AI Questions: Traits, [[-c-]] exchanging & copying AI Keywords to Image Keywords

[Jingo]

So.. recently been playing around with the OpenAI service and must admit - it is able to identify more birds from images than Mistral has been (neat!).  So, I have three questions...

1 - Let say the AI returns the following set of keywords for a Mallard Image:

AI|Aquatic; AI|Green Head; AI|Serene; AI|Tranquil; Animal; Animals|Animal; Animals|Bird|Duck; Animals|Bird|Mallard; feathers; Objects|Reflection; Objects|Water; Objects|Water|Pond; outdoors; Places|Nature; swimming; wildlife

Since I set the AI to add the keywords to the AI Keywords and not Hierarchical... I would like to copy SELECT hierarchical keywords to the image - mostly those that are hierarchical and linked to the thesaurus... do I have to manually do this or is there a mechanism via the Metadata panel to do so?  I have a metadata template setup to copy the AI components to XMP fields.. but it copies them ALL and not the selected items.  I can write an APP to do this but wanted to check and see if something exists first.

2 - I still cannot get any traits to file into the AI Tags... I have the following setup in OpenAI settings:



When I run the AI on any image, I get no traits returns in the AI Tags section... I would have thought the tags would get set up even with empty results given my prompt but perhaps not?

3 - I have set up my preferences to use the AI tags and the prompt within is simply set to [[-c-]] for all three items, description, keywords and landmarks.  In the autotagger prompt, I then enter the prompt as: 

Code Select Expand

Describe this image in a style to make it easily searchable. Use simple English, common words, factual language, and simple sentences.
Avoid describing anything not directly observable from the image.
Return ten to fifteen keywords describing this image.
{File.MD.hierarchicalkeywords|hasvalue:This image has the following keywords: {File.MD.hierarchicalkeywords}.}
Return the names of known landmarks and tourist spots in this image. If you cannot detect any landmarks or tourist spots, return nothing.
{File.MD.city|hasvalue:This image was taken in {File.MD.city},}{File.MD.country|hasvalue: {File.MD.country}.}

This seems to replace EACH of the [[-c-]] prompts with these values which seems to work.. but perhaps is wasteful to the AI token usage? Should I just hardcode each of the prompts in preferences with the appropriate section from above once I get something I like set?  Or - should we be able to define overrides for each section on the fly... something like a [[-k-]] for keywords, [[-d-]] for description and [[-l-]] for landmarks?

Thx - Andy.

[Mario]

1. You can either let the AI update hierarchicalKeywords or write AI.keywords.
There is no selection mechanism to control what goes where. You can map keywords (hierarchical or AI) via the Mapping options available in AutoTagger. Which enables you to exclude keywords you don't want, add only keywords with mappings etc.

If you later want to copy "select" keywords from AI.keywords to hierarchical keywords, a Metadata Template or maybe the Metadata Mechanic is the feature to use. This is quite particular to your workflow and I doubt there will ever be a "copy "these" keywords from AI. keywords to XMP hierarchical keywords.

2. I have several settings which use AI traits for OpenAI, e.g. one for AI.Headline and one for AI.Lighting.

Tip: Switch to debug logging and then search the log for OAI-RESPONSE to see the data OpenAI has returned. Your AI traits will show in the result, e.g. {"AI.Lighting\":[\"Bright and sunny with natural daylight\",

3. AutoTagger replaces [[-c--]] with the text you provide. There is only one placeholder, and it will be applied to all tags/traits where you include the [[-c--]]. This can be used to provide additional context, which is not that often needed when your prompt works.

If you find yourself in situations where you really need different manually entered placeholders for different tags/traits (which should be really rare), make one setting per tag an run them individually, providing placeholder context as needed.

Or, maybe setup a global variable for each tag, reference the variable in each corresponding tag and then set the global variables before you run AutoTagger.

I have never seen a need for this, but these are two ways to deal with that.

but perhaps is wasteful to the AI token usage?

The tokens are calculated based on the length of the prompt, the length of the response and a general fee per access.
How you construct the prompt (setting, placeholder, variables) is irrelevant.

Usually you set the prompts in the AutoTagger setting and you're done. Just run it.
When you occasionally see a need to provide AutoTagger with more context to help it, use a placeholder. Usually the context improves both keywords and description, which is why there is only one placeholder needed.

[Jingo]

Thanks Mario...

#1 - just seems like the AI returns a lot of "extra" keywords that I would never want to add to my images (and eventually thesauraus)... especially the flattened ones or duplicates that still get added. 

I'll come  up with an app to allow me to copy them in some mechanism beyond what the metadata template can do I think.

#2 - I checked the log (attached) and do not see ANY OAI-RESPONSE tags .. only OIA-PROMPT. you can see this morning at the end of the log where I sent a new OAI prompt for a bird photo... no response is shown in the log and no traits were returned.  It did however correctly identify the bird (YES!) and provide some nice description info...

#3 - Thx for the info... I set up my settings now so each item has the ability to include the extra details and only the prompt info relevant for that item.

[Mario]

The extra keywords you don't want can be excluded via the Keyword Mapping feature.

I checked the log (attached) and do not see ANY OAI-RESPONSE tags

My bad. That's for developers (aka me) only, sorry.

[Jingo]

The extra keywords you don't want can be excluded via the Keyword Mapping feature.

I checked the log (attached) and do not see ANY OAI-RESPONSE tags

My bad. That's for developers (aka me) only, sorry.

Too bad.. that would be great to see in the log!  I'm still not sure why I cannot get any trait information back.. has anyone else successfully setup traits and received results using OpenAI?  If so, can you share your traits so I can try them out and see if it something on my end?  Thx!

[Jingo]

For #1 - I just came up with the following variable to restrict non-hierarchical keywords from propagating from the AI keywords to my hierarchical keywords:

{File.MD.AI.keywords|filter:^AI|Animals|Colors|Events|Objects|People|Places|Special Items}

These represent the top levels of my Thesaurus so, if the the AI keyword has mapped correct to the thesaurus, then running this variable in the metadata template only transfers "valid" keywords.  I can then manually map other keywords it finds for subsequent runs to maximize keywords.. neat!

Just thought I would share in case this is helpful to others who only want to transfer thesaurus level keywords to their images.

Thx - Andy.

Still wondering if traits are working with OpenAI for anyone?

[Mario]

Still wondering if traits are working with OpenAI for anyone?

Works for me. Just tried. Both my Headline trait and Lighting trait were successfully filled by OpenAI for 10 different images.
Maybe rework the prompt?

[Jingo]

Good to know it is working at least.. I'll try reworking a few of them.. I was using a sample traits from the help file so figured it would at least fill the trait with "No Animals" if I ask:

If there are animals (cats, dogs, horses, fish, birds etc.) visible in this image, respond with the text 'Animals' else respond with the text 'No Animals'.

I'm just getting no traits set up at all.

[thrinn]

I'm just getting no traits set up at all.

One thing to keep in mind (because I stumbled upon this myself): If you create your own traits and you want to see the results in the Metadata Panel, make sure to add the corresponding AI tag to the layout. In my case I wondered why the group "Traits" was always empty. Simple solution: The tag was not part of the MD layout.

[Mario]

And when you try the same prompt for description?
Note that AIs are fickle and some trial and error is required. Prompts often need to be tailored to the AI.
Check out https://platform.openai.com/docs/guides/prompt-engineering where OpenAI provides details about how to prompt them for specific purposes.

[Mario]

One thing to keep in mind (b

Yes. You can also add your custom AI tags to a File Window layout of course.

[Jingo]

Thanks Thorston... that was it!  Total miss on my end... somehow thought by adding a trait to the preferences they were going to be added automatically to the Traits Metadata template (feature request?)... 

The traits are of course working just fine for the AI.. as soon as I added these values, they showed right up for all images I had already taken.  Cool!

[Mario]

There is no Traits Metadata template. And IMatch does not dynamically update or change Metadata Templates, there is no infrastructure for that. IMatch removes deleted AI tags from Metadata Panel layouts, though.

The "AI Tags" included in IMatch 2025 uses the "always there" tags (description, landmarks, and keywords). All else is up to the user. I shall place a todo box in the help to mention this explicitly.

IMatch could show a series of dialogs, asking users if and to which Metadata Panel templates he wants to add an AI tag, and in which branch or maybe even a new branch. Then one could argue, the same could be "useful" for adding AI tags to one or more File Window layouts. Or maybe automatically add a new data-driven category for the tag under a parent category selected by the user, or ...

I pondered all this and moved it to "maybe, when there is actually sufficient demand" list. I'm sure in a couple of months from now, when a larger share of users has used AutoTagger, we'll know better what is missing and what needs to be improved.

[Jingo]

Sorry... I did mean the standard AI Tags metadata template which shows a "traits" subheading automatically.  I know once you start down one path... it opens things up to more paths. 

I think a note in the help is enough for now (I read the help multiple times but didn't know I had to manually add them) but an option to auto-add them down the road would be super as well!

[Mario]

I have added a note in both the AutoTagger and Prompting/Traits topic. Let's wait a couple of months and see how many people use Traits and request some sort of automatically adding trait tags to one or more Metadata Panel layouts.