Confused why Categories @keywords does not match keywords/ keyword thesaurus

[CollieDog]

I've been through the manual, but am having trouble figuring out why the entries in @keywords do not reflect the hierarchical keywords that I created in Keywords and are so clearly reflected in the Keyword thesaurus.

In the attached screenshots, many of the keywords appearing in @keywords are appearing "flat" and are not displayed in their corresponding hierarchical order. Why?
As far as I recall, I have never created created anything in @keywords and have relied exclusively on the keyword panel to create a hierarchical order.

As I understand, @keywords are only keywords within the IMatch database and do not get written to XPM sidecar files. Conversely, keywords entered in the Keyword panel are propogated into XML sidecars. The @keywords lists are created by IMatch based on the keywords created in the Keyword Panel. But if this is the case, why did the hierarchical order disappear?

In the screenshot, Social Science Concepts in the Keyword Panel contains many sub-topics. But within Categories @keywords, Social Science Concepts contains very few of the sub-topics.

Thanks for patience as I sort this out 



[attachment deleted by admin]

[Mario]

The thesaurus just stores text for any kind of metadata tag, including keywords. It is not linked with the special @Keywords category.

@Keywords is a data-driven category which is filled automatically from the hierarchical keywords in your file. It reflects the actual keywords you have added to your files.
Your thesaurus can contain 10,000 keywords, but your files may only use 500 of them. And then @Keywords will also have only 500 child categories.

[CollieDog]

Thanks Mario,

I do understand that the Thesaurus Manager simply stores keywords (and metadata tags?) as a master lookup table.

I was under the impression that if I added hierarchical keywords to images from the Keyword Panel, then the same hierarchical structure would appear in the @keywords category.
In fact, I have completely got this backwards. I thought the Keywords Panel was data-driven (reading from XML sidecars) and the @keywords category only stored keywords within the IMatch internal catalogue.

Just to clarify that I understand properly: The keywords appearing in the @keywords Category appear flat (without the tree structure) because the photo was tagged only with the keyword "prayer flag lungta" from the Keyword Panel Thesaurus entry "SOCIAL SCIENCE CONCEPTS#|Anthropology Sociology|religion|Tibetan religion|prayer flag lungta". IMatch 5 does not assign the parent keywords to the photo "SOCIAL SCIENCE CONCEPTS#|Anthropology Sociology|religion|Tibetan religion|", and that is why "prayer flag lungta" appears as a flat keyword in @keywords Category. Is this correct?

If so, I do find this puzzling. Similar to As another user garypcook noted, I also thought the point of using higherarchical keywords was to avoid the need to add all the parent keywords individually (https://www.photools.com/community/index.php?topic=2682.msg17400#msg17400)

I strive to clarify this because I'm working to resolve a big problem. Several months ago, my RAID failed and I lost my most recent IMatch catalogue. When I imported my image folders (ie 2010, 2011, 2012, 2013, 2014), some of which had metadata sidecars of hierarchical keywords created in IMatch, I thought that the Keyword Panel thesaurus would be re-created during the import.

But it wasn't. My Keyword Panel Thesaurus no longer contains a full record of all the keywords of imported images and XMP sidecars.

Instead, the @keywords Category contains many hierarchical keywords from sidecars. These keyword now appear flat (@keywords does not reflect the hierarchical category from the Keywords Panel where I created them). More problematic is that the Keywords Panel thesaurus no longer includes the keywords from @keywords Category. When I view photos, I can see that it has the "SOCIAL SCIENCE CONCEPTS#|Anthropology Sociology|religion|Tibetan religion|prayer flag lungta" keyword attached, and this keyword is in @keywords Category. But its not in the Keywords Panel thesaurus which is where I need it because I do all my assignment of keywords (tagging) from there.

It seems that my problem above is the same as Joel23 here https://www.photools.com/community/index.php?topic=2526.msg16363#msg16363
I read the post, but am still not clear how to resolve this issue.

Is there an easy solution to get my hierarchial keywords from @keywords Category back into the Keywords Panel thesaurus?
Or should I give up on the Keywords Panel thesaurus, and try to reconstruct my hierarchical keywords in the @keywords Category?

I'm so close to figuring this all out. Thanks again for your patience.

[Mario]

@Keywords uses the hierarchical keywords in your file, and rebuilds their structure.
Please have a look at the keywords your files really contain, e.g. by looking at the keywords field in the Metadata Panel (default layout). If you see keywords like Parent|Child|Child there you will also see the same structure in @Keywords. This is usually a no-brainer all all automatic. Please show us what you see in the keyword panel (the actual keywords, not just the thesaurus section).

Your "social concepts" top-level category has [] so you have configured it to be a group-only level. Which is not stored as an actual keyword. It then will of course also not show up in @Keywords. You can control all this via the thesaurus, and even use multiple levels there which never show up in your keywords. I explain this in the corresponding help section and the thesaurus help.

If you enable the check box for "Chinese Religion" the resulting hierarchical keyword is thus "religion|Chinese Religion". This is stored in the file, and this is also where it will show up in @Keywords as well.

[CollieDog]

I've been looking at the hierarchical keywords in the Metadata Panel.
The keywords there SOCIAL SCIENCE CONCEPTS#|Anthropology Sociology|religion|Tibetan religion|prostration are indeed mirrored over in @keywords.

The problem is that SOCIAL SCIENCE CONCEPTS#|Anthropology Sociology|religion|Tibetan religion|prostration does not appear in the Keyword Thesaurus (with check-boxes), nor does it appear in the Thesaurus Manager.
How can I get the keywords in @keywords to be included in the  Keyword Thesaurus (with check-boxes) and the Thesaurus Manager?

For some reason, there are many keywords in @keywords that appear as flat keywords rather than as hierarchical order. Perhaps this is a throwback to when I was using Lightroom to add keywords and used flat keywords. What should I do to fix this mess and have all keywords sorted according to the order in the Master Thesaurus?

My guess is: Move around the keywords in @keywords as needed and build my hierarchical structure that I was trying to create in the Keyword Thesaurus. But if I do that, how can I get the Keyword Thesaurus in proper order?

I want to ensure that my keywords and metadata is stored in XML sidecars so that this data can be used to rebuild the IMatch catalogue in case of massive failure of my disks and backups.




[attachment deleted by admin]

[Mario]

How can I get the keywords in @keywords to be included in the  Keyword Thesaurus (with check-boxes) and the Thesaurus Manager?

IMatch does not automatically import keywords into your thesaurus, or synchronize it with whatever keywords are in your files. This is not what users want. You can use the commands available in the thesaurus manager to import the keywords in your files. IMatch here scans the database for hierarchical keywords and adds them to the thesaurus. Please see the help for the thesaurus in the IMatch help system for details.

Adobe extended the XMP standard with a proprietary XMP-lr namespace in order to store LR data and hierarchical keywords. The XMP standard has no concept for hierarchical keywords, except for some recent additions which may take years to adapt, or never will. Anyway, since LR introduced hierarchical keywords, a file suddenly could contain keywords in 3 places (!): legacy IPTC keywords (flat, no hierarchy), standard XMP keywords (dc:subject, flat, no hierarchy) and XMP Lightroom keywords (hierarchical, proprietary).

An application which deals with keywords is supposed to always synchronize hierarchical and flat keywords. Not all applications do it. Not all applications do it in the same way. Some just write the leaf keywords, others come up with some schema to 'flatten' hierarchical keywords into XMP and IPTC keywords. LR used several ways over the years, and when I recall correctly, some of them were broken and caused the keywords to fall apart. Users don't noticed that as long as they used only LR, but it caused trouble when they dared to use their files with non Adobe software, real DAM systems or typical agency workflows. Your files may be affected by this.

IMatch has many options which allow you to control if and how hierarchical keywords are 'flattened', how existing flat keywords are treated on import (e.g. how to map them to hierarchical keywords, using your thesaurus). This usually results in a no-brain 'just works' workflow, but allows users to fix problems introduced by other applications they may have used over the years. I've literally spent months developing all this - something that is really important and can save users weeks of work - but also something that is not suitable for reviews or flashy ads 

I cannot look into your files or tell you why your keywords are wrong. You can see the keywords in your file by adding IPTC keywords, XMP subject and the XMP-lr hierarchcialSubject tags to a metadata panel layout. Or use the ExifTool Command Processor in IMatch to check the keywords in your files. There is a "List Keywords" preset which shows all keywords from IPTC, XMP and hierarchical Keywords side-by-side.

Check out the import options for keywords (Edit > Preferences > Metadata) and the help (press <F1> while in that dialog box) to see if the settings you have chosen match whatever is stored in your files, and your thesaurus.

Yes, this is complicated. Not my fault. Adobe and other software vendors has made a mess out of this. And if you have used more than one software to add or edit keywords in your files, you probably have caused trouble, out-of-sync keywords or keywords which have been flattened in ways you did not want.

IMatch can only pick up the pieces and offer options to fix these problems, if possible.