Confusion re @keywords and Keyword panel

[CollieDog]

Hi!

I'm still having troubles figuring out @Keywords and keywords that I assign in the Keyword Panel.
In my main photo database, I have used a custom-built controlled vocabulary to assign keywords to many thousands of research photos.

The Keyword Thesaurus and keywords displayed in the Keyword Panel display the same hierarchy. They are ordered and tidy.
However, when I examine @Keywords, I see a chaotic mess of keywords. In @Keywords, the many keywords that I assigned based on carefully-ordered hierarchies in the Keyword Panel appear partially in the same structured order, but many appear in one long list as "flat" keywords.

To try to get to the bottom of this mystery, I created a new simple database where I could experiment with applying keywords in the Keyword Panel and see how this affected @Keywords.
After several simple experiments, I am still mystified as to why the Keyword Panel does NOT reflect the changes (rearranging of nested keywords) in @Keywords.

I have attached several screenshots to illustrate my question.

Using the Keyword Panel, I assigned students "Bagdagul" and "Ayan" to the subcategory of Cafeteria. I then used @Keywords and moved these student names into the category "Students" where they belong.

To my surprise, the Keyword Panel continues to show Ayan and Bagdagul as non-selected keywords in "Cafeteria".  Why were these names not entirely removed from Cafeteria in the Keywords Panel when I had previously moved them into "Students" in the @Keywords panel?

It seems to me that the ways this functions will only create huge mess and discordance in my database. Can I not use @Keywords structure the order of my images and have this exact order reflected in Keywords Panel?

My Goal: In @Keywords, I would like to see a perfect hierarchical display all of my keywords so that I can click on any @keyword and see all of the photos associated with it.
My Problem: The @Keywords  are a mess and appear in a flat alphabetical order that looks messy and seems to be mixed in to my hierarchical ordering system. I am having trouble understanding how this is useful and would greatly appreciate some gentle guidance in figuring this out.  I'd like to understand this once and for all and hopefully clean up my current database and avoid this problem in the future.

Thanks everyone!

[Jingo]

Hi there... I think you are confusing what the @Keywords "special" category is and what it is used for... this category is a listing of all HIERARCHICAL keywords.. they should mirror the keywords in your files.

In your example - you do not have your Keywords/Thesaurus setup with the same hierarchical layout that you want for you @Keywords... and thus your confusion.  If you want to assign Ayan to the People subdir of Cafeteria - you should also reflect his in your Keywords panel...

If you were to check your file after doing the @Keywords drag/drop - you find your image files match the @Keywords... because they are written back to the file in that hierarchy... the Keywords panel does NOT reflect your image keywords nor the @Keywords category...

Think of the keywords panel as a list of possibilities.. think of @Keywords as the ACTUAL reflection of keyword hierarchy in your images.  The 2 don't need to match... but certainly can if you keep Keywords up to date.

Hope this helps.. sure Mario and others will give more details and a better description than I just did!
 

[sinus]

Hi there... I think you are confusing what the @Keywords "special" category is and what it is used for... this category is a listing of all HIERARCHICAL keywords.. they should mirror the keywords in your files.

In your example - you do not have your Keywords/Thesaurus setup with the same hierarchical layout that you want for you @Keywords... and thus your confusion.  If you want to assign Ayan to the People subdir of Cafeteria - you should also reflect his in your Keywords panel...

If you were to check your file after doing the @Keywords drag/drop - you find your image files match the @Keywords... because they are written back to the file in that hierarchy... the Keywords panel does NOT reflect your image keywords nor the @Keywords category...

Think of the keywords panel as a list of possibilities.. think of @Keywords as the ACTUAL reflection of keyword hierarchy in your images.  The 2 don't need to match... but certainly can if you keep Keywords up to date.

Hope this helps.. sure Mario and others will give more details and a better description than I just did!

Yes, Jingo is right, I think.
And, as far as I know, the thesaurus you can only change with the thesaurus manager. If you do something in you categories or @keywords, this will NOT change something in the thesaurus, hence not in the keyword panel.
You must look it the other round.

Use a VERY small DB with only 2-3 image (like you did) and do again some tests.
For example, if you delete a keyword, this will also delete this keyword in the @keywords (except you say IMatch in the prefs, it should not do so). But this deleting will NOT delete the keyword in the thesaurus.

It is not easy to understand it, but once done, you will see the system behind it.

[Mario]

The @Keywords category reflects the actual keywords contained in your files.

This is unrelated to the contents of the IMatch thesaurus. The IMatch Universal thesaurus enables you to store "text" for every metadata tag and to use that text later. This works exceptionally well for repeating text like headlines, descriptions, copyright notices, legal statements. And it works well if you want to setup a controlled vocabulary for your hierarchical keywords.

See the Universal Thesaurus help topic in the IMatch help system. And, for keywords, the corresponding article Free Controlled Vocabularies in the IMatch knowledge base.

If you make changes to keywords in your files, this does not affect the contents of your thesaurus. And if you change the contents of your thesaurus, this will not change the existing keywords in your files. In your example, Your thesaurus is not linked to the keywords in your files.

In your example, you drag and drop keywords around in your files, utilizing the advanced capabilities of the @Keyword category. While this will change the keywords assigned to your files, it will not change the contents of the thesaurus. And hence you will see changing keywords in the Keyword Panel, but of course the thesaurus panel will not change.

To change the contents of your thesaurus, use the Thesaurus Manager. Click on Help or press <F1> while the Thesaurus Manager dialog box is visible to open the corresponding help.

[CollieDog]

Thanks everyone for your responses.

I've done more experimentation and am getting closer to a clear understanding.

The problem I face now and which I find immensely confusing is how my @Keywords categories became so messy?  I used the Thesaurus Manager to create a controlled vocabulary of hierarchical terms and used the Keywords Panel to assign tag photos with items from my controlled vocabulary.

If everything is working normally, then the items appearing in @Keywords should be an exact duplicate of the controlled vocabulary that I created in the Thesaurus Manager (and reflected in the Keywords Panel). But this is not the case!  The items appearing in my @Keywords quite messed up. Many hundreds of items are displaying as flat keywords that are not nested in the hierarchy. Please see the attached screen shot.

ie)
When I created the keyword "kinship terminology" in Keyword Panel, I created it as a child of "SOCIAL SCIENCE CONCEPTS".
In @Keywords , why is "kinship terminology" listed as a flat keyword that is NOT nested inside of "SOCIAL SCIENCE CONCEPTS"?

How can I clean up my @Keywords so that all items appear in the hierarchical structure that I created in the controlled vocabulary (in the Thesaurus Manager)?

I sincerely hope I've not asked a question that is surely too "dunce-like"    I spent a great deal of time creating a controlled vocabulary suitable for my social science research, but now am mystified why the items in @Keywords are not reflecting this.

[Mario]

The problem I face now and which I find immensely confusing is how my @Keywords categories became so messy?

It just reflects the keywords in your files. If the keywords in your files are messy, so will be the @Keywords category.

I'm not sure why you end up with such a mess. Usually this just works and is a no brainer. Did you look at the actual keywords already in your files? How did you setup the thesaurus? Do you use group levels? Exclude levels?

If you assign the keyword BLA|blu|blo to a file, this is exactly what will be assigned. You should see it that way in the Keyword Panel. If the keywords become messed up during write-back (!) it's because of how you have changed the default keyword settings in IMatch, or because of the keywords already in your files...

[CollieDog]

"I'm not sure why you end up with such a mess. Usually this just works and is a no brainer."

Hahahahaha.... er... I'm not sure what to say     
Anyway, the problem is getting closer to resolution.

It seems that when I set up the controlled vocabulary, something was wrong with the setup of group levels and exclude levels. I'm examining this now.

I cannot figure out why PEOPLE - NAME -- Various Names  is creating flat keywords in @Keywords.  Please see the attached screenshot that shows the hierarchy and settings in Thesaurus Manager.
Why are these settings resulting in flat keywords in @Keywords?  Is there an easy way to resolve this?

Some of the other flat keyword problems originated when I imported my tagged images from Adobe Lightroom into IMatch.

Is there any way to open the Thesaurus Manager separately so that I can check it against the @Keywords and make changes? Right now, if I open Thesaurus Manager, I can't scroll through the @Keywords to cross-check.  Perhaps I should be using the Keyword Panel to do this cross-checking and correction?

[Jingo]

Right now, if I open Thesaurus Manager, I can't scroll through the @Keywords to cross-check.  Perhaps I should be using the Keyword Panel to do this cross-checking and correction?

AFAIK - the Thesaurus Manager is a separate process and will not allow interaction with the underlying windows while it is open.  However, I would suggest you just export the thesaurus as a text file and then work on it in your favorite text editor or update your @Keywords while looking at it that way.  If you make changes to the thesaurus in the text editor, then you can import that file back into the system again... much simpler.

Good Luck!

[CollieDog]

After puzzling over this all day, it seems that I must have changed a setting or involked a command that entirely messed up my @Keywords and changed them from hierarchical keywords to flat.

I examined PEOPLE - NAME -- Various Names  to try understand why IMatch created flat keywords in @Keywords.  I attached a screenshot last time showing the hierarchy in the Thesaurus Manager, and the corresponding flat keywords in @Keywords.

Why are these settings resulting in flat keywords in @Keywords?  Is there an easy way to resolve this?
The screenshot attached illustrates the problem.

[Mario]

Why are these settings resulting in flat keywords in @Keywords?  Is there an easy way to resolve this?

By the looks of it, you have configured both PEOPLE and name to be group levels in your thesaurus. See the [ and ] around them? That's the indicator.
Check your keywords int he thesaurus and remove the group level option.

Group levels exist only in the thesaurus, to, well, group your keywords. They do not become part of the keyword.

A keyword like PEOPLE|name|Paul with group levels PEOPLE and name produces only the keyword Paul when you add it in the thesaurus. This is visible in the Keyword Panel. When you assign the keyword from the thesaurus, only Paul is added.

[CollieDog]

By the looks of it, you have configured both PEOPLE and name to be group levels in your thesaurus. See the [ and ] around them? That's the indicator.
Check your keywords int he thesaurus and remove the group level option.

Thanks! I removed the 'group level' option and ran "Update Metadata". That solved the problem! The keywords are now showing in proper hierarchical order in @Keywords.
If I sift through the Thesaurus Manager and remove all of the group level options, then all of my keywords will appear in proper hierarchical order in @Keywords.

The only problem that I envisage with this is that now, for example, the photo of a Tibetan prayer flag will no longer only be tagged with "prayer flag lungta" but will be tagged with all of the keywords of the hierarchy eg) "SOCIAL SCIENCE CONCEPTS#|Anthropology Sociology|religion|Tibetan religion|prayer flag lungta"

If these top-level categories (eg SOCIAL SCIENCE CONCEPTS, Anthropology Sociology) are now included as keywords, will this cause a mess in my catalogue and make it difficult to search for important keywords later?  I simply don't have enough experience with this to foresee what problem that having this full hierarchical structure may cause.

[Mario]

IMatch gives you a lot of options to manage your keywords and the mapping between keywords and hierarchical keywords. There is no other application out there which gives you this amount of control. The default options and thesaurus settings usually work best. If you add group levels to your thesaurus without knowing how this affects the resulting keywords, I suggest you re-read the thesaurus help topic. It explains nicely all the options, when to use group levels, how they affect the mapping between keywords on export and import etc.

Either you want hierarchical keywords, or you don't. I don't see a problem. Unless you mess up your thesaurus or disable options, IMatch can map back from flat keywords to hierarchical keywords when it re-imports IPTC and XMP. No mess is caused in your keyword hierarchy.

I recommend you re-read the IMatch help topics about the thesaurus and hierarchical keywords. And how IMatch overcomes the old problem of mapping between flat keywords used by XMP and IPTC and the more versatile hierarchical keywords used by Lr, IMatch and other software.

[Jingo]

Just to add... the whole flat keywords vs hierarchical keywords debate can be discussed for a long.. long time.  In the end, it usually comes down to how you manage your images and WHAT you keyword for.

I started out as a flat keyworder.... "cat", "Vacation", "kingfisher"... it was all well and good for my personal catalog and enabled me to find keywords very quickly... somewhere along the way, I switch to a hierarchical structure... "cat" became  "Animals|cat", "Vacation" became "Events|Vacation", "kingfisher" became "Animals|Birds|kingfisher".... I could still find keywords quickly and it helped when I was adding more specific keywords like "Objects|Crafts|Ribbon|Red"... because I might also have... "Objects|Cars|Red"...

Lately - I've been thinking of moving back to flat keywords again... because I don't need that level of detail and can instead use a flat keyword of "Red" and a flat keyword of "Ribbon" and just search for both words to quickly find them...

In the end.. it really depends on how you search for things, how you use your system, etc... flat IPTC keywords can be seen via lots of software... hierarchical.... not so much (unless these are also written to IPTC and then the hierarchy makes them a single keyword string).   

Good luck with things... I'm still figuring keywording out 15 years after I first started!!

[Mario]

I tend to KISS.

Do I really need something complex like WHAT|vehicle|car|Volvo  or would Volvo just do?

If you look at what pros do, they usually use very simple keywording schemas. See, for example:

Getty: http://www.guliver.ro/wp-content/uploads/Guliver_-_Getty_Images_Keyword_Guide.pdf

shutterstock: http://www.shutterstock.com/blog/how-to-master-the-art-and-science-of-keywording-your-images

In IMatch you have options to control how the hierarchical keyword is flattened: "Volvo" or maybe "car|Volvo"?

And you have the very powerful, fast and truly dynamic IMatch categories to manage any number of complex hierarchies and taxonomies outside your keywords.

It's best to have a big think in the beginning and then come up with whatever works. Consistency is more important than everything else. See also https://www.photools.com/4588/free-controlled-vocabularies-imatch/

[CollieDog]

Thanks Mario and Jingo,

Your replies and patience are appreciated.

I have been systematically going through the help file and simultaneously experimenting with a test database to clarify functions.

I didn't realize that Categories were different from @Keywords. If I understand correctly, if I want to make special categories or albums, I should do it as subnodes of Categories, and NOT in @Keywords.  The @Keywords reflects the keywords that are added to photos.

One thing that mystifies me is that it seems I cannot change group-level placeholder helper words into keywords and add these in front of the hierarchical keywords that follow. I hope the following explanation clarifies my problem.

Originally, I set up my Thesaurus Manager like this: 

WHO (as group-level) |
PERSON (as group-level)  |
NAMES | Mr Desolation 

This structure was reflected in the Keyword Panel thesaurus and when I apply "Mr Desolation" to a photo, the hierarchical keyword assigned to the file is "NAMES | Mr Desolation"

But because @Keywords only shows NAMES | Mr Desolation and does not show WHO and PERSON, I found that the lack of WHO and PERSON leads to chaos in my @Keywords.

To try to resolve this, I opened Thesaurus Manager and removed the group-level configuration of WHO and PERSONS. Accordingly the group-level designations were removed in the Keyword Panel. This is show in the screenshot.

I ran "metadata writeback", but the former group-level placeholder helper words ( WHO and PERSON) did not NOT appear in front as WHO | PERSON | NAMES | Mr Desolation

The Keyword Panel thesaurus shows that WHO and PERSON are no longer a group-level designation so why were they not added in front of names and why did @Keywords not reflect the change? Is there a way to automatically add WHO | PERSON to appear in front of NAMES | Mr Desolation as a result of turning off the group-level designation of WHO and PERSON?

My tests seem to indicate that I can't fix my current mess of flat or leaf keywords in @Keywords unless I
a) remove group-level designation for all keywords in Thesaurus Manager
b) manually reassign all of the hierarchical keywords

Thank you for helping me through this.

[Mario]

Changing your thesaurus does not affect keywords already in your files.
If you change group levels or other things in the thesaurus, this also does not reflect in your keywords. The thesaurus is just a way to conveniently store text for metadata tags, including keywords.

In short: Remove the unwanted keyword and assign it again from the thesaurus. What then gets added to your file is whatever thesaurus holds in that moment.
Changing the thesaurus later will not change your keywords. Consider a user accidentally changing something in his thesaurus and then IMatch starts to update 300,000 files to reflect the changes done...

If you want to change the keywords in your files, use the Keyword Panel.
Or work with the @Keywords category hierarchy. Which has some special features not available in regular categories.

[Erik]

My own experience has been the hierarchical keywords are more for our convenience.  Outside of a few DAM software (Lightroom for instance), most everything else rely on the flat keywords.  As a result, the hierarchy is best for me with regard to the thesaurus. 

Of course, when we change the thesaurus, it can cause confusion with images that may have varying keywords (if I have rearranged keywords in the thesaurus for instance).  I used to go in and adjust the keywords around, but I find that it is just as easy to search for files containing a specific keyword than adjusting all the files because my thesaurus was changed.  Usually I only change the thesaurus for the sake of assigning keywords.  I don't need to worry much about what I've already assigned.

Now for flat keywords, I find myself relying on using the "exclude" option rather than the "group" option.  This keeps the hierarchy clean while making sure that specific levels of the hierarchy never end up in the flat keywords.  I do contemplate using @Keywords to only show flat keywords (if possible) perhaps grouping by the first letter for some order.  That might allow me to find images a little more quickly, but I'm not sure whether I'll do that yet.  I've not even played with the @keyword settings to see if this is possible.

[CollieDog]

( :)I just found that this was inadvertently posted in the wrong thread! Sorry about that )

Hi everyone,

First of all, thanks to everyone who addressed my previous questions?
Over the last few weeks, I've placed my use of IMatch on hold while I've been studying the help files and experimenting with organizational methods.

I have already catalogued (added keywords to) a few thousand images in my research photo collection about cultural traditions in rural north China. Before I begin working on the remaining majority of images in my research collection, I want to avoid the mistakes that I've made in the past. This problem is described in my overview of "PROBLEMS" below.

In order to better-understand IMatch's powerful tools, I created an experimental database where I've been experimenting with Categories, @Keywords and the keywords in the thesaurus manager.
I believe I understand now the function of keywords, @keywords, and categories.

However, I'm still very much confused as to how to set up my workflow.
Allow me to explain the a) keywording system I have used, b) the problems, and c) my desired goals. 

a) SYSTEM USED:

I imported a controlled vocabulary into IMatch 5 Thesaurus Manager
The main headings of my controlled vocab in the Thesaurus Manager look like this:

Keywords

[AGRICULTURE")
ACTION "
AGES
AGRICULTURE
ANIMAL *
ART"
BUILDING 0
CELESTIAL BODY"
COLOUR
CONCEPT "
DOCUMENTS "
ELEMENT"
EVENT "
FOOD AND DRINK "
GENDER"
LANDFORM *
LOCATION"
MATERIAL "
NATURAL PHENOMENON *
NATURE AND ENVIRONMENT »
NUMBER"
OBJECT*
PEOPLE *
PHOTOGRAPHY*
PLACE*
PROFESSION *
SOCIAL SCIENCE CONCEPTS*
SPORT*
THOROUGHFARE *
TIME*
TRANSPORTATION *
WATER *

and modified it to include a group of keywords on various social science topics. See the example below.

SOCIAL SCIENCE CONCEPTS#
   anthropology- medical
   Anthropology Sociology
      festivals
         outdoor kitchen daguocai
   China folk arts and practices
      dance
      junledui music band
      painter
      paper cuts
      yangge dance
   Co-ops
      {co-operatives}
      {cooperative}
   conflict
   cooperation
   festivals
      Huanghui festival
      Jiaozhuhui festival
         {Dizang Pusa festival}
         {Jiaozhu Assembly}
      Kaiguang
         {Kaiguang festival}
      Nauryz Kazakh New Year
      temple festival
         {miaohui}
   food security
      dumpster dive
         {deli-dive}
         {food scavenging}
         recovered food
            {salvaged food}
   industry
      mining
         Yancoal Mining Co.
   interview
   kinship terminology
   migrant workers
   nationalism
   
This organization system has been modified many times as I catalogue the images.
I've been trying to find information on standard methods for building an organizational structure for social science concepts, but I've had little luck finding good examples.


b) PROBLEM:

1. Necessary to constantly modify my hierarchical list.
Many of the keywords assigned to my images need to be re-arranged into different categories.
Currently, because I used keywords written into metadata, the only way to do this is to use @keywords to find similarly tagged images, and then manually assign new keywords using the thesaurus in the Keyword Panel. This procedure is far too time-consuming- especially considering that I will continue to refine my organizational system until they collection has been processed.

2. Need to find a better way to organize.
a) My controlled vocabulary does not use Who What Where Why group-level categories. I am thinking of adopting the new vocab list from "Lightroom Keyword List Project"

b) I am also considering going with a simpler list from UNESCO
http://vocabularies.unesco.org/browser/thesaurus/en/page/concept378


c) GOAL:

To have my photos sorted using hierarchical keywords according my organizational scheme.
Whenever I change or revise the hierarchical order of my controlled vocabulary, I'd like IMatch to automatically change the metadata in all my files so that they match accordingly.

I'd like @keywords to display the images as they are categorized in a hierarchy.

I'd like the Thesaurus Manager, or at least the Universal Thesaurus (keyword panel) to reflect the changes that I make in my keywording hierarchy.

SOLUTIONS?

What type of workflow will allow me to achieve my goals? Should I stop using the Keyword Manager altogether and rely only on @keywords? @Keywords does not allow the creation of synonyms - a fact which further complicates my keywording plans.

On the forum ( https://www.photools.com/community/index.php?topic=6006.0 ) Mario advised "You usually draw the line between categories and keywords by what you want to keep in the file (and under @Keywords) and what you only want/need in IMatch."
I'm unsure what information I would want only in IMatch. Usually if I distribute jpegs to friends or upload them, I strip out all metadata. It seems to me that I want all the metadata in XMP sidecar files that are readable by other software.

I would greatly appreciate suggestions and feedback before I proceed with organizing and tagging my image collection.

[Mario]

@Keywords does not allow you to create synonyms because it only reflects the keywords in your file.
@Keywords is basically a sophisticated data-driven category that reads all keywords from all your files and then creates child categories for each keyword and assigns the files with that keyword.

Synonyms are no concept of XMP or IPTC keywords. This is an IMatch concept implemented in the thesaurus for your convenience. Once the synonyms have been added, they become normal keywords. IMatch uses the thesaurus to check each keyword in the keyword panel, in order to identify keywords you have configured as synonyms in your thesaurus.

'Moving' a keyword is simple in IMatch.
For example, to move "county|city|bombay' to 'country|city|mombai'  you select the bombay category under @Keywords. Select all files. In the Keyword panel, delete 'country|city|bombay' and add 'country|city|mombai'. This takes only a few seconds, and IMatch will update the keywords in all affected files, and also @Keywords.

Alternatively, you can just drag the file in the @Keywords category from bombay to mumbay. Couldn't be easier.

Operations on keywords like this are sometimes required when a city or place name changes, object name changes or maybe a species is moved within a taxonomy. Or when you change your mind about how you use certain keywords.

Large-scale re-arranging of keyword hierarchies is usually rarely needed. And when, it can be considered a project and a nuisance - even if you have powerful tools like IMatch at your disposal. The major agencies and the IPTC have stable keyword hierarchies for decades, only added new keywords here and there if new topics need to be covered.

Moving segments of your keyword hierarchy (e.g. changing "cars|classic|...." to "cars|vintage|..." can mean to change segments of a keyword hierarchy 10 levels deep.
IMatch allows you to do that simply by drag-and-drop in @Keywords. You just select all children of "cars|classic" and drag them to "cars|vintage". IMatch updates keywords accordingly. I'm not aware of any other software that can do that, or as easily.

Keeping keywords simple, using only as many levels (if at all) as absolutely required and spending time up-front to consider the best hierarchy for a given collection are key factors for successful keyword management in a DAM system.

For volatile and frequently changing hierarchies and taxonomies, IMatch categories are usually the better alternative. You can do many things with categories that are impossible with keywords. And much faster. And with powerful tools like category formulas, Alias categories, data-driven categories etc. There is no DAM out there that can compete with IMatch on this level.

[akirot]

This thread and especially Mario's post is a good summary of the features and concept of keywords and categories.
Personally I have implemented a detailed structure of categories (for private use only) plus a flat simplified structure for keywords (for publishing).
I miss the feature of synonyms for categories.
With the thesaurus (plus synonyms) for keywords one can implement a taxonomy for e.g. animals including their names in various languages. Going down this road one would flood the affected images with all that stuff.
Having synonyms for categories would be better (for personal organization, not for publishing).
(I know the alias feature for categories but I find aliases too complicate to implement for taxonomies.)
Is anybody else facing this challenge? How do you approach this request?
Or should a feature request be raised?

[Mario]

I miss the feature of synonyms for categories.

Consider using category splasher favorites. These enable you to assign a file to multiple categories at once.
To create a splasher favorite, select two or more categories in the category panel or category view and drop them to the Favorites panel. When asked, use the option to create a splasher.