Cannot get rid of an old keyword

[Herens]

Hi,

I want to assign the following keyword (a child of the category @Keyword) to 4 photographs (I'm a botanist),  :

Végétaux|Classification|Angiospermes|Dicotylédones vraies|Dicotylédones vraies supérieures|Super Astéridées|Astéridées|Lamiidées ou Euastéridées I|Boraginales|Boraginacées|Consoudes (Symphytum)|Grande consoude (Symphytum officinale)

But the field automatically reverts to an old and incorrect keyword (that I created years ago). I tried to delete it but it reappears everytime after a writeback :

Plantes|Fleurs|Espèces|Boraginacées|Bourraches|Grande consoude (Symphytum officinale)

I don't know what to do anymore...

[jch2103]

You may have conflicting legacy IPTC metadata. Run the Metadata Analyst app on a problem image to see what the likely issue is. (It may be helpful if you save the report here - be sure to zip it first because this forum may not allow you to upload it otherwise.) If that's the problem, you can run the ExifTool Command Processor preset 'Delete Legacy (IIM) IPTC Metadata' on the problem file(s).

[Mario]

See The Metadata Analyst for details.
If errors or warnings are shown, use the green button at the top of the MDA to copy them into the clipboard and paste them into your reply.

As jch2103 pointed out, the most likely cause is a mismatch between hierarchical and flat keywords in your files, and IMatch being unable to resolve this automatically.
This happens e.g. if you have used software which only updated one set of keywords, or which wrote unmatched information into hierarchical and flat keywords.

[Herens]

Thanks Mario and John.

Here are the warnings of the Metadata analyst for one of the litigious photographs :

Metadata Analyst Results. Version 2020.14.2. 5/28/2021 3:25:25 PM
File analyzed: H:\Mistralnet Photos\Botanique\2021\2021 05\2021 05 20 Woluwe\Originaux\2021-05-20_12.53.22.cr2
Errors: 0
Warnings: 12

Warning: [XMP] Embedded XMP record (Unknown) and XMP sidecar file (photools.com IMatch 20.14.0.2 (Windows)) found.
Warning: [XMP] Embedded XMP rating is 0.
Warning: [XMP] [ExifIFD]:DateTimeOriginal not mapped to [XMP-exif]:DateTimeOriginal (embedded).
Warning: [XMP] [ExifIFD]:CreateDate not mapped to [XMP-xmp]:CreateDate (embedded).
Warning: [XMP] [ExifIFD]:DateTimeOriginal not mapped to [XMP-photoshop]:DateCreated (embedded).
Warning: [XMP] [IFD0]:Copyright not mapped to [XMP-dc]:Rights (embedded).
Warning: [XMP] [IFD0]:Artist not mapped to [XMP-dc]:Creator (embedded).
Warning: [XMP] [IFD0]:Artist not mapped to [XMP-tiff]:Artist (embedded).
Warning: [XMP] [IFD0]:Orientation not mapped to [XMP-tiff]:Orientation (embedded).
Warning: [XMP] [ExifIFD]:UserComment not mapped to [XMP-dc]:Description (embedded).
Warning: [XMP] [GPS]:GPSLatitude not mapped to [XMP-exif]:GPSLatitude (embedded).
Warning: [XMP] [GPS]:GPSLongitude not mapped to [XMP-exif]:GPSLongitude (embedded).

[Mario]

Dreadful!

Just kidding.
The problem is that you have both XMP data in a sidecar file and an XMP record embedded in the image.
RAW files, by standard, use XMP data in sidecar files, not embedded.
Having 2 XMP records for the same file will cause problems. There can be only one source of truth.

Solution: Delete the embedded XMP record using the ExifTool Command Processor
Select the .CR2, start the ECP with F9,E and run the "Delete XMP metadata" preset.

What I don't see are error warnings about out-of-sync keywords, though.
Maybe this is a side-effect of the two XMP records, I dunno. The problem you see is in 99% of all cases caused by different keywords in flat XMP keyword / legacy IPTC and hierarchical keywords.
Fix the double XMP problem first, then we'll see.

[Herens]

Hi Mario, I did what you recommended, but it didn't solve the problem.

I would say it's not dreadful but creepy  :

I tried the following :
I deassigned the wrong keyword to the files and did a writeback.
I then deleted this keyword in the category panel.
I assigned the correct keyword via the category panel.
I took a look in the metadata panel, which still showed (for those files) the old keyword (supposedly deleted)!
After a new writeback, the new and correct keyword had been deselected in the category panel (and replaced by the old one).

and I thought botany was complicated!


 

[Mario]

Did you delete the XMP record in the CR2 as I asked above?

[Herens]

yes, I tried it 3 times with no result.

The metadata panel doesn't change, it always shows the old keyword.
And if remove it manually, it reverts to it automatically after a few seconds.

[Mario]

Where do you remove the keyword?
Please use the Keywords Panel. Show us what you see there (screen shot).

[Herens]

Sorry for my late answer, I was in the field because of the nice weather.

Tonight I did it as you said, removed the wrong keyword, ran the ECP and assigned the right keyword using the keyword panel.
I did it for one photograph, and it worked, even after the writeback !
In the next days I will do the same for the other litigious files : 337 in total, the oldest ones from 2005 and the latest from May.
(DB has approx. 180.000 files)
I will inform you of the result, but already thank you for your help !


I usually assign or deassign a keyword via the categories panel. I also create new keywords there.
Shouldn't I do it in this way ?

Stéphane

[Mario]

The @Keywords category hierarchy just conveniently mirrors the actual keywords in your files.
Usually the dedicated Keywords Panel is the best place to work with keywords. But @Keywords also works.

If there is trouble with keywords, it's usually not IMatch but rubbish stuck into your files by some other, less competent, applications.

[Herens]

Hi Mario,

by using the Keyword panel I just unearthed something strange.
It doesn't mirror exactly the structure I have under the Categories panel.

Example: I want to assign the following keyword, which is visible under the Categories panel (attachment 1) :
Végétaux : Date : Mai

But it didn't appear in the keyword panel. In this panel, under Végétaux, I only had the subdivision Classification and Type.

So each time I added, via the Categories panel, Végétaux : Date : Mai, it was transformed to an old keyword (which I never could remove from the DB), which  exists in the keyword panel : Plante : Fleurs : Date : Mai.

I have now added Végétaux : Date : Mai in the keyword panel as well (attachment 2), and it now seems to behave as I want (when using the ECP to remove XMP data).

This seems to be a new behaviour since April/May, because i have allocated  Végétaux : Date : Mai before, to more than 9.000 files, without problem.

Stéphane

[Herens]

Beside what I mentioned in my earlier post, I still have an issue with 6 files (from 2005).
When I try to run the ECP to remove the XMP metadata, I get this error message :

Error: File not found - H:/Mistralnet Photos/Belgique/Luxembourg/2005 06 01 Bertrix ardoisières/2005 06 01 Bertrix Originaux/2005 06 01 Bertrix-0032.CR2
Error: File not found - H:/Mistralnet Photos/Belgique/Luxembourg/2005 06 01 Bertrix ardoisières/2005 06 01 Bertrix Originaux/2005 06 01 Bertrix-0042.CR2
Error: File not found - H:/Mistralnet Photos/Belgique/Luxembourg/2005 06 01 Bertrix ardoisières/2005 06 01 Bertrix Originaux/2005 06 01 Bertrix-0043.CR2
Error: File not found - H:/Mistralnet Photos/Belgique/Luxembourg/2005 06 01 Bertrix ardoisières/2005 06 01 Bertrix Originaux/2005 06 01 Bertrix-0044.CR2
Error: File not found - H:/Mistralnet Photos/Belgique/Luxembourg/2005 06 01 Bertrix ardoisières/2005 06 01 Bertrix Originaux/2005 06 01 Bertrix-0057.CR2
Error: File not found - H:/Mistralnet Photos/Belgique/Luxembourg/2005 06 01 Bertrix ardoisières/2005 06 01 Bertrix Originaux/2005 06 01 Bertrix-0058.CR2
    0 image files updated
    6 files weren't updated due to errors

But the files (CR2 and XMP) exist.

[sinus]

Could this have something to do with the umlauts and accents and such stuff like è é and so on?
Though it is a shame, but I use not more umlauts in folders or filenames, instead of ö I write oe ...

If I use äöü ... I have maybe no troubles with program 1, but with the next program or the web or links .... it is really shame. 

Though I have not problems with IMatch.

[Mario]

Example: I want to assign the following keyword, which is visible under the Categories panel (attachment 1) :

This is not the categories panel. This is the thesaurus panel. It shows your Thesaurus and the keywords you have added there.

When you assign keywords from the thesaurus, it will add exactly what is in the thesaurus.
It will remove group or ignore levels. It will apply synonyms.
IMatch does not make keywords up.

If assigning the keyword Végétaux|Date|Mai works, but assigning the keyword Mai from the thesaurus does not, check the parent keywords of Mai and Mai itself in the thesaurus manager. Make sure you did not add synonyms, group levels and similar.

Also check under Edit > Preferences > Metadata how you let IMatch flatten hierarchical keywords into XMP and (optionally) legacy IPTC.

When I try to run the ECP to remove the XMP metadata, I get this error message :

Your folder names contain non-ASCII characters. If you have still old ExifTool Command Processor presets, make sure to update them to use UTF-8 encoded file names:

-G1
-all
-a
-charset
filename=UTF8
{Files}


The
-charset
filename=UTF8
is the important bit.

I have no issues with file names like C:/data/Image Tests/Version/Outdsoor/France/Bertrix ardoisières/2005 06 01 Bertrix Originaux\2005 06 01 Bertrix-0032.CR2 using this.

The requirement to change existing ECP presets was included in the release notes, but years ago. I did this change when ExifTool added support for UTF-8 encoded file names.

[Herens]

Hi Mario, thanks for the clarification.

Under Edit > Preferences > Metadata, I have the following options selected (see attachment)

[Mario]

Lookup via thesaurus should be on. Only then IMatch can map existing flat keywords in your files to your hierarchical keywords.
Assigning a keyword via the @Keywords hierarchy does the same as assigning the keyword via the Keywords Panel.
If you assign a file to a @Keywords category and the keyword is not added, either your thesaurus prevents that (group level keyword or exclusion level) or you have found a bug that was never reported (unlikely).
Many users, including myself, assign keywords via the Category Panel. Or the thesaurus. Or the Keywords Panel.

But you said above that the problem was solved after you removed the duplicate XMP record, so I assume all is well now.
Just do the same for your other files. Having two XMP records for the same file is bad.

[Herens]

OK, I have now (this morning) selected Lookup via thesaurus.

And added
-G1
-all
-a
-charset
filename=UTF8
{Files}

to the ECP preset.

Yesterday I still met some problems with writebacks (old keywords coming back ...)
But after the changes I did this morning, things seem to be ok. I cross the fingers !

Many thanks for your help (and patience) !

[Mario]

When keywords 'come back' or a re-import after write-back produces new keywords, the reason is one of:

+ IMatch has been configured to flatten hierarchical keywords in a way that produces new flat keywords, and the thesaurus does not allow Match to match these flat keywords back to their hierarchical origin

+ The file contains a mess of out-of-sync keywords in flat/hierarchical XMP/legacy IPTC (pretty common when you touch metadata for your files with multiple applications, and some of them do not properly synchronize keywords or just don't care at all). Often, IMatch can sort this out with two write-backs. If not, the user must manually fix the out-of-sync keywords. The Metadata Analyst app can detect these problems and reports them.