Can't delete certain keywords

[LRAT]

I want to remove ALL keywords from a series of imported pictures.
Initially I selected all the pictures, then I selected the Keyword panel and clicked in the top RH corner hamburger symbol and selected: "Delete all keywords".
I then clicked on the green arrow.
Then clicked on the yellow pencil symbol.
This removed the majority of the keywords.

However, out of 242 pictures about 70 still had the following keywords in it:
~Where|Antarctic; ~Where|Antarctica;
No matter what I do, these keywords continuously return to the keyword field after the yellow pencil has been clicked.
I then checked this forum and somebody mentioned to click F9+E. There's an option to delete all the keywords.
I ran this but to no avail.

So, it seems that it is impossible to remove the keywords.
When I hover the mouse over the yellow pencil symbol then a message shows up:

This file has unwritten metadata.
Click the icon to write that data now.
List of tags to write:
IPTC::ApplicationRecord\Keywords
XMP::Lightroom\HierarchicalSubject
XMP::dc\Subject

It seems that these fields are containing the keywords that cannot be deleted. No matter what I do it keeps on returning.
Any other tricks that I could apply to strip away all the old keywords?
Thanks for your help in advance!
Luke

[Mario]

Looks like your IPTC and XMP keywords are out-of-sync in a way that makes it impossible for IMatch to set them straight.
Can happen and depends on whatever other software you have used that modified metadata.

1. Run Metadata Analyst on a file wit the problem.
Use the GREEN BUTTON at the top to copy/paste the results into your reply.

2. Deleting legacy IPTC keywords in the ExifTool Command Processor is usually the way to fix that. IMatch will then flush out a fresh set of synchronized keywords.

See Metadata Problems and Pitfalls for more information.

As always in cases like "I have problems writing metadata to this file", having the actual file for analysis is a big help.
Upload to your cloud space and post a link, or send to support email address with a link back to this topic (I get many emails per day and cannot always remember what's what).

Note: I have a special queue for "metadata problems", because a) it usually costs a lot of time to figure out what the problem is and b) it is usually not caused by IMatch/ExifTool.
This queue is always full and I process such problem reports in the order they came in. Typically it will take a week or so before I can look into your particular issue. Too many people with to many metadata problems caused by too many other applications... 

[musashi77]

Im also experiencing this exact same problem on a bunch of jpgs. I dont mean to hijack this thread but Ive posted the output. Should I create a new thread to upload an image for?

Metadata Analyst Results. Version 2020.14.2. 7/23/2022 12:50:15 AM
File analyzed: \\qnap6\PICS\PHOTOS_ALL\2019 - (Current) Chifley\2020_\2020-03_\2003 Phone - Daniel_\20200313_215355.jpg
Errors: 0
Warnings: 10

Warning: [System] File has unwritten metadata (pending write-back).<br/>The metadata loaded from the image and the data in the database may not match.
Warning: [Detailed Validation] [minor] Unknown APP4 segment
Warning: [Detailed Validation] [minor] Unknown APP5 segment
Warning: [Detailed Validation] [minor] Unknown APP11 segment
Warning: [Detailed Validation] ExifIFD tag 0xa434 LensModel requires ExifVersion 0230 or higher
Warning: [Detailed Validation] ExifIFD tag 0x9010 OffsetTime requires ExifVersion 0231 or higher
Warning: [Detailed Validation] ExifIFD tag 0x9012 OffsetTimeDigitized requires ExifVersion 0231 or higher
Warning: [Detailed Validation] ExifIFD tag 0x9011 OffsetTimeOriginal requires ExifVersion 0231 or higher
Warning: [Detailed Validation] [minor] IFD1 tag 0x0100 ImageWidth is not allowed in JPEG
Warning: [Detailed Validation] [minor] IFD1 tag 0x0101 ImageHeight is not allowed in JPEG

[Mario]

None of these warnings is related to keywords. And important info is missing, e.g. if there are different keywords in the XMP and legacy IPTC. Impossible to tell what the problem is.
But there is pending metadata, so the MDA did not see the actual data anyways.

Please write-back first.
If this does not remove the pen, point the mouse cursor at the pen to see what IMatch wants to write.
This would give at least a minimum of info to work with.

Upload the file with this problem and post a link.
Or send it to support email address and I will look into this next week or the week after.

Usually this kind of problem is caused by legacy IPTC keywords which are out-of-sync with XMP, in a way that IMatch cannot fix.
Did you read Metadata Problems and Pitfalls?

[RobiWan]

@LART and musashi77,

what software created the files?
Some times ago I had same problem and have spoken with Phill (ExifTool developer). He said he has never seen anything like this, the keywords would be written in 2 places and Exiftool doesn't know what to do with them.

Here the link to my thread by ExifTool https://exiftool.org/forum/index.php?topic=13198.msg73642#msg73642

[musashi77]

So Ive had this problem for a while, after a while I just filtered them away and ignored it, but now I finalizing the importing and tagging every photo Ive ever take, all 350K of them.

I have 2 Keywords on 104 files (1 keyword stuck on 103 files and the other attached to a single file) They are all jpgs from a mobile device, there are no XMP sidecars.

I am able to remove the keywords in questions and successfully "write back", everything looks great but then when I run clean up tasks on the database:  "database tools" -> "Database Diagnostics" or "Compact and Optimize" then the keywords continually return to the same 104 files.

I then checked this forum and somebody mentioned to click F9+E. There's an option to delete all the keywords.

Can someone point me towards this, I tried searching the forum for this reference but couldn't fine it.

@LART and musashi77,

what software created the files?
Some times ago I had same problem and have spoken with Phill (ExifTool developer). He said he has never seen anything like this, the keywords would be written in 2 places and Exiftool doesn't know what to do with them.

Here the link to my thread by ExifTool https://exiftool.org/forum/index.php?topic=13198.msg73642#msg73642

I'm pretty sure that all the keywords on these files have only been done in iMatch, the files may have gone through Lightroom or Daminion however I only really started working on a Hierarchical keyword structure, and these keywords came from an early structure created in IMatch that evolved.

I also just want to point out that Ive only ever used programs like Lightroom and Imatch to write keywords into the file, so I really am confused about things like exif, IPTC, whats legacy where or how it was saved to certain places etc..

I'ld like to try these steps for example but Im really just trying to figure out where to start with these kind of steps.
"Deleting legacy IPTC keywords in the ExifTool Command Processor is usually the way to fix that."
"The command to delete all keywords would normally be: exiftool -keywords= -subject= FILE"


I have emailed some samples to support, below is the MDA on the files sent to support directly after "write back"
________________________________________________________________________________________________________________________
Metadata Analyst Results. Version 2020.14.2. 7/23/2022 11:54:09 PM
File analyzed: \\qnap6\PICS\PHOTOS_ALL\2019 - (Current) Chifley\2019_\2019-04 05 Japan_\1904-05 Japan Mobile Daniel_\20190515_141610.jpg
Errors: 0
Warnings: 10

Warning: [Detailed Validation] [minor] Unknown APP4 segment
Warning: [Detailed Validation] [minor] Unknown APP5 segment
Warning: [Detailed Validation] [minor] Unknown APP11 segment
Warning: [Detailed Validation] ExifIFD tag 0xa434 LensModel requires ExifVersion 0230 or higher
Warning: [Detailed Validation] ExifIFD tag 0x9010 OffsetTime requires ExifVersion 0231 or higher
Warning: [Detailed Validation] ExifIFD tag 0x9012 OffsetTimeDigitized requires ExifVersion 0231 or higher
Warning: [Detailed Validation] ExifIFD tag 0x9011 OffsetTimeOriginal requires ExifVersion 0231 or higher
Warning: [Detailed Validation] Missing required JPEG ExifIFD tag 0x9101 ComponentsConfiguration
Warning: [Detailed Validation] [minor] IFD1 tag 0x0100 ImageWidth is not allowed in JPEG
Warning: [Detailed Validation] [minor] IFD1 tag 0x0101 ImageHeight is not allowed in JPEG
__________________________________________________________________________________________________________________________
Metadata Analyst Results. Version 2020.14.2. 7/23/2022 11:54:56 PM
File analyzed: \\qnap6\PICS\PHOTOS_ALL\2019 - (Current) Chifley\2019_\2019-12_\1912 Phone - Hiromi_\20191224_211418.jpg
Errors: 0
Warnings: 8

Warning: [Detailed Validation] [minor] Unknown APP4 segment
Warning: [Detailed Validation] [minor] Unknown APP5 segment
Warning: [Detailed Validation] [minor] Unknown APP11 segment
Warning: [Detailed Validation] ExifIFD tag 0x9010 OffsetTime requires ExifVersion 0231 or higher
Warning: [Detailed Validation] ExifIFD tag 0x9012 OffsetTimeDigitized requires ExifVersion 0231 or higher
Warning: [Detailed Validation] ExifIFD tag 0x9011 OffsetTimeOriginal requires ExifVersion 0231 or higher
Warning: [Detailed Validation] [minor] IFD1 tag 0x0100 ImageWidth is not allowed in JPEG
Warning: [Detailed Validation] [minor] IFD1 tag 0x0101 ImageHeight is not allowed in JPEG
_____________________________________________________________________________________________________________________________

[Mario]

I also just want to point out that Ive only ever used programs like Lightroom and Imatch to write keywords into the file, so I really am confused about things like exif, IPTC, whats legacy where or how it was saved to certain places etc..

Metadata usually works behind the curtain. Users only notice it when something goes wrong, or they have used problematic tools or workflows that left their metadata in a sad state.
For a DAM like IMatch, managing metadata is one of the most important features of course.
I recommend you at least skim over Metadata for Beginners

I run clean up tasks on the database:  "database tools" -> "Database Diagnostics" or "Compact and Optimize" then the keywords continually return to the same 104 files.

That would be quite unlikely. And the diagnosis would surely report warnings and errors in the log file, so you could learn more about the problem.

I'ld like to try these steps for example but Im really just trying to figure out where to start with these kind of steps.
"Deleting legacy IPTC keywords in the ExifTool Command Processor is usually the way to fix that."

Select the problem file in a File Window.
Run Commands menu > ExifTool Command Processor to start the The ExifTool Command Processor
Select the "Delete legacy IPTC (IIM) metadata" template and run it.

BUT, only do that when the problem you are encountering is actually caused by mangled or out-of-sync legacy IPTC metadata.
Nothing in the Metadata Analyst results seems to indicate that.
It would show when there is legacy IPTC data and it would also show when there are out-of-sync keywords.

The warnings we see in your outputs are just ExifTool being very precise and picky. Many files produced even by modern cameras have some invalid EXIF data - the camera vendors don't care. EXIF data is not selling hardware.

I will look into the files you have sent. Hopefully you have included a link to this topic since it may take two weeks for me to look into this. I get 50 emails per day, and many users send emails about "metadata problems". And usually it turns out that IMatch is not the problem and I've wasted yet another hour of my life to diagnose and explain metadata problems. Nobody pays me for that and it's always time-consuming. I look at the "metadata mess" queue in my inbox only when there is nothing else to do. And the queue always has 10 or 15 email conversations.

[musashi77]

I run clean up tasks on the database:  "database tools" -> "Database Diagnostics" or "Compact and Optimize" then the keywords continually return to the same 104 files.

That would be quite unlikely. And the diagnosis would surely report warnings and errors in the log file, so you could learn more about the problem.

Hi Mario, I can assure you that this is what is happening, its repeatable. I can remove the keyword from the photo, do a clean write back, delete the keyword from the @keyword list in the catergory pain, and everytime I do a "Database Diagnostics" that keyword will be back in the keyword tree and the same photo applied to it. I have attempted to capture the logs while running through the steps. And I do always get some errors or warnings when running the diagnostics, up until now I just ignore them and run the diagnostics a second time and they errors are gone. This is not new behavior, Ive had these files and been dealing with them for quite some time (a year at least) was filtering them out of site until I finished my overall goal of importing my collection.

The attached logs are from the diagnostics and a debug.

I did also try deleting the IPTC data from the exiftool as mentioned above, however that didn't make a difference.
Also checked out link for more information on metadata, and its is helpful, however as it mentions most people are oblivious to it until it goes wrong, which is true, however I did spend a lot of time researching the best way to handle metadata across multiple or as many applications as possible which is exactly why I ended up using Imatch as my source of truth. Thats the reason I recently went way back into my archives to have everything imported into Imatch to have everything consistently tagged dating back more then 20years. So Im just at a stage where its time for me to address the oddities within my collection. 104 photos and 600 odd video files (another thread I started) is not bad from 350K in the database that are all tagged now.   

[Mario]

I see only one warning in the "IMATCH6_DIAGNOSIS_LOG_pass03.txt"  log file:

Warning: Missing keywords for entity [60] for file [25449]. Fixed.

This warning is logged when there is a face annotation linked to a person and the person has one or more keywords, but these keywords are not found in the database for the file.
The diagnosis then re-applies the keywords.

You can find the name of the file via the "File Finder" app (App Manager). Enter the number into the Find File By Oid field.

If you have set keywords for the person and you want to remove these keywords, and the diagnosis sees that the person is assigned but the keywords are not, maybe this is the problem?
Check the persons assigned to faces in the image to see which keywords you have added to the persons.

[musashi77]

Warning: Missing keywords for entity [60] for file [25449]. Fixed.

This warning is logged when there is a face annotation linked to a person and the person has one or more keywords, but these keywords are not found in the database for the file.
The diagnosis then re-applies the keywords.

Perfect, this was exactly what was causing it, went into the face annotations and updated or removed the corresponding keyword, and all is well.

thanks Mario, much appreciated.

Hopefully the OP gets something out of this to.