Writing metadata cause keywords to disappear. Forcing update they reappear.

[Damit]

Unfortunately, I keep on running into problems with this program. I have invested countless hours trying to get things to work, probably spending 5x more time troubleshooting than actually tagging and it is really frustrating!
 
I did a diagnostic and got more of the "Warning: Missing keywords for entity [X] for file [X]. Fixed." As I have said before, I believe Mario was incorrect in concluding that these messages were an error by the diagnostic rather than an actual error because each file named does, in fact have issues.

I have begun looking into them to try to figure out how I can actually get this program and my database, which I have completely started anew after having such difficulty with my previous two databases, to work reliably. All of this I believe is related to the issues discussed here:
https://www.photools.com/community/index.php/topic,14017.msg98601.html#msg98601

The current issue I have is with a set of .tiffs (though this problem is present with jpegs as well) in a folder which were discovered with the aforementioned diagnostic warning. I noticed that there actually were missing keywords, just as the diagnostic reported.  The keywords were related to two entities that had keywords propagated using the "Keyword" field in the Person editor.
Before I force reload, the photo is missing the keywords for those people. The persons exists in the "Person in Image" XMP IPTC Extension and MWG Regions but do NOT exist in the IPTC application record, the XMP Dublin Core Subject, or the XMP Lightroom Hierarchical Subject.

When I force update the photo, the keywords related to the entities/persons reappear and are propagated and visible in @Keywords. The XMP Lightroom Hierarchical Subject now contains the keywords propagated by IMatch as instructed in the Person Editor for the entities/persons, which is their name and for one, her relationship. However, these keywords are not reflected in the IPTC application record, the XMP Dublin Core Subject even after the forced update!   

As soon as I write the metadata to save it, the keywords disappear from XMP Lightroom Hierarchical Subject and @Keywords. This cycle can go on forever. I force reload, they appear, I save, they disappear. This occurs even if I set the Keep existing XMP and Protect existing XMP both to On or Off. I was having these same problems with my previous 2 databases. I realize now that what I thought was lost face annotations in my original database was do to this problem. It would be sincerely appreciated if this problem was finally resolved. I have spent a lot of time (which has value as well) doing everything I can to get this program to behave and there is a repeatable issue. I have rebuilt my database two separate times trying to get things to work. I am attaching the 2 diagnostic logs with the errors, one which was run right after the other, and a link to the debug log that was running while writing and force-freshing the file. I will email the file in question, which is file 19 in the error report, and the database to support, but my old database had the same issues and support has that already. I will note that I work with IMatch and Lightroom, going back and forth but I do have a strict protocol of having each read prior to working with the files and saving the metadata after any editing.  This is advertised as a capability of IMatch, so this should not be a problem, but I thought I would mention it anyway.

All of the files reported in the diagnostic that I have investigated to this point have this issue.

Also, please answer:
1.) How can you find out who is an entity.  The report mentions file numbers and entities.  I know the file finder can find the files, how can I find out who is which entity?

2.) How can you find the file number of a photo? Is there a properties or something of that sort which will list IMatch's file number for a photo?

Log

[Mario]

As soon as I write the metadata to save it, the keywords disappear from XMP Lightroom Hierarchical Subject and @Keywords.

Open the ExifTool output panel (View menu > Panels) before you write back.
Write back.
Copy the contents of the output panel into Notepad, save and attach the text file.
This shows us exactly which tags are written, the tag values written etc.

Please disclose all non-standard metadata settings you have made under Edit > Preferences > Metadata and ... Metadata 2.
If you use metadata propagation via versions, please include your propagation settings.

1. File.Persons.OID
2. File.OID

[Damit]

As soon as I write the metadata to save it, the keywords disappear from XMP Lightroom Hierarchical Subject and @Keywords.

Open the ExifTool output panel (View menu > Panels) before you write back.
Write back.
Copy the contents of the output panel into Notepad, save and attach the text file.
This shows us exactly which tags are written, the tag values written etc.

Thank you, Mario. I have attached the output. It does contain warning about the IPTCDigest not being current and that the XMP may be out of sync.

Please disclose all non-standard metadata settings you have made under Edit > Preferences > Metadata and ... Metadata 2.

95% of the time, I have Protect XMP set to YES (Default is NO) and Keep XMP data selected to YES, as you advised in another thread. I only change Keep Existing XMP to NO for brief instances when @Keywords does not update, but I don't think I have had to do that many times recently as I have been adding folders piece-meal and being very meticulous adding files, constantly checking @Keywords and it is functioning well. Unfortunately, I just added a bunch of files right before noticing this issue, but it is present with some of the earliest files I imported when rebuilding this last time. Even so, I am luckily NOT experiencing problems with @Keywords updating. 
When I ran the last save, which generated the exif output I attached, I did have the Keep Existing XMP to NO and the Protect XMP to YES, which are opposite the default settings, but I tried to save with the default settings and the result was the same.  Everything reverts.

If you use metadata propagation via versions, please include your propagation settings.

I had versions and propagations in my older databases but had not set them up in this one to avoid any issues they may cause.  I was then going to ask advice here before putting some in action, which was hopefully going to occur next week. So, at this time, nothing in Buddies and Versioning is enabled.

I have also attached the Exif Output of me trying the same thing on a jpeg. I do not seem to get the same errors there. I performed that attempt with the default settings in metadata 2.

[Damit]

I also want to add that I just reviewed these files in Lightroom and they show the proper keywords, but now they are showing they are in need of update. If I do update, the keywords disappear. I did have Lightroom read all metadata that IMatch performed from the files and saved that metadata prior to this last import of files into IMatch. It could be thought of as a back up of the work performed in Imatch. The fact that there is now a difference in the files means that somehow things have changed in the database since the last time I updated and saved prior to the import of new files. I don't know how importing files could screw up files already in place, if that is, in fact, what happened, but it seems that the metadata was somehow altered when the new files were added to IMatch because, if not, Lighroom would not show that the metadata it just saved from the work performed in IMatch would need updating. Strangely, in IMatch, most of these files show all the keywords as present. Most don't even show they need a writeback, but if you do writeback, the keywords will disappear.  I only discovered them by working on neighboring adjacent files that were flagged by the diagnostic, but in Lightroom they do show as needing an update. The other strange thing is that the IPTC Lightroom Hierarchical, IPTC Application record Subject and Dublin Core Subject are the fields that are reset, but not the MWG face regions or the annotations. Those remain.

Worst still is I can perform the changes in Lightroom that IMatch refuses to perform. I was working with one of these files in IMatch.  I wrote the data and the keywords disappeared.  I then went to Lightroom and hit Ctrl+S to write the old metadata saved in it that previously written in IMatch to the file.  I then went back to IMatch and did a refresh.  All the keywords reappeared because they were written by Lightroom. Now when I save the metadata in IMatch with a write-back, the keywords do not disappear! So I was able to accomplish with Lightroom what I could not with IMatch.

[Mario]

Please don't change any of the protection settings. Doing so may cause most if not all of the problems you report. And which is also the reason than 99.999% of all IMatch users have absolutely no problems doing something simple as adding keywords to files.

You always write lots of text, jump from here to there, from IMatch to Lightroom (which has many issues with metadata, requires an explicit reload of XMP when modified by other applications etc.) and all that just adds to confusion.

Use the The ExifTool Command Processor in IMatch with the "All Keywords" preset to see the actual keywords in your files.
That's the base. What IMatch brings into the database depends on your protection settings, your thesaurus (!), your settings under Edit > Preferences > Metadata and Metadata 2. I cannot figure all this out not knowing precisely which settings you have enabled and when and for how long and in which phase of your workflow...

What changes did you do to metadata for the two ExifTool outputs?
The 2024-03-10 only writes a XMP face regions and some metadata timestamps. No keywords.
The second writes XMP regions and one hierarchical keyword: "Who|People|In Pic|Maria Tata Talamas" but no XMP subject , which is very weird and must be caused by some settings you did?

Because IMatch always writes XMP subject (flat keywords) when it writes hierarchical subjects. When you change hierarchical keywords in the keywords panel, IMatch maps these internally to XMP subject and, when legacy IPTC exists for the file, also into legacy IPTC Keywords. But not in your case.

I've just made a test. For a file I've added the hierarchical keyword IMatch|Test in the Keywords Panel and then told IMatch to write back. As expected, it writes this:

Code Select Expand

-XMP-lr:HierarchicalSubject=IMatch|Test
-XMP-dc:Subject=IMatch|Test
plus some timestamps and other stuff.

When I now run the "All Keywords" preset in the ExifTool Command Processor, I get exactly what I would have expected:

Code Select Expand

[XMP-dc]        Subject                        : IMatch|Test
[XMP-lr]        Hierarchical Subject            : IMatch|Test
The new keyword has been written, both as a hierarchical keyword and as a flat keyword (I have configured IMatch to write hierarchies in flat keywords under Edit > Preferences > Metadata).

When I repeat my test with an old file that still has legacy IPTC metadata, adding the same flat keyword produces the following write instructions for ExifTool:

Code Select Expand

-IPTC:Keywords=IMatch|Test
-XMP-lr:HierarchicalSubject=IMatch|Test
-XMP-dc:Subject=IMatch|Test
and when I look into the file with the ECP with the "All Keywords preset", I see

Code Select Expand

[IPTC]          Keywords                        : IMatch|Test
[XMP-dc]        Subject                        : IMatch|Test
[XMP-lr]        Hierarchical Subject            : IMatch|Test
which is the perfect result.

Why does this not happen in your case? How did you add "Maria Tata Talamas" to the file? In is not a region keyword, so it must have been added in the Keywords Panel or in some other way? But then why IMatch is not writing the flat XMP Subject keyword.
Must be some settings you did.

It does contain warning about the IPTCDigest not being current and that the XMP may be out of sync.

This is a frequent warning. Many applications out there don't follow the rules and don't put checkusms (digests) for legacy IPTC data into the metadata. ExifTool does.
Digests are used to tell if the legacy IPTC metadata and their corresponding tags in XMP IPTC are in-sync.

As a first try, go to Edit > Preferences > Metadata 2 and click on the "Default button". Then save changes by closing with OK.
Open Edit > Preferences > Metadata and make a screen shot. Save to a JPG file and attach to your reply.

Then select some files giving you problems in a File Window. Press Shift+Ctrl+F5 and choose "Reload Metadata" to bring in metadata using the correct standard settings. Note that when the files contain flat keywords in legacy IPTC / XMP Subject, your E > P > Metadata and thesaurus WILL impact how the flat keywords are mapped to hierarchical keywords. I mean group levels and exclusions you may have configured in your thesaurus. See thesaurus help for more info.

If you think the keywords IMatch shows for a file are wrong, select the file, run the ExifTool Command Processor with <F9>,<E>, select the "All Keywords" and look at the actual keywords stored in the file.
In the same way you can determine if IMatch has written the keywords you think it should write.

UPDATE

Just received your sample TIFF file.
It contains one hierarchical keyword in IPTC:keywords, XMP:subject and XMP:hierarchicalSubject.
When I add two more keywords in the IMatch Keywords Panel and write back, all keywords show up in the file, in all three keyword tags.

Photoshop, Lightroom etc. show the keywords.
Perfect result.

UPDATE 2

I've downloaded your database and added the sample TIF to the database.
The file contains these keywords:

Code Select Expand

[IPTC]          Keywords                        : Who|People|In Pic|Padre Oscar
[XMP-dc]        Subject                        : Who|People|In Pic|Padre Oscar
[XMP-lr]        Hierarchical Subject            : Who|People|In Pic|Padre Oscar
After import, the Keywords Panel shows the following keywords:

Code Select Expand

Who|People|In Pic|Padre Oscar;
Who|People|In Pic|Padre Mario Viscaino;
Who|People|In Pic|Maria Eugenia Salazar;
Who|Relationship|Family|Sister
The additional keywords come from the 3 persons in the image.

I add IMatch|Test again as a new keyword and write back. The file now shows these keywords:

Code Select Expand

[IPTC]          Keywords                        : Who|People|In Pic|Padre Oscar, Who|People|In Pic|Padre Mario Viscaino, Who|People|In Pic|Maria Eugenia Salazar, Who|Relationship|Family|Sister, IMatch|Test
[XMP-dc]        Subject                        : Who|People|In Pic|Padre Oscar, Who|People|In Pic|Padre Mario Viscaino, Who|People|In Pic|Maria Eugenia Salazar, Who|Relationship|Family|Sister, IMatch|Test
[XMP-lr]        Hierarchical Subject            : Who|People|In Pic|Padre Mario Viscaino, Who|People|In Pic|Maria Eugenia Salazar, Who|Relationship|Family|Sister, Who|People|In Pic|Padre Oscar, IMatch|Test
All keywords created from the persons in the image have been written, also the new keyword I have added manually in the Keyword Panel.

Perfect result. No keywords missing or duplicated. Person keywords imported and mapped. Keywords synchronized across XMP and legacy IPTC. Using your database with your thesaurus and settings. Not sure what else I can do.

[Damit]

This response was written before I saw your update 2. But like I said it exists with the original file as it sits in the database. I am not sure how that can be replicated but a remote session will prove to you that it is happening if eveything else I have written is not sufficient

What changes did you do to metadata for the two ExifTool outputs?

I have also attached the Exif Output of me trying the same thing on a jpeg. I do not seem to get the same errors there. I performed that attempt with the default settings in metadata 2.

As I wrote, it was not the same file. I was submitting it to show you that the problem was occurring with Jpegs as well.

Please don't change any of the protection settings. Doing so may cause most if not all of the problems you report. And which is also the reason than 99.999% of all IMatch users have absolutely no problems doing something simple as adding keywords to files.

You specifically asked me to keep the settings to Yes on both on this post (Bold added) where you said these setting had nothing to do with my problem. I submit that I am still having the same issues as those on that thread, though they are manifesting different symptoms:

I think settings "Protect existing XMP" and "Keep existing XMP"

Protect existing XMP ensures that metadata you have modified in IMatch but not written yet is not overwritten when you also change the file in another application. See Protect existing XMP
See Keep existing XMP
You want these settings to be enabled. They are unrelated to your problem.

So you asked me to set the both to Yes, even though the default is No for Protect XMP as stated here Protect existing XMP, So, what is your recommended setting? Both to Yes?
BTW, I had them both set to yes for 95% of the time and only set Keep XMP data to No in rare occasions where the @Keywords was not updating, to recreate the Rich XMP data as instructed in you manual here: Keep existing XMP.  I can only go by the manual, and I was doing things as instructed therein. Like I said, I believe I actually have not done this recently. Anyhow, please let me know how I should set things for this not to supposedly happen, because I can tell you it has happened with all combinations of settings.  All set to yes, all set to no, and either one set to yes.  The whole erasing when saving and repopulation of data when force reloading continues to happen. 

You always write lots of text, jump from here to there, from IMatch to Lightroom (which has many issues with metadata, requires an explicit reload of XMP when modified by other applications etc.) and all that just adds to confusion.

I have read you chide others for long posts and I don't see how its helpful. It is off-putting. Believe me, I find it both laborious and tedious to have to report these issues at all. They are complicated to detail, and I am just doing my best to convey what is happening. I also feel like I have to prove to you that the problem exists by giving you many examples because it seems, at times, that you will not accept that the problem is as stated. For example, in this thread https://www.photools.com/community/index.php/topic,14017.msg98601.html#quickreply_anchor it took until post 10 for you to accept I had the problem I delineated in my original post. So I am sorry they are so long and confusing; they are not purposefully so.

Lightroom (which has many issues with metadata, requires an explicit reload of XMP when modified by other applications etc.) and all that just adds to confusion.

This program is advertised to play nicely with Lightroom. It was chief among the reasons I bought it. And I am very happy I am using both because lightroom has clued me into problems in IMatch. It also gives me another avenue to recover data if things screw up in IMatch, as they have.
I also have to reload changes performed in Lightroom when I come into IMatch. I have a strict protocol to try to avoid these problems. When I work in Lightroom I save the changes and then have lightroom read back the metadata to make sure nothing changes.  Nothing ever does in lightroom. Then I come into iMatch and I rescan the file (because I don't always want to wait for the program to write the changes) and save the changes. It is always in IMatch that things get funky, where face regions don't transfer even running the rebuild face regions using XMP unless I rescan first and where files constantly show a need of a metadata update even though I have not touched the file for a long time.
After I make sure things are correct in IMatch, I then go to Lightroom and read the data again and make sure nothing changes. I do this because all the problems I have faced in IMatch have made me leery and paranoid, and with good reason. Frankly, I have encountered much more metadata problems with IMatch than Lightroom as evidenced by my activity on this forum and just what is going on with this file, where lightroom performed changes on the original file that IMatch refused to do in any metadata2 setting. It is curios that Lightroom was able to load the data and set the data appropriately with no problems even though it is supposedly inferior in handling metadata.

The second writes XMP regions and one hierarchical keyword: "Who|People|In Pic|Maria Tata Talamas" but no XMP subject , which is very weird and must be caused by some settings you did?

No changes in setting.  Like I said it was a different file but the behavior is very weird and it is being performed by IMatch.

As a first try, go to Edit > Preferences > Metadata 2 and click on the "Default button". Then save changes by closing with OK.
Open Edit > Preferences > Metadata and make a screen shot. Save to a JPG file and attach to your reply.

Done!
OK, so I have done some tests, all with the default Metadata 2 settings and Metadata 1 as shown in the screen shot. I have no "exclude" keywords of which I am aware and no Versioning set. I got some interesting results. I looked at the .tiff I sent you and it behaved completely normal, as you reported. All the keywords related to other entities that were not Padre Oscar remained when I did a save or a reload. I also looked at the info in exiftool and everything was present as it should be.
So, I went to the original file in my database, and it did not behave the same. When I would tell IMatch to write the metadata, it would erase all the keywords associated and propagated by the other entities as shown in the thumbnail (I have my thumbnails listing keywords to monitor things like this), just keeping Padre Oscar, as far as keywords.
I have attached the Exif tool output of the file (Named File 19 after writing metadata). There are some curious things. You will notice there are 3 people in XMP-iptcExt persons in image, and all those people are supposed to be producing Hierarchical Keywords which are supposed to be propagated by IMatch based on what I inputted in the People Person Editor. These people also appear in the XMP-Mwg-rs fields. However, you will notice that there is only one Hierarchical Subject for Padre Oscar. That is an error. There should be inputs for the other entities. Why is IMatch not producing the keywords for the other People when I save as it should as instructed by the settings in the person editor? They are present in the IMatch database and displayed in the Thumbnail if I do a Force Reload, but ONLY THERE, if I look at the exif tool output, those keywords are not in the file itself, only displayed in the database (See included screen-shot file name starts with "IMatch and Exif tool mismatch") and when I try to write them, they disappear from the Thumbnail! (See attached jpeg named IMatch and Exif tool output after saving metadata 2024-03-11 105430, which shows the keywords missing from the thumbnail after trying to save the metadata.)
So, something is stopping IMatch from writing those keywords into the file or producing them in the first place, but only within this database and only with the original. If I copy this photo to another location and import it into the same database, it will initially show the same thing in the exif tool, with only Padre Oscar listed in the hierarchical keywords, but in the thumbnail it will show the missing keywords and it has a pencil, even though the file was just imported into IMatch. Why?  Because they are being propagated by IMatch based on the Persons in Image and/or the Mwg regions based on the settings in the Person Editor in People. If I then ask IMatch to write the metadata, all the keywords are populated, and everything looks as it should. But this will not happen with the original file as it sits in its original location If I go to the original and try to get the hierarchical keywords to write, they do not (as the previous Exif Output showed). The only thing written is the Padre Oscar hierarchical keyword. This is why you cannot reproduce the problem, because it exists only with the original files as they exist in my database. So, there is something that will not let IMatch write the data it knows is missing (as evidenced by the diagnostic and the fact that the keywords keep showing up in the thumbnail while doing a forced update) in a file that has been previously manipulated in IMatch. THAT is where the problem lies. In the database and how it "protects" and "preserves" or how it propagates keywords based on a People person editor setting. It is like IMatch remembers the previous state of the file and wants to "protect" and retain it. But if I import it as it was a new file IMatch writes everything as it should. So why is this not happening with the original file?

[Damit]

I have just tried it with another file, and I got the same result.  I cannot write the person driven hierarchical keywords in the original file in its original location but if I copy that file to another folder in the same database, it will write the keywords. If I then copy the file in the same folder as the original (with an altered name obviously), nothing changes, all the keywords are written. So, it has something to do with the previous state of the file in its database prior to trying to update. Hopefully that helps you narrow down where this problem may be occurring.

Also, FYI for a manual correction, I just noticed that when I set the defaults in Metadata 2, both Keep and Protect XMP default to Yes, which is different from what the manual states in https://www.photools.com/help/imatch/rmh_config_metadata2.htm?dl=hid-45 so perhaps this section merits a correction?

[Mario]

The Default button is more right than the help.
If you find incorrect info in the help, use the feedback link at the bottom of the help page to let me know.
If you cannot work with files in one folder, move the files to another folder.

So, something is stopping IMatch from writing those keywords into the file or producing them in the first place, but only within this database and only with the original.

Unlikely. Failure to write to files will be logged. But is not.
ExifTool will report warnings and errors when it failed to write data. In the output panel and in the IMatch log file.
But nothing is logged.At least not in the log files I have seen so far and the stuff you have sent to me in emails on top.

I have done for you what I could do.

Downloaded 10 GB of databases. No repro case. All working perfectly.
Used your sample files. No repro case. All working perfectly.
Answered tons of questions from you and read entire short novels in form of community postings from you.

I'm done.
My support capacity is limited and there are many other users who need my support and advice.

[Damit]

Proof of failure

Here is a link to a video of it occurring within IMatch. The is no debating what is going on. So, likely or not, it is happening and the failure to record output is not logged. Maybe IMatch is not presenting the keywords to be written but the problem has been documented. 

The question is will you actually support a customer when the problem is not easy to fix.  "Top-notch support" would.

[PandDLong]

I haven't read all your posts but your frustration is apparent.

I took a look at that video and have two observations which may or may not be useful:

1. I am not seeing the 'Pending Updates' icon on your selected file so not sure what is being sent to exiftool as an update on the write-back (if anything).

2. I don't believe VarToy is designed as a real-time metadata viewer - I always use a couple of the iMatch tools to understand what is happening (eg. a panel) if something doesn't seem right.  I have also used a simple exiftool viewer to look at the file as it provides an unfiltered look.


Michael

[Damit]

Thank you Michael. Yes I am very frustrated and distressed. I only have a limited amount of time to try to tag images. I chose this program despite warnings about going with an outlier program. It was specifically pointed out to me that there was a single person for support and development and that may cause issues, but I still chose IMatch because its potential was so tempting.  I never thought I would encounter so many issues and, despite starting over twice, still being mired in them. I have taken great pains to document the issue. There are many unanswered questions addressing screen shots that obviously show things are amiss in my system yet support believes it is fair to abandon the user solely because he reports he cannot replicate it.

To address your points: Yes the file is not showing a need for an update, but the point I was trying to make was that if I did write, it should be writing what is showing in the thumbnail precisely because the thumbnail is not showing it needs updating. Regardless, I then force a reload and that brings in the correct information, but somehow this never gets written into the file because, as you see in the video, as soon as I write the metadata it disappears once again.  There is no disputing that there is an issue, and one would think Mario would want to figure out what was going on. I mean I am showing IMatch cannot even write metadata to a file! But I guess not.

2.) I was using the VarToy to show the database being used and the file and entity IOD in case that would be helpful should support decide to address the issue.

[PandDLong]

Okay, now I did read through a bunch of the past posts on this topic - you have spent a lot of time on this.

I went through a frustrating time in the past with Location data (likely not as extensive as yours but I did burn a lot of cycles figuring it out).  I would make changes, it would get changed back to the old, etc  (you know the scenario very well).    What I traced it to was the inter-play between tags that are "supposed" to have the same value - which I think applies to your situation as well - with both the Lightroom and Dublin Core having tags that should mimic each other.

In my case, I got weird behaviour when these tags would get out of sync and between exiftool and iMatch they would reconcile the differences - and sometimes not in the manner I had expected. Once I sorted out the why, I was able to establish a process whereby I made sure they were synchronized in the way I expected and wanted.

You mention wanting to eliminate Dublin Core tags and refer to them as legacy.  I haven't come across that industry data that DC is out-of-date and - as you pointed out - iMatch locks out manual changes to the DC Keyword tag. Perhaps this is affecting the tag synchronization if you are eliminating that tag externally.   But I am speculating.

I also note from your past posts that some of these keywords are to be added due to people being tagged in the image.  I have found that if the people keywords get out of sync, one has to force some type of update in iMatch with People to have these keywords get generated.  No changes means iMatch has no reason to redo all of the people data - for example, you can manually modify the Person-in-Image tag and it becomes out of sync with the People identified in the image - but if you make any changes in iMatch with the faces or people, the tag becomes synchronized again.  This is strictly my observation and may not be correct.

Have you set up a panel (or VarToy) to show all of the tags that should (or may) contain keywords and be synchronized?

It could be that when doing a metadata writeback in iMatch, as there is nothing to write, iMatch does some sync of keyword tags and precedence is given to a tag you don't pay attention to?  (This is what I discovered in the past).

As a side note, as iMatch only instructs exiftool to update the tags it has identified as needing a change, if there is no identified "pending update" I would think nothing is written - which is why when you do a forced reload the "old data" is still there.

Hopefully this is useful food for thought.

Michael

[Mario]

As a side note, as iMatch only instructs exiftool to update the tags it has identified as needing a change,

Yes. That's the best way to handle this.
IMatch updates the tags modified by the user, then runs the XMP->EXIF, XMP->IPTC, XMP-GPS ExifTool scripts and ExifTool works it's magic of updating linked tags, timestamps, checksums and all that stuff.

If there is nothing to write, a write-back does nothing. As it should. You can always force a write-back of selected tags by marking them as modified with the pen icon in the Metadata Panel, if so desired.

Works beautifully since IMatch switched to ExifTool with IMatch 5 in 2016.

[Damit]

Thank you, Michael! It is good to hear I am not alone. I can't tell you how much I really appreciate you trying to help. I have spent ALOT of time trying to get things to work in IMatch, simply because it has so much potential, but I must say my trust in this being a rock-solid reliable program that will protect my metadata has taken a serious hit, which is truly sad. I am considering just dealing with the limitations of the other solutions I was using before, like ACDSee and Lightroom. The feature set is severely limited compared to IMatch but reliability and integrity is paramount, and at least their support won't abandon you. I still in shock that support would actually punt on an issue that is very well documented.

I went through a frustrating time in the past with Location data (likely not as extensive as yours but I did burn a lot of cycles figuring it out).  I would make changes, it would get changed back to the old, etc  (you know the scenario very well).    What I traced it to was the inter-play between tags that are "supposed" to have the same value - which I think applies to your situation as well - with both the Lightroom and Dublin Core having tags that should mimic each other....You mention wanting to eliminate Dublin Core tags and refer to them as legacy.  I haven't come across that industry data that DC is out-of-date and - as you pointed out - iMatch locks out manual changes to the DC Keyword tag. Perhaps this is affecting the tag synchronization if you are eliminating that tag externally.   But I am speculating.

So, it seems this is not a one user problem. Yet again, I am told I am the only one experiencing something and then someone finally chimes in to let it be known they had the problem as well. Very Frustrating! What you are describing is exactly what is happening to me. Things seemed to get out of sync, but I thought IMatch was supposed to be good at resolving such conflicts, at least it is advertised as such.
The most important thing is that Mario removes whatever changes he did to the diagnostic, if any were done, that may causes issues like this not to be caught because I have not updated and every single file that the diagnostic found to have missing keywords did, in fact, have missing keywords (I would really like to hear how and why they determined these were errors) and often because of a stubborn Dublin Core tag that would not change and was out of sync with the other keywords. The Database diagnostic was working properly and should not have been altered to not catch these issues. Mario, did you make changes to the Database diagnostic, and if so are you going to change it back? Will I be able to rely on it catching these issues in the future? As I shifted through the errors in the diagnostic, I finally stumbled on a solution, no thanks to support, who should have been able to spot some of these issues, but instead dismissed them. I realized that a lot of them had to do with specific entities. Troubleshooting on my own, I posited that it may have to do with the linked keywords, so I deleted the linked keywords, saved, and then added the keywords back again. More than half of the errors were gone after that. The fact that this fixed some errors certainly seems like these issues are an IMatch related problem.

I also note from your past posts that some of these keywords are to be added due to people being tagged in the image.  I have found that if the people keywords get out of sync, one has to force some type of update in iMatch with People to have these keywords get generated.

This is exactly what I described above.  When I made these changes, the people updated and corrected a bunch of files that had those people in them. Maybe I needed to update the people, like you mentioned. I could not find how to do this. Maybe Mario can instruct on how to update people, because maybe this would have done the same thing as what I did in a more efficient manner.

For other files I had to delete and redraw face regions, but these were only a few of the dozens I have already processed. On a couple I had to delete all the metadata through the exif tool to get rid of the Dublin Core information and then just wrote back with the information in lightroom because IMatch struggled to write it. It just would not and always reverted to the older out of sync data. Seems like another IMatch bug. Lightroom had no problem writing the correct information and IMatch read it.  I still have hundreds more to go through! 

You can always force a write-back of selected tags by marking them as modified with the pen icon in the Metadata Panel, if so desired.

I tried and this did not work. 
There should be a way to get IMatch to overwrite what is in the file and write what it knows (As evidenced in the thumbnail) should be written. I am shocked that it is unable to do this. 
Only doing what I described above is resolving the very real issues I am experiencing. Worst still is I went to my old database, well Attempt 3, because now I am in attempt 4 of just trying to get my pictures up to tag, and imported the modified people.  I went to the same files I just fixed in database 4 and they still had the erroneous information and would not update. Even worse, it would not reload with the correct data, even though it exists in the exiftool and database 4. So it seems I have to try again, piece meal, checking for errors using the relatively slow database diagnostic to deal with errors as they pop up (I really wish I could take some of those functions out-why do I always have to make the graph and optimize, sometimes you just want to see the errors). If this attempt does not work, I am done with this program and my reviews will reflect the experience I have had here.

Mario