Old IPTC keywords - or not?

[jch2103]

I have approximately 70 NEF files for which I get recurring yellow pencils (for IPTC::ApplicationRecord\Keywords). Various metadata editors were used on these files over the years, including GeoSetter and various versions of IMatch. No other files in my database show this behavior. In theory, it should be easy to delete any existing IPTC keywords and make this issue go away.

However, according to EXIFToolGUI, there are no IPTC tags for these files. I get the same result when I list all metadata with the ExifTool Command Processor (i.e., no IPTC lags are listed - see attached). If I use the ECP (or ExifToolGUI) to attempt to delete IPTC metadata in the NEF file (-overwrite_original_in_place -IPTC:All= {Files}), I get the message that the file is left unchanged.

If I run Rescan Now/Reload Metadata, the yellow pencil goes away and stays away in my IMatch session. It also stays away if I try Rescan Now/Normal Rescan or Rescan Now/Force Update. If I restart IMatch, the yellow pencil will not be there.

But (!) at some later time when I start a new IMatch session, I find these same files pending in Collections/Pending Metadata Writeback. This is beginning to drive me a little nuts, as I think I have the problem solved only to have it come back later.

Any ideas or suggestions on how to fix this issue?


[attachment deleted by admin]

[Aubrey]

John,
i have faced issues with IPTC keywords reappearing. In fact it again happened last week. I went back and reviewed an earlier discussion I had had in the forum.
https://www.photools.com/community/index.php?topic=1930.msg12439#msg12439

In a nutshell what I have just done, and it finally sorted the issue for me.
Change the metadata2 NEF settings from default to ensure that they are not written again.
Manually delete the IPTC keywords in the file

I modified my metadata2 as shown in the attached jpg. See Mario's comment in the thread (dated 17 March 2014 time 10:09:54 am)

To delete from the NEF file (I expect it works with other file types also), I used the script, to be run from exif command (you are a long time member so I expect you understand what to do) given by Joerg (joel23) in the thread referred to above:

-overwrite_original_in_place
-IPTC:Keywords=
-XMP:Subject=
-XMP:HierarchicalSubject=
{Files}

Perhaps this will resolve your issue.

Aubrey.



[attachment deleted by admin]

[RalfC]

I have approximately 70 NEF files ...

Any ideas or suggestions on how to fix this issue?

NEF and Metadata: a never ending story. Mario rants about it frequently (felt: at least once a week). 

Did you try what happens if you write the Metadata once? (If not, try it with one file first and check the exiftool messages in the output monitor)

NEF-Files might contain XMP-Data.
Depending on your Metadata settings, this might cause that IMatch intends to create/update an (external) XMP-file.

Regards,
Ralf

[jch2103]

Thanks, Aubrey and RalfC. I tried your suggestions and experimented with the files in question. It appears that the root of the problem was that XMP keywords data had gotten embedded in the NEF files at some point, which caused further synchronization issues (somehow including ITPC) in conflict with the XMP sidecar files. I probably should have realized this before; I had changed capitalization of some keywords in the files, but the changes weren't showing up in my metadata panel. If I'd pursued the issue at the time I might have cleared it up earlier.

Bottom line: I *think* I've cleared up the problem with these files (but of course I thought I'd fixed them several times before, and the problem came back...!). Thanks again!

[Edit] Hmm! I wrote too soon! I ran IMatch diagnostics after doing the above, and got a warning that there were 70 files with metadata issues. Sure enough, the same 70 files came back in Pending Metadata Write-back. This time, I again set the Metadata2 preferences as suggested, and clicked the pencil. The only message in the output panel was a usual one about sub-second issues. The pending files all went away. I then re-ran IMatch diagnostics, this time with no warnings or errors. (Yea!) It would seem that the 'IPTC' data had gotten lodged in the IMatch database. I hope things are finally cleared up now...

As I've said before, you never know how bad your metadata are unless you work with an excellent tool like IMatch!

[Aubrey]

John,
Glad to hear that you have finally resolved your issue.
The issue with my files arose in a similar manner. I was using GPISYNC and allowed the default "Geonames in IPTC" when I was on the learning curve with this GPS sync software. I now know better.

Aubrey.

[Carlo Didier]

I had similar problems with some jpeg files and the IPTC::ApplicationRecord\Keywords field.
Exiftool showed two keywords in the XMP:Subject field. No way of deleting either field solved the problem. I could add a new keyword which would then appear as the only one and once I deleted that, the two old ones re-appeared. The only way to get rid of them was to completely remove ALL metadata. Drove me nuts!

[Mario]

The re-appearing keywords problem is usually caused by

a) legacy IPTC data in the image file, and IMatch is not allowed to update IPTC when writing back

and/or (!)

b) Duplicate XMP records (embedded in the file, plus an external XMP record in a sidecar file).


The solution is either to allow IMatch to synch (IPTC) or to remove the duplicate and competing XMP record (NEF files processed by Capture, mostly).

[Carlo Didier]

Mario, this was outside of iMatch. I was using standalone Exiftool only. No sidecar files existed.

[jch2103]

Just to prolong this discussion, my original post noted that:

However, according to EXIFToolGUI, there are no IPTC tags for these files. I get the same result when I list all metadata with the ExifTool Command Processor (i.e., no IPTC lags are listed - see attached). If I use the ECP (or ExifToolGUI) to attempt to delete IPTC metadata in the NEF file (-overwrite_original_in_place -IPTC:All= {Files}), I get the message that the file is left unchanged.

To further clarify, there was no IPTC::ApplicationRecord\Keywords data in the xmp sidecar file either.

Based on the behavior I saw, it seemed as if the IPTC::ApplicationRecord\Keywords existed somehow in the IMatch database, or alternatively was somehow hidden from ExifTool in the NEF files. Neither 'explanation' seems logical, though.

[Mario]

There is legacy IPTC data, which can be in the NEF file.
There is IPTCCore and related data, which can be in the XMP inside the NEF, or in the sidecar, or both.
ExifTool looks only at the file you tell, it does not consider XMP sidecar files when you look at a NEF.
IMatch produces legacy IPTC data when a) enabled and b) the NEF file already has legacy IPTC data. It does not create new IPTC data by default, in accordance with the MWG rules.

[jch2103]

There is legacy IPTC data, which can be in the NEF file.
There is IPTCCore and related data, which can be in the XMP inside the NEF, or in the sidecar, or both.
ExifTool looks only at the file you tell, it does not consider XMP sidecar files when you look at a NEF.
IMatch produces legacy IPTC data when a) enabled and b) the NEF file already has legacy IPTC data. It does not create new IPTC data by default, in accordance with the MWG rules.

"ExifTool looks only at the file you tell" >> ExifTool reported no IPTC data in the file.
"it does not consider XMP sidecar files when you look at a NEF" >> A review of the XMP sidecar file showed there were no IPTC data in it.

Hence my confusion: If there were no IPTC data in the NEF or in the XMP, and I was able to clear the 'write' flag from the file, why did IMatch subsequently (apparently only after running diagnostics) reinstate the 'write' flag? Where was the IPTC 'data' hiding in the meantime?

[Carlo Didier]

Well, I had the exact same problem as John described, but with simple Jpeg files without xmp sidecars. Maybe somthing was propagated during raw conversion into the final Jpeg.
As there is no way, even with ExifTool to get rid of this, it is really very annoying.

[RalfC]

Hence my confusion: If there were no IPTC data in the NEF or in the XMP (...) Where was the IPTC 'data' hiding in the meantime?

A NEF can contain IPTC and/or XMP-Metadata.  If there is XMP-Data inside of the NEF you will see such a bevaiour as well as if there would be IPTC data.
Did you actually check if there was no XMP-Metadata inside the NEF?

Regards,
Ralf

[jch2103]

Did you actually check if there was no XMP-Metadata inside the NEF?

I probably wasn't clear enough in https://www.photools.com/community/index.php?topic=4449.msg29821#msg29821.
Yes, per your suggestion there was in fact XMP data in the NEF, and yes, I did then delete it. At that point, the 'pencil' went away, and I thought the problem was resolved. But when I ran Diagnostics in a subsequent session, the pencil (proposing to write to IPTC::ApplicationRecord\Keywords) came back. That's why I'm perplexed.


To finally resolve the problem, I again set Metadata2 to prevent writes of IPTC to the NEF and clicked the pencil with the Output window open. Nothing was written in the Output window, but the pencil went away. I reran Diagnostics (no problems detected) and changed Metadata2 back to NEF defaults. All is good now, except for fully understanding what happened.

[Mario]

(proposing to write to IPTC::ApplicationRecord\Keywords) came back. That's why I'm perplexed.

If you have configured IMatch to update existing legacy IPTC (this is the default) and your NEF file contains an IPTC record, IMatch will do exactly that. This is required in order to not let XMP and legacy IPTC get out of synch.

[jch2103]

(proposing to write to IPTC::ApplicationRecord\Keywords) came back. That's why I'm perplexed.

If you have configured IMatch to update existing legacy IPTC (this is the default) and your NEF file contains an IPTC record, IMatch will do exactly that. This is required in order to not let XMP and legacy IPTC get out of synch.

Yes, of course. But at the risk of beating a dead horse, at the point when the pencil went away, the NEF file did NOT contain any IPTC and I had just deleted all XMP data in the NEF. And the sidecar xmp file did not contain any IPTC data. (Matadata2 was set for NEF default at that point.) But the pencil came back, only after I ran Diagnostics.

[Mario]

IMatch only writes IPTC to the NEF when it detects (even a spurious) IPTC record. Deleting XMP from the NEF file will not delete the legacy IPTC record.