Metadata going out of sync

rolandgifford · December 08, 2023, 09:06:41 PM

I'm working through selecting images for a project. I'm doing this by running searches and setting a new Keyword to all images found, another search to assign the same keyword and so on till I have the keyword assigned to all candidates following which I remove the keyword from images I don't want to be included.

There my be other/better ways but this is what I do.

I periodically write back metadata to 'save' where I'm up to. All images before I start have had metadata written back and IMatch showed no images outstanding.

I currently have 493 images pending writeback and running the command to writeback all outstanding has no effect on this number. Many of them are raw files which are usually part of a 4 image set, with a the out of camera JPG as the master.

Showing a single image and viewing the output panel shows no errors. I have uploaded metadata analysis for a single image, a copy of the output panel and a debug log.

I'm partly wondering why this has happened and partly wondering how to fix it

rolandgifford · December 09, 2023, 12:41:21 PM

I should perhaps have added that I never enter details directly against raw files. I only ever make changes to the master of the version set and everything gets propagated from there

Mario · December 09, 2023, 01:10:43 PM

Note:Use the GREEN button in Metadata Analyst to copy the relevant results into the clipboard and paste them into your reply.
Wading though JSON extracts is a lot more time consuming for me.

The output panel shows something strange.
IMatch is only writing legacy IPTC data to the RW2 file, but no XMP keywords are written in this output operation.
What are your propagation settings? Which tag groups to you propagate?
Do you use any non-standard settings for metadata?
Did you change keywords in the Keyword Panel for the RAW or a version?
IMatch always writes XMP hierarchical keywords and flat keywords. And legacy IPTC only when the output file already contains legacy IPTC data.

rolandgifford · December 09, 2023, 01:24:22 PM

Metadata Analyst Green button

====================================

Metadata Analyst Results. Version 2023.4.6. 12/9/2023 12:15:57 PM
File analyzed: H:\Selected Photos\Holidays\2018_0902 Sulawesi\RolandSulawesi 2018_0911_101222.RW2
Errors: 1
Warnings: 11

Warning: [System] File has unwritten metadata (pending write-back).<br/>The metadata loaded from the image and the data in the database may not match.
Warning: [Legacy IPTC] Character Set Encoding: unspecified.
Warning: [XMP] Embedded XMP record (photools.com IMatch 23.1.0.22 (Windows)) and XMP sidecar file (photools.com IMatch 23.4.0.6 (Windows)) found.
Warning: [XMP] Embedded XMP rating is 0.
Warning: [XMP] [ExifIFD]:CreateDate and [XMP-xmp]:CreateDate (embedded) mismatch.
Warning: [XMP] [IFD0]:Copyright and [XMP-dc]:Rights (embedded) mismatch.
Warning: [XMP] [IPTC]:CopyrightNotice and [XMP-dc]:Rights (embedded) mismatch.
Warning: [XMP] [XMP-iptcExt]:LocationCreatedGPSLatitude not mapped to [XMP-exif]:GPSDestLatitude (embedded).
Warning: [XMP] [XMP-iptcExt]:LocationCreatedGPSLatitude not mapped to [XMP-exif]:GPSDestLatitude (sidecar).
Warning: [XMP] [XMP-iptcExt]:LocationCreatedGPSLongitude not mapped to [XMP::iptcExt]:GPSDestLongitude (embedded).
Warning: [XMP] [XMP-iptcExt]:LocationCreatedGPSLongitude not mapped to [XMP::iptcExt]:GPSDestLongitude (sidecar).
Error: [Keywords] Different XMP keywords in embedded XMP record and sidecar file.

ADMIN: PlEASE never copy tons of debug or output data in your replies. This makes them hard to read and process and also fills the community search engine with junk, making it less useful for all users.
This time I have saved the output file dump to a text and and attached it.

What are your propagation settings? Which tag groups to you propagate?

Everything between Categories and File Protection
XMP Keywords
XMP All Data, disabled crop

Don't copy XMP Orientation
Don't copy XMP Regions (Face data)

Do you use any non-standard settings for metadata?

No

Did you change keywords in the Keyword Panel for the RAW or a version?

The master for the version set. I don't ever change any child in the version set

Mario · December 09, 2023, 02:07:05 PM

QuoteWarning: [XMP] Embedded XMP record (photools.com IMatch 23.1.0.22 (Windows)) and XMP sidecar file (photools.com IMatch 23.4.0.6 (Windows)) found.

Why has the RAW file an XMP record written by IMatch?
IMatch by default never writes XMP metadata to RAW files. It always uses the XMP sidecar file.

QuoteError: [Keywords] Different XMP keywords in embedded XMP record and sidecar file.

This is caused by the RW2 file having embedded XMP data.

As a first step, remove the embedded XMP metadata with the ExifTool Command Processor and the "Delete XMP Metadata" preset. Select the RW2 file, run the ECP and apply the preset.

Clear the Output Panel.
Change a keyword in the Keywords Panel for the master and write back the master. This will trigger propagation and write-backs for all files.

Copy the contents of the output panel to a text file and attach.

rolandgifford · December 09, 2023, 02:55:19 PM

Quote from: Mario on December 09, 2023, 02:07:05 PMWhy has the RAW file an XMP record written by IMatch?
IMatch by default never writes XMP metadata to RAW files. It always uses the XMP sidecar file.

No idea. This photo will originally have been updated by software other than IMatch. It has been successfully written by IMatch in the past.

I followed your steps and the raw file is no longer in the Pending list. Output panel contents attached

Mario · December 09, 2023, 03:03:48 PM

This time hierarchicalSubject were copied, not only legacy IPTC data.

I assume the problem is solved?

rolandgifford · December 09, 2023, 03:55:42 PM

Quote from: Mario on December 09, 2023, 03:03:48 PMI assume the problem is solved?

Yes and no

Removing XMP metadata from the raw files and updating the master for each (presumably one at a time as I don't know how to select them all at once) fixes the broken data.

It doesn't tell me what broke the data. These files have previously been written back OK and I haven't changed any of the raw files directly

Mario · December 09, 2023, 04:31:18 PM

I have no idea, either.
The only way to make IMatch write XMP into RAW files is to change the file format settings in Edit > Preferences > Metadata : File Formats. Or doing it manually via ExifTool. I think.

The XMP data in the RAW was written by 23.1.22, which was released on August 8 and replaced by IMatch 2023.2 on August 15.
I don't see anything related to propagation or IMatch embedded XMP into RAW files for some reason in the release notes.

rolandgifford · December 09, 2023, 06:05:34 PM

Quote from: Mario on December 09, 2023, 04:31:18 PMI have no idea, either.
The only way to make IMatch write XMP into RAW files is to change the file format settings in Edit > Preferences > Metadata : File Formats. Or doing it manually via ExifTool. I think.

I have checked and I haven't changed any file formats from the default on the Metadata2 settings. I certainly haven't done ExitTool manual updates.

However, I have a vague recollection a discussion on here about embedded and sidecar settings for some video format and me changing some setting as a result of that. I can't remember what it was but possibly linked, which is why I mention it.

I'll go away and fix these images now