What's wrong with my files? Metadata is written back partially.

[lbo]

Hi all,

maybe this is a FAQ, so a pointer to a previous thread is welcome. I searched the discussions, but couldn't resolve the problem, please excuse my ignorance.

My CR2 raw files have embedded metadata written by GeoSetter, no XMP sidecar file. To me, they don't look that bad. I trust in ExifTool and DxO reads all metadata correctly.

Find a demo file here: Description_by_Geosetter.CR2 (24,1 MB): https://app.box.com/s/gwoh3x0nbxyt4a99k6ez9y8rirvaj6yn

IMatch has the default Metadata and Metadata2 settings. Fresh install, no fiddling.

When IMatch scans the file the first time, the file window shows the pencil because HierarchicalSubject has to be written. No problem, I click the pencil and a XMP file is created and the CR2 is written.

BTW: HierarchicalSubject isn't written to the CR2 file. But the yellow pencil is away so I think IMatch considers the CR2+XMP file being o.k.

Then I modify Headline and Description by appending the string "IMatch changed" in the "Default" Metadata panel, hit Ctrl-S and click the pencil to write the changes from the database to the CR2+XMP file pair.

Two seconds later, my changes vanish in the Metadata panel.

In the CR2 file, IPTC Headline, IPTC Caption-Abstract, [IFD0] Image Description get the change, [ExifIFD] User Comment becomes the new description (was empty from GeoSetter).

But XMP-dc Description and XMP-photoshop Headline are not changed in the CR2 file.

What happens?

How can I use my images with IMatch?

Oliver

P.S.: Using the latest trial 2018.10.5

[Mario]

My CR2 raw files have embedded metadata written by GeoSetter, no XMP sidecar file.

The XMP ISO standard and the Metadata Working community are quite clear. The XMP metadata for RAW files should go into XMP sidecar files. Only one (!) XMP record should exist per file.
Although Canon also embeds a rudimentary metadata record inside the RAW in some camera - causing additional trouble.

No problem, I click the pencil and a XMP file is created and the CR2 is written.

Problem. IMatch writes the XMP sidecar file, but does not write XMP data to the RAW file. Hence you create two XMP records for the same file, which cause problems the next time IMatch has to rescan the file, because by default, the XMP data embedded in the RAW overrides the sidecar data. You need to decide if you follow the standard or want to work with XMP embedded in the file. Don't use two XMP records for the same file.

See also Metadata Storage

[lbo]

well, I have to deal with the current situation (roughly 8000 CR2 files with embedded metadata).

Several options come to mind:

1. Switch "keep existing XMP" to "off" and trust that IMatch keeps old data in sync

2. Manipulate existing CR2 files with ExifTool to move metadata to XMP sidecars

Do you have other suggestions?

Oliver

[Mario]

Have you looked at the XMP sidecar IMatch has created?
If will contain all the XMP that is also in the CR2, because IMatch reads the XMP from the CR2 and merges it with the XMP from the sidecar (if there is a sidecar).

If you first index the file, no sidecar exists. IMatch thus produces a rich XMP record by importing and mapping EXIF/legacy IPTC and the XMP data from the CR2.
When you write back, this XMP record is written to the sidecar.

From then on, the XMP record in the CR2 is superfluous and only gets in the way.

[lbo]

I didn't expect anything else, but how does your posting relate to my question?

Do you mean I should use option 2 or did you want to suggest a method I didn't mention yesterday?

Oliver

[Mario]

I commented on your option 2.
You don't need ExifTool to "move" your metadata into the sidecar. As I tried to explain, IMatch does that already.
If the XMP contains all you want, remove the superfluous XMP record from the RAW.
You must avoid to have two XMP records for the same file.
The standard is to keep the XMP in the sidecar for RAW files. This is also the default in IMatch.

[lbo]

You don't need ExifTool to "move" your metadata into the sidecar. As I tried to explain, IMatch does that already.

As far as I see, IMatch does not move but copy data to a sidecar. I guess, therefore you suggest a second step to complete the "move":

If the XMP contains all you want, remove the superfluous XMP record from the RAW.

What method do you suggest to "remove the superfluous XMP record from the RAW"?

Maybe that this is what I meant with "Manipulate existing CR2 files with ExifTool".

Oliver

[Mario]

IMatch does not delete XMP records from RAW files.
If you want to remove the XMP record from the RAW to have only one, in the sidecar, do as explained in this FAQ: https://www.photools.com/community/index.php?topic=4102.0

[jch2103]

IMatch does not delete XMP records from RAW files.
If you want to remove the XMP record from the RAW to have only one, in the sidecar, do as explained in this FAQ: https://www.photools.com/community/index.php?topic=4102.0

This is part of my standard workflow for one of my cameras that also embeds Rating in the RAW file (Nikon in this case). Quick and easy.

[lbo]

IMatch does not delete XMP records from RAW files.
If you want to remove the XMP record from the RAW to have only one, in the sidecar, do as explained in this FAQ: https://www.photools.com/community/index.php?topic=4102.0

This is part of my standard workflow for one of my cameras that also embeds Rating in the RAW file (Nikon in this case). Quick and easy.

no doubt that this is the right way for new files, and especially to remove data with no value.

In my case, it's about manipulating thousands of legacy raw files with embedded valuable metadata therefore I have to be rather sure to get everything right.

The method suggested by Mario consists of two separate steps:

• Let IMatch create the XMP sidecar
• Delete the embedded XMP data with the command processor

How can I be sure that I made no error in step 1 and I left raw files without sidecar?

Or, the other way round, how can I be sure that I run step 2 only on raw files for which a sidecar with the correct data has been created?

Oliver

[Mario]

1. NEVER manipulate your original (from camera) files. Always keep them backed up.

2. Check the contents of the side car file in ECP and also in the Metadata Panel in IMatch.

3. Use test files and verify.

You can see the data contained in the RAW in ECP.
You can also see the data contained in the XMP in ECP.
This allows you to compare, check for missing data etc.
If satisfied, remove partial XMP record from RAW file.