sidecar files for metadata

[mvkuilen]

Now that I've got my 380k images into iMatch, I have a few tasks ahead.
Learn iMatch
Review my workflow
Get out of the trial version.

As for workflow, up to now, I've used Downloader Pro to get the images off of the SD card and rename them yyyy-mm-dd-hh-mm-ss and store them in a similarly named folder in a local drive. Then a quick review and apply basic corrections in DXO. For the potential keepers, I apply more customized corrections in DXO. The local folder with these corrections is then copied to a network NAS. This is periodically backed up so there are at least two copies of all images at all times. 
I then plan on applying iMatch keywords in the NAS since iMatch knows about all the images in the NAS which is pretty stable. If I later edit the images some more on the local drive and then these revised images are then updated on the NAS, will all iMatch work be deleted? If so, what is the best way to preserve iMatch data associated with each image? Can this be done without having to copy the images back to the local drive? Can the metadata also be saved in sidecar files by iMatch?
The local drive is constantly changing so I don't tell iMatch about it. Once a folder has been worked on and no further work is planned, it is deleted from the local drive.

[Mario]

I then plan on applying iMatch keywords in the NAS since iMatch knows about all the images in the NAS which is pretty stable. If I later edit the images some more on the local drive and then these revised images are then updated on the NAS, will all iMatch work be deleted?

If you mean by this that you replace a file that has metadata embedded by IMatch with another file produced by whatever image editor you use, then yes. How could IMatch prevent that?
Note that DxO is not very good with metadata. It does not sync. It does produce only a subset of XMP. This means that whatever it outputs will be sub-standard compared to the metadata IMatch produces. And since you keep only one copy of your files in IMatch, IMatch versioning cannot help you either.
I recommend to edit metadata in IMatch for the file when DxO is done with it.

I would recommend that you consider the files you place on your NAS and add to IMatch as "final". If you really have to edit one of these files, pull it back to your local disk so DxO or whatever editor you use has at least a chance to retain the metadata. After modifying the file in your editor, double check the metadata to see what has been modified or stripped before replacing the file on the DAM.

Can the metadata also be saved in sidecar files by iMatch?

Where metadata must go is clearly defined in the IPTC/EXIF/XMP standards. This is not optional.
IPTC/EXIF/GPS always must be stored in the original image and synchronized with the corresponding elements in the XMP record.
XMP metadata must be stored in the image itself for file formats like JPG, TIFF, PNG, GIF, DNG, PSD, PSB,.... and in external XMP sidecar files for other formats, like RAW images.

The usual workflow for your scenario is to finish the files on the local disk.
This also gives the nest performance. When the files are final, move them to the NAS. All under the control of IMatch.
In the rare case that you have to re-edit an archived image, copy if back from the NAS to the local disk (in a folder managed by IMatch). Make your edits. You can then undone potential metadata problems by copying metadata from the file on the NAS back to the edited copy (again, in IMatch) before copying the edited copy back to the archive.

[mvkuilen]

Well I now have 1 out of 3 done (no more trial version, instead - imatch_licensed_2020_7_4_x64). Next on the agenda - learn iMatch and rethink my workflow.
With the NAS serving as both the final repository for finished files and as backup for the local drive while the images are being worked on, I think I will exclude the work-in-progress files in the NAS from iMatch. They change too often to have them moving back and forth between the NAS and the local drive. There are enough old files to keep iMatch busy for quite a while.

The majority of the files are RAW and they do have XMP sidecars. Since the RAW files are non-destructively edited with all changes stored in DOP sidecars, would any metadata not be affected if the DOP files are updated with new instructions?

[Mario]

For Learning:

The IMatch Learning Center has many free video tutorials.
And The IMatch Help System is only a <F1> key press away.

Since the RAW files are non-destructively edited with all changes stored in DOP sidecars, would any metadata not be affected if the DOP files are updated with new instructions?

I suppose. Unless DxO also makes changes to the XMP metadata. Or the EXIF/GPS in the RAW, which must be kept in sync with the XMP, both ways.
Making changes to an image file usually requires some corresponding XMP timestamps to change, maybe digest information updated, history data, ...etc. I don't know if and how much of the standard XMP workflows and requirements DxO implements. Or if they care at all. They go their own way with their proprietary DOP files...

[WebEngel]

I then plan on applying iMatch keywords in the NAS since iMatch knows about all the images in the NAS which is pretty stable. If I later edit the images some more on the local drive and then these revised images are then updated on the NAS, will all iMatch work be deleted?

If you mean by this that you replace a file that has metadata embedded by IMatch with another file produced by whatever image editor you use, then yes. How could IMatch prevent that?

Hi Mario,

are you not too modest about Imatch's capabilities?  Isn't the metadata setting "Protect existing XMP" doing exactly what the user wants?  I have a similar workflow as the user above, and IMatch is the perfect tool for it.

martin

[Mario]

The protect metadata feature tells IMatch not to replace XMP or modified tags when re-ingesting an externally modified file.
The idea is to prevent users from accidentally wiping out unwritten changes in the IMatch database.
It depends on your workflow and which other applications you are using if this is needed or useful.

If you work with multiple applications on the same files, you should always make sure IMatch has written back modified metadata so the other applications can see the data. And vice-versa.
It is common practice among IMatch users to modify metadata only in IMatch - because most other applications out there are not 'good' at working with metadata as ExifTool/IMatch. They don't handle keywords properly, don't synchronize EXIF/IPTC/GPS data, may strip out maker notes and all these unpleasant things.

Metadata was not designed (not even XMP) to be modified concurrently by multiple applications. There is no locking or versioning mechanism. There is sorts of digest information and some tracking in XMP, but virtually no application really uses that. It's complicated to implement and most vendors just don't care enough. Especially not RAW processor vendors who often don't see metadata management as a priority.

IMatch uses metadata protection.
Lightroom explicitly asks the user before importing metadata from files that have been modified by other applications.
Each application has to deal with this somehow, but there is still a chance that a non-compliant or just not caring application causes problems.

I usually recommend to set/add metadata as the last step in your workflow, when the image and versions are final. This way you avoid all potential issues caused by multi-application workflows.