Reverse metadata propagation from Version to Master

[cytochrome]

As it happens I have several thousand jpg with IPTC/XMP metadata. The corresponding Raw master files have either xmp  sidecars or nothing.

I like to have the (standard) IPTC/XMP written in the RAW. Is it feasible to reverse propagate  metadata in IMatch?

With the sidecars it is easy, but with nothing, that is directly from jpg to raw Of course I can do it with Exiftool but it is a pain since the JPGs are in sub-folders to the Raw. Would be much nicer if I could select the JPGs and let the file relations and Exiftool do the rest

Francis

[Mario]

Is it feasible to reverse propagate  metadata in IMatch?

No. It's complicated enough to propagate metadata from masters to versions. But you decide what you consider a master or a version. It is not mandatory to make the RAW the master. And you can use one relation rule now, and later delete it and setup another relation rule... did you try that?

[Ferdinand]

As it happens I have several thousand jpg with IPTC/XMP metadata. The corresponding Raw master files have either xmp  sidecars or nothing.
I like to have the (standard) IPTC/XMP written in the RAW. Is it feasible to reverse propagate  metadata in IMatch?

Now I understand your problem.  As Mario said, you're not going to get this natively in IMatch.  A script could do it.  It would be possible to combine bits of this reverse copy ratings script with bits of my cats to keywords script to do this, but it would be some work and I'm not in a position to offer at the moment.  It's also a slightly dangerous script IMHO.

[Mario]

I like to have the (standard) IPTC/XMP written in the RAW

This requires a full copy of the metadata, potentially merging already existing embedded XMP data in the master with XMP data from the version or the sidecar files. Quite an effort and with some potential pitfalls as well.

IMatch does not work that way with metadata. When updating metadata, IMatch uses the current state of the database (meaning the metadata imported from the file and looking at which data was changed) and then writes out the changes as a "diff" to the file on disk, adding, updating and removing only the required tags. This ensures that ExifTool can operate efficiently and that the metadata in the file remains unchanged except for where it actually needs to be updated.

I would consider setting up a solitary relation rule which makes the JPEG the master and the RAW the version. Then do a manual propagation via the corresponding command. This is basically a "reverse" propagation, at least if one always considers the RAW as the master. If this works, the temporary rule can be dropped and the standard RAW = master rule re-enabled.

[Ferdinand]

I can imagine a situation where someone has a RAW format which didn't have IPTC support in IMatch 3.6.  So they entered IPTC in the JPG version (rather than trying to use the XMP editor).  Now that they've got IMatch 5, or soon will have, they'd like to copy this to the XMP sidecar for the RAW.  All you're looking to do is to copy the IPTC fields into their XMP equivalents.  I'd have thought that this was possible and safe to do in a script if the RAW had no XMP.  If it did then you'd be overwriting the previous values.

[Mario]

"Metadata? Keep it as simple and standard as possible."

This is not directed specifically at you, Ferdinand. But you came up with a theoretical case and I use that to add some info - for the benefit of future readers.


This is a very specialized requirement. Something that ExifTool is very good at.
"Fixing" metadata or copying/merging/bridging metadata in such special cases is not the scope of a DAM like IMatch. Although IMatch versioning and propagation is quite capable and flexible, it cannot cater for all fringe cases. Sorry for that, but the system is complex enough as it is.

By default IMatch 3 imports IPTC into XMP. So you start with the IPTC Core in the XMP and then you need to devise (depends on your requirements) if you want to copy this IPTC data from the JPEG sidecar to the RAW sidecar. Or copy the classic IPTC into the RAW as well to prevent an out-of-synch between IPTC embedded in the RAW and IPTC in the XMP. There are many special situations which need to be taken care of here, and these usually depend on what unusual thing the user did with the metadata in his files in the past. All that is way to special for a general feature, and also out of scope for versioning or propagation. In typical workflows (Adobe or standard RAW processors, image editors, pano software etc.) such special cases usually don't occur. The majority of users will never have these "problems", thankfully.


What you describe in your theoretical situation calls for either some custom ExifTool command lines or even a shell or IMatch script which understands the situation and constellation of available and desired metadata. For example:

+ RAW files may contain embedded classic IPTC
+ RAW files may contain embedded XMP, where the XMP optionally contains an IPTC Core name space. This IPTC core name space may/shall contain a copy of the classic IPTC data in the file (if there is some), but may not
+ RAW files may have an external sidecar XMP file
+ In worse cases, RAW files have embedded metadata (XMP and/or IPTC) and a XMP sidecar file with potential out-of-synch IPTC/XMP data - compared to the embedded XMP in the RAW
+ By definition, a XMP file belongs to all files with the same name, which means that RAW and JPEG share the same data. Unless you move your JPEG files into other folders or you keep a separate set of XMP data embedded in the XMP. It depends on the application (or configuration) whether the external XMP sidecar is used for the JPEG or the embedded XMP record. If your application gives you no control over the process they usually ignore XMP in sidecar files if the JPEG has embedded XMP data - or quite the opposite.

All these are special cases (and there are more) which require some extra care and attention to handle properly. Most of this is out of scope for IMatch. Luckily, the majority of users does not face such problems, ever. IMatch 3 synchronizes IPTC and XMP automatically (IPTC->XMP) and keeps XMP sidecar files for RAW files by default. Very good interoperability. Adobe applications embed XMP in standard formats like JPEG or DBG, and maintain XMP in sidecar files for RAW. Other RAW processors usually do the same. But as soon as a user steps from the standard path, additional work or even problems may be the result.

As long as a user has done nothing special with his metadata, files will just flow into IMatch 5 and all metadata will work out of the box. If a user took liberties or utilized unusual schemes (like using different metadata for JPEG and the associated RAW file) he has to solve that manually before the files enter IMatch. But these are (hopefully) rare and fringe cases. As soon as the data has been modified to fit into one of the established and supported standard workflows, IMatch 5 will take care for the rest.

[cytochrome]

I more or less guessed about the answers, but I asked my question anyway, maybe others would be interested...

First Ferdinand got it exactly: "Now I understand your problem." Yes you do and "I can imagine a situation where someone has a RAW format which didn't have IPTC support in IMatch 3.6.  So they entered IPTC in the JPG version (rather than trying to use the XMP editor).  Now that they've got IMatch 5, or soon will have, they'd like to copy this to the XMP sidecar for the RAW.  All you're looking to do is to copy the IPTC fields into their XMP equivalents.  I'd have thought that this was possible and safe to do in a script if the RAW had no XMP.  If it did then you'd be overwriting the previous values."

Exactly. Since 2 years, in addition to my Nikons I shoot also a Panasonic. The RW2 has NO XMP I'm aware of, only Exif/Makernotes. 

- since many years Imatch supports writing IPTC/XMP to the NEF. I use it, no sidecars, like it, and for consistency in my work flow I would like to do the same for the RW2.

- the IPTC I use are strictly Author, Copyright, Headline, Description and full Location. No keyword or exotica

- I ingest with Photomechanic. It did not write to the RW2 so I had it write xmp sidecars. I lost half of these (quite many) after converting RW2 to jpg. The JPGs retain the IPTC, the rest is lost

- lately there was a PM version that could write IPTC/XMP to the RW2. Photomechanic dropped it because some files were no longer opened by Adobe products. I don't use Adobe, in ASP they look fine and ASP reads in metadata from the RXW2 and propagates it to the JPGs. I kept this PM version for my Panasonic files, so all my recent RW2 have in-file XMP.

- IM3.6 would not read nor write to the RW2. On an individual basis it is easy to do with ExifTool, so I did it but it is a pain

- when I saw IM5 I was thrilled, it could read the in-file RW2 metadata and versions + file relations opende great hopes. I thought voila la solution à mon problème...

I understand Mario, it is certainly not the task of a DAM to solve such particular needs. It is only the Swiss army knife aspect of iM5 that gave me hopes...

Just for the fun of discussion, Mario raises many objections, but they don't apply in this case:

- RW2 files contain NO IPTC, classic or XMP embeded
- I can restrict to the folders where RW2 have no xmp sidecars
- my JPGs are all in sub-folders to the RW2 folders

I will try what Mario suggests, try to define a one time rule where the JPG in my Panasonic file branch are the master and the RW2 the versions. And see what happens.

In the worst case I can go back to PM. It has a setting for copying metadata from JPG to Raw. But looks complicated and unsure. I would have more confidence in an IMatch solution.

Francis

[Mario]

I'm sure you will get some success with either

- A reverse versioning rule
- A Exiftool Command Processor statement maybe, which you run on the JPEG and which updates the corresponding RAW?
- A small script which uses Application.ShellExecute to run ExifTool with some command line arguments and the name of the JPEG source and RAW destination

there are enough ways in IMatch. But for your specific problem you will need a very specific solution.

[Ferdinand]

- A small script which uses Application.ShellExecute to run ExifTool with some command line arguments and the name of the JPEG source and RAW destination

I had in mind a script that read the metadata variable(s) in the JPG and used this value to set the corresponding variable(s) for the RAW, all using native IMatch 5 methods.  That way the XMP Export preferences would be respected.

I realise that you're (Mario) often pointing out ways to use the native functionality of IMatch rather than using scripts, as we did in IMatch 3.6.  This is good - we need to understand the capabilities of V5 better and get out of our old paradigms.  The thing that worries me in this case is that reverse propagation is dangerous - you need to make sure that that temporary rule is deleted or off.  At least with a script you only run it manually and you choose the images to run it on.

[Mario]

you need to make sure that that temporary rule is deleted or off.

Yes. But my understanding is that Francis needs to do this only once - to fix his existing metadata and to standardize it. Remembering to delete the temporary rule afterwards should pose no problem.

I guess that we'll encounter more users who have used different ways to store metadata over the years. To work around problems in other software or to setup a metadata storage system which fit into their workflows. Things change and metadata is no exception.  And maybe these users need to fix their metadata in some way, e.g. re-combining XMP, strip embedded XMP from RAW files into sidecar files (or the opposite), map/bridge metadata, ...

All these are usually custom problems which need custom solutions. ExifTool is ideal for this kind of one-time metadata processing. Either via the built-in ExifTool Command Processor in IMatch 5, via thee free ExifTool GUI software or native via the ExifTool command line. The ExifTool web site has tons of examples and FAQ articles for the most common operations, plus a user forum full of detail solutions for thousands of problems solved so far.

An IMatch script could provide some additional comfort but would not necessarily make things easier.

For example, last week I had to fix some metadata myself.  Basically I had to produce an ARG file for ExifTool with some arguments and a list of several hundred file names to process.

I fired up Notepad, entered the ExifTool arguments required for the job. I found them on the ExifTool user forum in a post 

Then I selected the files in IMatch 5 via a combination of Filters on the entire database and finally used <Ctrl>+<C> to copy the file names into the Windows clipboard. From there I pasted the file names into my ARG file in Notepad and saved it.

Ran it via

exiftool -@ mario.args

on the command line and ExifTool did the rest. Repaired metadata in about 300 files from various folders and time periods in less than five minutes. A quick rescan in IMatch 5 and the repaired metadata was available there as well.

[cytochrome]

Thank you so much Mario and Ferdinand for this discussion, it gives me ideas and clues. I will first go the relation files way (and not forget to disable them after use).

But right now I am frenching (mon dieu!!) the strings (!!!) in imatch5res_str.xml... takes more time than expected. My reverse thing will come after.

Francis

[Mario]

Thank you for your help with translating IMatch. I'm fully aware that the initial amount of work is high. But once this is done, new versions of IMatch will only require a few new strings, messages and dialogs.


Help Wanted!
If you want to help translating IMatch in YOUR language, help us out. See

https://www.photools.com/community/index.php?topic=56.0

for details.

It's fun. You'll be famous! Your name in the IMatch About dialog!!!

[cytochrome]

I was not complaining... I volunteered and while translating I am "pre-learning" about functions I  never used!!

Will be useful some day.

And I am almost through this one. Wehn done I'll be back in the translator section for more questions.

Francis

[Mario]

I was not complaining...

I know.

I volunteered and while translating I am "pre-learning" about functions I  never used!!