XMP Export setting

[philippe28]

Hi Mario
The setting Allow to create XMP files (disabled) is marked as disabled.
But it seems that setting no actually prevents IMatch to create the xmp file.
This is good for me but I don't understand why it shown as disabled.
Could you explain ? Will we lose this feature in the future ?
Thanks
Philippe

[Mario]

This has historical reasons. It should have no effect.
What exactly did you do?
Not creating XMP files is no option anymore and I just left in the option. For quite some time. I shall remove this at some point.

[philippe28]

That would be really a shame  for me not to have the option not to write to file (in fact the option to do it only when I need).
Today I'm not obliged to create xmp and to write back metadata and I'm very happy with that.
The import/export features of IMatch are key to share data with other applications, but also, as you explain, not to be prisoner of the tool, whatever the reasons... that really great ... when needed.
I love not to be obliged to use files. The fact to be obliged would be a constraint.
Not having to transit through files guarantees to get the maximum speed for all internal Imatch operations.
On the other hand I do not need the photo editor knows the GPS coordinates or the name of my aunt.
I don't want to worry about non-wished interactions or pollution on shared XMP files. If IMatch, and probably some other applications, manage properly XMP it is not the case of all applications.
At publishing time, I like that IMatch gives me control of which data I share. Sometime it is useful to share GPS coordinates sometime I prefer not.
I have already seen some internal functions not working without write back like metadata propagation or filter on GPS data (there may be others). That's a bit of pain for me, but so far that's not a roadblock thanks to the scripting possibilities you offer.
I understand that exiftool brings some benefits on the design side, but I would appreciate to continue to enjoy both the speed of internal operations on database and the power of communication with other tools... when I need it.

[Mario]

The only way to store XMP data outside the database are XMP files (for RAW images and all images not in the JPG, DNG, TIF, PNG or PSD format).

[philippe28]

The only way to store XMP data outside the database are XMP files. That's perfectly clear and I don't see any issue there.
The only point for me is to be able to use them when I want to do it (i.e. when I want to import or export data) and not to be obliged to have them and to deal with them the rest of the time. At least, that's my wish.

[Mario]

If you don't write-back, IMatch does not create XMP files. Or update your images.
But when you write-back, IMatch will always create an XMP sidecar file for file formats which don't support embedded XMP data.
This is inevitable and there are many complex processes which depend on it. From proper synchronization between IPTC, EXIF, GPS (which are always stored in the image) and XMP to file relations, buddy file processing and versioning.

[philippe28]

Thank you Mario for the explanation.
That's good for me then.

[philippe28]

Another feature which doesn't work without writing back: when exporting images, the metadata of the images are not written, even the exif ones.

I've then made a try writing back on my working folder (700 images). Several minutes to have it done.
Then I can propagate metadata with the standard feature (300 images). Again some minutes.

Some issues I've faced during my operations:
- when the master is a jpg, I can run the propagation to version, the metadata are actually written in the version file, but I don't see them in IMatch, even after rescan.
- when Rescan or write back, the selection is not the one I expect.
  - Once I've launched a rescan on Version files. The versions have not been rescanned but the master files have been instead (probably the last selection I'd made in fact). This is not a problem when all xmp are up to date, but I've lost a lot of metadata with this operation. I may have missed something but the result is weird. How can I be sure of what is rescanned ?
- When I wanted to write back the Master files, I made sure that these files were selected, but IMatch included also the version files (displayed number of files). Is there some setting to limit the operation to selected files ?

[Mario]

When you write back, the master and naturally all versions are written. A must.

What do you mean with "exporting images"? There is no such feature in IMatch.


I have removed the "Don't create XMP files" option for the next release.

[philippe28]

I have an album of images (nef) with metadata but without xmp.
When I select the images and Write Back, all metadata disappear.
Hopefully I've a backup but I don't figure out how I get out of this situation.
Do you have any advice ?
Thanks
Philippe

[Mario]

When you write back to NEF files, all metadata disappears? Even EXIF, IPTC and GPS information? Is the metadata panel completely blank afterwards?
Or does only some metadata disappear?
Does IMatch report any problems writing to the file in the log file?
Do you use the default settings for metadata?
Does your NEF file contain legacy IPTC data?

You can upload a NEF file somewhere and I can see if I can write metadata to it. I use NEF files with IMatch all day, and they work just fine.

[philippe28]

Only metadata I've input previously disappear. The exif ones are still there. The Nef are the original ones.
I've attached the log file where I've done the following actions (after a restore of the database):
- select one image and write back.
- as no xmp is written, I've entered another field (City) and write-back again
- the xmp file does appear but former metadata (keywords, gps) have disappeared.
- on the corresponding version image the metadata are still there.
Note: when I started to work with Imatch I've set the flag "Allow to create XMP files (disabled)" to no. But seen the different difficulties that creates I'm switching to the write back mode.
I've attached the related files.
Thank you

[Mario]

This option has no effect. IMatch ignores it.

The log file tells us:

WriteBack created XMP file [56055] 'D:\DOCUME~1\Images\Photos\2010\2017\201706~2\201706~4.XMP
WriteBack successful for file [56055] 'D:\Documents\Images\Photos\2010\2017\20170605_Cahuzères\20170605_Cahuzères_008.NEF' in 3 s

Can you please open "D:\Documents\Images\Photos\2010\2017\20170605_Cahuzères\" and check if the XMP file for 20170605_Cahuzères_008.NEF exists?

[philippe28]

Yes the xmp files exists. That's the one I've attached in the previous post.

[Mario]

The XMP file is OK.

It contains an entry for city: Cahuzères in both the Iptc4xmpExt and the photoshop XMP namespaces. Perfect.

I have copied your XMP file as abc.XMP in the same folder as an abc.NEF.
After importing the file into IMatch, the City shows correctly in the metadata panel.

This looks all perfectly working to me.

Did you perhaps change other options?
Or does your NEF file contain XMP data (Nikon software has embedded XMP data directly in the NEF, which takes precedence over sidecar files by default).

Select your NEF file and then open the ExifTool Command Processor: <F9>,<E>
Select the "List Metadata" preset at the top and then pres <F9>.

Does the ECP show XMP data in the output?
In that case you have XMP data both on the NEF file and the sidecar file, which is always problematic.

[philippe28]

I've attached the output of ECP. I don't see any xmp data in it.
Yes, once the xmp file is written the process runs as expected. I've added Country and write back is fine.
... I think I understand my mistake... If I delete the xmp, the next write back doesn't create it again. But if I enter State and write back again, IMatch creates back the xmp with the new State but City and Country have gone.
So I think I've deleted the xmp files at some point of the time, and this lead to that confusion.
Is there a way to make IMtach write again the xmp with database metadata and not to lose them ? 

[Mario]

If I delete the xmp, the next write back doesn't create it again.

That would explain the first entry in your log file: "Writeback - Nothing to do".
If you delete the XMP file and than write-back again (e.g. after changing data that does not affect XMP) IMatch will not create another XMP file, because it does not write XMP data. But that would be a very special case.

I don't see why IMatch would lose any data during write-back. If there is pending data in the database, it will write that data. But if you have deleted the XMP files manually after IMatch has written them, there will be no pending XMP data in the database and thus nothing to write.

You have intentionally tricked out the system by manually deleting the XMP data from disk. Bad mistake.
Naturally, when IMatch reimports the files later for some reason, the XMP data in the database will vanish. IMatch reads the files, finds no XMP data and loads that into the database. Even the option "protect unwritten data" will not help because due to your peculiar action there is no unwritten data in the database  - and no XMP data anymore in your files.

I suggest you select a bunch of files for which you have deleted the XMP files in a file window.
Then mark all relevant XMP data in the Metadata Panel as modified (click the pen icon in front ot the field).
This tells IMatch to consider these tags as "pending" and when you then write back it will write out these tags, plus the surrounding XMP header and footer data.

[philippe28]

I understand my mistake and your suggestion works perfectly.
Thank you for your kind support.

[Mario]

Great. In the future: Don't work against the system or the flow. It's always best to keep things simple and let IMatch/ExifTool take care for the metadata management.