Icon to show where actual metadata is stored

[cytochrome]

Metadata shown in the Metadata Panel can originate from image (raw, jpg, etc), from the xmp sidecar or (I suppose) from the Database.

When I look atan image I don't know where the info shown in  the panel comes from. This is annoying. In the attached exemple the metadata was fully written (author, etc). Then I changed the Location field. After updating, evrything but the new Loation was gone.

I understand how this happened, Mario gave me a special crash course in metadata handling in IM5 (https://www.photools.com/community/index.php?topic=596.0) but still it would be handy to have a small icon or colored letter like Raw, Xmp, etc, for example beside the icon for Metadata template and Thesaurus (where my white mouse marker is located on the image).

Maybe I am not the only one who sometimes shifts from sidecar to raw or reverse, it would be of help, and anyway much faster than opening ExifToolGUI and have a look

Francis

[attachment deleted by admin]

[Ferdinand]

Francis - I realise that you're trying to do some reverse propagation, but most of the time this isn't an issue.  I have my IMatch configured so that RAW files have XMP in sidecars and so-called standard formats have it embedded.   So I know where the metadata is being read from and written to.  If you know and understand your Meatadata2 preferences then this isn't an issue.  Switching back and forth is not that common.

[cytochrome]

Hello Ferdinand,

Yes I am in the process of updating all my RW2 with metadata. By the way this is no longer a problem, works fine either with an ExifTool command-line, or from within IMatch with the EPC.

Nevertheless, when one has a mixed environment (sidecar + in-file metadata) and one opens a new folder it is not immediately clear where the metadata displayed in the panel is copied from. One has to do a little thinking: are there sidecars in this folder? for all raw files? how is my metadata2 set at this moment?

So a small icon would be informative and save my olsneurons a little choline-esterase storm...

Francis

[sinus]

Since I am also in the progress to deal with Metadata, the idea from Francis is very good. Would finally help me also.

[JohnZeman]

So a small icon would be informative and save my olsneurons a little choline-esterase storm...

Oh my.  Don't you just hate it when that happens?

[cytochrome]

Of course I hate it, synapses en folie at my grand age!! Seriously, a little icon would help.

I fiddled so much with settings when experimenting with the metadata write back to Panasonic rw2 that in some folders I have a mix up of metadata still in xmp, already in rw2, only in the JPG, perhaps in the IMatch limbs (this is worst).

It is annoying when you spend lots of time cataloging photos and you don't even know exactly where the data is kept. The only sure way I found is to open Exiftoolgui at the same time to see what is written where.

Francis

[Ferdinand]

There are two separate but related concepts here - (i) where the file has metadata stored, be it in the file or the sidecar and whether it's IPTC or XMP, and (ii) where IMatch is reading it from.  It sounds to me like you want an icon for the first definition, whereas what you're likely to get is an icon for the second.  Knowing where it's reading it from won't tell you everything you need to know about what is stored where.

[Mario]

EXIF/IPTC are always read from and stored to the image file.

XMP can be read from

1. The image file
2. The XMP sidecar file
3. Both

XMP can be written to

1. The image file
2. The sidecar file

All this is controlled with the Metadata 2 options.

I see little value for the majority of users in yet another icon or other 'information item' which tells them where their metadata is saved to or read from. Most users don't have that many variations, really. For JPEG, TIFF, PSD, DNG files, EXIF/IPTC/XMP is embedded. For RAW files it is in sidecar files. Period. That's how Adobe does it, which means that 100% of the professional market does it that way. Most third party RAW software follows that example.

If a user finds himself in the situation where part of his RAW files have embedded XMP data, and other RAW files have not, or, even worse, XMP sidecar files have been used for file formats which according to the XMP standard have embedded XMP data, my advice is to clean-up that situation before adding his files to IMatch 5. I did that recently, cleaned up my own metadata mess (I used Nikon software in the past, and ended up with embedded XMP data). Moved it all into XMP sidecar files and ditched Capture entirely.

ExifTool makes it easy to either import XMP into the RAW, or extract XMP from the RAW and create XMP sidecar files. There are many examples in the ExifTool help and on the ExifTool user forum. This process needs to be done only once, and then all files use a single approach. Then add your files to IMatch 5.

This not only voids the need for features which show from where the XMP data was read and where it will be stored, but also gives you a more solid basis and stable workflow for the future.

[cytochrome]

Ite missa est... Amen.

Ferdinand made a pertinent analysis of the situation and its true it is complicated.

Mario I see I am annoying you with my in-raw metadata writing. And I agree that 99.9% of users go the Adobe sidecar way, in most cases even without knowing it, which is perfect.

Francis

[Mario]

Mario I see I am annoying you with my in-raw metadata writing.

Not at all.

And I agree that 99.9% of users go the Adobe sidecar way, in most cases even without knowing it, which is perfect.

I have to estimate for every feature request how many users will benefit from it. And to weight that against the development and documentation efforts involved.

I perfectly understand that there are users with a "mixed tools and concepts" history, which have very specific problems with their metadata. I think that these users go the extra mile and clean up their data before adding their files to IMatch 5. This is usually no big deal thanks to ExifTool. Now is the ideal time.

I just don't see that adding features and logic to indicate where metadata came from will be helpful. Because it will end like this:

EXIF, IPTC: 100% of all cases this data is stored in the image
XMP: 95% of all users: Adobe/MWG style. Which means embedded XMP in the formats suggested by MWG, sidecars for the rest (including RAW files).

Many users don't even think about this at all. Or care for where their metadata is stored. They use the default settings in LR/PS/Bridge and IMatch, which are all MWG-compliant.

[sinus]

EXIF/IPTC are always read from and stored to the image file.

XMP can be read from

1. The image file
2. The XMP sidecar file
3. Both

XMP can be written to

1. The image file
2. The sidecar file

All this is controlled with the Metadata 2 options.

Sorry for my lack of quick understanding. A big difference is mostly, is the file a "standard"-file like jpg, dng, tif and so on OR is it a RAW-file, like NEF or other RAW-images, what cameras produces.

Specialy I am puzzled, because if I read here some threads, I think, XMP can be stored only in sidecars, but then I read something like XMP can also be embedded in RAW?
Hence my question, what can im5 write in a RAW?

I am really sorry, but I am puzzled a lot about this stuff.
Finally I want only to know FOR SURE, where can IM5 store IPTC and where XMP?!

So, the "table" from Mario I understand.
Is my additon in red also true??

XMP can be read from
1. The image file
2. The XMP sidecar file
3. Both

XMP can be written to
1. The image file (RAW and Standard-files)
2. The sidecar file

Thanks a lot!

[Mario]

IMatch can embed XMP in RAW files if you enable that. As long as ExifTool supports the file.
It's just off by default and under normal conditions you should always use XMP in sidecar files for RAW files.

Nikon made a big mistake, for whatever reason, when they started embedded XMP in NEF and NRW files. And to support this is the only reason why I added that as an experimental feature in IMatch 3 - reluctantly.

Unless you are forced so use embedded XMP data in RAW files you should not do it. The MWG is pretty clear about that as well.

XMP shall be embedded in standard formats like JPG, PSD, DNG, TIFF and in sidecar files for all other formats.

[Ferdinand]

IMatch can embed XMP in RAW files if you enable that. As long as ExifTool supports the file.
It's just off by default and under normal conditions you should always use XMP in sidecar files for RAW files.

I thought that the default option for XMP write mode is "Use existing sidecars else use mode 1", which would mean that IMatch will only use a sidecar if it is already there.  My opinion FWIW, is that the default option should be mode 5 - "Always write to xmp sidecar files".

To answer Markus' other question, IPTC can only be in the image file.  There is no provision to store it anywhere else.

If a user finds himself in the situation where part of his RAW files have embedded XMP data, and other RAW files have not, or, even worse, XMP sidecar files have been used for file formats which according to the XMP standard have embedded XMP data, my advice is to clean-up that situation before adding his files to IMatch 5. I did that recently, cleaned up my own metadata mess (I used Nikon software in the past, and ended up with embedded XMP data). Moved it all into XMP sidecar files and ditched Capture entirely.

I agree strongly with this sentiment.  It's part of the reason why I wrote that cats to keywords migration script which runs in V3.6.  It doesn't migrate XMP from one location to the other, but perhaps I could add that in for those who don't want to have to deal with ExifTool directly themselves.

[sinus]

IMatch can embed XMP in RAW files if you enable that. As long as ExifTool supports the file.
It's just off by default and under normal conditions you should always use XMP in sidecar files for RAW files.

Nikon made a big mistake, for whatever reason, when they started embedded XMP in NEF and NRW files. And to support this is the only reason why I added that as an experimental feature in IMatch 3 - reluctantly.

Unless you are forced so use embedded XMP data in RAW files you should not do it. The MWG is pretty clear about that as well.

XMP shall be embedded in standard formats like JPG, PSD, DNG, TIFF and in sidecar files for all other formats.

Thanks a lot, Mario, now this is very clear for me. I will go the sidecar-route.

[sinus]

IMatch can embed XMP in RAW files if you enable that. As long as ExifTool supports the file.
It's just off by default and under normal conditions you should always use XMP in sidecar files for RAW files.

I thought that the default option for XMP write mode is "Use existing sidecars else use mode 1", which would mean that IMatch will only use a sidecar if it is already there.  My opinion FWIW, is that the default option should be mode 5 - "Always write to xmp sidecar files".

To answer Markus' other question, IPTC can only be in the image file.  There is no provision to store it anywhere else.

So, Ferdinand, this is now also clear for me, hence I will go further with xmp-sidecars. What I will do with all my Nefs from 3.6 (without Sidecars), I have to think about, but I am not worried about this. Thanks, also your other threads about metadata are very helpful!

[Mario]

I thought that the default option for XMP write mode is "Use existing sidecars else use mode 1", which would mean that IMatch will only use a sidecar if it is already there.  My opinion FWIW, is that the default option should be mode 5 - "Always write to xmp sidecar files".

You are right, of course. I had changed that for the 106 build but then did not publish it because it was not all done. The help topic has also been updated.

The defaults (MWG mode) are

Always write to XMP sidecar files
in combination with
Embed XMP in standard formats.

This gives you embedded XMP in JPEG, PSD, TIFF, DNG etc. and sidecar files for everything else.

[Mario]

What I will do with all my Nefs from 3.6 (without Sidecars), I have to think about, but I am not worried about this.

You'll need embedded XMP if you work with Nikon software and Nikon continues to support only embedded XMP data (I no longer use Capture so I don't keep up with whatever nik/Nikon are doing).

Or you convert the NEF files by extracting the XMP into sidecar files and deleting the embedded XMP data afterwards with ExifTool.

[Ferdinand]

So, Ferdinand, this is now also clear for me, hence I will go further with xmp-sidecars.

I think that's a good idea, but be sure to read my replies to you in the other thread about LR settings and how to avoid having LR and IMatch making conflicting XMP edits.

What I will do with all my Nefs from 3.6 (without Sidecars), I have to think about, but I am not worried about this.

There are solutions to this which can be described.   It's not urgent, but I suggest that you deal with this before you move your production DB to the public release of V5, when it's available for sale.

[Mario]

when it's available for sale.

Ooh, that will be heaven... 

[sinus]

The defaults (MWG mode) are

Always write to XMP sidecar files
in combination with
Embed XMP in standard formats.

This gives you embedded XMP in JPEG, PSD, TIFF, DNG etc. and sidecar files for everything else.

Great. I think, I will use exactly this default.

[sinus]

You'll need embedded XMP if you work with Nikon software and Nikon continues to support only embedded XMP data (I no longer use Capture so I don't keep up with whatever nik/Nikon are doing).

Or you convert the NEF files by extracting the XMP into sidecar files and deleting the embedded XMP data afterwards with ExifTool.

Since I use NO Nikon software (only LR, CS6 and IMatch), I will have to go your second suggestion, what is definitively ok for me.
Thanks for your answer!

[sinus]

So, Ferdinand, this is now also clear for me, hence I will go further with xmp-sidecars.

I think that's a good idea, but be sure to read my replies to you in the other thread about LR settings and how to avoid having LR and IMatch making conflicting XMP edits.

What I will do with all my Nefs from 3.6 (without Sidecars), I have to think about, but I am not worried about this.

There are solutions to this which can be described.   It's not urgent, but I suggest that you deal with this before you move your production DB to the public release of V5, when it's available for sale.

Thanks, Ferdinand, I will do so, like you writes and of course I will read your other comments. Well, in fact, I have to read it VERY carefully, because English is not that easy for me, specialy if it goes into (important) details.

To be honest, I will work further with IM 3.6 as long as I am not sure, how to deal with IM5. Because I guess, I will work again a long time with IM5, hence I will be sure to make nothing stupid. And to avoid this, I will long enough "play" with IM5 and take the old images from IM3.6 into IM5, when I am ready for it.