Questions about date write-back

Started by janb83, December 17, 2023, 04:01:18 PM

Previous topic - Next topic

janb83

When files without any Exif/XMP tags etc. are indexed, it seems the following four tags are changed and marked for write-back:

XMP::exif\DateTimeOriginal
XMP::photoshop\DateCreated
XMP::xmp\CreateDate
XMP::xmp\ModifyDate

Testing shows that later modifying XMP::photoshop\DateCreated (which is "Date Subject Created" aka File.DateTime internally) also modifies XMP::exif\DateTimeOriginal. I guess this is for compatibility reasons. Editing XMP::xmp\CreateDate does not automatically change any other tags.

A couple of questions:

1) If XMP::exif\DateTimeOriginal is overwritten when changing XMP::photoshop\DateCreated (aka "Date Subject Created"), why is XMP::exif\DateTimeOriginal not a read-only tag by default?

2) Why is XMP::xmp\ModifyDate marked for write-back after indexing, but not when changing some metadata like face annotations?

3) Most importantly: Are the fields (except ModifyDate) EVER changed by IMatch again after the initial indexing, unless I manually change them?
My reason for asking this third question is that I need to make sure I permanently preserve the original creation/modification date (whichever is older) in case I ever need to retrace who scanned an image and how. Yes, ideally this information would be contained in the metadata already. Unfortunately, that's just how it is with thousands of scans done by other people up to 30 years ago. With a preserved original modification/creation date, I at least have a rough idea of who might have done the scan (because e.g. I know my dad did a lot of them in 2000).
I might just copy this information to some custom unused tag anyways though, just be to safe :(

Mario

Many tags are linked to tags in other metadata formats or XMP schemata. I've explained this a number of times in this community and I don't want to explain it again. It's usually not relevant for users since IMatch and ExifTool take care for this automatically.

For example, XMP::photoshop\DateCreated\DateCreated is linked to XMP::exif\DateTimeOriginal\DateTimeOriginal and vice-versa. XMP::photoshop\City\City is linked to Composite\MWG-City\City and XMP::iptcExt\LocationShownCity\LocationShownCity and so on.

I've explained why DateSubjectCreated and CreateDate are added/set during ingest and how here: How IMatch uses Date and Time Information

Quote1) If XMP::exif\DateTimeOriginal is overwritten when changing XMP::photoshop\DateCreated (aka "Date Subject Created"), why is XMP::exif\DateTimeOriginal not a read-only tag by default?
Date subject created is the important timestamp. This is why IMatch offers it and "Create Date" in the Metadata Panel. The other tag is not shown anywhere by default, except in the browser mode I believe. And when you change it, it will change the linked tag too. Links word bidirectional. 
I recommend to just stick to the two date and time tags shown in the "Default" layout in the Metadata Panel and you're good. Things will just work. 

Don't directly manipulate other time stamps unless you have read the specifications for XMP and IPTC and EXIF and also the EXIFTool docs. I have been bitten many times by metadata mess myself.
You're a new IMatch user based on your post count. Don't try to do too much to quickly. IMatch cannot prevent you from shooting yourself into your own foot when modifying metadata the wrong way.


Quote2) Why is XMP::xmp\ModifyDate marked for write-back after indexing, but not when changing some metadata like face annotations?

This tag is always automatically set when ExifTool writes the file. IMatch must not manually set it every time.

QuoteMost importantly: Are the fields (except ModifyDate) EVER changed by IMatch again after the initial indexing, unless I manually change them?
My reason for asking this third question is that I need to make sure I permanently preserve the original creation/modification date (whichever is older) in case I ever need to retrace who scanned an image and how.

That's never a good idea. Even opening files containing XMP data or XMP in sidecar files in other applications may change time stamps, depending on how good the other application is implemented, which Adobe XMP toolkit it uses (there were many issues over the years) etc.

Important data like this should go only into XMP tags designed to take and retain user-provided data.
Not into timestamps that are somewhat considered volatile.

Do yourself a favor and select an XMP tag like description or title or headline or one of the many other tags reserved for user-specified data and copy the time stamps you want to keep into it for safekeeping.

In IMatch this can be easily done via a Metadata Template.

Run the template once to copy the relevant data, then write-back the files.
The data is now persistent and changes regarding how XMP deals with timestamps, changes in IMatch or ExifTool, bugs, other applications that fiddle with timestamps cannot affect it anymore. Don't forget to make and retain backups of your files.

You can use variables to produce the content of the tag, maybe even producing text that lists the values of various time stamps, including file system timestamps. The result will also be viewable the Metadata Panel.
-- Mario
IMatch Developer
Forum Administrator
http://www.photools.com  -  Contact & Support - Follow me on 𝕏 - Like photools.com on Facebook

janb83

Thanks again for your answers Mario. I do really appreciate you taking the time, but can I make one polite request please? I wrote specific questions about specific points because I researched and tested stuff first. Otherwise, I would not be able to ask these questions. I even mentioned that I did specific tests. In your and my interest, please don't start at zero by linking basic FAQs if my questions are already more advanced. I get where you are coming from because you have plenty of users who don't look at or don't understand the documentation and you must start at zero for them. But I don't want to waste your time by having you look up links etc. to stuff I already checked. I thought it was clear from my questioning that I did my research first.

As an example: I know fields are linked, actually tested this and mentioned this. I did not ask about this general mechanic at all, yet you felt the need to say you "don't want to explain it again". I never asked you too, I asked specific detailed questions not covered in the help and forum explicitly (btw, another thing not mentioned on the date documentation: IPTC dates are also considered before file creation/modification dates are used as a last fallback for File.DateTime).

Anyways, on to the specifics:

Quote from: Mario on December 17, 2023, 04:59:43 PMFor example, XMP::photoshop\DateCreated\DateCreated is linked to XMP::exif\DateTimeOriginal\DateTimeOriginal and vice-versa. XMP::photoshop\City\City is linked to Composite\MWG-City\City and XMP::iptcExt\LocationShownCity\LocationShownCity and so on.

[...]

And when you change it, it will change the linked tag too. Links word bidirectional.


This is what I assumed, but it's not true, I tested it. Changing XMP::exif\DateTimeOriginal\DateTimeOriginal does NOT change XMP::photoshop\DateCreated\DateCreated. It's only the other way around.

QuoteDon't directly manipulate other time stamps unless you have read the specifications for XMP and IPTC and EXIF and also the EXIFTool docs. I have been bitten many times by metadata mess myself.
You're a new IMatch user based on your post count. Don't try to do too much to quickly. IMatch cannot prevent you from shooting yourself into your own foot when modifying metadata the wrong way.

I actually have read the specifications, at least the parts about dates. I'm not trying to do too much, I am doing the absolute minimum necessary to ensure I don't cause permanent damage to files and/or irretrievably lose metadata. That is precisely why I am making sure I fully understand how IMatch works in this regard and asking these questions to fill knowledge gaps not covered by the docs.

Just because I have a low post count does not mean I am an inexperienced user. I have been a PC power user for over 30 years and have used countless applications of similar complexity, even working on some. I tried writing well-articulated, clear, and precise questions. Sorry if you got a different impression.


Quote
Quote2) Why is XMP::xmp\ModifyDate marked for write-back after indexing, but not when changing some metadata like face annotations?

This tag is always automatically set when ExifTool writes the file. IMatch must not manually set it every time.

I know that the modification date is always set, this can be easily tested. That is precisely why I asked the question. I don't understand why IMatch does not treat both cases identically. Why does it tell me about a write-back in one case but not the other, if in fact in both cases it will be written? As a developer, I can guess that for some reason you explicitly set it internally in one scenario, but not in the other where it works implicitly. From a user perspective, this distinction does not make much sense.

Quote
QuoteMost importantly: Are the fields (except ModifyDate) EVER changed by IMatch again after the initial indexing, unless I manually change them?
My reason for asking this third question is that I need to make sure I permanently preserve the original creation/modification date (whichever is older) in case I ever need to retrace who scanned an image and how.

That's never a good idea. Even opening files containing XMP data or XMP in sidecar files in other applications may change time stamps, depending on how good the other application is implemented, which Adobe XMP toolkit it uses (there were many issues over the years) etc.

Important data like this should go only into XMP tags designed to take and retain user-provided data.
Not into timestamps that are somewhat considered volatile.

Do yourself a favor and select an XMP tag like description or title or headline or one of the many other tags reserved for user-specified data and copy the time stamps you want to keep into it for safekeeping.

In IMatch this can be easily done via a Metadata Template.

Run the template once to copy the relevant data, then write-back the files.
The data is now persistent and changes regarding how XMP deals with timestamps, changes in IMatch or ExifTool, bugs, other applications that fiddle with timestamps cannot affect it anymore. Don't forget to make and retain backups of your files.

You can use variables to produce the content of the tag, maybe even producing text that lists the values of various time stamps, including file system timestamps. The result will also be viewable the Metadata Panel.

Thanks. I am aware of the risk of manipulation, which is exactly why I asked if IMatch(!) ever changes these fields unless done manually. I have backup and sync procedures in place which will tell me if any other application would change a file, that's not an issue for me. But I need to be sure that IMatch does not "accidentally" also modify these fields while I work on other metadata. Because in that specific case, my syncing will not be able to tell the difference, it just knows a change is OK because I worked on the file in IMatch.

You are likely correct in saying that textual fields are less likely to get changed by other applications, but in reality, ANY metadata can be violated by any application. But, I already wrote that I will likely copy the date so some other fields additionally to be safe, an idea which your explanation supports. I had already set up a Metadata Template for this. I still would like to know if I can assume that the dates (XMP::photoshop\DateCreated etc.) will never be changed by IMatch other than through explicit user actions (manually setting it, setting via a Template etc.) or a potential bug. If it's NOT like that and you DO change these dates, I would like to know this so I don't rely on it for other purposes like data driven categories etc.