Difficulty identifying all file without Dates

[Tveloso]

I'm not sure if this is a bug, but I'm finding that in some cases, files that I know do not contain any usable Dates, are appearing with values in Date Digitized (XMP::xmp\CreateDate\CreateDate\0), and Date Created (XMP::photoshop\DateCreated\DateCreated\0), in the Metadata Panel's Default Layout.

I know that IMatch must assign Dates to files, and that it must fall back to the FileSystem Dates when there are no usable Dates in the appropriate Tags in the Metadata.  But when this happened in the past, it seems the "defaulted-to dates" didn't actually appear in the MetaData Panel.

Quite some time ago, per Mario's good advice, I set up Data Driven Categories on both Date Digitized and Date Created, and the "Other" Category within those, showed the files that actually lack Dates.  These two Data Driven Categories essentially re-create the TimeLine, but also identify and isolate, the files having no Dates (and that are appearing in an "incorrect place" in the TimeLine, due to IMatch having had to use the FileSystem Dates for them).

But now, files that I believe were previously in that "Other" Category, are no longer appearing there, and appear in the same "Date Category" as they do the TimeLine.

For example, the Metadata Panel, for a file from the "Other" Category in my Data Driven Category on Date Created (which ECP confirms "contains" only the FileSystem Dates...i.e. it actually has no Dates):

Code Select Expand


[System]            - File Modification Date/Time     : 2018:12:09 09:58:31-05:00
[System]            - File Access Date/Time           : 2019:01:21 21:04:48-05:00
[System]            - File Creation Date/Time         : 2018:12:09 09:58:31-05:00


...looks like this (i.e the Metadata Panel agrees that the file contains no Dates):



...(and that file is appearing in the TimeLine Node for December 9, 2018, as expected - which is not correct, but the only alternative currently, in the absence of Dates in the File's Metadata).

But another file that also lacks dates:

Code Select Expand


[System]            - File Modification Date/Time     : 2020:03:02 23:59:21-05:00
[System]            - File Access Date/Time           : 2020:03:02 23:59:21-05:00
[System]            - File Creation Date/Time         : 2020:03:02 23:59:21-05:00


...has these values in the Metadata Panel (i.e the Metadata Panel indicates that the file contains Dates, when it does not):



This is not a new "issue" in IMatch2020...and I'm not sure if this is even an issue at all (or just what happens when a file of this type is updated, and then re-loaded in IMatch)...because I have other files that don't appear in the "no Dates" Category (but do lack dates, as in example 2 above), and show dates in 2019, 2018, and 2017.

So my questions, are: is it correct that some files without dates, are showing values in Date Digitized and Date Created?...(while most don't - and the Metadata Panel agrees with what they actually contain)...and how could I identify all such files?

I tried filtering for the absence of other standard metadata items (such as "Model") using a value filter, but I'm not sure if that's capturing all of my "problem files".

[Mario]

These fields are filled by ExifTool during import.
Or by the user by hand or a Metadata Template or...

IMatch uses the rules explained in How IMatch uses Date and Time Information only to fill the internal global File.DateTime 'value', it does not fill XMP tags when they have no value.

Maybe ExifTool found something usable in your files?
Maybe these tags where updated by you or a MD template?
What does the Metadata Analyst app tell you about errors and issues found in these files?

[Tveloso]

Mario, the Metadata Analyst shows this for the first example file (the one where the Metadata Panel agrees with the File's contents - no Dates):

   

...and this for the second (the one showing Dates in the Metadata Panel - though it too currently contains no Dates):

   

For file-2, I added some faces, but did not write back yet.

I think I have probably just talked myself into thinking that under those conditions (some change to the Metadata of a file with no Dates), that the file would still not contain Dates, if the user didn't add them (only the newly added Metadata would be there).  In fact, as I write this, the more I think that I used to know this isn't the case...(IMatch must include Dates when it creates the File's XMP Data), and I just convinced myself that Dates would remain "empty" with changes to other Metadata.

I have been very slow in dealing with my files that lack dates...(in truth, I've hardly worked on that group of files at all - mostly because for many, I know I'm going to have a hard time determining even an approximate date).  That's a whole different subject that I've seen discussed here several times in the past - how to deal with "fuzzy dates".

And now, with all the excitement of IMatch2020, it will be even more difficult to turn to that mundane task.

I'll just create a Category for those files to work on them over time...(and will add to that category, as I encounter files like #2 above - the used to not have dates, and now have incorrect ones, due to my previous updates).

Thank you so much for your support.

[Mario]

See How IMatch uses Date and Time Information
If there is no EXIF date and time information. XMP created and subject created will be empty, unless you fill them in the MD panel.

[Tveloso]

Mario, that's true at the initial ingest of a file lacking those dates.

I was thinking that IMatch used to "preserve the absence" of those Dates, unless the user explicitly populated them, but now it seems that they are being populated as a byproduct of other updates.

For example, for my file #1 (which lacked those dates), I added a face annotation, and did a Write-back...and the previously empty MD Panel now contained Keywords (by virtue of the Keywords entered on the Person record), and also Dates.

To be clear, after adding the Annotation, the MD Panel contained only the Keywords, and the Dates were still blank.  But after I did a Write-back, these MD Panel Dates:

• Date Digitized
• Date Created
• Date Modified

...now contained today's date (where they were blank before), and the file now contained:

Code Select Expand


[System]            - File Modification Date/Time     : 2020:03:29 17:20:30-04:00
[System]            - File Access Date/Time           : 2019:01:21 21:04:48-05:00
[System]            - File Creation Date/Time         : 2019:01:21 21:04:48-05:00
[XMP-xmp]           - Metadata Date                   : 2020:03:29 17:12:35-04:00
[XMP-xmp]           - Modify Date                     : 2020:03:29 17:12:35-04:00


This caused this file to jump from the TimeLine Node for December 9, 2018 (where it was before) to the TimeLine Node for today.  And that file was now also no longer in the my Date DD Category "Other", but in the branch of its 2020 sibling category for today's date.

It seems that it's been behaving that way for quite a few versions (populating XMP created and subject created, as a byproduct of other updates), as I have files with dates in the past several years, that I believe originally did not have those dates populated).

[Mario]

That's perfectly normal.
When you change a file in any application, the file system write time will change. When you access it, the access time will change.
The XMP metadata and modify dates must change when metadata is modified.

But that does not impact XMP::xmp\CreateDate\CreateDate or XMP::photoshop\DateCreated\DateCreated which are still not filled.
IMatch this will fall back to the "modifed" date reported by the file system for the File.DateTime global.
If you use the Metadata Panel to set a date and time, IMatch will use that instead.

[jch2103]

That's perfectly normal.
When you change a file in any application, the file system write time will change. When you access it, the access time will change.
The XMP metadata and modify dates must change when metadata is modified.

But that does not impact XMP::xmp\CreateDate\CreateDate or XMP::photoshop\DateCreated\DateCreated which are still not filled.
IMatch this will fall back to the "modifed" date reported by the file system for the File.DateTime global.
If you use the Metadata Panel to set a date and time, IMatch will use that instead.

I'm seeing the same behavior for the few images for which I still don't have Original and Digitized dates: Both these metadata fields are empty as is Modified date in the metadata panel. If I run face recognition on one of these images, the metadata fields remain blank until I click the write metadata pen, after which Original, Digitized and Modified date/times are all changes to the moment when the metadata were written. Exactly the same thing happens when I make and save any other metadata changes.

However, I found some older images w/o Original and Digitized date/times (but with a Modified date/time) that I had processed with IMatch 5.6.0.12 in 2016. That suggests that the behavior for IMatch we're discussing has changed since 2016.

The cure of course is to assign Original and Digitized date/times to all images, even if those dates are approximate. Once those tags are filled in by the user, they don't change because of other metadata changes. (@Tvelsoso: I assume these are mostly scanned image? That's where most of mine w/o these tags came from, anyway. I used to have a few thousand of these, but I've been able to reduce it to just a few now.)

[Tveloso]

I'm seeing the same behavior for the few images for which I still don't have Original and Digitized dates: Both these metadata fields are empty as is Modified date in the metadata panel. If I run face recognition on one of these images, the metadata fields remain blank until I click the write metadata pen, after which Original, Digitized and Modified date/times are all changes to the moment when the metadata were written. Exactly the same thing happens when I make and save any other metadata changes.

Thank you for confirming John.

The cure of course is to assign Original and Digitized date/times to all images, even if those dates are approximate. Once those tags are filled in by the user, they don't change because of other metadata changes. (@Tvelsoso: I assume these are mostly scanned image? That's where most of mine w/o these tags came from, anyway. I used to have a few thousand of these, but I've been able to reduce it to just a few now.)

Yes, these are mostly scans.  But there are also a few images originally taken with a Digital Camera, that I recognize as having been edited by me (years ago), and it appears that whatever I used to edit them, discarded all metadata.  I still have over 1,800 of those images all together (probably a few more that over time have been "assigned dates").  I have assigned them to a Missing Invalid and Incorrect Dates Category so they'll now "stay put", and I can work on them over time.

That's perfectly normal.
When you change a file in any application, the file system write time will change. When you access it, the access time will change.
The XMP metadata and modify dates must change when metadata is modified.

So the only "issue" is that the MD Panel is now displaying dates, once these files have been updated, leading us to believe that those dates have actually been assigned to them.  So we have:

Date #1
MD Panel Label = Date Digitized
IMatch Tag Name = Create Date
IMatch Tag ID = XMP::xmp\CreateDate\CreateDate\0
ExifTool Tag = [XMP-exif] - Date/Time Original

This date has a value in the MD Panel following a Write-Back, but is not actually stored in the file

Date #2
MD Panel Label = Date Created
IMatch Tag Name = Date Subject Created
IMatch Tag ID = XMP::photoshop\DateCreated\DateCreated\0
ExifTool Tag = [XMP-photoshop] - Date Created

This date has a value in the MD Panel following a Write-Back, but is not actually stored in the file

Date #3
MD Panel Label = Date Modified
IMatch Tag Name = Modified Date
IMatch Tag ID = XMP::xmp\ModifyDate\ModifyDate\0
ExifTool Tag = not sure of it's [XMP-xmp] - Metadata Date or [XMP-xmp] - Modify Date

This date has a value in the MD Panel following a Write-Back, and is also stored in the file

So this is not a very significant issue (if it's an issue at all)...these files otherwise behave as expected.  Since they lack "actual metadata dates", each time they are updated, they get the current date in File.DateTime...and the only issue is that this date is also being shown in XMP Created and Subject Created in the Metadata Panel (when they used to be blank).

[Mario]

The XMP dates are filled by ExifTool during ingesting the file (from EXIF data or one of the other tags explained here: How IMatch uses Date and Time Information)

If your files have no usable timestamps (no metadata or no EXIF), they will be empty. IMatch does not make these up.
When you change metadata and write-back, ExifTool may be able to use one of the newly created time stamps in the file or falls back to the last modified date and time.

Pro Tip: If your files have no usable date and time, fill the two date and time values in the MD Panel yourself. By setting a date or time or by conjuring it up with a Metadata Template or the Metadata Wizard from a usable date and time source. This way you control the date and time used.