Date/Time Change Issues

[Whaler]

A new user and first posting. Let me start by saying IMatch is even better than I had anticipated, I truly appreciate the effort that's gone into creating such a rich and powerful program.
My biggest frustration has been in the discovery of the errors either I or other software has made of my metadata. I've been impressed with how much of it I have been able to repair in such a short period of time considering I have 60K images. Thanks to the forum and help files I have managed to navigate my way around the various controls available.
One of my discoveries is that the timestamps on about 1000 raw files are not consistent throughout the metadata. The XMP-exif Date/Time Digitized is correct, but the XMP-exif Date/Time Original and XMP-Photoshop Date Subject Created, are not correct. All three time and dates are correct for the corresponding JPGs.
Working on a few images I was able to use the Metadata Templates to insert the Date/Time Digitized into the XMP-Photoshop Date Subject Created. The date was replaced with the correct time (great) but the time was offset by an hour. I would appreciate any help in figuring out why the time would be offset by an hour before I venture further with more files and correcting  the XMP-exif Date/Time Origional.

[Mario]

When you click the pen icon in the Metadata Panel in front of the date/time tags you want to update and then write back, IMatch updates the XMP, Photoshop, legacy IPTC and EXIF timestamps to the same value. Did you try that?

The problem with manually updating timestamps is the time-zone. EXIF does not record time-zone information but XMP does. This can cause issues when mapping during write-back and re-mapping during import.

[Whaler]

Thanks Mario for your reply.
Clicking the pen icon works and updates all the information when I copy the correct time and paste it into the incorrect field. I have updated about 100 files this way, copy correct time (digitized) and paste it into the labels that are incorrect (Photoshop-created, Original, Created Date). I am only adjusting the XMP tags and when I do click the pen all the exif tags are updated to the correct time without the time-zone information,as you said. This is rather time consuming as I have to copy, and then paste 3 different tags for every file.
Initially I had hoped to propogate from the corresponding JPG files which all have the correct date and time but I could not find a way to limit it to only the affected tags. Propogating all XMP data creates more problems as it also affects anything related to file type. I then experimented with the Metadata Templates to fill the XMP Photoshop Created Date with the correct Digitized Date/Time and this is where I encountered the strange addition of an hour. I had assumed Metadata Templates would simply copy from Digitized and paste it into the Created tag but some additional step is taking place that I can't figure out (why add an hour?).

As I said earlier, I have been able to manually copy and paste into the required XMP tags and correct the errors, I was hoping that I could find a way to automate the process as I still have about 800 files with errors.

Again I much appreciate your efforts.

[Mario]

Meztadata Templates indeed just copy the tag values. There should be no time shift, unless your source values contain time-zone offsets?

Also, just in case. Did you use the correct formatting YYYY-MM-DD HH:MM:SS when using a variable as the source (this depends on the variable/tag used).
Because DateTime variables in IMatch return date and time data formatted based on the user settings (Windows date and time formats). This is not the ISO date and time format expected by ExifTool when writing tags.

For example: {File.MD.XMP::xmp\CreateDate\CreateDate|format:YYYY-MM-DD hh:mm:ss} ensures that the datetime format is correct for setting date and time tags.

See Variables for details.

[Whaler]

Mario, that was it. I had adjust for daylight savings time in my Windows time date and time formats. Disabling that ensured I got the correct time. Thanks, I should have figured that one out. Much appreciated.

[Whaler]

I've come back to this topic because adjusting for daylight savings only worked for a small group of the photos with incorrect time stamps.  For three years I tried to use Lightroom as my photo manager and all those photos, thousands of them, have various incorrect time stamps. The only consistently correct time stamp I have for those photos is the XMP-exif Date\Time Digitized. Of course, this means the time viewed in the thumbnail and timeline is incorrect.

The last couple of weeks I have read almost all the help files and scoured the forum looking for a way to correct the problem. I have conducted several tests on multiple files and have not been able to find a solution that works on more than one file at a time.

For the testing I used copies of the same photo, in each step I took I wrote the contents of the Metadata Panel to a text document, I ran the ExifTool-List File Dates and Times and copied that information into a text document, I than put all that information into Excel where I could sort and compare the different tags with each test I performed. I ran tests with MWG Compliance On and Off.

I won't bore you with all the details but just the ones that have raised some questions for me.

This is the relevant Original File Info as it appears when first added into IMatch and copied from the Metadata Panel:
File Name   2007-09-10_19-03-28e.JPG
Create Date   2007:09:10 19:03:28
Date/Time Original   2007:09:10 19:03:28
Offset Time   
Offset Time Original   
Offset Time Digitized   
Date Created   2007:09:11
Time Created   03:03:28Z
Date Subject Created   2007:09:11 03:03:28Z
Date/Time Digitized   2007:09:10 19:03:28-07:00
Date/Time Original   2007:09:10 19:03:28
Created Date   2007:09:10 19:03:28.0
The file has a pencil icon with List of tags to write: XMP::dc\Subject (as do all my photos) and the time on the thumbnail is 3:03 AM.

First question:
Why does the XMP-exif Date\Time Original and the XMP-xmp Created Date not display the UTC offset? When I view the metadata in ExifTool or other readers the original file does have the -07:00 appended but it almost never shows up in the IMatch Metadata panel even after writing to the file. Could it be there is some formatting at work in the Metadata Panel display, or it's mapped to a different tag like composite?
From ExifTool I get - [XMP:XMP-xmp]   CreateDate                      : 2007:09:10 19:03:28-07:00, and [XMP:XMP-exif]  DateTimeOriginal                : 2007:09:10 19:03:28-07:00.


At this point I write the XMP::dc\Subject to the file and when I do offsets have been written to the file and the tags EXIF Create Date plus XMP-exif Date/Time Original have been changed to 2007:09:11 03:03:28 AM which I'm sure came from the XMP-photoshop Date Subject Created tag. I get that IMatch may need to update the metadata upon the first writing, I was just surprised that it didn't come up in the tags to write or whether it affects outcomes later.

What did work for me but only on a single file at a time, is to Copy the XMP-exif Date/Time Digitized value and then Paste it into XMP-photoshop Date Subject Created. After I save the changes in the Metadata panel and write to the file I end up with the correct time in all relevant tags (again IMatch does not display the appended time offset even though it's there in the files relevant tag).

This is what the Metadata panel reads after this test:
File Name   2007-09-10_19-03-28e.JPG
Create Date   2007:09:10 19:03:28
Date/Time Original   2007:09:10 19:03:28
Offset Time   -07:00
Offset Time Original   -07:00
Offset Time Digitized   -07:00
GPS Date Stamp   2007:09:11
GPS Time Stamp   02:03:28
Date Created   2007:09:10
Time Created   19:03:28-07:00
GPS Timestamp   2007:09:11 02:03:28Z
Date Subject Created   2007:09:10 19:03:28-07:00
Date/Time Digitized   2007:09:10 19:03:28-07:00
Date/Time Original   2007:09:10 19:03:28
Created Date   2007:09:10 19:03:28.0
Thumbnail now reads the correct time as 7:03 PM

Metadata Templates indeed just copy the tag values. There should be no time shift, unless your source values contain time-zone offsets?"

Not sure I understand why I wouldn't have time zone offsets, without the time zone my computer would read 7 hours ahead of my local time. Windows writes the offset time, eg. - FileModifyDate:  2018:10:16 11:49:46-07:00

So I tried the following Metadata Template – {File.MD.XMP::photoshop\DateCreated\DateCreated\0} fill with {File.MD.XMP::exif\DateTimeDigitized\DateTimeDigitized\0} thinking it would be the same as a direct copy and paste into the relevant tags but it produces a different set of results.

File Name   2007-09-10_19-03-28c.JPG
Create Date   2007:09:10 19:03:28
Date/Time Original   2007:09:10 07:03:28
Offset Time   -07:00
Offset Time Original   
Offset Time Digitized   -07:00
GPS Date Stamp   2007:09:11
GPS Time Stamp   02:03:28
Date Created   2007:09:10
Time Created   07:03:28-07:00
GPS Timestamp   2007:09:11 02:03:28Z
Date Subject Created   2007-09-10 7:03:28 PM
Date/Time Digitized   2007:09:10 19:03:28-07:00
Date/Time Original   2007:09:10 07:03:28 (AM on Metadata Panel)
Created Date   2007:09:10 19:03:28.0
Thumbnail now reads 7:03 AM

Second question:
Why does this template not produce the same results as a direct copy and paste of the same tags?

It's definitely the Photoshop time that created the conflicts and I have tried deleting that tag which also deletes the Date/Time Origial tag. This clears up the errors but seems to be a rather dramatic move when replacing it with the correct time would be preferred.

Moving forward I was hoping to accomplish two items:
1-   Correct the time stamps in a batch process so that it produces the same results as copy and paste.
2-   Figure out a way to search for all files with unequal time stamps, either through a filter or dynamic category. Not sure if it's possible to compare two tags and determine if they are equal or not.

Any help or observations would be greatly appreciated.

[hluxem]

Hello,

I had something similar in the past and date & time tags are sensitive, they only work if the tag is formatted correctly. In my case I had to use the format function.
Looks like the timezone is not part of the string transferred, at least thats what it looks like when I copy the date time into a text tag with the metadata template.
As I always had the same time zone I could add it manually, like in below:
{File.MD.XMP::exif\DateTimeDigitized\DateTimeDigitized\0|format:YYYY:MM:DD hh:mm:ss-04:00}

I couldn't find a way to format the date time value including the time zone. I think normally zzz or k is used to add the time zone. Nothing like that is listed in the help for the formatting arguments (almost all the way to the bottom), so it may not be possible.

https://www.photools.com/help/imatch/#var_basics.htm


I think to find the files with bad time stamps you have to use a script, compare the time stamp for a selection and add them to a category or collection. I did something similar to compare the time stamp with the file name as my names are date and time based.


Heiner

[Whaler]

Thanks Heiner for your reply.
I remember reading the help file you inserted but may have missed the significance of it. While I love the user control of IMatch, it does become daunting trying to learn it all.

I have a great deal of trial and error to go through.

I used your idea of a template to copy the date time into a text tag, what I got the following string "2007-09-10 7:03:28 PM" which is interesting because if I copy (control C) and paste it into a text document I get "2007:09:10 19:03:28-07:00"

I tried using {File.MD.XMP::exif\DateTimeDigitized\DateTimeDigitized\0|format:YYYY:MM:DD hh:mm:ss-07:00} as my source to copy from but it didn't work, I will try using the formatting for the destination tag and see what happens.

For now scripts will have to wait, still a ways to go.

Before using IMatch I didn't even know I had any conflicts with my metadata 

Again thanks for your reply,

Rene

[hluxem]

Hello Rene,

There is indeed a lot to learn and a lot of trial and error with all the different date and time tags. Scripting is a whole new chapter for me and therefore very time consuming. I was able to use what I call "copy and paste" programming to get 2 scripts running. One is pretty closed to what you what in comparing time stamps and assigning files to a category of a file selection.

Without testing it too much, try this statement. That's more inline with what I'm using.

{File.MD.XMP::exif\DateTimeDigitized\DateTimeDigitized\0|format:YYYY:MM:DD hh:mm:ss}-07:00

Just in case you work with pictures in different time zones, I think if the time zone in the picture is different than the local timezone your computer is in, the time value written to the tag is shifted according to the local time for the computer. I'm at -4 right now, if the following is used as the source value 2016:10:08 17:18:19-07:00, the result is 2016:10:08 20:18:19-07:00 with above instruction. That's obviously not what you want, I think the only way around that is to change the time zone temporally when you work with a template.

Good luck,

Heiner

[Whaler]

Heiner, that worked.

Playing around with it a little more I realized I was making this more complicated than it should be. After 12 years of using the same DAM application I had come into IMatch with unconscious expectations and assumptions about the metadata, many of those were incorrect.

Thanks again,

Rene

[Mario]

Date and time are always somewhat challenging, especially when you have to deal with (and reconcile) legacy IPTC, legacy EXIF and XMP. And time zones (which are not supported by all standards).
That's a real mess.

I still wonder why camera vendors bring out space-age cameras with built-in smart phones, WiFi, NFC, Bluetooth, mirror-less etc. And still stick to EXIF metadata which has been defined 30 years ago...

Why don't they just throw out all the old junk and write one, clear XMP record when the image is generated?
This could hold proper date and time, GPS data, photographer info, rating if the camera supports this etc.
IMatch would just ingest that XMP record and would not need to care for mapping between legacy EXIF, XMP, GPS etc.

[Whaler]

Thanks Heiner and Mario,
I was able to consolidate all dates and times to perfectly matching metadata.

Why don't they just throw out all the old junk and write one, clear XMP record

This got me thinking and I used the ECP to remove old proprietary metadata, some of which contained time stamps. I then used the ECP to remove all of the incorrect time stamps which also cleaned up the composite tags.

Applying the Metadata Template to write the correct time {File.MD.XMP::exif\DateTimeDigitized\DateTimeDigitized\0|format:YYYY:MM:DD hh:mm:ss}-07:00 into the now empty XMP time/date relevant tags, corrected everything.
Much appreciate your help.
Rene