Seeking help with an approach to iPhone mov file dating

[dcb]

I'd like to know how others are dating iPhone/iPad movie (.mov) files. Before I get too far down a rabbit hole chasing my tail.

Background
We are household of multiple iOS devices, all of different ages. Until now I've been using the Dropbox Camera upload function to transfer photos and movies to my PC. The advantage of Dropbox is that it names each file with a datestamp so instead of IMG_1994 I will get a file named 2017-11-21 18.11.39.mov. This is always the correct time the movie was created. The disadvantages of Dropbox are that it is slow (my internet is not that great) and only downloads for a time as a background process then stops.

A new approach
I have been testing transfers of files from a direct connection of my iPhone and copying from the DCIM folders. All JPG files are perfectly timed. When iMatch reads the creation date it is the time the photo was taken. MOV files are different as there is an inconsistency (at least under iOS 11) with the way Apple treats the timestamps. It is very obvious when the photo is a "Live" photo. These are images where a JPG is produced as well as 3 seconds of video. The timestamps are different in one very important way (but it applies to all MOV files).

A "Live" photo taken at 11:30 this morning will produce a create date of:

• JPG - 2018:04:10 11:30 AM
• MOV - 2018:04:10 01:30 AM

The JPG is correct for UTC+10:00. The MOV is correct but does not specify the time offset. If I was just concerned with the date I could copy the data using a YYYY:MM:DD format and leave off the time. Not pretty but it works.

However, because I live on the east coast of Australia, our UTC time offset is either +10:00 or +11:00. If the same photo was taken at 8:45 AM I would have:

• JPG - 2018:04:10 08:45 AM
• MOV - 2018:04:09 22:45 PM

Different days!

Quicktime Dates
There are various dates stored within Quicktime. Only one of them is the right creation date to give me what I need. ExifTool showed it and this evening I was eventually able to find out which Quicktime tag to ingest using iMatch. Problem solved. I can copy the field using a metadata template from Quicktime to Created Date and I'm happy.

But...

Apple has changed Quicktime in such a way that the name of the date variable has changed over time. A December 2017 video uses {File.MD.QuickTime::ItemList\1.5\CreationDate\0} and a November 2013 file uses {File.MD.QuickTime::ItemList\2.2\CreationDate\0}.

In all there appear to be about 6 different tags in my database.

What would you do?
Because I've been using Dropbox up until now, video that has been taken recently is still being properly named via Dropbox. If I start manually copying from the DCIM folder I run into the date problem above. But because these are new files and everything else prior to that is correct in the iMatch database I can probably get away with just using the most recent Quicktime variable and update as iOS changes (I have not yet tested each household device).

If devices are all different I can have a metadata template for each = messy.

I may be able to get away with ExifTool command processing to copy the fields as the field always seems to be called Creation Date by ExifTool, no matter the video.

Code Select Expand

-overwrite_original_in_place
-model<quicktime:model
-make<quicktime:make
-mwg:CreateDate<quicktime:creationdate
-mwg:DateTimeOriginal<quicktime:creationdate
-xmp:metadatadate=now
{File.Path}{File.Name}.xmp

I'd prefer not to go down the path of scripting unless I have to and make determinations on the Software version used by Apple.

Has anyone else solved this problem?
Do you just ignore it?

Thanks in advance, David

[Mario]

Metadata + DateTime + TimeZones + Apple MOV + Apple ("everything we do is right, better believe it") software policies => problems.
If DropBox is getting it right (why?) ExifTool should be able to get it right too.

When producing the global "File Date & Time" from MOV files, IMatch looks at a whole bunch of tags because Apple produced so many variants of their metadata over time...something I still don't understand.  The MOV format is of Apple's making. The QuickTime metadata format is of Apple's design. Why are they unable or unwilling to create a "date and time movie taken" metadata field and stick to it.

The tags used and the sequence they are tried is based on heuristics and on the files I have analyzed over time.
If no XMP data exists, IMatch begins starting with QuickTime::UserData, then QuickTime::TrackHeader (multiple versions), then QuickTime::MetaData (multiple versions) and finally ASF::FileProperties. If all that does not contain a usable date and time, IMatch uses the file system date and time.
Some of the Apple tags may contain a time-zone offsets. Others may not but be in UTC or local time. No way to tell. Depends on the producing device and how serious the programmer were when writing the firmware.

Usually it's best to overcome the problems with setting the XMP created/digitized in IMatch (manually or by MD template from some other tags) and then stick to it. IMatch prefers the XMP timestamps and uses them for all date-based features.

Video metadata mess is even worse than image metadata mess.

Let me know which of the tags in "your files" contain the right data or send me some sample files with info about which tags contain the right time stamps. I may add some additional checks to the MOV date & time processing routine.

[dcb]

Thanks Mario. I couldn't agree more. I hope to get some more testing done over the weekend. At this point I'm going to next look at the ExifTool command processor to generate an XMP file as that's the only place I can see where it is reliably getting a date.

Form your list of past experience date checking I think you may check other QuickTime dates before getting to the one my files may need and if it's a case of first-come, first served, that may not work. Changing it one way may break something else.

Will get back to you once I've done some more testing.

[ubacher]

you may check other QuickTime dates before getting to the one my files may need and if it's a case of first-come, first served, that may not work

I was wondering about how to do this in a metadata template. Would it work to list the various (source) date fields in sequence and mark Insert only if tag is empty?
Then the first entry which fills the date field would disable the following ones.
If this works, could you, Mario, make such a template available based on your experience?

[Mario]

This should work but I don't see why I should create such a template.
IMatch performs the processing internally automatically when detecting a MOV file with one or more QuickTime records.

[dcb]

Got to test this evening instead of the weekend and have come up with a working solution that uses the ExifTool Command Processor, set to run once for each file.

Code Select Expand

-tagsfromfile
{File.FullName}
-overwrite_original_in_place
-extractEmbedded
-mwg:CreateDate<quicktime:creationdate
-mwg:DateTimeOriginal<quicktime:creationdate
-make<quicktime:model
-model<quicktime:make
-xmp:metadatadate=now
{File.Path}{File.Name}.xmp

Here's the breakdown for anyone else who is interested.

Code Select Expand

-tagsfromfile
{File.FullName}

I'm writing to an .xmp file so I first need to specify where I'm getting the QuickTime information from. Can't use the XMP file I'm going to add it to.

Code Select Expand

-overwrite_original_in_place
-extractEmbedded

The first line updates the xmp file in place (or creates it if necessary). The second line lets ExifTool search within the movie container for extra data. Not needed in my case to get the data I want, but does hide a warning message. If I was running for 1000's of file's I'd exclude this.

Code Select Expand

-mwg:CreateDate<quicktime:creationdate
-mwg:DateTimeOriginal<quicktime:creationdate
-make<quicktime:model
-model<quicktime:make
-xmp:metadatadate=now


Copy data from each of the QuickTime tags to the more standard metadata tags and get consistency with the rest of the database. The last line sets the time the metadata was updated to now. Not strictly necessary but I think it's good practice.

Code Select Expand

{File.Path}{File.Name}.xmp
Give the name of the output file. As I'm using the "Run for each file in selection" option I can't get away with {Files} and have to specify the full path and name with the new extension.

On my test data it works fine. Your mileage may vary.

(Now to work out why the GPS Lat and Long are ok but Altitude isn't showing up. It's there)

[Mario]

It seems that every QucikTime verson / file / device produces different data in one of the many QuickTime records. Pretty messy. Bad for users of course. Users should not have to deal with these things, Apple should do that for them by using one "Date and time this file was created" tag. But that's how Apple is in the details...

I asked above and I ask again. Can you record a small sample file and attach? Just record the lens cap if you will. I need to see the data.

ExifTool reports one that to IMatch that looks like the creation date you use, from the Keys group: QuickTime::Keys\creationdate\CreationDate Can you confirm that?

[dcb]

It seems that every QucikTime verson / file / device produces different data in one of the many QuickTime records. Pretty messy. Bad for users of course. Users should not have to deal with these things, Apple should do that for them by using one "Date and time this file was created" tag. But that's how Apple is in the details...

I asked above and I ask again. Can you record a small sample file and attach? Just record the lens cap if you will. I need to see the data.

ExifTool reports one that to IMatch that looks like the creation date you use, from the Keys group: QuickTime::Keys\creationdate\CreationDate Can you confirm that?

I don't get anything in QuickTime::Keys\creationdate\CreationDate as you've listed above. I added that field to the metadata browser, then went into Preferences Metadata 2 Tag Manager to add the group. Rescanned with force update and still nothing.

I have been getting the value in any one of QuickTime\ItemList\1.4\CreationDate\0, QuickTime\ItemList\1.5\CreationDate\0, QuickTime\ItemList\1.6\CreationDate\0 and that was only after I added the right tag group in Metadata 2. Which ItemList depends on the age of the file and that's what made using a Metadata Template impractical.

No matter which file, ExifTool always reports the right data as a single value in QuickTime:CreationDate which is why the ExifTool Command Processor is working.

Happy to send a sample. I think best is one of the Apple live photos because it has both the jpg correctly dated and the corresponding mov file so you can see how the dates are handled so badly. These are bigger than I can upload through the forum. Shall I email them?

[Mario]

Please don't write into quotes. It makes your post unreadable.

No matter which file, ExifTool always reports the right data as a single value in QuickTime:CreationDate which is why the ExifTool Command Processor is working.

The problem is that many of these magic tags are not part of the XML stream IMatch receives from ExifTool when it imports the data. I need to figure out which actual tags are used by ET to produce these synthetic tags and if I can access them somewhere in the XML stream.

[dcb]

Sorry. Writing into quotes was unintentional. Have fixed that.

Can I email the sample files to REMOVED BY ADMIN. As I said between 2-3MB each but with the JPG and MOV side by side it may make things clearer.

[Mario]

You can use that address. But PLEASE DO NEVER POST email addresses in public forums.
Robots are constantly polling communities like this to harvest email addresses for SPAMming.
I have reasons not to include emails in my signature etc.