Video Timestamps (Was Re: [NB] Metadata Template)

[joel23]

Existing XMP data of course overrides the QuickTime data. That's the key point here.
The original question was which timestamp is used by IMatch for various move files if there is no XMP data. I answered that, and also explained how XMP data can be used to override the non-standard QuickTime timestamps.

Well Mario...
it's clear for me now...  but IMatch creates XMP as soon the movies are ingested. Even when users have "immediately write back metadata" disabled, I can't imagine they live with a yellow pencil on there movie files for long. And we of course want to apply other metadata like GPS, Copyright and Keywords to them. 
By this the question, which timestamp is used when there is no XMP, is in a way obsolete.

As you might understand now, this looks (or at least the way it was explained) as a bug to users. The whole sub-thread could have been avoid, except the answer to the original question "how to do that with a metadata template".

Since more and more cams also capable to produce videos, here might be the requirement for IMatch treating videos the same way it treats photos.

@Frank
You either shouldn't create XMP (never click on the yellow pencil nor click Metadata write back -> for all pending files) or use a metadata template as suggested.

[Mario]

Since more and more cams also capable to produce videos, here might be the requirement for IMatch treating videos the same way it treats photos.

Then we should wait if some of the big vendors bother to setup some common standard for "XMP for video". Like the MWG.

Or, cameras could just add an EXIF record to the movie. Should be doable.
Or, better, camera makers could adopt to the 10 year old XMP standard and include one in the generated image. This would solve all these problems nicely, and the camera vendors can use the camera clock or even the GPS to fill in lots of useful data...truth is, they don't care.

We here could setup a poll and allow users to vote which timestamp contained in whatever video format they use should be used by IMatch for the initial filling of the XMP EXIF date and time tags or the "File Date & Time". I'm sure many users have ideas about this. I don't even know all video formats in use today, or which metadata these files may contain. QuickTime has about 4 to 8 timestamps, depending on the QuickTime variant in use, which software was used on the file etc.


@All camera users:

Please send sample files, show me the metadata extracted by IMatch, which timestamps you would like to see in the XMP metadata for your video files. Since there is no standard, we are free to do whatever we want. Not sure how future proof this is, but setting a few XMP timestamps from arbitrary video metadata should not harm.

Otherwise I can only make a small project out of this - evaluate all common video formats, try to find samples, try to find out if and which metadata these files contain (may vary between camera models and software used), figure out what data ExifTool can extract from these files and then come up with some sort of mapping table. Then implement a new args file for ExifTool or talk to Phil if he is willing to do this as a standard  in future ExifTool versions.


Since all this can take months, I encourage you to use a Metadata Template for the time being.

I was able to set the XMP created timestamp from the QuickTime metadata by filling it simply from:

{File.MD.QuickTime::MediaHeader\1\MediaCreateDate\0|value:formatted}

After applying this template, the "File Date & Time" used by IMatch everywhere (including the time line) was also using this date and time. Perfect.
I could use the same template to set any other XMP data from whatever QuickTime data I like to copy in whatever XMP tag.

[jch2103]

@All camera users:

Please send sample files, show me the metadata extracted by IMatch, which timestamps you would like to see in the XMP metadata for your video files. Since there is no standard, we are free to do whatever we want. Not sure how future proof this is, but setting a few XMP timestamps from arbitrary video metadata should not harm.

Mario - Sounds like a good project. How do you want us to send you sample files (esp as video files are so large)? I'm certainly willing to participate, as I find dealing with video date/time metadata from my camera annoying. It would be useful to have a 'standard' way of dealing with this, even if it's limited to IMatch users!

[Richard]

I doubt that the size of a file would effect how metadata is stored within files. I would think a few second video would work.

[joel23]

Since more and more cams also capable to produce videos, here might be the requirement for IMatch treating videos the same way it treats photos.

We here could setup a poll and allow users to vote which timestamp contained in whatever video format they use should be used by IMatch for the initial filling of the XMP EXIF date and time tags or the "File Date & Time". I'm sure many users have ideas about this. I don't even know all video formats in use today, or which metadata these files may contain. QuickTime has about 4 to 8 timestamps, depending on the QuickTime variant in use, which software was used on the file etc.

Regarding the Create Date:
all movies I have so far from six different (also older) cams, plus two samples from Canon 7D and Sony Alpha 7 II I just downloaded and three different movie editors I use, write a QT record at least with the "MovieHeader Create Date" and "MediaHeader Create Date" tags. Regardless which QuickTime "variants" respectively MOV or MP4 versions ares used.
In other words: as soon a device or software creates a QT record, the dates are to find there.

If I understand the QTFF specification right, the "QTFF movie atom" acts as a container for several other atoms, like (several) media atoms and track atoms:

The movie atom is essentially a container of other atoms. These atoms, taken together, describe the contents of a movie. At the highest level, movie atoms typically contain track atoms, which in turn contain media atoms.

By this I think it is safe to use QTs "MovieHeader Create Date", since this is the creation date of the whole container or as we use to say: the movie (MP4, MOV)
I would map that to XMP:CreateDate and/or xmp:EXIF:DateTimeOriginal.


BTW, not sure if this might be relevant: XMP specs 3 tells (see Table 21, also for artist, comments, etc.pp) that at least for IFF/RIFF based formats (AVI, WAV, AIFF) ICRD should / can be mapped to xmp:CreateDate.
I don't think it would (or should) be much different or of harm when it's done this way for other container formats as well.  ICRD seems the be the equivalent to or can be mapped to EXIFs DateTime Original (http://dev.w3.org/2008/video/mediaann/mediaont-1.0/mediaont-1.0.html)
ICRD (ContentCreateDate) is not to find in Phils' Quicktime tag list, but according QTFF it is internally mapped to "©day" (which is to find in his QuickTime ItemList Tags)

[jch2103]

@All camera users:

Please send sample files, show me the metadata extracted by IMatch, which timestamps you would like to see in the XMP metadata for your video files. Since there is no standard, we are free to do whatever we want. Not sure how future proof this is, but setting a few XMP timestamps from arbitrary video metadata should not harm.

I've attached ExifTool Command Processor List Metadata dumps (-G1 -all {Files}) from two short videos, one from a Nikon D600 and the other from a Nikon 1 V1. Both were added to IMatch (to run ECP) but with no metadata edits.

The two short (~4 second) clips are too big (24 MB total) to attach here; a download link is https://www.sugarsync.com/pf/D6380252_62506476_7543650

[attachment deleted by admin]

[Frank]

Hi all,
I have created a google spreadsheet for adding all files with the necessary information, so that Mario must not figure everything out.

So everybody is free to add the necessary information to that spreadsheet. Just follow the link
https://docs.google.com/spreadsheets/d/18mdRCJiD3zXo2MzKOb-Kok8hWXUdmNvJOdSYg0IfgME/edit?usp=sharing

If this is a way that Mario find it useful I can add all my Vidoformats within next week in this way. Some Videos I will not sent this way therfor I just added the information extracted via ExifTool Command Processor.

[joel23]

@Frank,
good Idea - I was about to suggest something similar. Otherwise there would be duplicate uploads of ExifTool Outs and uploaded movies soon.

[Mario]

Maybe somebody puts this in to a FR? This discussion is in the solved bug report board and I will not keep track of it.

[Ferdinand]

I spit the thread and moved it to a FR.

[rainforest1155]

I was able to set the XMP created timestamp from the QuickTime metadata by filling it simply from:
{File.MD.QuickTime::MediaHeader\1\MediaCreateDate\0|value:formatted}

After applying this template, the "File Date & Time" used by IMatch everywhere (including the time line) was also using this date and time. Perfect.

That works great for my Canon videos. However, on Apple devices, this field seems to include the time in UTC format and thus depending on the time zone your device is set to, the time may be off. Checking the metadata of my different Apple videos, there seem to be 2 different tags that contain the local time with the time zone included as well
File.MD.QuickTime::ItemList\1.1\CreationDate\0
and
File.MD.QuickTime::ItemList\2.2\CreationDate\0
There's also a third tag with \2.5\, but in my case none of my files had any date in it, but I guess others may have different Apple files that might contain a date there.

To have IMatch automatically extract the date of any of the field, just replace Mario's data with:

Code Select Expand


{File.MD.QuickTime::ItemList\1.1\CreationDate\0|default:{File.MD.QuickTime::ItemList\2.2\CreationDate\0|default:{File.MD.QuickTime::ItemList\2.5\CreationDate\0|default:{File.MD.QuickTime::MediaHeader\1\MediaCreateDate\0|value:formatted}}}}

That should check for any Apple time metadata and if those are blank, use the MediaCreateDate instead.

I've tested this with my own videos (not just the Apple ones) and it's working fine. Finally, all videos are properly displayed when sorting by capture time.

Hope this helps others with Apple devices.

Sebastian