How to propagate certain metadata between video files?

[DaweP]

I have some problems related to metadata of video files.

Video files taken with Nikon Z6 and indexed in IMatch show some useful metadata in thumbnails (date and time, aperture, shutter speed, and lens used).
However, if I make some short clips derived from the original video files in DaVinci Resolve, and index these in IMatch, these files show date-time related to when the file was created by DaVinci Resolve, not actually filmed. This is technically correct, yet not something I want to see. But importantly, these clips contain no other metadata (aperture, shutter speed, and lens used). These data were clearly not incorporated to the files by DaVinci Resolve.
Trying to address this annoyance, I created File Relation between the original video files (from Z6), considered Masters, and the cuts (from DaVinci Resolve), considered Versions. Then I am able to propagate the desired date-time from the masters to the versions. Problem partly solved.
However, it would be nice to propagate also the metadata on aperture, shutter speed, and lens used. But these fields remain blank. I tried several options of "What to propagate", even the option "All Metadata", yet still these data are not propagated.

So the questions are:
1) Do you now whether DaVinci Resolve can be configured to include the metadata to the files it creates? This would be the best solution. I have not found this setting, though.

2) Where is the problem causing that I cannot propagate the the metadata on aperture, shutter speed, and lens used? The file relation is set correctly, masters and versions are identified as expected and date-time is propagated correctly...

I wonder about the general handling of metadata in video files, I have little experience with this. Perhaps somebody can clarify it.
Interestingly, ISO value is not shown even for the master files... I set the layout editor to visualize the data using the following Custom template:
{File.MD.shutterspeed|value:formatted}___F {File.MD.aperture}___ISO {File.MD.iso}
and
{File.MD.photools.com::IMatch\2900\lens\0|default:Unspecified Lens}

It works very well for photos, but ISO is not shown for video files.

I appreciate your help.
David

[Mario]

Is this the same question we've discussed already via email today?
Then I repeat my questions:

1. Provide a sample video for download (from camera)
2. Provide a sample video produced by your video editing software.
3. Show us the propagation rules you have created (screen shots, we need to see all boxes you have ticked!)
4. If DaVinci creates XMP files, also include them.

Upload to your cloud space and send a link to my support email: support email address

Video files usually use no EXIF metadata, so Nikon may do something special again.
We need to see the actual video files and the metadata they contain and which metadata tags you propagate.

[jch2103]

So the questions are:
1) Do you now whether DaVinci Resolve can be configured to include the metadata to the files it creates? This would be the best solution. I have not found this setting, though.

As far as I've been able to determine, there isn't a configuration option in Resolve to include metadata from the video source file(s). It's possible I've missed something, but I don't think so.

[DaweP]

Is this the same question we've discussed already via email today?
Then I repeat my questions:

1. Provide a sample video for download (from camera)
2. Provide a sample video produced by your video editing software.
3. Show us the propagation rules you have created (screen shots, we need to see all boxes you have ticked!)
4. If DaVinci creates XMP files, also include them.

Upload to your cloud space and send a link to my support email: support email address

Video files usually use no EXIF metadata, so Nikon may do something special again.
We need to see the actual video files and the metadata they contain and which metadata tags you propagate.

Yes, this is the same question. I posted here, as instructed.

Well, concerning the metadata in video files (in general), I was thinking about it and it is actually logical that video files should not contain the data that can be adjusted during filming (ISO, aperture, focal length). As I film mostly in "manual everything", I did not realize it, only now... My fault! Considering this, it is strange that IMatch shows some data anyway... (perhaps as these were not adjusted during filming, i.e., were constant value during the whole clip)?

Concerning the files to provide, I currently have here only some video files of my 5 months old daughter that I would rather not share. I prepare some testing videos tomorrow and come back with the required details.

[Mario]

That's why I asked for a sample in my email this morning and again here in my reply.
Film a wall for 5 seconds. That'll do.

We needs to see if and which metadata Nikon stores in your file. Video files don't contain EXIF data, so I wonder where the ISO etc. should come from.
Timestamps are recorded in a QuickTime stream in MP4 files, and IMatch can use that of course. See How IMatch uses Date and Time Information to learn more.

[DaweP]

Here is a link to Dropbox ZIP file that contains a sample video from Nikon Z6, a short cut from DaVinci Resolve (suffix: _C01) and screenshots that illustrate what I see in IMatch screen (Fig1-Fig3) and how I set the propagation (Fig4-Fig5).

https://www.dropbox.com/s/j3aeq19o572sw3p/DP_video_metadata.zip?dl=0

The screenshots (darkened to highlight the important parts) are commented here:
Fig1: Two video files indexed in IMatch. Original left, cut version right. You see that the original file contains some information on shutter speed, aperture and lens. It also shows actual time of filming.
Fig2: Time shows when it was created by DaVinci Resolve. This is what I want to change to reflect the actual time of filming. Note that it does not show any data on lens or camera settings.
Fig3: After propagation all metadata, Time is now matching (this is what I wanted), but still no camera settings from the master file.
Fig4: How the detection rules are set...
Fig5: ...and what is propagated. I guess that "All metadata" should cover this.

[Mario]

When I try to download the ZIP I only get a corrupted file with 500KB. Is the file still uploading or is it damaged?

Propagating ALL METADATA is potentially dangerous. I would not recommend this. Many metadata tags contain things like dimensions or profile data, and this data does not necessarily match the file you propagate it into. This can render the files unusable. Beware.

[DaweP]

Strange. A new archive (27898kB) is here. Please, retry:
https://www.dropbox.com/s/f9i78uxr4x6jykm/DP_video_metadata2.zip?dl=0

I know that "All Metadata" is not optimal. This was my last resort when previous trials were not successful.

[Mario]

This one worked.

The original video contains a set of proprietary Nikon maker notes. QuickTime metadata. Some tracks. Some XMP.
ExifTool does some magic to copy some of the proprietary Nikon maker notes into Composite Aperture and Shutter Speed, which is used by IMatch. That is what you see in the File Window.

The second image has none of the proprietary Nikon maker notes. So no shutter speed or Aperture "virtualization" by ExifTool.
Nikon maker notes are a block of binary data, usually with pointers (offsets pointing at specific byte addresses in the original file).

I doubt pretty much that it would be safe to copy these. And ExifTool does not support copying maker notes between video files.
-tagsfromfile does not copy them, which means ExifTool considers them unsafe for copy.
That's all non-standard, Nikon-only, secret sauce stuff. Copying this data into another file would most likely ruin it. Since the byte addresses would still point to the data blocks in the original image - which don't exist or at a different place in the target file.

Why does your camera embed some XMP tags, but Nikon does not bother to record the shutter speed, ISO, aperture etc. in the XMP EXIF record? This would solve the problem nicely.
But they don't bother.

If you want to copy this info between files, write a Metadata template which copies the info from the Composite tags / Nikon maker notes into XMP metadata (of the original video).
The you can just propagate XMP and the info will carry safely in to versions.

Of course it would be smarter if Nikon would do that. They already embed a few XMP tags in the video, why not also embed some useful tags like shutter speed, aperture, ISO, lens etc.
That would be helpful right? I suggest you ask them. Will be laugh.

[DaweP]

OK, thanks for resolving this out. So in the end, it is Nikon's matter, as I expected 
Actually, these data are not essential for me, it just would be nice to have these. But as these settings might actually be "variable" during video recording, the only one which makes full sense is Lens used. I might try to follow your suggested Metadata template path if I find some time to dive into this.

And what option would you recommend to use to safely synchronize date and time from masters (originals) to versions (cuts)?

Speaking about time, there is one more strange thing I found. The time is one hour off (less) than should be. My camera is set correctly. All my still images show correct time, so it is specific to video files. All original video files are one hour behind the actual time when displayed in IMatch (correct in Windows Explorer). Do you have any idea why this happens?

[Mario]

IMatch shows local time (local time-zone and used it for File.DateTime).
See How IMatch uses Date and Time Information

Like with images, use XMP for everything, including date subject created and create date. This is easy and safe to propagate.
Your original file uses different time stamps in track data, quick time, XMP etc. Use the ExifTool Command Processor to see them all.

[DaweP]

So now I have to deal with two issues:
1) Propagation of Date-Time from masters to version...
Do you mean that I should run the ExifTool for my video files to create sidecar XMP files for all these video files and then propagate the correct time from the masters to the versions? I did not succeed with this so far... I created the XMP files, but the desired data are time is not propagated (checked XMP, All Data). If I now run the command for the two files, I get the following output (see the attached file) - the content is different. But honestly I do not know whether this shows metadata in the file itself, or in the sidecar XMP files.

2) To visualize Date-Time in the thumbnails
Here I struggle to find what metadata field I should choose as a source for Date-Time. I do not understand why almost all lines show wrong time (7:59:47) and only few lines correct one (8:59:47). Intuitively, I take the ExifTool output as a guide to identify fields with correct time (this is [MakerNotes:Nikon] as a source in Layout Editor. But I cannot find these...

Rather than dealing with all this metadata stuff, which is getting more and more complex, it seems that easier way is to use TimeWiz and manually set the correct time for every file. Not what I expected... 

[Mario]

1. Have you read How IMatch uses Date and Time Information, especially the section about videos?

Your original file contains timestamps in the Apple QuickTime namespace:

[QuickTime]     Create Date                     : 2022:02:20 07:59:47
[QuickTime]     Modify Date                     : 2022:02:20 07:59:47

and IMatch uses these to fill the XMP date subject created and create date timestamps.
As documented in the above help topic.
The MD Panel shows this time and the File Window as well. At least here and for your sample video.
When you propagate XMP, these will be propagated, no?`

So, you need to do nothing special. Unless you consider these timestamps wrong and want to use other metadata. Then fill the same XMP tags from whatever you want.

2. See 1.

That's all I can say about this. We have spent way too much time on this already. Emails and community postings. An analyzing your video files and metadata.
Just let IMatch doe what it does, don't try to work with proprietary Nikon maker notes in video files.
Use only XMP and you're set. IMatch does not use external sidecar XMP files for MP4 videos.

[DaweP]

Yes, we have spent too much time on this, I know it and appreciate your support. Just for the record, I have read the linked part of Help already before starting this discussion and again now.

Theoretically, all should work well for me with how the system is set "by default" - according to the hierarchy of "where IMatch looks for data" in the Help section.
Practically, I have some strange issues with something as simple as showing correct Date Subject Created for video.
To narrow it down to a most simple statement: if I take a photo and then switch to film and create some short clip, then index both files in IMatch... Photo will show correct time, video will show time -1 hour. Without me doing anything with the metadata. I consider this strange and the Help page does not indicate I am doing anything wrong.

All the things of metadata copying to file versions are different thing, not as important. But the shift in displayed time is strange.
Sorry for this, I really do not want to bother you. I appreciate your work, I have always appreciated it. But sometimes I just encounter something that cannot resolve myself, however I try.

[Mario]

Photo will show correct time, video will show time -1 hour.

Check your local time-zone offset.
Check if the metadata in the image / video contains time-zone offsets.
Check your time-zone related settings under Edit > Preferences > Metadata 2. See the Date and Time help topic for details about time zones.

[DaweP]

Done. I have not identified anything what would explain this.

As a last resort, please download this ZIP containing one photo file and one video file. Both created 2 seconds apart, today at 15:44. No modification, just downloaded from the card.
https://www.dropbox.com/s/y7jzg9dgbeq5xnd/time_test.zip?dl=0

Please, index both files in your database and tell me what time you see.

Windows Explorer correctly shows 15:44 for both files (see 1st attached file). Yet in IMatch, the video shows 14:44, i.e., one hour less (see 2nd attached file).
I have tested the same with photo/video files from different (Nikon) camera, Z50, and replicated the issue.

I have not changed anything from default IMatch time settings, no reason to do so - all was fine with images until now. So it all should be set at default - and thus should work well.

[Mario]

What's your local time zone.
What settings do you have for time zone?
What's the time zone in your video?
1 hour off and your local time zone is one hour off or two?
QuickTime has no time zone so local time zone is assumed. Did you check that?


I have explained this several times now. The File.DateTime IMatch uses is in local time (that's what the File Window shows), set during ingest.
See How IMatch uses Date and Time Information for details.

If your input file has no time-zone, IMatch assumes local tz unless you configure otherwise. You have all the options. Apply time-zone in the settings.
Check what's in the file, check your settings, configure as needed. Windows Explorer does weird things and does show things like File modification date, not the timestamp in the file for many videos.
What "Date" shows in Windows Explorer is usually the file creation date, and this is often unrelated to the timestamps in the metadata which IMatch uses.
But that is all explained in How IMatch uses Date and Time Information

The original question was about propagating metadata between files. Not it's time-zone issues with QuickTime metadata.
If you want me to spend even more time on this, open a bug report and I will schedule a new time slot.
Analyzing metadata takes so much time, and I have spent more than enough for one support case. I have still 30 emails from other users to process today.

[DaweP]

OK, I finally resolved this, so that the video files show correct time after import.
I am sincerely sorry that it took so long. Having a crying baby in the background did not facilitate it, honestly.

At the same time, I am sorry that the originally reported issue of metadata propagation resulted is something different. This problem was discovered only when dealing with the former... and turned out into something more important to resolve.

Thank you for the support and sorry again for so much of your time.
David