Tags Renamed or no longer Imported

[Tveloso]

I have (Version and Buddy) File Relations configured for iPhone Live Photos (photos taken with this feature produce both a JPG or HEIC, and a MOV File when moved to the PC).

In order to ensure that I am not inadvertently "connecting" unrelated files that happen to have the same or similar names on Input, I also use the Limit by Variables option:

       

...with what I found to be a UUID that both files have in common.

Starting with IMatch 2021 (or possibly with a later 2020 version - it's been a little while since I've added new files to my database), I found the that the follow is happening:

• Content Identifier Tag in the Apple Section has been renamed
  For new HEICs and JPGs indexed, the Content Identifier Tag ({File.MD.Apple::Main\17\ContentIdentifier\0}) is now named ({File.MD.Apple::Main\17\MediaGroupUUID\0}).
  
  I believe I recall seeing here, or in the Release Notes, that ExifTool has renamed some Tag identifiers, and at the time thinking that my IMatch Installation would probably not be impacted by this.  But it looks like it has been.
  
  This particular change in behavior is not really a problem (I can simply update my relation definitions), but the next item is actually causing an issue for my relation definitions.
  
• Content Identifier Tag in one of the QuickTime Sections is no longer being imported
  The Tag that I found for MOV Files ({File.MD.QuickTime::ItemList\1.1\ContentIdentifier\0}), which matched the one discussed above, appears to no longer be imported for new files.
  
  I checked Tag Manager, and all references to Tags with that Name (in both the Apple and QuickTime groups) appear to be checked:
  
         
  
  ...but IMatch is no longer importing this Tag.
  

I found that when re-scanning existing files (with a force update), the tag in the still is renamed, as discussed above, and the Tag that's now not being imported for new MOV Files does not get discarded:

       

...but if I add that same MOV file to my test Database, the tag is missing there (in fact the entire QuickTime ItemsList section is missing):

       

Might this be a bug?...or is there possibly another item in Tag Manager that I must now check, in order to get that Tag back into IMatch?

[Mario]

You are working with proprietary Apple tags.
ExifTool makes tag names up for this kind of tags. There is no guarantee that Phil changes the tag names of these proprietary tags when he does some updates or reorganizations in ExifTool.

Keep in mind that all the stuff Apple puts in their files is neither fully documented nor standardized in any way.
They change it often, rearrange things when a new iOS or MaxOS update is shipped etc. Apple is not about compatibility with other platforms, they only care for their own ecosystem.

When a new ExifTool version is shipped with IMatch, IMatch re-imports all tag names into the database. This keeps existing tag names for downward compatibility. But when ExifTool version n+1 has changed some tag names compared to version n and now delivers 'the same' data under a different name, IMatch just imports this data as delivered.

I frankly don't care for the 10,000+ proprietary maker notes camera vendors include, or not include in their files. They change all the time and I have enough to do to keep up with the ISO standardized XMP metadata and the so-so- standardized EXIF stuff.
I don't know anything about this proprietary Apple maker notes you are using. And if things like "List" is included in a tag name, it means that ExifTool has enumerated something in list form, and that the numbering and order may change between files, depending on how many things are in that list per file.

Does Apple not store anything in a standar

[jch2103]

@Tveloso - For what it's worth, the ExifTool Version History https://exiftool.org/history.html lists more than a few changes/updates related to QuickTime, etc. Kudos to Phil and all the folks who contribute to keeping ExifTool up to date in the face of all the ongoing changes by Apple, camera companies, etc.!

[Tveloso]

Mario, I was really only concerned about that Tag that IMatch no longer imports.

Exiftool does still show that Tag when I run the List Metadata ECP Preset:

        [QuickTime]         8 Handler Type                    : Metadata Tags
        [Keys]              - Location Accuracy Horizontal    : 65.000000
        [Keys]              - Content Identifier              : A0A0C227-4A97-400F-A4A7-F3CEFF4E71A2

...so I thought that perhaps the ExifTool changes responsible for the rename in the other Tag, might also have possibly caused this one to now be mapped to a Tag that is not set by default in Tag Manager?...(and this might be something easy to identify)...

But I completely understand that you cannot devote time to something like this, which might be effecting just one user, and is squarely in the realm of "propriety Apple stuff".

I can just use DateCreated as my limiting variable in the relation definitions

@Tveloso - For what it's worth, the ExifTool Version History https://exiftool.org/history.html lists more than a few changes/updates related to QuickTime, etc. Kudos to Phil and all the folks who contribute to keeping ExifTool up to date in the face of all the ongoing changes by Apple, camera companies, etc.!

Thank you for the Link John.  Yes, I can only imagine what a monumental task dealing with those constant (and undocumented) changes must be!

One item under v12.27 looked like it might speak to this:

• Fixed family 7 group names for QuickTime Keys tags

...since the Tag I'm looking for is in that Keys group.  Perhaps I'll look around a bit more in Tag Manager in IMatch to see if I can locate something that just needs to be enabled.  But if not, I can certainly use DateCreated.

[Mario]

IMatch imports several hundred tags by default, and keeps over 10,000 possible tags out of the database (because most of them are not useful).
It also updates the database with all tags ExifTool reports. If the tag is not imported, you  need to find and enable it in the Tag Manager.
Use the -X option in the ECP to see what IMatch sees when it imports a file.

Feel free to send me a sample file (with a link back to this thread) and the info which tags you want to import.

[Mario]

I've had a look at your HEIC files.
The UUID you mention exists in the tag:

[Apple] Media Group UUID: A0A0C227-4A97-400F-A4A7-F3CEFF4E71A2

in the HEIC file and it is imported by default by IMatch.

In the MOV file, the tag is listed by the ECP as

[Keys] Content Identifier : A0A0C227-4A97-400F-A4A7-F3CEFF4E71A2

Keys is not one of the "tag groups" ExifTool lists as groups, and the only "Keys" group I could find is QuickTime::Keys, but it does not contain a Context-Identifier tag, or anything named similarly.

In the XMP data IMatch uses to ingest ExifTool data, the tag is listed as

<Keys:ContentIdentifier>A0A0C227-4A97-400F-A4A7-F3CEFF4E71A2</Keys:ContentIdentifier>

Since the Keys group is unknown to IMatch (not part of the group list ExifTool exports), IMatch rejects the values during ingest. It cannot import values from unknown groups.
This is something only Phil can answer (why is this group not listed, maybe created on-the-fly somehow)?
I will ask him.

[Tveloso]

Thank you so much Mario.

[Mario]

I can see the data in the QuickTime::ItemList\1.2\ContentIdentifier for the MOV file, and in the Apple::Main\17\MediaGroupUUID for the HEIC file.

[Tveloso]

I can see the data in the QuickTime::ItemList\1.2\ContentIdentifier for the MOV file...

Yes, that's where it was being imported when indexing these types of MOV Files under IMatch 2020 (in a Tag under the QuickTime::ItemList Section, although in a slightly different Tag Name from the one you mention above), but under IMatch 2021, that Tag Section was no longer present in the Metadata Panel for the file.

The sample file I provided shows that QuickTime::ItemList Section in my Production Database (where the file was originally indexed by IMatch 2020):

       

...but in my Test Database (where that same file was newly indexed using IMatch 2021), the QuickTime::ItemList\1.2\ContentIdentifier Tag was not present:

       

I found that a QuickTime ItemList Group was not enabled in Tag Manager, and when I checked that box:

       

...and did a Force Update, that Tag Section was back, and it included the "new" Tag you mentioned above:

       

I'm sorry to have bothered you with this Mario...I should have seen that QuickTime ItemList Group was not enabled in Tag Manager.

I did look around a bit there, and found what I thought corresponded to the Tag that I was interested in (several items actually had relevant-looking Names), but everything I found was already enabled.

So it seems that ExifTool did "rename" this tag, as it did for the corresponding HEIC File (or, moved the data into another Tag with a similar Name?)

Thanks again for pointing me in the right direction.

[Mario]

As Phil said in our discussion, he sometimes renames tags when new data becomes available or new things are learned.
I usually take care of this on the database level - but when you directly access proprietary maker notes like in this case, you will be affected by changes in naming ExifTool might introduce.

That's why I've introduced the Metadata Tag ShortCodes many years ago and encourage users to use them.
For the 10,000+ make note tags of various brands and vendors, there is no such thing, of course.