Tag manager and COMPOSITE values: How are they handled.

[ubacher]

There seems to be no entry for COMPOSITE in the tag manager.
But the COMPOSITE values seem to have gone after a metadata tag trim.
Can someone confirm this please.

Is there a way to recover these? - they are in the file - shown by exiftool.

[Ferdinand]

My understanding is that composite values are *not* in the file, they are values that ExifTool derives from certain fields on-the-fly and displays.  So if a composite value has gone that's because the field(s) that it was based on are gone.

[ubacher]

Exiftool does show COMPOSITE fields. I assume it is exiftool which creates them and IM just reads
and stores them. Except it does not seem to any longer.

[Mario]

Composite tags are made up by ExifTool on the fly.
99% of the data in these tags is duplicated from other tags, which is why IMatch skips them completely - except some convenient tags which IMatch uses internally.

[ubacher]

IM does not seem to set all the composite fields any more.
Example: Screenshot

[attachment deleted by admin]

[Ferdinand]

The ECP output shows that these tags still exist in the file.  This makes sense as IMatch won't remove them from the file.

The metadata panel shows that these fields were not imported by IMatch (the ones you highlighted do not appear to be composite fields).

This is either because (i) you didn't enable them in the Tag Manager; or (ii) you did enable them but haven't re-read the metadata for this file since doing so.

[ubacher]

(i) you didn't enable them in the Tag Manager;

How do I enable? There is no tag group named COMPOSITE. Another name?

[Ferdinand]

As we tried to explain, these composite fields do *not* exist in the file, they are constructed by ExifTool on the fly from certain other fields.  So they do not appear in the Tag Manager, which only lists fields that *do* exist in some of your files.

If you want IMatch to display a composite field then in the Tag Manager you need enable those certain other fields that are used to calculate it.

What are those other fields?  They are listed here:
http://www.sno.phy.queensu.ca/~phil/exiftool/TagNames/Composite.html

So for example, to calculate DoF, it would appear that you need to enable:
FocalLength
Aperture
CircleOfConfusion
FocusDistance
SubjectDistance
ObjectDistance

[Mario]

IMatch does not longer import the Composite tags, because 99% of these tags just duplicate information in other metadata tags. This change alone saves maybe 30% of disk space and, more important, reduces the number of metadata entries IMatch needs to store per file considerably.

Composite tags are made up by ExifTool. Furthermore, writing to (updating) a Composite tag has sometimes rather strange side effects, which is why IMatch does not do it. All the information you are looking for is part of XMP or EXIF and are imported by IMatch automatically. For some camera models, unusual metadata like CircleOfConfusion is not part of standard metadata, but stored somewhere in a proprietary maker note. If you need this information, you need to enable the corresponding groups and tags in the Tag Manager (Ferdinand pointed that out above).

Please see the release notes for the IMatch version you are using if you have not yet heard about the Tag Manager.

[Ferdinand]

IMatch does not longer import the Composite tags

Not all all?  OK, I missed that.  (Still on 5.2.8 on the production machine until I finish a big job).

[Mario]

No. The bulk of the Comp takes is skipped. Only some tags IMatch uses internally are still imported. As said, 99% of the comp tags just duplicate information already in other tags (IPTC, EXIF) and I should never have imported the Composite tags at all.

[ubacher]

I find the Composite values for Depth of Field, Hyperfocal Distance and maybe Light Value interesting.

Since these values are the only ones which are displayed in the Metadata browser but no longer filled I suspect they have
just been overlooked.

It would not make sense for me to calculate these values when they are present in the metadata of the file.

[Mario]

All these tags are either part of the EXIF metadata, or some custom maker note produced by your camera model. This depends on your camera, and firmware. I'm sure you'll find the tags quickly in the Metadata Panel in Browser mode. If you don't see them, enable the maker notes for your camera model. Use the ECP to see the data actually in the file to find what needs to be imported. Don't import anything. Your database has 200,000 files and IMatch will be very happy when 10 or 20 million records can be removed from the database.

[ubacher]

We did not have Depth of Field in IM3 so I can surely do without it in IM5

Too much trouble to write a script which needs to be run every time one has ingested a file the first time.

When I want to see the DoF I can just display the exiftool output.
Or maybe I just make a script which reads the value from the exiftool output - on demand only.

====================
Related comment: What really pisses me off is that modern cameras do not display Depth of Field.
We need to carry along tables or smartphones with a DoF program to look up what the DoF is.

Looking at DoF metadata in previous versions of IM helped me to explain why certain images lacked sharpness.
Made me pay much more attention to DoF when setting the camera.

[Mario]

The CircleOfConfusion tag (I now what it is but I doubt it will be of any use) seems to be a rather special maker note. According to the Composite tag description at

http://www.sno.phy.queensu.ca/~phil/exiftool/TagNames/Composite.html

ExifTool calculates it from another Composite tag named

ScaleFactor35efl as: calculated as D/1440, where D is the focal plane diagonal in mm. This value may be incorrect if the image has been resized.

The ScaleFactor35efl Composite tag is filled from a wide variety of tags:

LensType
FocalLength
MaxAperture
MaxApertureValue
MinFocalLength
MaxFocalLength
LensModel
LensFocalRange
LensSpec
LensType2

and that's maybe not even all.

As said, I can re-enable certain Composite tags if there is demand. It's all configured in a configuration file. Pretty easy. But the bulk (99/100) of Composite tags are just a waste of space because they duplicate 1:1 other information.

If there are specific wishes, please open a feature request and explain which composite tags you would like to have back and why. I can then ship an updated configuration file with the next build and after reloading the metadata for your files the composite tag(s) will be back.

[ubacher]

I see a big advantage in using composite values which are calculated from other fields: Exiftool
figures out which fields it needs, does the calculation and outputs it. There is no need to load all
those fields which are only necessary to do the calculation.

As the exiftool docu says most of the fields are convenience fields.

DoF and Hyperfocal distance and Focus distance are calculated.

[Ferdinand]

I thought that part of the purpose of composite tags was to deal with the situation where different camera makers store the same information in different fields.  So if you have several such cameras you can just view the composite field rather than having to find and display several fields.  That ExifTool page on composite tags isn't clear on whether my understanding is correct or not.  Is it?

[One variable that I'd like to see imported by default is the shutter count, which is different for different camera makers.  But don't do this just on my account - I'm going to set aside some time to go through the tag manager in a forensic manner.]

[Mario]

As I said, put the Composite tags you want into a feature request and I'll enable them on a must-have basis.
There are about 120 or so Composite fields, and most just mirror other info 1:1.

[sinus]

Related comment: What really pisses me off is that modern cameras do not display Depth of Field.
We need to carry along tables or smartphones with a DoF program to look up what the DoF is.

Looking at DoF metadata in previous versions of IM helped me to explain why certain images lacked sharpness.
Made me pay much more attention to DoF when setting the camera.

In the attachement my current Metadata-Browser, Composite-group.

I wonder really, for what you like to see Depth of Fields. Do not take me wrong, I wonder simply. For me the DoF is rarely important, if, then mostly in the macro or portrait-photographie. But also there seldom. If, I can look also at my lens, most lenses (Nikon) have roughly graved the DoF, depening on the distance.

But why do you need this after the shooting?

For the composite-stuff, maybe you should do, like Mario wrote, create a feature request.

BTW: I take the Tag Manager like Ferdinand, I guess, only single steps and checking, if I change there something.
Globally I think, the Tag Manager is a nice tool, but as we can see here in some threads, it is not that intuitive, like it could be (if it would, we had not such a lot of questions about it).
But I guess also, this is a lot of work for Mario, and normally we change the Tag Manager not a lot.



[attachment deleted by admin]

[Richard]

normally we change the Tag Manager not a lot

Or for someone like me, not at all. I tend to think that most users are simply glad to get rid of garbage and leave it at the default settings.

[Mario]

Or for someone like me, not at all. I tend to think that most users are simply glad to get rid of garbage and leave it at the default settings.

I also assume that most users never will do even as much as look at the Tag Manager.

Anyway, I have no re-enabled the following composite tags for the next build:

Flash, LightValue, CircleOfConfusion, FOV, HyperfocalDistance, ScaleFactor35efl, DOF.

All other Composite tags were available anyway (because IMatch uses them in various internal routines). The Composite group is just protected and hence not visible in the Tag Manager.

[ubacher]

I'm happy. Thanks.

(A case of: It's always difficult to take away something from people after they got used to having it.)