Exiftool and characterset (diacritics)

[jmhdassen]

I have loaded a bunch of JPG files and finally managed to do writeback to get the data the way I want it, except for some final 500 images.

All these images have location names with "non-latin" characters, many from eastern europe. But to keep it simple: also "Île-de-France" does not work. I have to change it to "Ile-de-France" for it to be written back.

I have read many of the discussion about this. Back in 2014 it was apparently already an issue. The advise is to use UTF8. This apparently is already the default in Metadata setting. But I tried default and explicit 'UTF8' and it does not make a difference.

I would think many people have location names with not-latin characters. So I am probably missing something?
Anyone has some advise on this ??

I have Windows-10 with american english as the default language. I normally do not work in any other language on my computer.
What really surprises me is that all names display fine in the IMatch GUI and the XMP files.
But Exiftool always gives this "Warning: Some character(s) could not be encoded in Latin" message. After which the location changes to "?Île-de-France" in the metadata editor. There seems to be a mismatch between exiftool and IMatch.
attached an example exiftool output.

I have finalized loading my data for the moment by 'latinizing' all the failing location names. But this is not really what I want.

Regards, Jozef Dassen

[Mario]

XMP can handle UTF-8. GPS not really. Legacy IPTC only when the special UNICODE encoding variant is used. EXIF can only handle ASCII.
If you store non-ASCII characters in metadata fields which must be migrated between multiple metadata formats, you may be better of to stick to XMP alone, e.g. removing legacy IPTC data from your files to avoid interference.
XMP has solved all these problems, but if users still need to support legacy IPTC and camera vendors insist on sticking to the 30 year old EXIF metadata format instead of using XMP exclusively, there will always be problems when mapping data between metadata standards. This is covered in the IMatch help, e.g. under Metadata

[jmhdassen]

As usual, a very defensive and not quite useful answer.
I am not trying to stick to any specific format.
I just want my location data to show up correctly. As far as I can see, it is only in XMP tags (XMP:photoshop and XMP:lightroom).
The camera is not 30 years old. It is common Nikon NEF file processed by Capture One, also fairly standard.

If you could give my a more practical answer that would be nice.
This is after all a fairly common scenario I would think.

For instance: how do I remove IPTC data from my files (if that would solve the problem).

If I understand you well, you are saying that IMatch can not handle "older" files. Not very good for a Catalog...... It would then indeed be better to be able to switch off this exiftool writing (because that is where the problem comes from).

JD

[jmhdassen]

For your information: I just tested with Photo Supreme. They write these location  names with foreign characters without error to the same JPG file.

Not that I like Photo Supreme more than IMatch. But at least they show it is possible (I do not know if they use exiftool).

JD

[Mario]

I set the GPS coordinates for a JPG image the coordinate found when I search for Île-de-France in the Map Panel.
IMatch reverse-gecodes the file and adds:

City   77200 Torcy
Location   
ISO Country Code   FRA
Country   Frankreich
State/Province   Île-de-France
GPS Latitude   48.84992
GPS Longitude   2.637041
GPS Altitude   85.23


I click the pen to write-back the data. The file is a JPEG file.
No problems. Île-de-France remains unchanged. Note that the JPEG has no legacy IPTC metadata and I use the default settings in IMatch (embedded XMP).
XMP data in the JPEG shows:

[XMP-iptcExt]   Location Shown Province State   : Île-de-France
[XMP-photoshop] State                           : Île-de-France

Perfect result.

Tip: To delete legacy IPTC data in your files, use the "Delete legacy IPTC Data" preset in the The ExifTool Command Processor

[jmhdassen]

That is good. So it is possible that exiftool writes these characters.
But then there still is a problem. If I enter "Île-de-France" in the Metadata Editor I can see it and save it to the database. But writeback/rescan screws it up. So looks like a software problem to me. Probably a character set issue. Should be easy to trace.

Dont expect me to reverse geocode my locations. That would be an ugly workaround to a software problem. I do not have GPS data to begin with.

Thanks, Jozef Dassen

[mastodon]

I had exactly the same problem, and clear all IPTC data was the solution.

[Mario]

That is good. So it is possible that exiftool writes these characters.
But then there still is a problem. If I enter "Île-de-France" in the Metadata Editor I can see it and save it to the database. But writeback/rescan screws it up. So looks like a software problem to me. Probably a character set issue. Should be easy to trace.

Dont expect me to reverse geocode my locations. That would be an ugly workaround to a software problem. I do not have GPS data to begin with.

Thanks, Jozef Dassen

This has nothing to do with reverse-geocoding. This process fills the same metadata tags you can also fill by hand or via a Metadata Template.
As I said, check your files for legacy IIM3 IPTC data, which handles only ASCII or local code pages. And this may clash with you adding non-ASCII characters or characters not covered by the code page in your legacy IPTC data. IIM3 IPTC data has been retired almost 20 years ago. For good reasons.