I cannot write back pending metadata for 91 photos

[OdysseusL]

Hello,

I still test IMatch. Now I have a strange behaviour: I still need to write back pending metadata for 91 photos. If I select either one or all of these photos and press the "pencil button" the pencil button vanishes for a few seconds, and than it appears again to be active. It's every time the same 91 pictures (Panasonic RAW format: RW2). I have thousands other RW2 photos, which do not show this "effect".

Any hints or tips?

Thanks in advance.

[Mario]

I suggest you start with Metadata Problems and Pitfalls, especially Files Pending Again After Write-Back, which explains the most common reasons for IMatch being unable to finish the write-back.

1. Write-back twice. This usually sorts out issues caused by out-of-sync metadata inserted by other software.

2. If this does not help, point the mouse at the pen icon to see which tags need to be written. This usually helps to analyze the problem.

3. Run the Metadata Analyst app included in IMatch on one of your RAW files to check the metadata for problems.
If problems are reported, use the GREEN BUTTON at the top to copy the warning and error messages into the clipboard and paste them into your reply.

4. If possible, upload a sample RAW (with the XMP sidecar file) to your cloud space and post a link.
This allows us to look at the actual file causing the problem.

Include information about which other software you have used before, if you use versioning in IMatch or non-standard metadata settings.
Since you are new to IMatch, I assume that you don't use versioning yet and left all settings to default.

[OdysseusL]

I stopped IMatch and restarted it. And now it works. "Stranger Things" ;-)

[Mario]

If this happens again, make a copy of the IMatch log file (see log file).

If restarting IMatch solved this, there is a good chance that your virus checker was somehow interfering (e.g., blocking ExifTool) or that the files processed were maybe locked by another application.

I have also just encountered, and resolved, an obscure issue with Windows WIC keeping locks on files that have already been closed by IMatch.
This, sometimes, and for some file formats, and only on some computers, prevented IMatch from writing back files which were displayed recently in the Quick View Panel.
Not relevant for your 90 files, just mentioning it here to introduce the https://www.photools.com/release-notes/ to you.


In both cases IMatch will log this to the log file, the ExifTool output panel (The Output Panel) and, usually, also mark the files as having problems with the yellow warning icon in the File Window. These files then will also show up as problem files in the The IMatch Dashboard.

I mention all this because you are apparently new to IMatch and most likely don't know yet about these features yet.
The log file and the Dashboard can be very helpful in figuring out issues or problems with your file collection.
They also help me to help you.

Yes. I believe it is better to give the user as much information as possible when something goes wrong.
I hate when an application tells me "Oops, something went wrong", without any additional info about what went wrong or details that may help me to figure out the problem...

[OdysseusL]

Thank you very much. I will take care of your hints and tips, before I post (from my point of view) a problem.

[OdysseusL]

So, the behaviour of writing back pending metadata for 91 photos happens again.

It is the same 91 (RW2) photos since starting this thread. I write back the metadata, the yellow pencil vanishes for a few seconds, and then it appears again. Only the Explorer, Firefox, and IMatch are running.

If I hover the mousepoint over the yellow pencil IMatch shows me the following text (for all 91 pendig RW2 files):
"This file has unwritte metadata. Click the icon to write that data now. List of tags to write: XMP::dc\Subject".

I've attached a log file. Starting "debug log" right before I press the yellow pencil for one photo (Img_62160.RW2) . I switch back to "normal log" right after the yellow pencil icon appears again.

I've attached the output of my local installed exiftool.exe of one of the appropriate RW2 file (Img_62160.RW2), too.

Don't hesitate to reply, if someone needs more information.

Thanks in advance.

[Mario]

IMatch wants to write a flat (!) XMP keyword. How can this happen?

You never change flat keywords in IMatch directly, only hierarchical keywords.
IMatch then produces the flat keyword during write-back, or when you change hierarchical keywords.

Why did the content of this tag change? Very unusual.

Do you perhaps manually change the XMP-dc:Subject tag somehow? Metadata Panel? Metadata Template?

Switch the Metadata Panel to the "Keyword" layout.
Make a screen shot, crop and attach to your reply.

Also let us know how you add keywords, if your RAW file contains legacy IPTC data (run the Metadata Analyst) etc.

Pulling data from the file via ExifTool is not helpful, because what you see there will differ from the database, you will not have MWG mapping applied etc.

[sinus]

I have no clue, because I am not an expert in this.

But in the exiftool, it seems to me a bit curious, that there are so many weired letters, like here:

Code Select Expand

Subject                         : Architektur, Zoo_und_Tierpark, Wildparadies_Tripsdrill, Natur, Tiere, V├Âgel, St├Ârche, Orte, L├ñnder, Deutschland, Baden-W├╝rttemberg, Cleebronn_Tripsdrill, Wildparadies_Tripsdrill

Maybe this has nothing to do with your problem, but maybe it would be worth to correct this in one file to try.

[Mario]

But in the exiftool, it seems to me a bit curious, that there are so many weired letters, like here:

The OP probably did not know about character set encodings and did not configure his console to the proper character set.
What you see in the text file is what you get when you don't know about all the metadata character set technicalities, UTF-8 encoding and suchlike.
Then you mangle and destroy metadata values not encoded in ASCII or your local character set.
I'm sure the keywords look OK in IMatch, because IMatch deals with character encodings.

[OdysseusL]

Yes, the characters looks find in IMatch (UTF-8). The weird characters are a result from copy from the Windows terminal window. Sorry my mistake.

I created the keywords within Adobe Lightroom Classic (current version), as all other almost 75k photos. Now I have migrated to IMatch via including all these photos in IMatch. From the 75k photos only these 91 show this "behaviour".

No I have not change any "XMP-dc:Subject tag" by myself. It this possible via IMatch?

I've attached the Keyword-Panel (I hope it is the right panel). I haven't changed or added any keywords since importing these 91 files in IMatch.

How may I run the "Metadata-Analyst"?

[JohnZeman]

How may I run the "Metadata-Analyst"?

Menu > Tools > Metadata Analyst

[sinus]

It seems to me, that you have here in these keywords for some keys a space after the |
Like in Natur|Tiere| Vögel

I am at home with an older IMatch. Here I have this behaviour also. If I delete this space and write
Natur|Tiere|Vögel

then it works. I can remember, I had this behaviour some time, but I do not know here on my old computer, if this is still the case with the newest IMatch. 

You can also try to delete all keys in ONE image and then try add a key, then two and so on ... 

Maybe this helps also nothing, but it is worth to try.

[Mario]

No I have not change any "XMP-dc:Subject tag" by myself. It this possible via IMatch?

Yes. But not by accident. Only when a user really needs to change these manually, e.g. to repair some problems caused by other software.

I've attached the Keyword-Panel (I hope it is the right panel). I haven't changed or added any keywords since importing these 91 files in IMatch.

That's the right tool for working with keywords in IMatch. And also the @Keywords Category

But as @sinus pointed out, I see some leading/trailing blanks (Leerzeichen):

Natur| Tiere
Natur|Tiere| Vögel

The entire hierarchy seems to be messed up, and the flat keywords in the file are broken up elements from the hierarchical keywords, with some blanks added? Looks pretty random...

I would expect that a hierarchical keyword like

Natur|Tiere|Vögel|Störche

would be flattened (if at all) as

Natur
Tiere
Vögel
Störche

or, some software out there instead breaks the hierarchy into individual levels, basically exploding the keyword into it's fragments:

Natur
Natur|Tiere
Natur|Tiere|Vögel
Natur|Tiere|Vögel|Störche

but this should be done without adding additional blanks. But in your screen shot and ExifTool output we see these blanks.
Looking at the ExifTool output you've posted, I cannot really tell, if there are extra blanks:

Subject: Architektur, Zoo_und_Tierpark, Wildparadies_Tripsdrill, Natur, Tiere, V├Âgel, St├Ârche, Orte, L├ñnder, Deutschland, Baden-W├╝rttemberg, Cleebronn_Tripsdrill, Wildparadies_Tripsdrill

Hierarchical Subject: Architektur, Architektur|Zoo_und_Tierpark, Architektur|Zoo_und_Tierpark|Wildparadies_Tripsdrill, Natur, Natur|Tiere, Natur|Tiere|V├Âgel, Natur|Tiere|V├Âgel|St├Ârche, Orte, Orte|L├ñnder, Orte|L├ñnder|Deutschland, Orte|L├ñnder|Deutschland|Baden-W├╝rttemberg, Orte|L├ñnder|Deutschland|Baden-W├╝rttemberg|Cleebronn_Tripsdrill, Orte|L├ñnder|Deutschland|Baden-W├╝rttemberg|Cleebronn_Tripsdrill|Wildparadies_Tripsdrill

Did you produce these keywords in an older Lightroom version?
Lightroom had many problems over the years with storing and mapping keywords, but when Adobe fixed them, this did not necessarily fix the keywords in older files.

Please upload one or two of these files to your cloud space (Dropbox, OneDrive, GoogleDrive, ...) and post a link.
Or send a link to support email address, if the images are private.
Include a link back to this topic. I get many emails per day.

I need to see what's actually contained in these files and what ExifTool and IMatch make out of the mess during import.

[OdysseusL]

First of all thank you very much for all your hints and tipps. This feedback, and of course IMatch itself, will be the reason to buy (latetest tomorrow) the software.

Now I have "circumvated the problem": I have converted these 91 RW2 files via Adobe DNG Converter to DNG files, and the problem has vanished (at least for the last hours). I can edit the keywords to pure hierachical keywords and everthing seems to work. I hope this situation will stay stable for these 91 fotos.

I've backuped the 91 RW2 files of course.

@Mario, if you still are interested in a few RW2 files with this "strange behaviour" I may provide a link to them. They are not "private".

[Mario]

If you have a RW2 (and the corresponding XMP sidecar file if there is one), post a link.
This should allow us to determine what the problem is with the file.

Usually, there is nothing special about RW2 or other RAW files, unless one of the known problems: Embedded XMP and sidecar file, mismatching keywords in flat and hierarchical keywords in combination with the user's thesaurus not allowing for properly mapping the flat keywords into his/her hierarchy etc.
All this has been analyzed, discussed here and solved many times over the years.

[OdysseusL]

Hello,

here is a link to 2 of 91 pictures with the descriped behaviour above: https://www.dropbox.com/sh/tb0uxp1no12lbag/AABtkrGOxb1Vl2i5Oz58bDPda?dl=0

I hope it helps. Thanks in advance.

[Mario]

As expected, the problem was easy to identify. The  Metadata Analyst points right at the source of the problem, see screen shot below.

1. Your RW2 files have embedded XMP data, which is very unusual. RAW files should use XMP is sidecar files.
2. Your RW2 files even have embedded legacy IPTC data.
3. There is also a XMP sidecar file.

Typical metadata mess again. Different keywords in different metadata formats, competing XMP metadata.

Solution:
Remove the embedded XMP record via the ExifTool Command Processor (ECP).
Remove the legacy IPTC record also via the ECP unless you really need it, e.g. for clients or your agency.
Then you have only one source of metadata and one set of keywords.

The software that has embedded the XMP record in the RAW and also the legacy IPTC record has only used the name "Ver.2.1", so I cannot tell what it is.
Did you add the XMP data and legacy IPTC data in a software, in the camera, something that your camera vendor provided...?

[OdysseusL]

Hello Mario,

no, I don't have.

Ok, I will follow your described steps to "clarify the situation". Thank you very much!

1) As described in my previous post, I have converted these 91 files from RW2 to DNG. There it was possible to clean the metadata, but no XMP file has been created by IMatch. Does IMacht not create XMP sidecar files for DNG files?

2) Only one question: The XMP data has not been embedded by Adobe Lightroom Classic. But during importing to IMatch (and writing back the metadata, the raw files change date has been changed. Does IMatch write metadata in RAW files, too?

[Mario]

Does IMacht not create XMP sidecar files for DNG files?

No, because by the XMP standard, DNG files use embedded XMP metadata. Same for JPG, PNG, PSD, PSB, GIF PDF, ...

Does IMatch write metadata in RAW files, too?

Of course! The original file may contain EXIF, GPS and legacy IPTC data. XMP contains copies of this data. So, when certain XMP tags are changed, they have to be synchronized with EXIF/GPS/IPTC embedded in the RAW, and written back to the RAW. RAW files are just TIFF files with a different file extension and a proprietary "RAW" matrix "heap".

If all this is new to you, please read Metadata for Beginners in the IMatch help system.