Need for second writeback for some file!

[KimAbel]

Hello

I just did a rescan of all my folders and files (76528 files)

Then a lot of files is showing up with a need for metadata writeback. The field that shows up as edited is xmp::dc/subject. Why does these files show up as edited (they are originally imported by IM5)? I havent done anything with the them in IM5.

Another case is when I do an edit of metadata then for some files this field also shows up as edited after a metadata writeback. So for some files I have to excecute metadata writeback two times in order to complete. Why?

Is it files with IPTC that shows up a second time with dc/subject as edited?

My metadata settings is the standard one in IM5. I use Lightroom for editing raws, but apart from labels and ratings I do not edit metadata in Lightroom.

Regards Kim Abel

[KimAbel]

Not much response on my initial post..

The following question solves only part of my initial post. I still dont know why some files need two metadata writebacks and why so many files shows up with an edited xmp::dc/subject after my rescan (not just rawfiles).

After my rescan of all my folders I have ended up with a few thousand files which needed a metadata writeback. After excecuting a metadata writeback on these files I end up with a little over 1300 files which fails in doing this. After I have excecuted the writeback they disappears from the collection "Pending metadata writeback", but then IM5 is doing "reading metadata" and "adding files and folders" in batches of 5-25 files and they slowly appears one more time in the collection "Pending metadata writeback". I have then discovered that these files (all raw files) have embedded iptc metadata (in addition to my xmp files that I have set IM5 to use). I am trying to use ExifTool command processor in IM5 to delete all the Iptc and xmp information in my rawfiles, but I get a strange result! I use the commands:

-overwrite_original
-Iptc:all=
{Files}

and

-overwrite_original
-xmp:all=
{Files}

For some of my files I succeed, but for some files I get a result with a deleted raw file (marked as offline in IM5) and a backup file of the raw with a different name (which are imported into IM5).

When I am doing the same in ExifToolGUI it all works fine. No deleted raw files and the yellow pencil disappears for good.

What is different in ExifTool Command processor in IM5 and the same operation in ExifToolGUI (I use "modify/remove metadata/xmp:all and Iptc:all)?

Its much easier to do the erase command in IM5 because the files that I need to edit is on many different folders, and in the ExifToolGUI I have to navigate to each folder in order to erase xmp and Iptc. With IM5 I have them all gathered in one collection.

Kim Abel

[DigPeter]

FWIW see https://www.photools.com/community/index.php?topic=1409.msg8461#msg8461 .  I have not attempted to use exif-tool, so cannot help there, but I and others have had the same problem with having to write back the same files more than once before the dreaded YP disappears.

[KimAbel]

Thanks DigPeter for your link :-)

I have looked at your linked threads, but cant find any similarities with my case. All my files that shows up a second time with the yellow pencil is supported by IM5, and are image files. So why are these files having this behaviour?

The files where I cant get rid of the yellow pencil with the second writeback is files with conflicting metadata (embedded and sidecar). These files can be fixed, but the ExifTool command processor in IM5 behaves not in the same way as ExifToolGUI!

Kim abel

[joel23]

[...]
For some of my files I succeed, but for some files I get a result with a deleted raw file (marked as offline in IM5) and a backup file of the raw with a different name (which are imported into IM5).

When I am doing the same in ExifToolGUI it all works fine. No deleted raw files and the yellow pencil disappears for good.

What is different in ExifTool Command processor in IM5 and the same operation in ExifToolGUI (I use "modify/remove metadata/xmp:all and Iptc:all)?

Kim Abel

The difference is that - depending on your metadata settings - IMatch uses (IMHO most of the time unnecessary) so called arg files for ExifTool on read/write. But for telling for sure I need to know your metadata settings for the RAW.

Sometimes Exiftool launched by IMatch can't process the file in time. In that case the original file already is deleted but the tmp-files can't be renamed back to the original file name. Which leads to missing files marked as offline and an import of the tmp-files.
ps

The files where I cant get rid of the yellow pencil with the second writeback is files with conflicting metadata (embedded and sidecar).

This additionally can be an issue. ExifToolGui only works on the file marked and ignore side cars if they are not marked. IMatch might not.

[KimAbel]

Thanks very much jJoel23

My settings in Metadata 2 is the default choice. I have done nothing to change them.

This additionally can be an issue. ExifToolGui only works on the file marked and ignore side cars if they are not marked. IMatch might not.

In IM5 only the raw are affected, not the sidecar. The values are still in the sidecars after I have deleted Iptc and xmp from the raw.

Is there a way to make IM5 work the same way as ExifToolGUI? That way I dont have to navigate to hundreds of folders to delete embedded metadata.

Kim Abel

[joel23]

Thanks very much jJoel23

My settings in Metadata 2 is the default choice. I have done nothing to change them.

Okay. I might have missed it - but about which RAW files we are talking about?

Is there a way to make IM5 work the same way as ExifToolGUI? That way I dont have to navigate to hundreds of folders to delete embedded metadata.

I believe not. You never know what a programer has implemented to his code for launching a secondary application (here ExifTool) - aside the statements used we can see within the arg files.

Maybe a script can help here. There is a script for Geosetter which launches Geosetter on the marked files only, regardless where they are (in which folders they are)
Maybe something else is possible with ExifToolGui (I doubt it) but it should be possible with Exiftool.exe on command line.

Something like: write a file with all the marked files to disk, run Exiftool with "for all %n in %filename do -overwrite_original -Iptc:all= -xmp:all="

[KimAbel]

Did you forget to type something in the last reply Joerg :-) I can only see a full quote of the former post.

Something like: write a temporary file with all the marked image files to disk, run Exiftool with "for all %n in %filename do -overwrite_original -Iptc:all= -xmp:all="

Thats the nice thing with ExifToolGUI. I dont have to understand the command line :-) Im not sure how to get from my collection in IM5 to exiftool.exe! The best thing would be:

Sometimes Exiftool launched by IMatch can't process the file in time. In that case the original file already is deleted but the tmp-files can't be renamed back to the original file name. Which leads to missing files marked as offline and an import of the tmp-files.

If this is the case I would think that Mario has to change how ExifTool is working in IM5. For instance increase the handling time for each file. Im not sure if this is possible, but I hope that it can be done :-) I guess that both Mario and the users that uses ExifTool in IM5 dont want files to be renamed after excecuting ExifTool.

Kim Abel

[joel23]

Did you forget to type something in the last reply Joerg :-) I can only see a full quote of the former post.

LOL. I don't know what happened. There was nothing to say anymore. But still: what kind of RAW files we are talking about and how large they are?

If this is the case I would think that Mario has to change how ExifTool is working in IM5. For instance increase the handling time for each file. Im not sure if this is possible, but I hope that it can be done :-) I guess that both Mario and the users that uses ExifTool in IM5 dont want files to be renamed after excecuting ExifTool.

Kim Abel

That is the way ExifTool works - always and everywhere, even when working with the GUI. It applies changes by writing them to a tmp-file, deletes the old file and renames the tmp-file back to the original file name.
Usually it does not happen, that the old file is deleted and the tmp left alone; seems here the tmp-file still was blocked what makes it impossible for ExifTool to rename it.

[KimAbel]

what kind of RAW files we are talking about and how large they are?

Forgot that :-) My raw files are Cr2. Most of them are in the range 6-15 Mb, but some are 25-30 Mb. Its from several different Canon cameras over the years.

One thing that is different between ExifTool in IM5 and ExifToolGUI is that in GUI I have set it not to make backup copies of the file.

The command -overwrite_original_in_place and checking the box "Run for each file in selection" seem to do the trick in IM5

I will test this more.

Thanks for guiding me in the right direction :-)

Kim Abel

[joel23]

Ah yes,
there is the difference ;-)

-overwrite_original                  Overwrite original by renaming tmp file
-overwrite_original_in_place     Overwrite original by copying tmp file