Pushing Metadata from IM5 to Photo Files

StanRohrer · December 03, 2014, 01:27:09 PM

I made a big mistake and wrote identical keywords and description fields to over 1500 photo files thus wiping out many hours of work. I have a very recent Pack N Go backup of the database which I can Restore that would have good data. What commands do I use, or, how do I push this data from IM5 back into the photo files?

A ReScan would take file data and put it into IM5 but this is the wrong flow direction for this problem. Is there some way to push a mass of IM5 fields to photo files with something like the Metadata Write-Back? Or what?

Thanks for your help and suggestions!

Mario · December 03, 2014, 01:41:43 PM

I would try this:

1. Backup all image files so you can rollback.
2. Restore the old database to a new folder.
3. Select all 1500 files you want to change
4. For each metadata tag you want to write back, click the pen icon in the Metadata Panel (the icon next to each tag).
5. This pen marks the tag as "updated" and thus IMatch will write the contents of that tag from the database to the file when you trigger the write.back

Ferdinand · December 03, 2014, 01:42:50 PM

Ouch!!

Another option is to restore the files from a backup and re-read the metadata - can you do this?

StanRohrer · December 03, 2014, 02:21:08 PM

Mario,
It will be 8 hours before I can get back and analyse the full extent of the damage. It was the Pen tool next to the Keywords and Description fields that got me into this jam.

I have been doing many Copy and Paste Attributes. I neglected to bring IPTC data and wanted to fill those fields as IPTC is used at my stock photo agencies and I wanted it complete in my database. I selected all files in a large folder, clicked the Pen icons next to Description and Keywords in the Metadata window. I expected the various data to remain and be written back to the files. However, I think the data of the highlighted file was written to each of the group of selected files. Thus all files now resulted in identical data.

I just did a quick test on another computer with just 2 files and ended up with identical wrong data in the files. Please try this and see if you can confirm. For this test I used "file1" as the keyword and description entry in the first file. Similarly, "file2" was put into the second file keyword and description. Then I selected both files and clicked the pen icons and clicked the check button to process. The descriptions and keywords of both files now matched and were not as original.

Ferdinand,
My file backup is a few days older but certainly is my second choice of recovery. A ReScan would come close to catching up.

Mario · December 03, 2014, 02:27:10 PM

You are right. When you use the pen in the Metadata Panel, you tell IMatch to update all files with the value currently displayed. I just came up with that from the tip of my head. So this approach will not work. You would have to set the pen for each file individually to push the data from the database into the file.

It could be doable with a custom script, which iterates over all files, and sets the tags you want to update by a) fetching their value, b) setting the same value. This should mark the value as updated, and the write-back should push that data into the file.

I'm stressed out and don't have the time to test this or write the script for you. You can start with the "Metadata" sample script which shows how to read/write metadata.

StanRohrer · December 03, 2014, 03:10:05 PM

Doing a test of 3 files "file1", "file2", "file3" seems to show the keywords from all files are combined and saved back to all files. Description from the last clicked (focused) file is written to all files. This is the same test as the above 2 file test.

Mario · December 03, 2014, 03:11:12 PM

Yes. That's what I said. Setting the pen writes the data from the focused file to all selected files. That's the actual purpose of the pen icon - to push the value of the focused file to all selected files. This is why this cannot work for your situation.

StanRohrer · December 03, 2014, 03:19:05 PM

It looks like I will attempt a file recovery and ReScan per Ferdinand. My rusty programming skills are 15 years old and were then aimed at electronic circuit board testing. I don't have experience with the IM scripting and tools. It will likely be faster for me to do a file recovery.

Should I make a Feature Request to have the Metadata Pens become iterative by file instead of flat processing by group? I have not used those pens enough to guess if it would affect other peoples standard work flow. The way it is now burned me pretty bad - but just because I went on a large scale update mission as my first attempt at use.

StanRohrer · December 03, 2014, 03:21:22 PM

Quote from: Mario on December 03, 2014, 03:11:12 PM
Yes. That's what I said. Setting the pen writes the data from the focused file to all selected files. That's the actual purpose of the pen icon - to push the value of the focused file to all selected files. This is why this cannot work for your situation.

Not quite. Keywords from all files are combined and written back to all files. Keywords are not just from the focused file.

Mario · December 03, 2014, 06:41:31 PM

I suggest you file that as a bug then. I'm not sure right now how this is supposed to work for repeatable tags like keywords, if this should be a replace or a merge. And I will not have time to look into this for a while, so better add a bug report.

Ferdinand · December 03, 2014, 11:56:42 PM

I have a feeling I fell into the same trap at some point, on a smaller scale fortunately. That pen icon in the metadaa panel is a little dangerous.

I have been thinking about this as well. Have you tried to restore the database from another machine using P&G? Because, assuming that automatic writeback is off, once you've done this IMatch should see a discrepancy between what's in the database for these files and what's in the files and sidecars. It should regard what is in the database as having precedence, unless you do a forced rescan of the metadata, which clearly you won't. Given this, it should set writeback for these 1500 files. This should solve your problem. I can't be sure about this, but it's worth a try. I see that Mario suggested that a script might be necessary to get the writeback status set, so perhaps I am not correct in suggesting this.

If you tell me which fields are affected, I might be able to write a simple script, if it proves to be the case that such a script is necessary, although time is a little tight at present, so I can't make any guarantees.

Another idea - would a metadata template that sets the same value in the field in question set the writeback flag?

Failing that there is my previous suggestion.

StanRohrer · December 04, 2014, 12:47:50 AM

Ferdinand,
The fields are Description and Keywords. Images attached with the path ID's.

Is Immediate Write Back saved in the database file? I don't know if I can open the good database and disable the setting and not have it update. I might try.

For Keywords I think I could add something like "TEST1" and IM knows how to add this and not scramble the keywords among files (a awesome feature that generally works right!). Then come back with another pass to remove TEST1 from all files. I have tested and think that would work.

However in my testing the Description field does not work like this and would make all entries the same when working them as a group selection. So I am still stuck on this field. Any effort I can think of to push this field back to the files has to be on an individual file by file basis. Hence the need of a script.

I've just started a multi-hour task of pulling the affected files out of my backup system. The actual count is 1629 files across 5 folders.

[attachment deleted by admin]

StanRohrer · December 04, 2014, 01:39:49 AM

Bug Report entered here:
https://www.photools.com/community/index.php?topic=3851.msg25750#msg25750

StanRohrer · December 04, 2014, 02:29:00 AM

Ferdinand,
Thanks for the thoughts of creating a script. At this point I have restored the image files from my recent backup and started a Rescan. So I think a script may not be needed. I may have backed up my processing by a couple of hours but not to bad.

Now to decide if I should go back to the backup version of the IM database and see if I can recover some keyword and description processing from after the file backup. I'm concerned I'll hit the same roadblock - how do I push data from IM back into files in a reasonable fashion. (My files backup is older than my IM backup).

This is just another lesson in keeping database and file backups current. I'll give myself an B+ grade. Not a seamless recovery but reasonably effective. I probably lost an evenings worth of work and an an evenings worth of recovery at this point.

Ferdinand · December 04, 2014, 10:03:19 AM

Best of luck

StanRohrer · December 06, 2014, 08:02:50 PM

I have recovered from my disaster here. Thanks for the luck and support.

After recovery I had about a dozen files still missing IPTC Caption-Abstract. At this level I did file updates one file at a time. I added and the removed an extra period at the end of the text so the field would be processed and pushed to the file.

I have opened a Feature Request here:
https://www.photools.com/community/index.php?topic=3862.msg25858#msg25858