Metadata write back speed

[dp7602]

Hi !
In relation with the «Question about restoring with Pack&Go» which is solved, here are the logs after : 
1. writing back metadata in 5 TIFF files (from film scanning, so approx. 130 Mo each).
2. writing back metadata in 5 CR2 files (approx. 11 Mo each).

I use an AMD Ryzen 5 2600 processor (6 cores) and the files are on a USB 3 hard disk.
I know that performing this on a USB drive is not the best in terms of speed but It is convenient for me since I can easily access the pictures using a laptop while on the go.

First impression : slow with TIFF files which are very large and quite speedy with CR2 files.
Regardless the actual location of the files (hard drive, USB Hard drive, SSD), it seems that the size of files matters a lot.

Thanks for helping.

Pierre

[Mario]

Yes, of course the file size matters a lot.
If ExifTool finds no "room" in a TIFF file, it has to split the file into segments to allocate more space for the metadata. At that time it allocates a bit extra space for future updates.
This also depends on how the TIFF file was generated (some applications allocate some extra space for metadata updates, some don't), the size of the TIFF file (50 MB or 500 MB) etc.

Both log files just show an IMatch start. No information about even a single file written back.
Remember than you must copy the log file before restarting IMatch. Use Help > Support > Copy Log file after writing a few files.

[Aubrey]

Yes I have noticed that writing Metadata to large Tiff files generated after digitizing (Epson v 500). Take quite a while.

There was a discussion on threads for writing back Metadata to these large files and possible pitfalls in the thread:
https://www.photools.com/community/index.php?topic=8492.msg59745#msg59745

Aubrey

[Mario]

For reference, on my system ExifTool/IMatch needs ~5s to write back to a 600MB (!) TIFF file with 14,000 x 19,000 pixel.
I've added rating, description, keywords, author info. The following re-index / import takes 1.4s, with 300 tags read.

Round that up to 7-8 seconds to write-back and re-import a 600 MB TIFF file.
And IMatch does this for several files in parallel (depending on the core count).

Photoshop takes about 5-6 seconds to save the file from memory to disk.

[dp7602]

Hi !

Here is a log file after writing back metadata in 5 CR2 and 5 TIFF files.

Thanks,

Pierre

[Mario]

This log file just shows IMatch starting up. No files were read or written.
Remember to use "Help menu > Support > Copy Log File" after you have written your files.

[dp7602]

Is this file OK ?

Pierre

[Mario]

No write-back logged.
You exported text from the thesaurus?

Make sure you enable debug logging under Help menu > Support.
But write backs are logged always.

[dp7602]

Hi Mario,

That's right, I exported the thesaurus as a txt file to reorganize it.

Regarding the log file, am I right saying  :

1. perform the write back
2. copy Log file by "Help menu > Support > Copy Log File"

What else ?

This is what I did.
I'm rather puzzled by the fact that you don't see the metadata write back in the log file.
Maybe enable the debug logging.

Regards,
Pierre

[Mario]

This log shows IMatch starting and loading a database.
Then text is exported from the thesaurus.
Then I see a warning from ExifTool about an image file with an invalid URI in the MicrosoftPhoto section.
This is not uncommon, because several Microsoft applications messed their own metadata up and Microsoft never fixed it before discontinuing it.

I see no write back. Make sure you enable debug logging via Help > Support. Then change data in 10 or 20 files (a mix of your typical files) and click the pen icon.
You can search the log for the word WriteBack afterwards to see what was logged.

[dp7602]

Hi Mario !

I did as you said. This time there are some writeback events in the log file : 

For 5 TIFF (130 Mo each) and 5 CR2 files.
Approx. 1 minute for each TIFF file and 12 seconds for each CR2 file.

I used my laptop (A I5 6200U microprocessor with 2 cores and 4 threads) and the files are on a USB3 hard disk.

Thanks for helping.

Pierre

[Mario]

If it does not get faster the second time you write back to the same files, the culprit is most likely either the system or the USB disk.
ExifTool may need to make room for new data during the first write, requiring the file to be spliced and re-combined, which costs a lot of time.
A 600 MB TIFF on my system takes only 5 seconds to write-back (SSD, not external UBS disks). USB disks usually take twice as long.

If you write back to a TIFF image of the same size on your SSD, how long does that take?

You may also try to reduce the number of parallel threads IMatch uses (Process Control (Advanced Setting)) to 1 or 2, this may improve overall performance if you system is struggling with too many parallel operations.
You can test write back for one TIFF file before to see if this is proportional faster.-

Finally, your virus checker may slow down IMatch considerably if it interferes with every disk I/O operation. Did you make IMatch an exception in your virus checker?