Very slow write back and strange numbers

[Mees Dekker]

I'm currently doing a lot of write-back actions, since I added copyright etc to my files (via a metadata panel).

I noticed that writing back these metadata takes for ages. And secondly the numbers don't match (see screenshot). When I filter on "files with unwritten metadata: top righthand corner) I get a selection of 65 files. These are all shown. I then select them all: again 65 files. So far, so good.

But when I hit the pencil icon on one of the files: a window comes up and states that IMatch is working on 167 files and slows down to an absolute crawl. Hitting the cancel button does not cancel the process. Had to kill Imatch through taskmanager. Debug log is attached.

Any clue, since this is rather new, since a week or so. Before that I could easily select 2000 files and write back them  all in a rather smooth scheme. 

[akirot]

I can (must) confirm the "strange" numbers as described.
(As IMatch did its job I ignored that and considered it as a minor glitch.)

[Mario]

Do you use versioning?
Writing masters must also write back versions.

"Slow" can mean many things. Which timing information does IMatch log in the log file? It records how long each write-back takes.

[mopperle]

Not using versioning and can not confirm the behaviour above.
I recently changed a lot of data in my database, selecting always around up to 200 files and let IM write back. Shown number was always correct and it was never "slow" for me.

[Mees Dekker]

Do you use versioning?
Writing masters must also write back versions.

"Slow" can mean many things. Which timing information does IMatch log in the log file? It records how long each write-back takes.

The logfile was attached. Each writeback (CR2) takes about 40 sec and yes: I do use versioning.

[Mario]

and yes: I do use versioning.

This explains the difference in numbers then. You write-back n masters but that may require n*2 or similar numbers of actual files to write, depending on how many versions.
Write the master. Copy metadata from master to version(s). Re-read the master and modified versions. Repeat for each master.

Didn't see the log, sorry.

First thing I see is an ExifTool warning Temporary file already exists: P:/Foto's/Klanten/Markenheem/2022/Gebouw F/4 Wassinkbrink/Wassinkbrink (003).psd_exiftool_tmp.
This means that ExifTool already tried to write to the file, but somehow did not complete (crashed?), leaving a temporary file behind. Unusual, but might happen with corrupted files or when a virus checker kicks in and blocks ExifTool in mid-write. Hard to tell.

Delete these temporary files in Windows Explorer before you try to write-back again.

Then WriteBack successful for file [256990] 'P:\Foto's\Klanten\Markenheem\2022\Gebouw F\3 't Zand\'t Zand blokhut (027).CR2' in 40 s
and WriteBack successful for file [258722] 'P:\Foto's\Klanten\Markenheem\2022\Gebouw F\3 't Zand\'t Zand zandweg (007).CR2' in 5 s

Why does the one file need 40 seconds and the other 5? Do they differ?
Maybe some previous write-backs timed out after 120 seconds and this is why there are left-over temporary files?

What kind of drive is P:?

Two things I would check:

0. Make sure your virus checker is not interfering, blocking ExifTool etc.

1. Try to reduce the number of parallel write-backs via Edit > Preferences > Application: Threads for Write-back. Maybe set this to two.
If your system does not cope with too many concurrent write-backs for some reason, this should make a difference.

2. When you copy these two files from P: to a folder on your SSD and write-back there, does ExifTool perform faster or the same? Just to rule out that the P: drive is the culprit..

[Mees Dekker]

P: is a folder (and subfolders)  on my Synology NAS. It is excluded from scans by Windows Defender, as are IMatch and Exiftool (and of course the folder on my internal SSD drive in which the actual database resides).

I did investigate a bit further and it seems that there are 3 explanations for the the slow performance:

• I use versioning (psd, psb and jpg are versions and buddies of the original CR2 file). It seems as if Exiftool writes metadata to the the master, propagates them to the versions, writes metatdata to the versions, but does that at least 2 times for each jpg version. Some kind of loop?
• psd files take a lot of time to write metadata, can't explain why
• I had some very big files (over 1,4 Gb each) and combined with the point above it took forever, possibly resulting in time-outs for Exiftool and freezing of IMatch

[Mario]

NAS => Slowest possible case (worst-case scenario), even when you run a high-end NAS and a 10GB wired network. Lots of data to transfer.
Can be 1000 times slower than doing the same on a SSD in your PC.

writes metatdata to the versions, but does that al least 2 times for each jpg version. Some kind of loop?

This sounds like you have to masters in the chain. Maybe something like CR2 and PSD being masters?
Writing back a master causes a write-back to all its versions. And when one of these version is also a master, writing it back triggers a write-back to its versions...maybe this is the case here.

I had some very big files (over 1,4 Gb each)

Writing back such large files on your NAS requires ExifTool to download the file over the network, make the required changes, writing the file back over the network.
A 1GB network transfers maybe 90MB per second, so only downloading the file will take 15 seconds in the best case. And when IMatch writes-back multiple files in parallel (the default), the net bandwidth is divided between 4, 8, ... parallel write backs, meaning each download takes 4 or 8 times as long as when only one file is processed at the time.

I recommend you "finish" these files on your local PC before moving then for long-term storage to your NAS.