IMatch stoppt import

[RobiWan]

Hi,

I'm trying to import some directories to my "travel" database, but IMatch stoppt allways the impoort (hothing happens anymore) and I can't see whats wrong here.

Cheers Robert

[Mario]

Indexing seems to be extremely slow.

IMatch starts indexing files at 10.22 16:24:25 in the "C:\ExtHDD\ExtHDD 2508\robert\Fotografie\DSLR\ExtHDD 2508\EOS R6 Mark II\RAW\2023\05\06\" folder. WIC takes about 300 milliseconds to load a file. 20 ms for the check sum.
ExifTool takes 10 seconds to extract metadata from sets of 5 files. Almost 700 tags are imported for each file.

10.22 16:27:19, IMatch loads "C:\ExtHDD\ExtHDD 2508\robert\Fotografie\DSLR\ExtHDD 2508\EOS R6 Mark II\RAW\2023\05\06\_rr60040.cr3" in 156 ms. Checksum in 0ms.

10.22 16:27:20: IMatch processes "C:\ExtHDD\ExtHDD 2508\robert\Fotografie\DSLR\ExtHDD 2508\EOS R6 Mark II\RAW\2023\05\06\_rr60181.cr3". Now checksum takes almost 8 seconds (!)
For C:\ExtHDD\ExtHDD 2508\robert\Fotografie\DSLR\ExtHDD 2508\EOS R6 Mark II\RAW\2023\05\06\_rr60201.cr3 it takes almost 9 seconds.

To calculate the checksum, IMatch reads the entire image once. This is usually fast. And at the beginning, IMatch does this in 20 to 30 ms for the files mentioned. Suddenly, this drops to 7 to 8 seconds, 20 times longer.

Now, WIC also starts to slow down drastically. Instead of 200 ms per file, WIC now needs 30 seconds (!) per file.

This looks as something external is making the C: disk very busy, dropping performance for IMatch.

Look at Windows Task Manager to check on disk utilization and on the "Details" tab show the writes/reads per second to identify which applications are reading and writing lots of data.

And, reboot once. This often helps.

[RobiWan]

Thank you for your quick reply. After all, it was Sunday.
Well, I almost thought it was something with WIC again. I know that's useless now, for me the WIC story is always the point of negative attention in IMatch.
As soon as there are too many images in a directory, the Windows WIC eventually fails. Always supposedly 100 images before the end. Disk activity goes to 0 and it doesn't happen anymore. IMatch can then also no longer be terminated.
With a change to IMatch internal RAW Processor it seems to run at the moment.

Robert

[Mario]

As soon as there are too many images in a directory, the Windows WIC eventually fails. Always supposedly 100 images before the end.

Interesting. I have never experienced such a thing.
When I finish a shoot I usually come home with 2,000 or more NEF RAW files from a weekend. I start IMatch and push the images from the cards to my incoming folder. IMatch indexes the files without any problem.

I've just re-indexed my Canon RAW test file library (about 1,200 CRW, CR2 and CR3 files from a wide range of Canon models). All files were added to a new database without a problem using the standard Windows 11 WIC codecs.
I deleted the database, created it new, switched IMatch to LibRaw and repeated the test. LibRaw failed to process one CRW file, but forcing a rescan then worked.

What kind of computer do you use? An older system perhaps or maybe a notebook / laptop?
From experience, when something fails under heavy and prolonged load (like IMatch indexing RAW files), maybe some system instability / overload is the reason for the problem.

If you can reproduce the problem by indexing more than 100 RAW files using WIC, we can easily test that.

1. Open Edit menu > Preferences > Application.
2. Disable photools.com RAW processing
3. Search for process and set all four values to 2 (see Process Control (Advanced Setting))
4. Create a new database via Database menu > New. Use your fastest disk (SSD) for the database.
5. Drag a folder containing several hundred of your CR3 files into the Media & Folders View to index them

If the problem is some instability under load, using less parallel threads (2, in this case) should solve this and all files should be added without problems.

IMatch by default tries to utilize all processors in the system to be quick. And that for a loooong time, depending on how many images are read. Especially when automatic face recognition is enabled. And disk utilization will also go up, for the disk containing the database and for the disk containing the images.

[RobiWan]

Me too

without a problem using the standard Windows 11 WIC codecs.

Some moths ago I have had a problem with Windows WIC because IMatch RAM consumption. Link
I believe here is something like this - some Microsoft update or so.

What kind of computer do you use?

My desktop is with AMD Ryzen 9 3900X 12-Core, 64 GB RAM, My Laptop is with i7 - 13700H and also 64 GB RAM too. Both machines never have any issues with load at all.

With IMatch RAW processor I have no issues at the moment.

Robert

[Mario]

If one of you have a free CR3 files to share which cause this problem, send them to support email address or upload them somewhere and send me a link.

I would like to see this error in action and maybe figure out how to prevent it or detect it and inform the user somehow. Via a banner or the Dashboard perhaps.