Unexplained delays while viewing files

[ubacher]

First let me report that the problem with "illegal argument" and associated closing of Imatch has been
solved with the newest version of Imatch.


Today I experienced long delays while switching images in the viewer. I looked at the log file to see
if I could find the reason.

Code Select Expand

07:01:37+59484 [0CEC] 05  M> >  0 CIMDXWnd::LoadFile  'V:\develop\IMatch5\src\IMatchNG\IMDXWnd.cpp(791)'I noticed that the same file is shown as being loaded multiple times??? Is that kosher?

The same at 07:01:57 and 07:02:18.
At 07:04:10 I switched to debug logging.

A long delay - different reason -at:

Code Select Expand

07:05:13+23657 [0CEC] 10  M> >  0 CIMEventManager::GetFileEvents  'V:\develop\IMatch5\src\IMEngine\IMEventManager.cpp(2267)and again at 07:05:49
Note that I have only 5 events in my db.


The following files (304 to 308) I switched back and forth in the viewer - so this is not unusual.

Later on in the log file I was processing files in the batch processor - and there were no unexplained delays.

Logfile attached.

[Mario]

I see a ton of low-memory warnings.
Could be the reason that Windows WIC is underperforming.
Reboot your system. Retry. Don't open too many applications at the same time.
Your computer has 32 GB of memory, yet only 8GB are available when IMatch starts!

[ubacher]

Sorry - I did notice that Imatch used 5GB when I experienced the delay - which I found strange , but I forgot to mention this.

Of course PS is the memory hog.
So would the delays be caused by windows swapping in/out memory even though there were many GB still available?

[Mario]

IMatch itself will never use 5GB of RAM for a tiny database with only 14K files.

But I see that you have used the Viewer, where IMatch uses memory to cache files in advance in memory.
The Viewer is allowed to consume up to 80% of the available memory. Which is of course not really helpful when 70% of the available memory is already used up when IMatch starts.

IMatch uses Windows WIC to load the JPG files you view in the Viewer. The memory consumed by WIC is counted towards the memory working set of IMatch. This is the last memory log I see.  WIC is known to use excessive amounts of memory when it deals with corrupted or "wrongly" formatted files. But with JPG images, this is pretty rare. But who knows, maybe WIC bugged out and took several GB to load one JPG file? Impossible to test from here.

You then close the Viewer, categories are recalculated, file histories are updated.
Then the Viewer opens again, some files are viewed
Then categories and events are updated.
Batch Processor runs.
Metadata is propagated or manually copied.

The database is loaded in 5 seconds, but initializing the UI takes over 10 seconds more. Which seems a bit on the slow side.

The database does not seem to perform well. For a 14K database I don't expect to see as many delayed transactions where the database system has to wait for the disk. What kind of drive is D: ?

The results of that are many warnings about "slow" categories. Because recalculating categories is very database-intensive, and if the database does not perform well, everything will be slow.

Categories like 'STYLE' or 'TYPE OF FOTO' are reported as slow. Also the large "IMatch Workflow Categories".

I also see this:

[33968ms #sl] CIMTaskGroup::Assign

33 seconds for assigning files to a category? I don't see such slow operations on databases with a million files.

This then results in logs like

CIMCollection::InnerCalculate: Failed to get transaction or CS lock for 'Today'

where IMatch has given up recalculating the collection today since the database was too slow to respond.
For a database with only 14K! Which is nothing.

Make sure you store your database on your fastest disk. If D: is your fastest disk, make sure your virus checker is not interfering and that the D: disk is not maxed out by other applications.

[ubacher]

IMatch itself will never use 5GB of RAM for a tiny database with only 14K files.

The Task Manager showed this amount under Imatch.
(Aside: I also noticed that having the Recycle Bin open requires a lot of memory - 5GB im my case.)

What kind of drive is D: ?

It is an SSD (M2).

33 seconds for assigning files to a category?

I have the same system for many years now - nothing changed (but Imatch version); This is why I noticed the unusual delays I experience. A wild guess is that the program has to wait until a stuck sub-process is released.

[Mario]

If a WIC codec allocates 4GB when dealing with a bad image, Windows will consider this memory as being used by IMatch - since IMatch has loaded the WIC codec.

33 seconds to add some files to a category for a database with only 14K files is insane. Unless some other software utilizes the D: disk all the time. Is Windows installed on D: ? Then it could be thrashing caused by low mem.

Make sure you have made an exclusion for folder containing the database in your virus checker, and maybe also for IMatch.
Reboot your PC and stop all unneeded background processed (especially stuff from Adobe).
Then retry.

Maybe move the database from D: to C: and see if this changes anything.