Long delay in Collections & Categories views

[DigPeter]

What is going on here?  Always after starting IM there is a long delay when I go to Collections & Categories views;   the blue busy circle rotates for a long time.  Attached is a screen grab and the logfile.

[attachment deleted by admin]

[Carlo Didier]

Same problem here, but I supposed that my calculated categories (with regular expressions) were the cause

[DigPeter]

Same problem here, but I supposed that my calculated categories (with regular expressions) were the cause

Yes - I have a few of those, but not many.  That should not affect collections? 

[Mario]

Your database has 42,000 files and about 5000 categories.

Does this happen every time you switch between the M & F View and the Categories View, or only once after startup?
Because with 5000 categories, and many data-driven and formula based, IMatch will need some time to calculate every category.
Do you display category counts in the Category View? Then IMatch needs to calculate all visible categories in the tree, not just the selected one.
If you use many formulas, data-driven categories etc. this may take a short while when nothing is yet in the cache.

The database loads in 13 seconds and the load is finished at 12:03:31.
On 12:03:32 IMatch loads a file window for a category, which takes only about 2 seconds.
Nothing much happens till 12:04:17.

Then you switch to the Collection View.
Then IMatch starts calculation all collections, which is done in about one second.
A File window is loaded for 214 files. 12:04:20

12:04:45, another file window is loaded, 39 files. 0.5 seconds total processing time.

IMatch is now most idle, except until 12:07:29, when another file window is loaded.

12:04:33 Another file window is loaded. Takes about 0.5 seconds.

Except for loading the database, the log file reports no "slow" operations, no database locks or extended waits. Looks normal to me.

[jeknepley]

FWIW, not a problem here. Maybe 3-4 seconds upon opening Collections for the first time during a session; the same or less for Categories.

The data base is on the large side (255K files, 8K folders, 8GB, 2550 categories ) so it's unlikely that size is the chief source of the problem.

My primary laptop is on the "fast-side" so that may contribute to my results. 32GB RAM; DB is on C:/ which is a SSD; files are on a USB3 connected hard drive and I run the IM5 DB optimize routine regularly. The PC is the ASUS Republic of Gamers top of the line model.

The identical DB (synced to the aforementioned one) runs on a 4-5 year old Dell Studio 1747 laptop - 8GB RAM; DB is on C:/ which is a SSD; files are on a USB2 connected hard drive. I just now started IM5 and it immediately began Reading Metadata and Adding/ updating some files. I clicked on Collections while this was going on and the response was almost instantaneous. Categories took 10-15 seconds (the screen turned white??) but then everything worked as expected.

Hope this helps illustrate that short delays are possible.

[DigPeter]

Thanks for this analysis, Mario.

Your database has 42,000 files and about 5000 categories.

Does this happen every time you switch between the M & F View and the Categories View, or only once after startup?
Because with 5000 categories, and many data-driven and formula based, IMatch will need some time to calculate every category.

No, but quite often.  This particular post is mainly concerned with switching to Collections view.

Do you display category counts in the Category View? Then IMatch needs to calculate all visible categories in the tree, not just the selected one.
If you use many formulas, data-driven categories etc. this may take a short while when nothing is yet in the cache.

Yes

12:04:45, another file window is loaded, 39 files. 0.5 seconds total processing time.

IMatch is now most idle, except until 12:07:29, when another file window is loaded.

This is the long period I referred to, after I had switched from green to blue dot.  You say IM was idle, but the blue busy circle was active the whole time.

[Mario]

You are getting more precise.

The problem is the collections view.
The delay happens when you switch between one dot collection and another.

Do you notice high CPU activity when this happens (keep Windows Task Manager open).
What happens when you disable counts in the Collections View (Gear Icon)?
Do you have many collection nodes expanded?
Do you have a filter panel with collections open?
Do you use variables in the file window which refer to collections? Or categories with formulas which refer to collections?
Do you have the category panel open?

The delay is most likely caused by IMatch needing to re-calculate the collection contents. From what I can see in the log file, no operation takes longer than a few milliseconds, and the file window is loaded in 0.5 to 1.5 seconds every time.

[DigPeter]

Do you notice high CPU activity when this happens (keep Windows Task Manager open).

I have just run IM and am not now getting delays on initially switching. on swithching activity peaks at around 60%.

What happens when you disable counts in the Collections View (Gear Icon)?

Peaks at about 40%

Do you have many collection nodes expanded?

Dots  pins only

Do you have a filter panel with collections open?

Not initially.  I use filters intermittently.

Do you use variables in the file window which refer to collections? Or categories with formulas which refer to collections?

No

Do you have the category panel open?

No