#SLOWCAT: how to turn these cats to manual update?

[Jingo]

Howdy... so, looks like I have 2 categories that have the #SLOWCAT in my log.. 5 seconds for one and 6 seconds for another.  So, I figured, I would just turn these to manual update since I hardly ever use them.

I chose the category (No Title), right clicked and chose "Data-Driven Properties" and noticed the auto-update was set to "ON"




So, I chose OFF like the description box says to do.. but the OK button remains low lit.



So, I checked the help and it says that these values may be controlled by a global parameter on the Background Processing Tab.



So, I went to the settings -> Bkg Processing Tab.. but don't see a unique setting for Data-Driven Processing:



Is it controlled by the Indexing parameter?  Thoughts?

[Mario]

Enable the level. Enabled is still set to "No".

[Jingo]

Yikes.. .how did I miss that!

But - if the level is not enabled (and I have 0 active levels) - then why is the data-driven category being auto-updated at all?  Shouldn't 0 active levels render the category inactive?

Sorry for the silly question - I don't use categories at all and guess I should brush up on my IMatch knowledge of data-driven cats!

[Mario]

It might be scheduled for update, but the update should run in no time, without any enabled levels
But you cannot close the data-driven setup dialog without having at least one level enabled, so a zero active level data-driven category cannot happen.

[Jingo]

Not sure why my log is showing SLOWCATS for these then... I found an old string from a few years ago (https://www.photools.com/community/index.php/topic,10273.msg73431.html)  where I discussed the same exact thing in my log so it has been happening for awhile now...  it continues to cause the screen to "freeze" and display "wonky" before unfreezing and snapping the resolution back again. 

I've attached the past days worth of Debug log in case this is something you have a moment to review.  Nothing major but really more curious than anything.

Thx! - Andy.

[Mario]

The log file is missing the header so I cannot tell how large your database is or which IMatch version you are using.

Many operations are logged as "slow", because they take longer than 5 seconds.
This does not necessarily indicate a problem, it just might be that your database is huge or your computer just takes longer to process things (longer than 2s, in this case).

Database-bound steps like updating data-driven categories (e.g. for standard categories like "Make and Model" or "No Title") take between 4 and 8 seconds. The only #slowcat entries in this log are for  'No Title' (7 seconds) and 'No Copyright' (4 seconds).

If you don't work with these categories, collapse their parent category "IMatch Standard Categories" and make sure that parent has the "Direct Assignments only" enabled (which is the default). IMatch then does not have to re-calculate them all the time.

Category updates run in separate background processes and should not interfere with the user interface.
If this happens again, make a note of the time (with seconds) and use the Help menu > Copy Logfile command to store a snapshot of the log. Near the end of the log we might see what's causing the issue.

If the database is on your fastest disk (SSD), check the log of your virus checker to see if it scans the database on reach access. This can bring down performance real bad. Make sure there is an exception for the folder containing the database.

[Jingo]

Thx Mario - latest version of IMatch (just updated this morning actually) and database contains 89000 JPG images.  Database itself is 4GB according to the dashboard.

I do have the Direct Assignments only enabled so figured they wouldn't get calculated at all.. but they still show in the log all the time.  I just collapsed the parent in the categories tree so I'll see if that makes a difference.  Will keep an eye on things and be sure to share the entire log when it happens again... I didn't include the log earlier because it was over 6MB with debug logging enabled.

Only using standard windows virus scanner so no interference from that.

Thx!

[Mario]

If the category is visible somewhere (Category View, Category Panel, Category Filter) it will be calculated in order to show the counts (number of files). The results are cached, but if you perform some task that may cause these cause to be invalid, like editing metadata in some way, the categories will be scheduled for recalculation.

When you collapse the parent, the categories are no longer visible and the direct assignments only option means that IMatch must not calculate the children in order to calculate the count of the parent category.

[Jingo]

Thx Mario.. since I do not use categories (except for the magical @keywords of course), I will just remove the extra category panel from my workspace and be sure the main Category tab is fully collapsed.  Thx again for your help!

[Jingo]

Sorry to bring this back up again...  but - I have once again encountered the same problem today.  

While working on a group of images (Media&Folder tab open), I selected an image to view, hit delete to reject it and the software hung up (when this happens, my screen gets "blurry" and elongated) as it has in the past.

Reviewing the log file (luckily I was in debug mode still), I see the aforementioned #SLOWCAT entries. However, I have no category panels open and when I clicked the Categories Tab, the only path that was expanded was @Keywords... the IMATCH WORKFLOW header was collapsed as previously suggested.  So, what caused IMatch to recalculate these items?

Any thoughts on preventing these cats from ever calculating?  Why are only 3 categories reported as SLOWCAT when I have others that also have 50,000+ entries:

Code Select Expand

03.13 10:12:35+    0 [1744] 02  I> #SLOWCAT 'No Title' (5750)
03.13 10:12:38+    0 [1744] 02  I> #SLOWCAT 'No Copyright' (3110)
03.13 10:12:42+    0 [1744] 02  I> #SLOWCAT 'No GPS Data' (2344)


Thanks.. and sorry!

[Mario]

So, what caused IMatch to recalculate these items?

Many potential reasons. Something has, for some reason, requested the file count for these categories. Which is not that uncommon.
I would need to know how your workspace looks, and what you did at the time before the categories were requested to update.
In the log before the #sl the Keywords or Metadata panel was updated, relations where updated, and image was loaded into the Viewer or QVP, another thread fetched file information from IMWS, a rating was changed, buddy files were retrieved, collections were updated and stored, the file history was updated and then the categories were updated.

Keep in mind that IMatch recalculates many categories in parallel in the background and, in almost all cases, this has no effect on the user interface.
The category updates are done in background threads and unless something in the user interface has to actually wait for the categories to update, you won't even notice this. And the "has to wait" case is very rare.

Your database is on quite small, with less than 100K files and only 2,000 categories.
Still, loading the database takes 10 seconds, which is a bit slow.

In order to update the "No Title" category, IMatch has to evaluate the title tag for 100K files and assign all files with an empty title. The search operation takes 5750ms. Which is OK. For 'No Copyright' it takes only 3200ms.

What I see during the same time period are several calls to IMWS to fetch file data via the "files" endpoint and also location data. Was the Map Panel open or other panels / apps which work with GPS data?
The calls to these endpoints are usually very fast, but around the same time the slow cats are reported, IMatch also reports a whooping 11s for as the response time for the "files" endpoint. Which is a lot.
I wonder what you did at that time.

Since these categories are formula-based, they don't offer an option to make them "manual update". This is available only for data-driven categories.

[Jingo]

Thx for looking Mario.. yes, my workspace has quick a few panels docked (not necessarily open but that might not matter) including: 

Right Monitor: Open -> the Map panel, Metadata panel (with GPS Details loaded), the quick view panel  Closed but docked: Output  

Left Monitor: Open -> Keywords Panel, App Manager, Media & Folders Tab   Closed but docked: Filter, App, Info & Activity, Did You Know?, Favorites Panel.

The image chosen has GPS and Location data.. it shows in the panel and on the map.

Thx!