UI delay since upgrade to 2025.1.14

[HG]

Hello, I opened a similar thread yesterday and received a lot of good response. Unfortunately, by accident, I deleted the thread. My fault. Carefully I read all the good tips and did some changes. Unfortunately, the problem still exists. In thumbnail view, when selecting multiple files (~20) and assigning a category or pin to the selected files, the UI delays. Changing the directory after updating files sometimes is slow as well. Sometimes it is fast, sometimes really slow, what makes it difficult to reproduce. If there is a delay in the UI, there is a risk to do right things to wrong images. This happened to me several times. This issue is making me careful but reduces the speed of my workflow.
CPU, Mem and SSD usage are low all the time. EXE and db folder are excluded from from MS virus scan. Face recognation not enabled.
Attached the LOG.
Hopefully there are more ideas how to fix. Thank you all in advance.
HGF

[Mario]

IMatch has found two folders which are outdated and are scheduled for rescan in the background. You can find the folders by searching for W> in your log file.

When you looked at the Info & Activity panel as suggested, was IMatch still processing files?
That would be the most likely explanation.

IMatch reports a whooping 80s for starting the application (including 20 seconds for loading the 200K database). That's unusual, unless IMatch was very busy processing out-of-date folders in the database. See https://www.photools.com/release-notes?search=2661 for the reason why IMatch may find many folders to rescan initially.

Then the log file shows many files being processed, indexed, metadata read and then you shut down IMatch.
Was IMatch finished with processing files at that time? It does not look like it.

If your PC does not copy well, try setting the performance profile to Balanced or Low. See Performance Profile for more information.

[HG]

I did a complete rescan of the database and an optimazation. Monitoring of CPU, mem, disk utilization is low. No peaks. My workflow is exactly the same as in privious versions. In previous versions I never excluded anything from viruschecker, nevertheless Imatch performance was almost satisfying. Now exe and db are excluded. Since version 2025 I'm in trouble. It coulkd be true that 2025 is the fasted version since ever. Nevertheless I'm facing peaks of low response in GUI. Last time this morning. Response time of GUI in thumnails window was poor and I wrode descriptions into wrong images, because the screen was not updated correctly. And there were no open activities in Imatch. This happend now to me several times and is really frustrating.   This never happened in previous versions. I guess, in V2025 there is a general problem. I tried to reproduce the issue, unfortunately it happend temporarily is impossible to reproduce. How can I provide support to the development to fix this?

Regards HGF

[Mario]

I guess, in V2025 there is a general problem. I tried to reproduce the issue, unfortunately it happend temporarily is impossible to reproduce.

IMatch 2025 is out for six weeks now, and user upgrade rate is exceptional. Hundreds of users are already using IMatch 2025 daily. If there would be a general performance problem, I would know about it for many months and this community would be flooded with reports.

Note: Please press <Enter> occasionally to make your posts readable. A post like the one above is a screen-high wall of text on mobile devices, which I use often. Thank you.

There are not many reports about "bad" performance.

One from @ubacher we tried to analyze over the past days seems to actually be caused be a severe hardware problem, as he found out today. Which is in line from what I could determine from the log files he provided.

One from another user, who forgot to update his virus checker exception to IMatch 2025. After doing so, performance was superb again.

I never excluded anything from viruschecker,

This does mean nothing. Virus checkers get updated frequently, and what worked yesterday just fine may be broke today. Your virus checker may not have interfered with previous versions, but the new components, 3rd party tools and Microsoft framework IMatch 2025 ships with may upset it for some reason.

How can I provide support to the development to fix this?

As always with fuzzy, nob.repeatable issues encountered by one user on one PC, analysis is difficult.
You did not answer my questions from my last post. Please do so.

Switch IMatch to debug logging via Help > Support and when you encounter the bad performance again, go to Help > Support < Copy Logfile and save the log file to disk. ZIP and attach. We may be lucky and actually see in the log which operations were performed and what took the most time.

When IMatch is idle, CPU consumption is < 0.5%. When you even experience problems with Windows repainting the File Window, the only explanation I could come up ad-hoc is that IMatch is processing files in the background. Which is why I asked the questions I did but did not get answers.

Things like face recognition and face reclustering running in the background may cause performance issues if the PC is not up to it. Go to Edit menu > Preferences: Application and search for performance. Reduce the performance profile to Balanced or Low to see if this makes any difference.

And attach a log file of a session where you encountered sub-optimal performance.

[HG]

Just had a new thumbnail view where the catagories were listed from the prevoius screen. It took ~5sec 'til the catagories window was refreshed. This is just a sample. Sometime I'm facing delays in different operations, what is making the isolation of the issue difficult.
As requested, attached the debug log and a screenshot of the activities screen.

[HG]

Happened again. In thumbnail view, open catagories register and refresh >10 sec.

[Mario]

Just had a new thumbnail view where the catagories were listed from the prevoius screen. It took ~5sec 'til the catagories window was refreshed. This is just a sample. Sometime I'm facing delays in different operations, what is making the isolation of the issue difficult.
As requested, attached the debug log and a screenshot of the activities screen.

Your database has 190,000 files.
Which kind of drive is F:, which holds your database?

Looking at the log file, the first thing that is interesting:

Parsing variable '{File.Categories.Direct|filter:@All|Personen;count:true}' done in 85594ms (12 threads), producing 110012 elements. [86859ms #sl] PTDataGrouperGroupDelegate::Invoke


IMatch needed 86 (!) seconds to calculate the category Personen_Anzahl.
And this massive runtime of course will slow down any other category calculation and database operation.

There are also many write-backs going on, with metadata propagation taking 6 to 12 seconds each. That's expensive.
Of course also causing IMatch to have re-reads metadata from many files, invalidating all categories, including the super-expensive "Personen_Anzahl" category etc.

The write-backs go to the same disk F: which contains the database, further reducing performance.
You should try to keep the database on the fasted disk (SSD!) and not on the same disk as the files you manage. A 200,000 files database is larger than what many stock photo agencies deal with, and IMatch cannot overcome physics. Moving your database to the fastest SSD (C:, usually) should already improve performance.

Or, maybe your system is overloaded already. Did you change the performance profile as I suggested?


While all the write-backs going on, metadata being reloaded, categories being invalidated all the time, the performance for tasks like propagation drop dramatically.

Also, I see [90735ms #sl] PTMetabase::SetFileValues which means that IMatch took 90 (!) seconds to write back metadata for one or more files. Which is usually measured in milliseconds, not seconds.

The database system reports excessively long transaction times, which means the disk containing the database is maxed out, not being able to read and write data.

Which is understandable, when IMatch is writing and reading metadata from to/from the same disk that contains the database. When the disk is at 100%, that's it.

You say you see no "peaks" in Task Manager. But if the disk performance drops so badly, it will show on the Performance tab.

Summary

At the time you report the system as slow:

- IMatch was writing back many of files

- IMatch was propagating metadata to even more files

- IMatch was re-indexing files and invalidating/updating categories visible somewhere

- You have constructed a category named Personen_Anzahl  that is so slow that it takes almost 90 seconds to recalculate. Categories based on variables are expensive (aka slow) and you use a very expensive variant with category lookups and filters. IMatch cannot cache variable-driven categories because of their inherent volatile nature. Each file write-back invalidates all categories, including this expensive one.

- You store the files you manage in IMatch on the same disk as the database. This means that write-backs, re-indexing and database operations compete with each other - until performance drops dramatically.

Suggestions:

Put the database on your fastest SSD, not on F:
Remove or set to manual update the Personen_Anzahl category
Expect IMatch to respond more slowly while is is writing back, propagating metadata on disk, indexing the updated files
Avoid maxing out the disk by reducing the performance profile in IMatch to Low or Balanced.

Let us know how performance is after implementing these measures.

[HG]

Mario, many thanks for investigation and providing solution options. My drive F is a fast SSD and you are right, files and db is on this drive. I'll move db to drive D, what is a SSD as well. I try to avoid having operational data on drive c  , because in case of windows crash I want avoid to loose user data. Should I move the preview cache as well?
Personen_Anzahl was created in the past to help a filter categrory to find images with a single person. However, no problem to remove this category for testing purposes. I'll setup all your sugestions and will come back with the result. Many thanks in advance, HGF

[Mario]

If you need such a category, set it to manual. Or, maybe a data-driven category on one of the person tags with a numeric filter can be used. This will be much faster than parsing and processing ~200,000 variables for each update of this category.

Should I move the preview cache as well?

Normally the cache is not causing much I/O, except perhaps when IMatch runs face recognition and pulls 20 or more files from the cache at the same time.

[HG]

Here are my results:
I moved the db (and previewcache) from F: (SSD 870 EVO 4T) to D: (SSD 850 EVO 500G) and excluded this folder from Defender. The improvement was noticeably better. Contrary to my original statement about the Windows drive, I nevertheless moved the db and the previewcache to C: (NVME 970 EVO Plus) in a 2nd attempt. The NVME is once again significantly faster and this is noticeable when dealing with Imatch. I have not noticed any UI delays since then. I have changed the "Persons_Number category" to manual and that has brought another advantage. Changing the "performance profile" to balanced then no longer produced any noticeable advantage.
Conclusion:
The separation of data and database on different drives was the decisive factor, which in retrospect, after Mario's detailed explanation, is also logical. And as a supplement, it is definitely an advantage if the db is located on the fastest drive. Also, when setting up some great things, like in my case the "persons_number category", you have to be careful not to produce any background processes that might paralyze the whole system.
I would like to thank Mario for his patience and the community for the many good suggestions, which all contributed to the solution and initiated a learning effect.
I will send a final debug log and then switch off debugging again.
Yours sincerely
HGF

[Mario]

Very good. My guess is that the disk was maxed out and the super-slow category on top made things even worse.
IMatch is fast, but there is physics...

Not all SSD drives are equally fast and the database should always be on the fasted available SSD - because IMatch is basically a high-performance database server with a GUI. Performance depends on disk speed.

The more processors your system has (telemetry tells me that the average number of processor cores is rising all the time), the more the disk becomes the bottleneck - especially for things like category updates, collection updates.

The log shows that IMatch used 12 parallel threads to calculate your variable-based category. Which is awesome, because it tells me that IMatch neatly broke down the category calculation into 12 independent blocks with roughly 16.600 files each and then set all 12 processor cores to process the data in parallel. Which guarantees optimal performance - if the disk can deliver the data fast enough. Else the processors have to wait.

Which was the case, because IMatch was also writing and reading files at the same time and on the same disk containing the database.

If IMatch has finished processing all files and is idle, trigger the calculation of the expensive category and see how long it takes now. I'm sure it will be a lot faster.

Tip: Keep an eye on the Performance Panel in the IMatch Dashboard. It lists categories and collections which take too long to update.