Version 2025 performance

[theresamarie1]

I am a long time user of iMatch, although not a power user.   I've used it to keep track of images and the types of images (topic, subject, location,etc).   And I've upgraded over the years which includes mapping the databases.   The 2025 upgrade has surprised me in how slow it is performing.  It seems to do some 'meta' data items (says reading) in the background that make the interface mostly unusable.   It's just way too slow to navigate.   I have a fast machine that I use for large astronomical image processing (AMD threadripper, 64G, 40TB of disk, large GPU).   The program suggests that I needed to do a DB compaction today on my 80K files.     It says anywhere from 2 hours to 45 hours and bounces around on this outlook.  The program is unusable while it's doing this (which isn't a problem if it's 2 hours but 45 hours seems like there is an issue.   

What is other users experience here?   

[Mario]

IMatch 2025 is the fastest version ever.

1. Make sure that you have excluded IMatch from your virus checker if you have done so before.
See TODO 1 in Welcome to IMatch 2025!

2. IMatch may have found many folders which have to be rescanned and that keeps it busy.
Did you check the Info & Activity Panel to see what IMatch is doing?

3. Did you read the release notes, especially this one? Note the blue alert box.
See the note Folders With Non-indexed Files

4. Always attach the log file from the IMatch session where you encountered a problem. See ##llogfile
This gives us a minimum of information to work with.

[sybersitizen]

I have a fast machine that I use for large astronomical image processing (AMD threadripper, 64G, 40TB of disk, large GPU). The program suggests that I needed to do a DB compaction today on my 80K files. It says anywhere from 2 hours to 45 hours and bounces around on this outlook.  The program is unusable while it's doing this (which isn't a problem if it's 2 hours but 45 hours seems like there is an issue. 

What is other users experience here?   

There are a number of different Threadripper processors, but even if yours is one of the lowest end 19XX variants, it's faster than my 5-year-old Ryzen 3 3200G system. My database of more than 90,000 files is on an inexpensive SSD. A full Pack & Go takes between 6 and 7 minutes. That and all other IMatch operations are not any slower than they were with v2023. (All my image files are on a NAS, but I wouldn't expect that to matter much for the Pack & Go process.)

Oh, a couple of things just occurred to me. File count isn't the only factor to consider. If you store a lot more data about each file than I do, that surely creates a larger database. My database is about 4GB in size. Also, Pack & Go does much more than database optimization, so I timed only Compact and Optimize within IMatch itself: 90 seconds.

[Mario]

Compact & Optimize is a database operation that creates a copy of the database without any slack or deleted areas.

The performance depends 99% on disk IO. If this takes hours for a small database with 80K files, the database is probably on a super-slow drive or the virus checker is kicking in.

Just did a Compact & Optimize for a 100K files database on the SSD in my five year old PC.
Total runtime 1 minute and 3 seconds!

We will know more when the OP performs the steps required for new IMatch 2025 users and provides a log file so we get a minimum of information.

[HG]

@theresamarie1, I'm facing a similar issue. Temporarily the UI is not updated or updating slow. I did all the tips coming from the community, unfortunately nothing worked well. My machine is fast enough, this could not be the reason, when monitoring I never got a peak. Poor UI updating causes failures when writing into images. Hopefully, development will fix this soon.
Regards HGF

[mopperle]

Hopefully, development will fix this soon.

Compact and Optimize of 40k files db took 36 sec. on my 6 year old notebook, so not sure what you expect Mario to fix.

If you experience any problems, you should at least attach a debug logfile and/or the IMatch diagnosis logfile after running a database diagnosis. For more details read the according section in the helpfile:
https://www.photools.com/help/imatch/rmh_dlg_dbdiag.htm

[theresamarie1]

Thanks to those who provided helpful feedback and confirmed that there isn't a general 2025 version performance issue. I can't say exactly how long it took to perform the compaction and optimization, but it did end in less than an hour overall but much more than a minute or two.  I wrote my original post while it was running and giving a broad range of ETAs (up to 45 hours).  Today I re-ran this and it took less than a minute to optimize 191K files (I underestimated the size).  As I mentioned, it runs on a threadripper, one of the faster models, and runs on SSD rather than spinning media, although i use some large hard disk to stop backups and video.  Maybe the initial run was due to my not having done this in some time.   But right now the performance seems mostly normal and working as I expect.   Thank you again.

Terri

[Mario]

About 98%-99% of the time of Compact & Optimize is spent "in" the database system.
Basically it creates a copy of the database, leaving out stale sections of the database, no longer needed pre-allocated sections, performs some index maintenance and stuff.

The runtime thus depends on the speed of the drive containing the database. The estimate the dialog is showing is calculated whenever the database "calls back" IMatch, based on the percentage completed at this point in time and the runtime so far. If a virus checker hits the database or copy when the process starts and stalls the database system for several seconds before it is allowed to read or write data, the estimate will vary widely, even showing dozens of hours. This should equalize quickly, as the database system continues.