Version 2025 performance

Started by theresamarie1, Today at 04:41:14 PM

Previous topic - Next topic

theresamarie1

I am a long time user of iMatch, although not a power user.   I've used it to keep track of images and the types of images (topic, subject, location,etc).   And I've upgraded over the years which includes mapping the databases.   The 2025 upgrade has surprised me in how slow it is performing.  It seems to do some 'meta' data items (says reading) in the background that make the interface mostly unusable.   It's just way too slow to navigate.   I have a fast machine that I use for large astronomical image processing (AMD threadripper, 64G, 40TB of disk, large GPU).   The program suggests that I needed to do a DB compaction today on my 80K files.     It says anywhere from 2 hours to 45 hours and bounces around on this outlook.  The program is unusable while it's doing this (which isn't a problem if it's 2 hours but 45 hours seems like there is an issue.   

What is other users experience here?   

Mario

#1
IMatch 2025 is the fastest version ever.

1. Make sure that you have excluded IMatch from your virus checker if you have done so before.
See TODO 1 in Welcome to IMatch 2025!

2. IMatch may have found many folders which have to be rescanned and that keeps it busy.
Did you check the Info & Activity Panel to see what IMatch is doing?

3. Did you read the release notes, especially this one? Note the blue alert box.
See the note Folders With Non-indexed Files

4. Always attach the log file from the IMatch session where you encountered a problem. See ##llogfile
This gives us a minimum of information to work with.

sybersitizen

#2
Quote from: theresamarie1 on Today at 04:41:14 PMI have a fast machine that I use for large astronomical image processing (AMD threadripper, 64G, 40TB of disk, large GPU). The program suggests that I needed to do a DB compaction today on my 80K files. It says anywhere from 2 hours to 45 hours and bounces around on this outlook.  The program is unusable while it's doing this (which isn't a problem if it's 2 hours but 45 hours seems like there is an issue. 

What is other users experience here?   
There are a number of different Threadripper processors, but even if yours is one of the lowest end 19XX variants, it's faster than my 5-year-old Ryzen 3 3200G system. My database of more than 90,000 files is on an inexpensive SSD. A full Pack & Go takes between 6 and 7 minutes. That and all other IMatch operations are not any slower than they were with v2023. (All my image files are on a NAS, but I wouldn't expect that to matter much for the Pack & Go process.)

Oh, a couple of things just occurred to me. File count isn't the only factor to consider. If you store a lot more data about each file than I do, that surely creates a larger database. My database is about 4GB in size. Also, Pack & Go does much more than database optimization, so I timed only Compact and Optimize within IMatch itself: 90 seconds.

Mario

Compact & Optimize is a database operation that creates a copy of the database without any slack or deleted areas.

The performance depends 99% on disk IO. If this takes hours for a small database with 80K files, the database is probably on a super-slow drive or the virus checker is kicking in.

Just did a Compact & Optimize for a 100K files database on the SSD in my five year old PC.
Total runtime 1 minute and 3 seconds!

We will know more when the OP performs the steps required for new IMatch 2025 users and provides a log file so we get a minimum of information.