Is there any way to speed up thumbnail loading?

[DavidOfMA]

Short of buying a new computer or a graphics card, is there any way to speed up thumbnail loading? When I switch from one folder to another, IMatch freezes for a few seconds and then takes about 10 seconds to display the thumbnails in the new folder (and there are always fewer than 50 images in each folder, so it's not trying to display a large number). There is no filtering applied and the thumbnails are only 300 pixels wide, so that doesn't seem to be a factor. Also, when I try to page down in a particular folder, IMatch usually takes several seconds to display the next screen of thumbnails. The Viewer is similarly slow in displaying the first image in a folder (about 12 seconds) and in moving from one image to the next (4-6 seconds).

In IMatch 3.6, all of this was very fast, and although overall I like a lot about IMatch 5.5, it's not very usable on my computer because it's running so slowly. The computer is a Gateway running Windows 7 64-bit, with an Intel four-core processor and 6GB system memory. Unfortunately, the BIOS limits the amount of system memory I can dedicate to the onboard graphics processor to 128MB, for reasons that must have made sense to some programmer, but not to me.

Thanks,
David

[Mario]

Thumbnail loading is usually very fast in IMatch.
IMatch loads only the thumbnails needed to fill a page, and pre-loads batches in advance.

While file window layout do you use?
How many thumbnails do you show per page?
Where is the database located (local disk, SSD, network)?
Do you Optimize your database regularly?

Is it faster when you use the "Thumbs only" layout, which does not need to load collection, rating, label, relations etc?

Please enable debug logging and provide a log file from a session where you

Start IMatch
Open a folder, scroll in the file window
Switch to another folder, scroll in the file window
Go back to the original folder and scroll again a bit
Close IMatch, ZIP and attach the log file.

[DavidOfMA]

While file window layout do you use? I use a customized thumbnail layout that is slightly modified from "Default." It displays a few pieces of information about the file and displays the ratings.

How many thumbnails do you show per page? Usually 12.

Where is the database located (local disk, SSD, network)? On my fastest hard drive, a 2TB Seagate SATA drive that was the fastest when I compared database diagnostics speeds. IMatch 3.6 displays a database of the same images almost instantaneously, both when I first select a file and when I scroll in the file window. On IMatch 5.5, folder selection display that takes less than half a second on IMatch 3.6 takes about 12 seconds. This is similar to the slowness of database diagnosis. It takes IMatch 3.6 40 seconds to diagnose my 130K file database, while IMatch 5.5, even on the fastest disk, takes about 11 minutes (down from 60, originally).

Do you Optimize your database regularly? Yes. This database was optimized a few days ago, and I have added no images since. It is just as slow immediately after optimization.

Is it faster when you use the "Thumbs only" layout, which does not need to load collection, rating, label, relations etc? No, "Thumbs only" is actually slower, because with my usual layout IMatch tries to display 2 rows of images, whereas with "Thumbs only," it tries to display 2 rows plus the partially-in-view third row that would otherwise be occupied with file information. In either case, when I scroll, first it freezes, then it scrolls with some blank thumbnails that only have the text, then IMatch fills in the blank thumbnails with images. Something similar happens when I switch folders. First it freezes, then it selects the folder and displays the "Loading" message in the status bar, and finally it displays the first couple rows of thumbnails. NOTE: IMatch is also very slow to close the Viewer. Usually, I see the "program not responding" message, click Wait, and finally it closes.

Please enable debug logging and provide a log file from a session where you

Start IMatch
Open a folder, scroll in the file window
Switch to another folder, scroll in the file window
Go back to the original folder and scroll again a bit
Close IMatch, ZIP and attach the log file.

Attached.

NOTE: I checked CPU use for IMatch when I scroll in a folder and it's struggling to paint thumbnails, and when I try to open a file in the Viewer. It's 25-30% when I click from one folder to another, the same when I scroll within the folder, and up to 50% when I open a file in the Viewer, even if the file is a JPEG. This seems like a lot of crunching and seems to be what is slowing the whole thing down.

Thanks,
David

[attachment deleted by admin]

[Mario]

Everything on your system that involves reading data from the database is slow, slow, slow...
It's like your disk is running with one foot firmly on the brake.

The file Window reports that it needs almost 10 (ten!) seconds to load a group of files from the database.
It's slow to load the relation data, needed for the file window.
It's slow to load verb data, which is needed for the file window (to tell if proxies are used)
It takes literally 9 seconds to load the data required to 'fill' the file window from the database.
IMatch even reports 'slow' database transactions, which is something you might expect when IMatch is busy importing new files in the background while the user performs options in the front-end which require lots of slow database operations. Not when the system is idle and you are only loading a file window.

Are you sure you are not using a virus checker or some other software which keeps the disk very busy all the time?

10 seconds to load a file window on your system.  Hum...

To get some measures, I've made a check with a 220,000 files database here. That's the largest database I have.
It's on a quite new standard hard disk (2 TB WD).
My computer is about six years old, Windows 10.

I used the Default file window layout for this test.

Loading a folder with between 6,000 and 10,000 files into the file window takes only between 1.4 and 2.5 seconds.
My file window shows 4 rows of 7 files, and a bit of the fifth row (~ 35 thumbnails per page).

When I switch to a thumbnail only layout, my file window shows about 140 files.
It takes ~ 2 seconds to scroll from page to page in that modus, for a folder with 7,000 files.

Smaller databases or a database on a SSD will be even faster of course.

The interesting log entry is PTFileWnd::LoadGroup with the < in front. This reports the overall time the file window required to load and prepare the data for display.
On your system this method takes almost 10 seconds.

It's like your disk is slowed down by a virus checker or something else. And IMatch cannot display files faster than it can load data from the database. Even if you only show thumbnails without any metadata, IMatch needs to load the file info, the thumbnails (several KB per file), the relation information etc. Performance stands and falls with how fast the disk can deliver the data.

[DavidOfMA]

Yes, I understand that this is abnormal behavior for IMatch. What I don't understand is how to fix the problem so that I can use the program.

Previously, I tried turning off the virus checker (Avast!) and also Windows Defender, which I don't run in real time but which could, perhaps, have somehow been triggered by IMatch. Turning off the virus checkers had no effect on IMatch's poor performance on my system then, and when I just did it again, now, it also had no effect. I've also marked the IMatch database extension, and the folder in which I keep IMatch, as items for the virus checker to ignore. Still no improvement in performance.

What else can I do to try to get this program to work properly on my machine? As I said, IMatch 3.6 works perfectly, and so do all my other programs. Copying the database itself is very quick, too. But somehow, IMatch 5, and only IMatch 5, runs extremely slowly on my system.

Thanks for any further help you can provide.

David

[Mario]

No idea at the moment.
The file window "load" code is in place basically since version 5.0 back in June 2014.
Must be something that's interfering with any disk access the database system does on your machine.

Have you tried to create a new database, add a few thousand files and then check the performance?

[DavidOfMA]

I've had the problem with loading being very slow since I started using IMatch 5. I just haven't had time to work with the program until recently. It's not something new with 5.5.

I will try a new database and see if that makes any difference, though I have checked and optimized this database several times -- shouldn't that fix any database issues?

If it turns out a fresh database solves the problem, what is the best way to repair the existing database? Or, in the worst case, transfer all my attributes, categories, bookmarks, etc. to the new database?

Thanks,
David

[DavidOfMA]

I think I found the problem. I turned off "enable color management" for the thumbnails, and now the folders open much more quickly. Perhaps you could take a look at that code. Turning color management on for thumbnails in IMatch 3.6 had only a negligible effect on display speed. Something's different about it in IMatch 5. Have you tested your databases with color management for thumbnails turned on? That would tell me if this is a local problem or an IMatch problem.

The Viewer is still slow, but it's lightning fast when I view grayscale images. This leads me to believe it would also speed up dramatically if I turned off color management there. Is there a way to do this? I didn't see anything like that in Edit Preferences. I'd much rather lose a little in color fidelity if it made a huge difference in speed, which it seems it would. If this turning off color management for the viewer is not possible now, I'd like to suggest it as a feature.

David

[Mario]

As explained in the help, color management should be off for thumbnails unless you really, really need it. And then turn if off again. It is off by default for a reason. Running each thumbnail through Windows ICM slows down thumbnail loading by a factor of 10 or more.

[DavidOfMA]

I see that now, though as I said, it isn't a problem on IMatch 3.6.

Is there a way to turn it off for the viewer and quickview?

[Mario]

There is no code change between IMatch 3 and 5 in how color management is performed. This is still all in-software. Nobody really needs CM for thumbnails.

You cannot disable color management for the Viewer, Slide Show or Quick View Panel.
The typical time for color management with the new hardware based render is about one second, and it's done in the background, while the image is pre-loaded.

[DavidOfMA]

Thanks. That explains why switching to folders with four or five images took four or five seconds when color management was enabled, and fuller folders took 12. Also why viewing preloaded files is fast but there is a delay. when going faster than the preload. The Viewer was very fast with the grayscale files. No "Loading" message at all, though these were big files. It would be nice to have the choice to have that speed with RGB files, too, even if ICC profiles were ignored.