File Window Pause Button Anomaly

Started by Darius1968, April 08, 2014, 06:12:02 AM

Previous topic - Next topic

Darius1968

As is, my database now has 175,965 files.  The pause button seems to function properly in the Categories or Timeline view, but not in the Media & Folders.  To validate this, I make sure all three vies are identically configured, ie., the sort order is set to default, the Pause button is set to pause if over 5,000 files, and "Show Hierarchy" is unchecked. 

If I'm in either the Categories or the Timeline view and I clicked on something over the 5K limit, an IMMEDIATE pause of the file window is effected.  On the other hand, if I'm in the Media & Folders view, and I click one of my two hard drive nodes (either C or E), pausing is not immediate, but rather it can take many seconds or even close to a minute to pause to kick in.  Why is this? 

Mario

If you click a drive, IMatch needs to process all folders on that drive to bring them into the file window. Only when this step has been performed, IMatch "knows" that there are more than 5K files, pauses the file window and stops loading the thumbnail and other data. The file window decides whether it loads the files it should display, and for that the M&F must first determine how many folders your drive has, how many files each folder has and then forward this info to the file window.

A database with nearly 200,000 files would require dedicated server hardware and a massive amount of CPU, memory and disk resources with any of the other DAM vendors. It would be considered a rather major database. There are many stock photo agencies in the world which sell images over the net and which don't have half that amount of images, running their DAM system on a multi-system server farm...

IMatch can handle this amount of images, but there are physical limits not even IMatch can overcome. I'm tuning performance all the time, but there are scenarios which are not likely to be encountered by many users (like this), and tuning for these scenarios is lower on my list.

Why do you click on a drive if you don't want to see all the files on that drive? Would it not be more efficient to just click on the folders(s) you want to see, maybe using the hierarchical mode in the file window to also show the sub-folders?

Loading 50,000 or even 100,000 files in a file window is possible. But since IMatch has to perform a number of operating system calls to query file system states this will take time because file system interaction is really slow compared to the IMatch database itself.
-- Mario
IMatch Developer
Forum Administrator
http://www.photools.com  -  Contact & Support - Follow me on 𝕏 - Like photools.com on Facebook

Darius1968

I don't intentionally click on the drive node itself to test IMatch's abilities, this would happen if I didn't pause IMatch first before entering something in the Media & Folders filtering.  Why is it though that if I for instance, click on the @ALL category in the Categories view, then all 175 K files will show in 7 seconds? 

Mario

Categories are an unique IMatch feature. I designed this and wrote the code.
To determine how many files are in @All, IMatch does not need to call file system routines or produce a deep hierarchy of nested folders etc. This saves a lot of time.
-- Mario
IMatch Developer
Forum Administrator
http://www.photools.com  -  Contact & Support - Follow me on 𝕏 - Like photools.com on Facebook