Orientation & Dimensions Vs Data Driven

Started by Darius1968, March 28, 2018, 08:44:31 AM

Previous topic - Next topic

Darius1968

Okay, so, I seek all images in my database that have a net ratio (in their dimensions) of 2,048Wx1,536H, or 2,048Hx1,536W. 

For said task, I have a choice of either, Orientation & Dimensions global filtering, or I can set up a data driven category. 

If I set up the data driven cat., I find that I have to use {File.Width}, {File.Height} variables, instead of recommended tags, which takes  a whole few minutes to calculate all such 1,893 image files in my database!  The reason I can't use tags, is because many of the files are not the original files, but rather, they are files that are now, no longer 2048x1536, but instead, maybe, 1024x768, because they are a 'version' file derived from some original file I don't have. 

What, exactly, is the Orientation & Dimensions using as its source, for the selection of files that have an actual width of either 2048 or 1536, that is so quick?  Surely, it can't be using the variables! 

Mario

The ability to use variables in data-driven categories gives you a very powerful tool to handle very complex cases.
But as documented, this comes at a price. Using variables is a lot slower than using the metadata directly.

A "few minutes" is rather long. How many files does your database have? Database on SSD? Computer speed?
I would guess that for 50,000 files calculating two variables takes about 5 seconds or so...

The Orientation & Dimension filter utilizes a special query module inside the IMatch engine. For the aspect ratio filter it uses the file width/height. The same information that is used for the {File.Width}, {File.Height} variables, but of course more directly, in-memory cached, without the additional overhead of variables or data-driven categories.
-- Mario
IMatch Developer
Forum Administrator
http://www.photools.com  -  Contact & Support - Follow me on 𝕏 - Like photools.com on Facebook

Darius1968

I'm not surprised at the speed compromise, for my not using right now, such a stellar system - everything being on two (one terabyte) hard drives, on an old dual-core processor, with about 450K files. 

Mario

This explains that, then.
Half a million assets managed on a regular mid-range system, with regular hard disk. There are limits, even for IMatch.
I guess your DB holds about 150 million to 200 million metadata records. Sieving through that to update a data-driven category is already a challenge.

Doing the same, fetching 500,000 file records, producing two variables each, then perform the data-driven category magic on that will cause even a current 6 core system to be busy for a minute.
-- Mario
IMatch Developer
Forum Administrator
http://www.photools.com  -  Contact & Support - Follow me on 𝕏 - Like photools.com on Facebook

Darius1968

If I right now, get an SSD storage device for my database, could I expect any speed improvement on my current system that is also constrained to a USB 2.0 connection? 

Mario

Switching to an SSD is always a big improvement.
But connecting it via USB 2.0 will be no good. USB 2.0 cannot barely handle the transmission rates of a modern hard disk. And by no means can it handle SSD speeds.
Make sure you add the SSD to your system directly, replacing the C: disk (on which Windows is installed). This will speed up everything.
Usually you get an easy "disk copy" program bundled with your SSD which allows you to clone the existing C: disk, then replace it with the SSD.
-- Mario
IMatch Developer
Forum Administrator
http://www.photools.com  -  Contact & Support - Follow me on 𝕏 - Like photools.com on Facebook