Search Time Benchmarks

Started by Darius1968, February 03, 2020, 09:55:52 AM

Previous topic - Next topic

Darius1968

There are currently about 400K files in my database.  My system has SSD storage, an I9 processor, and 16 GB RAM. 
I had a read of this post, in which you say that you are able to do a File Window Search (Advanced Search, Search everywhere) for the term "Paris" in 13 sec.  This was a few years ago, on a system that is in some regards, below the specs of my system.  With that said, my search is taking about 23 sec., and I was just wondering why this would be.  You said in that post that the same search (with "Frequently used tags" enabled) took 5 sec.  This way, it takes about 3 sec. on my system. 
Finally, I've surprisingly found that if I do a search (Advanced Search, Search everywhere) for two terms (Paris AND France), then the search time is cut in half, to about 11sec!  But, that search, with the OR operator takes about 45 sec. 
I've attached the log file in Debut Mode, if that helps. 

Mario

Searching 400,000 files in 10 or 20 seconds is amazingly fast, especially when you force IMatch to search all metadata tags ("search everywhere") or many tags ("common tags").
On a 400,000 files database with approximately 300 tags per file (could be lots more!) this means IMatch needs to search 120 million data items every time you run a search.

Search performance varies greatly, depending on parameters, scope size, other things going on on your system, state of search index etc.
"Search Everywhere" searches the entire database and is thus slowest. Searching frequent tags still searches about 60 or 80 tags - per file in the scope!

The first time you run a search and every time you change the scope or a search parameter, IMatch needs to re-build the search index in-memory.
It does this in the background.

Which means that when you wait 5 seconds and run a search, the index may be ready and the search takes 3 seconds.
When you change a parameter and don't wait, IMatch has to first build the index (5 seconds) and then search, which results in a "search time" of 8 seconds.

When the search index is built and you do repeated searches without changing the scope or parameters, searching will be really fast - because the index exists and can be used.
-- Mario
IMatch Developer
Forum Administrator
http://www.photools.com  -  Contact & Support - Follow me on 𝕏 - Like photools.com on Facebook

Darius1968

I wasn't complaining that the time was too long.  I was just wondering why a system with somewhat higher specs was taking twice as long to do the same thing that an older system with an I7 processor would take. 

Mario

IMatch reports that it could not use the search index.
This usually means you are searching everywhere or the specific combination of tags/options/scope you are using prevents IMatch from using the search index - or it would be inefficient to create the search index.
-- Mario
IMatch Developer
Forum Administrator
http://www.photools.com  -  Contact & Support - Follow me on 𝕏 - Like photools.com on Facebook