280'000 files in IMatch (well, to bridge the summer slump)

Started by sinus, July 25, 2019, 09:54:26 AM

Previous topic - Next topic

sinus

As you can see, I have now over 280'000 files in IMatch and it works still very good.
And yes, I know, with an SSD all would be quicker including the "compacting and optimizing", but one day ...  ;D

Have a good time!

BTW: Sometimes the number of files is smaller than before because I delete old files from time to time.
Best wishes from Switzerland! :-)
Markus

Mario

Your database would be 2 to 4 times faster if you would spend 50€ on a SSD...even if you don't switch Windows and just add an SSD as a second drive.
Optimizing is not needed very often, unless you regularly delete large sets of files (several thousand).
-- Mario
IMatch Developer
Forum Administrator
http://www.photools.com  -  Contact & Support - Follow me on 𝕏 - Like photools.com on Facebook

sinus

Quote from: Mario on July 25, 2019, 10:55:58 AM
Your database would be 2 to 4 times faster if you would spend 50€ on a SSD...even if you don't switch Windows and just add an SSD as a second drive.
Optimizing is not needed very often, unless you regularly delete large sets of files (several thousand).

Thanks, Mario

I will do for sure, but maybe in some weeks, have a lot to do now.
Best wishes from Switzerland! :-)
Markus

sinus

Quote from: Mario on July 25, 2019, 10:55:58 AM
Your database would be 2 to 4 times faster if you would spend 50€ on a SSD...even if you don't switch Windows and just add an SSD as a second drive.
Optimizing is not needed very often, unless you regularly delete large sets of files (several thousand).

I checked this, and  :-[

My Computer is an "Medion Erazer X5341 F" with 16 GB RAM.
It has a SSD-harddisk  as C with 256 GB.

And on this SSC is Win 10 AND IMatch AND the IMatch-Database.

This means, IMatch runs on a SSD.

What is NOT on the SSD, but on a a S-ATA III-Harddrive (with HotSwap), and on this are all my real files.
But the size is roughly about 4 TB and to buy an SSC for this files are very expensive.

The only thing, what I could change, are the cache-files.
At the moment they are 140 GB big.
And they are NOT on the SSD (because too big), but on a intern HD.
Do you mean, this would help a lot, if I could store the Cache-files on a SSD?

Hence, I guess, at the moment I cannot make IMatch quicker.
But for me the speed is ok and quite quickly.

Thanks for you tips.





Best wishes from Switzerland! :-)
Markus

Mario

This is quite a fast system.

If the database is on the SSD, it is as fast as it can get.

The location of the original files is irrelevant, they are only touched once (when ingesting) or when you write-back metadata.

A 140 GB cache folder size is quite a lot and not really helpful. Unless you access files from all times often, IMatch re-creates cache files on-demand as needed. So recently used images are always in the cache, and images not viewed for long time don't waste disk space. But that's only relevant if you need to conserve disk space. The size of the cache folder only minimally impacts the time IMatch needs to load a file from the cache.

What puzzles me is that your "optimize" time is 30 minutes (when I recall correctly) and on my system it takes only ~40 minutes (when I recall correctly) to optimize a 600,000 files database. Strange.
Do you an anti-virus software? Maybe it somehow interferes with IMatch.
-- Mario
IMatch Developer
Forum Administrator
http://www.photools.com  -  Contact & Support - Follow me on 𝕏 - Like photools.com on Facebook

sinus

Quote from: Mario on July 29, 2019, 01:35:46 PM
This is quite a fast system.

If the database is on the SSD, it is as fast as it can get.

The location of the original files is irrelevant, they are only touched once (when ingesting) or when you write-back metadata.

A 140 GB cache folder size is quite a lot and not really helpful. Unless you access files from all times often, IMatch re-creates cache files on-demand as needed. So recently used images are always in the cache, and images not viewed for long time don't waste disk space. But that's only relevant if you need to conserve disk space. The size of the cache folder only minimally impacts the time IMatch needs to load a file from the cache.

What puzzles me is that your "optimize" time is 30 minutes (when I recall correctly) and on my system it takes only ~40 minutes (when I recall correctly) to optimize a 600,000 files database. Strange.
Do you an anti-virus software? Maybe it somehow interferes with IMatch.

Thanks, Mario, for your detailled answer.
I think, I understand all.

No, no, my time to optimize is only mostly about 10 minutes! And I think, for this amount of files that is ok!?
Yes, I have Avira-antivirus, but I have set there, to not include IMatch-DB.

So, finally, I think, all is allright.  :D
Maybe end of next year (or so) a new computer would be a nice present for x-mas 2020.  8)
Best wishes from Switzerland! :-)
Markus

Mario

10 minutes is good for almost 300,000 files.

300,000 files is already a corporate grade (many companies get away with far less files, including the big marketing companies which are the main DAM users).
You are managing all this with a 100€ software on a consumer-grade (actually, a gamer) PC.

An i7 with 16 GB RAM is quite good. Not much has improved in that area.
I've recently checked how much it would cost to replace one of my 4 year old PCs and how much faster the new machine would be...
It would cost about 2,000€ - 2,500€ to get a noticeably faster PC with 12 cores and 2 TB SSD and 4 TB RAID spinning disk and a modern AI-ready graphic card (which costs about 800€ alone).
I guess I'll wait till the end of the year to get more bang for the buck.
-- Mario
IMatch Developer
Forum Administrator
http://www.photools.com  -  Contact & Support - Follow me on 𝕏 - Like photools.com on Facebook