Database size?

Started by Aubrey, January 02, 2014, 10:16:54 PM

Previous topic - Next topic

Aubrey

IMatch 3.6 database is 927,537 KB
IMatch 5 database was 1,698,280 KB

After rebuilding search engine index and compacting (I did not check size at each step) the new size is
2,018164 KB

Is it normal for the new DBase to be 2x the 3.6 database?

Thanks,
Aubrey.

JohnZeman

Mine is about 1.5 times larger.

Mario

That's caused by all the metadata stored in the database. IMatch stores 50 to 500 metadata tags for each file, depending on whatever ExifTool can extract and what's not filtered out. That makes up for a lot of data.

I have test databases with 50 GB, no problem.
-- Mario
IMatch Developer
Forum Administrator
http://www.photools.com  -  Contact & Support - Follow me on 𝕏 - Like photools.com on Facebook

Aubrey

I'm using Backblaze for my cloud backup. This means a lot of backup time when I change database.
Upload rate is only 512k in cyprus... Pretty horrible! Download 5mb
To a upgrade to higher upload rate means a significant cost... Even 10 mb download gives me same upload rate!
Any thoughts on splitting Dbase file.
aubrey

Mario

Even a very small and cheap USB stick can hold multiple copies of your database. If your cloud upload is so slow, consider this as an alternative.
IMatch 5 is a pro-level DAM system. All the features require a bit of disk space. A minimal cost, really. ZIP the database to save some 30% or so.
-- Mario
IMatch Developer
Forum Administrator
http://www.photools.com  -  Contact & Support - Follow me on 𝕏 - Like photools.com on Facebook

Aubrey

#5
Just checked my database size- it is 4.5 Gb with ca. 33,000 raw images

Others have database of similar size wit this number of images?

Aubrey.

Mario

I have a 100,000 files database with less than 5 GB.
Size varies depending on the size of the thumbnails chosen, and more by the amount of metadata contained in your files.
-- Mario
IMatch Developer
Forum Administrator
http://www.photools.com  -  Contact & Support - Follow me on 𝕏 - Like photools.com on Facebook

Aubrey

I checked through my settings, thumbnail size was set to 800  ???

I changed it to the recommended 300 and re-sampled files. Database size now down to 2.6 Gb!

Thanks,
Aubrey.

sinus

Quote from: Mario on January 03, 2014, 08:20:16 AM
I have test databases with 50 GB, no problem.

This if good to know!

I have a (TEST)-db with 20'000 image at the moment and the thumbnail-size on 500 pixel.

The DB-size is now 1.5 GB.
At the end of the "moving from IM3 to IM5" I will have about 200'000 files. So I will have ROUGHLY a DB-size of 15 GB.
This seems to me a lot, but I think, nowadays are huge files and huge DB not more a very big problem.

Therefore I like to hear such a statement from Mario.

And the thumbnail-size, at the end, I will reduce more, maybe 300 pixel. On the other hand, monitors gets bigger and bigger - it seems, nowadays almost  ;D everything tends to be bigger.  :)
Best wishes from Switzerland! :-)
Markus

Mario

300 pixel thumbnails usually work pretty well for screen resolutions up to ~ 2000 pixels. We have the Quick Preview and Viewer and can look at large images any time.

If you use 27" or 30" monitors with  ~ 3000 pixel resolution you may need bigger thumbnails.

Tip: Under Edit > Preferences > Database you can select a thumbnail size and then click on the magnifier button to see it 1:1 and to get an estimate about the size required to store 1,000 of these thumbnails:

300 pixel: ~ 24 MB for 1000 files.
500 pixel: ~ 58 MB for 1000 files.
800 pixel: ~ 146 MB for 1000 files.

Of course the database space requirements vary, depending on the image content and how effective the compression is. It's a lot of data.

And 200,000 files is a lot, too. I can tell you that large stock agencies don't even attempt to manage this collection size on a single PC. That's rather a job for a server rack plus Enterprise-level disks, IO-systems and UPS. But then, they don't use IMatch  ;)
-- Mario
IMatch Developer
Forum Administrator
http://www.photools.com  -  Contact & Support - Follow me on 𝕏 - Like photools.com on Facebook