divide a DB into 3 pieces

Started by sinus, February 27, 2017, 06:20:19 PM

Previous topic - Next topic

sinus

Only as an info:

I will divide my DB (about 240'000 files, 14.5 GB DB) into three pieces.

I think, all will then be quicker.

At the moment I think about, how to do it  ;D
I did not find the info (it was here once) and in the help-file I find also nothing (maybe I have overlooked it).

I think, one way could be:

1) copy the db
2) give other names, like DB1, DB2, DB3
3) then open one and "remove from Database" some folders.
4) and in the other DB I remove other folders.

I must think about, what is also involved here, hm, for sure, I think, the cache.
And maybe other stuff.

I will report my progress.  8)
Best wishes from Switzerland! :-)
Markus

Mario

240,000 files is not that much.

It should work quite fast on any modern computer. SSD storage is highly recommended.

I work with my 350,000 files database on a 1,300€ PC with 6 cores, 32 GB RAM and full 512 GB SSD storage and it is very fast. Even with 20,000 categories - but I already use IMatch 2017 with the new low-memory storage.

I really would think twice before splitting a database. It will work the way you suggest, but still...
Do you have SSD storage?
Did you consider removing not needed data-driven categories?

Maybe wait for IMatch 2017 and see if this is faster?
-- Mario
IMatch Developer
Forum Administrator
http://www.photools.com  -  Contact & Support - Follow me on 𝕏 - Like photools.com on Facebook

sinus

Quote from: Mario on February 27, 2017, 06:48:45 PM
240,000 files is not that much.

It should work quite fast on any modern computer. SSD storage is highly recommended.

I work with my 350,000 files database on a 1,300€ PC with 6 cores, 32 GB RAM and full 512 GB SSD storage and it is very fast. Even with 20,000 categories - but I already use IMatch 2017 with the new low-memory storage.

I really would think twice before splitting a database. It will work the way you suggest, but still...
Do you have SSD storage?
Did you consider removing not needed data-driven categories?

Maybe wait for IMatch 2017 and see if this is faster?

Hmm, ok, I will think about it.
Maybe you are right, it is not not the best time, better wait on IMatch 2017.

Hm, yep, I  will wait and as you proposed, looking, if I could do such things like removing data-driven-cats.
Thanks!!  :D
Best wishes from Switzerland! :-)
Markus

ubacher

I have split databases exactly as you described. Mostly to carry a subset with me on a laptop.

The difficulty is keeping Imatch in synch. If you change some templates, scripts, filter settings, favorites etc. or add some categories then you will want to have the same on the other databases.

I keep track of the changes manually and apply them when I am back home to the
other (main) database. Laborious process but I see no other way.

Sooner or later we have to split our large databases if they continue to grow. The question is only when (at how many files.)


sinus

Thanks for this, interesting.

Yep, there are reasons for and against splitting.
On the other hand, there are some users with over 350'000 files, so I have still a bit of time.  ;D

And, computers and disks are also quicker (and cheaper).

And yes, holding in sync, this is something, what we must looking at.
Best wishes from Switzerland! :-)
Markus

Mario

Quote. If you change some templates, scripts, filter settings, favorites etc.

That's what Pack & Go is for. It does this automatically. Except for machine-specific settings or database-specific settings of course.

Quoteor add some categories then you will want to have the same on the other databases.

Yes. Categories are per-database.
If you maintain multiple databases (which may affect 0.1% of all users) you will have to manually synchronize your categories between both databases.
-- Mario
IMatch Developer
Forum Administrator
http://www.photools.com  -  Contact & Support - Follow me on 𝕏 - Like photools.com on Facebook