Data-driven category (time to build)

[axel.hennig]

I know, that this question depends, amongst other things, mostly on the hardware one is using, but I would like to get an idea how long the following takes on ohter computers:

Building a Data-driven category (XMP Date/Time) on 80'000 pictures (mostly *.jpg).

IM5 hasn't finished since approx. 30 min.

Hardware:
- 512GB SSD (Win8 (64bit), IM5, Pictures and DB on it)
- 8GB RAM
- Intel i7 2.60 GHz

[jch2103]

Building a data-driven category based on a date/time field is going to generate a very large number of entries (and take a long time), given the hours/minutes/seconds/sub-seconds involved. You may be better off using the built-in Timeline.

[Mario]

Building a data-driven category on XMP date and time is a really bad idea.
The result will most likely create one category for each file in your database, because the XMP Date/Time includes seconds and it only when two files were taken within the same second they will end up in one category. IMatch should abort the creating after 10,000 sub-categories have been created, to protect your database.

Use the timeline, which is much better at this.
Use a substring to create categories only based on the date portion etc.

The times required to create and/or update data-driven categories (updates are much faster because this usually does not create a massive amount of new categories) are written to the log file.

[axel.hennig]

Hello jch2103, hello Mario,

thank you for your fast reply.

It was my fault, I didn't explain exactly: I made a data-driven category from XMP Date/Time with 'Use part of value' -> '1,10' and 'Detect Hierarchies' -> ':'. So I get nearly the same as the timeline, except: I also used the 'Other' element to find images with no XMP date/time.

[Mario]

Just wait then until IMatch completes.
This will tell us (Attach the logfile) how long it took.
Loading the metadata for 80,000 files, extracting the date and time stamp, extracting the sub-string, applying the other options you have configured, build the groups, sorting etc. it's quite a bit. I would try it here but I have 34 other points on my to-do list for today (Sunday!). So well wait until you tell us how long it took on your computer. Should not take as long as I invested writing this reply.

[axel.hennig]

Attached is the logfile (the one with '_1.txt') when I created the first time the data-driven category. The logfile with '_2.txt' is the one after deleting the data-driven category and rebuilding exact the same one. The second time was much much faster (< 1sec ?).

@Mario: You do not have to test this. I was just interested how long this will take for other users/on other computers. There are enough other forum participants, perhaps someone else is also interested in this.

[attachment deleted by admin]

[joel23]

Attached is the logfile (the one with '_1.txt') when I created the first time the data-driven category. The logfile with '_2.txt' is the one after deleting the data-driven category and rebuilding exact the same one. The second time was much much faster (< 1sec ?).

@Mario: You do not have to test this. I was just interested how long this will take for other users/on other computers. There are enough other forum participants, perhaps someone else is also interested in this.

2.5 mins for 20.000 files here.

[axel.hennig]

Hello Joerg,
thank you for your reply. I made some further tests with data-driven categories (camera make/model, xmp rating, ISO-speed with numeric ranges,...) and everything now was really fast (< 5sec).

[joel23]

Hello Joerg,
thank you for your reply. I made some further tests with data-driven categories (camera make/model, xmp rating, ISO-speed with numeric ranges,...) and everything now was really fast (< 5sec).

You are welcome.
Yes, the others are much faster:
RefreshGroup-Completed: 'Maker' in 514 ms
RefreshGroup-Completed: 'ISO' in 546 ms