Find pictures of every event over all the years

[sinus]

Hello all
I have a problem where I don't know if and how this can be solved (automatically).
Because with 250'000 pictures it is difficult to do this "by hand".

All pictures have the same name structure:
20120329-1457-167232-s-gou-clouds_a
19840403-1555-032173-s-dar-picture_stone_m
20110601-0732-150549-s-gou-teacher_a
20110601-1344-250536-s-gou-event_a
20180816-1024-347077-s-coo-volk_a
20180817-0948-347205-s-coo-wildterrine_m_v1
20180817-1421-357205-s-coo-nopas_m_v1
yyyymmdd-time

Like most of you I take several pictures at one event.
Now I want to mark a picture (e.g. with a bookmark) of every event from all years, from every day.

That means in the structure above, it should always be the first picture of an event of one day.
This is differentiated by the date (20180816). If there is only one event on this day, then it goes on to the next day. If there is a second event on this day, then this second event is also marked on the same day.
And so on, year after year, month after month, day after day.

For the search with this structure these data are probably important:

20180816 for the date (year-month-day)
20180816-xxxx-xxxxxxxx--x-xxx-text (like above -wildterrine or -nopas, these are different events)

So from the above list all would be marked, because all were photographed on another day except 2 entries:
20120329-1457-167232-s-gou-clouds_a
20110601-0732-150549-s-gou-teacher_a
20110601-1344-250536-s-gou-event_a
20180817-0948-347205-s-coo-wildterrine_m_v1
20180817-1421-357205-s-coo-nopas_m_v1

but they have different text-description.

I don't know if I described my problem well.
Think, that you photographed yesterday one event.
Today 3 different events.

Then the result should be 4 files bookmarked, the first of each event.

And above all, I don't know how to solve it.
Does anyone have a clever idea for a solution?
A script would work, but unfortunately I can't script, so there might be some solution I didn't think of.

I know, this is a special problem and difficult to explain and to solve, but maybe someone of you has a clever idea.

I am still completely in the dark.

Thanks for thinking!
Markus

[Mario]

Maybe try with a data-driven category based on variables.

Level 1: {File.Name|substr:0,8} which groups by day
Level 2: {File.Name|substr:27} which groups by the part after the coo- or gou-

Impossible to exclude the _m_v1  this way, though.
This is why many use event codes which are always the same length and in the same spot.

Stuffing all this into the file name and not into real metadata is probably not a good idea.
If you would have a proper keyword strategy which uses an "Event|teacher" or "Event|wildterrine" keyword for each file, this would be super-easy. Or use categories instead of keywords.
Everything is easier than encoding all this stuff in the file name and then later trying to get it out again to actually use it for something. That's not what a DAM is for.

[sinus]

Maybe try with a data-driven category based on variables.

Level 1: {File.Name|substr:0,8} which groups by day
Level 2: {File.Name|substr:27} which groups by the part after the coo- or gou-

Impossible to exclude the _m_v1  this way, though.
This is why many use event codes which are always the same length and in the same spot.

Stuffing all this into the file name and not into real metadata is probably not a good idea.
If you would have a proper keyword strategy which uses an "Event|teacher" or "Event|wildterrine" keyword for each file, this would be super-easy. Or use categories instead of keywords.
Everything is easier than encoding all this stuff in the file name and then later trying to get it out again to actually use it for something. That's not what a DAM is for.

Thanks, Mario
Very interesting, I will try this out.

I have over the years tried several filename-strategies.
To be honest, this is far the best and I would not like to change it.

It is also very good for me to backup and once it safed me for example, when I was in a real hurry and I had trouble to opens IMatch (really, was the case, some monthes agao), I was able to pickup the correct files in theh windows explorer and edit them with PS and send them.

And last but not least, in the past (maybe now, with the new IMatch-search this is not more that important) IMatch was very quickly to search the filename, far quicker than searching for other tags.

But as always, I'll think about it, of course. For example, cut all the description in the filenames would be easy and also possible for me, because I have of course every event also in other metadata-fields.

Thanks really for your idea!!

[ubacher]

As to encoding data in the filename: this is how I started before Imatch. But categories is the way to go when you use Imatch.

Events: I have an EVENTS category tree. In this tree I have an Index to Events category. For each event I take one shot, a characteristic one,
and give it this index category. This way I can display EVENTS|Index to Events and get an overview (an index) of all events.
The thumbnail display I then use shows me the file name (which shows the date) and the folder name (which usually holds the name of the event).

When I started doing this I actually wrote a script to help me find events for which I had not yet set the Index to Events category.

(My events are ...B-day, Party, Fest, Concert, Theater, Parade, Wedding, Church-Religion-Tradition, Extreme-Weather, Other)

I do the same with a category tree which should be called Foto-Shoots.

[sinus]

As to encoding data in the filename: this is how I started before Imatch. But categories is the way to go when you use Imatch.

Thanks, ubacher, for your input!

Here I cannot fully agree. A DAM should take and work with every filename. IMatch does this of course.
As I pointed out above, there are situations, where you cannot use IMatch. Or another DAM.
For example, I have a notebook, IMatch works simply not with my huge DB.
Or I have an old PC at home, where my wife works ... with Windows XP.  No chance for IMatch.

Or I want backup or restore some stuff, here it helps me a lot, if I see not only the date, but also a SHORT description (usually one word).
Or I do transfer some pics on a stick ... great (for me) to see the name of files, I have only look at the filename in the explorer, without even starting a program, and I know, what it is.

All these things helped me a lot over the years. At the beginning of IMatch (2001) I had also only filenames with numbers.
But after some years I "studied" the whole stuff and I am still today sure, that it has a lot of advantages to add a small word into the filename.

The only thing,what could maybe even better, is to restrict this word to always the same amount of characters.
But at the end I could easy change this again (with IMatch), and until now this was not a real problem.

If I would start again, I would for sure take the same naming-convention.

But of course, for you and others this might not be good, each user has an own "best solution".

Events: I have an EVENTS category tree. In this tree I have an Index to Events category. For each event I take one shot, a characteristic one,
and give it this index category. This way I can display EVENTS|Index to Events and get an overview (an index) of all events.
The thumbnail display I then use shows me the file name (which shows the date) and the folder name (which usually holds the name of the event).

When I started doing this I actually wrote a script to help me find events for which I had not yet set the Index to Events category.

(My events are ...B-day, Party, Fest, Concert, Theater, Parade, Wedding, Church-Religion-Tradition, Extreme-Weather, Other)

I do the same with a category tree which should be called Foto-Shoots.

That is a good and clever thing, like you work.
I do the same and additionally I do work with (automatic) stacking.
Hence a monthly folder with 2000 images or so ends up with 10 or 30 images, each image is an "info-master" in my system (with additonally information about the event).
I do this since maybe 3 years and this works very good.
And with the new "stacking-unstacking" possiblity of IMatch it is even better.

But the problem are all the images before I did this and all the images, what are not from me.
And these images, what have not such a category or another remark, are really a lot.

Thanks, I will look how it is best to achieve a solution.

[sinus]

Maybe try with a data-driven category based on variables.

Level 1: {File.Name|substr:0,8} which groups by day
Level 2: {File.Name|substr:27} which groups by the part after the coo- or gou-

Impossible to exclude the _m_v1  this way, though.
This is why many use event codes which are always the same length and in the same spot.

Stuffing all this into the file name and not into real metadata is probably not a good idea.
If you would have a proper keyword strategy which uses an "Event|teacher" or "Event|wildterrine" keyword for each file, this would be super-easy. Or use categories instead of keywords.
Everything is easier than encoding all this stuff in the file name and then later trying to get it out again to actually use it for something. That's not what a DAM is for.

Your idea is a cool thing, Mario.
Only a remark: curious, if I use your 2 levels above, IMatch works forever (over 10 minutes), then I ended IMatch.
The same with the variables "headline" on the 2. level.

And the same with this two variables:
{File.Name|substr:0,8}
{File.Name|substr:26,9}

Again cancel IMatch (and starts again) and put this variables above instead in 2 levels, I putted it only in the first level as one line:

{File.Name|substr:0,8}{File.Name|substr:26,9}

Boah, super, after a few seconds all was done! And looks perfect for me, as yoou can see in my attachement.

The only thing still will be done "by hand" is to pickup the first image of each event and give it a bookmark. But ok, not all is possible with IMatch (except with a script).

Thanks for your idea!

[Ger]

Hi Sinus,

This solution is not as elegant as what's described above, but given it's a one-time effort, I would normally take a detour. Using the excellent export and import functions in IMatch. I would use the Copy Data app to export the filenames and import them into excel. In excel I would use a few simple formulas to define the firsts and then import the matching filenames (using the FileFinder app).

I have attached the excel formulas in case you would explore this solution.

Ger

[sinus]

Hi Ger,

Uh, thanks a lot for your idea and your effort with the formulas in Excel!
Very nice of you. 

I will look at it, but not sure, if I will use it.

[Mario]

Only a remark: curious, if I use your 2 levels above, IMatch works forever (over 10 minutes), then I ended IMatch.

Your database has 250,000 files. Data-driven categories based on variables are much slower than normal data-driven categories. Using a nested data-driven category with two variable-based levels on 250,000 files will run a while, even on a modern computer with 8 or 12 cores, lots of RAM and super-fast SSD storage. There is only so much IMatch can do.

Calculating 62,500,000,000 variables, performing the string functions etc. is a lot of work.
If you do this only on one level it reduces to 500,000 operations, which is of course much faster.

[sinus]

Only a remark: curious, if I use your 2 levels above, IMatch works forever (over 10 minutes), then I ended IMatch.

Your database has 250,000 files. Data-driven categories based on variables are much slower than normal data-driven categories. Using a nested data-driven category with two variable-based levels on 250,000 files will run a while, even on a modern computer with 8 or 12 cores, lots of RAM and super-fast SSD storage. There is only so much IMatch can do.

Calculating 62,500,000,000 variables, performing the string functions etc. is a lot of work.
If you do this only on one level it reduces to 500,000 operations, which is of course much faster.

Hm, I am sorry, it seems, I have not waited enough long. SORRY! 

At least the two variable above ({File.Name|substr:0,8}{File.Name|substr:26,9}) on one level takes about 15 seconds! Superspeed, I think.  and it does, what I want.

[Mario]

Yes. As I said. It's 250,000 operations when you use one level, but 62,500,000,000, which can take along time.
Data-driven categories can do magic and be fast even with two or three levels. Variables need to be evaluated individually, once for each file in the database. And when you have two levels, once for each file for each level at the level above. This can be 10 or 1000 times slower, depending on the elements on level 1.