Sort by part of file name?

Started by Carlo Didier, January 07, 2015, 07:47:19 PM

Previous topic - Next topic

Carlo Didier

I'd like to sort my files by the filename without the first character. Is that possible?

ubacher

My approach: put the trimmed file name into an attribute and sort by this attribute.
(Attributes show up in the list of options to sort by)

Carlo Didier

Quote from: ubacher on January 07, 2015, 11:24:38 PM
My approach: put the trimmed file name into an attribute and sort by this attribute.
(Attributes show up in the list of options to sort by)

I didn't have any need for attributes yet, but I tried to add one. Problem is, I can't automatically fill it with the truncated file name, at least apparently not at the definition of the attribute.
So I might have to fill the attribute by template or script?

sinus

Quote from: Carlo Didier on January 08, 2015, 10:08:16 AM
Quote from: ubacher on January 07, 2015, 11:24:38 PM
My approach: put the trimmed file name into an attribute and sort by this attribute.
(Attributes show up in the list of options to sort by)

I didn't have any need for attributes yet, but I tried to add one. Problem is, I can't automatically fill it with the truncated file name, at least apparently not at the definition of the attribute.
So I might have to fill the attribute by template or script?

You could automatically let fill a metadata-field by a metadata-template. There you can use variables like
<Run FontFamily='Tahoma' FontSize='12pt'  FontWeight='normal' >{File.Name|substr:1,11} </Run>

and then create a sort-profile for the choosen meta-field.
Quite easy to create ONCE, then you have done it and can use it all the time.
Best wishes from Switzerland! :-)
Markus

Carlo Didier

Quote from: sinus on January 08, 2015, 10:26:34 AMYou could automatically let fill a metadata-field by a metadata-template. There you can use variables like
<Run FontFamily='Tahoma' FontSize='12pt'  FontWeight='normal' >{File.Name|substr:1,11} </Run>

and then create a sort-profile for the choosen meta-field.
Quite easy to create ONCE, then you have done it and can use it all the time.

Sounds feasable, but I wouldn't want to apply a template to over 72000 files (as the metadata has to be stored in the files). That would result in triggering new full backups of ~1.2TB of data (1x on NAS, 2x on USB and 1x online) ...

sinus

Quote from: Carlo Didier on January 08, 2015, 10:58:40 AM
Quote from: sinus on January 08, 2015, 10:26:34 AMYou could automatically let fill a metadata-field by a metadata-template. There you can use variables like
<Run FontFamily='Tahoma' FontSize='12pt'  FontWeight='normal' >{File.Name|substr:1,11} </Run>

and then create a sort-profile for the choosen meta-field.
Quite easy to create ONCE, then you have done it and can use it all the time.

Sounds feasable, but I wouldn't want to apply a template to over 72000 files (as the metadata has to be stored in the files). That would result in triggering new full backups of ~1.2TB of data (1x on NAS, 2x on USB and 1x online) ...

Carlo, of course, every change has its drawbacks (mostly).
Forgive me, but I always wonder, why such things (changing 100'000 files, backups and so on) are that bad. I do such things over night, who cares, I am sleeping at this time. If I know, it works more than one night, well, then a weekend will do it.  8)
I have against 200'000 images (nefs,jpg, tifs...) and did such things several times. OK, security could be a problem because of saving a file several times, but to be honest, I personally have very seldom seen corrupted files, because I saved them several times.
Best wishes from Switzerland! :-)
Markus

Carlo Didier

Markus, it's not changing the files itself that bothers me, but the implications on my backups. A full online backup of 1.2+TB takes 2-3 weeks on my connection, many hours on the NAS and USB disks. And, due to the history of changes being kept, it takes a LOT of space on the NAS and USB disks. Fortunately, my backup software for the online backup and on the NAS is intelligent enough to just backup changes in the files, not the whole files, but still ...

And don't tell me I don't need all those backups. Backups are one of my main concerns at work, so I know how important they are and I couldn't sleep well without  :)

sinus

Quote from: Carlo Didier on January 08, 2015, 04:01:05 PM
Markus, it's not changing the files itself that bothers me, but the implications on my backups. A full online backup of 1.2+TB takes 2-3 weeks on my connection, many hours on the NAS and USB disks. And, due to the history of changes being kept, it takes a LOT of space on the NAS and USB disks. Fortunately, my backup software for the online backup and on the NAS is intelligent enough to just backup changes in the files, not the whole files, but still ...

And don't tell me I don't need all those backups. Backups are one of my main concerns at work, so I know how important they are and I couldn't sleep well without  :)

So you could do only one backup and .... ahem, sorry, could not resist, only a joke.   ;D

Of course backups are important.
OK, in this case I see, that this is a problem for you ... 2-3 weeks, uh, not possible, of course.

After this knowing, I am running out of ideas ... hm, maybe my tip simply without writing back? After all, it is only one metafield, if lost, it could be simply and easy renewed.

Another way, like ubacher said, by Attributes, filled by a script?

Or ... hm, running out of ideas!  ???
Best wishes from Switzerland! :-)
Markus

Ger

I would like to have the option to use a part of the filename for sorting as well. I do have a workaround, but that's not a 100% solution.

I will put in a feature request, after all we must keep Mario busy :-)

Ger

sinus

Quote from: Ger on January 08, 2015, 08:38:14 PM
I would like to have the option to use a part of the filename for sorting as well. I do have a workaround, but that's not a 100% solution.

I will put in a feature request, after all we must keep Mario busy :-)

Ger

Would nice to hear about your workaround.
Best wishes from Switzerland! :-)
Markus

Ger

I was afraid of that question, Markus!

As mentioned, the workaround is very specific. The normal sorting is simply on file name (format: yyyymmdd_hhmmss_zzzz-<text>.ext). The <zzzz> normally is the camera image number, but for some images (compilations, panoramas etc) it can be different and starting with a letter instead of a number. As I have Canon cameras and photograph in RAW, the extension of the masters is CR2, and alphabetically before TIF or JPG. So instead of fiel name, I sort on {File.Date} ascending and then on {File.Ext} descending with the result that images with the same file date and time are sorted in such way that the result files (JPG, TIF) are before the original(s).

Example:
20150109_075301_1234.CR2
20150109_075301_1235.CR2
20150109_075301_1234-pA01.TIF (a panorama of x1234 and x1235)

will be shown as
20150109_075301_1234-pA01.TIF
20150109_075301_1234.CR2
20150109_075301_1235.CR2

Hope it's clear.

Of course I thought about both mentioned solutions (metadata field and attributes), but obviously my problem wasn't big enough to fully explore them or spend the time to test :)

Ger

sinus

Hi Ger,

Ah, I see. Thanks for your good explanation.
Interesting, how we have different workflows, starting simply with different folderstructure and filenaming strategies.

At least, if it works, then all is fine.
Best wishes from Switzerland! :-)
Markus

Ger

Hi Markus,

Very true. I think the combined user group of IMatch uses many different workflows, all facilitated by IMatch. It's also reflected in the forum quality.

For most of us it's a long path from the start of cataloging images till where we are now. And along that road everybody made decisions  that turn out good or bad. Some you will correct, some you live with or find workarounds.  Also backgrounds are different, for me photography is just a hobby.

Ger


Carlo Didier

Just to explain my case: If I want to sort my files chronologically, I can't sort on the filename because it has this format: XYYYYMMDDNNN... where, evidently, YYYMMDD is the date, NNN a running number and X can be a letter like "D" for digital originals, "S" from scanned slides, "P" from scanned paper and some others to identify images taken by other family members. As I'd like sometimes to have all images of any sources together but in correct chronological order, I need to sort on just the date part of the filename. I can't use the "Date taken" or "Date original" metadata fields as these are not correctly filled out for scans and may often be totally incorrect for images from family members, but I usually set the correct information in the filenames.

Ger

Carlo,

As your case also refers to wrong meta data fields, it could be worth wile to update one of the File Date metadata fields with the correct date and time using a metadata template and then use this metadata field for sorting. What's more, the Timeline View then will (can?) become useful as well...

Guess it's a trade-off (backup time!) that can only be made by you.

Ger



Carlo Didier

Quote from: Ger on January 09, 2015, 09:56:09 AM
Carlo,

As your case also refers to wrong meta data fields, it could be worth wile to update one of the File Date metadata fields with the correct date and time using a metadata template and then use this metadata field for sorting. What's more, the Timeline View then will (can?) become useful as well...

Guess it's a trade-off (backup time!) that can only be made by you.

Ger

Yes, I thought about correcting the metadata, but for many of those scans I only have a date (best case) but often even not a complete date, but only a year and month or even just a year (worst case, quite rare though) which makes it a bit difficult to decide what to enter into the metadata (I simply put "00" for the day and/or month in the filename if I don't know more).

I wouldn't want to enter fictive dates (e.g. the first of the month), because that would hide the fact that I don't really know the exact date. Having "00" in the filename clearly points to the fact that the date is approximate, which in itself is an additional information. Of course, that information could be replaced by attributes or categories, but then it wouldn't be so obvious anymore.

Ger

I understand your point. I remember a very old threat in the IM3 forum (and probably you know it as well) where somebody explained his naming convention for these kind of images.

On the others side, if you don't know the day in a specific month anymore, how big is the chance that you confuse the actual date/time with the fictitious date/time: 19960501 00:00:01 for an image made in the month of May 1996 or an image made on May 1, 1995 at precisely 1 second after Midnight?

I did solve this issue like this for the (limited number of) scanned images I have in my database. And this way i can use the timeline view in IM5, which I like as well.

Ger

Carlo Didier

That works quite well if you don't have digital original files (from digital cameras) and scanned files from the same time span. Then you don't immediately see that some have a real "date taken" of, for example, May 1st whilst others only have that date assigned because the day wasn't known. I'd prefer to have the files without a specific day seperate from those where the day is correctly known.

In my case, I could live with your solution as I don't have digital originals from the periods where my negatives or slides were taken. But still, I hate to enter wrong data if I don't have complete data. It's just not clean.

sinus

Quote from: Ger on January 09, 2015, 03:28:08 PM
I understand your point. I remember a very old threat in the IM3 forum (and probably you know it as well) where somebody explained his naming convention for these kind of images.
Ger

I know, that Richard does work also with a lot of old pictures, where the correct data is not known, but he has a good solution. And also others too, but I do not remember as well.
Best wishes from Switzerland! :-)
Markus

Richard

Hi Markus,

You have a good memory. I do use fictitious dates and times. Except for New Years parties, the  chance that a picture was taken one second before or after midnight are extremely slight.

If I know a picture was taken sometime in 1864 I would use 18640630 11:59:59. In other words, as close as I can get to the middle of the year.

If I know a picture was taken some time in May of 1906 I use 19060515 11:59:59.

If the best I can do is guess that a picture was taken sometime in the 1860's or before, the date/time for that image would be 18691231 11:59:59. For me that date/time simply means that the picture was taken before 18691231 11:59:59.

A picture taken some time in 1870 or after would be 18700101 00:00:01.

sinus

Quote from: Richard on January 09, 2015, 04:40:59 PM
Hi Markus,

You have a good memory.

Thanks, Richard, for your post with the clarification. I am sure, this is a good solution.
But, BTW, my memory is unfortunately not that good as yours, I have detected this at several occasions, when you wrotes some posts.  8)
Best wishes from Switzerland! :-)
Markus