I'm probably going to ask something stupid... Master without versions?

[nacho02]

Hi,

as mentioned. I've now more or less managed to get the versioning to work, and I realised that my files without versions, do not count as "Masters". Whicle I can understand the logic, I was seriously wondering why not? Does a Master need a version??

Versionless masters are... what?

I am thinking about adding these files as masters. Any suggestions as to what file relation I could use (if any?).

Thanks in advance,

Ignacio

[lenmerkel]

Versionless masters are... what?

In the context of IMatch, a Master is any file that has a relation to one or more other files. Those other files might be:

• Copies (e.g. an archive/backup copy of a working raw file, perhaps stored on removable media). This relationship is a version relationship. Your working raw file is the master, while the archive copy is the version. This might be done as a manual version link, for example as a Copy step in the Renamer.
• Derivatives (e.g. a JPG derivative from a raw file). This relationship is also a version relationship. Your raw file is the master, while the JPG is the version.
• Buddies (e.g. a sidecar file containing settings from raw conversion software - a DOP file, for example, from DxO Optics Pro). This relationship is a buddy relationship. Your raw file is the master file, while the sidecar is the buddy.

So, for a file to be considered a master, it has to have versions, buddy files, or both.

The Configuring File Relations in the IMatch help explains this in detail.

[P.Jones]

Hi,

as mentioned. I've now more or less managed to get the versioning to work, and I realised that my files without versions, do not count as "Masters". Whicle I can understand the logic, I was seriously wondering why not? Does a Master need a version??

Versionless masters are... what?

I am thinking about adding these files as masters. Any suggestions as to what file relation I could use (if any?).

Thanks in advance,

Ignacio

You could create a category like, 'Master without versions', this would help you find all of those Masters when needed, its what I do.

Then if you do create a version sometime in the future you can remove that category from the master photo as it will now become a Master in the eyes of IM.

[Ferdinand]

I have a lot of sympathy for the OP.  I mostly shoot RAW, and I regard all my RAW files (and occasionally a few JPGs) as masters, whether they have versions or not.  However IMatch does not agree. 

This issue was discussed a long time ago in the alpha testing phase, but Mario was resistant to the idea of a master without versions.

So I use categories for this kind of thing, which is what I did in V3.6.

[sinus]

From my point of view:

An image without versions is an image, but not a master! It makes not sense for me, to assign such an image as a master, if there is not a version, so yes, a master needs a version. Simple and logic, at least for me.

I think, this would make thinks more complicates.
Now we know, if a master has the icon to be a master, then there must be a version.

Want you sign all RAWs (all images except version?) as masters, you can do this easily with categories or collections.
For example create a cat, where all images belongs execpt versions. I do not know, if this is really possible, but I guess, there are ways to do so.
If not, then you could create a feature request.

But I wonder, why do you want to sign images, what have not version, as masters? Is there a specific reason?

I am glad, to have icons, what shows me exactly, this image is a master, this is a version, and if there is no sign, I know exatly, and this image has no versions, in my case this means, such images has not been edited, because not necessary or they are not good enough or whatever.

[thrinn]

I am more comfortable with the current approach, that is: a master is only a master when there is at least one version. I think this is an easy rule to understand. If this would change the question arises how files that are not masters should be defined or identified? It would not make sense to define all files as masters. If you need that just check the @All category . One approach could be do define all files at masters that have at least one relation rule defined that could produce versions. But then, would you want all relation definitions taken into account? Or only active ones?
Keep also in mind that it is possible to define version chains (a picture might be a version of a master and at the same time a master for another version). This sounds all very complicated to me.

[Erik]

Semantics aside,

I look at this situation in a similar manner as the original poster. 

1. If a file is not a master and not a version, it isn't related to any other file by definition set by IMatch
2. The files that fit this description are potential Masters in that they can only move into the master category as theoretically they should not become a version.

What you call these files is just semantics, but I do find they are important because these are often files I haven't processed, yet.  They are essentially the files I am looking for to become Masters.

I've felt like it would be nice to have these files as a collection to go with the Masters and Versions collections, but categories work fine for that purpose, too.  As Ferdinand mentioned, we've had these discussions in the past on the boards, and ultimately, its a minor issue (to me).

[sinus]

1. If a file is not a master and not a version, it isn't related to any other file by definition set by IMatch

Except stacked files

We have a lot different files.

A file becomes a master, if we create a version from this master.
Then we have a master and a version (or several masters).

They are indicated by icons.

All files without such an icon is NOT a master or version in a RELATION!

That's it. If you now want "collect" the files, what are not masters and not versions, I think, there are possibilities to do so in IMatch.

[nacho02]

Thank you all for your replies.

We are indeed talking about semantics here, but I did feel that whether a master has a version or not, it is (for me ) still a master, in the sense that it is the "digital negative". Yes, this applies also for non-raw out-of-camera images. These represent the source/quelle/origin/fuente... you know.

Now, I may, or may not process them further (thus creating versions), but regardless of this, these images represent the "originals".

Why would I want to mark them?

1.Well because I want to make sure that none of the JPGs (they are mostly JPGs) which I have in my DB are forgotten in the process.
2. When I have the images in  a folder, I usually use the viewer to rate them... I cannot do this with the versions, since the ratings and the rest of the info will not copy to the originals. So I choose originals (mostly by selecting a NEF or CR2 Filter). But these mean that I can easily forget the JPGs (from a different camera, or cell phone) which are also wanting to be rated and culled.
3. Because I want to count them.

I will create a data driven category in the form of "all images  <> Versions". I am trying to avoid data-driven cats, but why not.

Thanks!

[herman]

For what it is worth, I solved this problem (even before I knew it could ever be a problem, even years before I used IMatch) by setting up a folder structure (see attachment).

In my folder hierarchy "Originals" is just what it says: it contains only out-of-camera original files and their sidecars, created by raw converters.

The "Processed" folder contains only images derived from the originals: raw converter output, edited images, resized images, you name it.

With such a folder structure it is easy enough to create formula-driven categories and filters dealing with relations.
It is even a piece of cake to find orphaned versions 

[attachment deleted by admin]

[Erik]

1. If a file is not a master and not a version, it isn't related to any other file by definition set by IMatch

Except stacked files

We have a lot different files.

A file becomes a master, if we create a version from this master.
Then we have a master and a version (or several masters).

They are indicated by icons.

All files without such an icon is NOT a master or version in a RELATION!

That's it. If you now want "collect" the files, what are not masters and not versions, I think, there are possibilities to do so in IMatch.

Right, I was purposely keeping stacked files out of it.  Stacked files aren't defined through the file relations preferences and despite the options to create automatic stacks, the stacks are only grouped but metadata is never tied.

I was mostly just stating like the Op and others, I look at photos that are not part of a version or master relationship as essentially an original (whether they are part of a stack or not).  As an original, they can become a master if I derive files from them. 

In my 3.6 category scheme, I actually had three categories for versioning

Original
Original.Has Version
Version

I had used color coding so that I could quickly see whether a file is a version or not.  I'm doing something similar in IM5 for a couple of reasons.  1. The Icons for stacks and Versions aren't clear enough for me to see quickly and 2. I want to accommodate and identify the situation this thread is discussing.

And, with Sinus's comment, I actually have and still have a separate category tree for Stacks:

Stacks
Stacks.Panorama Components
Stacks.Bracketed Exposures
Stacks.Series (e.g. for a shoot or time lapse setting)
Stacks.Other (usually for graphics, composite images using Photoshop Layers, etc).

Stacks help for cleaning up my displays (couldn't do that with 3.6 too easily), but keeping my separate stack categories has helped remind me why I have images stacked.

And, per Herman's mention, I do have file locations and names set to identify my originals from versions quite easily.  Of course, with data drive categories or formulas based on the collections, it's fairly easy to set up.

[Ferdinand]

In my 3.6 category scheme, I actually had three categories for versioning

Original
Original.Has Version
Version

I think that's a useful way of thinking about it.  Originals is what I've always called them.  I guess in V5, "Original.Has Version" becomes "Original and Master".  I don't think you need this as a separate category now, as you can filter on the Originals category and Master Collection to get these files, can't you (assuming "Originals" covers all original files)?

My folder structure also separates Originals from Non-originals, although I think in a different way to Herman, and so it's harder to use in a formula category.  I mainly use my naming schema to identify originals and versions, although it's a little bit messy and should clean it up to make it easier.

[nacho02]

Hello again!

I think this is consolidating... good. I believe we are all just about in the same wavelength here.

It's interesting to see how you guys deal with this issue.

Thanks for the exchange!

[Ferdinand]

... as you can filter on the Originals category and Master Collection to get these files, can't you (assuming "Originals" covers all original files)?

This statement of mine was a late-night piece of silliness.  Masters are a subset of Originals, the subset with versions.

[Joe Austin]

I have the same view of raw files, they are 'masters' in my view until I create a .psd version of them that has enhancements I want to keep in any future versions of the file I may create.

I had a category scheme similar to Ferdinand's in 3.6, that used categories to describe what types of version a file was, and what types of version a file had.

In Imatch 5 I use this:

I have a category called "Mastering Versions".  this category contains the version of the file I want to send to PS, or ACR, whenever I want to process something more than a preview version.

For any given image the "Mastering Version" is the raw file if there is no .psd,   otherwise the .psd.   I also have .psd originals from slide scans that are "masters"   ,   I also have a few .jpg originals that are "masters"

My formula for the "Mastering Version" is:

"@All" NOT ( "@Collection[Relations|Master|raw->psd]" OR  "@Collection[Relations|Version|psd original -> jpg]" OR "@Collection[Relations|Version|raw->jpg]" OR "@Collection[Relations|Version|raw->nis]" )

This category then becomes the sole qualifier in a saved filter.   Invoking this filter will, within the scope of the file window, filter out all raw imatch-masters that have a .psd version, and all versions other than .psd's.

This leaves me with only raw files that have no .psd, all .psd's,  and jpegs that are not a version (originals)

If I do start creating multiple .psd's I'll have to revise this, but it seems to work for now, and I don't have to be meticulous in categorizing all my versions like I did in 3.6.

[Ferdinand]

"Mastering Versions" seems like a contradiction in terms to me.  I call these "Finals". 

[Joe Austin]

Hehe,  "A Rose, by any other name...."

Actually, in my scheme, such versions  are always used to create other end products so would never be "final".

[Erik]

In my 3.6 category scheme, I actually had three categories for versioning

Original
Original.Has Version
Version

I think that's a useful way of thinking about it.  Originals is what I've always called them.  I guess in V5, "Original.Has Version" becomes "Original and Master".  I don't think you need this as a separate category now, as you can filter on the Originals category and Master Collection to get these files, can't you (assuming "Originals" covers all original files)?

My folder structure also separates Originals from Non-originals, although I think in a different way to Herman, and so it's harder to use in a formula category.  I mainly use my naming schema to identify originals and versions, although it's a little bit messy and should clean it up to make it easier.

Based on your question about needing only one category, you're right.  Filters could take care of that need.  As it was, even in IM3.6, I only had two true categories as I used a formula to set up the third as all files not in the other two.

Currently, my originals and versions are separated by naming scheme, too, partially influenced by examples you and others posted long ago and partially to work with what I need.  I have originals that are JPGs from my pre-dslr days, so the file name is the only way I could distinguish JPGs that were originals from those that were versions.  With IM5, I've set fairly rigid file relation rules for identifying masters such that only images that match my naming scheme for Originals are allowed to be masters.  This mostly keeps me out of the situation where IM identifies a version as a master (because of a subversion).  This basically means that my files are either originals or versions, and some of the originals are also masters.

The question becomes how to I quickly identify these three types of files.  The version relation icons are ok, but I struggle with them (my eyes just don't catch them that well or confuse them with the stacks).  Color coding categories doesn't work if they are based on formulas like I had done in IM3.6, and labels only work if they don't propagate down to versions, which I'll implement using the script posted recently.

But, you give me a great idea for using a filter to identify separate the originals among those that are masters and those that are not masters.

... I realize that it is going to take me a while to figure out exactly how I should implement my workflow.

[Ferdinand]

... I realize that it is going to take me a while to figure out exactly how I should implement my workflow.

When you sort it out please let us know.  I'm still struggling with this.  Part of my problem is that to extract the part of the file name that tells me whether a file is an original or version and if a version then what kind, I have to use regex.  I have to use a script for this, as I can't get the regex to work in a dynamic category.  I'm contemplating a renaming exercise to solve this.  Ugghhhh.

[Erik]

... I realize that it is going to take me a while to figure out exactly how I should implement my workflow.

When you sort it out please let us know.  I'm still struggling with this.  Part of my problem is that to extract the part of the file name that tells me whether a file is an original or version and if a version then what kind, I have to use regex.  I have to use a script for this, as I can't get the regex to work in a dynamic category.  I'm contemplating a renaming exercise to solve this.  Ugghhhh.

I haven't gotten very far along that process, and I had actually assumed I would use a Regex to define my categories, although I'm not sure what you mean by a "dynamic" category.  Are you referring to a Data Driven Category?  or another type of category?

I just played with this a bit.  I'm not sure how what I've done would differ from what you've tried, but a category for files that are Originals AND Masters in my workflow would have a formula like this:

"@All" AND "@FileRegExp[^\d{8}_.{3}_\d{3,4}\.]" AND "@Collection[Relations|Master]"

My originals all have a format YYYYMMDD_CAM_NNNN.ext.  Versions, if they exist, have a -XN (where N is a number, X is a letter) tacked on one or more times to the original file name.

I'm not sure the formula above reflects exactly how I'll go about this process, but it is a start of how I envisioned working through it.  Note that in the example, I am not using an extension (my originals all have the same root file name structure independent of extensions since I do have many JPG and TIFF images that are originals from non-DSLR (non-RAW) cameras.

I think I'll be more likely to create a category Originals that uses the regex part of the formula and then a subcategory that is essentially the first category AND the collection portion.

I haven't played with it much, but for all my versions, I can stick with the @Collection portion and skip the file name part. This is partially because I've decided that I don't want any versions to be Masters as far as IM is concerned.  In terms of file derivation, I may not follow that as I often go from RAW to TIFF to JPG, but the file name will let me see the intermediate step, and I don't need IM to track that.

As it is part of the reason I haven't gotten too far into this is that I'm in a big file renaming overhaul as my file names slightly morphed over time, and I need them more consistent so the above and my file relations will work uniformly.  I also had a lot of dots in my file names and folders that I wanted to get rid of due to some issues I've had with certain software. 
Unfortunately, such a renaming operation has been a lot more work than I expected, so I definitely wouldn't recommend that exercise unless you really have to.  I did.

Last, I'm a little concerned about how performance might become affected as I do utilize a lot of formula categories.  I haven't seen an issue, yet, and I never did with IM3.6, but we'll see how it goes.

-Erik

[Ferdinand]

I had been trying to get a data-driven (dynamic) category to work, but your idea of a set of category formulas to work is better.  I have to know the values I'm looking for to create the set of formula categories, but that's not hard, as I know those.  The data-driven approach would save this trouble - but I can't get the data-driven approach to work as the required regex is too complex, whereas the formula approach is simple and works.

My naming schema, as you may have read, reserves the character "-" for versions, so anything without it is an original.  (There are a few exceptions, but those are easy to clean up.)  The first character after the dash indicates what sort of version it is, and the second character indicates how it was produced.  So I can easily create sets of dynamic cats using formulas like "@FileRegExp[-p]", with "@All" NOT"@FileRegExp[-]" for the originals, and "@FileRegExp[-.5]" for the raw converter.  Simple.  Thanks!!

I sympathise about the file rename exercise - I did a big one of those many years ago and got myself into a big mess.

I turned off the auto-update of data driven cats - even on a fast machine I found it annoying.  We will see what effect the formula cats have.

[Erik]

I had been trying to get a data-driven (dynamic) category to work, but your idea of a set of category formulas to work is better.  I have to know the values I'm looking for to create the set of formula categories, but that's not hard, as I know those.  The data-driven approach would save this trouble - but I can't get the data-driven approach to work as the required regex is too complex, whereas the formula approach is simple and works.

My naming schema, as you may have read, reserves the character "-" for versions, so anything without it is an original.  (There are a few exceptions, but those are easy to clean up.)  The first character after the dash indicates what sort of version it is, and the second character indicates how it was produced.  So I can easily create sets of dynamic cats using formulas like "@FileRegExp[-p]", with "@All" NOT"@FileRegExp[-]" for the originals, and "@FileRegExp[-.5]" for the raw converter.  Simple.  Thanks!!

I sympathise about the file rename exercise - I did a big one of those many years ago and got myself into a big mess.

I turned off the auto-update of data driven cats - even on a fast machine I found it annoying.  We will see what effect the formula cats have.

See... you reminded me how simple I could have set things up.  I basically use the same type of file naming scheme as you (at least down to the - for versions and the first letter indicating type of version.  I've never really worried about the how it's produced part much for file names besides having a tree of -'s to know when versions are derived from other versions).  As it stands, I could easily use the same simple RegEx as you are.  I'll have to look at that again.

I haven't looked much at using RegEx in Data Driven Categories, but the one advantage would be the ability to color code the categories, if that was desired.  With formula categories, we can't do that (although we can use Aliases I guess).

[sinus]

Yes, with a good file-schema, things can be very easy! I do also a good (I think) file-schema, though a long one.
But I can "fish out" easy master or version (like Ferdinand), the client, a unique number (besides date-time) and the event.

[Ferdinand]

I haven't looked much at using RegEx in Data Driven Categories, but the one advantage would be the ability to color code the categories, if that was desired.  With formula categories, we can't do that (although we can use Aliases I guess).

A dynamic category based on part of a file name works best if you can extract the portion you want by character position.  My file name is of the form:
date_time_cameraID_sequencenumber-versionID.xxx
and if the cameraID and sequencenumber were always the same character length then I could extract the character in position 19, say. 

But the mistake I made was that these two variables vary in length, and so the characters I want to extract vary in their position.  I have contemplated a rename to fix this, but it's a big job and with flow-on effects everwhere, since I store cameraID_sequencenumber in XMP/IPTC and in Attributes, all of which would have to be updated.

In data-driven categories there isn't full access to regex.  I can't just extract a pattern in the middle.  I have to find and remove the portion before the character I want and likewise for the portion afterwards.  It's been a while since I tried this, but I don't think I could quite manage it.  You may have better luck if youcan get away with a simpler, regex.

[Erik]

I can probably get away with it since my naming scheme ends up with fixed file name lengths for all original images, anyway. I essentially have the same file name format although each field in the name is a fixed size, which may be helpful.

As it is, I just finished my renaming and reorganizing process.  There are certainly a small percentage of my versions that need some name cleanup.  I just noticed as I wrapped up this process, that I hastily output files with default names (or don't change them) when processing, and lots of programs like to throw underscores to the original file name.  It's easy enough to fix, but not easy enough to find... Actually it will be easy to find, I just haven't done it yet.

Oh well, the next step is to actually put all the file relation stuff to work and categorization per this thread and other topics.  I'm a bit exhausted by just how "screwy" my files were (file names, spurious keywords, etc). 

[jch2103]

It's the sign of an excellent database program (e.g., IMatch 5) that it will reveal the 'garbage' that's been lurking unobserved for many years...