Versioning strategy question

[paloscsaba]

Earlier in iMatch 3 I used to keep only my JPGs in iMatch, and stored the corresponding RAW files elsewhere.
Now I would like to migrate this database (containing mainly category info) to iMatch 5, and let iMatch 5 know where the RAW files reside.

I would also like to add some thousand newer RAW files and categorize them, but they don't have corresponding JPGs.

How would you set up the new database? For the older files the JPGs should be the masters, for the newer ones the RAWs (.MRW).

How can I retain the category information of the old files?

[Ferdinand]

For the older files the JPGs should be the masters, for the newer ones the RAWs (.MRW).

My initial reaction is that this is going to be the hard part.  I was going to say that I doubt that it's going to be possible, but on reflection it probably is, but with a complex file relations set-up.

There was a discussion some time ago about setting JPGs as masters and RAW as versions.  There were some complications with this approach, although I don't recall what they were.  It is possible to configure IMatch this way, but file relations seem to function most smoothly when RAW are the versions. 

One option would be to first import any older RAW, set up the file relations so that RAW are the versions, propagate any information from the JPG to the RAW, then reverse the definition.  Some people here in your position have done that.

As to how to configure file relations for the newer files, perhaps if you tell us how your files and folders are organised then we can advise.  My own view is that it's easiest when a version is in a subfolder of the master.  But IMatch is flexible and there are other ways.  However that said, there is one situation where IMatch has difficulty distinguishing masters and versions.  If your versions are in a completely separate folder tree to your masters, and if the file names are all the same other than the sufffix, e.g.  DSC12345.RAW and DSC12345.JPG, and there are multiple JPG versions named like this in separate folders, then there are issues.  My advice is to avoid this situation if possible.

[Mario]

Versions should be uniquely and certainly identifiable.
If you have masters and versions arranged in a sub-folder hierarchy, this is usually easy.
Also when you have masters in one folder hierarchy, and versions in another.

Problems usually only arise when you have a naming schema which makes it impossible to associate a version with the master (because you have used the same names for unrelated files) and the versions cannot be determined via their location (e.g. a sub-folder of the folder containing the master). But such situations are rare, because most users use the standard versions/buddies in a sub-folder of the folder containing the master folder.

@paloscsaba

Have have got yourself into a cinch there.

I understand that you have a database with only JPEG files. Now you want to add the corresponding RAW files and copy over the categories from the JPEG files.

You also have new RAW files without corresponding JPEG files, and you want to bring them into the database.

I suggest to try the following:

Configure a file relation in IMatch which makes the JPEG the master. Then add the RAW files for which there are JPEG. Let IMatch propagate (copy) the categories from the JPEG to the RAW files.

Remove the file relation.

Create a file relation which makes your MRW files the master, and JPEG the version.

Add the RAW files which have no JPEG.

This way you end up with a standard setup, where the RAW is the master and files you create from it later (e.g., JPEG) become versions.

[paloscsaba]

Dear Ferdinand and Mario,
wow, thank you for your quick and detailed answer.

I wish the company I work for would offer such a great support to its users...
Csaba

[Ferdinand]

@paloscsaba:  You're welcome.  It just so happens that your case has been discussed before, most recently in the last day or two.

Also when you have masters in one folder hierarchy, and versions in another.

Not necessarily.  If I have  DSC12345.RAW as the master file in one folder hierarchy, and several versions all named named DSC12345.JPG in another folder hierarchy, but (obviously) in separate sub-folders, then one or more of the JPGs may well be identified as the master ahead of the real master.

Is there really no chance that you (Mario) will ever add an option to constrain masters to be in a specific folder hierarchy, or perhaps a specific category?  Because without this, the people in this situation can't have versioning (propagation), and I'm going to have to find the time to migrate my old image synchronisation script so that they can.  I have some sympathy for such people, as they adopted an approach to naming and folders in good faith that was fine in IMatch 3.6, but as it turns it's not supported in IMatch 5, at least not without a large scale renaming / reorganisation exercise, which for many such  people is not practicable.

[Mario]

At least according to this community, there are not many users who run into this kind of problem. I think you or another user has made a feature request, but it's not on my top-list and I would need to search for it in the FR board. Versioning is already more complicated than it should be, by adding more stuff to handle yet another special case.

Adding another criteria to limit master detection to certain folders or categories will cause a whole rats tail of addition options. I can easily think up 5 to 10 "what if..." questions for either folder or categry case.

A proper folder hierarchy is so easy to maintain. A controlled file naming schema which avoids using the same file names for totally unrelated files is easy to setup and stick to. Even if you have duplicate file names, IMatch versioning gives you many tools already to keep them separate. That's more than sufficient. A few fringe cases will always remain, but then maybe these users need to refactor a bit, adapting their file storage and naming schema to support their current workflow.

Yes, I know, I know...there is always this and that. And good reasons for certain naming schemata. Things have grown over a decade years and cannot be changed...and all that. But refactoring from time to time is part of DAM, and usually does good in many ways.

My general drift is to keep things as they are for now. Of course, when a feature request is made and gets a ton of likes, that's different of course.

[DavidOfMA]

Adding another criteria to limit master detection to certain folders or categories will cause a whole rats tail of addition options. I can easily think up 5 to 10 "what if..." questions for either folder or category case.

Just to be clear, since I'm one of those other users, and I initiated the feature request, what I was suggesting was an optional criteria that users like me, of whom there appear to be only a few, could specify either a Masters folder or category as the only place to find Masters. Other users would be unaffected. But, as you say, so far only a couple of us have come forward with this problem.

A great appeal to me back in 2001 was that I didn't have to be overly concerned with file location or naming schemes because I could use categories to identify related files. So, I set up my folders in a way that best suited my project-oriented workflow and used categories to manage everything. This was confirmed when Ferdinand wrote his versioning script and dramatically improved that process, so I had no reason to reconsider this until IMatch 5, when a completely different way of organizing files was the underlying assumption. Nobody's fault, but too late for me to change the order of 130,000 files with filenames and locations embedded in various book projects.

David

[paloscsaba]

I also have some JPGs that have no RAW counterparts (pictures received from other people, smartphone pics).
Can these remain masters while I am defining my RAW files as masters?

I.E: if a RAW exist, RAW is master, else JPG is master?

Csaba

[Mario]

Can these remain masters while I am defining my RAW files as masters?

This is tricky. You need to avoid that the same file becomes both a master and a version. So be very careful with your file name masks and the folders you search for versions. If you create a setup which runs in circles later, strange things may happen. Make sure that you keep your RAW masters and JPEG masters separate and that IMatch can always uniquely identify them, e.g. by a strict naming schema or clear separation into different folder hierarchies.

[Ferdinand]

I also have some JPGs that have no RAW counterparts (pictures received from other people, smartphone pics).
Can these remain masters while I am defining my RAW files as masters?
I.E: if a RAW exist, RAW is master, else JPG is master?

A point you may not have noticed is that a file is only identified by IMatch as a master if a version is detected for it under the file relations rules.  An image on its own without any versions is not identified by IMatch as a master.

I suspect that some of what you want to do can be done with sophisticated file relations definitions, but getting those right may be a challenge for someone new to IMatch5.  Simplicity is the key to an easier life and less mistakes.

@David - I'll try to allocate time to migrate the script, but I have a several projects with deadlines at the moment.  One thing that would simplify the migration process is if I removed the code for propagating properties / attributes.  Unless someone screams, that's what I propose to do.