Sharing: file relations

[herman]

In this post I share the file relation settings I currently use.

As I am slowly growing into IM5 things may change (actually they have changed a lot already), but this is what I have now and what works for me.

The attachment shows the directory structure I currently use.
This structure will most likely change when IM5 becomes my production platform, for the beta test phase I just keep it as it is.

I can think of many simplifications and streamlining for these file relation definitions, however, I found this granularity helpful for beta testing. Once everything is settled I will go for the streamlining and efficiency.

I hope this may help those who struggle with the concepts of buddy files, versions and regular expressions.



Buddy files.

My old Canon Powershots could record a sound track to go with an image.
This relation definition covers the sound files recorded with a .jpg image

Relation definition name
JPG Buddy Files

Master Expression
\.(jpg)$

Replacement Expression
<empty>

Link Expression
^{name}\.(wav)$

Where to search
Master Folder - Specified Folder Only

-----

Raw files produced by my old Canon Powershots
A .thm buddy was produced by the camera, sometimes I shot in raw+jpg mode, so that a .jpg may also be present which I consider a buddy file.

Relation definition name
CRW Buddy Files

Master Expression
\.(crw)$

Replacement Expression
<empty>

Link Expression
^{name}\.(thm|jpg)$

Where to search
Master Folder - Specified Folder Only

-----

Raw files produced by the more recent Canon cameras
When I shoot raw+jpg I consider the .jpg a buddy file

Relation definition name
CR2 Buddy Files

Master Expression
\.(cr2)$

Replacement Expression
<empty>

Link Expression
^{name}\.(jpg)$

Where to search
Master Folder - Specified Folder Only

-----

The Canons could record some video format as well which was accompanied by a .thm file

Relation definition name
Video Buddy Files

Master Expression
\.(avi|mov)$

Replacement Expression
<empty>

Link Expression
^{name}\.(thm)$

Where to search
Master Folder - Specified Folder Only

-----

My Leica M9 creates .dng files as it's native raw format
When I shoot raw+jpg I consider the .jpg a buddy file

Relation definition name
DNG Buddy Files

Master Expression
\.(dng)$

Replacement Expression
<empty>

Link Expression
^{name}\.(jpg)$

Where to search
Master Folder - Specified Folder Only

-----

Corel AfterShot Pro creates sidecar files containing the processing parameters for an image.

Relation definition name
ASP Buddy Files

Master Expression
\.(jpg|crw|cr2|dng)$

Replacement Expression
<empty>

Link Expression
^{filename}\.(xmp)$

Where to search
Master Folder - Specified Folder Only

-----

DxO Optics Professional creates sidecar files containing the processing parameters for an image.

Relation definition name
DxO Buddy Files

Master Expression
\.(jpg|cr2|dng)$

Replacement Expression
<empty>

Link Expression
^{filename}\.(dop)$

Where to search
Master Folder - Specified Folder Only

-----

PictureCode Photo Ninja creates preview files in a subdirectory of the original file
It also creates XMP sidecar files containing the processing parameters for an image. These sidecars need no definition here as IM5 treats XMP sidecars as buddy files anyway.

Relation definition name
PN Buddy Files

Master Expression
\.(jpg|crw|cr2|dng)$

Replacement Expression
<empty>

Link Expression
^{filename}\.(preview)$

Where to search
List of Folders – Specified Folder Only – Pattern – {p0}\_PhotoNinja

[edit]
Please refer to this post for a better relation definition for PhotoNinja buddy files.
[/edit]




Versioning.

The file names I use evolved over the years. Some examples:
Images shot by my S40 are identified as YYYYMMDD_xxxx.ext (e.g. 20040330_0934.jpg)
Images shot by my G11 are identified as YYYYMMDD_IMG_xxxx.ext
Images shot by my M9 are identified as YYYYMMDD_Lxxxxxxx.ext

The best thing to do here would be some renaming, so that there is a clear and uniform file-naming structure.
Renaming can be done using the flexible and powerful IM5 renamer, and I plan to do that when I switch from IM3 to IM5 for production work.
Until that time the file relation mechanism in IM5 is flexible enough to accomodate the current state.
The common denominator between all original filenames is that there are always four numerical digits before the extension. This is reflected in the master expression.

I currently store the processed files in their own directory structure.
As refreshing IM relations is much faster when the derived images can be found in a sub-directory of their masters I may change the directory structure for performance reasons when I switch from IM3 to IM5 for production work,

Derived images receive a suffix indicating the program used to derive the image, e.g. _DxO or _PN.
Following this suffix there may be more suffixes like _mail or _web.
The important thing in this scheme is that a version always starts with an underscore following the filename.
My current definition allows for versions like xxxx_DxO.jpg, but also xxxx_PN_v1_web.jpg


Relation definition name
Versions

Master Expression
[0-9]{4, }\.(crw|cr2|jpg|dng)$

Replacement Expression
<empty>

Link Expression
^{name}_([0-9|a-z])+([_|0-9|a-z])*\.(jpg|tif)$

Where to search
List of folders - Down - 2 Levels – Processed folder

[attachment deleted by admin]

[clpratt]

Thank you so much for the buddy expressions given. I was at this moment struggling with a definition for linking my Canon AVIs and their THM files.
Sorted.
Chris

[claudermilk]

I have several in my test databases, but they are all very similar--mainly just changing the extensions to look for. The regex expressions are all based on Mario's NEF sample since they pretty much just work.

My directory structure looks like this:
<drive>
|_Photography
   |_YYYY
      |_YYYYMM
         |_YYYYMMDD - <Shoot name>
            |_RAW
            |_Working
            |_Done
               |_Web (these web directories moved over time, so I have to account for both)
            |_Web

Really only the last two levels matter due to using the relative paths in the definition.       

CR2 Versioning

• Link via: File Name
• Master Expression:
• \.(cr2)$
• Replacement Expression: ^_*//
• Link Expression: ^(_*{name})([+\-_]*[0-9|a-z]*)*\.(jpg|jpeg|psd|tif|tiff|dng)$
• Where to search: List of folders
• Direction: Specified Folders
• {p1}\RAW
• {p1}\Working
• {p1}\Done
• {p1}\Done\Web
• {p1}\Web
• Versioning/What to Propagate: Categories, Rating, Label, All Metadata

[jch2103]

For users of VueScan scanning software (subject to further refinement), to supplement the excellent posts above:

VueScan Buddy Files

Link via: File Name
Master Expression: \.(tif|dng)$ (VueScan can generate either TIF or DNG 'raw' files.)
Replacement Expression: ^_*//
Link Expression: ^(_*{name})[+\-_]*[0-9|a-z]*\.(jpg|jpeg|jpg\.dop)$ (I sometimes use DxO for additional processing.)
Where to search: Master Folder
Direction: Specified Folder Only (I keep all original and derived files in the same folder - for now)

VueScan Versioning

Link via: File Name
Master Expression: \.(tif|dng)$ (VueScan can generate either TIF or DNG 'raw' files.)
Replacement Expression: ^_*//
Link Expression: ^(_*{name})[+\-_]*[0-9|a-z]*([_|0-9|a-z])*\.(jpg|jpeg|psd)$
Where to search: Master Folder
Direction: Specified Folder Only (I keep all original and derived files in the same folder - for now)

[schwarzvogel]

I have several in my test databases, but they are all very similar--mainly just changing the extensions to look for. The regex expressions are all based on Mario's NEF sample since they pretty much just work.

My directory structure looks like this:
<drive>
|_Photography
   |_YYYY
      |_YYYYMM
         |_YYYYMMDD - <Shoot name>
            |_RAW
            |_Working
            |_Done
               |_Web (these web directories moved over time, so I have to account for both)
            |_Web

Really only the last two levels matter due to using the relative paths in the definition.       

CR2 Versioning

• Link via: File Name
• Master Expression:
• \.(cr2)$
• Replacement Expression: ^_*//
• Link Expression: ^(_*{name})([+\-_]*[0-9|a-z]*)*\.(jpg|jpeg|psd|tif|tiff|dng)$
• Where to search: List of folders
• Direction: Specified Folders
• {p1}\RAW
• {p1}\Working
• {p1}\Done
• {p1}\Done\Web
• {p1}\Web
• Versioning/What to Propagate: Categories, Rating, Label, All Metadata

Hallo zusammen,
ich versuche gerade Versionierung zu verstehen ...
Im Beispiel oben und im Hilfesystem (Thema: Folder Patterns) werden Platzhalter für die Ordnerhierarche verwendet wie {p0}...{p4} oder {d0}...{d4} für nur 4 Ebenen.
Ist "4" das Maximum ?
Wie und wo (in den "Einstellungen" ?) wird die Anzahl der Ebenen definiert ? (z.B. auch {p6}/{d6} für Laufwerk c:\)
Erkennt Imatch dies automatisch aus dem System ?
Im Hilfesystem oder der Community habe ich leider keine Antwort gefunden.
Vielen Dank !
mfG schwarzvogel

[Mario]

Ist "4" das Maximum ?

Nein.

Wie und wo (in den "Einstellungen" ?) wird die Anzahl der Ebenen definiert ? (z.B. auch {p6}/{d6} für Laufwerk c:\)
Erkennt Imatch dies automatisch aus dem System ?

Da verstehe ich die Frage nicht.
Die Folder und Directory patterns erlauben Dir, relative Pfade zu definieren. Üblicherweise kommst Du mit {p0} zurecht, weil das der Master folder ist, und mit {p0}\RAW usw. kannst Du Deine Verzeichnisstruktur abbilden.

[schwarzvogel]

Hallo Mario,
bitte entschuldige mein Mißverständnis.
ich versuche weiter zu verstehen ... 
mfG schwarzvogel

[Mario]

Schaur dir nochmal die Beispiele in der Hilfe an. Wenn es dann noch Fragen gibt, einfach posten. Am besten mit einem konkreten Anwendungsfall, das macht es mir leichter.

[schwarzvogel]

Hallo zusammen,
Es wird Zeit sich zu äußern.
Ich bekomme das Versioning inzwischen einigermaßen hin.
Zur Zeit übernehme ich nur "Alle Metadaten" und "XMP-Daten" in die Versionen.
Es ist aber seltsam, daß der Inhalt einiger Felder (bei mir z.B. 'Autor' und 'Keywords') sowohl im Master als auch in  den Versionen dupliziert oder sogar mehrfach geklont werden !
Dieser Fehler wiederholt sich leider immer wieder trotz manueller Berichtigung mit nachfolgender Aktualisierung.
Nur wenn dieser Fehler behoben ist, ist es nach meiner Meinung sinnvoll bei Bedarf weitere Metadaten oder überhaupt Kategorien in die Prozedur aufzunehmen.
Das Verwenden von Verzeichnismustern "{p0}" usw. funktioniert nach einer geringfügigen Änderung meiner Dateistruktur inzwischen auch.
Zu dieser Frage sollte der entsprechende Teil des Hilfesystems zum besseren Verstehen jedoch unbedingt nachgebessert werden:
1. Hinweissatz: "{p0}" bezeichnet  immer das Verzeichnismuster für den Ordner  der Master-Datei.
2. Die Beispiele sollten zeigen, daß es gibt auch andere mögliche Dateistrukturen als nur "c:\data\images\travel\summer". Es sollte mindestens noch eine beispielhafte Alternative angegeben werden.
3. Das Datei-Beispiel im Eingabefenster des Verzeichnismusters unter "Einstellungen -> Dateibeziehungen -> Detection -> ,,Muster hinzufügen" sollte flexibel das eigene Dateisystem des Nutzers zeigen !
mfG schwarzvogel

[Mario]

Please do not cross-post (posting the same text in multiple threads). Thank you.