Configuring File Relations

File Relations allow you to handle buddy files and to control how IMatch copies metadata from master files to versions.
IMatch ships with pre-configured relation definitions for many standard usage scenarios. You can use these presets, customize them or add new definitions for your specific needs. This topic explains how to edit and create relation definitions.

See also File Relations for general information about how IMatch works with Versions and Buddy Files.

To configure file relations, go to Edit > Preferences > File Relations.

The File Relations Configuration Dialog

This dialog may look a bit daunting at first, but actually it is quite simple to use.

Performance Tip: Disable or delete all relation rules you don't need anymore.
IMatch has to process all rules whenever it scans or rescans a folder. Folders with thousands of files in combination with many relation rules can cause sluggish performance.

Relation Definitions List

On the left side of the dialog you see the list of existing relation definitions. IMatch automatically applies all enabled (checked) definitions when it adds or updates folders in your database or when you run the command Commands > Relations > Refresh Relations.

Buddy Files and Versions

IMatch automatically manages XMP sidecar files. You don't need to (and must not!) create any file relations for the .XMP file extension.

Combining Buddy Files and Versions

Usually, you create buddy file relations and version relations for the same type of files.

• If you create JPEG versions from a RAW master file, you usually want IMatch to keep the files together when you move/rename/copy the master file. This is done via a buddy file relation.
• You also often want to propagate metadata like rating, label or description from the RAW master to the JPEG version. This is controlled via a version relation.
• A file can be both a version and a buddy of a master file (and often is). You can combine buddy and version relations freely, as you need them.

There are some situations or workflows where a set of files is only a version set or a buddy relation. This is why we keep both types separate.

Adding and Deleting Relation Definitions

To add, remove and reorder relations definitions in this list, use the buttons below the list. When you click the + button IMatch shows a dialog where you select the type of definition you want to create.

Creating a new Relation Definition

Relation Order

The order of the relation definitions in this list is important. Relations are applied in the order they appear in this list, top to bottom. The last definition found matching "wins". This allows you to specialize relations, aka to handle version chains. Usually you don't need this but it is supported for special cases.

Version Chain

Consider you have a definition which declares .RAW => JPEG,DNG,PSD,TIF. And you have a definition which declares .TIF => .JPG.

If IMatch finds File.RAW and File.JPG and File.TIF in a folder, the JPG and TIF become versions of the RAW file. IMatch now continues checking the definitions and finds that there is a .TIF => .JPG definition. The result of applying these definitions is that the TIF becomes a version of the RAW and the JPG becomes a version of the TIF.

Renaming a Definition

Relation definitions cannot be renamed directly. If you want to rename a relation definition, just create a new definition and select the original definition in the Copy settings from combo box (see above). Enter a name of your liking for the new definition. This creates a copy of the original definition under a different name. You can then delete the original definition.

Keep It Simple (KISS).
Although IMatch file relations are very powerful and allow to handle even complex cases, chances are that you only need something very simple.

Detection

The Detection tab holds controls which allow you to define how and where IMatch looks for related files.

A version set. The .RAW file is the master and it has 4 versions / buddy files.

Link Via

This drop-down list allows you to choose how the relation definition finds matching files. The available choices are:

Matching File Names

The three fields Master Expression, Replacement Expression and Link Expression work together to find related files based on the file name of the master file. All three fields support regular expressions for a maximum of flexibility.

See the topic on Regular Expressions for more information.

Before you start, you need to answer these questions:

• What are the names of my master files?
  Which extension do I use for my master files?
• What are the names of the version files?
  Do they use the same name as the master, just with a different extension?
  Or do I append additional letters or digits to the master file name to create the version file name?

The simplest versioning schema would be something like: When my master file is named image.raw, the JPEG version of it is named image.jpg. Very easy. Another common schema is to append additional letters or digits to the version name: image_web.jpg, image123.jpg or image_final.jpg.

Having the same file extension for master and versions (e.g. some JPEG files are masters while others are versions of these masters) will cause problems because there is no way to tell by looking at a folder whether a file is a version or a master. This all then depends on which folder is processed first, and it cannot be guaranteed that the folder containing the versions will be processed after the folder containing the masters (e.g. when IMatch reacts on file system notifications about modified files and folders). Always use different file formats for masters and versions, or at least use a naming schemata which allows to tell whether a file is a master or a version - e.g. by adding a suffix to the version name: file001.jpg (master) file001_web.jpg (version).

Master Expression

This expression defines how IMatch detects the master files for a relation definition. The simplest expression is:

\.RAW$

This expression finds all files ending with .RAW. Just replace RAW with the file extension used by your master files (e.g. .NEF, .CR3, .ORF, .RAF, ...) and you have a good start.

The period . has a special meaning in regular expressions. We have to escape (override) its meaning by adding a \ in front to tell the regular expression parser that it should consider it as a literal (real) dot.

The $ tells IMatch that the file name must end with the given expression (in this case, end with .RAW).

More Examples

\.(nef|nrw)$

This expression is typically used for Nikon raw files. Nikon uses .NEF and .NRW as extensions, which are both covered in this regular expression. It reads "Match all names ending in .nef or .nrw". The regular expressions are not case sensitive, they will match .NEF as well as .nef.

For Canon users:

\.(crw|cr2)$

For Sony users:

\.(srf|sr2|arw)$

and so on.

Doubled Extensions

Some applications append their own extension to the file extension. For example: .NEF.BIB or .CR2.DOB, .NEF.DXO.

We can deal with this easily following this schema:

\.(nef|nef\.bib)$

This matches .nef and .nef.bib

\.(cr2|cr2\.dob|cr3|cr3\.dxo)$

This matches .cr2, .cr2.dob, .cr3 and .cr3.dxo

The Link Expression

The link expression defines how IMatch finds versions for a master. Again, you use regular expressions to be as flexible as possible. IMatch supports some additional special tokens in regular expressions used for links:

These tokens are used to refer to the master file name in the regular expression of the link. IMatch replaces these tokens before applying the regular expressions.

If your version has the same file name as the master file, just a different extension (image.raw =>image.jpg), the link expression is also simple:

^{name}\.jpg$

This already is a valid regular expression for a link. If the master file has the name:

File123.RAW

this expression becomes:

^File123\.jpg$

which matches the file name "File123.jpg". This relation definition would thus match JPG files with the same name as the master file.

The ^and the $

We use the ^ to ensure that the version file name starts with the name of the master file. This is to prevent the accidental matching of a file named bigpicture.jpg to the master file picture.jpg. We also use the $ regular expression token to ensure that the version file name ends with the .jpg extension. This prevents accidental matches for files using multiple extensions, like .jpg.cfg, which are produced by some image editing applications to store custom settings.

This following expression would match all .TIF files with the same name as the master file:

^{name}\.tif$

If you use multiple extensions for your version files, you have to update the expression to:

^{name}\.(tif|tiff)$

and if you want to link both TIF and JPG files to a master file you would use:

^{name}\.(jpg|jpeg|tif|tiff)$

This expression matches all files with the same name as the master and one of the extensions .jpg, .jpeg, .tif or .tiff.

When your version file names have additional letters or digits

If you append letters or digits to your version file name, you can cover this by changing the link expression slightly:

^{name}.*\.jpg$

By adding the .* to the expression, you tell IMatch that the original file name can be followed by any number of letters, digits or other characters. For a master file like image.raw, this allows for version file names like image.jpg, image_web.jpg, image123.jpg or image_final_123.jpg.

Using only a part of the Master file name

Use this when the version file name contains only a part of the master file name.

If the master file name is longer than the version file name, things become a bit more complicated. Consider the following example:

20140101-78921.raw  (Master)
20140101-web.jpg (Version)

These two file names cannot be matched with the default settings. For example, if you use a link expression mask like

^{name}.*\.jpg$

IMatch first replaces {name} with the name of the master file, and then evaluate the regular expression to find matching versions. For our current example, the regular expression becomes:

^20140101-78921.*\.jpg$

and this can never match version file names like 20140101-web.jpg. The -78921 part of the master file name gets into the way. This part is not contained in the version file name. To handle this situation, you can append a postfix in the format :n to the {name} pattern (also to {ext} and {filename}). n specifies the number of characters to use from the original file name when replacing the {name} token.

We use this for our example:

^{name}:8.*\.jpg$

This tells IMatch to replace {name} with the first 8 characters of the master file file name. The regular expression hence becomes:

^20140101.*\.jpg$

and this matches version file names like 20140101-web.jpg just fine.

Advanced Examples

The following expression is a full-fledged, cover-all-bases expression, if you use very complicated naming conventions for your version files:

^(_*{name})[+\-_]*[0-9|a-z]*\.(jpg|jpeg|dng|psd|tif)$

Note the '\' in front of the '-' in the [+\-] part. This is required because the - has a special meaning when used inside [].
See Regular Expressions for more information.

The above expression matches all JPG, JPEG, DNG or PSD or TIF files with the same name as the master file. In addition, it handles file names created from the master by appending letters or digits, e.g.:

Master: _DSC1234.RAW

_DSC1234.jpg
_DSC1234_1.jpg
_DSC1234_New.jpg
_DSC1234-Final.jpg
_DSC1234-001.jpg
...

The [+\-_]* allows for 0 or more + - _ after the master name. The [0-9|a-z]* allows for 0 or more letters or digits.

Replacement Expression (Advanced Topic)

What this expression cannot cover is a version file name like:

DSC1234.jpg

That's just the name of the master file but without the leading underscore. This file name will never match the regular expression above.

Remember, the regular expression (simplified) is:

^(_*{name})\.jpg$

and {name} will be replaced with the name of the master file, which gives us:

^(_*_DSC1234)\.jpg$

The _DSC1234 part in the regular expression enforces that the version file name begins with an underscore. There is no way around that, at least not with a regular expression.

We stress this issue because it is a common case. The leading _ is often used to denote files in AdobeRGB color space. When a RAW file is converted and the color space is changed to, say, sRGB, image editors often remove the leading underscore from the file name.

Since this situation cannot be handled by a single regular expression, IMatch uses a second regular expression to "clean" the master file name before it is inserted into the link regular expression.

A Replacement regular expression follows this format:

/replace this/with that/

Where the "with that" part can be empty: /replace this//
You can add multiple replacement expressions: /a/b/c/d/_DSC//   (replace a with b and c with d and _DSC with "nothing")

For example:

/^_*//

This replacement expression replaces all occurrences of _ at the beginning of the master file name with "nothing". A file name like

_DSC1234.nef

will be transformed to

DSC1234.nef

before inserting it into the link expression. In our example above, we end up with

^(_*DSC1234)\.jpg$

which is what we need. Since we do not require a leading underscore any more, link file names like "_DSC1234.jpg" and "DSC1234.jpg" will both be possible.

Another example would be a (rather strange) master name like:

DSC1234-master.RAW

Leave the replacement expression empty if you don't need it. This improves performance when IMatch scans folders to apply relation rules.

Testing Expressions

Regular expressions are powerful but may become complex and hard to understand by just looking at them.

IMatch provides a cool test feature which allows you to try out your expressions. Click the Test Expressions button to open the Test Expressions dialog box:

This dialog allows you to try out all three types of expressions. The Master Filename and Version Filename take the sample names you want to test. Fill in a master file name and click the Test button to see if it matches the Master Expression defined in the dialog. Fill in a Version file name and click Test to find out if it matches the Link Expression and Replacement Expression.

Click OK to close the dialog and to copy the expressions into the Relation Definition properties.

Where to Search

In this section of the dialog box you specify where IMatch is looking for buddy files and versions. IMatch can search only the folder containing the master file, include sub-folders of that folder in its search (one or more levels deep) and even look in selected folders or the entire database.

It is advisable to keep relations rules as simple as possible, and to use only rules that are really needed.
IMatch has to process all rules every time you change something in a folder (e.g. add, update or remove files). If you work with folders containing thousands of files this can require quite some time and may slow down overall IMatch performance.

Master Folder

This is the usual folder to look for versions. With the drop-down box Direction you can control whether IMatch also searches sub-folders of the master folder, and how many levels.

If you only want to search the master folder, select Master Folder in the Where to search box and Specified Folder only in the Direction box. This limits the search to the folder containing the master file.

If you keep versions or buddy files in sub-folders of the master folder, choose Down in the Direction box and enter 1 or more levels in the Levels box. This tells IMatch to look in the folder containing the master file and in one or more levels of sub-folders.

Master Folder         | Master folder only
|-- Settings          | Master folder, down 1 level
|-- Edits             | Master folder, down 1 level
|-- Output            | Master folder, down 1 level
  |-- Print           | Master folder, down 2 levels
  |-- Web             | Master folder, down 2 levels

Looking in the Master folder and one or two levels of its sub-folders is sufficient for almost all cases. It covers buddy files which are usually in the same folder as their Master file, and the usual scenarios where applications keep buddy files a sub-folder of the Master folder.

If you want to work efficiently with versioning and stacking, you should use a simple file and folder hierarchy. Keep master files and derivatives like buddy files and versions within the same folder or a sub-folder hierarchy not more than two levels deep. Do not spread your files all over your hard disk.

This not only makes versioning easier but also tasks like backup and archiving final files.

List of Folders

If you use a more complicated folder structure, or you spread your files over different disks or network resources, you can also handle them with IMatch. Select List of Folders in the Where to search box. Now you can use the drop-down button on the right of the folder list to select folders from your computer or add folder patterns.

Select Folder

This command opens a standard folder selection dialog where you can check one or more specific folders to include. The dialog allows you to select folders with or without including sub-folders. Selecting fixed folders is sometimes useful when you keep final versions of your images on a specific drive or folder on your network server, and you want to let IMatch look for versions there too.

IMatch has to search all folders you specify here whenever it rescans the master folder or you run the Commands > Relations > Refresh command.
IMatch rescans folders when you write back metadata or other applications make changes to the file system. If you specify too many folders to search for related files, or these folders are on slow media like network servers or NAS storage, you may slow down IMatch considerably.

Again, keeping a simple folder structure not only makes it easier for you but also for IMatch.

Folder Patterns

With the Add Pattern command you can add a new folder pattern. Folder patterns allow you to specify folders by using parts of the master folder name.

See also Folder Patterns for additional information and examples.

Consider this folder hierarchy:

We assume that the blue folders "Beach" and "People" contain the master files and that the "Edits" folder contains different versions of the master. The "Settings" folder contains buddy files created by a RAW processing application. The yellow "Archive" folder has two sub-folders containing the archived (final) versions of the images in the blue folders.

If you set up your relation definition to search in the master folder and its sub-folders, IMatch will find all buddy files and versions of the master in "Edits" and "Settings". It will not find the archived versions in the yellow "Archive" folder hierarchy however.

The "Archive" folder is on the same level as the Master folders "Beach" and "People", and hence IMatch would have to search upwards from the master folder and then again down to find archived versions.

Constructing folder names to search for buddy files and versions using folder patterns is an advanced topic and you probably will never need it. But if you have a complex folder structure or you spread versions over multiple disks, folder patterns are indispensable.

Settings made for Direction and Levels only apply to folders indexed by your database.
When you use folder patterns to specify folders not contained in your database for buddy files you have to explicitly specify each of these folders via a folder pattern.

This is where folder patterns come in. These special constructs allow you to produce folder names from the folder name of the master file and literal text. If you choose the Add Pattern command, IMatch opens a dialog box where you can enter a pattern.

The Folder Pattern dialog box

See also Folder Patterns for additional information and examples.

Using Patterns

Back to our example from above. We want to find all versions for a folder like "Beach" in its sub-folders and in the corresponding "Archive" folder:

Using folder patterns we have several ways to achieve this goal:

c:\data\images\archive\{d0}

would be one. A runtime, when IMatch is searching for versions for a master file in "Beach", it expands the above path to

c:\data\images\archive\Beach

because {d0} always returns the name of the folder containing the master file.

Another, more flexible solution would be to specify the path of the Archive folder relative to the folder containing the master file. This way we can move the entire folder hierarchy to another disk or into another folder (than c:\data\images) and all relation definition rules would still work:

{p1}\Archive\{d0}

{p1} returns the parent folder of the folder containing the master file. {d0} as before is the name of the master folder. Together with some literal text this resolves to:

c:\data\images\archive\Beach
    
or, when the master file currently examined is in "People":

c:\data\images\archive\People

Manual Versions

If you choose this type for your relation, IMatch does not attempt to find masters and versions. You link masters and versions manually via the corresponding commands provided in the File Window context menu or the associated keyboard shortcuts.

To link files, you first select the file(s) that will become the version, and finally you select the file which will become the master (the focused file will always become the master). Then you press Shift+Ctrl+M to create the master and version link.

IMatch will bring up a dialog box where you choose which of the existing version relation definitions you want to use for your manual link:

The properties and settings of the relation chosen here control the actions IMatch later performs when you change the master, e.g. which metadata or other properties are propagated.

When to Use

Manual versions are not often used, but when you need them it is very good to have them. For example, of none of the automatic mechanisms to detect master and versions works for your files. Or you want to have special propagation rules only for some of your files. Or maybe you want to manually define a visual proxy for specific files in your collection. All that can be easily achieved with manual versions definitions.

It is advisable to keep relations rules as simple as possible, and to use only rules that are really needed.
IMatch has to process all rules every time you change something in a folder (e.g. add, update or remove files). If you work with folders containing thousands of files this can require quite some time and may slow down overall IMatch performance.

Using Variables (Advanced Topic)

You can optionally further limit which files are considered as versions by using IMatch Variables. This can be used to detect versions from the metadata they contain, e.g., the name of the software used to create the file, the camera or photographer name, ISO value, ICC profile or any other metadata tags and IMatch variables. Or maybe you only want consider files to be a version which have a specific rating, label or category.

This is an advanced topic and not often used in typical workflows. But it can be extremely helpful to implement complex versioning not covered by any of the other options available in IMatch.

Example: Restriction versions to files with a Blue label.

There are three parts to consider:

• The Master Expression, which can be a variable (referencing the master file) or a literal (text or number). This is evaluated once, for the master file.
• The Version Expression, which is evaluated for each file designated as a version by the other options in this dialog.
• The comparison operator.

It works as follows: For each detected master, the Master Expression is evaluated. For each version file detected. the Version Expression is evaluated. Then the two results are compared with the selected comparison operator. If there is a match, the file is considered a version.

In the example above, the Master Expression is simply the text Blue. For each potential version, the Version Expression variable extracts the label of the file. The result is compared with the word Blue. If there is a match, the file becomes a version.

If you instead use the variable {File.MD.XMP::xmp\Label\Label\0} for the Master Expression, only files with the same XMP label as the master will become versions.

Variables and Buddy Files

When IMatch looks for buddy files it not only looks for files indexed by the IMatch database, but also for files in the file system. In many cases buddy files are secondary files (like settings or configuration data) produced by other applications and you don't include these files in your IMatch database. Still, IMatch needs to find the buddy files and process them (move, copy, rename, ...).

Since these external buddy files have no representation in the database, IMatch cannot consider variables when searching for these files. If you configure a buddy file relation which uses variables for filtering, keep that in mind.

The Versioning Tab

This tab is enabled for relation definitions of type Version (definitions with the [V] prefix in the relation definition list).

In this tab you configure what IMatch does with versions matching this definition. This includes Propagation and Uses.

Configuring Propagation and Uses in the Versioning Tab

Propagation

Propagation is the process of copying metadata from the master to its versions. IMatch can copy categories, collections, attributes, annotations and metadata from the master to versions. In the What to propagate list you can enable individual attributes you want to propagate.

If there is more than one propagation set, e.g. for copying XMP data, select only one in the list. If you select multiple sets, all the data of these sets will be propagated. The result may not be what you expect! It is highly recommended that you try out propagation of metadata with a couple of test images first. Open the resulting files in all your favorite applications to see if they can still be read and if the resulting metadata is exactly what you expected.

Please see the corresponding topic for details about propagation.

Common Propagation Pitfalls

IMatch data like collections, categories, Attributes etc. is usually safe to copy.
Before you start copying metadata like IPTC, EXIF or XMP between files, make sure you understand the metadata format you work with and the consequences of copying metadata between files.

IMatch gives you fine control over which data you propagate (copy from the master file to its versions). All the options like "Copy XMP data" or "Don't copy XMP Regions" may sound intimidating - but they are needed.

Here are some common pitfalls to consider:

Copying EXIF data between different files is usually a bad idea. EXIF data was just not designed for this. EXIF contains camera settings and other data that is linked to the image as it was created by the camera or scanner. This includes things like the width and height of the image, its orientation (rotation) and other measures which usually change when you produce a new file from a RAW image. Especially copying the EXIF orientation or camera vendor maker notes between files often leads to unwanted side effects. If you really have to copy EXIF data, use the EXIF (Safe Tags only) option when possible.

Another pitfall is copying propriety 'settings' metadata, e.g. the crs namespace Adobe puts into XMP records when you process a file in LR or ACR. This information is tied to the RAW file. Copying it to another file (e.g., a JPEG you've created from the RAW) is useless and may even confuse LR or other Adobe software when you open the JPEG in them. IMatch offers options to strip this kind of proprietary XMP data when copying.

Copying only XMP data when the target file also contains legacy IPTC data is another typical problem case. If you copy only XMP data, the IPTC data in the target file remains unchanged. Since IMatch imports legacy IPTC into XMP during the following import, the old IPTC data in the version will override at least some of the XMP data copied over from the master. Solution: If your versions have IPTC data, always enable the option to copy IPTC data for propagation when you propagate XMP data. If the master has no IPTC data, IPTC data will be deleted in the version as well. Else the master IPTC data will be copied, ensuring that XMP and IPTC data are synchronized properly.

Categories

If you propagate categories, you must choose which categories you want to propagate in the Categories tree. If you want to propagate all categories, just check the @All node in the tree. Otherwise, check specific categories.

Via the check boxes you can include and exclude individual categories, and also decide whether or not you want to include child categories.

The following check states are available:

Uses

In the Use this version list you can control specific uses for the relation definition. Uses include visual proxies or specific versions you designate for printing or email purposes.

The following uses are supported:

Color

The Color control allows you to define a color for a relation definition. If you enable the color, the relation icon in the file window will show an outline in the chosen color.

Standard relation icons (top) and colored relation icons (bottom, yellow color)

The idea behind using colors is to make it easy to determine which of your relation definitions was used to link a master (left) to its version (right). Using different colors for different relation definitions gives you an additional visual clue right in the file window.

Refreshing Relations

Whenever you make a change in your File Relations, IMatch needs to refresh the relation data in the database. This operation updates links between master and versions, updates proxies etc. You start this process via the Commands > Relations > Refresh Relations command.

This command works on the selected object(s) and its children in the Media & Folders view: a folder, a media, drive or the entire database. Select the object containing the master files and then run this command.

You can speed up the process by refreshing only the relations in certain folders. If you are unsure, select the Database node and refresh the relations in your entire database. It is important to run this command on the folder/category etc. containing the master files.

The refresh process is usually fast, unless you have specified very complex rules or IMatch needs to scan very deep folder hierarchies to detect versions.

Performance Tip: Disable or delete all relation rules you don't need anymore.
IMatch has to process all rules whenever it scans or rescans a folder. Folders with thousands of files in combination with many relation rules can cause slow performance.

Automatically Refreshing Relations

When IMatch adds or updates folders in your database it checks if one or more of the added/updated files are master files. If this is the case, IMatch processes all relation definitions to check for versions of these masters.

IMatch does not, however, check if one of the added/updates files is a version. Doing so would slow down the add/update process considerably. Because the master file(s) for the new/updated files could be anywhere. In the parent folder. In a folder several levels above. Maybe even on another disk or somewhere in the database.

If you add new versions to your database, you have to perform a manual relation update of whatever folder contains the master file(s) for the new version files. Select the master folder in the Media & Folders tree and then press F4,R to refresh the relations and to link the new versions to their masters.

You can disable the automatic relation update in the Background Processing options, if you have to for some reason.