Problem with Version Relation and Folder Patterns

[banzai]

Hi there,

I have a folder with .mp4 files. For each .mp4 file there is a subfolder named like the video file (without extension) containing .jpg files belonging to the equally named video. 

base [folder]
 - 123.mp4
 - 124.mp4
 - 123 [folder]
    - 001.jpg
    - 002.jpg
 - 124 [folder]
    - 001.jpg
    - 002.jpg
      


I tried to setup a version relation for this:

master expression: "\.MP4$"
replacement expression: ""
link expression: "\.jpg$"

where to search: list of folders
direction: specified folder only
list:
"{p0}\{n}"


But somehow this does not work, no versions are found.
I suspect the folder pattern is incorrect, but from reading the docs, I figured it should resolve to the path containing the jpgs.
{p0} should give "base" in my example.
{n} should give "123" for the first potential master (123.mp4)

And the folder base\123 contains the correct images.

So what am I missing here?

[Mario]

File Relations currently only support the {p*} and {d*} tokens. See the dialog when you add a pattern.
Other IMatch modules also use folder patterns but also support the file name-based patterns.

IMatch only considers the folder containing the master file when determining where to search for versions based on the relation definitions.

I frankly have never seen a folder schema like the one which you use.

It seems the names of your versions are just numbers and have no relation to the master files in the folder above them. And that you have any number of uniquely named masters in the base folder, but the names of the versions are the same for different masters and are all on the same level (one folder below the master).

This makes this impossible to handle detect for IMatch at the current time.

Usually you have either file names which link versions and masters (same name or derivations) or the versions are in a sub-folder of the master.

A folder schema like yours came never up before. I would need to enhance the "folder name finding" routine to support {f} and the other file name-based tokens to support this rather unique schema.

Not sure if this would impact performance of matching (IMatch frequently has to match millions of master-version pairs) when users index new folders...

Currently, when IMatch scans a folder for masters, it determines the folders which may contain versions for the masters in this folder. Then it reads the file names in the folder to find masters, and for each master, checks the potential version folders for versions.

To support your schema, IMatch would have to change this to first identify the masters, and then determine potential version folders for each master based on the file name of the master.

I think this would slow down things considerably, because to determine if a folder produced by something like {p0}{f} exists, IMatch must call some Windows file system routines. And the "check if folder nnn exists" is very slow.
Currently IMatch calls this function once for each potential folder for each master folder. To support your schema, it would have to call this one for each master file and potential folder. Which could easily cause thousands or even tens of thousands extra "does this folder exist" Windows function calls. Bringing the version check down to a crawl...

[banzai]

Thanks for the quick reply, Mario.

I understand your performance concerns and you are hesitating to implement something in that direction just cause of my strange folder schema. It is indeed, but can't change it without causing other problems.

For now I will live with manual versions I guess.

But what about another feature:
A small checkbox next to "Link Expression": "Match against fully qualified path instead of just filename".
That way I could filter for the correct subdir myself (as {name} is available).
And it shouldn't cause performance problems as users have to manually turn it on. The regex check will be slightly slower due to the longer string, but when I turn this on, I will do it for a reason and know what can happen.

[Mario]

I have analyzed the problem, implemented the change in a clever way that does not cause any loss of performance for users not using {n*} or {f*} or {e} patterns and rolled the change into the upstream release branch for the next version of IMatch. See https://www.photools.com/release-notes/ #01768 for more information.

[banzai]

Wow, that's great news!
Big thanks for your quick help  👍