photools.com Community

IMatch Discussion Boards => General Discussion and Questions => Topic started by: MrPete on August 28, 2024, 05:27:43 AM

Title: File Relations->List of Folders->Direction: what is "Up and Down"?
Post by: MrPete on August 28, 2024, 05:27:43 AM
Given a fixed folder X:\photos\thisType\Master
 - with some subfolders
AND archive X:\photos\thisType\Archive
 - also with some subfolders

(Essentially like in the Help page https://www.photools.com/help/imatch/rel_config.htm )

What does "Up and Down" direction mean, and how does the "Levels" count modify that?
Title: Re: File Relations->List of Folders->Direction: what is "Up and Down"?
Post by: Mario on August 28, 2024, 08:41:41 AM
Up and down n levels from the folder where a master was found. See Where to Search (https://www.photools.com/help/imatch/rel_config.htm?dl=hid-26)
Title: Re: File Relations->List of Folders->Direction: what is "Up and Down"?
Post by: MrPete on September 02, 2024, 11:09:31 PM
Quote from: Mario on August 28, 2024, 08:41:41 AMUp and down n levels from the folder where a master was found. See Where to Search (https://www.photools.com/help/imatch/rel_config.htm?dl=hid-26)
Since Where to Search doesn't describe this option, let's make it clear.

If I have a folder structure of:
D:\a1\b1\c1\master\e1\f111
                  \e1\f112
                  \e1\f113
                  \e2\f121
                  \e2\f122
                  \e2\f123
            \c1\d2\e1\f211
               \d2\e2\f221
               \d3\e3\f331
D:\a1\b1\c2\d4\e4\f441
                 \f442
                 \f443
        \c3\d5\e5\f551
                 \f552
                 \f553

ie d2 and d3 have the same parent as master (\a1\b1\c1)
c2 and c3 have the same grandparent (\a1\b1)

Are you saying "up and down" 2 levels, will capture everything in the above list?
- all share the same grandparents (up 2 levels)
- all are within 2 levels down from master (grand nephews ;) )
Title: Re: File Relations->List of Folders->Direction: what is "Up and Down"?
Post by: Mario on September 03, 2024, 10:11:09 AM

c:\a\b\c\d\e\f

When the master is found in c, a, b, d, e will be checked when you enable 2 levels up/down.

But it you even consider this, think twice and maybe make your file structure easier. 
Most users use "master folder" or "one level down", since they keep versions in the same folder as the master or in sub-folders below the folder containing the master. Don't force IMatch to scan thousands of files every time it has to re-check relations. This can become a performance problem very quickly.
Tip: the log file reports how many folders and files IMatch had to process when refreshing a relation.

It is very easy to shoot yourself in the foot and force IMatch to search in too many folders for versions every time.

Just had a support case yesterday where a user reported that the Batch Processor was super-slow when he processed files in one specific folder. But not in others. Moving the files to another folder brought the Batch Processor runtime down from 26 minutes (!) to 4 seconds.

Turned out the problem folder had 200,000 files in 1,800 sub-folders, and the file relation rule he had setup forced IMatch to search all 1,800 folders and 200,000 files every time file relations had to be refreshed. Resolving that issue dropped the Batch Processor runtime from 26 minutes to 4 seconds.
Title: Re: File Relations->List of Folders->Direction: what is "Up and Down"?
Post by: Tveloso on September 03, 2024, 05:42:44 PM
Quote from: Mario on September 03, 2024, 10:11:09 AMc:\a\b\c\d\e\f

When the master is found in c, a, b, d, e will be checked when you enable 2 levels up/down.
Will this not also include all the siblings of the folders mentioned?  I'm pretty sure that's true for the "down" direction (d and e)...all the siblings of d, and all the siblings of e, will also be checked for Versions, when a Master is found in c.

If that's also true for the "up" direction (a and b), and I believe that it is, then that means that all the siblings of a (i.e. everything in the root of the C drive), and all the siblings of b, will also be checked for Versions.

So, depending on your actual directory structure, this could very well turn into what Mario warns about:
Quote from: Mario on September 03, 2024, 10:11:09 AMIt is very easy to shoot yourself in the foot and force IMatch to search in too many folders...

The use of Folder Patterns, as Mario suggested in another topic, can cover the "up" direction, and prevent that "accidental inclusion" of the sibling folders there.

The configuration options IMatch provides in Where to Search (Folder Patterns - including literal strings where needed, combined with Direction), should be able to cover any requirement...
Title: Re: File Relations->List of Folders->Direction: what is "Up and Down"?
Post by: MrPete on September 04, 2024, 09:21:35 PM
Quote from: Mario on September 03, 2024, 10:11:09 AMWhen the master is found in c, a, b, d, e will be checked when you enable 2 levels up/down.
(Oh boy do I get it about excess searching ;) )

I'm simply trying to understand what this function actually does. Tony is asking the same thing.

In my example, "master" is really at level "d", and I have defined a-f. Didn't want the example to include the root level.


YES, obviously if one were to aim the searcher at a complex tree with lots of folders in these levels, the searching would be close to endless (possibly an infinite loop if links are present ;) )
Title: Re: File Relations->List of Folders->Direction: what is "Up and Down"?
Post by: Mario on September 05, 2024, 10:13:06 AM
A quick test revealed, in a folder structure like

B
 - b1
 - b2
C
 - c1
 - c2
 - D
  - d1
  - d2

if the master is in C and the rule specifies Down 1 Level, versions will be searched in c1, c2 and D. Not in d1 or d2.
Searching up one level finds the version in B, not in b1 or b2.

Feel free to make your own tests using your actual folder layout!
- move the version to a folder
- select the folder containing the master
- press <F4>,<R> and then choose "Rebuild"

This will immediately show you if your versions are detected or not using the settings you use. And you can quickly redo the test after changing your relation settings. Very helpful. Instant feedback.