File Relations->List of Folders->Direction: what is "Up and Down"?

Started by MrPete, August 28, 2024, 05:27:43 AM

Previous topic - Next topic

MrPete

Given a fixed folder X:\photos\thisType\Master
 - with some subfolders
AND archive X:\photos\thisType\Archive
 - also with some subfolders

(Essentially like in the Help page https://www.photools.com/help/imatch/rel_config.htm )

What does "Up and Down" direction mean, and how does the "Levels" count modify that?
  • In the above example, I'd want to go Up one level, then allow going Down 2-3 levels, to collect the archives
  • If I choose Up and Down even 2 levels, I'm guessing it will go Up to X:\photos which expands the search a crazy amount.
  • If Up and Down actually means "Up 1, then down <levels count>", then we've got something quite useful!

Mario

-- Mario
IMatch Developer
Forum Administrator
http://www.photools.com  -  Contact & Support - Follow me on 𝕏 - Like photools.com on Facebook

MrPete

Quote from: Mario on August 28, 2024, 08:41:41 AMUp and down n levels from the folder where a master was found. See Where to Search
Since Where to Search doesn't describe this option, let's make it clear.

If I have a folder structure of:
D:\a1\b1\c1\master\e1\f111
                  \e1\f112
                  \e1\f113
                  \e2\f121
                  \e2\f122
                  \e2\f123
            \c1\d2\e1\f211
               \d2\e2\f221
               \d3\e3\f331
D:\a1\b1\c2\d4\e4\f441
                 \f442
                 \f443
        \c3\d5\e5\f551
                 \f552
                 \f553

ie d2 and d3 have the same parent as master (\a1\b1\c1)
c2 and c3 have the same grandparent (\a1\b1)

Are you saying "up and down" 2 levels, will capture everything in the above list?
- all share the same grandparents (up 2 levels)
- all are within 2 levels down from master (grand nephews ;) )

Mario


c:\a\b\c\d\e\f

When the master is found in c, a, b, d, e will be checked when you enable 2 levels up/down.

But it you even consider this, think twice and maybe make your file structure easier. 
Most users use "master folder" or "one level down", since they keep versions in the same folder as the master or in sub-folders below the folder containing the master. Don't force IMatch to scan thousands of files every time it has to re-check relations. This can become a performance problem very quickly.
Tip: the log file reports how many folders and files IMatch had to process when refreshing a relation.

It is very easy to shoot yourself in the foot and force IMatch to search in too many folders for versions every time.

Just had a support case yesterday where a user reported that the Batch Processor was super-slow when he processed files in one specific folder. But not in others. Moving the files to another folder brought the Batch Processor runtime down from 26 minutes (!) to 4 seconds.

Turned out the problem folder had 200,000 files in 1,800 sub-folders, and the file relation rule he had setup forced IMatch to search all 1,800 folders and 200,000 files every time file relations had to be refreshed. Resolving that issue dropped the Batch Processor runtime from 26 minutes to 4 seconds.
-- Mario
IMatch Developer
Forum Administrator
http://www.photools.com  -  Contact & Support - Follow me on 𝕏 - Like photools.com on Facebook

Tveloso

Quote from: Mario on September 03, 2024, 10:11:09 AMc:\a\b\c\d\e\f

When the master is found in c, a, b, d, e will be checked when you enable 2 levels up/down.
Will this not also include all the siblings of the folders mentioned?  I'm pretty sure that's true for the "down" direction (d and e)...all the siblings of d, and all the siblings of e, will also be checked for Versions, when a Master is found in c.

If that's also true for the "up" direction (a and b), and I believe that it is, then that means that all the siblings of a (i.e. everything in the root of the C drive), and all the siblings of b, will also be checked for Versions.

So, depending on your actual directory structure, this could very well turn into what Mario warns about:
Quote from: Mario on September 03, 2024, 10:11:09 AMIt is very easy to shoot yourself in the foot and force IMatch to search in too many folders...

The use of Folder Patterns, as Mario suggested in another topic, can cover the "up" direction, and prevent that "accidental inclusion" of the sibling folders there.

The configuration options IMatch provides in Where to Search (Folder Patterns - including literal strings where needed, combined with Direction), should be able to cover any requirement...
--Tony

MrPete

Quote from: Mario on September 03, 2024, 10:11:09 AMWhen the master is found in c, a, b, d, e will be checked when you enable 2 levels up/down.
(Oh boy do I get it about excess searching ;) )

I'm simply trying to understand what this function actually does. Tony is asking the same thing.

In my example, "master" is really at level "d", and I have defined a-f. Didn't want the example to include the root level.

  • If I tell it to look down one level, does that search all child folders under the master? (e1, e2 etc)
  • If I tell it to look up one level, does that search all siblings? (d2, d3 under c1)
  • If I tell it to look up two levels, does that search all folders under the grandparent? (c1, c2, c3 under b1)?
  • And most important, if I tell it to go "up and down" two, does that do all of the above, AND search two levels below those additional parent/sibling folders?

YES, obviously if one were to aim the searcher at a complex tree with lots of folders in these levels, the searching would be close to endless (possibly an infinite loop if links are present ;) )

Mario

A quick test revealed, in a folder structure like

B
 - b1
 - b2
C
 - c1
 - c2
 - D
  - d1
  - d2

if the master is in C and the rule specifies Down 1 Level, versions will be searched in c1, c2 and D. Not in d1 or d2.
Searching up one level finds the version in B, not in b1 or b2.

Feel free to make your own tests using your actual folder layout!
- move the version to a folder
- select the folder containing the master
- press <F4>,<R> and then choose "Rebuild"

This will immediately show you if your versions are detected or not using the settings you use. And you can quickly redo the test after changing your relation settings. Very helpful. Instant feedback.
-- Mario
IMatch Developer
Forum Administrator
http://www.photools.com  -  Contact & Support - Follow me on 𝕏 - Like photools.com on Facebook