A use case for folder exclusion in relations

Started by MrPete, August 27, 2024, 09:36:47 PM

Previous topic - Next topic

MrPete

My sweetie has the following simple workflow for a few project types:

Top folder: PROJECT
Sub folders:
  • 1 Originals
  • 2 Cleaned
  • 3 Signed and color profile (for print house)
  • 4 Final resize done
  • 5 Ready to upload

The easiest relation definition for this would seem to be:
MASTER: Project/1 Originals
VERSIONS: Project *except* Project/1 Originals

But that's not available.

I have three questions about this:
a) Do I maybe not need to worry about excluding the "1 Originals" folder? Master files are masters anyway (due to being *.NEF)???

b) It appears that version folders must already be added to the index. Correct?

c) Is there a simple way to accomplish what's needed? The only alternatives I see right now are:
    - Complicate her structure by moving steps 2-5 under 1 (the folders are actually numbered)
  OR
    - Add all four other folders to the versions list



Mario

a) If Originals does not contain NEF files, no masters will detected in these folders.

b) Yes. IMatch can only find versions in folders indexed by your database.

c) Your structure makes limiting where to find versions hard, since the folder containing masters is on the same level as the folders containing potential versions. All of this revolves around the folder where IMatch found a master. In your case, it will find masters in the "1 Originals" folder (I assume). Now how to you tell IMatch that there are other folders on the same level as the master folder which might contain versions?
The typical way users store their versions is in the same folder as the master or in a sub-folder under the master folder (in your case, sub-folders under 1 Originals. You use a different structure which causes extra challenges.
Folder patterns is in this particular case the probably way to go, to avoid IMatch having to search everywhere for versions. See Folder Patterns. You use the master folder as the base, and then add "2 Cleaned" and "3 Signed...", all as separate patterns.
Or cleanup and move folders 2 to 5 below 1 and then use a simple "one level below master" rule.
-- Mario
IMatch Developer
Forum Administrator
http://www.photools.com  -  Contact & Support - Follow me on 𝕏 - Like photools.com on Facebook

MrPete

Quote from: Mario on August 27, 2024, 09:58:00 PMc) Your structure makes limiting where to find versions hard, since the folder containing masters is on the same level as the folders containing potential versions. 
I realize you're used to what you have, and have encouraged others to use. NOT trying to convince you to adopt a different workflow.

At the same time, it's only hard to find versions because iMatch doesn't have a "same level" option -- which AFAIK is a pretty simple structure. I'm confident you as a SW dev have plenty of software that understands "same level". Separate same-level folders for different file purposes (src, bin, lib, etc etc)... all within a common folder for that project.  ;)

Advantages:
  • Each folder is visible; the structure is immediately visible
  • Don't have to open the actual original photo folder or see those photos, to get to a processed-photo folder (ie when time to upload, etc... it can be unwieldy to go to the parent folder when all I want is to get to Upload-ready images)
  • Minimal risk of performance degradation (Windows slows considerably as file count grows in each folder. This of course will depend on how many buddy / variant files are involved.) (WHY? Directories are stored as B-trees. with 4KB node size. Each new level has slower performance. File ops/sec  drops ~50% at 1-2k files per folder, Dir ops/sec drops ~50% at ~25-300 folders in a level. (NTFS, SSD storage) File performance depends on filename length of course.

Mario

#3
QuoteMatch doesn't have a "same level" option
Of course it has. I've suggested using folder patterns above several times.
Use "List of Folders" and "Specified Folders only" and then input this folder pattern:

{p1}\Versions

if your masters are in "c:\projects\bla\Originals" and your versions are in c:\projects\bla\Versions, this will work fine.
{p1} is the "folder above the folder containing the master".

When IMatch finds a master in "Originals" it uses the folder pattern. {p1} returns "c:\projects\bla\" and thus it will look in c:\projects\bla\Versions for versions. If you have versions in other folders, just add more folder patterns:

{p1}\2 Cleaned
{p1}\3 Signed and ...
{p1}\4 ...

If the folders have sub-folders which may contain versions, set the "Down" and "level" options accordingly.
If not, "Specified folder only" is best to limit where IMatch is looking for versions.

Or, if NEF files are really only in the "1 Originals" folder, you can use just one pattern to grab them all:

{p1}

together with the "Down" option and 1 level (or more).

This means if a master is found in <drive>:\<A>\<B>, look for versions for it in <drive>:\<A> and the first level of sub-folders. <A> will  be scanned for versions, but since it does not contain any version (no NEF files), that's OK.

I'd say that IMatch is extremely flexible when it comes to control where to find masters and versions.
This concept is now in place since IMatch 5, back in 2016.
-- Mario
IMatch Developer
Forum Administrator
http://www.photools.com  -  Contact & Support - Follow me on 𝕏 - Like photools.com on Facebook

MrPete

I guess the {p1} would be as close as it gets to "same level". I'd much prefer not having to specify every same-level folder; that clearly is different.

Yes, I think we can run with the assumption that masters are all NEF, at least in this case. :)

Mario

{p1} and one level down will search for versions in all folders below the parent folder of the folder containing the masters. Which matches your use case.
-- Mario
IMatch Developer
Forum Administrator
http://www.photools.com  -  Contact & Support - Follow me on 𝕏 - Like photools.com on Facebook

MrPete

Quote from: Mario on September 01, 2024, 05:43:13 PM{p1} and one level down will search for versions in all folders below the parent folder of the folder containing the masters. Which matches your use case.
Yep... including searching the masters folder. As long as we're careful, that works fine. "Same level" presumably would search the same set of folders excluding the current folder, but honestly I don't care -- this will be fine :)