Finding/Removing orphaned Versions / derived files

[afx]

Hi,
did not find this in help/google or very much likely I am searching wrong.

I have directory of raw files and then subdirectories of derivative JPGs and TIFFs.
IMATCH is set up to us the JPGs as display proxies via versions.
Time, Gremlins and my addled brain occasionally lead to situations where I do have more JPGs in the subdirectories than masters. Typically because I created JPGs initially, sent them of to someone and later on further refined/pruned my NEF collection and forgot to nuke the JPG folder first before recreating the JPGs.

Is there an easy way to clean this mess with IMATCH 2017?

thx
afx

[Mario]

Maybe the hide buddy files (and the inverted option) of the File Properties filter in the Filter panel may help?

A buddy file itself does not know if it is a buddy or not. It even may belong to several masters, depending on your setup and what your folders contain.
Buddy files are moved, copied, renamed and deleted when the master is deleted. If you have buddy left-overs, something with your file relations is wrong, or you move your buddy files out of the reach of IMatch or something...

[herman]

IMATCH is set up to us the JPGs as display proxies via versions.

When you have indeed set them up as versions (and not as buddies as the topic title suggests...): I have in my database a category to find Orphaned versions.
It is a formula based category, the definition is shown in attached screenshot.
Works like a charm here.

[afx]

When you have indeed set them up as versions (and not as buddies as the topic title suggests...): I have in my database a category to find Orphaned versions.

Thanks Hermann, sounds like what I need, but I guess I have to define the Collection first as well?

cheers
afx

[herman]

You need to define the relations first (what is a master file, how to link a version to a master).
When you have done that refresh the relations.
The Collection with Relations, Masters and Versions is automagically maintained by IMatch, no need to define it.

Hope this helps,

[afx]

I do have my relations defined, so that I see the JPG proxies instead of the NEFs.
But when I try to enter this formula:

Code Select Expand

"@All" NOT "@Collection[Relations|Version]"
I get an error stating that the Category does not exist.
That's why I thought Collection is something that needs to be define explicitly.

thx
afx

[herman]

Nope, the Version Collection should be there when you have defined your versions.
See attached screenshot.
Do you see the Relations collection there?

[herman]

Hmmmm, you may have stumbled over a bug......

I defined this category ages ago, when IMatch 5 was just out.
It has been faithfully doing what it has to do, and it still does.
However, when I open the Category Formula editor (for the working formula!) and click the Test button I receive an error message.
See attached screenshot.

[afx]

Hmm,
this is what I see..
cheers
afx

[afx]

Hmmmm, you may have stumbled over a bug......

That is a silly habit of mine ;-)

Yes, your screenshot looks like my error.

thx
afx

[herman]

Great, when you see the same error I see it is fortunately not isolated to one machine.
Usually this is good news, meaning chances are that Mario can reproduce the error message 

I wil raise a bug report referring to this topic.

[herman]

For the bug report see topic https://www.photools.com/community/index.php?topic=6881.0
Mario just said it is fixed for the next release.

[afx]

Great! The fast turnaround is cool!

Got the update and now the Category works... Sort of...
The results seem to be inconsistent.
@All is 140K, the orphan check is 67K images which sounds a bit too much already.
To drill down further into manageable chunks I then used the filter panel to only filter for file names containing 2006 (my images are named with YYYYMMDD in the name).
The first image I see is an image where I definitely have a version and the thumb also confirms this, see attachment.
And I got more like this...

So, what am I doing wrong?

thx
afx

[herman]

You said in a post earlier in this topic:

Code Select Expand

"@All" NOT "@Collection[Relations|Version]"

This expression will return ALL images that are not a version, and that is what it does.

In my setup it works because I know where the versions are to be found, they all live in a dedicated folder called Processed, so the expression

Code Select Expand

"@Folder[file://D:/Herman/Pictures/Fotografie/Processed/]" NOT "@Collection[Relations|Version]"

will return ALL images that live in the dedicated version folder but don't have a version relation.

So you may have to define an expression which identifies the folder(s) where your versions are supposed to live, and then see if there are any files in those folders which don't have a version relation.

An alternative may be to work with categories.
For example, when you can catch all your Out-Of-Camera-Original Files in a category the versions are the files not fitting the OOC category.

Without knowing how you organized your OOC and version images in folders and categories I can't be more specific.
I don't know if I could when I knew all details, but I can try.....

[afx]

You said in a post earlier in this topic:

Code Select Expand

"@All" NOT "@Collection[Relations|Version]"
This expression will return ALL images that are not a version, and that is what it does.

Ok, so I should see all masters with this.
Misunderstood than.
I thought it would show me everything for which a version does not exist, a slight difference ;-)

In my setup it works because I know where the versions are to be found, they all live in a dedicated folder called Processed, so the expression

Code Select Expand

"@Folder[file://D:/Herman/Pictures/Fotografie/Processed/]" NOT "@Collection[Relations|Version]"
will return ALL images that live in the dedicated version folder but don't have a version relation

So you may have to define an expression which identifies the folder(s) where your versions are supposed to live, and then see if there are any files in those folders which don't have a version relation.

Tricky ;-)
My processed images are in subfolders of the raw files (JPG, PP and WM for various forms of derivatives)

An alternative may be to work with categories.
For example, when you can catch all your Out-Of-Camera-Original Files in a category the versions are the files not fitting the OOC category.

But I am trying to find dangling/orphaned versions:
For all non raw files show me the files that have no RAW or TIFF master
is probably the best summary of what I want to archive.
I have RAW files as masters but also TIFFs from scans (and a few odd JPGs, but they are irrelevant)

So a first idea would be

Code Select Expand

("@All" NOT "@Collection[Relations|Version]") NOT "DB Maint|RAW"
Which seems to work for the NEFs.
RAW is a category filtering on various raw types with @FileRegExp.
Probably not the best performance wise, but will do for now.

But to make this more robust, I should check for the subdirectories, so occasional JPG only directories are not touched.

Code Select Expand

(("@All" NOT "@Collection[Relations|Version]") NOT "DB Maint|RAW") AND "@FolderRegExp[jpg$|PP$|wm$]"
Which seems to work nicely.

Now I get about 3K files back, which seems to be reasonable.

Further checking is needed, but so far it looks like I am on the right track.

Thanks for all your help!
afx

[herman]

For all non raw files show me the files that have no RAW or TIFF master

When you now have something that works please ignore what I am going to say now, otherwise:

You may (for the purpose of finding orphans) define a new temporary relation definition where the master files are all JPG / PP / WPM
and the versions are TIF / NEF / ....

When that relation works you have to look for JPG / PP / WPM without a version.

When finished disable this relation definition and activate your normal relation definitions.

In IMatch there are usually more ways to your goal, pick whatever is the easiest for you :-)