Renamer for distributing files

[jmsantos]

Sometimes I shoot RAW+JPG. I download the photos with the DownloaderPro application, which extracts from the memory card, renames and distributes with the following folder hierarchy:

E:\PHOTOS\2023\2023-06\2023-06_Project\ (for any RAW file)
E:\PHOTOS\ 2023\2023-06\2023-06_Project\JPG (for JPG files)

That is, the JPG images are in a subfolder of the one that contains the RAW files, and the RAW file folder does not include the extension in the folder name.

I'm trying to centralize the whole process in IMatch by doing this with the Renamer tool. I copy the files from the memory card to a Download folder in Windows Explorer. When iMatch indexes that folder, in addition to entering basic copyright metadata, it should rename and move based on the folder layout I've been following for years.

However, so far I've only managed to distribute the files to folders with the file extension:
E:\PHOTOS\{File.DateTime|format:YYYY\YYYY-MM}\{File.DateTime|format:YYYY-MM}_{Renamer.UserInput}\{File.ExtS}

which results

E:\PHOTOS\YYYY\YYYY-MM}\YYYY-MM_Project\NEF (for Nikon RAW files) or
E:\PHOTOS\YYYY\YYYY-MM}\YYYY-MM_Project\RAF (for Fuji RAW files)
E:\PHOTOS\YYYY\YYYY-MM}\YYYY-MM_Project\JPG (for JPG files)

Is it possible to automatically distribute in iMatch as I'm doing so far, so that the RAW files go to a folder above the JPG ones and the RAW file folder does not include the extension in the folder name?

[Mario]

Is it possible to automatically distribute in iMatch as I'm doing so far, so that the RAW files go to a folder above the JPG

This would require a conditional variable that picks different folders based on the file extension.
Can be tricky and I currently don't have the time to work on this, sorry.

RAW file folder does not include the extension in the folder name?

Your mask ends with {File.ExtS} so the extension is always appended. Because it is not conditional, does not check for the file extension. For example:

{File.Ext|is:NEF,,{File.Ext},false}

returns 'nothing' if the file is a NEF file, else the file extension. Use this as an example for your experiments.

Note: In general, you should use {File.Ext} instead, ExtS is for specific purposes only.
 

[jmsantos]

{File.Ext|is:NEF,,{File.Ext},false}
 

Thank you, Mario. I will experiment with that clue.

[jmsantos]

I have experimented with {FileExt} variables and various functions to filter RAW files.

The result is different if I add the function "|is:" or "|contains:". Just with "contains" I can add multiple extensions separated by semicolons.

In this way, the variable
{File.Ext|contains:NEF;contains:RAF;contains:CR2,,{File.Ext},false;to_upper}
will create only subfolders with any extension not included in the list.

I can't find if there is any way to filter all RAW files. Can there be?

[Mario]

I can't find if there is any way to filter all RAW files. Can there be?

IMatch does not maintain a "This is a RAW file" attribute or property.
All RAW files are basically TIFF files with some undocumented and proprietary Bayer matrix stored in one of the TIFF heaps (this is the raw part). And a thumbnail and preview image in other heaps. And metadata in other heaps.

[Tveloso]

Perhaps instead of trying to identify/filter the Raw types, you could explicitly refer to the JPEG type instead?...

It seems like you were using the {File.ExtS} variable in order to cover the different extension variations for JPEG Files (.JPEG or .JPG).  That can also be done with the IMatch File Format variable. So this variable:

    E:\PHOTOS\{File.DateTime|format:YYYY\YYYY-MM}\{File.DateTime|format:YYYY-MM}_{Renamer.UserInput}{File.Format|is:JPEG,\JPG;is:xxx,,\}

...might be able to do what you need.  It will return this:

    E:\PHOTOS\YYYY\YYYY-MM\YYYY-MM_UserInputValue\JPG\

...when the file it a JPEG, otherwise (for all other FileTypes), it will return this:

    E:\PHOTOS\YYYY\YYYY-MM\YYYY-MM_UserInputValue\

The literal backslash dividing the directory resolved from the user Input, and the JPG Directory has been "moved" into the result of the is function so as to always have a trailing backslash (and to not have two consecutive ones for the "parent path" when the file is not a JPEG).  And since there can be multiple is functions in a variable, you can add other formats that you want to include in the JPG subdirectory...for example, this variable:

    E:\PHOTOS\{File.DateTime|format:YYYY\YYYY-MM}\{File.DateTime|format:YYYY-MM}_{Renamer.UserInput}{File.Format|is:JPEG,\JPG;is:OTHER,\JPG;is:xxx,,\}

...will cause renamer to move two explicit file formats into the JPG directory, and all other formats into the parent.

[jmsantos]

Thanks for this advice, Tony. I'll try it.

I had thought of that solution too, but sometimes I make some video (rarely) and I also wanted to separate it into another specific folder of videos. So there would be a main folder only for RAW and subfolders for JPG, MOV, MP4, etc. I can do this very easily with DownloaderPro, which with a simple token, groups all the RAW files regardless of the format (NEF, RAF, CR2...), or all the videos in a MOV subfolder, even if the file format is AVI, MOV or MP4.

It was trying to do the same thing automatically in the indexing process in IMatch, which now integrates renaming as well as metadata. But what I have achieved so far is much more complex. When I download only JPG files there is no problem, iMatch creates the JPG subfolder and moves the images there. But there is a problem when I shoot RAW+JPG: having the JPG as buddy and visual files of the RAW, iMatch doesn't allow to move them separately with the Renamer tool and moves both to the same main folder.

I'll do some more research, but I'm afraid I'll have to stick with DownloaderPro.

[thrinn]

You mean something like this?

Code Select Expand

Z:\TEST\{File.Ext|regexp:jpg//!,JPG,,false}{File.Ext|regexp:MOV|MP4//!,VIDEOS,,false}{File.Ext|regexp:zip|txt//!,OTHERS,,false}To explain: I use a MOVE step with multiple concatenated RegExp. (I use a fixed folder Z:\TEST\ as base target folder. You can easily replace this part with an expression to create date and project based folders).

The first part:

Code Select Expand

{File.Ext|regexp:jpg//!,JPG,,false}returns "JPG" if the extension is jpg and nothing if not. I use "false" as third parameter to ignore the extension case. This means: If the file is a jpg file it returns JPG as subfolder name. Else nothing.

The second part is nearly the same. But because we are talking about RegExp, it is very easy to check for additional extensions: This one checke for MOV or MP4, you can add additional ones using |. If one of the extensions is matched, this part returns VIDEOS as sub folder name. So all movies will be moved to a VIDEOS sub folder.

Code Select Expand

{File.Ext|regexp:MOV|MP4//!,VIDEOS,,false}
Just for fun, let's move ZIP and TXT files automatically to an OTHERS subfolder:

Code Select Expand

{File.Ext|regexp:zip|txt//!,OTHERS,,false}
Note that I have no part for raw files. That's because a RAW file extension won't be catched by any of the RegExp, so the result will be the base folder.

[Tveloso]

Nice!

So using regexp with File.Ext gives direct control over the specific extensions that will route to a given Subfolder.  And using is with File.Format lets IMatch "group the extensions" according to the Format, and we refer to the format Identifiers in the variable instead, to do the same thing (I think MP4s and MOVs are "MCI").  IMatch variables are awesome!

But there is a problem when I shoot RAW+JPG: having the JPG as buddy and visual files of the RAW, iMatch doesn't allow to move them separately with the Renamer tool and moves both to the same main folder.

You may be able to solve this by adjusting your Buddy Definition so that it expects the Buddy Files to already be in a JPG subdirectory of the Master (via a Folder Pattern), so that the JPGs of the Raw+JPG pair will not be considered buddies initially while in the ingest folder (and won't become buddies until Renamer has moved them).

[jmsantos]

Thanks to Thorten and Tony.

I've been offline for a few days. I'll check these tips as soon as I can. In any case, the issue of companion files I think the only solution is to disable it. Distributing the JPGs to a separate folder from the RAWs when I shoot RAW+JPG is what I would expect iMatch to do. I can't do it because the tool used to auto layout into folders based on extension is Renamer.

In this case I see that it is not worth doing without DownloaderPro, because doing the same in IMatch means complicating things too much. And we already know that it is better to keep things simple if they work.

[Mario]

Distributing the JPGs to a separate folder from the RAWs when I shoot RAW+JPG is what I would expect iMatch to do.

You can certainly do that with the Renamer.

But not if you also make the JPG a buddy files of the RAW right from the start.
The job of buddy files is to rename, move, delete the buddy file when the master file is renamed, moved, deleted.
This will collide with that you are trying to do in the Renamer.

If your buddy rule for your RAW file makes IMatch search only in sub-folders of the RAW for buddy files, this will work.
When you run the Renamer, a JPG in the same folder as the RAW will not be detected as a buddy file. And hence it can be renamed and distributed into its own folder.

Once the JPG is in a sub-folder of the RAW, it will be detected as a buddy file, and IMatch's buddy file management kicks in.

[jmsantos]

If your buddy rule for your RAW file makes IMatch search only in sub-folders of the RAW for buddy files, this will work.

Sorry, Mario. After carefully reading the help and trying various options, I don't know how to do that. 

[Mario]

This is where you control where IMatch is looking for buddy files or versions.
You can also use folder patterns to make that dynamic etc.

See Where to Search in the File Relations help topic.

[jmsantos]

I have read several times the help related to Indexing, File Relations and Renamer. My configuration is the one you indicate, as these screenshots show.

The process I follow is to copy the images with Windows file explorer from the card to a DOWNLOADS folder, iMatch automatically indexes that folder, enters basic authoring metadata, renames and moves to the folder structure I set up in Renamer. But I can't get it to automatically create a subfolder for the JPGs, both RAW+JPG files are moved to the same folder.

[thrinn]

In your case, you should not use the settings from Mario's post for the Buddy relation.
Instead:

• Set "Where to search" to "List of folders"
• Set "Direction" to "Specified folder only"
• Then use a folder pattern like in the screen shot. {p0} refers to the base folder (where your NEF should be). So, {p0}\JPG tells IMatch to only look in the JPG sub folder for Buddy Files.

[jmsantos]

In your case, you should not use the settings from Mario's post for the Buddy relation.
Instead:

• Set "Where to search" to "List of folders"
• Set "Direction" to "Specified folder only"
• Then use a folder pattern like in the screen shot. {p0} refers to the base folder (where your NEF should be). So, {p0}\JPG tells IMatch to only look in the JPG sub folder for Buddy Files.

Yeah, that solves the question! It has been difficult, but the help of this forum is invaluable.

Thank you so much, Thorsten.

[thrinn]

You're very welcome.
IMatch can do so much for many different people with different requirements... But this kind of flexibility always comes with a price in terms of complexity. So, don't hesitate to ask - we always are happy to help.