Exiftool: up to 6 instances, but only one working?

[Carlo Didier]

When I run an exiftool command through the command processor on a large number of files, iMatch starts up to 6 instances of exiftool, but only one seems to realy do something.
The PC has a 4-core i5 processor with 8GB of RAM.

Only when I specify something different from 0 in Preferences, Process control, iMatch only starts as many exiftool instances as specified and they all seem to really do something.

It doesn't make a difference whether I select "Run once" or "Run for each file in selection".
Actually, the second option has a bug: Selected 35 files and the option to "Run for each file in selection". Apparently, it runs 35 times on all 35 files!

Is that normal behaviour?

[attachment deleted by admin]

[Richard]

It is normal for there to be multiple instances of exiftool. In fact six is fairly low. What is not normal is for instances of exiftool to remain when IMatch closes but I have not seen that in a long time.

[Mario]

The command processor uses one ExifTool instance.

[Carlo Didier]

The command processor uses one ExifTool instance.

Could it be made to use more than one? This seems inefficient.

Also, even with the Process Control values set to 4, I still see 5-6 instances and 99% of the time, even with lots of files to re-scan, only one is doing something (always the same instance).

[Mario]

Changing the ECP to split into multiple instances would be a major effort.
Multiple instanfes of ET are normal, IMatch creates them on demand as needed, drops them if no longer needed, etc.

[Carlo Didier]

Ok. Maybe for a later feature request for version 5.5... 
Other things are more important.

[Mario]

All this has been designed carefully and is in place for two years.
I don't see the need to make adjustments to the ECP. You can always run multiple ExifTool instances on the command line, or use something like EXIFToolGUI if yo need more.

The feature request list is full and has work at leas for a full man year. Consider this before adding more requests.

[Ferdinand]

Actually, the second option has a bug: Selected 35 files and the option to "Run for each file in selection". Apparently, it runs 35 times on all 35 files!

This issue wasn't addressed.  There is an aspect of using the ECP that isn't documented very well, and certainly isn't clear in the ECP itself.  If you switch to "run once for each file" then you need to replace {Files} with {File.FullName}.  I discussed this in:
https://www.photools.com/community/index.php?topic=2324.msg15206#msg15206
and I described the different syntax used in either option.

I have had some issues with the performance of the ECP when using it for larger numbers of files (>1,000).   I found it better to run in smaller batches.  I tried both "run once" and "run once for each file" (changing the syntax to suit) and I thought "run once" was faster.  I don't use the ECP on this number of files often enough (i.e. rarely) to get a better understanding of the issue.

[Erik]

After the 5.1.2 release, I used the ECP extensively to clean up some issues with my files.  I quickly found I need the ECP to "run once for every file" based on some tricky stuff I had going on and feedback from the EXIFTool forums (for setting up my args).

My finding is that running once for every file does slow things down considerable but the ECP doesn't choke as easily on large batches of files.

I don't think the slowness is unexpected or unreasonable.  EXIFTool is running a unique instance for each file when you run it that way.  While it's a fast program no matter how you do it, the use of this type of setup illustrates and compounds (when using in large batches) the effects of starting a program-stopping-starting-stopping-and on and on... vs. starting the program once letting it run through some files and then stopping. 

I believe Running Once is the preferable method for speed and efficiency but the Run Once For Each File is there for flexibility.

Back to running multiple instances simultaneously.  I can see that being complicated, at least from the ECP perspective.  As it is, that tricky thing I had going on before was a multistep process where each arg file would executer ExifTool 3 times when all was said and done.  I can't imagine the effort it would take to somehow account for cases like this, and would rather see feature work spent elsewhere.

[Carlo Didier]

Actually, the second option has a bug: Selected 35 files and the option to "Run for each file in selection". Apparently, it runs 35 times on all 35 files!

This issue wasn't addressed.  There is an aspect of using the ECP that isn't documented very well, and certainly isn't clear in the ECP itself.  If you switch to "run once for each file" then you need to replace {Files} with {File.FullName}.  I discussed this in:
https://www.photools.com/community/index.php?topic=2324.msg15206#msg15206
and I described the different syntax used in either option.

I have had some issues with the performance of the ECP when using it for larger numbers of files (>1,000).   I found it better to run in smaller batches.  I tried both "run once" and "run once for each file" (changing the syntax to suit) and I thought "run once" was faster.  I don't use the ECP on this number of files often enough (i.e. rarely) to get a better understanding of the issue.

Thanks for the explanation, Ferdinand. That's exactly the point I was tripping over and which isn't wll explained in the help, as you say, especially for those who might use some of the predefined commands without understanding all the syntax and functions of the ECP and/or Exiftool.