Files in a selection not updating when a new version created

[Stefanjan]

I have a large number of scanned tifs in a catalogue spread across multiple date folders.

I select a subset of the tifs and send them to Photolab.

I process the tifs in Photolab and then export them as Jpegs to the original folders.

The visual versions do not updated unless I identify all the folders and rescan them.

Am I missing a way of updating a selection so that the visual version is updated.

I've attached a screenshot of my Background Processing settings

[Mario]

What do you mean by "visual version"? A visual proxy version?
What do you mean by "not updated"? What is not updated?

The more info we have, the better.
Are the new versions added to the database?
Does IMatch not detect the versions when rescanning the folders where your external software makes modifications?
If you select the master, do the versions show in the Version Panel?
Is the folder in which you let your other software place the files covered by your file relations?
Did you check the IMatch log file for warnings (lines containing W>) or errors (lines containing E>) See log file

I made a quick test:

folder (contains the master A and some versions of A)
|- sub-folder (some other versions of A)

I create a new sub-folder:

folder
|- sub-folder
|- New Folder

I open an image image editor and do a "save as" to "New Folder". I use a file name that makes the new file a version of A.

After a couple of seconds, IMatch rescans "New Folder", the new file is ingested and it becomes a new version for master A.
The version shows in the Version Panel.
Looks OK to me.

[Stefanjan]

Sorry, I thought my original question was clear but hope this further detail helps:

524 Tif images across multiple folders in a single catalogue
16 files selected and selection created CTRL G, R
16 files sent to Photolab for processing
Files processed in Photolab and sent back as jpegs to original image folder
Preferences File Relations setup up so Jpeg version is a visual proxy.
When I view the selection in the imatch File Window the masters do not show versions
If I switch to one of the folders, the new jpeg versions are not displayed but the folder shows the icon signifying the folder has changed.
If I rescan the folder then jpeg versions are correctly displayed everywhere as a visual proxy version.

To answer your questions:

What do you mean by "visual version"? A visual proxy version?
A. Yes a visual proxy version

What do you mean by "not updated"? What is not updated?
A. The Tif Master does not show a jpeg version unless I go to the folder and manually rescan

Are the new versions added to the database?
A. Not until I go to the folders and manually rescan

Does IMatch not detect the versions when rescanning the folders where your external software makes modifications?
A. Only if I manually rescan each folder, odesn't seem to do it automatically in the scenario outlined above.

If you select the master, do the versions show in the Version Panel?
A. No unless I manually rescan

Is the folder in which you let your other software place the files covered by your file relations?
A.  The files are placed in the same folder as the masters.

Did you check the IMatch log file for warnings (lines containing W>) or errors (lines containing E>) See log fil
A. No errrors in the log file

[Mario]

If I switch to one of the folders, the new jpeg versions are not displayed but the folder shows the icon signifying the folder has changed.

IMatch will rescan the folder at the next opportunity.
It waits for at least 5-10 seconds after the last file system update to the folder in order to deal with things like temporary files, files locked by other applications etc.
Until the folder has been rescanned, now versions can be detected.

How long did you wait for IMatch to rescan the folder?
Does your other software maybe keep files locked for an extended time?
Try closing the other application after you have saved the files.

[Stefanjan]

How long did you wait for IMatch to rescan the folder?
Try closing the other application after you have saved the files.

Tried following experiment:
13 TIF files in a selection sent to DXO Photolab
All processed in DXO Photolab batch selected and exported back into original folders as jpegs
Closed DXO Photolab (don't think that makes a difference)
Switched back to Selection in imatch
Waited without touching PC
Last file exported by Photolab displayed version proxy in 8 minutes 17 seconds
Remaining files displayed version proxy in 9 minutes 30 seconds

Is there anything I can do to force imatch to update the selection faster.

Currently I am having to identify the folders and rescan each folder which results in an immediately displaying the visual proxies.

[Mario]

No idea.
Switch IMatch to debug logging (Help > Support) and repeat your test. ZIP and attach the log file.
See log file for more info.

Folders for which IMatch has noticed external changes are marked with the refresh icon.
If "Import new and updated folders" is enabled under Edit > Preferences > Application, IMatch will automatically enqueue modified folders and processes them in turn.

Try this:
In Windows Explorer, copy an image into the clipboard.
In Windows Explorer, paste this file into a folder indexed by IMatch while IMatch is running.

IMatch should mark the folder with the new file as pending (blue icon) after about 10 seconds, and process it as soon as possible afterwards.
I've just tried this here on two computers and two databases, and it works.

[Stefanjan]

Switch IMatch to debug logging (Help > Support) and repeat your test. ZIP and attach the log file.

Log file attached, took a very long time to display proxies but did in the end.

If "Import new and updated folders" is enabled under Edit > Preferences > Application, IMatch will automatically enqueue modified folders and processes them in turn.

In my version of imatch this "Import new and updated folders" appears under Edit > Preferences >Background Processing not Edit > Preferences > Application.

Try this:
In Windows Explorer, copy an image into the clipboard.
In Windows Explorer, paste this file into a folder indexed by IMatch while IMatch is running.

Took about 8 minutes for the file to appear

[Mario]

The log file shows

05.23 14:01:09+ 1703 [41FC] 10  I>                   FSWEEP: Folder [38297] 'E:\RedmiCamera\' modified.
05.23 14:03:55+ 3047 [41FC] 10  I>                   FSWEEP: Folder [38832] 'D:\Photos Prior Years\1991\1991-08-18 ¦\' modified.
05.23 14:03:57+ 1235 [41FC] 10  I>                   FSWEEP: Folder [38860] 'D:\Photos Prior Years\1991\1991-05-26 ¦\' modified.
05.23 14:10:48+  375 [41FC] 10  I>                   FSWEEP: Folder [38666] 'D:\Photos Prior Years\1991\1991-07-01 ¦\' modified.
05.23 14:13:09+  500 [41FC] 10  I>                   FSWEEP: Folder [38668] 'D:\Photos Prior Years\1991\1991-12-24 ¦\' modified.

Between each of these folder sweeps (rescans to check for new and updated files), IMatch processes hundreds if not even thousands of other file system notifications sent by Windows. I see things like

EngineFolderScanner: Signaled. Enabled: 1, idle for 0 ms. Event count: 1028

which is quite a lot. Since the file system is so busy, sending so many events, IMatch delays rescanning, to let the file system settle.
It makes not much sense to rescan folders while Windows is continuously sending "folder update" events.

Things cool down at around 05.23 14:17:00 and at  05.23 14:17:04, IMatch starts to process file system changes, adding new files etc. 469 files are added or updated in this log file.

I have no idea what is causing Windows to send this many events about changed files and folders. Maybe some other process is running in the background or something?

[Stefanjan]

I have no idea what is causing Windows to send this many events about changed files and folders. Maybe some other process is running in the background or something?

Thanks Mario, for looking into this for me. I will have to try and track down what is doing this. Any suggestions how I could identify cause this?

[Mario]

https://docs.microsoft.com/en-us/sysinternals/downloads/process-explorer is a good starting point to analyze such things.
Or the built-in Windows Resource Monitor.

Good candidates are Windows Explorer indexing stuff in folders (depends on your indexing settings), virus checkers,  etc.

Also, look at your log file. Search for FSWEEP to find the name of folders IMatch was adding to the background queue for processing, AddOrUpdate to find the names of folders and files IMatch was processing because of modified events being sent. Maybe this helps to identify what you did or which software you used at the time is doing all the work.

[Stefanjan]

Thanks Mario