iMatch adds new photos to catalog automatically?

[timoteo]

I had read that iMatch watches the folders in its catalog and whenever a new file showed up in a folder it would be automatically added to the folder in the tool.

I had added a "Texas" folder with no subfolders, only thousands of files, a few days ago. I later deleted this "Texas" folder and replaced it with a backup copy. The backup "Texas" folder has hundreds of sub-folders and sub-sub-folders. The sub-folders didn't show up in "Texas" folder in iMatch when I opened it after this.  I didn't see this. I imagined that a watch process ran in the background all the time, even when iMatch wasn't running, and would add new photos to the iMatch catalog in the background.

Can someone explain to me how this really works., as opposed to how I imagined it to work?

[Mario]

I imagined that a watch process ran in the background all the time, even when iMatch wasn't running,

IMatch does not constantly monitor your system.
When you load a database, IMatch does a background scan to find all folders with a date newer than the corresponding folder record in the database. For all folders found during this step, IMatch scans the folder for new and updated files and automatically updates the database.

See: Rescanning Folders

IMatch does by default not automatically add sub-folders during the automatic indexing, because this could have unwanted side effects. It adds sub-folders when you manually trigger a folder rescan via Shift+F5 or the context menu of the folder.

By restoring a backup you have probably messed up the file system timestamps and IMatch cannot automatically detect that the folder has changed. Or something similar. Do a manual rescan to bring in all new sub-folders. Just select the folder in the Media & Folders tree and press Shift+F5.

[Aubrey]

A couple of questions related to this subject:
Q1:
I have all my images stored on a WD drive attached to my laptop with USB3 (They are of course backed up to other sources).
Should I change my Indexing to: "Only for local folders", will this be faster than "Import new and updated files automatically"?

Q2:
I use DXO and or Affinity for processing. When I export from DXO/Affinity a processed file I notice that it can take quite a long time to appear in IMatch folder (maybe 30+ seconds), though export has clearly finished in DXO/Affinity. Is there a way to update files faster. Perhaps the time delay is IMatch reading file and updating metadata before making it available and thus nothing to be done?

Sometimes I use CRTL SHIFT F5 and that "appears" to force a faster update; is this just a subjective opinion?

Aubrey.

[Mario]

A1: Makes no difference. This only applies to network-based folders, not USB storage.

A2: This is a safety feature. I think I have explained this in the help somewhere but I don't recall where.
Many applications work with temporary files during save operations. They don't write the file to the correct file name first, but instead create a temporary file in the same folder. When the save operation completes successfully, they delete the existing file (if any) and then rename the temporary file to the correct name. The precise mechanism depends on the application and can be changed at any time.

If IMatch would now rescan a folder at the wrong time, e.g. while the original file has been removed but the temporary file has not yet been renamed, it may remove the file from the database before the other application can complete whatever it does. And similar issues.

IMatch deals with this successfully for many years by just waiting for a certain period of time when Windows sends "something in this folder has changed". This not only handles the temp/rename/move dances some applications perform but also prevent IMatch from rescanning the same folder over and over again while the other application is still at work. ~20s has been proven to work well in all environments, from local SSD storage to remote network connections. You cannot change that.

Note: You can always Shift+F5 the folder in IMatch to force an immediate rescan if you really, really need to update the folder right now and you cannot wait for IMatch to rescan it automatically.

[Aubrey]

Mario,
Thank you for the great explanation to Q2.
The security file for safety makes so much sense.
Understanding what is happening in background is most informative. I'm happy to wait for the new file to "pop up" especially knowing the processes in action.

Aubrey.