Why is iMatch updating files?

[Carlo Didier]

Once again, like several times in the past, iMatch suddenly thinks it has to update thousands of files when absolutely nothing has changed ...

All I did was a relocate of a folder tree. The corresponding folder has been in both locations (internal SSD and NAS) since months and has been constantly synchronised using GoodSync, so both sides are absolutely identical (verified).

Also, it re-reads other folders than the one I have relocated...

This random updateing has happened in the past, also with previous iMatch versions and even when nothing at all had changed.
What could be causing this? File and folder timestamps have not changed.

[ubacher]

Interesting! There have been many times in the past where I was surprised by Imatch updating a large
number of files. Could there be some weird problem rearing its head every now and then?

[Carlo Didier]

Note that I ran "Compact and Optimize" and it started "adding and updating" right afterwards.

[Carlo Didier]

Interesting! There have been many times in the past where I was surprised by Imatch updating a large
number of files. Could there be some weird problem rearing its head every now and then?

Al least I am not alone with this!

[Mario]

This screen shot does not tell me much.
The log file will show AddOrUpdateFolder entries for each folder IMatch has scanned. If the folder has a "last modified" timestamp newer than recorded in the database, IMatch will check the folder for new or updated files. The same happens when IMatch receives "Folder Changed" events from Windows.

When scanning folders, IMatch checks for new files, and files where the "last modified on disk" timestamp is newer than recorded in the database.
You can see the entries AddOrUpdateFile in the database.

If new or modified files are found, they are processed.
You then see additional entries, like ET-Extract.
Files with the same last modified timestamp as recorded the last time the file was scanned are skipped.

If this happens out of the blue, try to figure out what may have changed the files.
Check the last modified timestamp in Windows Explorer etc.

[Carlo Didier]

As I said before, nothing has changed. File time stamps are unchanged. Files and folders which are rescanned haven't been touched in months.

All I did before the rescan was:
- relocate folder "D:\Photography\Photos Carlo\2010" on local SSD to "\\LochEwe\Photography\Photos Carlo\2010" on my NAS (where I already had an identical copy)
- run Compact and optimize

Absolutely NOTHING else happened on the PC!

After that, when iMatch re-opened the database, it started the rescan, mainly of the folder "D:\Photography\Photos Carlo\2015" on the local SSD.

It's not the first time that I have seen this strange behavior, although it happens rarely. But always without any logical reason as to what is rescanned.

[Mario]

When IMatch rescans a folder and does not find new and updated files, it does nothing.
Actually, it does this every time you open a database. It checks each folder. Folders with newer dates are scanned. Files with newer dates are re-processed.

Maybe your NAS delivers different time stamps? UTC? Local Time? May depend on the SAMBA version you are running on your NAS and whatnot...

[Carlo Didier]

When IMatch rescans a folder and does not find new and updated files, it does nothing.
Actually, it does this every time you open a database. It checks each folder. Folders with newer dates are scanned. Files with newer dates are re-processed.

Maybe your NAS delivers different time stamps? UTC? Local Time? May depend on the SAMBA version you are running on your NAS and whatnot...

The NAS has the same time zone, etc. As I already said, all timestamps are unchanged.
And iMatch is not just scanning, but "Adding and updating" tens of thousands of unchanged files on the local SSD, not the NAS, wich takes hours ...

[Mario]

If IMatch found files to change, there is a timestamp difference.
IMatch uses the Windows API CompareFileTime to compare the timestamp stored in the database and the timestamp returned by Windows for the file.
For files already in the database, it also checks if the file name case has changed or if the extension has changed or if the associated XMP file has changed.
This check also accounts for potential differences due to DST (in case of older FAT file systems which don't use UTC) and ignores them.

You can see if a file was detected as modified in the log file. You see one of File needs update (timestamp) or  File name or extension case changed or File needs update (XMP file updated).

[Carlo Didier]

There must be a bug somewhere. As already said, nothing has changed in the files and folders which iMatch updates. They haven't been touched in months.

Unfortunately, I don't have the log file of that session anymore. I'll keep in mind to save that next time this happens.

I will also try to restore a version of the database before the update and see what happens when I open that one.

[ubacher]

Would there be a way to show (for diagnostic purposes in the log?) the reason a file was updated?
If the modification date is the only criteria the something like "1:06:03 younger"

[Jingo]

I might suggest checking out some "watch" software that can monitor and alert you when a file or folder changes - this may help you narrow down a process that is updating a modified date unbeknownst to you...  Here is one such program (though old): http://leelusoft.blogspot.com/2013/07/watch-4-folder-25-free-version.html  or  this one (newer): https://www.diskpulse.com

[Mario]

Would there be a way to show (for diagnostic purposes in the log?) the reason a file was updated?
If the modification date is the only criteria the something like "1:06:03 younger"

I've explained about what IMatch logs just a few posts above, including the actual messages logged!

[Carlo Didier]

Just to be very, very clear: I checked (by looking at the backups) and NONE of the dates (file created/modified, metadata, ...) of any of the files which iMatch updates had changed. There is absolutely no reason for iMatch to update those files.

I have tried with a 3 days older version of the database (before the problem occured). Nothing has changed in the files (they are the same as when iMatch re-loaded them). This time, iMatch did not update those files but others ... which hadn't changed either.

I will leave iMatch in debug log until this happens again.

[ubacher]

I have been running debug logging for a while now and when I had again
a large number of files updating I checked the log file. Mario said:

You can see if a file was detected as modified in the log file. You see one of File needs update (timestamp) or  File name or extension case changed or File needs update (XMP file updated).

I could not see any entries of that kind.?

[Mario]

I could not see any entries of that kind.?

This is an example log file entry:

AddOrUpdate: C:\data\Image Tests\_DSC-9282910.NEF
File needs update (timestamp)

The file has been detected as modified because the timestamp in the file system has changed.