Delete File Slow

Started by ubacher, Today at 10:49:07 AM

Previous topic - Next topic

ubacher

Just want to report that I also have found unusual long delays when Delete Rejected Files and Rescan <Folder>.


Mario

Quote from: ubacher on Today at 10:49:07 AMJust want to report that I also have found unusual long delays when Delete Rejected Files and Rescan <Folder>.


And you have kept the log files to provide a minimum of details for your post?
Or was this from a thread you remember but I don't?

Deleting a file can perform very differently, depending on factors like local or remote file, the structure of the database, the amount of data to remove from the database, the number of categories in the database and many other factors.

It takes about twice as long to delete a file from my one million file test database than it takes for a smaller 100,000 files test database.

ubacher

My report is based on experience on using Imatch heavily (and for many years).
My inspection of the log file did not show a clear reason.

Previous experience with upgrades tells me that it takes a few months for these kinds of problems to
get corrected. It seems mostly when other problems are corrected such slowdowns/hangups disappear also.

After the previous update I swore that next time I would wait a few months before I upgrade the next time.
But the new PEEK feature and the handling of dng V1.7 files made me update right away!

Mario

QuotePrevious experience with upgrades tells me that it takes a few months for these kinds of problems to
get corrected. It seems mostly when other problems are corrected such slowdowns/hangups disappear also.
The log file in debug logging mode records a lot of details, e.g. how long removing a file actually takes, and which of the steps required to finish the task take. Look out for CIMEngine5::FileOpDeleteFile and how long it takes.

On my system with a 100K database it takes between 1 and 3 seconds. 3 seconds for a file with multiple faces and persons assigned and confirmed. Because removing these faces updates all categories, collections, the associated person data, invalidates caches and whatnot. How long does it take on your system to delete a file?

ubacher

I just had another case of unusual long delay. Please look at the log file and search for
processed 42 folders in 0ms
There is a long delay shown on the line following for each entry.

The last entry happened when i switched from the Info&Activity panel to show the categories panel. (Screenshot)

NB: I switched logging to debug midway through the file.

Screenshot 2025-03-12 142142.jpg


Mario

QuoteInfo&Activity panel to show the categories panel. (Screenshot)
This for sure means that IMatch has to load all categories into the panel, recalculate categories as needed, and, depending on which categories you have, recalculate persons, events, collections and whatnot. This will delay other backgrounds tasks and tasks waiting for database access.

The database has less than 1,000 files, but IMatch needs 12 seconds (!) to load it. What else is going on on that computer? This is a system with four cores and it should perform much better. Is D: your fastest disk?

No files were deleted in this session. No FileOpDeleteFile entries in the log.