Forever To Delete 19 Tif Files

[Darius1968]

The attachment to this post is a log, accounting for what happened, while I was simply trying to delete 19 TIF Files - Created by an Epson flatbed scanner - that I no longer needed.  The viewer was open - A later-deletion of some TIF Files, with the viewer closed, seemed to be a somewhat smoother process - when the progress-dialog just seemed to become frozen.  Is there anything remarkable about this log file? 

[sinus]

I am not a log-specialist.

Hence I can see almost nothing.
What I can see, that you have a lot of folders and a very lot of categories, if I compare it with my DB.

But this should be not the source of your problem, I think.

Here are the points, what I means, just for your information, some "big" differences:

Darius
06.07 20:39:14+    0 [10F4] 02  I>        Database Stats
06.07 20:39:14+    0 [10F4] 02  I>          6073 folders
06.07 20:39:14+    0 [10F4] 02  I>          403607 files
06.07 20:39:14+    0 [10F4] 02  I>          20639 categories

06.07 20:39:21+ 5141 [10F4] 00  S>      #STS#: "timeline.loadtime"         0      0      0.00      "5141ms"


Markus
06.09 08:04:12+    0 [4CDC] 02  I>         Database Stats
06.09 08:04:12+    0 [4CDC] 02  I>           861 folders
06.09 08:04:12+    0 [4CDC] 02  I>           307039 files
06.09 08:04:12+    0 [4CDC] 02  I>           3339 categories

06.09 08:04:15+  406 [4CDC] 00  S>       #STS#: "timeline.loadtime"         0      0      0,00      "797ms"


btw/OT
This brings me up to an old wish from me: it would be great, if we could compare DBs in some way. Like here, I guess, you want also to know, is your DB ok or is there something special bad? And I think, with some skill, someone could create an app or I do not know, the main-points of a DB to compare.
Because, IF someone has troubles, this could be the first thing to look at, if a IMatch-DB looks "normal" or are there some points, what are really out of normality.
But, OK, this is only in my mind since ages and it seems, I am the only one with such a wish. 

[Mario]

Windows failed to delete the cache files for some reason. Virus checker?
Each deletion attempt caused a long delay. Windows was probably waiting for something or retrying the delete several times and then giving up after a number of seconds, returning an error code to IMatch.

Check your virus checker. Check for other software doing things on your system which may cause IMatch/Windows to delete files.
Make sure all Windows updates are installed. IMatch loads cache files via Windows WIC, and maybe WIC had the cache files locked for some reason after IMatch has loaded them.
Usually such strange effects go away after rebooting Windows once.

[Mario]

@sinus

Databases vary widely.
There is nothing really to compare between databases.
All the relevant info (database size, number of files, folders, categories) is listed the Activity panel and also logged in the log file and written to the diagnosis log.

And there are some simple rules:

- Operations taking 1 second for a database with 100,000 files take (at least) 5 seconds for a database with 500,000 files.
Usually more, because most algorithms have no linear performance characteristics

- If you have 20,000 categories, all category operations will take 10 times as long compared to a database with 2,000 categories

- Some users put their database on a SSD, others put it on a external USB drive or even a NAS box - which makes a BIG difference in performance

- Users sometimes do crazy things with variables in their custom File Window layouts - forcing IMatch to constantly re-calculate categories and collections.
This may not cause any issues, until, some day, it does. This is why I usually ask to switch to the Default layout in such cases.

- Users run stuff (programs, utilities, ...) in the background without even knowing. Most users these days are just "users", without much deeper know-how.
And all that stuff can cause momentarily performance drops or may interfere with IMatch or Windows in strange ways.

- Virus checkers do crazy things now and then - from stalling IMatch by running an on-access check on the entire database every time IMatch reads or writes data to terminating ExifTool or one of the other external helper utilities IMatch uses.

- Since Windows no longer really reboots, but only goes to sleep "to improve startup time", stuff accumulates in memory and the file system, causing strange effects sometimes.
A simple reboot from the START menu (which is a real reboot) fixes these issues in most cases. Users just don't know about this - because many users today are just "users", without any knowledge of how their PC works.

[sinus]

Thanks, Mario, for your long explanations.

Darius has even 100.000 files more than I do have.

To be honest, I tend to split this DB in the next time, because now I can feel, that some things does take longer than before.
I do only hesitate, because two DBs are more difficult to "maintain", if I  change something in one DB, it is not changed in the other.
Some preferences does work for both DBs, some does not.

I know, that we can a lot of things import, but it is always time-consuming and we must remember to do so.
That is why I have not splitted them.

I wonder, if Darius does have the same feeling, that with such a lot of files, IMatch gets slower (what is very understandable).

[Mario]

My largest test database has over 800,000 files in thousands of folders and performs quite well. And that is really a corporate-grade file collection, including 4,000 videos, PDFs etc.
I run it on an one year old PC, AMD 12 core CPU, 32 GB RAM, m.2 SSD storage for the database.

I use standard IMatch settings, including category-color codes in the File Window (for @Keywords) etc. Even folders with 10,000 files display in < 1s.
Of course searching the entire database takes much longer than searching a 100,000 files database. And operations like updating data-driven categories or collections too.
But I can work with this database, add new files, view files, filter, search, rate, label, add metadata etc. super-fast. Nothing gets in the way.

Of course when I write back a completed folder with 5,000 to 10,000 files, I know that IMatch will show a progress dialog and that I have to wait a while.
And I know that the compact & optimize takes about 30 minutes, so I do it when I don't need the database.