Categories load in ~minute+ despite no diagnostic warning, files for writing

[Damit]

I really don't know what is up with this program. After carefully adding folders year by year, checking everything to make sure things are ok and addressing any warning or error that came up, I just added 2000+ of the 2012 files and, after waiting over 1.5 hours for the program to load the metadata, it still locks up just switching views from any view to categories, initially locking up 4-5 times for over 20 minutes while photools.exe (IMatch) only uses 6% of my CPU and the disk and memory are not taxed at all. Worst of all, the display expands taking the whole screen (long standing problem with this program that I have experience since day 1), so one cannot even use other programs. There is no progress bar – nothing to let you know what is going on and how long the program needs to do whatever it is doing in the background. This is with an optimized database that had just gone through a database diagnostic with no errors and 0 files to be written and all versions propagated. I also had all @Keywords updated, which requires me to go to metadata 2 and change the Protect XMP data to off.  Unless I do this, they will never update! This has been occurring since 2-3 days after restarting my database for the 4th time and has been present, to some extent in the last three attempts to get a stable database running. 
After this "initial" 20-minute waiting period (despite already waiting 1.5 hours for the files to be read and all metadata updated prior to this) every time I go to categories the program locks up. Each time it takes a minute, if not more, to load. The last time I tried it took over 5 minutes! All other views load in less than 2 seconds.
I have attached two logs and one database diagnostic (showing no errors). One log is regular, and then I ran a debugging log while switch from collections to categories. Hopefully something in there can explain the hanging. I also experienced a crash an hour before this and am sending a link to Mario as it is over 350mb. These incessant problems have to be resolved and I have a strong inclination that they are all related.

I have an extremely fast computer employing an i9-13900k, 128GB of memory, and Gen4 Nvmes throughout my whole system. This should not be happening. I am reaching my wit's end and I would really REALLY appreciate some support that brings this to a working resolution.

[Mario]

If your computer is so super fast, but I see the File Window needing 17 or more (!) seconds to load for a tiny database with less than 50K files. Something is really wrong. The File Window is constantly reloading, but the log is not in debug logging mode (see log file) so there are not many details.

Also there is a 13 second difference between the database load time and the IMatch UI init time, which is highly unusual. Show us a screen shot of your work space please.

This looks like something is badly interfering with the database system in IMatch and its file I/O.
The typical cause of this in 99% of all "IMatch is slow" is the installed virus checker that constantly scans the database on every file access, bringing IMatch down to a crawl.

FIRST THING: Configure an exception for the folder (!) containing the IMatch databases you use (you seem to use many different databases, why?) and IMatch itself. This often works wonders.

Please let me know which virus checker you use and when you have configured the exceptions.
Is D: your fastest SSD? I wonder, since with such small databases, IMatch usually flies, even on old hardware.

The second log file is yet from another database, with only 20K files.
This time the database loads in 5 second, but the UI needs 22 seconds to initialize. The typical time would be 2 seconds. Maybe one second, since you have such a super fast PC. My PC is four years old and loads a database with 150,000 files in the same time. Again, please show us a screen shot of the workspace for this database.

The diagnosis log is clean.

I really don't know what is up with this program.

Do you have a license for IMatch? Maybe just consider uninstalling IMatch and asking my distributor for a refund?

I mean, many users work very happily and daily with IMatch, without constantly running into problems all over the place.
Maybe IMatch is not for you and you should look for another DAM software that works better for you?
Here is a comprehensive list (no affiliation): Capterra

I'll be happy to agree to a refund request when you make one at https://www.digitalriver.com/shopper-support/
I think this will be the best solution for both of us. You get your money back and can find a DAM that works for you as good as IMatch works for all other users.

I'm sad to lose a user for IMatch. But if my product works so well for so many users, but not for you, it is how it is.

[Mario]

I've downloaded and tested the 3. database you've sent today.
No diagnostic warnings or errors.
Loads in 8 seconds on my 4 year old PC.
No excessive timings with categories or File Window loads.
Refreshing all data-driven categories shows times between 60ms- and 750ms.
Works very smoothly.

[Damit]

OK, first off, thank you for addressing this issue, and a huge mea culpa on the switching to Categories problem. After reading this and doing testing, trying to figure things out, I realized that it was because I had set visual similarity on the sorting window looking at some duplicates and when I reopened the program it was defaulting to @Keywords and that is why it was taking so long to load. Frankly, doing visual similarity on all @Keywords in 1-5 minutes is pretty fast for this program. So, I sincerely apologize that I did not recognize that. Sorry for any inconvenience! I should have recognized that!
Still, you did find some troubling things that still need to be addressed. I really don't want to leave IMatch for many reasons but mostly because I have spent a year and a half investing a ton of time trying to learn and troubleshooting it. While a refund is nice, and I really appreciate the offer, it does not make up for all the time I have spent on this program, and I would much rather just get things working. I am willing to do anything, within reason, to figure out and help resolving this issue. Tell me what to look for or do, and I will do it and report back. If you advise not to do something, I won't. I cannot tell you how many times I wanted to do something but did not due to warnings on your help pages.
-I will not be quoting you in my message to shorten the length of this post as you seem prefer brevity over comprehensiveness, please let me know if you would prefer otherwise. I am willing to do anything to make this process smoother. As you said it, something is off! I just want to figure out what it is and how to solve it.
-My computer is fast, it is not a claim or about bragging, see the Cinebench test attached. I am willing to run any other test you request. I built this computer specifically for photo management and also for IMatch. This is the second computer I run IMatch on, and I had significant problems on the 1st as well, similar ones to the one I am experiencing now. You even made some of the same observations about files taking long to access. Again, that was a totally separate computer. The only thing that is not top notch in my current build is the graphics card (2070s), though it is no slouch for editing since photoshop does not use the GPU for testing. It should be more than enough for IMatch, and I have no problems with thumbnails. So, I do not think this is hardware based.
-I have included a screenshot of the exclusion that I had already set up in Defender, which is the only virus checker I use, when I first installed IMatch. Every database is under that folder. Do I need to set up an exclusion for another folder?  I sent you the crash report by email, as requested on the pop-up. Did that provide any clues of any issues in my system?
-Yes, my D drive is my fastest drive. It is a Western Digital SN850x Gen4 Nvme SSD, one of the fastest around, or at least 6-7 months ago it was. It is only 55% full, even with several testing databases. I store all of my photo files in a 3 disk array of Lexar NM790 Gen4 Nvmes running in JBOD.
-I do not know how the second one is from a different database. It should be from the same one.  All I did was restart IMatch and run the debugging log while going from collections to categories. Are you sure it is from another database?  How would I check?
I have been adding files to my database piecemeal dealing with problems and I finally got everything in with no diagnostic errors, so hopefully things will be smoother from here on out. Like you everything loads quickly and smoothly, which is why I was sad to hear that you reported some issues accessing files. I did notice some additional lag writing metadata recently. I thought things were finally stable and was very disappointed to confront another issue. Believe me, I would prefer not to have to come here and report problems!

[Mario]

I had set visual similarity on the sorting window looking at some duplicates and when I reopened the program it was defaulting to @Keywords and that is why it was taking so long to load.

Well, yeah. There is no free lunch. And this is the most computing-intensive sort algorithm IMatch offers. Running it for 20,000 files can take 20s or more.

Is there a question in your post a community member or I should look into?

I'm currently working mobile and your post is a wall of text, filling screen after screen after screen on my phone.
And I'm neither experienced in Cinebench or any other benchmark nor interested for whatever purpose.

IMatch is designed to run fast even on older computers. I use it happily on an almost 5 year old PC and on a notebook, with databases containing between 50,000 and 900,000 files.

[Damit]

All I was trying to point out with the benchmark results is that my computer runs fast, and this is not a hardware problem.

Well, yeah. There is no free lunch. And this is the most computing-intensive sort algorithm IMatch offers. Running it for 20,000 files can take 20s or more.

Yes, I know. It took over 5 minutes once in my system. It would be nice if IMatch would tell you it is sorting rather than just provide a spinning circle and not responding (maybe it does, and I don't see where it is displaying this info). Perhaps this is also why there is currently a feature request to allow one to escape out of this sorting when it is active, so you are not stuck waiting in case you began the sort in error.

Is there a question in your post a community member or I should look into?
I'm currently working mobile and your post is a wall of text, filling screen after screen after screen on my phone.

Questions: 

I see the File Window needing 17 or more (!) seconds to load for a tiny database with less than 50K files. Something is really wrong. The File Window is constantly reloading, but the log is not in debug logging mode (see log file) so there are not many details.

Also there is a 13 second difference between the database load time and the IMatch UI init time, which is highly unusual. Show us a screen shot of your work space please.

This looks like something is badly interfering with the database system in IMatch and its file I/O.
The typical cause of this in 99% of all "IMatch is slow" is the installed virus checker that constantly scans the database on every file access, bringing IMatch down to a crawl.

1.) I have included a screenshot of the exclusion that I had already set up in Defender, which is the only virus checker I use, when I first installed IMatch. Every database is under that folder. Do I need to set up an exclusion for another folder? If not, I also sent you a list of all the programs in my computer to show you I do not have any other antivirus installed, and hopefully the benchmark proves there is no hardware problem, so why would there be so many delays accessing the files?

2.) I sent you the crash report by email, as requested on the pop-up. Did that provide any clues of any issues in my system?

[Mario]

I sent you the crash report by email, as re

I'm sure I have replied. You are sending so many emails and making so many posts in so many community topics that I cannot keep track. I see no email from you in the inbox, so I have replied. Or commented in the community post, since you often make posts and send other info via email, which makes things really confusing.

A crash can happen, for many reasons. DUMP files are often inconclusive.
If a crash is reproducible (is it?) then we can have a look and spend some hours on deeper analysis.

[Damit]

I don't understand. I cannot attach anything bigger than 5mb, and you specifically ask to email and attach a link of the thread, which I did. 
No, you have not replied to this question:
You specifically said

Something is really wrong. The File Window is constantly reloading, but the log is not in debug logging mode (see log file) so there are not many details.

Also there is a 13 second difference between the database load time and the IMatch UI init time, which is highly unusual. Show us a screen shot of your work space please.

This looks like something is badly interfering with the database system in IMatch and its file I/O.
The typical cause of this in 99% of all "IMatch is slow" is the installed virus checker that constantly scans the database on every file access, bringing IMatch down to a crawl.

1.) I have included a screenshot of the exclusion that I had set up in Defender when I first installed IMatch. This is the only virus checker I use. Every database is under that folder I excluded in D. Do I need to set up an exclusion for another folder? If not, I also sent you a list of all the programs in my computer (now attaching here since you seem confused) to show you I do not have any other antivirus installed, and hopefully the benchmark proves there is no hardware problem, so why would there be so many delays accessing the files? I am trying to resolve these issues to make sure IMatch is stable because you indicated above that it was not.