People View and Face Manager issues

[mrmsa_2022]

In the People view, the text "No files" appears for several people, even though there are files for those individuals.

In the Face Manager view, after the F2 function, the window takes quite a long time to open, much longer than in the previous software version.

When typing a person's name into the Face Manager view, the letters appear in the input field with a noticeable delay and searching for names takes a very long time.

The mouse click between different people doesn't seem to show the clicked name as active, so the user can't see which person is active.

Virus scanner Avira Security, ...photools.com/... folder excluded from virus scanner including executables IMatch2023x64.exe, exiftool.exe etc.
Virus scanner turned off, no change for the behavior, same issues.

[mastodon]

I have this No files issue, too. But after some time it shows the correct numbers. Ex. I click on "Show only persons with unconfirmed faces." and take some time, more then a minute...

[Mario]

I have this No files issue, too. But after some time it shows the correct numbers. Ex. I click on "Show only persons with unconfirmed faces." and take some time, more then a minute...

How large is your database (files), how many persons do you have, which tab is active in the People View at the time, which layout do you show?

The File Window is updated as soon as you click a person, usually. There should be no delay at all, unless there are tens of thousands of files for a person.

Can you please switch to debug logging and repeat the steps, then ZIP and attach the log.
There more info and details about a behavior I get, the more likely it is for me to reproduce and fix it.

[mrmsa_2022]

I have this No files issue, too. But after some time it shows the correct numbers. Ex. I click on "Show only persons with unconfirmed faces." and take some time, more then a minute...

I never had this same issue with older version software. Yes, it is true that you need to wait view to update, older version did this way much better.

[mrmsa_2022]

I have this No files issue, too. But after some time it shows the correct numbers. Ex. I click on "Show only persons with unconfirmed faces." and take some time, more then a minute...

How large is your database (files), how many persons do you have, which tab is active in the People View at the time, which layout do you show?

The File Window is updated as soon as you click a person, usually. There should be no delay at all, unless there are tens of thousands of files for a person.

Can you please switch to debug logging and repeat the steps, then ZIP and attach the log.
There more info and details about a behavior I get, the more likely it is for me to reproduce and fix it.

How large is your database (files), how many persons do you have, which tab is active in the People View at the time, which layout do you show?
Files: 135638
Persons: 8846
Tab: People
Layout: People (I tried Default, no difference)

The File Window is updated as soon as you click a person, usually. There should be no delay at all, unless there are tens of thousands of files for a person.
I do not have any person having tens of thousands of files, mostly those are from 2-10 files around, I guess that is the great majority of files per person, some may have hundredths but only few of those.

Can you please switch to debug logging and repeat the steps, then ZIP and attach the log.
There more info and details about a behavior I get, the more likely it is for me to reproduce and fix it.
Sure, I do collect the that information the next, 30 minutes and you'll get that information.

[Mario]

Persons: 8846

Gosh! That may be it.

The average user has between 50 and 100 users. The extreme so far was almost 1,000 persons.
I never tested more than 2,000 persons, and that was a stretch already.

Do you work commercially with IMatch? What kind of work do you to that requires to manage 9,000 persons?

9,000 persons is stretching anything quite a bit. That's a log of data to maintain and move between the app managing the People View and IMatch.

[mrmsa_2022]

Log file here. Is there something special you want me to do with the software to create the log file?

[Mario]

Log file here. Is there something special you want me to do with the software to create the log file?

No. I believe we've found the problem already. 9,000 managed persons!

[mrmsa_2022]

Persons: 8846

Gosh! That may be it.

The average user has between 50 and 100 users. The extreme so far was almost 1,000 persons.
I never tested more than 2,000 persons, and that was a stretch already.

Do you work commercially with IMatch? What kind of work do you to that requires to manage 9,000 persons?

9,000 persons is stretching anything quite a bit. That's a log of data to maintain and move between the app managing the People View and IMatch.

I do not use IMatch for commercial purposes. Number of people comes from my hobbies SLR and d-SLR from total +30 years of time including negative and positive collections starting from early 1900. All digitized now, thus amount of people which I do not keep anything special to be honest. I do have another software, true AI, which can handle that number of people without any troubles.
Btw, old version of IMatch was able to handle that amount of people, newest not.

[Mario]

Btw, old version of IMatch was able to handle that amount of people, newest not.

There is not much difference in the Person part of the People View between IMatch 2021 and 2023.
Families and Groups are new, but they don't affect the other tab.
I wonder what cause this performance issue on your PC. No other reports, but I doubt many users (who have already upgraded) have 9000 persons.

I've made a quick test with my 2,000 person database and it shows immediate response when clicking a person (File Window Update) searching for persons or scrolling.

When clicking is slow with so many persons I should be able to simulate this.
Usually I process bug reports in the order they come in (unless it makes sense to do otherwise) and your bug report is from today. So it might take a few days before I can look into this.

[mastodon]

It is weird. Now, at first it was good, but after clicking on "Show only persons with unconfirmed faces." and then switch to several times to People-Falmilies-Groups, the file number counting failed. IMatch version 2013.1.12

IMATCH6_LOG.TXT

[mrmsa_2022]

" In the People view, the text "No files" appears for several people, even though there are files for those individuals. "

Leaving here an example.
These numbers does not follow the reality. From top to bottom, 3rd person does not have any unconfirmed faces when opening the Person Manager. Same for image 4, 5 and 7. Last person on bottom, does have two  files and system shows "No files"

[Mario]

The counts you see in the People View are fetched on-demand when needed.
Maybe IMatch is still struggling to get all 9,000 persons in your database sorted.
I have no idea. I have never created 9,000 persons and hence I don't know if there are some performance issues caused by firing thousands of requests to calculate file counts and confirm counts per person.

As I said, I will know more once I have created a database with 10K persons and enough files and then assign some files to each of the 10K persons. This can take days to setup alone...
Currently I'm busy analyzing and fixing bugs reported before your bug report, sorry.

[mrmsa_2022]

The counts you see in the People View are fetched on-demand when needed.
Maybe IMatch is still struggling to get all 9,000 persons in your database sorted.
I have no idea. I have never created 9,000 persons and hence I don't know if there are some performance issues caused by firing thousands of requests to calculate file counts and confirm counts per person.

As I said, I will know more once I have created a database with 10K persons and enough files and then assign some files to each of the 10K persons. This can take days to setup alone...
Currently I'm busy analyzing and fixing bugs reported before your bug report, sorry.

No problem at all, just wanted to leave here an example. I've left IMatch running for 4 hours with those selected person thus I do believe there is enough time to count the number of files and, previous version never had this trouble.

[Mario]

Just for the fun of it I've created a database, added 10,000 persons to it.
Dashboard loads in 7 seconds. It shows the first 200 persons without gender:



All persons use a copy of the same face image, but that does not change performance or the amount of data the People View has to load. The database holds 10,000 cover images.

People View takes a few seconds to load and refresh when there are changes. Searching takes maybe 5 to 8 seconds.
Toggling the "Unconfirmed" button takes a few seconds to refresh the view.


I have not yet assigned files to each person, that will have to wait until I have more time. Should not make much of a difference, though. I will find out.

I tested this on a 3 year old PC.
In general, this seems to work just fine with 10,000 persons.
Figuring out why it takes only 7 sec. to load the dashboard here and so much longer on your computer will be tricky.
Are you sure your virus checker or other security software is not slowing down IMatch?

[mrmsa_2022]

Just for the fun of it I've created a database, added 10,000 persons to it.
Dashboard loads in 7 seconds. It shows the first 200 persons without gender:



All persons use a copy of the same face image, but that does not change performance or the amount of data the People View has to load. The database holds 10,000 cover images.

People View takes a few seconds to load and refresh when there are changes. Searching takes maybe 5 to 8 seconds.
Toggling the "Unconfirmed" button takes a few seconds to refresh the view.


I have not yet assigned files to each person, that will have to wait until I have more time. Should not make much of a difference, though. I will find out.

I tested this on a 3 year old PC.
In general, this seems to work just fine with 10,000 persons.
Figuring out why it takes only 7 sec. to load the dashboard here and so much longer on your computer will be tricky.
Are you sure your virus checker or other security software is not slowing down IMatch?

What I could try to do is to make completely new database and play with that a bit, reading that only from one fast SSD drive and see what will be the effect.

[Mario]

A SSD for database storage is highly recommended.
You can export your persons from your database and then import them into a new database. There will be no file assignments to the persons initially, but you will have the same mass of persons to work with.

[mastodon]

Did so see my report? Is there any cue?

[Mario]

Is this reproducible?
You did not answer my questions so I don't have much to work with.

It is usually better to file your own bug report. Else things will get mixed, lost and forgotten...

[mrmsa_2022]

What I could try to do is to make completely new database and play with that a bit, reading that only from one fast SSD drive and see what will be the effect.

I've build new database partially from files I'm using for the large database, 11000 files, 1000 persons. High speed SSD, only one drive.
What I've noticed so far:
- IMatch is more responsive, but it could be better
- At some point, hours of play with IMatch, people view got not responsive, no matter what I put into a people search field IMatch is not searching anything, steps I did continuously before that event
1. In Collection view I started to make metadata write-back manually, sorting view by "persons" pull-down menu.
2. Selecting group of persons I want to write-back, sometimes multi-selection, and used Ctrl-Alt-S for write-back
3. I noticed some names were wrongly assigned, wrong person, and I found some duplicate persons named slightly different/typos
4. Swtiched into People view and started to search those dublicates/typos by search field adding there p:"person name", this worked in the beginning but stopped working completely after sometime
5. After I found this, I noticed that what ever person visible on People view left side of the screen I do click, that person field is not changing the color indicating that to be as selected person, I had precisely the same problem with large dataset but it was more notable there from the very first moment.

I still need to work with this small dataset and look if I can found something else from there. When I close IMatch, and start again, People view works as I do except that to work. I haven't yet replicate that but I do try to do that keeping in mind my time constraints for my daily work.

[mrmsa_2022]

What I could try to do is to make completely new database and play with that a bit, reading that only from one fast SSD drive and see what will be the effect.

I've build new database partially from files I'm using for the large database, 11000 files, 1000 persons. High speed SSD, only one drive.
What I've noticed so far:
- IMatch is more responsive, but it could be better
- At some point, hours of play with IMatch, people view got not responsive, no matter what I put into a people search field IMatch is not searching anything, steps I did continuously before that event
1. In Collection view I started to make metadata write-back manually, sorting view by "persons" pull-down menu.
2. Selecting group of persons I want to write-back, sometimes multi-selection, and used Ctrl-Alt-S for write-back
3. I noticed some names were wrongly assigned, wrong person, and I found some duplicate persons named slightly different/typos
4. Swtiched into People view and started to search those dublicates/typos by search field adding there p:"person name", this worked in the beginning but stopped working completely after sometime
5. After I found this, I noticed that what ever person visible on People view left side of the screen I do click, that person field is not changing the color indicating that to be as selected person, I had precisely the same problem with large dataset but it was more notable there from the very first moment.

I still need to work with this small dataset and look if I can found something else from there. When I close IMatch, and start again, People view works as I do except that to work. I haven't yet replicate that but I do try to do that keeping in mind my time constraints for my daily work.

Continued with this and ended up again this Person view strange behavior, problem goes away after closing IMatch and start again.
Now this came after I re-scanned SSD drive, and for my surprise, I do not understand how and why, IMatch suddenly recognized there new images about 1000 files, even folders were scanned once already.
After this operation Person view was behaving completely illogical, people file numbers and unconfirmed face are not matching what could be seen when clicking a person and observing from the right side view (layout default, sorting default).

[Mario]

1000 files, even folders were scanned once already. 

Always include the log file (see log file).
It for example will tell us why IMatch has scanned the folders and how many files with a "last modified" timestamp newer than the recorded timestamp in the database were found.
IMatch does not reindex folders or files if the "last modified" timestamp reported by Windows matches what is stored in the database.

If IMatch was busy in the background, e.g. doing face recognition, all file and face counts will shift very often. Did you check that IMatch had completed face recognition? When you click a person, the current snapshot of that person is determined and loaded into the File Window. If background activity is running, the person may be updated one second later, changing all face and file counts again, depending on what face recognition is doing.

Not sure what's going on, but without a log file in debug mode (see log file) there is not even a chance to find out.

[mrmsa_2022]

One log file from today here.

[Mario]

What were you doing in that session?

I see many recluster operations for person. In this operation IMatch scans the database for faces matching a person, assigning and un-assigning persons along the way. If faces are assigned to a person, or replaced with another "better" person, this will invalidate all person- and face-related data (like file counts) very often.
The database has only 12K files, so it is quite small. Is this the database with the 9,000 persons?

[mrmsa_2022]

What were you doing in that session?

The database has only 12K files, so it is quite small. Is this the database with the 9,000 persons?

What were you doing in that session?
Organizing and confirming unconfirmed faces on Collections, "Files with Unconfirmed Faces"

The database has only 12K files, so it is quite small. Is this the database with the 9,000 persons?
This is not the same database, I made this purposely and spend days to organize that and it is still ongoing. Just to try if smaller database is having same issues like larger one.

[mrmsa_2022]

I see many recluster operations for person. In this operation IMatch scans the database for faces matching a person, assigning and un-assigning persons along the way. If faces are assigned to a person, or replaced with another "better" person, this will invalidate all person- and face-related data (like file counts) very often.

Would it be possible that notable amount of wrongly assigned persons, meaning same person is assigned for multiple names at the same time, could cause issues for IMatch behaviour?

I remember, long time ago, I tried "Automatically confirm faces" and I forgot that setting "Only when confident" for couple of days, and I noticed that it had confirmed faces here and there, most wrongly, but that was already too late.

I noticed when playing with this smaller database, which I did partially from those same images used on that large database, that names are quite frequently confirmed to wrong person, and I think this must come from the large database as metadata been written to those files already with wrong person assignment.

[Mario]

I remember, long time ago, I tried "Automatically confirm faces" and I forgot that setting "Only when confident" for couple of days, and I noticed that it had confirmed faces here and there, most wrongly, but that was already too late.

The wrong person being confirmed for a face will reduce recognition rate, especially when the face was also trained.
You should remedy that by correcting the person in the Viewer or via the Face Manager. Otherwise you will always get less than optimal results.

I noticed when playing with this smaller database, which I did partially from those same images used on that large database, that names are quite frequently confirmed to wrong person, and I think this must come from the large database as metadata been written to those files already with wrong person assignment.

If the files contain XMP person regions with tags, IMatch will use that info to create a corresponding face annotation and assign and confirm the person. This is how XMP face regions are supposed to be used.

If there are no XMP face regions in the metadata, IMatch runs face recognition and assigns the best matching person. Depending on your settings, the person will be also confirmed. In this case, wrongly assigned and confirmed persons will influence the person selected by the AI.

[mrmsa_2022]

The wrong person being confirmed for a face will reduce recognition rate, especially when the face was also trained.
You should remedy that by correcting the person in the Viewer or via the Face Manager. Otherwise you will always get less than optimal results.

I do have the latest version .14 of IMatch now, I'll go and try this again if same issue exists from where this thread started at.
At the same time, I am going to clean the wrongly assigned persons way or another so that person recognition is more accurate, I can see there to be quite many wrongly confirmed faces. 

I tested that using smaller database and noticed that recognition works very accurately when I just deleted all persons and started to assign those again for perhaps 30 person or so, it went really well.

I may need a week or so to return this thread with some conclusion and findings if anything new to report.

[Mario]

I tested that using smaller database and noticed that recognition works very accurately when I just deleted all persons and started to assign those again for perhaps 30 person or so, it went really well.

If there are no wrongly assigned persons anymore and you start "fresh" by assigning persons to faces, the AI learns and should be (within the given limits of what is doable these days very accurate and fast.
The less unconfirmed faces, the faster FR can re-evaluate when you assign a person or add a new face.

[mrmsa_2022]

When typing a person's name into the Face Manager view, the letters appear in the input field with a noticeable delay and searching for names takes a very long time.

I switched back to big database, 9000 person, and I can say that Face Manager is painfully slow. If I do remove person, like I've now done for couple of persons, updating Face Manager after that may take a minute or two.

Is there anything I could do for that because that definitely was not the case with older version of IMatch, it did that slow but it was seconds, not tens of seconds or minutes?

Could there be some background processes affecting Face Manager performance? Something I could switch off and try? Background indexing is OFF. Metadata Write-back not immediately, only manually. Anything else I could try?

[mrmsa_2022]

Is there anything I could do for that because that definitely was not the case with older version of IMatch, it did that slow but it was seconds, not tens of seconds or minutes?

I changed the People Manager layout to person thumbnails only, not showing the number of assigned and confirmed persons. This makes browsing of people way much more faster. Does IMatch count those numbers continuously as when I switch back to the view where numbers are visible, and scroll that view, it comes really slow. Does that needs to be like that? Nevertheless, when I am using the other layout just thumbs, I can scroll persons very fast.

[Mario]

These numbers are calculated on-demand, when the person scrolls into view.
And they have to be re-calculated every time you change something with a person, like assigning, confirming, ... persons or adding removing faces etc.

[Mario]

I've made some performance enhancements (and small bug fixes) for the people view for IMatch 2023.1.16.
On-demand loading improved, render time improved, parallelism improved.

I've also introduced a limit for the list-style layouts if there are more than 500 persons.
The list then shows the Show more / Show All controls like the grid layouts always show to reduce the initial work the people view has to do. This only affects users with a unusual high number of persons.

The average number of persons per database is about 150, according to telemetry.
The maximum number of persons is 16,000. The vast majority of databases manages way less than 1,000 persons.
Average number of faces is 7,000 and the maximum is 480,000 faces.

This makes your database with 9,000 a bit of an outlier and extreme.

[mrmsa_2022]

I've made some performance enhancements (and small bug fixes) for the people view for IMatch 2023.1.16.
On-demand loading improved, render time improved, parallelism improved.

I've also introduced a limit for the list-style layouts if there are more than 500 persons.
The list then shows the Show more / Show All controls like the grid layouts always show to reduce the initial work the people view has to do. This only affects users with a unusual high number of persons.

Sounds good, I do have .16 version now installed.
I can confirm on-demand loading to be notably faster than .14 version was. This is very good improvement.
From where I could find the Show more / Show All controls on list style layout?

[Mario]

From where I could find the Show more / Show All controls on list style layout?

You will see these only when needed. Usually IMatch knows when you scrolled to the "end" and loads the next batch of persons automatically. This might rarely fail (screen size, window sizes, other factors) and in that case you'll see the "Load More" and "Load All" controls.