People: How to "start over" for a person?

[Jingo]

Ok... I know this has probably been discussed many times since IM2020 has been released.. but I'm still a bit confused over how best to "start over" with a person. 

In my database, I have thousands of family images from birth to age 16... I used my existing database and started at the beginning to train the AI on some family members...  obvious to me know - this was not the best approach because new images are generally not being assigned correctly for family members that were training as babies (nor do I expect it to).  Since I had already keyworded these members, I can easily "ignore" the older photos and just start using the People feature moving forward.... I'm more interested in the AI simply adding in the proper keywords to my images via the People feature rather than using it to show the exact face in every photo.

However, I am concerned that if I simply delete the person from the DB, not only will the fingerprint still be there if I recreate it - but this note from the help might mess up existing keywords on images: Deleting a person cannot be undone.  All keywords and categories associated with the person are removed from the files with that person. Depending on how many files are linked with that person, this can take some time.

So... what is the best way to delete a person so their fingerprint is completely removed from the system and the AI can retrain while retaining existing keywords that already are attached to the image?

Thx - Andy.

[Mario]

A person has many fingerprints - one for each face the person is assigned to.

I would not do anything rash now. I have planned some additions regarding training people and checking / correcting wrongly trained persons.

But, if you delete the person, all face associated with it become unassigned and are re-assigned to other persons later maybe.
When you add a new person and assign faces / confirm automatically assigned faces, the person will be trained fresh.

As said in the People help topic, try to assign/confirm some representative faces first. Say, the first 20 or 30 faces you assign to the person - assuming that the person appears in your photos at very different ages or a large variety of facial expressions. For typical faces in typical camera-facing portrait / group photos, 3 to 5 faces for training are usually enough to exceed a 92%-95% recognition rate.

[hluxem]

try to assign/confirm some representative faces first.

That's ok if you start from scratch.
I imported a lot of face annotations. Mostly family members over the past 30 years, baby pictures with sun glasses and hats and all different angles. So I can see that the fingerprint is not very good for these pictures.

I do like the idea of some additions regarding the training of people. I would be fine to just reset a persons fingerprint and then retrain again.

Heiner

[Stefan]

I would be fine to just reset a persons fingerprint and then retrain again.

+1
As I had too much trouble with too many annotations and unconfirmed faces I just started with a fresh database (without writing back the metadata and annotation of the images I had made so far). So having a "reset person fingerprint" would be better than starting a new database.

[Mario]

A person has no fingerprint.
A person has 1, 100 or ... 10,000 assigned faces. Each face has a fingerprint.

To find a matching person for a face F, IMatch theoretically needs to compare all other faces with a person link in the database with that face.
But this would be way to slow. So IMatch picks a number of faces for each person, say 20 or 30, based on some mathematical model to get a fairly 'wide' range of possible faces for any given person. And these faces are then used to perform comparisons. IMatch builds this face set based on trained faces, where faces with persons manually assigned by the user (via the person selector) or confirmed are considered as trained.
Depending on how a user works and how 'good' the automatic face recognition works for him/her, a database may contain 1, 5, 10 or several hundred of trained faces for each person. There are no rules.

Showing the user the trained faces somehow and allowing him/her to 'untrain' one or more of these faces may help users who encounter sub-optimal face recognition performance caused by either the nature of their image collection, their assignment / confirmation workflow or similar factors. That's my general idea. I'm still working on this.

Currently, bug reports have priority. I need to get as many bugs weeded out before the mass of users migrates to IMatch 2020.

[hluxem]

That's my general idea. I'm still working on this.

Not sure if it helps as I don't know much about the AI technology, but here are some thoughts I had in regards to face annotations. Not worth a FR but it may trigger some other ideas.

As life moves on, so does the circle of people you take pictures of. Out of my 650 persons I could exclude a lot of people from being a potential match. Should certainly be faster, but maybe the recognition rate goes up too.

Only use images from the past x years as a reference for a face or consider birthdate and date taken in image in the process.

Not quite serious, but makes me smile. I saw multiple times 2 faces in one image tagged as the same person. Most of the time a person only shows up once in an image . Although on a recent vacation I realized how you can use the panorama function to show a person multiple times.

Currently, bug reports have priority.

Agreed!

[Mario]

What is written in a few lines in a forum post may actually take weeks to design and implement.

I have many photos showing the same people: Studio images with mirrors, event posters, images with twins or triplets, images with reflections in water, ... not that uncommon.

[Jingo]

A person has many fingerprints - one for each face the person is assigned to.

I would not do anything rash now. I have planned some additions regarding training people and checking / correcting wrongly trained persons.

Ok.. I'll hold off for now... I'm really interested in getting this to work moving into the future rather than linking 20 years of photos for one person... so, if training on current photos will make future imports correct - that is a better approach for me.

My concern with removing the "older images" from the person is that the keyword will be removed as well according to the help... I suppose I can just add these to a temp collection or result tab ahead of time and then re-assign the keyword.

BTW: for those interested... I had included these articles about the AI behind the face detection... it got buried in the other thread but is quite interesting so I figured I would include the links here again... enjoy!

https://www.pyimagesearch.com/2018/06/18/face-recognition-with-opencv-python-and-deep-learning
http://blog.dlib.net/2014/02/dlib-186-released-make-your-own-object.html

[plastikman]

Other software I use has no problems whatsoever with my personal photos from me being a baby (scanned analog photos) till 35 years old. I believe they can trace the evolution and might have multiple phases of your face profile stored. Only with babies etc. it is often important to tag a lot of photos for the system not to confuse them with other babies. Also it shouldn't be that hard to bias the recognition based on the most recent matches in terms of image creation date.

[Mario]

"Other software" being?

It is pretty hard for face recognition to compare the face of a 6 month old toddler with that of a 70 year old person. Even if it is the same person.

[Stefan]

I have many photos showing the same people: Studio images with mirrors, event posters, images with twins or triplets, images with reflections in water, ... not that uncommon.

While it is not uncommon, it is kind of unlikely as I encountered this issues in almost every picture that has 3 or more unconfirmed faces. I had several group photos with around 10 people and family members (that were trained and confirmed as person in the database with hundreds of pictures), but very often none of the persons in the group photo are correctly annotated by the face detection, like 8 out of 10 people get the same name suggestion. And the only person where the name would fit gets actually also a wrong name. So if I have persons A,B,C,D,E,F,G,H,I,J from left to right in the photo the automatic detection suggests B,K,B,B,B,B,B,L,B,B or something similar ... even more so after more and more trained persons.
As I assume that the recognition is based on likelihood, I think it would make sense to give the name with the highest likelihood in the photo that name and then at least reduce (not exclude) the likelihood of names that would be repeating.

[Mario]

So if I have persons A,B,C,D,E,F,G,H,I,J from left to right in the photo the automatic detection suggests B,K,B,B,B,B,B,L,B,B or something similar ...

it is impossible to understand what you mean without actual samples.
I don't remember having encountered anything similar. And I have tested IMatch with larger sets of choirs and group photos and large family photos.

Hard to tell. Feel free to send me a set of sample images. You can resize them as long as the images are > 100px and zip them up. I will only use them for analysis and destroy them after the analysis is done.

Using my own files and larger sets of "in the wild" photos collected from the Internet gives me actually a great variety in my 100K sample library. From high-gloss studio photos down to crappy smart phone images. But there are always edge cases...

[Mario]

Ok.. I'll hold off for now... I'm really interested in getting this to work moving into the future rather than (...)

One feature (among several other planned for the next couple of releases) will be a way to see the faces you have trained and to 'untrain' faces.
I plan this as an extension to Person Editor. It will look like this:

[plastikman]

"Other software" being?

It is pretty hard for face recognition to compare the face of a 6 month old toddler with that of a 70 year old person. Even if it is the same person.

Lightroom, Google Photos and Apple Photos. They pick out all my childhood photos as well as my more recent adult ones without a hitch. Google Photos is even quite good with named pets, something other software struggles with a lot.

[Jingo]

"Other software" being?

It is pretty hard for face recognition to compare the face of a 6 month old toddler with that of a 70 year old person. Even if it is the same person.

Lightroom, Google Photos and Apple Photos. They pick out all my childhood photos as well as my more recent adult ones without a hitch. Google Photos is even quite good with named pets, something other software struggles with a lot.

As Mario has said in the past - Google, Apple and Adobe have hundreds of programmers doing nothing but work on recognition AI... and they are probably spending hundreds of millions on the project.  For a single developer to incorporate this type of AI into software that costs less than $100... simply outstanding!

[Jingo]

Ok.. I'll hold off for now... I'm really interested in getting this to work moving into the future rather than (...)

One feature (among several other planned for the next couple of releases) will be a way to see the faces you have trained and to 'untrain' faces.
I plan this as an extension to Person Editor. It will look like this:

This looks fantastic and will be exactly what folks need to help identify and narrow down photos that may be deflecting the system away from ID'ing certain faces... or for me - to remove hundreds of baby photos from someone who is 16 and for which I will only use the AI moving forward.  Even more important - it will allow you to KEEP those already id'd faces in the Person while just removing their fingerprint moving forward.... outstanding!

[Tveloso]

This looks fantastic and will be exactly what folks need to help identify and narrow down photos that may be deflecting the system away from ID'ing certain faces... or for me - to remove hundreds of baby photos from someone who is 16 and for which I will only use the AI moving forward.  Even more important - it will allow you to KEEP those already id'd faces in the Person while just removing their fingerprint moving forward.... outstanding!

Yes, fantastic!

Mario, in addition to the functionality for keeping the Face Assignment, and just removing the selected fingerprints from training the AI, could this function also provide a "not this person" command?...(so we can clean up incorrect assignments as well)...

[Stefan]

One feature (among several other planned for the next couple of releases) will be a way to see the faces you have trained and to 'untrain' faces.

Hi Mario, that would be great. It would not only be a tool for the users to correct mistakes they made, it would be a good debugging tool, too. At least that would be my hope to understand better why my recognition rate is so bad and it would be easier to support you with trouble shooting.
Just make sure that in case the list for one person could get very long (more than the 30 you mentioned, like hundreds of faces), that all trained faces can be removed at once to be able to quickly start over.
Thanks!

[hluxem]

That looks nice!

As someone with 600+ imported persons it would be nice if there is somewhere an option to reset all fingerprints for the database, maybe under database tools or utilities. 

I agree with Tveloso on the additional "not this person" command. A window like this would be ideal for batch processing faces too.

[Mario]

this function also provide a "not this person" command?.

Won't work for several reasons in relation to this.
Trained faces are always confirmed. Not this person often not.
Trained faces are only a subset of all faces in your database.
Very unlikely that you have hundred of trained faces, unless your images are so special that you had to assign each and every face manually. But then there is something wrong - which you cannot fix. I would need to see the images causing the problem to see what the AI makes of them.
And other reasons.

As I said above, more features are coming in future versions.
Keep in mind that IMatch 2020 is less than two weeks old. And that I have been working extremely long hours over the past two weeks to get all the emails and community posts answered.
And I've fixed over 50 bugs reported by early adaptors.

This leaves barely room for new features.
I have read most of what's been posted in the FR board and many things posted there are already on my road map for this year.

Currently my main focus is getting my inbox empty and all bugs reported so far fixed (almost done!).
Bugs affecting more than one or a handful of users always get priority over new features.

[jch2103]

One feature (among several other planned for the next couple of releases) will be a way to see the faces you have trained and to 'untrain' faces.

I agree with the others who think this will be very helpful. Should save a lot of time in the future in dealing with faces.

[plastikman]

this function also provide a "not this person" command?.

As I said above, more features are coming in future versions.
Keep in mind that IMatch 2020 is less than two weeks old. And that I have been working extremely long hours over the past two weeks to get all the emails and community posts answered.
And I've fixed over 50 bugs reported by early adaptors.

This leaves barely room for new features.
I have read most of what's been posted in the FR board and many things posted there are already on my road map for this year.

Currently my main focus is getting my inbox empty and all bugs reported so far fixed (almost done!).
Bugs affecting more than one or a handful of users always get priority over new features.

Nothing but respect for you Mario! Once I decide to buy I have no problem waiting for things to improve, what is there is already very good. Just wanting to make sure it has all my needs covered.

The new features in this release work better, faster and more user friendly than what Adobe brings to the table and they must have at least 20+ developers on Lightroom that leverage the work of more likely 100+ developers working on Creative Cloud and other Adobe products with shared frameworks.

I must even say that this Events and People features in this release remind me of the Steve Jobs Apple era of personal computing, basically early iPhoto period. Super user friendly, well thought out and still very much customizable to individual needs.

[Mario]

they must have at least 20+ developers on Lightroom

Adobe has over 22,000 employees world-wide.
To get an (incomplete) impression of how many people work on each product, check the "About" dialog of the software.
For the current Lr, the About dialog lists about 200 people working on Lr (and incorporated technologies like RAW).