Side portrait are not recognised as a face

[mastodon]

I found that most of times side portrait are not recignised as a face. Why?

[jch2103]

The face recognition algorithms (not just in IMatch) generally expect face-on portraits (they use information like distance between eyes/nose/ears/etc. to recognize faces). With a side portrait, much of this information is missing...

[mastodon]

Picasa did that quite well. BUT with face-on portraits IMatch does much faster and better than Picasa. So, it is a feature of the algorithm. OK, I know Mario, it will we better soon.

[Mario]

This is just a limitation in the algorithms used and how the IMatch face AI works. Not all cases can be handled.

If Picasa does this better:

a) Google has 150,000 employees and the majority of them are much smarter than me
b) Use Picasa for these faces, store face in XMP, import in IMatch
c) Or use a manual face annotation

[DigPeter]

This is just a limitation in the algorithms used and how the IMatch face AI works. Not all cases can be handled.

@Mario
I sent you some images of faces in profile before latest version was available.  Are you saying that in general, face profiles are not detectable?  I do not mind if they are not, but just would like to know.

[Mario]

I sent you some images of faces in profile before latest version was available

I have read and replied to over 600 emails in the past 3 weeks.  Plus the community. Plus all the bug fixes and changes I've done in these 3 weeks....
Can you be a bit more specific, e.g. the date and email used?

Detecting frontal faces and angled faces and sideway faces requires differently training AIs. A different set of of maybe 100,000 to 200,000 labeled files. Several weeks of 24/7 computing time training each.
Creating an AI that handles all faces is impossible. There are always trade-offs. If you train the AI to detect sideway faces better, it will suddenly fail to detect many frontal faces. Or angled faces.

Like all AIs the AI in IMatch is tuned for portrait situations, family photos, groups etc. This is what works best for 99% of the user base.
If many of your images use sideway faces or faces in profile and IMatch does not handle them, there is nothing I can do. This is as good as it gets for now.

[medgeek]

a) Google has 150,000 employees and the majority of them are much smarter than me

The first part may be true, but the second surely is not!

[PaulS]

The first part may be true, but the second surely is not!

+1

[mastodon]

I think the face recognition of IMatch is a BIG improvement. Recognising front faces better than side ones is a feature and not a bug.
With IMatch 2020 the handling of family photos is very easy, so standing ovation for Mario!

[Mario]

I think the face recognition of IMatch is a BIG improvement. Recognising front faces better than side ones is a feature and not a bug.
With IMatch 2020 the handling of family photos is very easy, so standing ovation for Mario!

Just wait. But I'm not finished yet. The next release will contain the Face Reviewer feature, which makes confirming, rejecting and training faces even easier.

[Tveloso]

The first part may be true, but the second surely is not!

+1

+1

Just wait. But I'm not finished yet. The next release will contain the Face Reviewer feature, which makes confirming, rejecting and training faces even easier.

Awesome!  Can't wait to see it!

[ColinIM]

The first part may be true, but the second surely is not!

+1

+1 again ...

Certainly not the majority of Google employees 

[plastikman]

I sent you some images of faces in profile before latest version was available

Detecting frontal faces and angled faces and sideway faces requires differently training AIs. A different set of of maybe 100,000 to 200,000 labeled files. Several weeks of 24/7 computing time training each.
Creating an AI that handles all faces is impossible. There are always trade-offs. If you train the AI to detect sideway faces better, it will suddenly fail to detect many frontal faces. Or angled faces.

Like all AIs the AI in IMatch is tuned for portrait situations, family photos, groups etc. This is what works best for 99% of the user base.
If many of your images use sideway faces or faces in profile and IMatch does not handle them, there is nothing I can do. This is as good as it gets for now.

Would it be possible to split these processes and have a front and side profile for each person that is run in a separate process but still connected to the same people ID profile?

I understand how you put your resources, makes total sense to focus on the front faces.

[Mario]

IMatch is very well able to identify profile faces (to some extent), angled faces, partially occluded faces, faces with glasses, changing hair style or beards, ...
But not every case can be handled.

Companies like Google/Microsoft/Facebook/Amazon spend years and millions over millions of dollars to create and improve their face recognition technology.
You cannot expect the same from me. I have neither the resources, nor the money not the time these companies put into this technology.
These companies know they will reap billions from exploiting face recognition to track us all and everywhere. And then sell this data.

My AI will satisfy a majority of the user base when I'm lucky. And I may sell a couple of additional licenses because IMatch now has integrated face recognition.
And yes. I will also work on that over the years to come, adding improvements as publicly available AI technology evolves.
This is just the IMatch AI 1.0. And it is about 3 weeks old.

I've made the explicit decision to not integrate cloud-based face recognition technology in IMatch. Although the results may (!) be better for some edge cases. Not sure.
Because using cloud-based face recognition technology means sending images of your family and friends to the servers maintained by these companies.
Which not only creates legal issues (GDPR) but IMHO is not a good service you do to your friends and family.
When you look at the fine print in the agreements you have to sign before using this technology (similar in the Facebook usage agreement etc.), you know that they will 'keep' the images forever and use them to "improve their technology". Which can mean anything - and usually means your images help them to track you and to make money.
Google ClearView for some background.

There are plenty of specialized face recognition applications out there.
And as long as they can store the resulting information as standard XMP face regions, IMatch can import and use that data.
If your images are not handled well by the IMatch AI, this gives you other choices without losing the integration with IMatch.
Whatever works for you is good.

[DigPeter]

I sent you some images of faces in profile before latest version was available

I have read and replied to over 600 emails in the past 3 weeks.  Plus the community. Plus all the bug fixes and changes I've done in these 3 weeks....
Can you be a bit more specific, e.g. the date and email used?

Detecting frontal faces and angled faces and sideway faces requires differently training AIs. A different set of of maybe 100,000 to 200,000 labeled files. Several weeks of 24/7 computing time training each.
Creating an AI that handles all faces is impossible. There are always trade-offs. If you train the AI to detect sideway faces better, it will suddenly fail to detect many frontal faces. Or angled faces.

Like all AIs the AI in IMatch is tuned for portrait situations, family photos, groups etc. This is what works best for 99% of the user base.
If many of your images use sideway faces or faces in profile and IMatch does not handle them, there is nothing I can do. This is as good as it gets for now.

Thank you Mario - I just wanted to be clear about this.  I do not have many profile views.  I was not moaning or complaining.  I think that your AI is doing a great job.

[Mario]

A "face" for the AI has two eyes, two eyebrows, a mouth, a nose, two ears, a chin etc.
If several of these face landmarks are missing, face detection (finding faces in an image) and/or face recognition (matching trained person faces to the face) will have a hard time.

Unless you train the AI explicitly with faces in profile, where only one eye, one eyebrow, one ear etc. is visible. But then of course all regular faces will fail to detect.
The way to deal with this is to run "multiple AI's" and check each image and face with each of these AIs. Which is possible, but maybe not practical.

As I said above, this is the IMatch AI 1.0.
And there was a long way for IMatch 1.0 to IMatch 2020.
A good chance that the AI will improve over the years.

If you can send me some of your problem files or let me know the date you have sent them, I can add them to my test suite and use them when I'm improving the AI.

[DigPeter]

If you can send me some of your problem files or let me know the date you have sent them, I can add them to my test suite and use them when I'm improving the AI.

Email to support address on 7 March, reference  https://www.photools.com/community/index.php?topic=9822.0;topicseen

[Mario]

Ah. 7. of Match. Several hundred emails back. No wonder I could not remember...
Thanks. I will include your images in my test library (no public exposure or distribution). All these faces are indeed in full profile or even facing mostly away from the camera. Almost impossible to recognize unless a neutral network trained explicitly for this is used. And this would need to be very specific...

[Tveloso]

I've made the explicit decision to not integrate cloud-based face recognition technology in IMatch. Although the results may (!) be better for some edge cases. Not sure.
Because using cloud-based face recognition technology means sending images of your family and friends to the servers maintained by these companies.
Which not only creates legal issues (GDPR) but IMHO is not a good service you do to your friends and family.

Thank you so very much for that Mario.

I was very excited to see Face Recognition in IMatch...and doubly so, about the fact that it's a "Local AI" that doesn't use any Cloud-Based Services.

I was also initially excited about Auto-Tagger, but haven't yet started to learn about it...and am having second thoughts about whether or not I'll actually use it, for that very reason (not sure I want to send my family photos to the cloud service provider).

[Mario]

This is your decision of course.
And I want to explicitly remark  that I have made extra efforts to not force you to use one specific AI service. I have created integrations for four popular services. You can choose between Clarifai, Google, imagga and Microsoft. All have their pro's and con's - technically and legally.

'Private' users usually can get good results with manual keywording using a well maintained controled vocabulary in the thesaurus. And the many time-saver features IMatch offers for working with keywords. For institutional or corporate users, or people starting fresh with DAM and facing 300,000 files without keywords, AutoTagger can be invaluable.

[Tveloso]

For institutional or corporate users, or people starting fresh with DAM and facing 300,000 files without keywords, AutoTagger can be invaluable.

I absolutely agree that AutoTagger can be invaluable...even for my small less than 50K-File Database, I was very excited to start using it.

'Private' users usually can get good results with manual keywording...And the many time-saver features IMatch offers for working with keywords.

Those time-saver features (and the concern about using a cloud service) are why I started going back and forth on whether to use Auto-Tagger.  My initial impression was that I would use it to "get caught up" on KeyWording, and then just stay on top of it for my on-going indexing, with the existing powerful IMatch features, and by developing/improving my workflow in that area.

In the end though, I'll probably read the Help on the Auto-Tagger, get inspired by it, and then say "Heck with it - I'm doing it", and take the plunge with one of the Services IMatch supports (probably Google Vision).

Thank you so much Mario.

[Mario]

All services offer a generous free monthly quota (usually 1000 images) so you can try without risk.