External Face detection

[PierreC]

Hello
After some work and time, I am now a big user of the Faces module of IMatch, and a happy one, for a personal database of 60K+ photos.
The main tool I use for confirming faces is Face Manager (not the viewer) as I can process large quantities of faces detected by iMatch in the pictures very fast.

My main issue so far is the first step of Face detection :detecting that there are faces in the pictures : I have recently added a large number of photos coming from iphone, and the detection of faces is sometimes very good, sometimes very bad (probably due to the pictures themselves as people may have sunglasses,... ) but the point is that I have now several 000s of photos where one face is detected out of 3 persons obviously (to my human eye) in the photo or no face is detected while there are obviously (to my human eye again) people in it.
As I don't want to scroll all my new pictures one by one with the viewer and manually add, this is now a big issue for me.
Mario advised me to look at external tools to get the faces detected (and so have the annotation "green frame" added by an external tool), then have iMatch take over and assign and manage the person to the face + my tuning, thru Face Manager.

My question to the community is : does anybody use an external tool for this first step of face detection, and then iMatch to assign and manage the persons, for significant number of pictures? If so which tool?

Thanks

[Mario]

If you have no problem uploading your person images to companies like Google or Microsoft (and you have the explicit (written) permission of all people depicted in your photos when you live in the EU (GDPR reasons)) you can do some tests to see if Google/Microsoft with their vast computing resources and AI systems can detect your faces better. They provide freely accessible test sites. Just search for "Google face detection test" or "Microsoft Face Detection test".

Partially occluded faces are always difficult and it depends on how the AI was trained if it can be successful at detecting them or not. An AI trained to detect faces with sun glasses will often fail to detect normal faces.

I though Apple phones all have built-in face detection?
I know that because Apple messes up how they store XMP face regions and this (for some users) causes issues when they let IMatch automatically import XMP face regions from images taken with their iPhones.
Are your images taken with an iPhone? Do they contain face regions in XMP? In that case, make a check if the imported face region is really enclosing the face. Apple devices often exports the XMP face region wrong (with a horizontal or vertical shift) and this will upset the face detection in IMatch. Apple compensates for this bug in their own software, but nobody else can.

[PierreC]

Yes my photos are taken with iPhone, but I had to convert a number of them from HEIC to JPG

How do I find out about the XMP face regions in the file?

I am also interested in experience of others using external tools, if any

[Mario]

How do I find out about the XMP face regions in the file?

ExifTool Command Processor.
List Metadata
Search for Region on the right side.

[PierreC]

So I ran the ExifTool Command Processor for the first time / Preset = List Metadata
Which text should I look for in the output?

Thx

[thrinn]

Search for Region on the right side.

I think this answers your question?

[PaulS]

You can also use the following arguments in ExifTool Command Processor to only return the MWG Region metadata:

-G1
-XMP-mwg-rs:all
-a
{Files}

If these are the tags that Apple is using, this may make it easier for you.

[Tveloso]

I have recently added a large number of photos coming from iphone, and the detection of faces is sometimes very good, sometimes very bad (probably due to the pictures themselves as people may have sunglasses,... ) but the point is that I have now several 000s of photos where one face is detected out of 3 persons obviously (to my human eye) in the photo or no face is detected while there are obviously (to my human eye again) people in it.

The thing that may need clarification here is whether the Face Annotations in question are actually being created via Face Detection in IMatch (Ctrl-M, F), or via importing the XMP Face Regions that the iPhone images already contained.

In my experience with indexing iPhone Photos, I was surprised to find that the resulting Face Annotations were sometimes not very good in terms of placement, and that in some cases not all people were being detected, when, to my human eye - as you said - they should have been (their faces were not averted, or obscured).  But in particular, it was the size and placement of the Face Regions by the iPhone, that I "had trouble with".

For example, given two faces that are very close together:

                      ,-.,~~.     
           ,-.,~~.  ,'///||\\`.   
         ,'///||\\`.//(((||)))\\.
        ///(((||)))\(         )))
       (((         )))        |(_
       _)))        |/\       /\\_.
      ._//\       /\_/`-._.-'\-`-'
      `-'_/`-._.-'\- \/=._.=\/ hjw
        ' \/=._.=\/
        (*Note: this ASCII Art was copied from this Web page: https://www.ludd.ltu.se/~vk/pics/ascii/junkyard/techstuff/tutorials/Hayley_Wakenshaw.html)

...the iPhone sometimes created very large face regions:

          ......................
          :                    :
     .....:................    :
     :    :           ,-.,:~.  :
     :    :,-.,~~.  ,'///|:\\`.:
     :   ,:///||\\`.//(((|:)))\:.
     : // :(((||)))\(     :   ):)
     : (((:        )))    :   |:_
     : _)):        |/\    :  /\:_.
     :._//:       /\_/`-._:-'\-:-'
     :`-'_:...............:....:
     :  ' \/=._.=\/       :
     :....................:

...such that when looking at one of them in Face Manager, it can be difficult to determine which person exactly, the Face Region is supposed to be enclosing:

     ......................
     :                    :
     :                    :
     :           ,-.,~~.  : 
     :,-.,~~.  ,'///||\\`.: 
     :///||\\`.//(((||)))\:
     :(((||)))\(         ):
     :        )))        |:
     :        |/\       /\:
     :       /\_/`-._.-'\-:
     :....................: 

So after Mario gave us the ability to separately configure Importing XMP Face Regions at Ingest, and Creating XMP Face Regions (from Face Annotations) during Write-Back, I turned off the Import side, and am using Face Detection in IMatch exclusively.

You may already know this, but in your case, you should determine whether the Face Annotations that you're finding are not always correct, have really been created by IMatch...(it could be that they're the existing Face Regions that the iPhone has created - and not from Face detection in IMatch).  There are two options to ensure that the IMatch Face Detection is where the Annotations have come from:

• Turn off the Import XMP Face Regions parameter
  In Edit->Preferences->Metadata 2, set the Import XMP Face Regions parameter to No, so the iPhone's XMP Regions data does not create Face Annotations at Ingest.  Then use IMatch Face Detection to create the Face Annotations, either manually (via Ctrl-M, F), or by setting the Automatc Face Recognition parameter to Yes, in Edit->Preferences->Application.
  
• Uncheck the Ignore images with existing face annotations option
  When running Face Detection manually (Ctrl-M,F), if you un-check the Ignore images with existing face annotations option, this allows IMatch to replace the "Apple-created" Face Regions even if they have been imported.
  

As I write this, I'm thinking that it might in fact be interesting to turn the Import XMP Face Regions parameter back on in my IMatch installation, in order to see how the two Face Recognition technologies compare (by first reviewing the imported XMP face regions, and then replacing them, by letting IMatch re-process those files during Face Detection).