Should I apply Face Tags to obscured or turned faces?

[Damit]

I have been waffling back and forth on this and my searches on the web have not been fruitful. There does not seem to be at least a partial consensus on the proper methodology to tag faces. Some only tag visable faces, some also include partial faces and some include the back of the head. In fact, one topic I encountered in my search was several articles addressing Facebook's ability to tag the back of the head. Up until now I have been tagging faces, with annotation regions, to people who have partially or fully obscured faces, such as when their face is turned. My logic is that they are in the picture, though their face may not be showing. Then I thought about how I would be using face tags and if I wanted to have a bunch of photos where the person's face is not visible. 

One thing I am planning on employing is a Present tag, where I tag all photos where a person, to my knowledge, was present, even though they are not in the photo at all. Maybe they are behind the camera, taking the picture. This way, photos that are pertinent or have some meaning to the person, as they were present for the event, can be grouped for whatever purpose, perhaps to be displayed in a slide show. Still, just because someone was present, does not mean they are in the picture, so this distinction alone does not preclude me from tagging obscured faces, as there is a difference between being in the picture with your face obscured, being in the picture with your face visible or just being present at the event where the picture was taken.

One issue is that I am thinking that I would like to be able to differentiate if a person face-tagged in a picture has a visible or obscured face. I really can't figure out how to do this other than make a duplicate people tag with a suffix or (Obscured), but that is an ugly and messy solution I do not want to even entertain. So, as of now, I am considering just doing what I have been doing and deal with having some pictures where the face is not visible.

How are others tagging their faces?  Any advice on dealing with this problem would be greatly appreciated.

[Mario]

IMatch offers the unique feature to link persons to images, even if the face is obscured and cannot be detected. That's one way to deal with this. If you are only interested in the fact which persons are in an image, this is quick and easy.
And a person being linked implicitly conveys the information that the face was not detectable for some reason.

Another way is to manually place face annotations, even if the person is only shown from the back. This links the file to the persona and allow gives IMatch the spacial data to decide whether an image contains a single person, two persons or a group of persons. This is reflected in the standard Category Face Arrangement category (see Face Arrangement).

You can also use categories like "Partially Obscured", "Face Invisible" etc. to further enrich the information recorded by the image. If you work in the Viewer and keep the Category Panel open, this is quick to do. Or create a number of favorites and execute them with <F3>.

Or maybe add a keyword like "obscured faces", if this works better.

Not sure how Facebook could recognize persons from the back of their head, without any face features at all visible.
On the other hand, Facebook getting millions of people to feed the Facebook AI for free by manually tagging friends and family members in photos was a good trick. They made it sound cool and social, but actually just wanted to know how every Facebook user (and their friends and family members) look.
Very precious data, very easy to link with other data they collect and worth millions over millions when selling the data to other companies and Facebook selected partners.
Maybe their AI has learned to identify people from the surroundings or from other persons in the image. At least well enough to make suggestions.

[Damit]

Thank you so much for your well considered response.

I was thinking of using links, it is a good idea, but I still want to reserve that option instead of Face Tags in the future. Face tagging can be quite laborious and tagging a bunch of images at once is becoming more appealing, which is probably one of the reasons it exists. I am surprised that links are not used for face arrangements information. 

Your category idea is interesting, but I would have to make one for each person so as to designate which face was obscured. At least I can find solace in knowing that others tag obscured faces and there may be good reasons for doing so. I think I may just continue with my current strategy unless someone else suggests something that is better.

[Mees Dekker]

I've been dealing with this problem too.

What I use as a workable solution:

• go to the Timeline tab and find all files for "to day". That shows all pictures taken on this specific date over the years. In my case I find a couple of 100's files per day.
• I then revise each of them. Either by using the automated face recognition or (placing by hand) individual face annotations.  
• It will take a full year to "hit" all pictures/files in your database, but at least it keeps the workload manageble.

[PandDLong]

I waffled over this one as well.  

I started with adding face annotations for partial, obscured and back of heads as it associated the person with the picture (which was my primary goal).

When the feature was added for manual links, I found it a nice upgrade.  I now use face annotations for faces - including partials - but I use the manual links for people without their face in the picture (perhaps it is even just some body parts without a head!).   This way everyone associated with a picture is identified and can be found through the People function and/or the 'People in Image' tag (which I use for a category).   

I have found the manual link faster to add and the data is now available to understand that the face is not visible.  I will likely refine my categories in the future to be able to just identify people with visible faces (while still having the data to see all pictures for a given person - with or without a face).

Everyone has their unique balance of effort and reward (and keeping future options).  This is working for me (for now).

Michael

[Tveloso]

I now use face annotations for faces - including partials - but I use the manual links for people without their face in the picture (perhaps it is even just some body parts without a head!).

I have manually placing Face Annotations, even in the case of body parts without the head.  But using Face Links for that situation sounds like a much better option.

As Mario says:

And a person being linked implicitly conveys the information that the face was not detectable for some reason.

Thank you! I'm going to start using links in this situation as well

[Damit]

I waffled over this one as well. 

I started with adding face annotations for partial, obscured and back of heads as it associated the person with the picture (which was my primary goal).

When the feature was added for manual links, I found it a nice upgrade.  I now use face annotations for faces - including partials - but I use the manual links for people without their face in the picture (perhaps it is even just some body parts without a head!).   This way everyone associated with a picture is identified and can be found through the People function and/or the 'People in Image' tag (which I use for a category).   

I have found the manual link faster to add and the data is now available to understand that the face is not visible.  I will likely refine my categories in the future to be able to just identify people with visible faces (while still having the data to see all pictures for a given person - with or without a face).

Everyone has their unique balance of effort and reward (and keeping future options).  This is working for me (for now).

Michael

I now use face annotations for faces - including partials - but I use the manual links for people without their face in the picture (perhaps it is even just some body parts without a head!).

I have manually placing Face Annotations, even in the case of body parts without the head.  But using Face Links for that situation sounds like a much better option.

As Mario says:

And a person being linked implicitly conveys the information that the face was not detectable for some reason.

Thank you! I'm going to start using links in this situation as well

Thank you both for your input.  It seems that the idea for using links for undetectable faces has been used by others and perhaps this is the way to go. I still want to grasp on to it for those bursts with a single individual where face recognition may catch 80% of the faces and tag them, but then you still have to into the set to label the the others with regions.  It would be SO much easier to just select all those shots an link a person to them.  There are many examples this functionality would make my life easier. 

That being said I have still used face annotations because I am anal and I like to see those boxes around faces. I guess I need to decide on a path to follow soon, I just don't want to regret that decision.

Now if only we could get filtered face detection, where you can tell IMatch there is a face and the face will be this person, or limited to a set of persons, my decision would be easier.

[Damit]

When the feature was added for manual links, I found it a nice upgrade.  I now use face annotations for faces - including partials - but I use the manual links for people without their face in the picture (perhaps it is even just some body parts without a head!).  This way everyone associated with a picture is identified and can be found through the People function and/or the 'People in Image' tag (which I use for a category). 

I have found the manual link faster to add and the data is now available to understand that the face is not visible.  I will likely refine my categories in the future to be able to just identify people with visible faces (while still having the data to see all pictures for a given person - with or without a face).

As Mario says:

And a person being linked implicitly conveys the information that the face was not detectable for some reason.

Thank you! I'm going to start using links in this situation as well

Thank you both for your input.  It seems that the idea for using links for undetectable faces has been used by others and perhaps this is the way to go. I still want to grasp on to it for those bursts with a single individual where face recognition may catch 80% of the faces and tag them, but then you still have to into the set to label the the others with regions.  It would be SO much easier to just select all those shots an link a person to them.  There are many examples this functionality would make my life easier. 

That being said I have still used face annotations because I am anal and I like to see those boxes around faces. I guess I need to decide on a path to follow soon, I just don't want to regret that decision.

Now if only we could get filtered face detection, where you can tell IMatch there is a face and the face will be this person, or limited to a set of persons, my decision would be easier.
[/quote]

[Mario]

It would be SO much easier to just select all those shots an link a person to them.  There are many examples this functionality would make my life easier.

Why don't you do that?
When you apply a person link, IMatch does that for all selected files.

Now if only we could get filtered face detection, where you can tell IMatch there is a face and the face will be this person, or limited to a set of persons, my decision would be easier.

This is what you do when you assign a person to a face, or when you place a manual face annotation since IMatch was unable to detect a face. If IMatch cannot assign a person to a face, it is irrelevant if you give it a subset of persons to choose from or not. Except the set has only one person.

[Damit]

It would be SO much easier to just select all those shots an link a person to them.  There are many examples this functionality would make my life easier.

Why don't you do that?
When you apply a person link, IMatch does that for all selected files.

Oh I am strongly considering doing this, but as I said, I am anal and I like those annotations coupled with a face region centered on the person's face. I just have to determine how feasible it is for me to do draw face annotations and label unrecognized faces for all my pictures and even if it was feasible, would I be better off waiting 5 years for AI to reach the level where it could do this almost as accurately as I?

I know myself, and as much as I want to say I will be practical, I probably will choose to continue face tagging rather than linking.

But you mentioned something in your first response that also got me thinking.  Does the linking of faces to an image provide IMatch with that spacial data used for face arrangements? If, not, then this is another reason to use Face Annotations rather than just linking.

Now if only we could get filtered face detection, where you can tell IMatch there is a face and the face will be this person, or limited to a set of persons, my decision would be easier.

This is what you do when you assign a person to a face, or when you place a manual face annotation since IMatch was unable to detect a face. If IMatch cannot assign a person to a face, it is irrelevant if you give it a subset of persons to choose from or not. Except the set has only one person.

Yep you are right.

[Mario]

You can add links first, then revisit the files when you have time and drag the link to the actual face.
This will convert the link to a regular face annotation.

Does the linking of faces to an image provide IMatch with that spacial data used for face arrangements?

No. Links are placed in the upper right corner an d have neither a place nor dimension. They cannot be used for face arrangements.

be better off waiting 5 years for AI to reach the level where it

Google and Facebook have invested large sums of money into face recognition.
Google because it wants to identify persons in images in's bot harvests from all web sites. To monetize and sell the info.
Facebook to identify persons in images uploaded to face book to monetize the information and sell the data.

Even after having spent all that money, none of their AI is 100% correct. Not even people are always 100% correct.
The AI I have developed for this purpose is pretty good, but not 100% and probably never will be. If a face cannot be detected, it could only guess.

When you look at the photo, you probably recognize the person by hair style or color or because you took the photo and know the persons shown. This is not something an AI can do, or at least not in the next 10 years. But who knows.

I don't have the money Google and Meta and Microsoft can throw at AI. And most persons I know have problems uploading faces of their friends and family to an AI serviced for recognition (and what else). Here in Europe this is even illegal, unless you have the explicit permission of each person in the photo. For good reasons.

So, the IMatch face AI is as good as it gets. It works great, but not in 100% of all cases.
Linking persons to files is a quick way to integrate them with all the cool people features in IMatch - except face arrangement.
If you want to use face arrangement, you'll need to support the AI for the files where it cannot find a face or a matching person.

[Damit]

Here in Europe this is even illegal, unless you have the explicit permission of each person in the photo. For good reasons.

How civilized. Unfortunately, here in the US we have no protections of this sort.

Thanks for your explanation. It is sincerely appreciated. As always, you have me thinking.