Face Detection Improvement

[marco88]

Hi Mario,

I played with the face detection a little bit in the viewer but it is too cumbersome for me as I want to detect and classify 100s of images with faces in it so I propose a list of improvement to the current features available:

1) make face detection available through the search menu

i.e. have a search for images in a set with faces appearing in them

I could then look for images listed in the resultset that are not categorized yet and conclude I need to assign a person name to it (i.e. a category under my people branch).

2) make face detection available through scripting

i.e. have a function that returns images with faces in it  from a set of images.

I looked into the scripting doc and I found the find similar function. If you could make the face detection feature available too in scripting I could write a script to attempt to auto categorize.

However I would need to be able to compare images too i.e. see if images that both have faces in it and taking closely in time are visually very similar then I could conclude this is a photo of the same person and assign the same category name to it.

These hopefully are small and realistic improvements to the current feature set and might be useful to others.

Hopefully in the long term if we could have face recognition with auto categorization that would be the ideal world!

Kind regards
Marc.

[Mario]

The idea of adding the face detection to the Viewer was to make it interactive, allowing the user to quickly spot false positives, too many detected faces, wrong face detections etc. and to correct them immediately.

A Command like "Detect faces in all selected images and create face annotations from the results" would be doable, but you would have to check each file manually afterwards to delete false positives or other failures. The same would be true for a scripting method. You cannot be sure that all faces have been detected or that a file has some false positives without a manual check..!

[marco88]

Thanks Mario,

I suppose it will depends how well the face detection work. Probably if it gets 80% right that is ok. If it gets 50% or below maybe not.

I already categorized manually a large number images with faces so in theory I could pass that as a training set to the face detection functions.

Could you expose in your scripts some of the API from the image library you are using to train or tune the face detection function a little?

Cheers
Marc.

[marco88]

By the way personally I would not need the function to use annotation as I don't use them (yet).

I use only category right now:

people
--me
-->kids
------> kids 1
...

So for me if the function just returns the images where it thinks there is a face (or more) that is good enough.

I can then go set the right category (me or kid 1 or kid 2) after when reviewing the result set.


But maybe other people used annotations already and have ideas on that?

Cheers
Marc

[Mario]

The face detection implemented in IMatch is linked to the annotation feature. IMatch also detects existing XMP face regions in your files and produces face annotations.

Detection rate depends on your images, face orientation etc.

Did you consider just using Google Picasa or one of the other paid face detection application which produce proper XMP regions? IMatch will use these results automatically.

There are also web services which provide face detection for a fee between 50 and 500 dollars per month e.g. https://www.mashape.com/animetrics/animetrics-face-recognition/pricing

[marco88]

Yes I checked all these out quickly this afternoon. I use IMatch for storing my personal photos family pics travel ... Not keen to pay.

Google Picasa face recognition works great. Tried it a while ago at my parents. That is probably my best bet. Only problem is now I have 2 tools.

Cheers
Marc.

[Mario]

We discussed this here on many occasions.
There is no affordable or free library I could use to improve face detection or recognition. Only the multi-billion dollar companies currently have the technology, and Google and Facebook bought most of the companies who offered face detection technology for cheap.

The few available open source toolkits have detection rates way below what Picasa does, and I cannot use the code in IMatch because of license restrictions anyway.

Companies like Facebook and Google pay for the technology and offer them for free because they want your data. Face data combined with what Google / FB know about you can be sold for lots of money. I don't sell your data.


The only tools that would probably work for IMatch 5 are web-based paid services like the one I linked to above. But when I integrate their system into IMatch, this means two things:

1. It only works when the IMatch user is on-line

2. IMatch needs to upload all photos to the web site of the vendor to perform the detection. Which is a privacy nightmare and legally challenging in many countries. I don't have a law department so I don't even consider this.

3. I would need to pay several hundred dollars per month from my own pocket for the service. For 300 US$ I get 5000 face recognitions per month, so when IMatch users have used up that amount, we all have to wait for another month. I don't think I will go that way.

As it seems, Google will no longer develop or support Picasa. They've lost interest because they cannot get the data out of Picasa unless you explicitly upload the images to Google photo books or whatever it is called now. Their new product 'Google Photos' only works when you are online, and only performs face detection when you upload the files to Google servers. That way Google gets your data and can sell it and use it for whatever purpose they like. <Shudder>

I don't know yet where Adobe bought their face recognition technology from, but I'd wager that I cannot afford to buy the same....

[marco88]

No problems I ran Google Picasa on my hard disk it found over 30,000 photos.

It took 1.5 days to scan this and identified faces. I reviewed and sorted some. It looks like it worked well just too just took a long time to scan all this. Then I will try to export as per your doc says to XMP and reload to IMatch see what happens.

I am pretty excited to have my 10+ years of images sorted by people faces. It would be saving me months of work. No way I could have manage that on my spare time (manually).

Cheers,
Marc.

[Mario]

See the related article Using Google Picasa™ Face Tags in IMatch in the IMatch knowledge base for helpful information. Make backup copies of all your files before letting Picasa update them. Just in case Picasa damages a file when writing the XMP data.