facial recognition and location metadata importing

[GKent]

Please move this topic wherever is appropriate.  I wasn't sure where to put it.  Please also excuse my ignorance about the matters on which I write and would like advice.

+++++++++++++++

As I understand it, iMatch does not currently have facial recognition technology built-in, but it can import that metadata.  So, my question is, what PC software, or online service, does a good job of creating facial recognition metadata that can be easily imported into iMatch?  I have the same question in regard to location metadata.

I've experimented just a little with Google Photos.  I like the quick way it does the facial recognition search of my pictures.  But as far as I can tell, Google Photos does not have any means for you to export that facial recognition metadata.  Maybe it will in the future.

Thanks in advance for the help.

Greg

[Mario]

1. Yes, IMatch imports existing XMP face region data automatically. But you can also create it in the Viewer. Just click on the smiley icon.
IMatch has no face recognition, but it has face detection. It has also a very quick way to add tags (names aka keywords) to faces in the Viewer. Usually this is as fast as letting some software do it automatically and then go through every picture and check for problems. The systems are not 100% accurate and you need to check.

2. IMatch automatically imports GPS data contained in your files. It can do manual or automatic reverse geo-coding (mapping GPS coordinates to country, city and location). If your files have no GPS coordinates yet, you can add then in the IMatch Map Panel.

See the help for the Viewer and the Map Panel for all details.


Google Picasa had face recognition and could save this data in XMP. IMatch automatically reads the face data on import.

But Google Picasa has been discontinued in favor of Google Photos. Google has no interest anymore in Picasa because it cannot get at your data when you use Picasa on your PC.
The only reason for Google to provide Google Photo for free is to get at your data. Get your face data and the faces of other people in your images. This way they can learn about you, know how you look, who you know, where you take your photos and all that.

Always keep that in mind when you use some "free" software, especially from companies like Google. You pay with your privacy and the privacy of the other people on your photos. In Germany it is even illegal to upload photos to web sites which do face recognition, unless you have the explicit consent of each person on the image. And this for very good reasons. Uploading faces of unknowing people and thus allowing Google and it's clients to identify these people everywhere is not good.

Since the big companies give away the software for free and make it "social" and "cool" to do this, billions of people feed the Google and Facebook A.I.s for free. Whether or not the other people on the photos want that.

Yes. I'm concerned about privacy  . Like most IT people.

Adobe Lr has built-in face recognition which works OK for frontal faces. They (at leas know and as far as I know) off-line, without uploading your images into the Adobe cloud. Lr also writes XMP face regions IMatch will read and convert into face annotations automatically.

See also this know-how article in the IMatch DAM knowledge base.

[GKent]

Thanks, Mario.  So, it sounds like it would be just as fast to use IMatch face detection and tagging as to import the face recognition data from Adobe Lr.  Thanks for the caution about Google Photos.  -- Greg

[Mario]

Yeah, give it a try. It's quite a smooth process. You have full access to your thesaurus and thus can just pick people from a list.

Take a minute and remember the keyboard shortcuts for adding tags and switching between faces and you're done in no time. And of course "you" do it always right, even for faces not looking straight at the camera. Not like some more or less intelligent algorithm 

[medgeek]

The only reason for Google to provide Google Photo for free is to get at your data. Get your face data and the faces of other people in your images. This way they can learn about you, know how you look, who you know, where you take your photos and all that.

Always keep that in mind when you use some "free" software, especially from companies like Google. You pay with your privacy and the privacy of the other people on your photos.

Mario is right, of course. Privacy should be a concern for everyone.  It's good to know Germany takes privacy more seriously than we do on the other side of the pond.  An easy way to summarize it:  if you are getting something for free, then you are the product (whose private information is sold to advertisers, etc.).

[Mario]

Mario is right, of course. Privacy should be a concern for everyone.  It's good to know Germany takes privacy more seriously than we do on the other side of the pond.  An easy way to summarize it:  if you are getting something for free, then you are the product (whose private information is sold to advertisers, etc.).

That is one of the reasons why IMatch allows you to choose between GeoNames.org and Google for reverse geo-coding or why you can choose between OpenStreetMap, Googe Maps and Bing Maps in the Map Panel. Or why there are no Google ads in this community. Or why I don't use active "Like" buttons on my web pages, just regular links. This way FB cannot track you unless you explicitly click on the link.

[mastodon]

I use the last version of Picasa for face recognition and it is quiet useful for autoimatic tagging (you have to play a little bit with the "suggestion and cluster threshold" options of the face recognition). Don't forget to save data to jpgs, that is not automatic.
And you can try digikam, that is a free and developing software, BUT it might have painful bugs. Does anybode have experiences with Digikam?

[Mario]

Last time I looked the face recognition was not really that good. I could do it much faster manually and 100% correct in the IMatch Viewer 

[tspear]

Mario,

In some previous threads I read you have utilized some opencv.org functionality.
Have you kept up with http://cmusatyalab.github.io/openface/ which is based on opencv for facial recognition?

Tim

[Mario]

I know about OpenFace. But the results were not really good the last time I've checked. It also need a Python runtime environment and suchlike. Doable, but..
I get excellent results from my initial tests with some of the major face recognition web services. That's rather the route I would go.
And face recognition is only a part of the task. Recognizing builds, landscapes, objects, moods etc. to make keyword entry less of a nuisance is as important.

[mastodon]

Can we use that face recognition web services now? Can we use it without IMatch support, integration?

[Mario]

Can we use that face recognition web services now?

No. I'm still working on that. That's a big feature because it requires tight integration with features like the Thesaurus, @Keywords etc. to be really useful.

Can we use it without IMatch support, integration?

No, why?
You can always buy or lease computing time from one of the specialized vendors and upload your files for face recognition.
The typical cost is here about 1.5 - 2.0 US$ per 1000 face recognition operations, plus monthly storage cost, depending on the number of images you want to use for recognition.

Or you use LightRoom CC, where Adobe offers face recognition as part of the monthly subscription, I believe.
Canto has powerful face integration in their Enterprise-level DAM now, which starts already at several hundred US$ per user per month.

[Arthur]

LR6 standalone has this also, but this is not reliable, if the face is directed partially to the side or turned away. You must inspect each file anyway to ensure, that the person keywords are set correctly. If you have to look into each file anyway, you can set the keywords by hand by using multi selection of photos with the same person in files window and assigning a keyword once. The functional advantage of having face annotations/regions in addition to keywords I did never understand...

[Mario]

LR6 standalone has this also, but this is not reliable, if the face is directed partially to the side or turned away.

This is normal. I don't know which technology Adobe includes in LR6 but most face recognition systems have problems with identifying faces not directly looking at the camera (full frontal).

One has to check each image manually to add missing faces or correct wrongly tagged faces.
Given the tools provided by the IMatch Viewer for face face annotations I think you are almost as fast with doing it manually right 100% the first time than using an A.I.-based approach and manual review.

And if you only need the keywords but not the face regions, you are super-fast with IMatch and adding keywords anyway.

On the other hand, the A.I. algorithms get better all the time. But the massive computing power to run the A.I. for proper face recognition is not (usually) available on normal desktop PCs or notebooks or mobile devices. The A.I.s run on specialized hardware (often: clusters of graphic cards) or specialized chips. The same is true for the massively huge networks required for object detection, text recognition etc.
It's much more efficient and cheaper to rent the computing power required for all this for as long as it is required in the cloud.

There are some vendors which offer face recognition which can be included in software like IMatch. But they are very expensive (I would have to pay 10,000 US$+ per year!) and, frankly, they are not even close to what some of the cloud-based systems offer. And that's just faces...

There is also object detection, sentiment analysis, gender, age and more.
Some systems can even formulate sentences like "This image shows two doves in front of a blue sky ", "group of people celebrating" or "image of the tower bridge in London".

Now, guess how much time we all could save when IMatch could not only assign keywords to files automatically but also make suggestions for the he the headline and caption 
This would most likely cause some additional cost (for the company providing the cloud service). But I guess that for many users spending a couple of $ to get a few thousand images tagged, classified and provided with headline/caption automatically would be acceptable.

This would be strictly optional of course. If you don't use it, you don't pay for it.