Who uses annotions - annotations?

[sinus]

Hello,
I have read in the helpfile about annotiones.
Then I searched here in the forum and found 3 entries.

So I wonder who works with annotions and how.
Would be interesting to hear about the use (or not use) of this interesting feature.

[sinus]

Hello,
I have read in the helpfile about annotiones.
Then I searched here in the forum and found 3 entries.

So I wonder who works with annotions and how.
Would be interesting to hear about the use (or not use) of this interesting feature.

Well, if someone is that stupid (like me)  to write a word not correct, then 
Sorry, folks!

So, with the correct word "annotations" I can find a lot of informations in this forum.

Nevertheless, if someone want to write about hers/his use of annotations, you are welcome! 

[ubacher]

When I have a series of group shots I use annotations to mark the faces with closed eyes.
( Or other deficiencies).
Similarly I mark spots in images which need retouching/fixing.

When I first tried annotations I added a big R and L over stereo images - to make it clear
that the image is part of a stereo pair. Haven't done this retro-actively yet. I will be looking for a way of doing this automatically
based on a category I have set. --> Future work.

[sinus]

When I have a series of group shots I use annotations to mark the faces with closed eyes.
( Or other deficiencies).
Similarly I mark spots in images which need retouching/fixing.

When I first tried annotations I added a big R and L over stereo images - to make it clear
that the image is part of a stereo pair. Haven't done this retro-actively yet. I will be looking for a way of doing this automatically
based on a category I have set. --> Future work.

Ubacher, really interesting, never thought an such things. Thanks for sharing! 

[Mario]

Annotations are a feature most useful for corporate and scientific users. (Pro) Photographers usually don't have much use for them.

I have already the code which can create 'Face' annotations from Picasa face tags found in XMP. When I have some idle time I will write the code which stores IMatch 'Face' annotations also in XMP, and then I will include this as another feature for Annotations. This will make Annotations more appealing also for photographers and especially amateur photographers who often 'tag' people in photos.

I don't like this at all, because adding names to faces and then maybe even uploading the images gives Facebook and other companies and 'services' very valuable data. Facebook sells this data for a lot of money. But if you keep the files under your control, or strip the face data when exporting the files (e.g. in the IMatch Batch Processor), having automatic face tagging can be very useful indeed.

[sinus]

Thanks, Mario, for this interesting informations.
Yep, if used really only private, than such annotations are surely very good.

But we should be aware of some danger too, yes.

[lnh]

The Picasa connection for face tags sounds interesting. Several years ago I played with a couple thousand picture dataset and was amazed at the accuracy of their facial recognition technology. As an option would it be possible to just extract and create hierarchical keywords from the Picasa data rather than actually assign the tag area? Like many Google projects, you have to wonder if Picasa will end up in the trash bin at some point soon as it's been pretty static stuck at 3.9.x for about 3 years now.

I remember reading some metadata handling analysis done by Geoff Coupe  (e.g. http://gcoupe.wordpress.com/category/photography/) where it wasn't clear that Picasa always played nice with your metadata and might have even stripped maker notes in some cases. My memory is a bit hazy on the details, but remember being cautious about considering the use of Picasa as a "pre-processor" just to get tagging.

I always strip metadata before uploading to Facebook. Interesting that they data mine your metadata, but don't maintain it in the uploaded image (everything including copyright and/or CC license terms gets stripped away by Facebook). They also compress your image so much that invisible watermarks barely survive a round trip.

[jch2103]

The Picasa connection for face tags sounds interesting. ...  As an option would it be possible to just extract and create hierarchical keywords from the Picasa data rather than actually assign the tag area?
...
My memory is a bit hazy on the details, but remember being cautious about considering the use of Picasa as a "pre-processor" just to get tagging.

Yes, Google is pretty good at search functions including facial recognition. Picasa face recognition can store data in two different places: either in a Picasa.ini file in each folder, or (if you enable it) in XMP data in the image. If you choose the XMP route, Picasa uses a standard MWG XMP structure that includes face region (i.e., the boxes around the faces) and the names that you assign. You can convert Picasa from using one system to another, and there's also a third party solution to convert Picasa.ini data to XMP in-image data (Andreas Vogel's AVPicFaceXMPTagger - http://www.anvo-it.de/wiki/avpicfacexmptagger:main). See also http://davoweb.net/2014/01/face-tagging-and-lightroom/ which includes a short discussion of naming conventions.

I did some limited tests of using this with IMatch data-driven categories, and it works pretty well. I've attached some sample images.

However, I still haven't done this for most of my images. One extra note: if you're concerned about privacy, you can use the desktop version of Picasa, but don't sign in to your Google account...I don't think your faces get transmitted to Google then (though I can't prove it).

My memory on Picasa data glitches is also hazy, but I have avoided recommending it or using it for metadata entry for a long time (as a free solution for others to use), although its search functions are OK once metadata's securely in place. Perhaps it's more reliable now, but I'd still be hesitant. Users on this forum, of course, have vastly better tools available in IMatch.


[attachment deleted by admin]

[Roadrunner]

I have just downloaded Imatch5 for evaluation.

I'm a genealogist and I have scanned over 10.000 old photos and would like to use Imatch5 to keep track of them. Many are group photos and it would be fantastic to use face annotations and have the possibility to search all photos for eg "John Doe". I have done some test and I can see the face annotations in the collection view under faces. How do I link a face to a category like Relatives/Family/John Doe ?

I have to say that I don't like the face recognition facilities that are used on internet, Facebook etc. It can easily be used for wrong purposes and I always delete all metadata before publishing anything on Internet.   

/Ulf

[sinus]

Hi Ulf,
welcome on board

We have here at least two users, who work as (or like, I don't know) genealogists, I hope, they can give you some hints. Though, face recognition is a special thing, I think.

[Roadrunner]

Thank you!

I will add that I'm not looking for face recognition. Just manually link a face to a person in categories using annotation. In a group photo I will link every known person to a persons name so I can do a search for a specific person or persons in a family. In that case I understand that I need to have family as a category and person as child.

/Ulf

[Richard]

Hi Ulf,

Years ago, when a annotations were first mentioned for "The Next Generation of IMatch", I had wished that I could select the category for "John Doe", select a group photo in which he appeared, and mark his face. Then do the same for all the others in that group photo. Then when a relative with a copy of my database selected "John Doe", he would be indicated by the annotation, however that is not possible as far as I know. Unless I made a separate copy of the file for each person in the image.

[Ferdinand]

Years ago, when a annotations were first mentioned for "The Next Generation of IMatch", I had wished that I could select the category for "John Doe", select a group photo in which he appeared, and mark his face. Then do the same for all the others in that group photo. Then when a relative with a copy of my database selected "John Doe", he would be indicated by the annotation, however that is not possible as far as I know. Unless I made a separate copy of the file for each person in the image.

From the help file page on annotations (in the viewer):  "Region annotations can be used to tag faces in photos. This special type of annotation automatically synchronizes the tags you assign to the face with the hierarchical keywords in the database."

[Ferdinand]

Actually I replied to soon from only a quick read of the help file - there are also face annotations.  Won't these do what you want?

[Roadrunner]

First I made a Category like People|Relatives|Family|John Doe.
Then I did a face annotation in the viewer with the name John Doe in both Tag and Description.
After that John Doe appear in top level in category, but I can't figure out how to link it or move it to John Doe in the right hierarchy level. I'm newbie so there are tons of functions and I haven't found the right way to do it. I have read the help text about annotation but I struggle with the last part about synchronization with category.

/Ulf 

[Ferdinand]

I've only played with this a little, but it appears to me that you create keywords from annotations, rather than create annotations from keywords.

That is, you enter "tag" for a face annotation, and this creates a keyword.  It doesn't have to be at the top level - there is an option in Metadata2 preferences to create it further down the keyword tree.

As far as I can see you can't tell a face annotation to use an existing keyword. 

I was thinking about making a feature request to do this.  It would be more useful to manage these keywords outside of annotations and use them in annotations, than to manage them entirely inside annotations.

[Roadrunner]

Ok, I think I understand what I have done now.

How do I create annotation from an existing keyword?

[Ferdinand]

How do I create annotation from an existing keyword?

What I was trying to say is you can't.  Not that I can see. 

I think that the ability to do this would make a good feature request.  Unfortunately there's a long line of feature requests.

[Richard]

Actually I replied to soon from only a quick read of the help file - there are also face annotations.  Won't these do what you want?

As far as I can tell face annotations will only work for one face in a file. If you have a group of 18 persons and make 18 copies of that file and assign a different copy to each person's category, then face annotations would accomplish what I wished for but only as a workaround.

[Roadrunner]

Thank you for your kind help.

I understand that there are lots of requests and the future will tell.
I already like many parts of the powerful software, so I'm likely to buy a license anyway.

/Ulf

[Ferdinand]

As far as I can tell face annotations will only work for one face in a file. If you have a group of 18 persons and make 18 copies of that file and assign a different copy to each person's category, then face annotations would accomplish what I wished for but only as a workaround.

Well I added multiple face tags for multiple people on one image with no problems.  You just keep clicking the face annotation button on the annotations panel and after each push you move and resize the face annotation rectangle to fit that persons face and enter the tag text and then create another face annotation.  I had multiple keywords automatically created from multiple face annotations on one image.  I think that this is neat.

My issue is the one I highlighted above - I want to create annotations from keywords, not keywords from annotations.

[Richard]

You just keep clicking the face annotation button on the annotations panel and after each push you move and resize the face annotation rectangle to fit that persons face and enter the tag text and then create another face annotation.

All well and good but no gain at all over workarounds that I employed years ago. Using the IPTC Caption field I would enter each person's name in the order in which their right pupils appeared in the group image. If the order was hard to see, I would include a copy with red numbers, in the correct order, on each person.

Another workaround was to copy the image and crop each person's face and save that cropped image file to that person's category.

[Ferdinand]

What would constitute an advance then?  I'd have thought that this was an advance for your situation.

[Richard]

What I have always wanted was to be able to assign the same image file to all categories for people in that image, then select that file in John's category and annotate John. Open that same file for Mary and repeat. Same for all shown in that image. The goal would be that the annotation AND category assignment together would determine which annotation appears.

[Mario]

1. You can create a face annotation for every face in the image. Or let IMatch detect the faces automatically.
2. If you enter an already existing keyword as the tag, IMatch will automatically assign the file to the existing keyword under @Keywords.
3. You have have any number of face annotations. If an image shows Peter, Paul and Mary and you create three face annotations, the file will be added to the corresponding @Keywords and the keywords will be assigned as well. You can also see the image in the corresponding region collection.

[Richard]

I shall experiment with this in my Test database before I comment farther.

[Roadrunner]

Mario,

I have created a face annotation for every face in the image.
After that I enter an existing keyword as the tag, but IMatch created the "same" keyword but in the top level of hierarchy. Maybe I'm doing something wrong? Do I need to enter the hierarchy path in the tag and if so how to do that?

/Ulf

[jch2103]

After that I enter an existing keyword as the tag, but IMatch created the "same" keyword but in the top level of hierarchy. Maybe I'm doing something wrong? Do I need to enter the hierarchy path in the tag and if so how to do that?

From the Help:

Tags as Keywords
Tags can optionally be synchronized with the hierarchical keywords stored in the database. You can enable or disable this functionality under Edit > Preferences > Metadata 2.

But it seems you're limited to one base like 'people' or 'WHO|people'. That would seem to rule out a hierarchical structure like People|Relatives and People|Friends.

As the Help says, this is still a work in progress.

[jch2103]

At the risk of going off-topic, is there any way to connect face annotations with MWG face regions (or vice versa)? Or is that something that may occur in the future?

[Mario]

At the risk of going off-topic, is there any way to connect face annotations with MWG face regions (or vice versa)? Or is that something that may occur in the future?

I have written about this occasionally. I have code which converts MWG region annotation of type face/pet into IMatch face annotations. I don't have the code which does the opposite yet. To many other things on my list and I lose 60% of my time trying to get all the metadata special cases done and handled users come up with. I should have enforced MWG from the start. Millions of Lightroom users seem to be able to cope with that...

[Mario]

But it seems you're limited to one base like 'people' or 'WHO|people'. That would seem to rule out a hierarchical structure like People|Relatives and People|Friends.

I'm not sure that I understand the problem.

You can enter one tag for a face annotation (or for any annotation).
IMatch optionally uses this tag for face annotations to assign hierarchical keyword(s) to the file.

For example, I have an image of Velvet and her Grandpa.

I let IMatch detect the faces and then set the tag WHO|Velvet for the face annotation for Velvet and WHO|Grandpa to Grandpa's face annotation. This produces the two hierarchical keywords:

@Keywords
  |-WHO
    |- Grandpa
    |- Velvet


The two keywords are added to the file, and the file is assigned to the two corresponding categories under @Keywords.
Of course I could also use tags like WHO|Persons|Family|Velvet as the face annotation tag, if I would use a deeper hierarchy for keywords.
It's actually quite simple and elegant.

If you want to assign other keywords to the file, do it using the Keyword Panel as usual.

IMatch also automatically assigns the image to the Collections Annotations|Region|Face|WHO|Grandpa and Annotations|Region|Face|WHO|Velvet because that's how IMatch handles all annotation objects (with or without tags).

[Ferdinand]

After that I enter an existing keyword as the tag, but IMatch created the "same" keyword but in the top level of hierarchy. Maybe I'm doing something wrong? Do I need to enter the hierarchy path in the tag and if so how to do that?

But it seems you're limited to one base like 'people' or 'WHO|people'. That would seem to rule out a hierarchical structure like People|Relatives and People|Friends.

I agree.  Note new feature request:
https://www.photools.com/community/index.php?topic=3344.0

The more support it gets the better chance we have of seeing it sooner.

[Roadrunner]

Yes! 

Mario,
your explanation of using hierarchical keywords in the tag did the trick.

Now I have used face annotation on several group photos with up to 10 peoples and it works fine for me.

Thank you/Ulf
 

[marco88]

I wish it would do face recognition ala Picasa!

Tried it this summer. Very impressive for a free software.

Marc.

[marco88]

Maybe FaceRecognizer is a solution?

http://docs.opencv.org/trunk/modules/contrib/doc/facerec/

Cheers
Marc.

[jch2103]

There are some earlier discussions on the forum about face recognition and some of the difficult issues associated with it, including Mario's experience with OpenCV: https://www.photools.com/community/index.php?topic=1373.msg8251#msg8251

Here's another link from earlier this year: http://davoweb.net/2014/01/face-tagging-and-lightroom/

[lnh]

I've played around with the people annotation feature a little bit. I can't figure out an easy way of assigning tags to people short of typing it in each time. Since I keep my people keywords in a 3 level hierarchy, typing all this stuff is a lot of work, and I'll probably end up using the feature for just special cases like identifying relatives in a picture from an era prior to my birth where the knowledge could get lost as people pass on.

You can also just type names in the tag field and they will end up at the root level of your categories. From there you can move them into your people hierarchies, but the tag field in the annotations which also controls what text gets placed in the box doesn't update. Nevertheless, the fact that the tag gets mapped to your existing category structure is nice.

Maybe in some future version of IMatch the picking of names will get streamlined via a connection with the thesaurus or some other way.

[Mario]

You can also just type names in the tag field and they will end up at the root level of your categories.

We have discussed this in another thread just the other day. You can use hierarchical keywords for the face annotation tag, e.g. WHO|Family|Paul. Moving / changing a keyword has no effect on the tags you assign to face or other annotations.

[marco88]

Tks jch2103 maybe OpenCV will become more useable in the future a new open source technology will come out.

Google Picasa does really a good job. It works well enough and is really simple to use.

Just what I need to clear up my 26.0000 odd files and file them in my people categories.


Anyway currently using the recent panel to (re-)assign categories. It works fine.

Marc.

[Mario]

The Face detection in IMatch (Viewer Annotations) is based on OpenCV.
If you click on the smiley, IMatch can detect faces in standard situations automatically, without the need for you to manually position the face annotations.

But face recognition is a totally different thing. Google and Facebook have spent millions and millions of dollars to build face recognition technology and their face databases. Because they can make a lot of money from selling this data to corporations, marketing companies, governments and others. Just image how this data, combined with traffic cameras or the cameras mounted in every store or mall these days, can be used...

What could not be achieved by algorithms was then socialized, by making "tag your friends" 'cool'. Millions of Facebook and Picasa users helped these corporations to improve the overall quality of their face databases - which yields more money when selling the data in the end.

There is no comparable technology available on the market. At least nothing I could afford to license. There were a few companies who offered on-line services for face recognition. I could have used that, maybe only a s a fee-based add-on for IMatch (to cover the license fees). But most of these companies either went bust of have been bought by Facebook or Google to get their technology. Besides, using such a service in IMatch would be a privacy nightmare, and illegal (for good reasons) in many countries - without an airtight waiver from each IMatch user. And I would have to hire a law firm which can create and adapt such a waiver for each country IMatch is sold in. A nightmare.

So, in short. I don't like what Google/Facebook are doing with face data.
There is only a small chance that (unless somebody knows something I could use without spending months of development) that IMatch will have face recognition comparable to what Facebook/Google have.

I plan to link XMP region data (faces, pets) with the Face annotation feature in IMatch. I already have the code which imports face regions as face annotations into IMatch. But not the code which stores face annotations in XMP etc. This is one of my pet projects, and there is plenty of other work to do...

[marco88]

Personally I would not want to move the information to image files or on the web at all (privacy issues indeed) but instead keep it all in Imatch dbms only for querying example would be (I use categories extensively for that):

give me all the photos with person X when age 8 to 10
or
give me all the photos with X in it which were taken between X and Y date?
or
give all the photos with both person X and Y in it
or better
give all the photos with both person X and Y in it but nobody else

It is the sort of thing I try to do when printing or emailing photos to my family.


If openCV allows in the future to recognize face as well as detect them I think it would be for me a great addition to IMatch and make these type of queries more possible.

Note one thing that Picasa did not appear to have is age related querying i.e. child face change a lot in the early year so what do you do when you want baby photo of child X as oppose to photo when 10 year old?

Just some ramblings and offering ideas for futures.

Cheers.
Marc.

[Richard]

IMO, face recognition software could lead to problems. I have photos of siblings which others claimed were of me. I also had a mother misidentify which of her daughters was in a photo. Children's faces do change a lot and at one age child A may look like child B at a different age. The mother only caught her error when she looked at other details in the photo.

[Mario]

It is the sort of thing I try to do when printing or emailing photos to my family.

All this can be easily done with IMatch, except for an automatic detection of faces. When I implement the feature, IMatch will be able to create face annotations from XMP regions.

I can already foresee that people will demand a lot of fancy options for how to map the name assigned to a region to keywords, or categories, or maybe both in some way. Using the thesaurus. Or not. Looking at different spots in their keyword hierarchy. Maybe even map the same face into different sections of their keyword hierarchies. Implementing all this will take much longer (and cause more bug reports, long discussions here in the community, feature requests etc.)  than the simple task of creating face annotations from XMP regions, or XMP regions from IMatch face annotations.

And this is the reason why I do not work on this right now. I fear that the simple region->face anno->region idea will generate tons of work once users use it. And we have 200+ feature requests already.

[marco88]

Yes a bit of minefield I agree.

Cheers,
Marc.

[lnh]

Found a way of making face annotations a little more convenient. I store people in a structure like: PEOPLE|RELATIVES|Joe. Typing "PEOPLE|RELATIVES|" every time I have someone to face tag in this keyword hierarchy makes for a lot of typing. The best solution I've come up with is to create a User Annotation which contains the common upper levels of the hierarchy. So in this case, the User created annotation would be just "PEOPLE|RELATIVES|" and then I just have to type in the name.

Not as nice as having access to the thesaurus, or a keyword panel, but an improvement in lessening the typing load.