Face annotation suggestions

[hluxem]

I started testing face annotations for the "Who" categorization. I'm actually surprised how well the face recognition in LR works and I believe I'm going to be faster categorizing pictures using face annotations rather than just using categories.  While starting to work with face annotation in Imatch and LR I wrote done some things which I think may make it easier to work with annotations in Imatch in general. Some of them are more specific to tagging people. Sometimes it's hard to describe, but because it's mostly people related it did not like to add snapshots with pictures and names to the post. Please let me know if a picture would help for some points. For my purposes it's fine to have the face detection done in LR, but there are still many cases where I just like to do the tagging manually in Imatch.
This is my wish list, I have no idea what it means to implement and that's up to Mario anyhow. I don't expect that everything can and will be implemented and hope that other users have some comments and suggestions as well.

1.   Display annotation
Add an option to display the annotation outside the face box and an option for the font used.
I use a very big 4K monitor (40") and the font is in general much bigger than needed.

2.   Change the way an annotation is manually placed and named
Instead of placing the annotation in the middle, use click and drag like in LR. First select the standard or user symbol, then left click to position the left corner of the annotation, drag the mouse to size the annotation. This should be easier for tagging faces missed by the face detection as well as to place any other symbols on an image. Eliminate the need to hit F2 by either highlighting the text field after placing the box or allowing to just click and edit.

3.   Selection dialog
Currently the selection dialog box opens always on the same position, it's easier if it opens right in place where the annotation is. When starting to type text, the names underneath get filtered, but a mouse click is required to select a name from the list. I like the first entry of the filtered list to be highlighted and selected if I hit enter or leave the dialog.

4.   Copy and paste annotation to the same layer as in source image
I like the ability to copy and paste annotations, currently when I paste a face annotation from the layer Persons on an image, its copied to the default layer on another image.

5.   Rename annotation
I like to be able to rename an annotation just like a category (I actually would like to see them in the category tree), currently I can only edit a single annotation. But if I misspelled a name multiple times I have to go to all pictures with this person. I don't use keywords, but I believe when I change the keyword, the face annotation is not changed.

6.   Creating user icons.
When I add a face annotation to the gallery, an icon is placed in the user tab. Not sure how this symbol is created, it looks like just some portion of a letter, there is no way to recognize the different icons created. It would be very helpful if this symbol could be the initials or the first 2 letters in the annotation. Somebody could even get the idea to have the face displayed :>)

The following features are more specific to face tagging and dealing with people.

7.   Show all pictures of one person
You can do that now, but you get the whole picture. I think it is great to have thumbnails displaying the content of the box (If possible I would actually like to be able to adjust how much more or less of the box is displayed).

8.   Show all persons
Again, you can theoretically do that today, but you get the full picture and you get all pictures of a person. Only one picture of the face for each person is what I'm looking for.

9.   Group annotations like family, college, work, relatives ...
It would be nice to be able to group the annotations and create a hierarchy. This works currently if I would include the group names in the annotation name.  If I name people like "...... Family -  First Last" I can create data driven categories with a structure, but then the group name is always included in the person's name. I thought of other ways to make this work, but can't find a solution which works for multiple persons on an image. Maybe the field description in the basic box under the tag name can be used? Could attributes be available for annotations, that would allow to even add more information? For me a manual grouping would be fine too.

10.   Sort option
I usually use the format first name last name for the face annotations. Would be nice if I could sort the annotation for the last name.

I hope that I didn't miss some functionality already available in Imatch, I did read the help and tried to check everything out, but sometimes you just don't see it.

Heiner

[Mario]

Thanks for taking the time to think about this feature and to come up with suggestions. Always appreciated.

I have a lot of changes done to face annotations for the upcoming IMatch release. To make them easier and faster to use. Improving the workflow, etc.

I think that most of your suggestions have already been implemented or are now void,
I suggest you wait for the next release, then review and update this FR as needed.

1. done

2. Not easily doable. Annotations work in a specific way, and face annotations are just a sub-type.

3. No longer valid.

4. Fixed already.

5. Annotations are stored per file. They are not linked to any category or keyword. IMatch allows you create keywords from face annotations to add extra value, and because LR still has problems with face regions which are not also in the keywords. But when you change a keyword, IMatch does not go through all files in your database, loads the annotations, checks if there are face annotations with the same tag as the keywords you just have renamed and then changes the tag in the annotation. Consider a database with 100,000 or 300,000 files...how long would that take?

6.

7. On my to-do list, but not much demand.
A first step would be to use a selected face as the thumbnail for the corresponding collection.

8. See 7.

9. Annotations are automatically grouped in the collection view. You cannot add additional groups for annotations. I doubt very much that such a feature would be of interest for many users. Annotations are mostly used in corporate and scientific environments.

What you request here sounds more like you would to have features already available, in more flexible ways, in IMatch categories. You can use the keywords associated with your face annotations and group them in categories in any way you can imagine. Either via formulas or Alias categories. Works super.

10. Sorting where? In the Collection View? @Keywords Category?

[hluxem]

Code Select Expand

Thanks for taking the time to think about this feature and to come up with suggestions. Always appreciated.

Sounds like it took me longer to write it down than for you to implement new features
I don't know how you manage all of this while you are also working on Imatch Anywhere. Thank you for all the hard work. I'm definitely looking forward to the next release.

Annotations are mostly used in corporate and scientific environments.

I maybe wrong, but I think this is going to change. Just as with GPS data, I think at some point face annotation will be part of most images, at least the one taken with a phone :>), that's part of the information Google, Facebook and all the others are looking for. Although I don't like that, you don't have to upload these data to the cloud and with Imatch you can strip out the data easily. I do like to use the available tools to make my categorization easier, why assign a category "Who" manually when I can have the tagging done mostly automatic and even get the option to display the persons name in the picture.

You can use the keywords associated with your face annotations and group them in categories in any way you can imagine. Either via formulas or Alias categories. Works super.

2 Problems with using keywords, first of all, I don't use them, second they are not really associated to the annotation as you wrote in 5. That's why I don't use keywords. The data is already in the file and I don't want to duplicate it and then keep all redundant data synchronized. There is enough mess in the meta data and I fell I create more redundancy and possible pitfalls by adding keywords. I did try to use data driven categories, but could not achieve what I want. I was only able to duplicate the collections into a category list. I have never worked with alias categories, I will look into that feature next.

[Mario]

Some well-known tools out there expect that the name of a face region is also a keyword. And many agencies demand it. You have the option to disable this in IMatch, but I would not. If you change the annotation, the keyword changes as well. And a few bytes more in the file don't matter.
Having the annotation tags also in keywords opens up access to all keyword and @Keywords category related features.

Adding the same features again, this time based on a specific kind of XMP region ("face") or multiple region types would be a lot of effort and I don't see any benefit.

It would be easier to add a feature which allows something "Change the tag in all face annotations from "Peter" to "Paul", and matching keyword updates). I can do this. I can also extract the faces from images and present them in a new view (similar to categories, just with all face images so you can pick all images of a person by clicking on the thumbnail.

All this is doable. It depends on how many users request this, and of course there are many other feature requests as well.

And I always have to consider that this needs to work also for databases with 300,000 or 400,000 files - and I'm not sure that the other software which presents features like this (Picasa, Lr) can handle that. But IMatch would need to, and this makes things usually harder to implement.

Let's see how many Likes you can get for this.

[mastodon]

This is my whislist too!
And: 2.   Change the way an annotation is manually placed and named
Yes, that would be nice. I like the way of drawing the rectangle in Picasa.

And in the Viewer window it would be nice to have a forward and a back arrow in the picture (on the left and the right side), so that it is not necessery to click on the thumbnails on the ribbon. That would make the tagging faster.

[Mario]

And in the Viewer window it would be nice to have a forward and a back arrow in the picture (on the left and the right side),

Why not use the mouse wheel or one of the many keyboard shortcuts for navigation? Much faster than aiming the mouse at some sort of arrow on the sides of the image - especially if you have a larger monitor...

And you don't need to create each face annotation manually. If the automatic detection fails for your files, you can just copy/paste face annotations and then quickly move them to the right spot!

The next version of IMatch has a different resizing method for face annotations which makes this even faster. And a command to automatically detect faces in selected files. And an in-line editor for face tags, with 'recent' list and pick-up lists based on the thesaurus etc. Very fast to work with, especially when used in combination with the automatic face detection and the copy/paste feature. A dozen files per minute or more.

[mastodon]

Thanks, I try the options you have mentioned, sounds great and I am wating for the new release. 

[Mario]

If you had time looking at the changes made to face annotations in the 5.7.2 release, maybe you can review your original feature request post.

[hluxem]

Sorry for posting so late, unfortunately I have been traveling and not been able to finish my tests.
I do love the changes you made with the last version! I did not expect all this changes that fast.
I hope the bug with the display of the layers can be fixed easily.
There are still features I like to see in future version, but I'm really happy with the way things work now and I hope more users are discovering the face annotation features.

Here are my comments to each point:

1)   Very nice, big improvement!

2)   a: Renaming works now much faster, great implementation.
        b: No change in placing the annotation. I think that could be a useful change for annotations and notes in some future version.

3)   Better than I anticipated!

4)   Not sure about that, the bug with the layer display not updating makes it hard be sure what layer the annotation is on. In fact, I have only used the annotations on my test database because I don't want to get the annotations on the wrong layer and then have to fix them later on.

5)   No change here.
I was going to try and modify an existing script to see if I get that done. My idea is to select a set of images, input the name I want to change, read the region name tag and replace the string. That maybe beyond my skills, but I will try and if I have questions post them in the scripting section.

6)   No change here.
I was wondering if Imatch saves the icons in a folder and I could manually replace them. That would help me a lot!

7) & 8 ) No change, both points are related.
This came up from using LR and seeing pictures of my kids from baby to teenagers. At that point I thought it is just a nice feature. Face tagging about 3000 old family pictures, I learned that for fast face tagging, it's not really that important to recognize the right person. LR did that only for about 20% of the pictures, the rest remains unassigned. After batch detecting a certain number of files, LR shows all unassigned faces and then you can select and assign all faces from one person. In my opinion, rather than spending money on a different face recognition software which identifies the person, implementing a solution where you can show all faces of a selection of pictures and allow multi select and assign names to a group of faces would be a better step for the future. For me being able to display the faces of a group of pictures seems to be something which could be the base of making face tagging easier, allows to show all persons in a database and shows you how people change over time.
8 )   See 7

9)   No change, but I found a work around.
Just for the record, I don't use keywords for annotations because I hate redundant information and having annotations and keywords only synched in one directions calls for inconsistent meta data in the files.
I created a data driven category based on the annotations. This gives me all persons as a category. Then I create a category with the hierarchy structure and manually create reference categories to the persons under the right hierarchy. As a next step I was going to test if I can export the data driven annotation categories to Excel, define the hierarchy in Excel and then import the hierarchy categories with the reference categories.

10)   No change here.
Would be nice to have, but I can live without it:>). Maybe that's something I can do with the reference categories as well.

So again, thanks for implementing all the new features. Short term, I would appreciate if the layer issue could be resolved and if I could manually change the icons for user created annotations.

Heiner

[Mario]

The bug has already been fixed. see the "Solved for next version".

2b. Face Annotations are now created and modified entirely differently than before.

4. IMatch now asks you if you paste into a different layer

5. You can always assign another name. But that will not change other face annotations / keywords. If this is what you expect. This would require to add a special feature which remembers all faces in your database and when you change the "face" Susan to John all other Susans will also change. Quite a massive enhancement, nothing that I can throw in in a regular update. Something for a major version.

6. That's a rendition of the Annotation. The icons used here are dynamic and based on vector graphics. If you plan to have the "face" as the annotation, this would be a separate feature request and linked to 5. Major version.

7-9: See 5.

10. See 5.

So far your requests have two Likes. Implementing 5-10 would require several weeks of development. It sounds like you would like to re-create a mix of Picasa and some Lr features in IMatch. Lets see if you can get another set of Likes for all this. It makes no sense to implement all this for a handful of people.

[hluxem]

The bug has already been fixed. see the "Solved for next version".

That's great! Didn't see that the thread was moved to solved.

6. That's a rendition of the Annotation. The icons used here are dynamic and based on vector graphics. If you plan to have the "face" as the annotation, this would be a separate feature request and linked to 5. Major version.

I attached a picture showing the user tab for user created annotations. I included some other items like a call out and a text box. All the green dots and the green corner are for face annotations, they are just not distinguishable and I don't see a way to work with them as it is now. Even displaying a numbering from 1 to n would help. Each Icon added to the user gallery needs to be named, displaying the first two letters of the name would be perfect. While displaying a face would be nice, that's certainly would be overkill and a waste of your time.

like to re-create a mix of Picasa and some Lr features in IMatch

Neither Picasa nor LR can be used as a proper DAM product. They do have some nice features in the area of managing the WHO portion of a database. Looks like not many people are using face annotations at the moment. It is a fairly new feature. That's surprising to me, because I assume most users do have a "WHO" branch in their database. With the current feature set it still takes me a little longer to categorize  new pictures with face annotations instead of just categories in Imatch. But I get the additional benefit of having the persons pointed out in the pictures. I started out with face tagging all pictures in LR, currently I only use LR as an intermediate step if there are many persons in the pictures. Then I can go through all the faces and multi select the same ones. Makes face tagging for hundreds of pictures really fast.

[Mario]

The Gallery feature was not designed to hold annotations for every face. It has been designed to host customized annotation objects, like custom arrows and such. A face annotation is just a green rectangle, and hence they all look the same. If IMatch would manage a face database, which it doesn't at this time, you would not use this feature anyway.

That's surprising to me, because I assume most users do have a "WHO" branch in their database.

Not really. If you are only concerned about the person in the image (and not the position of the face in the image), quickly assigning the associated keyword from your WHO hierarchy is super-fast. I really doubt that many users need the exact location of the face in the image for anything. That's rather a by-product of the face recognition algorithms the other software use. Most people are just interested to find all images with aunt Paula in them.

[jch2103]

I really doubt that many users need the exact location of the face in the image for anything. That's rather a by-product of the face recognition algorithms the other software use. Most people are just interested to find all images with aunt Paula in them.

Probably true that most users don't care about the exact location of the face in the image, but I know that it can be important for documenting things like historical/family group photos (i.e., Aunt Paula is in the back row, third from the left, Uncle Max is..., etc.). Probably not enough IMatch users to justify doing this, but it is useful for some purposes (perhaps with other tools).

[mastodon]

Face tagging is much improved in IMatch 5.7, and now I have implemented it in my workflow (I used to work with keywords before). I have been doing face annotations with Picasa, and then import the pictures in to IMatch. But IMatch still lacks some goodies, as hluxem said. BUT here I always have to emphasize that Mario reads all our minor complains and take into consideration so we have the possibility to influence the development of IMatch. Thank you, Mario.
I think home users like me have photos of relatives, colleagues and friends and they know most of them, let's say 80-90% but the remaining 10-20% they do not know, especially in photos of a group of people. In these cases the face annotation is a great help to recognise everybody. That's why I do face tagging and the keywords are not enough. AND then there is/will be IMatch Anywhere used by people who do not know the people in the pictures, so it would be a great feature to have tags showing the names.
The 7. Shows all pictures of one person, 8. Show all persons (only the face), 9. Group annotations like family, college, work, relatives feature and an application like Piacasa's "People manager" to change a face tag in the complete database would be nice. Last is essential when I realise that I have a second John Smith and I want to change the name to let's say John Simth Jr. to the person already existing in the database. Hungarians have the family name first, so the 10. sorting (in the Collection view I suppose) is not much of a problem for us but it is for others. Well, I do not know how to implement the sorting feature, because there is only one field for the name in the face tag, so it is hard to recognize the first and the family name.
Actually, this face tagging feature might be a complete project for Mario (even the user interface might need some new menus or buttons) if we want a mix of Picasa and some Lr features in IMatch, as he summarized our requests.