Link Persons-DB also to Images, not only to face annotations

[voronwe]

As far as I understand the  discussion with Mario (https://www.photools.com/community/index.php?topic=11152.0), the connection between a Person in the Person-DB and an Image is only done via Face-recognition, that means if an Image has no face recgonised it is not connected to a person.
This is from my point of view a disadvantage, because a person is more the only a face.
With the current situation pictures of a person which does not show the face (e.g. picture from the backside, body-parts ect.) can not be connected to a person in a proper way.

Also, the face-recoginition is very time consuming in cases where it is absolutely clear which person is on the image - for example shootings with one person, where in the end you have e.g. 300 pictures of the same person.

My proposal is: Use the "IPTC Extension\Person in Image" as a second opinion to connect a person to an Image. In this case it would be very fast to connect a lot of Image to a person (just mark the Images and write the Name to the tag).
In this case, the person DB can also check whether the person was set by face-recognition or manually by the user. In the latter case, do not use them for the Face-recognition AI (this point is of course unclear, because I do not know how the AI works )
-> This means the DB could have a new flag.

In this case the person Database would be more flexible. In the End it is called "Persons" and not "Faces"

[Mees Dekker]

Exactly for this reason, I created a category for each person. Also it is easier to build a hierarchy of persons (like "friends, collegues, relatives, etc.).

And by doing so, it is easier to combine them  in a filter or in the (category-) builder (e.g.: find all pictures that show myself and my father, but not my mother in the year 19.. while on vacation in England).

[voronwe]

Exactly for this reason, I created a category for each person. Also it is easier to build a hierarchy of persons (like "friends, collegues, relatives, etc.).

And by doing so, it is easier to combine them  in a filter or in the bulder (e.g.: show all pictures that show myself and my father, but not my mother in the year 19.. while on vacation in England).

But then you have a category and the Person-DB, which contains additional information about the person. Or how did you connect the Category to the Person-DB?

And how do you handle persons with multiple hirachies?
Example: Person A is a member of multiple organisations. How to show this in a hierarchy? (Beside the problem that it seems that currently no multiple organisations are possible in the Person-DB, but that is a different story (https://www.photools.com/community/index.php?topic=11042.0) )

[Mees Dekker]

I don't use face recognition for this reason.

Categories can have aliases; see: https://www.photools.com/help/imatch/#cat_alias.htm

[voronwe]

I don't use face recognition for this reason.

Categories can have aliases; see: https://www.photools.com/help/imatch/#cat_alias.htm

So you do not use Face recognition or the Person DB at all?

Do you write the categories back to the Metadata? Means the Name of the Person?
I was thinking about filling the category data-Driven from the "Person in Image", but currently I'm struggeling with subcategories as well as having a correct sorting (because the Names in the "Person in Image" are First Name, Familiy Name, but a correct sorting should be of course by familiy name.

And also the Face-recognition is a benefit if you have a group of people on the image.

Using categories in this case would mean to have 2 Person DBs. Does not sound good to me

[Carlo Didier]

I don't use face recognition for this reason.

Categories can have aliases; see: https://www.photools.com/help/imatch/#cat_alias.htm

Same here.

[sinus]

I do also not use face recognition.
The reason for me is simple: I created for every person a keyword.
So all names of persons are in the metadata.

Once you have keywords created, you can do a lot with them and with the help of categories, like other here pointed out.
No problem finding one person or 2 specific persons or all persons alone and so on.

The key for me is a good structure of keywords or at least categories.

Face recognition is fascinating, and maybe I will use it one day, but because I have already a person-structure in IMatch, it is not really necessary.

[Mario]

You can achieve much with keywords or categories, or a combination of both.

But keep in mind that assigning a person to a face (automatically or manually) also assigns keywords associated with the person to the file.
A person also represents much more than a simple keyword, from the name, previous names, birth date and other person data.
And future versions of IMatch will build on that, added more features in this area, more ways to work with persons and events and files in all sorts of ways.

[voronwe]

And future versions of IMatch will build on that, added more features in this area, more ways to work with persons and events and files in all sorts of ways.

And this is why I'm asking for other ways to fill the persons DB . And as I see here, some (many?) do not use it at all, but Kategories, so it seems that also a way to fill the Persons DB from categories would be nice.

[Mario]

There is no "Persons DB".
IMatch manages one or more face annotations per image. A face annotation is optionally associated with a person.

Looking at a tag like your PersonsInImage (one or more one-word text strings) or a bunch of categories won't do much.
No annotation can be created from that. No person can be linked, because no annotation. Etc. I've wrote it above.

The persons feature depends on persons being annotated in images (where the person is). All else is linked to that.
IMatch cannot create face annotations from a bunch of words in a random tag or a bunch of categories. It does the inverse alreaady- assigning a person to a face in an image sets keywords and categories automatically.

[voronwe]

There is no "Persons DB".
IMatch manages one or more face annotations per image. A face annotation is optionally associated with a person.

Looking at a tag like your PersonsInImage (one or more one-word text strings) or a bunch of categories won't do much.
No annotation can be created from that. No person can be linked, because no annotation. Etc. I've wrote it above.

The persons feature depends on persons being annotated in images (where the person is). All else is linked to that.
IMatch cannot create face annotations from a bunch of words in a random tag or a bunch of categories. It does the inverse alreaady- assigning a person to a face in an image sets keywords and categories automatically.

Hi Mario

ok, I do not know the internal structure of your DB, but for me I would assume the following:

There is a Table with the Persons in it. So if an Image has an face annotation, I would think that the Image-Entry would have the information that there is an annotiation (with position, size and so on), and that this annotation would have a link to one entry in the Person Table.
Am I correct so far?

[Mario]

A file can have any number of face annotations.
Face annotations can be linked to a person object in the IMatch graph.

[voronwe]

A file can have any number of face annotations.
Face annotations can be linked to a person object in the IMatch graph.

Ok, so far I understood it correct. So my question would be to have the link from an Image to a Person not only via the face annotation, but directly, like this:

Image1 -> Face Annotation 1 -> Person 50
          -> Face Annotaion 2 ->  Person 42
         -> Direct Link 1        -> Person 20
        ->  Direct Link 2        -> Person 12

(Ok, this is probably not a use case, because either you mark all person as Face annotation or none, but I hope my point is understandable)

Image 2 -> Direct Link 1  -> Person 50

And now, when I ask  in PersonView "Give me all Images with Person 50", I would like to see Image1 and Image2.
Currently, it is giving me only Image1, because it relies only on Face Annotations.

I am not asking for creating a face annotation from a name. This was never my idea.
My request is completely based on Text:
- When adding a new Name, use the Person Table as a Theasaurus and connect the image to the person
- When there are allready Names in the Image -> search for that names in the Person Table. -> If it is allready there, connect it, if not, ask the user how to connect (maybe misspelling) or add a new person.

[Mario]

This would not really work. Or I would have to rip apart substantial parts of how IMatch works with face annotations, files and persons and rewrite them. Very unlikely.
Don't wait for this. I recommend you use face recognition and annotations and let IMatch do all this automatically.

[Tveloso]

I do use IMatch's Face Recognition functionality, and I find it to be very near 100% accurate with my family members.  For them, the face recognition operation usually creates already confirmed annotations that are correct...and the unconfirmed ones are usually correct as well.

And for me it takes a lot of manual efford, because the regognition is often wrong - for me it seems that it has problems especially with children.

In the early days of Face Recognition in IMatch, I did experience issues with incorrectly assigned persons (particularly with my wife and my daughter), but other users here pointed out that having a lot of  Rejected Faces might be contributing to the lowered hit rate, and when I deleted the rejected faces, IMatch then began almost always identifying them correctly.

Perhaps you're running into that Rejected Faces problem?

In fact, with my grandchildren, who look very similar, it's uncanny how well IMatch recognizes them.  There have been times where I thought IMatch had confused my new granddaughter with her older brother (who is just 16 months older), based upon the Thumbnail in Face Manager...but when I opened the photo in the integrated viewer, I found that IMatch had it right, and it was me that had it wrong!

What will happen with images with persons where you can not see the face (Photographed from the backside, body-parts only etc...) - How to handle such pictures?...

It's true that when a face is not detected, a bit of manual work is needed to get the Face Annotation added, but I think it's worth the effort to do it.

I think that even if I had had already established Keywords and/or Categories to identify Persons, as many here have done, I might still want to use the Face Recognition functionality in order to get Face Annotations added to the images...

[voronwe]

Good morning

This would not really work. Or I would have to rip apart substantial parts of how IMatch works with face annotations, files and persons and rewrite them. Very unlikely.
Don't wait for this. I recommend you use face recognition and annotations and let IMatch do all this automatically.

And exactly what should IMatch do automatically? As far as I understand, I have to set a dummy face annotation somewhere , then I have to add a person to this face annotation manually (because the recognition will not handle it of course), then I have to copy this to each file one-by-one with STRG+V.
Is there a better way to automatise this, I would be glad to hear it.
And, may I quote you from the other thread:

IMatch links persons to face annotations in the image, not to the image itself.
IMatch would need a feature which creates a face annotation from the name and places it at an arbitrarily position and site somewhere in the image.
And then produce useless face regions during export, potentially confusing or breaking other applications. Not good.

I understand this in a way of "Do not do this". Maybe I overread the solution how to handle Images of persons without faces, but I can not remember it.

All I ask for is something like this for Persons

SELECT <Image> FROM <Image_table> WHERE <PersonInImage> LIKE %<Name>%

This is not possible with your Database-structure?

What I get so far from the answers is that the Person-View is only for Face-annotations and nothing else. It is a little bit frustrating for me as a user, when I see such a feature like persons, but it is stucked in a cage, because it is only useful in a very special way of using.

And this:

And future versions of IMatch will build on that, added more features in this area, more ways to work with persons and events and files in all sorts of ways.

sounds very strange to me when it can not fulfill my basic needs.

In the early days of Face Recognition in IMatch, I did experience issues with incorrectly assigned persons (particularly with my wife and my daughter), but other users here pointed out that having a lot of  Rejected Faces might be contributing to the lowered hit rate, and when I deleted the rejected faces, IMatch then began almost always identifying them correctly.

Perhaps you're running into that Rejected Faces problem?

I checked it and I foud for my smaller daughter 5 rejected faces. Anyway, why should I delete them, they are rejected, so why should the AI consider it.

Just an interesting case: I have about 400 pictures of my younger daughter face-recognised, and 5 of my nephew. When I run the face-recognition on a pictures with my daughter, in nearly half the cases it detects my nephew. Why? Statistically it is much more likely that it is my daughter (An AI, in the End, is mostly statistics).

Not to mention that in many cases it does not detect a face at all. I run the face recognition on the attached picture. It does not detect a face at all. This is a picture of the shooting I mentioned earlier: 300 images, about 150 only are recognised as faces.

In the end (I think I mentioned this earlier): The Goal is to have an easy way to search for images with persons with the Person-View. Face-recognition is only a tool for filling it with data, but from my point of view not the only one. But what I read out from Marios statements (an please correct me with examples when I'm wrong) is, that this is the only way and anything else will not work because of the DB-structure.
For me this is like: You allways have to use an electric screwdriver for putting a picture on the wall, even if a hammer would do it.

[Mario]

I understand that IMatch is not working like you want it to work for your specific use case.
This is just how it is. IMatch cannot do everything in every possible way.

There is no way to produce face annotations vector objects from arbitrary text data found in metadata tags.
And these are the base for face recognition, linking persons to images (the persons are linked to the annotation, not to the image!).
The IMatch AI, face recognition, vector face annotations and the IMatch Graph work together to make the People View and all related concepts work.

Implementing all the changes you want to make your use case work would probably cost two week of work. Or more.
Because of all the extra tests to write, the documentation updates, figuring out how to deal with face annotations which have been produced at random from tag data but which have no real meaning, size or position.
Which must always be excluded from face recog and hidden in the Viewer and Quick View panel. Yet there must still be a way to remove them or update them when you or another application changes the tag you want to use as the source for these people. I don't see this happen.

I recommend you create a data-driven category based on the tag you use for storing people names. This will organize your files by the people they contain, and you can see them, combine them, use them for filtering etc.

Just an interesting case: I have about 400 pictures of my younger daughter face-recognised, and 5 of my nephew. When I run the face-recognition on a pictures with my daughter, in nearly half the cases it detects my nephew. Why? Statistically it is much more likely that it is my daughter (An AI, in the End, is mostly statistics).

Please review these two persons in the Face Manager.
Most likely some very similar faces have been used for training, spoiling the recognition.
Untrain these faces to improve recognition quality.

[Carlo Didier]

...the persons are linked to the annotation, not to the image!...

This is the main reason why I don't use face recognition and the persons functionality either.
I want to link persons to images. This could be through annotations from face recognition, but it should not be limited to that.
I have currently many pictures associated with persons (through my manually managed categories, but it could also be keywords or something else) and only maybe half of those have recognizable faces in the image.

A person record in the persons db could be associated with face annotations or any other information (categories, keywords, even folder names!)

The same as in real life, a person can be referenced by his picture or the social security number or the ID card number, ...

[ben]

Very interesting discussion. 
From what i followed in this forum within the last year it reflects the two different point of views.


Personally, I think Carlo is summarizing it quite well:

Quote from: Carlo
I want to link persons to images. This could be through annotations from face recognition, but it should not be limited to that.

As always i am looking forward to new features that Mario is planning.
But i fear, the current design is locking out users because they don't use face annotations or additionally use keywords/categories.

Quote from: Mario
And future versions of IMatch will build on that, added more features in this area, more ways to work with persons and events and files in all sorts of ways.

There is one statement that i absolutely disagree:

I understand that IMatch is not working like you want it to work for your specific use case

This is not a specific use case. Unless you handle only portrait pictures (faces are detected very likely),
having persons with undetectable faces is a very common use case in my eyes (people from the side or from the back or partially hidden).


So, i really hope that one day all the peoples features will allow the flexibility to link people by a face annotation and also by other means.
Until then, face recognition is nice to have but i stick to keywords/categories.

[sinus]

So, i really hope that one day all the peoples features will allow the flexibility to link people by a face annotation and also by other means.
Until then, face recognition is nice to have but i stick to keywords/categories.

I think exactly the same and do also the same.

I do really not know, if this can be roughly compared:
A relative of me is deceised these days.
Thanks to my own person-system (with keywords/categories) I have no problems to find all images or videos from this person.
Very good and I can use this at the moment very good.

But now I have for the funeral some other files like music, speach, drawings and so on.
And also images, where this person is not on the image, it is only linked because the deceised person had interest in this theme.

Now makes it IMatch easy, to "link" all these files to one image (in this case the death person): I can use simply stacks. VERY good!

Those, who works with the face annotations, can for sure better decide, if we can comparing this somewhow.
Maybe this is a complete other story, I do not know.

But what is true, I have also a lot of pictures, where we cannot see the face, but I would like to link it to a person.

[Mario]

But what is true, I have also a lot of pictures, where we cannot see the face, but I would like to link it to a person.

Just add a face annotation where the person is approximately then?
The rest is fully automatic.

All face- and person-based features in IMatch will immediately work with that file.
Keywords and categories associated with the person will be added.
The "PersonsInImage" tag will be updated.


@All

If my statements above were not clear enough: This works how I have designed it. And it works very well. I'm happy with it.
I will not break the links between faces and persons in the IMatch Graph or somehow hack this in, adding all sorts of extra code paths, switch statements, special handling etc. Dealing with persons linked to faces in files, or to the files directly, or a mix of the two.
While this would probably be doable, it would also be a large amount of work, and changes on such a substantial level will inevitably introduce new bugs, cause compatibility issues with existing databases, maybe reduce performance for all users etc. It took me almost a year (!) to get face recognition and the person-related features working for IMatch 2020. And it works very well I dare say.

If 50 or more users support your request below, I will think about it again. Then it would be a problem affecting many and that needs a solution.
Until then, I concentrate on IMatch 2021 and the open bug list.

[voronwe]

Hi Mario

But what is true, I have also a lot of pictures, where we cannot see the face, but I would like to link it to a person.

Just add a face annotation where the person is approximately then?
The rest is fully automatic.

Which is a hack. A solution of the problem which will work now, but it is a dirty one, because it misuses face recognition

If my statements above were not clear enough: This works how I have designed it. And it works very well. I'm happy with it.
I will not break the links between faces and persons in the IMatch Graph or somehow hack this in, adding all sorts of extra code paths, switch statements, special handling etc. Dealing with persons linked to faces in files, or to the files directly, or a mix of the two.
While this would probably be doable, it would also be a large amount of work, and changes on such a substantial level will inevitably introduce new bugs, cause compatibility issues with existing databases, maybe reduce performance for all users etc. It took me almost a year (!) to get face recognition and the person-related features working for IMatch 2020. And it works very well I dare say.

If 50 or more users support your request below, I will think about it again. Then it would be a problem affecting many and that needs a solution.
Until then, I concentrate on IMatch 2021 and the open bug list.

It is a feature request. I do not see feature requests as to be done as soon as possible, but at least to be put in a kind of backlog.
But please consider it for the future, even if you say that it can not be done for 2021 (which is absolutely fine, there must be a cut called "feature freeze", otherwise a software would never be delivered).

I'm not completely sure how you are working, but as far I understood, you are the one and only person who works on IMatch. If so, I think this is a dangerous situation, especially for designing a software, because it sometimes leads to a blinker-View (Scheuklappenblick auf Deutsch). This happens involuntary, simply because one person can not oversee everything. And I know. from my personal experience, that if I developed a great feature, which works well, but now people from outside come with new ideas, about which I have not thought (simply because no one can see every possiblity), that I start also fighting for my original design.

But: It seems that I'm not the only one who is not so happy with this feature, as it is implemented now. From my experiences, new feature ideas come in mainly from two sources:
(1) Ideas developed in-house (brainstormings, someone got notice about fancy stuff, ect.)
(2) Feature requests from customers.

Number (2) is more important, because this means that there is someone outside in the "real world" who is working with my product, and who is facing a problem, which can not be solved with the current version. Means this is something I have not considered because it did not came to my mind so far (reasons: see above). And answering this with some like "I will not do it, because it is a lot of work" is in my opinion not a good idea, because in the end, the customer pays for a product, not for the work it takes to make it.
You say "This works how I have designed it. And it works very well. I'm happy with it.". It seems that I'm not the only one who disagrees with the second sentence. Because it work very well in your special usecase, but there are usescases (pointed out by different persons) where  it does not work that well.

The question is, what would be better? A developer who is happy with his software or a customer who is happy?


These are just my thoughts about it. I'm saying this as somebody who loves IMatch and as someone who is trying to help to have a product which makes everybody happier.

Greetings

Thorsten

[Mario]

Which is a hack. A solution of the problem which will work now, but it is a dirty one, because it misuses face recognition

I beg to differ.
Adding a face annotation manually when a face is not detected (for whatever reason) is not a hack.
It is a normal procedure. No AI is perfect.
If no face is detected, you add a annotation and link it to the person.
If a face is detected but IMatch cannot find a matching person, you assign a person manually. This is how the AI learns.

The question is, what would be better? A developer who is happy with his software or a customer who is happy?

It seems I have made you angry for some reason. I apologize.
But please keep in mind that I'm one person developing IMatch in his spare time. And there are many users. With many requests. Most of which get no likes or support at all.

No software can make all users happy all the time. This is impossible. Whether it is created by one person or by a team of hundreds of developers.
I'm proud that most people think that IMatch works very well and that my support is better than average. It seems you don't think so. Which is OK.

[Mees Dekker]

Should we not calm down a bit?

Your support is fantastic, Thorsten (Voronwe) does not say anything else. Imatch is a superb product, especially when you take the low price into account. Exactly for this reason, there is a very loyal customerbase here. So you can be very proud indeed of what IMatch is today and how it performs.

However: any product can benefit from users idea's. What works for you, does not necessarily work for everybody. There are a zillion workflows out there. So keep an open mind to these idea's.

That is not to say that everything is technically possible or doable by one single person named Mario. Time is scarse and precious, priorities have to be set. Any user will understand that.

[voronwe]

The question is, what would be better? A developer who is happy with his software or a customer who is happy?

It seems I have made you angry for some reason. I apologize.
But please keep in mind that I'm one person developing IMatch in his spare time. And there are many users. With many requests. Most of which get no likes or support at all.

No software can make all users happy all the time. This is impossible. Whether it is created by one person or by a team of hundreds of developers.
I'm proud that most people think that IMatch works very well and that my support is better than average. It seems you don't think so. Which is OK.

In fact yes, you did, I can not deny it and I tried to put my word as polite a possible, trying allways to see the other side. For me I see this big, great feature "Persons", which I can not use in a proper way, because allways I try something there I run agains hidden walls. This frustrates me, especially in a tool like a DB, which should simply work and which is nothing where I want to spend creativiy on. Maybe only for finding easies ways to make the adding of data less time consuming - but also this with a limited efford. I want to spend more time in tools like Photoshop where I can be creative than in a DB. See the DB like a Camera - it is only a tool and when this tool gives me or does not work well, I get disappointed, because I dont want to spend my time in finding workarounds to use a tool in a way I want to.

The point is also that I paid money for this software. So I think I can expect something. Not all, of course, but more than I would expect from a software given away for free. If IMatch would be a free Software like XNView, I would not go till this point, because in this case I would say, well, it is for free, so I can not claim something, only ask. But as soon as money comes in, it is a different story. And please see, that US$ 129.99 is not cheap. Maybe compared to other DAM-systems (To be honest, I do not know the price), but if you compare it to something like Affinity or even Photoshop (I think one year of Photoshop is in the same direction), it is quite high, especially for a hobby-user as I am. (Of course you could say, this is selled in higher amount, but companies like Adobe or Affinity have also higher costs).

Please do not understand me wrong, I will not question your business-model, but from the professional view of your website and the product I was expecting at least that you are working fulltime on. If you do it only spare-time, Chapeau and many thanks for your great support. I sometimes wounder if you have a life outside IMatch, as much as you are posting here (An thats why I had the feeling, that this would be your fulltime-job)

I know that it is impossible to make everybody happy,  but the reason you gave why not came that fast and ultimate (and it was only a small request I made from my point of view - I hope I made this clear with the SQL-Line), that I was very disappointed.
I had the same quite often, where I said fast "It can't be done", but after a while and rethinking it, I found a way.

Some ideas need time to mature, so what I'm asking for is: Take your time, rethink my request, and maybe, you will find a way with not so much effort to handle it.

Thank you for your great work (and please see this as a real thank you, there is nothing ironic behind it)

[Carlo Didier]

Hi Mario

But what is true, I have also a lot of pictures, where we cannot see the face, but I would like to link it to a person.

Just add a face annotation where the person is approximately then?
The rest is fully automatic.

Which is a hack. A solution of the problem which will work now, but it is a dirty one, because it misuses face recognition

Exactly!

And it leaves out all those who already have tagged persons in images through other means (like categories).
Nobody will take his person categories and manually add fake face annotations to hundreds or thousands of images ...

[Mario]

So, this makes two or maybe three users who support this big change.
Let's see how many other users support this.

[Mees Dekker]

+1

[David_H]

So, this makes two or maybe three users who support this big change.
Let's see how many other users support this.

I think I would like something similar - even if its just an additional attribute on the person record to store a 'manually assigned' category; thus the files with a person in consists of the distinct result of all those with their face annotations AND those from the manually assigned category.
This might also be useful in that a 'master' would be the one marked up, and transient copies (Edited, cropped, etc) would still show up even without the face annotations (assuming correct version propagation)

[ben]

+1 for

I want to link persons to images. This could be through annotations from face recognition, but it should not be limited to that.

For me it's important to say: This is a feature request and this is of course independent of what i pay for iMatch. Just wanted to mention it.

[LeslieP]

Seems to me that this post can be moved to the SOLVED folder, since the capability will be introduced in v2021.

https://www.photools.com/community/index.php?topic=11360.0

I'm pretty excited about all of this. Just upgraded to v2020 and will be cleaning up my face annotations, and getting rid of the detritus from having used Mylio for its face recognition capabilities. The IMatch FacialRecog capabilites are so far superior that I'm happy to convert, and thus reduce the number of "tools I use" by one.

My plan is to use IMatch as a true DAM, managing all of my digital assets related to my family history. There will be JPGs with no people in them, there will be PDFs, and the ability to link them to the related people is important.

[voronwe]

Just want to say:
I tried this new feature today and it works like a gem.

That is exactly what I needed - Took me about 10seconds to annotate 200 images with the same Person

Thank you very much for implementing it