Vision Based keywording

[StanRohrer]

Mario,
Have you had any thoughts about incorporating vision based keywording into IM?

https://www.eyeem.com/tech
Try EyeEm Vision about half way down the page.

https://eyeem.3scale.net/


https://creative.adobe.com/addons/products/16333#.WMQs6fnys_U

https://photofocus.com/2017/03/11/fast-easy-photo-keywording-from-the-cloud/

[Mario]

Yes.

The ability to integrate such services will be much easier once IMatch 2017 with the integrated IMatch Web ServicesTM is available.

Naturally, users will have to make their own arrangements with the vendors (getting a free or paid key). I cannot pay for that, sorry.
Most services offer a reasonable free request package, though. But if you plan to process tens of thousands of files, it will cost you.

And also consider carefully the privacy issues. Most vendors reserve the right to retain the images you send to them for "improvement" or research purposes.

You did not list the Microsoft Vision site:

https://www.microsoft.com/cognitive-services/en-us/computer-vision-api

which is quite advanced and ahead in many areas. And they offer reasonable packages as well.

[StanRohrer]

The Microsoft free version would certainly be useful for us small time photogs. I'd be willing to fork out a little money to a provider if the interface worked well enough. I played with the EyeEm version today with some images. It was sort of hot/cold. It identified some images very well and some poorly. But it will be very helpful as the technology gets better.

[Mario]

I have written a neat IMatch 2017 app for testing purposes a while ago.

It runs in an App Panel on the right side of the File Window.
When I click the button, it sends the thumbnail of the selected file to the external vendor and retrieves the analysis results. Takes less than one second.

From the result it produces suggestions for keywords and the description.
Depending on the vendor, my app even gets GPS coordinates (it a known location is shown in the image, e.g. a building). And often even the name of the building "National Gallery, London".
Or information about the age and mood of the persons shown in the image.

I can then just edit/tick of the information I want to save to the image metadata.

This is how I would integrate such services into IMatch.
I could even envision to allows users to run this in the background, when new images are added to the database.
If the technology gets better or your images are 'easy', this would reduce the manual input required considerably. With the odd exception of course.

There are several vendors providing cloud-based image analysis. Some are better than others, the big players like Google or Microsoft have some clear advantages. All vendors have different programming interfaces so a general app would have to manage that behind the curtain.

As I said in my initial post, the switch to JavaScript programming and the integration of IMatch WebServices in IMatch 2017 will make integrating such services much, much easier. This is one of the main reasons for the switch to web technologies from Basic.

[Jingo]

+1 from me... so long as my images and analysis data are not stored by MS or Google (hard to believe they won't be) - this is a great option.  Looks like a few of these offer API's already so I can see how this could be added into IM as you describe Mario...  Looking forward to IM2017 more and more each day!

[Mario]

+1 from me... so long as my images and analysis data are not stored by MS or Google (hard to believe they won't be) - this is a great option.

Google stores everything about you. And they feed their AI with whatever you do or upload. FB, Microsoft etc. all the same.
All vendors who offer cloud-based face recognition keep copies of the faces you upload. This is required, else they could not identify people.

I have not found a "local only" face recognition technology that works nearly as good as the one from MS etc.
At least none that does not cost at least a five-digit sum per year. Adobe can afford this, I don't.

When I look into this as some later time (users are invited to implement their own apps of course) I will consider privacy implications and make sure that users know about this.