Excire support planned?

[Lord_Helmchen]

Mario do you have planned to support (or better start a cooperation) with Excire (https://www.excire.com/)? I would be interested in some automatic categorization of photo content - as extra module to purchase. Would this technically be possible (I haven't checked how it works in detail) and would you be interested in supporting this?

I have read some very positive reviews and also heard good experience from other photographers. And as it's available as plugin for LR, I think technically that would be possible for IM too.

<edit>
I know that you can't implement this by your own. But possibly you could get in touch to that small German company.
</edit>

[Mario]

They provide per-call fee-based web services. I've looked into them when they published. I've asked for pricing and privacy months ago but only got a generic reply.

Before you judge, you may also want to check out the similar services provided by Google:

https://cloud.google.com/vision/

(Use the "Drag image file here...") box in the middle.

and Microsoft:

https://azure.microsoft.com/en-us/services/cognitive-services/computer-vision/

Both services well worth to play around with. I think they are equal or even better than Exciire in many areas.

I commented on this several times in recent posts: The availability of such A.I. based services was one of the driven factors for the massive amount of work that went into the embedded IMWS in IMatch 2017 and the entire app ecosystem.

This groundwork will allow me to integrate such services much easier. From face recognition to license places, from text recognition to automatic mood and sentiment analysis. Runner numbers for sports photographers etc.

Of course payment will be a challenge. Excire does not publish prices but Google and Microsoft do. This will given you an impression of how much this costs.
Furthermore, these services require authentication (A "key", like for Google Maps or Bing). I can either let the users insert they own keys so they deal with Excire, Google, MS themselves. Or I route this through a proxy server I need to setup so I can use my own API key (in secret) and I need to cover the cost - and charge users of course for the service.

I think this will be one of the challenges with integrating such 3rd party services into IMatch. And of course privacy. If a user wants to use these services he has to allow IMatch to send his images (or thumbnails) to Excire, Google, Microsoft, Frauenhofer or whoever.

[Lord_Helmchen]

Thanks for your quick response. I have not looked into details of the services mentioned by you. But IIRC, the others are SaaS, so they would get my photos. And that would be a no-go for be.

Therefore I hopes there could be a short connection from a small German software developer to another small German software developer.

[Mario]

I've contacted them several months ago but only got a generic reply. I shall contact them again when there is time.
Why don't you use the Lr plug-in? The keywords should show up in IMatch just fine afterwards.

From the web site it does not look like they sell any kind of development API I could integrate?

But to give you a figure: There are some vendors out there which offer APIs for face recognition etc. I could integrate. None of these lists prices so I had to contact them.
The cheapest product I could integrate in IMatch had an annual license fee of about 12,000 US$. That's nothing for a company like Adobe but I'm sure you can understand that this is nothing I can afford.

I just wrote Excite another email. Maybe we can figure something out.
Their "Seearch engine" seems to work like a locally installed web service - basically the same IMatch and IMatch Anywhere do.

[Lord_Helmchen]

If I'd use LR, I would have thought about that. But I use DXO as raw developer and IMatch as DB - so no need for LR. I only have a old, very old version and don't want to use another tool for processing my photos. Main reason for kicking LR was the need of importing all photos additionally to IM...

The price for the LR plugin (I think similar range as IM) would be acceptable. But I could understand that you don't want to invest $12k without knowing that you'll get it back from additional licenses.

Hope they will answer now.

[Mario]

This is an expensive technology.

If you use one of the cloud-based providers for face recognition / image analysis you pay about 1.5 US$ for 1000 "calls".
This means sending 1000 images for face detection or image analysis will cost 1.5 US$.
If you want face and feature detection / labeling  (for keywords, descriptions) 1.5US$ dollar will mean 500 images.

Of course if a user pays for himself, and he has, say, whooping 100,000 images to process, he will end up with something like 150 US$.
Or only 15 bucks when he has only 10,000 files. That's affordable, considering the potential time savings.

But these examples show that it will be impossible for me to provide such a service for free.
I can only implement and give my users the infrastructure but I cannot pay for it from my own pockets.

Excire wrote me an email to day and asked for my phone number. I made it clear that this is a spare time project and my budget is minimal. Maybe we can work something out.

[Lord_Helmchen]

I don't know, if Adobe pays for Excire. I think they are "too big" to pay a small company. From my understanding Excire is an offline application that has to be paid by the end user. So on your side would be the implementation effort. Possible chargeable to users who want that option...

[Mario]

Excite sells the Lr Plug-in. This is unrelated to Adobe.

[Lord_Helmchen]

Yes, and it would be great if a similar solution would be technically possible for IM - even for small user base of IM compared to LR.

[Mario]

Adobe sells millions of copies of their software and has implemented a nice life-long subscription schema for most of their products now. Huge market.
The IMatch market is a niche compared to that. Maybe I can work something out with Excite.

[jcldl]

I did not know "excire" before reading this post. But after that I tried it in lightroom, I think it's interesting and I would be happy if I could use it in Imatch.

[Mario]

The Excire bundle costs 99US$.
Would you be willing to pay that much for an IMatch Ad-on?

Have you tried out the demos from Microsoft and Google I've linked to above. Quite interesting.
These show how far these companies are, including age grouping, sentiment analysis, automatically locating buildings on images, text recognition...quite impressive.

[jcldl]

OK thanks, I shall try Google.

[mastodon]

An APi for face recognition would be nice. I have a lot of work with family photos to set the face tags.
And we would pay per use (15-20 $ for 10000 images are nuts, if it much better than Picasa), and you would share the fee with the vendor. I would pay even just for the API, so for the possibility in advance. So, that way you would be work for nuts or nothing.

[Mario]

One problem is billing. Google, Microsoft etc. are not prepared for many individuals which need 1000 units each.
The billing is designed for a 3rd party (me) to utilize Google's/MS's services in one of their products (IMatch) and pay the bill for everybody.

It is of course clear that I cannot pay for all IMatch users. And a user who does not use these services should not pay more.
We thus need to find a way to

a) make users pay only for what they use
b) somehow compensate me for the work and time required to make all this work.

My current general working model idea for integrating "pay-per-use" web services (like the computer vision technologies from Google/Microsoft/others) is as follows:

1. I learn how all this works (Sigh)
2. I implement apps for Google/MS etc. which are shipped with IMatch

Now, there are two ways:

A)  I write the app in a way that the user can enter his own API key / user account for Google/MS. Similar to the Map Panel.
The user then has full control over the process, if he exceeds the 'free' units he gets each month, how he settles payment etc.
This is the easiest solution, but may be complicated for the IMatch user.

B)  I setup and maintain a webservice infrastructure in the cloud to handle this.
The IMatch app then utilizes this service, which in turn utilizes the services provided by Google/MS etc.
This allows me to keep my API keys secret, control the usage per user and total cost, and it shields the user's IP from Google/MS (all requests come from my server).
I can also handle load balancing and API changes more easily.

Now, If a user wants to use the app, he needs to pay extra. Say, 5 $ a months for 2000 face detections and auto labeling.
My distributor MyCommerce offers subscription payment models which would make that fairly easy.
You pay for what you use.If you exceed your monthly allowance, the app will not function anymore until the next month.

Technical challenges are manifold and a lot of work would go into that. But it is one big step into SaaS, and that's how things are going anyway.

[Art]

A)  I write the app in a way that the user can enter his own API key / user account for Google/MS. Similar to the Map Panel.
The user then has full control over the process, if he exceeds the 'free' units he gets each month, how he settles payment etc.
This is the easiest solution, but may be complicated for the IMatch user.

B)  I setup and maintain a webservice infrastructure in the cloud to handle this.
The IMatch app then utilizes this service, which in turn utilizes the services provided by Google/MS etc.
This allows me to keep my API keys secret, control the usage per user and total cost, and it shields the user's IP from Google/MS (all requests come from my server).
I can also handle load balancing and API changes more easily.

Now, If a user wants to use the app, he needs to pay extra. Say, 5 $ a months for 2000 face detections and auto labeling.
My distributor MyCommerce offers subscription payment models which would make that fairly easy.
You pay for what you use.If you exceed your monthly allowance, the app will not function anymore until the next month.

Technical challenges are manifold and a lot of work would go into that. But it is one big step into SaaS, and that's how things are going anyway.

Personally I would prefer option A, IF I would use such a service. Setting up another webservice inbetween the user and the actual service provider increases the risk of things going wrong and could possibly mix up files/descriptions if multiple users would use the same filename though the unique id might solve most of that. I think that writing and maintaining code  to avoid the latter etc. would take away too much time from IMatch development. The load balancing of option 2 would not really be necessary if the add-on would directly interface from the user to Google (or others)

I know that several software programs have API hooks that you only have to connect to and then run the add-on and only need rewrite of your side of the code if there are really major changes. Maybe Google has something similar already?

[Mario]

For option A, IMatch users would have to open developer accounts with Google and find their way through the English Google developer sites and documentation.

Google changes and retires interfaces for their web services regularly. This is not a problem, I just have to keep the app / web service up-to-date. The same happens with OpenLayers, GeoNames, Google Maps, Bing etc.

Option B would work for all users out-of-the-box. No danger for confusing anything, the communication between IMatch and my service / Google will be within the scope of a "session", which is linked to a specific IP address / IMatch instance. My service manages the sessions automatically, and forgets all data used as soon as it is no longer required (after an image has been processed).

Option A is definitely easier for me. This is why I used this for Google Maps and Bing.

When I have some free play time I will learn about the vision apis provided by Google and Bing and see if I can come up with a useful app for that. This will be interesting.

Maybe Excite calls back and we can work out how we can integrate their technology in IMatch. They have a C++ interface which should work just nicely. It will all boil down to how much it will cost. Technology like this is always expensive and if there is no way to offload the cost to the users, there will be no way for me to integrate it.

[Mario]

I just had a quite fruitful telephone conference with a representative from Excire.

I will need to study the documentation and sign an NDA, but I will donate some of my time into looking into ways to integrate Excire into IMatch.

My current plan (subject to change without notice, bla-bla) is

1. learn about the options and how it all works.
2. Then, depending on the outcome:

- provide the Excire integration in IMatch
- Integrate it's search function, somehow integrate it into the Keyword Panel etc.
- Provide a dedicated app that eases the processing of the 'backlog' of existing images
- Integrate it into the indexing process so it runs automatically for new files

Note: If a user wants to use Excire in IMatch, (s)he will have to purchase a license of sorts. I envision this to be simple, like purchasing the Lr plug-in. You buy a license key from Excire, enter it in IMatch and the technology is unlocked and becomes functional.

No changes for users who don't want to use Excire.

[Aubrey]

Cool!looking forward to seeing this.
BTW I think the software is Exire!

Excite was a pre Google search engine in the '90s. A great product but never got monitized.

[Mario]

I'm sure this will be very welcome for many. I cannot say more, need to sign the NDA to get a test library to make a test integration and see how well it performs.
One of the best things for me is

a) it's local, no data transmitted to 3rd party servers (this was actually one of their goals)
b) they are OK with my plan to provide the integration in IMatch and they handling the license sales.

For me the benefit is that this integration makes IMatch more attractive, especially to users who have to add keywords and descriptions to many files.  Even if the Excire integration reduces the time to add keywords only by half, people can save minutes, hours, days or even weeks of work.

Not to mention the added benefits of features "Find me all files with a tree and a mountain" 

For users who don't need this kind of features, nothing changes.

[Menace]

Könnte eine Win-Win-Situation sein. Minimalanforderungen sind:

Mehrkernprozessor mit 64-bit Unterstützung
Betriebssystem   Mac OS X 10.7 (oder neuer) oder Windows 7/8.1/10 (64-bit)

Passt das mit IMatch (32-bit) zusammen?

[Mario]

Diese Angaben gelten für das Lr Plug-in.

[cytochrome]

This is very interesting and I would gladly (well...) pay for a face recognition tool linked to IMatch.

But I am not sure I get it right: on the Excire site they only talk about Lightroom and the LR catalogue. I don't use LR so the question is: would the Excire technology apply to images stored in the IM catalogue and write to IM categories/keywords?

Francis

[Jingo]

This is very interesting and I would gladly (well...) pay for a face recognition tool linked to IMatch.

But I am not sure I get it right: on the Excire site they only talk about Lightroom and the LR catalogue. I don't use LR so the question is: would the Excire technology apply to images stored in the IM catalogue and write to IM categories/keywords?

Francis

Yes... I believe that would be the goal... Imatch integration just like it exists in LR currently.

For me, the biggest benefit (depending on how well it works) is to find images and keyword them when I haven't added the keywords myself...  For instance, I am keen on historical marker signs and take photos of them plus the surrounding area whenever I can.  But - sometimes I just dump the photos into the catalog and forget to add specific keywords for things like "sign".  To be able to take a months worth of images and have Excire analyze them and show me all the ones that have a "sign" in it so I can then keyword it is priceless.... my hope is it can handle things like this and not just the more "obvious" stuff like trees, mountains and buildings...

[Mario]

If I create a Excire extension for IMatch, it will be fully integrated into IMatch.

You will not need the Lr plug-in. Excire will not create a plug-in for IMatch - I will.
You just buy a "key" from Excire and if IMatch gets that key it unlocks all features based on Excire.

I don't know too much about the actual capabilities or how I may integrate it - we're still in the sign papers phase.
They offer at least three ways of integration and I will check them all and then decide. I want a loose coupling, if possible.
Once I have access to their APIs and libraries, I will know better how everything works.

I expect a tight integration, including Excire suggestions in the Keyword Panel (or even a separate IMatch App), new content-based search functions, face recognition and more. Everything running locally and integrated into the IMatch database.

Other A.I. based technologies I'm currently investigating would be integrated in similar ways.

In short: I need to play with this, think deep and important thoughts and then everything will be nice and shiny 

[cytochrome]

Well, if Excire technology -or similar technology from Google or Microsoft- can be integrated into IMatch it would be a BIG step forward  for people like me who have tons of images with family, friends, local people etc. 

I would buy this. The simplest business scheme would be the best, pay a fee for the IMatch "plugin" and a license for Excire or the like.

Francis

[Mario]

Well, if Excire technology -or similar technology from Google or Microsoft- can be integrated into IMatch it would be a BIG step forward  for people like me who have tons of images with family, friends, local people etc. 

Precisely.

I would buy this. The simplest business scheme would be the best, pay a fee for the IMatch "plugin" and a license for Excire or the like.
Francis

I currently have no plans require a fee for the Excire-based plug-in. Unless I have to spend months over months to develop it (I doubt that). The benefit for me will be that my users can buy an "Excire IMatch Key" license and then can use their fabulous technology in IMatch to speed up keywording and searching.

[Lord_Helmchen]

Sorry for being quite for such long time as initiator of this thrad, but I'm on vacation and war more focusing on that... 

That's really great news - and exactly what I was hoping for. If you need some beta tester, just let me know.

[Jingo]

That's really great news - and exactly what I was hoping for. If you need some beta tester, just let me know.

Same here.. happy to put in the time to test this out...

[Mario]

I had some time and downloaded and installed Excire in Lr...

I think that this functionality will be a most welcome addition for many IMatch users (despite the extra cost for the Excire license).
And I hope that I can provide a much better integration in IMatch as Excire could do in Lr (the limits are in Lr, not Excire).

I have send out the papers and I hope to get the test components soon. Let's see what I can make of it.

For example, Excire has about 150 (?) classifiers (tags / keywords) they assign to each image. And I guess they will add more over time.
This would be ideal for a new collection hierarchy in IMatch, or maybe a category hierarchy similar to @Keywords.
All files in the database would then automatically show up on the corresponding collection, e.g. "House", "Mountain", "One face"., "Multiple Faces" and so on.
And when we have this info in collections or categories, we can use it for sorting, searching, filtering, variables, ....

The Excire "Find similar images" function would be integrated as a new IMatch command into the Search menu.

Let's see what other information and characteristics Excire can pull out of images and how I can make that useful for IMatch users.

[Lord_Helmchen]

I really like it when feature requests are analyzed and (most likely) be implemented by software vendors. 

Again great support by Mr. IMatch!

[ianrr]

+1

[Kucera]

+1

[mastodon]

+1

[medgeek]

At the risk of being repetitious, +1

[Frank]

+1

[mgm1965]

+1

[Lord_Helmchen]

Mario,

The Excire "Find similar images" function would be integrated as a new IMatch command into the Search menu.

Any news about progress of Exire support? Can we expect it for near future? 

[Mario]

So far I did not receive anything from them. I've signed their NDA and now I'm waiting for them to send me something to play with.

[mastodon]

I would pay even in advance for this plugin. I know, Mario, would do that.

[Mario]

I wrote my contact an email the other day. I'm sure they are busy and I'm just a small fish for them.

But after having installed their Lightroom plug-in, I'm sure that I could do lots better in IMatch. Lr does not offer much comfort for integrating something like Excire. But I can. And, making the data available via IMWS would allow me and others to create awesome apps using this technology. Lets...

[Lord_Helmchen]

Mario, do you think it would be helpful to contact them from customer side and tell them, that we're interested and could create some additional revenue on their side?

[Jingo]

Mario, do you think it would be helpful to contact them from customer side and tell them, that we're interested and could create some additional revenue on their side?

That is what I was just thinking as well.. "persuade" them to get the ball rolling a bit quicker with community input!   

[Mario]

I'm not sure that this will help much, but who knows. Thanks anyway.

The mailed me back today. Some internal shuffling and decision making. They'll give me a call tomorrow.

Google and MS make this easier. Just register for their services, start working. If you don't exceed a certain number of requests, your using their services for free.
Of course the major disadvantage is that the images (or thumbnails thereof) have to be uploaded into the cloud. Not so with Excire, which works locally.

Excire has no face recognition, though.
It can find images with faces (IMatch does that already) and it can, to some extent, find images by age or smile.

So far the only affordable way (meaning, less than 10,000 US$ per year) I have found are the services provided by Google, MS and some other cloud services....

[Lord_Helmchen]

Ok, I will just contact them and tell, that I'd be interested in getting Exire if it supports IMatch.

They can be contacted via following Web form or via email (listed on that site):
https://www.excire.com/contact-en/

[lnh]

Google and MS make this easier. Just register for their services, start working. If you don't exceed a certain number of requests, your using their services for free.
Of course the major disadvantage is that the images (or thumbnails thereof) have to be uploaded into the cloud. Not so with Excire, which works locally.

Excire has no face recognition, though.
It can find images with faces (IMatch does that already) and it can, to some extent, find images by age or smile.

So far the only affordable way (meaning, less than 10,000 US$ per year) I have found are the services provided by Google, MS and some other cloud services....

If it can't do facial recognition to the level of suggesting a person in your categories, Excire is of less interest to me. It might still be useful for everything else, but I'd rather pay for a Google or MS cloud service key to get a richer capability to at least get person level recognition. Privacy is always a concern, but no one is requiring connection to the cloud service and every IMatch user could make their own choice.

I haven't upgraded LR since v4, but I understand person level facial recognition is now part of the package. If it works well (?) then Excire could be viewed as a complement giving a full featured recognition capability even if the implementation is currently clumsy.

I do remember getting the developer key to connect Google Maps to IMatch. Kinda guessed about the right choices (and it worked), but it's hardly an end-user friendly process right now. However, short of Google making it simpler, a quick tutorial/help page would go part way towards addressing this wart on Google's side.

[Mario]

We discussed ways to integrate the Google / Microsoft Visual Intelligence features above.

Getting an API key from Google is usually nothing a end-user ever does. Usually the company (me) gets a key and then incorporates that into their web site or application. The end-user then just uses the services, and the company (me) pays for that. The problem is the "pays for that" part 

Even if I wanted, I cannot afford paying Google so you can do face recognition in IMatch. There is a certain amount of free calls, but that is quickly exceeded and then I have to pay 1.5US$ for 1000 face detections or so. Impossible. Consider 100 users doing an initial face detection on the 100,000 files in their database. 15 dollar cost for each - out of my pocket...

The easiest solution is for me to provide the feature to use Google/Microsoft and all the user needs is his own key.
A more user friendly solution would be for me to route everything through one of my web services, only allowing users to access the service which have paid extra for face recognition and other visual features. If a user exceeds his monthly budget my service would refuse. But that's getting complicated quickly as you can imagine.

The face recognition in Lr works OK. These are never perfect and you need to check each image manually for false positives. I have found that I'm usually as fast in IMatch doing in manually with face annotations and <F2> as I'm in Lr. Other users mileage may vary. I usually do only 100 or so images in a row, and using IMatch's annotation copy/paste also saves a lot of work. Copy/Paste. Maybe move around a bit. Done. Next file.

[Lord_Helmchen]

If it can't do facial recognition to the level of suggesting a person in your categories, Excire is of less interest to me. It might still be useful for everything else, but I'd rather pay for a Google or MS cloud service key to get a richer capability to at least get person level recognition. Privacy is always a concern, but no one is requiring connection to the cloud service and every IMatch user could make their own choice.

I do not like cloud services for privacy relevant things. Especially if I would provide other peoples data to a cloud service that is based in a country with very light weight privacy protection. Local tools without uploading my photos (e.g. Picasa) would be ok, but not cloud services at data kraken like Microsoft, Apple, Google, Amazon etc.

So this local solution would be very interesting, even without face recognition.

BTW: I send out my request to Excire site and shortly after my mail, I got a response from project that they are already in contact with Mario. The mail was sent out at 9 p.m. local time.

[Mario]

Keeping the data local is one of the main advantages of Excire. I think they are improving their software all the time, and maybe they will add face recognition at some time.

Using cloud-based services for face recognition is not only relevant to privacy. There is a legal component as well, which is often ignored by people. For example, in Germany, you need to retrieve the explicit (!) agreement of each person in the photo before you upload it. Especially if you upload the image for reasons like identifying people.

I know that Facebook and others have spend a lot of time and money to make "find your friends" and face recognition 'social' and 'fun'. But the sole reason for this is to be able to identify people in images and get their face features on record. This data will be used to identify and track people everywhere at some point. Combine this data with the millions of cameras i public places and you can monitor everybody...Orwell is rotating in his grave...

I really like the cool features provided by AI algorithms and what they can be used for in a DAM environment. But I'm by nature also concerned about privacy for me and of course all IMatch users. I don't sell your data (I could make make some good money from that), and I don't allow ads on my web site or in this community (ads cannot only be annoying, they can also be used to track you).

[Mario]

I have spoken to Excire representatives this afternoon. The will send me some developer libraries to try out within a couple of weeks and then I'll know more about what is possible and how to best make use of it in IMatch.