Excire Foto

[plastikman]

German company Excire just launched a new product: Excire Foto. Mario should definitely take a look at it. I am testing their trial now and find it very interesting and promising. They were already known for their plug-ins, and now they launched a stand-alone product.

It highlights what I already told Mario I would like to see: simple local machine vision based on a small set of criteria, all-stored separate from metadata until you decide to apply it.

Some of the useful things it tags: smiling or not, eyes open or closed, approximate age, portrait, if there is bokeh in the shot, how many persons are in the shot, frontal or side of a face etc. This to me is extremely useful information to cull/sort my images. These types of tags are NOT the focus of Google/Microsoft and they barely show up if you run their machine vision.

Accuracy is not a 100% and should be a bit better but I am sure this could be setup to only tags essential things (e.g. no objects, species etc. which are more prone to mistakes).

I hope to see this type of functionality in IMatch 2021.

[Mario]

I'm aware of Excire Photo 

I have worked with Excire in the past, while analyzing available computer vision technology for IMatch. See Computer Vision and Machine Learning in IMatch for more information. Unfortunately, I was not able to publish the results produced with Excire image analysis technology.

At the end of my evaluation, I've decided to not integrate Excire technology and others, like Amazon computer vision, into IMatch. I cannot say more about Excire, because I've signed a NDA at the time and I respect that.

The IMatch AutoTagger app, which supports all leading computer vision AIs (Google, Microsoft, Clarifai and imagga) is included for free with IMatch 2020. Keywords, categories and even AI-based location and GPS assignments for known landmarks and places, everything can be done with AutoTagger. For some services, even multiple specialized models or even custom models for specific image domains (Wedding, Landmark, Food, Apparel, Machinery, Chemistry, Biology ...) are supported. This is the top-end of what is possible today. The same technology available in high-priced corporate DAM systems - right there in your IMatch 2020.

AutoTagger allows you to automatically organize your files based on their visual content, landmarks, known places and the resulting keywords. It is fully integrated into IMatch, including the IMatch Universal Thesaurus, @Keywords, controlled vocabularies and more. Powerful features for mapping, bridging and cleaning keywords are available. For all supported (over 100) file formats. And with top-notch metadata processing as we know from IMatch and ExifTool.

AutoTagger has been designed to be extensible and if other powerful AI services become available for the public, I can easily integrate them. The source code for AutoTagger is included in IMatch 2020 - which makes it possible for interested users and especially corporate and institutional users to add additional services and to extend it for their needs - which is already done in several projects.

There are indeed many other DAM systems on the market. Excire Photo has just been announced and is one of them. Welcome!
Many of the current top-line DAMs integrate AI-based technologies of sorts. Original vendor or 3rd party, running locally, on-premise or in the cloud.

If you are looking for an alternative for IMatch, check out the comprehensive market overview at Capterra. No affiliation.
From Mylio to ACDSee to Photo Supreme to DigiKam (free) to DAMNation to Canto, Extensis, FotoWare, AssetBank, Bynder, Widen, ... Excire Photo . I'm sure you can find a DAM that fits your needs.

[plastikman]

I am very happy with IMatch Mario, just sharing this product and what I find interesting about it. And this product is a DAM light, light, light from my first impression. I was just curious about the machine vision.

I just would like to have some useful grouping using local machine vision, saved in an index that is separate from the keywords metadata field... I recommended this before. Too aid local search and discoverability similar to how "find similar pictures already works". E.g. detect landscapes, portraits, dominant color, one-two-three faces, bokeh, some composition elements.

Again, no criticism just constructive feedback in relation to an industry announcement.

p.s. my first expression is that their machine vision is terribly inconsistent and inaccurate compared to other options I tried.

[sinus]

Thanks for these interesting stuff to read.

Well, about this "tagging" like smiline, eyes open, kneeling and so on, every user has a unique desire.

And it depends, do you start with a DAM, or do you have already a big Database in IMatch.
Are your files family or business or whatever.

In my case I have quite a big DB, family and business included (separated by some personal things like a "tag" in the filename and dots.

From the beginning I have tagged the personal photos with names and the amount of photos (2,3...).
I have even stored, is it a portrait, only head, or whole person. But after some times I decided against this, because it was simply not necessary.

To let do this a AI is interesting. And fascinating.
But like I have read here and in other "recensions" about AI for people, I am not sure, if finally it does really help to save time, at leas for me.

If I have family photos, I see e.g. a group of 5 people. In the background there is aunt Jane. But she is quite fare away and she is a bit unsharp and looks to the side. She will not like this image. Now a AI would add 5, I guess.
But I would add 4, because it makes more sense for me. I am sure, you can imagine other situations, where you would decicde something else as the AI does.

And so on. There are a lot of decisions, what an AI has to do ... or I have to do it.
For family stuff I like it more, if I do this myself.

And the biggest point for me is, that I have already tagged my family photos over the years, I have not troubles to find a photo from aunt Jane with child Phil, where they only these two are on the photo (2 persons).
The same with the age.
With help of the various possibilities in IMatch since years I am tagging family members with the birthdate and the actual age, when the photo is taken, simply by one click. And I can see this under each thumb and in other outputs (Design&Print, html...).

This is cool stuff for me ... and AI from IMatch would do this also, but automatically, but you have to check the results.

That is why I do not use now AI for people and the same is true for events.

But these stuff will be better and better and in the future, maybe I will turn also on this AI-stuff.
Because it is fascinating.

And Mario did it integrate into IMatch very good.

[plastikman]

Markus, great input.

I agree with your assessment.

AI will never be 100% accurate versus human (if alone because every human is different and this thing called "vision" is not the same because we all have different backgrounds and beliefs).

But, and I repeat myself, if machine vision is used as an aid, for quick search results, without the user having to put 1 minute of effort in establishing that database, I believe it can be of great value. Even when it is not 70% accurate, it still helps you find things without much effort. But you don't want those faulty tags in your metadata.

So in my view:
- there is the absolute source of truth called metadata (verified by user)
- there is an approximate source of truth, getting better when technology matures, that is a handy aid in finding pictures, without the need for excessive tagging on the user behalf

Again,

I am talking about things like 1,2,3,4 faces in image
Landscape or portrait
Wide shot or a portrait
Strong lines in image, no lines in image (depth vs flat)
Smiling or not smiling
Eyes open or eyes closed
Dominant color

This set of criteria is very useful.

With regards to your faces + aunt example.... I do the same. I use face recognition for people, and if there are faces that are not relevant to the photo, I delete them even if the recogition is right.

So if Mario would count the identified faces, the count should be right and better than what you get in Excire Foto. Actually, this can already be achieved with formulas (and the count value), but that is for nerds like me and not for the average joe who I think this feature is valuable for.

[plastikman]

The IMatch AutoTagger app, which supports all leading computer vision AIs (Google, Microsoft, Clarifai and imagga) is included for free with IMatch 2020. Keywords, categories and even AI-based location and GPS assignments for known landmarks and places, everything can be done with AutoTagger. For some services, even multiple specialized models or even custom models for specific image domains (Wedding, Landmark, Food, Apparel, Machinery, Chemistry, Biology ...) are supported. This is the top-end of what is possible today. The same technology available in high-priced corporate DAM systems - right there in your IMatch 2020.

Which service offers the specialization? BTW, I never got Microsoft Azure to work with my API key. Kept throwing errors. Probably did something wrong on my side. Also: I think Azure has functionality to "describe images" e.g. "red car in front of a building". Any chance you can incorporate that in the future? I find it very fascinating.

Anyway, IMatch is the DAM product I recommend everywhere DAM questions pop-up (Dutch forum, DPReview, reddit) with Photo Supreme as an alternative for Mac users. So don't worry about me, I am very happy

[Mario]

Double-check your Azure setup. AutoTagger should log all errors received from Azure in the IMatch log.
I have considered the describe feature in Azure, but I found it so inaccurate, often misleading or plain funny, that it would produce a lot of work afterwards to review each description. And it doubles the cost per image, so users would pay double for having extra work.

Clarifai and imagga and Microsoft allow you to train your own models. Google too.
That's of course pretty advanced and not aimed at normal people. More at companies which need to handle specific types of images.

Carifai offers custom training and multiple pre-made models for different domains (wedding, food, ...) which you can select in AutoTagger.

I am talking about things like 1,2,3,4 faces in image
Landscape or portrait
Wide shot or a portrait
Strong lines in image, no lines in image (depth vs flat)
Smiling or not smiling
Eyes open or eyes closed
Dominant color

The number of faces in an image is available via variables so you can use it to create a data-driven category to automatically organize your images based on the number of faces shown.

Landscape or portrait: Similar.

Dominant colors can be used in the Filter panel to find files.
I could make this available via a variable but there was never a request for this. I doubt many people need to organize their images by things like "mostly red" constantly.

Similar, your "eyes closed" or "smiling" requirements: I'm not sure how many people really need to organize their images based on this - or would be willing to pay for it (AI services charge extra for this kind of analysis per image) ...

All this can already be done so precisely (100% correct) and quickly during a single pass review...

Keywords for shot type, motive type, eye state, smile, whatever.
Favorites for each keyword (or even combinations of frequently used combinations of keywords like "portrait, eyes open, smiling").
Then you can, with a few mouse clicks or key presses, add these keywords to your files. 100% correct. And once done, persistent in the metadata, forever.

Even it you produce 300 keepers from a one day 2,000 shots session, adding keywords to these 300 files takes only 10, maybe 20 minutes. Even if you do it every day, it's virtually no time compared to taking the shots or the entire overhead of studio, traveling or editing.

AutoTagger and similar services can be a massive help when you start from scratch with 100,000 files. Or you are a company or institute which faces the challenge to get some order into an image collection spreading over 50 years and 19 hard disks. It produces enough order to allow you to dig in and review the files over time to correct the wrong keywords and add more.

But for a hobby or pro photographer, it's often a gimmick or a nice-to-have. Because, especially given the tagging features in IMatch, you can achieve a lot by hand, and quickly. And always correct. None of the AIs always assigns the correct keywords and frequently misses keywords. So you have to review the results anyway at some point in time.

If you rely only on automatic organization of AI, you may find that you don't find many images ("faster than ever") because the AI failed and sorted your Hawaii images into your Clothes folder 

[sinus]

Again,

I am talking about things like 1,2,3,4 faces in image
Landscape or portrait
Wide shot or a portrait
Strong lines in image, no lines in image (depth vs flat)
Smiling or not smiling
Eyes open or eyes closed
Dominant color

This set of criteria is very useful.

Thanks, plastikman

I agree with you.

BTW, Landscape/portrait should be also in the exif or xmp.

Dominant color: maybe 1 or 2 years I did also mention this in a tag inside IMatch.
But then I did not more, because it was for me not necessary and then came the search with a kind of AI and the color-seach in IMatch and this was better. 

Of course, you are right, the more info the better. Although sometimes it is simply too much info, and a lot, what is not necessary.
If I look at the Exif-xmp-iptd and whatelse is in ONE image, it is simply too much.

I mean, in the analog-times you had basically nothing. If you wanted to remember only the date-time, not to speak from other stuff, you had to write it somewhere (best on a paper and take a shot before the event was a good thing  ).

But nowadays we have well, a lot, really a lot of information and yes, even now sometimes the information, what we would like to have, we have not.