Articles regarding AI

[axel.hennig]

Was reading the following (not that up-to-date) articles:

• Building a private, local photo search app using machine learning
• rclip: An AI-Powered Command-Line Photo Search Tool

Just wanted to know if anybody has tried something like this (not necessarily one of the above tools, but one that runs locally not cloud-based). I'm not able to get both of them to work (either because it runs just on Linux (rclip) or my programming skills are not good enough).

[Mario]

Image AI is coming to us. The billions spend by companies like Google and Facebook and Apple and Microsoft to develop technologies to analyze images people are uploading to the web drips down and makes the tech available to normal people. At some point in time.

Don't get your hopes up just now, though.

I play with many of these tools and see if they could be useful to IMatch users. I also do my own experiments.
Working with different training sets, models, different implementations on different platforms...
The failure quote is usually too high to be useful, they don't work outside their trained domains, produce to many random errors, false positives etc.

Keep in mind that the training sets models used by commercial vendors like Google, Clarifai, imagga and Microsoft are massive and run on very expensive hardware. Not because the want to (it's expensive) but because there is no tech that can do the same with the same quality on normal hardware. But things change all the time in AI and the understanding of how things work and how they can be made to work on normal hardware of even smart phones increases all the time. I'm positive.

And even the big AI's and models fail and produce plain nonsense as output.
IMatch supports the Google, Claridai, imagge and Microsofts AI and results vary widely.

For now, if an image classifier has an error rate of only 5% (which would be ranked very good), 5 out of 100 images you let it work on are classified wrong. And you often get things like "image contains a cat" when the image is actually showing a church or car or mountain. You have to check each image and weed out the mistakes manually.

As I wrote in the IMatch help, the IMatch AutoTagger can be a massive help if you sit in front of a heap of unsorted and uncategorized heap of 200,000 or 500,000 images.

For normal folks, it is often easier and quicker to manually assign keywords to files - and to get it right 100% first time.

If I find something that a) works, b) has a license which allows me to use it, c) does not have 500 software patents attached to it and d) actually works, I will be happy to add it to IMatch. For the benefit of all users.

So far, the product marketing is always a lot better than the actual results. Not only for commercial offerings but also for blog posts and tutorials. Let me know when I'm wrong.

[digedag]

For normal folks, it is often easier and quicker to manually assign keywords to files - and to get it right 100% first time.

I tried imagga for quite a while with many different photographs - I can fully confirm Mario's findings!
Bernhard

[monochrome]

For normal folks, it is often easier and quicker to manually assign keywords to files - and to get it right 100% first time.

I think a problem with image labeling is that we don't have any good system for what's called "knowledge management". I'd definitely use an image search engine on my photos if it had a 5% failure rate - but I'd also want to transparently re-index the photos when a better engine came around, and prioritize manually added tags over machine-generated ones, and so on.

Tagging my photos permanently with 5% error rate is a non-starter. (And current offerings from Amazon et al are more like 20% error and 80% missing the point.)

(And I say that as someone who actually rolled my own image labeling app for IMatch back in 2018: https://monochrome.sutic.nu/2018/02/03/imception.html )

[Mario]

As I said, I do my own experiments and are keeping track of AI with potential impact on DAM.
So far, it always ended up with "Yeah, OK. Works, but with too many errors. Not really worth it".

With IMatch AutoTagger, every user can try out how good the image classification AI's from Google, Microsoft, imagga and Clarifai are. For free, for a limited amount of images per month.

IMHO, it makes sense to spend maybe 100 bucks to get at least some sort of organization into a collection of 100,000 or 200,00 files. And save weeks of your time by automatically classifying and tagging your images. Even if 5 or more out of 100 images are classified wrong. The error rate is high and none of the AI's I've tried so far was better than 70% to 80% correct. Which causes a lot of manual clean-up and re-tagging in the end. I don't buy into the marketing. I actually test stuff.

Image classification AI's work OK-ish in limited areas, e.g. Google making some sense of images their bots find on the web or Facebook / Instagram / Twitter learning about images their users post, to support target advertising to these users. Or, via face recognition, learning who knows who, who your friends are, where you travel to, which car or furniture you have etc. Again, for target advertising, data sales and selling you stuff.

One possible way I see for IMatch would be to allow users to train their AI themselves, for the type of images they create. Whether you mostly do personal photography, stock, studio or maybe you're a scientist working with images or biological species, animals, machine parts, planes, cars or whatnot.

Basically users setting up a category hierarchy of tagged images (images with keywords) to train the AI and then let it run on all their other images to automatically tag them. This works, somewhat, but the error rate is still very high. Users can easily train the wrong images and mess everything up. This is actually science and requires a lot of trial and error.

When I find an actually working solution, IMatch users will find out soon.

If you have found something that works well and is not behind a paywall or dozens of software patents, let me know. I' listening and I'm open to anything that helps IMatch users.

[monochrome]

One possible way I see for IMatch would be to allow users to train their AI themselves, for the type of images they create

Hey that would actually be cool

[Mario]

One possible way I see for IMatch would be to allow users to train their AI themselves, for the type of images they create

Hey that would actually be cool

You can do that today already. Microsoft, Clarifai and imagga allow you to train your own models.
But it is not easy to train models, unless you train something for a very specific and tiny area of expertise.
There are models for almost everything out there, from insects to weddings to food to clothes to mushrooms.