IMatch 2023 - IM Autotagger film strip?

[jch2103]

I started playing with the new autotagger - pretty neat! I found what appears to be a bug, though. The first several times I tried it, it showed a film strip of the selected images at the top of the app window. But during my last few tests, no images showed up in the film strip, although I had selected 6 images to try out. See attached screenshot.

[Mario]

Nothing has changed regarding the film strip in AutoTagger between the 2021 and 2023 release.
Did you use AutoTagger before?
What happens if you close AutoTagger and reopen it. Does the film strip show?
If so, when does it stop showing, e.g. after which command or operation?
I cannot reproduce this, so the more details you can give me, the better.

Tip: You can run the AutoTagger (while IMatch is also running) in your web browser.
Open this URL: http://127.0.0.1:50519/imatch/apps/auto-tagger/
Press <F12> to open the developer tools in the browser and make sure the "Console" tab is visible.
Then use the AutoTagger normally in the browser.

If the film strip vanishes again, are there any errors logged in the "Console" or "Network" tab in the browser?

[Mario]

Any additional info on this?

[jch2103]

I tried Autotagger within IM about 7-8 times. The first batch were all in one folder and showed the film strip. I closed Autotagger and switched to a different folder, also had success. I again closed Autotagger and tried a third folder (all of these folders had very different kinds of images) and this time no film strip. 17 total AOI requests. 

After that, I tried the web link you posted - it also didn't show the filmstrip. In another session it did, then not in another.  I've attached the log of the current session (after closing Autotagger again but leaving IM running). Maybe it will show something useful...

ps - It seems to think all yellow flowers including sunflowers are daisies. 

[Mario]

Did the console window in the browser's debugger show any warnings when AutoTagger did not show the thumbnails?
I see nothing related in the log file.
I see some warnings about high memory usage of 84% (IMatch reports a peak usage of about 2GB while using the Viewer, which is normal) and a few minor warnings from ExifTool about out-of-sync IPTC digests while indexing files.

Is your sequence to reproduce that opening AT multiple times, with images from different folders and then, at some point, the film strip stops working? How many files have you selected?

I've played with that for 30 minutes, switching between folders, opening the AT, looking up some files, closing the AT, other folder, select files, open AT and repeat. I could not get the thumbnail strip to not show.
This must be an issue with the app and then running it in the web browser while the developer tools are open press <F12> and looking at the "Console" in the developer tools should reveal JavaScript error messages when something is wrong with the thumbnail strip.

ps - It seems to think all yellow flowers including sunflowers are daisies.

That's AI for you.

One could train his/her own model to identify flowers. Starting with a few hundred or better thousand images of various flowers, all tagged with their species and variant. Then train a fresh model from start or use transfer learning to build on top of an existing model to save time and money. All possible and maybe doable for normal persons at some point in the future.
The big AI vision companies do that and they offer e.g. models for apparel, wedding photography or food. There are also some models used by apps which specialize on plants, fungi, birds and stuff.

The "general" model used by IMatch at this time has been trained on 2 million images. But AutoTagger supports different models also for the IMatch AI, so we might see specialized models to chose from at some point in the future.

[jch2103]

Is your sequence to reproduce that opening AT multiple times, with images from different folders and then, at some point, the film strip stops working? How many files have you selected?

That's what I've tried so far (between 1 and 8 images at at time). I'll try your other suggestions later. 

I realize there are lots of advantages and disadvantages of AI in something like the Autotagger. As you say, the training set is crucial. 

[Mario]

I could so far not reproduce this. Not sure if other users have the same issue (no linked bug reports) or if other users have not tried AutoTagger yet.

Any further details much appreciated.
Errors from the Browser console etc., something that gives me a hint.

AutoTagger is around for over two years now, and the thumbnail strip has not changed for IMatch 2023.

Maybe it is a graphic card issue?
AutoTagger makes heavy use of the graphic card's vector units (when compatible) and AutoTagger itself is of courser a browser window and the browser uses the graphic card too.
The graphic card drivers shipped with Windows are often months or more out of date...

[jch2103]

I'll have to dig into this later. I hadn't used the Autotagger before IM 2023, so perhaps why I hadn't run into this before. I'm using the latest Nvidia driver for my almost obsolete GTX 950. 

[Mario]

Any further info on this?
I could so far not reproduce it and not other user reported this problem.

[jch2103]

Still happening. Just now, I selected 4 images for Autotagger, all 4 thumbnails showed up. I closed Autotagger and tried again w/ 2 different images: no thumbnails. Attached is a console log; means nothing to me, but a number of error messages. Not sure if I loaded the console properly. 

[Mario]

These entries don't tell me much and none of them is related to AutoTagger.
Your Chome seems to host some extensions / trackers which become active, trying to connect remote hosts and fail.
DevTools failed just means that you don't have source map files for JavbaScript that is run by the extensions you have installed in Chrome. Which is normal. mss/boq-one-google seems to be something Google hosts, maybe another tracker or widget you have installed.

I see no error messages related to Auto Tagger.

Can you please repeat the test but this time open the "Network" tab?
When you then press <F5> to reload Auto Tagger you will see a large number of requests when Auto Tagger loads resources, connects to IMWS to fetch data.
Click on "Lookup and review...".

The film strip at the top will load some the images.
The Network tab will show entries starting with image? for each image loaded.
If there is a problem, the lines will be shown in red color. And when you click on it, the right side will show the Status Code which is of interest and when you click on the Network tab, you'll see the error message returned by IMWS.

If an image is successfully loaded and you click on it, the Preview tab shows the image returned by IMWS.

See screen shots below.
But, if there are network errors, we would see these messages also in the console. So I wonder...

[jch2103]

After (re-)acquainting myself with the Chrome Console, I tried again with Autotagger. See attached screenshots based on a session with 6 images selected first in IM; no network errors, but no images show in Autotagger although the image is present in the Console (image 2).

How should I proceed? I do see some error messages on the bottom right re IM, but I can't decode them.

[jch2103]

When I ran Autotagger again, I got these messages:

DevTools failed to load source map: Could not load content for http://127.0.0.1:50519/imatch/apps/auto-tagger/lib/tf.min.js.map: HTTP error: status code 404, net::ERR_HTTP_RESPONSE_CODE_FAILURE
DevTools failed to load source map: Could not load content for http://127.0.0.1:50519/imatch/apps/auto-tagger/lib/coco-ssd.min.js.map: HTTP error: status code 404, net::ERR_HTTP_RESPONSE_CODE_FAILURE

[thrinn]

Hi John,
you can safely ignore the "DevTools failed to load source map" messages: they only tell you that Chrome does not find the original source code files on your PC. But this does not have any impact on the functionality of the App. It only means that debugging would be less comfortable because the code is less readable by humans.

[Mario]

The DevTools warnings can be safely ignored.
If there are no red error messages in the network tag, all images were fetched and delivered successfully.
This means that the problem is in the app itself, or in the component that does the film strip.
AutoTagger seems to have processed the image alright, showing keywords.

Have you tried resizing the AutoTagger window?
This enforces a re-layout of everything, including the film strip.

If this does nothing, let's have a look at the HTML generated.




If you click on the button shown below, and then hover the mouse over the AutoTagger app, Chrome will show the HTML under the cursor and when you click an element, it will show the surrounding HTML on the right side.



In this example, I've clicked on the first image on the left. If you don't see images and you click there, we hopefully can see in the HTML code shown on the right why there is no image:



The HTML code shown in the "Elements" tab after clicking into the image. This HTML is what makes the image show.

[jch2103]

Resizing the Autotagger window didn't change anything.

Unfortunately, when no images show in the filmstrip, there are no images to click on, so no imageid data in Elements. See screenshots 5 & 6.

When I closed and reopened IM, running Autotagger showed two of six images selected in IM. See screenshot 7 with one of the two in filmstrip selected. Of course, I couldn't select the missing images.

I did notice one odd thing: the filmstrip only seems to show images without keywords, at least in this particular test. But on the next test, the filmstrip only showed 4 images w/o keywords of 10 selected images (9 of which had no keywords). Strange an inconsistent.

Given this, I don't know how to shown the HTML for the missing images. Is there another way to dup status data from the app?

[jch2103]

Further checking shows that only raw (NEF) files are being shown on the filmstrip, regardless of whether they have keywords!

[thrinn]

This is all very strange. I tried the AutoTagger with different files, (JPG, ARW, ORF, even PDF) - and the thumbnails are always displayed. Including files with and without existing keywords. All worked. So I can not reproduce your issue.

Just as an idea:The AutoTagger allows to select different thumbnail sizes in the settings. Don't know if this could make any difference.

Maybe I overlooked the info, so I will just ask: Are the same files sometimes displayed, sometimes not? Or are only specific files missing? If the latter: Maybe try to recreate the cache images for the missing ones?

[jch2103]

Thanks for the ideas to check. I've tried re-sizing the thumbnails; it doesn't change the problem. 

I've tried reloading the Autotagger to see if different images are shown; doesn't seem to make a difference. I also checked using my laptop instead of my desktop to see if that made a difference; it didn't. 

More investigation: My Nikon NEF files are all shown, but not JPG images derived from those (mostly processed with various versions of DxO PhotoLab). However, I do see JPGs from non-Nikon cameras and straight-from-camera JPGs from my Nikon cameras (I have very few of these). The rest of the Autotagger functionality appears OK; the Autotagger will suggest keywords based on images that aren't in the filmstrip, but it's a handicap not to see the image. 

I've attached a sample Z6 JPG (processed with DxO PhotoLab) that doesn't appear in the Autotagger film strip in case anyone want to check it. 

[axel.hennig]

Hi John,

I haven't fully followed this discussion, just from time to time "a little bit".

Your attached file is shown when I run it (together with another file) in AutoTagger (see screenshot).

[jch2103]

Thanks for checking! Now all I have to do is figure out why I'm not seeing it on either my desktop or my laptop! I don't have any issues with images elsewhere in IMatch or in any other application. 

[Mario]

No problem here either with that file.
I've also tried a mix of 300 Nikon regular and Z series files, both RAW and processed. The film strip always worked.

This is all very strange.
No errors in the console or network tab.
Sometimes works, sometimes not.
App seems to work fine otherwise...

[jch2103]

I agree - very strange, especially that it happens on both my desktop and laptop. The issue is consistent in that it seems that no JPGs developed from my Nikon cameras show in the AutoTagger filmstrip (not just for DxO PhotoLab, but also for Affinity Photo 2 and latest version of Photoshop based on my tests today). JPG images (and MOV files) directly taken with my Nikon cameras do appear in the filmstrip. 

Any suggestions for me to try to resolve this would be much appreciated!

[Mario]

Versions? Proxy images?

[jch2103]

Not in the images I've tried so far. 

[jch2103]

@Mario - given the isolated issues I'm running into (w/ Autotagger app and Imatch Log Reader), does it make sense to uninstall and reinstall IMatch? If so, is there anything else I should do?

[Mario]

I have no idea, frankly.
Maybe close IMatch and delete the folder "C:\ProgramData\photools.com\IMatch6\browser"

If you use a "virus checker" other than Windows Defender, maybe make sure it's not interfering somehow?

[jch2103]

Thanks. Unfortunately, that didn't resolve the issue. I use Windows Defender; haven't noticed any issues with it. 

[jch2103]

Further investigation: 

I uninstalled IM on my laptop (Win 11), restarted it then reinstalled IM. Same issues (except I hadn't loaded latest version of Log Reader - first version worked OK): Autotagger refuses to display thumbnails of JPGs developed from NEFs, although this isn't an issue elsewhere in IM or in any of the photo or other software I have on both my laptop and desktop (incomplete list includes DxO PhotoLap 6, Topaz Sharpen AI, Photoshop, Lightroom, Affinity Photo 1 & 2, etc.). 

Next, I installed IM on my wife's Win 11 laptop (both laptops are MS Surface Books, but hers is a more recent version than mine). Result: Autotagger works properly, showing thumbnails of JPGs and NEF files. 

Any advice on what to do next? My suspicion is software differences, but I don't know where to start. 

[Mario]

The AutoTagger uses the thumbnails stored in the database. The same used e.g. for the File Window.
There are no dependency on anything external or anything.

You can try that out in your browser. If you know the OID of an image, you can use this URL to fetch the thumbnail in your web browser:

http://127.0.0.1:50519/v1/files/image?auth_token=&id=<ID_OF_YOUR_FILE>&imagesize=dbthumbnail&default=true

If you put this variable {File.OID|cast:int} in the VarToy, it shows the OID you can use with the URL.

This is basically what the mini File Window in the AutoTagger does. Or the Pinboard App.

[jch2103]

Thanks.

The thumbnail of a file that doesn't show up in the Autotagger filmstrip does display properly using the expression you provided (see screenshot). So does the thumbnail of a file that does appear in the filmstrip. Both these files also appear in the Pinboard app. So the thumbnails are there and otherwise display in other apps/uses. 

Must be some other problem that causes the app not to display some images.  

[Mario]

That was my guess. Otherwise you would see red errors in the Network tab when you run AutoTagger in your browser.
I currently have no further clue.

[ubacher]

I tried AutoTagger today - no thumbnails showing. This makes it unusable for more than one image at a time.
Win 10.
Running it in the browser it shows one thumbnail only (4 files selected)

Network errors shown:

Code Select Expand

Source map error: Error: NetworkError when attempting to fetch resource.
Resource URL: moz-extension://6163dbea-d3e5-43eb-a134-172a45f1090d/model/static/DOMPurify/purify.min.js
Source Map URL: purify.min.js.map

Source map error: Error: request failed with status 404
Resource URL: [url=http://127.0.0.1:50519/system/moment/moment-with-locales.min.js]http://127.0.0.1:50519/system/moment/moment-with-locales.min.js[/url]
Source Map URL: moment-with-locales.min.js.map

Source map error: Error: request failed with status 404
Resource URL: [url=http://127.0.0.1:50519/imatch/apps/auto-tagger/lib/tf.min.js]http://127.0.0.1:50519/imatch/apps/auto-tagger/lib/tf.min.js[/url]
Source Map URL: tf.min.js.map

Source map error: Error: request failed with status 404
Resource URL: [url=http://127.0.0.1:50519/imatch/apps/auto-tagger/lib/coco-ssd.js]http://127.0.0.1:50519/imatch/apps/auto-tagger/lib/coco-ssd.js[/url]
Source Map URL: coco-ssd.min.js.map


[Mario]

The source map errors are normal. IMatch does not include source maps. You can safely ignore them.
The first one is from a plug-in you run in your FF browser.

[jch2103]

@Ubacher - Thanks for confirming the issue. 

[Mario]

So, that makes two users with this problem.
What is the same? What is different?
If there are no red network errors when IMatch tries to fetch the cover images and the AutoTagger makes keyword suggestions, the images are loaded, just not displayed for some reason.

Does the IMatch Clipboard work for you?
It uses the same component to render thumbnails.

[thrinn]

One cause for pictures not showing up in the thumbnail strip is if they are "protected" by the shield icon and the "Protected files visible" option is unchecked. But in this case, the thumbnails should also be hidden in the file window.

[Mario]

Good point.
Protected files are ignored by the AutoTagger, it does not load them.
This situation is indicated on the home page of the AutoTagger:

[thrinn]

This situation is indicated on the home page of the AutoTagger:

An important information which I totally missed . Because I was completely focused on the tagger page, I did not pay attention to the home screen, just clicked through to "Lookup and Review". I will create a small feature request to make this information a bit more prominent.

[Mario]

For those affected by the "no thumbnail" problem and not working with protected files, please try this:

in the file "c:\ProgramData\photools.com\IMatch6\webroot\imatch\apps\auto-tagger\src\app.js"

change the line

import '/system/vue/vue.min.js'

right at the top into

import '/system/vue/vue.js'

This enables debugging for the Vue component used by IMatch.

Now run AutoTagger in your web browser and look at the Console to see if there are any errors reported when you don't see thumbnails.

Don't forget to change the line back after your test is complete for optimal performance.

[ubacher]

Running with vue in debug mode shows the attached error screenshot:

[Mario]

That's normal. IMatch first tries to find the -us resource and then falls back to the neutral en resource.
But that's not the console tab, it looks more like the network tab.
The error messages of the Vue component will show in the console tab, not in the network tab.

[jch2103]

So, that makes two users with this problem.
What is the same? What is different?
...
Does the IMatch Clipboard work for you?
It uses the same component to render thumbnails.

The IMatch Clipboard works fine. As a test, I picked 6 images, 4 NEF and 2 JPG. All showed correctly in the Clipboard, but only the NEFs were visible for the same 6 images in the Autotagger filmstrip. None of these images are protected.

I'll try your other suggestion next.

[Mario]

but only the NEFs were visible for the same 6 images in the Autotagger filmstrip. None of these images are protected.

Hm, another strange effect.
If you add the same NEF/JPG files to a new database, does the AutoTagger work or fail with that database too?
If it fails (good!) you can send me the 4 NEF and 2 JPG files and I can hopefully reproduce the problem here.
When I can reproduce it, I can fix it.

Maybe the Vue debugging as described above will provide me with the info I need to fix this.

[jch2103]

Here's a screenshot of the debugger info (hopefully the 'right' console data). Hope it helps; if not, please let us know what to try. 

Haven't tried loading images into a new database yet, but this behavior (Autotagger filmstrip shows NEFs but not 'developed' JPGs) seems to be consistent with all my images so far. 

[Mario]

Nothing unusual in the console.
Is this from a situation where the JPG files failed to load?
No clue there, unfortunately.

What happens if you select only the JPG files and start the AutoTagger?

[jch2103]

Nothing unusual in the console.
Is this from a situation where the JPG files failed to load?
No clue there, unfortunately.

What happens if you select only the JPG files and start the AutoTagger?

Yes, the prior screenshot was a situation where the JPGs failed to load. 

The attached screenshot is where only one JPG from the prior set was selected for the Autotagger. 

[ubacher]

Just tried selecting NEF files: They show OK, Tagger works OK
Jpgs fail to show.

[Mario]

And if you only select the JPG files?
I still have no clue where to dig in.
Two users. Two similar problems.
No network errors. Nothing in the console.
I have no issues with using JPEG files or NEF or CR2 or even video files or PDF files.
Where is the connection...?

[thrinn]

Just tried selecting NEF files: They show OK, Tagger works OK
Jpgs fail to show.

All JPGs? Or only some?
It is hard to help here because I haven't been able to reproduce the behaviour. For me, all file types are working fine. I also tried the JPG John attached to his reply #19, but it displays flawlessly here.