Filter Panel - Filtering Descriptions

[JohnZeman]

Mario this one has me stumped.

It's been awhile but I used to use this filter to find images in the current file window that have identical descriptions.  And in the past it has always worked fine.

But now it always returns "There are too many elements to display in the Value Filter for this tag", even when there are only a few images (like 6) in the file window.

If I change the Description tag to something else like State/Province or Location then the filter works fine but it fails every time with Descriptions.

Any ideas?

[Mario]

This filter starts by considering all files in the database and then applies the scope.
If there are more than 10,000 different values for the tag in the database to start with, it aborts with the error message. To avoid excessive resource usage.
I think I've remember looking into this / making a change for 2021, but I don't remember right now. Maybe I've just upped the limit, or used one of the new database system functions to apply the scope before fetching. No time right now to look into the details.

[Mario]

I remembered: This is a 10,000 different values for the same tag in the database limit.
IMatch does a quick check before starting the slow op to retrieve all values and then weed out the values not in the current scope.

Since all this is based on internal caches to make the Filter Panel work fast, it cannot be (easily) changed to consider the files in the current scope - or else the cache would need not only store the values, but also the scope used for these values - which makes the cache very large and slow to use.

Tags like file name or description, which produce more than 10,000 different values  are not suited for this specific filter.
Maybe, with today's fast computers and SSD storage, I can raise that limit. Need to make some tests over the weekend.

[JohnZeman]

Thanks Mario, I hadn't thought of the possibility that IMatch considers all files in the database instead of just the ones in the current file window.

That could explain my problem though.  Because I'm sure I have more differing descriptions in my 126,000+ photo database than I have differing anything else (like Locations, Province/State, and so on).  Hopefully you'll find a way to increase the limit in IMatch 2021 because it is a very useful filter for me.

Thanks again.

[Mario]

The logic in IMatch is to load the unique values of the tag on first request and keep them in memory. The Filter Panel is updated and re-applied very often and the in-memory caching takes load of the database.
To apply the filter, IMatch uses the in-memory cache, applies the scope and - done.

When I wrote that code for IMatch 5, 10,000 unique values were a lot (still are) unless you use tags which basically produce unique values for each file in the database, like file name, create date etc.
I will have a look over the weekend. This is probably something I can improve utilizing one of enhancements made to the database system over the past year. I have optimized other parts using that new tech for IMatch 2021 already.

[thrinn]

John,
as an idea for a workaround: What about creating a Sort Profile on the Description and combining it with a tabular layout where the description is shown? This should give you one line per picture, and because pictures with the same description are displayed under each other, it should be not too difficult to spot visually if they are identical. At least if you descriptions are not too long.

[JohnZeman]

Thanks Mario and Thorsten.

I haven't tried your suggestion yet Thorsten but in the mean time I've figured out another way I can still achieve what I want by creating a Descriptions chart in the Statistics panel.

Doing that allows me to group the descriptions of the selected images in a pie chart.  Then I can hover my mouse pointer over each section of the pie chart to find the description I want.

And when I find what I'm looking for right clicking that section of the pie chart lets me select the files with identical descriptions.

[thrinn]

That's also a very clever idea! And much easier than creating a sort profile and file layout!

[JohnZeman]

John,
as an idea for a workaround: What about creating a Sort Profile on the Description and combining it with a tabular layout where the description is shown? This should give you one line per picture, and because pictures with the same description are displayed under each other, it should be not too difficult to spot visually if they are identical. At least if you descriptions are not too long.

Thorsten I just tried your suggestion and that works too!

That's one of the many things I love about IMatch, there are so many ways we can do what we want.