Telemetry in Future IMatch Versions

[Mario]

As recently announced future versions of IMatch include a telemetry feature.

This feature optionally records which IMatch features you use and some basic information about your database and computer. This data is completely anonymous and not linked to your license or community user account. When IMatch closes, it transmits the telemetry data to photools.com for analysis and statistics.

Telemetry is optional. You will be asked if you allow IMatch to collect and transmit telemetry data. Please do!

I want to learn about how IMatch users work with the software. Which features are used most often and which features are not used at all. What the common screen resolutions are or how many files IMatch users manage on the average.

Please see this page for details about telemetry and the how and why:

https://www.photools.com/application-telemetry/

This is a new page. Let me know if you find it unclear or you spot a typo.

Note: I did not use a 3rd party software to add telemetry to IMatch. The data is send straight from IMatch via SSL (HTTPS) to photools.com and only used by me.

[JohnZeman]

I will certainly enable it.

[Mario]

Thanks! Much appreciated.

I hope that most users will allow telemetry. The more users help, the better the results are.

[Jingo]

Same here.. it will be on and I hope it helps you create an even better product.

Found one item to change in the article:

old: Sadly. we don't know much about how user work with IMatch or our other products.
new: Sadly. we don't know much about how users work with IMatch or our other products.

[jch2103]

I'm in also. Should be very useful information.

One observation: A complex product like IMatch has many possible functions, some of which are used quite infrequently (i.e., IMatch has a 'long tail'). The more people who participate, the more likely there will be statistically useful information about these lesser-used functions. Even so, picking this out of the statistical noise could be challenging ('Is this a function worth keeping?'). At the beginning of this triage process, though, I'm sure there will be lots of low-hanging fruit for improvements.

[Mario]

@Jingo: Thanks. Typo fixed.

The more people who participate, the more likely there will be statistically useful information

Yes. The more users participate, the better.
Also, data will get better over months of telemetry data (some features are needed only occasionally, but are still important).

I will learn about the data from the data. Maybe I need to tweak things, add more detailed telemetry for some areas.
But after running this with a few test users for a week I guess it will provide a good picture. This will tech me some things!

[mastodon]

I will add you as much telemetry data as you need. I think it is a need for developing IMatch.

[hro]

To me, telemetry or user feedback features are an issue of trust. Do I trust the supplier who asks for them. Mostly I don't trust a supplier and will not allow them to use my data. This is unfortunate but is the world we live in.

With Imatch this is different. I am a user for many years now and like the way Mario engages with his customer base. I appreciate his frankness and openness. Happy to share my data with him.

Cheers. Hartmut.

[Mario]

Happy to share my data with him.

Thanks!

But to make this clear: You are not sharing your data.

My server just collects the telemetry data produced by IMatch instances running somewhere, and then combines this data with the data from all other users to produce statistics.
I don't need nor want to identify individual users. Telemetry does not care if the data comes from Sean from Ireland or Hideki from Japan.

[hro]

Thanks to make this clear. Appreciated.

[ubacher]

One thing one needs to keep in mind when analyzing usage statistics: Some users may have used some
function and found it difficult, or could not get it to work, or did not understand how it works - and thus stop using the function.
This could then give the impression that a function is not much used when in reality it would need to be questioned why it is not used.

And then there are us old-time users. Many of us, I am sure, still do things the way we did it in IM3 and we never did pick up the
much easier/better ways of doing things now. If Mario does share the results then many of us might get triggered into looking into
apparently much used function which are unused by us.

[Mario]

Some users may have used some function and found it difficult, or could not get it to work, or did not understand how it works

If Mario does share the results then many of us might get triggered into looking into apparently much used function which are unused by us.

These are some of the key reasons for adding telemetry to IMatch.

I currently don't know much about how IMatch users use it in the wild.
I learn from this community and the testing groups. And from commercial clients.
But that is only a fraction of the total user base. Most users are silent and only contact me when they run into a bug or problem.

As I wrote on the telemetry explain page if we learn that some functions / features / panels are not used by many, we can start to dig deeper, e.g. asking users "why" via a newsletter or here in the community.

If a feature is not used because it is too hard or users just don't know for what it's good for, I can create a new help topic, knowledge-base article or video tutorial.
If a feature is not used because nobody needs it, I will remove it and reduce development complexity and application footprint.
Every feature costs 'money' in terms of maintaining it, keeping it working between versions, upgrading 3rd party toolkits etc.

We first need to get some data to work with.
I may need to adust the telemetry for a couple of releases to get useful results, but I hope to learn a lot from it.

I have learned a lot already since the IMatch help system is available online.
I now for the first time have statistics about the pages which are visited most and, which search terms users use etc.
That helped me to improve the help system considerably.

[Aubrey]

+1 I'm also in!
Perhaps when features not being used are identified, you might pose the question why?
Not useful
Too difficult to learn and apply
Need training
etc... (trying to think of other reasons!)

Aubrey.

[Mario]

Perhaps when features not being used are identified, you might pose the question why?
Not useful
Too difficult to learn and apply
Need training
etc... (trying to think of other reasons!)

Aubrey.

That would fall under the "improve telemetry over time".
Currently the locally cached telemetry data is deleted once it has been sent. This is for privacy reasons.

To implement this kind of questions I would need to retain telemetry data on the users PC for a couple of months. And then poke occasionally at it to find if features marked as relevant have been used or not. But asking users such things would open yet another can of worms, may raise privacy issues etc. I need to have a deep think first.

Another way would be to present the findings of the telemetry after some months on the photools.com web site, and make IMatch tell the user about this page.
Rarely used but 'assumedly' useful features could be highlighted, and links to the help or a tutorial could be provided. This could probably bring the features to the attention of more users.
We could also let users vote to remove a feature etc.
Something simple like that.

Engaging users is really awfully complicated these days. So many web sites, social media gadgets and streaming services compete for the attention of people...

[thrinn]

+1
I will also participate. As already discussed before, I trust Mario to handle the data gathered this way responsibly and also to inform his users openly should more than anonymous statistics be collected in the future.

[sinus]

I trust Mario also! And will participate.

As written in this thread ( https://www.photools.com/community/index.php?topic=8225.msg57816#msg57816 ), I find this interesting not only for Mario, but also for the users.
If only a part of the results of course.

If a user could compare some data (size DB, how many categories have others, speed loading, search etc) with other users, I am still convinced that this would be a good thing for users and for Mario.

1) It would give the user a good feeling.
2) It would avoid some posts with questions like "is that normal..." and so on.

Only my thoughts, means not, they are good. 

[BanjoTom]

+1  --  Count me in, too.  I've come to trust Mario, and think this collection of telemetry data can only help a great product become even better, for ALL users. 

[Mario]

+1  --  Count me in, too.  I've come to trust Mario, and think this collection of telemetry data can only help a great product become even better, for ALL users.

Thanks. I hope hat the anonymous (!) telemetry data gathered will be useful.
Over the past days I've fleshed out the prototype version, adding more elements to the tracking.
For example, the maximum and average cache image creation and load time, the max/avg time for re-calculating data-driven categories.
These are critical metrics, because they directly impact how 'fast' users experience IMatch.

I plan to start with 'too many' metrics and then reduce this to the truly useful set of metrics over a couple of IMatch releases.
Logging too much will flood the database (not good) and logging too little will limit the statistics and the overall usefulness (also not good).
I also have no idea whether 5% or 50% of the user base will allow telemetry. We'll see.
This will be a 'learn as you go' project.

IMatch 2018 is a rock-solid and super-performant platform for future developments. Telemetry should tell me where to go.
I have also some big plans (and some prototypes!) for the IMWS / Server side. But it's too early to talk about that.

[ben]

+1 Count me in as well.
In 99% of the cases, I disable these functions.
Mario is one of the few I trust.

[Jingo]

Mario is one of the few I trust.

Agree 100%... BTW: Just realized yesterday that Win10 is sending info by default as part of their "Customer Experience Improvement Program" which appears to be turned ON by default in Win10.  I had to modify a group policy in order to shut it off and prevent M$oft from retrieving it... Amazing!

[ColinIM]

+1 Count me in as well.
In 99% of the cases, I disable these functions.
Mario is one of the few I trust.

I agree with all three lines in ben's post!

Yes, I will be happy to enable this IMatch metadata option.

Here are some notes about my use of IMatch before I ask a question below:

• I run two instances of IMatch on two different computers, but I also maintain different databases (with different photo sets) on those computers.
• On my 'main' computer (with its newer / better hardware) I maintain my 'live' database with my full photo collection of over 70,000 images - therefore the IMatch metadata taken from this computer would properly represent my normal use of IMatch.
• On my older computer I use IMatch only for testing and experimenting with IMatch's many features and options. On this older computer I keep multiple, small 'test' databases, and in one database (for example) I might sometimes leave a Layout arrangement unfinished, or leave some other parts of a database 'untidy' for a while.
• Clearly the IMatch metadata taken from this older computer would not represent my normal use of IMatch.

So, my question is:  On that older computer should I turn off the option to gather and send back IMatch metadata?

I would be happy to keep 'metadata' enabled on both computers, but I suspect that my older computer's metadata would only be unwelcome and unrepresentative 'noise' among the bigger metadata picture - so to speak!

Looking forward to future improvements on this already superb DAM product, based on its real-life use from (hopefully) almost 100% Imatch user participation 

[Mario]

Just keep it enabled. I can tell from the platform/cpu codes in the telemetry if this is a slow or fast computer.

[ColinIM]

Just keep it enabled. I can tell from the platform/cpu codes in the telemetry if this is a slow or fast computer.

Thank you Mario ...

... but - perhaps I shouldn't have labelled these computers as the 'slow' one and the 'faster' one because the reason for my doubt about keeping it enabled on both computers was not about their relative speed or performance.

I was concerned that my pattern of use of IMatch on that other computer was not at all representative of my actual use of IMatch for about 90% of my time spent on IMatch DAM activities.

So now (I think) ... if this 'occasionally used' IMatch computer is really just my test platform for my IMatch activities, then I should simply disable the metadata option on it, and therefore prevent its non-realistic IMatch metadata from skewing your statistics?

I'm sorry to have opened this statistics 'Pandora's Box'     and I'm probably thinking too hard on your behalf  

I am actually very confident that your stat's-gathering scheme will cope perfectly well with a certain degree of 'noise' and some untidy data from a proportion of your many IMatch users!

[sinus]

Colin
I understand your "concern" very well, that honors you and many do not waste any thoughts on it. Among those who don't think like that, probably I belong rather.

(Except that I just think that some of these data might be interesting for some users to see how your IMatch compares to others and might change something.)

If I were you, I'd let both computers query. Because even if you use your test computer "strangely", it doesn't matter. There are also users who have a single computer to use IMatch and still use the software "strangely". 

[Mario]

I was concerned that my pattern of use of IMatch on that other computer was not at all representative of my actual use of IMatch for about 90% of my time spent on IMatch DAM activities.

That's OK and will probably not be even a blib somewhere in the stats.

[ColinIM]

Thank you Markus (sinus) and thank you Mario.

(....) There are also users who have a single computer to use IMatch and still use the software "strangely". 

Ha Ha! Probably very true