Importing data into attributes without adding new rows

[ccs]

It would be really helpful to be able to import new data or replace existing data just for the specified fields versus adding a new row with just new values (or replacing all data except for the newly imported fields if replace is specified).  I saw some discussion of this topic in 2023, but didn't see a subsequent feature request.  For us, this is important to our use of the program.

We have been using Imatch for a very long time (maybe two decades now?).  Attributes (formerly properties) have always been a critical feature for us and are central to our workflow. We code aspects of each image and never want more than one row per image but routinely have over 30 columns.  We regularly import and export data as csv files between those fields and external database programs for analysis. We also use the attribute importer app to pull other types of data into the attributes for analysis.  It has taken us way too long to fully to transition away from version 3.6 because that version's data management was exactly what we needed.  We are doing our best to stick with 2023, but attributes are now much harder for us to enter and manage.  It would be extremely helpful to have the option to import data into the attributes requiring addition/update without adding a new row or deleting all of the other attribute values in that row.

Perhaps it should be a separate request, but it would also be helpful to easily tell how many attribute rows exist per image (without looking at each image individually, which is impossible in large databases), to automatically add a blank row for a newly added image if there are attributes field defined, and/or to be able to ensure that there is only one attribute record per image (for those who do not want to inadvertently add more).

I hope this makes sense.  Thanks in advance for considering these feature requests.

[Mario]

Attributes are one of the least used features in IMatch.
I don't even have telemetry data, which means that none of the users participating in telemetry has even one Attribute Set.
This gives them a quite low priority in my book.

If Attributes are such an integral and important part of your workflow, have you considered writing a PowerShell or Python script or an IMatch app that exports and imports data in exactly the way you want?
Any programming language that can work with REST APIs works with IMatch (Python, Java, JavaScript, C++, C#, Rust, PowerShell, curl etc. Even Excel Basic!).

The v1/attributes/ set of IMWS endpoints are fully documented and you can add, update and delete Attribute values and Sets.
IMatch ships with a sample app named "Attributes" that shows how to create and delete Sets and modify Attribute data.
To see it in the App Manager, enable developer mode in Edit > Preferences > Application.

[thrinn]

I don't even have telemetry data, which means that none of the users participating in telemetry has even one Attribute Set.

That is strange: I at least use Attributes in my two databases (one for photos and related stuff, one for documents), and I always have Telemetry enabled. So you should see "something" at least. Or did you mean that there is no metric for attributes?

[Mario]

You're right. My ad-hoc query was wrong. I have telemetry data
There are about 250 users with more than 1,000 attribute records.

[sinus]

Yep, and one of these 250 users is me 

But I must say, I mostly use it with a script, years ago, where for sure Thorsten helped me, and I believe, Jingo, too. 
Because I fînd it a bit akward to use it, for me no surprise, that only a few users use it. 

[Mario]

@sinus

If you want a change in the Attributes Panels, feel free to add a feature request, describing what you want to have changed, why and how.

[Jingo]

If you want to go the custom app route, take a look at the app I created a few years ago: Attribute Finda nd Translate: https://www.photools.com/community/index.php/topic,9402.0.html

It's been awhile, but I believe this lets you load a file (translations in this case) and set specific columns of data from those values... shouldn't be that hard to modify it to allow you to load a file, search for a specific key and then only set the columns from that data row.

Enjoy!

[ccs]

I wonder also if those people who used attributes intensively were, like us, very slow to move forward with new versions of Imatch.  If so, then they perhaps are not contributing to telemetry data, like I think we were not, even though they are very loyal and appreciative customers (purchasing the new versions, but still actually using 3.6). 

Thank you for your suggestions, which I will look into.  I would greatly appreciate any help from other attribute users who have already made strides in this area.

Finally, just a vote for attributes to remain an integral part of imatch.  It is one of the (many) features that makes this program stand apart from other things on the market (especially for science users). 

Thank you.

[ccs]

Thank you, Jingo.  I will have a look at your app!

[Mario]

Finally, just a vote for attributes to remain an integral part of imatch.  It is one of the (many) features that makes this program stand apart from other things on the market (especially for science users). 

Attributes will stay. It's just so that after introducing this feature with IMatch 5 in 2016 (!), it basically never changed. It just works well for most users, I believe.

I doubt many users have the need to constantly import and export Attributes.
And if they do, there is usually more involved, like an external database or data managed in Excel. And in such cases, writing an Excel script or Python/PowerShell script that does all the work is usually the way to go.
It can do all the tricks you need, including merging new data into existing rows. Or add empty rows to files without Attributes.

If you need a user interface, writing an IMatch app in HTML and JavaScript is an option.
The CSV Importer app, for example, is an IMatch app written in JavaScript that can optionally import into Metadata or Attributes. It can serve as ab example how to import into Attributes.

It's not hard if you have somebody who has some programming experience. And since IMatch IMWS is language-agnostic, you can use any programming language you like.

[ccs]

Thank you, Mario

[Mario]

"Working with Attributes" on the recipes page is a good starter: https://www.photools.com/dev-center/doc/imatch/tutorial-recipes.html

[thrinn]

I wonder also if those people who used attributes intensively were, like us, very slow to move forward with new versions of Imatch.

I do use attributes, but mainly through my own app. And I don't have to export/import attributes on a regular basis, so my usecase is a bit different. But I moved to every new IMatch version as soon as it was available.

[thrinn]

Perhaps it should be a separate request, but it would also be helpful to easily tell how many attribute rows exist per image (without looking at each image individually, which is impossible in large databases)

This part should be already doable using variables and a data driven category. For example, let's say I have an Attribute Set called Dokumentinfo which contains an attribute DocID which I know is always filled (this last part is important). Then,

Code Select Expand

{File.AT.Dokumentinfo.DocID|count:true}counts the number of attribute records.

You can now create a data-driven category using this variable expression like this:
You cannot view this attachment.

Each level of the category represents a grouping on the "number of attribute records", the number in brackets is the number of corresponding files. For example, in my test database, I have 1561 files without any attribute records, 5 files with 1 record, and 1 file with two records.
You cannot view this attachment.

Just make sure to choose an attribute which is always filled. The count function will count records without a value in the corresponding attribute. Also, for performance reasons, I would switch off "Automatic update" and do the refresh manually as needed.

[Mario]

Neat. That's the way I'd do it too.

[ccs]

Thank you, Thrinn!  That's a good idea.