Write-back Metadata to JPG file

[jmhdassen]

I am having a lot of problems with write-back action modifying data in database.
I already abandoned XMP sidecars (which I would prefer, because they cause unexpected interference). I have set the Metadata2 JPG properties to default, but still have some problems.
I pay a lot of attention to metadata, especially location data. The main reason I use IMatch is because its Thesaurus makes it easy to get consistent metadata.
But I can not get my head around the write-back to JPG. I know about the MWG stuff, but I think the problem I have is not related to that.
See 4 screenshots attached for sequence of events:
1. I have data in the database that I want to write-back to image. According to the pen icon only Keywords and Subject need to be written. I press the pen.
2. This seems to have worked, but the Darwin Core Location data has disappeared from the database.
3. I now manually reenter the the DWC data and the pen shows it is pending for write-back
4. After write-back this time, this things seem to be OK.

But why do I need to manually reenter the DWC data ?? I am sure it was in the database since I repeatedly closed and reopened IMatch.
I must add that the DWC tags (and some others) are updated by an external program, using the the IMWS interface, specifically the '/v1/metadata' POST API.
This IMWS interface seems to work beautifully. The results of this "stick' in the database.
I suspect however that these programmatically updated tags are wiped out by the write-back. Why??

I have some 50,000 images to be updated like this. Doing it manually is not feasible.

Regards, Jozef Dassen

[Mario]

This seems to be a rather complicated workflow.
Write-back is a simple process. All metadata tags marked as modified (pen icon set in the Metadata Panel) are written, then IMatch runs some ExifTool scripts to map XMP back to EXIF, GPS and legacy IPTC.
JPEG files must always use embedded XMP data (that's in the XMP standard and also industry standard). Why did you want to use XMP files? This will cause many problems.

Which tags IMatch writes can be seen in the ExifTool output Panel (View menu > Panels > Output Panel...).
Do the tags you have modified in your custom app appear there? Do they show as modified in the Metadata Panel?
What happens if you change some Dublin Core tags in the Metadata Panel and write back?

[jmhdassen]

Ah, ok. So if you compare screenshot 1 with screenshot 3 then you can see that the tags I have updated via the IMWS API do not have the pen icon set. I checked for several other images and it is the same.

So the question reduces to: why do tags updated with v1/metadata POST not get the pen icon??
I have been using the simple task 'set' command to send to v1/metadata POST and it obviously wrote correctly to the database.
But I guess it fails to trigger something that manual entry triggers.

                task = {}
                task['id'] = id
                task['tag'] =  'DarwinCore::Main\\dctermsLocationCounty\\DCCounty\\0'
                task['op'] = 'set'
                task['value'] = iptc_city
                task_list.append(task)

The reasons for my workflow are many:
- Linux is my main platform and has been since before PC existed (Unix back then).
- I am using Capture One for my pictures and Capture One is as bad for metadata as it is good for RAW processing.
- I am absolutely dissatisfied with the simplistic country/state/city/location tags. DWC tags are a lot better(but still not perfect). I have a country specific logic for updating these tags.

As I said I can not do all my updates manually. Is there a way to trigger the pen icon through IMWS
If not, this v1/metadata POST command is kind of useless, at least for me.

Thanks and regards, Jozef Dassen

[Mario]

I have made a quick check with the "Metadata" sample app included in IMatch (you see the sample apps when you enable the Developer Mode in Edit > Preferences > Application).
You can also run it directly via http://127.0.0.1:50519/imatch/samples/metadata/index.html

The Metadata app has a button to change the rating and title. When I use these, the rating and title are changed, and marked as modified (pen icon in the MD panel switches).
The tooltip of the pen icon for the modified file shows correctly that Rating/Title have been modified. For example:

tasks: [{"idlist":"@imatch.filewindow.active.selection","tag":"title","op":"set","value":"Hello"}]

When I write-back, the modified rating and title are written.
So it seems to work in principle.

I've changed the app to set the tag DarwinCore::Main\\dctermsLocationCounty\\DCCounty instead of the title.
The tag shows up in the MD panel as modified and is also written back correctly.

Please run this app and change the rating or title. Does this work correctly and flat these tags as modified (tooltip of the pen in the FW)?
If so, there must be a difference in your app...?

[jmhdassen]

Thanks Mario,

That does indeed work but does not solve my problem directly.
My application is a python program that runs on Linux (part of a bigger thing).
I have tried to make calls to the IMWS interface from Linux (191.168.1.21:50519), but I never got a connection. I opened the firewall for the port, tried with firewall off.
It seems it only responds to 127.0.0.1 and not to any actual ip address, even on the  windows machine running IMatch. Maybe that is done on purpose. Fine with me.
So what I did is put a Cherrypy webserver on windows. Linux sends the request to the cherrypy server and cherrypy translates the request to IMWS commands uses 127.0.0.1 to connect to IMWS.
Proper responses are received back

HOST_URL = 'http://127.0.0.1:60520'    ## I changed the port
req = requests.post(HOST_URL + '/v1/metadata', params, timeout=REQUEST_TIMEOUT)

So this works for reading data, reading thesaurus, adding values to thesaurus. It also seems to write the metadata to the right places. I even tested deleting files and deleting values from thesaurus.
I have scripts to verify all the metadata against thesaurus etc, etc.
It all works fine, except for activating the pen icon.

The only difference is that my stuff is in python while the app is in nodejs

Ah, just looked at it again:
I am using an image file id in my tasks. This image is normally not selected/active.
Your app uses the IMatch.idlist.fileWindowSelection. This a list of active selections. Implicitly selected.
Could that make a difference ?

Regards,
Jozef Dassen

[Mario]

I have tried to make calls to the IMWS interface from Linux (191.168.1.21:50519), but I never got a connection.

The integrated IMWS in IMatch only accepts requests from the same machine.
For all else, consider using IMatch Anywhere (https://www.photools.com/imatch-anywhere/)
Which is the server version of IMatch Anywhere WebServices.

So what I did is put a Cherrypy webserver on windows

So you are setting up a kind of proxy web sever to deliberately circumvent the built-in restriction for local connections only in IMatch.  How not nice

Yes, it works whether or not I use an id list or a fixed list of ids.

I recommend you double-check your proxy server and ensure that it works as you think it does.
Please understand that I will not provide any further support for this.
The official and licensed way of doing this works. I've tested it twice.

[jmhdassen]

Got your point.
I will try to build a test windows version of my python scripts.
I have IMatch anywhere, but that does not help me update the metadata in the JPGs which is my goal.

or maybe I do need to go back to external XMP files, but make them myself on Linux (quite easy with python bs4).

I see IMatch as a useful tool, not a goal in itself.

cheers, JD

[Mario]

JPG files must use embedded XMP metadata, by the XMP standard.
IMatch will read the XMP sidecar files, but many software will ignore it. IMatch will only write embedded XMP data for JPG files, unless you force it not to (NOT recommended).
Writing metadata via an app and then writing-back is a simple thing, and done by me and many users.
It's most likely your setup, app and proxy that makes it fail. Only you can check this.