GPS - POI Info

Started by hannes_hab, September 13, 2021, 04:55:30 PM

Previous topic - Next topic

hannes_hab

One of my cameras (nikon coolpix p330) has a function to save the GPS Position and also add POIs (Point of interest) - so when showing the photos in the camera also a note what it is is shown: like town XX and castle YY.
I wonder where these data ist stored and if Imatch reads these Infos.
I searched the POI Info but did not find it.

Maybe you can help me

Mario

This sounds like one of the many proprietary Nikon metadata fields.
IMatch by default does not import most of the proprietary maker notes.

1. Check in the ExifTool Command Processor (List Metadata preset) to see if ExifTool knows about these tags.

2. If so, enable them in the IMatch Tag Manager

3. Select the files with this data and then press Shift+Ctrl+F5 and use "Reload Metadata".
NOTE: Write back before step 3.

You can then add this data to a custom Metadata Panel layout, use it with variables etc.
-- Mario
IMatch Developer
Forum Administrator
http://www.photools.com  -  Contact & Support - Follow me on 𝕏 - Like photools.com on Facebook

hannes_hab

Thank you - I tried this but:

Exif Tools says :

[Nikon]         Location                        : Baluardo San Pietro

but this is shown in the Nikon Location field:
QmFsdWFyZG8gU2FuIFBpZXRybwAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA==

Mario

#3
Looks like a binary data block. Maybe Base64-encoded. Maybe encrypted...
Feel free to ask about it in the https://exiftool.org/forum/, maybe somebody is interested in decoding this and Phil can add it to ExifTool.

Nikon may not be interested in other software being able to use this, so I don't think they will document this anywhere.

Maybe the Nikon software allows you to export it as standard track log file? Then you can import and visualize it on the IMatch Map panel: GPX Track Log Import
-- Mario
IMatch Developer
Forum Administrator
http://www.photools.com  -  Contact & Support - Follow me on 𝕏 - Like photools.com on Facebook

hannes_hab

its no big problem but I thought - exif-tool (ExifTool Command Processor (List Metadata preset)) shows it  the right way - therefore it it not an exif-tool problem. Am I wrong?

Mario

As I said, the problem is that ExifTool does not announce the [Keys] group and hence IMatch does not know about it.
If ExifTool would tell IMatch that there is a Keys group with these n tags, IMatch would import the data automatically. Something with Keys is special. Maybe ExifTool invents it on the fly when finding proprietary Apple data. Apple does many strange things - many of them designed to lock customers into the Apple ecosystem.
-- Mario
IMatch Developer
Forum Administrator
http://www.photools.com  -  Contact & Support - Follow me on 𝕏 - Like photools.com on Facebook

hannes_hab

ok - thank you very much - I will report it there - but its no big problem

Mario

As I said, I will ask Phil. And I did.
Lets give it a couple of days for him to find some time for answering. We do all this in our spare time, so...
-- Mario
IMatch Developer
Forum Administrator
http://www.photools.com  -  Contact & Support - Follow me on 𝕏 - Like photools.com on Facebook

Tveloso

Mario, I think you may have confused an item in this post (a Base64-encoded string in Nikon Maker Notes), with my post (about an Apple/QuickTime "Keys" Tag).

Both relate to things that ExifTool may not be making available to IMatch, or that IMatch does not import by Default...and you're probably moving through these posts at 100-miles-an-hour...

But I'm glad to see that I'm not the only that does this type of thing... :)
--Tony

Mario

I have answered about 60 emails today, plus community posts here in the public and private boards. it happens that I confuse things.
-- Mario
IMatch Developer
Forum Administrator
http://www.photools.com  -  Contact & Support - Follow me on 𝕏 - Like photools.com on Facebook

Mario

Quote from: hannes_hab on September 13, 2021, 08:16:58 PM
its no big problem but I thought - exif-tool (ExifTool Command Processor (List Metadata preset)) shows it  the right way - therefore it it not an exif-tool problem. Am I wrong?

What does this mean?
Shows it the right way? You said

but this is shown in the Nikon Location field:
QmFsdWFyZG8gU2FuIFBpZXRybwAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA==


so what should IMatch make of this?
-- Mario
IMatch Developer
Forum Administrator
http://www.photools.com  -  Contact & Support - Follow me on 𝕏 - Like photools.com on Facebook

hannes_hab

I did everything as you told me:

Tools -> ExifTool Command Processor -> List Metadata
shows different metadata and also:

[Nikon]         Location                        : Baluardo San Pietro

so I thought exif tool can read and show it as it should be.

Then I have enable this tag in the IMatch Tag Manager

Added this tag to my custom Metadata Panel layout, made a write back and reloaded the metadata. New Metadata field (location / Nikon) :

In this field this : QmFsdWFyZG8gU2FuIFBpZXRybwAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA==
is shown.

I thought it should be: Baluardo San Pietro

Mario

#12
Just importing an arbitrary maker note into the database will not magically make IMatch use this maker note instead of the standard XMP Country, City or Location metadata.
IMatch has no idea that the proprietary maker note Location contains a location information, or how to use it.
You can configure the Metadata Panel to show the tag and you can use it, e.g. via variables or for searching/sorting, but that's it. For IMatch this tag is just a tag, not related to anything with locations.

By the looks of the data in the MD Panel, ExifTool did somehow mess the character encoding. Or delivered a binary data stream to IMatch, which is unexpected...

Please upload a sample image somewhere and post a link so we can have a look. I have not seen something like this before.


For display in the Metadata Panel, reverse geo-coding, the Map panel, sorting, searching and all other location-based features, IMatch uses the official XMP location tags (via the special Composite tags provided by ExifTool).  IMatch fills these tags when you do Reverse Geocoding in IMatch, or you can fill them manually. These tags are also filled by other standard software, e.g. Adobe products.

If the Nikon software you use (or the camera) has decided to put the location into a proprietary and undocumented NIKON maker note instead of the ISO standard XMP tags, that's their problem. IMatch is not aware of this and will not use it. I wonder why they do that - the only thing I can think of is to force you to use their software and ecosystem.

But that's just a location, right? Not a complete data set.
Where does Nikon store the country name or city name in your scheme?
All the standard location fields in your MD panel are empty. Which is not good.
A full location set consists of GPS coordinates, country, city, location, iso code. And that usually both for the location created and the location shown.

If there are GPS coordinates in your images, why don't you just use IMatch's Reverse Geocoding?
This fills all the right tags with the right values, allowing you to use that data also in other software.
Using proprietary maker notes for that
-- Mario
IMatch Developer
Forum Administrator
http://www.photools.com  -  Contact & Support - Follow me on 𝕏 - Like photools.com on Facebook

hannes_hab

Thank you.
I do not expect that. I only wanted to see the information that is wirten by this camera and shown by exiftool.
I know about reverse geotagging but as posted in the "feature request" (reverse geocoding provider also in the geocoding panel) I sometimes want to see other infos about what is seen so the POI feature is maybe sometimes helpful.
When I change the reverse geocoding provider I get different information about the location, sometimes Geonames or Google or Here gives me better information about where I am and what is seen.
So it would be fine to switch fast between these infos and also have additional info like the nikon POI which is also named "location".

As I said - its no big problem but I will upload an example file in the evening.

Mario

The POI thing is a proprietary Nikon thing.
Usually you have two sets of location data for a file (in XMP): location created and location shown. Each with GPS coordinates, country, city and location data.

These are basically the two possible POI for an image. An image cannot now or track that you were looking at a POI five minutes before the image was taken (unless you store that in the description or one of the other text fields in XMP).

Not sure where other POIs can come in here. Or how your Nikon software would encode that.
Usually POI data is logged in a track log, which is unrelated to images.
Images may be taken at some points along the track, but the track itself is independent.
You can create tack logs 2 hours long, create POIs entries every 5 minutes with GPS coordinates and description, but take an image maybe only every 30 minutes.
This type of POI data cannot be stored in a single image, but only in a track log of sorts (which IMatch supports in the Map Panel).

I can tell you more about the encoding issue when I have seen a file your camera produces.
-- Mario
IMatch Developer
Forum Administrator
http://www.photools.com  -  Contact & Support - Follow me on 𝕏 - Like photools.com on Facebook

hannes_hab

I just uploaded the example files:

https://www.dropbox.com/sh/9u9f546xkcnyebj/AADUwv6OzmflUF7qEM4BDELda?dl=0

I also made screenshots what exiftool shows and what reverse geocoding, provided by geonames, google and here, would do.

Unfortunatly the best info is provided by the cameras POI function. No Idea how nikon does it but maybe its a google fucnction ...

I also can open the map and see the info there and then write it, but if the info is already there its fine to just copy and paste from one field to another or use a function Imatch provides.

Mario

In addition to reverse geo-coding, Google offers additional services (each at a cost) for "place details" like opening hours, user reviews, icons, formatted address and whatnot. That's an extra, you first have to retrieve their place id via reverse geo-coding and then use that place id to query for additional data (which costs additional money).

IMatch does not do this. It uses the "formatted address" as provided by Google, plus the country data.
This does not return things like "Eiffel Tower" (location shown as seem from a human perspective) but the location for the camera standpoint and the location shown in form of street addresses.

Maybe Nikon does use this API to fill in additional data in their software? Do you pay for this software or does Nikon pay all the Google bills? These APIs are charged by the 1,000 invocations, so this may get expensive first (unless Nikon sells your data back to Google to offset the cost).
-- Mario
IMatch Developer
Forum Administrator
http://www.photools.com  -  Contact & Support - Follow me on 𝕏 - Like photools.com on Facebook

Mario

#17
I have downloaded the file.
First I've made some tests with the extra "places" API from Google after retrieving the place_id via IMatch reverse geo-coding.
I could not get better data than what IMatch already gets via the reverse geo-coding. Nowhere was anything close to "Chiesa di San Leonardo" returned.
Which Nikon software do you use for this (or is this done by the camera?) Are you sure it is using Google and not Here or Bing?

The data contained in the [Nikon] Location field is returned as a Base64-encoded binary data stream.
This format is only used for binary data without any text interpretation. IMatch treats it as such and stores the binary blog as a string in the database.
Not sure why ExifTool delivers this as a binary stream of bytes and not as a text. Or why ExifTool shows it as a readable text (maybe some extra logic for this field built-in?)
When I let ExifTool output XML it delivers the binary stream. This is what IMatch is using to ingest results from ExifTool.
When I let ExifTool output JSON or text, it delivers the tag as text.

This might be a bug or particularity for XML output in ExifTool. I need to ask Phil about this.
-- Mario
IMatch Developer
Forum Administrator
http://www.photools.com  -  Contact & Support - Follow me on 𝕏 - Like photools.com on Facebook

thrinn

I do not own such a Nikon camera, but according to their Web Site it looks like the POIs are stored in form of a database inside the camera memory. I would assume that this is a offline solution, without calling some Google API or whatever. I have no idea if it would be possible to update this database in-camera. Maybe a similar approach like for some (often expensive) built-in navigation systems in cars, where you have to pay (and pay handsomely!) for map updates, after a certain period of time.
Thorsten
Win 10 / 64, IMatch 2018, IMA

hannes_hab

It is done by the camera, after getting the GPS-signal (GPS symbol turns from red to white) the POI is displayed on the screen (almost immediately - not all the time helpful - sometimes you only get police-office and so on - but when travelling its fine ), and after making the picture it is also displayed when viewing the image.
I never used a nikon software to display or develop the pictures. My Raw-Software displays only the GPS coordinates (and had a tool to do reserve geocoding but after changes in the Google API this is not working any more). I thought maybe Imatch can display this infos.

Mario

#20
IMatch does display the info  ;D
The problem is that ExifTool does not export the text but encodes it as a binary string - this is what IMatch gets and shows.
I have posted a question in the ET forum, let's give it a couple of days.

If the camera uses an integrated POI database, that explains that. But not why Nikon buries the data in a proprietary maker note and does not update the XMP record to make this data accessible for other applications.

You can use a Metadata Template to copy the data from the Nikon maker notes to XMP if you like.
Once the ExifTool issue is resolved.
-- Mario
IMatch Developer
Forum Administrator
http://www.photools.com  -  Contact & Support - Follow me on 𝕏 - Like photools.com on Facebook

hannes_hab


hannes_hab

Any news from the ET forum?

Mario

Yes. And No.

This is how ExifTool behaves. For this particular tag with these particular contents.
It seems that whatever camera/software has written the data into this tag has not written text, but a mix of text and binary data (I assume filling the tag with 0-bytes for padding or whatever).

When  ExifTool is used to extract the data in plain text format (e.g. in the ExifTool Command Processor), it somehow detects this problem and strips the binary data and returns only the text.
The same is done when ExifTool is used to extract the data in JSON format (with the -JSON parameter).
But when data is requested in XML format (as IMatch does when ingesting data from ExifTool), the data is returned not as text but as a binary blob, Bse64-encoed. And it is even lead-in with a line-feed character, which is illegal for Base64-encoded data.

IMatch gets the data as a Base64-encoded character string:

<rdf:Description rdf:about='E:/data/download/community/11746/DSCN9362.jpg'
  xmlns:et='http://ns.exiftool.org/1.0/' et:toolkit='Image::ExifTool 12.26'
  xmlns:Nikon='http://ns.exiftool.org/MakerNotes/Nikon/1.0/'>
<Nikon:Location rdf:datatype='http://www.w3.org/2001/XMLSchema#base64Binary'>
Q2hpZXNhIGRpIFNhbiBMZW9uYXJkbwAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA==
</Nikon:Location>
</rdf:Description>
</rdf:RDF>


This is what IMatch gets. You get the same when you request XML output in the ExifTool Command Processor by adding the -X parameter.

Based on what Phil wrote, this behavior is as it is and historical. It may only affect your files, or only some of your files where the Nikon:Location not contains pure text but a mix of text and binary data - which triggers the Base64-encoding in ExifTool for XMP output. I don't see this changing in ExifTool.

What I did last night was to add a special case (yet another one) for the Location tag in the Nikon::LocationInfo group when Base64-encoded binary data is returned.
IMatch now strips characters not valid for Base64, then decodes the result then strips everything that is not ASCII or a blank or tab and puts the result into the Nikon::Location tag.
For the two sample images I have this worked well.

Did I tell you that I HATE metadata, and especially camera vendors filling files with crap? Because, in the end it all boils up in IMatch I have to to spend enormous amounts of time to deal with this shit.
Why does Nikon not just fill the pretty, neat, standard XMP tags for location? They are standardized, documented, ISO-certified. Why bury this crap somewhere in a proprietary maker note. Why? Because to lock you into their software and ecosystem.

I'm sick of this.
-- Mario
IMatch Developer
Forum Administrator
http://www.photools.com  -  Contact & Support - Follow me on 𝕏 - Like photools.com on Facebook

sinus

Quote from: Mario on September 23, 2021, 06:37:04 PM

Did I tell you that I HATE metadata, and especially camera vendors filling files with crap? Because, in the end it all boils up in IMatch I have to to spend enormous amounts of time to deal with this shit.
Why does Nikon not just fill the pretty, neat, standard XMP tags for location? They are standardized, documented, ISO-certified. Why bury this crap somewhere in a proprietary maker note. Why? Because to lock you into their software and ecosystem.

I'm sick of this.

On the other hand, what would be IMatch without Metadata?
So, deep in your heart, you cannot hate them.  8) ;)

You did and did several times the job of Nikon, Canon, Olympus ... and so on guys, what did it not correct.
Maybe you should send them all a bill, for work as a part-time employee. ROFL  ;D :D :)

Well, sometimes we have to make a joke.  ;)
Best wishes from Switzerland! :-)
Markus

Mario

Quote from: sinus on September 23, 2021, 09:37:59 PM
On the other hand, what would be IMatch without Metadata?
So, deep in your heart, you cannot hate them.  8) ;)

The purpose of a DAM and my precious spare time is not to deal with non-standard, proprietary and undocumented metadata crap camera vendors come up with.
Especially not NIKON, which introduced encryption and obfuscation in their metadata to prevent others from legally accessing it - until Adobe cried out and NIKON had to cave in.

They just give a shit about you once they have your money. Encrypting metadata to prevent the use in other software, burying important things like white balance info in proprietary maker notes instead of the official EXIF tags, forcing you to use their cloud to unlock certain camera features. And users let them get away with all this boo-shite. I don't like this at all.
-- Mario
IMatch Developer
Forum Administrator
http://www.photools.com  -  Contact & Support - Follow me on 𝕏 - Like photools.com on Facebook

hannes_hab

Quote from: Mario on September 23, 2021, 06:37:04 PM
Yes. And No.

Thank you

Quote from: Mario on September 23, 2021, 06:37:04 PM
What I did last night was to add a special case (yet another one) for the Location tag in the Nikon::LocationInfo group when Base64-encoded binary data is returned.
IMatch now strips characters not valid for Base64, then decodes the result then strips everything that is not ASCII or a blank or tab and puts the result into the Nikon::Location tag.
For the two sample images I have this worked well.

Fine - sounds great!

Quote from: Mario on September 23, 2021, 06:37:04 PM
I'm sick of this.

I understand that

hannes_hab

I wonder if there is an approach to simulate something like this, what obviously the camera does in this case, and get infos about this place (POIS, sights, church, castle ... ) from wikipedia or google with an app ...

Mario

There are commercial POI providers, e.g. https://datarade.ai/data-categories/point-of-interest-poi-data or https://carto.com/spatial-data-catalog/poi-data/
When you use the IMatch AutoTagger with Google, known landmarks, popular buildings etc. are often returned as part of the keywords.
Maybe OpenStreetMap has a dedicated API for POIs as well.
-- Mario
IMatch Developer
Forum Administrator
http://www.photools.com  -  Contact & Support - Follow me on 𝕏 - Like photools.com on Facebook

hannes_hab

Thank you for that hint - I now only use clarifai - I will try google too!
Concerning Imatch AutoTagger or generaly APPS - some can stay open but some you have to open and close when switching to new pictures.
Is there a chance to have it (AutoTagger or reverse Geotagging too) open all the time?

Mario

-- Mario
IMatch Developer
Forum Administrator
http://www.photools.com  -  Contact & Support - Follow me on 𝕏 - Like photools.com on Facebook

hannes_hab

this is the big advantage of this older APP "Super GeoLocator App" it can be open all the time:
https://www.photools.com/community/index.php?topic=9093.0

Mario

What is the advantage?


You select the files you want to reverse geo-code, you run the dialog, done.
You select the files you want to auto-tag, the dialog opens, you auto-tag, dialog closes, done.
No resources wasted by constantly keeping the window and browsers loaded, no performance impact by monitoring the event bus in IMatch and reacting to events when not needed.

There is a big difference between something like Auto Tagger, the Renamer, Design & Print, reverse geo-coding which uses a dialog approach and app which must always be open, e.g. the Map Panel or Event View in order to be useful. IMatch had over 100 dialog boxes. Why should it keep them all open all the time?
-- Mario
IMatch Developer
Forum Administrator
http://www.photools.com  -  Contact & Support - Follow me on 𝕏 - Like photools.com on Facebook

hannes_hab

I am sure its my fault. I simply want to do things I am used to. I am (still) new to IMATCH and for me a DAM only should help me (fast). My main thing are the pictures and the software to develop these pictures - the best thing would be if I can do everything inside this software, unfortunately this is not working. You are familiar to your software - you know the best way to use it. I have to try out many things - hope I find a good way to work with it.

I dont want to have them ALL open ALL the time, but for some I would wish it.
I try to use some of the auto functions (e.g. reverse Geotagging along with Super GeoLocator) maybe this helps me.

Mario

I'm still not getting your point.

When you want to do a reverse geo-code for some files, you select them (or use Ctrl+A to select all files) and then run the dialog with <Ctrl>+<M>,<G>.
If your settings are correct, you geo-code all files with <Alt>+<A> or a click on "Lookup All".

If the reverse geo-coding would be somehow running in a separate panel, you would sill have to select the files you want to process, and then click one or two buttons on the app.
I don't see any saved steps using that approach.

If an efficient workflow is your goal, make sure you get acquainted with IMatch features like Favorites, Metadata Templates (can save a lot of reverse geo-coding when you shoot the same places often), Keywords Panel etc.

The Saving Time topic in the IMatch help lists many of the features designed to getting things done quickly.
-- Mario
IMatch Developer
Forum Administrator
http://www.photools.com  -  Contact & Support - Follow me on 𝕏 - Like photools.com on Facebook

hannes_hab

I want it fast, but also have control.

With Super GeoLocator I can do following:
Click retrieve Data - I get some results - maybe one sounds good for me - ok - if I am not sure I can zoom in with the open map an see whats there and maybe confirm or correct.

With standard reverse Geocoding I cannot do that, I have to close the app, zoom in (the map) and open it again. Or am I wrong?

In the older Bibble and ASP software when GPS-plugin worked (unfortunately corel has no will to update these plugins) there was a combined map/reverse geocoding tool - of course this was in the Metadata panel group and could be open all the time. (its still here but not working any more). There also was an option to have an addition big open floating map - (linux specific I think) on top or in the background, opening or closing with one click, without any lags on an older computer. Everything concerning Metadata is done suddenly - no waiting.

Thats the same with the Nikon POI Information - If I would have it within IMATCH I would have additional infos the would help me.
Not so big problems but annoying.

And speed .... I am not sure, sometimes its not that fast but ok, but sometimes (still on the NAS) everything freezes -dont know, maybe (only) Windows is shit. I have enough CPU-Power and RAM. But I can only do one simple task - write back Metadata - every other task or software is blocked.

Thank you for the Saving Time hint - I will read it ...

Mario

#36
QuoteIf I would have it within IMATCH I would have additional infos the would help me.

I've explained how you can transfer this proprietary Nikon data into proper XMP data.
The data comes from a database embedded in the camera firmware and is not accessible from the outside.
Nikon probably pays a lot of money every year for this database (if they license it from a POI vendor).

QuoteWith standard reverse Geocoding I cannot do that, I have to close the app, zoom in (the map) and open it again. Or am I wrong?

Not sure about that. I usually use a workspace like this for reverse geo-coding. I see the image in big, I see the map in big, and there is usually no doubt if what the provider returns is correct.



QuoteAnd speed .... I am not sure, sometimes its not that fast but ok, but

What does that mean? Reverse geo-coding is done in the database. Storing the data received from the vendor takes maybe 0.1 seconds.
I process 100 files in a few seconds, no issues?
Where does the NAS come in? Or have you set IMatch to write-back immediately after every metadata change? That would be a bad idea and real bad for performance.

When does IMatch freeze?
You are aware of the log file and this is the first thing you should ZIP and attach when you experience performance issues in IMatch?
Because the log tells us what takes how long and where the problem is. Or if you process 10 MB JPG files, 50 MB RAW files or 2 GB video files... Slow is relative.

Also keep a close eye on your virus-checker. Because in 90% of all "IMatch is slow" cases it is not Windows shit but the virus checker interfering, constantly scanning the IMatch database and every file IMatch touches. IMPORTANT: Virus Checkers

Of course a NAS box performs 100 to 1000 times slower than a local SSD.
NAS are for long-term archival, not for actively working with files.

Note that the performance of reverse geo-coding depends on the service you use and your internet connection.
If you use a commercial service like Google or HERE, response times are one or two seconds.
If you rely on the free GeoNames.org service, response times will vary greatly. This service is operated and paid for by volunteers.

You can also enable automatic reverse-geocoding and IMatch then updates the data automatically when you place or more the file marker on the Map Panel.
And when you import new files.
If you need precision, this is what you should do. You place the marker on the map, no need to use the reverse geocoding dialog at all!

Automatic reverse geocoding
-- Mario
IMatch Developer
Forum Administrator
http://www.photools.com  -  Contact & Support - Follow me on 𝕏 - Like photools.com on Facebook

hannes_hab

Quote from: Mario on September 26, 2021, 03:31:53 PM
QuoteIf I would have it within IMATCH I would have additional infos the would help me.

I've explained how you can transfer this proprietary Nikon data into proper XMP data.

Hmm I did not get that - I tough its working for you - and maybe will be available i a next release.

Quote
What I did last night was to add a special case (yet another one) for the Location tag in the Nikon::LocationInfo group when Base64-encoded binary data is returned.
IMatch now strips characters not valid for Base64, then decodes the result then strips everything that is not ASCII or a blank or tab and puts the result into the Nikon::Location tag.
For the two sample images I have this worked well.

I dont understand - What do I have to do, so that it is displayed correctly?

hannes_hab

Quote from: Mario on September 26, 2021, 03:31:53 PM
Also keep a close eye on your virus-checker. Because in 90% of all "IMatch is slow" cases it is not Windows shit but the virus checker interfering, constantly scanning the IMatch database and every file IMatch touches. IMPORTANT: Virus Checkers

I only have Windows Defender and the folder where the IMATCH database is, is not scanned. Maybe I should exclude the folders with the pictures too?

Mario

Your NAS runs some sort of Linux and on that runs some sort of SAMBA which simulates a Windows server to the outside. Lots of things can get into the way. The network speed is also important.
I recommend finishing your files on your local had disk before moving them to the NAS for long-term archival if you experience slow performance when working directly on the NAS.
-- Mario
IMatch Developer
Forum Administrator
http://www.photools.com  -  Contact & Support - Follow me on 𝕏 - Like photools.com on Facebook

hannes_hab

here is the log-file
Last action I have written only 1 file with Metadata changes - no chance to do anything (in Imatch, or with ASP) in the meantime - I dont know this from linux on my older PC

Mario

Please use debug logging (Help menu > Support > Debug Logging).
This log file contains only a minimum amount of data and no info about write-back or write-back timings.
Use ZIP to compress the log before uploading.
-- Mario
IMatch Developer
Forum Administrator
http://www.photools.com  -  Contact & Support - Follow me on 𝕏 - Like photools.com on Facebook

hannes_hab

#42
here are the new LOG files
last actions:
copy of Metadata from one file to one other: about 1 minute duration
writing of one changed file: about 1 minute duration

Both operation in ASP:  immediately (1 second) coping and writing of Metadata (in Catalog on local disk and copy the the xmp to the NAS / without writing into the picture File)

Mario

QuoteCopy of Metadata from one file to one other: about 1 minute duration

This means:

1. IMatch has to write-back the source file
2. Re-import the metadata  to make the database consistent
3. Copy the data from the source file to the target file
4. Re-import the metadata from the target file

If you do all this over a network and on your NAS box, a lot of data has to travel.

What is ASP?

Keep in mind that IMatch/ExifTool not only update a bit of data in the XMP sidecar.
Depending on the data you have changed, ExifTool must also splice and update the image itself, to ensure that metadata is properly synchronized.
Does your ASP that too? I doubt it.

Also, ExifTool is designed to do safe streaming updates, which means it always processes the entire file to be able to properly deal with EXIF maker notes and offset shifts due to larger XMP data  This takes longer, but is much safer. Many applications out there wipe or break maker notes when updating files, ExifTool does not.

Repeat your test with the same files on your local disk or SSD.

The difference in execution time is the NAS penalty in your environment.

52 seconds to write a NRW file is really slow.
On my system this takes less than one second (local disk) and about 6 seconds on my (pro-grade) NAS.
-- Mario
IMatch Developer
Forum Administrator
http://www.photools.com  -  Contact & Support - Follow me on 𝕏 - Like photools.com on Facebook

hannes_hab

Quote from: Mario on September 26, 2021, 06:16:16 PM
What is ASP?
Aftershot Pro (former Bibble) is my raw-converter and picture editing program.
https://www.aftershotpro.com

Quote
Repeat your test with the same files on your local disk or SSD.

I tried it on the local ssd - yes much faster - about 2 seconds - I think I will buy a external SSD for my recent pictures.

But then I have to synchronize them to the NAS - so then I should use Imatch for this action again?

hannes_hab

Quote from: Mario on September 24, 2021, 09:10:53 AM
There are commercial POI providers, e.g. https://datarade.ai/data-categories/point-of-interest-poi-data or https://carto.com/spatial-data-catalog/poi-data/
When you use the IMatch AutoTagger with Google, known landmarks, popular buildings etc. are often returned as part of the keywords.
Maybe OpenStreetMap has a dedicated API for POIs as well.

I have tried the Autotagger with Google but did not get better results - maybe this only works with the most popular buildings ...

Mario

Most likely. This is why there are specialized and expensive POI databases.
Not sure if access to them is affordable for normal people. But I think that Nikon pays dearly for the data they embed in the in-camera database, when they buy the license from a specialized vendor.

To sad that Nikon stores the info in a proprietary and undocumented metadata field and not in the standard XMP location data field. Probably just another subtle lock-in mechanism...
-- Mario
IMatch Developer
Forum Administrator
http://www.photools.com  -  Contact & Support - Follow me on 𝕏 - Like photools.com on Facebook

hannes_hab

Quote from: hannes_hab on September 26, 2021, 04:29:56 PM
Quote from: Mario on September 26, 2021, 03:31:53 PM
QuoteIf I would have it within IMATCH I would have additional infos the would help me.

I've explained how you can transfer this proprietary Nikon data into proper XMP data.

Hmm I did not get that - I tough its working for you - and maybe will be available i a next release.

Quote
What I did last night was to add a special case (yet another one) for the Location tag in the Nikon::LocationInfo group when Base64-encoded binary data is returned.
IMatch now strips characters not valid for Base64, then decodes the result then strips everything that is not ASCII or a blank or tab and puts the result into the Nikon::Location tag.
For the two sample images I have this worked well.

I dont understand - What do I have to do, so that it is displayed correctly?
Once again, please can you tell me what I have to do - unfortunately I dont did not get it.
Thank you in advance.

Mario

#48
Just copy the data from the proprietary Nikon tag into XMP location or wherever you want it.
Use a Metadata Template.

The 2021.10.2 release of IMatch contains the work-around for the binary data issue in this Nikon tag.
You just need to force a reload of the metadata with Shift+Ctrl+F5, "Reload Metadata" after selecting the affected files in a File Window.
This produces a "cleaned up" Nikon::Location value, which you then can copy into XMP::location or use however you like.
-- Mario
IMatch Developer
Forum Administrator
http://www.photools.com  -  Contact & Support - Follow me on 𝕏 - Like photools.com on Facebook

hannes_hab

YES - its working - thank you very much Mario!