Reverse Geocoding (Lookup All vs Lookup of single image gives different results)

[kiwilink]

I have noticed several times when I choose 9 or 10 images and choose Lookup All and then scroll thru the images one at a time (before I say O.K.) the location will start incorrectly repeating to the next image.  I have attached an example to explain.  On 4 of 9 of the attachment the location is correct (New Hampshire).  However, the next image (5 of 9) says the same location (New Hampshire) but 5 of 9 was shot in Canada.  When I choose just the image I shot in Canada (by itself) and do a Reverse Geocoding it correctly identifies the Latitude and Longitude as Canada.

What am I doing wrong?

Thanks

Kiwilink

[Mario]

Send me these 9 images and tell me in order they are sorted when you open the dialog. => support email address.
Which Geo-coding service do you use?
I need to reproduce this here to see what the geo-coding service returns.

In the "All" mode IMatch just loops over the images and sends each coordinate pair to the geo-coding service. Then it takes the first address returned.

[Jingo]

In the "All" mode IMatch just loops over the images and sends each coordinate pair to the geo-coding service. Then it takes the first address returned.

Not to get sidetracked from the issue - but when you say it "takes the first address returned".. I always thought Lookup All would loop through all the selected images and use their individual coordinates to get the individual location data... in other words -I could select 30 images that are front different countries and always return their individual location results.  Is this correct or have I misunderstood?

[Mario]

If you use Lookup All", IMatch retrieves the "first/best" address for each file individually (making 10 requests to the geo-coding service). Make sure you don't abuse GeoNames.org or that you don't exceed the limits of your Google plan when you reverse geo-code many files.

It does not display the "choose address" dialog for each file or anything. If you want to pick addresses, navigate to the file you want to pick an address and then use "Lookup". This will show you to top-10 addresses returned for that file.

[kiwilink]

Mario:

I sent the files to you along with the sort order.  I noticed that the first image has location data in the Metadata.  It appears that the lookup is copying location data from the first image and applying it to several images (but not all) when I choose "Lookup All"

Kiwilink

[Mario]

I don't think so. Each file is processed individually, at least this is what I remember.
The lookup just loops over the selected files, gets the GPS coords, does the rev. geo-coding and picks the first result returned by the gc service.

Since I need to re-implement all this anyway for IMatch 2018 due to adding support for destination coordinate pairs, I will look into this in a couple of weeks.

[sinus]

What is to say still, at least for me, if I select 30 images I have to hit "Lookup all" several times, because the geo-server does not a lot of images edit at once.
And if I have no box more "that I should try it later", then often the altitude is not set.

Hence I do "lookup all" seveal times, until no warn-box pop-ups and then I check quickly the altidude. If it is not 0, then I click ok.

[Mario]

Which geo-coding service do you use?

GeoNames.org and Google apply certain limits, e.g. how many lookups you can do in a certain amount of time. If you abuse the service, it may block future requests.
I'm quite sure that IMatch should log the error messages received from the service you use to the log file. Knowing the exact error would be helpful. Most likely it's a "user exceeded quota" or similar.

If you need to geo-code many files, you will need to switch to a paid or 'pro' account. These services are not free, at least not for heavy use. If you are on a free plan, you might need to change that.

I think I recall that Google has announced to change their reverse geo-coding and Google Maps usage in June, requiring a paid account for everybody, with a certain number of free Google Maps uses and address lookups. If you use Google Maps and/or Google for reverse geo-coding and you have an account you should have been noticed already.

https://cloud.google.com/maps-platform/pricing/

Everybody seems to get 200US$ of free usage per month, which is a lot.

Here is what Google charges per map lookup, find address operation or reverse geo-coding call (per image): https://cloud.google.com/maps-platform/pricing/sheet/

GeoNames.org is financed by donations only and by some commercial services.
If you are on a free and not a paid account your requests may be blocked when you send to many in a too short time.

IMatch applies a certain wait time for each request as to not exceed the maximum number of requests per second. But if you fire Lookup All for many files in quick succession, GeoNames.org / Google may block one or more requests.

Please understand that this is not a limitation in IMatch.
You are using services provided by 3rd parties for free. If you have a Google account which includes the usage for the reverse geo-coding and Google Map usage, you have much higher limits, but Google will charge you if you exceed your monthly free limit. It's not that expensive, though.

If you use the Map feature in Lightroom. you are locked into Google. Adobe pays for your usage, which is easy for them to do, given that they make billions of dollars. And I'm sure Adobe gets good rates from Google as well. And there is maybe some data transfer as well, providing Google not only with your IP address but also information about your software environment and whatnot. That's worth a lot for Google.

I don't do that. I give you options to choose from. And I'm surely unable to pay for the Google Maps, Bing Maps or Google reverse geo-coding usage from all IMatch users.

[kiwilink]

Mario:

Thanks for the feedback.  I am still testing this and if I find out more I will post it here.  Glad you can look at it when you get time.  No hurry here at all.  I just want to make sure that if I do 20 at a time I don't have to step thru each image every time to make sure it finds the right location.  I probably will only do 60 or 70 a week so I don't think I will exceed Google's limitation.

Thanks!

Kiwilink

[Mario]

Keep an eye on the log file to see if Google returns error messages when geo-coding. If you exceed your limit, it will show in the log at least.

[Jingo]

Keep an eye on the log file to see if Google returns error messages when geo-coding. If you exceed your limit, it will show in the log at least.

Hi Mario - just throwing this out there... the log is a great resource but without knowing all the "ins-and-outs" of error/warning messages or what to look for, kinda hard to use for the average user. 

Have you given any thought to updating the "info and activity" panel to provide more details about what IM is currently doing (activity taking a long time AND errors/warnings)?  Ie: it often takes IM about 15 seconds to load fully... I see things in the log like:

05.24 07:59:28+   31 [4774] 10  I>                  InnerSearchMetabase: Pattern: '^True', op: 1000, groups: 0, tags: 1, scope: 0, lang: '(null)', flags: 0
05.24 07:59:28+    0 [4774] 10  M>                   > 18 CIMQueryManager::InnerSearchMetabase  'IMQueryManager.cpp(1912)'
05.24 07:59:38+ 9844 [4774] 10  I>                     1003 results fetched (bv)
05.24 07:59:38+    0 [4774] 10  M>                   < 18 [9844ms #sl] CIMQueryManager::InnerSearchMetabase

looks like this process took 9 secs to complete... but - what is it and what is it doing?  Have some of this info (along with error messages like the Google Maps one above) will make things MUCH easier for users to figure out... things like Google reporting limits and issues is important for the user to know (especially if the reverse geo is not working) and having it listed someplace in the software in easy terms will make IM errors/issues much easier to review.  I often go to that panel.. but it (and the activity gear icon) doesn't provide me with the info I'm looking for.. I often then go to the log (which is setup as FULL all the time) - but it has so much info that I often get lost and don't know what to look for.

[Mario]

This is a formula-based category updating in the background. If this takes such a huge toll on startup time on your machine, consider deleting data-driven / formula-based categories you don't need.
The long execution time may be caused by other tasks with higher priority pausing the search. Category updates have only background priority and may be intercepted by more important tasks.

IMatch is complicated application. It does many things, and many thing simultaneously. Adding more info to the Info & Activity panel will do no good. I'm actually thinking about removing because I doubt many users look at it more than once. Most users are not interested in more details.

But this is totally unrelated to your initial question and we should not distract this thread.

[kiwilink]

Mario:

After several tests I have concluded that when I pick even a small number of lookups (like 3 images) and choose Lookup All,  if the first image already has location data then the other two images will come back from the lookup with the same location as the first image. 

Consequently, If I pick 3 images and the first image doesn't have location data but the second one does then the first image will be correct, the 2nd image already had location data and is correct but the 3rd image will come back incorrectly with the location data from the 2nd image.

Therefore, it looks like as long as you don't grab an image that has location data everything works fine but if one image has location data then it will duplicate the location to the following image.  I hope you can duplicate this on your end.

Thanks!

Kiwilink

[Mario]

With "location data" you mean GPS coordinates or things like City/Country/....?

[kiwilink]

Mario:

I am referring to Composite Metadata.  I have attached a screen capture of what I am referring to by "Location".  It is the Composite data that shows up when I open up the Metadata panel (see attachment).

Mike

[kiwilink]

Mario:

I am sending you two images via email that you can try. 

kiwilink

[kiwilink]

Mario:

I was doing more testing on this issue and tried changing the Provider Search and the Geocoding Service to Geonames.org to see if I would get different results but I did not.  Any word on this or is there anything you would like for me to try? 

Thanks!

Kiwilink

[Mario]

I've made several tests with the two sample files you've send, and also the larger set. Selection them all or in various combinations, using the Lookup All button in the Reverse Geo-coding dialog.

This chooses the first returned address from the geo-coding service for each file. All files get different location data assigned.
The two sample images get Belleville/IL and Mascoutah/IL as expected and as you say in your email.

Unless I do something completely different than you this was either a strange glitch nobody ever noticed or I have fixed it for IMatch 2018.
Please re-test when IMatch 2018 is out and let me know.