Blue Hour Calculator for Keywords?

[jch2103]

This is definitely off-topic. I wonder if anyone know of a blue hour calculator that could assign a 'blue hour' keyword to files. I ask only because someone on a dpreview.com forum posed a challenge for blue hour images, which led me to this question.

I can understand why this wouldn't exist, as it would need to determine blue hour (and likely other 'hour' keywords like golden hour), based on specific location at the time the images were taken. Seems quite complicated to me, but I'm curious.

[Mario]

There's a bit of science involved in this:

However, the blue color spectrum is most likely to emerge when the Sun is between 4 and 8 degrees below the horizon. By this definition, the blue hour encompasses parts of both nautical twilight and civil twilight.

Quote from https://www.timeanddate.com/astronomy/blue-hour.html
This web site provides extensive tables for all kinds of time-related stuff, including the base for calculating the blue hours (there are two per day I think) for a specific location/city.
The manual lookup is free, using the API, e.g. from an IMatch app, costs money.

This is also interesting: https://jekophoto.eu/tools/twilight-calculator-blue-hour-golden-hour/index.php which integrates this into a Google Maps lookup, as it seems.

[Aubrey]

I have used a book Astronomical algorithms by Jean Meeus for generating sun positions.
Easiest way to implement is:
take date and time from image convert to UTC
calculate sun position. 
Look at photo position and calculate sun's  apparent declination. (relatively simple spherical trig.)
Check if this fits blue hour/golden hour.
This is the easy part to code intoJavascript.
For me the tough bit would be getting time and location out of database!

Aubrey

Ps. I'd be happy to write the Java code if somebody else could get the parameters out of IMatch...
Fun collaborative project...
I wrote sun and planet position program for hp49 about 18 years ago

[Mario]

Getting time and location out of the database is easy using the files endpoint.
For example, if you paste this in any IMatch app (for example, one of the sample apps),

Code Select Expand


                 IMWS.get('v1/files',{
                     idlist: IMatch.idlist.fileWindowSelection,
                     fields: 'id,datetime',
                     rtaglat: 'gpslatitude',
                     rtaglng: 'gpslongitude'
                 }).then(response => {
                     console.log(JSON.stringify(response,null,2));
                 });


it fetches the id, date & time and latitude/longitude for each file currently selected in the active file window.
It returns an array of file objects. Each object contains the id, datetime, lat and lng. The JSON result looks like this:

{
  "files": [
    {
      "id": 3947,
      "dateTime": "2007-01-16T10:18:33",
      "lat": "51.53055",
      "lng": "-0.127161"
    }
  ],
  "hiddenByStacks": 0
}

[Jingo]

Sounds like a fun APP to create though ideally there would be a free API to plug into... most require a credit card for API key even if they offer some free endpoint hits.  I'm happy to review and create if there is interest.

[Aubrey]

Working on it...
Almost got the sun position coded. Just checking against nautical almanacs for possible errors in code.

A.

[Jingo]

   Very nice.. looking forward to seeing it!

[jch2103]

Very cool. Thank you!

[Aubrey]

OK, I've got the javascript up and running. I have to manually input a time and a location... the next stage is to read these from IMatch - I'll look at json tomorrow. But this maybe where I hand off project.

I would like to tidy up the code a little (all the console.log commands) then I'll zip and others can work with it.

I've checked the results against Exsate android app. It looks good.
I'm getting:
Astronomical twilight, sun -18 - 12 degrees (below horizon)
Nautical twilight, sun -12 to -6 degrees below horizon (this is where one takes star sights at sea - horizon is just visible)
Civil twilight, sun  -6 to -0.8 degrees (below horizon)

Blue hour: sun -9 to -3 degrees (below horizon)
Golden hour: sun -0.9 to 6 degrees above horizon

Note that due to refraction that when we see the lower limb of the sun touches the horizon, in reality the center of the sun is on the horizon. So the -0.9 degrees should be at sunset just before the upper limb dips below the horizon and vise versa for sunrise.

 

[jch2103]

Excellent!

One fly in the ointment, of course, is that unlike twilight, there's no common and excepted definition of 'blue hour'. I have no objection to using -9 to -3 degrees, but I do see some other definitions are also used. For example:

- Wikipedia: No definition, but cites timeanddate.com: "Although the blue hour does not have an official definition, the blue color spectrum is most prominent when the Sun is between 4° and 8° below the horizon."
- photopils.com: "Blue hour (from -6° to -4°)"
- https://app.photoephemeris.com/: "We define golden hour as the time between sunrise and when the sun climbs to +6° above the horizon (and the time between +6° and sunset)." But it doesn't define 'blue hour' at all, instead referring to https://medium.com/the-photographers-ephemeris/magical-gold-and-blue-confusion-in-the-twilight-zone-862ba821f295, which cites several conflicting definitions!
- Other citations of 'blue hour' just use a general description.
- For what it's worth, the guy who posed the challenge on DPReview (which led me to originally post here) gave yet another definition: https://www.dpreview.com/forums/post/64192326 (which eventually led to the usual exchange of insults/opinions on the forum...).

So, as is often the case, a can of worms...

[Aubrey]

I could put in a user selection for altitudes together with default values - lets think about version 2.

I'm making progress on loading the image file data. Based on Mario's hint earlier.
I've watched some videos from Traversy media on how to access json values (https://www.youtube.com/watch?v=wI1CWzNtE-M)

I'm going to work a bit more on the project and see if I can't get together an IMatch app.
(38C (100F)and 25% humidity so not much incentive to work in garden).

Aubrey.

[Mario]

Do you know the IMatch Developer Center? https://www.photools.com/dev-center/doc/imatch/

Especially good for starters is the Recipes section: https://www.photools.com/dev-center/doc/imatch/tutorial-recipes.html

[Jingo]

I could put in a user selection for altitudes together with default values - lets think about version 2.

I'm making progress on loading the image file data. Based on Mario's hint earlier.
I've watched some videos from Traversy media on how to access json values (https://www.youtube.com/watch?v=wI1CWzNtE-M)

I'm going to work a bit more on the project and see if I can't get together an IMatch app.
(38C (100F)and 25% humidity so not much incentive to work in garden).

Aubrey.

When I was learning how to code IM apps, I found both the sample apps and the applications extremely helpful... also, if you review the APP forum, all the apps I've coded and shared are heavily commented to assist others in re-using code bits and learning the ropes.  Always happy to lend a hand so just let me know! 

[Aubrey]

OK it's just running (on 3, not 6 cylinders).
I can select files.
Get the time, convert the time to UTC (Summer time will have to be controlled by a radio switch)*

Then calculate sun's altitude....

I'm not sure how to add say "Golden Hour" to a keyword set. Would someone point me in the right direction for writing back to files - needless to say I'm using a test database and files.

Also need to build html interface - should be OK here even if it's not pretty 

* I thought I'd be able to sort out Summer Time using a longitude based method for European and USA Summer times. This won't work as longitude is not sufficient. Many countries in Africa do not apply Summer time and are at same longitude as Europe!

Aubrey.
PS: It's been interesting - learned a lot. I'm also appreciating (even more) what a genius Mario is as I scratch the surface of the available functions.

[Jingo]

For keyword and other database writing - check out the Metadata Sample APP... or any of my apps that include IMWS.post('v1/metadata',...) functions for example (not tested.. just off the top of my head):

Code Select Expand



  var postData = {        idlist: IMatch.idlist.fileWindowSelection,
                                tag: "hierarchicalkeywords",
                                op: "set",
                                value: [<YOUR KEYWORD]]
                            }

IMWS.post('v1/metadata',postData).then(function(response) {
                     console.log("SUCCESS");
                    },

function (error) {
                        console.log("ERROR");
                    });



[jch2103]

Very good; sounds promising!

* I thought I'd be able to sort out Summer Time using a longitude based method for European and USA Summer times. This won't work as longitude is not sufficient. Many countries in Africa do not apply Summer time and are at same longitude as Europe!

Yes, DST is really complicated. In the US, we have some states that don't switch to DST, some that stay on DST year round, some that are spit, etc., and things are always in flux. Australia is similar. And you already know about other countries. The only way to automate lookup would be to have a geographically fine-grained database that's constantly updated. I have no idea if such a thing exists in non-proprietary form, especially in open source. (I'm sure Garmin, et. al., do.)

[Mario]

I'm not sure how to add say "Golden Hour" to a keyword set. Would someone point me in the right direction for writing back to files - needless to say I'm using a test database and files.

I've posted the links to the IMatch Developer Center yesterday.

On the recipes page, there is a section entitled "Updating Metadata" which explains how to update metadata. For individual files or for all files in an idlist (like the current selection in the File Window).

it even has a sub-section entitled "Adding Keywords"
Pretty much copy&paste for you.

[Aubrey]

I've posted the links to the IMatch Developer Center yesterday....

it even has a sub-section entitled "Adding Keywords"
Pretty much copy&paste for you.

Mario and Jingo thank you for your info on posting.

I cut'n'pasted from Mario's section adding keywords.
All now working writes back nicely.

Now all remains is the html interface so user can decide how to shift time to UTC.
Give a list of files with golden/blue hour and decide allow user to decide whether to write back. keywords.

[Aubrey]

Yes, DST is really complicated...

The only way to automate lookup would be to have a geographically fine-grained database that's constantly updated. I have no idea if such a thing exists in non-proprietary form, especially in open source. (I'm sure Garmin, et. al., do.)

I've done a preliminary look up on lat, long and time offset to get to UTC. Looks like there is an amount out there. This could be added to a version 2. For the moment I plan to calculate a default value and then allow user to modify as he sees fit.

Aubrey.

PS: As usual some project creep is happening!

[jch2103]

Thanks. Sounds very reasonable.

[Aubrey]

Update:
Took yesterday off... (Mario would not approve  ) still working on it.
It is mostly formatting and how I want user to interact.

I have also realized another issue  to consider and code.
What happens if one is on 1st of month and correction to UTC makes date go less than 1? need to back off one month

Also what happens on 1st of year and UTC sets date to less than 1 (This one is more lightly to happen as everybody takes images on New years eve etc.). New year in Australia is still old year at 0 deg long (London).

Aubrey.

[Mario]

Date and time arithmetic can be complex, especially if you throw in time zones, DST etc.

IMatch ships with (and uses in several modules) (e.g. TimeWiz, Map Panel) the fabulous moment.js library: https://momentjs.com/docs/
This library understands and handles date and time arithmetic and makes things like "subtract one hour from this date" easy. And correct.

You can include it in your app by adding

<script src="/system/moment/moment-with-locales.min.js"></script>

or

import moment from 'moment';

if you use ES6 for your app.

How to create a moment.js datetime value from a metadata value as used by IMatch is easy:

const m = new moment.parseZone(metadataDateAsString,'YYYY:MM:DD HH:mm:ssZ');

then you can use m to do arithmetic, like adding or subtracting time.
When you're finished, you can convert m back into a string (using the format method) and store the date back into the database (if your app needs this).

[Aubrey]

Mario,
Thanks. This is useful.
I was starting to write code from scratch to subtract hours and then thought about leap years and added complications... I was about to do a search on js functions for subtracting a few hours... when your timely advise arrived.

Once I get to UTC all works fine.

I'm learning a lot about javascript html ccs (ajax) and putting default values into html. Nothing like a real project better learn.

Aubrey.

[Mario]

'm learning a lot about javascript html ccs (ajax)

Excellent!

JavaScript, HTML and AJAX is what drives the Internet. All the apps on your smart phone use Ajax/REST to communicate with the "cloud" and get stuff done. And many use front-ends build using HTML and JavaScript. If you use the fab Visual Studio Code editor...it is written in JavaScript and HTML.

[Aubrey]

If you use the fab Visual Studio Code editor...it is written in JavaScript and HTML.

Using Apache NetBeans... seems to work fine.

[Mario]

Whatever works for you is good!

[Aubrey]

An update:
You haven't been forgotten!
Working with Ajax which is giving me headaches... but I'm learning.
I've now got the summer time button updating the file list correctly, without rewriting everything.

I've added an additional correction for users who need to make additional changes to get to UTC.
(Time zones don't strictly follow lines of longitude).

I've attached a screen shot of the layout.
User can make changes to time, reselect images in IMatch etc. before submitting to have keywords added.

It'll take a little more time but progress is good.

Aubrey.
PS: I now see why Mario does not give dates of major new releases!! 

PPS: Don't analyse date too much, I've added some "manual offsets" in the js code to get the golden hour appearing, even though time clearly shows it cannot be.

[Mario]

Near interface!

[Aubrey]

The project app can be found at:
https://www.photools.com/community/index.php?topic=10550.0

Aubrey.

[Aubrey]

If you use the fab Visual Studio Code editor...it is written in JavaScript and HTML.

Using Apache NetBeans... seems to work fine.

Starting to use Visual Studio after watching some tutorials by Traversy.
Very nice interface. I've added bash terminal (git-scm.com) so I have a nice unix terminal on bottom of page.

Maybe Net-Beans can do same but looks like Studio has many useful extensions so I'm committing to VS. 

Aubrey.

[Mario]

Visual Studio and Visual Studio Code are two different applications. I guess you mean Visual Studio Code https://code.visualstudio.com/ ?
VSCode is amazing. Especially considering that it is free and comes from Microsoft 

[Jingo]

Visual Studio and Visual Studio Code are two different applications. I guess you mean Visual Studio Code https://code.visualstudio.com/ ?
VSCode is amazing. Especially considering that it is free and comes from Microsoft 

Agree... for free - it is really really good!