Writing the first script: various questions/problems that popped up

[ubacher]

The learning curve is steep.....

As I try to write my first script... here the questions I run into:
(Background:  I like to get the selected files and move them to a sub-folder. I attach my script)

1.

Code Select Expand

IMWS.get('v1/files',{
                    // This idlist represents all files currently selected in the active file window.
                    idlist: IMatch.idlist.fileWindowSelection,
                    // we only need the id and the file name
                    fields: 'id,filename'

How can I find out what fields I can specify?                        
I am looking to get the folder/path of the selected files.
( I think I saw a sample script which returns/shows all the fields - I could use
this info but I like to learn where in the docu I find this)
I suspect I will have to go via the Id to get the folder name.

2.
In the VS CODE editor: I ctrl click on:

Code Select Expand

<script src="/system/imws/imwslib.js"></script>
but it tells me :
Unable to open 'imwslib.js': File not found (\system\imws\imwslib.js).

3.
I tried using consolLog in place of console.log
I get the error message that it is not defined.
I can not find it in the IMWS nor the Imatch library.

4. Soon I will want to know:
    Since I don't need to pop up a widow for this app - how do I avoid opening a window
   or (if unavoidable) how do I close the app window once done?
   
5. To get the selected files I should be able to just use getSelectedFiles
   Haven't figured out the syntax and haven't found it in a sample.
   Maybe later I will ask how to remove all code I won't need for this script i.e. make it
   smaller/simpler.

6. What is the best way to search thru a widows folder full of scripts to find a particular text.
    Just the window explorer search? I used to use find from the command line.
    This will allow me to find scripts which contain a particular
    code/function.   
   
Am I the only one struggling with this chore of converting scripts? Is everyone waiting
for others to develop scripts and post them as examples/starting points?

Please: Do post any working examples you have developed - no matter how trivial - so that others may "steal" and learn.
(I did all my basic scripting by copy/paste!!!)

[sinus]

Am I the only one struggling with this chore of converting scripts? Is everyone waiting
for others to develop scripts and post them as examples/starting points?

Please: Do post any working examples you have developed - no matter how trivial - so that others may "steal" and learn.
(I did all my basic scripting by copy/paste!!!)

I think, you are not the only one.

And I can only stress your asking for examples. Would be help for sure.

I personally have not downloaded IM2017 now (but I will of course), but I am quite sure, that I will run into the same and/or other problems in my try, to convert or create new scripts.

[Mario]

How can I find out what fields I can specify?

See the documentation of the /files endpoint for the names of all files you can request.

1. Open the IMatch WebServices Documentation app.
2. In the search box at the top, type files to quickly find the files endpoint.
3. Scroll down until you see the fields parameter.

The fully-qualified file name is returned in the filename field.

Did you study the Files Sample App. It shows all fields and how to work with them.

In the VS CODE editor: I ctrl click on:

Open the folders pane on the left (Attachment 1 below). Then scroll down until you see the system/imws folder, which contains the imwslib.js. But there is nothing interesting in that file.

3.

console.log("Bla");

is a built-in function in JavaScript. Note the . between console and log.
JavaScript is also case-sensitive! Console is not the same as console!

4. All Apps require a window. You can use a modal window that shows up with your app runs and closes itself.

See, for example, the "Progress Bar" or "Process Files" apps which do that.

The Process Files app may be special interesting because it shows how to to the most typical stuff: Show a modal with a progress bar, iterate over all selected files to do something, then close the modal window again. Check it out so see how this works.

5. There are many examples which show how to get the selected files.

6. What is the best way to search thru a widows folder full of scripts to find a particular text.

Visual Studio Code has powerful search functions (The magnifier icon in the toolbar on the left).

1. In VC Code go to File > Open Folder.
2. Open C:\ProgramData\photools.com\IMatch6\webroot

This brings all apps into the folder panel on the left. Now you can look at every app, search the HTML and JS code etc.

Am I the only one struggling with this chore of converting scripts? Is everyone waiting
for others to develop scripts and post them as examples/starting points?

As I said often: There are not many users who develop scripts. Also, I'm just rolling out IMatch. So far only users who monitor this community know about IMatch 2017, I have not yet sent out emails to notify all users. It will take a couple of weeks/month before more users will look into writing apps.

[Ger]

I fully support your scream of help!!

Here's a  first script I did (well, steal with pride, adoption/combination of elements in Mario's scripts). Attached the full script, here's some background.

I removed all the bling-bling Mario includes (fancy layout is for later). I left in jquery)

First idea was to write output in html, so a few html sections described first.
The first scripting part is a 1on1 copy from Mario's scripts: checking for IM and IMWS, auto update of the script.

The function LoadFileData accesses all selected images and grabs some information (metadata).

Code Select Expand


       // Load the data for the currently focused file and show it in the data pre.
        function loadFileData() {

            var params = {
                idlist: IMatch.idlist.fileWindowSelection,
                fields: 'id,name,label',
                tagHeadline: 'headline',
            }
            IMWS.get('v1/files', params).then(function (response) {
                //$('#filejson').text(JSON.stringify(response, null, 2));

                // For each file in the current page, add lines
                for (var x = 0; x < response.files.length; ++x) {
                    thisImageID = response.files[x].id;
                    baseLine = thisImageID + '|' + response.files[x].name + '|' + response.files[x].label + '|' + response.files[x].Headline;
                    outLine = getAttributes(thisImageID, baseLine);
                }
            }, this);
        };


If you want to see the structure of the JSON element containing the file information, unquote the //$('#filejson...) line and the JSON record will be written in the html page (example attached).

I also want to add the attributes for the selected images and add this information to the metadata. For this I added the function getAttributes.

Code Select Expand


       // Load the Attributes per image
        function getAttributes(thisImageID, baseLine) {
            var parAtt = {
                set: 'FotoGY',
                id: thisImageID,
            }
            var tmpArray = [];
            IMWS.get('v1/attributes', parAtt).then(function (response) {
                //$('#attrjson').text(JSON.stringify(response, null, 2));
                for (var y = 0; y < response.result.length; ++y) {
                    for (var z = 0; z < response.result[y].data.length; ++z) {
                        thisLine = baseLine + response.result[y].data[z].Page + '|' + response.result[y].data[z].HTML + '|' + response.result[y].data[z].Sequence;
                        console.log(thisLine);
                    }
                }
            });
            return;
        };


I basically get all the attributes for the attribute set (file, not global) FotoGY and add the elements 'Page', 'HTML' and 'Sequence' to the image line. If an image has more than 1 attribute, a line for every attribute is created.
Every completed line is finally written to the console:

Code Select Expand


console.log(thisLine);


So far so good - and happy with this result. But I'm struggling with the output: it's great to write the line to console (in the getAttributes function, but I am not able to keep the variables available outside of the function.
I basically wanted to add all lines to an array (outLine) but as soon as I leave the getAttributes function, the content of is gone.

My final goal with this script (as in my original Basic script) is to write all lines (containing metadata and attributes) to a text file (for being picked up later in Excel). Help and tips welcome!


As mentioned, I'm struggling a lot with the scripting (but yet determined to find out!) and I support ubacher's ask for support/help!

Mario, you did a great job in providing the examples, but I have to learn Javascript, HTML, CSS and the IMatch libraries. That's for me the reason to simply throw away the layout libraries, for now they're making my life only more difficult.
As ubacher, I'm at this moment at level 0.001 and the most fundamental things are difficult.

Ger

[Mario]

If you want to see the structure of the JSON element containing the file information, unquote the //$('#filejson...) line and the JSON record will be written in the html page (example attached).

Tip: If you run your app in your browser, the "Network" tab (or "Net" sometimes) usually shows you the RAW response from the server (IMWS) and also, because IMWS marks the response as JSON, a nicely formatted). Screen shot from Google Chrome.

That's for me the reason to simply throw away the layout libraries,

You don't need to include the Boostrap library and .js file. This CSS library just makes sure that Apps look good. If you only want to write plain HTML, this will work too.
The most basic app would be:

Code Select Expand


<!DOCTYPE html>
<html>
    <head>
        <meta charset="utf-8">
        <meta http-equiv="X-UA-Compatible" content="IE=edge" />

        <script src="/system/imws/imwslib.js"></script>
        <script src="/system/imatch/imatchlib.js"></script>
    </head>

    <body>
        <h1>Hello World</h1>
        <script>
        </script>
    </body>
   
</html>


It just shows "Hello World" in the App Panel.
If you let the Bootstrap css an js files in the header, it does the same. But looks better ;-)

I basically wanted to add all lines to an array (outLine) but as soon as I leave the getAttributes function, the content of is gone.

In JavaScript there are two scopes for variables: Global and Function.
If you declare a variable inside a function, the variable is only available inside that function.
To declare a global variable, declare outside of any function.

The variable x is declared inside the function Boo and will vanish when the function ends.

globalVar and globalArray are, well, global and are accessible in all functions and the <script> block itself.

See: https://developer.mozilla.org/en/docs/Web/JavaScript/Reference/Statements/var

Code Select Expand


<script>

    var globalVar = 123;

    var globalArray = [];


    function Boo()
    {
        var x = 5;
        globalArray.push(1);
        globalArray.push(42);
     }

</script>


[Ger]

Mario wrote:

In JavaScript there are two scopes for variables: Global and Function.
If you declare a variable inside a function, the variable is only available inside that function.
To declare a global variable, declare outside of any function.

Yes, I know but I keep losing all data. I started to think it has to do with the IMWS calls, but that doesn't seem a logical explanation either.
I defined the outLine array (as well as other variables) as global, but once leaving the function calls... values are gone...

On line 118, the array is filled and displayed, when done with everything at the end of loadFileData (line 144) the array is empty. Must be something I'm overlooking, but I have no clue...

Ger

[Mario]

Line 144 is executed while the IMWS call is still running!

You do the IMWS.get(...) which immediately returns (it is processed asynchronously).
Then you do the console.log(outline).

A few milliseconds later, IMWS replies and your code that fills outline is executed.

Move the console.log(...) into the t

hen(function(response) {...

branch.

[Ger]

When I move like this:

Code Select Expand


       // Load the data for the currently focused file and show it in the data pre.
        function loadFileData() {

            var params = {
                idlist: IMatch.idlist.fileWindowSelection,
                fields: 'id,name,label',
                tagHeadline: 'headline',
            }
            IMWS.get('v1/files', params).then(function (response) {
                $('#filejson').text(JSON.stringify(response, null, 2));

                // For each file in the current page, add lines
                for (var x = 0; x < response.files.length; ++x) {
                    thisImageID = response.files[x].id;
                    baseLine = thisImageID + '|' + response.files[x].name + '|' + response.files[x].label + '|' + response.files[x].Headline;
                    getAttributes(thisImageID, baseLine);
                }
            }, this);
        };
            // Here the outLine array gives empty [] list
            console.log(outLine);
    </script>


I still don't get output... I'm sorry...

[Mario]

Code Select Expand


            IMWS.get('v1/files', params).then(function (response) {
                $('#filejson').text(JSON.stringify(response, null, 2));

                // For each file in the current page, add lines
                for (var x = 0; x < response.files.length; ++x) {
                    thisImageID = response.files[x].id;
                    baseLine = thisImageID + '|' + response.files[x].name + '|' + response.files[x].label + '|' + response.files[x].Headline;
                    getAttributes(thisImageID, baseLine);
                }

// ===> You must place it inside the function that is called when the response from IMWS is processed.
// ===> Then the console.log() is executed after your code filled the outLine array.

                // Here the outLine array gives empty [] list
                console.log(outLine);

            }, this);
        };
    </script>


[Ger]

I feel very stupid now...

I do add the line to the array in the getAttributes function, and the array indeed is filled correctly (line 115). In the getAttribute function i can see the filled array.
But when i return to the loadFileData and finalize, I want to be able to access the array... but it's empty

I'm sorry...

[Mario]

Same thinking error. Typical for programmers who are new to JavaScript or similar languages.
Fell for that myself many times...

It's all about asynchronous processing and when the promises resolve (when the background processing is done).
If you nest asynchronous requests that way, your script no longer behaves linearly.
Many operations run at the same time, and each operation may require a different time to complete.

Your logic is correct, if all operations would 'halt' the script until they are completed. But IMWS.get does not. It is asynchronous. You call it. It does the job and calls you back when it is done. That is the then logic.

Now...

1. In your loadFileData you call IMWS.get to get some data for all selected files.
2. Then this request has been completed, you iterate in a loop over each file. For each file you call your getAttributes function.
3. getAttributes calls IMWS.get to fetch the Attributes for one file. This is an asynchronous call.
4. getAttributes returns to loadFileData while the request to load data from IMWS is still running inside it.
5. The loop continues and calls getAttributes for the next file.
...

This means that, most likely, several calls to getAttributes are processed at the same time, each taking a few milliseconds. The then(function(response... part of getAttributes is called as soon as IMWS has returned the data. But not necessarily in the same order in which you called getAttributes. And probably some getAttribute calls are still running when your for loop in loadFileData is already completed. Maybe each call takes 0.1 seconds and you call it 10 times. The loop itself probably takes 0.00001 seconds to complete...

Doing one asynchronous call is simple. The then(...) logic is easy to use, once understood.
Doing multiple asynchronous calls at the same time is also simple. Same logic.
Nesting multiple asynchronous calls...now that's more difficult. This is actually quite advanced for your first script.

There are several ways to handle such more complex scenarios (nesting multiple asynchronous calls). I have demonstrated that in multiple sample scripts.

Before I go into the details:

Did you notice that the /attributes endpoint can retrieve data for any number of files!
You can just specify the same idlist you used for the /files request and get all Attributes for these files back in one request call.
I would recommend using this.


Doing it the hard way (nice learning experience, may cause some gray hairs as well):

What you actually want to do in your loop is:

1. For each file in this list, fetch the Attributes.
2. When the data for all files has been retrieved, continue with this program part.


You need to wait for all outstanding getAttributes calls to resolve first. jQuery has a when() function that takes multiple promises (each call to IMWS.get() returns a primise) and when all promises are resolved, when calls a function.

This is create when you have to wait for a specific number of promises. For example, when you need to load the user info from IMWS and the general info and when both is available call IMatchTranslate to translate your script.

In your situation this will probably not work because your loop may process 5, 100 or 1000 files...or 100,000.

I recommend the following approach:

1. in loadFileData you retrieve the list of files to process.
2. Store this in a global variable and also have a global index.
3. The getAttributes function processes one file, the file at the current index position.
4. In the then() part it increments the index and stores the attribute data it has received i your global array.
5. It then calls itself (!) if there is another file to process (index < filelist.length)
5. If all files have been processed, it calls a function that does whatever you want to do with the attributes.

[Ger]

Mario, thanks for that long explanation, much appreciated - I will try tomorrow (not behind my database now) and report back!

Same thinking error. Typical for programmers who are new to JavaScript or similar languages.
Fell for that myself many times...

Hmm... I don't consider myself as a programmer (the last 30 years). Especially not after translating IMatch 5 - it gave me a good insight in all the balls a true programmer must keep in the air at the same time - the core functionality probably being the easiest. But then it starts: optimizing/performance, leveraging windows/third party dlls, prioritization of processes...

Ger

[ubacher]

I have just been struggling with exactly the same problem: Asynchronous calls - tried nesting and got unexpected results.

Here my program: I thought it was simple enough for a start!

I want to get the selected files.
If only one or too many files selected I want to pop up a warning (window.alert) and when it is acknowledged I want to exit the script
    (something like "exit sub" is what I look for)

I want to move the selected files into a sub-folder.
   To get the folder name I use splitPath and then put together the folder name.

(Eventually I will first want to check that all files are from the same folder and only then do the move.
I will also have to handle the case of the sub-folder not yet existing.... And other safety checks)

Mario: Can you explain again how to handle this? Do I have to use a "when" function to "serialize" my calls?
and
How does a script exit? When all promises have been fulfilled?

(I will study Ger's script, maybe I can then make sense of you explanations. Need to cool my brain first).

[thrinn]

Yes, I am also struggling 
But to check if at least one file hase been selected, I would use the following snippet (which is, by the way, taken more or less from the processfiles sample app):

Code Select Expand


            // We first get the id and file name of all selected files in the active file window.
            IMatch.get('v1/files', {
                idlist: IMatch.idlist.fileWindowSelection,
                fields: 'id,filename'
            }).then(function (response) {
                // Store in a variable and start processing
                files = response.files;
                if (files.length < 1) {
                    alert('Select at least one file!')
                }
                else {
                    // Call a function to do something with your selection
                    // do_something( files);
                }

            })

I use the JavaScript alert function because it is easy to use for a simple message.
Because the do_something function is called only if the promise has been fulfilled and takes all files at once as parameter, there should be no async problems - at least at this point...

[Mario]

An app never apps. It is loaded into the App panel and then the browser runs the code in the <script> section. If there is no script or the script decides to do nothing, nothing will happen. Just the HTML code will be visible.

"Modal" apps run in a separate window, not in an App Panel. This makes these apps ideal for what you have in mind. You can control when your script "ends" by calling modalClose() from your script code. You can also control how large the modal window is, what it shows etc.

Have a look at the sample Apps Progress Bar (which counts up to 100% and then exists) and Process Files which shows how to write an app that iterates over all selected files, showing a progress bar. Select maybe 20 or 50 files before running it for a good effect.

I want to get the selected files.

Define "get". What do you want to get? Information about the files or just how many files are selected?
The typical way would be to call the /files endpoint which returns information about files. And to specify the idlist "IMatch.idlist.fileWindowSelection" which represents all selected files in the active file window. This will give you info about how many files are selected and whatever info you want about each file.

Did you look at the Code Recipes, which has examples exactly like the one above, with explanations? Look at the "Iterating over All Selected Files" example there. This is copy/paste code you can copy into your <script> and then see how it works.

For example:



With two selected files this returns the following data in the console:


2 files selected.
D:\data\__File Format Test Suite\GPS\Berlin\berlin-957470.jpg
D:\data\__File Format Test Suite\GPS\Berlin\berlin-86190.jpg
ORIGINAL RESPONSE:
{
  "files": [
    {
      "id": 4338,
      "fileName": "D:\\data\\__File Format Test Suite\\GPS\\Berlin\\berlin-957470.jpg"
    },
    {
      "id": 4336,
      "fileName": "D:\\data\\__File Format Test Suite\\GPS\\Berlin\\berlin-86190.jpg"
    }
  ],
  "hiddenByStacks": 0,
  "debug": {
    "runtime": "0ms"
  }
}


You can copy/paste this code into your app and then look at the output panel in IMatch. Or, better, open your app in your web browser, <F12> for the developer tools and then look at the console there. Set a breakpoint in the line with console.log(response.files.length... and then step through. Look at what the variables contain etc. to get a feel for things.

Don't forget to check out my Code Recipes. Very useful resource.

[thrinn]

Do I have to use a "when" function to "serialize" my calls?

My understanding (at the moment!) is: Yes, you have to use the then part of the promise because you need the result of the get. The list of selected files, in this case.

Alas, to work with the result (the files variable),  you can now use an ordinary JavaScript function. The asynchronous calls come into play again when you call some "internet" functions (IMWS endpoints or IMatch functions).

[Mario]

As explained in great detail on the jQuery web site (https://api.jquery.com/jquery.get/) there is a lot of functionality in the AJAX get request. I just wrapped that for you in the IMatch.get() and IMWS.get() to shield you from too much complexity.

You MUST use the then() logic for all get/post/delete calls.

JavaScript does not wait for web servers to respond - it is designed to provide responsive experiences.
Your script continues immediately when get() returns. It does not wait.

Keep in mind: A IMWS.get(...) call can take 0.1s or 10s - depending on how many files you are processing, how much data there is to return, how busy IMatch or IMatch WebServices are etc.

If you work with the IMWS embedded in IMatch, response times are usually 0.1 seconds or less.
If you run your script against an IMWS 6000 miles away, response times may be 1 or more seconds.
Your script can make any number of get() calls at the same time. IMWS will process them all and respond to each.

So, IMWS.get(...) just means: "Get me the data and call me back when you are done".

The "call me back when you are done" is the then() part. And there is also an optional error part, to handle errors:

Code Select Expand


IMWS.get(...).then(function(response) {
  // Success, response contains whatever IMWS has returned.
},error(status) {
  // Opps, problem reaching IMWS (network error, invalid endpoint etc.)
});


get takes a number of functions in addition to the endpoint name and parameters. The first function is called when the call is successful, the second when the call fails.

Whenever you do an AJAX call (AJAX is the fancy name for the technology that lets JavaScript interface with web services) you use the then() logic. All IMatch endpoints. All IMWS endpoints. When you use other web services (GeoNames, Google, Flickr. ...) it always works like that.

And there is even more. For example, .always(function...  can be used to run code whether or not the call is successful. Very important if your script needs to do something in all cases...

jQuery web site: https://api.jquery.com/jquery.get/

jQuery tutorials: https://www.w3schools.com/jquery/

This takes a while to settle. I can recall that it took me a while to wrap my head around all this asynchronous stuff and the Promises. But this is the way modern applications work.

Have a look at the Code Recipes for more examples and copy/paste code.

[Ger]

I'm afraid I'm still too much in my outdated 3rd generation coding ideas 

Just discovered the HTML Report app, maybe this helps a bit

to be continued...

[Mario]

Looked at the coding recipes yet? Easy to grab working code from there and to re-combine in your app.

[Ger]

I learned a lot last days, but a much bigger lot is still to be done!
Still struggling with my first tryout: writing some file data + attributes into a text file.

From Mario's comments and the code recipe page I understood that the then-clause will be executed after a promise is delivered. I embedded this in my script and the script runs perfectly fine... with the output to the console:

Code Select Expand


        // Load the Attributes per image
        function getAttributes(thisImageID, baseLine) {
            var parAtt = {
                set: 'FotoGY',
                id: thisImageID,
            }
            var thisLine = [];
            IMWS.get('v1/attributes', parAtt).then(function (response) {
                for (var y = 0; y < response.result.length; ++y) {
                    for (var z = 0; z < response.result[y].data.length; ++z) {
                        thisLine = thisLine + baseLine + response.result[y].data[z].Page + '|' + response.result[y].data[z].HTML + '|' + response.result[y].data[z].Sequence + '\r\n';
                    }
                }

                IMatch.writeTextFile({
                    filename: 'd:\\pcxtemp\\imdata ' + thisImageID + '.txt',
                    data: thisLine,
                    //append: true
                }).then(function (response) {
                    console.log(thisLine);
                });
            }, this);
        }


        // Load the data for the currently focused files
        function loadFileData() {

            var params = {
                idlist: IMatch.idlist.fileWindowSelection,
                fields: 'id,name,label',
                tagHeadline: 'headline',
            }

            IMatch.writeTextFile({
                filename: 'd:\\pcxtemp\\imdata.txt',
                data: 'header line\r\n',
            }).then(function (response) {

                IMWS.get('v1/files', params).then(function (response) {
                    response.files.forEach(function (f) {
                        thisImageID = f.id;
                        baseLine = thisImageID + '|' + f.name + '|' + f.label + '|' + f.Headline + '|';
                        getAttributes(thisImageID, baseLine);
                    }, this);
                });
            });
        };


Every attribute gets its own line on the console, nicely listing file data and attribute information. So far so good.
However, then (I think) the asynchronous aspect comes back when trying to write the line to a text file. When trying to add all output files to one text file (imdata.txt), only a subset is written, probably because the writeTextFile-routine is too slow (as mentioned, the console output in the then-clause is perfect.
If i create a text file per image (as in the code above), a text file for every image is written and contains all attribute lines.

I've been trying several things to a) store the thisLine into an array and b) find a way to write this array to a text file after all processing has finalized. But maybe my frame of thinking is wrong. Any help, ideas, code, tips, tricks are welcome...

---
Talking is easy... show me the code (Linus Torvalds)

[Mario]

Why not buffer everything in a string variable and then call WriteTextFile once at the end?
Even if your script produces many megabytes of output, this should work fine.

Also it seems that you call /attributes for individual files...this either means you are doing multiple /attributes call simultaneously or you are doing some other strange things...

You can call /attributes it once for all the files you want to work on. Much more efficient, much easier to handle the response.
If you want to get the attributes for all selected files, for example. This would simplify your script as well.

1. You get all the attribute data you are interested it in one call.
2. You iterate over the result (which contains a separate set of attribute data for each file)
3. You build whatever you want to write into the text file in a string in memory.
4. Finally, you call writeTextFile  once.

For example: To get all attributes for the set 'Notes' for the files currently selected in the file window, you do:

Code Select Expand


IMWS.get('v1/attributes',{
    idlist: IMatch.idlist.fileWindowSelection,
    set: 'Notes',
}).then(function(response) {
    console.log(JSON.stringify(response,null,2));
});

That's all! Copy/paste this in your script, put in the right Attribute Set name and watch the output in the Output panel or the browser console.
If you only want specific attributes, use the attributes parameter in addition.

IMWS returns the following JSON response:

1. result is an array. For each requested file (in IMatch.idlist.fileWindowSelection) it returns one record.
This record consists of an id (the file id) and a data element.

2. The data record is an also an array. It contains 0 to many Attribute records.

In this example, file 2250 has two Attribute records, file 2251 has only one.

{
  "result": [
    {
      "id": 2250,
      "data": [
        {
          "instanceId": 1,
          "Note": "This is a note 1",
          "Description": "Description 1",
          "Count": "0"
        },
        {
          "instanceId": 2,
          "Note": "This is yet another boring note for this file",
          "Count": "50"
        }
      ]
    },
    {
      "id": 2251,
      "data": [
        {
          "instanceId": 1,
          "Note": "This is a note 2",
          "Description": "Description 2",
          "Count": "0"
        }
      ]
    },
...


Now we produce an output file from that.

1. The first row is the header
2. The columns are separated by a tabular (ASCII 9): JavaScript uses \t for that.
3. The records are delimited with carriage-return / line-feed (ASICC 13,10). JavaScript uses \r\n for that:

Code Select Expand


IMWS.get('v1/attributes',{
    idlist: IMatch.idlist.fileWindowSelection,
    set: 'Notes',
}).then(function(response) {
    // console.log(JSON.stringify(response,null,2));

    // The header
    var data = 'File Id\tNote\r\n';

    // For each returned file record
    response.result.forEach(function(record) {
        // NOTE:    A file may have more than one entry in data.
        //          We add one row in the output for each note found.
        record.data.forEach(function(d) {
            if (d.Note !== undefined) {
                data += record.id + '\t' + d.Note + '\r\n';
            }
        });

    });

    // Dump to console, write to text file, whatever...
    console.log(data)
});                   


The output looks like this:


File Id   Note
2250   This is a note 1
2250   This is another note for file 1
2251   This is a note 2


Note: The

if (d.Note !== undefined) {

just checks if the file has a Note Attribute. It may have none, and then we don't produce the row.

[sinus]

Ha! 

This thread is very interesting for  me, because it handles the only real problem what I have to change to IMatch2017.

I have in VB a script, what does read all attributes out of a file, does some calculating and put this calculated fields into a word-doc, neatly formated.
At the moment I am not able to do such a script in JavaScript and I have also not the time for this - because instead 1 or 2 hours, what skilled programmers would use for this, I must for sure have 1-2 weeks.

Pull out some fields from one or more files is not that difficult, and put it in a string or in a text-file is also quite easy possible, but put the different fields nicely in another program like Word is another thing, what is for me very difficult.

But this thread and the explanations in the last post from Mario will help me later a big deal, I think. 
Hence, thanks for this thread.

[Mario]

You can always format your output text file as HTML. Word can import HTML very nicely, so you can even produce tables that way.
The example above basically produces CSV, which can be imported straight into Excel or Word as well.

And doing a few sums and percentages in JavaScript with data from Attributes is easy.

IMatch ships with an example script named HTML Report which does exactly that. Check it out. Import & Export panel.

[sinus]

You can always format your output text file as HTML. Word can import HTML very nicely, so you can even produce tables that way.
The example above basically produces CSV, which can be imported straight into Excel or Word as well.

And doing a few sums and percentages in JavaScript with data from Attributes is easy.

IMatch ships with an example script named HTML Report which does exactly that. Check it out. Import & Export panel.

Oh, good idea, html into word. Never thought at that, thanks, Mario!

I will surely have a look at your pointed html-Report.

[Ger]

Getting all attributes for all images in the selection works like a charm, also the output to the text (csv) file is ok.

In your example you only take the Attributes record into account, where in my desired output every attribute record consists of some v1/files data (name, label, headline etc), completed with the attributes information (Notes.Note in your example).

Sample output file:
ID|File name|Label|Headline|Attribute.Page|Attribute.HTML|Attribute.Sequence
175|20060606_165137_1074-LR.jpg|Green|Blue wildebeest in Kgalagadi TP|Beautiful images|satopff|2
175|20060606_165137_1074-LR.jpg|Green|Blue wildebeest in Kgalagadi TP|Flora and Fauna - Mammals in focus|ff_blue_wildebeest|3
180|20060607_173641_0721-LR.jpg||Late afternoon sun on the main road through the Auob river bed|Kgalagadi TP - Landscape|parks_saf_ktp_landscape_auob|7
180|20060607_173641_0721-LR.jpg||Late afternoon sun on the main road through the Auob river bed|Beautiful images|satoplandscape|8
180|20060607_173641_0721-LR.jpg||Late afternoon sun on the main road through the Auob river bed|Dagboek 2006|dag20060607|9

Sinus: i promise to post the full script once it's working!

[sinus]

Sinus: i promise to post the full script once it's working!

Ger, thanks a lot, would be cool! 

Your project is really very interesting. 
Finally this is, I think, often the case, that we pull some information out of a file (files) and work then with them.

I wish you further good luck ... and hope, I can later also profite from your script. 

[Mario]

where in my desired output every attribute record consists of some v1/files data (name, label, headline etc), completed with the attributes information (Notes.Note in your example).

No problem there.
You first do your /files call, and when this completes you do the /attributes call for the same set of files.
Now you have two sets of data. You can easily find what is related by the file id, right?
Iterate over the files in the response. Then search the matching attributes in the response from /attributes. Combine as need to produce new rows for your output.

[Ger]

Thanks, I haven't yet tried this, but I'm just wondering whether this approach would be best in terms of performance.
The current VB script runs over about 13.000 images which have some 20.000 attributes, which means that for every attribute I have to loop over 13.000 images (until found).

This was the reason I originally wanted to process by file (as I did in the VB script).

[thrinn]

I have to loop over 13.000 images (until found).

The files endpoint return an array of objects. Each object has an numerical id (the internal file id). Now you could create a new array with the id as array index and the object (representing 1 file) as content. If you loop over your attribute, you can access the file info in your new array directly, using the file id as array index.

I must admit I didn't try it. But I think this approach would work and give a good performance.

[Mario]

Thanks, I haven't yet tried this, but I'm just wondering whether this approach would be best in terms of performance.
The current VB script runs over about 13.000 images which have some 20.000 attributes, which means that for every attribute I have to loop over 13.000 images (until found).

This was the reason I originally wanted to process by file (as I did in the VB script).

Calling /attributes 13,000 times instead of once is real bad for performance. Iterating over an array in memory is much, much faster than doing Webservice calls. Chromium handles this in almost zero time.

[Ger]

My export script works! Thanks to Mario and his useful replies.
If you see the end code, it's looks (and actually IS) quite simple.

Will play and try with the program a bit and post the program (a bit more tidy) on the forum.

Ger

... and then on to the next level!

[Mario]

If you see the end code, it's looks (and actually IS) quite simple.

This is normal. The next script will be easier, and the one afterwards again.
Then you'll throw everything away and do it again - with all the know-how you then have 
Or, in new-speak hipster language you 'refactor' 

[Ger]

It took some time, but my first script is converted to Javascript. It's actually one of the more straightforward programs: creating a text file (tab delimited).
The data in the text file contains attribute information per image. For every set of attribute information a line is added to the file; the line contains some image metadata as well.

The attribute set the program is looking into is named 'FotoGY' (which is defined as a constant). The elements within this attribute set are Page, HTML and Sequence. Important to know in case you want to run the program yourself. If the image does not have these elements, no output is generated

The app will work with the files selected; processing starts with asking the path and name of the output file. After that the actual processing starts with getting the id's and metadata of the images in scope. Some of the metadata is changed a bit (some keywords are filtered out, geography information is manipulated).
For every image ID the attributes are requested from IMatch and for every attribute a line is generated.

Finally, the collected output date (in the variable OutputData) is written to a text file.

What did I learn?
JS works asynchronous, which means that program execution will continue while a request to (e.g.) IMatch is made. So if you 'need' the output ('promise') from IMatch, you have to make sure you only continue processing after the  'promise' has been fulfilled. That's where the  then(function clause comes into play.

JS arrays can be leveraged flexible. Traditionally, an array started with element 0 (or 1) and the index went up sequentially. JS gave the opportunity to use a more flexible index: by using the ImageID element as array index, processing became easy. All image info was added to an array with the imageID as index. When this information is required later, the imageID can be used to very efficiently pick up the info.

I'm fully aware that also the coding itself can be nicer, better etc, but my first focus is to have (a few) things running and step-by-step understand more and get better.

With posting the JS (in the attached zip file) I hope somebody can get something out of it. I at least like examples and investigate how and what they function.

Up to the next steps... many things to do!

Ger

[Mario]

The first script is always the hardest.

[Ger]

The first script is always the hardest.

Hmm... before I started i thought i picked a very easy script to start with

The nice thing about this is that we (ok: I) are forced to jump into something new. Besides all the swearing and frustration, it's also fun the explore a bit!

Ger

[sinus]

Ger,

congratulations, you did it! 

I wish, I had also the time now, but IMatch2017 and scripting must wait a bit (except sometimes in the night with my slow laptop).

Finally your script runs, great (I could not test it yet), this is the most important thing. The next script you will write even better. 

[sinus]

Please: Do post any working examples you have developed - no matter how trivial - so that others may "steal" and learn.
(I did all my basic scripting by copy/paste!!!)

Right, Ubacher, exactly!

And this is what I did with your script 

Thanks a lot. You would not believe it: I tried and tried and tried read Marios comments, help (as far as I undstood them  ) and I was not able to write a small script. No chance.
I tried also Marios app "my first app" ... even this was too complicated! 

I took your script ... and it worked!!! 

Well, to be correct, almost.
First I selected (randomly) only one image and then a warning pop-up-window has been displayed, that I should select more than 1 file.

Cool, then I knew, at least your lines worked in my IMatch-app.

I selected 3 images and nothing happend.
Then I saw in your lines something about console. But how can I choose or made visible this console? I have only IMatch and the app. I have read about console but was not able to find where I can choose this. I guess, Mario cannot imagine, what troubles users like me has to create a simply script. What he does during the sleep, I do not understand or/and have hours for figure it out.

Finally I saw, that you had a windows alert for the warning. So I changed the console log against also the window alert box ...

and success! Suddenly a windows alert box popped and showed me the name (I believe) from my 3 images ... it popped 3 times on.

Thanks, Ubacher, you made my day! 

This gives me the first time some lines to start.
All other stuff, what I have read or tried from IMatch and the help (Mario) was simply too complicated, still.

I think, what users like me must have, is a kind of VERY easy start.

Like I would explain a camera for a you child:
I would not say, you must first clean the camera, would not explain time and lenses open and would not explain something from blue and yellow light and would not explain how to sharpen the lenst.

I would say something like:

take the camera
look through the viewer
push the button

That's ist.
Then he would have first a success. If the resulst is too dark or unsharp or yellowish or whatever: does not matter. He/She could then work on that step by step.

The first success is important, I think.
With your lines, ubacher, you gave me this, thanks. 


BTW: I want not say, that Mario did not a great job! He did so. I guess, even a fantastic job.
But users like me are specialy "dumb" and must have specialy easy starting points.

[Mario]

Did you see the Code Recipes. There are copy/paste examples, e.g. for iterating over all selected files (just copy/paste into "My App") or for getting the focused file and many other things.

Learning to program is never easy. You said the same you said above 10 years ago when IMatch introduced the BASIC language 

[ubacher]

Sinus: you can see the console.log output in the Output panel (script tab). Just like with VB.

You can also try:
When you call up the app in the app panel select the icon to put the app into the browser.
I use firefox. Press f12 and then F5 to run the app. On the bottom half of the screen you should be able to see a console window showing the output.
(Have not managed to use breakpoints etc. yet.)

Mario points us to the sample apps but of course they never quite show what we want to accomplish. That's why it is so important for "scripting users" to
share their scripts. Users usually have similar scripting ideas/needs.

[sinus]

Did you see the Code Recipes. There are copy/paste examples, e.g. for iterating over all selected files (just copy/paste into "My App") or for getting the focused file and many other things.

Learning to program is never easy. You said the same you said above 10 years ago when IMatch introduced the BASIC language 

Hm. 10 years?  Hmm, could be.

I will try, thanks!

[sinus]

Hey, Ubacher, thanks a lot.
That is maybe the solution.

I will try to do so, thanks and have a good time! 

[Mario]

(Have not managed to use breakpoints etc. yet.)

See (Firefox)  click to zoom image:



Your break point will be hit when you reload the page.
At the shown location, the /info call will have returned the result and the app is creating the table from the contents.
You can now look at variable contents by pointing at them or by setting watches like in VB.