Understanding setTimeout and file processing

[dcb]

I am working on a script to copy several folders from Dropbox into my iMatch import processing folder. I'm building up from the file processing app, plus a script Mario posted recently that gives me a nice solution to iterate over the multiple folders. Things work okay until I add the second loop and it seems to be in the setTimeout function that's causing something else to happen which I don't understand.

Here's the script:

Code Select Expand

            var files = [];
            var folder = '';
            var index = 0;

            $(document).ready(function () {

                var foldersToCopy = ['',' from David'];
                var nextFolderIndex = 0;

                // Now fill all folders to be copied into the foldersToCopy array...

                // Copy the first folder. This function calls itself when a folder has been copied until all folders have been copied.

                moveNextFolder();


                function moveNextFolder()
                {
                    if (nextFolderIndex  >= foldersToCopy.length)  {
                        return; // All done.
                    }

                    IMatch.folderScan({
                        foldername: 'C:\\Users\\XXXXXX\\Dropbox\\Camera Uploads' + foldersToCopy[nextFolderIndex],
                        filemask: '*.mov,*.png,*.jpg,*.mp4'
                    }).then(function(response) {
                       // Folder copied. Copy the next folder
                        files = response.folders.files;
                        folder = response.folders.path;
                        index = 0;
                        $('#headline').text('Processing ' + files.length + ' file(s)...');
                        nextFile();

                        nextFolderIndex++;
                        moveNextFolder();
                    });
                }

               
                // Handle the abort button
                $('#btn-stop').click(function() {
                    // We just move the index, the rest will
                    // be done automatically in nextFile().
                    index = files.length;
                    $('#btn-stop').prop('disabled',true);
                });
            });


            // This function "processes"" the next file.
            function nextFile() {

                // If all files have been processed, move the
                // process bar to 100%, wait one second and then
                // close the modal window.
                if (index >= files.length) {
                    $('#pbar').css('width','100%');
                    $('#pbar').text('100%');

                    setTimeout(function() {
                        IMatch.modalClose();
                    },1000);
                    return;
                }

                // Update the file name
                $('#filename').text(folder + files[index]);

                // Update the progress bar
                var perc = Math.floor((index * 100) / files.length);
                $('#pbar').css('width',perc + '%');
                $('#pbar').text(perc + '%');

                // Wait a short time, then call this function again
                // to process the next file.

                console.log(folder + files[index]);

                setTimeout(function() {
                    index++;
                    nextFile();
                },50);


When I run it this way, I get one file from the first folder, and all the files from the second.

If I change the final few lines as follows I get all filenames on the console. I don't at this point see the progress bar update because it's too fast. That's not necessarily a problem for processing but impacts me because I don't understand why there is a difference and if the index and nextFile instructions are within the timeout it only appears to fire once before moving on.

Code Select Expand


setTimeout(function() {
  ;
},50);

index++;
nextFile();


Thanks in advance, David

[Mario]

1. setTimeout(function() {
  ;  2. // The code you place here is executed when the timeout expires (after 50 milliseconds).
},50);

This code is processed immediately (while timeout is still waiting!).
3. index++;
4. nextFile();

Your code executes 1. 3. 4. 1. 3. 4. ..... and then 2. after 50 ms.
You call setTimeout probably hundreds of times before the first call expires after 50ms.


What you probably want is

Code Select Expand

function nextFile()

setTimeout(function() {
  // Do some work with the file
  index++;
  nextFile();
},50);


Call nextFile every 50ms, when the timer expires.

[dcb]

Thanks Mario. That's what I understand should be happening but not what I'm seeing when I run it. When I have code at 2, it executes only once.

Let's say I have 2 files sets, A and B, each with 3 files in them. What I should see is

Code Select Expand

A.1
A.2
A.3
B.1
B.2
B.3

To get that I have had to move the instructions outside of the timeout function. Because, if inside the timeout function (as with your original example) code I get:

Code Select Expand

A.1
B.1
B.2
B.3

This only occurs when I have the second folder of files (the B set). If it is just the A set then I get:

Code Select Expand

A.1
A.2
A.3

In other words, on the first of 2 sets of files, "2. // The code you place here is executed when the timeout expires (after 50 milliseconds)." executes only once. I can't see why this would be the case.

Thanks, David

[Mario]

setTimeout is a JavaScript function and has nothing to do with IMatch. Please see the documentation for setTimeout in the MDN for details about setTimeout works and how you can apply it in your JavaScript code. If you put code outside setTimeout in your code it will be executed while setTimeout is still running, or simultaneously. I recommend using the Debugger in your browser to see what's going on.

[thrinn]

Or to put it another way: setTimeout is not something like "wait 50 ms doing nothing, then continue with the rest of my code". It executes what is inside the function you supply to setTimeout after 50ms, but executes immediately and in parallel whatever follows the setTimeout call.
In your case I suspect the nextFile function will return while setTimeout is still waiting, causing the statements after the nextFile to execute in parallel. That means, your code moves to the next Folder while the first setTimeout is still waiting.

I would have to try for myself (unfortunately, I am not accustomed enough with JS yet to have only to look at some code to be sure whta happens), but maybe this is some hint where to look.

[Mario]

Advanced JavaScript constructs like setTimeout can be hard to grasp, even if you have some experience. Using the Debugger in a browser and setting breakpoints at the begin of the nextFile function, inside setTimeout and below the call to setTimeout should make clear what's happening. But running in a Debugger of course causes timing differences between the normal execution and the debugging. Probably temporarily increasing the 50ms in setTimeout to 1000 ms or so will make this more clear.

As thrinn said, setTimeout does not halt execution of nextFile. It is just a way to say "Run this code in 50ms". All other code continues to run.

[dcb]

That's what I'm seeing. Still trying to learn my way around it.

[Mario]

Quick: https://www.w3schools.com/jsref/met_win_settimeout.asp

Detailed: https://developer.mozilla.org/en-US/docs/Web/API/WindowOrWorkerGlobalScope/setTimeout
The MDN is the reference.