the 'process files' sample app

[peterverm]

i have a question regarding the 'process files' sample app.

Looking in index.html, i find that it calls nextFile() once, and that nextFile() calls itself, see extract below.

function nextFile() {

...

    // NOTE: This is where you would do some actial work
    //       with the file.

    setTimeout(function() {
        index++;
        nextFile();
    },files.length < 50 ? 100 : 50);
}

It works fine, no problem, but it forces sequential processing of the files.

I wanted to try parallel processing, so i changed the first call to

for (MyFilesIndex = 0; MyFilesIndex < FilesLength; MyFilesIndex++) {
    processFile(MyFilesIndex);
};

where processFile treats only 1 file and returns.  As a result, processing of all files is started in parallel.

Works also fine.  I tried for maximum 28 files, and then i decided to ask your advise.

Is there a maximum number of files that i can launch in parallel ?  Or should i revert to your solution of sequential processing ?

I asked the question because, although it worked previously, today IMatch stopped working.  I include the log.  Maybe you can advise ?

Thanks,

Peter

[Mario]

My sample script calls calls processFile via a timer. This has two effects:

a) it allows the app to simulate that processFile takes some time (else the process bar would just jump from 0 to 100) and
b) it de-couples each call of processFile. processFile does not call itself recursively. Instead the timer calls it asynchronously so no recursion happens.

You can make multiple calls to IMWS endpoints in parallel.
Each call will consume resources and block a connection for the duration of the call. The maximum number of parallel connections is limited to 50.
It also depends on what you are actually doing in your app. Some operations cannot be performed in parallel (for example, you cannot delete and rename the same file at the same time).

Your app seems to be causing write-backs, and the log file is full of

Cannot write to off-line/read-only file C:\Users\peter archive\Pictures\199x\2

warnings. And, more importantly,

Database is on a read-only media or the current user has no write privileges to the folder containing the database file.

I don't know what your app does, but it seems IMatch was suddenly unable to write to the database file. Virus checker?

[peterverm]

Mario,

thanks.

1. the "Cannot write to off-line/read-only file" warning has nothing to do with my app, i caused it manually.  Now, I made the files read/write.  So this is OK now.
2. i 'll keep your warning about 50 parallel connections in mind, and yes, as i programmed it (without timer) the app is lightning fast.
3. my app works again - i did no changes.  I include a log of today of a fresh start.  Do you see anything weird ?

Regards,

Peter

[Mario]

I searched the log file for E> and found:

AppManager::Error parsing file 'C:\ProgramData\photools.com\IMatch6\webroot\user\update-attributes-1\app.json': 'default document does not exist.'
AppManager::Error parsing file 'C:\ProgramData\photools.com\IMatch6\webroot\user\update-attributes-2\app.json': 'default document does not exist.'
AppManager::Error parsing file 'C:\ProgramData\photools.com\IMatch6\webroot\user\update-attributes-3\app.json': 'default document does not exist.'

This means that the defaultDocument specified in the app.json (usually index.html) does not exist. The App Manager cannot run these apps because it does not know which HTML file to load.

[peterverm]

OK, these are old directories, i removed them.  Now, there are no more E> in the log file.  We can close the post.

Thanks for the help.

Peter