Updating to 2018.10.6

[dwags]

Hello,
I have tried updating to the new version of IMatch, but I get an error message that reads:

Preparation error
There was a problem preparing the downloaded file or while transfering it to IMatch. Please restart IMatch and try again.

I have restarted IMatch as well as my computer on several occasions, but nothing works.

What am I missing?

Dale Wagler

[jch2103]

Try downloading the update file directly from https://www.photools.com/customer/. It's possible the original download got glitched during the download process.

[Mario]

The manual download from the customer portal of course works.

But I would like to get to the bottom of this issue in the automatic updater which seems to affect only a few users...
The IMatch log file from that session (see log file) will contain detailed information about what went wrong. The preparation step is where the app stores the file on the local hard disk so it can be executed. My guess is that this triggers some virus checkers which then block the write operation. Or something of that kind.

[dwags]

Thanks for your reply Mario.
I have attached a copy of the latest log file.

Dale

[Mario]

Do you get this result every time you are trying to download the update?

The error message s a bit fuzzy but it seems to indicate that either the path was not found (the TEMP folder on your system...?) or that Windows could not or did not want to save the file to disk.

This is always the error when a user reports that the Updater App fails. So far recall 3 maybe 4 users where this has failed. Probably something like a virus checker interfering or something...??

[dwags]

I had always been trying to install from the "update available" within IMatch, and continually got the preparation error message, even after disabling my anti virus.
I finally took John's advice and downloaded from the web portal, and got a message to the affect that Windows Smartscreen could not determine if this is a trusted site. I simply clicked download anyway and all is good.

Thanks to all.

Dale

[Mario]

You can always download updates from the customer portal. The App is just a convenience feature.


This Smart Screen nonsense becomes a real PITA.

If you use a Microsoft browser (Jeez!) it sends information about every download you make to Microsoft. Microsoft records for each file downloaded the URL and the number of downloads, and compares that with the MS databases. The AI then determines if the file has been download often enough, if it is signed with a certificate, if the file has been reported as a virus etc.
If a file has not been downloaded often enough, Microsoft browsers flag it as "potentially insecure, because our AI does not know it". That's what MS calls "Smart Screen" for normal users.

All my installer files are signed with my personal digital  certificate, which is a pain to get and authorize for and which costs me a lot of money every year.
But that's no longer enough.
To suppress the Smart screen message from the start, Microsoft now demands a "extended digital certificate" which costs thousands of dollars and requires a local notary / lawyer to certificate that I am me. Stupid. I won't spend that much money, sorry. Even the notary would cost several hundred dollars just to sign the papers I need to begin with the authorization process for the extended certificate.

[Aubrey]

I can't remember when I last used the Microsoft browser.

Chrome or Opera for me... I've started using Opera again after leaving them for a few years.

I've killed as much of Cortina as I am able on my machine
Aubrey

[Jingo]

Well - for so many folks that don't know better, MS Edge is what gets thrown up on the screen for each Win 10 installation... so, they think "hey - my browser"... that's how MS likes it just like Apple wants folks to only use it's preloaded Safari.

I've used Firefox pretty exclusively over the years with a bit of Opera lately... must admit the latest version of Firefox is 100x faster and more stable than earlier and I"m happy I stuck it out through the "bad times".. many of the addons I use dont' work on the new version... but there are alternatives.

[Carlo Didier]

I must confess that I'm switching browsers several times a year. Went with IE for some time, then switched to Firefox for no specific reason, then to Chrome.
Went away from Chrome because it was a huge memory hog, so went with Edge for a while and then back to IE again.

Now back to Chrome again because they got a better memory management and I found the OneTab extension very useful. Can't say though that I'd prefer any browser over the others. For what I use them (i.e. maybe 2% of their "features"), they're all the same.

For me, the so-called browser wars are about lots of features I never use or need. Browser have just become bloatware.

[Mario]

I use FF for browsing.
I use Edge for testing only.
I use Chrome for development because a) it is the same engine used by IMatch (Chromium) and it b) has the best tools for developers.

[greg]

Hello,
I have tried updating to the new version of IMatch, but I get an error message that reads:

Preparation error
There was a problem preparing the downloaded file or while transfering it to IMatch. Please restart IMatch and try again.

I have restarted IMatch as well as my computer on several occasions, but nothing works.

What am I missing?

Dale Wagler

I continue to have the same problem as described. I have gone to using a direct download from the iMatch website, but it bothers me that there is something causing a problem.

[Mario]

Please switch IMatch to debug logging and repeat. Attach the log file in zipped form. If it's the same Windows "cannot save file" error we'll have another hint.

[greg]

Please switch IMatch to debug logging and repeat. Attach the log file in zipped form. If it's the same Windows "cannot save file" error we'll have another hint.

I'm already up to date so I'll try that on the next update. If there is a way to force a download again for debug purposes I would try that.

[Mario]

Yes. You can edit the file

C:\ProgramData\photools.com\IMatch6\webroot\imatch\apps\updater\src\app.js

In line 250 or thereabouts you'll see

Code Select Expand

if (IMatchInfo.developerMode) {
    //vinstalled = vavail-1;
}


Remove the // comment in front of vinstalled:

Code Select Expand

if (IMatchInfo.developerMode) {
    vinstalled = vavail-1;
}


and then run the App. It will report that an update is available.
Don't forget to undo the change afterwards.

Which virus checker do you run?

[greg]

Norton Antivirus.

Will give your suggestion a try after I grab a cup of coffee  .

[greg]

I tried as shown, but I must have missed something. The updater app still said I already had the latest version installed. I added the statement within the if to outside and the same thing, nothing changed.

Code Select Expand

let vavail = svavail[0] * 10000000 + svavail[1] * 1000 + svavail[2];
            let vinstalled = svinstalled[0] * 10000000 + svinstalled[1] * 1000 + svinstalled[2];

            if (IMatchInfo.developerMode) {
                 vinstalled = vavail-1;
            }
            vinstalled = vavail-1 //delete this when done

[Mario]

Probably a cache issue.
You can delete the browser cache while IMatch is not running. Delete the folder C:\ProgramData\photools.com\IMatch6\browser

[greg]

Hi Mario. I deleted browser, deleted all chrome history, changed/saved the file and restarted the computer. Still the updater says the version is current. It seems like there is caching somewhere else.

[Mario]

Hm, works here. Do you start the Updater App from the App Manager (The normal update check via the menu will not be modified by this).

[greg]

After I found developer mode in preferences I was able to capture the failure.

[Mario]

Download OK. WebServer ran into error 3 during upload or saving the file on the disk. Windows error 3 usually means "path not found", which would indicate that the TEMP folder does not exist, which cannot be. None the wiser, unfortunately.

[greg]

Clearly the TEMP directory exists and when I checked TEMP contents IMATCH has written files to it. Perhaps the mystery will someday become clear as this issue occurs for at least 2 of us, whatever the commonality might be. I spend time on these things as they indicate something different on my computer, configuration, software or potentially virus.

[Mario]

This is very mysterious.
I've compared a log and a debug trace from one of my machines with your log. I see no field_stored entry, which means that the web server never stored the file or never received it...?

Can you please repeat your test, this time running the Updater in Google Chrome (or FF, if you don't have Chrome).

Just open this URL while IMatch is running:

http://127.0.0.1:50519/imatch/apps/updater/index.html

Click <F12> to open the debugger tools and switch to the Network Tab.
Then click download in the app.
When you click download, the app will first download the file into memory, and then make a POST request to the IMWS running in IMatch to store the file on disk (Apps cannot directly store files in the file system).

In Chrome this looks like in the attached image (see below).
The App is calling the upload IMWS endpoint to upload data just downloaded to IMWS in IMatch, which then saves it to disk.

Does this look similar on your machine?

[greg]

Hi Mario,

Upload is in red and the right hand message is "Failed to load response data". See attachments. Attachment 4 is the information associated with addlogentry.

[Mario]

This matches what we see in the log file.

What I would like to see are the headers (see my screen shot) of the upload request. This shows us the data IMatch sends to IMWS, the size of the data etc.

[greg]

Here they are.

[Mario]

Thanks. This looks good. The file is transferred correctly and with the proper headers.

Still, the web server does not call the handler in IMWS which activates the file transfer.
What usually happens is that the web server software tells IMWS that a form request is processed and IMWS waits for a specific field (filename) to provide the web server with the name under which the file should be stored. This is logged in the IMatch log file, but fails in your log. This means that the web server never called IMWS back. I have no idea why.

[greg]

Hi Mario, Thanks for spending time on this. Is there any further checking that can be done from my end? Can you see anything from the web server end?

[Mario]

There is nothing I can see on the server side. The file has already been downloaded to your system, but the Updater app cannot save the file to your hard disk because the web server embedded in IMatch does not process the form request (upload) correctly. I will need to analyze the source code of the web server, ask questions in the board etc. to dig deeper into this. I will do so when I find the time. Or I need to find a way to reproduce that here, but I already tried four computers.

[greg]

You have put in quite a bit of effort already. This issue currently manifests itself as low impact and it occurs infrequently. In my experience this issue would be classified as low severity. Perhaps the effort spent could turn out to be useful if related issues come up in the future.

[Mario]

Yes, it's prio 3. But it bugs me nevertheless.

I wanted the Updater App to make it easier for all users to download/install updates. That's why I've spent time designing and developing it.
That it does not work for some users is unfortunate and I don't like this. But currently my work plan is full, up to 120% until IMatch Anywhere 2018 is out.

[KchoPrro]

I am another affected by this problem, I upload my LOG in case it is helpful.

I will install manually, thanks!

KchoPrro

[Mario]

Same error (path not found) as with the other posters. I've also noticed that your memory is utilized almost to a 100%, which is pretty rare these days...?

[KchoPrro]

Same error (path not found) as with the other posters. I've also noticed that your memory is utilized almost to a 100%, which is pretty rare these days...?

Yes, I just wanted to leave a record here and add the LOG, I know that, right now, there is no solution and it is not a problem either because we can manually install the update. I just added my answer in case it helps.

Indeed, Imatch may need 100% of my memory, although I have 4GB of RAM, my motherboard only allows 3GB, my computer is already a few years old but it is enough for what I need.

Thanks for the reply, as always, Mario.

KchoPrro

[Mario]

IMatch uses less than 300 MB RAM, but the memory was utilized to almost 100%.
Maybe run less applications at the same time. With a memory utilization that high Windows may be forced to swap memory to disk and this makes everything much slower.

1.  Which virus checker do you use?
Since this problem happens only on a few PCs, it might be the virus checker somehow blocking IMWS to write the file...

2.  Can you switch IMatch to Debug logging (Help menu > Support > Debug Logging) and repeat the test, and attach the log file from that test?
This has more info about what is going on when the Updater is trying to upload (save) the downloaded file to disk.

[Carlo Didier]

... although I have 4GB of RAM, my motherboard only allows 3GB, my computer is already a few years old but it is enough for what I need.

That may well be a problem and explains why your memory is used 100%. On my system with 16GB RAM, an average of 3-4GB is already in use without any applications running. Just starting a browser like Chrome already eats up 500-900MB!

[KchoPrro]

... although I have 4GB of RAM, my motherboard only allows 3GB, my computer is already a few years old but it is enough for what I need.

That may well be a problem and explains why your memory is used 100%. On my system with 16GB RAM, an average of 3-4GB is already in use without any applications running. Just starting a browser like Chrome already eats up 500-900MB!

It is possible, although iMatch only needs 300MB of RAM, Windows 10 Pro x64 already uses a lot. I do not use Chrome, I use Firefox, but it can also get a lot of memory because I usually work with several tabs at the same time. At the moment I did not consider changing the equipment, although I was studying the possibility of expanding the memory to 8GB by changing my 2x 2GB modules by 2x 4GB. My motherboard does not support anymore.

IMatch uses less than 300 MB RAM, but the memory was utilized to almost 100%.
Maybe run less applications at the same time. With a memory utilization that high Windows may be forced to swap memory to disk and this makes everything much slower.

1.  Which virus checker do you use?
Since this problem happens only on a few PCs, it might be the virus checker somehow blocking IMWS to write the file...

I usually use iMatch, Photoshop CC, Canon Digital Photo Professional and Firefox but the normal thing is just iMatch and Firefox (only when I want to edit a photo I open the other two applications).

I do not use antivirus (Windows Defender ), I have the system recently formatted and clean, I prefer to work with clean copies of the system, every so often, I restore and everything improves.

2.  Can you switch IMatch to Debug logging (Help menu > Support > Debug Logging) and repeat the test, and attach the log file from that test?
This has more info about what is going on when the Updater is trying to upload (save) the downloaded file to disk.

You mean to update iMatch in that mode? It's late for that, I downloaded the update and installed it manually, no problem in it, it was very easy.
Edit:: Ok, I upload the log

[Mario]

I usually use iMatch, Photoshop CC, Canon Digital Photo Professional and Firefox

There you go then. Photoshop is a known memory hog and usually reserves 50% of the installed RAM in a non-cooperative (!) mode. Windows cannot longer make this RAM available to other applications.
Even starting a modern Web Browser will cost you 500 MB or more. Each tab maybe another 100 MB.

I'M still proud that IMatch can manage a 500,000 files database in less than 1 GB RAM. Your web browser needs the same amount of memory just to display 3 web pages