Viewer extremely slow

[grauschaf]

After Updating IM to 2019.1.4 the File Viewer is almost not usable. I loaded 10 RAW-Files into the viewer. To show 2 of them side-by-side takes about 30 sec. There is a the loading... message visible. The next two images take 30 sec as well. Turning back to the first two images (which were already loaded) I have to wait again.
Closing the viewer also takes some time if I interrupt the loading process.

No Problem with jpg files, only with raws!

From time to time I get the message "This file could not be displayed". After reopening the viewer they are displayed. This problem is still present. Maybe this information helps.

I provide the log file and the result of the WIC Codec check. I use the Fast Image Codec Pack. Hope for help

[Mario]

I see tons of error messages like

"Der Prozess kann nicht auf die Datei zugreifen, da sie von einem anderen Prozess verwendet wird."

in your log file. It seems that something is blocking Windows WIC from accessing the files in the cache...?
For files which are not affected by this, the load time is about 0.4s per cache file. Still a bit on the slow side but OK.

I also the message "Only thumbnail or small preview extracted. Falling back to ptc RAW processing." many times. This is usually logged when IMatch tries to read a RAW file (e.g for creating the cache image) and the WIC  codec can only extract a 160 pixel thumbnail or a preview smaller than the size configured for the cache (in your case 2000 pixel, but the embedded preview is only 1920 pixel). Apparently loading the full RAW also gives files with only 1920 pixel so IMatch then tries to load the file using LibRaw again.

Of course al this slows down IMatch massively.
Have you tried to reboot your system and close other applications when running IMatch?

[grauschaf]

Hi Mario
Thank you for your prompt help. Yes, I have closed all other applications and restarted my computer.
I have also made a test with another picture viewer. There are no problems with the raw files.
You mentioned that IMatch tries to get a cache image of 2000 px and the preview of the Fuji RAW is only 1920px. Does IMatch always uses the embedded preview of raws to generate cache files? If so, why do I need the WIC Codec then?
Do you think it might be a good idea to set the cache size to 1920px? I'll try this evening.

[Mario]

IMatch by default uses the embedded preview. I have explained how IMatch uses previews, full-size RAW files, how to configure the cache size in the help:

The Cache

1920 is a HD preview and quite small for modern monitors. IMatch uses a default of 2000 pixels for good reasons. To get a decent size cache image even if the embedded preview is too small.
Experience tells me that files either contain a full-size (100%) preview or the preview is smaller than 2000 pixel.

IMatch reads previews or fill RAW via the installed WIC codec. Always. Or via LibRaw. IMatch does not manually fiddle with RAW files to extract the preview differently than the full RAW.
For most WIC codecs there is only a very small performance difference between extracting the embedded preview and developing the RAW. Of course the embedded preview is to be preferred because it usually looks like what you have seen in camera. For DNG, the preview looks like whatever you have seen in your RAW developer.

Show us a WIC diagnosis from one of your files. This will tell us which codec is installed, and what it can access.
Select a RAW file in a file window, then go to Help > Support > https://www.photools.com/help/imatch/#wic_diag.htm

Have you perhaps tried to view files in the Viewer while IMatch was writing metadata back to them? Or which other process could have looked your RAW files and images in the IMatch cache folder (!) while IMatch was trying to access them? Your "other" software has no problems because it probably does not try to load JPEG files from the IMatch cache folder, right? Or it may not use Windows WIC installed on your system.

To me thus sounds like a virus checker blocking IMatch or something. Check the detailed logs of your virus scanner to see if it has blocked files, and which.

[grauschaf]

Hi Mario. I attached the WIC diagnostic result to my initial post yesterday already

[grauschaf]

The WIC Diagnosis shows the analysis of the raw file took about 17 seconds.

[Mario]

   Thumbnail: Codec 'Fuji Raw Decoder (FastPictureViewer Codec Pack)'
      () 160x104 pixel in 0 ms.
   Preview: Codec 'Fuji Raw Decoder (FastPictureViewer Codec Pack)'
      (GetPreview failed (88982F81 Der Vorgang wird nicht unterstützt.).) 0x0 pixel in 0 ms.
   Full resolution: Codec 'Fuji Raw Decoder (FastPictureViewer Codec Pack)'
      () 1920x1280 pixel in 0 ms.


Have you configured the FastPictureViewer codec to always provide the embedded preview? This is an option and it looks like you have. The preview and the full resolution have the same size (1920 pixel) and I doubt that this is correct. I think this is an option that you can set when installing the FPV codecs. You can force it to always use the embedded preview, no matter what.

Since the cache size is by default 2000 pixel and your WIC codec only delivers 1920 pixel, this is what happens:

1. IMatch tries to load the preview. It comes out too small.
2. IMatch tries to load the full RAW. It also comes out too small.
3. IMatch falls back to loading the file with the LibRaw codec.
4. Preview still too small.
5. Full RAW works.

This takes of course a lot more time than using using the embedded preview in step 1.
And, we still don't know why so many of your cache file (the JPEG files in the IMatch cache folder) are reported as locked by Windows when it tries to load them via WIC.

[Carlo Didier]

1920 is a HD preview and quite small for modern monitors.

But maybe not for many users, especially non-professionals.
Would be interesting if you could see some stats about monitor resolutions from your telemetry (the stats would at least have 4 monitors from 2 PCs from me which are "only" 1920 HD ...)

[Mario]

Switch the minimum preview size to something below 1920 under Edit > Preferences > Caching. IMatch will then use the small previews in your RAW files.

But this does not solve the problem. If you tell the Cache than 1920 pixel is good enough, you will never be able to zoom in or see your files at 100% in IMatch!
You should definitely configure FPV codecs to return the full RAW (requires re-install? when I remember correctly).

[grauschaf]

I Re-Installed FastPictureViewer Codec Pack. No settings during installation available. Windows Photo Viewer, FastStone and Photo Mechanic show pictures in a size of 1920 x ... px for Fuji RAF Files and even smaller for Sony ARW Files. Nikon NEF Files are shown in full resolution. Same as IMatch.
My problem has nothing to do with the Update of IMatch.

In IMatch I changed the Preview Size to 1920px. No problem with the viewer any more (except the low resolution ).

My next Steps:
- Try to get information concerning the FPV Codec. Why only a preview in 1920px resolution?
- Have a look for Fuji Film Codecs (X-T2, X-T3) and Sony RX100.

A recent thread deals with a similar problem: https://www.photools.com/community/index.php?topic=7194.0

[Mario]

Have you looked at the Windows control panel and there at the FPV options? (Open the Windows control panel and search for Fast)

Make sure you have not checked this option:



If you have, uncheck for all and force IMatch to do a forced reload of affected files using Shift+Ctrl+F5.

Damn, I'd with Microsoft could force camera vendors to supply a codec for their cameras so MS can integrate them (and automatically update them) with Windows automatically.

[grauschaf]

Hi Mario
The hint with the Codec Pack Control Panel was the solution for one of my problems. The Codec Pack forces to use embedded previews of my RAW files. Unchecked it, set the IM Cache to 2000px and every is ok .

One problem is still existing - the file viewer message " This file cannot be displayed" and the log file entry "Der Prozess kann nicht auf die Datei zugreifen, da sie von einem anderen Prozess verwendet wird". There are no other appications blocking my file and I'm not shure if this is the reason. The behavior of the viewer is strange. Example:

- 3 Files in the viewer. 2 files are displayed simultaniosly. Picture 1 and 2 are displayed when focus is on picture one.
- Putting focus on picture 2, both are still displayed.
- Putting focus on picture 3, picture 2 has the problem and is not displayed. Picture 3 is displayed.
- Going back to picture 1, picture 2 is displayed again.
- Stepping through all pics in the viewer, closing viewer and reopen it with the same pictures -> all pictures are displayed.
- The amount of not displayed pictures varies. Could be one of ten or 6 of ten. Sometimes they are all displayed.

[Mario]

Maybe something on your system is so slow that, when the Viewer tries to load the same file multiple times, it breaks and blocks the file?
Each Viewer window is separate and does its own background caching and pre-loading of files. If it happens that two windows try to load the same file at the same time...but IMatch/WIC open the files in read-only and shared mode so this should never be a problem. I have not experienced this, not even with 8 files in view at the same time. Is your disk unusually slow perhaps?

One reason for this message would be that two windows try to load a file from the cache, don't find it and create it. When both windows write the file at exactly the same time, they may run into problems. But that is expected and IMatch will just retry a few times. If this fails and Windows still returns "file locked" you will see this message.

Have you tried to move the cache folder to another disk? Exclude it in your virus checker? ...

[grauschaf]

The cache is located on c:\ which is a SSD that should be fast enough. I excluded it in the bitdefender software.
I have to take some new pics with my camera to see wether the exclusion make it better. The problem occurs only with files which haven't been displayed already.
I will post my experiences within the next days. Thank you very much for your help Mario.

Gerd

[Mario]

You create cache files on-demand and view multiple new files at the same time in the Viewer side-by-side.
Do you experience the "locked" problem also when you only view one file at a time?

[grauschaf]

Do you experience the "locked" problem also when you only view one file at a time?

Yes, the problem doesn't depend on the amount of simultanously displayed files in the viewer (I think ). To be sure I will do some more tests.
Question: After the files have been loaded once to the viewer the problem does not exist any more. Is it a good idea to delete the preview cache folder to get the problem again?

Another idea: I use compressed Fuji RAW files. I will try with uncompressed files too.

[Mario]

When you clear the cache folder, IMatch will re-create cache files on-demand again.
If this somehow produces a problem on your machine, this will be useful to force it.
But one Viewer panel window (one image at a time) means only one background thread which pre-loads files. Not much chance of locking, unless some 3rd party software or component interferes...

[grauschaf]

Did some research on my problems:

1. The problem "File can not be displayed":
- I deleted the Preview Cache Folder and startet IMatch
- I opened the viewer with 6 RAW files.
- Two of them could not be displayed even if there is only one file in the viewer window
- There are 6 jpg files in the cache folder. I have full access to both problem files (able to rename them), so I don't think they are blocked by another process.
- I closed the file viewer and reopened it. No Problem any more.
- I repeated this procedure. Problem looks different every time.

2. Problem with very long loading time
- I set the Minimal Cache Size to 1920px (size of Fuji XT2 preview). Fast loading now but preview only in 1920px. Because I don't the the chance to solve the problem at the moment this is ok for me.

Question: Watching the content of the cache folder I don't see a difference between the two cache working modes (on demand and default). In both modes five previews are generated when I load files into the viewer. Browsing the viewer window more files are generated. I expected a different behavior in default mode.

I hope there is something helpful for you,
Gerd

[Mario]

Default creates cache files when you add files to your database (or existing files are rescanned because of an update).

I have no idea why Windows reports file locks on your system. IMatch loads cache images via the built-in WIC JPEG codec in Windows (or if another JPEG codec is installed, by whatever takes precedence).

On your machine this looks like "Windows, loads file "bla.jpg". Windows: "Cannot, file is locked": That's all I can tell. I have never experienced this and I'm not sure about what I could do. The Microsoft documentation just says that this error code is returned when the coded is unable to access a file because of file permission errors or file locking...

If you see this problem in the Viewer even if only one file is open...and even if all files are cached already...
The error code I see in the log is not even documented a a possible error code for the Load... function....

[grauschaf]

I've got used to this problem over years. Stepping through all files in the viewer, closing the viewer and reopen it solves the problem. So I can live with it.
Thank you for your help. And if there is something new on my problem I will report it in this thread.

Gerd