Issue: Image viewer displays low resolution image

[iamjwk]

Mario,

I am using automatic caching on two folders containing aproximately 70000 images. Of course this takes some time  When selecting a subfolder, the images were not cached yet and hence the process started for this folder. Thumbnails are fine, but the image in the image viewer is of low resolution. Images are DNG files based on Nikon D800 files. I already deleted the whole cache and restarted the webservices; did not fix the issue. It is also not happening for all images.

When I look into the logfile I see some errors w.r.t. loading file and lines with suggest that thumbnail resolution and preview resolution are the same. From the NUC device running IMatch webservices I can access the image files on the other computer. So rights should be fine. I attached the logfile.

• Can you have a look what is going wrong?


• Is it possible to filter on already cached images?


• How to display the ID you are using for caching beneath the thumbnails, so I can delete these specific cached image files and see what happens when cache images are rebuild for a specific image.

Regards,
Jan-Willem

[Mario]

You are accessing files on a remote system. Do you have configured IMWS to run under a user account which has access to the remote system?

See the section Accessing Files or Databases on Different Computers (Servers, NAS) in the IMatch Anywhere help for more info.

[iamjwk]

Double checked it. I configured it via the control dialog & looked it up in the service properties. It is set correctly.

See the new attached logfile: image D800-7280.DNG loads fine, but D800-7278.DNG doesn't.

Jan-Willem

[Mario]

The first thing I see is a warning about excessive memory usage:

IMWS is using 2.3 GB of RAM, then even over 3 GB (!) which is highly unusual and the source of the problem. IMWS runs out of memory.

This happens while IMWS is apparently scanning a large cache hierarchy for outdated or missing files.
This is usually only a file system check, looking for a file with the proper name.
Some files seem to be missing or not be available in all required sizes so IMWS creates them.

IMatch processes several files, and the memory peaks first at 1.8, then at 2.5 and finally at 3.5 GB.

I see also excessive load times when creating cache images. Some files take almost 3 minutes (!) to load from the NAS and save again.

Unfortunately, the cache update routine does not log the file names of the files it finds missing or outdated, only the cache file names. This makes it impossible to tell you which files are causing the problem. I will add more log output for the next release.

You said you are processing 70,000 files into the cache. And that on an Intel NUC computer, which is barely larger than an USB stick. I'm not sure that these little machines are up to the stress caused by processing 70,000 JPEG or RAW files in one go. Maybe they cause stability problems or just overheat and then throttle the CPU down a lot.

Excessive memory usage is often caused by damaged files which cause memory peaks in the WIC codec (RAW files) or the built-in JPEG processor in IMatch. Since you process so many files in one batch, it's hard to tell what the reason is.

When you search your log file for Peak you'll find the entries which indicate memory peaks. And if you scroll upwards, you'll find the files and folders processed. Maybe this gives some clues about which files are the problem.

The folder reported nearby is

\\PC-WOONKAMER...tscherstrasse (D800)\RAW\D800-7295.DNG with 7360 x 4912

and you process 36 MP RAW files. These require a lot of memory to start with, and if there is a broken file that just happens to not cause a real error/abort but just a massive memory peak, it will be tough to track down.

I see error messages like

Failed to save cache image [117069] E:\Server Folders\Websites - Intel NUC Server\imatch.krusesite.nl\Cache\...

this may be a result of the memory shortage or maybe the disk was full or there was another problem.

I suggest you don't let IMWS create 70,000 cache files in a go. Do it folder by folder.
I will add some more debug logging for the next release. So far there were never problems reported in this area and hence there is no fine-level logging code.
Maybe we can then find out which files are causing the problem.

[iamjwk]

Mario,

Will do automatic cache setup for smaller bunch of files and see what happens. I will target the folder with the possible problem & another one and see what happens.

Note: the intel NUC computer I have is not an USB stick mini computer. Configuration is:
Computer: Intel NUC D54250WY
Processor: Intel Core i5-4250U (2 Cores, 4 Logical Processors)
RAM: 8.00 GB
OS: Windows Server 2012 Standard
Running IMatch anywhere & the server from a SSD.

So should be enough power/RAM to do the job I think. When next update comes with more extensive logging details we can have a look again if I have not nailed it down by then.

Jan-Willem