Problem with DNG files

[vbt]

After loading all my files onto IMatch I noticed a problem with DNG files. Namely when opened in Viewer or Quickview they seem to only be showing the embedded preview file (which is smaller in size) and vertical files are appearing as horizontal (including in all thumbnails).

This seemed odd as I hadn't noticed the problem when testing. So I opened the file in an earlier IMatch Database I had used for testing and this time the same photo appeared full size.

The following database settings were the same in both databases:
Application - Prefer photools.com RAW Processing  - No
Cache - Allow thumbnail enlargements beyond 100% - No
Cache - Use Embedded Preview - No
Autostack copies - Ask
Autostack Version Copies - No


I also ran WIC Diagnostics on the same photo in both databases and the output seemed to be the similar -

From Database that works i.e. displays DNGs full size:

WIC: Testing file 'D:\Pictures stored on IMatch\2019\2019-02-27\20190227-1412-2.DNG'
   Thumbnail: Codec 'DNG Decoder'
      (GetThumbnail failed (88982F44 The bitmap codec does not support a thumbnail.).) 0x0 pixel in 0 ms.
   Preview: Codec 'DNG Decoder'
      (GetPreview failed (88982F81 The operation is unsupported.).) 0x0 pixel in 0 ms.
   Full resolution: Codec 'DNG Decoder'
      () 1024x769 pixel in 0 ms.

IMatch Integrated RAW Processing.

Analyzing image 'D:\Pictures stored on IMatch\2019\2019-02-27\20190227-1412-2.DNG'
    Preview: (Format:1, Dimensions: 1024x769) in 0ms. Result: 0
    Original Image: (4608x3464 4608x3464) in 2281ms. Result: 0

Version: 0.19.0-Beta1


From Database that fails i.e. only displays smaller image (and with vertical photos horizontal):

WIC: Testing file 'D:\Pictures stored on IMatch\2019\2019-02-27\20190227-1412-2.DNG'
   Thumbnail: Codec 'DNG Decoder'
      (GetThumbnail failed (88982F44 The bitmap codec does not support a thumbnail.).) 0x0 pixel in 0 ms.
   Preview: Codec 'DNG Decoder'
      (GetPreview failed (88982F81 The operation is unsupported.).) 0x0 pixel in 0 ms.
   Full resolution: Codec 'DNG Decoder'
      () 1024x769 pixel in 16 ms.

IMatch Integrated RAW Processing.

Analyzing image 'D:\Pictures stored on IMatch\2019\2019-02-27\20190227-1412-2.DNG'
    Preview: (Format:1, Dimensions: 1024x769) in 0ms. Result: 0
    Original Image: (4608x3464 4608x3464) in 2313ms. Result: 0

Version: 0.19.0-Beta1


Finally tried with photos from two cameras (both Panasonic) and both listed as supported by IMatch Integrated RAW Processing.


So why is the photo not displaying correct in one database but not the other?

One other thing, in the database that is working I noticed the thumbnail in the DNG file originally displays horizontal, but fixes itself after forcing a rescan by selecting it, then pressing cntl, shift F5, force update. But in the database that isn't working a forced update does not change anything.

[Mario]

The installed WIC codec cannot handle the DNG format variant you use.

WIC: Testing file 'D:\Pictures stored on IMatch\2019\2019-02-27\20190227-1412-2.DNG'
   Thumbnail: Codec 'DNG Decoder'
      (GetThumbnail failed (88982F44 The bitmap codec does not support a thumbnail.).) 0x0 pixel in 0 ms.
   Preview: Codec 'DNG Decoder'
      (GetPreview failed (88982F81 The operation is unsupported.).) 0x0 pixel in 0 ms.
   Full resolution: Codec 'DNG Decoder'
      () 1024x769 pixel in 0 ms.


it can only extract a 1024 pixel image and considers it as the full-size RAW.

Either save the DNG files in a standard format that is suppored by the DNG codec installed on your system (I assume this is the Windows codec, not sure from the small except of the WIC diagnosis you have posted) or disable the use of WIC codecs via Edit > Preferences > Application: Favor photools.com RAW processing. This switches IMatch to use LibRaw. Then select the files with problems, press Shift+Ctrl+F5 and use the "Force Update" option to tell IMatch to re-process the files.

Why the installed WIC codec does not handle the DNG files you use - I don't know. Adobe and other vendors really have made a big mess out of the DNG format, pushing out so many different variants over the years. If you are using the built-in Windows 10 codec, you may want to contact Microsoft for support. I don't know how you created the DNG files, from camera or by using a software...

You can also try to install the latest set of WIC codecs from Microsoft.

In all cases, you will need to do a forced update of affected files in IMatch (Select then Shift+Ctrl+F5). Installing a new WIC codec or switching from WIC to IMatch's built-in RAW processing does not cause IMatch to re-create all cache files.


As always when reporting such issues, include

- the IMatch Logfile (log file) from a session where you processed some of your files (e.g. scanned a folder)
- All settings under Edit > Preferences > Cache
- The WIC diagnosis log file (an except is nice, but having the complete file with the header and footer data is better.
- Send me a sample DNG file to support email address (include a link to this thread in your email).
This will show me if the problem also happens on my machine, a clean W10 installation, what the Microsoft WIC tools say about the file etc.

RAW processing is not database-dependent. Why this should work with one database but not the other - I have no idea. The RAW processing in IMatch is always the same. Unless you configured different cache and RAW-processing options for both databases, of course.

[vbt]

Thanks for the quick reply.

The bit that helped was "Why this should work with one database but not the other - I have no idea. The RAW processing in IMatch is always the same. Unless you configured different cache and RAW-processing options for both databases, of course."

So I looked at the cache settings and the database that wasn't working had "minimal size" on the cache tab in preferences set to 640 whereas the database that was working had this set to 2000. So the database that I thought wasn't working was correctly using the embedded thumbnails, as these were 1024x769. So I changed the value to 2000 on that database and then forced the files to update. Which then produced the right result. (Hooray!)

So the reason was simple, but I have no recollection of changing that setting, but accept I must have.

The WIC codec issue was presumably irrelevant as the protools.com must be working, and is called either as default or when WIC codec fails.

Also for some other reason (that is not important) if the embedded thumbnails are used for these DNG files then the orientation of vertical photos is not correct.

Thanks again.

[Mario]

If the WIC codec fails, IMatch falls back to using LibRaw. This is what will happen for your installation.

And, apparently (search for ORF here in the board for more info), LibRaw returns that the image needs to be rotated, even if it must not. Apparently sometimes the preview is stored in a 'corrected' orientation in the RAW file (ORF only so far), but this is not reflected in the EXIF data returned. So IMatch rotates the file (because Exif tells so) but should not. This so far only affected ORF files with embedded RAW.

Using a proper DNG WIC codec would solve this. Are you sure your system is up-to-date? Usually the Windows DNG codec just works. Unless your DNG files are very special - and this could probably cause some archival nightmare in a couple of years. DNG is supposed to be standardized and not cause issues with the standard Windows 10 WIC codecs for this format.
Try the Adobe DNG coded, if you use Adobe software. Maybe this is yet another specialize Adobe has added to DNG because they needed it for one of their products - and now everybody else has to play catch up to support that special DNG flavour - or not...

[vbt]

I am not certain my windows DNG codecs are up to date. I don't have any problems but then I probably only ever use windows to display thumbnails (e.g. when using Windows Explorer) and these thumbnails display correctly but windows explorer is probably just using the embedded jpegs.

The DNGs I have will not be unusual - common cameras with DNGs generated by Lightroom. I also don't intend to generate new DNGs. In future I will just keep the raw files alongside jpgs. I also intend to regard the raw files, in some ways, as secondary to the jpgs. I intend to always store a final jpg after being happy with any development of the raw file, because I don't want to always be reliant on keeping the raw engine that generated the final image. (I do still want to retain the raw files in case I ever wish to generate a new image from that file. One of the biggest strengths for me, of IMatch, is good version handling.) 

I can "fix" the orientation of all my existing DNG files (and I don't have that many vertical DNG files) using EXIF data to find the files and then forcing them to update. Since I won't be generating new DNG files this won't be an ongoing task.

PS I think the WIC Codec Download Sites link target for Adobe DNG on https://www.photools.com/1167/wic-support-codec-availability/ must have moved.

[Mario]

Probably Adobe does not make a DNG codec anymore. I can only find archived information.
Maybe they rely on Microsoft for updating the Windows 10 DNG codec whenever Adobe comes up with an "extension" - which they do whenever they need something for one of their products (Lr, mos often). Then it can take months or longer for the DNG codec to update and support it (at least this was the case when Adobe "enhanced" DNG to support fast previews, which they needed to make Lr faster).

I have no problematic DNG files in my sample collection. All work here with the FPV codecs and whatever is included in Windows 10 (clean install).
Send me one of yours to support email address if you like (with a link back to this thread so I know what's what).

[vbt]

I have just sent an email with a large file.

That said, I am happy with the way everything is working now. However it would be interesting to know if the WIC codec on your PC handles this file.