NEF Thumbnails Not Showing

Started by TomH, April 15, 2025, 02:53:49 PM

Previous topic - Next topic

TomH

I have occasionally had issues where I could not see certain thumbnails within specific folders. To be clear, it only applied to a few files, not all of them. The only way I have been able to correct this is by removing the folder from the database and then adding it back in. Purging and rebuilding the cache never helped.

Yesterday I had this happen again but to a larger group of files (32 in total). I ran the file verifying tool, not knowing what it would do, and the results were 32 files with checksum errors which equates to how many files that are not showing thumbnails. I don't know where to go from here.  My workflow me going back and forth between Imatch and DXO Photolab 8. Is there something that could be corrupt the files? I don't know where to go from here to both solve the issue and also prevent it from happening again.

Mario

The File Verifier compares the checksum IMatch has created when ingesting a file with the current checksum of the file.
If there is a difference, the file was modified without IMatch noticing. Either the file was modified by the Windows "last modified" timestamp was not changed (deliberate action) or the file has been damaged somehow.

If IMatch cannot read a NEF file, there are about 20 possible reasons.
And it will log the reason to the log file (see log file).

Select one of the problem files in a File Window and then run Help menu > Support > WIC diagnosis.
This will try to load the image using the NEF WIC codec installed on your system and also via LibRaw.

Save the result and ZIP / attach to your reply. This will tell us what's what, and I can advise.

Open Edit menu > Preferences > Application. Search for photools. The state of "Prefer photools.com RAW processing" (use LibRaw) is by default on since IMatch 2025. If this is off on your computer, IMatch is using the installed WIC codec that ships with Windows (unless you have installed the Nikon NEF codec).

There are now over 50 or so different NEF variants in use, all of them undocumented. The ability to read these files or to at least extract the embedded preview image depends on whether or not the WIC codec or LibRaw support the NEF variant. If it is a new camera with a new NEF format, it can take months until Microsoft and the LibRaw project catch up. The joy of using undocumented RAW formats.

You don't need to remove the folder from the database.
You can just select the NEF and press Shift+Ctrl+F5 to re-read the file as it was new. This will create the thumbnail, cache image etc. If the file is pending for write-back (yellow pen icon) write-back before forcing a reload to avoid losing metadata you have entered in IMatch.

If this happens again, make sure to make a copy of the log file (Help menu > Support > Copy Logfile) so we have something to work with.

graham1

I often have this, with Canon CR3 files and Sony ARW files. I can now fairly well predict when it will happen, which is when there is a large ingest or moving of files. In fact, as I type this I am waiting for IMatch to re-read a large number of files to create thumbnails - I rearranged in another application about 10,000 safari images into folders representing different types of animals. IMatch picked up the changes, but maybe 3-4,000 thumbnails did not survive the moves. 

I have stopped worrying about it.  All I do is select the files without thumbnails then ctrl-right click to rescan with a full force update. This always works. Sometimes the update completely fixes this, other times the files get the yellow pencil to update the metadata. I just do this in downtime and let IMatch do its thing while I am doing something else. Strangely, even though the thumbnail might not have been generated, the image is visible in the Quick View window, which I always have visible in my default screen layout. 

My guess as to why this happens is that with large 45 or 60 MP files, reading a whole load of them all at once and their metadata does not leave enough resources to generate the thumbnails. Whatever the reason, I have learned not to worry about it because there is a simple solution. It would be nice to have an option simply to force update of thumbnails (I have not found it, if it exists), but I can manage without it. 

Graham

Tveloso

Quote from: graham1 on April 15, 2025, 07:20:32 PMIt would be nice to have an option simply to force update of thumbnails
The Dashboard's Actions Panel does have a Files without thumbnail item, in the Files with Problems section, which should allow you to isolate the files you'll need to run a Force-Update for...
--Tony

TomH

Quote from: Mario on April 15, 2025, 03:48:20 PMOpen Edit menu > Preferences > Application. Search for photools. The state of "Prefer photools.com RAW processing" (use LibRaw) is by default on since IMatch 2025. If this is off on your computer, IMatch is using the installed WIC codec that ships with Windows (unless you have installed the Nikon NEF codec).

There are now over 50 or so different NEF variants in use, all of them undocumented. The ability to read these files or to at least extract the embedded preview image depends on whether or not the WIC codec or LibRaw support the NEF variant. If it is a new camera with a new NEF format, it can take months until Microsoft and the LibRaw project catch up. The joy of using undocumented RAW formats.
Thank you Mario. I have attached a zip file of one of the suspect files after running the WIC diagnostic for your information.

My camera is old by today's standards (Nikon D7500). For some reason, my setting were defaulting to "No". I have since changed it to "Yes".

Quote from: Mario on April 15, 2025, 03:48:20 PMYou don't need to remove the folder from the database.
You can just select the NEF and press Shift+Ctrl+F5 to re-read the file as it was new. This will create the thumbnail, cache image etc.
That fixed it! I had to resort to force updating but that solved it. Regular rescans had not for whatever reason.

As a sidenote, on the dashboard where files with missing thumbnails are noted, there were 3 files listed and none related to what I was trying to resolve.




Mario

I understand that switching o LibRaw fixed it?

Which means that the default Windows WIC codec for NEF just don't handle your NEF files.
Microsoft may or may not support the NEF variant your camera produces. No guarantees.

There is a reason why NEF is not a file format allowed for long-term archival.

Install the official Nikon WIC codec or use LibRaw (photools.com RAW processing). And hope that one handles your NEF variant.
Don't rely on the proprietary and undocumented NEF format for long-term archival. Use a standard format like DNG or TIFF to be safe.

TomH

Quote from: Mario on April 15, 2025, 11:33:11 PMI understand that switching o LibRaw fixed it?

Which means that the default Windows WIC codec for NEF just don't handle your NEF files.
Microsoft may or may not support the NEF variant your camera produces. No guarantees.

There is a reason why NEF is not a file format allowed for long-term archival.

Install the official Nikon WIC codec or use LibRaw (photools.com RAW processing). And hope that one handles your NEF variant.
Don't rely on the proprietary and undocumented NEF format for long-term archival. Use a standard format like DNG or TIFF to be safe.

It's hard to say what fixed what Mario. I changed the default setting and then did a force rescan so its hard to say which solved it. For what it is worth, I have been working with Photools and the same camera's NEF files since last November and it all works 99% of the time but there have been a few incidents like this where select files need some encouragement.

I'll take your archiving recommendations into consideration as well.

Mario

Maybe it's a random stress issue, caused when IMatch processes many files in parallel and your system becomes "overloaded"?
When you now use photools.com RAW processing, let's hope that the issue is gone. It behaves more stable under heavy load, WIC codecs sometimes fail.