Failed to load image with PTD2D.

[Pawel]

There was some HEIF codec update today from Microsoft and afterwards my IMA stopped displaying most of iPhone HEIF files. I see a lot of "Failed to load image with PTD2D. 0 -   'V:\develop\IMatch5\src\ptpicore\PlugIns\ptpimm\PTPIMultiMedia.cpp(1289)'" or "Failed to load image with PTD2D. 0 -   'V:\develop\IMatch5\src\ptpicore\PlugIns\ptpimm\PTPIMultiMedia.cpp(1305)'" errors in service log file.

What is interesting, all the HEIC files load correctly when opened from within file system using Windows Photos app for example - so the WIC codec generally works I assume. Also, some HEIC files do load correctly in IMatchViewer.

For those that don't load IMA displays some random thumbnails from other HEIC files (completely unrelated) and when clicked sometimes show big version of the thumbnail is displayed - like slightly distorted image - and sometimes I can see a big file icon only.

I rebooted the IMA server, cleared IMatch Anywhere cache, reinstalled HEVC extensions - the problem still persist.

Any idea, what should I do next?

[Mario]

I understand that you or Windows has installed an update for the HEIC WIC codec and it no no longer works in IMatch Anywhere WebServices.

Do the files work in IMatch? What happens if you select a file that failed and do a forced update by pressing Shift+Ctrl+F5 in a File Window?

Please provide a minimum of information:

1. The IMWS/IMatch log file (see log file)
2. The WIC diagnosis result of one of your HEIF files which fails to load (WIC Diagnostics)
3. A HEIF file which fails to load. Provide a link or send it to support email address
There are so many variants in use now, and Apple changes the format frequently.

Having one of your files here in the lab will help me to analyze this further. Not sure if this is a WIC problem or somehow caused how IMatch uses WIC (although there is only one way to open a file via WIC)...

Did you reboot your system once?
Are these files on the same computer on which IMWS is running or on another computer or NAS?

WIC is pretty much a black box and IMatch basically asks Windows nicely to "load this image". And when Windows returns an error code, IMatch logs it and continues with the next file.
IMatch must rely on the Windows HEIC codec because the HEIC/HEIF file format is infected by dozens of software parents from Apple, Nokia and others. I won't touch a HEIC/HEIF file myself, ever. I don't have the money for the lawyers. Patent-infested patents are a real pest. And HEIC/HEIF is a very bad example for this.

Why would IMatch display random thumbnails? The thumbnail is stored in the database under the same id as the file. IMatch does not select thumbnails randomly.
Maybe something else is broken and messes IMatch's in-memory caches up?
Does this problem go away when you restart IMatch? This clears all memory caches.

I have just checked the HEIF/HEIC images files (about 50, from different devices and users) in my sample library.
Some files fail to load with WIC error messages. But that was always the case.
I also don't get previews for these files in Windows Explorer.
Some of the files load in Affinity Photo / Designer, though.
HEIC/HEIF support on platforms other than Apple is still pretty sketchy.
If I try to load one of the files IMatch displays OK in the current Photoshop, Ps tells me to download the HEIC codec I already have...

[Pawel]

I would gladly get rid of HEIC files, too. Some day I will do something about it but for now I index them with IMatch.

As to the problem: the files do work in IMatch - I can see them as thumbnails and in the viewer both on my main computer and on the computer running IMatch Anywhere. What is interesting though, IMatch WIC codec diagnostic log shows "GetThumbnail failed (88982F44..." errors for the files - on both computers. I tried to force rescan and update cache of one of the HEIC files and it is still visible both as a thumbnail and in the viewer.

I am sending IMWS and IMatch logs together with the problematic HEIC files to the support address.

[Pawel]

Oh, just one thing to add - I restarted IMatch Anywhere service and the computer IMatch Anywhere works on many times. I also manually deleted the cache directory configured in IMatch Anywhere and restarted everything.

[Mario]

Match WIC codec diagnostic log shows "GetThumbnail failed (88982F44..." errors for the files -

A RAW can contain up to 3 separate images: thumbnail, preview and full-sized RAW. WIC diagnosis tests all 3 formats.
IMatch prefers the preview, unless it is smaller than the minimum size configured.

WIC diagnosis also tries to load the RAW via LibRaw - which always fails with HEIC because LibRaw cannot support this format due to Apple/Nokia patents and license fees.

These license fees to pay to Apple/Nokia are also the reason why the HEIF WIC codec is not installed by default in Windows and you have to download it separately if you need it.
Microsoft thus pays license fees to Apple/Nokia only for the actually installed WIC codecs.
And this is also the reason why the HEIC/HEIF video codec costs money.

Why anybody would store her/his image image files in such a patent-infested file format totally under the control of Apple/Nokia and some others escapes me.
This is even worse than the undocumented and proprietary RAW formats we have to deal with.

Apple clearly came up with HEIC/HEIF to get away with smaller memory sizes on their phones (at premium prices, no less) and to gain more control over the digital assets of iPhone users.
Make it easier to keep all your files in the Apple walled garden and making it much harder to get your files out and use them on non-Apple devices...
Apple like you. They don't want you to leave. Ever.

[Mario]

I have received your HEIC and MOV files.
The MOV file is processed without problems.

For the HEIC files, Windows WIC reports:

Thumbnail: Codec 'Microsoft HEIF Decoder'
    (GetThumbnail failed (88982F44 The bitmap codec does not support a thumbnail.).) 0x0 pixel in 0 ms.

Preview: Codec 'Microsoft HEIF Decoder'
    (GetPreview failed (88982F81 The operation is unsupported.).) 0x0 pixel in 0 ms.

Full resolution: Codec 'Microsoft HEIF Decoder'
    (File loaded with 3024 x 4032 but WIC CopyPixels failed (C00D5212 No suitable transform was found to encode or decode the content.)) 0x0 pixel in 0 ms.


which means that Windows WIC can load the file, but not convert the RAW data into a usable image (this is the CopyPixels part).

It seems that this HEIC format variant is not yet supported by Windows.
Maybe Apple introduced another breaking change or Microsoft had not yet have time to update the WIC codec. Or a recent update broke something.

Not even the "shell thumbnail handler" in Windows supports this format. I don't get thumbnails in Windows Explorer on two separate computers.
Affinity Designer loads the file, but Photoshop fails to load it.
The Microsoft Window WIC test tools also fail to load this file.

I recommend you contact Apple or Microsoft for a solution for this problem.
I can neither change how Apple writes HEIC files nor if and when Microsoft supports this format. Nothing I can do, sorry.
Not even LibRaw supports HEIC files, for patent and license fee reasons.

[Pawel]

What is interesting is that despite WIC diagnostic's negative output the HEIC files are visible in IMatch and in Windows Explorer preview window. It is not a new variant as the files are several months old and were processed by IMatch already with no problems - I have them indexed, face tagged etc.

I understand however that it is probably something that's changed on Microsoft side that broke the support for the files in IMA (I am sure the files were visible in IMA several days ago) so I will try to fix something on Microsoft side or get rid of the format altogether as per your recommendation.

Just one question remains: why is IMA displaying random MOV frames as thumbnails for most of the HEIC files? I'd rather see it displaying the empty slide icon.

[Mario]

1. If the file was previously created, IMatch will use the existing cache file.
I'm quite sure when you force a rescan of one of the working files, it will also fail to load now. Seems like MS has broken something in the HEIF codec.

2. IMatch may use a fall-back on failed files which evolves FFMpeg (a HEIF can be a video, an image, an audio file or a combination of all these).
But this could not produce random previews (from other files?). I did not see this effect with the files you have provided.

Again, HEIF is a very volatile format designed for the specific needs of Apple and Nokia and other patent and license holders.
If Apple and Microsoft don't get their act together (Affinity seems to cope, Adobe Ps does not handle it but Lr does), I cannot do anything about this.

I rely on the HEIF WIC codec because this is the only way.
Reading the HEIF files with my own code would require me to make a contract with the MPEG, Apple, Nokia and all license and patent holders (see https://en.wikipedia.org/wiki/High_Efficiency_Video_Coding#Patent_holders for a list). Samsung alone has 4249 (!) patents which are somehow affected by HEIF. Software patents are insane.
I cannot even afford the lawyers which would tell me which lawyers can make such a deal...

Just don't use such proprietary and not widely supported image formats if possible.
There are so many flexible image formats around which are supported on Windows, Mac, iOS, Android and Linux. Why HEIF?

[Mario]

I have solved it. Apple. Microsoft. HEIC format customer lock-in shit again.

I've had another user which ran into the problem that HEIC files no longer displayed in IMatch.
He sent me a sample image, and again WIC loaded the file just fine, but the operation which converts it into something IMatch can use (CopyPixels) returned the dreaded C00D5212 No suitable transform was found to encode or decode the content. error message again.

After trying different things for an hour and Googling for another hour, I gave up.
Then I had the idea to try opening the file in Windows Photo. It failed to load. But Photo told me that a HECV codec was missing and that I can buy it (from Microsoft) in the Windows shop. For 1€.

HECV Video Extension.

So I bought it and Windows installed it.

And, voilà - after a forced update with Shift+Ctrl+F5, the image loaded just fine in IMatch 

What I think happened is: HEIC/HEIF is a container format. It can contain any types of formats: images, vides, audio etc. Like MP4 but tailored to the needs of Apple.
I think recent iPhones and probably other Apple devices encode images in a format that requires royalty payments and is thus not available in the free WIC HEIF codec included in Windows.
Buying the HEIF codec in the shop adds support for these formats (and money to the pockets of Microsoft and Apple) and then the WIC codec can also encode these formats.

So, if you run into this problem: it is not IMatch's fault. You need to buy the extra HEIF codec for Windows. Which is thankfully cheap.
Interestingly, this worked for some time, but a Windows update seems to have disabled this...not sure.

Thanks to Apple for forcing their users to use a parent-infested and royalty-ridden file format.
To me this seems to be just another measure to lock people into the walled Apple garden.

And WIC should return an error message like "You can only decode this format when you have the fee-based HEIC codec installed" instead of the other, meaningless error message. This would have saved me hours of time.

[dpop100]

I am experiencing the same issues with Apple files. I have never had issues with any Apple file format in iMatch but I recently upgraded to an iPhone 13. iMatch continues to crash at every launch until I remove the Apple files. Using the debug dump information led me to this thread. (should I start a  new one?) I did install the new HEVC Video extension mentioned but it continues to crash attempting to scan these files. I understand your  position that this is not a iMatch issue, but like it or not, my workflow does include images I take with my phone. I would be okay to NOT include them in the scans but I do not want to have to move them outside of my folder structure to avoid scanning. Under Preferences, I can choose to skip hideen files or folders and skip an entire folder. Is there an option I am missing to not scan *heic or *.dng?

David Poplawski

[sinus]

Thanks to Apple for forcing their users to use a parent-infested and royalty-ridden file format.
To me this seems to be just another measure to lock people into the walled Apple garden.

Nice written   and from my point of view very true.
Apple has some wondeful products, really, but they want to lure people into their (expensive) paradise and woe betide them if they want to get out again.
That is why I never bought an apple-product, although most people in my trade (photographer, graphic designer, artist...) work with Apple.

[Mario]

I am experiencing the same issues with Apple files. I have never had issues with any Apple file format in iMatch but I recently upgraded to an iPhone 13. iMatch continues to crash at every launch until I remove the Apple files. Using the debug dump information led me to this thread. (should I start a  new one?) I did install the new HEVC Video extension mentioned but it continues to crash attempting to scan these files. I understand your  position that this is not a iMatch issue, but like it or not, my workflow does include images I take with my phone. I would be okay to NOT include them in the scans but I do not want to have to move them outside of my folder structure to avoid scanning. Under Preferences, I can choose to skip hideen files or folders and skip an entire folder. Is there an option I am missing to not scan *heic or *.dng?

David Poplawski

I have quite a collection of HEIC/HEIF files produced by different cameras and devices, including Apple hardware.
I'm using the latest HEIC WIC codec provided by Microsoft. They are into the HEIC/HEIF consortium so I'd guess they keep their codecs up-to-date. If Apple has made a breaking change in their HEIC files with new models which cause the WIC codec to crash (I have never seen that), you can switch IMatch to using LibRaw and try your luck.

Edit > Preferences > Application: Use photools.com RAW processing.
Then WIC codecs are no longer used. I'm not sure from the top of my head if LibRaw supports HEIC/HEIF, though...

If LibRaw also fails to load your Apple files or even crashes, we are out of options for the moment. That's the risk in using patent-protected proprietary file formats.

You also mention DNG files. I have about 1,000 DNG files in my test suite, created with dozens of cameras and smart phone modes and converted with various versions of Adobe's ACR and other software. None causes any trouble.

Please upload some HEIC/HEIF/DNG files that give you trouble to your cloud space (OneDrive, DropBox, Google Drive, Apple cloud) and send me a link to support email address (include a link back to this topic). I can have a look and see if the files also crash the WIC codec here.

If the Microsoft WIC codec crashes, I can report it via the official developer channels to Microsoft and they are hopefully able and willing to fix it within the next couple of months. Same for Libraw, but the developer is much more responsive.

[dpop100]

Thank Mario. I was in a huge rush to get these images cataloged for a project and just wanted it to work, hassle free. I did not have adequate time to investigate the error logs or the exact images that were causing the crash. I will pull those files back in and try again and send those sample files to you for examination.