Olympus .ORF images shot in portrait orientation...

[spiff]

IMatch only uses the embedded preview if it is large enough (depending on your cache settings). This would explain the discrepancy.
When you disable the "Use embedded preview" option under Edit > Preferences > Cache and then select one of your problem files, press Shift+Ctrl+F5 and use "force update", does IMatch then display the file correctly?

I did so. Sorry to say it did not work.
1.) "Vorschaubild vewenden" is "Nein"
2.) Rescaned the directory.
I does not change anything, Portrait orf are still shown horizontal.

[spiff]

There is an additional bug:
After dissable "use embedded preview" and "force update" imatch is developing a preview out of the orf. Clearly visible because the preview out of orf is a bit dull compared to the out of the cam jpg (which looks already identical as the embedded preview). So far, so good.
But; when enabling again "use embedded preview" and "force update" imatch is still using the developed orf preview and does not use the embedded preview anymore.
The only way to force imatch using again emedded preview is right click on directory and "aus Datenbank entfernen...", then reimport the directory and only now imatch is using the emedded preview again.

[spiff]

complement, there is another way to shift imatch to use the embedded preview again than "remove (directory) from database.." and reimport. Imatch cannot be made to use the emedded ORF preview anymore as long as version (jpg) and master (orf) are in the same directory. If you first move the version (jpg) to another directory and than "force update" the ORF, imatch is able to use the embedded preview. Afterwards move the version back to the original directory where the master is.

[Mario]

Do you perhaps use the JPEG as the version proxy?
IMatch would then use the JPEG as the source when displaying the ORF. And this would explain this effect.

of orf is a bit dull compared to the out of the cam jpg

This is normal. The preview has been created whatever settings and development process your camera applies.
When IMatch is forced to use the RAW, the WIC codec 'develops' the RAW - but this is of course no RAW processor.

This is why camera vendors should provide the WIC codec. They can produce great results by using the same code they use in camera.
But Olympus does not care enough for their customers to provide their own codec when I'm not mistaken.

[spiff]

Do you perhaps use the JPEG as the version proxy?
IMatch would then use the JPEG as the source when displaying the ORF. And this would explain this effect.

If i use the jpg as version proxy the problem is masked because jpg Portraits are shwon in vertical orientation. But if you work just with orf files the problem is still there. I have here 3 types of orf files from 2012 to 2020, and all in Portrait orientation taken orf files are affected. Cameras are:
Olympus E-P1
Olympus E-M5
Olympus E-M1MarkII

Have you tried it with my files i sent you? Please just import the orf to keep it simple, you must see that imatch is not able to show them in correct orientation. 

This is normal. The preview has been created whatever settings and development process your camera applies.
When IMatch is forced to use the RAW, the WIC codec 'develops' the RAW - but this is of course no RAW processor.

What i described "But; when enabling again "use embedded preview" and "force update" imatch is still using the encoded orf preview and does not use the embedded preview anymore." - no way to bring imatch back to use the ebedded preview even when you enable it in cache presets and force a scan, this is not normal. But i mix here two problems in on thread, better focus to the false orf orientation.

[spiff]

I made a Video, you can see it here:

https://www.youtube.com/watch?v=L-XpaVvU610

[Mario]

Thanks for the sample images.
I was able to reproduce this problem on some of my Windows installations.

1. On systems where no WIC codec is installed and IMatch has to fall back to LibRaw, the orientation is correct (IMatch 2020 contains the orientation work-around for LibRaw and ORF).
2. On systems with older Microsoft WIC codecs installed, the orientation is correct.
3. On systems with FastPictureViewer WIC codecs for ORF installed, the orientation is correct.

4. On systems with the latest WIC codecs from Microsoft, the orientation is wrong.

The problem seems to be the 3rd party imaging library which does not seem to like what the WIC codec returns - at least it does not consider the 90° CW EXIF orientation... (?)
IMatch never sees this and never knows this, it just instructs the library to load the file. Which it does. Just with the wrong orientation.

I have no idea what makes these ORF files so special, or why the orientation is correct for all codecs, except this specific set. Or if this is something in the 3rd party image library I use - which usually just works.

I have now solved this by re-routing ORF files through my own code which I have written to interface directly with WIC.
I use this for certain WIC-supported file formats (like HEIF or WEBP) to treat these in special ways.

When I let IMatch process ORF files using my code  things work much better and these ORF files are rotated properly for all WIC codec variants!

Probably I should sit down and write by own image processing library from scratch some day. My own code usually works. Or at least I can find bugs real quick and fix them real quick.
But writing an image processing library requires special know-how and time...and I have not enough of both.

As far as I'm converted, this now fixes this ORF issue with all 3 variants of WIC codecs in use today. The rotation issue for these files and LibRaw was already fixed by ignoring the orientation when the preview is used and just assuming that it is rotated neutral.

This is what I see for your sample images:

[spiff]

Thank you very much for your engagement! I assume you can fix this now with a next release? Sorry for late response, i do a training about the new features in imatch 2020 these days.

As i described in this thread earlier i do also have some problems with files should be in Portrait orientation but are not respective .jpg virtual proxy developed of old Canon .crw RAW files. But things are different to the item with Olympus .orf files. Here the Canon .crw Portrait orientation files are shown in correct orientation in imatch as well as Windows Foto / Windows Explorer. But not the developed .jpg (virtual proxy). Imatch shows the .jpg in horizontal orientation altough the Exif orientation information for .jpg and .crw is identical and correct (rotation 90°). I checked that with imatch Metadata Analyst. I can get rid of the problem if i manually remove the exif rotation information for the .jpg so it gets the information "horizontal" imatch can show it in the viewer in Portrait mode, but this is bungling. I send you 2 .crw files with .jpg, one original and one manualy removed orientation information.

If you can`t see what i try to explain, let me please know. I will then make a video for easy understanding.

Regards Björn

[Mario]

Make sure the EXIF orientation in your JPEG matches the actual bitmap orientation.

[spiff]

I did not understand what you menaning. Have you checked the files? Metadata Analyst telling me "Embedded XMP record (photools.com IMatch 20.3.0.4 (Windows)) and XMP sidecar file (photools.com IMatch 20.3.0.4 (Windows)) found." Does this mean Canon .crw RAW files are able to contain XMP meta data and i now have them double? If yes, how to handle that ideal? Anyway there is no warning about rotation missmatch.

[Mario]

Which files?

If you did something via email - my inbox has currently 160 unread emails. And about 20 of them contain links to databases I should check or images not working in some way or ...

CRW files can contain embedded XMP. Some Canon cameras store a partial XMP record in the CRW, usually with a "rating=none" entry to make things extra hard. Nikon does the same with some models.
This actually required me to add an "ignore rating in embedded XMP" to overcome that - users were complaining that the rating they set in IMatch was vanishing.
RAW files should only use XMP sidecars - not embedded XMP. But Canon and Nikon give a shit and both companies have a very bad history for everything related to software.

You wrote:

n but are not respective .jpg virtual proxy developed of old Canon .crw RAW files.

You have configured a JPEG to be used as a visual proxy for your RAW file?
IMatch then displays the JPEG instead of the RAW. If the orientation is wrong, the EXIF in the JPEG is wrong. The orientation of the RAW file is irrelevant.
I asked you to check the orientation of the JPEG file you use as the visual proxy for the RAW to see if it is correct.

[spiff]

It is the email i sent you 11.03.2020 00:55 Uhr.

As i can see in Metadata app the exif orientation of .jpg seems o.k (90°). But it is shown horizontal in imatch, not in Windows Explorer (when opened, as thumbnail also horizontal in Windows Explorer). This is why i ask you for your help.

...Here the Canon .crw Portrait orientation files are shown in correct orientation in imatch as well as Windows Foto / Windows Explorer. But not the developed .jpg (virtual proxy). Imatch shows the .jpg in horizontal orientation altough the Exif orientation information for .jpg and .crw is identical and correct (rotation 90°). I checked that with imatch Metadata Analyst. I can get rid of the problem if i manually remove the exif rotation information for the .jpg so it gets the information "horizontal" imatch can show it in the viewer in Portrait mode, but this is bungling. I send you 2 .crw files with .jpg, one original and one manualy removed orientation information.

Do you fix the problem with .orf orientation with the next imatch patch?

Regards Björn

[Mario]

I still have 100 unread emails in my inbox. Your email is most likely in that heap.
I have never seen a wrong rotation for JPEG files. It's always RAW files that cause issues. Be it in WIC or LibRaw.

[spiff]

Will you fix the problem with false orientation for ORF (Olympus RAW file) within the next imatch releases?
For me this is essential it will be done not immediately but not too far because i fully switched to olympus a few years ago and can not life forewer with that bug (no face detection for all Portrait mode pictures etc.)

Thanks Björn

[Mario]

For a list of all bugs fixed in the upcoming version, please read the release notes. I write them for a reason.

https://www.photools.com/release-notes/

and if you do that you will find that a WIC-related problem for ORF files has been fixed. Release note #01033
Just search for ORF on that page and you fill with the release note without problems. It contains the explanation of the bug fix etc.

I don't know if this fixes your problem. To many ORF flavours around, no support from Olympus for developers available etc...

Do you have a WIC coded that handles your ORF format? If so, which WIC codec?
The rotation problem was only in the most recent WIC codec from Microsoft (separate download) but not in the FastPictureViewer codecs or the standard WIC codecs shipped with Windows.
But the latest WIC codec in the shop are based on LibRaw, which would explain that the problem only appears in this specific WIC variant.

The ORF rotation back when LibRaw is used was already fixed in the current version.

[Mario]

It is the email i sent you 11.03.2020 00:55 Uhr.
Regards Björn

I have downloaded the files in the ZIP file and added then to an IMatch 2020 database on a fresh Windows 10 installation.
This is what I see. I see no wrong orientation. The CRW files are all processed by the standard WIC codec for CRW in Windows.

[spiff]

have you used the latest Windows codec?

[Mario]

Yes. I have one Windows with the default WIC codecs (whatever that is) and one with the LibRaw-based codecs from the shop.

I recommend to wait for the next IMatch release which has yet another bug fix for yet another WIC issue regarding rotation. This was discussed in this thread for some length.

[spiff]

it was discussed for another Raw format (orf). But we will wait for the next release and see.

[Mario]

WIC is a black box.
I ask Windows to "load" the image. When this is successful I ask WIC for the required orientation. And then rotate the image as indicated before display.
For some files this fails because the originating device has written the embedded preview in a different orientation than indicated in the EXIF data.

IMatch now ignores the EXIF data when the preview is used. This will "fix" the problem for the reported case.
Until a user reports a bug because his camera does not use a neutral rotation for the preview but requires the preview to be rotated like the RAW data EXIF indicates.
At this time I will have to add options to override this fucking orientation on a per file format or even per camera basis.
Just because camera vendors can't get their shit in order.

[abgestumpft]

I still have problems with Portrait oriented .ORF files being displayed in horizontal Orientation using the WIC Codec in latest release 2020.3.6.
When I use the internal iMatch decoder the orientation is correct.

Can others confirm this too?

[spiff]

-

[abgestumpft]

There was a new release yesterday (2020.3.6), but I still have the same issues with that release. That's why I'm asking if the problems are still there for others too...
The Topic of that thread is "Olympus .ORF images shot in portrait orientation...", so I expect my question is valid in this one...

[Mario]

Did you force an update of all affected files with Shift+Ctrl+F5?

Just installing an IMatch update does not re-create all thumbnails and cache images.
Also make sure you use the WIC codecs from the Microsoft Store, which seem to deal better with the ORF rotation issue.

And of course you should contact Olympus and demand a proper WIC codec which supports the files your camera produces.
You pay for the hardware and Oly f**s you by not supporting WIC and not providing a proper codec. That is their responsibility.

[abgestumpft]

Did you force an update of all affected files with Shift+Ctrl+F5?
Just installing an IMatch update does not re-create all thumbnails and cache images.

Yes, and I also importet some completly new RAW files that haven't been in the catalog before.
Everytime the same: WIC Codec = wront orientation, internal iMatch codec = correct orientation

Also make sure you use the WIC codecs from the Microsoft Store, which seem to deal better with the ORF rotation issue.

I have the "RAW Image Extetion" from Microsoft store installed. In Windows Explorer the Thumbnails of the .ORF files have the correct orientation. I've sent you a mail today at 21:57 with an example RAW file and some screenshots + WIC Diagnostic logs

[spiff]

There was a new release yesterday (2020.3.6), but I still have the same issues with that release. That's why I'm asking if the problems are still there for others too...

+1
Same here. I have installed the latest WIC coded, your update is based on information about my installed WIC codec + Olympus RAW i sent you. Unfortunately it is still false orientaded in imatch.

[abgestumpft]

One thing I just recognized (maybe that helps):
1. When I open a .CR2 file with the MS Foto app this is right-away in portrait rotation

2. When I open a .ORF file with the MS Foto app it is first loaded=shown in horizontal position and changed to portrait orientation after 1 sec. or so. Also when I change the Window Size of "Fotos" the picture is back in horizontal orientation until I stop scaling, then it's back in portrait.

[Mario]

I have no other options for processing ORF files.
Since these are the only files producing this irritating behavior and I recommend you contact Olympus for a working WIC codec. They will laugh in your face, most likely.

I can either rotate the image as WIC tells me (which does not work for these ORF files). Or I can ignore the orientation. Which also seems not to work.
There is no 3rd option. Rotate or ignore. IMatch cannot guess or apply a random orientation. Its either rotate as defined by the EXIF orientation or not.

So, I'm out of options. And I have plenty of other things to do. Please use a virtual rotation in IMatch to rotate these files in whatever orientation you would like them.

[sinus]

A pity, that Olympus does not a better job.
They have very good cameras, but they could never really keep up with companies like Nikon or Canon, at least this is my impression (in the professional area).

But there are other companies that do not do a very good job when it comes to providing digital aids like WICs or other software.

[spiff]

Is there a work arround that makes possible using face recognition, does this work with virtual rotation?

Mario, this is really bad news for me because i mainly use Olympus. Because oft that, one request: Do you please eanble with a very next release that we can selective enabling for which RAW files we want to use imatch RAW processing? So we get rid of it via that way, but still able using embded RAW thumbnails where imatch can handle it well.

[abgestumpft]

On thing that I think has changed to the behaviour Conrad described here:
https://www.photools.com/community/index.php?topic=9446.msg66759#msg66759

At his test the Settings:
"Prefer photools.com RAW processing" = "No", "Use embedded preview" = "No" resulted in wrong orientation of portrait oriented .ORF files

Now when using this with a short test on my .ORF files in portrait mode the orientation is shown correct!
So situation with WIC Codec is this:
"Prefer photools.com RAW processing" = "No", "Use embedded preview" = "Yes"  -> Wrong orientation
"Prefer photools.com RAW processing" = "No", "Use embedded preview" = "No"  -> Correct orientation

This could also explain the behaviour of the Microsoft Fotos App:
It first shows the file in horizontal (quick loading of embedded JPG -> what iMatch is using -> also wrong orientation here) and then changes the display to portrait orientation after 1-2sec (processed RAW file?).

Can others confirm than the "No"- "No" option is now working on .ORF files?

What would be the impact of using "Use embedded preview" = "No"? My understanding is that instead of using the embedded JPG iMatch will process each RAW file to create a preview image. This would mean:
1. Much longer processing time when importing new Fotos
In case the embedded JPG dimensions < RAW (which is the case for .ORF):
2. More disk space usage of Cache folder
3. In the Viewer the 100% view will provide a more detailed image

[spiff]

Thank you for this information. At the moment imatch is in progress for a new DB i will give a feedback latest this afternoon.

[Mario]

"Prefer photools.com RAW processing" means that IMatch is not using WIC codecs but LibRaw for processing files.
This is where I made the first of several "Olympus only" changes. To ignore the EXIF orientation when processing the preview.

As I said in the release notes, to rotation bug with WIC (no photools.com RAW processing) only happened with the latest WIC codecs from Microsoft (from the Shop), which they have based on LibRaw themselves. So the same problem here. Also solved.

The standard Windows 10 WIC codecs or the old FastPictureViewer codecs don't show this behavior.
For FPV only unless you enable the option in the FastPictureViewer codec to always rotate the embedded preview before delivering it to the calling application - which causes other problems because it still returns the original EXIF orientation.

IMatch ignores EXIF orientation when the embedded preview is used. It applys the EXIF orientation when processing the RAW - which it almost never does as long as the embedded preview is large enough.
I have no other options. Either I believe the EXIF orientation returned by WIC / LibRaw or I ignore it. Both seem to fail for some Oly files only.
So, this goes to the Olympus support. They just need to provide you with a WIC codec which handles all the crappy details of their files - other camera vendors like Nikon can do that, so Oly should also not let their paying customers stand in the rain.

[abgestumpft]

Just to make it clear what is currently working OK in iMatch 2020.3.6 with Olympus .ORF files in Portrait orientation:

These settings give correct rotation of the files:
1. Use the internal iMatch LibRAW
"Prefer photools.com RAW processing" = "Yes"
2. Use the WIC Codec, but ignore the embedded JPGs -> Process the RAWs
"Prefer photools.com RAW processing" = "No", "Use embedded preview" = "No"
3. Use iMatch Detault settings + Manual correction of orientation
"Prefer photools.com RAW processing" = "No", "Use embedded preview" = "Yes" AND apply virtual rotation as stated in the Orientation tag manually for the files (on a first check this also works on faces, but maybe there is some showstopper using that option)

It looks like with Olympus .ORF in Portrait and WIC the tag for the embedded JPG is wrong. For the RAW it is correct.
And also the Microsoft Foto App (using the same WIC codec) seems to have the same issue, see what I wrote earlier:

This could also explain the behaviour of the Microsoft Fotos App:
It first shows the file in horizontal (quick loading of embedded JPG -> what iMatch is using -> also wrong orientation here) and then changes the display to portrait orientation after 1-2sec (processed RAW file?).

Windows Explorer might show a preview of the RAW instead of the embedded JPG and therefore orientation is correct here.

My conclusino for now is:  On Olympus .ORF files the RAW part of the WIC codec is OK, but something is wrong with orientation information of embedded JPG.

[spiff]

i can confirm that it is the same here. I decided to work with
3. Use iMatch Detault settings + Manual correction of orientation
because it performs better and saves space (no RAW processing). If it works with face detection proper this is fine for me. If imatch in future can handle embedded previews with faulty orientation information it is easy to filter for virtual rotated files and remove the tag.

[Mario]

I have already handled this.

For all RAW files this works the same.

1. IMatch tells WIC to load the embedded preview.
If this large enough (as per user settings) it uses it.
IMatch asks WIC how to rotate the image for display.

2. If the preview is to small or usage is disabled, load the full RAW via WIC.
The processing done on the image depends on the WIC codec. This is why WIC codecs should be produces by camera vendors. Because they know their RAW files best (supposedly).
IMatch asks WIC how to rotate the image for display.

3. If all that fails, fall back to LibRaw.
Load preview. If OK, ask LibRaw how to rotate it.
If preview no good, load RAW. Ask LibRaw how to rotate it.

In case of these ORF files, the rotation for the preview delivered by WIC and LibRaw is wrong. Apparently.
Because when IMatch rotates the image as reported, users tell me the "orientation is wrong".
So I no longer rotate when the preview is loaded. This is bug fix. I can either rotate or not. Both fails with these peculiar images. They are just messed up.
Still uses tell me "The orientation is wrong".

Both methods fail for these ORF files. And I have spent too much time on this already.

Ask Olympus support to supply you with a working codec. Then all is well. This is how this is supposed to work.
Windows has the framework, the camera  vendor delivers the WIC codec ("driver") and all applications can just display the image and are done.

If Olympus tells you they don't have one, buy your next camera from a vendor who actually cares for its customers.
And provides a WIC. Or at least provides Microsoft with information about how to optimally process the files.

[abgestumpft]

Hi Mario,

when you check my other thread here:
https://www.photools.com/community/index.php?topic=9948.0

This is how .ORF behaves with different options:


Ant his:

The latest Olympus WIC codecs are obviously buggy. Also in the Microsoft Fotos App (based on WIC), when loading an .ORF RAW in portrait orientation, the initial preview of the file is shown horizontal (JPG preview) until the RAW file is loaded (changing to portrait orientation). -> JPG Previews using WIC on portrait oriented .ORF files is wrong (does NOT only applys to iMatch). RAW is correct.

This means the Microsoft Foto App has the same problem like iMatch -> when it load the preview JPG of a portrait-ORF file it is shown horizotally, until the Foto App loads the RAW and then shows the converted RAW in Portrait rotation.
When you check the both attatched Fotos:
The first one shows how Fotos is displaying the embedded JPG
The second one is the embedded RAW

This is exaclty the same way that iMatch behaves. So the root cause is a bug in the WIC codec for orientation of embedded JPG provided by Olympus (I will check were I can open a Bug report at Olympus, let's see if something is happening)

[spiff]

Gentlemen, because i had to install Win 10 new (backup from 2016) and my system untill now has no WIC codec installed Portrait orientation of ORF files for E-P1, E-M5, EM1MkII (2012-2020) works fine when using embedded thumbnails. Its clear that Windows at the moment is not able to show ORF files. Trouble starts when installing WIC codec i think.

Small question can you please explain me what this shaded lines are telling me, can´t find it in help?

[Mario]

1. Windows 10 always comes with WIC codecs.
They just may not be able to understand the ORF format you are using. Olympus has produced many different ORF variants over the years and unless they give you a WIC codec that handles the formats you have, you depend on whatever ORF WIC codec Microsoft provides with Windows.

2. "Shaded lines".... do you mean the label indicator?
This just means that your files use an XMP label name that is unknown to IMatch. There is no standard.
See Metadata