color shift after batch processor

[ChrisMatch]

Maybe I still don't understand color profiles - maybe this effect is explainable - but I don't know what and why this is happening...

From time to time I notice that some images look like the colors are off in comparison to the orginial developed JPG.
Usually I start with a NEF file
-> develop it with PhotoNinja
-> and export a JPG (with embedded sRGB profile)
-> this JPG goes to the batch processor where the image gets a border and a text overlay
    (Color Profile Settings: Convert to profile = sRGB, Embed profile = ON)

I have attached a screenshot where you can see both images (before and after batch processor).

Is there a setting to avoid this effect - or is this actually a bug?

[attachment deleted by admin]

[ChrisMatch]

As it seems no one was able to come up with an explanation
so maybe we can move this to the bug section.

Thanks
Chris

[jarraun]

Chris,

Just in case, try to disable the "Convert to profile" setting in the "Color profile settings" of the "Batch processor". I may be wrong but your images have already an embeded profile, don´t need to apply it anymore, just embed the actual profile in the new image. Hope this helps.

Best regards,

Javier

[Winfried]

This seemes to be a typical sRGB <--> AdobeRGB mismatch.
Specially because of the green.
Check which profile the input image has?
I bet it is AdobeRGB.
What are the setting "Convert to profile"?
Did you embed a profile?

Winfried

[ChrisMatch]

Thanks for trying to help.

@Winfried:
The profile of the original image is sRGB.
My settings of the batch processor can be seen in my first post (screenshot).

@Javier
I tried it with 'convert to profile = off' but with the same result (see attachment).

best regards
chris

[attachment deleted by admin]

[Winfried]

Knowing that this "green" is quite tricky, I used this in my book about colormanagement 10 years ago.
I created 4 files, with the rbg-value 102,204,102 and assigned or converted to different icc-profiles

If your browser is colormanaged you should see differences.
If you use AdobeRGB as your color space you need the rgb-values 140,203,108 to get the "right" color.

This little example shows the effect of color profiles to the image, but it is no explination for the observe effect.

Well, I played a bit with this files and found the bug 

If I switch on Lab-sharpening the following colormanagement settings are not used.
If I switched off Lab-scharpening all was fine.

Winfried







[attachment deleted by admin]

[ChrisMatch]

Thanks again for having a look at it.

In my case sharpening was turned off!


[attachment deleted by admin]

[jarraun]

Chris,

Please, can you find in your metadata panel this variable ICC_Profile::Main\desc\ProfileDescription\0 ?, once you have it, check if the files edited with Photo Ninja and exported as JPG with embeded srgb profile, actually show sRGB IEC61966-2.1 in the ICC_Profile::Main\desc\ProfileDescription\0 variable. You can also try to open these files with another color managed software to check if they have really the embeded profile in them.

[ChrisMatch]

I couldn't find this variable in the metadata panel but this is the exif output:

[ICC-header]    Profile CMM Type                : lcms
[ICC-header]    Profile Version                 : 4.2.0
[ICC-header]    Profile Class                   : Display Device Profile
[ICC-header]    Color Space Data                : RGB
[ICC-header]    Profile Connection Space        : XYZ
[ICC-header]    Profile Date Time               : 2014:09:06 18:14:21
[ICC-header]    Profile File Signature          : acsp
[ICC-header]    Primary Platform                : Microsoft Corporation
[ICC-header]    CMM Flags                       : Not Embedded, Independent
[ICC-header]    Device Manufacturer             :
[ICC-header]    Device Model                    :
[ICC-header]    Device Attributes               : Reflective, Glossy, Positive, Color
[ICC-header]    Rendering Intent                : Perceptual
[ICC-header]    Connection Space Illuminant     : 0.9642 1 0.82491
[ICC-header]    Profile Creator                 : lcms
[ICC-header]    Profile ID                      : 0
[ICC_Profile]   Profile Description             : sRGB
[ICC_Profile]   Profile Copyright               : No copyright, use freely
[ICC_Profile]   Media White Point               : 0.9642 1 0.82491
[ICC_Profile]   Chromatic Adaptation            : 1.04787 0.02292 -0.0502 0.02957 0.99048 -0.01707 -0.00925 0.01506 0.75179
[ICC_Profile]   Red Matrix Column               : 0.43605 0.22249 0.01392
[ICC_Profile]   Blue Matrix Column              : 0.14304 0.06061 0.7139
[ICC_Profile]   Green Matrix Column             : 0.3851 0.7169 0.09708
[ICC_Profile]   Red Tone Reproduction Curve     : (Binary data 524 bytes, use -b option to extract)
[ICC_Profile]   Green Tone Reproduction Curve   : (Binary data 524 bytes, use -b option to extract)
[ICC_Profile]   Blue Tone Reproduction Curve    : (Binary data 524 bytes, use -b option to extract)
[ICC-chrm]      Chromaticity Channels           : 3
[ICC-chrm]      Chromaticity Colorant           : Unknown (0)
[ICC-chrm]      Chromaticity Channel 1          : 0.64 0.33
[ICC-chrm]      Chromaticity Channel 2          : 0.3 0.60001
[ICC-chrm]      Chromaticity Channel 3          : 0.14999 0.06
[ICC_Profile]   Device Mfg Desc                 : PictureCode
[ICC_Profile]   Device Model Desc               : sRGB

[jarraun]

Mmm, strange. See what ECP shows for one of my files with sRGB IEC61966-2.1 profile embeded:

[XMP-xmpRights] Usage Terms                     : Todos los derechos reservados. Prohibida la reproducción sin permiso previo
[ICC-header]    Profile CMM Type                : Lino
[ICC-header]    Profile Version                 : 2.1.0
[ICC-header]    Profile Class                   : Display Device Profile
[ICC-header]    Color Space Data                : RGB
[ICC-header]    Profile Connection Space        : XYZ
[ICC-header]    Profile Date Time               : 1998:02:09 06:49:00
[ICC-header]    Profile File Signature          : acsp
[ICC-header]    Primary Platform                : Microsoft Corporation
[ICC-header]    CMM Flags                       : Not Embedded, Independent
[ICC-header]    Device Manufacturer             : IEC
[ICC-header]    Device Model                    : sRGB
[ICC-header]    Device Attributes               : Reflective, Glossy, Positive, Color
[ICC-header]    Rendering Intent                : Perceptual
[ICC-header]    Connection Space Illuminant     : 0.9642 1 0.82491
[ICC-header]    Profile Creator                 : HP
[ICC-header]    Profile ID                      : 0
[ICC_Profile]   Profile Copyright               : Copyright (c) 1998 Hewlett-Packard Company
[ICC_Profile]   Profile Description             : sRGB IEC61966-2.1
[ICC_Profile]   Media White Point               : 0.95045 1 1.08905
[ICC_Profile]   Media Black Point               : 0 0 0
[ICC_Profile]   Red Matrix Column               : 0.43607 0.22249 0.01392
[ICC_Profile]   Green Matrix Column             : 0.38515 0.71687 0.09708
[ICC_Profile]   Blue Matrix Column              : 0.14307 0.06061 0.7141
[ICC_Profile]   Device Mfg Desc                 : IEC http://www.iec.ch
[ICC_Profile]   Device Model Desc               : IEC 61966-2.1 Default RGB colour space - sRGB
[ICC_Profile]   Viewing Cond Desc               : Reference Viewing Condition in IEC61966-2.1
[ICC-view]      Viewing Cond Illuminant         : 19.6445 20.3718 16.8089
[ICC-view]      Viewing Cond Surround           : 3.92889 4.07439 3.36179
[ICC-view]      Viewing Cond Illuminant Type    : D50
[ICC_Profile]   Luminance                       : 76.03647 80 87.12462
[ICC-meas]      Measurement Observer            : CIE 1931
[ICC-meas]      Measurement Backing             : 0 0 0
[ICC-meas]      Measurement Geometry            : Unknown
[ICC-meas]      Measurement Flare               : 0.999%
[ICC-meas]      Measurement Illuminant          : D65
[ICC_Profile]   Technology                      : Cathode Ray Tube Display
[ICC_Profile]   Red Tone Reproduction Curve     : (Binary data 2060 bytes, use -b option to extract)
[ICC_Profile]   Green Tone Reproduction Curve   : (Binary data 2060 bytes, use -b option to extract)
[ICC_Profile]   Blue Tone Reproduction Curve    : (Binary data 2060 bytes, use -b option to extract)
[File]          Image Width                     : 4655

Do you have the ICC Profile sRGB IEC61966-2.1 installed in your system ?. Go to Windows Control Panel and then to Color Management and under ICC Profiles check it is included see attachment. If not download it from the WEB. Then make sure the ICC Profile sRGB IEC61966-2.1 is the one that Photo Ninja picks when embeding profile.

[attachment deleted by admin]

[ChrisMatch]

Yes, it is installed on my system.

The reason only sRGB is shown is, that I can choose between sRGB and sRGB IEC61966-2.1 when creating the output image with PhotoNinja.
I have choosen sRGB because it sounded more general and I have no clue what the difference is.

[attachment deleted by admin]

[jarraun]

But can you find any plain, just "sRGB" profile in the Color Management of your Windows system ?
I would try embed in Photo Ninja the sRGB IEC61966-2.1  ICC Profile. May be the IM5 Batch Processor doesn´t recognize a plain "srgb" profile.

[ChrisMatch]

ok, did the test with sRGB IEC61966 with the same result (see attachment).

Exif Tool now shows the new profile:
[ICC-header]    Profile CMM Type                : lcms
[ICC-header]    Profile Version                 : 2.1.0
[ICC-header]    Profile Class                   : Display Device Profile
[ICC-header]    Color Space Data                : RGB
[ICC-header]    Profile Connection Space        : XYZ
[ICC-header]    Profile Date Time               : 1998:02:09 06:49:00
[ICC-header]    Profile File Signature          : acsp
[ICC-header]    Primary Platform                : Microsoft Corporation
[ICC-header]    CMM Flags                       : Not Embedded, Independent
[ICC-header]    Device Manufacturer             : IEC
[ICC-header]    Device Model                    : sRGB
[ICC-header]    Device Attributes               : Reflective, Glossy, Positive, Color
[ICC-header]    Rendering Intent                : Perceptual
[ICC-header]    Connection Space Illuminant     : 0.9642 1 0.82491
[ICC-header]    Profile Creator                 : lcms
[ICC-header]    Profile ID                      : 0
[ICC_Profile]   Profile Copyright               : Copyright (c) 1998 Hewlett-Packard Company
[ICC_Profile]   Profile Description             : sRGB IEC61966-2.1
[ICC_Profile]   Media White Point               : 0.95045 1 1.08905
[ICC_Profile]   Media Black Point               : 0 0 0
[ICC_Profile]   Red Matrix Column               : 0.43607 0.22249 0.01392
[ICC_Profile]   Green Matrix Column             : 0.38515 0.71687 0.09708
[ICC_Profile]   Blue Matrix Column              : 0.14307 0.06061 0.7141
[ICC_Profile]   Device Mfg Desc                 : IEC http://www.iec.ch
[ICC_Profile]   Device Model Desc               : IEC 61966-2.1 Default RGB colour space - sRGB
[ICC_Profile]   Viewing Cond Desc               : Reference Viewing Condition in IEC61966-2.1
[ICC-view]      Viewing Cond Illuminant         : 19.6445 20.3718 16.8089
[ICC-view]      Viewing Cond Surround           : 3.92889 4.07439 3.36179
[ICC-view]      Viewing Cond Illuminant Type    : D50
[ICC_Profile]   Luminance                       : 76.03647 80 87.12462
[ICC-meas]      Measurement Observer            : CIE 1931
[ICC-meas]      Measurement Backing             : 0 0 0
[ICC-meas]      Measurement Geometry            : Unknown
[ICC-meas]      Measurement Flare               : 0.999%
[ICC-meas]      Measurement Illuminant          : D65
[ICC_Profile]   Technology                      : Cathode Ray Tube Display
[ICC_Profile]   Red Tone Reproduction Curve     : (Binary data 2060 bytes, use -b option to extract)
[ICC_Profile]   Green Tone Reproduction Curve   : (Binary data 2060 bytes, use -b option to extract)
[ICC_Profile]   Blue Tone Reproduction Curve    : (Binary data 2060 bytes, use -b option to extract)


[attachment deleted by admin]

[jarraun]

Just one last thing, my knowledge of color management is limited  . Normally in image editors we have two options: "Assign ICC Profile" or "Convert to ICC Profile", I don´t know how is it managed in Photo Ninja, but if you assign a profile to an image that already has one, many color shifts can be noticed. It is normally adviced convert to a ICC profile unless you are working in RAW images and want assign a customized profile to an image or your image has no ICC profile embeded.
Sorry Chris I´ve run out of ideas.

[ChrisMatch]

Sorry Chris I´ve run out of ideas.

Never mind - I am glad we tried a few things - at least we were able to rule out some possible problems.

[Ferdinand]

These things can be frustratingly difficult to track down.  Perhaps when Mario finds and fixes the lab sharpening bug he might also find and fix yours.

In the meantime, if it's causing you problems then you could try my Photoshop CS Batch Processor Script.  I wrote this many years ago because the IMatch 3.6 batch processor was very limited in its handling of non-sRGB colour profiles.  Mostly the script is not needed in IMatch 5, but it has some uses and it may help in your case until the bug is squashed.

[Mario]

I don't see it (maybe you have already). Did you attach the original file you are using for your test?

Using a file format with lossy compression also impacts colors. If you use a JPEG for input and a JPEG for output, color shifts can happen, caused by the compression algorithm used for JPEG. The amount varies depending on the amount of compression used.

I can look into this when you show me the original file used for the test.

[ChrisMatch]

I will provide the files in the evening.

I started with a TIF file (to simulate what in my normal workflow is a NEF file).
The TIF file got converted by PhotoNinja (my raw converter) to a JPG file.
This JPG file then goes to the batch converter which again produces a JPG file.
    >>   TIF   --- 'PhotoNinja' --->   JGP    --- 'batch converter' --->   JPG


This simulates my normal workflow where I usually start with a NEF image.
Let the raw converter generates a final JPG image.
Which get's a last transformation for web upload (e.g. add a frame, size transformation) via the batch converter.

As far as I know I use a compression level of 85% - but I will check this in the evening too.

Thanks so far.
Chris

[ChrisMatch]

Hi

I tried a new conversion:
  TIF   --- 'PhotoNinja' --->   TIFF --- 'batch converter' --->   JPG
but using a tiff as input for the batch converter didn't help neither.




[attachment deleted by admin]

[ChrisMatch]

Here are some example files...

[attachment deleted by admin]

[Winfried]

One problem I see with this workflow: Your TIF is garbage. It does not contain a profile.
A rgb-image without an icc-profile ist garbage. Nobody knows what the values for r,g,b really mean.
The TIF is created with paint.net. It seemes to me that this program isn't able to handle colors correctly, i.e. doesn't understand color-management.
And PhotoNinja does not know how to handle this garbage, i.e. how to interpret rgb-data without a profile.
Maybe try a test with my file green_102-204-102_sRGB-1.jpg, which contains a valid icc-profile

Winfried

[Ferdinand]

Garbage is a strong word.  I do agree that trying to assess colour management using an image without a profile does seem a little pointless.

That said, you should be able to process this image through the batch processor successfully so long as the option to convert to a profile is unchecked.  After all, you could open it in Photoshop, leave it as colour unmanaged, do some manipulations and save a colour unmanaged version.

Where I think the problem probably lies is in the Lab sharpening option.  If the batch processor converts to Lab to sharpen in the L channel, it needs a profile to convert back to, and the image doesn't have one, and the batch processor settings don't specify one, so what is it supposed to do?  Actually, it probably has to assume a profile in order to convert to Lab in the first place, so it has had to make this assumption twice.

I don't see how Lab sharpening could work reliably with a colour unmanaged image.

[ChrisMatch]

Maybe try a test with my file green_102-204-102_sRGB-1.jpg, which contains a valid icc-profile

Thanks for the suggestion - I tried it - and got the color shift again...



[attachment deleted by admin]

[ChrisMatch]

Thanks for having a look at the files
but I don't find your argument cogent
(hope this is the right word - in German I would say 'nachvollziehbar').

The original TIFF should be of no relevance.
It is the JPG that goes into the batch processor
and a JPG coming out of the batch processor.
And I see a color shift between those two JPGs.

I can also reproduce this if I send a TIFF (coming out of PhotoNinja - which also should contain a color profile)
into the batch processor with the same color shift.
I didn't upload this TIFF because it exceeds the allowed size limit.

[Winfried]

Very, very strange.... 
My files work fine (i.e. as expected) in  my environment without problems.
I have to do some more tests with different settings.

I still have problems to understand your file "colors.tif"
Who/which application created this file?
Exiftools shows:
Image Width                     : 800
Image Height                    : 600
Bits Per Sample                 : 8 8 8 8
So, although it seemes to be a rgb-file (=3 chanels), it has 4 chanels.
Tif-files can have alpha-chanels. Maybe PhotoNinja cannot handle this (according to my tests).
But this is no explanation why my files do not work in your environment.

Winfried

[ChrisMatch]

I have to do some more tests with different settings.

Thanks for this idea.
I think I found the setting that is causing the problem.

If I disable canvas, the color shift is gone !?

[attachment deleted by admin]

[Winfried]

Yes, I can confirm this

Winfried

[ChrisMatch]

Yes, I can confirm this

Winfried

Great - and many thanks for all the help. 
Now I am confident Mario will be able to fix it.

[Mario]

Both effects have been fixed for the 5.2.16 build.

When you use a canvas, IMatch has to consolidate color profiles (the color profile IMatch uses internally for all operations on the canvas, the color space of the background image you load into the canvas (if you do) and the color profile which may be contained in the image currently being processed. This did not work as I had designed it.

When implementing the LAB sharpen I did not notice that the required splitting of the image and the later re-combining after sharpening the L channel under some conditions "lost" the color profile.

[Ferdinand]

When implementing the LAB sharpen I did not notice that the required splitting of the image and the later re-combining after sharpening the L channel under some conditions "lost" the color profile.

What happens if LAB sharpen is enabled for batch processing an image without an embedded colour profile?  Does IMatch assume something like sRGB for convenience, in order to do the conversion and sharpening and reverse conversion?

[Mario]

If an image has no embedded color profile, no color management can be performed.
(The Windows color management functions return only an error message in this case, also the 3rd party image libraries).

This is when things get messy and fuzzy and which in general should be avoided. I'm too tired to spawn yet another discussion about color management, the lack of support for a proper method to specify a color profile in EXIF, the often mysterious camera vendors and the DCF tried to work around that...

After loading an image, IMatch asks the render queue if the file has a profile. The image library checks if the file has an embedded profile. If this is not the case, it also checks if the file uses the so-called DCF Optional Color Space as specified by "Design rule for Camera File System" (DCF) Version 2.0. This means that it checks the ColorSpace, the WhitePoint, the PrimaryChromaticities, the YCbCrCoefficients and the WhitePoint for specific values recorded in the DCF 2.0. If these values are matched, the file is assumed to be in the Adobe RGB color space and an AdobeRGB color profile is loaded. This then again allows color management.

If this is also not the case, IMatch must assume that the file is sRGB in order to transform it into a target color space (for internal processing or display).

[Ferdinand]

I'm too tired to spawn yet another discussion about color management

Me too.  Thanks for the answer, it made perfect sense.