Photoshop - mask transparency math

I had previously reported that in Photoshop, when we paint on a pixel mask (either the layer pixel mask or the Selection mask) with a color whose RGB values are all C (a gray color), the transparency coefficient of the mask, t, is made approximately:

t=(C/255)^1.25


I have since gotten some more insight into this very curious matter.

First, that exponent value (1.25) only pertains if:

• The working RGB color space is sRGB

• The working gray color space is Gray Gamma 1.8

How does that come about?

I can't really explain why Adobe chose to do it this way. But, reverse engineered, it seems to work this way:

• The relative luminance of the RGB color, Y, is determined (based on the definitions of the working RGB color space).

• The grayscale coordinate value K, that would, in the working gray color space, give that same relative luminance, Y, is determined.


• The transparency coefficient of the mask is made 1-(K/100).


Note that once the transparency coefficients are established for a mask, it does not matter of the working color spaces are changed - the transparency coefficients are absolute once determined, and do not change.

Now, where does the 1.25 come from? Well, if we go through the math, we find that:

• If the gamma of the RGB color space is gRGB
• and the gamma of the grayscale color space is gGS

then the transform becomes:

t=(C/255)^(gRGB/gGS)

For the sRGB there is not a simple gamma, but the value 2.25 is a good stand-in. The default grayscale color space is Gray gamma 1.8, and it's gamma is - well, you know. Thus the exponent on the transform is 2.25/1.8=1.25.

An interesting related matter is the working of the Info panel when examining a pixel mask.

First we will review the ordinary use of the Info panel (in examining image pixels). If we set one sub-panel to show the grayscale values of image pixels, it reports in terms of the grayscale color space coordinate K. This works just like R, G, or B, except upside down, on a scale of 0-100% (not 0-255), and of course there is only one, since this is a grayscale color space, only requiring one coordinate.

Thus, the way we "decode" a grayscale image, with coordinate K, is this way:

Y=(1-(K/100))^gGS

where Y is the relative luminance implied by the value of K and gGS is the gamma of the particular grayscale color space.

Now when we use the info panel, set up this way, to examine a mask (in black/white mode), K tells us the mask opacity coefficient (the inverse of the mask transparency coefficient) at the particular point; that is, K=0% means the mask is fully transparent, K=100% means that the mask is fully opaque, and K=30% means that the mask is 70% transparent.

This is independent of the two color spaces used when the mask was painted (the resulting transparency coefficients are "absolute"), and the two color spaces now in effect.

If the other sub-panel is set to show the RGB values of the image pixels (according to the working RGB color space)normally in normal use, now it shows what RGB values (under the current working RGB color space), and considering the current working grayscale color space, would produce the transparency coefficient we have at this point if we wanted to paint it someplace else.

By the way, if we set the working RGB color space to sRGB, and the working grayscale color space to sGray (which follows exactly the same nonlinear gamma precompensation as does sRGB), then, if we paint on the mask with a gray RGB color R,G,B=C,C,C, the transparency coefficient established is just C/255.

Honest, guys, could I make something like this up?

Ah, the mind of Adobe!

Best regards,

Doug

This drawing shows what seems to be the chain of calculations that leads from a color used to paint on a mask to the resulting transparency, t, of the mask in that region:


This shows the apparent chain of calculations that affect the indications given in the Photoshop Information dialog when inspecting a mask:


Best regards,

Doug

Hi Doug,

Thanks for sharing your findings. I never really thought about the implications of the Grayscale workingspace settings on the effect a mask has. One more reason to not send a layered file to a client, since we have no control over their Photoshop color settings.

Cheers,
Bart

Hi Doug,

impressive find, I never thought about the correlation of the gray scale working space and masks!

However I just did some testings as well and it seems like it really doesn't matter, because the profile is not "converted to" but rather just "assigned" (which means the actual numbers do never change, whatever your working profile is set to).
However of course it will change, when you do convert to another profile, so you have to measure the K-value of the actual color profile in the info palette.

So no worries, Bart, the appearance won't change with the gray scale profile .

I attached two screenshots to illustrate my point. Watch the gray scale profile as well as the color readout for the K and the L*.

Hi, Jonas,

Jonas Wendorf said:

impressive find, I never thought about the correlation of the gray scale working space and masks!

However I just did some testings as well and it seems like it really doesn't matter, because the profile is not "converted to" but rather just "assigned" (which means the actual numbers do never change, whatever your working profile is set to).
However of course it will change, when you do convert to another profile, so you have to measure the K-value of the actual color profile in the info palette.

So no worries, Bart, the appearance won't change with the gray scale profile .

I attached two screenshots to illustrate my point. Watch the gray scale profile as well as the color readout for the K and the L*.

Click to expand...

Thanks for doing this testing. Your findings seem consistent with my understanding.

Thanks again.

Best regards,

Doug

Jonas Wendorf said:

So no worries, Bart, the appearance won't change with the gray scale profile .

Click to expand...

Hi Jonas,

As much as I appreciate the consolation, there is an issue.

Try the following:

• Create an image filled with pure white (I made a new 100x100 pixel image, 16-b/ch RGB, ProPhoto RGB).
• Double click the Background layer to promote the layer to a common layer which can have a mask and transparency.
• Add a mask, and fill it with 50% gray.

The info panel will read 128,128,128 (50% opacity) when the Gray working space is set to Gray gamma 1.8 (the same gamma as the ProPhoto working space, just like I've set my colorsettings up). However, the layer opacity results in 109, 109, 109 when the Gray working space gamma is changed to 2.2 (or a client has it set to that).

Therefore, the layer contribution in a composite varies with the mismatch between the gammas of the RGB and Gray working spaces. The mask values stay the same, but their effect on the layer opacity changes.

Cheers,
Bart

Bart_van_der_Wolf said:

Thanks for sharing your findings. I never really thought about the implications of the Grayscale workingspace settings on the effect a mask has. One more reason to not send a layered file to a client, since we have no control over their Photoshop color settings.

Click to expand...

Note however that, once the mask has been made, its nature of the mask (that is, its transparency coefficient at each point) unaffected by the color profiles in effect.

Of course, the appearance of the composite image may well be effected by the profile in effect. I plan to do some further work on that matter as time allows.

I do not mean by the above to suggest that it is attractive to "publish" PSD files (or Word DOC files, or any other proprietary file) except in the case of well controlled collaboration.

By the way, thanks for your numerous insights into this matter from time to time.

Best regards,

Doug

Hi, Bart,

I think your point is a good one, but I need to be certain about some details (see embedded).

Bart_van_der_Wolf said:

Try the following:

• Create an image filled with pure white (I made a new 100x100 pixel image, 16-b/ch RGB, ProPhoto RGB).
• Double click the Background layer to promote the layer to a common layer which can have a mask and transparency.
• Add a mask, and fill it with 50% gray.

Click to expand...

Tell me what you mean "50% gray". Do you mean fill the mask with a gray color whose K is 50%? That does not give (necessarily) give t=0.5. (If the gray and RGB profiles in effect when the mask is "painted" are the same, then it will. That is perhaps what you mean.)

How did you "fill it with 50% gray"?

The info panel will read 128,128,128 (50% opacity) when the Gray working space is set to Gray gamma 1.8 (the same gamma as the ProPhoto working space, just like I've set my colorsettings up).

Click to expand...

What are you looking at with the Info panel? The mask or the image?

If you are looking at the mask, yes, the RGB readings at each point are affected by the mismatch between the grayscale and RGB profiles then in effect. But this is just a manifestation of the curious way mask transparency is controlled (sort of "working in reverse"). (See the second figure in my illustrated message.)

If you are inspecting the mask, only the K value of a grayscale display in the Info panel will tell us the mask opacity at that point.

Best regards,

Doug

Doug Kerr said:

Tell me what you mean "50% gray".
[...]
How did you "fill it with 50% gray"?

Click to expand...

Hi Doug,

The Photoshop menu Edit|Fill... offers an option Use: 50% Gray. That will amount to RGB 128, 128, 128 (or just 128 for a grayscale layer such as a mask) being used as rounded 8-bit values in the info panel.

What are you looking at with the Info panel? The mask or the image?

Click to expand...

I'm looking at the image itself (while hovering the mouse pointer over the image itself), which shows the expected 50% opacity of White as 128,128,128, but only when the Photoshop color settings have been setup with a matched gamma for the RGB and Gray working space.

Since the intended effect of using a mask is to adjust the layer opacity (its contribution to the composite of layers), it helps if the composited effect is predictable and as intended.

On my own computers, I have the RGB and Gray working spaces setup to identical gammas, because I want predictable changes from RGB to Grayscale mode or vice versa. Therefore I never realised the importance for the effect of masks before.

Thanks for pointing out the connection.

Cheers,
Bart

Hi, Bart,

Bart_van_der_Wolf said:

Hi Doug,

The Photoshop menu Edit|Fill... offers an option Use: 50% Gray. That will amount to RGB 128, 128, 128 (or just 128 for a grayscale layer such as a mask) being used as rounded 8-bit values in the info panel.

Click to expand...

Oh, yes, duh! Thanks.

Just to link this in to the theme here, if the RGB color space in use and the nominated grayscale color space have the same "gamma function", that would produce a mask transparency of 0.5 (a mask opacity of 0.5 or 50%). Otherwise it will produce a different mask transparency.

I'm looking at the image itself (while hovering the mouse pointer over the image itself), which shows the expected 50% opacity of White as 128,128,128, but only when the Photoshop color settings have been setup with a matched gamma for the RGB and Gray working space.

Click to expand...

I'm a little confused here. When actually looking at the image in the layer of interest (click on the "image" thumbnail) I don't find that the pixel color indications in the Info panel are affected at all by the mask. I assume that's because the mask actually works by "modulating" the opacity of the pixels, not their color.

For example, with an image of 255,255,255 and a mask color of 146,146,146 (leading to a mask transparency of 0.5 - a "K" of 50% when examining the mask) (this is in sRGB with the grayscale space nominated as Gray Gamma 1.8), when examining the image, the Info box shows the color of the pixels to be 255,255,255. (Their opacity, not reported by the Info box, is presumably 50%.)

What am I missing?

Best regards,

Doug

Hi Bart,

true, now that you're saying it, I see!

However for a real world test just take two images, use a gradient mask to combine them and change the gray working space to whatever you like while watching the composite image (not the mask!).
If you don't believe me that there's no difference, do a stamp visible every time and compare them with the "Difference" blend mode.

Works for me .

Jonas

Edit: Some changes. I misread your answer at first.

Hi, Bart,

Note that is we want to see the "effective" color of the pixels in our test layer, as affected by a mask on that layer, we need to put a black layer underneath (and of course look at the composite image).

Then with the subject layer at 255,255,255 and the mask on that layer at t=0.5, the composite image will show 128,128,128.

That is because the upper-layer pixels (255,255,255) are given by the mask a transparency of 0.5, and thus only contribute to the image buildup as 128,128,128.

The background layer (0,0,0) contributes to the overall buildup by the fraction <1- the opacity of the pixels in the upper layer>, which is also 0.5, but of course 0.5 times 0,0,0 is 0,0,0.

Thus the net color of the observed pixels in the composite image is [128,128,128] + [0,0,0], or [128, 128,128].

I plan to do some further work today to see how the workings of the mask transparency is affected by changes in the working color space.

That is, we know that the impact of a 60% mask transparency is to reduce the pixel opacity to 60%, and the implication of that is that the effective values of all the color coordinates are changed to 60% of their actual values. That is, a pixel whose coordinates are 260, 200, 150 acts in the buildup (for example, if seen over a black background) to have coordinates 150,120,90.

Now, if we change the color space so that the coordinates are different, will that ratio still hold? I should know that shortly.

Best regards,

Doug

A small update and refinement.

If the file has no attached profile, then the "scaling" between the RGB gray color (C,C,C) used to paint on the mask and the resulting mask transparency coefficient, t, is based on (to use Bart's apt phrase) the "mismatch" in gamma between the working RGB color space and the nominated grayscale color space.

If the file has an attached profile, then the "scaling" between the RGB gray color used to paint on the mask and the resulting mask transparency coefficient, t, is based on the "mismatch" in gamma between the attached RGB color space (whether or not it is the working color space) and the nominated grayscale color space.

Not really surprising. I was just getting my ducks lined up before I proceeded to the next stage of my work, and this came out of a more complete review than previously!

Best regards,

Doug

Doug Kerr said:

Hi, Bart,

Note that is we want to see the "effective" color of the pixels in our test layer, as affected by a mask on that layer, we need to put a black layer underneath (and of course look at the composite image).

Click to expand...

That's correct, that's the goal we're trying to achieve, a composite of layers. However, there is also another factor to consider; The "Blend RGB Colors Using Gamma" setting. To avoid wrong blends (remember the green/magenta edge example in an earlier thread about gradients) it should really be active and set to gamma 1.

The plot thickens.

Cheers,
Bart

Let me summarize my current outlook on these matters and illuminate some areas on which I plan to next work.

a. When we draw on a pixel mask with the RGB gray color C,C,C that mask region is given a transparency, t, that is based on C but scaled by the discrepancy in the nonlinear transfer functions (we may say, simplistically, the "gammas") of the attached profile color space (or, if non, the current working color space) and the nominated grayscale color space.

b. When those two color spaces have consistent transfer functions (as, for example, for sRGB and sGray), then:


c. Once the mask has been created, the map of t is unaffected by changes in the attached profile color space, working color space, or nominated grayscale color space.

d. When we examine the mask using the Info panel with a grayscale subpanel, the reported value of K indicates the mask transparency, thus:


d. When we examine the mask using the Info panel with an RGB subpanel, the reported RGB indicates the color that, if used to paint on the mask under the current color space situation, would produce the indicated transparency.

e. [This needs some further confirmation as to the precise details.] The effect of a mask on the pixels under its jurisdiction (those in its layer) is to scale their opacity (ordinarily, 100% at birth) by the factor t. Thus, pixels having an actual opacity of 100%, "seen through" a mask region where t=0.6, are identical to the same pixels where we have in come other way set their opacity to 60%.

f. [This needs some further confirmation as to the precise details.] The "net effect" of the pixels is as if all their RGB coordinates are scaled by their opacity (and thus by t).

Thus a pixel with RGB=200,150,100, under a mask region where t=0.6, will participate in the image buildup as if it had coordinates 120, 90, 60.

However, to see/measure that directly, there must be a black layer underneath. Else we will see 40% of the underlying pixel RGB values as well.

My next work will be to investigate the implications of item (f) when the working color space is not an RGB space, or when the working space now is different from that when the mask was created, or when the image has been converted to a different working space (but the mask remains in place).

Best regards,

Doug

Hi, Bart,

Bart_van_der_Wolf said:

That's correct, that's the goal we're trying to achieve, a composite of layers. However, there is also another factor to consider; The "Blend RGB Colors Using Gamma" setting. To avoid wrong blends (remember the green/magenta edge example in an earlier thread about gradients) it should really be active and set to gamma 1.

Click to expand...

Thanks. I usually dodge that by presuming the "normal" blend mode (and saying so, but I didn't here)!

The plot thickens.

Click to expand...

It do indeed!

Best regards,

Doug

The matter has been raised as to what happens to Photoshop PSD files with layers and masks if transported to another platform where an unexpected working color space situation may be in effect.

As I understand it, if the file has an attached profile, then on a normal open, that profile will be used as the working color space.

But suppose that there is no attached profile. Then the working space will be the current default .

In any case, the RGB values of the pixels are not changed by the color space situation at the "receiving" end. The interpretation of those RGB values as colors will, of course, vary (as for any image file).

As I have mentioned before, the transparency values of any mask are not changed by a change in the color space environment.

And thus the "effective" RGB values of the image pixels, affected by their opacity, in turn affected by the transparency of the mask, will not change either. Of course, the "appearance" of the composite image will vary with the color space in effect, as for any image file.

Best regards,

Doug