• Please use real names.

    Greetings to all who have registered to OPF and those guests taking a look around. Please use real names. Registrations with fictitious names will not be processed. REAL NAMES ONLY will be processed

    Firstname Lastname

    Register

    We are a courteous and supportive community. No need to hide behind an alia. If you have a genuine need for privacy/secrecy then let me know!
  • Welcome to the new site. Here's a thread about the update where you can post your feedback, ask questions or spot those nasty bugs!

What does higher ISO in digital give us? Is it really a higher ISO?

Hi, Bart,

Well, I'm working my way thorough all this. I am having a little problem with your chart. I have assumed that the six "grayscale" patches on the mini CC that you mention are intended to be the patches often called 24 through 19 (from darkest to lightest).

That's correct. 1=19 (White), 2=20, 3=21, 4=22, 5=23 and 6=24 (Black)

But I am surprised that the noise values for "patch 6" on your table are all smaller than for "patch 5".

Normally, we would expect the noise values (for any given exposure setup and ISO setting) to rise monotonically with luminance, owing to the shot noise varying as sqrt(N).

That's correct at the Raw level, but we're looking at demosaiced and gamma/tonemap adjusted images here. I wanted to see what we end up with after the data crunching, as that is what we'll base our visual judgement on. Maybe C1 has a built in shadow noise reduction, we don't know for certain. I switched off all noise reduction and sharpening that I could.

All I can think of is that your exposure was so high that the lightest patch always saturated (which of course would cause a decrease in digital noise), but that doesn't seem likely given the relationship between the various entries in the "ISO 1600" group.

The final target values for the white patch 1/19 were R=G=B=231, and for the Black patch 6/24 R=G=B=46, so there was no clipping involved. The black isn't very black, because it is semi-glossy/matte. I didn't want to stretch the contrast as one might do if there are no real blacks in an image, I only tweaked the exposure slider for identical output RGB values.

What am I missing?

Nothing, but I can't explain it either. It's perhaps best to do one's own test, even if it is only with a slightly defocused gray card instead of a step wedge scale.

One more question while I've got you up. Is the "native ISO" the one at which the photodetectors are saturated from an electronic standpoint (the so-called "well full" level) at the same photometric exposure at which the DN tops out?

That would depend on one's definition of native ISO. My measurements at the Raw non-demosaiced level show a maximum Dynamic range (engineering definition) and minimum gain at ISO 100 for the 1Ds3 I have (ISO 80 for my 1Ds2). Others have reported similar findings for other Canon models. ISO 100 is also the lowest standard ISO the camera's offer. Nikons seem to be closer to ISO 200 (although I don't know if that depends on the source of the chips, Sony or another manufacturer), but I haven't measured that myself. That would indeed require an exposure at the clipping level, and a noise floor, and must be measured at the Raw data level. ISO 'L' is ineffective in the 1Ds3 as far as Dynamic range is concerned (It's just ISO 100 gain pulled 1 stop in the camera).

Cheers,
Bart
 

Doug Kerr

Well-known member
Hi, Bart,

Thanks for clarifying those issues. I know there are many mysteries we don't fully understand at this time.

But I am having trouble knowing where you are going, and how you get there.

The key to your premise seems to be this:

"To get an optimal noise performance we need to find a balance between under-exposure noise, and amplified system-noise."

I'm assuming that by "underexposure noise" you are referring to the fact that the signal-to-shot-noise ratio decreases with photometric exposure.

But can't find anything about the relationships involved that suggests the concept of a "balance". And what are the variables we want to change will trying for that "balance"?

It sounds as if the actual practice being discussed would be how to choose the best ISO setting and amount of "push" to get the best noise performance in a certain situation. Your description suggests that, in comparing alternatives, we would always "push" (if needed) to get a consistent "exposure result" (for example, consistent RGB values in the delivered developed image). Makes sense as a premise for comparison.

Now what might be our constraints as we seek to find this optimum for a particular shot? I only find two:

a. Amount of exposure bias (basically, what do we need to do to overcome the imperfection of the metering scheme as it deals with this particular scene, or scene of this type!) (Note that there is a tendency to describe this as "how much headroom do we need", but in fact we want to have, if we can arrange for it, no headroom at all. If we do need headroom, it is because of the fact that we cannot predict exactly how the photometric exposure will be distributed)

b. Available photographic exposure (Ev); that is, what aperture do we have available and care to use, and what shutter speed is "acceptable". (Example: we decide that we need an aperture of f/8 to get the DOF we want, and need a shutter speed of 1/500 sec to capture the action on the field.)

If the metering system (with zero EC) does not in fact give us the headroom we need, we cater to that by setting a negative EC. We can't contemplate a corresponding "push" in development; if straightforward operation led us to a photometric exposure in which the highlights saturated the system (meaning that the highlights got R, G, or B values "above 255"), then if we apply an EC of -1 Ev, and then apply a"push" of 1 Ev, we will still have the highlights with R, G, or B values above 255.

Now, this having been said, what choice of ISO sensitivity will give us the best noise result? It seems to me that it would always be the lowest one that just meets criterion (b), above, given the EC already decided upon.

Your chart seems to suggest this. It assumes a consistent "exposure result". And that being so, the best noise performance comes from the lowest ISO settings, with no EC and no push.

So I must be missing something here. What is it?

Thanks.

Best regards,

Doug
 

Asher Kelman

OPF Owner/Editor-in-Chief
Taking an overhead lit stage with overall low light and harsh dynamic range.

Perhaps, Doug and Bart, at least for my needs and to get a concrete practical example into play, let me offer some limits to work with.

I can narrow the parameters to f3.5 to 4.5 and speeds from 1/120 to 1/200 as the max range contemplated. As far as the most black, blacks, these are already swallowed up in noise no matter what ISO over 800 so we can just accept that the very black blacks are lost and it doesn't in practice matter. The specular light will still bring out details in the tuxedos and black dresses the performers wear.

The job is to get the performers hands faces and instruments and white shirts well rendered with the least noise. Let's arrive with a spot meter to take a max reading off a forehead or white shirt. What else would you need to start to rationalize how to choose the ISO and the -EV used at the lower ISO than one would have otherwise have chosen. (Since we know from experience that exposing to the right under these conditions means lost highlights and the degradation of finest detail above 230 or so)

Asher
 

Doug Kerr

Well-known member
Hi, Asher,

The job is to get the performers hands faces and instruments and white shirts well rendered with the least noise. Let's arrive with a spot meter to take a max reading off a forehead or white shirt. What else would you need to start to rationalize how to choose the ISO and the -EV used at the lower ISO than one would have otherwise have chosen. (Since we know from experience that exposing to the right under these conditions means lost highlights and the degradation of finest detail above 230 or so)

Here is my suggestion.

It is based on the assumption that the camera uses the "SOS" approach to assessing ISO sensitivity (as do contemporary Canon cameras) and the spot meter is calibrated consistent with ISO 2720.
There are many ways you can play the strategy. Perhaps the most straightforward way to do it is to find the EI setting (ISO speed setting) for the meter such that, when metering off a "whitest object", an acceptable aperture/shutter speed combination is indicated.

Then, use that same aperture and shutter speed in the camera for the shot, with the camera set to an ISO sensitivity 6.4 times the EI set in the meter (2-2/3 stops higher).

EC is not involved, since the camera's metering system is not involved.

If you camera only accepts ISO sensitivity setting in "full stop" increments from ISO 100 (or you wish to only use such), then you need to try in the spot meter only EIs in the series 16, 32, 64, 125, 250, 500, etc.

Best regards,

Doug
 

Asher Kelman

OPF Owner/Editor-in-Chief
Perhaps the most straightforward way to do it is to find the EI setting (ISO speed setting) for the meter such that, when metering off a "whitest object", an acceptable aperture/shutter speed combination is indicated.

Then, use that same aperture and shutter speed in the camera for the shot, with the camera set to an ISO sensitivity 6.4 times the EI set in the meter (2-2/3 stops higher).
Thanks Doug,

I'm a little flummoxed here as it would appear that you would be pushing the brightest whites into white out! Or, perhaps, you're you saying that I expose for the brightest spots as if it was 18% gray? That would place the brightest whites in the center of the histogram. If that's the case, I can see that there's headroom for your maneuver. This would presumably deal with getting the highlights safely.

For lowering noise, one could then go one stop down and give +1 Ev EC.

However, if I'm not correct on the 18% grey presumption, then I guess I'm not sure now how we are using the light meter in your scenario.

I did a quick test with the camera at Manual exposure, f4.0, 1/160. The camera focused on a shiny Apple white brick power supply under the overhead kitchen light, to reproduce the overhead incandescent stage lights. I adjusted the ISO setting so that the leading brightest edge of the histogram was no further than the center of the X axis. That turned out to be ISO 400. Then I moved up the ISO 2 full stops to ISO 1600 took another shot. The resulting image was underexposed a tad, but that's what I want. I'd like to be able to push to recover the highlights and have less noise. So at least now I have a paradigm to test in the performance halls.

Thanks,

Asher
 

Doug Kerr

Well-known member
Hi, Asher,

I'm a little flummoxed here as it would appear that you would be pushing the brightest whites into white out! Or, perhaps, you're you saying that I expose for the brightest spots as if it was 18% gray? That would place the brightest whites in the center of the histogram. If that's the case, I can see that there's headroom for your maneuver. This would presumably deal with getting the highlights safely.

Let me first articulate what I will assume you mean by your "18% gray" comment:

1. A situation in which a shot of a uniform-luminance "neutral" scene, taken at the exposure chosen by the camera's automatic exposure control ("metering") system will, after "standard" development, result in a JPG image in which the image has RGB values that (in accordance with the applicable color space) represents a "luminance" 18% of the luminance represented by the highest possible RGB code, 255,255,255.

A corollary often assumed from that specific situation is:

2a. If, with the same ISO sensitivity in effect, we expose that scene with an exposure 100/18 times the exposure in the first case (2.5 stops higher), the image will have RGB values of (essentially) 255,255,255 (that is, will be "exactly saturated" throughout).

or

2b. If, using the same exposure as in the first case, but with an ISO sensitivity 100/18 that of the first case (5.6 times as much), the image will have RGB values of (essentially) 255,255,255 (that is, will be "exactly saturated" throughout).

In a situation where (1) is exactly true, (2a) and (2b) will not necessarily be true. This is a result of the way the development process (in-camera or post) deals with tone mapping and the like, and can be effected by various available options (such as the parameters controlled by the selection of a "Picture style" in Canon cameras, white balance setting, etc.).

Now, in my Canon EOS 40D (with the various auxiliary settings set however I found them last night), if I take a metered shot of a uniform-luminance scene, and take the JPEG output, the average RGB codes across a central region of the image are 106,106,106, which imply (under a strict interpretation of the sRGB color space) a "luminance" about 14% of that implied by RGB=255,255,255 (about 2.8 stops below that).

If I then keep the same aperture and shutter speed but raise the ISO sensitivity by 3 stops (e.g., from ISO 100 to ISO 800), we might expect the image to be "fully blown out", with RGB values of 255,255,255. But rather the average RGB values in the developed image were 242,242,242, (reflecting a headroom of about 1/6 stop).

If I were to keep the same aperture and shutter speed, but instead raise the ISO sensitivity to ISO 640 (6.4 times as high, almost exactly 2-2/3 stops), the average image RGB values were 226,226,226 (reflecting a headroom of about 0.4 stop). (Hold this thought.)

The procedure I suggested follows this latter situation. It would, for this camera, result in the "metered shirt" ending up in the image with about 0.4 stops of headroom.

For lowering noise, one could then go one stop down and give +1 Ev EC.

I'm not sure what you mean by "go one stop down" and by "give +1Ev EC".

I'll put aside for then moment comment on what is and is not appropriately denominated in Ev units, and make the assumption that by this phrase you mean:

• Expose at one stop less than what my procedure would suggest, and
• Apply a one-stop push in development

I have been trying to follow Bart's discussion of what I think is this ploy, and I am not yet able to figure out what it does, and why. Perhaps that will unfold better later in the day. For the moment, I just cannot comment on that.

However, if I'm not correct on the 18% grey presumption, then I guess I'm not sure now how we are using the light meter in your scenario.

Let me summarize the technique I suggest. I urge you not to try to connect it to the phrase "18% gray". Instead, refer to the passage highlighted in blue above.

• Using a spot meter, measuring on a "brightest object" in the scene, choose an EI setting (ISO setting) in the meter such that a "workable" combination of aperture and shutter speed is indicated.

• Set the camera ISO sensitivity to 6.4 times the EI you used in the spot meter, set the aperture and shutter speed to those coming out of the work with the spot meter, and shoot.

I did a quick test with the camera at Manual exposure, f4.0, 1/160. The camera focused on a shiny Apple white brick power supply under the overhead kitchen light, to reproduce the overhead incandescent stage lights. I adjusted the ISO setting so that the leading brightest edge of the histogram was no further than the center of the X axis.
I have no idea where that would be on a relative luminance scale (or the relative exposure scale, which as we say above do not exactly track). But is doesn't matter, as this is an empirical determination (just as was mine with my 40D).

That turned out to be ISO 400. Then I moved up the ISO 2 full stops to ISO 1600 took another shot. The resulting image was underexposed a tad, but that's what I want. I'd like to be able to push to recover the highlights and have less noise. So at least now I have a paradigm to test in the performance halls.

Indeed. A perfectly practical approach.

Best regards,

Doug
 

Asher Kelman

OPF Owner/Editor-in-Chief
Now, in my Canon EOS 40D (with the various auxiliary settings set however I found them last night), if I take a metered shot of a uniform-luminance scene, and take the JPEG output, the average RGB codes across a central region of the image are 106,106,106, which imply (under a strict interpretation of the sRGB color space) a "luminance" about 14% of that implied by RGB=255,255,255 (about 2.8 stops below that).

If I then keep the same aperture and shutter speed but raise the ISO sensitivity by 3 stops (e.g., from ISO 100 to ISO 800), we might expect the image to be "fully blown out", with RGB values of 255,255,255. But rather the average RGB values in the developed image were 242,242,242, (reflecting a headroom of about 1/6 stop).

If I were to keep the same aperture and shutter speed, but instead raise the ISO sensitivity to ISO 640 (6.4 times as high, almost exactly 2-2/3 stops), the average image RGB values were 226,226,226 (reflecting a headroom of about 0.4 stop). (Hold this thought.)

The procedure I suggested follows this latter situation. It would, for this camera, result in the "metered shirt" ending up in the image with about 0.4 stops of headroom.

Thanks Doug for your careful reasoning and test.

The procedure you used getting an average reading of a uniform-luminance scene does not address the tiny and most important over illuminated areas of a violinists delicate hands. These maximum illuminated and reflected areas have the most critical detail and contribute little to the "average" reading. Also the stage is not uniformly lit and the lighting rakes the face and hands. So that's why, I instead chose the leading edge of the histogram to be at the center of the graph. That would, I believe, correspond to using a spot meter on the brightest area, if one could catch them, and putting that spike in the same position in the histogram.

I just wanted to let you know where our measurements with the initial test exposure differ. From this, would you still give the value of x 640 by which to increase the ISO figure?

I'd like to have a headroom of ~ 1 full stop. So the exposure to the final image would be - 1EV. Then, according to Bart's ploy, I'd push the exposure in RAW processing to recover that by + 1 EV.

Asher
 

Doug Kerr

Well-known member
Hi, Asher,

Thanks Doug for your careful reasoning and test.

The procedure you used getting an average reading of a uniform-luminance scene does not address the tiny and most important over illuminated areas of a violinists delicate hands.
My procedure did not presume an average reading of a substantial area. It involved an average reading of a small area (perhaps a hand). The word average, while apt (that's what a spot meter always measures, the average over its "regard") perhaps misdirected your attention.

These maximum illuminated and reflected areas have the most critical detail and contribute little to the "average" reading. Also the stage is not uniformly lit and the lighting rakes the face and hands. So that's why, I instead chose the leading edge of the histogram to be at the center of the graph. That would, I believe, correspond to using a spot meter on the brightest area, if one could catch them, and putting that spike in the same position in the histogram.

I understand, and that seems very reasonable.

I just wanted to let you know where our measurements with the initial test exposure differ. From this, would you still give the value of x 640 by which to increase the ISO figure?

Since I have no idea what photometric exposure on the sensor (vis_à-vis saturation) your datum point on the histogram scale represents, I can't come to any conclusion on that.

I'd like to have a headroom of ~ 1 full stop. So the exposure to the final image would be - 1EV. Then, according to Bart's ploy, I'd push the exposure in RAW processing to recover that by + 1 EV.

Well, I have no idea what headroom your basic scheme gives, so all I can say is that if you increase it by one stop, you will have at least one stop.

You'll of course find out as you examine test shots.

Whether or not a one-stop push in development is appropriate depends in part on what is the range is in the developed image. I would think you probably want to push until the "100% reflectance" objects are at about RGB 245 or so.

A one-stop push to recover from a one stop underexposure, when the "base" exposure is arbitrarily arrived at, is not a "plan" - just an "activity".

Best regards,

Doug
 

Doug Kerr

Well-known member
Hi, Asher,

Just as a point of reference:

Tests with my Canon EOS 40D show that if we look at the exposure recommendation from spot metering with the in-camera metering system a small area, with a certain ISO sensitivity in effect, and:

• For the actual shot, hold that exposure (aperture/shutter speed) and use an ISO setting 6.4 times the one set during metering (2-2/3 stops)

or

• For the actual shot, hold the ISO setting set during metering and increase the exposure by 2-2/3 stops

we then get, in the developed JPEG image, for that object, an RGB triple representing a luminance very nearly 1/3 stop below saturation.

Best regards,

Doug
 

Doug Kerr

Well-known member
Hi, Bart,

Is it possible that the "unexpected" lower noise value for the "black" CC patch is a result of "single-sided truncation"?

If we think of just abstract numbers (with no limit on range), and, for example, have a random variable with mean=0.1 and standard deviation=1, then many of the values will be negative. And that's fine.

But if we are operating on, for example, a luminance scale, where negative values cannot exist, then those "should be negative" random values will be clipped at zero (and "scored" as such). Then the observed standard deviation will be less than the actual standard deviation.

This is why the ISO standard for dynamic range in a digital camera does, which is based on the luminance at which theh SNR is 1.0, does not actually measure the noise at that luminance (in which case a significant fraction of the values would be negative in luminance, which is impossible).

And I guess this is also the reason for the offset in the Canon scale of values for raw data.

Just a thought.

Best regards,

Doug
 
Hi, Bart,

Is it possible that the "unexpected" lower noise value for the "black" CC patch is a result of "single-sided truncation"?

If we think of just abstract numbers (with no limit on range), and, for example, have a random variable with mean=0.1 and standard deviation=1, then many of the values will be negative. And that's fine.

But if we are operating on, for example, a luminance scale, where negative values cannot exist, then those "should be negative" random values will be clipped at zero (and "scored" as such). Then the observed standard deviation will be less than the actual standard deviation.

Hi Doug,

That's a very good hypothesis, so I double checked. Unfortunately, for the hypothesis, there is no clipping involved and the distribution of the noise is roughly Gaussian (even at Gamma 2.2):

Patch 6 (24), exposed as ISO 400, and no push in post-processing
ISO400_Patch6(24).png


Patch 6 (24), 'under-exposed' 2 stops as ISO 1600, pushed +2EV in post-processing
ISO400+Push2EV_Patch6(24).png


This is why the ISO standard for dynamic range in a digital camera does, which is based on the luminance at which theh SNR is 1.0, does not actually measure the noise at that luminance (in which case a significant fraction of the values would be negative in luminance, which is impossible).

And I guess this is also the reason for the offset in the Canon scale of values for raw data.

Whatever the reason is that Canon does it, it makes perfect DSP sense, because it allows lots of advanced postprocessing which follows the laws of physics.

Cheers,
Bart
 

Doug Kerr

Well-known member
Hi, Bart,

That's a very good hypothesis, so I double checked. Unfortunately, for the hypothesis, there is no clipping involved and the distribution of the noise is roughly Gaussian (even at Gamma 2.2):

Thanks so much for checking. I really didn't think that would be involved here, but the hypothesis was tantalizing!

By the way, when you describe a certain exposure as "underexposed by x stops", I assume you mean as compared to the exposure "recommendation" of the internal metering system. That is for what metering approach (pattern, AF point, aiming if the pattern is "spot", etc.)?

Thanks so much for all this great work.

Best regards,

Doug
 

Doug Kerr

Well-known member
Asher,

Further to the matter of the histogram scale and exposure metering.

In simple tests (defined as doable without my getting out of my chair; I do keep various test targets on the opposite wall of the office just in case) with my Canon EOS 40D, I find that a metered exposure of a uniform-luminance target (one that results in image pixel levels of 118,118,118, or implies a relative luminance of about 18.1%) shows on the histogram at about 0.45 (treating full scale as 1.0).

So in fact, in that camera, having the histogram manifestation of the brightest element (i.e., the leading edge of the "lobe") at "half scale" is a good approximation of spot meterage of that element.

Then, I would expect an exposure 2-2/3 stops higher (of the same exposure with an ISO sensitivity 6.4 times that use in metering) would leave about 1/3 stop of "headroom".

An exposure only 2 stops greater (or an ISO sensitivity 4 times higher) would be expected to yield about one stop of headroom. (It's really a little less - perhaps 2/3 stop - in actual fact, owing to the nonlinearities introduced through image processing, but this is all approximate anyway.)

So that could be a very handy "scheme" for you to employ in your concert work.

As to "push", my own leaning is for a delivered image in which the relative luminance corresponds roughly to the reflectance of the object (essentially the actual underlying objective of the Zone System, although we rarely hear it characterized that way). That would suggest pushing until the "brightest" object shows an RGB triple in the area of 250,250,250.

But you may have other objectives.

Best regards,

Doug
 
Hi, Bart,

Thanks for clarifying those issues. I know there are many mysteries we don't fully understand at this time.

But I am having trouble knowing where you are going, and how you get there.

The key to your premise seems to be this:

"To get an optimal noise performance we need to find a balance between under-exposure noise, and amplified system-noise."

Yes, where the system noise amplification comes from the ISO (gain) setting, and under exposure may be needed at a given ISO to allow short shutterspeeds (e.g. to reduce subject motion).

I'm assuming that by "underexposure noise" you are referring to the fact that the signal-to-shot-noise ratio decreases with photometric exposure.

Correct.

But can't find anything about the relationships involved that suggests the concept of a "balance". And what are the variables we want to change will trying for that "balance"?
[...]
If the metering system (with zero EC) does not in fact give us the headroom we need, we cater to that by setting a negative EC. We can't contemplate a corresponding "push" in development; if straightforward operation led us to a photometric exposure in which the highlights saturated the system (meaning that the highlights got R, G, or B values "above 255"), then if we apply an EC of -1 Ev, and then apply a"push" of 1 Ev, we will still have the highlights with R, G, or B values above 255.

I'm not taking the metering sytem into account. I'm assuming we have already determined the perfect exposure level. What remains is to achieve the best image quality at that exposure level.

Now, this having been said, what choice of ISO sensitivity will give us the best noise result? It seems to me that it would always be the lowest one that just meets criterion (b), above, given the EC already decided upon.

Indeed, the lower we can set the ISO (gain), the lower the (pre-)ADC system noise amplification is. So we need to find the lowest ISO that will still allow a short enough exposure time.

Cheers,
Bart
 
By the way, when you describe a certain exposure as "underexposed by x stops", I assume you mean as compared to the exposure "recommendation" of the internal metering system.

No, the internal exposure metering is too blunt an instrument for challenging lighting scenarios, like the one Asher is confronted with. I am assuming that a perfect exposure level has already been established, by whatever means. Since the aperture and shutterspeed are more or less a given for Asher's concert stage scenario, he probably uses M mode, the only one left is ISO which we want to keep as low as possible.

At high ISOs (above unity gain) there is mostly photon shot noise and read-noise in the shadows, i.e. little useful signal we would lose by 'under-exposing'. But because under-exposure means we can use a lower ISO setting to under-expose (shutterspeed and aperture remain fixed), we also reduce overall noise amplification, and we gain highlight clipping latitude. Postprocessing allows to push the exposure by one or two stops, and still have less noise than from using the next higer ISO setting and getting a correct exposure. That's the balance I mentioned, ISO (gain) noise amplification versus under-exposure+push shot+read noise.

Bart
 

Doug Kerr

Well-known member
Hi, Bart,

Thanks so much for the clarification. All makes sense to me.
I'm not taking the metering sytem into account. I'm assuming we have already determined the perfect exposure level. What remains is to achieve the best image quality at that exposure level.
I understand.

So we need to find the lowest ISO that will still allow a short enough exposure time.

Very succinct. I think I missed this elegantly stated objective amid the details of the entire discussion!

Again, thanks so much.

Best regards,

Doug
 

Doug Kerr

Well-known member
Hi, Bart,

At high ISOs (above unity gain) there is mostly photon shot noise and read-noise in the shadows, i.e. little useful signal we would lose by 'under-exposing'.
Yes, this is a pivotal concept to which I didn't give sufficient recognition.

But because under-exposure means we can use a lower ISO setting to under-expose (shutterspeed and aperture remain fixed), we also reduce overall noise amplification, and we gain highlight clipping latitude. Postprocessing allows to push the exposure by one or two stops, and still have less noise than from using the next higher ISO setting and getting a correct exposure.
The clipping latitude issue is eay to grasp. But why does the digital (numeric) multiplication of the DN's in "pushing" not also increase the noise component? One might think it would do so about as much as a corresponding increase in the pre-ADC analog gain (for a higher ISO setting) would.

(The data in your chart, of course, confirms your description. But I do not yet understand the underlying mechanism. Perhaps it involves something I do not recognize about the behavior of the pre-ADC amplifiers.)

Thanks.

Best regards,

Doug
 

Asher Kelman

OPF Owner/Editor-in-Chief
I'm not taking the metering sytem into account. I'm assuming we have already determined the perfect exposure level. What remains is to achieve the best image quality at that exposure level.
I understand.

Doug Kerr said:
So we need to find the lowest ISO that will still allow a short enough exposure time.

Bart and Doug, in alphabetical order, LOL! :)

With using a lower ISO than the one in which one has metered (and then exposing a compensatory amount in either talking the shot and/or in else recovery in processing the RAW file), one is now working with a setting of the camera in which the dynamic range is also generally larger, sometimes by even a whole stop. Saying that, am I merely restating the notion that noise is less under these lower ISO settings?

Asher
 

Doug Kerr

Well-known member
Hi, Asher,

With using a lower ISO than the one in which one has metered (and then exposing a compensatory amount in either talking the shot and/or in else recovery in processing the RAW file), one is now working with a setting of the camera in which the dynamic range is also generally larger, sometimes by even a whole stop. Saying that, am I merely restating the notion that noise is less under these lower ISO settings?
We must be carefully not to toss about "changes in exposure" and changes in development push/pull as though they were interchangeable.

And your example (using a lower ISO sensitivity than the one at which we metered, presumably keeping the exposure constant) might well result in "blowout" of the highlights such that a "pull" in development could not recover them.

I'm not sure your first statement leads to the second, but the second is generally true subject to the proper qualifications: a lower ISO setting will yield better noise performance (signal-to-noise ratio) on a particularize scene object assuming that the photometric exposure on the sensor is the same; that is, that the photographic exposure (combination of aperture and shutter speed) is varied to match the ISO setting (as using a metered exposure would do).

Let's review some carefully-defined comparisons.

Phase A

A1. First, we set a certain ISO sensitivity and shoot at the metered exposure. (Exposure=1)

A2. Then we set the ISO sensitivity to, say, 1/2 the value for the first shot, and then shoot again with a metered exposure (or, if we wish to think of it this way, shoot with twice the exposure used in the first shot), assuming that the aperture and shutter speed this calls for are still "acceptable". (Exposure=2)

With "standard" development in both cases, the two images have the same RGB values for each scene element.

We can expect a better noise result in A2 than in A1.

Phase B

B1. First, we set the same ISO sensitivity as in A1 and shoot at the metered exposure. (Exposure=1)

B2. Then we set twice that ISO sensitivity and shoot again at the metered exposure (or, if we wish to think of it this way, shoot with half the exposure used in the first shot). (Exposure=0.5)

With "standard" development in both cases, the two images have the same RGB values for each scene element.

We can expect a worse noise result in B2 than in B1.

Phase C

C1. First, we set the same ISO sensitivity as in A1 and shoot at the metered exposure. (Exposure=1). We use "standard development" for this shot.

C2. We keep that ISO sensitivity and shoot at half the metered exposure. (We could do that manually, or just set EC=-1.) (Exposure=0.5) We then push the development so as to get the same RGB values as in case C1.

We can expect a worse noise result in C2 than in C1.

Overall

In shot C2 the exposure is the same as in shot B2. We might expect the two to have essentially identical noise performance.

Bart's data suggests that the noise performance for shot C2 is slightly better than for shot B2 (if I properly understand the definitions of his various cases.)

I do not yet grasp the mechanism that would bring this about.

In any case, setting aside that conundrum, we need to recognize this as a classical case of what information theory tells us, which is that the more energy we deliver to the sensor (say, for a certain scene item), the better will be the signal-to-noise ratio for it.

Punch line warning

That says that, for a scene element of a certain luminance, the greater the photographic exposure, the better the noise performance in the delivered image for that scene element (in terms of signal-to-noise ratio, not absolute noise levels). (This is even true for exposures that will "blow out" brighter elements, although of course we would probably not want to use such.

Now the conundrum I mentioned above is that, as we play with both analog and digital gains in the camera, the signal-to-noise ratio for a given scene element (with a constant photometric exposure on the sensor) seems to change slightly. Again, I do not at this point understand the mechanism behind that.

Best regards,

Doug
 

Doug Kerr

Well-known member
Hi, Bart,

I suspect that the standard deviations you report are taken from the Photoshop histogram statistics panel, and have been reported "as found" (rather than, for example, having been converted to a "luminance referred" basis, essentially backing out the effect of the - you should pardon the expression - "gamma" curve of the sRGB color space).

The matter of that curve might explain the curiosity I had questioned in the values for the "black" patch, patch 6 vs. those for patch 5. There, one unit of R/G/B can represent a much smaller actual standard deviation in the DNs than one unit of R/G/B at the higher luminance of your "patch 5".

I don't know what RGB values your patches had, so I'll do an exercise with made up (but self-consistent) numbers.

For example, if your exposure was such that your "patch 1" ended up at RGB 240, your patch 5 at RGB 48, and your patch 6 at RGB 9, then one RGB unit on patch 5 represents almost 4 times the amount of noise as one RGB unit at patch 6 (because of the nonlinear transfer curve).

And one unit of RGB at patch 1 would represent almost 8 times the amount of noise as one RGB unit at patch 5.

Then, using your data for the ISO 400 case, the actual noise at patch 1 is about 3.75 times the noise at patch 5.

Now, since the luminance at patch 1 is about 29 times the luminance at patch 5, we might expect the shot noise at patch 1 to be about 5.4 times the shot noise at patch 5 (since shot noise rises essentially as the square root of photometric exposure).

But of course there is true read noise, which is roughly constant with photometric exposure. And thus the actual ratio of the total noise at patch 1 to the total noise at patch 5 would be a bit less than 5.4 times - maybe 3.75 times.

Just a story, of course.

Best regards,

Doug
 
Last edited:
C2. We keep that ISO sensitivity and shoot at half the metered exposure. (We could do that manually, or just set EC=-1.) (Exposure=0.5) We then push the development so as to get the same RGB values as in case C1.

We can expect a worse noise result in C2 than in C1.

Overall

In shot C2 the exposure is the same as in shot B2. We might expect the two to have essentially identical noise performance.

Bart's data suggests that the noise performance for shot C2 is slightly better than for shot B2 (if I properly understand the definitions of his various cases.)

I do not yet grasp the mechanism that would bring this about.

Hi Doug,

It is possibly also caused by the way the ISO amplification (gain) is implemented (at least in Canon cameras). Apparently (witnessed by how 'intermediate' ISOs can boost noise beyond the next higher) it is a 2-stage process. It involves a pre-amplification and a post-amplification step, pre/post ADC that is.

May I suggest some practical testing to get a grip on the more fundamental matters. One can start with a uniform exposure, e.g. by shooting the sky through a piece of opal glass or translucent white plexiglass, or a slightly out of focus uniformly lit surface. One can also shoot at different ISOs with aperture priority (to keep vignetting/light fall-off constant, as well as photometric exposure). One can also underexpose (-1 EV and -2 EV). This will produce a set of images at average metering exposure levels.

The Raw images can be inspected, and preliminary conclusions can be drawn, with "Rawnalyze" (Download / Documentation).

A better, more scientific, approach would be to shoot pairs of images and subtract them from each other (to remove systematic noise), add an offset (to avoid negative values), and divide the standard deviation of the result by Sqrt(2). It's best to use a crop from the center of the image without dead/hot/stuck pixels. One can use a program like IRIS to do all those steps based on the Raw image data (before demosaicing), but it'll require getting used to the IRIS functions.

The Raw conversion is only followed by a gamma adjustment (and stuff like white balancing and color management) of sorts to arrive at the final RGB values, but the solution is in the Raw data. Rawnalyze already allows to do some of that. It is also a nice utility to determine things like the distance in 1/6th stops between an average exposure and the actual clipping level (to check metering). Recommended for the toolchest. I also use it when I want to select the best exposure from a bracketed series seeking the highest non-clipped exposure to maximize DR.

Cheers,
Bart
 

Asher Kelman

OPF Owner/Editor-in-Chief
The Raw conversion is only followed by a gamma adjustment (and stuff like white balancing and color management) of sorts to arrive at the final RGB values, but the solution is in the Raw data. Rawnalyze already allows to do some of that. It is also a nice utility to determine things like the distance in 1/6th stops between an average exposure and the actual clipping level (to check metering). Recommended for the toolchest. I also use it when I want to select the best exposure from a bracketed series seeking the highest non-clipped exposure to maximize DR.
So Bart,

Is this part of the first program or Iris?

Asher
 

Doug Kerr

Well-known member
Hi, Bart,

It is possibly also caused by the way the ISO amplification (gain) is implemented (at least in Canon cameras). Apparently (witnessed by how 'intermediate' ISOs can boost noise beyond the next higher) it is a 2-stage process. It involves a pre-amplification and a post-amplification step, pre/post ADC that is.
Indeed, the key to the mystery may be in that area.

I'm glad you limited your work to the "full-stop" ISO values just to avert any further curiosities!

Of course the "discrepancy" is really very tiny (typically about 1/4 stop, as in the case of your patch 1 ISO 400 + push 1 vs. ISO 800 data). I don't even get that degree of consistency in exposure result in moving from one ISO setting to another with metered exposure.

May I suggest some practical testing to get a grip on the more fundamental matters. One can start with a uniform exposure, e.g. by shooting the sky through a piece of opal glass or translucent white plexiglass, or a slightly out of focus uniformly lit surface.

I use a WB diffuser for that (not a ColorRight!).

One can also shoot at different ISOs with aperture priority (to keep vignetting/light fall-off constant, as well as photometric exposure).

Well, photometric exposure won't be constant if you change the ISO setting and use a metered mode, but I know what you mean.

One can also underexpose (-1 EV and -2 EV). This will produce a set of images at average metering exposure levels.

Well, at average metering levels -1 and -2 stops.

The Raw images can be inspected, and preliminary conclusions can be drawn, with Rawnalyze.
Yes, I have it, but have only used it a little. It is a nice tool.

I probably won't get to the more sophisticated testing, but who knows!

There's more fun available than I have time (and energy) for!

Thanks.

Best regards,

Doug
 
Top