Quantizing (and) noise in digital photography

We often hear of "quantizing noise" as one of the noise components in an overall digital imaging chain.

That term is borrowed (inappropriately and unprofitably) from another field, the digital representation of audio waveforms.

Here, for reference, is some background on that.

*********

When we represent an audio waveform in digital form, quantizing error gives errors in the individual waveform samples as reconstructed, and thus to "distortion" of the reconstructed waveform (quantizing distortion).

When the digital representation of audio waveforms was first introduced into telecommunication practice, there was need for a metric that would describe how "severe" quantizing distortion was (in terms of its perceptual impact to the listener) - one with a clear theoretical definition, and hopefully one that could be determined in an actual case with practical test instruments. (At the time, there was no computer-based or even digital measurement - most instruments involved only filters, amplifiers, maybe modulators, rectifiers, and analog meters!)

This need was met by the following train of thought:

• Expand our definition of "noise" to include anything "in" the delivered waveform that was not "in" the original waveform.

• Then "noise" is manifest by the difference between the delivered waveform and the original waveform - even if that is a result of quantizing distortion.

• If we subtract the original waveform from the delivered waveform (or remove it in some other way), we have a waveform that is the "distortion", isolated - which we treat as a special form of "noise".

• We then analyze this "noise" waveform using the same technique used to assess noise proper, which includes recognition of the differing sensitivity of the human ear to waveform components of different frequency.

We do this using "noise measuring instruments" that we already have, with the addition of a simple accessory to "remove the original waveform". (It was actually just a notch filter; we always made the test with an "original waveform" that was a sine wave at a certain frequency, which this filter just removed from the reconstructed waveform before it was fed to the "noise meter")

We found that the result of this (our "metric" for quantizing distortion) tracked pretty well with the human perception of the "badness" of quantizing distortion. And even if was not ideal in that regard, we could measure it with instruments we had widely deployed in the field. So it was adopted. And the term "quantizing noise" came into use in recognition of the back story - perhaps life imitating art.

*********

Now lets go to digital photography. What does quantizing error do to the image constructed from data from our sensor?

If we have areas of slowly varying color (color of course including luminance) we see the impact as banding. We easily recognize that this is not "natural" ("the sky never looks like that"). It's not handsome, and its an "impairment", but certainly from the viewer's standpoint, it is hard to think of it as "noise".

If we have a scene area with a lot of small detail, not forming an expected geometric pattern (think a gravel road), or even a shag rug (remember those?), the quantizing error in fact changes the image from what it "should be", but not in a way that we probably recognize by examining the image in any usual way (if at all).

So the characterization of the impact of quantizing error as noise isn't apt from the viewer's perspective.

But what about from a technical perspective? When we speak of such things as "signal to noise ratio", we mean by noise a randomly varying departure of the reported pixel color from what the sensor would "report" if there were no noise. The way we measure it is to apply a consistent photometric exposure (perhaps to a grid of sensels) and look at the variation in the digtial values (DN) the ADC reports across those pixels. (We typically use the measure "standard deviation" as the metric of that noise.) Or we might do it by taling several shots with the camera locked in position and looking at the variation in the DN reported by a single sensel.

How does quantization get in on this?

Lets think of the latter test model, with a constant photometric exposure on a certain sensel. Then in general, quantization causes no "randomly varying" discrepancy. It causes an error, but it is fixed (because the actual sensel votage is fixed - we're ignoring for now sources of "real" noise). Hardly noise-like.

Now, suppose we do this with a luminance that we vary slowly between tests. At any given stage of this, the effect of the error is fixed (and does not constitute a "random variation", nor any variation at all), but as the test luminance crosses certain boundaries, the error jumps (never being greater than half a DN number step). Perhaps we should consider the RMS error over a range of such slow change in luminance. If we feel a real need to talk about "quantizing noise", then we could consider that determination to be a measure of it. But it's not clear what that tells us. And of course the magntiude of the RMS assessment will be in the area of a fraction of a DN unit - very small.

But there is an effect of quantizing on the impact of "real" random noise. If the random noise in the usual sense (shot noise, read noise, etc), for a certain range of pixel luminance, is small compared to the size of the quantizing "step" (one unit of the data number, or DN), then for certain luminance values, the noise as observed at the digital output of the ADC will be zero; for other luminance values, the noise as observed at the digital output of the ADC will be greater than the actual random "voltage". Overall, for a random range of photometric exposures, the noise as seen in the DN output of the ADC may be on average greater than the actual noise on an analog basis going into the ADC, or it may be smaller.

But it's hard to think of this as noise caused by quantizing. (If quantizing causes the randomness of the digital output of the ADC to be less than the actual noise voltage, would that be a negative quantizing noise?)

Rather, it is a (slight) corruption of the transfer of the noise from analog form to digital form through the ADC, just as quantizing corrupts the transfer of the signal voltage from analog form to digital form through the ADC.

And it is almost negligible as soon as the RMS value of the actual noise gets as large as one quantizing step (one DN unit). At that point, the average noise (as seen out of the ADC) is about 1.125 times the actual noise voltage into the ADC.

If the RMS noise voltage into the ADC is 5 times the quantizing step (one DN unit), then the noise as seen out of the ADC (in digital form) is a little less than the noise voltage into the ADC.

*********

Overall, the bottom line is this: the concept of "quantizing noise" is a useful conceit in the matter of the digital representation of audio waveforms, since it allows us to make an assessment of the impact of quantizing distortion using concepts, and actual test instruments, developed for the measurement of actual noise.

Trying to transport that concept to the area of digital imaging does not seem to lead to anything profitable.

Doug,

I was an excellent student, following your logic until the noise changes in going up different DN units and also "
And it is almost negligible as soon as the RMS value of the actual noise gets as large as one quantizing step (one DN unit). At that point, the average noise (as seen out of the ADC) is about 1.125 times the actual noise voltage into the ADC.

"If the RMS noise voltage into the ADC is 5 times the quantizing step (one DN unit), then the noise as seen out of the ADC (in digital form) is a little less than the noise voltage into the ADC."

I was relieved that you didn't find the line of thinking of quantizing digital noise as useful, so the fact that I lost my way near the end means I did not, after all, lose out!

Asher

Hi, Asher,

Asher Kelman said:

I was an excellent student, following your logic until . . .

Click to expand...

I appreciate your diligence. These ""Physics Review Letters" do not have the didactic refinement of my actual articles (which often take a whole day or so to write). (They generally don't even get Cherokee copy editing!)

I actually wrote this while standing by to help Carla as she prepared for tomorrow morning's colonoscopy. (I had to write a simulation in the middle of it to find out what quantizing does do to the "real" noise in the digital imaging chain!)

I was relieved that you didn't find the line of thinking of quantizing digital noise as useful, so the fact that I lost my way near the end means I did not, after all, lose out.

Click to expand...

Delighted to hear that, and nicely said.

It's a phrase that is probably best not mentioned within the precincts of digital photography. It serves no useful purpose, and can only stir those with some insight into spasms of trying to connect to it, like a listener struggling with a pronouncement that "Although the Moon is smaller than the Earth, it is further away".

One of my next tricks is to figure out what higher ADC bit depth really does and how. (Yes, yes - dynamic range, exposure latitude, enhanced karma, etc.) As you know, I am very reticent about accepting (or, God forbid, repeating) any description of what something does unless I either hear or can construct at least one credible explanation of how!

"So", said my step-grandson's father-in-law at a family New Years' Eve party, "what do you do now that you are retired?"

Thanks again for suffering through this latest diatribe. I hope to finish the real article within the week.

Best regards,

Doug

Hi, Asher,

Let me restate somewhat more carefully the part of my discussion of quantization (and) noise that may have been problematical to you.

Fist imagine that there is no noise on the "signal" voltage from a sensel.

For a small range of sensel voltage, the analog-to-digital converter (ADC) gives us the same digital output (the same digital number, DN). For only one specific sensel voltage is that DN a correct representation; for the others it is incorrect. The discrepancy is quantizing error.

The error of course does not lie in the digital output varying when it shouldn't (like errors in reading a laboratory thermometer). Rather it lies in the digital output not varying when it should (as if the technician just failed to record tenths of a degree in the thermometer readings).

A common manifestation of this in an image is when we have "banding" of a region of slowly-varying luminance.

Now consider the real situation of a random component - "noise" - in the sensel voltage. And think of a constant luminance so that the ideal sensel voltage - without the noise - would be constant. And imagine that the noise component is "small". (If the noise voltage is truly Gaussian, its potential "swing" is always infinite; so we can't describe its "size" in terms of voltage swing. Rather we describe it by its statistical property, its standard deviation, from which we can predict for what percentage of the time does the instantaneous noise voltage exceed a certain value - positive or negative).

Because of the quantizing process, where for a range of sensel voltages the digital output is constant, the digital output does not accurately reflect the vagaries of the sensel voltage resulting from the noise component. And exactly how that happens depends on the "base" voltage (the signal voltage itself, not including the noise).

A little parable about a temperature recording system may clarify that point. Suppose we in fact have, over some period, a nominally-constant temperature, but that the output of the temperature detector has a random variation (and let's assume it is only up to a discrepancy of ±0.03° F). We capture the detector output only to the nearest 0.1° F in our "recorder" (that is, we quantize that value).

If the nominal temperature is 76.50°, but with the "noise" the output from the detector varies from 76.47° to 76.53°, the recorder will always record 76.5°, and the effect of the noise is not seen. The "swing" in the readings is 0.

Later, if the nominal temperature is 76.55°, but with the "noise" the output from the detector varies from 76.52° to 76.68°, the recorder will about half readings as 76.5° and the other half at 76.6°. The "swing" in the readings is 0.1°, which we can think of as ±0.05°.

Now, from an examination of the record, can we tell how great the random variation in the temperature detector output - it's "noise" - is?

No, we can't.

But suppose, by way of testing our instrument, we subject it to a slowly varying actual temperature, and observe "swing" of the recorded readings as the "nominal" value changes. We will see, in the readings, an "average" swing of 0.25° (which we can think of as an average variation of ±0.125°).

So it is with the "random variation" observed in the digital output of our camera ADC as a manifestation of the random variation ("noise") in sensel voltage.

The story is more complex than in the temperature recorder parable, because the noise voltage does not uniformly occupy a range of a certain span, but rather, as I mentioned above, it has differing probabilities of having any value we can imagine over a theoretically unbounded range (there is of course a practical limit to the value the instantaneous sensel voltage can take on).

But still, as in the case of the temperature recorder parable, the swing in digital values due to that noise differs depending on exactly what the "background" voltage (the one actually representing luminance) happens to be.

So we may need to think in terms of the "average" swing in digital output, resulting from the noise voltage, as we vary the "background" voltage slowly over a range.

Now, for very small noise levels (compared to the "quantizing step"), the average magnitude of the random variation at the digital output (stated as always in terms of the standard deviation of the instantaneous digital output) will be slightly larger than the random variation of the noise voltage proper (we have an average "enlargement" of the noise at the digital output because of quantizing).

It may be this enlargement that some who speak of "quantizing noise" are thinking of (but I doubt it).

Now, as the magnitude of the noise increases (into the range we are usually dealing with), it turns out that the variation in the digital output is slightly smaller than the noise itself (we have an average "diminution" of the noise at the digital output because of quantizing).

Those who hope to find evidence of "quantizing" noise will have to think that in this situation (the one in which we are most often interested) that "quantizing noise" must be negative.

Best regards,

Doug

Further to the above reports.

Let me review the context:

• We are intersted in comparing the noise in the digital output of the ADC with the noise in the sensel voltage into the ADC

• We recognize that this relationship depends on where the "base" voltage (representing the ideal sensel "signal" with no noise) falls in the quantizing band.

• So we determine the relationship for ten different positions of the "base" voltage across the width of the quantizing band, and report the average of those ten values.

The results are (these are updated in the face of an error in my original model):

• For a noise in the sensel voltage whose RMS value is the same as the width of the quantizing band (one DN unit), the RMS value of the noise in the digital output is about 1.08 times the noise in the voltage.

• For an noise in the voltage whose RMS value is 1.7 times the width of the quantizing band, the RMS value of the noise in the digital output is about equal to the noise on the voltage.

• For an noise in the voltage whose RMS value is 5 times the width of the quantizing band, the RMS value of the noise in the digital output is about 0.87 times the noise in the voltage.


So the effect of quantizing on the representation of the sensel noise in the digital output, for sensel noise greater than 1.7 times the quantizing band, is to "diminish" the noise.

Best regards,

Doug