EOS 1Ds Mark III analysis, Read Noise

[Bart_van_der_Wolf]

Hi folks,

This is the first in a series of technical analyses I performed as based on my 1Ds3.

One of the useful (for analysis purposes) key figures in the analysis of a camera/sensor array performance is; Read Noise. Read noise is the lowest amount of noise that a camera will produce, it can't get better than this (unless one resorts to (postprocessing) tricks). It is the (electronic) noise that is generated while reading data from the sensor, and as recorded in the Raw data as a noise floor. It is an important parameter in the determination of Dynamic Range.

The data was collected from so-called Black frames. Black frames are Raw files that received no exposure, by recording 'images' without lens but with bodycap in place, viewfinder covered, and the shortest possible 'exposure' time (1/8000th second) selected.

The frames were shot in pairs, at 4 second intervals (to avoid potential noise from the writing action to the CF card), thus allowing to reduce the potential effects of hot pixels and pattern noise by calculating the standard deviation of the difference between the files, divided by Sqrt(2). All available ISO settings were tested in the same manner, and all files were shot at approx. 20 degrees Celcius ambient temperature. The actual sensor elements used were a central crop of the same 400x400 sensels as would be later used for other (actual exposure) analyses. That means that for each of the G/R/G/B Bayer CFA filtered sensels, 40,000 samples were available (a quantity which should be enough for statistically relevant sample populations). To allow comparisons with data collected by others, the frequently used IRIS software (version 5.51) was used to read the Raw (non-color balanced, non-demosaiced) 14-bit data from the files.

The 14-bit quantized data was scaled to 16-bit to facilitate comparison with other bit depths, divide by 4 to compare with other 14-bit results, or by 16 to compare with 12-bit quantization results. Here are the results:


Several things can be learned from looking at the data. With the exception of ISO 500, only the 'regular' ISOs (100, 200, 400, 800, 1600) are useful for lowest read noise performance. The 'intermediate' ISOs have more read-noise than their next higher 'regular' ISO. There is also no read noise benefit to using the special 'L' and 'H' settings, other than allowing for more extreme exposure levels. Unlike with the 1Ds2, ISO 'L' doesn't improve the read noise performance and the associated dynamic range. ISO 'H' is clearly a simple multiplication result of an (underexposed) ISO 1600 sensitivity.

Given the fact that the maximum clipping level of all sensels appears to be 15,280 data numbers (plus or minus 1 DN or ADU) in 14-bit quantization space, one can also conclude that the theoretically maximum Dynamic range, according to the common engineering definition, is Log(15280 / 5.7625) / Log(2) =~ 11.3 stops. While that is not an improvement when compared to the 1Ds2, it is almost the same. A higher DR would have been welcomed, but Canon made a different quality trade-off in favor of resolution at the expense of DR improvement.

Bart

[Johnny_Johnson]

Hi Bart,

What am I missing? It looks like that 1000 & 1250 have the same advantage that 500 does.

Later,
Johnny

[Bart_van_der_Wolf]

Originally Posted by Johnny_Johnson

Hi Bart,

What am I missing? It looks like that 1000 & 1250 have the same advantage that 500 does.

Later,
Johnny

You're right, thanks for pointing it out. I overlooked it because I am personally mostly interested in lower ISOs for studio work or architecture.

Bart

[Doug Kerr]

Hi, Bart,

Thanks for the very interesting report. I need to contemplate your results for a while to be sure I grasp their significance.

Originally Posted by Bart_van_der_Wolf

Given the fact that the maximum clipping level of all sensels appears to be 15,280 data numbers (plus or minus 1 DN or ADU) in 14-bit quantization space, one can also conclude that the theoretically maximum Dynamic range, according to the common engineering definition . . .

Would you explain a little more what your notion of "the common engineering definition" of the dynamic range of a digital camera is. (I'll be able to reverse engineer your definition from your numbers, but I'm too tired to do that right now!)

Is it essentially the same as the definition of dynamic range defined by the ISO standard?

For reference, that definition is essentially:

The ratio of (a) the luminance* implied by the maximum digital output number to (b) the RMS noise (on the basis of the variation in luminance that would cause the observed random variation in digital output) experienced at a base luminance of 0.01 the luminance in (a).

*Precisely, photometric exposure, but that essentially corresponds to relative luminance since f/number, shutter speed, and the like would be constant between the two parts of the comparison.


Again, thanks for this nice report and the clear presentation of the data.

[Joseph A. Kurkjian]

Bart, thanks very much for posting your test results.

Regards,

Joe Kurkjian

[Asher Kelman]

Originally Posted by Bart_van_der_Wolf

Given the fact that the maximum clipping level of all sensels appears to be 15,280 data numbers (plus or minus 1 DN or ADU) in 14-bit quantization space, one can also conclude that the theoretically maximum Dynamic range, according to the common engineering definition, is Log(15280 / 5.7625) / Log(2) =~ 11.3 stops. While that is not an improvement when compared to the 1Ds2, it is almost the same. A higher DR would have been welcomed, but Canon made a different quality trade-off in favor of resolution at the expense of DR improvement.

Bart, could you show where the 15280 and 5.7625 come from?

It would be nice to know to what extent noise increased by extending exposure to 1/1,000 1/250, 1/50 sec. Might it not be that on a hot day, for example, just going to 1/30 sec from 1/8,000 sec might be worse than going from ISO 200 to 500. After all there would be more time for the noise to accumulate.

Thanks

Asher

[Ralph Eisenberg]

Thanks very much for having taken the time and gone to the trouble to do this testing, as well as allowing us to benefit from your results.

[Bart_van_der_Wolf]

Originally Posted by Doug_Kerr

Would you explain a little more what your notion of "the common engineering definition" of the dynamic range of a digital camera is.

It simply is the signal saturation level, 15280 in this case, divided by the read noise, approx. 5.77 for both ISO 'L' and ISO 100. As such it doesn't take 'acceptable' levels of noise into consideration, but rather the maximum attainable under controlled situations.

A slightly more detailed way of defining it is given by: http://www.ccd.com/ccd111.html.

Again, thanks for this nice report and the clear presentation of the data.

You're welcome. I like to share this type of info because it allows a better founded comparison between different camera solutions for different applications, and it takes the guesswork factor out of discussions. It helps people to make better decisions, and allows to tweak solutions for dealing with equipment limitations (e.g. HDR imaging if expose-to-the-right is not enough help).

In the particular case of the 1Ds3, which is intended as a studio camera (where lighting conditions can be controlled) just like the 1Ds2 was/is, it is clear that optimal exposure (ETTR) is very important for the best results. For critical outdoor work, one therefore might want to consider more exposure bracketing if the best image quality is needed. The 1Ds3 allows to bracket up to 7 shots in a little over 1 second (depending on shutter time of course).

Bart

[Torsten Kötting]

Originally Posted by Bart_van_der_Wolf

A higher DR would have been welcomed, but Canon made a different quality trade-off in favor of resolution at the expense of DR improvement.

do you think Canon had a choice? my impression is das DR hasnt improved for a while now because they dont know how to do so ...

[Bart_van_der_Wolf]

Originally Posted by Asher Kelman

Bart, could you show where the 15280 and 5.7625 come from?

The 15280 (+/- 1) is the highest data number that can be found in the raw 14-bit image data. It can be found in overexposed and clipped images. It follows from another test that I'll share, where a series of increasing exposures of a uniform surface is used for the determination of the 'gain' and 'unity gain' parameters (the latter is important if one wants to control noise in highlight areas).

The 5.7625 is just the simple average of the read noise at ISO 'L' in its native 14-bit data. You can also average the 16-bit equivalents shown in the chart table (23.05) and divide by 4 to scale to 14-bit values.

It would be nice to know to what extent noise increased by extending exposure to 1/1,000 1/250, 1/50 sec.

In earlier tests I (and others) have done, it doesn't make a significant difference. I have yet to verify it for the 1Ds3, but I expect it to be the same as for other Canon cameras that I've tested. Let's say that in handheld shooting conditions, e.g. up to 1/30th sec., it would be hard to measure a significant increase in noise, let alone see it.

Might it not be that on a hot day, for example, just going to 1/30 sec from 1/8,000 sec might be worse than going from ISO 200 to 500. After all there would be more time for the noise to accumulate.

The major contributor to temporal noise, is called the 'dark count'. It accumulates over time, and it increases exponentially with temperature. In my experience it is swamped by other noise sources in regular handheld exposure scenarios. It becomes an increasingly more important factor when we approach 1 second exposure times, or longer. To reduce it's influence as much as possible in the read noise tests, it is therefore customary to use the shortest possible integration time when testing.

Bart

[Bart_van_der_Wolf]

Originally Posted by Torsten Kötting

do you think Canon had a choice? my impression is das DR hasnt improved for a while now because they dont know how to do so ...

There are 2 parameters that define dynamic range. One is the saturation level, and the other is read noise.

Read noise is pretty low as it is. We're talking about something like 23 electrons amidst noisy (camera) electronics and cosmic rays. I don't expect (but one can only hope for) revolutionary improvements there, evolutionary at best.

The saturation level is governed by ADC bit depth and sensel size. Just increasing the ADC bit depth alone doesn't necessarily improve the situation by much. Increasing the sensel size (storage capacity, 'well depth') does make a difference in dynamic range, but then one has to sacrifice resolution if one is restricted to a given physical sensor array dimension. So Canon clearly chose for resolution in the 1Ds3, and offers more dynamic range in the 1D3 with its larger sensels but otherwise same chip technology.

Bart

[Doug Kerr]

Hi, Bart,

Originally Posted by Bart_van_der_Wolf

It simply is the signal saturation level, 15280 in this case, divided by the read noise, approx. 5.77 for both ISO 'L' and ISO 100.

Sure, a very useful ratio. It is essentially the signal-to-noise ratio at the saturation luminance.

By the way (and if I read your stuff throughly enough I would probably know!), is the metric of read noise RMS-based?

As such it doesn't take 'acceptable' levels of noise into consideration, but rather the maximum attainable under controlled situations.

"Maximum attainable" what? The implication is that it is the maximum attainable "dynamic range". But again, that depends on what concept of dynamic range we adopt.

If the ratio is 2650, does that imply that we can get a "useful" image that has a ratio between the base luminance of the highlight areas to the base luminance of the shadow areas in which we are interested of 2650? And if so, why does that follow?

There is little question that the better the SNR, the greater ("all other things being equal") will be the "dynamic range" of the system (under several, but not all, reasonable definitions). But I think we have to be careful in suggesting that the saturation luminance SNR is a tells us the maximum of some metric for dynamic range.

By the way, I haven;'t yet read the reference you cite (which I plan to do shortly).

None of the above is to any way demean the value of your report. I'm just trying to make sure that we don't draw equivalences that may not follow some particular definition.

Best regards,

Doug

[Emil Martinec]

Thanks for the effort on this. Will you also be measuring the "gain"?

[Bart_van_der_Wolf]

Originally Posted by Emil Martinec

Thanks for the effort on this. Will you also be measuring the "gain"?

Yes, I've sofar collected ISO 'L', 100 and 200 uniform area exposure series from average exposure to clipping exposure. These are the levels I was most interested in for the short term, for personal imaging requirements. Higher ISO's will follow later as lighting conditions and time permit, to satisfy curiosity.

I'm a bit puzzled by the small read noise difference between ISO 'L' and 100, but it seems to be confirmed by the preliminary gain figures (around 4 electrons/ADU for 'L' and 100 in 14 bit terms).

I'll probably post details tomorrow.

Bart

[Steve Saunders]

Thanks Bart.

[Emil Martinec]

Just for grins, I analyzed the 1Ds3 read noise data. I assumed a two-stage amplification, with the first stage responsible for the main ISO's of 100,200,400,800, and 1600; and a second stage amplification of G2 = 1, 1.26, or 1.59 depending on whether one wants the 100-200-400 etc ISO series, the 125-250-500 etc ISO series, or the 160-320-640 etc series (1.26 and 1.59 are the slightly more accurate 1/3 stop amplifications). Then I assumed noises adding in quadrature as

N^2 = (G2 G1 N0)^2 + (G2 N1)^2 + N2^2 (*)

where roughly N0 would be the read noise of the sensel, which then gets amplified by both gains; N1 would be the noise of the on-chip amplifier for the main ISO's, which then gets further amplified by the second stage amplifier whose gain is G2 and whose noise is N2 (or perhaps more precisely N2 is the combined noise of the second stage amplifier and everything downstream of it, eg the ADC). In another thread here is reported a gain of 16.12 electrons/14-bit ADU at ISO 100, so we can take G1 = (1/16.12) x (ISO/100) (so long as we remember that N0 is measured in electrons, while N1 and N2 are in ADU).

Doing best fit parabolas for the three separate ISO series from the noise data and then backing out the various parameters (here I did a bit quick and dirty analysis by analysing pairs of ISO series and taking the average; probably one could do a bit better), I get

N0^2 = 410 electron^2 (about 1.6 ADU^2 at ISO 100)
N1^2 = 23 ADU^2
N2^2 = 12 ADU^2

and plotting the three series of N^2 vs ISO, I get




Here the horizontal axis is ISO/100, and the vertical axis is (noise)^2 in ADU^2; the points are Bart's data, the curves are equation (*) evaluated on the fit parameters. ISO 1600 data seems a little off from the curve, not sure why at the moment; but the rest fits remarkably well, IMO.

It would be interesting to work this out for other bodies.

[John Sheehy]

Originally Posted by Bart_van_der_Wolf

A higher DR would have been welcomed, but Canon made a different quality trade-off in favor of resolution at the expense of DR improvement.

Bart

I suspect that maybe your camera has the gain a little bit on the high side. The other cameras I've measured seemed to be slightly better (not by much, though). This is something I've noticed with these cameras. The read noises vary from camera to camera by a small amount, but the ratio between the read noises at various ISOs are much closer. My first 30D had what I considered very high read noise, considering the fact that it had the same sensor as the 20D, and when I compared their actual sensitivities (RAW levels of the same absolute exposures) the 30D was clearly over-amplifying by 1/3 stop. The camera had other issues (lost the time and date; resumed with random settings after a shot), so I exchanged it, and the new 30D had read noise and sensitivity close to what the 20D had.

As far as DR is concerned, With the exception of one camera, the K10D (4500:1), the engineer's DR of individual pixels really doesn't get any better than 2500:1 to 3000:1 for current DSLRs. The resolution makes a big difference for practical DR, though. I'd rather have 100M pixels with 2000:1 than 4M pixels with 2500:1; the former would have far more DR in practice; not just better resolution. If DR of the image was strictly determined by the DR of the pixel, then any tiny crop from a sensor would have the same DR as the full sensor.

[Emil Martinec]

A stringent cross-check of the model of two-stage amplification is that curves of the two gains for fixed values of the other gain properly intersect the data.
The noise squared vs gain for two stages of amplification is

N^2 = (G2 G1 N0)^2 + (G2 N1)^2 + N2^2

where G2=1, 1.26, 1.59 is the gain that generates the intermediate ISO's, and G1=100, 200, 400, 800, 1600 are the main ISO gains.

There are only three independent parameters -- the noises N0, N1, and N2 of the various stages of sensor readout and amplification. An approximate best fit for these curves is
N^2 = .0487 x^2 + 2^(2 n/3) 2.55 + .9

and

N^2 = (.0487 + 2.55/2^(2 n) ) x^2 + .9

for n=0,1,2,..., and x=ISO/100. Thus N0^2=.0487, N1^2=2.55, and N2^2=.9 (all in ADU squared). Plotting these curves together with the data yields



for which all data fits the curves rather well, apart from the ISO 1600 data (note that the intersection of the two curves predicts where the read noise should lie, above the measured point, but correctly at ISO 1600). That both sets of curves intersect the data using the same parameters N0,N1, and N2, is a rather nontrivial check of the model against the read noise data.

It would be interesting to understand what is responsible for this anomaly at ISO 1600.

A couple of conclusions may immediately be drawn:

1) The dominant source of noise at low ISO is the noise N1 of the main ISO amplification. If Canon is to improve dynamic range, it needs this amplifier to be quieter.

2) There is a noise advantage to using intermediate ISO's, vs underexposing and pushing in raw conversion (the latter has an advantage in highlight headroom though). Pushing an underexposure amplifies the noise N2 as well as N1 and N0, whereas using the intermediate ISO's amplify N1 and N0 but not N2.

[Emil Martinec]

Sorry, that last post was for data from the 5D. Here is the revised plot for Bart's 1Ds3 data:



Again the two sets of curves represent holding one of the two gains fixed and varying the other. The parameters in the noise model

N^2 = (G2 G1 N0)^2 + (G2 N1)^2 + N2^2

for the 1Ds3 are N0^2=1.64, N1^2=24.5, and N2^2=7, all in 14-bit ADU squared. The pattern for the 5D and 1Ds3 is basically the same -- the noise model is a good fit to all but ISO 1600.

Referred to 12-bit units, these are N0^2=.103, N1^2=1.53, and N2^2=.44; compared to the values for the 5D of N0^2=.0487, N1^2=2.55, and N2^2=.9. A tentative conclusion is that the amplifier noises have been roughly halved in going from the 5D to the 1Ds3 (actually, N2 includes not just the noise of the second stage of amplification, but also any noise in the ADC, etc); the noise N0 from the sensor readout is roughly double for the 1Ds3, perhaps this is inversely correlated to the size of the photosite -- the 5D pixels are 1.64 times the area of the 1Ds3 pixels.

[Hans Polak]

Hi Emil,

This thread is already one year old, but hopefully you are still there.
I have read several of your publications, and they were amongst the best to shed light on the issues of noise.

Reading this thread I have two questions:

1)
You describe the total (digital) read noise as being:
N^2 = (G2 G1 N0)^2 + (G2 N1)^2 + N2^2, where N0 is the Sensel Noise, N1 the onboard amplifier noise and N2 all the noise behind the second amplifier.

The onboard amplifier however has a certain input noise, that remains almost the same independent of its amplification.
When defining N1 as output noise of G1 as you did, makes this noise component N1 dependent on the ISO setting.

When having N1 presented as input noise of the onboard amplifier, it results in G1 G2 N1, where N1 is independent of ISO setting.
The input noise of the external amplifier G2 is probably neglectable to the rest of the noise behind G2 (if available at all, see below point 2) and can probably be disregarded, the most obvious one here being the ADC noise.

So when redefining N1 as the input noise of the onboard amplifier, one could rewrite the formula as:
N^2 = (G2 G1 N0)^2 + (G2 G1 N1)^2 + N2^2 .
or
N^2 = (N0^2 + N1^2)(G2 G1)^2 + N2^2
Here N0, N1 and N2 all independent of ISO.
In this presentation, it is more obvious that it is probably impossible to asses the individual values of N0 and N1.
If so, it hard to say if the onboard read amplifier is the more dominant noise source of or the read out circuitry of the sensel.
Anyhow, you have tried to fit a parabolum through the points measused by Bart, and found that the 1600 ISO point is a bit off.
But If you take the formula Sqrt(y)= (ax)^2 + b^2, then for higher ISO values where (ax)^2 is much larger as b^2, then the formula reduces to y = ax as an asymtote, and ISO 1600 is no longer an exception.
Solving a and b in Barts case, gives me approx ax=sqrt((N0^2 + N1^2)(G1 G2))=77*ISO/1600 and b=N2=22.5 (all based on 16 bit words)
All points measured by Bart fit perfectly on the curve, icluding ISO 1600.
As from ISO 470, the Sensol plus Read amplifier are the dominant noise producers.
Do you agree on this ?

2)
A second question is, that looking at Barts noise figures, it seems as if there is no second amplifier at all, but that the intermediate ISO values are made AFTER A/D conversion.
Else I see no reason why the Noise is going up from100 to 125 and 160 with exactly the mathematical factor 1.25 and 1.6, and then going down again at 200 ISO.
The same goes on between ISO 200 / 400 and ISO 400 / 800, where 320 is higher as 400, and 640 has the same noise as ISO 800.
This cannot be explained with a noisy amplifier G2 as far as I see it.

Hans

[Emil Martinec]

Originally Posted by Hans Polak

Hi Emil,

This thread is already one year old, but hopefully you are still there.

Not often; random chance brought me back.

Reading this thread I have two questions:

1)
You describe the total (digital) read noise as being:
N^2 = (G2 G1 N0)^2 + (G2 N1)^2 + N2^2, where N0 is the Sensel Noise, N1 the onboard amplifier noise and N2 all the noise behind the second amplifier.

The onboard amplifier however has a certain input noise, that remains almost the same independent of its amplification.
When defining N1 as output noise of G1 as you did, makes this noise component N1 dependent on the ISO setting.

When having N1 presented as input noise of the onboard amplifier, it results in G1 G2 N1, where N1 is independent of ISO setting.
The input noise of the external amplifier G2 is probably neglectable to the rest of the noise behind G2 (if available at all, see below point 2) and can probably be disregarded, the most obvious one here being the ADC noise.

I'm not knowledgeable enough to say what specifically sources the noise; for the purpose of modelling, it doesn't really matter; all that goes into the fit is that there are sources of noise prior to amplification, and sources of noise after amplification.

So when redefining N1 as the input noise of the onboard amplifier, one could rewrite the formula as:
N^2 = (G2 G1 N0)^2 + (G2 G1 N1)^2 + N2^2 .
or
N^2 = (N0^2 + N1^2)(G2 G1)^2 + N2^2
Here N0, N1 and N2 all independent of ISO.
In this presentation, it is more obvious that it is probably impossible to asses the individual values of N0 and N1.

Yes, there is no distinction between N0 and N1 in this formula -- they are both noise sources present before any signal amplification takes place, so they are indistinguishable from in the output.

If so, it hard to say if the onboard read amplifier is the more dominant noise source of or the read out circuitry of the sensel.
Anyhow, you have tried to fit a parabolum through the points measused by Bart, and found that the 1600 ISO point is a bit off.
But If you take the formula Sqrt(y)= (ax)^2 + b^2,

I assume you mean y^2=(ax)^2+b^2

then for higher ISO values where (ax)^2 is much larger as b^2, then the formula reduces to y = ax as an asymtote, and ISO 1600 is no longer an exception.
Solving a and b in Barts case, gives me approx ax=sqrt((N0^2 + N1^2)(G1 G2))=77*ISO/1600 and b=N2=22.5 (all based on 16 bit words)
All points measured by Bart fit perfectly on the curve, icluding ISO 1600.
As from ISO 470, the Sensol plus Read amplifier are the dominant noise producers.
Do you agree on this ?

I don't see how you can fit all the points to a single parabola; there are two interleaved parabolas, as my graph indicates. That is why the model I chose has two gains. Are you only fitting the ISO 100-200-400-800-1600 data? If so, I agree that there is a parabola that fits those better than the two-gain model, however the parameters that make that fit better make the fit to the intermediate ISO data worse. Each of the intermediate ISO sequences also have better parabolic fits; however it is much harder to fit ALL the data with only three parameters.

2)
A second question is, that looking at Barts noise figures, it seems as if there is no second amplifier at all, but that the intermediate ISO values are made AFTER A/D conversion.
Else I see no reason why the Noise is going up from100 to 125 and 160 with exactly the mathematical factor 1.25 and 1.6, and then going down again at 200 ISO.
The same goes on between ISO 200 / 400 and ISO 400 / 800, where 320 is higher as 400, and 640 has the same noise as ISO 800.
This cannot be explained with a noisy amplifier G2 as far as I see it.

Hans

If the intermediate ISO's were done as digital multiplication after A/D, that would show up as gaps in the histogram of the RAW data. For instance, a 1/3 stop boost is multiplication by roughly 5/4, and every fifth RAW level would be unpopulated. Such gaps are not seen in the 1 series (though they are present in the xxD series, indicating that their intermediate ISO's are done in software).