EOS 1Ds Mark III analysis, Read Noise

[Bart_van_der_Wolf]

Hi folks,

This is the first in a series of technical analyses I performed as based on my 1Ds3.

One of the useful (for analysis purposes) key figures in the analysis of a camera/sensor array performance is; Read Noise. Read noise is the lowest amount of noise that a camera will produce, it can't get better than this (unless one resorts to (postprocessing) tricks). It is the (electronic) noise that is generated while reading data from the sensor, and as recorded in the Raw data as a noise floor. It is an important parameter in the determination of Dynamic Range.

The data was collected from so-called Black frames. Black frames are Raw files that received no exposure, by recording 'images' without lens but with bodycap in place, viewfinder covered, and the shortest possible 'exposure' time (1/8000th second) selected.

The frames were shot in pairs, at 4 second intervals (to avoid potential noise from the writing action to the CF card), thus allowing to reduce the potential effects of hot pixels and pattern noise by calculating the standard deviation of the difference between the files, divided by Sqrt(2). All available ISO settings were tested in the same manner, and all files were shot at approx. 20 degrees Celcius ambient temperature. The actual sensor elements used were a central crop of the same 400x400 sensels as would be later used for other (actual exposure) analyses. That means that for each of the G/R/G/B Bayer CFA filtered sensels, 40,000 samples were available (a quantity which should be enough for statistically relevant sample populations). To allow comparisons with data collected by others, the frequently used IRIS software (version 5.51) was used to read the Raw (non-color balanced, non-demosaiced) 14-bit data from the files.

The 14-bit quantized data was scaled to 16-bit to facilitate comparison with other bit depths, divide by 4 to compare with other 14-bit results, or by 16 to compare with 12-bit quantization results. Here are the results:


Several things can be learned from looking at the data. With the exception of ISO 500, only the 'regular' ISOs (100, 200, 400, 800, 1600) are useful for lowest read noise performance. The 'intermediate' ISOs have more read-noise than their next higher 'regular' ISO. There is also no read noise benefit to using the special 'L' and 'H' settings, other than allowing for more extreme exposure levels. Unlike with the 1Ds2, ISO 'L' doesn't improve the read noise performance and the associated dynamic range. ISO 'H' is clearly a simple multiplication result of an (underexposed) ISO 1600 sensitivity.

Given the fact that the maximum clipping level of all sensels appears to be 15,280 data numbers (plus or minus 1 DN or ADU) in 14-bit quantization space, one can also conclude that the theoretically maximum Dynamic range, according to the common engineering definition, is Log(15280 / 5.7625) / Log(2) =~ 11.3 stops. While that is not an improvement when compared to the 1Ds2, it is almost the same. A higher DR would have been welcomed, but Canon made a different quality trade-off in favor of resolution at the expense of DR improvement.

Bart

[Johnny_Johnson]

Hi Bart,

What am I missing? It looks like that 1000 & 1250 have the same advantage that 500 does.

Later,
Johnny

[Bart_van_der_Wolf]

Originally Posted by Johnny_Johnson

Hi Bart,

What am I missing? It looks like that 1000 & 1250 have the same advantage that 500 does.

Later,
Johnny

You're right, thanks for pointing it out. I overlooked it because I am personally mostly interested in lower ISOs for studio work or architecture.

Bart

[Doug Kerr]

Hi, Bart,

Thanks for the very interesting report. I need to contemplate your results for a while to be sure I grasp their significance.

Originally Posted by Bart_van_der_Wolf

Given the fact that the maximum clipping level of all sensels appears to be 15,280 data numbers (plus or minus 1 DN or ADU) in 14-bit quantization space, one can also conclude that the theoretically maximum Dynamic range, according to the common engineering definition . . .

Would you explain a little more what your notion of "the common engineering definition" of the dynamic range of a digital camera is. (I'll be able to reverse engineer your definition from your numbers, but I'm too tired to do that right now!)

Is it essentially the same as the definition of dynamic range defined by the ISO standard?

For reference, that definition is essentially:

The ratio of (a) the luminance* implied by the maximum digital output number to (b) the RMS noise (on the basis of the variation in luminance that would cause the observed random variation in digital output) experienced at a base luminance of 0.01 the luminance in (a).

*Precisely, photometric exposure, but that essentially corresponds to relative luminance since f/number, shutter speed, and the like would be constant between the two parts of the comparison.


Again, thanks for this nice report and the clear presentation of the data.

[Joseph A. Kurkjian]

Bart, thanks very much for posting your test results.

Regards,

Joe Kurkjian

[Asher Kelman]

Originally Posted by Bart_van_der_Wolf

Given the fact that the maximum clipping level of all sensels appears to be 15,280 data numbers (plus or minus 1 DN or ADU) in 14-bit quantization space, one can also conclude that the theoretically maximum Dynamic range, according to the common engineering definition, is Log(15280 / 5.7625) / Log(2) =~ 11.3 stops. While that is not an improvement when compared to the 1Ds2, it is almost the same. A higher DR would have been welcomed, but Canon made a different quality trade-off in favor of resolution at the expense of DR improvement.

Bart, could you show where the 15280 and 5.7625 come from?

It would be nice to know to what extent noise increased by extending exposure to 1/1,000 1/250, 1/50 sec. Might it not be that on a hot day, for example, just going to 1/30 sec from 1/8,000 sec might be worse than going from ISO 200 to 500. After all there would be more time for the noise to accumulate.

Thanks

Asher

[Ralph Eisenberg]

Thanks very much for having taken the time and gone to the trouble to do this testing, as well as allowing us to benefit from your results.

[Bart_van_der_Wolf]

Originally Posted by Doug_Kerr

Would you explain a little more what your notion of "the common engineering definition" of the dynamic range of a digital camera is.

It simply is the signal saturation level, 15280 in this case, divided by the read noise, approx. 5.77 for both ISO 'L' and ISO 100. As such it doesn't take 'acceptable' levels of noise into consideration, but rather the maximum attainable under controlled situations.

A slightly more detailed way of defining it is given by: http://www.ccd.com/ccd111.html.

Again, thanks for this nice report and the clear presentation of the data.

You're welcome. I like to share this type of info because it allows a better founded comparison between different camera solutions for different applications, and it takes the guesswork factor out of discussions. It helps people to make better decisions, and allows to tweak solutions for dealing with equipment limitations (e.g. HDR imaging if expose-to-the-right is not enough help).

In the particular case of the 1Ds3, which is intended as a studio camera (where lighting conditions can be controlled) just like the 1Ds2 was/is, it is clear that optimal exposure (ETTR) is very important for the best results. For critical outdoor work, one therefore might want to consider more exposure bracketing if the best image quality is needed. The 1Ds3 allows to bracket up to 7 shots in a little over 1 second (depending on shutter time of course).

Bart

[Torsten Kötting]

Originally Posted by Bart_van_der_Wolf

A higher DR would have been welcomed, but Canon made a different quality trade-off in favor of resolution at the expense of DR improvement.

do you think Canon had a choice? my impression is das DR hasnt improved for a while now because they dont know how to do so ...

[Bart_van_der_Wolf]

Originally Posted by Asher Kelman

Bart, could you show where the 15280 and 5.7625 come from?

The 15280 (+/- 1) is the highest data number that can be found in the raw 14-bit image data. It can be found in overexposed and clipped images. It follows from another test that I'll share, where a series of increasing exposures of a uniform surface is used for the determination of the 'gain' and 'unity gain' parameters (the latter is important if one wants to control noise in highlight areas).

The 5.7625 is just the simple average of the read noise at ISO 'L' in its native 14-bit data. You can also average the 16-bit equivalents shown in the chart table (23.05) and divide by 4 to scale to 14-bit values.

It would be nice to know to what extent noise increased by extending exposure to 1/1,000 1/250, 1/50 sec.

In earlier tests I (and others) have done, it doesn't make a significant difference. I have yet to verify it for the 1Ds3, but I expect it to be the same as for other Canon cameras that I've tested. Let's say that in handheld shooting conditions, e.g. up to 1/30th sec., it would be hard to measure a significant increase in noise, let alone see it.

Might it not be that on a hot day, for example, just going to 1/30 sec from 1/8,000 sec might be worse than going from ISO 200 to 500. After all there would be more time for the noise to accumulate.

The major contributor to temporal noise, is called the 'dark count'. It accumulates over time, and it increases exponentially with temperature. In my experience it is swamped by other noise sources in regular handheld exposure scenarios. It becomes an increasingly more important factor when we approach 1 second exposure times, or longer. To reduce it's influence as much as possible in the read noise tests, it is therefore customary to use the shortest possible integration time when testing.

Bart