D300 vs 40D controlled tests

I did an analysis of the D300 vs the 40D image quality using raw data taken under controlled conditions.
The end result is that the raw data are quite comparable, with the 40D doing better by some measures,
and the D300 faring better by other criteria. Here's the URL:

http://theory.uchicago.edu/~ejm/pix/20d/posts/tests/D300_40D_tests/

Emil Martinec said:

I did an analysis of the D300 vs the 40D image quality using raw data taken under controlled conditions.

Click to expand...

Hi Emil,

Even though you're probably not going to get as much response here as on DPReview, I just wanted to let you know that your info (and mentioning it here on OPF) is highly appreciated.

Your write-up is well done and provides an informative overview. Of course there are also other factors besides noise and quantum efficiency that define image quality, e.g. MTF response, but it is important to know how our basic building blocks (photons) are converted into (hopefully) clean data.

Bart

Emil Martinec said:

I did an analysis of the D300 vs the 40D image quality using raw data taken under controlled conditions.

Click to expand...

Hi Emil,

Just a question, relating to 'gain' determination. As a side note, I'm not sure "quantum efficiency" is a better descriptor than "gain". To me QE is more about the percentage of photons from the incoming photon flux that do get converted to charge. I also agree that "gain" is not an entirely appropriate description, but it has a tradition (in electronic amplification).

You determine 'gain' from a single exposure point (which saves time). Do you think you might get an even more accurate/robust number by determining the slope of a linear regression between several exposure levels close to saturation?

As an example, from my 1Ds2:


Note: the saturation/clipping exposures have been omitted from the regression calculation/chart, due to non-linear response.

Bart

Bart_van_der_Wolf said:

Hi Emil,

Just a question, relating to 'gain' determination. As a side note, I'm not sure "quantum efficiency" is a better descriptor than "gain". To me QE is more about the percentage of photons from the incoming photon flux that do get converted to charge. I also agree that "gain" is not an entirely appropriate description, but it has a tradition (in electronic amplification).

You determine 'gain' from a single exposure point (which saves time). Do you think you might get an even more accurate/robust number by determining the slope of a linear regression between several exposure levels close to saturation?

As an example, from my 1Ds2:
http://www.xs4all.nl/~bvdwolf/temp/OPF/1Ds2_ISO200_Gain.png

Note: the saturation/clipping exposures have been omitted from the regression calculation/chart, due to non-linear response.

Bart

Click to expand...

I had a series of test shots run at +2EV and at +3 EV exposure. I used the +2EV images to do the 'gain' measurements vs ISO, and did a spot check with a few +3EV measurements to see that I was getting the same answer to within a few percent (and I was). I'm not sure the camera-to-camera variation is less than the gain in precision one might get from a multi-EV combined measurement. Anyway it's moot; my associate who's doing the shooting legwork is about return the (borrowed) 40D being used for the tests. I may be able to squeeze out an MTF comparison before that happens; we'll see.

As for semantics, I think QE is more appropriate, or at least 'relative QE'; if you divide the per pixel measurement by the pixel area, you get an unnormalized measure of the photons being captured per unit area (Table 8 of my little essay); so this is a solid number whose ratio between different sensors is their ratio of absolute QE's.

And on another note, I was unhappy with the quality of the test I used for judging relative sensitivity, so I redid it and the revised results are in an update of the webpage. Bottom line -- both cameras expose similarly, the D300 is NOT more sensitive.

Emil Martinec said:

I had a series of test shots run at +2EV and at +3 EV exposure. I used the +2EV images to do the 'gain' measurements vs ISO, and did a spot check with a few +3EV measurements to see that I was getting the same answer to within a few percent (and I was).

Click to expand...

Okay, that's what makes the judgement more robust, by not depending on a single observation.

There is another benefit to a linear regression though, it takes away the dependency on (the accuracy of) the read noise determination. But since the read noise will probably be determined anyway, it just shows that there are more routes leading to the same destination. Personally I like the visual feedback that a trendline offers, but then as a photographer I have a visual orientation.

Bart

Bart_van_der_Wolf said:

Okay, that's what makes the judgement more robust, by not depending on a single observation.

There is another benefit to a linear regression though, it takes away the dependency on (the accuracy of) the read noise determination. But since the read noise will probably be determined anyway, it just shows that there are more routes leading to the same destination. Personally I like the visual feedback that a trendline offers, but then as a photographer I have a visual orientation.

Bart

Click to expand...

At ISO 200, subtracting out the read noise makes a 1/2 percent correction to the total noise in determining the shot noise. I don't think it makes any difference. Perhaps at high ISO where the signal and therefore the shot noise is closer in magnitude to the read noise, one might want to do the regression.

A couple more data points to add. I looked at the ISO extension. The low ISO extension appears to be ISO 125 in actual fact (based on the raw level compared to ISO 200, shot under the same conditions). Saturation at the low ISO is the same raw level as at ISO 200. The high ISO extension isn't obviously implemented in software -- there are no gaps in the raw histogram as there is in Canons.

Emil Martinec said:

I looked at the ISO extension. The low ISO extension appears to be ISO 125 in actual fact (based on the raw level compared to ISO 200, shot under the same conditions). Saturation at the low ISO is the same raw level as at ISO 200. The high ISO extension isn't obviously implemented in software -- there are no gaps in the raw histogram as there is in Canons.

Click to expand...

I don't think there are any gaps in ISO 3200 in anything but the 20D, 30D and 40D. The 5D and 1D*mk2 series use an amplifier to get 3200 - but not at the photosite - just like most CCD cameras get all their ISOs.

Also, Nikon tends to rescale data after capture. You will often see a periodic pattern superimposed on the full histogram - this means that the numbers have been altered, and if there were an integer push, it could be somewhat hidden by this. Gaps in every other histogram value tells you that the push is integer, but the lack of gaps does not prove that there is no such push.

John Sheehy said:

Also, Nikon tends to rescale data after capture. You will often see a periodic pattern superimposed on the full histogram - this means that the numbers have been altered, and if there were an integer push, it could be somewhat hidden by this. Gaps in every other histogram value tells you that the push is integer, but the lack of gaps does not prove that there is no such push.

Click to expand...

Yes, at ordinary ISO's the red channel and blue channel data have gaps in them as I mentioned in the review. Do you know why they do this? It seems to introduce rounding errors without any obvious benefit.

Emil Martinec said:

Yes, at ordinary ISO's the red channel and blue channel data have gaps in them as I mentioned in the review. Do you know why they do this? It seems to introduce rounding errors without any obvious benefit.

Click to expand...

Canon does things like this too, but usually globally; not separately for the channels. I suppose they have some target sensitivity for each channel, in the case of the Nikon, and for all the channels, in some Canons (the 5D always, the 30D sometimes when a fast lens is used wide open), and some kind of post-digitization compensations are done. Personally, I think they should leave the image data alone and just put the desired factors into the RAW files' metadata, for any compensations that are the least bit destructive.

Emil Martinec said:

Yes, at ordinary ISO's the red channel and blue channel data have gaps in them as I mentioned in the review. Do you know why they do this? It seems to introduce rounding errors without any obvious benefit.

Click to expand...

Canon does things like this too, but usually globally; not separately for the channels. I suppose they have some target sensitivity for each channel, in the case of the Nikon, and for all the channels, in some Canons (the 5D always, the 30D sometimes, even at the main ISOs, when a fast lens is used wide open), and some kind of post-digitization compensations are done. Personally, I think they should leave the image data alone and just put the desired factors into the RAW files' metadata, for any compensations that are the least bit destructive.