black frame noise of my 40D

I was curious to know the black frame noise of my 40D so I downloaded IRIS, installed it, and tried to get it to say something useful.

The result is a sigma value of 21.57 at ISO 1600 for 16 bit values. Presumably the 12 bit equivalent would be 5.4. Does anyone have equivalent values for the other Canon DSLRs? I'm curious how my 40D compares.

I'm not sure I did it correctly. I shot a 14 bit RAW file with the lens cap on and the viewfinder blocked. Shot at ISO 1600 with shutter speed of 800. Then I simplly installed IRIS, loaded the raw file via the "load a RAW file" command, drew a rectangle (size didn't matter much), right clicked and looked at the sigma value. Do I need to do something else? The value was very consistent across the whole image. Can one of you experts confirm this is valid approach?

Thanks!

Byron

BJ Sheppard said:

I'm not sure I did it correctly. I shot a 14 bit RAW file with the lens cap on and the viewfinder blocked. Shot at ISO 1600 with shutter speed of 800. Then I simplly installed IRIS, loaded the raw file via the "load a RAW file" command, drew a rectangle (size didn't matter much), right clicked and looked at the sigma value. Do I need to do something else? The value was very consistent across the whole image. Can one of you experts confirm this is valid approach?

Click to expand...

Shooting with the lens cap on and blocking the viewfinder is usually adequate, but if you want to also eliminate potential effects from the lens electronics, try shooting with the body cap on, no lens. Also make sure you've disabled any noise reduction setting in the camera (or try both ON/OFF to see if it makes a difference on the darkframe noise).

You've correctly loaded as a Raw, so the resulting 'image' is indeed the monochrome Bayer CFA you are looking at. However, since the image is not colorbalanced yet, the differences between the R/G/B sensels may seem to be larger than they actually are. For Black/Dark frames this is only a minor effect, but if you want to be more accurate the following is recommended.

Display the IRIS "Commands window" by clicking its button in the toolbar. With the Raw file loaded, type the command >split_cfa s1 s2 s3 s4 which will separate the Red, Green 1 and 2, and the Blue sensels into 4 separate files named s1...s4. You can set the location where they are saved in the File|Settings dialog under the working path (while you're there, you may want to set the File type to PIC). You can now sequentially reload and analyze the 4 separate files by typing >load s1 or >load s2, etc. After each loading (the active file name is reflected in the top of the IRIS window frame) you can draw a rectangle and right-click for the statistics.

Which color the S1...S4 separations represent is not directly obvious in IRIS, you'll have to derive that from an image with clear R, G, and B patches, and look for the brightest patch. The brightest patch corresponds with the color.

Important! The statistics values are not 16-bits domain values as suggested at the bottom right status, they are still in the, in your case, 14-bit domain. So you'd need to divide statistics results by 4 to compare with other 12-bit results, or multiply by 4 to directly compare with other 16-bit results.

Bart

Thanks for the reply.

This is a typo in my original post... I said 16 bits but meant 14. That's why I divided 21.57 by 4 to give the 12 bit equivalent value. I can't find a way to edit my original post...

I redid the test using the body cap with same results. I then changed the shutter speed to 8000 and also tried shooting with and without high ISO noise reduction and saw no difference either. This suggests the high ISO noise reduction is a post processing step and nothing is done to the raw data.

I'm still looking for similar tests done for other Canon cameras. If anyone knows of them please post up!

And thanks for the IRIS comments...quite helpful since I'm trying to figure this program out. I will try the split command when I get a moment.

Byron

BJ Sheppard said:

I was curious to know the black frame noise of my 40D so I downloaded IRIS, installed it, and tried to get it to say something useful.

The result is a sigma value of 21.57 at ISO 1600 for 16 bit values. Presumably the 12 bit equivalent would be 5.4.

Click to expand...

Yep. What also matters is what the clipping point is, and what the absolute sensitivity is.

Does anyone have equivalent values for the other Canon DSLRs? I'm curious how my 40D compares.

Click to expand...

My 20D is 4.7. The first 30D I bought was about 6.0, and I figured that it was using excessive gain, and losing 1/3 stop of DR, so I replaced it with one that turned out to be 4.3. The 1Dmk3 is about 3.2, IIRC (finally justifying its price in the high ISO dept. - the mk2 was no better than the 20D). As I said above, though, absolute sensitivity is important. You may be seeing a higher level of noise, in ADUs, than a 20D or 30D, but the same real-world signal level could be higher in the 40D, making the absolute S/N higher for the 40D, while losing a little bit of DR. You need to shoot the same scene through the same lens, with the same manual exposure, to really compare the absolute sensitivities. To check traditional DR, however, all you need to know is the clipping point, the black point, and the noise.

Also, you need to scale the noise to the MP count. 5.4 ADU noise in a 10MP sensor is cleaner than 5.4 in an 8.

I'm not sure I did it correctly. I shot a 14 bit RAW file with the lens cap on and the viewfinder blocked. Shot at ISO 1600 with shutter speed of 800.

Click to expand...

That should do.

Then I simplly installed IRIS, loaded the raw file via the "load a RAW file" command, drew a rectangle (size didn't matter much), right clicked and looked at the sigma value. Do I need to do something else? The value was very consistent across the whole image. Can one of you experts confirm this is valid approach?

Click to expand...

IRIS only allows you to draw a rectangle in 100 pixel view mode. You just have to be careful with the 40D that you keep away from the edges of the RAW "image" because IRIS understands the CR2 compression scheme, but it does not know what is really part of the image (as it is not aware of the 40D camera), so on the top, left, and bottom edges of the frame, there may be some non-image pixels that are not supposed to be black. Including them may boost the sigma a little, especially in a small rectangle.

Bart_van_der_Wolf said:

However, since the image is not colorbalanced yet, the differences between the R/G/B sensels may seem to be larger than they actually are. For Black/Dark frames this is only a minor effect, but if you want to be more accurate the following is recommended.

Click to expand...

The readout hardware in the Canons (and most CFA cameras, although some are rumored to have different gains for different channels, but I've never seen any proof) is completely blind to the CFA. There is no need to separate the channels for non-exposures.

John Sheehy said:

The readout hardware in the Canons (and most CFA cameras, although some are rumored to have different gains for different channels, but I've never seen any proof) is completely blind to the CFA. There is no need to separate the channels for non-exposures.

Click to expand...

Since I'm learning...I tried it anyway. Took one of my black frames and split the file into 4 parts comparing the statistics. Results were:

Initially loaded RAW file = 21.86
S1 = 21.59
S2 = 21.87
S3 = 22.12
S4 = 22.15

I tried to make sure the statistics rectangle was in the same place each time. Multiple runs produced repeatable results to within +/- 0.01.

S3 and S4 looked suspiciously similar (I haven't verified which color is which channel yet) so I tried a couple other files and that correlation disappeared. This file actually had the highest variance between the four channels, all other files were closer and in fact in some files S1-S4 were the same down to and including the 1/10 decimal place. So I conclude nothing special is happening between the various channels and most likely they are being read out simply as sensels.

BJ Sheppard said:

Since I'm learning...I tried it anyway.

Click to expand...

Yes, and (besides the learning experience) it is also important IMHO not to assume the behavior of (especially) a new camera model based on other models. Afterall, we cannot know a priori if the new DIGIC III behaves the same as previous versions. I prefer to verify before assuming.

Took one of my black frames and split the file into 4 parts comparing the statistics. Results were:

Initially loaded RAW file = 21.86
S1 = 21.59
S2 = 21.87
S3 = 22.12
S4 = 22.15

I tried to make sure the statistics rectangle was in the same place each time. Multiple runs produced repeatable results to within +/- 0.01.

Click to expand...

So while the differences are small between Raw files, the maximum difference between the channels in this particular sample still exceeds 1%. Whether that's considered as much or not, depends on the ensuing use of the results.

So I conclude nothing special is happening between the various channels and most likely they are being read out simply as sensels.

Click to expand...

At this level of (non-)exposure, that seems to be the case, but always be aware of what happens when you do increase exposure. Then the channels may start to exhibit more different results between them, QED.

As for your earlier question about how this relates to other cameras, I did some analyses on a couple of 1D Mark III files made available by John Nevill, and for the same ISO 1600 setting at 1/8000 second exposure time, IRIS reports an overall Sigma of 12.2 (also in the 14 bits domain, after cropping out the non-image edge sensel data). That would suggest the 40D signal is amplified more than the 1D Mark III's signal (which would be somewhat in line with differences due to smaller sensels in the 40D's sensor array), but maybe there are (also) other reasons.

I've also measured (same ISO and 'exposure' time) an overall Sigma of 7.7 for a 1Ds Mark II in the 12 bits domain (would become 30.8 @ 14 bits), and an overall Sigma of 4.5 for a 20D in the 12 bits domain (would become 18 @ 14 bits).

That would suggest that there is noticable progress in the most recent models.

Bart

John Sheehy said:

....As I said above, though, absolute sensitivity is important. You may be seeing a higher level of noise, in ADUs, than a 20D or 30D, but the same real-world signal level could be higher in the 40D, making the absolute S/N higher for the 40D, while losing a little bit of DR. You need to shoot the same scene through the same lens, with the same manual exposure, to really compare the absolute sensitivities.

Click to expand...

I only have a 300D to compare with the 40D.

I shot a piece of white paper (defocussed) with my trusty 100-400L using the same manual exposure setting on both cameras and at ISO 800. I shot a test using halogen lighting and then another using natural lighting.

The halogen lighting case (for example) produced the following:

300D:
S1 = 445
S2 = 275
S3 = 366
S4 = 443

40D:
S1 = 1714
S2 = 1294
S3 = 1525
S4 = 1714

From my black frame noise shots I determined the black point of the 300D as 128 and the 40D as 1024.

Clipping points for the 300D are 3989 for the S1 and S3 channels and 3962 for the S2 and S4 channels ( I measured this by WAY overexposing the sheet of paper until all channels clipped). Clipping points for the 40D are 16224 in all channels (S1-S4).

For both cameras the channel order seems to be GBRG for S1-S4 respectively.

Now...what do I do with all these numbers!

Taking the green channels, I calculate the 300D sensitivity as being 445-128 = 317 ADU for this scene. The 40D is 1714 - 1024 = 690. But being 14 bits I scale it down to 690/4 = 172 equivalent. This makes the 40D log (317/172) / log 2 = 0.88 stops less sensitive.

Assuming I haven't messed it all up....this seems to indicate the 40D needs to use approximately one stop higher ISO to get the same net image brightness as the 300D. So the noise improvement is not as great as one might think from simply comparing noise at the same ISO values. Correct?

To check traditional DR, however, all you need to know is the clipping point, the black point, and the noise.

Click to expand...

At ISO 800,

300D clip = 3989, black point = 128, noise = 10.28
40D clip = 16224, black point = 1024, noise = 12.77

So, 300D DR would be (3989 - 128) / 10.28 = 375 or 8.5 stops; and the 40D DR would be (16224-1024) / 12.77 = 1190 or 10.2 stops.

The difference is a bit misleading though given the sensitivity difference! Not sure how one addresses the sensitivity difference in the equations...I seem to have calculated two pieces of information but don't know how to combine them except empirically.

As an aside, I noticed the 40D metering was 2/3 stops less than the 300D. So in practice if I let the 40D meter the way it wanted I would end up with 2/3 stop slower shutter speed and about the same image brightness as the 300D. This suggests the 40D to has maintained the same/similar metering standard (at least when using center weighted metering on a white sheet of paper!).

IRIS only allows you to draw a rectangle in 100 pixel view mode.

Click to expand...

I'm seeing 100% pixels and happily drawing rectangles. Is there another mode and how would I get to it?

You just have to be careful with the 40D that you keep away from the edges of the RAW "image"

Click to expand...

Yes, I noticed that. Thanks.

Bart_van_der_Wolf said:

...So while the differences are small between Raw files, the maximum difference between the channels in this particular sample still exceeds 1%. Whether that's considered as much or not, depends on the ensuing use of the results.

Click to expand...

FWIW, my 0.01 variance was on the SAME raw file...I was just testing the variablility when redrawing the statistics rectangle.

I've noticed when doing different RAW files (especially at high ISO such as this) the data seems to move around a bit in a random fashion. Often I see variability in the numbers that approaches 1% or so. For example, this result here was my worst example while my best example was at least 10x better! All under the same conditions.

Such is the nature of noise I guess...

Since it is pouring rain, I thought I'd summarize some of the black frame noise values we've posted in this thread. All at ISO 1600. I've listed 12 bit and 14 bit equivalent numbers.

300D
20.5 (82)

20D
4.5 (18)
4.7 (18.8)

30D
6.0 (24)
4.3 (13.2)

40D
5.4 (21.7)
5.5 (21.9)

1DMk3
3.1 (12.2)

1DsMk2
7.7 (30.8)

My 300D cheats on ISO 1600 as it appears to be just ISO 800 numbers doubled. The black point goes from 128 to 253, the noise doubles from 10 to 20, and the histogram develops alternating gaps. Interestingly the gaps appear to be dithered in some manner to avoid making them zero.

My 40D does something similar starting at ISO 3200. The noise suddenly doubles to 44 and the histogram develops those "alternating dithered gaps". Unlike the 300D however, the black point remains at 1024.

Why does Canon set the black point at 1024? Based on the range of noise data I've seen, 512 would be plenty adequate to avoid clipping.

Also how does "highlight tone priority" mode work? I activated it for an ISO1600 test and the noise value and black point changed to the ISO 800 values suggesting ISO800 level gain rather than normal ISO1600 gain. The clipping point remained the same however.

BJ Sheppard said:

Why does Canon set the black point at 1024? Based on the range of noise data I've seen, 512 would be plenty adequate to avoid clipping.

Click to expand...

I'm not certain Canon does, they might, but I'm not 100% sure. All we do know is that IRIS (using the DCRAW decoding fundamentals) reports it as such.

Also how does "highlight tone priority" mode work? I activated it for an ISO1600 test and the noise value and black point changed to the ISO 800 values suggesting ISO800 level gain rather than normal ISO1600 gain. The clipping point remained the same however.

Click to expand...

HTP is a gain reduction, thereby creating a stop of overexposure/clipping latitude (at the expense of shadow detail). Also see the conclusion of this thread.

Bart

Bart_van_der_Wolf said:

I'm not certain Canon does, they might, but I'm not 100% sure. All we do know is that IRIS (using the DCRAW decoding fundamentals) reports it as such.

Click to expand...

Why would there be doubt? One data number is the same as any other so if we believe all the other numbers we're getting then I don't see the reason to doubt the value of 1024. Am I missing something?

HTP is a gain reduction, thereby creating a stop of overexposure/clipping latitude (at the expense of shadow detail). Also see the conclusion of this thread.

Click to expand...

Great, thanks. So it does what I thought...seems kind of lame.

Out of curiosity I tested a friends 30D and compared with my 40D.

The black frame noise (12 bit equivalent) was:

ISO 100:
30D = 1.95
40D = 1.42

ISO 200:
30D = 2.07
40D = 1.59

ISO 400:
30D = 2.37
40D = 2.05

ISO 800:
30D = 3.20
40D = 3.19

ISO 1600:
30D = 4.92
40D = 5.45

ISO 3200:
30D = 9.9
40D = 11.11

So at lower ISO my 40D was better while at ISO 1600 and 3200 the 30D was better. But the differences are not large and on an actual high iso image developed in DPP with same settings (indoor scene with tungsten light) looked pretty similar to me.

Sensitivity testing showed the 40D to be 0.35 stops less sensitive in the Green channel. Blue was 0.27 stops less sensitive and Red was 0.5 stops less sensitive.

Metering on spot metering was usually identical but evaluative metering results varied between 1/3 stop higher to 2/3 stop less depending on scene lighting/composition.

Resolution was about 11% better on test charts but not visible on real images.

Clipping was 3398 at ISO 100 and 4071 at ISO200-1600 for the 30D, it was 13825 at ISO 100 and 16225 at ISO200-1600 for the 40D.

All in all, pretty similar results. Makes me wonder if I should have saved my money and just bought a cheap 30D! Changing lenses produced a much bigger image quality difference than changing cameras...!

First of all, Hi to you all. I am new here, though a few of you know me from DPReview. My particular interest is in understanding the raw characteristics of cameras. (Yes, I am a pixel peeper).

BJ Sheppard said:

Clipping was 3398 at ISO 100 and 4071 at ISO200-1600 for the 30D, it was 13825 at ISO 100 and 16225 at ISO200-1600 for the 40D

Click to expand...

I don't have this camera (yet), but I received perhaps a dozen raw images with different ISO. The clipping point is always under 14000 in these images.

Do you mind posting or sending me an image, where the clipping (saturation) is 16255 or anthing close that?

Another issue: I saw it several times, that the black point of the 40D is 1024. This is not so. The black level (black point is IMO not a suitable term for this) is specified per pixel raw for the Canon DSLRs. There are 2602 black point specifications for the 40D, these are between 1020 and 1030 (ISO 1250), between 1019 and 1029 (ISO 3200), 1022 and 1030 (ISO 100 and ISO 200). These are examples, I don't know, if the ranges are always the same for a given ISO.

This fact has an interesting relevance: pixels, which "clip together", should show different values, depending on the related black level - but this is not so. For example all the clipped red pixels of the 40D show the very same value - which should mean, that they clip at different points, with the same variation as the black levels are. However, their identical clipped values stink. This points to some game driven by the DIGIC processor.