The Canon sRaw format - New technical article

The Canon "sRaw" output format, which debuted on the EOS 1D Mark II, has been the source of little awe and much mystery. Canon of course doesn't say anything about exactly what it is that means anything, and doesn't anymore bother to mention why it's good.

Fortunately, a "daisy chain" of wonks have managed to reverse-engineer the critter, but of course their reports don't really encapsulate just what we have.

I decided to wade into this mass of information and see if I could figure out what is really going on and, beyond that, how it could be explained. The result, as you might well expect, is a new mass of information (several times the size of the collected wonking, in fact- but hopefully more entertaining).

I refer to my new technical article, "The Canon sRaw and mRaw Output Formats", available here:

http://dougkerr.net/Pumpkin/index.htm#sRaw

So to answer one burning question, "Is the data in the sRaw file really raw?"

Well, it's sort of one-third cooked, still a little raw around the edges.

"Read and be amazed" (the new mantra for the family book-publishing business).

Hi Doug,

I have printed out your original writing on sRAW but I have nor read it fully to go beyond the concept of partially cooked to why should use it. When I have a break from shooting, I'll try the new article. Thanks for bringing us the science!

Asher

Doug_Kerr said:

..."Read and be amazed" (the new mantra for the family book-publishing business).

Click to expand...

Hi Doug,
An excellent paper and yes, as always, I'm amazed!
Now that the small cooling fan in my brain has done it's job (my brain is always over-clocked when consuming your papers!) I have two questions I'll risk embarassment by asking:

- Within each sensel there are 4 components; 1 Red, 2 Green and 1 Blue. - why are there 2 Green spectral filters for each Red and Blue?

- Considering the R/GG/B arrangement and the formula:
R =r0.45 G =g0.45 B =b0.45
wouldn't G's value be the square root of g (which would account for the two G filters)?


Hope I don't sound too lame...

Hi, John,

John Angulat said:

Hi Doug,
An excellent paper and yes, as always, I'm amazed!
Now that the small cooling fan in my brain has done it's job (my brain is always over-clocked when consuming your papers!) I have two questions I'll risk embarassment by asking:

Click to expand...

- Within each sensel there are 4 components; 1 Red, 2 Green and 1 Blue. - why are there 2 Green spectral filters for each Red and Blue?

Well, as I understand it, the greater number of green sensels allows the demosaicing algorithm to do a "more accurate" job of deducing the color of each pixel with respect to the "green" part of the spectrum, where the eye has greater spatial acuity.

Maybe later tonight I'll look at Bayer's patent and see what he actually said.

Considering the R/GG/B arrangement and the formula etc etc etc.

Click to expand...

Those three formulas only tell us how the actual r, g, and b components of a color are (individually) converted to a nonlinear form (as R, B, and B) before being put in a file. It is not at all related to anything about relative properties of the three channels, or the four sensels in a "cluster", or the relative sensitivities of the different kind of sensels, or any notion of adding together the two G sensel outputs.

This all started out a just a way to let a CRT display to directly run from the R, G, and B values, given that, for each "gun", the luminous output varies as about the 2.2 power of the control voltage. (This avoided having to put a nonlinear circuit between the "received values" and the gun control voltages, which would have been complicated when this was adopted, about 1940.) Later this was found to have some conceptual advantages as well.

Incidentally, if the arrangement was actually (for some reason) that "G was the square root of g", then written in that same form, that would be:

G=g^0.5

(nearly what we have here for all three values).

Isn't this all fun!

Best regards,

Doug

Doug_Kerr said:

- Within each sensel there are 4 components; 1 Red, 2 Green and 1 Blue. - why are there 2 Green spectral filters for each Red and Blue?

Well, as I understand it, the greater number of green sensels allows the demosaicing algorithm to do a "more accurate" job of deducing the color of each pixel with respect to the "green" part of the spectrum, where the eye has greater spatial acuity.

Click to expand...

Yup, Green filtered sensels also carry the majority of the luminance information (just like human vision resolution peaks at medium wavelengths). One only has to inspect a e.g. L*a*b* conversion of a color image to recognise the importance of the Luminance channel. The human eye also has many more rods across the field of view (luminance sensitivity, especially at lower luminance levels) than cones (chrominance sensitivity, although the concentration of cones is concentrated to a small angle (when pupil dilation is smallest).

A common approximation of luminance is 0.3*R + 0.59*G + 0.11*B . That doesn't mean that the recorded signals have that same proportion for a spectrally uniform input signal though. The Bayer CFA filters are also optimized for a) spectral band pass characteristics, and b) for transmission (i.e. quantum efficiency of the total sensor compromise, photons striking the sensel vs recorded photons/electrons), while allowing for relatively accurate color reproduction along the human visible spectrum.

Cheers,
Bart

Doug, Bart -
Thanks so much for the follow-up explanation.
You guys are limitless in your knowlege (and I'm all the better for it!)

Actually, in Bayer's original patent, each cluster consisted of a red-filtered sensel, a blue-filtered sensel, and two green-filtered sensels. But he thought of the green-filtered ones as essentially picking up luminance.

Why did he not provide unfiltered pixels for picking up luminance? Perhaps to avoid conflict with the earlier Banning patent, which used 2x2 clusters with R, G, B, and W (white; that is, unfiltered) sensels!

Why did he not think in terms of red, green, and blue-filtered elements? Perhaps to avoid conflict with an even earlier patent by Banning, which described "triad" clusters with - you got it, one red, one green, and one blue element each.

Nevertheless, he still mentioned that an advantage is that the green sensels occur most frequently, providing finer sampling of the green component, for which the eye has greatest acuity.

(C'mon Bruce - are they for luminance, or for green? A little forked tongue here.)

In his Claim 1, an array is described that includes luminance-sensitive elements, which occupy at least half of the element positions, along with two other kinds of elements with sensitivity to two different specific parts of the spectrum.

In Claim 2, an array is described, like the one in Claim 1, in which the luminance-sensitive elements are green-sensitive and the other two kinds of elements are red- and blue-sensitive.

Got that?

In any case, even if the third kind of sensel truly sensed luminance, we would still be able to ascertain color; if we know Y, R, and B, we can ascertain G.

Incidentally, a recent Kodak patent describes a CFA scheme in which the clusters (4x4) have 2 blue, 2 red, 4 green, and 4 unfiltered ("luminance") sensels.

Another recent patent describes a CFA scheme in which each cluster (2x2) has one red, one green, one blue, and one unfiltered sensel.

Many neat things to think about.

Meanwhile, I just got a call, which turned out to be for for Carla (from a national political party office) and the CNAM (the caller's name in the caller ID) was "Octothorpe". Kind of eerie.

Best regards,

Doug

Doug_Kerr said:

Meanwhile, I just got a call, which turned out to be for for Carla (from a national political party office) and the CNAM (the caller's name in the caller ID) was "Octothorpe". Kind of eerie.

Click to expand...

My dear sir, I believe your past is coming back to haunt you!

Hi, John,

John Angulat said:

My dear sir, I believe your past is coming back to haunt you!

Click to expand...

Well, so it seems. As some sage said recently, "we can run, but we can't hide."

By the way, I asked the caller, before I went off to fetch Carla for him, why his "caller ID name" was 'octothorpe'. He had no idea that it was, or why, or of course even what that was. He was just a boiler room caller.

"Octothorpe" is, as you may know, an extremely common misspelling of the term (which is "octatherp") - perhaps even the most commonly-encountered form.

Best regards,

Doug

Doug_Kerr said:

Why did he not provide unfiltered pixels for picking up luminance?

Click to expand...

Perhaps because the Green filtered luminance portion, together with the Red and Blue filtered portions, allowed for more accurate color reconstruction at all spatial sampling positions at the same time (kind of doing one thing but not forgetting to do another thing as well, leading to a better overall result, synergy)?

Human vision is most sensitive to Green(ish) wavelengths, and also shifts towards relatively shorter wavelengths than full spectrum at lower luminance levels (AKA Punkinje effect, e.g. exploited in night vision goggles)

Cheers,
Bart

Hi, Bart,

Bart_van_der_Wolf said:

Perhaps because the Green filtered luminance portion, together with the Red and Blue filtered portions, allowed for more accurate color reconstruction at all spatial sampling positions at the same time (kind of doing one thing but not forgetting to do another thing as well, leading to a better overall result, synergy)?

Click to expand...

A very nice insight and description. Thanks.

Human vision is most sensitive to Green(ish) wavelengths, and also shifts towards relatively shorter wavelengths than full spectrum at lower luminance levels (AKA Punkinje effect, e.g. exploited in night vision goggles)

Click to expand...

Purkinje, although we here near Punkin Center, Texas (as well as being the proprietors of The Pumpkin) appreciate the thought.

Best regards,

Doug