Doug Kerr
Well-known member
In a "true tristimulus" color camera, each little region of the scene that is destined to be represented as a pixel of the delivered image is examined, in the image on the focal plane, by three photodetectors having different spectral responses. In most cases, we identify there as the "R", "G", and "B class of photodetector.
In a "three chip" color camera (many video cameras are of this type), the three photodetectors that regard a pixel of the image are in different arrays, the three arrays being each provided their own copy of the image through beam splitting arrangements. (The filters that confer the different spectral responses to the three classes of detector work "in bulk" as part of the three paths.)
In a color camera with a "Foveon" sensor, the three photodetectors that regard a pixel of the image are stacked at the same site on a single array. The top two of them are "semitransparent", so that all three receive light from the illumination on that spot (much as with the three layers on typical color film).
But in Canon cameras to date, a color filter array (CFA) sensor is used. This is not a true tristimulus sensor.
In it, the collection of "R", "G", and "B" detectors are interleaved across the array, at locations that correspond to the pixels of the native delivered image. Thus each pixel of the image is not regarded by three detectors, but rather by only one - with one of the three responses.
By a clever "intelligent interpolation process" we speak of as demosaicing, the camera is able to make an intelligent estimate of what all three sensor values would be for each pixel of the image. Thus, in effect, a real, if not always accurate, tristimulus representation of the image is attained.
Of course, as I emphasize with my "qualified" language, the color rendition may in fact not be accurate.
In the newly-announced Canon C300 "Cinema EOS" camera, a true tristimulus sensor system is employed. The actual sensor array consists, just as in a CFA sensor, of "R", "G", and "B" detectors, interleaved across the image, but not one in each pixel location. In fact, a "quad" of four of them (yes, one "R", one "B", and two "G"s) occupies the area of one pixel of the delivered image. (Not just that amount of area, but explicitly a particular pixel's real estate.) And all four regard the light falling on that pixel's "land".
The "R", "G", and "B" photodetector outputs from each such quad gives us a fairly-accurate indication of the light on that pixel spot. ("Fairly-accurate" not because of any shortcoming in this particular system, but for the same reason alluded to earlier: the spectral responses of the "R", "G", and "B" photodetectors don't really lead to that. In fact, theoretically, they can't!)
The language Canon uses to describe this is of course a creature of their life in the CFA world. They say that "the sensor is 3840 x 2160 pixels in size, but that the delivered image is 1920 x 1080 pixels in size".
Of course what they mean is that the sensor is 3840 x 2160 photodetectors in size. And it is 1920 x 1080 pixels in size. (We sometimes say "sensels" to help avoid perpetuating this misunderstanding.)
Now, exactly how does Canon arrange for the light that falls on one pixel's "real estate" on the sensor to be uniformly divided over the four "tenants"? Well, I don't know yet.
It very likely involves the same technique used to make the anti-aliasing filter used in many CFA sensors. There, each point of the image (as formed by the lens) is blown out to four points (on the face of the sensor) in a square pattern, with the vertical and horizontal spacing being approximately, if not exactly, the same as the detector spacing.
This is of course done by a "bulk" filter plate (called a "four spot filter" in the technical literature). That won't do for our purpose here; it would result in a given "R" detector receiving light from all across its "pixel of loyalty" and from some adjacent pixels as well - not part of the playbook here. So the doodad must be somehow partitioned. I will have my Canon patent guru look for evidence of how that is to be done.
Very likely, the optical device that does this is a combination of the microlens and the "four-spot filter" techniques.
Now what about an antialising filter - an "optical low-pass filter. Do we need one here? Well, in theory, we always need one. Well, isn't the "four spot filter" one? In reality, it isn't by itself a very complete one, and in our cameras other phenomena are called upon to complete the job. (The details are beyond this note.)
And its cutoff frequency is suitable for the pixel pitch of the CFA sensor array (which is the photodetector pitch). But the pixel pitch here is twice the photodetector pitch. So an anti-aliasing filter would have to have a different cutoff frequency that the one that comes from a "four-spot" filter plate.
Now, we have seen it said that the need for an antialising filter only comes from the use of the CFA approach (perhaps specifically from the use of demosaicing). That is not true. The cause of aliasing, or the artifacts it produces, are not creatures of the CFA approach.
But is it true that under the CFA approach the visible impact of aliasing can be far greater than in a true tristimulus camera (or a monochrome camera), because of the working of demosaicing.
Thus, going in the other direction, in a true tristimulus camera (such as one with a Foveon sensor), we may be able to dispense with an overt antialising filter. (I understand that there is indeed none in the familiar Foveon sensor cameras.)
Bur we still have working for us the mysterious "other phenomena" to which I alluded above, which lend a hand in the "anti-alising war". And the visual impact of aliasing is not aggravated in a true tristimulus camera as it is in a CFA camera.
But then shouldn't there be some visible artifacts of aliasing (for example, the classical "color moiré patterns) in the images from a Foveon sensor camera - maybe just nowhere near as bad as would be in a CFA camera with no explicit anti-aliasing filter?
Well, some workers have in fact reported the presence of such artifacts. (I have not examined their reports.)
Well, I've gotten of a little off to the side. Back to the C300: does it have an overt anti-aliasing filter, or is that explicit functionality somehow built into the gadget that does the distribution of the light on a pixel over the four "tenant" photodetectors? I just don't know.
In any case, the whole notion looks like a step forward.
Best regards,
Doug
As an aside, we must be careful not to read too much into these three arbitrary designations. In particular, the set of output values from the three photodetectors regarding a certain scene "spot" are not the R, G, and B coordinates of the color of the spot in any actual color space we use, not really even close. The set of three values, nevertheless, gives a "fairly-accurate" (in most cases) indication of the color of the spot. Its actual R, G, and B, coordinates can then be derived from those sensor "R", "G", and "B" values.
In a "three chip" color camera (many video cameras are of this type), the three photodetectors that regard a pixel of the image are in different arrays, the three arrays being each provided their own copy of the image through beam splitting arrangements. (The filters that confer the different spectral responses to the three classes of detector work "in bulk" as part of the three paths.)
In a color camera with a "Foveon" sensor, the three photodetectors that regard a pixel of the image are stacked at the same site on a single array. The top two of them are "semitransparent", so that all three receive light from the illumination on that spot (much as with the three layers on typical color film).
But in Canon cameras to date, a color filter array (CFA) sensor is used. This is not a true tristimulus sensor.
In it, the collection of "R", "G", and "B" detectors are interleaved across the array, at locations that correspond to the pixels of the native delivered image. Thus each pixel of the image is not regarded by three detectors, but rather by only one - with one of the three responses.
By a clever "intelligent interpolation process" we speak of as demosaicing, the camera is able to make an intelligent estimate of what all three sensor values would be for each pixel of the image. Thus, in effect, a real, if not always accurate, tristimulus representation of the image is attained.
Of course, as I emphasize with my "qualified" language, the color rendition may in fact not be accurate.
In the newly-announced Canon C300 "Cinema EOS" camera, a true tristimulus sensor system is employed. The actual sensor array consists, just as in a CFA sensor, of "R", "G", and "B" detectors, interleaved across the image, but not one in each pixel location. In fact, a "quad" of four of them (yes, one "R", one "B", and two "G"s) occupies the area of one pixel of the delivered image. (Not just that amount of area, but explicitly a particular pixel's real estate.) And all four regard the light falling on that pixel's "land".
The "R", "G", and "B" photodetector outputs from each such quad gives us a fairly-accurate indication of the light on that pixel spot. ("Fairly-accurate" not because of any shortcoming in this particular system, but for the same reason alluded to earlier: the spectral responses of the "R", "G", and "B" photodetectors don't really lead to that. In fact, theoretically, they can't!)
The language Canon uses to describe this is of course a creature of their life in the CFA world. They say that "the sensor is 3840 x 2160 pixels in size, but that the delivered image is 1920 x 1080 pixels in size".
Of course what they mean is that the sensor is 3840 x 2160 photodetectors in size. And it is 1920 x 1080 pixels in size. (We sometimes say "sensels" to help avoid perpetuating this misunderstanding.)
Now, exactly how does Canon arrange for the light that falls on one pixel's "real estate" on the sensor to be uniformly divided over the four "tenants"? Well, I don't know yet.
It very likely involves the same technique used to make the anti-aliasing filter used in many CFA sensors. There, each point of the image (as formed by the lens) is blown out to four points (on the face of the sensor) in a square pattern, with the vertical and horizontal spacing being approximately, if not exactly, the same as the detector spacing.
This is of course done by a "bulk" filter plate (called a "four spot filter" in the technical literature). That won't do for our purpose here; it would result in a given "R" detector receiving light from all across its "pixel of loyalty" and from some adjacent pixels as well - not part of the playbook here. So the doodad must be somehow partitioned. I will have my Canon patent guru look for evidence of how that is to be done.
Very likely, the optical device that does this is a combination of the microlens and the "four-spot filter" techniques.
Now what about an antialising filter - an "optical low-pass filter. Do we need one here? Well, in theory, we always need one. Well, isn't the "four spot filter" one? In reality, it isn't by itself a very complete one, and in our cameras other phenomena are called upon to complete the job. (The details are beyond this note.)
And its cutoff frequency is suitable for the pixel pitch of the CFA sensor array (which is the photodetector pitch). But the pixel pitch here is twice the photodetector pitch. So an anti-aliasing filter would have to have a different cutoff frequency that the one that comes from a "four-spot" filter plate.
Now, we have seen it said that the need for an antialising filter only comes from the use of the CFA approach (perhaps specifically from the use of demosaicing). That is not true. The cause of aliasing, or the artifacts it produces, are not creatures of the CFA approach.
But is it true that under the CFA approach the visible impact of aliasing can be far greater than in a true tristimulus camera (or a monochrome camera), because of the working of demosaicing.
Thus, going in the other direction, in a true tristimulus camera (such as one with a Foveon sensor), we may be able to dispense with an overt antialising filter. (I understand that there is indeed none in the familiar Foveon sensor cameras.)
Bur we still have working for us the mysterious "other phenomena" to which I alluded above, which lend a hand in the "anti-alising war". And the visual impact of aliasing is not aggravated in a true tristimulus camera as it is in a CFA camera.
But then shouldn't there be some visible artifacts of aliasing (for example, the classical "color moiré patterns) in the images from a Foveon sensor camera - maybe just nowhere near as bad as would be in a CFA camera with no explicit anti-aliasing filter?
Well, some workers have in fact reported the presence of such artifacts. (I have not examined their reports.)
Well, I've gotten of a little off to the side. Back to the C300: does it have an overt anti-aliasing filter, or is that explicit functionality somehow built into the gadget that does the distribution of the light on a pixel over the four "tenant" photodetectors? I just don't know.
In any case, the whole notion looks like a step forward.
Best regards,
Doug