Doug Kerr
Well-known member
Representation by sampling
In the classical model of the representation of a continuous variable (such as the voltage of an audio waveform) by sampling, we capture, at regular intervals, the instantaneous value of that variable. From this series of values, we can precisely reconstruct the entire original "waveform" if that waveform contains no frequency components whose frequencies are at or above half the frequency at which we take the samples.
When we do this electrically, we in effect open a gate for an infinitesimal time at every sampling instant to allow the instantaneous voltage right then through to be captured. Of course, is the gate is open. We can still capture it with, for example, and amplifier followed by a storage capacitor. We can then "measure" this voltage at our leisure. (In a digital system, we will in fact develop a digital description of the value and "send that on" to the distant end for use in reconstructing the original waveform.)
Suppose that for whatever reason, we open the gate each time for a finite duration, and that the circuit is such that the value that is "captured" is the average of the voltage over that duration. We can show that this is precisely equivalent to the original case (with an infinitesimal sampling time) but where the waveform is first passed thorough a certain type of "low-pass" filter. The frequency response of that filter is exactly that of a filter whose time response is a "spread function" that is constant for the sampling duration (and we can determine that response by taking the Fourier transform of a "rectangular pulse" of that duration).
That frequency response turns out to have the classical "sine x over x" form. The parameter x here is frequency, scaled by a factor based on the sampling duration.
One implication of doing this is that, after we "reconstruct the"original" waveform from the samples (perhaps conveyed in digital form), the signal as been afflicted by a "drooping" frequency response. Thus, we must use an "post-reconstruction equalizer" that restores all the frequency components to their original relative magnitude (else the restored audio waveform will have a "high-frequency rolloff").
This phenomenon is often called "aperture distortion", where "aperture" refers to the sampling process as a window (aperture) of finite time width through which the signal voltage is observed.
Aliasing
We were reminded earlier that for the sampling process to work properly it is necessary that the waveform being sampled contain no frequency components at or above half the sampling rate (a frequency limit called the "Nyquist frequency"). If this is not true, the "overfrequency" components are not just lost. Rather, they appear in the reconstructed waveform as components whose frequency is as far below the Nyquist frequency as their original frequency was above it. These spurious components are a source of "corruption" in the reconstructed waveform.
This phenomenon can be called "foldover distortion" (because of what happens to the frequency of the component, which is "folded" about the Nyquist frequency)s or, commonly, "aliasing".
The premise of that name is this. If the sampling rate is 8000 kHz, and thus the Nyquist frequency is 4000 Hz, then a component of the original signal at 4100 Hz will have exactly the same representation in the train of samples as a component with a frequency of 3900 Hz. That is, it travels "with the papers of" a 3900 Hz component - it travels under an "alias".
To avert this, we in general must filter out of the signal, before it is samples, all frequency components at or above the Nyquist frequency . The lowpass filter used for this purpose is often called an "antialising filter".
If we have a finite "sampling window", then the aperture distortion effect in effect provides us with the equivalent of a pre-sampling lowpass filter. Can that be our antialiasing filter? Not actually. The response of this virtual filter does not decline rapidly enough (and after, at a certain frequency, it drops to zero, it then rises again).
Still, in many cases of digital audio, we use a less-than-ideal antialising filter for many reasons, and it is not out of the question that, to simplify system design, we might use a very wide sampling duration and accept the resulting virtual lowpass filter as our "antialising filter".
Now, to digital photography
Of course, digital photography also involves the representation of a continuous phenomenon (in this case, the color of the image) with discrete samples. The principles I discussed above all basically apply, but there are many matters that change the details of their implications.
Let us for a moment assume a "monochrome" sensor.
The matter of aliasing is still with us. If in fact the variation of luminance (the only aspect of the color of the image that is pertinent, given that I have assumed a monochrome sensor) contain spatial frequency components at spatial frequencies higher than the spatial Nyquist frequency (determined by the pitch of the sensor grid), then the delivered image will have luminance components not in the original image. If the image also has components not too far below the Nyquist frequency, then the improperly-reconstructed components will interact with these other components to produce, visually, "beats" (which we describe as moiré patterns).
To avert this, we would apply some type of optical lowpass filter (working in the spatial frequency domain) before the image is sampled by the sensor array.
Our sampling aperture
Our sampling organ is the array of detectors. To attain theoretically ideal sampling, each would have to only respond to the illuminance over an infinitesimal region of the image. But then the luminous energy captures would be infinitesimal. We cannot amplify it before by the photodetector, so this will not work.
So we in fact strive (with such tools as microlenses) to equip each detector to accept luminous energy from as large a region of the image as possible, usually a region that is nearly as large as suggested by the spacing between photodetectors. This is a very large sampling aperture.
This of course gives us a serious "aperture distortion" problem, in fact the major cause of the decline in the MTF of the sensor itself with increasing spatial frequency.
But, so it shouldn't be a total loss, can this declining spatial frequency response in fact be used as our antialising filter? Its response is hardly ideal, but then we get it free.
Well, just as in simple digital audio systems, an ill-suited lowpass filter may be better than none. And thus in fact, in many monochrome cameras, there is no overt antialiasing filter. We just use the frequency response caused by our "large sampling aperture" to do that for us, such as it is.
Next: with a CFA sensor.
Best regards,
Doug
In the classical model of the representation of a continuous variable (such as the voltage of an audio waveform) by sampling, we capture, at regular intervals, the instantaneous value of that variable. From this series of values, we can precisely reconstruct the entire original "waveform" if that waveform contains no frequency components whose frequencies are at or above half the frequency at which we take the samples.
When we do this electrically, we in effect open a gate for an infinitesimal time at every sampling instant to allow the instantaneous voltage right then through to be captured. Of course, is the gate is open. We can still capture it with, for example, and amplifier followed by a storage capacitor. We can then "measure" this voltage at our leisure. (In a digital system, we will in fact develop a digital description of the value and "send that on" to the distant end for use in reconstructing the original waveform.)
Suppose that for whatever reason, we open the gate each time for a finite duration, and that the circuit is such that the value that is "captured" is the average of the voltage over that duration. We can show that this is precisely equivalent to the original case (with an infinitesimal sampling time) but where the waveform is first passed thorough a certain type of "low-pass" filter. The frequency response of that filter is exactly that of a filter whose time response is a "spread function" that is constant for the sampling duration (and we can determine that response by taking the Fourier transform of a "rectangular pulse" of that duration).
That frequency response turns out to have the classical "sine x over x" form. The parameter x here is frequency, scaled by a factor based on the sampling duration.
One implication of doing this is that, after we "reconstruct the"original" waveform from the samples (perhaps conveyed in digital form), the signal as been afflicted by a "drooping" frequency response. Thus, we must use an "post-reconstruction equalizer" that restores all the frequency components to their original relative magnitude (else the restored audio waveform will have a "high-frequency rolloff").
This phenomenon is often called "aperture distortion", where "aperture" refers to the sampling process as a window (aperture) of finite time width through which the signal voltage is observed.
Aliasing
We were reminded earlier that for the sampling process to work properly it is necessary that the waveform being sampled contain no frequency components at or above half the sampling rate (a frequency limit called the "Nyquist frequency"). If this is not true, the "overfrequency" components are not just lost. Rather, they appear in the reconstructed waveform as components whose frequency is as far below the Nyquist frequency as their original frequency was above it. These spurious components are a source of "corruption" in the reconstructed waveform.
This phenomenon can be called "foldover distortion" (because of what happens to the frequency of the component, which is "folded" about the Nyquist frequency)s or, commonly, "aliasing".
The premise of that name is this. If the sampling rate is 8000 kHz, and thus the Nyquist frequency is 4000 Hz, then a component of the original signal at 4100 Hz will have exactly the same representation in the train of samples as a component with a frequency of 3900 Hz. That is, it travels "with the papers of" a 3900 Hz component - it travels under an "alias".
To avert this, we in general must filter out of the signal, before it is samples, all frequency components at or above the Nyquist frequency . The lowpass filter used for this purpose is often called an "antialising filter".
If we have a finite "sampling window", then the aperture distortion effect in effect provides us with the equivalent of a pre-sampling lowpass filter. Can that be our antialiasing filter? Not actually. The response of this virtual filter does not decline rapidly enough (and after, at a certain frequency, it drops to zero, it then rises again).
Still, in many cases of digital audio, we use a less-than-ideal antialising filter for many reasons, and it is not out of the question that, to simplify system design, we might use a very wide sampling duration and accept the resulting virtual lowpass filter as our "antialising filter".
Now, to digital photography
Of course, digital photography also involves the representation of a continuous phenomenon (in this case, the color of the image) with discrete samples. The principles I discussed above all basically apply, but there are many matters that change the details of their implications.
Let us for a moment assume a "monochrome" sensor.
The matter of aliasing is still with us. If in fact the variation of luminance (the only aspect of the color of the image that is pertinent, given that I have assumed a monochrome sensor) contain spatial frequency components at spatial frequencies higher than the spatial Nyquist frequency (determined by the pitch of the sensor grid), then the delivered image will have luminance components not in the original image. If the image also has components not too far below the Nyquist frequency, then the improperly-reconstructed components will interact with these other components to produce, visually, "beats" (which we describe as moiré patterns).
To avert this, we would apply some type of optical lowpass filter (working in the spatial frequency domain) before the image is sampled by the sensor array.
Our sampling aperture
Our sampling organ is the array of detectors. To attain theoretically ideal sampling, each would have to only respond to the illuminance over an infinitesimal region of the image. But then the luminous energy captures would be infinitesimal. We cannot amplify it before by the photodetector, so this will not work.
So we in fact strive (with such tools as microlenses) to equip each detector to accept luminous energy from as large a region of the image as possible, usually a region that is nearly as large as suggested by the spacing between photodetectors. This is a very large sampling aperture.
This of course gives us a serious "aperture distortion" problem, in fact the major cause of the decline in the MTF of the sensor itself with increasing spatial frequency.
But, so it shouldn't be a total loss, can this declining spatial frequency response in fact be used as our antialising filter? Its response is hardly ideal, but then we get it free.
Well, just as in simple digital audio systems, an ill-suited lowpass filter may be better than none. And thus in fact, in many monochrome cameras, there is no overt antialiasing filter. We just use the frequency response caused by our "large sampling aperture" to do that for us, such as it is.
The same if true, for the same reason, of many "true tricolor sensor" cameras, such as video cameras with three sensors, or a still camera with a sensor such as the Foveon type.
Next: with a CFA sensor.
Best regards,
Doug