On the "resolution" of a photographic system - again

The original concept of resolution of an optical system

The concept of the resolution of an optical system goes back to astronomical telescopes (a special kind of camera). There, in an important part of their use, the objects are of infinitesimal size - points of light. An important property of a telescope is its ability to let the observer recognize, as separate, two point objects at a very small angular spacing. This is said to be the ability to "resolve" those objects.

The resolution of a general-purpose camera

Although this notion can be applied to a general-purpose photographic system, it does not fit our most common interest. There, we are typically concerned with the ability of the system to portray in the image fine detail in the scene. But just what that means is not a simple matter. And it is not straightforward to craft a numerical metric which, in one number, will express how well the system does that for us.

In any case, by (inexact) parallel to the astronomical telescope property, we speak quantitatively of this ability as the resolution of our photographic system (or of the images it delivers).

One approach that has been (and still is) widely used is to imagine a test scene comprising alternate light and dark bands (same width for light and dark), and reduce the spacing of the pattern until, in the image, one can no longer see a pattern of light and dark, but only a uniform gray. That spacing of the bands is then said to be the resolution of the entire photographic system.

There are several conventions used to do this. One of the least ambiguous is to consider the joint width of a dark band and the adjacent light band to be a "line pair". If we are working with, for example, a camera negative, we can then state the resolution in line pairs per millimeter. To recognize that the image on the negative may be reproduced for the viewer as a print of varying size, we can state the resolution in line pairs per picture height.

Note that in terms of the spatial frequencies contained in a scene pattern, for a pattern of f line pairs per picture height, the fundamental frequency is f cycles per picture height.

Note that there is some uncertainty as to exactly at what spacing of the pattern can the pattern "no longer be seen".

The modulation transfer function (MTF)

The actual behavior of the photographic system that leads to a certain resolution (as defined above) being observed is the spatial frequency response of the system, which we speak of as its modulation transfer function (MTF). If we have before us a plot of the MTF, can we determine from that, in some "scientific" way, what resolution (as described above) will be observed? Not really.

However, in many cases of interest, the resolution (determined from observation of line patterns), stated in line pairs per picture height, will be about the same as the spatial frequency (in cycles per picture height) at which the MTF drops to 10% of its value for low spatial frequencies.

The MTF of a digital photographic system is basically composed of these subordinate MTFs:

• The MTF of the lens proper (which reflects its aberrations)

• The MTF that represents the effects of diffraction

• The MTF of the antialising filter (if present)

• The MTF that represents the fact that out photodetectors do not pick up the illuminance of the optical image at a point but rather picks up the illuminance more-or-less averaged over a finite region (for example, the "intake port" of the photodetector microlens). This is spoken of as the "sampling aperture" MTF.

So, for example, the spatial frequency at which the MTF drops to 10% of its value at low spatial frequencies might be where

• The MTF of the lens was 30%

• The MTF from diffraction was 67%, and

• The sampling aperture MTF was 50%

• There is no antialising filter.

It is tempting to say, in a certain case, that "the resultion here is limited by the lens resolution". If in fact, for example, the lens MTF drops to 10% at a much lower spatial frequency than the other MTFs in the chain, that is a meaningful notion. But otherwise it is not that simple.

Or to say, "the resultion here is not limited by the lens resolution". If in fact, for example, the lens MTF drops to 10% at a much higher spatial frequency than the other MTFs, that is a meaningful notion. But otherwise it is not that simple.

The Nyquist limit

An upper bound on the possible resolution of an image in a digital system is its Nyquist frequency, which is the reciprocal of twice the pixel spacing. No digital image can convey a frequency component at or above the Nyquist frequency. Never. No way, no how.

If there are, in the optical image being "sampled" by the sensor, any frequency components at or above the Nyquist frequency, they will not be represented in the digital image. But that image will, as a result of the presence of those "out of range" frequencies in the image being sampled, contain spurious frequencies (below the Nyquist frequency) not present in the optical image. This is the phenomenon of aliasing.

Of course, if the amplitude of these "out of range" components is small, then the spurious frequencies they morph into in the digital image will be small, and will not consequentially degrade the image.

In order to keep the amplitudes of any "out of range" components "low", we rely on the system MTF reaching a low value as we approach the Nyquist frequency. In many cases, the combination of lens MTF, diffraction MTF, and aperture MTF will bring this about. If not, we may need to add an antialising filter, the decline of whose MTF as frequency increases will complete the needed suppression of "out of range" frequencies.

Note that when we use a CFA sensor, the Nyquist frequency for the R and B aspects is half that of the final digital image. (For the G aspect, it is the same as that of the final digital image.) How can we then suppress "out of range" frequency components (to avert aliasing) without reducing the resolution to substantially less that the one the Nyquist frequency of the image would allow? Well, we can't really.

Time for breakfast.

Best regards,

Doug

Here we will do a little exercise, actually related to something many of us go through.

We have interchangeable lens camera A, which includes a certain lens. It has a sensor with Na sensels. The resultion of this camera is reported to be Ra line pairs per picture height.

We contemplate getting a new body, with a sensor of the same size but with 1.5 times the number of sensels in the v and h directions (2.25 times the total sensel count). We assume the same sampling aperture size in terms of the sensel pitch, and assume that the response of the antialising filter is scaled to match the Nyquist frequency of the new sensor. We will use the same lens on the new body (making camera B). Assume operation in all cases at an f-number such that the effects of diffraction can be essentially ignored.

Now, what might we expect to be the resolution of the new camera (Rb)?

Well, firstly, the Nyquist frequency for the overall digital image is 1.5 times what it was for camera A, and this means an upper limit on the possible resolution of 1.5 times of the upper limit for camera A.

Now, we look at two extreme cases:

Case 1

The frequency at which the lens MTF drops to 10% (MTF10) is much less than the Nyquist frequency of camera A. We can fairly say that the resolution we enjoy in camera A is "lens limited".

In this case, we can expect a resolution with camera B not much different than that with camera A. We could fairly (if simplistically) say that in camera B, the resolution is still "lens limited".

Case 2

The frequency at which the lens MTF drops to 10% (MTF10) is over 1.5 times the Nyquist frequency of camera A (and thus over the Nyquist frequency of camera B). We can fairly say that the resolution we enjoy in camera A is not "lens limited".

In this case, we can expect a resolution with camera B of almost 1.5 times that with camera A. We could fairly (if simplistically) say that in camera B, the resolution is not "lens limited".

Now of course for in-between cases the result would be — in between those results.

Time now to do my physical therapy exercises. I am fighting a long-lasting and rather debilitating pain in right leg believed (not definitively) to be due to some injury to my right piriformis muscle. We try to cure this injury to the muscle by repeatedly and frequently abusing it in a scientifically-planned way.

Best regards,

Doug

Doug Kerr said:

Time now to do my physical therapy exercises. I am fighting a long-lasting and rather debilitating pain in right leg believed (not definitively) to be due to some injury to my right piriformis muscle. We try to cure this injury to the muscle by repeatedly and frequently abusing it in a scientifically-planned way.

Best regards,

Doug

Click to expand...

What did the sadist say to the masochist?
"Hit me"
The masochist replied: "No"

I empathize with your discomfort, Doug. A symptom of the aging process no less. But don't take it out on us. I'm feeling enough mental pain trying to figure out all this stuff your posting. You should take up something less weighty in your age of enlightenment. PG Wodehouse is a good read.

Hi, Tom,

Tom dinning said:

What did the sadist say to the masochist?
"Hit me"
The masochist replied: "No"

Click to expand...

Yes, one of my favorites!

I often render it: "The masochist replied (with a sneer): 'No'."

I empathize with your discomfort, Doug. A symptom of the aging process no less. But don't take it out on us. I'm feeling enough mental pain trying to figure out all this stuff your posting.

Click to expand...

Misery loves company!

Please note the caveat at the head of this forum section.

I'm afraid you may have disqualified yourself from the prospect that Asher would, on any day one of my technical diatribes appears in this section, refund the part attributable to that day of what you pay to subscribe to OPF.

You should take up something less weighty in your age of enlightenment. PG Wodehouse is a good read.

Click to expand...

Thanks for your encouragement.

Best regards,

Doug

Now we will do a similar exercise with a more specialized configuration.

Various high-performance digital backs (mostly of the MF size range) offer what is known as multishot technique, and often an advanced form called microstep multishot technique.

A popular form of the basic multishot technique is called "4X multishot". There, the image formed by the lens is captured by a CFA sensor four times in rapid succession. Between each of the exposures the sensor is moved, in the x and then y direction, by the photodetector pitch. The result is that each point in the image that will become a pixel of the digital image is visited in sequence by an R photodetector, two G's, and a B.

In effect, we have created a virtual "trichromatic" sensor with photodetector pitch equal to the photodetector pitch of the real sensor (and the pixel pitch of the digital image). Thus no CFA interpolation is needed to develop the full-color digital image.

Note before we move on that the Nyquist frequency is the same for the R, G, and B "aspects" of the sampling process, and is the same as that of the digital image itself.

This is advantageous for a couple of reasons:

• No CFA interpolation is required.

• We have the potential for chromaticity resolution on a par with the luminance resolution (and thus approaching the limit for the pixel dimensions of the digital image).

Now we move to the more advanced form of the technique, in a common form called "16X microstep multishot technique".

Here, we actually sample the optical image 16 times, moving the sensor in x and y increments of half the photodetector pitch. Thus four times as many points in the image as before are now each visited by an R detector, two Gs, and a B. In effect, we have created a virtual true trichromatic sensor whose "pixel pitch" is half the photodetector pitch of the actual sensor. From this we get, without any need for CFA interpolation, a digital image with four times the pixel count we had with the "4X" technique (or which we would have using the sensor in the normal CFA scheme).

Thus, the Nyquist frequency of the system, for all three chromatic aspects, and for the overall digital image, is twice that of the "4X" technique.

Thus the limit of possible resolution (imposed by the Nyquist limit), for all three chromatic aspects, as well as for the final digital image, is twice that of the 4X technique.

We begin in case A, in which our CFA sensor is operated in 4X multishot mode. The Nyquist limit for all three chromatic aspects (R, G, and B), and for the digital image itself, is the reciprocal of twice the photodetector pitch.

Thus, dependent upon the cooperation of the lens' MTF and the other MTF's involved, we have the potential of resolution (both luminance and chromaticity) approaching that Nyquist frequency.

Now in case B, we change to 16X microstep multishot mode, the same sensor and lens being in place.

Now, the pitch at which the optical image is samples is half that of case A. As a consequence, the Nyquist frequency is twice what it was in case A.

What about resolution? Well, as before, we consider two extreme cases:

Case 1

The frequency at which the lens MTF drops to 10% (MTF10) is much less than the Nyquist frequency of case A. We can fairly say that the resolution we enjoy in case A is "lens limited".

In this case, we can expect a resolution in case B not much different than that for case A. We could fairly (if simplistically) say that in case B, the resolution is still "lens limited".

Case 2

The frequency at which the lens MTF drops to 10% (MTF10) is over 2 times the Nyquist frequency of of case A (and thus over the Nyquist frequency for case B). We can fairly say that the resolution we enjoy in Case B is not "lens limited".

In this case, we can expect a resolution in case B of almost perhaps 2 times that for case A. We could fairly (if simplistically) say that in case B, the resolution is not "lens limited".

Now of course for in-between cases the result would be — in between those results.

Case C

Now in case C, we actually use a sensor with four times as many sensels as the one used in cases A and B (the sensel pitch being half that of the other sensor). Assume that in it, the sampling apertures are the same as for the other sensor in relation to the photodetector pitch. We operate it in "4X" mode, so as to get the advantages of a virtual "true trichromatic sensor".

What might we expect to be the resultion we will then attain, compared to case B?

Well, the sampling pitch will be the same as in case B. Thus the Nyquist frequency, which is the upper limit on the attainable resolution, will be the same as for case B.

But there is a difference between the overall system MTF for the two cases.

In case B, if the sampling aperture width is indeed the same as the photodetector pitch (the largest we can actually make it, physically), it becomes twice the sample pitch (owing to the "microstep" technique). This means a fairly severe decline in the sampling aperture MTF as we approach the Nyquist frequency, a factor that detracts from the attainable resolution.

In case C, if the sampling aperture width is indeed the same as the photodetector pitch, it is equal to the sample pitch (owing to the "microstep" technique). This means a less severe decline in the sampling aperture MTF as we approach the Nyquist frequency.

Thus, we have the expectation of a lower resolution for case B (the microstep technique) than for case C (sensor with four times the pixel count).

Now, given the likely cost of a sensor with four times the number of pixels, the microstep technique seems like a pretty good bargain.

Note that in cases B and C, the implications of the lens MTF on the resultion that is attained are the same.

Best regards,

Doug

Now, it has been asserted by a colleague that (and I necessarily paraphrase):

• The resolution we can attain in case B is greater than in case C (at least for certain lens MTFs).

• The attained resolution in Case B is not as dependent on the limiting effect of the lens MTF as in case C. In fact, the intimation is that by using the technique of case B, we can attain resolution in the digital image beyond that in the actual optical image (the latter of course being largely "limited" by the lens MTF). Thus, by using microstep technique, we can to some extent "evade" the limitation on image resolution imposed by the lens' MTF.

I am unable to construct any model of the process which, in light of the well-understood concepts of sampling theory and the interaction of MTFs, would make this result possible.

Best regards,

Doug

Tom dinning said:

What did the sadist say to the masochist?
"Hit me"
The masochist replied: "No"

Click to expand...

Wasn't it the other way 'round (masochist asking sadist)?

Easy to remember when you think of the two persons responsible for these words:
Leopold von Sacher-Masoch and Donatien Alphonse François, Marquis de Sade.

Resolution (no, not this one) is an important element for a picture, but not everything as far as I am concerned.

Best regards,
Michael

Hi, Michael,

Michael Nagel said:

Wasn't it the other way 'round (masochist asking sadist)?

Click to expand...

Oh, of course. (I told you I am off my game these days!)

Thanks.

Best regards,

Doug

This is further to the matter of the "microstep" form of multishot technique.

In what follows, I will assume that in each case we have applied the "X4" aspect of multishot technology such that we have in effect a true trichromatic sensor.

Case A

In this figure we show the operation of the microstep multishot technique. We do not see the basic X4 sampling, but rather show the equivalent trichromatic sensor.


The left-hand portion represents the sampling by the sensor. It occurs in four "phases", at time t1, t2, etc., the sensor being moved between them by one half the photodetector pitch.

For now, we will assume (here and in the following case) that the sensor is a "point sample" type; that it, its sampling aperture is of infinitesimal size. We use small (not infinitesimal!) dots to show the sampling locations.

Section a1 shows the sampling by the sensor at time t1, section a2 the sampling at time t2, and so forth. All the samples are gathered up to form the sample set for the digital image, seen on the right.

Its pixel pitch is half the detector pitch of the sensor. It has a certain Nyquist frequency, which is the upper limit on the resolution that could be attained in the digital image.

I have made the dot that shows a certain one of the sampling locations, and the corresponding sample of the complete sample set, purple, as I will tell a story about it.

Case B

In this case, we replace the original sensor with one having four times as many photodetectors (over the same sensor area).


Again I will assume that we again have applied the "X4" aspect of multishot technology such that we have in effect a true trichromatic sensor.

Here all the sampling is done "at once" (of course there were really four passes for the X4 aspect, but we don't illustrate that, illustrating a true trichromatic sensor).

We end up with the very same sample set, with the very same Nyquist frequency, as in case A.

The upper limit on attainable resolution (the Nyquist frequency) is thus the same in both cases.

And there is no difference in the sample sets. For example, consider the sampling location on the image shown here (and in the prior figure) in purple. Of course it has the same illuminance in both cases (we assume the very same optical image). The sample value (the purple dot in the digital image) is the same whether that point is examined in the "t1" phase of case A or in the single sampling phase in case B.

Best regards,

Doug

Michael Nagel said:

Wasn't it the other way 'round (masochist asking sadist)?

Easy to remember when you think of the two persons responsible for these words:
Leopold von Sacher-Masoch and Donatien Alphonse François, Marquis de Sade.

Resolution (no, not this one) is an important element for a picture, but not everything as far as I am concerned.

Best regards,
Michael

Click to expand...

On the money, Michael. I'm still recovering from the after effects of anaesthesia. Such a delight with the bite of a serpent. Excuse me while I fall over again.

Michael Nagel said:

Wasn't it the other way 'round (masochist asking sadist)?

Easy to remember when you think of the two persons responsible for these words:
Leopold von Sacher-Masoch and Donatien Alphonse François, Marquis de Sade.

Resolution (no, not this one) is an important element for a picture, but not everything as far as I am concerned.

Best regards,
Michael

Click to expand...

Who was it that said something about clear ideas and sharp pictures?

Now we will address the mysterious matter of the sampling aperture MTF. You will recall that the larger the size of the sampling aperture (compared to the sampling pitch), the faster does the overall system MTF decline with increasing spatial frequency.

We will first revisit case B: the use of the "fine pitch" sensor without microstep technique,


Considering our famous "purple" sampling point, the green square shows the sampling aperture (based on its being as large as we can physically make it—a "100% fill" design).

Now we will first revisit case A: the use of the "basic" sensor with microstep technique.


Again considering our famous "purple" sampling point, the green square shows the sampling aperture (again based on its being as large as we can physically make it—a "100% fill" design—on the physical sensor).

But this sensor has twice the physical pitch of the one above. So the size of the sampling aperture is now twice as large as before.

So what does that do?

It makes the overall system MTF for Case A (microstep multishot technique) decline more rapidly than for case B (no microstep, finer pitch sensor).

What does that not do?

There is seemingly afoot in certain circles the misconception that a larger sampling aperture "gathers more information" from the optical image (because it spans a greater area of it).

But in fact each sample result—regardless of the size of the sampling aperture—is just a single number. How can that carry more or less information about the image?

We get "more information" about the image only by collecting more samples—samples at a finer spacing. The size of the sampling aperture doesn't figure into that notion.

Best regards,

Doug

Doug Kerr said:

But in fact each sample result—regardless of the size of the sampling aperture—is just a single number. How can that carry more or less information about the image?

We get "more information" about the image only by collecting more samples—samples at a finer spacing. The size of the sampling aperture doesn't figure into that notion.

Click to expand...

As you have indicated, small sampling always is better as long as one is not aperture limited!

A larger sampling aperture means that proportionally less of the light wave front is scraped against the edges of that aperture. So there are less ripples in the light landing at a hoped-for focus point! The finer the spaces in the sensor, the more it's beneficial to have an open aperture and not have overlapping waves smearing detail limiting the resolution that would otherwise be realized!

Asher

In this exercise, the examples actually fit in with the story of microstep multishot technique, and you may wish to think of it in those terms. But the concept is perfectly general.

We will work from this figure:


We will assume a monochrome system.

The red outline represents a tiny optical image (perhaps really a small region of a larger one). In case a1, we sample it, at the points indicated by the little dots, at a pitch of p1.

The Nyquist frequency of this sampling process (fN1) is 1/(2p1).

We will assume that there are no components in the variation of illuminance on the image whose spatial frequencies are at or above that Nyquist frequency.

The set of sample data comprises:

• nine numbers.

But that suite of nine numbers:

• completely and exactly describes the illuminance all over the image—at every place on the image, no matter closely separated are the places we consider.

In case a2, we sample this very same image, again with pitch p1, but with the locations of the sampling points offset from where they were in case a1.

The set of sample data comprises:

• nine numbers. These are almost certainly different from the nine numbers in case a1.

But that suite of nine numbers:

• completely and exactly describes the very same image as the suite of data in case a1.

So, is the information in data set a2 the same as in data set a1?

Well, seen as a set of numbers, not the same at all.

Seen as a complete and exact description of the illuminance over a rectangular image, exactly the same!

In cases a3 and a4, we sample with the sample points in and third and fourth position. In these cases, the suites of nine numbers are different from before. But in each case, the suite of data completely and exactly describes the very same image.

What if, contrary to our stipulation, there are components in the variation of luminance across the image whose frequencies are above the Nyquist frequency for this sampling process?

Well, then:

• None of the four suites of data describe the original image. (They are corrupted by aliasing.)

• The corrupted images they describe may be different.

Now, in case b, we sample our same little image at the points indicated, with a pitch, p2, that is half of the pitch in the "a" cases (p1). Accordingly, the Nyquist frequency here (fN2) is twice that of the "a" cases.

We might do this with a sensor with detectors at pitch p2. Or we might just consolidate the the four suites of data from cases a1-a4.

The result is identical between the two possibilities, and is:

• The set of 36 values completely and exactly describes the little image.

• But the process could now do this if the illuminance variation in the image did contain components whose frequencies were above fN1 (but not at or above fN2).

Suppose that, as originally assumed, there were no components whose frequencies were at or above fN1. Then does sample suite "b" better describe the little image than sample suites "a1", or "a2", or "a3", or "a4"?

No. They all completely and exactly describe the image (every place on it).

Ain't science grand!

Next: Father Barbour enjoys his favorite dessert*, despite tensions in the family.


Best regards,

Doug

About Father Barbour's Dessert

Father Barbour was the patriarch of a large and complicated family whose life was chronicled in the long-running and iconic radio serial drama, "One Man's Family".

His favorite dessert, bread with butter and applesauce drenched in heavy cream, was actually only mentioned in one episode. It figured as an offering of reconciliation after a period of family tension. But it was occasionally referenced in advertising (I think for a creamery company, but I don't remember for certain)—"Make Father Barbour's favorite dessert tonight".

In any case, when I was very young, my mother would occasionally say, "Tonight, should we have Father Barbour's Dessert?" I enjoyed it so very much, not just because it was so very tasty, but as well because it was part of a very comforting tradition - more for the Kerr family than for the Barbour family.

I had a wonderful childhood, a good launch for a wonderful life.

Best regards,

Doug

Excellent work, Doug and kudos for the drawing skills! I'd encourage folks to reread this.

Asher

Asher Kelman said:

Sampling at different phases​

Excellent work, Doug and kudos for the drawing skills! I'd encourage folks to reread this.

Asher

Click to expand...

Is that so I can not understand it the second time?
On my first read, I was too frightened to pick up my camera for fear of lightning striking me.
Then I turned on the studio lights today and a capacitor in the main light blew up. Then a storm passed overhead and the tree out the front was hit by lightning.
Who do you know I don't, Doug?
I'm sleeping under the bed tonight.
And I'll promise I'll never say another word against you beautiful and informative posts, Doug.
Please forgive me!
Your eternal servant, Tom

Hi, Tom,

Do you not read the masthead for this forum section?

Here it is, so you don't have to scroll up for it (emphasis added):

Tom dinning said:

Who do you know I don't, Doug?

Click to expand...

Well, tens of thousands of people.

I'm sleeping under the bed tonight.

Click to expand...

Well, be thankful that dust bunnies, after they copulate, don't fall over and giggle like real bunnies.

Will Christine make Father Barbour's Dessert for you?

Best regards,

Doug

Hi, Asher,

Asher Kelman said:

Excellent work, Doug and kudos for the drawing skills! I'd encourage folks to reread this.

Click to expand...

Thanks so much.

In fact, one of the motivations for this series was for me to hone my basic skills with CorelDraw. I had for many years used Micrografx Designer as my technical illustration tool. But for various reasons, I decided to shift to CorelDraw. But it takes a lot of getting used to, eh?

For example, if I want to deposit a text label (loose text, not in a container, called "artistic text") in 12-point Arial with the Text tool, I can't just set the font to 12-point Arial and shoot. I must set the "default" font for artistic text to 12-point Arial (fairly deep in an obscure menu) and then shoot.

If I want to grab a rectangle by one of its corners and drag the corner to a point on the grid (to align the rectangle to the grid), I have to not only turn on "snap to grid" (makes sense) but also must turn on "snap to objects", else my grabber will not really grab the corner but maybe some place very close to it, and move that point on the rectangle to a point on the grid. But, with "snap to objects" turned on, the corner will try and snap to various existing object features as well as to the grid, not handy in a crowded area of the drawing.

Aargh, eh?

Best regards,

Doug

By the way, here (from our eclectic archive) is the episode of One Man's Family (first run 1949.09.12) in which the special dessert is mentioned:

http://dougkerr.net/audio/one_mans_...emade_bread_and_applesauce_peace_offering.mp3

It runs 29:02.

Best regards,

Doug