John Sheehy
New member
I've created a model for determining the minimum needed levels to represent RAW data, with a custom LUT.
The basic idea is that, according to my tests, what you need in the RAW data is a next- and previous-levels from the one in question, where the standard deviation of noise at the central level in question is approximately 1.4x the distance to those steps. IOW, if the standard deviation is 100 photons, then the next level above and below should be about 71 photons away. That value is actually quite conservative, and covers any extreme pushing or boost of contrast in conversion; the number is overkill for low-contrast conversions.
Armed with that value, I create a spreadsheet where, on the first row, I start with the maximum number of photons in a given camera at a given ISO. On the same row, I calculate the shot noise for that many photons, and add it in quadrature to the (fixed) read noise in electrons for that ISO. Then, I divide this noise in photons/electrons by 1.4, and that is the step size, so I subtract that many photons from the total in the first row, for the number of photons in the second row. I calculate the total noise for that amount of photons, which is slightly less than the previous row, divide by 1.4, and subtract again, repeat, lather and rinse until there are no photons left. In the first row, at the right, I have the number 1, 2 in the second row, etc, to facilitate counting.
I plugged various numbers into the spreadsheet, based on full well capacities and read noises I knew from various high-end DSLRs down through my point-and shoots. I also made the 1.4 constant variable with a reference to easily change it for the entire spreadsheet. Not a single camera I thought of needed more than about 700 or 800 levels at base ISO. Photon count was far more important than relative read noise (and therefore, DR), which makes sense, when you consider the fact that altering read noise makes little effect in the midtone and highlight noise, which is photon-noise-dominated. The D2X, according to my spreadsheet, needs only 64 levels at ISO 1600. The FZ50 needs about 40 levels at ISO 1600. The 5D2 at ISO 25,600 needs 13 levels.
Here's what it looks like for the 5D2 at ISO 100. The max photons are 65,700, and the read noise is 25.6 electrons. We hit zero photons left at 650 levels:
This is not intended as a proof that 1.4 is sufficient; this just shows the ramifications if it is (I believe/know that it is, if the math is done properly).
Of course, there are distribution issues to look out for, but if they are properly balanced, there shouldn't be any issues.
The basic idea is that, according to my tests, what you need in the RAW data is a next- and previous-levels from the one in question, where the standard deviation of noise at the central level in question is approximately 1.4x the distance to those steps. IOW, if the standard deviation is 100 photons, then the next level above and below should be about 71 photons away. That value is actually quite conservative, and covers any extreme pushing or boost of contrast in conversion; the number is overkill for low-contrast conversions.
Armed with that value, I create a spreadsheet where, on the first row, I start with the maximum number of photons in a given camera at a given ISO. On the same row, I calculate the shot noise for that many photons, and add it in quadrature to the (fixed) read noise in electrons for that ISO. Then, I divide this noise in photons/electrons by 1.4, and that is the step size, so I subtract that many photons from the total in the first row, for the number of photons in the second row. I calculate the total noise for that amount of photons, which is slightly less than the previous row, divide by 1.4, and subtract again, repeat, lather and rinse until there are no photons left. In the first row, at the right, I have the number 1, 2 in the second row, etc, to facilitate counting.
I plugged various numbers into the spreadsheet, based on full well capacities and read noises I knew from various high-end DSLRs down through my point-and shoots. I also made the 1.4 constant variable with a reference to easily change it for the entire spreadsheet. Not a single camera I thought of needed more than about 700 or 800 levels at base ISO. Photon count was far more important than relative read noise (and therefore, DR), which makes sense, when you consider the fact that altering read noise makes little effect in the midtone and highlight noise, which is photon-noise-dominated. The D2X, according to my spreadsheet, needs only 64 levels at ISO 1600. The FZ50 needs about 40 levels at ISO 1600. The 5D2 at ISO 25,600 needs 13 levels.
Here's what it looks like for the 5D2 at ISO 100. The max photons are 65,700, and the read noise is 25.6 electrons. We hit zero photons left at 650 levels:
This is not intended as a proof that 1.4 is sufficient; this just shows the ramifications if it is (I believe/know that it is, if the math is done properly).
Of course, there are distribution issues to look out for, but if they are properly balanced, there shouldn't be any issues.
