Tool for examing Canon raw values?

[Doug Kerr]

I have so far rarely shot to the raw output, and I am not very familiar with the tools in that area.

From time to time I am interested in examining the data in a raw output file (Canon .CR2, mostly), perhaps to see the values at individual photosites but also to see the distribution as a histogram (and I guess I would really want to see it in a histogram "by channel").

Can anybody recommend a good software tool for this? It might even be something I have, and don't know it.

Some while ago I did some of this with Canon's DPP. In the version I had at the time, the raw histogram display could have its horizontal axis set to work in "raw units" (0-4095), but that seemed to disappear in the next version. Now, it seems to be in an arbitrary scale in "stops", and I don't know what the origin is or even the values at the ends.

Thanks for any help.

[John_Schwaller]

Hi, Doug....

I have been shooting only RAW for many years. I'd never consider going back to jpeg....not just only...at all.

I am not sure what you are looking for, but you probably have the best chance nosing around DCRAW.

Here are a couple possible starting points:
http://www.cybercom.net/~dcoffin/dcraw/
http://www.insflug.org/raw/software/tools/dcraw.php3

John

[John Sheehy]

Originally Posted by John_Schwaller

Hi, Doug....

I have been shooting only RAW for many years. I'd never consider going back to jpeg....not just only...at all.

I am not sure what you are looking for, but you probably have the best chance nosing around DCRAW.

DCRAW only outputs the true RAW data in "document mode" with the "-D" option, but unless it has changed recently, this output is not useful in photoshop, because DCRAW uses the least significant bits of the 16-bit output for the RAW data, and photoshop is not really 16-bit, and discards the LSB.

[John_Schwaller]

Originally Posted by John Sheehy

DCRAW only outputs the true RAW data in "document mode" with the "-D" option, but unless it has changed recently, this output is not useful in photoshop, because DCRAW uses the least significant bits of the 16-bit output for the RAW data, and photoshop is not really 16-bit, and discards the LSB.

I thought Doug's request was tools to analyze the RAW data, not input it to PS. I thought he might find it (a tool) around discussions on DCRAW. I vaguely remember someone (on some unremembered forum) discussing a tool which was able to work with raw data, but it was too far back...and too little interest on my part.

Actually...the "-D" option might be what he wants...he'll have to let us know.

John

[Bart_van_der_Wolf]

Originally Posted by Doug_Kerr

From time to time I am interested in examining the data in a raw output file (Canon .CR2, mostly), perhaps to see the values at individual photosites but also to see the distribution as a histogram (and I guess I would really want to see it in a histogram "by channel").

Can anybody recommend a good software tool for this?

Two applications come to mind (there may be others as well), both are mainly intended to handle astronomical imagery, although they both come with a personal concern I have with the blackpoint selection.
IRIS is a free tool, and ImagesPlus is a payware tool.

They both allow to inspect the Raw file data, down to the recorded sensel values before Bayer CFA demosaicing.

Bart

[Doug Kerr]

Hi, John,

Originally Posted by John_Schwaller

I thought Doug's request was tools to analyze the RAW data, not input it to PS.

Indeed.

I thought he might find it (a tool) around discussions on DCRAW. I vaguely remember someone (on some unremembered forum) discussing a tool which was able to work with raw data, but it was too far back...and too little interest on my part.

Actually...the "-D" option might be what he wants...he'll have to let us know.

I'll have to look into that. Thanks for the "tip".

Best regards,

Doug

[Doug Kerr]

Hi, Bart,

Originally Posted by Bart_van_der_Wolf

Two applications come to mind (there may be others as well), both are mainly intended to handle astronomical imagery, although they both come with a personal concern I have with the blackpoint selection.

Yes, I have used IRIS for the purpose. But there are many mysteries in it that make it hard for me to use it. Mainly, between uses, I forget how I have to set it up. In particular, the "threshold" stuff often confuses me.

Thanks.

Best regards,

Doug

[John Sheehy]

Originally Posted by Doug_Kerr

Some while ago I did some of this with Canon's DPP. In the version I had at the time, the raw histogram display could have its horizontal axis set to work in "raw units" (0-4095), but that seemed to disappear in the next version. Now, it seems to be in an arbitrary scale in "stops", and I don't know what the origin is or even the values at the ends.

I use IRIS:

http://www.astrosurf.com/buil

It has historically loaded Canon, Nikon, Pentax, Olympus RAW data literally, and now supports DNG so any RAW that the DNG converter can convert can be opened in IRIS as well.

If you choose the "load a RAW file" menu option, IRIS will load it as a greyscale image. You can then have it interpolated to RAW color, or, if you want, or just separate it into four color channel greyscales with the command, "split_cfa a b c d" and it will display a. The letters will represent the positions from the upper left of the RAW image in this pattern:

cd
ab

For Canon, which has red in the upper left, I use "split_cfa g2 b r g1" for:

r g1
g2 b

In the command window, you can type "load a" and save b", etc, to load and save the CFA subimages.

IRIS' histogram is uni-channel (R, G, and B of a color image are treated as three different pixels), so if you want a nice RGB RAW histogram you want to load the image somewhere else. You can just make an image with the command "merge_rgb r g1 b", which has no interpolation within it, and is good for getting a nice RGB histogram out of another program.

IRIS' number space is 16-bit signed. When you save out an image in 16-bit formats, it checks to see if there are any negative numbers. If there are not, it maps the 0 to 32767 range to the 16-bit output (doubles the values), if there are negative numbers, then it treats the output data as 16-bit signed, which is incomprehensible to most, if not all image processing programs, so clipping at 0 should be done first. When saving 8-bit output, IRIS maps only the visible image to the 8 bits. IRIS has two sliders in a threshold tool, the top one representing max and the lower, min, and this window of the data is displayed on the screen, and is used in 8-bit output.

Before making any sense out of the RAW values, you need to know where white and black are for your camera at a given ISO. I recommended shooting a set of blackframes at all ISOs, and a set of completely clipped images to see the value (or values; sometimes it's on a line-by-line basis) of the clipping point. Generally, it is a good idea to subtract the mean black level from a RAW before doing much else with it. Interpolation to RGB, binning, downsampling, etc, are all done across the zero line into the negative numbers, which lowers the noise slightly and preserves very faint signals by not clipping half of them away. Clipping at zero is the last thing to do, for minimal noise. You can also multiply the data to use the full range of 0 to 32767 before any of those operations or white balance (or gamma), to get more mathematical precision. Subtle benefits all, but benefits they are.

[John Sheehy]

Originally Posted by John_Schwaller

I thought Doug's request was tools to analyze the RAW data, not input it to PS.

Well, it has to be analyzed somewhere. You can't analyze it with DCRAW, you can only extract it into a usable format.

I thought he might find it (a tool) around discussions on DCRAW. I vaguely remember someone (on some unremembered forum) discussing a tool which was able to work with raw data, but it was too far back...and too little interest on my part.

Actually...the "-D" option might be what he wants...he'll have to let us know.

The -D option outputs the literal RAW data into a 16-bit format, but as I said, this format is mutilated by Photoshop upon loading into its 15.000044 bit format. The -D option should be integer-scalable, but it is not, or was not, the last I checked.

[Doug Kerr]

Hi, John,

[I thought I had already responded to this, but the message seems to have evaporated someplace here]

Thanks so much for the info on IRIS. Actually, since I last wrote, I reacquainted myself with IRIS, and have figured out some of that (in a basic way), but your detailed explanation is extremely helpful as I proceed.

Often, the biggest problems are little ones, like, "How do I open the command window?". (I finally found reference to the little toolbar icon in the online ISIS tutorial.)

I think I will be able to do exactly what I want with IRIS.

Thanks again so much. You have always been so supportive of my work.

Best regards,

Doug

[Giovanni Brembati]

Hi Doug,
a free software to plot detailed histograms from 16-bit images

http://www.guillermoluijk.com/softwa...mmar/index.htm