• Please use real names.

    Greetings to all who have registered to OPF and those guests taking a look around. Please use real names. Registrations with fictitious names will not be processed. REAL NAMES ONLY will be processed

    Firstname Lastname

    Register

    We are a courteous and supportive community. No need to hide behind an alia. If you have a genuine need for privacy/secrecy then let me know!
  • Welcome to the new site. Here's a thread about the update where you can post your feedback, ask questions or spot those nasty bugs!

Lossy compression DNG files

Doug Kerr

Well-known member
I have just become aware that there is such a thing as a lossy compression DNG file, which is smaller than the regular DNG file.

Using the Adobe DNG Converter on a CR2 file output by my Canon Powershot G16, where the CR2 file has a size of 14.6 MB, the regular DNG file has a size of 14.1 MB, and the lossy compression DNG file a size of 7.1 MB.

I have no idea how much of a compromise in the quality of the ultimately developed image might be encountered with the use of the lossy compression DNG file (or how we could even know).

The choice to use the lossy compression format for DNG files is found in the Preferences panel of the Adobe DNG Converter (accessible by pressing the Change Preferences button on the main page).

Best regards,

Doug
 

Tom dinning

Registrant*
I have just become aware that there is such a thing as a lossy compression DNG file, which is smaller than the regular DNG file.

Using the Adobe DNG Converter on a CR2 file output by my Canon Powershot G16, where the CR2 file has a size of 14.6 MB, the regular DNG file has a size of 14.1 MB, and the lossy compression DNG file a size of 7.1 MB.

I have no idea how much of a compromise in the quality of the ultimately developed image might be encountered with the use of the lossy compression DNG file (or how we could even know).

The choice to use the lossy compression format for DNG files is found in the Preferences panel of the Adobe DNG Converter (accessible by pressing the Change Preferences button on the main page).

Best regards,

Doug

Another quandary. I have the same suspicions when the snotty nosed, pimple faced teenager delivers the pizza. Did he add something and will it ultimately alter my experience?
The choice is simple: I don't buy pizza, nor do I use dng converter. It seems to be another step in the chain that could be tampered with.
Or am I just paranoid? After all, the said teenager is an ex student and I might have unfairly judged him.
 

Asher Kelman

OPF Owner/Editor-in-Chief
I have just become aware that there is such a thing as a lossy compression DNG file, which is smaller than the regular DNG file.

Using the Adobe DNG Converter on a CR2 file output by my Canon Powershot G16, where the CR2 file has a size of 14.6 MB, the regular DNG file has a size of 14.1 MB, and the lossy compression DNG file a size of 7.1 MB.

I have no idea how much of a compromise in the quality of the ultimately developed image might be encountered with the use of the lossy compression DNG file (or how we could even know).

The choice to use the lossy compression format for DNG files is found in the Preferences panel of the Adobe DNG Converter (accessible by pressing the Change Preferences button on the main page).

Best regards,

Doug

Doug,

One way to look at this is to look at a picture of your good wife in a draw hat! If you process the RAW file with a set of Changes made in a separate layer in Photoshop, the correction layer is always copied from another file). With say 6 aggressive filters, if you look at the expanded histogram, do you see any jagged, "toothcomb", effect or on the image loss of detail in the edges of the straw or posterization near highlights or shadows especially??

If the histogram is smooth and your pictures seem flawless, likely as not, the compression in the DNG did no harm. Printing out the image would help too. Do sure Bart has more scientific tests!

Asher
 

Doug Kerr

Well-known member

Doug Kerr

Well-known member
Hi, Asher,

Doug,

One way to look at this is to look at a picture of your good wife in a draw hat! If you process the RAW file with a set of Changes made in a separate layer in Photoshop, the correction layer is always copied from another file). With say 6 aggressive filters, if you look at the expanded histogram, do you see any jagged, "toothcomb", effect or on the image loss of detail in the edges of the straw or posterization near highlights or shadows especially??

I'm not sure I understand the part I have highlighted in red. Do you mean that whether I have the regular DNG file or the lossy-compressed DNG file loaded, I apply a consistent correction layer?

I don't know how to move correction layers from one file to another.

Interesting approach. Thanks for suggesting it.

Best regards,

Doug
 

Andrew Stannard

pro member
Doug - Glad you found the article useful. I don't know if you followed the link in that page through to an adobe forum thread, but if you didn't it's worth doing so. It contains even further detail around the mechanism of compression used, and reasons why.

Andrew.
 

Doug Kerr

Well-known member
Hi, Andrew,

Doug - Glad you found the article useful. I don't know if you followed the link in that page through to an adobe forum thread, but if you didn't it's worth doing so. It contains even further detail around the mechanism of compression used, and reasons why.

Yes, I did read the referenced thread. That is all way amazing!

Thanks again.

Best regards,

Doug
 

Doug Kerr

Well-known member
Hi, Asher,

One way to look at this is to look at a picture of your good wife in a draw hat! If you process the RAW file with a set of Changes made in a separate layer in Photoshop, the correction layer is always copied from another file). With say 6 aggressive filters, if you look at the expanded histogram, do you see any jagged, "toothcomb", effect or on the image loss of detail in the edges of the straw or posterization near highlights or shadows especially??

If the histogram is smooth and your pictures seem flawless, likely as not, the compression in the DNG did no harm. Printing out the image would help too. Do sure Bart has more scientific tests!

I don't understand the premise behind this suggested route of investigation.

Let's start by assuming that the CR2 file out of the camera will, when developed, lead to the desired image.

Next, let's assume that a normal DNG file generated from the CR2 file will do likewise.

Now we address a "lossy compression" DNG file generated from that same CR2 file. At issue, I believe, is to what extent does the use of lossy compression lead to this raw file developing into into a different image, perhaps one that does not as accurately reflect the scene. Maybe some detail is compromised in some way, or some artifacts emerge.

Is it your premise that if the lossy compression DNG file develops into a significantly different image than the normal DNG file, we can note that by subjecting the images developed from both DNG files to the same regimen of "aggressive" filter actions; if the file from the lossy compression DNG file is significantly different from the image from the regular DNG file, that will manifest by increased misbehavior under these aggressive filter actions?

Remember, the filter actions work on the image developed from the raw file by ACR. I can't quite grasp what unwanted differences in the image developed from the lossy compression DNG file might be expected to make that image more susceptible to misbehavior under aggressive filter actions.

What am I missing here?

Best regards,

Doug
 

Asher Kelman

OPF Owner/Editor-in-Chief
Hi, Asher,



I'm not sure I understand the part I have highlighted in red. Do you mean that whether I have the regular DNG file or the lossy-compressed DNG file loaded, I apply a consistent correction layer?

I don't know how to move correction layers from one file to another.

Interesting approach. Thanks for suggesting it.

Best regards,

Doug


Hi Doug,

Add the correction layer to the file base layer with one version of the original file and then stage that layer to the image from the other, differently compressed file.

The object of the harsh testing is to discover whether or not the file still maintains great edges and gradations. Or perhaps the colors become posterized with sections of different hue, no longer gradually changing scrim one hue to another and instead jumping from A to B with obvious differences as in a comic book.

Of course one could use Imatest or some other program.

This is essentially the same question that jpg afficionados challenge RAW devotees with!

Asher
 

Doug Kerr

Well-known member
Hi, Asher,

Add the correction layer to the file base layer with one version of the original file and then stage that layer to the image from the other, differently compressed file.

I don't know how to do that with Photoshop. (Recall my skill with Photoshop is very limited!)

This is essentially the same question that jpg afficionados challenge RAW devotees with!

Ah, yes.

Thanks.

Best regards,

Doug
 

Doug Kerr

Well-known member
Here we see a visual comparison of a small crop from the JPG files developed (by Canon DPP) from two DNG files, generated by Adobe DNG Converter from same CR2 file generated by my Canon PowerShot G16 camera. The shot was at ISO 400.

The original shot resulted in a JPG image with dimensions of 4000 px × 3000 px. The reconstructed JPG images (from the CR2->DNG chain) had those same dimensions.

The crop on the left is developed from a DNG file with normal compression; the one on the right from a DNG file with lossy compression. They are both presented on a pixel-for-pixel basis.

G03461A-C1.jpg
G03461B-C1.jpg


Left: from DNG with normal compression. Right: from DNG with lossy compression.​

Both developments were with the default settings of DPP.

We note that various clever steps are taken during the generation of a DNG file with lossy compression which may in fact have a beneficial effect on the noise in the ultimately recovered image.

Best regards,

Doug
 

Doug Kerr

Well-known member
Well, something has obviously gone wrong in the preparation of my report above. Of course the two DNG files could not have been developed in DPP - it doesn't work on DNG files.

So I need to reconstruct exactly what I did do. I'll be back when I get that straightened out.

Sorry for the bum steer.

Best regards,

Doug
 

Doug Kerr

Well-known member
Well, here is the report afresh, hopefully correct this time. The change is highlighted in blue (but the images are new also)

************

Here we see a visual comparison of a small crop from the JPG files developed (by Photoshop/ACR) from two DNG files, generated by Adobe DNG Converter from same CR2 file generated by my Canon PowerShot G16 camera. The shot was at ISO 400.

The original shot resulted in a JPG image with dimensions of 4000 px × 3000 px. The reconstructed JPG images (from the CR2->DNG chain) had those same dimensions.

The crop on the left is developed from a DNG file with normal compression; the one on the right from a DNG file with lossy compression. They are both presented on a pixel-for-pixel basis.

G03461A-PS-C1.jpg
G03461B-PS-C1.jpg


Left: from DNG with normal compression. Right: from DNG with lossy compression.​

Both developments were with the default settings of PS/ACR.

I think I would have to say that I find a very slight deterioration in image quality overall from the lossy compression DNG route.

Best regards,

Doug
 

Doug Kerr

Well-known member
Now, for something a little different, a similar comparison between:

• (On the left) A crop from the JPG image developed by Photoshop/ACR from the regular DNG file generated from the camera CR2 file.

• (On the right) A crop from the JPG image developed by Raw Therapee, using the vaunted AMaZE demosiaicing algorithm, from the regular DNG file generated from the camera CR2 file.

G03461A-PS-C1.jpg
G03461A-RT-C1.jpg


Left: PS/ACR development. Right: RawTherapee development​

The noise performance is clearly inferior in the Raw Therapee development.

In addition, in the blue sky over the target house (no crops of that shown here) the noise in the RawTherapee development is greatly worse.

Of course, I did not optimize the various settings in RawTherapee (the development was with the default settings).

Interesting.

Best regards,

Doug
 

Doug Kerr

Well-known member
Well, indeed I did not have noise reduction turned on in Raw Therapee.

Here are the two files again with noise reduction set to a modeate level in Raw Therapee:

• (On the left) A crop from the JPG image developed by Photoshop/ACR from the regular DNG file generated from the camera CR2 file.

• (On the right) A crop from the JPG image developed by Raw Therapee, using the vaunted AMaZE demosiaicing algorithm, from the regular DNG file generated from the camera CR2 file.

G03461A-PS-C1.jpg
G03461A-RT2-C1.jpg


Left: PS/ACR development. Right: Raw Therapee development​

We see that this level of noise reduction seriously impacts image sharpness (note for example the license plate).

Of course again my settings may not have been "careful".

Just playing.

Best regards,

Doug
 
Top