Large GB files in Photoshop!

What's your experience in the time it takes to open crop or save 20-30 GB files in photoshop. It takes for ever like 40-50 minutes for every procedure!

I have a raid (~empty) 1 TB scratch disk, a fast SSD drive for Mac OS 10.6 and CS4 on a dual intel Xenon processor, admittedly only 16 GB RAM and 1.88 dual duo core processors.. These files are from Autopano Giga and I suspect that they are not as lean as they could be. It's only 243 layers of each 1 Canon 5D II files in 8 BIT and one stitched layer.

Nothing running at the same time. There's some 160 GB of virtual RAM and the processor is going at about 20% to 80-90% (whereas AP Giga uses up to 300% of the processors when it's working).

Thanks for ideas on how you work with such monster files. Obviously, I can cut the large file into say 5 logical pieces just to optimize things beautifully and then reassemble without all the layers once they are no longer needed.

I'm going to install a maxed-out graphics card to see if that can sweeten the machine. The next thing to try would be be a sawn-off shotgun, point blank inside the beast! I would, except I don't actually own the Mac box!

Asher

Photoshop CS4 is a 32 bits application under OS x. Upgrading to CS5, which is a 64 bits application, may improve things.

I am not sure whether your files are huge because of the resolution or because of the many layers. In the latter case, maybe you should consider merging some of the layers to reduce the data.

I don't think that changing the graphic cards is going to help.

Jerome Marot said:

Photoshop CS4 is a 32 bits application under OS x. Upgrading to CS5, which is a 64 bits application, may improve things.

I am not sure whether your files are huge because of the resolution or because of the many layers. In the latter case, maybe you should consider merging some of the layers to reduce the data.

I don't think that changing the graphic cards is going to help.

Click to expand...

Hi Jerome,

CS4 hands over a major load to the graphics card. That's been one of the biggest advances. Autopano Giga and Aperture also use that extra computing power for graphics calculations.

I can only get rid of layers by dividing the pano to segments and then most of the layers are not needed for that segment. I am starting by cutting the picture into 5 zones and the each will have the non-relevant layers discarded. Then I can work on each layers to do any repairs. after that I can discard the layers and just keep the correct segments of the pano and stitch them back again. By that time, the total will be less than 4 GB and I can use a compressed TIFF file.

Asher

Asher Kelman said:

Hi Jerome,

CS4 hands over a major load to the graphics card. That's been one of the biggest advances. Autopano Giga and Aperture also use that extra computing power for graphics calculations.

Click to expand...

Hi Asher,

I'm not sure about CS4 dealing with memory that well. I skipped CS4, but saw it in action (or not) on other people's computers. If I recall correctly, as Jerome pointed out, it may take CS5 to improve performance, including the use of the GPUs for faster processing on top of 64-bit memory use. The software requires the proper drivers to be installed to use the GPUs, but not all operations are currently making use of the accelerated performance, yet.

I believe Photoshop will use it's swap files even if there is enough Ram, so make sure the swap files are on the faster drives, and that those drives are not fragmented too much.

There are limitations on the size of TIFFs, both in pixel dimensions and in total file size, so you may want to consider saving as .PSB files (Photoshop's big file format).

Cheers,
Bart

Bart_van_der_Wolf said:

Hi Asher,

I'm not sure about CS4 dealing with memory that well.

Click to expand...

Bart,

Thanks for taking a stab at this! TIFF files have to be for size below 4GB so that's not on the table. Actually, I'm shocked to find that CS4 can only use 3GB on the MAC under 32 BIT! One needs to go to CS5 which I'll now do ASAP and add the new GPU I purchased.

CS5 in 64 BITS mode can use as much RAM as one throws at it!

Also, I should use my 120GB SSD for the initial scratch disk ahead of the 1 TB hard drive RAID if have. Once CS5 is launched, there's no speed benefit really for it being on an SSD, The use as a scratch disk makes more sense. That's straight from Adobe here.

It will be something to see this work!

Asher

Asher Kelman said:

Bart,

Thanks for taking a stab at this! TIFF files have to be for size below 4GB so that's not on the table. Actually, I'm shocked to find that CS4 can only use 3GB on the MAC under 32 BIT! One needs to go to CS5 which I'll now do ASAP and add the new GPU I purchased.

Click to expand...

I suggest you first install the Graphics card. I don't know if Photoshop installs different compnents depending on the GPU configuration it finds at the time of installing. There is no guarantee that Photoshop agrees with the specific GPUs it finds, but I think most nVidia based offerings will be fine.There is a list of 'recommended' cards here.

Cheers,
Bart

Asher Kelman said:

CS4 hands over a major load to the graphics card. That's been one of the biggest advances. Autopano Giga and Aperture also use that extra computing power for graphics calculations.

Click to expand...

Certainly. But your problems appear to be memory problems. A new graphic card will not help for that.

I can only get rid of layers by dividing the pano to segments and then most of the layers are not needed for that segment.

Click to expand...

I really don't understand how you use layers for this work and how you come to files as big as 60 GB. And I have done fairly large panoramas myself. What is the resolution of your panorama?

Note that I did not say that you should discard the layers, but flatten them (or a part of them). This limits you in the sense that you can't go back on these adjustments before the point where you merged the layers, but it does not make you divide the image in chunks.

Bart_van_der_Wolf said:

I suggest you first install the Graphics card. I don't know if Photoshop installs different compnents depending on the GPU configuration it finds at the time of installing. There is no guarantee that Photoshop agrees with the specific GPUs it finds, but I think most nVidia based offerings will be fine.There is a list of 'recommended' cards here.

Cheers,
Bart

Click to expand...

I have the Radeon 6770 with 1GB on board and it's newer than the series in Adobe's article. Finger's crossed!

Asher

Found that, in case you upgrade soon...
http://www.adobe.com/content/dam/Adobe/en/products/photoshop/pdfs/PhotoshopCS5_performance.pdf

HTH...