Open Photography Forums  
HOME FORUMS NEWS FAQ SEARCH

Go Back   Open Photography Forums > Digital Darkroom > Digital Asset Management: access & storage of image files.

Reply
 
Thread Tools Display Modes
  #1  
Old January 17th, 2007, 05:41 AM
Ray West Ray West is offline
Senior Member
 
Join Date: Jun 2006
Location: UK - Somerset
Posts: 1,703
Default long term, or longer term

There are a number of discussions here on long term storage of digital images, etc. Different folk have different requirements.

Just out of curiosity, I was wondering how much of the stuff you consider important now, images and the like, are, say, more than five years old?

I have some bmp files dated feb 1995, but I know I have some older ones somewhere, and some text files dated 1989. The text files were written, iirc using wordperfect, I can read them fine in notepad, but the formatting is lost. No real effort has been made to keep these, just copied over to hdds, as my pc's have been upgraded. I have much earlier stuff on paper tape, I expect...., plus 5 1/4 floppies, cassette tape and the like.

So, in 20 years time, where will your images be? Will you be able to use them? What age are the oldest files you use now?

Best wishes,

Ray
Reply With Quote
  #2  
Old January 17th, 2007, 02:37 PM
Angela Weil Angela Weil is offline
Member
 
Join Date: Oct 2006
Location: Heidelberg, Germany
Posts: 46
Default

Good question. Your mentioning of 'Word Perfect' ticked me off. I have a large unfinished text in this format (200 + pages) dating from 1995 or so and earlier.
If you think that only photographers have this problem: My spouse has music files on a most amazing array of formats, analogue media, digital media - a lot of which are no longer in use. In fact, our storage space looks like a history of storage media. I have no idea how this will develop. But we did start to copy and reformat a number of 'old' things before we discard 'old' media. He keeps a few 'old' computers, because they provide the only opportunity to read now obsolete formats (floppy disk drive, anyone?). We even have a record player (!) and a way to digitalize records or tapes to burn CD's or turn them into MP3's.
On the image side, there are files with negatives and files with slides, a flatbed scanner, a film scanner, CD's, DVD's, an assortment of hard disks and a back up server. Unbelievable. Bmp files - I remember them.
However, I always keep paper copies of important things, texts or images. That's a funny development with the digital revolution.
There are studies on large corporations trying to save money and to be more considerate of the environment by encouraging everyone to write e-mails instead of letters on paper. It does not work: Everybody prints everything and possibly copies it several times, thus piling up costs for the consumables.:-))
Angela
Reply With Quote
  #3  
Old January 17th, 2007, 11:58 PM
Cem_Usakligil Cem_Usakligil is offline
Senior Member
 
Join Date: Jun 2006
Posts: 5,939
Default

Interesting question Ray, as usual :-).
Firstly, I've got tons of off-line stuff residing on:
- Some punch cards containing a Fortran FFT (Finitie Fourier Transform) program which was run on a Univac 1106 mainframe (kept as a sentiment)
- Some telex paper tape containing a self written BIOS and the bootstrap to load it for a Motorola 6800 processor based controller system which I had developed for my graduation project (again, kept just for sentiment's sake)
- IBM tape reels dating from 1985-87 made on a S/38, containing some IBM text prcessing docs. (kept as a relic, no way to read the data I'm afraid)
- 5.25" soft floppies from 1985-1992 period containing WP and early word docs (no drives with which to read them)
- 3.5" floppies from 90s and onwards with documents, programs, OS (OS/2 Warp for instance) (mostly accessible)
- Tape backups made on casette tapes (Commodore), DDS, DAT, OnStream, etc (all unrecoverable I'm afraid. That's is the reason I've stopped using tapes at a certain moment)
- Millions of CD-ROMs (e.g. MSDN subscriptions, magazine gifts, data backups): mostly recoverable although some of the earlier CD-Rs I've burnt for backup purposes have passed away.
- Very many off-line SCSI and ATA hard disks in denominations from 20MB to 40GB (mostly recoverable)
- Lots of paper prints (LOL)
- Recent off-line backups residing on DVDs (mostly -hopefully- recoverable)
- Recent off-line stuff residing on external USB/FW hard disks in denominations from 80GB to 500GB (some 8 TB in total). (actual and recoverable)

Re. the on-line data, the earliest I can find are WP documents from 1990. As you've written yourself, I can only see them in text format. There are also some documents made with SDW (Software Development Workbench) in 1990 and some associated drawings and flowcharts, but I cannot decipher them anymore.

So where does this leave me at? I'll have to think about it for a while before -if at all- I can give an answer :-).

Cheers,

Cem
Reply With Quote
  #4  
Old January 18th, 2007, 03:25 AM
Ray West Ray West is offline
Senior Member
 
Join Date: Jun 2006
Location: UK - Somerset
Posts: 1,703
Default

Hi Angela & Cem, et al,

At the back of this question, I guess, are my concerns for folk looking at long term storage on dvd's. I see/imagine a whole raft of problems. Maybe OK if you are keeping the data for a year or two, but after that you'll probably have to copy the whole lot, if you can, onto whatever other media is the current fashion. I had a lot of fun, a few years back, with Syquest, Iomega and MD drives. And as for writing cd's - never even walk across the room, in the early days.... I think the answer is, there is no answer.

Best wishes,

Ray
Reply With Quote
  #5  
Old January 18th, 2007, 12:37 PM
Asher Kelman Asher Kelman is offline
OPF Owner/Editor-in-Chief
 
Join Date: Apr 2006
Posts: 33,801
Default

Ray and Cem and Angela the very talented photographer from Heidelberg,

We are all looking for the solution to these problems. Unfortunately technology is not yet matured to a point that solutions will not require extra safeguards for near term safeguards for media/drive mechanism deterioration.

I'm hoping that one of the versions of Blue Ray will have a chemical, not merely an organic dye for storing the information. The former hass ben now developed and should be marketed some time this year. A long lasting chemical rec ord is better than a fragile light sensitive orgaqnic layer.

So perhaps this year we'll have ability to store in 50 GB chunks and that now means a pracitical way of doing regular incremental backups or versions.

For me at least 5GB is too small to fiddle with unless it is for delivering a job. I must admit, I prefer backup to hard drives at present.

Asher
Reply With Quote
  #6  
Old January 18th, 2007, 03:59 PM
John_Nevill John_Nevill is offline
Senior Member
 
Join Date: May 2006
Location: Hampshire, UK
Posts: 581
Default

I'd hate to think what old data formats I have lying around. Nostalgically they are very interesting, 51/4 floppies, Amstrad, Oric and BBC micro cassette tapes with various word processing files embedded, together with early AutoCAD files and even Gem stuff.

I also have a lot on paper, likewise a few thousand slides and negatives. Ironically, I feel more confident in the longevity of film than digital stuff.

It took me 12 hours to duplicate a USB hard drive last weekend (using synctoy 1.4 ) and it felt uninspiring!

Maybe Blue-Ray will offer better permamancy and larger capacities but will I be able to open the files in 20 years? I can't find a slot for the cassette tape on my laptop.

Maybe I should print more!

Now where's that slide I was looking for, there its is....no computer needed!
Reply With Quote
  #7  
Old January 18th, 2007, 04:06 PM
Asher Kelman Asher Kelman is offline
OPF Owner/Editor-in-Chief
 
Join Date: Apr 2006
Posts: 33,801
Default

I still own two Apple 2E machines!

Unfortunately I gave away all my DOS and PC machines last year but did transfer all the files to the Mac.

Asher
Reply With Quote
  #8  
Old January 18th, 2007, 04:07 PM
Angela Weil Angela Weil is offline
Member
 
Join Date: Oct 2006
Location: Heidelberg, Germany
Posts: 46
Default long term storage....

Ok, future oriented: For the time being, my bet is on hard drives. I keep image back-ups on three type of hard drives, one in the working computer, one on a server and some additional, older and smaller hard drives in a drawer, not connected to a working system - together with a brief printed description of the content. We use small hard drives for that purpose, small as in 30, 40 or 60 gig. I gave up on burning CD's or DVD's. They are too small, too time consuming to do and too fragile. Ok, hard drives can fail as well but if I have some degree of redundancy, it should work. Maybe, hopefully.
Angela
Reply With Quote
  #9  
Old January 18th, 2007, 04:22 PM
John_Nevill John_Nevill is offline
Senior Member
 
Join Date: May 2006
Location: Hampshire, UK
Posts: 581
Default

Yes, I agree, hardrives are currently the cheapest and most flexible. Although drive connectivity has changed significantly, Today's PCs will not read a hard drive that's 10 years old without specialist hardware. This means we will have to keep transferring digital data to newer mediums. The rate of change will also become more frequent, so method will be as important as the medium itself.
Reply With Quote
  #10  
Old January 18th, 2007, 09:24 PM
Erik DeBill Erik DeBill is offline
Member
 
Join Date: Aug 2006
Location: Austin, TX
Posts: 224
Default

Quote:
Originally Posted by Angela Weil
Ok, future oriented: For the time being, my bet is on hard drives. I keep image back-ups on three type of hard drives, one in the working computer, one on a server and some additional, older and smaller hard drives in a drawer, not connected to a working system - together with a brief printed description of the content. We use small hard drives for that purpose, small as in 30, 40 or 60 gig. I gave up on burning CD's or DVD's. They are too small, too time consuming to do and too fragile. Ok, hard drives can fail as well but if I have some degree of redundancy, it should work. Maybe, hopefully.
Angela
I've heard that you need to rev up those offline drives every few months or a year to keep them working. I know that I've had some old ones stop working when I wasn't looking myself.

Personally I keep all my files online. One copy on the file server, 2 copies on the backup server. That way I at least know if a hard drive fails. It's expensive, though, and a strategy designed before I started producing data as quickly as I do now.
Reply With Quote
  #11  
Old January 19th, 2007, 06:37 AM
Marian Howell Marian Howell is offline
Member
 
Join Date: May 2006
Location: massachusetts south coast
Posts: 225
Default

i work with arts archivists and their guidelines (and the library of congress) recommend transference of files to another medium every 10 years as standard precedure. and of course, "exercising" those drives is a good idea as well. assuming you have them backed up, no harm can be done because if the drive does crash you are covered. this is a bonus of the digital age. i spent many years in the audio world, and exercising tapes is a double-edged sword since the media is so fragile but not exercising them leads to other storage-related issues. so i have multiple hard drive backups (1 set not stored on site). as our data files get larger and larger with every new camera/software generation, this approach can get expensive, and, even more importantly to me, more time consuming.
__________________
marian
Reply With Quote
  #12  
Old January 19th, 2007, 06:45 AM
Nill Toulme Nill Toulme is offline
Senior Member
 
Join Date: Jun 2006
Location: Atlanta, Georgia USA
Posts: 1,407
Default

Quote:
Originally Posted by Asher Kelman
I still own two Apple 2E machines!

Unfortunately I gave away all my DOS and PC machines last year but did transfer all the files to the Mac.
I've never forgiven Apple for abandoning the II line (and me). I think perhaps, deep down, that's why I will never go near a Mac. :-(

Nill
~~
www.toulme.net
Reply With Quote
  #13  
Old January 19th, 2007, 10:43 AM
Stan Jirman Stan Jirman is offline
Member
 
Join Date: Jun 2006
Location: Santa Clara, CA
Posts: 189
Default

My important data is stored on two RAID-5 systems (one master, one backup) where the 2nd is at a different location. I synchronize them relatively often. The really important stuff is also burned onto two sets of DVD, one set is mailed to a friend in a different state.

At the time the images are downloaded, they are MD5 checksummed and these checksums are written into a file. I have a program that I can run periodically that checks the MD5 checksums and can tell me which files have rotten - that way I know if a DVD is bad, or if the original / backup copy have spontaneously changed.

I agree that 4GB disks are totally useless and the size is the main reason why I back up only the most important stuff to DVD. As for future planning I always copy stuff to newer type media as soon as reasonably feasible. All my NeXT ODs are now on CD and on my RAID. The only stuff I don't have readable in a current day format are my floppies from the C64 - but we are talking 1981 here :) God, I am old.
__________________
Stan Jirman
http://www.jirman.com/
Canon & Apple equipment
Reply With Quote
  #14  
Old January 19th, 2007, 11:34 AM
Dave See Dave See is offline
New Member
 
Join Date: Jan 2007
Location: Cali, USA
Posts: 14
Default what is saving, and not a question

Hi all,

A very important topic indeed...
Influencing my perspective is an incident which happened many years ago, when I'd bought (into) the Psion PDA. Must have been mere negelct on my part, but I did not monitor the /back up/ battery status well and lost all recorded data when both that and the AA cells died. I then discovered that my short-term memory had also gone with the batteries! Since then, I've never used a PDA and my short-term memory has recovered too.

I've got lots of old documents and negatives. As for digital, and specifically "how long on which medium/in which format", I've been exercising a sort of weight loss program this passed year. Email, old scripts, directories of texts and notes have all been severly scrutinized and most of what was is now deleted. Very liberating. Very un-GoogleMail ;)

I bought a film/slide scanner and considered "digitizing" some negatives. I too have several hundred vinyl recordings, a few CDs... decided not to store them digitally: they are in the most robust medium now, and translating them to digital makes them another image entirely.

Recently I've made the shift from mostly film to mostly digital capture. Colorspaces, White Balance(sic) and "Exposing Right" have most of my attention now, yet the data is piling up... and when I get a logical goup of files, I write them to CD, or DVD. I still have a DAT drive... and considered the USB harddrive... yes, this is an important thread.

I will come to accept a certain amount of data loss, just as I have the odd "pop" in a vinyl track. By accepting digital data as mutable and impermanent, I tend to be less an archivist of /all/ images and consider first why to keep this, or that file. Before I concern myself with the long term storage media, I must first stop the hoarding of all things digital in the name of "saving" or "archiving"... or because, "I can tweak it in post later".

Martin's post reminded me, that truely archiving assets is a process of data transfer... so if you're planning on keeping anything, you will be re-saving between storage media--perhaps even file format--every few years. I believe CD/DVD will give me about 10 years, don't require electricity(energy) and are in an open format (my DVDs are 4.3GB ISO9660 images). So in 10 years I'll review the saved data, and hopefully weed out a lot of it.

I still have the plates(intaglio) for prints hanging on the wall, but some artists "break the plate" to ensure the value of the print folio... is this done for photography?

rgds,
Dave
Reply With Quote
  #15  
Old January 19th, 2007, 11:43 AM
Dave See Dave See is offline
New Member
 
Join Date: Jan 2007
Location: Cali, USA
Posts: 14
Default

Quote:
Originally Posted by Stan Jirman
[snipped]
At the time the images are downloaded, they are MD5 checksummed and these checksums are written into a file. I have a program that I can run periodically that checks the MD5 checksums and can tell me which files have rotten - that way I know if a DVD is bad, or if the original / backup copy have spontaneously changed.
[snipped]
This is a great idea Stan! And although less likely for us, a fella I met who works in DNA/bio-tech related that his folks generated so many files that two files of different content returned the same MD5 sum. I suspect MacOSX has `sha1sum'... anyhoo, your method is a great idea.

rgds,
Dave
Reply With Quote
  #16  
Old January 19th, 2007, 01:22 PM
Marian Howell Marian Howell is offline
Member
 
Join Date: May 2006
Location: massachusetts south coast
Posts: 225
Default

i think dave has brought up a couple of good issues.
Quote:
I too have several hundred vinyl recordings, a few CDs... decided not to store them digitally: they are in the most robust medium now, and translating them to digital makes them another image entirely.
if digital is not the native media, then this is true, a scan will never be an exact copy of the negative or the print. but sometimes you want/need to save something and the native format is no longer valid/useful/accessible, the original is deteriorating, and any save is better than no save. to continue the analogy of the vinyl discs, i would ask dave when do you stop playing the disc before the needle wears the grooves? the nature of the playback system is destructive. prints fade too if you keep them visible and not stored away unseen in the vault.
Quote:
I still have the plates(intaglio) for prints hanging on the wall, but some artists "break the plate" to ensure the value of the print folio... is this done for photography?
sure!
and lastly, your point about filtering *what* to save is spot on for me!
__________________
marian

Last edited by Marian Howell; January 19th, 2007 at 05:41 PM.
Reply With Quote
  #17  
Old January 19th, 2007, 05:12 PM
Ray West Ray West is offline
Senior Member
 
Join Date: Jun 2006
Location: UK - Somerset
Posts: 1,703
Default

Thanks, for your thoughts. I think filtering is important, I do not normally do it. I am surrounded by junk, in case 'it may come in useful'. This is not only in a relatively small area of binary info, but in lumps of metal, wood and other stuff - like two or three barn full's of it... (not a problem - just keep enough material in the barns to build another barn)

A bit of background, in a former life, I came across companies who used the 'grandfather-father-son' type of tape backup. The problem was, they could not restore. They ran the system for a year or so, but unknowingly copying to corrupted tapes, or the data was corrupted, and they never tried reading the tapes properly, until they needed to, then they couldn't.

If you can remember the analogue mobile phones - they failed gracefully. If you went into a bad reception area, you could walk back out again, or strain your ears. Digital just switches out. Vinyl records, even shellac ones, are playable, even with quite severe damage, but once in the digital form, it is too easily broken - never mind the bandwidth. I like paper tape, I like steam engines, I like lead acid batteries - all pretty reliable old stuff - Stan, your'e just a boy ;-). Pressed cd's are possibly OK, but I do not know if anyone uses them. Magnetic tape, re. Marian's note, it prints through, if not rewound - my 'old grey whistle test' half inch tapes self erased in 5 or 6 years. I wonder about the long term crystalline state of dvd's (Philip's early c'ds failed in 6 months or so - silver layer oxidising or something - and Philips is a pretty techy type company, who did all the testing, I suspect.) .

afaik, usa gov archive in tif images, if in digital image, but the format choice is another can of worms (re. our old word perfecrt files.)

Best wishes,

Ray
Reply With Quote
  #18  
Old April 27th, 2007, 06:16 AM
Ray West Ray West is offline
Senior Member
 
Join Date: Jun 2006
Location: UK - Somerset
Posts: 1,703
Default

I was just sent this link, it looks as if it may be of interest. A more automatic back up system it seems. I have no knowledge about it, other than it may exist.

http://www.drobo.com/


Best wishes,

Ray
__________________
All images posted in OPF are assumed to be for Comment & Critique, unless otherwise designated Comment Only or Edit and Repost.
Reply With Quote
Reply

Bookmarks

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are Off
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT -7. The time now is 06:53 AM.


Posting images or text grants license to OPF, yet of such remain with its creator. Still, all assembled discussion 2006-2017 Asher Kelman (all rights reserved) Posts with new theme or unusual image might be moved/copied to a new thread!