Dumb or DAM? Admit it! Folders or keywords, how do you organize your pictures?

John Beardsworth has raised the important challenge to those advocating organizing by some folder hierarchy. Do you take the approach the he does that we should rely on keywords so that we can find images no matter what or do you cling to the foolish notion that a folder hierarchy is sufficient.

Read John passionate but logical wake up call for those of us still taking shelter in a structure that is not capable of withstanding shifts in nomenclature, the city name changed, the files in the wrong folder, the last flower shoot in with the sunsets in Maine. The files needed in 3 places with the butterfly, the river and the sunset.

Read John's blog here and then admit what you really do!

Asher

Let me confess first!

After ingestion, all my images go to iviewMedia Pro for keywording.

I keep folders by year and month and topic. Derivatives get subfolders. That way if I'm working on a project and need to continue with it while traveling I'll grab a copy of 2007_12_17_Symphony and 2008_01_26_symphony and take them with me on my laptop. All new work is put in a folder, laptop_derivatives" and when I return, these derivatives will be added to the appropriate iviewmedia Pro catalogs, keyworded coded and then placed in the appropriate folder.

So I admit to a hybrid system. However, not one sorted by subject.

When I really trust the software and it wont blink at 50,000 files or double that, then I'll put everything in one catalog. At present, I use overlapping catalogs: by subject and by hard drive.

So how do you handle your files and what software do you trust?

Asher

John writes:

So what happens when a photo fits two categories? Do you put one copy in each folder? Or add more subfolders?

Click to expand...

For me, thats easy, I make a collection.

I have just the case he mentioned. I had two LR libraries based on a years work (say 2006 and 2007). In each I have a folder named "Dogs" with dog images. The naming structure I use is to include the Folder Name token in LR. that way, the actual folder name is "Dogs 2006" and "Dogs 2007". This allows a simple F2 key to rename images based on that folder name.

OK, now I decided to incorporate both years into one bigger library. That's easy, you can use the Export to Catalog and then Import to Catalog function. But I have two separate folders of Dogs based on two years and I want to keep them that way. No problem. Make a Collection called Dogs, drag both folders into that. They remain on the HD in their original folder locations and the naming token still works yet I can find ALL dogs by clicking on the Collection named Dogs. Further, I can keep two separate and smaller LR Libraries for travel. The bigger mongo library is easily updated should changes be made on the road from the smaller ones (Sync Catalog or Export/Import).

I'm not saying I've figured out a better system or it would work for John, but it works for me.

I do use Image Ingester to sort files by date (DAM Book style) and to insert copyright and basic shoot metadata. But keywords are silly amounts of time and work to invest when you do not have the time and make no money of the time it would save you.

When I want to find an image of a sort, I go to either my web-released image folder, on of the print ready file folders, or go to one of my books of 4x6 prints and get a filename. From there due the per camera sequence numbering I can get into the right month or so and from there forward I prefer to browse and see what catches my rather than seeing what caught my eyes six months ago. My tastes are constantly evolving and unpredictable.

my $0.02 and another test of Quick Reply,

Sean

I pretty much do what Andrew suggests.

I use a number of raw converters and sort with Lightroom. I have two main libraries one for general and one for wildlife.

The general library has all images organised in folders by date and event e.g. /2008/02-10-Festival X/ etc.

The wildlife has images organised by genre e.g. /cats/snow leopards/ etc.

I initially used the stack feature in Lightroom, placing the PSD alongside the raw.

This approach was eating disk space at a phenomenal rate and although accessible, it put quite an overhead on the individual library folder.

So I've since changed and now move all PSDs into a similar folder hierarchy on a different drive using the same filename as the original. The PSDs are then cross referenced using virtual collections.

This folder based organisation enables me to locate and edit raws and psds in other applications. A quick folder sync in Lightroom maintains library integrity.

I also tend to keyword on import, so any derivative is easy to find.

If I find an image sits in both general and wildlife, I'll import the file into both libraries keeping the file in the same location.

Folders for me

Since I shoot mostly people and travels, I have folders by the names of the people or by the trip/location. Subfolders for RAW and JPG out of camera and a subfolder called edited and another called discards. I don't trash the discards but am not using them on initial edits. I do go back and find gems there sometimes. This system lets me work in any of the programs that I have and makes it easy for me to upload to my website and copy to my backup drives too.

This is a good topic for the dead-of-winter, Asher. (It's 6F here in Chicago as I write this.)

File Naming Convention
This simple naming convention is at the core of my organizational system and has proven itself to be exceptionally durable and useful in the four years since I've adopted it.

All of my image files are renamed (with EXIF Renamer) immediately after ingestion but before any other operations are performed. I use a simple but extremely effective format for all image file names.
_________ <CAMERA PREFIX><YYYY><MM><DD>-<HH><MM>-<SERIAL>
The year, month, day-of-month, hour, and minute substrings (extracted from the file's EXIF data) need no explanation.

The <CAMERA PREFIX> is an identifier for the particular camera used to capture the image. For example, a Leica M8 image would be prefixed "M8". Files from my Canon 1 bodies are prefixed with the unique first 4 characters of their file names (ex: WY0R, GF7M).

<SERIAL> is the serialized number that the camera assigned to the image file during capture.

So for example:
M820080218-0933-000050 is an M8 image captured Feb 18, 2008 at 9:33am.
WY0R20080213-1426-6279 is a Canon 1Ds Mark II image (from the WY0R prefix) captured February 13, 2008 at 2:26 pm.


Physical Organization
All of my image files are organized into hierarchical folders on disk by:
____ YEAR
________ MONTH
____________ DAY-OF-MONTH/TOPIC

For example, image files recorded today would be located in the folder: 2008 / 02 / 19 . I can have any number of folders for a particular date, in case I want to segregate images according to topic.

This produces an organization that's simple to search and navigate independently of any other software cataloging system, although it also makes folder navigation extremely self-evident from within Lightroom and Bridge.

It's saved my bacon many times, too.


Topical Organization
I use Adobe Lightroom for this. I use key-wording / key-phrasing to categorize characteristics intrinsic to an image's content. I use collections for image usage applications. For example, I will use a hierarchy of collections to assemble and organize images for a particular project, images to be printed, images to be placed online, etc.

So that's my system in a nutshell.

Ken and others using file folder hierarchies,

So when you travel, you take with a collection presumably if you think you are going to work further on them. Now when you return, how do you file the derivatives back to the Folder hierarchy?

Asher

Asher Kelman said:

Ken and others using file folder hierarchies,

So when you travel, you take with a collection presumably if you think you are going to work further on them. Now when you return, how do you file the derivatives back to the Folder hierarchy?

Asher

Click to expand...

Export Catalog, Import Catalog. Works like a charm.

Asher Kelman said:

Ken and others using file folder hierarchies,

So when you travel, you take with a collection presumably if you think you are going to work further on them. <....>

Asher

Click to expand...

I never take image files (unless they represent a motive for the trip). If I did, however, I would take then in the same folder structure they have at home, even if it was just one file. Consistency is a key to reliability

Ken Tanaka said:

I never take image files (unless they represent a motive for the trip). If I did, however, I would take then in the same folder structure they have at home, even if it was just one file. Consistency is a key to reliability

Click to expand...

So let's imagine you need a Lightroom "Collection" you have spent the effort to make? You would, I understand take the 4,000 images in the entire folder structure related to the 30 pictures you need, even when the add to up to say 20GB?

This could be photographs from over 4-5 years! If you do take all the related folder hierarchy set, at least it would be consistent. How would you accomplish this. In Lightroom, for example, how would you specify that without doing it one date at a time.

Asher

No. If I needed to take 30 image files from my collection with the expectation that I would need to subsequently replace them back into the collection I would probably transport them in the their home folder STRUCTURE, but not with all of the other images within that structure.

If I had no expectation of having to replace the files into the main archive I would probably just yank copies onto a flat folder.

Either way, however, my image file name format makes it simple to relocate the original and proper archive folder location very quickly.

I quickly settled on a similar folder structure. My naming is slightly redundant to be descriptive and then the file name duplicates the info in the folders, just so the file can stand on its own somewhere else.

Finally, the hour and minute serve as my serial number and then a sequence number on the end (if needed) to prevent duplicates, like so:

{2007}\{2007-12}\{Dec-23}\{2007-Dec-23-0545}(1)

So if I have a second photo taken at 5:45am, it would be

{2007}\{2007-12}\{Dec-23}\{2007-Dec-23-0545}(2)

After imbibing my pictures into LR, I noticed that with the functionality in the metadata browser, I didn't need this structure anymore, but I haven't been able to implement a better one. One idea I have is that they could just be dumped into folders that don't exceed 2GB so I can easily back them up on DVDs.

However, I don't have any utilities that support that kind of decision making...

Pragmaticality is the way to go....

Hi folks,

Lots of good input here. It goes to show that there is no "right" or "wrong" way of doing this that'll work for everybody alike. One ends up doing what works for him/her.

Firstly, let me start by saying that I have been a Peter Krogh adept, have learnt a lot from his good book and read all the related bits and pieces published on the net. I never got to the point that I could comply with more than 70% of what he's been advocating. It just didn't work for me.

Nowadays, I start by ingesting the RAW files into the ingestion folders by using the incredible tool Image Ingester Pro. When doing that, I create folders like this:
camera\2008\2008_02\2008_02_21. I don't create job or topic related folders/directories. I now depend on LR to do that for me using the keyword browser. Like Andrew, I also use catalogues a lot.

I create the basic meta data for the shoot using the IIP during ingestion. Then I import files into LR from their local directories. In LR, I do the rest of keywording, albeit not very consistently I'm ashamed to admit.

I copy at the same time all the ingested files to another set of folders for keeping them intact as ingested and for backup purposes. I do not backup to DVD's, it is too much of an hassle. I use multiple external hard disks some of which are kept off-site. I do a complete synchronisation of the folder structure on the internal hard disks to the external hard disks without using any complicated back up tools. Data does not get compressed or changed in the process. I also keep a third online copy on my file server. So in total, I have 3 online copies of my files and at least 2 off-line copies. The drives get updated to higher capacities and newer technologies (such as IDE to eSATA) every couple of years, so my data keeps getting refreshed in that process and I stay ahead of obsolescence issues.

Just my $ 0.02 anyway.

Cheers,

Cem

Hi Cem,

A solid and safe approach. Can you use ingester Pro to set up your backups and send the files to LR?

Asher

Asher Kelman said:

A solid and safe approach. Can you use ingester Pro to set up your backups and send the files to LR?

Click to expand...

Hi Asher,

The IIP is a very versatile program and does many thing like creating a backup directory during ingestion, but it does not import the files into LR. That remains to be an extra step that still needs to be done, but it doesn't bother me ATM. The alternative is to import them in one go from within LR (i.e. ditch IIP), but I like IIP way too much for going that route. Marc Rochkind, who is the creator of IIP, has been very cooperative at the beginning when IIP was still in the beta testing phase. I have built this rapport with him during that period and it would feel like I am betraying him if I'd switch over. Just call me sentimental....

Cheers,

Cem

I am still amazed at the lack of versatile import options in most imaging software programs.

I became spoiled early on with a (mostly) free program called cam2pc. To this day, it has import options for creating directories and naming files that no one has implemented (at least the last time I checked around).

Sadly, it seems to not be supported any longer but you can still download it and there is a shareware version (www.nabocorp.com)

Edward Bussa said:

I am still amazed at the lack of versatile import options in most imaging software programs.

I became spoiled early on with a (mostly) free program called cam2pc. To this day, it has import options for creating directories and naming files that no one has implemented (at least the last time I checked around).

Click to expand...

Image Ingester Pro has nearly complete freedom for naming files and directories.


Mind you, the syntax is not consumer friendly, but then control never has been consumer friendly.

Add in the responsive support from the developer (Marc Rochkind) and that it that IIP will verify/validate ingested images along with making a backup and I have everything I want from a downloader (that and the fact that it does not automatically start like the garbage Adobe releases but waits for my command).

It also adds metadata at the users choice and it does not do any image processing beyond format conversions. Hence no destructive brightness controls or other image data destroying processes (w/ the exception of DNG conversion which should be non-destructive) which means it is safe and no extraneous features I do not want in the field. At the end of the day managing your assets is important, but nothing (and I mean nothing) trumps a physical print for reviewing your work (unless your goal is not related to physical prints).

some thoughts,

Sean

Sean DeMerchant said:

Image Ingester Pro has nearly complete freedom for naming files and directories.

Mind you, the syntax is not consumer friendly, but then control never has been consumer friendly.

Add in the responsive support from the developer (Marc Rochkind) and that it that IIP will verify/validate ingested images along with making a backup and I have everything I want from a downloader (that and the fact that it does not automatically start like the garbage Adobe releases but waits for my command).

It also adds metadata at the users choice and it does not do any image processing beyond format conversions. Hence no destructive brightness controls or other image data destroying processes (w/ the exception of DNG conversion which should be non-destructive) which means it is safe and no extraneous features I do not want in the field. At the end of the day managing your assets is important, but nothing (and I mean nothing) trumps a physical print for reviewing your work (unless your goal is not related to physical prints).

some thoughts,

Click to expand...

These are more than just some thoughts, Sean, much more than that. And I agree with you 200%. Thanks for the concise explanation.

Cheers,

Cem

cam2pc options

Just to clarify, these options may be of no use to you, and there are definitely options that IIP provides that cam2pc does not.

However, as I said, I am still amazed that I can not point to an "ultimate" product that provides a more than adequate number of options for formatting directories and filenames. Case in point is LR, which I expected to be much better in this regard.

Here are some screen shots from cam2pc - note the %a option in the first one that allows you to put a three letter month abbr. in a folder OR file name:



Next, note how much of the EXIF information can be pulled. Although I don't find these useful, I think its a nice feature set.




Now, I just went and looked at the product page for IIP, thinking I had checked this product out before. However, now I realize that I was thinking of another product - $40 is a reasonable price for a sophisticated downloader. I will be downloading it to check it out. Thanks!

I say all this about cam2pc because I think its an excellent product that was years ahead of its time and am saddened that its ongoing development has died. For me it combined the speed of FastStone (years before) and bested most downloaders for naming flexibility, and its feature set stopped evolving several years ago!

However, my needs have grown as I need to be more sophisticated with application of metadata and I'm curious how the backup features of IIP might help.

Thanks again,

Not complicated here. Folders, folders below that, and on in a tree structre.

For example:

Washington Freedom
Washington Freedom 2008
Washington Freedom 2007, etc.

Each sub-folder gets the same sortable name: 2008, 02, 25, Freedom vs Wildcats

Save the CR2s only long-term except for portfolio shots, save the tiffs also for those. Save the jpegs and tiffs for maybe six months till I can reasonably sure most orders are done. I rename image files as they all are nested in folders and don't mix.

I do a lot of sorts when processing by different metadata tho, mostly the time of capture. Since I shoot with two and sometimes three cameras, mixing the files later is important for sequencing. That is when the Bridge appending of at digit on the front end of the time name is useful, for posting the jpegs for sale.

I've tried in the past to rename files and find is annoying to work with later. Using Bridge file rename to add a number the front of a jpeg during conversion for posting to the web. But having the orginial CR2 name in the new file name makes finding it later to process for a sale much easier...IMO.

I save my image files by the date taken and what is included (example: 8-10-07 Rock park or 8-14-07 Smith Session) this makes it easier for me to find what I need easily. I also include in each folder an EDIT folder so that I can keep my reworked images separate from my original SOOC shots.

Now to just get last years images off my computer and onto discs - lol

FYI, Lightroom allows you to sequence images in the browser regardless of the filename. Then, upon export, you can have it assign a sequence number based on the order in the browser. I think. I can't say I've actually done this, but it appears that is how it works...

Ingesters

Hi All

No one has mentioned Photo Mechanic which has the most powerful ingestion routine of all the apps I have looked at. I suspect many people do not look 'under the hood' but there is virtually nothing PM cannot do when it comes to renaming, folder generation, metadata templates, backup etc.

It even has a very clever code replacement feature whereby you can assign short text strings to commonly used words/names - think of gridiron players' shirt numbers and their corresponding full names - type in the shirt number and the full name is inserted in the metadata.

PM also opens images super-fast. Editing stars as soon as the first images are ingested and using DNG/RAW files I can step through 100% previews almost as fast as I can press the keys. There is a good reason Sports Illustrated use this software for their editing.

It's no DAM hovever; for that I use iView and it's Folder Watch facility. Set it up first, use Photo Mechanic to ingest to the watched folder with metadata templates applied and iView will catalog those files as they are copied off the card. Two steps in one.

RAW work is done in Bridge with ACR4.1 or Peter Kroghs RAW settings tool for Bridge. Lightroom is software I am trying to like, it's an excellent RAW converter but a rubbish browser. I cannot abide it's attempts at being a DAM - give me keywords and DNG files any day.

BTW, John B has scripts for iView to copy metadata from IPTC fields to keywords. I use mostly location as my criteria at it takes very little time to copy the 4 IPTC place fields to the keywords.

Thanks so much Nick! How do you catalog in iview? Do you have different catalogs by year or one massive one or by topic? I have Catalogs by Topics and also an overlapping set by Hard Drives. I find it can cause crashes if I try to make just one huge catalog.

Concerning Photomechanic, you are right, we are not mentioning it enough! I wonder how that's happened? So certainly, we should all look closely at Photomechanic. So let's provide the links for the stuff we are talking about:

Photo Mechanic is a photo "browser" designed with speed and ease of use in mind. You can call it a "photo editor" as well. But unlike other photo editing programs that are designed to work with one photo at a time in order to edit its pixels, Photo Mechanic is designed to work with groups of photos together in order to manage them. Professional photo editors who handle lots of photos daily, often under deadline, understand this distinction. Their job involves selecting photos, not selecting pixels.

Let's tell folks where to look:

Click and get a trial version or buy $150 for Mac or IBM!

Everyone should have read Peter Krogh's book, even if you are not going to follow his advice!

click to order, I get no percentage

PM is the grandfather of smarts in initial image handling, sorting and backup. Peter Krogh includes it in his workflow diagrams throughout his famous DAM book. Here's chapter one for free, courtesy Peter himself, Download Chapter One here It's beautifully illustrated. I have two copies. Also there's a video series:

Click to order, not here, either, unfortunately

for $99.95 covering Adobe Bridge, ACR, DNG, Iview MediaPro and more!

Back to Ingester: It also does file copying verification. does PM do this?

It's free because I want to get the widest possible use; I can make money on ImageIngesterPro and ImageVerifier, and hope to sell lots of upgrades. There's no catch—no donations, advertising, upgrades, trials, or tie-ins. (IIP starts as an IIP trial, but you can switch the app to free II, which isn't a trial—it's really free.)

Click to visit the website, after all it's free and there's other great software there too. "Verifier" is worth looking at by itself!

Anyone, but does Ingester do anything better than PM??

Asher

Asher Kelman said:

Thanks so much Nick! How do you catalog in iview? Do you have different catalogs by year or one massive one or by topic? I have Catalogs by Topics and also an overlapping set by Hard Drives. I find it can cause crashes if I try to make just one huge catalog.

Asher

Click to expand...

I use one big catalog for all my images. I don't use image previews, only thumbs, and my catalog of 35000 images is about 360Mb. If you include previews the file size blows out, but you must have all images online if you want to have access to a bigger image in iView. In which case I would have catalogs by year; IView can search across multiple catalogs at once so this is not a big deal.

I generally catalog by location using the place finder but this does not matter if you use full IPTC metadata and key words. You can display the catalog in any way you choose - where the files are is irrelevant, hence the flexibility of the bucket system.

Incidentally I had been using a very similar system for about a year before I read Peter's book - it was affirming to see I had been working along the right lines!

Not sure if Photo Mechanic 'verifies' a copy exactly, it has yet to fail to copy after 3 years of using it exclusively so I guess it's as fool proof as you can get.

Nick

Hi Nick,

I'm thinking of all the miles you must have trudged with your Ebony camera on your shoulder, ready to use. To get to 350,000, I guess that the last 300,000 must be digital?

Asher

That's 35,000, not 350,000 !!

Prolly 34000 digi and 1000 scanned film. Lots of un-scanned film though.

N

Well 35,000 is still a lot!

For the digital landscapes, what do you use? As you might have noticed, we're interested in professional choices in digital for large images.

Also, I wonder whether you look at older film images differently now you can scan images and have such wonderful editing tools.

Lastly, what scanner do you use?

Asher

Asher Kelman said:

Well 35,000 is still a lot!
For the digital landscapes, what do you use? As you might have noticed, we're interested in professional choices in digital for large images.

Also, I wonder whether you look at older film images differently now you can scan images and have such wonderful editing tools.

Lastly, what scanner do you use?
Asher

Click to expand...

Currently using a 5D with 24TS or 17-40. Getting excellent results with Autopano Pro or CS3. Soon to be a 1DsM3. I can get perfect 100cm wide prints out of this setup.

My commercial work for magazines comes up really well at double page spread size so my clients are happy.

Latest shoot here, a gallery I did for my client:

http://www.nickrains.com/galleries/AG2007/index.html

No panos, these to follow. Here is a gallery with some panos, last three particularly.

www.nickrains.com/galleries/KimberleyCoast/index.html

Also some from Papua New Guinea before Xmas.

http://www.nickrains.com/galleries/PNG2007/index.html

Scanning.

I use an 'old' Imacon Flextight Photo - 275Mb from 6x12 film. Plenty good enough. I also have a Nikon 4000ED for my 35mm slides.

Cheers

Nick