The Image Ingestion Phase

Transferring your images from the camera to the computer is one of the most critical yet often one of the least examined stages of your workflow. It's critical because at this stage, your images exist only on the camera media. It's not that Compact Flash, Secure Digital, or microdrives are dramatically more fragile than other storage media, it's simply that there's only one copy! Losing previews or camera raw settings is irritating, but you can redo the work. If you make mistakes during ingestion, though, you can lose entire images.

The following ground rules have stood me in good stead for several yearsI've had my share of equipment problems, but thus far, I've yet to lose a single image.

• Don't use the camera as a card reader. Most cameras will let you connect them to the computer and download your images, but doing so is a bad idea for at least two reasons. Cameras are typically very slow as card readers, and when the camera is being used as a card reader, you can't shoot with it.

• Never open images directly from the camera media. It's been formatted with the expectation that the only thing that will write to it is the camera. If something else writes to it, maybe nothing will happen, but then again, maybe something bad will.

• Don't rely on just one copy of the imagesalways copy them to two separate drives before you start working.

• Don't erase your images from the camera media until you've verified the copiessee "The Image Verification Phase," later in this chapter.

• Always format the cards in the camera in which they will be shot, never in the computer.

Following these rules may take a little additional time up front, but not nearly as long as a reshoot (assuming that lost images can in fact be reshot).

Camera Media and Speed

All CF cards (or SD cards, or microdrives) are not created equal, but vendor designations like 4x, 24x, 40x, 80x, Ultra, and Write Accelerated aren't terribly reliable guides as to the performance you'll get with your personal setup.

There are two distinctly different aspects to CF card speed.

• Your burst shooting rate is dictated by the speed with which the CF card writes images in the camera.

• Your image offloading speed is dictated by the speed with which images can be read from the CF card onto your computer's hard disk.

In either case, the bottleneck may be the CF Card, or it may be the hardware used to write to it (your camera) or read from it (your card reader).

Compact Flash write speed

Most of today's high-speed CF cards can write data as fast as the camera can send it. However, older cameras may not be able to deliver the data fast enough to justify the premium prices the fastest cards command.

The best source I know for comparative data on different cameras' write speeds to different cards can be found on Rob Galbraith's Web site, www.robgalbraith.comlook for the CF Database link on the front page. Note that the database no longer gets updated for some older cameras, so if the notes say something to the effect of "this camera will benefit from the fastest card available," look in the table to check which card that actually was and when that page was last updated.

Compact Flash read speed

The card reader and even the operating system can play an equal role in determining read speed to that of the card itself. Card readers almost invariably use one of three interfaces: USB 1.1, USB 2.0, or FireWire.

Almost any card available today can max out the speed of a USB 1.1 reader. In theory, USB 2.0 is faster than FireWire, but in practice, as the EPA says, "your mileage may vary"I've generally found FireWire to be both faster and more reliable than USB 2.0, particularly with fast cards such as the SanDisk Ultra II and Extreme and the Lexar 80x product lines.

Mac OS X users should take note that OS X versions prior to Panther (OS 10.3) were very slow at reading 2GB and larger cards that use FAT-32 formatting. Panther fixed the problem.

Microdrives

In addition to solid-state Compact Flash cards, microdrivesminiature hard disks in Compact Flash form factorare also available. Microdrives were introduced when solid-state CF cards were still quite limited in both speed and capacity.

Today, solid-state CF cards have outstripped microdrives in both capacity and speed, and they also have enormous advantages in durability. Like all hard drives, microdrives use moving parts machined to very fine tolerances, so they don't respond well to impactsit's easy to destroy both the data and the drive itself by dropping it. Solid-state CF cards are a great deal more robustwhile I don't recommend abusing them in any way, I have one that survived being run over by a Ford Explorer!

Microdrives may make a comeback, with much higher capacities than before, but right now such designs are still on the drawing board.

Secure Digital (SD) cards

If microdrives are the wave of the past, Secure Digital (SD) cards are the wave of the future, though at the time of this writing only a handful or so of cameras support them. The main impetus behind the development of SD is the built-in encryption, which is inviting for the music and movie industries, since it will let them distribute copyrighted material digitally.

For camera use, SD is still pretty new, and relatively few cameras use itthe Canon EOS 1DsMkII lets you use both CF and SD, but the SD card seems intended as a spare for when the CF card gets full. The capacities of the current generation are still lower than the largest CF cardsat the time of writing, 1GB cards are common and we're just starting to see 2GB SD cardsbut the fastest SD cards are slightly faster in the camera than are the fastest CF cards, though they're considerably slower at transferring data from the card to the computer. Both of these statements are subject to change. All the recommendations for handling and using CF cards apply equally to SD.

Formatting Camera Media

Always format your camera media, whether CF card, microdrive, or SD card, in the camera in which it will be used! Your computer may appear to let you format the card while it's loaded in the card reader, but it's quite likely that it will either do so incorrectly or offer you a set of options from which it's easy to make the wrong choice.

Formatting CF cards on Windows systems can, at least in theory, be done correctly, but the only time I'd recommend doing so is if you've used software supplied by the card vendor to perform data recovery or diagnostics and the software recommends formatting. Formatting CF cards under any flavor of the Mac OS is a recipe for disaster. Formatting cards in the camera in which they will be used is always safe and guarantees that the format will be the one your camera can use.

Tip

When Disaster Strikes. If you wind up with a card that's unreadable but contains data you want to recover (it's rare, but it can be caused by doing things like pulling the card out of the reader without first ejecting it in software), do not format it! Doing so will guarantee that any data that was still on the card will be permanently consigned to the bitbucket. Major CF card vendors such as SanDisk and Lexar include data-recovery software with the cards (which for my money is sufficient reason to stick with those brands). Before attempting anything else, try the recovery software. If that fails, and the data is truly irreplaceable, several companies offer data recovery from CF cards, usually at a fairly hefty pricea Google search for "Compact Flash Data Recovery" will turn up all the major players.

Camera Card Capacities

Bigger isn't always better, and in the case of CF cards, large capacities often come at premium prices. A 4GB card will generally cost more than twice as much as a 2GB one, and so on.

Using two smaller cards rather than one big one offers an immediate workflow advantage. When the first card is full, you can switch to the second one to continue shooting while the first card is being copied to the computer. By the time the second card is full, the first one will have finished copying, and you can format it in the camera and continue shooting.

Ingesting Images

I always copy images onto a hard drive before attempting to open them. (Actually, I always copy the images onto two different hard drives. I may be paranoid, but I've yet to lose a digital capture.)

It's possible to connect the camera to your computer and actually open the images while they're still on the CF card. It's likewise possible to put the CF card in a card reader and open the images directly from the CF card. But "possible" doesn't mean it's a good idea! It's possible to run a Porsche on kerosene or to perform brain surgery with a rusty Phillips screwdriver, and I consider either one about as advisable as opening images directly from the camera media.

I always copy to two hard drives for the simple reason that hard drives break, usually at the least convenient moment they could possibly choose to do so. If you simply can't take the time to make two copies, consider setting up a mirrored (not striped) RAID array. Mirrored RAID arrays copy the data to two drives simultaneously, so unless both drives fail simultaneously (which is extremely unlikely), you'll always have a copy of the data. RAID level 5 (distributed parity) offers a good compromise between speed and reliabilitythe data is distributed with parity bits across three drives, so even if one drive fails, the array can be rebuilt (though it takes a while to do so).

You can even kill two birds with a single stone by using a casing that allows hot-swapping of the drives, and use the drive mechanisms themselves, suitably boxed, to archive the datahard disks are much faster than CD-R or DVD-R; will almost certainly last at least as long, particularly if they're simply being stored; and can cost less than a dollar per gigabytesee the next section, "Archiving Images."

When I'm shooting in the field and using a laptop to offload the images, I carry a portable 100GB FireWire drive, and make a second copy of all the images on the portable drive. When time permits, I make both copies directly from the camera media. Failing that, I make one copy from the camera media, then copy from one hard drive to the other. When there's absolutely no time to spare, I may work from a single copy of the images, but in that situation, I always verify the images by letting Bridge build previews before I format the camera card from which they came.

However you choose to accomplish the task, my overriding recommendation is that you wait until the copy from camera media to hard drive is complete and verified before you try to do anything at all to the images.

Archiving Images

I've heard of photographers who don't bother to archive their raw images once they've processed them to gamma-corrected color ones. That seems about as sensible to me as throwing out all your negatives because you've made prints that you like! Given the huge amount of processing that goes into converting a digital raw capture and the fact that the techniques for doing said conversions are only likely to get better, it seems extraordinarily short-sighted at best not to archive your raw captures. The issues then become when, in what form, and on what media you archive them.

When to archive

When I first copy the raw images to the computer, I almost always copy them to two different drives. One copy becomes my working copy; the other serves immediately as a short-term backup and then, after conversion to DNG with the original raw file embedded, as a long-term archive.

Once I've done my selecting, sorting, ranking, and renaming, I've applied initial Camera Raw edits, and I've saved the files in DNG format to bind the metadata, I archive these too. Yes, it makes for a heavy storage requirement, but storage space is relatively inexpensive, time is expensive, and images are irreplaceable.

What to archive

You should archive anything you want yourself or someone else to be able to retrieve at some unspecified time in the future. It's really that simple.

Don't confuse archives and backups. Backups are usually automated, incremental copies that reflect the current state of your system. Archives are long-term storage, designed to remain undisturbed unless and until the data is required. An archive isn't a substitute for backups, and backing up isn't a substitute for archiving!

Archive media

Strictly speaking, there's no such thing as an archival medium for digital storageany of the even slightly convenient solutions available for recording ones and zeroes will degrade over time. Archives must be maintained!

There are really two problems in archival storage. The obvious one is the integrity of the storage medium. The less obvious but equally critical one is the availability of devices that can read the storage medium. There's probably still magnetic tape from 1970s mainframes that has good data on it, but good luck finding a drive that can read it.

Any archiving strategy must include periodic refreshing of the data onto new media, preferably taking advantage of improvements in technology. I've migrated most of my early-'90s CD-ROMs onto either DVD-Rs or to large-capacity hard disks, and unless something better comes along I'll probably refresh that data onto the even larger, faster, and cheaper hard disks that will be available in the future with capacities measured in terabytes rather than gigabytes.

Burnable CDs and DVDs, both read-only and rewritable, differ from commercially pressed CDs and DVDs in an important way. In the commercially manufactured disks, the data is stamped on a foil layer made of metal. (It's about the same thickness as the foil in a cigarette pack, but it's metal nonetheless.) Burnable CDs and DVDs use a photosensitive dye layer to record the datathe dye changes color when the laser writes to it. Photographers should be well aware of the fragility of dyes…. So use whatever storage medium you find convenient, but recognize that it will fail, and plan accordingly.