My recent image processing workflow in the field

Curious Walrus (A2_084372)
Svalbard

Having recently spent long periods away from home generating new photographs and video content, I’ve of course been practising and refining my in-field image management workflow. I’ve tweaked a few things over the last year, so here’s an overview.

While I’m leading photography expeditions I do spend a lot of time helping people make their own photos, but I also make a lot of my own photographs. Part of what I try to do is “lead by example”. So at the end of a day I might have made anywhere between several hundred and several thousand images. Regardless, by the time I sit down with expedition participants at dinner I’ve usually done my basic processing. Adobe Photoshop Lightroom is the core tool I use for this, and at the beginning of each trip I usually start a fresh catalog, which I will import into my central catalog when I return home.

First steps

The first thing I do with my images is import all my memory cards and initiate a backup (which synchronises the image folders to matching folders on external backup drives). My laptop has enough storage for all my trip photos on internal disks, so once the backup has finished I can eject the external backup drives and continue to work on the photos. I could store the primary copies of the photos on an external disk, but I prefer to reduce the number of drives I need to carry, and the internal drive is much faster.

While the backup is running, I have Lightroom generating previews for all the new images. Sometimes 1:1 previews, but for the sake of speed often just “Standard” previews. Depending on how many photographs I’m importing this can take a little while, so I’ll multi-task. I might be changing out of wet polar gear, having a shower to get rid of the dust, or even going to dinner. But usually it’s done by the time I’m ready for it.

Letting all this run quickly is one of the reasons I use fast card readers (Lexar readers which depending on the cards will get me 30-50 MB/s). If it’s been a busy session I might have a collection of 16 GB and 8 GB cards to import, and I use my own PteroFile software’s ingest function to allow me to copy/rename files directly to the hard drive and kick off the backups automatically. I can ingest multiple cards concurrently (although only one backup will run at a time). Once they’re all ingested (and the last backup running) I get Lightroom to import them (via the Synchronize Folder) function and start preview generation.

However this isn’t required, and even though some of the Mac users on my trips choose to use the backup component of PteroFile, it is simplest for many of them to just use Lightroom’s Import function to read in each card in turn, and just kick off a PteroFile Sync after the last card is imported. Overall the process is slower, but it works.

On ship-borne expeditions where we might have several landings per day, I’ll do this process each time I get back on board. On driving expeditions (e.g. African safaris) where we often have a morning drive and then another later in the day (or sometimes one huge day out) I’ll do this process each time I get back to camp.
So come lunch or dinner time (or anytime we have some “down” time) I should have my laptop with all my new photos imported, and know that it’s all been backed up to my main backup drive (I’ll mention an extra backup drive further below). If I need to go straight back out again and put more images onto the cards, I’m confident at that point about formatting the cards. But if I get a chance I’ll start on the image editing work.

Editing workflow

This happens in several passes (and in no particular order). On the whole my workflow is set up so I can be interrupted at any point and smoothly resume later. This helps in the field, and it also helps at home when I’m working through a large volume of images.

Aladdin’s Ice Cave (A2_087569)
(in the face of a Svalbard glacier)

Image development and assessing quality
For this I’ll pick out “good” images by giving them star ratings.

My system for “star” ratings is simple, starting off with the fact that an image with zero stars has not yet been rated (rather than implying it isn’t worth any stars). 2 stars means it’s good (1 star means it might be good). 3 stars and upwards imply various levels of “better”.

If I find an image that should be deleted I just apply a “To Delete” label (red in my system) and move on to the next image. Every now and then I’ll select all the images with that label, review, and delete in a batch (this helps avoid accidental deletion).

Note that I often don’t get through rating every image during a trip. I will often come back with lots of 2-star images, many unrated, and a scattering of others. I have to have something left to do when I get home!

I’ll get Lightroom to display all the photos since my last edit (usually either by selecting date-based folders or through date-filtering the “All Photographs” collection). Because I have already generated previews for all the images, flipping through them in the Library’s Loupe mode is very fast, and I can quickly assign ratings and labels with the keyboard as I race through them. Occasionally I’ll want to zoom in to check fine detail, and that’s made much easier if I’ve let the system pre-generate 1:1 previews. And occasionally I’ll hop over to the Develop module to tune exposures and cropping (for the sake of speed it is important to switch back to the Loupe mode before continuing to the next images). In some cases I’ll make white balance adjustments and then synchronise them to multiple photos (and then I might re-generate previews) but on the whole I don’t spend a lot of time in the field fine-tuning the development of photos.

It really doesn’t take long to run through hundreds of photographs just picking out interesting shots and throwing away the obvious duds. If I’ve been interrupted in this task it doesn’t take long to find my place again.
Keywording
I assign many keywords to my images, and have a huge hierachical dictionary of keywords in my Lightroom catalog. Participants in the LuminOdyssey expeditions receive a subset of this to import into their own catalogs or Aperture libraries at the start of the expedition.
My keywords cover multiple areas, for example actions like “fighting”, descriptors like “glacier”, animal species names, and people’s names. So far I haven’t been trying to completely keyword all my images during a field trip, but I find I can make big in-roads on the job, and it helps me manage the photos during the trip. Two areas I try to complete keywording for in the field are animal and people names. In this way at the end of a trip I can quickly extract all the photos I’ve taken of each participant and provide them with JPEG copies to take home. It also helps me focus my later image processing to view things like “all my hippo photos”.
I have two “workflow” keywords to help me manage this: “√animal species” and “√PIK: People I Know“. The funky √ character at the start makes it easy for me to type into the keyword fields (as I have very few keywords that start with this, the auto-completion works well): the magic keystroke for this on OS X is Option-V. I based this system on an idea by Eric Scouten. I have two Smart Collections in the catalog which put these to use: “Might have people I know” and “Animals not IDed“.

As you can see, the collection will contain anything by me which does not have the √animal species keyword. The Might have people I know collection is similar. I include the rule about Creator because I sometimes import photos from participants into my catalog, for things like group critique/processing and for end-of-trip slideshows (on ships where we also have non-photographers on board this is very popular). I don’t need to waste time keywording those photos too.
Assigning people keywords
I view the “Might have people I know” collection and can quickly assign the √PIK keyword to all photographs without people in them. On trips where I have hundreds of consecutive landscape or wildlife images this is very quick to do: select them all then assign the keyword. As the keyword is assigned, they disappear from the smart collection. Then I can go through and assign participant names to the remaining photographs. When I’ve assigned keywords for every person in a photo I also assign the √PIK keyword (again this can be done in groups). Ideally the number of images in the people collection should approach zero very quickly, although I often end up leaving photos of mixed groups until I can be bothered sorting out the details.
At the start of the trip I’ve already put a group of keywords into the dictionary containing each person’s name (correctly spelt) with each keyword set to not be exported. So to label an image containing Fred Smith I just need to start typing “Fre” and the keyword completion will show Fred’s name as an option. By providing these keywords to everyone at the start of the trip we avoid people mis-spelling the names of strangers, and by marking the keywords as not to be exported, we won’t have people’s names unexpectedly appearing as tags on anything uploaded to online services. If there’s a name that wasn’t in the list (e.g. the name of a local guide) we just have to type it once and it gets added to the dictionary (we can later drag it to the appropriate position in the hierarchy, etc).
Eagles, vultures, and one of our vehicles
(including visible people)
Animal keywords
Similarly I provide keywords for most of the animals we expect to encounter, grouped by animal type and with synonyms so that scientific names, etc are automatically inherited. I use this hierarchy in my own work, and by providing it to participants I’ve found it provides a concrete example of the uses of keyword hierarchies. There are usually new animals encountered during the trip which we don’t have keywords for, but they’re easy to add.

To keyword the day’s images, I view the Animals not IDed collection, and select groups of photos to assign animal names to. If an image has two species in it, it will be assigned both keywords. Once an image is keyworded, the √animal species keyword will automatically drop it from the collection. In fact one of the first things I do is assign this keyword to any photo without animals: usually a quick and efficient process. Very quickly, the only things left in the collection are either complex mixtures of animals (e.g. there can be many different scavengers clustered around a wildebeest carcass) or animals I can’t identify.
mystery Hornbill
(female Van der Decken’s Hornbill)

Note that I’m usually not bothered about image quality at this stage. Either I’ve already assessed this or I will get to it later. Because I’m usually applying a keyword to many selected images I’m hopefully not wasting much effort on individual keywords for images I will later delete. I am sometimes side-tracked into developing and rating particular images as I come across them, but it’s easy to get back to the job of keywording again.

I have a fairly extensive natural history background (to start with, I got into photography as a teenager through my interest in birdwatching) so I’m usually able to identify most animals. But sometimes I can only identify it to the level of type (e.g. “hornbill”). In some cases I might explicitly assign the hornbill keyword (the “parent” of the hornbill species names in the dictionary) but in general I don’t assign the √animal species keyword until it’s identified. It was very convenient on the recent African trip to sit down with our local drivers over breakfast and skim through the unidentified photos with them to assign names before I’d forgotten the details of the previous day’s experiences.
Without these “marked-complete” keywords and the matching smart collections, I would find it hard to stay on top of keywording the images each day. But with them, I brought back >11,000 images from Africa with less than 100 photos unidentified, and many of the animal groups already edited down to just the “good” images.
Image review is important!
At the start of each day (or the start of each outing, if I’d looked at images in the middle of the day) I had a pretty good idea of what I’d managed to produce in the previous day. This helped me feel “in control” of my collection, but it also let me focus on my next outing. I know there are times I’ve heard people say “Naah, move on. We’ve photographed zebras already.” while I’ve had to say “I know we took lots of zebra shots yesterday, but today I want one that’s got the right bits in focus!” I encourage participants to review their own photos during the trip so that they have the best chance of going home with great images.
By then I would have looked at each image at least twice, having considered keywords as well as skimmed them for overall quality/content. By assigning 2+ ratings as I went I was easily able to share with my expedition participants an overview of my take of the outing (often over drinks before dinner, or around a campfire after dinner) by just applying a filter. We talk a bit about what we saw and photographed, and sometimes the conversation gets drawn into topics of composition and technique.

Trunk stretch (A2_089979)
Samburu Reserve, Kenya
And as an added benefit, as our recent African trip progressed I was able to prepare and upload web postings of several images for each day.

Backups

As described above, my import process includes backing up files to an external drive. Because it’s done via file synchronisation (rather than by Lightroom making a second copy of each file) I have the choice of importing all the files without the backup drive connected, and kicking off a separate backup later (although for efficiency and to allow me to safely format cards I prefer to get the backup done early).

This backup drive is currently a rugged 1 TB drive which connects via Firewire 800, often providing 50+ MB/s transfers without putting overhead on the laptop’s processor. It includes a separate partition for OS X’s Time Machine to manage backing up the rest of the machine (everything except my media folders, Lightroom catalogs, and “scratch” folders).

Having a backup of everything provides great peace of mind, but I do go one step further. I have another drive (750 GB) which is housed in a rechargeable Nexto ND-2700 enclosure. This can be mounted via USB as an “MS-DOS” hard drive, but also provides a backup to the laptop in that I could use it to copy files directly from flash cards to the hard drive. But its main use is to hold a second backup copy of the media folders and catalogs. PteroFile is configured to update whatever backup drives are connected when it runs, so all I need to do is connect the drive and start a backup: all the new/changed files since the last backup to that drive are then copied. The USB transfer rate is typically only 12-16 MB/s, so it takes longer to update than the main backup drive, but it can run in the background while I’m working. I usually update this drive every couple of days (and especially before I embark on airplane flights, as I prefer to get a fellow traveller to carry it in their luggage for safety).

I’ll talk about the safety of redundant backups in another blog post, as there are still some easy mistakes that can be made in this area.

Other parts of the process

I haven’t talked here about the geo-coding process, but at various stages during the trip I would also download and backup the data from our GPS units, and towards the end of the trips I start applying that to the photos. I also provide GPS data to the trip participants.

After the trip

Upon return home there’s a lot of work to do, but I can almost immediately respond to the requests of friends, family, and clients who want to see images from the most-recent trip straight away.

I still have development and editing to do on many photos (including throwing away many due to subtle errors). I populate more of the metadata, using the GPS data to pull up maps of each location in order to apply appropriate locality and landmark information (we usually don’t have enough network connectivity to do this in the field). There’s also a lot more keywording that can be done on the images once I’ve edited them down to a useful set.

With 11,000+ images there’s still a lot of work to do, but at least I’ve “broken the back” of the job by the time I return.

Overwhelmed (A2_018519)
Easter Island

I hope this article helps inspire some of you to review your own image-management workflow, both while on trips and while you’re at your home base. Running Lightroom on a laptop in the field can be a great enhancement to your workflow. Similar backup techniques to PteroFile Sync can be achieved with a variety of “file synchronization” software. Although the best hardware/software configuration still depends on things like your camera image size and your laptop capacity, a well-thought-out workflow can be very efficient.

Leave a Reply