Managing your files

Having brought back thousands of new images from the LuminOdyssey August African trip, I’m in the process of editing that collection down, and integrating the trip’s Lightroom catalog into my main system. In this process I also made some discoveries of “old” files on my system!

Cleaning up the mess (A2_099492)

My backups are based on file synchronisation: that is, fundamentally they mirror the source folders to the backup folders on other drives (some of which are later stored offline). These folders contain more than just the photo and video files that Lightroom manages, they also contain things like GPS track logs and audio recordings. Before Lightroom arrived I used Expression Media to catalog and manage these files, and prior to Lightroom 3.x I was still managing the video files with Expression Media.

My central Lightroom catalog currently contains over 125,000 photos/videos, dating back over 10 years. These are spread across multiple source “media folders” across several drives. I have more media folders than drives, mainly because I haven’t bothered to merge them. For example I might have had media-A and media-B folders (each containing aud, gps, img, and vid trees) each on separate 200 GB drives, but when I consolidated them onto a single larger drive, I didn’t need to immediately merge the trees and I just copied them onto the same drive. As long as the files have been imported into a single catalog, Lightroom (and Expression Media) lets me search/view/edit them as one set without having to think about which folder they actually reside in. Having separate media folders also allows me to have multiple Lightroom catalogs (each with separate folders) without getting into a mess about which catalog is managing each file.

As part of this cleanup work inside Lightroom I did a “Synchronize Folder” on each of these “media” folders, searching for new/missing files. As expected, I didn’t encounter any missing files, but across them all I did find about a hundred “new” files. They weren’t actually new, but they weren’t in the Lightroom catalog. The backups had been maintaining the files safely, but behind the scenes. It turns out these were mainly files which earlier versions of Lightroom had not supported and thus not imported (but my import process outside Lightroom had copied over anyway).

In my case almost all of these “new” files were immediately marked for deletion (I use a “To Delete” label inside Lightroom for that) as they weren’t actually all that useful. But it’s good to know that I don’t have any strange goings-on in my huge archive of images.

When doing maintenance like this, it’s important to not just blindly accept the results of the Synchronize dialog. In the example above, there are 104 images in the catalog with missing files, and 153 files without matching Lightroom records. But we don’t know if any of the “missing” files have just been renamed/moved and thus are part of the 153 “new” files. If we went ahead and let Lightroom throw away those images and then re-import them, it would lose any flags, collection membership, and virtual copies, and if the metadata hadn’t been saved in XMP form, it would also lose any keywords, develop settings, etc.

In this case I would use the “Show Missing Photos” button to make a temporary collection containing those 104 images, and either use the “Find Missing File/Folder” functions to re-link the files, or explicitly delete them. Then repeat the Synchronize function and import any remaining “new” files. I usually have the “Show import dialog before importing” box checked, again so there’s nothing surprising going on in the background. If those new files are things I don’t actually want to import, I do in fact import them, but I then go through the “Previous Import” collection to delete them (and when I delete them from Lightroom I get it to move them to the Trash/Recycle Bin). With files that may be scattered across multiple sub-folders, this is the simplest way to clean them up without making messy mistakes.

I also took the time today to merge several of the media folders for the sake of tidiness. This involves just moving folders/files inside Lightroom, but it does mean that the next backup will take a lot longer while it reorganises the contents of the backup drives!

Leave a Reply