Backups, backups

Every photographer should already have a good solid backup routine in place. Of course, every single computer user should have a backup system setup, but photographers are a slightly different breed since we honestly need large amounts of backup space. For a long time we’ve had various incarnations of NAS boxes storing our photographs with software RAID ensuring that every photograph is automatically stored on two physical drives.

HP ProLiant MicroServer and external USB drives

As a base, that offers a good solid approach that should work well enough. However, our first RAID suffered a couple of unexpected shutdowns due to power fluctuations caused by thunderstorms. As a result of one of these outages, the RAID got mixed up and couldn’t recover. It took me several weeks of work in the evenings to get everything back up to speed and in the end we lost only about 400 images (the Lightroom previews were saved though). But that was two weeks of time that could have been spent better.

Sure, at the time we made offsite backups on to DVDs, but that was already starting to be impractical (and is even more so now). Of course, because making the DVD backups was such a pain, at the moment I was a year behind in making the backups. Of course the images that were lost were from slightly after the time I had last made backups…

cd-rom and dvd disks

So external USB drives were then utilized to every now and then copy the data from our NAS to and then store in my office at work. At the end of last year our NAS at the time with 1.5TB of space was filling up and I bought some 3TB drives which naturally wouldn’t work in that NAS. So a friend and I both ordered HP ProLiant MicroServers (N40 to be exact) at an insanely cheap price – they are much more expensive now. Of course, I never got around to setting up our offsite backup systems decently once the new server was up (it’s called U96 ;) ). So now I finally got around to gathering up all of our external USB drives and am in the process of creating offsite backups of all of our images.

Inside of a HP ProLiant MicroServer

Of course, in addition to our photographs we are making backups of our computer systems as well. Anna’s computer stores Time Machine backups on a separate non-RAID drive on U96. My laptop mirrors its drive to an external drive automatically each day and my home directory is also backed up to CrashPlan (should set it up for Anna’s machine as well). Specifically – the CrashPlan backup serves as a decent offsite backup for both of our machines.

And on why we don’t use a cloud service as the offsite backup location of our photographs? Look at the table below with the amount of data in digital photographs we have for each year:

Year Amount of data
2003 2.2 GB
2004 14.1 GB
2005 45 GB
2006 45 GB
2007 50 GB
2008 80 GB
2009 160 GB
2010 220 GB
2011 390 GB
2012 425 GB
2013 (to date) 210 GB

Naturally the data shows some progression on the image sizes that each new camera produced (and the switch from JPG to raw is there somewhere as well). In late 2003 we got our first digital camera, a Minolta 3.2 megapixel point and shoot. In December of 2004 we bought a 300D, November 2006 a 30D, October 2010 our 7D. The final switch to all raw shooting happened in June of 2008. An interesting side note thus is that we initially used Lightroom with JPGs since we’ve used LR from the 1.0 days in 2007.

But back to why we don’t back our images to the cloud. Simply put – a 1Mbps upstream does not lend itself to speedy backups (automatically or manually) and recovering the data in case of failures would be an equally painful process. That’s also the reason why so far I’ve ditched all ideas of using semi-automated backups to a machine running at work and rely on manually transporting the drives to and from work.

That’s it for my rant of the importance of backups (and how we do ours).

Backup, backup, backup…

yellow flatcoated retriever eating grasses

Just keep repeating the title until you set up a good backup regime. If you already have one, try what happens when you imagine one drive just broke.

Still here? Then this post is not for you ;)

In all honesty, please build a good backup system where everything is not on one drive and in one device. Our basic setup so far has been two 1.5TB drives in a RAID-1 setup in a separate NAS box. Sounds technical right? Basically, we have a device hooked up to our home network that has two drives that show as one as everything is automatically mirrored on both drives. In addition to this, I periodically make backups onto an external 1.5TB USB drive. Which, *cough* in theory, should be stored offsite between weekly backups.

As I’ve mentioned, I’ve been going through last years images. While I was doing this I got (yet another) email from our NAS box telling me that the error count of one drive was increasing. I ignored it because at the same time I’m building a new image server for our house with more storage. Last Sunday, while Anna was editing photographs, our NAS died. Even after several reboots it did nothing.

So, out with the drive that was reporting errors and now everything was up and running. Except for the fact that I only had all images on a single working drive. So, frantically I hooked up the external drive to make sure that I had everything on at least two drives again. This moral of the story? Make sure you have at least two copies of everything that is important – you never know when a drive may fail.

BTW, both of our computers are continuously backed up to an external drive and I’m also in the process of setting up Crashplan on both our computers. Alas, I still don’t know how to sensibly get the image server backed up to Crashplan as 1.5TB is a bit slow to stream…

The picture above is one found during my archival romp and is from early 2012.