Like many people, I’ve struggled over the years to find the perfect backup solution for all of my digital content.  As file sizes creep ever upwards, it’s been a permanent upgrade process that’s often caused more than its fair share of headaches.  All drives will fail at some point, it’s an absolute certainty.  The only questions are when will it happen, and will it affect your business when it does?  You can’t answer that first question, but with some careful planning, you can make sure the answer to the second one is no.

Looking back on 2014, it’s the first time that I haven’t had to make any sweeping changes to my backup routine.  I think I’ve finally found the perfect solution so I thought I would share it with you guys, and talk about some of the ups and downs that have lead me to my setup.  I will talk specifics when it comes to various hard drive models, but in reality it’s more the process that’s important, and the lessons I have learned along the way.  You could substitute other drives into my setup and much of it would work in just the same way.

My love/hate/love again relationship with Drobo

When the very first 4-bay Drobos came out, it seemed like a perfect solution for a small creative freelancer.  The ability to add drives on the fly, without them needing to be of the same size or brand, was very appealing.  My first Drobo purchase cleared my desk of the familiar rats nest of cables and external drives and for a short while I was happy.  The Drobo gives your files redundancy with their version of RAID, and the whole thing seemed to be very simple, if a little on the slow side back then with firewire.  And then things started to go wrong… This was quite some time ago now and I must confess to no longer remembering the specifics of the issues, but they were plentiful and included things like needing to change power supplies seemingly endlessly, and my Drobo falling into a continuous loop of restarting. To Drobo’s immense credit, their customer service was always top-notch and after a string of issues, my 1st gen Drobo was replaced with a second generation one for no charge.  Sadly I continued to have issues and at some point in this whole process a Drobo completely failed on me.  This is where I learned my first big lesson when it comes to hard drives and RAID type systems.  Whilst your files are redundantly protected on the drives, if the actual unit fails, the box that you put the drives into, you don’t have access to your files until you can get your hands on another unit!  It seems obvious to me now, but until it happened, I didn’t appreciate what an effect this can have on your business if it happens at an inopportune moment.  I live in a fairly remote place, I can’t just pop to the store to buy another Drobo.  Nor would I really want to when they don’t come cheap and I’m within my warranty period anyway. 

A period time passed where all I had was the drives from the Drobo while they shipped me a replacement.  No data should be lost when swapping the drives to a new unit, but your mind sure doesn’t settle when you just see a pile of naked drives on your desk and not a happy little Drobo covered in green lights!  Eventually Drobo swapped out my 2nd Gen unit for the newer 5-bay Drobo S and I went and purchased a second one.  Not because I needed more drive space, but so that I could make a copy of the first Drobo.  I looked at various other options out there, but the Drobos still worked out to be the most cost effective solution for the amount of storage space that I needed.  With a second unit on my desk that was a perfect copy of the first unit, if one of the units died again, I would not have that period of time where I was awaiting replacement.  I now had my files stored redundantly, twice. I purchased a piece of software called Carbon Copy Cloner which gives you incredible control over drive cloning activities, and since then, Drobo 1 has been mirrored to Drobo 2 every night while I’m asleep.  I was starting to hone in on the perfect setup…

Over the years, I’ve had a few drive failures of the drives in the Drobos.  I keep a couple of spare drives in the office and if it happens, you just pull the dead drive out and slot in the new one.  I used the WD Caviar Green drives and I don’t think I had an abnormal number of failures.  It’s pretty much impossible to pick drives based on online reviews since the review scores always get skewed negatively by angry people.  When was the last time you bothered to leave a review of something that just anonymously worked perfectly in the background for years and years?  I settled on always using the WD drives simply to prevent decision fatigue and keep things easy.  By this time I was on my third generation of Drobo but I continued to have sporadic issues of various types.  They released a new Drobo called the 5D, the first to have a Thunderbolt connection, and after more conversations with their support team I was sent a pair of 5Ds for testing.  Understandably, at this point I was a little jaded, but the head of the company even called me up to talk about it and explain some of the sweeping changes that were made in creating the 5D.  Whilst it looked the same on the outside as the S version, it was an entirely different machine.  I agreed to run the 5D alongside my S for a while to “torture test” them and if they survived everything I threw at them, I’d give them a shot.

For a couple of months I put data on them, pulled drives out of them randomly, turned the power off while copying files and generally threw everything I could at the new 5Ds.  They passed with flying colours.  I couldn’t break them, I couldn’t even trip them up.  It was, as I had been told, like a different machine.

In the end, I’m really glad that I stuck with it.  Since that point, I’ve been immensely happy with the Drobo 5D and I regularly recommend them to people now.  It took them several years, and a few iterations to get there, but these are now excellent storage devices in my opinion.  I use them as my main media archives and Carbon Copy Cloner takes care of mirroring the two of them to provide me with double redundancy just in case one of the units ever fails.

Time Machine – Useful but not perfect

The Drobos take care of my media archive but there’s obviously a ton of important files on the computer as well.  I can only really speak to this next part from a Mac perspective, but even if you are a PC user, please read on because I also learnt an important lesson here as well.

I use a 4GB G-Tech GDrive as my Time Machine backup that’s plugged into my iMac.  Time Machine is a simple system and it’s saved my ass on more than one occasion where I’ve accidentally deleted an important document.  It’s pretty much a set-and-forget system but use it long enough and it’ll probably save you at some point.  My iMac is set up to boot the operating system and programs from a fast SSD, with an internal 4GB drive for all my everyday documents.  Time Machine takes care of backing up both of those drives and for the longest time, I thought this was a near perfect solution…

One day I got to my computer to find that the SSD boot drive had failed on me.  Let that be a warning to anyone that thinks an SSD is infallible because it doesn’t contain the moving parts of an HDD. Wrong!  My Time Machine drive had backups of all the data that used to be on the boot drive, but what was I going to do with it?  I don’t keep a spare $600 SSD lying around the office, and the dead drive was within warranty so I hardly wanted to purchase one from a store. The problem is that a Time Machine backup can restore data to another drive, but it is not bootable itself.  You can’t simply turn on your system and run it from the TM backup.  With a ton of work to do, and no computer to do it with, I’d learnt another lesson.  The warranty SSD replacement was going to take 10 days to get to me!

At this point I added another piece to my backup puzzle and it was Carbon Copy Cloner that quickly provided the solution.  CCC can create a bootable copy of a drive, complete with the recovery partition. After replacing my SSD boot drive I added another G-Tech GDrive to the office that CCC mirrors my boot drive to every night after it has finished mirroring the pair of Drobos.  If my boot drive ever fails again, I can simply boot the computer from this clone and be back up and running in two minutes as if nothing had ever happened.  Assuming you’ve had Time Machine running as well, you won’t even lose any documents that you created between the previous nights clone of the boot drive, and the time at which the boot drive failed.  To some people this might seem like a pretty over-the-top step, but imagine if your boot drive went down on a day where you are on a crazy deadline and charging your client a price in the hundreds, if not thousands of dollars for your work? 

Learn from my mistakes and misfortunes!  Always have a bootable copy of your computer files.  Call it bad luck, or maybe poor choice in SSD, but I actually had a second failure of my SSD boot drive in 2014.  Thanks to my system, I was back online in a few minutes and working off the bootable backup created by Carbon Copy Cloner.

Files in the cloud

This year also marked the start of my expansion into the cloud, specifically by using Google Apps and Google Drive for Work.  I began the year on a $1.99/month Google Drive plan that gave me 100GB of cloud storage but when the Drive for Work was launched with it’s unlimited plan for only $10/month per user, it seemed too good to pass up.  I quickly migrated all of my business documents to Drive and since they also remain on your computer, these get backed up by Time Machine and Carbon Copy cloner as well.  Backup in triplicate, with the added bonus of having the files available via the cloud from any other device I’m using.  Whilst the cloud space is unlimited in theory, upload speed prevent it being a useable offsite backup solution for large media files, but it’s still proven extremely useful to me.

Cloud + Carbon Copy Cloner

A neat little trick I developed is to use Google Drive and Carbon Copy Cloner together to sync system files between machines automatically.  Syncing files on Drive is no problem, but it will only sync files that are in your Drive folder.  What if you want to sync system files that are buried deep in other folders?  One such example is my font files.  I want to have the same fonts on all my machines so that they are available in Adobe programs like Photoshop and Premiere. The beauty of Carbon Copy Cloner is that you don’t have to clone entire drives, you can just select specific folders.  Overnight, my system font folder is cloned to Google Drive.  On my laptop I have another copy of CCC which clones the font folder within the Drive folder, over into the system font folder.  I always have the same fonts available on both computers, and I have cloud access to them if I need to load them onto additional machines.  I use the same technique to sync preference files, actions and presets as well!

Offsite backup

It’s important to have an offsite backup of all of your data to protect from flood, fire, and theft.  This year Google Drive was responsible for offsite backup of all my business documents and personal files, but that still leaves the media files.  This is always a tricky thing to deal with, especially with the size of 4k video files these days.  For most people, some form of automated cloud backup isn’t really an option due to limitation in upload speeds.  Certainly where I live in Canada, I can get some stellar download speeds, but faster upload speeds require an upgrade to insanely expensive plans.  Everything we’ve talked about so far happens totally automatically, but my offsite media file backups are still done the old(er) fashioned way with a Pelican case full of hard drives.  After every big job, I grab the case from its off site location and transfer files to one of the drives in the case.  I don’t use fancy drives for this, just naked drives that I put in a drive toaster on my desk.  Occasionally I’ll spin up all the drives in the case just to make sure they are working.

How do you handle your offsite backups?  I’d love to hear your solutions in the comment section because this is definitely the weakest area in my setup.

Backup plan summary

• Media files are stored on a Drobo 5D and that Drobo is cloned to a second Drobo every night to provide both double redundancy in the files and also redundancy in the hardware.
• Both internal drives from my iMac are backup up continuously with Time Machine to a 4TB G-Tech GDrive.
• The iMac’s boot drive is also cloned nightly by Carbon Copy Cloner onto another G-Tech GDrive which is bootable.
• All of my documents (personal & business) from the iMac are stored within the Google Drive folder so these are synced automatically to the cloud continuously.  This then causes them to be downloaded to my laptop as well when that gets turned on.
• Carbon Copy Cloner also runs various other nightly tasks to copy important system files onto Google Drive so that they can be easily synced to my laptop as well.
• Offsite backups are handled the old-fashioned way, with a Pelican case full of drives since this is, at the moment, the most cost effective way that I can think to do it.

A note on laptop backup

All documents on my laptop reside within the Google Drive Folder in a “Laptop” subfolder.  That takes care of backup and syncing for them, although I do also travel with a LaCie Rugged Drive that acts as my Laptop Time Machine. As for media files on the road… that’s an ever changing process depending on the size of the job, although I am very interested to check out the new LaCie Rugged RAID drives.  The downside of a built in RAID system like that is that you can’t split the drives up during transit.  At the moment I tend to use two identical drives (Carbon Copy Cloner to the rescue again!) and keep one on me at all times, while the other travels in my suitcase.  If I lose a drive or get robbed, I’ve always got the copy in my suitcase, or if the airline loses my bags then I’ve got my copy.

As I mentioned at the beginning of this piece, I feel like I’ve finally got this pretty much figured out.  I wish I had a slightly better solution for offsite backup, but I also have to maintain a realistic budget for all of this so I’m willing to put in a bit of leg work in that area at the moment.  Whether you use Drobos or some other similar RAID box, there’s definitely a few lessons I’ve learnt here that I hope can save you guys a few headaches.  If you’ve got some different processes in place that you are proud of, let’s hear about them in the comments below!