When it comes to archiving an edit, especially a large documentary edit, the archive struggle is real. I recently wrapped up a low-budget documentary edit, and near the end, I brought up the archiving of the project and realized there wasn’t a plan in place. As an editor for hire, I don’t do a full media archive of all the projects I work on. While I do archive masters and sub-masters of most projects, archiving all of the raw media just isn’t possible. Big edit jobs these days often have multi-terabytes of media, so offering an archive service is not something I’ve elected to offer.
When discussing the end of this recent edit, I offered the client a few thoughts about archiving options.
- Cheaper kinda scary way – No real archiving of edit media. We generate the Master files for you and your client (I’ll keep a copy of those too) and then I delete all the media from my server. I can send you a lot of the supporting files so if ever you needed to open the edit, in theory you could using your backup drives, but it’s never that easy.
- Just ok – you make sure you can open a final Premiere project of just the final color-graded and audio-mixed master. You need to have all the graded and mix media for that. I would need to generate that Premiere project for you.
- Good – ship me a hard drive and I can mirror the working drive with a backup, confirm we can open the project from the archive drive and then I keep that on my shelf for an offsite archive or send it back to you.
- Better – The same as good but send two archives drives and I keep one, you keep one so there are two backups that live in separate places. Or I send both back to you and you keep one and send the other elsewhere so they live separately. (I’m happy to keep one in case there are changes or versions needed in the future)
- Best – Archive on LTO tape which is the gold standard for archiving (that service is offered in my building but there are others). That’s the most expensive option but very safe.
- Perfect and better than best – A perfect archive is both LTO for long-term archive and then the Good option. That’s rare, though. 😊
In the end, the better option was chosen, which didn’t surprise me as that’s a great, affordable option. That’s the option that most productions seem to choose. While I didn’t do any hand-holding during the selection of the archive drives, what was ultimately sent to me perhaps made this better option more difficult than it needed to be. Lesson learned: recommend the archive drives and charge a fee if different options are chosen.
Despite the SSD world we live in, there are times when spinning hard drives are still the right, or perhaps only, choice when it comes to archiving large amounts of media. This OWC article makes a good argument for spinning disks and tries really hard to sell you some OWC gear. The sticking point with this doc was that we had about 4.5 TBs of media that needed to be archived. Perhaps it could have been trimming down to below 4 TBs but that would have taken work that the production couldn’t afford. And since we were at the tail end of the budget, 4 TB SSDs were also cost-prohibitive.
In the end, I was sent two 5 TB Westen Digital My Passport spinning hard drives. Most editors would probably cringe when these things show up in their edit suite, but this wasn’t my money being spent, and in the end, you often just want to get the work done.
My first indication of trouble was when I saw the FedEx driver literally drop the box on my doorstep. But then I plugged the first one in.
Hear that? Not the sound you want to hear from a @westerndigital My Passport drive. These things are 💩 though. Didn’t help the @FedEx driver threw the box on my doorstep. pic.twitter.com/DR9u9hv280
— Scott Simmons (@editblog) July 26, 2024
If you don’t want to give TwitterX any extra clicks these days then give this a listen 🎧⬇️
That is indeed the last sound you want to hear from a spinning hard drive.
Since I was sent two drives, I plugged in the second one, and it worked. After a quick reformat from Windows ExFat I was ready to archive the media.
Archive it all, relink the media
My preference for an archive of this type is to archive the entire working folder, including the non-linear editing project and then relink the archived project to the archived media. Easy-peasy since all the edit media resides in a single folder.
While the file copy setup is easy, the My Passport drives are slow—very, very slow. Offshot is my file copy tool of choice, as it’s as fast and reliable as you get.
Yes, that’s 10 hours to copy nearly 5 TBs of media to a USB 2.0 hard drive from an iodyne ProData via Thunderbolt 4.0 on a Mac Studio. If there is ever a bottleneck, the My Passport drive is it.
When all was said and done, the Offshoot report told me exactly how long this took.
And the second archive?
It’s often said that without at least two archives of a project, you don’t have an archive. Since one of the My Passport drives was DOA, we needed to source another. Rather than ship another drive to me, I took advantage of what is probably the only good thing about a Western Digital My Passport drive: easy availability.
I do have to admit that $150 for 5 TBs is an amazing price, but I guess the time it takes to transfer that much media to such a slow drive has to be weighed. It would not have been my choice, but here we were in this archive process.
This 3rd My Passport drive was alive and well, and the file copy commenced.
After completing these two large file transfers, it’s a matter of copying the project file (in this case and Adobe Premiere Pro Production) to each archive and relinking each project to the media on that drive. That way, in the future, once everyone has forgotten everything about the edit, it’s easy to open up the project and get to an edit timeline without the reconnection dance.
And how do you charge for this?
For an archive such as this, I would charge a flat fee. I don’t think it’s fair to charge hourly for a 12 hour file transfer that happens overnight. But it is a service that should be charged for. Perhaps some of the weight of the lengthy archiving process is on me as I wasn’t specific about why drives should be purchased for archving. You would think that after all these years and thousands of GBs archived, I would have known better, but I did not.
While I will keep the archive drive myself, the other one has to be shipped back to the client, so there is a slight chance of damage during travel. I always encourage the client to plug in their archive copy when they get the drive just to ensure it survived the trip. Yes these are spinning drives and there is always the question of will a spinning archive drive still spin if you boot it years later. Usually, yes, but maybe no.
This discussion reminds me of some old articles where we discuss these very topics, still worth a read in our SSD days:
- Are you performing end of year hard disk maintenance? (controversial for sure.)
- A Q and A about those hard drive docking stations we all use
Uhh… cloud?
This article has neglected to mention online cloud storage as an archiving option. Yes, there are options out there, but it’s still seemingly the wild west. Imagine a few years of video projects, and you’re looking at 50, 100 TBs to live in the cloud for what could be decades. If you have an affordable cloud archiving solution for this that doesn’t require keeping drives plugging in, companies going out of business, changing their terms-of-service or breaking the bank, please let us know in the comments below.