With Support from Apple

(Page 1 of 2 pages for this article  1 2 >)

Thursday, December 18, 2008

Filed under: EditingPost Production

Do It Yourself 6TB Video Array on a Mac

vittiPhoto | 12/18

Because you can never be too large or too fast…

image

With high definition, multi camera productions and emergent video formats all the digital video rage, power video users are refocusing efforts on computer storage.  Last year, Norco Technology approached this author to write an article building and testing a storage system around the Norco DS-1220, a 12 bay, port multiplier SATA chassis for the Do-It-Yourself Mac video community.  A test regime to benchmark and field test using Final Cut Pro Studio 2 (FCP ST2) was devised.  In August 2008, Fini.tv brought me on board to edit one of four 1 hour segments

(1)

, build and manage a shared storage system for five FCP ST2 workstations. This project would ultimately create 3.9TB of material with the four programs edited simultaneously and delivered under a tight deadline.  The opportunity to build and test a secondary storage system of this caliber in a real post production environment was a rare coincidence and a valuable test.

The CHALLENGES

Drawing from the computer’s secondary storage, video ‘data’ differs from other kinds of data transfer.  Successful multimedia data transfer mixes large video files with smaller audio files delivered together in time.  A prime concern is balancing the system for the different file types and their timely playback.  Latency, dropped frames, and sync issues are not acceptable to editors or their collaborators. In their pursuit of picture lock,  these creative teams can devise solutions that push an edit system beyond it’s benchmarking, a recipe for failure.  The expectation is that a robust non-linear edit system should be capable exceed the edit burdens placed on it, but how can one measure this headroom?  When benchmarking isn’t enough, a real world test is just the ticket.

The fundamental task was to build an array on a Mac Pro, benchmark and field test the system with the edit and finish of 4 hours of broadcast content.  A MacIntosh Mac Pro dual quad core 2.8, the DS-1220 with 12 500GB Maxtor SATA drives and a Highpoint Technology’s RocketRAID 2314 Host Bus Adaptor (controller card or HBA) comprised the test rig.  The field test consisted of a healthy mix of strait interviews, two camera, and film-style reenactments. imageThe target codec was DVCProHD 720/59.94, a sprightly video format requiring 100 Mb/s per stream.  The sequences were onlined, titled and finished in Final Cut Pro Studio 2, round-tripped through Color for color correction, and delivered on HDCAM at 1080i 29.97 to the network’s QC department.  Starting from scratch, we had roughly six weeks to complete 4 one hour programs, and to increase our blood pressure, the four programs were aired on consecutive evenings. 

The Arizona MacIntosh User’s Group (AMUG), a prominent user group with an extensive storage systems review archive, gave the DS-1220 great marks.  Tested with a variety of controller cards including the Norco/Silicon Image, Sonnet Technologies and Highpoint Technologies, the AMUG provides deep benchmark data for each in ten percent volume filled increments.  AMUG’s reviews also contain useful construction details and comparisons. This test began with benchmark testing using the Norco controllers 4629 and 4618, but as the new field test emerged, the Highpoint RocketRAID 2314 host bus adaptor was chosen for our field test.

A 5.0 Terabyte, striped array was specified with media file backup on secondary volumes and project backups on local workstations.  System benchmarking employed the AJA KONA System Test (2GB file sweep) for read/write speeds at four stages as the array filled from 1.5 TB to 3.9TB. The file sweep function provides nine different video frame sizes and at 2 GB file size provides more representative drive speed results.  For comparison, the system test application averages read/write speeds for the frame sizes and this average was reported.  The AJA KONA System Test application is included with Kona and IO products as well as available as a free download which does not require AJA Video hardware installation.

Planning DIY

There are several considerations for the do-it-yourself terabyte scale storage system. The chassis technology used in this test is SATA Port Multiplier (PM) which aggregates several fast and inexpensive SATA drives on each host bus port. 

Beginning with the host computer, identify the PCI bus the controller card will inhabit.  imageA G-5, Mac Pro or XSERVE can fulfill the host computer role most handily

(2)

. Consider an HBA based on the desired RAID functionality and PCI card format.  Most HBA cards come in both PCI X and PCIe card formats reducing card format as a mitigating factor. For instance, Highpoint Technologies offers three card classes:  the value line, an enterprise level offering additional RAID functionality and other features and a third line utilizing older equipment (ie, PCI). Select an array chassis that offers enough drive bays to create the performance and volume size needed.  Table one generalizes the performance of RAID0 arrays with 2-12 drives.

Fewer drive devices in an array generally means lower performance, and safe RAID configurations can eat into performance or volume size through lower efficiency.  The DS-1220 has port multipliers on bays 1-10, but 11 and 12 are strait connections, one cable per drive device.  Confirm that the HBA and the chassis are compatible, the Highpoint Technologies website lists host card and some chassis compatibility.

TIP: base your device selection on 500 to 750 GB drives.  These offer the best value (¢ / GB) and the flexibility to scale your volume up or down by changing with larger or smaller devices, which are readily available.

With a chassis selected, choose drive devices that are compatible with your HBA, and base the device size on the array size needed.  When estimating the array volume size, the usable volume size is roughly 8% less than the stated device capacity.  On top of this, most RAID arrays can chew up 3% (striped fast), 40% for parity schemes and 53% (mirrored safe) in net volume capacity.  For example, our test production estimated the four programs would use 4 TB of space total.  A 10 disc array was built using 500GB devices which yielded, when striped, a usable 4.55TB volume.  Be aware that SATA devices read/write performance degrades as the drives fill. Stored data builds on the platters from fast outside to slower inside sectors. With 40% or less capacity remaining, disk performance drop may be significant on a single SATA device.  Striped RAID0 arrays seem to mitigate this performance drop, but it’s prudent to be aware of this behavior.  As SATA

(3)

drives fill, you may experience a rapid drop in read/write speed. 

TIP: Controller cards are generally equipped with mini SAS 10Gb/s external or eSATA connectors, chassis use eSATA or infiniband connectors.  Be sure to acquire the appropriate cabling solution.

image

There are more cabling options today with SATA, infiniband, internal (4x) and external (10X) mini SAS connectors.  eSATA connectors are easier to disconnect than other connector systems, a consideration for a system in a busy location.

The ARRAY

In addition to the DS-1220 chassis, the kit as tested shipped with the Norco 4629 controller card, 4 eSATA cables, 12 drive trays, mounting screws, and a screw driver.  4 screws mount each drive to a sled, times 12 sleds, plan on spending a little preparation time mounting drives.

The DS-1220’s chassis design is strait forward; a 12 slot, 3U rack mountable solution with front loading, vertical drive trays, three cooling fans and a 360 watt power supply. The backside sports 4 eSATA connectors and two power toggle switches; the top is a power standby, below the power cord is the main power switch.

TIP: Use all four screw when mounting drives to minimize unwanted vibrational noise. After the first 40 hrs or so, check the drive mounting screws, some will loosen up through use.

The chassis is pretty massive, at around 35 lbs, and deep (about 20”), it may take two to muscle this unit into a rack.  Installation in this rack was hampered by the vertical handles partially blocking screwdriver access to the mounting screws. Careful alignment of chassis and rack rails may help. Once the chassis is mounted in the rack, the front loading drives make it pretty easy to load up.  imageWith the drives and rack mounted, the sleds click into the chassis with a reassuring physical click, as well they require a positive effort to open and disengage.  Each sled has a green activity LED.  Two more LED’s indicate the chassis status representing fan and temperature alarms. Installing the RAID management utilities (referred to as drivers), connecting the eSATA cables and powering up the unit and our array is good to go.

TIP: For non rack installations, the chassis comes equipped with rubber feet, but only for position 1, it’s lowest profile. Standing up on it’s short side (position 3) can work, the DS-1220 is about as wide as a Mac Pro is tall.

Serial ATA port multiplier spec provides for up to 15 devices per SATA host port

(4)

and is likened to a USB hub in function.  imageOn the RocketRAID 2314 card, the four host ports can support up to 5 devices each.  The Norco DS-1220 chassis has 4 eSATA connectors and the host bus controller cards used all had eSATA connectors.  With 12 device bays in the DS-1220 chassis, here’s how the device bays map to the ports: bays 1-4 on port 1, 5-10 port 2, bay 11 on port 3 and bay 12 on port 4.

The host bus controller ports map to the DS-1220’s PM backplane connectivity offering a variety of host options.  One option is passthrough mode where drive devices are seen as individual volumes by the computer and managed using the Apple Disk Utility.  This usually yields lower read/write speeds

(5)

. Optimizing data speed and security entails RAID management through the HBA. 

There are a variety of RAID management solutions for controllers range from hardware dip switches, scripts, applications or browser based management utilities.  The Norco cards use a javascript RAID management utility where the RocketRAID 2314 uses a browser based solution.

TIP:  Check the HBA’s support or downloads webpage for the drivers that match the host computer’s operating system version.  SATA drivers can been more closely tied to OS development, older drivers on newer OS versions might result in stability issues.

For the field test, we’ve installed the DS-1220 chassis, RocketRAID 2314 controller, drives, RAID management utility, cabling and we’ve created two RAID arrays: a 10 disc striped (RAID0) array and a 2 disc striped array

(6)

.  Each array appears on the desktop as a single volume, accordingly, the 10 disc stripe is approximately 4.5TB and the 2 disc stripe a tad under 1.0TB. 


1. deliverable was 45 minutes per 1 hour program segment
2. Apple tech specs:  http://support.apple.com/specs/
3. The good news is SATA II device specification is currently 3GB/s bandwidth, is spec’d to attain 6Gb/s for 2009 and an impressive12GB/s by 2012.
4. the host port, limited to 3Gb/s generally will reach saturation with 3 drive devices
5. A 10 disc striped array software formatted array yielded 117.4/165.8MB/s read/write,the hardware formatted array yielded 284.4/529.2MB/S
6. See the Computer Specs for hardware and software details.

(Page 1 of 2 pages for this article  1 2 >)

               


Name:

Email:

Location:

URL:

Smileys

Remember my personal information

Notify me of follow-up comments?

 
Blackmagic Design Releases Support For Final Cut Pro X 10.0.3 Broadcast Monitoring
PVC News Staff | 02/01

Broadcast monitoring in Final Cut Pro 10.0.3 allows video output to external monitors and other equipment

image

Blackmagic Design today released Desktop Video 9.2 beta 1, a software update for its capture and playback products that adds broadcast monitoring support with the new Final Cut Pro X 10.0.3 update.…

Update Alert: Final Cut Pro X goes to 10.0.3
Scott Simmons | 01/31

A big update adds multicam, manual relinking, broadcast monitoring and the ability to move a project over from FCP7

image

Is it early 2012? It is and Apple has kept its promise with an update that takes Final Cut Pro X to 10.0.3. The promised features…

Upcoming Webcast: Moving to Adobe Premiere Pro and Using an HP Workstation
PVC News Staff | 01/13

Award winning editor talks about switching from Final Cut Pro to Adobe Premiere CS5.5

image

Creative Director, Producer, and Editor Christine Steele talks about her journey switching from Final Cut Pro to Adobe Premiere CS5.5 and discusses the powerful performance…







Copyright © 2011, HD Expo, LLC a division of Diversified Business Communications. DBA Createasphere

All rights reserved. HD EXPO, High Def EXPO, Createasphere, E-Tech, Entertainment Technology Exposition, 3D Production Workshop, VariCamp, P2 Camp, ColorCamp 101, and Lighting, Filters & Gels for HD are all trademarks of HD Expo, LLC.

Terms of Use  |  Privacy Policy

Check PageRank