Quantcast
Proper storage for an FX library- need advice - Gearspace.com
The No.1 Website for Pro Audio
Proper storage for an FX library- need advice
Old 28th February 2014
  #1
Gear Nut
 
🎧 5 years
Proper storage for an FX library- need advice

Hi All,
Currently, I have about 1.2 TB of FX on one of my internal drives in my Mac Pro tower. It's backed up on a combination of the other internals and firewire drives.
I expect my library to increase and know this isn't an optimal setup. What's the best storage method for a large FX library? Should I put everything on a RAID? If so, what brand? Thanks!!!
Old 28th February 2014
  #2
Lives for gear
 
mikevarela's Avatar
 
🎧 10 years
Best option would be a RAID array. I'd look at RAID 5 for this option. You could use Raid 1 (A Mirror), and you'd get redundancy, but RAID 5 is a better option for more space and better speed.

I'd look at the Promise RAID boxes, they connect with multiple types of connections. Easy is FW 800, but best would be eSATA. You buy the box, and then you fill it with 4 drives. 1 Gets used for parity, so 4 1 TB drives becomes a 3 TB RAID.

I also like Synology, they're often cheaper and work well, but Promise has a rock solid pedigree.

SmartStor DAS for OS X
Old 28th February 2014
  #3
Gear Maniac
 
Sndnerd's Avatar
 
🎧 5 years
I was in a similar situation but was looking for a rack mount solution, I ended up going with Addonics: Addonics Category: 1U RAID Rack It was around $400, filled it with 4x4Tb RAID 5 FW800, haven't looked back.
Old 28th February 2014
  #4
Lives for gear
 
JSt0rm's Avatar
 
🎧 15 years
what you want is redundancy. I have a portable copy of my sound effects on a 4tb disk. I have a 16tb 1u raid box that hold my sfx plus all old projects and I have a cloud backup of 5tb of data. If my house burns down I lose no data.
Old 28th February 2014 | Show parent
  #5
Lives for gear
 
🎧 10 years
Quote:
Originally Posted by JSt0rm ➑️
what you want is redundancy. I have a portable copy of my sound effects on a 4tb disk. I have a 16tb 1u raid box that hold my sfx plus all old projects and I have a cloud backup of 5tb of data. If my house burns down I lose no data.
Where are you storing 5TB in the cloud and how much does it cost?
Old 28th February 2014 | Show parent
  #6
Lives for gear
 
JSt0rm's Avatar
 
🎧 15 years
Quote:
Originally Posted by SaschaP ➑️
Where are you storing 5TB in the cloud and how much does it cost?
crashplan pro. It took me like 2 years to be fully uploaded but they have unlimited storage for so little i can't even remember. I had to do some low level stuff to the software because my data set is probably the largest they are dealing with. lol.

If I ever lost all my personal backups I would need to order drives from them and that would cost me an amount but you get it back when you return the drives.

Online Data Backup - Offsite, Onsite, & Cloud - CrashPlan Backup Software
Old 28th February 2014
  #7
Lives for gear
 
JSt0rm's Avatar
 
🎧 15 years
Let me add I also use a time machine scheduler that does a auto back up every night at midnight to another drive. This is a copy of my project drive/os drive in its current state. My macpro is also set to wake at 9:30am and sleep at 1am.

The only thing I don't backup is my video drive. If they are coming to the sound effects editor looking for picture then something is horribly wrong
Old 1st March 2014
  #8
Lives for gear
 
🎧 10 years
Did you make large zip files or upload wav/aiff folders as they are on your hard drives straight up?
Old 1st March 2014 | Show parent
  #9
Lives for gear
 
JSt0rm's Avatar
 
🎧 15 years
Quote:
Originally Posted by SaschaP ➑️
Did you make large zip files or upload wav/aiff folders as they are on your hard drives straight up?
The software scans your system and does everything for you. It looks for duplicates so it won't upload the same file twice. You can exclude folders/harddrives if you want. I have it not uploading my video drive as thats a ton of data i don't need backed up but everything else is in the cloud. I've been pleased with crashplan from a tech and service stand point. Obviously its going to take a long time to upload this kind of data but once the majority is up its just keeping up with new stuff added. I also needed to do some command line stuff in the software as it needed access to more ram because my data set is beyond what any sane person would put up
Old 1st March 2014 | Show parent
  #10
Gear Head
 
🎧 10 years
Quote:
Originally Posted by JSt0rm ➑️
The software scans your system and does everything for you. It looks for duplicates so it won't upload the same file twice. You can exclude folders/harddrives if you want. I have it not uploading my video drive as thats a ton of data i don't need backed up but everything else is in the cloud. I've been pleased with crashplan from a tech and service stand point. Obviously its going to take a long time to upload this kind of data but once the majority is up its just keeping up with new stuff added. I also needed to do some command line stuff in the software as it needed access to more ram because my data set is beyond what any sane person would put up
does your FX drive have to be connected at all times for crashplan to back it up?

I use my FX drive via a hard drive dock and only connect it when I cut FXs, would crash plan still be able to back up my FX library?
Old 1st March 2014 | Show parent
  #11
Gear Guru
 
charles maynes's Avatar
 
🎧 15 years
Quote:
Originally Posted by mhuang ➑️
does your FX drive have to be connected at all times for crashplan to back it up?

I use my FX drive via a hard drive dock and only connect it when I cut FXs, would crash plan still be able to back up my FX library?
your drive must be online to backup to crashplan. If it is offline, it will be skipped in the process-

Crashplan takes forever (I have about 3.5tbs up on my account) but it is really inexpensive.

I also have 2 copies of my library on physical drives as immediate backup.
Old 3rd March 2014
  #12
Gear Head
 
airbornesound's Avatar
 
🎧 10 years
For the past year or so I've been using a Synology box for my sound library. I got the 413j, which is fairly basic, but works well. I stocked it with WD "Red" drives and they're working perfectly. Big fan of Synology - great software, support, and implementation. As a bonus, the unit is dead silent (which wasn't the case for other units I used).

I also backup to the cloud using a combination of the Mac app Arq ($29), and Amazon's S3 "Glacier" service. That's only a few dollars a month. Also been using that for a long time and it has done well.

I actually wrote about both on my blog, if you'd like more info.

Synology post:
Review: Synology NAS – A Turbo-Charged Way to Store Sound Libraries | Airborne Sound

Arq + Amazon S3 Glacier post:
The Cheap & Easy Way to Backup Sound Libraries | Airborne Sound
Old 3rd March 2014
  #13
Lives for gear
 
NReichman's Avatar
 
🎧 10 years
I tried Backblaze and Crashplan. Both of them gave me stress. I'm using Arq/Amazon Glacier now instead and am liking it more- more efficient, more robust, and not that much more expensive.
Old 3rd March 2014
  #14
Gear Head
 
🎧 10 years
Glacier is exactly what I've been looking for. Straight up archival storage - not a synching service, and no need for quick access. It's more like a safe deposit box model. Thanks!

-a
Old 3rd March 2014
  #15
Lives for gear
 
NReichman's Avatar
 
🎧 10 years
You'll also find that Arq is incredibly light on the CPU. It never interferes with work.
Old 3rd March 2014
  #16
Deleted e479b20
Guest
Quote:
Originally Posted by Xcurve ➑️
Hi All,
Currently, I have about 1.2 TB of FX on one of my internal drives in my Mac Pro tower. It's backed up on a combination of the other internals and firewire drives.
I expect my library to increase and know this isn't an optimal setup. What's the best storage method for a large FX library? Should I put everything on a RAID? If so, what brand? Thanks!!!
I'm sure you know this and it has been said elsewhere but RAID is not a backup. Even if you have a RAID you will still need to backup your data so the question is what would be the benefit of a RAID in your working situation vs a normal backup that you will need anyway?

RAID is for people who either need super-fast streaming (not the case with audio applications) or people who need 24/7 reliability and an option to swap drives without shutting down the system.

For one-person studios it's highly over-rated and it offers zero protection against user-error. The only thing it might protect you from are single drive failures. See here from wikipedia:

Quote:
Drive capacity has grown at a much faster rate than transfer speed, and error rates have only fallen a little in comparison. Therefore, larger capacity drives may take hours, if not days, to rebuild. The re-build time is also limited if the entire array is still in operation at reduced capacity.[55] Given a RAID with only one drive of redundancy (RAIDs 3, 4, and 5), a second failure would cause complete failure of the array. Even though individual drives' mean time between failure (MTBF) have increased over time, this increase has not kept pace with the increased storage capacity of the drives. The time to rebuild the array after a single drive failure, as well as the chance of a second failure during a rebuild, have increased over time.[56]
Some commentators have declared that RAID 6 is only a "band aid" in this respect, because it only pushes the problem a little further down the road.[56] However, according to a 2006 NetApp study of Berriman et al., the chance of failure decreases by a factor of about 3,800 (relative to RAID 5) for a proper implementation of RAID 6, even when using commodity drives.[57] Nevertheless, if the currently observed technology trends remain unchanged, in 2019 a RAID 6 array will have the same chance of failure as its RAID 5 counterpart had in 2010.[57][50]
Mirroring schemes such as RAID 10 have a bounded recovery time as they require the copy of a single failed drive, compared with parity schemes such as RAID 6, which require the copy of all blocks of the drives in an array set. Triple parity schemes, or triple mirroring, have been suggested as one approach to improve resilience to an additional drive failure during this large rebuild time.[57]
RAID has become a little bit of a marketing hype at least among consumer-devices like disk-stations that only do raid 0,1 and 5 etc. In reality a straight chronosync backup gets you more protection on the same ticket.

RAID is something for 24/7 enterprises that can not allow for a millisecond of downtime.
Old 3rd March 2014 | Show parent
  #17
Gear Nut
 
🎧 5 years
Quote:
Originally Posted by apple-q ➑️
RAID has become a little bit of a marketing hype at least among consumer-devices like disk-stations that only do raid 0,1 and 5 etc. In reality a straight chronosync backup gets you more protection on the same ticket.

RAID is something for 24/7 enterprises that can not allow for a millisecond of downtime.
So your suggestion is for me to continue to use my internal HDs and various FW drives vs. a RAID+cloud backup?
Old 3rd March 2014 | Show parent
  #18
Deleted e479b20
Guest
Quote:
Originally Posted by Xcurve ➑️
So your suggestion is for me to continue to use my internal HDs and various FW drives vs. a RAID+cloud backup?
well raid+cloud is good as is regular drive plus backup is as well. Imagine a worst case scenario where your raid is down and then you have to spend 1 month or more downloading that 1TB of backup from the cloud.

My point was that just a RAID doesn't equal a backup. So what ever you do don't rely on just one drive or one raid.
Old 5th March 2014
  #19
Lives for gear
 
🎧 10 years
RAID is not synonymous with backup but it is cheap insurance against downtime. With the tight schedules and turnarounds we all face I see no reason not to include a redundant array in your setup along with external backups and cloud solutions. All of these are vastly cheaper than tape and M-O backup systems of yore.
Old 6th March 2014
  #20
Gear Nut
 
brainspin's Avatar
 
🎧 5 years
Regarding backing up to a cloud, there's another solution that's arguably just as reliable and definitely more efficient: just keeping an offsite drive(s) you rotate out at regular intervals.

If you need to be regularly backing up more than 3-4 TB of data, then it could get more complex cause you'd need to rotate out multiple drives per cycle given their space limitations, so that's something to consider.
Old 6th March 2014
  #21
Gear Nut
 
🎧 5 years
Thanks for all the response. It seems most of the current RAIDs are thunderbolt only but I did find a few FWs. Does anyone have any noise issues with using a RAID in close proximity?
Old 6th March 2014
  #22
Lives for gear
 
NReichman's Avatar
 
🎧 10 years
Yes. My MacGurus' Burly eSATA RAID/JBOD enclosure is powerful, but too noisy to work in the same room with. I put it in a closet in the hallway.

Call MacGurus and they'll let you know which of their RAIDs are silent.
Old 6th March 2014
  #23
Lives for gear
 
TVPostSound's Avatar
 
🎧 10 years
Quote:
Originally Posted by Xcurve ➑️
Hi All,
Currently, I have about 1.2 TB of FX on one of my internal drives in my Mac Pro tower. It's backed up on a combination of the other internals and firewire drives.
I expect my library to increase and know this isn't an optimal setup. What's the best storage method for a large FX library? Should I put everything on a RAID? If so, what brand? Thanks!!!
I have 3TB of FX on a firewire. I have a copy backup.

Nothing more, been like this for 10 years.
Old 13th March 2014
  #24
Lives for gear
 
🎧 10 years
Heads up, Google dropped their "drive" pricing to $10/mo for 1TB. That's on par with Amazon Glaciers pricing with a much friendlier interface and no retrieval time limit penalties.

You Can Now Get 1TB of Google Drive Storage For Just $10 a Month
πŸ“ Reply
Topic:
Post Reply

Welcome to the Gearspace Pro Audio Community!

Registration benefits include:
  • The ability to reply to and create new discussions
  • Access to members-only giveaways & competitions
  • Interact with VIP industry experts in our guest Q&As
  • Access to members-only sub forum discussions
  • Access to members-only Chat Room
  • Get INSTANT ACCESS to the world's best private pro audio Classifieds for only USD $20/year
  • Promote your eBay auctions and Reverb.com listings for free
  • Remove this message!
You need an account to post a reply. Create a username and password below and an account will be created and your post entered.


 
 
Slide to join now Processing…

Forum Jump
Forum Jump