/raidlevel/ general

>>62046443
you don't sound qualified to be making these type of choices

Who the fuck decided to back up a raid via USB? You? It better fucking not have been

>raid
>backup
nice b8

>>62046508
It is pretty common in small filmproduction companys. Cut the movie on a mirrored Lacie 2Big then backup to 5xHDD USB RAID Enclosure.
I did not choose this workflow. I am used to centralized network storage and backing up to 4TB single drives, but I have to live with this for now.

>>62046443
>RAID for backing

repeat after me: raid is not a backup solution, it's a redundancy solution

>>62046569
So do you have an actual useful solution? I know this is shit and this is why I am asking here. JBOD and then make a virtual single volume from it? I want to avoid single HDDs because of the pains of spanning a project over multiple HDDs (there are databases involved to keep track of media files, some of the video files can be larger than 100GB etc)

>>62046569
also the reason for raid5 being discouraged is that modern disks are far larger from when raid5 was popular, rebuilds are the time disks are most likely going to fail as you're putting them through a severe amount of abuse and if your array has anything larger than 1tb~ disks a rebuild is going to take a significant amount of time, if you bought all your disks at the same time and one fails it's not unreasonable to assume others will fail in a similar time frame especially if you're about to put them through 30-40 hours of array rebuilding

raid6 offers dual disk redundancy but it's a stopgap solution and might put you in the exact situation raid5 is putting you in, raid isn't a backup but redundancy solution and if you can't spare the storage requirements for proper redundancy with raid10 there's a good chance you don't have a backup solution at all

>>62046612
Exactly this is what I discovered when a disk a failed. I fully understand the concept of this. Yet I have to live with over 80TB of data stored on RAID5. I can make propositions for an upgrade path, and it is okay if it costs 2k but I want to have a real plan before I go about this. Migration can take at least a week via USB and manual copying so I want to be prepared. Most video production related online resources tend to be written by dudes in their 40s crying over the downfall of tape, the same kind of people who put RAID5 as a backup solution in place. I want to be smarter than that.

>>62046701
>over 80TB of data stored on RAID5
>on USB
You're gonna get fucked.
These people need a real datacenter.

>>62046443
and the solution is?

>>62046443
I read your one sentence as
>I'm in charge of IT and procrastination

1.Don't use a hardware controller.
2.Use RaidZ or ZFS Mirrors

>>62046443
Something about if your raid controller(discrete card or motherboard) craps out your volume is lost forever.
People think it's neat because you can switch a borked hard drive, but forget to think what happens when the machine itself dies.
Whereas if your FreeNAS runs ZFS and craps out you can move the whole RAID Z to another machine and it will recoqnize what it is.

>>62046569
>raid is not a backup
It is if the file is already on an external hard drive somewhere else you dunce

>>62046443

No dude, don't use RAID for backup. Use backup software that just, you know, backs up to the other drive. Safer. You can also make older copies of content to provide backup redundancy if a file somehow got corrupted and then copied to the backup drive.

>>62046443
Remember to keep the backup in another building, so they don't both die incase of fire.

>>62047324
That makes the external hard drive the backup, dunce.

>>62046701
honestly the biggest problem with hardware raid these days is trying to replace the hw raid card if that dies, not that I'd trust usb hw raid further than I could throw it and software raid has corruption issues on power loss

it'll be an expensive upgrade no matter what you do and I wouldn't want to be in your shoes, all I can recommend is getting a proper raid solution to get redundancy sorted out, raid6 would be preferable over raid5 but it's far less supported and might have issues on your dodgy usb hw raid cards and it's still bad for all the reasons raid5 is, so raid10 would be preferable, and then I'd recommend getting a proper backup solution in place whatever that may be, nas works but isn't ideal especially if it's connected over the network, tapes are cheap but the readers are expensive and there's problems with restoring from backups, offsite is expensive and if you're dealing with terabytes requires a decent network connection etc

if you have to explain it to rationalise the cost and they're film students try to put it in an analogy they might get, like having all your open volatile film canisters on a desk around someone that occasionally smokes - you'd strongly want offsite copies in a secured building just like you'd want backups of your data in case the raid array fails and rebuilding isn't possible

>>62046443
how to make a software raid in 3 easy steps:

step 1: install a linux

step 2: install a software packages sometimes known as mdam

step 3: configure that shit up, yo

File: IMG_0061.jpg (2MB, 2400x3000px) Image search: [Google]

2MB, 2400x3000px

>>62046443

I recently had some of the same questions regarding what to do with my home NAS, it's a Mac mini with a thunderbolt 4 drive bay attached.

Ended up going with 2x striped mirrored vdevs in a single ZFS zpool. Hell of a lot nicer having a single pool for storage and it's effectively RAID10 but with a much better file system.

Like folks said though, it's not backup. I recently found Google apps suite for business gives you unlimited Google Drive storage for $10 a month. I made a single user "business" and now have unlimited storage. I use rclone to encrypt then back up the contents of the. NAS to GDrive.

>>62047349
Chicken and egg

>>62047387
This.
Dual striped mirrors have nice I/O because it's double the read speeds.

why not just a 2-3 4TB disks, mirrored, per project?
also, you should be using something like btrfs/zfs software raid, as they also provide checksumming
not to mention you're not relying on a specific kind of hardware raid card (you might have trouble if that card dies and you can't find the same one to replace it)

>>62047359
thanks for taking the time to write an actual response. I got the management on my side. I explained the situation thoroughly and they are now checking the financial side of things. Looks like its going to be either tape or cloud.

>>62047660
cloud with a proper respectable provider that has a good SLA is probably your best bet if the network thing isn't an issue, tape is a bit of a shit solution and restoring from backup isn't always guaranteed to work, it's mostly for when you need hundreds of terabytes of data backed up redundantly still - I recall reading an article a few years ago where a good % of tape backups won't work, but I can't remember if the article ever explained whether it was the tapes failing or if the backup method just wasn't that good

File: a.jpg (61KB, 1037x432px) Image search: [Google]

61KB, 1037x432px

you're not getting usb external hdds and just storing those, right?
it'd be cheaper just to use bare hdd's, plugged in through hot swap bays, and stored in hdd storage/shipping boxes (better protection in those as well, external hdd's aren't well protected on their own)

>>62047359
>software raid has corruption issues on power loss
only while writing, and things like journaling/cow/checksumming of modern filesystems like btrfs/zfs, make it both difficult to corrupt things at all, and also easy to detect
for a backup solution, all you need to do is read back the contents to ensure it's good, and you're done. since it's a backup, it won't be further written to, so no risk of corruption

>>62046530
>>62046569
https://wiki.archlinux.org/index.php/RAID#RAID_level_comparison

File: 1487706230410.jpg (259KB, 1000x1500px) Image search: [Google]

259KB, 1000x1500px

How long does it take to repopulate HDD in RAID?

Certain datacenter that I use, had an issue regarding hardware equipments, making 2 petabytes of data unaccessible.
They say that i can access the data when the primary HDD is populated.