RAID help

>>57185510
Don't use RAID 6.

>>57185517

Why not?

>>57185529
Because 5TB is fucking huge. Somewhere along the line, one of your drives will fail, you'll replace it with a spare and while trying to reconstruct the data two more disks will fail, making all your data gone forever.

>>57185510
Software

Always software

>>57185586

I have almost 6TB of data on my current HDDs. I want something that will be future proof for at least 5 years. What are my options?

>>57185611

Why do you say that?

zpool create

>>57185618
Hardware raid came about because doing raid operations took a significant amount of available CPU time way back when.
Nowadays, the usage is completely and entirely negligible, and software raid technology has far surpassed hardware.

For one, software raid is easier to develop, so more development tends to happen for it. If a bug is discovered in a software raid driver, it's fixed and updated. If a bug is discovered in a hardware raid controller, there's nothing that can be done about shipped units, and retooling existing designs costs extra money so even future shipment may continue having bugs.
Because hardware raid controllers need to be implemented with physical transistors, manufacturers tend to skimp on features they don't think will be used, while software raid always comes with everything

This is all well documented on the internet if you cared to look for it.

>>57185664

I have been looking for a while, but recent articles (2015+) have been difficult to find, so I was suspect about their reliability.

My only concern with software RAID is what happens if the motherboard dies? Can I simply buy another suitable motherboard and migrate the drives to the new system? (The OS will be set up on SSD's separate from the actual RAID HDDs.)

>>57185664
>If a bug is discovered in a hardware raid controller, there's nothing that can be done about shipped units

This is blatant misinformation

I work at the National Center for Supercomputing Applications, use RAID 5.

>>57185681
>what happens if the motherboard dies? Can I simply buy another suitable motherboard and migrate the drives to the new system?
Yes. The motherboard doesn't matter at all to software raid

>>57185681
Back up the raid info and build your raid from it

>>57185697

Why 5 instead of RAID 6? Especially consider the size of modern day HDDs and the possibility of encountering an URE?

>>57185700

So any motherboard with suitable requirements for the rest of the system will work? Doesn't need to be the exact same motherboard?

>>57185712

In an ideal world, I would. But for now, I don't have the money to buy 30TB worth of HDD, the system to build it -and- backup drives for it as well.

>>57185715
only use hardware raid 5 if you have a battery-backed raid controller otherwise you will be a "write hole" victim

>>57185715
>So any motherboard with suitable requirements for the rest of the system will work?
Yes
>Doesn't need to be the exact same motherboard?
Nope. Again, software raid doesn't care about your mobo. It cares about your disks and only your disks

>>57185730
Don't ever use RAID 5 or 6 with such large drives, OP. If you insist on hardware RAID do RAID 1 or RAID 10.

>>57185750

Not insisting on hardware raid at all, honestly. Trying to determine between onboard (motherboard) or software raid.

I can supply a BBU, if it is needed. I haven't looked at what wattage would be needed yet, but yeah.

>>57185730

I wasn't really considering RAID 5, I wanted to go with RAID 6, considering the size of the HDDs.

>>57185715
If zero loss of data is a priority, use RAID 6 and consider magnetic tapes. In the real world, we use RAID 5 for the lower overhead and the less complicated controller algorithm. The slowest component of your machine (up to a very large scale) is the storage, and RAID 0 performance, especially with slower larger disks, is desired. The added redundancy of RAID 5 is just enough when you can replace the drive in a reasonable amount of time.

>>57185773
>motherboard
Huh? Are you talking about Intel RSTe? That's software.

>>57185823

It's mostly just a setup for home use (movie/TV show server). Data loss would be extremely annoying, considering the amount I would need to get back again. But it's nothing that would bankrupt me like a business.

My concern is that RAID 5 isn't enough with the size of modern day HDDs, as the possibility of an URE increases with the size of the HDD.

Basically, RAID 6 seems like the new RAID 5, these days.

>>57185872
RAID 5 absolutely isn't enough with 5TB drives and neither is RAID 6 with consumer hard disks.

>>57185888

Define consumer HDDs?

I'm using RAID rated HDDs for this build.

>>57185586
I am aware of that feel while you rebuild your RAID6 array but it has always turned out just fine. Just make sure you replace dead hdds in a timely fashion. It also helps to have a hot spare.

>>57185912

I plan on having 1 HDD on stand by at all times. Just trying to determine if software RAID is an acceptable use for this situation and if Windows 7 is suitable for setting it up in.

How about using raid 10 with 2x3 (for better speed) or 3x2 (for better reliability)?

>>57185618
Linux mdadm runs on anything everywhere. You just need any computer to access your data.

Hardware Raid or BIOS software fake hardware Raid depends on hardware. Your hw raid controller breaks and you need to get that exact hw raid controller.

Just usd linux mdadm.

>>57185935

Speed isn't a terrible issue, as the most imposing thing I plan on putting it through is possibly streaming 2 1080p videos at the same time.

Reliability is a concern, but I'm not sure that 10TB is enough for future proofing. I'm using 6TB now.. 4 more is.. not that much.

>>57185906
Anything with UREs within 1x10^14 bytes read. So, anything that isn't near line SAS or SAS.

>>57185961
2x3 would make 15TB, but if you want to add more disks to the raid "1" (reliability part) you must do it with three (i.e. now 3x3)

for why shouldn't you use raid 5/6 with fuckhuge disks:
http://www.zdnet.com/article/why-raid-6-stops-working-in-2019/
http://storagegaga.com/4tb-disks-the-end-of-raid/

also you really shouldn't use raid on windows, it's more of a hack than a proper implementation really.
https://plus.google.com/+AlanCoxLinux/posts/QWnM6s2zKWm

>>57185614
MD RAID 10. Use a far layout if speed is an issue.

>>57185912
>I am aware of that feel while you rebuild your RAID6 array but it has always turned out just fine
We've had it not turn out fine. Lost two disks with unblemished smart statuses the first night it was running. We ended up buying three or four WD NAS, copied everything off our second RAID6 array to them, and then flew an intern down to London with them in a suitcase to rebuild the lost array as an MD RAID 10.

Frankly, it was the copying off our second array that felt the worst, since it was reading sectors it likely hadn't touched in years. We decided that our next NAS is probably going to run GlusterFS.

>>57186020

I've seen the first article you linked. It argues that with fuckhuge disks, RAID 6 starts behaving like RAID 5 instead of RAID 6.

That level of reliability is fine with me, since I plan on maintaining a hot-swap HDD at all times anyways.

My only concern is if performance would be an issue with modern hardware, considering a Windows OS.

I know it's not the best OS for RAID, but would it be enough for simply serving movie/TV show video files for home use?

>>57186043
shit, I was writing the post but accidentally ctrl+w'ed out of the tab, so sorry being late.

>first article
he also argues that raid 5 is no more better than a single disk in his earlier posts

>hot-swap HDD at all times
my point is that another drive dies while (automatically) rebuilding the hot-swap and a third one dies while building the second new and so on
(I'm not sure if I understand hot-swapping correctly though)

>simply serving movie/TV show video files
I'd say even a single disk would be sufficient, and raid would certainly perform better than that

>Data loss would be extremely annoying but not bankruptcy-inducing
just make sure you don't store anything that is not reobtainable like family photos, important documents etc. that are not backed up as if the raid didn't exist.

>>57185510
linux, software raid (MDADM)

linux will do everything you need for an HTPC/steam streaming box, steam is natively supported with big picture mode and streaming from a windows box available, and for media, there's kodi/emby/etc

mdadm can be grown to add new disks

>>57186277

My main concern with this is I am completely unfamiliar with any sort of Linux enviroment. I would like to add this new server onto my current network with minimal headache.

>>57186314
you can use something simple like xubuntu or whatever
installing kodi/emby/steam and creating an mdadm raid is not difficult to do, if you're competent with windows, it won't take much effort to do this

the hardest part will be creating the mdadm raid, but there's plenty of guides and tutorials for that, and it'll be much less of a pain later on than if you used a fakeraid/hardware raid

>>57186025
>We decided that our next NAS is probably going to run GlusterFS.
I didn't know what GlusterFS was, so I read a little bit about it.
That sounds like way the fuck over kill for something like a home NAS.

>>57186373

Fair enough, anon. I will look into it.

>>57186043
You meant a hot spare?

You can't be serious. A hot spare is really uneconomical for home use.

>>57186735
>Uneconomical
The cost difference is only as much as the price of a drive drops from now to when the first drive failure happens, so not a lot for the benefit that the system can automatically drop a failing drive and start the rebuilding process immediately. Also allows you to go shop for new drives at a more convinient time

>>57185664
>software raid technology has far surpassed hardware.
Okay then, enjoy your DMI-caused throughput bottleneck