So I know that RAID-5 with today's high capacity drives is a recipe for suicide, because if one drive dies then a single URE while resilvering will destroy the array.
But what about RAID-10? It's the most recommended these days, I assume for a good reason. What happens when a drive dies in a RAID-10 array and then you suffer a URE in the mirrored drive while rebuilding? Will it just result in a corrupt file for whatever sitting on that block/stripe or will it also ruin the array?
Please help a noob understand the risks between mirroring vs parity.
You actually do a lot fewer reads rebuilding a RAID1, whether standalone or part of a RAID10, than you do rebuilding a RAID5, because they're just sequential bulk reads instead of parity checking.
Yes I'm aware that a drive being physically broken while rebuilding RAID-5 is pretty unlikely, and much less so for RAID1 where all the controller does is copy the contents over from a single drive.
But that's not what makes RAID-5 rebuilding so dangerous, it's the likelihood of a read error. Things like this happen all the time on a desktop drive but it doesn't matter, nothing breaks except for a single corrupt file. During rebuilding RAID-5 though, it can't tolerate a single lost bit and it's actually likely to encounter a read error.
But what I don't understand is how this dynamic plays for RAID-1 (and by extension, RAID-10). What if a URE happens during a RAID-1 rebuild? Which isn't all that unlikely anyways?
>RAID 5 arrays protect against the failure of 1 disk in the array. When (not if) a drive in the array fails the drive must be replaces and the redundancy of the array must be rebuilt on the new drive. This requires reading every bit of data on the remaining non failed drives. Typically SATA drives will encounter an unrecoverable read error rate (URE) once every 10^14 reads. During the rebuild process it is VERY likely that one of the disk in the degraded array will experience an URE. When this happens the RAID controller will detect that a 2nd disk has failed and will fail the entire array.
>But that's not what makes RAID-5 rebuilding so dangerous, it's the likelihood of a read error
> Implying we don't run all our SANs with RAID5
> Implying this is even an issue with > 400 disks
>But what I don't understand is how this dynamic plays for RAID-1
Nearly all of this RAID5 bashing comes from these storage startups with he Supermicro + JBOD + software RAID combination.
In order to attack the competition, you get paper after paper complaining about the most common RAID levels, until people who's run RAID5 on everything for ten years suddenly think it's bad.
Not just Asians.
The UN is recommending insect farming as the most viable source of nutrition to support Earth's overpopulation in the coming decade.
Insects are extremely efficient at converting plant mass into protein, with minimal energy waste.
>it can't tolerate a single lost bit
Yeah, maybe on a dedicated raid card from 1992 with seriously dumb firmware. You think a software raid-5 isn't smart enough to retry a sector read before it fails a major operation like that?
i had a conversation with some guy on the bus like this
just a few decades ago we in north america thought eating raw fish was fucked. now everyone's doing it. the next is probably insects.
RAID 5 can be recovered after multiple errors across drives (with data loss). 3ware controllers had/have an option to use zero's for the stripe where a read error occurs on multiple disks. If two disks don't spin up, you're fucked though.
I've recovered from a similar situation with linux software raid 5. That required dealing with (holey) images from the disks. You just have to deal with silent corruption of data in the effected areas and the possibility for a super block being corrupted, which should be somewhat redundant in the file system.
Not saying recovery is easy, it can take several days to a week to get something. You will loose data, but that would hopefully be confined to a few files.
Not very soon, but you're right. Overpopulation is an extremely real threat, and scarcity will make efficiency necessary.
I've read stories of people choosing cannibalism over dieing of starvation.
In that perspective, insects aren't so bad if you're starving in an African warzone.
In most western countries, the opposite is the problem. The push to import illegals in the us is to prevent a demographic crisis because we quit making babies in the financial crisis. Japan is going to try to build robots to solve their massive demographic problem. Japan is too xenophobic to import foreigners. Europe is importing Muslims.
The solution to overpopulation: get rich. People quit making babies, and invest more into the kids they do have.
Ive been looking at a raid solution since I'm running out of space on my current drives.
The issues that they seem to have is putting me off a bit especially a failure resulting in a fully lost array, I mean WTF I dont want to risk 16tb of data.
Since I work with Openstack in my job I was thinking of using a slightly modified version of Swift instead. Using 2x Replication provides slightly less space that Raid 5 iirc but means that any number of drives can fail and you wont have a chance of losing that data on the remainng drives.
Also allows multiple sizes of disks to be used together, great if you want to upgrade from 4x2tb to 5X4tb or something by replacing one drive at a time.
Also they are working on high availabilty sharding which could result in 2/3/4x replicas (whatever redunancy you want) to occupy 40% less space than the actual number of copies.
IMO it seems like an easy decision but as far as I know it hasnt been run as a NAS before so who knows.
Overpopulation is an extremely real threat to countries that for practical purposes, have little to do with us.
These are countries in Africa that receive aid from the UN, or places like India that literally cannot stop raping, or places like China where they rushed into industrialization and modernization when their economy was previously agricultural.
Third world countries and countries who base their economy on agriculture tend to have more children. First world countries that are more varied and have strong infrastructure have less kids.
Basically, jungle people have children.
>The solution to overpopulation: get rich. People quit making babies, and invest more into the kids they do have.
Wealth doesn't work like that. It's a comparative thing, if everybody in the world had 10 billion USD, the USD would become worthless.
Wealth bracket doesn't directly correspond to number of children.
You can see this in Russia.
In Russia, every family has maybe 1 or 2 kids. More would be a lot.
Most of Russia is poor bydlo. So why don't they have more kids?
Well keep in mind that the RAID is not a backup (thought it is more robust against hardware failure, it's main benefit is better read/write performance).
I think what I'll end up doing is using the RAID for daily work, and back up individual projects to a single high cap drive.
System drive is an SSD with its own backup.
Yeah, But having a decent 128GB SSD and a 1TB/8GB hybrid I dont really seem to suffer much in performance.
I can store most things that ill want to access quickly on my laptop and the rest on the Swift system and take the ~10-50 second copy time before starting to watch a movie etc
To each his own though
I know. But wealth is a physical thing, something our reserve banks have forgotten. Do you have enough food to eat? Are you healthy? Is your future secure? Can you afford to have children? Can you retire? Do you have free time, or are you living hand to mouth?
The answer to the above is true wealth. The paper shit in your wallet is mostly worthless, and made more so with every click-clack of a printing press.
One reason the Chinese had so many kids is that it was the only way to guarantee you would be supported in your old age. When one child per family came along, hmmm, lots of girl babies didn't make it.
Turns out, maybe our govt's guarantees may not mean very much in the end, but we are veering far from RAID.
OP, RAID6 on a mega-raid card on high end (supermicro) hardware that is updated at least every 4 years is the only way I'd fly. I handle ~20TB on older 3ware controllers at work spread across multiple machines.
Russia is a funny place; the men are worthless drunks, and the ladies hard as nails. In general though I think the correlation holds across populations.
It's not about wealth, it's just simple economics.
In some places (mainly NA and Europe), people have an economic disincentive to have children, since the parents are expected to provide basic needs, education, misc until adulthood, when the child fucks off without giving a cent back to the parents.
In other places, children are basically slave labour.
> more robust against hardware failure, it's main benefit is better read/write performance
What is... no?
Key word here being redundant.
Doesn't matter. If it did, we'd all be running RAID 6.
I've watched Clariion RAID 10's split shelves and fail because of a data cable. Drives 1-4 and 12-16 dropped out. The other half of the mirror picks up the slack.
Usually enterprise gear will take the drives, mirror the individual members, and then stripe the array.
The real danger of RAID is a power supply taking a shit and applying line voltage to the computer. I've seen it happen, and cleaned up the mess.
Regardless a fire or lightning strike will do the same. RAID provides only limited security. You have to keep multiple rotating offsite backups. Fortunately for OP's case he has a small enough data set that he can make backups across cheap external HDD's.
What, you don't like 1792 errors on HP array controllers?
If you use a battery on your array controller, it's less of an issue.
If Windows VSS backups to something off RAID are the only way to fly...