[Boards: 3 / a / aco / adv / an / asp / b / bant / biz / c / can / cgl / ck / cm / co / cock / d / diy / e / fa / fap / fit / fitlit / g / gd / gif / h / hc / his / hm / hr / i / ic / int / jp / k / lgbt / lit / m / mlp / mlpol / mo / mtv / mu / n / news / o / out / outsoc / p / po / pol / qa / qst / r / r9k / s / s4s / sci / soc / sp / spa / t / tg / toy / trash / trv / tv / u / v / vg / vint / vip / vp / vr / w / wg / wsg / wsr / x / y ] [Search | Free Show | Home]

How do I protect my system from data degradation and bit rot?

This is a blue board which means that it's for everybody (Safe For Work content only). If you see any adult content, please report it.

Thread replies: 34
Thread images: 4

File: IMG_0113.jpg (78KB, 250x236px) Image search: [Google]
IMG_0113.jpg
78KB, 250x236px
How do I protect my system from data degradation and bit rot? I have it running in a faraday cage with a primary SSD mirrored by a RAID0 HDD array, both ssd and hdd array changed every six months, and an emergency redunancy high speed SD flash card memory array changed every quarter. It feels like I'm in a constant battle to maintain data integrity and I feel like my process is only barely cutting it.

Would it be worth it to invest in a UPS/generator protected ECC ram drive NAS unit, or does the network transfer risk further data degradation?
>>
>>59630629
Now this is autism
>>
>>59630629
Just store everything important in the cloud you fucking luddite.
>>
>>59630643
Great insight, but half of my rare anime collection just isn't available on torrent trackers anymore and my optical copies degrade just as fast as my magnetic and NAND memory.
>>
>>59630629
perform a hexdump of everything you need saved and then print the output on a4 printing paper
>>
>>59630691
FUCKING IDIOT NOT EVEN CONVERTING IT INTO A QR CODE
>>
>>59630629
w-what are you hiding?
>>
Bitrot is p. rare anon. Just use btrfs or something and forget about it.
>>
>>59630666
Checked. As soon as you get comfortable storing your degenerate cartoon porn on the cloud the sooner you'll find a girlfriend who likes it too
>>
>>59630666
I have too much invested to trust the cloud and there is no reliable provider here in Ukraine. I'm thinking of building a server rack with 512gb ecc memory ram drive NAS units, maybe 12 of them, and having them cycle data in rotation from the ssd/hdd raid array.
>>
>>59630629
I hope you do all that, then get hit by a bus the next day.
>>
>>59630629
>RAID0
>data integrity

nigga hope you have backups cuz RAID ain't no backup and RAID0 ain't even redundant

Use ZFS or Btrfs for the file system.

>>59630718
>even giving a shit about backing up your media

Also store locally and get something offsite. This is basic 3-2-1 shit and anyone who disregards "cloud backups" is a retard
>>
>>59630629
TAPE
BACKUPS
TAPE
BACKUPS
TAPE
BACKUPS
TAPE
BACKUPS
TAPE
BACKUPS
TAPE
BACKUPS


damn thx hp
>>
>>59630834
This doesn't provide on demand storage. I want to be able to stream my rare anime archive hence the investments.
>>
>>59630869
wtf? then you weekly mirror the precious anime tapes to lead cased ssds (obviously some uber redundant raid array of ssds)

bitch you better make your cables out of gold conductor and lead shielding
>>
>>59630985
>ssds for archival storage
Brainlet detected.
>>
File: jew_basic.jpg (38KB, 600x600px) Image search: [Google]
jew_basic.jpg
38KB, 600x600px
>>59630666

>cloud
>>
>>59630629
learn some basic computer science
questions like
>does the network transfer risk further data degradation?
display such a severe lack of knowledge, that there's no way to convince you of what a good strategy would be, since you wouldn't understand why it would work
>>
>>59630629
Just use a FS with redundancy and checksumming (e.g., ZFS) and schedule periodic parity scrubbing.

Sector-level forward error correction in drives (HDD and SSD) is more robust than you realize, and RAID6/RAID-Z2/whatever-multi-parity can save your ass from a lot of problems if you can deal with the cost/performance hit.
>>
>>59631490
Using non-volatile memory is begging for bit flips and critical data loss.
>>
>>59630629
Printouts, retard.
>>
>>59631667
>paper storage of terabytes of rare 90s anime
>>
>>59631008
no the tapes ar archive and the ssds are copies you refresh weekly


F
U
CK
YO
U
>>
>>59631691
Do you want to keep your waifu safe or not?
>>
>>59631695
>can't vertical type
Fuck off newfriend
>>
>>59631704
A tulpa is the safest waifu storage medium.
>>
>>59630629
ZFS faggot.
>>
File: f7FdEdG.jpg (10KB, 260x194px) Image search: [Google]
f7FdEdG.jpg
10KB, 260x194px
>>59631756
Can confirm.
>>
>>59630629
>faraday cage

Depends on how many hertz it is running
>>
>>59630629
>I have it running in a faraday cage with a primary SSD mirrored by a RAID0 HDD array, both ssd and hdd array changed every six months, and an emergency redunancy high speed SD flash card memory array changed every quarter

You have none of those things.

To prevent bitrot, use a filesystem that is aware of the drives, such as ZFS or btrfs.
>>
>>59632354
>such as ZFS or btrfs.
memefs you mean
>>
>>59632406
Feel free to suggest alternatives.

Oh right, those are the best options. Well then.
>>
>>59631647
Undetected bit flips are only an issue in ECC-less RAM.

All non-volatile storage has a ton of redundancy built in that requires large numbers of corrupt bits in a block to be corrupted.

It's no substitute for a backup and it does have a shorter lifespan than magnetic storage, but it's not gonna magically undetectably flip bits in your file.
>>
>>59630629
Your method is completely useless.
The act of transferring the data also puts it at risk. Hardware RAID controllers also pose a risk as you cannot audit them to ensure they know all the thousands of ways data can go bad.
Use ZFS.

There are no unstable releases.
It is feature full with no known data destructive bugs and a perfect track record when it comes to stability so long as you stick to the defaults.
Your data is secure as long as you have redundancy and don't try to micromanage on a block level.
Your file system can handle all of that for you.

It's literally one time setup, walk away for a decade and come back to resilver the array and replace a faulted disk or two.
ECC is more secure but memory errors can be caught, in some respects, with FULL end to end checksums you can enable through kernel params.
Thread posts: 34
Thread images: 4


[Boards: 3 / a / aco / adv / an / asp / b / bant / biz / c / can / cgl / ck / cm / co / cock / d / diy / e / fa / fap / fit / fitlit / g / gd / gif / h / hc / his / hm / hr / i / ic / int / jp / k / lgbt / lit / m / mlp / mlpol / mo / mtv / mu / n / news / o / out / outsoc / p / po / pol / qa / qst / r / r9k / s / s4s / sci / soc / sp / spa / t / tg / toy / trash / trv / tv / u / v / vg / vint / vip / vp / vr / w / wg / wsg / wsr / x / y] [Search | Top | Home]

I'm aware that Imgur.com will stop allowing adult images since 15th of May. I'm taking actions to backup as much data as possible.
Read more on this topic here - https://archived.moe/talk/thread/1694/


If you need a post removed click on it's [Report] button and follow the instruction.
DMCA Content Takedown via dmca.com
All images are hosted on imgur.com.
If you like this website please support us by donating with Bitcoins at 16mKtbZiwW52BLkibtCr8jUg2KVUMTxVQ5
All trademarks and copyrights on this page are owned by their respective parties.
Images uploaded are the responsibility of the Poster. Comments are owned by the Poster.
This is a 4chan archive - all of the content originated from that site.
This means that RandomArchive shows their content, archived.
If you need information for a Poster - contact them.