/mlp/ Preservation Project

File: 1399998051609.png (296KB, 500x500px) Image search: [Google]

296KB, 500x500px

>>30293454
>Using Glimmy
Eh

The first thing we should do is create some type of archive that anyone can access. We need to start saving every dumb pony video you watch on youtube, every green written on this board, every pony song you listen to and every piece of artwork and animation you like.

File: 1398114178180.gif (63KB, 333x260px) Image search: [Google]

63KB, 333x260px

>>30293464
Any suggestions on how to achieve that?

>>30293464
Well, for starters fimfic has a torrent archive. We need an anon who could host this.

>>30293469
Well the saving thing is easy enough. As for the archive, that could just be a site like yayponies. It isn't hard to set up a simple site like that and host files.

>>30293470
Fimfic is not about to close you know.

In fact it's about to go Genfic and spread to every fandom out there.
Knighty is already making a living from Fimfic, with Genfic he'll become rich

>>30293478
Yeah, but we're talking about 20-40-60 years down the line. Youtube isn't going down anytime soon either but a collection is a nice idea.

File: 1379987393213.png (201KB, 500x480px) Image search: [Google]

201KB, 500x480px

>>30293476
Who would handle the donations? that's a serious issue as people is anonymous and could steal the money at any time, or be dishonest.
Could a patreon or something similar be worked out so there is a reduced risk?

>>30293481
Most of us will be dead in 60 years

>>30293482
I think if someone was actually hosting the site for us I wouldn't care if he was blowing the money. Just as long as it stayed running.

>>30293478
Is this true?

>>30293481
Youtube isn't all that reliable.
A lot of pony videos get their sound tracks removed because of copyright infringement, some videos are also already down for the same reason.
However hosting them might have legal implications for these same issues, so I dunno what could be done about it.
>>30293484
That's the intention of the project anon. To pass the flag once our time comes.

>>30293482
I'd trust myself to do that. I know saying that is pointless, as I can't prove a thing to you guys. But I was really lucky to be born into a well off family, I have no need at all for more money, I live comfortably. Hell, I have enough to probably fund it myself

>>30293490
The Genfic part yes. The rich part I don't know

>>30293484
Hence preserving it for the future generations. Ponies shouldn't be forgotten, ever.
>>30293496
Well Yayponies hasn't been glassed, maybe we could ask them how they do it?

File: 18951370_1814771702173101_1379440737570760455_n.jpg (67KB, 750x698px) Image search: [Google]

67KB, 750x698px

>>30293454
>>30293464
>>30293469
See the problem here is that _you people_ don't want to send me stuff to archive.
Send me all your money though

>>30293470
>>30293476
http://iwatchitfortheplot.romesilvanus.io/Pony/Stories/

>>30293497
You shouldn't take the load all alone, it would make it an obligation and if you have an emergency or change your mind later it would be even worse since you wouldn't like to feel compromised about some stuff you no longer care about (I know we love ponies to death, but just being realistic here).

>>30293506
I forgot about this. How/where could we aggregate content for you?

File: 1392459650202.png (184KB, 421x431px) Image search: [Google]

184KB, 421x431px

>>30293506
>Send me all your monies
Lol
Obviously we are talking about covering the costs comfortably (maybe with a small extra fee for your troubles) and never really putting you out of your day job. I mean if we could, great but honestly I would prefer it if we could try and secure the host for a long time than to waste the people's funds and spirit quickly in the very first years.

>>30293508
I would never leave you guys high and dry. On the very slim chance I do move on, unlikely, I'd make sure that all the content was left in good hands and that many copies existed to keep it from being lost forever. Besides, I'd love the responsibility, I do little in my life besides /mlp/ and uni.

>>30293525
Never make impossible promises anon.
What if you lose your memory or something? I don't want to sound pessimistic but you know how life goes.

>>30293533
Well then I'd share it with some of you guys, just in case. But still, I could provide a lot of the funds myself.

File: hNwfAzp.gif (2MB, 640x480px) Image search: [Google]

2MB, 640x480px

>>30293514
https://app.syncwerk.de/u/d/806ae7340a/
Or where ever you like really.

>>30293518
Picture very related.

I run this off of my own money because I feel like it and that's just how it is. You specifically can send me all your money anyway.

>>30293536
I am also wealthy but I say that we should try and act wisely about it, like set up some investments or something so it can survive on its own, what do you say? this might be complicated though.
It is much better to use then money in a way it can generate money instead of feeding it by yourself all the time.

>>30293550
Definitely.
Also >>30293547 and >>30293506 are not me

>>30293547
Did you decide to remove /zoo?

>>30293555
I just moved it for a bit to rub it into the face of that faggot on /b who shows off with it.

File: 1428918215130.gif (2MB, 384x271px) Image search: [Google]

2MB, 384x271px

>>30293542
Eh this project was about saving /mlp/ for the next generations originally, I don't know anything about the pony religion stuff.

>>30293569
It was an accidental post I'm retarded, ignore it.

>>30293570
Yeah I also confused you thinking you followed our previous conversation in the other thread until I read your posts closely.

>>30293547
Alright, now that we have a place to upload, what's on our list? We need to figure what's first priority.

>>30293547
Oh I know that you have ran that server for a while now for free.
I am talking about running it in a permanent manner in the unfortunate case that you don't feel like running it or you can't afford to.
So what do you suggest?

How will we avoid people who fucked up the original fandom. Cause at this point bronies are seen as a dying fandom. Not saying we should use that name but pointing out the obvious. Cause we can't start anew because if try to incorporate everything it might blow up in our face but quicker.

Alright boys, we should set up some sort of schedule. Remember, this is a long term project, not something we'll dive into overnight. The main goal of this is to save as much pony content as we possibly can.

So, passively as you browse /mlp/, save greens. Even if they are just small, 4 post greens or whole stories, save them to notepad files. They take up barely and room.

Music is a different story. There is a lot. The best we can do is save anything we come across, obscure or popular. Direct downloads are preferred, but youtube audio rips are acceptable if that is not available.

Now we get to pony videos. What we can do is save all of them that you watch. From small 10 second clips to full hour long projects (Like anthology), save it all. As with music, direct downloads are preferred but youtube rips will suffice if this is not a possibility.

Any other types of content should also be saved to your computers. This should be our goal for the next while, pony isn't on the verge of destruction by any means and with s8 confirmed, this content isn't going anywhere in the next few years.

So, priority right now is to save all pony content that you see. We should make this a regular thread to keep track of how we are going.

We can compile the content at a later date.

That's my take on what we should do, thoughts?

File: 1400239658687.png (130KB, 900x876px) Image search: [Google]

130KB, 900x876px

>>30293579
The first PMVs is my pick, unless they are already there
https://www.youtube.com/watch?v=7LE6DJj0ngU
https://www.youtube.com/watch?v=dg0-RBlMHoE

>>30293598
This sounds good. Bit by bit.
>>30293547
Does this work with you? Can you categorize for us?

File: 1425230449213.png (116KB, 1166x267px) Image search: [Google]

116KB, 1166x267px

I have this list of videos too I have enjoyed
https://www.youtube.com/playlist?list=PLoAwCWM4nZKuf7RH570c7JGKKXZh2oo_7

>>30293580
You can always download it, so feel free to. Though I add/remove/move things around from time to time.
http://iwatchitfortheplot.romesilvanus.io/Folder%20sizes.txt

>>30293605
I can, but it does help if you send it sorted and with as much additional information as possible.

Right now I still have 1.114TB free to use.

>>30293630
>Only 1 tb
I don't think that is going to be enough.
I remember one anon mentioning he downloaded around 1tb only of music and it wasn't even close to being it all.
I guess we are going to have to provide an extra hard drive.

>>30293598
Are we going to bring porn to the site cause if we are I'm pretty sure the site won't make it that far I'm not saying we shouldn't but look at what happened when they do bring porn into things if anything derpibooru should keep that part but whatever you guys think is best.

>>30293651
I'm thinking we do bring porn, but perhaps to a separate archive, dedicated solely to porn.

>>30293636
The server has 6TB. 1.1 is still free.
I will need a new one with more space anyway if I'm going to finish my Derpibooru clone.

File: 1389059245948.gif (470KB, 544x525px) Image search: [Google]

470KB, 544x525px

>>30293651
Porn needs a separate server simply because it will eventually be knocked by take down requests from artists, specially if you start allocating paid content (which is a must because it is generally the best).

File: 1379536479404.gif (134KB, 320x322px) Image search: [Google]

134KB, 320x322px

>>30293672
What do you have saved in there anon?
Are you already handling duplicates? I can see those are a complicated issue, specially to try and optimize time and space so people doesn't waste theirs uploading stuff that is already there.

>>30293690
If you don't wan't to upload duplicates then take a look first, or use the search/filter mask.

I try to keep duplicates minimized. But sometimes there will be multiple copies of the same files in different directories simply because they belong together.
Except in 'Shared by other people'. I don't touch those.

>>30293662
As long as I don't have to see it I'm fine with it. Thou that doesn't matter but it's great knowing what we can reach an agreement on something.

File: 1425853962379.jpg (10KB, 215x220px) Image search: [Google]

10KB, 215x220px

>>30293718
You stink of anti clop, but for the sake of unity and peace I'll drop the friction.

>>30293724
Not really you do you as long as I don't have to see it. It's a free country.

>>30293732
You won't have to see it since the server can simply keep it in a completely different folder and address so the main menu just says SFW content and NSFW content and that's all you need to see, then you can click SFW without any drama.

File: 1495713398561 (1).png (551KB, 1366x768px) Image search: [Google]

551KB, 1366x768px

>>30293651
>>30293677
>>30293718
>>30293732
Stop your whining and grow a spine.

>>30293732
Everything needs to be archived. Even the pron. This is our goddamn history man!

>>30293745
I don't have a problem with it in all honesty I really don't. I was pretty much playing a joke but what I meant to say is that it may turn away some potential people who wonder into the site.

>>30293751
It is easier to reach a compromise anon.
I am all in favor of clop, but I am also in favor of not forcing it in the eyes of the people that doesn't want anything to do with it.
It is also necessary in the end when handling new kids, you aren't going to show a horse pussy to an actual 5 year old you are showing the show in hopes he likes it or when you are showing fandom content, do you?

>>30293761
Or, instead of bending over to whiny people you can just tell them to grow a spine.
People who complain about stuff they don't want to see have the worst kind of behaviour. Be quite and deal with it.

Then don't show it to them. Young kids are not the target here. Even then they can handle it since it's not their usual interests.

>>30293783
There not the target but they are the next generation.

>>30293753
I know it's my history and you know I'll wear not proudly but I'll still wear it.

File: 1387603139543.png (727KB, 1600x900px) Image search: [Google]

727KB, 1600x900px

>>30293783
We want to preserve the show for future generations.
Are you paying attention about what this project is about? What is the point of preserving everything if new kids aren't going to grow up and wish to preserve it when we are about to die?
Come on anon.

>>30293579
The Antologies?
Ponified trailers?
SFMPonies?
SOOOOOOOONGS!! EVERY ONE OF THE FAN-MADE SONG!!!

>>30293823
I think we should stop arguing we are going to piss someone off and ruin everything we need to learn self control.

File: 1403438549067.png (92KB, 500x572px) Image search: [Google]

92KB, 500x572px

>>30293850
I am just explaining what this project is about.

>>30293783
I understand where you're coming from and you know what yes I know those type of people are annoying but we have to preserve what we have left and try not to let this fandom die when we are gone so I'm sorry if I didn't come off with a strong head and seemed too frail.

>>30293860
No I understand but it's best if we drop it for now then go over it later.

File: 1429074332072.gif (62KB, 706x720px) Image search: [Google]

62KB, 706x720px

https://www.youtube.com/watch?v=ydVe51sFmzY
This is what the project is about.
Making reality what you see in this video.

Let's just put pron in a separate hidden folder. We still have it, but it's no in the main flow. Bam.

>>30293901
It is already done. Got that link of 65GB folder somwhere...

https://discord.gg/UhCFQr

I made the Discord, use it if you want. It exists more as a means to stay organized and strategize.

>>30293562
>have /zoo link on a sticky note on my desktop for half a year now
>go to it for fap material yesterday a bit ago, link doesn't work
>decide not to fap
you have caused me to forgo a masturbation session. this is all your fault and i will never forgive you for this demonizing transgression. if it is not up within five seconds of this post i will jihad your house

>>30294342
rekt

>>30293562
>>30293901
>>30293909
What's the link for the pr0n folder?

>>30293579
Greentexts. They're the most likely to be lost in the coming years. Most Bronies have never even heard of them, and /mlp/ is a very small section of the fandom.

Make the title of the greentext the post ID of the first post in it, followed by a hyphen, then the title. That will prevent duplicates of the same greentext. We can figure out how to tag them later. Name is filename.

File: IMG_6964.png (352KB, 852x2058px) Image search: [Google]

352KB, 852x2058px

>>30294503
Forgot picture.

>>30294503
You can use the Pastebin downloader from MULPWiki:
http://mulpwiki.org/documents/Pastebins/outdated/GTDownloader_v1.0.jar

You can download their archives here:
http://mulpwiki.org/documents/Pastebins/

I did already mirror them (plus in extracted form, but it can take ages to load that many files in-browser):
http://iwatchitfortheplot.romesilvanus.io/Pony/Stories/MULPWiki%20Pastebin%20archive/

I'd begin by looking for existing archives. Maybe as part of the project Anons can upload their personal collections, which may contain items that would've otherwise been lost. There's also the board archives: desuarchive, arch.b4k.co, archived.moe, and 4tan. Perhaps setup teams to search through them oldest -> newest and organize what they find? That probably sounds retarded. I don't know.

To alleviate the cost of hosting, a torrent index site may be an option. A "piratebay of pony content", or basically a more elaborate yayponies. Put everything into greentext / fanfic, music, image, and video categories.

I agree with >>30293598. Getting into the habit of saving things is a good first step.

https://github.com/bibanon/BASC-Archiver can save whole threads.

I dont any of that computer stuff but here's some stuff if ut might help ya'll. http://minuette.net/Pony/

>>30294622
How do you upload stuff to the server?

>>30295466
I have a Google Drive with unlimited space. And I use rclone and some bash scripts to sync everything down.

Stuff in 'Shared by other people' are either coming by BTSync, wget, Torrent, or other diverse ways.

If you want to send help me out you can either use https://app.syncwerk.de/u/d/806ae7340a/ or any other mean you want.

>>30295538
I was thinking that first of all things, we need to upgrade the server with like 10 tb more and then develop a script or mini app that can directly grab youtube videos from their website, which we just remotely add to some interface.
I only got the idea, I don't know shit about code though and I don't know how hard (and abusable too) it will be.

File: Screenshot 2017-06-12 22.49.52.png (364KB, 1365x1024px) Image search: [Google]

364KB, 1365x1024px

>>30295561
Space isn't necessarily the problem for me. I need to upgrade it anyway in the future. And it shouldn't fill up that quickly.

I do plan on finishing up some sites to help displaying stuff in a more organized manner. E.g.: like picture related.

>>30295626
Well if you aren't on discord I was commenting how we need to come with some small business or investing that is reliable and provides us with the much needed solid ground we will need to be able to maintain the server independently of donations - the business or whatever will maintain most if not all if possible costs of everything so we can just worry about keep the business afloat and nothing else.
Tell me what you guys think.

>>30295726
>/mlp/ starts a business
What could go wrong?

>>30295726
That's too complicated and dreamy.

What we should do is have everyone start creating their own personal archives, but NAME ALL THE FILES THE SAME WAY, so if two people save the same green text separately they will have the SAME FILE NAME on both computers.

Then, anyone with this archival system can merge their archive with ANYONE else who is using the same naming system and just check the box that says "Delete all duplicate names".

All archives are decentralized, which will prevent copyright takedowns. We can merge archives on a regular basis, which will also seed the archives. I believe a similar system was used to create the do/k/ument, available publicly at murdercube.com.

>>30295816
It is the only realistic thing I got.
You can't seriously think a patreon will run for 60+ years right? Not to mention that patreon will attract scruffy negative attention and you know how that goes, all the work down the drain.
But please suggest what you have in mind, I am open to everything.

>>30295824
This relies too much on people having internet at all times and a working computer.
Life can go bad at anytime.
However your strong point is that it will be difficult to take down 20+ mirrors from people.
Provided we can get that many.

It is a solely private project, and will remain a private project. A community project will not run well.
It doesn't cost me that much to run, and even though I'm not stopping you from giving a tip, I don't go around asking for it like some big pony sites like to do.

>>30295726
>>30295824
>>30295836
This will never work.

File: 1426217404825.jpg (8KB, 300x168px) Image search: [Google]

8KB, 300x168px

>>30295874
You won't be able to sustain that weight forever by yourself.

>>30295859
Doesn't require an Internet connection at all times, more like once a month, or year. Not everyone hosts the files publicly on a website, they just sit on each individual's personal hard drive. Occasionally someone might upload their archive to Mega, or whatever equivalent there is 20+ years from now, and everyone else with an archive downloads theirs, absorbing and merging them. At the same time, anyone can start their own archive at any time as long as they know the naming system. It's low effort, which is important on 4chan.

>>30295874
>A community project will not run well.
Isn't a community project the best way to save the most possible content?

>>30295886
Then download his archive when he puts it up and create your own mirror to carry the torch when he's dead and gone.

>>30295886
>>30295886
I'm pretty sure I'll be able to afford 30 to 60 euro a month for quite some time.

>>30295905
The point of the archive is for it to be available constantly, so if one anon has the chance to show it off (without loking like a pedo or creep) to kids so they see if they like it and give it a chance so they can experiement the magic in a similar way as we did.
It is the only way we can ensure that pony will remain in their hearts for years to come and possibly, generations.

>>30295917
That's the idea of multiple mirrors anon.
I think that one can be carried no matter what, since it doesn't depend on a single user, they just have to find a way to make their files available on demand to anons of here from any available mirror.

>>30295908
Man power is absolutely required to gather the most fan content possible, yes.

>>30293454
I've been doing my part. I save every scrap of pony music, fics, videos, and images I like

File: Pinkie_Pie_wide_grin_EG.png (560KB, 1920x1080px) Image search: [Google]

560KB, 1920x1080px

>>30295982
>Man power is absolutely required to gather the most fan content possible, yes.
I read that with MAS Pinkie's voice, yes.

I think we need some type of accepted nomenclature for all types of files that we all use. I'd make some up but I'm shitting right now and I gotta get back to work. Keep it up, though. I'll be with you all in spirit until I get off.

>>30296020
MAS! Someone, or a lot of someones, needs to save every episode of MAS.

>>30296051
I got all the old school ones, with actual show scenes

>>30296058
Absolute legend.

Top 10 Pony videos:

1. https://www.youtube.com/user/TheMovieBrony/videos?flow=grid&view=0&sort=da

2. https://www.youtube.com/user/jhaller2/playlists?view=1&sort=da&flow=grid

3. https://www.youtube.com/user/LittleshyFiM/playlists?sort=dd&view=1&flow=grid

File: 1425865840621.gif (625KB, 400x312px) Image search: [Google]

625KB, 400x312px

I stood awake and pulled an all nighter, I'll take a nap and come back in some hours.
Try to not fuck shit up while Iam gone.
things that seem to have popped so far are
>multiple mirrors from random anons
>script for downloading youtube videos directly from the tube into the server
>donations or a way to help anon to keep the main server running which is top priority over everything right now
>maybe get a dev volunteer or in the case we can't, hire someone to do code for us (unless one anon or server anon can do code stuff)
>make some system to have the mirror files of anons (on demand if possible at anytime, in case the worst happens and we lose the main server)
See ya in a while.

>>30296120
If someone could make a list of the 10 tens, and then all videos in them that'd be amazing

>>30296051
I'm working on the naming standards right now. Will post a pastebin once I'm finished.

>>30296243
Cool. Should we rely on file tags for stuff like artist, year, etc?

>>30293454
You guys should dump all your pictures from artists that put themselves on Derpibooru's DNP list somewhere. That way we can finally give a nice fuck you to artists that want to control the internet.

>>30296243
Let's name everything from derpibooru with the default name it gives when you try to download it.

Name all YouTube videos with their own title.

Name all Greentexts with the post number of the first post of the greentext.

Name all Pastebins with their Pastebin ID.

There. Everything is the same. This >>30295824 Anon's dream is become a reality.

All songs are on YouTube so they use the same naming scheme.

If we want proper titles, then put a - and then the title. If we want tags too, or author, then put a - and then the tags, or a - and then the author. Doesn't matter very much for songs since the author is typically already in the video title (all of 4everfree's stuff, and most others).

There's your naming scheme.

https://pastebin.com/4nh6xxuP
Finished the naming standards for now. Of course, this doesn't cover everything, but it should make our lives easier when it comes to the bulk of what we want to archive. Let me know what you think. It's important we decide on this now to make our lives easier in the future.

>>30296486
Tagging always makes things easier when it comes to the content of the image. It doesn't have to be super specific, but a few tags can make it easier to search for an image before adding so we can avoid duplicates.

>>30296692
I think for images we should upload them to Derpibooru, properly tag them and then download them off of it. That way, when we download images off of Derpibooru we can just save them directly instead of having to rename each and every single image that we download. Folderizing the images is another matter.

>>30296862
To clarify, when you download an image off of Derpibooru, it bases the title of the image off the ID AND ALL THE TAGS THAT THE IMAGE HAS. If we do it this way it'll save us a LOT of time and effort.

>>30296862
The idea was we could still add stuff to the archive even if Derpibooru, Pastebin, etc ceased to exist. I guess it wouldn't really matter since we'll probably upload must of the stuff within the near future while they're still up. Uploading to Derpibooru and then downloading from there would make things pretty easy since it adds the tags automatically to the file name. For Pastebin though, since the ID doesn't really mean anything, we should still have the author and title in the name so we can find specific stories.

>>30297097
Yes, you're completely right. But we should have the pastebin ID before anything else to prevent duplicates, same as the greentexts.

>>30296879
Windows has a file name limit of 216 characters.

>>30297097
Yes, you're completely right. But we should have the pastebin ID before anything else to prevent duplicates, same as the greentexts.

>>30297324
They save alright.

>>30293591
You create an intelligent atmosphere like /tg/ or take control of all future websites and ban idiots starting with Sethisto, Knightly, Orange, owner of Derpibooru.

Sadly as you can see you weren't quick enough and the /b/tards and fatlard sjw furfags communists took control of all the sites(even 4chan) and let all the idiots inside to propagate.

>>30295816
Start with bitcoins.

It's really sad how the cartoon show's currency was called bits, yet 0% of this fandom knew about bitcoins despite being created in 2009, one year earlier than FiM.

>>30298203
Never knew about bitcoins*

>>30298203
Isn't bitcoin fucking dead?

There's a right way to do this, but it requires a very much not-small (but not impossibly massive) starting investment, in the order of around $14k: https://pcpartpicker.com/list/LZ282R On top of that, an optional investment in an LTO-7 tape drive (+ tapes 1x/year) would not be a bad idea.

Slap FreeBSD on here with 1/2 of the m.2 drive dedicated to swapspace (make sure to harden the installation), and make the entire 24x SATA drives a ZFS pool with mirroring, sha512 verified dedup, and lz4 compression (optionally, bump up to an OC'd 1700 or 1800x and go for gzip-9), so that there's ~120 terabytes of available space (+dedup, +compression).

Stick this gorgeous piece of hardware into rented rackspace somewhere with a fat pipe and high levels of power redundancy (eg lots and lots of UPS's). Set up SSH to a personal computer using key-based authentication, turn off all ports on the machine except a few for RSYNC, HTTP GET, torrents, and SSH.

Back up every pone-related youtube video using youtube-dl, all of /mlp/ in a per-minute pull in a full static mirror, every MLP episode in every official version (DVD, blu ray, web-dl, EQG, etc), all of fimfic in plaintext (with the script they use for epub/html/pdf conversion bundled), best-possible quality music backups from anything posted to EQD and anything else that can be found, and any other sources of pone-related content any other anons can think of.

Every day, encrypt and upload a snapshot of the entire pool to Amazon Cloud Storage using RSYNC. Every year, write the entire pool to a series of LTO-7 tapes twice (either over SSH cp, or by physically going to the rack and plugging the tapedrive in), storing one set in one location, and another far away (eg in another state if in the US). Preferably, do it in a secure lockbox, one with fire/earthquake/flood protection and climate control. You can set the first part up with a simple cronjob.

I've thought this through very extensively.

I also don't have $14k. Or even $1k.

File: sold mine at 2500.png (4KB, 227x97px) Image search: [Google]

4KB, 227x97px

>>30299047
Very much no. You can buy any item from Dell or Newegg using bitcoin only, to name the 2 biggest points of sale for actual products. There's also thousands of smaller places using bitcoin, and Japan is moving to a bitcoin payment system as a standard alternative for cash or credit in almost every store in the entire country in a couple of years.

>>30299393
I didn't understand a god damn word you just said.

>>30299420
That's okay. I'm cross-board with /g/, and have spent a number of years outside of that place researching data integrity.

tl;dr: properly backing up every piece of MLP content reliably and faithfully using something like a bunch of external USB drives, or google drive (as I've seen some anons suggest) would be a *terrible* idea (I didn't elaborate as to why it was a bad idea in the post itself because >2k characters). The only way to do it *verifiably correctly* is to spend around $14k (+ maybe $(7500 + 250/year), depending on the amount of time you need to keep the backups alive for) as a starting cost, and do a lot of fancy software stuff afterwards.

>>30299393
That would be the very very end game, we were discussing ways to generate money to achieve something of this magnitude.
We have to think reasonably and make do with small stuff to start, just some basic scripts and backing up what anons come with slowly to start.
This is why we need to set a good database and take care of duplicates elegantly right from the start to optimize our early efforts.

>>30299465
We can organise that type of stuff in the future. For now we should be focussed on archiving as much content to our own machines as possible in anticipation of when we are able to properly put it all together.

>>30293496
Most of the claims are bs
IS fair use some of these poni videos

We could start a bitcoin donation

>>30299420
He is saying we run a computer with a lot of special functions that will be automatic and will do backups and other very useful stuff on it's own.

>>30299542
I don't have much currently in my pocket though.
Maybe in a few months.

>>30299511
>very very end game
Of course. I couldn't make that clearer in the character limit given all the other info, but this is obviously the real endgame of a working pony archive. A youtube-dl on popular pone channels would not be remiss. I have a bit of storage myself, which isn't fully reliable (but gets the job done), which I'll start backing up FimFic to. I don't know if I have enough space for any youtube, though.

A good database will be the most difficult part, probably. Most sources in the DB will probably drop like flies after a year or two.

>>30299512
Again, fair. Just remember: if you want to do it correctly, back up 3x, 2 onsite and 1 offsite (if you have an office, for example, leave a hard drive in there that you take back home once a month to sync up).

Ok so I made a possible nomenclature system that we could use.

Firstly, get mp3tag. It's a free software that edits the metadata of audio and video files. This will let us add stuff like titles, years, genres, authors/artists/creators and other stuff to video and audio files. Find it here:
http://www.mp3tag.de/en/download.html

>Music
Song name - Artist
e.g. Loyalty - Mandopony ft. AcousticBrony
Just an example

>Videos
Video name - Creator
e.g. Friendship is Witchcraft Episode 1 - SherclopPones

For both videos and music, use the previously mentioned program to give it tags like creator, title and year released. This will help us to sort stuff. Also, in the comment section of the metadata, put a link to the video and the channel.

>Greentext
Save this as a .txt file. Also, if you save greens from a general, like /nst/ for example, make a subfolder in your green folder for all greens from that general.

Title - Author - Pastebin ID
e.g. Obession - Sea Urchin - 5ECNnjxF

If the green was just posted in some thread and not on pastebin, leave out the pastebin ID. Make the author Anonymous if there is no name. If the green does not have a title and is, say, a few posts long, name it something to do with the green. Like 'Anon helps Twi find a book'.

>Art/animations
Use derpibooru filename

>Games
Game title
e.g. Rainbow Battle Simulator (made up name)

If it's a .swf or .exe or any other singular file, rename the file. If it's a folder, put it into a .zip and rename that.

Thoughts?

>>30300406
While the naming suggestions are great, I'd recommend MusicBrainz Picard:
https://picard.musicbrainz.org/
Or PuddleTag:
http://docs.puddletag.net/
To do the tagging. Alternatively, ffmpeg with copy streams can edit metadata of any container.
But yes, the naming suggestions are perfect. If and when pastebin goes down, however, I'd get rid of the pastebin ID from the txt, which can be done with a simple bash script.
Also, there are likely some images/animations/etc that aren't on derpi but are on e621, in which case we'd have to rename them to avoid collisions (since e621 names are just md5's).

Another thing I've seen suggested is a centralized database of content with tags on content type (and tags with creator names), and a primary and secondary source (eg the youtube video, and a local copy that you have and are willing to upload). With this, an endgame server like the one suggested in >>30299393 would be able to archive content quickly and automatically using the database.

You guys understand this is how archive.horse -> desustorage -> desuarchive started?

Hell, we (2 guys) spent $2000 on hardware and even wrote a tumblr archiver. It was bad ass.

The worst part is the hardware to host it. If you are serious, be ready to spend lots of money you'll never get back.

>>30301929
Follow up to this, if anyone wants to get in touch with us, #anonarchive on Rizon.

>>30301929
Does this archiver actually save the Tumblr theme?
Because the only working one I know is https://github.com/bbolli/tumblr-utils and that one saves the blog in its own layout.

File: file.png (255KB, 1840x1332px) Image search: [Google]

255KB, 1840x1332px

>>30302042
We used a DB to hold everything. And no, theme saving was something we were working on, but never got around to. The project has been abandoned for quite some time now.

>>30302082
Maybe you can contribute it to the github project I linked?
I'd really like the possibility of archiving a blog as it is. But using tools like wget/curl/httrack/etc requires way too much work to get everything working.

>>30302125
https://github.com/woodenphone/tumblrsagi

File: file.png (242KB, 2498x1460px) Image search: [Google]

242KB, 2498x1460px

>>30302144
Also

>>30301929
>If you are serious, be ready to spend lots of money you'll never get back.
I am killing myself once the thing gets rolling anyways.

>>30302981
We could have someone take out ALL the loans and then become An Hero for the rest of us.

>>30303120
Eh I don't hate my family.
Unless you are 100% sure there is no fucking way they can charge them if I killed myself.

>>30303165
They definitely would charge them.

BTW don't kill yourself, this is all we have you know. If you aren't happy with your life then change it.

>>30303358
I already made up my mind.

Aight