Ask a guy who archives everything anything? Pic probably related

>>551839536
What do you use to archive?

>>551839536
Archive this!

>>551839536
I archived your mom last night

Archive this!
**Points to genitals**

>>551840086
I programmed it myself.

>>551840221
Already have.

>>551840698
Nice m8

>>551839536
Why?

>>551841212
So I can get statistics on what you shitheads talk about and then sell it to large companies for mass profit. Why else?

>>551840929
Could you upload a copy of the software? I archive a lot of threads too, and wouldnt mind streamlining the process a bit more.

>>551841283
Can you post the 10 most frequent words on this board?
>inb4 top 3 is neufag, summer and autism

>>551841283
lel
you're just going to make everyone look like a faggot

go ahead and do it

>>551841354
It's a web service using SQL, I don't have it in the form of an executable unfortunately. It also has proprietary image detection to identify bodies to remove any explicit images that get archived.

>>551841283
99% of threads are shit

>>551841426

you forgot op and fag

How far back does this go?

>>551839536
Is this archive accessible on the web somewhere?

>>551841484
>remove any explicit images
>archives 4chan
Why bother?

>>551841610
I assumed 2 and 3 character words would get cut out of the search for performance reasons, that's how i'd do it to get faster results anyway(tip: count the number of them on my post)

File: 1372457999333.jpg (109KB, 640x963px) Image search: [Google]

109KB, 640x963px

>>551841484

>>551841674
Just over 4 years

>>551841513
dog is dead wat do

the most used words are prolly nigger, op, is, a, fag, /b/, mfw and rate

>>551841774
I can tell you that roll is the most commonly posted word, pretty boring.

>>551841484
post the code on github or pastebin plz

>>551841928
4chan confirmed for shitty.

>>551841989
This is my income, no.

>>551842095
The number of non-oc OC threads that people believe are OC which aren't are upsettingly high. Also, the general frequency of new posts do not change much between summer and any other season.

>>551842401
People before profits, man. Don't be a nigger... give us the code

>>551842701
nigger is the 14th most common word on 4chan, man this place sucks

>>551842401
According to this, this place is full of newfaggotry then.
>>551843065
We need more nigger.

If "faggot" more popular than "nigger"?

my friend has been being harassed by /b/tards and it would be great to know the time distribution of when the threads are up. No chance of you hosting this archive online is there?

I don't believe you. You have no proof.

>>551841283
what the hell kind of fucked up company wants /b/'s info?

>>551845103
fcking true

>>551840086
use fireshot an extension for firefox

>>551845190
I see threads all the time where guys are trying to shoot bigger loads. If someone had developed a pill that let you shoot massive wads every time, I feel like marketing it towards /b/tards would be a great tool.

>>551845190
oh shit it's no company
<spoiler>he's nsa</spoiler>

>>551845317
playah ID

>>551845373
>implying btards have sex
there's already a market for that pill; it's called l.a.

>>551845521
Never heard of it. Why don't all of those threads just end with a link to the product? Horrid marketing.

>>551839536
this is fucking impressive GJ op. Also how many legit camwhores are there?

File: heaven.png (135KB, 580x512px) Image search: [Google]

135KB, 580x512px

>>551845190
You'd be surprised.

>>551845103
>"The stories and information posted here are artistic works of fiction and falsehood."
>Expecting truth
Heaven you never cease to amaze me.

>>551845380
It's getting a bit warm in here.

>>551846079
surprise me

>>551846270
A lot of web hosting companies and SEO web masters take interest in this type of information.

>>551846079
thats actually really cool

do you have a shitton of pictures?

>>551839536

Is this you OP?

https://www.youtube.com/watch?v=P4hmcy4zmM0

how much memory does all this take up?

>>551846674
/b/ rapes SEO for lulz, doesn't make a lot of sense. we're not a target market for anything good in this world.

>>551847086
kekked a little, ty anon

>>551847224
About 40GB of images a day. It's a terrifying amount of data, for the very same reason there is a small page limit on all the boards.

File: chomp2.webm (2MB, 1280x720px)

2MB, 1280x720px

>>551847395
saving webms, too?

>>551847612
Yup, they're mostly porn though. They actually cause quite a bit of trouble since it's hard to gauge whether they are explicit or not because it's not as easy to identify content like you can in a static image.

Do you have the third part of Getting Clean for Karina. The thread was posted yesterday

>>551847885
You're in luck, give me a moment

>>551847748
your favorite thread of the last 4 year?

>>551848202
I will be wainting fo you then. Meanwhile, do you think there is a pattern in making subjects or replying ? I always thought that /b/ is always about the same things again and again.

>>551839536
do you know who UTV is?

File: ka.png (151KB, 548x350px) Image search: [Google]

151KB, 548x350px

>>551847885
>>551849160
Was this thread removed by a moderator? I'm finding references to it but the thread has been removed from the DB, if there was a substantial amount of explicit stuff or a moderator deleted the threat, the likelihood is the system purged the thread from the DB to avoid storing CP and other stuff we could get in trouble for.

If you have the title of the thread I can have another look.

>>551849995
A stalker

hmm i need proof, give me a cp thread from one of your archives

>>551849998
It 404 but there was no CP or anything shady involvee. There were the regular hateful comments about OP that's all. I don't have the title of thread.

Thanks for trying mate

>>551846893
i like to expand on this, as i have quite a collection of pictures that i mostly saved off 4chan.
do you know hydrus network client? it's the tagging program that i use and it makes finding relevant pictures much easier.

how do you find shit in your programm? do you 'tag' roll threads as such? what would you need to do to get a list of all spaghetti greentext stories from the last month?

do you organise pictures for private use?

and another thing: could your system be used to reliably remove recurring threads like (for example) gay furry threads? they almost always have another OP text + pic, so what's the criteria here? dont want to set up a filter that hides meta-discussion about those threads.

>>551845407
how ironic

>>551850663
>hail hydra, confirmed for joke

>>551839536
There was someone who used to post under the name "Jack Ryan" or "House" with pictures of Ben Affleck and Gregory house on /b/ when you used to be able to post with usernames... can you give me any information on this person or the last time they were around and where?

>>551850663
Images still maintain a relationship with the parent thread. Because of this, images can be pulled up if you search by a string which exists from within the parent thread. It can also find duplicate images and give statistics on how often an image has been reposted, this is great for determining whether something is actually OC, but this can take a while.

To get threads with greentext stories would be quite difficult to do quickly, but it is generally as simple as making a SQL query to fetch posts that have greentext (the DB maintains the html formatting/spans) and then pull up all the parent threads.

I do organize some pictures for private use, it's quite fun to flip through them all.

About removing furry threads, I'm not too sure, you could perhaps detect images which have been reposted from threads known to be furry threads but otherwise it's risky to do this automatically as you may end up losing an interesting thread with just one furry picture posted at random.

>>551839536
Do you remove the Celeron Processors?

File: 1174770651129.jpg (103KB, 1350x832px) Image search: [Google]

103KB, 1350x832px

>>551851552
how does image detection work? you've mentioned that you can filter nude pictures. is it failsafe? what about things like this? does it use nipple detection?

>>551851552
How many ylyl threads do you own?

>>551851745
>nipple detection
I lol'd. I'm not the one responsible for programming most of the image detection, but I have a vague understanding of how it works. As far as I know it attempts various methods of detection, ranging from forming a basic skeleton, detecting skin colours and faces, if any one of these flags up anything even slightly then it will remove the image. It's extremely sensitive so there are lots of times where it will delete something not even remotely sexual, this can be seen in wallpaper threads. But it is a fine price to pay for essentially not saving CP and having to worry about that.

I can't find the image you posted in the DB so I should imagine it detected it.

>>551851745
>Nipple detection.
Kek'd

File: dasodpsi.png (6KB, 134x42px) Image search: [Google]

6KB, 134x42px

>>551852059
An eternity of keks...

>>551852412
oh shit nigger, nearly 5k

>>551839536

Where's MH370 ?

>>551852412
You also own a picture of my dick, you fag

>>551852893
Ordered by size in descending order, I'd never find it anyway.

>>551839536

Do you have the thread with 100000000 get? Please post son.

File: 1378824348092.jpg (2KB, 126x126px) Image search: [Google]

2KB, 126x126px

>>551852994
oh no you didn't

>>551852994
It was a fun picture with eyes and teeth.

>>551853013
I looked but couldn't find, I don't think that happened in the last four years. Sorry to disappoint.

>>551853608
2mil? 3mil? 4mil then?

do you have any traces of mod works? do you know when a thread is being forcibly removed instead of just 'expiring'?
as a lurker, you don't really see much being removed and if you do then it's pure luck. could your tools be useful for mod work?


also didnt there used to be an official 4chan archive? i know that i used the filenames there to gauge the date that i started lurking.

>>551851270
OP, please this!

File: 1294149432.jpg (5KB, 222x227px) Image search: [Google]

5KB, 222x227px

>>551853835
Try 300 mil "Get!" - Pic related

>>551853864
Sometimes a thread disappears moments seconds after a post was made which suggests it can't have 404'd. Usually this is because of mod intervention, I think I have a few examples actually where a user uploads a picture of someones credit card, then the thread gets removed promptly, I will try and find it.