Thread replies: 13
Thread images: 1
Thread images: 1
Anonymous
877.06 GB (compressed) CSV file 2017-02-28 07:51:05 Post No. 59168288
[Report] Image search: [Google]
877.06 GB (compressed) CSV file 2017-02-28 07:51:05 Post No. 59168288
[Report] Image search: [Google]
File: Screen Shot 2017-02-28 at 2.50.10 PM.png (28KB, 394x163px) Image search:
[Google]
28KB, 394x163px
What kind of a computer do I need to analyze this data set?
>>
define analyze
pdp-8 would be more than enough, just give it time
>>
>>59168310
text search with <5 sec response time
>>
>>59168288
Something with at least 1TB RAM?
>>
>>59168310
also, regex search (something like 10.*.1.3[4-5]) or arbitrarily masked IP address ranges
>>
>>59168388
*of
>>
>>59168288
is it actually json? i had issues finding a json parser that didn't try to eat the whole file, i had to write one myself.
>>
The new Apple™ MacBook Pro© with Retina® Display
>>
>>59168288
Store it in a compressed database like Apache Cassandra.
Cassandra can also easily distribute datasets and queries across multiple machines.
Its downside is a restricted dialect of SQL called CQL.
>>
>>59168678
>Not using Hadoop and Spark
>>
You probably want to index it first.
>>
Perl can do it in 5 seconds on an Amiga 500.
>>
>>59168288
How data contains file?.
Thread posts: 13
Thread images: 1
Thread images: 1