Does /g/ check their checksums?

if you're concerned about someone MITM'ing your downloads, why would you trust a checksum sent to you along the same (encrypted or unencrypted) connection? why wouldn't they be able to MITM that as well and just send you a different checksum?

checksums are only helpful for verifying that your download transferred correctly — that is, without some random packets being dropped or whatever. using it for security is retarded unless you're getting the checksum from a completely different method that you trust with absolute confidence. which is probably unlikely.

>>57859236
What this guy said

>>57859236
or checking your 20gb reaction face folder for duplicate files.

>>57859236
The example that comes to mind is the 64bit linux mint cinnamon17.3 iso on feb 20.
The website was hacked and the download link was changed to a trojanized iso on a bulgarian file server. The md5 link was unchanged. So anybody who checked would have been OK. I bet that's how the hack got noticed.

Data integrity mostly, checksums were never to provide security.

If you want authentication you use certificates/signatures

>>57859396
checksums get used for all sorts of weird shit, but this is just a weird hack of its use. checksum matching only tells you if you have absolutely identical images. if one image is off by a literally imperceptible amount, or even differently scaled, it won't match up.

>>57859514
i was going to add that the only way checksum can be useful for determining fraud is if the hacker in question is both reasonably competent (at least competent enough to pull off a hack) and also simultaneously so hapless as to be borderline retarded. like if you imagine Mr. Magoo being a hacker. but i thought "no, surely no hacker has ever been that stupid".

i guess i was proven wrong!

this is a little like the warrant canary defense. it's not clear (and certainly not established in any precedent that i'm aware of) that a warrant canary is at all effective. it's possible that a secret court could compel a company to lie about the status of its warrant canary and we would never know. i'm not even sure whether a warrant canary is considered legally binding in the first place, so the argument that it puts a person between a rock and a hard place (on the one hand to disclose a secret subpoena, and on the other to perjure oneself) is at best murky and at worst really ill-thought-out. an affidavit or a statement given under oath would be dicier, but i'm pretty sure a judge would tell you to perjure yourself rather than give up secrets that they already ruled to be material to national security or whatever horse shit.

>>57859675
and to the first comment bit, checksum obviously has the same issue with other kinds of files but a PDF or text file is just a lot easier to parse for differences.

and checksum is incredibly frustrating for image duplicate checking because the outputs for two almost identical images are generally completely different from one another. the output is absolutely useless in determining similarity; just identicalness.

a much much more robust system would be something that would fundamentally "look" at the image, but obviously most people aren't interested in running something like that. checksum comparisons are insanely easy and really fast. some sort of computer vision is pretty much the opposite in both dimensions.

>>57859720
>>57859396

actually, similar images are still not identical, you could want them both, different resolutions or color balances.

BUT for all medias, checksum is stupid simply because of metadata.

i dont bother to check for duplicates, even tens of thousands of highres porn photos are just a fraction of video, and videos sorted by type or actress are easily humanly checked and too complex for computer to check.


what i really want is an image set-grouper, that can group photos of the same person across different angles and clothes, including when face isnt visible, that would be so porn-autistically satisfying

>>57859184
I don't give a shit about security. I just use everything Russian as I trust them more. That's the extent of my security efforts.

All the games I play make such a big deal about checksums even though I have no idea what they are and what they do. So I just ignore it and keep playing.

>>57859184

Bruce Schneier is maybe a miserable computer user. Poor guy. His cap is cute.

>>57859184

Where do I get those contact lenses.

>>57859236
what is gpg??

>>57861401

You could feasibly construct a checksum scheme for images making use of locality-sensitive hashing.

As others have mentioned, checksums do not guarantee security. You're thinking of message authentication codes or signatures (in the asymmetric case).

>>57859236
>checksums are only helpful for verifying that your download transferred correctly — that is, without some random packets being dropped or whatever

>Ethernet has checksums so corrupted frames can be identified
>TCP has checksums and sequence numbering so that corrupted and/or missing segments can be identified
>application layer software should do error/consistency checking as well
>somehow user still has to check the hash himself

>>57861553
I refuse to believe you've never downloaded a fucked up file

>>57861566
OK, but how does that even still happen if there's integrity checking implemented all over the whole network stack? That was the question.

>>57861577
I actually have a background in networking, so I can speak to this a little bit.

The checks built into the protocols arent perfect. Most of the standard ones (excluding weird physical-layer stuff) uses an algorithm called the inverted one's complement sum. Basically, there are certain patterns of bit corruption (one every 16 bits), that can go undetected. Some protocols will even allow the checksums to be zeroed out and be considered valid (though, this usually doesn't happen).

>>57861674

This. In theory, yes, you can devise a statistically optimal checksum scheme, but for practical reasons, be they related to legacy or efficiency, we choose something simple yet slightly flawed. The public Internet is also not uniform in operation, meaning that entire sub-sequences of nodes along any one path may completely skip integrity checks.

>>57861401
i think i have an issue with lots of duplicates just making navigation too confusing. i have something like 17 TB of capacity, maybe 10-13 of it used, so i have *plenty* of space to download more, but i've definitely got redundant copies of a bunch of JAV porn. some of it is higher quality than others (higher resolution, no watermark, etc...) but in some cases there are just redundant copies in other directories, or even in the same directory under slightly different names. it probably represents.... 200GB? just a random guess.

i've thought about crowdsourcing the solution, but i don't want to serve 10+ TB of porn to the world and have them all start cataloging it and potentially get in trouble for any number of shit (copyright infringement would be the first hazard).

>>57861505
the safest way to use GPG is to get the key to a recipient using a method that you trust. the problem is that if you trust the internet, then why are you even really bothering with GPG? the imperfect solution is to send it via the internet but along a different path. so for instance you broadcast your public key on your personal site and hope/trust that someone intercepting your emails isn't also intercepting traffic from your site traffic.

but this goes to one of the most important maxims of security: prepare for the threat you'll encounter. if your most sensitive work is planning a surprise birthday party, you probably don't need to worry about a comprehensive state sponsored attack. if (people know that) you're negotiating the sale of weapons grade uranium to north korea, you probably can't trust anyone to receive your GPG key if you send it over the internet. you should just assume it's been modified in transit. you need to coordinate an in-person meeting and exchange keys in person.

>>57861553
checksums are a solution to a problem from a different time. now they're anachronistic, but some people still use them.

>>57861674
The physical layer provides error detection, but not recovery. The bad frames are simply dropped. Though, the missing segments should be detected at the transport layer, which (TCP at least) performs both error detection _and_ recovery, and should request any missing segments to be resent. There there's possible integrity checking at the application layer. How does that all fail?

>>57861577
Because sometimes the file on the server is damaged and nothing is damaged in transit.

>>57861785

A lot of it comes down to the exact integrity mechanisms in place. If you're just performing a simple file transfer over a vanilla TCP socket, the ones-complement sum checksum has mathematical "blind spots," so it is possible for errors to remain undetected.

It does depend on your stack. If you have mechanisms for guaranteed, ordered delivery with cryptographically enforced message integrity (e.g. using an HMAC), you don't need to do any manual checks. This latter example would be the case if you were to, say, do the same file transfer over a TLS socket.

>>57861785
I'm not suggesting TCP can't recover from most data corruption; simply stating that some corruption *can* go undetected.

However, something TCP can't really protect from is a premature 'reset' or 'fin' packet. If your client doesn't receive the whole file, it might not convey this to the user.

>>57861804
also, this.

File: 1461623264949.png (269KB, 389x431px) Image search: [Google]

269KB, 389x431px

>>57859184
Yes because the checksum is sent over https, but the download often over http to save encryption overhead on the servers.