I'm involved in the development of an openssl engine. Prior to this project, i collected a lot of experience with the higher level functions of the openssl API. But god damn, the documentation for the low level operations is almost nonexistent.
Anyways, we mean to replace the ECDH algorithm with a new candidate. To keep it dynamic we chose to make it an engine. The ECDH replacement will be set with the ENGINE_set_ECDH function but I have no focking clue when this function will be called in the actual TLS process. Which brought me to this dark place.
Are any experienced openssl developers present who know how this clusterfuck works internally?
lol bro unless youre writing that shit in asm, its going to suck
>>55677079
I see you already mastered this field
>>55676980
You're looking to /g/ for advice on development involving a major FOSS project? Are you high?
>>55677392
Desperate anon, only desperate. You wouldn't believe me how exhausting it is to fight yourself through this jungle of undocumented code
As a slightly more than casual user --
I'm under the impression OpenSSL is a gruesome library, and that LibreSSL and BoringSSL forks will eventually replace it.
Is that possible?
>>55677435
I admittedly know nothing of the SSL process, but you can always fall back on stack traces and step-by-step debugging to discover the program flow. It'll suck, but at least you'll have a better grasp of the internals. Couldn't you also inject your "engine" as a skeleton just to see how it's invoked or even modify an existing one with your own trace code to identify it as yours?
>>55677472
I would switch this moment to a forked openssl but considering that the final product is intended to be delivered to a broad userbase we have to stick with the mainstream. Sucks, i know.
>>55677497
Sadly I'm already doing it. Taking into account that we already have a working standalone implementation of the algorithm, it's just incredibly frustrating that it's not possible to just add it to openssl with a few lines of code. Guess i have to keep using their code, a debugger and other engines as reference material.
> FML
>>55676980
There was one written in Rust, it might be clearer to read
https://github.com/ctz/rustls
>>55676980
> OP on thinkpad
>>55677718
Looks more like a replacement for openssl instead of an engine which can be loaded dynamically at runtime.
I appreciate your help though
>>55676980
>Are any experienced openssl developers present who know how this clusterfuck works internally?
If #openssl can't help you, anon, noone can.
That said, the ENGINE api is for cryptographic accelerators, so I imagine you implement your accelerator, load it, then select it as the active implementation explicitly withENGINE_set_default(engine, ENGINE_METHOD_ECDH);before starting any TLS transactions.
>>55677792
I'm going to ask around on #openssl then
Who told you the ENGINE api is for accelerators? AFAIK it's meant to give dev's an 'easy' way to change/add/remove digests, ciphers, rand functions, etc.
O' I'm well capable of loading my engine and setting it as default. Still the underlying structure of the key exchanges are not so transparent from the 'ENGINE' perspective. It's easier to just add your algorithm to openssl's codebase. But then you'd have a fork of openssl. Which kills the mainstream points and requires ressources to maintain it. So it's not really an option for me
>>55677888
>Who told you the ENGINE api is for accelerators?
That's what it was (originally) intended for, not that it really matters:
http://web.mit.edu/crypto/src/openssl-0.9.6a/README.ENGINE
The name is a giveaway: Hardware implementations of cryptographic primitives are traditionally called 'cryptographic engines'.
The actual key exchange process will of course be opaque to the engine - cryptographic engines (both hardware and software) only implement the primitives used by a cryptographic process, not the process itself. Unless your key exchange process is exactly analogous to ECDH, you won't be able to implement it as an alternative ECDH engine - the actual key exchange that happens on the wire is implemented by OpenSSL itself, not the engine.
But desu you probably know more about this than me anyway, since I've been lucky enough not to have to work with OpenSSL internals myself.
>>55678274
> That's what it was (originally) intended for, not that it really matters:
The more you know...
That's exactly the problem here though. As far as I understood it, there shouldn't be a problem with the actual encryption algorithm (once it's implemented). But to make the whole concept secure against a specific kind of attack, a minor change in the TLS protocol is needed. Just placing the servers keyexchange a bit later in the handshake. So even if I manage to implement the damn algorithm, I still have no focking option to change the handshake without forking ssl. inb4 i don't have the ressources to maintain a whole fork. You woulnd't know another way to accomplish my goal, would you?
> sauces
> https://www.openssl.org/docs/manmaster/crypto/ENGINE_set_ECDH.html
> https://wiki.openssl.org/index.php/Creating_an_OpenSSL_Engine_to_use_indigenous_ECDH_ECDSA_and_HASH_Algorithms