Incorrect handling of OpenSSL errors causes dropped connections #39
Description
Asio does not handle errors reported by OpenSSL properly. This may (has for me) result in errors later on which are very hard to troubleshoot.
When an error occur in OpenSSL, errors-codes are pushed onto a queue. A single call to OpenSSL may result in multiple error-codes being pushed to the queue. For example, a low-level component in OpenSSL may push one error to the queue, while another higher-level component may push another.
After Asio has executed an SSL function, it only removes the first error from the queue. If more than one item have been pushed to the queue, the remaining items will be left behind in the queue. When Asio calls a SSL function later on, and checks the queue, it may pick up the error from a previous operation, and think that the current operation has faulted, even though it hasn't.
For example, you may call async_read and that will fail with an error related to loading of private keys, even though the call to load the private_key was made yesterday.
Easiest way to see this issue:
context.set_password_callback(GetPasswordCallbackFunc);
context.use_private_key_file(keyfile, boost::asio::ssl::context::pem);
int err_after = ::ERR_peek_error();
Make sure that GetPasswordCallbackFunc reports an incorrect password, so that the private key file cannot be opened. When this happens, OpenSSL will push two errors to the error queue, but Asio will only remove the first item. The call to ::ERR_peek_error() will show that there are errors left behind in the queue. If you some point later in time tries to perform another SSL operation, such as async_read, that will fail because there's old items left on the error queue. Or rather, the actual read will succeed but Asio will report it as failed)
I tried clearing the queue in use_private_key_file but there are other SSL operations which pushes multiple errors to the queue. (I'm not sure which though)
Some of the functions in Asio calls....
::ERR_clear_error();
.. prior to do anything but far from all of them. According to what I've read, always calling this method prior to calling OpenSSL functions is the proper way to go.