(Configurable) hard limit on the number of open TCP connections

[Firobe]

In line with #534, this is Edwin's patch to put the hard limit to the number of TCP listens in progress, with some light modifications to make this configurable at connect time. It's intended as a last resort limit for this layer (but still before another limit in the lower layer: mirage/mirage-net-solo5#49)

cc @edwintorok

[Firobe]

Added a pair of commits to refresh the CI a bit :)

[hannesm]

Added a pair of commits to refresh the CI a bit :)

I guess that'd be best as a separate PR to be merged quickly.

So, you propose a "(Configurable) hard limit on the number of open TCP connections"? Your implementation seem to only care about the listens table, which are the partially opened server connections -- and not about the channels (fully established connections) neither the connects (partially opened client connections).

I wonder why, and wouldn't esp. for the listens table be an implementation of syncookies be sufficient (to avoid having to store much on the server side for any incoming SYN packet)? Is the mitigation primarily for syn floods, and in this case is it sufficient and the best solution?

[edwintorok]

I wonder why, and wouldn't esp. for the listens table be an implementation of syncookies be sufficient (to avoid having to store much on the server side for any incoming SYN packet)?

Although SYN cookies are a widely deployed solution, it can lead to silent data loss.
See SYN cookies ate my dog, then told me I never had a dog, just a low MSS

Having the option to turn SYN cookies on or off might still be useful though, especially if the application has other ways of detecting missing or corrupted data (e.g. through TLS, so SYN cookies could be turned on by default when TLS is in use without the fear of undetectable data loss).

From the linked article:

The problem of missing data can potentially affect any client-server TCP application in which SYN cookies are enabled on the server, the client speaks first after establishing a connection, the first packet it sends to the server is three bytes or smaller, and further data is sent before the server is expected to reply and before the client TCP stack resends the first packet.
On Linux, if the client’s first packet is more than three bytes and a second packet is sent, this will not cause data to go silently missing from the TCP stream but it may cause the connection to be reset.

I was not actually aware of this until I started seeing some problems with SYN cookies myself, where legitimate traffic was getting stuck.

It can also lead to performance issues, although if you only turn them on when you detect that you are being flooded that may be acceptable.

Is the mitigation primarily for syn floods, and in this case is it sufficient and the best solution?

SYN floods are one reason (and SYN cookies help there), but there is another attack which is not thwarted by SYN cookies. Just like how like SYN cookies avoid having to keep state server-side for connections, similar techniques can be used by an attacker client application, i.e. it is possible to write an almost entirely stateless TCP client capable of opening a nearly unbounded amount of connections by fully completing a 3-way handshake without actually keeping any per-connection state (I've written such a stateless client myself, after finding a C implementation of this from 2008).

So it might be useful to limit both the amount of connections that are being established, and the amount of already established connections.

[Firobe]

Added a pair of commits to refresh the CI a bit :)

I guess that'd be best as a separate PR to be merged quickly.

Done in #536

So it might be useful to limit both the amount of connections that are being established, and the amount of already established connections.

This makes me realize that the naming introduced in mirage/mirage#1630 (comment) is unclear. I refer to open connections, while we're actually talking about "pending" connections I guess. If we end up limiting both listens and channels I'll revise the naming.