Proposal: Improved Browser Connectivity using Secure WebSockets

[marten-seemann]

This is a proposal to make it easier to use secure websockets (wss). Currently, users wishing to offer wss to the network need to run an HTTPS server (e.g. an nginx) with a valid TLS certificate configured in front of their libp2p node.

Benefits

Browser nodes have limited options to connect to the rest of the libp2p network. There's no way to establish raw TCP or QUIC connections, and WebSockets / WebTransport connections are subject to the Web Security Model (i.e., the server has to present a CA-signed TLS certificate).
The only exception to this rule is WebRTC, which uses self-signed certificates. However, browsers enforce pretty strict limits on the number of concurrent WebRTC connection.

Limitations

It is understood that this is only an option for public nodes, and only if they have a domain name configured. Therefore, only some of the public nodes offer wss support will actually make use of wss. However, this is still a lot better than the status quo, as this is an additional (and not as strictly limited as WebRTC) connection option for libp2p nodes.

Obtaining a Certificate

There are 3 options to get a valid certificate:

1. Import an existing key / cert file. This is probably not used very commonly nowadays.
2. Use the ACE HTTP or TLS challenge. Note that this involves starting a HTTP or endpoint on port 80 or 443, respectively (this is a limitation imposed by the ACME protocol). Note that this will pose a problem for nodes that run a HTTP(S) server on their node, or nodes which can't bind to ports < 1024.
3. Use DNS challenges. This will require user configuration, as DNS entries have to be modified temporarily to pass the ACME challenge (see the certmagic documentation for details)

In practice, it's only possible to obtain certificates for domain names, but not for IP addresses. It thus only makes sense to dial /dnsaddr/domain.xyz/wss addresses. Dialing /ipX/<ip>/tcp/<port>/wss addresses can still make sense for non-browser use cases, where we can ignore certificate validation errors (or, if we want to be fancy, distinguish between these two uses cases and use a libp2p-tls certificate in that case).

Implementation Details

go-ws-transport will add a new configuration option WithCertificate(*tls.Config). When set, it is possible to listen on /wss addresses.

certmagic looks like it has everything we need, and we can offer all of the 3 options listed above to configure the certificate.

The domain names that we obtain certificates for could be configured manually, or extracted them from /dnsaddr/ multiaddrs. It is unclear if this logic should be part of libp2p, or left to the application.

cc @willscott @vyzo @aschmahmann @aarshkshah1992 @mxinden @Stebalien @patrickwoodhead

[achingbrain]

This is going to solve a lot of problems for browser nodes 👍

Tangentially related but I put a POC together a while back that does a reverse-DNS lookup on your router's external interface and uses uPNP to configure port forwarding for long enough to do the ACME handshake based on that domain name - https://gist.github.com/achingbrain/8d9a00190f2e13182579d26d138f5b4a - It also means you don't need to run the server on privileged ports.

[BigLep]

@marten-seemann : I've seen conversations from @lidel on this in the past. Please make sure he's looped in.

[BigLep]

There is also protocol/web3-dev-team#70 from 2021.

[lidel]

There are also some notes in ipfs/in-web-browsers#181,
TLDR being the various degrees of automation:

• Stage 0: bring your own certs, manual config (issuance and renewal happens outside)
• Stage 1: bring your own domain, manual initial config (then issuance and renewal are automated via ACME challenges)
• Stage 2: TLS out-of-the-box, no manual config (either PL provides a subdomain, or we do reverse-DNS lookup trick, or both depending on network topoligy, but the end goal is that user does nothing)

/wss story has been blocked for years because we were aiming at full automation (2), which comes with some caveats. I'm 👍 for doing this one step at a time, to keep the scope in check.

👉 I suggest we focus on (0) first, and allow users to manually provide certs for domains via config:

• ACME is convenient and popular, but there will always be security-sensitive deployments which prefer to do own key and cert management, so manual cert support will remain a useful (and requested) feature,
• Small scope, easy to test end-to-end, and to wire up in upstream projects like go-ipfs.

When we have this, we could look into (1) and (2).

(I'm 👍 for having built-in ACME support but that usually baloons complexity across the stack. This is because when we have ACME, full automation (2) is very close. For example, PoC by @achingbrain is worth exploring, as it would work in many home NATs – the publicly diallable IP could be read via AutoNAT, and thanks to liberal defaults in consumer routers we could attempt to map public port 80 to local port >1024 for HTTP Challenge. It will fail because some router models may block low port 80, but then we could fall-back to a subdomain and DNS-challenge via centralized infra provided by PL)

[marten-seemann]

either PL provides a subdomain, or we do reverse-DNS lookup trick, or both depending on network topoligy, but the end goal is that user does nothing

I agree this should be out of scope. Providing a subdomain brings us into all kinds of trouble, first and foremost ACME rate-limiting. Reverse DNS seems straightforward enough, but will potentially require some more wiring up into the libp2p stack (just obtaining the cert is only the first half, we also need to advertise a /dns/<domain>/wss address).

Assuming a wss listener support in go-ws-transport (I have a working prototype that shouldn't be too far from being shippable), both (0) and (1) seem pretty straightforward (this is mostly due to the fact that certmagic is such a terrific library).

I created a small wrapper in https://github.com/marten-seemann/go-libp2p-certbot/. It supports both options:

• (0): it's 1 LOC, see https://github.com/marten-seemann/go-libp2p-certbot/blob/c6961b3edd687b2735bea526c3fb549f6dd673c7/certmanager.go#L123-L129
• (1): given a list of (/dns) multiaddrs, the library performs ACME for every domain. I'd call this semi-automatic, since the user only has to configure its listen addrs.

The tls.Config obtained from go-libp2p-certbot can directly be plugged into go-ws-transport.

---

@achingbrain's POC looks very interesting, but will probably require us to assign (sub)domains, right? Home IP addresses change very frequently, so we'd probably need a way to quickly update DNS records or obtain a new subdomain. That's a whole lot more complexity, and I'd consider this out-of-scope for my proposal here. Let's focus on public nodes with pre-configured domains for now.

[marten-seemann]

Closing, since we now have WebTransport and that's 100x better than WebSocket. Specifically, it doesn't require any DNS setup.