MG_TLS_BUILTIN: optional two-way auth [was: malformed empty Certificate message in TLS 1.3 (RFC 8446 violation)]

[saidalbardawil]

• My goal is: Connect to an MQTTS broker using MG_TLS_BUILTIN without providing a client certificate. The broker sends a CertificateRequest during the TLS 1.3 handshake but does not require a client certificate (fail_if_no_peer_cert = false).

• My actions were: Configured mg_tls_opts with only the server CA certificate and skip_verification = 1. No client cert or key was set. Connected to a TLS 1.3 MQTT broker on port 8883.

• My expectation was: The TLS handshake should complete successfully with an empty Certificate message (no client cert), and the MQTT CONNECT should be accepted — the same way mosquitto_pub (OpenSSL) connects to the same broker without a client certificate.

• The result I saw: The TLS handshake appears to complete on the client side (keys are derived, Certificate + Finished are sent, MQTT CONNECT is sent encrypted), but the server silently closes the connection every time. Logs:

138c   3 mongoose.c:4693:mg_connect_svc 1 -1 mqtts://broker:8883
13d7   3 mongoose.c:6807:mg_connect_res 1 192.168.x.x:54890 -> x.x.x.x:8883
16fd   3 mongoose.c:4658:mg_close_conn  1 -1 closed
2714   3 mongoose.c:4693:mg_connect_svc 3 -1 mqtts://broker:8883
2a7a   3 mongoose.c:4658:mg_close_conn  3 -1 closed
3a9c   3 mongoose.c:4693:mg_connect_svc 4 -1 mqtts://broker:8883
3d83   1 mongoose.c:685:mg_error        4 -1 peer RST
3d83   3 mongoose.c:4658:mg_close_conn  4 -1 closed

The same broker works fine with mosquitto_pub without a client cert:

$ mosquitto_pub -h broker -p 8883 --capath /etc/ssl/certs/ -t "test" -m "hello" -d
Client null sending CONNECT
Client null received CONNACK (0)

• My question is: The mg_tls_send_cert() function sends a malformed empty Certificate message when cert_der.len == 0. Instead of an empty certificate list (length 0), it sends a list with one entry containing 0-length cert_data, which violates RFC 8446 Section 4.4.2 (cert_data minimum length is 1). Could this be fixed?

Root Cause Analysis

In mg_tls_send_cert(), when cert_der.len == 0:

size_t total_size = tls->cert_der.len + 5;  // = 0 + 5 = 5

This creates a certificate list with one entry (3 bytes cert length + 0 bytes data + 2 bytes extensions = 5), but a 0-length cert_data is invalid per RFC 8446.

What Mongoose sends (wrong):

0B 00 00 09 00 00 00 05 00 00 00 00 00
                        ^^^^^^^^^ 0-length cert entry (invalid per RFC)

What RFC 8446 requires (empty list, no entries):

0B 00 00 04 00 00 00 00
               ^^^^^^^^^ empty certificate list (length 0)

Suggested Fix

Add an early return at the top of mg_tls_send_cert() for clients with no certificate:

static bool mg_tls_send_cert(struct mg_connection *c, bool is_client) {
  struct tls_data *tls = (struct tls_data *) c->tls;
  if (is_client && tls->cert_der.len == 0) {
    uint8_t empty[8] = {MG_TLS_CERTIFICATE, 0, 0, 4, 0, 0, 0, 0};
    mg_sha256_update(&tls->sha256, empty, 8);
    return mg_tls_encrypt(c, empty, 8, MG_TLS_HANDSHAKE);
  }
  int send_ca = !is_client && tls->ca_der.len > 0;
  // ... rest unchanged

Environment

• mongoose version: 7.14 (also verified on latest master — same bug)
• Compiler/IDE and SDK: STM32CubeIDE 1.x, ARM GCC
• Target hardware/board: STM32H743ZI (Nucleo-H743ZI)
• Connectivity chip/module: Built-in Ethernet (LAN8742 PHY)
• Target RTOS/OS (if applicable): FreeRTOS

[scaprile]

There is no reason for Mongoose to be sending a certificate unless you configured your server in two-way (mutual) TLS authentication.

mongoose/src/tls_builtin.c

Lines 1839 to 1849 in 0ec1112

	if (tls->cert_requested && tls->cert_der.len > 0) { // two-way auth
	// generate application keys at this point, keep using handshake keys
	struct tls_enc hs_keys = tls->enc;
	mg_tls_generate_application_keys(c);
	tls->app_keys = tls->enc;
	tls->enc = hs_keys;
	if (!mg_tls_send_cert(c, true) || !mg_tls_send_cert_verify(c, true) ||
	!mg_tls_client_send_finish(c))
	return false;
	tls->enc = tls->app_keys;
	} else {

You are using Two-way (mutual) authentication but that is not mandatory, I don't think your setup is correct, and if is, please explain why and point to authoritative documentation. We currently do not support something like that, Mongoose currently works either with one-way or two-way auth, not what I think is your setup; you either don't configure a cert and are not requested for one, or if you will be requested a cert, then you configure one.
https://mongoose.ws/documentation/tutorials/tls/
Please, instead of explaining us what we should fix and how, I suggest you tell us how you configure Mongoose and provide a TLS dialog capture, so we can see what the actual problem is, and, in case it were needed, can also fix our CI/CD tests that pass; or add new.
your struct mg_tls_opts and where do you call mg_tls_init
Thank you.

[saidalbardawil]

Thank you for the quick response.

My configuration:

struct mg_tls_opts tls_opts;
memset(&tls_opts, 0, sizeof(tls_opts));
tls_opts.ca = mg_str(TLS_CA);           // server CA certificate (PEM)
tls_opts.name = mg_url_host(url);
tls_opts.skip_verification = 1;
// NO cert or key set — one-way TLS, not mutual
mg_tls_init(c, &tls_opts);

This is called inside the MQTT connect handler after mg_mqtt_connect(), when c->is_tls is true. No client certificate or key is configured. This is standard one-way TLS.

The scenario:

The server sends a CertificateRequest during the TLS 1.3 handshake, but does not require a client certificate (optional client auth). This is a common and valid configuration — many brokers and web servers do this (e.g., EMQX, Mosquitto with require_certificate false, nginx with ssl_verify_client optional).

Per RFC 8446 Section 4.4.2 (https://www.rfc-editor.org/rfc/rfc8446#section-4.4.2):

"If the corresponding certificate type extension was not negotiated in EncryptedExtensions, or the server did not send a CertificateRequest message, or no certificates are available, the client MUST send a Certificate message containing no certificates (i.e., with the "certificate_list" field having length 0)."

The keyword is MUST. In TLS 1.3, when the server sends CertificateRequest, the client must always respond with a Certificate message — even if it's empty. This is different from TLS 1.2, where the client could skip it entirely.

Looking at the code you referenced:

if (tls->cert_requested && tls->cert_der.len > 0) {  // two-way auth

When cert_requested == true but cert_der.len == 0 (no client cert configured), this block is skipped entirely and no Certificate message is sent. That works with lenient servers, but strict TLS 1.3 implementations will close the connection because they expect the Certificate message in the handshake transcript.

[scaprile]

You are configuring one-way in Mongoose, but your server is requesting two-way.
As said, we currently do not support that.
We never claimed to be RFC-compliant; we are, as long as it doesn't interfere with our goals.
The code I posted has a comment at the first line: "two-way". We're not handling "one or two, you decide" situations; as said, it is either one-way or two-way, not maybe both. If you're requested a certificate, you have to provide one.
Thanks for explaining the same again, I'd preferred a pointer to authoritative docs where that configuration is valid; independently of what brokers do.
None of our clients has requested that, so far, so they don't seem to have problems configuring their servers for either one-way or two-way; and they don't use "I'd rather like two-way, but if you want to go one-way that's fine" modes. It doesn't make sense to me, IMHO, you either enforce or don't.

Said that, it doesn't seem too time consuming to add support for that. I'll mark this as a feature request and it will be handled in due time. If you're a paying customer, please contact us at Support and we'll speed up things for you.
Thank you.

[saidalbardawil]

Thanks a lot for the quick and detailed response —and for accepting this as a feature request!

You asked about authoritative documentation for this configuration. It's part of the TLS 1.3 spec itself:

• RFC 8446, Section 4.3.2 (https://www.rfc-editor.org/rfc/rfc8446#section-4.3.2) describes CertificateRequest as an optional message the server can send.
• RFC 8446, Section 4.4.2 (https://www.rfc-editor.org/rfc/rfc8446#section-4.4.2) says the client MUST reply with a Certificate message even when it has no certificate to offer.

So the "ask but don't enforce" model is baked into TLS 1.3 by design it's not something specific to any broker.

I totally understand it hasn't come up before from your users. I have a local workaround that works well for now, so there's no urgency at all on my side. Whenever you get to it is fine.

Thanks again for maintaining such a great library — keep up the awesome work!

[scaprile]

Added support for no certificates, hence sending empty lists.
Clients should now always respond to a cert request, and just send an empty list. However, a certificate being requested still implies two-way auth, so Mongoose will derive keys at that point as it will assume it is in a two-way auth scenario.
Let me know if this branch contents work for you: https://github.com/cesanta/mongoose/tree/refs/heads/nocert

[saidalbardawil]

Thanks for the quick update.

I tested it on my side, and unfortunately it still does not work for my case.
As you mentioned, this branch still treats CertificateRequest as a two-way-auth path internally. In my optional client-certificate setup, that seems to be why the connection still closes.

So for my environment, this branch does not fix the issue yet.

[scaprile]

Can you please attach a pertinent log and network sniff capture ? Mongoose is supposed to respond with an empty certificate list to the CertificateRequest. If it does not, I need to see what it does. Perhaps that is fine but the key generation must not be done there when there are no certificates ? I need info.
We support the standard OpenSSL way of logging keys, if you're running on an OS: https://mongoose.ws/documentation/tutorials/tls/#debugging-tls-traffic