HTTPS Listener limitations - why are the TLS settings not used to distinguish listeners or allow a list of hostnames?

[funkluk]

Hi all
I’m not sure if this is the right place, but I hope to get some insight into how the intended use of the Gateway API is.

My setup is that I have several hosts using HTTP and HTTPS, some that serve several hostnames (alias).

Now, when I apply/translate this into the Gateway API, I’ve trouble wrapping my head around resp. I’m running very quickly into the limits.

As the listener only supports one hostname, I must create a separate listener for each hostname and alias, repeating the TLS certificate settings (i.e., using a single certificate with all the SANs).

My first approach was to define the hostnames in the HTTPRoute and not in the HTTPS Listener. This works great in the HTTP Listener setup, but not for HTTPS, since I can’t have more than one HTTPS Listener. HTTPS listeners with the same port but different TLS settings are not considered unique.

from the GatewaySpec Listener field documentation:

Note that the tls field is not used for determining if a listener is distinct, because

The only option in this scenario would be a wildcard certificate. But if I have completely different domain names, wildcard certificates are not possible.

So, I’m back to defining a separate listener for each hostname and alias, but then I’m running really quickly into the listener limit of 64. And in addition, I must reference all these listeners in every HTTPRoute I create.

I’ve seen that there are issues around increasing this limit and the introduction of ListenerSets, but still, for me, the whole configuration is a bit strange.

Therefore, I try to understand the intention of the API definition as it is now a bit better.

First, did I miss an option on how to configure my setup correctly?

One option to come by my issues would be to include the tls field in the uniqueness validation for the HTTPS Listeners. Or can somebody explain the design decisions why the HTTPS Listener does not consider the TLS settings to distinguish the listeners?

Another option would be to use a list of hostnames in the listener. Any reasoning on why the listener hostname is a single value and not a list as in the HTTP route?

I'd really love to get more insight into the API design to understand these points better.

Thanks a lot!

[howardjohn]

to make sure I understand this a bit, what does your certificate look like (in your ideal state)? do you have 1 cert with N SANs?

[funkluk]

I‘d say one certificate with N SANs for related hostnames.
For example we have a website with a .com and .ch domainname serving the same site. So both domain names are in the certificate.
For other sites or applications, they should have their own certificates.
So in the end there would be multiple certificates with each having one to N SANs.

I hope this clarifies it better.

[youngnick]

We definitely did not consider the "same site, different top-level domain" case when designing Listeners. Sorry.

Are the SANs just because of the general perception that wildcard certificates should not be used? When we initially designed Gateway API, cert-manager, ACME, and Let's Encrypt were not in wide use, so the design kind of expects the use of wildcard certificates for larger deployments. To that end, we have controls intended to make it so that Gateways can use certificates that the Gateway owner cannot read themselves (although, of course, the Gateway implementation must be able to).

We're also working with cert-manager on figuring out how to make this work; I'm working on a document at the moment that will lay out some expectations for how this could work.

I did also want to correct one thing, though. You said:

My first approach was to define the hostnames in the HTTPRoute and not in the HTTPS Listener. This works great in the HTTP Listener setup, but not for HTTPS, since I can’t have more than one HTTPS Listener. HTTPS listeners with the same port but different TLS settings are not considered unique.

That's technically correct, but I think you've been misled by the phrase "TLS Settings". That means that the tls stanza is not a determinant on if a Listener is distinct. But, importantly, hostname is. So, having different hostnames, even if they use the same certificate is (and this is actually one of the reasons for that requirement, so that we don't require implementations to unwrap and parse certificates in order to determine Listener validitiy.)

However, today, for this sort of setup, the best you can do is:

• one Listener per TLD of the company site. So *.company.com and *.company.ch would need separate Listeners. These could be on a Gateway, or on a ListenerSet. Each of these can use the same certificate, stored in the same Secret. These Listeners are distinct because they have different hostname fields.
• Each HTTPRoute includes both service.company.com and service.company.ch hostnames in its hostnames field. This will allow a single HTTPRoute to attach to both Listeners.
• If you are managing your own certificates, without cert-manager, or without using cert-manager's existing Gateway API integration, then you are done. In this case, the hostname is only serving to filter what HTTPRoutes can attach to that Listener.
• If you are using cert-manager's existing Gateway API integration, you may be out of luck; I haven't checked how it works in a while, so I'm not sure.

Personally, my ideal setup here would be to recommend the following:

• a single wildcard certificate, managed by cert-manager, using a Certificate object in a special certs namespace, that only infrastructure admins (or a small set of people) have access to. This creates a Secret in that namespace - lets call it wildcard for ease of example. This would cover the base company name in all the TLDs you care about - so, in this example *.company.com and *.company.ch.
• A ReferenceGrant in the certs namespace that allows access to Secrets from Gateways in other namespaces. This can also be limited to specific namespaces, if you wish.
• A Gateway with a Listener for each of *.company.com and *.company.ch.
• HTTPRoutes with routes for each of the services (which I am assuming are servicename.company.(com|ch). When you specify the parentRef for this HTTPRoute, you can specify just the Gateway, and the hostname intersection calculation will ensure that only HTTPRoutes that have hostnames that intersect will attach to those Listeners. So you don't need to have separate parentRefs in the HTTPRoutes for each Listener. The implementation will ensure that only traffic that matches one of the hostnames on the HTTPRoute arrives at the backend - based on both the SNI for the TLS connection, and the Host header or :authority on the HTTP request.

This will then allow you to attach an arbitrary number of HTTPRoutes, with an arbitrary number of subdomain services, to the Listeners, without needing to re-mint the certificate, or edit the Gateway at all.