Add support for Fowarded header (RFC 7239)

[Kludex]

Discussed in #2236

^{Originally posted by nhairs January 28, 2024}
We should probably support the Forwarded header which was standardised in RFC 7239.

One of the benefits of this header over X-Forwarded-* headers is that it supports including the port of the client rather than just the address.

See also: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Forwarded

Important

• We're using Polar.sh so you can upvote and help fund this issue.
• We receive the funding once the issue is completed & confirmed by you.
• Thank you in advance for helping prioritize & fund our backlog.

[Kludex]

Reference for myself: https://systemoverlord.com/2020/03/25/security-101-x-forwarded-for-vs-forwarded-vs-proxy.html

[Kludex]

@nhairs Do you have any proposal on how we should implement this?

Can we have the X-Forwarded-* and Forwarded at the same time?

[nhairs]

Can we have the X-Forwarded-* and Forwarded at the same time?

Per the code comments I left in my open PR, I initially thought this was the case. i.e. use the official headers if available otherwise fallback to the x-forwarded headers. But I suspect that such behaviour might introduce vulnerabilities into user's applications. It might be better to take a PEP20 "Explicit is better than implicit" approach which leads me to...

@nhairs Do you have any proposal on how we should implement this?

My gut feeling is that we're better off making users explicitly choose which headers they want to extract info from. Trying to support all of them from the commandline seems like a lot of work though. What about supporting X-Real-IP headers?

Which leads to my suggestion on #2231:

[Kludex]

Makes sense to be explicit on what is supported.

[Kludex]

Maybe we can check how others are doing it? Are we going to pioneer this "choices"?

[pkucmus]

Please tell me, if there's progress on this? How can this be picked up? This is a wide spread problem as AWS's API Gateway v2 converts all x-forwarded headers into the new forwarded (ref). It's most painful when using url_for in FastAPI - once behind that proxy one will always get a http url as a result (even if the app is behind https).

[nhairs]

I've been thinking about this issue on and off over the last 12 months.

Maybe we can check how others are doing it? Are we going to pioneer this "choices"?

So in theory most ASGI apps much be encountering this issue so surely there is some fairly standard ASGI wrapper/middleware that could be lifted used across many projects. Personally checking for existing solutions would be where I would start.

However there is also the question of how it is integrated into Uvicorn. When writing the old PR there ended up being 3 commons ways that forwarded headers appear, forwarded, X-Forwarded-for, and X-RealIP (with name variations). Handling each of these cases using command-line switches feels like the wrong place to be doing this (at least to me).

It almost feels like if we have standard ASGI wrappers / middleware available we should instead push the inclusion / configuration of these down into the application layer as although there are "common" ways of resolving this, there are many ways of doing it.

[Kludex]

There isn't such a thing as standard ASGI wrapper/middleware. You mean a third party package or more like discussing in the spec itself?

A third party package works, but the idea is to make it easier for users, and discoverable if it's a configuration.

But then your proposal is to not do anything else here?