Auth header is url-encoded

[jshufro]

Describe the bug

Lodestar seems to be URL-encoding the username:password pair for basic auth before base64 encoding and setting the Authorization header.

(I will see about PRing a fix)

Expected behavior

Auth headers aren't part of the URL and need not be encoded

Steps to reproduce

Set a username with an = in it
Observe that the validator queries with a Authorization header with a value that decodes to %3D

Additional context

This has broken the rescue node's work flow again ;)

Operating system

Linux

Lodestar version or commit hash

v1.11.3

[nflaig]

Auth headers aren't part of the URL and need not be encoded

The problem is we are not setting these explicitly but as part of the URL. I wonder how other clients handle this but if you use a basic auth credentials (username:password) via browser it would certainly be URL encoded.

But I would agree we don't need to do that and align behavior with other clients. Also noticed it not URL encoding every character. e.g. = is encoded while & is not and # just throws an error..

> new URL("http://user:passw=rd@localhost")
URL {
  password: 'passw%3Drd',
}

> new URL("http://user:passw&rd@localhost")
URL {
  password: 'passw&rd',
}

> new URL("http://user:passw#rd@localhost")
Uncaught TypeError [ERR_INVALID_URL]: Invalid URL

I remember in the past that I had configured my password manager to only use URL safe characters for generation because various services had similar behavor (including browser)... this is bad should definitely not have any character restriction in passwords.

[nflaig]

@jshufro Let me know if you are working on this, else I can take a look at it as well.

Just a few notes from my side

• if we want to support any character e.g. # or ? then have to extract basic auth credentials before validating URL else this will throw an error
  

  lodestar/packages/api/src/utils/client/httpClient.ts

  Lines 132 to 134 in 9dae25f

  	if (!isValidHttpUrl(urlOpts.baseUrl)) {
  	throw Error(`HttpClient.urls[${i}] must be a valid URL: ${urlOpts.baseUrl}`);
  	}

• test case needs to be updated to cover all cases as outlined in my previous comment
  

  lodestar/packages/api/test/unit/client/httpClient.test.ts

  Line 138 in 5201ac4

  it("should set user credentials in URL as Authorization header", async () => {

• could build the auth header in constructor of HttpClient class, this would also avoid recreating the header on every request

[jshufro]

I'll take a stab. I agree pulling it out before passing into the client is probably best:

1. It lets us fix the immediate issue
2. You probably shouldn't log the fully qualified uri- the username/password as somewhat sensitive.

[jshufro]

Also- since we use a URL-safe base64 library, it should be a nop for us to decode the Auth header, urldecode the username:password pair (which are base64 for us), and then re-encode the header in a header rewrite, so i'm going to attempt that before i start this

[jshufro]

haproxy made it too complicated so i adjusted the upstream service. I'll have a PR up for lodestar in a bit :)