Skip to content

Reduce allocations in QueryHelpers.AddQueryString - #70200

Open
davidfowl wants to merge 7 commits into
mainfrom
perf/http-add-query-string
Open

davidfowl wants to merge 7 commits into
mainfrom
perf/http-add-query-string

Conversation

@davidfowl

@davidfowl davidfowl commented Oct 31, 2026 •

Copy link
Copy Markdown
Member

Reduce allocations in QueryHelpers.AddQueryString

  • You've read the Contributor Guide and Code of Conduct.
  • You've included unit or integration tests for your change, where applicable.
  • You've included inline docs for your change, where applicable.
  • There's an open issue for the PR that you are making.

Fewer allocations and faster QueryHelpers.AddQueryString.

Description

QueryHelpers.AddQueryString builds its result with a default-capacity StringBuilder, which grows chunk by chunk. The single name/value overload also wraps the pair in an array and goes through IEnumerable enumeration. The single-pair overload runs on hot paths such as cookie authentication login/access-denied redirects (ReturnUrl), and the dictionary overload runs when building OAuth/OIDC challenge URLs.

Changes (all internal to QueryHelpers, no public API change):

  • Both overloads build into the shared ValueStringBuilder, starting on stackalloc char[256] and moving to ArrayPool<char> for longer output. Names and values are encoded in place with the span-based UrlEncoder.Encode(ReadOnlySpan<char>, Span<char>, ...), so the result string is the only allocation. Growth is done locally, so pooled buffers are cleared before they're returned.
  • AddQueryString(string uri, string name, string value) no longer wraps the pair in an array and enumerates it. The separator is still decided exactly as before (is there a ? before the first #).

The output is identical: same separators, same UrlEncoder.Default encoding, null values skipped, anchor preserved.

Benchmark

QueryHelpersAddQueryStringBenchmark (added): BenchmarkDotNet InProcess (--config perflab, 3 warmup / 15 iterations), 3 interleaved rounds, mean of the rounds. "Previous" is the earlier string.Create/Encode(string) revision of this PR (cfd749c).

Case main previous PR main alloc previous alloc PR alloc
ReturnUrl (single pair, value needs encoding) 153.3 ns 86.1 ns 84.2 ns (−45%) 600 B 216 B 136 B (−77%)
Simple (single pair, no encoding) 65.0 ns 26.4 ns 45.6 ns (−30%) 304 B 56 B 56 B (−82%)
LongValue (602-char value, no encoding) 242.2 ns 105.5 ns 200.3 ns (−17%) 2,768 B 1,264 B 1,264 B (−54%)
EncodeHeavy (short non-ASCII/reserved value) 202.6 ns 154.1 ns 149.3 ns (−26%) 760 B 392 B 208 B (−73%)
OAuth (7-entry dictionary) 484.1 ns 363.8 ns 420.2 ns (−13%) 2,072 B 952 B 768 B (−63%)

Percentages are relative to main. Compared with the previous revision, the span-based encoder saves allocations whenever something gets encoded, but it's slower when nothing does, because Encode(string) returns the input instance without copying.

Tests

Added AddQueryStringMatchesReferenceImplementation, which compares both the single-pair and the enumerable overloads against the previous StringBuilder algorithm. Cases cover empty URIs, a trailing ?, # inside the anchor and multiple #, a ? inside the fragment, non-ASCII and surrogate pairs, characters that need encoding, null values, 1,400-char keys and values (longer once encoded) that force the pooled buffer to grow, lone surrogates, a 300-char value that expands 9× when encoded (several regrowths), surrogate pairs straddling the 256-char stack buffer, and a URI longer than the stack buffer. A mutation that skips regrowth after DestinationTooSmall fails all 12 cases. All 518 WebUtilities tests pass.

🤖 Posted by ASP.NET Core Perf 2 (AI Agent)

Build the single name/value overload with string.Create and use an
ArrayPool-backed buffer instead of StringBuilder for the enumerable
overloads. Output is unchanged.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings October 31, 2026 17:16
@github-actions github-actions Bot added the area-networking Includes servers, yarp, json patch, bedrock, websockets, http client factory, and http abstractions label Oct 31, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues were identified for the allocation-reduction changes to QueryHelpers.AddQueryString.

No open issues 🎉

What changed in this PR

Reduces allocations in QueryHelpers.AddQueryString, which is used to construct URLs for authentication redirects and other HTTP scenarios, without changing its public API.

Changes:

  • Uses string.Create for single-pair calls and a pooled character buffer for enumerable calls.
  • Adds reference-comparison tests covering encoding, fragments, null values, and buffer growth.
  • Adds allocation and timing benchmarks for simple queries and authentication URLs.
File Description
src/​Http/​WebUtilities/​test/​QueryHelpersTests.cs Adds compatibility tests against the previous algorithm.
src/​Http/​WebUtilities/​src/​QueryHelpers.cs Replaces StringBuilder with direct string creation and pooled buffering.
src/​Http/​WebUtilities/​perf/​Microbenchmarks/​QueryHelpersAddQueryStringBenchmark.cs Adds single-pair and OAuth dictionary benchmarks.

Review effort: Balanced

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@PureWeen

PureWeen commented Nov 1, 2026

Copy link
Copy Markdown
Member

/review

@davidfowl davidfowl added the agent:working An AI agent is actively working on this issue/PR label Nov 4, 2026 — with Copilot SWE Agent
Query values may contain sensitive data, so clear the used portion of the
rented buffer before returning it, both on Dispose and when growing.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@davidfowl davidfowl removed the agent:working An AI agent is actively working on this issue/PR label Nov 4, 2026
@davidfowl davidfowl added the agent:working An AI agent is actively working on this issue/PR label Nov 4, 2026 — with Copilot SWE Agent
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@davidfowl
davidfowl requested review from a team, SamMonoRT and wtgodbe as code owners November 4, 2026 02:31
Restore the full tree from c17c895 after the previous empty commit was
created with an empty index. No content change relative to c17c895.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copy link
Copy Markdown
Member Author

Retriggering CI once for an unrelated test failure on c17c895.

Failing test: Microsoft.AspNetCore.Mvc.FunctionalTests.TempDataPropertyTest.TempDataPropertyAttribute_TempDataKept (Ubuntu.2404.Amd64.Open only), at TempDataPropertyTest.cs:113, Assert.Equal() Failure: Strings differ with actual "Success (from Temp Data)100". Links: build 1662849, Helix console. Build Analysis lists a 0.03% historical failure rate for this test.

Why it's unrelated: this PR only changes QueryHelpers.AddQueryString in src/Http/WebUtilities (plus its tests and a benchmark). The test POSTs to TempDataProperty/CreateNoRedirect, then GETs TempDataKept and ReadTempData. Those actions return OkResult or plain strings and use cookie TempData. There are no redirects, views or query-string building on that path. The only Mvc caller of AddQueryString is DefaultFileVersionProvider (asp-append-version), which this test doesn't use.

Commits: my first Retrigger CI commit (a3b814c) was created from an empty index by mistake and removed all files. The next commit (496a1b0) restores the exact tree of c17c895. The PR diff is unchanged: the same 3 files, +182/−15, with no content change from c17c895.

🤖 Posted by ASP.NET Core Perf 2 (AI Agent)

@davidfowl davidfowl removed the agent:working An AI agent is actively working on this issue/PR label Nov 4, 2026
@davidfowl davidfowl added the agent:working An AI agent is actively working on this issue/PR label Nov 6, 2026 — with Copilot SWE Agent
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@davidfowl davidfowl removed the agent:working An AI agent is actively working on this issue/PR label Nov 6, 2026
@davidfowl davidfowl added the agent:working An AI agent is actively working on this issue/PR label Nov 6, 2026 — with Copilot SWE Agent

Copy link
Copy Markdown
Member Author

The aspnetcore-ci failure on cfd749c (build 1667826) is a known Kestrel timing flake and isn't caused by this PR.

Failing test: Microsoft.AspNetCore.Server.Kestrel.Core.Tests.Http2TimeoutTests.HEADERS_ReceivedWithoutAllCONTINUATIONs_WithinRequestHeadersTimeout_AbortsConnection on Ubuntu.2404.Amd64.Open (Build Analysis: 0.05% failure rate)
Moq.MockException: Expected invocation on the mock once, but was 0 times: h => h.OnTimeout(TimeoutReason.RequestHeaders) (Http2TimeoutTests.cs:190)

Why it's unrelated

I didn't retrigger CI because main E2E is currently flaky. A maintainer can rerun the failed job if needed.

🤖 Posted by ASP.NET Core Perf 2 (AI Agent)

@davidfowl davidfowl removed the agent:working An AI agent is actively working on this issue/PR label Nov 6, 2026
@dotnet-policy-service

Copy link
Copy Markdown
Contributor

Looks like this PR hasn't been active for some time and the codebase could have been changed in the meantime.
To make sure no conflicting changes have occurred, please rerun validation before merging. You can do this by leaving an /azp run comment here (requires commit rights), or by simply closing and reopening.

@dotnet-policy-service dotnet-policy-service Bot added the pending-ci-rerun When assigned to a PR indicates that the CI checks should be rerun label Nov 13, 2026
@davidfowl davidfowl added the agent:working An AI agent is actively working on this issue/PR label Nov 16, 2026 — with Copilot SWE Agent

davidfowl commented Nov 16, 2026 •

Copy link
Copy Markdown
Member Author

ASP.NET Core Perf 2 has stopped: @DeagleGross's review is addressed in 70459f6, with the A/B results in the threads and the PR description.

🤖 Posted by ASP.NET Core Perf 2 (AI Agent)

Build the result in a ValueStringBuilder that starts on a 256-char
stackalloc buffer and falls back to ArrayPool, and encode names and
values in place with UrlEncoder.Encode(ReadOnlySpan<char>, Span<char>, ...)
so the result string is the only allocation. Growth is handled locally so
pooled buffers are cleared before they are returned.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@davidfowl davidfowl removed the agent:working An AI agent is actively working on this issue/PR label Nov 16, 2026

@DeagleGross DeagleGross left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ValueStringBuilder doesn't clear its pooled arrays, which is why this needs the local Append, Grow and ReturnRentedBuffer helpers and the ref char[]? rentedBuffer parameters. RefPooledArrayBufferWriter<char> (src/Shared/Buffers) starts on a stackalloc span, grows through ArrayPool, and clears on grow and Dispose, so I think it removes all of that. The encode loop becomes Encode(source, writer.GetSpan(source.Length), out consumed, out written, isFinalBlock: true), then writer.Advance(written), and on DestinationTooSmall slice the source by consumed and loop.

can u try use that instead and re-run the benchmarks?

Copy link
Copy Markdown
Member Author

Merge assessment

Verdict: Merge after DeagleGross's RefPooledArrayBufferWriter<char> request is answered and aspnetcore-ci is green. The allocation cut is real, but there is an open request from a write-access reviewer and CI on the head is red.

Size M: +95/−14 production lines, +65 test, +38 benchmark; 4 files, 1 production assembly (Microsoft.AspNetCore.WebUtilities) plus test and perf projects; links the existing src/Shared/ValueStringBuilder without editing it
Impact Medium: ReturnUrl 153 → 84 ns (−45%), 600 → 136 B (−77%); OAuth dictionary −13% time, −63% bytes; used on login/challenge redirects, not per request
Risk Medium: WebUtilities URL builder; pooled buffers are returned and cleared within the call
Tests AddQueryStringMatchesReferenceImplementation (12 cases) compares both overloads with the old StringBuilder algorithm; parity tests pass on main by design, body reports a skip-regrowth mutation fails all 12
Review No approvals; gfoidl (read) and BrennanConroy (write) threads resolved; DeagleGross's (write) two threads were addressed in 70459f6, but his later review asking for RefPooledArrayBufferWriter<char> and re-run benchmarks has no reply
CI Red on 70459f6: aspnetcore-ci reports 1 pipeline error with 0 of 86,889 tests failed; Platform tests Test: macOS never reported; root cause unknown (AzDO logs not checked)
Complexity Adds local Append/Grow/ReturnRentedBuffer helpers because ValueStringBuilder doesn't clear pooled arrays; the requested writer would remove most of that
Reversibility Easy: one file of logic
Security sensitivity Not assessed here (out of scope under the repo's review guidance)
Encoding/escaping parity Same UrlEncoder.Default, string overload replaced by the span overload; no custom fast-path predicate; reference tests cover lone surrogates, surrogate pairs across the 256-char stack boundary and 9× expansion
Behavior / public API change None

Reasons to merge

  • Result string is the only allocation when encoding is needed, cutting ReturnUrl bytes from 600 to 136.
  • Output is pinned against the old algorithm across the edge cases that matter for UrlEncoder chunking.
  • Small, single-file change in one assembly.

Reasons not to merge yet

  • DeagleGross's latest review is unanswered and would reshape the code.
  • No-encode paths are slower than the PR's previous revision (LongValue 105 → 200 ns), though still faster than main.
  • CI on the head is red and the failing leg is not identified.

What would change this verdict: a reply to DeagleGross with re-run numbers and a green CI run would make this Lean merge.

🤖 Posted by ASP.NET Core Perf 2 (AI Agent)

@PureWeen

Copy link
Copy Markdown
Member

/review

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area-networking Includes servers, yarp, json patch, bedrock, websockets, http client factory, and http abstractions pending-ci-rerun When assigned to a PR indicates that the CI checks should be rerun

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants