Skip to content

Use exact-size header arrays for output cache entries - #70195

Open
davidfowl wants to merge 1 commit into
mainfrom
perf/mw-outputcache-header-arrays
Open

davidfowl wants to merge 1 commit into
mainfrom
perf/mw-outputcache-header-arrays

Conversation

@davidfowl

@davidfowl davidfowl commented Oct 31, 2026 •

Copy link
Copy Markdown
Member

Use exact-size header arrays for output cache entries

  • You've read the Contributor Guide and Code of Conduct.
  • You've included unit or integration tests for your change, where applicable.
  • You've included inline docs for your change, where applicable.
  • There's an open issue for the PR that you are making.

Allocate output cache header arrays at their exact size instead of renting them.

Description

OutputCacheEntryFormatter.Deserialize (every cache hit) and OutputCacheEntry.CopyHeadersFrom (every stored response) rent their header array from ArrayPool<(string, StringValues)>.Shared, but the arrays are never returned: OutputCacheEntry.Dispose is a no-op, and the entry keeps the array. The shared pool therefore never has an array to hand back, so each call allocates a new pool-sized array with at least 16 elements (16 B each) plus the pool lookup overhead.

This change allocates an array of exactly the needed length. The stored format, cache keys and the headers written to the response are unchanged. Because nothing is rented, no array can be shared between entries.

Benchmark: the existing EndToEndBenchmarks.ReadAsync, which deserializes a cached entry with 6 headers. Three interleaved A/B rounds:

PayloadLength Before (main) After
10 154-170 ns / 488 B 136-143 ns / 328 B
1000 150-154 ns / 488 B 137-139 ns / 328 B
65553 151-162 ns / 488 B 138-139 ns / 328 B
262161 149-158 ns / 488 B 137-142 ns / 328 B

The store path (CopyHeadersFrom) follows the same pattern: before, it rented a 16-element array for the 7 response headers; now it allocates a 7-element array.

Tests: Microsoft.AspNetCore.OutputCaching.Tests, 403/403. The existing formatter and middleware tests cover header round-tripping, including ignored headers.

🤖 Posted by ASP.NET Core Perf 2 (AI Agent)

The header arrays were rented from ArrayPool but never returned (the
entry's Dispose is a no-op), so every cache hit and every stored response
allocated a fresh pool-sized array (minimum 16 elements).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings October 31, 2026 16:24
@github-actions github-actions Bot added the area-middleware Includes: URL rewrite, redirect, response cache/compression, session, and other general middlewares label Oct 31, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

There are no active review issues for the header-array allocation changes.

No open issues 🎉

What changed in this PR

This PR reduces output-cache header allocation overhead by replacing pooled arrays that were not returned with directly allocated arrays, while preserving serialization and response-header behavior.

Changes:

  • Allocate header arrays using the requested count during deserialization and response storage.
  • Remove pool-return handling and simplify header-memory assignment.
File Description
src/​Middleware/​OutputCaching/​src/​OutputCacheEntryFormatter.cs Allocates an exact-length array for deserialized headers and passes it directly to SetHeaders.
src/​Middleware/​OutputCaching/​src/​OutputCacheEntry.cs Replaces array rental with direct allocation while retaining ignored-header filtering and slicing.

Review effort: Balanced

@PureWeen

PureWeen commented Nov 1, 2026

Copy link
Copy Markdown
Member

/review

@davidfowl davidfowl added the agent:working An AI agent is actively working on this issue/PR label Nov 6, 2026 — with Copilot SWE Agent
@davidfowl davidfowl removed the agent:working An AI agent is actively working on this issue/PR label Nov 6, 2026
@dotnet-policy-service

Copy link
Copy Markdown
Contributor

Looks like this PR hasn't been active for some time and the codebase could have been changed in the meantime.
To make sure no conflicting changes have occurred, please rerun validation before merging. You can do this by leaving an /azp run comment here (requires commit rights), or by simply closing and reopening.

@dotnet-policy-service dotnet-policy-service Bot added the pending-ci-rerun When assigned to a PR indicates that the CI checks should be rerun label Nov 13, 2026

Copy link
Copy Markdown
Member Author

Merge assessment

Verdict: Lean merge. It is a 16-line change that drops an ArrayPool rent whose arrays were never returned, with a repeatable ~10% and −160 B gain on every output-cache hit.

Size S: +4/−12 production lines, +0 test, +0 benchmark; 2 files, 1 assembly (OutputCaching)
Impact Medium: EndToEndBenchmarks.ReadAsync ~150–170 → 136–143 ns and 488 → 328 B per cache hit, 3/3 rounds, all payload sizes
Risk Low: OutputCaching middleware (Medium), lowered one tier because ArrayPool.Rent is replaced by new T[n] and every consumer already reads through a length-bounded ReadOnlyMemory
Tests No new tests; relies on existing OutputCacheEntryFormatterTests and middleware header round-trip tests (403/403 per body); not a bug fix, so nothing to be red on main
Review No review from a user with write access; Copilot reviewer found no issues; PureWeen (write) posted /review on Nov 1 with no follow-up
CI Green on a7024c2 (aspnetcore-ci, Helix, platform tests); mergeable, no conflicts, 470 commits behind main
Complexity Lower than main: removes the rent/return branch in OutputCacheEntry.CopyHeadersFrom
Reversibility Trivial single-commit revert
Security sensitivity Not assessed (repo review guidance)
Encoding/escaping parity n/a (header storage only; serialized format unchanged)
Behavior / public API change None

Reasons to merge

  • OutputCacheEntry.Dispose is a no-op, so rented arrays never went back to the pool and each call paid pool overhead plus a ≥16-element array.
  • Timing ranges for main and PR do not overlap in any of the 4 payload sizes.
  • Serialized format, cache keys and response headers are unchanged.

Reasons not to merge yet

  • The store path (CopyHeadersFrom) is not benchmarked; the body only describes its array size.
  • CopyHeadersFrom still sizes the array to headers.Count, including headers it then skips, so "exact size" holds only for the read path.
  • CI ran on Oct 31 against a base now 470 commits old.

What would change this verdict: A plan to make OutputCacheEntry.Dispose return pooled arrays (as its comment intends) would make the pool worth keeping and this change a Hold.

🤖 Posted by ASP.NET Core Perf 2 (AI Agent)

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area-middleware Includes: URL rewrite, redirect, response cache/compression, session, and other general middlewares pending-ci-rerun When assigned to a PR indicates that the CI checks should be rerun

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants