Skip to content

Reduce per-request allocations in StaticFileMiddleware - #70173

Open
davidfowl wants to merge 6 commits into
mainfrom
perf/staticfiles-request-path
Open

davidfowl wants to merge 6 commits into
mainfrom
perf/staticfiles-request-path

Conversation

@davidfowl

@davidfowl davidfowl commented Oct 31, 2026 •

Copy link
Copy Markdown
Member

Reduce per-request allocations in StaticFileMiddleware

  • You've read the Contributor Guide and Code of Conduct.
  • You've included unit or integration tests for your change, where applicable.
  • You've included inline docs for your change, where applicable.
  • There's an open issue for the PR that you are making.

Remove avoidable allocations from the UseStaticFiles request path.

Description

Every request that reaches StaticFileMiddleware calls FileExtensionContentTypeProvider.TryGetContentType. That call allocated an extension substring for the lookup. Each served or revalidated file also paid for a few small allocations in StaticFileContext. This PR removes them. All changes are internal.

  • FileExtensionContentTypeProvider
    • When Mappings is exactly a Dictionary<string, string> whose comparer supports span lookups, the constructor caches an AlternateLookup<ReadOnlySpan<char>>. This covers the default instance (OrdinalIgnoreCase) and user dictionaries with Ordinal or OrdinalIgnoreCase.
    • The extension is then looked up as a span, so no substring is allocated. Every other mapping type, including derived dictionaries, still uses IDictionary.TryGetValue(string).
    • The alternate lookup reads the same dictionary instance, so later changes to Mappings are still seen. The extension rules (IsNullOrWhiteSpace, then last . to the end) are unchanged.
  • StaticFileContext.LookupFileInfo
    • The ETag string is built with a single string.Create(InvariantCulture, $"\"{hash:x}\""), replacing '"' + Convert.ToString(hash, 16) + '"' (two strings).
    • The output is the same lowercase two's-complement hex. I checked this against Convert.ToString for 1M random values plus edge cases, and a new test covers negative hashes.
  • StaticFileContext.ApplyResponseHeadersAsync
    • Writes HeaderUtilities.FormatDate(lastModified) and etag.ToString() straight into Response.Headers, so it no longer allocates a typed ResponseHeaders wrapper.
    • These are exactly the values the typed setters wrote. The typed wrapper is still created lazily for Content-Range.
  • GetMaxPreconditionState: no longer takes a params PreconditionState[], which allocated an array per request.

Headers, status codes and log output are unchanged.

Benchmark

I used a local BenchmarkDotNet harness (not included). It runs StaticFileMiddleware with a PhysicalFileProvider on a DefaultHttpContext. Timings are on 2 cores, 3 runs each.

FileExtensionContentTypeProvider.TryGetContentType (default provider)

Path Before After
/css/site.css 25.6–25.9 ns, 32 B 18.3–19.0 ns, 0 B (−27%)
/lib/bootstrap/dist/js/bootstrap.bundle.min.js 26.2–26.7 ns, 32 B 16.2–16.3 ns, 0 B (−38%)
/unknown.foo 20.2–21.9 ns, 32 B 13.0–13.2 ns, 0 B (−36%)
/api/values (no extension) 3.69–3.72 ns, 0 B 3.68–3.78 ns, 0 B (unchanged)

StaticFileMiddleware.Invoke: time is dominated by the physical file stat, so these show only the allocation drop.

Request Before After
HEAD (200) 2765–2825 ns, 2344 B 2800–2875 ns, 2200 B (−144 B)
GET If-None-Match (304) 3013–3143 ns, 2752 B 3033–3063 ns, 2608 B (−144 B)
GET If-Modified-Since (304) 2960–2995 ns, 2560 B 2966–3047 ns, 2416 B (−144 B)
GET file not found 1502–1567 ns, 1592 B 1530–1533 ns, 1560 B (−32 B)
GET full body ~38 µs, ~20.6 KB ~39 µs, ~20.5 KB (I/O noise)

That is four fewer allocations per served or revalidated file, and one fewer for every request that has an extension.

Tests

  • Microsoft.AspNetCore.StaticFiles.Tests: 315 passed, 51 skipped.
  • Microsoft.AspNetCore.StaticFiles.FunctionalTests: 20 passed, 5 skipped.
  • Microsoft.AspNetCore.StaticAssets.Tests: 64/64.
  • New tests:
    • ApplyResponseHeaders_WritesExpectedETagAndLastModified checks the exact ETag against the old Convert.ToString(hash, 16) formula, including negative hashes, and checks Last-Modified, Accept-Ranges and Content-Type.
    • Content-type provider tests cover a case-sensitive custom mapping, changes to Mappings after construction, a trailing-dot key, a non-Dictionary mapping (SortedDictionary), and a derived dictionary that re-implements TryGetValue.

Benchmark (3 interleaved rounds)

New StaticFileBenchmark in src/Middleware/StaticFiles/perf/Microbenchmarks, added in this PR. It replaces the local harness described above. StaticFileMiddleware serves /css/site.css (1 KB) from an in-memory IFileProvider, so the numbers measure the changed CPU and allocation path without disk I/O.

  • TryGetContentType_Default (span lookup): 20.1–21.1 ns → 12.0–12.1 ns (about −40%), 32 B → 0 B.
  • StaticFileMiddleware.Invoke (Head, Get, GetIfNoneMatch → 304): −144 B per request in every case. Time is within noise: the PR was 2–6% faster in rounds 1–2 and 2–6% slower in round 3.
  • TryGetContentType_NonDictionaryMappings (a SortedDictionary mapping on the unchanged string fallback path): still 32 B. It is about 1 ns slower (12.7–13.2 ns → 13.6–14.9 ns) because of the added _useSpanLookup check. This affects only non-Dictionary custom mappings.
Round Method main Mean PR Mean main Allocated PR Allocated
1 TryGetContentType_Default 20.1 ns 12.0 ns 32 B 0 B
1 TryGetContentType_NonDictionaryMappings 13.2 ns 13.6 ns 32 B 32 B
1 Head 688.3 ns 665.7 ns 1,912 B 1,768 B
1 Get 914.7 ns 883.7 ns 2,000 B 1,856 B
1 GetIfNoneMatch 1.042 us 984.1 ns 2,288 B 2,144 B
2 TryGetContentType_Default 20.1 ns 12.0 ns 32 B 0 B
2 TryGetContentType_NonDictionaryMappings 12.7 ns 14.9 ns 32 B 32 B
2 Head 676.0 ns 656.0 ns 1,912 B 1,768 B
2 Get 911.5 ns 889.4 ns 2,000 B 1,856 B
2 GetIfNoneMatch 1.003 us 949.9 ns 2,288 B 2,144 B
3 TryGetContentType_Default 21.1 ns 12.1 ns 32 B 0 B
3 TryGetContentType_NonDictionaryMappings 12.7 ns 13.8 ns 32 B 32 B
3 Head 659.6 ns 700.8 ns 1,912 B 1,768 B
3 Get 871.4 ns 897.2 ns 2,000 B 1,856 B
3 GetIfNoneMatch 984.2 ns 1.003 us 2,288 B 2,144 B

Machine: AMD EPYC 9V74, 2 cores, Ubuntu 24.04 container; .NET SDK 12.0.100-alpha.1.26521.103, runtime 12.0.0 (12.0.26.52911); BenchmarkDotNet 0.13.0 --config perflab (InProcess, MemoryDiagnoser), WarmupCount=3, IterationCount=15. Order per round: main, then PR. "main" is the PR branch with the product source files reverted to the merge base (4f556f7), so both builds run the same benchmark source. Means and bytes come from the BDN JSON exporter.

🤖 Posted by ASP.NET Core Perf 2 (AI Agent)

- Look up content types by extension span via Dictionary alternate lookup instead of allocating a substring.
- Build the ETag string with a single allocation.
- Write Last-Modified/ETag directly to the header dictionary instead of allocating ResponseHeaders.
- Avoid the params array when computing the precondition state.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings October 31, 2026 08:00
@github-actions github-actions Bot added the area-middleware Includes: URL rewrite, redirect, response cache/compression, session, and other general middlewares label Oct 31, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues remain for the static-file allocation optimizations.

No open issues 🎉

What changed in this PR

Reduces allocations in the static-file request path while preserving content-type lookup behavior and response headers.

Changes:

  • Uses span-based extension lookups for compatible dictionaries, retaining the string-based fallback for other mappings.
  • Removes intermediate ETag strings, typed response-header wrappers, and precondition arrays.
  • Adds regression tests for custom mappings, mapping mutations, and exact response-header values.
File Description
src/​Middleware/​StaticFiles/​test/​UnitTests/​StaticFileContextTest.cs Checks exact ETag and response headers, including negative hash values.
src/​Middleware/​StaticFiles/​test/​UnitTests/​DefaultContentTypeProviderTests.cs Covers mapping compatibility, mutations, and extension edge cases.
src/​Middleware/​StaticFiles/​src/​StaticFileContext.cs Reduces allocations in ETag formatting, header writing, and precondition evaluation.
src/​Middleware/​StaticFiles/​src/​FileExtensionContentTypeProvider.cs Adds allocation-free span lookups for compatible dictionaries.

Review effort: Balanced

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copy link
Copy Markdown
Member Author

Retriggering CI with an empty commit. The failures are unrelated to this change (it only touches StaticFiles content-type lookup and header writes):

🤖 Posted by ASP.NET Core Perf 2 (AI Agent)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copy link
Copy Markdown
Member Author

Pushed empty commit a1d008f to retrigger CI. The only failures were in aspnetcore-components-e2e (Blazor E2E, Linux, Mono server):

Neither test depends on content-type mapping or ETag formatting. The static file responses those tests receive keep byte-identical headers, which the StaticFiles tests cover.

🤖 Posted by ASP.NET Core Perf 2 (AI Agent)

…n PRs)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copy link
Copy Markdown
Member Author

Pushed empty commit 38d1685 to retrigger CI. Everything except aspnetcore-components-e2e passed. These e2e failures, in the Mono client/WebAssembly and server legs:

  • VirtualizationTest.VariableHeight_HtmlTable_DoesNotProduceNestedTrElements
  • RedirectionTest.NavigationException_InAsyncContext_DoesNotBecomeUnobservedTaskException
  • BlazorWebJsInitializersTest.InitializersRunsModernCallbacksByDefaultWhenPresent (2 cases)

are the same set failing right now on unrelated PRs: #69752 (only touches the RequestDelegateGenerator) fails exactly these four, and #70068 fails the first two. That's 3 of the ~12 recent PRs I sampled. Static file content types are unchanged by this PR. FileExtensionContentTypeProvider returns the same mapping for every extension (covered by the existing provider tests plus the new ones), and .js, .mjs, .wasm and .dll resolve identically.

🤖 Posted by ASP.NET Core Perf 2 (AI Agent)

@PureWeen

PureWeen commented Nov 1, 2026

Copy link
Copy Markdown
Member

/review

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@davidfowl davidfowl added the agent:working An AI agent is actively working on this issue/PR label Nov 6, 2026 — with Copilot SWE Agent
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@davidfowl davidfowl removed the agent:working An AI agent is actively working on this issue/PR label Nov 6, 2026
@davidfowl davidfowl added the agent:working An AI agent is actively working on this issue/PR label Nov 6, 2026 — with Copilot SWE Agent

Copy link
Copy Markdown
Member Author

On 55c4a17, aspnetcore-ci passed. The only failure is in the aspnetcore-components-e2e Mono server leg (build 1667861), and it isn't caused by this PR.

Failing test: Microsoft.AspNetCore.Components.E2ETest.ServerExecutionTests.ServerRoutingTest.NavigationLock_CanBlockExternalNavigation. Build Analysis gives it a 31.89% failure rate.
BrowserAssertFailedException: Expected: http://127.0.0.1:41537//myexternalpath3, Actual: http://127.0.0.1:41537/subdir/

Why it's unrelated

I didn't retrigger CI because main E2E is currently flaky. A maintainer can rerun the failed leg if needed.

🤖 Posted by ASP.NET Core Perf 2 (AI Agent)

@davidfowl davidfowl removed the agent:working An AI agent is actively working on this issue/PR label Nov 6, 2026
@dotnet-policy-service

Copy link
Copy Markdown
Contributor

Looks like this PR hasn't been active for some time and the codebase could have been changed in the meantime.
To make sure no conflicting changes have occurred, please rerun validation before merging. You can do this by leaving an /azp run comment here (requires commit rights), or by simply closing and reopening.

@dotnet-policy-service dotnet-policy-service Bot added the pending-ci-rerun When assigned to a PR indicates that the CI checks should be rerun label Nov 13, 2026

Copy link
Copy Markdown
Member Author

Merge assessment

Verdict: Merge after a fresh CI run and an area-middleware review. The content-type lookup win is real and the header-write changes produce the same bytes as main, but the last CI run is from Nov 6 and nobody with write access has reviewed it.

Size M: +43/−24 production lines, +105 test, +139 benchmark (+5 solution-file lines); 10 files, 1 production assembly (StaticFiles; 3 projects incl. test and new benchmark project)
Impact Medium: TryGetContentType 20.1–21.1 ns → 12.0–12.1 ns, 32 B → 0 B; StaticFileMiddleware.Invoke −144 B per HEAD/GET/304, time within noise (±2–6% across rounds)
Risk Medium: StaticFiles middleware, single assembly, no shared source
Tests 5 new FileExtensionContentTypeProvider cases (case-sensitive comparer, Mappings mutation, trailing dot, SortedDictionary, derived dictionary) and ApplyResponseHeaders_WritesExpectedETagAndLastModified incl. negative hashes; these are parity tests and pass on main by design; no allocation assertion
Review No approvals; only the Copilot bot reviewed; no review threads
CI aspnetcore-ci green on 55c4a17; aspnetcore-components-e2e Mono server leg red on NavigationLock_CanBlockExternalNavigation, a known Blazor flake (comment); mergeable, no conflicts
Complexity Small: span-lookup branch plus two fields; params array replaced by 4 compares
Reversibility Easy, self-contained
Security sensitivity Not assessed (repo review policy)
Encoding/escaping parity n/a: no escaping; ETag hex ({hash:x} vs Convert.ToString(hash, 16)) and Last-Modified text are compared against main's formula in the new test
Behavior / public API change None

Reasons to merge

  • Removes the per-request extension substring for the default provider and any exact Dictionary<string,string> with a span-capable comparer.
  • Header values are the same strings the typed ResponseHeaders setters wrote (HeaderUtilities.FormatDate, EntityTagHeaderValue.ToString).
  • Derived and non-Dictionary mappings keep the IDictionary.TryGetValue(string) path, covered by tests.

Reasons not to merge yet

  • End-to-end Invoke time is unchanged within noise, so the user-visible gain is −144 B per served file.
  • The SortedDictionary fallback is about 1 ns slower (12.7–13.2 → 13.6–14.9 ns).
  • It adds a new benchmark project and solution entries that need upkeep.
  • CI is 11 days old and dotnet-policy-service asked for a rerun.

What would change this verdict: a green rerun plus one area-owner approval makes this a straight merge.

🤖 Posted by ASP.NET Core Perf 2 (AI Agent)

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area-middleware Includes: URL rewrite, redirect, response cache/compression, session, and other general middlewares pending-ci-rerun When assigned to a PR indicates that the CI checks should be rerun

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants