Repository navigation
Conversation
Read small UTF-8 section bodies into a pooled buffer and decode them directly, falling back to StreamReader (with identical read boundaries) whenever byte order mark detection could apply or the body exceeds 64 KB. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
🟡 Changes recommended
The UTF-8 fast path can return buffered multipart content despite an already-canceled token, requiring an explicit cancellation check and a regression test.
1 open issue
What changed in this PR
This PR reduces per-field allocations when WebUtilities reads multipart form values by adding a pooled-buffer fast path for typical UTF-8 content.
Changes:
- Decodes small UTF-8 bodies directly, retaining a
StreamReaderfallback for other encodings, possible byte order marks, short initial reads, and large values. - Adds compatibility tests covering encodings, read boundaries, invalid UTF-8, and payload sizes.
- Adds multipart-reading benchmarks across field counts and value lengths.
| File | Description |
|---|---|
| src/Http/WebUtilities/test/MultipartSectionStreamExtensionsTests.cs | Adds decoding and stream-boundary compatibility tests. |
| src/Http/WebUtilities/src/MultipartSectionStreamExtensions.cs | Implements pooled UTF-8 decoding and prefixed-stream fallback. |
| src/Http/WebUtilities/perf/Microbenchmarks/MultipartSectionReadAsStringBenchmark.cs | Measures multipart field-reading performance and allocations. |
Review effort: Balanced
| buffer = newBuffer; | ||
| } | ||
|
|
||
| var read = await section.Body.ReadAsync(buffer.AsMemory(length), cancellationToken); |
There was a problem hiding this comment.
Thanks. I checked this against main and the premise doesn't hold. StreamReader.ReadToEndAsync(cancellationToken) only observes the token through the underlying ReadAsync calls. A buffered section body completes those without checking it, so on main ReadAsStringAsync with an already-canceled token returns the value instead of throwing. Adding an explicit ThrowIfCancellationRequested would therefore change behavior for UTF-8 sections. I verified this: with that check added, the UTF-8 case throws while the StreamReader-based charset path doesn't.
I added ReadAsStringAsync_BufferedSection_CanceledToken_MatchesStreamReader instead. It uses a section returned by MultipartReader with a canceled token, for both the UTF-8 fast path and a non-UTF-8 (StreamReader) charset, and it passes against both main's source and this PR. WebUtilities tests: 548 passed.
There was a problem hiding this comment.
Leaving this open: the suggested change would alter behavior. On main, StreamReader.ReadToEndAsync(token) observes the token only through the underlying ReadAsync calls, and a buffered MultipartReader section completes them without checking it. So with an already-canceled token, main returns the value. An upfront ThrowIfCancellationRequested would make UTF-8 sections throw while other charsets still return.
Evidence: e6418cd adds ReadAsStringAsync_BufferedSection_CanceledToken_MatchesStreamReader (UTF-8 fast path and iso-8859-1 StreamReader path). It passes against both main's implementation and this PR.
Separately, 6f13d3d limits the pooled fast path to sections created by MultipartReader. A caller-provided Body stream can no longer see, or keep, the pooled buffer.
🤖 Posted by ASP.NET Core Perf 2 (AI Agent)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…ions MultipartSection.Body can be set to an arbitrary stream, which would see the pooled buffer passed to ReadAsync and could retain it after it is returned. Limit the fast path to MultipartReaderStream, which only copies buffered data into the buffer, and use StreamReader for other bodies as before. Parity tests now cover both sections created by MultipartReader and caller-provided bodies. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
|
/review |
|
Looks like this PR hasn't been active for some time and the codebase could have been changed in the meantime. |
|
ASP.NET Core Perf 2 is addressing the 2026-11-16 review feedback on this PR. Update: done. Review feedback addressed in 5d3ad57; claim released. 🤖 Posted by ASP.NET Core Perf 2 (AI Agent) |
…fixedReadStream - Clear the used part of each pooled buffer before returning it (the whole buffer if a read fails part way). - A first read shorter than 3 bytes no longer forces the StreamReader fallback. If the body ends there (for example "1" or "on"), decode it directly; if more data arrives, fall back with the original read boundaries preserved. - Move PrefixedReadStream to src/Shared, replaying the first read and the rest of the buffered bytes as separate reads, and add tests for it. - Add 1- and 2-byte cases to MultipartSectionReadAsStringBenchmark. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
DeagleGross
left a comment
There was a problem hiding this comment.
I'd like us to simplify this path instead of replaying data into StreamReader. proposal:
- Use the existing
PooledArrayBufferWriter<byte>:
using var writer = new PooledArrayBufferWriter<byte>(4096);
while (true)
{
var read = await section.Body.ReadAsync(writer.GetMemory(1024), cancellationToken);
if (read == 0) break;
writer.Advance(read);
}- Read the whole body first, then decode once from
writer.WrittenSpan
- If the first byte can't start a BOM (not
EF,FE,FF,00), callstreamEncoding.GetString(span)directly. That's the common case and allocates only the result string. - Otherwise, or if the body is under 3 bytes, use
new StreamReader(new MemoryStream(...), streamEncoding, detectEncodingFromByteOrderMarks: true)over the buffered bytes.StreamReaderthen sees the whole body as one read, and the rare path costs the usual allocations.
the "under-3-bytes" is important to keep behavior the same as streamreader does. if you know how to do it without stream fallback let me know.
we will avoid the prefixedreadstream at all here then. remove it if the plan works
|
A little complex |

Avoid StreamReader allocations in MultipartSection.ReadAsStringAsync
Avoid StreamReader in MultipartSection.ReadAsStringAsync for UTF-8 values
Description
Makes
MultipartSection.ReadAsStringAsync(used byFormReader/FormFeaturefor every multipart form field) skip the per-callStreamReaderand its buffers for typical UTF-8 values.Today each call allocates a
StreamReaderalong with its 1 KB byte buffer, its char buffer and aStringBuilderforReadToEndAsync. That's about 3.5 KB of garbage per form field, even for a 16-byte value.With this change:
MultipartReader: the body is read into anArrayPoolbuffer. It starts at 4 KB and doubles up to a 64 KB cap. At EOF we decode it directly withEncoding.GetString, so only the result string is allocated (56 B for a small field, down from 3,512 B). The used part of each pooled buffer is cleared before it's returned (the whole buffer if a read fails).StreamReadercode, fed by the bytes already read and then the rest of the body. That covers:Bodywas set by the caller. Only the internalMultipartReaderStreamever sees the pooled buffer, and it only copies already-buffered data into it, so caller code can't observe the buffer or keep it after it's returned;EF/FE/FF/00);1oron) still takes the fast path.Behavior is unchanged, including edge cases:
StreamReader's BOM detection depends on read boundaries: it only detects on a read of ≥2 bytes, and it stays armed after shorter reads.StreamReaderwould have ruled out every BOM (and the UTF-8 preamble) on its first read; in the second it never sees another read. Either way it would have decoded the bytes as-is.StreamReadersees the same read boundaries as before: the sharedPrefixedReadStream(now insrc/Shared) replays the first read and the rest of the buffered bytes as separate reads.New tests compare the result against
StreamReaderreading the same stream with the same read pattern. They cover:These parity tests pass both on main and with this change; see Tests below for the tests that fail without it.
No public API changes.
Each parity case runs both a
MultipartReadersection (fast path) and a caller-provided body (StreamReaderpath), and checks both againstStreamReaderunder identical read chunking.Benchmark (3 interleaved rounds)
MultipartSectionReadAsStringBenchmark(added in this PR,src/Http/WebUtilities/perf/Microbenchmarks), now including 1- and 2-byte values. Measured on head 5d3ad57; the same benchmark file was compiled into the main build.Environment: AMD EPYC 9V74, 2 vCPU (container), Ubuntu 24.04; .NET SDK 12.0.100-alpha.1.26521.103; BenchmarkDotNet
--config perflab(InProcess, MemoryDiagnoser), WarmupCount=3, IterationCount=10. "main" is the merge base4f556f73a6. Runs were interleaved main, PR, main, PR, main, PR. Each cell shows mean / allocated per operation; allocations are exact bytes from the JSON exporter.StreamReaderfallback. Time is within noise (+4.5% / −1.1% median, with PR rounds on both sides of main), and allocation is essentially unchanged.Tests
src/Http/WebUtilities/test: 573 passed.MultipartSectionStreamExtensionsTestscovers parity withStreamReader(including 1–2 byte values and short reads before a near-boundary), that small values only allocate the result string (fails on main and on the previous head for 1–2 byte values), and that pooled buffers are cleared (fails without the clearing).src/Shared/test/Shared.Tests: 14 newPrefixedReadStreamTestspassed (read boundaries, destination limits, sync/async, argument validation).🤖 Posted by ASP.NET Core Perf 2 (AI Agent)