Repository navigation
Slice source text instead of copying it line by line for outlining - #20443
xperiandri wants to merge 18 commits into
Conversation
✅ Release-note check exemptedThe NO_RELEASE_NOTES label or localization-bot authorship exempts this pull request. |
BenchmarkBenchmarkDotNet, both versions of the changed code copied verbatim into a standalone harness (the compiler-internal helpers they use are Inputs are two real files from this repo:
"Build + scan" is what one outlining pass pays outside the AST walk, and the editor pays it per keystroke. On the large file that is 4.4 MB of garbage per pass before, 1.0 MB after, a 77% reduction; on the small file 76%. Where the allocations went:
Variance is high on the multi-modal rows (the harness reports bimodal distributions on the scan benchmarks, and the full-pass rows carry a wide StdDev), so treat the time ratios as order-of-magnitude rather than precise. The allocation numbers are exact and are the substance of the change. Two caveats worth stating plainly:
|
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
793ffee to
8e642e7
Compare
This comment has been minimized.
This comment has been minimized.
ac31157 to
9546828
Compare
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
Reverts the copies of dotnet#20443 that came in with an earlier rebase: "Slice source text instead of copying it line by line for outlining" and "Track comment lines by number instead of storing their text". They changed the public FSharp.Compiler.Service surface (Structure.getOutliningRanges took ReadOnlyMemory<char>[]) and added a System.Memory reference that fails restore with NU1510 - neither belongs here, and dotnet#20443 carries that work on its own. The Copilot snippets go back to the string[] lines Structure.getOutliningRanges takes on main. Everything outside the Copilot files is now as on main. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This comment has been minimized.
This comment has been minimized.
T-Gro
left a comment
There was a problem hiding this comment.
🤖🕵️ Please shorten the description using this guidance. Focus on the problem and why the change is needed, in simplified technical English. Leave the implementation inventory to the Files tab and retain necessary caveats.
91ebfe4 to
15cfe79
Compare
Dismissed: the approved commit is stale after a new head, and the current exact-head delta review is still pending.
|
Re the review of 13 Oct about comments. Counted the way that review counted, the diff now adds two comment lines (198 characters):
The 12 implementation and test comment lines are gone since Separate from comments: the editor now slices a pooled buffer instead of one big string. The numbers on a real VS snapshot are in the thread on |
The editor's block structure built the sourceLines array for Structure.getOutliningRanges by calling ToString() per line, allocating a fresh string for the entire file on every outlining pass, once per keystroke. getOutliningRanges now takes ReadOnlyMemory<char>[] and slices the already-materialized source text once (SourceText.GetLinesAsMemory()) instead. ReadOnlySpanCharExtensions in illib mirrors the existing Ordinal string helpers so span call sites read the same way string call sites do. A local recursive function closing over a ReadOnlySpan<char>-typed sibling cannot be compiled - the CLR disallows instantiating FSharpFunc<ReadOnlySpan<char>, _> as a closure field (FS0412) - so commentTypeOf moves to module scope, next to the CommentType it classifies. StructureTests.fs slices its own lines the same way at the call site, and FSharp.Compiler.Service.Tests needs a direct System.Memory PackageReference: FSharp.Compiler.Service's own reference to it is only transitive through the net472 ProjectReference's SetTargetFramework override, mirroring the FSharp.Core pin already in this project for the same reason. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
CommentList kept a copy of every comment line next to its line number, but the number alone identifies the line in the source array the function already holds, and only the first and last lines of a group are ever read back to compute the fold's columns. Store the numbers and index the source at the end, so grouping comments allocates no tuple per line. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
CheckCodeFormatting flagged illib.fsi for a stray space before the colon in the ReadOnlySpanCharExtensions signatures; dotnet fantomas fixes it mechanically, no signature changes. check_release_notes also requires an entry for changes under vsintegration/src. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Plain_Build_Windows and Plain_Build_Linux both failed with NU1510: on the .NET Core inner build System.Memory ships with the framework, and NuGet's package-pruning check treats an unconditional explicit PackageReference to it as an error. The pin is only needed on net472, where FSharp.Compiler.Service's own PackageReference to System.Memory doesn't flow through the netstandard2.0 SetTargetFramework override. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Wrap commentTypeOf's doc comment in <summary>, move the FS0412 rationale into <remarks>, and reference the types through <see cref> rather than inline code spans. Use the shorthand lambda for the whitespace check, per review suggestion. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Slicing before searching makes the result relative to the slice, while the String siblings these mirror return an index into the whole string. A call ported from the string path would land a column short by startIndex, and "not found" would come back as -1 from the slice rather than from the string. Nothing calls them. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Restores the startIndex overloads dropped in c9fbf5a, this time reporting the position in the span they were given rather than in the slice they searched, which is what the String siblings they mirror return. A miss still comes back as -1 rather than as startIndex - 1. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
A doc comment that carries markup like <see>/<paramref> needs that text inside <summary> - otherwise it renders as raw text in the generated XML and in tooltips. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
getCommentRanges recurses once per line, threading a three-way state through every call; a reference tuple heap-allocates on each of those recursive calls, a struct tuple doesn't. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Changing the signature made every FSharp.Compiler.Service consumer of the outlining API pay for a caller the editor alone has. The scanner now sits behind two entry points: the public one keeps its string[] parameter and maps to memory, and getOutliningRangesFromLineSlices takes the slices the editor already holds. The public surface is unchanged, so the surface-area baseline and the StructureTests calls return to what they were, and the release note moves out of Breaking Changes. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ve its FCS note to 11.0.200 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The editor passes the lines as slices of one string, each from its own offset; the structure tests only went through the string[] wrapper. The test feeds such slices - mixed line endings, indentation, trailing whitespace - to getOutliningRangesFromLineSlices, compares with getOutliningRanges over a string per line, and pins the ranges that are read off the line text. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The note was moved to 11.0.200 but its first copy stayed behind. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The doc blocks on commentTypeOf, the slice entry point of the signature file, GetLinesAsMemory and the structure test restated what their names say or defended the change. The test helper is renamed to say what it does. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
The editor's entry point had no test of its own. Compare its slices with the lines of the SourceText for text with \n, \r\n and \r line breaks. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
SourceText.ToString() copies the whole text into a new string on every outlining pass. For a Visual Studio snapshot that string is a large object for any real source file. WithLinesAsMemory fills a pooled char[] with SourceText.CopyTo and hands the line slices to a callback; the buffer goes back to the pool when the callback returns. getOutliningRangesFromLineSlices reads the slices before it returns, so its result does not point into the buffer. A new test overwrites the buffer after the call to keep it that way. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
The structure tests keep their ranges in arrays and compare them with the collection form of Assert.Equal, the source text of the parity test is a seq, and createBlockSpans reads the line number with the _.Range.StartLine shorthand. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
ec3d54a to
6138028
Compare

The editor's block structure rebuilt a
string[]of every line of the file on each outlining pass — that is, once per keystroke — just to hand the scanner something to read.The scanner now reads
ReadOnlyMemory<char>slices of one pooled copy of the source text, and classifies comment lines over spans instead of building trimmed strings.Structure.getOutliningRangeskeeps its publicstring[]signature, soFSharp.Compiler.Serviceconsumers are unaffected and the surface-area baseline is untouched; the editor calls an internal entry point that takes the slices directly.For a 13.7k-line file, the lines of one pass cost 220 KB instead of 2.1 MB on a real Visual Studio snapshot (.NET Framework 4.8), and the comment scan allocates 0.8 MB instead of 2.6 MB (measurements, scan benchmark).
🤖 Generated with Claude Code