Repository navigation
Native AOT with a JIT profile compiles the automaton's closure right, and releases publish with Trusted Publishing - #58
Merged
Conversation
…JIT profile compiles it right Given the profile Corvus.Text.Json 5.8.5 shipped, the native AOT compiler devirtualised the enumeration of an int[] through IEnumerable<int> in Dfa.DfaBuilder.Closure wrongly: the closure came out empty, and patterns the automaton answers, such as ^[0-9]+(ns|ms|us|µs|s|m|h)$ and ^(0|[1-9]\d*)\.(0|[1-9]\d*)$, matched nothing. The JIT, and native AOT without the profile, were right. Closure now takes a ReadOnlySpan<int>, and ClosureOfList a List<int>, so there is no interface call for a profile to devirtualise. Corvus.Regex.Ecma262.AotProfileTests is a native AOT program compiled with that profile (the part that covers this library). It checks 19 answers: 8 were right before the change, and all 19 are now. The aot workflow publishes and runs it, and the AOT sample. Releases publish to NuGet.org with Trusted Publishing, from a Publish job of build.yml (nuget.org matches a token by the workflow file its job runs in, so the shared pipeline's Publish job cannot), as the user in the organisation variable ZF_BUILD_DOTNET_NUGET_TRUSTED_PUBLISHING_USER. The expired NUGET_APIKEY is no longer used. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011rjLiHWYWRFfJb3dkSsnE3
Contributor
Code Coverage Summary Report - Linux (net10.0)Summary
CoverageCorvus.Regex.Ecma262 - 82.4%
|
Contributor
Contributor
Code Coverage Summary Report - Windows (net481)Summary
CoverageCorvus.Regex.Ecma262 - 83.2%
|
Contributor
Code Coverage Summary Report - Windows (net10.0)Summary
CoverageCorvus.Regex.Ecma262 - 82.4%
|
Contributor
|
The documentation preview for this pull request is at https://corvus-oss.org/Corvus.Regex.Ecma262/pr-preview/pr-58/ (built from 7241ec6). GitHub Pages takes a minute or two to publish it. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The bug
Corvus.Text.Json ships a static profile (
profiles/Corvus.Text.Json.mibc) that it hands to the native AOT compiler. With the profile from 5.8.0 to 5.8.5, an application published withPublishAotgot wrong answers from this library: patterns that the automaton answers matched nothing. Two examples are^[0-9]+(ns|ms|us|µs|s|m|h)$on30sand^(0|[1-9]\d*)\.(0|[1-9]\d*)$on1.2. It showed on the jsonschema-benchmark ascorvus-net-optcoldfailing cspell, jsconfig, krakend, ui5 and ui5-manifest.The JIT is right. Native AOT without the profile is right too.
How it was found
[MethodImpl(NoOptimization)]was bisected over the profiled methods down toDfa.DfaBuilder.Closure. Turning off optimisation for that method alone makes every answer right.foreachover itsIEnumerable<int> from, recorded on the JIT:List<int>andint[], withList<int>.EnumeratorandSZGenericArrayEnumerator<int>. The AOT compiler uses it to devirtualise the enumeration. That enumeration of an array throughIEnumerable<int>is where native AOT's implementation differs from CoreCLR's, and the closure came out empty.This is an AOT compiler fault. I could not cut it down to a standalone program yet: a minimal copy of
Closurewith its own JIT profile compiles right. It will be reported to dotnet/runtime with the repro above.The change
Engine/Dfa.cs:Closuretakes aReadOnlySpan<int>(the callers pass[start]and theint[]of a set), andClosureOfListtakes aList<int>, so there is no interface call for a profile to devirtualise. Nothing else changes.Corvus.Regex.Ecma262.AotProfileTests: a native AOT program compiled withprofiles/Corvus.Text.Json-5.8.5.mibc. That file is the part of 5.8.5's profile that covers this library (dotnet-pgo merge --include-reference Corvus.Regex.Ecma262 --include-reference System.Private.CoreLib). It checks 19 answers as UTF-16 and as UTF-8. Before the change 8 were right. Now all 19 are..github/workflows/aot.yml: publishes and runs that program, and the AOT sample, on every pull request and push to main.Publishjob ofbuild.yml, as Corvus.JsonSchema does since 5.8.7. The user is the organisation variableZF_BUILD_DOTNET_NUGET_TRUSTED_PUBLISHING_USER(endjin-nuget), and this repository can see it. The expiredNUGET_APIKEYis no longer used, which is why 1.0.14's tag build failed to publish.Verification
mcr.microsoft.com/dotnet/sdk:10.0.401-noble-aot(the jsonschema-benchmark's image): the profile tests give 8/19 right without the change and 19/19 with it, and the AOT sample runs.dotnet build Solutions/Corvus.Regex.Ecma262.slnx -c Release: 0 errors. The only warnings are the local SDK's NU1903 on the ILCompiler package (also on the AOT sample on main) and the net481 binding remap that main has.dotnet teston net10.0: 630 passed, 0 failed.No label, so merging releases 1.0.15. That needs a Trusted Publishing policy on nuget.org first: owned by
endjin-nuget, for repository ownercorvus-dotnet, repositoryCorvus.Regex.Ecma262and workflowbuild.yml(the file name, no environment).🤖 Generated with Claude Code
https://claude.ai/code/session_011rjLiHWYWRFfJb3dkSsnE3