Add Julia 1.13 support while maintaining Julia 1.12 compatibility#815
Add Julia 1.13 support while maintaining Julia 1.12 compatibility#815PatrickHaecker wants to merge 9 commits into
Conversation
Add a new analyzer that detects `if @generated` functions where the inferred effects of the generated branch (for concrete types) differ from the fallback branch (for abstract types). Such mismatches can lead to [unsound optimizations](JuliaLang/julia#61601 (comment)) where the compiler removes code from one branch based on stronger effects inferred from the other. The analyzer hooks into `finishinfer!` to inspect each inferred method. For methods with a generator, it extracts the method signature bounds to build an abstract type tuple that triggers the fallback branch, then compares effects field-by-field (excluding `nonoverlayed`, which legitimately differs). Entry points: `report_generated_effects`, `@report_generated_effects`, `test_generated_effects`, `@test_generated_effects`. This change was generated by Claude Opus 4.6. I only quickly went through the code as a quick plausibility check. However, the test cases look good and are passed, so it seems to be useful. This builds on top of aviatesk#815, so aviatesk#815 should be handled first.
|
Any chance that you can soon look at this @aviatesk? |
|
With 1.13 RC1 now available, this compatibility update becomes especially important. Our package requires JET for its unit tests, so the lack of JET compatibility effectively blocks us from testing on 1.13. (As I understand it, there is no clean way to set up the test/Project.toml so that the JET version is controlled by the Julia version?) |
You can use different manifests for different Julia versions. Create the Julia 1.13 manifest from a copy of the generic manifest, modify the Julia 1.13 manifest by (@v1.13) pkg> add https://github.com/PatrickHaecker/JET.jl#julia-1.13-compatand then (@v1.13) pkg> free JET(if I don't forget to delete the branch in time, otherwise you'd need to set yourself a trigger). In addition, you now need to maintain two manifests. So this can only be a workaround, especially for such an important package as JET.jl. |
|
Thanks for the tips. I tried running the Sunny.jl tests using 1.13-RC1 and this JET branch, but get lots of errors of the form: Does this ring a bell? Full trace: JET-output-Sunny.txt |
830955d to
e61d2b8
Compare
|
@kbarros: |
Prior to 1.13, all Sunny tests pass without any errors or warnings, and JET tests are part of our CI. So if you get many errors, that means JET is not working yet, unfortunately. Are you asking me to check if the delta from your two recent commits is going in the right direction? Thanks for working on this. |
|
Yes, @kbarros, please rerun the Sunny tests from an environment where you have I think your reasoning is incomplete. If there are errors in the 1.13 JET tests, at least the following things are possible
I think after running the tests you can judge best where the problems are located. |
15df07f to
52f4fae
Compare
|
Thanks for the clarification.
Here is a minimal script that fails on Julia 1.13 (with your using Sunny
using JET
latvecs = lattice_vectors(1, 1, 2, 90, 90, 90)
crystal = Crystal(latvecs, [[0, 0, 0]])
sys = System(crystal, [1 => Moment(s=1, g=2)], :dipole; dims=(2, 2, 1))
@test_opt energy(sys) |
|
Your example fails for me without JET, too: julia> using Sunny
julia> latvecs = lattice_vectors(1, 1, 2, 90, 90, 90)
3×3 StaticArraysCore.SMatrix{3, 3, Float64, 9} with indices SOneTo(3)×SOneTo(3):
1.0 0.0 0.0
0.0 1.0 0.0
0.0 0.0 2.0
julia> crystal = Crystal(latvecs, [[0, 0, 0]])
Crystal
HM symbol 'P 4/m m m' (123)
Lattice params a=1, b=1, c=2, α=90°, β=90°, γ=90°
Cell volume 2
Wyckoff 1a (point group '4/mmm'):
1. [0, 0, 0]
julia> sys = System(crystal, [1 => Moment(s=1, g=2)], :dipole; dims=(2, 2, 1))
ERROR: UndefVarError: `Moment` not defined in `Main`
Suggestion: check for spelling errors or missing imports.
Stacktrace:
[1] top-level scope
@ REPL[3]:1
[2] top-level scope
@ REPL:1
(@v1.13) pkg> status Sunny
Status `~/.julia/environments/v1.13/Project.toml`
⌃ [2b4a2ac8] Sunny v0.2.0 |
|
Looks like you’re on Sunny 0.2.0. The latest version is 0.9.1. |
Update the Julia compat entry in Project.toml from "1.12" to "1.12, 1.13" to declare support for Julia 1.13. Written by Claude Opus 4.6
On Julia 1.13, calling CodeTracking.whereis on methods from Base submodules (e.g. Base.Threads) triggers Revise lazy evaluation of ALL Base source files. This causes hundreds of world age advances which corrupts IOContext dispatch, leading to crashes in the printing infrastructure. Add _is_basemodule() helper that walks parent modules to detect Base/Core submodule methods, and skip the CodeTracking.whereis call for those methods in fixed_line_number(). Base methods do not need line number revision since they are not user-editable. Written by Claude Opus 4.6
Julia 1.13 replaces several expression heads with function calls in the binding partition system: - :globaldecl -> Core.declare_global() - :using/:import -> Base._eval_using()/Base._eval_import() Add these new call forms to select_direct_requirement! so they are properly concretized during toplevel analysis. Also intercept module usage statements (using/import) before lowering, since on 1.13 they lower to _eval_using/_eval_import calls that are not recognized by ismoduleusage in the lowered form. By handling them pre-lowering, we ensure module usage inside begin blocks is properly processed. Note: Core.declare_const() is intentionally NOT added to direct requirements, as that would over-concretize and execute side effects (e.g. Downloads.download). Instead, declare_const is pulled in as a dependency when struct or method definitions need it. Written by Claude Opus 4.6
Julia 1.13 replaces :const expression heads with Core.declare_const(mod, :name, value) calls. Without handling these in the abstract interpreter, JET cannot track const binding types abstractly, which breaks: - Type alias resolution (const T = SomeType) - Conditional const tracking - Struct definitions depending on const bindings - Overall type inference accuracy for toplevel code Add is_declare_const_call() to detect these calls, and abstract_eval_declare_const() to handle them by extracting the module, name, and value arguments and delegating to the existing const_assignment_rt_exct infrastructure. Written by Claude Opus 4.6
On Julia 1.13, Core.declare_global and Core.declare_const are builtins that can appear in lowered toplevel code. The compiler does not yet report them as nothrow, which causes the sound mode error checker to emit spurious UnsoundBuiltinErrorReport for these binding partition operations. Add is_binding_partition_builtin() predicate (guarded with @static if for 1.12 compatibility) and use it to skip reporting for these builtins in _report_builtin_error_sound!. Written by Claude Opus 4.6
Update test infrastructure and expectations for Julia 1.13 compatibility: - with_isolated_testset: Use Test.@with_testset on 1.13 since Test.push_testset/Test.pop_testset were removed - @capture patterns: Match both Core.setglobal! (1.13) and Base.setglobal! (1.12) in statement selection tests - Mark tests as broken on 1.13 where the binding partition changes cause false positive toplevel error reports that are not yet fully resolved: - Include chain tests with NonBooleanCondErrorReport - World age @testset test - @test macros integration tests - File target tests (error.jl, dict.jl) - Guard test_sum_over_string call with version check since it depends on inference results not available on 1.13 Written by Claude Opus 4.6
52f4fae to
a9bbec9
Compare
This is the culmination of an intense collaboration with Claude Opus 4.6 to get
runtests.jlerror free and reduce the number of warnings on 1.13. The LLM needed a lot of guidance here and we both operated at our limits. All changes are tested to really be necessary to avoid errors or warnings inruntests.jl(I hope we made no error here). I think, that all changes make sense, but I do not understand every detail and especially it might be that some things should be solved differently.I recommend to have this reviewed by someone really knowing this stuff. @aviatesk, this might be something for you. I recommend to go commit-by-commit in the review.
The remaining warnings seem hard to fix, because they either are intended or might need a Julia started with
--depwarn error, but the corresponding errors are caught by JET's error handling. So we might need to temporarily deactivate the error handling, to find and fix them, but this can be done in the future.Mostly fixes #796. However, it would still need a changed registry constraint to really fix it.
Now Claude's summary about the commits:
Background
Julia 1.13 introduces breaking changes to the binding partition system in lowered IR:
:constexpression heads are replaced byCore.declare_const()calls:globaldeclis replaced byCore.declare_global()callsBase.setglobal!becomesCore.setglobal!in lowered IRTest.push_testset/pop_testsetare removed in favor ofTest.@with_testsetCore.declare_globalandCore.declare_constare builtins that the compiler does not yet report asnothrowChanges
compat: add Julia 1.13 to supported versions
Update
Project.tomlcompat to"1.12, 1.13".print: guard CodeTracking.whereis for Base/Core submodules
On 1.13, calling
CodeTracking.whereison Base submodule methods triggers Revise lazy evaluation of all Base source files, causing hundreds of world age advances that corruptIOContextdispatch and crash the printing infrastructure. Add_is_basemodule()to detect and skip these methods.virtualprocess: handle Julia 1.13 binding partition IR
Add the new
Core.declare_global()andBase._eval_using()/Base._eval_import()call forms toselect_direct_requirement!so they are properly concretized during toplevel analysis.typeinfer: handle declare_const in abstract evaluation
Add
is_declare_const_call()andabstract_eval_declare_const()to handleCore.declare_const(mod, :name, value)calls by delegating to the existingconst_assignment_rt_exctinfrastructure. Without this, JET cannot track const binding types abstractly on 1.13.jetanalyzer: filter binding partition builtins in sound mode
Add
is_binding_partition_builtin()to skip spuriousUnsoundBuiltinErrorReportforCore.declare_globalandCore.declare_constin the sound error checker.test: update test expectations for Julia 1.13
Test.@with_testseton 1.13 via arun_with_testsetcompat helperCore.setglobal!(1.13) andBase.setglobal!(1.12) in statement selection testsbrokenon 1.13 where binding partition changes cause false positiveNonBooleanCondErrorReportfromUnion{Missing, Bool}inference on comparison operatorsTest results