CI with Julia v1.11

[navidcy]

This PR switches the CI to use Julia v1.11.
It also adds a Manifest with v1.11 ending so that there is still compatibility with previous versions.

Note the the distributed CI still does not have Julia v1.11 (right @Sbozzolo?) so there Julia v1.10 is used. This is possible because there are two Manifests.

[Sbozzolo]

Note the the distributed CI still does not have Julia v1.11 (right @Sbozzolo?) so there Julia v1.10 is used. This is possible because there are two Manifests.

I installed julia 1.11 on the Caltech cluster, but we haven't made a module yet (but it's coming today or so)

[glwagner]

This is possible because there are two Manifests.

There's only one manifest?

[ali-ramadhan]

This is possible because there are two Manifests.

There's only one manifest?

I assume he's referring to this new feature of Julia 1.11: https://julialang.org/blog/2024/10/julia-1.11-highlights/#manifest_versioning

[navidcy]

There's only one manifest?

The PR adds another Manifest specifically for v1.11 and keeps the older Manifest that works for v1.10. Or we can just have one Manifest (the one for v1.11) and drop the one for v1.10

[glwagner]

keeps the older Manifest that works for v1.10

The Manifest was deleted in #3783

[navidcy]

oh great! I missed that!
I wanted to delete it long now but I got resistance doing that before... Great! So I'll delete the new Manifest then as well!

[glwagner]

oh great! I missed that! I wanted to delete it long now but I got resistance doing that before... Great! So I'll delete the new Manifest then as well!

Deleting it seemed to help increase the likelihood that CI passed. Although, it did not fully solve the problem (and note a few other changes were also made on #3783).

[navidcy]

Noting that internal_tide.jl gives NaN with Julia v1.11 while all is OK with Julia v1.10; something with immersed boundaries....? I'm looking into it.

[simone-silvestri]

I think it's a plotting issue. We are filling up the immersed boundaries with NaN and, apparently, we cannot plot NaNs anymore? The error says:

ERROR: LoadError: On worker 2:
  | Looking up a non-finite or NaN value in a colormap is undefined.

[navidcy]

I ran the script and the actual simulation NaN-ed.

[glwagner]

I ran the script and the actual simulation NaN-ed.

That means Oceanangians isn't compatible with julia 1.11.

Do any other tests catch the issue? We can use this opportunity to add more tests.

[navidcy]

I’m trying to make an mwe

[navidcy]

@glwagner, note that the ocean mixing example NaNs....

[ali-ramadhan]

Since Debugger.jl seems to have started working with Julia 1.11 (JuliaDebug/Debugger.jl#361) I thought I should take another look here.

But with Julia 1.11.6 all the examples seem to run just fine. I built the docs and ran all the examples, and the outputs all look good. So maybe no examples are blowing up anymore?

Looking at a recent-ish test build, if the docs will build now then it's just an Enzyme test failure plus a few test failures (see #4663 (comment)) that might be fixed by #4734 (tentatively)?

I'll run the full test suite locally with Julia 1.11 (with and without this potential fix: #4663 (comment)) and report back.

[glwagner]

Since Debugger.jl seems to have started working with Julia 1.11 (JuliaDebug/Debugger.jl#361) I thought I should take another look here.

But with Julia 1.11.6 all the examples seem to run just fine. I built the docs and ran all the examples, and the outputs all look good. So maybe no examples are blowing up anymore?

Looking at a recent-ish test build, if the docs will build now then it's just an Enzyme test failure plus a few test failures (see #4663 (comment)) that might be fixed by #4734 (tentatively)?

I'll run the full test suite locally with Julia 1.11 (with and without this potential fix: #4663 (comment)) and report back.

Are you on ubuntu? I believe the errors only manifest on ubuntu (possibly only on CPU). I'd expect them to get fixed eventually though.

[ali-ramadhan]

Are you on ubuntu? I believe the errors only manifest on ubuntu (possibly only on CPU). I'd expect them to get fixed eventually though.

I ran the examples locally on Arch Linux / EndeavourOS (where the examples have NaNed before). The tests are currently running on Debian.

Curious why it would be specific to a Linux distro.

[glwagner]

Curious why it would be specific to a Linux distro.

Yes, this is the question of the hour!

[simone-silvestri]

Not sure if it is something relevant, but this was giving me a bounds error in julia 1.11
d739326

[ali-ramadhan]

Got pretty far in before test_distributed_model.jl complained about the instantiate function for some reason which we might need to fix for Julia 1.11. But so far so good I think. Then I realized I only ran the CPU tests lol.

Maybe it will be easier to run tests for Julia 1.10 + 1.11 (CPU + GPU) once CI is back online (hopefully next week!).

Oceananigans: Error During Test at /home/alir/atdepth/Oceananigans.jl/test/runtests.jl:20
  Got exception outside of a @test
  LoadError: invalid method definition in Main: function Main.instantiate must be explicitly imported to be extended
  Stacktrace:
    [1] top-level scope
      @ none:0
    [2] top-level scope
      @ ~/atdepth/Oceananigans.jl/test/test_distributed_models.jl:38
    [3] include
      @ ./sysimg.jl:38 [inlined]
    [4] macro expansion
      @ ~/atdepth/Oceananigans.jl/test/runtests.jl:178 [inlined]
    [5] macro expansion
      @ ~/.julia/juliaup/julia-1.11.6+0.x64.linux.gnu/share/julia/stdlib/v1.11/Test/src/Test.jl:1709 [inlined]
    [6] (::var"#11#12")()
      @ Main ~/atdepth/Oceananigans.jl/test/runtests.jl:22
    [7] task_local_storage(body::var"#11#12", key::Symbol, val::GPUArraysCore.ScalarIndexing)
      @ Base ./task.jl:315
    [8] allowscalar(f::Function)
      @ GPUArraysCore ~/.julia/packages/GPUArraysCore/aNaXo/src/GPUArraysCore.jl:179
    [9] top-level scope
      @ ~/atdepth/Oceananigans.jl/test/runtests.jl:18
   [10] include(fname::String)
      @ Main ./sysimg.jl:38
   [11] top-level scope
      @ none:6
  in expression starting at /home/alir/atdepth/Oceananigans.jl/test/test_distributed_models.jl:38
Test Summary:                               |    Pass  Error  Broken    Total       Time
Oceananigans                                | 4062298      1      10  4062309  210m22.2s
  Unit tests                                | 4006064                 4006064    5m10.9s
  AbstractOperations and broadcasting tests |     933                     933    2m44.5s
  TripolarGrid tests                        |      46                      46      26.9s
  Poisson Solvers 1                         |     286                     286    4m04.7s
  Poisson Solvers 2                         |     660                     660   20m38.6s
  Matrix Poisson Solvers                    |      31                      31      20.2s
  General Solvers                           |      70                      70       6.2s
  Simulation tests                          |   19101                   19101   10m54.3s
  Lagrangian particle tracking tests        |    1771                    1771    4m36.3s
  Model and time stepping tests (part 1)    |    2992              2     2994   11m28.1s
  Model and time stepping tests (part 2)    |   28606                   28606   22m27.3s
  Model and time stepping tests (part 3)    |     633              1      634   58m05.0s
  Turbulence closures tests                 |     218                     218    3m22.7s
  Shallow Water Models                      |      42                      42    1m27.6s
  HydrostaticFreeSurfaceModel tests         |     254              7      261    8m23.1s
  Multi Region tests                        |     591                     591   55m34.4s
ERROR: LoadError: Some tests did not pass: 4062298 passed, 0 failed, 1 errored, 10 broken

[ali-ramadhan]

This is worth trying again now that CUDA.jl v5.8.4 is out which should fix some of the GPU failures on Buildkite.

[giordano]

Superseded by #4836!