Regression: Bug with nospecialize and world age

[NHDaly]

:/ This is a bit of a complicated MRE... It appears that there has been some sort of codegen regression introduced in julia master, where applying heavy @nospecialize can cause a function to not pick up method invalidation, and/or to run in the wrong world age.

There should be a dynamic dispatch in this test, and it should pick up a method added by the test.

I can't reproduce it with a smaller example, but you can see it in the tests for this PR:
JuliaServices/ExceptionUnwrapping.jl@043150c from JuliaServices/ExceptionUnwrapping.jl#14.

The tests are failing on julia master, but not in the other versions. And I can reproduce it locally.

I added some printlns which seem to show the issue, although it's still hard to explain.
I added this diff:

 #     Cons: possibly hiding intermediate exceptions that might have been helpful to see.
@@ -110,7 +112,10 @@ function _summarize_exception(io::IO, e::CompositeException, stack; prefix 
= not
     end
 end
 # This is the overload that prints the actual exception that occurred.
function _summarize_exception(io::IO, exc, stack; prefix = nothing)
+    @show exc
+    @show is_wrapped_exception(exc)
+    global EXC = exc
     # First, check that this exception isn't some other kind of user-defined
     # wrapped exception. We want to unwrap this layer as well, so that we are
     # printing just the true exceptions in the summary, not any exception

and now you can see that the call to is_wrapped_exception inside the function behaves differently than the call at the top level. If i define a new type, at first is_wrapped_exception(e) returns false. But then, when i define unwrap_exception(e::X) = e.x, now is_wrapped_exception should return true. And it does at the top-level, but it doesn't inside the function:

julia> e = try try
           @assert false
       catch e
           rethrow(X(e))
       end catch e; 
           @show e
           @show is_wrapped_exception(e) 
           ExceptionUnwrapping.summarize_current_exceptions(stdout)
       end
e = X(AssertionError("false"))
is_wrapped_exception(e) = false
=== EXCEPTION SUMMARY ===

exc = X(AssertionError("false"))
is_wrapped_exception(exc) = false
X(AssertionError("false"))
 [1] top-level scope
   @ REPL[23]:2

julia> ExceptionUnwrapping.unwrap_exception(e::X) = e.x

julia> e = try try
           @assert false
       catch e
           rethrow(X(e))
       end catch e; 
           @show e
           @show is_wrapped_exception(e) 
           ExceptionUnwrapping.summarize_current_exceptions(stdout)
       end
e = X(AssertionError("false"))
is_wrapped_exception(e) = true
=== EXCEPTION SUMMARY ===

exc = X(AssertionError("false"))
is_wrapped_exception(exc) = false
X(AssertionError("false"))
 [1] top-level scope
   @ REPL[25]:2

julia> is_wrapped_exception(ExceptionUnwrapping.EXC)
true

summarize_current_exceptions seems to somehow not be picking up the newly defined method.

If i remove all the @nospecalizes in the package, everything works again.

Thanks, sorry this is pretty specific.

[vtjnash]

narrowed down so far to this weird little recursion gadget:

julia> module ExceptionUnwrapping
       @nospecialize
       unwrap_exception(@nospecialize(e)) = e
       unwrap_exception(e::Base.TaskFailedException) = e.task.exception
       @noinline function _summarize_task_exceptions(io::IO, exc, prefix = nothing)
           _summarize_exception((;prefix,), io, exc)
           nothing
       end
       @noinline function _summarize_exception(kws, io::IO, e::TaskFailedException)
           _summarize_task_exceptions(io, e.task, kws.prefix)
       end
       # This is the overload that prints the actual exception that occurred.
       @noinline function _summarize_exception(kws, io::IO, @nospecialize(exc))
           @show unwrap_exception(exc) === exc
           if unwrap_exception(exc) !== exc # something uninferrable
               return _summarize_exception(kws, io, unwrap_exception(exc))
           end
       end
       struct X; x; end
       end;

julia> let e = ExceptionUnwrapping.X(nothing)
           @show ExceptionUnwrapping.unwrap_exception(e) === e
           ExceptionUnwrapping._summarize_task_exceptions(stdout, e)
       end
ExceptionUnwrapping.unwrap_exception(e) === e = true
unwrap_exception(exc) === exc = true

julia> ExceptionUnwrapping.unwrap_exception(e::ExceptionUnwrapping.X) = e.x

julia> let e = ExceptionUnwrapping.X(nothing)
           @show ExceptionUnwrapping.unwrap_exception(e) === e
           ExceptionUnwrapping._summarize_task_exceptions(stdout, e)
       end
ExceptionUnwrapping.unwrap_exception(e) === e = false
unwrap_exception(exc) === exc = true

If we look closely, we see it decided that _summarize_task_exceptions(::IO, ::Any, ::Any) does not need to track any edges, even though the inlining pass clearly should have added them (to invoke _summarize_exception).

[NHDaly]

Is that enough to look into? I guess this is a bug that will start showing up in 1.10.

[gbaraldi]

#49404 @vtjnash this got bisected to that PR.

[vtjnash]

That likely doesn't tell you much other than inlining has a bug, but we already knew that.

[gbaraldi]

I guess that PR just exposes it then?

[NHDaly]

Whatever is broken in this example only started breaking on 1.10 though; it works fine on 1.9.

Are you saying that it might have just coincidentally exposed a bug that was already occurring even before 1.10 then? 👀