Pasting multi-line input eats comment

[lgoettgens]

We experience this in Oscar.jl (cf. oscar-system/Oscar.jl#4402 (comment))

Try pasting the full code block into the REPL:

julia> x = 1
       # hello
1

If you paste this in 1.11 it looks exactly like here, in 1.12-nightly (08d3c70) and nightly (f6e2b98) it changes to:

julia> x = 1
1

cc @benlorenz @c42f

[benlorenz]

It will also ignore all code following a comment (outside of arrays/tuples/blocks), pasting this

julia> x = 1
       #hello
       y = 2

will just print:

julia> x = 1
1

and y is not assigned.

For 1.11 and 1.10 it prints:

julia> x = 1
       #hello
1

But y is also ignored.

[KristofferC]

Bisected to 3e08cb1. Will see what commit in JuliaSyntax broke it.

[KristofferC]

JuliaSyntax commit that introduced this regression is in JuliaLang/JuliaSyntax.jl#493. cc @timholy

[mlechu]

@benlorenz I don't think we can change this without breakage. The REPL ignores output lines in this sort of paste (which I didn't know before looking into it), so anything not consumed by a Meta.parse starting at the prompt does not make the cut.

@KristofferC you're right that this issue is the same thing as #57345. I thought I could change the REPL behaviour independent of what Meta.parse does, but the ignored lines in a prompted paste are defined as "everything Meta.parse doesn't consume" and skipping over comments manually would be unwise.

Do you think we should revert JuliaLang/JuliaSyntax.jl#493 and restore the 1.10 and 1.11 behaviour? Given package test failures for either, I'm not sure what the compatible choice is.

[timholy]

I don't know either. CodeTracking transiently worked around this by disabling a test: JuliaDebug/CodeTracking.jl@f3a9f9b; that's not necessary after JuliaLang/JuliaSyntax.jl#493, but the current lack of a good substitute is not ideal. I'm not sure how serious the Documenter issues were, perhaps @mortenpi would know.

[vtjnash]

@JeffBezanson and I looked into this closer, and we don't think the old behavior was necessarily correct anyways. Trailing text should be assumed to be part of the output, so it is typically correct to strip anything that isn't necessary to complete the parse. That does mean there are cases where pasting text will remove part of the intended input, but we could never distinguish those cases before either, and we just accepted and intended that it is just a best-effort attempt. If you remove the "julia>" prompt text, then everything pastes in as expected without triggering these heuristics.