JuliaSyntax: fix anonymous function parsing

[IanButterworth]

Fixes #60202

julia> function zip_missing(::Tuple{}, longer)
		      map(function (second_one)
		          (missing, second_one)
		      end, longer)
		  end
zip_missing (generic function with 1 method)

Written by Claude

[Keno]

Just for reference, @KristofferC just touched this code in JuliaLang/JuliaSyntax.jl#580

[IanButterworth]

The fix here has become quite complicated.. @c42f @KristofferC

[fogti]

Would it be possible to add a unit test for this? (incl. for the previous change that affected this part of the code)

Did we actually find out where the error comes from (by bisecting starting from Julia-1.12)?

[IanButterworth]

There are a couple of tests added already.

I believe the issue was introduced by JuliaLang/JuliaSyntax.jl#580

[fogti]

I think it would make a lot of sense to split the logic with a case distinction on _maybe_grouping_parens:

Suggested change

	# Check if there's a newline between `)` and the next `(` or `.`.
	# We need to find where `)` is and check what immediately follows it.
	# If peek(1, skip_newlines=false) is `)`, we're directly before it.
	# Otherwise there's whitespace/newline before `)`.
	next_token_pos = if peek(ps, 1, skip_newlines=false) == K")"
	# Directly before ), token after ) is at 2
	2
	else
	# There's whitespace before ), so ) is at 2
	# and what follows ) is at 3
	3
	end
	token_after_paren = peek(ps, next_token_pos, skip_newlines=false)
	# If token_after_paren is a newline, this is an anonymous function
	has_newline_after_paren = _maybe_grouping_parens && token_after_paren == K"NewlineWs"
	# Get the next significant token to determine if we need to parse a call
	next_kind = peek(ps, 2, skip_newlines=_maybe_grouping_parens && !has_newline_after_paren)
	next_kind = if _maybe_grouping_parens
	# Check if there's a newline between `)` and the next `(` or `.`.
	# We need to find where `)` is and check what immediately follows it.
	# If peek(1, skip_newlines=false) is `)`, we're directly before it.
	# Otherwise there's whitespace/newline before `)`.
	next_token_pos = if peek(ps, 1, skip_newlines=false) == K")"
	# Directly before ), token after ) is at 2
	2
	else
	# There's whitespace before ), so ) is at 2
	# and what follows ) is at 3
	3
	end
	token_after_paren = peek(ps, next_token_pos, skip_newlines=false)
	# If token_after_paren is a newline, this is an anonymous function
	# Get the next significant token to determine if we need to parse a call
	peek(ps, 2, skip_newlines= token_after_paren != K"NewlineWs")
	else
	# Get the next significant token to determine if we need to parse a call
	peek(ps, 2, skip_newlines=false)
	end

[fogti]

Subsequently, it makes sense to inspect more closely what happens if _maybe_grouping_parens is true: If the next tokens are:

• K")", K"NewlineWs", then we call peek(ps, 2, skip_newlines=false) = token_after_paren
• K")", !K"NewlineWs", then we call peek(ps, 2, skip_newlines=true) = token_after_paren
• !K")", x/*we expect K")" here, but don't check that*/, K"NewlineWs", then we call peek(ps, 2, skip_newlines=false)=token_after_paren (I think (hope I interpret it correctly) that __lookahead_index treats the case with skip_newlines=true such that it also skips newlines entirely, including in the count up to n=2)
• !K")", x, !K"NewlineWs", then we call peek(ps, 2, skip_newlines=false)=token_after_paren`

So, I think we can omit the peek(ps, 2, skip_newlines= token_after_paren != K"NewlineWs") entirely.

Suggested change

	# Check if there's a newline between `)` and the next `(` or `.`.
	# We need to find where `)` is and check what immediately follows it.
	# If peek(1, skip_newlines=false) is `)`, we're directly before it.
	# Otherwise there's whitespace/newline before `)`.
	next_token_pos = if peek(ps, 1, skip_newlines=false) == K")"
	# Directly before ), token after ) is at 2
	2
	else
	# There's whitespace before ), so ) is at 2
	# and what follows ) is at 3
	3
	end
	token_after_paren = peek(ps, next_token_pos, skip_newlines=false)
	# If token_after_paren is a newline, this is an anonymous function
	has_newline_after_paren = _maybe_grouping_parens && token_after_paren == K"NewlineWs"
	# Get the next significant token to determine if we need to parse a call
	next_kind = peek(ps, 2, skip_newlines=_maybe_grouping_parens && !has_newline_after_paren)
	next_token_pos = if _maybe_grouping_paren
	# Check if there's a newline between `)` and the next `(` or `.`.
	# We need to find where `)` is and check what immediately follows it.
	# If peek(1, skip_newlines=false) is `)`, we're directly before it.
	# Otherwise there's whitespace/newline before `)`.
	if peek(ps, 1, skip_newlines=false) == K")"
	# Directly before ), token after ) is at 2
	2
	else
	# There's whitespace before ), so ) is at 2
	# and what follows ) is at 3
	3
	end
	else
	2
	end
	# Get the next significant token to determine if we need to parse a call
	next_kind = peek(ps, next_token_pos, skip_newlines=false)