feat: Add token methods to JSONSourceCode

[nzakas]

Prerequisites checklist

• I have read the contributing guidelines.

What is the purpose of this pull request?

Adds token methods to JSONSourceCode.

What changes did you make? (Give an overview)

Added the following methods to JSONSourceCode:

• getTokenBefore()
• getTokenAfter()
• getTokenOrCommentBefore()
• getTokenOrCommentAfter()

Also added tests for these methods.

Additionally, I changed comments to a getter so it can be lazily initialized.

(The tokens array from Momoa contains both tokens and comments, so it makes sense to lazily initialize it all together.)

Related Issues

fixes #83

Is there anything you'd like reviewers to focus on?

I wasn't sure if we needed getTokenOrCommentBefore() or getTokenOrCommentAfter(). These appear as convenience methods on SourceCode in the eslint package, but getTokenBefore() and getTokenAfter() also have a second argument that lets you specify whether or not to include comments. We could go that way here and eliminate two methods, or we could keep these as-is. I'm not sure what the better path is as we'd likely want to encourage other languages to match whatever we decide on here.

[lumirlumir]

Could you take a look at the CI failure? Bun CI is currently throwing an error.

Also, I’ve left comments on a few parts!

[nzakas]

Bun test looks like it was a temporary issue.

[lumirlumir]

Overall looks good to me! I've left a few comments on some missing parts.

[mdjermanovic]

I wasn't sure if we needed getTokenOrCommentBefore() or getTokenOrCommentAfter(). These appear as convenience methods on SourceCode in the eslint package, but getTokenBefore() and getTokenAfter() also have a second argument that lets you specify whether or not to include comments. We could go that way here and eliminate two methods, or we could keep these as-is. I'm not sure what the better path is as we'd likely want to encourage other languages to match whatever we decide on here.

getTokenOrCommentBefore() and getTokenOrCommentAfter() are deprecated in the eslint package:

https://eslint.org/docs/latest/extend/custom-rules#deprecated-sourcecode-methods

So I think we shouldn't have these methods here, but an option to include comments in getTokenBefore() and getTokenAfter().

[nzakas]

@mdjermanovic good call. Don't know how I missed that. 👍

[lumirlumir]

LGTM, Thanks!

I’ve left a minor typo fix in the comments. Would like @mdjermanovic to verify the changes before merging.

[lumirlumir]

Currently, the CI fails for the same reason that happened in eslint/css#182.

I've looked into the Prettier repository and found that there is a fix ongoing for this:

• Markdown: newline removed around code block in v3.6.0 prettier/prettier#17652
• Add missing blank line around code block prettier/prettier#17675

So, if this new feature addition is not in a hurry, I think we can wait until the Prettier bug fix is released.
If not, we can fix the CI like we did in eslint/css#182.

[nzakas]

I've fixed the Prettier error so this should be ready for review.

[lumirlumir]

Re-approving. Still pending review from @mdjermanovic before merging.

[nzakas]

ping @mdjermanovic

[mdjermanovic]

LGTM, thanks! Leaving open for @lumirlumir to re-review.

[lumirlumir]

LGTM, Thanks!

Pending until the CI error is resolved.

(Maybe #117 could be a workaround.)

[nzakas]

I think we can merge this because the CI failures are known to not be related.