Skip to content

Add address-review command#959

Merged
justin808 merged 1 commit intomainfrom
jg/add-address-review-cmd
Mar 8, 2026
Merged

Add address-review command#959
justin808 merged 1 commit intomainfrom
jg/add-address-review-cmd

Conversation

@justin808
Copy link
Copy Markdown
Member

@justin808 justin808 commented Mar 8, 2026

Summary

Adds the address-review command that fetches GitHub PR review comments and creates todos to address them. This command was ported from the React on Rails repository and provides functionality to extract, filter, and organize review feedback from GitHub PRs.

Pull Request checklist

  • [ ] Add/update test to cover these changes
  • [ ] Update documentation
  • [ ] Update CHANGELOG file

Other Information

This is a new custom command for the project, no CHANGELOG update needed.


Note

Low Risk
Documentation-only addition of a workflow command; no application/runtime code paths are modified.

Overview
Adds a new Claude custom command, /.address-review, documented in .claude/commands/address-review.md, to fetch GitHub PR review/issue comments via gh api, filter to actionable top-level (non-bot) feedback, and turn them into a pending todo list for the user to select from.

Also documents the follow-up workflow for replying back to the appropriate GitHub comment threads after fixes are made, including the relevant API calls and expected response content.

Written by Cursor Bugbot for commit b276a2f. Configure here.

Summary by CodeRabbit

  • Documentation
    • Added a comprehensive guide for handling GitHub PR review comments and converting them into actionable items, including workflows for repository detection, API strategies, filtering criteria, error handling, and todo creation.

@coderabbitai
Copy link
Copy Markdown

coderabbitai bot commented Mar 8, 2026

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: d63b198c-ccea-443f-b95c-deb2356e439b

📥 Commits

Reviewing files that changed from the base of the PR and between d67309a and b276a2f.

📒 Files selected for processing (1)
  • .claude/commands/address-review.md

Walkthrough

A new markdown guide is added at .claude/commands/address-review.md that documents a workflow for fetching GitHub PR review comments and converting them into actionable todos, including repository detection, API strategies, filtering criteria, error handling, and post-addressing reply mechanics.

Changes

Cohort / File(s) Summary
Documentation & Command Guide
.claude/commands/address-review.md
Comprehensive workflow documentation including repository detection logic, API fetch strategies for different comment types, filtering rules, error handling procedures, todo creation format, user presentation flow, and reply mechanics. Includes command examples, URL parsing rules, and sample outputs.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Poem

🐰✨ A guide hops into view,
GitHub comments turned to tasks anew,
Todos bloom from feedback seeds,
Each review addressed with rabbit speed! 📋🌿

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch jg/add-address-review-cmd

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@justin808 justin808 merged commit 11c0ba9 into main Mar 8, 2026
3 of 4 checks passed
@justin808 justin808 deleted the jg/add-address-review-cmd branch March 8, 2026 03:45
**If only PR number is provided (fetch all PR review comments):**

```bash
gh api repos/${REPO}/pulls/{PR_NUMBER}/comments | jq '[.[] | {id: .id, path: .path, body: .body, line: .line, start_line: .start_line, user: .user.login, in_reply_to_id: .in_reply_to_id}]'
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The gh api command supports --paginate which automatically handles multi-page responses. Since pagination is called out as a known limitation, this is an easy fix:

Suggested change
gh api repos/${REPO}/pulls/{PR_NUMBER}/comments | jq '[.[] | {id: .id, path: .path, body: .body, line: .line, start_line: .start_line, user: .user.login, in_reply_to_id: .in_reply_to_id}]'
gh api --paginate repos/${REPO}/pulls/{PR_NUMBER}/comments | jq '[.[] | {id: .id, path: .path, body: .body, line: .line, start_line: .start_line, user: .user.login, in_reply_to_id: .in_reply_to_id}]'

**If only PR number is provided (fetch all PR review comments):**

```bash
gh api repos/${REPO}/pulls/{PR_NUMBER}/comments | jq '[.[] | {id: .id, path: .path, body: .body, line: .line, start_line: .start_line, user: .user.login, in_reply_to_id: .in_reply_to_id}]'
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, when fetching by PR number only, top-level issue comments (/issues/{PR_NUMBER}/comments) are not fetched — only inline review comments are. Reviewers frequently leave general feedback as top-level comments on the PR discussion thread. Consider adding a second fetch:

gh api --paginate repos/${REPO}/issues/{PR_NUMBER}/comments | jq '[.[] | select(.user.login | endswith("[bot]") | not) | {id: .id, body: .body, user: .user.login, html_url: .html_url}]'

Then merge and de-duplicate results before creating todos.

- One todo per actionable review comment/suggestion
- For file-specific comments: `"{file}:{line} - {comment_summary} (@{username})"` (content)
- For general comments: Parse the comment body and extract actionable items
- Format activeForm: `"Addressing {brief description}"`
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

activeForm is not a field in the TodoWrite tool — todos only have content, status, and id. This looks like leftover terminology from a different tool or an earlier draft. Suggest removing this line to avoid confusing the AI that executes the command:

Suggested change
- Format activeForm: `"Addressing {brief description}"`
- For file-specific comments: `"{file}:{line} - {comment_summary} (@{username})"` (content)
- For general comments: Parse the comment body and extract actionable items
- All todos should start with status: `"pending"`

Comment on lines +84 to +100
## Step 6: Address Items and Reply

When addressing items, after completing each todo item, reply to the original review comment explaining how it was addressed.

**For issue comments (general PR comments):**

```bash
gh api repos/${REPO}/issues/{PR_NUMBER}/comments -X POST -f body="<response>"
```

**For PR review comments (file-specific, replying to a thread):**

```bash
gh api repos/${REPO}/pulls/{PR_NUMBER}/comments/{COMMENT_ID}/replies -X POST -f body="<response>"
```

**For standalone review comments (not in a thread):**
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The three reply strategies lack clear decision criteria, and the "standalone review comment" case is rarely correct when responding to existing feedback. A clearer decision tree would help:

  • If the comment has a comment_id (inline review comment): use POST /pulls/{PR}/comments/{COMMENT_ID}/replies
  • If it's a top-level issue comment (from the /issues/{PR}/comments endpoint): use POST /issues/{PR}/comments
  • The standalone POST /pulls/{PR}/comments path (with commit_id, path, line) is for creating new review threads, not replying — it should be removed or moved to a separate "creating new comments" section to avoid confusion.

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b276a2f0f9

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

**If a specific review ID is provided (`#pullrequestreview-...`):**

```bash
gh api repos/${REPO}/pulls/{PR_NUMBER}/reviews/{REVIEW_ID}/comments | jq '[.[] | {id: .id, path: .path, body: .body, line: .line, start_line: .start_line, user: .user.login}]'
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Retrieve review body for pullrequestreview URLs

Using GET /pulls/{PR_NUMBER}/reviews/{REVIEW_ID}/comments only returns inline diff comments, so a review URL like #pullrequestreview-... that contains only an overall review message (no line comments) will produce an empty result and lose actionable feedback. This breaks the advertised flow for specific review URLs because the command never fetches the review object's own body text.

Useful? React with 👍 / 👎.

**If only PR number is provided (fetch all PR review comments):**

```bash
gh api repos/${REPO}/pulls/{PR_NUMBER}/comments | jq '[.[] | {id: .id, path: .path, body: .body, line: .line, start_line: .start_line, user: .user.login, in_reply_to_id: .in_reply_to_id}]'
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Fetch issue comments when PR number is provided

The PR-number path only calls GET /pulls/{PR_NUMBER}/comments, which excludes general PR discussion comments from issues/{PR_NUMBER}/comments; as a result, /address-review 12345 misses non-inline reviewer feedback unless the user supplies an #issuecomment-... URL explicitly. This causes incomplete todo generation for common reviews that mix inline and general comments.

Useful? React with 👍 / 👎.


- Rate limiting: GitHub API has rate limits; if you hit them, wait a few minutes
- Private repos: Requires appropriate `gh` authentication scope
- Large PRs: PRs with many comments may require pagination (not currently handled)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As noted above, gh api --paginate handles pagination automatically and is a one-word fix. Suggest either fixing it or removing this from Known Limitations once fixed:

Suggested change
- Large PRs: PRs with many comments may require pagination (not currently handled)
- Large PRs: PRs with many comments use `--paginate` to fetch all pages automatically

Copy link
Copy Markdown

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 2 potential issues.

Bugbot Autofix is ON, but it could not run because the branch was deleted or merged before autofix could start.


## Step 2: Parse User Input

Extract the PR number and optional review/comment ID from the user's message:
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing $ARGUMENTS prevents receiving user's PR input

High Severity

The command never references the $ARGUMENTS template variable, so when a user invokes /address-review 12345 or /address-review https://github.com/org/repo/pull/12345, the PR number or URL is not injected into the prompt. The sibling command update-changelog.md correctly references $ARGUMENTS on its line 7. Without it, Claude won't know which PR the user wants to review, making the command non-functional.

Fix in Cursor Fix in Web


```bash
gh api repos/${REPO}/pulls/{PR_NUMBER}/comments -X POST -f body="<response>" -f commit_id="<COMMIT_SHA>" -f path="<FILE_PATH>" -f line=<LINE_NUMBER> -f side="RIGHT"
```
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Standalone comment reply creates new disconnected comment

Medium Severity

The "standalone review comments" case in Step 6 uses the POST /repos/{owner}/{repo}/pulls/{pull_number}/comments endpoint with path, line, and side parameters, which creates a brand-new standalone review comment rather than replying to the original. The correct endpoint for replying to any review comment (standalone or threaded) is the same /replies endpoint used in the case above it: POST .../comments/{COMMENT_ID}/replies.

Fix in Cursor Fix in Web

@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps bot commented Mar 8, 2026

Greptile Summary

This PR adds a new Claude custom command (.claude/commands/address-review.md) that orchestrates fetching GitHub PR review comments via the gh CLI and creating a structured todo list so developers can methodically address reviewer feedback. The command follows the same pattern as the existing update-changelog.md command in the repo.

Key findings:

  • Missing general PR discussion comments: When fetching by PR number only, the command calls GET /pulls/{PR}/comments (inline code comments) but omits GET /issues/{PR}/comments (general discussion comments left in the PR timeline). Reviewers frequently leave actionable feedback as general comments not tied to specific lines, so these would be silently ignored unless a specific issue comment ID is provided.
  • Undefined <COMMIT_SHA> placeholder: The standalone review comment POST call (Step 6) requires a commit_id parameter, but the command never explains how to obtain the PR head commit SHA — the placeholder is left unresolved, making the instruction incomplete for an agent trying to post new comments.

Confidence Score: 2/5

  • The command has two functional gaps that prevent complete usage of its features, particularly the undefined COMMIT_SHA placeholder which blocks the standalone review comment workflow.
  • Two substantive issues remain: (1) issue-level PR discussion comments are not fetched when using the "PR number only" input format, causing actionable general feedback to be silently missed; (2) the COMMIT_SHA placeholder in Step 6's standalone comment endpoint lacks guidance on how to obtain the value, making that code path unusable without external knowledge. While the command is safe to merge from a security perspective (it's documentation), these gaps mean it will produce incomplete or broken results in common scenarios.
  • .claude/commands/address-review.md — needs fixes for fetching issue comments and defining how to obtain COMMIT_SHA.

Sequence Diagram

sequenceDiagram
    participant User
    participant Claude
    participant ghCLI as gh CLI
    participant GitHubAPI as GitHub API

    User->>Claude: /address-review <PR number or URL>
    Claude->>ghCLI: gh repo view --json nameWithOwner
    ghCLI-->>Claude: REPO

    alt specific issuecomment ID
        Claude->>GitHubAPI: GET /repos/{REPO}/issues/comments/{COMMENT_ID}
        GitHubAPI-->>Claude: single comment
    else specific pullrequestreview ID
        Claude->>GitHubAPI: GET /repos/{REPO}/pulls/{PR}/reviews/{REVIEW_ID}/comments
        GitHubAPI-->>Claude: review comments
    else PR number only
        Claude->>GitHubAPI: GET /repos/{REPO}/pulls/{PR}/comments
        GitHubAPI-->>Claude: inline review comments
        Note over Claude,GitHubAPI: ⚠️ Issue-level comments NOT fetched
    end

    Claude->>Claude: Filter replies & bots (per steps 54-58)
    Claude->>Claude: TodoWrite — create pending todos
    Claude-->>User: Present N actionable items, wait for direction

    User->>Claude: "Address items 1, 2"
    Claude->>Claude: Implement fixes
    Claude->>GitHubAPI: POST reply to comment thread
    GitHubAPI-->>Claude: 201 Created
    Claude-->>User: Summary of changes made
Loading

Last reviewed commit: b276a2f

Comment on lines +48 to +52
**If only PR number is provided (fetch all PR review comments):**

```bash
gh api repos/${REPO}/pulls/{PR_NUMBER}/comments | jq '[.[] | {id: .id, path: .path, body: .body, line: .line, start_line: .start_line, user: .user.login, in_reply_to_id: .in_reply_to_id}]'
```
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

General PR discussion comments not fetched

When using the "PR number only" case, only inline code review comments are retrieved via /pulls/{PR_NUMBER}/comments. However, general discussion comments left in the PR timeline (e.g., a reviewer's overall feedback posted in the "Leave a comment" box) are stored at a different GitHub API endpoint: /issues/{PR_NUMBER}/comments.

These are two distinct resources:

  • GET /repos/{REPO}/pulls/{PR_NUMBER}/comments → inline code review comments only
  • GET /repos/{REPO}/issues/{PR_NUMBER}/comments → general PR discussion/timeline comments

Reviewers frequently leave actionable feedback as general comments (not tied to a specific line), so these would be silently skipped with the current approach. Consider also fetching issue-level comments when a PR number is provided:

gh api repos/${REPO}/issues/{PR_NUMBER}/comments | jq '[.[] | {id: .id, body: .body, user: .user.login, html_url: .html_url}]'

Comment on lines +100 to +104
**For standalone review comments (not in a thread):**

```bash
gh api repos/${REPO}/pulls/{PR_NUMBER}/comments -X POST -f body="<response>" -f commit_id="<COMMIT_SHA>" -f path="<FILE_PATH>" -f line=<LINE_NUMBER> -f side="RIGHT"
```
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

<COMMIT_SHA> placeholder is undefined

The standalone review comment POST call on line 103 requires a commit_id parameter, but the command does not explain how to obtain <COMMIT_SHA>. Without this guidance, an agent following these instructions would be unable to correctly fill in this placeholder.

The commit SHA should be fetched from the PR's head commit. Add this before the POST call:

COMMIT_SHA=$(gh api repos/${REPO}/pulls/{PR_NUMBER} | jq -r '.head.sha')

Then use $COMMIT_SHA in the POST command on line 103.

justin808 added a commit that referenced this pull request Mar 8, 2026
### Summary

Adds the address-review command that fetches GitHub PR review comments
and creates todos to address them. This command was ported from the
React on Rails repository and provides functionality to extract, filter,
and organize review feedback from GitHub PRs.

### Pull Request checklist

- ~[ ] Add/update test to cover these changes~
- ~[ ] Update documentation~
- ~[ ] Update CHANGELOG file~

### Other Information

This is a new custom command for the project, no CHANGELOG update
needed.

<!-- CURSOR_SUMMARY -->
---

> [!NOTE]
> **Low Risk**
> Documentation-only addition of a workflow command; no
application/runtime code paths are modified.
> 
> **Overview**
> Adds a new Claude custom command, `/.address-review`, documented in
`.claude/commands/address-review.md`, to fetch GitHub PR review/issue
comments via `gh api`, filter to actionable top-level (non-bot)
feedback, and turn them into a pending todo list for the user to select
from.
> 
> Also documents the follow-up workflow for replying back to the
appropriate GitHub comment threads after fixes are made, including the
relevant API calls and expected response content.
> 
> <sup>Written by [Cursor
Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit
b276a2f. Configure
[here](https://cursor.com/dashboard?tab=bugbot).</sup>
<!-- /CURSOR_SUMMARY -->
justin808 added a commit that referenced this pull request Mar 8, 2026
## Summary

- Added [Unreleased] changelog entries for two merged PRs missing from
CHANGELOG.md:
- **PR #963**: Fixed `Env#current` crashing when Rails is not loaded
(bug fix by @ihabadham)
  - **PR #900**: Added Node package API documentation (by @justin808)
- Skipped non-user-visible PRs: #958, #957, #959

## Test plan

- [x] `yarn lint` passes
- [x] Changelog formatting matches existing conventions

🤖 Generated with [Claude Code](https://claude.com/claude-code)

<!-- CURSOR_SUMMARY -->
---

> [!NOTE]
> **Low Risk**
> Documentation-only change confined to `CHANGELOG.md`, with no runtime
or build behavior impact.
> 
> **Overview**
> Updates `CHANGELOG.md` under **[Unreleased]** to document two
previously-merged changes: a fix preventing `Env#current` from crashing
when Rails isn’t loaded, and new documentation for the Node package
JavaScript API (`docs/node_package_api.md`).
> 
> <sup>Written by [Cursor
Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit
81bf2b8. Configure
[here](https://cursor.com/dashboard?tab=bugbot).</sup>
<!-- /CURSOR_SUMMARY -->

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant