Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .github/pull_request_template.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,11 @@

[ What changed? Feel free to be brief. ]

## AI Code Review

- **Team members only**: AI review runs automatically when PR is opened or marked ready for review
- Team members can also trigger a review by commenting `@continue-review`

## Checklist

- [] I've read the [contributing guide](https://github.com/continuedev/continue/blob/main/CONTRIBUTING.md)
Expand Down
186 changes: 186 additions & 0 deletions .github/workflows/continue-review.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,186 @@
name: Continue CLI Code Review

on:
pull_request:
types: [opened, ready_for_review]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Workflow is not re-run when new commits are pushed, so the AI review comment can fall out-of-date.

Prompt for AI agents
Address the following comment on .github/workflows/continue-review.yaml at line 5:

<comment>Workflow is not re-run when new commits are pushed, so the AI review comment can fall out-of-date.</comment>

<file context>
@@ -0,0 +1,164 @@
+name: Continue CLI Code Review
+
+on:
+  pull_request:
+    types: [opened, ready_for_review]
+
+permissions:
</file context>

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In my experience. Only "ready to review" is great to limit credit spend, but I don't think contributors will start with a draft. So makes sense to have this as opened, but probably makes sense"ready to review" unnecessary

issue_comment:
types: [created]

permissions:
contents: read
pull-requests: write
issues: write

jobs:
code-review:
name: AI Code Review
runs-on: ubuntu-latest
timeout-minutes: 10
# Only run if:
# - It's a PR event from a team member (with write/admin permissions)
# - OR it's a comment with @continue-review on a PR from a team member
if: |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using the literal block indicator | preserves the newline characters, so the if expression is passed to the runner with embedded new-lines. GitHub Actions fails to parse multi-line if expressions containing newlines, causing the job to be skipped or to error. Use the folded style (>) so the expression is evaluated as a single line.

Prompt for AI agents
Address the following comment on .github/workflows/continue-review.yaml at line 22:

<comment>Using the literal block indicator `|` preserves the newline characters, so the `if` expression is passed to the runner with embedded new-lines. GitHub Actions fails to parse multi-line `if` expressions containing newlines, causing the job to be skipped or to error. Use the folded style (`&gt;`) so the expression is evaluated as a single line.</comment>

<file context>
@@ -0,0 +1,172 @@
+name: Continue CLI Code Review
+
+on:
+  pull_request:
+    types: [opened, ready_for_review]
+  issue_comment:
+    types: [created]
+
+permissions:
</file context>

(github.event_name == 'pull_request' &&
(github.event.pull_request.author_association == 'OWNER' ||
github.event.pull_request.author_association == 'MEMBER' ||
github.event.pull_request.author_association == 'COLLABORATOR')) ||
(github.event_name == 'issue_comment' &&
github.event.issue.pull_request &&
contains(github.event.comment.body, '@continue-review') &&
(github.event.comment.author_association == 'OWNER' ||
github.event.comment.author_association == 'MEMBER' ||
github.event.comment.author_association == 'COLLABORATOR'))

steps:
- name: Checkout Repository
uses: actions/checkout@v4
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

External actions are referenced only by version tags (e.g., actions/checkout@v4) instead of immutable commit SHAs. Tag drift can introduce unreviewed changes or malicious code into your workflow and is discouraged by GitHub’s security guidelines.

(Based on your team's feedback about pinning third-party GitHub Actions to commit SHAs for supply-chain security.)

Prompt for AI agents
Address the following comment on .github/workflows/continue-review.yaml at line 36:

<comment>External actions are referenced only by version tags (e.g., `actions/checkout@v4`) instead of immutable commit SHAs. Tag drift can introduce unreviewed changes or malicious code into your workflow and is discouraged by GitHub’s security guidelines.

(Based on your team&#39;s feedback about pinning third-party GitHub Actions to commit SHAs for supply-chain security.)</comment>

<file context>
@@ -0,0 +1,172 @@
+name: Continue CLI Code Review
+
+on:
+  pull_request:
+    types: [opened, ready_for_review]
+  issue_comment:
+    types: [created]
+
+permissions:
</file context>

with:
fetch-depth: 0 # Fetch full history for better context

- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: 20

- name: Install Continue CLI
run: npm install -g @continuedev/[email protected]
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we want to pin to this CLI version?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, that's by design. I think we should pretty much always do this when using a dependency with ad-hoc npm install. Otherwise, we may end up using a version with a breaking change, or even a version with a security issue.


- name: Build PR Review Prompt
run: |
# Get PR number based on event type
if [ "${{ github.event_name }}" = "pull_request" ]; then
PR_NUMBER="${{ github.event.number }}"
else
# For issue_comment event on a PR
PR_NUMBER="${{ github.event.issue.number }}"
fi

# Get PR diff
gh pr diff $PR_NUMBER > pr_diff.txt

# Create review prompt
cat > review_prompt.txt << 'EOF'
You are conducting a code review for a pull request. Below is the git diff showing all the changes:

EOF

echo "--- PR DIFF START ---" >> review_prompt.txt
cat pr_diff.txt >> review_prompt.txt
echo "--- PR DIFF END ---" >> review_prompt.txt

cat >> review_prompt.txt << 'EOF'

Please analyze these changes and provide a comprehensive code review. Consider:

1. **Code Quality**: Are there any bugs, performance issues, or code smells?
2. **Best Practices**: Does the code follow established patterns and conventions?
3. **Security**: Are there any potential security vulnerabilities?
4. **Testing**: Are appropriate tests included or updated?
5. **Documentation**: Is documentation adequate for the changes?
6. **Architecture**: Do the changes fit well with the existing codebase structure?
Comment on lines +75 to +80
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since we have the repo cloned, will the agent be following our rules in the .continue folder? If not, could we just cat them into the context window or something?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought about this but we have a lot of different rules with different triggers, we'd have to either put all the rules in the context (noisy), or handle rule resolution for each file individually (complicated). I'm not against it but it feels out of the scope of this PR.


You can use the available tools to explore the codebase and understand context better.

Format your response as a markdown code review with the following structure:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My hunch is that this will get annoying with it following the exact structure even when unnecessary. Might be helpful to add a comment like "You do not need to follow this exact structure, use it for inspiration while still keeping your review concise and focused."

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I made it more rigid on purpose - the odds are most people won't read this thoroughly but will skim it. The more consistent the structure, the easier it will be to find the exact data you need.


## Code Review Summary

### ✅ Strengths
- **[Aspect]**: [Description of what was done well]

### ⚠️ Issues Found

Only include the severity subheaders below if you actually found issues at that level:

#### Critical
- **[Issue Title]**: [Description of critical issues that must be fixed before merging]

#### High
- **[Issue Title]**: [Description of high-priority issues that should be fixed]

#### Medium
- **[Issue Title]**: [Description of issues that should be addressed]

#### Low
- **[Issue Title]**: [Description of minor issues or nice-to-have improvements]


### 💡 Suggestions
- **[Suggestion Title]**: [Description of improvement recommendations]

### 🚀 Overall Assessment
[Provide overall recommendation: APPROVE, REQUEST_CHANGES, or COMMENT]

Only call the 'exit' tool if you find critical security vulnerabilities or bugs that would break production.
EOF
env:
GH_TOKEN: ${{ github.token }}

- name: Run Continue CLI Review
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The step executes unconditionally although its required input file (review_prompt.txt) is only created for non-draft PRs, so draft PRs will make the workflow fail. Add the same condition (if: github.event.pull_request.draft == false) or check for the file’s existence before reading it.

Prompt for AI agents
Address the following comment on .github/workflows/continue-review.yaml at line 82:

<comment>The step executes unconditionally although its required input file (`review_prompt.txt`) is only created for non-draft PRs, so draft PRs will make the workflow fail. Add the same condition (`if: github.event.pull_request.draft == false`) or check for the file’s existence before reading it.</comment>

<file context>
@@ -0,0 +1,137 @@
+name: Continue CLI Code Review
+
+on:
+  pull_request:
+    types: [opened, ready_for_review]
+
+permissions:
+  contents: read
+  pull-requests: write
</file context>

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Run Continue CLI Review executes even when the PR is still a draft, but review_prompt.txt is only created for non-draft PRs, causing the job to fail in draft mode.

Prompt for AI agents
Address the following comment on .github/workflows/continue-review.yaml at line 81:

<comment>Run Continue CLI Review executes even when the PR is still a draft, but review_prompt.txt is only created for non-draft PRs, causing the job to fail in draft mode.</comment>

<file context>
@@ -0,0 +1,136 @@
+name: Continue CLI Code Review
+
+on:
+  pull_request:
+    types: [opened, ready_for_review]
+
+permissions:
+  contents: read
+  pull-requests: write
</file context>

run: |
echo "Running Continue CLI with prompt:"
echo "=================================="
cat review_prompt.txt
echo "=================================="
echo ""

# Run the CLI with hardcoded assistant and pipe output to code_review.md
cat review_prompt.txt | cn --readonly --org continuedev --config continuedev/review-bot -p > code_review.md
env:
CONTINUE_API_KEY: ${{ secrets.CONTINUE_API_KEY }}

- name: Upload Review Results
uses: actions/upload-artifact@v4
if: always()
with:
name: code-review-results
path: |
code_review.md
review_prompt.txt
pr_diff.txt
retention-days: 30

- name: Comment PR with Review
if: always()
uses: actions/github-script@v7
with:
script: |
const fs = require('fs');

try {
let reviewContent = '';

if (fs.existsSync('code_review.md') && fs.statSync('code_review.md').size > 0) {
reviewContent = fs.readFileSync('code_review.md', 'utf8');
} else {
reviewContent = '⚠️ AI review completed but no review output was generated. Check the action logs for details.';
}

// Get PR number based on event type
let prNumber;
if (context.eventName === 'pull_request') {
prNumber = context.payload.pull_request.number;
} else {
// For issue_comment event
prNumber = context.payload.issue.number;
}

// Add a header if triggered by comment
if (context.eventName === 'issue_comment') {
reviewContent = `*Triggered by @${context.payload.comment.user.login}'s request*\n\n${reviewContent}`;
}

// Create new comment
await github.rest.issues.createComment({
issue_number: prNumber,
owner: context.repo.owner,
repo: context.repo.repo,
body: reviewContent
});
console.log(`Successfully created new comment on PR #${prNumber}`);
} catch (error) {
console.log('Failed to post comment:', error.message);
console.log('Error details:', error);
}
env:
GITHUB_TOKEN: ${{ github.token }}
Loading