-
Notifications
You must be signed in to change notification settings - Fork 3.7k
Continuous ai review bot #7099
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Continuous ai review bot #7099
Changes from 12 commits
1dbba32
1bc7791
32a8eed
b04c6f6
3de7baa
82c7cc2
2b384e7
8b2686c
ae02893
7dec41f
f15c140
7c2cb65
a7e2d6f
e623242
45c19fa
fafea05
9150b6f
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,137 @@ | ||
| name: Continue CLI Code Review | ||
|
|
||
| on: | ||
| pull_request: | ||
| types: [opened, ready_for_review] | ||
|
|
||
| permissions: | ||
| contents: read | ||
| pull-requests: write | ||
| issues: write | ||
|
|
||
| jobs: | ||
| code-review: | ||
| name: AI Code Review | ||
| runs-on: ubuntu-latest | ||
| timeout-minutes: 10 | ||
|
|
||
| steps: | ||
| - name: Checkout Repository | ||
| uses: actions/checkout@v4 | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. External actions are referenced only by version tags (e.g., (Based on your team's feedback about pinning third-party GitHub Actions to commit SHAs for supply-chain security.) Prompt for AI agents |
||
| with: | ||
| fetch-depth: 0 # Fetch full history for better context | ||
|
|
||
| - name: Setup Node.js | ||
| uses: actions/setup-node@v4 | ||
| with: | ||
| node-version: 20 | ||
|
|
||
| - name: Install Continue CLI | ||
| run: npm install -g @continuedev/[email protected] | ||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Do we want to pin to this CLI version?
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yeah, that's by design. I think we should pretty much always do this when using a dependency with ad-hoc npm install. Otherwise, we may end up using a version with a breaking change, or even a version with a security issue. |
||
|
|
||
| - name: Build PR Review Prompt | ||
| if: github.event.pull_request.draft == false | ||
| run: | | ||
| # Get PR diff | ||
| gh pr diff ${{ github.event.number }} > pr_diff.txt | ||
|
|
||
| # Create review prompt | ||
| cat > review_prompt.txt << 'EOF' | ||
| You are conducting a code review for a pull request. Below is the git diff showing all the changes: | ||
|
|
||
| EOF | ||
|
|
||
| echo "--- PR DIFF START ---" >> review_prompt.txt | ||
| cat pr_diff.txt >> review_prompt.txt | ||
| echo "--- PR DIFF END ---" >> review_prompt.txt | ||
|
|
||
| cat >> review_prompt.txt << 'EOF' | ||
|
|
||
| Please analyze these changes and provide a comprehensive code review. Consider: | ||
|
|
||
| 1. **Code Quality**: Are there any bugs, performance issues, or code smells? | ||
| 2. **Best Practices**: Does the code follow established patterns and conventions? | ||
| 3. **Security**: Are there any potential security vulnerabilities? | ||
| 4. **Testing**: Are appropriate tests included or updated? | ||
| 5. **Documentation**: Is documentation adequate for the changes? | ||
| 6. **Architecture**: Do the changes fit well with the existing codebase structure? | ||
|
Comment on lines
+75
to
+80
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Since we have the repo cloned, will the agent be following our rules in the
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I thought about this but we have a lot of different rules with different triggers, we'd have to either put all the rules in the context (noisy), or handle rule resolution for each file individually (complicated). I'm not against it but it feels out of the scope of this PR. |
||
|
|
||
| You can use the available tools to explore the codebase and understand context better. | ||
|
|
||
| Format your response as a markdown code review with the following structure: | ||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. My hunch is that this will get annoying with it following the exact structure even when unnecessary. Might be helpful to add a comment like "You do not need to follow this exact structure, use it for inspiration while still keeping your review concise and focused."
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I made it more rigid on purpose - the odds are most people won't read this thoroughly but will skim it. The more consistent the structure, the easier it will be to find the exact data you need. |
||
|
|
||
| ## Code Review Summary | ||
|
|
||
| ### ✅ Strengths | ||
| - [List positive aspects] | ||
|
|
||
| ### ⚠️ Issues Found | ||
| - [List any problems with severity levels] | ||
|
|
||
| ### 💡 Suggestions | ||
| - [List improvement recommendations] | ||
|
|
||
| ### 🚀 Overall Assessment | ||
| [Provide overall recommendation: APPROVE, REQUEST_CHANGES, or COMMENT] | ||
|
|
||
| Only call the 'exit' tool if you find critical security vulnerabilities or bugs that would break production. | ||
| EOF | ||
| env: | ||
| GH_TOKEN: ${{ github.token }} | ||
|
|
||
| - name: Run Continue CLI Review | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The step executes unconditionally although its required input file ( Prompt for AI agents
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Run Continue CLI Review executes even when the PR is still a draft, but review_prompt.txt is only created for non-draft PRs, causing the job to fail in draft mode. Prompt for AI agents |
||
| run: | | ||
| echo "Running Continue CLI with prompt:" | ||
| echo "==================================" | ||
| cat review_prompt.txt | ||
| echo "==================================" | ||
| echo "" | ||
|
|
||
| # Run the CLI with hardcoded assistant and pipe output to code_review.md | ||
| cat review_prompt.txt | cn --auto --org continuedev --config continuedev/review-bot -p > code_review.md | ||
|
||
| env: | ||
| CONTINUE_API_KEY: ${{ secrets.CONTINUE_API_KEY }} | ||
|
|
||
| - name: Upload Review Results | ||
| uses: actions/upload-artifact@v4 | ||
| if: always() | ||
| with: | ||
| name: code-review-results | ||
| path: | | ||
| code_review.md | ||
| review_prompt.txt | ||
| pr_diff.txt | ||
| retention-days: 30 | ||
|
|
||
| - name: Comment PR with Review | ||
| if: always() | ||
| uses: actions/github-script@v7 | ||
| with: | ||
| script: | | ||
| const fs = require('fs'); | ||
|
|
||
| try { | ||
| let reviewContent = ''; | ||
|
|
||
| if (fs.existsSync('code_review.md') && fs.statSync('code_review.md').size > 0) { | ||
| reviewContent = fs.readFileSync('code_review.md', 'utf8'); | ||
| } else { | ||
| reviewContent = '⚠️ AI review completed but no review output was generated. Check the action logs for details.'; | ||
| } | ||
|
|
||
| const prNumber = context.payload.pull_request.number; | ||
|
|
||
| // Create new comment | ||
| await github.rest.issues.createComment({ | ||
| issue_number: prNumber, | ||
| owner: context.repo.owner, | ||
| repo: context.repo.repo, | ||
| body: reviewContent | ||
| }); | ||
| console.log(`Successfully created new comment on PR #${prNumber}`); | ||
| } catch (error) { | ||
| console.log('Failed to post comment:', error.message); | ||
| console.log('Error details:', error); | ||
| } | ||
| env: | ||
| GITHUB_TOKEN: ${{ github.token }} | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Workflow is not re-run when new commits are pushed, so the AI review comment can fall out-of-date.
Prompt for AI agents
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In my experience. Only "ready to review" is great to limit credit spend, but I don't think contributors will start with a draft. So makes sense to have this as opened, but probably makes sense"ready to review" unnecessary