Skip to content

Use GitHub App tokens in Claude workflows#2936

Merged
wwwillchen merged 2 commits intodyad-sh:mainfrom
wwwillchen:chore/github-app-workflows
Mar 7, 2026
Merged

Use GitHub App tokens in Claude workflows#2936
wwwillchen merged 2 commits intodyad-sh:mainfrom
wwwillchen:chore/github-app-workflows

Conversation

@wwwillchen
Copy link
Copy Markdown
Collaborator

@wwwillchen wwwillchen commented Mar 6, 2026

Summary

  • switch the listed Claude automation workflows from PAT/default token auth to GitHub App installation tokens
  • use fork-scoped app tokens for PR responder and rebase push paths, plus base-repo app tokens for labels/comments/API calls
  • rename the Actions variable and secret references to DYAD_GITHUB_APP_ID and DYAD_GITHUB_APP_PRIVATE_KEY

Test plan

  • npm run fmt
  • npm run lint:fix
  • npm run ts

πŸ€– Generated with Claude Code


Open with Devin

@wwwillchen wwwillchen requested a review from a team March 6, 2026 23:34
@gemini-code-assist
Copy link
Copy Markdown
Contributor

Note

Gemini is unable to generate a summary for this pull request due to the file types involved not being currently supported.

Copy link
Copy Markdown
Contributor

@devin-ai-integration devin-ai-integration bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

βœ… Devin Review: No Issues Found

Devin Review analyzed this PR and found no potential bugs to report.

View in Devin Review to see 4 additional findings.

Open in Devin Review

@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps bot commented Mar 6, 2026

Greptile Summary

This PR migrates all listed Claude automation workflows from PAT/default GITHUB_TOKEN authentication to GitHub App installation tokens, enabling fork-scoped pushes that trigger real PR synchronize events (unlike pushes with GITHUB_TOKEN, which GitHub silently suppresses). The dual-token pattern β€” a base-repo token for labels/comments/API calls and a fork-scoped head-repo token for pushing β€” is a well-structured approach and the implementation is largely correct across the 8 changed workflows.

Key changes and observations:

  • All workflows now use actions/create-github-app-token@v2 with fine-grained permission-* inputs scoped to what each job actually needs.
  • claude-rebase.yml and pr-review-responder.yml correctly split token creation into a base-repo token and a conditional fork-scoped head-repo token.
  • pr-review-responder.yml and claude-rebase.yml properly wire github-token into actions/github-script steps.
  • claude-pr-review.yml drops permission-contents: read from the app token β€” the previous GITHUB_TOKEN had this via the job-level permissions block, and the Claude Code action's skill may invoke GitHub API content endpoints that require it.
  • claude-deflake-e2e.yml passes the full-scope app token to npm ci which typically only needs package-download access (or no token at all for a public repo).
  • The previously used PAT secrets (DEFLAKE_E2E_PAT_GITHUB_TOKEN, PR_CONTENTS_RW_GITHUB_TOKEN) are now unused across all workflows and should be revoked from repository settings to reduce secret sprawl.

Confidence Score: 4/5

  • PR is safe to merge with minor permission gaps that may cause subtle API failures in the PR review workflow.
  • The core token migration logic is correct and the dual-token pattern for fork-scoped pushes is sound. The main concern is the missing permission-contents: read in claude-pr-review.yml, which could cause 403 errors on content API calls but won't break the workflow completely. The deflake workflow's over-privileged token for npm ci is a best-practice issue, not a functional one.
  • claude-pr-review.yml (missing contents:read permission) and claude-deflake-e2e.yml (over-privileged token for npm ci step).

Important Files Changed

Filename Overview
.github/workflows/claude-deflake-e2e.yml Adds broad-scoped app token (actions/contents/issues/pull-requests: write) replacing PAT in npm ci, GH_TOKEN, and github_token. App token scope is wider than needed for npm ci step.
.github/workflows/claude-pr-review.yml App token only requests permission-pull-requests: write, dropping the contents: read that job-level GITHUB_TOKEN previously provided. Could cause 403s on GitHub API content endpoints used by the pr-review skill.
.github/workflows/claude-rebase.yml Correctly splits into base-app-token (contents/pull-requests: write) and fork-scoped head-app-token (contents: write). Conditional creation of head token and correct env mapping.
.github/workflows/pr-review-responder.yml Correctly introduces dual-token pattern (base-app-token for labels/comments/API, head-app-token for fork push). github-token wired to github-script steps. All GITHUB_TOKEN substitutions look correct.

Sequence Diagram

sequenceDiagram
    participant GH as GitHub Actions Runner
    participant AppSvc as create-github-app-token
    participant Base as Base Repo (dyad-sh/dyad)
    participant Fork as Fork Repo (contributor)
    participant Claude as claude-code-action

    GH->>AppSvc: Request base-repo installation token<br/>(contents:write, pull-requests:write)
    AppSvc-->>GH: base token

    GH->>AppSvc: Request fork-scoped installation token<br/>(owner=fork owner, contents:write)
    AppSvc-->>Fork: Authenticate via GitHub App
    Fork-->>AppSvc: Grant token
    AppSvc-->>GH: fork token

    GH->>Base: Checkout repository (default GITHUB_TOKEN)
    GH->>GH: Configure git push URL<br/>to use fork token

    GH->>Claude: Run skill<br/>(github_token = base token)
    Claude->>Base: PR labels, comments, API calls
    Claude->>Fork: git push via configured push URL
    Fork-->>GH: Triggers PR synchronize event
    Note over Fork,GH: CI and downstream workflows fire normally
Loading

Last reviewed commit: 7668462

@wwwillchen
Copy link
Copy Markdown
Collaborator Author

@BugBot run

Comment on lines +211 to +219
- name: Create GitHub App token for PR head repo
if: steps.pr-info.outputs.should_continue == 'true'
id: head-app-token
uses: actions/create-github-app-token@v2
with:
app-id: ${{ vars.DYAD_GITHUB_APP_ID }}
private-key: ${{ secrets.DYAD_GITHUB_APP_PRIVATE_KEY }}
owner: ${{ github.event_name == 'pull_request_target' && github.event.pull_request.head.repo.owner.login || github.event.workflow_run.head_repository.owner.login }}
repositories: ${{ github.event_name == 'pull_request_target' && github.event.pull_request.head.repo.name || github.event.workflow_run.head_repository.name }}
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

GitHub App must be installed on all allowed fork repositories

The Create GitHub App token for PR head repo step requests an installation token scoped to the contributor's fork repository (using the owner + repositories parameters). This step will fail with a 404 error if the GitHub App (DYAD_GITHUB_APP_ID) is not installed on the fork.

This is a critical operational prerequisite for all allowed PR authors (wwwillchen, wwwillchen-bot, dyadbot, princeaden1). If any of their forks lack the App installation, this step will fail and the entire workflow will abort β€” even though the earlier should_continue check was true.

The same concern applies to claude-rebase.yml at its equivalent step (lines 43–51). Consider either:

  1. Adding a fallback or graceful error handler for missing App installations, or
  2. Documenting that the GitHub App must be pre-installed on all contributor fork repos as a required setup step before this workflow can function.
Prompt To Fix With AI
This is a comment left during a code review.
Path: .github/workflows/pr-review-responder.yml
Line: 211-219

Comment:
**GitHub App must be installed on all allowed fork repositories**

The `Create GitHub App token for PR head repo` step requests an installation token scoped to the contributor's fork repository (using the `owner` + `repositories` parameters). This step will fail with a 404 error if the GitHub App (`DYAD_GITHUB_APP_ID`) is not installed on the fork.

This is a critical operational prerequisite for all allowed PR authors (`wwwillchen`, `wwwillchen-bot`, `dyadbot`, `princeaden1`). If any of their forks lack the App installation, this step will fail and the entire workflow will abort β€” even though the earlier `should_continue` check was `true`.

The same concern applies to `claude-rebase.yml` at its equivalent step (lines 43–51). Consider either:
1. Adding a fallback or graceful error handler for missing App installations, or
2. Documenting that the GitHub App must be pre-installed on all contributor fork repos as a required setup step before this workflow can function.

How can I resolve this? If you propose a fix, please make it concise.

cubic-dev-ai[bot]

This comment was marked as resolved.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Mar 6, 2026

πŸ” Dyadbot Code Review Summary

Verdict: πŸ€” NOT SURE - Potential issues

Reviewed by 3 independent agents: Correctness Expert, Code Health Expert, UX Wizard.

Issues Summary

Severity File Issue
🟑 MEDIUM .github/workflows/claude-check-workflows.yml:31-35 Workflow missed from migration

🟑 MEDIUM: claude-check-workflows.yml was not migrated to GitHub App tokens

This workflow still references secrets.DEFLAKE_E2E_PAT_GITHUB_TOKEN for both GH_TOKEN (line 31) and github_token (line 35). Every other Claude workflow was converted in this PR, but this one was missed. If the old PAT is rotated or removed as part of this migration, this workflow will break silently.

Suggestion: Add the same actions/create-github-app-token@v2 step and replace both DEFLAKE_E2E_PAT_GITHUB_TOKEN references with steps.app-token.outputs.token, matching the pattern used in claude-deflake-e2e.yml.

🟒 Low Priority Notes (2 items)
  • Fork app installation requirement β€” pr-review-responder.yml and claude-rebase.yml: The head-app-token step requires the GitHub App to be installed on each fork. Unlike the previous PAT, this will fail if the app isn't installed on a contributor's fork. Since workflows are restricted to a small trusted user list, this is likely intentional but worth documenting.
  • Token creation block duplicated across 7 files β€” The same 6-line token creation step is copy-pasted in 7 workflows (9 total with fork-scoped variants). Consider extracting a reusable composite action if this pattern grows further.
🚫 Dropped False Positives (3 items)
  • Fork app token failure leaves PR labels stuck β€” Dropped: The base-app-token is created unconditionally (no if guard), and all label cleanup/failure steps use base-app-token, not head-app-token. Labels will be cleaned up correctly even if the head-app-token step fails.
  • Rebase workflow fails silently for forks β€” Dropped: The failure() handler runs using base-app-token and adds cc:rebase-failed, providing user feedback.
  • App token expiration during long workflows β€” Dropped: GitHub App tokens last 1 hour; these workflows complete well within that window.

Generated by Dyadbot multi-agent code review

@wwwillchen
Copy link
Copy Markdown
Collaborator Author

@BugBot run

Copy link
Copy Markdown

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

βœ… Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

πŸ’‘ Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 766846250d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with πŸ‘.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

private-key: ${{ secrets.DYAD_GITHUB_APP_PRIVATE_KEY }}
owner: ${{ github.event_name == 'pull_request_target' && github.event.pull_request.head.repo.owner.login || github.event.workflow_run.head_repository.owner.login }}
repositories: ${{ github.event_name == 'pull_request_target' && github.event.pull_request.head.repo.name || github.event.workflow_run.head_repository.name }}
permission-contents: write
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Add workflows permission to head-repo app token

This token is used for the git push path, but it only requests permission-contents: write; GitHub rejects pushes that modify .github/workflows/* unless the credential also has workflow-write permission. In PRs where /dyad:pr-fix updates workflow files, the push step will fail and the automation loop cannot complete, so this token should also request permission-workflows: write (the same risk applies to the analogous head-token setup in claude-rebase.yml).

Useful? React with πŸ‘Β / πŸ‘Ž.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Mar 6, 2026

πŸ” Dyadbot Code Review Summary

Verdict: βœ… YES - Ready to merge

Reviewed by 3 independent agents: Correctness Expert, Code Health Expert, UX Wizard.

βœ… No new issues found by multi-agent review.

The token migration is clean and consistent across all 8 workflow files. The dual-token pattern (base-repo + fork-scoped) in claude-rebase.yml and pr-review-responder.yml is correctly implemented with proper guards. Permission scoping on each app token follows least-privilege.

🚫 Dropped False Positives (5 items)
  • head-app-token expression edge case with null head.repo β€” Dropped: The should_continue guard in pr-info correctly sets false when headRepo is null, and the head-app-token step is gated on should_continue == 'true'. The expression cannot reach a state where both sides of the || are empty.
  • Triage app token missing contents:read β€” Dropped: The triage workflow only runs gh issue and gh search commands via allowed_tools. Contents are accessed through local checkout using the default token, not the app token.
  • permissions: {} misleading in deflake-e2e β€” Dropped: Restricting the default GITHUB_TOKEN to no permissions while using an explicitly-scoped App token is correct defense-in-depth. The app token permissions are clearly declared in the step.
  • Vestigial GITHUB_TOKEN permissions blocks β€” Dropped: Keeping restrictive permissions: on the default token is good practice even when it's not actively used β€” it limits blast radius if someone later adds a step that inadvertently uses it.
  • Bot identity change affects UX β€” Dropped: PR comments/labels will now appear from the GitHub App instead of the PAT user. This is intentional and expected behavior for the migration.

Generated by Dyadbot multi-agent code review

Copy link
Copy Markdown

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

βœ… Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

@github-actions github-actions bot added the needs-human:review-issue ai agent flagged an issue that requires human review label Mar 6, 2026
@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Mar 7, 2026

🎭 Playwright Test Results

❌ Some tests failed

OS Passed Failed Flaky Skipped
🍎 macOS 249 1 9 6

Summary: 249 passed, 1 failed, 9 flaky, 6 skipped

Failed Tests

🍎 macOS

  • annotator.spec.ts > annotator - capture and submit screenshot
    • TimeoutError: locator.click: Timeout 120000ms exceeded.

πŸ“‹ Re-run Failing Tests (macOS)

Copy and paste to re-run all failing spec files locally:

npm run e2e \
  e2e-tests/annotator.spec.ts

⚠️ Flaky Tests

🍎 macOS

  • edit_code.spec.ts > edit code edits the right file (passed after 1 retry)
  • fix_error.spec.ts > fix error with AI (passed after 1 retry)
  • refresh.spec.ts > refresh preserves current route (passed after 1 retry)
  • select_component.spec.ts > select component next.js (passed after 1 retry)
  • setup_flow.spec.ts > Setup Flow > setup banner shows correct state when node.js is installed (passed after 1 retry)
  • setup.spec.ts > setup ai provider (passed after 1 retry)
  • switch_versions.spec.ts > switch versions (isomorphic git) (passed after 1 retry)
  • toggle_screen_sizes.spec.ts > Toggle Screen Size Tests > should persist device mode after rebuild (passed after 1 retry)
  • undo.spec.ts > undo (passed after 1 retry)

πŸ“Š View full report

@wwwillchen wwwillchen merged commit 51fc07e into dyad-sh:main Mar 7, 2026
11 of 12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

needs-human:review-issue ai agent flagged an issue that requires human review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant