How to efficiently skip unnecessary jobs in a monorepo GitHub Actions workflow?

[Litis-Trion250]

Why are you starting this discussion?

Question

What GitHub Actions topic or product is this about?

Workflow Deployment

Discussion Details

We have a large monorepo that uses GitHub Actions for CI/CD. The workflow currently runs all jobs (e.g., build-frontend, test-backend, deploy-docs) on every push to main, even if only a single, unrelated project directory has changed.This is causing our CI/CD pipeline to take an excessive amount of time and consume too many minutes.Goal: I need a robust and efficient way to conditionally skip specific jobs in the workflow based only on which directories have changed in the current commit or pull request.For example, if only files in the docs/ directory have changed, only the deploy-docs job should run, and all other jobs like build-frontend and test-backend should be skipped.What is the recommended, best-practice approach to implement this file-path-based conditional job execution in a GitHub Actions workflow for a monorepo?

[Rocky-Dewan]

This is a very common and critical challenge in monorepos. The best and most efficient solution is to use the paths-filter action, which leverages the git diff command under the hood to determine the changed files and set output variables that can be used in the if condition of your jobs.Here is a step-by-step guide and the YAML implementation:1. Use the paths-filter Action

The dorny/paths-filter action is the industry standard for this use case. It allows you to define a map of paths and will output a boolean (true or false) for each map key, indicating if any files in that path have changed.2. Implementation in Workflow YAML

You will add a new job (or step in an existing job) to calculate the changes, and then use its output in the if condition of your other jobs.

name: Monorepo CI
on:
  push:
    branches:
      - main
  pull_request:
    branches:
      - main

jobs:
  # 1. This job calculates which paths have changed
  filter-paths:
    runs-on: ubuntu-latest
    outputs:
      frontend: ${{ steps.filter.outputs.frontend }}
      backend: ${{ steps.filter.outputs.backend }}
      docs: ${{ steps.filter.outputs.docs }}
    steps:
    - uses: actions/checkout@v4
    - uses: dorny/paths-filter@v3
      id: filter
      with:
        # Define the mapping of paths to output variables
        filters: |
          frontend:
            - 'packages/frontend/**'
          backend:
            - 'packages/backend/**'
          docs:
            - 'docs/**'

  # 2. Conditional job for the frontend
  build-frontend:
    needs: filter-paths
    if: needs.filter-paths.outputs.frontend == 'true'
    runs-on: ubuntu-latest
    steps:
      # ... build steps ...
      - name: Frontend Build
        run: echo "Building frontend..."

  # 3. Conditional job for the backend
  test-backend:
    needs: filter-paths
    if: needs.filter-paths.outputs.backend == 'true'
    runs-on: ubuntu-latest
    steps:
      # ... test steps ...
      - name: Backend Test
        run: echo "Testing backend..."

  # 4. Conditional job for the documentation
  deploy-docs:
    needs: filter-paths
    if: needs.filter-paths.outputs.docs == 'true'
    runs-on: ubuntu-latest
    steps:
      # ... deployment steps ...
      - name: Deploy Docs
        run: echo "Deploying documentation..."

Key Takeaway

The critical part is the needs: filter-paths and the if: needs.filter-paths.outputs.YOUR_KEY == 'true' condition on the subsequent jobs. This ensures the job is only queued and executed if a change was detected in the specified path, drastically reducing your CI/CD time and cost.

[anilgoswamistartbitsolutions]

If each workflow YAML corresponds to a project (frontend, backend, docs, etc.), the simplest method is to split workflows and scope their triggers.

Example:
`# .github/workflows/frontend.yml
on:
push:
branches: [main]
paths:
- 'frontend/'
pull_request:
paths:
- 'frontend/'

jobs:
build-frontend:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- run: npm ci && npm run build

Then another workflow for backend:

.github/workflows/backend.yml

on:
push:
paths:
- 'backend/**'`