Skip to content

Commit 8cbcd19

Browse files
benfredbenfred
authored
Add a script to check for breaking C ABI changes (#1749)
This adds a script that can check for changes that would break our C-ABI. It works by comparing two sets of header files: the `c/include` header files for the published ABI, as well as the `c/include` header files inside a new pull request. Each set of header files is parsed using libclang, and the differences between the old and new header files are examined for changes that would cause a breaking ABI change. Currently the following checks are made and flagged by this tool: * Functions that have been removed from the C-ABI * Functions that have extra parameters added * Functions that have parameters removed * Functions that have the type of any parameter changed * Structs that have been removed * Structs that have have members removed * Structs that have the types of members changed * Enum values that have been removed * Enum values that have their value changed This is just currently a POC of using libclang to flag breaking c-abi changes - and is meant as an alternative to tools that require debug symbols for detecting abi breaks from the compiled library, and isn't ready to merge just yet. To fully get this integrated, we will need to store the headers for the stable c-abi to use as a baseline somewhere, and then download the stable c-abi headers on each PR and use them to run in a GHA workflow to flag breaking changes. When run on a branch that has a breaking ABI change (#1496 which has a number of breaking c-abi changes), the output is: ``` $ ./check_c_abi.py ~/code/cuvs_stable_abi/c/include ~/code/cuvs/c/include Error: Function has been removed. Symbol cuvsCagraMergeParamsCreate from /home/ben/code/cuvs_stable_abi/c/include/cuvs/neighbors/cagra.h:584 Error: Function has been removed. Symbol cuvsCagraMergeParamsDestroy from /home/ben/code/cuvs_stable_abi/c/include/cuvs/neighbors/cagra.h:587 Error: Function has a changed argument type 'cuvsCagraMergeParams_t' to 'cuvsCagraIndexParams_t for argument 'params'. Symbol cuvsCagraMerge from /home/ben/code/cuvs/c/include/cuvs/neighbors/cagra.h:882 Error: Function has a changed argument type 'cuvsCagraIndex_t' to 'cuvsFilter for argument 'output_index'. Symbol cuvsCagraMerge from /home/ben/code/cuvs/c/include/cuvs/neighbors/cagra.h:882 Error: Function has a new argument 'cuvsCagraIndex_t output_index'. Symbol cuvsCagraMerge from /home/ben/code/cuvs/c/include/cuvs/neighbors/cagra.h:882 Error: Struct has been removed. Symbol cuvsCagraMergeParams from /home/ben/code/cuvs_stable_abi/c/include/cuvs/neighbors/cagra.h:576 ``` Also, this script runs in < 100ms on my workstation, and we could potentially add this as a pre-commit hook closes #1739 Authors: - Ben Frederickson (https://github.com/benfred) - Mike Sarahan (https://github.com/msarahan) Approvers: - Mike Sarahan (https://github.com/msarahan) - Kyle Edwards (https://github.com/KyleFromNVIDIA) URL: #1749
1 parent 48e9950 commit 8cbcd19

11 files changed

Lines changed: 952 additions & 0 deletions

File tree

‎.github/workflows/check-c-abi.yaml‎

Lines changed: 131 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,131 @@
1+
name: C ABI Compatibility Check
2+
3+
on:
4+
workflow_call:
5+
6+
jobs:
7+
# Check PRs for breaking ABI changes
8+
check-pr:
9+
runs-on: ubuntu-latest
10+
steps:
11+
- name: Checkout PR branch
12+
uses: actions/checkout@v4
13+
with:
14+
fetch-depth: 0
15+
16+
- name: Set up Python
17+
uses: actions/setup-python@v5
18+
with:
19+
python-version: '3.11'
20+
21+
- name: Install Python dependencies
22+
run: |
23+
pip install --upgrade pip
24+
pip install ci/check_c_abi
25+
26+
- name: Get dlpack dependency
27+
run: |
28+
git clone https://github.com/dmlc/dlpack
29+
30+
- name: Find merge base commit
31+
id: merge-base
32+
run: |
33+
git fetch origin main
34+
MERGE_BASE=$(git merge-base HEAD origin/main)
35+
echo "merge_base_sha=${MERGE_BASE}" >> $GITHUB_OUTPUT
36+
echo "Merge base commit: ${MERGE_BASE}"
37+
38+
- name: Try to download baseline for merge-base commit (most accurate)
39+
id: download-merge-base
40+
continue-on-error: true
41+
uses: dawidd6/action-download-artifact@8a338493df3d275e4a7a63bcff3b8fe97e51a927 # v19
42+
with:
43+
name: c-abi-baseline-${{ steps.merge-base.outputs.merge_base_sha }}
44+
workflow: check-c-abi.yaml
45+
commit: ${{ steps.merge-base.outputs.merge_base_sha }}
46+
path: baseline/
47+
48+
- name: Try to download latest main baseline (fallback 1)
49+
id: download-main
50+
if: steps.download-merge-base.outcome == 'failure'
51+
continue-on-error: true
52+
uses: dawidd6/action-download-artifact@8a338493df3d275e4a7a63bcff3b8fe97e51a927 # v19
53+
with:
54+
name: c-abi-baseline-main
55+
workflow: check-c-abi.yaml
56+
branch: main
57+
path: baseline/
58+
59+
- name: Extract baseline ABI from main branch (fallback 2)
60+
if: steps.download-merge-base.outcome == 'failure' && steps.download-main.outcome == 'failure'
61+
run: |
62+
echo "⚠️ No baseline artifacts found, extracting from main branch..."
63+
echo "This is slower but ensures we always have a baseline for comparison."
64+
git worktree add ../cuvs-main main
65+
mkdir -p baseline
66+
check-c-abi extract \
67+
--header-path ../cuvs-main/c/include \
68+
--include-file cuvs/core/all.h \
69+
--output-file baseline/c_abi.json.gz \
70+
--dlpack-include-path dlpack/include
71+
72+
echo "✓ Baseline ABI extracted from main branch"
73+
74+
- name: Report baseline source
75+
run: |
76+
if [ "$DOWNLOAD_MERGE_BASE_OUTCOME" == "success" ]; then
77+
echo "✓ Using baseline from merge-base commit: $MERGE_BASE_SHA"
78+
elif [ "$DOWNLOAD_MAIN_OUTCOME" == "success" ]; then
79+
echo "✓ Using latest main baseline (merge-base baseline not yet available)"
80+
else
81+
echo "✓ Using freshly extracted baseline from main branch"
82+
fi
83+
env:
84+
DOWNLOAD_MERGE_BASE_OUTCOME: ${{ steps.download-merge-base.outcome }}
85+
MERGE_BASE_SHA: ${[ steps.merge-base.outputs.merge_base_sha }}
86+
DOWNLOAD_MAIN_OUTCOME: ${{ steps.download-main.outcome }}
87+
88+
- name: Analyze current branch for ABI breaking changes
89+
run: |
90+
check-c-abi analyze \
91+
--abi-file baseline/c_abi.json.gz \
92+
--header-path c/include \
93+
--include-file cuvs/core/all.h \
94+
--dlpack-include-path dlpack/include
95+
96+
- name: Comment on PR with results
97+
if: failure()
98+
uses: actions/github-script@v7
99+
with:
100+
script: |
101+
github.rest.issues.createComment({
102+
issue_number: context.issue.number,
103+
owner: context.repo.owner,
104+
repo: context.repo.repo,
105+
body: `## ⚠️ C ABI Breaking Changes Detected
106+
This PR introduces breaking changes to the C ABI. Please review the changes carefully.
107+
108+
Breaking ABI changes are only allowed in major releases. If this is intentional for a major release,
109+
the baseline ABI will need to be updated after merge.
110+
111+
See the job logs for details on what specific changes were detected.
112+
113+
### What are breaking ABI changes?
114+
115+
Breaking ABI changes include:
116+
- Removing functions from the public API
117+
- Changing function signatures (parameters or return types)
118+
- Removing struct members or changing their types
119+
- Removing or changing enum values
120+
121+
### Next steps
122+
123+
1. Review the changes flagged in the CI logs
124+
2. If these changes are unintentional, update your PR to maintain ABI compatibility
125+
3. If these changes are required, ensure:
126+
- This is part of a major version release
127+
- The changes are documented in the changelog
128+
- Migration guide is provided for users
129+
130+
For more information, see the [C ABI documentation](../docs/source/c_developer_guide.md).`
131+
});

‎.github/workflows/pr.yaml‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,7 @@ jobs:
1212
- check-nightly-ci
1313
- changed-files
1414
- checks
15+
- check-c-abi
1516
- conda-cpp-build
1617
- conda-cpp-tests
1718
- conda-cpp-checks
@@ -315,6 +316,9 @@ jobs:
315316
with:
316317
enable_check_generated_files: false
317318
ignored_pr_jobs: "telemetry-summarize"
319+
check-c-abi:
320+
needs: telemetry-setup
321+
uses: ./.github/workflows/check-c-abi.yaml
318322
conda-cpp-build:
319323
needs: checks
320324
secrets: inherit

‎.github/workflows/store-c-abi-baseline.yaml‎

Lines changed: 73 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,73 @@
1+
name: Archive C ABI Baseline for Release
2+
3+
on:
4+
release:
5+
types: [published]
6+
workflow_dispatch:
7+
inputs:
8+
version:
9+
description: 'Version tag to archive baseline for (e.g., v26.02.00)'
10+
required: true
11+
type: string
12+
13+
jobs:
14+
archive-baseline:
15+
runs-on: ubuntu-latest
16+
steps:
17+
- name: Checkout repository
18+
uses: actions/checkout@v4
19+
20+
- name: Download main baseline artifact
21+
uses: dawidd6/action-download-artifact@8a338493df3d275e4a7a63bcff3b8fe97e51a927 # v19
22+
with:
23+
name: c-abi-baseline-main
24+
workflow: check-c-abi.yaml
25+
branch: main
26+
path: baseline/
27+
28+
- name: Archive baseline with release version
29+
uses: actions/upload-artifact@v4
30+
with:
31+
name: c-abi-baseline-${{ github.event.release.tag_name || inputs.version }}
32+
path: baseline/c_abi.json.gz
33+
retention-days: 400 # ~13 months
34+
35+
- name: Commit baseline to repository (for long-term storage)
36+
run: |
37+
git config user.name "github-actions[bot]"
38+
git config user.email "github-actions[bot]@users.noreply.github.com"
39+
40+
# Create a baselines branch if it doesn't exist
41+
git fetch origin baselines:baselines 2>/dev/null || git checkout --orphan baselines
42+
git checkout baselines 2>/dev/null || true
43+
44+
# Copy the baseline file with version name
45+
mkdir -p c-abi-baselines
46+
cp baseline/c_abi.json.gz c-abi-baselines/c_abi_$RELEASE_TAG_NAME.json.gz
47+
48+
# Commit and push
49+
git add c-abi-baselines/
50+
git commit -m "Archive C ABI baseline for $RELEASE_TAG_NAME"
51+
git push origin baselines
52+
env:
53+
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
54+
RELEASE_TAG_NAME: ${{ github.event.release.tag_name || inputs.version }}
55+
56+
- name: Create release comment
57+
if: github.event_name == 'release'
58+
uses: actions/github-script@v7
59+
with:
60+
script: |
61+
const tagName = context.payload.release.tag_name;
62+
63+
github.rest.repos.createCommitComment({
64+
owner: context.repo.owner,
65+
repo: context.repo.repo,
66+
commit_sha: context.payload.release.target_commitish,
67+
body: |
68+
`✅ C ABI baseline archived for release ${tagName}
69+
This baseline has been archived from the main branch and will be available for historical reference.
70+
The baseline is stored in:
71+
- Artifact: \`c-abi-baseline-${tagName}\` (available for ~13 months)
72+
- Branch: \`baselines\` (permanent storage)`
73+
});

‎.github/workflows/update-c-abi-baseline.yaml‎

Lines changed: 58 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,58 @@
1+
name: C ABI Update Baseleine
2+
3+
on:
4+
workflow_dispatch: # Allow manual trigger for bootstrap
5+
push:
6+
branches:
7+
- main
8+
paths:
9+
- 'c/include/**'
10+
- 'ci/check_c_abi/**'
11+
12+
jobs:
13+
# Extract and store baseline ABI from main branch
14+
update-baseline:
15+
runs-on: ubuntu-latest
16+
steps:
17+
- name: Checkout main branch
18+
uses: actions/checkout@v4
19+
20+
- name: Set up Python
21+
uses: actions/setup-python@v5
22+
with:
23+
python-version: '3.11'
24+
25+
- name: Install Python dependencies
26+
run: |
27+
pip install --upgrade pip
28+
pip install -e ci/check_c_abi
29+
30+
- name: Get dlpack dependency
31+
run: |
32+
git clone https://github.com/dmlc/dlpack
33+
34+
- name: Extract ABI from main branch
35+
run: |
36+
mkdir -p baseline
37+
check-c-abi extract \
38+
--header-path c/include \
39+
--include-file cuvs/core/all.h \
40+
--output-file baseline/c_abi.json.gz \
41+
--dlpack-include-path dlpack/include
42+
echo "ABI extracted from main branch (commit: $COMMIT)"
43+
env:
44+
COMMIT: ${{ github.sha }}
45+
46+
- name: Store commit-specific baseline
47+
uses: actions/upload-artifact@v4
48+
with:
49+
name: c-abi-baseline-${{ github.sha }}
50+
path: baseline/c_abi.json.gz
51+
retention-days: 90 # Keep for 3 months
52+
53+
- name: Store main baseline (latest, never expires)
54+
uses: actions/upload-artifact@v4
55+
with:
56+
name: c-abi-baseline-main
57+
path: baseline/c_abi.json.gz
58+
retention-days: 0 # Never expire

‎ci/check_c_abi/LICENSE‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
../../LICENSE

‎ci/check_c_abi/VERSION‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
../../VERSION

‎ci/check_c_abi/check_c_abi/__init__.py‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
#
2+
# SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION.
3+
# SPDX-License-Identifier: Apache-2.0
4+
#

0 commit comments

Comments
 (0)