[build] Add ccache support for C/C++ compilation (SONIC_CONFIG_USE_CCACHE) by rustiqly · Pull Request #25650 · sonic-net/sonic-buildimage

rustiqly · 2026-02-24T19:52:17Z

Description

[agent]
Add optional ccache integration to dramatically speed up incremental C/C++ builds. When SONIC_CONFIG_USE_CCACHE=y is set, ccache intercepts gcc/g++ calls and caches object files, making recompilation of unchanged source near-instant.

Motivation

The critical build path includes three C++ heavy packages:

libswsscommon: 57 .cpp files, 2m build time
libsairedis: 3m18s build time
swss: 4m49s build time

These packages are on the serial critical path (libswsscommon → libsairedis → swss = ~10m). With a warm ccache, incremental rebuilds (e.g., when only one file changes in swss) drop from minutes to seconds.

Changes

File	Change
`rules/config`	Add `SONIC_CONFIG_USE_CCACHE` knob (default: `n`)
`slave.mk`	Define `CCACHE_ENV`, inject into build commands, add `ccache-stats`/`ccache-clear` targets
`sonic-slave-bookworm/Dockerfile.j2`	Install `ccache` package
`sonic-slave-trixie/Dockerfile.j2`	Install `ccache` package

How it works

ccache is installed in the sonic-slave container (adds ~1MB)
When enabled, /usr/lib/ccache is prepended to PATH — this directory contains symlinks (gcc, g++, cc, c++) that route through ccache
Cache is stored at target/ccache/<distro>/ — persists across builds, per-distro to avoid cross-contamination
CCACHE_COMPILERCHECK=content ensures cache invalidation on compiler upgrades
When disabled (default), CCACHE_ENV is empty — zero impact

Usage

# Enable in config.user (persistent)
echo 'SONIC_CONFIG_USE_CCACHE = y' >> rules/config.user

# Or one-shot
make SONIC_CONFIG_USE_CCACHE=y target/sonic-vs.img.gz

# Check cache statistics
make ccache-stats

# Clear cache
make ccache-clear

Expected Impact

Scenario	Without ccache	With warm ccache
Full clean build	~53m	~53m (first build populates cache)
Rebuild after swss change	~5m (swss)	~10s (only changed files recompile)
Rebuild after no changes	~10m (full critical path)	~30s (all cache hits)

Backward Compatibility

Default n — no behavior change unless explicitly enabled
ccache package is installed but dormant when not enabled
No impact on cross-builds (CCACHE_ENV applied uniformly)
DPKG cache (SONIC_DPKG_CACHE_METHOD) is orthogonal and works alongside ccache

Testing

Full VS build test pending with CI.

rustiqly · 2026-02-24T19:52:21Z

@copilot review

mssonicbld · 2026-02-24T19:52:51Z

/azp run Azure.sonic-buildimage

azure-pipelines · 2026-02-24T19:53:04Z

Azure Pipelines successfully started running 1 pipeline(s).

mssonicbld · 2026-02-24T22:08:43Z

/azp run Azure.sonic-buildimage

azure-pipelines · 2026-02-24T22:08:55Z

Azure Pipelines successfully started running 1 pipeline(s).

mssonicbld · 2026-02-24T22:28:09Z

/azp run Azure.sonic-buildimage

azure-pipelines · 2026-02-24T22:28:20Z

Azure Pipelines successfully started running 1 pipeline(s).

Copilot

Pull request overview

Adds an opt-in SONIC_CONFIG_USE_CCACHE build knob to integrate ccache into the SONiC build environment, aiming to speed up incremental C/C++ rebuilds by caching compilation outputs.

Changes:

Introduces SONIC_CONFIG_USE_CCACHE configuration option (default n).
Installs ccache in the bookworm and trixie sonic-slave images.
Injects a CCACHE_ENV prefix into Debian package build commands and adds ccache-stats / ccache-clear helper targets.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 5 comments.

File	Description
`rules/config`	Adds `SONIC_CONFIG_USE_CCACHE` knob and documentation comment.
`slave.mk`	Defines `CCACHE_ENV`, prefixes build commands, prints knob value, and adds ccache helper targets.
`sonic-slave-bookworm/Dockerfile.j2`	Installs `ccache` in the bookworm build container.
`sonic-slave-trixie/Dockerfile.j2`	Installs `ccache` in the trixie build container.

slave.mk

mssonicbld · 2026-02-24T22:33:32Z

/azp run Azure.sonic-buildimage

azure-pipelines · 2026-02-24T22:33:44Z

Azure Pipelines successfully started running 1 pipeline(s).

mssonicbld · 2026-02-25T00:47:18Z

/azp run Azure.sonic-buildimage

azure-pipelines · 2026-02-25T00:47:31Z

Azure Pipelines successfully started running 1 pipeline(s).

mssonicbld · 2026-02-25T04:49:13Z

/azp run Azure.sonic-buildimage

azure-pipelines · 2026-02-25T04:49:24Z

Azure Pipelines successfully started running 1 pipeline(s).

mssonicbld · 2026-02-25T05:47:27Z

/azp run Azure.sonic-buildimage

azure-pipelines · 2026-02-25T05:47:38Z

Azure Pipelines successfully started running 1 pipeline(s).

mssonicbld · 2026-02-25T07:50:17Z

/azp run Azure.sonic-buildimage

azure-pipelines · 2026-02-25T07:50:28Z

Azure Pipelines successfully started running 1 pipeline(s).

mssonicbld · 2026-02-25T08:59:37Z

/azp run Azure.sonic-buildimage

azure-pipelines · 2026-02-25T08:59:49Z

Azure Pipelines successfully started running 1 pipeline(s).

mssonicbld · 2026-03-05T15:03:45Z

/azp run Azure.sonic-buildimage

azure-pipelines · 2026-03-05T15:03:57Z

Azure Pipelines successfully started running 1 pipeline(s).

mssonicbld · 2026-03-06T15:09:34Z

/azp run Azure.sonic-buildimage

azure-pipelines · 2026-03-06T15:09:47Z

Azure Pipelines successfully started running 1 pipeline(s).

mssonicbld · 2026-03-08T17:03:22Z

/azp run Azure.sonic-buildimage

azure-pipelines · 2026-03-08T17:03:34Z

Azure Pipelines successfully started running 1 pipeline(s).

mssonicbld · 2026-03-09T14:02:54Z

/azp run Azure.sonic-buildimage

azure-pipelines · 2026-03-09T14:03:07Z

Azure Pipelines successfully started running 1 pipeline(s).

yejianquan

LGTM. Well-designed opt-in knob:

Default off, zero impact when disabled (CCACHE_ENV is empty)
/usr/lib/ccache PATH prepend is the standard ccache integration pattern
Per-BLDENV cache dir prevents cross-contamination
CCACHE_COMPILERCHECK=content handles compiler upgrades correctly
Error check for missing ccache binary is good
ccache-stats/ccache-clear helper targets are useful

The build-report target removal looks unrelated -- was that intentional?

🤖 Posted by DevAce, Jianquan's AI Agent, on his behalf.

mssonicbld · 2026-03-10T14:01:12Z

/azp run Azure.sonic-buildimage

azure-pipelines · 2026-03-10T14:01:28Z

Azure Pipelines successfully started running 1 pipeline(s).

mssonicbld · 2026-03-11T14:03:42Z

/azp run Azure.sonic-buildimage

azure-pipelines · 2026-03-11T14:03:54Z

Azure Pipelines successfully started running 1 pipeline(s).

mssonicbld · 2026-03-13T02:59:27Z

/azp run Azure.sonic-buildimage

azure-pipelines · 2026-03-13T02:59:39Z

Azure Pipelines successfully started running 1 pipeline(s).

…ACHE) Add optional ccache integration to speed up incremental C/C++ builds. When SONIC_CONFIG_USE_CCACHE=y is set in rules/config, ccache intercepts gcc/g++ calls via /usr/lib/ccache PATH prepend and caches compilation results under target/ccache/<distro>/. Changes: - rules/config: Add SONIC_CONFIG_USE_CCACHE knob (default: n) - slave.mk: Define CCACHE_ENV with PATH, CCACHE_DIR, CCACHE_BASEDIR, and CCACHE_COMPILERCHECK; inject into dpkg-buildpackage and make build commands; add ccache-stats and ccache-clear targets; print config status in build info - sonic-slave-bookworm/Dockerfile.j2: Install ccache package - sonic-slave-trixie/Dockerfile.j2: Install ccache package Benefits: - First build populates the cache (minimal overhead from ccache lookups) - Subsequent builds with unchanged sources: near-instant recompilation - Key packages on the critical path (libswsscommon 2m, libsairedis 3m, swss 5m) benefit most from cache hits during iterative development - Cache is per-distro (bookworm/trixie) to avoid cross-contamination - CCACHE_COMPILERCHECK=content ensures cache validity across compiler updates Usage: echo 'SONIC_CONFIG_USE_CCACHE = y' >> rules/config.user make ccache-stats # view hit/miss statistics make ccache-clear # wipe cache Signed-off-by: Rustiqly <[email protected]>

mssonicbld · 2026-03-13T14:03:15Z

/azp run Azure.sonic-buildimage

azure-pipelines · 2026-03-13T14:03:28Z

Azure Pipelines successfully started running 1 pipeline(s).

…ACHE) (sonic-net#25650)

rustiqly requested review from lguohan, qiluo-msft and xumia as code owners February 24, 2026 19:52

lihuay requested a review from Copilot February 24, 2026 22:27

Copilot started reviewing on behalf of lihuay February 24, 2026 22:27 View session

Copilot AI reviewed Feb 24, 2026

View reviewed changes

slave.mk Outdated Show resolved Hide resolved

slave.mk Outdated Show resolved Hide resolved

slave.mk Outdated Show resolved Hide resolved

slave.mk Show resolved Hide resolved

slave.mk Outdated Show resolved Hide resolved

rustiqly force-pushed the ccache-support branch from 6faeb06 to e85a3e6 Compare February 25, 2026 00:47

rustiqly force-pushed the ccache-support branch from e85a3e6 to ba589dc Compare February 25, 2026 04:49

rustiqly force-pushed the ccache-support branch from ba589dc to 02a21f5 Compare February 25, 2026 05:46

rustiqly force-pushed the ccache-support branch from 02a21f5 to 2ce38e7 Compare February 25, 2026 07:47

rustiqly force-pushed the ccache-support branch from 2ce38e7 to 0a346c7 Compare February 25, 2026 08:58

rustiqly dismissed yxieca’s stale review via 6b480a2 March 5, 2026 04:07

rustiqly force-pushed the ccache-support branch 2 times, most recently from 6b480a2 to 3ae64e4 Compare March 5, 2026 15:01

rustiqly force-pushed the ccache-support branch from 3ae64e4 to dc2d618 Compare March 6, 2026 15:01

rustiqly force-pushed the ccache-support branch from dc2d618 to 855fcb9 Compare March 8, 2026 17:03

rustiqly force-pushed the ccache-support branch from 855fcb9 to 9192ef7 Compare March 9, 2026 14:01

yejianquan approved these changes Mar 9, 2026

View reviewed changes

rustiqly force-pushed the ccache-support branch from 9192ef7 to fc11338 Compare March 10, 2026 14:01

rustiqly force-pushed the ccache-support branch from fc11338 to 0841f03 Compare March 11, 2026 14:01

rustiqly force-pushed the ccache-support branch from 0841f03 to 94ef299 Compare March 13, 2026 02:57

rustiqly force-pushed the ccache-support branch from 94ef299 to 7cf0db8 Compare March 13, 2026 14:01

lihuay merged commit db6b897 into sonic-net:master Mar 14, 2026
20 checks passed

yue-fred-gao pushed a commit to yue-fred-gao/sonic-buildimage that referenced this pull request Mar 16, 2026

[build] Add ccache support for C/C++ compilation (SONIC_CONFIG_USE_CC…

deaa28f

…ACHE) (sonic-net#25650)

Conversation

rustiqly commented Feb 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Motivation

Changes

How it works

Usage

Expected Impact

Backward Compatibility

Testing

Uh oh!

rustiqly commented Feb 24, 2026

Uh oh!

mssonicbld commented Feb 24, 2026

Uh oh!

azure-pipelines bot commented Feb 24, 2026

Uh oh!

mssonicbld commented Feb 24, 2026

Uh oh!

azure-pipelines bot commented Feb 24, 2026

Uh oh!

mssonicbld commented Feb 24, 2026

Uh oh!

azure-pipelines bot commented Feb 24, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

mssonicbld commented Feb 24, 2026

Uh oh!

azure-pipelines bot commented Feb 24, 2026

Uh oh!

mssonicbld commented Feb 25, 2026

Uh oh!

azure-pipelines bot commented Feb 25, 2026

Uh oh!

mssonicbld commented Feb 25, 2026

Uh oh!

azure-pipelines bot commented Feb 25, 2026

Uh oh!

mssonicbld commented Feb 25, 2026

Uh oh!

azure-pipelines bot commented Feb 25, 2026

Uh oh!

mssonicbld commented Feb 25, 2026

Uh oh!

azure-pipelines bot commented Feb 25, 2026

Uh oh!

mssonicbld commented Feb 25, 2026

Uh oh!

azure-pipelines bot commented Feb 25, 2026

Uh oh!

mssonicbld commented Mar 5, 2026

Uh oh!

azure-pipelines bot commented Mar 5, 2026

Uh oh!

mssonicbld commented Mar 6, 2026

Uh oh!

azure-pipelines bot commented Mar 6, 2026

Uh oh!

mssonicbld commented Mar 8, 2026

Uh oh!

azure-pipelines bot commented Mar 8, 2026

Uh oh!

mssonicbld commented Mar 9, 2026

Uh oh!

azure-pipelines bot commented Mar 9, 2026

Uh oh!

yejianquan left a comment

Choose a reason for hiding this comment

Uh oh!

mssonicbld commented Mar 10, 2026

rustiqly commented Feb 24, 2026 •

edited

Loading