Skip to content

Update sonic-swss-common pipeline build for Bullseye #716

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
saiarcot895 merged 14 commits into from
Mar 28, 2023
Merged

Update sonic-swss-common pipeline build for Bullseye #716

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
fb3574e
Don't install python-swsscommon in docker-sonic-vs
saiarcot895 Nov 24, 2022
cec1a08
Change AzP to build for Bullseye only
saiarcot895 Nov 24, 2022
87f5077
Don't install python-swsscommon_*.deb.
saiarcot895 Nov 24, 2022
37fcdbd
Merge remote-tracking branch 'origin/master' into bullseye-docker-son…
saiarcot895 Nov 29, 2022
5d46a4f
Use pytest-3 instead of pytest
saiarcot895 Nov 29, 2022
bc7b119
Clean up build rules to only support Bullseye builds
saiarcot895 Nov 29, 2022
510b43f
Update Microsoft repo URL for Bullseye
saiarcot895 Nov 30, 2022
1fbb507
Merge remote-tracking branch 'origin/master' into bullseye-docker-son…
saiarcot895 Feb 2, 2023
35eaff7
Merge remote-tracking branch 'origin/master' into bullseye-docker-son…
saiarcot895 Mar 20, 2023
b26c0bb
Fix sairedis build for Bullseye
saiarcot895 Mar 20, 2023
bdb85ec
Fix swss build.
saiarcot895 Mar 20, 2023
3a366e2
Temporarily bump up time limit for sairedis build
saiarcot895 Mar 20, 2023
e1506de
Replace deb packages instead of remove and install
saiarcot895 Mar 22, 2023
1f67e67
Empty commit to make new commit hash.
saiarcot895 Mar 28, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 4 additions & 7 deletions .azure-pipelines/build-sairedis-template.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ parameters:

- name: timeout
type: number
default: 60
default: 90

- name: sonic_slave
type: string
Expand Down Expand Up @@ -67,22 +67,19 @@ jobs:
sudo apt-get update
sudo apt-get install -qq -y \
qtbase5-dev \
libdbus-glib-1-2 \
libdbus-glib-1-dev \
libpcsclite-dev \
docbook-to-man \
docbook-utils \
aspell-en \
libhiredis0.14 \
libhiredis-dev \
libnl-3-dev \
libnl-genl-3-dev \
libnl-route-3-dev \
libnl-nf-3-dev \
libnl-cli-3-dev \
swig3.0 \
libpython2.7-dev \
libzmq5 libzmq3-dev
swig4.0 \
libzmq3-dev

sudo apt-get install -y redis-server
sudo sed -ri 's/^# unixsocket/unixsocket/' /etc/redis/redis.conf
Expand Down Expand Up @@ -129,7 +126,7 @@ jobs:
set -ex
rm ../*.deb || true
./autogen.sh
DEB_BUILD_OPTIONS=nocheck fakeroot dpkg-buildpackage -b -us -uc -Tbinary-syncd-vs -j$(nproc)
DEB_BUILD_OPTIONS=nocheck DEB_BUILD_PROFILES=nopython2 fakeroot dpkg-buildpackage -b -us -uc -Tbinary-syncd-vs -j$(nproc)
# Add SYS_TIME capability for settimeofday ok in syncd test
sudo setcap "cap_sys_time=eip" syncd/.libs/syncd_tests
make check
Expand Down
2 changes: 1 addition & 1 deletion .azure-pipelines/build-swss-template.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,7 @@ jobs:
sudo apt-get install -y libzmq5 libzmq3-dev
sudo apt-get install -qq -y \
libhiredis-dev \
swig3.0
swig4.0
sudo apt-get install -y libdbus-1-3
sudo apt-get install -y libteam-dev \
libteam5 \
Expand Down
12 changes: 6 additions & 6 deletions .azure-pipelines/build-template.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ parameters:

- name: debian_version
type: string
default: buster
default: bullseye

jobs:
- job:
Expand Down Expand Up @@ -94,7 +94,7 @@ jobs:
set -ex
rm ../*.deb || true
./autogen.sh
fakeroot debian/rules DEB_CONFIGURE_EXTRA_FLAGS='--enable-code-coverage' $( if [[ ${{ parameters.debian_version }} == "bullseye" ]]; then echo "DEB_BUILD_PROFILES=nopython2"; fi ) CFLAGS="" CXXFLAGS="--coverage -fprofile-abs-path" LDFLAGS="--coverage -fprofile-abs-path" binary
fakeroot debian/rules DEB_CONFIGURE_EXTRA_FLAGS='--enable-code-coverage' DEB_BUILD_PROFILES=nopython2 CFLAGS="" CXXFLAGS="--coverage -fprofile-abs-path" LDFLAGS="--coverage -fprofile-abs-path" binary
mv ../*.deb .
displayName: "Compile sonic swss common with coverage enabled"
- ${{ if eq(parameters.run_unit_test, true) }}:
Expand All @@ -121,11 +121,11 @@ jobs:

sudo dpkg -i libswsscommon_*.deb
sudo dpkg -i libswsscommon-dev_*.deb
sudo dpkg -i python-swsscommon_*.deb
sudo dpkg -i python3-swsscommon_*.deb

./tests/tests
redis-cli FLUSHALL
pytest --cov=. --cov-report=xml
pytest-3 --cov=. --cov-report=xml
mv coverage.xml tests/coverage.xml
gcovr -r ./ -e ".*/swsscommon_wrap.cpp" --exclude-unreachable-branches --exclude-throw-branches --gcov-ignore-parse-errors -x --xml-pretty -o coverage.xml
make -C goext
Expand All @@ -142,9 +142,9 @@ jobs:
set -ex
# Install .NET CORE
curl -sSL https://packages.microsoft.com/keys/microsoft.asc | sudo apt-key add -
sudo apt-add-repository https://packages.microsoft.com/debian/10/prod
sudo apt-add-repository https://packages.microsoft.com/debian/11/prod
sudo apt-get update
sudo apt-get install -y dotnet-sdk-5.0
sudo apt-get install -y dotnet-sdk-6.0
displayName: "Install .NET CORE"
- task: PublishCodeCoverageResults@1
inputs:
Expand Down
14 changes: 1 addition & 13 deletions .azure-pipelines/docker-sonic-vs/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,16 +4,4 @@ ARG docker_container_name

ADD ["debs", "/debs"]

RUN dpkg --purge python-swsscommon python3-swsscommon swss libsairedis sonic-db-cli libswsscommon libsaimetadata libsaivs syncd-vs

RUN dpkg -i /debs/libswsscommon_1.0.0_amd64.deb
RUN dpkg -i /debs/python-swsscommon_1.0.0_amd64.deb
RUN dpkg -i /debs/python3-swsscommon_1.0.0_amd64.deb
RUN dpkg -i /debs/sonic-db-cli_1.0.0_amd64.deb

RUN dpkg -i /debs/libsaimetadata_1.0.0_amd64.deb
RUN dpkg -i /debs/libsairedis_1.0.0_amd64.deb
RUN dpkg -i /debs/libsaivs_1.0.0_amd64.deb
RUN dpkg -i /debs/syncd-vs_1.0.0_amd64.deb

RUN dpkg -i /debs/swss_1.0.0_amd64.deb
RUN dpkg -i /debs/libswsscommon_1.0.0_amd64.deb /debs/python3-swsscommon_1.0.0_amd64.deb /debs/sonic-db-cli_1.0.0_amd64.deb /debs/libsaimetadata_1.0.0_amd64.deb /debs/libsairedis_1.0.0_amd64.deb /debs/libsaivs_1.0.0_amd64.deb /debs/syncd-vs_1.0.0_amd64.deb /debs/swss_1.0.0_amd64.deb
Copy link
Copy Markdown
Contributor

@liushilongbuaa liushilongbuaa Mar 23, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

libswsscommon depends on libboost1.71-dev. (in debian/control Line5)
But docker-sonic-vs uninstalled libboost1.71. This line should fail. @saiarcot895 can you rerun the pipeline? Maybe the former PR check used old docker-sonic-vs image.

Copy link
Copy Markdown
Contributor Author

@saiarcot895 saiarcot895 Mar 23, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The dependency string is specified as libboost-dev | libboost1.71-dev, which means either libboost-dev or libboost1.71-dev needs to be installed. Since libboost-dev is installed in Bullseye, the dependency is met.

In Buster, Boost 1.71 is installed through the libboost1.71-dev (and associated) packages. The Buster default is Boost 1.67, and I guess something explicitly wanted to use Boost 1.71, so all packages are built with that. Because libswsscommon might still get built for Buster (it's needed for the nat and sflow containers), that part of the dependency string cannot be removed just yet.

Copy link
Copy Markdown
Contributor

@liushilongbuaa liushilongbuaa Mar 24, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/mnt$ docker run --rm -ti --entrypoint bash docker-sonic-vs:latest
root@9f79d2f3aae7:/# apt list| grep libboost

WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

libboost-serialization1.74.0/now 1.74.0-9 amd64 [installed,local]

Copy link
Copy Markdown
Contributor

@liushilongbuaa liushilongbuaa Mar 24, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

docker-sonic-vs didn't install libboost-dev or libboost1.71-dev

Copy link
Copy Markdown
Contributor Author

@saiarcot895 saiarcot895 Mar 24, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The docker container shouldn't need the dev package. The dev package is only for building libswsscommon, and should only be present in the slave container.

Copy link
Copy Markdown
Contributor Author

@saiarcot895 saiarcot895 Mar 24, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Line 5 of debian/control lists the build-time dependencies. This does not necessarily correspond to runtime dependencies. The runtime dependency list is automatically generated, and gets filled into the shlibs:Depends variable for each package. For the libswsscommon package, this dependency list is libboost-serialization1.74.0 (>= 1.74.0), libc6 (>= 2.14), libgcc-s1 (>= 3.0), libhiredis0.14 (>= 0.14.0), libnl-3-200 (>= 3.2.21), libnl-nf-3-200 (>= 3.2.7), libnl-route-3-200 (>= 3.2.7), libstdc++6 (>= 9), libuuid1 (>= 2.16), libyang, libzmq5 (>= 4.0.1+dfsg) (for a package built on Bullseye).

14 changes: 1 addition & 13 deletions azure-pipelines.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ resources:
parameters:
- name: debian_version
type: string
default: buster
default: bullseye
variables:
- name: BUILD_BRANCH
${{ if eq(variables['Build.Reason'], 'PullRequest') }}:
Expand Down Expand Up @@ -119,18 +119,6 @@ stages:
artifact_name: sonic-swss-common.arm64
debian_version: ${{ parameters.debian_version }}

- stage: BuildBullseye
dependsOn: Build
condition: succeeded('Build')
jobs:
- template: .azure-pipelines/build-template.yml
parameters:
arch: amd64
timeout: 180
sonic_slave: sonic-slave-bullseye:$(BUILD_BRANCH)
artifact_name: sonic-swss-common.bullseye.amd64
debian_version: bullseye

- stage: BuildSairedis
dependsOn: Build
condition: succeeded('Build')
Expand Down