[dockers]: Upgrade SNMP docker to stretch build by samaity · Pull Request #2620 · sonic-net/sonic-buildimage

samaity · 2019-02-28T19:41:38Z

Signed-off-by: Sangita Maity samaity@linkedin.com

- What I did
Move SNMP to stretch based. Requires below two pull requests to build SNMP Docker successfully.

changes needed for SNMP docker upgrade to stretch build sonic-platform-common#23
more changes needed for SNMP docker upgrade to stretch build sonic-platform-common#26

needed some changes to be compatible with python3 version. otherwise, it was giving import error or no module found for "exceptions" module.

integrating py.test into setuptools sonic-snmpagent#103

Integrate py.test into setuptools as vanilla 'python3 setup.py test' does not work.

- How I did it

samaity@Server09:~/github_srcCode/src_feb26/sonic-buildimage$ BLDENV=stretch  make configure PLATFORM=broadcom
+++ Making configure +++
make -f Makefile.work configure
make[1]: Entering directory '/home/samaity/github_srcCode/src_feb26/sonic-buildimage'
SONiC Build System

Build Configuration
"CONFIGURED_PLATFORM"             : "broadcom"
"SONIC_CONFIG_PRINT_DEPENDENCIES" : ""
"SONIC_BUILD_JOBS"                : "1"
"SONIC_CONFIG_MAKE_JOBS"          : "24"
"USERNAME"                        : "admin"
"PASSWORD"                        : "YourPaSsWoRd"
"ENABLE_DHCP_GRAPH_SERVICE"       : ""
"SHUTDOWN_BGP_ON_START"           : ""
"ENABLE_PFCWD_ON_START"           : ""
"INSTALL_DEBUG_TOOLS"             : ""
"ROUTING_STACK"                   : "quagga"
"ENABLE_SYNCD_RPC"                : ""
"ENABLE_ORGANIZATION_EXTENSIONS"  : "y"
"HTTP_PROXY"                      : ""
"HTTPS_PROXY"                     : ""
"ENABLE_SYSTEM_TELEMETRY"         : ""
"SONIC_DEBUGGING_ON"              : ""
"SONIC_PROFILING_ON"              : ""
"KERNEL_PROCURE_METHOD"           : "build"
"BUILD_TIMESTAMP"                 : "20190226.225454"
"BLDENV"                          : "stretch"

Docker moved to stretch build.

samaity@Server09:~/github_srcCode/src_feb27/sonic-buildimage/target$ ls -la docker-snmp-sv2.gz*
-rw-r--r-- 1 samaity samaity 135539759 Feb 27 15:19 docker-snmp-sv2.gz
-rw-r--r-- 1 samaity samaity     50123 Feb 27 15:19 docker-snmp-sv2.gz.log

- How to verify it

BUILD SERVER
And all the debian packages of the stretch version required for snmp->

samaity@Server09:~/github_srcCode/src_feb27/sonic-buildimage/target/debs/stretch$ ls -lrt *snmp*
-rw-r--r-- 1 samaity samaity   23980 Feb 27 10:56 snmptrapd_5.7.3+dfsg-1.7+deb9u1_amd64.deb
-rw-r--r-- 1 samaity samaity 1594854 Feb 27 10:56 libsnmp-base_5.7.3+dfsg-1.7+deb9u1_all.deb
-rw-r--r-- 1 samaity samaity 2267844 Feb 27 10:56 libsnmp30_5.7.3+dfsg-1.7+deb9u1_amd64.deb
-rw-r--r-- 1 samaity samaity 2252048 Feb 27 10:56 libsnmp30-dbg_5.7.3+dfsg-1.7+deb9u1_amd64.deb
-rw-r--r-- 1 samaity samaity 1054846 Feb 27 10:56 libsnmp-dev_5.7.3+dfsg-1.7+deb9u1_amd64.deb
-rw-r--r-- 1 samaity samaity 1497778 Feb 27 10:56 libsnmp-perl_5.7.3+dfsg-1.7+deb9u1_amd64.deb
-rw-r--r-- 1 samaity samaity   19554 Feb 27 10:56 python-netsnmp_5.7.3+dfsg-1.7+deb9u1_amd64.deb
-rw-r--r-- 1 samaity samaity 1092653 Feb 27 10:56 libsnmp-base_5.7.3+dfsg-1.7+deb9u1_all.deb.log
-rw-r--r-- 1 samaity samaity     323 Feb 27 10:56 libsnmp-base_5.7.3+dfsg-1.7+deb9u1_all.deb-install.log
-rw-r--r-- 1 samaity samaity     335 Feb 27 10:56 libsnmp30_5.7.3+dfsg-1.7+deb9u1_amd64.deb-install.log
-rw-r--r-- 1 samaity samaity  153046 Feb 27 10:56 snmp_5.7.3+dfsg-1.7+deb9u1_amd64.deb
-rw-r--r-- 1 samaity samaity   56414 Feb 27 10:56 snmpd_5.7.3+dfsg-1.7+deb9u1_amd64.deb

samaity@Server09:~/github_srcCode/src_feb27/sonic-buildimage/target/python-wheels$ ls -lrt asyncsnmp-*
-rw-r--r-- 1 samaity samaity  66526 Feb 27 12:39 asyncsnmp-2.1.0-py3-none-any.whl
-rw-r--r-- 1 samaity samaity 113223 Feb 27 12:39 asyncsnmp-2.1.0-py3-none-any.whl.log

Sonic Switch

admin@lnos-x1-a-csw04:~$ docker images
REPOSITORY                 TAG                 IMAGE ID            CREATED             SIZE
docker-snmp-sv2            latest              0d878b276eb2        19 hours ago        328.9 MB


admin@lnos-x1-a-csw04:~$ docker ps -a
CONTAINER ID        IMAGE                             COMMAND                  CREATED             STATUS              PORTS               NAMES
6530c012c419        docker-snmp-sv2:latest            "/usr/bin/supervisord"   4 seconds ago       Up 3 seconds                            snmp

admin@lnos-x1-a-csw04:~$ docker exec -it snmp bash
root@lnos-x1-a-csw04:/# ps -ef
UID        PID  PPID  C STIME TTY          TIME CMD
root         1     0  7 19:19 ?        00:00:01 /usr/bin/python /usr/bin/supervisord
root        19     1  0 19:19 ?        00:00:00 /usr/sbin/rsyslogd -n
Debian-+    24     1  4 19:19 ?        00:00:00 /usr/sbin/snmpd -f -LS4d -u Debian-snmp -g Debian-snmp -I -smux
root        26     1 25 19:19 ?        00:00:03 python3.6 -m sonic_ax_impl
root        28     0  0 19:19 ?        00:00:00 bash
root        34    28  0 19:19 ?        00:00:00 ps -ef

root@lnos-x1-a-csw04:/# supervisorctl status
rsyslogd                         RUNNING   pid 19, uptime 0:02:40
snmp-subagent                    RUNNING   pid 26, uptime 0:02:37
snmpd                            RUNNING   pid 24, uptime 0:02:39
start.sh                         EXITED    Feb 28 07:19 PM

src/snmpd/patch-5.7.3+dfsg/series

dockers/docker-snmp-sv2/Dockerfile.j2

qiluo-msft

As comment

…GetN.patch

samaity · 2019-04-03T21:11:25Z

mistakenly closed it. #Closed

qiluo-msft · 2019-04-03T21:24:35Z

Why include src/sonic-platform-common? #Closed

samaity · 2019-04-03T23:02:21Z

Why include src/sonic-platform-common?

@qiluo-msft , I made some changes in src/sonic-platform-common. Those are merged in azure/master. I need those changes during build to test. #Closed

samaity · 2019-04-09T17:25:22Z

@lguohan , I reverted back the snmpd version to 5.7.3+dfsg-1.5 as discussed.However, I am unable to build to libsnmp-base_5.7.3+dfsg-1.5_all.deb as libssl1.0-dev got removed during "sonic-slave-stretch" build.

Step 6/29 : RUN apt-get -y build-dep linux
 ---> Running in ebf2e4b9e2a6
Reading package lists...
Building dependency tree...
Reading state information...
The following packages will be REMOVED:
  libssl1.0-dev

Need your help here.

qiluo-msft · 2019-04-10T01:34:47Z

Could you resolve the conflict first? #Closed

samaity · 2019-04-10T17:51:38Z

Could you resolve the conflict first?

resolved.

qiluo-msft · 2019-04-11T01:04:14Z

~~I believe the change of src/sonic-platform-common is independent with this one~~. Please submit a standalone PR for that.

In reply to: 479689169 [](ancestors = 479689169)

samaity · 2019-04-11T02:59:45Z

I believe the change of src/sonic-platform-common is independent with this one. Please submit a standalone PR for that.

In reply to: 479689169 [](ancestors = 479689169)

@qiluo-msft , Yes, I already raised the PR and it got merged into sonic-platform-common master branch.

…kport support

samaity · 2019-04-12T16:30:57Z

@qiluo-msft @lguohan kindly review it and let me know if I have to do anything else from my side.

lguohan · 2019-04-14T00:19:58Z

@samaity , build are still failing for multiple targets. #2783 can help some build failures, but vs test pr build failed on building snmp, can you look into them?

lguohan · 2019-04-14T17:58:34Z

retest this please

samaity · 2019-04-15T05:20:19Z

@samaity , build are still failing for multiple targets. #2783 can help some build failures, but vs test pr build failed on building snmp, can you look into them?

@lguohan, As per log in test case below, docker-snmp-sv2 got built successfully on buildimage-brcm-all-pr.

make[1]: Entering directory '/data2/johnar/workspace/broadcom/buildimage-brcm-all-pr'
SONiC Build System

Build Configuration
"CONFIGURED_PLATFORM"             : "broadcom"
"SONIC_CONFIG_PRINT_DEPENDENCIES" : ""
"SONIC_BUILD_JOBS"                : "1"
"SONIC_CONFIG_MAKE_JOBS"          : "16"
"USERNAME"                        : "admin"
"PASSWORD"                        : "YourPaSsWoRd"
"ENABLE_DHCP_GRAPH_SERVICE"       : ""
"SHUTDOWN_BGP_ON_START"           : ""
"ENABLE_PFCWD_ON_START"           : ""
"INSTALL_DEBUG_TOOLS"             : ""
"ROUTING_STACK"                   : "quagga"
"ENABLE_SYNCD_RPC"                : ""
"ENABLE_ORGANIZATION_EXTENSIONS"  : "y"
"HTTP_PROXY"                      : ""
"HTTPS_PROXY"                     : ""
"ENABLE_SYSTEM_TELEMETRY"         : "y"
"SONIC_DEBUGGING_ON"              : ""
"SONIC_PROFILING_ON"              : ""
"KERNEL_PROCURE_METHOD"           : "build"
"BUILD_TIMESTAMP"                 : "20190414.181755"
"BLDENV"                          : "stretch"
"VS_PREPARE_MEM"                  : "yes"

[ building ] [ target/docker-snmp-sv2.gz ] 
[ finished ] [ target/docker-snmp-sv2.gz ]

But it got failed for vs test pr build. I believe vs test pr build is based on debian jessie as per DockerFile. So, It's trying to build libsnmp-base_5.7.3+dfsg-1.7+deb9u1_all.deb on debian jessie. Do you think we should have a separate Dockerfile for docker-sonic-vs stretch build?

SONiC Build System

Build Configuration
"CONFIGURED_PLATFORM"             : "vs"
"SONIC_CONFIG_PRINT_DEPENDENCIES" : ""
"SONIC_BUILD_JOBS"                : "1"
"SONIC_CONFIG_MAKE_JOBS"          : "16"
"USERNAME"                        : "admin"
"PASSWORD"                        : "YourPaSsWoRd"
"ENABLE_DHCP_GRAPH_SERVICE"       : ""
"SHUTDOWN_BGP_ON_START"           : ""
"ENABLE_PFCWD_ON_START"           : ""
"INSTALL_DEBUG_TOOLS"             : ""
"ROUTING_STACK"                   : "quagga"
"ENABLE_SYNCD_RPC"                : ""
"ENABLE_ORGANIZATION_EXTENSIONS"  : "y"
"HTTP_PROXY"                      : ""
"HTTPS_PROXY"                     : ""
"ENABLE_SYSTEM_TELEMETRY"         : "y"
"SONIC_DEBUGGING_ON"              : ""
"SONIC_PROFILING_ON"              : ""
"KERNEL_PROCURE_METHOD"           : "build"
"BUILD_TIMESTAMP"                 : "20190414.175943"
**"BLDENV"                          : ""**
"VS_PREPARE_MEM"                  : "yes"
[ building ] [ target/debs/libsnmp-base_5.7.3+dfsg-1.7+deb9u1_all.deb ] 
[ finished ] [ target/debs/libsnmp-base_5.7.3+dfsg-1.7+deb9u1_all.deb ] 
[ FAIL LOG START ] [ target/debs/libsnmp-base_5.7.3+dfsg-1.7+deb9u1_all.deb

qiluo-msft

Please also fix the build checkers.

lguohan · 2019-04-16T05:38:11Z

please upgrade vs docker to stretch.

qiluo-msft · 2019-04-16T19:18:07Z

Offline discussed with @zhenggen-xu , the solution should be use the same version of snmpd as Jessie, and hopefully pass all the build checkers.

* [dockers]: Upgrade SNMP docker to stretch build * Removed patch-> 0003-CHANGES-BUG-2743-snmpd-crashes-when-receiving-a-GetN.patch * update platform-common submodule * adding PyYAML package to stretch dockerfile * Installing redis package via pip in stretch dockerfile * again updating platform-common submodule * revert back the snmpd version to 5.7.3+dfsg-1.5 * upgrading the snmpd version to 5.7.3+dfsg-1.7+deb9u1 with openssl backport support * update sonic-snmpagent submodule * purge libdpkg-perl package in Dockerfile * revert back the snmpd version to 5.7.3+dfsg-1.5 * minor change in series file

[dockers]: Upgrade SNMP docker to stretch build

a97e44b

stcheng added the Enhancement ➕ label Feb 28, 2019

jleveque requested a review from qiluo-msft March 1, 2019 01:39

qiluo-msft reviewed Mar 1, 2019

View reviewed changes

src/snmpd/patch-5.7.3+dfsg/series Outdated Show resolved Hide resolved

qiluo-msft reviewed Mar 1, 2019

View reviewed changes

src/snmpd/patch-5.7.3+dfsg/series Outdated Show resolved Hide resolved

qiluo-msft reviewed Mar 1, 2019

View reviewed changes

dockers/docker-snmp-sv2/Dockerfile.j2 Show resolved Hide resolved

qiluo-msft requested changes Mar 1, 2019

View reviewed changes

samaity mentioned this pull request Mar 20, 2019

[dockers]: Upgrade LLDP docker to stretch build #2633

Merged

Removed patch-> 0003-CHANGES-BUG-2743-snmpd-crashes-when-receiving-a-…

f688dcf

…GetN.patch

samaity mentioned this pull request Mar 26, 2019

changes needed in test file for SNMP docker upgrade to stretch build sonic-net/sonic-snmpagent#99

Closed

samaity added 3 commits April 2, 2019 11:18

update platform-common submodule

9d3914f

adding PyYAML package to stretch dockerfile

d5f15af

Installing redis package via pip in stretch dockerfile

9b47ae0

samaity closed this Apr 3, 2019

samaity reopened this Apr 3, 2019

again updating platform-common submodule

a22667d

revert back the snmpd version to 5.7.3+dfsg-1.5

cf7d1bd

Merge branch 'master' into snmp_stretch

d221d5a

samaity mentioned this pull request Apr 11, 2019

integrating py.test into setuptools sonic-net/sonic-snmpagent#103

Merged

samaity added 2 commits April 10, 2019 20:13

upgrading the snmpd version to 5.7.3+dfsg-1.7+deb9u1 with openssl bac…

66a543f

…kport support

update sonic-snmpagent submodule

e032208

sonic-net deleted a comment from samaity Apr 12, 2019

purge libdpkg-perl package in Dockerfile

65fda48

qiluo-msft approved these changes Apr 16, 2019

View reviewed changes

Merge branch 'master' into snmpstretch

9b6070c

samaity added 2 commits April 16, 2019 12:31

revert back the snmpd version to 5.7.3+dfsg-1.5

da9733d

minor change in series file

b1ba67e

qiluo-msft merged commit e798b93 into sonic-net:master Apr 17, 2019

samaity deleted the snmpstretch branch April 17, 2019 18:48

Conversation

samaity commented Feb 28, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

qiluo-msft left a comment

Choose a reason for hiding this comment

Uh oh!

samaity commented Apr 3, 2019 • edited by qiluo-msft Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

qiluo-msft commented Apr 3, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

samaity commented Apr 3, 2019 • edited by qiluo-msft Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

samaity commented Apr 9, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

qiluo-msft commented Apr 10, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

samaity commented Apr 10, 2019

Uh oh!

qiluo-msft commented Apr 11, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

samaity commented Apr 11, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

samaity commented Apr 12, 2019

Uh oh!

lguohan commented Apr 14, 2019

Uh oh!

lguohan commented Apr 14, 2019

Uh oh!

samaity commented Apr 15, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

qiluo-msft left a comment

Choose a reason for hiding this comment

Uh oh!

lguohan commented Apr 16, 2019

Uh oh!

qiluo-msft commented Apr 16, 2019

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

samaity commented Feb 28, 2019 •

edited

Loading

samaity commented Apr 3, 2019 •

edited by qiluo-msft

Loading

qiluo-msft commented Apr 3, 2019 •

edited

Loading

samaity commented Apr 3, 2019 •

edited by qiluo-msft

Loading

samaity commented Apr 9, 2019 •

edited

Loading

qiluo-msft commented Apr 10, 2019 •

edited

Loading

qiluo-msft commented Apr 11, 2019 •

edited

Loading

samaity commented Apr 11, 2019 •

edited

Loading

samaity commented Apr 15, 2019 •

edited

Loading