Install python-redis package to docker containers by liuh-80 · Pull Request #14632 · sonic-net/sonic-buildimage

liuh-80 · 2023-04-13T01:26:18Z

Install python-redis package to docker containers

Why I did it

This this bug: #14531
The 'flush_unused_database' is part of docker-database, and docker-database does not install python-redis package by itself. it's using redis installed by sonic-py-swsssdk.
So after remove sonic-py-swsssdk from container, this script break.

To this this bug and avoid similer bug happen again, install python-redis to docker containers which removed sonic-py-swsssdk .

How I did it

Install python-redis to containers.

How to verify it

Pass all UT.
Create new UT to cover this scenario: sonic-net/sonic-mgmt#8032

Which release branch to backport (provide reason below if selected)

Description for the changelog

Improve sudo cat command for RO user.

Ensure to add label/tag for the feature raised. example - PR#2174 under sonic-utilities repo. where, Generic Config and Update feature has been labelled as GCU.

Link to config_db schema for YANG module changes

A picture of a cute animal (not mandatory but encouraged)

qiluo-msft · 2023-04-13T07:01:31Z

Is it true that only flush_unused_database inside docker-database uses redis python package? If yes, we can change only docker-database Dockerfile.

Is it possible to replace this dependency by libswsscommon? If yes, create a new issue to track.

liuh-80 · 2023-04-13T07:34:50Z

Is it true that only flush_unused_database inside docker-database uses redis python package? If yes, we can change only docker-database Dockerfile.

Is it possible to replace this dependency by libswsscommon? If yes, create a new issue to track.

The swsssdk been removed from all these containser by this PR: https://github.com/sonic-net/sonic-buildimage/pull/11469/files

To avoid potensial risk that other script may break because python-redis missing, maybe we install python-redis to all containers?

Also it's possible replace the dependency by libswsscommon, new issue: #14634

nazariig · 2023-04-13T07:59:02Z

dockers/docker-snmp/Dockerfile.j2

        smbus

+# Install python-redis
+RUN pip3 install redis==4.5.4


@liuh-80 why do we need this?

The swssdk depends on python-redis.
So before we remove swssdk by this PR, python-redis also install to docker-snmp when install swsssdk: https://github.com/sonic-net/sonic-buildimage/pull/11469/files

So to avoid potensial risk that some other script also using python-redis break because swsssdk removed from this docker, maybe we install python-redis to this container.

Fixed, code removed.

nazariig · 2023-04-13T07:59:15Z

dockers/docker-platform-monitor/Dockerfile.j2

 RUN pip3 install libpci

+# Install python-redis
+RUN pip3 install redis==4.5.4


@liuh-80 why do we need this?

same reason as docker-snmp

Fixed, code removed.

xumia · 2023-04-13T08:15:19Z

Hua, do we need to install redis to all docker config engine images, or only swss layer enough? Such as docker-swss-layer-bullseye, docker-swss-layer-buster.

liuh-80 · 2023-04-13T08:26:42Z

Hua, do we need to install redis to all docker config engine images, or only swss layer enough? Such as docker-swss-layer-bullseye, docker-swss-layer-buster.

Seems those docker layer is based on docker config engine images?

https://github.com/sonic-net/sonic-buildimage/blob/d57de0987a26553e94b8c31caed385e8b156c70c/dockers/docker-swss-layer-buster/Dockerfile.j2
{% from "dockers/dockerfile-macros.j2" import install_debian_packages, install_python_wheels, copy_files %}
FROM docker-config-engine-buster-{{DOCKER_USERNAME}}:{{DOCKER_USERTAG}}

The original swsssdk and python-redis been installed to docker config engine images, so there are potensial gap if we only install python-redis to these layer.

nazariig · 2023-04-13T11:09:19Z

Seems those docker layer is based on docker config engine images?

@liuh-80 that is exactly my question: why do we need to have the same package in database docker when it is already installed in config engine? What is the motivation to have it in both places?

liuh-80 · 2023-04-14T02:03:16Z

Seems those docker layer is based on docker config engine images?

@liuh-80 that is exactly my question: why do we need to have the same package in database docker when it is already installed in config engine? What is the motivation to have it in both places?

@nazariig, I check these docker layer file, currently the python-redis does not installed in it.

Also the docker-database is based on config engine, not using swss layer.

qiluo-msft · 2023-04-15T00:04:24Z

dockers/docker-platform-monitor/Dockerfile.j2

 RUN pip3 install libpci

+# Install python-redis
+RUN pip3 install redis==4.5.4


This is already in base image docker-config-engine-bullseye* #Closed

Fixed, code removed.

qiluo-msft · 2023-04-15T00:05:14Z

dockers/docker-snmp/Dockerfile.j2

        smbus

+# Install python-redis
+RUN pip3 install redis==4.5.4


This is already in base image docker-config-engine-bullseye* #Closed

Fixed, code removed.

qiluo-msft · 2023-04-15T00:06:35Z

dockers/docker-config-engine-bullseye/Dockerfile.j2

 RUN pip3 uninstall -y enum34

+# Install python-redis
+RUN pip3 install redis==4.5.4


==4.5.4

@xumia, do we need to specify the version?

==4.5.4

@xumia, do we need to specify the version?

If python3-redis installed (redis=3.5.3 currently), then need to specify the version, or use pip3 install --upgrade redis. If the version not specified, then the installation will be skipped, use 3.5.3.

If we need the version 4.5.4 exactly, then "pip3 install redis==4.5.4" is good.
If we want to use the latest version, and at least 4.5.4, use "pip3 install redis>=4.5.4", prefer to auto version upgrade.

Hi, we need exactly 4.5.4, because swsssdk install 4.5.4:

https://github.com/sonic-net/sonic-py-swsssdk/blob/master/setup.py
dependencies = [
'redis==4.5.4;python_version >= "3.0"',
'redis>=3.5.3;python_version < "3.0"',
'redis-dump-load',
]

Install python-redis package to docker containers #### Why I did it This this bug: sonic-net#14531 The 'flush_unused_database' is part of docker-database, and docker-database does not install python-redis package by itself. it's using redis installed by sonic-py-swsssdk. So after remove sonic-py-swsssdk from container, this script break. To this this bug and avoid similer bug happen again, install python-redis to docker containers which removed sonic-py-swsssdk . #### How I did it Install python-redis to containers. #### How to verify it Pass all UT. Create new UT to cover this scenario: sonic-net/sonic-mgmt#8032 #### Description for the changelog Improve sudo cat command for RO user.

mssonicbld · 2023-04-20T15:32:34Z

Cherry-pick PR to 202211: #14775

Install python-redis package to docker containers #### Why I did it This this bug: #14531 The 'flush_unused_database' is part of docker-database, and docker-database does not install python-redis package by itself. it's using redis installed by sonic-py-swsssdk. So after remove sonic-py-swsssdk from container, this script break. To this this bug and avoid similer bug happen again, install python-redis to docker containers which removed sonic-py-swsssdk . #### How I did it Install python-redis to containers. #### How to verify it Pass all UT. Create new UT to cover this scenario: sonic-net/sonic-mgmt#8032 #### Description for the changelog Improve sudo cat command for RO user.

**What I did** Add new UT for flush_unused_database scripts. **Why I did it** The flush_unused_database script not cover by UT. **How I verified it** Pass all UT. **Details if related** This UT depends on PR: sonic-net/sonic-buildimage#14632

Ad redis package to docntainers

3567dfb

liuh-80 changed the title ~~[POC] Add redis package to docker containers~~ [POC] Install python-redis package to docker containers Apr 13, 2023

This was referenced Apr 13, 2023

[database]: Traceback during flush of unused database #14531

Closed

Add UT for flush_unused_database sonic-net/sonic-mgmt#8032

Merged

liuh-80 changed the title ~~[POC] Install python-redis package to docker containers~~ Install python-redis package to docker containers Apr 13, 2023

liuh-80 marked this pull request as ready for review April 13, 2023 06:11

liuh-80 requested review from lguohan and qiluo-msft as code owners April 13, 2023 06:11

qiluo-msft requested a review from xumia April 13, 2023 06:56

Add missing change

c8824d3

nazariig requested changes Apr 13, 2023

View reviewed changes

liuh-80 mentioned this pull request Apr 13, 2023

Add UT to check sudo cat command sonic-net/sonic-mgmt#8034

Merged

qiluo-msft reviewed Apr 15, 2023

View reviewed changes

Revert unecessary code

064a533

xumia approved these changes Apr 19, 2023

View reviewed changes

nazariig approved these changes Apr 19, 2023

View reviewed changes

qiluo-msft approved these changes Apr 20, 2023

View reviewed changes

qiluo-msft merged commit a14cc76 into sonic-net:master Apr 20, 2023

liuh-80 added the Request for 202211 Branch label Apr 20, 2023

StormLiangMS added the Approved for 202211 Branch label Apr 20, 2023

mssonicbld added the Created PR to 202211 Branch label Apr 20, 2023

mssonicbld mentioned this pull request Apr 20, 2023

[action] [PR:14632] Install python-redis package to docker containers #14775

Merged

mssonicbld added Included in 202211 Branch and removed Approved for 202211 Branch Created PR to 202211 Branch labels Apr 23, 2023

Conversation

liuh-80 commented Apr 13, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Why I did it

How I did it

How to verify it

Which release branch to backport (provide reason below if selected)

Description for the changelog

Ensure to add label/tag for the feature raised. example - PR#2174 under sonic-utilities repo. where, Generic Config and Update feature has been labelled as GCU.

Link to config_db schema for YANG module changes

A picture of a cute animal (not mandatory but encouraged)

Uh oh!

qiluo-msft commented Apr 13, 2023

Uh oh!

liuh-80 commented Apr 13, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

xumia commented Apr 13, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

liuh-80 commented Apr 13, 2023

Uh oh!

nazariig commented Apr 13, 2023

Uh oh!

liuh-80 commented Apr 14, 2023

Uh oh!

qiluo-msft Apr 15, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

qiluo-msft Apr 15, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

xumia Apr 15, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mssonicbld commented Apr 20, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

liuh-80 commented Apr 13, 2023 •

edited

Loading

liuh-80 commented Apr 13, 2023 •

edited

Loading

xumia commented Apr 13, 2023 •

edited

Loading

qiluo-msft Apr 15, 2023 •

edited

Loading

qiluo-msft Apr 15, 2023 •

edited

Loading

xumia Apr 15, 2023 •

edited

Loading