Skip to content

[setup-container]: Fix incompatible packages between virtual env and global env#9440

Merged
Pterosaur merged 1 commit intosonic-net:masterfrom
Pterosaur:fix_env
Aug 16, 2023
Merged

[setup-container]: Fix incompatible packages between virtual env and global env#9440
Pterosaur merged 1 commit intosonic-net:masterfrom
Pterosaur:fix_env

Conversation

@Pterosaur
Copy link
Copy Markdown
Contributor

Description of PR

Summary:
In setup-container stage, all packages under the user, AzpDevOps, will be reinstalled to the virtual environment of host user. But there are a few incompatible packages that cannot be installed by pip command of the virtual environment.

Type of change

  • Bug fix
  • Testbed and Framework(new/improvement)
  • Test case(new/improvement)

Back port request

  • 201911
  • 202012
  • 202205

Approach

What is the motivation for this PR?

In setup-container stage, all packages under the user, AzpDevOps, will be reinstalled to the virtual environment of host user. But there are a few incompatible packages that cannot be installed by pip command of the virtual environment.

How did you do it?

Filter out these incompatible packages at the re-installing step.

How did you verify/test it?

Check setup-container command in the locally.

$ ./setup-container.sh -n sonic-mgmt  -d /data
'/home/zegan/.ssh/id_rsa_docker_sonic_mgmt' -> '/tmp/tmp.Ssj2kL0x43/id_rsa'
'/home/zegan/.ssh/id_rsa_docker_sonic_mgmt.pub' -> '/tmp/tmp.Ssj2kL0x43/id_rsa.pub'
[+] Building 66.9s (27/27) FINISHED                                                                                                                                                                                                                                               docker:default
 => [internal] load build definition from Dockerfile                                                                                                                                                                                                                                        0.1s
 => => transferring dockerfile: 2.31kB                                                                                                                                                                                                                                                      0.0s
 => [internal] load .dockerignore                                                                                                                                                                                                                                                           0.1s
 => => transferring context: 2B                                                                                                                                                                                                                                                             0.0s
 => [internal] load metadata for sonicdev-microsoft.azurecr.io:443/docker-sonic-mgmt:latest                                                                                                                                                                                                 0.0s
 => [ 1/22] FROM sonicdev-microsoft.azurecr.io:443/docker-sonic-mgmt                                                                                                                                                                                                                        0.0s
 => [internal] load build context                                                                                                                                                                                                                                                           0.1s
 => => transferring context: 3.25kB                                                                                                                                                                                                                                                         0.0s
 => CACHED [ 2/22] RUN if getent group zegan; then groupmod -o -g 1000 zegan; else groupadd -o -g 1000 zegan; fi                                                                                                                                                                            0.0s
 => CACHED [ 3/22] RUN if getent passwd zegan; then userdel zegan; fi                                                                                                                                                                                                                       0.0s
 => CACHED [ 4/22] RUN useradd -o -l -g 1000 -u 1000 -m -d /home/zegan -s /bin/bash zegan;                                                                                                                                                                                                  0.0s
 => CACHED [ 5/22] RUN if getent group docker; then groupmod -o -g 998 docker; else groupadd -o -g 998 docker; fi                                                                                                                                                                           0.0s
 => CACHED [ 6/22] RUN if [ 'zegan' != 'AzDevOps' ]; then /bin/bash -O extglob -c 'cp -a -f /var/AzDevOps/!(env-*) /home/zegan/'; for hidden_stuff in '.profile .local .ssh'; do /bin/bash -c 'cp -a -f /var/AzDevOps/$hidden_stuff /home/zegan/ || true'; done fi                          0.0s
 => CACHED [ 7/22] RUN usermod -a -G sudo zegan                                                                                                                                                                                                                                             0.0s
 => CACHED [ 8/22] RUN usermod -a -G docker zegan                                                                                                                                                                                                                                           0.0s
 => CACHED [ 9/22] RUN echo 'zegan ALL=(ALL) NOPASSWD:ALL' > /etc/sudoers.d/zegan                                                                                                                                                                                                           0.0s
 => CACHED [10/22] RUN chmod 0440 /etc/sudoers.d/zegan                                                                                                                                                                                                                                      0.0s
 => CACHED [11/22] RUN chown -R '1000:1000' /home/zegan                                                                                                                                                                                                                                     0.0s
 => CACHED [12/22] RUN sed -i -E 's/^#?PermitRootLogin.*$/PermitRootLogin yes/g' /etc/ssh/sshd_config                                                                                                                                                                                       0.0s
 => CACHED [13/22] RUN echo 'root:root' | chpasswd                                                                                                                                                                                                                                          0.0s
 => CACHED [14/22] RUN echo 'zegan:12345' | chpasswd                                                                                                                                                                                                                                        0.0s
 => CACHED [15/22] COPY --chown=1000:1000 id_rsa id_rsa.pub /home/zegan/.ssh/                                                                                                                                                                                                               0.0s
 => CACHED [16/22] RUN chmod 0700 /home/zegan/.ssh                                                                                                                                                                                                                                          0.0s
 => CACHED [17/22] RUN chmod 0600 /home/zegan/.ssh/id_rsa                                                                                                                                                                                                                                   0.0s
 => CACHED [18/22] RUN chmod 0644 /home/zegan/.ssh/id_rsa.pub                                                                                                                                                                                                                               0.0s
 => CACHED [19/22] RUN cat /home/zegan/.ssh/id_rsa.pub >> /home/zegan/.ssh/authorized_keys                                                                                                                                                                                                  0.0s
 => CACHED [20/22] RUN chmod 0600 /home/zegan/.ssh/authorized_keys                                                                                                                                                                                                                          0.0s
 => CACHED [21/22] WORKDIR /home/zegan                                                                                                                                                                                                                                                      0.0s
 => [22/22] RUN if [ 'zegan' != 'AzDevOps' ] && [ -d /var/AzDevOps/env-python3 ]; then /bin/bash -c 'python3 -m venv ${HOME}/env-python3'; /bin/bash -c '${HOME}/env-python3/bin/pip install pip --upgrade'; /bin/bash -c '${HOME}/env-python3/bin/pip install wheel'; /bin/bash -c '${HO  61.1s
 => exporting to image                                                                                                                                                                                                                                                                      5.6s
 => => exporting layers                                                                                                                                                                                                                                                                     5.6s
 => => writing image sha256:fd47f5fa0ac90e0f2084037cb9f7fe85e48577c3854031578e585fbcb6630c03                                                                                                                                                                                                0.0s
 => => naming to docker.io/library/docker-sonic-mgmt-zegan:master                                                                                                                                                                                                                           0.0s
b419771c024f1b8dc5e59270f24bf55df86e695e62da7fdb4c3d09d62417bc06
 * Restarting OpenBSD Secure Shell server sshd
   ...done.
******************************************************************************
EXEC: docker exec --user zegan -ti sonic-mgmt bash
SSH:  ssh -i ~/.ssh/id_rsa_docker_sonic_mgmt [email protected]
******************************************************************************

Any platform specific information?

Supported testbed topology if it's a new test case?

Documentation

@Pterosaur Pterosaur marked this pull request as ready for review August 14, 2023 03:12
@Pterosaur Pterosaur merged commit b10f9b6 into sonic-net:master Aug 16, 2023
AharonMalkin pushed a commit to AharonMalkin/sonic-mgmt that referenced this pull request Jan 25, 2024
…global env (sonic-net#9440)

Description of PR
Summary:
In setup-container stage, all packages under the user, AzpDevOps, will be reinstalled to the virtual environment of host user. But there are a few incompatible packages that cannot be installed by pip command of the virtual environment.

Type of change
 Bug fix
 Testbed and Framework(new/improvement)
 Test case(new/improvement)
Back port request
 201911
 202012
 202205
Approach
What is the motivation for this PR?
In setup-container stage, all packages under the user, AzpDevOps, will be reinstalled to the virtual environment of host user. But there are a few incompatible packages that cannot be installed by pip command of the virtual environment.

How did you do it?
Filter out these incompatible packages at the re-installing step.

How did you verify/test it?
Check setup-container command in the locally.

$ ./setup-container.sh -n sonic-mgmt  -d /data
'/home/zegan/.ssh/id_rsa_docker_sonic_mgmt' -> '/tmp/tmp.Ssj2kL0x43/id_rsa'
'/home/zegan/.ssh/id_rsa_docker_sonic_mgmt.pub' -> '/tmp/tmp.Ssj2kL0x43/id_rsa.pub'
[+] Building 66.9s (27/27) FINISHED                                                                                                                                                                                                                                               docker:default
 => [internal] load build definition from Dockerfile                                                                                                                                                                                                                                        0.1s
 => => transferring dockerfile: 2.31kB                                                                                                                                                                                                                                                      0.0s
 => [internal] load .dockerignore                                                                                                                                                                                                                                                           0.1s
 => => transferring context: 2B                                                                                                                                                                                                                                                             0.0s
 => [internal] load metadata for sonicdev-microsoft.azurecr.io:443/docker-sonic-mgmt:latest                                                                                                                                                                                                 0.0s
 => [ 1/22] FROM sonicdev-microsoft.azurecr.io:443/docker-sonic-mgmt                                                                                                                                                                                                                        0.0s
 => [internal] load build context                                                                                                                                                                                                                                                           0.1s
 => => transferring context: 3.25kB                                                                                                                                                                                                                                                         0.0s
 => CACHED [ 2/22] RUN if getent group zegan; then groupmod -o -g 1000 zegan; else groupadd -o -g 1000 zegan; fi                                                                                                                                                                            0.0s
 => CACHED [ 3/22] RUN if getent passwd zegan; then userdel zegan; fi                                                                                                                                                                                                                       0.0s
 => CACHED [ 4/22] RUN useradd -o -l -g 1000 -u 1000 -m -d /home/zegan -s /bin/bash zegan;                                                                                                                                                                                                  0.0s
 => CACHED [ 5/22] RUN if getent group docker; then groupmod -o -g 998 docker; else groupadd -o -g 998 docker; fi                                                                                                                                                                           0.0s
 => CACHED [ 6/22] RUN if [ 'zegan' != 'AzDevOps' ]; then /bin/bash -O extglob -c 'cp -a -f /var/AzDevOps/!(env-*) /home/zegan/'; for hidden_stuff in '.profile .local .ssh'; do /bin/bash -c 'cp -a -f /var/AzDevOps/$hidden_stuff /home/zegan/ || true'; done fi                          0.0s
 => CACHED [ 7/22] RUN usermod -a -G sudo zegan                                                                                                                                                                                                                                             0.0s
 => CACHED [ 8/22] RUN usermod -a -G docker zegan                                                                                                                                                                                                                                           0.0s
 => CACHED [ 9/22] RUN echo 'zegan ALL=(ALL) NOPASSWD:ALL' > /etc/sudoers.d/zegan                                                                                                                                                                                                           0.0s
 => CACHED [10/22] RUN chmod 0440 /etc/sudoers.d/zegan                                                                                                                                                                                                                                      0.0s
 => CACHED [11/22] RUN chown -R '1000:1000' /home/zegan                                                                                                                                                                                                                                     0.0s
 => CACHED [12/22] RUN sed -i -E 's/^#?PermitRootLogin.*$/PermitRootLogin yes/g' /etc/ssh/sshd_config                                                                                                                                                                                       0.0s
 => CACHED [13/22] RUN echo 'root:root' | chpasswd                                                                                                                                                                                                                                          0.0s
 => CACHED [14/22] RUN echo 'zegan:12345' | chpasswd                                                                                                                                                                                                                                        0.0s
 => CACHED [15/22] COPY --chown=1000:1000 id_rsa id_rsa.pub /home/zegan/.ssh/                                                                                                                                                                                                               0.0s
 => CACHED [16/22] RUN chmod 0700 /home/zegan/.ssh                                                                                                                                                                                                                                          0.0s
 => CACHED [17/22] RUN chmod 0600 /home/zegan/.ssh/id_rsa                                                                                                                                                                                                                                   0.0s
 => CACHED [18/22] RUN chmod 0644 /home/zegan/.ssh/id_rsa.pub                                                                                                                                                                                                                               0.0s
 => CACHED [19/22] RUN cat /home/zegan/.ssh/id_rsa.pub >> /home/zegan/.ssh/authorized_keys                                                                                                                                                                                                  0.0s
 => CACHED [20/22] RUN chmod 0600 /home/zegan/.ssh/authorized_keys                                                                                                                                                                                                                          0.0s
 => CACHED [21/22] WORKDIR /home/zegan                                                                                                                                                                                                                                                      0.0s
 => [22/22] RUN if [ 'zegan' != 'AzDevOps' ] && [ -d /var/AzDevOps/env-python3 ]; then /bin/bash -c 'python3 -m venv ${HOME}/env-python3'; /bin/bash -c '${HOME}/env-python3/bin/pip install pip --upgrade'; /bin/bash -c '${HOME}/env-python3/bin/pip install wheel'; /bin/bash -c '${HO  61.1s
 => exporting to image                                                                                                                                                                                                                                                                      5.6s
 => => exporting layers                                                                                                                                                                                                                                                                     5.6s
 => => writing image sha256:fd47f5fa0ac90e0f2084037cb9f7fe85e48577c3854031578e585fbcb6630c03                                                                                                                                                                                                0.0s
 => => naming to docker.io/library/docker-sonic-mgmt-zegan:master                                                                                                                                                                                                                           0.0s
b419771c024f1b8dc5e59270f24bf55df86e695e62da7fdb4c3d09d62417bc06
 * Restarting OpenBSD Secure Shell server sshd
   ...done.
******************************************************************************
EXEC: docker exec --user zegan -ti sonic-mgmt bash
SSH:  ssh -i ~/.ssh/id_rsa_docker_sonic_mgmt [email protected]
******************************************************************************


Signed-off-by: Ze Gan <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants