Skip to content

[pytest] Test the feature of monitoring critical processes by Monit.#3398

Merged
yozhao101 merged 6 commits intosonic-net:masterfrom
yozhao101:monitoring_processes_by_monit
May 5, 2021
Merged

[pytest] Test the feature of monitoring critical processes by Monit.#3398
yozhao101 merged 6 commits intosonic-net:masterfrom
yozhao101:monitoring_processes_by_monit

Conversation

@yozhao101
Copy link
Contributor

Signed-off-by: Yong Zhao [email protected]

Description of PR

Summary:
Fixes # (issue)

Type of change

  • Bug fix
  • Testbed and Framework(new/improvement)
  • [x ] Test case(new/improvement)

Approach

What is the motivation for this PR?

This PR aims to test the feature of monitoring critical processes by Monit in 20191130 image.

How did you do it?

The logic of this script is:
Step 1: Manually generate the expected regex of alerting messages for critical processes of containers in each namespace
Step 2: Kill each critical process one by one of containers in namespaces
Step 3: Wait for 70 seconds and loganalyzer will check whether the alerting messages fired by Monit in syslog match the regex created at Step 1.
Step 4: Post-check and restart containers by running the command sudo config reload.

How did you verify/test it?

I tested this pytest script on physical devices: str-n3164-acs-2 (Multi-ASIC) and str-msn2700-03 (Single ASIC). For the device
str-n3164-acs-2, I tested the 20191130 image. For device str-msn2700-03, I tested 202012 and 20191130.70 images

Any platform specific information?

N/A

Supported testbed topology if it's a new test case?

Documentation

@yozhao101 yozhao101 requested a review from a team as a code owner April 22, 2021 21:42
@yozhao101 yozhao101 requested review from jleveque and yxieca April 22, 2021 21:51
@lgtm-com
Copy link

lgtm-com bot commented May 4, 2021

This pull request introduces 1 alert and fixes 1 when merging 1030352 into dfaceac - view on LGTM.com

new alerts:

  • 1 for Syntax error

fixed alerts:

  • 1 for Unused import

Signed-off-by: Yong Zhao <[email protected]>
@lgtm-com
Copy link

lgtm-com bot commented May 4, 2021

This pull request fixes 1 alert when merging 4d36c90 into dfaceac - view on LGTM.com

fixed alerts:

  • 1 for Unused import

@jleveque jleveque added the New label May 4, 2021
@lgtm-com
Copy link

lgtm-com bot commented May 4, 2021

This pull request fixes 1 alert when merging 3acddcb into 058fc1c - view on LGTM.com

fixed alerts:

  • 1 for Unused import

@yozhao101 yozhao101 merged commit b4a6c2f into sonic-net:master May 5, 2021
@yozhao101 yozhao101 deleted the monitoring_processes_by_monit branch May 5, 2021 06:38
saravanansv pushed a commit to saravanansv/sonic-mgmt that referenced this pull request May 6, 2021
…onic-net#3398)

What is the motivation for this PR?
This PR aims to test the feature of monitoring critical processes by Monit in 20191130 image.

How did you do it?
The logic of this script is:
Step 1: Manually generate the expected regex of alerting messages for critical processes of containers in each namespace
Step 2: Kill each critical process one by one of containers in namespaces
Step 3: Wait for 70 seconds and loganalyzer will check whether the alerting messages fired by Monit in syslog match the regex created at Step 1.
Step 4: Post-check and restart containers by running the command sudo config reload.

How did you verify/test it?
I tested this pytest script on physical devices: str-n3164-acs-2 (Multi-ASIC) and str-msn2700-03 (Single ASIC). For the device
str-n3164-acs-2, I tested the 20191130 image. For device str-msn2700-03, I tested 202012 and 20191130.70 images

Any platform specific information?
N/A
bingwang-ms pushed a commit to bingwang-ms/sonic-mgmt that referenced this pull request Aug 18, 2021
We are migrating from Jenkins to azure pipeline. This PR added azure pipeline yaml files and dependent template files for nightly tests. Pipeline yaml files only added for 3 testbeds yet. The pipelines were originally added to branch azp-test of repo https://dev.azure.com/mssonic/internal/_git/sonic-mgmt-int. They have been tested on Azure DevOps.

Now we formally add these pipelines to the Networking-acs-sonic-mgmt repo. Currently the internal branch of Networking-acs-sonic-mgmt is synched to same internal branch of the sonic-mgmt-int repo. After this PR is merged, we need to update configuration of the created pipelines to formally use these files from the internal branch.

If the pipelines work fine with yaml files from internal branch, we will create more pipeline files based on current templates for rest of the nightly testbeds.

Related work items: sonic-net#3021, sonic-net#3073, sonic-net#3135, sonic-net#3153, sonic-net#3162, sonic-net#3176, sonic-net#3238, sonic-net#3241, sonic-net#3346, sonic-net#3352, sonic-net#3378, sonic-net#3389, sonic-net#3395, sonic-net#3397, sonic-net#3398, sonic-net#3407, sonic-net#3410, sonic-net#3411, sonic-net#3412, sonic-net#3413, sonic-net#3414, sonic-net#3415, sonic-net#3434, sonic-net#3437, sonic-net#3445, sonic-net#3446, sonic-net#3447, #9740131, #9821349
vmittal-msft pushed a commit to vmittal-msft/sonic-mgmt that referenced this pull request Sep 28, 2021
…onic-net#3398)

What is the motivation for this PR?
This PR aims to test the feature of monitoring critical processes by Monit in 20191130 image.

How did you do it?
The logic of this script is:
Step 1: Manually generate the expected regex of alerting messages for critical processes of containers in each namespace
Step 2: Kill each critical process one by one of containers in namespaces
Step 3: Wait for 70 seconds and loganalyzer will check whether the alerting messages fired by Monit in syslog match the regex created at Step 1.
Step 4: Post-check and restart containers by running the command sudo config reload.

How did you verify/test it?
I tested this pytest script on physical devices: str-n3164-acs-2 (Multi-ASIC) and str-msn2700-03 (Single ASIC). For the device
str-n3164-acs-2, I tested the 20191130 image. For device str-msn2700-03, I tested 202012 and 20191130.70 images

Any platform specific information?
N/A
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants