Skip to content

[docker-base]: Rate limit priority INFO and lower in syslog#5666

Merged
theasianpianist merged 2 commits intosonic-net:masterfrom
theasianpianist:log-rate-limit
Oct 20, 2020
Merged

[docker-base]: Rate limit priority INFO and lower in syslog#5666
theasianpianist merged 2 commits intosonic-net:masterfrom
theasianpianist:log-rate-limit

Conversation

@theasianpianist
Copy link
Copy Markdown
Contributor

@theasianpianist theasianpianist commented Oct 19, 2020

Signed-off-by: Lawrence Lee lawlee@microsoft.com

Temporary fix for #5570

- Why I did it
There is currently a bug where messages from swss with priority lower than the current log level are still being counted against the syslog rate limiting threshhold. This leads to rate-limiting in syslog when the rate-limiting conditions have not been met, which causes several sonic-mgmt tests to fail since they are dependent on LogAnalyzer. It also omits potentially useful information from the syslog. Only rate-limiting messages of level INFO and lower allows these tests to pass successfully.

- How I did it
Add $SystemLogRateLimitSeverity option to rsyslog.conf in the base Dockerfiles

- How to verify it
Build a new orchagent image and install it. Run the ACL tests, which should now pass. No rate limiting should occur in the syslogs.

- Which release branch to backport (provide reason below if selected)

  • 201811
  • 201911
  • 202006

- Description for the changelog

- A picture of a cute animal (not mandatory but encouraged)

Signed-off-by: Lawrence Lee <lawlee@microsoft.com>
Copy link
Copy Markdown
Contributor

@jleveque jleveque left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you also please update the related comments to read:

# Set a rate limit on messages from the container of INFO level (level 6) or lower

@jleveque
Copy link
Copy Markdown
Contributor

You may also want to make the same change here: files/image_config/rsyslog/rsyslog-container.conf.j2

* Update comments to reflect new behavior
* Also change files/image_config/rsyslog/rsyslog-container.conf.j2

Signed-off-by: Lawrence Lee <lawlee@microsoft.com>
@theasianpianist theasianpianist merged commit 207587d into sonic-net:master Oct 20, 2020
@theasianpianist theasianpianist deleted the log-rate-limit branch October 20, 2020 18:52
santhosh-kt pushed a commit to santhosh-kt/sonic-buildimage that referenced this pull request Feb 25, 2021
…t#5666)

There is currently a bug where messages from swss with priority lower than the current log level are still being counted against the syslog rate limiting threshhold. This leads to rate-limiting in syslog when the rate-limiting conditions have not been met, which causes several sonic-mgmt tests to fail since they are dependent on LogAnalyzer. It also omits potentially useful information from the syslog. Only rate-limiting messages of level INFO and lower allows these tests to pass successfully.

Signed-off-by: Lawrence Lee <lawlee@microsoft.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

orchagent continuously prints out "set: setting attribute 0x10000004 status: SAI_STATUS_SUCCESS"

3 participants