Skip to content

[202012][dockers][supervisor] Increase event buffer size for process exit listener; Set all event buffer sizes to 1024#7203

Merged
jleveque merged 1 commit intosonic-net:202012from
jleveque:inc_sup_event_buffer_202012
Apr 1, 2021
Merged

[202012][dockers][supervisor] Increase event buffer size for process exit listener; Set all event buffer sizes to 1024#7203
jleveque merged 1 commit intosonic-net:202012from
jleveque:inc_sup_event_buffer_202012

Conversation

@jleveque
Copy link
Copy Markdown
Contributor

Why I did it

Backport of #7083 to the 202012 branch.

To prevent error messages like the following from being logged:

Mar 17 02:33:48.523153 vlab-01 INFO swss#supervisord 2021-03-17 02:33:48,518 ERRO pool supervisor-proc-exit-listener event buffer overflowed, discarding event 46

This is basically an addendum to #5247, which increased the event buffer size for dependent-startup. While supervisor-proc-exit-listener doesn't subscribe to as many events as dependent-startup, there is still a chance some containers (like swss, as in the example above) have enough processes running to cause an overflow of the default buffer size of 10.

This is especially important for preventing erroneous log_analyzer failures in the sonic-mgmt repo regression tests, which have started occasionally causing PR check builds to fail. Example here.

I set all supervisor-proc-exit-listener event buffer sizes to 1024, and also updated all dependent-startup event buffer sizes to 1024, as well, to keep things simple, unified, and allow headroom so that we will not need to adjust these values frequently, if at all.

…exit listener; Set all event buffer sizes to 1024
@jleveque
Copy link
Copy Markdown
Contributor Author

/azp run

@azure-pipelines
Copy link
Copy Markdown

Azure Pipelines successfully started running 1 pipeline(s).

@jleveque jleveque merged commit dd9be59 into sonic-net:202012 Apr 1, 2021
@jleveque jleveque deleted the inc_sup_event_buffer_202012 branch April 1, 2021 19:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants