Skip to content

[orchagent.sh] mask SIGHUP before starting orchagent#22207

Merged
yxieca merged 1 commit intosonic-net:masterfrom
stepanblyschak:mask-sighup
Apr 30, 2025
Merged

[orchagent.sh] mask SIGHUP before starting orchagent#22207
yxieca merged 1 commit intosonic-net:masterfrom
stepanblyschak:mask-sighup

Conversation

@stepanblyschak
Copy link
Collaborator

@stepanblyschak stepanblyschak commented Apr 2, 2025

Why I did it

FIXES #21962

logrotate uses SIGHUP to let orchagent reopen SAI Redis recording. If SIGHUP is sent by logrotate before orchagent register its handler the default action takes place - termination. If we mask the signal in the start script orchagent inherits masked signals and will ignore SIGHUP until signal handler is registered.

This is a rare race condition observed during 1000 warm-reboot testing.

Work item tracking
  • Microsoft ADO (number only):

How I did it

Mask SIGHUP

How to verify it

Place sleep inside orchagent before signal handler for SIGHUP is registered. Send SIGHUP as if it would be sent from logrotate script, ensure no orchagent termination.

Which release branch to backport (provide reason below if selected)

  • 201811
  • 201911
  • 202006
  • 202012
  • 202106
  • 202111
  • 202205
  • 202211
  • 202305
  • 202405
  • 202411

Request to backport to all active branches.

Tested branch (Please provide the tested image version)

Description for the changelog

Link to config_db schema for YANG module changes

A picture of a cute animal (not mandatory but encouraged)

logrotate uses SIGHUP to let orchagent rotate SAI Redis recording. If
SIGHUP is sent by logrotate before orchagent register its handler the
default action takes place - termination. If we mask the signal in the
start script orchagent inherits masked signals and will ignore SIGHUP
until signal handler is registered.

This is a rare race condition observed during 1000 warm-reboot testing.

Signed-off-by: Stepan Blyschak <[email protected]>
@stepanblyschak stepanblyschak requested a review from lguohan as a code owner April 2, 2025 09:17
@mssonicbld
Copy link
Collaborator

/azp run Azure.sonic-buildimage

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

Copy link
Contributor

@prsunny prsunny left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@prabhataravind for viz

@bingwang-ms
Copy link
Contributor

@prsunny, @stepanblyschak Will the signal SIGHUP be trapped by orchagent.sh forever? If so, does that cause trouble?

@stepanblyschak
Copy link
Collaborator Author

@bingwang-ms If SIGHUP is sent to orchagent before handlers are registered the orchagent will terminate

@yxieca yxieca merged commit be01013 into sonic-net:master Apr 30, 2025
21 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[swss] Orchagent terminated by SIGHUP because logrotate sent SIGHUP on boot after 202405->202411 warm upgrade

7 participants