Skip to content

Fix ctrmgrd occasionally restart service problem#13342

Closed
kuanyu99 wants to merge 1 commit intosonic-net:masterfrom
kuanyu99:fix_ctrmgrd_restart_service
Closed

Fix ctrmgrd occasionally restart service problem#13342
kuanyu99 wants to merge 1 commit intosonic-net:masterfrom
kuanyu99:fix_ctrmgrd_restart_service

Conversation

@kuanyu99
Copy link
Copy Markdown
Contributor

Why I did it

After I enable the Kubernetes on the SONiC, the database container keeps restarting.
And I will keep seeing the following message.

Jan 11 03:07:33.820227 sonic DEBUG ctrmgrd.py: run: Received message : '('database', 'SET', (('state', 'enabled'),))'
Jan 11 03:07:33.820655 sonic DEBUG ctrmgrd.py: on_state_update: database init=True old_remote_state= remote_state=none
Jan 11 03:07:33.842777 sonic DEBUG ctrmgrd.py: is_systemd_active: system status for database: 0
Jan 11 03:07:33.842894 sonic DEBUG ctrmgrd.py: handle_update: feat=database set=local ct=none state=none restart=True label_add=False
Jan 11 03:07:33.842975 sonic DEBUG ctrmgrd.py: mod_db_entry: mod_db_entry: db=STATE_DB tbl=KUBE_LABELS key=SET data={'database_enabled': 'false'}
Jan 11 03:07:33.843618 sonic DEBUG ctrmgrd.py: restart_systemd_service: Restart service database to owner:local

How I did it

Due to the current_owner "none" is the default state for the container, I intend to make the ctrmgrd skip changing the state of the container. This operation should be handled by the "container" script instead of the "ctrmgrd.py".

In the set_owner="kube" condition, it also ignores the case when the current_owner equal to "none".

How to verify it

After the change, the containers will not be restarted by the ctrmgrd.py unintentionally.

Which release branch to backport (provide reason below if selected)

  • 201811
  • 201911
  • 202006
  • 202012
  • 202106
  • 202111
  • 202205
  • 202211

Description for the changelog

  • Originally, the container script will set the current_owner to none when it try to stop that feature. Later, when the container script want to start the feature, it will set local/kube as the owner and then start it. However, the ctrmgrd.py will do the same thing for the feature. It will try to restart the feature when the owner changed from none to local. The function of ctrmgrd is duplicate of the container script.

  • Prevent the ctrmgrd.py from restarting the feature when the owner is changed from none to local.

Ensure to add label/tag for the feature raised. example - PR#2174 under sonic-utilities repo. where, Generic Config and Update feature has been labelled as GCU.

Link to config_db schema for YANG module changes

A picture of a cute animal (not mandatory but encouraged)

* Originally, the container script will set the current_owner to none when
  it try to stop that feature.
  Later, when the container script want to start the feature, it will set
  local/kube as the owner and then start it.
  However, the ctrmgrd.py will do the same thing for the feature. It will
  try to restart the feature when the owner changed from none to local.
  The function of ctrmgrd is duplicate of the container script.

* Prevent the ctrmgrd.py from restarting the feature when the owner is
  changed from none to local.
@kuanyu99 kuanyu99 requested a review from lguohan as a code owner January 12, 2023 02:12
@kuanyu99
Copy link
Copy Markdown
Contributor Author

I found the PR-#12997 can fix the same problem.
Close this PR.

@kuanyu99 kuanyu99 closed this Jan 16, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant