Skip to content

[multi-asic][warm_restart] add Multi-ASIC support for warm_restart commands #99

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
stepanblyschak wants to merge 2 commits into from
Closed

[multi-asic][warm_restart] add Multi-ASIC support for warm_restart commands #99

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
9a2a040
[warm_restart] add Multi-ASIC support for warm_restart commands
stepanblyschak Oct 7, 2025
68607ee
fix tests after rebase
stepanblyschak Dec 26, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
148 changes: 106 additions & 42 deletions config/main.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3812,92 +3812,156 @@ def warm_restart(ctx, redis_unix_socket_path):
# Note: redis_unix_socket_path is a path string, and the ground truth is now from database_config.json.
# We only use it as a bool indicator on either unix_socket_path or tcp port
use_unix_socket_path = bool(redis_unix_socket_path)
config_db = ConfigDBConnector(use_unix_socket_path=use_unix_socket_path)
config_db.connect(wait_for_init=False)

# warm restart enable/disable config is put in stateDB, not persistent across cold reboot, not saved to config_DB.json file
state_db = SonicV2Connector(use_unix_socket_path=use_unix_socket_path)
state_db.connect(state_db.STATE_DB, False)
TABLE_NAME_SEPARATOR = '|'
prefix = 'WARM_RESTART_ENABLE_TABLE' + TABLE_NAME_SEPARATOR
ctx.obj = {'db': config_db, 'state_db': state_db, 'prefix': prefix}
ctx.obj = {'prefix': prefix}

asic_namespaces = multi_asic.get_namespace_list()
all_namespaces = asic_namespaces
if multi_asic.is_multi_asic():
all_namespaces = [multi_asic_util.constants.DEFAULT_NAMESPACE] + asic_namespaces
ctx.obj["all_namespaces"] = all_namespaces
ctx.obj["asic_namespaces"] = asic_namespaces
ctx.obj["state_db"] = {}
ctx.obj["config_db"] = {}
for namespace in all_namespaces:
config_db = ConfigDBConnector(namespace=namespace, use_unix_socket_path=use_unix_socket_path);
config_db.connect(wait_for_init=False)
state_db = SonicV2Connector(namespace=namespace, use_unix_socket_path=use_unix_socket_path);
state_db.connect(state_db.STATE_DB, False)
ctx.obj["state_db"][namespace] = state_db
ctx.obj["config_db"][namespace] = config_db


@warm_restart.command('enable')
@click.option('--namespace', '-n', 'namespace', default=None, help='Namespace name')
Copy link
Copy Markdown

@oleksandrivantsiv oleksandrivantsiv Dec 19, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you use this decorator instead, you won't namespace validation inside funtion:

@click.option('-n', '--namespace', help='Namespace name', required=True if multi_asic.is_multi_asic() else False,
              type=click.Choice(multi_asic.get_namespace_list()), default=multi_asic.DEFAULT_NAMESPACE)

Copy link
Copy Markdown

@oleksandrivantsiv oleksandrivantsiv Dec 19, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will reduce the number of changes in this file

Copy link
Copy Markdown
Owner Author

@stepanblyschak stepanblyschak Dec 26, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@oleksandrivantsiv I see the benefit of doing this, but not sure how to mock multi_asic functions later in a test case. Since this decorator runs on module import time, it likelly needs reloading module after mock setup. I tried this but seems this does not play well with ClickRunner I will check what I can do.

Copy link
Copy Markdown
Owner Author

@stepanblyschak stepanblyschak Jan 9, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@oleksandrivantsiv I find it less suitable for this command. The choice enumerates asic namesapces while my commands use global and asic namespaces. Also makes mocking harder
The only benefit of choice is CLI feedback, but this is not a user facing CLI

@click.argument('module', metavar='<module>', default='system', required=False)
@click.pass_context
def warm_restart_enable(ctx, module):
state_db = ctx.obj['state_db']
config_db = ctx.obj['db']
def warm_restart_enable(ctx, namespace, module):
if namespace is not None:
Copy link
Copy Markdown

@oleksandrivantsiv oleksandrivantsiv Dec 19, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

minor: if namespace and namespace not in ctx.obj["all_namespaces"]

Copy link
Copy Markdown
Owner Author

@stepanblyschak stepanblyschak Jan 9, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

need check None, which means user did not provide. '' - is global ns

if namespace not in ctx.obj["all_namespaces"]:
raise click.UsageError("Invalid namespace: {}".format(namespace))
namespaces = [namespace] if namespace else ctx.obj["all_namespaces"]

config_db = ctx.obj["config_db"][multi_asic_util.constants.DEFAULT_NAMESPACE]
feature_table = config_db.get_table('FEATURE')
if module != 'system' and module not in feature_table:
sys.exit('Feature {} is unknown'.format(module))
prefix = ctx.obj['prefix']
_hash = '{}{}'.format(prefix, module)
state_db.set(state_db.STATE_DB, _hash, 'enable', 'true')
state_db.close(state_db.STATE_DB)

for namespace in namespaces:
state_db = ctx.obj["state_db"][namespace]
state_db.set(state_db.STATE_DB, _hash, 'enable', 'true')
state_db.close(state_db.STATE_DB)


@warm_restart.command('disable')
@click.option('--namespace', '-n', 'namespace', default=None, help='Namespace name')
@click.argument('module', metavar='<module>', default='system', required=False)
@click.pass_context
def warm_restart_disable(ctx, module):
state_db = ctx.obj['state_db']
config_db = ctx.obj['db']
def warm_restart_disable(ctx, namespace, module):
if namespace is not None:
if namespace not in ctx.obj["all_namespaces"]:
raise click.UsageError("Invalid namespace: {}".format(namespace))
namespaces = [namespace] if namespace else ctx.obj["all_namespaces"]

config_db = ctx.obj["config_db"][multi_asic_util.constants.DEFAULT_NAMESPACE]
feature_table = config_db.get_table('FEATURE')
if module != 'system' and module not in feature_table:
sys.exit('Feature {} is unknown'.format(module))
prefix = ctx.obj['prefix']
_hash = '{}{}'.format(prefix, module)
state_db.set(state_db.STATE_DB, _hash, 'enable', 'false')
state_db.close(state_db.STATE_DB)

for namespace in namespaces:
state_db = ctx.obj["state_db"][namespace]
state_db.set(state_db.STATE_DB, _hash, 'enable', 'false')
state_db.close(state_db.STATE_DB)


@warm_restart.command('neighsyncd_timer')
@click.option('--namespace', '-n', 'namespace', default=None, help='Namespace name')
@click.argument('seconds', metavar='<seconds>', required=True, type=int)
@click.pass_context
def warm_restart_neighsyncd_timer(ctx, seconds):
db = ValidatedConfigDBConnector(ctx.obj['db'])
def warm_restart_neighsyncd_timer(ctx, namespace, seconds):
if namespace is not None:
if namespace not in ctx.obj["asic_namespaces"]:
raise click.UsageError("Invalid namespace: {}".format(namespace))
namespaces = [namespace] if namespace else ctx.obj["asic_namespaces"]

if ADHOC_VALIDATION:
if seconds not in range(1, 9999):
ctx.fail("neighsyncd warm restart timer must be in range 1-9999")
try:
db.mod_entry('WARM_RESTART', 'swss', {'neighsyncd_timer': seconds})
except ValueError as e:
ctx.fail("Invalid ConfigDB. Error: {}".format(e))

for namespace in namespaces:
db = ValidatedConfigDBConnector(ctx.obj["config_db"][namespace])
try:
db.mod_entry('WARM_RESTART', 'swss', {'neighsyncd_timer': seconds})
except ValueError as e:
ctx.fail("Invalid ConfigDB. Error: {}".format(e))


@warm_restart.command('bgp_timer')
@click.option('--namespace', '-n', 'namespace', default=None, help='Namespace name')
@click.argument('seconds', metavar='<seconds>', required=True, type=int)
@click.pass_context
def warm_restart_bgp_timer(ctx, seconds):
db = ValidatedConfigDBConnector(ctx.obj['db'])
def warm_restart_bgp_timer(ctx, namespace, seconds):
if namespace is not None:
if namespace not in ctx.obj["asic_namespaces"]:
raise click.UsageError("Invalid namespace: {}".format(namespace))
namespaces = [namespace] if namespace else ctx.obj["asic_namespaces"]

if ADHOC_VALIDATION:
if seconds not in range(1, 3600):
ctx.fail("bgp warm restart timer must be in range 1-3600")
try:
db.mod_entry('WARM_RESTART', 'bgp', {'bgp_timer': seconds})
except ValueError as e:
ctx.fail("Invalid ConfigDB. Error: {}".format(e))

for namespace in namespaces:
db = ValidatedConfigDBConnector(ctx.obj["config_db"][namespace])
try:
db.mod_entry('WARM_RESTART', 'bgp', {'bgp_timer': seconds})
except ValueError as e:
ctx.fail("Invalid ConfigDB. Error: {}".format(e))


@warm_restart.command('teamsyncd_timer')
@click.option('--namespace', '-n', 'namespace', default=None, help='Namespace name')
@click.argument('seconds', metavar='<seconds>', required=True, type=int)
@click.pass_context
def warm_restart_teamsyncd_timer(ctx, seconds):
db = ValidatedConfigDBConnector(ctx.obj['db'])
def warm_restart_teamsyncd_timer(ctx, namespace, seconds):
if namespace is not None:
if namespace not in ctx.obj["asic_namespaces"]:
raise click.UsageError("Invalid namespace: {}".format(namespace))
namespaces = [namespace] if namespace else ctx.obj["asic_namespaces"]

if ADHOC_VALIDATION:
if seconds not in range(1, 3600):
ctx.fail("teamsyncd warm restart timer must be in range 1-3600")
try:
db.mod_entry('WARM_RESTART', 'teamd', {'teamsyncd_timer': seconds})
except ValueError as e:
ctx.fail("Invalid ConfigDB. Error: {}".format(e))
ctx.fail("bgp warm restart timer must be in range 1-3600")

for namespace in namespaces:
db = ValidatedConfigDBConnector(ctx.obj["config_db"][namespace])
try:
db.mod_entry('WARM_RESTART', 'teamd', {'teamsyncd_timer': seconds})
except ValueError as e:
ctx.fail("Invalid ConfigDB. Error: {}".format(e))


@warm_restart.command('bgp_eoiu')
@click.option('--namespace', '-n', 'namespace', default=None, help='Namespace name')
@click.argument('enable', metavar='<enable>', default='true', required=False, type=click.Choice(["true", "false"]))
@click.pass_context
def warm_restart_bgp_eoiu(ctx, enable):
db = ValidatedConfigDBConnector(ctx.obj['db'])
try:
db.mod_entry('WARM_RESTART', 'bgp', {'bgp_eoiu': enable})
except ValueError as e:
ctx.fail("Invalid ConfigDB. Error: {}".format(e))
def warm_restart_bgp_eoiu(ctx, namespace, enable):
if namespace is not None:
if namespace not in ctx.obj["asic_namespaces"]:
raise click.UsageError("Invalid namespace: {}".format(namespace))
namespaces = [namespace] if namespace else ctx.obj["asic_namespaces"]

for namespace in namespaces:
db = ValidatedConfigDBConnector(ctx.obj["config_db"][namespace])
try:
db.mod_entry('WARM_RESTART', 'bgp', {'bgp_eoiu': enable})
except ValueError as e:
ctx.fail("Invalid ConfigDB. Error: {}".format(e))


def vrf_add_management_vrf(config_db):
"""Enable management vrf in config DB"""
Expand Down
62 changes: 55 additions & 7 deletions show/warm_restart.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
import click
import utilities_common.cli as clicommon
import utilities_common.multi_asic as multi_asic_util
from sonic_py_common import multi_asic
from swsscommon.swsscommon import SonicV2Connector, ConfigDBConnector
from tabulate import tabulate

Expand All @@ -11,14 +13,38 @@ def warm_restart():


@warm_restart.command()
@click.option('--namespace', '-n', 'namespace', default=None, help='Namespace name')
@click.option('-s', '--redis-unix-socket-path', help='unix socket path for redis connection')
def state(redis_unix_socket_path):
def state(namespace, redis_unix_socket_path):
"""Show warm restart state"""
kwargs = {}

if redis_unix_socket_path:
kwargs['unix_socket_path'] = redis_unix_socket_path
click.secho(
"Warning: '-s|--redis-unix-socket-path' has no effect and is left for compatibility",
fg="red", err=True)

if namespace and namespace not in multi_asic.get_namespace_list():
raise click.UsageError("Invalid namespace: {}".format(namespace))

asic_namespaces = multi_asic.get_namespace_list()
all_namespaces = asic_namespaces
if multi_asic.is_multi_asic():
all_namespaces = [multi_asic_util.constants.DEFAULT_NAMESPACE] + asic_namespaces
if namespace is not None and namespace not in asic_namespaces:
raise click.UsageError("Invalid namespace: {}".format(namespace))

namespaces = [namespace] if namespace else all_namespaces

for namespace in namespaces:
if len(namespaces) > 1:
namespace_str = namespace or "global"
click.echo(f"\nFor namespace {namespace_str}:\n")

show_warm_restart_state_for_namespace(namespace)

db = SonicV2Connector(host='127.0.0.1')

def show_warm_restart_state_for_namespace(namespace):
db = SonicV2Connector(namespace=namespace)
db.connect(db.STATE_DB, False) # Make one attempt only

TABLE_NAME_SEPARATOR = '|'
Expand Down Expand Up @@ -53,19 +79,41 @@ def remove_prefix(text, prefix):


@warm_restart.command()
@click.option('--namespace', '-n', 'namespace', default=None, help='Namespace name')
@click.option('-s', '--redis-unix-socket-path', help='unix socket path for redis connection')
def config(redis_unix_socket_path):
def config(namespace, redis_unix_socket_path):
"""Show warm restart config"""
kwargs = {}
if redis_unix_socket_path:
kwargs['unix_socket_path'] = redis_unix_socket_path
config_db = ConfigDBConnector(**kwargs)

if namespace is not None and redis_unix_socket_path:
raise click.UsageError("Cannot specify both namespace and redis unix socket path")

asic_namespaces = multi_asic.get_namespace_list()
all_namespaces = asic_namespaces
if multi_asic.is_multi_asic():
all_namespaces = [multi_asic_util.constants.DEFAULT_NAMESPACE] + asic_namespaces
if namespace is not None and namespace not in asic_namespaces:
raise click.UsageError("Invalid namespace: {}".format(namespace))

namespaces = [namespace] if namespace else all_namespaces

for namespace in namespaces:
if len(namespaces) > 1:
namespace_str = namespace or "global"
click.echo(f"\nFor namespace {namespace_str}:\n")

show_warm_restart_config_for_namespace(namespace, **kwargs)

def show_warm_restart_config_for_namespace(namespace, **kwargs):
config_db = ConfigDBConnector(namespace=namespace, **kwargs)
config_db.connect(wait_for_init=False)
data = config_db.get_table('WARM_RESTART')
# Python dictionary keys() Method
keys = list(data.keys())

state_db = SonicV2Connector(host='127.0.0.1')
state_db = SonicV2Connector(namespace=namespace)
state_db.connect(state_db.STATE_DB, False) # Make one attempt only
TABLE_NAME_SEPARATOR = '|'
prefix = 'WARM_RESTART_ENABLE_TABLE' + TABLE_NAME_SEPARATOR
Expand Down
Loading
Loading