Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
STARTUP_TSB_TIMER=900
14 changes: 14 additions & 0 deletions dockers/docker-fpm-frr/base_image_files/TS
Original file line number Diff line number Diff line change
Expand Up @@ -65,3 +65,17 @@ else
docker exec -i bgp /usr/bin/$1
fi
fi

# If TSA was issued by startup_tsa_tsb.service, show it
if [[ $1 == "TSC" ]]; then
service='startup_tsa_tsb.service'
if [[ $(/bin/systemctl show $service --property ActiveState --value) == "active" ]] && \
[[ $(/bin/systemctl show $service --property SubState --value) == "running" ]]; then
TSA_TSB_CONF=/usr/share/sonic/device/$PLATFORM/startup-tsa-tsb.conf
[ -f $TSA_TSB_CONF ] && . $TSA_TSB_CONF
service_start_sec=$(date -d "$(systemctl show --property=ActiveEnterTimestamp $service --value)" +%s)
service_elapsed_sec=$(( $(date +%s) - service_start_sec))
tsb_time_remaining=$((STARTUP_TSB_TIMER - service_elapsed_sec))
echo "TSB : Pending (Time Remaining:$tsb_time_remaining seconds, service:$service)"
fi
fi
11 changes: 10 additions & 1 deletion dockers/docker-fpm-frr/base_image_files/TSA
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,15 @@ then
sudo config mux mode standby all
fi

if [ -z "$STARTED_BY_TSA_TSB_SERVICE" ]; then
service='startup_tsa_tsb.service'
if [[ $(/bin/systemctl show $service --property ActiveState --value) == "active" ]] && \
[[ $(/bin/systemctl show $service --property SubState --value) == "running" ]]; then
echo "Stopping $service before configuring TSA"
systemctl stop $service
fi
fi

/usr/bin/TS TSA
if [[ "$(sonic-cfggen -d -v DEVICE_METADATA.localhost.type)" == *"SpineRouter"* ]] ; then
if [[ "$1" != "chassis" ]] ; then
Expand All @@ -29,4 +38,4 @@ if [[ "$(sonic-cfggen -d -v DEVICE_METADATA.localhost.type)" == *"SpineRouter"*
fi
else
echo "Please execute 'sudo config save' to preserve System mode in Maintenance after reboot or config reload"
fi
fi
9 changes: 9 additions & 0 deletions dockers/docker-fpm-frr/base_image_files/TSB
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,15 @@ then
sudo config mux mode auto all
fi

if [ -z "$STARTED_BY_TSA_TSB_SERVICE" ]; then
service='startup_tsa_tsb.service'
if [[ $(/bin/systemctl show $service --property ActiveState --value) == "active" ]] && \
[[ $(/bin/systemctl show $service --property SubState --value) == "running" ]]; then
echo "Stopping $service before configuring TSB"
systemctl stop $service
fi
fi

/usr/bin/TS TSB
if [[ "$(sonic-cfggen -d -v DEVICE_METADATA.localhost.type)" == *"SpineRouter"* ]] ; then
if [[ "$1" != "chassis" ]] ; then
Expand Down
4 changes: 4 additions & 0 deletions files/build_templates/sonic_debian_extension.j2
Original file line number Diff line number Diff line change
Expand Up @@ -871,6 +871,7 @@ sudo LANG=C cp $SCRIPTS_DIR/telemetry.sh $FILESYSTEM_ROOT/usr/local/bin/telemetr
sudo LANG=C cp $SCRIPTS_DIR/mgmt-framework.sh $FILESYSTEM_ROOT/usr/local/bin/mgmt-framework.sh
sudo LANG=C cp $SCRIPTS_DIR/asic_status.sh $FILESYSTEM_ROOT/usr/local/bin/asic_status.sh
sudo LANG=C cp $SCRIPTS_DIR/asic_status.py $FILESYSTEM_ROOT/usr/local/bin/asic_status.py
sudo LANG=C cp $SCRIPTS_DIR/startup_tsa_tsb.py $FILESYSTEM_ROOT/usr/local/bin/startup_tsa_tsb.py

# Copy sonic-netns-exec script
sudo LANG=C cp $SCRIPTS_DIR/sonic-netns-exec $FILESYSTEM_ROOT/usr/bin/sonic-netns-exec
Expand Down Expand Up @@ -899,6 +900,9 @@ echo "mgmt-framework.timer" | sudo tee -a $GENERATED_SERVICE_FILE
sudo cp $BUILD_TEMPLATES/pmon.timer $FILESYSTEM_ROOT_USR_LIB_SYSTEMD_SYSTEM
echo "pmon.timer" | sudo tee -a $GENERATED_SERVICE_FILE

sudo cp files/build_templates/startup_tsa_tsb.service $FILESYSTEM_ROOT_USR_LIB_SYSTEMD_SYSTEM/
echo "startup_tsa_tsb.service" | sudo tee -a $GENERATED_SERVICE_FILE

sudo cp $BUILD_TEMPLATES/sonic.target $FILESYSTEM_ROOT_USR_LIB_SYSTEMD_SYSTEM
sudo LANG=C chroot $FILESYSTEM_ROOT systemctl enable sonic.target

Expand Down
15 changes: 15 additions & 0 deletions files/build_templates/startup_tsa_tsb.service
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
[Unit]
Description= STARTUP TSA-TSB SERVICE
Requires=updategraph.service database.service
After=updategraph.service database.service
Copy link
Contributor

@abdosi abdosi Jan 10, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does it make sure we are running this service before swss/syncd/teamd/bgp services comes up ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is no dependency on these services right now. Ensure this service starts before bgp should be sufficient right?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added "Before=bgp.service"

Before=bgp.service
ConditionPathExists=!/etc/sonic/chassisdb.conf
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what is need for this ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updategraph dependency is to start the tsa-tsb service after config is loaded and chassisdb.conf file check is to start the service only in IMM.


[Service]
Environment="STARTED_BY_TSA_TSB_SERVICE=1"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

after the service is stopped do we need to reset STARTED_BY_TSA_TSB_SERVICE

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reset the env variable

ExecStart=/usr/bin/python3 -u /usr/local/bin/startup_tsa_tsb.py start
ExecStop=/usr/bin/python3 -u /usr/local/bin/startup_tsa_tsb.py stop
RemainAfterExit=yes
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what is the behavior of this services in case of config reload and if the swss docker restart ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

config reload or swss restart doesn't restart the tsa service


[Install]
WantedBy=multi-user.target
156 changes: 156 additions & 0 deletions files/scripts/startup_tsa_tsb.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,156 @@
#!/usr/bin/env python3

# Name: startup_tsa_tsb.py, version: 1.0
#
# Description: Module contains the definitions to the VOQ Startup TSA-TSB service
from sonic_py_common import multi_asic, device_info
from sonic_py_common.logger import Logger
import subprocess
import sys, getopt
from threading import Timer
import os

# Global Logger class instance
logger = Logger("startup_tsa_tsb")
logger.set_min_log_priority_info()

def get_tsb_timer_interval():
platform = device_info.get_platform()
conf_file = '/usr/share/sonic/device/{}/startup-tsa-tsb.conf'.format(platform)
file = open(conf_file, 'r')
Lines = file.readlines()
for line in Lines:
field = line.split('=')[0].strip()
if field == "STARTUP_TSB_TIMER":
return line.split('=')[1].strip()
return 0

def get_sonic_config(ns, config_name):
if ns == "":
return subprocess.check_output(['sonic-cfggen', '-d', '-v', config_name.replace('"', "'"), ]).strip()
else:
return subprocess.check_output(['sonic-cfggen', '-d', '-v', config_name.replace('"', "'"), '-n', ns.replace('"', "'")]).strip()

def get_sub_role(asic_ns):
sub_role_config = "DEVICE_METADATA['localhost']['sub_role']"
sub_role = (get_sonic_config(asic_ns, sub_role_config)).decode()
return sub_role

def get_tsa_config(asic_ns):
tsa_config = 'BGP_DEVICE_GLOBAL.STATE.tsa_enabled'
tsa_ena = (get_sonic_config(asic_ns, tsa_config)).decode()
if asic_ns == "":
logger.log_info('CONFIG_DB.{} : {}'.format(tsa_config, tsa_ena))
else:
logger.log_info('{} - CONFIG_DB.{} : {}'.format(asic_ns, tsa_config, tsa_ena))
return tsa_ena

def get_tsa_status(num_asics):
if num_asics > 1:
counter = 0
for asic_id in range(int(asic_num)):
asic_ns = 'asic{}'.format(asic_id)
sub_role = get_sub_role(asic_ns)
if sub_role == 'FrontEnd':
tsa_enabled = get_tsa_config(asic_ns)
if tsa_enabled == 'false':
counter += 1
if counter == int(asic_num):
return True;
else:
tsa_enabled = get_tsa_config("")
if tsa_enabled == 'false':
return True;
return False;

def config_tsa():
num_asics = multi_asic.get_num_asics()
tsa_ena = get_tsa_status(num_asics)
if tsa_ena == True:
logger.log_info("Configuring TSA")
subprocess.check_output(['TSA']).strip()
else:
if num_asics > 1:
logger.log_info("Either TSA is already configured or switch sub_role is not Frontend - not configuring TSA")
else:
logger.log_info("Either TSA is already configured - not configuring TSA")
return tsa_ena

def config_tsb():
logger.log_info("Configuring TSB")
subprocess.check_output(['TSB']).strip()
tsb_issued = True
return

def start_tsb_timer(interval):
global timer
logger.log_info("Starting timer with interval {} seconds to configure TSB".format(interval))
timer = Timer(int(interval), config_tsb)
timer.start()
timer.join()
return

def print_usage():
logger.log_info("Usage: startup_tsa_tsb.py [options] command")
logger.log_info("options:")
logger.log_info(" -h | --help : this help message")
logger.log_info("command:")
logger.log_info("start : start the TSA/TSB")
logger.log_info("stop : stop the TSA/TSB")
return

def reset_env_variables():
logger.log_info("Resetting environment variable")
os.environ.pop('STARTED_BY_TSA_TSB_SERVICE')
return

def start_tsa_tsb(timer):
#Configure TSA if it was not configured already in CONFIG_DB
tsa_enabled = config_tsa()
if tsa_enabled == True:
#Start the timer to configure TSB
start_tsb_timer(timer)
return

def stop_tsa_tsb():
reset_env_variables()
Copy link
Contributor

@judyjoseph judyjoseph Jan 11, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In case the user entered TSA/TSB command manually, shouldn't we explicitly stop/cancel timer thread which we started earlier in start_tsb_timer() ? Or does the systemd cleans it up during exit

Here we just reset the env variables only.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will set the daemon flag for the Timer thread so that the thread will exit when the main thread is killed by the service stop

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@judyjoseph , i tested with prctl and the threads started by the main thread are stopped when the service is stopped and main python script is killed with SIGTERM. So we don't need to explicitly stop the timer.

return

def main():
platform = device_info.get_platform()
conf_file = '/usr/share/sonic/device/{}/startup-tsa-tsb.conf'.format(platform)
#This check should be moved to service file or make this feature as configurable.
#Adding it here for now.
if not os.path.exists(conf_file):
logger.log_info("{} does not exist, exiting the service".format(conf_file))
return
if len(sys.argv) <= 1:
print_usage()
return

# parse command line options:
try:
opts, args = getopt.getopt(sys.argv[1:], 'h:', ['help' ])
except getopt.GetoptError:
print_usage()
return

for opt, arg in opts:
if opt in ("-h", "--help"):
print_usage()
return

for arg in args:
if arg == 'start':
tsb_timer = get_tsb_timer_interval()
start_tsa_tsb(tsb_timer)
elif arg == 'stop':
stop_tsa_tsb()
else:
print_usage()
return

return

if __name__ == "__main__":
main()