Skip to content

[warm/fast reboot] some service docker might have been stopped already #668

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
yxieca merged 4 commits into from
Sep 25, 2019
Merged

[warm/fast reboot] some service docker might have been stopped already #668

Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
4ce8c38
[warm/fast reboot] some service docker might have been stopped already
yxieca Sep 24, 2019
6c9baa1
log and ignore all errors while killing containers
yxieca Sep 24, 2019
0f49488
add debug message
yxieca Sep 24, 2019
c551c32
put redirect at the right place
yxieca Sep 24, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 10 additions & 2 deletions scripts/fast-reboot
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -423,12 +423,14 @@ if [[ "$REBOOT_TYPE" = "fast-reboot" ]]; then
# then immediately call `systemctl stop teamd` to prevent the service from
# restarting the container automatically.
# Note: teamd must be killed before syncd, because it will send the last packet through CPU port
debug "Stopping teamd ..."
docker exec -i teamd pkill -USR2 teamd || [ $? == 1 ]
while docker exec -i teamd pgrep teamd > /dev/null; do
sleep 0.05
done
docker kill teamd > /dev/null
systemctl stop teamd
debug "Stopped teamd ..."
fi

# Kill swss Docker container
Expand Down Expand Up @@ -464,7 +466,7 @@ if [[ "$REBOOT_TYPE" = "warm-reboot" || "$REBOOT_TYPE" = "fastfast-reboot" ]]; t
# Send USR1 signal to all teamd instances to stop them
# It will prepare teamd for warm-reboot
# Note: We must send USR1 signal before syncd, because it will send the last packet through CPU port
docker exec -i teamd pkill -USR1 teamd || [ $? == 1 ] > /dev/null
docker exec -i teamd pkill -USR1 teamd > /dev/null || [ $? == 1 ]
debug "Stopped teamd ..."
fi

Expand All @@ -476,10 +478,16 @@ debug "Stopped syncd ..."
# We call `docker kill ...` to ensure the container stops as quickly as possible,
# then immediately call `systemctl stop ...` to prevent the service from
# restarting the container automatically.
debug "Stopping all remaining containers ..."
for CONTAINER_NAME in $(docker ps --format '{{.Names}}'); do
docker kill $CONTAINER_NAME > /dev/null
CONTAINER_STOP_RC=0
docker kill $CONTAINER_NAME &> /dev/null || CONTAINER_STOP_RC=$?
systemctl stop $CONTAINER_NAME
if [[ CONTAINER_STOP_RC -ne 0 ]]; then
debug "Failed killing container $CONTAINER_NAME RC $CONTAINER_STOP_RC ."
fi
done
debug "Stopped all remaining containers ..."

# Stop the docker container engine. Otherwise we will have a broken docker storage
systemctl stop docker.service
Expand Down