[Smartswitch] Kill stale initramfs udevd process before updating udevd rules#17
Open
tirupatihemanth wants to merge 2 commits intomasterfrom
Open
[Smartswitch] Kill stale initramfs udevd process before updating udevd rules#17tirupatihemanth wants to merge 2 commits intomasterfrom
tirupatihemanth wants to merge 2 commits intomasterfrom
Conversation
Signed-off-by: Hemanth Kumar Tirupati <tirupatihemanthkumar@gmail.com>
There was a problem hiding this comment.
Pull request overview
This PR addresses a SmartSwitch DPU boot-time failure mode where a stale initramfs systemd-udevd process can survive switch_root and later crash when DPU udev rules are updated, by adding logic to detect and terminate the stale daemon before writing rules.
Changes:
- Add
kill_stale_udevd()to detect the systemd-managed udevd PID and kill othersystemd-udevd --daemonprocesses. - Invoke
kill_stale_udevd()before generating/etc/udev/rules.d/92-midplane-intf.rules. - Update the file copyright header year range.
dgsudharsan
approved these changes
Mar 19, 2026
vivekrnv
approved these changes
Mar 19, 2026
Signed-off-by: Hemanth Kumar Tirupati <tirupatihemanthkumar@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why I did it
On SONiC SmartSwitch platforms with DPUs, systemd-udevd crashes with SIGABRT on every reboot when DPU firmware initialization is slow. During the initramfs boot phase, a standalone systemd-udevd daemon is started to handle device discovery. If DPU firmware takes longer than the 60-second udevadm settle timeout (BlueField-3 DPUs can take 120 seconds each in the failure case when they are stuck), the initramfs cannot stop this udevd before switch_root. The stale process survives into the real system but is never chrooted into the overlayfs root, leaving it with a broken filesystem view. When dpu-udev-manager.sh writes udev rules, the stale udevd detects the change and crashes on an assertion in systemd's chase() path resolution (assert(path_is_absolute(p)) at chase.c:648), because dir_fd_is_root() returns false for a process whose root still points to the initramfs rootfs rather than the overlayfs.
This triggers a systemd issue : systemd#29559 which maintainers doesn't consider as a bug from systemd side. Raising this fix for our usecase.
How I did it
Added a kill_stale_udevd() function to dpu-udev-manager.sh that runs before writing the udev rules. It identifies the systemd-managed udevd PID via systemctl show, then kills any other systemd-udevd --daemon process that doesn't match -- these are leftover initramfs instances. If no stale process exists (e.g. DPUs are healthy and the initramfs udevd exited cleanly), the function is a no-op.
How to verify it