Skip to content

fix(console): skip extra newline after username in login_stage_2()#23295

Merged
StormLiangMS merged 1 commit intosonic-net:masterfrom
lipxu:fix/console-login-extra-newline
Mar 26, 2026
Merged

fix(console): skip extra newline after username in login_stage_2()#23295
StormLiangMS merged 1 commit intosonic-net:masterfrom
lipxu:fix/console-login-extra-newline

Conversation

@lipxu
Copy link
Copy Markdown
Contributor

@lipxu lipxu commented Mar 25, 2026

Description

SSHConsoleConn.login_stage_2() has an unconditional write_channel(self.RETURN) at the bottom of every loop iteration. After sending the username, the next loop iteration reads the terminal echo of the username ("admin"), which matches neither the login-prompt pattern nor the Password-prompt pattern — so all if branches are skipped and the unconditional write_channel(RETURN) fires, sending a blank newline to the DUT before Password: is shown.

The DUT receives the blank newline as the password → authentication fails → Linux pam_faildelay enforces a 3-second mandatory delay → the test framework times out → socket closes. All 3 retry attempts fail the same way.

dut_console/test_escape_character.py had a 2.6% pass rate (out of 11,323 runs over 30 days) due to this bug. Rare passes happen only when network timing causes the real password to race ahead of the accidental blank Enter.

Root Cause

The loop-bottom write_channel(self.RETURN) is intended to wake up the terminal when no pattern is matched (e.g., before the login prompt appears). It should only fire while waiting for the username prompt, not after the username has been sent:

# login_stage_2() — before fix
if not user_sent and re.search(username_pattern, output, flags=re.I):
    self.write_channel(username + self.RETURN)
    ...
    user_sent = True
    # next iteration reads "admin" echo — matches no pattern — falls through to:

self.write_channel(self.RETURN)   # ← sends blank \n as accidental password

Fix

Guard the loop-bottom write_channel(RETURN) so it only fires when still waiting for the username prompt:

if not user_sent and re.search(username_pattern, output, flags=re.I):
    self.write_channel(username + self.RETURN)
    ...
    user_sent = True
# username echo iteration and all subsequent iterations skip the blank CR:

if not user_sent:                   # ← fix: only wake up terminal before username is sent
    self.write_channel(self.RETURN)
time.sleep(0.5 * delay_factor)
i += 1

Verification

Manually confirmed on bjw-can-7060-1 via console (bjw-can-e1031-2, line 27):

  • Sending empty Enter at Password: prompt → ~3 seconds silence → "Login incorrect" (pam_faildelay)
  • Normal login with correct password → shell in under 1 second
  • DUT PAM config: pam_faildelay.so delay=3000000

Re-run on vms11-t0-7060-cx32-1 with partial fix confirmed the terminal-echo-iteration is the remaining bug (log showed write_channel: b'\n' between username and password).

https://elastictest.org/scheduler/testplan/69c3c20fa32086e7e883bc08

Type of change

  • Bug fix

Tested

  • Pending test run on testbed with the corrected fix

@mssonicbld
Copy link
Copy Markdown
Collaborator

/azp run

@azure-pipelines
Copy link
Copy Markdown

Azure Pipelines successfully started running 1 pipeline(s).

@lipxu lipxu force-pushed the fix/console-login-extra-newline branch from 4b78112 to f96255f Compare March 25, 2026 09:02
@mssonicbld
Copy link
Copy Markdown
Collaborator

/azp run

@azure-pipelines
Copy link
Copy Markdown

Azure Pipelines successfully started running 1 pipeline(s).

@lipxu lipxu force-pushed the fix/console-login-extra-newline branch from f96255f to db8af0e Compare March 25, 2026 09:13
@mssonicbld
Copy link
Copy Markdown
Collaborator

/azp run

@azure-pipelines
Copy link
Copy Markdown

Azure Pipelines successfully started running 1 pipeline(s).

The login loop in SSHConsoleConn.login_stage_2() sends an unconditional
write_channel(RETURN) at the bottom of every iteration. After the username
is sent and user_sent is set to True, the loop falls through to this
write_channel, sending a blank newline to the DUT before the Password:
prompt is read.

The DUT receives the blank newline as the password input, causing
authentication to fail. Linux pam_faildelay then enforces a 3-second
delay before responding with Login incorrect. The test framework times
out within this 3-second window, closes the connection, and retries -
all three attempts fail the same way.

Fix: add continue after user_sent = True so the loop restarts at the
top and waits for the Password: prompt before taking any further action.

Root cause confirmed via live manual test on bjw-can-7060-1:
- Sending empty Enter at Password: prompt -> 3-second silence -> Login incorrect
- Normal login with correct password succeeds in under 1 second
- Pass rate of dut_console/test_escape_character.py was 2.6% due to this race

Signed-off-by: Liping Xu <xuliping@microsoft.com>
@lipxu lipxu force-pushed the fix/console-login-extra-newline branch from db8af0e to b5cf26f Compare March 25, 2026 11:02
@mssonicbld
Copy link
Copy Markdown
Collaborator

/azp run

@azure-pipelines
Copy link
Copy Markdown

Azure Pipelines successfully started running 1 pipeline(s).

Copy link
Copy Markdown
Collaborator

@yxieca yxieca left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AI agent on behalf of Ying. Reviewed; no issues found.

@StormLiangMS StormLiangMS merged commit 8f515a2 into sonic-net:master Mar 26, 2026
18 checks passed
@StormLiangMS StormLiangMS added the Request for 202511 branch Request to backport a change to 202511 branch label Mar 26, 2026
mssonicbld pushed a commit to mssonicbld/sonic-mgmt that referenced this pull request Mar 26, 2026
…onic-net#23295)

The login loop in SSHConsoleConn.login_stage_2() sends an unconditional
write_channel(RETURN) at the bottom of every iteration. After the username
is sent and user_sent is set to True, the loop falls through to this
write_channel, sending a blank newline to the DUT before the Password:
prompt is read.

The DUT receives the blank newline as the password input, causing
authentication to fail. Linux pam_faildelay then enforces a 3-second
delay before responding with Login incorrect. The test framework times
out within this 3-second window, closes the connection, and retries -
all three attempts fail the same way.

Fix: add continue after user_sent = True so the loop restarts at the
top and waits for the Password: prompt before taking any further action.

Root cause confirmed via live manual test on bjw-can-7060-1:
- Sending empty Enter at Password: prompt -> 3-second silence -> Login incorrect
- Normal login with correct password succeeds in under 1 second
- Pass rate of dut_console/test_escape_character.py was 2.6% due to this race

Signed-off-by: Liping Xu <xuliping@microsoft.com>
Signed-off-by: mssonicbld <sonicbld@microsoft.com>
@mssonicbld
Copy link
Copy Markdown
Collaborator

Cherry-pick PR to 202511: #23362

ravaliyel pushed a commit to ravaliyel/sonic-mgmt that referenced this pull request Mar 27, 2026
…onic-net#23295)

The login loop in SSHConsoleConn.login_stage_2() sends an unconditional
write_channel(RETURN) at the bottom of every iteration. After the username
is sent and user_sent is set to True, the loop falls through to this
write_channel, sending a blank newline to the DUT before the Password:
prompt is read.

The DUT receives the blank newline as the password input, causing
authentication to fail. Linux pam_faildelay then enforces a 3-second
delay before responding with Login incorrect. The test framework times
out within this 3-second window, closes the connection, and retries -
all three attempts fail the same way.

Fix: add continue after user_sent = True so the loop restarts at the
top and waits for the Password: prompt before taking any further action.

Root cause confirmed via live manual test on bjw-can-7060-1:
- Sending empty Enter at Password: prompt -> 3-second silence -> Login incorrect
- Normal login with correct password succeeds in under 1 second
- Pass rate of dut_console/test_escape_character.py was 2.6% due to this race

Signed-off-by: Liping Xu <xuliping@microsoft.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants