Skip to content

Distinguish between initial and resumed watch phases for streaming lists#1882

Merged
clux merged 2 commits into
kube-rs:mainfrom
doxxx93:fix/streaming-lists-reconnect-1844
Jan 3, 2026
Merged

Distinguish between initial and resumed watch phases for streaming lists#1882
clux merged 2 commits into
kube-rs:mainfrom
doxxx93:fix/streaming-lists-reconnect-1844

Conversation

@doxxx93
Copy link
Copy Markdown
Member

@doxxx93 doxxx93 commented Jan 3, 2026

Motivation

Fixes #1844

When using streaming_lists() mode, users observe duplicate events every ~290 seconds. This happens because watch connections timeout after 290s (Kubernetes API limitation), and when reconnecting, the watcher incorrectly sets sendInitialEvents=true, causing the API server to resend all objects.

The root cause is that to_watch_params() always sets send_initial_events=true for StreamingList mode, regardless of whether it's an initial watch or a reconnection.

Solution

Introduce a WatchPhase enum to distinguish between initial and resumed watches:

enum WatchPhase {
    Initial,  // from State::Empty - requests initial events
    Resumed,  // from State::InitListed - no initial events
}
  • WatchPhase::Initial: Used when starting a fresh streaming list watch, sets sendInitialEvents=true
  • WatchPhase::Resumed: Used when reconnecting after timeout, sets sendInitialEvents=false

This aligns with how client-go handles streaming lists - it uses watchList() for initial sync and watch() for reconnections, never requesting initial events on reconnection.

@doxxx93 doxxx93 force-pushed the fix/streaming-lists-reconnect-1844 branch from 0eb30f4 to 42c3bb9 Compare January 3, 2026 00:34
@doxxx93 doxxx93 requested a review from clux January 3, 2026 00:37
@codecov
Copy link
Copy Markdown

codecov Bot commented Jan 3, 2026

Codecov Report

❌ Patch coverage is 70.37037% with 8 lines in your changes missing coverage. Please review.
✅ Project coverage is 76.1%. Comparing base (10efa00) to head (7357151).
⚠️ Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
kube-runtime/src/watcher.rs 70.4% 8 Missing ⚠️
Additional details and impacted files
@@           Coverage Diff           @@
##            main   #1882     +/-   ##
=======================================
+ Coverage   76.0%   76.1%   +0.2%     
=======================================
  Files         85      85             
  Lines       8267    8285     +18     
=======================================
+ Hits        6275    6301     +26     
+ Misses      1992    1984      -8     
Files with missing lines Coverage Δ
kube-runtime/src/watcher.rs 48.6% <70.4%> (+4.8%) ⬆️

... and 5 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@doxxx93
Copy link
Copy Markdown
Member Author

doxxx93 commented Jan 3, 2026

I will add some tests

@clux clux added the changelog-fix changelog fix category for prs label Jan 3, 2026
@clux clux added this to the 3.0.0 milestone Jan 3, 2026
Copy link
Copy Markdown
Member

@clux clux left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot! This looks great to me. Did a quick sanity run and it does indeed fix the underlying problem.

@doxxx93 doxxx93 closed this Jan 3, 2026
@doxxx93 doxxx93 deleted the fix/streaming-lists-reconnect-1844 branch January 3, 2026 06:34
@doxxx93 doxxx93 restored the fix/streaming-lists-reconnect-1844 branch January 3, 2026 06:34
@doxxx93 doxxx93 reopened this Jan 3, 2026
@clux
Copy link
Copy Markdown
Member

clux commented Jan 3, 2026

Happy to merge?

@doxxx93
Copy link
Copy Markdown
Member Author

doxxx93 commented Jan 3, 2026

Yeah 😀

@clux clux merged commit 5d9aa40 into kube-rs:main Jan 3, 2026
33 checks passed
@doxxx93 doxxx93 deleted the fix/streaming-lists-reconnect-1844 branch January 3, 2026 10:15
@clux clux mentioned this pull request Jan 6, 2026
hugoponthieu pushed a commit to hugoponthieu/kube that referenced this pull request Jan 29, 2026
…sts (kube-rs#1882)

* fix(watcher): distinguish between initial and resumed watch phases for streaming lists

Signed-off-by: doxxx93 <[email protected]>

* test(watcher): add unit tests for WatchPhase behavior

Signed-off-by: doxxx93 <[email protected]>

---------

Signed-off-by: doxxx93 <[email protected]>
cchndl pushed a commit to cchndl/kube that referenced this pull request Feb 19, 2026
…sts (kube-rs#1882)

* fix(watcher): distinguish between initial and resumed watch phases for streaming lists

Signed-off-by: doxxx93 <[email protected]>

* test(watcher): add unit tests for WatchPhase behavior

Signed-off-by: doxxx93 <[email protected]>

---------

Signed-off-by: doxxx93 <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog-fix changelog fix category for prs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Possible inconsistency in behaviour with streaming_list and listwatch

2 participants