Reduce log noise for transient RPC failures in multi-URL setups#292
Open
ameten wants to merge 1 commit intoindexsupply:mainfrom
Open
Reduce log noise for transient RPC failures in multi-URL setups#292ameten wants to merge 1 commit intoindexsupply:mainfrom
ameten wants to merge 1 commit intoindexsupply:mainfrom
Conversation
Author
|
@ryansmith3136, @lechengfan, could you please take a look on this change? |
When multiple RPC URLs are configured, Shovel's round-robin naturally rotates to healthy endpoints on retry. Previously, every single retry was logged at error level, creating noise when only one of several RPCs was temporarily down. This change tracks consecutive failures and only escalates to error level after all configured URLs have been tried and failed. Individual URL failures are logged at warn level. The per-block loading error in the load() function is also downgraded to warn since it propagates up to runTask where it is logged at the appropriate level. Co-Authored-By: Claude Opus 4.6 <[email protected]>
ameten
force-pushed
the
reduce-converge-retry-log-noise
branch
from
dc9e10f to
335fd73
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
warnlevel instead oferrorerroronly after all configured URLs have been tried and failedloading blockserror inload()is downgraded towarnsince it propagates up torunTaskwhere it's logged at the appropriate levelNumURLs() intto theSourceinterface andjrpc2.ClientMotivation
Shovel's round-robin URL rotation naturally recovers from single-endpoint failures on the next retry. Logging every retry at
errorlevel creates noise in monitoring when only one of several RPCs is temporarily down, making it harder to spot real issues where all endpoints are failing.Test plan
go build ./...passesgo test ./shovel/ -shortpasses🤖 Generated with Claude Code