Skip to content

Sync#153

Merged
TheMarstonConnell merged 5 commits intomainfrom
marston/sync
Nov 20, 2025
Merged

Sync#153
TheMarstonConnell merged 5 commits intomainfrom
marston/sync

Conversation

@TheMarstonConnell
Copy link
Member

@TheMarstonConnell TheMarstonConnell commented Nov 8, 2025

Switching to sync to make things better again.

Summary by CodeRabbit

  • New Features

    • Automatic retry for account sequence mismatches with intelligent recovery.
  • Bug Fixes

    • More reliable transaction broadcasting and sequence handling.
    • Improved stray item list stability and correct repopulation behavior.
  • Chores

    • Updated rate-limit defaults to allow higher throughput and larger bursts.
    • Tightened pagination bounds for stray listings.
  • Tests

    • Added comprehensive tests covering sequence error extraction and edge cases.

@coderabbitai
Copy link
Contributor

coderabbitai bot commented Nov 8, 2025

Walkthrough

Updates default rate-limit values, switches queue broadcasts to synchronous, adds parsing/retry for account-sequence-mismatch errors, introduces tests for the parser, and refactors stray-manager pagination and internal types (including changing lastSize to uint64 and safer slice population).

Changes

Cohort / File(s) Summary
Rate Limit Configuration
config/types.go
Updated DefaultRateLimitConfig: PerTokenMs from 400 → 100, Burst from 10 → 30.
Queue broadcast & sequence handling
queue/queue.go
Added extractExpectedSequence(errorMsg string) (uint64, bool) using regex/strconv; switched broadcast from BroadcastTxCommitBroadcastTxSync; on account-sequence-mismatch parse expected sequence and retry (fallback: increment); mirrored extraction logic on success path; removed a 10-minute batching skip; added regexp and strconv imports.
Queue tests
queue/queue_test.go
New tests for extractExpectedSequence covering normal cases, large numbers, zero, malformed messages, whitespace/newlines, and negative cases; asserts both returned value and found flag.
Stray manager refactor
strays/manager.go, strays/types.go
Introduced maxReturn constant; changed StrayManager.lastSize type from int64uint64; tightened pagination logic to use maxReturn; avoid resetting s.strays unnecessarily; build new slice with pointers to each file to avoid loop-variable aliasing; adjusted random offset and page-limit handling.

Sequence Diagram(s)

sequenceDiagram
    participant Caller
    participant Queue.BroadcastPending as BroadcastPending
    participant Extract as extractExpectedSequence
    participant RetryLoop as Retry

    Caller->>BroadcastPending: BroadcastPending(tx, account)
    rect rgb(220,240,255)
    Note over BroadcastPending: Send via BroadcastTxSync
    BroadcastPending->>BroadcastPending: Call RPC BroadcastTxSync
    end

    alt RPC returns sequence-mismatch error
        BroadcastPending->>Extract: extractExpectedSequence(err.Error())
        alt extraction success
            Extract-->>BroadcastPending: expectedSequence (uint64), true
            BroadcastPending->>BroadcastPending: set account sequence = expectedSequence
        else extraction failed
            Extract-->>BroadcastPending: 0, false
            BroadcastPending->>BroadcastPending: increment sequence (fallback)
        end
        BroadcastPending->>RetryLoop: retry broadcast with updated sequence
    else RPC success
        BroadcastPending->>BroadcastPending: check res.RawLog for "account sequence mismatch"
        alt RawLog contains mismatch
            BroadcastPending->>Extract: extractExpectedSequence(res.RawLog)
            alt extraction success
                Extract-->>BroadcastPending: expectedSequence
                BroadcastPending->>BroadcastPending: set sequence = expectedSequence
            else
                Extract-->>BroadcastPending: 0,false
                BroadcastPending->>BroadcastPending: increment sequence
            end
        end
        BroadcastPending-->>Caller: return result
    end
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

  • Areas needing extra attention:
    • Regex correctness and edge-case handling in extractExpectedSequence.
    • Correctness and race-safety of sequence update + retry logic in queue/queue.go.
    • Impact of switching BroadcastTxCommitBroadcastTxSync on ordering/semantics.
    • strays/manager.go changes: lastSize type change, pagination math, and removal of prior reset logic.
    • Tests: ensure coverage matches intended behavior and edge cases.

Possibly related PRs

  • more block tweaks #147: Overlaps changes to config/types.go rate-limit defaults and queue/queue.go broadcast behavior and sequence handling.

Pre-merge checks and finishing touches

❌ Failed checks (1 warning, 1 inconclusive)
Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 33.33% which is insufficient. The required threshold is 80.00%. You can run @coderabbitai generate docstrings to improve docstring coverage.
Title check ❓ Inconclusive The title 'Sync' is extremely vague and does not clearly convey the specific changes made in the pull request, which involve multiple distinct modifications across config, queue, and strays packages. Use a more descriptive title that highlights the main changes, such as 'Update rate limiting config and improve account sequence handling' or 'Refactor queue broadcast and sequence error recovery'.
✅ Passed checks (1 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
✨ Finishing touches
  • 📝 Generate docstrings
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch marston/sync

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

📜 Review details

Configuration used: CodeRabbit UI

Review profile: ASSERTIVE

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between e181a29 and 87cf58a.

📒 Files selected for processing (3)
  • config/types.go (1 hunks)
  • queue/queue.go (5 hunks)
  • queue/queue_test.go (1 hunks)
🔇 Additional comments (5)
config/types.go (1)

70-72: Verify rate limit changes and clarify incomplete broadcast migration.

The rate limiting defaults have been significantly relaxed (4x faster token generation, 3x larger burst capacity). Additionally, the codebase shows mixed broadcast implementations: queue/queue.go uses BroadcastTxSync, but core/app.go still uses BroadcastTxCommit in three locations (lines 133, 155, 177), suggesting an incomplete migration.

The sequence mismatch handling mentioned in the summary is already implemented in queue/queue.go (lines 246–248, 263–265).

Please verify:

  1. Are these rate limit values (100ms per token, burst of 30) appropriate for the expected load and tested under realistic conditions?
  2. Is the incomplete BroadcastTxCommit usage in core/app.go intentional or does it need migration to BroadcastTxSync for consistency?
queue/queue_test.go (2)

110-131: LGTM! Comprehensive edge case coverage.

The edge case tests appropriately verify that the extraction handles whitespace variations, newlines, and long messages correctly.


84-88: No issues found—the comma-formatted test case is documented edge-case handling.

The regex pattern expected\s+(\d+) (queue.go:30) intentionally matches only contiguous digits. All real Cosmos SDK v0.45.17 error messages in the test suite (lines 19, 25, 112) use non-comma format ("expected 1471614"). The test case at lines 84–88 with comma formatting is explicitly documented as a known limitation: "Regex will match the first digit sequence '1'". This shows developers are aware of the edge case and it does not represent a production concern.

queue/queue.go (2)

146-147: LGTM! Simplified polling logic.

The 100ms sleep interval is reasonable for queue polling, and the simplified time check improves readability.


234-234: Verify the broadcast method change aligns with system requirements.

The switch from BroadcastTxCommit to BroadcastTxSync is a significant behavioral change. BroadcastTxSync returns after passing CheckTx without waiting for block inclusion, making it faster but providing less certainty about transaction finality.

Please confirm:

  1. The system correctly handles the asynchronous nature of BroadcastTxSync
  2. There are appropriate mechanisms to track transaction inclusion in blocks
  3. Error handling accounts for transactions that pass CheckTx but fail in block execution

You may also want to verify that downstream systems don't depend on the finality guarantees that BroadcastTxCommit provided.

Comment on lines +26 to +38
// extractExpectedSequence extracts the expected sequence number from an account sequence mismatch error message.
// It looks for the pattern "expected <number>" in the error message, allowing for optional whitespace.
// Returns the expected sequence number and true if found, or 0 and false if not found.
func extractExpectedSequence(errorMsg string) (uint64, bool) {
re := regexp.MustCompile(`expected\s+(\d+)`)
matches := re.FindStringSubmatch(errorMsg)
if len(matches) > 1 {
if expectedSeq, parseErr := strconv.ParseUint(matches[1], 10, 64); parseErr == nil {
return expectedSeq, true
}
}
return 0, false
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick | 🔵 Trivial

Optimize regex compilation for better performance.

The regex is compiled on every function call, which is inefficient. Consider moving the compilation to a package-level variable.

Apply this diff to optimize the regex compilation:

+var (
+	sequenceRegex = regexp.MustCompile(`expected\s+(\d+)`)
+)
+
 // extractExpectedSequence extracts the expected sequence number from an account sequence mismatch error message.
 // It looks for the pattern "expected <number>" in the error message, allowing for optional whitespace.
 // Returns the expected sequence number and true if found, or 0 and false if not found.
 func extractExpectedSequence(errorMsg string) (uint64, bool) {
-	re := regexp.MustCompile(`expected\s+(\d+)`)
-	matches := re.FindStringSubmatch(errorMsg)
+	matches := sequenceRegex.FindStringSubmatch(errorMsg)
 	if len(matches) > 1 {
 		if expectedSeq, parseErr := strconv.ParseUint(matches[1], 10, 64); parseErr == nil {
 			return expectedSeq, true
 		}
 	}
 	return 0, false
 }
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
// extractExpectedSequence extracts the expected sequence number from an account sequence mismatch error message.
// It looks for the pattern "expected <number>" in the error message, allowing for optional whitespace.
// Returns the expected sequence number and true if found, or 0 and false if not found.
func extractExpectedSequence(errorMsg string) (uint64, bool) {
re := regexp.MustCompile(`expected\s+(\d+)`)
matches := re.FindStringSubmatch(errorMsg)
if len(matches) > 1 {
if expectedSeq, parseErr := strconv.ParseUint(matches[1], 10, 64); parseErr == nil {
return expectedSeq, true
}
}
return 0, false
}
var (
sequenceRegex = regexp.MustCompile(`expected\s+(\d+)`)
)
// extractExpectedSequence extracts the expected sequence number from an account sequence mismatch error message.
// It looks for the pattern "expected <number>" in the error message, allowing for optional whitespace.
// Returns the expected sequence number and true if found, or 0 and false if not found.
func extractExpectedSequence(errorMsg string) (uint64, bool) {
matches := sequenceRegex.FindStringSubmatch(errorMsg)
if len(matches) > 1 {
if expectedSeq, parseErr := strconv.ParseUint(matches[1], 10, 64); parseErr == nil {
return expectedSeq, true
}
}
return 0, false
}
🤖 Prompt for AI Agents
In queue/queue.go around lines 26 to 38, the regular expression is being
compiled on every call to extractExpectedSequence which hurts performance; fix
it by moving regexp.MustCompile(`expected\s+(\d+)`) to a package-level variable
(e.g., var expectedSeqRe = regexp.MustCompile(...)) with a brief comment, then
update extractExpectedSequence to use that precompiled
expectedSeqRe.FindStringSubmatch(errorMsg) and keep the same parsing and return
logic.

Comment on lines +246 to +256
if strings.Contains(err.Error(), "account sequence mismatch") {
if expectedSeq, found := extractExpectedSequence(err.Error()); found {
data = data.WithSequence(expectedSeq)
continue
}
// Fallback to incrementing if extraction fails
if data.Sequence != nil {
data = data.WithSequence(*data.Sequence + 1)
continue
}
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick | 🔵 Trivial

Consider extracting duplicated sequence mismatch handling logic.

The sequence mismatch handling logic is duplicated between the error path (lines 246-256) and the response path (lines 263-273). This reduces maintainability and increases the risk of inconsistencies if the logic needs to be updated.

Consider refactoring into a helper function:

// handleSequenceMismatch attempts to recover from a sequence mismatch by extracting
// the expected sequence from the error message or incrementing the current sequence.
// Returns updated TransactionData and true if recovery was attempted, or original data and false otherwise.
func handleSequenceMismatch(data walletTypes.TransactionData, errorMsg string) (walletTypes.TransactionData, bool) {
	if expectedSeq, found := extractExpectedSequence(errorMsg); found {
		return data.WithSequence(expectedSeq), true
	}
	// Fallback to incrementing if extraction fails
	if data.Sequence != nil {
		return data.WithSequence(*data.Sequence + 1), true
	}
	return data, false
}

Then use it in both places:

// In error path:
if strings.Contains(err.Error(), "account sequence mismatch") {
	if newData, handled := handleSequenceMismatch(data, err.Error()); handled {
		data = newData
		continue
	}
}

// In response path:
if strings.Contains(res.RawLog, "account sequence mismatch") {
	if newData, handled := handleSequenceMismatch(data, res.RawLog); handled {
		data = newData
		continue
	}
}

Also applies to: 263-273

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

📜 Review details

Configuration used: CodeRabbit UI

Review profile: ASSERTIVE

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 87cf58a and 2c78a04.

📒 Files selected for processing (2)
  • strays/manager.go (3 hunks)
  • strays/types.go (1 hunks)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
  • GitHub Check: lint

@TheMarstonConnell TheMarstonConnell merged commit 965e84f into main Nov 20, 2025
6 checks passed
@TheMarstonConnell TheMarstonConnell deleted the marston/sync branch November 20, 2025 19:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant