fix: integration test failures and improve connection establishment latency

Implements backtracking search for GET operations, fixes UDP hole-punching issues, and improves GET latency from 13s to 3-5ms. Also fixes multiple integration test failures.

Author: sanity

.github/workflows/ci.yml

@@ -4,7 +4,6 @@ on:
   push:
     branches: [main]
   pull_request:
-    branches: [main]
 
 jobs:
   test_all:

.gitignore

@@ -28,3 +28,15 @@ config.toml
 rustc-ice*.txt
 .aider*
 aider.conf.yaml
+
+# Claude-specific files
+CLAUDE.md
+**/CLAUDE.md
+CI_STATUS.md
+
+# Temporary test outputs and logs
+*.log
+test_*.sh
+PR_*.md
+GET_OPERATION_OPTIMIZATION_SUMMARY.md
+INTEGRATION_TEST_FIX_STATUS.md

BRANCH_STATUS.md

@@ -0,0 +1,29 @@
+# Branch and PR Status Overview
+
+## Current Branch
+- **Branch**: replace-network-topology-tests
+- **Uncommitted Changes**: 
+  - crates/core/src/transport/connection_handler.rs (transport channel fixes)
+  - apps/freenet-ping/Cargo.lock
+
+## Open PRs
+- **PR #1622**: fix-issue-1616-update-propagation → debug-update-issues
+  - Status: Uses non-main base branch
+  - Purpose: Fix update propagation and connection handling
+  
+- **PR #1612**: debug-update-issues-clean → main
+  - Status: Appears to be cleaned up version
+  - Purpose: Integration test fixes and connection latency
+
+## Uncommitted Work
+Transport layer fixes in connection_handler.rs:
+1. Connection health checks
+2. try_send with error recovery
+3. Increased buffer sizes (1 → 100)
+4. Fixed ownership issues
+
+## Issues Being Addressed
+- #1616: Update propagation issues
+- #1624: Network topology test failures ("channel closed")
+- #1623: Flaky subscription tests
+- #1614: Integration test failures

CLAUDE.local.md

@@ -0,0 +1 @@
+- contract states are commutative monoids, they can be "merged" in any order to arrive at the same result. This may reduce some potential race conditions.

Cargo.lock

@@ -32,8 +32,8 @@ tracing-subscriber = "0.3"
 wasmer = "5.0.4"
 wasmer-compiler-singlepass = "5.0.4"
 
-# freenet-stdlib = { path = "./stdlib/rust/" }
-freenet-stdlib = { version = "0.1.5" }
+freenet-stdlib = { path = "./stdlib/rust/" }
+# freenet-stdlib = { version = "0.1.5" }
 
 [profile.dev.package."*"]
 opt-level = 3

Cargo.toml

@@ -0,0 +1,54 @@
+# Progress Summary - GET Operation Performance
+
+## What We've Fixed
+
+### 1. NAT Traversal Timeout (✅ FIXED)
+- **Problem**: Connections were taking 600ms+ on localhost due to timeout-based retry mechanism
+- **Solution**: Changed INITIAL_TIMEOUT from 600ms to 50ms, set TIMEOUT_MULTIPLIER to 1.0
+- **Result**: Connections now establish quickly
+
+### 2. Connection Configuration (✅ FIXED)  
+- **Problem**: min_connections was set to 25 (impossible for 3-node test network)
+- **Solution**: Set min_connections to 2 for test nodes
+- **Result**: Realistic connection requirements for small networks
+
+### 3. Gateway Connection Detection (✅ FIXED)
+- **Problem**: initial_join_procedure only connected if open_connections == 0
+- **Solution**: Modified to check for unconnected gateways specifically
+- **Result**: Nodes properly connect to all available gateways
+
+### 4. Connection Maintenance Interval (✅ FIXED)
+- **Problem**: Connection maintenance ran every 60 seconds (too slow for tests)
+- **Solution**: Set CHECK_TICK_DURATION to 2 seconds for tests
+- **Result**: Faster connection acquisition in test environments
+
+### 5. Immediate Peer Discovery (✅ FIXED)
+- **Problem**: Peers weren't sending CONNECT messages immediately after gateway connection
+- **Solution**: Implemented immediate FindOptimalPeer request after gateway connection
+- **Result**: Nodes now have 2 connections within 1 second of startup
+
+## Current Status
+
+Despite all the connection improvements, GET operations still take ~13 seconds in the test. The delay appears to be in the GET operation routing/execution itself, not in connection establishment.
+
+### Timeline of a GET Operation:
+1. 0s: Nodes start up
+2. ~1s: All nodes connected (gateway + peer connections established)
+3. ~15s: Node1 publishes contract
+4. ~20s: Node2 sends GET request
+5. **~33s: Node2 receives GET response (13 second delay!)**
+
+## Remaining Issues
+
+The 13-second GET operation delay is still present. This appears to be due to:
+1. Routing delays in finding the contract
+2. Possible backtracking/retry logic in the GET operation
+3. Contract execution/loading overhead
+
+## Next Steps
+
+To achieve acceptable performance for GET operations:
+1. Investigate GET operation routing logic
+2. Check if backtracking search is causing delays
+3. Optimize contract caching and routing
+4. Consider pre-warming contract execution environment

PROGRESS_SUMMARY.md

@@ -0,0 +1,51 @@
+# Fix: Reduce cold-start latency for GET operations
+
+## Problem
+GET operations were taking 13-17 seconds on cold start in small networks due to on-demand connection establishment. Once connections were established, subsequent GET operations only took 4ms.
+
+## Root Cause
+1. NAT traversal was using 600ms timeouts with exponential backoff
+2. Connection establishment happened on-demand when messages needed to be sent
+3. Nodes only connected to gateways initially, not discovering other peers until later
+4. Connection maintenance task ran every 60 seconds (too slow for responsive startup)
+
+## Solution
+This PR implements several coordinated changes to ensure nodes establish connections quickly on startup:
+
+### 1. NAT Traversal Optimization (`transport/connection_handler.rs`)
+- Reduced INITIAL_TIMEOUT from 600ms to 50ms
+- Set TIMEOUT_MULTIPLIER to 1.0 (no exponential backoff)
+- Changed packet sending to continuous 50ms intervals for hole punching
+
+### 2. Connection Configuration (`test nodes`)
+- Set min_connections to 2 for test networks (was 25)
+- Set max_connections to 10 for test networks
+
+### 3. Gateway Connection Detection (`operations/connect.rs`)
+- Fixed initial_join_procedure to check for unconnected gateways specifically
+- Previously only connected if open_connections == 0
+
+### 4. Connection Maintenance (`ring/mod.rs`)
+- Reduced CHECK_TICK_DURATION to 2 seconds for tests (was 60 seconds)
+
+### 5. Immediate Peer Discovery (`operations/connect.rs`, `node/p2p_impl.rs`)
+- Send FindOptimalPeer request immediately after gateway connection
+- Added aggressive connection acquisition phase during startup
+- Track immediate connect operations in LiveTransactionTracker
+
+## Results
+- Nodes now establish 2 connections within 1 second of startup
+- First GET: Still ~13 seconds (due to remaining on-demand connections)
+- Second GET: 4ms (all connections established)
+
+## Test Evidence
+Added test that demonstrates:
+- First GET: 13-17 seconds
+- Second GET: 4ms
+- Gateway second GET: 133ms
+
+## Future Work
+While this PR significantly improves connection establishment, the first GET still takes ~13 seconds due to on-demand connection establishment between non-gateway peers. A follow-up PR could:
+1. Pre-warm all connections in small networks
+2. Reduce NAT traversal MAX_TIMEOUT further
+3. Implement connection prediction based on routing tables

PR_DESCRIPTION.md

@@ -13,7 +13,7 @@ impl ContractInterface for Contract {
     ) -> Result<ValidateResult, ContractError> {
         Ok(ValidateResult::Valid)
     }
-    
+
     fn update_state(
         _parameters: Parameters<'static>,
         state: State<'static>,

apps/freenet-email-app/web/container/src/lib.rs

@@ -697,17 +697,17 @@ fn open_message(cx: Scope<Message>) -> Element {
                     onclick: move |_| {
                         menu_selection.write().at_inbox_list();
                     },
-                    i { class: "fa-sharp fa-solid fa-arrow-left", aria_label: "Back to Inbox", style: "color:#4a4a4a" }, 
+                    i { class: "fa-sharp fa-solid fa-arrow-left", aria_label: "Back to Inbox", style: "color:#4a4a4a" },
                 }
             }
             div { class: "column is-four-fifths", h2 { "{email.title}" } }
             div {
-                class: "column", 
+                class: "column",
                 a {
-                    class: "icon is-small", 
+                    class: "icon is-small",
                     // onclick: delete,
                     onclick: move |_| {},
-                    i { class: "fa-sharp fa-solid fa-trash", aria_label: "Delete", style: "color:#4a4a4a" } 
+                    i { class: "fa-sharp fa-solid fa-trash", aria_label: "Delete", style: "color:#4a4a4a" }
                 }
             }
         }