Skip to content

test: evaluate enabling large scale test (50 nodes) in CI #2026

@sanity

Description

@sanity

Parent Issue

Part of #2021 - fixing ignored integration tests

Problem

The 50-node test is currently marked as manual-only and ignored in CI.

Affected test:

  • apps/freenet-ping/app/tests/test_50_node_operations.rs::test_50_node_operations

Current status:

#[ignore = "large scale test - run manually"]

Test Description

This is a large-scale integration test that spins up 50 nodes to verify system behavior under higher peer counts. It's currently disabled from automated CI runs due to resource requirements.

Current State

Unlike the other ignored tests, this one is not broken - it's deliberately excluded from CI due to:

  • High resource consumption (50 node processes)
  • Long runtime
  • Potential CI environment limitations

Investigation Tasks

  • Measure actual resource requirements (CPU, memory, time)
  • Compare against CI environment limits
  • Evaluate if test can run with fewer nodes (e.g., 20-30) while still providing value
  • Check if test could run on specific CI triggers (e.g., pre-release, nightly)
  • Consider splitting into smaller tests that cover specific scenarios
  • Evaluate if test provides unique coverage vs existing multi-node tests

Options

Option 1: Enable in CI with modifications

  • Reduce node count to fit CI resources
  • Run only on specific branches or tags
  • Add as optional/manual trigger in CI

Option 2: Keep as manual test

  • Document when/how to run manually
  • Add to pre-release checklist
  • Consider running in nightly builds with higher resources

Option 3: Replace with targeted tests

  • If 50-node test is primarily for stress testing
  • Create focused tests for specific edge cases
  • Keep large scale test for local/manual verification only

Questions to Answer

  1. What specific scenarios does 50-node test cover that smaller tests don't?
  2. What's the minimum node count needed to reproduce those scenarios?
  3. What are the actual CI resource limits?
  4. How long does the test take to run?
  5. Is the test deterministic or does it have flakiness issues?

Success Criteria

  • Decision documented: enable in CI, keep manual, or replace
  • If enabling: test runs successfully in CI environment
  • If keeping manual: documented in testing guide and pre-release checklist
  • If replacing: new targeted tests created and validated
  • Either remove #[ignore] or update reason with tracking info

Impact

Low priority - test is working, just question of automation strategy. However, having large-scale testing coverage is valuable for catching scalability issues before production.

[AI-assisted debugging and comment]

Metadata

Metadata

Assignees

No one assigned

    Labels

    A-developer-xpArea: developer experienceE-mediumExperience needed to fix/implement: Medium / intermediateT-trackingType: Meta-issue tracking multiple related issues

    Type

    No type

    Projects

    Status

    Triage

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions