Skip to content

Enhanced Test Isolation with Advanced Dependency Injection Patterns #17

@behrangsa

Description

@behrangsa

Summary

Enhance the existing dependency injection system with advanced isolation patterns to achieve complete test isolation, improve mock capabilities, and establish industry-standard testing practices for the Samoid codebase.

Current State Analysis

Existing Implementation ✅

The codebase already implements a solid foundation:

  • Trait-based abstractions: Environment, CommandRunner, FileSystem
  • Production implementations: SystemEnvironment, SystemCommandRunner, SystemFileSystem
  • Mock implementations: MockEnvironment, MockCommandRunner, MockFileSystem
  • Thread-safe mocks: Using Arc<Mutex<T>> for concurrent testing
  • Builder pattern: Fluent API for mock configuration

Current Limitations ❌

  1. Incomplete isolation: Tests can still affect each other through shared state
  2. Limited mock capabilities: Fixed responses, no dynamic behavior simulation
  3. No temporal isolation: Tests cannot simulate time-dependent scenarios
  4. Missing failure injection: Cannot simulate partial failures or flaky operations
  5. No resource cleanup: Manual cleanup required, risk of test pollution
  6. Limited observability: Cannot inspect what operations were performed during tests

Detailed Enhancement Requirements

1. Complete Test Isolation Framework

1.1 Enhanced Environment Manager

pub trait EnvironmentManager {
    fn create_isolated_environment(&self) -> Box<dyn IsolatedEnvironment>;
}

pub trait IsolatedEnvironment: Environment + Send + Sync {
    fn set_var(&mut self, key: &str, value: &str);
    fn unset_var(&mut self, key: &str);
    fn clear_all_vars(&mut self);
    fn get_all_vars(&self) -> HashMap<String, String>;
    fn reset_to_defaults(&mut self);
}

Requirements:

  • Each test gets a completely isolated environment
  • No cross-test contamination
  • Deterministic variable state
  • Ability to simulate missing/corrupt environment variables

1.2 Advanced Filesystem Isolation

pub trait IsolatedFileSystem: FileSystem + Send + Sync {
    fn create_temp_workspace(&mut self) -> PathBuf;
    fn cleanup_workspace(&mut self);
    fn simulate_disk_full(&mut self, enable: bool);
    fn simulate_permission_denied(&mut self, path: &Path, enable: bool);
    fn simulate_network_filesystem(&mut self, enable: bool);
    fn get_operation_log(&self) -> Vec<FileSystemOperation>;
}

#[derive(Debug, Clone)]
pub enum FileSystemOperation {
    Read { path: PathBuf, success: bool },
    Write { path: PathBuf, size: usize, success: bool },
    CreateDir { path: PathBuf, success: bool },
    SetPermissions { path: PathBuf, mode: u32, success: bool },
}

Requirements:

  • Automatic temporary directory creation/cleanup
  • Realistic filesystem error simulation
  • Operation logging for verification
  • Cross-platform path handling

1.3 Advanced Command Runner with Behavior Simulation

pub trait IsolatedCommandRunner: CommandRunner + Send + Sync {
    fn set_command_response(&mut self, cmd: &str, args: &[&str], response: CommandResponse);
    fn simulate_command_not_found(&mut self, cmd: &str, enable: bool);
    fn simulate_timeout(&mut self, cmd: &str, duration: Duration);
    fn simulate_intermittent_failure(&mut self, cmd: &str, failure_rate: f32);
    fn get_command_history(&self) -> Vec<CommandExecution>;
    fn clear_history(&mut self);
}

#[derive(Debug, Clone)]
pub struct CommandResponse {
    pub exit_code: i32,
    pub stdout: Vec<u8>,
    pub stderr: Vec<u8>,
    pub delay: Option<Duration>,
}

#[derive(Debug, Clone)]
pub struct CommandExecution {
    pub program: String,
    pub args: Vec<String>,
    pub timestamp: SystemTime,
    pub result: io::Result<Output>,
}

Requirements:

  • Complex command behavior simulation
  • Realistic timing simulation
  • Command execution history tracking
  • Flaky command simulation for robustness testing

2. Test Context Management

2.1 Isolated Test Context

pub struct TestContext {
    pub environment: Box<dyn IsolatedEnvironment>,
    pub filesystem: Box<dyn IsolatedFileSystem>,
    pub command_runner: Box<dyn IsolatedCommandRunner>,
    pub temp_dir: PathBuf,
    pub test_id: String,
}

impl TestContext {
    pub fn new(test_name: &str) -> Self;
    pub fn with_git_repository(&mut self) -> &mut Self;
    pub fn with_files(&mut self, files: HashMap<&str, &str>) -> &mut Self;
    pub fn with_env_vars(&mut self, vars: HashMap<&str, &str>) -> &mut Self;
    pub fn with_command_responses(&mut self, responses: HashMap<(&str, &[&str]), CommandResponse>) -> &mut Self;
    pub fn cleanup(self);
}

impl Drop for TestContext {
    fn drop(&mut self) {
        // Automatic cleanup
    }
}

Requirements:

  • Single object for all test dependencies
  • Automatic resource management
  • Fluent builder API
  • Deterministic cleanup

2.2 Test Scenario Builder

pub struct TestScenarioBuilder {
    context: TestContext,
}

impl TestScenarioBuilder {
    pub fn new(test_name: &str) -> Self;
    
    // Git repository scenarios
    pub fn empty_git_repo(self) -> Self;
    pub fn git_repo_with_commits(self, commits: &[&str]) -> Self;
    pub fn corrupted_git_repo(self) -> Self;
    pub fn large_git_repo(self, file_count: usize) -> Self;
    
    // Environment scenarios
    pub fn ci_environment(self) -> Self;
    pub fn development_environment(self) -> Self;
    pub fn production_environment(self) -> Self;
    
    // Failure scenarios
    pub fn git_command_failures(self) -> Self;
    pub fn filesystem_errors(self) -> Self;
    pub fn permission_denied_errors(self) -> Self;
    
    pub fn build(self) -> TestContext;
}

Requirements:

  • Predefined realistic scenarios
  • Complex scenario composition
  • Failure condition simulation
  • Performance testing scenarios

3. Enhanced Mock Capabilities

3.1 State Machine Mocks

pub trait StatefulMock {
    type State;
    fn set_state(&mut self, state: Self::State);
    fn get_state(&self) -> &Self::State;
    fn transition(&mut self, trigger: &str) -> Result<(), String>;
}

pub struct StatefulCommandRunner {
    current_state: CommandRunnerState,
    transitions: HashMap<(CommandRunnerState, String), CommandRunnerState>,
    state_responses: HashMap<CommandRunnerState, HashMap<String, CommandResponse>>,
}

#[derive(Debug, Clone, PartialEq, Eq, Hash)]
pub enum CommandRunnerState {
    Normal,
    GitUnavailable,
    NetworkDown,
    DiskFull,
    PermissionDenied,
}

Requirements:

  • Stateful behavior simulation
  • State transition validation
  • Complex failure scenario modeling
  • Realistic system state changes

3.2 Time-Aware Mocks

pub trait TimeProvider {
    fn now(&self) -> SystemTime;
    fn sleep(&self, duration: Duration);
}

pub struct MockTimeProvider {
    current_time: Arc<Mutex<SystemTime>>,
    time_multiplier: f64,
}

impl MockTimeProvider {
    pub fn new() -> Self;
    pub fn advance_time(&self, duration: Duration);
    pub fn set_time_multiplier(&mut self, multiplier: f64);
    pub fn freeze_time(&mut self);
    pub fn unfreeze_time(&mut self);
}

Requirements:

  • Time-dependent behavior testing
  • Fast-forward time simulation
  • Deterministic timing in tests
  • Race condition testing

4. Verification and Assertion Framework

4.1 Operation Verification

pub trait OperationVerifier {
    fn verify_command_called(&self, cmd: &str, args: &[&str]) -> bool;
    fn verify_command_call_count(&self, cmd: &str, expected: usize) -> bool;
    fn verify_file_written(&self, path: &Path, content: &str) -> bool;
    fn verify_directory_created(&self, path: &Path) -> bool;
    fn verify_environment_variable_read(&self, key: &str) -> bool;
}

pub struct TestVerifier {
    context: TestContext,
}

impl TestVerifier {
    pub fn new(context: TestContext) -> Self;
    
    pub fn assert_git_config_set(&self, key: &str, value: &str);
    pub fn assert_hooks_installed(&self, hook_names: &[&str]);
    pub fn assert_files_created(&self, files: &[&str]);
    pub fn assert_permissions_set(&self, path: &Path, expected_mode: u32);
    pub fn assert_no_side_effects(&self);
}

Requirements:

  • Comprehensive operation verification
  • Side-effect detection
  • Assertion helpers for common operations
  • Clear failure messages

5. Implementation Tasks

Phase 1: Core Infrastructure (8 story points)

  • 5.1 Implement IsolatedEnvironment trait and MockIsolatedEnvironment
  • 5.2 Implement IsolatedFileSystem trait with temp directory management
  • 5.3 Implement IsolatedCommandRunner with advanced response simulation
  • 5.4 Create TestContext with automatic cleanup
  • 5.5 Add comprehensive logging for all mock operations

Phase 2: Advanced Features (5 story points)

  • 5.6 Implement StatefulMock system for complex behavior simulation
  • 5.7 Add TimeProvider abstraction for time-dependent testing
  • 5.8 Create TestScenarioBuilder with predefined scenarios
  • 5.9 Implement failure injection capabilities
  • 5.10 Add cross-platform compatibility enhancements

Phase 3: Verification Framework (3 story points)

  • 5.11 Implement OperationVerifier and TestVerifier
  • 5.12 Add comprehensive assertion helpers
  • 5.13 Create side-effect detection mechanisms

Phase 4: Integration and Migration (5 story points)

  • 5.14 Update all existing tests to use new isolation framework
  • 5.15 Create migration guide and documentation
  • 5.16 Add performance benchmarks for new isolation system
  • 5.17 Implement backward compatibility layer
  • 5.18 Create comprehensive examples and best practices guide

6. Acceptance Criteria

6.1 Complete Test Isolation ✅

  • No test can affect another test's state
  • Each test gets fresh, isolated dependencies
  • Automatic cleanup prevents resource leaks
  • Deterministic test execution order independence

6.2 Enhanced Mock Capabilities ✅

  • Simulate complex system behaviors (disk full, network down, etc.)
  • State machine-based behavior modeling
  • Time-dependent scenario testing
  • Failure injection and flaky operation simulation

6.3 Comprehensive Verification ✅

  • Track all operations performed during tests
  • Verify expected operations occurred
  • Detect unexpected side effects
  • Clear assertion failure messages

6.4 Developer Experience ✅

  • Fluent, intuitive API for test setup
  • Predefined scenarios for common cases
  • Comprehensive documentation with examples
  • Migration path from existing tests

6.5 Performance and Reliability ✅

  • Test execution time < 5 seconds for full suite
  • Zero memory leaks from test isolation
  • Cross-platform compatibility (Unix, Windows)
  • Thread-safe mock implementations

7. Success Metrics

  • Isolation Score: 100% of tests are completely isolated
  • Coverage: >95% test coverage maintained after migration
  • Performance: <10% test execution time increase
  • Reliability: Zero test flakiness due to isolation issues
  • Maintainability: Reduced test setup code by >50%

Priority: High

Effort: 21 story points
Phase: Construction

Dependencies

  • Depends on: Current test infrastructure (Comprehensive Test Infrastructure #5) ✅
  • Blocks: Advanced integration testing, Performance testing improvements
  • Relates to: Error handling improvements, Cross-platform compatibility

Source

Identified through code review and testing best practices analysis. This enhancement addresses fundamental testing architecture improvements needed for enterprise-grade reliability.


Note: This issue represents a significant architectural improvement that will establish Samoid as having industry-leading test isolation practices. The investment will pay dividends in test reliability, maintainability, and developer productivity.

Metadata

Metadata

Assignees

No one assigned

    Labels

    effort: 8Effort: 8 story points - large workmeta:architectureArchitectural decision or significant designmeta:house-keepingTechnical debt and unnecessary features to removepriority:lowLow priority - nice to have improvementsrisk:highHigh risk item requiring mitigationsource:stakeholderSource: Stakeholder - requirement from project stakeholderstatus:approvedStatus: Approved - requirement accepted for implementationtechnical:testingTesting infrastructure

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions