🤖 Saorsa Robotics

Production-ready Rust framework for autonomous robotic control with local AI models

Saorsa Robotics provides a comprehensive, safety-first framework for robotic control systems with Vision-Language-Action (VLA) models running entirely on local hardware. Built in Rust for memory safety, performance, and reliability.

🌟 Key Features

🧠 Local AI Models

• MolmoAct Integration: Action Reasoning Model with 3D spatial understanding and Chain-of-Thought planning
• Candle ML Framework: Lightweight Rust-native inference without Python dependencies
• OpenVLA Support: Compatible with cloud and local VLA models
• On-Device Learning: Continual improvement through OFT adapters and intervention learning

🛡️ Safety-Critical Design

• Formal Constraint System: Expression-based DSL for defining safety boundaries
• Real-Time Monitoring: Watchdog systems with automatic intervention
• Zero Panic Guarantee: No unwrap(), expect(), or panic!() in production code
• Comprehensive Testing: 100% test coverage on safety-critical paths

🎯 Advanced Capabilities

• Multi-Modal Control: Voice commands, vision processing, and haptic feedback
• Stereo Vision: Depth perception with dual camera calibration
• CAN Bus Integration: Direct hardware control for motors and actuators
• Intent Parsing: Natural language to robot action conversion

⚡ Performance

• Zero-Copy Data Paths: Efficient memory usage
• Async/Await: Non-blocking I/O throughout
• Action Chunking: Smooth control despite network latency
• Real-Time Capable: Deterministic timing for critical operations

🚀 Quick Start

Prerequisites

# Install Rust
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh

# Clone repository
git clone https://github.com/dirvine/saorsa-robotics
cd saorsa-robotics

Build and Test

# Build all crates
cargo build --release

# Run all tests
cargo test --all

# Run with safety checks
cargo run --bin sr-cli -- --safety-enabled

Run Examples

# VLA Policy Demo
cargo run --example vla_policy_demo

# Wake Word Detection
cargo run --example wake_word_demo

# Safety Constraints Demo
cargo run --bin safety-demo

📦 Architecture

saorsa-robotics/
├── apps/                       # Application binaries
│   ├── sr-cli/                # Main CLI interface
│   ├── brain-daemon/          # Central coordination daemon
│   ├── safety-demo/           # Safety system demonstration
│   └── kyutai-stt-app/       # Speech-to-text application
├── crates/                    # Core library crates
│   ├── vla-policy/           # Vision-Language-Action models
│   ├── safety-guard/         # Safety constraint engine
│   ├── voice-local/          # On-device voice processing
│   ├── vision-stereo/        # Stereo vision and depth
│   ├── intent-parser/        # NLU and command parsing
│   ├── can-transport/        # CAN bus communication
│   ├── device-registry/      # Hardware device management
│   └── continual-learning/   # Online learning framework
├── examples/                  # Example applications
├── configs/                   # Device and system configs
└── docs/                     # Technical documentation

🔧 Core Components

VLA Policy System (vla-policy)

Implements multiple Vision-Language-Action models for robot control:

use vla_policy::{create_policy, PolicyConfig, Observation};

// Create MolmoAct policy with 3D reasoning
let config = PolicyConfig {
    model_type: "molmoact".to_string(),
    model_path: "models/molmoact-7b".to_string(),
    // ... configuration
};

let policy = create_policy(config)?;
let action = policy.predict(&observation).await?;

Features:

• MolmoAct with Chain-of-Thought reasoning
• Waypoint generation for complex tasks
• Skills framework (Pick, Place, Reach)
• Mock policy for testing

Safety Guard (safety-guard)

Expression-based constraint system ensuring safe operation:

use safety_guard::{SafetyGuard, Constraint};

let mut guard = SafetyGuard::new();

// Define workspace boundaries
guard.add_constraint(Constraint::expression(
    "workspace_x",
    "x >= -0.5 && x <= 0.5"
)?);

// Check if action is safe
if guard.check_action(&action)? {
    robot.execute(action)?;
}

Features:

• Mathematical expression constraints
• Real-time evaluation with evalexpr
• Watchdog monitoring
• Automatic intervention on violations

Voice Control (voice-local)

On-device speech recognition and wake word detection:

use voice_local::{KyutaiProvider, WakeWordDetector};

let provider = KyutaiProvider::new(config)?;
let detector = WakeWordDetector::new("hey robot")?;

// Process audio stream
if detector.detect(&audio_frame)? {
    let command = provider.transcribe(&audio_buffer)?;
    execute_command(command)?;
}

Features:

• Kyutai/Mimi model integration
• Real-time transcription
• Wake word detection
• Plugin architecture for custom models

Stereo Vision (vision-stereo)

Depth perception and 3D scene understanding:

use vision_stereo::{StereoCamera, DepthEstimator};

let camera = StereoCamera::new(config)?;
camera.calibrate()?;

let (left, right) = camera.capture()?;
let depth_map = DepthEstimator::compute(&left, &right)?;
let tags = detect_april_tags(&left)?;

Features:

• Dual camera calibration
• Real-time depth estimation
• AprilTag detection
• Point cloud generation

CAN Transport (can-transport)

Hardware control via CAN bus:

use can_transport::{SlcanTransport, Message};

let transport = SlcanTransport::new("/dev/ttyUSB0")?;

// Send motor command
let msg = Message::new(0x123, &[0x01, 0x02, 0x03])?;
transport.send(&msg)?;

Features:

• SLCAN protocol support
• ODrive motor control
• T-Motor actuator support
• Mock transport for testing

🧪 Testing

All crates maintain 100% test coverage on critical paths:

# Run all tests
cargo test --all

# Run with coverage
cargo tarpaulin --out Html

# Run safety-critical tests
cargo test -p safety-guard

# Run benchmarks
cargo bench

Current test status:

• ✅ safety-guard: 13/13 passing
• ✅ vla-policy: 21/21 passing
• ✅ voice-local: All doctests passing
• ✅ intent-parser: 1/1 passing
• ✅ Zero compilation warnings

🔐 Safety & Security

Production Standards

• No Panics: Zero unwrap(), expect(), or panic!() in production
• Error Handling: All errors properly propagated with Result<T, E>
• Memory Safety: Guaranteed by Rust's ownership system
• Concurrency Safety: Safe parallelism with Send/Sync traits

Safety Features

• Formal constraint verification
• Watchdog timers on all operations
• Automatic failsafe modes
• Comprehensive audit logging

📚 Documentation

• Architecture Overview
• VLA Policy Design
• Safety System
• CAN Protocol
• Vision System
• Voice Control
• Research Notes

🗺️ Roadmap

Near Term

• ONNX runtime integration for broader model support
• ROS2 bridge for ecosystem compatibility
• Web dashboard for monitoring and control
• Simulation environment with Bevy

Long Term

• Distributed multi-robot coordination
• Federated learning across robot fleets
• Custom silicon accelerator support
• Formal verification of safety properties

🤝 Contributing

We welcome contributions! Please see CONTRIBUTING.md for guidelines.

Key areas for contribution:

• Additional VLA model implementations
• Hardware device drivers
• Safety constraint patterns
• Documentation and examples

📄 License

MIT License - see LICENSE for details.

🙏 Acknowledgments

• Built with Candle for ML inference
• Inspired by LeRobot for robot learning
• Safety patterns from aerospace and automotive industries

📬 Contact

• GitHub: @dirvine
• Project: Saorsa Labs

---

For the original Python implementation for SO-101 arms, see archive/python-so101