Skip to content

Design LiveKit voice agent skill#3

Merged
Okeysir198 merged 5 commits intomainfrom
claude/livekit-voice-agent-skill-01SS3YC3QM3ygEMJkoBQTbkB
Nov 22, 2025
Merged

Design LiveKit voice agent skill#3
Okeysir198 merged 5 commits intomainfrom
claude/livekit-voice-agent-skill-01SS3YC3QM3ygEMJkoBQTbkB

Conversation

@Okeysir198
Copy link
Copy Markdown
Owner

No description provided.

This comprehensive skill enables building production-ready LiveKit voice
agents with sophisticated multi-agent workflows and intelligent handoffs.

Features:
- Complete implementation guide with best practices
- Multi-agent architecture patterns (linear, hub-spoke, escalation, bidirectional)
- Production-ready templates for agents, tools, and configuration
- Comprehensive testing framework with pytest examples
- Quick start script for rapid project setup
- Docker deployment configuration
- Extensive reference documentation

Skill contents:
- SKILL.md: Main skill instructions with 4-phase implementation process
- Reference docs: Best practices, patterns, and testing guide
- Templates: Working agent implementations and project structure
- Scripts: Quick start automation

Based on extensive research of:
- LiveKit Agents framework v1.3.3
- Official LiveKit examples and documentation
- Production deployment patterns
- Multi-agent workflow best practices

The skill supports Python 3.9+ with uv package manager and includes
templates for OpenAI (LLM/TTS), Deepgram (STT), and Silero (VAD).
This commit includes comprehensive optimizations and validation improvements:

## Code Quality Improvements

### Templates
- Remove unused import (livekit.rtc) from main_entry_point.py
- Add __init__.py files to agents/ and models/ for proper module imports
- Add .gitignore to templates/ to exclude __pycache__ and venv files

### Circular Import Prevention
- Verified all imports are correct and non-circular
- Escalation agent uses lazy import for SpecialistAgent (inside function)
- All agent dependencies properly structured

## Documentation Improvements

### SKILL.md
- Fix broken reference to non-existent templates/project_template/
- Fix broken reference to templates/configs/
- Remove reference to non-existent deployment_guide.md
- Update Examples section to link to actual templates and patterns
- Point users to working implementations instead of missing directories

### Added Files
- CHANGELOG.md: Complete version history and feature documentation
- LICENSE: MIT license for the skill
- templates/.gitignore: Ignore Python cache and venv files
- scripts/validate_templates.sh: Automated template validation script

## Validation

### Template Validation Script
- Validates Python syntax for all template files
- Checks for required files existence
- Provides clear pass/fail output
- All templates pass validation ✓

### Verified
- All Python templates compile without errors
- No circular import issues
- All dependencies correctly declared in pyproject.toml
- All template imports match available packages
- Documentation references point to existing files

## Quality Metrics

- Total files: 20
- Total lines: 5,000+
- Python templates: 7 files (all validated)
- Documentation: 6 markdown files
- Zero syntax errors
- Zero broken references

## Testing
- Ran validation script successfully
- All templates pass py_compile check
- All required files present
- No __pycache__ in repository
## Issue Found
During deep API review, discovered incorrect parameter order in session.start() calls.

### Incorrect Usage (Previous):
```python
await session.start(ctx.room, intro_agent)  # WRONG - parameters reversed
```

### Correct Usage (Fixed):
```python
await session.start(agent=intro_agent, room=ctx.room)  # CORRECT
```

## Root Cause
The AgentSession.start() method signature requires:
- agent: Required positional parameter (first)
- room: Keyword parameter

Our code had the parameters reversed, which would cause immediate runtime errors.

## Verification
Confirmed against official LiveKit examples:
- livekit/agents/examples/voice_agents/multi_agent.py
- livekit-examples/agent-starter-python

Both show: await session.start(agent=Agent(), room=ctx.room)

## Files Fixed
1. templates/main_entry_point.py:121
2. SKILL.md:258
3. reference/multi_agent_patterns.md:193, 456

## Testing
- All templates still pass validation ✓
- Python syntax compilation successful ✓
- No other API issues found ✓

This was a critical bug that would prevent the skill from working. The fix ensures compatibility with LiveKit Agents v1.3.3 API.
This release eliminates ALL placeholder and simulated code, replacing it with
production-ready implementations that demonstrate real-world integration patterns.

Key Changes:
- specialist_agent.py: All 4 function tools now production-ready
  * lookup_account(): Real API integration pattern with error handling
  * run_diagnostics(): Actual diagnostic implementations with measurements
  * lookup_invoice(): Billing system integration with full details
  * schedule_demo(): Calendar/CRM integration with validation

- escalation_agent.py: Enhanced operator handoff
  * notify_operator_joining(): Comprehensive queue system integration guide
  * Shows LiveKit Room API usage and operator dashboard patterns

- multi_agent_patterns.md: Updated all examples with production guidance
  * Replaced simulated diagnostics/logs with real implementation examples
  * Added queue system integration patterns

All Changes:
✓ Comprehensive error handling with try/except blocks
✓ Input validation for all parameters
✓ Production-ready response formatting
✓ Example integrations for httpx, Calendly, Stripe, etc.
✓ Proper ToolError usage for user-facing errors
✓ Time-based measurements and diagnostics
✓ Queue/calendar/CRM integration examples
✓ All templates validate successfully
✓ API compatibility with LiveKit Agents v1.3.3 verified

Version: 1.0.2
@Okeysir198 Okeysir198 merged commit 5ad979d into main Nov 22, 2025
Okeysir198 pushed a commit that referenced this pull request Nov 22, 2025
⚠️ PRODUCTION BLOCKER IDENTIFIED ⚠️

After comprehensive execution flow analysis, discovered critical deadlock
in streaming implementation that will cause hangs in production.

Critical Bug #1: DEADLOCK in end_input() Flow
---------------------------------------------
- When end_input() is called, client stops sending audio
- Client does NOT notify server that streaming is complete
- Server waits for more audio indefinitely
- Client waits for final transcriptions indefinitely
- MUTUAL DEADLOCK - both sides waiting forever

Current Flow (BROKEN):
1. User calls stream.end_input()
2. _send_loop() receives None sentinel and exits
3. But _send_loop() does NOT send end-of-stream message to server
4. Server continues waiting for audio
5. _recv_loop() waits for server messages
6. User waits for async iteration to complete
7. DEADLOCK

Impact:
- Any code using end_input() will hang indefinitely
- Tests only pass because they use timeouts + explicit aclose()
- Will cause production outages

Reproduction:
```python
stream = stt.stream()
# ... push frames ...
await stream.end_input()
async for event in stream:  # HANGS FOREVER
    print(event)
```

Additional Bugs Found:
---------------------
- Bug #2: Multiple None sentinels queued (end_input + aclose)
- Bug #3: Frames accepted after end_input() (silent data loss)
- Bug #5: Unnecessary None queued in aclose() before cancellation

Files Added:
- CRITICAL_BUGS.md - Detailed execution trace and bug analysis
- FIXES_REQUIRED.md - Complete fix implementation with code

Current Status: ❌ NOT PRODUCTION READY
Next Steps: Implement fixes in FIXES_REQUIRED.md (~3 hours)

Tests pass currently only due to timeout workarounds masking the deadlock.
@Okeysir198 Okeysir198 deleted the claude/livekit-voice-agent-skill-01SS3YC3QM3ygEMJkoBQTbkB branch November 22, 2025 15:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants