Skip to content

Audit: Standardize agent definitions, cross-references, and QA authority #46

@mriechers

Description

@mriechers

Description

A comprehensive audit of all 8 agent definitions revealed significant inconsistencies in naming, missing cross-references between agents, UI gaps, and authority issues with the QA Manager. This issue tracks the work needed to bring all agent definitions into alignment.

Audit Findings

Naming Inconsistencies

Agent Backend ID Prompt Name UI Name
Analyst analyst transcript-analyst "Analyst"
SEO seo seo-specialist "SEO Specialist"
Manager manager "Quality Assurance Manager" "QA Manager"
Timestamp timestamp "chapter marker agent" MISSING

Missing UI Representation

  • Timestamp agent exists in backend (worker.py, timestamp.md) but is not listed in web/src/constants/agents.ts — invisible to users

QA Manager Authority Gaps

  • Manager prompt implies authority to RETRY/ESCALATE/FIX/FAIL, but worker.py actually executes those decisions — prompt language is misleading
  • Manager does not check Formatter's needs_review flag
  • Manager has no quality criteria for Timestamp output
  • Manager should have clear, documented authority to flag problems, re-run jobs, and manage the pipeline

Missing Cross-References

  • Upstream agents (Analyst, Formatter, SEO) don't know the Manager QA checkpoint exists
  • SEO doesn't know Manager's validation rules in advance
  • Timestamp agent has no QA criteria in Manager's prompt

Orphaned Agent

  • AP Style Bot (ap_style_bot.md) has a prompt file but no pipeline integration, no UI, no config — needs to be integrated or archived

Conflicting Instruction Sources

  • Copy Editor has two instruction files: canonical (EDITOR_AGENT_INSTRUCTIONS.md) and CLI override (.claude/agents/copy_editor.md)

Scope of Work

Agent Prompt Updates (.claude/agents/*.md)

  • Standardize naming conventions across all prompts
  • Add pipeline awareness section to each agent (who comes before/after, what the Manager checks)
  • Update Manager prompt to clarify authority model and add timestamp QA criteria
  • Add needs_review flag awareness to Manager
  • Resolve or archive AP Style Bot

UI Updates (web/src/constants/agents.ts)

  • Add Timestamp agent to constants
  • Ensure display names match standardized naming

Config/Backend Alignment

  • Verify config/llm-config.json and worker.py use consistent identifiers
  • Consider creating a single agent registry as source of truth

Files Involved

  • .claude/agents/analyst.md
  • .claude/agents/formatter.md
  • .claude/agents/seo.md
  • .claude/agents/manager.md
  • .claude/agents/timestamp.md
  • .claude/agents/copy_editor.md
  • .claude/agents/ap_style_bot.md
  • web/src/constants/agents.ts
  • claude-desktop-project/EDITOR_AGENT_INSTRUCTIONS.md
  • api/services/worker.py
  • config/llm-config.json
  • AGENTS.md

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions