docs: consolidate and improve documentation structure (#19)

- Fix TLS flag inconsistency (--no-tls-enabled -> --tls-enabled=false)
- Add PyPI trusted publishing guide to docs index
- Add CLI alternatives throughout quickstart guide
- Improve cross-references between documentation files
- Enhance documentation structure and organization
- Update OpenAPI spec with correct TLS flag

Addresses #4

Author: Nigel Brown

README.md

@@ -43,7 +43,7 @@ uv run python -m mcp_tef
 **Security:**
 - **Default**: HTTPS with auto-generated self-signed certificate
 - **Custom certs**: `--tls-cert-file` and `--tls-key-file` flags
-- **Development**: `--no-tls-enabled` for HTTP (⚠️ not for production)
+- **Development**: `--tls-enabled=false` for HTTP (⚠️ not for production)
 
 See [docs/testing-with-ollama.md](docs/testing-with-ollama.md) and [docs/tls-configuration.md](docs/tls-configuration.md) for details.
 

docs/README.md

@@ -33,6 +33,16 @@ This directory contains detailed documentation for the mcp-tef project.
 - Production deployment guidelines
 - Troubleshooting TLS issues
 
+### Publishing and Deployment
+
+**[PyPI Trusted Publishing Setup](pypi-trusted-publishing-setup.md)** - PyPI publishing configuration
+- Configure trusted publishing for `mtef` CLI package
+- GitHub Actions OIDC setup
+- PyPI project configuration
+- Troubleshooting publishing issues
+
+> **Note**: Currently, CLI releases are published to GitHub Releases. This guide is for future PyPI publishing setup.
+
 ### Project Information
 
 **[Current Specification](current-specification.md)** - Comprehensive system specification and requirements
@@ -61,10 +71,18 @@ This directory contains detailed documentation for the mcp-tef project.
 - **Main README**: [`../README.md`](../README.md) - Project overview and quick start
 - **Development Guide**: [`../CLAUDE.md`](../CLAUDE.md) - Development workflow and principles
 
-## Getting Started
+## Documentation Guide
+
+**For new users:**
+1. Start with the [**Quickstart Guide**](quickstart.md) for step-by-step tutorials
+2. See the main [`README.md`](../README.md) for project overview and features
+
+**For developers:**
+1. Read [`CLAUDE.md`](../CLAUDE.md) for development workflow and principles
+2. See [`testing-with-ollama.md`](testing-with-ollama.md) for testing strategies
+3. Check [`tls-configuration.md`](tls-configuration.md) for HTTPS/TLS setup
 
-1. Start with the main [`README.md`](../README.md) for project overview and quick start
-2. Read [`testing-with-ollama.md`](testing-with-ollama.md) for testing strategies
-3. Check [`tls-configuration.md`](tls-configuration.md) for HTTPS setup
-4. See [`CLAUDE.md`](../CLAUDE.md) for development workflow
+**For deployment:**
+1. Review [`tls-configuration.md`](tls-configuration.md) for production TLS setup
+2. See [`pypi-trusted-publishing-setup.md`](pypi-trusted-publishing-setup.md) for PyPI publishing (future)
 

docs/openapi.yaml

@@ -23,7 +23,7 @@ servers:
   - url: https://localhost:8000
     description: Local development server (HTTPS with auto-generated cert)
   - url: http://localhost:8000
-    description: Local development server (HTTP, use --no-tls-enabled)
+    description: Local development server (HTTP, use --tls-enabled=false)
 
 tags:
   - name: mcp-servers

docs/pypi-trusted-publishing-setup.md

@@ -2,6 +2,8 @@
 
 This document describes how to configure PyPI trusted publishing for the `mtef` package.
 
+> **Note**: Currently, the CLI is published to GitHub Releases only. This guide is for future PyPI publishing setup. The workflow file referenced (`release-cli.yml`) currently creates GitHub releases but does not publish to PyPI yet.
+
 ## Overview
 
 Trusted publishing allows GitHub Actions to publish to PyPI without storing API tokens. It uses OpenID Connect (OIDC) to establish trust between GitHub and PyPI.
@@ -53,7 +55,7 @@ If the `mtef` project doesn't exist on PyPI yet:
 
 ## How It Works
 
-The release workflow (`.github/workflows/release-cli.yml`) uses:
+When PyPI publishing is enabled, the release workflow (`.github/workflows/release-cli.yml`) would use:
 
 ```yaml
 permissions:
@@ -110,6 +112,11 @@ When the workflow runs:
   - A pending publisher configured on your PyPI account
   - Manual project creation on PyPI first
 
+## Related Documentation
+
+- [Main README](../README.md) - Project overview and release process
+- [CLI Documentation](../cli/README.md) - CLI usage and installation
+
 ## References
 
 - [PyPI Trusted Publishing Documentation](https://docs.pypi.org/trusted-publishers/)

docs/quickstart.md

@@ -50,15 +50,29 @@ cp .env.example .env
 
 ### Start the Server
 
+**Option 1: Run directly (Python)**
+
 ```bash
 # Default: HTTPS with auto-generated self-signed certificate
 uv run python -m mcp_tef
 
 # OR: HTTP mode for development (insecure)
-uv run python -m mcp_tef --no-tls-enabled
+uv run python -m mcp_tef --tls-enabled=false
+```
+
+The server starts at `https://localhost:8000` (or `http://localhost:8000` with `--tls-enabled=false`).
+
+**Option 2: Deploy with CLI (Docker)**
+
+```bash
+# Install the CLI
+uv tool install "mtef @ git+https://github.com/StacklokLabs/mcp-tef.git#subdirectory=cli"
+
+# Deploy the server
+mtef deploy --health-check
 ```
 
-The server starts at `https://localhost:8000` (or `http://localhost:8000` with `--no-tls-enabled`).
+> **💡 Tip**: The CLI deploys mcp-tef as a Docker container from GitHub Container Registry. See [CLI documentation](../cli/README.md) for details.
 
 ### Access API Documentation
 
@@ -72,7 +86,7 @@ Open your browser (accept the self-signed certificate warning if using HTTPS):
 # HTTPS (default)
 curl -k https://localhost:8000/health
 
-# HTTP (with --no-tls-enabled)
+# HTTP (with --tls-enabled=false)
 curl http://localhost:8000/health
 ```
 
@@ -123,6 +137,8 @@ Stateless analysis that:
 
 Use your actual MCP server URLs - no need to register servers first!
 
+**Option A: Using the REST API (curl)**
+
 ```bash
 curl -k -X POST https://localhost:8000/test-cases \
   -H "Content-Type: application/json" \
@@ -141,6 +157,23 @@ curl -k -X POST https://localhost:8000/test-cases \
   }'
 ```
 
+**Option B: Using the CLI**
+
+```bash
+mtef test-case create \
+  --name "Weather query test" \
+  --query "What is the weather in San Francisco?" \
+  --expected-server "https://my-mcp-server.com/sse" \
+  --expected-tool "get_weather" \
+  --expected-params '{"location": "San Francisco"}' \
+  --servers "https://my-mcp-server.com/sse,https://another-server.com/sse"
+```
+
+> **Note**: The `--servers` flag accepts comma-separated URLs. See [CLI documentation](../cli/README.md) for more examples.
+```
+
+> **💡 Tip**: The CLI provides a simpler interface for common operations. See [CLI documentation](../cli/README.md) for full details.
+
 **Response**:
 ```json
 {
@@ -168,6 +201,8 @@ curl -k -X POST https://localhost:8000/test-cases \
 
 Execute the test with runtime API key and model configuration:
 
+**Option A: Using the REST API (curl)**
+
 ```bash
 # Using OpenRouter (or OpenAI, Anthropic)
 curl -k -X POST https://localhost:8000/test-cases/550e8400-e29b-41d4-a716-446655440000/run \
@@ -197,6 +232,21 @@ curl -k -X POST https://localhost:8000/test-cases/550e8400-e29b-41d4-a716-446655
   }'
 ```
 
+**Option B: Using the CLI**
+
+```bash
+# Using OpenRouter
+mtef test-run execute 550e8400-e29b-41d4-a716-446655440000 \
+  --model-provider openrouter \
+  --model-name anthropic/claude-3.5-sonnet \
+  --api-key sk-or-v1-...
+
+# Using Ollama (no API key needed)
+mtef test-run execute 550e8400-e29b-41d4-a716-446655440000 \
+  --model-provider ollama \
+  --model-name llama3.2:3b
+```
+
 **What Happens**:
 1. **Tool Ingestion** (1-5s per server): Fresh tools loaded from all `available_mcp_servers`
 2. **LLM Evaluation**: LLM receives query and all tool descriptions, selects one
@@ -231,10 +281,18 @@ curl -k -X POST https://localhost:8000/test-cases/550e8400-e29b-41d4-a716-446655
 
 ### Step 3: View Detailed Results
 
+**Option A: Using the REST API (curl)**
+
 ```bash
 curl -k https://localhost:8000/test-runs/660e8400-e29b-41d4-a716-446655440001/result
 ```
 
+**Option B: Using the CLI**
+
+```bash
+mtef test-run get 660e8400-e29b-41d4-a716-446655440001
+```
+
 **Response**:
 ```json
 {
@@ -270,10 +328,14 @@ curl -k https://localhost:8000/test-runs/660e8400-e29b-41d4-a716-446655440001/re
 
 After running multiple tests:
 
+**Using the REST API (curl)**
+
 ```bash
 curl -k https://localhost:8000/metrics/summary
 ```
 
+> **Note**: Aggregate metrics are currently only available via the REST API. CLI support may be added in future releases.
+
 **Response**:
 ```json
 {
@@ -312,6 +374,8 @@ curl -k https://localhost:8000/metrics/summary
 
 The easiest way - just provide your MCP server URLs:
 
+**Using the REST API (curl)**
+
 ```bash
 curl -k -X POST https://localhost:8000/similarity/analyze \
   -H "Content-Type: application/json" \
@@ -325,10 +389,21 @@ curl -k -X POST https://localhost:8000/similarity/analyze \
   }'
 ```
 
+**Using the CLI**
+
+```bash
+mtef similarity analyze \
+  --server-urls "https://my-server-1.com/sse,https://my-server-2.com/sse" \
+  --threshold 0.85 \
+  --recommendations
+```
+
 ### Option B: Analyze Tools Directly (No Server Fetch)
 
 If you already have tool definitions:
 
+**Using the REST API (curl)**
+
 ```bash
 curl -k -X POST https://localhost:8000/similarity/analyze \
   -H "Content-Type: application/json" \
@@ -356,6 +431,8 @@ curl -k -X POST https://localhost:8000/similarity/analyze \
   }'
 ```
 
+> **Note**: Direct tool list analysis is currently only available via the REST API. CLI support requires MCP server URLs.
+
 ### Understanding Similarity Results
 
 **Response**:
@@ -420,6 +497,8 @@ curl -k -X POST https://localhost:8000/similarity/analyze \
 
 For visualization or analysis of all pairwise similarities:
 
+**Using the REST API (curl)**
+
 ```bash
 curl -k -X POST https://localhost:8000/similarity/matrix \
   -H "Content-Type: application/json" \
@@ -429,6 +508,16 @@ curl -k -X POST https://localhost:8000/similarity/matrix \
   }'
 ```
 
+**Using the CLI**
+
+```bash
+mtef similarity matrix \
+  --server-urls "https://my-server.com/sse" \
+  --threshold 0.7
+```
+
+> **Note**: The `--server-urls` flag accepts comma-separated URLs for multiple servers.
+
 #### Generate Overlap Matrix
 
 Multi-dimensional analysis (semantic + parameters + description):
@@ -455,6 +544,8 @@ curl -k -X POST https://localhost:8000/similarity/overlap-matrix \
 
 ### Analyze Tools from MCP Server(s)
 
+**Using the REST API (curl)**
+
 ```bash
 # Single server
 curl -k "https://localhost:8000/mcp-servers/tools/quality?server_urls=https://my-server.com/sse&model_provider=openrouter&model_name=anthropic/claude-3.5-sonnet" \
@@ -468,6 +559,29 @@ curl -k "https://localhost:8000/mcp-servers/tools/quality?server_urls=https://se
 curl -k "https://localhost:8000/mcp-servers/tools/quality?server_urls=https://my-server.com/sse&model_provider=ollama&model_name=llama3.2:3b"
 ```
 
+**Using the CLI**
+
+```bash
+# Single server with OpenRouter
+mtef tool-quality \
+  --server-urls "https://my-server.com/sse" \
+  --model-provider openrouter \
+  --model-name anthropic/claude-3.5-sonnet \
+  --api-key sk-or-v1-...
+
+# Multiple servers (comma-separated)
+mtef tool-quality \
+  --server-urls "https://server1.com/sse,https://server2.com/sse" \
+  --model-provider openrouter \
+  --api-key sk-or-v1-...
+
+# Using Ollama (no API key)
+mtef tool-quality \
+  --server-urls "https://my-server.com/sse" \
+  --model-provider ollama \
+  --model-name llama3.2:3b
+```
+
 ### Understanding Quality Results
 
 **Response**:
@@ -692,7 +806,7 @@ rm mcp_eval.db mcp_eval.db-shm mcp_eval.db-wal
 curl -k https://localhost:8000/health
 
 # Option 2: Disable TLS for development
-uv run python -m mcp_tef --no-tls-enabled
+uv run python -m mcp_tef --tls-enabled=false
 # Then use http://localhost:8000
 ```
 
@@ -805,6 +919,7 @@ ollama pull llama3.2:3b
 5. **Understand Data Model**: [docs/data-model.md](data-model.md)
 6. **Deep Dive: Ollama Testing**: [docs/testing-with-ollama.md](testing-with-ollama.md)
 7. **Production TLS Setup**: [docs/tls-configuration.md](tls-configuration.md)
+8. **Browse All Documentation**: [docs/README.md](README.md)
 
 ---
 
@@ -817,7 +932,7 @@ ollama pull llama3.2:3b
 uv run python -m mcp_tef
 
 # Start server (HTTP)
-uv run python -m mcp_tef --no-tls-enabled
+uv run python -m mcp_tef --tls-enabled=false
 
 # Run tests
 task test
@@ -852,5 +967,30 @@ open https://localhost:8000/docs
 
 ---
 
+## Choosing Between API and CLI
+
+Both the REST API and CLI provide access to mcp-tef functionality:
+
+| Use Case | Recommended Method |
+|----------|-------------------|
+| **Interactive exploration** | REST API (curl) - See examples in workflows above |
+| **Scripting and automation** | CLI (`mtef`) - Better for shell scripts and CI/CD |
+| **Docker deployment** | CLI (`mtef deploy`) - Simplifies container management |
+| **Direct server control** | Python (`uv run python -m mcp_tef`) - Full control over configuration |
+
+**CLI Installation:**
+
+```bash
+# Install from GitHub release
+uv tool install https://github.com/StacklokLabs/mcp-tef/releases/download/cli-v0.1.0/mcp_tef_cli-0.1.0-py3-none-any.whl
+
+# Or install from Git
+uv tool install "mtef @ git+https://github.com/StacklokLabs/mcp-tef.git#subdirectory=cli"
+```
+
+See the [CLI documentation](../cli/README.md) for complete CLI usage and examples.
+
+---
+
 **Need help?** Check the [troubleshooting section](#troubleshooting) or review the [full documentation](README.md).
 

docs/testing-with-ollama.md

@@ -470,6 +470,8 @@ pytest --tb=short
 
 ## Related Resources
 
+- [Quickstart Guide](quickstart.md) - Complete user workflows and examples
+- [TLS Configuration](tls-configuration.md) - HTTPS/TLS setup for production
 - [Ollama Documentation](https://github.com/ollama/ollama)
 - [Ollama API Reference](https://github.com/ollama/ollama/blob/main/docs/api.md)
 - [Available Models](https://ollama.ai/library)