Integrate api-simulator for Quota Guard End-to-End HTTP Testing

[Trecek]

Summary

Add api-simulator as a dev dependency and use its mock_http_server pytest fixture to test the quota guard's real HTTP path end-to-end. Currently all quota tests monkeypatch _fetch_quota at the function level — the actual httpx client construction, header injection (Authorization: Bearer, anthropic-beta), response parsing, and error handling are never exercised. This plan introduces a base_url parameter to _fetch_quota and check_and_sleep_if_needed, then writes 7 tests that point the real httpx client at mock_http_server to exercise the full HTTP path.

Files changed: 3 (pyproject.toml, src/autoskillit/execution/quota.py, new tests/execution/test_quota_http.py)
Existing tests: Unchanged — all monkeypatch-based tests in test_quota.py remain as-is.

Requirements

DEP — Dependency Integration

• REQ-DEP-001: The system must include api-simulator as a dev-only dependency with a pinned git tag source.
• REQ-DEP-002: The api-simulator dependency must not appear in production runtime dependencies.

CFG — URL Configurability

• REQ-CFG-001: _fetch_quota must accept a base_url parameter defaulting to https://api.anthropic.com.
• REQ-CFG-002: check_and_sleep_if_needed must thread the base_url parameter through to _fetch_quota at both call sites.
• REQ-CFG-003: The production behavior must be unchanged when base_url is not explicitly provided.

HTTP — HTTP Path Verification

• REQ-HTTP-001: Tests must exercise the real httpx client construction path, not monkeypatch _fetch_quota.
• REQ-HTTP-002: Tests must verify that the Authorization: Bearer header is sent on the request.
• REQ-HTTP-003: Tests must verify that the anthropic-beta: oauth-2025-04-20 header is sent on the request.
• REQ-HTTP-004: Tests must verify correct JSON response parsing for the five_hour utilization shape.

ERR — Error Handling Verification

• REQ-ERR-001: Tests must verify fail-open behavior on HTTP 4xx/5xx responses.
• REQ-ERR-002: Tests must verify fail-open behavior on network timeout.
• REQ-ERR-003: Tests must verify that the above-threshold path triggers a double-fetch (two HTTP requests).

COMPAT — Backward Compatibility

• REQ-COMPAT-001: Existing test_quota.py tests must continue to pass unchanged.

Architecture Impact

Process Flow Diagram

%%{init: {'flowchart': {'nodeSpacing': 50, 'rankSpacing': 60, 'curve': 'basis'}}}%%
flowchart TB
    classDef terminal fill:#1a237e,stroke:#7986cb,stroke-width:2px,color:#fff;
    classDef stateNode fill:#004d40,stroke:#4db6ac,stroke-width:2px,color:#fff;
    classDef handler fill:#e65100,stroke:#ffb74d,stroke-width:2px,color:#fff;
    classDef phase fill:#6a1b9a,stroke:#ba68c8,stroke-width:2px,color:#fff;
    classDef newComponent fill:#2e7d32,stroke:#81c784,stroke-width:2px,color:#fff;
    classDef detector fill:#b71c1c,stroke:#ef5350,stroke-width:2px,color:#fff;

    START([START: check_and_sleep_if_needed])

    subgraph GatePhase ["Gate Phase"]
        direction TB
        ENABLED{"config.enabled?"}
        DISABLED(["RETURN<br/>should_sleep: false"])
    end

    subgraph CachePhase ["Cache Phase"]
        direction TB
        CACHE["_read_cache<br/>━━━━━━━━━━<br/>Read local JSON cache"]
        CACHE_HIT{"Cache fresh?<br/>━━━━━━━━━━<br/>age ≤ max_age?"}
    end

    subgraph FetchPhase ["HTTP Fetch Phase"]
        direction TB
        FETCH["● _fetch_quota<br/>━━━━━━━━━━<br/>★ base_url parameter<br/>httpx.AsyncClient GET"]
        BASEURL["★ base_url<br/>━━━━━━━━━━<br/>default: api.anthropic.com<br/>test: mock_http_server.url"]
        PARSE["Parse Response<br/>━━━━━━━━━━<br/>five_hour.utilization<br/>Z→+00:00 normalization"]
    end

    subgraph DecisionPhase ["Threshold Decision"]
        direction TB
        THRESHOLD{"utilization<br/>≥ threshold?"}
        RESETS_AT1{"resets_at<br/>is None?<br/>(Gate 1)"}
        REFETCH["● _fetch_quota re-fetch<br/>━━━━━━━━━━<br/>★ base_url threaded<br/>Double-fetch for accuracy"]
        RESETS_AT2{"resets_at<br/>still None?<br/>(Gate 2)"}
    end

    subgraph Results ["Results"]
        BELOW(["RETURN<br/>should_sleep: false"])
        FALLBACK1(["RETURN<br/>should_sleep: true<br/>reason: unknown_reset<br/>fallback ≥ 60s"])
        FALLBACK2(["RETURN<br/>should_sleep: true<br/>reason: unknown_reset<br/>fallback ≥ 60s"])
        SLEEP(["RETURN<br/>should_sleep: true<br/>sleep_seconds computed"])
        FAILOPEN(["RETURN<br/>should_sleep: false<br/>error key present"])
    end

    subgraph TestInfra ["★ Test Infrastructure (test_quota_http.py)"]
        direction TB
        MOCK["★ mock_http_server<br/>━━━━━━━━━━<br/>api-simulator fixture<br/>HTTP server"]
        REGISTER["★ register / register_sequence<br/>━━━━━━━━━━<br/>Custom endpoint responses<br/>Status codes, delays"]
        INSPECT["★ get_requests / request_count<br/>━━━━━━━━━━<br/>Header verification<br/>Double-fetch assertion"]
    end

    START --> ENABLED
    ENABLED -->|"false"| DISABLED
    ENABLED -->|"true"| CACHE
    CACHE --> CACHE_HIT
    CACHE_HIT -->|"fresh + below threshold"| BELOW
    CACHE_HIT -->|"miss or expired"| FETCH
    FETCH --> BASEURL
    BASEURL --> PARSE
    PARSE --> THRESHOLD
    THRESHOLD -->|"below"| BELOW
    THRESHOLD -->|"above"| RESETS_AT1
    RESETS_AT1 -->|"None"| FALLBACK1
    RESETS_AT1 -->|"present"| REFETCH
    REFETCH --> RESETS_AT2
    RESETS_AT2 -->|"None"| FALLBACK2
    RESETS_AT2 -->|"present"| SLEEP
    FETCH -.->|"HTTP error / timeout"| FAILOPEN

    MOCK -.->|"serves responses to"| BASEURL
    REGISTER -.->|"configures"| MOCK
    INSPECT -.->|"verifies headers / count"| FETCH

    class START terminal;
    class DISABLED,BELOW,FALLBACK1,FALLBACK2,SLEEP,FAILOPEN phase;
    class ENABLED,CACHE_HIT,THRESHOLD,RESETS_AT1,RESETS_AT2 stateNode;
    class CACHE,PARSE handler;
    class FETCH,REFETCH handler;
    class BASEURL,MOCK,REGISTER,INSPECT newComponent;

Loading

Color Legend:

Color	Category	Description
Dark Blue	Terminal	Entry point
Teal	State	Decision points and routing
Orange	Handler	Processing nodes (cache read, HTTP fetch, parse)
Green	New Component	★ New base_url parameter and test infrastructure
Purple	Phase	Result return paths

Development Diagram

%%{init: {'flowchart': {'nodeSpacing': 50, 'rankSpacing': 60, 'curve': 'basis'}}}%%
flowchart TB
    classDef cli fill:#1a237e,stroke:#7986cb,stroke-width:2px,color:#fff;
    classDef stateNode fill:#004d40,stroke:#4db6ac,stroke-width:2px,color:#fff;
    classDef handler fill:#e65100,stroke:#ffb74d,stroke-width:2px,color:#fff;
    classDef phase fill:#6a1b9a,stroke:#ba68c8,stroke-width:2px,color:#fff;
    classDef newComponent fill:#2e7d32,stroke:#81c784,stroke-width:2px,color:#fff;
    classDef output fill:#00695c,stroke:#4db6ac,stroke-width:2px,color:#fff;
    classDef detector fill:#b71c1c,stroke:#ef5350,stroke-width:2px,color:#fff;

    subgraph Deps ["● DEPENDENCY MANIFEST (pyproject.toml)"]
        direction TB
        PYPROJECT["● pyproject.toml<br/>━━━━━━━━━━<br/>hatchling build backend<br/>requires-python ≥ 3.11"]
        DEVDEPS["● dev optional-dependencies<br/>━━━━━━━━━━<br/>pytest, pytest-asyncio,<br/>pytest-httpx, pytest-xdist,<br/>pytest-timeout, ruff,<br/>import-linter, packaging"]
        APISIM["★ api-simulator<br/>━━━━━━━━━━<br/>New dev dependency<br/>HTTP mock fixture provider"]
        UVSRC["★ [tool.uv.sources]<br/>━━━━━━━━━━<br/>api-simulator pinned<br/>git: TalonT-Org/api-simulator<br/>branch: main"]
        UVLOCK["● uv.lock<br/>━━━━━━━━━━<br/>Regenerated with<br/>api-simulator entry"]
    end

    subgraph Quality ["CODE QUALITY GATES (pre-commit)"]
        direction TB
        FORMAT["ruff format<br/>━━━━━━━━━━<br/>Auto-fix code style<br/>reads + modifies src"]
        LINT["ruff check<br/>━━━━━━━━━━<br/>Auto-fix lint violations<br/>reads + modifies src"]
        TYPES["mypy<br/>━━━━━━━━━━<br/>Type checking<br/>reads src, reports only"]
        UVCHECK["uv lock check<br/>━━━━━━━━━━<br/>Verifies lockfile sync<br/>reads uv.lock"]
        SECRETS["gitleaks<br/>━━━━━━━━━━<br/>Secret scanning<br/>reads staged files"]
        IMPORTLINT["import-linter<br/>━━━━━━━━━━<br/>Layer contract enforcement<br/>IL-001 through IL-007"]
    end

    subgraph Testing ["TEST FRAMEWORK"]
        direction TB
        PYTEST["pytest + pytest-asyncio<br/>━━━━━━━━━━<br/>asyncio_mode=auto<br/>timeout=60s signal"]
        XDIST["pytest-xdist -n 4<br/>━━━━━━━━━━<br/>Parallel test workers<br/>worksteal distribution"]
        UNITQUOTA["● test_quota.py<br/>━━━━━━━━━━<br/>23 unit tests<br/>monkeypatch _fetch_quota<br/>mock signature updated"]
        HTTPQUOTA["★ test_quota_http.py<br/>━━━━━━━━━━<br/>7 end-to-end HTTP tests<br/>real httpx client path<br/>no monkeypatching"]
        MOCKSERVER["★ mock_http_server fixture<br/>━━━━━━━━━━<br/>api-simulator provides<br/>register / register_sequence<br/>get_requests / request_count"]
    end

    subgraph EntryPoints ["ENTRY POINTS"]
        CLI["autoskillit CLI<br/>━━━━━━━━━━<br/>autoskillit.cli:main"]
    end

    PYPROJECT --> DEVDEPS
    DEVDEPS --> APISIM
    APISIM --> UVSRC
    UVSRC --> UVLOCK

    PYPROJECT --> FORMAT
    FORMAT --> LINT
    LINT --> TYPES
    TYPES --> UVCHECK
    UVCHECK --> SECRETS
    SECRETS --> IMPORTLINT

    IMPORTLINT --> PYTEST
    PYTEST --> XDIST
    XDIST --> UNITQUOTA
    XDIST --> HTTPQUOTA
    APISIM -.->|"provides fixture"| MOCKSERVER
    MOCKSERVER -.->|"injected into"| HTTPQUOTA

    PYPROJECT --> CLI

    class PYPROJECT,DEVDEPS,UVLOCK phase;
    class APISIM,UVSRC,HTTPQUOTA,MOCKSERVER newComponent;
    class UNITQUOTA handler;
    class FORMAT,LINT,TYPES,UVCHECK,SECRETS,IMPORTLINT detector;
    class PYTEST,XDIST handler;
    class CLI output;

Loading

Color Legend:

Color	Category	Description
Purple	Build Config	pyproject.toml, dev deps, lockfile
Green	New Component	★ api-simulator dep, uv.sources, HTTP test file, mock fixture
Orange	Test Framework	pytest, xdist, existing test_quota.py
Red	Quality Gates	ruff, mypy, uv lock check, gitleaks, import-linter
Dark Teal	Entry Points	CLI entry point

Closes #607

Implementation Plan

Plan file: /home/talon/projects/autoskillit-runs/impl-20260404-190816-816130/.autoskillit/temp/make-plan/integrate_api_simulator_quota_guard_plan_2026-04-04_191500.md

🤖 Generated with Claude Code via AutoSkillit

Token Usage Summary

Step	input	output	cached	count	time
plan	5.5k	76.5k	6.0M	5	32m 41s
verify	3.1k	86.2k	5.4M	5	31m 25s
implement	1.1k	116.2k	22.6M	6	50m 55s
fix	214	28.4k	3.5M	5	30m 58s
audit_impl	137	58.9k	3.1M	5	19m 28s
open_pr	100	51.3k	3.9M	3	16m 38s
Total	10.2k	417.5k	44.5M		3h 2m

[Trecek]

AutoSkillit PR Review — Verdict: changes_requested

Found 12 inline findings (1 critical, 11 warnings). See inline comments.

[Trecek]

[warning] tests: _reset_mock is autouse=True and calls mock_http_server.reset() per-test. If mock_http_server is session-scoped (common for server fixtures), route registrations from parallel xdist workers sharing the same server can cross-contaminate. Verify that mock_http_server is function-scoped before relying on this reset pattern.

[Trecek]

Investigated — this is intentional. pytest-xdist with -n 4 creates a separate OS process per worker; session-scoped fixtures are initialized once per worker process, not globally. Each worker gets its own mock_http_server instance on a different port (confirmed via api_simulator/plugin.py). Tests within a worker run sequentially, so the autouse _reset_mock cleanly isolates state. No cross-worker contamination is possible in this architecture.