Skip to content

feat: add context7-cache skill — local documentation cache#685

Open
m2015agg wants to merge 2 commits intoanthropics:mainfrom
m2015agg:add-context7-cache-skill
Open

feat: add context7-cache skill — local documentation cache#685
m2015agg wants to merge 2 commits intoanthropics:mainfrom
m2015agg:add-context7-cache-skill

Conversation

@m2015agg
Copy link
Copy Markdown

Summary

Adds context7-cache — a skill that caches Context7 library documentation locally in SQLite+FTS5, serving docs via CLI instead of MCP round-trips.

  • npm package: @m2015agg/context7-skill (npm | GitHub)
  • Auto-detects project dependencies from package.json, requirements.txt, pyproject.toml, etc.
  • Pre-caches documentation weighted by import frequency
  • FTS5 full-text search across all cached libraries
  • Version-aware caching from pinned dependency versions
  • Offline resilient — serves from cache when API is unreachable

Anthropic SkillsBench Results

Benchmarked using the skill-creator framework with LLM grader (Sonnet 4.6). 3 evals × 3 runs per config.

Metric CLI Cache (with_skill) MCP Server (without_skill) Delta
Pass Rate 93% ± 10% 87% ± 12% +6%
Speed 17.9s 31.9s 1.8x faster
Hallucinations None Fabricated FastAPI scope param (3/3 runs)

Per-eval breakdown

  • FastAPI Depends(): CLI 100% vs MCP 75% (MCP hallucinated non-existent scope parameter)
  • Celery+Redis config: CLI 80% vs MCP 87% (both missed task decorator sometimes)
  • Supabase RLS policies: CLI 100% vs MCP 100% (tied)

Key finding

The MCP server returned Context7 docs that included a fabricated scope parameter on FastAPI's Depends() — caught by the LLM grader in all 3 runs. The CLI cache served pre-verified cached docs without this hallucination.

Files added

  • skills/context7-cache/SKILL.md — 131 lines, follows spec
  • skills/context7-cache/references/benchmark.json — full benchmark data
  • skills/context7-cache/references/benchmark.md — human-readable summary
  • skills/context7-cache/scripts/check-setup.sh — setup verification

Test plan

  • SKILL.md follows Agent Skills spec (name format, description < 1024 chars, < 500 lines)
  • Directory name matches name field
  • Benchmarked with skill-creator grader agent
  • Fair comparison: MCP given 10 turns (enough for resolve + fetch + respond)
  • npm package published and installable

🤖 Generated with Claude Code

@m2015agg m2015agg marked this pull request as draft March 26, 2026 16:56
@m2015agg m2015agg marked this pull request as ready for review March 26, 2026 16:56
matthgreen and others added 2 commits April 2, 2026 15:52
Caches Context7 library docs in SQLite+FTS5, serving via CLI instead
of MCP round-trips. Auto-detects project dependencies, pre-caches docs,
and provides FTS5 search across all cached libraries.

Benchmarked with Anthropic SkillsBench (LLM grader, Sonnet 4.6):
- CLI cache: 93% pass rate (± 10%) in 17.9s
- MCP server: 87% pass rate (± 12%) in 31.9s
- CLI is 1.8x faster, no hallucinations observed
- MCP fabricated a non-existent FastAPI scope parameter in 3/3 runs

3 evals: FastAPI dependency injection, Celery+Redis config, Supabase RLS
Each eval: 3 runs per config, LLM-graded assertions with evidence

Includes: SKILL.md, benchmark.json, benchmark.md, setup check script.
Requires: npm install -g @m2015agg/context7-skill

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Remove non-standard fields (compatibility, metadata, allowed-tools)
- Match format of existing skills (claude-api, webapp-testing)
- Add LICENSE.txt

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants