Skip to content

Commit 0e01a14

Browse files
lyonzinlyonzin
committed
feat: enterprise CI/CD pipeline + 55 tests
CI/CD (4 GitHub Actions workflows): - ci.yml: lint (ruff) + test matrix (Python 3.11/3.12 × Ubuntu/Windows) + coverage - security.yml: CodeQL SAST + pip-audit dependency scan (push + weekly cron) - release.yml: auto PyPI publish (trusted publisher) + Docker build + GitHub release on tag - dependabot.yml: weekly dependency + GitHub Actions updates Tests (55 tests, 4 modules): - test_config: 9 tests (no Ollama refs, model, formats, expansions, CVE aliases) - test_ingestion: 13 tests (all parsers, markdown chunking, code block protection, min size) - test_search: 19 tests (query expansion, BM25, cache hit/miss/evict, keyword routing) - test_tools: 14 tests (input validation, error handling for all 12 MCP tools) Quality tooling: - ruff lint + format check in CI - pytest-cov with 70% minimum threshold - pyproject.toml configured for ruff, pytest, coverage
1 parent 47ceec8 commit 0e01a14

14 files changed

Lines changed: 858 additions & 23 deletions

‎.github/dependabot.yml‎

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
version: 2
2+
updates:
3+
- package-ecosystem: "pip"
4+
directory: "/"
5+
schedule:
6+
interval: "weekly"
7+
reviewers:
8+
- "lyonzin"
9+
labels:
10+
- "dependencies"
11+
open-pull-requests-limit: 5
12+
13+
- package-ecosystem: "github-actions"
14+
directory: "/"
15+
schedule:
16+
interval: "weekly"
17+
reviewers:
18+
- "lyonzin"
19+
labels:
20+
- "ci"

‎.github/workflows/ci.yml‎

Lines changed: 59 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,59 @@
1+
name: CI
2+
3+
on:
4+
push:
5+
branches: [master]
6+
pull_request:
7+
branches: [master]
8+
9+
permissions:
10+
contents: read
11+
12+
jobs:
13+
lint:
14+
name: Lint & Format
15+
runs-on: ubuntu-latest
16+
steps:
17+
- uses: actions/checkout@v4
18+
- uses: actions/setup-python@v5
19+
with:
20+
python-version: "3.12"
21+
- run: pip install ruff
22+
- name: Ruff lint
23+
run: ruff check mcp_server/ tests/
24+
- name: Ruff format check
25+
run: ruff format --check mcp_server/ tests/
26+
27+
test:
28+
name: Test (Python ${{ matrix.python-version }}, ${{ matrix.os }})
29+
needs: lint
30+
runs-on: ${{ matrix.os }}
31+
strategy:
32+
fail-fast: false
33+
matrix:
34+
python-version: ["3.11", "3.12"]
35+
os: [ubuntu-latest, windows-latest]
36+
37+
steps:
38+
- uses: actions/checkout@v4
39+
- uses: actions/setup-python@v5
40+
with:
41+
python-version: ${{ matrix.python-version }}
42+
43+
- name: Install dependencies
44+
run: |
45+
python -m pip install --upgrade pip
46+
pip install -r requirements.txt
47+
pip install pytest pytest-cov
48+
49+
- name: Run tests with coverage
50+
run: pytest tests/ -v --cov=mcp_server --cov-report=xml --cov-report=term-missing
51+
52+
- name: Upload coverage
53+
if: matrix.python-version == '3.12' && matrix.os == 'ubuntu-latest'
54+
uses: codecov/codecov-action@v4
55+
with:
56+
file: coverage.xml
57+
fail_ci_if_error: false
58+
env:
59+
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}

‎.github/workflows/release.yml‎

Lines changed: 84 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,84 @@
1+
name: Release
2+
3+
on:
4+
push:
5+
tags: ["v*.*.*"]
6+
7+
permissions:
8+
contents: write
9+
id-token: write # Required for PyPI Trusted Publisher
10+
11+
jobs:
12+
test:
13+
name: Pre-release Tests
14+
runs-on: ubuntu-latest
15+
steps:
16+
- uses: actions/checkout@v4
17+
- uses: actions/setup-python@v5
18+
with:
19+
python-version: "3.12"
20+
- run: |
21+
pip install -r requirements.txt
22+
pip install pytest
23+
pytest tests/ -v
24+
25+
publish-pypi:
26+
name: Publish to PyPI
27+
needs: test
28+
runs-on: ubuntu-latest
29+
environment: pypi
30+
steps:
31+
- uses: actions/checkout@v4
32+
- uses: actions/setup-python@v5
33+
with:
34+
python-version: "3.12"
35+
- run: pip install build
36+
- run: python -m build
37+
- uses: pypa/gh-action-pypi-publish@release/v1
38+
with:
39+
attestations: true
40+
41+
docker:
42+
name: Build & Push Docker
43+
needs: test
44+
runs-on: ubuntu-latest
45+
steps:
46+
- uses: actions/checkout@v4
47+
- uses: docker/setup-buildx-action@v3
48+
- uses: docker/login-action@v3
49+
with:
50+
username: ${{ secrets.DOCKERHUB_USERNAME }}
51+
password: ${{ secrets.DOCKERHUB_TOKEN }}
52+
if: secrets.DOCKERHUB_USERNAME != ''
53+
- uses: docker/build-push-action@v6
54+
with:
55+
context: .
56+
push: ${{ secrets.DOCKERHUB_USERNAME != '' }}
57+
tags: |
58+
lyonzin/knowledge-rag:latest
59+
lyonzin/knowledge-rag:${{ github.ref_name }}
60+
cache-from: type=gha
61+
cache-to: type=gha,mode=max
62+
63+
github-release:
64+
name: GitHub Release
65+
needs: [publish-pypi, docker]
66+
if: always() && needs.publish-pypi.result == 'success'
67+
runs-on: ubuntu-latest
68+
permissions:
69+
contents: write
70+
steps:
71+
- uses: actions/checkout@v4
72+
with:
73+
fetch-depth: 0
74+
- name: Generate changelog
75+
id: changelog
76+
run: |
77+
PREV_TAG=$(git tag --sort=-creatordate | sed -n '2p')
78+
if [ -z "$PREV_TAG" ]; then PREV_TAG=$(git rev-list --max-parents=0 HEAD); fi
79+
echo "## Changes since $PREV_TAG" > /tmp/changelog.md
80+
git log ${PREV_TAG}..HEAD --pretty=format:"- %s" >> /tmp/changelog.md
81+
- uses: softprops/action-gh-release@v2
82+
with:
83+
body_path: /tmp/changelog.md
84+
generate_release_notes: true

‎.github/workflows/security.yml‎

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,35 @@
1+
name: Security
2+
3+
on:
4+
push:
5+
branches: [master]
6+
schedule:
7+
- cron: "0 6 * * 1" # Weekly Monday 6am UTC
8+
9+
permissions:
10+
security-events: write
11+
contents: read
12+
13+
jobs:
14+
codeql:
15+
name: CodeQL Analysis
16+
runs-on: ubuntu-latest
17+
steps:
18+
- uses: actions/checkout@v4
19+
- uses: github/codeql-action/init@v3
20+
with:
21+
languages: python
22+
- uses: github/codeql-action/analyze@v3
23+
24+
dependency-audit:
25+
name: Dependency Audit
26+
runs-on: ubuntu-latest
27+
steps:
28+
- uses: actions/checkout@v4
29+
- uses: actions/setup-python@v5
30+
with:
31+
python-version: "3.12"
32+
- run: |
33+
pip install pip-audit
34+
pip install -r requirements.txt
35+
pip-audit --strict

‎mcp_server/__init__.py‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,6 @@
44
__author__ = "Ailton Rocha (Lyon.)"
55

66
from .config import Config
7-
from .ingestion import DocumentParser, Document
7+
from .ingestion import Document, DocumentParser
88

99
__all__ = ["Config", "DocumentParser", "Document"]

‎mcp_server/config.py‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
"""Configuration for Knowledge RAG System v3.0"""
22

3-
from pathlib import Path
43
from dataclasses import dataclass, field
4+
from pathlib import Path
55
from typing import Dict, List
66

77
BASE_DIR = Path(__file__).parent.parent

‎mcp_server/ingestion.py‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -4,13 +4,13 @@
44
Supports: MD, PDF, TXT, PY, JSON, DOCX, XLSX, PPTX, CSV
55
"""
66

7-
import re
8-
import json
97
import hashlib
10-
from pathlib import Path
8+
import json
9+
import re
1110
from dataclasses import dataclass, field
12-
from typing import List, Dict, Optional, Any
1311
from datetime import datetime
12+
from pathlib import Path
13+
from typing import Any, Dict, List, Optional
1414

1515
# PDF support (optional)
1616
try:

‎mcp_server/server.py‎

Lines changed: 14 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -25,16 +25,15 @@
2525
By Lyon :) Legal Ne?
2626
"""
2727

28-
import json
2928
import hashlib
29+
import json
3030
import re
31-
import time
3231
import threading
33-
import numpy as np
34-
from pathlib import Path
35-
from typing import List, Dict, Optional, Any, Tuple
36-
from datetime import datetime
32+
import time
3733
from collections import OrderedDict
34+
from datetime import datetime
35+
from pathlib import Path
36+
from typing import Any, Dict, List, Optional, Tuple
3837

3938
# ChromaDB
4039
import chromadb
@@ -43,20 +42,19 @@
4342
from fastembed import TextEmbedding
4443
from fastembed.rerank.cross_encoder import TextCrossEncoder
4544

45+
# FastMCP
46+
from mcp.server.fastmcp import FastMCP
47+
4648
# BM25 for keyword search (hybrid search)
4749
from rank_bm25 import BM25Okapi
50+
from watchdog.events import FileSystemEventHandler
4851

4952
# File watcher for auto-reindex
5053
from watchdog.observers import Observer
51-
from watchdog.events import FileSystemEventHandler
52-
53-
# FastMCP
54-
from mcp.server.fastmcp import FastMCP
5554

5655
# Local imports
5756
from .config import config
58-
from .ingestion import DocumentParser, Document, parse_documents
59-
57+
from .ingestion import Document, DocumentParser
6058

6159
# =============================================================================
6260
# QUERY CACHE
@@ -144,7 +142,7 @@ def __init__(self, model: str = None):
144142
self._dim = config.embedding_dim
145143
print(f"[INFO] Loading embedding model: {self.model_name} ({self._dim}D)...")
146144
self._model = TextEmbedding(model_name=self.model_name)
147-
print(f"[INFO] Embedding model loaded successfully")
145+
print("[INFO] Embedding model loaded successfully")
148146

149147
def __call__(self, input: List[str]) -> List[List[float]]:
150148
"""
@@ -206,7 +204,7 @@ def _ensure_model(self):
206204
if self._model is None:
207205
print(f"[INFO] Loading reranker model: {self.model_name}...")
208206
self._model = TextCrossEncoder(model_name=self.model_name)
209-
print(f"[INFO] Reranker model loaded successfully")
207+
print("[INFO] Reranker model loaded successfully")
210208

211209
def rerank(
212210
self,
@@ -466,7 +464,7 @@ def _check_dimension_mismatch(self) -> bool:
466464
error_msg = str(e).lower()
467465
if "dimension" in error_msg:
468466
print(f"[MIGRATION] Embedding dimension mismatch detected: {e}")
469-
print(f"[MIGRATION] Nuclear rebuild required.")
467+
print("[MIGRATION] Nuclear rebuild required.")
470468
return True
471469
# Other error — don't trigger rebuild
472470
print(f"[WARN] Dimension check query failed (non-dimension error): {e}")
@@ -803,7 +801,7 @@ def query(
803801
where_filter = {"category": routed_category}
804802

805803
# Parallel Semantic + BM25 search (threaded for latency reduction)
806-
from concurrent.futures import ThreadPoolExecutor, as_completed
804+
from concurrent.futures import ThreadPoolExecutor
807805

808806
semantic_results = {}
809807
bm25_results = {}
@@ -985,7 +983,6 @@ def _expand_with_adjacent_chunks(self, results: List[Dict], window: int = 1) ->
985983
return results
986984

987985
for result in results:
988-
doc_id_chunk = result.get("content", "")
989986
source = result.get("source", "")
990987
chunk_idx = result.get("chunk_index", 0)
991988

‎pyproject.toml‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -77,3 +77,22 @@ exclude = [
7777
"scripts/",
7878
".gitignore",
7979
]
80+
81+
[tool.pytest.ini_options]
82+
testpaths = ["tests"]
83+
pythonpath = ["."]
84+
85+
[tool.ruff]
86+
target-version = "py311"
87+
line-length = 120
88+
89+
[tool.ruff.lint]
90+
select = ["E", "F", "W", "I"]
91+
ignore = ["E501"] # Line length handled by formatter
92+
93+
[tool.coverage.run]
94+
source = ["mcp_server"]
95+
96+
[tool.coverage.report]
97+
show_missing = true
98+
fail_under = 70

0 commit comments

Comments
 (0)