Developed by Mutasem Kharma (معتصم خرما)

🛡️ Enterprise LLM Firewall

Zero-Trust AI Gateway & DLP Proxy for Enterprise Security

📖 Overview

The Enterprise LLM Firewall is a robust, low-latency API proxy designed to sit between internal corporate applications and GenAI providers (OpenAI, Ollama, Anthropic). It acts as a Zero-Trust shield, ensuring that sensitive data (PII) never leaves the corporate perimeter, and preventing malicious prompt injections using local Machine Learning algorithms.

✨ Core Features

• 🔒 Data Loss Prevention (DLP): Intercepts streaming and synchronous requests to dynamically tokenize PII (Emails, Credit Cards) before hitting external LLMs.
• ⚡ Streaming SSE Vault: Real-time sliding-buffer tokenization that securely swaps sensitive tokens ([EMAIL_1]) back into plain text with 0ms latency continuously during Server-Sent Events.
• 🧠 AI Semantic Guardrails: Bypasses legacy regex constraints by utilizing local HuggingFace text-classification NLP models to evaluate the intent of requests and detect Zero-Day Prompt Injections.
• 🏢 Multi-Tenancy: Secure mapping of API Keys to Tenants, generating persistent SecurityPolicy and AuditLog rules in PostgreSQL.
• 🚀 Redis High-Performance Caching: Policies are cached via Redis async configurations to guarantee ultra-fast route processing.
• 📊 CISO Command Center Dashboard: A React/Tailwind-inspired frontend natively served via FastAPI for real-time traffic monitoring and single-click policy adjustments.

🏗️ Architecture Design

graph TD
    Client((Corporate App)) <-->|Streaming / Sync| FW[LLM Firewall Gateway]
    FW -->|1. JWT Check| Auth(API Key / Auth)
    FW -->|2. Redis Cache| RC[(Redis Policy)]
    
    FW -->|3. DLP Vault| Extract[Presidio / Regex Masking]
    FW -->|4. AI Guards| Guard[HuggingFace Semantic Guard]
    
    Extract --> Proxy[Intelligent Proxy]
    Guard --> Proxy
    
    Proxy <-->|Sanitized Request| OpenAI(OpenAI API)
    Proxy <-->|Local Airgapped| Ollama(Local Ollama)
    
    FW -.->|Audit Logs| DB[(PostgreSQL)]
    
    CISO((CISO / Security)) -->|Login over JWT| Dash[Web Dashboard]
    Dash -.-> DB
    Dash -.-> RC

Loading

🚀 Quick Start

1. Docker Compose (Enterprise Production)

The quickest way to launch the full stack (API, PostgreSQL, Redis) securely:

docker-compose up --build -d

The API will be available at http://localhost:8000/docs.

2. Local Development (Uvicorn)

# 1. Activate Environment
python -m venv venv
venv\Scripts\activate   # Windows

# 2. Install Dependencies
pip install -r requirements.txt

# 3. Setup SQLite MVP & Inject API Key
python bootstrap.py

# 4. Start Server
uvicorn app.main:app --reload --port 8000

🛡️ CISO Dashboard

Head over to the Visual Command Center to manage your live policies:

• URL: http://localhost:8000/dashboard
• Login: admin / password123

📁 Repository Structure

llm_firewall/
├── app/
│   ├── api/             # FastAPI Routes & Admin Endpoints
│   ├── core/            # Config, JWT Security
│   ├── db/              # SQLAlchemy Async Models (PostgreSQL)
│   ├── engine/          
│   │   ├── anonymizer/  # PII Vault Strategy
│   │   ├── detectors/   # HuggingFace Semantic Guards, Presidio, Regex
│   │   ├── providers/   # OpenAI, Ollama Integrations
│   │   └── streaming.py # Sliding-Window Stream Buffer
│   ├── frontend/        # Splendid HTML/Tailwind CISO Dashboard
│   ├── schemas/         # Pydantic Schemas
│   └── services/        # Orchestrator & Redis Cache logic
├── alembic_migrations/  # Database State Migrations
├── tests/               # Client streaming test scripts
├── docker-compose.yml   # Infrastructure Orchestration
└── Dockerfile           # Optimized Python 3.10 deployment

🔐 Disclaimer

This software is intended for Enterprise network boundaries. Always ensure local NLP models and Redis caches are properly air-gapped from public networks.

---

Developed by Mutasem Kharma (معتصم خرما) — GitHub | Portfolio | Twitter/X