How to Build a Self-Hosted LLM Firewall with Resk: A Practical Tutorial

How to Build a Self-Hosted LLM Firewall with Resk TL;DR Resk is a deployable full-stack LLM firewall (FastAPI + React) that sits in front of any OpenAI-compatible provider. It adds RBAC with a 64-bit capability bitmask per role, editable filtering policies, and logits-level filtering via resklogits . This tutorial shows you how to set it up and why you need it. The Problem: Why Ordinary Defenses Fail When you expose an LLM endpoint to users, you face a unique set of risks. Prompt injection can trick the model into ignoring your system instructions. Data leakage can occur if the model outputs sensitive information. And without fine-grained access control, any user with a valid API key can call expensive models or use tools they shouldn't. Traditional defenses like input sanitization or output filtering are often too coarse. They either block too much (breaking legitimate use) or too little (letting attacks through). Moreover, they don't address the core issue: you need a policy layer that understands both the application context and the LLM's behavior. Before — The Vulnerable Way Consider a simple FastAPI endpoint that proxies requests to OpenAI. Without a firewall, your code might look like this: from fastapi import FastAPI, Request import httpx app = FastAPI() @app.post("/v1/chat/completions") async def chat(request: Request): body = await request.json() # Directly forward to OpenAI async with httpx.AsyncClient() as client: resp = await client.post( " https://api.openai.com/v1/chat/completions ", json=body, headers={"Authorization": "Bearer YOUR_API_KEY"} ) return resp.json() This is vulnerable because: No authentication: anyone can call your endpoint. No policy enforcement: users can request any model, use any tool, and send any prompt. No filtering: malicious or banned content passes through unchanged. No logging: you have no audit trail of what was sent or received. After — The Resk Way Resk provides a complete solution. Here's how you'd set it up and use it. Quick Start Clone the repository and run the one-command launcher: ./start.sh This creates a Python venv, installs dependencies, seeds the SQLite DB with a default admin ( admin / changeme ), starts the backend on :8000 , and the frontend on :5173 . Manual Backend Setup If you prefer manual steps: cd backend python -m venv .venv && . .venv/bin/activate pip install -e . cp .env.example .env uvicorn resk_app.main:app --reload --port 8000 Frontend Setup cd frontend bun install bun run dev Using the Firewall Endpoint Once running, you can send requests to the firewall endpoint. First, obtain a JWT by logging in: JWT=$(curl -s -X POST http://localhost:8000/api/auth/login \ -H 'Content-Type: application/json' \ -d '{"username":"admin","password":"changeme"}' \ -c /tmp/resk_cookies.txt | jq -r '.access_token') Then call the firewall: curl -X POST http://localhost:8000/v1/chat/completions \ -H "Authorization: Bearer$JWT" \ -H "Content-Type: application/json" \ -H "X-Provider-Id: " \ # optional -d '{ "model": "gpt-4o-mini", "messages": [{"role": "user", "content": "Hello"}] }' The JWT carries the user's roles and capabilities_mask. Resk applies: Tool gating (bit 0): blocks tool calls if the user's mask doesn't allow it. Policy compilation: banned phrases are converted into token bans via resklogits (or naive post-filtering for distant backends). Provider routing: if X-Provider-Id is absent, it falls back to LLM_BACKEND_URL and LLM_BACKEND_API_KEY env vars. What Changed Authentication : The request now requires a Bearer JWT, obtained via /api/auth/login . This ensures only authorized users can access the firewall. RBAC : The JWT contains the user's roles and capabilities_mask. Resk checks bit 0 (can_call_tools) before allowing tool calls, returning 403 if not set. Policy enforcement : Resk compiles policies into banned phrases and token biases. If resklogits is installed, it uses a ShadowBanProcessor for logits-level filtering; otherwise, it falls back to naive substring post-filtering. Provider routing : You can specify a provider via the X-Provider-Id header, allowing you to route different requests to different backends (OpenAI, vLLM, Ollama, custom). Logging : Every request is logged in the RequestLog , giving you full auditability. Honest Limitations Setup complexity : While start.sh simplifies things, production deployment requires PostgreSQL and careful configuration of environment variables like JWT_SECRET_KEY and PROVIDER_ENCRYPTION_KEY . Performance overhead : Logits-level filtering adds latency. The naive post-filtering is faster but less precise. Not a silver bullet : Resk cannot prevent all prompt injection attacks; it's a defense layer, not a complete solution. Dependency on resklogits : For best results, you need to install resklogits separately. Without it, the firewall uses simpler filtering. Conclusion Resk gives you a self-hosted, full-stack LLM firewall that you can deploy in front of any OpenAI-compatible provider. It adds essential security controls: RBAC, policy enforcement, and multi-provider routing, all with an admin console for management. Ready to secure your LLM endpoints? Check out Resk for enterprise AI security tools, and the GitHub repository for the open-source code. Start building safer AI applications today.