BrighTO_Router

BrighTO-Router — Rust LLM Gateway, Model Load Balancer, and SystemOne Router

Million-token AI traffic, simple Rust fast path, one Docker install.

BrighTO-Router, also searchable as Brighto LLM Router, is a free open-source, self-hosted LLM gateway, AI router, and SystemOne decision router written in Rust. It gives teams one stable API endpoint for OpenAI-compatible, Anthropic-compatible, cloud, local, Ollaya/Laya, Quyết-compatible, and JEV/DJEV-style System One backends with model load balancing, fallback routing, team API keys, token budgets, usage analytics, and privacy-first logging. A single-file Python SDK covers every public inference API.

Why this exists

Most teams do not need a large hosted AI platform to start. They need a fast, understandable LLM API proxy that they can run, audit, and maintain. BrighTO-Router focuses on the traffic path: authenticate, enforce budget, choose model route or Model Group, forward the request, stream the response, and record usage metadata.

Core features

• OpenAI-compatible /v1/chat/completions, /v1/completions, /v1/responses, /v1/embeddings
• Anthropic-compatible /v1/messages
• Claude CLI bridge in 1.3.0: an OpenAI Chat route can also serve /v1/messages, so Claude Code can use an OpenAI-compatible model while the same public model keeps serving /v1/chat/completions. Qwen 3.8 local is validated through both protocols.
• Agent client setup for Claude Code, Codex, Hermes, and OpenClaw, using matching protocols or the explicit Claude CLI bridge, with tool-capable backends
• System One decisions through /v1/systemone and /v1/decisions
• Single-file Python SDK for every public inference API, including Quyết-compatible System One decisions
• Rerank and ASR adapter routes
• Round-robin and weighted Model Groups for load balancing
• Team API keys, expiry, token budgets, RPM limits, and concurrency limits
• PostgreSQL usage ledger and Portal dashboard
• Hardened admin auth (constant-time key check, per-IP rate limiting) and Portal security headers since 1.2.0
• No prompt or response content stored by default
• One-line Docker install with amd64 and arm64 images; Linux servers, Windows through Docker Desktop/WSL2, and macOS through Docker Desktop

Quick start

curl -fsSL https://raw.githubusercontent.com/thusinh1969/BrighTO_Router/main/install.sh | bash

Then open:

http://127.0.0.1:18080/

Architecture

• Architecture diagram — Portal, Admin API, PostgreSQL snapshot, Rust hot path, adapters, Model Groups, SystemOne, ledger, and release gates.
• PostgreSQL data flow — durable control plane, Model Group counters, usage ledger, and JSONL fallback.
• Migration 0008 — required additive database migration for the Claude CLI bridge, with backup, rolling-upgrade, rollback, and test coverage.
• Python SDK and API examples — one-file client for chat, streaming, Responses, embeddings, rerank, ASR, System One, and Anthropic Messages, plus a Quyết backend example.
• Load-balancing test notes — round-robin and weighted Model Group validation.

Repository and Docker image

• GitHub: thusinh1969/BrighTO_Router
• Docker: thusinh1969/brighto_airouter:v1.3.0 for linux/amd64 and linux/arm64
• Release: v1.3.0

Benchmarks

BrighTO-Router publishes deterministic mock-backend benchmarks to measure router overhead, not model inference speed. The 1M-token and Model Group artifacts are in the repository under benchmarks/artifacts/.

Keywords

LLM gateway, LLM router, AI gateway, AI router, OpenAI API proxy, Anthropic API router, Claude Code gateway, Claude CLI bridge, Qwen3.8 Anthropic gateway, local Qwen gateway, Anthropic Messages proxy, self-hosted LLM proxy, model router, model load balancer, SystemOne router, System One decisions, JEV router, DJEV router, Ollaya router, Laya model, Quyết decision model, Quyet-1.0-Small, Python SDK, fallback routing, token budget, cost reduction, LiteLLM alternative, Bifrost alternative, Rust API gateway.

This site is open source. Improve this page.