LLM Router
Decides per question whether a cheap model will do. When it says escalate it makes you choose, so you can see whether the pricier answer was actually better.
What It Does
We don't need Fable for everything do we? This is an experiment with different levels of LLM and ability to rank with confidence the difficulty of a query and judge which LLM can answer it accurately. This was inspired by the GiveDirectly's experiment in Rwanda giving unrestricted LLM access to villagers. I was wondering for them and for most people who use LLMs in fact what model would be able to reliably answer 80% of queries and would a router be able to escalate intelligently when needed.
How it works
Every message you send goes first to a cheap model whose only job is to size up the question as easy/medium/hard, what topic, and whether it's worth spending real money on. If it says cheap, you just get the cheap model's answer immediately; if it says escalate, it stops and hands you two buttons so you can decide whether to pay for the stronger model and see for yourself if the better answer was worth it. Conversation history lives in SQL so the thread survives reload.
Built with
- Built with: Python/FastAPI backend around a single model registry — DeepSeek V4 Flash both routes (classifying each request as answer-now or escalate) and serves the cheap tier, escalating on request to DeepSeek V4 Pro, Claude Haiku 4.5, Sonnet 5, or Opus 5; every model is declared once in config.MODELS with its own pricing and token budget and called through one complete() interface, with per-call latency and USD cost logged next to the conversation in SQLite. Served to a single vanilla-JS page (no framework, no build step) by uvicorn bound to the LAN so it's usable from a phone.