Send each request to the cheapest model that can handle it
Route requests between LLMs with a 0.8B decision model instead of a prompt or a rule. Saina Helm 2 Routing scores your candidate models from the request and each candidate’s description and price, so the expensive model only runs when it is needed. Open weights, experimental.
In short: Describe each candidate model in plain words with its price, send the request, and Helm 2 Routing returns a probability per candidate. Route on the top score, fall back to your default when it is unsure, and measure the whole thing against a fixed cheap model before you switch. The router is open weights and experimental; the benchmark harness is public.
Saina Helm 2 Routing
Text
Gateways, agents and pipelines that run more than one model and pay for the wrong one too often
The step
“Routing only earns its place if its decisions beat the cheapest acceptable fixed model on the real workload, including the router’s own overhead and its mistakes. Show me that number or I’ll pin one model and move on.”
What people running LLM gateways say, and they are rightMost teams running several models route with one of two things.
- Rules. Token count, a keyword, which endpoint the request came from. Cheap and predictable; wrong whenever difficulty doesn’t follow length.
- A classifier prompt on a chat model. Ask a small LLM “is this easy or hard?”. Now there is a prompt to maintain, a text answer to parse, latency on every request, and no measure of how sure the classifier was.
A router is a decision between options you know in advance, scored from the request. That is a decision model’s shape, and a 0.8B one adds tens of milliseconds, not seconds. The honest part is the measurement: a router has to beat “always use the cheap model” and “always use the good one” on quality, total cost and failures before it is worth running.
What the request looks like
{
"model": "saina-helm-2-routing-0.8b",
"state": "Write a short thank-you email to a customer after today's
onboarding call. Keep it to four sentences.",
"mode": "distribution",
"questions": {
"route": {
"type": "single_choice",
"question": "Which candidate should answer this request, at the lowest cost that will do the job well?",
"options": {
"economy": "Text model for short everyday writing and simple questions. About $0.0001 per request.",
"standard": "General model for most tasks, moderate reasoning. About $0.0005 per request.",
"advanced": "Frontier model for complex reasoning, long documents, demanding code. About $0.004 per request."
}
}
}
}And what comes back:
{
"model": "saina-helm-2-routing-0.8b",
"answers": {
"route": {
"type": "single_choice",
"selection": "economy",
"probabilities": {
"economy": 0.79,
"standard": 0.18,
"advanced": 0.03
},
"confidence": 0.69
}
}
}The candidates are plain-language descriptions with a price, not model names, so the router never has to know your vendors. Route on the top score; when the top two are close, route to your safe default rather than the cheap one. The probabilities describe the routing decision, not the quality of the downstream answer, which is why the next section is about measuring.
Putting it in place
Benchmark it against two fixed arms
Take a sample of real requests with a quality judgement (a rubric, a reviewer, a test). Run three arms: always-cheap, always-good, and routed. For each, record quality, total cost including the router’s own call, p50 and p95 latency, route share, and failures. Compare with confidence intervals; if the routed arm doesn’t beat always-cheap on quality at lower cost than always-good, pin a model and move on. We publish misses as well as wins; if you run this on a public harness, we’d like to see the numbers either way.
Write candidates the way you’d brief a person
“Frontier model for complex reasoning, long documents, demanding code. About $0.004 per request.” beats a model name. Keep prices in the description: the question asks for the lowest cost that will do the job, and the model reads the price as part of the option.
Wire it in
Helm 2 Routing is text-only and runs where you run it: pip install 'saina[local,server]' and load the weights (Apache-2.0, pinned to a revision). Call /v1/ask on your own server from the gateway or the agent loop before the request goes to a provider; the answer is the candidate ID. The request format is the same as every other Saina model, so a decision model integration you already have works unmodified. In n8n, the Saina Helm node’s “Choose an LLM tier for each request” template does this with Helm 2 and the same question; swap the model name once you self-host Routing.
What it won’t do
- Experimental. No benchmark results are published yet for this model. Measure on your workload before relying on it; the harness is the measurement, the router is the hypothesis.
- Text only, and not on the hosted API today: self-host it.
- It routes; it does not check the answer that comes back. Put a rubric check (Helm 2, yes/no) after the expensive model if you need that too.
Frequently asked questions
How is this different from a rules-based or embedding router?
Rules and embedding similarity route on surface features of the request. Helm 2 Routing reads the request and the candidates’ descriptions and prices together and scores each candidate, with a probability you can set a threshold on. Whether that beats your current router on your workload is a measurement, not a promise.
Do I have to name the models?
No. Each candidate is an ID with a plain-language description, including its price. The router never sees a vendor name unless you put one in.
What does it cost to run?
It is a 0.8B open-weights model you self-host; a routing call is one scoring pass on a CPU or a small GPU, tens of milliseconds. Count it in the total cost of the routed arm when you benchmark.
Is it on the hosted API?
Not yet. Download the weights and run it with the saina package; the request format is the same as the hosted /v1/ask.