Route from evidence
Use representative evaluation results, operational constraints, and real workload data. A router should encode a decision already supported by evidence—not hide an unsupported preference.
Every route
needs a reason.
An effective router is a visible operational policy: it explains what matched, why a model was selected, what it may cost, and what happens when it fails.
OPEN ROUTER WORKSPACE ↗Keep rules inspectable
Rules should be ordered, narrow, and readable. Task type, request size, explicit keywords, budget, and required capability are understandable signals; opaque routing should be treated as a separate learned system.
Prefer the smallest sufficient model
A cheaper route is valuable only when it satisfies the task. Establish a quality floor with evaluations, then use cost and latency to choose among models that clear it.
Fail deliberately
Fallbacks are for execution failures and capacity problems, not silent quality repair. Keep the chain short, avoid circular routes, apply timeouts, and record every attempt.
Guard before execution
Estimate cost before dispatch and enforce a hard maximum. Actual cost can differ because providers tokenize and bill differently, so monitor real usage as well as estimates.
Observe and revise
Record the selected rule, model, latency, tokens, cost, fallback attempts, and outcome. Re-run evaluation suites when models, prices, prompts, or provider behavior change.
PRODUCTION CHECKLIST
Before a policy goes live.
- Representative evaluation evidence exists for every routed task class.
- Every rule has an owner, reason, order, and bounded match condition.
- Cost ceilings, timeouts, provider limits, and maximum output are enforced.
- Fallbacks use compatible capabilities and never form circular chains.
- Decision traces exclude secrets and sensitive prompt content.
- Routing quality, spend, latency, and failure rate are reviewed continuously.