Start with the decision
Define what the evaluation will decide: a production model, a routing tier, a fallback, or a prompt revision. A score without a decision context is noise.
Evidence needs
a method.
The studio helps teams run controlled qualification tests. It does not turn a tiny dataset or a lexical match into scientific proof.
OPEN EVALUATION STUDIO ↗Use representative cases
Choose real task shapes, edge cases, and failure modes. Three to six cases are useful for a fast qualification pass, but they are not a substitute for a statistically meaningful production dataset.
Control the run
Every model receives the same system instruction, case prompt, temperature, and output limit. AXON//RADAR uses temperature zero for evaluations to reduce avoidable variation.
Separate checks from judgment
Expected-term coverage is a transparent lexical check—not a claim of semantic correctness. Human scores should assess factuality, relevance, instruction-following, clarity, and unacceptable failure modes.
Measure operational fit
Quality is only part of deployment fitness. Review completion rate, latency, token volume, estimated cost, access conditions, privacy requirements, and the consequences of provider failure.
Reproduce before committing
Provider routing, model versions, prices, and inference behavior can change. Export the record, repeat important runs, expand the dataset, and validate the winning configuration in the intended production environment.
RECOMMENDED HUMAN RUBRICScore the answer,
Score the answer,
not the model’s reputation.
- 1 — Unusable
Wrong, unsafe, irrelevant, or materially violates the instruction.
- 2 — Major revision
Some value, but important errors or omissions prevent use.
- 3 — Acceptable
Mostly correct and useful with ordinary editing or verification.
- 4 — Strong
Correct, relevant, clear, and needs only minor refinement.
- 5 — Excellent
Complete, precise, production-ready for the defined task.