LLM evals with decision models: thresholds, trade-offs and calibration

Turning a model's 0–1 score into a reliable yes/no, and knowing when you can trust that score.

Source: Hamel Husain & Antaripa Saha, “How To Use Decision Models for Evals”, 20 Oct 2026 · X post · full post with interactive charts

1. Summary

• The experiment: 100 human-labelled traces from an apartment-leasing AI agent. Task: flag traces with an error (failed booking, missed hand-off to a human).
• Default 0.50 is arbitrary. A model can rank perfectly and still catch nothing at 0.50 if all its scores sit below that line.
• Set requirements first. For an alerting system they wanted ≥80% precision and ≥80% recall. Only Perplexity Decider, Liquid d1-ultra and OpenAI Decisions met it.
• Cost vs. quality. Perplexity: F1 0.79 at 0.82¢ per 100 runs. OpenAI Decisions: F1 0.87 at 3.72¢, about 4.5× the cost.
• Calibration varies. Jev and d1-ultra were already well calibrated. Platt scaling helped Perplexity, d1 and Clef Flash, but slightly hurt OpenAI.
• None of this is new. It's classic classifier evaluation. Data scientists and ML engineers already have these skills.

2. Why this matters for robotics

• Perception: an object detector outputs confidence. The threshold decides whether the robot "sees" the obstacle. Low threshold means phantom braking. High threshold means collisions.
• Grasp success and anomaly detection: "is this grasp going to fail?" and "is the motor current abnormal?" are binary classifiers with exactly the same precision/recall trade-off.
• Safety stops: safety-critical systems lean hard toward recall (never miss a human), and accept false stops. That's the "cancer screening" end of the trade-off.
• Calibration feeds planning: probabilistic planners and sensor fusion (e.g. Bayesian filters) multiply probabilities together. Badly calibrated scores silently break that maths.
• Evaluating robot policies: as robot foundation models (VLAs) grow, teams use LLM judges and decision models to label rollout success from logs and video. This is the same eval workflow.

3. Deep dive: decision thresholds

A decision model outputs a score s ∈ [0, 1]. The API converts it into a label with a rule: flag as fail if s ≥ t. The default t = 0.50 assumes the scores are calibrated and that false alarms and misses cost the same. Neither is usually true.

The authors' example: a model gives every failure 0.40 and every pass 0.10. It separates the classes perfectly (any threshold between 0.10 and 0.40 is flawless) but at 0.50 it catches nothing. Ranking quality and threshold choice are separate problems.

In real data the two score distributions overlap. Moving the threshold slides a line across both. Try it:

4. Deep dive: precision, recall and minimum bars

Pick the trade-off from the business (or safety) consequences, not from habit:

• Precision-heavy: criminal trials. Don't convict the innocent. In robotics: auto-rejecting parts on a line, where each false reject costs money.
• Recall-heavy: cancer screening. Don't miss cases. In robotics: human detection near an industrial arm.

The key move in the article is to set hard minimums before optimising, e.g. ≥80% on both for an alerting system. Plot precision and recall against threshold. The usable region is where both curves clear the bar. Models with no such region are eliminated, whatever their F1.

5. Deep dive: tuning without fooling yourself

With only 100 labelled examples, picking the threshold that looks best on all of them overfits. The authors use stratified five-fold cross-validation:

1. Split the labelled traces into 5 folds, keeping the failure ratio equal in each (stratified).
2. For each fold, choose the threshold on the other 4 folds, restricted to thresholds meeting both minimums.
3. Measure precision, recall and F1 on the held-out fold.
4. Use the same splits for every model so the comparison is fair.
5. If no threshold qualifies, report it. Don't quietly relax the bar.

Important practical detail: save the scores once and reuse them. Tuning is free after the first API pass. They give a ready-made prompt for a coding agent:

Using my saved model scores and binary labels (1 means failure), tune a decision threshold to maximize F1 while requiring at least 80% precision and 80% recall. Use stratified five-fold cross-validation… Reuse the saved scores without making new model calls.

import numpy as np
from sklearn.model_selection import StratifiedKFold
from sklearn.metrics import precision_score, recall_score, f1_score

def tune(s, y, min_p=.8, min_r=.8, seed=0):
    ts = np.linspace(.01, .99, 99)
    out = []
    for tr, va in StratifiedKFold(5, shuffle=True, random_state=seed).split(s, y):
        ok = [(f1_score(y[tr], s[tr] >= t), t) for t in ts
              if precision_score(y[tr], s[tr] >= t, zero_division=0) >= min_p
              and recall_score(y[tr], s[tr] >= t) >= min_r]
        if not ok:
            out.append(None); continue
        t = max(ok)[1]
        p = s[va] >= t
        out.append((t, precision_score(y[va], p, zero_division=0),
                    recall_score(y[va], p), f1_score(y[va], p)))
    return out

Result in the article: the three candidate models gained a little from tuning. The weaker models (Liquid d1, Clef Flash) gained more but still fell below the minimums. Small precision/recall shifts can still matter a lot in production.

6. Deep dive: performance vs. cost

Use a Pareto frontier: for each quality level, the cheapest model that reaches it. Anything above and to the left of the frontier is dominated. Latency was left out because it depends on things outside the experiment, but in robotics it's often the binding constraint. A 300 ms classifier can't sit inside a 100 Hz control loop.

The authors' heuristic: start with a big model that performs acceptably, then try smaller ones to see if they match it for less. Also, narrower judges (one per failure mode) tend to shrink the gap between models.

7. Deep dive: can you trust the probability?

A score is calibrated if, among cases scored ~0.80, about 80% actually fail. Check with a reliability diagram: bin predictions (~20 traces per bin), plot mean predicted probability vs. observed failure rate.

• Points above the diagonal: model underestimates risk.
• Points below: model overestimates risk.
• A striking example: one Clef Flash bin predicted ~8% failure, but 60% actually failed.

Platt scaling fixes this by fitting a logistic regression on the log-odds of the score: p' = σ(a · logit(s) + b). Clip scores away from 0 and 1 first. Fit it inside cross-validation, and judge it with the Brier score (mean squared error of probabilities, lower is better).

Calibration isn't free improvement. It helped three models, barely changed two that were already good, and slightly hurt OpenAI. Measure it, don't assume it.

Calibration doesn't change ranking, so it won't change the best threshold's precision/recall. It matters when you use the number itself: risk scores, expected-cost decisions, fusion with other probabilities.

8. Playbook

1. Write a clear pass/fail criterion and get it right first (prompt and labels).
2. Label a set of real examples by hand.
3. Set minimum precision and recall from the consequences of each error type. Drop models that can't meet them.
4. Score everything once, save the scores, and tune each model's threshold with stratified CV on shared splits.
5. If you'll use scores as confidence, check calibration and try Platt scaling. Keep it only if the Brier score improves.
6. Compare surviving models on quality vs. cost (and latency). Pick from the Pareto frontier.
7. Iterate. A new prompt means re-tuning the threshold and re-checking calibration.

9. Self-check

10. Glossary & further reading

• Full post with interactive charts and footnotes
• Hamel: Creating an LLM judge that drives business results
• Why binary pass/fail instead of 1–5 scales
• Randy Olson: When to use a decision model instead of an LLM
• scikit-learn: precision, recall, F-measure
• Related: Module 19: the evals field guide: where decision models fit among code graders, AI judges and detectors, and how to check any judge with kappa
• Related: Module 07: success detectors and reward models are classifiers too, so tune and calibrate them before using them as rewards
• Related: Module 04: evaluation that doesn't fool you (day splits, drift cutoffs, paired bootstrap), and the "start big, then go smaller" idea taken all the way to fine-tuning
• Related: Module 03: shared autonomy, where a robot's confidence decides how much it takes over, so calibration matters there too
• Related: Module 02: TensorTonic practice track (has "Precision and Recall at K", mIoU and Type I/II error problems to drill these ideas)
• Gallatin & Albon, Machine Learning with Python Cookbook, 2nd ed.