Model Uncertainty Scores: What They Are and How to Use Them

"Confidence" and "uncertainty" get used interchangeably, and the sloppiness leads to the wrong gates in workflows. A model uncertainty score is any of several signals you can compute from a model's output, and each one catches a different failure mode. This guide maps the signals to the failures they catch. (For where scores come from in the first place, see the AI confidence scores guide.)

Four model uncertainty score signals

The first three are standard functions of the output distribution p₁ ≥ p₂ ≥ … ≥ pₙ over n options. In the literature they're called least confidence, margin and entropy.

1. Top-option probability (least confidence)

The model's probability for its own best answer, p₁ (0.94 for billing). The uncertainty form is least confidence, 1 − p₁. Low p₁ = the model doesn't strongly favor anything.

• Catches: broad indecision; a case that fits no option well (partially, see the table).
• Misses: two-way races at low thresholds. With many options and a low bar, say threshold 0.45, an answer like billing 0.48 / refunds 0.46 / other 0.06 clears the bar while being close to a coin flip between two options.

2. Margin between the top two

The gap p₁ − p₂. Small margin = a two-way race.

• Catches: ambiguity between adjacent categories (billing vs. refunds), where the choice matters even when the winner's probability is decent.
• Misses: cases where the model is uniformly unenthused but the ranking is stable (top probability low, margin fine; the model is out of its depth, not torn).

3. Entropy

Entropy measures how spread out the whole distribution is: H = −Σᵢ pᵢ log pᵢ. It is 0 when all mass is on one option and reaches its maximum, log n, when the distribution is uniform. Because that maximum grows with the number of options, compare questions using normalized entropy, H / log n, which runs from 0 to 1. High entropy = probability smeared across many options.

• Catches: diffuse confusion; a new topic that maps to nothing ("is this even in my option set?").
• Misses: direction. Entropy says "confused", not "confused between these two". A good tripwire, a poor router.

4. Out-of-distribution inputs

A different animal: not a property of the output distribution but of the input: text unlike anything the model handles well (wrong language, novel product, pasted logs instead of prose). Call it out-of-distribution, or OOD.

• Catches: the failures probabilities under-report. A model can be confidently wrong on an input type it has never seen; all three signals above can look calm.
• How to get it: cheap proxies (language detection, length extremes, character-set oddities, a "does this look like our traffic" classifier) rather than deep OOD methods.

Aleatoric vs. epistemic uncertainty

Two sources of uncertainty behave differently. Aleatoric uncertainty is in the data: the ticket genuinely could be billing or refunds, and more training wouldn't settle it; the fix is sharper option definitions or a human. Epistemic uncertainty is the model's lack of knowledge: inputs unlike its training data. OOD inputs are the epistemic case, and that is why output-based signals often miss them. Output probabilities are better at expressing the aleatoric part; for the epistemic part, add input-side checks (OOD proxies) and outcome monitoring.

Which signal catches which failure

Top probability is only partial for "fits no option" because probabilities are normalized over the options you supplied: if nothing fits, the mass still has to land somewhere, and it can pile onto the least-bad option. Add an explicit "other / none of these" option so the model has somewhere to put it. For two racing options, top probability catches the race only when it pushes p₁ below the threshold.

The last row matters most: no uncertainty signal catches the confident-and-wrong case. Only a labeled eval set and monitoring of downstream outcomes (reopened tickets, reverted actions) touch it. Signals reduce risk; they don't repeal it.

Using them together in a workflow

All three output signals can be computed from any returned distribution. For a decision model, Helm's decision mode applies the first two for you, as threshold (on p₁) and min_margin (on p₁ − p₂). Helm's separate confidence field is a normalized version of p₁: (p₁ − 1/n) / (1 − 1/n) for single choice, and 2·p₁ − 1 for yes/no, which for two options is exactly the margin. A sensible stack:

1. OOD proxy first (cheap, catches the weird): language/length/charset gates.
2. Threshold on top probability (the main bar).
3. Margin on the top two (the adjacency bar), where it can bind; see below.
4. Entropy as a monitored metric, alerting on distribution shifts rather than gating individual items.

How threshold and margin interact

For single-choice and yes/no questions the probabilities sum to 1, so p₁ ≥ t implies p₁ − p₂ ≥ 2t − 1. At Helm's default t = 0.8, every accepted answer already has a gap of at least 0.6, so a min_margin of 0.05 or 0.2 never rejects anything. A margin bar only does work when min_margin > 2t − 1: lower thresholds with many options, e.g. threshold 0.5 with min_margin 0.2. A close race under a high threshold shows up as below_threshold, not below_margin. (Choosing the pair: Choosing a probability threshold.)

Reading the reason codes

Every decision-mode answer carries a reason, checked in this order: tie (top two exactly equal) → below_threshold (p₁ < threshold) → below_margin (p₁ − p₂ < min_margin) → accepted. Ratings ignore min_margin; multi_choice questions only return accepted (at least one label cleared) or below_threshold (none did).

Don't diagnose from the reason alone. At a high threshold, a two-way race is reported as below_threshold, the same reason as "fits nothing". Diagnose from the logged top-two gap: many rejections with a small p₁ − p₂ (and most mass on the same two options) mean those options overlap and need sharper descriptions; rejections with low p₁ and mass spread across options (high normalized entropy) suggest a gap in the option set. If you want the reason itself to separate the two cases, use a lower threshold plus a margin bar so races land in below_margin.

A note on calibration

These signals are probabilities, and the exact numbers are only as trustworthy as the model's calibration, measured on your data rather than assumed (see the pillar guide). A poorly calibrated model's margins are still directionally useful (torn is torn), but the values your thresholds compare against deserve a reliability check before you fix them in place.

Checklist

• Signals chosen per question: threshold at minimum, plus margin where min_margin > 2t − 1.
• Explicit "other / none" option where inputs may fit nothing.
• OOD proxies in front for the weird inputs.
• Full distribution logged per deferral; reasons and top-two gaps reviewed weekly.
• Normalized entropy (or distribution) monitored for drift, not gated per item.
• Nobody expects uncertainty signals to catch confident-and-wrong; the eval set and outcome monitoring own that.

Related posts

• AI confidence scores (pillar)
• Choosing a probability threshold
• Build a human handoff trigger