CORE GUIDE
Confidence, variance and sample-size humility
AI evaluation results are samples from variable systems, so confidence requires repeated observations, uncertainty estimates and enough data for the decision.
Mental model
A measured success rate is an estimate, not a property carved into the model. Its usefulness depends on sample size, trial variance, task mix and whether the observed slice represents the production decision you care about.
Why it matters
Small eval sets can produce dramatic percentage swings from only a few examples. Stochastic generation adds another source of variance, and aggregate scores can hide unstable subgroups. Without statistical humility, teams may ship regressions because a noisy improvement looks real, or block useful changes because a tiny slice happened to fail.
01
Measure the uncertainty around the metric
Run enough tasks and repeated trials to characterize variability that matters to the product. Report counts and intervals or other uncertainty summaries alongside point estimates. Break results into predefined slices, and compare candidates under matched conditions. For high-impact release decisions, require enough evidence that the decision would not reverse under plausible sampling noise.
02
Example: 90% versus 94% on twenty tasks
A new model passes 19 of 20 tasks while the baseline passes 18. Calling that a five-point improvement exaggerates what the sample can support: one additional failure reverses much of the story. A larger matched suite and repeated trials may reveal whether the improvement is stable or just noise.
Common failure modes
- Reporting percentages without the underlying numerator and denominator.
- Treating one model run per task as if stochastic variance did not exist.
- Using aggregate improvements to hide regressions in important slices.
Engineering heuristics
- Show raw counts next to rates and scores.
- Increase sample size around decisions with costly false positives or false negatives.
- Predefine critical slices and examine their variance separately.
Takeaways
- 01Point estimates need uncertainty context.
- 02Small samples invite overconfident release decisions.
- 03Variance is part of system behavior, not an inconvenience to average away.
Related concepts from the Knowledge Graph
These relationships come from the canonical graph, not a separate Guide taxonomy.