CORE GUIDE
Bias and fairness
Bias and fairness engineering asks whether an AI system produces systematically different errors, opportunities or burdens across relevant groups and contexts, and whether those differences violate the product's intended policy.
Mental model
Fairness is not one universal metric. It is a product and governance decision about which outcomes matter, which groups or slices are relevant, and which disparities are acceptable under the real decision process.
Why it matters
Aggregate accuracy can hide concentrated harm. A system may perform well overall while one language, accessibility mode or demographic proxy receives more refusals, lower-quality evidence or harsher decisions. Treating fairness as an evaluation problem makes the claim measurable and connects it to human review, auditability and release gates.
01
Define the decision, affected groups and acceptable disparity
Start from the consequential outcome rather than a generic bias score. Identify groups or operational slices that could receive different treatment, choose metrics that reflect the product decision, and inspect both error rates and downstream consequences. Control for sample size and legitimate task differences, document policy choices, and require human or governance review when the system cannot justify a disparity.
02
Example: a screening assistant has equal average accuracy but unequal false negatives
A candidate-screening helper achieves the same overall score before and after an update. Slice analysis shows that applicants with non-standard career histories now receive more false negatives because the model overweights conventional chronology. The team blocks the release, revises features and review policy, and measures the specific disparity rather than celebrating the unchanged average.
Common failure modes
- Declaring a system fair because its aggregate score is high.
- Choosing a fairness metric without connecting it to the actual product consequence.
- Reading large percentage gaps from tiny samples as definitive evidence.
Engineering heuristics
- Define fairness around a concrete decision and harm model.
- Measure important slices with uncertainty and sample-size context.
- Use human review and release vetoes when disparity creates consequential risk.
Takeaways
- 01Fairness is a system-level evaluation claim, not a model adjective.
- 02Aggregate quality can hide concentrated failure.
- 03The right metric follows from the decision and policy being enforced.
Reading evidence
This records actions you actually took; it does not claim mastery, proficiency, or certification.
Used in
This Concept is reused across these canonical learning paths.