CORE GUIDE
Capability is not a guarantee
A model demonstrating a capability in examples or benchmarks does not guarantee that the behavior will occur for every production input; reliability requires contracts, evidence and controls around probabilistic capability.
Mental model
Capability asks whether a model can perform a behavior under some conditions. Guarantee asks what the surrounding system can promise across a defined distribution, failure budget and consequence level. The gap between them is where production engineering begins.
Why it matters
AI demos often turn one successful output into an implicit promise. That creates fragile products because model behavior varies with inputs, context and runtime conditions. Separating capability from guarantee keeps teams from using model confidence as an SLA and directs them toward evaluation, fallbacks, human review and deterministic enforcement.
01
Convert capability into bounded system claims
Define the behavior, input distribution and consequence you care about. Measure performance across representative and adversarial cases, identify unacceptable failures, then add validation, fallback, approval or hard constraints until the system-level claim is defensible. Phrase promises in terms the system can observe and enforce, not in terms of what the model usually seems able to do.
02
Example: 'the model can cite sources' becomes a product guarantee
A model often produces citations correctly in demos, so a product promises every answer is source-backed. In production, some citations are missing or irrelevant. The guarantee becomes defensible only after the application requires structured citations, validates source existence and support, and blocks or labels answers when evidence is insufficient.
Common failure modes
- Turning a successful demo into a universal reliability promise.
- Using benchmark averages as guarantees for high-consequence edge cases.
- Assuming model self-confidence can replace external validation.
Engineering heuristics
- State the input distribution and failure budget behind every strong product claim.
- Use evaluation to measure capability and runtime controls to bound guarantees.
- Reserve hard promises for behavior the surrounding system can verify or enforce.
Takeaways
- 01Capability is probabilistic evidence, not a contract.
- 02Guarantees belong to the whole system, not the model alone.
- 03Production engineering is the work of bounding the gap between the two.
Reading evidence
This records actions you actually took; it does not claim mastery, proficiency, or certification.
Used in
This Concept is reused across these canonical learning paths.
Related concepts from the Knowledge Graph
These relationships come from the canonical graph, not a separate Guide taxonomy.