Vibe Coding × Evaluation·Applied Mission·14 min

Would You Merge This AI-Generated PR?

Your coding agent changed 17 files. CI is green. The diff looks plausible. One architectural boundary is still broken. Decide whether the evidence is strong enough to merge.

MISSION · AI-NATIVE SOFTWARE ENGINEERING

Green tests are not a merge decision.

You are the engineer responsible for the merge.

An agent implemented a cache optimization and added a dependency. Existing tests pass, but the generated change bypasses a transaction boundary and touches unrelated files. The PR is fast to accept and expensive to misunderstand.

Objective

Reduce regression and architecture risk while keeping review effort bounded.

Stakes

If the hidden boundary violation ships, production can serve stale state after a successful transaction.

In one sentence

AI-generated code should be judged against an explicit change contract, repository architecture, targeted tests, and dependency evidence—not against how convincing the diff looks.

Loading Mission Engine…

Key takeaways

Spec before generation

A stable target makes it possible to tell whether the agent solved the requested problem or merely produced plausible code.

Tests are scoped evidence

Green CI only proves what the current test suite checks.

Review the boundary

Most expensive regressions occur where generated changes violate architecture, data or operational contracts.

ENGINEERING DEBRIEF

The review target is the system contract, not the generated patch.

Passing tests are evidence, not proof that generated code belongs in the repository.

AI coding compresses implementation time and expands verification responsibility. A strong workflow bounds the requested change, checks architectural invariants, creates tests for the new behavior and likely failure modes, and reviews dependency movement. This turns AI code review from aesthetic approval into an evidence process.

✓Write the change contract before judging the generated implementation.
✓Review repository-level invariants that line-by-line diff inspection can miss.
✓Generate tests from risks, not only from the happy-path specification.
✓Treat new dependencies as part of the architecture decision.

LEARNING CONTEXT

Reusable mental model

Passing tests are evidence, not proof that generated code belongs in the repository.

Not seen

CONCEPTS PRACTICED

concept-specification-before-generationSpecification before generation
concept-ai-code-reviewAI code review
concept-test-first-aiTests as executable acceptance evidence
concept-evaluation-evidenceEvaluation environment and verifier design

Suggested backfill

No shipped prerequisite is required before entering this incident.

Transfer the model

Move from generated code to generated claims: build a research workflow that can show why each important statement should be believed.

View full learning path

Next: Research Evidence Mission

Move from generated code to generated claims: build a research workflow that can show why each important statement should be believed.

Open research mission