Synthetic data and distillation

Synthetic data can scale examples and transfer behavior, but it also transfers the generator's blind spots unless quality and diversity are controlled.

Mental model

Synthetic data is generated training evidence, not free ground truth. Distillation can compress useful behavior from a stronger teacher into a target model, but the resulting dataset inherits the teacher, prompt, sampling and filtering policy.

Why it matters

Teams use synthetic examples to cover rare cases, expand instruction data and reduce labeling cost. The danger is feedback loops: a teacher systematically omits difficult cases, generated examples become stylistically homogeneous, or automatic filters reward the same biases used to create the data. Training on more synthetic rows can then make the target model confidently narrower rather than genuinely better.

Generate broadly, filter independently, validate on real data

Define the capability or behavior to transfer, then generate examples across explicit difficulty and domain slices rather than one prompt distribution. Attach generation metadata, apply deterministic checks where possible, and use independent reviewers or models for quality filtering. Most importantly, hold out real or independently curated evaluation data so the synthetic generator is not grading its own worldview.

Example: synthetic tool-call training data

A teacher model produces thousands of tool-use conversations. Instead of training on all of them, the pipeline checks schema validity, balances tool and error categories, removes near duplicates and sends uncertain edge cases to review. The final adapter is evaluated on real production-like tool tasks that were never generated by the teacher.

Common failure modes

Treating generated labels as ground truth because they are cheap and plentiful.
Using the same model and rubric to generate, filter and evaluate the data.
Scaling dataset size without measuring diversity, duplication or difficult slices.

Engineering heuristics

Track generator, prompt, seed and filter provenance for synthetic examples.
Balance coverage intentionally instead of sampling only easy high-confidence cases.
Keep an independent real-data or human-curated evaluation set.

Takeaways

01Synthetic data scales supervision, not truth.
02Distillation transfers both strengths and systematic errors.
03Independent evaluation is the main defense against synthetic feedback loops.