Can a large-scale linguistic model (LLM) improve on code generation using only its raw outputs, without verifier, teacher model, or reinforcement learning? We answer by confirming that it is a simple self-distillation (SSD): sample solutions from a model with a certain temperature and reduction setting, then fine-tune those samples with regular supervised fine-tuning. SSD improves Qwen3-30B-Instruct from 42.4% to 55.3% pass@1 in LiveCodeBench v6, with benefits focused on difficult problems, and covers all Qwen and Llama models in 4B, 8B, and 30B scales, including both instructional and reasoning variants. To understand why such a simple approach can work, we trace these benefits to precision testing conflicts in LLM coding and show that SSD reshapes the distribution of tokens in a context-dependent manner, suppressing bug tails where precision is important while preserving useful diversity where testing is important. Together, SSD offers a comprehensive post-training guide to improve LLM code generation.
