HOLAD: Breaking the No-Recovery Bottleneck in Long-Horizon Reasoning

Long-term modeling of Large-scale Language Models (LLMs) remains unstable even when advanced techniques are provided. Examining controlled algorithmic puzzles, we show that although decomposition is essential for stability, excessive decomposition creates a “bottleneck of no return”. We show that this bottleneck becomes critical due to non-uniform error distributions, where the errors are constant in several “hard” irreversible steps. To address this, we propose Lookhead Advanced Atomic Decomposition (LEAD). By combining short-horizon future proofing and integrated release, LEAD provides enough isolation to maintain stability while maintaining enough spatial context to correct errors. This enables the o4-mini model to solve Jumping Checkers up to complexity n = 13, while the extreme decomposition fails to exceed n = 11.