The latest distribution models enable high-quality video production, but suffer from slow runtimes. The large transformer-based backbones used in these models are constrained by spatiotemporal attention. In this paper, we identify that a significant portion of the token-to-token connections always yields null points across the various inputs, and their patterns tend to repeat across queries. Therefore, the calculation of attention in these cases can be skipped with little effect on the result. This view continues to be held for linking between local token blocks. Inspired by this, we present CalibAtt, a free training method that accelerates video production with minimal attention. CalibAtt performs an offline calibration pass that identifies block-level minimum and stable repetition patterns across the input, and combines these patterns into optimized attention functions for each layer, head, and propagation time step. During decision-making, we enumerate the selected connections depending on the density input, and skip the unselected ones in a hardware-efficient manner. Extensive tests on Wan 2.1 14B, Mochi 1, and multi-step fused models at various resolutions show that CalibAtt achieves 1.58× end-to-end speedup, outperforming existing untrained methods while maintaining video production quality and text-to-video alignment.
- † Tel Aviv University
- ** Work done while at Apple
