Chain-of-thought (CoT) information is a common de-facto method of eliciting responses such as reasoning from large linguistic models (LLMs), allowing them to state individual steps before providing a final response. Although the similarity to human thinking is undeniable, the driving force behind the success of CoT thinking remains unclear. In this work, we perform an in-depth analysis of CoT sequences from competitive level statistical questions, with the aim of better understanding how, and which components of CoT actually contribute to the final answer. In this case, we introduce the concept of power, estimating how much a particular part of the CoT increases the probability of correct completion. When we examine the traces of thinking through the lens of power, we identify surprising patterns including (1) non-monotonicity that is often strong (due to thinking), (2) very sharp but sometimes difficult to explain spikes (details of thinking and jumping) and (3) random guesses at times, when the model arrives at any correct answer without giving. While some of the behaviors of energy are easily interpreted and aligned with human understanding (such as details and tangents), others remain difficult to understand from a human perspective. In order to further measure the reliance of LLMs on reasoning information, we investigate the concept of CoT transfer, where we measure the strength of a weak model under a partial CoT from another, stronger model. Indeed in line with our previous results, we find that as little as 20% of partial CoT can “open” the performance of a weak model in previously unsolvable problems, highlighting that a large part of the mechanics supporting CoT is transferable.
