HoloCoT: Holographic Chain-of-Thought via Orthogonal Subspace Anchoring for Persistent Long-Horizon Reasoning

Abstract

Long-horizon reasoning in large language models (LLMs) suffers from plan drift: the gradual divergence of a reasoning trace from the high-level intent established by the initial prompt, caused by the autoregressive objective’s lack of any penalty for discarding global constraints. We propose HoloCoT (Holographic Chain-of-Thought), a training framework that enforces persistent plan awareness by requiring every intermediate hidden state to maintain a sufficient statistic of the overarching plan. The key contributions are: (1) an orthogonal plan-tracking subspace via a learned projection W_proj ∈ ℝ^(d×k), isolated from the linguistic stream by an orthogonality regulariser; (2) a temporally modulated InfoNCE contrastive loss that respects early-step causal uncertainty whilst enforcing strict plan fidelity as the trace solidifies; and (3) a unified objective incurring zero inference overhead. Across five benchmarks spanning multi-step arithmetic, competition and Olympiad mathematics, and embodied planning, HoloCoT applied to models from 7B to 72B parameters consistently outperforms Chain-of-Thought, Plan-and-Solve, and Self-Consistency baselines, yielding +7.4% average accuracy and reducing our novel Plan Drift metric by 63%. Formal analysis establishes convergence and an information-preservation bound.