CLaRa: Integrating Retrieval and Generation with Continuous Latent Consultation
Retrieval-augmented generation (RAG) improves large-scale linguistic models (LLMs) with external knowledge but still suffers from long-term conditions and the integration of retrieval-augmented generation. In this work, we propose CLaRa (Continuous Latent Reasoning), an integrated framework that performs embedding-based compression and joint optimization in a continuous shared environment. In order to obtain semantically rich and retrievable compressed vectors, thus reducing the length of the generated document, we present SCP, a key-preserving data integration framework based on query-answering and word-specific monitoring. CLaRa then trains the reranker and the generator end-to-end with a single language modeling loss, with gradients flowing in both modules using a high-k differential approximation. In theory, this integrated setting balances retrieval consistency and response quality. Tests across multiple QA benchmarks show that CLaRa achieves state-of-the-art compression and reproducibility, even with a text compression level of 16, well-configured text-based baselines that perform extremely well.
- † University of Edinburgh
- ** Work done while at Apple
