Large-scale linguistic models (LLMs) now sit at the core of search, help, and agent workflows, making semantic caching critical to reducing decision costs and delays. Production deployments often use a tiered design with dynamic states: a static cache of curated, offline-checked responses mined from the log, backed by a dynamic online cache. In practice, both categories are often governed by a single embedding constraint, which encourages a difficult trade-off: conservative constraints miss safe reuse opportunities, while strict constraints risk giving statistically incorrect answers. Introducing Krites, an adaptive buffering policy, judged by LLM that extends static coverage without changing supply decisions. In the sensitive mode, Krites behaves exactly like the normal threshold policy. If the static nearest neighbor of the input falls just below the static threshold, Krites automatically invokes the LLM judge to determine whether the static response is acceptable for the new input. Authorized matches are developed in a dynamic cache, allowing iterations and future iterations to reuse selected static responses and to increase static access over time. In tracking-driven simulations of chat and search activity, Krites increases the proportion of requests that are provided with selected static answers (direct direct hits and confirmed promotions) up to 3.9 times with chat traffic and search-style queries relative to tuned bases, with unchanged critical path latency.
