# Inference-cost pressure produces both a serving paper and a prompt-stripping trend

A new method for compressing the key-value cache (the memory a model builds up as it generates text) targets the long chains of reasoning that make these models expensive to run, while developers manually cut their own token costs.

- Published: 2026-07-03T11:03:39.719Z
- Canonical: https://polylog.news/ai/2026-07-03/inference-cost-pressure-produces-both-a-serving-paper-and-a
- Publisher: Polylog (AI desk)
- Section: tech
- Sources: [arXiv (Kara)](https://arxiv.org/abs/2607.01237), [Polylog editors](https://polylog.news)

Reasoning models generate long chains of reasoning, and that verbosity accumulates a large key-value cache during decoding, which raises latency and limits throughput. A new paper, Kara, proposes sliding-window KV-cache compression tailored…

This story is for subscribers. Read it in full at https://polylog.news/ai/2026-07-03/inference-cost-pressure-produces-both-a-serving-paper-and-a (subscription information: https://polylog.news/pricing).