▼The Inference-Cost Efficiency Race

Score history

Daily conviction score, 0 to 100. Higher means the thesis is more strongly corroborated.

Now 69 · +3 since Aug 27 · ranged 69 to 72

Showing the last few days. Unlock full score history.

Why the conviction moved

• Aug 28
  Strengthened +3

  New research on binarized network pruning and multi-drafter speculative decoding targets inference cost from both the weight-footprint and the tokens-per-query side. Multi-drafter speculative decoding in particular cuts serving cost without changing the model, keeping efficiency a competitive axis independent of capability releases.

• Aug 25
  Strengthened +5

  KVBoost reuses key-value tensors at the chunk level and recomputes only where deviation is large, attacking prefill cost on prompts that share content but not a leading prefix — the case prefix caching structurally cannot serve. Agent and RAG traffic is dominated by exactly those reordered-context prompts, so the addressable share of prefill spend is large.

Showing the last 2 days. Unlock the full record.

Source trail

• Supporting · August 28, 2026

  Small Models and Compression Research Push More Inference Off the Cloud

  New research on binarized network pruning and multi-drafter speculative decoding targets inference cost from both the weight-footprint and the tokens-per-query side. Multi-drafter speculative decoding in particular cuts serving cost without changing the model, keeping efficiency a competitive axis independent of capability releases.

  Hacker News

• Supporting · August 25, 2026

  A New Cache Method Attacks the Prefill Cost That Prefix Caching Cannot Reach

  KVBoost reuses key-value tensors at the chunk level and recomputes only where deviation is large, attacking prefill cost on prompts that share content but not a leading prefix — the case prefix caching structurally cannot serve. Agent and RAG traffic is dominated by exactly those reordered-context prompts, so the addressable share of prefill spend is large.

  arXiv cs.AI