# AI Inference Shifts to Consumer Devices

Over the next 3-6 months, smaller efficient architectures and inference-cost optimizations push capable AI off the cloud and onto laptops, phones, and mobile NPUs.

- Conviction: 54 / 100 (weakening)
- 7-day move: -14
- 30-day move: -23
- Horizon: Medium term (3-9 months)
- Tracking since: 2026-06-15T00:00:00.000Z
- Last updated: 2026-07-31T14:01:48.572Z
- Canonical: https://polylog.news/ai/trends/on-device-ai-inference
- Publisher: Polylog
- Affected regions: Global

## Recent score history

- 2026-07-30: 56
- 2026-07-31: 54

## Recent evidence

- [confirms] A New Local Speech-to-Text Library Aims to Widen On-Device Transcription (2026-07-19): transcribe.cpp offers drop-in whisper.cpp support with Rust bindings and numerically validated models for cross-platform offline speech recognition per tech reporting, expanding the tooling that runs capable inference locally rather than in the cloud.
- [confirms] PrismML's 1-Bit Bonsai 27B Compresses a 27-Billion-Parameter Model to 3.9 Gigabytes (2026-07-15): PrismML's 1-bit quantization-aware training shrinks a 27B model to 3.9GB, small enough to load on consumer hardware. Compressing frontier-scale models to a handful of gigabytes removes the memory barrier that keeps such models cloud-bound.

17 more evidence entries, the full score history, the conviction-driver timeline, and affected assets are for subscribers: https://polylog.news/pricing