Morning Edition · Wednesday, September 23, 2026Published at 7:26 AM EDT · New York

The Choice of 4-Bit Quantizer Decides Whether a Fine-Tuned Model Leaks Private Data

Researchers found that calibration-free GGUF Q4_K_M reproduced 5.3 percent of planted private records while calibration-based AWQ and GPTQ reproduced none, at the same bit width.

Organizations routinely fine-tune small language models on private data and then compress them to four bits so they can run on modest hardware. A paper published on arXiv by researchers Cristhian Kapelinski and Diego Kreutz, presented at th…