Same model, same Q4_K_M label: 5.02, 5.07 and 5.27 bits per weight

The article describes Picchio, a Python tool that measures local large language model (LLM) performance across three metrics: effective bits per weight, inference speed (tok/s) across prefill/decode/wallclock phases, and GPU utilization. The tool identifies measurement inconsistencies in quantized models—such as four Q4_K_M quantizations of the same model measuring between 5.02-5.27 bits per weight—and detects silent CPU fallbacks that can cause performance variations of up to 22x.