Huawei releases MACaQ, its internal quantizer for local inference

Huawei has released MACaQ, a model-agnostic quantizer that enables efficient local inference for text, image, video, and audio models across multiple frameworks including llama.cpp, vLLM, Diffusers, and PyTorch. The tool can quantize a 122-billion-parameter model in 64 minutes using just 32.5 GB of memory and achieves lower perplexity than competing quantizers like MR-GPTQ while requiring 11 times less quantization time.