Inkling-2-Mini in 5.4 GB Memory
LookAtMyQuant is a quantization compiler that compresses AI models by measuring how much each component affects output quality when compressed, then optimally allocates compression codes to fit size budgets. The technique achieved notable results, including compressing Qwen3.5-4B to 1.23 GB (2.3 bits per weight) while maintaining strong performance on math problems, with lower quality loss compared to competing methods like Prism ML's Bonsai. The method uses post-training quantization with trellis codes, calibrated on the model's own solutions, followed by distillation steps.