Sub-1-Bit LLM Compression via Latent Factorization

LittleBit is a compression technique that reduces large language models to sub-1-bit precision (as low as 0.1 bits per weight) by factorizing weight matrices into low-rank binary factors with learned scales, while maintaining the original model architecture at inference time. LittleBit-2, an improved version, enhances compression quality through latent geometry alignment during initialization using Joint-ITQ, with no additional inference overhead. The open-source implementation supports multiple popular LLM architectures including Llama, OPT, Qwen, Gemma, and Phi.