Llama-modchip: 284B DeepSeek on 64GB RAM, no GPU
Llama-modchip is a software project that enables running extremely large language models (up to 284B parameters) on standard CPUs with 64GB RAM without requiring GPUs, achieving speeds of 44-111 tokens per second depending on the model and hardware configuration. The technology uses advanced compression techniques including Hessian-based weight compression and quantization to reduce model sizes from 76.87 GiB down to 56.56 GiB while maintaining performance across 256K context windows.