I'm making LLMs work on small and old devices

A developer has created "smolmaxxing," a project that optimizes large language models to run on older and low-power devices like a 2013 iPad Air and budget TV boxes by using custom kernels, intelligent memory management, and weight streaming techniques. The project demonstrates that these legacy devices can run models ranging from 260K to 16 billion parameters with practical inference speeds—such as 24-26 tokens/second on the iPad Air for a 1.7B model and 5-7 tokens/second for an 8B model on the TV box. All performance claims are validated through actual hardware testing with detailed documentation of each optimization step.