| |
Show HN: Shoehorn – Quantize any model down to run on your machine
Shoehorn is a tool that quantizes large language models to run on consumer hardware by scanning Hugging Face's most-downloaded models and recommending ones that fit within your machine's memory constraints. Users select their hardware (Mac, GPU, or custom specs), and the browser-based tool ranks available models by quality for that budget, then uses llama.cpp as an inference backend to enable local chatting. The application is available for macOS, Linux, and Windows with a simple one-button installation and interface.
Read Full Article →
← More Tech news