|
Show HN: Maple – Sub-2-BPW Moe Pruning to Run Gemma 4 26B at 45 TPS on a 6GB GPU
Maple is a pruning technique that compresses Gemma 4 26B from 9.92 GB to 4.75 GB by removing 80 dormant experts, enabling the model to run entirely on a 6GB GPU laptop at 45.4 tokens per second. The method uses domain-specific calibration to identify and preserve only the most relevant experts for specialized tasks like coding and technical reasoning, making it suitable for edge deployment.
Read Full Article →
← Back to latest