Show HN: KVBoost – chunk-level KV cache reuse for HuggingFace, 5–48x faster TTFT

KVBoost is a Python library that optimizes LLM inference through chunk-level KV cache reuse, achieving 5–48x faster time-to-first-token (TTFT) without requiring model changes or extensive VRAM. It uses techniques like FlashAttention-2, AWQ layer streaming, and CPU-based KV cache offloading to enable efficient inference of large models (32B+) on consumer hardware with as little as 8GB VRAM. The system demonstrates 86% average KV reuse and maintains consistent 20ms TTFT across multi-turn conversations while baseline HuggingFace inference degrades to 122ms by turn 8.