Show HN: Spinf-serve – inference server for bulk scoring and decisions

Spinf-serve is an open-source inference server designed for bulk classification and decision-making tasks on a single 24GB NVIDIA GPU, offering an OpenAI-compatible API. Unlike chat-based servers that re-read content for each question, it processes multiple questions about the same content in a single forward pass, making it faster and more efficient for high-throughput applications like scoring millions of documents. The tool is optimized for returning probabilities over fixed answer sets rather than generating text, and can be self-hosted on compatible NVIDIA hardware.