WebLLM: high-performance in-browser LLM inference engine

WebLLM is a high-performance inference engine that runs large language models directly in web browsers using WebGPU acceleration, eliminating the need for server-side processing. The engine is fully compatible with OpenAI's API and supports features like streaming, JSON mode, and structured generation across a range of open-source models while maintaining user privacy through local computation.