Training a 3.8B LLM to 0.384 CORE for $998 – Hugo Vergnes

A researcher trained a 3.8 billion-parameter language model scoring 0.384 on the CORE benchmark for $998 using rented B200 GPUs over 43 hours on 65 billion tokens. The project demonstrates that meaningful model training is achievable for individual researchers with modest budgets, outperforming comparable prior work like nanochat while leveraging good infrastructure design and Llama-style architecture choices. The work highlights that B200s offer better value than H100s and shows how disciplined software engineering enables efficient experimentation in AI development.