CS336: Language Modeling from Scratch

CS336 is a Stanford course that teaches students to build language models from scratch, covering the complete pipeline from data collection and preprocessing through transformer architecture implementation, training, and evaluation. The course emphasizes hands-on implementation with minimal scaffolding, requiring students to write significant amounts of Python code and optimize models for efficient GPU training across multiple machines. Prerequisites include proficiency in Python, deep learning, and foundational knowledge in mathematics, statistics, and machine learning.