USDB: Scaling Storage for Next-Generation Recommender Systems at ByteDance

ByteDance's USDB is a unified storage system designed to handle the storage asymmetry in modern recommender systems, which require processing long, attribute-rich user histories with frequent small updates and large repeated reads. The system uses innovations like time-based compaction staging, differentiated replicas, and hybrid columnar formats to achieve 85.8% lower write latency and 42.1% lower infrastructure costs compared to optimized baselines. USDB enables efficient data reuse across ingestion, inference, and training while supporting both streaming and batch workloads at production scale.