| |
Show HN: TurboGPT: train 22KiB transformer in 13s
TurboGPT is a CUDA C++ implementation for training tiny transformer models efficiently, capable of training a 22KiB transformer in 13 seconds. The project includes tools for checkpoint management, TensorBoard-compatible logging, and can achieve 2.52435 BPB (bits per byte) loss on a 1.5G token dataset. It supports Windows with Visual Studio 2022 and CUDA 13.4, allowing users to train and resume models with customizable configurations.
Read Full Article →
← More Tech news