AI is now capable of developing its own inference hardware

OpenTPU, an open-source AI accelerator developed using AI design principles, demonstrates that AI agents can successfully design and build hardware capable of running their own inference workloads. The project successfully executes ten modern language models on an FPGA card, achieving competitive performance with decode speeds ranging from 3.78 to 85.8 tokens per second depending on model size and quantization, while utilizing 82-94% of peak DRAM bandwidth. OpenTPU also serves as an educational resource, providing a complete end-to-end implementation from hardware design to software drivers that illustrates how AI accelerators function.