ESP32S3 cluster running 1.58-bit (BitNet) Language model

A team has created a distributed inference system running a 1.58-bit quantized language model (BitNet) across a cluster of seven ESP32S3 microcontrollers, with one serving as a master node handling tokenization and embedding while the other six compute nodes process transformer layers in parallel via high-speed SPI communication. The system slices a 0.5B parameter model across the cluster, with the master node orchestrating input/output and the compute nodes handling the intensive attention and MLP operations using ultra-low-precision (1.58-bit) weights to fit within the microcontrollers' memory constraints.