Deterministic Simulation Testing in Celld

Cloudflare is developing celld, a distributed system runtime, and using deterministic simulation testing (DST) to identify and reproduce bugs that occur due to unpredictable event ordering in distributed systems. DST runs celld's production code in a controlled simulator environment that can manipulate event sequences, storage operations, and timing—allowing developers to repeat failing scenarios with the same seed value and trace root causes. The approach has already uncovered previously unknown bugs that would be difficult to catch through conventional testing.