The Dataflow Model Revisited

The authors of the influential "Dataflow Model" paper reflect on their work eleven years later, acknowledging that while core concepts like event time and strong consistency remain sound, they overemphasized windowing and triggers in favor of simpler database approaches like SQL and materialized views. They reveal that the key insight missed was understanding streams and tables as different representations of the same object, and that snapshot-consistent refresh mechanisms ultimately reached far more users than the more complex watermark-based approach by asking less of them.