What Running Kafka on VMs Taught Us About Systems Thinking

A team at Moniepoint proactively rebuilt their Kafka infrastructure from manually-managed VMs running Docker Compose to Strimzi before experiencing a crisis, recognizing that their setup didn't scale despite functioning adequately. The original VM-based approach required manual SSH access for every configuration change, risked downtime with each restart, suffered from inconsistent versions across instances, and lacked standardization for provisioning new pipelines. This proactive architectural decision demonstrates the value of systems thinking—identifying and addressing scalability problems before they become operational disasters.