Every unit of work in a data processing system should aspire to be small, independently processable, and consistently sized. When these three properties hold, scaling becomes almost trivially simple. Reality rarely cooperates, which is why understanding these properties matters so much for platform engineering.

Batch processing architecture has a clean pattern that scales elegantly: decouple batch systems asynchronously from everything else. When you get this right, your real-time system stays stable regardless of batch volume, and you never need elaborate job scheduling to avoid infrastructure strain.

When designing data platforms, I frequently encounter teams trying to build one unified system that handles both real-time streaming and batch analytics. The instinct makes sense: both workloads operate on the same underlying data, so why not share infrastructure?

Getting this architecture right has real consequences.

The challenge is that these workloads have fundamentally different characteristics. Supporting both well on a single platform is expensive and complex. In most cases, you get better results by separating them early and letting each system lean into its strengths.

Most architecture discussions devolve into abstract debates about microservices, monoliths, and database choices. After years of explaining these concepts to engineers and product leaders, I've found that thinking about software architecture like a physical building cuts through the noise and makes the tradeoffs viscerally clear.

This isn't just a teaching metaphor. It's a decision framework that surfaces why some changes cost weeks and others cost months, why certain tech debt compounds silently while other debt screams at you daily, and how to gauge the right amount of architectural runway to build.