Kubernetes Storage Architectures: How the Approaches Compare

Most Kubernetes storage evaluations start with a vendor shortlist and end in a feature-matrix stalemate, because feature lists converge: everything has snapshots, everything has a CSI driver, everything claims high performance. What does not converge is architecture. Where storage runs, how data is protected, how the software is packaged, how the data path is built, and how the estate is shaped are decisions that set the cost curve and the operational model for years, and no feature checkbox undoes them.

This guide compares the approaches along those five axes rather than by product name. Each axis is a genuine trade-off with a right answer that depends on your estate, not a strawman. The last section says plainly where simplyblock sits on each axis, where that combination is the right call, and where it is not. Worked figures below are arithmetic from stated example assumptions; the TCO calculator runs the same math on your numbers.

The five axes that decide the outcome

The axes are largely independent: a platform picks a point on each, and most of the meaningful differences between Kubernetes storage options reduce to different combinations of these five choices.

Table 1: The five axes. A storage decision is a position on each axis; a product is one fixed combination of positions.

The order below follows the money: placement and protection dominate the hardware bill, packaging sets the licence curve, the data path decides what applications feel, and topology shapes operations.

Where storage runs: hyperconverged or disaggregated

The first choice is physical: does storage run on the same nodes as the workloads, or on a separate tier of storage servers reached over the network?

Disaggregated designs put storage on dedicated targets. The pool is managed independently of compute, can grow independently, and can serve many consumers beyond one Kubernetes cluster. The price is a second hardware footprint to buy, rack, power, and operate, plus an unavoidable network hop on every I/O. Disaggregation earns its keep in estates where a very large shared pool serves heterogeneous consumers, or where storage and compute genuinely grow on unrelated schedules.

Hyperconverged designs run the storage software on the compute nodes themselves, pooling the NVMe drives already in those servers into one distributed system. There is one hardware footprint, data can be served with node-local awareness, and there is no separate tier with its own lifecycle, firmware matrix, and on-call rotation. The historical objection to hyperconverged storage was that the storage stack stole CPU from workloads; whether that holds depends entirely on the data path efficiency covered in axis 4.

For Kubernetes and Red Hat® OpenShift® estates, where the unit of growth is the node and workloads are scheduled across the same servers that hold the drives, hyperconverged placement is the natural fit. Simplyblock is hyperconverged: it runs on the compute nodes and presents block volumes over NVMe/TCP, so adding a node adds compute, capacity, and performance together.

How data is protected: replication or erasure coding

Every distributed storage system must survive drive and node failures. The two mechanisms are copies and math, and the choice between them is the single largest driver of drive count at scale.

Three-way replication stores three full copies of every block on different nodes. It is simple, fast for the smallest random writes, and brutally expensive in capacity: every usable terabyte needs three raw terabytes.

Erasure coding splits data into chunks and computes parity. A 4+2 scheme writes four data chunks and two parity chunks across six nodes; any two can fail and the data survives, which is the same two-failure tolerance as triple replication at half the raw capacity. The historical objection was CPU cost and write amplification, which modern implementations address by coding in user space at NVMe speed; that is again an axis 4 question.

The arithmetic at 1 PB usable, on 15 TB NVMe drives:

Table 2: The same usable petabyte and the same fault tolerance, at double or half the drive count. The gap scales linearly with capacity: at 10 PB usable it is 1,000 drives.

Replication keeps a narrow edge for the very smallest random writes, since it avoids the read-modify-write of partial-stripe updates. Erasure coding wins decisively on capacity economics, and at petabyte scale capacity economics is the bill.

How the software ships: your hardware or an appliance

Two systems with identical architecture can have completely different economics depending on how they are sold.

The appliance model bundles storage software with a fixed hardware SKU, usually licensed per node or per device. Procurement is simple and support is one throat to choke. The costs arrive later: hardware refresh on the vendor’s schedule and price list, no reuse of servers you already own, capacity expansion quantized to the vendor’s node sizes, and a licence meter that runs on infrastructure units rather than on what you actually store.

Software on commodity hardware decouples the two. You choose the servers, fold in existing hardware where it meets spec, expand in whatever increments make sense, and switch hardware vendors without touching the storage layer. Licensing can then follow the platform you already run: aligned to the cores you license for OpenShift or Kubernetes when hyperconverged, or to usable capacity when storage is a separate disaggregated tier, rather than to the vendor’s appliance SKUs and capacity tiers.

Table 3: Same architecture, different economics. The licence meter matters most at scale: appliance capacity tiers penalize exactly the dense, efficient clusters you want to build.

Simplyblock ships as software on commodity hardware and is priced like your platform: by the CPU capacity of your worker nodes, counted the same way as your OpenShift or Kubernetes subscription, and data volume does not change the price. Storage spend moves with the platform budget you already plan.

How the data path is built: NVMe-first or retrofitted

Two storage systems can use identical NVMe drives and deliver very different latency, because what sits between the workload and the flash differs.

Retrofitted stacks predate NVMe. Their data paths were designed for spinning disks and SCSI semantics: single-queue protocols, kernel block layers tuned for devices a thousand times slower, interrupt-driven I/O, and multiple context switches and buffer copies per request. Put NVMe drives behind such a path and the drives are no longer the bottleneck; the software is. The overhead shows up twice, as CPU consumed per I/O and as latency variance under load.

NVMe-first stacks are built for flash semantics end to end: multi-queue from the workload to the drive, poll-mode drivers in user space instead of interrupts, and zero-copy paths that avoid the kernel block layer entirely. The same I/O costs fewer CPU cycles, and on a hyperconverged system those cycles belong to workloads, which connects this axis back to axis 1: an efficient data path is what makes storage on the compute nodes affordable.

The metric that separates the two designs is not average latency, where both look respectable on an idle system. It is tail latency, the p99 and beyond, under real concurrent load. Applications feel the tail: a database commit waits for its slowest write, not its average one. When you evaluate, benchmark the tail at your real queue depths, on loaded systems, or the numbers will not survive contact with production.

Simplyblock’s data path is NVMe-first, user-space, and poll-mode, and presents volumes over NVMe/TCP on standard Ethernet, so the multi-queue path extends across the network without RDMA hardware. The details are in the NVMe/TCP guide.

How the estate is shaped: one large cluster or many small

The final axis is topology. Some estates concentrate everything in a few large clusters; others run many small ones across sites, factories, branches, or availability zones.

One large cluster concentrates capacity and administration: one pool to monitor, one upgrade to run, maximum statistical efficiency from shared headroom. The trade-off is blast radius and physics: everything shares a failure domain boundary, and workloads that live far from the cluster pay for the distance in latency.

Many small clusters put storage next to the workloads and cap the blast radius of any failure at one site. Aggregate scale arrives by adding clusters, not by growing one. The trade-off is operational: whatever the per-cluster overhead is, you pay it N times, which is why this topology punishes storage systems with a heavy per-cluster footprint or minimum node counts.

The honest answer on this axis is that the workload map decides, not the storage software. What the storage layer owes you is indifference: the same software, operating model, and licence from a three-node edge cluster to a hundred-node core cluster, so topology remains an infrastructure decision rather than a procurement event. Simplyblock runs the same way at both ends of that range, which is the extent of its opinion on the matter.

Where simplyblock fits, and where it does not

Simplyblock is one fixed combination of positions on the five axes: hyperconverged placement, erasure-coded protection, software on commodity hardware priced in line with the platform, an NVMe-first user-space data path over NVMe/TCP, and indifference to cluster topology. That combination is deliberate, and like every architecture it has a domain where it wins and a domain where it does not.

Table 4: Fit by estate profile. The “no” rows are load-bearing: an architecture optimized for everything is optimized for nothing.

If you are weighing these axes for a specific move, the VM migration guide applies them to the highest-stakes case: sizing and cutting over a virtual machine estate to OpenShift Virtualization. And for any estate, the five-axis comparison reduces to arithmetic faster than to opinion: put your own capacity and node figures through the math and see which combination your numbers actually favor.