Low-latency block storage over standard Ethernet for OpenShift and Kubernetes: native NVMe multi-queue, no special network cards, and no proprietary drivers.
NVMe/TCP carries native NVMe commands over standard TCP/IP. For OpenShift and Kubernetes it delivers low-latency block storage on ordinary Ethernet, without RDMA-capable adapters and without out-of-tree drivers. simplyblock uses NVMe/TCP as the transport for erasure-coded, hyperconverged storage that runs on the compute nodes, so there is no separate storage tier to rack or operate. This guide covers how NVMe/TCP differs from iSCSI, why an NVMe-first design matters once a cluster reaches petabyte scale, how it integrates through CSI, and the cases where NVMe/TCP is not the right transport.
The Storage Transport Problem
Once a cluster carries hundreds of pods and virtual machines and grows toward petabyte scale, the choice of block protocol and its placement in the cluster stop being details.
iSCSI wraps SCSI commands in TCP and carries a single command queue per session, so concurrent I/O is funnelled through one queue and reassembled. For a cluster where many pods and virtual machines drive I/O at the same time, that serialisation is a practical ceiling, and it carries a SCSI translation layer kept only for backward compatibility.
NVMe drives placed behind a data path originally designed for spinning disks and SCSI add a translation layer, an interrupt, and a context switch on every operation. At petabyte scale the limit is rarely raw bandwidth; it is the CPU cost of each I/O and the gap between average and worst-case latency, which is the latency applications actually feel.
Placing storage on a dedicated target that compute nodes reach over the network adds a tier to rack, license, and operate, and it puts a network hop between every workload and its data. Both hardware count and latency grow with it.
How simplyblock Approaches It
What Teams Gain
Storage runs on the compute nodes, so there is no separate storage tier to buy, rack, or operate. Fewer moving parts, one server class, and one thing to scale.
An NVMe-first, poll-mode data path holds tail latency steady during the node drains, rebuilds, and rolling upgrades that a platform team runs every week, not just in a quiet benchmark.
Distributed erasure coding protects data at a fraction of the raw-capacity overhead of replication, so the same usable capacity needs far fewer NVMe drives.
NVMe/TCP is an open, in-kernel standard on commodity hardware. No proprietary drivers, no appliance, and no RDMA fabric requirement to get started.
Run the numbers
Enter your usable capacity and compare 1.5× raw with erasure coding against 3× with replication, plus the compute you reclaim from lower latency. No sign-up, and the result is printable.
iSCSI carries SCSI commands over TCP with a single command queue per session, which serialises concurrent I/O. NVMe/TCP carries native NVMe commands and preserves the NVMe multi-queue model, so many pods and virtual machines can drive parallel I/O without a single-queue bottleneck. Both run on standard Ethernet, so NVMe/TCP keeps iSCSI's operational simplicity while removing its architectural limits.
No. NVMe/TCP runs over standard Ethernet and standard NICs, and the initiator has been in the mainline Linux kernel since 5.0. RDMA-capable adapters and a lossless fabric are only needed for NVMe/RoCE, which simplyblock also supports where the lower latency is worth the additional fabric complexity.
Three cases. If you already run a lossless RDMA fabric and need the lowest possible latency, NVMe/RoCE will beat NVMe/TCP. If the workload is not latency or IOPS sensitive, such as cold archival data or large sequential batch, a cheaper capacity tier is usually a better fit. And if storage never leaves a single node, a networked block protocol adds coordination without a matching benefit. Most block workloads at scale sit outside these cases.
On the compute nodes. simplyblock is hyperconverged: storage and compute share the same servers, so there is no separate storage tier to rack or license. NVMe/TCP is the transport between storage services across those nodes, and storage follows the workload to preserve local-read latency.
Ask about NVMe/TCP versus iSCSI, running storage on the compute nodes, CSI integration, or where NVMe/TCP is the wrong choice.
Free, no obligation
Book a 30-minute call with a simplyblock storage architect. We will help you cut storage cost and hit your IOPS and latency targets on Kubernetes, with practical recommendations you can act on right away.
Book my session