Rob Pankow 
Two processes hold the same volume at the same instant. Whether that is routine or catastrophic depends entirely on which layer is arbitrating, and on the platform most teams are leaving, nobody had to choose. VMFS was a clustered filesystem with a distributed lock manager built into it, so many hosts could mount one LUN concurrently and the question was answered before anyone thought to ask it. vMotion leaned on that guarantee. HA restart leaned on it. Shared disk guest clustering leaned on it.
On Kubernetes the same guarantee has a name, ReadWriteMany, and it is a request rather than an inheritance. Something underneath has to actually implement it, and the declaration in a PersistentVolumeClaim does not say what that something is.
The part that gets missed is that ReadWriteMany does not describe one mechanism. It describes two, and which one you get depends on the volume mode. One of them is nearly free. The other is a distributed filesystem problem, and the ordinary way of solving it puts every client behind a single server. That is the one KubeVirt reaches for when a virtual machine needs shared state on a filesystem, and it is where parallel NFS stops being an acronym and starts being the difference between a VM that can be evacuated from a node and one that cannot.
What ReadWriteMany actually asks the storage layer to do
Kubernetes access modes are a contract about attachment, not an implementation of anything. ReadWriteOnce says one node may mount the volume for reading and writing. ReadWriteMany says many nodes may. The control plane enforces the declaration by refusing attachments that exceed it, and the CSI driver reports which modes it can honour. Neither of them supplies concurrency safety. They police the claim and leave the semantics to whatever sits below.
Below, there are two genuinely different implementations hiding under the same access mode.
With volumeMode: Block, the volume is handed to the consumer as a raw device. There is no filesystem in the path, so there is nothing to keep consistent. RWX here means only that two nodes are permitted to open the same device at once, and absolutely nothing arbitrates what they write. That is safe exactly when the consumer knows it is sharing: a cluster-aware application with its own locking, or QEMU coordinating a handover during live migration. Give the same device to two pods that each believe they own it and the result is the corruption you would expect.
With volumeMode: Filesystem, RWX means a real shared filesystem, and now three separate obligations appear that a block device never had:
- Namespace and metadata consistency. A file created on one node has to be visible, with the right size and attributes, from the others. Directory operations have to be atomic against concurrent ones.
- Lock arbitration. Byte-range and whole-file locks have to mean something across nodes. A lock granted on one client has to be honoured on every other.
- Cache coherence. Clients cache aggressively because the alternative is unusable. Something has to invalidate those caches when another client writes, or readers quietly serve stale data.
VMFS supplied all three inside the hypervisor, which is why the concurrency question never surfaced as an architectural decision on the old platform. It was a property of the datastore. On Kubernetes it is a property of whatever you chose to provide RWX, and the choices differ from each other far more than the access mode suggests.
How pNFS splits the control path from the data path
Classic NFS puts metadata and bulk data through the same door. The client talks to one server for LOOKUP, OPEN and LOCK, and it talks to the same server for every byte of every read and write. That single door is simple, well understood, and it is also a ceiling: aggregate throughput for every client on the export is bounded by one machine’s network interface, one machine’s CPU, and one machine’s memory bandwidth.
Parallel NFS, standardised as part of NFSv4.1 in RFC 5661, removes the bulk data from that path. It is a protocol feature rather than a vendor extension, which matters because it means the client side is in the mainline Linux NFS stack rather than in something you install.
The mechanism is a layout. The client contacts the metadata server and asks for a layout on a file. The layout is a description of where that file’s data physically lives and how to reach it directly. Having been granted one, the client performs reads and writes straight to the data servers, in parallel, with the metadata server entirely out of the data path. The metadata server keeps what it is good at: the namespace, open state, locks and delegations.
The layout is described in one of several formats, and the choice determines what the data servers actually are. The file layout expresses the data as stripes across NFS data servers. The block and SCSI layouts let the client address the blocks over a storage fabric. The flexible file layout, added in RFC 8435, describes the file as a set of mirrors across loosely coupled servers. The common property is that the client learns the map once and then goes directly to the data.
Three details are worth holding onto, because they are where the implementations differ.
A layout is a lease, not a grant. The metadata server can recall it with a CB_LAYOUTRECALL callback, and the client is obliged to flush its outstanding writes and hand the layout back. This is how the server keeps the right to reorganise, to fence a misbehaving client, and to stay authoritative about where data lives even though it is not carrying it.
The metadata path stays sized for metadata. Once the bulk transfer is elsewhere, the control plane’s job is small and latency sensitive rather than large and bandwidth hungry, which are very different machines to build.
Throughput scales by addition rather than by replacement. More data servers and more fabric means more aggregate bandwidth to the same export. On a single head design, the only way to go faster is to make that one head bigger, and eventually there is no bigger.
Shared access is a storage decision, not a checkbox on the PVC. If your migration plan has VMs that must live migrate and state that must be RWX, it is worth an hour with someone who has built the path before you commit to a filesystem stack beside your block storage. Talk to a storage architect
Where a single NFS server head runs out
The failure modes of shared filesystem access are not exotic. They are the predictable consequences of where each obligation is discharged, which is why it is worth setting them out by layer rather than by symptom.
Table 1: What each layer of a shared-access path owes, and how it fails
| Layer | What it has to do | How it fails |
|---|---|---|
| Namespace and lock state | Serialise metadata operations, grant and honour locks across all clients | A busy client storm makes lock latency the slowest thing in the cluster, and every other client waits behind it |
| Bulk data path | Move the actual reads and writes at the rate the workload needs | On a single head, every client shares one NIC and one CPU budget, so one noisy consumer starves the rest |
| Client cache coherence | Invalidate cached data and attributes when another client writes | Stale reads that do not announce themselves, which surface as application-level inconsistency rather than as storage errors |
| Layout lease and recall | Take access back from a client that is unreachable or misbehaving | Without a recall path, fencing a bad client means restarting it, and a split handover has no clean resolution |
The usual answer to RWX on Red Hat® OpenShift® is to run a filesystem beside the block storage, which in practice means CephFS through OpenShift Data Foundation. It is worth being precise about what that does and does not solve, because the lazy version of this comparison is wrong.
CephFS is a genuine distributed filesystem. Metadata is served by a scalable set of metadata daemons, and native clients read and write directly against the object storage daemons rather than through a gateway, so it is not the single head shape. Ceph remains a good option when performance is not the main goal, and that line is not a courtesy.
The distinction is narrower and more practical. The client protocol is RADOS, so the parallel path exists for native CephFS clients but not as pNFS, and any consumer that needs an actual NFS export gets it through a gateway that reintroduces the single door the architecture just avoided. The second cost is structural: it is a second data path, a second failure domain and a second thing to upgrade, running alongside the block storage that is already serving the VM disks. Neither of those is fatal. Both are real, and both are usually discovered after the platform is in production rather than during the design. Based on publicly available information as of September 2026, and not independently verified, pNFS parallel access is not a capability OpenShift Data Foundation presents.
Simplyblock serves RWX with pNFS parallel access out of the same pool that serves the block volumes, which is the reason the comparison is worth making at all. There is no separate filesystem stack to stand up, and the shared volume is drawn from the same erasure-coded capacity, over the same fabric, as everything else the cluster runs.
What live migration does with the volume during handover
Live migration is the place where all of this stops being theoretical, because it is the one operation that deliberately creates the condition the access mode exists to describe.
When KubeVirt migrates a running VM, it does not detach and reattach. It starts a second virt-launcher pod on the target node while the source is still running, pre-copies memory across, then pauses for a very short window and switches execution. For that window, and for the whole pre-copy before it, both pods reference the same PersistentVolumeClaim. The volume is genuinely held twice.
That is why the access mode is a hard prerequisite rather than a tuning parameter. With RWO, the target pod cannot attach the volume while the source holds it. The migration does not degrade, it does not start. The practical consequence is larger than one failed operation: a VM whose storage cannot be held twice is a VM that cannot be evacuated from its node, so node drains stall, and the cluster upgrade that depends on draining stalls behind them. The Multi-Attach error for volume message is the same condition, arriving in the form of an event rather than a design review.
Which of the two RWX mechanisms you need follows from what the volume holds.
VM disks are the easy case. They are block volumes, and RWX block asks only that both nodes may open the device. QEMU on either side coordinates the handover, so no filesystem and no distributed lock manager are involved. This is the path that should carry the bulk of the data, and it is the one that benefits directly from a block protocol with a short path to the device.
Firmware and small state volumes are the hard case. As covered in the earlier piece on where Windows VM firmware state lives after the exit, a vTPM’s contents land on a PVC with volumeMode: Filesystem, and that volume also has to be RWX if the VM is ever to live migrate. So a few megabytes of firmware state, not the disks, is what forces a shared filesystem into the design. This is precisely the volume that does not justify standing up a second storage system beside the first, and precisely the reason it matters whether the platform can serve shared filesystem access from the pool it already has.
The order of operations that follows is unglamorous and worth writing down. Decide which volumes need Filesystem mode before you choose the storage class, not after. Confirm the class can actually honour RWX in that mode rather than only in block mode, because a driver can support both and a class can be configured for one. Then test an eviction, not a migration you triggered by hand: a drain is what an upgrade will do to you at two in the morning, and it exercises the same path with none of the patience.
Questions and Answers
What is pNFS?
Parallel NFS is a feature of NFS version 4.1, standardised in RFC 5661, that separates the metadata path from the data path. The client asks a metadata server for a layout describing where a file’s data lives, then reads and writes directly to the data servers in parallel, without the metadata server in the data path. The practical effect is that aggregate throughput scales with the number of data servers and the fabric rather than being capped by a single NFS server’s network interface and CPU.
Does KubeVirt live migration require ReadWriteMany?
Yes, for any volume the VM keeps during migration. KubeVirt starts a target pod while the source is still running and both reference the same PersistentVolumeClaim, so the volume is attached twice for the duration of the handover. With ReadWriteOnce the target cannot attach and the migration never starts, which also means the VM cannot be evacuated from its node during a drain or a cluster upgrade.
What is the difference between RWX block and RWX filesystem?
They are two different mechanisms behind one access mode. RWX with volumeMode: Block hands a raw device to both nodes and arbitrates nothing, which is safe when the consumer coordinates, as QEMU does during live migration. RWX with volumeMode: Filesystem requires a genuine shared filesystem that supplies namespace consistency, cross-node lock arbitration and cache coherence. The second is substantially harder to deliver at speed, and it is the one a vTPM or EFI state volume needs.
Does OpenShift Data Foundation support pNFS?
Based on publicly available information as of September 2026, and not independently verified, it does not. CephFS is a real distributed filesystem and its native clients do read and write directly against the object storage daemons, so it is not a single server design. The parallel path is RADOS rather than pNFS, and where an NFS export is required it is served through a gateway that reintroduces a single door. Ceph remains a sensible choice when performance is not the primary goal.
How does simplyblock provide RWX for KubeVirt?
Simplyblock serves RWX volumes with pNFS parallel access from the same pool that provides the block volumes, so shared filesystem access does not require a second storage system beside the block storage. Client traffic goes to the data servers in parallel rather than through one server head, and the shared volume draws on the same erasure-coded capacity and the same NVMe/TCP fabric as the VM disks it sits next to.
