Chris Engelbert 
A disk fails in a twelve node cluster at 02:14, and nothing pages anyone. Nothing is down. The cluster is doing exactly what it was built to do, which is rebuild the lost copies from the survivors as fast as the network will carry them. By 02:40 the database team has filed a ticket about write latency. By 03:10 somebody is reading a backfill throttle tuning guide for the first time, on a cluster that has been in production for four months.
That night is the part of the open source storage decision that never reaches the evaluation spreadsheet. The spreadsheet compares what each option gives you. It does not compare what each option then asks of you, every week, for the next five years.
The gap matters more after a vSAN exit than it would have before one, because vSAN was a clustered storage system that almost nobody operated as one. VMware® vSphere presented it as a checkbox and a storage policy. The rebuild logic, the placement rules, the capacity arithmetic and the failure handling were all there, and all of it was somebody else’s job. Replace it with an open source clustered storage system and that job becomes yours, together with the licence savings that made the swap attractive.
This post is about the second half of that trade. It is not a survey of which project to install: we keep a separate tool survey for that question. The question here is narrower and harder to answer from a feature matrix. Once it is running, what do you run?
What you actually get on day one
Start with the honest inventory, because the open source options in this category are not weak software and pretending otherwise wastes everybody’s time.
Ceph is the serious one. Unified block, file and object from a single cluster, genuinely mature, operated at petabyte scale by institutions that have run it for a decade, and the reference implementation for most of what people mean when they say software defined storage. Ceph is a good option when performance is not the main goal. That sentence is the entirety of the fair assessment, and it is not a backhanded one: for capacity tiers, archives, object workloads and estates with a platform team that already knows RADOS, it is frequently the right answer and the licence savings are real.
The lighter projects trade breadth for simplicity. Longhorn and OpenEBS take a node local approach that is far easier to stand up and far easier to reason about, at the cost of the placement sophistication and the multi protocol reach that a vSAN estate was quietly relying on. We have written up where that simplicity stops being an advantage when the workload is a VM estate rather than a handful of stateful containers.
Then there is the packaged route, and it contains the fact that surprises the most buyers. Red Hat® OpenShift® Data Foundation is Ceph, productised, supported, and licensed. ODF Essentials is included in OpenShift Platform Plus but not in OpenShift Container Platform. ODF Advanced, which is where metro synchronous replication, regional asynchronous replication and VM level disaster recovery orchestration live, is a separate licence sold per two cores and is included in neither, with VM level DR additionally requiring RHACM. A team that assumed its OpenShift subscription covered storage after the exit is often looking at a second purchase, and the alternative to that stack is a question worth asking before the renewal rather than after it.
So the fork is not really open source against commercial. It is this: the supported build of the open source option is a purchase, and the free build is the one your team operates. Day one, the free build hands you pooled capacity, replication or erasure coding, snapshots, a CSI driver and no licence invoice. Everything below is what arrives in the months after that.
What breaks first, and what each break costs
These are ordered the way estates actually hit them, not by severity. The first two land before anything has gone wrong at all, which is exactly why they are missed during evaluation.
Week one: the capacity you bought is not the capacity you have
Most open source clustered storage defaults to three way replication, because three way replication is simple, fast to rebuild and easy to reason about. It is also 3:1. Every usable terabyte costs three raw terabytes.
If the estate is coming off a vSAN RAID-1 mirroring policy at FTT1, the baseline being replaced was 2:1, and the destination sizing was very likely done against the datastore used figure rather than against the new overhead ratio. That is a capacity plan that is out by fifty percent before a single VM has migrated.
Erasure coding exists in these systems and closes much of the gap on paper. In practice it moves the cost rather than removing it: on a replication oriented architecture, erasure coded pools pay for the saved capacity in CPU and in small write latency, which is precisely the profile a VM estate generates. Teams that turn it on for the capacity and turn it off again for the latency are common enough to be a pattern.
Month one: the storage stack is running on the nodes you sized for workloads
A hyperconverged design puts the storage software on the same hosts as the workloads, which is the point of it. What the evaluation rarely quantifies is how much of the host the storage software takes.
The efficiency spread between storage stacks here is not marginal. Based on publicly available information as of September 2026, and not independently verified, a Ceph derived stack delivers on the order of 5,000 write IOPS per CPU core, against roughly 100,000 for a stack built on a polled, user space I/O path. An inefficient storage layer can consume something close to a quarter of a host before it has served a single unusual workload.
That cost does not appear as a storage line. It appears as compute you have to buy, on nodes you are licensing for something else, which is an uncomfortable place for it to hide when the whole exercise started as a licence reduction.
The first disk failure: the rebuild is the outage nobody declared
This is the 02:14 story, and it is structural rather than a tuning defect.
When a replica based cluster loses a device, it restores redundancy by re streaming data across the surviving nodes. The cluster is healthy throughout and no alert fires for unavailability, because nothing is unavailable. What degrades is latency, for every workload sharing those nodes and that network, for as long as the rebuild takes. On a dense node with modern drive capacities that window is measured in hours.
Getting that window under control is real work: backfill throttles, recovery priorities, scrub scheduling, and a placement policy that does not put the rebuild traffic through the same path as production. None of it is mysterious. All of it is yours now.
Month six: what you actually bought was headcount
By the second quarter the pattern is clear. Version upgrades have to be sequenced and rehearsed. Placement rules need revisiting every time the hardware profile changes. Capacity has to be kept below the threshold where a full cluster becomes an incident rather than a purchase order. And the client protocol turns out to matter: RADOS is a rich protocol with its own semantics and its own client behaviour to learn, where NVMe over Fabrics is a block protocol the kernel already speaks.
None of this is an argument that the software is bad. It is an argument that the line item moved. It did not disappear.
| Symptom | Root cause | What it costs you |
|---|---|---|
| Destination sizing is short by roughly half | Three way replication at 3:1 replacing a 2:1 mirroring policy | Unplanned capacity purchase in the first quarter |
| Workload nodes need more cores than modelled | Storage stack consuming host CPU, up to about a quarter of a host | Compute and platform licences, billed as something other than storage |
| Latency complaints with no outage and no alert | Rebuild re streaming replicas across production nodes and network | Hours of degraded service per device failure, no SLA breach recorded |
| Upgrades slip and drift accumulates | Placement, scrub and recovery tuning owned in house | A named engineer, permanently, plus an on call rotation |
Table 1: The four costs that replace the vSAN licence line, in the order estates encounter them.
Sizing a vSAN replacement and unsure which of these four you are about to absorb? Talk to us before the destination hardware is ordered, when the capacity and CPU arithmetic can still change the bill of materials. Talk to a storage architect
Where the line falls between free to licence and free to run
The useful question is not whether open source clustered storage is good. It is whether your estate’s profile puts the operating cost below the licence saving, and that has a reasonably clear answer in both directions.
Open source is the right call when performance is not the primary constraint, when capacity growth matters more than tail latency, when the workloads are object, archive, backup targets or analytics rather than latency sensitive VMs, and, decisively, when you already employ or intend to employ people who want to operate storage. If a platform engineer on your team is genuinely interested in this problem, that changes the arithmetic more than any feature comparison does.
It is the wrong call when the estate arriving from vSphere carries latency sensitive database and application tiers with commitments attached, when the platform team is already at capacity with the migration itself, or when the exit business case depended on not adding headcount. A licence saving funded by an unbudgeted engineer is not a saving, it is a transfer.
For the estates in that second group, the peer option worth putting alongside the open source candidates is a storage layer that keeps the efficiency properties without handing you the operations. simplyblock is one, and the structural differences are the ones this post has been walking through rather than a feature list. Erasure coding at roughly 1.25:1 at FTT1 instead of about 3:1, so the capacity arithmetic does not move against you. Far more write IOPS per host CPU core, so less of each node is spent on storage. NVMe over Fabrics as the client protocol, which the Linux kernel already speaks, rather than a protocol your team has to learn. And volumes that detach and re attach elsewhere rather than rebuilding, which is what turns the 02:14 story into a non event. Both deployment models are supported, hyperconverged and disaggregated, so that choice stays a deployment decision you can revisit later rather than a vendor decision you make once.
Two practical notes that matter for an exit specifically. simplyblock is priced like the platform it runs on: by the CPU capacity of your worker nodes, counted the same way your OpenShift or Kubernetes subscription counts them, and data volume does not change the price. And backing devices are not restricted to local NVMe, so LUNs an existing array presents to the storage nodes can become pool capacity, which is a real answer to “we are not refreshing storage this year”. Two caveats travel with that: local NVMe remains the production performance recommendation, so treat array backed pools as a transition option rather than a greenfield design, and the devices are formatted when they join the pool, so the capacity is reused but the data is still migrated.
If you want the wider option set rather than this narrower operating question, the vSAN alternatives comparison covers commercial and open source together, and there is a direct simplyblock and Ceph comparison for readers who have already narrowed it to those two.
Questions and Answers
What is clustered storage, and is it the same thing as vSAN?
Clustered storage pools the drives in several nodes into one logical storage system, keeps redundant copies or erasure coded fragments spread across those nodes, and survives the loss of a node or a device without losing data. vSAN was one implementation of that idea, integrated into the hypervisor and presented through storage policies. Open source projects such as Ceph implement the same idea with the integration and the operational work left to you, and simplyblock implements it with the operational work kept inside the product. The architecture is comparable; what differs is who runs it.
Is open source clustered storage actually free?
The licence is free. The operating cost is not, and this post is a walk through where it lands: capacity overhead from replication factors, host CPU consumed by the storage stack, degraded latency windows during rebuilds, and the engineering time to own placement, upgrades and tuning. For some estates that total is comfortably below the licence it replaces. For a latency sensitive VM estate leaving vSAN with no spare platform capacity, it usually is not.
Should we use Ceph to replace vSAN?
Ceph is a good option when performance is not the main goal. If the workloads coming off vSphere are capacity oriented, or if you have a team that wants to operate storage, it is a credible and well proven choice. If the estate is latency sensitive VMs with commitments attached and the platform team is already fully committed to the migration, the efficiency and operating model differences described above will matter more than the licence line, and a managed alternative is likely to cost less in total.
Does ODF come with our OpenShift subscription?
Partly, and the distinction is expensive. ODF Essentials is included in OpenShift Platform Plus but not in OpenShift Container Platform. ODF Advanced, which carries metro synchronous replication, regional asynchronous replication and VM level disaster recovery orchestration, is a separate licence sold per two cores and is included in neither edition, with VM level DR additionally requiring RHACM. Check which edition you hold before assuming storage and DR are covered.
How do we size the destination correctly?
Take the raw to effective ratio of the candidate architecture rather than the vSAN figure you are replacing, apply it to real consumed capacity rather than the datastore used figure, and add the host CPU the storage stack will take on every workload node. Those two corrections account for most destination sizing errors we see after a vSAN exit, and both are knowable before anything is ordered.
