VM Migration: The Storage Layer

Moving virtual machines off a legacy hypervisor platform is mostly a storage exercise. Compute translates cleanly: a vCPU is a vCPU, memory is memory, and Red Hat® OpenShift® Virtualization runs the same guest operating systems through KubeVirt. Storage is where the assumptions live. Live migration depends on it, snapshot and backup regimes are built around it, sizing mistakes are buried in it, and every cutover risk in the project plan traces back to it.

This guide assumes you are doing the migration, not deciding whether to. It walks the storage work in the order a migration actually happens: measure the estate, size the target, sequence the cutover, know what breaks, keep a rollback path, and validate before declaring victory. The worked examples use round numbers so you can substitute your own; the TCO calculator does the same arithmetic interactively.

Why the storage layer decides the migration

Three properties of a VM estate make storage the hard part of any migration.

First, sizing is inferred, not known. A VM estate accumulates provisioned capacity for years: disks sized generously at creation, clones of clones, templates with padded volumes. The sum of provisioned disk sizes routinely lands at two to three times what the guests have actually written. Migrate on provisioned figures and you buy hardware you will never fill. Migrate on guesses and you undersize the target and discover it during cutover week.

Second, operational capabilities are storage capabilities. Live migration, the ability to move a running VM between hosts without downtime, only works when both hosts can access the same disk simultaneously. Snapshot-based backup only works when the storage layer produces consistent snapshots on schedule. Neither capability transfers automatically. Each must be verified on the target, and each fails quietly if it is not.

Third, cutover risk concentrates in data movement. Rehosting a VM definition is a metadata operation measured in seconds. Moving its disks is a data operation measured in hours, and it is the step that cannot be trivially undone once the source is decommissioned. The order in which you move data, and what you keep until validation passes, is the difference between a reversible step and a bet.

The rest of this guide treats each phase in sequence.

Measure the estate you actually have

The output of this phase is a per-VM inventory with four columns that matter: written capacity, peak IOPS, peak throughput, and workload class. Everything downstream is sized from this table, so resist the temptation to substitute provisioned figures.

Written capacity, not provisioned. Most hypervisor management tooling reports both the provisioned size of each virtual disk and the actual space consumed on the datastore. Export the consumed figure per disk, per VM. For thick-provisioned disks where consumed equals provisioned, check the guest: filesystem usage inside the VM is the honest number, and the gap between guest usage and datastore consumption is space you will reclaim through thin provisioning on the target.

Peak I/O, not average. Averages hide the nightly batch job and the Monday-morning login storm. Pull IOPS and throughput per VM at peak over at least two representative weeks, including a month-end if the estate runs financial workloads. The target cluster must absorb the coincident peak of each migration wave, not the average of a quiet Tuesday.

Workload class. Tag each VM as latency-sensitive (databases, message queues, anything with a p99 SLO), general purpose, or cold (rarely powered on, archival, dev machines nobody has touched in a quarter). The classes drive wave planning in step 3, and the cold class is a candidate for not migrating at all.

A minimal inventory looks like this:

Table 1: The four columns that drive every downstream decision. Provisioned capacity is recorded only to measure how much it overstates the estate.

Two practical notes. Guests without a guest agent report less accurate consumption data; install the agent fleet-wide before the measurement window, since you will need it on the target anyway for consistent snapshots. And record access mode requirements while you are in the inventory: any VM that must live-migrate needs shared access to its disk, which becomes a storage class decision in step 2.

Size the target from the working set

Sizing converts the inventory into three numbers: usable capacity, raw capacity, and node count. The arithmetic is short, and the difference between doing it from written capacity versus provisioned capacity is usually the largest single cost decision in the project.

Work through an example estate: 1,200 VMs, 2.4 PB provisioned, 800 TB written, measured coincident peak of 400,000 IOPS.

Usable capacity. Start from written capacity and add a growth allowance for the planning horizon. A 30 percent allowance over the migration-plus-one-year window puts the example at roughly 1,040 TB, call it 1 PB usable. Thin provisioning on the target means you do not pre-buy the gap between written and provisioned; guests keep their large virtual disks, and physical capacity is consumed only as they write.

Raw capacity. This is where the data protection scheme sets the hardware bill. Triple replication stores three full copies, so 1 PB usable needs 3 PB raw. A 4+2 erasure coding scheme stores four data chunks plus two parity chunks, tolerating the same two concurrent failures at 1.5× overhead: 1 PB usable needs 1.5 PB raw. Simplyblock uses erasure coding as its native protection scheme, which is why the same usable target is roughly half the drives.

Table 2: The example estate sized four ways. Written-capacity input plus erasure coding versus provisioned-capacity input plus replication is the full range between the honest number and the default mistake.

Node count. Simplyblock is hyperconverged: storage runs on the compute nodes that also run the VMs, over NVMe/TCP, so there is no separate storage tier to rack. Node count is therefore set by whichever binds first: VM density (vCPU and memory per node), aggregate NVMe capacity per node, or peak IOPS per node. Divide the coincident peak from step 1 across the candidate node count and confirm the per-node figure sits comfortably inside what the hardware delivers, then add at least one node of failure headroom per failure domain.

One design decision belongs in this step because it is expensive to change later: the storage class layout. VMs that need live migration require volumes with ReadWriteMany access in block mode. Define a storage class for that profile from day one and make it the default for VM workloads:

1
apiVersion: storage.k8s.io/v1
2
kind: StorageClass
3
metadata:
4
  name: vm-live-migratable
5
provisioner: csi.simplyblock.io
6
parameters:
7
  # shared block access for KubeVirt live migration
8
volumeBindingMode: WaitForFirstConsumer
9
allowVolumeExpansion: true

With the matching PVC profile (volumeMode: Block, accessModes: [ReadWriteMany]), every VM disk provisioned during the migration is live-migratable by construction rather than by later remediation.

Sequence the cutover in waves

The single most effective de-risking decision in a VM migration is refusing the big bang. Stand the target storage up alongside the source, move the estate in waves scoped to namespaces, and validate each wave before starting the next. Every step stays independently reversible, and the blast radius of any failure is one namespace.

Wave composition. Build waves from the workload classes in step 1:

Table 3: Wave ordering by risk. The latency-sensitive wave goes late deliberately: by then the team has runbooks, real performance data from waves 0 and 1, and a tested rollback.

The data path. Two mechanisms move disks, and most projects use both. The Migration Toolkit for Virtualization, Red Hat’s migration tooling for OpenShift Virtualization, handles the VM conversion: it reads disks from the source platform, converts them, and writes them into PersistentVolumeClaims through the Containerized Data Importer. For warm migration it replicates changed blocks in the background while the source VM keeps running, shrinking the final cutover window to minutes. Once VMs are on the target, all further movement (clones for testing, rehearsal copies, per-namespace duplicates) uses standard CSI snapshot and clone against the simplyblock storage class.

Rehearse wave 0 twice. The first run of the pipeline surfaces the boring failures: a guest agent missing, a driver the converted VM needs, a network mapping typo. Run wave 0, tear it down, run it again clean, and only then schedule wave 1. The second rehearsal is cheap; discovering the same failures during wave 2 is not.

Cutover mechanics per namespace. Freeze changes on the source VMs, run the final incremental sync, power down the source VMs, start the target VMs, and run the validation battery from step 6. The source VMs stay powered off but intact. Total downtime per VM is the final sync plus boot, which warm migration keeps in minutes.

What breaks, and the checks that catch it

The failure modes of VM migrations are unusually predictable. Each of these has a cheap check that belongs in the wave 0 and wave 1 runbooks.

Live migration silently lost. The classic failure: VMs land on volumes without shared block access, everything appears to work, and weeks later the first node drain fails because the VMs on it cannot move. The check costs one command per wave: pick a running migrated VM and drive a live migration with virtctl migrate <vm-name>, then confirm it completed and the guest never dropped a connection. If the storage class from step 2 is the enforced default, this check passes by construction; run it anyway.

Snapshot semantics differ. Hypervisor snapshots on the source were likely quiesced through platform tooling, and backup products built schedules around them. On the target, CSI snapshots are crash-consistent by default. For filesystem consistency, KubeVirt virtual machine snapshots invoke the QEMU guest agent to freeze and thaw guest filesystems around the snapshot, which is why the agent went in fleet-wide during step 1. The check: take a VM snapshot of a running database VM, restore it to a clone, and verify the database opens clean. Then re-point backup schedules at the CSI mechanism and confirm the first scheduled run completes, per namespace, before the wave is signed off.

Performance regressions from sizing shortcuts. If a workload is slower after migration, the cause is almost always discoverable in the step 1 inventory: a peak that was measured as an average, or a latency-sensitive VM misclassified as general purpose. The check is a before-and-after comparison against the inventory numbers, not against feel. Simplyblock exposes per-volume latency and IOPS metrics; alert on deviation from the recorded baseline during the first week after each wave.

Boot order and dependency surprises. Multi-VM applications encode startup dependencies that the source platform satisfied by accident. Document dependency order per namespace during assessment, and start target VMs in that order during cutover. This is a checklist discipline, not a storage feature, but it fails during the storage cutover window and gets blamed on storage.

Keep a rollback path until the end

A rollback plan you cannot execute is a slide, not a plan. The per-namespace structure makes rollback concrete and cheap enough to keep until the very end.

The rule: source VMs are powered off at cutover, never deleted, until their namespace passes validation. Rollback is then mechanical: stop the target VMs, power the source VMs back on, and re-point DNS or the load balancer. Nothing needs restoring from backup because nothing was destroyed.

Scope the gates explicitly:

• Rollback window. Define per wave, typically one to two weeks of production traffic on the target. Inside the window, rollback is the default response to any unexplained regression.
• Decommission gate. Source capacity for a namespace is reclaimed only after its window closes with validation green. The temptation to reclaim source hardware early is real, especially when the source platform bills per host; resist it for the active wave at minimum.
• Data written after cutover. The one asymmetry in rollback: writes on the target after cutover do not exist on the source. For most namespaces the answer is accepting the window’s data loss on rollback, documented and signed off in advance. For the few systems where that is unacceptable, keep an export path (database-level replication back to the source) alive during the window.

Because each wave is scoped to a namespace, a rollback never puts more than one wave at risk, and waves already validated stay migrated.

Validate, then plan headroom

Validation is a battery run per namespace, against recorded numbers, before the wave is declared done. The battery from the earlier steps, collected in one place:

1. Live migration: one virtctl migrate per wave completes with no guest disruption.
2. Snapshot and restore: a guest-agent-frozen snapshot of a running stateful VM restores to a clone that opens clean.
3. Backup: the first scheduled backup run against CSI snapshots completes and a test restore succeeds.
4. Performance: per-volume latency and IOPS match the step 1 baseline through at least one business peak.
5. Dependencies: a full namespace stop-and-start in documented order comes up without manual intervention.

When the last wave passes, the migration is done but the sizing work has one step left: headroom policy. Thin provisioning means physical consumption grows as guests write, not when disks are created, so capacity planning becomes a consumption-rate exercise. Set an alert threshold on pool utilization, and size the expansion increment (drives or nodes) to arrive comfortably ahead of the trend line. Simplyblock expands capacity online, so growth after the migration is an operational task rather than a project.

The estate is now on storage that was sized from measured reality, protected by erasure coding at 1.5× raw instead of 3×, with live migration and consistent snapshots verified rather than assumed. Just as importantly, the organization now has a rehearsed, reversible pattern for moving VM workloads, which is worth keeping in the runbook long after the source platform is gone.