Chris Engelbert 
KubeletHasDiskPressure is one of those conditions that always has an obvious answer and never has a permanent one. A node crosses a threshold, the kubelet starts evicting pods, someone prunes unused images or truncates a runaway log file, the condition clears, and everyone moves on. Two weeks later it happens on a different node, and the same runbook gets executed again.
The reason the loop repeats is that the usual fixes treat disk pressure as a housekeeping failure, when the recurring version of it is a design decision. The kubelet’s eviction thresholds are measured against node-local filesystems, so anything a workload writes to node-local disk, container logs, the writable container layer, emptyDir, becomes an input to whether that workload survives on that node. If your application’s working state lives there, pod survival is a function of node disk capacity, and you will be back at the runbook every time the cluster gets denser.
This post covers what the kubelet actually measures, why reclaim only buys time, and how moving workload state off the node removes it from the eviction calculus entirely.
What Kubelet Actually Measures When It Reports DiskPressure
The kubelet monitors a fixed set of eviction signals on each node and compares them against configured thresholds. Four of those signals raise the DiskPressure node condition:
| Signal | What it describes | Common default hard threshold |
|---|---|---|
nodefs.available | Free space on the filesystem the kubelet uses for emptyDir volumes, container logs, and its own working directory | < 10% |
nodefs.inodesFree | Free inodes on that same filesystem | < 5% |
imagefs.available | Free space on the filesystem holding container images and writable container layers, when it is separate from nodefs | < 15% |
imagefs.inodesFree | Free inodes on the image filesystem | < 5% |
Table 1: The four eviction signals behind the DiskPressure node condition. Thresholds are configurable per kubelet and vary by distribution, so check your own --eviction-hard values rather than assuming the defaults.
Two details in that table explain most of the confusion around disk pressure. The first is that nodefs is not “the disk,” it is a specific filesystem that also happens to hold your emptyDir volumes and every container’s stdout log. The second is that inode exhaustion triggers the condition just as readily as byte exhaustion, which is why a node can report disk pressure while df -h looks comfortable. A workload writing millions of tiny files will hit nodefs.inodesFree long before it hits nodefs.available.
When a threshold is crossed, three things happen roughly at once. The node condition flips to DiskPressure=True, the node picks up the node.kubernetes.io/disk-pressure:NoSchedule taint so the scheduler stops placing new pods on it, and the kubelet begins reclaiming. Only if reclaim is insufficient does it start ranking pods for eviction. That ranking is not random: pods whose local ephemeral storage usage exceeds their requests go first, then pod priority breaks ties, then usage relative to requests. A pod with no ephemeral-storage request at all is treated as exceeding it immediately, which is why the workloads that get evicted are so often the ones nobody wrote resource requests for.
Worth separating from all of the above: a pod that exceeds its own ephemeral-storage limit is evicted regardless of node conditions. That is a per-pod limit check, not node-pressure eviction, and it produces a very similar-looking Evicted status with a different cause.
Why Reclaim and Pruning Only Buy You Time
Before evicting anything, the kubelet tries to free space itself. It garbage-collects terminated pods and dead containers, and when the imagefs signal is the one in breach, it deletes unused images oldest-first. This is the same work an operator does by hand during an incident, which is why manual pruning feels effective: you are racing the kubelet to do its own job.
Reclaim works on exactly one category of data, though: things the kubelet is allowed to throw away. It cannot reclaim an emptyDir volume that a running pod still has mounted. It cannot reclaim a writable container layer that an application is actively writing into. It cannot shrink a log file that a container is still appending to at the rate it chose. When the growth is coming from any of those three, reclaim frees a few gigabytes of image cache, the condition clears for a while, and the underlying consumer keeps growing at exactly the rate it was before.
That is the shape of the recurring case. A one-off disk pressure event usually is housekeeping: an image cache that nobody had ever pruned, a debug log left at verbose. The version that comes back every few weeks is almost always a workload keeping real state on node-local disk, and the honest fixes for that are limited. You can raise the eviction threshold, which trades pod stability for a smaller safety margin before the node itself gets into trouble. You can overprovision the node’s disk, which means every node in the pool carries headroom sized for the worst-case workload that might land on it. You can set ephemeral-storage requests and limits, which is genuinely worth doing because it makes the eviction ranking predictable and stops one greedy pod from taking down its neighbours, but it converts a node-level surprise into a pod-level one rather than removing it.
None of those change the underlying coupling: the workload’s state and the node’s operating margin are stored on the same filesystem, and the kubelet is measuring that filesystem to decide whether the workload gets to keep running.
If DiskPressure keeps coming back on different nodes, the problem is not the runbook. Talk to us about which of your workloads still keep working state on node-local disk, and what it takes to move them off. Talk to a storage architect
Where Workload State Lives, and What That Costs You
The practical question is not “how do I clear the alert” but “which of these five places is my data in, and what does each one cost me when a node gets tight, drained, or lost.”
Table 2: Where workload state can live on a Kubernetes node, and how each option behaves under disk pressure and node lifecycle events.
| Where state lives | Counts toward the DiskPressure threshold | Survives node drain | Survives node failure | Recovery action | How you add capacity |
|---|---|---|---|---|---|
| Container logs and writable layer | Yes, on nodefs or imagefs | No | No | Rotate logs, prune images, rebuild | Bigger disk on every node |
emptyDir | Yes, on nodefs | No, deleted with the pod | No | Rebuild from source, cold start | Bigger disk on every node |
hostPath | Yes, if it sits on nodefs | No, node-bound | No | Manual, node-specific | Bigger disk on that node |
| Local persistent volume | Yes, it consumes node disk | Only if rescheduled to the same node | No | Restore from backup or replica | Bigger disk on every candidate node |
| simplyblock NVMe/TCP PVC | No, the volume is not on node filesystems | Yes | Yes | Volume reattach, seconds | Grow the shared pool once |
The last row is the only one where the capacity question is answered once for the cluster rather than once per node. That difference compounds: headroom kept on node-local disk to stay clear of an eviction threshold is headroom you buy on every node in the pool, whether or not the workload that needs it is ever scheduled there.
Taking Workload State Out of the Eviction Calculus
A persistent volume attached over a storage fabric does not live on nodefs or imagefs, so the kubelet does not count it when evaluating disk pressure. That is not a tuning trick, it is a consequence of where the bytes physically are: on a shared NVMe/TCP pool reached over standard Ethernet, not on the node’s root disk. Move a workload’s data directory, spill files, or local cache from emptyDir onto a persistent volume claim and that workload stops contributing to the node’s eviction signals altogether.
What is left on node-local disk is what should have been there in the first place: the image cache, container logs, and genuinely disposable scratch. Those grow at rates you can bound with log rotation, image garbage collection, and ephemeral-storage limits, which makes node disk sizing a capacity-planning exercise rather than a recurring incident.
The second half of the argument is what happens during the events that surround disk pressure. Because a node under DiskPressure is tainted NoSchedule, the usual response is to drain it, and draining is where the difference between storage models becomes visible. Workloads on node-local disk have to be rebuilt somewhere else, which is exactly when a cold cache or a full resync shows up as an application-level outage. With simplyblock, the CSI driver detaches the volume from the drained node and reattaches it wherever the pod is rescheduled, typically in seconds, with the data intact. That reattach path is also where a generic CSI driver alone stops being enough: the interface describes the operation, but how long it takes is a property of the storage layer underneath it. The failure domain of the node and the failure domain of the data are no longer the same thing.
That separation is also what makes the capacity story different. Simplyblock scales compute and storage independently, so the answer to “we need more storage headroom” is to grow the pool, not to grow the root disk on every node in the cluster. Simplyblock is priced like the platform it runs on: by the CPU capacity of your worker nodes, counted the same way your OpenShift or Kubernetes subscription counts them, and data volume does not change the price. And because the storage layer is not tied to a platform, the same volumes work across bare-metal Kubernetes, Red Hat® OpenShift®, KubeVirt, and traditional hypervisors, in either hyperconverged or disaggregated topologies.
To be clear about what this does and does not solve: moving state to network-attached volumes removes that state from the eviction signals, but it does not fix a container logging at 200 MB an hour, and it does not excuse skipping ephemeral-storage requests. Do both. The point is that once workload state is off the node, the remaining consumers of node disk are ones you control directly, and disk pressure goes back to being a capacity number you set rather than an alert you answer.
Questions and Answers
What does DiskPressure mean in Kubernetes?
DiskPressure is a node condition the kubelet sets when free space or free inodes on the node’s filesystems fall below configured eviction thresholds. It applies to nodefs (which holds emptyDir volumes, container logs, and kubelet working files) and imagefs (container images and writable layers). When the condition is true, the node is tainted NoSchedule so no new pods land on it, and the kubelet begins reclaiming resources and then evicting pods.
Why do my pods keep getting evicted with disk pressure even after I prune images?
Because image pruning only reclaims data the kubelet is allowed to delete. It cannot reclaim an emptyDir volume that a running pod has mounted, a writable container layer an application is actively writing to, or a log file a container is still appending to. If the growth is coming from any of those, pruning frees the image cache, the condition clears temporarily, and the real consumer keeps growing at the same rate.
Which pods does the kubelet evict first under disk pressure?
The kubelet ranks pods by whether their local ephemeral storage usage exceeds their requests, then by pod priority, then by usage relative to requests. A pod with no ephemeral-storage request is treated as exceeding it immediately, which is why workloads without resource requests are usually the first to go. Setting ephemeral-storage requests and limits is the most direct way to make that ranking predictable.
Do persistent volume claims count toward the disk pressure eviction threshold?
No. The kubelet’s disk eviction signals measure the node’s own filesystems, and a volume attached over a storage fabric does not sit on them. That is the architectural reason moving workload state from emptyDir onto a network-attached PVC removes the workload from the eviction calculus, rather than just giving it more room before the same threshold is crossed.
How does simplyblock stop disk pressure from recurring?
Simplyblock keeps volumes on a shared NVMe/TCP pool instead of node-local disk, so workload data is not measured by the kubelet’s eviction signals, and node disk goes back to serving only the image cache, logs, and disposable scratch. When a node under disk pressure is drained, the CSI driver reattaches the volume to the rescheduled pod in seconds rather than forcing a rebuild, and storage headroom is added once at the pool instead of on every node in the cluster.
Can I just raise the eviction threshold instead?
You can, and for a genuinely one-off event that is a reasonable call. The tradeoff is that the eviction threshold is the safety margin protecting the node itself: lower it too far and instead of losing a pod you risk the kubelet, the container runtime, or the node’s own logging running out of space. Raising the threshold buys room, it does not decouple your workload’s survival from your node’s disk.
Why does a node report disk pressure when there is still free space?
Almost always inode exhaustion. nodefs.inodesFree and imagefs.inodesFree raise the same DiskPressure condition as the byte-based signals, so a workload writing very large numbers of small files can trigger eviction while df -h still shows capacity available. Check df -i alongside df -h when the byte numbers look fine.
