Firecracker on nested KVM: 1.38 s median to a healthy agent
305 production cold boots on nested KVM, and the guest clock fix when two documented methods fail.
On September 25, 2026, plori started to run new agent sessions in Firecracker microVMs. Each node is itself a KVM guest, so the microVMs use nested KVM. These numbers cover the period to September 27.
The short answer
- 305 cold boots: the median time from create request to a healthy agent process was 1.38 s and the 95th percentile was 1.76 s. Firecracker itself configured and started each guest in a median of 10 ms.
- Guest clock:
clock_realtimeand KVM PTP both fail on our nodes, so guests use NTP with the node as time server. A guest restored from a 120 s old snapshot read 120 s behind for the first 3 s, and had offset 0 s at 5 s. - Memory: at an overcommit ratio of 1.5, a node holds nine sessions instead of six.
Production boot phases, 305 cold boots
| Phase | Median | 95th percentile |
|---|---|---|
| create | 19 ms | 32 ms |
| boot | 10 ms | 18 ms |
| health | 1.34 s | 1.72 s |
| total | 1.38 s | 1.76 s |
When the answer changes
- Your host clock source: both clock methods need a TSC-based clock source on the host. Read
/sys/devices/system/clocksource/clocksource0/current_clocksource. Withtscboth can work. Withkvm-clock, as on our nodes, serve NTP from the node. The problem is not certain on every nested host. - Concurrency: at most five VMs were in use at the same time in the week measured. We have not measured boot time under higher concurrency.
- Not proven in production: snapshot restore and memory pressure handling ran only on test nodes. Production ran at ratio 1.5 from September 26, and the memory pressure loop took zero steps.
The full post has the jailer and reflink root disks, the per-VM network rules, and the memory admission settings.