Appendix C: KVM ioctl Reference

KVM's userspace API is organized by file descriptor. The same numeric command space is used for system, VM, vCPU, and device operations, so an ioctl name is not complete documentation until its target fd and capability requirements are known.

This appendix indexes the ioctls used by the book. Chapter 5 develops the three-fd model, Chapter 6 covers memory slots and dirty tracking, Chapter 7 covers interrupts and time, and Chapter 8 covers exits.

Opening /dev/kvm and issuing these ioctls requires a bare-metal Linux host or a VM with nested virtualization. Treat experiments as isolated lab work. The tables are an API reference, not commands to run on a workstation.

Read The Encoding Carefully

KVM defines KVMIO as 0xAE. On architectures using the generic ioctl layout, _IOC(dir, type, nr, size) packs the command number like this:

The macro names describe data movement from the kernel's point of view: _IOW copies input from userspace, _IOR copies output to userspace, and _IOWR does both. Historical exceptions make those direction bits unreliable as semantic documentation. For example, KVM_GET_DIRTY_LOG is _IOW because its top-level struct contains a userspace pointer, KVM_GET_ONE_REG is also _IOW, and KVM_SET_IRQCHIP is spelled _IOR in the UAPI header. Use the published macro, not a reconstructed "corrected" encoding.

_IO commands have no encoded struct size, so their generic value is simply 0x0000AE00 | nr. Other raw values depend on the argument layout and ioctl encoding of the build architecture. The operation number and macro are the portable reference.

Follow The File Descriptors

VM ioctls must be issued from the process address space that created the VM. The KVM documentation says vCPU ioctls should remain on the thread that created the vCPU; moving them can impose a first-call performance cost. Device ioctls must remain in the VM creator's address space. A few ioctls deliberately accept more than one fd class, so check each entry rather than assuming that a command number uniquely determines scope.

System Ioctls

These target the fd returned by open("/dev/kvm", O_RDWR).

Reject an API version other than 12. Extensions evolve through KVM_CHECK_EXTENSION, not by incrementing that base version. Most capability queries return zero or one, but some return a count, size, bitmask, or limit. When KVM_CAP_CHECK_EXTENSION_VM is available, the VM-fd form can report the capabilities of that configured VM and is preferred where the distinction matters.

KVM_GET_VCPU_MMAP_SIZE supplies the length for mapping offset zero of every vCPU fd. Do not substitute sizeof(struct kvm_run). The returned layout may also account for the historical coalesced-MMIO page and dirty-ring pages.

VM Ioctls

These target the fd returned by KVM_CREATE_VM unless noted otherwise.

KVM_CREATE_VCPU accepts an ID in [0, max_vcpu_id). Query KVM_CAP_NR_VCPUS for the recommended count, KVM_CAP_MAX_VCPUS for the hard count limit, and KVM_CAP_MAX_VCPU_ID for the ID-space limit. Count and ID are separate constraints.

On x86, KVM_CREATE_IRQCHIP creates two PICs and an IOAPIC, and arranges for future vCPUs to receive local APICs. Order therefore matters: call it before creating vCPUs when the machine design requires in-kernel LAPICs. On arm64 the ioctl creates a GICv2; KVM_CREATE_DEVICE is the preferred interface and is required for other GIC versions such as GICv3.

KVM_SET_TSS_ADDR is an x86 Intel-host requirement. Its three-page guest physical range must remain below 4 GiB and must not overlap RAM or MMIO. The guest must not use it. This is not a guest task-state segment supplied by the guest OS; it is KVM's VMX compatibility reservation.

Memory And Dirty Tracking

The basic memory-slot struct connects a guest-physical range to an existing userspace virtual range:

struct kvm_userspace_memory_region {
    __u32 slot;
    __u32 flags;
    __u64 guest_phys_addr;
    __u64 memory_size;
    __u64 userspace_addr;
};

Slot IDs occupy bits 0-15; with KVM_CAP_MULTI_ADDRESS_SPACE, bits 16-31 hold the address-space ID. Slots cannot overlap within one address space. Setting memory_size to zero deletes a slot. An existing slot can move or change flags but cannot be resized in place. KVM recommends matching the low 21 bits of guest_phys_addr and userspace_addr so guest large pages can be backed by host large pages.

KVM_MEM_LOG_DIRTY_PAGES enables bitmap tracking. By default, KVM_GET_DIRTY_LOG returns dirties since the preceding call and clears the bits before returning. With KVM_CAP_MANUAL_DIRTY_LOG_PROTECT2, userspace instead clears and reprotects selected ranges through KVM_CLEAR_DIRTY_LOG. The dirty-ring interface is a separate per-vCPU producer path; after harvesting entries, userspace marks them reset and calls KVM_RESET_DIRTY_RINGS.

KVM_SET_USER_MEMORY_REGION2 adds guest_memfd and offset fields. The KVM_MEM_GUEST_MEMFD flag belongs to that extended struct, not the legacy one. Its capability and memory-attribute contracts are part of confidential-guest support and must be negotiated together.

Interrupt Eventfds

KVM_IOEVENTFD and KVM_IRQFD remove the userspace vCPU loop from common notification paths:

guest write -> KVM_IOEVENTFD -> eventfd -> device backend
device event -> eventfd -> KVM_IRQFD -> routed guest interrupt

An ioeventfd registration identifies a PIO or MMIO address, optional data match, access length, and eventfd. A matching guest write signals the eventfd instead of producing a userspace exit. With KVM_CAP_IOEVENTFD_ANY_LENGTH, a zero length ignores the guest write width.

An irqfd registration binds an eventfd to a GSI. KVM_SET_GSI_ROUTING maps that GSI to an irqchip pin or MSI route. With KVM_CAP_IRQFD_RESAMPLE, a second eventfd reports deassertion after a level-triggered interrupt is resampled; the device backend must requeue the interrupt if its condition remains true.

vCPU Ioctls

These target a vCPU fd.

Snapshot implementations also use architecture-specific state families such as XSAVE/XCRS, debug registers, PIT state, irqchip state, and device attributes. The pair must be restored in the ordering required by each ioctl; a bag of structs is not a portable snapshot format.

With KVM_CAP_VM_TSC_CONTROL, the TSC-frequency pair also accepts a VM fd. The set operation then establishes the initial frequency for subsequently created vCPUs and must run before vCPU creation. Otherwise the pair uses the vCPU fd under its vCPU capabilities.

Set the vCPU model before first run. In particular, KVM warns that changing CPUID with KVM_SET_CPUID2 after KVM_RUN may destabilize the guest. Direct kernel boot must also establish the architecture's entry register state rather than assuming firmware initialized it.

The KVM_RUN Contract

KVM_RUN has no explicit argument. It exchanges inputs and outputs through the mapping at offset zero of the vCPU fd:

struct kvm_run {
    __u8  request_interrupt_window;       /* in */
    __u8  immediate_exit;                 /* in */
    /* ... */
    __u32 exit_reason;                    /* out after return value 0 */
    __u8  ready_for_interrupt_injection;  /* out */
    __u8  if_flag;                        /* out, conditionally valid */
    /* exit-specific union follows */
};

A return value of zero makes exit_reason and its matching union member valid. A return of -1 is an error path; for example, an unmasked pending signal produces EINTR. Do not dispatch a stale exit_reason after an error.

immediate_exit is checked once when a KVM_RUN begins. Setting it alone does not eject a vCPU already executing. The scalable kick pattern sends a signal to interrupt the vCPU thread and sets immediate_exit in the signal handler so a racing re-entry returns EINTR instead of running again. It requires KVM_CAP_IMMEDIATE_EXIT and correct cross-thread memory ordering.

For exits that carry an unfinished read or write, re-entering through KVM_RUN completes the operation. This matters for KVM_EXIT_IO and KVM_EXIT_MMIO: userspace must consume write data or provide read data before the next call. See Chapter 8 for the exit-specific union fields.

Capability Checks

Probe the operation, not the kernel version. The capability families used most often in this book are:

Do not treat every positive result as Boolean. Read the capability's API entry to learn whether the result is a count, size, bitmask, or versioned mode, and whether it must also be enabled with KVM_ENABLE_CAP.

Sources

• KVM API documentation
• Generic ioctl encoding
• KVM common UAPI header
• KVM x86 UAPI header
• ioctl(2)
• Appendix research note