MicroVMs From The Bottom Up

Chapter 22: MicroVMs As Containers

Every runtime that runs a container on Linux eventually calls execve. The process on the other end of that call is still subject to the same host kernel, the same syscall table, and the same kernel bugs as everything else on the node. The namespace and cgroup boundaries are real, but they are software: a kernel vulnerability can walk past them. The microVM exists precisely to put a hardware boundary between the workload and the host — a second kernel, a separate address space enforced by EPT rather than by Linux's own memory management. The question this chapter answers is what happens when you want both: the operational interface of a container — the OCI image, the CRI API, the Kubernetes pod spec — wrapped around the isolation of a virtual machine.

Three projects answer that question in different ways. firecracker-containerd replaces the runC shim with a Firecracker VMM while keeping the containerd daemon and its entire API surface intact. Kata Containers does the same at the CRI level, routing every Kubernetes pod into a dedicated VM, transparent to kubelet. flintlock takes the idea one level up: instead of running a container inside a microVM, it provisions a microVM that is a Kubernetes node, managed through an OCI-friendly API and controlled by a Cluster API provider. The first two care about what runs inside the VM; the third cares about the VM as the unit of infrastructure.

The place where all three projects touch the containerd book is the containerd shim v2 protocol. Understanding that handoff is the key to understanding everything else.

The Shim v2 Protocol As The Pivot Point

Three radically different systems — runC containers, Firecracker microVMs, and QEMU-backed Kata pods — appear identical to containerd above a single protocol boundary. That boundary is the shim v2 protocol, and understanding it is what makes the rest of this chapter legible.

When containerd starts a task, it does not call runC directly. It finds a shim binary, forks it, and from that point forward speaks the containerd.task.v2.Task ttrpc service defined in api/runtime/task/v2/shim.proto. The shim is responsible for everything below that service boundary: creating the container, managing its lifecycle, mounting its rootfs, and forwarding stdio. Containerd does not care what the shim actually does. It calls Create, Start, Kill, and Delete over ttrpc and trusts the shim to make it happen.

The runtime name in the container spec determines which shim binary runs. Containerd takes the last two dot-separated components of the runtime name, replaces every . with - within those components, and prepends containerd-shim-. The runtime name io.containerd.runc.v2 yields containerd-shim-runc-v2; aws.firecracker yields containerd-shim-aws-firecracker; io.containerd.kata.v2 yields containerd-shim-kata-v2. Starting with containerd 1.6.0, the binary path can be given directly instead of relying on the derivation.

The shim service has seventeen RPC methods: State, Create, Start, Delete, Pids, Pause, Resume, Checkpoint, Kill, Exec, ResizePty, CloseIO, Update, Wait, Stats, Connect, and Shutdown. The shim must publish four events in strict order — TaskCreateEventTopic, TaskStartEventTopic, TaskExitEventTopic, TaskDeleteEventTopic — and is responsible for mounting the container rootfs into the rootfs/ subdirectory of the OCI bundle path it receives in CreateTaskRequest. In containerd 2.3 and later, the shim reads a protobuf BootstrapParams message from stdin and writes a BootstrapResult to stdout. Legacy shims write a JSON object like {"version": 2, "address": "/path", "protocol": "grpc"} to stdout.

Everything above the shim — container creation, image pulling, snapshot management, event routing, the CRI API — is identical whether the shim is containerd-shim-runc-v2 or containerd-shim-aws-firecracker. The hardware boundary lives entirely below the seventeen ttrpc methods.

firecracker-containerd

One Shim, One VM, Many Containers

The naive mapping from containerd's task model to Firecracker would fork one VMM per container. That is the wrong answer: each Firecracker process carries its own memory overhead, its own guest kernel boot, and its own vCPU threads. firecracker-containerd instead multiplexes: one shim process manages all containers within a single microVM. The first Create call for a given VM ID starts the shim and boots the VMM; subsequent tasks in the same VM reuse the running instance. Containerd locates an existing shim for a VM via pid and socket files at /var/lib/firecracker-containerd/shim-base/<vm_id>/. If the client supplies no VM ID, the shim generates a UUID v4, giving the default topology of one container per VM.

The project ships four components: the control plugin (compiled directly into a specialized containerd binary — not a standalone process or a socket-based plugin), the host-side runtime shim (containerd-shim-aws-firecracker), the in-VM agent, and a root filesystem image builder. The control plugin manages microVM lifecycle and exposes an API modeled on the Firecracker lifecycle, defined in proto/firecracker.proto. Its configuration types — FirecrackerMachineConfiguration (fields: CPUTemplate, HtEnabled, MemSizeMib, VcpuCount), CNIConfiguration (NetworkName, InterfaceName, BinPath, ConfDir, CacheDir, Args), and FirecrackerDriveMount (HostPath, VMPath, FilesystemType, Options, RateLimiter, IsWritable, CacheType) — let the caller declare the VM's hardware before any container is created.

flowchart TB subgraph host["Host process tree"] ct["containerd (with control plugin)"] shim["containerd-shim-aws-firecracker"] fc["firecracker VMM"] ct -->|"TaskService ttrpc"| shim shim -->|"Firecracker REST API"| fc end subgraph vm["MicroVM (guest kernel)"] agent["in-VM agent"] runc1["containerd-shim-runc-v1 (container A)"] runc2["containerd-shim-runc-v1 (container B)"] agent --> runc1 agent --> runc2 end shim -->|"ttrpc over AF_VSOCK port 10789"| agent

VSOCK: The Control Plane Across The Hardware Boundary

The host shim connects to the in-VM agent over AF_VSOCK using the ttrpc protocol. Two port constants govern this, both confirmed in runtime/service.go: defaultVsockPort is 10789, used for the ttrpc control channel, and minVsockIOPort is 11000 (a uint32), used as the base port for stdio multiplexing. Stdio for each container in the VM occupies three consecutive ports starting at minVsockIOPort: stdin on port 11000, stdout on 11001, stderr on 11002; a second container gets 11003–11005, and so on.

The VSOCK device in the Firecracker Go SDK is configured with ID: "agent_api", a GuestCid cast from uint32 to int64, and a UdsPath pointing to a host-side Unix domain socket. Firecracker's VSOCK multiplexing protocol adds one indirection: to reach a given port in the guest, the host connects to the UDS path and sends the string "CONNECT <port_num>\n"; Firecracker responds with "OK <assigned_hostside_port>\n". Guest-to-host connections work in the opposite direction: the guest targets CID 2 (the host CID), and Firecracker forwards the connection to <uds_path>_<port_number> on the host.

Block Devices, Not Filesystems

The snapshotter is an out-of-process gRPC proxy plugin that implements containerd's snapshotter API. firecracker-containerd ships two implementations. The naive snapshotter does a full file copy per snapshot — useful as a proof of concept but produces no deduplication. The devmapper snapshotter uses device-mapper thin provisioning for copy-on-write layering, matching what production systems need. The thin pool is named fc-dev-thinpool with base_image_size = "10GB" and metadata rooted at /var/lib/firecracker-containerd/snapshotter/devmapper. Containerd identifies it as io.containerd.snapshotter.v1.devmapper in config.toml.

The important design choice is that snapshots are exposed to Firecracker as block devices. There is no virtiofs, no 9p, no filesystem-level sharing across the hardware boundary. The host exports a device-mapper thin volume; the guest kernel mounts it directly. This is a deliberate constraint: virtiofs requires a shared memory region and a dedicated virtiofsd process, introducing host-kernel surface area that firecracker-containerd's threat model rules out.

Firecracker has no hot-plug for block devices. Every drive must be declared before the VM boots. The runtime solves this with drive stub pre-allocation: before boot it attaches sparse stub files (minimum 128 bytes) or /dev/null aliases to reserve drive IDs. When a container is created and its snapshot is ready, the runtime calls PATCH /drives/{drive_id} — the Firecracker REST endpoint PatchGuestDriveByID — to swap the stub for the real block device. The mountDrives function at around line 821 of runtime/service.go iterates s.driveMountStubs and calls PatchAndMount on each before the in-VM agent becomes available. Inside the VM, the agent correlates drives to containers either by position (lsblk sorted by MAJOR:MINOR) or by content (a drive ID written into the stub file that persists on the real device).

The Network: tc-redirect-tap

Network configuration in firecracker-containerd happens at VM-creation time, not per-container. The caller passes a CNIConfiguration to CreateVM; the runtime runs a CNI chain of three plugins. First, ptp creates a veth pair inside a dedicated network namespace. Second, host-local handles IPAM. Third, tc-redirect-tap — a custom Firecracker plugin — creates a TAP device inside that same network namespace and installs Linux TC (Traffic Control) U32 filters to redirect traffic bidirectionally between the TAP and the veth at the ingress/egress filter level.

This is the same TAP-plus-TC pattern Chapter 21 describes for raw Firecracker networking, but the CNI chain wraps it so the IP address and network plumbing come from the standard CNI IPAM path rather than being hardcoded. The Firecracker Go SDK creates and manages the network namespace, invokes CNI within it, starts the VMM inside that namespace, and handles CNI teardown on VM termination. Inside the guest, the IP and DNS arrive via Linux kernel boot parameters; the guest /etc/resolv.conf is symlinked to /proc/net/pnp. The sample network is named "fcnet" with interface name "veth0"; CNI config lives under /etc/cni/conf.d and plugin binaries under /opt/cni/bin.

The In-VM Agent And Boot Args

The in-VM agent must be embedded in the root filesystem image and configured to start on boot. Once running, it accepts control instructions from the host shim over the VSOCK ttrpc channel, invokes standard Linux containers via containerd-shim-runc-v1 (runC v1, not v2) inside the VM, emits events and metrics back to the shim, and proxies stdio over the per-container VSOCK port triples. The production kernel command line used with firecracker-containerd is:

console=ttyS0 noapic reboot=k panic=1 pci=off nomodules ro systemd.unified_cgroup_hierarchy=0 systemd.journald.forward_to_console systemd.unit=firecracker.target init=/sbin/overlay-init

The init=/sbin/overlay-init argument indicates a custom init binary that handles the overlay filesystem setup before handing control to the agent — the agent may be overlay-init itself or a child it launches. The pci=off nomodules arguments reflect Firecracker's stripped device model: without PCI and with a pre-configured driver set baked into the kernel, module loading is neither needed nor possible.

The containerd socket for firecracker-containerd lives at /run/firecracker-containerd/containerd.sock, with state at /run/firecracker-containerd and content root at /var/lib/firecracker-containerd/containerd. The CRI plugin is explicitly disabled in config.toml: disabled_plugins = ["io.containerd.grpc.v1.cri"]. This is not the containerd instance kubelet talks to; it is a dedicated daemon for the Firecracker integration.

Kata Containers

The Same Shim Protocol, A Different VM Strategy

Kata Containers ships as a single binary, containerd-shim-kata-v2, derived from the runtime handler string io.containerd.kata.v2 by the same naming convention. Like firecracker-containerd, one shim instance manages all containers in a single pod's VM — there is no per-container shim fork. Unlike firecracker-containerd, Kata targets the Kubernetes CRI path directly. Kubernetes RuntimeClass (stable since Kubernetes 1.12) routes pods to containerd-shim-kata-v2 via runtimeClassName: kata in the pod spec. The CRI runtime signals sandbox versus container membership to the shim via an OCI annotation: io.kubernetes.cri.container-type when containerd is the CRI runtime (the primary path in this book), or io.kubernetes.cri-o.ContainerType when CRI-O is. Both carry values sandbox or container.

The mapping is clean: each Kubernetes pod becomes one Kata VM, and each container in that pod becomes one process inside that VM. The API boundary above the shim — the CRI RunPodSandbox and CreateContainer calls — remains identical to what kubelet sends for a runC pod. The VM appears nowhere in the Kubernetes API.

kata-agent: Rust, PID 1, 43 Methods

The in-VM agent for Kata, kata-agent, has been written in Rust since Kata 2.0. It runs as a long-running process inside the VM and is responsible for the full container lifecycle within that VM. When the guest image is an initrd, kata-agent runs as PID 1 at /sbin/init, built with AGENT_INIT=yes. When the guest image is a rootfs, systemd starts as PID 1 and launches kata-agent as a systemd service. The agent listens on vsock address vsock://-1:1024 (VMADDR_CID_ANY on port 1024), accepting connections regardless of the guest CID the hypervisor assigns. A separate configurable vsock port (the agent.log_vport option) is reserved for log forwarding; it defaults to 0 and must be set explicitly.

The agent exposes the AgentService ttrpc service defined in src/libs/protocols/protos/agent.proto (package grpc). In the current main branch it has 43 RPC methods. The categories span sandbox lifecycle (CreateSandbox, DestroySandbox), container lifecycle (CreateContainer, StartContainer, StatsContainer, and four more), process operations (ExecProcess, SignalProcess, WaitProcess), I/O, networking (UpdateInterface, UpdateRoutes, GetIPTables, SetIPTables, and three more), storage (GetVolumeStats, ResizeVolume, AddSwap), guest introspection (GetGuestDetails, GetMetrics, GetOOMEvent), hotplug (OnlineCPUMem, MemHotplugByProbe), and policy.

The breadth reflects Kata's broader scope relative to firecracker-containerd: interface updates, routing changes, iptables management, and memory hotplug are all in-VM operations the shim must be able to drive without touching the host kernel.

Before vsock, Kata used a virtio serial port for agent communication and a separate kata-proxy process on the host for mux/demux. Vsock eliminated the proxy. The vhost_vsock kernel module must be loaded on the host (CONFIG_VHOST_VSOCK=y; sudo modprobe -i vhost_vsock); the kernel requirement is Linux 4.8 or later.

Before running the above command: loading vhost_vsock is a host-kernel operation requiring root. Verify the module is available with modinfo vhost_vsock before attempting to load it. On shared hosts or cloud VMs without nested virtualization, the module may not be present.

OCI Bundle Delivery Over the Wire

With runC, delivering a container's rootfs to the container runtime is a local filesystem operation: the shim mounts the snapshotter's device into the bundle's rootfs/ directory before calling runc create. With Kata, there is a hardware boundary between the shim and the container, so the bundle cannot be mounted locally and then used. Kata handles this in two steps.

The OCI config.json is transmitted to kata-agent via ttrpc — specifically a CreateContainerRequest over vsock — rather than via a filesystem path inside the VM. The container rootfs travels by one of two mechanisms depending on configuration. The default is virtiofs: the shim starts a virtiofsd daemon on the host (one per VM), exports the snapshotter directory, and the agent mounts the exported directory as the container rootfs inside the VM. The alternative is virtio-scsi: when block-based graph drivers are configured, the block device is hot-plugged into the VM as a virtio-scsi device, appearing as /dev/sda or a similar SCSI path. A distinct case applies when using the devmapper snapshotter with Kata-on-Firecracker: there the block device is passed as a virtio-blk drive, visible inside the VM as /dev/vda.

The guest OS mini-image — the kernel and minimal userspace that runs inside each Kata VM — is mounted via DAX to avoid double-caching its pages in both the host and guest page caches. QEMU uses an NVDIMM device backed by a file (the guest sees it as /dev/pmem*); Cloud Hypervisor uses an emulated Persistent Memory (PMEM) device; both mount as ext4 inside the VM. virtio-9p is a supported but non-default alternative to virtiofs for rootfs sharing.

sequenceDiagram
    participant K as kubelet
    participant C as "containerd CRI"
    participant S as "containerd-shim-kata-v2"
    participant V as virtiofsd
    participant A as kata-agent
    participant R as "runc (inside VM)"

    K->>C: RunPodSandbox
    C->>S: Create (TaskService ttrpc)
    S->>S: boot hypervisor VM
    S->>V: start virtiofsd (one per VM)
    S->>A: CreateSandbox (AgentService vsock:1024)
    K->>C: CreateContainer
    C->>S: Create (container task)
    S->>A: CreateContainer (config.json over vsock)
    A->>V: mount virtiofs export as rootfs
    A->>R: exec container process
    R-->>A: process running
    A-->>S: container started
    S-->>C: task running
    C-->>K: container ready

Networking: TC Redirect Without MACVTAP

Hypervisors cannot use a veth interface directly: veth works at the network namespace boundary in the kernel, but the VM's network interface must attach at the TAP layer so the VMM can forward raw Ethernet frames into the guest. Kata's shim creates a TAP interface in the host network namespace and installs TC redirect rules to bridge traffic bidirectionally between the pod's veth interface (which CNI already populated with an IP and routes) and the VM's TAP interface, at the ingress and egress filter level. This replaced an earlier approach that used MACVTAP; the TC redirect approach is more composable with CNI chains and avoids the MAC address management burden MACVTAP introduces.

The flow across a node where kubelet has assigned the pod network via CNI is: CNI populates the veth in the pod's network namespace; the shim bridges traffic from that veth to the TAP; Firecracker or QEMU connects the TAP to a virtio-net device; the guest kernel sees a eth0 with the CNI-assigned IP. From the pod's perspective — and from kubelet's — the network is indistinguishable from a plain container's. From the kernel's perspective, every packet crosses the hardware boundary of the VM twice.

Four Hypervisors, One Shim

All four VMM backends are invoked through containerd-shim-kata-v2; the selection is made in configuration.toml. The current releases in the main branch are QEMU v10.2.1 (supporting x86_64, aarch64, ppc64le, s390x, and riscv, with the largest device footprint and broadest architecture coverage), Cloud Hypervisor v51.1 (x86_64 and aarch64, with fine-grained per-thread seccomp and an HTTP OpenAPI management interface), Firecracker v1.12.1 (x86_64 and aarch64, with the most restrictions: no virtiofs, no device hotplug, no VFIO, no CPU or memory hotplug, devmapper snapshotter required), and Dragonball (no separate released version; runs in-process with the runtime-rs shim at zero IPC overhead).

Firecracker as a Kata backend is configured via /etc/kata-containers/configuration-fc.toml. Because Firecracker does not support virtiofs, the devmapper snapshotter is mandatory and virtio-block is the only rootfs transport. This makes the Kata-on-Firecracker path more constrained than the Kata-on-QEMU path but also closer to what firecracker-containerd does directly.

Kata 3.0 introduced runtime-rs, a Rust-based shim with Dragonball as an embedded VMM running in the same process as the shim. Since the VMM and the shim share an address space, the REST API round-trip between them disappears entirely. Kata 4.0 — planned for July 2026 — makes runtime-rs the default; the Go runtime enters deprecation at that point, receiving only bug and security fixes with removal no earlier than version 5.0.

Flintlock: The VM As A Kubernetes Node

A Different Level Of Abstraction

firecracker-containerd and Kata Containers both put the microVM inside the container model: the container is the unit of work, and the VM is the isolation mechanism. Flintlock inverts the relationship. The microVM is the unit of infrastructure. Its primary use case is not running application containers but provisioning the nodes those containers will run on — Kubernetes worker and control-plane nodes, managed through an API that speaks OCI image references for kernel and disk images.

Flintlock (liquidmetal-dev/flintlock, formerly weaveworks/flintlock, MPL-2.0) is a host-level daemon written in Go. It exposes a gRPC service named MicroVM in package microvm.services.api.v1alpha1 with exactly five RPC methods: CreateMicroVM, DeleteMicroVM, GetMicroVM, ListMicroVMs, and ListMicroVMsStream. The gRPC endpoint defaults to localhost:9090; a grpc-gateway HTTP endpoint defaults to localhost:8090, with REST mappings for the first four methods (POST /v1alpha1/microvm, DELETE /v1alpha1/microvm/{uid}, GET /v1alpha1/microvm/{uid}, GET /v1alpha1/microvm/{namespace}).

MicroVMSpec: OCI Images As Disks

The MicroVMSpec protobuf message (package flintlock.types, proto3) captures the full declaration of a microVM:

string id = 1
string namespace = 2
map<string, string> labels = 3
int32 vcpu = 4
int32 memory_in_mb = 5
Kernel kernel = 6
optional Initrd initrd = 7
Volume root_volume = 8
repeated Volume additional_volumes = 9
repeated NetworkInterface interfaces = 10
map<string, string> metadata = 11
google.protobuf.Timestamp created_at = 12
google.protobuf.Timestamp updated_at = 13
google.protobuf.Timestamp deleted_at = 14
optional string uid = 15

The Kernel message (selected fields) carries string image = 1, map<string,string> cmdline = 2, optional string filename = 3 (to name the kernel file within the image), and bool add_network_config = 4: the kernel is an OCI image reference, not a local path. Volume.VolumeSource has optional string container_source = 1, meaning disk images are also specified as OCI image references, pulled through containerd. The NetworkInterface.IfaceType enum has two values: MACVTAP = 0 (required for Cloud Hypervisor) and TAP = 1 (the only option Firecracker supports). MicroVMStatus.MicroVMState tracks PENDING, CREATED, FAILED, and DELETING.

Containerd As The Storage Layer

Flintlock uses containerd as its storage and pull infrastructure, connecting to the local containerd socket at /run/containerd/containerd.sock. It operates in the "flintlock" namespace — separate from the "default" namespace that other tools use — with volumes backed by the devmapper snapshotter (ContainerdVolumeSnapshotter = "devmapper") and kernel and initrd images pulled using the native snapshotter (ContainerdKernelSnapshotter = "native"). State is rooted at /var/lib/flintlock; configuration lives in /etc/opt/flintlockd. The primary VMM backend is Firecracker via firecracker-microvm/firecracker-go-sdk v1.0.0; Cloud Hypervisor is experimentally supported, selected at startup with the --default-provider flag.

The pattern of block devices from devmapper snapshots being passed directly to Firecracker as virtio-blk drives is the same as in firecracker-containerd, but the purpose differs: firecracker-containerd passes a container's OCI layer as a block device to run an application inside the VM; flintlock passes an OS disk image as a block device to run a Kubernetes node on top of the VM.

The default resource limits, from github.com/weaveworks/flintlock/pkg/defaults: two vCPUs, 1024 MB of memory, a ten-second timeout on VM deletion, a ten-minute resync period, and a maximum of ten retries. The Firecracker binary is resolved from $PATH with FirecrackerDetach = true, meaning the VMM process is detached from the daemon's process group.

Cluster API Integration

The operational interface for flintlock at scale is liquidmetal-dev/cluster-api-provider-microvm (CAPMVM), a Cluster API Infrastructure Provider registered with clusterctl as InfrastructureProvider named "microvm". Installation is clusterctl init -i microvm; CNI integration (for example, Cilium) requires the feature flag EXP_CLUSTER_RESOURCE_SET=true.

CAPMVM defines two core types. MicrovmMachineSpec embeds microvm.VMSpec inline and adds SSHPublicKeys []microvm.SSHPublicKey and ProviderID *string. MicrovmMachineStatus carries Ready bool, VMState *microvm.VMState, Addresses []clusterv1.MachineAddress, FailureReason, FailureMessage, and Conditions — the standard CAPI machine infrastructure-provider contract, with GetConditions/SetConditions and a MachineFinalizer. MicrovmClusterSpec fields include ControlPlaneEndpoint clusterv1.APIEndpoint, SSHPublicKeys []microvm.SSHPublicKey, Placement, MicrovmProxy *flclient.Proxy, and TLSSecretRef string. MicrovmClusterStatus carries Ready bool, Conditions clusterv1.Conditions, and FailureDomains clusterv1.FailureDomains.

A lighter-weight alternative exists for smaller deployments: the liquidmetal-dev/microvm-operator, a plain Kubernetes operator (not CAPI) for creating batches of microVMs on flintlock-managed devices. Its README marks it as proof of concept.

Where The Two Books Meet

The containerd book traces the container from a containerd.Create call down through the shim v2 protocol, the snapshotter, the CNI plugin chain, and the TC networking primitives. This book traces the microVM from a Firecracker API call down through KVM ioctls, virtio transports, and the hardware isolation boundary. The junction between the two traces is the containerd shim.

Topic Containerd book MicroVM book extension
Shim v2 / TaskService io.containerd.runc.v2 runs OCI containers aws.firecracker / io.containerd.kata.v2 substitute a VM for the container sandbox
devmapper snapshotter Container layer storage on the host Block device passed to Firecracker virtio-blk instead of mounted in the host filesystem
TAP + TC redirect CNI networking for pods Veth-to-TAP bridge for VM network interfaces in both firecracker-containerd and Kata
ttrpc Containerd-to-shim control plane Shim-to-in-VM-agent control plane (firecracker-containerd: port 10789; Kata: port 1024)

The hardware enforcement point is what the table cannot convey on its own. In both microVM paths, EPT or NPT in the CPU enforces the isolation boundary — a kernel vulnerability on the guest side cannot reach the host side without also defeating the hardware virtualization extension. That boundary is absent from plain container isolation, where the same kernel enforces security on both sides of every namespace wall. The shim is the hinge.

Chapters 4 through 8 of this book built the hardware side of that hinge: the VMCS, the KVM_RUN loop, the virtio queues, and the two-dimensional page tables. The chapters in the containerd book build the software side: the snapshotter API, the ttrpc shim protocol, the CNI invocation, and the event routing.

Sources And Further Reading