Standardized but flexible I/O for self-virtualizing devices

  • Authors:
  • Joshua LeVasseur;Ramu Panayappan;Espen Skoglund;Christo du Toit;Leon Lynch;Alex Ward;Dulloor Rao;Rolf Neugebauer;Derek McAuley

  • Affiliations:
  • Netronome Systems;Netronome Systems;Netronome Systems;Netronome Systems;Netronome Systems;Netronome Systems;Georgia Tech;Netronome Systems;Netronome Systems

  • Venue:
  • WIOV'08 Proceedings of the First conference on I/O virtualization
  • Year:
  • 2008

Quantified Score

Hi-index 0.00

Visualization

Abstract

Moving device virtualization from the virtual machine monitor (VMM) to the devices improves virtual-machine performance significantly, but it requires support from the devices. PCI and PCI Express (PCIe) devices can provide VMs with direct and secure I/O through the use of multiple functions per card, but at significant cost and inflexibility. One solution to help reduce the costs is the PCIe SR-IOV standard, which introduces lightweight, virtual PCIe functions. We are currently developing a highly configurable and programmable PCIe networking device which can change its behavior at runtime and which can provide a number of different types of device functions to the host system (e.g., standard NIC, specialized packet capturing devices, or crypto offload engines). We have found the PCIe SR-IOV standard to be too inflexible to support these types of devices, primarily due to its mechanism for configuring virtual functions. In this paper we propose an alternative approach which does not require additional silicon and provides significantly higher flexibility than SR-IOV. We achieve this by delegating enumeration and configuration of "software configurable virtual functions" to the main device driver for the device. Our solution is compatible with the higher layers of the PCI device stack of modern operating systems and hypervisors so that we can leverage all the existing mechanisms for hot-plugging, discovering devices, loading device drivers, and assigning PCI devices to virtual machines (including providing DMA isolation with IO-MMUs). We present details of a prototype implementation for Linux and Xen.