The Heisenberg measuring uncertainty in lightweight virtualization testbeds

• Authors: Quan Jia;Zhaohui Wang;Angelos Stavrou
• Venue: CSET'09 Proceedings of the 2nd conference on Cyber security experimentation and test
• Year: 2009

The need for large-scale experimentation testbeds involving several hundred, or even thousands, of nodes is undeniable. Testbeds including Emulab [10], and Deter [5] are heavily used for both research and application testing. To scale even further and shed some of the limitations that the relative small number of physical nodes impose, researchers have turned to full virtualization [8] and lightweight, container-based virtualization [16, 10]. Virtualization allows running multiple virtual execution environments (VEEs) per physical host. n this paper, we evaluate the use of hundreds of lightweight containers as a testbed to measure the performance of simple applications. We show that, although economically and technically compelling, virtualization has some limitations due the sharing of host resources (CPU, network, memory and disk) among samehost VEEs. Determining the number of VEEs that can be deployed in a physical machine without interfering with the fidelity of the experiment is not a trivial task and it cannot be estimated or computed ahead of time using aggregate utilization of individual resource. Furthermore, monitoring the health of an experiment by measuring the individual resource utilization can affect the behavior of the service under test. Therefore, we observed what we call a "weak" form of the Heisenberg uncertainty principle for host resource measurements: increasing the precision and fidelity of the resource measurements can interfere with the behavior of the experiment. We believe that this observation holds in general but it becomes more pronounced when we instantiate hundreds of VEEs due to the necessary context switching.