Performance Measurement of an Integrated NIC Architecture with 10GbE

• Authors: Guangdeng Liao;Laxmi Bhuyan
• Venue: HOTI '09 Proceedings of the 2009 17th IEEE Symposium on High Performance Interconnects
• Year: 2009

The deployment of 10 Gigabit Ethernet (10GbE) connections to servers has been hampered by the "fast-network-slow-host" phenomenon. Recently, the integration of network interfaces (INICs) is proposed to tackle the performance mismatch.While significant advantages over PCI-based discrete NICs (DNICs) were shown in prior work using simulation methodologies, it is still unclear how INICs perform on real machines with 10GbE. This paper is the first to study the impact of INICs by extensive evaluations through micro-benchmarks on a highly threaded Sun Niagara 2 processor. The processor is the industry’s first "system on a chip," integrating two 10GbE NICs. We observe that the INIC only shows its advantage over the DNIC with large I/O sizes. It improves 7.5% network bandwidth while saving 20% relative CPU utilization. We characterize the system behaviors to fully understand the performance benefits with respect to different number of connections, OS overhead, instruction counts, and cache misses etc.All of our studies reveal that there is a benefit of integrating NICs onto CPUs, but the gain is somewhat marginal. More aggressive integrated NIC designs should be adopted for higher speed networks like the upcoming 40GbE and 100GbE.