An implementation of the Hamlyn sender-managed interface architecture
OSDI '96 Proceedings of the second USENIX symposium on Operating systems design and implementation
Managing energy and server resources in hosting centers
SOSP '01 Proceedings of the eighteenth ACM symposium on Operating systems principles
Wake on wireless: an event driven energy saving strategy for battery operated devices
Proceedings of the 8th annual international conference on Mobile computing and networking
An Evaluation of Current High-Performance Networks
IPDPS '03 Proceedings of the 17th International Symposium on Parallel and Distributed Processing
Self-tuning wireless network power management
Proceedings of the 9th annual international conference on Mobile computing and networking
Runtime Power Monitoring in High-End Processors: Methodology and Empirical Data
Proceedings of the 36th annual IEEE/ACM International Symposium on Microarchitecture
Making a Case for Efficient Supercomputing
Queue - Power Management
Transport Protocol Optimization for Energy Efficient Wireless Embedded Systems
DATE '03 Proceedings of the conference on Design, Automation and Test in Europe - Volume 1
Performance Comparison of MPI Implementations over InfiniBand, Myrinet and Quadrics
Proceedings of the 2003 ACM/IEEE conference on Supercomputing
Exploring the Energy-Time Tradeoff in MPI Programs on a Power-Scalable Cluster
IPDPS '05 Proceedings of the 19th IEEE International Parallel and Distributed Processing Symposium (IPDPS'05) - Papers - Volume 01
Using multiple energy gears in MPI programs on a power-scalable cluster
Proceedings of the tenth ACM SIGPLAN symposium on Principles and practice of parallel programming
Performance Characterization of a 10-Gigabit Ethernet TOE
HOTI '05 Proceedings of the 13th Symposium on High Performance Interconnects
A Power-Aware Run-Time System for High-Performance Computing
SC '05 Proceedings of the 2005 ACM/IEEE conference on Supercomputing
Just In Time Dynamic Voltage Scaling: Exploiting Inter-Node Slack to Save Energy in MPI Programs
SC '05 Proceedings of the 2005 ACM/IEEE conference on Supercomputing
A comparison of 4X InfiniBand and Quadrics Elan-4 technologies
CLUSTER '04 Proceedings of the 2004 IEEE International Conference on Cluster Computing
Adaptive, transparent frequency and voltage scaling of communication phases in MPI programs
Proceedings of the 2006 ACM/IEEE conference on Supercomputing
CLUSTER '07 Proceedings of the 2007 IEEE International Conference on Cluster Computing
Designing Energy Efficient Communication Runtime Systems for Data Centric Programming Models
GREENCOM-CPSCOM '10 Proceedings of the 2010 IEEE/ACM Int'l Conference on Green Computing and Communications & Int'l Conference on Cyber, Physical and Social Computing
Per-call energy saving strategies in all-to-all communications
EuroMPI'11 Proceedings of the 18th European MPI Users' Group conference on Recent advances in the message passing interface
A case for RDMA in clouds: turning supercomputer networking into commodity
Proceedings of the Second Asia-Pacific Workshop on Systems
Designing energy efficient communication runtime systems: a view from PGAS models
The Journal of Supercomputing
Energy saving strategies for parallel applications with point-to-point communication phases
Journal of Parallel and Distributed Computing
On limitations of network acceleration
Proceedings of the ninth ACM conference on Emerging networking experiments and technologies
E2SC '13 Proceedings of the 1st International Workshop on Energy Efficient Supercomputing
Hi-index | 0.00 |
Recently, high speed interconnects capable of remote direct memory access (RDMA) such as InfiniBand and iWARP have gained considerable popularity due to their superb latency and bandwidth. Most existing studies about RDMA have focused mainly on its performance aspect. However, as power management has become essential for high-end systems such as enterprise servers and high performance computing nodes which are often equipped with RDMA capable network adapters, it is very important for us to take a fresh look at the benefits of RDMA from the power perspective. In the paper, we provide a detailed empirical study of the benefits of RDMA in terms of power savings compared with traditional communication protocols such as TCP/IP. We used two popular RDMA adapters in our evaluations: Mellanox ConnectX InfiniBand HCAs and Chelsio T3 10GE RNICs. In order to isolate the impact of communication on power consumption, our evaluation focused on using micro-benchmarks which perform different communication patterns. We have also studied several important factors that may have an impact on the performance and the power consumption of RDMA adapters such as the use of polling versus blocking, CPU speeds, and extra memory copies. We show that using high speed RDMA adapters can result in significant amount of power consumption during communication. (In one test, the system power has increased by as much as 50 watts, or over 30% of the idle power.) We found that RDMA generally has better power efficiency compared to that of TCP/IP, especially for communication intensive phases, for example when large messages are transferred. The power savings of RDMA are achieved by minimizing the interactions between the network adapters and other system components such as the CPUs and the memory: Although nearly the same amount of data must be going through the network adapters for both RDMA and TCP/IP, RDMA requires much fewer CPU cycles for protocol processing and also generates less memory bus traffic, both of which contribute to its power savings. Overall, our research demonstrated that RDMA not only provides high communication performance, but also offers excellent power efficiency, making it a desirable choice in environments that have strict power/energy constraints and demand high communication performance.