Query mesh: multi-route query processing technology
Proceedings of the VLDB Endowment
| Hi-index | 0.00 |
Building high-performance clusters using one of the two leading network technologies, Myrinet and Infini- Band, has been thought as a de facto way to achieve several teraflops computing power. Meanwhile, main- taining both types of clusters, it appears, may have cre- ated an another challenge for the MPI programming system, the most popular parallel programming library that has been successfully used on both networks. The belief that extending cluster resources across two dif- ferent types of networks may increase computing paral- lelism has driven many researchers to tackle this chal- lenge with various viewpoints. We approach this chal- lenge with a different perspective, application trans- parency, which is accomplishing the goal without any modification of legacy MPI applications. We, therefore, focus on the design of an RDMA gateway that can re- lay messages very fast, and this design focus turns out to be a better way to preserve the application trans- parency. RDMA Gateway (RG), our prototyped sys- tem, has a very efficient memory management mecha- nism that prevents RG from showing irregular spikes of a memory usage under a heavy load condition. Exper- imental results show that running parallel applications over heterogeneous clusters can be very promising with low performance overhead.