Applying In-Transit Buffers to Boost the Performance of Networks with Source Routing

• Authors: José Flich;Pedro López;Manuel Perez Malumbres;José Duato;Tomas Rokicki
• Venue: IEEE Transactions on Computers
• Year: 2003

Clusters of workstations (COWs) are becoming increasingly popular as a cost-effective alternative to parallel computers. In these systems, processors are connected using irregular topologies, providing the wiring flexibility, scalability, and incremental expansion capability required in this environment. Myrinet is one of the most popular interconnection networks for COWs. Myrinet uses source routing and wormhole switching. The up*/down* routing algorithm is used to build the network routes. On the other hand, in Myrinet, network behavior is controlled by the software running at the network interfaces. Hence, new features such as new routing algorithms can be added by only changing this software. In previous work, we proposed the In-Transit Buffer (ITB) mechanism to improve the performance of source routing-based networks. The ITB mechanism temporarily ejects packets from the network at some intermediate hosts and later reinjects them into the network, performing a special kind of virtual cut-through switching at these hosts. We applied this mechanism to up*/down* routing, in order to remove the down \rightarrow up forbidden channel dependences that prevented minimal routing between every pair of hosts. Results showed that network throughput can be more than doubled on medium-sized (32 switches) networks. In this paper, we analyze in depth the effect of using ITBs in the network, showing that they not only serve for guaranteeing minimal routing, but also that they are a powerful mechanism able to balance network traffic and reduce network contention. To demonstrate these capabilities, we apply the ITB mechanism to improved routing schemes, such as DFS and smart-routing. These routing algorithms (without ITBs) are able to improve the performance of up*/down* by 30 percent and 90 percent, respectively, for a 32-switch network. The evaluation results show that, when ITBs are used together with these improved routing algorithms, network throughput achieved by DFS and smart-routing can still be improved by 56 percent and 23 percent, respectively. However, smart-routing requires a time to compute the routing tables that rapidly grows with network size, it being impossible in practice to build networks with more than 32 switches. This high computational cost is mainly motivated by the need of obtaining deadlock-free routing tables. However, when ITBs are used, one can decouple the stages of computing routing tables and breaking cycles. Moreover, as stated above, ITBs can be used to reduce network contention. In this way, in this paper, we also propose a completely new routing algorithm that tries to balance network traffic by using a simple and low time consuming strategy. The proposed algorithm guarantees deadlock freedom and reduces network contention with the use of ITBs. The evaluation results show that our algorithm obtains unprecedented throughputs in 32-switch networks, tripling the original up*/down* and almost doubling smart-routing.