Multiphase Complete Exchange on Paragon, SP2, and CS-2

• Authors: Shahid H. Bokhari
• Venue: IEEE Parallel & Distributed Technology: Systems & Technology
• Year: 1996

The overhead of interprocessor communication is a major factor in limiting the performance of parallel computer systems. The complete exchange is the severest communication pattern in that it requires each processor to send a distinct message to every other processor. This pattern is at the heart of many important parallel applications. There are three main algorithms for complete exchange, all designed for hypercubes: the direct exchange, the standard exchange, and the multiphase exchange. Most contemporary commercial multicomputer systems are not hypercubes. However, through special-purpose hardware and dedicated communication processors, these systems can achieve very high performance communication and can emulate hypercubes quite well. Multiphase complete exchange, which is actually a family of algorithms with standard and direct exchange as extreme cases, performs optimally for varying message sizes. The author has implemented multiphase complete exchange on three contemporary parallel architectures: the Intel Paragon, the IBM SP2, and the Meiko CS-2. He describes the essential features of these machines and discusses their basic interprocessor communication overheads. Then he evaluates the performance of multiphase complete exchange on each machine. He discovered that the Paragon executes the multiphase well and yields smooth performance plots, with the cyclic variations in these plots stemming from memory access patterns; the SP2 exhibits enormous fluctuations in its plots because of interference from other jobs; and the CS-2 exhibits small fluctuations and the largest differences between predicted and observed timings. The author concludes that the theoretical ideas developed for hypercubes also apply to these machines and that multiphase complete exchange can lead to major savings in execution time over traditional solutions.