Is Parallelism for You?

• Authors: Cherri M. Pancake
• Venue: IEEE Computational Science & Engineering
• Year: 1996

Parallelism is an intuitive and appealing concept. Consider a computational science or engineering problem you've been working on. If executing it on a single CPU yields results in, say, 10 hours, why not use 10 CPUs and get the results after just an hour?In theory, parallelism is that simple--applying multiple CPUs to a single problem. Besides offering faster solutions, applications that have been parallelized--converted into parallel programs--can solve bigger, more complex problems whose input data or intermediate results exceed the memory capacity of one CPU. Simulations can be run at finer resolution. Physical phenomena can be modeled more realistically.In practice, however, parallelism carries a high price tag. It involves a steep learning curve, it is effort-intensive, and a parallel computer's runtime environment is inherently unstable and unpredictable. It is perfectly possible to work months on parallelizing an application, only to find that it yields incorrect results or that it runs slower now than before.How do you know whether to make the investment? This article offers 16 practical rules of thumb and several case studies that can help you predict if parallelism might be worthwhile for your application. For instance, inherently serial content has a debilitating impact on theoretical speedup. This means that you probably shouldn't consider parallelizing a program with a potentially "parallel content" of less than 95 percent, unless you're already experienced in parallel programming, or unless you can replace a significant portion of the serial version with proven, efficient parallel algorithms. The techniques presented for estimating likely performance gains are drawn from the experiences of hundreds of computational scientists and engineers at national labs, universities, and research facilities.