An analytic model for parallel computation

• Authors: Roger M. Firestone
• Venue: AFIPS '77 Proceedings of the June 13-16, 1977, national computer conference
• Year: 1977

A multiprocessor (MP) system is defined to be two or more independent processing units accessing a common memory for instructions and data. The common memory may be divided into independently accessible banks. A parallel program is a program operating on an MP system which makes use of more than one processing unit to achieve its computational goals. An analytic model is constructed which exposes certain properties of parallel programs, based on certain idealized assumptions. These assumptions are that: 1. the program algorithm may be divided into independent parallel-executable sections in any way desired; 2. the program is compute-bound, so that input-output considerations do not affect its behavior; 3. there exist linear cost functions for elapsed time and for computer time used; 4. initiation and termination of each independent activity of a parallel program incurs an identical amount of overhead; 5. competition for access to storage causes degradation in performance, manifested as reduced computational speed; and 6. references to storage are independently and uniformly distributed. Under these assumptions, the model gives an optimal number of processing units to use for the program in order to achieve minimum cost. This optimum is dependent on the overhead, the degradation, and the ratio of the cost of elapsed time to the cost of computer time. In the limit of very large compute time, a bound for the cost ratio is established, below which the use of parallel methods is not economically feasible. This lower bound depends only on the behavior of the degradation factor. Further analysis shows that the degradation factor is linear in the number of processing units employed, inversely linear in the number of independent storage banks, and quadratic in the relative storage accessing rate. A Monte Carlo model was constructed by J. L. Rosenfeld which gave similar results for those situations where the assumptions of the two models coincided.