Almost control-free (indeterministic) parallel computation on permit schemes

• Authors: Karel Čulík
• Venue: POPL '78 Proceedings of the 5th ACM SIGACT-SIGPLAN symposium on Principles of programming languages
• Year: 1978

The parallelity means "simultaneous performance or execution" and it may concern either computer units of different sorts (e.g. a memory and a processor), or computer units of the same sort (e.g. several processors).The computers Illiac 4 and Burroughs Scientific Processor (BSP) (announced recently) have several arithmetic processors. They are designed for large-scale computations with a special sort of numeric data structures, i.e. with large matrices (arrays). Their parallelity concerns naturally and essentially the definitions of matrix operations.According to [Br] the software techniques for exploiting the parallelism of the BSP consists of a "vectorization" of an usual serial program. There are 16 processors in the BSP which are heavily dependent each on the other, because at each instant by each of these processors just one and the same operation may be performed. Therefore a synchronization of all processors is assumed, which is the most important difference from the concept of parallel program scheme of [KM], which allows to speak about a sequence of particular steps, each of which is represented by execution of particular statements in parallel.In this paper a parallel computer with m ≥ 1 processors is assumed, which is also synchronized [Cu 1], but the processors are fully independent each on the other, i.e. on different processors different operations may be performed at each instant. Further, the parallelity concerns arbitrary simple data and arbitrary operations in all generality (and not only matrices). Thus the inherent parallelity of any serial program should be discovered and used for speeding up the duration of computation at most m times, while the memory space requirements remain unchanged.In [Cu 2] parallel flow diagrams were introduced and further the following "parallelization " (i.e. a "computation" of a parallel execution sequence of steps) of a serial program, was discussed in two parts: 1) newly permitted statements are determined by a permitter, and then 2) a subset of m (or less) statements is selected from the set of all permitted (and not yet selected) statements by a selector. The selected statements are executed in parallel on m processors, etc. until all permitted statements are selected (and executed), and the computation terminates. The permitter and the selector should replace the statement counter of serial programs or flow diagrams, by which is determined which statement should be executed as the next one. It is nuclear whether a suitable hardware technology can be designed to perform as a permitter and selector require.The linearity of programs is connected with serial computers (having just one processor, thus m = 1) essentially, and therefore in the following flow diagrams (or program schemes) will be used instead, because they allow a natural and transparent modification to permit diagrams and schemes, which represent a new sort of computing prescription (not necessarily deterministic algorithms) being a generalization of well-known binary trees, by which usual arithmetic expressions are represented.It is well known that there is no inherent reason for performing an operation from the numerator of a fraction sooner than an operation from the denominator. This arbitrariness of the order conceals an intrinsic parallelism of any expression which contains an n-ary operation with n ≥ 2 (which corresponds to the fact that the value of such operation does not depend on the order in which the values of its n arguments were achieved). This obvious fact is less clear when algorithms instead of operations are considered.