Quinn dedicates significant attention to SIMD architectures (historically exemplified by the Connection Machine and vector processors).
I can’t link to copyrighted material directly. To obtain a copy:
Michael J. Quinn’s Parallel Computing: Theory and Practice is not merely a programming manual; it is a treatise on the mathematics of concurrency. It teaches that parallelism is not an optimization, but a fundamental rethinking of algorithm design. The text proves that locality (keeping data close to computation) and dependency analysis (avoiding race conditions) are the two immutable laws of high-performance systems. Parallel Computing Theory And Practice Michael J Quinn Pdf
This is the dominant paradigm in modern computing (multicore CPUs, clusters).
In the context of modern High-Performance Computing (HPC), Quinn’s work provides the "why" behind current trends. In the context of modern High-Performance Computing (HPC),
1. The Death of SIMD and its Resurrection: Quinn wrote extensively on SIMD, which fell out of favor in the late 90s. However, modern GPU computing (CUDA, OpenCL) is fundamentally SIMD (renamed SIMT—Single Instruction, Multiple Threads). Quinn’s theoretical breakdown of data parallelism is directly applicable to programming modern Nvidia/AMD GPUs.
2. The Message Passing Interface (MPI): While the book predates the ubiquity of cloud computing, its focus on Distributed Memory algorithms predicts the rise of MPI and MapReduce. The analysis of "owner-computes" rules (where the processor owning a memory location performs the calculation) is the foundational logic of MPI. modern GPU computing (CUDA
3. The Scalability Wall: Quinn’s treatment of isoefficiency functions—how memory and computation must scale to maintain efficiency—is a concept often ignored in modern "easy scaling" cloud environments. It explains why simply adding nodes to a cluster often results in zero performance gain for poorly designed algorithms (due to network saturation).
Quinn transitions from hardware to software, detailing specific strategies for parallel algorithm design.