How to Write Fast Numerical Code: A Small Introduction

Authors:
Srinivas Chellappa;Franz Franchetti;Markus Püschel
Affiliations:
Electrical and Computer Engineering, Carnegie Mellon University,;Electrical and Computer Engineering, Carnegie Mellon University,;Electrical and Computer Engineering, Carnegie Mellon University,
Venue:
Generative and Transformational Techniques in Software Engineering II
Year:
2007

Citing 34
Cited 3

Evaluating Associativity in CPU Caches

IEEE Transactions on Computers
Matrix multiplication via arithmetic progressions

Journal of Symbolic Computation - Special issue on computational algebraic complexity
Introduction to algorithms

Introduction to algorithms
A methodology for designing, modifying, and implementing Fourier transform algorithms on various architectures

Circuits, Systems, and Signal Processing
Computational frameworks for the fast Fourier transform

Computational frameworks for the fast Fourier transform
Numerical recipes in C (2nd ed.): the art of scientific computing

Numerical recipes in C (2nd ed.): the art of scientific computing
Optimizing matrix multiply using PHiPAC: a portable, high-performance, ANSI C coding methodology

ICS '97 Proceedings of the 11th international conference on Supercomputing
Applied numerical linear algebra

Applied numerical linear algebra
ScaLAPACK user's guide

ScaLAPACK user's guide
A fast Fourier transform compiler

Proceedings of the ACM SIGPLAN 1999 conference on Programming language design and implementation
LAPACK Users' guide (third ed.)

LAPACK Users' guide (third ed.)
Cramming more components onto integrated circuits

Readings in computer architecture
Generative programming: methods, tools, and applications

Generative programming: methods, tools, and applications
Introducing computer systems from a programmer's perspective

Proceedings of the thirty-second SIGCSE technical symposium on Computer Science Education
SPL: a language and compiler for DSP algorithms

Proceedings of the ACM SIGPLAN 2001 conference on Programming language design and implementation
FLAME: Formal Linear Algebra Methods Environment

ACM Transactions on Mathematical Software (TOMS)
Automatically tuned linear algebra software

SC '98 Proceedings of the 1998 ACM/IEEE conference on Supercomputing
Computer architecture: a quantitative approach

Computer architecture: a quantitative approach
An updated set of basic linear algebra subprograms (BLAS)

ACM Transactions on Mathematical Software (TOMS)
Automatic Performance Tuning in the UHFFT Library

ICCS '01 Proceedings of the International Conference on Computational Sciences-Part I
Iteration Space Tiling for Memory Hierarchies

Proceedings of the Third SIAM Conference on Parallel Processing for Scientific Computing
Short Vector Code Generation for the Discrete Fourier Transform

IPDPS '03 Proceedings of the 17th International Symposium on Parallel and Distributed Processing
A Dynamically Tuned Sorting Library

Proceedings of the international symposium on Code generation and optimization: feedback-directed and runtime optimization
A Complete Compiler Approach to Auto-Parallelizing C Programs for Multi-DSP Systems

IEEE Transactions on Parallel and Distributed Systems
The science of deriving dense linear algebra algorithms

ACM Transactions on Mathematical Software (TOMS)
Formal loop merging for signal transforms

Proceedings of the 2005 ACM SIGPLAN conference on Programming language design and implementation
Spiral: A Generator for Platform-Adapted Libraries of Signal Processing Algorithms

International Journal of High Performance Computing Applications
Sparsity: Optimization Framework for Sparse Matrix Kernels

International Journal of High Performance Computing Applications
FFT program generation for shared memory: SMP and multicore

Proceedings of the 2006 ACM/IEEE conference on Supercomputing
An automatically-tuned sorting library

Software—Practice & Experience
Generating FPGA-Accelerated DFT Libraries

FCCM '07 Proceedings of the 15th Annual IEEE Symposium on Field-Programmable Custom Computing Machines
Formal datapath representation and manipulation for implementing DSP transforms

Proceedings of the 45th annual Design Automation Conference
Automatic performance optimization of the discrete fourier transform on distributed memory computers

ISPA'06 Proceedings of the 4th international conference on Parallel and Distributed Processing and Applications
A Modified Split-Radix FFT With Fewer Arithmetic Operations

IEEE Transactions on Signal Processing

Operator Language: A Program Generation Framework for Fast Kernels

DSL '09 Proceedings of the IFIP TC 2 Working Conference on Domain-Specific Languages
Spectral accuracy in fast Ewald-based methods for particle simulations

Journal of Computational Physics
Power system probabilistic and security analysis on commodity high performance computing systems

HiPCNA-PG '13 Proceedings of the 3rd International Workshop on High Performance Computing, Networking and Analytics for the Power Grid

Quantified Score

Hi-index	0.00

Visualization

Abstract

The complexity of modern computing platforms has made it extremely difficult to write numerical code that achieves the best possible performance. Straightforward implementations based on algorithms that minimize the operations count often fall short in performance by at least one order of magnitude. This tutorial introduces the reader to a set of general techniques to improve the performance of numerical code, focusing on optimizations for the computer's memory hierarchy. Further, program generators are discussed as a way to reduce the implementation and optimization effort. Two running examples are used to demonstrate these techniques: matrix-matrix multiplication and the discrete Fourier transform.