This book is a comprehensive introduction to all the components of a high-performance parallel linear algebra library, as well as a guide to the PLAPACK infrastructure. PLAPACK is a library infrastructure for the parallel implementation of linear algebra algorithms and applications on distributed memory supercomputers such as the Intel Paragon, IBM SP2, Cray T3D/T3E, SGI PowerChallenge, and Convex Exemplar. This infrastructure allows library developers, scientists, and engineers to exploit a natural approach to encoding so-called blocked algorithms, which achieve high performance by operating on submatrices and subvectors. This feature, as well as the use of an alternative, more application-centric approach to data distribution, sets PLAPACK apart from other parallel linear algebra libraries, allowing for strong performance and significanltly less programming by the user. This book is a comprehensive introduction to all the components of a high-performance parallel linear algebra library, as well as a guide to the PLAPACK infrastructure. Scientific and Engineering Computation series

This is a textbook that teaches the bridging topics between numerical analysis, parallel computing, code performance, large scale applications.

The book contains reports about the most significant projects from science and industry that are using the supercomputers of the Federal High Performance Computing Center Stuttgart (HLRS). These projects are from different scientific disciplines, with a focus on engineering, physics and chemistry. They were carefully selected in a peer-review process and are showcases for an innovative combination of state-of-the-art physical modeling, novel algorithms and the use of leading-edge parallel computer technology. As HLRS is in close cooperation with industrial companies, special emphasis has been put on the industrial relevance of results and methods.

Containing over 300 entries in an A-Z format, the Encyclopedia of Parallel Computing provides easy, intuitive access to relevant information for professionals and researchers seeking access to any aspect within the broad field of parallel computing. Topics for this comprehensive reference were selected, written, and peer-reviewed by an international pool of distinguished researchers in the field. The Encyclopedia is broad in scope, covering machine organization, programming languages, algorithms, and applications. Within each area, concepts, designs, and specific implementations are presented. The highly-structured essays in this work comprise synonyms, a definition and discussion of the topic, bibliographies, and links to related literature. Extensive cross-references to other entries within the Encyclopedia support efficient, user-friendly searchers for immediate access to useful information. Key concepts presented in the Encyclopedia of Parallel Computing include; laws and metrics; specific numerical and non-numerical algorithms; asynchronous algorithms; libraries of subroutines; benchmark suites; applications; sequential consistency and cache coherency; machine classes such as clusters, shared-memory multiprocessors, special-purpose machines and dataflow machines; specific machines such as Cray supercomputers, IBM’s cell processor and Intel’s multicore machines; race detection and auto parallelization; parallel programming languages, synchronization primitives, collective operations, message passing libraries, checkpointing, and operating systems. Topics covered: Speedup, Efficiency, Isoefficiency, Redundancy, Amdahls law, Computer Architecture Concepts, Parallel Machine Designs, Benmarks, Parallel Programming concepts & design, Algorithms, Parallel applications. This authoritative reference will be published in two formats: print and online. The online edition features hyperlinks to cross-references and to additional significant research. Related Subjects: supercomputing, high-performance computing, distributed computing

Scientific applications involve very large computations that strain the resources of whatever computers are available. Such computations implement sophisticated mathematics, require deep scientific knowledge, depend on subtle interplay of different approximations, and may be subject to instabilities and sensitivity to external input. Software able to succeed in this domain invariably embeds significant domain knowledge that should be tapped for future use. Unfortunately, most existing scientific software is designed in an ad hoc way, resulting in monolithic codes understood by only a few developers. Software architecture refers to the way software is structured to promote objectives such as reusability, maintainability, extensibility, and feasibility of independent implementation. Such issues have become increasingly important in the scientific domain, as software gets larger and more complex, constructed by teams of people, and evolved over decades. In the context of scientific computation, the challenge facing mathematical software practitioners is to design, develop, and supply computational components which deliver these objectives when embedded in end-user application codes. The Architecture of Scientific Software addresses emerging methodologies and tools for the rational design of scientific software, including component integration frameworks, network-based computing, formal methods of abstraction, application programmer interface design, and the role of object-oriented languages. This book comprises the proceedings of the International Federation for Information Processing (IFIP) Conference on the Architecture of Scientific Software, which was held in Ottawa, Canada, in October 2000. It will prove invaluable reading for developers of scientific software, as well as for researchers in computational sciences and engineering.

Parallel processing has been an enabling technology in scientific computing for more than 20 years. This book is the first in-depth discussion of parallel computing in 10 years; it reflects the mix of topics that mathematicians, computer scientists, and computational scientists focus on to make parallel processing effective for scientific problems. Presently, the impact of parallel processing on scientific computing varies greatly across disciplines, but it plays a vital role in most problem domains and is absolutely essential in many of them. Parallel Processing for Scientific Computing is divided into four parts: The first concerns performance modeling, analysis, and optimization; the second focuses on parallel algorithms and software for an array of problems common to many modeling and simulation applications; the third emphasizes tools and environments that can ease and enhance the process of application development; and the fourth provides a sampling of applications that require parallel computing for scaling to solve larger and realistic models that can advance science and engineering.

This book aims to introduce graduate students to the many applications of numerical computation, explaining in detail both how and why the included methods work in practice. The text addresses numerical analysis as a middle ground between practice and theory, addressing both the abstract mathematical analysis and applied computation and programming models instrumental to the field. While the text uses pseudocode, Matlab and Julia codes are available online for students to use, and to demonstrate implementation techniques. The textbook also emphasizes multivariate problems alongside single-variable problems and deals with topics in randomness, including stochastic differential equations and randomized algorithms, and topics in optimization and approximation relevant to machine learning. Ultimately, it seeks to clarify issues in numerical analysis in the context of applications, and presenting accessible methods to students in mathematics and data science.