Numerical programs often use parallel programming techniques such as OpenMP to compute the program's output values as efficient as possible. In addition, derivative values of these output values with respect to certain input values play a crucial role. To achieve code that computes not only the output values simultaneously but also the derivative values, this work introduces several source-to-source transformation rules. These rules are based on a technique called algorithmic differentiation. The main focus of this work lies on the important reverse mode of algorithmic differentiation. The inherent data-flow reversal of the reverse mode must be handled properly during the transformation. The first part of the work examines the transformations in a very general way since pragma-based parallel regions occur in many different kinds such as OpenMP, OpenACC, and Intel Phi. The second part describes the transformation rules of the most important OpenMP constructs.

This book constitutes the refereed proceedings of the 19th International Conference on Parallel and Distributed Computing, Euro-Par 2013, held in Aachen, Germany, in August 2013. The 70 revised full papers presented were carefully reviewed and selected from 261 submissions. The papers are organized in 16 topical sections: support tools and environments; performance prediction and evaluation; scheduling and load balancing; high-performance architectures and compilers; parallel and distributed data management; grid, cluster and cloud computing; peer-to-peer computing; distributed systems and algorithms; parallel and distributed programming; parallel numerical algorithms; multicore and manycore programming; theory and algorithms for parallel computation; high performance networks and communication; high performance and scientific applications; GPU and accelerator computing; and extreme-scale computing.

A guide to the most recent, advanced features of the widely used OpenMP parallel programming model, with coverage of major features in OpenMP 4.5. This book offers an up-to-date, practical tutorial on advanced features in the widely used OpenMP parallel programming model. Building on the previous volume, Using OpenMP: Portable Shared Memory Parallel Programming (MIT Press), this book goes beyond the fundamentals to focus on what has been changed and added to OpenMP since the 2.5 specifications. It emphasizes four major and advanced areas: thread affinity (keeping threads close to their data), accelerators (special hardware to speed up certain operations), tasking (to parallelize algorithms with a less regular execution flow), and SIMD (hardware assisted operations on vectors). As in the earlier volume, the focus is on practical usage, with major new features primarily introduced by example. Examples are restricted to C and C++, but are straightforward enough to be understood by Fortran programmers. After a brief recap of OpenMP 2.5, the book reviews enhancements introduced since 2.5. It then discusses in detail tasking, a major functionality enhancement; Non-Uniform Memory Access (NUMA) architectures, supported by OpenMP; SIMD, or Single Instruction Multiple Data; heterogeneous systems, a new parallel programming model to offload computation to accelerators; and the expected further development of OpenMP.

High Performance Parallelism Pearls Volume 2 offers another set of examples that demonstrate how to leverage parallelism. Similar to Volume 1, the techniques included here explain how to use processors and coprocessors with the same programming – illustrating the most effective ways to combine Xeon Phi coprocessors with Xeon and other multicore processors. The book includes examples of successful programming efforts, drawn from across industries and domains such as biomed, genetics, finance, manufacturing, imaging, and more. Each chapter in this edited work includes detailed explanations of the programming techniques used, while showing high performance results on both Intel Xeon Phi coprocessors and multicore processors. Learn from dozens of new examples and case studies illustrating "success stories" demonstrating not just the features of Xeon-powered systems, but also how to leverage parallelism across these heterogeneous systems. - Promotes write-once, run-anywhere coding, showing how to code for high performance on multicore processors and Xeon Phi - Examples from multiple vertical domains illustrating real-world use of Xeon Phi coprocessors - Source code available for download to facilitate further exploration

This volume constitutes the refereed proceedings of the 11th International Conference on Applied Parallel and Scientific Computing, PARA 2012, held in Helsinki, Finland, in June 2012. The 35 revised full papers presented were selected from numerous submissions and are organized in five technical sessions covering the topics of advances in HPC applications, parallel algorithms, performance analyses and optimization, application of parallel computing in industry and engineering, and HPC interval methods. In addition, three of the topical minisymposia are described by a corresponding overview article on the minisymposia topic. In order to cover the state-of-the-art of the field, at the end of the book a set of abstracts describe some of the conference talks not elaborated into full articles.

This book constitutes thoroughly refereed post-conference proceedings of the workshops of the 16th International Conference on Parallel Computing, Euro-Par 2010, held in Ischia, Italy, in August/September 2010. The papers of these 9 workshops HeteroPar, HPCC, HiBB, CoreGrid, UCHPC, HPCF, PROPER, CCPI, and VHPC focus on promotion and advancement of all aspects of parallel and distributed computing.

The essential guide for writing portable, parallel programs for GPUs using the OpenMP programming model. Today’s computers are complex, multi-architecture systems: multiple cores in a shared address space, graphics processing units (GPUs), and specialized accelerators. To get the most from these systems, programs must use all these different processors. In Programming Your GPU with OpenMP, Tom Deakin and Timothy Mattson help everyone, from beginners to advanced programmers, learn how to use OpenMP to program a GPU using just a few directives and runtime functions. Then programmers can go further to maximize performance by using CPUs and GPUs in parallel—true heterogeneous programming. And since OpenMP is a portable API, the programs will run on almost any system. Programming Your GPU with OpenMP shares best practices for writing performance portable programs. Key features include: The most up-to-date APIs for programming GPUs with OpenMP with concepts that transfer to other approaches for GPU programming. Written in a tutorial style that embraces active learning, so that readers can make immediate use of what they learn via provided source code. Builds the OpenMP GPU Common Core to get programmers to serious production-level GPU programming as fast as possible. Additional features: A reference guide at the end of the book covering all relevant parts of OpenMP 5.2. An online repository containing source code for the example programs from the book—provided in all languages currently supported by OpenMP: C, C++, and Fortran. Tutorial videos and lecture slides.