This book serves as both a textbook and handbook on the benchmarking of systems and components used as building blocks of modern information and communication technology applications. It provides theoretical and practical foundations as well as an in-depth exploration of modern benchmarks and benchmark development. The book is divided into two parts: foundations and applications. The first part introduces the foundations of benchmarking as a discipline, covering the three fundamental elements of each benchmarking approach: metrics, workloads, and measurement methodology. The second part focuses on different application areas, presenting contributions in specific fields of benchmark development. These contributions address the unique challenges that arise in the conception and development of benchmarks for specific systems or subsystems, and demonstrate how the foundations and concepts in the first part of the book are being used in existing benchmarks. Further, the book presents a number of concrete applications and case studies based on input from leading benchmark developers from consortia such as the Standard Performance Evaluation Corporation (SPEC) and the Transaction Processing Performance Council (TPC). Providing both practical and theoretical foundations, as well as a detailed discussion of modern benchmarks and their development, the book is intended as a handbook for professionals and researchers working in areas related to benchmarking. It offers an up-to-date point of reference for existing work as well as latest results, research challenges, and future research directions. It also can be used as a textbook for graduate and postgraduate students studying any of the many subjects related to benchmarking. While readers are assumed to be familiar with the principles and practices of computer science, as well as software and systems engineering, no specific expertise in any subfield of these disciplines is required.

Peer-to-peer systems are now widely used and have become the focus of attention for many researchers over the past decade. A number of algorithms for decentralized search, content distribution, and media streaming have been developed. This book provides fundamental concepts for the benchmarking of those algorithms in peer-to-peer systems. It also contains a collection of characteristic benchmarking results. The chapters of the book have been organized in three topical sections on: Fundamentals of Benchmarking in P2P Systems; Synthetic Benchmarks for Peer-to-Peer Systems; and Application Benchmarks for Peer-to-Peer Systems. They are preceded by a detailed introduction to the subject.

A comprehensive collection of benchmarks for measuring dependability in hardware-software systems As computer systems have become more complex and mission-critical, it is imperative for systems engineers and researchers to have metrics for a system's dependability, reliability, availability, and serviceability. Dependability benchmarks are useful for guiding development efforts for system providers, acquisition choices of system purchasers, and evaluations of new concepts by researchers in academia and industry. This book gathers together all dependability benchmarks developed to date by industry and academia and explains the various principles and concepts of dependability benchmarking. It collects the expert knowledge of DBench, a research project funded by the European Union, and the IFIP Special Interest Group on Dependability Benchmarking, to shed light on this important area. It also provides a large panorama of examples and recommendations for defining dependability benchmarks. Dependability Benchmarking for Computer Systems includes contributions from a credible mix of industrial and academic sources: IBM, Intel, Microsoft, Sun Microsystems, Critical Software, Carnegie Mellon University, LAAS-CNRS, Technical University of Valencia, University of Coimbra, and University of Illinois. It is an invaluable resource for engineers, researchers, system vendors, system purchasers, computer industry consultants, and system integrators.

This book constitutes the thoroughly refereed proceedings of the 5th International Workshop, PMBS 2014 in New Orleans, LA, USA in November 2014. The 12 full and 2 short papers presented in this volume were carefully reviewed and selected from 53 submissions. The papers cover topics on performance benchmarking and optimization; performance analysis and prediction; and power, energy and checkpointing.

The scope of contemporary higher education is wide, and concerns about the performance of higher education systems are widespread. The number of young people with a higher education qualification is expected to surpass 300 million in OECD and G20 countries by 2030. Higher education systems are faced with challenges that include expanding access, containing costs, and ensuring the quality and relevance of provision. The project on benchmarking higher education system performance provides a comprehensive and empirically rich review of the higher education landscape across OECD countries, taking stock of how well they are performing in meeting their education, research and engagement responsibilities.

The Complete Guide to Optimizing Systems Performance Written by the winner of the 2013 LISA Award for Outstanding Achievement in System Administration Large-scale enterprise, cloud, and virtualized computing systems have introduced serious performance challenges. Now, internationally renowned performance expert Brendan Gregg has brought together proven methodologies, tools, and metrics for analyzing and tuning even the most complex environments. Systems Performance: Enterprise and the Cloud focuses on Linux(R) and Unix(R) performance, while illuminating performance issues that are relevant to all operating systems. You'll gain deep insight into how systems work and perform, and learn methodologies for analyzing and improving system and application performance. Gregg presents examples from bare-metal systems and virtualized cloud tenants running Linux-based Ubuntu(R), Fedora(R), CentOS, and the illumos-based Joyent(R) SmartOS(TM) and OmniTI OmniOS(R). He systematically covers modern systems performance, including the "traditional" analysis of CPUs, memory, disks, and networks, and new areas including cloud computing and dynamic tracing. This book also helps you identify and fix the "unknown unknowns" of complex performance: bottlenecks that emerge from elements and interactions you were not aware of. The text concludes with a detailed case study, showing how a real cloud customer issue was analyzed from start to finish. Coverage includes - Modern performance analysis and tuning: terminology, concepts, models, methods, and techniques - Dynamic tracing techniques and tools, including examples of DTrace, SystemTap, and perf - Kernel internals: uncovering what the OS is doing - Using system observability tools, interfaces, and frameworks - Understanding and monitoring application performance - Optimizing CPUs: processors, cores, hardware threads, caches, interconnects, and kernel scheduling - Memory optimization: virtual memory, paging, swapping, memory architectures, busses, address spaces, and allocators - File system I/O, including caching - Storage devices/controllers, disk I/O workloads, RAID, and kernel I/O - Network-related performance issues: protocols, sockets, interfaces, and physical connections - Performance implications of OS and hardware-based virtualization, and new issues encountered with cloud computing - Benchmarking: getting accurate results and avoiding common mistakes This guide is indispensable for anyone who operates enterprise or cloud environments: system, network, database, and web admins; developers; and other professionals. For students and others new to optimization, it also provides exercises reflecting Gregg's extensive instructional experience.

Wastewater treatment plants are large non-linear systems subject to large perturbations in wastewater flow rate, load and composition. Nevertheless these plants have to be operated continuously, meeting stricter and stricter regulations. Many control strategies have been proposed in the literature for improved and more efficient operation of wastewater treatment plants. Unfortunately, their evaluation and comparison – either practical or based on simulation – is difficult. This is partly due to the variability of the influent, to the complexity of the biological and biochemical phenomena and to the large range of time constants (from a few minutes to several days). The lack of standard evaluation criteria is also a tremendous disadvantage. To really enhance the acceptance of innovative control strategies, such an evaluation needs to be based on a rigorous methodology including a simulation model, plant layout, controllers, sensors, performance criteria and test procedures, i.e. a complete benchmarking protocol. This book is a Scientific and Technical Report produced by the IWA Task Group on Benchmarking of Control Strategies for Wastewater Treatment Plants. The goal of the Task Group includes developing models and simulation tools that encompass the most typical unit processes within a wastewater treatment system (primary treatment, activated sludge, sludge treatment, etc.), as well as tools that will enable the evaluation of long-term control strategies and monitoring tasks (i.e. automatic detection of sensor and process faults). Work on these extensions has been carried out by the Task Group during the past five years, and the main results are summarized in Benchmarking of Control Strategies for Wastewater Treatment Plants. Besides a description of the final version of the already well-known Benchmark Simulation Model no. 1 (BSM1), the book includes the Benchmark Simulation Model no. 1 Long-Term (BSM1_LT) – with focus on benchmarking of process monitoring tasks – and the plant-wide Benchmark Simulation Model no. 2 (BSM2). Authors: Krist V. Gernaey, Technical University of Denmark, Lyngby, Denmark, Ulf Jeppsson, Lund University, Sweden, Peter A. Vanrolleghem, Université Laval, Quebec, Canada and John B. Copp, Primodal Inc., Hamilton, Ontario, Canada