This book covers the history and recent developments of stochastic computing. Stochastic computing (SC) was first introduced in the 1960s for logic circuit design, but its origin can be traced back to von Neumann's work on probabilistic logic. In SC, real numbers are encoded by random binary bit streams, and information is carried on the statistics of the binary streams. SC offers advantages such as hardware simplicity and fault tolerance. Its promise in data processing has been shown in applications including neural computation, decoding of error-correcting codes, image processing, spectral transforms and reliability analysis. There are three main parts to this book. The first part, comprising Chapters 1 and 2, provides a history of the technical developments in stochastic computing and a tutorial overview of the field for both novice and seasoned stochastic computing researchers. In the second part, comprising Chapters 3 to 8, we review both well-established and emerging design approaches for stochastic computing systems, with a focus on accuracy, correlation, sequence generation, and synthesis. The last part, comprising Chapters 9 and 10, provides insights into applications in machine learning and error-control coding.

This book covers the history and recent developments of stochastic computing. Stochastic computing (SC) was first introduced in the 1960s for logic circuit design, but its origin can be traced back to von Neumann's work on probabilistic logic. In SC, real numbers are encoded by random binary bit streams, and information is carried on the statistics of the binary streams. SC offers advantages such as hardware simplicity and fault tolerance. Its promise in data processing has been shown in applications including neural computation, decoding of error-correcting codes, image processing, spectral transforms and reliability analysis. There are three main parts to this book. The first part, comprising Chapters 1 and 2, provides a history of the technical developments in stochastic computing and a tutorial overview of the field for both novice and seasoned stochastic computing researchers. In the second part, comprising Chapters 3 to 8, we review both well-established and emerging design approaches for stochastic computing systems, with a focus on accuracy, correlation, sequence generation, and synthesis. The last part, comprising Chapters 9 and 10, provides insights into applications in machine learning and error-control coding.

The end of Dennard scaling and demands for energy efficient, low power, and high density computing solutions over the past decade has forced exploration of new computing technologies. Stochastic computing is one of these alternative computing technologies which has enjoyed renewed interest and is the primary focus of this dissertation. Stochastic computing is a form of approximate computing which encodes values as probabilistic bitstreams where the ratio of 1s and 0s determines the encoded value. This representation allows stochastic computing to achieve lower operating power, higher computational density, and better error resilience compared to conventional binary-encoded circuits. In its current form, stochastic computing presents a number of challenges before it can become a practical replacement for conventional binary-encoded computing. First, there is little prior work detailing design methodologies to guide effective implementation and integration of stochastic computing into accelerator architectures. Second, the application space where stochastic computing yields compelling gains is far from obvious and has only seen limited exploration. Third, stochastic arithmetic circuits are unintuitive to design because they require careful consideration of correlation and quantization effects. This thesis focuses on new circuit components, applications, architectural considerations, and design techniques to improve the practicality of stochastic computing accelerators. I first propose novel stochastic circuits to improve the accuracy of stochastic computations and augment the range of implementable functions. I then evaluate the viability of stochastic computing with a design space exploration of end-to-end stochastic computing accelerator architectures. In this exploration, I evaluate under what design parameters and conditions stochastic computing accelerators are competitive alternatives to their binary-encoded counterparts. Using these guidelines, I use these results to establish a set of architecture design guidelines to help designers identify when and why they should consider stochastic computing. I then evaluate codesign opportunities and empirically measuring power, area, and energy efficiency for emerging applications. I also propose borrowing techniques from program synthesis such as stochastic synthesis and mixed integer linear programming to automatically synthesize novel stochastic circuits. Finally, I conclude with future directions for further improving the practicality of stochastic computing as well as additional research directions beyond stochastic computing.

The approximate and stochastic computing have been developed, on the one hand, to address the diminishing gains of technology scaling, and on the other hand, to exploit the intrinsic error resilience of many applications. They, indeed, take advantage of the disparity between the level of accuracy required by the application and that provided by the computing system, for achieving energy efficiency. As of the most important constitutes of an integrated circuit, arithmetic units often lie within the critical path of a processing system. They play a vital role in determining the performance and power consumption of the computing system. In the past decade, the design of the approximate arithmetic units has been in the center of attentions of the VLSI design research community; resulting in a numerous proposed approximate designs in the literature. In spite of a decade work on the approximate computing, there are still unresolved challenges faced by digital designers. The concept of acceptable quality of the results forms the foundation of the approximate and stochastic computing. In view of this fact, it is crucially decisive to have a clear, quantifiable definition of what signifies an acceptable quality. Indeed, the current metrics most often do not capture the requirements of a target application, and hence, mislead to sub-optimal design options for the application. Moreover, non-systematic designs, lack of fair comparisons and reproducible research have resulted in somewhat limited progresses in the field of approximate and stochastic computing. Besides, the accuracy requirements of an application is not a static property and may change across the different phases of the application. Therefore, it is important to systematically develop approximate and stochastic computing platforms which offer a variety of output qualities. In this dissertation, the aim is to take fundamental steps towards resolving the aforementioned challenges. Correspondingly, the following contributions are made in this dissertation. First, to palliate the lack of expressiveness of current metrics, a new parameterizable metric which correlates more precisely to the accuracy of the applications is proposed in this dissertation. Afterwards, the importance of fair comparisons for approximate computing units is underlined in this work. Subsequently, through generalizing and systematically optimizing an architectural template for approximate adders, an architecture is proposed which outperforms its existing counterparts. A conceptual framework for the systematic design of approximate adders including hybrid and non-equally segmented approaches is developed next. The framework discriminates the scenarios where approximate processing does not provide significant benefits from those where it does; in this latter case, it aids in obtaining optimal configurations for the adders. Furthermore, in order to address the dynamic configuration of the error characteristics, a stochastically-tunable adder is proposed which reduces the energy-delay product considerably in comparison with its conventional counterpart. In addition, we develop data-dependent corrections for truncated multipliers, where the proposed architectures surpass the existing approximate multipliers in the literature. The applicability of the proposed methods, and in general approximate computing units is eventually studied in modern applications. The correlation between the errors of a single unit and the whole system's accuracy is also investigated in the applications.

This book delves into various solution paradigms such as artificial neural network, support vector machine, wavelet transforms, evolutionary computing, swarm intelligence. During the last decade, novel solution technologies based on human and species intelligence have gained immense popularity due to their flexible and unconventional approach. New analytical tools are also being developed to handle big data processing and smart decision making. The idea behind compiling this work is to familiarize researchers, academicians, industry persons and students with various applications of intelligent techniques for producing sustainable, cost-effective and robust solutions of frequently encountered complex, real-world problems in engineering and science disciplines. The practical problems in smart grids, communication, waste management, elimination of harmful elements from nature, etc., are identified, and smart and optimal solutions are proposed.

The@ first graduate-level textbook to focus on fundamental aspects of numerical methods for stochastic computations, this book describes the class of numerical methods based on generalized polynomial chaos (gPC). These fast, efficient, and accurate methods are an extension of the classical spectral methods of high-dimensional random spaces. Designed to simulate complex systems subject to random inputs, these methods are widely used in many areas of computer science and engineering. The book introduces polynomial approximation theory and probability theory; describes the basic theory of gPC methods through numerical examples and rigorous development; details the procedure for converting stochastic equations into deterministic ones; using both the Galerkin and collocation approaches; and discusses the distinct differences and challenges arising from high-dimensional problems. The last section is devoted to the application of gPC methods to critical areas such as inverse problems and data assimilation. Ideal for use by graduate students and researchers both in the classroom and for self-study, Numerical Methods for Stochastic Computations provides the required tools for in-depth research related to stochastic computations. The first graduate-level textbook to focus on the fundamentals of numerical methods for stochastic computations Ideal introduction for graduate courses or self-study Fast, efficient, and accurate numerical methods Polynomial approximation theory and probability theory included Basic gPC methods illustrated through examples