Students and teachers of mathematics and related fields will find this book a comprehensive and modern approach to probability theory, providing the background and techniques to go from the beginning graduate level to the point of specialization in research areas of current interest. The book is designed for a two- or three-semester course, assuming only courses in undergraduate real analysis or rigorous advanced calculus, and some elementary linear algebra. A variety of applications—Bayesian statistics, financial mathematics, information theory, tomography, and signal processing—appear as threads to both enhance the understanding of the relevant mathematics and motivate students whose main interests are outside of pure areas.

Multi-armed bandits is a rich, multi-disciplinary area that has been studied since 1933, with a surge of activity in the past 10-15 years. This is the first book to provide a textbook like treatment of the subject.

The significantly expanded and updated new edition of a widely used text on reinforcement learning, one of the most active research areas in artificial intelligence. Reinforcement learning, one of the most active research areas in artificial intelligence, is a computational approach to learning whereby an agent tries to maximize the total amount of reward it receives while interacting with a complex, uncertain environment. In Reinforcement Learning, Richard Sutton and Andrew Barto provide a clear and simple account of the field's key ideas and algorithms. This second edition has been significantly expanded and updated, presenting new topics and updating coverage of other topics. Like the first edition, this second edition focuses on core online learning algorithms, with the more mathematical material set off in shaded boxes. Part I covers as much of reinforcement learning as possible without going beyond the tabular case for which exact solutions can be found. Many algorithms presented in this part are new to the second edition, including UCB, Expected Sarsa, and Double Learning. Part II extends these ideas to function approximation, with new sections on such topics as artificial neural networks and the Fourier basis, and offers expanded treatment of off-policy learning and policy-gradient methods. Part III has new chapters on reinforcement learning's relationships to psychology and neuroscience, as well as an updated case-studies chapter including AlphaGo and AlphaGo Zero, Atari game playing, and IBM Watson's wagering strategy. The final chapter discusses the future societal impacts of reinforcement learning.

Students and teachers of mathematics and related fields will find this book a comprehensive and modern approach to probability theory, providing the background and techniques to go from the beginning graduate level to the point of specialization in research areas of current interest. The book is designed for a two- or three-semester course, assuming only courses in undergraduate real analysis or rigorous advanced calculus, and some elementary linear algebra. A variety of applications—Bayesian statistics, financial mathematics, information theory, tomography, and signal processing—appear as threads to both enhance the understanding of the relevant mathematics and motivate students whose main interests are outside of pure areas.

This book constitutes the refereed conference proceedings of the 20th International Symposium on Research in Attacks, Intrusions, and Defenses, RAID 2017, held in Atlanta, GA, USA, in September 2017. The 21 revised full papers were selected from 105 submissions. They are organized in the following topics: software security, intrusion detection, systems security, android security, cybercrime, cloud security, network security.

Research on general video game playing aims at designing agents or content generators that can perform well in multiple video games, possibly without knowing the game in advance and with little to no specific domain knowledge. The general video game AI framework and competition propose a challenge in which researchers can test their favorite AI methods with a potentially infinite number of games created using the Video Game Description Language. The open-source framework has been used since 2014 for running a challenge. Competitors around the globe submit their best approaches that aim to generalize well across games. Additionally, the framework has been used in AI modules by many higher-education institutions as assignments, or as proposed projects for final year (undergraduate and Master's) students and Ph.D. candidates. The present book, written by the developers and organizers of the framework, presents the most interesting highlights of the research performed by the authors during these years in this domain. It showcases work on methods to play the games, generators of content, and video game optimization. It also outlines potential further work in an area that offers multiple research directions for the future.

This newly revised and expanded second edition of the popular Artech House title, Fuzzing for Software Security Testing and Quality Assurance, provides practical and professional guidance on how and why to integrate fuzzing into the software development lifecycle. This edition introduces fuzzing as a process, goes through commercial tools, and explains what the customer requirements are for fuzzing. The advancement of evolutionary fuzzing tools, including American Fuzzy Lop (AFL) and the emerging full fuzz test automation systems are explored in this edition. Traditional software programmers and testers will learn how to make fuzzing a standard practice that integrates seamlessly with all development activities. It surveys all popular commercial fuzzing tools and explains how to select the right one for software development projects. This book is a powerful new tool to build secure, high-quality software taking a weapon from the malicious hacker’s arsenal. This practical resource helps engineers find and patch flaws in software before harmful viruses, worms, and Trojans can use these vulnerabilities to rampage systems. The book shows how to make fuzzing a standard practice that integrates seamlessly with all development activities.