Stochastic Averaging and Extremum Seeking treats methods inspired by attempts to understand the seemingly non-mathematical question of bacterial chemotaxis and their application in other environments. The text presents significant generalizations on existing stochastic averaging theory developed from scratch and necessitated by the need to avoid violation of previous theoretical assumptions by algorithms which are otherwise effective in treating these systems. Coverage is given to four main topics. Stochastic averaging theorems are developed for the analysis of continuous-time nonlinear systems with random forcing, removing prior restrictions on nonlinearity growth and on the finiteness of the time interval. The new stochastic averaging theorems are usable not only as approximation tools but also for providing stability guarantees. Stochastic extremum-seeking algorithms are introduced for optimization of systems without available models. Both gradient- and Newton-based algorithms are presented, offering the user the choice between the simplicity of implementation (gradient) and the ability to achieve a known, arbitrary convergence rate (Newton). The design of algorithms for non-cooperative/adversarial games is described. The analysis of their convergence to Nash equilibria is provided. The algorithms are illustrated on models of economic competition and on problems of the deployment of teams of robotic vehicles. Bacterial locomotion, such as chemotaxis in E. coli, is explored with the aim of identifying two simple feedback laws for climbing nutrient gradients. Stochastic extremum seeking is shown to be a biologically-plausible interpretation for chemotaxis. For the same chemotaxis-inspired stochastic feedback laws, the book also provides a detailed analysis of convergence for models of nonholonomic robotic vehicles operating in GPS-denied environments. The book contains block diagrams and several simulation examples, including examples arising from bacterial locomotion, multi-agent robotic systems, and economic market models. Stochastic Averaging and Extremum Seeking will be informative for control engineers from backgrounds in electrical, mechanical, chemical and aerospace engineering and to applied mathematicians. Economics researchers, biologists, biophysicists and roboticists will find the applications examples instructive.

Extremum Seeking through Delays and PDEs, the first book on the topic, expands the scope of applicability of the extremum seeking method, from static and finite-dimensional systems to infinite-dimensional systems. Readers will find numerous algorithms for model-free real-time optimization are developed and their convergence guaranteed, extensions from single-player optimization to noncooperative games, under delays and PDEs, are provided, the delays and PDEs are compensated in the control designs using the PDE backstepping approach, and stability is ensured using infinite-dimensional versions of averaging theory, and accessible and powerful tools for analysis. This book is intended for control engineers in all disciplines (electrical, mechanical, aerospace, chemical), mathematicians, physicists, biologists, and economists. It is appropriate for graduate students, researchers, and industrial users.

A high school student can create deep Q-learning code to control her robot, without any understanding of the meaning of 'deep' or 'Q', or why the code sometimes fails. This book is designed to explain the science behind reinforcement learning and optimal control in a way that is accessible to students with a background in calculus and matrix algebra. A unique focus is algorithm design to obtain the fastest possible speed of convergence for learning algorithms, along with insight into why reinforcement learning sometimes fails. Advanced stochastic process theory is avoided at the start by substituting random exploration with more intuitive deterministic probing for learning. Once these ideas are understood, it is not difficult to master techniques rooted in stochastic control. These topics are covered in the second part of the book, starting with Markov chain theory and ending with a fresh look at actor-critic methods for reinforcement learning.

Outliers play an important, though underestimated, role in control engineering. Traditionally they are unseen and neglected. In opposition, industrial practice gives frequent examples of their existence and their mostly negative impacts on the control quality. The origin of outliers is never fully known. Some of them are generated externally to the process (exogenous), like for instance erroneous observations, data corrupted by control systems or the effect of human intervention. Such outliers appear occasionally with some unknow probability shifting real value often to some strange and nonsense value. They are frequently called deviants, anomalies or contaminants. In most cases we are interested in their detection and removal. However, there exists the second kind of outliers. Quite often strange looking data observations are not artificial data occurrences. They may be just representatives of the underlying generation mechanism being inseparable internal part of the process (endogenous outliers). In such a case they are not wrong and should be treated with cautiousness, as they may include important information about the dynamic nature of the process. As such they cannot be neglected nor simply removed. The Outlier should be detected, labelled and suitably treated. These activities cannot be performed without proper analytical tools and modeling approaches. There are dozens of methods proposed by scientists, starting from Gaussian-based statistical scoring up to data mining artificial intelligence tools. The research presented in this book presents novel approach incorporating non-Gaussian statistical tools and fractional calculus approach revealing new data analytics applied to this important and challenging task. The proposed book includes a collection of contributions addressing different yet cohesive subjects, like dynamic modelling, classical control, advanced control, fractional calculus, statistical analytics focused on an ultimate goal: robust and outlier-proof analysis. All studied problems show that outliers play an important role and classical methods, in which outlier are not taken into account, do not give good results. Applications from different engineering areas are considered such as semiconductor process control and monitoring, MIMO peltier temperature control and health monitoring, networked control systems, and etc.

This book provides a brief insight of various challenges and its mitigation techniques in microgrid due to power quality (PQ) issues. The central concept of this book revolves around the PQ issues in microgrid. The main objective of this book is to make aware of the power and control engineers with different innovative techniques to mitigate the challenges due to PQ issues in microgrid. The topics covered in this book are PQ disturbances in microgrid and different recent and innovative schemes to mitigate them. The book emphasizes technical issues, theoretical background, and practical applications that drive postgraduates, researchers, and practicing engineers with right advanced skills, vision, and knowledge in finding microgrid power quality issues, various technical challenges and providing mitigation techniques for the future sustainable microgrids.

Next, we present a new stochastic optimization algorithm based on the currently popular Simultaneously Perturbed Stochastic Approximation. The reason for the new algorithm is that in some constrained problems the standard SPSA may exhibit slow convergence due to "iterate bouncing against the constraints." We discuss the nature of this phenomenon, and present a modified SPSA algorithm that does not suffer from it.

Photovoltaic Power System: Modelling, Design and Control is an essential reference with a practical approach to photovoltaic (PV) power system analysis and control. It systematically guides readers through PV system design, modelling, simulation, maximum power point tracking and control techniques making this invaluable resource to students and professionals progressing from different levels in PV power engineering. The development of this book follows the author's 15-year experience as an electrical engineer in the PV engineering sector and as an educator in academia. It provides the background knowledge of PV power system but will also inform research direction. Key features: Details modern converter topologies and a step-by-step modelling approach to simulate and control a complete PV power system. Introduces industrial standards, regulations, and electric codes for safety practice and research direction. Covers new classification of PV power systems in terms of the level of maximum power point tracking. Contains practical examples in designing grid-tied and standalone PV power systems. Matlab codes and Simulink models featured on a Wiley hosted book companion website.