A comprehensive introduction to optimization with a focus on practical algorithms for the design of engineering systems. This book offers a comprehensive introduction to optimization with a focus on practical algorithms. The book approaches optimization from an engineering perspective, where the objective is to design a system that optimizes a set of metrics subject to constraints. Readers will learn about computational approaches for a range of challenges, including searching high-dimensional spaces, handling problems where there are multiple competing objectives, and accommodating uncertainty in the metrics. Figures, examples, and exercises convey the intuition behind the mathematical approaches. The text provides concrete implementations in the Julia programming language. Topics covered include derivatives and their generalization to multiple dimensions; local descent and first- and second-order methods that inform local descent; stochastic methods, which introduce randomness into the optimization process; linear constrained optimization, when both the objective function and the constraints are linear; surrogate models, probabilistic surrogate models, and using probabilistic surrogate models to guide optimization; optimization under uncertainty; uncertainty propagation; expression optimization; and multidisciplinary design optimization. Appendixes offer an introduction to the Julia language, test functions for evaluating algorithm performance, and mathematical concepts used in the derivation and analysis of the optimization methods discussed in the text. The book can be used by advanced undergraduates and graduate students in mathematics, statistics, computer science, any engineering field, (including electrical engineering and aerospace engineering), and operations research, and as a reference for professionals.

Noise is a common factor in most real-world optimization problems. Sources of noise can include physical measurement limitations, stochastic simulation models, incomplete sampling of large spaces, and human-computer interaction. Evolutionary algorithms are general, nature-inspired heuristics for numerical search and optimization that are frequently observed to be particularly robust with regard to the effects of noise. Noisy Optimization with Evolution Strategies contributes to the understanding of evolutionary optimization in the presence of noise by investigating the performance of evolution strategies, a type of evolutionary algorithm frequently employed for solving real-valued optimization problems. By considering simple noisy environments, results are obtained that describe how the performance of the strategies scales with both parameters of the problem and of the strategies considered. Such scaling laws allow for comparisons of different strategy variants, for tuning evolution strategies for maximum performance, and they offer insights and an understanding of the behavior of the strategies that go beyond what can be learned from mere experimentation. This first comprehensive work on noisy optimization with evolution strategies investigates the effects of systematic fitness overvaluation, the benefits of distributed populations, and the potential of genetic repair for optimization in the presence of noise. The relative robustness of evolution strategies is confirmed in a comparison with other direct search algorithms. Noisy Optimization with Evolution Strategies is an invaluable resource for researchers and practitioners of evolutionary algorithms.

This book is designed as a textbook, suitable for self-learning or for teaching an upper-year university course on derivative-free and blackbox optimization. The book is split into 5 parts and is designed to be modular; any individual part depends only on the material in Part I. Part I of the book discusses what is meant by Derivative-Free and Blackbox Optimization, provides background material, and early basics while Part II focuses on heuristic methods (Genetic Algorithms and Nelder-Mead). Part III presents direct search methods (Generalized Pattern Search and Mesh Adaptive Direct Search) and Part IV focuses on model-based methods (Simplex Gradient and Trust Region). Part V discusses dealing with constraints, using surrogates, and bi-objective optimization. End of chapter exercises are included throughout as well as 15 end of chapter projects and over 40 figures. Benchmarking techniques are also presented in the appendix.

This book constitutes the refereed proceedings of the 11th International Conference on Artificial Evolution, EA 2013, held in Bordeaux, France, in October 2013. The 20 revised papers were carefully reviewed and selected from 39 submissions. The papers are focused to theory, ant colony optimization, applications, combinatorial and discrete optimization, memetic algorithms, genetic programming, interactive evolution, parallel evolutionary algorithms, and swarm intelligence.

This volume contains the proceedings of the Second International Workshop on Optimal Design and Control, held in Arlington, Virginia, 30 September-3 Octo ber, 1997. The First Workshop was held in Blacksburg, Virginia in 1994. The proceedings of that meeting also appeared in the Birkhauser series on Progress in Systems and Control Theory and may be obtained through Birkhauser. These workshops were sponsored by the Air Force Office of Scientific Re search through the Center for Optimal Design and Control (CODAC) at Vrrginia Tech. The meetings provided a forum for the exchange of new ideas and were designed to bring together diverse viewpoints and to highlight new applications. The primary goal of the workshops was to assess the current status of research and to analyze future directions in optimization based design and control. The present volume contains the technical papers presented at the Second Workshop. More than 65 participants from 6 countries attended the meeting and contributed to its success. It has long been recognized that many modern optimal design problems are best viewed as variational and optimal control problems. Indeed, the famous problem of determining the body of revolution that produces a minimum drag nose shape in hypersonic How was first proposed by Newton in 1686. Optimal control approaches to design can provide theoretical and computational insight into these problems. This volume contains a number of papers which deal with computational aspects of optimal control.

The first contemporary comprehensive treatment of optimization without derivatives. This text explains how sampling and model techniques are used in derivative-free methods and how they are designed to solve optimization problems. It is designed to be readily accessible to both researchers and those with a modest background in computational mathematics.

Powerful optimization algorithms are key ingredients in science and engineering applications. In this thesis, we develop a novel class of discrete-time, derivative-free optimization algorithms relying on gradient approximations based on non-commutative maps–inspired by Lie bracket approximation ideas in control systems. Those maps are defined by function evaluations and applied in such a way that gradient descent steps are approximated, and semi-global convergence guarantees can be given. We supplement our theoretical findings with numerical results. Therein, we provide several algorithm parameter studies and tuning rules, as well as the results of applying our algorithm to challenging benchmarking problems.