Submodular functions are relevant to machine learning for at least two reasons: (1) some problems may be expressed directly as the optimization of submodular functions and (2) the Lovász extension of submodular functions provides a useful set of regularization functions for supervised and unsupervised learning. In this monograph, we present the theory of submodular functions from a convex analysis perspective, presenting tight links between certain polyhedra, combinatorial optimization and convex optimization problems. In particular, we show how submodular function minimization is equivalent to solving a wide variety of convex optimization problems. This allows the derivation of new efficient algorithms for approximate and exact submodular function minimization with theoretical guarantees and good practical performance. By listing many examples of submodular functions, we review various applications to machine learning, such as clustering, experimental design, sensor placement, graphical model structure learning or subset selection, as well as a family of structured sparsity-inducing norms that can be derived and used from submodular functions.

It has widely been recognized that submodular functions play essential roles in efficiently solvable combinatorial optimization problems. Since the publication of the 1st edition of this book fifteen years ago, submodular functions have been showing further increasing importance in optimization, combinatorics, discrete mathematics, algorithmic computer science, and algorithmic economics, and there have been made remarkable developments of theory and algorithms in submodular functions. The 2nd edition of the book supplements the 1st edition with a lot of remarks and with new two chapters: "Submodular Function Minimization" and "Discrete Convex Analysis." The present 2nd edition is still a unique book on submodular functions, which is essential to students and researchers interested in combinatorial optimization, discrete mathematics, and discrete algorithms in the fields of mathematics, operations research, computer science, and economics. - Self-contained exposition of the theory of submodular functions - Selected up-to-date materials substantial to future developments - Polyhedral description of Discrete Convex Analysis - Full description of submodular function minimization algorithms - Effective insertion of figures - Useful in applied mathematics, operations research, computer science, and economics

Active learning is a machine learning setting where the learning algorithm decides what data is labeled. Submodular functions are a class of set functions for which many optimization problems have efficient exact or approximate algorithms. We examine their connections. 1. We propose a new class of interactive submodular optimization problems which connect and generalize submodular optimization and active learning over a finite query set. We derive greedy algorithms with approximately optimal worst-case cost. These analyses apply to exact learning, approximate learning, learning in the presence of adversarial noise, and applications that mix learning and covering. 2. We consider active learning in a batch, transductive setting where the learning algorithm selects a set of examples to be labeled at once. In this setting we derive new error bounds which use symmetric submodular functions for regularization, and we give algorithms which approximately minimize these bounds. 3. We consider a repeated active learning setting where the learning algorithm solves a sequence of related learning problems. We propose an approach to this problem based on a new online prediction version of submodular set cover. A common theme in these results is the use of tools from submodular optimization to extend the breadth and depth of learning theory with an emphasis on non-stochastic settings.

This book presents a framework for the control of networked systems utilizing submodular optimization techniques. The main focus is on selecting input nodes for the control of networked systems, an inherently discrete optimization problem with applications in power system stability, social influence dynamics, and the control of vehicle formations. The first part of the book is devoted to background information on submodular functions, matroids, and submodular optimization, and presents algorithms for distributed submodular optimization that are scalable to large networked systems. In turn, the second part develops a unifying submodular optimization approach to controlling networked systems based on multiple performance and controllability criteria. Techniques are introduced for selecting input nodes to ensure smooth convergence, synchronization, and robustness to environmental and adversarial noise. Submodular optimization is the first unifying approach towards guaranteeing both performance and controllability with provable optimality bounds in static as well as time-varying networks. Throughout the text, the submodular framework is illustrated with the help of numerical examples and application-based case studies in biological, energy and vehicular systems. The book effectively combines two areas of growing interest, and will be especially useful for researchers in control theory, applied mathematics, networking or machine learning with experience in submodular optimization but who are less familiar with the problems and tools available for networked systems (or vice versa). It will also benefit graduate students, offering consistent terminology and notation that greatly reduces the initial effort associated with beginning a course of study in a new area.

In the late forties, Mathematical Programming became a scientific discipline in its own right. Since then it has experienced a tremendous growth. Beginning with economic and military applications, it is now among the most important fields of applied mathematics with extensive use in engineering, natural sciences, economics, and biological sciences. The lively activity in this area is demonstrated by the fact that as early as 1949 the first "Symposium on Mathe matical Programming" took place in Chicago. Since then mathematical programmers from all over the world have gath ered at the intfrnational symposia of the Mathematical Programming Society roughly every three years to present their recent research, to exchange ideas with their colleagues and to learn about the latest developments in their own and related fields. In 1982, the XI. International Symposium on Mathematical Programming was held at the University of Bonn, W. Germany, from August 23 to 27. It was organized by the Institut fUr Okonometrie und Operations Re search of the University of Bonn in collaboration with the Sonderforschungs bereich 21 of the Deutsche Forschungsgemeinschaft. This volume constitutes part of the outgrowth of this symposium and docu ments its scientific activities. Part I of the book contains information about the symposium, welcoming addresses, lists of committees and sponsors and a brief review about the Ful kerson Prize and the Dantzig Prize which were awarded during the opening ceremony.

This book is dedicated to Jack Edmonds in appreciation of his ground breaking work that laid the foundations for a broad variety of subsequent results achieved in combinatorial optimization.The main part consists of 13 revised full papers on current topics in combinatorial optimization, presented at Aussois 2001, the Fifth Aussois Workshop on Combinatorial Optimization, March 5-9, 2001, and dedicated to Jack Edmonds.Additional highlights in this book are an account of an Aussois 2001 special session dedicated to Jack Edmonds including a speech given by William R. Pulleyblank as well as newly typeset versions of three up-to-now hardly accessible classical papers:- Submodular Functions, Matroids, and Certain Polyhedranbsp;nbsp; by Jack Edmonds- Matching: A Well-Solved Class of Integer Linear Programsnbsp;nbsp; by Jack Edmonds and Ellis L. Johnson- Theoretical Improvements in Algorithmic Efficiency for Network Flow Problemsnbsp;nbsp; by Jack Edmonds and Richard M. Karp.

There is a strong case for electrical network topologists and submodular function theorists being aware of each other's fields. Presenting a topological approach to electrical network theory, this book demonstrates the strong links that exist between submodular functions and electrical networks. The book contains: . a detailed discussion of graphs, matroids, vector spaces and the algebra of generalized minors, relevant to network analysis (particularly to the construction of efficient circuit simulators) . a detailed discussion of submodular function theory in its own right; topics covered include, various operations, dualization, convolution and Dilworth truncation as well as the related notions of prinicpal partition and principal lattice of partitions. In order to make the book useful to a wide audience, the material on electrical networks and that on submodular functions is presented independently of each other. The hybrid rank problem, the bridge between (topological) electrical network theory and submodular functions, is covered in the final chapter. The emphasis in the book is on low complexity algorithms, particularly based on bipartite graphs. The book is intended for self-study and is recommended to designers of VLSI algorithms. More than 300 problems, almost all of them with solutions, are included at the end of each chapter.