Sequential quadratic optimization (SQP) methods are widely used to solve large-scale nonlinear optimization problems. We build two matrix-free methods for approximately solving exact penalty subproblems that arise when using SQP methods to solve large-scale optimization problems. The first approach is a novel iterative re-weighting algorithm. The second approach is based on alternating direction augmented Lagrangian technology applied to our setting. We prove that both algorithms are globally convergent under loose assumptions. SQP methods can be plagued by poor behavior of the global convergence mechanisms. Here we consider global convergence results that use an exact penalty function to compute step sizes. To confront this issue, we propose a dynamic penalty parameter updating strategy to be employed within the subproblem solver in such a way that the resulting search direction predicts progress toward both feasibility and optimality. We prove that does not decrease the penalty parameter unnecessarily in the neighborhood of points satisfying certain common assumptions. We also discuss a coordinate descent subproblem solver in which our updating strategy can be readily incorporated. In the final application of the thesis, we consider a block coordinate descent (BCD) method applied to graphical model learning with special structures, in particular, hub structure and latent variable selection. We tackle the issue of maintaining the positive definiteness of covariance matrices for general rank 2 updates. An active set strategy is employed to speed up BCD for hub structure problem. For latent variable selection problems, we propose a method for maintaining a low rank factorization for the covariance matrix while preserving the convexity of the subproblems for SBCD. We show that our proposed method converges to a stationary point of a non-convex formulation. Extensive numerical experiments are discussed for both models.

With contributions by specialists in optimization and practitioners in the fields of aerospace engineering, chemical engineering, and fluid and solid mechanics, the major themes include an assessment of the state of the art in optimization algorithms as well as challenging applications in design and control, in the areas of process engineering and systems with partial differential equation models.

Finally, systematic ways of generating a nonsingular basis for general nonlinear programs must be developed if this strategy is to be adapted to solve large, sparse problems efficiently. To deal with these problems, a two phase LP-based procedure is coupled to the RND algorithm. This strategy also serves to partition the variables into decisions and dependents, thereby generating a nonsingular basis. Any redundancies/degeneracies in the constraints are also detected and processed separately. The entire reduced SQP implementation is then interfaced with GAMS (Brooke et al. (1988)), a front end for representing and solving process models.

In its thousands of years of history, mathematics has made an extraordinary ca reer. It started from rules for bookkeeping and computation of areas to become the language of science. Its potential for decision support was fully recognized in the twentieth century only, vitally aided by the evolution of computing and communi cation technology. Mathematical optimization, in particular, has developed into a powerful machinery to help planners. Whether costs are to be reduced, profits to be maximized, or scarce resources to be used wisely, optimization methods are available to guide decision making. Opti mization is particularly strong if precise models of real phenomena and data of high quality are at hand - often yielding reliable automated control and decision proce dures. But what, if the models are soft and not all data are around? Can mathematics help as well? This book addresses such issues, e. g. , problems of the following type: - An elevator cannot know all transportation requests in advance. In which order should it serve the passengers? - Wing profiles of aircrafts influence the fuel consumption. Is it possible to con tinuously adapt the shape of a wing during the flight under rapidly changing conditions? - Robots are designed to accomplish specific tasks as efficiently as possible. But what if a robot navigates in an unknown environment? - Energy demand changes quickly and is not easily predictable over time. Some types of power plants can only react slowly.

This book reviews and discusses recent advances in the development of methods and algorithms for nonlinear optimization and its applications, focusing on the large-dimensional case, the current forefront of much research. Individual chapters, contributed by eminent authorities, provide an up-to-date overview of the field from different and complementary standpoints, including theoretical analysis, algorithmic development, implementation issues and applications.

On February 15-17, 1993, a conference on Large Scale Optimization, hosted by the Center for Applied Optimization, was held at the University of Florida. The con ference was supported by the National Science Foundation, the U. S. Army Research Office, and the University of Florida, with endorsements from SIAM, MPS, ORSA and IMACS. Forty one invited speakers presented papers on mathematical program ming and optimal control topics with an emphasis on algorithm development, real world applications and numerical results. Participants from Canada, Japan, Sweden, The Netherlands, Germany, Belgium, Greece, and Denmark gave the meeting an important international component. At tendees also included representatives from IBM, American Airlines, US Air, United Parcel Serice, AT & T Bell Labs, Thinking Machines, Army High Performance Com puting Research Center, and Argonne National Laboratory. In addition, the NSF sponsored attendance of thirteen graduate students from universities in the United States and abroad. Accurate modeling of scientific problems often leads to the formulation of large scale optimization problems involving thousands of continuous and/or discrete vari ables. Large scale optimization has seen a dramatic increase in activities in the past decade. This has been a natural consequence of new algorithmic developments and of the increased power of computers. For example, decomposition ideas proposed by G. Dantzig and P. Wolfe in the 1960's, are now implement able in distributed process ing systems, and today many optimization codes have been implemented on parallel machines.