In August 1995, I commissioned a one year study with the objective of assessing the effectiveness of the use of modeling and simulation (M & S) in the acquisition process. In particular, I was looking for the metrics by which the Department of Defense could ascertain the value-if any-that was returned on its investment in M & S in terms of the reduction in time, resources, and risk in weapons systems development and fielding and in terms of increase in the military utility of those systems. I also tasked the study team while gathering information to support the assessment to note technical and other challenges to realizing the postulated benefits and to report on specific M & S tools and processes being used to facilitate the acquisition of systems in the DoD and industry. This report, which documents the results of that study, provides tangible, quantitative indicators that the use of M & S can provide substantial benefit measured in time, cost, productivity, and system quality and performance. The evidence is consistent and pervasive, across both DoD and industry. I personally was impressed that the most significant return on investment was realized when M & S was used as an integrator of functions within the acquisition process, i.e., integrating design and manufacturing or linking requirements more closely to test. This leads me to believe that its real value lies as an enabler of Integrated Product and Process Development (IPPD).

The Committee on Modeling and Simulation Enhancements for 21st Century Manufacturing and Acquisition was formed by the NRC in response to a request from the Defense Modeling and Simulation Office (DMSO) of DOD. The committee was asked to (1) investigate next-generation evolutionary and revolutionary M&S capabilities that will support enhanced defense systems acquisition; (2) identify specific emerging design, testing, and manufacturing process technologies that can be enabled by advanced M&S capabilities; (3) relate these emerging technologies to long-term DOD requirements; (4) assess ongoing efforts to develop advanced M&S capabilities and identify gaps that must be filled to make the emerging technologies a reality; (5) identify lessons learned from industry; and (6) recommend specific government actions to expedite development and to enable maximum DOD and U.S. commercial benefit from these capabilities. To complete its task, the committee identified relevant trends and their impact on defense acquisition needs; current use and support for use of M&S within DOD; lessons learned from commercial manufacturing; three cross-cutting and especially challenging uses of M&S technologies; and the areas in which basic research is needed in M&S in order to achieve the desired goals for manufacturing and defense acquisition.

The Committee on Modeling and Simulation Enhancements for 21st Century Manufacturing and Acquisition was formed by the NRC in response to a request from the Defense Modeling and Simulation Office (DMSO) of DOD. The committee was asked to (1) investigate next-generation evolutionary and revolutionary M&S capabilities that will support enhanced defense systems acquisition; (2) identify specific emerging design, testing, and manufacturing process technologies that can be enabled by advanced M&S capabilities; (3) relate these emerging technologies to long-term DOD requirements; (4) assess ongoing efforts to develop advanced M&S capabilities and identify gaps that must be filled to make the emerging technologies a reality; (5) identify lessons learned from industry; and (6) recommend specific government actions to expedite development and to enable maximum DOD and U.S. commercial benefit from these capabilities. To complete its task, the committee identified relevant trends and their impact on defense acquisition needs; current use and support for use of M&S within DOD; lessons learned from commercial manufacturing; three cross-cutting and especially challenging uses of M&S technologies; and the areas in which basic research is needed in M&S in order to achieve the desired goals for manufacturing and defense acquisition.

During the past decade and a half, the National Research Council, through its Committee on National Statistics, has carried out a number of studies on the application of statistical methods to improve the testing and development of defense systems. These studies were intended to provide advice to the Department of Defense (DOD), which sponsored these studies. The previous studies have been concerned with the role of statistical methods in testing and evaluation, reliability practices, software methods, combining information, and evolutionary acquisition. Industrial Methods for the Effective Testing and Development of Defense Systems is the latest in a series of studies, and unlike earlier studies, this report identifies current engineering practices that have proved successful in industrial applications for system development and testing. This report explores how developmental and operational testing, modeling and simulation, and related techniques can improve the development and performance of defense systems, particularly techniques that have been shown to be effective in industrial applications and are likely to be useful in defense system development. In addition to the broad issues, the report identifies three specific topics for its focus: finding failure modes earlier, technology maturity, and use of all relevant information for operational assessments.

For every weapons system being developed, the U.S. Department of Defense (DOD) must make a critical decision: Should the system go forward to full-scale production? The answer to that question may involve not only tens of billions of dollars but also the nation's security and military capabilities. In the milestone process used by DOD to answer the basic acquisition question, one component near the end of the process is operational testing, to determine if a system meets the requirements for effectiveness and suitability in realistic battlefield settings. Problems discovered at this stage can cause significant production delays and can necessitate costly system redesign. This book examines the milestone process, as well as the DOD's entire approach to testing and evaluating defense systems. It brings to the topic of defense acquisition the application of scientific statistical principles and practices.