The Department of Defense (DoD) recently adopted evolutionary acquisition, a dynamic strategy for the development and acquisition of its defense systems. Evolutionary defense systems are planned, in advance, to be developed through several stages in a single procurement program. Each stage is planned to produce a viable system which could be fielded. The system requirements for each stage of development may be specified in advance of a given stage or may be decided at the outset of that stage's development. Due to the different stages that comprise an evolutionary system, there exists a need for careful reexamination of current testing and evaluation policies and processes, which were designed for single-stage developments. The Office of the Under Secretary of Defense for Acquisition, Technology and Logistics (USD-AT&L) and the Director of Operational Testing and Evaluation (DOT&E) asked the Committee on National Statistics (CNSTAT) of the National Academies to examine the key issues and implications for defense testing from the introduction of evolutionary acquisition. The CNSTAT was charged with planning and conducting a workshop to study test strategies for the evolutionary acquisition. The committee reviewed defense materials defining evolutionary acquisition and interviewed test officials from the three major test service agencies to understand the current approaches used in testing systems procured through evolutionary acquisition. The committee also examined possible alternatives to identify problems in implementation. At the workshop that took place on December 13-14, 2004, the committee tried to answer many questions including: What are the appropriate roles and objectives for testing in an evolutionary environment?, Can a systematic, disciplined process be developed for testing and evaluation in such a fluid and flexible environment?, and Is there adequate technical expertise within the acquisition community to fully exploit data gathered from previous stages to effectively combine information from various sources for test design and analysis?. Testing of Defense Systems in an Evolutionary Acquisition Environment provides the conclusions and recommendations of the CNSTAT following the workshop and its other investigations.

The Department of Defense (DoD) recently adopted evolutionary acquisition, a dynamic strategy for the development and acquisition of its defense systems. Evolutionary defense systems are planned, in advance, to be developed through several stages in a single procurement program. Each stage is planned to produce a viable system which could be fielded. The system requirements for each stage of development may be specified in advance of a given stage or may be decided at the outset of that stage's development. Due to the different stages that comprise an evolutionary system, there exists a need for careful reexamination of current testing and evaluation policies and processes, which were designed for single-stage developments. The Office of the Under Secretary of Defense for Acquisition, Technology and Logistics (USD-AT&L) and the Director of Operational Testing and Evaluation (DOT&E) asked the Committee on National Statistics (CNSTAT) of the National Academies to examine the key issues and implications for defense testing from the introduction of evolutionary acquisition. The CNSTAT was charged with planning and conducting a workshop to study test strategies for the evolutionary acquisition. The committee reviewed defense materials defining evolutionary acquisition and interviewed test officials from the three major test service agencies to understand the current approaches used in testing systems procured through evolutionary acquisition. The committee also examined possible alternatives to identify problems in implementation. At the workshop that took place on December 13-14, 2004, the committee tried to answer many questions including: What are the appropriate roles and objectives for testing in an evolutionary environment?, Can a systematic, disciplined process be developed for testing and evaluation in such a fluid and flexible environment?, and Is there adequate technical expertise within the acquisition community to fully exploit data gathered from previous stages to effectively combine information from various sources for test design and analysis?. Testing of Defense Systems in an Evolutionary Acquisition Environment provides the conclusions and recommendations of the CNSTAT following the workshop and its other investigations.

During the past decade and a half, the National Research Council, through its Committee on National Statistics, has carried out a number of studies on the application of statistical methods to improve the testing and development of defense systems. These studies were intended to provide advice to the Department of Defense (DOD), which sponsored these studies. The previous studies have been concerned with the role of statistical methods in testing and evaluation, reliability practices, software methods, combining information, and evolutionary acquisition. Industrial Methods for the Effective Testing and Development of Defense Systems is the latest in a series of studies, and unlike earlier studies, this report identifies current engineering practices that have proved successful in industrial applications for system development and testing. This report explores how developmental and operational testing, modeling and simulation, and related techniques can improve the development and performance of defense systems, particularly techniques that have been shown to be effective in industrial applications and are likely to be useful in defense system development. In addition to the broad issues, the report identifies three specific topics for its focus: finding failure modes earlier, technology maturity, and use of all relevant information for operational assessments.

In the military, information technology (IT) has enabled profound advances in weapons systems and the management and operation of the defense enterprise. A significant portion of the Department of Defense (DOD) budget is spent on capabilities acquired as commercial IT commodities, developmental IT systems that support a broad range of warfighting and functional applications, and IT components embedded in weapons systems. The ability of the DOD and its industrial partners to harness and apply IT for warfighting, command and control and communications, logistics, and transportation has contributed enormously to fielding the world's best defense force. However, despite the DOD's decades of success in leveraging IT across the defense enterprise, the acquisition of IT systems continues to be burdened with serious problems. To address these issues, the National Research Council assembled a group of IT systems acquisition and T&E experts, commercial software developers, software engineers, computer scientists and other academic researchers. The group evaluated applicable legislative requirements, examined the processes and capabilities of the commercial IT sector, analyzed DOD's concepts for systems engineering and testing in virtual environments, and examined the DOD acquisition environment. The present volume summarizes this analysis and also includes recommendations on how to improve the acquisition, systems engineering, and T&E processes to achieve the DOD's network-centric goals.

Since 1992, the Committee on National Statistics (CNSTAT) has produced a book on principles and practices for a federal statistical agency, updating the document every 4 years to provide a current edition to newly appointed cabinet secretaries at the beginning of each presidential administration. This fourth edition presents and comments on four basic principles that statistical agencies must embody in order to carry out their mission fully: (1) They must produce objective data that are relevant to policy issues, (2) they must achieve and maintain credibility among data users, (3) they must achieve and maintain trust among data providers, and (4) they must achieve and maintain a strong position of independence from the appearance and reality of political control. The book also discusses 11 important practices that are means for statistical agencies to live up to the four principles. These practices include a commitment to quality and professional practice and an active program of methodological and substantive research. This fourth edition adds the principle that statistical agencies must operate from a strong position of independence and the practice that agencies must have ongoing internal and external evaluations of their programs.

"So far, EA implementation of military space programs has produced mixed results. The capabilities and requirements definition and management processes are major challenges in all EA programs. EA programs require an evolutionary costing approach; most cost analysts interviewed expressed generally positive views about EA."--BOOK JACKET.