The Department of Defense (DOD) has proposed that the practices and policies of the Office of the Director of Operational Test and Evaluation (DOT&E) be modified to reduce the time and cost of developing and fielding new weapon systems. To help focus deliberations on DOD'S proposal, you asked us to review DOT&E'S operations and organizational structure for overseeing operational testing. Specifically, you asked us to assess (1) DOT&E's efforts and their impact on the quality of operational testing and evaluation in DOD and (2) the strengths and weaknesses of the current organizational framework in DOD for operational testing. As part of our review, we conducted 13 case studies of the testing of individual weapon systems. (Our scope and methodology are described in app. I, and brief descriptions of the 13 weapon systems are provided in app. II.)

For every weapons system being developed, the U.S. Department of Defense (DOD) must make a critical decision: Should the system go forward to full-scale production? The answer to that question may involve not only tens of billions of dollars but also the nation's security and military capabilities. In the milestone process used by DOD to answer the basic acquisition question, one component near the end of the process is operational testing, to determine if a system meets the requirements for effectiveness and suitability in realistic battlefield settings. Problems discovered at this stage can cause significant production delays and can necessitate costly system redesign. This book examines the milestone process, as well as the DOD's entire approach to testing and evaluating defense systems. It brings to the topic of defense acquisition the application of scientific statistical principles and practices.