DoD revamps testing organization

The Automatic Test System (ATS) Executive Directorate (ED) is charged with developing objectives for the Department of Defense (DoD) ATS modernization. The DoD ATS Framework Working Group (FWG) was established by the ATS ED to create a Framework addressing these objectives. The FWG consists of two sub-groups; Management, that includes members from all of the DoD Services, and Technical. The Framework is made up of key elements and associated specifications and standards that form the open system architecture approach for DoD ATSs. In addition to creating and updating key element definitions, the FWG focuses on identifying commercial ATS related specifications that might satisfy the elements in the Framework. The standards specified in the DoD ATS Framework are documented in the DoD's Information Technology Standards Repository (DISR) and thereby become a mandatory requirement for all DoD ATS acquisitions. The Framework is also being specifically called out in DoD ATS requirements documents so that its standards are incorporated in future DoD ATS acquisitions, such as the Navy's eCASS. Although the ATS Framework was originally defined more than 10 years ago, it continues to evolve as test technology evolves. A function of the FWG is to continually assess the Framework and plan the development and demonstration of specifications and standards to satisfy ATS Framework requirements. The FWG is currently participating in the standards development process via standards organizations, consortia, and working groups. The FWG is evaluating these standards in terms of applicability to the twenty plus key elements defined in the Framework. The FWG is also continually evaluating new test and information technologies to further refine and add new key elements. When necessary, the FWG provides requirements and potential solutions to standards development teams so that the resulting standard will fit within the Framework. When these standards are published and proven to meet the requirements of a particular Framework element, the FWG makes recommendations to the DoD's ATS Management Board (AMB) regarding how these standards should be represented in the DISR. Standards which are mandated will be required by the acquisition process, and vendors' products acquired by the DoD must conform to them. This paper will provide a summary of the current activities and issues impacting the ATS Framework with a discussion of the near term plans of the FWG. It will provide status on the key elements that the FWG members are currently working on. These elements include the Adapter Functional and Parametric Data (AFP), Diagnostic Data (DIAD), Diagnostic Services (DIAS), Instrument Functional and Parametric Data (IFP), Master Conformance Index (MCI), Maintenance Test Data (MTD), Resource Adapter Interface (RAI), Run Time Services (RTS), Prognostics and Health Management (PHM), and Test Station Functional and Parametric Data (TSFP). The standards that satisfy or partially satisfy these key elements, and the current status of these standards are discussed. The paper also describes the purpose of, and the FWG's role in, several industry working groups; including LAN extensions for Instrumentation (LXI), Institute of Electrical and Electronics Engineers (IEEE) Standards Coordinating Committee 20 (SCC20), Interchangeable Virtual Instruments (IVI), and Synthetic Instruments Working Group (SIWG). Finally, current issues, such as how signal modeling is to be incorporated within the Framework, the relationship to the DoD's efforts with those of the UK Ministry of Defense (MoD), the use of the DISR mandated elements in upcoming DoD ATS acquisitions, and how O-level test capabilities could be incorporated in the Framework are discussed.

The Automatic Test System (ATS) Executive Directorate (ED) is charged with developing objectives for the Department of Defense (DoD) ATS modernization. The DoD ATS Framework Working Group (FWG) was established by the ATS ED to create a Framework addressing these objectives. The FWG consists of two sub-groups: Management, that includes members from all of the DoD Services, and Technical. The Framework is made up of key elements and associated specifications and standards that form the open system architecture approach for DoD ATSs. In addition to creating and updating key element definitions, the FWG focuses on identifying commercial ATS related specifications that might satisfy the elements in the Framework. The standards specified in the DoD ATS Framework are documented in the DoD's Information Technology Standards Repository (DISR) and thereby become a mandatory requirement for all DoD ATS acquisitions. The Framework is also being specifically called out in DoD ATS requirements documents so that its standards are incorporated in future DoD ATS acquisitions, such as the Navy's eCASS.

The Versatile Automated Depot Test Station (VDATS) is a major component in solving the U.S. Air Forcepsilas Automatic Test System (ATS) proliferation concerns. VDATS addresses the proliferation issue by providing the capability to replace an estimated 260 depot-level test systems that assume 150 different configurations, as well as a large number of intermediate-level test systems.The Department of Defense (DoD) has invested billions of dollars in aging and increasingly obsolete ATS used to troubleshoot and diagnose components of military aircraft and weapon systems. The issues of increased downtimes, test system obsolescence, and nonportable Test Program Sets (TPS) further contributes to the DoD's continued reporting of spare parts shortages and the increasing impacts that ATS obsolescence has on the readiness of military aircraft and weapon systems. The VDATS provides a viable approach to resolving these issues. VDATS has an extensible designed-in expansion capability intended to support present and future Air Force workloads by providing an organically-developed ATS platform with a modular, open-systems architecture that is Flexible, Capable, Tailorable, and Sustainable. VDATS makes it technically feasible to now realize the laudable 1994 DoD policy goal stated in: Military Readiness: DoD Needs to Better Manage Automatic Test Equipment Modernization," GAO-03-451, March 2003.[1] VDATS meets this stated DoD policy goal of producing an ATS capable of supporting multiple weapon systems, aircraft, and eventually being interoperable between DoD services. In 2007 the Air Force's VDATS was officially designated as a DoD Standard Family of Testers, supporting the 2005 ATS Master Plan requirement meant to reduce ATS proliferation and life-cycle operations and support costs. This paper briefly examines the present-day issues and how the U.S. Air Force is resolving these with the VDATS program. It also explains the strategy to preserve present-day and near-future TPS re- host investments, while enhancing system capabilities to meet the Air Force's depot and intermediate-level ATS requirements for the foreseeable future.

For decades, the automatic test community has worked to develop hardware standardization and interface protocols that aid in mitigating the impact of equipment obsolescence on the repair and maintenance mission. The traditional view of Test Program Set (TPS) transportability in this environment has always been one of migrating a legacy TPS to a new hardware architecture design born out of a need to replace aging technology or provide new test capability. The Department of Defense (DoD) has recognized the benefit of standardizing automatic test systems (ATS) through policy guidance whose intent is to drive the services to a common configuration test environment, thereby reducing life cycle sustainment costs for all DoD systems. Today, hardware standardization has advanced to the point that many ATS look and function very similar to one another. However, there are two major hurdles that continuously thwart our efforts to develop a standard core test system, the lack of a standard UUT interface and the lack of an open architecture software design that facilitates operations in any environment and on any test system architecture. Automatic Test Markup Language (ATML) provides promise with regards to the development of standardized test software interfaces. Developing a standard hardware configuration for all test requirements is ultimately achievable with the technology currently available today except for one simple but hugely significant hurdle, the cost involved in re-hosting the myriad of test program sets currently in existence. Therefore, any significant advance in common interface design has to take into account the need to minimize the impact to existing test program sets. This paper will seek to explore the various technological possibilities for overcoming the barriers to true TPS transportability that exist in today's automatic test community.

The purpose of this paper is to present a conceptual approach and to make practical recommendations on how to improve the current Automatic Test Equipment (ATE) testing and maintenance capabilities by utilizing the existing cloud computing model to build a globally linked ATE maintenance system. The basic tenet of the ATE community is to support a multi-tiered maintenance concept which, in general, is a three tiered system that is composed of organizational maintenance (O-level), intermediate maintenance (I-level), and depot maintenance (D-level) organizations. The goal of the ATE is to (1) quickly and accurately detect and isolate each fault, (2) provide software tools for analyzing historical data, and (3) gather, manage, and distribute accurate and reliable maintenance information for the failed Unit Under Test (UUT). The ATE system should provide services that will (1) maintain a repository of information that will improve fault detection and isolation, allow for off-platform assessments, document failures, and help quantify corrective actions, (2) reduce false UUT pulls, and (3) reduce repair time by prompting repair procedures. Furthermore, the ATE system should provide additional services that will help optimize the time to diagnose problems by using collected failure information and by recommending entry points into the Test Program Set (TPS) software. It should also present information to the ATE maintainer to aid in informed repair decisions which could be in the form of pilot debrief results, platform Built In Test (BIT) results, O-level test outcomes and corrective actions, and maintenance and usage history of the platform and UUT. So, based on this definition of ATE maintenance the use of cloud computing can be used to provide services to improve the overall ATE testing throughput which will result in bottom line improvements to ATE life cycle costs. By using cloud computing, which is defined to be “a model for enabling ubiquitous, convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, servers, storage, applications, and services) that can be rapidly provisioned and released with minimal management effort or service provider interaction”, users can develop cloud computing models that will provide access to application software and databases that can be used to build a globally linked ATE maintenance system. This paper will discuss the essential characteristics of the cloud computing models and define the various flavors of cloud offerings available to designers today. This paper will also analyze the cloud computing model to arrive at a conceptual approach that can be used to enhance the current ATE Testing and Maintenance capabilities. Practical recommendations will be discussed on how to transform the current ATE Testing and Maintenance capabilities into the specific cloud computing model offerings in order to help configure a globally linked ATE maintenance system.

Recent changes in the state-of-the-art in software development and test instrumentation has provided the Department of Defense with opportunities to reduce the cost of its automatic test systems (automatic test equipment and the software and information required to drive that equipment). In order to guide and enhance this evolution, the three Services (USN, USAF, USA) have developed an automatic test system research and development program consisting of three broad areas; automatic test system functional interface convergence, next generation test environment, and automatic test system modernization methods. This program, when funded, will provide the technology necessary to transition to the open architecture test systems of the future. The balance of this paper covers the eleven identified tasks under these broad areas, and how they form a coordinated, integrated basis for an improved acquisition strategy. >

A 1994 study identified that between 1980 and 1992, the services spent approximately $50 billion on automatic test systems (ATS). Controlling proliferation of ATS became the DoD's goal and led to policy which stated that DoD programs shall select ATS families to meet all ATS acquisition needs, and that introduction of unique ATS must be minimized. Beginning in 1996, DoD and industry together began work toward an open architecture for ATS by identifying the set of critical interfaces or key elements for which standards needed to be defined. Shortly afterward, the Joint Service Nxtest integrated program team began efforts toward evolving test technologies for insertion into DoD ATS. The original architecture has evolved into the DoD ATS framework which is published in the defense information technology standards registry. The joint service ATS framework working group, under the NxTest IPT, is accelerating efforts toward completing the framework. Cooperation between the services and industry has reached new levels with active participation in working groups and standards development bodies. In addition to demonstrating emerging test technologies, the agile rapid global combat support (ARGCS) advance concept technology demonstration project is helping to mature the ATS framework. When the ATS framework is completed and DoD has a truly open architecture for ATS, it is anticipated that DoD's ATS policy may shift from a hardware (families and COTS) perspective to a situation wherein any tester that fully implements the ATS framework must satisfy DoD's ATS-related goals, and the actual hardware used becomes less important.

The electronics industry and the Department of Defense (DoD), has thousands of obsolete legacy automated test systems (ATS). There are many systems, with different hardware and software architectures, that cannot be upgraded. The inability to reliably test products, diagnose faults, and collect historical data is having an effect on mission readiness. This paper describes a test and diagnostic system model that provides a means to use historical test and repair data from all levels of operation. The process reduces rework costs and decreases maintenance and repair costs through earlier and more accurate fault isolation. This work recoups the efforts of the original developer and captures test and diagnostic knowledge for the future. Consequently, the concept has been proposed for implementation within the Army ATS/TPS centers for use at the Army maintenance depots. Additional benefits from this work: development of a reliability database for system, subsystem, component by test type and ATS; tracking system reliability and mission performance data for use in developing requirements for new or upgrade system procurement specifications; and for pushing diagnostic knowledge and support from the sustainment level to the field and vice versa. The work presented will change the process of developing, maintaining and migrating diagnostic test now and into the future.

Rapid growth in the development and production of software-defined radios encourages some countries to manufacture their own radios by the use of new technology transfer agreements. Deployment of the automatic test equipment is a fundamental step of transfer of technology. As the automatic test equipments will be scattered throughout the world, the test systems should be designed accordingly to meet the new needs introduced. Since the cost of maintaining old systems is more expensive, using the existing automatic test systems and updating them according to the newly introduced needs is not an option. This fact encourages us to take a big step and design a completely new automatic test system. This paper introduces the new challenges encountered in the design, deployment and maintenance processes and describes the structure of the developed automatic test system in detail. The system described in this paper covers the typical automatic test equipment units like test and measurement devices, switching units, input/output components, loads and test software. However, the main focus of the paper is on the new test software introduced.

The Department of Defense (DoD) manages capabilities through the Joint Interoperability and Capability Development System (JCIDS) process. As part of this process, sponsors develop a series of DoD Architecture Framework (DoDAF) products to assist analysts understand the proposed capability and how it fits into the broader network of DoD legacy systems and systems under development. However, the Joint Staff, responsible for executing the JCIDS process, often analyzes these architectures in isolation without considering the broader network of systems. DoD leadership, the Government Accountability Organization, and others have noted the lack of the DoD's ability to manage the broader portfolio of capabilities in various reports and papers. Several efforts have proposed merging DoDAF architecture into a larger meta-architecture based on individual system architectures. This paper specifically targets the Systems View 3 (SV-3), System-to-system matrix, as an opportunity to merge multiple DoDAF architecture views into a network of system and understand the potential benefits associated with analyzing a broader perspective. The goal of merging multiple SV-3s is to better understand the interoperability of a system within the network of DoD systems as network metrics may provide insights into the relative interoperability of a DoD system. Currently, the DoD's definition of interoperability focuses on the system or capability's ability to enter and operate within the DoD Information Network (DoDIN); however, this view limits the definition of interoperability as it focuses solely on information flows and not resource flows or physical connections that should be present in a SV-3. The paper demonstrates the importance of including all forms of connections between systems in a network by comparing network metrics associated with the different types of connections. Without a complete set of DoDAF architectures for each system within the DoD and based on the potential classification of these products, the paper collates data that should be included in an SV-3 from open source, unclassified references to build the overall network of DoD systems. From these sources, a network of over 300 systems with almost 1000 connections emerges based on the documented information, resource, and physical connections between these legacy and planned DoD systems. With this network, the paper explores the quantification of individual system's interoperability through the application of nodal and network metrics from social network analysis (SNA). A SNA perspective on a network of systems provides additional insights beyond traditional network analysis because of the emphasis on the importance of nodes, systems, in the network as well as the relationship, connections, between the nodes. Finally, the paper proposes future work to explore the quantification of additional attributes of systems as well as a method for further validating the findings.

: Agile Rapid Global Combat Support (ARGCS) system is a new Automatic Test System being developed through the Department of Defense's Advanced Concept Technology Demonstration program. This business case analyzes ARGCS benefits for the Marine Corps as an option to replace their current Automated Test Systems for ground equipment. The business case analyzes the Marine Corps' current systems, specifically the Marine Corps' Third Echelon Test Set, and quantifies the relevant differences between these systems and the Agile Rapid Global Combat Support System.

This chapter describes the DoD (Department of Defense) policy goal toward Net centric transformation as an example of enterprise architecture in practice. An integrated architecture across the enterprise provided by the DoD Global Information Grid (GIG) is a key ingredient toward meeting Net centricity. The chapter provides background information on key concepts and details the steps necessary to meeting the Net-ready key performance parameter (NR-KPP). The DoD architecture framework provides the Supporting System View, Operational View, and Technical Standards View architecture products that each DoD program must create to meet the Net-centric model. The Net-centric transformation is detailed based on the Net-centric data strategy, Net-centric IA strategy, use of service-oriented architecture, and use of a communications transport strategy. The path toward Net centricity is a significant and long-term effort and the chapter focuses on specific areas that affect DoD programs on their path toward Net-centric compliance. The implementation of enterprise services and the use of key technical standards are also discussed as emerging efforts.

For modern Automatic Test Equipment (ATE) one of the most daunting tasks is now Information Assurance (IA). What was once at most a secondary item consisting mainly of installing an Anti-Virus suite is now becoming one of the most important aspects of ATE. Given the current climate of IA it has become important to ensure ATE is kept safe from any breaches of security or loss of information. Even though most ATE are not on the Internet (or even on a network for many) they are still vulnerable to some of the same attack vectors plaguing common computers and other electronic devices. This paper will discuss some of the processes and procedures which must be used to ensure that modern ATE can continue to be used to test and detect faults in the systems they are designed to test. The common items that must be considered for ATE are as follows: The ATE system must have some form of Anti-Virus (as should all computers). The ATE system should have a minimum software footprint only providing the software needed to perform the task. The ATE system should be verified to have all the Operating System (OS) settings configured pursuant to the task it is intended to perform. The ATE OS settings should include password and password expiration settings to prevent access by anyone not expected to be on the system. The ATE system software should be written and constructed such that it in itself is not readily open to attack. The ATE system should be designed in a manner such that none of the instruments in the system can easily be attacked. The ATE system should insure any paths to the outside world (such as Ethernet or USB devices) are limited to only those required to perform the task it was designed for. These and many other common configuration concerns will be discussed in the paper.

The dial gauge is used widely in the production line at present. The dial gauge has many types and huge quantity. At present, testing this type of instrument is mainly used manual observation method. By man-made factors, the testing efficiency and accuracy is low. In order to improve the test efficiency and accuracy, it is required to use the automatic test methods for dial gauge. Although the control algorithm of control system in automatic test system is simple, the control accuracy is high demanding. So the design of control systems is essential. This paper takes the dial gauge automatic test method for the object and studied on the control system in the automatic test system. It presents the design principles, methods in fully automatic test system and studied on the design of control systems, mechanical drive parts and the software architecture. Finally, through experimental results verify the test system design is reasonable. Practical application results show that the system is feasible, and the motion control system is high precision and higher intellectualized. It also has good application prospect, especially for some special environment which is not suitable for testing by human eye, this system shows its superiority.

The increased reliance on commercial satellites for military operations has made it essential for the Department of Defense (DoD) to adopt a supply chain framework to address cybersecurity threats in space. This paper presents a satellite supply chain framework, the Cybersecurity Supply Chain (CSSC) Framework, for the DoD in the evaluation and selection of commercial satellite contracts. The proposed strategy is informed by research on cybersecurity threats to commercial satellites, national security concerns, current DoD policy, and previous cybersecurity frameworks. This paper aims to provide a comprehensive approach for safeguarding commercial satellites used by the DoD and ensuring the security of their supporting components. Inspired by the National Institute of Standards and Technology (NIST) 800-171 requirements and the DoD’s future Cybersecurity Maturity Model Certification (CMMC) process, the two-part framework significantly streamlines the NIST requirements to accommodate small businesses. It also extends key NIST requirements to commercial-off-the-shelf (COTS) suppliers. The CSSC Framework complements the CMMC certification process by addressing the need for cybersecurity requirements for all subcontractors supporting a commercial space asset. The framework incorporates a scoring process similar to CMMC scoring, granting points to a subcontractor for meeting the cybersecurity requirements outlined by the framework. In addition, the framework creates a space architecture overview that details the overall bid score and establishes a matrix based on individual requirements. This model and matrix allow DoD acquisition personnel to closely analyze each contract bid, comparing the subcontractor's strengths and weaknesses to other bidders. The CSSC Framework will allow the DoD to apply NIST standards to subcontractors who do not meet the requirements for CMMC certification.