Automated test systems test program software — making the invisible visible

The Versatile Automated Depot Test Station (VDATS) is a major component in solving the U.S. Air Forcepsilas Automatic Test System (ATS) proliferation concerns. VDATS addresses the proliferation issue by providing the capability to replace an estimated 260 depot-level test systems that assume 150 different configurations, as well as a large number of intermediate-level test systems.The Department of Defense (DoD) has invested billions of dollars in aging and increasingly obsolete ATS used to troubleshoot and diagnose components of military aircraft and weapon systems. The issues of increased downtimes, test system obsolescence, and nonportable Test Program Sets (TPS) further contributes to the DoD's continued reporting of spare parts shortages and the increasing impacts that ATS obsolescence has on the readiness of military aircraft and weapon systems. The VDATS provides a viable approach to resolving these issues. VDATS has an extensible designed-in expansion capability intended to support present and future Air Force workloads by providing an organically-developed ATS platform with a modular, open-systems architecture that is Flexible, Capable, Tailorable, and Sustainable. VDATS makes it technically feasible to now realize the laudable 1994 DoD policy goal stated in: Military Readiness: DoD Needs to Better Manage Automatic Test Equipment Modernization," GAO-03-451, March 2003.[1] VDATS meets this stated DoD policy goal of producing an ATS capable of supporting multiple weapon systems, aircraft, and eventually being interoperable between DoD services. In 2007 the Air Force's VDATS was officially designated as a DoD Standard Family of Testers, supporting the 2005 ATS Master Plan requirement meant to reduce ATS proliferation and life-cycle operations and support costs. This paper briefly examines the present-day issues and how the U.S. Air Force is resolving these with the VDATS program. It also explains the strategy to preserve present-day and near-future TPS re- host investments, while enhancing system capabilities to meet the Air Force's depot and intermediate-level ATS requirements for the foreseeable future.

Managing Automatic Test Systems in the 90's is more than writing BASIC test programs to control IEEE 488 instruments. Managing your Automatic Test System involves Test Program Set (TPS) development, management of re-usable test objects, data management, tester resource management and user interface management. In addition, the Automatic Test System must provide a Test Executive that controls the overall test development and execution environments. The benefits of this type of system are, lower TPS development cost, a common user interface and lower TPS development cost, a common user interface and lower operator training cost. This paper focuses on these aspects of Automatic Test System Management. >

This paper outlines an approach to developing an emulation environment embedded in an automated test system (ATS) controller's operating system kernel. By taking this approach, it is possible to design an emulator that is built for multiple ATS environments, allowing it to be seamlessly ported from ATS to ATS for use with many different test program sets (TPS). This paper then explores possible uses for such an emulator for the purpose of reducing the life-cycle costs associated with ATS and TPS development and maintenance.

This paper discusses a practical approach to the use of Commercial-Off-the-Shelf (COTS) tools to implement Automatic Test Equipment (ATE) and Unit Under Test (UUT) signal-based models to support both new Test Program Set (TPS) development and TPS rehost activities. This modeling technique also provides a standard to which existing and future ATEs and UUTs can be modeled and used for analysis and comparisons. The signal-based model is implemented via a Structured Query Language (SQL) Relational Data Base Management System (RDBMS). COTS Rapid Application Development (RAD) tools, such as Microsoft/sup (R)/ Visual Basic/sup (R)/ Microsoft Access/sup (R)/ and PowerBuilder/sup (R)/, are employed to implement interfaces to the SQL database model and to provide the necessary algorithms for data comparison, model reuse and other manipulations of the model data. After models are established for both the ATE and UUTs, ATU/UUT compatibility analysis can be performed to identify any potential incompatibilities. This information can be used to optimize the selection of a target ATE for a selected group of UUTs. It can also be used to identify any signal deficiencies in the target ATE that require ATE enhancements or Interface Test Adapter (ITA) signal conditioning, to ensure UUT compatibility.

For decades, the automatic test community has worked to develop hardware standardization and interface protocols that aid in mitigating the impact of equipment obsolescence on the repair and maintenance mission. The traditional view of Test Program Set (TPS) transportability in this environment has always been one of migrating a legacy TPS to a new hardware architecture design born out of a need to replace aging technology or provide new test capability. The Department of Defense (DoD) has recognized the benefit of standardizing automatic test systems (ATS) through policy guidance whose intent is to drive the services to a common configuration test environment, thereby reducing life cycle sustainment costs for all DoD systems. Today, hardware standardization has advanced to the point that many ATS look and function very similar to one another. However, there are two major hurdles that continuously thwart our efforts to develop a standard core test system, the lack of a standard UUT interface and the lack of an open architecture software design that facilitates operations in any environment and on any test system architecture. Automatic Test Markup Language (ATML) provides promise with regards to the development of standardized test software interfaces. Developing a standard hardware configuration for all test requirements is ultimately achievable with the technology currently available today except for one simple but hugely significant hurdle, the cost involved in re-hosting the myriad of test program sets currently in existence. Therefore, any significant advance in common interface design has to take into account the need to minimize the impact to existing test program sets. This paper will seek to explore the various technological possibilities for overcoming the barriers to true TPS transportability that exist in today's automatic test community.

In modern systems there is a desire to create Test Program Sets (TPSs) in multiple languages. However, most Automatic Test Equipment (ATE) is designed with only one test executive in mind and the exclusion of all others. This paper discusses a proven design to allow for Multiple Test Executives (MTE) on a single system. In addition, the test executives are divorced from the operator or maintainer so that they are not directly aware of the underlying test exec. This allows TPS programmers to work in a language of their choice and expertise in order to generate TPSs in the most efficient manner. This also allows for legacy TPSs to be more easily moved to the new system as it is more likely that there is a good fit between one of the legacy test programming languages and one of the many supported by the new system supporting MTE. By allowing MTE under one common operator interface, a consistent look and feel can be enforced while still allowing for MTE. This abstraction of operator interface from the actual test executive is done by taking advantage of several aspects of modern test executives and user interfaces. Most modern test executives actually live as an engine which can be controlled via software. While they may come with a GUI out of the box (Test Stand, Test Studio, ATLAS, etc), most modern test executives are run by an underlying engine which is controllable through similar interfaces. Point in fact, Test Stand, Test Studio, and PAWS ATLAS can (and are) controlled in this manner. By allowing for software control of the test executive, a common GUI can control another common run time engine which communicates with all the test executives and runs TPSs on the corresponding test executive. By also creating a common interface to the GUI and instruments, all of the test executives can communicate through the common software and be controlled by and control a common set of user interfaces and instruments.

The Automatic Test System (ATS) Executive Directorate (ED) is charged with developing objectives for the Department of Defense (DoD) ATS modernization. The DoD ATS Framework Working Group (FWG) was established by the ATS ED to create a Framework addressing these objectives. The FWG consists of two sub-groups; Management, that includes members from all of the DoD Services, and Technical. The Framework is made up of key elements and associated specifications and standards that form the open system architecture approach for DoD ATSs. In addition to creating and updating key element definitions, the FWG focuses on identifying commercial ATS related specifications that might satisfy the elements in the Framework. The standards specified in the DoD ATS Framework are documented in the DoD's Information Technology Standards Repository (DISR) and thereby become a mandatory requirement for all DoD ATS acquisitions. The Framework is also being specifically called out in DoD ATS requirements documents so that its standards are incorporated in future DoD ATS acquisitions, such as the Navy's eCASS. Although the ATS Framework was originally defined more than 10 years ago, it continues to evolve as test technology evolves. A function of the FWG is to continually assess the Framework and plan the development and demonstration of specifications and standards to satisfy ATS Framework requirements. The FWG is currently participating in the standards development process via standards organizations, consortia, and working groups. The FWG is evaluating these standards in terms of applicability to the twenty plus key elements defined in the Framework. The FWG is also continually evaluating new test and information technologies to further refine and add new key elements. When necessary, the FWG provides requirements and potential solutions to standards development teams so that the resulting standard will fit within the Framework. When these standards are published and proven to meet the requirements of a particular Framework element, the FWG makes recommendations to the DoD's ATS Management Board (AMB) regarding how these standards should be represented in the DISR. Standards which are mandated will be required by the acquisition process, and vendors' products acquired by the DoD must conform to them. This paper will provide a summary of the current activities and issues impacting the ATS Framework with a discussion of the near term plans of the FWG. It will provide status on the key elements that the FWG members are currently working on. These elements include the Adapter Functional and Parametric Data (AFP), Diagnostic Data (DIAD), Diagnostic Services (DIAS), Instrument Functional and Parametric Data (IFP), Master Conformance Index (MCI), Maintenance Test Data (MTD), Resource Adapter Interface (RAI), Run Time Services (RTS), Prognostics and Health Management (PHM), and Test Station Functional and Parametric Data (TSFP). The standards that satisfy or partially satisfy these key elements, and the current status of these standards are discussed. The paper also describes the purpose of, and the FWG's role in, several industry working groups; including LAN extensions for Instrumentation (LXI), Institute of Electrical and Electronics Engineers (IEEE) Standards Coordinating Committee 20 (SCC20), Interchangeable Virtual Instruments (IVI), and Synthetic Instruments Working Group (SIWG). Finally, current issues, such as how signal modeling is to be incorporated within the Framework, the relationship to the DoD's efforts with those of the UK Ministry of Defense (MoD), the use of the DISR mandated elements in upcoming DoD ATS acquisitions, and how O-level test capabilities could be incorporated in the Framework are discussed.

Migration, maintenance and regression of more than three hundred (300) Test Program Set (TPS) to support all existing (i.e., Consolidated Automated Support System (CASS) and Reconfigurable Transportable Consolidated Automated Support System (RTCASS)) and future (e.g., electronic Consolidated Automated Support System (eCASS)) CASS Family Automated Test Equipment (ATE) members can be an overwhelming task that requires compilation and build of source, translation, data transfer and data validation. CASS family automation tools can help TPS developers create and execute from Personal Computer (PC) environment CASS Virtual Address extension (VAX) Digital Command Line (DCL) scripting files without physically being on an ATE station; make Magneto-Optical (MO) disks (without user intervention) from VAX backup save set files; make automated Version Description Document (VDD) reports at Operational Test Program Set (OTPS) level; identify and fix corrupted Master Test Program Set Index (MTPSI) files; identify corrupted CASS Abbreviated Test Language for All Systems (ATLAS) files; identify missing CASS files (ATLAS source ATLAS lookup, graphics, backup save set, Digital Test Language, Digital Automated Test Program Generation (DATPG), shutdown, data, subassembly and Functional Extension Program); validate and create Automated Test System (ATS) Source Data Repository (ASDR) directory structure; create file list inventories for Configuration Management (CM) and Quality Assurance (QA); have a visual representation of the MTPSI card; and provide support for the Next Generation Operations Management Software (NxOMS) communication. This paper will demonstrate improvements from our previous suite of automation tools, before and after figures showing tools benefits, and faced hardware challenges.

The Automatic Test System (ATS) Executive Directorate (ED) is charged with developing objectives for the Department of Defense (DoD) ATS modernization. The DoD ATS Framework Working Group (FWG) was established by the ATS ED to create a Framework addressing these objectives. The FWG consists of two sub-groups: Management, that includes members from all of the DoD Services, and Technical. The Framework is made up of key elements and associated specifications and standards that form the open system architecture approach for DoD ATSs. In addition to creating and updating key element definitions, the FWG focuses on identifying commercial ATS related specifications that might satisfy the elements in the Framework. The standards specified in the DoD ATS Framework are documented in the DoD's Information Technology Standards Repository (DISR) and thereby become a mandatory requirement for all DoD ATS acquisitions. The Framework is also being specifically called out in DoD ATS requirements documents so that its standards are incorporated in future DoD ATS acquisitions, such as the Navy's eCASS.

Santa Barbara Infrared, Inc. has deployed IRWindows as an Electro-Optical test development and execution environment for military Test Program Sets (TPS). Advantages of TPS development for EO systems in the IRWindows environment are seen compared to the TPS development in ATLAS. The advantages of the IRWindows environment are: 1. Faster learning curve (graphical user interface is easier than test line interface) 2. Faster TPS development time (real time changes and asset control interface allows for faster development) 3. Asset control panel allows user to control assets real time and monitor all asset functions during development 4. Unit Under Test (UUT) image viewer allows user to set test parameters like Region of Interest more easily and more precisely 5. Continuous mode tests (like MTF allows user to real time adjustments) 6. Open architecture for test modifications This paper will outline the details of how these advantages are utilized and how not only development time is decreased but also how test execution time can be minimized making traditionally long TPS run times on EO systems more efficient.

This paper describes several capabilities that greatly improve the test program set (TPS) debug and integration efforts. As TPS debug and integration activities are significant cost factors in TPS development, improvements in this phase offer potentially high returns. Important considerations include hardware and software architecture decisions, instrument interface visibility, interactive control, degree of simulation, and various modes of controlling and monitoring execution. Specific operation of various capabilities is also discussed as are the benefits to TPS debug and integration.

The purpose of this paper is to present a conceptual approach and to make practical recommendations on how to improve the current Automatic Test Equipment (ATE) testing and maintenance capabilities by utilizing the existing cloud computing model to build a globally linked ATE maintenance system. The basic tenet of the ATE community is to support a multi-tiered maintenance concept which, in general, is a three tiered system that is composed of organizational maintenance (O-level), intermediate maintenance (I-level), and depot maintenance (D-level) organizations. The goal of the ATE is to (1) quickly and accurately detect and isolate each fault, (2) provide software tools for analyzing historical data, and (3) gather, manage, and distribute accurate and reliable maintenance information for the failed Unit Under Test (UUT). The ATE system should provide services that will (1) maintain a repository of information that will improve fault detection and isolation, allow for off-platform assessments, document failures, and help quantify corrective actions, (2) reduce false UUT pulls, and (3) reduce repair time by prompting repair procedures. Furthermore, the ATE system should provide additional services that will help optimize the time to diagnose problems by using collected failure information and by recommending entry points into the Test Program Set (TPS) software. It should also present information to the ATE maintainer to aid in informed repair decisions which could be in the form of pilot debrief results, platform Built In Test (BIT) results, O-level test outcomes and corrective actions, and maintenance and usage history of the platform and UUT. So, based on this definition of ATE maintenance the use of cloud computing can be used to provide services to improve the overall ATE testing throughput which will result in bottom line improvements to ATE life cycle costs. By using cloud computing, which is defined to be “a model for enabling ubiquitous, convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, servers, storage, applications, and services) that can be rapidly provisioned and released with minimal management effort or service provider interaction”, users can develop cloud computing models that will provide access to application software and databases that can be used to build a globally linked ATE maintenance system. This paper will discuss the essential characteristics of the cloud computing models and define the various flavors of cloud offerings available to designers today. This paper will also analyze the cloud computing model to arrive at a conceptual approach that can be used to enhance the current ATE Testing and Maintenance capabilities. Practical recommendations will be discussed on how to transform the current ATE Testing and Maintenance capabilities into the specific cloud computing model offerings in order to help configure a globally linked ATE maintenance system.

In military and aerospace applications test systems are expected to have a service life of more than 20 years. Over the life of the test system individual test instruments may become obsolete or may require upgrading due to changes in TPS requirements. Any changes to these legacy test instruments may cause problems with the deployed Test Program Set (TPS). Most test systems are designed with an instrument wrapper layer to isolate the TPS developer from the instrument driver Application Programming Interface (API). This wrapper layer will minimize the changes to the TPS due to changes in individual test instruments. This paper discusses some of the issues that may be encountered when either replacing obsolete instrumentation or upgrading instrumentation in a test system. Some of the issues are easy to discover and understand such as minor differences in the instrument specifications between the legacy test instrument and the replacement instrument. Other issues may not become apparent until attempting to use the replacement instrument within a test program or when upgrading system checkers. For example a test program may fail with a replacement instrument due to signal settling time. This problem can occur when the replacement instrument performs measurements faster than the legacy instrument. The problem can be fixed by either adding a wait in the test program or by adding a trigger delay in the test program or wrapper. The main focus of this paper will be on the problems related to the replacement of analog instruments. This paper will not discuss issues related to differences in the instrument driver API since this can usually be handled in the instrument wrapper layer. The problem discussions will look at classes of problems and site some specific examples in each class. Guidelines will be presented for solving or avoiding the problem. Some of this information may also be useful in the design of future test systems and TPSs to avoid similar issues.

New software technologies, including the World Wide Web, may seem far removed from the tasks facing test program set (TPS) developers, but they promise to revolutionize the way TPS data is organized, presented, and used. This paper will describe how an integrated TPS development and execution environment can capitalize on these new technologies to improve test programming efficiency.

New software technologies, including the World Wide Web, may seem far removed from the tasks facing test program set (TPS) developers, but they promise to revolutionize the way TPS data is organized, presented, and used. This paper will describe how an integrated TPS development and execution environment can capitalize on these new technologies to improve test programming efficiency.