Digital Test and Evaluation

The complex system engineering in the aerospace domain is undergoing a digital transformation, with Model-Based Systems Engineering (MBSE) serving as the core driving force. By establishing a unified model framework, MBSE has revolutionized traditional development paradigms and significantly enhanced multidisciplinary collaboration and full lifecycle management capabilities. However, MBSE still faces bottlenecks in model standardization, cross-domain integration, and dynamic decision-making. This paper examines MBSE model mappings across key aerospace mission phases: planning, requirement analysis, design, and on-orbit operations. It identifies core AI-empowered scenarios such as intelligent requirement capture, multidisciplinary collaborative optimization, and virtual-physical fusion verification. The study explores the synergistic application of AI and MBSE in aerospace complex system engineering, providing methodological support for the digital transformation and intelligent upgrading of China’s aerospace equipment. The complex system engineering in the aerospace domain is undergoing a digital transformation, with Model-Based Systems Engineering (MBSE) serving as the core driving force. By establishing a unified model framework, MBSE has revolutionized traditional development paradigms and significantly enhanced multidisciplinary collaboration and full lifecycle management capabilities. However, MBSE still faces bottlenecks in model standardization, cross-domain integration, and dynamic decision-making. This paper examines MBSE model mappings across key aerospace mission phases: planning, requirement analysis, design, and on-orbit operations. It identifies core AI-empowered scenarios such as intelligent requirement capture, multidisciplinary collaborative optimization, and virtual-physical fusion verification. The study explores the synergistic application of AI and MBSE in aerospace complex system engineering, providing methodological support for the digital transformation and intelligent upgrading of China’s aerospace equipment.

To keep pace with growing system complexity, Digital Engineering (DE) Model Based Systems Engineering (MBSE) capabilities must be expanded. A digital thread based approach is required to holistically evaluate measures of effectiveness/performance for missions. This paper describes the approach taken to connect requirements, SysML‐based architecture, and analytical Modeling and Simulation (M&S) yielding an interdisciplinary digital thread supporting mission analysis. The analysis was conducted in support of a multi‐domain (land, air, sea, and space) multi‐platform mission to detect, track, and engage the enemy. The approach used commercially available tools, strategically connected to establish an Authoritative Source of Truth (ASOT) for each piece of data, and gain insights about the System of Systems (SoS) beyond that which the component models could provide on their own. The results of these efforts produced a multipurpose SoS MBSE toolset enabling semi‐automated trade study analysis, while more importantly serving as a basis from which to expand systems engineering activities during subsequent phases of the systems' lifecycle. This paper concludes by identifying further opportunities for MBSE research and development to expand the developed capability.

<title>ABSTRACT</title> <p>The International Council on Systems Engineering <ext-link ext-link-type="uri" xlink:href="https://www.incose.org/">https://www.incose.org/</ext-link> is a recognized standards body defining a system engineering knowledge-base, but this knowledge falls short of fully recognizing manufacturing in the Systems Engineering (SE) framework. To be inclusive, Manufacturing needs to join in the initiative of Model Based Systems Engineering to be relevant and succeed in the digital transformation in the field of systems engineering. This paper addresses this need in manufacturing by applying Model Based System Engineering (MBSE) to the identification and management of key characteristics so that a more relevant set of Manufacturing requirements can be introduced into the MBSE construct and help realize manufacturing resilience and become a full SE partner.</p> <p><bold>Citation:</bold> “Identification and Management of Key Characteristics in Product Development Using Model Based Systems Engineering,” In <italic>Proceedings of the Ground Vehicle Systems Engineering and Technology Symposium</italic> (GVSETS), NDIA, Novi, MI, Aug. 15-17, 2023.</p>

Adopting model-based systems engineering (MBSE) tools signifies a shift in the complex nature of systems related to digital transformation. This study performs a comprehensive review and analysis to investigate organizational thinking toward training adult learners to use MBSE tools. Recognizing the wealth of experience and unique training needs characteristic of professionals, this study identifies and reflects on the prevalence of andragogy-based learning strategies to support future needs. By aligning MBSE tools with andragogy-based principles, this study seeks to contribute to the body of knowledge for both adult education and systems engineering, supporting a pathway for organizations to navigate and excel in a digitally transformed environment. This study found that organizations do not explicitly recommend a structured andragogy-based learning theory as a critical component to developing training for MBSE tools to support digital transformation. However, many of the principles ascribed to andragogy are commonly used by both organizations and researchers to promote MBSE training to adult learners. potential future development to further advance this work.

The Automatic Test System (ATS) Executive Directorate (ED) is charged with developing objectives for the Department of Defense (DoD) ATS modernization. The DoD ATS Framework Working Group (FWG) was established by the ATS ED to create a Framework addressing these objectives. The FWG consists of two sub-groups; Management, that includes members from all of the DoD Services, and Technical. The Framework is made up of key elements and associated specifications and standards that form the open system architecture approach for DoD ATSs. In addition to creating and updating key element definitions, the FWG focuses on identifying commercial ATS related specifications that might satisfy the elements in the Framework. The standards specified in the DoD ATS Framework are documented in the DoD's Information Technology Standards Repository (DISR) and thereby become a mandatory requirement for all DoD ATS acquisitions. The Framework is also being specifically called out in DoD ATS requirements documents so that its standards are incorporated in future DoD ATS acquisitions, such as the Navy's eCASS. Although the ATS Framework was originally defined more than 10 years ago, it continues to evolve as test technology evolves. A function of the FWG is to continually assess the Framework and plan the development and demonstration of specifications and standards to satisfy ATS Framework requirements. The FWG is currently participating in the standards development process via standards organizations, consortia, and working groups. The FWG is evaluating these standards in terms of applicability to the twenty plus key elements defined in the Framework. The FWG is also continually evaluating new test and information technologies to further refine and add new key elements. When necessary, the FWG provides requirements and potential solutions to standards development teams so that the resulting standard will fit within the Framework. When these standards are published and proven to meet the requirements of a particular Framework element, the FWG makes recommendations to the DoD's ATS Management Board (AMB) regarding how these standards should be represented in the DISR. Standards which are mandated will be required by the acquisition process, and vendors' products acquired by the DoD must conform to them. This paper will provide a summary of the current activities and issues impacting the ATS Framework with a discussion of the near term plans of the FWG. It will provide status on the key elements that the FWG members are currently working on. These elements include the Adapter Functional and Parametric Data (AFP), Diagnostic Data (DIAD), Diagnostic Services (DIAS), Instrument Functional and Parametric Data (IFP), Master Conformance Index (MCI), Maintenance Test Data (MTD), Resource Adapter Interface (RAI), Run Time Services (RTS), Prognostics and Health Management (PHM), and Test Station Functional and Parametric Data (TSFP). The standards that satisfy or partially satisfy these key elements, and the current status of these standards are discussed. The paper also describes the purpose of, and the FWG's role in, several industry working groups; including LAN extensions for Instrumentation (LXI), Institute of Electrical and Electronics Engineers (IEEE) Standards Coordinating Committee 20 (SCC20), Interchangeable Virtual Instruments (IVI), and Synthetic Instruments Working Group (SIWG). Finally, current issues, such as how signal modeling is to be incorporated within the Framework, the relationship to the DoD's efforts with those of the UK Ministry of Defense (MoD), the use of the DISR mandated elements in upcoming DoD ATS acquisitions, and how O-level test capabilities could be incorporated in the Framework are discussed.

This paper describes the goals, methods, approaches, and preliminary results of a research task within the Systems Engineering Research Center (SERC). Evidence across the Services and industry has affirmed digital engineering is a critical practice necessary to support acquisition in an environment of increasing global challenges, dynamic threats, rapidly evolving technologies, and increasing life expectancy of our systems currently in operation. Digital engineering updates the systems engineering practices to take full advantage of computational technology, modeling, data analytics, and data sciences. The Department of Defense's vision for digital engineering is to modernize how the Department designs, develops, delivers, operates, and sustains systems, while continuing to practice systems engineering efficiently and effectively. Digital transformation is fundamentally changing the way acquisition and engineering are performed across a wide range of government agencies, industries, and academia. As the Department of Defense transitions to digital engineering, there is a need to develop and maintain an acquisition workforce and culture that is literate in model-based engineering, competent in digital engineering models, methods, tools, and understands digital artifacts across the acquisition lifecycle.This paper outlines the development of the Digital Engineering Competency Framework by the Systems Engineering Research Center. The purpose of the Digital Engineering Competency Framework is to provide clear guidance for the Department of Defense acquisition workforce, in particular the engineering acquisition workforce, through clearly defined competencies that illuminate the knowledge, skills, abilities, and behaviors required for digital engineering professionals. The approach taken to develop the Digital Engineering Competency Framework draws from existing competency models discovered during a literature search and from the practicing Digital Engineering community. The development process for the Digital Engineering Competency Framework relied initially on literature findings to develop a draft model that was reviewed by subject matter experts through a series of workshops. No existing competency models focused on digital engineering (or model-based systems engineering or model-based engineering) were found during the literature review. However, several related competency models were identified, and these informed the initial draft model. At appropriate points, the draft model(s) were assessed through community presentations and industry review and appropriate revisions were made.The Digital Engineering Competency Framework has currently completed several iterations of subject matter expert review and revisions and is approaching a finalized general structure. The overarching structure of the DECF will consist of competency areas, proficiency levels within the competency, and constituting knowledge, skills, abilities, and behaviors. The specific knowledge, skills, abilities, and behaviors that will be associated with each competency area are the primary elements still changing in current iterations. Once the model is finalized, it will be compared with the existing Defense Acquisition University curriculum to determine what elements of such existing curriculum already support these competencies. Gaps will be noted, and training recommendations will be developed. As with the development of the model itself, this will be an iterative process. Another critical use case for the Digital Engineering Competency Framework is to enable hiring for Digital Engineering Positions, given Competency models and frameworks are used by hiring managers to ensure that individuals have the required and appropriate skillsets to adequately perform their jobs.

The digital thread represents the origination, flow, and consumption of data through machines, people, and systems across the enterprise of a manufacturer. In this paper, we explore unique manufacturing use cases and business opportunities derived from, and driven by, the digital thread and enabling technologies, with machine asset data as the cornerstone. The digital thread connects the various processes, entities, actions, and decisions across a company’s supply chain through data and data relationships, helping to ensure an organization runs efficiently and in unison. By stitching together critical product information with digital assets across the full product lifecycle, a digital thread empowers manufacturing companies to drive continuous improvement and innovative new business processes and models. The benefits of data-driven manufacturing are far too significant to ignore and will enable many to deliver competitive advantages in an ever-competitive landscape. Digital transformation is about changing business models and about companies not just taking advantage of the huge opportunities created by these latest technologies but also preparing for their constant evolution. Amid the hype surrounding Industry 4.0, IIOT, and digital transformation, the introduction of Industry 4.0 has caused a bit of a culture shock for manufacturers. To build a roadmap to digital transformation, most companies are looking into the future, attempting to visualize where they want or need to be in twenty years, and planning backwards. However, a great deal of hesitancy exists for many manufacturers to embrace the technology and modernization that solves these new challenges.

The literature argues that a real digital transformation of firms requires holistic changes of the business model. Despite knowledge about this ambitious goal, understanding of how digital business model transformation can be achieved is still very limited. In this article, we explore how firms achieve digital business model transformation. We apply a case study design to investigate how incumbents have changed their respective business model dimensions during digital transformation. Our findings center on interview data and complementary archival records from 15 cases. We present a framework for digital business model transformation along the dimensions of value proposition, value creation, and value capture. Our results emphasize the importance of a preparatory phase in which the strategic course is set. Furthermore, our findings demonstrate that transforming a company's business model is most effective when a single person, namely the Chief Digital Officer, is responsible. Our findings contribute to the business model literature, by providing a more holistic view on how business model innovation can be utilized during digital transformation.

Digital engineering (DE) is “an integrated digital approach that uses authoritative sources of systems’ data and models as a continuum across disciplines to support lifecycle activities from concept through disposal. A DE ecosystem is an interconnected infrastructure, environment, and methodology that enables the exchange of digital artifacts from an authoritative source of truth.” (DAU, 2017) Digital transformation is fundamentally changing the way acquisition and engineering are performed across a wide range of government agencies, industries, and academia and is characterized by the integration of digital technology into all areas of a business, changing fundamental operations and how results are delivered in terms of new value to customers. It includes cultural change centered on alignment across leadership, strategy, customers, operations, and workforce evolution. In 2020, the Systems Engineering Research Center (SERC) developed a Digital Engineering Competency Framework (DECF) to support the US Department of Defense (DoD). This paper provides an update to the DECF (version. 1.1) and a mapping of some existing DoD training resources against the DECF. It discusses gaps and provides a few high‐level recommendations on how to build DE competency in the DoD acquisition workforce. Though the DECF and the training included here are specific to the DoD, the competency framework is intended to be relevant to anyone working in the DE space.

Off‐the‐Shelf (OTS) strategies have become prevalent in the acquisition of naval platforms because the strategy is perceived as a means of reducing the acquisition cost and schedule risk. This paper covers a method to conduct an evaluation of shortlisted OTS naval platform design options that utilises Model‐Based Systems Engineering (MBSE). MBSE is used as a means of developing and managing the traceability of evaluation criteria, managing the evaluation itself and facilitating reuse of the design patterns present both in naval platforms and acquisition processes. The paper concludes with a description of a pilot test of the method for an Offshore Patrol Vessel, which found it to be useful to manage the evaluation criteria traceability, maintain design data and identify weak spots in OTS design options.

The success of systems engineering (SE) efforts to solve development issues benefit from the mutually advantageous relationship between academia and practice. This relationship can be threatened by lag in adoption and differences in focus between the two worlds. In an effort to help maintain alignment between SE research and practice, a 2‐day workshop was held at the University of Alabama in Huntsville on research needs in SE. The workshop was attended by individuals from academia, the Department of Defense, NASA, and the aerospace industry. This paper presents summaries of the discussions that took place, as well as possible research questions informed by the results of the workshop. The workshop consisted of an introductory roundtable and three breakouts sessions focusing on requirements, verification, and validation; configuration and data management; and systems acquisition and design. Several topics emerged as important concepts for future research, and research questions were developed relating to model‐based systems engineering (MBSE), requirements, modeling, sociotechnical issues, and education.

Focusing on businesses, digital transformation has a significant impact on various sectors. It enables innovation practices, new or improved designs and processes, and new business models, restructuring the business value chain. Digital transformation is not just about development through adoption of new technologies. Even though organisations differ regarding their digital transformation process, there is a need for digital maturity models in order to assess the status quo of their digital transformation. Therefore, there is a lack of understanding on which are the critical elements that organisations need to take into consideration for their digital transformation journey. The purpose of this paper is to propose both a scientific and practical framework for designing, developing, and implementing a digital transformation journey. Many different digital maturity models have been examined and analysed implementing a three-phase literature review methodology. Based on the prior literature of digital transformation maturity models, a conceptual framework is proposed and tested with a case study. As a result, the synthesized framework could serve as a blueprint of strategic decision-making in digital transformation. This research spreads light on the field of digital transformation by expanding the role of digital maturity models. It is stated that more research needs to be done aiming to gain a deeper understanding of this topic.

Purpose—While digital transformation has been widely implemented in large corporations, there is a big potential for startups and SMEs that are lagging behind in utilizing this opportunity. This chapter aims to raise awareness among startups and SMEs about digital transformation by developing a digital maturity model adopted, especially for international distributed work. The model serves as a framework for startups and SMEs to understand which dimensions influence and are important for digital transformation, and what specific questions are relevant for each of the dimensions.Design/methodology/approach—The research followed the Design Science Research with explorative interviews of two CEOs and development of case studies of companies currently undergoing the digital transformation process to get a deeper understanding of the different dimensions and which dimensions really matter. The artifacts are the dimensions that have a direct impact on the successful implementation of digital transformation, including a sample report of the digital maturity to position a company’s digital transformation against a baseline. Those dimensions were identified and chosen based on the review of 27 existing digital maturity models.Findings—The perception, that digital transformation is all about technology, has been disproved. Seven dimensions identified were (a) Culture and Leadership, (b) Digital Technologies, (c) Organization and Processes, (d) Employees and Digital Skills, (e) Customers, (f) Products and Innovation, and (g) Digital Strategy. Once a digital strategy is defined, a company can then focus on either internal or external environment. Culture and leadership need to be defined top-down as a base for driving change. Digital technologies are enablers, not drivers.Business implications—The use of a digital maturity model raises companies’ awareness about the current digital transformation phase and helps to understand, kick-off and accelerate their digital transformation to realize the aspired situation. While SMEs need to carefully plan and transform all aspects of a running company to benefit from digital opportunities, startups can launch digitally right from the start.Originality/value—Startups and internationally active SMEs can use the maturity model for a self-assessment as a first step to understand key dimensions to build a digital strategy and continuously work on improving maturity level as well as sustainability. Researchers in other countries can use the maturity model for classification and conduct a survey for comparative analysis among countries and different industries in different countries.

There are many limitations depend on traditional physical product testing methods. Firstly, the cost is high and adaptability is poor. Secondly, the test is lag and it can’t follow the new product design’s beat. Thirdly, it can’t converge with digital design method, and it difficult to meet with the requirement of industry information. Study a product’s digital test method which based on modern design technical, and discuss simulation testing technical come true which face to virtual prototyping of bag filter, and aiming at digital testing result of bag filter main working capability prove feasibility and effectiveness of this method. Firstly, analyses components of digital virtual testing system and theory of working, and provide design model of virtual prototyping testing and testing progress how to come true, and by changing the condition of working simulation, and sampling, and data processing, acquire the result of dust gas uniformity which is design prototyping of bag filter. Application prove that digital process, face to digital testing of product virtual prototyping, by testing object and testing instrument and testing progress, and so on, adopting model simulation, digital simulation, calculate experiment, simulate testing technical, complete prototyping testing working under fully virtual state in computer. Let prototyping design quality controlling which is based on quantizing move forward. Achieve parallel designing and testing, which is full with the zero-cost, zero-lag, zero error of large and complex mechanical equipment design testing.

Digital business model transformation (DBMT) necessitates new managerial capabilities, yet the existing literature lacks an empirical understanding of managerial capabilities as antecedents to strategic change and firm performance. This paper builds on dynamic managerial capabilities theory to argue that managerial human capital—composed of leadership and entrepreneurial skills—is a critical facilitator of DBMT and resultant firm performance. Further, the research model proposes that managers' social capital and cognition positively moderate the relationship between human capital and DBMT. The study's findings from a sample of German Industry 4.0 firms provide new insights into the significance of managerial capabilities in a digital economy. This study advances management literature by demonstrating that the benefits of managers' human capital for DBMT are contingent on its form: entrepreneurial skills facilitate digital transformation integral for firm performance, while leadership skills have no impact on firm performance—neither directly nor indirectly through DBMT. Thus, this study provides strong evidence of the importance of entrepreneurial skills in driving DBMT to increase firm performance. Further, the findings offer a nuanced account of the interrelationships between dynamic managerial capabilities, revealing that higher levels of social capital and lower levels of cognition increase the positive effect of entrepreneurial skills on DBMT. This study altogether reaffirms the significance of managers' dynamic capabilities for strategic change enabled by DBMT and their performance benefits, yet it reveals that the effect mechanisms differ from those found in nondigital research settings.