Intermittently failing tests in the embedded systems domain

The code of JUnit test cases is commonly used to characterize software testing effort. Different metrics have been proposed in literature to measure various perspectives of the size of JUnit test cases. Unfortunately, there is little understanding of the empirical application of these metrics, particularly which metrics are more useful in terms of provided information. This paper aims at proposing a unified metrics suite that can be used to quantify the unit testing effort. We addressed the unit testing effort from the perspective of unit test case construction, and particularly the effort involved in writing the code of JUnit test cases. We used in our study five unit test case metrics, two of which were introduced in a previous work. We conducted an empirical study in three main stages. We collected data from six open source Java software systems, of different sizes and from different domains, for which JUnit test cases exist. We performed in a first stage a Principal Component Analysis to find whether the analyzed unit test case metrics are independent or are measuring similar structural aspects of the code of JUnit test cases. We used in a second stage clustering techniques to determine the unit test case metrics that are the less volatile, i.e. the least affected by the style adopted by developers while writing the code of test cases. We used in a third stage correlation and linear regression analysis to evaluate the relationships between the internal software class attributes and the test case metrics. The main goal of this study was to identify a subset of unit test case metrics: (1) providing useful information on the effort involved to write the code of JUnit test cases, (2) that are independent from each other, and (3) that are the less volatile. Results confirm the conclusions of our previous work and show, in addition, that: (1) the set of analyzed unit test case metrics could be reduced to a subset of two independent metrics maximizing the whole set of provided information, (2) these metrics are the less volatile, and (3) are also the most correlated to the internal software class attributes.

Component-based software systems (CBSSs) enable modular design and reuse, but their integration poses substantial testing challenges. Integration faults often stem from interactions between components and are difficult to detect with isolated unit testing. Existing integration testing practices require significant manual effort, particularly in configuring test environments and maintaining test suites as systems evolve.This dissertation introduces the Interaction-based Integration (IBI) Testing Approach, a novel approach to automate integration testing by reusing unit test cases. The IBI testing approach comprises the IBI Metamodel, which captures structural, behavioral, and interaction expectation aspects of CBSSs, and the IBI Testing Process, which operationalizes the IBI Metamodel. The approach leverages unit test cases to derive interaction expectations, and generate so called interaction test cases, by reusing the test outputs of one unit test case as inputs for another. It thereby simulates system behavior, enabling integration testing without the need for dedicated integration test cases or environments. In contrast to existing approaches, which require dedicated integration environments and handcrafted test cases, this work enables integration validation through existing unit test artifacts.The IBI testing approach is formalized using property graphs and regular path queries. It is implemented in the InterACt tool, evaluated through a demonstration case study on a microservice system and an exploratory case study in collaboration with an industry partner. The results confirm the feasibility of the IBI Testing Approach while identifying practical limitations that guide future improvements.This work contributes a reusable integration testing model, a formalized testing process, and a tool-supported implementation that provide a novel approach to integration testing of CBSSs. The findings highlight the potential of reusing unit test cases for integration testing, while also addressing the challenges and limitations of this approach in practice.

In the software industry, software testing becomes more important in the entire software development life cycle. Software testing is one of the fundamental components of software quality assurances. Software Testing Life Cycle (STLC)is a process involved in testing the complete software, which includes Regression Testing, Unit Testing, Smoke Testing, Integration Testing, Interface Testing, System Testing & etc. In the STLC of Regression testing, test case selection is one of the most important concerns for effective testing as well as cost of the testing process. During the Regression testing, executing all the test cases from existing test suite is not possible because that takes more time to test the modified software. This paper proposes new Hybrid approach that consists of modified Greedy approach for handling the test case selection and Genetic Algorithm uses effective parameter like Initial Population, Fitness Value, Test Case Combination, Test Case Crossover and Test Case Mutation for optimizing the tied test suite. By doing this, effective test cases are selected and minimized the tied test suite to reduce the cost of the testing process. Finally the result of proposed approach compared with conventional greedy approach and proved that our approach is more effective than other existing approach.

Software testing seems to be a huge struggle for most developers. This is presumably because of the amount of manual work involved in software testing. In this article we propose a testing approach that makes the testing process less manual. The approach combines three well-known concepts Design by Contract, Unit Testing and Equivalence Partitioning into a new approach named Testing by Contract. The first step is to define a new programming language that supports an extended form of Design by Contract. The idea is to enhance the contracts with knowledge of Equivalence Partitions. The second step is to design a tool that exploits the enhanced contracts to build quality test cases automatically. The test cases are built as JUnit test cases, which allow them to be combined with manually written test cases. A prototype Java-like language with support for this extended form of Design by Contract has been developed. A compiler to regular Java-code as well as a tool for automatic generation of test cases is currently being built. This prototype language/tool will be used to further examine and evaluate the ideas described above.

Software quality assurance and testing is a vital phase in the software development lifecycle, oftentimes requiring a significant portion of the overall development budget. Providing authentic, real-world software quality assurance and testing experiences within the context of a Computer Science or Software Engineering curriculum is challenging. Based on prior experience leveraging Free and Open Source Software (FOSS) as an instructional medium for teaching Software Engineering concepts, this paper describes five increasingly involved learning interventions that utilize FOSS in a software testing course to provide students real-world hands-on software testing knowledge and experience. The proposed approach will be piloted and assessed in software testing courses at the undergraduate and graduate level.

Adapting automated test generation to GUI testing of industry applications

Development of a novel methodology for root cause analysis and selection of maintenance strategy for a thermal power plant: A data exploration approach

Fault in a software program can be detected by mutation testing. However, mutation testing is an expensive process in a software testing domain. In this paper, we have introduced a method based on genetic algorithm and mutation analysis for unit testing process. Software industry produces high quality software in which software testing has an important role. First, we make a program/software and intent some mutant in this program/software, find most critical path and optimise test cases using genetic algorithm for the unit testing. Initially generated test cases are refined using genetic algorithm. We use a mutant function for measuring the adequacy of the test case set. The given mutant function is used to calculate a mutant score. We have achieved 100% path coverage and boundary coverage using mutation testing. The objective is to produce a set of good test cases for killing one or more undesired mutants and produces different mutant from original software/program. Unlike simple algorithms, genetic algorithms provide suitability for reducing the data generation at a comparable cost. An optimised test case has been generated by proposed approach for cost reduction and revealing or killing undesired test cases.

Fault in a software program can be detected by mutation testing. However, mutation testing is an expensive process in a software testing domain. In this paper, we have introduced a method based on genetic algorithm and mutation analysis for unit testing process. Software industry produces high quality software in which software testing has an important role. First, we make a program/software and intent some mutant in this program/software, find most critical path and optimise test cases using genetic algorithm for the unit testing. Initially generated test cases are refined using genetic algorithm. We use a mutant function for measuring the adequacy of the test case set. The given mutant function is used to calculate a mutant score. We have achieved 100% path coverage and boundary coverage using mutation testing. The objective is to produce a set of good test cases for killing one or more undesired mutants and produces different mutant from original software/program. Unlike simple algorithms, genetic algorithms provide suitability for reducing the data generation at a comparable cost. An optimised test case has been generated by proposed approach for cost reduction and revealing or killing undesired test cases.

Unit testing is an established practice in professional software development. However, in high-performance computing (HPC) with its scientific applications, it is not widely applied. Besides general problems regarding testing of scientific software, for many HPC applications the effort of creating small test cases with a consistent set of test data is high.We have created a tool called FortranTestGenerator, that significantly reduces the effort of creating unit tests for subroutines of an existing Fortran application. It is based on Capture & Replay (C&R), that is, it extracts data while running the original application and uses the extracted data as test input data. The tool automatically generates code for capturing the input data and a basic test driver which can be extended by the developer to an appropriate unit test. A static source code analysis is conducted, to reduce the number of captured variables. Code is generated based on flexibly customizable templates. Thus, both the capturing process and the unit tests can easily be integrated into an existing software ecosystem.Since most HPC applications use message passing for parallel processing, we also present an approach to extend our C&R model to MPI communication. This allows extraction of unit tests from massively parallel applications that can be run with a single process.

Background & Problem Statement - Software testing is a critical phase in the software development lifecycle (SDLC), ensuring that applications function correctly, meet user requirements, and maintain high- quality standards. Traditional software testing approaches, including manual testing and rule-based automation, often face challenges in scalability, efficiency, and adaptability to dynamic software environments. Traditional testing methods are overwhelmed by complex software systems which slows down defect detection and extends both testing costs and release schedules. Machine Learning (ML) has emerged as a transformative solution, introducing predictive and adaptive capabilities that optimize test case selection, automate defect detection, and enhance overall software quality assurance (QA). This study explores the integration of ML in software testing, addressing the challenges of traditional QA methodologies and demonstrating how AI-driven frameworks improve testing efficiency. Methodology - To investigate the impact of ML in software testing, this research adopts a systematic approach by analyzing ML-driven test automation techniques, including predictive testing, adaptive test execution, and automated test case generation. Research reviews how Google Microsoft Facebook IBM and Deep Code put ML-based quality assurance frameworks into operation. The study leverages supervised learning, reinforcement learning, deep learning, and NLP-based techniques to demonstrate how ML models predict software defects, dynamically adapt test cases, and optimize testing resources. The research tests how ML-based testing models operate within CI/CD pipelines to improve ongoing testing and deployment flow. Analysis & Results - The analysis of ML-driven software testing reveals that predictive analytics improves early defect detection rates. It helps developers spend 37% less time debugging their work. Adaptive testing models, including self-healing test scripts, minimize maintenance costs by 50% and enhance test reliability in agile environments. The integration of NLP-based test case generation increases test coverage. NLP technology enables automatic connection between requirements and test cases at 89% success rate. Additionally, reinforcement learning techniques improve test case selection, reducing redundant test executions by 43%. Our research shows different ML methods work well to lessen incorrect error alerts. ML integration for QA surely increasing defect prediction accuracy and optimizing test execution time. Findings & Contributions - This research contributes to the field of AI-driven software testing by providing a comprehensive framework for ML-based QA methodologies. Our study shows that machine learning helps find more software problems better adapts test cases and lowers testing expenses to solve present software development needs. The study also identifies critical challenges, including data availability, model interpretability, and computational overhead, suggesting future research directions in Explainable AI (XAI), hybrid AI-ML testing models, and AI-driven security testing. As the industry moves toward AI-first software testing, this research paves the way for fully autonomous QA frameworks, enabling intelligent, scalable, and cost- effective software validation techniques. Keywords - Machine Learning, Software Testing, Quality Assurance, Predictive Testing, Adaptive Testing, Test Automation, Defect Prediction, Self- Healing Test Scripts, AI-Driven QA, Reinforcement

Software testing is a process where you have the stated requirements, pre-requisite software that is to be installed and the configuration of the machine that need to be done as per the test data for the test case and for a specific platform which when verified works perfectly. This paper describes about Manual test case process overview in the testing life cycle for a large software development project and how you track the effort for the same. A Tester need to know about the fundamentals of testing concepts. Documentation of the testing concepts and the process overview of the testing life cycle for a software product need to be done initially so that a new tester when recruited can understand the basic flow as to how it is carried out in real time in an organization practically. Initially a tester need to execute the manual test case for few platforms and some tougher manual test cases need to be learnt from the tester who is having some experience. The tougher manual test cases need to be identified initially from the experienced tester who will be giving a session for the execution of such test cases. So when the foundation for a new tester is powerful the plan assigned for the execution of test cases is met. The effort tracking for the testing can be captured for a tester and should be matched with planned effort for the execution of test cases. Measurement of the testing effort plays a very important role as to how comfortable a new tester is and further root cause analysis is done if required when the new tester does not meet the planned effort.

The steady march of Moore's Law in semiconductors has enabled the creation of ever more complex systems with electronics playing a central role. As a result, thorough testing of individual components is no longer adequate to ensure overall system performance, quality, and reliability. The rising importance of system-level test (SLT) to supplement traditional component structural test has gained wide recognition recently. To understand this trend, we examine where traditional testing falls short and where SLT fills the gap. In many cases, system failures are the result of complex component interactions leading to abnormal scenarios not attributable to simple and single root causes. Rather than being confined by traditional gate-level fault models, it might be more appropriate to develop a system-level fault model derived as an emergent property of complex systems. New definitions of fault coverage and how to automate high-coverage test generation are obvious challenges in the system domain. Recent developments in design verification which faces similar challenges in dealing with system complexity may offer possible solutions for SLT to draw on. The aim is to stimulate new ideas and directions for future research in system-based testing as upcoming 5G/IoT/AI-based applications penetrate deeper and pervasively into our daily lives.

Unit testing has a significant role in software development and its impacts depend on the quality of test cases and test data used. To reduce time and effort, unit test generator systems can help automatically generate test cases and test data. However, there is currently no unit test generator for Kotlin programming language even though this language is popularly used for android application developments. In this study, we propose and develop a test generator system that utilizes genetic algorithm (GA) and ANTLR4 parser. GA is used to obtain the most optimal test cases and data for a given Kotlin code. ANTLR4 parser is used to optimize the mutation process in GA so that the mutation process is not totally random. Our model results showed that the average value of code coverage in generated unit tests against instruction coverage is 95.64%, with branch coverage of 76.19% and line coverage of 96.87%. In addition, only two out of eight generated classes produced duplicate test cases with a maximum of one duplication in each class. Therefore, it can be concluded that our optimization with GA on the unit test generator is able to produce unit tests with high code coverage and low duplication.

Software testing is a crucial yet time-consuming aspect of software development. Writing comprehensive unit tests that accurately verify whether a function or an entire program behaves as intended requires considerable effort from developers, particularly when handling numerous edge cases. This study explores how Large Language Models (LLMs) can streamline this process by automatically generating effective unit tests. We evaluate various LLMs on their capability to interpret problem specifications, analyze source code across multiple programming languages, and generate suitable test cases. The effectiveness of these test cases is assessed using the Pass@1 and line coverage metrics. Our findings reveal that LLMs perform significantly better when provided with both the problem description and the corresponding solution code, particularly in the C programming language. Additionally, we observe substantial performance improvements when example test cases are included in the prompt, leading to higher Pass@1 scores and enhanced code coverage, particularly with more advanced LLMs.