Software Error Detection through Testing and Analysis

Putting Requirements and Quality at the Core of Global Service Delivery: Current Efforts and Future Plans at Pace University

Software quality assurance and testing is a vital phase in the software development lifecycle, oftentimes requiring a significant portion of the overall development budget. Providing authentic, real-world software quality assurance and testing experiences within the context of a Computer Science or Software Engineering curriculum is challenging. Based on prior experience leveraging Free and Open Source Software (FOSS) as an instructional medium for teaching Software Engineering concepts, this paper describes five increasingly involved learning interventions that utilize FOSS in a software testing course to provide students real-world hands-on software testing knowledge and experience. The proposed approach will be piloted and assessed in software testing courses at the undergraduate and graduate level.

Background & Problem Statement - Software testing is a critical phase in the software development lifecycle (SDLC), ensuring that applications function correctly, meet user requirements, and maintain high- quality standards. Traditional software testing approaches, including manual testing and rule-based automation, often face challenges in scalability, efficiency, and adaptability to dynamic software environments. Traditional testing methods are overwhelmed by complex software systems which slows down defect detection and extends both testing costs and release schedules. Machine Learning (ML) has emerged as a transformative solution, introducing predictive and adaptive capabilities that optimize test case selection, automate defect detection, and enhance overall software quality assurance (QA). This study explores the integration of ML in software testing, addressing the challenges of traditional QA methodologies and demonstrating how AI-driven frameworks improve testing efficiency. Methodology - To investigate the impact of ML in software testing, this research adopts a systematic approach by analyzing ML-driven test automation techniques, including predictive testing, adaptive test execution, and automated test case generation. Research reviews how Google Microsoft Facebook IBM and Deep Code put ML-based quality assurance frameworks into operation. The study leverages supervised learning, reinforcement learning, deep learning, and NLP-based techniques to demonstrate how ML models predict software defects, dynamically adapt test cases, and optimize testing resources. The research tests how ML-based testing models operate within CI/CD pipelines to improve ongoing testing and deployment flow. Analysis & Results - The analysis of ML-driven software testing reveals that predictive analytics improves early defect detection rates. It helps developers spend 37% less time debugging their work. Adaptive testing models, including self-healing test scripts, minimize maintenance costs by 50% and enhance test reliability in agile environments. The integration of NLP-based test case generation increases test coverage. NLP technology enables automatic connection between requirements and test cases at 89% success rate. Additionally, reinforcement learning techniques improve test case selection, reducing redundant test executions by 43%. Our research shows different ML methods work well to lessen incorrect error alerts. ML integration for QA surely increasing defect prediction accuracy and optimizing test execution time. Findings & Contributions - This research contributes to the field of AI-driven software testing by providing a comprehensive framework for ML-based QA methodologies. Our study shows that machine learning helps find more software problems better adapts test cases and lowers testing expenses to solve present software development needs. The study also identifies critical challenges, including data availability, model interpretability, and computational overhead, suggesting future research directions in Explainable AI (XAI), hybrid AI-ML testing models, and AI-driven security testing. As the industry moves toward AI-first software testing, this research paves the way for fully autonomous QA frameworks, enabling intelligent, scalable, and cost- effective software validation techniques. Keywords - Machine Learning, Software Testing, Quality Assurance, Predictive Testing, Adaptive Testing, Test Automation, Defect Prediction, Self- Healing Test Scripts, AI-Driven QA, Reinforcement

One of the main concerns in the software industry continues to be the development of high quality software. This concern will be exacerbated as software systems become more complex. The training of software developers continues to grow in academia since more institutions are offering software engineering (SE) courses. However, the list of topics that are expected to be covered in this course leaves little or no time for topics that focus on developing quality software, such as software testing and the use of testing tools.In this paper we describe an approach that non-intrusively integrates the use of software testing tools in SE courses. The cornerstone of our approach is the interaction students have with a Web-Based Repository of Software Testing Tools (WReSTT) that contains tutorials on testing concepts and testing tools. WReSTT employs both collaborative learning and social networking features that are attractive to students. We present the results of preliminary study performed in two SE courses that show how using the resources in WReSTT can potentially impact the students' understanding of software testing and the use of testing tools.

A Framework for Managing, Measuring, and Predicting Attributes of Software Development Products and Processes Reflecting the immense progress in the development and use of software metrics in the past decades, Software Metrics: A Rigorous and Practical Approach, Third Edition provides an up-to-date, accessible, and comprehensive introduction to software metrics. Like its popular predecessors, this third edition discusses important issues, explains essential concepts, and offers new approaches for tackling long-standing problems. New to the Third Edition This edition contains new material relevant to object-oriented design, design patterns, model-driven development, and agile development processes. It includes a new chapter on causal models and Bayesian networks and their application to software engineering. This edition also incorporates recent references to the latest software metrics activities, including research results, industrial case studies, and standards. Suitable for a Range of Readers With numerous examples and exercises, this book continues to serve a wide audience. It can be used as a textbook for a software metrics and quality assurance course or as a useful supplement in any software engineering course. Practitioners will appreciate the important results that have previously only appeared in research-oriented publications. Researchers will welcome the material on new results as well as the extensive bibliography of measurement-related information. The book also gives software managers and developers practical guidelines for selecting metrics and planning their use in a measurement program.

As software becomes more ubiquitous and complex, the cost of software bugs continues to grow at a staggering rate. To remedy this situation, there needs to be major improvement in the knowledge and application of software validation techniques. Although there are several software validation techniques, software testing continues to be one of the most widely used in industry. The high demand for software engineers in the next decade has resulted in more software engineering (SE) courses being offered in academic institutions. However, due to the number of topics to be covered in SE courses, little or no attention is given to software testing, resulting in students entering industry with little or no testing experience. We propose a minimally disruptive approach of integrating software testing into SE courses by providing students access to a collaborative learning environment containing learning materials on testing techniques and testing tools. In this article, we describe the learning environment and the studies conducted to measure the benefits accrued by students using the learning environment in the SE courses.

Software testing is an important process in software quality assurance(SQA). Software testing consumes the majority of the software developers' time and cost of all the phases of system development. An important problem in software testing is to determine whether a program has been tested enough with a testing criterion and when one could stop testing. On the other hand, software measurement is used to estimate the quality of a software. Complexity measures can be used to compare software quality and development cost. This paper proposes a new testing approach which choosing testing path based on path complexity. The process is to use a set of testing paths that satisfying all-uses testing criterion, and then choose the testing path with the most complexity to test. This process is continued until the program can be accepted. The proposed formula is used to determine if it is possible to stop testing before satisfying all-du-paths testing criterion. The main contribution of the paper is that software developers can choose a proper testing action with lower testing effort.

Software testing is an extremely important work link in software engineering. It runs through the entire life cycle of software engineering to ensure that software development meets requirements and can meet certain software quality requirements. With the continuous expansion of the scale of information systems, simple software testing can no longer meet the requirements of system and software quality assurance. This requires practitioners to gradually begin to think about and improve the testing methods of large-scale information software. For large-scale information software, testing will be embedded in each unit, and the software testing process will be organized in a more macroscopic way of thinking and organization. This article will summarize a series of methods suitable for large-scale information software testing through the research of software testing and the practice of the author. It is hoped that the research in this article can provide some help for the testing of large-scale information software, and also provide help for the precipitation and improvement of the author's own work.

Search-based software engineering (SBSE) consists of the application of metaheuristic search techniques—such as genetic algorithms, simulated annealing, hill climbing, and tabu search—to software engineering problems. This idea is based on the observation that some software engineering activities can be thought of as optimization problems, and on the fact that techniques providing exact solutions are, very often, unable to adequately scale to large, real-world software engineering problems. Define fitness functions to guide an automated search for solutions, rather than attempting to construct solutions from scratch by hand. SBSE moves the engineer up the abstraction chain, from the tedious and expensive role of detailed solution construction, to the potentially more productive and more cost-effective area of fitness function design. This approach has been repeatedly demonstrated to have successful applications across the software engineering life cycle from requirements and project planning to maintenance and testing. This special issue focuses on applications of search SBSE to the maintenance phase of the life cycle. After software testing 2, software maintenance is the most widely studied application area for SBSE research. Work on SBSE applications to problems in software maintenance can be traced back to the pioneering work of the Mancoridis et al. 3, which led to the development of the Bunch tool for software re-modularization 4. In the past five years there has been a significant increase in SBSE applications to software maintenance and evolution. Search-based techniques have been applied to post-development code improvement problems, such as object-oriented refactoring 5, 6, and imperative language slicing and transformation 7. Other applications are related to software engineering economics and project management problems, such as release planning 8-12, effort, and cost estimation 13-16, and project staffing and planning 17-20. Last, but not the least, SBSE has been applied to deal with service-oriented architectures, for example for services QoS-aware composition 21, service-level agreement negotiation 22, and testing 23. A complete overview of SBSE with a discussion of future research trends can be found in the recent International Conference on Software Engineering (ICSE) ‘Future of Software Engineering’ paper 24. This issue of the Journal of Software Maintenance and Evolution: Research and Practice features three outstanding papers, dealing with different aspects of software maintenance and evolution: release planning and staffing, re-factoring, and component substitution. The paper ‘Optimized Staffing for Product Releases—Focused Search and its Application at Chartwell Technologies’ by Puneet Kapur, An Ngo-The, Günther Ruhe, and Andrew Smith presents an approach for optimizing the staffing for product releases. The problem they solve can be viewed as part of the more general release planning problem, which assigns features to releases such that technical, resource, risk, and budget constraints are met. In particular, the genetic search-based approach the authors propose aims at assigning human resources to software development tasks, taking them from a pool of developers having different levels of skills. The applicability of the proposed approach is demonstrated with an industrial case study. The paper ‘Search-Based Refactoring: an Empirical Study’ by Mark O'Keffe and Mel Ó Cinnéide presents an approach to automatically re-factor, using search-based techniques, object-oriented systems that undergo repeated addition of functionality. Such addition can, very often, deteriorate the quality of the system's underlying design. Their re-factoring is guided by the QMOOD quality model 25, and the paper compares different optimization techniques, such as Genetic Algorithms, Simulated Annealing, and Multiple Ascent Hill-Climbing. The results of experiments carried out on five software systems suggest that the latter technique outperforms the others. The paper ‘Search-based Many-to-one Component Substitution’ by Nicolas Desnos, Marianne Huchard, Guy Tremblay, Christelle Urtado, and Sylvain Vauttier presents a search-based approach that allows the replacement of an obsolete component with a composition of multiple components. When a component becomes obsolete, the approach searches for a minimal assembly of other components, exposing their functionality by means of ports that preserves the behavior of the original component. The proposed approach has been evaluated on a set of randomly generated components. We hope that readers will enjoy this special issue and find the selected papers insightful and inspiring. We would like to thank all the authors who presented papers for this special issue: there were many good quality papers, but unfortunately we could accept only three of them. Last, but not the least, we would like to thank the reviewers who provided timely, detailed and constructive feedbacks to the authors. Without their precious help this special issue would not have been possible.

Many computer science students complete their undergraduate degrees with insufficient testing skills and knowledge. To understand the gaps in students' testing skills and knowledge, we analyzed 1014 software tests written by 12 groups in an undergraduate Software Quality Assurance (SQA) course project. In the project the student groups were provided a requirements document and were instructed to follow Test Driven Development (TDD) practices using black-box tests. To understand how the groups applied black-box testing in their project, we created an automatic tool to sort the tests into categories or "test buckets." By analyzing the test bucket data, we were able to assess the effectiveness and efficiency of student-written tests. We observed that the student groups were significantly more likely to test for explicit requirements than implicit requirements and significantly more likely to test happy paths than invalid inputs. Furthermore, students inefficiently tested happy paths, invalid inputs and explicit requirements resulting in a higher proportion of software tests with duplicate intent. Based on these results we provide insights into how black-box test education can be improved.

Background: The development of technology, both hardware and software, is rapidly advancing due to the growth of Artificial Intelligence (AI), which simplifies many human activities, including the software testing process. This research journal discusses how software testing is conducted through a Quality Management System (QMS). Objective: The journal examines whether software testing processes are related to QMS, how significantly QMS supports software testing, and the differences between testing with QMS and without QMS. Methods: This study outlines the stages of software testing through QMS, such as Software Quality Assurance (SQA), Quality Control (QC), Test Maturity Model Integration (TMMI), and performance indicators. According to the American Society for Quality (ASQ), QMS is a formal and documented system consisting of policies, processes, and procedures designed to achieve an organization's quality policies and objectives. Results: Several experts also define QMS as a system that focuses on improving customer satisfaction, process consistency, efficiency, and the overall quality of products or services. Software, according to Roger S. Pressman, is a set of instructions (programs), data structures that allow information manipulation, and related documentation describing operations and functions. Conclusion: Overall, QMS not only improves software product quality but also strengthens the testing process, making it more optimal, efficient, and aligned with organizational goals. The integration of QMS frameworks such as ISO 9001:2015, CMMI, and TMMI provides organizations with a competitive advantage by ensuring consistent delivery of high-quality software products that meet or exceed customer expectations.

Security affects every software component in different types of computing systems. Many vulnerabilities and attacks on software systems are due to security weaknesses in the software itself. During the process of software specification, development, or testing, security issues are either taken into consideration insufficiently or not at all. Such software, due to internal weaknesses is prone to new attacks. By teaching secure software engineering techniques for designing and developing software modules, students would learn systematic secure software development techniques, such as defect detecting and security testing. This paper presents a series of modules that are designed to be integrated into undergraduate software engineering courses from a security perspective. The goal of the modules is to teach the building of robust software security requirements, secure software design and development, and secure software verification through a secure software development lifecycle.

Time pressure in software engineering: A systematic review

Software Engineering courses are perceived as being dry and boring. The education community has been looking at active learning and hands-on training techniques to engage students better and thereby provide a better understanding of concepts. This paper presents a web-based game called Scrumify, developed by undergraduate students, that is designed to introduce students to the Scrum software development process through story elements, and quiz mechanics. The game is designed to reinforce software engineering concepts more specifically agile software processes. Quizzes within the game are designed to grant points that inject a sense of competition for the users and motivation to learn the agile methodology concepts. Scrumify serves two objectives: software engineering education for audience of the game and web-based game development and software testing education for the developers of the game. Use of this game in classroom provides an active learning technique for agile software processes. Students are more engaged with games and thereby absorb the concepts better. The game was evaluated in a sophomore software engineering class and the results are presented.

Software Quality Assurance (SQA) and Software Process Improvement (SPI) are topics of crucial importance for software engineers; however, teaching them in a lecture room comes with several limitations due to lack of practical experience. With that in mind, we created KUALI-Brick, a LEGO-based activity that brings SQA and SPI concepts together applying them in order to successfully build a LEGO city. This hands-on activity has been carried out in a fourth-year Software Engineering course at the University of Canterbury, with current results showing high levels of fun, increased engagement and an improved learning experience. We present a step-by-step guide to replicate the activity as well as lessons learned after conducting the activity for three consecutive years.