Practitioners’ Perspectives on Change Impact Analysis for Safety-Critical Software – A Preliminary Analysis

"Change Impact Analysis" is the process of determining the consequences of a modification to software. In theory, change impact analysis should be done during software maintenance, to make sure changes do not introduce new bugs. Many approaches and techniques are proposed to help programmers do change impact analysis automatically. However, it is still an open question whether and how programmers do change impact analysis. In this paper, we conducted two studies, one in-depth study and one breadth study. For the in-depth study, we recorded videos of nine professional programmers repairing two bugs for two hours. For the breadth study, we surveyed 35 professional programmers using an online system. We found that the programmers in our studies did static change impact analysis before they made changes by using IDE navigational functionalities, and they did dynamic change impact analysis after they made changes by running the programs. We also found that they did not use any change impact analysis tools.

Software change impact analysis (CIA) methods enable developers to understand potential impacts of a code change so that the change can be executed confidently without affecting reliability of the software. However, existing CIA approaches do not support CIA for all source code granularities. Additionally, they lack support for inter-granular change impact queries and hidden dependencies. This paper introduces Augur, an automated static code analysis-based CIA approach that addresses these shortcomings. Augur infers and maintains semantic and environment dependencies along with data and control dependencies between source code entities across granularities. Additionally, Augur uses Change Impact Query Language, a novel query language for impact analysis proposed in this paper, to support inter-granular CIA queries with batch querying feature. Augur has been realized as a Visual Studio extension called Augur-Tool. We have conducted quantitative evaluation on two open-source and two industrial projects to assess the accuracy of the tool. Results from the evaluation indicate that Augur provides CIA with high accuracy (average precision 55% and average recall 85%).

FCA–CIA: An approach of using FCA to support cross-level change impact analysis for object oriented Java programs

Small changes can have major and nonlocal effects in object oriented languages, due to the extensive use of subtyping and dynamic dispatch. This makes it difficult to understand value flow through a program and complicates life for maintenance programmers. Change impact analysis provides feedback on the semantic impact of a set of program changes.The change impact analysis method presented in this thesis presumes the existence of a suite of regression tests associated with a Java program and access to the original and edited versions of the code. The primary goal of our research is to provide programmers with tool support that can help them understand why a test is suddenly failing after a long editing session by isolating the changes responsible for the failure. The tool analyzes two versions of an application and decomposes their difference into a set of atomic changes. Change impact is then reported in terms of affected tests whose execution behavior may have been modified by the applied changes. For each affected test, it also determines a set of affecting changes that were responsible for the test's modified behavior.The first contribution of this thesis is the demonstration of the utility of the basic change impact analysis framework of [51], by implementing a proof-of-concept prototype, Chianti, and applying it to Daikon, for an experimental validation.The second contribution is the definition and implementation of the dependences between atomic changes. Extensive experiments show that our dependences can help build the intermediate programs automatically in most cases.Another contribution is the heuristics for ranking the atomic changes for fault localization. This thesis proposes a heuristic that ranks method changes that might have affected a failed test, indicating the likelihood that they may have contributed to a test failure. Our results indicate that when a failure is caused by a single method change, our heuristic ranked the failure-inducing change as number 1 or number 2 of all the method changes in 67% of the delegate tests (i.e., representatives of all failing tests).

A systematic Change Impact Analysis (CIA) is being used for better change management of software. Also, CIA process is evolved continuously to make it more effective. Software metrics play an important role to evaluate CIA process. Two types of metrics are used to evaluate CIA. First types of metrics are the standard metrics used to evaluate the performance of CIA techniques for example Precision, Recall, F-measure etc. These are most commonly used by researchers. Second types of metrics are those which are used to quantify the change impact which is based on the code/design features. This paper is aimed at identification of these second types of metrics available in literature.

Software change impact analysis (CIA) is a key technique to identify the potential ripple effects of the changes to software. Coarse-grained CIA techniques such as file, class and method level techniques often gain less precise change impacts, which are difficult for practical use. Fine-grained CIA techniques, such as slicing, can be used to gain more precise change impacts, but need more time and space cost. In this paper, by combining the features of the coarse-grained technique and the fine-grained technique, a variable-method (VM) correlation-based CIA technique called VM-CIA is proposed to improve the precision of static CIA. First, the VM-CIA technique uses the abstract syntax tree (AST) of program to construct a novel intermediate representation called variable and method triple (VMT), which is used to analyze the correlation between the variables and methods. Second, the VM-CIA technique proposes the single-change impact analysis algorithm and multi-change impact analysis algorithm to compute the impact set based on the VMT representation. In addition, the VM-CIA technique can get a sorted impact set which is more accurate than the existing CIA techniques. The empirical results show that the VM-CIA technique can greatly improve the precision (19%) over traditional the CIA techniques, while at the cost of a little recall (5%). Moreover, the empirical studies also show that the VM-CIA technique predicts a ranked list of potential impact results according to the distance measure, which can greatly facilitate the practical use.

As business services become increasingly dependent on information technology (IT), it also becomes increasingly important to maximize the decision support for managing IT. Configuration Management Data Bases (CMDBs) store fundamental information about IT systems, such as the system's hardware, software and services. This information can help provide decision support for root cause analysis and change impact analysis. We have worked with our industrial research partner, CA, and with CA customers to identify challenges to the use of CMDBs to semi-automatically solve these problems. In this paper we propose a framework called DRACA (Decision Support for Root Cause Analysis and Change Impact Analysis). This framework mines key facts from the CMDB and in a sequence of three steps combines these facts with incident reports, change reports and expert knowledge, along with temporal information, to construct a probabilistic causality graph. Root causes are predicted and ranked by probabilistically tracing causality edges backwards from incidents to likely causes. Conversely, change impacts can be predicted and ranked by tracing from a proposed change forward along causality edges to locate likely undesirable impacts.

The documentation of software architecture relations as a kind of traceability information is considered important to help people understand the consequences or ripple-effects of architecture evolution. Traceability information provides a basis for analysing and evaluating software evolution, and consequently, it can be used for tasks like reuse evaluation and improvement throughout the evolution of software. To date, however, none of the published empirical studies on software architecture traceability have examined the validity of these propositions. In this paper, we hypothesize that impact analysis of changes in software architecture can be more efficient when supported by traceability links. To test this hypothesis, we designed two controlled experiments that were conducted to investigate the influence of traceability links on the quantity and quality of retrieved assets during architecture evolution analysis. The results provide statistical evidence that a focus on architecture traceability significantly reduces the quantity of missing and incorrect assets, and increases the overall quality of architecture impact analysis for evolution.

Software quality attributes, such as reliability, security, and usability, are often well-defined and understood at the requirement level. They lay the ground foundation necessary to achieve high-quality, robust, user-friendly, and trustworthy software systems. However, when addressing these attributes at the code level, two significant challenges emerge. First, they tend to scatter across the codebase due to improper encapsulation of object-oriented classes, hampering the visibility of quality-related components across the codebase. Second, they become tangled within a single module due to intricate interdependencies with functional aspects of the code. Addressing quality concerns in the presence of scattered and tangled code can lead to unforeseen issues. For example, software developers may inadvertently introduce new and latent bugs or incorrectly implement code components deviating from the original system-wide requirements. To tackle these pressing issues, this dissertation proposes a series of state-of-the-art solutions integrating ML-based techniques and NLP-based techniques, including static program analysis techniques, to automatically and effectively detect and repair quality concerns present at the code level, even when scattered across the codebase. Additionally, we introduce program structural analysis and change impact analysis, complemented by other unsupervised ML-based techniques, to disentangle quality-related changes from functional ones, to gain a holistic understanding of a particular maintenance task. From a practical stance, our techniques can be integrated for quality risk assessment purposes and incorporated into safety-critical continuous integration systems to monitor the evolving nature of quality concerns. Furthermore, they can enhance bidirectional traceability across evolving software artifacts, such as bug reports, commit messages, and source code, and facilitate bug triaging systems for prioritizing the resolution of highly impacted quality-related bugs.

Chapter 5 - Change-Effects Analysis for Evolving Software

Change impact analysis is generally regarded as a very difficult program comprehension problem. One of the reasons for this is that there is no universal definition for dependency between software artifacts, only algorithms that approximate the dependencies. In the past two decades, different kinds of algorithms have been developed by researchers. But which algorithm is the most suitable in a specific situation, which one finds the relevant dependencies in the best way? What kinds of dependencies are important for the programmers? What kinds of algorithms do they work with? Finding the most relevant dependencies is difficult, and it is essentially a creative mental task. A possible way to answer the above questions is to involve programmers in a survey, and listen to their subjective opinions based on expertise and experience in program comprehension. In this paper, we present the results of our survey on this theme. We wanted to know what the difference was between the results of some well-known algorithms and programmers´ opinions and, in addition, among programmers´ opinions as well. Hence we conducted a case study.

When modifying a cyber-physical system, the consequences of changes need to be understood beforehand to adequately assess risks and costs. Model-based change impact analysis is key for estimating the impact of a change before actually modifying the system. Existing change impact analysis approaches apply very similar algorithms for change propagation to instances of domain-specific metamodels. However, they lack fundamental concepts for domain-spanning change impact analysis. In this paper, we propose a generic methodology for domain-spanning change impact analysis to address limitations of existing approaches. Evaluation results show the relevancy and comprehensives of the methodology for several domains.

The way practitioners perform maintenance tasks in practice is little known by researchers. In turn, practitioners are not always up to date with the proposals provided by the research community. This work investigates the gap between software maintenance techniques proposed by the research community and the software maintenance practice. We carried out a survey with 112 practitioners from 92 companies and 12 countries. We concentrate on analyzing if and how practitioners understand and apply the following subjects: bad smells, refactoring, software metrics, and change impact analysis. This study shows that there is a large gap between research approaches and industry practice in those subjects, especially in change impact analysis and software metrics.

Enterprise architecture (EA) models the whole enterprise in various aspects regarding both business processes and information technology resources. As the organisation grows, the architecture of its systems and processes must also evolve to meet the demands of the business environment. Evolving an EA model may involve making changes to various components across different levels of the EA. As a result, an important issue before making a change to an EA model is assessing the ripple effect of the change, i.e. change impact analysis. Another critical issue is change propagation: given a set of primary changes that have been made to the EA model, what additional secondary changes are needed to maintain consistency across multiple levels of the EA. There has been however limited work on supporting the maintenance and evolution of EA models. This article proposes an EA description language, namely ChangeAwareHierarchicalEA, integrated with an evolution framework to support both change impact analysis and change propagation within an EA model. The core part of our framework is a technique for computing the impact of a change and a new method for generating interactive repair plans from Alloy consistency rules that constrain the EA model.

Software change is inevitable in the process of software development. At present, there are a lot of researches on the methods of the change impact analysis, but most of them are aimed at a certain stage of the software development process, and these technologies are not suitable for other stages of software development. In this paper, based on the whole software development process, five kinds of entity dependency types and five types of change are proposed and formalized, and their properties are discussed. This paper is to provide a theoretical basis for the research of rule based change impact analysis in the software development process.