Normalizing source code vocabulary to support program comprehension and software quality

<italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Feature Location (FL)</i> is a core software maintenance activity that aims to locate observable functionalities in the source code. Given its key role in software change, a vast array of Feature Location Techniques (FLTs) have been proposed but, as more and more FLTs are introduced, the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">selection of an appropriate FLT</i> is an increasingly difficult problem. One consideration is the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">characteristics of the features</i> being sought. For example, in the code associated with the feature, programmers may have named identifiers consistently, and with meaningful naming conventions, or not, and this may impact on the suitability of different FLTs. The suggestion that such characteristics matter has implicit support in the literature: An analysis of existing FLT empirical studies reveals that the system under study can often have a stronger impact on FLT performance than differing FLTs themselves. To understand this interaction between feature characteristics and FLTs better, this paper proposes <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">a suite of feature-characteristic metrics</i> that are postulated to control FLTs’ performance, holistically across FLTs and impacting on individual FLTs to different degrees. To evaluate the suite, a controlled experiment is performed, using 878 features, to probe the relationship between the metrics and the performance of four FTL techniques: three commonly-used techniques and one state-of-the-art technique. The evaluation is performed using four commonly used evaluation measures and extended by employing 41 other established source-code metrics as extraneous variables. Results of the empirical evaluation suggest that the feature-metric suite presented impacts FLT performance holistically, and impacts different FLTs to different degrees. Thus, this paper moves towards the more standard selection of appropriate FLTs, with respect to the prominent feature characteristics in the software systems under study, and more rigorous consideration of the features selected to compare FLTs.

Feature Location (FL) aims to locate observable functionalities in source code. Considering its key role in software maintenance, a vast array of automated and semi-automated Feature Location Techniques (FLTs) have been proposed. To compare FLTs, an open, standard set of non-subjective, reproducible “compare-to” FLT techniques (baseline techniques) should be used for evaluation. In order to relate the performance of FLTs compared against different baseline techniques, these compare-to techniques should be evaluated against each other. But evaluation across FLTs is confounded by empirical designs that incorporate different FL goals and evaluation criteria. This paper moves towards standardizing FLT comparability by assessing eight baseline techniques in an empirical design that addresses these confounding factors. These baseline techniques are assessed in twelve case studies to rank their performance. Results of the case studies suggest that different baseline techniques perform differently and that VSM-Lucene and LSI-Matlab performed better than other implementations. By presenting the relative performances of baseline techniques this paper facilitates empirical cross-comparison of existing and future FLTs. Finally, the results suggest that the performance of FLTs partially depends on system/benchmark characteristics, in addition to the FLTs themselves.

Presence of large number of Feature Location Techniques (FLTs) poses difficulties when selecting an appropriate FLT, given a software maintenance task. This problem is aggravated by extensive heterogeneity in empirical designs employed to evaluate the FLTs and one such heterogeneity that may feed into conflicting findings across studies, is the feature set sought in evaluations. An analysis of the empirical findings of FL studies suggests that (sought) feature characteristics can have a stronger impact on FLTs performance than differing FLTs. Towards understanding their impact, this paper proposes two feature metrics that are hypothesized as affecting FLTs performances. To evaluate the presented metrics, a controlled experiment on 461 features gathered from four software systems was performed. The focus was to establish the relationship between the metrics and FLT performance. Results of the empirical evaluation suggest that the presented feature metrics strongly impact performance of different FLTs, as measured using established evaluation measures. Thus, this paper facilitates a more standard, transparent selection of feature benchmarks towards fair comparison of FLTs.

In the past decade, there have been many well-publicized cases of source code leaking from different well-known companies. These leaks pose a serious problem when the source code contains sensitive information encoded in its identifier names and comments. Unfortunately, redacting the sensitive information requires obfuscating the identifiers, which will quickly interfere with program comprehension. Program comprehension is key for programmers in understanding the source code, so sensitive information is often left unredacted. To address this problem, we offer a novel approach for REdacting Sensitive Information in Software arTifacts (RESIST). RESIST finds and replaces sensitive words in software artifacts in such a way to reduce the impact on program comprehension. We evaluated RESIST experimentally using 57 professional programmers from over a dozen different organizations. Our evaluation shows that RESIST effectively redacts software artifacts, thereby making it difficult for participants to infer sensitive information, while maintaining a desired level of comprehension.

Software companies need to provide a large set of features satisfying functional and non-functional requirements of diverse customers, thereby leading to variability in space. Feature location techniques have been proposed to support software maintenance and evolution in space. However, so far only one feature location technique also analyses the evolution in time of system variants, which is required for feature enhancements and bug fixing. Specifically, existing tools for managing a set of systems over time do not offer proper support for keeping track of feature revisions, updating existing variants, and creating new product configurations based on feature revisions. This paper presents four challenges concerning such capabilities for feature (revision) location and composition of new product configurations based on feature/s (revisions). We also provide a benchmark containing a ground truth and support for computing metrics. We hope that this will motivate researchers to provide and evaluate tool-supported approaches aiming at managing systems evolving in space and time. Further, we do not limit the evaluation of techniques to only this benchmark: we introduce and provide instructions on how to use a benchmark extractor for generating ground truth data for other systems. We expect that the feature (revision) location techniques maximize information retrieval in terms of precision, recall, and F-score, while keeping execution time and memory consumption low.

Software maintainers are often challenged with source code changes to improve software systems, or eliminate defects, in unfamiliar programs. To undertake these tasks a sufficient understanding of the system (or at least a small part of it) is required. One of the most time consuming tasks of this process is locating which parts of the code are responsible for some key functionality or feature. Feature (or concept) location techniques address this problem. This paper introduces Conclave, an environment for software analysis, and in particular the Conclave-Mapper tool that provides a feature location facility. This tool explores natural language terms used in programs (e.g. function and variable names), and using textual analysis and a collection of Natural Language Processing techniques, computes synonymous sets of terms. These sets are used to score relatedness between program elements, and search queries or problem domain concepts, producing sorted ranks of program elements that address the search criteria, or concepts. An empirical study is also discussed to evaluate the underlying feature location technique.

Feature location is a maintenance task to identify the implementation of a feature within the source code. To automate or support the task, extensive studies have been conducted on feature location techniques. In this paper, we focus on certain static and dynamic constraints regarding feature additions to object-oriented programs, and construct an interactive feature location procedure based on the constraints. We manually conducted a case study for several features of a real-world program on the assumption that the user always correctly answers the questions asked by the procedure. The results show that over 75% of the feature's implementation could be efficiently covered by the procedure with relatively small number of execution traces.

This special issue of EMSE comprises four selected papers on program comprehension, an unusually wide-ranging research area within software engineering. It encompasses the human activities of comprehending software, the technologies for supporting the program comprehension, and their evaluation. It is notable for combining aspects of cognitive psychology, tool design, and empirical study of users, tools, and software artifacts. In “High-MCC Functions in the Linux Kernel”, the authors challenge the assumption that functions with high values of McCabe Cyclomatic Complexity (MCC) are likely to be problematic to understand and maintain. They explore the nuances of what a high MCC value might mean, and examine functions from the Linux kernel with high MCC values. They find that often the structure of these functions is found to be easy to understand by users, leading to the conclusion that syntactic measures alone may not be a good metric for understandability. In “Measuring and Modeling Programming Experience”, the authors describe a controlled experiment in assessing how the experience of software developers can best be measured. This is a particularly important problem in program comprehension research, as developer experience is often a key factor that must be explicitly taken into account when studying how programmers perform problem-solving tasks. In “An Empirical Study on the Impact of Static Typing on Software Maintainability”, the authors tackle one of the classic programming language debates: How does static typing affect human software development capabilities? The authors approach this question from an empirical perspective. In a carefully designed experiment, the authors assess the impact of static typing on understanding undocumented code, fixing type errors, and fixing semantic errors. The results show rigorous empirical evidence that static types are indeed beneficial to these activities, except when fixing semantic errors. In “Labeling Source Code with Information Retrieval Methods: An Empirical Study”, the authors address augmenting source code with explanatory labels. Automated techniques for this exist, which are based on, e.g., Vector Space Models, Latent Semantic Indexing (LSI), and latent Dirichlet allocation (LDA). The authors describe an experiment in which such Empir Software Eng (2014) 19:1259–1260 DOI 10.1007/s10664-014-9329-5

Feature location, an important task in program comprehension, occurs when the developer identifies the source code entity or entities responsible for implementing a functionality. Researchers have applied static analysis techniques to multiple software maintenance tasks, including feature localization. Static analysis techniques operate on a document corpus. Configuration and preprocessing decisions are required to build a suitable source code corpus for a static analysis technique. Currently, there is little guidance in the software engineering literature for making such configuration decisions. This paper focuses on two preprocessing methods for source code corpora, identifier splitting and stop word lists. We experiment on three open source Java test suites, i.e. Mylyn 1.0.1, Rhino 1.5R5, and Rhino 1.6R5. Our results indicate that identifier splitting and stop word list decisions do not significantly affect the performance of the LDA based feature location technique.

To support program comprehension, software artifacts can be labeled--for example within software visualization tools--with a set of representative words, hereby referred to as labels. Such labels can be obtained using various approaches, including Information Retrieval (IR) methods or other simple heuristics. They provide a bird-eye's view of the source code, allowing developers to look over software components fast and make more informed decisions on which parts of the source code they need to analyze in detail. However, few empirical studies have been conducted to verify whether the extracted labels make sense to software developers. This paper investigates (i) to what extent various IR techniques and other simple heuristics overlap with (and differ from) labeling performed by humans; (ii) what kinds of source code terms do humans use when labeling software artifacts; and (iii) what factors--in particular what characteristics of the artifacts to be labeled--influence the performance of automatic labeling techniques. We conducted two experiments in which we asked a group of students (38 in total) to label 20 classes from two Java software systems, JHotDraw and eXVantage. Then, we analyzed to what extent the words identified with an automated technique--including Vector Space Models, Latent Semantic Indexing (LSI), latent Dirichlet allocation (LDA), as well as customized heuristics extracting words from specific source code elements--overlap with those identified by humans. Results indicate that, in most cases, simpler automatic labeling techniques--based on the use of words extracted from class and method names as well as from class comments--better reflect human-based labeling. Indeed, clustering-based approaches (LSI and LDA) are more worthwhile to be used for source code artifacts having a high verbosity, as well as for artifacts requiring more effort to be manually labeled. The obtained results help to define guidelines on how to build effective automatic labeling techniques, and provide some insights on the actual usefulness of automatic labeling techniques during program comprehension tasks.

Before performing software maintenance and evolution tasks, developers must expend significant effort to understand the design of the subject software system. Reverse engineering and reengineering tools can extract artifacts, such as class diagrams, from source code so that the effort expended on program comprehension can be greatly reduced. Choosing suitable and appropriate tools is itself a difficult process. Many metrics and benchmarks have been proposed to evaluate the quality of reengineering tools, but the process is still complicated and requires much human effort. Furthermore, ignored by most evaluation experiments and approaches is the understanding of differences and similarities of the output artifacts produced by different tools. This paper proposes a novel approach for tool evaluation making a direct comparison of the output artifacts of the chosen tools. We apply our methodology on class diagrams extracted from source code. We utilize a class matching model to automate the process. In this study, we use ten open-source programs as test cases and two open source reverse engineering tools Doxygen and StarUML. We evaluate the output of these two tools based on class matching given the same input. This approach, we believe, provides developers with good guidance in tool selection.

Recovering traceability links between source code and software document can be very helpful for Software Maintenance and Software Reuse. Existing work has already achieved good results in extracting code elements (classes, methods, etc.) from software documents. However, it will lead to a lot of noise links if we link a document to all the code elements existing in it. In this paper, we propose an approach to identify the contextual code elements and the salient code elements in a software document, then we can weight the traceability links between source code and software document so that those noise traceability links can be filtered effectively. We measure the saliency of each code element in a document with four kinds of document-related features and three kinds of code-related features, and we adopt TransR-based code embedding technology to evaluate the distance between code elements. In the experiments, we get a precision of 70.7% in recognizing salient code elements of StackOverflow answer documents, which is more than 12% improvement compared with Rigby's work. At the same time, we can filter about 56.5%~69.3% noise traceability links compared with the RecoDoc approach. It will improve the quality of traceability links between source code and related software documents.

Identifying and extracting parts of a system's implementation for reuse is an important task for re-engineering system variants into Software Product Lines (SPLs). An SPL is an approach that enables systematic reuse of existing assets across related product variants. The re-engineering process to adopt an SPL from a set of individual variants starts with the location of features and their implementation, to be extracted and migrated into an SPL and reused in new variants. Therefore, feature location is of fundamental importance to the success in the adoption of SPLs. Despite its importance, existing feature location techniques struggle with huge, complex, and numerous system artifacts. This is the scenario of ArgoUML-SPL, which stands out as the most used case study for the validation of feature location approaches. In this paper we use an automated feature location technique and apply it to the ArgoUML feature location challenge posed.

Software Quality Assurance is a sub-process that ensures that developed software meets and complies with defined or standardized quality specifications. Focusing on source code, there are characteristics that can be used to evaluate the quality. Introductory courses must encourage freshmen students to improve internal quality of their source code, but only as sophomore they have contact with Software Engineering concepts, including Quality Assurance. In this paper we present a tool to source code quality evaluation aimed at supporting students to improve their source code and, consequently, their programming skills. The proposed tool uses quality reports (available to professional environment integrate with software repositories) to analyze students' source code and provide a feedback about the student coding. The proposed tool run locally, with few computational resources. In addition, we proposed the methodology to use the proposed tool: it consists of challenging students to perform a set of maintenance tasks in a controlled environment. We prepared a source code by introducing common defects, what decreases the quality of source code, and ask to students to perform maintenance tasks in order to both eliminate the introduced defects and introduce new features. After each modification, the students must evaluate their code using the proposed tool to obtain a feedback about quality of source code. To evaluate the approach and the tool, we created a survey and applied to students and the teacher. As a result, we show the benefits of using the proposed tool to both teachers and students perspectives. The results are positive to enhance the teaching-learning Software Quality Assurance to Software Engineering students.

Change impact analysis techniques estimate the potential effects of changes made to software. Directed Incremental Symbolic Execution (DiSE) is an intraprocedural technique for characterizing the impact of software changes on program behaviors. DiSE first estimates the impact of the changes on the source code using program slicing techniques, and then uses the impact sets to guide symbolic execution to generate path conditions that characterize impacted program behaviors. DiSE, however, cannot reason about the flow of impact between methods and will fail to generate path conditions for certain impacted program behaviors. In this work, we present iDiSE, an extension to DiSE that performs an interprocedural analysis. iDiSE combines static and dynamic calling context information to efficiently generate impacted program behaviors across calling contexts. Information about impacted program behaviors is useful for testing, verification, and debugging of evolving programs. We present a case-study of our implementation of the iDiSE algorithm to demonstrate its efficiency at computing impacted program behaviors. Traditional notions of coverage are insufficient for characterizing the testing efforts used to validate evolving program behaviors because they do not take into account the impact of changes to the code. In this work we present novel definitions of impacted coverage metrics that are useful for evaluating the testing effort required to test evolving programs. We then describe how the notions of impacted coverage can be used to configure techniques such as DiSE and iDiSE in order to support regression testing related tasks. We also discuss how DiSE and iDiSE can be configured for debugging; finding the root cause of errors introduced by changes made to the code. In our empirical evaluation we demonstrate that the configurations of DiSE and iDiSE can be used to support various software maintenance tasks.