Ontology-based Program Comprehension Tool Supporting Website Architectural Evolution

Cognitive science emphasizes the strength of visual formalisms for human learning and problem solving. In software engineering, a clear, visual presentation of a system's architecture can significantly reduce the effort of comprehension. Yet, all too often the documentation of complex software systems lacks clear identification of the architectural constituents and insufficiently relates them to the source code. It is our contention that visualization of the architectural constituents within the source code model is an indispensable aid for the guided evolution of large-scale software systems. We present a prototype tool for visualizing both published, generic design patterns as well as well-thought, ad-hoc design solutions, given the reverse-engineered source code of a system. We discuss the architecture and core functionality of this tool, addressing source code reverse engineering, design repository, design representation, and design clustering. Then, we present our visualization objectives and detail our techniques for pattern visualization. A case study example helps explicate and illustrate our work.

This paper describes the architecture of an ongoing developed Flexible and Extensible Program comprehension Support System (FEPSS). Program comprehension is a process for supporting users to understand a software system and is required for forward, reverse engineering, and re-engineering. In a program comprehension process some artifacts such as control-flow graphs and data-flow graphs need to be generated to help users in understanding and some artifacts such as design rationale and testing cases also need to be recorded for further re-engineering. It is impossible to predict all the types of artifacts to be supported in a program comprehension process as at least no one makes a re-engineering plan when designing a new software system. Therefore a program comprehension system should support a great flexibility and extensibility in artifact modelling. The approach which FEPSS takes is to provide a support system where the users can add new artifacts even at run time. FEPSS uses a relational object model to link the artifacts in order to integrate various activities in a program comprehension process as a whole. To help users to understand the artifacts and their relationships FEPSS uses 3D visualization to display artifacts. As the number of the artifacts for a software system can be very large FEPSS provides a navigation tool for users to control what artifacts to display. Based on these mechanisms, FEPSS can support users to analyze, model, and explore the artifacts through pictures.

In the past decade, there have been many well-publicized cases of source code leaking from different well-known companies. These leaks pose a serious problem when the source code contains sensitive information encoded in its identifier names and comments. Unfortunately, redacting the sensitive information requires obfuscating the identifiers, which will quickly interfere with program comprehension. Program comprehension is key for programmers in understanding the source code, so sensitive information is often left unredacted. To address this problem, we offer a novel approach for REdacting Sensitive Information in Software arTifacts (RESIST). RESIST finds and replaces sensitive words in software artifacts in such a way to reduce the impact on program comprehension. We evaluated RESIST experimentally using 57 professional programmers from over a dozen different organizations. Our evaluation shows that RESIST effectively redacts software artifacts, thereby making it difficult for participants to infer sensitive information, while maintaining a desired level of comprehension.

Comprehending software is hard. There's an inversely proportional relationship between a software system's complexity and size on the one hand and the usability of existing approaches for program comprehension on the other. Ironically, the need for these approaches increases directly proportional with the size of the system. In this work, we describe our approach and tool Osiris for automatic generation of concern models from source code. Our approach helps in program comprehension by identifying coherent sets of methods, grouping them into concerns and propagating these concerns along the concern graph of the application.

Developers spend a substantial amount of their time with program comprehension. To improve their comprehension and refresh their memory, developers need to communicate with other developers, read the documentation, and analyze the source code. Many studies show that developers focus primarily on the source code and that small improvements can have a strong impact. As such, it is crucial to bring the code itself into a more comprehensible form. A particular technique for this purpose are explicit feature traces to easily identify a program’s functionalities. To improve our empirical understanding about the effects of feature traces, we report an online experiment with 49 professional software developers. We studied the impact of explicit feature traces, namely annotations and decomposition, on program comprehension and compared them to the same code without traces. Besides this experiment, we also asked our participants about their opinions in order to combine quantitative and qualitative data. Our results indicate that, as opposed to purely object-oriented code: (1) annotations can have positive effects on program comprehension; (2) decomposition can have a negative impact on bug localization; and (3) our participants perceive both techniques as beneficial. Moreover, none of the three code versions yields significant improvements on task completion time. Overall, our results indicate that lightweight traceability, such as using annotations, provides immediate benefits to developers during software development and maintenance without extensive training or tooling; and can improve current industrial practices that rely on heavyweight traceability tools (e.g., DOORS) and retroactive fulfillment of standards (e.g., ISO-26262, DO-178B).

The literature reports that source code lexicon plays a paramount role in program comprehension, especially when software documentation is scarce, outdated or simply not available. In source code, a significant proportion of vocabulary can be either acronyms and-or abbreviations or concatenation of terms that can not be identified using consistent mechanisms such as naming conventions. It is, therefore, essential to disambiguate concepts conveyed by identifiers to support program comprehension and reap the full benefit of Information Retrieval-based techniques (e.g., feature location and traceability) whose linguistic information (i.e., source code identifiers and comments) used across all software artifacts (e.g., requirements, design, change requests, tests, and source code) must be consistent. To this aim, we propose source code vocabulary normalization approaches that exploit contextual information to align the vocabulary found in the source code with that found in other software artifacts. We were inspired in the choice of context levels by prior works and by our findings. Normalization consists of two tasks: splitting and expansion of source code identifiers. We also investigate the effect of source code vocabulary normalization approaches on software maintenance tasks. Results of our evaluation show that our contextual-aware techniques are accurate and efficient in terms of computation time than state of the art alternatives. In addition, our findings reveal that feature location techniques can benefit from vocabulary normalization approaches when no dynamic information is available.

This special issue of EMSE comprises four selected papers on program comprehension, an unusually wide-ranging research area within software engineering. It encompasses the human activities of comprehending software, the technologies for supporting the program comprehension, and their evaluation. It is notable for combining aspects of cognitive psychology, tool design, and empirical study of users, tools, and software artifacts. In “High-MCC Functions in the Linux Kernel”, the authors challenge the assumption that functions with high values of McCabe Cyclomatic Complexity (MCC) are likely to be problematic to understand and maintain. They explore the nuances of what a high MCC value might mean, and examine functions from the Linux kernel with high MCC values. They find that often the structure of these functions is found to be easy to understand by users, leading to the conclusion that syntactic measures alone may not be a good metric for understandability. In “Measuring and Modeling Programming Experience”, the authors describe a controlled experiment in assessing how the experience of software developers can best be measured. This is a particularly important problem in program comprehension research, as developer experience is often a key factor that must be explicitly taken into account when studying how programmers perform problem-solving tasks. In “An Empirical Study on the Impact of Static Typing on Software Maintainability”, the authors tackle one of the classic programming language debates: How does static typing affect human software development capabilities? The authors approach this question from an empirical perspective. In a carefully designed experiment, the authors assess the impact of static typing on understanding undocumented code, fixing type errors, and fixing semantic errors. The results show rigorous empirical evidence that static types are indeed beneficial to these activities, except when fixing semantic errors. In “Labeling Source Code with Information Retrieval Methods: An Empirical Study”, the authors address augmenting source code with explanatory labels. Automated techniques for this exist, which are based on, e.g., Vector Space Models, Latent Semantic Indexing (LSI), and latent Dirichlet allocation (LDA). The authors describe an experiment in which such Empir Software Eng (2014) 19:1259–1260 DOI 10.1007/s10664-014-9329-5

We introduce an efficient parametric model checking (ePMC) method for the analysis of reliability, performance and other quality-of-service (QoS) properties of software systems. ePMC speeds up the analysis of parametric Markov chains modelling the behaviour of software by exploiting domain-specific modelling patterns for the software components (e.g., patterns modelling the invocation of functionally-equivalent services used to jointly implement the same operation within service-based systems, or the deployment of the components of multi-tier software systems across multiple servers). To this end, ePMC precomputes closed-form expressions for key QoS properties of such patterns, and uses these expressions in the analysis of whole-system models. To evaluate ePMC, we show that its application to service-based systems and multi-tier software architectures reduces the analysis time by several orders of magnitude compared to current parametric model checking methods.

System evolution challenges persist as IS scholarship remains entrenched in static models inadequately capturing how systems drift over time, while siloed team and architecture studies obscure their dynamic interplay. This fragmentation obscures the mechanisms—the “how”—through which coevolving team and technical architectures shape evolutionary outcomes. Our theory, building on Simon’s decomposability idea, traces systems’ evolution to the dynamic syncing of drifting technical and team architectures. We theorize two mechanisms of system evolution—module evolution (internal changes) and architecture evolution (interconnection changes)—each differentially catalyzed by system and team architecture; their syncing simultaneously activates both mechanisms. Econometric analysis of 1,303 open-source systems encompassing 10.1 million evolutionary events across 4.8 billion lines of code supports the proposed ideas. Our contributions are threefold. First, we reconceptualize system and team architectures as temporally dynamic properties, shifting the discourse from static snapshots to dynamic evolutionary trajectories. Second, we introduce module evolution and architecture evolution as mechanistically-distinct pathways differentially catalyzed by system and team architecture, enlarging the discourse from magnitude to the content of change. Third, we show how dynamic system-team syncing sustains systems’ evolution by continuously restoring alignment as inevitable architectural drift erodes their congruence. Overall, our mechanistic theory underscores the sociotechnical dynamics through which architectural syncing drives systems evolution.

Many organizations and open-source projects use the C preprocessor (CPP) to implement configurability in their software systems. Despite extensive research, existing studies on the effects of CPP use on program comprehension are still limited to experiences, opinions, and empirical studies with narrow scopes. So, it is unclear whether the CPP actually leads to what is sometimes referred to as "#ifdef hell." In this paper, we expand the existing evidence on program comprehension in the presence of CPP directives, but we also highlight a surprising dilemma. We conducted an empirical study, including an experiment and a questionnaire, on the impact of refactoring CPP directives with 521 experienced software developers. The results indicate that, in contrast to previous findings, comprehension performance slightly worsened in terms of correctness when our participants worked on code with refactored CPP directives. However, in alignment with previous findings, they preferred the refactored code, considering it more comprehensible and easier to work with. This dilemma of objective performance versus subjective preference is a surprising outcome that has not been found before. We argue that our work motivates the need for more studies to understand this dilemma-which may significantly impact common beliefs in research and practice.

Finding errors in the source code is a typical kinds of test in the programming courses, which is helpful to assess the students' ability in program comprehension and debugging. In this paper, we propose an analysis scheme to interpret the measured line-level eye movement in an error-finding test of C language. An eye-tracking measurement system is implemented to collect the student's eye movement; during the test, the students select the error lines by mouse click in the web pages. We divide the students' eye movement into multiple cognitive tasks according to their decision actions, i.e. the mouse click events. For each cognitive task, we formulate the line-level program execution sequence around the code line selected by the students. A new metric, namely Matched Movement Ratio (MMR), is proposed to describe the proportion of line-level eye-focus jumping events that matches with the program execution sequence. We validate the calculation of MMR by two case studies, and further utilize this metric to conduct group study for 19 students in a test about bubble-sorting algorithm. Experiment results show that, MMR can differentiate the students by their code tracing performance in error-finding missions. Although our scheme can not infer the whole cognitive process in test, MMR can work as a reference quantifying the students' cognitive efforts tracing the code lines. Our results provide evidences for the teacher to diagnose the weakness of students and provide improvement suggestions in programming courses.

With increasing demands and changing environment on software systems, a major challenge for those systems is to evolve themselves to adapt to these variations, especially during their running, where dynamic evolution of software architectures has been a key issue of software dynamic evolution research. Most current research in this direction focuses on describing dynamic evolution process of software architectures, and lack consistency analysis of dynamic evolution of software architectures. In this paper, we propose to represent software architectures with constraint hypergraphs, model dynamic evolution of software architectures with constraint hypergraph grammars, and discuss the consistency condition and corresponding consistency decision method of dynamic evolution of software architectures. Our approach provides a formal theoretical basis and a user-friendly graphical representation for consistency analysis of dynamic evolution of software architectures.

Before performing software maintenance and evolution tasks, developers must expend significant effort to understand the design of the subject software system. Reverse engineering and reengineering tools can extract artifacts, such as class diagrams, from source code so that the effort expended on program comprehension can be greatly reduced. Choosing suitable and appropriate tools is itself a difficult process. Many metrics and benchmarks have been proposed to evaluate the quality of reengineering tools, but the process is still complicated and requires much human effort. Furthermore, ignored by most evaluation experiments and approaches is the understanding of differences and similarities of the output artifacts produced by different tools. This paper proposes a novel approach for tool evaluation making a direct comparison of the output artifacts of the chosen tools. We apply our methodology on class diagrams extracted from source code. We utilize a class matching model to automate the process. In this study, we use ten open-source programs as test cases and two open source reverse engineering tools Doxygen and StarUML. We evaluate the output of these two tools based on class matching given the same input. This approach, we believe, provides developers with good guidance in tool selection.

To support program comprehension, software artifacts can be labeled--for example within software visualization tools--with a set of representative words, hereby referred to as labels. Such labels can be obtained using various approaches, including Information Retrieval (IR) methods or other simple heuristics. They provide a bird-eye's view of the source code, allowing developers to look over software components fast and make more informed decisions on which parts of the source code they need to analyze in detail. However, few empirical studies have been conducted to verify whether the extracted labels make sense to software developers. This paper investigates (i) to what extent various IR techniques and other simple heuristics overlap with (and differ from) labeling performed by humans; (ii) what kinds of source code terms do humans use when labeling software artifacts; and (iii) what factors--in particular what characteristics of the artifacts to be labeled--influence the performance of automatic labeling techniques. We conducted two experiments in which we asked a group of students (38 in total) to label 20 classes from two Java software systems, JHotDraw and eXVantage. Then, we analyzed to what extent the words identified with an automated technique--including Vector Space Models, Latent Semantic Indexing (LSI), latent Dirichlet allocation (LDA), as well as customized heuristics extracting words from specific source code elements--overlap with those identified by humans. Results indicate that, in most cases, simpler automatic labeling techniques--based on the use of words extracted from class and method names as well as from class comments--better reflect human-based labeling. Indeed, clustering-based approaches (LSI and LDA) are more worthwhile to be used for source code artifacts having a high verbosity, as well as for artifacts requiring more effort to be manually labeled. The obtained results help to define guidelines on how to build effective automatic labeling techniques, and provide some insights on the actual usefulness of automatic labeling techniques during program comprehension tasks.

The purpose of this study was to identify fault-prone functions that are likely to contain faults in a given software system. Five metrics were used: Di, an internal design metric which incorporates factors related to a function's internal structure; De, an external design metric which focuses on a function's external relationships to the rest of the software system; D(G), a composite design metric which is a linear combination of Di and De; and the union and intersection of Di, De, and D(G). Since the system being considered was already developed, a very important aspect of our study was to extract the design information directly from the source code rather than from the corresponding design documentation which may not exist or, if it does exist, it may be incomplete, difficult to understand, or not updated. To make the analysis more accurate and efficient, a metric analysis tool (χMetrics) was implemented. We conducted experiments using χMetrics on part of a distributed software system, written in C, with a client–server architecture, and identified a small percentage of its functions as good candidates for fault-proneness. Files containing these functions were then validated by the real defect data collected between a recent major release and its subsequent release for their fault-proneness. The results indicate that our metrics are good indicators of fault-prone functions. Two extra experiments were also conducted to show that function size cannot replace any of our metrics; and where the function size was factored out our metrics performed better than the normalized metrics. The important benefit of our metrics is that they help project managers determine where additional testing effort should be spent and possibly which fault-prone functions should be assigned to more experienced programmers if modifications are required. Copyright © 2000 John Wiley & Sons, Ltd.