Measuring Program Comprehension: A Large-Scale Field Study with Professionals

During software development and maintenance, developers spend a considerable amount of time on program comprehension. Previous studies show that program comprehension takes up as much as half of a developer's time. However, most of these studies are performed in a controlled setting, or with a small number of participants, and investigate the program comprehension activities only within the IDEs. However, developers' program comprehension activities go well beyond their IDE interactions.

Program comprehension is a required activity for many software development and maintenance tasks. A common means of understanding software is though UML diagrams. UML diagrams model the design and implementation of an application, describing both its structure and behaviour. However with rapid software development life cycles, UML diagrams often become inconsistent with the implementation of an application. This limits their effectiveness in program comprehension activities. This paper presents a program analysis framework to reverse engineer sequence diagrams from application source code. Part of this framework is the ability to filter out irrelevant operations from a sequence diagram in order to simplify the representation of an application. This is achieved by identifying important data, and following its use in the application. The purpose of this framework is to aid in program comprehension activities by providing up to date representations of an application; while also enabling developers to identify the logical operation of a program without interference from irrelevant or supporting operations.

Many techniques and tools have been developed to improve program comprehension but most tools are unsuitable for novices. Some tools are not user friendly, some designs are more suitable to expert programmers and some integrated development environments (IDEs) are very simple and fail to support program comprehension needs among the users. These factors hinder the learning process and may become obstacles to users who have no programming background. This paper attempts to improve program comprehension by using Zoom Visual Flow (ZViF) technique that represents source code in graphical views. It also gives some insights on how to improve visual presentation method in a program editor or an IDE. A lab experiment was conducted to determine the effectiveness of the technique. The respondents were asked to determine their preference between ZViF technique and Control Structured Diagram (CSD). CSD is one of the best techniques that uses graphic to visualize a program. The result shows that users prefer the proposed technique compared to CSD as the ZViF helps to improve program comprehension among novices.

An automated approach for noise identification to assist software architecture recovery techniques

Developers spend a substantial amount of their time with program comprehension. To improve their comprehension and refresh their memory, developers need to communicate with other developers, read the documentation, and analyze the source code. Many studies show that developers focus primarily on the source code and that small improvements can have a strong impact. As such, it is crucial to bring the code itself into a more comprehensible form. A particular technique for this purpose are explicit feature traces to easily identify a program’s functionalities. To improve our empirical understanding about the effects of feature traces, we report an online experiment with 49 professional software developers. We studied the impact of explicit feature traces, namely annotations and decomposition, on program comprehension and compared them to the same code without traces. Besides this experiment, we also asked our participants about their opinions in order to combine quantitative and qualitative data. Our results indicate that, as opposed to purely object-oriented code: (1) annotations can have positive effects on program comprehension; (2) decomposition can have a negative impact on bug localization; and (3) our participants perceive both techniques as beneficial. Moreover, none of the three code versions yields significant improvements on task completion time. Overall, our results indicate that lightweight traceability, such as using annotations, provides immediate benefits to developers during software development and maintenance without extensive training or tooling; and can improve current industrial practices that rely on heavyweight traceability tools (e.g., DOORS) and retroactive fulfillment of standards (e.g., ISO-26262, DO-178B).

Feature location is a program comprehension activity, the goal of which is to identify source code entities that implement a functionality. Recent feature location techniques apply text retrieval m...

This special issue of EMSE comprises four selected papers on program comprehension, an unusually wide-ranging research area within software engineering. It encompasses the human activities of comprehending software, the technologies for supporting the program comprehension, and their evaluation. It is notable for combining aspects of cognitive psychology, tool design, and empirical study of users, tools, and software artifacts. In “High-MCC Functions in the Linux Kernel”, the authors challenge the assumption that functions with high values of McCabe Cyclomatic Complexity (MCC) are likely to be problematic to understand and maintain. They explore the nuances of what a high MCC value might mean, and examine functions from the Linux kernel with high MCC values. They find that often the structure of these functions is found to be easy to understand by users, leading to the conclusion that syntactic measures alone may not be a good metric for understandability. In “Measuring and Modeling Programming Experience”, the authors describe a controlled experiment in assessing how the experience of software developers can best be measured. This is a particularly important problem in program comprehension research, as developer experience is often a key factor that must be explicitly taken into account when studying how programmers perform problem-solving tasks. In “An Empirical Study on the Impact of Static Typing on Software Maintainability”, the authors tackle one of the classic programming language debates: How does static typing affect human software development capabilities? The authors approach this question from an empirical perspective. In a carefully designed experiment, the authors assess the impact of static typing on understanding undocumented code, fixing type errors, and fixing semantic errors. The results show rigorous empirical evidence that static types are indeed beneficial to these activities, except when fixing semantic errors. In “Labeling Source Code with Information Retrieval Methods: An Empirical Study”, the authors address augmenting source code with explanatory labels. Automated techniques for this exist, which are based on, e.g., Vector Space Models, Latent Semantic Indexing (LSI), and latent Dirichlet allocation (LDA). The authors describe an experiment in which such Empir Software Eng (2014) 19:1259–1260 DOI 10.1007/s10664-014-9329-5

Developers use a large amount of their time to understand the system they work on, an activity referred to as program comprehension. Especially software evolution and forgetting over time lead to developers becoming unfamiliar with a system. To support them during program comprehension, we can employ knowledge recovery to reverse engineer implicit information from the system and the platform (e.g., GitHub) it is hosted on. However, to recover useful knowledge and to provide it in a useful way, we first need to understand what knowledge developers forget to what extent, what sources are reliable to recover knowledge, and how to trace knowledge to the features in a system. We tackle these three issues, aiming to provide empirical insights and tooling to support developers during software evolution and maintenance. The results help practitioners, as we support the analysis and understanding of systems, as well as researchers, showing opportunities to automate, for example, reverse-engineering techniques.

There are two essential ingredients for collecting information from the Internet: 1) a World Wide Web (Web) browser, and 2) an Internet search engine. The two most widely used Web browsers are Microsoft's Internet Explorer and Netscape Communications Corporation's Navigator. There are, however, more than 120 browser software possibilities available. These browsers have been cataloged by Adrian Roselli and may be found at Evolt, a site that supports the exchange of Web design information. Web browsers are like brands of shoes; some fit better than others. By visiting Evolt, whose uniform resource locator (URL) is listed below, one might find a browser that works better. For example, some Internet users prefer Firefox (Figure 1) and Safari, because Web browsers have different efficiencies in displaying Internet information on different computer platforms. Figure 1 Search page from the Web browser known as Firefox. Image used with permission of Mozilla, parent of Firefox. For a free copy of Firefox, which operates on either Apple or IBM compatibles, visit http://www.mozilla.org. Evolt http://browsers.evolt.org/ As is the case with Web browsers, there are a number of Internet search engines. Each search engine has a variety of features that address different objectives for finding information from the Internet. The State University of New York at Albany has organized an excellent list of Internet search engines.

Many organizations and open-source projects use the C preprocessor (CPP) to implement configurability in their software systems. Despite extensive research, existing studies on the effects of CPP use on program comprehension are still limited to experiences, opinions, and empirical studies with narrow scopes. So, it is unclear whether the CPP actually leads to what is sometimes referred to as "#ifdef hell." In this paper, we expand the existing evidence on program comprehension in the presence of CPP directives, but we also highlight a surprising dilemma. We conducted an empirical study, including an experiment and a questionnaire, on the impact of refactoring CPP directives with 521 experienced software developers. The results indicate that, in contrast to previous findings, comprehension performance slightly worsened in terms of correctness when our participants worked on code with refactored CPP directives. However, in alignment with previous findings, they preferred the refactored code, considering it more comprehensible and easier to work with. This dilemma of objective performance versus subjective preference is a surprising outcome that has not been found before. We argue that our work motivates the need for more studies to understand this dilemma-which may significantly impact common beliefs in research and practice.

Human factors in software development: On its underlying theories and the value of learning from related disciplines. A guest editorial introduction to the special issue

How to identify class comment types? A multi-language approach for class comment classification

Code comment generation typically refers to the process of generating concise natural language descriptions for a piece of code, which facilitates program comprehension activities. Inline code comments, as a part of code comments, are also crucial for program comprehension. Recently, the emergence of large language models (LLMs) has significantly boosted the performance of natural language processing tasks. This naturally inspires us to explore the performance of the LLMs in the task of inline code comment generation. To this end, we evaluate open-source LLMs on a large-scale dataset and compare the results with the current state-of-the-art methods. Specifically, we explore the model performance in the following scenarios based on the widely used evaluation metrics (i.e. BLEU, Meteor, and ROUGE-L): (1) generation with simple instruction; (2) few-shot-guided generation with random examples selected from the database; (3) few-shot-guided generation with similar examples selected from the database; and (4) adopt the re-ranking strategy for the output of LLMs. Our findings reveal that: (1) under the simple instruction scenario, LLMs could not fully show the potential in the task of inline comment generation compared to the state-of-the-art models; (2) random few-shot leads to a slight improvement; (3) similar few-shot and re-ranking strategy could significantly enhance the performance of LLMs; and (4) for inline comment and code snippet pairs with different intents, why category achieves the best performance and what category achieves relatively poorer performance. That remains consistent across all four scenarios. Our findings shed light on future research directions for using LLMs in inline comment generation tasks.

Program comprehension is an important activity in software maintenance, as software must be sufficiently understood before it can be properly modified. The study of a program's execution, known as dynamic analysis, has become a common technique in this respect and has received substantial attention from the research community, particularly over the last decade. These efforts have resulted in a large research body of which currently there exists no comprehensive overview. This paper reports on a systematic literature survey aimed at the identification and structuring of research on program comprehension through dynamic analysis. From a research body consisting of 4,795 articles published in 14 relevant venues between July 1999 and June 2008 and the references therein, we have systematically selected 176 articles and characterized them in terms of four main facets: activity, target, method, and evaluation. The resulting overview offers insight in what constitutes the main contributions of the field, supports the task of identifying gaps and opportunities, and has motivated our discussion of several important research directions that merit additional consideration in the near future.

SmartLife ecosystems are emerging as intelligent user-centered systems that will shape future trends in technology and communication. Biological metaphors of living adaptable ecosystems provide the logical foundation for self-optimizing and self-healing run-time environments for intelligent adaptable business services and related information systems with service-oriented enterprise architectures. The present research in progress work investigates mechanisms for adaptable enterprise architectures for the development of service-oriented ecosystems with integrated technologies like Semantic Technologies, Web Services, Cloud Computing and Big Data Management. With a large and diverse set of ecosystem services with different owners, our scenario of service-based SmartLife ecosystems can pose challenges in their development, and more importantly, for maintenance and software evolution. Our research explores the use of knowledge modeling using ontologies and flexible metamodels for adaptable enterprise architectures to support program comprehension for software engineers during maintenance and evolution tasks of service-based applications. Our previous reference enterprise architecture model ESARC--Enterprise Services Architecture Reference Cube--and the Open Group SOA Ontology was extended to support agile semantic analysis, program comprehension and software evolution for a SmartLife applications scenario. The Semantic Browser is a semantic search tool that was developed to provide knowledge-enhanced investigation capabilities for service-oriented applications and their architectures.