Integrating reuse into the rapid, continuous software engineering cycle through test-driven search

Software engineers have to cope with uncertainties and changing requirements. Agile methods provide flexibility towards changes and the emergence of continuous delivery has made regular feedback loops possible. The abilities to maintain high code quality through reviews, to regularly release software, and to collect and prioritize user feedback, are necessary for continuous software engineering (CSE). However, there exists no software process metamodel that handles the continuous character of software engineering. In this paper, we describe an empirical process metamodel for continuous software engineering called CSEPM, which treats development activities as parallel running workflows and allows tailoring and customization. CSEPM includes static aspects that describe the relations between specific CSE concepts including reviews, releases, and feedback. It also describes the dynamic aspect of CSE, how development workflows are activated through change events. We show how CSEPM allows to instantiate linear, iterative, agile and continuous process models and how it enables tailoring and customization.

Continuous software engineering refers to the organizational capability to develop, release and learn from software in very short rapid cycles, typically hours, days or a very small numbers of weeks. This requires not only agile processes in teams but in the complete research and development organization. Additionally, the technology used in the different development phases, like requirements engineering and system integration, must support the quick development cycles. Finally, automatic live experimentation for different system alternatives enables fast gathering of required data for decision making. The workshop, the second in the series after the first one at ICSE 2014, aims to bring the research communities of the aforementioned areas together to exchange challenges, ideas, and solutions to bring software engineering a step further to being a holistic continuous process. The workshop program is based on eight papers selected in the peer-review process and supplemented by interaction and discussions at the workshop. The topics range from agile methods, continuous software engineering practices to specific techniques, like visualization and testing.

This book provides essential insights on the adoption of modern software engineering practices at large companies producing software-intensive systems, where hundreds or even thousands of engineers collaborate to deliver on new systems and new versions of already deployed ones. It is based on the findings collected and lessons learned at the Software Center (SC), a unique collaboration between research and industry, with Chalmers University of Technology, Gothenburg University and Malm University as academic partners and Ericsson, AB Volvo, Volvo Car Corporation, Saab Electronic Defense Systems, Grundfos, Axis Communications, Jeppesen (Boeing) and Sony Mobile as industrial partners. The 17 chapters present the Stairway to Heaven model, which represents the typical evolution path companies move through as they develop and mature their software engineering capabilities. The chaptersdescribe theoretical frameworks, conceptual models and, most importantly, the industrial experiences gained by the partner companies in applying novel software engineering techniques. The books structure consists of six parts. Part I describes the model in detail and presents an overview of lessons learned in the collaboration between industry and academia. Part II deals with the first step of the Stairway to Heaven, in which R&D adopts agile work practices. Part III of the book combines the next two phases, i.e., continuous integration (CI) and continuous delivery (CD), as they are closely intertwined. Part IV is concerned with the highest level, referred to as R&D as an innovation system, while Part V addresses a topic that is separate from the Stairway to Heaven and yet critically important in large organizations: organizational performance metrics that capture data, and visualizations of the status of software assets, defects and teams. Lastly, Part VI presents the perspectives of two of the SC partner companies. The book is intended for practitioners and professionals in the software-intensive systems industry, providing concrete models, frameworks and case studies that show the specific challenges that the partner companies encountered, their approaches to overcoming them, and the results. Researchers will gain valuable insights on the problems faced by large software companies, and on how to effectively tackle them in the context of successful cooperation projects.

International audience

In this paper, we present a coverage-based regression test selection (RTS) approach and a developed tool for Python. The tool can be used either on a developer’s machine or on build servers. A special characteristic of the tool is the attention to easy integration to continuous integration and deployment. To evaluate the performance of the proposed approach, mutation testing is applied to three open-source projects, and the results of the execution of full test suites are compared to the execution of a set of tests selected by the tool. The missed fault rate of the test selection varies between 0–2% at file-level granularity and 16–24% at line-level granularity. The high missed fault rate at the line-level granularity is related to the selected basic mutation approach and the result could be improved with advanced mutation techniques. Depending on the target optimization metric (time or precision) in DevOps/MLOps process the error rate could be acceptable or further improved by using file-level granularity based test selection.

When encountering multiple and complicated models, multisource structured and unstructured data, complex requirements analysis, the platform design and integration of hydroinformatics systems become a challenge. To properly solve these problems, we describe a distributed software framework and it’s continuous integration process in hydroinformatics systems. This distributed framework mainly consists of server cluster for models, distributed database, GIS (Geographic Information System) servers, master node and clients. Based on it, a GIS - based decision support system for joint regulating of water quantity and water quality of group lakes in Wuhan China is established.

Software is a critical component of nearly every product or service. Improvements in software can lead to substantial competitive advantages. At the same time, software and surrounding engineering teams have become increasingly complex.The adoption of continuous integration and delivery is a recent trend to radically improve software release speed. However, its adoption is far from straightforward. Specifically, rethinking processes, organizational culture, ways of working, and business models require buy-in from diverse stakeholders that may have conflicting objectives. Such situations are explored by organizational conflict research.This paper reports on early lessons from an ongoing research project in continuous software engineering, specifically investigating adoption challenges from an organizational conflict perspective. We identify catalysts, symptoms, and outcomes of organizational conflicts hindering the adoption process.We conclude that predictable conflicts emerge when adopting continuous engineering. Engineers, managers, and other teams can proactively prepare for and allocate resources to resolve them. Proper analysis and management can help avoid wasted time, impeding processes, and frustration.

This paper covers the challenges in software validation, especially in the context of complex, embedded systems. Problems related to limited validation environment resources (DUT - Devices Under Test, workload and traffic generators, measurement setups, test probes), stability and complexity of systems will be highlighted. Authors will describe validation strategy which responds to quality assurance and risk management requirements applied for real products provided to the market. Presented technologies covers solutions based on Continuous Integration and Continuous Delivery process of modern, complex, embedded systems. Authors present both known and new solutions and approaches to known challenges, also suggest possible further development directions to improve efficiency and optimize product development cost.

Allure Reports, an open-source framework, significantly enhance the visualization and comprehensibility of test results in software testing. This paper explores the integration of Allure Reports with GitHub Workflows to automate the generation, publication, and management of detailed and interactive test reports. The primary focus is on improving communication, debugging processes, and overall testing efficiency through auto- mated, real-time reporting solutions. Key features of Allure Re- ports include interactive visualizations, comprehensive overviews, detailed test case information, customizable reports, and seamless integration with Continuous Integration/Continuous Deployment (CI/CD) pipelines. Additionally, the paper discusses the role of GitHub Pages and GitHub Artifacts in making test results accessible and supporting continuous integration and delivery processes. The implementation of this integration resulted in enhanced transparency, collaboration, and efficiency within development teams, as well as improved test result visibility and debugging capabilities. Empirical studies support our findings, highlighting improvements in task efficiency and reduced debugging time through the use of automated reporting and continuous integration practices [1- 3].

Faults in the automated continuous integration (CI) process can seriously impact the development of industrial code. To reduce manual intervention in automated CI processes, we want to understand better the CI systems' failure distribution to improve efficiency, reliability, and maintainability. This paper investigates failures in CI in four large industrial projects. We gather 11 731 builds over six months, identifying 1 414 failing builds. We also identify the distribution of different types of build failures in each of the four CI projects. Our results show that compilation is the most significant individual cause of failure with 47 %, followed by testing at 36 %. The checkout step with associated checks also incurs a non-negligible portion of failures with 12 %. Furthermore, we identify 14 distinct types of failures in the testing step. We conclude that configuration problems are a significant issue, as pipeline scripting and dependency errors make up a large number of failures.

We propose a new approach to solving important practical problems of complex debugging, joint testing, and analysis of the execution time of software module versions in a heterogeneous distributed computing environment that integrating Grid and cloud computing. These problems arise in the process of supporting the continuous integration of modules of distributed applied software packages. The study focuses on the packages that are used to conduct large-scale computational experiments. The scientific novelty of the proposed approach is to combine the methodology for creating the packages with modern software development practices based on its continuous integration using knowledge about the specifics of the problems being solved. Our contribution is multifold. We expanded the capabilities of continuous integration tools by developing new additional tools for the markup and transformation of data from poorly structured sources and predicting modules execution time. In addition, we developed a technological scheme of the joint applying our developed tools and external systems for continuous integration. Therefore, we provide a more large range of capabilities of continuous integration in relation to the processes of creating and using the packages in comparison with the well-known tools. The fundamental basis of their functioning is a new conceptual model of the packages. This model supports the specification, planning, and execution of software continuous integration processes taking into account the specific subject data and problems being solved. Applying the developed tools in practice leads to a decrease in the number of errors and failures of applied software in the development and use of the packages. In turn, such decrease significantly reduces the time for large-scale computational experiments and increases the efficiency of using resources of the environment. The results of practical experiments on the use of system prototype for continuous integration of applied software show their high efficiency.

Continuous delivery of cloud systems requires constant running of jobs (build processes, tests, etc.). One issue that plagues this continuous integration (CI) process are intermittent failures - non-deterministic, false alarms that do not result from a bug in the software or job specification, but rather from issues in the underlying infrastructure. At Mozilla, such intermittent failures are called oranges as a reference to the color of the build status indicator. As such intermittent failures disrupt CI and lead to failures, they erode the developers' trust in the jobs. We present a novel approach that automatically classifies failing jobs to determine whether job execution failures arise from an actual software bug or were caused by flakiness in the job (e.g., test) or the underlying infrastructure. For this purpose, we train classification models using job telemetry data to diagnose failure patterns involving features such as runtime, cpu load, operating system version, or specific platform with high precision. In an evaluation on a set of Mozilla CI jobs, our approach achieves precision scores of 73%, on average, across all data sets with some test suites achieving precision scores good enough for fully automated classification (i.e., precision scores of up to 100%), and recall scores of 82% on average (up to 94%).

The Biopython Project is a long-running distributed collaborative effort, supported by the Open Bioinformatics Foundation, which develops a freely available Python library for biological computation. This presentation was given at the Bioinformatics Open Source Conference (BOSC) 2016 in Orlando. We presented details of the latest Biopython releases – 1.66 and 1.67 – and development version – 1.68-dev – and new developments regarding Biopython docker containers, our continuous integration process, and our website. The video for this keynote presentation at BOSC 2016 can be viewed here .

This paper presents the design, development and evaluation of a software tool to assist the localisation of root causes of test case failures in distributed embedded systems, specifically vehicle systems controlled by a network of electronic control units (ECUs). We use data visualising to provide sensible information from a large number of test execution logs from large-scale software integration testing under a continuous integration process. Our goal is to allow more efficient root-cause identification of failures and foster a continuous feedback loop in the fault localisation process. We evaluate our solution in-situ at the Research and Development division of Volvo Car Corporation (VCC). Our prototype helps the failure debugging procedures by presenting clear and concise data and by allowing stakeholders to filter and control which information is displayed. Moreover, it encourages a systematic and continuous analysis of the current state of testing by aggregating and categorising historical data from test harnesses to identify patterns and trends in test results.

Unified Modeling Language (UML) plays an important role in the modern software development process by offering standardized methods for visual modeling, analysis, and design of complex systems. This paper discusses the main aspects of storing, editing, and integrating UML documents with integrated development environments (IDEs). The main focus is on comparing the two main formats for storing UML models – XML Metadata Interchange (XMI) and JavaScript Object Notation (JSON). XMI, being an official standard developed by Object Management Group, provides a high degree of detail and compatibility with various UML tools, making it preferable for large and complex projects that require support for the full UML specification. At the same time, JSON is simple and flexible, which makes it suitable for projects where speed of development and ease of integration are important, although it is inferior to XMI in describing complex aspects of UML models. The paper analyzes in detail the methods of editing UML documents, including manual editing and automated approaches, using APIs and scripts. Manual editing is useful when you need to make small changes to the structure of a UML document, but it can be time-consuming and error-prone when working with large projects. Automation, on the other hand, provides effective tools for mass editing and generating UML elements, which significantly speeds up development and minimizes the likelihood of errors, especially since modern tools such as StarUML offer the necessary tools for programmatically modifying UML documents by integrating them into development processes. Combining of UML documents with integrated development environments also plays a key role in increasing development efficiency, as this allows automating code generation based on UML models, maintaining synchronization between code and diagrams, and facilitating visualization and documentation of architectural solutions. Despite the significant advantages, there are challenges associated with limited support for all UML features and potential conflicts between models and code. This only emphasizes the need for further research in this area. Prospects for further research include the development of new methods and tools for working with UML documents, improving their integration with development environments, and using artificial intelligence to automate the analysis and design of UML models. Attention should also be paid to the using of UML with continuous integration processes, which can significantly increase the flexibility and adaptability of software development. Thus, the study of the structure of storing, editing, and integrating UML documents is an important area for optimizing the design and development of modern software systems.