Quality Indicators in Search-based Software Engineering

In Search-Based Software Engineering (SBSE), users typically select a set of Multi-Objective Search Algorithms (MOSAs) for their experiments without any justification, or they simply choose an MOSA because of its popularity (e.g., NSGA-II). On the other hand, users know certain characteristics of solutions they are interested in. Such characteristics are typically measured with Quality Indicators (QIs) that are commonly used to evaluate the quality of solutions produced by an MOSA. Consequently, these QIs are often employed to empirically evaluate a set of MOSAs for a particular search problem to find the best MOSA. Thus, to guide SBSE users in choosing an MOSA that represents the solutions measured by a specific QI they are interested in, we present an empirical evaluation with a set of SBSE problems to study the relationships among commonly used QIs and MOSAs in SBSE. Our aim, by studying such relationships, is to identify whether there are certain characteristics of a QI because of which it prefers a certain MOSA. Such preferences are then used to provide insights and suggestions to SBSE users in selecting an MOSA, given that they know which quality aspects of solutions they are looking for.

Many software engineering problems are multi-objective in nature, which has been largely recognized by the Search-based Software Engineering (SBSE) community. In this regard, Pareto- based search algorithms, e.g., Non-dominated Sorting Genetic Algorithm II, have already shown good performance for solving multi-objective optimization problems. These algorithms produce Pareto fronts, where each Pareto front consists of a set of non- dominated solutions. Eventually, a user selects one or more of the solutions from a Pareto front for their specific problems. A key challenge of applying Pareto-based search algorithms is to select appropriate quality indicators, e.g., hypervolume, to assess the quality of Pareto fronts. Based on the results of an extended literature review, we found that the current literature and practice in SBSE lacks a practical guide for selecting quality indicators despite a large number of published SBSE works. In this direction, the paper presents a practical guide for the SBSE community to select quality indicators for assessing Pareto-based search algorithms in different software engineering contexts. The practical guide is derived from the following complementary theoretical and empirical methods: 1) key theoretical foundations of quality indicators; 2) evidence from an extended literature review; and 3) evidence collected from an extensive experiment that was conducted to evaluate eight quality indicators from four different categories with six Pareto-based search algorithms using three real industrial problems from two diverse domains.

This special issue of Empirical Software Engineering consists of revised and extended versions of five selected papers originally presented at the 1st International Symposium on Search Based Software Engineering (SSBSE 2009). The symposium was held at Cumberland Lodge, Windsor, UK, in May 2009, and brought together the academic researchers and industrial practitioners that make up the international Search Based Software Engineering (SBSE) community for the first conference dedicated to the subject. Search Based Software Engineering is the application of search-based optimization techniques—for example, genetic algorithms, hill climbing, simulated annealing, taboo search, and ant colony optimization—to the solution of software engineering problems. Many such problems are often difficult or impossible to solve manually, and moreover, as modern software becomes more complex and adopts distributed and highly parallel architectures, are proving to be increasingly intractable using traditional automated techniques. In contrast, SBSE approaches can leverage sophisticated optimisation algorithms running on affordable computing infrastructure to solve these problems. SBSE approaches also demonstrate other advantages: they can be used to find acceptably good solutions when time, or resources, are limited, and—by utilizing multi-objective search algorithms—can derive a range of solutions that illustrate potential trade-offs between competing goals. For these reasons, both academic and industrial interest in SBSE is increasing rapidly. A recent survey,1 undertaken by the UK-based SEBASE project, indicated an exponential growth of

Search-based software engineering (SBSE) consists of the application of metaheuristic search techniques—such as genetic algorithms, simulated annealing, hill climbing, and tabu search—to software engineering problems. This idea is based on the observation that some software engineering activities can be thought of as optimization problems, and on the fact that techniques providing exact solutions are, very often, unable to adequately scale to large, real-world software engineering problems. Define fitness functions to guide an automated search for solutions, rather than attempting to construct solutions from scratch by hand. SBSE moves the engineer up the abstraction chain, from the tedious and expensive role of detailed solution construction, to the potentially more productive and more cost-effective area of fitness function design. This approach has been repeatedly demonstrated to have successful applications across the software engineering life cycle from requirements and project planning to maintenance and testing. This special issue focuses on applications of search SBSE to the maintenance phase of the life cycle. After software testing 2, software maintenance is the most widely studied application area for SBSE research. Work on SBSE applications to problems in software maintenance can be traced back to the pioneering work of the Mancoridis et al. 3, which led to the development of the Bunch tool for software re-modularization 4. In the past five years there has been a significant increase in SBSE applications to software maintenance and evolution. Search-based techniques have been applied to post-development code improvement problems, such as object-oriented refactoring 5, 6, and imperative language slicing and transformation 7. Other applications are related to software engineering economics and project management problems, such as release planning 8-12, effort, and cost estimation 13-16, and project staffing and planning 17-20. Last, but not the least, SBSE has been applied to deal with service-oriented architectures, for example for services QoS-aware composition 21, service-level agreement negotiation 22, and testing 23. A complete overview of SBSE with a discussion of future research trends can be found in the recent International Conference on Software Engineering (ICSE) ‘Future of Software Engineering’ paper 24. This issue of the Journal of Software Maintenance and Evolution: Research and Practice features three outstanding papers, dealing with different aspects of software maintenance and evolution: release planning and staffing, re-factoring, and component substitution. The paper ‘Optimized Staffing for Product Releases—Focused Search and its Application at Chartwell Technologies’ by Puneet Kapur, An Ngo-The, Günther Ruhe, and Andrew Smith presents an approach for optimizing the staffing for product releases. The problem they solve can be viewed as part of the more general release planning problem, which assigns features to releases such that technical, resource, risk, and budget constraints are met. In particular, the genetic search-based approach the authors propose aims at assigning human resources to software development tasks, taking them from a pool of developers having different levels of skills. The applicability of the proposed approach is demonstrated with an industrial case study. The paper ‘Search-Based Refactoring: an Empirical Study’ by Mark O'Keffe and Mel Ó Cinnéide presents an approach to automatically re-factor, using search-based techniques, object-oriented systems that undergo repeated addition of functionality. Such addition can, very often, deteriorate the quality of the system's underlying design. Their re-factoring is guided by the QMOOD quality model 25, and the paper compares different optimization techniques, such as Genetic Algorithms, Simulated Annealing, and Multiple Ascent Hill-Climbing. The results of experiments carried out on five software systems suggest that the latter technique outperforms the others. The paper ‘Search-based Many-to-one Component Substitution’ by Nicolas Desnos, Marianne Huchard, Guy Tremblay, Christelle Urtado, and Sylvain Vauttier presents a search-based approach that allows the replacement of an obsolete component with a composition of multiple components. When a component becomes obsolete, the approach searches for a minimal assembly of other components, exposing their functionality by means of ports that preserves the behavior of the original component. The proposed approach has been evaluated on a set of randomly generated components. We hope that readers will enjoy this special issue and find the selected papers insightful and inspiring. We would like to thank all the authors who presented papers for this special issue: there were many good quality papers, but unfortunately we could accept only three of them. Last, but not the least, we would like to thank the reviewers who provided timely, detailed and constructive feedbacks to the authors. Without their precious help this special issue would not have been possible.

Despite preceding related publications, works dealing with the resolution of software engineering problems by search techniques has especially risen since 2001. By its first decade, the Search Based Software Engineering (SBSE) approach has been successfully employed in several software engineering contexts, using various optimization techniques. Aside the relevance of such applications, knowledge regarding the publication patterns on the field plays an important role to its understanding and identity. Such information may also shed light into SBSE trends and future. This paper presents the first bibliometric analysis to SBSE publications. The study covered 740 publications of the SBSE community from 2001 through 2010. The performed bibliometric analysis concerned mainly in four categories: Publication, Sources, Authorship, and Collaboration. Additionally, estimates for the next years of several publication metrics are given. The study also analyzed the applicability of bibliometric laws in SBSE, such as Bradfords and Lotka.

This challenges and open problems paper discusses the role of software engineers when combining modeling with Search-Based Software Engineering (SBSE). With SBSE techniques increasingly providing insights into difficult software modeling problems, the role of the software engineer might be thought of as diminishing. However, given the crucial role of the model understanding in SBSE, we argue that the role of the software engineer will become more important through interactive SBSE, although exploiting this interaction remains a challenge for the field.

In the past five years there has been a dramatic increase in work on Search-Based Software Engineering (SBSE), an approach to Software Engineering (SE) in which Search-Based Optimization (SBO) algorithms are used to address problems in SE. SBSE has been applied to problems throughout the SE lifecycle, from requirements and project planning to maintenance and reengineering. The approach is attractive because it offers a suite of adaptive automated and semiautomated solutions in situations typified by large complex problem spaces with multiple competing and conflicting objectives. This article 1 provides a review and classification of literature on SBSE. The work identifies research trends and relationships between the techniques applied and the applications to which they have been applied and highlights gaps in the literature and avenues for further research.

Search-Based Software Engineering (SBSE) tools offer automatic support by applying search-based techniques to solve different optimization problems of various software engineering areas. However, configuring SBSE tools often requires expertise in optimization techniques, posing a significant barrier for non-specialist users. This task involves selecting suitable algorithms, tuning parameters, choosing appropriate objective functions and search operators, and making decisions that affect optimization quality and performance. To address this challenge, we propose a multiple-LLM approach that integrates Large Language Models (LLMs), specifically ChatGPT-4, into SBSEworkflows to provide context-aware and natural language suggestions for parameter configuration. The approach follows a Divide and Conquer strategy, in which specialized LLMs are trained for distinct parameter configuration subproblems. As a proof of concept, we implemented OPLA-Wizard that works with OPLA-Tool v2.0, for Product Line Architecture (PLA) design. This assistant supports users in configuring algorithm settings, search operators, and objective functions through a user-friendly interface. Preliminary results show that the multiple-LLM approach improves accuracy, explanation clarity, and output format adherence compared to a single-LLM approach. This work contributes to the democratization of SBSE by reducing cognitive load, preventing misconfigurations, and making advanced optimization techniques more accessible to a broader audience.

FP-ABC: Fuzzy-Pareto dominance driven artificial bee colony algorithm for many-objective software module clustering

Software is an engineering material to be optimised. Until comparatively recently many computer scientists doubted this; why would one want to optimise something that could be made perfect by pure logical reasoning? However, the wider community has come to realise that, while very small programs may be perfect in isolation, larger software systems may never be (because the world in which they operate is not perfect). Once we accept this, we soon arrive at evolutionary computation as a means of optimising software. However, software is not merely some other engineering material to be optimised. Software is virtual and inherently adaptive, making it better suited to evolutionary computation than any other engineering material. This is leading to breakthroughs at the interface of software engineering and evolutionary computation, though there are still many exciting open problems for evolutionary commutation researchers to get their teeth into. This talk will cover some of these recent developments in Search Based Software Engineering (SBSE) and Dynamic Adaptive SBSE.

Big data driven genetic improvement for maintenance of legacy software systems

Search Based Software Engineering: Introduction to the Special Issue of the IEEE Transactions on Software Engineering

Energy-efficient permutation flow shop scheduling problem using a hybrid multi-objective backtracking search algorithm

It is very important that the sequencing and lot sizing in the flow-line environment are not independent. They should be integrated. In this paper, a multiobjective hybrid evolutionary search algorithm which combines a genetic algorithm and a simulated annealing algorithm is proposed and the performance of the proposed algorithm is compared with the existing genetic algorithm and the simulated annealing algorithm. The algorithms are coded independently and the performance is compared with randomly generated test problems. The objective functions considered for evaluation are the minimisation of makespan, minimisation of overtime and minimisation of holding cost. The scalar fitness function combining all the three objective functions, which minimises total cost, is used. The results are presented in tables and figures. The results show that the proposed hybrid algorithm performs better than the genetic algorithm and the simulated annealing algorithm.

This research investigates the multi-response of both material removal rate (MRR) and surface roughness (Ra) for the wire electrical discharge machining (WEDM) of two stainless steel alloys: AISI 304 and AISI 316. Experimental results are utilized to compare the machining responses obtained for AISI 316 with those obtained for AISI 304, as previously reported in the literature. The experimental work is conducted through a full factorial experimental design of five running parameters with different levels: applied voltage, transverse feed, pulse-on/pulse-off times and current intensity. The machined workpieces are analyzed using an image processing technique in order to evaluate the size of cut slots to allow the calculation of the MRR. Followed by the characterization of the surface roughness along the side walls of the slots. Different mathematical regression techniques were developed to represent the multi-response of both materials using the MATLAB regression toolbox. It was found that WEDM process parameters have a fuzzy influence on the responses of both material models. This allowed for multi-objective optimization of the regression models using four different techniques: multi-objective genetic algorithm (MOGA), multi-objective pareto search algorithm (MOPSA), weighted value grey wolf optimizer (WVGWO) and osprey optimization algorithm (OOA). The optimization results reveal that the optimal WEDM parameters of each response are inconsistent to the others. Hence, the optimal results are considered a compromise between the best results of different responses. Noteworthily, the multi-objective pareto search algorithm outperformed the other candidates. Eventually, the optimal results of both materials share the high voltage, high transverse feed rate and low pulse-off time parameters; however, AISI 304 requires low pulse-on time and current intensity levels while AISI 316 optimal results entail higher pulse-on time and current levels.