Unveiling the microservices testing methods, challenges, solutions, and solutions gaps: A systematic mapping study

This doctoral research proposal is designed to address the challenges on employing microservices architecture (MSA) in DevOps. These two practices are relatively recent trends on the crowded street of software engineering paradigms. We believe that the coexistence of these practices may bring new challenges for architects about making and implementing decisions of microservices in DevOps. We plan to gain a deep understanding of the challenges/solutions, architecture description, migration, refactoring, and evolution in MSA with DevOps. Our expected outcome would be: (i) a systematic mapping study on MSA in DevOps, (ii) an evidence based framework for synthesizing MSA challenges/solutions in DevOps, (iii) recommendations for documenting MSA in DevOps based on industrial practices, (iv) a migration process from monolithic applications/architecture to MSA in DevOps, and (v) empirical reports on MSA refactoring and evolution in DevOps.

SummaryContext: Software performance is crucial for ensuring the quality of software products. As one of the non‐functional requirements, the few efforts devoted to software performance have often been neglected until a later phase in the software development life cycle (SDLC). The lack of clarity of what software performance research literature is available prevents researchers from understanding what software performance research fields are available. It also creates difficulty for practitioners to adopt state‐of‐the‐art software performance techniques. Software performance research is not as organized as other established research topics such as software testing. Thus, it is essential to conduct a systematic mapping study as a first step to provide an overview of the latest research literature available in software performance. Objective: The objective of this systematic mapping study is to survey and map software performance research literature into suitable categories and to synthesize the literature data for future access and reference. Method: This systematic mapping study conducts a manual examination by querying research literature in noble journals and proceedings in software engineering in the past decade. We examine each paper manually and identify primary studies for further analysis and synthesis according to the pre‐defined inclusion criteria. Lastly, we map the primary studies based on their corresponding classification category. Results: This systematic mapping study provides a state‐of‐the‐art literature mapping in software performance research. We have carefully examined 222 primary studies out of 2000+ research literature. We have identified six software performance research categories and 15 subcategories. We generate the primary study mapping and report five research findings. Conclusions: Unlike established research fields, it is unclear what types of software performance research categories are available to the community. This work takes the systematic mapping study approach to survey and map the latest software performance research literature. The study results provide an overview of the paper distribution and a reference for researchers to navigate research literature on software performance.

The Microservice Architecture Style (MSA) has emerged as a significant computing paradigm in software engineering, with companies increasingly restructuring their monolithic systems to enhance digital performance and competitiveness. However, the migration process, particularly the microservice identification phase, presents complex challenges that require careful consideration. This study aimed to provide developers and researchers with a practical roadmap for microservice identification during legacy system migration while highlighting crucial migration steps and research requirements. Through a systematic mapping study following Kitchenham and Petersen’s guidelines, we analyzed various microservice identification approaches and developed a middleweight ontology that can be queried for key inputs, data modeling, identification algorithms, and performance evaluation metrics. Our research makes several significant contributions: a comprehensive analysis of existing identification methodologies, a multi-dimensional framework for categorizing and evaluating approaches, an examination of current research trajectories and literature gaps, an ontological framework specifically designed for microservice identification, and an outline of pressing challenges and future research directions. The study concluded that microservice identification remains a significant barrier in system migration efforts, highlighting the need for more research focused on developing effective identification techniques that consider various aspects, including roles and dependencies within a microservice architecture. This comprehensive analysis provides valuable insights for professionals and researchers working on microservice migration projects.

The microservices architecture is a set of small, autonomous services that can work together to form a single application that has seen widespread adoption by practitioners. However, deploying this architecture creates several technical challenges. This study examines the state of the art of microservices deployment in the literature, informing researchers and practitioners about the techniques and technologies used in the deployment of microservices and, practices used in the DevOps culture. We conducted a systematic mapping study driven by four research questions related to the deployment of microservices and DevOps practices, and 21 studies were identified from which information was synthesized using the meta-aggregation method. With the information synthesis, 43 findings were extracted and classified into seven categories. We can summarize the findings of this study as follows: (i) We identified the essential DevOps practices in the deployment of microservices. (ii) We defined the stack of technologies with the highest incidence in the studies. (iii) We found three groups of challenges in microservices deployment. (iv) Finally, we present a set of recommendations for microservices deployment.

Quality model is regarded as a well-accepted approach for assessing, managing and improving software product quality. There are three categories of quality models for software products, i.e., definition model, assessment model, and prediction model. Quality assessment model (QAM) is a metric-based approach to assess the software quality. It is typically regarded as of high importance for its clear method on how to assess a system. However, the current state-of-the-art in QAM research is under limited investigation. To address this gap, the paper provides an organized and synthesized summary of the current QAMs. In detail, we conduct a systematic mapping study (SMS) for structuring the relevant articles. We obtain a total of 716 papers from the five databases, and 31 papers are selected as relevant studies at last. In summary, our work focuses on QAMs from the following aspects: software metrics, quality factors, aggregation methods, evaluation methods and tool support. According to the analysis results, our work discovers five needs that researchers in this area should continue to address: (1) new method and criteria to tailor a quality framework (i.e., structure of software metrics and quality factors) according to different specifics, (2) systematic investigations on the effectiveness, strength and weakness of different aggregation methods to guide the method selection in different context, (3) more investigations on evaluating QAMs in the context of industrial cases, (4) further investigations or real-world case studies on the QAMs related tools, and (5) building a public and diverse software benchmark which can be adopted in different application context.

A key part of software evolution and maintenance is the continuous integration from collaborative efforts, often resulting in complex traceability challenges between software artifacts: features and modules remain scattered in the source code, and traceability links become harder to recover. In this paper, we perform a systematic mapping study dealing with recent research recovering these links through information retrieval, with a particular focus on natural language processing (NLP).Our search strategy gathered a total of 96 papers in focus of our study, covering a period from 2013 to 2021. We conducted trend analysis on NLP techniques and tools involved, and traceability efforts (applying NLP) across the software development life cycle (SDLC). Based on our study, we have identified the following key issues, barriers, and setbacks: syntax convention, configuration, translation, explainability, properties representation, tacit knowledge dependency, scalability, and data availability.Based on these, we consolidated the following open challenges: representation similarity across artifacts, the effectiveness of NLP for traceability, and achieving scalable, adaptive, and explainable models. To address these challenges, we recommend a holistic framework for NLP solutions to achieve effective traceability and efforts in achieving interoperability and explainability in NLP models for traceability.

In todays world, software security has become essential for protecting the overall organizations operations. Mostly software development companies are adopting various strategies to build secure software to cope with the challenges of the client organization. Information become threatened due to connection with the cyber world, and it requires better security mechanisms. Software development organizations are receiving pressure from their clients to focus on the enhancement of security during the whole software development life cycle (SDLC). This protocol aims to review the literature in a systematic way to identify the state-of-the-art of software security to be considered by Global Software Development (GSD) vendor organizations during the development of a secure software as it evolves from requirements engineering to its final disposal. In order to improve security processes in the context of software development, our current research developed a Systematic Mapping Study (SMS) protocol. Presently, we are in the implementation phase of the protocol for the development of secure software. The expected outcomes of this SMS will be a list of security measurements and their solutions to be incorporated by GSD vendor organizations in each phase of the SDLC. This will also give a direction for new research in this area.

Microservice architecture (MSA) is a type of software and systems architecture that is based on the modularization principle. It proposes designing systems employing small-scaled, loosely coupled, and independently deployable microservices. There are several benefits of microservices architecture in terms of maintainability, scalability, and productivity which have led to rise in its popularity. Even though there are several studies about development in MSA, the studies on the quality of the microservice-based systems are limited. In this study, we propose a quality model based on fuzzy logic to measure and assess quality attributes of systems in MSA that can be used by software architects, developers, and project managers. We focus on maintainability of microservices because it is one of the most important quality attributes of software systems. We identified sub-characteristics and properties of microservices that affect maintainability, and constructed a hierarchical quality model based on ISO/IEC 250xy standard SQuaRE (System and Software Quality Requirements and Evaluation). Our fuzzy model measures maintainability of microservices in three levels, i.e., low, medium, and high. We provided a basis for the development and application of quality models in industrial practice as well as a basis for further extension. To demonstrate and evaluate our methodology, we used open-source applications designed in MSA. The results show that our method can assess maintainability of microservices realistically.

Despite the importance of software qualities, they are not well understood, especially in the context of the interrelationships between qualities. A number of systematic mapping studies have been conducted prior to 2015 to summarize the literature on the topic and to identify research gaps. To provide a better understanding of the current state of the art, we conducted a systematic mapping study on relevant studies from 2015 to 2019 through a database search and a subsequent snowballing approach. In total, 18 studies were selected as the study subjects wherein we evaluated the types of software quality interrelationships and the qualities that comprise them. Based on our findings, we report on the progress made to address previously identified research gaps.

Early software defect prediction: A systematic map and review

Background The relatively low universal health coverage (UHC) effective coverage indices in most African countries is a cause for concern. Some African countries have effective coverage indices as low as 25 when the global average is 44-48. Although several African countries have initiated UHC activities, their implementation paces and approaches vary. The aim of this systematic mapping study is to describe the publication frequencies on UHC-related activities across African countries. The revealed publication patterns can guide the enhancement of UHC research, implementation, and publication in Africa. Methods This systematic mapping study was conducted by first establishing a research scope. Then, relevant publications were identified from PubMed, Google Scholar, Google, WHO, and uhc2030 search engines. A classification scheme was developed to guide the systematic mapping of the selected 202 publications. This systematic mapping study describes the variations in UHC-related publication types, contributions, focus areas, and dimensions between 2013 and 2023 across African countries. Results Analyzing the publications based on the developed classification scheme revealed that WHO was the most frequent author with 17 publications. Secondly, primary research articles (28.2%, n = 57) were the commonest while legislation (0.5%, n = 1) were the rarest. Most of the publications were on the financing and performance of health systems. Conclusions Overall, publication numbers and evidence value increased as time progressed from 2013 to 2023. The generated systematic maps indicate opportunities for UHC-promoting publication activities. Supporting the development of legislation and authoring systematic reviews are some of the opportunities that stakeholders can leverage to steer UHC in Africa.

This study focuses on the application of neural networks for anomaly detection in microservice architectures. The increasing complexity of modern microservice systems makes traditional anomaly detection methods less effective, as they struggle to adapt to dynamic environments and large volumes of data. Neural networks offer a promising solution due to their ability to model complex patterns and detect subtle deviations that may indicate system failures or security threats. The aim of this work is to compare the effectiveness of different neural network-based approaches to anomaly detection, evaluate their strengths and weaknesses, and determine the most suitable method for practical implementation. To achieve this, several key tasks were outlined: an in-depth review of contemporary anomaly detection techniques applied to microservices; an analysis of neural network-based models, with a particular focus on autoencoders and recurrent neural networks (RNNs); a comparative assessment of these methods in terms of accuracy, computational efficiency, and robustness to noise; an exploration of hybrid models that combine different techniques to improve detection performance; and the formulation of recommendations for selecting the optimal approach based on specific application scenarios. This research examines the use of deep neural networks for detecting behavioral anomalies in microservice environments. Various architectures are tested and compared to identify trade-offs between detection accuracy, processing speed, and resistance to false positives. The findings indicate that deep neural networks, particularly autoencoders and recurrent models, achieve high precision in identifying anomalies, but their efficiency varies depending on the architecture and the nature of the dataset. The study highlights that while autoencoders effectively capture deviations in data distribution, recurrent networks excel in detecting temporal anomalies in sequential microservice interactions. The conclusions confirm that neural networks provide a powerful tool for anomaly detection in microservice monitoring, significantly outperforming conventional statistical approaches. However, challenges remain, including the need for adaptive models capable of learning in real time and optimizing computational resources. Further research should focus on hybrid approaches that integrate multiple neural architectures to enhance detection accuracy and scalability. The development of self-learning and auto-tuning models could also improve adaptability to evolving microservice behaviors, ensuring robust and efficient anomaly detection in complex distributed systems.

: The Rome Air Development Center has developed the Software Quality Framework as a means to specify software quality goals and measure software quality. Much of the work to date has focused on metrics applicable to software code. This report describes an effort undertaken to measure the quality of software products earlier in the software development life cycle, during the design phase, and to automate the capture of metric data from design media. Metrics of software quality, primarily those related to the criterion simplicity (or conversely, complexity), were reviewed. This review includes those metrics previously developed in the Software Quality Framework. Two metrics, Halstead's Software Science and McCabe's Cyclomatic Complexity were chosen for their amenability to measurement during design and their potential for automation. Two design media were used: Design Aids for Real-Time Systems, an experimental automated design tool developed at the Charles Stark Draper Laboratory; and Ada as a program design language.

This paper presents an overview of quantitative analysis techniques for software quality and their applicability during the software development life cycle (SDLC). This includes the Seven Basic Tools of Quality, Statistical Process Control, and Six Sigma, and it highlights how these techniques can be used for managing and controlling the quality of software during specification, design, implementation, testing, and maintenance. We verify whether or not these techniques, which are generally accepted for most projects, most of the time, and have value that is recognized by the peer community, have indeed been included in the SWEBOK Guide.

Domain-Specific Languages: A Systematic Mapping Study