Prioritization of User Story Acceptance Tests in Agile Software Development Using Meta-Heuristic Techniques and Comparative Analysis

[Context] In agile software development, requirements are frequently represented in lightweight artifacts such as user stories (USs). To support their validation, USs may be complemented with additional artifacts such as acceptance criteria and test cases. [Problem] Existing studies propose frameworks for US writing, or investigate high-level factors that contribute to productivity, such as teamwork and support from management. However, empirical evidence on the impact of requirements artifacts on team efficiency is scarce. [Method] Through a mixed-methods case study at a large Dutch organization, we investigate how the use of requirements artifacts affects efficiency. We collected data from eight agile teams: 1,345 USs from nineteen sprints, US quality indicators, the quantity of associated acceptance criteria and test cases, and cycle time. We statistically analyzed the existence of correlations between these variables, and we triangulated our results through interviews with product owners and scrum masters from the involved teams. [Findings] Our employed indicators for US quality do not seem to have an effect on on-time completion, with our interviewees highlighting the trade-off between investing on writing quality and short-term efficiency. We found, instead, that the existence of acceptance criteria may be a relevant factor for on-time completion, as confirmed both by statistical tests and the interviewees.

<italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Context:</i> User stories have been widely accepted as artifacts to capture the user requirements in agile software development. They are short pieces of texts in a semi-structured format that express requirements. Natural language processing (NLP) techniques offer a potential advantage in user story applications. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Objective:</i> Conduct a systematic literature review to capture the current state-of-the-art of NLP research on user stories. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Method:</i> The search strategy is used to obtain relevant papers from SCOPUS, ScienceDirect, IEEE Xplore, ACM Digital Library, SpringerLink, and Google Scholar. Inclusion and exclusion criteria are applied to filter the search results. We also use the forward and backward snowballing techniques to obtain more comprehensive results. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Results:</i> The search results identified 718 papers published between January 2009 to December 2020. After applying the inclusion/exclusion criteria and the snowballing technique, we identified 38 primary studies that discuss NLP techniques in user stories. Most studies used NLP techniques to extract aspects of who, what, and why from user stories. The purpose of NLP studies in user stories is broad, ranging from discovering defects, generating software artifacts, identifying the key abstraction of user stories, and tracing links between model and user stories. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Conclusion:</i> NLP can help system analysts manage user stories. Implementing NLP in user stories has many opportunities and challenges. Considering the exploration of NLP techniques and rigorous evaluation methods is required to obtain quality research. As with NLP research in general, the ability to understand a sentence’s context continues to be a challenge.

User story management is a critical aspect of agile software development, as it involves understanding and prioritizing the needs of stakeholders, and translating them into actionable tasks for development teams. Furthermore, the identification of new activities within products is crucial for enhancing software quality assurance, and user story management is a crucial component of agile software development and software testing. Reusing manually written test steps would be a waste of time and effort for the testers because requirements in agile software development are continually changing and eventually becoming out-dated. Therefore, developers need to implement the necessary functions and write test steps for user stories to determine the desired behaviour or desired result of the program. In an agile world, the user story is the link between the customer and the development team, as well as the main pillar on which the development team relies to understand the product requirements. Therefore, developers need to implement the necessary functions and write test steps for user stories to determine the desired behaviour or desired result of the program. Since agile testing can detect defects early in the software life cycle and deliver a high-quality product, we propose our approach to generating test cases using natural language processing to analyse the user story. Therefore, Neural Language Processing (NLP) techniques help the development team to obtain clear data and achieve customer satisfaction.

A user story is commonly applied in requirement elicitation, particularly in agile software development. User story is typically composed in semi-formal natural language, and often follow a predefined template. The user story is used to elicit requirements from the users' perspective, emphasizing who requires the system, what they expect from it, and why it is important. This study aims to acquire a comprehensive understanding of user stories in requirement elicitation. To achieve this aim, this systematic review merged an electronic search of four databases related to computer science. 40 papers were chosen and examined. The majority of selected papers were published through conference channels which comprising 75% of total publications. This study identified 24 problems in user stories related to requirements elicitation, with ambiguity or vagueness being the most frequently occurring problem reported 18 times, followed by incompleteness reported 11 times. Finally, the model approach was the most popular approach reported in the research paper, accounting for 30% of the total approaches reported.

In Agile software development, user stories play a vital role in capturing and conveying end-user needs, prioritizing features, and facilitating communication and collaboration within development teams. However, automated methods for evaluating user stories require training in NLP tools and can be time-consuming to develop and integrate. This study explores using ChatGPT for user story quality evaluation and compares its performance with an existing benchmark. Our study shows that ChatGPT’s evaluation aligns well with human evaluation, and we propose a “best of three” strategy to improve its output stability. We also discuss the concept of trustworthiness in AI and its implications for non-experts using ChatGPT’s unprocessed outputs. Our research contributes to understanding the reliability and applicability of Generative AI in user story evaluation and offers recommendations for future research.

Context: Agile software development (ASD) uses ‘agile’ artefacts such as user stories and product backlogs as well as ‘non-agile’ artefacts, for instance designs and test plans. Rationales for incorporating especially non-agile artefacts by an agile team mainly remain unknown territory. Goal: We start off to explore influences on artefacts usage, and state our research question as: To what extent does maturity relate to the usage of artefacts in ASD in software product organizations? Method: In our multiple case study 14 software product organizations were visited where software product management maturity was rated and their artefacts usage listed. Results: We found maturity to be negatively correlated with the non-agile/all artefacts ratio. In other words, the more mature software product management is, the fewer non-agile artefacts are used in ASD. Conclusions: This suggests that an organizational factor influences an agile team in its artefacts usage, contradictory to the concept of self-organizing agile teams.

Agile development has become increasingly popular among software development teams due to its capacity to deliver and update software rapidly while accommodating evolving requirements. Within this dynamic context, access control policies are critical for ensuring the security of systems by defining who can access specific resources under given conditions. However, identifying and documenting these policies often rely on manual, time-intensive processes prone to errors and oversight. This paper proposes an innovative framework leveraging Retrieval-Augmented Generation (RAG) and Large Language Models (LLMs) to automate the extraction and organization of access control policies from user stories and software documentation. The framework focuses on the early stages of the development lifecycle, capturing access control requirements as expressed in natural language artifacts. It comprises two core components: (1) a pipeline for extracting and categorizing access control policies, enabling precise mappings between roles, actions, and resources, and (2) an interactive chatbot designed to support Security Operations Center (SOC) analysts in evaluating suspicious access requests by providing contextualized insights into access policies. By integrating advanced natural language processing techniques with retrieval-based augmentation, the framework aims to reinforce access control mechanisms by improving visibility, and providing contextualized insights for security analysts.

&lt;p&gt;eXtreme programming (XP) is one of a new breed of methods, collectively known as the agile methods, that are challenging conventional wisdom regarding systems development processes and practices. Practitioners specifically designed the agile methods to meet the business problems and challenges we face building software today. As such, these methods are receiving significant attention in practitioner literature. In order to operate effectively in the world of vague and changing requirements, XP moves the emphasis away from document-centric processes into practices that enable people. The Customer is the primary organisational facing role in eXtreme Programming (XP). The Customer's explicit responsibilities are to drive the project, providing project requirements (user stories) and quality control (acceptance testing). Unfortunately the customer must also shoulder a number of implicit responsibilities including liaison with external project stakeholders, especially project funders, clients, and end users, while maintaining the trust of both the development team and the wider business. This thesis presents a grounded theory of XP software development requirements elicitation, communication, and acceptance, which was guided by three major research questions. What is the experience of being an XP Customer? We found that teams agree that the on-site customer practice is a drastic improvement to the traditional document-centric approaches. Our results indicate, however, that the customers are consistently under pressure and commit long hours to the project in order to fulfil the customer role. So while this approach to requirements is achieving excellent results, it also appears to be unsustainable and thus constitutes a great risk to XP projects. Who is the XP Customer? The initial definition of XP resulted in many people interpreting the onsite customer to be a single person. This research has highlighted that a customer team always exists, and goes further to outline the ten different roles that were covered on the team, which range from the recognised "Acceptance Tester" role to the less recognised roles of "Political Advisor" and "Super-Secretary". What are the practices that support an XP Customer to perform their role effectively on a software development project? An additional eight customer-focused practices have been uncovered to supplement the existing XP practices. These customer-focused practices together enable customers to sustainably drive XP projects to successful completion. The practices range from those that specifically focus on interaction (both with the programmer team and the larger organisation) e.g. "Programmer On-site" and "Roadshows" to those that specifically look to the well-being and effectiveness of the customer (e.g. "Pair Customering") to those that highlight the key steps or activities that need to occur along the way (e.g. "Big Picture Up-Front" and "Recalibration").&lt;/p&gt;

In agile software development, product backlog items (PBI) are used to capture the user requirements prior to the product implementation. Many types of requirements can be observed within a software project. Proper classification of PBI can positively impact the software development process. PBI can be classified into three categories: user stories, foundational stories, and spikes. After the extreme literature survey, no research was held on classifying the PBI into the categories mentioned above. This paper proposed a machine learning (ML) based approach to classify the PBI into three categories. 4,721 PBI were collected from different software projects and manually labelled into the three classes mentioned above. Then the PBI were cleaned using different pre-processing techniques. Classification models were constructed using ML techniques. The performance of each ML model was evaluated using accuracy, precision, recall, and F1 score. Support vector machine (SVM) outperformed other ML models by providing 88% accuracy.

&lt;p&gt;Agile software development offers a deceptively simple means to organise complex multi-participant software development while achieving fast delivery of quality software, meeting customer requirements, and coping effectively with project change. There is little understanding, however, of how agile software development projects achieve effective coordination, a critical factor in successful software projects. Agile software development provides a unique set of practices for organising the work of software projects, and these practices seem to achieve effective project coordination. Therefore, this thesis takes a coordination perspective to explore how agile software projects work, and why they are effective. The outcome of this research is a theory of coordination in co-located agile software development projects. To build a coordination theory, evidence was drawn from a multi-case study following the positivist tradition in information systems. Three cases of agile software development contributed to the theory, along with one additional non-agile project that contributed contrasting evidence. The findings show that agile software development practices form a coordination strategy addressing three broad categories of dependency: knowledge dependencies, task dependencies, and resource dependencies. Most coordination is for managing requirement, expertise, historical, and task allocation dependencies; all forms of knowledge dependency. Also present are task dependencies, which include activity or business process dependencies, and resource dependencies, which include technical or entity dependencies. The theory of coordination explains that an agile coordination strategy consists of coordination mechanisms for synchronising the project team, for structuring their relations, and for boundary spanning. A coordination strategy contributes to coordination effectiveness, which has explicit and implicit components. The primary contribution of this theory is an explanation of how agile software development practices act together to achieve effective project coordination. The coordination strategy concept can be used to select practices from agile methods to ensure software projects achieve effective coordination. In addition, once operationalised in future work, the well-grounded theoretical concepts developed in this research will provide valuable tools for measuring the coordination effectiveness of agile method adoption and adaptation.&lt;/p&gt;

&lt;p&gt;Agile software development offers a deceptively simple means to organise complex multi-participant software development while achieving fast delivery of quality software, meeting customer requirements, and coping effectively with project change. There is little understanding, however, of how agile software development projects achieve effective coordination, a critical factor in successful software projects. Agile software development provides a unique set of practices for organising the work of software projects, and these practices seem to achieve effective project coordination. Therefore, this thesis takes a coordination perspective to explore how agile software projects work, and why they are effective. The outcome of this research is a theory of coordination in co-located agile software development projects. To build a coordination theory, evidence was drawn from a multi-case study following the positivist tradition in information systems. Three cases of agile software development contributed to the theory, along with one additional non-agile project that contributed contrasting evidence. The findings show that agile software development practices form a coordination strategy addressing three broad categories of dependency: knowledge dependencies, task dependencies, and resource dependencies. Most coordination is for managing requirement, expertise, historical, and task allocation dependencies; all forms of knowledge dependency. Also present are task dependencies, which include activity or business process dependencies, and resource dependencies, which include technical or entity dependencies. The theory of coordination explains that an agile coordination strategy consists of coordination mechanisms for synchronising the project team, for structuring their relations, and for boundary spanning. A coordination strategy contributes to coordination effectiveness, which has explicit and implicit components. The primary contribution of this theory is an explanation of how agile software development practices act together to achieve effective project coordination. The coordination strategy concept can be used to select practices from agile methods to ensure software projects achieve effective coordination. In addition, once operationalised in future work, the well-grounded theoretical concepts developed in this research will provide valuable tools for measuring the coordination effectiveness of agile method adoption and adaptation.&lt;/p&gt;

Agile development practices have been widely adopted in a variety of organizations, yet only a few tools are available to help make the practical process of managing agile teams less painful and more successful. HP Agile Manager is a purpose-built SaaS-based Agile planning tool. HP Agile Manager provides a simpler, smarter way to manage collaborative development. Liran Tal provides a practical, concise approach to using Agile Manager in a variety of settings to better plan, conduct, and manage software releases within development teams. His step-by-step approach will show you how to plan your products features, streamline the agile sprint process, work with user stories, and track defects throughout the development process. Agile Manager can work for small startups, mid-sized teams, as well as scale up for bigger organizations as a cost-effective and flexible tool to apply agile techniques to improve your software development process. What youll learnHow to apply the concepts of agile management in software development teams to better execute and increase performance and delivery How a software release cycle works, how to manage sprints, set timelines, build a products backlog, and manage day to day team tasks How to leverage agile best practices to guide your team through effective release planning and team collaboration Understand and apply agile software delivery concepts such, estimating and prioritizing user stories. Streamlining sprints planning, execution, and retrospective How to setup, and use HP Agile Manager to effectively manage software lifecycle through agile methodology, and learn how to leverage Agile Manager's flexible workflow to meet your team requirements and processes Using Agile Manager's dashboards for real-time view into the current status of releases, and sprints to gain insights and pro-actively respond to changes, as well as tracking your team progress, work-load, and keeping an eye on tasks burn-down, or cumulative flow Who this book is for Developers, quality engineers, development team leaders and new managers will find this book to serve them well in learning and applying agile methodologies in their teams.

Writing a user story is essential to communicate with end users and developers. It is popular practices found in most agile methodologies. Therefore, to reuse the user stories from the previous successful projects becomes necessary. In this paper, we propose the business process ontology for defining user story. The ontology is a knowledge base designed to collect the user stories in term of N-triple. The ontology schema is designed using classes and hierarchy relation along with the properties according to Role-Action-Object relations. We introduce the synonym property to simplify and reduce the number of the nodes in the ontology as well. Finally, we demonstrate how to initially populate and build the ontology from the historical user stories.

Stakeholder involvement is vital in the requirements elicitation process. However, a lack of stakeholder involvement in the project often occurs. Various data sources, such as business processes and software documentation, will help determine what stakeholders need. The limitation of these documents is that they are technical and do not contain numerous stakeholder's needs. This paper aims to extract information related to software requirements from online news using domain specificity. The online news comprises elements that compose the user story, namely: the aspect of who (stakeholder), the aspect of what (functional), and the aspect of why (reason). User stories are standard practices used to capture and write functional software requirements in agile software development. This paper proposes to use domain specificity to extract aspects of what from online news to increase understanding of domain knowledge, especially software functionality. We use SRS documents and online news as data sources. Based on these datasets, domain specificity is calculated to produce software specific vocabulary. POS tags are being used to extract software requirements-related information from online news. When examining two datasets, our approach improved precision and recall with average values of 0.56 and 0.579.

A Lagrangian heuristic for sprint planning in agile software development