A case study investigating the characteristics of verification and validation activities in the software development process

<div class="WordSection1"><p>I am honored for being an alumnus of KEMC, and am delighted to write this editorial for the Annals. I have, ever since done my research and taught qualitative methods, felt that clinical methods are quite akin to qualitative research. Then, why only few clinical researchers use qualitative methods? A witness to this assumption is the research published in the Annals. In this treatise, I try to identify some parallels between the two types of methods and argue for the clinicians to invest in learning qualitative research methods to practice clinical methods well.</p><p> Qualitative research is to identify and interpret issues from the perspective of participants, their experience of: illness or disability, using health service, and to appreciate the meanings they give to the behaviour, events or objects in the context of their social and cultural norms. In this type of research, the emphasis is on exploring the associations and understanding the phenomenon in its holism; and not, like in quantitative research, from an outsider’s perspective and for certain specific aspects.<sup>1</sup> It requires participants with specific characteristics, selected purposely that can best inform the research topic. More participants, identified inductively during data collection, are added to develop full and multiple perspectives about the cases.<sup>2</sup></p><p> No preset data collection tool is used, instead qualitative researcher guided by a research question acts as an instrument, since the line of enquiry he changes during data collection as new understanding is gained and/or the situation changes.<sup>3</sup> The data for research is derived from the observation, interviews or verbal interactions, focus group discussions, document reviews, life histories etc. and the researcher asks why, how and under what circumstances things occur; and not just what, where and when. It is recorded in words or pictures and log book is used to record notes arising from interviews, observations, extracts from documents etc.</p><p> In health care settings patients are the subjects for clinical methods. The clinician, even prior to any verbal communication, observes the patient, e.g. for his gait and appearance. If in a bed or examination couch, his posture could give some clue to the illness. Inspection, a clinical method, is like systematic observation, which is qualitative method, should be holistic. In my third year during bedside teaching, Professor (late) Rashid Ahmed Qureshi said, <em>“patient has come to you as a whole and not his stomach in a tray”</em>, when a student straight went to examine abdomen of a patient with acute abdomen. History taking, another clinical method, is like conducting semi structured in depth interview – a qualitative research method. In both disciplines, we are told, <em>“not to ask leading, but follow up and probing questions”; </em>and Professor (late) Alamgir Khan, while teaching clinical methods, would add, <em>“if a good history is taken, you will establish diagnosis in over 65% of cases”. </em>Likewise, as part of history taking, documents related to patient’s illness history and treatment are reviewed similar to document review in qualitative research.</p><p> The two approaches however differ in how the data is analysed. In clinical practice, diagnosis is established based on the pathophysiological knowledge or patient’s clinical condition is discussed in clinicopathological conference.<sup>4</sup> On the other hand, qualitative research employs meaning based data analysis, whereby the qualitative data is transformed into some form of explanation, understanding or interpretation of people and situation that is investigated.<sup>5,6</sup></p><p> In conclusion, the history, the observation and the review of document related to patient are since obtainned using qualitative methods, the clinician trained in these methods could not only conduct these methods well but also interpret the data to identify and detect obstacles to the change in clinical condition and the reasons why improvement does or does not occur.<sup>7</sup> Finally, while it is heartening that research forums are organised in the institutions affiliated with KEMU, in order the research is richer, the researchers’ skills in qualitative research methods should be built.</p></div>

A Three-Layered Approach to Radiation Therapy Software Recall Analysis

The software development process (SDP) contains many variables depending on the methodology chosen. Software development methodologies such as Agile or Waterfall can specify how and in what sequence software is developed. However, the SDP remains a hot topic among practitioners and academics, who seek to clarify the key aspects that influence the successful implementation of the SDP. As the SDP is a dynamic and knowledge-intensive business process (BP), one of its most impact components is the human factor (HF). It is difficult to predict how the HF impacts the sequence and results of SDP implementation. This paper proposes a new approach to investigate the impact of the HF on the SDP through the perspectives of different software development methodologies. The advantages and novelties of this approach include modelling HF uncertainties through their fuzzification in SDP activities, modelling different SDP methodologies using a case-handling approach, and simulating a dynamic, case-based SDP with real HF-related data collected from several IT organizations. The results show that the impact of the HF on SDP performance differs across the Waterfall and Agile methodologies. The results also allow researchers and practitioners working on software development projects to familiarize themselves with the impact of the HF on different SDPs, and to assess the degree of development risk associated with the SDP depending on the methodology chosen.

Many papers proposed in the software engineering and information systems literature are dedicated to analysis of software projects missing their schedules, exceeding their budgets, delivering software products with poor quality and in some cases even wrong functionality. The expression “software crisis“ has been coined since the late 60's to illustrate this phenomenon. Various solutions has been proposed by academics and practitioners in order to deal with the software crisis, counter these trends and improve productivity and software quality. Such solutions recommend software process improvement as the best way to build software products needed by modern organizations. Among the well-known solutions, many are based either on software development tools or on software development approaches, methods, processes, and notations. Nevertheless, the scope of these solutions seems to be limited and the improvements they provide are often not significant. We think that since software artifacts are accumulation of knowledge owned by organizational stakeholders, the software crisis is due to a knowledge gap resulting from the discrepancy between the knowledge integrated in software systems and the knowledge owned by organizational actors. In particular, integrating knowledge management in software development process permits reducing the knowledge gap through building software products which reflect at least partly the organization's know-how. In this paper, we propose a framework which provides a definition of knowledge based on information systems architecture and describes a knowledge-oriented software development process to help organizations in reducing the software crisis impacts.

In the current era of artificial intelligence, large language models such as ChatGPT and BARD are being increasingly used for various applications, such as language translation, text generation, and human-like conversation. The fact that these models consist of large amounts of data, including many different opinions and perspectives, could introduce the possibility of a new qualitative research approach: Due to the probabilistic character of their answers, “interviewing” these large language models could give insights into public opinions in a way that otherwise only interviews with large groups of subjects could deliver. However, it is not yet clear if qualitative content analysis research methods can be applied to interviews with these models. Evaluating the applicability of qualitative research methods to interviews with large language models could foster our understanding of their abilities and limitations. In this paper, we examine the applicability of qualitative content analysis research methods to interviews with ChatGPT in English, ChatGPT in German, and BARD in English on the relevance of computer science in K-12 education, which was used as an exemplary topic. We found that the answers produced by these models strongly depended on the provided context, and the same model could produce heavily differing results for the same questions. From these results and the insights throughout the process, we formulated guidelines for conducting and analyzing interviews with large language models. Our findings suggest that qualitative content analysis research methods can indeed be applied to interviews with large language models, but with careful consideration of contextual factors that may affect the responses produced by these models. The guidelines we provide can aid researchers and practitioners in conducting more nuanced and insightful interviews with large language models. From an overall view of our results, we generally do not recommend using interviews with large language models for research purposes, due to their highly unpredictable results. However, we suggest using these models as exploration tools for gaining different perspectives on research topics and for testing interview guidelines before conducting real-world interviews.

Background. The use of qualitative research (QR) methods is recommended as good practice in discrete choice experiments (DCEs). This study investigated the use and reporting of QR to inform the design and/or interpretation of healthcare-related DCEs and explored the perceived usefulness of such methods. Methods. DCEs were identified from a systematic search of the MEDLINE database. Studies were classified by the quantity of QR reported (none, basic, or extensive). Authors (n = 91) of papers reporting the use of QR were invited to complete an online survey eliciting their views about using the methods. Results. A total of 254 healthcare DCEs were included in the review; of these, 111 (44%) did not report using any qualitative methods; 114 (45%) reported “basic” information; and 29 (11%) reported or cited “extensive” use of qualitative methods. Studies reporting the use of qualitative methods used them to select attributes and/or levels (n = 95; 66%) and/or pilot the DCE survey (n = 26; 18%). Popular qualitative methods included focus groups (n = 63; 44%) and interviews (n = 109; 76%). Forty-four studies (31%) reported the analytical approach, with content (n = 10; 7%) and framework analysis (n = 5; 4%) most commonly reported. The survey identified that all responding authors (n = 50; 100%) found that qualitative methods added value to their DCE study, but many (n = 22; 44%) reported that journals were uninterested in the reporting of QR results. Conclusions. Despite recommendations that QR methods be used alongside DCEs, the use of QR methods is not consistently reported. The lack of reporting risks the inference that QR methods are of little use in DCE research, contradicting practitioners’ assessments. Explicit guidelines would enable more clarity and consistency in reporting, and journals should facilitate such reporting via online supplementary materials.

The purpose of this focused issue is to advance understanding of qualitative research methods in the international business context. Debates on qualitative research methods in the key international business (IB) journals have been rare. There has been no focused issue on this topic previously, either in mir or in other journals in the field, making this a 'special' issue. During the process of editing a recent handbook on qualitative research methods (Marschan-Piekkari/Welch 2004), we realized that while this collection covered many topics, often for the first time in the IB field, there were many others still outstanding. There was thus scope for continuing a dialogue about qualitative methods in the IB research community. The challenge of making qualitative research count in the quantitative world of IB remains. Qualitative research is difficult to define, since the term encompasses many different research traditions, research strategies and methods for data collection and analysis (Prasad 2005). A typical definition is that everything non-numerical is qualitative research (Marschan-Piekkari/Welch 2004, p. 19). However, even this very simplistic definition soon breaks down, since research strategies which are typically considered to be qualitative, such as case studies, can combine numerical with non-numerical data (Hurmerinta-Peltomaki/Nummela 2004). There is also no clear divide between qualitative and quantitative research in terms of research traditions, with much qualitative research in management and IB sharing the positivist assumptions of quantitative research (Prasad/Prasad 2002). This focused issue is not about reporting the empirical findings of IB studies applying qualitative methods; rather it is about the process of undertaking qualitative research in the IB field. Of course, there is a large volume of literature on qualitative research methods already. However, we would argue that the IB context warrants special treatment. As the authors of this focused issue discuss, the distinctiveness of the field stems from its multiple linguistic and cultural settings; organizational complexity; and its intellectual roots in post-War academic institutions in the USA. These issues receive little treatment in general texts on qualitative methodology. There has been some methodological debate in previous issues of mir. Earlier articles in mircan be grouped into three categories: first, reviews of existing methodological practices in IB publications (e.g., Cavusgil/Das 1997, Nasif et al. 1991); second, proposals for new quantitative techniques (e.g., the use of key informants in cross-cultural studies, as proposed by Lenartowicz and Roth 2004 and a linguistic-based measure of cultural distance as developed by West and Graham 2004); and third, calls for new methodological approaches (Boddewyn/Iyer 1999). The absence of qualitative research methods in this debate perhaps reflects the limited amount of empirical qualitative research published in this journal. A recent analysis of mir issues between 1990 and 1999 revealed that only 5 percent of articles published in this period used qualitative methods (Welch/Welch 2004). This focused issue is therefore a response to those, such as Boddewyn and Iyer (1999) in this journal, who have called for alternatives to surveys and secondary data analysis. This focused issue attracted 39 submissions from more than 15 countries. Given that there have been limited outlets for methodological articles in IB, this is a considerable number. It perhaps suggests a level of interest to which IB journals have not responded to date. Of the total number of submissions, 11 were selected for review. On the basis of reviewers' comments, six of these papers were ultimately rejected. The final five papers underwent two and, in some cases, up to four rounds of revisions. In selecting the articles, we used the following criteria: appropriateness of the topic for the focused issue; demonstrated knowledge of IB and methodological literature; quality of argument; originality and innovativeness of the contribution to IB methodology; organization and clarity of the paper; and potential application to research practice in IB. …

In particular, an early integration of performance specifications in the SDP has been recognized during the last few years as an effective approach to improve the overall quality of a software. Performance related problems are becoming more and more strategic in the software development, especially recently with the advent of Web Services and related business-oriented composition techniques (software as a service, Web 2.0, orchestration, choreography, etc.). The goal of our work is the definition of a software development process that integrates performance evaluation and prediction. The software performance engineering development process (SPEDP) we specify is focused on performance, which plays a key role driving the software development process, thus implementing a performance/QoS-driven (software) development process. More specifically, in this paper our aim is to formally define the SPEDP design process, posing particular interest on the basis, on the first step of SPEDP, the software/system architecture design, modeling and/or representation. We define both the diagrams to use and show how to model the structure of the software architecture, its behavior and performance requirements. This is the first mandatory step for the automation of the SPEDP into a specific tool.

Performance related problems are becoming more and more strategic in the software development, especially recently with the advent of Web Services and related business-oriented composition techniques (software as a service, Web 2.0, orchestration, choreography, etc.). In particular, an early integration of performance specifications in the SDP has been recognized during the last few years as an effective approach to improve the overall quality of a software. The goal of our work is the definition of a software development process that integrates performance evaluation and prediction. The software performance engineering development process (SPEDP) we specify is focused on performance, which plays a key role driving the software development process, thus implementing a performance/QoS-driven (software) development process. More specifically, in this paper our aim is to formally define the SPEDP design process, posing particular interest on the basis, on the first step of SPEDP, the software/system architecture design, modeling and/or representation. We define both the diagrams to use and show how to model the structure of the software architecture, its behavior and performance requirements. This is the first mandatory step for the automation of the SPEDP into a specific tool, which we have partially implemented as a performance plug-in for ArgoUML, ArgoPerformance.

This scientific article presents the results of a study focused on the current practices and future prospects of AI-tools usage, specifically large language models (LLMs), in software development (SD) processes within European IT companies. The Pan-European study covers 35 SD teams from all regions of Europe and consists of three sections: the first section explores the current adoption of AI-tools in software production, the second section addresses common challenges in LLMs implementation, and the third section provides a forecast of the tech future in AI-tools development for SD. The study reveals that AI-tools, particularly LLMs, have gained popularity and approbation in European IT companies for tasks related to software design and construction, coding, and software documentation. However, their usage for business and system analysis remains limited. Nevertheless, challenges such as resource constraints and organizational resistance are evident. The article also highlights the potential of AI-tools in the software development process, such as automating routine operations, speeding up work processes, and enhancing software product excellence. Moreover, the research examines the transformation of IT paradigms driven by AI-tools, leading to changes in the skill sets of software developers. Although the impact of LLMs on the software development industry is perceived as modest, experts anticipate significant changes in the next 10 years, including AI-tools integration into advanced IDEs, software project management systems, and product management tools. Ethical concerns about data ownership, information security and legal aspects of AI-tools usage are also discussed, with experts emphasizing the need for legal formalization and regulation in the AI domain. Overall, the study highlights the growing importance and potential of AI-tools in software development, as well as the need for careful consideration of challenges and ethical implications to fully leverage their benefits.

Verification activities have a very important role in software development process, especially in safety critical software such as airborne systems. Test processes have a continuity within the software development cycle. For this reason, tests are automated as much as possible to save time and resources. Automation of visual verification is of paramount importance as it covers a significant part of software tests. The aim of this study is to design an automated visual verification system for cockpit display software tests. To achieve this goal, we develop methods for different verification types such as text, color, font size, and object verification. Experiments are performed on a cockpit display simulation software. Detailed information about the experimental setup is given and the results are discussed.

This IIA assessed the flow down of institutional 830 Software Quality Assurance requirements through three required document templates to the CASTLE-PX software effort and the implementation of those SQA requirements. The templates flow down the DOE O 414.1D consensus standard requirements for Safety Software. This assessment did not include the flow down of NAP-24, Weapon Quality Policy, requirements. The assessment focused on the CASTLE-PX project’s software development and release processes. It did not assess Pantex’s acceptance or usage of the software. The assessment resulted in 3 Deficiencies, 5 Observations, 1 Recommendation, and 3 Strengths. Overall the CASTLE-PX team demonstrated it values quality and has worked to integrate quality practices into its software development processes. Improvement in documentation will enhance their SQA implementation.

Working in iterations and repeatedly improving team workflows based on collected feedback is fundamental to agile software development processes. Scrum, the most popular agile method, provides dedicated retrospective meetings to reflect on the last development iteration and to decide on process improvement actions. However, agile methods do not prescribe how these improvement actions should be identified, managed or tracked in detail. The approaches to detect and remove problems in software development processes are therefore often based on intuition and prior experiences and perceptions of team members. Previous research in this area has focused on approaches to elicit a team's improvement opportunities as well as measurements regarding the work performed in an iteration, e.g. Scrum burn-down charts. Little research deals with the quality and nature of identified problems or how progress towards removing issues is measured. In this research, we investigate how agile development teams in the professional software industry organize their feedback and process improvement approaches. In particular, we focus on the structure and content of improvement and reflection meetings, i.e. retrospectives, and their outcomes. Researching how the vital mechanism of process improvement is implemented in practice in modern software development leads to a more complete picture of agile process improvement.

High-speed digital signal processing requires sophisticated solutions for both software and hardware. Enabling software support and control over hardware functionality is a problem that will be emphasised as systems become more complex. Developing hardwarebound software is not, however, application development, and the related requirements cannot be covered in full by traditional software development processes. This paper describes experiences from the development of hardware-related software development processes at Nokia Networks. As most important characteristics of hardware-related software development, hard real-time requirements, experimental nature of work, documentation requirements, and the role of testing are presented. Characteristics of software process improvement in a hardware-dependent software environment are reported. As a result, a current-state analysis of hardware-related software development was made. Better understanding of the development processes was gained, facilitating further process improvement activities.

The low-code approach is an important area of research being developed to improve the rapid creation and performance of software applications. This approach allows developers and users to easily create software applications through an interactive interface. Existing research shows that the low-code approach accelerates the development process in innovative ways, saves effort and time, and reduces code complexity. This study analyses and compares automatic code generation and transformation techniques in low-code platforms. The study analyses the contributions of automatic code generation and transformation to software engineering processes and evaluates the impact of these techniques on software product quality and development speed. The importance of the Model-Based Development (MBD) approach in automated code generation processes is highlighted. It is stated that MBD provides benefits such as speeding up the software development process and reducing errors by enabling automatic code generation from high-level abstract models. In the study, various literature studies were evaluated to examine the impact of automatic code generation and transformation techniques on the application development process. These studies have shown how automatic code generation and transformation techniques are applied, what results are achieved and what contributions these techniques make to the software development process. The results of the study show that automatic code generation and transformation techniques have a great impact on the application development process. In particular, the acceleration of the software development process, the reduction of errors and the ability to produce more effective and efficient software products are important benefits of these techniques. In addition, it has been found that this approach increases the success of software projects by enabling software developers to produce fast and accurate solutions in complex systems. In this context, the importance of automatic code generation and transformation techniques plays an important role in software development processes.