Design quality is vital if software is to be maintainable. What practices do developers actually use to achieve design quality in their day-to-day work and which of these do they find most useful? To discover the extent to which practitioners concern themselves with object-oriented design quality and the approaches used when determining quality in practice, a questionnaire survey of 102 software practitioners, approximately half from the UK and the remainder from elsewhere around the world was used. Individual and peer experience are major contributors to design quality. Classic design guidelines, well-known lower level practices, tools and metrics all can also contribute positively to design quality. There is a potential relationship between testing practices and design quality. Inexperience, time pressures, novel problems, novel technology, and imprecise or changing requirements may have a negative impact on quality. Respondents with most experience are more confident in their design decisions, place more value on reviews by team leads and are more likely to rate design quality as very important. For practitioners, these results identify the techniques and tools that other practitioners find effective. For researchers, the results highlight a need for more work investigating the role of experience in the design process and the contribution experience makes to quality. There is also the potential for more in-depth studies of how practitioners are actually using design guidance, including Clean Code. Lastly, the potential relationship between testing practices and design quality merits further investigation.

To increase user satisfaction and enhance a positive image, the quality of software needs to be continuously improved. This study empirically investigates the importance of 15 quality characteristics and evaluates how well the Web 2.0 applications perform on those characteristics from a data quality perspective. Based on questionnaire responses from 279 participants and the results of importance–performance analysis, the performance of all data quality characteristics was found to be below the end user expectation. Confidentiality showed the greatest discrepancy between importance and performance.

The virtue of quality is not itself a subject; it depends on a subject. In the software engineering field, quality means good software products that meet customer expectations, constraints, and requirements. Despite the numerous approaches, methods, descriptive models, and tools, that have been developed, a level of consensus has been reached by software practitioners. However, in the model-driven engineering (MDE) field, which has emerged from software engineering paradigms, quality continues to be a great challenge since the subject is not fully defined. The use of models alone is not enough to manage all of the quality issues at the modeling language level. In this work, we present the current state and some relevant considerations regarding quality in MDE, by identifying current categories in quality conception and by highlighting quality issues in real applications of the model-driven initiatives. We identified 16 categories in the definition of quality in MDE. From this identification, by applying an adaptive sampling approach, we discovered the five most influential authors for the works that propose definitions of quality. These include (in order): the OMG standards (e.g., MDA, UML, MOF, OCL, SysML), the ISO standards for software quality models (e.g., 9126 and 25,000), Krogstie, Lindland, and Moody. We also discovered families of works about quality, i.e., works that belong to the same author or topic. Seventy-three works were found with evidence of the mismatch between the academic/research field of quality evaluation of modeling languages and actual MDE practice in industry. We demonstrate that this field does not currently solve quality issues reported in industrial scenarios. The evidence of the mismatch was grouped in eight categories, four for academic/research evidence and four for industrial reports. These categories were detected based on the scope proposed in each one of the academic/research works and from the questions and issues raised by real practitioners. We then proposed a scenario to illustrate quality issues in a real information system project in which multiple modeling languages were used. For the evaluation of the quality of this MDE scenario, we chose one of the most cited and influential quality frameworks; it was detected from the information obtained in the identification of the categories about quality definition for MDE. We demonstrated that the selected framework falls short in addressing the quality issues. Finally, based on the findings, we derive eight challenges for quality evaluation in MDE projects that current quality initiatives do not address sufficiently.

Software engineering research now considers that no system is an island, but it is part of an ecosystem involving other systems, developers, and users. When a framework or a library evolves, its clients often must adapt. For example, client developers might need to adapt to functionalities, client systems might need to be adapted to a new API, and client users might need to adapt to a new user interface. The consequences of these changes are yet unclear: what proportion of the ecosystem might be expected to react, how long might it take for a change to diffuse in the ecosystem, do all clients react in the same way? This paper reports an exploratory study aimed at observing API evolution and its impact on a large software ecosystem, Pharo, which has about 3,600 distinct systems, and six years of evolution. We analyze 118 API changes in the context of method replacement and suggestion, and answer research questions regarding the magnitude, duration, extension, and consistency of such changes in the ecosystem. The results of this study help to characterize the impact of API evolution in large software ecosystems and provide the basis to better understand how such impact can be alleviated.

The quality of requirements is typically considered as an important factor for the quality of the end product. For traditional up-front requirements specifications, a number of standards have been defined on what constitutes good quality : Requirements should be complete, unambiguous, specific, time-bounded, consistent, etc. For agile requirements specifications, no new standards have been defined yet, and it is not clear yet whether traditional quality criteria still apply. To investigate what quality criteria for assessing the correctness of written agile requirements exist, we have conducted a systematic literature review. The review resulted in a list of 16 selected papers on this topic. These selected papers describe 28 different quality criteria for agile requirements specifications. We categorize and analyze these criteria and compare them with those from traditional requirements engineering. We discuss findings from the 16 papers in the form of recommendations for practitioners on quality assessment of agile requirements. At the same time, we indicate the open points in the form of a research agenda for researchers working on this topic .

Cyber-physical systems (CPSs) are ubiquitous systems that integrate digital technologies with physical processes. These systems are becoming configurable to respond to the different needs that users demand. As a consequence, their variability is increasing, and they can be configured in many system variants. To ensure a systematic test execution of CPSs, a test system must be elaborated encapsulating several sources such as test cases or test oracles. Manually building a test system for each configuration is a non-systematic, time-consuming, and error-prone process. To overcome these problems, we designed a test system for testing CPSs and we analyzed the variability that it needed to test different configurations. Based on this analysis, we propose a methodology supported by a tool named ASTERYSCO that automatically generates simulation-based test system instances to test individual configurations of CPSs. To evaluate the proposed methodology, we selected different configurations of a configurable Unmanned Aerial Vehicle, and measured the time required to generate their test systems. On average, around 119 s were needed by our tool to generate the test system for 38 configurations. In addition, we compared the process of generating test system instances between the method we propose and a manual approach. Based on this comparison, we believe that the proposed tool allows a systematic method of generating test system instances. We believe that our approach permits an important step toward the full automation of testing in the field of configurable CPSs.

One of the grand challenges for adequately testing complex software is due to the oracle problem. Metamorphic Testing (MT) is a promising technique to alleviate the oracle problem through using one or multiple Metamorphic Relations (MRs) as test oracles. MT checks the satisfaction of every MR among the outputs of the MR-related tests instead of the correctness of individual test outputs. In practice, it is fairly easy to find MRs for testing any program, but it is very difficult to develop “good” MRs and evaluate their adequacy. A systematic approach for developing MRs and evaluating their adequacy in MT remains to be developed. In this paper, we propose a framework for evaluating MT and iteratively developing adequate MRs monitored by MT adequacy evaluation. The MT adequacy is measured by program coverages, mutation testing, and testing MRs with mutation tests. The MT evaluation results are used for guiding the iterative development of MRs, generating tests, and analyzing test outputs. We explain the framework through a testing example on an image processing program that is used for building the 3-dimensional structure of a biology cell based on its confocal image sections. In order to demonstrate the effectiveness of the proposed framework, we reported a case study of testing a complex scientific program: a Monte Carlo modeling program that simulates photon propagations in turbid tissue phantoms for accurate and efficient generation of reflectance images from biological tissues. The case study has shown the effectiveness of proposed MT framework for testing scientific software in general and the necessity of the MT enhancement in the development of adequate MRs. The case study results can be easily adapted for testing other software.

Automated program repair (APR) tools apply fault localization (FL) techniques to identify the locations of likely faults to be repaired. The effectiveness, performance, and repair correctness of APR depends in part on the FL method used. If FL does not identify the location of a fault, the application of an APR tool will not be effective--it will fail to repair the fault. If FL assigns the actual faulty statement a low priority for repair, APR performance will be reduced by increasing the time required to find a potential repair. In addition, the correctness of a generated repair will be decreased since APR will modify fault-free statements that are assigned a higher priority for repair than an actual faulty statement. We conducted a controlled experiment to evaluate the impact of ten FL techniques on APR effectiveness, performance, and repair correctness using a brute force APR tool applied to faulty versions of the Siemens Suite and two other large programs: space and sed. All FL techniques were effective in identifying all faults; however, Wong3 and Ample1 were the least effective FL techniques since they assigned the lowest priority for repair in more than 26 % of the trials. We obtained the worst APR performance significantly when Ample1 was used since it generated a large number of variants in 29.11 % of the trials, and took the longest time to produce potential repairs. Jaccard FL improved repair correctness by generating more validated repairs---potential repairs that pass a set of regression tests, and generating potential repairs that failed fewer regression tests. Also Jaccard's performance is noteworthy in that it never generated a large number of variants during the repair process compared to the alternatives.

Human resource management practices are key for the success of software development projects. Practices that promote knowledge sharing and organizational learning are positively related to development---effort curves, and thus software companies are looking for different alternatives oriented to promoting these practices. The model-driven development (MDD) paradigm is positioned as one of the best alternatives for reutilization of development knowledge. In particular, this paradigm considers the specification of conceptual models that can be used as input for automatic code generation to different target platforms. However, testing of applications developed through MDD solutions is still performed by the manual definition and execution of test cases by testers, which negatively impacts in the time reduction obtained from automatic code generation and the reutilization of knowledge generated during the MDD project execution. To address this issue, this paper presents a testing approach that automatically generates executable test cases for software developed by using MDD technologies.