Chapter 11 - Test Driven Development

Agile development approaches are increasingly being followed and favored by the industry. Test Driven Development (TDD) is a key agile practice and recent research results suggest that the successful adoption of TDD depends on different limiting factors, one of them being insufficient developer testing skills. The goal of this paper is to investigate if developers who are educated on general testing knowledge will be able to utilize TDD more effectively. We conducted a controlled experiment with master students during the course on Software Verification & Validation (V&V) where source code and test cases created by each participant during the labs as well as their answers on a survey questionnaire were collected and analyzed.Descriptive statistics indicate improvements in statement coverage. However, no statistically significant differences could be established between the pre- and post-course groups of students. By qualitative analysis of students’ tests, we noticed a lack of test cases for non-stated requirements (“negative”tests) resulting in a non-detection of bugs. Students did show preference towards TDD in surveys.Although further research is required to fully establish this, we believe that identifying specific testing knowledge which is complementary to the testing skills of a new TDD developer would enable developers to perform their tasks in a more efficient manner.Keywordstest driven developmentcontrolled experimentsoftware testing

Web services are increasingly being used in business critical environments as a mean to provide a service or integrate distinct software services. Research indicates that, in many cases, services are deployed with robustness issues (i.e., displaying unexpected behaviors when in presence of invalid input conditions). Recently, Test-Driven Development (TDD) emerged as software development technique based on test cases that are defined before development, as a way to validate functionalities. However, programmers typically disregard the verification of limit conditions, such as the ones targeted by robustness testing. Moreover, in TDD, tests are created before developing the functionality, conflicting with the typical robustness testing approach. This chapter discusses the integration of robustness testing in TDD for improving the robustness of web services during development. The authors requested three programmers to create a set of services based on open-source code and to implement different versions of the services specified by TPC-App, using both TDD and the approach presented in this chapter. Results indicate that TDD with robustness testing is an effective way to create more robust services.

Test driven development (TDD) is the practice of writing unit tests before writing the source. TDD practitioners typically start with example-based unit tests to verify an understanding of the software's intended functionality and to drive software design decisions. Hence, the typical role of test cases in TDD leans more towards specifying and documenting expected behavior, and less towards detecting faults. Conversely, traditional criteria-based test coverage ignores functionality in favor of tests that thoroughly exercise the software. This paper examines whether it is possible to combine both approaches. Specifically, can additional criteria based tests improve the quality of TDD test suites without disrupting the TDD development process? This paper presents the results of an observational study that generated additional criteria-based tests as part of a TDD exercise. The criterion was mutation analysis and the additional tests were designed to kill mutants not killed by the TDD tests. The additional unit tests found several software faults and other deficiencies in the software. Subsequent interviews with the programmers indicated that they welcomed the additional tests, and that the additional tests did not inhibit their productivity.

In this paper we propose the combination of several techniques into an agile formal development process: model-based testing, formal models, refinement of models, model checking, and test-driven development. The motivation is a smooth integration of formal techniques into an existing development cycle. Formal models are used to generate abstract test cases. These abstract tests are verified against requirement properties by means of model checking. The motivation for verifying the tests and not the model is two-fold: (1) in a typical safety-certification process the test cases are essential, not the models, (2) many common modelling tools do not provide a model checker. We refine the models, check refinement, and generate additional test cases capturing the newly added details. The final refinement step from a model to code is done with classical test-driven development. Hence, a developer implements one generated and formally verified test case after another, until all tests pass. The process is scalable to actual needs. Emphasis can be shifted between formal refinement of models and test-driven development. A car alarm system serves as a demonstrating case-study. We use Back's Action Systems as modelling language and mutation analysis for test case generation. We define refinement as input-output conformance (ioco). Model checking is done with the CADP toolbox.

Test Driven Development (TDD), as a quality promotion approach, suffers from some shortages that discourage its usage. One of the most challenging shortcomings of TDD is the low level of granularity and abstraction. This may lead to production of software that is not acceptable by the end users. Additionally, exploiting of TDD is not applicable in the enterprise systems development. To overcome this defect, we have merged TDD with Model Based Testing (MBT) and suggested a framework named Model Based Test Driven Development (MBTDD). According to TDD, writing test cases comes before programming, and based on our improved method of TDD, modeling precedes writing test cases. To validate the applicability of the proposed framework, we have implemented a use case of Human Resource Management (HRM) system by means of MBTDD. The empirical results of using MBTTD show that our proposed method overwhelms existing deficiencies of TDD.

Test-driven development (TDD) is a software development methodology for achieving high reliability. However, the practice of TDD at the integration level is rare. In this paper, we propose a test-driven component integration approach with the support of UML 2.0 Testing and Monitoring Profile (U2TMP) that is also proposed in this paper. In our approach, automated integration-level test cases are specified using U2TMP before writing the glue-code that integrates components. U2TMP enhances UML 2.0 diagrams for specifying how test scenarios should be set up and how the integrated components are expected to interact under these scenarios. Test cases written in U2TMP can be automatically transformed into test artifacts that setup the test environment, generate test stimuli, and monitor the component interactions at runtime. These test cases serve as a guideline for writing the glue-code and a regression test- bed. With this approach, the software developers may produce glue-code of high reliability.

Coercing new programmers to adopt disciplined development practices such as thorough unit testing is a challenging endeavor. Test-driven development (TDD) has been proposed as a solution to improve both software design and testing. Test-driven learning (TDL) has been proposed as a pedagogical approach for teaching TDD without imposing significant additional instruction time. This research evaluates the effects of students using a test-first (TDD) versus test-last approach in early programming courses, and considers the use of TDL on a limited basis in CS1 and CS2. Software testing, programmer productivity, programmer performance, and programmer opinions are compared between test-first and test-last programming groups. Results from this research indicate that a test-first approach can increase student testing and programmer performance, but that early programmers are very reluctant to adopt a test-first approach, even after having positive experiences using TDD. Further, this research demonstrates that TDL can be applied in CS1/2, but suggests that a more pervasive implementation of TDL may be necessary to motivate and establish disciplined testing practice among early programmers.

Due to embedded co-design considerations, testing embedded software is typically deferred after the integration phase. Contrasting with the current embedded engineering practices, Test-Driven Development (TDD) promotes testing software during its development, even before the target hardware becomes available. Principally, TDD promotes a fast feedback cycle in which a test is written before the implementation. Moreover, each test is added to a test suite, which runs at every step in the TDD cycle. As a consequence, test-driven code is well tested and maintainable. Still, embedded software has some typical properties which impose challenges to apply the TDD cycle. Essentially, uploading software to target is generally too time-consuming to frequently run tests on target. Secondary issues are hardware dependencies and limited resources, such as memory footprint or processing power. In order to deal with these limitations, four methods have been identified and evaluated. Furthermore, a number of relevant design patterns are discussed to apply TDD in an embedded environment.

Test-Driven Development (TDD) is probably the most important agile engineering practice. Since it was first described in detail in [1], this development practice has been adopted widely. This adoption has been also well supported with tools that provide a framework for defining and executing unit tests on the different development platforms. Test-Driven Development provides a guideline how to develop applications unit by unit, resulting in well-designed, well maintainable quality software. TDD focuses on units and it ensures that the atomic building blocks and their interactions are specified and implemented correctly. There is certainly a need for automating other tests as well in order to ensure a working integration environment, to validate performance or to ensure that the application finally behaves as it was specified by the customer. Usually for automating these non-unit tests, developers (mis-)use the unit test frameworks and the unit test execution tools. This paper investigates the potential problems caused by misusing unit test tools for automated functional tests in cases when these functional tests are defined through the development technique called Acceptance Test Driven Development (ATDD). The misuse of the unit testing tools may have direct impact on the testing efficiency and it may also “close the doors” for features specialized for automated functional tests. Some results of this investigation have been prototyped in a tool called SpecRun, which aims to provide better automated functional testing efficiency.

Writing secure code requires a programmer to think both as a defender and an attacker. One can draw a parallel between this model of thinking and techniques used in test-driven development, where students learn by thinking about how to effectively test their code and anticipate possible bugs. In this study, we analyzed the quality of both attack and defense code that students wrote for an assignment given in an introductory security class of 75 (both graduate and senior undergraduate levels) at NYU. We made several observations regarding students' behaviors and the quality of both their defensive and offensive code. We saw that student defensive programs (i.e., assignments) are highly unique and that their attack programs (i.e., test cases) are also relatively unique. In addition, we examined how student behaviors in writing defense programs correlated with their attack program's effectiveness. We found evidence that students who learn to write good defensive programs can write effective attack programs, but the converse is not true. While further exploration of causality is needed, our results indicate that a greater pedagogical emphasis on defensive security may benefit students more than one that emphasizes offense.

Test automation and Test-Driven Development (TDD) have emerged as key strategies for improving code quality and learning effectiveness, while artificial intelligence tools such as GPT are increasingly applied to automated test case generation. However, simply increasing the number of test cases or relying on automation tools does not guarantee sufficient fault detection performance, and empirical evidence of their educational effectiveness for novices remains limited. This study implements a TDD-based practice environment for Python beginners and compares researcher-designed cases, GPT-generated cases, GPT with strategy-guided prompts (GPT+Strategy), and student-written cases. Results show that GPT+Strategy cases achieved higher fault detection and efficiency than even the researcher-designed cases. TDD-based practice strengthened learners’ logical thinking and error recognition skills. These findings provide practical implications for programming curriculum design, AI-assisted learning tool development, and software quality assurance education.

Many computer science students complete their undergraduate degrees with insufficient testing skills and knowledge. To understand the gaps in students' testing skills and knowledge, we analyzed 1014 software tests written by 12 groups in an undergraduate Software Quality Assurance (SQA) course project. In the project the student groups were provided a requirements document and were instructed to follow Test Driven Development (TDD) practices using black-box tests. To understand how the groups applied black-box testing in their project, we created an automatic tool to sort the tests into categories or "test buckets." By analyzing the test bucket data, we were able to assess the effectiveness and efficiency of student-written tests. We observed that the student groups were significantly more likely to test for explicit requirements than implicit requirements and significantly more likely to test happy paths than invalid inputs. Furthermore, students inefficiently tested happy paths, invalid inputs and explicit requirements resulting in a higher proportion of software tests with duplicate intent. Based on these results we provide insights into how black-box test education can be improved.

Putting Requirements and Quality at the Core of Global Service Delivery: Current Efforts and Future Plans at Pace University

The Android platform does not support interoperability among different platforms, and this causes difficulty for developers because they must test applications on different platforms and devices. To solve this problem, this study proposes a test automation system applying test driven development (TDD) that allows automatic performance of repetitive tests. To design the proposed system, test case generation was automated using Annotation.

A survey of software testing practices in Canada