Crowdsourcing in Software Development: Empirical Support for Configuring Contests

Code search is an essential task in software development. Developers often search the internet and other code databases for necessary source code snippets to ease the development efforts. Code search techniques also help learn programming as novice programmers or students can quickly retrieve (hopefully good) examples already used in actual software projects. Given the recurrence of the code search activity in software development, there is an increasing interest in the research community. To improve the code search experience, the research community suggests many code search tools and techniques. These tools and techniques leverage several different ideas and claim a better code search performance. However, it is still challenging to illustrate a comprehensive view of the field considering that existing studies generally explore narrow and limited subsets of used components. This study aims to devise a grounded approach to understanding the procedure for code search and build an operational taxonomy capturing the critical facets of code search techniques. Additionally, we investigate evaluation methods, benchmarks, and datasets used in the field of code search.

Software development is a collaborative process in which programmers build software by integrating all the stages of the software development life cycle (SDLC). A software repository is a central file storage location where various software packages are stored, and these packages are retrieved and shared between all the software development team members at various locations. The software repositories are divided into various categories based on cooperation, coordination, and communication among the stakeholders as well as evolutionary changes to various software arti-facts such as source code repositories, software bug repositories, historical repositories, run-time repositories and requirement documents, and other documentation. The software bug repository is an essential repository among the entire repositories since the completion of the software is entirely dependent on the bug fixing mechanism associated with this repository in software development. Today's software systems are larger and more complex as they go through various stages from the requirement 98analysis phase to the maintenance phase. A variety of tasks and activities are carried out in each stage of software development, and these are expensive and vulnerable to errors. During software development, a large number of software bugs are continuously generated, and that has become the main reason for the delay in software completion. Hence, there is a vast demand for computational intelligent techniques to accomplish various tasks of software development. In recent years, fuzzy logic techniques emerged and played an important role in various fields of data mining and text mining. Since most of the content related to software bug repositories is text in nature, it is possible to effectively use fuzzy logic techniques to analyze these software bugs.

MARVEL is a kernel for software development environments that enacts a processing modeling notation, called the MARVEL Strategy Language (MSL). The software process is defined in MSL as a collection of modular strategies, where different (sets of these) strategies are appropriate for different tool suites, user roles, phases of the lifecycle, management policies, etc. Each strategy specifies a partial object-oriented data model and a partial rule-based process model. The strategies are supplemented by a collection of envelopes, each of which describes how to apply one or more of the external tools used during software development activities; this includes the mapping from the inputs/outputs of these tools and the MARVEL objectbase and the command lines to invoke these tools. Multiple MSL strategies are merged to obtain the full data and process models for enaction by the MARVEL kernel.Each software development activity is described in MSL by a rule specifying the precondition and multiple postconditions of the activity. Both preconditions and postconditions are written in a subset of the first order predicate logic, stylized to incorporate directives for efficient queries and updates. A precondition is a logical statement that must be true before the activity is initiated; a postcondition is a logical statement asserted to be true when the activity completes. Multiple mutually exclusive postconditions reflect the multiple possible results of many activities. An activity is typically a particular combination of software development tool, options specific to that tool, and typed arguments. The notion of activity could be extended to include loading a new set of strategies to take over at this point in the process, to support both nesting of hierarchical activities and successive activity phases.Enaction of an MSL process model is implemented by what we call opportunistic processing, consisting primarily of forward and backward chaining among the rules. When a user initiates an activity, MARVEL applies backward chaining to attempt to satisfy its precondition. If this is not possible, MARVEL could use the rules in the style of an expert system to explain the problem. Once an activity has completed, MARVEL asserts one of its postconditions and uses forward chaining to automatically carry out other activities whose results are presumed to be needed in the near future.We have designed two facilities for controlling this processing, to prevent runaway automation. Hints are additional MSL preconditions that affect automatic but not user-initiated invocation of tools. Implicit queries represent a reflective capacity to keep tabs on estimated costs of automatic behavior and request user confirmation when certain user-selected thresholds are reached.MARVEL version 2.10 was recently completed, an upgrade over the version 2.01 that was demonstrated at the 3rd PSDE conference in November 1988. This latest version of MARVEL adds an X11 windows graphical user interface and new subsystems called Marvelizer and Organ. Marvelizer is a tool for immigrating existing software systems from the Unix file system or (a restricted class of) existing software development environment into a MARVEL objectbase. Organ is a set of commands for reorganizing MARVEL objectbases as the project and/or system structure evolves. Marvelizer has been used to immigrate Marvel itself, including about 35,000 lines of C code, into a C/Marvel objectbase. We would like to use the C/Marvel set of MSL strategies, which support activities using the user's favorite editor, lint, cc, ar, ranlib, less (and soon rcs, dbx and mail), for our own further development of MARVEL. Both MARVEL and C/Marvel were in use during the Spring 1989 semester by students in my E6123y Programming Environments and Software Tools course (admittedly a captive audience).We have discovered several difficulties with the MARVEL paradigm that seem likely to apply at least partially to enaction of other notations for software process modeling. First, constructing and debugging a process model is hard. We have run into many unanticipated interactions among rules, particularly among those defined in different strategies that happen to be loaded into the MARVEL kernel at the same time. We are working on debugging facilities that will uncover all direct and transitive dependencies by constructing a graphical dataflow graph of the currently loaded rules. It is feasible for a user to enable and disable individual rules, both during debugging to see how that affects the dataflow among rules and also during operation because certain behavior is not currently desired. This would be dangerous, however, if done by a naive user: if rules were turned off and on at arbitrary points, it would be possible for MARVEL to get into what is an inconsistent state with respect to the full set of rules in such a way that automatic recovery is impossible.Similar difficulties seem likely to arise for any enactable process modeling paradigm. The following questions need to be answered: How does an implementor go about writing, testing and debugging a process model? For non-trivial processes, the model is likely to be very large and hence should be modularized. What is the appropriate style of interface among process model modules and how is it enforced? Is the process model visible to the users and, if so, how is it presented? If the users can modify the process model during enaction, how does this affect continuing the process? Should (can) consistency be enforced or should (can) inconsistencies be detected and repaired?Second, constructing envelopes is even harder. We currently use the Shell languages (e.g., CShell) as notations for defining envelopes. Writing the Shell scripts requires an intimate knowledge of both the tool and the mapping of the MARVEL objectbase onto the file system. Although there is no way to avoid the former, we hope to minimize the latter by developing a higher level envelope language that understands the MARVEL objectbase implementation so the envelope writer doesn't have to. However, this is currently a relatively low priority due to lack of manpower.Again, similar difficulties seem inevitable for other enaction mechanisms intended to work with external tools. The questions: Is it possible (or desirable) to support existing tools, particularly tools where retrofitting is impractical? If only new tools are supported, can a relatively simple standard interface be developed and published so these tools can be developed separately by vendors? Would such an interface pre-empt too many design decisions, or would it be feasible for vendors to upgrade their current products?Third, supporting multiple users is hardest of all. The current MARVEL kernel does not provide any facilities for concurrency control. It would be easy to add locking of the entire objectbase, but a more flexible transaction model is needed. We are working on a number of alternative models that we think are promising for incorporation into the MARVEL kernel. Commit-serializability introduces two new transaction operations, split-transaction and join-transaction, that manipulate in-progress transactions to support coordination among multiple users. Participant transactions (AKA participation domains) distinguishes participants in group software development tasks from observers, and relaxes serializability as viewed by participants to permit cooperation.It is essential for enactable process models to support coordination and cooperation among the members of software development teams. There are two sets of questions: First, what are appropriate processes for programming-in-the-many? This is itself an active area of research, and there is certainly no consensus. One opinion, voiced at the previous workshop, is that the community should not be enacting process models until we understand the processes we are modeling. An opposing opinion is that enaction gives the community the capability to experiment with different processes without prematurely making a full scale commitment to a particular process. Assuming agreement with the second view, what process modeling notations and enaction mechanisms can support a reasonable range of cooperative work facilities?The database and operating system communities have agreed on atomic and serializable transactions, which can be implemented using a wide variety of technologies, as the basis for data consistency, failure recovery and concurrency control across their traditional application areas. Can the software engineering community develop an equally all-encompassing “transaction” model? Or is there some characteristic of the software development process such that coordination and cooperation policies will always require special support on a case-by-case basis? If so, what does this say about enaction of process models in general?In summary, MARVEL is implemented and in limited use, it works in some cases, it doesn't work in others, it raises a lot of questions, and both we the MARVEL group and we the software engineering community have a lot more work to do.

Leveraging the abundant community resources, e.g. campus students, programming fans, software amateurs, is promising to be a flexible and effective approach for relieving the enterprise's IT resource shortage. However, the concern about confidential knowledge exposure makes most enterprises hesitate to outsource software development to community resources. Unlike the traditional legal protection mechanism that prevents community resources from exposing the already known confidential knowledge, this paper presents a proactive confidential knowledge protection technology, which prevents the community resources from learning the confidential knowledge in software development activities. First, the client is facilitated on identifying confidential knowledge involved in the software development activity. Then, the risky software design elements that might expose confidential knowledge are identified in both qualitative and quantitative way. Based on the identification result, a holistic proactive protection solution is presented by orchestrating task reservation, task pretending and task partition activities. Finally, the processed software development tasks are dispatched to community developers without worrying about the confidential knowledge exposure. The technology has been preliminarily validated in two community-based software development cases.

Program comprehension is a required activity for many software development and maintenance tasks. A common means of understanding software is though UML diagrams. UML diagrams model the design and implementation of an application, describing both its structure and behaviour. However with rapid software development life cycles, UML diagrams often become inconsistent with the implementation of an application. This limits their effectiveness in program comprehension activities. This paper presents a program analysis framework to reverse engineer sequence diagrams from application source code. Part of this framework is the ability to filter out irrelevant operations from a sequence diagram in order to simplify the representation of an application. This is achieved by identifying important data, and following its use in the application. The purpose of this framework is to aid in program comprehension activities by providing up to date representations of an application; while also enabling developers to identify the logical operation of a program without interference from irrelevant or supporting operations.

With the increasing range of software development and applications, the scale of software development is becoming larger and larger. The larger software's scale is the more possible failed in the phase of design, development, and implementation in software. As a starting point to improve the management and development of software, this paper designs a service platform for data services - JCFXBL. JCFXBL is mainly used to solve two issues, one is the management of software development, and the other is the technology of software development. In combination with two the practical application cases based on JCFXBL, from software development, software design, project management, the article describes the role of JCFXBL platform in the “Project Management” and “systems integration projects”, and verifies that JCFXBL platform can reduce the complexity of software development, improve software quality and solve software development, management issues, by actual software development.

Crowdsourcing divides a task into small pieces that are carried out by the crowd. In Software Engineering, crowdsourcing divides the software development tasks of to be carried out online by the crow and is simply called Software Crowdsourcing (SW CS). Most SW CS platforms support this emerging software development strategy and operate within a framework of competition among the crowd. Competitive SW CS platforms intentionally minimize communication and collaboration among the parties involved (customer, platform, and crowd) while they compete in the software development tasks. The goal of this paper is to investigate platform moderators in SW CS challenges. Platform moderators are individuals who work for the SW CS platforms to mediate customer and crowd. A qualitative analysis of the content of the communication forums hosted on the TopCoder platform was performed to analyze the messages exchanged by the platform moderators and the crowd. Our empirical results indicate that co-pilots enforce and, at the same time, extend the limitations of the documentation associated with the tasks to support crowd members, provide technical help to crowd members during the competitions, and engage the crowd in the challenges. Co-pilots are organized, work diligently, worrying about being fair, and, at the same time, seeking to find a balance between autonomy and dependency on the customer. We conclude by providing insights to improve the design of software crowdsourcing platforms.

Role stereotypes indicate generic roles that classes play in the design of software systems (e.g. controller, information holder, or interfacer). Knowledge about the role-stereotypes can help in various tasks in software development and maintenance, such as program understanding, program summarization, and quality assurance. This paper presents an automated machine learning-based approach for classifying the role-stereotype of classes in Java. We analyse the performance of this approach against a manually labelled ground truth for a sizable open source project (of 770+ Java classes) for the Android platform. Moreover, we compare our approach to an existing rule-based classification approach. The contributions of this paper include an analysis of which machine learning algorithms and which features provide the best classification performance. This analysis shows that the Random Forest algorithm yields the best classification performance. We find however, that the performance of the ML-classifier varies a lot for classifying different role-stereotypes. In particular its performs degrades for rare role-types. Our ML-classifier improves over the existing rule-based classification method in that the ML-approach classifies all classes, while rule-based approaches leave a significant number of classes unclassified.

Bug localization, i.e., locating bugs in code snippets, is a frequent task in software development. Although static bug-finding tools are available to reduce manual effort in bug localization, these tools typically detect bugs with known project-independent bug patterns. However, many bugs in real-world code snippets are project-specific. To address this issue, in this paper, we propose a novel approach for LOcating Bugs By Searching the most similar sample snippet (LOBBYS). LOBBYS detects bugs with the help of crowd-contributed correct code, which implement the function that buggy code is expected to implement. Given a buggy code snippet, LOBBYS takes two steps to locate the bug: (1) normalize the bug-gy snippet, and then search for the most similar sample snippet from the code base; (2) align the buggy code and sample code snip-pets, find the difference between the two code snippets, and generate a bug report based on the difference. To evaluate LOBBYS, we build one algorithm-oriented code base and select some buggy snippets from two real-world systems. The result shows that LOBBYS can effectively locate bugs for buggy snippets with high precision. Under the similarity of 50%, 70% and 90%, LOBBYS achieves bug-localization precision as 67%, 83%, and 92%.

Program entanglement, feature interaction and the feature language extensions

Debugging is a time-consuming task in software development and maintenance. To accelerate this task, several approaches have been proposed to automate fault localization. In particular, testing based fault localization (TBFL), which utilizes the testing information to localize the faults, seem to be very promising. However, the similarity between test cases in the test suite has been ignored in the research on TBFL. In this paper, we investigate this similarity issue and propose a novel approach named similarity-aware fault localization (SAFL), which can calculate the suspicion probability of each statement with little impact by the similarity issue. To address and deal with the similarity between test cases, SAFL applies the theory of fuzzy sets to remove the uneven distribution of the test cases. We also performed an experimental study for two real-world programs at different size levels to evaluate SAFL together with another two approaches to TBFL. Experimental results show that SAFL is more effective than the other two approaches when the test suites contain injected redundancy, and SAFL can achieve a competitive result with normal test suites. SAFL can also be more effective than applying test suite reduction to current approaches to TBFL.

Code reading is a prerequisite of program comprehension which is a fundamental task in software development. Strategies employed on code reading affect the programmer’s success rate of understanding tasks such as debugging. However, there is still limited knowledge about the code reading strategies used by students while performing bug finding task. In this paper, the author describes a summary of her research on novice programmer debugging skills using eye tracking data as a methodology. Eye tracking data were extracted and analyzed using visual effort metrics and sequential analysis of scanpaths using a clustering algorithm to determine common code reading patterns. The author’s research findings revealed differences on the code reading patterns and code reading strategies of high and low performing students. Empirical evaluation on the effectiveness of the strategies used by high performing students was also conducted which suggests that by teaching these strategies to students, improved debugging performance can be observed.

Reading and understanding source code is a major task in software development. Code comprehension depends on the quality of code, which is impacted by code structure and identifier naming. In this paper we empirically investigated whether longer but more descriptive identifier names improve code comprehension compared to short names, as they represent useful information in more detail. In a web-based study 88 Java developers were asked to locate a semantic defect in source code snippets. With descriptive identifier names, developers spent more time in the lines of code before the actual defect occurred and changed their reading direction less often, finding the semantic defect about 14% faster than with shorter but less descriptive identifier names. These effects disappeared when developers searched for a syntax error, i.e., when no in-depth understanding of the code was required. Interestingly, the style of identifier names had a clear impact on program comprehension for more experienced developers but not for less experienced developers.

Large Language Models (LLMs) have demonstrated powerful capabilities in the field of code generation, with a deep understanding of the semantics and functionality of code. Building websites is one of the most important tasks in software development, as it utilizes rich frontend displays and backend processing to achieve various service functions. It is one of the most widely used interactive software models. Although there have been some efforts in Web website generation, these efforts have been limited to the automation of generating Web pages. The advent of LLMs provides a new approach to Web site generation tasks. However, there is currently a lack of comprehensive evaluation of the generation performance of LLMs in this context, making it difficult to optimize and improve the generated results in a targeted manner. To address this issue, this paper conducts a multi-angle investigation and analysis of the performance of LLMs in Web site generation tasks. Firstly, Web generation requirements are collected, and effective prompt engineering is designed. These prompts are then input into different LLMs to initiate the self-iteration process. Next, the generated code is fed back into the LLM for security self-iteration, where the model performs vulnerability detection and repair on the code it has generated. The security-enhanced code is subsequently subjected to manual review, where it is evaluated using predefined quantitative metrics to generate indicator values. Finally, through testing, the quality, security, and code defects of the generated front-end and back-end Web code across different LLMs are analyzed, providing a comprehensive evaluation of the generation results. The experiments demonstrate that LLM systems perform well in completing and implementing the functions and layouts of pages in prompts for Web generation tasks, but there remains room for improvement in the security of the Web code.

One central task in software development by means of graph-based techniques is to inspect and to query the underlying graph. Important issues are, for example, to detect general graph properties like connectivity, to explore more special features like the applicability of left-hand side rules in graph transformations, or to validate snapshots of evolving systems by checking properties in an on-the-fly way. We propose a new approach combining visual and textual techniques for exploring graphs. We emphasize a particular aspect of the underlying graph by showing or hiding nodes and edges. We offer three different ways to explore (meta-)model snapshots which may be combined: (1) selection by object identity and class membership, (2) selection by OCL expression, and (3) selection by path length. One main motivation for our work is to access large or complicated graphs in a systematic way. We evaluate our approach by different middle-sized scenarios. Our evaluation shows that the approach works for large graphs with about 1000 nodes and 2000 edges and for graphs which instantiate metamodels representing software engineering artifacts.