Analysis and Comparison of Neural Network Models for Software Development Effort Estimation

Estimation of Software Development Effort from Requirements Based Complexity

Function point analysis is a widely used metric in the software industry for development effort estimation. It was proposed in the 1970s, and then standardized by the International Function Point Users Group, as accepted by many organizations worldwide. While the software industry has grown rapidly, the weight values specified for the standard function point counting have remained the same since its inception. Another problem is that software development in different industry sectors is peculiar, but basic rules apply to all. These raise important questions about the validity of weight values in practical applications. In this study, we propose an algorithm for calibrating the standardized functional complexity weights, aiming to estimate a more accurate software size that fits specific software applications, reflects software industry trends, and improves the effort estimation of software projects. The results show that the proposed algorithms improve effort estimation accuracy against the baseline method.

Inconsistency of expert judgment-based estimates of software development effort

The accurate estimation of software development effort has major implications for the management of software development in the industry. Underestimates lead to time pressures that may compromise full functional development and thorough testing of the software product. On the other hand, overestimates can result in over allocation of development resources and personnel [1]. Many models for effort estimation have been developed during the past years; some of them use parametric methods with some degree of success, other kind of methods belonging to the machine learning techniques, such as Neural Networks (NN), have been also studied in this field showing more accurate estimations, and finally the Genetic programming (GP) techniques are being considered as promising tools for the prediction of effort estimation. In this work we are to compare the behaviour of GP and Genetic Algorithms (GA) to the Neural Network approach (backpropagation).

The estimation of software development effort has been centralized mostly on the accuracy of estimates through dealing with heterogeneous datasets regardless of the fact that the software projects are inherently complex and uncertain. In particular, Analogy Based Estimation (ABE), as a widely accepted estimation method, suffers a great deal from the problem of inconsistent and non-normal datasets because it is a comparison-based method and the quality of comparisons strongly depends on the consistency of projects. In order to overcome this problem, prior studies have suggested the use of weighting methods, outlier elimination techniques and various types of soft computing methods. However the proposed methods have reduced the complexity and uncertainty of projects, the results are not still convincing and the methods are limited to a special domain of software projects, which causes the generalization of methods to be impossible. Localization of comparison and weighting processes through clustering of projects is the main idea behind this paper. A hybrid model is proposed in which the software projects are divided into several clusters based on key attributes (development type, organization type and development platform). A combination of ABE and Particle Swarm Optimization (PSO) algorithm is used to design a weighting system in which the project attributes of different clusters are given different weights. Instead of comparing a new project with all the historical projects, it is only compared with the projects located in the related clusters based on the common attributes. The proposed method was evaluated through three real datasets that include a total of 505 software projects. The performance of the proposed model was compared with other well-known estimation methods and the promising results showed that the proposed localization can considerably improve the accuracy of estimates. Besides the increase in accuracy, the results also certified that the proposed method is flexible enough to be used in a wide range of software projects.

Crowd-Sourced software development (CSSD) is getting a good deal of attention from the software and research community in recent times. One of the key challenges faced by CSSD platforms is the task selection mechanism which in practice, contains no intelligent scheme. Rather, rule-of-thumb or intuition strategies are employed, leading to biasness and subjectivity. Effort considerations on crowdsourced tasks can offer good foundation for task selection criteria but are not much investigated. Software development effort estimation (SDEE) is quite prevalent domain in software engineering but only investigated for in-house development. For open-sourced or crowdsourced platforms, it is rarely explored. Moreover, Machine learning (ML) techniques are overpowering SDEE with a claim to provide more accurate estimation results. This work aims to conjoin ML-based SDEE to analyze development effort measures on CSSD platform. The purpose is to discover development-oriented features for crowdsourced tasks and analyze performance of ML techniques to find best estimation model on CSSD dataset. TopCoder is selected as target CSSD platform for the study. TopCoder’s development tasks data with development-centric features are extracted, leading to statistical, regression and correlation analysis to justify features’ significance. For effort estimation, 10 ML families with 2 respective techniques are applied to get broader aspect of estimation. Five performance metrices (MSE, RMSE, MMRE, MdMRE, Pred (25) and Welch’s statistical test are incorporated to judge the worth of effort estimation model’s performance. Data analysis results show that selected features of TopCoder pertain reasonable model significance, regression, and correlation measures. Findings of ML effort estimation depicted that best results for TopCoder dataset can be acquired by linear, non-linear regression and SVM family models. To conclude, the study identified the most relevant development features for CSSD platform, confirmed by in-depth data analysis. This reflects careful selection of effort estimation features to offer good basis of accurate ML estimate.

Background. Currently there are several definitions of measures that should represent the size of software functional requirements. These measures have gained a quite relevant role, since they are one of the few basis upon which effort estimation can be based. However, traditional Functional Size Measures do not take into account the amount and complexity of the elaboration required, concentrating instead on the amount of data accessed or moved. This is a problem, when it comes to effort estimation, since the amount and complexity of the required data elaborations affect the implementation effort, but are not adequately represented by the current measures (including the standardized ones).

Software development effort estimation (SDEE) generally involves leveraging the information about the effort spent in developing similar software in the past. Most organizations do not have access to sufficient and reliable forms of such data from past projects. As such, the existing SDEE methods suffer from low usage and accuracy. We propose an efficient SDEE method for open source software, which provides accurate and fast effort estimates. The significant contributions of our article are (i) novel SDEE software metrics derived from developer activity information of various software repositories, (ii) an SDEE dataset comprising the SDEE metrics’ values derived from approximately 13,000 GitHub repositories from 150 different software categories, and (iii) an effort estimation tool based on SDEE metrics and a software description similarity model . Our software description similarity model is basically a machine learning model trained using the PVA on the software product descriptions of GitHub repositories. Given the software description of a newly envisioned software, our tool yields an effort estimate for developing it. Our method achieves the highest standardized accuracy score of 87.26% (with Cliff’s δ = 0.88 at 99.999% confidence level) and 42.7% with the automatically transformed linear baseline model. Our software artifacts are available at https://doi.org/10.5281/zenodo.5095723.

Due to the intangible nature of "software", accurate and reliable software effort estimation is a challenge in the software Industry.It is unlikely to expect very accurate estimates of software development effort because of the inherent uncertainty in software development projects and the complex and dynamic interaction of factors that impact software development.Heterogeneity exists in the software engineering datasets because data is made available from diverse sources. This can be reduced by defining certain relationship between the data values by classifying them into different clusters. This study focuses on how the combination of clustering and regression techniques can reduce the potential problems in effectiveness of predictive efficiencydue to heterogeneity of the data.Using a clustered approach creates the subsets of data having a degree of homogeneity that enhances prediction accuracy.It was also observed in this study that ridge regression performs better than other regression techniques used in the analysis.

Effort estimations are a crucial aspect of software development. The tasks should be completed before the start of any software project. Accurate estimations increase the chances of project success, and inaccurate information can lead to severe issues. This study systematically reviewed the literature on effort-estimating models from 2015-2024, identifying 69 relevant studies from various publications to compile information on various software work estimation models. This review aims to analyze the models proposed in the literature and their classification, the metrics used for accuracy measurement, the leading model that has been chiefly applied for effort estimation, and the benchmark datasets available. The study utilized 542 relevant articles on software development, cost, effort, prediction, estimation, and modelling techniques in the search strategy. After 194 selections, the authors chose 69 articles to understand ML applications in SEE comprehensively. The researchers used a scoring system to assess each study’s responses (from 0 to 5 points) to their research questions. This helped them identify credible studies with higher scores for a comprehensive review aligned with its objectives. The data extraction process identified 91% (63) of 69 studies as either highly or somewhat relevant, demonstrating a successful search strategy for analysis. The literature review on SEE indicates a growing preference for ML-based models in 59% of selected studies. 17% of the studies chosen favor hybrid models to overcome software development challenges. We qualitatively analyzed all the literature on software effort estimation using expert judgment, formal estimation techniques, ML-based techniques, and hybrid techniques. We discovered that researchers have frequently used ML-based models to estimate software effort and are currently in the lead. This study also explores the application of feature importance and selection in machine learning models for Software Effort Estimation (SEE) using popular algorithms like support Vector Machine (SVM), AdaBoost (AB), Gradient Boost (GB), and Random Forest (RF) with six benchmark datasets like CHINA, COCOMO-NASA2, COCOMO, COCOMO81, DESHARNAIS, and KITCHENHAM. We analyze the dataset descriptions and feature importance of the dataset analysis using ML models for choosing crucial play attributes in SEE.

Inaccurate estimates of software development effort is a frequently reported cause of IT-project failures. We report results from a study that investigated the effect of introducing lessons-learned sessions on estimation accuracy and the assessment of uncertainty. Twenty software professionals were randomly allocated to a Learning group or a Control group and instructed to estimate and complete the same five development tasks. Those in the Learning group but not those in the Control group were instructed to spend at least 30 minutes on identifying, analyzing, and summarizing their effort estimation and uncertainty assessment experience after completing each task. We found that the estimation accuracy and the realism of the uncertainty assessment were not better in the Learning group than in the Control group. A follow-up study with 83 software professionals was completed to better understand this lack of improvement from lessons-learned sessions. The follow-up study found that receiving feedback about other software professionals' estimation performance led to more realistic uncertainty assessments than receiving the same feedback of one's own estimates. Lessons-learned sessions, not only in estimation contexts, have to be carefully designed to avoid wasting resources on learning processes that stimulate rather than reduce learning biases.

Software cost and time estimation is the process of estimating the cost and time required to develop a software system. Software cost and time estimation supports the planning and tracking of software projects. Effectively controlling the expensive investment of software development is one of the important issues in software project management. Estimating software development cost with high precision is still a great challenge for project managers, because it allows for considerable financial and strategic planning. Software cost estimation refers to the predictions of the likely amount of effort, time, and staffing levels required to build a software system. A very helpful form of cost estimation is the one made at an early stage during a project, when the costing of the project is proposed for approval. However, estimates at the early stages of the development are the most difficult to obtain. In this paper a novel Constructive Cost Model (COCOMO) based on soft computing approach is proposed for software cost estimation. This model carries some of the desirable features of neural networks approach, such as learning ability and good interpretability, while maintaining the merits of the COCOMO model. Unlike the standard neural networks approach, the proposed model can be interpreted and validated by experts, and has good generalisation capability. The model deals effectively with imprecise and uncertain input and enhances the reliability of software cost estimates. From the experimental results, it was concluded that, by the proposed neural network model, the accuracy of cost estimation can be improved and the estimated cost can be very close to the actual cost.

The development effort estimation is one of the most difficult problems in software project management. It is one of the most critical aspects in the early stages of the software project. Several software development effort estimation models have been proposed, however, these models are not able to obtain more than a 25 percent of accuracy, neither provide an understandable model for experts in the application area. Therefore, in this paper, we present EEpred, an explanatory model to estimate the development effort based on data of known software projects. It is a serial multiple classifier system based-on several decision trees. The model performance was evaluated by an internal validation procedure, analyzing their robustness and predictive performance. This procedure demonstrates that EEpred is able to estimate the software development effort with a 71 percent of precision. The main advantage of EEpred, regarding to other algorithms, is its ability to translate the process into a collection of simple decision rules, providing more easily interpretable knowledge that can help software engineer to improve decision-making on development planning.

In project management, effective cost estimation is one of the most crucial activities to efficiently manage resources by predicting the required cost to fulfill a given task. However, finding the best estimation results in software development is challenging. Thus, accurate estimation of software development efforts is always a concern for many companies. In this paper, we proposed a novel software development effort estimation model based both on constructive cost model II (COCOMO II) and the artificial neural network (ANN). An artificial neural network enhances the COCOMO model, and the value of the baseline effort constant A is calibrated to use it in the proposed model equation. Three state-of-the-art publicly available datasets are used for experiments. The backpropagation feedforward procedure used a training set by iteratively processing and training a neural network. The proposed model is tested on the test set. The estimated effort is compared with the actual effort value. Experimental results show that the effort estimated by the proposed model is very close to the real effort, thus enhanced the reliability and improving the software effort estimation accuracy.

In this paper we present the results of our effort estimation analysis of a European Space Agency database consisting of 108 software development projects. We develop and evaluate simple empirical ...