Improved Software Effort Estimation Through Machine Learning: Challenges, Applications, and Feature Importance Analysis

<i>Introduction</i>: The precise estimation of software effort is a significant difficulty that project managers encounter during software development. Inaccurate forecasting leads to either overestimating or underestimating software effort, which can be detrimental for stakeholders. The International Function Point Users Group&#x2019;s Function Point Analysis (FPA) method is one of the most critical methods for software effort estimation. However, the practice of using the FPA method in the same fashion across all software areas needs to be reexamined. <i>Aim</i>: We propose a model for evaluating the influence of data clustering on software development effort estimation and then finding the best clustering method. We call this model the effort estimation using machine learning applied to the clusters (EEAC) model. <i>Method</i>: We cluster the dataset according to the clustering method and then apply the FPA and EEAC methods to these clusters for effort estimation. The clustering methods we use in this study include five categorical variable criteria (Development Platform, Industrial Sector, Language Type, Organization Type, and Relative Size) and the k-means clustering algorithm. <i>Results</i>: The experimental results show that the estimation accuracy obtaining with clustering consistently outperforms the accuracy without clustering for both the FPA and EEAC methods. Significantly, using the FPA method, the average improvement rate from using clustering as opposed to non-clustered was highest at 58.06%, according to the RMSE. With the EEAC method, this number reached 65.53%. The Industry Sector categorical variable achieves the best accuracy estimation compared to the other clustering criteria and k-means clustering. The improvement in accuracy in terms of the RMSE when applying this criterion is 63.68% for the FPA method and 72.02% for the EEAC method. <i>Conclusion</i>: Better results are obtained through dataset clustering compared to no clustering for both the FPA and EEAC methods. The Industry Sector is the most suitable clustering method among the tested clustering methods.

Estimating the software size and effort helps in early prediction of uncertainties in software development. Determining the size helps in ascertaining the effort, cost, and schedule for entire project. In context of mobile software, the existing techniques for software effort estimations could be adapted as such or with modifications. There are many software size and effort estimation metrics. In this paper, Use Case Point (UCP) metric is aimed for estimating size and effort for mobile application. Five android mobile applications are considered as a case study, and difference in actual effort and estimated effort is evaluated. Modified UCP has been also proposed to improve the results by considering mobile-specific characteristics.

Software cost and effort estimation using dragonfly whale optimized multilayer perceptron neural network

Crowd-Sourced software development (CSSD) is getting a good deal of attention from the software and research community in recent times. One of the key challenges faced by CSSD platforms is the task selection mechanism which in practice, contains no intelligent scheme. Rather, rule-of-thumb or intuition strategies are employed, leading to biasness and subjectivity. Effort considerations on crowdsourced tasks can offer good foundation for task selection criteria but are not much investigated. Software development effort estimation (SDEE) is quite prevalent domain in software engineering but only investigated for in-house development. For open-sourced or crowdsourced platforms, it is rarely explored. Moreover, Machine learning (ML) techniques are overpowering SDEE with a claim to provide more accurate estimation results. This work aims to conjoin ML-based SDEE to analyze development effort measures on CSSD platform. The purpose is to discover development-oriented features for crowdsourced tasks and analyze performance of ML techniques to find best estimation model on CSSD dataset. TopCoder is selected as target CSSD platform for the study. TopCoder’s development tasks data with development-centric features are extracted, leading to statistical, regression and correlation analysis to justify features’ significance. For effort estimation, 10 ML families with 2 respective techniques are applied to get broader aspect of estimation. Five performance metrices (MSE, RMSE, MMRE, MdMRE, Pred (25) and Welch’s statistical test are incorporated to judge the worth of effort estimation model’s performance. Data analysis results show that selected features of TopCoder pertain reasonable model significance, regression, and correlation measures. Findings of ML effort estimation depicted that best results for TopCoder dataset can be acquired by linear, non-linear regression and SVM family models. To conclude, the study identified the most relevant development features for CSSD platform, confirmed by in-depth data analysis. This reflects careful selection of effort estimation features to offer good basis of accurate ML estimate.

Background: Developing and maintaining a software effort estimation (SEE) data set within a company (within data) is costly. Often times parts of data may be missing or too difficult to collect, e.g. effort values. However, information about the past projects-although incomplete- may be helpful, when incorporated with the SEE data sets from other companies (cross data).Aim: Utilizing cross data to aid within company estimates and local experts; Proposing a synergy between semi-supervised, active and cross company learning for software effort estimation.Method: The proposed method: 1) Summarizes existing unlabeled within data; 2) Uses cross data to provide pseudo-labels for the summarized within data; 3) Uses steps 1 and 2 to provide an estimate for the within test data as an input for the local company experts. We use 21 data sets and compare the proposed method to existing state-of-the-art within and cross company effort estimation methods subject to evaluation by 7 different error measures.Results: In 132 out of 147 settings (21 data sets X 7 error measures = 147 settings), the proposed method performs as well as the state-of-the-art methods. Also, the proposed method summarizes the past within data down to at most 15% of the original data.Conclusion: It is important to look for synergies amongst cross company and within-company effort estimation data, even when the latter is imperfect or sparse. In this research, we provide the experts with a method that: 1) is competent (performs as well as prior within and cross data estimation methods) 2) reflects on local data (estimates come from the within data); 3) is succinct (summarizes within data down to 15% or less); 4) cheap (easy to build).

Accurate software effort estimation is one of the key factors to a successful project by making a better software project plan. To improve the estimation accuracy of software effort, many studies usually aimed at proposing novel effort estimation methods or combining several approaches of the existing effort estimation methods. However, those researches did not consider the distribution of historical software project data which is an important part impacting to the effort estimation accuracy. In this paper, to improve effort estimation accuracy by least squares regression, we propose a data partitioning method by the accuracy measures, MRE and MER which are usually used to measure the effort estimation accuracy. Furthermore, the empirical experimentations are performed by using two industry data sets (the ISBSG Release 9 and the Bank data set which consists of the project data performed in a bank in Korea).

Probabilistic estimation of software size and effort

GA-based method for feature selection and parameters optimization for machine learning regression applied to software effort estimation

Enhanced Software Effort Estimation Using Multi Layered Feed Forward Artificial Neural Network Technique

An effective method for finding the relevant number (window size) and the elapsed time (window age) of recently completed projects has proven elusive in software effort estimation. Although these two parameters significantly affect the prediction accuracy, there is no effective method to stratify and sample chronological projects to improve prediction performance of software effort estimation models. Exemplary projects (Bellwether) representing the training set have been empirically validated to improve the prediction accuracy in the domain of software defect prediction. However, the concept of Bellwether and its effect have not been empirically proven in software effort estimation as a method of selecting exemplary/relevant projects with defined window size and age. In view of this, we introduce a novel method for selecting relevant and recently completed projects referred to as Bellwether moving window for improving the software effort prediction accuracy. We first sort and cluster a pool of N projects and apply statistical stratification based on Markov chain modeling to select the Bellwether moving window. We evaluate the proposed approach using the baseline Automatically Transformed Linear Model on the ISBSG dataset. Results show that (1) Bellwether effect exist in software effort estimation dataset, (2) the Bellwether moving window with a window size of 82 to 84 projects and window age of 1.5 to 2 years resulted in an improved prediction accuracy than the traditional approach.

Data Mining is a method of finding patterns from vast quantities of data and information. The data sources include databases, data centers, the internet, and other data storage forms; or data that is dynamically streaming into the network. Estimation of effort is very important in the cost estimation of a software development project, and very critical in the software life development cycle planning process. This paper offers a description of the latest data mining techniques used in estimating software effort, and these techniques are divided into two, namely: Classical and Modern, based on when they were developed and when they started to be used in business administration. The Classical techniques are the ones that have been in use for decades and are still relevant until today, while the Modern ones are the ones that have been introduced recently and have gained wide acceptance in the system. The Classical techniques are Statistical methods, Nearest Neighbours, Clustering and Regression Analysis, while Neural Networks, Rule Induction Systems and Decision Trees are included in the Modern techniques. This paper offers an overview of these strategies in terms of their features, benefits, drawbacks and use areas.

Prediction of software development is the key task for the effective management of any software industry. The accuracy and reliability of the prediction mechanisms used for the estimation of software development effort is also important. A series of experiments are conducted to gradually progress towards the improved accurate estimation of the software development effort. However, while conducting these experiments, it was found that the size of the training set was not sufficient to train a large and complex artificial neural network (ANN). To overcome the problem of the size of the available training data set, a novel multilayered architecture based on a neural network model is proposed. The accuracy of the proposed multi-layered model is assessed using different criteria, which proves the pre-eminence of the proposed model.

Conclusion instability is the absence of observing the same effect under varying experimental conditions. Deep Neural Network (DNN) and ElasticNet software effort estimation (SEE) models were applied to two SEE datasets with the view of resolving the conclusion instability issue and assessing the suitability of ElasticNet as a viable SEE benchmark model. Results were mixed as both model types attain conclusion stability for the Kitchenham dataset whilst conclusion instability existed in the Desharnais dataset. ElasticNet was outperformed by DNN and as such it is not recommended to be used as a SEE benchmark model.

Software development effort estimation (SDEE) generally involves leveraging the information about the effort spent in developing similar software in the past. Most organizations do not have access to sufficient and reliable forms of such data from past projects. As such, the existing SDEE methods suffer from low usage and accuracy. We propose an efficient SDEE method for open source software, which provides accurate and fast effort estimates. The significant contributions of our article are (i) novel SDEE software metrics derived from developer activity information of various software repositories, (ii) an SDEE dataset comprising the SDEE metrics’ values derived from approximately 13,000 GitHub repositories from 150 different software categories, and (iii) an effort estimation tool based on SDEE metrics and a software description similarity model . Our software description similarity model is basically a machine learning model trained using the PVA on the software product descriptions of GitHub repositories. Given the software description of a newly envisioned software, our tool yields an effort estimate for developing it. Our method achieves the highest standardized accuracy score of 87.26% (with Cliff’s δ = 0.88 at 99.999% confidence level) and 42.7% with the automatically transformed linear baseline model. Our software artifacts are available at https://doi.org/10.5281/zenodo.5095723.

Software cost and time estimation is the process of estimating the cost and time required to develop a software system. Software cost and time estimation supports the planning and tracking of software projects. Effectively controlling the expensive investment of software development is one of the important issues in software project management. Estimating software development cost with high precision is still a great challenge for project managers, because it allows for considerable financial and strategic planning. Software cost estimation refers to the predictions of the likely amount of effort, time, and staffing levels required to build a software system. A very helpful form of cost estimation is the one made at an early stage during a project, when the costing of the project is proposed for approval. However, estimates at the early stages of the development are the most difficult to obtain. In this paper a novel Constructive Cost Model (COCOMO) based on soft computing approach is proposed for software cost estimation. This model carries some of the desirable features of neural networks approach, such as learning ability and good interpretability, while maintaining the merits of the COCOMO model. Unlike the standard neural networks approach, the proposed model can be interpreted and validated by experts, and has good generalisation capability. The model deals effectively with imprecise and uncertain input and enhances the reliability of software cost estimates. From the experimental results, it was concluded that, by the proposed neural network model, the accuracy of cost estimation can be improved and the estimated cost can be very close to the actual cost.