The Fitting Optimization Path Analysis on Scale Missing Data: Based on the 507 Patients of Poststroke Depression Measured by SDS

Missing data are frequently encountered in various disciplines and can be divided into three categories: missing completely at random (MCAR), missing at random (MAR), and missing not at random (MNAR). Valid statistical approaches to missing data depend crucially on correct identification of the underlying missingness mechanism. Although the problem of testing whether this mechanism is MCAR or MAR has been extensively studied, there has been very little research on testing MAR versus MNAR. A critical challenge that is faced when dealing with this problem is the issue of model identification under MNAR. In this paper, under a logistic model for the missing probability, we develop two score tests for the problem of whether the missingness mechanism is MAR or MNAR under a parametric model and a semiparametric location model on the regression function. The implementation of the score tests circumvents the identification issue as it requires only parameter estimation under the null MAR assumption. Our simulations and analysis of human immunodeficiency virus data show that the score tests have well-controlled type I errors and desirable powers.

Background: PIn air quality studies, it is very often to have missing data due to reasons such as machine failure or human error. The approach used in dealing with such missing data can affect the results of the analysis. The main aim of this study was to review the types of missing mechanism, imputation methods, application of some of them in imputation of missing of PM10 and O3 in Tabriz, and compare their efficiency. Methods: Methods of mean, EM algorithm, regression, classification and regression tree, predictive mean matching (PMM), interpolation, moving average, and K-nearest neighbor (KNN) were used. PMM was investigated by considering the spatial and temporal dependencies in the model. Missing data were randomly simulated with 10, 20, and 30% missing values. The efficiency of methods was compared using coefficient of determination (R2 ), mean absolute error (MAE) and root mean square error (RMSE). Results: Based on the results for all indicators, interpolation, moving average, and KNN had the best performance, respectively. PMM did not perform well with and without spatio-temporal information. Conclusion: Given that the nature of pollution data always depends on next and previous information, methods that their computational nature is based on before and after information indicated better performance than others, so in the case of pollutant data, it is recommended to use these methods.

Leaf mass per area (LMA) and leaf nitrogen concentration (LNC) are both essential parameters in plant ecology, which can reflect the growth status of plants. The features of LMA and LNC can be captured using spectral reflectance in a remote sensing approach. While the relationships between spectra and leaf trait variance across different species with estimation performance are unclear, the development of assessment and transferable models to predicate LMA and LNC are prevented. Hence, we analyzed the variance of raw spectra and spectral data difference with four pretreated approaches (SG—Savitzky–Golay filter, SNV—Standard Normalized Variate, MSC—Multiplicative Scatter Correction analysis, and normalize), LMA, and LNC over six remote sensing datasets by a transfer component analysis (TCA) approach. Spectra combined with the Successive Projections Algorithm (SPA) were also presented to extract wavelengths with higher important coefficients to minimize the redundancy of datasets. The variance of normalized spectra between different datasets showed a minor degree of variance, and LNC spectra variance was decreased by the SPA. The results also showed that a smaller LMA and LNC variance is presented over different datasets when the trait values with higher distribution probabilities are close to each other. The LNC and LMA estimation performance in transfer models established by partial least squares regression (PLS), support vector regression (SVR), extreme gradient boosting (XGB), and random forest regression (RFR) algorithms across different datasets were employed, in which the RFR transfer models performed good prediction results. The relationships between spectra and leaf trait variance and estimation performance in RFR transfer models over different datasets were evaluated. LMA distance has a significant influence on estimation performance in the transfer model, and the variance of spectra with all pretreated approaches showed a very significant effect on LNC accession performance. Furthermore, we proposed a weight coefficient of spectral data updating combined with the TCA and RFR approach (WDT-RFR) transfer model to improve transferability between datasets and promote estimation performance in the transfer model. Compared to the RFR transfer model using spectra without updating, the root mean square error (RMSE) of the WDT-RFR transfer model with 5% samples transferred to estimate LMA and LNC increased by 7.9% and 4.8% on average, respectively. The estimation results showed that our transfer model showed a superior estimation performance.

BackgroundAdvanced prediction of the daily incidence of COVID-19 can aid policy making on the prevention of disease spread, which can profoundly affect people's livelihood. In previous studies, predictions were investigated for single or several countries and territories.ObjectiveWe aimed to develop models that can be applied for real-time prediction of COVID-19 activity in all individual countries and territories worldwide.MethodsData of the previous daily incidence and infoveillance data (search volume data via Google Trends) from 215 individual countries and territories were collected. A random forest regression algorithm was used to train models to predict the daily new confirmed cases 7 days ahead. Several methods were used to optimize the models, including clustering the countries and territories, selecting features according to the importance scores, performing multiple-step forecasting, and upgrading the models at regular intervals. The performance of the models was assessed using the mean absolute error (MAE), root mean square error (RMSE), Pearson correlation coefficient, and Spearman correlation coefficient.ResultsOur models can accurately predict the daily new confirmed cases of COVID-19 in most countries and territories. Of the 215 countries and territories under study, 198 (92.1%) had MAEs <10 and 187 (87.0%) had Pearson correlation coefficients >0.8. For the 215 countries and territories, the mean MAE was 5.42 (range 0.26-15.32), the mean RMSE was 9.27 (range 1.81-24.40), the mean Pearson correlation coefficient was 0.89 (range 0.08-0.99), and the mean Spearman correlation coefficient was 0.84 (range 0.2-1.00).ConclusionsBy integrating previous incidence and Google Trends data, our machine learning algorithm was able to predict the incidence of COVID-19 in most individual countries and territories accurately 7 days ahead.

In longitudinal studies, missing data are ubiquitous. This article made a comparison of three model-based techniques for handling different types of missing data (i.e., missing at random (MAR)-based maximum likelihood (ML) approach, missing not at random (MNAR)-based Diggle–Kenward (DK) selection model and MNAR-based pattern mixture (PM) model) in longitudinal studies through a Monte Carlo simulation study. Two influential factors were considered: the dropout rates (5%, 10%, 20%, and 40%) and the sample sizes (100, 300, 500, and 1000) under MAR and MNAR missingness mechanisms respectively. The results indicated that the model selection was a crucial issue when researchers were dealing with missing data in longitudinal studies because under MNAR mechanism, DK method outperformed MAR-based ML approach, but PM method performed worse than MAR-based method did. The differences of the parameter estimation among three methods became more significant as the sample size and the dropout rate increased.

We analytically derive the fixed-effects estimates in unconditional linear growth curve models by typical linear mixed-effects modelling (TLME) and by a pattern-mixture (PM) approach with random-slope-dependent two-missing-pattern missing not at random (MNAR) longitudinal data. Results showed that when the missingness mechanism is random-slope-dependent MNAR, TLME estimates of both the mean intercept and mean slope are biased because of incorrect weights used in the estimation. More specifically, the estimate of the mean slope is biased towards the mean slope for completers, whereas the estimate of the mean intercept is biased towards the opposite direction as compared to the estimate of the mean slope. We also discuss why the PM approach can provide unbiased fixed-effects estimates for random-coefficients-dependent MNAR data but does not work well for missing at random or outcome-dependent MNAR data. A small simulation study was conducted to illustrate the results and to compare results from TLME and PM. Results from an empirical data analysis showed that the conceptual finding can be generalized to other real conditions even when some assumptions for the analytical derivation cannot be met. Implications from the analytical and empirical results were discussed and sensitivity analysis was suggested for longitudinal data analysis with missing data.

To analyse and predict the healing time of upper limb fractures in children, machine learning algorithms were used. Paediatric orthopaedic data was obtained from the University Malaya Medical Centre. The data set includes radiographs of upper limb fractures involving the radius, ulna, and humerus in children under the age of twelve, with ages recorded from the date and time of initial injury. Inputs assessment included: age, gender, bone type, the number of bones involved, fracture type, angulation and the distance of the fracture. Random Forest (RF) and Support Vector Regression (SVR) algorithms were used to predict and identify variables associated with fracture healing time. Self Organizing Maps was then used for visualization and ordination of factors associated with healing time. Algorithms performance was measured using root mean square error (RMSE). A significant determinant in fracture healing includes age, bone part, fracture angulation, and distance. The Wilcoxon signed ranked test reported there is a significant difference between the prediction result of the SVR model (RMSE = 2.56) and the RF model (RMSE = 2.66). Predicting healing time can be used in the treatment process and follow up period for general practitioners and medical officers. The algorithm is deployed online at https://kidsfractureexpert.com/.

Missing data are uncollected data but meaningful for the statistical analysis due to clinical relevancy of the data for properly specified estimands in clinical trials. Meanwhile the efforts to prevent or minimize missing data are commonly applied in clinical trials, in practice, missing data still occurs. Choosing a statistical method for imputation that deals with missing data targeting specified estimands provides the more reliable estimates of treatment effects. We considered longitudinal clinical settings that have different degrees of missing data and treatment effects, and simulated different missing mechanisms using data from randomized, double-blind, placebo-controlled phase 3 confirmatory clinical trials of approved drugs. We compared four commonly used statistical methods to deal with missing data in clinical trials. We find that, when the data are missing not at random (MNAR) with higher missing rates, mixed model for repeated measurements (MMRM) method overestimates treatment difference. Pattern-mixture model estimates were seen to be more conservative in our studies than MMRM given MNAR assumptions, which are more realistic with missing datain clinical trials. We emphasize the importance of prevention of missing data and specifying the estimand based on trial objectives beforehand. The specified proper estimand and the proper statistical method might be key features to value the clinical trial results despite missing data.

BackgroundMissing data are a common problem in prospective studies with a long follow-up, and the volume, pattern and reasons for missing data may be relevant when estimating the cost of illness. We aimed to evaluate the effects of different methods for dealing with missing longitudinal cost data and for costing caregiver time on total societal costs in Alzheimer’s disease (AD).MethodsGERAS is an 18-month observational study of costs associated with AD. Total societal costs included patient health and social care costs, and caregiver health and informal care costs. Missing data were classified as missing completely at random (MCAR), missing at random (MAR) or missing not at random (MNAR). Simulation datasets were generated from baseline data with 10–40 % missing total cost data for each missing data mechanism. Datasets were also simulated to reflect the missing cost data pattern at 18 months using MAR and MNAR assumptions. Naïve and multiple imputation (MI) methods were applied to each dataset and results compared with complete GERAS 18-month cost data. Opportunity and replacement cost approaches were used for caregiver time, which was costed with and without supervision included and with time for working caregivers only being costed.ResultsTotal costs were available for 99.4 % of 1497 patients at baseline. For MCAR datasets, naïve methods performed as well as MI methods. For MAR, MI methods performed better than naïve methods. All imputation approaches were poor for MNAR data. For all approaches, percentage bias increased with missing data volume. For datasets reflecting 18-month patterns, a combination of imputation methods provided more accurate cost estimates (e.g. bias: −1 % vs −6 % for single MI method), although different approaches to costing caregiver time had a greater impact on estimated costs (29–43 % increase over base case estimate).ConclusionsMethods used to impute missing cost data in AD will impact on accuracy of cost estimates although varying approaches to costing informal caregiver time has the greatest impact on total costs. Tailoring imputation methods to the reason for missing data will further our understanding of the best analytical approach for studies involving cost outcomes.Electronic supplementary materialThe online version of this article (doi:10.1186/s12874-016-0188-1) contains supplementary material, which is available to authorized users.

Droughts are becoming more frequent in the karst region of southwest China due to climate change, and accurate monitoring of karst agricultural droughts is crucial. To this end, in this study, based on random forest (RF) and support vector regression (SVR) algorithms, the monthly precipitation, monthly potential evapotranspiration, monthly normalised difference vegetation Index (NDVI), elevation, and karst development intensity from January to December 2001–2020 were used as independent variables, and the standardised soil moisture index (SSI) calculated by GLDAS soil moisture was used as the dependent variable to construct karst agricultural drought monitoring models at different timescales, using Guizhou Province as an example. The performance of the models constructed by the two algorithms was also evaluated using root mean square error (RMSE), coefficient of determination (R2), and correlation analysis, and the spatial and temporal evolution trends of karst agricultural drought at different timescales were analysed based on the model with better performance. The prediction of karst agricultural drought from January to December 2021–2025 was based on the seasonal difference autoregressive moving average (SARIMA) model and the analysis of change trends was performed using the Bayesian estimator of abrupt change, seasonal change, and trend (RBEAST). The results showed that (1) the drought model constructed by the RF regression algorithm performed better than the SVR algorithm at 1-, 3-, 6-, 9-, and 12-month timescales and was superior for monitoring karst agricultural drought. (2) The model showed that the overall trend of agricultural drought at different timescales was alleviated; 2010, 2011, and 2012 were typical drought years. At the same time, most regions showed a trend of drought mitigation, whereas a few regions (Bijie City, Liupanshui City, and Qianxinan Prefecture) showed a trend of aggravation. (3) The study predicted an overall high west–east distribution of drought intensity by 2021–2025. The 1- and 3-month timescales showed a trend of agricultural drought mitigation, and the 6-, 9-, and 12-month timescales showed a trend of aggravation; in 2021, 2022, and 2024, the abrupt change rates of autumn and winter droughts were higher. The results can provide a reference basis for the monitoring of agricultural drought in karst agriculture and the formulation of drought prevention and anti-drought measures.

Estimating monthly average temperature by remote sensing in China

Lithology classification is crucial for efficient and sustainable resource exploration in the oil and gas industry. Missing values in well-log data, such as Gamma Ray (GR), Neutron Porosity (NPHI), Bulk Density (RHOB), Deep Resistivity (RS), Delta Time Compressional (DTCO), Delta Time Shear (DTSM), and Resistivity Deep (RD), significantly affect machine learning classification accuracy. This study applied three algorithms, extreme gradient boosting (XGBoost), K-nearest neighbours (KNN), and the artificial neural network (ANN), to handle missing values in well-log datasets, particularly datasets with extreme missing data (30 %). Results indicated that XGBoost was the most efficient and accurate, especially for RHOB, NPHI, DTCO, and DTSM, with the lowest Mean Absolute Percentage Error (MAPE) and Root Mean Square Error (RMSE) values. The ANN also performed effectively, particularly on the GR, RS, and RD features, after the use of preprocessing techniques such as isolation forest and bias correction. However, the ANN can suffer from overfitting and requires large datasets for optimal performance. In contrast, KNN struggled with missing-not-at-random (MNAR) data due to its reliance on the k parameter and distance metric, making it less effective in mapping missing data relationships.•Missing values in well-log data can hinder lithology classification accuracy for efficient resource exploration in the oil and gas industry.•This research aims to address the problem of missing values in well-log datasets by applying machine learning algorithms such as XGBoost, ANN, and KNN to enhance classification performance.•XGBoost demonstrated superior performance in handling extreme missing data (30 %) in well-log datasets. ANN was effective but prone to overfitting for small datasets, while KNN struggled with missing-not-at-random (MNAR) data due to limitations in its distance-based approach.

Incomplete data is a pervasive challenge across a wide range of data types, including tabular, sensor, time-series, image, and textual data. Its presence stems from various real-world factors and gives rise to different missingness mechanisms. While much of the existing research focuses on the Missing Completely At Random (MCAR) assumption, the more complex and realistic mechanisms-Missing At Random (MAR) and Missing Not At Random (MNAR)-remain relatively underexplored despite their prevalence and impact. This PhD project aims to systematically investigate the challenges posed by diverse Incomplete data mechanisms and to develop robust machine learning methods that can perform reliably across MCAR, MAR, and MNAR scenarios. The research spans multiple data modalities and focuses on improving both the theoretical understanding and practical handling of incomplete data. By addressing mechanism-specific imputation challenges and proposing broadly applicable solutions, this work contributes to building more resilient and trustworthy data-driven systems in real-world settings.

Artificial intelligence algorithms require large datasets for better performance for all kinds of tasks such as classification and regression. In this paper, we explore the potential of the Random Forest Regression (RFR) algorithm and Support Vector Machine (SVM) algorithm with minimum features, such as signal power and its variants, for Visible Light Communication (VLC) based indoor positioning. We explore the performance of the RFR algorithm and SVM by using variations of the received signal power to increase the accuracy and reduce the computation complexity. The simulation results demonstrate that both techniques have estimated the location with high accuracy, however, RFR outperforms SVM in terms of mean error.

The objective of this study was to develop a low-cost method for rice growth information obtained quickly using digital images taken with smartphone. A new canopy parameter, namely, the canopy volume parameter (CVP), was proposed and developed for rice using the leaf area index (LAI) and plant height (PH). Among these parameters, the CVP was selected as an optimal parameter to characterize rice yields during the growth period. Rice canopy images were acquired with a smartphone. Image feature parameters were extracted, including the canopy cover (CC) and numerous vegetation indices (VIs), before and after image segmentation. A rice CVP prediction model in which the CC and VIs served as independent variables was established using a random forest (RF) regression algorithm. The results revealed the following. The CVP was better than the LAI and PH for predicting the final yield. And a CVP prediction model constructed according to a local modelling method for distinguishing different types of rice varieties was the most accurate (coefficient of determination (R2) = 0.92; root mean square error (RMSE) = 0.44). These findings indicate that digital images can be used to track the growth of crops over time and provide technical support for estimating rice yields.