Regression models for functional data by reproducing kernel Hilbert spaces methods

Background: Nowadays, forecasting models based on nonparametric models have been developed in many branches of science such as mathematics and economics. However, the relatively complicated structure of these models has made them less practical in medical sciences. Objectives: In this study, we investigated the application of a nonparametric regression model to predict the psychological symptoms appearing six months after mild traumatic brain injury. We also made a comparison between the performance of nonparametric regression and artificial neural network models in predicting the psychological symptoms. Methods: In a six-month period during 2015 - 2017, information of 100 mild traumatic patients was included in a prospective cohort study. The data were then divided randomly into two groups of training (n = 50) and testing (n = 50) for 100 times. In the training group, the focus was on 100 artificial neural network and nonparametric regression models. However, in the testing group, a comparison was made between the values obtained using the two final models. To compare the models, the ROC curve and the accuracy rate were finally applied. Results: According to the obtained results, the nonparametric regression model showed an accuracy rate of 91.25% while the neural network model had an accuracy rate of 85.34%. In the experimental set for both neural network and nonparametric regression models, the area under the ROC curve appeared to be 81.51% and 85.73 %, respectively. Conclusions: The nonparametric regression model appeared to be more powerful than the neural network model in predicting psychological symptoms.

ملخص:- تحتوي نماذج الانحدار على مکونين أحدهما مکون عشوائي يمثل حد الخطأ (Error Term) والأخر مکون غير عشوائي يمثل متغير الاستجابة(Response Variable) وهذا المکون هو محل الاهتمام فيتم تقديره بإحدى الطريقتين هما: الطريقة المعلمية أو الطريقة اللامعلمية أو کلاهما. نموذج الانحدار المعلمي (Parametric Regression Model) هو النموذج الأکثر شيوعا حيث أنه يفترض أن متغير الاستجابة له صيغه دالية محدده مسبقا والتي تم الحصول عليها من معلومات سابقه حول هيکل دالة الانحدار من خلال ذلک يتم تقدير فئة من المعالم. ورغم أن النماذج المعلمية تعد أحد أهم أدوات تحليل البيانات؛ لأنها فعاله ويمکن تفسيرها بسهوله، إلا انها لم تستطع ان تکون کافية في بعض الحالات. أما نموذج الانحدار اللامعلمي (Nonparametric Regression Model) فانه يعتمد على تقدير دالة الانحدار مباشرة من البيانات کما أنه لا يفترض فروضا معينه لتقدير معالم النموذج مثل التي يفترضها النموذج المعلمي وبالتالي فهو يتمتع بمرونة أکثر لاکتشاف البيانات التي قد تکون مفقودة او في الحالات التي لا تتوافر فيها معلومات سابقة. يتم انشاء نموذج الانحدار شبه المعلمي (Semi parametric Regression Model) من خلال مزج الطريقتين السابقتين معا. فغالبا ما يتم استخدام هذا النموذج في المواقف التي تکون فيها الافتراضات المعلمية غير محددة وغير متسقة، أو أن النموذج اللامعلمي لا يعمل بشکل کامل. يشتمل هذا البحث على تطبيق نماذج الانحدار المعلمية وشبه المعلمية بالإضافة إلى النموذج اللامعلمى کحالة خاصة من نموذج الانحدار شبه المعلمى لتقدير الدالة الافتراضية وتحديد شکل العلاقة بين المتغير التابع والمتغيرات المستقلة بالاضافة الى شکل انتشار البيانات مع خط (أو منحني) التقدير وذلک على بيانات واقعية تم الحصول عليها من الإتحاد المصري لألعاب القوى والمعروفة بـبيانات لعبة العشاري . Regression models contain two components, one of which is a random component that represents the Error Term and the other is a non-random component that represents the response variable and this component is of interest and is estimated in one of two methods: the parametric method or the nonparametric method or both. Parametric regression model is the most common model since it assumes that the response variable has a predetermined functional formula which was obtained from previous information about the structure of the regression function by which a class of parameters is estimated. Although parameter models are one of the most important tools for analyzing data; Because it is effective and can be explained easily, but it could not be sufficient in some cases. As for the nonparametric regression model, it relies on estimating the regression function directly from the data, as it does not assume specific assumptions for estimating the model parameters, such as those assumed by the parametric model, and thus it has more flexibility to discover data that may be missing or in cases where previous information is not available. The semi-parametric regression model is constructed by blending the two previous methods together. This model is often used in situations where the parametric assumptions are indeterminate and inconsistent, or the nonparametric model is not fully functional. This research includes the application of parametric and semi-parametric regression models in addition to the nonparametric model as a special case of the semi-parametric regression model for estimating the hypothetical function and determining the form of the relationship between the dependent variable and the independent variables in addition to the data spread form with a line (or curve) estimation on realistic data On it from the Egyptian Athletics Federation, known as Decathlon data.

In major domestic cities, the development of urban expressway is network oriented. The traffic flow forecasting system is the important prerequisite and foundation for realizing real-time traffic management and control. However, the traffic flow forecasting research is mainly based on highways. Research and application of short-term traffic forecasting for urban expressway is severely insufficient. Therefore, the study of urban expressway flow forecasting is discussed and a short-term traffic flow forecasting system for urban expressway based on k-NN nonparametric regression model is proposed in this study. First, the study analyzes the characteristics and needs of the urban expressway traffic flow, introduces the k-NN nonparametric regression model, and designs the short-term traffic flow forecasting system based on k-NN overall. Then, the short-term urban expressway flow forecasting system based on k-NN is established in three aspects: the historical database, the search mechanism and algorithm parameters, and the forecasting plan. Finally, a short-term traffic forecasting for urban expressway based on k-NN nonparametric regression model is developed in the VS2010 VC++ platform. Utilizing the Shanghai urban expressway section measured traffic flow data, the comparison of average and weighted k-NN nonparametric regression model is discussed and the reliability of the forecasting result is analyzed. Results show that the accuracy of the proposed method, under the five-minute interval, is over 90%, which best proves the reasonableness of the proposed forecasting model based on the k-NN nonparametric model.

In genome-enabled prediction, parametric, semi-parametric, and non-parametric regression models have been used. This study assessed the predictive ability of linear and non-linear models using dense molecular markers. The linear models were linear on marker effects and included the Bayesian LASSO, Bayesian ridge regression, Bayes A, and Bayes B. The non-linear models (this refers to non-linearity on markers) were reproducing kernel Hilbert space (RKHS) regression, Bayesian regularized neural networks (BRNN), and radial basis function neural networks (RBFNN). These statistical models were compared using 306 elite wheat lines from CIMMYT genotyped with 1717 diversity array technology (DArT) markers and two traits, days to heading (DTH) and grain yield (GY), measured in each of 12 environments. It was found that the three non-linear models had better overall prediction accuracy than the linear regression specification. Results showed a consistent superiority of RKHS and RBFNN over the Bayesian LASSO, Bayesian ridge regression, Bayes A, and Bayes B models.

The paper studies stochastic optimization problems in Reproducing Kernel Hilbert Spaces (RKHS). The objective function of such problems is a mathematical expectation functional depending on decision rules (or strategies), i.e. on functions of observed random parameters. Feasible rules are restricted to belong to a RKHS. This kind of problems arises in on-line decision making and in statistical learning theory. We solve the problem by sample average approximation combined with Tihonov's regularization and establish sufficient conditions for uniform convergence of approximate solutions with probability one, jointly with a rule for downward adjustment of the regularization factor with increasing sample size.

In this article, the complete convergence for the maximum of weighted sums of widely negative orthant dependent (WNOD, in short) random variables is investigated under some suitable moment conditions. Some sufficient conditions to prove the complete convergence are provided. The results obtained in the paper generalize some corresponding ones for some dependent random variables. As an application, the complete consistency for the weighted estimator in a nonparametric regression model is established. Finally, we present some simulations to show the consistency for the nearest neighbour weight function estimator in a nonparametric regression model.

Compositional data arise in many real-life applications and versatile methods for properly analyzing this type of data in the regression context are needed. When parametric assumptions do not hold or are difficult to verify, non-parametric regression models can provide a convenient alternative method for prediction. To this end, we consider an extension to the classical k- Nearest Neighbours (k-NN) regression, that yields a highly flexible non-parametric regression model for compositional data. A similar extension of kernel regression is proposed by adopting the Nadaraya–Watson estimator. Both extensions involve a power transformation termed the alpha -transformation. Unlike many of the recommended regression models for compositional data, zeros values (which commonly occur in practice) are not problematic and they can be incorporated into the proposed models without modification. Extensive simulation studies and real-life data analyses highlight the advantage of using these non-parametric regressions for complex relationships between compositional response data and Euclidean predictor variables. Both the extended K-NN and kernel regressions can lead to more accurate predictions compared to current regression models which assume a, sometimes restrictive, parametric relationship with the predictor variables. In addition, the extended k-NN regression, in contrast to current regression techniques, enjoys a high computational efficiency rendering it highly attractive for use with large sample data sets.

This chapter reviews the literature on variable selection in nonparametric and semiparametric regression models via shrinkage. We highlight recent developments on simultaneous variable selection and estimation through the methods of least absolute shrinkage and selection operator (Lasso), smoothly clipped absolute deviation (SCAD), or their variants, but restrict our attention to nonparametric and semiparametric regression models. In particular, we consider variable selection in additive models, partially linear models, functional/varying coefficient models, single index models, general nonparametric regression models, and semiparametric/nonparametric quantile regression models.

This paper is about optimal estimation of the additive components of a nonparametric, additive isotone regression model. It is shown that asymp- totically up to first order, each additive component can be estimated as well as it could be by a least squares estimator if the other components were known. The algorithm for the calculation of the estimator uses backfitting. Conver- gence of the algorithm is shown. Finite sample properties are also compared through simulation experiments. In this paper we discuss nonparametric additive monotone regression models. We discuss a backfitting estimator that is based on iterative application of the pool adjacent violator algorithm to the additive components of the model. Our main result states the following oracle property. Asymptotically up to first order, each additive component is estimated as well as it would be (by a least squares estimator) if the other components were known. This goes beyond the classical finding that the estimator achieves the same rate of convergence, independently of the number of additive components. The result states that the asymptotic distribution of the estimator does not depend on the number of components. We have two motivations for considering this model. First of all we think that this is a useful model for some applications. For a discussion of isotonic additive regres- sion from a more applied point, see also Bacchetti (1), Morton-Jones et al. (32) and De Boer, Besten and Ter Braak (7). But our main motivation comes from statistical theory. We think that the study of nonparametric models with several nonparamet- ric components is not fully understood. The oracle property that is stated in this paper for additive isotone models has been shown for smoothing estimators in some other nonparametric models. This property is expected to hold if the estimation of the different nonparametric components is based on local smoothing where the lo- calization takes place in different scales. An example are additive models of smooth functions where each localization takes place with respect to another covariate. In Mammen, Linton and Nielsen (28) the oracle property has been verified for the lo- cal linear smooth backfitting estimator. As local linear estimators, also the isotonic least squares is a local smoother. The estimator is a local average of the response variable but in contrast to local linear estimators the local neighborhood is chosen � Research of this paper was supported by the Deutsche Forschungsgemeinschaft project MA

International audience

Tensor dimensionality reduction (TDR) is a hot research topic in machine learning, which learns data representations by preserving the original data structure while avoiding convert samples into vectors and solving the problem of the curse of dimensionality of tensor data. In the work, a novel TDR approach based on mode product and Hilbert–Schmidt Independence criterion (HSIC) is proposed. The contributions of authors' work is described as following: (1) HSIC measures the statistical correlation of two random variables. However, instead of measuring the statistical correlation of two random variables directly, HSIC first transforms the two random variables into two reproducing kernel Hilbert spaces (RKHSs), and then measures the statistical correlation of transformed random variables by using Hilbert–Schmidt operators between the two RKHSs. The exploitation of RKHS increases the flexibility and applicability of HSIC. Although HSIC is widely used in machine learning, the authors have not seen its application to dimensionality reduction (DR)(except for authors' previous work). (2) A novel HSIC‐based TDR approach is proposed, which first applies HSIC to capture statistical information of tensor data set for DR. The authors give the mathematical derivation of HSIC for tensor data and establish a framework of TDR based on HSIC, named HSIC‐TDR for short, which aims to improve the DR results of tensor by exploring and preserving the statistical information of original data set. (3) Furthermore, to solve the out‐of‐sample problem, the authors learn an explicit expression between the dimensionality‐reduced tensors and the higher‐dimensional tensors by introducing mode product to HSIC‐TDR. The experimental results between the proposed method and other state‐of‐the‐art algorithm on various datasets demonstrate the well performance of the proposed method.

In this paper, the complete convergence for maximum of weighted sums of widely negative orthant dependent (WNOD) random variables are investigated. Some sufficient conditions for the convergence are provided and a relationship between the weight and the boundary function is revealed. Additionally, a Marcinkiewicz-Zygmund type strong law of large number for weighted sums of WNOD random variables is obtained. The results obtained in this paper generalize some corresponding ones for independent and some dependent random variables. As an application, the strong consistency for the weighted estimator in a non-parametric regression model is established. MR(2010) Subject Classification: 60F15; 62G05.

Consider the nonparametric regression modelYni=g(tni) +εi,i= 1, 2, …,n, n≥ 1, whereεi, 1 ≤i≤n, are asymptotically negatively associated (ANA, for short) random variables. Under some appropriate conditions, the Berry-Esseen bound of the wavelet estimator ofg(⋅) is established. In addition, some numerical simulations are provided in this paper. The results obtained in this paper generalize some corresponding ones in the literature.

Nelson Goodman’s new riddle of induction forcefully illustrates a challenge that must be confronted by any adequate theory of inductive inference: provide some basis for choosing among alternative hypotheses that fit past data but make divergent predictions. One response to this challenge is to distinguish among alternatives by means of some epistemically significant characteristic beyond fit with the data. Statistical learning theory takes this approach by showing how a concept similar to Popper’s notion of degrees of testability is linked to minimizing expected predictive error. In contrast, formal learning theory appeals to Ockham’s razor, which it justifies by reference to the goal of enhancing efficient convergence to the truth. In this essay, I show that, despite their differences, statistical and formal learning theory yield precisely the same result for a class of inductive problems that I call strongly VC ordered, of which Goodman’s riddle is just one example.

A novel nonparametric regression model is developed for evaluating the covariate-specific accuracy of a continuous biological marker. Accurately screening diseased from nondiseased individuals and correctly diagnosing disease stage are critically important to health care on several fronts, including guiding recommendations about combinations of treatments and their intensities. The accuracy of a continuous medical test or biomarker varies by the cutoff threshold (c) used to infer disease status. Accuracy can be measured by the probability of testing positive for diseased individuals (the true positive probability or sensitivity, Se(c), of the test), and the true negative probability (specificity, Sp(c)) of the test. A commonly used summary measure of test accuracy is the Youden index, YI=max{Se(c)+Sp(c)-1:c∈ℝ}, which is popular due in part to its ease of interpretation and relevance to population health research. In addition, clinical practitioners benefit from having an estimate of the optimal cutoff that maximizes sensitivity plus specificity available as a byproduct of estimating YI. We develop a highly flexible nonparametric model to estimate YI and its associated optimal cutoff that can respond to unanticipated skewness, multimodality, and other complexities because data distributions are modeled using dependent Dirichlet process mixtures. Important theoretical results on the support properties of the model are detailed. Inferences are available for the covariate-specific Youden index and its corresponding optimal cutoff threshold. The value of our nonparametric regression model is illustrated using multiple simulation studies and data on the age-specific accuracy of glucose as a biomarker of diabetes.