Model Selection by Linear Programming

With the help of 5G network, edge intelligence (EI) can not only provide distributed, low-latency, and high-reliable intelligent services, but also enable intelligent maintenance and management of smart city. However, the constantly changing available computing resources of end devices and edge servers cannot continuously guarantee the performance of intelligent inference. In order to guarantee the sustainability of intelligent services in smart city, we propose the Adaptive Model Selection and Partition Mechanism (AMSPM) in 5G smart city where EI provides services, which mainly consists of Adaptive Model Selection (AMS) and Adaptive Model Partition (AMP). In AMSPM, the model selection and partition of deep neural network (DNN) are formulated as an optimization problem. Firstly, we propose a recursive-based algorithm named AMS based on the computing resources of edge devices to derive an appropriate DNN model that satisfies the latency demand of intelligent services. Then, we adaptively partition the selected DNN model according to the computing resources of edge devices. The experimental results demonstrate that, when compared with state-of-the-art model selection and partition mechanisms, AMSPM not only reduces latency but also enhances computing resource utilization.

Accurate hydrological prediction faces challenges due to diverse datasets and the absence of universally applicable models. This study investigates meta-learning's role in identifying optimal models for hydrological time series forecasting in specific contexts. We explore limitations in model identification and introduce meta-learning as a solution. The proposed methodology formalizes the model selection process by using meta-features from a dataset that includes time series, geology, and the experimental configuration and results of predictions . These features are crucial for empowering the meta-learner to make informed model selections. The study encompasses localized and national analyses, offering insights into the meta-learner's performance across distinct geographic regions. In this initial phase of the experiment, we present the results of the Meta-Learners within the context of a localized evaluation in Ireland, we evaluated the Neagh Bann River Basin District ( NB RBD) which covers an area of around 5740 km2. It includes all of County Armagh, large parts of Counties Antrim, Londonderry, Down and Tyrone and a small area County Fermanagh in Ireland.  Using the NB RBD dataset which comprises 16 monitoring stations, we constructed the details of 12  prediction models.  The second phase of the experiment widens its scope by encompassing data from multiple locations across Ireland. This broader approach allows us to draw upon a more extensive and diverse dataset, comprehensively evaluating meta-learner performance within a national context. By incorporating data from various regions in Ireland, we aim to capture a more holistic understanding of the meta-learners' adaptability and effectiveness in a nationally diverse landscape.  We used data from 249 hydrometric stations in Ireland, each corresponding to a distinct geographical location. This comprehensive dataset encompasses various locations across the country, offering a rich and diverse perspective for our analysis. Evaluation results demonstrate the performance of 12 models. Models like Random Forest and K-Nearest Neighbors show promise, while Support Vector Machine struggles consistently. Addressing class imbalance through resampling techniques proves effective, underscoring the importance of tailored model selection strategies. Key findings highlight model performance variability, concerns about overfitting, and the significance of appropriate resampling strategies. The meta-learner's application showcases its value in leveraging strengths across classifiers and mitigating weaknesses. In conclusion, this study contributes to advancing adaptive model selection methodologies in hydrological time series forecasting.  Keywords: Meta-learning, model selection, hydrologic time-series prediction.

A prediction algorithm for time series based on adaptive model selection

Model selection: A Lagrange optimization approach

Most model selection procedures use a fixed penalty penalizing an increase in the size of a model. These nonadaptive selection procedures perform well only in one type of situation. For instance, Bayesian information criterion (BIC) with a large penalty performs well for “small” models and poorly for “large” models, and Akaike's information criterion (AIC) does just the opposite. This article proposes an adaptive model selection procedure that uses a data-adaptive complexity penalty based on a concept of generalized degrees of freedom. The proposed procedure, combining the benefit of a class of nonadaptive procedures, approximates the best performance of this class of procedures across a variety of different situations. This class includes many well-known procedures, such as AIC, BIC, Mallows's Cp, and risk inflation criterion (RIC). The proposed procedure is applied to wavelet thresholding in nonparametric regression and variable selection in least squares regression. Simulation results and an asymptotic analysis support the effectiveness of the proposed procedure.

This work deals with a class of nonlinear bilevel programming problems with nonconvex objective functions, in which the follower objective is a function of linear expression of all variables and the follower constraints are linear. For the leader functions, there are no any restrictions on the convexity as well as the differentiability. The distinguished feature of the problem is the nonconvexity of the follower objective function, which breaks through the barrier that the follower must be convex or concave in literature. First, for any fixed leader variable x, two linear programming are got from the follower and used to obtain the follower optimal solution y. In addition, in order to avoid solving directly the follower problem for each x, a real-binary encoding scheme is given which consists of x and the bases of two linear programming. Finally, a new crossover operator is designed based on the characteristics of the mixed encoding, and a novel genetic algorithm is proposed. The numerical results on 15 examples illustrate that the proposed algorithm is effective and stable.

In this paper we address the problem of dense depth map estimation from sparse noisy range data to reconstruct large heterogeneous outdoor scenes. We propose a surface inpainting solution through energy minimisation with an adaptive selection of surface regularisers among a set of well known convex and non-convex regularisers. In fact, the selection of norm is pivotal with respect to the intrinsic surface characteristics. Our goal is to show how dense interpolation of sparse range data can be leveraged of more exotic and non-convex regularisers such as the log and logTGV [1] which can better capture the scene geometry. In contrast to state of the art solutions, we do not restrict ourselves to this set of norms, instead we search for the most apt norm for each semantically segmented part of the scene. Our energy model selection use Bayesian optimisation to learn the best choice of free parameters. This results in an adaptive model selection and the generalisation of well studied regularisation norms. We conclude with a detailed experimental analysis of our approach using a basis of four norms over a set of challenging outdoor scenes.

Least squares techniques are commonly used to estimate model parameters from observed data. Often, such data do not become available all at the same time but is rather the result of successive measurements at periodic intervals. In such a case, it is desirable to have methods for updating the parameter estimates using new data as it becomes available. This is particularly important if the underlying model is not stationary. Many algorithms for performing such recursive estimation exist and are widely used in many branches of adaptive signal processing and control theory. At the same time, especially if the model is nonlinear, the number of potential parameters can be extremely large and it is necessary to have some means of picking only the most significant parameters, and discarding the rest. Within the last decade efficient procedures for carrying out such model selection have been developed and are increasingly being used in nonlinear system identification. They have also been applied to a certain class of neural networks to minimize the number of neurons required to accurately model a given problem. The aim of this paper is to combine the above two techniques in order to give an algorithm which can adaptively adjust its choice of model parameters in the light of new data. An example of the application of this algorithm to the adaptive selection of radial basis centres in the prediction of chaotic time series is given.

Consider the simple normal linear regression model for estimation/prediction at a new design point. When the slope parameter is not obviously nonzero, hypothesis testing and information criteria can be used for identifying the right model. We compare the performances of such methods both theoretically and empirically from different perspectives for more insight. The testing approach at the conventional size of 0.05, in spite of being the “standard approach,” performs poorly in estimation. We also found that the frequently told story “the Bayesian information criterion (BIC) is good when the true model is finite-dimensional, and the Akaike information criterion (AIC) is good when the true model is infinite-dimensional” is far from being accurate. In addition, despite some successes in the effort to go beyond the debate between AIC and BIC by adaptive model selection, it turns out that it is not possible to share the pointwise adaptation property of BIC and the minimax-rate adaptation property of AIC by any model selection method. When model selection methods have difficulty in selection, model combining is a better alternative in terms of estimation accuracy.This work was completed when the author was on leave from Iowa State University and was a New Direction Visiting Professor at the Institute for Mathematics and its Applications (IMA) at the University of Minnesota. The fundings from both IMA and ISU are greatly appreciated. The work was also partly supported by NSF CAREER grant DMS0094323. The author thanks Xiaotong Shen and Hannes Leeb for very helpful discussions. The paper also benefited from the questions and comments from the participants at the statistics seminars the author gave at the University of Minnesota and Duke University. The author is very grateful to the anonymous reviewers and the co-editor Benedikt Potscher for carefully reading earlier versions of the paper, bringing my attention to several closely related previous and current results, and making many very valuable suggestions, which significantly improved the paper in both content and presentation.

This paper presents an accurate distress classification method via adaptive Bayesian network model selection for maintenance inspection of road structures. The main contribution of this paper is adaptive selection of two Bayesian network models concerning classification performance. The proposed method trains a tag-based Bayesian network model based on inspection items and estimates its classification performance. Furthermore, for distresses that degrade the classification performance of the tag-based Bayesian network model, the proposed method trains another multifeature Bayesian network model based on inspection items and distress images. Consequently, the proposed method can adaptively select optimal Bayesian network models according to the estimated performance of the tag-based Bayesian network model. In actual maintenance inspection, distresses are generally classified either from inspection items alone or from both inspection items and visual information of distress images—i.e., distress classification has two patterns. Therefore the adaptive model selection approach is suitable for this classification scheme. Experimental results show that the proposed method outperforms several comparative methods and is suitable for actual maintenance inspection due to its low computation costs.

Real-time machine vision applications running on resource-constrained embedded systems face challenges for maintaining performance. An especially challenging scenario arises when multiple applications execute at the same time, creating contention for the computational resources of the system. This contention results in increase in inference delay of the machine vision applications, which can be unacceptable for time-critical tasks. To address this challenge, we propose an adaptive model selection framework that mitigates the impact of system contention and prevents unexpected increases in inference delay by trading off the application accuracy minimally. The framework has two parts, which are performed pre-deployment and at runtime. The pre-deployment part profiles the system for contention in a black-box manner and produces a model set that is specifically optimized for the contention levels observed in the system. The runtime part predicts the inference delays of each model considering the system contention and selects the best model according to the predictions for each frame. Compared to a fixed individual model with similar accuracy, our framework improves the performance by significantly reducing the inference delay violations against a specified threshold. We implement our framework on the Nvidia Jetson TX2 platform and show that our approach achieves greater than 20% reductions in delay violations over the individual baseline models.

Adaptive model selection for time series prediction in wireless sensor networks

Given $n$ independent replicates of a jointly distributed pair $(X, Y) \in \mathscr{R}^d \times \mathscr{R}$, we wish to select from a fixed sequence of model classes $\mathscr{F}_1, \mathscr{F}_2,\dots$ a deterministic prediction rule $f: \mathscr{R}^d \to \mathscr{R}$ whose risk is small.We investigate the possibility of empirically assessing the complexity of each model class, that is, the actual difficulty of the estimation problem within each class. The estimated complexities are in turn used to define an adaptive model selection procedure, which is based on complexity penalized empirical risk. The available data are divided into two parts. The first is used to form an empirical cover of each model class, and the second is used to select a candidate rule from each cover based on empirical risk. The covering radii are determined empirically to optimize a tight upper bound on the estimation error. An estimate is chosen from the list of candidates in order to minimize the sum of class complexity and empirical risk. A distinguishing feature of the approach is that the complexity of each model class is assessed empirically, based on the size of its empirical cover. Finite sample performance bounds are established for the estimates, and these bounds are applied to several nonparametric estimation problems. The estimates are shown to achieve a favorable trade-off between approximation and estimation error and to perform as well as if the distribution-dependent complexities of the model classes were known beforehand. In addition, it is shown that the estimate can be consistent, and even possess near optimal rates of convergence, when each model class has an infinite VC or pseudo dimension. For regression estimation with squared loss we modify our estimate to achieve a faster rate of convergence.

In this paper we propose an iterative algorithm for solving the problem of extracting a sparse source signal when a reference signal for the desired source signal is available. In the proposed algorithm, a nonconvex objective function is used for measuring the diversity (antisparsity) of the desired source signal. The nonconvex function is locally replaced by a quadratic convex function. This results in a simple iterative algorithm. The proposed algorithm has two different versions, depending on the measure of closeness between the extracted source signal and the reference signal. The proposed algorithm has useful applications to EEG/MEG signal processing. This is demonstrated by an example in which eye blink artifacts are automatically removed from a real EEG data.

Profit Allocation in the Global Supply Chain with Transfer Pricing and Exchange Rate