Sparsity in function and derivative approximation via the empirical feature space

Internet of autonomous unmanned vehicles (IAUV) is a global network of sensors, robots, and autonomous vehicles. Person re‐identification (Re‐ID) is an important intelligent transportation application in IAUV, which needs to be solved using artificial intelligence algorithms. In this study, a domain adaptation metric learning method embedded with structural information (called DAML‐ESI) is designed for person Re‐ID in IAUV. Due to the lack of labeling information in the target domain, DAML‐ESI realizes person Re‐ID with the help of the discriminative and structural information of pedestrian images of related domains. DAML‐ESI projects pedestrian images selected from different domains into a common metric space and establishes a discriminative metric learning model, which requires that the positive sample pair be mapped to a point, and the distance distribution of the negative sample pair be mapped to a fixed value. The projection matrix learned by DAML‐ESI is used to eliminate the distribution differences between different domains, and the distance metric is used to ensure that the learned metric learning model has strong discriminative ability in the metric space. To verify the effectiveness of DAML‐ESI, experimental comparisons are conducted on three person Re‐ID datasets, and DAML‐ESI achieves satisfactory recognition performance.

In this paper we deal with multiobjective linear and quadratic programming problem with uncertain information. So far in the field of statistical analysis and data mining, e.g., mean-variance portfolio problem, support vector machine and their varieties, we have encountered various kinds of quadratic and linear programming problems with multiple criteria. Moreover coefficients in such problems have uncertainty that is expressed by interval, probabilistic distribution or possibilistic (fuzzy) distribution. In this paper, we define a robust basis for all possible perturbation of coefficients within intervals in objective functions and constraints that is regarded as secure and conservative solution under uncertainty. According to the conventional multi-objective programming literature, it is required to solve test subproblem for each basis. Therefore, in case of our interval problem excessive computational demand is estimated. In this paper investigating the properties of robust basis by means of combination of interval extreme points we obtained the result that the robust basis can be examined by working with only a finite subset of possible perturbations of the coefficients

Adaptive online extreme learning machine by regulating forgetting factor by concept drift map

This chapter provides a literature review of optimization problems in the context of grey system theory, as proposed by various authors. The chapter explains the binary interactive algorithm approach as a problem-solving method for linear programming and quadratic programming problems with uncertainty and a genetic-algorithm-based approach as a second problem-solving scheme for linear programming, quadratic programming, and general nonlinear programming problems with uncertainty. In the chapter, details on the computation procedures involved for solving the aforementioned optimization problems with uncertainty are presented and results from these two approaches are compared and contrasted. Finally, possible future work area in the subject is suggested.

Ship dynamic models serve as the foundation for designing ship controllers, trajectory planning, and obstacle avoidance. Support vector regression (SVR) is a commonly used nonparametric modelling method for ship dynamics. Achieving high accuracy SVR models requires a substantial amount of training samples. Additionally, as the number of training samples increases, the computational efficiency for solving the quadratic programming problem (QPP) of SVR decreases. Ship controllers demand dynamic models with both high accuracy and computational efficiency. Therefore, to enhance the prediction accuracy and computational efficiency of SVR, this paper proposes a nonparametric modelling method based on twin SVR (TSVR). TSVR replaces a large QPP with a set of smaller QPPs, significantly enhancing generalizability and computational efficiency. To further improve the predictive accuracy of TSVR, the puma optimizer algorithm is employed to determine the optimal hyperparameters. The performance of the proposed method is validated using a Mariner class vessel. Gaussian white noise is introduced into the modelling data to simulate measurement error. The TSVR model accurately predicts various zigzag and turning circle manoeuvring motions under disturbance conditions, demonstrating its robustness and generalizability. Compared to the SVR model, the TSVR model achieves lower root mean square error and computational time, confirming its superior predictive accuracy and computational efficiency.

Learning a high-dimensional dense representation for vocabulary terms, also known as a word embedding, has recently attracted much attention in natural language processing and information retrieval tasks. The embedding vectors are typically learned based on term proximity in a large corpus. This means that the objective in well-known word embedding algorithms, e.g., word2vec, is to accurately predict adjacent word(s) for a given word or context. However, this objective is not necessarily equivalent to the goal of many information retrieval (IR) tasks. The primary objective in various IR tasks is to capture relevance instead of term proximity, syntactic, or even semantic similarity. This is the motivation for developing unsupervised relevance-based word embedding models that learn word representations based on query-document relevance information. In this paper, we propose two learning models with different objective functions; one learns a relevance distribution over the vocabulary set for each query, and the other classifies each term as belonging to the relevant or non-relevant class for each query. To train our models, we used over six million unique queries and the top ranked documents retrieved in response to each query, which are assumed to be relevant to the query. We extrinsically evaluate our learned word representation models using two IR tasks: query expansion and query classification. Both query expansion experiments on four TREC collections and query classification experiments on the KDD Cup 2005 dataset suggest that the relevance-based word embedding models significantly outperform state-of-the-art proximity-based embedding models, such as word2vec and GloVe.

The static Young's Modulus (Estatic) is an important parameter affecting the design of different aspects related to oil and gas producing wells. It is significantly changing based on the type of the formation, and hence, an accurate method of identifying Estatic is required. This study evaluates the performance of support vector regression (SVR) for prediction of the Estatic. The SVR model was learned to evaluate the Estatic from the well logs of the bulk formation density in addition to compressional and shear transit time. It was learned and tested on 592 training datasets of the inputs and their corresponding Estatic, these datasets were obtained from a sandstone formation in Well-A. The learned SVR model was then validated on 38 data points from Well-B, the performance of the optimized SVR on predicting the Estatic for the validation data was also compared with these of the early optimized artificial neural networks (ANN) and functional neural networks (FNN). As a result, all machine learning models showed high precision in predicting the Estatic for the validation data where Estatic was estimated with average absolute percentage errors of 3.80%, 2.54, and 2.03% and correlation coefficients of 0.991, 0.997, and 0.999 using the optimized ANN, FNN, and SVR models, respectively. This result shows the high accuracy of the SVR on predicting the Estatic.

In this paper, division algebras are proposed as an elegant basis upon which to extend support vector regression (SVR) to multidimensional targets. Using this framework, a multitarget SVR called epsilon(Z)-SVR is proposed based on an epsilon-insensitive loss function that is independent of the coordinate system or basis used. This is developed to dual form in a manner that is analogous to the standard epsilon-SVR. The epsilon(H)-SVR is compared and contrasted with the least-square SVR (LS-SVR), the Clifford SVR (C-SVR), and the multidimensional SVR (M-SVR). Three practical applications are considered: namely, 1) approximation of a complex-valued function; 2) chaotic time-series prediction in 3-D; and 3) communication channel equalization. Results show that the epsilon(H)-SVR performs significantly better than the C-SVR, the LS-SVR, and the M-SVR in terms of mean-squared error, outlier sensitivity, and support vector sparsity.

A guide on the use of SVMs in pattern classification, including a rigorous performance comparison of classifiers and regressors. The book presents architectures for multiclass classification and function approximation problems, as well as evaluation criteria for classifiers and regressors. Features: Clarifies the characteristics of two-class SVMs; Discusses kernel methods for improving the generalization ability of neural networks and fuzzy systems; Contains ample illustrations and examples; Includes performance evaluation using publicly available data sets; Examines Mahalanobis kernels, empirical feature space, and the effect of model selection by cross-validation; Covers sparse SVMs, learning using privileged information, semi-supervised learning, multiple classifier systems, and multiple kernel learning; Explores incremental training based batch training and active-set training methods, and decomposition techniques for linear programming SVMs; Discusses variable selection for support vector regressors.

The performance of visual object tracking depends largely on the target appearance model. Benefited from the success of CNN in feature extraction, recent studies have paid much attention to CNN representation learning and feature fusion model. However, the existing feature fusion models ignore the relation between the features of different layers. In this paper, we propose a deep feature fusion model based on the siamese network by considering the connection between feature maps of CNN. To tackle the limitation of different feature map sizes in CNN, we propose to fuse different resolution feature maps by introducing de-convolutional layers in the offline training stage. Specifically, a top-down modulation is adopted for feature fusion. In the tracking stage, a simple matching operation between the fused feature of the examplar and search region is conducted with the learned model, which can maintain the real-time tracking speed. Experimental results show that, the proposed method obtains favorable tracking accuracy against the state-of-the-art trackers with a real-time tracking speed.

In recent years, the quick deployment of Machine Learning in Small and Medium-sized Enterprises (SMEs) has been delayed by the complexity of model design and the lack of skilled personnel. However, traditional Automated Machine Learning (AutoML) models often restart learning from scratch for each task, resulting in high computational costs and delayed optimization. Therefore, to address these issues, a meta-learning-enhanced AutoML with Preference-Based Optimization (MetaAutoMPO) framework was proposed. Initially, it integrates a meta-learning layer that extracts taskspecific meta-features and retrieves optimal model configurations from similar tasks before starting the training process. Subsequently, regression models were constructed using a Multilayer Perceptron (MLP) with hyperparameters tuned via Bayesian optimization. The learned models are then used as objective functions in a reference-point-based Nondominated Sorting Genetic Algorithm II (NSGA-II) to generate preference-aware process parameterizations. Finally, experimental validation demonstrates that MetaAutoMPO achieves <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\mathbf{3 5 \%}$</tex> faster convergence and improves productivity (<tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\uparrow 3.19 \%$</tex>) while reducing defect rates (<tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\downarrow$</tex> 2.15%) compared with conventional AutoML methods.

Abstract. Despite showing great success of applications in many commercial fields, machine learning and data science models generally show limited success in many scientific fields, including hydrology (Karpatne et al., 2017). The approach is often criticized for its lack of interpretability and physical consistency. This has led to the emergence of new modelling paradigms, such as theory-guided data science (TGDS) and physics-informed machine learning. The motivation behind such approaches is to improve the physical meaningfulness of machine learning models by blending existing scientific knowledge with learning algorithms. Following the same principles in our prior work (Chadalawada et al., 2020), a new model induction framework was founded on genetic programming (GP), namely the Machine Learning Rainfall–Runoff Model Induction (ML-RR-MI) toolkit. ML-RR-MI is capable of developing fully fledged lumped conceptual rainfall–runoff models for a watershed of interest using the building blocks of two flexible rainfall–runoff modelling frameworks. In this study, we extend ML-RR-MI towards inducing semi-distributed rainfall–runoff models. The meaningfulness and reliability of hydrological inferences gained from lumped models may tend to deteriorate within large catchments where the spatial heterogeneity of forcing variables and watershed properties is significant. This was the motivation behind developing our machine learning approach for distributed rainfall–runoff modelling titled Machine Induction Knowledge Augmented – System Hydrologique Asiatique (MIKA-SHA). MIKA-SHA captures spatial variabilities and automatically induces rainfall–runoff models for the catchment of interest without any explicit user selections. Currently, MIKA-SHA learns models utilizing the model building components of two flexible modelling frameworks. However, the proposed framework can be coupled with any internally coherent collection of building blocks. MIKA-SHA's model induction capabilities have been tested on the Rappahannock River basin near Fredericksburg, Virginia, USA. MIKA-SHA builds and tests many model configurations using the model building components of the two flexible modelling frameworks and quantitatively identifies the optimal model for the watershed of concern. In this study, MIKA-SHA is utilized to identify two optimal models (one from each flexible modelling framework) to capture the runoff dynamics of the Rappahannock River basin. Both optimal models achieve high-efficiency values in hydrograph predictions (both at catchment and subcatchment outlets) and good visual matches with the observed runoff response of the catchment. Furthermore, the resulting model architectures are compatible with previously reported research findings and fieldwork insights of the watershed and are readily interpretable by hydrologists. MIKA-SHA-induced semi-distributed model performances were compared against existing lumped model performances for the same basin. MIKA-SHA-induced optimal models outperform the lumped models used in this study in terms of efficiency values while benefitting hydrologists with more meaningful hydrological inferences about the runoff dynamics of the Rappahannock River basin.

In this chapter we present a method for learning a compositional model in a minimax entropy framework for modeling object categories with large intra-class variance. The model we learn incorporates the flexibility of a stochastic context free grammar (SCFG) to account for the variation in object structure with the neighborhood constraints of a Markov random field (MRF) to enforce spatial context. We learn the model through a generalized minimax entropy framework that accounts for the dynamic structure of the hierarchical model. We first learn the SCFG parameters using the frequencies of object parts, then pursue spatial relations in order of greatest information gain. The learned model can generalize from a small set of training samples (n < 100) to generate a combinatorially large number of novel instances using stochastic sampling. To verify our learning method and model performance, we present plots of KL divergence minimization as the algorithm proceeds, and show that samples from the model become more realistic as more spatial relations are added. We also show the model accurately predicting missing or undetected parts for top-down recognition along with preliminary results showing that the model can learn a large space of category appearances from a very small (n < 15) number of training samples. This process is similar to “recognition-by-components”, a theory that postulates that biological vision systems recognize objects as composed from a dictionary of commonly appearing 3D structures. Finally, we discuss a compositional boosting algorithm for inference and show examples using it for object recognition. This article is a chapter from the book Object Categorization: Computer and Human Vision Perspectives, edited by Sven Dickinson, Ales Leonardis, Bernt Schiele, and Michael J. Tarr (Cambridge University Press). University of California Los Angeles, Los Angeles, CA. Lotus Hill Research Institute, EZhou, China.

Based on the urgent need for agricultural intelligence in various northern countries, this study develops an algorithm for crop pest and disease identification based on the basic theory of convolutional neural networks (CNN). The existing related algorithms are basically based on platforms that can carry multiple cameras to work to (e.g., tractors or combine harvesters). Such algorithms are not applicable to cultivation scenarios such as terraces. Therefore, in this study, unlike other designs based on multiple nets, the learning model of this learning uses only a single net for the train dataset. The main research objective of this learning is to achieve satisfactory result using a simplified CNN model design. The main approach of this learning model is to use the convolutional layer in the model to learn the crop pest and disease features in the given images, then use the pooling layer to reduce the number of parameters, and finally use the Fully connected layer to classify them into two categories: diseased and healthy. After learning through the train dataset with a sample size of 7000, the test dataset was imported into the completed learned model for testing, and the result was an accuracy of 94.5% and a loss rate of 0.1385.

State-of-the-art 2D image compression schemes rely on the power of convolutional neural networks (CNNs). Although CNNs offer promising perspectives for 2D image compression, extending such models to omnidirectional images is not straightforward. First, omnidirectional images have specific spatial and statistical properties that can not be fully captured by current CNN models. Second, basic mathematical operations composing a CNN architecture, e.g., translation and sampling, are not well-defined on the sphere. In this paper, we study the learning of representation models for omnidirectional images and propose to use the properties of HEALPix uniform sampling of the sphere to redefine the mathematical tools used in deep learning models for omnidirectional images. In particular, we: i) propose the definition of a new convolution operation on the sphere that keeps the high expressiveness and the low complexity of a classical 2D convolution; ii) adapt standard CNN techniques such as stride, iterative aggregation, and pixel shuffling to the spherical domain; and then iii) apply our new framework to the task of omnidirectional image compression. Our experiments show that our proposed on-the-sphere solution leads to a better compression gain that can save 13.7% of the bit rate compared to similar learned models applied to equirectangular images. Also, compared to learning models based on graph convolutional networks, our solution supports more expressive filters that can preserve high frequencies and provide a better perceptual quality of the compressed images. Such results demonstrate the efficiency of the proposed framework, which opens new research venues for other omnidirectional vision tasks to be effectively implemented on the sphere manifold.