Online Kernel Selection via Tensor Sketching

Online kernel selection in continuous kernel space is more complex than that in discrete kernel set. But existing online kernel selection approaches for continuous kernel spaces have linear computational complexities at each round with respect to the current number of rounds and lack sublinear regret guarantees due to the continuously many candidate kernels. To address these issues, we propose a novel hypothesis sketching approach to online kernel selection in continuous kernel space, which has constant computational complexities at each round and enjoys a sublinear regret bound. The main idea of the proposed hypothesis sketching approach is to maintain the orthogonality of the basis functions and the prediction accuracy of the hypothesis sketches in a time-varying reproducing kernel Hilbert space. We first present an efficient dependency condition to maintain the basis functions of the hypothesis sketches under a computational budget. Then we update the weights and the optimal kernels by minimizing the instantaneous loss of the hypothesis sketches using the online gradient descent with a compensation strategy. We prove that the proposed hypothesis sketching approach enjoys a regret bound of order O(√T) for online kernel selection in continuous kernel space, which is optimal for convex loss functions, where T is the number of rounds, and reduces the computational complexities at each round from linear to constant with respect to the number of rounds. Experimental results demonstrate that the proposed hypothesis sketching approach significantly improves the efficiency of online kernel selection in continuous kernel space while retaining comparable predictive accuracies.

We study kernel selection for online kernel learning, also known as online kernel selection which can be treated as a sequential decision problem and thus must balance the regret and the time complexity. Existing online kernel selection approaches via expert advice and classical adversarial bandit model can not meet the issue. In this work, we propose a novel grouped adversarial bandit solution to the problem. We first correspond each candidate kernel to a basic arm of an adversarial bandit problem. Then, all of the kernels are divided into several groups where each group is abstracted as a super arm. At each round, we choose a super arm and a basic kernel within the selected super arm, and make prediction by an online kernel learning algorithm. Besides, we introduce a Bernoulli random variable to decide whether to choose all of the rest super arms. Theoretical analysis shows the proposed approach balances the regret and the time complexity explicitly, which could enjoy better pseudo-regret and high probability regret bound than classical adversarial bandit model and lighter time complexity than expert advice model. Experimental results on benchmark datasets verify that the proposed approach balances the efficiency and effectiveness better.

Deoxynivalenol (DON) in concentration range 0.1-156.6 mg/kg, was found in 48 samples of barley kernels collected from heads with visible symptoms of head scab after inoculation in a field with Fusarium culmorum and Fusarium graminearum. Four fractions of the kernels were collected according to their size: > 2.8 mm (I); < or = 2.8-2.5 mm (II); > 2.5-2.2 mm (III) and < 2.2 mm (IV). In plants infected with Fusarium (when compared to the control) number of kernels in the fraction of the largest kernels (> 2.8 mm) decreased from 38% to 18%, while in small kernels (fractions < 2.5-2.2 mm and < 2.2 mm) numbers increased respectively from 16 to 22% and from 14 to 28%. Average level of DON concentration in these fractions was as follows: (I) 3.5 mg/kg--(4%); (II) 9.1 mg/kg--(16%); (III) 35.5 mg/kg--(29%) and (IV) 43.3 mg/kg--(51%). The highest DON accumulation was observed in fraction of small (< 2.5 mm) kernels. The fraction contributed from 77% up to 94% of total DON content in kernels with low (< 2 mg/kg) and high (> 100 mg/kg) contamination level, respectively. These results indicate that as well level of samples contamination with DON as human and animal health risk in consequence can be reduced by kernels selection (according their size) followed by rejection of the smallest kernels fraction.

As an important part in network management system, rapid and accurate anomaly detection is one of the preconditions to guarantee the effective work of the network. In general, the network management system will monitor the data of multiple indicators, and the manifestations of network anomalies are complex in the data. Most existing anomaly detection approaches have encountered some trouble in network anomaly detection because of the periodicity of time series data or high time complexity and memory cost working on high-dimensional data. Aiming at the deficiency of present methods of network anomaly detection, we propose an adapted method based on iForest algorithm. We construct some feature extractors through statistical methods to highlight different abnormal behaviors in different indicator and then use the extracted feature data for the construction and prediction of iForest. Combining specific feature extractors, we can eliminate the effects of periodicity, or specify the detection of peaks or troughs to adapt to different indicators. By means of iForest algorithm which has linear time complexity and low memory requirement, our method makes a rapid detection with large dataset and works well in high dimensional network management data.

This paper considers online convex optimization (OCO) problems - the paramount framework for online learning algorithm design. The loss function of learning task in OCO setting is based on streaming data so that OCO is a powerful tool to model large scale applications such as online recommender systems. Meanwhile, real-world data are usually of extreme high-dimensional due to modern feature engineering techniques so that the quadratic regression is impractical. Factorization Machine as well as its variants are efficient models for capturing feature interactions with low-rank matrix model but they can't fulfill the OCO setting due to their non-convexity. In this paper, We propose a projective quadratic regression (PQR) model. First, it can capture the import second-order feature information. Second, it is a convex model, so the requirements of OCO are fulfilled and the global optimal solution can be achieved. Moreover, existing modern online optimization methods such as Online Gradient Descent (OGD) or Follow-The-Regularized-Leader (FTRL) can be applied directly. In addition, by choosing a proper hyper-parameter, we show that it has the same order of space and time complexity as the linear model and thus can handle high-dimensional data. Experimental results demonstrate the performance of the proposed PQR model in terms of accuracy and efficiency by comparing with the state-of-the-art methods.

Long-term time series forecasting (LTSF) is a critical task across diverse domains. Despite significant advancements in LTSF research, we identify a performance bottleneck in existing LTSF methods caused by the inadequate modeling of Temporal Dependencies within the Target (TDT). To address this issue, we propose a novel and generic temporal modeling framework, Temporal Dependency Alignment (TDAlign), that equips existing LTSF methods with TDT learning capabilities. TDAlign introduces two key innovations: 1) a loss function that aligns the change values between adjacent time steps in the predictions with those in the target, ensuring consistency with variation patterns, and 2) an adaptive loss balancing strategy that seamlessly integrates the new loss function with existing LTSF methods without introducing additional learnable parameters. As a plug-and-play framework, TDAlign enhances existing methods with minimal computational overhead, featuring only linear time complexity and constant space complexity relative to the prediction length. Extensive experiments on six strong LTSF baselines across seven real-world datasets demonstrate the effectiveness and flexibility of TDAlign. On average, TDAlign reduces baseline prediction errors by <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1.47%</b> to <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">9.19%</b> and change value errors by <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">4.57%</b> to <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">15.78%</b>, highlighting its substantial performance improvements.

The discovery of information encoded in biological sequences is assuming a prominent role in identifying genetic diseases and in deciphering biological mechanisms. This information is usually encoded in patterns frequently occurring in the sequences, also called motifs. In fact, motif discovery has received much attention in the literature, and several algorithms have already been proposed, which are specifically tailored to deal with motifs exhibiting some kinds of regular structure. Motivated by biological observations, this paper focuses on the mining of loosely structured motifs, i.e., of more general kinds of motif where several exceptions may be tolerated in pattern repetitions. To this end, an algorithm exploiting data structures conceived to efficiently handle pattern variabilities is presented and analyzed. Furthermore, a randomized variant with linear time and space complexity is introduced, and a theoretical guarantee on its performances is proven. Both algorithms have been implemented and tested on real data sets. Despite the ability of mining very complex kinds of pattern, performance results evidence a genome-wide applicability of the proposed techniques.

Low-rank decomposition meets kernel learning: A generalized Nyström method

Wrap&Zip decompression of the connectivity of triangle meshes compressed with Edgebreaker

Large-scale eigenvector approximation via Hilbert Space Embedding Nyström

The article introduces a method of finding optimal routes on raster planes. The method presented takes advantage of a new algorithm that tends to minimize a number of direction changes within a route, while steering clear of the obstacles. Two different schemes, suitable for restricted area Vessel Traffic Service (VTS) system and collision avoidance system located on the own ship are described. The VTS-oriented scheme supports VTS priority policy that may extend or override international give-way regulations. The own-ship routing scheme in a give-way situation is capable of determining the shortest safe path to the destination point. The method takes into account own ship dynamics. It has linear time and space complexities and therefore is sufficiently fast to perform real-time routing on the raster grids. Both the general method and the algorithm it uses are presented in detail in the paper. Implementation issues are also discussed.

Weighted median filters are increasingly being used in signal processing applications and thus fast implementations are of importance. This paper introduces a fast algorithm to compute the weighted median of N samples which has linear time and space complexity as opposed to O(N logN) which is the time complexity of traditional sorting algorithms. The proposed algorithm is based on Quickselect which is closely related to the well known Quicksort. We compare the runtime and the complexity to Floyd and Rivest's algorithm SELECT which to date has been the fastest median finding algorithm and show that our algorithm is up to 30% faster.

In this work, we formalize a generic fast hue-preserving histogram equalization method based on the RGB color space for image contrast enhancement and two versions of that generic process. The first method estimates a RGB 3D histogram to be equalized using R-red, G-green, and B-blue 1D histograms, while the second method employs RG, RB, and GB 2D histograms. The histogram equalization is performed using shift hue-preserving transformations, avoiding unrealistic colors. Our methods have linear time and space complexities with respect to the size of the image and do not need to apply conversions from a color space to another in order to perform the image enhancement. Such design complies with real-time applications requirements. An objective assessment comparing our methods and others is performed using a contrast measure and a color image quality measure, where the quality is established as a weighting of the naturalness and colorfulness indexes. We analyze 300 images from the dataset of the University of Berkeley. Experiments show that the value of the image contrast produced by our methods is in average 50% greater than the original image value, keeping the quality of the produced images close to the original one.

Algorithm selection and hyperparameter tuning are critical steps in both academic and applied machine learning (ML). These steps are becoming increasingly delicate due to the extensive rise in the number, diversity, and distributed nature of ML resources. Multi-agent systems, when applied to the design of ML platforms, bring about several distinctive characteristics, such as scalability, flexibility, and robustness, just to name a few. This article proposes a fully automatic and collaborative agent-based mechanism for selecting distributed ML algorithms and simultaneously tuning their hyperparameters. Our method builds upon an existing agent-based hierarchical ML platform and augments its query structure to support the aforementioned functionalities without being limited to specific learning, selection, and tuning mechanisms. We have conducted theoretical assessments, formal verification, and analytical study to demonstrate the correctness, resource utilization, and computational efficiency of our technique. According to the results, our solution is algorithmically correct and exhibits linear time and space complexity in relation to the size of available resources. To further verify its correctness and demonstrate its effectiveness and flexibility across a range of algorithmic options and datasets, the article also presents a series of empirical results on a system composed of 24 algorithms and 9 datasets. The findings not only highlight the efficiency and scalability of the proposed approach, but also show its flexibility and openness to responding to the dynamic and distributed ML ecosystem.

Text-to-Graph extraction aims to automatically extract information graphs consisting of mentions and types from natural language texts. Existing approaches, such as table filling and pairwise scoring, have shown impressive performance on various information extraction tasks, but they are difficult to scale to datasets with longer input texts because of their second-order space/time complexities with respect to the input length. In this work, we propose a Hybrid Span Generator (HySPA) that invertibly maps the information graph to an alternating sequence of nodes and edge types, and directly generates such sequences via a hybrid span decoder which can decode both the spans and the types recurrently in linear time and space complexities. Extensive experiments on the ACE05 dataset show that our approach also significantly outperforms state-of-the-art on the joint entity and relation extraction task.