Sketch Kernel Ridge Regression Using Circulant Matrix: Algorithm and Theory.

This paper focuses on Dynamic Implementation using Array and Linked list. In the data structure concept, the list plays a major role in the allocation of data where it is a general overview of creating a node in a linked list using an array with the initialization of size. The node contains data and is interconnected to the next node in the sequence via their links which perform the search operation where the space and time complexity are not wasted in these analyses. Linked lists are a great way to store a theoretically infinite amount of data with a small and versatile amount of code. The amount of data stored increases linearly with the number of nodes in the list (space-o(n)) where linked lists are more efficient than arrays. It is observed that considering the time complexity when there is a variation in data set linked array is almost reaches close to array whenever dataset is high also linked array has better time complexity than linked list when the searching element is in middle and last. Considering the space complexity linked array is almost 2 times and 4 times less space is needed than array and linked list respectively.

WITHDRAWN: Spearman rank correlation based bagging ensemble bayes optimal classification for genomic predictive pattern analytics

Binary embedding refers to methods for embedding points in $\mathbb {R}^{d}$ into vertices in the Hamming cube of dimension ${\mathcal{O}}(d)$ , such that the normalized Hamming distance between the codes preserves a prespecified distance between vectors in the original space. A common approach to binary embedding is to use random projection , followed by one-bit quantization to produce binary codes. Of particular interest, in this letter, is sparse circulant binary embedding (SCBE), where a sparse random circulant matrix with random sampling at a rate of $1-s$ for $s\in (0, 1)$ is used for random projection. The SCBE has the space complexity ${\mathcal{O}}((1-s)d)$ , while unstructured random projection has the space complexity ${\mathcal{O}}(d^2)$ . We present an asymptotic analysis of SCBE, when $d$ approaches $\infty$ , showing that the performance of SCBE is comparable to that of binary embedding with unstructured random projection whilethe former has the space complexity ${\mathcal{O}}((1-s)d)$ and the time complexity ${\mathcal{O}}(d\log d)$ but the latter has both space and time complexities ${\mathcal{O}}(d^2)$ .

The classic buffer insertion algorithm of van Ginneken has time and space complexity O(n/sup 2/), where n is the number of possible buffer positions. For more than a decade, van Ginneken's algorithm has been the foundation of buffer insertion. In this paper, we present a new algorithm that computes the same optimal buffer insertion, but runs much faster. For 2-pin nets, our time complexity is O(nlogn) and space complexity is O(n). For multipin nets, our time complexity is O(nlog/sup 2/n) and space complexity is O(nlogn). The speedup is achieved by four novel techniques: predictive pruning, candidate tree, fast redundancy check, and fast merging. On industrial test cases, the new algorithms is 2-80 times faster than van Ginneken's algorithm and uses 1/4-1/500 of the memory. Since van Ginneken's algorithm and its variations are used by most existing algorithms on buffer insertion and buffer sizing, our new algorithm significantly improves the performance of all these algorithms. The predictive pruning technique has been applied to buffer cost minimization (Shi et al., 2004), and significantly improved the running time.

BackgroundThe cellular function of a vast majority of proteins is performed through physical interactions with other biomolecules, which, most of the time, are other proteins. Peptides represent templates of choice for mimicking a secondary structure in order to modulate protein-protein interaction. They are thus an interesting class of therapeutics since they also display strong activity, high selectivity, low toxicity and few drug-drug interactions. Furthermore, predicting peptides that would bind to a specific MHC alleles would be of tremendous benefit to improve vaccine based therapy and possibly generate antibodies with greater affinity. Modern computational methods have the potential to accelerate and lower the cost of drug and vaccine discovery by selecting potential compounds for testing in silico prior to biological validation.ResultsWe propose a specialized string kernel for small bio-molecules, peptides and pseudo-sequences of binding interfaces. The kernel incorporates physico-chemical properties of amino acids and elegantly generalizes eight kernels, comprised of the Oligo, the Weighted Degree, the Blended Spectrum, and the Radial Basis Function. We provide a low complexity dynamic programming algorithm for the exact computation of the kernel and a linear time algorithm for it’s approximation. Combined with kernel ridge regression and SupCK, a novel binding pocket kernel, the proposed kernel yields biologically relevant and good prediction accuracy on the PepX database. For the first time, a machine learning predictor is capable of predicting the binding affinity of any peptide to any protein with reasonable accuracy. The method was also applied to both single-target and pan-specific Major Histocompatibility Complex class II benchmark datasets and three Quantitative Structure Affinity Model benchmark datasets.ConclusionOn all benchmarks, our method significantly (p-value ≤ 0.057) outperforms the current state-of-the-art methods at predicting peptide-protein binding affinities. The proposed approach is flexible and can be applied to predict any quantitative biological activity. Moreover, generating reliable peptide-protein binding affinities will also improve system biology modelling of interaction pathways. Lastly, the method should be of value to a large segment of the research community with the potential to accelerate the discovery of peptide-based drugs and facilitate vaccine development. The proposed kernel is freely available at http://graal.ift.ulaval.ca/downloads/gs-kernel/.

Kernel Machines, such as Kernel Ridge Regression, provide an effective way to construct non-linear, nonparametric models by projecting data into high-dimensional space and play an important role in machine learning. However, when dealing with large-scale problems, high computational cost in the prediction stage limits their use in real-world applications. In this paper, we propose hashing based prediction, a fast kernel prediction algorithm leveraging hash technique. The algorithm samples a small subset from the input dataset through the locality-sensitive hashing method and computes prediction value approximately using the subset. Hashing based prediction has the minimum time complexity compared to the state-of-art kernel machine prediction approaches. We further present a theoretical analysis of the proposed algorithm showing that it can keep comparable accuracy. Experiment results on most commonly used large-scale datasets, even with million-level data points, show that the proposed algorithm outperforms the state-of-art kernel prediction methods in time cost while maintaining satisfactory accuracy.

We initiate the systematic study of the energy complexity of algorithms (in addition to time and space complexity) based on Landauer's Principle in physics, which gives a lower bound on the amount of energy a system must dissipate if it destroys information. We propose energy-aware variations of three standard models of computation: circuit RAM, word RAM, and transdichotomous RAM. On top of these models, we build familiar high-level primitives such as control logic, memory allocation, and garbage collection with zero energy complexity and only constant-factor overheads in space and time complexity, enabling simple expression of energy-efficient algorithms. We analyze several classic algorithms in our models and develop low-energy variations: comparison sort, insertion sort, counting sort, breadth-first search, Bellman-Ford, Floyd-Warshall, matrix all-pairs shortest paths, AVL trees, binary heaps, and dynamic arrays. We explore the time/space/energy trade-off and develop several general techniques for analyzing algorithms and reducing their energy complexity. These results lay a theoretical foundation for a new field of semi-reversible computing and provide a new framework for the investigation of algorithms.

Minimum variance based-Bayes Combination for prediction of soil properties on Vis-NIR reflectance spectroscopy

A faster FPTAS for knapsack problem with cardinality constraint

Models built with deep neural network (DNN) can handle complicated real-world data extremely well, seemingly without suffering from the curse of dimensionality or the non-convex optimization. To contribute to the theoretical understanding of deep learning, this work studies the nonparametric perspective of DNNs by considering the following questions: (1) What is the underlying estimation problem and what are the most appropriate data assumptions? (2) What is the corresponding optimal convergence rate and does the curse of dimensionality occur? (3) Is the optimal rate achievable for DNN estimators and is there any optimization guarantee? These questions are investigated on two of the most fundamental problems --- regression and classification. Specifically, statistical optimality of DNN estimators is established under various settings with special focuses on the curse of dimensionality and optimization guarantee.In the classic binary classification problem, statistical optimal convergence rates that suffer less from the curse of dimensionality are established under two settings:(1) Under the smooth boundary assumption, I show that DNN classifiers with proper architectures can benefit from the compositional smoothness structure underlying the high dimensional data in the sense that the optimal convergence rates only depend on some effective dimension d*, potentially much smaller than the data dimension d. (2) Under a novel teacher-student framework that assumes the Bayes classifier to be expressed as ReLU neural networks, I obtain a dimension-free rate of convergence O(n^{-2/3}) for DNN classifiers, which is also proven optimal. The optimization of DNN is highly complicated with generally no algorithmic guarantee on finding the global minimizer. To this end, I turn to the recently proposed neural tangent kernel (NTK) literature where the similarity between overparametrized DNN trained by gradient descent (GD) and kernel methods are established. Specifically, through a comprehensive analysis of L2-regularized GD trajectories, I prove that for overparametrized one-hidden-layer ReLU neural networks with L2 regularization, the output from GD is close to that from the kernel ridge regression with the corresponding NTK and optimal rate of L2 estimation error can be achieved.

Optimal rates for coefficient-based regularized regression

This paper introduces a new heuristic function with high efficiency for an optimum solving of the 8 - puzzle, this one being the double of the Chebyshev distance. The comparative study is realized among this new heuristic (Chebyshev heuristic), the Hamming h euristic and the Manhattan heuristic using A* algorithm implemented in Java. The Chebyshev heuristic function is more informed than Hamming and Manhattan heuristics. This paper also presents the necessary stages in object oriented development of an interactive software dedicated to simulate the A* search algorithm for 8 -puzzle. The modeling of the software is achieved through specific UML diagrams representing the phases of analysis, design and implementation, the system thus being described in a clear and practical manner. In order to confirm the obtained theoretical results which show that Chebyshev heuristic is more efficient, two performance criteria were used: space complexity and time complexity. The space complexity was measured by the number of generated nodes from the search tree, by the number of the expanded nodes and by the effective branching factor. The time complexity was measured by the running time. From the experimental results obtained by using the Chebyshev heuristic, improvements were obs erved regarding space and time complexity of A* algorithm versus Hamming and Manhattan heuristics.

This paper proposes a novel learning algorithm- SVM based MLP neural network algorithm (SVMMLP), which based on the Maximal Margin (MM) principle and take into account the idea of support vectors. SVMMLP has time and space complexities O(N) while usual SVM training methods have time complexity O(N3) and space complexity O(N2), where N is the training-dataset size. Intrusion detection benchmark datasets – NSL-KDD used in experiments that enable a comparison with other state-of-the-art classifiers. The results provide evidence of the effectiveness of our methods regarding accuracy, AUC, and Balanced Error Rate (BER).KeywordsMLP neural networkconvergenceSVMMLPintrusion detection

Multiple kernel learning (MKL) is demonstrated to be flexible and effective in depicting heterogeneous data sources since MKL can introduce multiple kernels rather than a single fixed kernel into applications. However, MKL would get a high time and space complexity in contrast to single kernel learning, which is not expected in real-world applications. Meanwhile, it is known that the kernel mapping ways of MKL generally have two forms including implicit kernel mapping and empirical kernel mapping (EKM), where the latter is less attracted. In this paper, we focus on the MKL with the EKM, and propose a reduced multiple empirical kernel learning machine named RMEKLM for short. To the best of our knowledge, it is the first to reduce both time and space complexity of the MKL with EKM. Different from the existing MKL, the proposed RMEKLM adopts the Gauss Elimination technique to extract a set of feature vectors, which is validated that doing so does not lose much information of the original feature space. Then RMEKLM adopts the extracted feature vectors to span a reduced orthonormal subspace of the feature space, which is visualized in terms of the geometry structure. It can be demonstrated that the spanned subspace is isomorphic to the original feature space, which means that the dot product of two vectors in the original feature space is equal to that of the two corresponding vectors in the generated orthonormal subspace. More importantly, the proposed RMEKLM brings a simpler computation and meanwhile needs a less storage space, especially in the processing of testing. Finally, the experimental results show that RMEKLM owns a much efficient and effective performance in terms of both complexity and classification. The contributions of this paper can be given as follows: (1) by mapping the input space into an orthonormal subspace, the geometry of the generated subspace is visualized; (2) this paper first reduces both the time and space complexity of the EKM-based MKL; (3) this paper adopts the Gauss Elimination, one of the on-the-shelf techniques, to generate a basis of the original feature space, which is stable and efficient.

A number of algorithms are available for computing the simulation relation on Kripke structures and on labelled transition systems representing concurrentsystems. Among them, the algorithm by Ranzato and Tapparo~[2007] has the best time complexity, while the algorithm by Gentilini et al.~[2003]~--~successivelycorrected by van Glabbeek and Ploeger~[2008]~--~has thebest space complexity. Both space and time complexities are critical issues in a simulation algorithm, in particular memory requirements are crucial in the context of model checking when dealing with large state spaces.We propose here a new simulation algorithm thatis obtained as a space saving modification of the time efficient algorithm by Ranzato and Tapparo: a symbolic representation of sets is embedded in thisalgorithm so that any set of states manipulated by the algorithm can be efficiently stored as a set of blocks of a suitable state partition. It turns out that this new simulation algorithm retains a space complexity comparable with Gentilini et al.'s algorithm while improving on Gentilini et al.'s time bound.