High Performance Hierarchical Tucker Tensor Learning Using GPU Tensor Cores

Nonconvex low-rank tensor approximation with graph and consistent regularizations for multi-view subspace learning

Despite the promise for performance and accuracy improvements of quantum inspired (QI) algorithms over classical machine learning (ML) algorithms, such gains have not been realized in practice. The quantum inspired algorithms can theoretically achieve significant speed up based on sampling assumptions and have thus far failed to outperform the existing classical ML models in practical applications. The speedup of quantum machine learning (QML) algorithms assume the access to data in quantum random access memory (QRAM) which is a strong assumption with current quantum architectures. QI algorithms assume sample and query (SQ) access to input vector and norms of matrices using a dynamic data structure. We explore the components of these models and the assumptions in this paper by surveying the recent works in QML and QI Machine learning (QIML) algorithms. We limit our study to QML and QIML models on achieving a speed up over classical ML techniques rather than individual proofs of these algorithms. This study highlights the assumptions being made that are currently not practical for QML and QIML algorithms in achieving performance advantage over classical ML algorithms.

Quantum Machine Learning (QML) is a new phraseology in the world today. With various upcoming Artificial Intelligence (AI) technologies, everyday data scientists and corporate around the world are trying to harness its power. QML borrows concepts of Quantum Computing and QML algorithms enhance the existing Machine Learning (ML) algorithms by processing datasets more efficiently. Quantum Computers are a powering force driving QML algorithms that boosts computational power and helps analyse data. QML algorithms can be applied in sectors involving number crunching, pattern identification and so on. In the existing world, QML can be a game-changer for organizations that work on identifying product correlations and customer behaviour in a brief time. This paper discusses studies related to principles of Quantum Mechanics, Quantum Computing and QML algorithm. Additionally, it explores opportunities and challenges faced in the field of QML. Lastly, the paper talks about QML applications and compares various available QML toolkits from the market. Future direction and exhaustive survey of papers from various spectrums related to Quantum Computing is presented in this paper.

With the advent of machine learning and its overarching pervasiveness it is imperative to devise ways to represent large datasets efficiently while distilling intrinsic features necessary for subsequent analysis. The primary workhorse used in data dimensionality reduction and feature extraction has been the matrix singular value decomposition (SVD), which presupposes that data have been arranged in matrix format. A primary goal in this study is to show that high-dimensional datasets are more compressible when treated as tensors (i.e., multiway arrays) and compressed via tensor-SVDs under the tensor-tensor product constructs and its generalizations. We begin by proving Eckart-Young optimality results for families of tensor-SVDs under two different truncation strategies. Since such optimality properties can be proven in both matrix and tensor-based algebras, a fundamental question arises: Does the tensor construct subsume the matrix construct in terms of representation efficiency? The answer is positive, as proven by showing that a tensor-tensor representation of an equal dimensional spanning space can be superior to its matrix counterpart. We then use these optimality results to investigate how the compressed representation provided by the truncated tensor SVD is related both theoretically and empirically to its two closest tensor-based analogs, the truncated high-order SVD and the truncated tensor-train SVD.

Emotion Recognition (ER) is an important capability that is increasingly being used to enhance the user experience in various fields such as, Human-Computer interaction, Cognitive Neuroscience, Neurofeedback, Robotics, Artificial Intelligence, Augmented Reality, Virtual Reality, Mental Health, etc. Emotions are generated in the limbic system, a set of brain structures that are involved in emotional processing, memory, and motivation. ER from electroencephalography (EEG) signals has gained attention from researchers in recent years as it provides a more accurate and non-intrusive way of understanding human emotions. This research paper presents a comprehensive study of the use of Quantum Support Vector Machines (QSVMs) and Quantum Neural Networks (QNNs) for ER using EEG signals. The aim of this study is to investigate the potential of quantum machine learning algorithms in improving the performance of ER tasks compared to classical Machine Learning (ML) algorithms. The model was simulated using IBM Qiskit due to the limited resources of quantum computers. Intriguingly, the experimental results contradict the speed-up claims of Quantum Computing (QC) by exhibiting longer computation times and lesser accuracy when compared to traditional methods. SVM and NN yield 68.5% and 69.2% accuracy while the quantum version gives 63% and 65.2% accuracy on the benchmark dataset SEED. This research provides evidence that Quantum Machine Learning (QML) algorithms need to be improved for ER from EEG signals and provide a reliable and non-intrusive way of understanding human emotions.

Network subspace clustering aims to uncover meaningful clusters of nodes by identifying low-dimensional subspaces embedded in complex and large-scale network data. Recent approaches have leveraged singular value decomposition and graph-boosting mechanisms to improve clustering performance; however, they often suffer from high computational complexity, sensitivity to noise, and the inability to capture nonlinear structural patterns. Moreover, existing methods fail to recover both the global low-rank structure of graphs and the concealed high-order relationships among nodes. To address these challenges, this paper proposes an efficient Network subspace Clustering framework using Low-rank Tensor singular value decomposition and Graph-Boosting (NCLTGB). Specifically, the tensor singular value decomposition module maps large-scale network data via a feature transition matrix into a clean low-rank feature space, effectively separating structural information from noise and retrieving high-quality latent features. After denoising and outlier elimination, a graph-boosting module—comprising an auxiliary graph and an enhanced graph—is employed to enforce long-range node relationships and strengthen global structural representations. Furthermore, NCLTGB incorporates a dual-guided pseudo-Siamese neural network to train the model, and the final subspace clustering is derived from the learned representations. Extensive experiments on multiple real-world datasets demonstrate that NCLTGB consistently outperforms advanced approaches, including MFK and SDAC-DA, in terms of both clustering accuracy and robustness.

Dynamic magnetic resonance imaging (dMRI) strikes a balance between reconstruction speed and image accuracy in medical imaging field. In this paper, an improved robust tensor principal component analysis (RTPCA) method is proposed to reconstruct the dynamic magnetic resonance imaging (MRI) from highly under-sampled K-space data. The MR reconstruction problem is formulated as a high-order low-rank tenor plus sparse tensor recovery problem, which is solved by robust tensor principal component analysis (RTPCA) with a new tensor nuclear norm (TNN). To further exploit the low-rank structures in multi-way data, the core matrix nuclear norm, extracted from the diagonal elements of the core tensor under tensor singular value decomposition (t-SVD) framework, is also integrated into TNN for enforcing the low-rank structure in MRI datasets. The experimental results show that the proposed method outperforms state-of-the-art methods in terms of both MR image reconstruction accuracy and computational efficiency on 3D and 4D experiment datasets, especially for 4D MR image reconstruction. Graphical abstract The flowchart of the proposed method to reconstruct the dynamic magnetic resonance imaging (MRI) from highly under-sampled K-space data in the kth iteration. To further exploit the low-rank structures in multi-way data, the core matrix nuclear norm, extracted from the diagonal elements of the core tensor under tensor singular value decomposition (t-SVD) framework, is also integrated into tensor nuclear norm (TNN) for enforcing the low-rank structure in MRI datasets. In each iteration, the first step is to get low-rank tensor ℓk - 1 by using soft thresholding on the singular values of ℓk - 1 = χk - 1 - ξk - 1, and an improved tensor nuclear norm method is proposed to process the low-rank tensor ℓk - 1 firstly. Then, the shrinkage operator is applied to ξk - 1 = χk - 1 - ℓk - 1 for sparse part ξk - 1. The final reconstructed d-MRI χk is obtained by enforcing data consistency that the residual in K-space is subtracted by the sum of the reconstructed low-rank tensor and sparse tensor.

In this work, we studied the use of Quantum Machine Learning (QML) algorithms for binary classification and compared their performance with classical Machine Learning (ML) methods. QML merges principles of Quantum Computing (QC) and ML, offering improved efficiency and potential quantum advantage in data-driven tasks and when solving complex problems. In binary classification, where the goal is to assign data to one of two categories, QML uses quantum algorithms to process large datasets efficiently. Quantum algorithms like Quantum Support Vector Machines (QSVM) and Quantum Neural Networks (QNN) exploit quantum parallelism and entanglement to enhance performance over classical methods. This study focuses on two common QML algorithms, Quantum Support Vector Classifier (QSVC) and QNN. We used the Qiskit software and conducted the experiments with three different datasets. Data preprocessing included dimensionality reduction using Principal Component Analysis (PCA) and standardization using scalers. The results showed that quantum algorithms demonstrated competitive performance against their classical counterparts in terms of accuracy, while QSVC performed better than QNN. These findings suggest that QML holds potential for improving computational efficiency in binary classification tasks. This opens the way for more efficient and scalable solutions in complex classification challenges and shows the complementary role of quantum computing.

At present, quantum computing and its applications are still in research. Nonetheless, the need to accelerate significantly computational processing that requires a considerable amount of time through classical computing for solving complex problems; are just a few reasons why quantum machine learning algorithms are being implemented in this field. Image classification is a frequent computer vision problems to solve using deep learning algorithms, evaluating their performance via well-known datasets. In this work, we compare the performance of the LeNet5 neural network with a quantum version of itself, in which a fixed non-trainable quantum circuit is used as a quanvolution kernel. The contribution of this work focuses on analyzing the disadvantages and advantages of a quanvolution kernel in image classification problems. The results show that using a quanvolutional layer achieves a favorable performance tradeoff over a classical CNN LeNet5 model. We used the MNIST hand-written digits dataset to perform the evaluation using well-known metrics such as accuracy, precision, F1 score, latency, throughput, and others.

<p style='text-indent:20px;'>In 2011, Kilmer and Martin proposed tensor singular value decomposition (T-SVD) for third order tensors. Since then, T-SVD has applications in low rank tensor approximation, tensor recovery, multi-view clustering, multi-view feature extraction, tensor sketching, etc. By going through the Discrete Fourier Transform (DFT), matrix SVD and inverse DFT, a third order tensor is mapped to an f-diagonal third order tensor. We call this a Kilmer-Martin mapping. We show that the Kilmer-Martin mapping of a third order tensor is invariant if that third order tensor is taking T-product with some orthogonal tensors. We define singular values and T-rank of that third order tensor based upon its Kilmer-Martin mapping. Thus, tensor tubal rank, T-rank, singular values and T-singular values of a third order tensor are invariant when it is taking T-product with some orthogonal tensors. Some properties of singular values, T-rank and best T-rank one approximation are discussed.</p>

Over the last couple of decades the demand for high precision and enhanced performance of physical systems has been steadily increasing. This demand often results in miniaturization and complex design, thus increasing the need for complex nonlinear control methods. Some of the state of the art nonlinear methods are stymied by the requirement of full state information, model and parameter uncertainties, mathematical complexity, etc. For many scenarios it is nearly impossible to consider all the uncertainties during the design of a feedback controller. Additionally, while designing a modelbased nonlinear control there is no standard mechanism to incorporate performance measures. Some of the mentioned issues can be addressed by using online learning. Animals and humans have the ability to share, explore, act or respond, memorize the outcome and repeat the task to achieve a better outcome when they encounter the same or a similar scenario. This is called learning from interaction. One instance of this approach is reinforcement learning (RL). However, RL methods are hindered by the curse of dimensionality, non-interpretability and non-monotonic convergence of the learning algorithms. This can be attributed to the intrinsic characteristics of RL, as it is a modelfree approach and hence no standard mechanism exists to incorporate a priori model information. In this thesis, learning methods are proposed which explicitly use the available system knowledge. This can be seen as a new class of approaches that bridge model-based and model-free methods. These methods can address some of the hurdles mentioned earlier. For example, i) a prior system information can speed up the learning, ii) new control objectives can be achieved which otherwise would be extremely difficult to attain using only model-based methods, iii) physical meaning can be attributed to the learned controller. The developed approach is as follows: themodel of the given physical system is represented in the port-Hamiltonian (PH) form. For the system dynamics in PH form a passivity-based control (PBC) law is formulated, which often requires the solution to a set of partial differential equations (PDEs). Instead of finding an analytical solution, the PBC control law is parameterized using an unknown parameter vector. Then, by using a variation of the standard actor-critic learning algorithm, the unknown parameters can be learned online. Using the principles of stochastic approximation theory, a proof of convergence for the developed method is shown. The proposedmethods are evaluated for the stabilization and regulation ofmechanical and electro-mechanical systems. The simulation and experimental results show comparable learning curves. In the final part of the thesis a novel integral reinforcement learning approach is developed to solve for the optimal output tracking control problem for a set of linear heterogeneous multi-agent systems. Unlike existing methods, this approach does not need to solve either the output regulator equation or requires a p-copy of the leader’s dynamics in the agent’s control law. A detailed numerical evaluation has been conducted to show the feasibility of the developed method.

Attempting to automatically learn to identify verb complements from natural language corpora without the help of sophisticated linguistic resources like grammars, parsers or treebanks leads to a significant amount of noise in the data. In machine learning terms, where learning from examples is performed using class-labelled feature-value vectors, noise leads to an imbalanced set of vectors: assuming that the class label takes two values (in this work complement/non-complement), one class (complements) is heavily underrepresented in the data in comparison to the other. To overcome the drop in accuracy when predicting instances of the rare class due to this disproportion, we balance the learning data by applying one-sided sampling to the training corpus and thus by reducing the number of non-complement instances. This approach has been used in the past in several domains (image processing, medicine, etc) but not in natural language processing. For identifying the examples that are safe to remove, we use the value difference metric, which proves to be more suitable for nominal attributes like the ones this work deals with, unlike the Euclidean distance, which has been used traditionally in one-sided sampling. We experiment with different learning algorithms which have been widely used and their performance is well known to the machine learning community: Bayesian learners, instance-based learners and decision trees. Additionally we present and test a variation of Bayesian belief networks, the COr-BBN (Class-oriented Bayesian belief network). The performance improves up to 22% after balancing the dataset, reaching 73.7% f-measure for the complement class, having made use only a phrase chunker and basic morphological information for preprocessing.

The deep computation model has been proved to be effective for big data hierarchical feature and representation learning in the tensor space. However, it requires expensively computational resources including high-performance computing units and large memory to train a deep computation model with a large number of parameters, limiting its effectiveness and efficiency for industry informatics big data feature learning. In this paper, a tensor-train deep computation model is presented for industry informatics big data feature learning. Specially, the tensor-train network is used to compress the parameters significantly by converting the dense weight tensors into the tensor-train format. Furthermore, a learning algorithm is implemented based on gradient descent and back-propagation to train the parameters of the presented tensor-train deep computation model. Extensive experiments are carried on STL-10, CUAVE, and SNAE2 to evaluate the presented model in terms of the approximation error, classification accuracy drop, parameters reduction, and speedup. Results demonstrate that the presented model can improve the training efficiency and save the memory space greatly for the deep computation model with small accuracy drops, proving its potential for industry informatics big data feature learning.

Despite the widespread adoption of deep learning models, shallow machine learning (SML) algorithms are still used for image classification due to simplicity, interpretability and efficiency. This study aims to bridge this gap by investigating the robustness dynamics of SML techniques under more complex scenarios, such as adversarial perturbations and geometric transformation. Five popular classification algorithms, including k-nearest neighbour and support vector machines, were employed to build classification models. Methodology involves investigating the robustness of proposed methods, first, on original and corrupted data by utilising benchmark datasets across several image domains. To strengthen the investigation, the models were trained using a new low-rank representation (LRR) strategy. This hybrid model simultaneously addresses two key limitations in classical LRR models: overcoming the sequential learning process and effectively capturing both local and global data structures. By introducing a dual regularisation mechanism, it integrates a k-nearest neighbour graph to preserve local consistency, while a global low-rank constraint ensures coherent data representation. Experimental results reveal significant drops in accuracy of most SML methods, especially under adversarial attacks and geometric transformations, but LRR approach mitigates these effects to a notable extent, boosting performance across data variations. The results also show that the proposed method outperforms state-of-the-art LRR techniques in most experiments.

Few-Shot Learning (FSL) algorithms have made substantial progress in learning novel concepts with just a handful of labelled data. To classify query instances from novel classes encountered at test-time, they only require a support set composed of a few labelled samples. FSL benchmarks commonly assume that those queries come from the same distribution as instances in the support set. However, in a realistic setting, data distribution is plausibly subject to change, a situation referred to as Distribution Shift (DS). The present work addresses the new and challenging problem of Few-Shot Learning under Support/ Query Shift (FSQS) i.e., when support and query instances are sampled from related but different distributions. Our contributions are the following. First, we release a testbed for FSQS, including datasets, relevant baselines and a protocol for a rigorous and reproducible evaluation. Second, we observe that well-established FSL algorithms unsurprisingly suffer from a considerable drop in accuracy when facing FSQS, stressing the significance of our study. Finally, we show that transductive algorithms can limit the inopportune effect of DS. In particular, we study both the role of Batch-Normalization and Optimal Transport (OT) in aligning distributions, bridging Unsupervised Domain Adaptation with FSL. This results in a new method that efficiently combines OT with the celebrated Prototypical Networks. We bring compelling experiments demonstrating the advantage of our method. Our work opens an exciting line of research by providing a testbed and strong baselines. Our code is available at https://github.com/ebennequin/meta-domain-shift.