Numerical linear algebra in data mining

Low rank approximation of matrices is an important tool in machine learning. Given a data matrix, low rank approximation helps to find factors, patterns, and provides concise representations for the data. Research on low rank approximation usually focuses on real matrices. However, in many applications data are binary (categorical) rather than continuous. This leads to the problem of low rank approximation of binary matrices. Here we are given a d x n binary matrix A and a small integer k < d. The goal is to find two binary matrices U and V of sizes d x k and k x n respectively, so that the Frobenius norm of A - U V is minimized. There are two models of this problem, depending on the definition of the dot product of binary vectors: The GF(2) model and the Boolean semiring model. Unlike low rank approximation of a real matrix which can be efficiently solved by Singular Value Decomposition, we show that approximation of a binary matrix is NP-hard, even for k=1.\nIn this paper, our main concern is the problem of Column Subset Selection (CSS), in which the low rank matrix U must be formed by k columns of the data matrix, and we are interested in the approximation ratio achievable by CSS for binary matrices. For the GF(2) model, we show that CSS has approximation ratio bounded by k/2+1+k/(2(2^k-1)) and this is asymptotically tight. For the Boolean model, it turns out that CSS is no longer sufficient to obtain a bound. We then develop a Generalized CSS (GCSS) procedure in which the columns of U are generated from Boolean formulas operating bitwise on selected columns of the data matrix. We show that the approximation ratio achieved by GCSS is bounded by 2^(k-1)+1, and argue that an exponential dependency on k is seems inherent.

We propose a new algorithm for the computation of a singular value decomposition (SVD) low-rank approximation of a matrix in the matrix product operator (MPO) format, also called the tensor train matrix format. Our tensor network randomized SVD (TNrSVD) algorithm is an MPO implementation of the randomized SVD algorithm that is able to compute dominant singular values and their corresponding singular vectors. In contrast to the state-of-the-art tensor-based alternating least squares SVD (ALS-SVD) and modified alternating least squares SVD (MALS-SVD) matrix approximation methods, TNrSVD can be up to 13 times faster while achieving better accuracy. In addition, our TNrSVD algorithm also produces accurate approximations in particular cases where both ALS-SVD and MALS-SVD fail to converge. We also propose a new algorithm for the fast conversion of a sparse matrix into its corresponding MPO form, which is up to 509 times faster than the standard tensor train SVD method while achieving machine precision accuracy. The efficiency and accuracy of both algorithms are demonstrated in numerical experiments.

Numerical linear algebra, digital signal processing, and parallel algorithms are three disciplines with a great deal of activity in the last few years. The interaction between them has been growing to a level that merits an Advanced Study Institute dedicated to the three areas together. This volume gives an account of the main results in this interdisciplinary field. The following topics emerged as major themes of the meeting: - Singular value and eigenvalue decompositions, including applications, - Toeplitz matrices, including special algorithms and architectures, - Recursive least squares in linear algebra, digital signal processing and control, - Updating and downdating techniques in linear algebra and signal processing, - Stability and sensitivity analysis of special recursive least squares problems, - Special architectures for linear algebra and signal processing. This book contains tutorials on these topics given by leading scientists in each of the three areas. A consider- able number of new research results are presented in contributed papers. The tutorials and papers will be of value to anyone interested in the three disciplines.

With the advancement of GPU architecture, matrix computation engines such as NVIDIA Tensor Cores now support double-precision (FP64) General matrix multiplications (GEMMs) with the same efficiency as single-precision (FP32) GEMMs. However, the adoption of this enhanced FP64 capability remains limited, primarily restricted to applications that involve multiple FP64 BLAS3 operations. Singular Value Decomposition (SVD), a fundamental decomposition in numerical linear algebra with numerous applications, can greatly benefit from exploiting this hardware feature. In this article, for FP32 SVD, we propose a novel algorithm, FP64 precision eigenvalue decomposition (EVD) based SVD, specifically designed to leverage the latest GPU architectural features. We provide a theoretical analysis demonstrating the feasibility of our approach on emerging GPU architectures and evaluate it from both accuracy and performance perspectives. Moreover, for FP64 SVD, we introduce a double-blocking band reduction technique combined with a GPU-based bulge chasing algorithm to further accelerate the overall SVD process. Experimental results show that, for FP32 SVD, our EVD-based SVD implementation achieves higher numerical accuracy and delivers speedups of up to 6.1× on H100 and 5.0× on A100 over the state-of-the-art cuSOLVER SVD solver. In the case of FP64 SVD, our method also achieves 4.9× and 4.8× speedups on H100 and A100, respectively. These results highlight the potential of our approach as a highly efficient and accurate solution for SVD on modern GPU platforms.

Text mining defines generally the process of extracting interesting features (non-trivial) and knowledge from unstructured text documents. Text mining is an interdisciplinary field which depends on information retrieval, data mining, machine learning, parameter statistics and computational linguistics. Standard text mining and retrieval information techniques of text document usually rely on similar categories. An alternative method of retrieving information is clustering documents to preprocess text. The preprocessing steps have a huge effect on the success to extract knowledge. This study implements TF-IDF and singular value decomposition (SVD) dimensionality reduction techniques. The proposed system presents an effective preprocessing and dimensionality reduction techniques which help the document clustering by using k-means algorithm. Finally, the experimental results show that the proposed method enhances the performance of English text document clustering. Simulation results on BBC news and BBC sport datasets show the superiority of the proposed algorithm.

The field of data mining increasingly adapts methods and algorithms from advanced matrix computations, graph theory and optimization. In these methods, the data is described using matrix representations (graphs are represented by their adjacency matrices) and the data mining problem is formulated as an optimization problem with matrix variables. With these, the data mining task becomes a process of minimizing or maximizing a desired objective function of matrix variables. Prominent examples include spectral clustering, non-negative matrix factorization, Principal Component Analysis (PCA) and Singular Value Decomposition (SVD) related clustering and dimension reduction, tensor analysis and L-1 regularization. These matrix-formulated optimization-centric methodologies are rapidly becoming a significant part of data mining, and have evolved into a popular and rapidlyexpanding research area for solving challenging data mining problems. First, they are amenable to vigorous analysis and benefit from the well-established knowledge in linear algebra, graph theory and optimization accumulated through centuries. Second, they can be efficiently computed thanks to the mature software tools developed by scientific computing communities. Third, they are simple to implement and easy to

Provides readers with the methods, algorithms, and means to perform text mining tasks This book is devoted to the fundamentals of text mining using Perl, an open-source programming tool that is freely available via the Internet (www.perl.org). It covers mining ideas from several perspectives--statistics, data mining, linguistics, and information retrieval--and provides readers with the means to successfully complete text mining tasks on their own. The book begins with an introduction to regular expressions, a text pattern methodology, and quantitative text summaries, all of which are fundamental tools of analyzing text. Then, it builds upon this foundation to explore: Probability and texts, including the bag-of-words model Information retrieval techniques such as the TF-IDF similarity measure Concordance lines and corpus linguistics Multivariate techniques such as correlation, principal components analysis, and clustering Perl modules, German, and permutation tests Each chapter is devoted to a single key topic, and the author carefully and thoughtfully introduces mathematical concepts as they arise, allowing readers to learn as they go without having to refer to additional books. The inclusion of numerous exercises and worked-out examples further complements the book's student-friendly format. Practical Text Mining with Perl is ideal as a textbook for undergraduate and graduate courses in text mining and as a reference for a variety of professionals who are interested in extracting information from text documents.

Multicarrier differential chaos shift keying (MC-DCSK) systems transmit the reference chaotic signal for information recovery to remove the complex synchronization circuit. However, transmission errors in the reference chaotic signal would induce the error propagations when directly used for demodulations. In order to address this issue, we propose a low-rank approximation of matrices (LRAM) detection method to improve the reliability performances. In our design, we exploit the low-rank characteristic of MC-DCSK signals sharing the reference chaotic signal. Instead of directly using the received reference chaotic signal for correlated demodulation, we propose to apply the LRAM method to jointly estimate the reference signal and the information-bearing signal. We use the singular value decomposition and generalized LRAM methods to minimize the distances between the estimates and transmitted data. Thus, the signal-to-noise ratio (SNR) can be improved to achieve lower bit error rate (BER). Moreover, theoretical BER expression is derived and LRAM-aided system is proved to be capable of proving the maximum-likelihood detection for received signals. Simulation results demonstrate the BER performances of our design outperform counterparts over additive white Gaussian noise and fading channels.

Singular value decomposition (SVD) method is a very important matrix decomposition method in linear algebra. It is widely used in signal processing, statistics, data compression and other fields. The paper introduces a SVD method to reduce dimension of original dataset and makes use of the attribute of LSA technique to combine SVD method with LSA technique, and then presents new methods for dual private protection data mining. Finally we conduct experiments to test and verify the proposed approach and get good results.

of works held in research and academic libraries and other relevant institutions, for research and private study; * clarify the current EU exception permitting the use of works that were made to be permanently located in the public space (the 'panorama exception'), to take into account new dissemination channels.

Approximation-free running SVD and its application to motion detection

There is global attention on new data analytic methods. Data scraping (a typical first step for advanced data analytics), text and data mining (TDM, the extraction of knowledge from data) and machine learning (ML, often also simply referred to as Artificial Intelligence or AI) are seen as critical technologies. The legal issues involved in the regulation of data range from privacy and data protection (such as the GDPR) to proprietary approaches (such as copyright, database rights, or proposed new rights in data themselves). This paper focusses on one specific intervention, the introduction of two exceptions for text and data mining in the Directive on Copyright in the Digital Single Market (CDSM). Art. 3 is a mandatory exception for text and data mining (TDM) for the purposes of scientific research; Art. 4 permits text and data mining by anyone but with rightsholders able to “contract-out” (Art. 4), for example preventing TDM use of publicly available online content by “machine-readable means”. We trace the context of using the lever of copyright law to enable emerging technologies and support innovation. Within the EU copyright intervention, elements that may underpin a transparent legal framework for AI are identified, such as the possibility of retention of (permanent) copies for further verification. On the other hand, we identify several pitfalls, including an excessively broad definition of TDM which makes the entire field of data-driven AI development dependent on an exception. We analyse the implications of limiting the scope of the exceptions to the right of reproduction (which leaves the communication of research results in a grey zone). We also argue that the limitation of Art. 3 to certain beneficiaries remains problematic; and that the requirement of lawful access is difficult to operationalize. In conclusion, we argue that there should be no need for a TDM exception for the act of extracting informational value from protected works. The EU’s CDSM provisions paradoxically may favour the development of biased AI systems due to price and accessibility conditions for accessing training data that offer the wrong incentives. We also identify some old and new areas of the EU acquis which will play a crucial role in the future relationship of EU copyright law with technological innovation.

Singular value decomposition (SVD) is a key step in many algorithms in statistics, machine learning and numerical linear algebra. While classical singular value decomposition has been made efficient in terms of computational complexity, classical algorithms are not able to fully utilise modern computing environments. The goal of this work is to survey various implementations and applications of randomized algorithms for SVD. Algorithms are compared in terms of accuracy and time of execution. On example of robust principal component analysis (RPCA), it is shown that using randomized algorithms can yield a significant speedup for image processing and similar applications.

We study a weighted low-rank approximation that is inspired by a problem of constrained low-rank approximation of matrices as initiated by the work of Golub, Hoffman, and Stewart [Linear Algebra Appl., 88/89 (1987), pp. 317--327]. Our results reduce to that of Golub, Hoffman, and Stewart in the limiting cases. We also propose an algorithm based on the alternating direction method to solve our weighted low-rank approximation problem and compare it with the state-of-art general algorithms such as the weighted total alternating least squares algorithm and the expectation maximization algorithm.

A fundamental mathematical field with many applications in science and engineering is linear algebra. This paper investigates the various applications of linear algebra in the fields of traffic flow analysis, electric circuits, cubic spline interpolation, and genetics. This research delves into individual applications while emphasizing cross-disciplinary insights, fostering innovative solutions through the convergence of genetics, cubic spline interpolation, circuits, and traffic flow analysis. The research employs specific methodologies in each application area to demonstrate the practical integration of linear algebra in genetics, cubic spline interpolation, electric circuits, and traffic flow analysis. In genetics, linear algebra techniques are utilized to represent genetic data using matrices, analyze genotype distributions across generations, and identify genotype-phenotype associations. For cubic spline interpolation, linear algebra is employed to construct smooth interpolating curves, involving the derivation of equations for spline functions and the determination of coefficients using boundary conditions and continuity requirements. In electric circuit analysis, linear algebra is crucial for modeling circuit elements, formulating systems of linear equations based on Kirchhoff's laws, and solving for voltage and current distributions in circuits. In traffic flow analysis, linear algebra techniques are used to represent traffic movement in networks, formulate systems of linear equations representing traffic flow dynamics, and solve for traffic flow solutions to optimize transportation networks. By addressing contemporary challenges, emerging research frontiers, and future trajectories at the intersection of linear algebra and diverse domains, this study underscores the profound impact of mathematical tools in advancing understanding and resolving complex real-world problems across multiple fields.