Handwritten Digit Recognition with Nonlinear Fisher Discriminant Analysis

Mixture-of-Experts (MoE) large language models (LLM) have memory requirements that often exceed the GPU memory capacity, requiring costly parameter movement from secondary memories to the GPU for expert computation. In this work, we present Mixture of Near-Data Experts (MoNDE), a near-data computing solution that efficiently enables MoE LLM inference. MoNDE reduces the volume of MoE parameter movement by transferring only the hot experts to the GPU, while computing the remaining cold experts inside the host memory device. By replacing the transfers of massive expert parameters with the ones of small activations, MoNDE enables far more communication-efficient MoE inference, thereby resulting in substantial speedups over the existing parameter of-floading frameworks for both encoder and decoder operations.

The basic idea of constructing locally-adaptive regression models (LAR models) consists in the use of regressors defined on the local subregions of factor values. The belonging of factor values to a particular local subdomain is set by indicator functions. Indicator functions by their nature are close to the well-known concepts of membership functions from the theory of fuzzy systems (Fuzzy Systems). As a rule, to provide the required smoothness of the required dependence of the response on the acting factors such local subdomains are defined with overlapping - in the form of the so-called fuzzy partitions. Type or type of indicator functions may be very different: triangular, trapezoidal, and non-linear. Specifying one or another type of indicator function determines the scheme of weighing local models. Each indicator function must be defined for the entire range of the corresponding factor. Triangular-type functions are used as indicator functions in this work. Linear factor models are considered as local models. It is noted that in their original form the proposed LAR models are not identifiable. The issue of identification of such models in the case of joint estimation of all parameters is considered. The procedure of model reduction is introduced. The resulting model is written out in the space of functions that allow estimation. In the case of dividing the domain of factor determination into two, three or four fuzzy partitions we propose the basis of functions allowing evaluation. The results of computational experiment on regression dependence reconstruction by ordinary polynomials of different degrees and by LAR models are given. The efficiency of LAR models in comparison with polynomials of degree 3 and 4 is noted.

This paper concentrates on the issue of detecting abnormal flows in distributed monitoring systems, which has many network management applications such as anomaly detection and traffic engineering. Collecting massive network traffic in real‐time remains a large challenge due to the limited system resource. Most existing approaches perform abnormal flow detection at one measurement point, while they cause large computation and memory overhead for recovering abnormal flows. In this paper, we propose a novel data streaming method that supports accurate abnormal flow detection with a low memory requirement. The key idea of our method is that each monitor compresses flow information to summary data structure, sends the generated data structure to the controller; then the controller aggregates the received data structures, recovers candidates of abnormal flows and estimates their size and change to find abnormal flows on the basis of the aggregated data structure. The experimental results based on real network traffic show that the proposed approach can detect up to 97% of abnormal flows with low memory and update requirements in comparison with related approaches.

Fault diagnosis plays a vital role in ensuring the operation safety of blast furnaces and improving the quality of molten iron in the ironmaking and steelmaking industry. The blast furnace ironmaking process (BFIP) is intrinsically nonlinear. To address the nonlinearity issue of BFIP, a novel fault diagnosis approach that combines the randomized method, local structure information, and Fisher discriminant analysis is proposed in this paper. Using a randomized feature map, the process data is first mapped onto a randomized explicit low‐dimensional feature space. Compared to kernel methods, explicit low‐dimensional random Fourier features considerably reduce the computational cost, particularly for real‐time fault diagnosis for a large training dataset or a large‐scale process. Additionally, the local structure information contained in the randomized low‐dimensional feature space is extracted. The fault diagnosis performance is improved through the exploration of the local structure of random Fourier features. Finally, the blast furnace iron‐marking process state is determined using Bayesian inference. Case studies on a real‐world BFIP are carried out to demonstrate the superior performance of the proposed method in comparison with other related methods.

The problem of determining whether two programs compute the same function is undecidable. We restrict the class of functions that a program can compute to be a finite dimensional vector space, enabling the problem to become tractable. We prove a theorem showing that the number of test points needed to distinguish any such programs is equal to the dimension of the vector space of functions which they compute, thus generalizing previous results of Howden. Our result unifies certain existing software testing theory and explains the successes and failures of one existing method for selecting test data called perturbation testing. We apply the theorem to obtain three results on the number of test points required to test in vector spaces of Boolean functions and polynomials. First, the Vandermonde matrix used together with our theorem yields the classical result that a polynomial of degree at most n is determined by n+1 points. Second, in vector spaces of Boolean functions we obtain useful results when the degree of the Boolean function is less than the number of inputs to the function. Third, we apply our theorem to describe a technique for testing nonpolynomial, real valued functions.

We investigate techniques to combine multiple representations of a handwritten digit to increase classification accuracy without significantly increasing system complexity or recognition time. We compare multiexpert and multistage combination techniques and discuss in detail in a comparative manner methods for combining multiple learners: voting, mixture of experts, stacking, boosting and cascading. In pen based handwritten character recognition, the input is the dynamic movement of the pentip over the pressure sensitive tablet. There is also the image formed as a result of this movement. On a real world database, we notice that the two multi layer perceptron (MLP) neural network based classifiers using these representations separately make errors on different patterns, implying that a suitable combination of the two would lead to higher accuracy. Thus we implement and compare voting, mixture of experts, stacking and cascading. Combined classifiers have an error percentage less than individual ones. The final combined system of two MLPs has less complexity and memory requirement than a single k nearest neighbor using one of the representations.

We propose a novel probabilistic framework that combines information acquired from different facial features for robust face recognition. The features used are the entire face, the edginess image of the face, and the eyes. In the training stage, individual feature spaces are constructed using principal component analysis (PCA) and Fisher's linear discriminant (FLD). By using the distance-in-feature-space (DIFS) values of the training images, the distributions of the DIFS values in each feature space are computed. For a given image, the distributions of the DIFS values yield confidence weights for the three facial features extracted from the image. The final score is computed using a probabilistic fusion criterion and the match with the highest score is used to establish the identity of a person. A new preprocessing scheme for illumination compensation is also advocated. The proposed fusion approach is more reliable than a recognition system which uses only one feature, trained individually. The method is validated on different face datasets, including the FERET database.

Mobility data of a moving object, called trajectory data, are continuously generated by vessel navigation systems, wearable devices, and drones, to name a few. Trajectory data consist of samples that include temporal, spatial, and other descriptive features of object movements. One of the main challenges in trajectory data analysis is to divide trajectory data into meaningful segments based on certain criteria. Most of the available segmentation algorithms are limited to processing data offline, i.e., they cannot segment a stream of trajectory samples. In this work, we propose an approach called Reactive Buffering Window - Trajectory Segmentation (RBW-TS), which partitions trajectory data into segments while receiving a stream of trajectory samples. Another novelty compared to existing work is that the proposed algorithm is based on multidimensional features of trajectories, and it can incorporate as many relevant features of the underlying trajectory as needed. This makes RBW-TS general and applicable to numerous domains by simply selecting trajectory features relevant for segmentation purposes. The proposed online algorithm incurs lower computational and memory requirements. Furthermore, it is robust to noisy samples and outliers. We validate RBW-TS on three use cases: (a) segmenting human-movement trajectories in different modes of transportation, (b) segmenting trajectories generated by vessels in the maritime domain, and (c) segmenting human-movement trajectories in a commercial shopping center. The numerical results detailed in the paper demonstrate that (i) RBW-TS is capable of detecting the true breakpoints of segments in all three usecases while processing a stream of trajectory points; (ii) despite low memory and computational requirements, the performance in terms of the harmonic mean of purity and coverage is comparable to that of state-of-the-art batch and online algorithms; (iii) RBW-TS achieves different levels of accuracy depending on the various internal parameter estimation methods used; and (iv) RBW-TS can tackle real-world trajectory data for segmentation purposes.

Impostors are image-based primitives commonly used to replace complex geometry in order to reduce the rendering time needed for displaying complex scenes. However, a big problem is the huge amount of memory required for impostors. This paper presents an algorithm that automatically places impostors into a scene so that a desired frame rate and image quality is always met, while at the same time not requiring enormous amounts of impostor memory. The low memory requirements are provided by a new placement method and through the simultaneous use of other acceleration techniques like visibility culling and geometric levels of detail.

This paper presents a new FSVQ scheme called Finite-State Residual Vector Quantization (FSRVQ) in which each state uses a Residual Vector Quantizer (RVQ) to encode the input vector. Furthermore, a novel tree- structured competitive neural network is proposed to jointly design the next-state and the state-RVQ codebooks for the proposed FSRVQ. Joint optimization of the next-state function and the state-RVQ codebooks eliminates a large number of redundant states in the conventional FSVQ design; consequently, the memory requirements are substantially reduced in the proposed FSRVQ scheme. The proposed FSRVQ can be designed for high bit rates due to its very low memory requirements and low search complexity of the state-RVQs. Simulation results show that the proposed FSRVQ scheme outperforms the conventional FSVQ schemes both in terms of memory requirements and perceptual quality of the reconstructed image. The proposed FSRVQ scheme also outperforms JPEG (current standard for still image compression) at low bit rates.© (1995) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.

Face recognition is one of the most fundamental functions for many kinds of intelligent robots. Among many methods proposed for face recognition, linear approaches such as Eigenface or principal component analysis (PCA), Fisher’s linear discriminant (FLD), Fisherface, and common vector (CV), have attracted great attention because of their simplicity. It is known that when the number of training examples is large enough, we should use FLD; otherwise, we should use CV. This paper proposes a rough common vector (RCV) approach. The basic idea of RCV is to divide the feature space into two subspaces. One is spanned by the eigenvectors corresponding to the largest eigenvalues of the within-class scatter matrix, and another is spanned by the eigenvectors corresponding to the smallest eigenvalues. The later plays the role of the null space of the within-class scatter matrix, and is important for extracting useful discriminative features for recognition. RCV can be used regardless the within-class scatter matrix is singular or not. Experimental results on four databases show that RCV outperforms nearest neighbor classifier, Eigenface, Fisherface, and CV in most cases.

The combination of integer lifting wavelet transform with set partitioning in hierarchical trees algorithm has been widely used in the field of image compression. Based on this algorithm, the paper presents a image coding algorithm with lower memory and higher speed. The algorithm further takes Human Visual System into consideration and modifies SPIHT algorithm is according to the characteristic of weighted wavelet coefficients. Experiment result shows that, under the situation of low bit-rate, the new algorithm can prove the visual effect of reconstructed image at a certain degree. In addition, because the new algorithm has low memory requirement, the coding speed accelerates a lot. It has a very broad appliance future in the occasion of high desire of memory and speed.

We study polynomial approximations of vertex singularities of the type r λ | log r| β on three-dimensional surfaces.The analysis focuses on the case when λ > -1 2 .This assumption is a minimum requirement to guarantee that the above singular function is in the energy space for boundary integral equations with hypersingular operators.Furthermore, such strong vertex singularities may appear in solutions to boundary integral formulations of time-harmonic problems of electromagnetism in Lipschitz domains.Thus, the approximation results for such singularities are needed for the error analysis of boundary element methods in three dimensions.Moreover, to our knowledge, the approximation of strong singularities (-1 2 < λ ≤ 0) by high-order polynomials is missing in the existing literature.In this note we prove an estimate for the error of polynomial approximation of the above vertex singularities on quasi-uniform meshes discretising a polyhedral surface.The estimate gives an upper bound for the error in terms of the mesh size h and the polynomial degree p. Introduction.In this note we analyse polynomial approximations of vertex singularities inherent to solutions of boundary integral equations (BIE) on a Lipschitz polyhedral surface Γ.In particular, denoting by r the distance to a vertex of Γ, we study approximations 2000 AMS Mathematics subject classification.

We study the elliptic Ruijsenaars model of type-BC, which is a difference analogue of the BC-type Calogero–Sutherland–Moser model with an elliptic pairwise potential. In the trigonometric limit, the difference operator of this model reduces to the Macdonald–Koornwinder operator by a gauge-transformation. We show that our difference operator acts on a finite-dimensional space of functions in a special case of the coupling constant.

We propose a novel online learning paradigm for nonlinear-function estimation tasks based on the iterative projections in the L <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> space with probability measure reflecting the stochastic property of input signals. The proposed learning algorithm exploits the reproducing kernel of the so-called dictionary subspace, based on the fact that any finite-dimensional space of functions has a reproducing kernel characterized by the Gram matrix. The L <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> -space geometry provides the best decorrelation property in principle. The proposed learning paradigm is significantly different from the conventional kernel-based learning paradigm in two senses: first, the whole space is not a reproducing kernel Hilbert space; and second, the minimum mean squared error estimator gives the best approximation of the desired nonlinear function in the dictionary subspace. It preserves efficiency in computing the inner product as well as in updating the Gram matrix when the dictionary grows. Monotone approximation, asymptotic optimality, and convergence of the proposed algorithm are analyzed based on the variable-metric version of adaptive projected subgradient method. Numerical examples show the efficacy of the proposed algorithm for real data over a variety of methods including the extended Kalman filter and many batch machine-learning methods such as the multilayer perceptron.