Analysis and generation of random vectors with copulas

Parallel VLSI generation of random analog vectors with controlled statistics from deterministic chaos is the key to applications such as analog encryption and secure communications, analog built-in self-test, stochastic neural networks, and simulated annealing optimization as well as perturbation learning in neural hardware. We present a class of analog additive cellular automata which generate parallel streams of statistically independent, uniformly distributed random analog values. The underlying noise-shaping mechanism is essentially that of a MASH cascade of delta-sigma modulators. We present theory on the noise-shaping properties, scalable parallel VLSI architectures, and include experimental results from an analog VLSI prototype with 65 channels. The cell for each channel implements a switched-capacitor delta-sigma modulator, and measures 100 /spl mu/m/spl times/120 /spl mu/m in 2 /spl mu/m CMOS technology. The 65 cells are connected in a chain on a 2-D grid, and can be rearranged for use in various VLSI applications that require a parallel supply of random analog vectors.

The paper discusses a general method of computer generation of random vectors based on transformations of uniformly distributed vectors. This method is then applied to build up efficient algorithms for generatingq-exponential random vectors and multivariate normal ort-distributed vectors. Comparisons and connections with other similar algorithms are also presented.

We present the “trivariate reduction extension” (TREx)—an exact algorithm for the fast generation of bivariate Poisson random vectors. Like the normal-to-anything (NORTA) procedure, TREx has two phases: a preprocessing phase when the required algorithm parameters are identified, and a generation phase when the parameters identified during the preprocessing phase are used to generate the desired Poisson vector. We prove that the proposed algorithm covers the entire range of theoretically feasible correlations, and we provide efficient-computation directives and rigorous bounds for truncation error control. We demonstrate through extensive numerical tests that TREx, being a specialized algorithm for Poisson vectors, has a preprocessing phase that is uniformly a hundred to a thousand times faster than a fast implementation of NORTA. The generation phases of TREx and NORTA are comparable in speed, with that of TREx being marginally faster. All code is publicly available.

Recurrent neural networks-guided vector-valued synchronized key exchange for secure and privacy-preserving communication in Industrial Internet of Things

We study generalized bootstrap confidence regions for the mean of a random vector whose coordinates have an unknown dependency structure. The random vector is supposed to be either Gaussian or to have a symmetric and bounded distribution. The dimensionality of the vector can possibly be much larger than the number of observations and we focus on a nonasymptotic control of the confidence level, following ideas inspired by recent results in learning theory. We consider two approaches, the first based on a concentration principle (valid for a large class of resampling weights) and the second on a resampled quantile, specifically using Rademacher weights. Several intermediate results established in the approach based on concentration principles are of interest in their own right. We also discuss the question of accuracy when using Monte Carlo approximations of the resampled quantities.

Mixtures of normal distributions provide a useful modeling extension of the normal distribution—both univariate and multivariate. Unlike the normal distribution, mixtures of normals can capture the kurtosis (fat tails) and nonzero skewness often necessary for accurately modeling a variety of real-world variables. An efficient analytical Monte Carlo method is proposed for considering multivariate mixtures of normal distributions having arbitrary covariance matrices. The method consists of a linear transformation of a multivariate normal having a computed covariance matrix into the desired multivariate mixture of normal distributions. The computed covariance matrix is derived analytically. Among the properties of the multivariate mixture of normals that we demonstrate is that any linear combination of mixtures of normal distributions is also a mixture of normal distributions. Methods of fitting mixtures of normal distributions are briefly discussed. A motivating example carried throughout this paper is the use of multivariate mixtures of normals for modeling daily changes in market variables.

On generation of elliptical distributions with Gaussian form covariance matrix

Testing Independence for a Large Number of High-Dimensional Random Vectors

Current verification methods for on-chip memory have been implemented using coverpoints that are generated based on a single operation. These coverpoints cannot consider the influence of other memory banks in a busy state. In this paper, we propose a method in which the coverpoints account for all operations executed on different memory banks. In addition, a new constrained random vector generation method is proposed to reduce the required random vectors for the multi-operation-based coverpoints. The simulation results on NAND flash memory show 100% coverage with 496,541 constrained random vectors indicating a reduction of 96.4% compared with conventional random vectors.

The NORTA method for multivariate generation is a fast general purpose method for generating samples of a random vector with given marginal distributions and given product-moment or rank correlation matrix. However, this method has been shown to fail to work for some feasible correlation matrices. (A matrix is feasible if there exists a random vector with the given marginal distributions and the matrix as the correlation matrix.) We investigate how this feasibility problem behaves as the dimension of the random vector is increased and find the problem to become acute rapidly. We also find that a modified NORTA procedure, augmented by a semidefinite program (SDP) that aims to generate a correlation matrix close to the desired one, performs well with increasing dimension.

Brain-inspired high-dimensional (HD) computing represents and manipulates data using very long, random vectors with dimensionality in the thousands. This representation provides great robustness for various classification tasks where classifiers operate at low signal-to-noise ratio (SNR) conditions. Similarly, hyperdimensional modulation (HDM) leverages the robustness of complex-valued HD representations to reliably transmit information over a wireless channel, achieving a similar SNR gain compared to state-of-the-art codes. Here, we first propose methods to improve HDM in two ways: (1) reducing the complexity of encoding and decoding operations by generating, manipulating, and transmitting bipolar or integer vectors instead of complex vectors; (2) increasing the SNR gain by 0.2 dB using a new soft-feedback decoder; it can also increase the additive superposition capacity of HD vectors up to 1.7times in noise-free cases. Secondly, we propose to combine encoding/decoding aspects of communication with classification into a single framework by relying on multifaceted HD representations. This leads to a near-channel classification (NCC) approach that avoids transformations between different representations and the overhead of multiple layers of encoding/decoding, hence reducing latency and complexity of a wireless smart distributed system while providing robustness against noise and interference from other nodes. We provide a use-case for wearable hand gesture recognition with 5 classes from 64 EMG sensors, where the encoded vectors are transmitted to a remote node for either performing NCC, or reconstruction of the encoded data. In NCC mode, the original classification accuracy of 94% is maintained, even in the channel at SNR of 0 dB, by transmitting 10,000-bit vectors. We remove the redundancy by reducing the vector dimensionality to 2048-bit that still exhibits a graceful degradation: less than 6% accuracy loss is occurred in the channel at − 5 dB, and with the interference from 6 nodes that simultaneously transmit their encoded vectors. In the reconstruction mode, it improves the mean-squared error by up to 20 dB, compared to standard decoding, when transmitting 2048-dimensional vectors.

We present a new approach to independent component analysis (ICA) by extending the formulation of univariate source signals to multivariate source signals. The new approach is termed independent vector analysis (IVA). In the model, we assume that linear mixing model exists in each dimension separately, and the latent sources are independent of the others. In contrast to ICA, the sources are random vectors, not just single variables, which means the elements of a random vector are closely related to the others. Thus, we assume the dependency between the elements of a source vector. In this manner, we define dependence between vectors as Kullback-Leibler divergence between the total joint probability of vectors and the product of marginal probabilities of vectors. Then, the model allows independence between multivariate source signals represented as random vectors, and dependence between the source signals within the vector representation. The proposed vector density model can for example capture variance dependencies within a vector source signal. There are several applications of this new formulation. In the separation of acoustic sources, the algorithm mitigates the permutation problem, i.e. the usual ICA algorithms applied in to the frequency domain mixture data suffer from the unknown permutation of the output signals. Although there are several engineering solutions to fix this problem after the ICA stage, the proposed method provides a natural solution to the problem by capturing the inherent dependencies of the acoustic signals. It therefore avoids the permutation problem and allows the separation of sources in very challenging environments for many sound sources.

Dependence within a high-dimensional profile of explanatory variables affects estimation and prediction performance of regression models. However, the strong belief that dependence should not be ignored, based on our well-proven knowledge of low-dimensional regression modeling, is not necessarily true in high dimension. To investigate this point, we introduce a new class of prediction scores defined as linear combinations of a same random vector, including the naive prediction score obtained when ignoring dependence and the Ordinary Least Squares (OLS) prediction score that, on the contrary, fully accounts for dependence by a preliminary whitening of the explanatory variables. Interestingly, the former class also contains Ridge and Partial Least Squares prediction scores, that both offer intermediate ways of dealing with dependence. Through a theoretical comparative study, it is first shown how the best handling of dependence should depend on the interplay between the structure of conditional dependence across explanatory variables and the pattern of the association signal. We also derive the closed form expression of the prediction score with best prediction performance within the proposed class, leading to an adaptive handling of dependence. Finally, it is demonstrated through simulation studies and using benchmark datasets that this prediction score outperforms existing methods in various settings. Supplementary materials for this article are available online.

A Markov tree is a random vector indexed by the nodes of a tree whose distribution is determined by the distributions of pairs of neighbouring variables and a list of conditional independence relations. Upon an assumption on the tails of the Markov kernels associated to these pairs, the conditional distribution of the self-normalized random vector when the variable at the root of the tree tends to infinity converges weakly to a random vector of coupled random walks called a tail tree. If, in addition, the conditioning variable has a regularly varying tail, the Markov tree satisfies a form of one-component regular variation. Changing the location of the root, that is, changing the conditioning variable, yields a different tail tree. When the tails of the marginal distributions of the conditioning variables are balanced, these tail trees are connected by a formula that generalizes the time change formula for regularly varying stationary time series. The formula is most easily understood when the various one-component regular variation statements are tied up into a single multi-component statement. The theory of multi-component regular variation is worked out for general random vectors, not necessarily Markov trees, with an eye towards other models, graphical or otherwise.

Random Vectors in High Dimensions