Scaling limit for small blocks in the Chinese restaurant process

The Pitman-Yor, or Chinese restaurant process, is a stochastic process that generates distributions following a power law with exponents lower than 2, as found in numerous physical, biological, technological, and social systems. We discuss its rich behavior with the tools and viewpoint of statistical mechanics. We show that this process invariably gives rise to a condensation, i.e., a distribution dominated by a finite number of classes. We also evaluate thoroughly the finite-size effects, finding that the lack of stationary state and self-averaging of the process creates realization-dependent cutoffs and behavior of the distributions with no equivalent in other statistical mechanical models.

We present a novel generative model for audio event transcription that recognizes “events” on audio signals including multiple kinds of overlapping sounds. In the proposed model, firstly, the overlapping audio events are modeled based on nonnegative matrix factorization into which Bayesian nonparametric approaches: the Markov Indian buffet process and the Chinese restaurant process, are incorporated. This approach allows us to automatically transcribe the events while avoiding the model selection problem by assuming a countably infinite number of possible audio events in the input signal. Then, Bayesian logistic regression annotates the audio frames with the multiple event labels in a semi-supervised learning setup. Experimental results show that our model can better annotate an audio signal in comparison with a baseline method. Additionally, we verify that our infinite generative model is also able to detect unknown audio events that are not included in the training data.

This paper presents an autonomous cognitive radio (CR) architecture, referred to as the Radiobot. This model goes beyond adaptive radio systems to exploit the main ingredients of cognition which, in this context, are mainly self-learning and self-reconfiguration. Without any prior knowledge of the RF environment, the Radiobot applies a sequence of increasingly sophisticated processing steps to detect and identify the sensed signals. In particular, in this paper, it applies a blind energy detection followed by a cyclostationary detection method to detect the active signals and extract their underlying periodic properties as reflected in cyclic frequencies. These extracted signal features are classified based on the Chinese restaurant process (CRP) and a learning algorithm is applied to achieve autonomous self-reconfiguration of the sensing module. We analyze the impact of fading and Doppler frequency shift on both the energy and cyclostationary detections, and show the receiver operating characteristic (ROC) of the carrier frequency detector. We show the robustness of the cyclostationary detection against channel fading and wide-sense stationary noise. Simulation results are presented to verify the multi-band operability and the reconfiguration ability of the Radiobot and to verify the convergence of the proposed learning algorithm.

Crowdsourcing has been proven to be an effective and efficient tool to annotate large data-sets. User annotations are often noisy, so methods to combine the annotations to produce reliable estimates of the ground truth are necessary. We claim that considering the existence of clusters of users in this combination step can improve the performance. This is especially important in early stages of crowdsourcing implementations, where the number of annotations is low. At this stage there is not enough information to accurately estimate the bias introduced by each annotator separately, so we have to resort to models that consider the statistical links among them. In addition, finding these clusters is interesting in itself as knowing the behavior of the pool of annotators allows implementing efficient active learning strategies. Based on this, we propose in this paper two new fully unsupervised models based on a Chinese restaurant process (CRP) prior and a hierarchical structure that allows inferring these groups jointly with the ground truth and the properties of the users. Efficient inference algorithms based on Gibbs sampling with auxiliary variables are proposed. Finally, we perform experiments, both on synthetic and real databases, to show the advantages of our models over state-of-the-art algorithms.

An important problem in evolutionary genomics is to investigate whether a certain trait measured on each sample is associated with the sample phylogenetic tree. The phylogenetic tree represents the shared evolutionary history of the samples and it is usually estimated from molecular sequence data at a locus or from other type of genetic data. We propose a model for trait evolution inspired by the Chinese Restaurant Process that includes a parameter that controls the degree of preferential attachment, that is, the tendency of nodes in the tree to subtend from nodes of the same type. This model with no preferential attachment is equivalent to a structured coalescent model with simultaneous migration and coalescence events and serves as a null model. We derive a test for phylogenetic binary trait association with linear computational complexity and empirically demonstrate that it is more powerful than some other methods. We apply our test to study the phylogenetic association of some traits in swordtail fish, breast cancer, yellow fever virus, and influenza A H1N1 virus. R-package implementation of our methods is available at https://github.com/jyzhang27/CRPTree.

Learning a distinct representation for each sense of an ambiguous word could lead to more powerful and fine-grained models of vector-space representations. Yet while `multi-sense' methods have been proposed and tested on artificial word-similarity tasks, we don't know if they improve real natural language understanding tasks. In this paper we introduce a multi-sense embedding model based on Chinese Restaurant Processes that achieves state of the art performance on matching human word similarity judgments, and propose a pipelined architecture for incorporating multi-sense embeddings into language understanding. We then test the performance of our model on part-of-speech tagging, named entity recognition, sentiment analysis, semantic relation identification and semantic relatedness, controlling for embedding dimensionality. We find that multi-sense embeddings do improve performance on some tasks (part-of-speech tagging, semantic relation identification, semantic relatedness) but not on others (named entity recognition, various forms of sentiment analysis). We discuss how these differences may be caused by the different role of word sense information in each of the tasks. The results highlight the importance of testing embedding models in real applications.

Traditional radar signal sorting (RSS) methods assume that all types of radar signals can be stored and trained offline. However, the presence of noncooperative radar emitters makes the assumption unrealistic in real-world RSS problems. This article proposes a real-time method for RSS in complex electromagnetic environments based on the Bayesian framework. Each type of radar signal is regarded as a collection of samples from Gaussian distribution. Bayesian inference is applied to calculate the likelihood of a new data point belonging to a known category, and the Chinese restaurant process (CRP) is adopted to provide some prior knowledge which helps calculate the probability of opening a new cluster. For the overclustering problem caused by dynamic and large-scale changes in pulse descriptor words (PDWs) of the signals from the same radar emitter, a dynamic cluster merging (DCM) algorithm is introduced. The Wasserstein distance and Hausdorff distance of the probability density functions (pdfs) of the PDWs between clusters are weighted, and the pulse repetition interval (PRI) is also considered to measure the similarity between clusters which helps merge clusters of the same type of signal. The method proposed in this article is verified under the conditions of large-scale changes in PDWs and dynamic changes in signal numbers over time. Higher accuracy of real-time signal sorting is achieved on the simulated dataset of signals from 30 types of radar emitters compared with existing methods. Even in the situations with a significant pulse loss rate, the proposed algorithm still attains high accuracy.

Metagenomes present assembly challenges, when assembling multiple genomes from mixed reads of multiple species. An assembler for single genomes can’t adapt well when applied in this case. A metagenomic assembler, Genovo, is a de novo assembler for metagenomes under a generative probabilistic model. Genovo assembles all reads without discarding any reads in a preprocessing step, and is therefore able to extract more information from metagenomic data and, in principle, generate better assembly results. Paired end sequencing is currently widely-used yet Genovo was designed for 454 single end reads. In this research, we attempted to extend Genovo by incorporating paired-end information, named Xgenovo, so that it generates higher quality assemblies with paired end reads.First, we extended Genovo by adding a bonus parameter in the Chinese Restaurant Process used to get prior accounts for the unknown number of genomes in the sample. This bonus parameter intends for a pair of reads to be in the same contig and as an effort to solve chimera contig case. Second, we modified the sampling process of the location of a read in a contig. We used relative distance for the number of trials in the symmetric geometric distribution instead of using distance between the offset and the center of contig used in Genovo. Using this relative distance, a read sampled in the appropriate location has higher probability. Therefore a read will be mapped in the correct location.Results of extensive experiments on simulated metagenomic datasets from simple to complex with species coverage setting following uniform and lognormal distribution showed that Xgenovo can be superior to the original Genovo and the recently proposed metagenome assembler for 454 reads, MAP. Xgenovo successfully generated longer N50 than Genovo and MAP while maintaining the assembly quality even for very complex metagenomic datasets consisting of 115 species. Xgenovo also demonstrated the potential to decrease the computational cost. This means that our strategy worked well. The software and all simulated datasets are publicly available online at http://xgenovo.dna.bio.keio.ac.jp.

Evaluation of glass evidence at activity level: A new distribution for the background population

We introduce diffusions on a space of interval partitions of the unit interval that are stationary with the Poisson–Dirichlet laws with parameters α,0 and α,α. The construction has two steps. The first is a general construction of interval partition processes obtained previously by decorating the jumps of a Lévy process with independent excursions. Here, we focus on the second step which requires explicit transition kernels and, what we call, pseudo-stationarity. This allows us to study processes obtained from the original construction via scaling and time-change. In a sequel paper we establish connections to diffusions on decreasing sequences introduced by Ethier and Kurtz (Adv. in Appl. Probab. 13 (1981) 429–452) and Petrov (Funktsional. Anal. i Prilozhen. 43 (2009) 45–66). The latter diffusions are continuum limits of up-down Markov chains on Chinese restaurant processes. Our construction is also a step toward resolving longstanding conjectures by Feng and Sun on measure-valued Poisson–Dirichlet diffusions and by Aldous on a continuum-tree-valued diffusion.

Probability density estimation is widely-known as an ill-posed statistical problem whose solving depends on extra constraints. We investigated what prior beliefs people might have in their learning of an arbitrary probability distribution, especially whether the distribution is believed to be unimodal or multimodal. In each block of our experiments, participants repeatedly reconstructed a one-dimensional spatial distribution after observing every 60 new samples from the distribution. The probability distribution function (PDF) they reported on each trial was submitted to a spectral analysis, where the powers for 1-cycle, 2-cycle, …, n-cycle components respectively indicate participants’ tendency of reporting unimodal, bimodal, …, n-modal distributions. In two experiments, the reported PDFs showed significant bimodality—that the 2-cycle power was above chance and even larger than the 1-cycle power—not only for bimodal distributions, but also for uniform distribution. Such illusory bimodality for uniform distribution was first found when we used an adaptive procedure analogous to “human MCMC”, updating the generative distribution of samples from trial to trial to reinforce potential biases in PDFs (Experiment 1). However, even when we fixed the generative distribution across trials (Experiment 2), the illusory bimodality did not vanish. The illusory bimodality was even observed before participants experienced any bimodal distributions in the experiment . We considered a few kernel density models and discuss further computational explanations (e.g. prior beliefs following Chinese Restaurant Process) for this new phenomenon.

We propose four variants of a novel hierarchical hidden Markov models strategy for rule induction in the context of automated sports video annotation including a multilevel Chinese takeaway process (MLCTP) based on the Chinese restaurant process and a novel Cartesian product label-based hierarchical bottom-up clustering (CLHBC) method that employs prior information contained within label structures. Our results show significant improvement by comparison against the flat Markov model: optimal performance is obtained using a hybrid method, which combines the MLCTP generated hierarchical topological structures with CLHBC generated event labels. We also show that the methods proposed are generalizable to other rule-based environments including human driving behavior and human actions.

Proteome coverage prediction with variants of the Chinese restaurant process construction

Chapter 13 - Bayesian Learning: Approximate Inference and Nonparametric Models

We use a natural ordered extension of the Chinese Restaurant Process to grow a twoparameter family of binary self-similar continuum fragmentation trees. We provide an explicit embedding of Ford’s sequence of alpha model trees in the continuum tree which we identified in a previous article as a distributional scaling limit of Ford’s trees. In general, the Markov branching trees induced by the two-parameter growth rule are not sampling consistent, so the existence of compact limiting trees cannot be deduced from previous work on the sampling consistent case. We develop here a new approach to establish such limits, based on regenerative interval partitions and the urn-model description of sampling from Dirichlet random distributions. AMS 2000 subject classifications: 60J80.