A Multi-view Clustering Model for Event Detection in Twitter

In recent years, multiview clustering algorithms have achieved promising performance by exploiting the complementarity and consistency of different views. However, many multiview spectral clustering methods only focus on the consistent information of views, and the time cost of feature decomposition is expensive. Moreover, these methods also require postprocessing (e.g., k-means) to obtain the final clustering results. To overcome these limitations simultaneously, we propose a novel multiview clustering algorithm. First, the method removes the inconsistent information of the views through cross-view measurement to maintain consistent information. These inconsistencies may be caused by noise, corruptions, or view-specific properties and will affect the quality of the similarity matrix. Then, we learn a consensus embedding matrix with nonnegative constraints by performing a low-rank decomposition of the consistency information. In this way, we can replace the eigendecomposition of the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">n</i> <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\bm {\times }$</tex-math></inline-formula> <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">n</i> Laplacian matrix in spectral clustering with the singular value decomposition of a <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">n</i> <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\bm {\times }$</tex-math></inline-formula> <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">c</i> low-rank matrix to reduce the computational burden, where <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">c</i> <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\bm {\ll }$</tex-math></inline-formula> <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">n</i> . Furthermore, due to the nonnegative constraint, we can directly obtain the clustering results. Also, to consider the diversity of views, adaptive weighting is applied to different view data. Compared to state-of-the-art multiview clustering methods on five benchmark multiview datasets, we demonstrate the superiority and effectiveness of our approach. We release the source code at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/hulu88/FAMvC</uri> .

With the advent of social networks and the rise of mobile technologies, users have become ubiquitous sensors capable of monitoring various real-world events in a crowd-sourced manner. Location-based social networks have proven to be faster than traditional media channels in reporting and geo-locating breaking news, i.e. Osama Bin Laden’s death was first confirmed on Twitter even before the announcement from the communication department at the White House. However, the deluge of user-generated data on these networks requires intelligent systems capable of identifying and characterizing such events in a comprehensive manner. The data mining community coined the term, event detection , to refer to the task of uncovering emerging patterns in data streams . Nonetheless, most data mining techniques do not reproduce the underlying data generation process, hampering to self-adapt in fast-changing scenarios. Because of this, we propose a probabilistic machine learning approach to event detection which explicitly models the data generation process and enables reasoning about the discovered events. With the aim to set forth the differences between both approaches, we present two techniques for the problem of event detection in Twitter : a data mining technique called Tweet-SCAN and a machine learning technique called Warble. We assess and compare both techniques in a dataset of tweets geo-located in the city of Barcelona during its annual festivities. Last but not least, we present the algorithmic changes and data processing frameworks to scale up the proposed techniques to big data workloads.

Tensorized multi-dimensional multi-view clustering based on nonnegative matrix factorization.

In the era of Industry 4.0, single‐view clustering algorithm is difficult to play a role in the face of complex data, i.e., multiview data. In recent years, an extension of the traditional single‐view clustering is multiview clustering technology, which is becoming more and more popular. Although the multiview clustering algorithm has better effectiveness than the single‐view clustering algorithm, almost all the current multiview clustering algorithms usually have two weaknesses as follows. (1) The current multiview collaborative clustering strategy lacks theoretical support. (2) The weight of each view is averaged. To solve the above‐mentioned problems, we used the Havrda‐Charvat entropy and fuzzy index to construct a new collaborative multiview fuzzy c‐means clustering algorithm using fuzzy weighting called Co‐MVFCM. The corresponding results show that the Co‐MVFCM has the best clustering performance among all the comparison clustering algorithms.

Event detection in twitter is the process of discovering popular events using messages generated by the twitter users. Event detection is an interesting research topic. Tweets are focused and may contain short forms. The tweets are noisy because there may be personal messages by the user also. In this paper we propose an algorithm to find top k popular events using keywords contained in the tweets. This paper classifies the popular events into different categories and the timeline is provided for every event. The timeline is useful to check when the event was popular. Geotagging is also done to find where the event was popular. We have implemented the algorithm using 14,558 users and 5, 27,548 tweets over a period of 10 months (22 June, 2015 to 25 April, 2016). The results are quite promising.

Latent Dirichlet allocation (LDA) is a new paradigm of topic model which is powerful to capture the latent topic information from natural language. However, the topic information in text streams, e.g. meeting recording, lecture transcription and conversational dialogue, are inherently heterogeneous and nonstationary without explicit boundaries. It is difficult to train a precise topic model from the observed text streams. Furthermore, the usage of words in different paragraphs within a document is varied with different composition styles. In this paper, we present a new hierarchical segmentation model (HSM) where the heterogeneous topic information in stream level and the word variations in document level are characterized. We incorporate the contextual topic information in stream-level segmentation. The topic similarity between sentences is used to form a beta distribution reflecting the prior knowledge of document boundaries in a text stream. The distribution of segmentation variable is adaptively updated to achieve flexible segmentation and is used to group coherent sentences into a topic-specific document. For each pseudo-document, we further use a Markov chain to detect the stylistic segments within a document. The words in a segment are accordingly generated by the same composition style, which differs from the style of the next segment. Each segment is represented by a Markov state, and so the word variations within a document are compensated. The whole model is trained by a variational Bayesian EM procedure and is evaluated on using TDT2 corpus. Experimental results show benefits by using the proposed HSM in terms of perplexity, segmentation error, detection accuracy and F measure.

The availability of many heterogeneous but related views of data has arisen in numerous clustering problems. Different views encode distinct representations of the same data, which often admit the same underlying cluster structure. The goal of multi-view clustering is to properly combine information from multiple views so as to generate high quality clustering results that are consistent across different views. Based on max-product belief propagation, we propose a novel multi-view clustering algorithm termed multi-view affinity propagation (MVAP). The basic idea is to establish a multi-view clustering model consisting of two components, which measure the within-view clustering quality and the explicit clustering consistency across different views, respectively. Solving this model is NP-hard, and a multi-view affinity propagation is proposed, which works by passing messages both within individual views and across different views. However, the exemplar consistency constraint makes the optimization almost impossible. To this end, by using some previously designed mathematical techniques, the messages as well as the cluster assignment vector computations are simplified to get simple yet functionally equivalent computations. Experimental results on several real-world multi-view datasets show that MVAP outperforms existing multi-view clustering algorithms. It is especially suitable for clustering more than two views.

Twitter First Story Detection (FSD) task refers to the detection of first tweet about the new event in tweet stream, which is a hard but important task in twitter event detection. However, the number of comparisons is too large in traditional online detection methods and there is a lack of global information of each event during detection process. To deal with the shortcomings above, we propose a novel FSD method based on nugget, which can describe the event concisely. Our approach generates and updates dynamically a nugget for each detected event in the process of detection. When a new tweet arrives, it is first compared with the nugget of each event, to be clustered into the event when it hits the nugget. Otherwise it is compared with individual tweets in the event. Our method improves the detection accuracy and reduces the number of comparisons. The experimental results on two public data sets show that our system has reached the state-of-the-art. Besides, we prove theoretically that our method possesses advantages in efficiency.

Multi-view Clustering and Multi-view Models

Asymmetric double-winged multi-view clustering network for exploring diverse and consistent information

Multi-view clustering via deep concept factorization

&lt;p&gt;Event detection from social media is a challenging task due to the volume, the velocity and the variety of&lt;br&gt; user-generated data requiring real-time processing. Despite recent works on this subject, a generalized and scalable&lt;br&gt; approach that could be applied across languages and topics has not been consolidated, yet. In this paper, we propose&lt;br&gt; a methodology for real-time event detection from Twitter data that allows users to select a topic of interest by&lt;br&gt; defining a simple set of keywords and a matching rule. We implement the proposed methodology and evaluate it&lt;br&gt; with real data to detect different types of events.&lt;/p&gt;

Towards a unified framework for graph-based multi-view clustering

Event Detection (ED) is a study area that attracts the attention of decision-makers from various disciplines in order to help them in taking the right decision. ED has been examined on various text streams like Twitter, Facebook, Emails, Blogs, Web Forums and newswires. Many ED models have been proposed in literature. In general, ED model consists of six main phases: Data collection, pre-processing, feature selection, event detection, performance evaluation and result representation. Among these phases, event detection phase has a vital rule in the performance of the ED model. Consequently, numerous supervised, unsupervised, semi-supervised detection methods have been introduced for this phase. However, unsupervised methods have been extensively utilized as ED process is considered as unsupervised task. Hence, such methods need to be categorized on such a way so it can help researchers to understand and identified the limitations lay in these methods. In this survey, ED models for text data from various Social Network sites (SNs) are analyzed based on domain type, detection methods, type of detection task. In addition, main categories for unsupervised detection methods are explicitly mentioned with revising their related works. Moreover, the major open challenges faced by researchers for building ED models are explained and discussed in detail. The main objective of this survey paper is to provide a complete view of the recent developments in ED field. Hence, help scholars to identify the limitations of existing ED models for text data and help them to recognize the interesting future works directions.

Multi-view clustering guided by unconstrained non-negative matrix factorization