IBVis: Interactive Visual Analytics for Information Bottleneck Based Trajectory Clustering

In a cloud environment, resources should be acquired quickly and automatically released at runtime. Traditional trajectory data partitions, indexing and query processing techniques are extended, so that they can take advantage of the cloud of large clusters of highly parallel processing capabilities. There are ways with trajectory data in the cloud database query processing. The advanced sensing techniques available today caused the existence of many types of trajectory datasets whose study can form a strong basis for decision making regarding a data context. Trajectory data generally includes trajectory classification, trajectory clustering, and trajectory associations and so on. All these tools need trajectory similarity measures to get comparisons in trajectory data. Some of the popular existing methods of trajectory similarity include Euclidean distance, semantic distance and there near variants. Each variant of these techniques exhibits notable differences in measurement of similarity and computational difficulty. This paper targeted to deal with this situation with the goal of lower computational efforts in trajectory similarity computations. A two‐phase trajectory similarity measure is defined. The first phase does point level analysis of trajectory points representing all trajectories and the following phase use the analysis to find the similarities. Using a hybrid similarity framework, new means for trajectory clustering and trajectory query processing are developed. From trajectory analysis followed by trajectory clustering, evaluation of cluster quality and processing of trajectory query are undertaken in this paper in different ways from the existing methods. All the proposed developments are tested with trajectory datasets with multiple attributes and the results are more favour to proposed framework than the existing ones.

Recent major developments in semantic communication systems stem from integration of deep learning (DL) techniques. Following the discovery of capacity achieving codes, the primary motivation for adopting the semantic approach, which retrieves meaning without requiring an exact reconstruction, is its potential to further conserve resources such as bandwidth and power. In this paper, we propose a novel semantic communication framework for textual data over additive white Gaussian noise (AWGN) channels via DL. Our framework leverages the information bottleneck (IB) principle to balance minimizing bit transmission under wireless channel rate constraints with maximizing semantic information retention. Unlike previous works, we integrate the bilingual evaluation understudy (BLEU) sentence similarity score into the training objective to enhance model performance. In particular, inspired by knowledge distillation, we utilize large language models (LLMs) during training to transfer their knowledge of text semantics into our model. Using IB principle, we train a neural semantic encoder at the transmitter and a neural semantic decoder at the receiver that incorporates into its objective function the rate constraint together with the BLEU score and the knowledge encoded in the soft probabilities produced by the LLM. Through extensive experiments, our proposed framework demonstrates a notable improvement of up to 45% in text semantic similarity compared to state-of-the-art benchmarks operating at the same channel capacity, significantly outperforming traditional communication systems. Moreover, it exhibits robustness to variations in signal-to-noise ratio (SNR) and achieves significant gains across both low and medium SNR regimes.

There has been an explosion in the usage of trajectory data. Clustering is one of the simplest and most powerful approaches for knowledge discovery from trajectories. In order to produce meaningful clusters, well-defined metrics are required to capture the essence of similarity between trajectories. One such distance function is Dynamic Time Warping (DTW), which aligns two trajectories together in order to determine similarity. DTW has been widely accepted as a very good distance measure for trajectory data. However, trajectory clustering is very expensive due to the complexity of the similarity functions, for example, DTW has a high computational cost O(n2), where n is the average length of the trajectory, which makes the clustering process very expensive. In this paper, we propose the use of hashing techniques based on Distance-Based Hashing (DBH) and Locality Sensitive Hashing (LSH) to produce approximate clusters and speed up the clustering process.

The common approaches to feature extraction in speech processing are generative and parametric although they are highly sensitive to violations of their model assumptions. Here, we advocate the non-parametric Information Bottleneck (IB). IB is an information theoretic approach that extends minimal sufficient statistics. However, unlike minimal sufficient statistics which does not allow any relevant data loss, IB method enables a principled tradeoff between compactness and the amount of target-related information. IB's ability to improve a broad range of recognition tasks is illustrated for model dimension reduction tasks for speaker recognition and model clustering for age-group verification.

Tremendous growth of Location-based technologies resulted in the generation of a huge volume of spatial data, which needs to be analyzed to get potentially important patterns. The spatial patterns extracted can be used to design a better infrastructure ensuring reliable service coverage. Trajectory data is one variant of spatial data that are generated by moving objects travelling across. It is represented as a sequence of spatial coordinates (latitude, longitude) of a location. Trajectory clustering tries to group similar spatial data points to extract the most common movement behaviors. Trajectory data poses major challenges including uncertainty, sampling rate, representation, relationships, spatial autocorrelation, serialization, redundancy, and triviality, which makes it hard to apply traditional clustering algorithms over trajectory data. In this paper, K-Means and DBSCAN (Density-based spatial clustering of applications with noise) clustering algorithms are analyzed using different similarity measures like Euclidean, Hausdorff and Haversine distances with the help of index measures say Adjusted Rand Index (ARI) and Fowlkes-Mallows scores (FMS). Experiment is carried out over two different trajectory datasets and it is proved that usage of Haversine distance for clustering is efficient than Euclidean and Hausdorff distances in terms of spatial trajectory data.

Disentangled representation learning is typically achieved by a generative model, variational encoder (VAE). Existing VAE-based methods try to disentangle all the attributes simultaneously in a single hidden space, while the separation of the attribute from irrelevant information varies in complexity. Thus, it should be conducted in different hidden spaces. Therefore, we propose to disentangle the disentanglement itself by assigning the disentanglement of each attribute to different layers. To achieve this, we present a stair disentanglement net (STDNet), a stair-like structure network with each step corresponding to the disentanglement of an attribute. An information separation principle is employed to peel off the irrelevant information to form a compact representation of the targeted attribute within each step. Compact representations, thus, obtained together form the final disentangled representation. To ensure the final disentangled representation is compressed as well as complete with respect to the input data, we propose a variant of the information bottleneck (IB) principle, the stair IB (SIB) principle, to optimize a tradeoff between compression and expressiveness. In particular, for the assignment to the network steps, we define an attribute complexity metric to assign the attributes by the complexity ascending rule (CAR) that dictates a sequencing of the attribute disentanglement in ascending order of complexity. Experimentally, STDNet achieves state-of-the-art results in representation learning and image generation on multiple benchmarks, including Mixed National Institute of Standards and Technology database (MNIST), dSprites, and CelebA. Furthermore, we conduct thorough ablation experiments to show how the strategies employed here contribute to the performance, including neurons block, CAR, hierarchical structure, and variational form of SIB.

This paper investigates task-oriented communication for multi-device cooperative edge inference, where a group of distributed low-end edge devices transmit the extracted features of local samples to a powerful edge server for inference. While cooperative edge inference can overcome the limited sensing capability of a single device, it substantially increases the communication overhead and may incur excessive latency. To enable low-latency cooperative inference, we propose a learning-based communication scheme that optimizes local feature extraction and distributed feature encoding in a task-oriented manner, i.e., to remove data redundancy and transmit information that is essential for the downstream inference task rather than reconstructing the data samples at the edge server. Specifically, we leverage Tishby’s information bottleneck (IB) principle (Tishby et al., 2000) to extract the task-relevant feature at each edge device, and adopt the distributed information bottleneck (DIB) framework of Aguerri and Zaidi, 2021, to formalize a single-letter characterization of the optimal rate-relevance tradeoff for distributed feature encoding. To admit flexible control of the communication overhead, we extend the DIB framework to a distributed deterministic information bottleneck (DDIB) objective that explicitly incorporates the representational costs of the encoded features. As the IB-based objectives are computationally prohibitive for high-dimensional data, we adopt variational approximations to make the optimization problems tractable. To compensate for the potential performance loss due to the variational approximations, we also develop a selective retransmission (SR) mechanism to identify the redundancy in the encoded features among multiple edge devices to attain additional communication overhead reduction. Extensive experiments on multi-view image classification and multi-view object recognition tasks evidence that the proposed task-oriented communication scheme achieves a better rate-relevance tradeoff than existing methods.

Trajectory clustering has played an essential role in trajectory mining tasks. It serves in a wide range of real-life applications, including transportation, location-based services, behavioral study, and so on. To support trajectory clustering analytics, a plethora of trajectory clustering methods have been proposed, which mainly extend traditional clustering algorithms by using spatio-temporal characteristics of trajectories. However, existing traditional trajectory clustering approaches based on raw trajectory representation highly rely on hand-craft similarity metrics, and can not capture hidden spatial dependencies in trajectory data, which is inefficient and inflexible for clustering analysis. To this end, we propose an end-to-end deep trajectory clustering framework via self-training, termed as E2DTC, inspired by the data-driven capabilities of deep neural networks. E2DTC does not require any additional manual feature extraction operations, and can be easily adapted for trajectory clustering analytics on any trajectory dataset. Extensive experimental evaluations on three real-life datasets show that our framework E2DTC achieves superior accuracy and efficiency, compared with classical clustering methods (i.e., K-Medoids) and state-of-the-art neural-network based approaches (i.e., t2vec).

Trajectory Data have been considered as a treasure for various hidden patterns which provide deeper understanding of the underlying moving objects. Several studies are focused to extract repetitive, frequent and group patterns. Conventional algorithms defined for Sequential Patterns Mining problems are not directly applicable for trajectory data. Space Partitioning strategies were proposed to capture space proximity first and then time proximity to discover the knowledge in the data. Our proposal addresses time proximity first by identifying trajectories which meet at a minimum of [Formula: see text] time stamps in sequence. A novel tree structure is proposed to ease the process. Our method investigates space proximity using Mahalanobis distance (MD). We have used the Manhattan distance to form prior knowledge that helps the supervised learning-based MD to derive the clusters of trajectories along the true spreads of the objects. With the help of minsup threshold, clusters of frequent trajectories are found and then in sequence they form [Formula: see text] length Sequential Patterns. Illustrative examples are provided to compare the MD metric with Euclidean distance metric, Synthetic dataset is generated and results are presented considering the various parameters such as number of objects, minsup, [Formula: see text] value, number of hops in any trajectory and computational time. Experiments are done on available real-time dataset, taxi dataset, too. Sequential Patterns are proved to be worthy of knowledge to understand dynamics of the moving objects and to recommend the movements in constrained networks.

Mobility data of a moving object, called trajectory data, are continuously generated by vessel navigation systems, wearable devices, and drones, to name a few. Trajectory data consist of samples that include temporal, spatial, and other descriptive features of object movements. One of the main challenges in trajectory data analysis is to divide trajectory data into meaningful segments based on certain criteria. Most of the available segmentation algorithms are limited to processing data offline, i.e., they cannot segment a stream of trajectory samples. In this work, we propose an approach called Reactive Buffering Window - Trajectory Segmentation (RBW-TS), which partitions trajectory data into segments while receiving a stream of trajectory samples. Another novelty compared to existing work is that the proposed algorithm is based on multidimensional features of trajectories, and it can incorporate as many relevant features of the underlying trajectory as needed. This makes RBW-TS general and applicable to numerous domains by simply selecting trajectory features relevant for segmentation purposes. The proposed online algorithm incurs lower computational and memory requirements. Furthermore, it is robust to noisy samples and outliers. We validate RBW-TS on three use cases: (a) segmenting human-movement trajectories in different modes of transportation, (b) segmenting trajectories generated by vessels in the maritime domain, and (c) segmenting human-movement trajectories in a commercial shopping center. The numerical results detailed in the paper demonstrate that (i) RBW-TS is capable of detecting the true breakpoints of segments in all three usecases while processing a stream of trajectory points; (ii) despite low memory and computational requirements, the performance in terms of the harmonic mean of purity and coverage is comparable to that of state-of-the-art batch and online algorithms; (iii) RBW-TS achieves different levels of accuracy depending on the various internal parameter estimation methods used; and (iv) RBW-TS can tackle real-world trajectory data for segmentation purposes.

Telecommunication fraud grows rapidly in recent years, which brings serious property loss or even life loss to victims. However, investigation and evidence collection in such crime cases is extremely difficult, as telecommunication fraud committed by a group usually has the features of long-distance and non-contact. However, in known scenarios of telecommunications fraud, multiple mobile phones or SIM cards are essentially utilized for role pretending, identity hiding and increasing the success rate of scams. Meanwhile, these phones and cards may expose plenty of real-time location data to intentional investigators. Based on such observation, this paper gives a new way to detect telecommunications fraud by finding potential fraudsters based on trajectory data mining. Through analyzing trajectory data, individuals with multiple phones can be found and recognized as potential telecommunications fraudsters which should be intensively monitored. The trajectory clustering and FP-growth algorithm are adapted in the proposed method, and the effectiveness of the method is validated on real-world data sets and simulation data sets. The proposed method provides a technical support for the prevention of fraudulent activities.

Dynamic trajectory partition optimization method based on historical trajectory data

As a frontier research topic in the field of scientific visualization, trajectory data visualization extracts valuable patterns and knowledge from trajectory data for decision support via spatiotemporal trajectory visualization techniques. We propose the concept of multivariate trajectory data and interpret two categories of attributes that are based on geographical space and abstract space. Properly analyzing multivariate trajectory data depends on many factors such as visualization task and data sparsity. Therefore, we generalize rich interactions to explore the evolution of trajectory events and transform the issue into a more intelligibly perceptual task, which derives our discussion regarding advantages and limitations of the analytical methods. This review endeavors to provide a quick and thorough cognition and comprehension with regard to fundamental features and numerous outcomes in visual analytics for trajectory data, seeks to promote comparisons and criticisms about the descriptive framework for multivariate spatiotemporal trajectory data visualization, and aims to encourage the exploration of emerging methods and techniques.

Inspired by the pioneering work of the information bottleneck (IB) principle for Deep Neural Networks’ (DNNs) analysis, we thoroughly study the relationship among the model accuracy, and , where and are the mutual information of DNN’s output T with input X and label Y. Then, we design an information plane-based framework to evaluate the capability of DNNs (including CNNs) for image classification. Instead of each hidden layer’s output, our framework focuses on the model output T. We successfully apply our framework to many application scenarios arising in deep learning and image classification problems, such as image classification with unbalanced data distribution, model selection, and transfer learning. The experimental results verify the effectiveness of the information plane-based framework: Our framework may facilitate a quick model selection and determine the number of samples needed for each class in the unbalanced classification problem. Furthermore, the framework explains the efficiency of transfer learning in the deep learning area.

In this work, we investigate the problem of out-of-distribution (OOD) generalization for unsupervised learning methods on graph data. To improve the robustness against such distributional shifts, we propose a Model-Agnostic Recipe for Improving OOD generalizability of unsupervised graph contrastive learning methods, which we refer to as MARIO. MARIO introduces two principles aimed at developing distributional-shift-robust graph contrastive methods to overcome the limitations of existing frameworks: (i) Invariance principle that incorporates adversarial graph augmentation to obtain invariant representations and (ii) Information Bottleneck (IB) principle for achieving generalizable representations through refining representation contrasting. To the best of our knowledge, this is the first work that investigates the OOD generalization problem of graph contrastive learning, with a specific focus on node-level tasks. Through extensive experiments, we demonstrate that our method achieves state-of-the-art performance on the OOD test set, while maintaining comparable performance on the in-distribution test set when compared to existing approaches. Our codes are available at: https://github.com/ZhuYun97/MARIO.