Representation Learning from Time Labelled Heterogeneous Data for Mobile Crowdsensing

Artificial intelligence has been extensively used to predict surface water quality to assess the health of aquatic ecosystems proactively. However, water quality prediction in data-scarce conditions is a challenge, especially with heterogeneous data from monitoring sites that lack similarity in water quality, hindering the information transfer. A deep learning model is proposed that utilizes representation learning to capture knowledge from source river basins during the pre-training stage, and incorporates meteorological data to accurately predict water quality. This model is successfully implemented and validated using data from 149 monitoring sites across inland China. The results show that the model has outstanding prediction accuracy across all sites, with a mean Nash-Sutcliffe efficiency of 0.80, and has a significant advantage in multi-indicator prediction. The model maintains its excellent performance even when trained with only half of the data. This can be attributed to the representation learning used in the pre-training stage, which enables extensive and accurate prediction under data-scarce conditions. The developed model holds significant potential for cross-basin water quality prediction, which could substantially advance the development of water environment system management.

Deep learning methods achieve remarkable predictive performance in modeling complex, large-scale data. However, assessing the quality of derived models has become increasingly challenging, as more classical statistical assumptions may no longer apply. These difficulties are particularly pronounced for spatio-temporal data, which exhibit dependencies across both space and time and are often characterized by nonlinear dynamics, time variance, and missing observations, hence calling for new accuracy assessment methodologies. This paper introduces a residual correlation analysis framework for assessing the optimality of spatio-temporal relational-enabled neural predictive models, notably in settings with incomplete and heterogeneous data. By leveraging the principle that residual correlation indicates information not captured by the model, enabling the identification and localization of regions in space and time where predictive performance can be improved. A strength of the proposed approach is that it operates under minimal assumptions, allowing also for robust evaluation of deep learning models applied to multivariate time series, even in the presence of missing and heterogeneous data. In detail, the methodology constructs tailored spatio-temporal graphs to encode sparse spatial and temporal dependencies and employs asymptotically distribution-free summary statistics to detect time intervals and spatial regions where the model underperforms. The effectiveness of what proposed is demonstrated through experiments on both synthetic and real-world datasets using state-of-the-art predictive models. • Proposes a novel residual correlation analysis to assess the quality of deep spatio-temporal models and identify regions where the predictions can be improved. • Complements traditional accuracy-based evaluations by offering an independent, metric-agnostic assessment of model quality. • Operates under minimal assumptions and remains effective even with missing or heterogeneous data, making it broadly applicable to real-world scenarios and deep learning models.

Traditional Chinese medicine (TCM) is found on a long‐term medical practice in China. Rare human brains can fully grasp the deep TCM knowledge derived from a tremendous amount of experience. In this big data era, a big electronic brain might be competent via deep learning techniques. For this prospect, the electronic brain needs to process various heterogeneous data, such as images, texts, audio signals, and other sensory data. It used to be a challenge to analyze the heterogeneous data by the computer‐aided system until the advances of the powerful deep learning tools. We propose a multimodal deep learning framework to mimic a TCM practitioner to diagnose a patient on the basis of multimodal perceptions of see, listen, smell, ask, and touch. The framework learns common representations from various high‐dimensional sensory data, and fuse the information for final classification. We propose to use conceptual alignment deep neural networks to embed prior knowledge and obtain interpretable latent representations. We implement a multimodal deep architecture to process tongue image and description text data for TCM diagnosis. Experiments illustrate that the multimodal deep architecture can extract effective features from heterogeneous data, produce interpretable representations, and finally achieve a higher accuracy than either corresponding unimodal architectures.

Mobile crowdsensing enables convenient sensory data collection from a large number of mobile devices and has found various applications. In the real practice, however, the sensory data collected from various mobile devices are usually unreliable. To extract truthful information from the unreliable sensory data in mobile crowdsensing, the topic of truth discovery has received wide attention recently, which essentially operates by estimating user reliability degrees and performing reliability-aware truthful aggregation. Despite the effectiveness, applying truth discovery in mobile crowdsensing faces several privacy and security challenges. First, the sensory data and reliability degrees of users may reveal privacy-sensitive information and, thus, demand strong protection. Second, the requester that initiates a crowdsensing application usually needs to have monetary investment, so the inferred truths can be the requester's proprietary information and should be protected as well. In this paper, we propose a new system architecture enabling encrypted truth discovery in mobile crowdsensing. We focus on general and realistic mobile crowdsensing scenarios with varying levels of user participation, and our security design is built on the confidence-aware truth discovery (CATD) approach for its state-of-the-art accuracy in such scenarios. In our system architecture, users send encrypted sensory data to the cloud, where CATD is then conducted in the encrypted domain. The final encrypted inferred truths are sent to the requester for decryption. Along the whole workflow, the sensory data and reliability degrees of users, as well as the inferred truths of the requester, are kept private. Extensive experiments over real-world mobile crowdsensing dataset show that our design achieves practical performance on mobile devices.

This paper presents a general Surface Reconstruction framework which encapsulates the uncertainty of the sam-pled data, making no assumption on the shape of the surface to be reconstructed. Starting from the input points(either points clouds or multiple range images), an Estimated Existence Function (EEF) is built which modelsthe space in which the desired surface could exist and, by the extraction of EEF critical points, the surface isreconstructed. The nal goal is the development of a generic framework able to adapt the result to different kindsof additional information coming from multiple sensors. Categories and Subject Descriptors (according to ACM CCS) : I.3.3 [Computer Graphics]: Shape Modeling, Uncer-tain data, Multi-sensor Data Fusion 1. Introduction 3D scanning devices are becoming more and more availableand affordable. Thanks to modern acquisition technologies,heterogeneous data can be acquired from multiple acquisi-tion sensors, which often incorporate information about un-certainty of the data sampling process. Surface reconstruc-tion techniques designed over a specic sensor often takeinto account uncertainty during the reconstruction process,but they are limited to work with a single device. On thecontrary, general techniques that can process data comingfrom different sensors usually disregard much part of sensor-specic information, and seldom take into account uncer-tainty.The basic concept of our approach is

Predictive maintenance, as one of the core components of Industry 4.0, takes a proactive approach to maintain machines and systems in good order to keep downtime to a minimum and the airline maintenance industry is not an exception to this. To achieve this goal, practices in Structural Health Monitoring (SHM) complement the existing Non-Destructive-Testing (NDT) have been established in the last decades. Recently, the increasing computational capability such as utilization of a graphical processing unit (GPU) in combination with advanced machine learning techniques such as deep learning has been one of the main drivers in the advancement of predictive analytics in condition monitoring.In our previous work, we proposed a novel approach using deep learning for guided wave based structural health called DeepSHM. As a study case, we treated an ultrasonic signal from guided Lamb wave SHM with a convolutional neural network (CNN). In that work, we only considered a single central frequency excitation. This led to a single governing wavelength which is normally good for the detection of a single damage size.In classical signal processing, applying a broader excitation frequency poses an analysis and interpretation nightmare because it contains more complex information and thus is difficult to understand. This problem can be overcome with deep learning; however, it creates another problem: while deep learning typically results in a more accurate result prediction, it is specifically made for solving only certain types of tasks. While many papers have already introduced deep learning for diagnostics, many of these works are only proposing novel predictive techniques, however the mathematical formalization is lacking, and we are not informed about why we should treat acoustic signal with deep learning. So, the basis of ‘explainable AI’ for SHM and NDT is currently lacking.For this reason, in this paper, we would like to extend our previous work into a more generalized. Rather than focusing on a novel technique, we propose a plausible theoretical perspective inspired from neuroscience for signal representation of deep learning framework to model machine perception in structural health monitoring (SHM), especially because SHM typically involves multiple sensory input from different sensing locations. To do this, we created a set of artificial data from a finite element model (FEM) and represented DeepSHM in two different ways: 1). Perpetual representation of observation and 2). Hierarchical structure of entities that is decomposable in a smaller sub-entity. Consequently, we assume two plausible models for DeepSHM: 1). Either it behaves as a single deciding actor since the observation is regarded as perpetual, and 2). Or it acts as a multiple actor with independent outputs since multiple sensors can form different output probabilities. These artificial data were split into several different input representations, classified into several damage scenarios and then trained with commonly used deep learning training parameters. We compare the performance metrics of each perception model to describe the training behavior of both representations.

Food security prediction from heterogeneous data combining machine and deep learning methods

The amount of available data has experienced significant growth as the result of technology advances in this era of Big Data. The biomedical domain, in particular, is one exemplar field where the number and scale of data sources have increased exponentially in the last decade. They are expected to keep growing even more rapidly to reach the level of Zetta bytes per year in the following year very soon. While more data obtained from advanced biotechnologies, such as high-throughput sequencing that encodes valuable information, are becoming overwhelmed, to discover knowledge from which for biology and medical research is still facing challenging problems with existing approaches. We study in this dissertation how to effectively and efficiently utilize these large-scare data from different numbers and types of sources for biomedical knowledge discovery. Raw data from biological organism such as microbiome usually have intrinsic high dimensionality of the feature space, which inevitably and exponentially raises the computational complexity of existing algorithms. We proposed a new approach using spectral interpolation technique to represent the high-dimensional data in low dimension space that not only greatly improves the efficiency of computing large-scale data but also preserves as much information as possible from original data. The resulting preferable outcomes for clustering and visualization tasks better facilitate the knowledge revealing of patterns and insights for microbial communities. We further studied how to enhance knowledge discovery while more than one data sources are available. Large-scale relational data such as protein-protein interactions (PPI) can be constructed in the form of network to invoke a system-wide perspective than traditional mechanistic approaches to interpret complex biological processes and functionalities. While bio-experiments are exhausted and costly, with two or more networks from different data sources we can apply computational comparative analysis such as Network Alignment to bridge the knowledge between well-studied species and under-examined species. We proposed new methods to globally align multiple large-scale biological networks from different species at the same time. We utilize both topological features and biological features of PPI networks and search heuristically for the best results. Representation learning for network is also integrated into our proposed framework to provide a new way to quantify the structural features of a node with its surrounding topology for the node embedding. The real data experiments showed promising results in finding homologous proteins as well as conserved protein complexes in poor-studied species for knowledge transferring from well-studied species. Besides utilizing homogeneous data from one and more data sources of one type, we keep exploring the possibility of harnessing sources of different types to take advantage of their underlying relational knowledge across heterogeneous data and capture the complex biomedical associations. The heterogeneous disease information networks we formulated in one research include types of sources from disease, pathway, and chemicals. They are filtered and calculated using Dynamic Time Warping (DTW) algorithm and meta path method for topological and semantics scores which lead to effective measurement of the similarity of diseases. In another study, we proposed a novel framework with Graph Convolutional Network to identify and predict disease-RNAs associations to better support the discovery of relational knowledge at the molecular level for medical applications such as disease diagnosis, therapy, and monitoring.

Deep learning (also known as deep structured learning, hierarchical learning, or deep machine learning) is a branch of machine learning based on a set of algorithms that attempt to model high-level abstractions in data by using a deep graph with multiple processing layers, composed of multiple linear and nonlinear transformations. Deep learning has been characterized as a class of machine learning algorithms with the following characteristics [257]: • They use a cascade of many layers of nonlinear processing units for feature extraction and transformation. Each successive layer uses the output from the previous layer as input. The algorithms may be supervised or unsupervised and applications include pattern analysis (unsupervised) and classification (supervised). • They are based on the (unsupervised) learning of multiple levels of features or representations of the data. Higher-level features are derived from lower-level features to form a hierarchical representation. • They are part of the broader machine learning field of learning representations of data. • They learn multiple levels of representations that correspond to different levels of abstraction; the levels form a hierarchy of concepts. These definitions have in common: multiple layers of nonlinear processing units and the supervised or unsupervised learning of feature representations in each layer, with the layers forming a hierarchy from low-level to high-level features. Various deep learning architectures such as deep neural networks, convolutional deep neural networks, deep belief networks (DBN), and recurrent neural networks have been applied to fields like computer vision, automatic speech recognition, natural language processing, audio recognition, and bioinformatics where they have been shown to produce state-of-the-art results on various tasks. In this chapter we start in Section 7.1 with an introduction, giving a brief history of this field, the relevant literature, and its applications. Then we study some basic concepts of deep learning such as convolutional neural networks, recurrent neural networks, backpropagation algorithm, restricted Boltzmann machines, and deep learning networks in Section 7.2. Then we illustrate three examples for Apache Spark implementation for mobile big data (MBD), user moving pattern extraction, and combination with nonparametric Bayesian learning, respectively, in Sections 7.3 through 7.5. Finally, we have summary in Section 7.6.

At present, how to make use of massive medical information resources to provide scientific decision-making for the diagnosis and treatment of diseases, summarize the curative effect of various treatment schemes, and better serve the decision-making management, medical treatment, and scientific research, has drawn more and more attention of researchers. Deep learning, as the focus of most concern by both academia and industry, has been effectively applied in many fields and has outperformed most of the machine learning methods. Under this background, deep learning based medical data analysis emerged. In this survey, we focus on reviewing and then categorizing the current development. Firstly, we fully discuss the scope, characteristic and structure of the heterogeneous medical data. Afterward and primarily, the main deep learning models involved in medical data analysis, including their variants and various hybrid models, as well as main tasks in medical data analysis are all analyzed and reviewed in a series of typical cases respectively. Finally, we provide a brief introduction to certain useful online resources of deep learning development tools.

Deep learning has emerged as a powerful tool in various domains, revolutionising machine learning research. However, one persistent challenge is the scarcity of labelled training data, which hampers the performance and generalisation of deep learning models. To address this limitation, researchers have developed innovative methods to overcome data scarcity and enhance deep model learning capabilities. Two prevalent techniques that have gained significant attention are transfer learning and self-supervised learning. Transfer learning leverages knowledge learned from pre-training on a large-scale dataset, such as ImageNet, and applies it to a target task with limited labelled data. This approach allows models to benefit from the learned representations and effectively transfer knowledge to new tasks, resulting in improved learning performance and generalisation. On the other hand, self-supervised learning focuses on training models using pretext tasks that do not require manual annotation, allowing them to learn valuable representations from large amounts of unlabelled data. These learned representations can then be fine-tuned for downstream tasks, mitigating the need for extensive labelled data. In recent years, transfer and self-supervised learning have found applications in various fields, including medical image processing, video recognition, and natural language processing. These approaches have demonstrated remarkable achievements, enabling breakthroughs in areas such as disease diagnosis, object recognition, and language understanding. However, while these methods offer numerous advantages, they also have limitations. For example, transfer learning may face domain mismatch issues between the pre-training and target domains, while self-supervised learning requires careful design of pretext tasks to ensure meaningful representations. This review paper explores the recent applications of these pre-training methods in various fields within the past three years. It delves into the advantages and limitations of each approach, assesses the performance of models employing these techniques, and identifies potential directions for future research. By providing a comprehensive review of current pre-training methods, this article offers guidance for selecting the best technique for specific deep learning applications to address the data scarcity issue.

Mobile crowdsensing (MCS) enables data collection by leveraging the sensing capabilities of distributed mobile devices (MDs). However, its performance is often constrained by limited coverage and connectivity of ground-based sensors. To overcome these limitations, this paper presents an efficient data collection framework in unmanned aerial vehicle (UAV)-assisted MCS systems. The proposed framework leverages the aerial mobility of UAVs to enhance the spatial coverage of MCS. By optimizing flight trajectories of UAVs, we aim to balance the age of information (AoI) and energy consumption. Specifically, this work considers the mobility of MDs and introduces a communication-enhanced trajectory optimization (CETO) algorithm to improve UAV coordination and adaptability in dynamic environments. Simulation results demonstrate that the proposed algorithm significantly outperforms other mainstream baseline methods employing deep reinforcement learning (DRL).

Mobile crowdsensing (MCS) is a new and promising paradigm of data collection in large-scale sensing and computing. A large group of users with mobile devices are recruited in a specific area to accomplish sensing tasks. An essential aspect of an MCS application is task allocation, which aims to efficiently assign sensing tasks to the recruited workers. Due to various resource and quality constraints, the MCS task allocation problem is often an NP-hard optimization problem. Traditional greedy or heuristic approaches are usually subject to performance loss in a certain degree so as to maintain tractability or accommodate special requirements such as incentive constraints. In this paper, we attempt to employ a deep reinforcement learning method to search for a more efficient task allocation solution. Specifically, we use a double deep Q-network (DDQN) to solve the task allocation problem as a path-planning problem with time windows. Our formulated problem takes into account location-dependency and time-sensitivity of sensing tasks, as well as the resource limits of workers in terms of maximum travelling distances. Simulations are conducted to compare the DDQN-based solution with two standard baseline solutions. The results show that our proposed solution outperforms the baseline solutions in terms of the platform’s profit and the coverage of tasks.

The complexity and diversity of equipments presented in the development of modern industrial technology brings the wide application of deep learning in fault diagnosis. However, the multi-source heterogeneous data collected from different sensors applied in industrial production cannot be used by traditional neural networks directly, which registers a difficulty to fault diagnosis based on deep learning. In addition, with only one of the data used, traditional deep learning fault diagnosis methods ignores the correlation between heterogeneous data, which may result in the loss of useful information and the accuracy of fault diagnosis. To solve the above problems, a fault diagnosis framework based on deep learning for multi-source heterogeneous data fusion is proposed. In the feature learning stage, stack self-encoder (SAE) and convolution neural network (CNN) are used to extract deep features from one-dimensional vibration data and two-dimensional image data respectively, and then feature fusion fault diagnosis is carried out. It solves the problem that traditional neural network can't make full use of multi-source heterogeneous data, and improves the accuracy of fault diagnosis. What's more, the bearing data of Case Western Reserve University are used to verify the validity of the diagnosis method proposed in this paper.

Despite the continuous evolution and significant improvement of cybersecurity mechanisms, malware threats remain one of the most important concerns in cyberspace. Meanwhile, Android malware plays a big role in these ever-growing threats. In recent years, deep learning has become the dominant machine learning technique for malware detection and continues to make outstanding achievements. Deep graph representation learning is the task of embedding graph-structured data into a low-dimensional space using deep learning models. Recently, autoencoders have proven to be an effective way for deep representation learning. However, it is not straightforward to apply the idea of autoencoder to graph-structured data because of their irregular structure. In this paper, we present DroidMalGNN, a novel deep learning technique that combines autoencoders with graph neural networks (GNNs) to detect Android malware in an end-to-end manner. DroidMalGNN represents each Android application with an attributed function call graph (AFCG) that allows it to model complex relationships between data. For more efficiency, DroidMalGNN performs graph representation learning in a supervised manner where two autoencoders are trained with benign and malicious AFCGs separately. In this way, it generates two informative embedding vectors for each AFCG in a low-dimensional space and feeds them into a dense neural network to classify the AFCG as benign or malicious. Our experimental results show that DroidMalGNN can achieve good detection performance in terms of different evaluation measures.