Soft-orthogonal constrained dual-stream encoder with self-supervised clustering network for brain functional connectivity data

Cancer genomics is dedicated to elucidating the genes and pathways that contribute to cancer progression and development. Identifying cancer genes (CGs) associated with the initiation and progression of cancer is critical for characterization of molecular-level mechanism in cancer research. In recent years, the growing availability of high-throughput molecular data and advancements in deep learning technologies has enabled the modelling of complex interactions and topological information within genomic data. Nevertheless, because of the limited labelled data, pinpointing CGs from a multitude of potential mutations remains an exceptionally challenging task. To address this, we propose a novel deep learning framework, termed self-supervised masked graph learning (SMG), which comprises SMG reconstruction (pretext task) and task-specific fine-tuning (downstream task). In the pretext task, the nodes of multi-omic featured protein-protein interaction (PPI) networks are randomly substituted with a defined mask token. The PPI networks are then reconstructed using the graph neural network (GNN)-based autoencoder, which explores the node correlations in a self-prediction manner. In the downstream tasks, the pre-trained GNN encoder embeds the input networks into feature graphs, whereas a task-specific layer proceeds with the final prediction. To assess the performance of the proposed SMG method, benchmarking experiments are performed on three node-level tasks (identification of CGs, essential genes and healthy driver genes) and one graph-level task (identification of disease subnetwork) across eight PPI networks. Benchmarking experiments and performance comparison with existing state-of-the-art methods demonstrate the superiority of SMG on multi-omic feature engineering.

Supervised deep learning methods have produced state-of-the-art results with large labeled datasets. However, accessing large labeled datasets is difficult in medical image analysis because of a shortage of medical experts, expensive annotations, and privacy constraints in the healthcare domain. Self-supervised learning is a branch of machine learning that exploits unlabeled data to encourage network weights toward a valid latent representation of the data during a so-called pretext task. The features learned by the model while solving pretext tasks are transferred to a downstream task where limited annotations are available. In this work, we propose PatchLoc, a novel pretext task whose objective is to find the location of a given patch from an image as a source of supervision. We validated the effectiveness of PatchLoc on a downstream segmentation task using three different medical datasets. PatchLoc yields substantial improvements compared to U-Net trained from scratch and other pretext task-based approaches in a low data regime.

Towards generalizable Graph Contrastive Learning: An information theory perspective

The integration of two fields, which are neuroscience and psychology has allowed a deeper understanding of the mental processes in the human mind, through the observation of biological and chemical processes in the brain and nervous system. Even though the gap between neuroscience and the human behavioral sciences is still large, the advancement of technology has helped narrow the gap, especially over the past two decades. This scoping review is aimed to explore the currently published research on self-reflection using the fMRI and the EEG. The present study describes the scope of existing research, and summarizes findings and recommendations for future work. The scoping review methodology guided by the standards of the PRISMA Extension for Scoping Reviews (PRISMA-ScR) was used. Comprehensive searches on the use of the fMRI and the EEG during self-reflection studies were conducted using several electronic databases including Scopus, Science Direct, Web of Science, Emerald Insights, and PubMed. The results obtained showed that all studies did not justify the sample size. The findings also revealed that various terms were used to describe the cognitive aspect of a self-reflection process. The brain functional connectivity data were taken through several experimental stages, which shows the flexibility of the experimental process. The activation of the prefrontal cortex (PFC) is one of the most commonly found regions in most studies, together with other areas associated with the self-reflection process. Based on the results obtained, this scoping review is believed to provide a comprehensive summary of the existing research on self-reflection using neuroimaging techniques.

Histopathological image segmentation is a laborious and time-intensive task, often requiring analysis from experienced pathologists for accurate examinations. To reduce this burden, supervised machine-learning approaches have been adopted using large-scale annotated datasets for histopathological image analysis. However, in several scenarios, the availability of large-scale annotated data is a bottleneck while training such models. Self-supervised learning (SSL) is an alternative paradigm that provides some respite by constructing models utilizing only the unannotated data which is often abundant. The basic idea of SSL is to train a network to perform one or many pseudo or pretext tasks on unannotated data and use it subsequently as the basis for a variety of downstream tasks. It is seen that the success of SSL depends critically on the considered pretext task. While there have been many efforts in designing pretext tasks for classification problems, there have not been many attempts on SSL for histopathological image segmentation. Motivated by this, we propose an SSL approach for segmenting histopathological images via generative diffusion models. Our method is based on the observation that diffusion models effectively solve an image-to-image translation task akin to a segmentation task. Hence, we propose generative diffusion as the pretext task for histopathological image segmentation. We also utilize a multi-loss function-based fine-tuning for the downstream task. We validate our method using several metrics on two publicly available datasets along with a newly proposed head and neck (HN) cancer dataset containing Hematoxylin and Eosin (H&E) stained images along with annotations.

Supervised learning technologies have been used in medical-data classification to improve diagnosis efficiency and reduce human diagnosis errors. A large amount of manually annotated data are required for the fully supervised learning process. However, annotating data information will consume a large amount of manpower and resources. Self-supervised learning has great advantages in solving this problem. Self-supervised learning mainly uses pretext tasks to mine its own supervised information from large-scale unsupervised data. And this constructed supervised information is used to train the network to learn valuable representations for downstream tasks. This study designs a general and efficient model for the diagnosis and classification of medical sensor data based on contrastive predictive coding (CPC) in self-supervised learning, called TCC, which consists of two steps. The first step is to design a pretext task based on the idea of CPC, which aims to extract effective features between different categories using its encoder. The second step designs a downstream classification task with lower time and space complexity to perform a supervised type of training using the features extracted by the encoder of the pretext task. Finally, to demonstrate the performance of the proposed framework in this paper, we compare the proposed framework with recent state-of-the-art works. Experiments comparing the proposed framework with supervised learning are also set up under the condition of different proportions of labeled data.

Self-supervised video representation learning leaves out heavy manual annotation by automatically excavating supervisory signals. Although contrastive learning based approaches exhibit superior performances, pretext task based approaches still deserve further study. This is because the pretext tasks exploit the nature of data and encourage feature extractors to learn spatiotemporal logic by discovering dependencies among video clips or cubes, without manual engineering on data augmentations or manual construction of contrastive pairs. To utilize chronological property more effectively and efficiently, this work proposes a novel pretext task, named serial restoration of shuffled clips (SRSC), disentangled by an elaborately designed task network composed of an order-aware encoder and a serial restoration decoder. In contrast to other order based pretext tasks that formulate clip order recognition as a one-step classification problem, the proposed SRSC task restores shuffled clips into the right order in multiple steps. Owing to the excellent elasticity of SRSC, a novel taxonomy of curriculum learning is further proposed to equip SRSC with different pre-training strategies. According to the factors that affect the complexity of solving the SRSC task, the proposed curriculum learning strategies can be categorized into task based, model based and data based. Extensive experiments are conducted on the subdivided strategies to explore their effectiveness and noteworthy laws. Compared with existing approaches, this work demonstrates that the proposed approach achieves state-of-the-art performances in pretext task based self-supervised video representation learning and a majority of the proposed strategies further boost the performance of downstream tasks. For the first time, the features pre-trained by the pretext tasks are applied to video captioning by feature-level early fusion, and enhance the input of existing approaches as a lightweight plugin.

Supervised learning methods have shown effectiveness in estimating spatial acoustic parameters such as time difference of arrival, direct-to-reverberant ratio and reverberation time. However, they still suffer from the simulation-to-reality generalization problem due to the mismatch between simulated and real-world acoustic characteristics and the deficiency of annotated real-world data. To this end, this work proposes a selfsupervised method that takes full advantage of unlabeled data for spatial acoustic parameter estimation. First, a new pretext task, i.e. cross-channel signal reconstruction (CCSR), is designed to learn a universal spatial acoustic representation from unlabeled multi-channel microphone signals. We mask partial signals of one channel and ask the model to reconstruct them, which makes it possible to learn spatial acoustic information from unmasked signals and extract source information from the other microphone channel. An encoder-decoder structure is used to disentangle the two kinds of information. By fine-tuning the pre-trained spatial encoder with a small annotated dataset, this encoder can be used to estimate spatial acoustic parameters. Second, a novel multi-channel audio Conformer (MC-Conformer) is adopted as the encoder model architecture, which is suitable for both the pretext and downstream tasks. It is carefully designed to be able to capture the local and global characteristics of spatial acoustics exhibited in the time-frequency domain. Experimental results of five acoustic parameter estimation tasks on both simulated and real-world data show the effectiveness of the proposed method. To the best of our knowledge, this is the first self-supervised learning method in the field of spatial acoustic representation learning and multi-channel audio signal processing. 1

A core step of mining human mobility data is to learn accurate representations for user-generated check-in sequences. The learned representations should be able to fully describe the spatial-temporal mobility patterns of users and the high-level semantics of traveling. However, existing check-in sequence representation learning is usually implicitly achieved by end-to-end models designed for specific downstream tasks, resulting in unsatisfactory generalizable abilities and poor performance. Besides, although the sequence representation learning models that follow the contrastive learning pre-training paradigm have achieved breakthroughs in many fields like NLP, they fail to simultaneously consider the unique spatial-temporal characteristics of check-in sequences and need manual adjustments on the data augmentation strategies. So, directly applying them to check-in sequences cannot yield a meaningful pretext task. To this end, in this paper we propose a contrastive pre-training model with adversarial perturbations for check-in sequence representation learning (CACSR). Firstly, we design a novel spatial-temporal augmentation block for disturbing the spatial-temporal features of check-in sequences in the latent space to relieve the stress of designing manual data augmentation strategies. Secondly, to construct an effective contrastive pretext task, we generate “hard” positive and negative pairs for the check-in sequence by adversarial training. These two designs encourage the model to capture the high-level spatial-temporal patterns and semantics of check-in sequences while ignoring the noisy and unimportant details. We demonstrate the effectiveness and versatility of CACSR on two kinds of downstream tasks using three real-world datasets. The results show that our model outperforms both the state-of-the-art pre-training methods and the end-to-end models.

BackgroundAttention-deficit/hyperactivity disorder (ADHD) is currently diagnosed by a diagnostic interview, mainly based on subjective reports from parents or teachers. It is necessary to develop methods that rely on objectively measureable neurobiological data to assess brain-behavior relationship in patients with ADHD. We investigated the application of a topological data analysis tool, Mapper, to analyze the brain functional connectivity data from ADHD patients.MethodsTo quantify the disease severity using the neuroimaging data, the decomposition of individual functional networks into normal and disease components by the healthy state model (HSM) was performed, and the magnitude of the disease component (MDC) was computed. Topological data analysis using Mapper was performed to distinguish children with ADHD (n = 196) from typically developing controls (TDC) (n = 214).ResultsIn the topological data analysis, the partial clustering results of patients with ADHD and normal subjects were shown in a chain-like graph. In the correlation analysis, the MDC showed a significant increase with lower intelligence scores in TDC. We also found that the rates of comorbidity in ADHD significantly increased when the deviation of the functional connectivity from HSM was large. In addition, a significant correlation between ADHD symptom severity and MDC was found in part of the dataset.ConclusionsThe application of HSM and topological data analysis methods in assessing the brain functional connectivity seem to be promising tools to quantify ADHD symptom severity and to reveal the hidden relationship between clinical phenotypic variables and brain connectivity.

Pre-trained language models (e.g., BERT) significantly alleviate two traditional challenging problems for Chinese word segmentation (CWS): segmentation ambiguity and out-ofvocabulary (OOV) words. However, such improvements are usually achieved on traditional benchmark datasets and not close to an important goal of CWS: practicability (i.e., low complexity as a standalone task and high beneficiality to downstream tasks). To make a trade-off between traditional evaluation and practicability for CWS, we propose a semisupervised neural method via pseudo labels. The neural method consists of a teacher model and a student model, which distills knowledge from unlabeled data to the student model so as to improve both in-domain and out-ofdomain CWS. Experiments show that our proposed method can not only keep the practicability of the lightweight student model but also improve the performance of segmentation effectively. We also evaluate a range of heterogeneous neural architectures of CWS on downstream Chinese NLP tasks. Results of further experiments demonstrate that our proposed segmenter is reliable and practical as a pre-processing step of the downstream NLP tasks at the minimum cost.

Neuroimaging plays an significant role in diagnosing and pathological study of brain diseases. Considering that both functional and structural abnormalities may lead to brain dis-eases and disorders, single modal neuroimaging approach may not fully characterize brain activities and working modes. Fusion of multimodal neuroimaging data is expected to provide more comprehensive characterization of brain diseases, given that the different modalities contain more complementary information. Recently, Graph Convolutional Networks (GCNs) is shown to have powerful capacity in representation learning for graph-structure data, which is considered to integrate both graph se-mantic structure and node information. Therefore, in this paper, we propose the Weighted Graph AutoEncoder (WGAE), a GCN- driven multimodal fusion model, to learn the combinational latent node representation of fMRI and DTI neuroimaging data, which are used as node features and graph structure respectively in the graph in unsupervised manner. Experimental results on two real-world datasets show the superiority of the proposed model over other existing single-modal or multi-modal methods in learning representations for disease prediction as the downstream task. Furthermore, ablation experiments also show the collaborative contribution of multimodal neuroimaging fusion in the proposed model, and also show the feasibility of assessing the respective importance of the two modalities during the disease prediction.

Deep convolutional networks have demonstrated state-of-the-art performance in a variety of medical image tasks, including segmentation. Taking advantage of images from different modalities has great clinical benefits. However, the generalization ability of deep networks on different modalities is challenging due to domain shift. In this work, we investigate the challenging unsupervised domain adaptation problem of cross-modality medical image segmentation. Cross-modality domain shift can be viewed as having two orthogonal components: appearance (modality) shift and content (anatomy) shift. Previous works using the popular adversarial training strategy emphasize the significant appearance/modality alignment caused by different physical principles while ignoring the content/anatomy alignment, which can be harmful for the downstream segmentation task. Here, we design a cross-modality segmentation pipeline, where self-supervision is introduced to achieve further semantic alignment specifically on the disentangled content space. In the self-supervision branch, in addition to rotation prediction, we also propose elastic transformation prediction as a new pretext task. We validate our model on cross-modality liver segmentation from CT to MR. Both quantitative and qualitative experimental results demonstrate that further semantic alignment through self-supervision can improve segmentation performance significantly, making the learned model more robust.

The goal of the hypergraph foundation model (HGFM) is to learn an encoder based on the hypergraph computational paradigm through self-supervised pretraining on high-order correlation structures, enabling the encoder to rapidly adapt to various downstream tasks in scenarios, where no labeled data or only a small amount of labeled data are available. The initial exploratory work has been applied to brain disease diagnosis tasks. However, existing methods primarily rely on graph-based approaches to learn low-order correlation patterns between brain regions in brain networks, neglecting the modeling and learning of complex correlations between different brain diseases and patients. This article proposes an HGFM for brain disease diagnosis, which conducts multidimensional pretraining tasks to explore latent cross-dimensional high-order correlation patterns on various brain disease datasets. HGFM is a high-order correlation-driven foundation model for brain disease diagnosis and effectively improves prediction performance. Specifically, HGFM first performs brain functional network link prediction tasks on individual brain networks and group interaction network link prediction tasks on group brain networks, constructing an HGFM for brain disease diagnosis. In downstream tasks, it achieves predictions for different brain disease diagnosis tasks through few-shot learning fine-tuning methods. The proposed method is evaluated on functional magnetic resonance imaging (fMRI) data from 4409 patients across four brain diseases. Results show that it outperforms existing state-of-the-art methods in all brain disease diagnosis tasks, demonstrating its potential value in clinical applications.

This paper serves as the first empirical study on self-supervised pre-training on partially supervised learning, an emerging yet unexplored learning paradigm with missing annotations. This is particularly important in the medical imaging domain, where label scarcity is the main challenge of practical applications. To promote the awareness of partially supervised learning, we leverage partially supervised multi-label classification on chest X-ray images as an instance task to illustrate the challenges of the problem of interest. Through a series of simulated experiments, the empirical findings validate that solving multiple pretext tasks jointly in the pre-training stage can significantly improve the downstream task performance under the partially supervised setup. Further, we propose a new pretext task, reverse vicinal risk minimization, and demonstrate that it provides a more robust and efficient alternative to existing pretext tasks for the instance task of interest.