Two-photon voltage imaging denoising by self-supervised learning

The availability of huge annotated datasets and deep learning have led to astounding Natural Language Processing (NLP) progress. However, there is a plethora of low-resource languages, lack the labelled corpora to train the models in the supervision. This kind of digital divide does not allow fair NLP development across language groups. Self-supervised learning (SSL) is a new paradigm in recent years that leverages large amounts of untagged text to learn powerful representations with little or no manual annotation. This paper is a review of self-supervised deep learning models in low-resource NLP tasks. It begins with the definition of the principles of the SSL and the difference between this approach and supervised and unsupervised approaches. We describe why methods such as masked language modeling, contrastive learning, and autoregressive modeling underlie modern pre-trained transformers such as BERT, GPT, and mBERT. A particular interest is paid to multilingual and cross lingual SSL schemes which allow knowledge transfer between high-resource and low resource languages. Low-resource tasks, including machine translation, sentiment analysis, speech-to-text, and information retrieval are reviewed. As per benchmark studies, even in small labelled samples, the SSL models can achieve big gains in accuracy and generalization. Computation cost, representational bias and morphologically rich and under-documented language evaluation, are however, problematic. The researchers observe that the study suffers certain limitations, such as over-reliance on high-resource pretraining information, inequity between linguistic groups, and the difficulty of deploying large-scale SSL models in resource-constrained circumstances. The future directions include lightweight multilingual models, federation of learning in NLP and symbolic linguistic knowledge and combination with SSL. Self-supervised deep learning bridges an essential gap between the high-resource and low-resource languages, and would be a highly valuable step toward inclusive, global NLP innovation.

This paper proposes a novel self-supervised learning framework that uses model reinforcement, REference-free LAtent map eXtraction with MOdel REinforcement (RELAX-MORE), for accelerated quantitative MRI (qMRI) reconstruction. The proposed method uses an optimization algorithm to unroll an iterative model-based qMRI reconstruction into a deep learning framework, enabling accelerated MR parameter maps that are highly accurate and robust. Unlike conventional deep learning methods which require large amounts of training data, RELAX-MORE is a subject-specific method that can be trained on single-subject data through self-supervised learning, making it accessible and practically applicable to many qMRI studies. Using quantitative mapping as an example, the proposed method was applied to the brain, knee and phantom data. The proposed method generates high-quality MR parameter maps that correct for image artifacts, removes noise, and recovers image features in regions of imperfect image conditions. Compared with other state-of-the-art conventional and deep learning methods, RELAX-MORE significantly improves efficiency, accuracy, robustness, and generalizability for rapid MR parameter mapping. This work demonstrates the feasibility of a new self-supervised learning method for rapid MR parameter mapping, that is readily adaptable to the clinical translation of qMRI.

The classification of marine plankton images is of great significance in ecological studies and environmental monitoring. In practical applications, plankton image classification faces several challenges, including sample imbalance, distinguishing between-class and within-class differences, and recognizing fine-grained features. To address these issues, we propose a few-shot self-supervised transfer learning (FSTL) framework. In FSTL, we design a new loss function that incorporates both supervised and self-supervised learning. The core of FSTL is a hybrid learning objective that integrates self-supervised contrastive learning for robust feature representation. Operating within a transfer learning paradigm, FSTL effectively adapts knowledge from head-classes to boost the few-shot classification performance on tail-classes. We applied FSTL to two datasets, in which plankton images were collected from Daya Bay and provided by the Woods Hole Oceanographic Institution (WHOI) datasets respectively. The experimental results demonstrated that our method showed better adaptability in the classification of plankton images. The findings of this study not only apply to the classification of plankton images but also offer the potential for classifying small-sample categories within long-tailed datasets.

Anomaly classification based on self-supervised learning and its application

Applying self-supervised deep learning improves the processing speed and accuracy of automatic modulation recognition (AMR). It reduces the dependence of previous deep networks on many labeled samples. However, affected by an incomplete signal representation modes set, previous models do not fully utilize the multiview property of signals in self-supervised learning. To deal with this issue, a hybrid-view contrastive model for AMR is proposed in this article based on self-supervised learning framework. First, star video is proposed to complete the set of signal representation modes. Next, a self-supervised learning framework based on hybrid-view contrastive learning, hybrid-view self-supervised framework (HVSF), is established to fully extract the signal features, where signals are augmented across views, including the discrete sequence, image, and video format. Considering the view-exclusive information loss and the model complexity, a weakly contrastive strategy and a Transformer-based view-shared feature extractor are finally constructed. Evaluation on four standard datasets demonstrates that the proposed model, HVSF, outperforms both the self-supervised models and supervised models, affirming its superior performance and stability.

We propose a novel probabilistic method employing Bayesian Model Averaging and self-cycle regularization for spatio-temporal correspondence learning in videos within a self-supervised learning framework. Most existing methods for self-supervised correspondence learning suffer from noisy labels that come with the data for free, and the presence of occlusion exacerbates the problem. We tackle this issue within a probabilistic framework that handles model uncertainty inherent in the path selection problem built on a complete graph. We propose a self-cycle regularization to consider a cycle-consistency property on individual edges in order to prevent converging on noisy matching or trivial solutions. We also utilize a mixture of sequential Bayesian filters to estimate posterior distribution for targets. In addition, we present a domain contrastive loss to learn discriminative representation among videos. Our algorithm is evaluated on various datasets for video label propagation tasks including DAVIS2017, VIP and JHMDB, and shows outstanding performances compared to the state-of-the-art self-supervised learning based video correspondence algorithms. Moreover, our method converges significantly faster than previous methods.

DrasCLR: A self-supervised framework of learning disease-related and anatomy-specific representation for 3D lung CT images

Precision agriculture requires accurate discrimination between winter crops and weeds, but there is a lack of annotated image data for winter cropping systems. This paper investigates a two-stage deep learning approach that integrates self-supervised feature learning with supervised fine-tuning for winter crop and weed image classification. A new winter crop and weed image dataset, WinterCropWeedDB, is proposed and used in this paper, which contains 1,136 high-resolution images of six winter crop species and four weed species collected from agricultural fields in central India. In the first stage of self-supervised learning, an EfficientNet-B3 model is pre-trained using a SimCLR-style self-supervised learning approach with an InfoNCE loss function (temperature τ = 0.5) on the images. The average contrastive loss value reduces from 2.0712 in the first iteration to 1.6835 at the end of pretraining. In the second stage of supervised fine-tuning, the pre-trained EfficientNet-B3 model is fine-tuned with a supervised classifier head on the images and tested on a single internal validation split (30%) of the dataset. The fine-tuned model reaches a maximum validation accuracy of 98.27%, with a macro-averaged F1-score of 0.98. Gradient-weighted class activation mapping (Grad-CAM) and Grad-CAM++ are used on the fine-tuned model to provide a qualitative visualization of the image regions that contribute to class predictions. The experiment outcomes demonstrate the viability of using self-supervised pretraining and supervised fine-tuning for the classification of winter crop and weed images on a region-specific dataset, while also emphasizing the importance of additional testing on independent test sets.

Background Histopathological endpoints are evolving as a treatment target in Inflammatory Bowel Disease (IBD). Use of histology to screen entrants could add value in IBD clinical trials; for example, by refining eligibility criteria to ensure studies recruit patients with definitive active inflammation at the microscopic level. Several histopathological indices have been developed, but the relative complexity of available scores hinders the development of an AI algorithm without large-scale labour-intensive annotation by a pathologist. We aimed to develop computer vision tools to assist decoding the complex clinical disease features at the histological level for both Crohn’s Disease (CD) and Ulcerative Colitis (UC). This will inform understanding of disease pathology and patient stratification to support clinical trial development strategies. Methods A total of 1397 clinically annotated Haematoxylin & Eosin (H&E) images were included from 418 CD and 218 UC patients enrolled in a multicentred longitudinal Study of a Prospective Adult Research Cohort with IBD (SPARC IBD) obtained from the IBD Plexus program of the Crohn’s & Colitis Foundation. We developed an image quality control (QC) algorithm to automatically identify image and tissue processing/staining artefacts negatively impacting analysis (e.g., out-of-focus, tissue folds, overstained regions) and excluded these regions (Fig. 1). Next, a self-supervised learning (SSL) deep learning computer vison model was developed and trained to predict disease relevant features including disease diagnosis and lesional macroscopic appearance (inflammation, erosions and ulcers). Finally, to better understand the model’s predictions, we generated heatmap overlays on the tissue that show regions which the model considers to be most predictive (Fig. 2) and shared these with pathologists for qualitative evaluation. Results We find that the SSL model performs well on different downstream classification tasks such as UC vs CD (area under curve (AUC) = 0.79) and normal vs lesional tissue (AUC = 0.76). Specialist pathologist collaboration further confirmed that the heatmap overlays identified clinically relevant tissue features, including inflammatory cell infiltrates (Fig. 2). Conclusion These encouraging results support further exploration of this deep-learning algorithm to distinguish disease specific characteristics in this set of images from CD and UC patients. Further work is ongoing to validate the heatmap approach on endoscopic scores, we also plan to validate our model on IBD clinical trial datasets.

The image reconstruction process in medical imaging can be treated as solving an inverse problem. The inverse problem is usually solved using time-consuming iterative algorithms with sparsity or other constraints. Recently, deep neural network (DNN)-based methods have been developed to accelerate the inverse-problem-solving process. However, these methods typically adopt supervised learning scheme, which requires ground truths, or labels of the solutions, for training. In many applications, it would be challenging or even impossible to obtain the ground truth, such as the tissue reflectivity function in ultrasound beamforming. In this study, a general framework based on self-supervised learning (SSL) scheme is proposed to train a DNN to solve the inverse problems. In this way, the measurements can be used as both the inputs and the labels during the training of DNN. The proposed SSL method is applied to four typical linear inverse problems for validation, i.e., plane wave ultrasound and photoacoustic image reconstructions, compressed sensing-based synthetic transmit aperture dataset recovery and deconvolution in ultrasound localization microscopy. Results show that, using the proposed framework, the trained DNN can achieve improved reconstruction accuracy with reduced computational time, compared with conventional methods.

To review the latest developments in self-supervised deep learning (DL) techniques for magnetic resonance imaging(MRI) reconstruction, emphasizing their potential to overcome the limitations of supervised methods dependent on fully sampled k-space data. While DL has significantly advanced MRI, supervised approaches require large amounts of fully sampled k-space data for training-a major limitation given the impracticality and expense of acquiring such data clinically. Self-supervised learning has emerged as a promising alternative, enabling model training using only undersampled k-space data, thereby enhancing feasibility and driving research interest. We conducted a comprehensive literature review to synthesize recent progress in self-supervised DL for MRI reconstruction. The analysis focused on methods and architectures designed to improve image quality, reduce scanning time, and address data scarcity challenges, drawing from peer-reviewed publications and technical innovations in the field. Self-supervised DL holds transformative potential for MRI reconstruction, offering solutions to data limitations while maintaining image quality and accelerating scans. Key challenges include robustness across diverse anatomies, standardization of validation, and clinical integration. Future research should prioritize hybrid methodologies, domain-specific adaptations, and rigorous clinical validation. This review consolidates advancements and unresolved issues, providing a foundation for next-generation medical imaging technologies.

Existing scene text recognition methods leverage large-scale labeled synthetic data (LSD) to reduce reliance on labor-intensive annotation tasks and improve recognition capability in real-world scenarios. However, the emergence of a synth-to-real domain gap still limits their efficiency and robustness. Consequently, harvesting the meaningful intrinsic qualities of unlabeled real data (URD) is of great importance, given the prevalence of text-laden images. Toward the target, recent efforts have focused on pre-training on URD through sequence-to-sequence self-supervised learning, followed by fine-tuning on LSD via supervised learning. Nevertheless, they encounter three important issues: coarse representation learning units, inflexible data augmentation, and an emerging real-to-synth domain drift. To overcome these challenges, we propose CCDPlus, an accurate character-to-character distillation method for scene text recognition with a joint supervised and self-supervised learning framework. Specifically, tailored for text images, CCDPlus delineates the fine-grained character structures on URD as representation units by transferring knowledge learned from LSD online. Without requiring extra bounding box or pixel-level annotations, this process allows CCDPlus to enable character-to-character distillation flexibly with versatile data augmentation, which effectively extracts general real-world character-level feature representations. Meanwhile, the unified framework combines self-supervised learning on URD with supervised learning on LSD, effectively solving the domain inconsistency and enhancing the recognition performance. Extensive experiments demonstrate that CCDPlus outperforms previous state-of-the-art (SOTA) supervised, semi-supervised, and self-supervised methods by an average of 1.8%, 0.6%, and 1.1% on standard datasets, respectively. Additionally, it achieves a 6.1% improvement on the more challenging Union14M-L dataset.

Scanning Electron Microscope (SEM) image analysis plays a vital role in semiconductor thin-film characterisation. In particular, defect detection and classification are performed using SEM images. However, conventional methods rely on labelled datasets or handcrafted features for classification, which limits their generalisation in real-world industrial inspection settings. The present work proposes a self-supervised multi-resolution learning framework for label-agnostic morphology representation learning and clustering of semiconductor thin-film defects. It uses a SEM image Dataset (4591 images) obtained from industrial wafer inspection. The framework starts with image pre-processing to remove acquisition artifacts. It employs a multi-resolution image pyramid for capturing surface morphologies at fine, intermediate, and coarse spatial scales. A shared-weight convolutional encoder that ensures alignment between embeddings across the three resolutions is trained (on an unlabelled dataset) and utilised for unsupervised defect morphology grouping. The framework learns morphological representations without defect labels. However, defect labels are used only for post-hoc evaluation by normalized mutual information (NMI) and visualization. Intrinsic clustering metrics and low-dimensional visualization are used to assess the algorithm's efficacy. Experimental results reveal that the proposed method, gray level co-occurrence matrix (GLCM), local binary patterns (LBP), wavelet-based features, and principal component analysis (PCA) on raw pixels obtained a silhouette score of 0.50, 0.43, 0.31, 0.45, and 0.22, respectively. While normalized mutual information (NMI) values remained moderate across the models. These results reflect the label-agnostic nature of the proposed SSL framework. Further, UMAP and t-SNE visualizations confirm the coherent manifold structure and the effectiveness of morphology-driven grouping. These results demonstrate the robust, scale-invariant quality of the proposed self-supervised multi-resolution learning framework for defect clustering.

Low dose CT is of great interest in these days. Dose reduction raises noise level in projection and decrease image quality in reconstructions. Model based image reconstructions can combine noise statistical model together with prior knowledge in an optimization problem so that significantly reduce noise and artefacts. In this work, we propose a deep learning neural network for low lose CT reconstruction in the sense of penalized weighted least-squares (PWLS) so that a self-supervised learning can be done with no ground-truth information needed. Instead of minimizing cost function for each image, the network learns to minimize the cost function for the whole training set. Features learned can be robust. No iterative processes are needed for real data reconstruction with such a trained network. We carried on our experiments on data from a practical dental CT. Applausible reconstructions with great noise reductions are obtained.

Self-supervised learning has shown potential in fine-grained visual recognition (FGVR). However, existing self-supervised learning methods are often susceptible to irrelevant patterns during training and lack the ability to capture the critical subtle differences in FGVR, leading to suboptimal performance. Moreover, existing approaches focus primarily on uni-modal visual concepts. Despite the emergence of powerful vision-language models (VLMs) in various high-level vision tasks, their potential in self-supervised FGVR remains largely unexplored. To this end, we propose a novel self-supervised learning (LearnMat) framework, that effectively filters out irrelevant feature interference and extracts more important and subtle discriminative features during training. Specifically, LearnMat consists of two key modules: the semantic awareness module (SAM) and the insight extraction module (IEM). In the SAM, we introduce a novel vision-language-grounded semantic distillation strategy using a corpus of generic, category-agnostic textual attributes, that injects explicit semantic constraints into self-supervised training and improves robustness to background interference. Complementarily, the IEM exploits gradient-based signals from the input image to highlight subtle differences and localize key discriminative regions, mitigating inter-class similarity and intra-class variation, and enhancing fine-grained discrimination. Extensive experiments across multiple popular FGVR datasets show that LearnMat significantly outperforms recent state-of-the-art methods, highlighting its marked effectiveness. Our code is avaliable at https://github.com/Heng-CHY/LearnMat.