Overcoming the preferred-orientation problem in cryo-EM with self-supervised deep learning.

The availability of huge annotated datasets and deep learning have led to astounding Natural Language Processing (NLP) progress. However, there is a plethora of low-resource languages, lack the labelled corpora to train the models in the supervision. This kind of digital divide does not allow fair NLP development across language groups. Self-supervised learning (SSL) is a new paradigm in recent years that leverages large amounts of untagged text to learn powerful representations with little or no manual annotation. This paper is a review of self-supervised deep learning models in low-resource NLP tasks. It begins with the definition of the principles of the SSL and the difference between this approach and supervised and unsupervised approaches. We describe why methods such as masked language modeling, contrastive learning, and autoregressive modeling underlie modern pre-trained transformers such as BERT, GPT, and mBERT. A particular interest is paid to multilingual and cross lingual SSL schemes which allow knowledge transfer between high-resource and low resource languages. Low-resource tasks, including machine translation, sentiment analysis, speech-to-text, and information retrieval are reviewed. As per benchmark studies, even in small labelled samples, the SSL models can achieve big gains in accuracy and generalization. Computation cost, representational bias and morphologically rich and under-documented language evaluation, are however, problematic. The researchers observe that the study suffers certain limitations, such as over-reliance on high-resource pretraining information, inequity between linguistic groups, and the difficulty of deploying large-scale SSL models in resource-constrained circumstances. The future directions include lightweight multilingual models, federation of learning in NLP and symbolic linguistic knowledge and combination with SSL. Self-supervised deep learning bridges an essential gap between the high-resource and low-resource languages, and would be a highly valuable step toward inclusive, global NLP innovation.

This paper proposes a novel self-supervised learning framework that uses model reinforcement, REference-free LAtent map eXtraction with MOdel REinforcement (RELAX-MORE), for accelerated quantitative MRI (qMRI) reconstruction. The proposed method uses an optimization algorithm to unroll an iterative model-based qMRI reconstruction into a deep learning framework, enabling accelerated MR parameter maps that are highly accurate and robust. Unlike conventional deep learning methods which require large amounts of training data, RELAX-MORE is a subject-specific method that can be trained on single-subject data through self-supervised learning, making it accessible and practically applicable to many qMRI studies. Using quantitative mapping as an example, the proposed method was applied to the brain, knee and phantom data. The proposed method generates high-quality MR parameter maps that correct for image artifacts, removes noise, and recovers image features in regions of imperfect image conditions. Compared with other state-of-the-art conventional and deep learning methods, RELAX-MORE significantly improves efficiency, accuracy, robustness, and generalizability for rapid MR parameter mapping. This work demonstrates the feasibility of a new self-supervised learning method for rapid MR parameter mapping, that is readily adaptable to the clinical translation of qMRI.

Background Histopathological endpoints are evolving as a treatment target in Inflammatory Bowel Disease (IBD). Use of histology to screen entrants could add value in IBD clinical trials; for example, by refining eligibility criteria to ensure studies recruit patients with definitive active inflammation at the microscopic level. Several histopathological indices have been developed, but the relative complexity of available scores hinders the development of an AI algorithm without large-scale labour-intensive annotation by a pathologist. We aimed to develop computer vision tools to assist decoding the complex clinical disease features at the histological level for both Crohn’s Disease (CD) and Ulcerative Colitis (UC). This will inform understanding of disease pathology and patient stratification to support clinical trial development strategies. Methods A total of 1397 clinically annotated Haematoxylin & Eosin (H&E) images were included from 418 CD and 218 UC patients enrolled in a multicentred longitudinal Study of a Prospective Adult Research Cohort with IBD (SPARC IBD) obtained from the IBD Plexus program of the Crohn’s & Colitis Foundation. We developed an image quality control (QC) algorithm to automatically identify image and tissue processing/staining artefacts negatively impacting analysis (e.g., out-of-focus, tissue folds, overstained regions) and excluded these regions (Fig. 1). Next, a self-supervised learning (SSL) deep learning computer vison model was developed and trained to predict disease relevant features including disease diagnosis and lesional macroscopic appearance (inflammation, erosions and ulcers). Finally, to better understand the model’s predictions, we generated heatmap overlays on the tissue that show regions which the model considers to be most predictive (Fig. 2) and shared these with pathologists for qualitative evaluation. Results We find that the SSL model performs well on different downstream classification tasks such as UC vs CD (area under curve (AUC) = 0.79) and normal vs lesional tissue (AUC = 0.76). Specialist pathologist collaboration further confirmed that the heatmap overlays identified clinically relevant tissue features, including inflammatory cell infiltrates (Fig. 2). Conclusion These encouraging results support further exploration of this deep-learning algorithm to distinguish disease specific characteristics in this set of images from CD and UC patients. Further work is ongoing to validate the heatmap approach on endoscopic scores, we also plan to validate our model on IBD clinical trial datasets.

The image reconstruction process in medical imaging can be treated as solving an inverse problem. The inverse problem is usually solved using time-consuming iterative algorithms with sparsity or other constraints. Recently, deep neural network (DNN)-based methods have been developed to accelerate the inverse-problem-solving process. However, these methods typically adopt supervised learning scheme, which requires ground truths, or labels of the solutions, for training. In many applications, it would be challenging or even impossible to obtain the ground truth, such as the tissue reflectivity function in ultrasound beamforming. In this study, a general framework based on self-supervised learning (SSL) scheme is proposed to train a DNN to solve the inverse problems. In this way, the measurements can be used as both the inputs and the labels during the training of DNN. The proposed SSL method is applied to four typical linear inverse problems for validation, i.e., plane wave ultrasound and photoacoustic image reconstructions, compressed sensing-based synthetic transmit aperture dataset recovery and deconvolution in ultrasound localization microscopy. Results show that, using the proposed framework, the trained DNN can achieve improved reconstruction accuracy with reduced computational time, compared with conventional methods.

To review the latest developments in self-supervised deep learning (DL) techniques for magnetic resonance imaging(MRI) reconstruction, emphasizing their potential to overcome the limitations of supervised methods dependent on fully sampled k-space data. While DL has significantly advanced MRI, supervised approaches require large amounts of fully sampled k-space data for training-a major limitation given the impracticality and expense of acquiring such data clinically. Self-supervised learning has emerged as a promising alternative, enabling model training using only undersampled k-space data, thereby enhancing feasibility and driving research interest. We conducted a comprehensive literature review to synthesize recent progress in self-supervised DL for MRI reconstruction. The analysis focused on methods and architectures designed to improve image quality, reduce scanning time, and address data scarcity challenges, drawing from peer-reviewed publications and technical innovations in the field. Self-supervised DL holds transformative potential for MRI reconstruction, offering solutions to data limitations while maintaining image quality and accelerating scans. Key challenges include robustness across diverse anatomies, standardization of validation, and clinical integration. Future research should prioritize hybrid methodologies, domain-specific adaptations, and rigorous clinical validation. This review consolidates advancements and unresolved issues, providing a foundation for next-generation medical imaging technologies.

As one of the main sources of remote sensing big data, synthetic aperture radar (SAR) can provide all-day and all-weather Earth image acquisition. However, speckle noise in SAR images brings a notable limitation for its big data applications, including image analysis and interpretation. Deep learning has been demonstrated as an advanced method and technology for SAR image despeckling. Most existing deep-learning-based methods adopt supervised learning and use synthetic speckled images to train the despeckling networks. This is because they need clean images as the references, and it is hard to obtain purely clean SAR images in real-world conditions. However, significant differences between synthetic speckled and real SAR images cause the domain gap problem. In other words, they cannot show superior performance for despeckling real SAR images as they do for synthetic speckled images. Inspired by recent studies on self-supervised denoising, we propose an advanced SAR image despeckling method by virtue of Bernoulli-sampling-based self-supervised deep learning, called SSD-SAR-BS. By only using real speckled SAR images, Bernoulli-sampled speckled image pairs (input–target) were obtained as the training data. Then, a multiscale despeckling network was trained on these image pairs. In addition, a dropout-based ensemble was introduced to boost the network performance. Extensive experimental results demonstrated that our proposed method outperforms the state-of-the-art for speckle noise suppression on both synthetic speckled and real SAR datasets (i.e., Sentinel-1 and TerraSAR-X).

Summary Surface obstacles or other technical issues result in gaps in surface seismic recording. To reconstruct these parts of the data, we use self-supervised deep learning. We rearrange a 2D survey line into a 3D data cube, sorting the traces by the source and receiver coordinates. First, we train a conventional 3D U-net and observe a poor performance due to the complexity of 3D reconstructions and the small number of training data samples. Next, we propose a new architecture and training strategy to benefit from the simplicity of 2D reconstructions while using 3D data samples. We show applications on synthetic and real data that have up to eight consecutive missing shots. Our multi-directional deep learning method offers a more accurate reconstruction compared to the 3D U-net and avoids the discontinuities that arise from 2D reconstruction methods.

Subtomogram averaging (STA) is a powerful image processing technique in electron tomography used to determine the 3D structure of macromolecular complexes in their native environments. It is a fast growing technique with increasing importance in structural biology. The computational aspect of STA is very complex and depends on a large number of variables. We noticed a lack of detailed guides for STA processing. Also, current publications in this field often lack a documentation that is practical enough to reproduce the results with reasonable effort, which is necessary for the scientific community to grow. We therefore provide a complete, detailed, and fully reproducible processing protocol that covers all aspects of particle picking and particle alignment in STA. The command line–based workflow is fully based on the popular Dynamo software for STA. Within this workflow, we also demonstrate how large parts of the processing pipeline can be streamlined and automatized for increased throughput. This protocol is aimed at users on all levels. It can be used for training purposes, or it can serve as basis to design user-specific projects by taking advantage of the flexibility of Dynamo by modifying and expanding the given pipeline. The protocol is successfully validated using the Electron Microscopy Public Image Archive (EMPIAR) database entry 10164 from immature HIV-1 virus-like particles (VLPs) that describe a geometry often seen in electron tomography.

Electron cryomicroscopy (cryo-EM) is a powerful tool in structural biology for determining 3D structures of macromolecular complexes. In the previous decade, many technological advancements have been introduced to improve the methodology of single particle cryo-EM, and hence the resolution of its protein reconstructions. These advancements include developments in both software algorithms and microscope hardware. Despite all these developments, the resolution of single particle cryo-EM cannot surpass 1.5 Å, a barrier blocking direct visualization of atomic details of the protein structure. Here, we show that by improving the optical performance of the electron microscope with new hardware, such resolution barrier can be removed, and atomic resolution of 1.25 Å can be reached with an engineered protein apoferritin. The atomic resolution protein structure helped set benchmarks for validating data quality in high resolution cryo-EM. We found that the new microscope also benefited to other proteins with lower contrast and higher flexibility, such as proteasome, chaperonin and fatty acid synthetase. For these proteins, sample properties and behavior were the major limitations. We anticipate that this would be alleviated by introducing other new microscope hardware, so that in the future, time and effort of structural biologists can concentrate on biochemical preparation of the sample, and single particle cryo-EM can routinely yield high quality protein structures at atomic resolution.

In myocardial T1 mapping, undesirable motion poses significant challenges because uncorrected motion can affect T1 estimation accuracy and cause incorrect diagnosis. In this study, we propose and evaluate a motion correction method for myocardial T1 mapping using self-supervised deep learning based registration with contrast separation (SDRAP). A sparse coding based method was first proposed to separate the contrast component from T1 -weighted (T1w) images. Then, a self-supervised deep neural network with cross-correlation (SDRAP-CC) or mutual information as the registration similarity measurement was developed to register contrast separated images, after which signal fitting was performed on the motion corrected T1w images to generate motion corrected T1 maps. The registration network was trained and tested in 80 healthy volunteers with images acquired using the modified Look-Locker inversion recovery (MOLLI) sequence. The proposed SDRAP was compared with the free form deformation (FFD) registration method regarding (1) Dice similarity coefficient (DSC) and mean boundary error (MBE) of myocardium contours, (2) T1 value and standard deviation (SD) of T1 fitting, (3) subjective evaluation score for overall image quality and motion artifact level, and (4) computation time. Results showed that SDRAP-CC achieved the highest DSC of 85.0 ± 3.9% and the lowest MBE of 0.92 ± 0.25 mm among the methods compared. Additionally, SDRAP-CC performed the best by resulting in lower SD value (28.1 ± 17.6 ms) and higher subjective image quality scores (3.30 ± 0.79 for overall quality and 3.53 ± 0.68 for motion artifact) evaluated by a cardiologist. The proposed SDRAP took only 0.52 s to register one slice of MOLLI images, achieving about sevenfold acceleration over FFD (3.7 s/slice).

With the increase in natural disasters, there is a need for better management and planning of post-disaster relief and rescue efforts to minimize loss of lives and property. An Unmanned Aerial Vehicle (UAV)-based system offers the advantage of mobility and a customized flight path that could be utilized to survey areas affected by a disaster. However, the images acquired by UAV must be analysed rapidly with minimum user intervention. In this context, the present work compares the performance of traditional handcrafted feature-based classifiers with that of deep learning methods for classifying images as flooded/non-flooded. The pixels corresponding to water in the UAV aerial image exhibit a characteristic texture as compared to roads, greenery etc. This motivated the use of handcrafted texture features (gray-level co-occurrence matrix (GLCM), local binary patterns (LBP)), which were then used to train a Support Vector Machine (SVM) classifier. Besides, Supervised (ResNet18) and Self-Supervised (Sim-CLR) deep learning methods are also studied for classifying UAV aerial images as flooded/non-flooded. The traditional and deep learning methods are compared on FloodNet dataset containing images acquired after hurricane Harvey. An F1 score of 0.84 for flooded class was obtained with the LBP texture classifier, compared to 0.87 using the self-supervised deep learning method. This result demonstrates that a hand-crafted texture-based classifier performs competitively with deep learning methods. Therefore, a traditional texture classifier could be preferred over deep learning methods for a rapid post- flood scene understanding in UAV aerial images.

Low cost, high performance GPU computing solution for atomic resolution cryoEM single-particle reconstruction

The attenuation of seismic field noise using self-supervised deep learning has gained attention due to its label-free training process. However, common self-supervised methods are limited by the pixelwise independence assumption, which does not align with field seismic noise characteristics, and suffer from signal leakage due to receptive fields containing inherent blind spots or traces. In this paper, we propose a self-supervised random noise attenuation method based on the non-pixelwise independence assumption. By considering the spatial correlation map of field noise, we extend the blind spot to a generalized blind neighborhood, ensuring that the prediction pixel is not influenced by neighboring pixels with noise correlation greater than zero. The blind neighborhood size controls how much spatial correlation is disrupted, allowing our method to handle random noise with varying spatial correlation. Since larger blind neighborhoods may lead to signal loss, we introduce an automatic trade-off between noise correlation disruption and signal preservation during training. Experiments on real seismic noise attenuation (including random and tracewise coherent noise) demonstrate the superiority of our method in destroying the spatial coherence of noise and preventing useful signal leakage.

Seismic data denoising is a critical component of seismic data processing, yet effectively removing erratic noise, characterized by its non-Gaussian distribution and high amplitude, remains a substantial challenge for conventional methods and deep-learning (DL) algorithms. Supervised learning frameworks typically outperform others, but they require pairs of noisy data sets alongside corresponding clean ground truth, which is impractical for real-world seismic data sets. In contrast, unsupervised learning (UL) methods, which do not rely on ground truth during training, often fall short in performance when compared with their supervised or traditional denoising counterparts. Moreover, current unsupervised DL methods fail to address the specific challenges posed by erratic seismic noise adequately. This paper introduces a novel zero-shot unsupervised DL framework designed specifically to mitigate random and erratic noise, with a particular emphasis on blended noise. Drawing inspiration from Noise2Noise (N2N) and data augmentation principles, we develop a robust self-supervised denoising network called robust Noiser2Noiser. Our approach eliminates the need for paired noisy and clean data sets as required by supervised methods or paired noisy data sets as in N2N. Instead, our framework relies solely on the original noisy seismic data set. Our methodology generates two independent recorrupted data sets from the original noisy data set, using one as the input and the other as the training target. Subsequently, we use a DL-based denoiser, denoising convolutional neural network, for training purposes. To address various types of random and erratic noise, the original noisy data set is recorrupted with the same noise type. Detailed explanations for generating training input and target data for blended data are provided. We apply our network to synthetic and real marine data examples, demonstrating significantly improved noise attenuation performance compared with traditional denoising methods and state-of-the-art UL Codes are available on https://github.com/Ji-seismic/N2N_deblending.

Advancements and outlooks in utilizing Convolutional Neural Networks for plant disease severity assessment: A comprehensive review