Deep learning improves macromolecule identification in 3D cellular cryo-electron tomograms.

Cryogenic electron tomography (cryoET) allows visualization of cellular structures in situ. However, anisotropic resolution arising from the intrinsic “missing-wedge” problem has presented major challenges in visualization and interpretation of tomograms. Here, we have developed IsoNet, a deep learning-based software package that iteratively reconstructs the missing-wedge information and increases signal-to-noise ratio, using the knowledge learned from raw tomograms. Without the need for sub-tomogram averaging, IsoNet generates tomograms with significantly reduced resolution anisotropy. Applications of IsoNet to three representative types of cryoET data demonstrate greatly improved structural interpretability: resolving lattice defects in immature HIV particles, establishing architecture of the paraflagellar rod in Eukaryotic flagella, and identifying heptagon-containing clathrin cages inside a neuronal synapse of cultured cells. Therefore, by overcoming two fundamental limitations of cryoET, IsoNet enables functional interpretation of cellular tomograms without sub-tomogram averaging. Its application to high-resolution cellular tomograms should also help identify differently oriented complexes of the same kind for sub-tomogram averaging.

Cytosolic and Membrane-Associated Ribosomes’ mRNA Profiles are Differentially Regulated by Learning and Sleep in Hippocampal Neurons

Cryo-electron tomography is an imaging technique that provides 3D images (tomograms) in situ of cells with sub-nanometer resolution. Typically, the first step in the analysis is to classify the tomogram voxels into different structures, named semantic segmentation. However, the segmentation results are sets of voxels, hindering further quantitative analysis. In this paper, we define and implement algorithms to convert the semantic segmentation of the main structures in a cellular cryo-electron tomogram (membranes, filaments, cytosolic and membrane-bound macromolecules) into specific skeletons, preserving their topological and geometrical information. Additionally, we have defined a metric for comparing segmentations in cryo-ET coming from different methods more robust than the standard DICE. We also demonstrate how this approach can be used to trace cellular features by analyzing several in situ cellular cryo-electron tomograms.

M-free: scoring the reference bias in sub-tomogram averaging and template matching.

Vision-based object positioning is very important in the electronic industry for assembly and inspection tasks. Many methods have been proposed to tackle the problem, either by traditional machine vision or by deep learning (DL) techniques. The traditional methods rely on template matching or feature point correspondence. They are computationally intensive and are easily affected by illumination changes and noise. DL models such as convolutional neural networks (CNNs) are computationally very efficient but are also sensitive against environmental changes. In this article, a deep reinforcement learning (DRL) model based on the Actor-Critic style Proximal Policy Optimization algorithm(s) (AC-PPO) is proposed. The proposed method is applied for the positioning of printed circuit boards (PCBs). The model uses as the current environment the sensed image and the reference template as a guide. It requires only a single manually marked template in the reference image. All possible training images are automatically and randomly generated during the neural network training without human intervention. The proposed reinforcement learning (RL) model is shown to be adaptive to environmental changes, including illumination, noise, de-focusing, and template occlusion, compared with the CNN regressor. Experimental results indicate that the proposed model on average can achieve estimation errors less than 1 pixel in translation and 1° in orientation, with fast evaluation for the real-time PCB positioning task.

In 3D computer graphics, mesh parameterization is a key technique for digital geometry processings such as morphing, shape blending, texture mapping, re-meshing and so on. Most of the previous approaches made use of an identical primitive domain to parameterize a mesh model. In recent works of mesh parameterization, more flexible and attractive methods that can create direct mappings between two meshes have been reported. These mappings are called “cross-parameterization” and typically preserve semantic feature correspondences between target meshes. This paper proposes a novel approach for parameterizing a mesh into another one directly. The main idea of our method is to combine a competitive learning and a least-square mesh techniques. It is enough to give some semantic feature correspondences between target meshes, even if they are in different shapes or in different poses.

To address the problems in the existing video super-resolution methods, such as noise, over smooth and visual artifacts, which are caused by the reliance on limited external training or mismatch of internal similarity patch instances, this study proposes a novel video super-resolution reconstruction algorithm based on deep learning and spatio-temporal feature similarity (DLSS-VSR). The video super-resolution reconstruction mechanism with the joint internal and external constraints is established utilizing the complementary advantages of both external deep correlation mapping learning and internal spatio-temporal nonlocal self-similarity prior constraint. A deep learning model based on deep convolutional neural network is constructed to learn the nonlinear correlation mapping between low-resolution and high-resolution video frame patches. A novel spatio-temporal feature similarity calculation method is proposed, which considers both internal video spatio-temporal self-similarity and external clean nonlocal similarity. For the internal spatio-temporal feature self-similarity, we improve the accuracy and robustness of similarity matching by proposing a similarity measure strategy based on spatio-temporal moment feature similarity and structural similarity. The external nonlocal similarity prior constraint is learned by the patch group-based Gaussian mixture model. The time efficiency for spatio-temporal similarity matching is further improved based on saliency detection and region correlation judgment strategy, which achieves a better tradeoff between super-resolution accuracy and speed. Experimental results demonstrate that the DLSS-VSR algorithm achieves competitive super-resolution quality compared to other state-of-the-art algorithms in both subjective and objective evaluations.

Background/Objectives: Accurate and efficient diagnosis of heart disease through electrocardiogram (ECG) analysis remains a critical challenge in clinical practice due to noise interference, morphological variability, and the complexity of overlapping cardiac signals. Methods: This study presents a comprehensive deep learning (DL) framework that integrates advanced ECG signal segmentation with transfer learning-based classification, aimed at improving diagnostic performance. The proposed ECG segmentation algorithm introduces a distinct and original approach compared to prior research by integrating adaptive preprocessing, histogram-based lead separation, and robust point-tracking techniques into a unified framework. While most earlier studies have addressed ECG image processing using basic filtering, fixed-region cropping, or template matching, our method uniquely focuses on automated and precise reconstruction of individual ECG leads from noisy and overlapping multi-lead images-a challenge often overlooked in previous work. This innovative segmentation strategy significantly enhances signal clarity and enables the extraction of richer and more localized features, boosting the performance of DL classifiers. The dataset utilized in this work of 12 lead-based standard ECG images consists of four primary classes. Results: Experiments conducted using various DL models-such as VGG16, VGG19, ResNet50, InceptionNetV2, and GoogleNet-reveal that segmentation notably enhances model performance in terms of recall, precision, and F1 score. The hybrid VGG19 + SVM model achieved 98.01% and 100% accuracy in multi-class classification, along with average accuracies of 99% and 97.95% in binary classification tasks using the original and reconstructed datasets, respectively. Conclusions: The results highlight the superiority of deep, feature-rich models in handling reconstructed ECG signals and confirm the value of segmentation as a critical preprocessing step. These findings underscore the importance of effective ECG segmentation in DL applications for automated heart disease diagnosis, offering a more reliable and accurate solution.

To solve three-dimensional structures of biological macromolecules in situ, large numbers of particles often need to be picked from cryo-electron tomograms. However, adoption of automated particle-picking methods remains limited because of their technical limitations. To overcome the limitations, we develop DeepETPicker, a deep learning model for fast and accurate picking of particles from cryo-electron tomograms. Training of DeepETPicker requires only weak supervision with low numbers of simplified labels, reducing the burden of manual annotation. The simplified labels combined with the customized and lightweight model architecture of DeepETPicker and accelerated pooling enable substantial performance improvement. When tested on simulated and real tomograms, DeepETPicker outperforms the competing state-of-the-art methods by achieving the highest overall accuracy and speed, which translate into higher authenticity and coordinates accuracy of picked particles and higher resolutions of final reconstruction maps. DeepETPicker is provided in open source with a user-friendly interface to support cryo-electron tomography in situ.

<sec>The realization of fast and high-quality three-dimensional particle-field image characterization is always highly desired in the areas, such as experimental fluid mechanics and biomedicine, for the micro-particle distribution status in a flow-field can characterize the field properties well. In the particle-field image reconstruction and characterization, a wildly used approach at present is the computed tomography. The great advantage of the computed tomography for particle-field image reconstruction lies in the fact that the full particle spatial distribution can be obtained and presented due to multi-angle sampling.</sec><sec>Recently, with the development and application of deep learning technique in computed tomography, the image quality has been greatly improved by the powerful learning ability of a deep learning network. In addition, the deep learning application also makes it possible to speed up the computed tomographic imaging process from sparse-sampling due to the ability of the network to strongly extract image feature. However, sparse-sampling will lead to insufficient acquirement of the object information during sampling for the computed tomography. Therefore, a sort of artifact noise will emerge and be accompanied with the reconstructed images, and thus severely affecting the image quality. As there is no universal network approach that can be applied to all types of objects in the suppression of artifact noise, it is still a challenge in removing the sparse-sampling-induced artifact noise in the computed tomography now.</sec><sec>Therefore, we propose and develop a specific lightweight residual and enhanced convergence neural network (LREC-net) approach for suppressing the artifact noise in the particle-field computed tomography. In this method, the network input dataset is also optimized in signal-to-noise ratio (SNR) in order to reduce the input noise and ensure the effective particle image feature extraction of the network in the imaging process.</sec><sec>In the design of LREC-net architecture, a five-layer lightweight and dual-residual down-sampling is constructed on the basis of typical U-net and Resnet50, making the LREC-net more suitable for the particle-field image reconstruction. Moreover, a fast feature convergence module for rapid particle-field feature acquirement is added to up-sampling process of the network to further promote the network processing efficiency. Apart from the design of LREC-net network itself, the optimization of network input dataset in SNR of images is achieved by finding a fit image reconstruction algorithm that can produce higher-SNR particle images in the computed tomography. This achievement reduces the input noise as much as possible and ensures effective particle-field feature extraction by the network.</sec><sec>The simulation analysis and experimental test demonstrate the effectiveness of the proposed LREC-net method, which involves the evaluations of SNR changes of the input-output images through the network, the proportion of residual artifact noise as ghost-particles (GPP) in the reconstructed images, and the valid-particle loss proportion (PLP). In contrast to the performances of U-net and Resnet50 under the same imaging conditions, all the data in SNR, GPP and PLP show the great improvement of the image quality due to the application of LREC-net method. Meanwhile, the designed LREC-net method also enhances the network running efficiency to a large extent due to the remarkable reduction of training time. Therefore, this work provides a new and effective approach for developing sparse-sampling-based fast and high-quality particle-field computed tomography.</sec>

This paper presents an enhanced ensemble classification framework for Synthetic Aperture Radar (SAR) Automatic Target Recognition (ATR) under diverse operational conditions, including Standard Operating Conditions (SOC) and Extended Operating Conditions (EOC). The proposed method integrates the strengths of Residual Neural Networks (ResNet) replacing Convolutional Neural Networks (CNN), Support Vector Machines (SVM), and template matching, leveraging majority voting to combine their complementary capabilities. The ensemble framework achieves improved robustness and classification accuracy across varied scenarios. The methodology employs ResNet, a deep learning architecture known for its superior feature extraction and classification capabilities, replacing AlexNet to address limitations in generalization and consistency. ResNet demonstrated better performance with average accuracies of 92.67% under SOC and 88.9% under EOC, showing consistent results across all six target classes, as compared to the CNN-based ensemble approach with average accuracies of 90.30% under SOC and 87.22% under EOC. The SVM is employed for its robustness in handling overfitting and classifying features extracted from 16 region properties. Template matching is included for its resilience in challenging conditions where deep learning techniques may underperform. Experimental validation using the MSTAR dataset, a standard benchmark for SAR ATR, highlights the effectiveness of this ensemble approach. The results confirm significant improvements in classification accuracy and robustness over individual classifiers, demonstrating the practical applicability of the ensemble approach to real-world SAR ATR challenges. This research advances SAR ATR by addressing critical challenges, including noise, occlusion, and variations in viewing angles while achieving high classification performance under diverse conditions. The integration of ResNet further enhances the framework’s adaptability and reliability.

In today’s ever-evolving healthcare landscape, our objective is to bring technology closer to human experience. In this paper, a vision-based hand gesture recognition (HGR) system is introduced as a human computer interface (HCI) that transforms natural movements into intuitive medium for device and application manipulation. This innovative approach is designed to empower the elderly and enhance patient monitoring by offering a seamless, nonintrusive interface. By bridging compassion with cutting-edge technology, this approach aims to redefine care and create a more connected, responsive healthcare environment. The study begins by reviewing both traditional and contemporary HGR methodologies, with a focus on vision-based systems, which have shown greater potential for practical applications compared to conventional sensor-based systems. Traditional approaches, such as contour analysis, colour segmentation, and template matching, have demonstrated limitations. In contrast, deep learning approaches have gained mass adaptation in vision-based systems for their ability to learn hierarchical features and better handling complex, dynamic scenarios. However, existing models struggle in real-time performance due to computational inefficiencies and environmental variations. This study introduces a deep learning-based detection framework which enables the practical application of HGR. For example, it supports the development of gesture-based virtual assistants for interacting with digital devices and allows elderly residents in care homes to give instructions to remote caregivers. The framework consists of two deep learning models, one for region of interest (ROI) detection and another for classification. The recognition process begins with capturing RGB images, chosen for versatility, affordability, and compatibility, which then passes through YOLOv8 (You Only Look Once version 8) for detection. The YOLOv8 model is trained on a subset, only 4.5%, of the HaGRID dataset, comprises over 550,000 RGB images, to accurately locate hand regions. Once detection is complete, several image processing techniques are applied to overcome typical HGR constraints. These include ROI extraction and gray scaling for faster computation, resizing for model optimization and distance viability, HSV conversion for lighting independence, and histogram equalization for enhanced feature extraction. The processed image is then fed into a convolutional neural network (CNN) for gesture classification. The network is designed with several convolutional layers, followed by dense layers with ReLU activation to introduce non-linearity, and SoftMax activation for classification. To improve the model's robustness, data augmentation techniques such as random flipping and rotation are applied. The HGR system demonstrates a staggering accuracy of 97% for training & validation. The proposed framework effectively addresses the limitations of conventional HGR systems—many of which have been developed and tested in controlled environments—demonstrating its applicability in real-life situations.

Automated segmentation of breast and fibroglandular tissue (FGT) is required for various computer-aided applications of breast MRI. Traditional image analysis and computer vision techniques, such atlas, template matching, or, edge and surface detection, have been applied to solve this task. However, applicability of these methods is usually limited by the characteristics of the images used in the study datasets, while breast MRI varies with respect to the different MRI protocols used, in addition to the variability in breast shapes. All this variability, in addition to various MRI artifacts, makes it a challenging task to develop a robust breast and FGT segmentation method using traditional approaches. Therefore, in this study, we investigated the use of a deep-learning approach known as "U-net." We used a dataset of 66 breast MRI's randomly selected from our scientific archive, which includes five different MRI acquisition protocols and breasts from four breast density categories in a balanced distribution. To prepare reference segmentations, we manually segmented breast and FGT for all images using an in-house developed workstation. We experimented with the application of U-net in two different ways for breast and FGT segmentation. In the first method, following the same pipeline used in traditional approaches, we trained two consecutive (2C) U-nets: first for segmenting the breast in the whole MRI volume and the second for segmenting FGT inside the segmented breast. In the second method, we used a single 3-class (3C) U-net, which performs both tasks simultaneously by segmenting the volume into three regions: nonbreast, fat inside the breast, and FGT inside the breast. For comparison, we applied two existing and published methods to our dataset: an atlas-based method and a sheetness-based method. We used Dice Similarity Coefficient (DSC) to measure the performances of the automated methods, with respect to the manual segmentations. Additionally, we computed Pearson's correlation between the breast density values computed based on manual and automated segmentations. The average DSC values for breast segmentation were 0.933, 0.944, 0.863, and 0.848 obtained from 3C U-net, 2C U-nets, atlas-based method, and sheetness-based method, respectively. The average DSC values for FGT segmentation obtained from 3C U-net, 2C U-nets, and atlas-based methods were 0.850, 0.811, and 0.671, respectively. The correlation between breast density values based on 3C U-net and manual segmentations was 0.974. This value was significantly higher than 0.957 as obtained from 2C U-nets (P<0.0001, Steiger's Z-test with Bonferoni correction) and 0.938 as obtained from atlas-based method (P=0.0016). In conclusion, we applied a deep-learning method, U-net, for segmenting breast and FGT in MRI in a dataset that includes a variety of MRI protocols and breast densities. Our results showed that U-net-based methods significantly outperformed the existing algorithms and resulted in significantly more accurate breast density computation.

Printed circuit boards (PCBs) are critical for interconnecting various components and allowing them to communicate with each other. It is critical to ensure that there are no small surface defects that can negatively impact PCB production. Therefore, template matching is often used in PCB surface inspection systems. Despite its popularity, this method can be improved because inspecting a PCB with a template is inefficient. Currently, integrating the surface inspection system with the deep learning method is proving to be more effective in solving this problem. This paper examines three popular deep learning object recognition methods in order to determine which one is the most effective in terms of attention. These three models are called Carafe, Empirical Attention, and ResNeSt. The experimental results showed that ResNeSt with split attention networks achieves the greatest accuracy in deep learning PCB surface inspection system with a mean average precision (mAP) of 99.2% and an average recall (AR) of 99.5%. The result of this study would improve the effectiveness of PCB surface inspection in controlling production lines.

The high rate of drug withdrawal from the market due to cardiovascular toxicity or lack of efficacy, the economic burden, and extremely long time before a compound reaches the market, have increased the relevance of human in vitro models like human (patient-derived) pluripotent stem cell (hPSC)-derived engineered heart tissues (EHTs) for the evaluation of the efficacy and toxicity of compounds at the early phase in the drug development pipeline. Consequently, the EHT contractile properties are highly relevant parameters for the analysis of cardiotoxicity, disease phenotype, and longitudinal measurements of cardiac function over time. In this study, we developed and validated the software HAARTA (Highly Accurate, Automatic and Robust Tracking Algorithm), which automatically analyzes contractile properties of EHTs by segmenting and tracking brightfield videos, using deep learning and template matching with sub-pixel precision. We demonstrate the robustness, accuracy, and computational efficiency of the software by comparing it to the state-of-the-art method (MUSCLEMOTION), and by testing it with a data set of EHTs from three different hPSC lines. HAARTA will facilitate standardized analysis of contractile properties of EHTs, which will be beneficial for in vitro drug screening and longitudinal measurements of cardiac function.