A multi-task deep learning-based framework for structural details and corrosion detection in steel bridges

Sea turtles exhibit high migratory rates and occupy a broad range of habitats, which in turn makes monitoring these taxa challenging. Applying deep learning (DL) models to vast image datasets collected from citizen science programs can offer promising solutions to overcome the challenge of monitoring the wide habitats of wildlife, particularly sea turtles. Among DL models, object detection models, such as the You Only Look Once (YOLO) series, have been extensively employed for wildlife classification. Despite their successful application in this domain, detecting objects in images with complex backgrounds, including underwater environments, remains a significant challenge. Recently, instance segmentation models have been developed to address this issue by providing more accurate classification of complex images compared to traditional object detection models. This study compared the performance of two state-of-the-art DL methods namely; the object detection model (YOLOv5) and instance segmentation model (YOLOv5-seg), to detect and classify sea turtles. The images were collected from iNaturalist and Google and then divided into 64% for training, 16% for validation, and 20% for test sets. Model performance during and after finishing training was evaluated by loss functions and various indexes, respectively. Based on loss functions, YOLOv5-seg demonstrated a lower error rate in detecting rather than classifying sea turtles than the YOLOv5. According to mean Average Precision (mAP) values, which reflect precision and recall, the YOLOv5-seg model showed superior performance than YOLOv5. The mAP0.5 and mAP0.5:0.95 for the YOLOv5 model were 0.885 and 0.795, respectively, whereas for the YOLOv5-seg, these values were 0.918 and 0.831, respectively. In particular, based on the loss functions and classification results, the YOLOv5-seg showed improved performance for detecting rather than classifying sea turtles compared to the YOLOv5. The results of this study may help improve sea turtle monitoring in the future.

Deep Learning (DL) techniques have enabled up to 6-fold acceleration in musculoskeletal magnetic resonance imaging (MRI) while preserving diagnostic image quality. Further, improvements in acceleration and generalization require novel approaches. We propose a DL framework that integrates Simultaneous Multislice (SMS) imaging with Parallel Imaging (PI) to enhance current DL-based reconstruction. To advance musculoskeletal Magnetic Resonance Imaging (MRI), by developing a DL reconstruction framework that combines SMS and PI, enabling acceleration of up to 8-fold and beyond, while maintaining image quality suitable for clinicalinterpretation. End-to-End (E2E) DL framework for reconstructing Turbo Spin Echo (TSE) MRI data acquired with SMS and PI acceleration. The method unrolls a proximal gradient algorithm with Nesterov momentum and integrates a novel DL network for joint regularization across simultaneously acquired slices. Slice separation and k-space-to-image reconstruction are unified by embedding the full SMS forward model into the DL architecture. Data Consistency (DC) is modulated to enhance denoising, and a super-resolution module improves image sharpness. The robust DL model was trained on over 200000 slices from 1.5T to 3T scans with diverse acquisitionsettings. The proposed E2E DL model outperforms prior methods at 8-fold and 12-fold acceleration, as measured by PSNR (peak signal-to-noise ratio) and SSIM (structural similarity index measure) metrics. Evaluation on prospectively acquired clinical scans by two radiologists confirms, that image quality and abnormality detection are comparable to standard acquisitions at loweracceleration. We extend state-of-the-art DL reconstruction frameworks by integrating slice separation directly into the model for SMS acquisitions. Our E2E DL approach achieves clinical-grade image quality at 8-fold acceleration across 20 subjects, reducing acquisition time by 27%. Preliminary results suggest potential for further acceleration up to 12-fold, demonstrating significant advancement beyond existing DLtechniques.

<p>Recently, Unmanned Aerial Vehicles (UAVs) equipped with high-resolution imaging sensors have become a viable alternative for ecologists to conduct wildlife censuses, compared to foot surveys. They cause less disturbance by sensing remotely, they provide coverage of otherwise inaccessible areas, and their images can be reviewed and double-checked in controlled screening sessions. However, the amount of data they generate often makes this photo-interpretation stage prohibitively time-consuming.</p><p>In this work, we automate the detection process with deep learning [4]. We focus on counting coastal seabirds on sand islands off the West African coast, where species like the African Royal Tern are at the top of the food chain [5]. Monitoring their abundance provides invaluable insights into biodiversity in this area [7]. In a first step, we obtained orthomosaics from nadir-looking UAVs over six sand islands with 1cm resolution. We then fully labelled one of them with points for four seabird species, which required three weeks for five annotators to do and resulted in over 21,000 individuals. Next, we further labelled the other five orthomosaics, but in an incomplete manner; we aimed for a low number of only 200 points per species. These points, together with a few background polygons, served as training data for our ResNet-based [2] detection model. This low number of points required multiple strategies to obtain stable predictions, including curriculum learning [1] and post-processing by a Markov random field [6]. In the end, our model was able to accurately predict the 21,000 birds of the test image with 90% precision at 90% recall (Fig. 1) [3]. Furthermore, this model required a mere 4.5 hours from creating training data to the final prediction, which is a fraction of the three weeks needed for the manual labelling process. Inference time is only a few minutes, which makes the model scale favourably to many more islands. In sum, the combination of UAVs and machine learning-based detectors simultaneously provides census possibilities with unprecedentedly high accuracy and comparably minuscule execution time.</p><p><img src="https://contentmanager.copernicus.org/fileStorageProxy.php?f=gnp.bc5211f4f60067568601161/sdaolpUECMynit/12UGE&app=m&a=0&c=eeda7238e992b9591c2fec19197f67dc&ct=x&pn=gnp.elif&d=1" alt=""></p><p><em>Fig. 1: Our model is able to predict over 21,000 birds in high-resolution UAV images in a fraction of time compared to weeks of manual labelling.</em></p><p> </p><p>References</p><p>1. Bengio, Yoshua, et al. "Curriculum learning." Proceedings of the 26th annual international conference on machine learning. 2009.</p><p>2. He, Kaiming, et al. "Deep residual learning for image recognition." Proceedings of the IEEE conference on computer vision and pattern recognition. 2016.</p><p>3. Kellenberger, Benjamin, et al. “21,000 Birds in 4.5 Hours: Efficient Large-scale Seabird Detection with Machine Learning.” Remote Sensing in Ecology and Conservation. Under review.</p><p>4. LeCun, Yann, Yoshua Bengio, and Geoffrey Hinton. "Deep learning." nature 521.7553 (2015): 436-444.</p><p>5. Parsons, Matt, et al. "Seabirds as indicators of the marine environment." ICES Journal of Marine Science 65.8 (2008): 1520-1526.</p><p>6. Tuia, Devis, Michele Volpi, and Gabriele Moser. "Decision fusion with multiple spatial supports by conditional random fields." IEEE Transactions on Geoscience and Remote Sensing 56.6 (2018): 3277-3289.</p><p>7. Veen, Jan, Hanneke Dallmeijer, and Thor Veen. "Selecting piscivorous bird species for monitoring environmental change in the Banc d'Arguin, Mauritania." Ardea 106.1 (2018): 5-18.</p>

The image quality of echocardiography is an important factor to affect cardiovascular disease diagnosis. Currently, the deep learning (DL) used in cardiac echocardiogram image quality assess model focus more on evaluating the whole dynamic video, but the outputs revealed less local anatomical details in judging the image quality in heart chambers. This study was aimed to achieve the local part image quality assess, specifically for the five locals in the left ventricle of A4C section for myocardium. The object detection model, YOLOv8 (You Only Look Once), were used to crop five local parts in the left ventricle myocardium of A4C section. Then, the ResNet-18 model was used to evaluate the image quality of each cropped part, that output from score 0 to 3, four quality levels. The YOLOv8 model demonstrated exceptional performance metrics with Precision of 98.77%, Recall of 98.84%, mAP50 of 98.95%, and mAP50-90 of 81.33%. Additionally, the model exhibited an average Inference Time of 215ms per frame. Comparatively, the ResNet-18 model achieved Accuracy scores of 79.34%, 82.41%, 77.82%, 82.33%, and 78.13%, which correspond to the assessment of the left ventricular myocardium in all five local A4C views. The aggregate performance of the ResNet-18 model was characterized by average Macro Precision of 66.77%, Recall of 59.89%, and F1 Score of 59.49%. Furthermore, the model displayed average Micro Precision of 67.42%, Recall of 70.00%, and F1 Score of 69.98%. This study determined the effectiveness of YOLOv8 to find the bounding box of local myocardium and ResNet-18 for real-time automatic quality assessment, and had the potential to improve the efficiency of diagnosis for the doctor using echocardiogram.

<title>Abstract</title> The future of agriculture is intertwined with automation. Accurate fruit detection, yield estimation, and harvest time prediction are crucial for efficient supply chain management by optimizing resources and logistic utilization. Computer vision can automate these tasks to reduce labour costs and improve efficiency by training deep learning models on appropriate data to perform knowledge-based tasks. Although fruit detection has been the focus of literature, yield prediction and harvest time estimation remain practical challenges for farmers. This is particularly important for high-value, highly perishable crops, such as raspberries, where contractual obligations require precise harvest timing. This paper addresses this gap by providing a learning-based framework for raspberry maturity detection and harvest time estimation. For this end, a phenology study of raspberry plants from three different cultivars was carried out and seven development stages were identified based on the BBCH-scale; for the first time, a phenology-based dataset of developing raspberry flowers and fruit was curated and made available publicly, which contains 1,853 high-resolution images and 6,907 manually labelled annotations. A comprehensive benchmark was developed from state-of-the-art object detection models, and finally, a tailored deep learning model capable of real-time inference was established that achieved 92.2% detection accuracy in the vertical farm field test. This paper provides a model that can estimate raspberry yield 28 to 33 days in advance of harvesting, a dataset that can be used by other researchers fpr object detection and segmentation model development, and a framework that can be extended for precise yield and harvest time estimation.

Deep learning (also known as deep structured learning, hierarchical learning, or deep machine learning) is a branch of machine learning based on a set of algorithms that attempt to model high-level abstractions in data by using a deep graph with multiple processing layers, composed of multiple linear and nonlinear transformations. Deep learning has been characterized as a class of machine learning algorithms with the following characteristics [257]: • They use a cascade of many layers of nonlinear processing units for feature extraction and transformation. Each successive layer uses the output from the previous layer as input. The algorithms may be supervised or unsupervised and applications include pattern analysis (unsupervised) and classification (supervised). • They are based on the (unsupervised) learning of multiple levels of features or representations of the data. Higher-level features are derived from lower-level features to form a hierarchical representation. • They are part of the broader machine learning field of learning representations of data. • They learn multiple levels of representations that correspond to different levels of abstraction; the levels form a hierarchy of concepts. These definitions have in common: multiple layers of nonlinear processing units and the supervised or unsupervised learning of feature representations in each layer, with the layers forming a hierarchy from low-level to high-level features. Various deep learning architectures such as deep neural networks, convolutional deep neural networks, deep belief networks (DBN), and recurrent neural networks have been applied to fields like computer vision, automatic speech recognition, natural language processing, audio recognition, and bioinformatics where they have been shown to produce state-of-the-art results on various tasks. In this chapter we start in Section 7.1 with an introduction, giving a brief history of this field, the relevant literature, and its applications. Then we study some basic concepts of deep learning such as convolutional neural networks, recurrent neural networks, backpropagation algorithm, restricted Boltzmann machines, and deep learning networks in Section 7.2. Then we illustrate three examples for Apache Spark implementation for mobile big data (MBD), user moving pattern extraction, and combination with nonparametric Bayesian learning, respectively, in Sections 7.3 through 7.5. Finally, we have summary in Section 7.6.

The images of oral potentially malignant disorders (OPMDs) frequently encounter problems related to visual quality and image distortion, which may lead to serious misdiagnosis and missed diagnosis. This study is to explore the auxiliary effects of different optical image enhancement algorithms on the object detection of OPMDs lesions. Digital images of OPMDs were collected, including white plaques, white stripes, and erosive lesions. The dataset was divided into a training set (6,488 images) and a validation set (2,592 images). Original images were processed using multiscale retinex (MSR), adaptive histogram equalization (AHE), and adaptive contrast enhancement (ACE), respectively. Object detection models based on You Only Look Once version 8 (YOLOv8) were used for lesion detection, and the diagnostic performance was evaluated using 328 images taken at different times. The model performance in the MSR-enhanced image set was superior to that in the original image set, with total accuracy increased for all three lesion types, and the sensitivity of complete correct recognition for complex multi-lesion images improved. Models trained with AHE and ACE preprocessing showed reduced diagnostic performance. Image enhancement algorithms can enhance the visual quality of OPMDs images, and the MSR algorithm is capable of strengthening the object detection ability in the computer vision model. This study provides an approach to reduce the misdiagnosis and missed diagnosis of OPMDs lesions in object detection model.

This study proposes a compact deep learning (DL) architecture and a highly parallelized computing hardware platform to reconstruct the blood flow index (BFi) in diffuse correlation spectroscopy (DCS). We leveraged a rigorous analytical model to generate autocorrelation functions (ACFs) to train the DL network. We assessed the accuracy of the proposed DL using simulated and milk phantom data. Compared to convolutional neural networks (CNN), our lightweight DL architecture achieves 66.7% and 18.5% improvement in MSE for BFi and the coherence factor β, using synthetic data evaluation. The accuracy of rBFi over different algorithms was also investigated. We further simplified the DL computing primitives using subtraction for feature extraction, considering further hardware implementation. We extensively explored computing parallelism and fixed-point quantization within the DL architecture. With the DL model's compact size, we employed unrolling and pipelining optimizations for computation-intensive for-loops in the DL model while storing all learned parameters in on-chip BRAMs. We also achieved pixel-wise parallelism, enabling simultaneous, real-time processing of 10 and 15 autocorrelation functions on Zynq-7000 and Zynq-UltraScale+ field programmable gate array (FPGA), respectively. Unlike existing FPGA accelerators that produce BFi and the β from autocorrelation functions on standalone hardware, our approach is an encapsulated, end-to-end on-chip conversion process from intensity photon data to the temporal intensity ACF and subsequently reconstructing BFi and β. This hardware platform achieves an on-chip solution to replace post-processing and miniaturize modern DCS systems that use single-photon cameras. We also comprehensively compared the computational efficiency of our FPGA accelerator to CPU and GPU solutions.

The aims of this work are (1) to explore deep learning (DL) architectures, spectroscopic input types, and learning designs toward optimal quantification in MR spectroscopy of simulated pathological spectra; and (2) to demonstrate accuracy and precision of DL predictions in view of inherent bias toward the training distribution. Simulated 1D spectra and 2D spectrograms that mimic an extensive range of pathological in vivo conditions are used to train and test 24 different DL architectures. Active learning through altered training and testing data distributions is probed to optimize quantification performance. Ensembles of networks are explored to improve DL robustness and reduce the variance of estimates. A set of scores compares performances of DL predictions and traditional model fitting (MF). Ensembles of heterogeneous networks that combine 1D frequency-domain and 2D time-frequency domain spectrograms as input perform best. Dataset augmentation with active learning can improve performance, but gains are limited. MF is more accurate, although DL appears to be more precise at low SNR. However, this overall improved precision originates from a strong bias for cases with high uncertainty toward the dataset the network has been trained with, tending toward its average value. MF mostly performs better compared to the faster DL approach. Potential intrinsic biases on training sets are dangerous in a clinical context that requires the algorithm to be unbiased to outliers (i.e., pathological data). Active learning and ensemble of networks are good strategies to improve prediction performances. However, data quality (sufficient SNR) has proven as a bottleneck for adequate unbiased performance-like in the case of MF.

Steering angle prediction plays a crucial role in the control of Autonomous Vehicles (AVs) and has garnered the interest of researchers, manufacturers, and insurance companies within the automotive industry. Various Deep Learning (DL) architectures have been utilized to forecast the steering angle of AVs across different scenarios. An examination focusing on steering angle prediction through deep learning algorithms can assist seasoned researchers in pinpointing areas that necessitate further advancement. Additionally, newcomers in the field can utilize this study as a foundation. This paper offers an extensive analysis of the recent progress achieved in DL architectures concerning the steering angle prediction of AVs. A comprehensive taxonomy outlining the application of DL in steering angle prediction of AVs has been developed. The survey provides a succinct synthesis, summary, and analysis of research findings. It is evident that Convolutional Neural Network (CNN) is the preferred choice among researchers for predicting the steering angle of autonomous vehicles, compared to other DL architectures. Furthermore, existing research challenges have been identified. The primary obstacle encountered in DL-based steering angle prediction of AVs is the scarcity of real-world datasets, leading researchers to heavily rely on data generated from simulated environments. Lastly, potential alternative approaches to address the identified research challenges have been suggested, indicating promising avenues for future research endeavours.

Deep learning based classification of breast tumors with shear-wave elastography

Object detection and semantic segmentation are two fundamental techniques of various applications in the fields of Intelligent Vehicles (IV) and Advanced Driving Assistance System (ADAS). Early studies separately handle these two problems. In this paper, inspired by some recent works, we propose a deep neural network model for joint object detection and semantic segmentation. Given an image, an encoder-decoder convolution network extracts a set of feature maps, these feature maps are shared by the detection branch and the segmentation branch to jointly carry out the object detection and semantic segmentation. In the detection branch, we design a PriorBox initialization mechanism to propose more object candidates. In the segmentation branch, we use the multi-scale atrous convolution to explore the global and local semantic information in traffic scenes. Benefiting from the PriorBox Initialization Mechanism (PBIM) and Multi-Scale Atrous Convolution (MSAC), our model presents the competitive performance. In the experiments, we widely compare with several recently-proposed methods on the public Cityscapes dataset, achieving the highest accuracy. In addition, to verify the robustness and generalization of our model, the extension experiments are also conducted on the well-known VOC2012 dataset.

Foreign bodies such as a surgical gauze can be retained in the body after surgery and in some cases cannot be detected by postoperative radiography. The aim of this study was to develop an object detection model capable of postsurgical detection of retained gauze in the body. The object detection model used deep learning using abdominal radiographs, and a phantom study was performed to evaluate the ability of the model to automatically detect retained surgical gauze. The object detection model was constructed using a Single Shot MultiBox Detector (SSD) 300. In total, 268 abdominal phantom images were used: 180 gauze images were used as training data, 20 gauze images were used as validation data, and an additional 34 gauze images and 34 nongauze images were used as test data. To evaluate the performance of the object detection model, a confusion matrix was created and the accuracy and sensitivity were calculated. True-positive (TP) rate, true-negative (TN) rate, false-positive (FP) rate, and false-negative (FN) rate were 0.92, 1.00, 0.00, and 0.08, respectively. Accuracy was 0.96, and sensitivity was 0.92. The object detection model could detect surgical gauze on abdominal phantom images with a high accuracy and sensitivity.

Stomatal observation and automatic stomatal detection are useful analyses of stomata for taxonomic, biological, physiological, and eco-physiological studies. We present a new clearing method for improved microscopic imaging of stomata in soybean followed by automated stomatal detection by deep learning. We tested eight clearing agent formulations based upon different ethanol and sodium hypochlorite (NaOCl) concentrations in order to improve the transparency in leaves. An optimal formulation—a 1:1 (v/v) mixture of 95% ethanol and NaOCl (6–14%)—produced better quality images of soybean stomata. Additionally, we evaluated fixatives and dehydrating agents and selected absolute ethanol for both fixation and dehydration. This is a good substitute for formaldehyde, which is more toxic to handle. Using imaging data from this clearing method, we developed an automatic stomatal detector using deep learning and improved a deep-learning algorithm that automatically analyzes stomata through an object detection model using YOLO. The YOLO deep-learning model successfully recognized stomata with high mAP (~0.99). A web-based interface is provided to apply the model of stomatal detection for any soybean data that makes use of the new clearing protocol.

Multi-modal biometric authentication systems face a critical trade-off: deep learning models achieve high accuracy but require substantial computational resources and lack interpretability making them unsuitable for edge devices and regulated environments. This paper addresses this gap by conducting the first comprehensive empirical comparison of Adaptive Neuro-Fuzzy Inference System (ANFIS) against five state-of-the-art deep learning architectures (CNN, DNN, RNN/LSTM, Transformer) for real-time multi-modal fusion of face, iris, and palmprint biometrics. Our novel contributions include: (1) a systematic evaluation framework demonstrating ANFIS achieves superior accuracy (98.1%) and lowest EER (1.5%) while requiring 93.7% less training time and 91.1% less inference time than Transformers; (2) statistical validation via 10-fold cross-validation proving ANFIS improvements are statistically significant (p<0.05); and (3) an adaptive weight adjustment mechanism enabling dynamic modality reweighting based on input quality. Results on three benchmark datasets (LFW, CASIA-Iris-V4, PolyU) with 20-user real-world validation demonstrate ANFIS as the optimal solution for resource-constrained, explainable biometric systems addressing critical deployment gaps in mobile, IoT, and privacy-sensitive applications.