Using Convolutional Neural Networks to Efficiently Extract Immense Phenological Data From Community Science Images

Deep learning (also known as deep structured learning, hierarchical learning, or deep machine learning) is a branch of machine learning based on a set of algorithms that attempt to model high-level abstractions in data by using a deep graph with multiple processing layers, composed of multiple linear and nonlinear transformations. Deep learning has been characterized as a class of machine learning algorithms with the following characteristics [257]: • They use a cascade of many layers of nonlinear processing units for feature extraction and transformation. Each successive layer uses the output from the previous layer as input. The algorithms may be supervised or unsupervised and applications include pattern analysis (unsupervised) and classification (supervised). • They are based on the (unsupervised) learning of multiple levels of features or representations of the data. Higher-level features are derived from lower-level features to form a hierarchical representation. • They are part of the broader machine learning field of learning representations of data. • They learn multiple levels of representations that correspond to different levels of abstraction; the levels form a hierarchy of concepts. These definitions have in common: multiple layers of nonlinear processing units and the supervised or unsupervised learning of feature representations in each layer, with the layers forming a hierarchy from low-level to high-level features. Various deep learning architectures such as deep neural networks, convolutional deep neural networks, deep belief networks (DBN), and recurrent neural networks have been applied to fields like computer vision, automatic speech recognition, natural language processing, audio recognition, and bioinformatics where they have been shown to produce state-of-the-art results on various tasks. In this chapter we start in Section 7.1 with an introduction, giving a brief history of this field, the relevant literature, and its applications. Then we study some basic concepts of deep learning such as convolutional neural networks, recurrent neural networks, backpropagation algorithm, restricted Boltzmann machines, and deep learning networks in Section 7.2. Then we illustrate three examples for Apache Spark implementation for mobile big data (MBD), user moving pattern extraction, and combination with nonparametric Bayesian learning, respectively, in Sections 7.3 through 7.5. Finally, we have summary in Section 7.6.

Background: Modern diagnostic pathology workflows involve the integration of histomorphologic, immunohistochemical (IHC), and molecular data to reach a final diagnosis. Recently, advances in deep learning have revolutionized pathology by providing the prospect for expert-level autonomous image analysis tools. Despite recent innovations in deep learning, integrating histomorphologic and molecular information found on respective H&E- and IHC-stained tissue sections still remains a challenge. Here, we aim to address this issue by incorporating computer vision tools, including deep learning and scale-invariant feature transform (SIFT), to align H&E-stained sections with accompanying IHC studies for automated subclassification of gliomas. Methods: To test the workflow, we trained the publicly available VGG19 convolutional neural network (CNN) using patches of pathologist-annotated H&E-stained WSIs to recognize histological patterns of 16 common tissue and brain tumor classes, including diffuse glioma, meningioma, metastatic carcinoma, and schwannoma. To complement the histomorphologic analysis, we optimized several deep learning classifiers, including Mask R-CNN, to recognize various IHC markers, such as Ki-67, IDH1-R132H and ATRX, relevant for molecular subclassification of gliomas. For the integrated analysis, we employed SIFT to align lesional regions of H&E and IHC images. Results: The histomorphologic classifier excelled at classification with accuracies of 100% for glioma, meningioma and metastatic carcinoma, and 93% for schwannoma (n = 125). The Mask R-CNN was tested on 147 images generated from 34 brain tumor Ki-67 WSIs and showed a high concordance with aggregate pathologists’ estimates (n = 3 assessors; y = 0.9712x -1.945, r = 0.9750). Using the deep learning classifiers and SIFT, we were able to identify tumor regions from the H&E images and align them to their corresponding IHC images. This resulted in significant improvement of ATRX and IDH1-R132H quantification compared to unaligned WSIs. Finally, SIFT and partially sampling the lesional regions decreased computational time significantly without compromising accuracy. Conclusion: SIFT can work in concert with deep learning tools to help examine the histomorphologic and molecular patterns of various brain tumors and provide a framework for integrating deep learning tools to provide automated diagnoses. Citation Format: Michael K. Lee, Kevin Faust, Madhumitha Rabindranath, Phedias Diamandis. Automating integration of histomorphologic and immunohistochemistry data using computer vision tools [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2022; 2022 Apr 8-13. Philadelphia (PA): AACR; Cancer Res 2022;82(12_Suppl):Abstract nr 455.

Application of Deep Learning Convolutional Neural Networks for Internal Tablet Defect Detection: High Accuracy, Throughput, and Adaptability

This paper presents a brief overview of vibration-based damage identification studies based on Deep Learning (DL) in civil engineering structures. The presence, type, size, and propagation of structural damage on civil infrastructure have always been a topic of research. In the last couple of decades, there has been a significant shift in the damage detection paradigm when the advancements in sensing and computing technologies met with the ever-expanding use of artificial neural network algorithms. Machine-Learning (ML) tools enabled researchers to implement more feasible and faster tools in damage detection applications. When an artificial neural network has more than three layers, it is typically considered as a ?deep? learning network. Being an important accomplishment of the ML era, DL tools enable complex systems which are made of several layers to learn implementations of data with outstanding categorization and compartmentalization capability. In fact, with proper training, a DL tool can operate directly with the unprocessed raw data and help the algorithm produce output data. Competitive capabilities like this led DL algorithms perform very well in complicated problems by dividing a relatively large problem into much smaller and more manageable portions. Specifically for damage identification and localization on civil infrastructure, Convolutional Neural Networks (CNNs) and Unsupervised Pretrained Networks (UPNs) are the known DL tools published in the literature. This paper presents an overview of these studies.

Pathology reports are a primary source of information for cancer registries which process high volumes of free-text reports annually. Information extraction and coding is a manual, labor-intensive process. In this study, we investigated deep learning and a convolutional neural network (CNN), for extracting ICD-O-3 topographic codes from a corpus of breast and lung cancer pathology reports. We performed two experiments, using a CNN and a more conventional term frequency vector approach, to assess the effects of class prevalence and inter-class transfer learning. The experiments were based on a set of 942 pathology reports with human expert annotations as the gold standard. CNN performance was compared against a more conventional term frequency vector space approach. We observed that the deep learning models consistently outperformed the conventional approaches in the class prevalence experiment, resulting in micro- and macro-F score increases of up to 0.132 and 0.226, respectively, when class labels were well populated. Specifically, the best performing CNN achieved a micro-F score of 0.722 over 12 ICD-O-3 topography codes. Transfer learning provided a consistent but modest performance boost for the deep learning methods but trends were contingent on the CNN method and cancer site. These encouraging results demonstrate the potential of deep learning for automated abstraction of pathology reports.

As the network goes deep into all aspects of people''s lives, the number and the complexity of network traffic is increasing, and traffic classification becomes more and more important. How to classify them effectively is an important prerequisite for network management and planning, and ensuring network security. With the continuous development of deep learning, more and more traffic classification begins to use it as the main method, which achieves better results than traditional classification methods. In this paper, we provide a comprehensive review of network traffic classification based on deep learning. Firstly, we introduce the research background and progress of network traffic classification. Then, we summarize and compare traffic classification based on deep learning such as stack autoencoder, one-dimensional convolution neural network, two-dimensional convolution neural network, three-dimensional convolution neural network, long short-term memory network and Deep Belief Networks. In addition, we compare traffic classification based on deep learning with other methods such as based on port number, deep packets detection and machine learning. Finally, the future research directions of network traffic classification based on deep learning are prospected.

PurposeThe aim of this study was to analyse and review deep learning convolutional neural networks for detecting and diagnosing early-stage dental caries on periapical radiographs.Materials and MethodsIn order to conduct this review, the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines were followed. Studies published from 2015 to 2021 under the keywords (deep convolutional neural network) AND (caries), (deep learning caries) AND (convolutional neural network) AND (caries) were systematically reviewed.ResultsWhen dental caries is improperly diagnosed, the lesion may eventually invade the enamel, dentin, and pulp tissue, leading to loss of tooth function. Rapid and precise detection and diagnosis are vital for implementing appropriate prevention and treatment of dental caries. Radiography and intraoral images are considered to play a vital role in detecting dental caries; nevertheless, studies have shown that 20% of suspicious areas are mistakenly diagnosed as dental caries using this technique; hence, diagnosis via radiography alone without an objective assessment is inaccurate. Identifying caries with a deep convolutional neural network-based detector enables the operator to distinguish changes in the location and morphological features of dental caries lesions. Deep learning algorithms have broader and more profound layers and are continually being developed, remarkably enhancing their precision in detecting and segmenting objects.ConclusionClinical applications of deep learning convolutional neural networks in the dental field have shown significant accuracy in detecting and diagnosing dental caries, and these models hold promise in supporting dental practitioners to improve patient outcomes.

Alzheimer's disease (AD) is a progressive brain disease. The goal of this study is to provide a new computer-vision based technique to detect it in an efficient way. The brain-imaging data of 98AD patients and 98 healthy controls was collected using data augmentation method. Then, convolutional neural network (CNN) was used, CNN is the most successful tool in deep learning. An 8-layer CNN was created with optimal structure obtained by experiences. Three activation functions (AFs): sigmoid, rectified linear unit (ReLU), and leaky ReLU. The three pooling-functions were also tested: average pooling, max pooling, and stochastic pooling. The numerical experiments demonstrated that leaky ReLU and max pooling gave the greatest result in terms of performance. It achieved a sensitivity of 97.96%, a specificity of 97.35%, and an accuracy of 97.65%, respectively. In addition, the proposed approach was compared with eight state-of-the-art approaches. The method increased the classification accuracy by approximately 5% compared to state-of-the-art methods.

1 - Congruence of deep learning in biomedical engineering: future prospects and challenges

The use of machine learning (ML) has been increasing rapidly in the medical imaging field, including computer-aided diagnosis (CAD), radiomics, and medical image analysis. Recently, an ML area called deep learning emerged in the computer vision field and became very popular in many fields. It started from an event in late 2012, when a deep-learning approach based on a convolutional neural network (CNN) won an overwhelming victory in the best-known worldwide computer vision competition, ImageNet Classification. Since then, researchers in virtually all fields, including medical imaging, have started actively participating in the explosively growing field of deep learning. In this paper, the area of deep learning in medical imaging is overviewed, including (1) what was changed in machine learning before and after the introduction of deep learning, (2) what is the source of the power of deep learning, (3) two major deep-learning models: a massive-training artificial neural network (MTANN) and a convolutional neural network (CNN), (4) similarities and differences between the two models, and (5) their applications to medical imaging. This review shows that ML with feature input (or feature-based ML) was dominant before the introduction of deep learning, and that the major and essential difference between ML before and after deep learning is the learning of image data directly without object segmentation or feature extraction; thus, it is the source of the power of deep learning, although the depth of the model is an important attribute. The class of ML with image input (or image-based ML) including deep learning has a long history, but recently gained popularity due to the use of the new terminology, deep learning. There are two major models in this class of ML in medical imaging, MTANN and CNN, which have similarities as well as several differences. In our experience, MTANNs were substantially more efficient in their development, had a higher performance, and required a lesser number of training cases than did CNNs. "Deep learning", or ML with image input, in medical imaging is an explosively growing, promising field. It is expected that ML with image input will be the mainstream area in the field of medical imaging in the next few decades.

Deep learning (DL) has shown explosive growth in its application to bioinformatics and has demonstrated thrillingly promising power to mine the complex relationship hidden in large-scale biological and biomedical data. A number of comprehensive reviews have been published on such applications, ranging from high-level reviews with future perspectives to those mainly serving as tutorials. These reviews have provided an excellent introduction to and guideline for applications of DL in bioinformatics, covering multiple types of machine learning (ML) problems, different DL architectures, and ranges of biological/biomedical problems. However, most of these reviews have focused on previous research, whereas current trends in the principled DL field and perspectives on their future developments and potential new applications to biology and biomedicine are still scarce. We will focus on modern DL, the ongoing trends and future directions of the principled DL field, and postulate new and major applications in bioinformatics.

We consider the question whether synchronization/alignment methods are still useful/necessary in the context of side-channel attacks exploiting deep learning algorithms. While earlier works have shown that such methods/algorithms have a remarkable tolerance to misaligned measurements, we answer positively and describe experimental case studies of side-channel attacks against a key transportation layer and an AES S-box where such a preprocessing remains beneficial (and sometimes necessary) to perform efficient key recoveries. Our results also introduce generalized residual networks as a powerful alternative to other deep learning tools (e.g., convolutional neural networks and multilayer perceptrons) that have been considered so far in the field of side-channel analysis. In our experimental case studies, it outperforms the other three published state-of-the-art neural network models for the data sets with and without alignment, and it even outperforms the published optimized CNN model with the public ASCAD data set. Conclusions are naturally implementation-specific and could differ with other data sets, other values for the hyper-parameters, other machine learning models and other alignment techniques.

Traditional methods for classification of soil types are time consuming, invasive and expensive. A non-invasive method like ground penetrating radar (GPR) provides a suitable way to classify soil types based on its electromagnetic properties. Deep learning algorithms have proven to be an effective tool for features extraction of GPR data. A deep convolutional neural network (CNN) model for automatic classification of soil types is proposed. A synthetic dataset is created using gprMax and used to train and validate the proposed CNN model. The proposed model shows good performance in classifying 7 different soil types from GPR B-Scan images. Upon testing the model on new and unseen data, its accuracy is found to be 97%.

In recent years, deep learning (DL) has revolutionized medical imaging, offering notable promise in orthopedic diagnostics. This systematic review explores how DL, particularly Convolutional Neural Networks (CNNs) and Generative Adversarial Networks (GANs), are applied to detect and analyze orthopedic anomalies such as fractures, spinal deformities, and foot deformities. We reviewed 63 peer-reviewed studies published between 2017 and 2025, analyzing their methodologies, datasets, performance metrics, and clinical relevance. The findings reveal significant advancements in fracture classification and vertebral labeling, though challenges persist for subtle anomalies and less-represented deformities. Despite encouraging results, limitations include small sample sizes, lack of external validation especially for rare pathologies. We conclude by identifying research gaps and proposing future directions for developing robust, clinically integrated DL tools to enhance diagnostic accuracy, data diversity, and anomaly detection in complex orthopedic scenarios.

An increased posterior tibial slope (PTS) corresponds with an increased risk of graft failure after anterior cruciate ligament (ACL) reconstruction (ACLR). Validated methods of manual PTS measurements are subject to potential interobserver variability and can be inefficient on large datasets. To develop a deep learning artificial intelligence technique for automated PTS measurement from standard lateral knee radiographs. It was hypothesized that this deep learning tool would be able to measure the PTS on a high volume of radiographs expeditiously and that these measurements would be similar to previously validated manual measurements. Cohort study (diagnosis); Level of evidence, 2. A deep learning U-Net model was developed on a cohort of 300 postoperative short-leg lateral radiographs from patients who underwent ACLR to segment the tibial shaft, tibial joint surface, and tibial tuberosity. The model was trained via a random split after an 80 to 20 train-validation scheme. Masks for training images were manually segmented, and the model was trained for 400 epochs. An image processing pipeline was then deployed to annotate and measure the PTS using the predicted segmentation masks. Finally, the performance of this combined pipeline was compared with human measurements performed by 2 study personnel using a previously validated manual technique for measuring the PTS on short-leg lateral radiographs on an independent test set consisting of both pre- and postoperative images. The U-Net semantic segmentation model achieved a mean Dice similarity coefficient of 0.885 on the validation cohort. The mean difference between the human-made and computer-vision measurements was 1.92° (σ = 2.81° [P = .24]). Extreme disagreements between the human and machine measurements, as defined by ≥5° differences, occurred <5% of the time. The model was incorporated into a web-based digital application front-end for demonstration purposes, which can measure a single uploaded image in Portable Network Graphics format in a mean time of 5 seconds. We developed an efficient and reliable deep learning computer vision algorithm to automate the PTS measurement on short-leg lateral knee radiographs. This tool, which demonstrated good agreement with human annotations, represents an effective clinical adjunct for measuring the PTS as part of the preoperative assessment of patients with ACL injuries.