Efficient deep neural networks for cancer detection on histopathology combining attention and image downsampling.

Background: Tumor infiltrating lymphocytes (TILs) are prognostic and predictive biomarkers in colorectal cancer and are associated with improved prognosis and response to immunotherapy. While TILs are routinely assessed by pathologists, a standardized technique (immunoSEQ, Adaptive Biotechnologies) that leverages targeted next-generation sequencing can also be used to quantify and characterize the T-cell receptor (TCR) repertoire of individual colorectal cancers. In a large, population-based study of incident colorectal cancer, the host immune responses were measured by an expert pathologist and ImmunoSEQ to understand the relationships between TILs, TCRs/cell and specific subgroups of colorectal cancer. Methods: Incident cases of adenocarcinoma of the colon or rectum from the Molecular Epidemiology of Colorectal Cancer (MECC) study included 1,000 cancers that were uniformly evaluated for TILs and other histopathologic features by one pathologist. FFPE-derived DNA from microdissected tumor tissue was extracted and sequenced using ImmunoSEQ analysis for the same 1,000 individuals. A resulting quantitative metric from this assay includes TCRs/cell, a measure of rearranged T cell quantity relative to all nucleated cells in a tumor sample. Gene expression in snap-frozen tissue available from 342/1,000 MECC colorectal cancers was measured with Affymetrix Human Genome U133 Arrays (U133A and U133 Plus2.0) as previously described. CMS classification was performed using the R package 3.5.1, CMS classifier, randomForest 4.6-14. Multivariate analysis assessed CMS by age, gender, TILs/HPF, TCRs/cell, MSI status, BRAF and KRAS mutational status. Results: TILs/HPF and TCRs/cell were significantly correlated among all 1000 cases (r=0.5, p<0.001). Among the 342 cases with available expression profiles, CMS1 constituted 12.0% of all CRC, with CMS2 (41.8%), CMS3 (8.5%), and CMS4 (13.7%) and unclassified (24%) representing the remaining distribution. There were statistically significant differences in the molecular and histopathologic features of colorectal cancers by CMS subgroups. MSI-H tumors were most frequently observed within CMS1 cancers (56.6% of CMS1 were MSI-H), with lower representation among CMS2 (1.5%), CMS3 (10%), CMS4 (3.5%), and unclassified CRC (9.5%) (p<0.0001). In addition, BRAF positive tumors were more frequently observed within the CMS1 group (12.2%, p =0.0065) and KRAS positive tumors within the CMS3 group (31%, p<0.0001). Consistent with prior reports, TILs/HPF were significantly higher in the CMS1 group (mean=7.7, p<0.0001). Similar statistically significant trends were observed across classes for TCRs/cell (mean=0.16, p=0.04). Conclusions: Subtypes of CRC have distinct histopathologic and molecular features that can be distinguished by expression profiles and immunoSEQ. Citation Format: Marilena Melas, Charalampos Lazaris, Stephanie L. Schmit, Asaf Maoz, Rebeca Sanz Pamplona, Chenxu Qu, Joel K. Greenson, Rork Kuick, Flavio Lejbkowicz, Hedy S. Rennert, Christopher P. Walker, Chase M. Bowen, Diane M. Da Silva, W. Martin Kast, Gregory E. Idos, Kevin J. McDonnell, Victor Moreno, Gad Rennert, Stephen B. Gruber. Tumor infiltrating lymphocytes, immunoSeq, and CMS classification in the molecular epidemiology of colorectal cancer study [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2019; 2019 Mar 29-Apr 3; Atlanta, GA. Philadelphia (PA): AACR; Cancer Res 2019;79(13 Suppl):Abstract nr 2332.

BackgroundHistopathology image analysis is a gold standard for cancer recognition and diagnosis. Automatic analysis of histopathology images can help pathologists diagnose tumor and cancer subtypes, alleviating the workload of pathologists. There are two basic types of tasks in digital histopathology image analysis: image classification and image segmentation. Typical problems with histopathology images that hamper automatic analysis include complex clinical representations, limited quantities of training images in a dataset, and the extremely large size of singular images (usually up to gigapixels). The property of extremely large size for a single image also makes a histopathology image dataset be considered large-scale, even if the number of images in the dataset is limited.ResultsIn this paper, we propose leveraging deep convolutional neural network (CNN) activation features to perform classification, segmentation and visualization in large-scale tissue histopathology images. Our framework transfers features extracted from CNNs trained by a large natural image database, ImageNet, to histopathology images. We also explore the characteristics of CNN features by visualizing the response of individual neuron components in the last hidden layer. Some of these characteristics reveal biological insights that have been verified by pathologists. According to our experiments, the framework proposed has shown state-of-the-art performance on a brain tumor dataset from the MICCAI 2014 Brain Tumor Digital Pathology Challenge and a colon cancer histopathology image dataset.ConclusionsThe framework proposed is a simple, efficient and effective system for histopathology image automatic analysis. We successfully transfer ImageNet knowledge as deep convolutional activation features to the classification and segmentation of histopathology images with little training data. CNN features are significantly more powerful than expert-designed features.

The field of histopathological image analysis has evolved significantly with the advent of digital pathology, leading to the development of automated models capable of classifying tissues and structures within diverse pathological images. Artificial intelligence algorithms, such as convolutional neural networks, have shown remarkable capabilities in pathology image analysis tasks, including tumor identification, metastasis detection, and patient prognosis assessment. However, traditional manual analysis methods have generally shown low accuracy in diagnosing colorectal cancer using histopathological images. This study investigates the use of AI in image classification and image analytics using histopathological images using the histogram of oriented gradients method. The study develops an AI-based architecture for image classification using histopathological images, aiming to achieve high performance with less complexity through specific parameters and layers. In this study, we investigate the complicated state of histopathological image classification, explicitly focusing on categorizing nine distinct tissue types. Our research used open-source multi-centered image datasets that included records of 100.000 non-overlapping images from 86 patients for training and 7180 non-overlapping images from 50 patients for testing. The study compares two distinct approaches, training artificial intelligence-based algorithms and manual machine learning models, to automate tissue classification. This research comprises two primary classification tasks: binary classification, distinguishing between normal and tumor tissues, and multi-classification, encompassing nine tissue types, including adipose, background, debris, stroma, lymphocytes, mucus, smooth muscle, normal colon mucosa, and tumor. Our findings show that artificial intelligence-based systems can achieve 0.91 and 0.97 accuracy in binary and multi-class classifications. In comparison, the histogram of directed gradient features and the Random Forest classifier achieved accuracy rates of 0.75 and 0.44 in binary and multi-class classifications, respectively. Our artificial intelligence-based methods are generalizable, allowing them to be integrated into histopathology diagnostics procedures and improve diagnostic accuracy and efficiency. The CNN model outperforms existing machine learning techniques, demonstrating its potential to improve the precision and effectiveness of histopathology image analysis. This research emphasizes the importance of maintaining data consistency and applying normalization methods during the data preparation stage for analysis. It particularly highlights the potential of artificial intelligence to assess histopathological images.

Every year, millions of women across the globe are diagnosed with breast cancer (BC), an illness that is both common and potentially fatal. To provide effective therapy and enhance patient outcomes, it is essential to make an accurate diagnosis as soon as possible. In recent years, deep-learning (DL) approaches have shown great effectiveness in a variety of medical imaging applications, including the processing of histopathological images. Using DL techniques, the objective of this study is to recover the detection of BC by merging qualitative and quantitative data. Using deep mutual learning (DML), the emphasis of this research was on BC. In addition, a wide variety of breast cancer imaging modalities were investigated to assess the distinction between aggressive and benign BC. Based on this, deep convolutional neural networks (DCNNs) have been established to assess histopathological images of BC. In terms of the Break His-200×, BACH, and PUIH datasets, the results of the trials indicate that the level of accuracy achieved by the DML model is 98.97%, 96.78, and 96.34, respectively. This indicates that the DML model outperforms and has the greatest value among the other methodologies. To be more specific, it improves the results of localization without compromising the performance of the classification, which is an indication of its increased utility. We intend to proceed with the development of the diagnostic model to make it more applicable to clinical settings.

Deep-Hipo: Multi-scale receptive field deep learning for histopathological image analysis.

<div>AbstractPurpose:<p>Histopathology evaluation is the gold standard for diagnosing clear cell (ccRCC), papillary, and chromophobe renal cell carcinoma (RCC). However, interrater variability has been reported, and the whole-slide histopathology images likely contain underutilized biological signals predictive of genomic profiles.</p>Experimental Design:<p>To address this knowledge gap, we obtained whole-slide histopathology images and demographic, genomic, and clinical data from The Cancer Genome Atlas, the Clinical Proteomic Tumor Analysis Consortium, and Brigham and Women's Hospital (Boston, MA) to develop computational methods for integrating data analyses. Leveraging these large and diverse datasets, we developed fully automated convolutional neural networks to diagnose renal cancers and connect quantitative pathology patterns with patients' genomic profiles and prognoses.</p>Results:<p>Our deep convolutional neural networks successfully detected malignancy (AUC in the independent validation cohort: 0.964–0.985), diagnosed RCC histologic subtypes (independent validation AUCs of the best models: 0.953–0.993), and predicted stage I ccRCC patients' survival outcomes (log-rank test <i>P</i> = 0.02). Our machine learning approaches further identified histopathology image features indicative of copy-number alterations (AUC > 0.7 in multiple genes in patients with ccRCC) and tumor mutation burden.</p>Conclusions:<p>Our results suggest that convolutional neural networks can extract histologic signals predictive of patients' diagnoses, prognoses, and genomic variations of clinical importance. Our approaches can systematically identify previously unknown relations among diverse data modalities.</p></div>

Nuclei Segmentation is a very essential and intermediate step for automatic cancer detection from H and E stained histopathology images. In the recent advent, the rise of Convolutional Neural Network (CNN), has enabled researchers to detect nuclei automatically from histopathology images with higher accuracy. However, the performance of automatic nuclei segmentation by CNN is fraught with overfitting, due to very less number of annotated segmented images available. Indeed, we find that the problem of nuclei segmentation is an unsupervised problem, because still now there is no automatic tool available which can make annotated images (nuclei segmented images) accurately, to the best of our knowledge. In this research article, we present a Logarithmic-Base2 of Gaussian (Log-Base2-G) Kernel which has the ability to track only the nuclei portions automatically from Colorectal Cancer H and E stained histopathology images. First, Log-Base2-G Kernel is applied to the input images. Thereafter, we apply an adaptive Canny Edge detector, in order to segment only the nuclei edges from H and E stained histopathology images. Experimental results revealed that our proposed method achieved higher accuracy and F1 score, without the help of any annotated data which is a significant improvement. We have used two different datasets (Con-SeP dataset, and Glass-contest dataset, both contains Colorectal Cancer histopathology images) to check the effectiveness and validity of our proposed method. These results have shown that our proposed method outperformed other image processing or unsupervised methods both qualitatively and quantitatively.

Pathological examinations play a critical role in the diagnosis process. Pathologists analyze biopsies to make a diagnosis. However, detection of nuclei in histopathology images is a costly procedure in terms of time. Because of the complexity of histopathology images, different observers might reach different conclusions. Recently, automatic digital pathology, which is faster therefore beneficial for patients and pathologists, draw many attention for research and clinical practice. In comparison to manual image analysis, computerized methods are not affected by the inter-observer variations. In this paper, we automated the nuclei detection process using deep convolutional neural networks (CNN) and simulated annealing to find center coordinates of nuclei in hematoxylin and eosin (H&E) stained histopathology images of colorectal adenocarcinoma.

In this study, a multiscale framework combining the Lifting Wavelet Transform (LWT) and multi-path Convolutional Neural Network (CNN) is proposed to enhance the analysis of histopathological breast cancer images. LWT is employed due to its capability to obtain multi-resolution features that effectively retain key textural details. In particular, a multi-path CNN facilitates the concurrent processing of features at multiple levels of wavelet decompositions, thereby preserving diagnostic information that sole-path CNN models may lose. The approach was evaluated on the BreakHis dataset, which contains 7,638 rated images at varying magnifications, achieving 99.34% test accuracy by combining magnification levels using a Haar wavelet filter. Results obtained from comparative analyses against general CNN models confirm that the new strategy performs better, thereby validating the efficacy of integrating wavelet-based textural extraction and state-of-the-art CNN models for early-stage breast cancer diagnosis.

Breast cancer (BC) is one of the diseases that women suffer most, especially in the world. Routine breast checks are vital for both early diagnosis and early treatment of the person concerned. Computer aided diagnosis systems have also come a long way in being a helpful tool for pathologists during diagnosis. In this work, a novel convolutional neural network (CNN) is proposed for the effective diagnosis of BC from histopathological images. Since classical CNNs have only one input, the network is to use only the raw images from the dataset in the training process. This limits the network from using an extra feature as an input. However, the proposed model has two inputs, unlike classical CNN structures. One input of the network uses histopathological raw images and the other input uses deep features of related images. All of the experimental studies were performed on the widely used BreaKHis dataset. For the test of performance, the accuracy criterion was preferred and the 5-fold cross-validation technique was taken into account. Accuracy scores of 99.94%, 98.94%, 99.05%, and 97.30% were obtained in 40×, 100×, 200× and 400× sub-datasets, respectively. While the results obtained were highly effective values for the diagnosis of BC, they were also far superior to other results reported in the literature.

This research introduces a novel approach utilizing a convolutional neural network (CNN) with MobileNetV2 as the base model for the analysis of histopathological images associated with stomach cancer in the Gastric Intestinal Tract. The study involves a comprehensive seven-layer architecture, incorporating convolution, pooling, dropout, dense, and fully connected layers. The model's performance was assessed in terms of accuracy, precision, recall, F1 score, and ROC curve, revealing promising outcomes. Despite the inherent complexities in predicting histopathological characteristics, MobileNetV2-based CNN achieved notable results with a training accuracy of 91.6%, testing accuracy of 61.47%, precision of 39.6%, recall of 82.3%, and an F1 score of 53.4%. The area under the ROC curve was measured at 0.77, indicating a satisfactory discriminatory capacity. These findings highlight the potential of MobileNetV2-based CNNs in assisting the classification of stomach cancer histopathological images, while also underscoring the need for further optimization to improve generalization and accuracy on unseen data.

ABSTRACTIn recent years, there has been a significant surge in the utilisation of deep learning and machine learning techniques for addressing complex and time‐intensive problems. The significance of employing deep learning becomes increasingly evident as the complexity of the problem increases. In the field of electromagnetics, the utilisation of deep learning techniques has exhibited exceptional efficacy across many applications, especially in wireless communications. In wireless communications, providing a structure that can simultaneously generate multiple beams and beamform them involves complexity and specific constraints. In this article, the time modulation technique is utilised to generate harmonics in the sidebands alongside the fundamental beam in various planar antenna arrays. By demonstrating the nonlinear shaping of the harmonic beams relative to each other, a novel approach is proposed that leverages deep learning techniques for the beamforming of both the fundamental beam and harmonic beams. In this regard, two models are proposed: a deep neural network (DNN) and a convolutional neural network (CNN). The input of CNN is comprised of two‐dimensional patterns of the main beam and harmonics. The input to DNN, on the other hand, includes useful details about the main beam and harmonics, such as their scanning angles, side lobe levels and directivities. The output of the models consists of the time modulation parameters of the array elements, including the pulse width and the pulse delay. The results demonstrate that DNN has achieved better accuracy and a shorter processing time in comprehending the relationship between the time modulation of array elements with different array dimensions and the radiation pattern of the fundamental beam and harmonic beams. Additionally, several samples are presented to evaluate the proposed model. The results demonstrate a high level of accuracy in fundamental beamforming, as well as in harmonic beamforming and beam steering.

Tumor mutational burden (TMB) is an indicator of the efficacy and prognosis of immune checkpoint therapy in colorectal cancer (CRC). In general, patients with higher TMB values are more likely to benefit from immunotherapy. Though whole-exome sequencing is considered the gold standard for determining TMB, it is difficult to be applied in clinical practice due to its high cost. There are also a few DNA panel-based methods to estimate TMB; however, their detection cost is also high, and the associated wet-lab experiments usually take days, which emphasize the need for faster and cheaper alternatives. In this study, we propose a multi-modal deep learning model based on a residual network (ResNet) and multi-modal compact bilinear pooling to predict TMB status (i.e. TMB high (TMB_H) or TMB low(TMB_L)) directly from histopathological images and clinical data. We applied the model to CRC data from The Cancer Genome Atlas and compared it with four other popular methods, namely, ResNet18, ResNet50, VGG19 and AlexNet. We tested different TMB thresholds, namely, percentiles of 10%, 14.3%, 15%, 16.3%, 20%, 30% and 50%, to differentiate TMB_H and TMB_L.For the percentile of 14.3% (i.e. TMB value 20) and ResNet18, our model achieved an area under the receiver operating characteristic curve of 0.817 after 5-fold cross-validation, which was better than that of other compared models. In addition, we also found that TMB values were significantly associated with the tumor stage and N and M stages. Our study shows that deep learning models can predict TMB status from histopathological images and clinical information only, which is worth clinical application.

Rationale and Objectives: Accurate diagnosis and staging of cervical precancers is essential for practical medicine in determining the extent of the lesion extension and determines the most correct and effective therapeutic approach. For accurate diagnosis and staging of cervical precancers, we aim to create a diagnostic method optimized by artificial intelligence (AI) algorithms and validated by achieving accurate and favorable results by conducting a clinical trial, during which we will use the diagnostic method optimized by artificial intelligence (AI) algorithms, to avoid errors, to increase the understanding on interpretation of colposcopy images and improve therapeutic planning. Materials and Methods: The optimization of the method will consist in the development and formation of artificial intelligence models, using complicated convolutional neural networks (CNN) to identify precancers and cancers on colposcopic images. We will use topologies that have performed well in similar image recognition projects, such as Visual Geometry Group Network (VGG16), Inception deep neural network with an architectural design that consists of repeating components referred to as Inception modules (Inception), deeply separable convolutions that significantly reduce the number of parameters (MobileNet) that is a class of Convolutional Neural Network (CNN), Return of investment for machine Learning (ROI), Fully Convolutional Network (U-Net) and Overcomplete Convolutional Network Kite-Net (KiU-Net). Validation of the diagnostic method, optimized by algorithm of artificial intelligence will consist of achieving accurate results on diagnosis and staging of cervical precancers by conducting a randomized, controlled clinical trial, for a period of 17 months. Results: We will validate the computer assisted diagnostic (CAD) method through a clinical study and, secondly, we use various network topologies specified above, which have produced promising results in the tasks of image model recognition and by using this mixture. By using this method in medical practice, we aim to avoid errors, provide precision in diagnosing, staging and establishing the therapeutic plan in cervical precancers using AI. Conclusion: This diagnostic method, optimized by artificial intelligence algorithms and validated by the clinical trial, which we consider “second opinion”, improves the quality standard in diagnosing, staging and establishing therapeutic conduct in cervical precancer.

This article explains how deep learning and machine learning techniques are being utilized to replace humans in a variety of activities, like item classification and sound addition. It emphasizes handwritten text recognition (HTR), which involves computers interpreting handwritten content from sources like paper documents and touchscreens. For recognizing handwritten digits, the study analyzes three algorithms: support vector machines (SVM), multilayer perception (MLP), and convolutional neural networks (CNN). This comparison aims to evaluate the correctness and execution times of different approaches. The evaluation is based on parameters like the dataset used, the number of epochs, algorithm complexity, accuracy, and hardware specifications. The study compares the effectiveness of CNN, Multilayer Perception (MLP), and Support Vector Machines using handwritten digits from the MNIST dataset. It evaluates these algorithms based on criteria such as runtime, complexity, rate of precision, the count of epochs, and the quantity of hidden layers specifically within deep learning techniques. SVM had the highest training data accuracy, but CNN outperformed others on the test dataset. The article’s primary goal is to identify the most precise model for recognizing handwritten digits. It achieves this by assessing the performance of different algorithms and comparing them based on various characteristics.