Intelligent framework for brain tumor grading using advanced feature analysis

Develop a predictive model utilizing weakly supervised deep learning techniques to accurately forecast major pathological response (MPR) in patients with resectable non-small cell lung cancer (NSCLC) undergoing neoadjuvant chemoimmunotherapy (NICT), by leveraging whole slide images (WSIs). This retrospective study examined pre-treatment WSIs from 186 patients with non-small cell lung cancer (NSCLC), using a weakly supervised learning framework. We employed advanced deep learning architectures, including DenseNet121, ResNet50, and Inception V3, to analyze WSIs on both micro (patch) and macro (slide) levels. The training process incorporated innovative data augmentation and normalization techniques to bolster the robustness of the models. We evaluated the performance of these models against traditional clinical predictors and integrated them with a novel pathomics signature, which was developed using multi-instance learning algorithms that facilitate feature aggregation from patch-level probability distributions. Univariate and multivariable analyses confirmed histology as a statistically significant prognostic factor for MPR (P-value< 0.05). In patch model evaluations, DenseNet121 led in the validation set with an area under the curve (AUC) of 0.656, surpassing ResNet50 (AUC = 0.626) and Inception V3 (AUC = 0.654), and showed strong generalization in external testing (AUC = 0.611). Further evaluation through visual inspection of patch-level data integration into WSIs revealed XGBoost's superior class differentiation and generalization, achieving the highest AUCs of 0.998 in training and robust scores of 0.818 in validation and 0.805 in testing. Integrating pathomics features with clinical data into a nomogram yielded AUC of 0.819 in validation and 0.820 in testing, enhancing discriminative accuracy. Gradient-weighted Class Activation Mapping (Grad-CAM) and feature aggregation methods notably boosted the model's interpretability and feature modeling. The application of weakly supervised deep learning to WSIs offers a powerful tool for predicting MPR in NSCLC patients treated with NICT.

Introduction/ Background Cancer is a disease that affects millions worldwide and accurate determination of whether lymph nodes (LNs) near the primary tumor contain metastatic foci is of critical importance for proper patient management. Histopathological evaluation is the only accepted method to make that determination. However, the current standard of care only examines a single central histological section per LN and yields an unacceptable false-negative rate. Aims To help pathologists in their examination we propose a method that extracts textural features from histopathological LN whole slide images (WSI) and then applies support vector machines (SVMs) to automatically identify regions suspicious for metastatic foci. Methods The database consisted of WSI from 44 LNs. Sections were stained with hematoxylin-eosin and examined at 20x (0.45μm resolution). Twenty-eight of the LNs were identified by an expert pathologist as positive for cancer (P), and the remaining sixteen were negative (N). This database was divided into two groups. Group 1 (15P and 5N) was used for training and Group 2 (13P and 11N) was used for testing the classification technique. For all analysis each WSI was divided into non-overlapping 1000 x 1000 pixel sub-images that will be referred to as high-power fields (HPFs). For each LN in Group 1, at least one WSI was annotated by a pathologist to identify rectangular, HPF-scale regions as locally cancerous or locally non-cancerous. From these annotated slides, 924 HPFs (462 P and 462 N) were obtained. For each of these HPFs, statistical features based on gray-level co-occurrence matrices [1] and Law’s texture energy measures [2, 3] were extracted from 9 derived images [4]. The extracted features were submitted to a sequential forward selection (SFS) method [5] to select few non-redundant features providing best class separation (cancerous vs. non-cancerous region). Combinations of the selected features were tested on the 924 HPFs using k-fold cross-validation to find those that produced the best results and consequently to train our SVM-based classifier. In Group 2, WSI were not annotated for cancerous and non-cancerous zones on a HPF scale. Each LN, however, had been labeled by a pathologist as positive or negative for cancer. For each WSI, each section was divided into contiguous HPFs, and those which mainly contain fatty tissue, background, and tears were automatically excluded. Each selected HPFs was classified as cancerous or non-cancerous using the previously trained classifier to obtain the total number of cancer-classified per LN. A receiver operating characteristics (ROC) curve was traced by changing the discriminator threshold (T) used to label the LN as P for cancer as a function of the total number of cancer-classified HPFs. Results During training, 5 Laws features were selected by SFS. Highly satisfactory k-fold cross-validation with a F-score of 0.996 ± 0.005 was obtained using only 2 statistical features computed at different scales. The ROC curve obtained by applying the SVM-classifier to the test set is shown in the next figure. Two valuable operating points can be identified which both guaranteed no false-negative. At T=11 we got 2 false-positives and an optimal F-score of 0.917, and with a more conservative approach, T=1, we got 7 false-positives and a F-score of 0.759. The top-left part of the slide displayed in next figure would have been proposed to the pathologist as the most suspicious region of the cancerous LN.

Background: Prostate cancer (PCa) frequently presents as multifocal lesions with histologic and molecular heterogeneity. A comprehensive understanding of tumor immune responses within the context of the complex PCa microenvironment remains elusive. Through the use of Digital Pathology and innovative Pathomics image analysis tools, tumor infiltrating lymphocytes (TILs) can be mapped, visualized, and quantified in whole slide images (WSIs) of H&amp;E stained PCa slides. We have developed a novel workflow to map and quantify TILs by Gleason histologic grade (Gleason-TIL maps) in whole radical prostatectomy specimens. Approach: WSIs from 211 H&amp;E tissue sections from eight whole radical prostatectomy specimens containing Gleason histologic patterns 3 and 4 were used in the development of the PCa Pathomics workflow. Pathomics tools generated PCa Tumor-TIL and Gleason-TIL maps for each corresponding H&amp;E WSI for all eight prostatectomy specimens. To support tissue sampling for correlative biomarker studies and genomic testing in PCa clinical trials, each Tumor-TIL and Gleason-TIL map was partitioned to quantify total PCa content, Gleason grade (3 vs 4/5) content, and intratumoral and peritumoral TILs per 0.5 x 0.5 cm area tile of tissue in the WSI. Results: Both Tumor-TIL and Gleason-TIL maps reveal complex and heterogeneous tumor immune interactions across multifocal lesions within each radical prostatectomy specimen. Our novel workflow added the ability to quantify PCa, Gleason grade, and TILs to specific 0.5 x 0.5 cm tissue tiles to quantify TILs relative to Gleason grade that varies within and across each case of multifocal PCa. Quantification of the TIL tile maps showed that in 6 of the 8 prostatectomies the “hot spot” tile with the most intratumoral TILs had more TILs in the Gleason 4/5 patches than in the adjacent Gleason 3 patches. Conclusionsand Future Studies: This novel workflow was developed to generate Tumor-TIL and Gleason-TIL maps for every PCa H&amp;E WSI to serve as powerful visual tools for real world laboratory tissue sampling for downstream genomics, immunohistochemistry, and other correlative studies for PCa clinical trials. Adding a 0.5 x 0.5 cm tile partitioning function increases the efficiency of identifying and quantitatively mapping TILs in the complex and heterogeneous PCa microenvironment. The preliminary association of TILs with higher grade Gleason 4/5 PCa appears provocative as potential actionable information as an immune biomarker in PCa. Citation Format: Yuwei Zhang, Rajarsi Gupta, Joel Saltz, Tahsin Kurc, Jakub Kaczmarzyk, Erich Bremer, Veronica Wallaengen, Amanda Galvez, Abigail Bartley, Leonard Salcedo, Alan Pollack, Sanoj Punnen, Oleksandr Kryvenko, Radka Stoyanova, Sandra M. Gaston. Mapping tumor infiltrating lymphocytes in whole prostatectomy specimens to visualize and quantify the immune microenvironment of prostate cancer [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2025; Part 1 (Regular Abstracts); 2025 Apr 25-30; Chicago, IL. Philadelphia (PA): AACR; Cancer Res 2025;85(8_Suppl_1):Abstract nr 5315.

Whole Slide Image (WSI) classification often relies on weakly supervised Multiple Instance Learning (MIL) methods to handle gigapixel-resolution images. In various MIL methods, attention-based approaches have shown great potential in modern medicine for cancer diagnosis and treatment. These approaches can model the interrelationships among instances to achieve enhanced bag representation using instance scores and thus promote bag-level classification performance. However, the existing attention-based MIL methods face two challenges: (1) The attention-based instance scores cannot accurately represent the contribution of instances to bag-level classification, making it difficult to identify the discriminative regions in WSIs. (2) Whole-slide pathological image analysis frequently suffers from model overfitting and insufficient representation of positive samples for training. To address the problem of poor discriminative regions in WSIs, we design a module to acquire the accurate contribution weights of instances by introducing the Class Activation Map suitable for WSI (WSICAM). For the second challenge, we adopt a Cross-Slide Augmentation (CSA) module to construct new samples with mixed labels on the basis of discriminative instances for model training. Our framework is composed of two WSICAM modules and one CSA module. The experimental results and visualizations demonstrate that our method achieves state-of-the-art in WSI classification on widely used datasets and exhibits robust capabilities in tumor lesion localization.

BackgroundTargeted therapy and immunotherapy put forward higher demands for accurate lung cancer classification, as well as benign versus malignant disease discrimination. Digital whole slide images (WSIs) witnessed the transition from traditional histopathology to computational approaches, arousing a hype of deep learning methods for histopathological analysis. We aimed at exploring the potential of deep learning models in the identification of lung cancer subtypes and cancer mimics from WSIs.MethodsWe initially obtained 741 WSIs from the First Affiliated Hospital of Sun Yat-sen University (SYSUFH) for the deep learning model development, optimization, and verification. Additional 318 WSIs from SYSUFH, 212 from Shenzhen People’s Hospital, and 422 from The Cancer Genome Atlas were further collected for multi-centre verification. EfficientNet-B5- and ResNet-50-based deep learning methods were developed and compared using the metrics of recall, precision, F1-score, and areas under the curve (AUCs). A threshold-based tumour-first aggregation approach was proposed and implemented for the label inferencing of WSIs with complex tissue components. Four pathologists of different levels from SYSUFH reviewed all the testing slides blindly, and the diagnosing results were used for quantitative comparisons with the best performing deep learning model.ResultsWe developed the first deep learning-based six-type classifier for histopathological WSI classification of lung adenocarcinoma, lung squamous cell carcinoma, small cell lung carcinoma, pulmonary tuberculosis, organizing pneumonia, and normal lung. The EfficientNet-B5-based model outperformed ResNet-50 and was selected as the backbone in the classifier. Tested on 1067 slides from four cohorts of different medical centres, AUCs of 0.970, 0.918, 0.963, and 0.978 were achieved, respectively. The classifier achieved high consistence to the ground truth and attending pathologists with high intraclass correlation coefficients over 0.873.ConclusionsMulti-cohort testing demonstrated our six-type classifier achieved consistent and comparable performance to experienced pathologists and gained advantages over other existing computational methods. The visualization of prediction heatmap improved the model interpretability intuitively. The classifier with the threshold-based tumour-first label inferencing method exhibited excellent accuracy and feasibility in classifying lung cancers and confused nonneoplastic tissues, indicating that deep learning can resolve complex multi-class tissue classification that conforms to real-world histopathological scenarios.

Pathology report generation from whole slide images with knowledge retrieval and multi-level regional feature selection.

BackgroundPersonalized medicine and accurate quantification of tumor and biomarker expression have become the cornerstone of cancer diagnostics. This requires Quality Control (QC) of research tissue samples to confirm adequate targeted tumor tissue sampling. Digitalization of stained tissue slides offer a precious way to archive, preserve, and retrieve necessary information when needed. This study is aimed to assess the most significant pre-analytic and analytic factors that might contribute to the efficacy of obtaining accurate whole slide images (WSIs) interpretation. Various studies are needed to identifysuch factors to allow for appropriate AI application and adequate tumor area/percentage quantification. MethodsHematoxylene and Eosine (H&E) satined WSIs collected from tissue specimens provided by the Cooperative Human Tissue Network (CHTN) Midwestern Division (CHTNMWD) were analyzed. Tissue specimens were processed, fixed, stained, and scanned contemporaneously (within 1 month). Two cohorts of malignant, colorectal cancer, 20X WSI (ScanscopeXT, Leica Biosystems, Illinois), were assembled. The study identified a "recent cohort" that included 76 WSIs created on 2018 or later. "Aged cohort" included 73 WSIs from specimens procured in the period of (2012-2014). Twenty recent WSIs of adenocarcinoma cases were used to construct WSIs analysis algorithms (VIS, Visiopharm A/S, Denmark) using machine learning to produce morphometric maps and calculate tissue and tumor areas. ResultsAlgorithmic analysis of 69 WSIs from rescanned aged slides vs. that of contemporaneous WSIs concluded 18 (28%) similar finding in tumor areas (within 10%), 56 (82%) had identicaltissue areas, and 54 (79%) had similar tumor percentages. ConclusionWSIs of aged H&E slides and stained paraffin block re-cuts produce different tumor quantification compared to those of original scanned sslides most likely due to pre-analytical factors. The difference in tumor area detected between original and rescanned WSIs trended upward in the period between 2012 and 2014. Less tumor area was detected as the slides age. Recut and H&E-stained tissues from stored paraffin blocks may detect more tumor due to excess eosinophilia. These results highlights the value of documenting archives of H&E WSIs collected at the procurement time. Such images provide a superior archive over glass slides and Formalin-Fixed Paraffin-Embedded (FFPE) blocks and contribute betterg to WSIs analysis application.

Colorectal cancer (CRC) stands as one of the deadliest cancer types worldwide. Identifying specific gene mutations crucial for targeted therapy is pivotal for CRC treatment. However, the current genetic testing method involves liquid biopsy, which is often cumbersome and cost-intensive. Histopathological whole-slide images (WSIs) are routinely generated for diagnostic purposes and can offer valuable insights into cellular heterogeneity. In this study, we aim to develop an integrated deep learning pipeline for predicting TP53 mutations in CRC, one of the most frequently mutated genes in CRC. We commenced by collecting 424 diagnostic WSIs from the TCGA-COAD cohorts. Each WSI was split as small patches at a resolution of 0.5 micrometers per pixel for following analyses. Quality control was implemented using Otsu thresholding and Gaussian blur filtering, both at the slide-level masking and patch-level filtering stages. Additionally, we applied the Macenko algorithm for color normalization to eliminate systematic errors. Following image preprocessing, we constructed a tissue-type classification model based on VGG19, which was trained and tested using the NCT-CRC-HE-100K and CRC-VAL-HE-7K datasets, both containing pixels with tissue types labeled by human experts. The trained classifier was then used to identify adenocarcinoma epithelium patches from TCGA-COAD. Subsequently, we applied multiple pre-trained Convolutional Neural Network (CNN) models, including ResNet-101, DenseNet-201, and Inception-ResNet-v2, to predict TP53 mutations for the identified adenocarcinoma epithelium patches from TCGA-COAD. A novel integration approach was employed to aggregate patch-level predictions into slide-level outcomes. We evaluated the trained models using various metrics, including accuracy, precision, recall, F1 score, AUC (Area Under the Curve), and Cohen's Kappa coefficient. The tissue-type classification model exhibited robust performance, achieving an overall accuracy of 94.7%, a Cohen's Kappa coefficient of 94%, a precision of 95.3%, a recall of 99.76%, and an F1 score of 97.48% for identifying adenocarcinoma epithelium patches. Regarding the TP53 mutation prediction models, ResNet-101, DenseNet-201, and Inception-ResNet-v2 exhibit the predictive ability to both patch level (AUC=65%) and slide level (AUC=70%). In summary, the proposed deep learning pipeline offers an efficient way to acquire gene mutation information from WSIs. Moreover, it demonstrates a capability to discover potential molecular aberration from the clinical images. Citation Format: Geng-Yun Tien, Liang-Chuan Lai, Tzu-Pin Lu, Mong-Hsun Tsai, Hsiang-Han Chen, Eric Y. Chuang. Prediction of gene mutation from colorectal adenocarcinoma whole slide images via integrated deep learning pipeline [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2024; Part 1 (Regular Abstracts); 2024 Apr 5-10; San Diego, CA. Philadelphia (PA): AACR; Cancer Res 2024;84(6_Suppl):Abstract nr 4938.

Reliability of whole slide images as a diagnostic modality for renal allograft biopsies

Image analysis in digital pathology is identified as a challenging field, particularly for AI-driven classification and search tasks. The high-resolution and large-scale nature of whole slide images (WSIs) present significant computational challenges in representing and analyzing these images effectively. The research endeavors to tackle these hurdles by presenting an innovative methodology grounded in self-supervised learning (SSL). Unlike prior SSL approaches that depend on augmenting at the patch level, the novel framework capitalizes on existing primary site information to directly glean effective representations from Whole Slide Images (WSIs). Moreover, the investigation integrates fully supervised contrastive learning to bolster the resilience of these representations for both classification and search endeavors. For experimentation, the study drew upon a dataset encompassing over 6,000 WSIs sourced from The Cancer Genome Atlas (TCGA) repository facilitated by the National Cancer Institute. The proposed architecture underwent training and assessment using this dataset. Evaluation primarily focused on scrutinizing performance across diverse primary sites and cancer subtypes, with particular attention dedicated to lung cancer classification. Impressively, the proposed architecture yielded outstanding outcomes, showcasing robust performance across the majority of primary sites and cancer subtypes. Furthermore, the study garnered the top position in validation for a lung cancer classification task.

The rise of Digital Pathology during the past few years is leading to the digitisation of the pathology field; the widespread use of Whole Slide Images (WSI) and the digitisation of the diagnostic process have allowed the introduction of AI-based methods to aid some parts of the process. In this framework, the CADIA project was raised in response to the Galician healthcare system digitisation needs. CADIA aims to develop an AI-based medical image analysis solution for the diagnosis of several pathologies and its demonstration on breast cancer diagnosis from WSIs. In this paper, we describe the development of CADIA, from the capture of requirements to the deployment and integration of the solution into the healthcare system infrastructure. We describe the opportunities, challenges and lessons learned during the project development.KeywordsDigital PathologyWhole Slide ImagesBreast cancer diagnosis

BackgroundSince microscopic slides can now be automatically digitized and integrated in the clinical workflow, quality assessment of Whole Slide Images (WSI) has become a crucial issue. We present a no-reference quality assessment method that has been thoroughly tested since 2010 and is under implementation in multiple sites, both public university-hospitals and private entities. It is part of the FlexMIm R&D project which aims to improve the global workflow of digital pathology. For these uses, we have developed two programming libraries, in Java and Python, which can be integrated in various types of WSI acquisition systems, viewers and image analysis tools.MethodsDevelopment and testing have been carried out on a MacBook Pro i7 and on a bi-Xeon 2.7GHz server. Libraries implementing the blur assessment method have been developed in Java, Python, PHP5 and MySQL5. For web applications, JavaScript, Ajax, JSON and Sockets were also used, as well as the Google Maps API. Aperio SVS files were converted into the Google Maps format using VIPS and Openslide libraries.ResultsWe designed the Java library as a Service Provider Interface (SPI), extendable by third parties. Analysis is computed in real-time (3 billion pixels per minute). Tests were made on 5000 single images, 200 NDPI WSI, 100 Aperio SVS WSI converted to the Google Maps format.ConclusionsApplications based on our method and libraries can be used upstream, as calibration and quality control tool for the WSI acquisition systems, or as tools to reacquire tiles while the WSI is being scanned. They can also be used downstream to reacquire the complete slides that are below the quality threshold for surgical pathology analysis. WSI may also be displayed in a smarter way by sending and displaying the regions of highest quality before other regions. Such quality assessment scores could be integrated as WSI's metadata shared in clinical, research or teaching contexts, for a more efficient medical informatics workflow.

Objective: It was the aim of this study to determine the diagnostic accuracy of high-risk prognostic factors and morphological characteristics of retinoblastomas using digital whole slide images (WSI) generated by a scanner. Methods: Forty-seven H&E sections of glass slides with high-risk morphological features of retinoblastoma were analyzed. Slides were scanned as WSI and reviewed. The results were compared with those obtained after reviewing the slides using a regular microscope as the gold standard. McNemar's test (MT), the percentage of agreement (POA), and sensitivity (S) and specificity (Sp) were evaluated between WSI and conventional microscopy. Results: There were no differences with respect to multicentricity, growth type, rosette formation, choroidal invasion, anterior chamber invasion, extraocular extension, scleral extension, optic nerve invasion, necrosis, or Azzopardi effect between WSI analysis and light microscopy (MT, p = 1.0; POA = 100%; S = 100%, and Sp = 100%). Discordance was found in 1 case where calcification could not be found using WSI (MT, p = 1.00; POA = 97.9%; S = 100%, and Sp = 97.8%). Conclusion: To the best of our knowledge, this is the first report using digital pathology (WSI) to evaluate prognostic factors in eyes containing retinoblastomas. Using WSI, the pathologist was able to detect high-risk morphological features in retinoblastoma. To date, WSI is an important tool, in particular for ophthalmic pathologists examining enucleation and exenteration specimens.

Early and reliable prediction of shunt-dependent hydrocephalus (SDHC) after aneurysmal subarachnoid hemorrhage (aSAH) may decrease the duration of in-hospital stay and reduce the risk of catheter-associated meningitis. Machine learning (ML) may improve predictions of SDHC in comparison to traditional non-ML methods. ML models were trained for CHESS and SDASH and two combined individual feature sets with clinical, radiographic, and laboratory variables. Seven different algorithms were used including three types of generalized linear models (GLM) as well as a tree boosting (CatBoost) algorithm, a Naive Bayes (NB) classifier, and a multilayer perceptron (MLP) artificial neural net. The discrimination of the area under the curve (AUC) was classified (0.7 ≤ AUC < 0.8, acceptable; 0.8 ≤ AUC < 0.9, excellent; AUC ≥ 0.9, outstanding). Of the 292 patients included with aSAH, 28.8% (n = 84) developed SDHC. Non-ML-based prediction of SDHC produced an acceptable performance with AUC values of 0.77 (CHESS) and 0.78 (SDASH). Using combined feature sets with more complex variables included than those incorporated in the scores, the ML models NB and MLP reached excellent performances, with an AUC of 0.80, respectively. After adding the amount of CSF drained within the first 14 days as a late feature to ML-based prediction, excellent performances were reached in the MLP (AUC 0.81), NB (AUC 0.80), and tree boosting model (AUC 0.81). ML models may enable clinicians to reliably predict the risk of SDHC after aSAH based exclusively on admission data. Future ML models may help optimize the management of SDHC in aSAH by avoiding delays in clinical decision-making.

Recently, many fields have widely used cluster analysis: psychology, biology, statistics, pattern recognition, information retrieval, machine learning, and data mining. Diagnosis of histopathological images of prostate cancer is one of the routine tasks for pathologists and it is challenging for pathologists to analyze the formation of glands and tumors based on the Gleason grading system. In this study, unsupervised classification has been performed for differentiating malignant (cancerous) from benign (non-cancerous) tumors. Therefore, the unsupervised-based computer-aided diagnosis (CAD) technique would be of great benefit in easing the workloads of pathologists. This technique is used to find meaningful clustering objects (i.e., individuals, entities, patterns, or cases) and identify useful patterns. Radiomic-based features were extracted for cluster analysis using the gray-level co-occurrence matrix (GLCM), gray-level run-length matrix (GLRLM), and gray-level size zone matrix (GLSZM) techniques. Multi-clustering techniques used for the unsupervised classification are K-means clustering, K-medoids clustering, Agglomerative Hierarchical (AH) clustering, Gaussian mixture model (GMM) clustering, and Spectral clustering. The quality of the clustering algorithms was determined using Purity, Silhouettes, Adjusted Rand, Fowlkes Mallows, and Calinski Harabasz (CH) scores. However, the best-performing algorithm (i.e., K-means) has been applied to predict and annotate the cancerous regions in the whole slide image (WSI) to compare with the pathologist annotation.