Histopathological Image Classification with Cell Morphology Aware Deep Neural Networks

Automated classification of histopathological images is an important research problem in medical imaging. Digital histopathology exhibits two principally distinct characteristics: 1) invariably histopathological images are multi-channel (color) with key geometric information spread across the color channels instead of being captured by luminance alone, and 2) the richness of geometric structures in such tissue imagery makes feature extraction for classification very demanding. Inspired by recent work in the use of sparsity for single channel image classification, we propose a new simultaneous Sparsity model for multi-channel Histopathological Image Representation and Classification (SHIRC). Essentially, we represent a multi-channel histopathological image as a sparse linear combination of training examples under suitable channel-wise constraints and classification is performed by solving a newly formulated simultaneous sparsity-based optimization problem. Experiments on two challenging real-world image databases: 1) provided by pathologists of the Animal Diagnostics Lab (ADL) at Pennsylvania State University, and 2) histopathological images corresponding to intraductal breast lesions [1], reveal the merits of the proposed SHIRC model over state of the art alternatives.

Histopathological image classification is an important task in medical image analysis. Recent approaches generally rely on weakly supervised learning due to the ease of acquiring case-level labels from pathology reports. However, patch-level classification is preferable in applications where only a limited number of cases are available or when local prediction accuracy is critical. On the other hand, acquiring extensive datasets with localized labels for training is not feasible. In this paper, we propose a semi-supervised patch-level histopathological image classification model, named CLASS-M, that does not require extensively labeled datasets. CLASS-M is formed by two main parts: a contrastive learning module that uses separated Hematoxylin images and Eosin images generated through an adaptive stain separation process, and a module with pseudo-labels using MixUp. We compare our model with other state-of-the-art models on two clear cell renal cell carcinoma datasets. We demonstrate that our CLASS-M model has the best performance on both datasets. Our code is available at github.com/BzhangURU/Paper_CLASS-M/tree/main.

Deep neural network incorporating domain and resolution transformations model for histopathological image classification

MF-OMKT: Model fusion based on online mutual knowledge transfer for breast cancer histopathological image classification

Breast cancer is a big concern for women due to its higher mortality compared to other cancers. Objective and accurate early diagnosis is primordial for the treatment and survival improvement of patients. Histopathological image classification is considered the gold standard and is usually the last and most dependent diagnosis approach for doctors to make patient treatment proposals. In particular, recent deep learning-based methods provide remarkable classification results. However, these methods ignore rationale or logical explanation that is important for diagnosis reliability and human-level understanding. This paper proposes a multi-instance classification network (MICNet) with the mechanism of visual explanation to achieve the explainable classification of histopathological breast cancer images. The method uses a simple two-dimensional convolution kernel to generate explanation maps (i.e., visual explanation) through features coming from the end of the feature extractor in the VGG11 model pre-trained by ImageNet. Multiple instance learning (MIL) based on mirror padding and overlap cropping is adopted to improve the network’s classification performance. We also design a weighted average pooling method to encourage the network to learn more accurate visual explanation. Experiments on BreakHis and Camelyon16 patch-based datasets demonstrate that our MICNet outperforms other CNN models in classification and is able to provide a logical visual explanation that supports the network’s prediction.

The field of histopathological image analysis has evolved significantly with the advent of digital pathology, leading to the development of automated models capable of classifying tissues and structures within diverse pathological images. Artificial intelligence algorithms, such as convolutional neural networks, have shown remarkable capabilities in pathology image analysis tasks, including tumor identification, metastasis detection, and patient prognosis assessment. However, traditional manual analysis methods have generally shown low accuracy in diagnosing colorectal cancer using histopathological images. This study investigates the use of AI in image classification and image analytics using histopathological images using the histogram of oriented gradients method. The study develops an AI-based architecture for image classification using histopathological images, aiming to achieve high performance with less complexity through specific parameters and layers. In this study, we investigate the complicated state of histopathological image classification, explicitly focusing on categorizing nine distinct tissue types. Our research used open-source multi-centered image datasets that included records of 100.000 non-overlapping images from 86 patients for training and 7180 non-overlapping images from 50 patients for testing. The study compares two distinct approaches, training artificial intelligence-based algorithms and manual machine learning models, to automate tissue classification. This research comprises two primary classification tasks: binary classification, distinguishing between normal and tumor tissues, and multi-classification, encompassing nine tissue types, including adipose, background, debris, stroma, lymphocytes, mucus, smooth muscle, normal colon mucosa, and tumor. Our findings show that artificial intelligence-based systems can achieve 0.91 and 0.97 accuracy in binary and multi-class classifications. In comparison, the histogram of directed gradient features and the Random Forest classifier achieved accuracy rates of 0.75 and 0.44 in binary and multi-class classifications, respectively. Our artificial intelligence-based methods are generalizable, allowing them to be integrated into histopathology diagnostics procedures and improve diagnostic accuracy and efficiency. The CNN model outperforms existing machine learning techniques, demonstrating its potential to improve the precision and effectiveness of histopathology image analysis. This research emphasizes the importance of maintaining data consistency and applying normalization methods during the data preparation stage for analysis. It particularly highlights the potential of artificial intelligence to assess histopathological images.

Histopathological image contains rich pathological information that is valued for the aided diagnosis of many diseases such as cancer. An important issue in histopathological image classification is how to learn a high-quality discriminative dictionary due to diverse tissue pattern, a variety of texture, and different morphologies structure. In this paper, we propose a discriminative dictionary learning algorithm with pairwise local constraints (PLCDDL) for histopathological image classification. Inspired by the one-to-one mapping between dictionary atom and profile, we learn a pair of discriminative graph Laplacian matrices that are less sensitive to noise or outliers to capture the locality and discriminating information of data manifold by utilizing the local geometry information of category-specific dictionaries rather than input data. Furthermore, graph-based pairwise local constraints are designed and incorporated into the original dictionary learning model to effectively encode the locality consistency with intra-class samples and the locality inconsistency with inter-class samples. Specifically, we learn the discriminative localities for representations by jointly optimizing both the intra-class locality and inter-class locality, which can significantly improve the discriminability and robustness of dictionary. Extensive experiments on the challenging datasets verify that the proposed PLCDDL algorithm can achieve a better classification accuracy and powerful robustness compared with the state-of-the-art dictionary learning methods. Graphical abstract The proposed PLCDDL algorithm. 1) A pair of graph Laplacian matrices are first learned based on the class-specific dictionaries. 2) Graph-based pairwise local constraints are designed to transfer the locality for coding coefficients. 3) Class-specific dictionaries can be further updated.

Accurate histopathological image classification plays a crucial role in cancer detection and diagnosis. In automated cancer detection methods, extraction of histological features of malignant and benign tissues is a challenging task. This paper presents a modified approach on octave convolution to extract high and low-frequency features which help to provide a comprehensive representation of histopathological images. Proposed octave convolution model is used to perform histopathological image classification using three different optimization strategies. Firstly, an optimal alpha value of 0.5 is used to give equal importance to both high-frequency and low-frequency feature maps. This balanced approach ensures that the model effectively considers critical high-frequency features as well as low-frequency features of cancerous tissues. Secondly, high-frequency and low-frequency feature maps are extracted and down sampled into half the spatial dimension size to reduce the computational cost compared to standard CNN. Thirdly, training and validation was conducted using ReLU, PReLU, LeakyReLU, ELU, GELU and Swish activation functions. From the experiment, it was concluded that PReLU is the best activation function for capturing intricate patterns inherent in cancer-related histopathological images. Combining all these optimization strategies, the proposed method proved to provide a classification accuracy of 93% and also to reduce the computational cost by 50%. Performance validation against pre-trained models, CNN variants and vision transformer-based models has also been conducted, which proved superior performance of the proposed model.

Uncertainty estimation in the classification of histopathological images with HER2 overexpression using Monte Carlo Dropout

The computer-assisted classification of breast cancer histopathological image in the future is an essential method for the improvement of the diagnostic performance, thus reducing breast cancer deaths. Although deep convolutional neural networks (DCNNs) have made dramatic breakthroughs, many image classification tasks still remain challenging due to the insufficiency of training data and the lack of the ability to focus on improving classification efficiency. To address these issues, we share a densely connected convolutional network (DenseNet) model that is composed of 3 layers DenseNet, pooling layer, and a classification layer for breast cancer classification in microscopic images. Each layer of DenseNet contains 4 dense blocks. Each dense block jointly uses the dense connection and a novel attention learning mechanisms to increase its ability for discriminative representation. Meanwhile, the transfer learning algorithm is applied to determine the model parameters to extract the features of the patient image that is performed. In order to ensure sufficient data volume, a data enhancement method based on the quad-tree principle is proposed for high-resolution images. On the other hand, the classification probability of each part after dicing is fused by three algorithms of addition, product, and maximum. We evaluated our DenseNet model on the BreastKHis dataset. Our results indicate that the DenseNet model and data enhancement method we adopted can adaptively focus on the study of breast cancer histopathological image classification, thus achieving the state-of-the-art performance in breast cancer classification. The results of the experiments are in terms of patient-level and image-level accuracy. The best recognition accuracy increased to 90.9%-92.5% and 89.3%-91.8%, respectively, compared with previous studies.

Fusion strategies for deep convolutional neural network representations in histopathological image classification

The application of machine learning techniques to histopathology images enables advances in the field, providing valuable tools that can speed up and facilitate the diagnosis process. The classification of these images is a relevant aid for physicians who have to process a large number of images in long and repetitive tasks. This work proposes the adoption of metric learning that, beyond the task of classifying images, can provide additional information able to support the decision of the classification system. In particular, triplet networks have been employed to create a representation in the embedding space that gathers together images of the same class while tending to separate images with different labels. The obtained representation shows an evident separation of the classes with the possibility of evaluating the similarity and the dissimilarity among input images according to distance criteria. The model has been tested on the BreakHis dataset, a reference and largely used dataset that collects breast cancer images with eight pathology labels and four magnification levels. Our proposed classification model achieves relevant performance on the patient level, with the advantage of providing interpretable information for the obtained results, which represent a specific feature missed by the all the recent methodologies proposed for the same purpose.

Diagnosis of different breast cancer stages using histopathology whole slide images is the gold standard in grading the tissue metastasis. Traditional diagnosis involves labor intensive procedures and is prone to human errors. Computer aided diagnosis assists medical experts as a second opinion tool in early detection which prevents further proliferation. Computing facilities have emerged to an extent where algorithms can attain near human accuracy in prediction of diseases, offering better treatment to curb further proliferation. The work introduced in the paper provides an interface in mobile platform, which enables the user to input histopathology image and obtain the prediction results with its class probability through embedded web-server. The trained deep convolutional neural networks model is deployed into a microcomputer-based embedded system after hyper-parameter tuning, offering congruent performance. The implementation results show that the embedded platform with custom-trained CNN model is suitable for medical image classification, as it takes less execution time and mean prediction time. It is also noticed that customized CNN classifier model outperforms pre-trained models when used in embedded platforms for prediction and classification of histopathology images. This work also emphasizes the relevance of portable and flexible embedded device in real time clinical applications.

Oral cancer is a prevalent disease happening in the head and neck region. Due to the high occurrence rate and serious consequences of oral cancer, an accurate diagnosis of malignant oral tumors is a major priority. Thus, early diagnosis is very effective to give the patient a prompt response to treatment. The most efficient way for diagnosing oral cancer is from histopathological imaging, which provides a detailed view of inside cells. Accurate and automatic classification of oral histopathological images remains a difficult task due to the complex nature of cell images, staining methods, and imaging conditions. The use of deep learning in imaging techniques and computational diagnostics can assist doctors and physicians in automatically analysing Oral Squamous Cell Carcinoma biopsy images in a timely and efficient manner. Thus, it reduces the operational workload of the pathologist and enhance patient management. Training deeper neural networks takes considerable time and requires a lot of computing resources, due to the complexity of the network and the gradient diffusion problem. With this motivation and inspired by ResNet's significant successes to handle the gradient diffusion problem, in this study we suggest the novel improved ResNet-based model for the automated multistage classification of oral histopathology images. Three prospective candidate model blocks are presented, analyzed, and the best candidate model is chosen as the optimal one which can efficiently classify the oral lesions into well-differentiated, moderately-differentiated and poorly-differentiated in significantly reduced time, with 97.59% accuracy.

Deep manifold feature fusion for classification of breast histopathology images