Breast Cancer Histopathology Image Classification and Localization using Multiple Instance Learning

The computer-assisted classification of breast cancer histopathological image in the future is an essential method for the improvement of the diagnostic performance, thus reducing breast cancer deaths. Although deep convolutional neural networks (DCNNs) have made dramatic breakthroughs, many image classification tasks still remain challenging due to the insufficiency of training data and the lack of the ability to focus on improving classification efficiency. To address these issues, we share a densely connected convolutional network (DenseNet) model that is composed of 3 layers DenseNet, pooling layer, and a classification layer for breast cancer classification in microscopic images. Each layer of DenseNet contains 4 dense blocks. Each dense block jointly uses the dense connection and a novel attention learning mechanisms to increase its ability for discriminative representation. Meanwhile, the transfer learning algorithm is applied to determine the model parameters to extract the features of the patient image that is performed. In order to ensure sufficient data volume, a data enhancement method based on the quad-tree principle is proposed for high-resolution images. On the other hand, the classification probability of each part after dicing is fused by three algorithms of addition, product, and maximum. We evaluated our DenseNet model on the BreastKHis dataset. Our results indicate that the DenseNet model and data enhancement method we adopted can adaptively focus on the study of breast cancer histopathological image classification, thus achieving the state-of-the-art performance in breast cancer classification. The results of the experiments are in terms of patient-level and image-level accuracy. The best recognition accuracy increased to 90.9%-92.5% and 89.3%-91.8%, respectively, compared with previous studies.

Owing to the increasing number of elderly “baby boomers” in Japan, the number of cancer patients is also expected to increase. Approximately 2 million baby boomers from nearby local areas are residing in metropolitan areas; hence, the geographical distribution of cancer patients will probably markedly change. We assessed the expected number of breast cancer (BC) patients in different regions (urban, outer city, town, rural) using estimates of the nation’s population and Kanagawa Cancer Registry data. To estimate future BC incidence for each region, we multiplied the 2010 rate by the predicted female population for each region according to age group. The incidence cases of BC in those aged ≥65 years is expected to increase in all areas; in particular, compared to rates in 2010, the BC incidence in urban areas was predicted to increase by 82.6% in 2035 and 102.2% in 2040. Although the incidence in all BC cases in urban areas showed an increasing trend, until peaking in 2040 (increasing 31.2% from 2010), the number of BC patients would continue to decrease in other areas. The number of BC patients per capita BC specialist was 64.3 patients in 2010; this value would increase from 59.3 in 2010 to 77.7 in 2040 in urban areas, but would decrease in other areas. Our findings suggest that the number of elderly BC patients is expected to increase rapidly in urban areas and that the demand for BC treatment would increase in the elderly population in urban areas.

MF-OMKT: Model fusion based on online mutual knowledge transfer for breast cancer histopathological image classification

Breast cancer is a big concern for women due to its higher mortality compared to other cancers. Objective and accurate early diagnosis is primordial for the treatment and survival improvement of patients. Histopathological image classification is considered the gold standard and is usually the last and most dependent diagnosis approach for doctors to make patient treatment proposals. In particular, recent deep learning-based methods provide remarkable classification results. However, these methods ignore rationale or logical explanation that is important for diagnosis reliability and human-level understanding. This paper proposes a multi-instance classification network (MICNet) with the mechanism of visual explanation to achieve the explainable classification of histopathological breast cancer images. The method uses a simple two-dimensional convolution kernel to generate explanation maps (i.e., visual explanation) through features coming from the end of the feature extractor in the VGG11 model pre-trained by ImageNet. Multiple instance learning (MIL) based on mirror padding and overlap cropping is adopted to improve the network’s classification performance. We also design a weighted average pooling method to encourage the network to learn more accurate visual explanation. Experiments on BreakHis and Camelyon16 patch-based datasets demonstrate that our MICNet outperforms other CNN models in classification and is able to provide a logical visual explanation that supports the network’s prediction.

Breast cancer is one of the major public health issues and is considered a leading cause of cancer-related deaths among women worldwide. Its early diagnosis can effectively help in increasing the chances of survival rate. To this end, biopsy is usually followed as a gold standard approach in which tissues are collected for microscopic analysis. However, the histopathological analysis of breast cancer is non-trivial, labor-intensive, and may lead to a high degree of disagreement among pathologists. Therefore, an automatic diagnostic system could assist pathologists to improve the effectiveness of diagnostic processes. This paper presents an ensemble deep learning approach for the definite classification of non-carcinoma and carcinoma breast cancer histopathology images using our collected dataset. We trained four different models based on pre-trained VGG16 and VGG19 architectures. Initially, we followed 5-fold cross-validation operations on all the individual models, namely, fully-trained VGG16, fine-tuned VGG16, fully-trained VGG19, and fine-tuned VGG19 models. Then, we followed an ensemble strategy by taking the average of predicted probabilities and found that the ensemble of fine-tuned VGG16 and fine-tuned VGG19 performed competitive classification performance, especially on the carcinoma class. The ensemble of fine-tuned VGG16 and VGG19 models offered sensitivity of for carcinoma class and overall accuracy of . Also, it offered an F1 score of . These experimental results demonstrated that our proposed deep learning approach is effective for the automatic classification of complex-natured histopathology images of breast cancer, more specifically for carcinoma images.

Breast cancer is a highly heterogeneous malignant tumor, and its accurate classification is of great significance for clinical diagnosis and treatment decision-making. In recent years, convolutional neural networks and Transformer have been widely used in pathological image analysis of breast cancer. Though the former excels at capturing local information and the latter is adept at modeling global dependencies, the former is limited by fixed sampling positions and is hindered to characterize irregular cell morphology, the latter is insufficient in describing the two-dimensional spatial structure of cells and tissues. Based on these, a Spatial-Frequency Domain Feature Extraction Model (S-FDFEM) proposed in this paper integrates spatial and frequency domain information to enhance feature learning for pathological image recognition. Specifically, in the spatial domain, Deformable Bottleneck Convolution (DBottConv) is utilized to effectively represent intricate cell and tissue morphological variations in pathological images and improve the expression ability of local features; in the frequency domain, wavelet low frequencies and Fourier high frequencies are generated based on the input pathological images to capture global approximation and fine structures, then statistical transformer and depth gradient feature extraction modules are integrated to operate on these two frequency domain components, enabling global dynamic focusing and two dimensions spatial characterization of pathological images. Experimental results show that the classification of breast cancer pathological image on BreakHis and BACH datasets verify the superiority of the S-FDFEM proposed in this paper.

Classify image into benign and malignant is one of the basic image processing tools in digital pathology for breast cancer diagnosis. Deep learning methods have received more attention recently by training with large-scale labeled datas, but collecting and annotating clinical data is professional and time-consuming. The proposed work develops a deep active learning framework to reduce the annotation burden, where the method actively selects the valuable unlabeled samples to be annotated instead of random selecting. Besides, compared with standard query strategy in previous active learning methods, the proposed query strategy takes advantage of manual labeling and auto-labeling to emphasize the confidence boosting effect. We validate the proposed work on a public histopathological image dataset. The experimental results demonstrate that the proposed method is able to reduce up to 52% labeled data compared with random selection. It also outperforms deep active learning method with standard query strategy in the same tasks.

Analysis of histopathological images (HIs) is crucial for detecting breast cancer (BR). However, because they vary, it is still very difficult to extract well-designed elements. Deep learning (DL) is a recent development that is used to extract high-level features. However, DL techniques continue to confront several difficult problems, such as the need for sufficient training data for DL models, which reduces the classification findings. In this study, an ensemble deep transfer convolutional neural network is presented to address this problem. The pre-trained models (ResNet50 and MobileNet) are employed to extract high-level features by freezing the front layer parameters while fine-tuning the last layers. In the proposed ensemble framework, KNN, SVM, logistic regression and neural networks are used as base classifiers. The majority vote and product approaches are used to integrate the predictions of each separate classifier. In the benchmark BreaKHis dataset, the suggested ensemble model is compared to some current approaches. It demonstrates that while the ensemble model obtains a considerable accuracy of 97.72% for the multiclass classification test, it achieves an accuracy of 99.2% for the binary task. The suggested ensemble model’s effectiveness in extracting useful features for BR images is demonstrated by comparison with existing cutting-edge models.

Breast cancer remains one of the most common malignancies in women, where early diagnosis is critical for effective treatment. While deep learning shows significant promise for histopathological image analysis, its performance is often hindered by two key challenges: the scarcity and imbalance of medical datasets, and the inherent difficulty of classifying images with large intraclass and small interclass variations. To address these issues, we propose a novel Center‐Border Dual‐Branch Network with Dynamic Weighted Fusion (CBDF‐Net) for breast cancer histopathology image classification. Our framework first counteracts data imbalance through a pathology‐informed augmentation pipeline (HistoAugment), then employs a Center‐Border Partition Module (CBPM) for spatially differentiated feature extraction. These distinct representations are subsequently processed by a dynamically fused dual‐branch network (DFD‐Net), and its outputs are then integrated by a dynamic fusion mechanism with a compensation shortcut. The entire architecture is trained under the supervision of our novel Adaptive Entropy Penalty (AEP) Loss, which mitigates overconfidence and enhances generalization. Extensive experiments on multiple public datasets demonstrate that CBDF‐Net consistently and significantly outperforms state‐of‐the‐art methods.

Microscopic analysis of breast tissues is necessary for a definitive diagnosis of breast cancer which is the most common cancer among women. Pathology examination requires time consuming scanning through tissue images under different magnification levels to find clinical assessment clues to produce correct diagnoses. Advances in digital imaging techniques offers assessment of pathology images using computer vision and machine learning methods which could automate some of the tasks in the diagnostic pathology workflow. Such automation could be beneficial to obtain fast and precise quantification, reduce observer variability, and increase objectivity. In this work, we propose to classify breast cancer histopathology images independent of their magnifications using convolutional neural networks (CNNs). We propose two different architectures; single task CNN is used to predict malignancy and multi-task CNN is used to predict both malignancy and image magnification level simultaneously. Evaluations and comparisons with previous results are carried out on BreaKHis dataset. Experimental results show that our magnification independent CNN approach improved the performance of magnification specific model. Our results in this limited set of training data are comparable with previous state-of-the-art results obtained by hand-crafted features. However, unlike previous methods, our approach has potential to directly benefit from additional training data, and such additional data could be captured with same or different magnification levels than previous data.

Breast cancer is the most prevalent cancer globally and presents a significant threat to human health. The low survival rate for advanced stages highlights the importance of early and accurate diagnosis. Pathology images, providing insights into cancer's cellular aggressiveness, are crucial for breast cancer diagnosis. Advances in deep learning have transformed automated medical image diagnosis. However, traditional deep learning methods, reliant on large labeled datasets, face challenges with the typically small size of medical image datasets. To overcome this, our study combines deep transfer learning (DTL) with convolutional neural networks(CNN). We utilize and enhance a ResNet50 model pretrained on a large dataset. Additionally, we apply four different deep transfer learning methods to feature extraction and classification of the BreakHis dataset, aiming to determine if breast cancer is benign or malignant. Ultimately, our system achieved the best average accuracy rates under four different magnifications: 98.57%, 96.83%, 97.76%, and 97.16%. These results demonstrate the accuracy and potential of deep transfer learning in classifying breast cancer histopathological images and its promising application in tumor prediction.

Breast cancer is the second leading cause of cancer death among women. Breast cancer is not a single disease, but rather is comprised of many different biological entities with distinct pathological features and clinical implications. Pathologists face a substantial increase in workload and complexity of digital pathology in cancer diagnosis due to the advent of personalized medicine, and diagnostic protocols have to focus equally on efficiency and accuracy. Computerized image processing technology has been shown to improve efficiency, accuracy and consistency in histopathology evaluations, and can provide decision support to ensure diagnostic consistency. We show that convolutional neural networks (CNN) can be an effective tool in classifying breast cancer histopathological images and evaluate its performance as a binary classifier in the field of breast cancer diagnosis using whole slide imaging.

Pathologists need a lot of clinical experience and time to do the histopathological investigation. AI may play a significant role in supporting pathologists and resulting in more accurate and efficient histopathological diagnoses. Breast cancer is one of the most diagnosed cancers in women worldwide. Breast cancer may be detected and diagnosed using imaging methods such as histopathological images. Since various tissues make up the breast, there is a wide range of textural intensity, making abnormality detection difficult. As a result, there is an urgent need to improve computer-assisted systems (CAD) that can serve as a second opinion for radiologists when they use medical images. A self-training learning method employing deep learning neural network with residual learning is proposed to overcome the issue of needing a large number of labeled images to train deep learning models in breast cancer histopathology image classification. The suggested model is built from scratch and trained.

Deep-learning-based breast cancer image diagnosis is currently a prominent and growingly popular area of research. Existing convolutional-neural-network-related methods mainly capture breast cancer image features based on spatial domain characteristics for classification. However, according to digital signal processing theory, texture images usually contain repeated patterns and structures, which appear as intense energy at specific frequencies in the frequency domain. Motivated by this, we make an attempt to explore a breast cancer histopathology classification application in the frequency domain and further propose a novel multi-branch spectral channel attention network, i.e., the MbsCANet. It expands the interaction of frequency domain attention mechanisms from a multi-branch perspective via combining the lowest frequency features with selected high frequency information from two-dimensional discrete cosine transform, thus preventing the loss of phase information and gaining richer context information for classification. We thoroughly evaluate and analyze the MbsCANet on the publicly accessible BreakHis breast cancer histopathology dataset. It respectively achieves the optimal image-level and patient-level classification results of 99.01% and 98.87%, averagely outperforming the spatial-domain-dominated models by a large margin, and visualization results also demonstrate the effectiveness of the MbsCANet for this medical image application.

Simple SummaryRecent advancements in AI have revolutionized cancer research, especially in the analysis of histopathological imaging data with minimal human involvement. Early detection of lymph node metastasis in breast cancer is vital for treatment outcomes. This paper introduces a novel approach that combines representation learning and deep learning (DL) to detect small tumors (STs) without neglecting larger ones. The proposed method uses representation learning to identify STs in histopathology images, followed by DL algorithms for breast cancer detection. Extensive evaluation shows remarkable accuracy in detecting STs without compromising larger-lesion detection. This approach enables early detection, timely intervention, and potentially improved treatment outcomes. The integration of representation learning and DL offers a promising solution for ST detection in breast cancer. By reducing human involvement and leveraging AI capabilities, the proposed method achieves impressive accuracy in identifying STs. Further research and validation could enhance diagnostic capabilities and personalized treatment strategies, ultimately benefiting breast cancer patients.The early diagnosis of lymph node metastasis in breast cancer is essential for enhancing treatment outcomes and overall prognosis. Unfortunately, pathologists often fail to identify small or subtle metastatic deposits, leading them to rely on cytokeratin stains for improved detection, although this approach is not without its flaws. To address the need for early detection, multiple-instance learning (MIL) has emerged as the preferred deep learning method for automatic tumor detection on whole slide images (WSIs). However, existing methods often fail to identify some small lesions due to insufficient attention to small regions. Attention-based multiple-instance learning (ABMIL)-based methods can be particularly problematic because they may focus too much on normal regions, leaving insufficient attention for small-tumor lesions. In this paper, we propose a new ABMIL-based model called normal representative keyset ABMIL (NRK-ABMIL), which addresseses this issue by adjusting the attention mechanism to give more attention to lesions. To accomplish this, the NRK-ABMIL creates an optimal keyset of normal patch embeddings called the normal representative keyset (NRK). The NRK roughly represents the underlying distribution of all normal patch embeddings and is used to modify the attention mechanism of the ABMIL. We evaluated NRK-ABMIL on the publicly available Camelyon16 and Camelyon17 datasets and found that it outperformed existing state-of-the-art methods in accurately identifying small tumor lesions that may spread over a few patches. Additionally, the NRK-ABMIL also performed exceptionally well in identifying medium/large tumor lesions.