Multi-level local feature classification for bleeding detection in Wireless Capsule Endoscopy images

A new feature extraction technique for the detection of lesions created from mucosal inflammations in Crohn’s disease, based on wireless capsule endoscopy (WCE) images processing is presented here. More specifically, a novel filtering process, namely Hybrid Adaptive Filtering (HAF), was developed for efficient extraction of lesion-related structural/textural characteristics from WCE images, by employing Genetic Algorithms to the Curvelet-based representation of images. Additionally, Differential Lacunarity (DLac) analysis was applied for feature extraction from the HAF-filtered images. The resulted scheme, namely HAF-DLac, incorporates support vector machines for robust lesion recognition performance. For the training and testing of HAF-DLac, an 800-image database was used, acquired from 13 patients who undertook WCE examinations, where the abnormal cases were grouped into mild and severe, according to the severity of the depicted lesion, for a more extensive evaluation of the performance. Experimental results, along with comparison with other related efforts, have shown that the HAF-DLac approach evidently outperforms them in the field of WCE image analysis for automated lesion detection, providing higher classification results, up to 93.8% (accuracy), 95.2% (sensitivity), 92.4% (specificity) and 92.6% (precision). The promising performance of HAF-DLac paves the way for a complete computer-aided diagnosis system that could support physicians’ clinical practice.

Self-supervised out-of-distribution detection in wireless capsule endoscopy images

This paper presents a methodology for detecting abnormal patterns in wireless capsule endoscopy (WCE) images. In particular, an average of 50,000 images are obtained during an WCE exam. Usually, these images are reviewed in a form of a video at speeds between 5 to 40 image-frames/sec. The time spent by a physician reading the results of WCE images varies between 45 to 180 minutes. This presents a major problem which is the reading process that consumes a significant amount of time and the results take several days before they become available since the physician has to find the time to study each video uninterrupted for up to 3 hours. The methodology presented here is based on the automatic detection of abnormal WCE patterns in an effort for reducing the reading time of the WCE images and the cost of the procedure as well. The methodology consists of a synergistic integration of image processing, analysis and recognition techniques for achieving the automatic detection of the WCE abnormal patterns.

Weakly supervised methods, such as class activation maps (CAM) based, have been applied to achieve bleeding segmentation with low annotation efforts in Wireless Capsule Endoscopy (WCE) images. However, the CAM labels tend to be extremely noisy, and there is an irreparable gap between CAM labels and ground truths for medical images. This paper proposes a new Discrepancy-basEd Active Learning (DEAL) approach to bridge the gap between CAMs and ground truths with a few annotations. Specifically, to liberate labor, we design a novel discrepancy decoder model and a CAMPUS (CAM, Pseudo-label and groUnd-truth Selection) criterion to replace the noisy CAMs with accurate model predictions and a few human labels. The discrepancy decoder model is trained with a unique scheme to generate standard, coarse and fine predictions. And the CAMPUS criterion is proposed to predict the gaps between CAMs and ground truths based on model divergence and CAM divergence. We evaluate our method on the WCE dataset and results show that our method outperforms the state-of-the-art active learning methods and reaches comparable performance to those trained with full annotated datasets with only 10% of the training data labeled. The source code is available at https://github.com/baifanxxx/DEAL.KeywordsActive learningSegmentationWCE images

With the development of wireless capsule endoscopy (WCE) and its extensive applications in clinic, doctors need to spend more time on reviewing the WCE images for lesions diagnosis. Therefore, automatic lesion detection for WCE has gradually become a research hotspot, which aims to reduce the pressure of doctors and improves the diagnosis efficiency. Many researchers adopted the traditional machine learning method to realize polyp detections, however, these methods need to extract the features manually, which were unable to find higher features of WCE images. So, in this study, we proposed a novel method that based on convolution neural network (CNN) to automatically recognize polyp in small bowel WCE image. We utilized the Alexnet architecture, one of the classical CNN, to extract the features of WCE images and classify polyp images from normal ones. We selected 14408 images from different patients, including 408 polyp images and 14000 normal images. Since the amount of initial polyp images is small, then, we did the data augmentation, including rotation, luminance change, blurring, and noise. At last the experimental results demonstrated that the method we proposed had a promising performance in polyp detection, whose accuracy, sensitivity and specificity can reach at 99.88%, 99.40% and 99.93%, respectively. Additionally, we evaluated ROC curve and its AUC value, which further confirmed that our model has a high accuracy and reliability in polyp detection. This proposed method has great potential to be used in the clinical examination to help doctors from the tedious image reviewing work.

Wireless capsule endoscopy (WCE) is a great breakthrough for Gastrointestinal (GI) Tract diagnoses, and it can view the entire gastrointestinal tract, especially the small intestine without invasiveness and sedation. However, a tough problem associated with this new technology is that too many images to be inspected by naked eyes cause a huge burden to physicians, so it is significant to find an automatic diagnosis method. In this paper, a new automatic algorithm for bleeding detection in WCE images is proposed. This new approach mainly focuses on color feature which is also a very effective clue used by physicians for diagnosis. We propose six color features in HSI color space to discriminate between bleeding and normal status. Then we use support vector classifier to verify the performance of the proposed features and judge the status of the images. Experimental results show that the proposed features and classification method is effective and the average accuracy can achieve approximately 97%.

In this paper, a two-dimensional to three-dimensional mapping of the wireless capsule endoscopy (WCE) images is proposed as well as the alignment of the consecutive images. The aim of this project is to reduce the diagnosis time of the doctor using the acquired data. This paper resolves a serious limitation of the inverse projection of the WCE images appeared previously. The novelty of this paper lies in three aspects. Firstly, as the main axis of the WCE is not always aligned with the center of the small bowel, the angle needs to be adjusted to obtain better visual effect. The inverse projected images considering the angle of the WCE are obtained from the innovative mathematical model proposed in this project. After compensating the angle, all the information in the interior wall of the small intestine will be clearly presented. Secondly, the redundancy of the images captured along a trajectory, where, they share a portion of overlapped information is exploited by motion estimation to enhance the final resolution. Therefore, doctors do not need to inspect the same region of the images without any benefit and the diagnosis time is significantly decreased. Finally, the cropped segments are merged into a single image which can fully represent the interior wall of the small intestine.

Diseases of the digestive tract, such as ulcers, pose a serious threat to human health. In fact, many types of endoscopy are employed to examine the patient’s gastrointestinal tract. Recently, wireless capsule endoscopy (WCE) is presented as an excellent diagnostic tool for evaluation of gastrointestinal diseases compared to traditional endoscopies. This diagnosis usually takes a long time, which is tiring, and so the doctors may miss parts where abnormalities of the gastrointestinal tract may present. Therefore, automated diagnostic technics to detect symptoms of gastrointestinal illness in WCE images is adopted as an excellent enhancement tool for these doctors. In this work, a new computer-aided diagnosis method for ulcer detection in WCE images is proposed. After a preprocessing step, fine-tuned convolutional neural network (CNN) is used to extract deep features from these images. Since the number of ulcer images in the available data sets is limited, the CNN networks used in this work were pre-trained on millions of labeled natural images (ImageNet). After the deep features extraction, a random forest classifier is employed to detect ulcer from WCE images. The proposed approach demonstrates promising results (96.73 % and 95.34 % in terms of precision and recall respectively). Those results are satisfactory when compared to recent state-of-the-art methods.

Deep convolutional neural networks with ensemble learning and transfer learning for automated detection of gastrointestinal diseases

Wireless capsule endoscopy (WCE) can painlessly capture a large number of images inside the intestine. However, only a small portion of these WCE images contain hemorrhage. It is thus critical to develop automated hemorrhage detection method to facilitate the diagnosis of intestinal diseases. However, automated hemorrhage detection is complicated by 1) the extreme imbalance between the amount of hemorrhage images and that of normal images; and 2) the variety of the appearance, texture, and luminance inside the intestine. In this paper, we proposed to learn a robust intestinal hemorrhage detection model via Convolutional Neural Networks (CNNs), because of CNNs' extraordinary performance in solving various image understanding tasks. Specially, we explored different CNN architectures and data augmentation methods. Besides, we investigated the correlation between hemorrhage detection accuracy and image quality. Across about 1.3k hemorrhage images and 40k normal images, the learned CNN model achieves an F-measure of 98.87%.

Wireless capsule endoscopy (WCE) is an innovative solution for gastrointestinal disease detection. Limited by WCE hardware and cost of manufacture, WCE image resolution is commonly low, which creates problems for attention to image details and visual perception in medical diagnosis. Under the sparse representation framework, we propose an adaptive dictionary pair learning method to obtain more appropriate representation of each patch with more relevant atoms according to patch content. Specifically, the dictionary pair is learned from high-low resolution cluster patches based on sparse constraint of input patches. Careful examination of the WCE images show there exist unnatural block areas. In order to further improve performance, the autoregressive model is applied to enhance local structure. Intensive experiments have been conducted on WCE image dataset and natural image dataset, including comparison test between the state-of-art methods and ours, and the results validate the effectiveness of the proposed method both on visual perception effect and objective indices.

Wireless capsule endoscopy (WCE), a widely used tool for gastrointestinal tract lesion detection, produces a large number of images. However, it is difficult and inefficient to manually search for and find bleeding lesions among these images. In this study, a new method for the automatic identification of bleeding frames in WCE has been proposed. The minimum spanning tree (MST) clustering algorithm was applied to reduce the background noise. Then, the images' features are extracted from the clustering result and classified by support vector machine (SVM). The experimental results, suggest that this method can correctly identify and mark the bleeding areas in WCE images. The final sensitivity, specificity and accuracy of our proposed method are 91.69%, 94.59%, and 94.10%, respectively.

As one of the most common human helminths, hookworm is a leading cause of maternal and child morbidity, which seriously threatens human health. Recently, wireless capsule endoscopy (WCE) has been applied to automatic hookworm detection. Unfortunately, it remains a challenging task. In recent years, deep convolutional neural network (CNN) has demonstrated impressive performance in various image and video analysis tasks. In this paper, a novel deep hookworm detection framework is proposed for WCE images, which simultaneously models visual appearances and tubular patterns of hookworms. This is the first deep learning framework specifically designed for hookworm detection in WCE images. Two CNN networks, namely edge extraction network and hookworm classification network, are seamlessly integrated in the proposed framework, which avoid the edge feature caching and speed up the classification. Two edge pooling layers are introduced to integrate the tubular regions induced from edge extraction network and the feature maps from hookworm classification network, leading to enhanced feature maps emphasizing the tubular regions. Experiments have been conducted on one of the largest WCE datasets with WCE images, which demonstrate the effectiveness of the proposed hookworm detection framework. It significantly outperforms the state-of-the-art approaches. The high sensitivity and accuracy of the proposed method in detecting hookworms shows its potential for clinical application.

High pooled performance of convolutional neural networks in computer-aided diagnosis of GI ulcers and/or hemorrhage on wireless capsule endoscopy images: a systematic review and meta-analysis

Gastrointestinal (GI) bleeding is a common problem and may lead to fatal consequences. The detection of bleeding is currently determined through conventional examination of wireless capsule endoscopy (WCE) images. Image-based bleeding detection is performed using only color pixels; therefore, image and color distortion affect the accuracy of RGB-based methods adversely. On the other hand, imaging modalities like multispectral features contain several unique characteristics of the target material, and therefore are less error-prone. In this regard, an on-chip bleeding detection sensor based on blood's optical properties is developed in this article. For creating this sensor, an array of 12 optical sensors, six in visible and six in near-infrared range, is tested with blood samples (BS) in various concentrations and non-blood samples (NBS), such as food pigments and natural foods (digestible and nondigestible). Various feature selection and machine learning approaches have been used to select the optimal bandwidth for the best detection accuracy. A capsule prototype is designed that uses 450 nm, 610 nm, and 810 nm of wavelengths. The prototype is tested in two in vitro experiments using two porcine intestines to validate the proof of concept. The results indicate that the designed system can distinguish occult and acute bleeding from a distance of 1 cm with angles of 45 and 90 degrees. An F1-score of 99% is achieved using the bagged decision tree algorithm. The proposed solution for detecting GI bleeding will eliminate the need to use a camera module and large data transmission, and thereby consumes 1.48 times less power than conventional WCE systems.