Hierarchical key frame extraction for wireless capsule endoscopy video based on the saliency map

Wireless Capsule endoscopy (WCE) has been developed to allow direct view of the entire small bowel in human body in a noninvasive manner for the first time. But the visual inspection of a large number of video frames produced in each examination poses a tedious task to physicians. In this paper, we propose a novel scheme aiming for reduction of the number of frames in a video so as to partially solve this problem. To achieve this goal, the original WCE video is first divided into different segments by detecting video boundary among the video sequence. Key frames from different segments are then extracted to summarize the video with k-means clustering. Preliminary experiments on our present video data verify that it is promising to employ the proposed scheme to abstract a WCE video.

Aiming at the problem of video key frame extraction, a density peak clustering algorithm is proposed, which uses the HSV histogram to transform high-dimensional abstract video image data into quantifiable low-dimensional data, and reduces the computational complexity while capturing image features. On this basis, the density peak clustering algorithm is used to cluster these low-dimensional data and find the cluster centers. Combining the clustering results, the final key frames are obtained. A large number of key frame extraction experiments for different types of videos show that the algorithm can extract different number of key frames by combining video content, overcome the shortcoming of traditional key frame extraction algorithm which can only extract a fixed number of key frames, and the extracted key frames can represent the main content of video accurately.

Temporal abnormality localization in long Wireless Capsule Endoscopy (WCE) videos is an important problem. The cost of obtaining frame level label for long WCE videos is prohibitive. In this paper, we propose an end-to-end temporal abnormality localization for long WCE videos using only weak video level labels. Physicians use Capsule Endoscopy (CE) as a non-surgical and non-invasive method to examine the entire digestive tract in order to diagnose diseases or abnormalities. While CE has revolutionized traditional endoscopy procedures, a single CE examination could last up to 8 hours generating as much as 100,000 frames. Physicians must review the entire video, frame-by-frame, in order to identify the frames capturing relevant lesion or abnormality. This, sometimes could be as few as just a single frame. Given this very high level of redundancy, analysing long CE videos can be very tedious, time consuming and also error prone. This paper presents a novel multi-step method for an end-to-end localization of target frames capturing abnormalities of interest in the long video using only weak video labels. First we developed an automatic temporal segmentation using change point detection technique to temporally segment the video into uniform, homogeneous and identifiable segments. Then we employed Graph Convolutional Neural Network (GCNN) to learn a representation of each video segment. Using weak video segment labels, we trained our GCNN model to recognize each video segment as abnormal if it contains at least a single abnormal frame. Finally, leveraging the parameters of the trained GCNN model, we replaced the final layer of the network with a temporal pool layer to localize the relevant abnormal frames within each abnormal video segment. We experimented with multiple real patients’ endoscopy videos and achieved an accuracy of 89.9% on the graph classification task and a specificity of 97.5% on the abnormal frames localization task.

Key frame extraction is one of the critical techniques in computer vision fields such as video search, video identification and video forgeries detection. The extracted key frames should be sufficient key frames that preserve main actions in a video with compact representation. The objective of this work is to improve our previous action key frames extraction algorithm (AKF) by adapting a threshold which selects action key frames as final key frames. The threshold adaptation was achieved by using the mean value, the standard deviation, and the L1-norm instead of the comparison of user summaries evaluation method to obtain a fully automatic video summarisation algorithm, and by eliminating the conditions in selecting the final key frames to reduce the complexity of the algorithm. We have validated our proposed Improved AKF on complex colour video shots instead of the simple grey level video shots. The Improved AKF algorithm was able to extract a compact number of action key frames by preventing redundant key frames, reduce processing complexity, and preserve sufficient information about the main actions in a video shot. We then evaluated the Improved AKF algorithm with the-state-of-the-art algorithms in terms of compression ratio using Paul videos and Shih-Tang dataset. The evaluation results showed that the Improved AKF algorithm achieved better compression ratio and retained sufficient information in the extracted action key frames under different testing video shots. Therefore, the improved AKF algorithm is a suitable technique for applications in computer vision fields such as passive object-based video authentication systems

This paper proposes an approach to eliminate redundant images adaptively for Wireless Capsule Endoscopy (WCE) video summarization by considering temporal correlation and feature similarity between adjacent WCE frames. The color and texture features, generated by HSV color histogram model and Gray Level Co-occurrence Matrix, have been taken into account. It is noted that frames from different WCE videos may have different dynamic information ranges. Hence a data-driven threshold termed as W-parametric mean value threshold (W-MVT) is developed to improve robustness of the proposed method. By comparing the color-texture feature similarity of adjacent WCE frames with W-MVT sequentially, the temporal correlated images with certain similarity are grouped into the same clip. Eventually, to consider gradient varying characteristic in one clip, the adaptive K-means clustering algorithm is adopted to keep key frames while remove redundant frames further. Experimental results show that two evaluation indicators-F-measure and compression ratio achieve 81.94% and 80.31%, which validates the effectiveness of the proposed WCE redundant image elimination (WCE-RIE) method.

Wireless Capsule Endoscopy (WCE) is a painless and noninvasive technique that allows physicians to visualize the entire small bowel. Before WCE was introduced, examining the entire small bowel was impossible without surgical procedure. Although WCE is a technology breakthrough, the manual video diagnosis session is time consuming and is prone to human cognition errors. Therefore, gastroenterologists urge computer vision researchers to develop computer-aided diagnosis systems to assist the review session. In this paper, we would like to present an ulcer detection scheme that identifies ulcers in WCE video. A saliency map is created by means of channel mixer to highlight suspicious regions. Once the suspicious regions are identified, we extract textural features along the contour and classify them using a trained classifier for ulcer validation.

To detect the region and bleeding frame in the wireless capsule endoscopy video, an automatic computer-aided technique is highly demanded to reduce the burden of physicians. The wireless capsule endoscopy (WCE), is an imaging technology which is recently established and doesn’t require any wired device. This device detects abnormalities in GI tract, i.e. (colon, esophagus, small intestine, and stomach). A WCE video consists of 57,000 images. It is very hard to examine by clinicians. To determine bleeding photos out of fifty-seven thousand WCE images makes the task very hard and expensive. The main goal is to develop an automatic obscure bleeding detection method by using superpixel segmentation and naive Bayes classifier. Naive Bayes and superpixel segmentation are used for this problem.

Purpose: Wireless video capsule endoscopy (VCE) is a new technology that enables us to visualize the entire small bowel mucosa. It involves swallowing a video capsule endoscope, which is painless and relatively safe. Its use has been established for suspected small bowel bleeding, and the role of capsule endoscopy in the investigation of inflammatory bowel disease, iatrogenic disease, polyposis syndromes and celiac disease is evolving. Aim: To describe the overall experience of video capsule endoscopy in a large tertiary care center. Methods: All charts of patient who underwent VCE between 2001 and 2005 were retrospectively reviewed. There were 466 charts available with the complete VCE reports. Results: There were 466 patients who underwent VCE. The mean age was 59.5 years (range 7–90). There were 265 female patients and 201 male patients. The most common indication for the procedure was obscure gastrointestinal bleeding (84.9%), followed by abdominal pain (8.8%), Crohn's (7.1%), diarrhea (5.4%) and abnormal imaging (3.2%). A possible cause of obscure gastrointestinal bleeding was identified in 76.8% of patients. VCE identified a possible cause of abdominal pain and diarrhea in 41.8% and 28% respectively. 16% of patients with diarrhea were found to have newly diagnosed Crohn's. 57.6% of patients with colonic Crohn's disease were found to have small bowel involvement. The etiology of abnormal imaging was found in 60% of patients. Small intestinal mass was identified in 3.6% of patients and 88.3% of these patients presented with obscure gastrointestinal bleeding. Conclusions: VCE is a clear choice for evaluation of obscure gastrointestinal bleeding VCE is an important tool in diagnosing previously unrecognized Crohn's disease. VCE is an important tool in diagnosing upper gastrointestinal involvement in established colonic Crohn's disease. VCE is an important supplementary diagnostic tool in investigating not only obscure gastrointestinal bleeding but also Crohn's disease.

Deep chroma prediction of Wyner–Ziv frames in distributed video coding of wireless capsule endoscopy video

Wireless Capsule Endoscopy (WCE) is a relatively new technology (FDA approved in 2002) allowing doctors to view most of the small intestine. WCE transmits more than 50,000 video frames per examination and the visual inspection of the resulting video is a highly time-consuming task even for the experienced gastroenterologist. Typically, a medical clinician spends one or two hours to analyze a WCE video. To reduce the assessment time, it is critical to develop a technique to automatically discriminate digestive organs and shots each of which consists of the same or similar shots. In this paper a multi-level WCE video segmentation methodology is presented to reduce the examination time.

In this paper, we present a new solution for extracting key frames from motion capture data using an optimization algorithm to obtain compact and sparse key frame data that can represent the original dense human body motion capture animation. The use of the genetic algorithm helps determine the optimal solution with global exploration capability while the use of a probabilistic simplex method helps expedite the speed of convergence. By finding the chromosome that maximizes the fitness function, the algorithm provides the optimal number of key frames as well as the low reconstruction error with an ordinary interpolation technique. The reconstruction error is computed between the original motion and the reconstruction one by the weighted differences of joint positions and velocities. The resulting set of key frames is obtained by iterative application of the algorithm with initial populations generated randomly and intelligently. We also present experiments which demonstrate that the method can effectively extract key frames with a high compression ratio and reconstruct all other non key frames with high quality.

Wireless capsule endoscopy (WCE) has been validated to be an important tool in the evaluation of gastrointestinal (GI) tract. Compared with traditional endoscope technologies, its non-invasiveness property meets with great favor of patients. However, from physician's point of view, WCE video suffers from low resolution, limited illumination, irregular movement, more importantly, imbalanced rate of abnormality which brings a lot of challenges for diagnosis. These challenges motivate us to devise an approach to guide the physicians to focus on the informative frames which could be convenient for review of the content of the GI tract. This paper presents a novel approach for automatic selection of the WCE video frames with lumen and coherent motility. We adopt lumen detection based on mean shift to provide robust and reliable selection of lumen. Together with the evaluation of coherent motility, we can provide a full and fast approach to select the informative frames for diagnosis. The experiments on real date are presented to show the performance of our proposed method.

For the existing traditional method to extract key frames from the video sequence with gradual change of content, there are problems of having too much redundancy or too much loss in the extraction results and other issues, a new key frames extracting method for adaptively extracting the video key frames has proposed in the paper. The experimental results have shown that the proposed method can effectively extract video key frames, especially for the video with gradual change of content, which can express more succinctly and comprehensively. The experiments on a large number of videos containing various scenes have shown that the key frames extracted by the proposed algorithm can more fully express the main content of video. When dealing with content gradient video, the effect is better. The content adaptively determines the number of key frames based on video.

Wireless video capsule endoscopy (CE) is a new technology that allows visualization of the entire small intestinal mucosa. It is indicated for the evaluation of obscure gastrointestinal bleeding (OGIB) and other disorders of the small intestine. Studies to date suggest that CE is safe and associated with few adverse events. A concern, which has not been studied, is the potential effect of CE on implanted cardiac devices such as implantable cardiac defibrillators (ICD) and other electromedical devices. We previously found CE to be safe in patients with cardiac pacemakers. The primary aim of this study was to evaluate the safety of CE in patients with ICDs who were being evaluated for OGIB. In addition, a secondary aim of the study was to determine whether ICDs had any effect on the images captured by CE. Patients referred for the evaluation of OGIB and who also had an ICD were enrolled into the study after informed consent. Five consecutive patients (four females and one male; mean age: 72 yr; range: 60-81 yr) with ICDs were studied. All patients had transvenous endocardial ICDs located in the chest. Prior to CE, patients had a baseline electrocardiogram (ECG) and ICD interrogation. Thereafter, CE was performed in a hospital setting with telemetry monitoring performed simultaneously. A post-procedure ICD interrogation was carried out to evaluate changes in programmed parameters. A cardiologist and ICD nurse specialist together reviewed both the telemetry monitor and the post-procedure ICD interrogation on each patient. When CE studies were reviewed, observations pertaining to technical difficulties and interference with video imaging were documented. No arrhythmia or other adverse cardiac events were noted during capsule transmission. No interference by the ICD on the CE video images was seen. CE was performed safely in these five patients with ICDs, and was not associated with any adverse cardiac events. ICDs also do not appear to interfere with video capsule imaging.

The Wireless Capsule Endoscopy (WCE) is a painless and non-invasive procedure that allows clinicians to visualize the entire Gastrointestinal Tract (GIT) and detect various abnormalities. During the inspection of GIT, numerous images are acquired at a rate of approximately 2 frames per second (fps) and recorded into a video footage (containing about 55,000 images). Inspecting the WCE video is very tedious and time consuming for the doctors, resulting in limited application of WCE. Therefore, it is crucial to develop a computer aided intelligent algorithm to process the huge number of WCE frames. This paper proposes an ulcerated frame detection method based on RGB and CIE Lab colour spaces. In order to select and provide the classifier with the bands containing most ulcer information, a statistical analysis of ulcerated images pixel based is proposed. The resulting band selection will enhance the classification results and increase the sensitivity and specificity with regards to ulcerated frame identification.