Color Content Matching of MPEG-4 Video Objects

Exploiting the relevant speech information that is embedded in facial images has been a significant research topic in recent years, because it has provided complementary information to acoustic signals for a wide range of automatic speech recognition (ASR) tasks. Visual information is particularly important in many real applications where acoustic signals are corrupted by environmental noises. This chapter reviews the most recent advances in feature extraction and representation for Visual Speech Recognition (VSR). In comparison with other surveys published in the past decade, this chapter presents a more up-to-date survey and highlights the strengths of two newly developed approaches (i.e., graph-based learning and deep learning) for VSR. In particular, we summarise the methods of using these two techniques to overcome one of the most challenging difficulties in this area-that is, how to automatically learn good visual feature representations from facial images to replace the widely used handcrafted features. This chapter concludes by discussing potential visual feature representation solutions that may overcome the remaining challenges in this domain.

Despite many years of research, object tracking remains a challenging problem, not only because of the variety of object appearances, but also because of the complexity of surrounding environments. In this research, we present an algorithm for single object tracking using a particle filter framework and color histograms. Particle filters are iterative algorithms that perform predictions in each iteration using particles, which are samples drawn from a statistical distribution. Color histograms are embedded in these particles, and the distances between histograms are used to measure likelihood between targets and observations. One downside of color histograms is that they ignore spatial information, which may produce tracking failure when objects appear that are similar in color. To overcome this disadvantage, we propose a saliency-based weighting scheme for histogram calculation. Given an image region, first its saliency map is generated. Next, its histogram is calculated based on the generated saliency map. Pixels located in salient regions have higher weights than those in others, which helps preserve the spatial information. Experimental results showed the efficiency of the proposed appearance model in object tracking under various conditions.

Image retrieval using color histograms generated by Gauss mixture vector quantization

We propose a new method for segmenting image sequences in sports scenes containing brisk movement. An important feature of this method is the computation of color histograms in the areas for the turf, which is done interactively during rehearsal. Another important feature is automatic morphological segmentation during broadcast. The morphological segmentation consists of two operations: coarse segmentation by binary reconstruction based on the areas detected by thresholding the color histogram, and fine segmentation by watershed transformation with markers. It is shown that this new method achieves accurate segmentation in sport scenes.

The video object mechanism is based on the appearance representation of the object. Tracking objects in a video has applications in video surveillance and other some applications. In previous method does not handle the case of full occlusion. The case of full occlusion can be handled by setting an LSK similarity threshold, which stops tracking when the object is lost. The objects can be tracked based on the compression domain and pixel domain. The spatial information of the object that is being tracked is retrieved using Local Steering Kernels. The Color features are extracted using color histograms. The features similarities between a candidate object ROI and the object ROI in the previous frame and the last stored object instance in the object model are measured. For extracting the foreground extended Kalman filter is used. This increases the tracking efficiency of the algorithm. Index Terms: Color histograms, localsteeringkernels, and visualobjecttracking.

Detecting visual changes in environments is an important computation with many applications in robotics and computer vision. Security cameras, remotely operated vehicles, and sentry robots could all benefit from robust change detection capability. We conjecture that if one has a mobile camera system the number of visual scenes that are experienced is limited (compared to the space of all possible scenes) and that the scenes do not frequently undergo major changes between observations. These assumptions can be exploited to ease the task of change detection and reduce the computational complexity of processing visual information by utilizing memory to store previous computations. We demonstrate a method to learn the distribution of visual features in an environment via a self-organizing map. Additionally, the spatial distribution of these features can be learned if a positional signal is available. Our method uses a low dimensional representation of visual features to rapidly detect changes in current visual inputs. The model encodes spatially-distributed color histograms of real world visual scenes captured by a camera moved through an environment. The distribution of the color histograms is learned using a self-organizing map with location (when available) and color data. We present tests of the model on detecting changes in an indoor environment.

Content based Image Retrieval (CBIR) allows automatically extracting target images according to objective visual contents of the image itself. Representation of visual features and similarity match are important issues in CBIR. Colour and texture features are important properties in CBIR systems. In this paper, a combined feature descriptor for CBIR is proposed to enhance the retrieval performance for CBIR. This method is developed by exploiting the wavelets and colour histogram moments. First, Haar wavelet is used to decompose colour images into wavelet coefficients. Then image feature extraction and similarity matching are performed by means of histogram moments. Ten categories of colour images are used to test the proposed technique. Experiment results show significant improvement of retrieval efficiency as compared to that of histogram moments as well as 2D Discrete Wavelet Transform.

The ever-increasing numbers of medical images in digital format are generated in clinical practice every day. These images include high-resolution and temporal data of various modalities and which constitute an important source of anatomical and functional information for diagnosis of diseases, research and education. So how to manage these large data and utilize them effectively and efficiently possess significant technical challenges. Thus, the technique of Content-based Medical Image Retrieval (CBMIR) emerges as the times require. However, current CBMIR is not sufficient to capture the semantic content of an image and difficult to provide good results according to the predefined categories in the medical domain for less using the medical knowledge. Accordingly, in this paper a mapping modelling of visual feature and knowledge representation is proposed to approach for medical image retrieval. Firstly, the low-level fusion visual features are extracted based on intensity, texture, and their extended versions. Secondly, a set of disjoint semantic tokens with appearance in lung CT images is selected to define a vocabulary based on medical knowledge representation. Finally, a mapping modeling is investigated to associate low-level visual image features with their high-level semantic. Experiments are conducted with a medical image database consisting of 220 lung medical images obtained from the Heilongjiang Province Hospital. And the comparison of the retrieval results shows that the approach proposed in this paper is effective.

Nowadays, the majority of images are in JPEG and MPEG compressed formats, and JPEG2000 is considered to be the next generation of compression standard due to the high- performance of discrete wavelet transform (DWT). It is time- consuming and occupies too much memory in conventional image retrieval ways. In order to solve these problems, we use grading retrieval techniques to implement image retrieval based on discrete cosine transform (DCT) compressed domain and DWT compressed domain. For image retrieval based on DCT domain, we use color features: color moment and color histogram, to describe content of images and propose a new dynamic color space quantization based on color distribution; For image retrieval based on DWT domain, we use texture features as two level feature vectors. The mean and standard deviation of low frequency sub-band coefficients are used as the first level retrieval. The means and standard deviations of selected high frequency sub-band coefficients are used as the second level retrieval. Furthermore, the third level retrieval is achieved by the fast wavelet histogram. Our experiment results clearly show that the two grading image retrieval algorithms work better than other algorithms: store memory is reduced and retrieval accuracy is improved. Index Terms—Content based image retrieval (CBIR), compressed domain, discrete cosine transform (DCT), discrete wavelet transform (DWT), color features, texture features

We present a video analysis framework that integrates prior knowledge in object tracking to automatically detect humans and faces, and can be used to generate abstract representations of video (key-objects and object trajectories). The analysis framework is based on the fusion of external knowledge, incorporated in a person and in a face classifier, and low-level features, clustered using temporal and spatial segmentation. Low-level features, namely, color and motion, are used as a reliability measure for the classification. The results of the classification are then integrated into a multitarget tracker based on a particle filter that uses color histograms and a zero-order motion model. The tracker uses efficient initialization and termination rules and updates the object model over time. We evaluate the proposed framework on standard datasets in terms of precision and accuracy of the detection and tracking results, and demonstrate the benefits of the integration of prior knowledge in the tracking process.

This work is about the relevance of Gibson's concept of affordances [1] for visual perception in interactive and autonomous robotic systems. In extension to existing functional views on visual feature representations, we identify the importance of learning in perceptual cueing for the anticipation of opportunities for interaction of robotic agents. We investigate how the originally defined representational concept for the perception of affordances - in terms of using either optical flow or heuristically determined 3D features of perceptual entities - should be generalized to using arbitrary visual feature representations. In this context we demonstrate the learning of causal relationships between visual cues and predictable interactions, and emphasize on a novel framework for cueing and hypothesis verification of affordances that could play an important role in future robot control architectures. We argue that affordance based perception should enable systems to react to environment stimuli both more efficient and autonomous, and provide a potential to plan on the basis of responses to more complex perceptual configurations. We verify the concept with a concrete implementation applying state-of-the-art visual descriptors and regions of interest within a simulated robot scenario and prove that these features were successfully selected for predicting opportunities of robot interaction.

Recently, the aspect of visual perception has been explored in the context of Gibson's concept of affordances [1] in various ways [4-9]. In extension to existing functional views on visual feature representations, we focus on the importance of learning in perceptual cueing for the anticipation of opportunities for interaction of robotic agents. Furthermore, we propose that the originally defined representational concept for the perception of affordances - in terms of using either optical flow or heuristically determined 3D features of perceptual entities - should be generalized towards using arbitrary visual feature representations. In this context we demonstrate the learning of causal relationships between visual cues and associated anticipated interactions, using visual information within the framework of Markov Decision Processes (MDPs). We emphasize a new framework for cueing and recognition of affordance-like visual entities that could play an important role in future robot control architectures. Affordance-like perception should enable systems to react to environment stimuli both more efficiently and autonomously, and provide a potential to plan on the basis of relevant responses to more complex perceptual configurations. We verify the concept with a concrete implementation of learning visual cues by reinforcement, applying state-of-the-art visual descriptors and regions of interest that were extracted from a simulated robot scenario and prove that these features were successfully selected for their relevance in predicting opportunities of robot interaction.

This work is about the relevance of Gibson’s concept of affordances [1] for visual perception in interactive and autonomous robotic systems. In extension to existing functional views on visual feature representations, we identify the importance of learning in perceptual cueing for the anticipation of opportunities for interaction of robotic agents. We investigate how the originally defined representational concept for the perception of affordances – in terms of using either optical flow or heuristically determined 3D features of perceptual entities – should be generalized to using arbitrary visual feature representations. In this context we demonstrate the learning of causal relationships between visual cues and predictable interactions, using both 3D and 2D information. In addition, we emphasize a new framework for cueing and recognition of affordance-like visual entities that could play an important role in future robot control architectures. We argue that affordance-like perception should enable systems to react to environment stimuli both more efficient and autonomous, and provide a potential to plan on the basis of responses to more complex perceptual configurations. We verify the concept with a concrete implementation applying state-of-the-art visual descriptors and regions of interest that were extracted from a simulated robot scenario and prove that these features were successfully selected for their relevance in predicting opportunities of robot interaction.

Sugarcane plant diseases are caused by various insects, bacteria, viruses etc., To reduce the impact of the diseases and thereby increase sugarcane yield, early detection of diseases is required. Existing techniques for detection of plant diseases focus on a single type of disease which does not have a significant impact on increasing the sugarcane crop yield. We propose an efficient Sugarcane Disease Classification system (SDC) using soft computing techniques. The proposed system has four different modules; colour space conversion, histogram generation, feature extraction and classification. Sugarcane image is represented by two different colour models RGB and La*b*.Wavelet and contourlet based energy features from both RGB and La*b* colour models are extracted and finally classified using a neural network. The proposed SDC system is tested on 1000 sugarcane images affected with ten different diseases (100 images per category) by a 10-fold cross validation approach. Results show that the SDC system provides an average classification accuracy of 99.89%

Introducing a weighted non-negative matrix factorization for image classification