Hybrid Codebook Model for Foreground Object Segmentation and Shadow/Highlight Removal

in order to solve the adverse effects of strong light and shadow on the test results, a fusion frame difference and background subtraction method in the HSV space is used in this paper. By using frame difference method to solve the effect of strong light, but frame difference method can not detect object when the object do not move, the method of background subtraction can detect it, building Gaussian background model in the HSV space can eliminate shadows. Empirical results show that the method of fusion frame difference and background subtraction in the HSV space can get overcome the effect of strong light and shadows. Fusion background subtraction and frame difference method based on establishing a Gaussian mixture model in HSV space can overcome the disadvantages of the frame difference method, at the same time it can also solve the false detection of object which result from the background subtraction method.

For health and safety monitoring in civil constructions such as bridges, roads and pavements, segmenting the regions of interest is the fundamental requirement for image analysis at high-level semantic. One of the major structural problems in concrete and asphalt surfaces are cracks, which start with harming the visual aspect of the construction and further lead to failure of the construction. This study makes four contributions to extract the image foreground from the background in order to address the crack detection task on the asphalt and concrete surfaces. Specifically, we model cracks as foreground objects and concrete or asphalt surfaces as textured background. The first contribution of this research is a hybrid image processing method for crack detection. In this method, the cracks are modelled as linear structures on the background of textured concrete or asphalt surfaces, which can be extracted by combining structure extraction with global pattern distribution. There are two phases in this model towards creating the final structure-texture map. The first phase is extracting strong structures or edges using relative total variation measures, which produces a structure feature map by preserving the edges and suppressing the background noises. The second phase calculates the spatial distribution of textures across the image. A bag-of-words model is used in this phase to quantise the texture pattern, which in crack detection application is the widely distributed road texture background. The local structure map and the global distribution map extract the crack structure and the textured background, respectively. The final crack is extracted by fusing these two maps and applying binarisation as the post-processing step. This model achieved a better result compared with the local structure extraction and the saliency method. As the second contribution, a large-scale dataset of asphalt and concrete crack images is prepared, including images with their corresponding high-resolution pixel-wise labels. To the best of our knowledge, until the completion of this thesis, there has not been any crack image dataset with pixel-wise ground truth labels available. The original crack image set includes 2532 images of cracks on brick and asphalt surfaces. This image set is split into training, validation and testing sets with the ratio of 50/25/25. Two augmentation techniques of rotation and ipping are applied to only the training set while validation and testing sets are locked in order to prevent the data leakage. All models are learned on the training set after fine-tuning the hyper-parameters. After each tuning, an early estimate of the model accuracy is obtained using the validation set. Finally, an unbiased performance estimation of the fitted model is provided on the testing set. The third contribution of this thesis is developing two encoder-decoder networks by exploring the recent advances of deep learning research for crack detection and applying them on our crack dataset. The first network is inspired by DeepLab, which is a modified ResNet architecture. In this network the last pooling layer is replaced with an Atrous Spatial Pyramid Pooling (ASPP) module. This encoder-decoder structure is designed to classify each image pixel into two foreground cracks or textured background. The second model is inspired by Full Resolution Residual Network (FRRN), which is a ResNetlike network with two residual and pooling streams to extract the high- and low-level features, respectively. The combination of di erent level features in this model improves the localization of crack pixels as well as the recognition of the crack structure as a whole. As FRRN outperforms DeepLab on crack classification, we select it as the baseline for further research. In our last contribution, we optimise the FRRN model by reducing the number of parameters. Inspired by the Inception module which significantly improved the utilization of the computing resources inside the GoogLeNet, we proposed Incepted FRRN (I-FRRN) network by embedding the Inception module inside the FRRN. Combining these two structures, our proposed model records 88.14% accuracy in classifying the positive class with 0.22% improvement, while having less than half the number of parameters compared with FRRN. The results show that the proposed architecture achieves significant computational e ciency gains and comparable or higher class-accuracy in crack classification task over the baseline model.

This paper proposes a novel shadow elimination method for solving the shadow occlusion problems of vehicle analysis. Different from traditional methods which only consider intensity properties in shadow modeling, this method introduces a new important feature to eliminate all unwanted shadows, i.e., lane line geometries. In this approach, a set of moving vehicles are first segmented from backgrounds by using a background subtraction technique. At this moment, each extracted vehicle may contain shadows which cause the failure of further vehicle analysis. To remove these unwanted shadows, a histogram-based method is then proposed for detecting different lane dividing lines from video sequence. According to these lines, a line-based shadow modeling process is then applied for shadow elimination. Two kinds of lines are used here for shadow elimination, i.e., the ones parallel and vertical to lane directions, respectively. Different type of lines has different capabilities to eliminate different kinds of shadows. Experiments demonstrate that approximately 92% of shadows can be successfully eliminated from moving vehicles.

We propose an effective online background subtraction method, which can be robustly applied to practical videos that have variations in both foreground and background. Different from previous methods which often model the foreground as Gaussian or Laplacian distributions, we model the foreground for each frame with a specific mixture of Gaussians (MoG) distribution, which is updated online frame by frame. Particularly, our MoG model in each frame is regularized by the learned foreground/background knowledge in previous frames. This makes our online MoG model highly robust, stable and adaptive to practical foreground and background variations. The proposed model can be formulated as a concise probabilistic MAP model, which can be readily solved by EM algorithm. We further embed an affine transformation operator into the proposed model, which can be automatically adjusted to fit a wide range of video background transformations and make the method more robust to camera movements. With using the sub-sampling technique, the proposed method can be accelerated to execute more than 250 frames per second on average, meeting the requirement of real-time background subtraction for practical video processing tasks. The superiority of the proposed method is substantiated by extensive experiments implemented on synthetic and real videos, as compared with state-of-the-art online and offline background subtraction methods.

In this paper, we propose a data-driven foreground object detection technique which can detect foreground objects from a moving camera. We propose to build a data-driven consensus foreground object template (CFOT) and then detect the foreground object region in each frame. The proposed foreground object detection technique is equipped with the following functions: (1) the ability to detect the foreground object captured by a fast moving camera ; (2) the ability to detect a low contrast (spatially/temporally) foreground object; and (3) the ability to detect a foreground object from a dynamic background. There are three contributions of our method: (1) a newly proposed data-driven foreground region decision process for generating the CFOT has been shown robust and efficient; (2) a foreground object probability is proposed for properly dealing with the imperfect initial foreground region estimations; and (3) a CFOT is generated for precise foreground object detection.

Foreground detection is one of the active research topics in computer vision and a precondition for intelligent video analytics. Background subtraction is often considered as a reliable method for extracting foreground from the video sequences. The real-world challenges such as intermittent object motion, dynamic backgrounds, bad weather conditions, shadows, and illumination variations affect the performance of foreground detection. Most of the conventional background subtraction methods result in faulty detection and fragmented foreground in case of complex scenes. In recent years, deep neural networks make revolutionary advances in the area of foreground detection. The convolutional neural networks based background subtraction methods are notable for their performance but their performance drop in case of unseen videos and incur high-computational cost. In this paper, we investigate the fusion of superpixel with an adaptive background modeling for foreground detection in several challenging videos. A non-iterative clustering method is adopted for superpixel clustering because of its computational efficiency. Integration of effective superpixel algorithm with background subtraction has paved the way for improvements in unsupervised background subtraction methods. The proposed method is evaluated on different videos of changedetection.net 2014 dataset in order to validate its effectiveness on critical challenges of background subtraction.KeywordsBackground modelingForegroundMoving objectsSuperpixelsUnsupervised

Background subtraction method is widely used in most of the video motion detection algorithms especially for video surveillance application. Background subtraction is used to extract moving or static foreground objects from the background scene. The efficiency of foreground-background segmentation heavily relies on background model which must be able to cope with changes in the scene and granularity of the foreground objects. The robust background model can produce good foreground segmentation results and it is still a great challenge to get accurate and high performance result today. In this paper, a video foreground-background segmentation approach is proposed. This approach is based on Codebook (CB) model with Kalman Filter. This approach can be used to extract foreground objects from the video stream. The Lab color space is used in this approach to calculate color difference between two pixels using CIEDE2000 color difference formula. Extracted foreground object from video sequence using this approach is useful for object detection in video surveillance applications. This approach has been tested with PETS and CDnet2014 datasets and segmentation results accuracy are evaluated compare with ground truth.

For the detection of moving objects, background subtraction methods are widely used. In case the background changes, we need to update the background in real-time for the reliable detection of foreground objects. An adaptive Gaussian mixture model (GMM) combined with probabilistic learning is one of the most popular methods for the real-time update of the complex and dynamic background. However, the probabilistic learning approach does not work well in high traffic regions. In this paper, we classify each pixel into four different types: still background, dynamic background, moving object, and still object, and update the background model based on the classification. For the classification, we analyze a sequence of frame differences at each pixel and its neighborhood. We experimentally show that the proposed method learn complex and dynamic backgrounds in high traffic regions more reliably, compared with traditional methods.

Cut set-based Dynamic Key frame selection and Adaptive Layer-based Background Modeling for background subtraction

Background subtraction models based on mixture of Gaussians have been extensively used for detecting objects inmotion in a wide variety of computer vision applications. However, background subtraction modeling is still an openproblem particularly in video scenes with drastic illumination changes and dynamic backgrounds (complexbackgrounds). The purpose of the present work is focused on increasing the robustness of background subtractionmodels to complex environments. For this, we proposed the following enhancements: a) redefine the modeldistribution parameters involved in the detection of moving objects (distribution weight, mean and variance), b)improve pixel classification (background/foreground) and variable update mechanism by a new time-space dependentlearning-rate parameter, and c) replace the pixel-based modeling currently used in the literature by a new space-timeregion-based model that eliminates the noise effect caused by drastic changes in illumination. Our proposed schemecan be implemented on any state of the art background subtraction scheme based on mixture of Gaussians toimprove its resilient to complex backgrounds. Experimental results show excellent noise removal and object motiondetection properties under complex environments.

We propose an effective background model using mixture of Gaussians and Laplacian pyramid decomposition for foreground object segmentation from complex scene containing stationary and moving objects. The Laplacian pyramid is employed to decompose the input image into a low-frequency big scale image and a high-frequency image. We build two mixtures of Gaussians in each pixel to represent the statistical characteristics of the stationary and moving points with proper feature vectors. Big scale foreground objects are obtained by fusing the results from stationary and dynamic models. Original foreground objects are then restored using the established model, low-frequency and high-frequency images. The experiments we performed here on complex scenes containing dynamic background objects have showed better performance and less memory cost compared.

Moving object detection is vitally used in video surveillance applications. Traditional Gaussian mixture model (GMM) based background subtraction (BGS) methods are usually performs well when background is stationary. However, they require parameter tuning to deal with dynamic backgrounds, whose background pixel values change over time. Particularly, the threshold which determines the pixels associated with moving objects from the resultant of BGS. To tackle this problem there is no ultimate solution. Considering that, this paper intents to present a novel idea to update the threshold of GMM based BGS with respect to color distortion, similarity and illumination measures in pixel level. Extensive experiments were carried out to demonstrate the effectiveness of the proposed method in comparison to some of the long-familiar GMM based BGS methods in literature. However, note that this paper is not attempted to provide a real-time technique, but rather to investigate the potential utilization of the aforementioned measures to set a threshold automatically to detect moving objects in video sequences.

The Iranian medicinal plants, such as Satureja bachtiarica have been utilized as traditional medicines by the indigenous people of Chaharmahal va Bakhtiari in Iran. In this study, Satureja bachtiarica were dried in suitable condition (in shadow) after extraction with watery, ethanol 96 %, methanol 96% and 20% glycerin antimicrobial effect of extract were determined by “screening antimicrobial activity” and “disk agar diffusion test” in 10, 20, 30 and 40 mg/ml concentration of the extract against Streptococcus pyogenes PTCC 1447, Pseudomonas aeruginosa PTCC 1310 and Staphylococcus epidermidis PTCC 1435. The results showed that aqueous, ethanol 96%, methanol 96% and 20% glycerin extracts were quite effective in 2 mg/ml concentration on Streptococcus pyogenes and Staphylococcus epidermidis and were prevented from growth them on medium, while extracts have no certain antimicrobial effect on Pseudomonas aeruginosa. In “disk agar diffusion method”, 10, 20, 30 and 40 mg/ml aqueous, ethanol 96%, methanol 96% and 20% glycerin extract concentrations, was inhibited effect on Streptococcus pyogenes and Staphylococcus epidermidis , but 40 mg/ml aqueous and 30 and 40 mg/ml ethanol 96%, methanol 96% and 20% glycerin extract concentrations, has inhibited effect on Pseudomonas aeruginosa prevent them growing. The results indicate that alcoholic and aqueous extracts of Satureja bachtiarica have the greatest effect on gram positive bacterium Streptococcus pyogenes . As a result, aqueous and alcoholic extracts of Satureja bachtiarica , have been strong antimicrobial activity against many food pathogen bacteria.

One of the fundamental problems of computer vision is to extract 3D shape and motion from images. This can be achieved when a scene or object is observed from different viewpoints or over a period of time. There is a wide range of applications, ranging from digitizing cultural heritage to vision-based autonomous robot navigation. This talk will present several approaches to solve this problem. First, we’ll discuss techniques for 3D shape recovery for static objects and scenes. One particular case is the 3D mapping and localization in large environments from images, e.g. urban 3D reconstruction from vehicleborne cameras or localization from cell-phone images. Next, we’ll shift our focus to modeling dynamic scenes, e.g. people who are moving around. In addition to explicitly 3D modeling an event, we'll consider the possibility to perform video-based rendering from casually captured videos. Speaker Bio-Sketch: Marc Pollefeys is a full professor in the Dept. of Computer Science of ETH Zurich since 2007 where he is the head of the Institute for Visual Computing and leads the Computer Vision and Geometry lab. He currently also remains associated with the Dept. of Computer Science of the University of North Carolina at Chapel Hill where he started as an assistant professor in 2002 and became an associate professor in 2005. Before this he was a postdoctoral researcher at the Katholieke Universiteit Leuven in Belgium, where he also received his M.S. and Ph.D. degrees in 1994 and 1999, respectively. His main area of research is computer vision. One of his main research goals is to develop flexible approaches to capture visual representations of real world objects, scenes and events. Dr. Pollefeys has received several prizes for his research, including a Marr prize, an NSF CAREER award, a Packard Fellowship and a European Research Council Starting Grant. He is the author or co-author of more than 130 peer-reviewed publications. He is the General Chair for the European Conference on Computer Vision 2014 (ECCV), was a Program Co-Chair for the IEEE Conference on Computer Vision and Pattern Recognition 2009 (CVPR), was general/program co-chair of the Third Symposium on 3D Data Processing, Visualization and Transmission and has organized workshops and courses at major vision and graphics conferences and has served on the program committees of many conferences. Prof. Pollefeys is/was on the Editorial Board of the IEEE Transactions on Pattern Analysis and Machine Intelligence and the International Journal of Computer Vision as well as several other journals in computer vision, graphics and robotics.

Cucumber fruit appearance quality is an important basis of growth status. In order to improve the quality detection accuracy and processing efficiency of cucumber color image under complicated background, an improved GrabCut algorithm was proposed to extract the cucumber boundary. Firstly, including pixel size normalization, rectangular box set and scale image resolution, pretreatments of cucumber image were adopted to reduce the iteration times and operation time of GrabCut algorithm. Then, the Gaussian mixture model was chosen to find out the possible prospect of target region and background region in the preprocessed rectangular frame on the preliminary modeling. Meanwhile, by the optimization of K-means cluster to the initial GMM model, the effective target area was extracted. Finally, the whole image noise and serrated boundary was removed by morphological operations to segment the outline of the complete target prospects with appropriate structure size. And then the cucumber appearance quality detection instrument was designed to extract the texture and shape features exactly, so that it could obtain cucumber appearance quality and evaluate its growth effectively. With the segmentation experiments by almost 300 cucumber original images from greenhouse in Shandong Province, the results showed that the improved GrabCut algorithm could effectively extract the complete and smooth boundary of cucumber. With relatively high segmentation evaluation index, the precision was 93.88%, the recall rate was 99.35%, the F-Measure reached 96.53%, and the misclassification error was controlled at minimum 5.84%. The average running time was shortened to 1.4023 s. The comparison results showed that the improved GrabCut algorithm was the best, followed by the SLIC and traditional GrabCut method. Cucumber appearance quality detection instrument could also extract more accurate feature parameters. And it could meet the basic growth condition assessment by automatic image processing. Keywords: cucumber, complicated background, quality detection, image processing, GrabCut DOI: 10.25165/j.ijabe.20181104.3090 Citation: Ye H J, Liu C Q, Niu P Y. Cucumber appearance quality detection under complex background based on image processing. Int J Agric & Biol Eng, 2018; 11(4): 193-199.