A novel foreground region analysis using NCP-DBP texture pattern for robust visual tracking

Dynamic backgrounds and sudden illumination changes are two of the major problems associated with background subtraction techniques. In this paper, we present a novel approach to background subtraction that addresses both of these challenges. In particular, we present an improved codebook background modelling and subtraction technique. We utilise image segmentation on the background image and model the background with a codebook for each pixel along with a pseudo background layer. We perceive background motion as an occlusion of one background layer by a nearby background layer. In other words, sliding of one background layer over a neighbouring layer causes background motion and will hence result in false segmentation. We present our approach of codeword spreading across layer boundaries to handle background motion. Furthermore, we present a two-step update of the background codebook to handle both sudden and gradual illumination changes.

The schooling behavior of both light- and dark-adapted Astyanax mexicanus (Fillipi) was observed under sudden changes in illumination in the range 10−7 to 100 ft-c at different times in the diel cycle. In darkness the fish were dispersed, swam slowly, and showed no signs of mutual recognition of one another. There was no simple relationship between the schooling response and the illumination, but in general the frequency, size, and durability of schools were proportional to the illumination. Schooling was maximal above 10−1 ft-c and approached zero at 10−6 ft-c. The schooling at any level of illumination varied with the testing procedures. The lowest responses were obtained with a gradual reduction of illumination in the evening. At illumination below 10−2 ft-c maximal schooling was obtained from dark-adapted fish. From 10−2 ft-c upward, it was obtained from light-adapted fish. The schooling rate (number of fish in a school) reached 100% for a few seconds at 10−3 ft-c and for longer periods at higher illumination, but was never sustained at the 100% level throughout any 4-min period. Schooling was generally enhanced by any sudden change in illumination but was never sustained at a high rate under monotonous conditions. The decline with monotony was equated with reduction of the fright situation. The decline with reduced illumination was attributed to the reduction of visual acuity and visibility, and occurred at a greater rate among faster-swimming fish. Notemigonus crysoleucas (Mitchill), swimming very slowly, exhibited high schooling rates at 10−5 ft-c. Earlier work showed that the retina of Astyanax in continuous darkness undergoes a rhythmic diel shift. The behavior showed that this retinal shift is not the classical transformation from scotopic to photopic vision. Dark-adapted during the day, the fish demonstrated rapid visual perception under sudden illumination at any value from 10−6 to 10−0.69 ft-c. Dark-adapted fish at night exhibited increasingly longer light-shock reactions with sudden illumination above 10−2 ft-c. According to schooling responses the cone threshold was about 10−3 ft-c, and the rod threshold about 10−6 ft-c. The mesopic range extended from about 10−3 to 10−2 ft-c.

The analysis of moving objects in video sequences has been a paramount issue in applications related to intelligent surveillance systems, robotics, and medicine. Although several works aimed to analyze objects in video sequences have been reported, many of them need manual parameter adjustments and they are not tolerant to illumination changes and dynamic backgrounds. Therefore, a novel scheme termed Dynamic Retinotopic SOM based on an adaptive artificial neural network, to detect moving objects is proposed in this work. The neural network is a model based on the mechanisms of the visual cortex that we called Retinotopic SOM (RESOM) and it is also proposed in this paper. Furthermore, RESOM is a real-time neural network that can adapt its learning parameters based on the scene behavior and it mimics perception abilities. A quantitative comparison with other segmentation methods reported in the literature using real video scenes showed that the proposed DR-SOM segmentation method automatically adjusts its parameters and outperforms the reported methods in condition of dynamic backgrounds, and gradual and sudden illumination changes.

It is important to efficiently segment motion objects from video in computer vision applications. A novel foreground segmentation approach has been developed based on structural similarity background modelling, which responds quickly to sudden illumination changes and dynamic background. Both structural similarity map and environmental variation parameters are taken as a dynamic feedback controller to update the background. A multi‐modal features fusion strategy has been proposed to segment foregrounds in a dynamic cluttered scene without any hypothesis for the scenario content in advance. Experiments for videos with some challenging content have been performed. Comparative study with state‐of‐the‐art methods has indicated the superior performance of the proposed method.

본 논문은 조명의 변화가 심한 영상에서 손 형태를 안정적으로 인지하는 기법에 관한 것이다. 제안한 방법은 HSI 색상공간에서 색상(Hue) 및 색상 기울기(Hue-Gradient)를 기반으로 정의된 배경모델을 구축하고, 실시간으로 입력되는 영상과의 배경차분(background subtraction)기법을 이용하여 배경과 손을 구분한다. 추출된 손의 영역으로부터 18가지의 특징요소를 추출하고 이를 기반으로 다중클래스 SVM(Support Vector Machine) 학습 기법을 사용하여 손의 형태를 인지한다. 제안 기법은 색상 기울기를 배경 차분에 적용함으로써, 조명 환경이 배경 모델의 조명과 다르게 급격한 변화가 이루어졌을 때도 안정적으로 손의 윤곽정보를 추출할 수 있도록 하였다. 또한, 실시간 처리를 저해하는 복잡한 손의 특성정보 대신, OBB의 크기에 대하여 정규화된 두 개의 고유값과 객체 기반 바운딩 박스(OBB)를 구성하는 16개 세부 영역에서의 손 윤곽픽셀의 개수를 손의 특성정보로 사용하였다. 본 논문에서는 급격한 조명 변화 상황에서 기존 RGB 색상요소를 기반으로 하는 배경차분법과 색상을 기반으로 하는 배경차분법, 본 논문에서 제안하는 색상 기울기 기반 배경 차분법의 결과를 비교함으로써 제안 기법의 안정성을 입증하였다. 6명의 실험대상자의 1부터 9까지의 수지화 2700개의 영상으로부터 손 특성 정보를 추출하고 이에 대하여 훈련을 통한 학습 모델을 생성하였다. 학습모델을 기반으로 실험자 6인의 손 형태 1620개의 데이터에 대하여 인지 실험을 실시하여 92.6%에 이르는 손 형태 인식 성공률을 얻었다. In this paper, we present a robust hand recognition approach to sudden illumination changes. The proposed approach constructs a background model with respect to hue and hue gradient in HSI color space and extracts a foreground hand region from an input image using the background subtraction method. Eighteen features are defined for a hand pose and multi-class SVM(Support Vector Machine) approach is applied to learn and classify hand poses based on eighteen features. The proposed approach robustly extracts the contour of a hand with variations in illumination by applying the hue gradient into the background subtraction. A hand pose is defined by two Eigen values which are normalized by the size of OBB(Object-Oriented Bounding Box), and sixteen feature values which represent the number of hand contour points included in each subrange of OBB. We compared the RGB-based background subtraction, hue-based background subtraction and the proposed approach with sudden illumination changes and proved the robustness of the proposed approach. In the experiment, we built a hand pose training model from 2,700 sample hand images of six subjects which represent nine numerical numbers from one to nine. Our implementation result shows 92.6% of successful recognition rate for 1,620 hand images with various lighting condition using the training model.

In order to improve the trajectory control effect of multi‐degree‐of‐freedom industrial robots, this paper combines visual image technology to conduct research on trajectory control of multi‐degree‐of‐freedom industrial robots. Aiming at the problem of video segmentation under sudden illumination changes, this paper uses a Gaussian mixture model based on the global illumination function to adopt a variety of illumination invariant features, and proposes a scene segmentation algorithm suitable for sudden illumination changes. Moreover, this paper compares and verifies the algorithm from the subjective and objective perspectives through experiments, which shows that the algorithm in this paper can segment the scene more accurately even in the environment of sudden changes in illumination. In addition, the results of the accuracy test and the trajectory control test show that the research method of the multi‐degree‐of‐freedom industrial robot trajectory control based on the visual image proposed in this paper can effectively improve the trajectory control effect of the robot.

Foreground segmentation in a video is a key process in many applications such as object detection, an object tracking, and a behavior analysis. Since the extracted foreground objects are often used in the analytical process, the quality of the foreground is a significant factor to the success of these applications. However, there are many key challenges in the foreground segmentation, including dynamic backgrounds, gradual illumination changes, sudden illumination changes, shadows, and long-term scene changes. This paper proposes a novel dynamic codebook method to address such challenges. The dynamic codebook aims to significantly improve the conventional well-known codebook technique by introducing a technique to make a dynamic boundary of each codeword. In this technique, the lab color space is used in order to make the model more resilient to the illumination change. The experimental results and comprehensive comparisons demonstrate that the proposed method can achieve very promising performance.

To solve the problem of indoor surveillance, this paper is concerned about the framework that allows the detection of moving objects based on IP camera. The current background subtraction algorithms cannot correctly classify foreground and background if there are sudden illumination changes. To eliminate the influence of illumination changes, intensity flicker correction technique is employed before background subtraction in this paper. Meanwhile to reduce the false alarms further, the union of two separate detection results is employed. In the experimental results, both global illumination changes and local ones are included. The results show that the improved subtractor proposed in this paper performs better than ViBe both in the aspect of correct detection rate and false alarms.

This paper proposes an efficient real-time background subtraction algorithm, which is essential in many computer vision applications. Initially, histograms of the intensity values for each channel of a pixel position in a set of training framesare constructed. A background model, histogram min-max bucket, is constructed from the minimum and maximum values of contiguous non-zero frequencies of the temporal intensity histogram. A novel feature of this algorithm is the use of a single sliding window to update the system adaptively, capturing the proper background even under sudden and/or gradual illumination changes in the scene. We incorporate both local and global information with the help of the current and the previously visited pixel values for binary classification of a pixel into foreground and background. This algorithm is compared with several state-of-the-art techniques and experimental studies show that the proposed method outperforms all these methods in terms of accurate binary classification.

In the real-time segmentation of the moving objects in a image sequence, methods based on background subtraction are used widely. Bi-polar Radial Reach Correlation (BP-RRC) has realized robust background subtraction by evaluating a local texture per pixel, suppressing the influence of brightness change. However, this method has a fault that is affected by the local illumination change of a background. This paper proposes a robust and stabilized background subtraction algorithm which can cope with texture change of various illumination changes by improving this texture background model. We verified the proposed method using the image sequence including a loose lighting change and a rapid lighting change, and adaptability higher than the conventional method on the sudden illumination changes was attained as a result of verification.

Developmental changes in behavioral and retinomotor responses of Pacific bluefin tuna on exposure to sudden changes in illumination

This study develops a highly reliable real-time visual tracking system capable of tracking a moving target in cluttered environment. The proposed system architecture can be divided into three main parts - the target detector, the target information extractor and the target tracker. The first part detects any moving object in the image, using the moving edges technique. Next, by integrating the information from the target detector and the attractable snake model, the second part automatically extracts the target's contour, illumination and color information to enable the target to be tracked. Finally, by adopting a new hybrid matching method, the third part enables the system to continuously track the target with great robustness and adaptability. Notably, this hybrid matching method incorporates the template matching method with a modified contour matching method. Furthermore, the visual tracking system developed in this study is established on a low-cost commercial camera platform, which can be controlled by a personal computer through an RS232 serial port. Finally, several experiments, including object tracking and human head tracking, are conducted to verify the robustness of the tracking system.

Visual odometry is an essential component in robot navigation and autonomous driving; however, visual sensors are vulnerable in fast motion or sudden illumination changes. This weakness can be compensated with inertial measurement units (IMUs), which maintain the short-term motion when visual sensing becomes unstable and enhance the quality of estimated motion with inertial information. An omnidirectional multi-view visual odometry (ROVO) has recently demonstrated superior performance and stability with the unceasing feature observation of the omnidirectional setup; however, the shortcomings of visual odometry remain. This letter introduced an omnidirectional visual-inertial odometry system (ROVINS) that could seamlessly integrate the inertial information into the omnidirectional visual odometer algorithm: (a) The soft relative pose constraints from the inertial measurement are first added to the pose optimization formulation, which enables blind motion estimation when all visual features are lost; (b) Using the prediction results from the estimated velocity, the visual features in tracking are initialized, resulting in feature tracking that is more robust to visual disturbances. The experimental results showed that the proposed ROVINS algorithm outperforms the vision-only algorithm by a significant margin.

Depth completion involves recovering a dense depth map from a sparse map and an RGB image. Recent approaches focus on utilizing color images as guidance images to recover depth at invalid pixels. However, color images alone are not enough to provide the necessary semantic understanding of the scene. Consequently, the depth completion task suffers from sudden illumination changes in RGB images (e.g., shadows). In this paper, we propose a novel three-branch backbone comprising color-guided, semantic-guided, and depth-guided branches. Specifically, the color-guided branch takes a sparse depth map and RGB image as an input and generates color depth which includes color cues (e.g., object boundaries) of the scene. The predicted dense depth map of color-guided branch along-with semantic image and sparse depth map is passed as input to semantic-guided branch for estimating semantic depth. The depth-guided branch takes sparse, color, and semantic depths to generate the dense depth map. The color depth, semantic depth, and guided depth are adaptively fused to produce the output of our proposed three-branch backbone. In addition, we also propose to apply semantic-aware multi-modal attention-based fusion block (SAMMAFB) to fuse features between all three branches. We further use CSPN++ with Atrous convolutions to refine the dense depth map produced by our three-branch backbone. Extensive experiments show that our model achieves state-of-the-art performance in the KITTI depth completion benchmark at the time of submission.

This study presents a hierarchical background modelling and subtraction approach for real‐time detection of moving objects. At the first level, a novel pixel‐wise background modelling method is proposed for coarse detection. The method can dynamically assign the optimal number of components for each pixel with the borrow–lend strategy. And a flexible learning rate which is variable and different for each component is presented to adapt to scene changes. Additionally, a new mechanism using a framework of finite state machine is introduced to maintain and update the background models. At the second level, in order to deal with sudden illumination changes, a block‐wise foreground validation approach is adopted for refined detection. The authors compare the proposed approach with state‐of‐the‐art methods and experimental results under various scenes demonstrate the robustness and effectiveness of the proposed approach.