Illumination robust template matching based on nearest neighbor

Video object segmentation and tracking are two essential building blocks of smart surveillance systems. However, there are several issues that need to be resolved. Threshold decision is a difficult problem for video object segmentation with a multi-background model. In addition, some conditions make robust video object tracking difficult. These conditions include nonrigid object motion, target appearance variations due to changes in illumination, and background clutter. In this paper, a video object segmentation and tracking framework is proposed for smart cameras in visual surveillance networks with two major contributions. First, we propose a robust threshold decision algorithm for video object segmentation with a multi-background model. Second, we propose a video object tracking framework based on a particle filter with the likelihood function composed of diffusion distance for measuring color histogram similarity and motion clue from video object segmentation. The proposed framework can track nonrigid moving objects under drastic changes in illumination and background clutter. Experimental results show that the presented algorithms perform well for several challenging sequences, and our proposed methods are effective for the aforementioned issues.

Visual tracking is a challenging issue in surveillance, human–computer Interaction, and intelligent robotics, among others. Managing appearance changes of the target object, illumination changes, rotations, non-rigid deformations, partial or full occlusions, background clutter, fast motion, and so forth is generally difficult. Among the numerous existing trackers, the correlation-filter-based tracker can achieve appealing performance with a fast speed for fast Fourier transform. Motivated by this property, the spatio-temporal context (STC) learning algorithm was proposed with the consideration of the information from the context around the target, and this algorithm achieved good results. However, STC only utilizes the overall intensity information. In this paper, we propose a multi-channel features STC learning algorithm with an improved scale-adaptive scheme. Our algorithm integrates powerful features, including Histogram of Oriented Gradients and color naming, using kernel methods on the basis of the STC algorithm to further enhance the overall tracking performance. Extensive experimental results obtained from various benchmark data sets demonstrate that the proposed tracker is promising for various challenging scenarios and maintains real-time performance at an average speed of 78 frames/s. According to the test results, our algorithm outperforms the STC algorithm and achieves performance that is competitive with the state-of-the-art algorithms.

Our goal is to establish a simple technique for dynamic human body tracking based on image sequence. In this paper, we present a novel method of template matching and predictive tracking for moving body in clutter background. The method exploits the fact that moving human body changed in approximate rigid parts. First, we use the moving difference to find the region of interest, then we make a template of a target through the morphology algorithm using the skeleton of the object, and by revising the mask combined Kalman predictive tracking method to archive real-time tracking. It has been proved that in clutter background and noisy situation this algorithm is still efficient.

This paper presents an eye gaze estimation system which robust against various users. Our method utilizes an IR camera mounted on glass to allow user’s movement. Pupil knowledge such as shape, size, location, and motion are used. This knowledge works based on the knowledge priority. Pupil appearance such as size, color, and shape are used as the first priority. When this step fails, then pupil is estimated based on its location as second priority. When all steps fail, then we estimate pupil based on its motion as the last priority. The aim of this proposed method is to make the system compatible for various user as well as to overcome problem associated with illumination changes and user movement. The proposed system is tested using several users with various race as well as nationality and the experiment result are compared to the well-known adaptive threshold method and template matching method. The proposed method shows good performance, robustness, accuracy and stability against illumination changes without any prior calibration.KeywordsGazeeye detectionpupilpupil knowledge

Abstract. In this paper we present an approach for detailed and precise automatic dense 3D reconstruction using images from consumer cameras. The major difference between our approach and many others is that we focus on wide-baseline image sets. We have combined and improved several methods, particularly, least squares matching, RANSAC, scale-space maxima and bundle adjustment, for robust matching and parameter estimation. Point correspondences and the five-point algorithm lead to relative orientation. Due to our robust matching method it is possible to orient images under much more unfavorable conditions, for instance concerning illumination changes or scale differences, than for often used operators such as SIFT. For dense reconstruction, we use our orientation as input for Semiglobal Matching (SGM) resulting into dense depth images. The latter can be fused into a 2.5D model for eliminating the redundancy of the highly overlapping depth images. However, some applications require full 3D models. A solution to this problem is part of our current work, for which preliminary results are presented in this paper. With very small unmanned aerial systems (Micro UAS) it is possible to acquire images which have a perspective similar to terrestrial images and can thus be combined with them. Such a combination is useful for an almost complete 3D reconstruction of urban scenes. We have applied our approach to several hundred aerial and terrestrial images and have generated detailed 2.5D and 3D models of urban areas.

Face tracking in realistic environments is a difficult problem due to pose variations, occlusions of objects, illumination changes and cluttered background, among others. The paper presents a robust and real-time face tracking algorithm. A novel likelihood is developed based on a boosted multi-view face detector to characterize the structure information. The likelihood function is further integrated with particle filter which can maintain multiple hypotheses. The algorithm proposed is able to track faces in different poses, and is robust to temporary occlusions, illumination changes and complex background. In addition, it enjoys a real-time implementation. Experiments with a challenging image sequence shows the effectiveness of the algorithm.KeywordsParticle FilterIllumination ChangeWeak LearnerCluttered BackgroundTemporary OcclusionThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Hand posture recognition has tremendous potential in the field of natural user interactions. There were many advances in research in recent years but there are still limitations regarding its usage in unfavorable live situations where hand posture variation, illumination change or background complexity are an issue. In cases like these, recognizing the hand posture is a difficult task. As such, we considered reducing the difficulty of the task by using curriculum learning with intermediate information. We proceeded to divide the complex architecture of the hand posture recognition task into two easier ones: 1) Extraction of the hand shape under clutter background with illumination change, 2) Recognition of the hand posture from a binary image. In order to do so, we propose here a bottom-up structured deep convolutional neural network incorporating a special layer for binary image extraction. Our proposed method also employs state-of-the art techniques for deep learning to obtain generalization. As a result, we achieved better recognition performances of the hand posture under clutter background compared to the baseline method.

Among the large number of cohort studies that employ propensity score matching, most match patients 1:1. Increasing the matching ratio is thought to improve precision but may come with a trade-off with respect to bias. To evaluate several methods of propensity score matching in cohort studies through simulation and empirical analyses. We simulated cohorts of 20,000 patients with exposure prevalence of 10%-50%. We simulated five dichotomous and five continuous confounders. We estimated propensity scores and matched using digit-based greedy ("greedy"), pairwise nearest neighbor within a caliper ("nearest neighbor"), and a nearest neighbor approach that sought to balance the scores of the comparison patient above and below that of the treated patient ("balanced nearest neighbor"). We matched at both fixed and variable matching ratios and also evaluated sequential and parallel schemes for the order of formation of 1:n match groups. We then applied this same approach to two cohorts of patients drawn from administrative claims data. Increasing the match ratio beyond 1:1 generally resulted in somewhat higher bias. It also resulted in lower variance with variable ratio matching but higher variance with fixed. The parallel approach generally resulted in higher mean squared error but lower bias than the sequential approach. Variable ratio, parallel, balanced nearest neighbor matching generally yielded the lowest bias and mean squared error. 1:n matching can be used to increase precision in cohort studies. We recommend a variable ratio, parallel, balanced 1:n, nearest neighbor approach that increases precision over 1:1 matching at a small cost in bias.

Key point correspondence plays a critical role in large distance lunar rover localization and navigation, which can decrease accumulative errors and facilitate the rover to approach probing target accurately. Lunar surface images often contain similar patterns, noisy points and obvious illumination changes, especially along with large distance movement, overlapping regions of image pairs have large deformations and sorely different scales. Traditional appearance based matching methods fail in such conditions as local appearance features become less distinctive. In this paper, we proposed a novel hierarchical graph matching strategy to fully utilize structure property of key points. For vertexes in bigger scale, their neighborhood graphs in smaller scale are regarded as vertex labels and the differences of neighborhood graphs are regarded as differences of vertexes. Based on the new strategy, both global and local constraint are considered such that the dependency on local appearance is alleviated. Furthermore, an active key point detection method is proposed to serve the graph matching method for better performance. Finally, several experiments are conducted on lunar surface images collected by Chang'E-3 rover, which demonstrates the robustness and effectiveness of the proposed method.

We introduce a system that processes a sequence of images of a front-facing human face and recognises a set of facial expressions. We use an efficient appearance-based face tracker to locate the face in the image sequence and estimate the deformation of its non-rigid components. The tracker works in real time. It is robust to strong illumination changes and factors out changes in appearance caused by illumination from changes due to face deformation. We adopt a model-based approach for facial expression recognition. In our model, an image of a face is represented by a point in a deformation space. The variability of the classes of images associated with facial expressions is represented by a set of samples which model a low-dimensional manifold in the space of deformations. We introduce a probabilistic procedure based on a nearest-neighbour approach to combine the information provided by the incoming image sequence with the prior information stored in the expression manifold to compute a posterior probability associated with a facial expression. In the experiments conducted we show that this system is able to work in an unconstrained environment with strong changes in illumination and face location. It achieves an 89% recognition rate in a set of 333 sequences from the Cohn–Kanade database.

This paper demonstrates how the nonlocal principle benefits video matting via the KNN Laplacian, which comes with a straightforward implementation using motion-aware K nearest neighbors. In hindsight, the fundamental problem to solve in video matting is to produce spatio-temporally coherent clusters of moving foreground pixels. When used as described, the motion-aware KNN Laplacian is effective in addressing this fundamental problem, as demonstrated by sparse user markups typically on only one frame in a variety of challenging examples featuring ambiguous foreground and background colors, changing topologies with disocclusion, significant illumination changes, fast motion, and motion blur. When working with existing Laplacian-based systems, we expect our Laplacian can benefit them immediately with an improved clustering of moving foreground pixels.

In this paper we present a novel and robust fusion algorithm for the infrared and visible Power-Equipment image, which is invariant to large-scale changes and illumination changes in the real operating environment of Power Equipments. Firstly, the Scale-Invariant Feature Transform (SIFT) algorithm is used to extract and describe the feature points from the infrared and visible images. Secondly, the best feature matching for each feature point of visible images is found by identifying its nearest neighbor in the database of feature points from infrared images. Thirdly, the Random Sample Consensus (RANSAC) technology is chose to select the appropriate geometric transformation model and estimate the transformation parameters of this geometric model. Finally, the bicubic interpolation method is employed to implement grayscale interpolation and coordinate transformation, then obtain the fusion image of visible and infrared images. Extensive experimental results on the Power-Equipment dataset demonstrate that our method has high stability and excellent performance.

Automatic detection and locating of objects such as poles, traffic signs, and building corners in street scenes captured from a mobile mapping system has many applications. Template matching is a technique that could automatically recognise the counterparts or correspondents of an object from multi-view images. In this study, we aim at finding correspondents of an object from wide baseline panoramic images with large geometric deformations from sphere projection and significant systematic errors from multi-camera rig geometry. Firstly, we deduce the camera model and epipolar model of a multi-camera rig system. Then, epipolar errors are analysed to determine the search area for pixelwise matching. A low-cost laser scanner is optionally used to constrain the depth of an object. Lastly, several classic feature descriptors are introduced to template matching and evaluated on the multi-view panoramic image dataset. We propose a template matching method combining a fast variation of a scale-invariant feature transform (SIFT) descriptor. Our method experimentally achieved the best performance in terms of accuracy and efficiency comparing to other feature descriptors and the most recent robust template matching methods.

Athlete identification is important for sp ort video content analysis since users often care about the video clips with their preferred athletes. In this paper, we propose a method for athlete id entification by combing th e segmentation, tracking and recognition procedures in to a coarse-to-fine scheme for jersey number (digital characters on sport shirt) detection. Firstly, image segmentation is employed to separate the jersey number regions with its background. And size/pipe-like attributes of digital characters are used to filter out candidates. Then, a K -NN ( K nearest neighbor) classifier is employed to classify a candidate into a digit in “0-9” or negative. In the recognition procedure, we use the Zernike moment features, which are invariant to ro tation and scale for digital shape recognition. Synth etic training samples with different fonts are used to represent the pattern of digital characters with non-rigid deformation. Once a character candidate is detected, a SSD (smallest square distance)-based tracking procedure is started. The recognition pro cedure is performed every several frames in the tracking process. After tracking tens of frames, the overall recognition results are co mbined to determine if a candidate is a true jersey number or not by a voting procedur e. Experiments on several types of sports video shows encouraging result. Keywords: Jersey number, sports vi deo, character recognition.

Facial expression recognition (FER) in uncontrolled, in-the-wild settings is a challenging task due to variations in pose, illumination, occlusion, and background clutter. The state-of-the-art methods on FER employ fixed size pooling in attention networks, resulting in suboptimal robustness in unconstrained environments. In this work, a novel Multi-Scale Attention Network (MSAN) is integrated with the YOLOv8 backbone architecture for expression classification. The YOLOv8 backbone provides efficient feature extraction with a lightweight, detection-oriented architecture, enabling real-time processing and strong spatial feature representation. The Convolutional Block Attention Mechanism (CBAM) is employed in the proposed work. The standard convolutional blocks in YOLOv8 backbone are replaced with MSAN, wherein the fixed-size pooling in the spatial attention mechanism of CBAM is replaced with a Pyramid Pooling Mechanism (PPM). The PPM employs adaptive average pooling at multiple resolutions to capture multi-scale contextual information, which is subsequently fused into a unified feature representation. This enables the spatial attention mechanism in MSAN to capture both global facial context and local region-level cues simultaneously, leading to superior robustness to scale variations, partial occlusions, and diverse expression variations. The work is evaluated on benchmark wild datasets including RAF-DB, FER2013, and AffectNet, with all experiments performed using 5-fold cross-validation. Experimental results demonstrate that the multi-scale attention network with PPM in YOLOv8 backbone achieved improvement in classification accuracy and robustness when compared with the state-of-the-art methods for expression classification in the wild. These findings highlight the effectiveness of integrating MSAN into lightweight detection-based backbones for real-world FER applications.