Spatial Neighborhood-Constrained Linear Coding for Visual Object Tracking

We present a Free Viewpoint Television (FTV) application with performance improved multi-view video coding (MVC) strategy, which is intended for multimedia access in broadband networks or for future mobile terminals. The significance of our strategy is to realize client-driven FTV playback with reduced decoder output buffer requirements. The advantages of our proposed free view application are improved occlusion detection, decoding efficiency, and enabling smooth switching between different target video sequences. In the proposed coding strategy, the state-of-the-art MPEG4/H.264 decoder is utilized to make the application more applicable for common development. The encoded video bit stream contains several inter-views, which are encoded in an interleaved format based on MPEG4/H.264 encoder. On the decoder side, the encoded multi-views can be recovered with one fold or parallel state-of-the-art MPEG4/H2.64 decoder(s). Based on the client’s choice, one view is shown on the client’s display device. The perspective of view is controlled by the client with guaranteed response time and the view switching happens smoothly. The proposed Buffer Efficient MVC (BE-MVC) strategy does not require powerful decoder for the viewer side to make the application more suited for real-time operation. The comparisons of the required output buffer and decoding performance are demonstrated in this paper.KeywordsFTVMVCBE-MVCmulti-viewfree view

This paper presents a novel visual tracking algorithm based on incremental two-dimensional Maximum Margin Criterion (2DMMC). 2DMMC is a promising discriminant criterion for image feature extraction and its specialities make it a good choice for visual tracking problem. The proposed approach uses the 2DMMC to learn a discriminant projection matrix that best separates the target from the background. The projection matrix is updated online by a incremental algorithm to handle the appearance variations of the target and background. A particle filter using an efficient likelihood function based on the projection matrix is used to predict the target location in each frame. Experiments show that the proposed tracking algorithm is able to track the target in complex scenarios.

Sparse representation has been widely applied for modeling object appearance in the tracking process. Various sparsity models are coded by single feature with fixed local patches, which neglects the potential spatial information inside the appearance model. This way also gives a low robust effect on

The present work investigated the right-left prevalence effect caused by the automatic activation of horizontal and vertical spatial codes in a task (Simon task) in which spatial information is task-irrelevant. Experiment 1 showed a horizontal Simon effect and a vertical Simon effect with a two-dimensional stimulus-response set. In Experiments 2 and 3, the right-left prevalence was obtained in two-dimensional Simon tasks with two contralateral effectors and four effectors respectively. Experiment 4 showed that horizontal coding is based on multiple spatial codes, whereas only one spatial code was formed for vertical coding. On the whole, these results support the notion that the right-left prevalence effect is a general phenomenon affecting spatial coding, and suggest that the horizontal dimension is prevalent because it is based on multiple spatial codes.

In visual tracking, developing a robust appearance model is a challenging task due to variations of object appearances such as background clutter, illumination variation and partial occlusion. In existing tracking algorithms, a target candidate is represented by linear combinations of target templates. However, the relationship between a target candidate and the corresponding target templates is nonlinear because of appearance variations. In this paper, we propose a kernelized convex hull based target representation for visual tracking. Namely, a target is represented by a nonlinear combination of target templates in a mapped higher dimensional feature space. The convex hull model can covers the target appearances that do not appear in the target templates. Experimental results demonstrate the robustness and effectiveness of the proposed tracking algorithm against several state-of-the-art tracking algorithms.

Content-based 3D shape retrieval (CBR) become a central research topic in the field of pattern recognition. The primary challenge for CBR methods is to extract proper features for discriminating diverse shapes of 3D models with non-rigid deformations. The bag-of-features (BoF) paradigm and Local Feature Descriptors (LFDs) have demonstrated impressive levels of performance. The success of this approach prompted several extensions to BoF, such as sparse coding approach. In this study, state-of-the-art content-based 3D shape retrieval methods based on local features and sparse coding are investigated in detail. Performance of the well-known local spectral descriptors (GPS, HKS, SIHKS, WKS and meshSIFT) is tested with several algorithms for dictionaries learning and coefficient learning. Experiments are performed on the standard dataset Shrec11 with different scenarios.

We present a novel color-invariant depth feature descriptor for color-shift-model (CSM)-based image processing. Color images acquired by a single camera equipped with multiple color-filter aperture (MCA) contain depth-dependent color misalignment. The amount and direction of the misalignment provides object's distance from the camera. The CSM-based image processing, which represents the combined image-acquisition and depth-estimation framework, requires a color-invariant feature descriptor that can convey depth information. For improving depth-estimation performance, color boosting is performed on a color image acquired by the MCA camera, and CSM-based channel-shifting descriptor vectors, or channel-shifting vectors (CSVs), are generated by using the feasibility test. Color-invariant features are also extracted in the luminance image. The proposed color-invariant three-dimensional (3-D) feature descriptor is finally obtained by combining the CSVs and luminance features. We present experimental analysis of the proposed feature descriptor and show that the descriptors are proportional to the depth of an object. The proposed descriptor can be used for feature-based image matching in various applications, including 3-D scene modeling, 3-D object recognition, 3-D video tracking, and multifocusing, to name a few.

Saliency detection integrating both background and foreground information

Making full use of temporal and spatial information is critical to cope with the appearance changes of objects in visual object tracking. However, existing methods in the tracking field, which employ a memory network at frame level to learn this information, bring redundancy and cannot build long-term relationships among historical frames due to the limited memory size. In this paper, we propose a novel memory network, Pixel-level Spatio-Temporal Memory (PSTM), which organizes object features in an efficient way to leverage temporal and spatial context information. Specifically, PSTM is constructed and updated by a memory writer, which includes a pixel-level updating strategy to maintain the temporal consistency and dynamically memorize the noteworthy variations. Furthermore, in order to exploit relationships between the object and search region and precisely estimate the state of the object, we propose a memory reader, Pixel-wise Matching and Refinement module (PMR), and model spatial context without a complex manual-designed mechanism. Comprehensive experiments and comparisons on challenging large-scale benchmarks, including GOT-10k, TrackingNet, LaSOT, OTB2015, VOT2020, and NfS, have demonstrated the effectiveness of our proposed method, which performs favorably against state-of-the-art trackers.

<strong class="journal-contentHeaderColor">Abstract.</strong> Hyperspectral images contain abundant spectral and spatial information about the earth's surface, labeling data processing and analysis more difficult, as well as the problem of sample labeling. In this paper, local binary pattern (LBP), sparse representation and mixed logistic regression model are introduced, and a sample labeling method based on neighborhood information and priority classifier discrimination is presented. Then, a hyperspectral remote sensing image classification method based on texture features and semi-supervised learning is implemented. The LBP is employed to extract features of spatial texture information from remote sensing images and enrich the feature information of samples. Then the multivariate logistic regression model is used to select the unlabeled samples with the largest amount of information, and the unlabeled samples with neighborhood information and priority classifier tags are selected to obtain the pseudo-labeled samples after learning. By making full use of the advantages of sparse representation and mixed logistic regression model, a new hyperspectral remote sensing image classification model based on semi-supervised learning is constructed to effectively achieve accurate classification of hyperspectral images. The data of Indian Pines, Salinas scene and Pavia University are selected to verify the validity of the proposed method. The experiment results show that the proposed classification method can obtain higher classification accuracy and show stronger timeliness and generalization ability.

The effect of spatial information characterization on 3D local feature descriptors: A quantitative evaluation

This paper shows that the communication capacity of CDMA systems can be enlarged further, by adopting array antennas, which involves a joint multiuser detection approach that is based on CDMA codes and their spatial information. In contrast to SDMA-CDMA systems the CDMA systems with array antennas in this paper do not have spatial coding, since it causes the frequent spatial handoff problem. Thus the new CDMA systems can be of much higher communication capacity than the CDMA systems which are not sectored. Analyzing the array pattern of array antennas, we show that the interference of other sectors, also multiple access interference (MAI), can be effectively suppressed by the array antennas for a CDMA system to be sectored by array antennas. The design and analysis of WLS multiuser detector in the new CDMA systems is presented to detect CDMA signals with different spatial information. Simulations show that a sectored CDMA system can be of better SINR (signal to inference plus noise ratio) and BER (bit error rate) performance even it has three times the users compared. to the number of cells of present CDMA systems.

Sparse coding based visual tracking: Review and experimental comparison

While many visual tracking methods have been proposed, the challenges such as severe deformation, cluttered background and heavy occlusion are still under well solved. This paper proposed a new visual tracking method by combing the sparse representation and global measure. We flrst construct an appearance model by sparse representation with the patches from target rectangles and their surrounding backgrounds. With this appearance model, we successfully identify the patches of target undergoing heavy occlusion or severe deformation. Then, we deflne a global measure based on the cues of saliency and color contrast to detect the tracking object. This measure performs well in distinguishing object from background especially in cluttered background. Many experiments on difierent challenge image sequences demonstrate that our method is much more robust and stable in tracking.

The main challenging issues in visual tracking can be listed as follows: significant variation of object’s appearance or background image, illumination changes, serious or even complete occlusion of object, etc. In order to deal with them, two modules are needed: one is an accurate appearance model updating online and the other one is a robust matching method to find the target according to the learned model. In this paper, we propose a novel tracking method in a particle filter framework based on IPCA and sparse representation, in which IPCA is used to model the object appearance adaptively and sparse representation is used in two aspects: first, it helps to formulate a robust updating scheme of the IPCA; second, it strengthens the matching process significantly when the tracker copes with very challenging cases as mentioned in the beginning. In experiments, we select three state-of-the-art tracking methods for comparison and demonstrate the superiority of our method over them on various data.