An adaptive wavelet shrinkage based accumulative frame differencing model for motion segmentation

Model fitting is a fundamental component in computer vision for salient data selection, feature extraction and data parameterization. Conventional approaches such as the RANSAC family show limitations when dealing with data containing multiple models, high percentage of outliers or sample selection bias, commonly encountered in computer vision applications. In this paper, we present a novel model evaluation function based on Gaussian-weighted Jensen–Shannon divergence, and integrate into a particle swarm optimization (PSO) framework using ring topology. We avoid two problems from which most regression algorithms suffer, namely the requirements to specify inlier noise scale and the number of models. The novel evaluation method is generic and does not require any estimation of inlier noise. The continuous and meta-heuristic exploration facilitates estimation of each individual model while delivering the number of models automatically. Tests on datasets comprised of inlier noise and a large percentage of outliers (more than 90 % of the data) demonstrate that the proposed framework can efficiently estimate multiple models without prior information. Superior performance in terms of processing time and robustness to inlier noise is also demonstrated with respect to state of the art methods.

Pedestrian trajectory prediction is a crucial component in computer vision and robotics, but remains challenging due to the domain shift problem. Previous studies have tried to tackle this problem by leveraging a portion of trajectory data from the target domain to fine-tune the model. However, such domain adaptation methods are impractical in real-world scenarios, as it is infeasible to collect trajectory data from all potential target domains. In this paper, we study a new task named generalized pedestrian trajectory prediction, with the aim of generalizing the model to unseen domains without accessing their trajectories. To tackle this task, we further introduce a Recurrent Aligned Network (RAN) to minimize the domain gap through domain alignment. Specifically, we devise a recurrent alignment module to effectively align the trajectory feature spaces at both time-state and time-sequence levels by the recurrent alignment strategy. Furthermore, we introduce a pre-aligned representation module to combine social interactions with the recurrent alignment strategy, which aims to consider social interactions during the alignment process instead of just target trajectories. We extensively evaluate our method and compare it with state-of-the-art methods on three widely used benchmarks. The experimental results demonstrate the superior generalization capability of our method. Our work not only fills the gap in the generalization setting for practical pedestrian trajectory prediction, but also sets strong baselines in this field.

Summary. Pattern recognition has as its objective to classify objects into different categories and classes. It is a fundamental component of artificial intelligence and computer vision. This chapter investigates the application of an efficient optimization method, known as Particle Swarm Optimization (PSO), to the field of pattern recognition and image processing. First a clustering method that is based on PSO is discussed. The application of the proposed clustering algorithm to the problem of unsupervised classification and segmentation of images is investigated. Then PSObased approaches that tackle the color image quantization and spectral unmixing problems are discussed.

A novel approach to motion segmentation based on gamma distribution

Most approaches to the visual perception of humans do not include high-level activity recognitition. This paper presents a system that fuses and interprets the outputs of several computer vision components as well as speech recognition to obtain a high-level understanding of the perceived scene. Our laboratory for investigating new ways of human-machine interaction and teamwork support, is equipped with an assemblage of cameras, some close-talking microphones, and a videowall as main interaction device. Here, we develop state of the art real-time computer vision systems to track and identify users, and estimate their visual focus of attention and gesture activity. We also monitor the users’ speech activity in real time. This paper explains our approach to high-level activity recognition based on these perceptual components and a temporal logic engine.Keywordsactivity recognitioncomputer visiontemporal logic

Lipschitz energy functional for anisotropic diffusion applications

Realistic 3D human model reconstruction is an important component in computer graphics and computer vision. In particular, texturing on the surface of models is a key stage of reconstruction. In this paper, we dispose the texture mapping on the model’s surface as an optimization of image stitching, and present an effective method to generate a seamless, integrated and smooth texture on the surface of 3D human model. First, we build a corresponding Markov Random Field model with respect to color images and triangular meshes of the surface. On the basis of alpha-expansion optimization for this Markov Random Field model, a 2D translation coordinate of color image, as an adaptive iterative factor, is introduced into the optimization to match the color content at the boundary of adjacent meshes. That compensates for the misalignment of adjacent color images, which caused by the inaccuracy of depth data and multi-view misregistration. Then we apply Poisson blending to a composite vector field in gradient domain, to resolve the small but noticeable illumination variations between different color images. To repair the blank regions, we parameterize the model’s surface and project it onto a 2D plane. Then the K-Nearest Neighbor algorithm is applied to fill up the blank regions with texture contents. Finally, we evaluate our method by comparison with another three advanced methods on some human models, and the results demonstrate that our method of images stitching creates a best texture on the surface of 3D human model both in visual effect and quantitative analysis.

Background modeling and subtraction is a basic component of many computer vision and video analysis applications. It has a critical impact on the performance of object tracking and activity analysis. In this paper, we propose an effective and adaptive background modeling and subtraction approach that can deal with dynamic scenes. The proposed approach uses a subspace learning method to model the background and the subspace is updated on-line with a sequential Karhunen-Loeve algorithm. A linear prediction model is also used to make the detection more robust. Experimental results show that the proposed approach is able to model the background and detect moving objects under various type of background scenarios with close to real-time performance.

Video summarization is an essential component of multimedia processing and computer vision, designed to condense extensive video content while maintaining its key contextual elements. The rapid expansion of video data across domains such as security surveillance, online media, and digital archiving necessitates the development of robust and efficient summa- rization techniques. This paper introduces LTC-SUM, a novel client-driven framework that leverages 2D Convolutional Neural Networks (CNNs) to generate adaptive, lightweight video sum- maries. Unlike traditional methods that rely on computationally intensive processes or simplistic heuristic techniques, LTC-SUM efficiently extracts significant frames while minimizing redundant information. The system is optimized for real-time applications and ensures low computational overhead. Extensive evaluations demonstrate that LTC-SUM surpasses conventional techniques in performance and accuracy, effectively balancing efficiency and contextual retention. Moreover, the framework’s adaptability extends its applicability to various fields, including surveillance monitoring, educational content processing, and automated me- dia analysis.

Recent Studies show that people respond to music, and this music and this music has a strong effect on human brain activity. The average urban human listens up to four hours of music every day. People listen to music based upon there mood and interests. This project focuses on creating an app to play users' songs based on their mood by capturing a facial expression. Computer visualization is an interdisciplinary field that helps to convey a high-level understanding of digital images or video to computers.. In this system, computer vision components are used to determine the user’s emotion by facial expression. When emotions are detected, the system plays a song of that emotion, saving a lot of user time by selecting and playing the songs themselves. Sentiment based music players also follow the details of the song and have genres like linear mode and random mode.

Most of parametric motion segmentation methods, formulated based on RANSAC technique, are designed to estimate and segment multiple motions in a sequential manner. This paper introduces a new random set theoretical approach to simultaneously estimate the parameters of, and segment multiple motions in a single run. In this approach, the parameters of multiple motions are modelled as a random finite set with multi-Bernoulli distribution. Simulation results involving segmentation of numerous motions show that our method outperforms state-of-art methods in terms of estimation error and correct estimation rate. In addition, it is highly parallelizable and well-suited for implementation by parallel processors. The fast convergence and highly parallelizable nature of the proposed approach make it an excellent choice for real-time estimation and segmentation of multiple motions in computer vision and robotic applications.

Motion segmentation is an important task in computer vision and several practical approaches have already been developed. A common approach to motion segmentation is to use the optical flow and formulate the segmentation problem using a linear approximation of the brightness constancy constraints. Although there are numerous solutions to solve this problem and their accuracies and reliabilities have been studied, the exact definition of the segmentation problem, its theoretical feasibility and the conditions for successful motion segmentation are yet to be derived. This paper presents a simplified theoretical framework for the prediction of feasibility, of segmentation of a two-dimensional linear equation system. A statistical definition of a separable motion (structure) is presented and a relatively straightforward criterion for predicting the separability of two different motions in this framework is derived. The applicability of the proposed criterion for prediction of the existence of multiple motions in practice is examined using both synthetic and real image sequences. The prescribed separability criterion is useful in designing computer vision applications as it is solely based on the amount of relative motion and the scale of measurement noise.

Motion segmentation is an important topic in computer vision. In this paper, we study the problem of multi-body motion segmentation under the affine camera model. We use a mixture of subspace model to describe the multi-body motions. Then the motion segmentation problem is formulated as an MAP estimation problem with model complexity penalty. With several candidate motion models, the problem can be naturally converted into a linear programming problem, which guarantees a global optimality. The main advantages of our algorithm include: It needs no priori on the number of motions and it has comparable high segmentation accuracy with the best of motion-number-known algorithms. Experiments on benchmark data sets illustrate these points.

Multi-view 3D reconstruction, namely, structure-from-motion followed by multi-view stereo, is a fundamental component of 3D computer vision. In general, multi-view 3D reconstruction suffers from an unknown scale ambiguity unless a reference object of known size is present in the scene. In this article, we show that multi-view images captured using a dual-pixel (DP) sensor can automatically resolve the scale ambiguity, without requiring a reference object or prior calibration. Specifically, the defocus blur observed in DP images provides sufficient information to determine the absolute scale when paired with depth maps (up to scale) recovered from multi-view 3D reconstruction. Based on this observation, we develop a simple yet effective linear method to estimate the absolute scale, followed by the intensity-based optimization stage that aligns the left and right DP images by shifting them back toward each other using cross-view blur kernels. Experiments demonstrate the effectiveness of the proposed approach across diverse scenes captured with different cameras and lenses. Code and data are available at https://github.com/lilika-makabe/dp-sfm-tpami.git.

Attention links sensing to recognition