Agnostic attribute segmentation of dynamic scenes with limited spatio-temporal resolution

Nowadays, moving object detection in sequence images has become a hot topic in computer vision research, and has a very wide range of practical applications in many fields of military and daily life. In this paper, fast detection of moving objects in complex background is studied, and fast detection methods for moving objects in static and dynamic scenes are proposed respectively. Firstly, based on image preprocessing, aiming at the difficulty of feature extraction of moving targets in low illumination at night, Gamma change is used to process. Secondly, for the fast detection of moving objects in static scenes, this paper designs a detection method combining background difference and edge frame difference. Finally, aiming at the fast detection of moving objects in dynamic scenes, a feature matching detection method based on the SIFT algorithm is designed in this paper. Simulation experiments show that the method designed in this paper has good detection performance.

Moving objects' detection in dynamic scenes is a very important task in video processing. In the applications of image processing (for example, video-surveillance), more attention is paid to whether there is an interested object in the scene rather than where the object is located. As a matter of fact, similar to static backgrounds, the statistical histograms of the most dynamic backgrounds have favorable stability, and these histograms would change clearly only when big and contrasting objects enter or move out of the scenes. This is a very important and interesting property for dynamic background. We propose a fast and simple algorithm, combining histograms in multiple color spaces and the superposition principle of statistical histogram, called Multiple Color Space Histogram Models (MCSHM). MCSHM first calculates statistical histograms of many color components in multiple color space and then use the changes of statistical histograms to determine whether there is an object rather than the changes in pixels or pixel-level regions. Thus, the computational complexity of MCSHM is kept at a very low level. The basic steps are as follows: firstly, convert each frame from RGB space to other color spaces and calculate the histograms of selected color components, then we can obtain the background histogram model; secondly, detect the objects using statistical histogram superposition principle; finally, update MCSHM by the result of detection. The experimental results demonstrate that our method can quickly and accurately detect moving objects in dynamic scenes.

Simultaneous localization and map building (SLAM) is crucial in autonomous robot navigation. However, existing SLAM systems generally assume a static environment, which makes it difficult to cope with the interference caused by moving objects in dynamic scenes, affecting the system’s localization accuracy and robustness. To address this challenge, this paper proposes YKD-SLAM, a visual SLAM system for indoor dynamic environments, which is based on the ORB-SLAM2 framework and incorporates YOLOv8 target detection, RCF-KMeans (Region-ConstrainedFastK-Means), and epipolar geometric constraints to realize the accurate rejection of dynamic feature points and improve the localization performance in dynamic environments. YKD-SLAM first uses YOLOv8 to detect dynamic objects in the scene, generates a detection frame, optimizes the depth map through open operations, and performs multi-region segmentation of the region within the detection frame by combining RCF-KMeans. Subsequently, through the dynamic feature point rejection strategy based on epipolar geometric constraints, different regions in the detection frame are discriminated into dynamic and static regions, and the feature points in the dynamic region are rejected to improve the localization accuracy and robustness of the system in dynamic environments. The experimental results show that YKD-SLAM performs well in several dynamic scenes in the TUMRGB-D dataset. Compared with ORB-SLAM2, its ATE is reduced by 98.37%; compared with DynaSLAM, the system operation efficiency is improved by 95.35%. In addition, practical experiments conducted in indoor dynamic scenes further validate its potential in real applications.

We present a mapping approach to road scene awareness based on active stereo vision. We generalise traditional static multi-camera rectification techniques to enable active epipolar rectification with a mosaic representation of the output. The approach is used to apply standard static depth mapping and optical flow techniques to the active case. We use the framework to extract the ground plane and segment moving objects in dynamic scenes using arbitrarily moving cameras on a moving vehicle. The approach enables an estimation of the velocity of the vehicle relative to the road, and the velocity of objects in the scene. We provide footage of preliminary results of the system operating in real-time, including dynamic object extraction and tracking, ground plane extraction, and recovery of vehicle velocity.

In this paper, we propose a novel object-level mapping system that can simultaneously segment, track, and reconstruct objects in dynamic scenes. It can further predict and complete their full geometries by conditioning on reconstructions from depth inputs and a category-level shape prior with the aim that completed object geometry leads to better object reconstruction and tracking accuracy. For each incoming RGB-D frame, we perform instance segmentation to detect objects and build data associations between the detection and the existing object maps. A new object map will be created for each unmatched detection. For each matched object, we jointly optimise its pose and latent geometry representations using geometric residual and differential rendering residual towards its shape prior and completed geometry. Our approach shows better tracking and reconstruction performance compared to methods using traditional volumetric mapping or learned shape prior approaches. We evaluate its effectiveness by quantitatively and qualitatively testing it in both synthetic and real-world sequences.

Moving vehicles segmentation based on Bayesian framework for Gaussian motion model

Most existing algorithms solve the visual odometry problem in dynamic scenes by removing dynamic objects. However, some scenarios such as autonomous driving, visual surveillance and virtual reality, require explicit motion information of the objects to help making decisions and understanding scenes. In this paper, we propose DMVO (Dymamic Multi-motion Visual Odometry), a robust and effective visual odometry algorithm that estimate camera pose as well as motion of objects in dynamic scenes using information from stereo cameras. DMVO combines disparity and depth estimation networks to obtain high-precision depth information, which improves the accuracy of the system. Local pose graph and global factor graph of dynamic and static structure are constructed for optimization. The DMVO system is tested on an indoor Oxford multi-motion dataset and an outdoor KITTI tracking dataset. The results show that our system achieves improved performance compared with the state-of-the-art algorithms.

The paper is devoted to the problem of constructing detailed 3D-reconstruction of objects in dynamic scenes when photographing the terrain by underwater autonomous vehicles. An image processing method is being considered that makes it possible to identify dynamic objects and points in a scene, which can be used to position an underwater autonomous vehicle. Identification of points in video stream images and 3D-reconstruction of point data are based on the ideas of the "seed" algorithm. Goal – increasing the accuracy of 3D-reconstruction of dynamic scenes when performing work by autonomous vehicles in previously unknown conditions. Statistical data characterizing the operation of algorithms for identifying objects and their points have been obtained. Conclusions are drawn about the features of working with small-polygonal and multi-polygonal objects. Approaches to determining the location of an underwater autonomous vehicle by analyzing point data are proposed. The use of image-based positioning systems is relevant in various applications, from unmanned vehicles and robot couriers to autonomous research vehicles. Decisions in this area may affect, for example, the effectiveness of search and rescue or repair work in difficult-to-reach conditions.

Observers’ memory for a person’s appearance can be compromised by the presence of a weapon, a phenomenon known as the weapon-focus effect (WFE). According to the unusual-item hypothesis, attention shifts from the perpetrator to the weapon because a weapon is an unusual object in many contexts. To test this assumption, we monitored participants’ eye movements while they watched a mock-crime video. The video was presented with sound and featured a female perpetrator holding either a weapon, a non-threatening unusual object, or a neutral object. Contrary to the predictions of current theories, there were no significant differences in total viewing times for the three objects. For the perpetrator, total viewing time was reduced when she held the non-threatening unusual object, but not when she held the weapon. However, weapon presence led to an attentional shift from the perpetrator’s face toward her body. Detailed time-course analyses revealed that the effects of object type were more pronounced during early scene viewing. Thus, our results do not support the idea of extended attentional shifts from the perpetrator toward the unusual objects, but instead suggest more complex attentional effects. Contrary to previous research, memory for the perpetrator’s appearance was not affected by object type. Thus, there was no WFE. An additional online experiment using the same videos and methodology produced a WFE, but this effect disappeared when the videos were presented without sound.

Visual simultaneous localization and mapping (VSLAM) is extensively utilized in the field of unmanned driving. Classical VSLAM typically postulates that the surroundings are stationary, and dynamic objects can seriously affect the accuracy of the system. However, detecting and tracking dynamic objects is beneficial for improving the autonomy and intelligence of robotics. In order to solve the problem of VSLAM distinguishing dynamic objects in dynamic scenes, a semantic VSLAM system based on the VDO-SLAM framework is proposed. The approach integrates semantic segmentation into the visual odometry and combines scene flow and geometric constraints to effectively reduce the negative impact of dynamic objects on system performance. Specifically, a transformer-based neural network architecture for optical flow estimation was employed, which effectively enhances the segmentation precision of foreground and background regions. And a geometric constraint was introduced to classify dynamic objects, which can reduction of erroneous segmentation of dynamic objects with effect. The data analysis results on the KITTI dataset demonstrate that our proposed algorithms can effectively enhance precision and stability of VSLAM system, reduce the tracking error of VSLAM in dynamic environments, and demonstrate good potential and application value.

A consensus-based method for tracking: Modelling background scenario and foreground appearance

This paper proposes an efficient ray-tracing scheme to render dynamic scenes by exploring the coherence between consecutive frames. By assuming that only few objects in a scene may move from frame to frame, the rays emitted from the same pixels of two consecutive frames may hit the same object with high possibility. By using extra buffers to store the index of triangle each ray intersects for the current scene, the trace of each ray for the following scene can start by testing if this ray intersects the same triangle. Once the ray hits the same triangle, the intersection depth can then be used to exclude the test operation of the other nodes during the tree traversal. In addition, instead of rebuilding the bounding volume hierarchies (BVH) tree structure of the next scene, it can also be obtained by updating the current one. The unmodified nodes can be annotated such that their intersection test can be simply skipped during the tree traversal for those rays which hit the same primitives in the current scene. For our own test scenes whose total number of modified triangles is less than 5% of the overall triangle count, our experimental results show that more than one half of ray-node and ray-triangle intersection operations can be saved, which can result in more than 50% reduction in the software ray-tracing time. For the common test scene TOASTER where more than 90% of triangles have been moved in consecutive scenes, the proposed scheme can still achieve the saving of trace time up to 24%.

In this paper, we propose a multi-camera video stitching approach, which conducts borderline adjusting based on foreground information. In real time applications of video stitching, one problem is to deal with dynamic scenes, where foreground objects often cause broken objects like artifacts in panoramic video. To address this problem, we propose a foreground based borderline adjusting method to achieve smooth video stitching. In this method, foreground objects are extracted from different viewpoints, and the stitching borderlines are adjusted according to the foreground content in dynamic scenes. Guided by the adjusted borderlines, foreground objects are smoothly synthesized into panoramic video. Experiment results show that this approach obtains more than 81% correct rate for dynamic scenes, compared with 25%~50% correct rate on the condition of not utilizing foreground information, and achieves processing speed of 10~13 frames per second.

This Perspective aims to inform the heterogeneous catalysis and materials science community about the recent advances in Time-of-Flight-Secondary Ion Mass Spectrometry (ToF-SIMS) to characterize catalytic solids by taking large model H-ZSM-5 zeolite crystals as a showcase system. SIMS-based techniques have been explored in the 1980–1990's to study porous catalyst materials but, due to their limited spectral and spatiotemporal resolution, there was no real major breakthrough at that time. The technical advancements in SIMS instruments, namely improved ion gun design and new mass analyser concepts, nowadays allow for a much more detailed analysis of surface species relevant to catalytic action. Imaging with high mass and lateral resolution, determination of fragment ion patterns, novel sputter ion concepts as well as new mass analysers (e.g. ToF, FTICR) are just a few novelties, which will lead to new fundamental insight from SIMS analysis of heterogeneous catalysts. The Perspective article ends with an outlook on instrumental innovations and their potential use for catalytic systems other than zeolite crystals.

Centre of mass model – A novel approach to background modelling for segmentation of moving objects