Real-time depth camera tracking with geometrically stable weight algorithm

The KinectFusion algorithm is now used routinely to reconstruct dense 3D surfaces at real-time frame rates using a commodity depth camera. To achieve robust pose estimation, the method conducts the frame-to-model tracking during camera tracking that must inevitably accompany the memory-bound, GPU-assisted volumetric computations for the model manipulation, to which mobile processors are often more vulnerable than PC-based processors. In this paper, we present an effective camera-tracking method that is based on the computationally lighter frame-to-frame tracking method. This method's tendency toward rapid accumulation of pose estimation errors is suppressed effectively via a predictor—corrector technique. By removing the costly volumetric computations from the pose estimation process, our camera tracking system becomes more efficient in terms of both time and space complexity, offering a compact implementation of depth sensor-based camera tracking on low-end platforms such as mobile devices in addition to high-end PCs.

The increase in popularity of fitness and at-home fitness has illuminated a major deficiency with traditional fitness tracking; the absence of immediate motion analysis and corrective feedback. Many conventional methods of instruction, as well as wearables, lack the system feedback of prompting users for proper movements, which can lead to many real workout injuries. The proposed system is the "Real-Time Exercise Tracking System". The proposed project is an intelligent method of intervention that uses computer vision and deep learning to provide a solution to the problem of lacking immediate feedback. The objective of the system is to accurately assess the form a user is exercising in, and properly provide the user with effective feedback on their form in real-time. The system utilizes a combination of MediaPipe for estimating accurate user pose, and a machine learning model to assesses the required movements of the user. The application provides the user with real-time voice and visual feedback on motion awareness. The application will assess the user's exercise awareness in terms of AI-based form correction, real-time performance analytics, propose new workouts based on previous fitness activities, and mitigate risk of injuries with alerts. The project will contribute to filling the gap between unmonitored home activity and professional fitness education, thus creating a safer, more effective, and happy fitness experience for users. Keywords – Real-Time Tracking, Pose Estimation, Computer Vision, MediaPipe, Fitness Technology, Deep Learning, Form Correction

Real-time camera tracking for marker-less and unprepared augmented reality environments

Designing a real-time visual tracking system to catch goldfishes is a very complex task because a fish’s precise relative 3D positions inside a water tank are difficult to determine. In this paper, we present a new visual servo control system using stereo vision to acquire the 3D positions of goldfishes. The continuously adaptive mean shift (CAMSHIFT) algorithm and optical flow vision tracking methods are applied to provide feedback of a tracked fish’s real-time position and motion trajectory in a water tank at a high frame-rate. Two low-cost, well calibrated cameras are used to track the goldfishes and a 5-axis robot manipulator is used to catch a specified goldfish. The designed visual tracking and servo system is less sensitive to lighting influences and thus performs more efficiently. Experiments of the proposed method yielded very good results as the real-time 3D vision successfully tracked two fishes and guided the manipulator to catching one of them in a net attached to its end-effector.

The ability to quickly acquire 3D models is an essential capability needed in many disciplines including robotics, computer vision, geodesy, and architecture. In this paper we present a novel method for real-time camera tracking and 3D reconstruction of static indoor environments using an RGB-D sensor. We show that by representing the geometry with a signed distance function (SDF), the camera pose can be efficiently estimated by directly minimizing the error of the depth images on the SDF. As the SDF contains the distances to the surface for each voxel, the pose optimization can be carried out extremely fast. By iteratively estimating the camera poses and integrating the RGB-D data in the voxel grid, a detailed reconstruction of an indoor environment can be achieved. We present reconstructions of several rooms using a hand-held sensor and from onboard an autonomous quadrocopter. Our extensive evaluation on publicly available benchmark data shows that our approach is more accurate and robust than the iterated closest point algorithm (ICP) used by KinectFusion, and yields often a comparable accuracy at much higher speed to feature-based bundle adjustment methods such as RGB-D SLAM for up to medium-sized scenes.

The Microsoft HoloLens is a head-worn mobile augmented reality device that is capable of mapping its direct environment in real-time as triangle meshes and localize itself within these three-dimensional meshes simultaneously. The device is equipped with a variety of sensors including four tracking cameras and a time-of-flight (ToF) range camera. Sensor images and their poses estimated by the built-in tracking system can be accessed by the user. This makes the HoloLens potentially interesting as an indoor mapping device. In this paper, we introduce the different sensors of the device and evaluate the complete system in respect of the task of mapping indoor environments. The overall quality of such a system depends mainly on the quality of the depth sensor together with its associated pose derived from the tracking system. For this purpose, we first evaluate the performance of the HoloLens depth sensor and its tracking system separately. Finally, we evaluate the overall system regarding its capability for mapping multi-room environments.

Calibration continues to receive significant attention in robotics because of its key impact on performance and cost associated with the operation of complex robots. Calibration of kinematic parameters is typically the first mandatory step. To this end, a variety of metrology systems and corresponding algorithms have been described in the literature relying on measurements of the pose of the end-effector using a camera or laser tracking system, or, exploiting constraints arising from contacts of the end-effector with the environment. In this work, we take inspiration from the behavior of infants and certain animals, who are believed to use self-stimulation or self-touch to “calibrate” their body representations, and present a new solution to this problem by letting the robot close the kinematic chain by touching its own body. The robot considered in this paper is sensorized with tactile arrays for a total of about 4200 sensing points. The correspondence between the predicted contact point from existing forward kinematics and the actual position on the robot’s ‘skin’ provides sample data that allows refining the kinematic representation (DH parameters). The data collection procedure is automated—self-touch is autonomously executed by the robot—and can be repeated at any time, providing a compact self-calibration system that does not require an external measurement apparatus.

As a damage element, high-speed fragments have a significant effect on the ammunition safety. The impact from the fragments are also one of the basic problems of ammunition safety tests. To clarify the reaction characteristics of combustion, explosion, detonation, and so on, when hypersonic fragments hit insensitive munitions, it is necessary to carry out corresponding research on the deceleration law of hypersonic fragment in the air. In this paper, a 30 mm caliber gun with large chamber, small caliber, and large aspect ratio is proposed to drive high-speed fragments. According to STANAG 4496 standard, a near-cylinder steel fragment with Brinell hardness HB ≤ 270 and mass of 18.6 g was designed. The test system was composed of zone interception velocity measurement, chamber pressure sensor, trajectory tracking system, high-speed camera, and other equipment were also established to obtain the pressure variations in the chamber, the velocity of the fragment, and its flight orientation. From the video taken by the high-speed camera and trajectory tracking system, the fragment and the projectile sabot achieve effective separation after the fragment travels out of the muzzle. As time goes on, the distance between the fragment and the projectile sabot gradually increases. The fragment is always in the front of the sabot and steadily flies to the target. The muzzle velocity of the fragment is controlled by adjusting the propellant charge, and the flight velocity in the air is measured by the zone interception velocity measuring device in the range of 5 Ma to 7 Ma. The theoretical models of fragment deceleration and the models of flight orientation are also established according to the experimental data. On this basis, F test and least square nonlinear regression fitting were used to analyze experimental data. Finally, the deceleration coefficient of quasi-cylindrical fragments between 5 Ma and 7 Ma stipulated in STANAG 4496 standard is 0.009312, and the average drag coefficient in air is 1.109.

In a single camera single source gaze tracking system, if we can accurately estimate the 3-D coordinates of the iris center and the center of corneal surface curvature, we can reconstruct 3-D gaze direction. A novel method is proposed in this paper, different from other gaze tracking systems, the user calibration in this paper is the calibration of the iris radius, corneal curvature radius and kappa angle. Through the simulation experiments, the feasibility of the algorithm proposed in this paper is analyzed, and the error of the algorithm is relatively small.

Oliva-Lozano, JM, Martín-Fuentes, I, Granero-Gil, P, and Muyor, JM. Monitoring elite soccer players physical performance using real-time data generated by electronic performance and tracking systems. J Strength Cond Res 36(11): 3224-3228, 2022-The aims of this technical report were to analyze the validity of real-time data collected by electronic performance and tracking systems (EPTS) and investigate the effect of varying real-time receiver's position on the real-time data collected. Physical performance data were collected from professional soccer players using EPTS. In addition, 3 real-time receivers, which were placed in different positions (i.e., central area of the stadium stands and right and left technical areas), were used to collect real-time data. The real-time data collected by each receiver were visualized on SVivo and compared with the data downloaded directly from the device on SPro. The results showed no statistically significant differences between the data collected by the real-time receivers compared with postsession data in any variable ( p > 0.05), except for total distance and high-speed running distance covered, which showed significant differences but trivial effect size ( p < 0.05; d = 0.01). The coefficient of determination ( R2 ) and intraclass correlation coefficient were greater than 0.97 and 0.99, respectively. Regarding the analysis of varying the receiver's position on the real-time data collected, the results showed that there was no significant effect of the receiver's position on any variable ( p > 0.05). Therefore, valid physical performance data may be obtained by real-time tracking systems such as SVivo, regardless of the position of the real-time receivers and distance to the players. Specifically, high-intensity running actions, distances covered at low and high speed, and accelerometer-derived variables such as player load may be accurately tracked by this real-time tracking software.

We report on the design of a real-time tracking system using precise space and time information of the hit. Tracking detectors with about 150 ps time resolution are already available and intense R&D is in progress to improve the performance. We describe the implementation of a real-time tracking algorithm in commercial FPGAs for a prototype tracking system based on 8 planes of silicon sensors used as a case study. We demonstrate that the use of the additional time information in the tracking algorithm allows to suppress noise hits while reducing fake track combinations and improving real-time track reconstruction. The system provides offline-like quality tracks with sub-microsecond latency and it is capable to determine the time of the track with picosecond resolution assuming 10 ps resolution for the hits.

In real time object tracking is an important task in various surveillance application. Nowadays, surveillance system very common in offices, collage campus, shopping malls etc. In this paper, an automated video surveillance system presented. Althrough tracking and counting system are commercially available today; there is need for further reaserach to address the challenges of real world scenarios. There is lot of surveillance cameras installed around us but there are no means to monitor all of them continuously. It is necessary to develop a computer vision based technologies that automatically process those images in order to detect situations or unusual behavior. In order to solve the problem of poor real time tracking performance using convolutional neural network. This paper proposes a fast and accurate real time video detection and tracking algorithm. Keyword: - Video surveillance system, Moving object detection, Tracking, Background subtraction. I. Introduction Real time human body detection and tracking in indoor environment deals with the concept of higher level application of image processing and computer vision. The following key components of identify to path of real time tracking are extracting a feature, background image subtraction. If two similar color objects come to close the tracker of one object can jump to associated object. The problem is called hijacking problem. Drifting problem occur when an object abruptly changes its direction to reverse. In that case it’s become very difficult to track the object because the motion model does not work. An object tracking can be understood as the problem of finding the path (i.e. Trajectory).

In this paper, a new real-time and intelligent target tracking system basing on the pan/tilt controlled-stereo camera embedded on an UGV (unmanned ground vehicle) is proposed. In the proposed system, firstly the face area of a moving person is detected from a sequence of left images captured by a stereo camera using a threshold value of YCbCr color model and then, the stereo camera is kept to track the moving target in real-time by controlling the pan/tilt system. Secondly a depth map of the target face is extracted by using a relationship between the geometry of a stereo camera system and the disparity map of a rectified stereo image. Finally, from this extracted depth map, a distance from the camera system to the target and the coordinates of the target are calculated and these target coordinates are used for extracting 3D information of the target to plan paths and construct the coordinates maps. From some experiments using 1,280 frames of the input stereo image pairs, it is analyzed that displacement on the center position of the face after tracking is kept to be very low values of 0.6% for 1,280 frames on average. Also, the error ratio between the measured and computed distance values of the target from the camera system is kept to be very low value of 0.5% on average. In addition, the proposed system has achieved a speed of 0.04 sec/frame for target detection and 0.06 sec/frame for target tracking. These experimental results suggest a possibility of implementing a practical stereo camera-based automatic target tracking system in the UGV having a high degree of accuracy and a very fast response time.© (2004) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.

First Demonstration of Combined kV/MV Image-Guided Real-Time Dynamic Multileaf-Collimator Target Tracking

Due to the rapid growth of computer science, image processing technology has recently been widely applied to various fields such as videoconferencing, computer vision and face tracking. However, the development of tracking systems is at a standstill because most studies have used the still camera and pure background. In an effort for advancement, this study proposes a wireless real-time tracking system based on background subtraction, the Gauss motion model, and the Otsu algorithm, to identify a moving object and then pursue it. This system consists of (a) a tracking platform with ARM and robot and (b) an image recognition system that regulates the location of the robot to trace the object and identifies the location of the object from the real-time image, respectively. In practice, the results of this study show that the proposed system can effectively trace the specific moving object and mark the object in the center of the view.