Real-time one-dimensional motion estimation and its application in computer vision

In-room MRI is a promising image guidance strategy in external beam radiotherapy to acquire volumetric information for moving targets. However, limitations in spatio-temporal resolution led several authors to use 2D orthogonal images for guidance. The aim of this work is to present a method to concurrently compensate for non-rigid tumour motion and provide an approach for 3D reconstruction from 2D orthogonal cine-MRI slices for MRI-guided treatments. Free-breathing sagittal/coronal interleaved 2D cine-MRI were acquired in addition to a pre-treatment 3D volume in two patients. We performed deformable image registration (DIR) between cine-MRI slices and corresponding slices in the pre-treatment 3D volume. Based on an extrapolation of the interleaved 2D motion fields, the 3D motion field was estimated and used to warp the pre-treatment volume. Due to the lack of a ground truth for patients, the method was validated on a digital 4D lung phantom. On the phantom, the 3D reconstruction method was able to compensate for tumour motion and compared favourably to the results of previously adopted strategies. The difference in the 3D motion fields between the phantom and the extrapolated motion was 0.4±0.3mm for tumour and 0.8±1.5mm for whole anatomy, demonstrating feasibility of performing a 3D volumetric reconstruction directly from 2D orthogonal cine-MRI slices. Application of the method to patient data confirmed the feasibility of utilizing this method in real world scenarios. Preliminary results on phantom and patient cases confirm the feasibility of the proposed approach in an MRI-guided scenario, especially for non-rigid tumour motion compensation.

In this paper, we present an extension to optical flow for estimating dense 2D motion fields with occlusion information. While optical flow computa- tion is based on two consecutive short-exposure images, we make also use of an additional intermediate long-exposure image. This motion-blurred in- termediate image captures scene motion directly. We introduce an image formation model that relates the long-exposure image to both short-exposure images in terms of dense 2D motion and local occlusion/disocclusion. Based on this image formation model, we describe a practical algorithm that en- ables estimating the 2D motion field of occluding and occluded regions as well as determining the instant of occlusion. Results for synthetic and real scenes demonstrate the validity of the proposed approach.

High-quality non-intrusive human motion capture is necessary for acquistion\nof model-based free-viewpoint video of human actors.\nSilhouette-based approaches have demonstrated that they are able to \naccurately recover a large range of human motion from multi-view video.\nHowever, they fail to make use of all available information, specifically that \nof texture information. This paper presents an algorithm \nthat uses motion fields constructed from optical flow in multi-view video \nsequences.\n\nThe use of motion fields augments the silhoutte-based method by incorporating \ntexture-information into the tracking process. \nThe algorithm is a key-component in a larger free-viewpoint video system of \nhuman actors. \nOur results demonstrate that our method accurately estimates pose parameters \nand allows for realistic texture generation in 3D video sequences.

Respiratory motion is a long-standing challenge for lung stereotactic body radiotherapy (SBRT), particularly for centrally located lung tumors where increased toxicity demands more precise motion management during treatment. Current two-dimensional imaging approaches are insufficient for 3D tumor deformable motion tracking. In this study, we developed and evaluated a Surface-derived Three-dimensional (3D) AI-driven Real-time (STAR) imaging system by transforming a surface imaging system into a 3D real-time imaging solution. STAR integrated two key components: (1) prior-model-free spatiotemporal implicit neural representation (PMF-STINR): a machine-learning sub-system for pretreatment dynamic cone-beam CT (CBCT) reconstruction and motion modeling; and (2) surface-to-deformation network (Surf2DefNet): a deep-learning model that correlates intra-treatment body surface images with internal 3D anatomy and motion fields, trained based on the dynamic CBCT and motion model output of PMF-STINR. Specifically, PMF-STINR reconstructs a reference CBCT and solves an eigenvector-based motion model from a pretreatment CBCT, while Surf2DefNet predicts the motion eigen-vector weightings from surface images, enabling it to infer real-time CBCTs and motion vector fields (MVFs) using intra-treatment surface maps acquired later. We evaluated STAR imaging using both a digital extended cardiac torso (XCAT) phantom with regular and irregular motion patterns and ten patient datasets. The relative error (RE), center-of-mass error (COME), 95th percentile Hausdorff distance (HD95), Dice coefficients (DICE), and Pearson correlation coefficient (PCC) metrics between STAR images and the 'GT' were evaluated. For the XCAT phantom and the ten patients, the mean COME values are within 1 mm for all but one patient (1.3 mm). The RE values were consistently low, and the DICE and PCC values exceeded 0.89 in all cases. The HD95 are all within 2 mm except for one patient (2. 78 mm). These results demonstrate that the STAR imaging system can achieve accurate spatiotemporal reconstructions from surface images, providing CBCTs and MVFs for intra-treatment real-time image-guidance, and has the potential to improve safety and efficacy of SBRT for lung cancer.

The industrial robots play an important role in modern industrial manufacturing processes, and their applications are becoming increasingly widespread, greatly improving manufacturing efficiency and significantly enhancing the digitalization level of the manufacturing industry. Therefore, achieving safe control of multi robot collaborative work is of great significance for the stable and efficient operation of robot systems. This article proposes a dual robot safety control technology based on the kinematic model of industrial robots. Firstly, a mathematical model of the kinematic space of the dual robots is constructed using D-H parameters; Then, use recursive traversal algorithm to calculate the minimum distance between each joint arm of the two robots in real time; Finally, the minimum distance between each joint arm will be calculated and compared with the set safety threshold to achieve real-time assessment and control of collision risk during the movement of the dual robots. A dual robot safety control testing platform was built using two KUKA industrial robots, integrated control software was developed, and relevant algorithms were validated. Through experiments, it was verified that this method can evaluate the collision risk of dual robots in real time during motion and effectively control the collision risk during the motion of dual robots.

A high‐speed field‐programmable gate array (FPGA) design is proposed to calculate angles and distances for biped robots in real time. A low‐complexity and high‐accuracy hardware‐oriented algorithm based on CORDIC was developed. To reduce hardware cost, a hardware sharing technique was used to realise a CORDIC scaling factor generator and three hardware sharing machines. Moreover, the multipliers and dividers were replaced by cost‐efficiency components, such as adders and shifters to further reduce hardware cost. In addition, the proposed design was implemented with a fully pipelined architecture, which achieved improved operating frequency and throughput efficiency. The proposed design was realised and verified by an FPGA device with a maximum operating frequency of 127 MHz, which achieved the calculation of angles and distances for biped robots in real time. Compared with the previous design, this work not only reduced hardware cost by at least 49.6% and average errors by at least 67.2%, but also improved the average executing performance by 62.7% when calculating ten angles for the biped robots.

Introduction: The position and pose measurement of the rehabilitation robot plays a very important role in patient rehabilitation movement, and the non-contact real-time robot position and pose measurement is of great significance. Rehabilitation training is a relatively complicated process, so it is very important to detect the training process of the rehabilitation robot in real time and accuracy. The method of the deep learning has a very good effect on monitoring the rehabilitation robot state. Methods: The structure sketch and the 3D model of the 3-PRS ankle rehabilitation robot are established, and the mechanism kinematics is analyzed to obtain the relationship between the driving input - the three slider heights - and the position and pose parameters. The whole network of the position and pose measurement is composed of two stages: (1) measuring the slider heights using the CNN based on the robot image and (2) calculating the position and pose parameter using the BPNN based on the measured slider heights from the CNN. According to the characteristics of continuous variation of the slider heights, a regression CNN is proposed and established to measure the robot slider height. Based on the data calculated by using the inverse kinematics of the 3-PRS ankle rehabilitation robot, a BPNN is established to solve the forward kinematics for the position and pose. Results: The experimental results show that the regression CNN outputs the slider height and then the BPNN accurately outputs the corresponding position and pose. Eventually, the position and pose parameters are obtained from the robot image. Compared with the traditional robot position and pose measurement method, the proposed method has significant advantages. Conclusion: The proposed 3-PRS ankle rehabilitation position and pose method can not only shorten the experiment period and cost, but also get excellent timeliness and precision. The proposed approach can help the medical staff to monitor the status of the rehabilitation robot and help the patient rehabilitation in training. Discussion: The goal of the work is to construct a new position and pose detection network based on the combination of the regression convolutional neural network (CNN) and the back propagation neural network (BPNN). The main contribution is to measure the position and pose of the 3-PRS ankle rehabilitation robot in real time, which improves the measurement accuracy and the efficiency of the medical staff work.

The last few years have been strongly marked by the presence of multimedia data (images and videos) in our everyday lives. These data are characterized by a fast frequency of creation and sharing since images and videos can come from different devices such as cameras, smartphones or drones. The latter are generally used to illustrate objects in different situations (airports, hospitals, public areas, sport games, etc.). As result, image and video processing algorithms have got increasing importance for several computer vision applications such as motion tracking, event detection and recognition, multimedia indexation and medical computer-aided diagnosis methods. In this paper, we propose a real time cloud-based toolbox (platform) for computer vision applications. This platform integrates a toolbox of image and video processing algorithms that can be run in real time and in a secure way. The related libraries and hardware drivers are automatically integrated and configured in order to offer to users an access to the different algorithms without the need to download, install and configure software or hardware. Moreover, the platform offers the access to the integrated applications from multiple users thanks to the use of Docker (Merkel, 2014) containers and images. Experimentations were conducted within three kinds of algorithms: 1. image processing toolbox. 2. Video processing toolbox. 3. 3D medical methods such as computer-aided diagnosis for scoliosis and osteoporosis.  These experimentations demonstrated the interest of our platform for sharing our scientific contributions related to computer vision domain. The scientific researchers could be able to develop and share easily their applications fastly and in a safe way.

InceptB: A CNN Based Classification Approach for Recognizing Traditional Bengali Games

Intrinsic image decomposition is an important topic in computer vision and computer graphics applications. However, this is a challenging problem by adopting the information of a single image. Therefore, additional priors or supplementary information such as multiply images or user interactions are necessary to address this problem. In this paper, we propose a novel scheme to use multiple images for intrinsic image decomposition, based on a similar strategy in robust principal component analysis (RPCA). RPCA utilizes the fact that the reflectance layer is common in multiple lights of a scene, and attempts to decompose the data matrix constructed from input images into a low-rank matrix and a sparse matrix. This is possible if the sparse matrix is sufficiently sparse, which is often not the case in computer vision applications. Moreover, the weighting parameter between the low-rank and sparse matrices greatly affects the accuracy of the results, and tuning this parameter can be tricky. This paper proposes a rank-constrained PCA algorithm (RCPCA) for solving background recovering problems. Fixing the rank of the low-rank matrix to be 1 allows RCPCA to better recover the low-rank matrix from the data matrix. Comprehensive tests show that RCPCA produces more stable and accurate results than RP-CA.

Deformable models are a useful tool in computer vision and computer graphics. A deformable model is a curve (in two dimensions) or a surface (in three dimensions), whose shape, position, and orientation are controlled through a set of parameters. Deformable models can represent manufactured objects, human faces and skeletons, and even bodies of fluid. In computer graphics, we use deformable models for animations and simulations, whereas in computer vision applications, such as tracking and fitting, deformable models help to restrict the family of possible solutions. We introduce the use of a directed acyclic graph (DAG) to describe the position and Jacobian of each point on the surface of deformable models. This data structure, combined with a topological description of the points, is simple, powerful, and extremely useful for both computer vision and computer graphics applications. We show a computer vision application, 3D deformable face tracking, and a computer graphics application, cyberglove data visualization and calibration.

This paper presents a novel framework for importance-aware adaptive data transmission, designed specifically for real-time computer vision (CV) applications where taskspecific fidelity is critical. An importance-weighted mean square error (IMSE) metric is introduced, assigning data importance based on bit positions within pixels and semantic relevance within visual segments, thus providing a task-oriented measure of reconstruction quality. To minimize IMSE under the total power constraint, a data-importance-aware waterfilling approach is proposed to optimally allocate transmission power according to data importance and channel conditions. Simulation results demonstrate that the proposed approach significantly outperforms margin-adaptive waterfilling and equal power allocation strategies, achieving more than 7 dB and 10 dB gains in normalized IMSE at high SNRs (<tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$&gt;10 ~\text{dB}$</tex>), respectively. These results highlight the potential of the proposed framework to enhance data efficiency and robustness in real-time CV applications, especially in bandwidth-limited and resource-constrained environments.

One-dimensional motion of interstitial clusters in iron-based binary alloys observed using a high-voltage electron microscope

Many computer vision applications need motion detection and analysis. In this research, a newly developed feature descriptor is used to find sparse motion vectors. Based on the resulting sparse motion field the camera motion is detected and analyzed. Statistical analysis is performed, based on polar representation of motion vectors. Direction of motion is classified, based on the statistical analysis results. The motion field further is used for depth analysis. This proposed method is evaluated with two video sequences under image deformation: illumination change, blurring and camera movement (i.e. viewpoint change). These video sequences are captured from a moving camera (moving/driving car) with moving objects.

In real time object tracking is an important task in various surveillance application. Nowadays, surveillance system very common in offices, collage campus, shopping malls etc. In this paper, an automated video surveillance system presented. Althrough tracking and counting system are commercially available today; there is need for further reaserach to address the challenges of real world scenarios. There is lot of surveillance cameras installed around us but there are no means to monitor all of them continuously. It is necessary to develop a computer vision based technologies that automatically process those images in order to detect situations or unusual behavior. In order to solve the problem of poor real time tracking performance using convolutional neural network. This paper proposes a fast and accurate real time video detection and tracking algorithm. Keyword: - Video surveillance system, Moving object detection, Tracking, Background subtraction. I. Introduction Real time human body detection and tracking in indoor environment deals with the concept of higher level application of image processing and computer vision. The following key components of identify to path of real time tracking are extracting a feature, background image subtraction. If two similar color objects come to close the tracker of one object can jump to associated object. The problem is called hijacking problem. Drifting problem occur when an object abruptly changes its direction to reverse. In that case it’s become very difficult to track the object because the motion model does not work. An object tracking can be understood as the problem of finding the path (i.e. Trajectory).