Multi-Camera Tracking System Applications Based on Reconfigurable Devices: A Review

Cardiac x-ray CT imaging is still challenging due to heart motion, which cannot be ignored even with the current rotation speed of the equipment. In response, many algorithms have been developed to compensate remaining motion artifacts by estimating the motion using projection data or reconstructed images. In these algorithms, accurate motion estimation is critical to the compensated image quality. In addition, since the scan range is directly related to the radiation dose, it is preferable to minimize the scan range in motion estimation. In this paper, the authors propose a novel motion estimation and compensation algorithm using a sinogram with a rotation angle of less than 360°. The algorithm estimates the motion of the whole heart area using two opposite 3D partial angle reconstructed (PAR) images and compensates the motion in the reconstruction process. A CT system scans the thoracic area including the heart over an angular range of 180° + α + β, where α and β denote the detector fan angle and an additional partial angle, respectively. The obtained cone-beam projection data are converted into cone-parallel geometry via row-wise fan-to-parallel rebinning. Two conjugate 3D PAR images, whose center projection angles are separated by 180°, are then reconstructed with an angular range of β, which is considerably smaller than a short scan range of 180° + α. Although these images include limited view angle artifacts that disturb accurate motion estimation, they have considerably better temporal resolution than a short scan image. Hence, after preprocessing these artifacts, the authors estimate a motion model during a half rotation for a whole field of view via nonrigid registration between the images. Finally, motion-compensated image reconstruction is performed at a target phase by incorporating the estimated motion model. The target phase is selected as that corresponding to a view angle that is orthogonal to the center view angles of two conjugate PAR images. To evaluate the proposed algorithm, digital XCAT and physical dynamic cardiac phantom datasets are used. The XCAT phantom datasets were generated with heart rates of 70 and 100 bpm, respectively, by assuming a system rotation time of 300 ms. A physical dynamic cardiac phantom was scanned using a slowly rotating XCT system so that the effective heart rate will be 70 bpm for a system rotation speed of 300 ms. In the XCAT phantom experiment, motion-compensated 3D images obtained from the proposed algorithm show coronary arteries with fewer motion artifacts for all phases. Moreover, object boundaries contaminated by motion are well restored. Even though object positions and boundary shapes are still somewhat different from the ground truth in some cases, the authors see that visibilities of coronary arteries are improved noticeably and motion artifacts are reduced considerably. The physical phantom study also shows that the visual quality of motion-compensated images is greatly improved. The authors propose a novel PAR image-based cardiac motion estimation and compensation algorithm. The algorithm requires an angular scan range of less than 360°. The excellent performance of the proposed algorithm is illustrated by using digital XCAT and physical dynamic cardiac phantom datasets.

The coding efficiency of the H.264/AVC standard enables the transmission of high quality video over bandwidth limited networks. Due to the use of multiple Macroblock (MB) partitions, the Motion estimation module has extremely high complexity that makes it unpractical for most real-time applications on resource-limited platforms such as hand held devices. In this paper we propose a novel algorithm that significantly reduces the encoding complexity while maintaining high rate distortion performance. The proposed method reduces the Motion estimation (ME) computational complexity by accurately predicting the optimal MB partitions and restricting the number of candidate modes based on a-priori probabilities computed from spatio-temporal information. The experimental results show that the speed up of UmHexagonS [1] (one of the most efficient ME algorithms) can be doubled while maintaining the coding efficiency of Full Search.

Aiming at the higher bit-rate occupation of motion vector encoding and more time load of full-searching strategies, a multi-resolution motion estimation and compensation algorithm based on adjacent prediction of frame difference was proposed. Differential motion detection was employed to image sequences and proper threshold was adopted to identify the connected region. Then the motion region was extracted to carry out motion estimation and motion compensation on it. The experiment results show that the encoding efficiency of motion vector is promoted, the complexity of motion estimation is reduced and the quality of the reconstruction image at the same bit-rate as Multi-Resolution Motion Estimation (MRME) is improved.

This paper proposes a method to improve the H.261 video encoder by using a new motion estimation (ME) algorithm instead of the full search block matching motion estimation algorithm (BMA). We thought that the traditional BMA based on the least square error or absolute error criterion may not be suitable for the H.261 video encoder. A good matching criterion should consider not only the interframe prediction error energy, but also the total bit number for encoding the motion vectors and interframe prediction error. Motivated by the above idea, we introduced a new motion estimation algorithm, which performs motion estimation in the DCT domain, and use a new matching criterion-the minimum bit rate criterion. Experimental results show that the performance of the H.261 encoder is improved greatly.

We present an efficient computation constrained block-based motion vector estimation algorithm for low bit rate video coding that offers good tradeoffs between motion estimation distortion and number of computations. A reliable predictor determines the search origin. An efficient search pattern exploits structural constraints within the motion field. A flexible cost measure used to terminate the search allows simultaneous control of the motion estimation distortion and the computational cost. Experimental results demonstrate the viability of the proposed algorithm in low bit rate video coding applications, achieving essentially the same levels of rate-distortion performance and subjective quality as that of the full search algorithm when used by the UBC H.263+ video coding reference software. However the proposed motion estimation algorithm provides substantially higher encoding speed as well as graceful computational degradation capabilities.

In order to ensure recognition accuracy, intelligent traffic video tracking system usually requires various types of information. Therefore, multi-features fusion becomes a good choice. In this paper, a new recognition approach for vehicle types based multi-feature fusion is proposed, which is used for vehicle tracking in a multi-camera traffic system. An improved Canny operator is presented for edge detection. SURF (Speeded Up Robust Features) is used for local feature extraction. To improve the performance of distance calculation between features, a refined method based on Hellinger kernel is put forward. A position constraint rule is applied to reduce unnecessary fake matchings. Finally, the information of vehicle types combined with LBP (Local Binary Pattern), HOG (Histogram of Oriented Gradients) is used for a multi-camera vehicle tracking platform, which adopts Hadoop to realize the parallel computing of the system. Experimental results show that the proposed approach has good performance for the platform.

This dissertation reports on rate-distortion optimized and computation-constrained video coding algorithms which yield effective and efficient block-based video encoders. We first propose a new cost function for motion estimation which can achieve good tradeoffs among motion vector bit rate, compensation distortion, and number of computations. Based on the new cost function, we propose a predictive motion estimation algorithm which can effectively determine the size of the search region according to the statistics of the motion field. The proposed motion estimation algorithm allows control of the motion vector bit rate and the computational cost, simultaneously. We also develop two efficient rate-distortion optimized coding mode selection algorithms, based on thresholding and finite state machine (FSM) methods, respectively. The thresholding method can eliminate consideration of the computationally expensive modes. The FSM method determines the test orders of the modes and associated parameters by exploiting the local statistics of the input video sequence. The proposed two methods can reduce substantially the computation requirements as compared to the full search mode selection method. Based on the proposed motion estimation and mode selection algorithms, we implement an H.263-based video coder and an MPEG-2, compliant video coder for very low bit rate video applications, and high bit rate interlaced video applications, respectively. The resulting video coders achieve excellent tradeoffs among bit rate, quality, and computational complexity. In fact, experimental results show that our video coders outperform the best known video coders in terms of compression performance and encoding speed.

In this paper, a novel motion estimation and compensation algorithm for frame rate up conversion is presented. Compared to conventional motion estimation methods, motion vectors are processed by different approaches for static region or internal object region and moving boundary of video stream in this proposed algorithm. In the static or internal object region, motion vectors are decided according to the interaction principle between adjacent blocks and easily obtained by the adjacent motion vectors; while for the moving boundary, motion vectors are firstly calculated according to the adjacent blocks, and then the algorithm detects the movement of block in the region by SAD values, finally, the motion vectors of moving blocks are refined and corrected.

Multi-camera tracking of vehicles on a city-scale level is a crucial task for efficient traffic monitoring. Most of the errors made by such multi-target multi-camera tracking systems arise due to tracking failures or misleading visual information of detection boxes under occlusion. Therefore, we propose an occlusion-aware approach that leverages temporal information from tracks to improve the single-camera tracking performance by an occlusion handling strategy and additional modules to filter false detections. For the multi-camera tracking, we discard obstacle-occluded detection boxes by a background filtering technique and boxes overlapping with other targets using the available track information to improve the quality of extracted visual features. Furthermore, topological and temporal constraints are incorporated to simplify the re-identification task in the multi-camera clustering. We give detailed insights into our method with ablative experiments and show its competitiveness on the CityFlowV2 dataset, where we achieve promising results ranking 4th in Track 3 of the 2021 AI City Challenge.

The rate-constrained (R-D) motion estimation techniques have been presented to improve the conventional block-matching algorithm by using a joint rate and distortion criterion. This paper presents two motion estimation algorithms using Kalman filter to further enhance the performance of the conventional R-D motion estimation at a relative low computational cost. The Kalman filter exploits the correlation of block motion to achieve higher precision of motion estimation and compensation. In the first algorithm, the Kalman filter is utilized as a postprocessing to raise the motion compensation accuracy of the conventional R-D motion estimation. In the second algorithm, the Kalman filter is embedded into the optimization process of R-D motion estimation by defining a new R-D criterion. It further improves the rate-distortion performance significantly.

A fast and efficient hybrid block-matching motion estimation algorithm is proposed in this paper. The fact that motion characteristics are relatively stable in a GOP (group of pictures) is utilized to reduce the computation burden without quality degradation. For P-frames, a multi-resolution block-matching algorithm is used. At the coarsest level, edge information is used to reduce searching positions before an exhaustive search. At the finer levels, only the several candidates chosen at the upper level are verified. For B-frames, MAE (mean absolute error) is used not only for the matching criteria, but also for classifying the macroblocks with the motion information extracted from P-frames. Concurrently, the search window size is adjusted using the contribution of different motion range to the improvement of the picture quality with motion compensation. When the decoded I-frame and P-frames are used as the reference, experimental results show that reconstructed digital frames have even better quality than the full search algorithm in terms of PSNR, under MPEG-1 coding environment.

One method to detect obstacles from a vehicle moving on a planar road surface is the analysis of motion-compensated difference images. In this contribution, a motion compensation algorithm is presented, which computes the required image-warping parameters from an estimate of the relative motion between camera and ground plane. The proposed algorithm estimates the warping parameters from displacements at image corners and image edges. It exploits the estimated confidence of the displacements to cope robustly with outliers. Knowledge about camera calibration, measuremts from odometry, and the previous estimate are used for motion prediction and to stabilize the estimation process when there is not enough information available in the measured image displacements. The motion compensation algorithm has been integrated with modules for obstacle detection and lane tracking. This system has been integrated in experimental vehicles and runs in real time with an overall cycle of 12.5 Hz on low-cost standard hardware.

This paper deals with structure and motion estimation by using image sequences captured by a multi-camera system moving in a scene. Single camera systems with a relatively small eld of view have limited accuracy because of the inherent confusion between translation and rotation. This is also the case for the stereo camera systems if the systems need the intersection of the elds of view for stereo correspondence. The cameras constituting the multi-camera system considered ultimately in this paper are arranged so that there is no, or a small, intersection of the elds of view. This conguration improves the accuracy of the structure and motion estimation because the ambiguity mentioned above decreases due to a large eld of view. We propose a recursive algorithm for real time applications of the multi-camera system using long term image sequences, which have appearing and disappearing point features. The results of the experiments with long term real image sequences are presented to demonstrate the feasibility of the algorithm.

High Efficiency Video Coding (HEVC) is the latest video coding standard that supports high resolution videos by providing approximately twice the compression efficiency as compared to its previous standard H.264. Motion Estimation (ME) in HEVC is the most computation-intensive block as a result it becomes a bottleneck in the design of the encoder while implementing video applications on various computing platforms such as general purpose and embedded processors. So developing computational efficient architectures on Field Programmable Gate Array (FPGA) and Application Specific Integrated Circuit (ASIC) platforms is inevitable. This paper proposes a fast hybrid search pattern algorithm and its hardware architecture for encoding UHD videos. The proposed Integer ME (IME) algorithm requires an average of 11.19% less encoding time than the default Test Zone Search (TZS) algorithm in HM reference software with compromising decrement in PSNR and increment in bit rate. The proposed architecture is implemented in both FPGA and ASIC platform with TSMC 90 nm technology library. It consumed 32-33% of resources in Virtex-7 FPGA and 2784.4 K equivalent gate count (in terms of NAND ) and 18 kB of memory, respectively. The results show that maximum frequency of the proposed architecture is 162 MHz and a total power consumption is 463.4 mW. The architecture provides a maximum throughput of 2.78 Gpixels/sec due to it process $$32\times 32$$ CU comparatively much less clock cycles (59.5) as compared to the state-of-the-art literature . Further, it supports 8K UHD $$(8192\times 4320)$$ @ 78 fps.

This paper introduces a novel scheme of motion estimation (ME) and motion compensation (MC) in wavelet transform (WT) video coding. The scheme is unique in that half-pixel resolution ME and MC are applied on filtered coefficients ahead of decimation to preserve motion consistency and yet achieve minimal prediction residue. An efficient motion-compensated WT video coding system is built based on the proposed ME and MC. In the coding scheme, according to the property of embedded zerotree wavelet (EZW) coding, the mean square error (MSE) criterion is used in ME instead of normally adopted sum of absolute difference, and an overlapped MC algorithm is developed to combat the block effect introduced by block-based ME and improve peak signal-to-noise ratio (PSNR) performance. The superior performance of the proposed scheme is demonstrated with experimental results.