Image convolution techniques integrated with YOLOv3 algorithm in motion object data filtering and detection

This chapter presents a novel MoG based method for foreground detection and segmentation in video surveillance. Normal MoG is difficult to deal with the foreground objects stay in the scene for a long time and segment different foreground objects from one blob. We improve MoG by adopting posterior feedback information of object tracking based on statistics to robustly modeling the background and to perfect the foreground segmentation result. Experiments and comparisons show that our method is robust and accurate for detecting foreground object in video surveillance.

Both visible and infrared images are important sources of intelligence information on the battlefield, and air‐to‐ground reconnaissance by UAV is an important means to obtain intelligence. However, there are great challenges in ground target detection and tracking, especially in complex battlefield environments. Aiming at the problem of insufficient accuracy of target detection by a single type of sensor in the battlefield environment at this stage, a target detection method by fusion of visible and infrared images is proposed in this paper, which is called ReconnaissanceFusion‐YOLO (RF‐YOLO), and with the help of infrared imagery, it can effectively improve the accuracy of target detection in the case of insufficient light. The performance of target detection in the battlefield is significantly improved by introducing two key innovative modules: dual feature fusion (DFF) module and feature fusion corrector (FFC) module. The DFF module enhances multi‐channel feature fusion through a novel concatenation and channel‐wise attention mechanism, while the FFC module performs feature correction between parallel streams using spatial and channel‐wise weights, addressing noise and uncertainty in different modalities. These modules are integrated on top of a dual‐stream YOLO architecture, allowing for effective fusion of visible and infrared information. RF‐YOLO was trained and evaluated using the FLIR data set, containing 5142 pairs of strictly aligned visible and infrared images. Results demonstrate that RF‐YOLO significantly outperforms benchmark networks in terms of robustness requirements. Specifically, the large model of RF‐YOLO achieves an mAP of 0.831, which is a significant improvement compared to the YOLOv5l inf benchmark's 0.739. This represents an improvement of over 12% in detection accuracy. Additionally, RF‐YOLO offers a Nano version that balances accuracy and speed. The Nano version achieves an mAP of 0.765, while maintaining a model size of only 11.5 MB, making it suitable for deployment on UAV edge computing devices with limited resources. To validate the practical applicability of our approach, this paper successfully implements target detection and tracking on a real UAV's edge computing device using the ROS system and SiameseRPN, combined with the proposed RF‐YOLO. Real‐world flight tests were conducted on an internal playground, demonstrating the effectiveness of our method in actual UAV applications. The system achieved a processing rate of approximately 10 fps at 640 × 640 resolution on an NVIDIA TX2 edge computing device, showcasing its real‐time performance capability in practical scenarios. This study contributes to enhancing UAV‐based battlefield reconnaissance capabilities by improving the accuracy and robustness of target detection and tracking in complex environments. The proposed RF‐YOLO method, along with its successful implementation on a real UAV platform, provides a promising solution for advanced military intelligence gathering and decision‐making support.

This paper discusses the integration of a geographic information system (GIS) and moving objects in surveillance videos (“moving objects” hereinafter) by using motion detection, spatial mapping, and fusion representation techniques. This integration aims to overcome the limitations of conventional video surveillance systems, such as low efficiency in video searching, redundancy in video data transmission, and insufficient capability to position video content in geographic space. Furthermore, a model for integrating GIS and moving objects is established. The model includes a moving object extraction method and a fusion pattern for GIS and moving objects. From the established integration model, a prototype of GIS and moving objects (GIS–MOV) system is constructed and used to analyze the possible applications of the integration of GIS and moving objects.

In recent years, visual object tracking has become a very active research field which is mainly divided into the correlation filter-based tracking and deep learning (e.g., deep convolutional neural network and Siamese neural network) based tracking. For target tracking algorithms based on deep learning, a large amount of computation is required, usually deployed on expensive graphics cards. However, for the rich monitoring devices in the Internet of Things, it is difficult to capture all the moving targets in each device in real time, so it is necessary to perform hierarchical processing and use tracking based on correlation filtering in insensitive areas to alleviate the local computing pressure. In sensitive areas, upload the video stream to a cloud computing platform with a faster computing speed to perform an algorithm based on deep features. In this paper, we mainly focus on the correlation filter-based tracking. In the correlation filter-based tracking, the discriminative scale space tracker (DSST) is one of the most popular and typical ones which is successfully applied to many application fields. However, there are still some improvements that need to be further studied for DSST. One is that the algorithms do not consider the target rotation on purpose. The other is that it is a very heavy computational load to extract the histogram of oriented gradient (HOG) features from too many patches centered at the target position in order to ensure the scale estimation accuracy. To address these two problems, we introduce the alterable patch number for target scale tracking and the space searching for target rotation tracking into the standard DSST tracking method and propose a visual object multimodality tracker based on correlation filters (MTCF) to simultaneously cope with translation, scale, and rotation in plane for the tracked target and to obtain the target information of position, scale, and attitude angle at the same time. Finally, in Visual Tracker Benchmark data set, the experiments are performed on the proposed algorithms to show their effectiveness in multimodality tracking.

Computer vision-based motion target detection and tracking, which is widely used in video surveillance, human-computer interaction, range interpretation, and other fields, is one of the current research hotspots in the field of computer vision. In engineering scenarios, the two are inseparable and need to work together to accomplish specific tasks. The related research is progressing rapidly, but there is still room for improving its timeliness, accuracy, and automation. In this paper, we summarize and classify some classical target detection methods, analyze the basic principles of convolutional neural networks, and analyze the classical detection algorithms based on region suggestion and deep regression networks. After that, we improve the SSD algorithm for the shortage of low-level feature convolution layers, which has insufficient feature extraction and leads to poor detection of small targets. For the motion target tracking problem, this paper studies the motion target tracking method based on support vector machine and proposes the tracking method of support vector regression and the corresponding online support vector regression solution method based on the analysis of support vector tracking method and structural support vector tracking method. In this paper, we propose a tracking method that fuses structural support vector machines and correlation filtering. The structure is based on the idea of Inception, which adds and replaces some feature convolution layers of the original network while maintaining the original lightweight backbone. The final experiments on the VOC data set demonstrate that the improved algorithm improves the average detection accuracy by 2.6% compared to the original algorithm and basically maintains the real-time speed as well. Experimental simulations on a subset of VOC data (human set) show a significant improvement in AP values and more effective and reliable detection tracking of moving targets. The stability and accuracy of motion target detection and tracking are improved by setting parameters, such as confidence level; the effectiveness and continuity of detection and tracking are judged by setting the interframe centroid distance.

Identifying and tracking objects in surveillance videos is an important means of mining information during surveillance. Currently, most object‐tracking methods rely only on image features, which cannot accurately express the motion of the tracked object in real geographical scenes and are easily influenced by occlusion and surrounding objects having similar features. However, tracked objects, such as pedestrians and vehicles, usually move in geographical space with fixed patterns of motion, and the motion in a short time is constrained by geographical space and time, the motion trajectory is predictable, and the range of motion is limited. Therefore, based on the SiamFC object tracking framework, this study introduces geographical spatiotemporal constraints into the tracking framework and proposes the GeoSiamFC method. The objective of this is to: (1) construct the mapping relationship between geographical space and image space to solve the problem that the pixel movement within the image after perspective imaging cannot accurately express the motion of the tracked object in a real geographical scene; (2) add candidate search areas according to the predicted trajectory location to correct the tracking errors caused by the occlusion of the object; and (3) focus on the search for the range of motion of the mapped pixel within the image space according to the limited geographical range of motion of the tracked objects in a certain time to reduce the interference of similar objects within the search area. In this study, separate experiments were conducted on a common test dataset using multiple methods to deal with challenges such as occlusion and illumination changes. In addition, a robust test dataset with noise addition and luminance adjustment based on the common test dataset was used. The results show that GeoSiamFC outperforms other object‐tracking methods in the common test dataset with a precision score of 0.995 and a success score of 0.756 compared with most other object‐tracking algorithms under the base condition of using only shallow networks. Moreover, GeoSiamFC maintained the highest precision score (0.970) and high success score (0.734) in the more challenging robust test dataset as well. The tracking speed of 59 frames per second far exceeds the real‐time requirement of 25 FPS. Geographical spatiotemporal constraints were considered to improve tracker performance while providing real‐time feedback on the motion trajectory of the target in geographical space. Thus, the proposed method is suitable for real‐time tracking of the motion trajectory of a target in real geographical scenes in various surveillance videos.

We propose a robust carried object recognition method from surveillance video. Though tremendous strides have been made in the object recognition by using the appearance-based method such as Faster R-CNN, YOLO, etc., recognizing carried object classes is still a difficult task. It is because the carried objects in surveillance video are of small size, low-resolution, occluded by people. To address the problem, we introduce prior knowledge of the location relation between carried objects and human body parts to improve the performance of the appearance-based method. Specifically, for the extracted human region, we detect human body parts and use prior knowledge to estimate the probability distribution of the existence of each class of carried objects. We update the score results of the appearance-based method by applying the probability distribution. In this way, we can reduce false positives and enhance confidence score of true positives. We evaluate the proposed method on four datasets and show that the proposed method outperforms Faster R-CNN by 6% in terms of average F-measure.

Target detection and tracking is a key technology in intelligent monitoring system. However, it is still difficult to design a robust, accurate and real-time target detection and tracking algorithm. Taking target Tracking as the research focus, this paper focuses on one of the current mainstream algorithms, namely, TLD (tracking-learning-detection) target Tracking algorithm. Based on the theory of TLD target tracking and corner detection, a new target tracking method was proposed to solve the existing problems. A classifier is built online to track and learn the target in real time, and a target contour detector is added in the system to complement the random forest detector to solve the problem of tracking failure. Experimental results show that the proposed algorithm can effectively overcome the influence of fast change or depth occlusion, and improve the tracking efficiency of moving targets.

As a key technology of detection and tracking system, moving target detection and tracking has attracted a great deal of attention. To achieve the goal of tracking target, conventional methods have to complete the detection of targets firstly, and then track the target. However, such kind of approaches require using the method of constant false-alarm rate to detect the whole frame image received at that time, which not only reduces the detection efficiency but also degrades the performance of target tracking. In order to implement real-time monitoring divers and other small moving targets under the conditions of strong underwater noise and clutter interference, an interactive algorithm for moving target detection and tracking is proposed based on acoustic image, which jointly analyzes the target detection system and the target tracking system. In the target tracking unit, the interactive algorithm takes advantage of data interconnection to remove the false targets produced by target detection; extrapolates the target's track, and predicts the location that the target may appear at the next moment by employing the Kalman filter. These messages will be used as priori information in the detection unit. In the detection unit, first of all, a gate is established whose center is the position that the target tracking unit predicted. For the image received at the next moment, detection in the predicted gate in implemented by using the method of constant false-alarm rate, followed by feeding back the detection results to the tracking unit. Compared with existed methods, the proposed algorithm can effectively improve the target detection efficiency and tracking performance, providing a better approach to monitor underwater small targets.

Feature expression is a crucial part of the target tracking process. The artificial feature is relatively simple and has strong real-time performance, but there is a problem of insufficient representation ability. It is prone to drift when dealing with problems such as rapid change and target occlusion. With the strong feature expression ability of deep neural network features in target detection and recognition tasks, deep neural network features are gradually used as feature extraction tools, but how to use and integrate these features is still worth studying. In this paper, the Residual Neural Network(ResNet) is the main researched object, and the influence of each layer on the target tracking performance is analyzed in detail. The feature fusion strategy of the convolutional layer and the addition layer is finally determined. We train a classifier separately for these layers. Then we search the multi-layer response maps to infer the target location in a coarse-to-fine fashion. The algorithm of this paper is verified on the OTB-50 dataset. The one-pass evalution(OPE) value can reach 0.612, which is better than the same type of algorithms.

The novel coronavirus has broken out from 2019 and quickly become a global pandemic. It spreads rapidly from person to person through droplets, aerosols and other carriers. In order to prevent the spread of the virus, people must wear masks when entering and leaving public places and taking public transport to reduce the risk of virus transmission. How to detect the wearing of masks in public places and other natural environments has become a new research problem. This paper proposes a lightweight deep neural network (E-YOLO) to realize mask wearing detection in real-time scenarios. E-YOLO improves the general target detection YOLOv3 algorithm by the follows methods. Firstly, the Efficient-Net series B2 backbone feature extraction network is used to replace the original Darknet53 feature extraction network. Combined with the spatial pyramid pooling module and the bidirectional feature pyramid structure to enrich the semantic information of the feature layer, so as to achieve the balance between the speed and accuracy of target detection in real-time scenes. The experimental results show that, compared with the YOLOv3 algorithm, the E-YOLO algorithm possesses the same accuracy and speed. The network floating point calculation amount is one twelfth of YOLOv3, and the model size is only one quarter of YOLOv3, which is more suitable for resource-constrained platforms.

This paper proposes a privacy preserving scheme for data security in the video surveillance. We firstly separate the foreground for each video frame, and obscure the separated human object by motion blur. For secure storage, each blurred foreground object is encrypted into N shares by visual cryptography, and stored into different servers. Each share is fully confidential and does not convey any meaningful information about the original video, so that breaking into one storage server do not induce any compromise. For legal requirement, the authorized users can recover the original content with better quality by non-blind deblurring algorithm. Moreover, thanks to our exploited foreground based encoding scheme, the data expansion introduced by distributed storage is greatly reduced. It is impossible for unauthorized users to recover the original content by the following reasons: 1) distributed video stream storage; 2) unknown blurring kernel; 3) inaccurate foreground content and mask. The performance evaluation on several surveillance scenarios demonstrates that our proposed method can effectively protect sensitive privacy information in surveillance videos.

Since real-time and communication amount is crucial for the wireless sensor network target tracking, the performance of target tracking in the wireless sensor network is critically depended on real-time and communication amount reduction. This paper presents a target tracking method based on distributed adaptive particle filtering in binary wireless sensor network. Based on dynamic clustering, the adaptive particle filter receives the observations from children nodes and formulates the local estimate with the cluster head as the processing center. Simulation results show that the method can effectively improve the real-time tracking and reduce communication amount.

In this paper an uncooled infrared camera with embedded target detection and tracking capability is presented. The camera is built upon four main electronic cards. Sensor card is designed to read and digitize the analog output signal generated by the infrared (IR) imaging sensor. FPGA card carries out the control process of the data transfer between sensor card and other peripherals. Target detection/tracking task is on the row image data provided by the FPGA card is carried out on the DSP card. Frame thresholding and differencing methods are used for target detection. Kalman filter is utilized for target tracking. According to the experimental results, camera is capable of performing target detection and tracking tasks in real time.

Accurate, fast and lightweight dense target detection methods are highly important for precision agriculture. To detect dense apricot flowers using drones, we propose an improved dense target detection method based on YOLOv8, named D-YOLOv8. First, we introduce the Dense Feature Pyramid Networks (D-FPN) to enhance the model’s ability to extract dense features and Dense Attention Layer (DAL) to focus on dense target areas, which enhances the feature extraction ability of dense areas, suppresses features in irrelevant areas, and improves dense target detection accuracy. Finally, RAW data are used to enhance the dataset, which introduces additional original data into RAW images, further enriching the feature input of dense objects. We perform validation on the CARPK challenge dataset and constructed a dataset. The experimental results show that our proposed D-YOLOv8m achieved 98.37% AP, while the model parameters were only 13.2 million. The improved network can effectively support any task of dense target detection.