Survey Paper on Multi-view Object Detection: Challenges and Techniques

Real-time video surveillance, especially CCTV systems, requires fast and accurate face detection. Object detection models with slow inference times are ineffective in real-time. This study addresses this challenge by improving the inference speed of the YOLOv8 model, a leading object detection framework known for its accuracy and speed. We focus on pruning the model's architecture, particularly the P5 head section, which detects larger objects. According to Bochkovskiy's 2020 research, this modification enhances the model's performance specifically for medium and small objects in CCTV footage. The standard YOLOv8 model and its modified version were compared for inference time, mean Average Precision (mAP), and model weight. The pruned YOLOv8 model cuts inference time by 15.56%, from 4.5 ms to 3.8 ms, and reduces model weight. The advantages mentioned above are offset by a 1.6% decrease in mean average precision. This research advances object detection technology by demonstrating architectural modifications' efficacy. These changes make the model faster and lighter, making it suitable for real-time surveillance. The accuracy trade-off is slight. The implications of these findings are crucial for implementing efficient object detection systems in CCTV surveillance. These findings also lay the groundwork for future research to improve such systems' speed-accuracy trade-off.

The chapter discusses the design and development of a novel object detection model keeping the motive that anyone and everyone with little to no prior knowledge can easily use it. The object detection is a part of computer vision which deals with detection and localization of objects in images or videos. These systems can be implemented for surveillance, keeping track of goods, robotics, medical field, self-driving cars, and much more. A lot of organizations are using it for various purposes such as it is used by NASA for analysis for deep space analysis, self-driving cars use it for making accurate decisions, and airports use it for security and surveillance. Although it has numerous use cases, but because of complex implementation and need of expertise in the field, it is not being widely used currently. In this chapter, we have made an effort to automate most of the process that a user should do in order to train and use models for object detection. The most complex part of using these technologies is training custom models, and only people having expertise in field are doing it. With the system designed by us, anyone would be able to train custom models on their own and they don’t have to do much for it. Normally, first users have to gather large amount of images and then annotate them 124one by one, which is a tedious and time taking process. One has to do this for thousands of images and each of those images can have multiple instances of various objects to train a model to give a good accuracy. We are automating the whole process with the help of AWS recognition module of AWS. You only look once (YOLO) object detection model is used for automation, as it is fast and accurate at the same time which gives it edge over the other models available for use. This model is based on deep learning to learn through the images provided by user in training phase and then later on can be used to perform detection on images or real-time videos. A graphic user interface (GUI) is also developed, so the users can easily interact with the system and train their own custom models and use as they want it to be used. Data are sent over the AWS servers and retrieve the results using a block of python code, then the response collected from AWS is manipulated and the data are extracted that is required to annotate the images and then processing is done according to the need of our target model. This format can be different for various models which can be used for object detection.

Object detection with deep learning model has achieved good results in many fields, but in some fields that think highly of data privacy, such as medical care, its applications is greatly limited by data. And Federated Learning allows clients to train a model together, while leaving their data in the local, without sharing with the server or other clients. Using the methods of Federated Learning, such as Federated Averaging(FedAvg), to train models can provide privacy, security benefits. Nonetheless, there is little experiment applying Federated Learning algorithms to train the model with a large number of parameters, such as deep learning object detection model. With non-IID data, the accuracy of object detection model trained by FedAvg reduces significantly, and need more rounds to coverage. In this work, we use Kullback-Leibler divergence(KLD) measure the weights divergence between different model trained with non-IID data. And we propose a useful scheme to improve FedAvg based Abnormal Weights Supression, reducing the influence of the weights divergence caused by non-IID and unbalanced data. As a representative of object detection, we choose Single Shot MultiBox Detector(SSD) as the base model. The results of the experiments show that the Mean Average Precision(mAP) get obvious improvement in Pascal VOC 2007 test dataset.

Backdoor Attacks with Wavelet Embedding: Revealing and enhancing the insights of vulnerabilities in visual object detection models on transformers within digital twin systems

Improved YOLOv7 model for underwater sonar image object detection

Artificial intelligence (AI) and edge computing have truly advanced in vehicular networks encouraging assessment of real‐time traffic conditions using kinetic information of autonomous vehicles with the help of road side units (RSUs). However, regardless of numerous improvements in sensor fusion technologies the existing vision/LIDAR‐based systems have found severe difficulties during perception on roads. In addition, the inter‐vehicular communications are hampered due to inefficient RSU placement techniques causing high‐latency issues during transmission of messages. Therefore, this article presents an AI‐driven vision‐based self driver assistance system (VSDAS) comprising a joint RSU deployment mechanism that utilizes enhanced memetic architecture‐based optimal RSU placement (MARP) method and an object detection model that implements an improved Haar‐cascade object detection algorithm for speedy identification of object. We have designed two varieties of genetic algorithm (GA) to solve optimal placement problem of RSUs: genetic architecture‐based with random restart hill climbing (GARRH) and the other is MARP for efficient placement of RSUs. After our experimental results, we see that the MARP algorithm shows best possible RSU locations over different generations achieving significantly better fitness scores than the GAHRC and GA ascribing to its local search process. In addition, Haar‐cascade achieves better mean average precision than local binary pattern and histogram of oriented gradients by selecting key frames. The experimental outcomes of our model reveals that the proposed enhanced memetic algorithm reduces the transmission delay to a greater extent. Additionally, the outcomes of our investigations on two public datasets (KITTI and Panasonic) showed that our improved algorithm clearly enhances the object detection performance.

Wheat is a primary carbohydrate-rich food commodity in the world, and the health of the wheat plants influences its production. Farmers can face a challenge due to decreased productivity caused by infected plants. However, wheat farmers have used conventional methods to survey diseases; however, these methods are often ineffective and inefficient, taking a lot of time and labor. And for enhanced wheat production, new strategies must be employed. Deep learning, specifically computer vision-based techniques, has proven significant capability in tasks like image classification, segmentation, and object detection. Deep learning techniques such as You Only Look Once (YOLO) models are state-of-the-art neural network algorithms used for accurate object detection. this study presents a comparative evaluation of two state-of-the-art object detection models, YOLOv5 and YOLOv8, for disease detection. Data augmentation techniques such as image noise, rotation, and flipping were implemented to improve the model’s performance during the training phase. The model’s performance was evaluated using metrics such as precision, recall, F1-score, and mean Average Precision (mAP).. The results show that the YOLOv5 and YOLOV8 models achieved good performance. They were able to detect the healthy and disease in images, These findings demonstrate that while both models are highly effective, but YOLOv8 offers greater robustness and accuracy for real-time detection.

Sea turtles exhibit high migratory rates and occupy a broad range of habitats, which in turn makes monitoring these taxa challenging. Applying deep learning (DL) models to vast image datasets collected from citizen science programs can offer promising solutions to overcome the challenge of monitoring the wide habitats of wildlife, particularly sea turtles. Among DL models, object detection models, such as the You Only Look Once (YOLO) series, have been extensively employed for wildlife classification. Despite their successful application in this domain, detecting objects in images with complex backgrounds, including underwater environments, remains a significant challenge. Recently, instance segmentation models have been developed to address this issue by providing more accurate classification of complex images compared to traditional object detection models. This study compared the performance of two state-of-the-art DL methods namely; the object detection model (YOLOv5) and instance segmentation model (YOLOv5-seg), to detect and classify sea turtles. The images were collected from iNaturalist and Google and then divided into 64% for training, 16% for validation, and 20% for test sets. Model performance during and after finishing training was evaluated by loss functions and various indexes, respectively. Based on loss functions, YOLOv5-seg demonstrated a lower error rate in detecting rather than classifying sea turtles than the YOLOv5. According to mean Average Precision (mAP) values, which reflect precision and recall, the YOLOv5-seg model showed superior performance than YOLOv5. The mAP0.5 and mAP0.5:0.95 for the YOLOv5 model were 0.885 and 0.795, respectively, whereas for the YOLOv5-seg, these values were 0.918 and 0.831, respectively. In particular, based on the loss functions and classification results, the YOLOv5-seg showed improved performance for detecting rather than classifying sea turtles compared to the YOLOv5. The results of this study may help improve sea turtle monitoring in the future.

Object detection is a challenging task in remote sensing because objects only occupy a few pixels in the images, and the models are required to simultaneously learn object locations and detection. Even though the established approaches well perform for the objects of regular sizes, they achieve weak performance when analyzing small ones or getting stuck in the local minima (e.g., false object parts). Two possible issues stand in their way. First, the existing methods struggle to perform stably on the detection of small objects because of the complicated background. Second, most of the standard methods used handcrafted features and do not work well on the detection of objects parts that are missing. We here address the above issues and propose a new architecture with a multipatch feature pyramid network (MPFP-Net). Different from the current models that, during training, only pursue the most discriminative patches, in MPFP-Net, the patches are divided into class-affiliated subsets, in which the patches are related, and based on the primary loss function, a sequence of smooth loss functions is determined for the subsets to improve the model for collecting small object parts. To enhance the feature representation for patch selection, we introduce an effective method to regularize the residual values and make the fusion transition layers strictly norm-preserving. The network contains bottom-up and crosswise connections to fuse the features of different scales to achieve better accuracy compared to several state-of-the-art object detection models. Also, the developed architecture is more efficient than the baselines.

PSNet: Perspective-sensitive convolutional network for object detection

Recent object detection models show promising advances in their architecture and performance, expanding potential applications for the benefit of persons with blindness or low vision (pBLV). However, object detection models are usually trained on generic data rather than datasets that focus on the needs of pBLV. Hence, for applications that locate objects of interest to pBLV, object detection models need to be trained specifically for this purpose. Informed by prior interviews, questionnaires, and Microsoft's ORBIT research, we identified thirty-five objects pertinent to pBLV. We employed this user-centric feedback to gather images of these objects from the Google Open Images V6 dataset. We subsequently trained a YOLOv5x model with this dataset to recognize these objects of interest. We demonstrate that the model can identify objects that previous generic models could not, such as those related to tasks of daily functioning - e.g., coffee mug, knife, fork, and glass. Crucially, we show that careful pruning of a dataset with severe class imbalances leads to a rapid, noticeable improvement in the overall performance of the model by two-fold, as measured using the mean average precision at the intersection over union thresholds from 0.5 to 0.95 (mAP50-95). Specifically, mAP50-95 improved from 0.14 to 0.36 on the seven least prevalent classes in the training dataset. Overall, we show that careful curation of training data can improve training speed and object detection outcomes. We show clear directions on effectively customizing training data to create models that focus on the desires and needs of pBLV.Clinical Relevance- This work demonstrated the benefits of developing assistive AI technology customized to individual users or the wider BLV community.

Object detection algorithms using convolutional neural networks (CNNs) achieve high detection accuracy, but it is challenging to realize real-time object detection due to their high computational complexity, especially on resource-constrained mobile platforms. In this article, we propose an algorithm-hardware co-optimization approach to designing a real-time object detection system. We first develop a compact object detection model based on a binarized neural network (BNN), which employs a new layer structure, the DenseToRes layer, to mitigate information loss due to deep quantization. We also propose an efficient object detection processor that runs object detection with high throughput using limited hardware rescources. We develop a resource-efficient processing unit supporting variable precision with minimal hardware overheads. Implemented in field-programmable gate array (FPGA), the object detection processor achieves 64.51 frames/s throughput with 64.92 mean average precision (mAP) accuracy. Compared to prior FPGA-based designs for object detection, our design achieves high throughput with competitive accuracy and lower hardware implementation costs.

Object detection is a hot topic with various applications in computer vision, e.g., image understanding, autonomous driving, and video surveillance. Much of the progresses have been driven by the availability of object detection benchmark datasets, including PASCAL VOC, ImageNet, and MS COCO. However, object detection on the drone platform is still a challenging task, due to various factors such as view point change, occlusion, and scales. To narrow the gap between current object detection performance and the real-world requirements, we organized the Vision Meets Drone (VisDrone2018) Object Detection in Image challenge in conjunction with the 15th European Conference on Computer Vision (ECCV 2018). Specifically, we release a large-scale drone-based dataset, including 8, 599 images (6, 471 for training, 548 for validation, and 1, 580 for testing) with rich annotations, including object bounding boxes, object categories, occlusion, truncation ratios, etc. Featuring a diverse real-world scenarios, the dataset was collected using various drone models, in different scenarios (across 14 different cities spanned over thousands of kilometres), and under various weather and lighting conditions. We mainly focus on ten object categories in object detection, i.e., pedestrian, person, car, van, bus, truck, motor, bicycle, awning-tricycle, and tricycle. Some rarely occurring special vehicles (e.g., machineshop truck, forklift truck, and tanker) are ignored in evaluation. The dataset is extremely challenging due to various factors, including large scale and pose variations, occlusion, and clutter background. We present the evaluation protocol of the VisDrone-DET2018 challenge and the comparison results of 38 detectors on the released dataset, which are publicly available on the challenge website: http://www.aiskyeye.com/. We expect the challenge to largely boost the research and development in object detection in images on drone platforms.

Object detection is a fundamental task in computer vision that involves identifying and localizing objects within images or video frames. This research focuses on implementing and evaluating the YOLOv5 (You Only Look Once version 5) model for real-time object detection. YOLOv5 is known for its efficiency, accuracy, and speed, making it a preferred choice for various applications such as autonomous driving, surveillance, and medical imaging. In this study, we explore the architecture, training process, and performance evaluation of YOLOv5 on benchmark datasets. The results demonstrate that YOLOv5 achieves high precision and recall, outperforming traditional object detection methods in terms of speed and accuracy. Keywords: Object Detection, YOLOv5, Deep Learning, Computer Vision, Real-Time Processing, Neural Networks.

Deep neural network approaches have demonstrated high performance in object recognition (CNN) and detection (Faster-RCNN) tasks, but experiments have shown that such architectures are vulnerable to adversarial attacks (FFF, UAP): low amplitude perturbations, barely perceptible by the human eye, can lead to a drastic reduction in labeling performance. This article proposes a new context module, called \textit{Transformer-Encoder Detector Module}, that can be applied to an object detector to (i) improve the labeling of object instances; and (ii) improve the detector's robustness to adversarial attacks. The proposed model achieves higher mAP, F1 scores and AUC average score of up to 13\% compared to the baseline Faster-RCNN detector, and an mAP score 8 points higher on images subjected to FFF or UAP attacks due to the inclusion of both contextual and visual features extracted from scene and encoded into the model. The result demonstrates that a simple ad-hoc context module can improve the reliability of object detectors significantly.