Transformer-Encoder Detector Module: Using Context to Improve Robustness to Adversarial Attacks on Object Detection

Object localization and detection is the emerging area of computer vision and image processing that finds out the location of the object in the video frames or in the digital images. Object localization and detection have many challenges like occlusion, scale variations, intraclass similarities, illumination conditions, pose variations, etc. Multi-view object detection focuses on different views of objects like the top, bottom, side view, etc. We aim to discuss the different Object detection models based on Machine learning and Deep neural network approaches in a multi-view environment. We also discuss different datasets used for object detections and their applications. We apply some object detection models like SSD and Faster R-CNN to specific object categories of Open Image Dataset V6 and compare the results based on mAP (Mean Average Precision).KeywordsObject detectionConvolution neural networksR-CNNFast R-CNNFaster R-CNNEfficientDetCenterNet

As the core technologies represented by Internet of Things (IoT) technology and computer vision technology booming, object detection algorithms based on deep learning have received high attention in various fields. Object detection is not only a core problem in the field of computer vision, but also a prerequisite and basis for many computer vision tasks, and has important applications in the fields of autonomous driving and video surveillance. The task of object detection is to find all the targets of interest in an image and determine their category and location. Object detection includes two tasks: object classification and object localization. However, object classification focuses on the most discriminative part of the feature map, and object localization requires a feature map focusing on the whole region of the object. Therefore, in order to improve the detection accuracy, we propose a CBAM-RetinaNet network model.CBAM (Convolutional Block Attention Module) is a lightweight convolutional attention module, which combines channel and spatial attention mechanism modules. The channel attention module focuses on the meaningful information in the input image, which would lead to an increase in the accuracy of object classification. The spatial attention module focuses on the location information of the object, which may improve the accuracy of object localization. The experiment results show that the proposed achieves better object detection performance on the PASCAL VOC2007 dataset, and the accuracy is 2% higher than that of the RetinaNet model, which has good experimental results.

Object detection is a major branch and fundamental task in computer vision, aiming to localize, identify and classify even the smallest objects of interest in images. Features can be extracted efficiently by deep convolutional neural networks (CNNs) as the backbone for real-time or near real-time object detection performance than the hand-crafted-based traditional methods. In the past few years, the advent of transformer-based models with robust self-attention mechanisms has not only raised object detection performance to a higher level but has also enabled it to produce excellent results. Many object detection tasks in the real world require that 3D information about the object be obtained, thus strengthening active research in 3D object detection. However, the algorithms for detecting 3D objects are not easy to propagate in real-world applications due to many factors, making reconstruction of 2D object detection algorithms to 3D object detection algorithms the suitable alternative. Therefore, we review the evolution of 2D object detection algorithms for digital imaging applications, focusing on their developments, models, applications, datasets, evaluation metrics, strengths and weaknesses, for better understanding of their landmarks and contributions to the advancement of the field.

Using multiple families of image features is a very efficient strategy to improve performance in object detection or recognition. However, such a strategy induces multiple challenges for machine learning methods, both from a computational and a statistical perspective. The main contribution of this paper is a novel feature sampling procedure dubbed "Tasting" to improve the efficiency of Boosting in such a context. Instead of sampling features in a uniform manner, Tasting continuously estimates the expected loss reduction for each family from a limited set of features sampled prior to the learning, and biases the sampling accordingly. We evaluate the performance of this procedure with tens of families of features on four image classification and object detection data-sets. We show that Tasting, which does not require the tuning of any meta-parameter, outperforms systematically variants of uniform sampling and state-of- the-art approaches based on bandit strategies.

Object detection and recognition is an essential task in computer vision with numerous real-world applications such as surveillance, self-driving cars, and robotics. In recent years, deep learning-based approaches have significantly improved the accuracy and speed of object detection and recognition. The You Only Look Once version 3 (YOLOv3) algorithm is a popular deep learning-based approach that can detect and recognize objects in real-time. The Common Objects in Context (COCO) dataset is a large-scale dataset with over 330,000 labeled images and more than 2.5 million object instances, making it a popular choice for object detection and recognition tasks. In this paper, we propose a deep learning-based approach for real-time object detection and recognition using the YOLOv3 architecture and COCO dataset. We evaluate our approach based on several performance metrics, including mean average precision (mAP), frames per second (FPS), total object detection time, object detection accuracy, false positive rate, number of detected objects, and mean intersection over union (mIoU). Our results show that our approach achieves a mean average precision of 0.76 on the COCO dataset and a real-time performance of 40 frames per second on a single GPU. Additionally, our approach achieves an object detection accuracy of 93.5%, a false positive rate of 6.5%, and a mean intersection over union of 0.65. Our proposed approach shows promising results for real-time object detection and recognition and can be applied to various real-world applications.

Addressing the challenge of efficiently detecting objects in ultra-high-resolution images during object detection tasks, this paper proposes a novel method called MegaDetectNet, which leverages foreground image for large-scale resolution image object detection. MegaDetectNet utilizes a foreground extraction network to generate a foreground image that highlights target regions, thus avoiding the computationally intensive process of dividing the image into multiple sub-images for detection, and significantly improving the efficiency of object detection. The foreground extraction network in MegaDetectNet is built upon the YOLOv5 model with modifications: the large object detection head and classifier are removed, and the PConv convolution is introduced to reconstruct the C3 module, thereby accelerating the convolution process and enhancing foreground extraction efficiency. Furthermore, a Res2Rep convolutional structure is developed to enlarge the receptive field and improve the accuracy of foreground extraction. Finally, a foreground image construction method is proposed, fusing and stitching foreground target regions into a unified foreground image. This approach replaces multiple divided sub-images with a single foreground image for detection, reducing overhead time. The proposed MegaDetectNet method’s effectiveness for detecting objects in ultra-high-resolution images is validated using the publicly available DOTA dataset. Experimental results demonstrate that MegaDetectNet achieves an average time reduction of 83.8% compared to the sub-image division method among various commonly used object detectors, with only a marginal 8.7% decrease in mAP (mean Average Precision). This validates the practicality and efficacy of the MegaDetectNet method for object detection in ultra-high-resolution images.

In the modern era, object detection and analysis is one of the crucial tasks. Numerous challenges have to be faced by the researchers to analyze the objects in images and real-time videos. Such analysis is required in multiple domains such as monitoring health, self-driving including detection of an anomaly, and many more. With the advanced usage of deep learning (DL) models and GPUs, the efficacy of object detection has been improved very well. Therefore, in this paper, we have provided a detailed study of both cases (images and videos) to detect specific objects using DL models. The existing approaches have been compared using on global wheat-head detection dataset to check the performance for object detection and recognition at different stages. From this analysis, we can say that DL is a backbone of computer vision to detect and recognize the various objects in a real-time scenario.KeywordsObject detectionGPUDeep learningYOLO

Carbonate rocks are important archives of past ocean conditions as well as hosts of economic resources such as hydrocarbons, water, and minerals. Geologists typically perform compositional analysis of grain, matrix, cement and pore types in order to interpret depositional environments, diagenetic modification, and reservoir quality of carbonate strata. Such information can be obtained primarily from petrographic analysis, a task that is costly, labor-intensive, and requires in-depth knowledge of carbonate petrology and micropaleontology. Recent studies have leveraged machine learning-based image analysis, including Deep Convolutional Neural Networks (DCNN), to automate description, classification and interpretation of thin sections, subsurface core images and seismic facies, which would accelerate data acquisition and reproducibility for these tasks. In carbonate rocks, this approach has been applied primarily to recognize carbonate lithofacies, and no attempt has been made to individually identify and quantify various types of carbonate grains, matrix, and cement. In this study, the applicability and performance of DCNN-based object detection and image classification approaches are assessed with respect to carbonate compositional analysis. The training data comprised of more than 13,000 individually labelled objects from nearly 4000 carbonate petrographic images. The dataset is grouped into six and nine different classes for the image classification and object detection tasks, respectively. Even with a small and relatively imbalanced training set, the DCNN was able to achieve an F1 score of 92% for image classification and 84% mean precision for object detection by combining one-cycle policy, class weight, and label mixup-smoothing methods. This study highlights the inefficiency of image classification as an approach to replicating human description and classification of carbonate petrography. By contrast, DCNN-based object detection appears capable of approaching human speed and accuracy in the area of carbonate petrography because it is able to individually locate and identify different carbonate components with greater cost-efficiency, speed, and reproducibility than conventional (human) petrographic analysis.

Convolutional Neural Networks (CNNs) have achieved significant success in image classification and object detection. CNN models generally consist of a single-stream and process single image data at once. In addition, multi-stream (or multi-modal) models have recently begun to be proposed that allow the processing of more than one input at the same time. The data can be an image, video, voice, or any other sensor data. Multi-modality may help us extract some hidden features of the same object. Furthermore, several new studies examine sharing feature maps between different streams of the same CNN. However, systematic studies that can adequately demonstrate the contribution of multi-modality and feature map sharing features to performance have not yet been conducted. Processing power and lack of available datasets are among the important factors that negatively affect progress. In this study, the contributions of multi-modality and feature map sharing (FMS) to increase the performance in object recognition are examined in detail. For this purpose, a new dataset and a new multi-modal multi-feature map sharing CNN model, which we call FMSNet, are developed. The proposed model achieved a 3.06% higher accuracy rate than its non-FMS counterpart, DenseNet-201, exceeding most of the state-of-the-art single-stream CNN models.

Deep convolutional neural network (DCNN) based computer vision methods have great progress in object detection tasks. And object detection is essential to many applications such as autonomous robots and vehicles. However, it is difficult for object detection system to be deployed on embedded platforms due to its intensive computing architecture. We present an object detector MobileNet-SSD which can be deployed on embedded platforms. The object detector presented in this work based on Single Shot Detector (SSD) framework and replace the feature extractor with a more light weight network MobileNetV1. General matrix to matrix multiplication is used to simplify the calculation of convolution operation and is further optimized using ARM NEON technology. Finally, the object detector MobileNet-SSD is deployed on embedded platform and achieves 1.13FPS with 0.72 mean average precision on PASCAL VOC dataset.

Generally, most approaches using methods such as cropping, rotating, and flipping achieve more data to train models for improving the accuracy of detection and segmentation. However, due to the difficulties of labeling such data especially semantic segmentation data, those traditional data augmentation methodologies cannot help a lot when the training set is really limited. In this paper, a model named OFA-Net (One For All Network) is proposed to combine object detection and semantic segmentation tasks. Meanwhile, using a strategy called “1-N Alternation” to train the OFA-Net model, which can make a fusion of features from detection and segmentation data. The results show that object detection data can be recruited to better the segmentation accuracy performance, and furthermore, segmentation data assist a lot to enhance the confidence of predictions for object detection. Finally, the OFA-Net model is trained without traditional data augmentation methodologies and tested on the KITTI test server. The model works well on the KITTI Road Segmentation challenge and can do a good job on the object detection task.

This paper presents a Cosine Similarity-Based Knowledge Distillation (CSKD) for robust, lightweight object detectors. Knowledge Distillation (KD) has been effective in enhancing the performance of compact models in image classification by leveraging deep CNN models. However, the complex and multifaceted nature of object detection, characterized by its modular design and multitasking requirements, poses significant challenges for traditional KD techniques. These challenges are further compounded by the conventional reliance on the Mean Squared Error (MSE) loss function and the limited application of enhanced feature representations to the training phase. Addressing these limitations, the proposed CSKD method combines cosine similarity guidance with MSE loss to facilitate a more effective knowledge transfer from the teacher model to the student model. This is achieved by distilling both intermediate features and prediction outputs, aided by an assistant prediction branch designed to learn directly from the teacher’s predictions. This dual-faceted distillation strategy enables the student model to better mimic the teacher model’s behavior, leading to improved performance. The proposed method demonstrates versatility and robustness across various object detector architectures without the need for additional feature enhancement layers during training. Notably, employing ResNet-50 as the teacher model and ResNet-18 as the student model, we achieve new benchmarks in KD for object detection across several architectures, including Faster-RCNN, RetinaNet, FCOS, and GFL, with respective mAP scores of 36.6, 35.2, 35.9, and 38.9. These results highlights the effectiveness of CSKD in advancing the state-of-the-art in KD for object detection, offering a compelling solution to the challenges previously faced by traditional KD methods in this domain. The code of the proposed CSKD is available at https://github.com/swkdn16/CSKD.

Efficient object detection based on selective attention

In recent years, Convolutional Neural Networks (CNNs) have repeatedly shown state-of-the-art performance for their accuracy in the task of object detection, but their heavy computational costs impede their ability for real-time detection when the supporting system is moving, particulary when it is accelerating. At the same time, recent progress on visual inertial systems takes great advantage of movement information to robustly estimate the robot state and its surrounding. This paper proposes to exploit the advantages of inertial odometry research for the purpose of real-time object detection system on mobile robots. We combine a CNN detector with VINS-Mono, a moving visual odometry system, and show reliable improvement in the detection process, especially when the robot accelerates or decelerates. Our system is ready-to-use in that it has very low deployment cost and requires no calibration. The resulting system allows for simultaneous robot state estimation and object detection, as well as object tracking. Lastly, this architecture proves to be flexible because not restrained to a specific object type or detector.

Object detection is an essential function for mobile robots, allowing them to carry out missions efficiently. In recent years, various deep learning models based on convolutional neural networks have achieved good performance in object detection. However, in cases in which robots have to carry out missions in a particular environment, utilizing a model that has been trained without considering the environment in which robots must conduct their tasks degrades their object detection performance, leading to failed missions. This poor model accuracy occurs because of the class imbalance problem, in which the occurrence frequencies of the object classes in the training dataset are significantly different. In this study, we propose a systematic solution that can solve the class imbalance problem by training multiple object detection models and using these models effectively for robots that move through various environments to carry out missions. Moreover, we show through experiments that the proposed multi-model-based object detection framework with environment-context awareness can effectively overcome the class imbalance problem. As a result of the experiment, CPU usage decreased by 45.49% and latency decreased by more than 60%, while object detection accuracy increased by 6.6% on average.