DeepHybridNet: A Novel CNN-Transformer Architecture for Robust Object Recognition and Categorization

Object detection is a hot topic with various applications in computer vision, e.g., image understanding, autonomous driving, and video surveillance. Much of the progresses have been driven by the availability of object detection benchmark datasets, including PASCAL VOC, ImageNet, and MS COCO. However, object detection on the drone platform is still a challenging task, due to various factors such as view point change, occlusion, and scales. To narrow the gap between current object detection performance and the real-world requirements, we organized the Vision Meets Drone (VisDrone2018) Object Detection in Image challenge in conjunction with the 15th European Conference on Computer Vision (ECCV 2018). Specifically, we release a large-scale drone-based dataset, including 8, 599 images (6, 471 for training, 548 for validation, and 1, 580 for testing) with rich annotations, including object bounding boxes, object categories, occlusion, truncation ratios, etc. Featuring a diverse real-world scenarios, the dataset was collected using various drone models, in different scenarios (across 14 different cities spanned over thousands of kilometres), and under various weather and lighting conditions. We mainly focus on ten object categories in object detection, i.e., pedestrian, person, car, van, bus, truck, motor, bicycle, awning-tricycle, and tricycle. Some rarely occurring special vehicles (e.g., machineshop truck, forklift truck, and tanker) are ignored in evaluation. The dataset is extremely challenging due to various factors, including large scale and pose variations, occlusion, and clutter background. We present the evaluation protocol of the VisDrone-DET2018 challenge and the comparison results of 38 detectors on the released dataset, which are publicly available on the challenge website: http://www.aiskyeye.com/. We expect the challenge to largely boost the research and development in object detection in images on drone platforms.

Efficient and adaptive computer vision systems have been proposed to make computer vision tasks, such as image classification and object detection, optimized for embedded or mobile devices. These solutions, quite recent in their origin, focus on optimizing the model (a deep neural network) or the system by designing an adaptive system with approximation knobs. Despite several recent efforts, we show that existing solutions suffer from two major drawbacks. First , while mobile devices or systems-on-chips usually come with limited resources including battery power, most systems do not consider the energy consumption of the models during inference. Second , they do not consider the interplay between the three metrics of interest in their configurations, namely, latency, accuracy, and energy. In this work, we propose an efficient and adaptive video object detection system— Virtuoso , which is jointly optimized for accuracy, energy efficiency, and latency. Underlying Virtuoso is a multi-branch execution kernel that is capable of running at different operating points in the accuracy-energy-latency axes, and a lightweight runtime scheduler to select the best fit execution branch to satisfy the user requirement. We position this work as a first step in understanding the suitability of various object detection kernels on embedded boards in the accuracy-latency-energy axes, opening the door for further development in solutions customized to embedded systems and for benchmarking such solutions. Virtuoso is able to achieve up to 286 FPS on the NVIDIA Jetson AGX Xavier board, which is up to 45× faster than the baseline EfficientDet D3 and 15× faster than the baseline EfficientDet D0. In addition, we also observe up to 97.2% energy reduction using Virtuoso compared to the baseline YOLO (v3)—a widely used object detector designed for mobiles. To fairly compare with Virtuoso , we benchmark 15 state-of-the-art or widely used protocols, including Faster R-CNN (FRCNN) [NeurIPS’15], YOLO v3 [CVPR’16], SSD [ECCV’16], EfficientDet [CVPR’20], SELSA [ICCV’19], MEGA [CVPR’20], REPP [IROS’20], FastAdapt [EMDL’21], and our in-house adaptive variants of FRCNN+, YOLO+, SSD+, and EfficientDet+ (our variants have enhanced efficiency for mobiles). With this comprehensive benchmark, Virtuoso has shown superiority to all the above protocols, leading the accuracy frontier at every efficiency level on NVIDIA Jetson mobile GPUs. Specifically, Virtuoso has achieved an accuracy of 63.9%, which is more than 10% higher than some of the popular object detection models, FRCNN at 51.1% and YOLO at 49.5%.

Object detection has become the cornerstone of various real-world applications, ranging from autonomous driving where it helps detect road objects to medical imaging for early disease prediction and gesture recognition systems. Due to their ability to operate continuously and cost-effectively, computer vision models are increasingly being employed in surveillance and monitoring tasks. Unlike humans, who can intuitively recognize and understand objects within images, machines require advanced algorithms to mimic such capabilities. A key challenge in computer vision lies in recognizing and tracking objects in real-time with accuracy and reliability. Human visual systems perform these tasks naturally and swiftly, enabling complex activities like driving. For computers to match such performance, efficient object detection models along with additional hardware such as sensors are essential. This study presents a comparative analysis of two popular deep learning-based object detection algorithms You Only Look Once (YOLO) and Faster Region-based Convolution Neural Network (Faster RCNN). Both models are evaluated based on their detection accuracy, speed, and performance in real-time scenarios. The paper aims to highlight the strengths and limitations of each model, offering insights into their suitability for different applications. The findings suggest that while both algorithms have their merits, the choice between them depends on the specific requirements of a task, such as the trade-off between detection speed and precision. Comparative study shows that Faster RCNN outperforms its accuracy compared with YOLO.

Tomato is one of the cultivations that is often used for gardening. Tomato also has high demands on the market as it’s used for daily needs and occurred for many cuisines. Tomato comes in several colors such as red, orange, and green. Their color could tell their maturity levels too. Tomato grows in several quantities even only on one branch. So as the technologies grow, the computer also could be trained to understand what tomato is and how does it look like. Using computer vision, the computer could tell tomatoes according to their color. For this study, the computer will be trained using Faster R-CNN models to recognize the tomato maturity as Faster R-CNN known support for the image classification and object detection. The accuracy for classification in validation stage about 98,70% in average. For the object detection the model has confidentiality about 96,20% to detect the tomato maturity.

Detecting objects from images is a challenging problem in the domain of computer vision and plays a very crucial role for wide range of real-time applications. The ever-increasing growth of deep learning due to availability of large training data and powerful GPUs helped computer vision community to build commercial products and services which were not possible a decade ago. Deep learning architectures especially convolutional neural networks have achieved state-of-the-art performance on worldwide competitions for visual recognition like ILSVRC, PASCAL VOC. Deep learning techniques alleviate the need of human expertise from designing the handcrafted features and automatically learn the features. This resulted into use of deep architectures in many domains like computer vision (image classification, visual recognition) and natural language processing (language modeling, speech recognition). Object detection is one such promising area immensely needed to be used in automated applications like self-driving cars, robotics, drone image analysis. This paper analytically reviews state-of-the-art deep learning techniques based on convolutional neural networks for object detection.

Computer vision (CV) and human-computer interaction (HCI) are essential in many technological fields. Researchers in CV are particularly interested in real-time object detection techniques, which have a wide range of applications, including inspection systems. In this study, we design and implement real-time object detection and recognition systems using the single-shoot detector (SSD) algorithm and deep learning techniques with pre-trained models. The system can detect static and moving objects in real-time and recognize the object's class. The primary goals of this research were to investigate and develop a real-time object detection system that employs deep learning and neural systems for real-time object detection and recognition. In addition, we evaluated the free available, pre-trained models with the SSD algorithm on various types of datasets to determine which models have high accuracy and speed when detecting an object. Moreover, the system is required to be operational on reasonable equipment. We tried and evaluated several deep learning structures and techniques during the coding procedure and developed and proposed a highly accurate and efficient object detection system. This system utilizes freely available datasets such as MS Common Objects in Context (COCO), PASCAL VOC, and Kitti. We evaluated our system's accuracy using various metrics such as precision and recall. The proposed system achieved a high accuracy of 97% while detecting and recognizing real-time objects.

Traffic violation apprehension is one of the traffic problems here in the Philippines. One example is the No Helmet No Ride Law that is implemented but many motorists still choose to ignore. To alleviate the problem the government has offered many solutions, one of which is the No Contact Traffic Apprehension Policy that uses CCTV Monitoring. To further enhance this solution the government has partnered with the De La Salle University to use artificial intelligence in the system. Computer Vision tasks like image classification and object detection can help automate the traffic apprehension system. Image classification and object detection are technologies which are used in computer vision in defining an image or coordinates of an object in an image. In this work, a novel approach to classifying motorcycle riders between wearing a helmet or not will be developed. It will be demonstrated using deep machine learning, specifically convolutional neural network and by utilizing different pre-trained models to a gathered dataset.

Deep learning models have revealed outstanding performance on image classification and object detection tasks. However, there is a crucial drop in performance when they are subject to learn new data incrementally in the absence of previous training data. They suffer from catastrophic forgetting—abrupt drop in performance. This phenomenon affects the implementation of artificial intelligence in practical scenarios. To overcome catastrophic forgetting, the previous method has either saved previous data in memory or generated the previous data. However, these methods are computationally complex and infeasible for real-time applications. In this article, we proposed the YOLOv3 as an object detection framework for incremental learning. A knowledge distillation loss is introduced for the prediction of previously learned knowledge without utilizing previous training data. Consequently, these predictions are updated while learning the current model. Experimental results on the Pascal VOC2007 indicate that the proposed method significantly improved the mean average precision up to 74% for two classes in comparison to the state-of-the-art methods.

Object detection, which is one of the most fundamental visual recognition tasks, has been a hotspot in computer vision. CNN (Convolutional Neural Networks) have been widely employed for building detector. Due to the success of RPN (Region Proposal Network), the two-stage detectors get both classification accuracy and precise regression bounding boxes. However, they still struggle in small-size object detection. In this paper, we present a deep network, namely Spatial Fine-Grained Network (SFGN). The SFGN that exploits Spatial Fine-Grained Features (SFGF) concatenates the higher resolution features, which is fine-grained with the low resolution features and high-level semantic by stacking spatial features for fine-grained features. An enhanced region proposal generator is proposed to get the objectless for small object to obtain a small set of proposal. The contextual information surrounding the region of interest is embedded using local spatial information for increasing the useful information and discriminating the background. For improving the detection performance, we use a simple yet surprisingly effective online hard example mining (OHEM) algorithm for training region proposal generator. It embeds an efficiently implemented soft non-maximum suppression (soft-NMS) for replacing with tradition NMS to obtain consistent improvements without increasing the computational complexity in inference. On PASCAL VOC 2007 and PASCAL VOC 2012 datasets, our SFGN improves baseline model from 81.2% mAP to 80.6% mAP. On MS COCO dataset, SFGN also performs better than baseline model. As intuition suggests, our detection results provide strong evidence that our SFGN improves detection accuracy, especially in small object test.

Review on self-supervised image recognition using deep neural networks

LPCL: Localized prominence contrastive learning for self-supervised dense visual pre-training

Object region proposal has a wide application on various domains, such as object detection, object tracking, robot navigation, and anomaly detection. Widely used region proposal methods are based on either grouping superpixels or sliding windows. Previous studies have been done by grouping either convolution features, superpixels, or windows. Moreover, histogram of gradients (HoGs) of lower-level features (pixels) can be used for the region proposal task. In this study, both the concepts of HoGs and convolution features are used for proposing object(s) region(s) in an image. Region proposal is done by HoGs of convolution features and statistical application on the gradient data of different feature maps. First, a sized image is passed through one convolution layer to generate reduced feature maps. Then, a window slides over each feature map and HoG is applied over each window. Thereafter, the histogram is analysed using the concept of Coefficient of Variation (CV) and angle (slope) applied on histogram data values. A window is considered as the object region if the CV and angle of its histogram data exceed some thresholds. For all feature maps, we have obtained several widows that represent approximated object regions. These windows are grouped to obtain the possible region(s) proposal. The effectiveness of the proposed method is demonstrated over one of the benchmark datasets, called `PASCAL VOC'. Our method is found superior to two other state-of-the-art, namely region-based segmentation (RBS), and combining efficient object localization and image classification (CEOLIC) in terms of recall.

Multi-level refinement enriched feature pyramid network for object detection

Complex background classification network: A deep learning method for urban images classification

This research explores the fusion of computer vision and agricultural quality control. It investigates the efficacy of computer vision algorithms, particularly in image classification and object detection, for non-destructive assessment. These algorithms offer objective, rapid, and error-resistant analysis compared to human inspection. The study provides an extensive overview of using computer vision to evaluate grain and fertilizer granule quality, highlighting granule size's significance. It assesses prevailing object detection methods, outlining their advantages and drawbacks. The paper identifies the prevailing trend of framing quality inspection as an image classification challenge and suggests future research directions. These involve exploring object detection, image segmentation, or hybrid models to enhance fertilizer granule quality assessment.