Improving an Object Detector and Extracting Regions Using Superpixels

An object detector based on convolutional neural network (CNN) has been widely used in the field of computer vision because of its simplicity and efficiency. The average accuracy of CNN model detection results in the object detector is greatly affected by the loss function. The precision of the localization algorithm in the loss function is the main factor affecting the result. Based on the complete intersection over union (CIoU) loss function, an improved penalty function is proposed to improve the localization accuracy. Specifically, the algorithm more comprehensively considers matching bounding boxes between prediction with ground truth, using the proportional relationship of the aspect ratio from both bounding boxes. Under the same aspect ratio of the two bounding boxes, the influence factors of the prediction box on localization accuracy were considered. In this way, the function of the penalty function is strengthened, and localization accuracy of the network model improved. This loss function is called Improved CIoU (ICIoU). Experiments on the Udacity, PASCAL VOC, and MS COCO datasets have demonstrated the effectiveness of ICIoU in improving localization accuracy of network models by using the one-stage object detector YOLOv4. Compared with CIoU, the proposed ICIoU improved average precision (AP) by 0.57% and AP75 by 0.12% on Udacity, AP by 0.26% and AP75 by 1.28% on PASCAL VOC, and AP by 0.06% and AP75 by 0.65% on MS COCO.

No matter how well-known the colleges are, there will always be concerns that people have during the application process and even after they have been accepted. The college hosts a variety of events, ranging from departmental activities to club activities. Not everyone is likely aware of all events. The chatbot bridges the gap between people and information. The world is becoming more automated, and people expect services to become more automated as well. Using a chatbot is the best approach to serve because, unlike a human, they are accessible 24 hours a day, seven days a week. A chatbot is a software that responds to user questions and provides information from a knowledge base. The purpose of this project is to create a chatbot for VNRVJIET that will answer queries raised about fests, departmental activities, events, clubs, infrastructure, placement data, admission procedure, and other topics. The proposed methodology consists of a chatbot built using Deep Neural Networks and speech recognition capabilities. The information is delivered in both speech and text modes using the proposed methodology. Data is collected and formatted in JSON format initially. The prepared data is pre-processed, and then the bag of words (BoW) algorithm is applied to it. The bag of words algorithm is most influential method for object categorization. The key aspect of using this algorithm is for converting the word vector to a numerical dataset for machine to do a deeper analysis. A deep neural network is created using tensorflow API, and the speech recognition function is de-fined for the input query and output response. Finally, the chatbot function is defined and utilized for generating responses for any given query.

Due to the reasonably acceptable performance of state-of-the-art object detectors, tracking-by-detection is a standard strategy for visual multi-object tracking (MOT). In particular, online MOT is more demanding due to its diverse applications in time-critical situations. A main issue of realizing online MOT is how to associate noisy object detection results on a new frame with previously being tracked objects. In this work, we propose a multi-object tracker method called CRF-boosting which utilizes a hybrid data association method based on online hybrid boosting facilitated by a conditional random field (CRF) for establishing online MOT. For data association, learned CRF is used to generate reliable low-level tracklets and then these are used as the input of the hybrid boosting. To do so, while existing data association methods based on boosting algorithms have the necessity of training data having ground truth information to improve robustness, CRF-boosting ensures sufficient robustness without such information due to the synergetic cascaded learning procedure. Further, a hierarchical feature association framework is adopted to further improve MOT accuracy. From experimental results on public datasets, we could conclude that the benefit of proposed hybrid approach compared to the other competitive MOT systems is noticeable.

The labeling cost of large number of bounding boxes is one of the main challenges for training modern object detectors. To reduce the dependence on expensive bounding box annotations, we propose a new semi-supervised object detection formulation, in which a few seed box level annotations and a large scale of image level annotations are used to train the detector. We adopt a training-mining framework, which is widely used in weakly supervised object detection tasks. However, the mining process inherently introduces various kinds of labelling noises: false negatives, false positives and inaccurate boundaries, which can be harmful for training the standard object detectors (e.g. Faster RCNN). We propose a novel NOise Tolerant Ensemble RCNN (NOTE-RCNN) object detector to handle such noisy labels. Comparing to standard Faster RCNN, it contains three highlights: an ensemble of two classification heads and a distillation head to avoid overfitting on noisy labels and improve the mining precision, masking the negative sample loss in box predictor to avoid the harm of false negative labels, and training box regression head only on seed annotations to eliminate the harm from inaccurate boundaries of mined bounding boxes. We evaluate the methods on ILSVRC 2013 and MSCOCO 2017 dataset; we observe that the detection accuracy consistently improves as we iterate between mining and training steps, and state-of-the-art performance is achieved.

We present an approach MSIL-CRF that incorporates multiple instance learning (MIL) into conditional random fields (CRFs). It can generalize CRFs to work on training data with uncertain labels by the principle of MIL. In this work, it is applied to saving manual efforts on annotating training data for semantic segmentation. Specifically, we consider the setting in which the training dataset for semantic segmentation is a mixture of a few object segments and an abundant set of objects' bounding boxes. Our goal is to infer the unknown object segments enclosed by the bounding boxes so that they can serve as training data for semantic segmentation. To this end, we generate multiple segment hypotheses for each bounding box with the assumption that at least one hypothesis is close to the ground truth. By treating a bounding box as a bag with its segment hypotheses as structured instances, MSIL-CRF selects the most likely segment hypotheses by leveraging the knowledge derived from both the labeled and uncertain training data. The experimental results on the Pascal VOC segmentation task demonstrate that MSIL-CRF can provide effective alternatives to manually labeled segments for semantic segmentation.

본 논문에서는 객체 검출에 사용되는 Haar-like feature의 정규화 방법에 대해 다룬다. 기존의 Haar-like feature의 분산 정규화는 후보 윈도우 픽셀들에 대한 표준편차 계산에 사용되는 별도의 적분 영상 생성을 위해 많은 연산을 필요로 했으며 밝기 변화가 작은 영역에서 오검출이 증가하는 문제를 가지고 있으나, 제안하는 정규화 방법은 별도의 적분 영상을 사용하지 않아 처리 속도가 빠르며 제안하는 방법을 사용하여 학습시킨 분류기는 밝기 변화에 대해 강건한 성능을 보인다. 실험 결과 제안한 방법을 사용했을 때 객체 검출기의 처리 속도는 26% 향상 되었으며, 제안한 방법을 사용하여 학습시킨 분류기들은 5% 이상 향상된 검출률을 보였으며, 밝기 변화가 심한 경우는 45% 향상된 검출률을 보였다. This paper describes a normalization method of Haar-like features used for object detection. Previous method which performs variance normalization on Haar-like features requires a lot of calculations, since it uses an additional integral image for calculating the standard deviation of intensities of pixels in a candidate window and increases possibility of false detection in the area where variance of brightness is small. The proposed normalization method can be performed much faster than the previous method by not using additional integral image and classifiers which are trained with the proposed normalization method show robust performance in various lighting conditions. Experimental result shows that the object detector which uses the proposed method is 26% faster than the one which uses the previous method. Detection rate is also improved by 5% without increasing false alarm rate and 45% for the samples whose brightness varies significantly.

Automatic detection of weapons is significant for improving security and well being of individuals, nonetheless, it is a difficult task due to large variety of size, shape and appearance of weapons. View point variations and occlusion also are reasons which makes this task more difficult. Further, the current object detection algorithms process rectangular areas, however a slender and long rifle may really cover just a little portion of area and the rest may contain unessential details. To overcome these problem, we propose a CNN architecture for Orientation Aware Weapons Detection, which provides oriented bounding box with improved weapons detection performance. The proposed model provides orientation not only using angle as classification problem by dividing angle into eight classes but also angle as regression problem. For training our model for weapon detection a new dataset comprising of total 7801 weapons images is gathered from the web and then manually annotated with position oriented bounding boxes. Our dataset provides not only oriented bounding box as ground truth but also horizontal bounding box. We also provide our dataset in multiple formats of modern object detectors for further research in this area. The proposed model is evaluated on this dataset, and the comparative analysis with off-the shelf object detectors yields superior performance of proposed model, measured with standard evaluation strategies. The dataset and the model implementation are made publicly available at this link: https://bit.ly/2TyZICF .

In this paper we give a convex optimization approach for scene understanding. Since segmentation, object recognition and scene labeling strongly benefit from each other we propose to solve these tasks within a single convex optimization problem. In contrast to previous approaches we do not rely on pre-processing techniques such as object detectors or super pixels. The central idea is to integrate a hierarchical label prior and a set of convex constraints into the segmentation approach, which combine the three tasks by introducing high-level scene information. Instead of learning label co-occurrences from limited benchmark training data, the hierarchical prior comes naturally with the way humans see their surroundings.

We formulate a model for multi-class object detection in a multi-camera environment. From our knowledge, this is the first time that this problem is addressed taken into account different object classes simultaneously. Given several images of the scene taken from different angles, our system estimates the ground plane location of the objects from the output of several object detectors applied at each viewpoint. We cast the problem as an energy minimization modeled with a Conditional Random Field (CRF). Instead of predicting the presence of an object at each image location independently, we simultaneously predict the labeling of the entire scene. Our CRF is able to take into account occlusions between objects and contextual constraints among them. We propose an effective iterative strategy that renders tractable the underlying optimization problem, and learn the parameters of the model with the max-margin paradigm. We evaluate the performance of our model on several challenging multi-camera pedestrian detection datasets namely PETS 2009 and EPFL terrace sequence. We also introduce a new dataset in which multiple classes of objects appear simultaneously in the scene. It is here where we show that our method effectively handles occlusions in the multi-class case.

In recent years, with the rapid development of artificial intelligence, simultaneous localization and mapping technology to solve robot perception problems has been widely used in automatic driving, robot navigation, augmented reality and other fields. In this paper, a method of reconstructing large outdoor 3D dense semantic map based on monocular vision is proposed. Firstly, the motion trajectory of the camera is estimated by visual odometer, and the depth data is obtained by monocular depth estimation algorithm. Meanwhile, a dense conditional random field (CRF) image segmentation system based on deep learning and super-pixel distribution is used for semantic segmentation. Then 2D image semantics are gradually transferred to 3D point clouds by Bayesian progressive label migration strategy and further optimize 3D labels through a novel CRF model. Super pixels are utilized to enforce smoothness and form robust PN high order potential. Finally, dense 3D semantic maps of urban environment is generated by arbitrary length image sequence. The algorithm is tested on KITTI dataset, and the constructed semantic map shows that the proposed algorithm can reconstruct globally consistent semantic map in large-scale outdoor scenes.

Human pose estimation (HPE) is a field focused on estimating human poses by detecting key points in images. HPE includes methods like top-down and bottom-up approaches. The top-down approach uses a two-stage process, first locating and then detecting key points on humans with bounding boxes, whereas the bottom-up approach directly detects individual key points and integrates them to estimate the overall pose. In this article, we address the problem of bounding box detection inaccuracies in certain situations using the top-down method. The detected bounding boxes, which serve as input for the model, impact the accuracy of pose estimation. Occlusions occur when a part of the target's body is obscured by a person or object and hinder the model's ability to detect complete bounding boxes. Consequently, the model produces bounding boxes that do not recognize occluded parts, resulting in their exclusion from the input used by the HPE model. To mitigate this issue, we introduce the Restoring Occluded Mask Image for 2D Human Pose Estimation (ROM-Pose), comprising a restoration model and an HPE model. The restoration model is designed to delineate the boundary between the target's grayscale mask (occluded image) and the blocker's grayscale mask (occludee image) using the specially created Whole Common Objects in Context (COCO) dataset. Upon identifying the boundary, the restoration model restores the occluded image. This restored image is subsequently overlaid onto the RGB image for use in the HPE model. By integrating occluded parts' information into the input, the bounding box includes these areas during detection, thus enhancing the HPE model's ability to recognize them. ROM-Pose achieved a 1.6% improvement in average precision (AP) compared to the baseline.

Recently, advanced ground-to-air missiles are developed to defend against a targeted aircraft. A conventional ground-to-air missile has infrared (IR) seekers or radar sensors. However, an anti-aircraft missile based on infrared images is easily distracted by flares, thus an aircraft deploys flares when an incoming missile is detected. In this paper, we propose a missile seeker that complements the weakness of the existing IR sensors, by switching to camera sensors in the case where IR sensors are disturbed. As we switch to camera sensors, we generate the initial bounding box by applying simple line detection algorithms on the target image. Once a bounding box is generated at the target aircraft, the box is tracked continuously using object trackers, which are based on camera images. The missile then uses the bounding box for tracking the target continuously. The object tracker is based on the camera image and is not disturbed by flares or chaffs. To the best of our knowledge, our paper is unique in proposing a missile seeker that complements the weakness of the existing IR sensors, by switching to camera sensors in the case where IR sensors are disturbed. The proposed tracking algorithm is simple and computationally efficient, thus is suitable for real time applications. As far as we know, our tracking algorithm is novel in using simple line detection algorithms for tracking a target aircraft. Using experiments, we show that the proposed tracking method outperforms the-state-of-the-art on object detector considering both time efficiency and tracking accuracy.

Natural Language Processing (NLP) is a field of computational linguistics related to interactions between computers and human languages. Parsing of input text of any human language is a part of NLP. This parsing technique requires processing of text on a word by word basis. To process any individual word especially in Sanskrit and all Dravidian languages, Sandhi splitting is a major task. Sandhi is also called Morphophonemics concerned with changes that occur when two words or separate morphemes come together to form a new word. Exact splitting point is essential for text processing tasks such as POS tagging and in turn parsing. We have adopted a novel approach to internal Sandhi splitting technique on Kannada language. Each Kannada word is split into morphemes according to valid morph patterns. After the division of each word into lexical morphemes, we have manually tagged each split word into root-begins, root-continuous and suffix. This work has been done on Kannada language. We have trained the system with a list of 1000 tagged words using a CRF (Conditional Random Fields) tool and nearly 400 raw split words (untagged words) are given as input to the CRF tool. The system generates a list of tagged split words for the given input according to the trained data. The system output has been compared with the manually tagged data. We have verified the data using 5 fold test, which takes five different combinations of trained data (1000 words) and test data (400 words). The average Precision, Recall and F-Measure of Tagging accuracy of CRF model for Kannada corpus in 5 fold test are nearly equal to 98.08, 92.91 and 95.43. This method can be successfully implemented in all other Dravidian languages for the Sandhi splitting.

Grounding DINO (GDINO) has strong potential for use in zero-shot detection and data annotation, but its use is limited by high computational costs. In addition, YOLOX allows real-time detection but struggles to perform well in complex scenes. To address this challenge, we propose an edge-cloud collaborative framework for an Advanced Driver Assistance System (ADAS) to enhance real-time detector performance on edge devices by leveraging the robust capabilities of cloud-based multimodal detectors to improve perception in complex environments. Our framework consists of cloud and edge components: on the cloud side, we propose a distillation method for multimodal object detectors, which is referred to as MMKD, to optimize the performance of GDINO. Specifically, we use a two-stage distillation strategy, including Cross-modal Listwise Distillation (CLD) and Risk-focused Pseudo-label Distillation (RPLD). With MMKD, we successfully deploy the GDINO model to the cloud, achieving a 1.4% improvement in average precision (AP) and a 1.7× increase in inference speed. On the edge side, leveraging this streamlined version of GDINO, we propose an ADAS data engine to construct a 1.5 Million-scale GDINO-based Dataset for ADAS, named GDDA1.5M. Impressively, on the basis of YOLOX-Lite, we develop a lightweight object detector that is optimized for the application of an ADAS on edge devices through pruning and architectural refinements. Leveraging the GDDA1.5M dataset and the RPLD training strategy, the model achieves a 7.5% improvement in AP, substantially surpassing its counterparts that were trained on 300K manually labeled images. After the YOLOX-Lite detector is deployed on edge devices within our proposed edge-cloud collaborative framework, it achieves an inference speed of 18 milliseconds on the Horizon X3E chip, while the cloud-based distilled model functions efficiently in complex environments.

Recently, fully convolutional network (FCN) has been successfully used to locate spliced regions in synthesized images. However, all the existing FCN-based algorithms use real-valued FCN to process each channel separately. As a consequence, they fail to capture the inherent correlation between color channels and the integrity of three channels. So, in this paper, quaternion fully convolutional network (QFCN) is proposed to generalize FCN to quaternion domain by replacing real-valued conventional blocks in FCN with quaternion conventional blocks. In addition, a new color image splicing localization algorithm is proposed by combining QFCNs and superpixel (SP)-enhanced pairwise conditional random field (CRF). QFCNs consider three different versions (QFCN32, QFCN16, and QFCN8) with different up-sampling layers. The SP-enhanced pairwise CRF is used to refine the results of QFCNs. Experimental results on three publicly available datasets demonstrate that the proposed algorithm outperforms the existing algorithms including some conventional algorithms and some deep learning-based algorithms.