A survey on class-agnostic counting: Advancements from reference-based to open-world text-guided approaches

The impressive performance of deep convolutional neural networks in single-view 3D reconstruction suggests that these models perform non-trivial reasoning about the 3D structure of the output space. Recent work has challenged this belief, showing that complex encoder-decoder architectures perform similarly to nearest-neighbor baselines or simple linear decoder models that exploit large amounts of per-category data, in standard benchmarks. A more realistic setting, however, involves inferring 3D shapes for categories with few available training examples; this requires a model that can successfully generalize to novel object classes. In this work we experimentally demonstrate that naive baselines fail in this few-shot learning setting, where the network must learn informative shape priors for inference of new categories. We propose three ways to learn a class-specific global shape prior, directly from data. Using these techniques, our learned prior is able to capture multi-scale information about the 3D shape, and account for intra-class variability by virtue of an implicit compositional structure. Experiments on the popular ShapeNet dataset show that our method outperforms a zero-shot baseline by over \(50\%\) and the current state-of-the-art by over \(10\%\) in terms of relative performance, in the few-shot setting.

The particular set of conceptual design methods proposed in this paper is, in fact, a functional and structural analysis, which includes: isolating the structure of the system from the environment, the sequence of performing a number of actions to achieve goals, etc. In some cases, the conceptual design methods conduct a systematic synthesis of the project under study. Object-oriented modeling is one of the main elements of such a methodology. The use of modern methods of analysis and synthesis of the structure of the object minimizes errors that are potentially possible during conceptual design and allows to obtain at this stage the maximum improvement of the structure of the tool, which most rationally fits the functional requirements of the technical task. Using the methodology of conceptual system design based on the universal modeling language UML, while using the integration of the necessary corresponding diagrams, we get the opportunity to create the necessary structural model of the system. The structural model is obtained as a result of combining two unary diagrams (target classes and object classes) through logical functional relationships of their entities in the form of a third (transitional) unary diagram of functional-target classes and is a hybrid object-target diagram of classes. This diagram describes the effects on the studied system of reclamation cultivation of slope and plain lands as well as the requirements for their treatment. Emerging goals are structured by the relationship with the elements of technological equipment from the set technological tasks. The resulting structured targets are combined into a basic structure through their aggregation relations of the tool for the system of reclamation cultivation of slope and plain lands. As a result of the theoretical studies of the reclamation treatment system, a new method of slope treatment was created, protected by the RF patent, as well as a number of inventions allowing to create a universal tillage tool used in the reclamation cultivation system for slope and plain lands.

A new preprocessing method, called image screening, is presented for improving the recognition rate and efficiency in statistical image recognition. The problem of detecting a specified object in an input image is treated as a two-class classification problem in which the image falls into a set of subimages in the target object (figure) class and the other set of subimages in the ground class. An image screening algorithm selects a candidate set of subimages which are similar to the object class and rejects the remaining set using screening filters whose design is based on projection pursuit. The classifiers of the two classes are taken from the candidate set. Experimental results for eye and mouth detection in face images and text-area detection in document images demonstrate that this method improves the recognition rate and efficiency.

We present an efficient algorithm for on-road vehicle (e.g. side and rear view of cars) detection problem using cascade of boosted classifiers. Adaptive boosting based classifier in cascaded structure is one of the few good approaches for object detection. This approach filters different non-target (negative) samples in different stages of cascaded structure according to their level of similarity with target object class. The boosted weak learners are quick and efficient for initial stages only, but in later stage of cascaded structure they are not efficient enough to remove the critical false alarms. In this paper, we propose a method of cascading complex features at the later stage of cascaded classifier to enhance the detection performance. We compared the performance of local and global texture features in combination with boosted haar like features. The best performance for on-road obstacle detection is achieved by Adaboost with Haar-like feature along with SVM and Histograms of Oriented Gradients (HOG) features.

Bottom-up, fully unsupervised segmentation remains a daunting challenge for computer vision. In the cosegmentation context, on the other hand, the availability of multiple images assumed to contain instances of the same object classes provides a weak form of supervision that can be exploited by discriminative approaches. Unfortunately, most existing algorithms are limited to a very small number of images and/or object classes (typically two of each). This paper proposes a novel energy-minimization approach to cosegmentation that can handle multiple classes and a significantly larger number of images. The proposed cost function combines spectral- and discriminative-clustering terms, and it admits a probabilistic interpretation. It is optimized using an efficient EM method, initialized using a convex quadratic approximation of the energy. Comparative experiments show that the proposed approach matches or improves the state of the art on several standard datasets.

In this paper we present a novel method for object class recognition. A vocabulary of object parts is automatically constructed from sample images of the object class by AdaBoost. Images are then represented using parts from this vocabulary. Based on this representation, the Sparse Network of Winnows (SNoW) learning architecture is employed to learn to recognize instances of the object class. Experimental results show that the method achieves high recognition accuracy on different data sets, and is highly robust to partial occlusion and background clutter.

Scene graph generation (SGG) is a fundamental task aimed at detecting visual relations between objects in an image. The prevailing SGG methods require all object classes to be given in the training set. Such a closed setting limits the practical application of SGG. In this paper, we introduce open-vocabulary scene graph generation, a novel, realistic and challenging setting, in which a model is trained on a set of base object classes but is required to infer relations for unseen target object classes. To this end, we propose a two-step method which firstly pre-trains on large amounts of coarse-grained region-caption data and then leverages two prompt-based techniques to finetune the pre-trained model without updating its parameters. Moreover, our method is able to support inference over completely unseen object classes, which existing methods are incapable of handling. On extensive experiments on three benchmark datasets, Visual Genome, GQA and Open-Image, our method significantly outperforms recent, strong SGG methods on the setting of Ov-SGG, as well as on the conventional closed SGG.KeywordsOpen-vocabulary scene graph generationVisual-language model pretrainingPrompt-based finetuning

Visual object counting aims to count the number of objects in a given image or video. Among many object counting methods, the counting by density estimation method draws attention because of its capability of counting and its object localization ability. The method utilizes the object density map of the image with multiple objects. The density is estimated by a regression model which learns the mapping between the local features of the given image and the density map generated from its corresponding object locations. Unlike conventional methods that only rely on object locations for density map generation, in this paper, we show that the system performance can be increased by considering object shapes as well as object locations. To this end, we propose two approaches to generate the ground truth density map from the object locations. Both methods generate the density map which reflects structural features of the objects. We show that the regression models trained with density maps which reflect the object shape outperforms the models trained with density maps generated by the conventional density map generation method on several challenging benchmarks. In other words, we observe that it is essential to generate the ground truth density map according to object shape in the image.

Few-shot object detection has arisen to address the issue of data scarcity due to the long-tail distribution problem. The goal of few-shot object detection is to allow a pre-trained detector to generalize novel objects with few-shot data samples. However, few-shot object detectors misclassify novel objects into pre-trained base classes frequently. Existing methods applied a large margin classifier to address this problem, but they do not work well in data scarcity settings. In this paper, we propose graph convolution network-based semantic relation network to capture the semantic relations for few-shot object detection. Regardless of the amount of data, the semantic relations among object classes remain constant, which can support the classification of novel objects. Semantic relations are trained by the attention-based module adaptively to training data. Graph convolution network updates the object classifier using semantic relations. The object classifier is integrated and trained with the visual features. Through the comparison experiments with Two-stage Fine-tuning Approach (TFA), we observe that our proposed method achieves higher performance on few-shot settings than TFA.

Recent research into recognizing object classes (such as humans, cows and hands) has made use of edge features to hypothesize and localize class instances. However, for the most part, these edge-based methods operate solely on the geometric shape of edges, treating them equally and ignoring the fact that for certain object classes, the appearance of the object on the “inside” of the edge may provide valuable recognition cues. We show how, for such object classes, small regions around edges can be used to classify the edge into object or non-object. This classifier may then be used to prune edges which are not relevant to the object class, and thereby improve the performance of subsequent processing. We demonstrate learning class specific edges for a number of object classes — oranges, bananas and bottles — under challenging scale and illumination variation. Because class-specific edge classification provides a low-level analysis of the image it may be integrated into any edge-based recognition strategy without significant change in the high-level algorithms. We illustrate its application to two algorithms: (i) chamfer matching for object detection, and (ii) modulating contrast terms in MRF based object-specific segmentation. We show that performance of both algorithms (matching and segmentation) is considerably improved by the class-specific edge labelling.

We demonstrate a system capable of tracking, in real world image sequences, landmarks such as eyes, mouth, or chin on a face. In a first version knowledge previously collected about faces is used for finding the landmarks in the first frame. In a second version the system is able to track the face without any prior knowledge about faces and is thus applicable to other object classes.

Automatic detection of objects at distance and in all weather conditions is a critical task in many visionbased safety applications such as video surveillance and vehicle forewarn collision warning. In such applications, prior knowledge about the object class (vehicle, pedestrian, tree, etc.) and imaging conditions (shadow, depth) is unavailable. What makes the task even more challenging is when the camera is non-stationary, e.g., mounted on a vehicle. The essential problem in this case lies in distinguishing between camera-induced motion and independent motion. This paper proposes a robust algorithm for automatic object detection at distance. The camera is mounted on a vehicle and operates in both day and night time. Through the utilization of the focus of expansion (FOE) and its associated residual map, the proposed method is able to detect and separate independently objects (IMOs) from the moving background caused by the camera motion. Experimentations on numerous realworld driving videos have shown the effectiveness of the proposed technique. Moving objects such as pedestrians and vehicles up to 40 meters away from the camera have been reliably detected at 10 frames per second on a 1.8GHz PC.

Optical character recognition (OCR) is a vital process that involves the extraction of handwritten or printed text from scanned or printed images, converting it into a format that can be understood and processed by machines. The automatic extraction of text through OCR plays a crucial role in digitizing documents, enhancing productivity, and preserving historical records. This article offers an exhaustive review of contemporary applications, methodologies, and challenges associated with Arabic OCR. A thorough analysis is conducted on prevailing techniques utilized throughout the OCR process, with a dedicated effort to discern the most efficacious approaches that demonstrate enhanced outcomes. To ensure a thorough evaluation, a meticulous keyword-search methodology is adopted, encompassing a comprehensive analysis of articles relevant to Arabic OCR. In addition to presenting cutting-edge techniques and methods, this article identifies research gaps within the realm of Arabic OCR. We shed light on potential areas for future exploration and development, thereby guiding researchers toward promising avenues in the field of Arabic OCR. The outcomes of this study provide valuable insights for researchers, practitioners, and stakeholders involved in Arabic OCR, ultimately fostering advancements in the field and facilitating the creation of more accurate and efficient OCR systems for the Arabic language.

The rapid digital transformation of industrial systems in the 21st century has led to an exponential growth in data generated by manufacturing processes and end-user products, particularly in the automotive sector. While this big data creates new opportunities for monitoring and diagnostics, it also introduces significant challenges related to system complexity, scalability, and nonlinearity, as well as the increasing shortage of experienced domain experts. These challenges motivate the adoption of intelligent, automated fault and anomaly detection techniques capable of operating reliably under real-world conditions. The primary objective of this paper is to provide a comprehensive and structured review of the anomaly detection methodologies for automotive applications, with particular emphasis on intelligent fault diagnosis, tolerance, and monitoring architectures. To this end, the paper systematically categorizes existing approaches, including model-based, data-driven, and hybrid techniques, and analyzes their underlying principles, data requirements, computational complexity, and applicability to safety-critical systems. Based on this analysis, the paper highlights current limitations, open research challenges, and emerging trends, including the integration of machine learning and artificial intelligence with domain knowledge and control-oriented frameworks. The main contribution of this work is a unified perspective that supports researchers and practitioners in selecting, designing, and deploying effective anomaly detection solutions for next-generation automotive and cyber-physical systems.

Object Goal Navigation requires a robot to find and navigate to an instance of a target object class in a previously unseen environment. Our framework incrementally builds a semantic map of the environment over time, and then repeatedly selects a long-term goal (’where to go’) based on the semantic map to locate the target object instance. Long-term goal selection is formulated as a vision-based deep reinforcement learning problem. Specifically, an Encoder Network is trained to extract high-level features from a semantic map and select a long-term goal. In addition, we incorporate data augmentation and Q-function regularization to make the long-term goal selection more effective. We report experimental results using the photo-realistic Gibson benchmark dataset in the AI Habitat 3D simulation environment to demonstrate substantial performance improvement on standard measures in comparison with a state of the art data-driven baseline.