A Knowledge-Based Strategy for Object Recognition and Reconstruction

In this paper, an object recognition method and a pose estimation approach using stereo vision is presented. The proposed approach was used for position based visual servoing of a 6 DoF manipulator. The object detection and recognition method was designed with the purpose of increasing robustness. A RGB color-based object descriptor and an online correction method is proposed for object detection and recognition. Pose was estimated by using the depth information derived from stereo vision camera and an SVD based method. Transformation between the desired pose and object pose was calculated and later used for position based visual servoing. Experiments were carried out to verify the proposed approach for object recognition. The stereo camera was also tested to see whether the depth accuracy is adequate. The proposed object recognition method is invariant to scale, orientation and lighting condition which increases the level of robustness. The accuracy of stereo vision camera can reach 1 mm. The accuracy is adequate for tasks such as grasping and manipulation.

The field of computer vision has undergone rapid development. Starting from recognition tasks such as classification, detection, and segmentation, the focus of visual analysis has gradually shifted towards learning spatiotemporal information. This thesis presents research on spatiotemporal learning, with a particular emphasis on motion estimation and visual recognition. First, we address the problem of video object tracking. Previous methods have primarily re-lied on learning improved appearance representations, while the spatiotemporal relationships of individual objects have been underexplored. We propose leveraging optical flow features to achieve higher generalization in semi-supervised video object segmentation, directly incorporating these features into both the target representation and the decoder network. Our experiments and analysis show that enriching feature representations with spatiotemporal information improves segmentation quality and generalization capability. Next, we investigate spatiotemporal learning in 3D for motion estimation, specifically scene flow estimation. Scene flow estimation as an important research topic in 3D computer vision is crucial for applications such as robotics, autonomous driving, embodied navigation, and tracking. We investigate the problem in different perspectives: 1. What is the best formulation for solving the problem and how to learn a better spatiotemporal feature representation? 2. Can we introduce uncertainty estimation to the task, which is of crucial importance for safety-critical downstream tasks? 3. How to scale the estimation to large-scale data, e.g., autonomous scenes, and leverage the temporal information without introducing much computation overheads? To answer these questions, we explore the use of transformers for improved feature representation, diffusion models for uncertainty estimation, and efficient feature learning methods for multi-frame, large-scale autonomous driving scenarios. Finally, we extend our research to joint visual segmentation, tracking, and open-vocabulary recognition in LiDAR sequences, particularly for autonomous scenes. In such environments, precise segmentation, tracking, and recognition of objects are essential for downstream analysis and control. Current human-annotated open-source datasets allow for reasonable tracking of traffic participants such as cars and pedestrians. However, we aim to advance beyond this towards segmenting and tracking any object in LiDAR data. To this end, we propose a pseudo-labeling engine that leverages the 2D visual foundation model SAM v2 and the vision-language model CLIP to automatically label LiDAR streams. We further introduce the SAL-4D model, capable of segmenting, tracking, and recognizing any object in a zero-shot manner. In summary, we explore the learning of spatiotemporal information in both 2D image and 3D point cloud domains. In the image domain, we demonstrate that spatiotemporal information improves video object segmentation quality and generalization. In the 3D point cloud domain, we show that spatiotemporal learning enables more accurate motion estimation and facilitates the first method for zero-shot segmentation, tracking, and open-vocabulary recognition of arbitrary objects.

Most previous methods for generic object recognition explicitly or implicitly assume that an image contains objects from a single category, although objects from multiple categories often appear together in an image.In this paper, we present a novel method for object recognition that explicitly deals with objects of multiple categories coexisting in an image.Furthermore, our proposed method aims to recognize objects by taking advantage of a scene's context represented by the co-occurrence relationship between object categories.Specifically, our method estimates the mixture ratios of multiple categories in an image via MAP regression, where the likelihood is computed based on the linear combination model of frequency distributions of local features, and the prior probability is computed from the co-occurrence relation.We conducted a number of experiments using the PASCAL dataset, and obtained the results that lend support to the effectiveness of the proposed method.

A method for object recognition using a multisensor model-based approach has been developed. The sensor algorithm research expert system (SARES) is a sun-based workstation for model-based object recognition algorithm development. SARES is a means to perform research into multiple levels of geometric and scattering models, image and signal feature extraction, hypothesis management, and matching strategies. SARES multisensor fusion allows for multiple geometric representations and decompositions, and sensor location transformations, as well as feature prediction, matching, and evidence accrual. It is shown that the fusion algorithm can exploit the synergistic information contained in IR and synthetic aperture radar (SAR) imagery yielding increased object recognition accuracy and confidence over single sensor exploitation alone. The fusion algorithm has the added benefit of reducing the number of computations by virtue of simplified object model combinatorics. That is, the additional sensor information eliminates a large number of the incorrect object hypotheses early in the algorithm. This provides a focus of attention to those object hypotheses which are closest to the correct hypothesis.

Various non-coal foreign objects in raw coal seriously affect the safety and efficiency of coal production. As an attractive object recognition method, object recognition based on computer vision has achieved fruitful results while the accuracy and speed are still great challenges because of the poor image quality results of high conveyor belt speeding and uneven illumination. In this paper, a non-coal foreign object recognition method based on deep learning was proposed. Firstly, based on the noise reduction, image enhancement and image segmentation, a combinatorial optimization method is proposed to preprocess the low-quality images. Secondly, from three aspects including network structure, activation function and decoupling prediction, an improved YOLOv5 algorithm is proposed. Finally, by fusing the visual attention mechanism, the feature extraction quality of the YOLOv5 model is further optimized to enrich the semantic information of foreign objects in the image. The experimental results show that, the proposed method can obtain 88.14% recognition accuracy for gangue objects and 87.10% recognition accuracy for iron objects. Meanwhile, we have achieved as high as 27fps recognition speed, which can meet the real-time recognition on high-speed coal belt conveyor.

As an important branch of artificial intelligence, computer vision plays a huge role in the rapid development of artificial intelligence. From a biological point of view, in the acquisition and processing of information, vision is much more important than hearing, touch, etc., because 70% of the human cerebral cortex is processing visual information. Therefore, advances in computer vision technology are critical to the development of artificial intelligence that is designed to allow machines to think and handle things like humans. The acquisition and processing of visual information has always been the focus of computer vision research, and it is also difficult. The main problem of traditional computer vision technology in the processing of visual information is that the extracted image features are less discriminative, the generalization ability of image features in complex background scenes is insufficient, and the recognition ability on object recognition is poor. In response to these problems, based on the visual neural mechanism, this paper establishes an appropriate computer model for the neuronal cells in the human primary visual cortex, models the recognition response mechanism of the visual ventral system, and performs image feature extraction on the training samples. And object recognition. The results show that compared with the traditional methods, the proposed method effectively improves the discrimination of image features, and the image features extracted under complex background scenes have good generalization ability. On this basis, the training samples can be effectively recognized. The results show that the model based on the visual neural mechanism, the recognition of the edge, orientation and contour of the training sample show the advantages of the biological vision mechanism in object recognition.

With the rapid development of computer, computer vision technology is also making rapid progress. In this paper, deep neural network algorithm is used to improve the technology of computer vision, improve the visual effect, and at the same time, innovative algorithm structure, improve the identification of pattern recognition. Computer vision and pattern recognition is a very cutting edge technology that has given people very advanced tools for vision and recognition. At present, the technology can control more accurate image resolution and improve the ability of pattern recognition. This article mainly explains the process of using deep neural network algorithm to improve computer vision and pattern recognition from the internal mechanism, and reveals the working principle and internal mechanism of computer vision and pattern recognition technology application. Data analysis proves that the pattern recognition application established by deep neural network algorithm performs very well in the field of vision and pattern recognition.KeywordsDeep neural network algorithmComputer visionPattern recognitionTechnical application

Identification of objects in a real world plays a key role for human-computer interaction in a computer-augmented environment using augmented reality techniques. For providing natural interaction in such environments, it is necessary for an interface system to know which objects a user is using. In the previously developed interface systems, real objects are identified by using specially designed tags attached to objects. In this work, we propose a new method for interactive object recognition and registration for more natural and intuitive interaction without using any tags. In particular, we introduce interactive object registration and recognition by combining direct manipulation with user's hands and a color-based object recognition algorithm.

Finding efficient and powerful techniques for representing 2D and 3D entities is central in many areas, such as computer vision, graphics, CAD or data visualization. The large amount of data often involved and the small processing time required place strong demands on modeling techniques. This chapter gives an overview of the possibilities wavelets offer for object modeling, specially from the point of view of object recognition in computer vision environments. For the 2D case, techniques based on segmented contours or raw grey level information are described. For 3D data, both the cases of dense, regularly sampled volumetric information and sparse, irregularly sampled geometric surface information are considered. Ideas on how Bayesian methods may enhance wavelet based techniques for object representation and recognition are outlined.

It is an important task for a robot to bring objects requested by human via voice. In order to achieve the task, object recognition using speech and images is needed. Ozasa et al. has proposed the method for the object recognition by integrating speech and image information. Although this method requires both speech (word) and image models, the speech models are automatically constructed by combining phonemic acoustic models according to the dictionary. However, the image models have to be constructed manually in advance. In this paper, instead of the manual construction of the image models, we propose an automatic image model construction method for object recognition using Web images. The effectiveness of the proposed method is verified in the object recognition by integrating speech and image information.

Object localization and recognition are important problems in computer vision. However, in many applications, exhaustive search over all object models and image locations is computationally prohibitive. While several methods have been proposed to make either recognition or localization more efficient, few have dealt with both tasks simultaneously. This paper proposes an efficient method for concurrent object localization and recognition based on a data-dependent multi-class branch-and-bound formalism. Existing bag-of-features recognition techniques which can be expressed as weighted combinations of feature counts can be readily adapted to our method. We present experimental results that demonstrate the merit of our algorithm in terms of recognition accuracy, localization accuracy, and speed, compared to baseline approaches including exhaustive search, implicit-shape model (ISM), and efficient sub-window search (ESS). Moreover, we develop two extensions to consider non-rectangular bounding regions-composite boxes and polygons-and demonstrate their ability to achieve higher recognition scores compared to traditional rectangular bounding boxes.

Feature-based object matching is a fundamental problem for many applications in computer vision, such as object recognition, 3D reconstruction, tracking, and motion segmentation. In this work, we consider simultaneously matching object instances in a set of images, where both inlier and outlier features are extracted. The task is to identify the inlier features and establish their consistent correspondences across the image set. This is a challenging combinatorial problem, and the problem complexity grows exponentially with the image number. To this end, we propose a novel framework, termed Robust Object Matching using Low-rank constraint (ROML), to address this problem. ROML optimizes simultaneously a partial permutation matrix (PPM) for each image, and feature correspondences are established by the obtained PPMs. Two of our key contributions are summarized as follows. (1) We formulate the problem as rank and sparsity minimization for PPM optimization, and treat simultaneous optimization of multiple PPMs as a regularized consensus problem in the context of distributed optimization. (2) We use the alternating direction method of multipliers method to solve the thus formulated ROML problem, in which a subproblem associated with a single PPM optimization appears to be a difficult integer quadratic program (IQP). We prove that under wildly applicable conditions, this IQP is equivalent to a linear sum assignment problem, which can be efficiently solved to an exact solution. Extensive experiments on rigid/non-rigid object matching, matching instances of a common object category, and common object localization show the efficacy of our proposed method.

The article considers the application of artificial intelligence and computer vision technologies to solve the automation of processing and analysis of botanical micro and macro objects (images of fern spores). Also, there is a problem of developing software for a digital twin of an agrobot. The first problem is an interdisciplinary research aimed at solving applied and fundamental problems in botanical objects' biosystematics and studying microevolutionary processes using computer vision technologies, methods of intelligent image analysis, machine learning, and artificial intelligence. The article presents the developed software module FAST (Functional Automated System Tool) for solving the direct problem — performing measurements from images obtained by scanning electron microscopy, virtual herbaria image library, entomological collections, or images taken in a natural environment.
 The second problem is software development for the digital twin of the agrorobot, designed for precise mechanical processing of plants and soil. The proposed solution includes several components: the control unit — NVIDIA Jetson NANO computing module; the actuator — 6-axis robotic arm; the machine vision unit based on an Intel RealSense camera; the chassis unit — tracked tracks and software drivers and components for their control. The digital twin of the robot considers the environmental conditions and the landscape of the operation area.
 The use of ROS (Robot Operating System) allows minimal effort to transfer a digital model to a physical one (prototype and serial robot) without changing the source code. Furthermore, consideration of the environmental conditions during the programming stage provides opportunities for further development and testing of real-life mathematical models for device control.

In this paper, we present a graphics processing unit (GPU)-based matching technique for the purpose of fast feature matching between different images. The scale invariant feature transform algorithm developed by Lowe for various feature matching applications, such as stereo vision and object recognition, is computationally intensive. To address this problem, we propose a matching technique optimized for GPUs to perform computations in less time. We optimize GPUs for fast computation of keypoints to make our system quick and efficient. The proposed method uses a self-organizing map feature matching technique to perform efficient matching between the different images. The experiments are performed on various image sets to examine the performance of the system under varying conditions, such as image rotation, scaling, and blurring. The experimental results show that the proposed algorithm outperforms the existing feature matching methods, resulting in fast feature matching due to the optimization of the GPU.

Automatization for the picking and placing of a variety of objects stored on shelves is a challenging problem for robotic picking systems in distribution warehouses. Here, object recognition using image processing is especially effective at picking and placing a variety of objects. In this study, we propose an efficient method for object recognition based on object grasping position for picking robots. We use a convolutional neural network (CNN) that can achieve highly accurate object recognition. In typical CNN methods for object recognition, objects are recognized by using an image containing picking targets from which object regions suitable for grasping can be detected. However, these methods increase the computational cost because a large number of weight filters are convoluted with the whole image. The proposed method detects all graspable positions from an image as a first step. In the next step, it classifies an optimal grasping position by feeding an image of the local region at the grasping point to the CNN. By recognizing the grasping positions of the objects first, the computational cost is reduced because of the fewer convolutions of the CNN.