Dealing with uncertain feature assessments in interactive object recognition

Identification of objects in a real world plays a key role for human-computer interaction in a computer-augmented environment using augmented reality techniques. For providing natural interaction in such environments, it is necessary for an interface system to know which objects a user is using. In the previously developed interface systems, real objects are identified by using specially designed tags attached to objects. In this work, we propose a new method for interactive object recognition and registration for more natural and intuitive interaction without using any tags. In particular, we introduce interactive object registration and recognition by combining direct manipulation with user's hands and a color-based object recognition algorithm.

A robot vision system can be called humanoid if it possesses an oculomotor system similar to human eyes and if it is capable to simultaneously acquire and process images of varying resolution. Designers of a number of humanoid robots attempted to mimic the foveated structure of the human eye. Foveation is useful because, firstly, it enables the robot to monitor and explore its surroundings in images of low resolution, thereby increasing the efficiency of the search process, and secondly, it makes it possible to simultaneously extract additional information – once the area of interest is determined – from higher resolution foveal images that contain more detail. There are several visual tasks that can benefit from foveated vision. One of the most prominent among them is object recognition. General object recognition on a humanoid robot is difficult because it requires the robot to detect objects in dynamic environments and to control the eye gaze to get the objects into the fovea and to keep them there. Once these tasks are accomplished, the robot can determine the identity of the object by processing foveal views. Approaches proposed to mimic the foveated structure of biological vision systems include the use of two cameras per eye (Atkeson et al., 2000; Breazeal et al., 2001; Kozima & Yano, 2001; Scassellati, 1998) (Cog, DB, Infanoid, Kismet, respectively), i. e. a narrow-angle foveal camera and a wide-angle camera for peripheral vision; lenses with space-variant resolution (Rougeaux & Kuniyoshi, 1998) (humanoid head ESCHeR), i. e. a very high definition area in the fovea and a coarse resolution in the periphery; and space-variant log-polar sensors with retina-like distribution of photo-receptors (Sandini &Metta, 2003) (Babybot). It is also possible to implement log-polar sensors by transforming standard images into log-polar ones (Engel et al., 1994), but this approach requires the use of high definition cameras to get the benefit of varying resolution. Systems with zoom lenses have some of the advantages of foveated vision, but cannot simultaneously acquire wide angle and high resolution images. Our work follows the first approach (see Fig. 1) and explores the advantage of foveated vision for object recognition over standard approaches, which use equal resolution across the visual field. While log-polar sensors are a closer match to biology, we note that using two cameras per eye can be advantageous because cameras with standard chips can be utilized. This makes it possible to equip a humanoid robot with miniature cameras (lipstick size and 1

Event Abstract Back to Event Logarithmic Hybrid Optical Neural Network-type systems: towards digital-optical cognitive models of robust object recognition Ioannis Kypraios1* 1 Oxford University, Centre for Innovation & Enterprise, APEM Computing Lab, United Kingdom Abstract - By combining the complex logarithmic r-theta mapping of a space-variant imaging sensor [1, 2] with the hybrid digital-optical neural network filter together with a window unit multiple objects of the same class or of different classes can robustly be recognized. The resulted logarithmic r-theta mapping for hybrid digital-optical neural network system or briefly referred to as logarithmic hybrid optical neural network system is shown to exhibit with a single pass over the input data simultaneously in-plane rotation, out-of-plane rotation, scale, log r-θ map translation and shift invariance, and good clutter tolerance by recognizing correctly the different objects within the cluttered scenes. Here, we study the biologically-inspired knowledge learning and knowledge representation [3] achieved by the logarithmic hybrid optical neural network-type of object recognition systems (see Fig. 1 and Fig. 2). We investigate the effects that altering the knowledge representation can have on the problem’s learned knowledge and the problem solving process, in overall. Further, we study the logarithmic unconstrained-, logarithmic constrained-, and logarithmic modified-hybrid optical neural network systems architectures’ designs [4, 5]. We show how the logarithmic unconstrained-hybrid optical neural network object recognition system applies an unconstrained representation of the problem’s knowledge to maximize the search of solutions, how the logarithmic constrained-hybrid optical neural network object recognition system uses a constrained representation of the problem’s knowledge to guide the search towards certain solutions over others in the multidimensional search space, and how the logarithmic modified-hybrid optical neural network object recognition system uses memory-like masks to recall certain solutions over others in the multidimensional search space. Fig. 1. (a) Simplified human retina and visual cortex model used for description purposes only; (b) A digital-optical computational model for the cognitive interaction between the retina and the human visual cortex with the general logarithmic hybrid optical neural network architercture. Fig. 2. Biologically-inspired knowledge learning and knowledge representation with the logarithmic hybrid optical neural network-type of object recognition systems. Figure 1 Figure 2 References [1] C. F. R. Weiman, “Video compression via log-polar mapping”, Real-Time Image Processing II, SPIE Symposium on OE/Aerospace Sensing, Vol. 1295, 266-277 (1990) [2] C. G. Ho et al., “Sensor geometry and sampling methods for space-variant image processing”, Pattern Analysis and Applications, Vol. 5, 369-384 (2002) [3] I. Lee, and B. Portier, "An empirical study of knowledge representation and learning within conceptual spaces for intelligent agents", 6th IEEE/ACIS International Conference Computer and Information Science, 463-468, Melbourne, Australia (2007) [4] I. Kypraios et al., “Fully invariant complex Logarithmic r- map for the Hybrid Optical Neural Network filter for object recognition within cluttered scenes”, 50th Anniversary International Symposium ELMAR 2008, IEEE Region 8/IEEE Croatia Section/EURASIP, Zadar-Croatia, Vol. 1, 141-146 (2008) [5] I. Kypraios et al., “Logarithmic r-θ mapping for hybrid optical neural network filter for object recogntion within cluttered scenes”, Recent Advances in Multimedia Signal Processing and Communications, SCI231, Springer-Verlag Berlin Heidelberg, 91-120 (2009) Keywords: digital-optical, cognitive, object recognition, Robust and Adaptive Systems, Knowledge representation, learning and memory Conference: Neuroinformatics 2013, Stockholm, Sweden, 27 Aug - 29 Aug, 2013. Presentation Type: Poster Topic: Neuromorphic engineering Citation: Kypraios I (2013). Logarithmic Hybrid Optical Neural Network-type systems: towards digital-optical cognitive models of robust object recognition. Front. Neuroinform. Conference Abstract: Neuroinformatics 2013. doi: 10.3389/conf.fninf.2013.09.00066 Copyright: The abstracts in this collection have not been subject to any Frontiers peer review or checks, and are not endorsed by Frontiers. They are made available through the Frontiers publishing platform as a service to conference organizers and presenters. The copyright in the individual abstracts is owned by the author of each abstract or his/her employer unless otherwise stated. Each abstract, as well as the collection of abstracts, are published under a Creative Commons CC-BY 4.0 (attribution) licence (https://creativecommons.org/licenses/by/4.0/) and may thus be reproduced, translated, adapted and be the subject of derivative works provided the authors and Frontiers are attributed. For Frontiers’ terms and conditions please see https://www.frontiersin.org/legal/terms-and-conditions. Received: 09 Apr 2013; Published Online: 11 Jul 2013. * Correspondence: Dr. Ioannis Kypraios, Oxford University, Centre for Innovation & Enterprise, APEM Computing Lab, Begbroke, Oxfordshire, OX5 1PF, United Kingdom, ioanniskyp@yahoo.com Login Required This action requires you to be registered with Frontiers and logged in. To register or login click here. Abstract Info Abstract The Authors in Frontiers Ioannis Kypraios Google Ioannis Kypraios Google Scholar Ioannis Kypraios PubMed Ioannis Kypraios Related Article in Frontiers Google Scholar PubMed Abstract Close Back to top Javascript is disabled. Please enable Javascript in your browser settings in order to see all the content on this page.

An effective human-robot interaction is essential for wide penetration of service robots into the market. Such robot needs a vision system to recognize objects. It is, however, difficult to realize vision systems that can work in various conditions. More robust techniques of object recognition and image segmentation are essential. Thus, we have proposed to use the human user's assistance for objects recognition through speech. This paper presents a system that recognizes objects in occlusion and/or multicolor cases using geometric and photometric analysis of images. Based on the analysis results, the system makes a hypothesis of the scene. Then, it asks the user for confirmation by describing the hypothesis. If the hypothesis is not correct, the system generates another hypothesis until it correctly understands the scene. Through experiments on a real mobile robot, we have confirmed the usefulness of the system.

In this paper we describe the first stage of a new learning system for object detection and recognition. For our system we propose Boosting [5] as the underlying learning technique. This allows the use of very diverse sets of visual features in the learning process within a common framework: Boosting — together with a weak hypotheses finder — may choose very inhomogeneous features as most relevant for combination into a final hypothesis. As another advantage the weak hypotheses finder may search the weak hypotheses space without explicit calculation of all available hypotheses, reducing computation time. This contrasts the related work of Agarwal and Roth [1] where Winnow was used as learning algorithm and all weak hypotheses were calculated explicitly. In our first empirical evaluation we use four types of local descriptors: two basic ones consisting of a set of grayvalues and intensity moments and two high level descriptors: moment invariants [8] and SIFTs [12]. The descriptors are calculated from local patches detected by an interest point operator. The weak hypotheses finder selects one of the local patches and one type of local descriptor and efficiently searches for the most discriminative similarity threshold. This differs from other work on Boosting for object recognition where simple rectangular hypotheses [22] or complex classifiers [20] have been used. In relatively simple images, where the objects are prominent, our approach yields results comparable to the state-of-the-art [3]. But we also obtain very good results on more complex images, where the objects are located in arbitrary positions, poses, and scales in the images. These results indicate that our flexible approach, which also allows the inclusion of features from segmented regions and even spatial relationships, leads us a significant step towards generic object recognition.

Object recognition is of fundamental importance in computer vision. In a few years, pedestrian detection, car detection, and more generally scene recognition will likely be reliable enough to allow fully-automated car navigation, and the human driver will be relegated to the back seat to sip his coffee. In this thesis we are interested in recognizing individual objects and categories. In order to reduce the volume of information one has to process, images are characterized by sets of features. These features, also called interest points, are targeted at image locations with high local information content. Various systems for detecting interest points and for describing the local image appearance near these points, have been proposed in the last two decades. We investigate which combinations from this plethora of detectors and descriptors, are most suited for object recognition tasks. On to the problem of object recognition, we are first interested in recognizing individual objects. In a few years, one can imagine that customers in shops, will take with their cell phone a picture of a product that looks interesting, send it to a remote server with a huge database of individual objects, and get back information about that specific product. We propose a system for individual object recognition, inspired from previous work on coarse-to-fine recognition. All steps of the recognition process are translated into principled probabilistic terms, which allows us to outperform a state-of-the-art commercial system for individual recognition. Regarding categories, faces are probably the category that has received the most attention in computer vision literature. Here we propose a system to recognize images of the same individual in large databases of images. This can be of high interest when looking for images of a given person over the internet. Our method's advantage is that it works on real-world images, as opposed to the face databases from the literature, collected in laboratories with controlled lighting, pose and background conditions. Finally, we are interested in recognition of object categories in general. Using support vector machines for the classification task, we propose a features-based kernel that improves recognition performance on object categories.

This paper reviews the work that has been done at Fraunhofer IOSB (and its predecessor institutes) in the past ten years in the area of laser gated-viewing (GV) in the short-wavelength infrared (SWIR) band. Experimental system demonstrators in various configurations have been built up to show the potential for different applications and to investigate specific topics. The wavelength of the pulsed illumination laser is 1.57 μm and lies in the invisible, retina-safe region allowing much higher pulse energies than for wavelengths in the visible or near-infrared band concerning eye safety. All systems built up, consist of gated Intevac LIVAR® cameras based on EBCCD/EBCMOS detectors sensitive in the SWIR band. This review comprises military and civilian applications in maritime and land domain—in particular vision enhancement in bad visibility, long-range applications, silhouette imaging, 3-D imaging by sliding gates and slope method, bistatic GV imaging, and looking through windows. In addition, theoretical studies that were conducted—e.g., estimating 3-D accuracy or modeling range performance—are presented. Finally, an outlook for future work in the area of SWIR laser GV at Fraunhofer IOSB is given.

In daily life, manipulating objects is one of the most frequent behaviors which is often difficult for the elderly and the disabled to complete due to physical factors. With the aging trend of society and the development of robot technology, it is a trend to use the escort robot to assist people in manipulating objects. Therefore, the object recognition and operation of the escort robot is one of the key research directions of robot technology. It contains many key technologies, such as object recognition technology, motion planning and control technology and communication technology, which determine the ability of escort robot to interact with the environment to provide high quality service. In this paper, an object recognition and operation system for escort robot is developed which includes object recognition module, motion planning and control module and communication module. In the object recognition module, an improved SURF algorithm is designed to match the feature points and recognize the objects, which has a high matching accuracy. Motion planning and control module based on Moveit realizes the planning and execution of actions independently. In the communication module, dedicated communication protocols are designed and applied to ensure that each module can work independently, stably and efficiently. In addition, the system is tested and verified in this paper.

IoT Enabled Automated Object Recognition for the Visually Impaired

A literature analysis has shown that object search, recognition, and tracking systems are becoming increasingly popular. However, such systems do not achieve high practical results in analyzing small moving living objects ranging from 8 to 14 mm. This article examines methods and tools for recognizing and tracking the class of small moving objects, such as ants. To fulfill those aims, a customized You Only Look Once Ants Recognition (YOLO_AR) Convolutional Neural Network (CNN) has been trained to recognize Messor Structor ants in the laboratory using the LabelImg object marker tool. The proposed model is an extension of the You Only Look Once v4 (Yolov4) 512 × 512 model with an additional Self Regularized Non–Monotonic (Mish) activation function. Additionally, the scalable solution for continuous object recognizing and tracking was implemented. This solution is based on the OpenDatacam system, with extended Object Tracking modules that allow for tracking and counting objects that have crossed the custom boundary line. During the study, the methods of the alignment algorithm for finding the trajectory of moving objects were modified. I discovered that the Hungarian algorithm showed better results in tracking small objects than the K–D dimensional tree (k-d tree) matching algorithm used in OpenDataCam. Remarkably, such an algorithm showed better results with the implemented YOLO_AR model due to the lack of False Positives (FP). Therefore, I provided a new tracker module with a Hungarian matching algorithm verified on the Multiple Object Tracking (MOT) benchmark. Furthermore, additional customization parameters for object recognition and tracking results parsing and filtering were added, like boundary angle threshold (BAT) and past frames trajectory prediction (PFTP). Experimental tests confirmed the results of the study on a mobile device. During the experiment, parameters such as the quality of recognition and tracking of moving objects, the PFTP and BAT, and the configuration parameters of the neural network and boundary line model were analyzed. The results showed an increased tracking accuracy with the proposed methods by 50%. The study results confirmed the relevance of the topic and the effectiveness of the implemented methods and tools.

Object recognition and pose estimation are of significant importance for robotic visual servoing, manipulation and grasping tasks. Traditionally, contour and shape based methods have been considered as most adequate for estimating stable and feasible grasps (Bicchi and Kumar, 2000). A new research direction has been advocated in visual servoing where image moments are used to define a suitable error function to be minimized. Compared to appearance based methods, contour and shape based approaches are also suitable for use with range sensors such as, for example, lasers. In this paper, we evaluate a contour based object recognition system building on the method in Nelson and Selinger (1998), suitable for objects of uniform color properties such as cups, cutlery, fruits etc. This system is one of the building blocks of a more complex object recognition system based both on stereo and appearance cues, (Bjorkman and Kragic, 2004). The system has a significant potential both in terms of service robot and programming by demonstration tasks. Experimental evaluation shows promising results in terms of robustness to occlusion and noise

Side-scan sonars are commonly deployed on autonomous underwater vehicles for seafloor imaging and search applications. In recent years, deep neural networks have become a common solution for object recognition problems with these systems, achieving impressive performance. However, adaptability and interpretability of these systems remain a challenge. The models used in these systems are typically tuned for recognition of specific categories of objects and, due to the labor required to create segmentation mask labels, are often limited to bounding box prediction. Meta's Segment Anything Model (SAM), released in 2023, provides a solution to add instance segmentation capabilities to existing object recognition systems. SAM may be fine-tuned on side-scan sonar data to predict separate object and acoustic shadow masks, allowing for automatic and robust measurement of undersea objects. In addition, this fine-tuned variant of SAM may then be integrated with existing object recognition algorithms tuned for side-scan sonar image analysis to transform them into instance segmentation algorithms. The end result is a highly descriptive recognition system with customizable reporting behavior based on the dimensions of the object(s) of interest.

We propose an end-to-end system for text detection and recognition in natural scenes and consumer videos. Maximally Stable Extremal Regions which are robust to illumination and viewpoint variations are selected as text candidates. Rich shape descriptors such as Histogram of Oriented Gradients, Gabor filter, corners and geometrical features are used to represent the candidates and classified using a support vector machine. Positively labeled candidates serve as anchor regions for word formation. We then group candidate regions based on geometric and color properties to form word boundaries. To speed up the system for practical applications, we use Partial Least Squares approach for dimensionality reduction. The detected words are binarized, filtered and passed to a hidden Markov model based Optical Character Recognition (OCR) system for recognition. We show significant improvement in text detection and recognition tasks over previous approaches on a large consumer video dataset. Furthermore, the event detection system built upon the OCR output of this approach outperformed multiple other OCR-only based submissions in the recently concluded NIST TRECVID 2013 multimedia event detection evaluations.

The object of this work is the recognition algorithms of aerial photography objects, namely, the analysis of recognition accuracy based on data sets with different aggregation classes. To solve this problem, an information system for object recognition based on aerial photography data has been developed. An architecture based on neural network architectures of the ConvNets group with structural modifications was chosen and used to create the information system. The use of a convolutional neural network of the ConvNets group in the architecture of the information system for the recognition of objects of aerial photography gives high accuracy rates when training the information system and validating its results. But the authors did not find any studies on the learning of the neural network of the ConvNets group. Therefore, it was decided to conduct an analysis in which case the ConvNets network will provide validation results with higher accuracy when the training takes place on datasets with or without class aggregation. The authors performed an analysis of the accuracy of recognition of aerial photography objects based on data sets with different aggregation classes. The dataset used for neural network training consisted of 3-channel labeled images of 64x64 pixels size. Based on the analysis, the optimal number of epochs for training is selected, which makes it possible to recognize aerial photography objects with greater accuracy and speed. It was concluded that greater accuracy in image classification is achieved for sampling without crossing data from different classes (without aggregation of classes). The result of the work is recommended for use in the automation of dataset filling and information filtering of visual images

Object identification in pictures, videos, and signal processing is not a big frontier and has been around for a few years. Though object recognition in static pictures has proven highly promising in terms of specific items such as facial recognition systems, illness diagnosis, and so on, it has been tricky when it comes to video processing and real-time image processing. Traditional object identification techniques are coupled with machine learning methodologies to improve algorithms' speed and accuracy. The paper focuses on real-time object recognition systems and how recent advances in the realm of object recognition and identification have been made while bearing in mind the real-time scenarios for recognizing varied objects.The paper discusses the current implementations illustrated by varied authors using far more widely used algorithms including YOLO for real-time visualizations. Keywords: Object recognition, YOLO, Machine learning, RCNN, faster RCNN