Robust elevator button detection in completely reflective surfaces

In this paper, we propose a method to detect elevator buttons and recognize their labels from images for blind navigation. First, a pixel-level mask of elevator buttons is segmented based on fully convolutional networks. Then a fast scene text detector is applied to recognize the text labels in the image as well as to extract their spatial vectors. Finally, all the detected buttons and their associated labels are paired by combining the button mask and spatial vectors of labels based on their location distribution. To evaluate the proposed method, we collect an elevator button dataset that contains 1,000 images with buttons captured from both inside and outside of elevators and annotate the locations and labels of all buttons. Preliminary results demonstrate the robustness and effectiveness of the proposed method for elevator button detection and associated label recognition.

To successfully move a robot into the building, the elevator button and elevator floor number detection and recognition can play an important role. It can help a robot move in the building, just as it also can help a visually impaired person who wants to move another floor in the building. Due to vision-based approach, the difference in lighting condition and the complex background are the main obstacles in this research. A hybrid image classification model is presented in this research to overcome all these difficulties. This hybrid model is the combination of histogram of oriented gradients and bag of words models, which later reduces the dimension of image features by using the feature selection algorithm. An artificial neural network has been implemented to get the experimental result by training and testing. In order to get training performance, 1000 training image samples have been used and additional 1000 image samples also been used to get the testing performance. The experimental results of this research indicate that this proposed framework is important for real-time implementation to implement the elevator button and elevator floor number recognition framework.

Decision letter: Visual and motor signatures of locomotion dynamically shape a population code for feature detection in Drosophila

The performance evaluation of an AI network model is the important part for building an effective solution before its deployment in real-world on the robot. In our study, we have implemented YOLOv3-tiny and YOLOv4-tiny darknet based frameworks for performance evaluation of the elevator button recognition task and tested both variants on image and video datasets. The objective of our study is two-fold: First, to overcome the limitation of elevator buttons dataset by creating new dataset and increasing its quantity without compromising the quality; Second, to provide a comparative analysis through experimental results and the performance evaluation of both detectors using four machine learning metrics. The purpose of our work is to assist the researchers and developers in decision making of suitable detector selection for deployment in the elevator robot towards button recognition application. The results show that YOLOv4-tiny outperforms YOLOv3-tiny with an overall accuracy of 98.60% compared to 97.91% at 0.5 IoU.

Simultaneous application of two laser-induced fluorescence approaches for film thickness measurements in annular gas-liquid flows

In order to solve the problems of the high cost and environmental requirements of existing contactless service, this paper proposes a novel RFID-based contactless service implementation scheme RF-Notouch, RF-Notouch provides a practical solution based on the IoT that can recognize the user's button action without contact. In RF-Notouch, tags are pasted on the elevator buttons to form a tag matrix to perceive the user's non-contact button actions and a model from radio frequency signal collection to final key press recognition is constructed. In viewing of the environmental noise that appears, this paper uses asymmetric left-right extension, matrix coding, and normalization methods to filter so as to remove the influence of different user button habits and different environments, and improve the accuracy of action recognition. The long and short-term memory network (LSTM) in meaningful learning is used to identify the user's specific button to complete the contactless button service. Real experimental results show that in different environments, RF- Notouch's elevator non-contact button action recognition accuracy rate reaches more than 93%

The usability of mobile robots for surveillance and search and rescue missions can be significantly improved by adding intelligent functionalities that decrease the cognitive load on the operator or that even allow autonomous behavior of the robots, e.g., when communication fails. Mobility in this regard is not only the mechatronic endeavor of developing suited locomotion apparatuses, but also a perception, modeling, and planning challenge. Here, the perception issue of detecting drivable ground is addressed, a particularly important issue for safety, security, and rescue robots, which have to operate in a vast range of unstructured, challenging environments. The simple yet efficient approach is based on the Hough transform of planes. The idea is to design the parameter space such that drivable surfaces can be easily detected by the number of hits in the bins corresponding to drivability. A decision tree on the bin properties increases robustness as it allows the robot to handle uncertainties, especially sensor noise. In addition to the binary distinction of drivable/nondrivable ground, a classification of terrain types is possible. The algorithm is applied to 3D data obtained from two different sensors, namely, a near‐infrared time of flight camera and a stereo camera. Experimental results are presented for typical indoor as well as outdoor terrains, demonstrating robust realtime detection of drivable ground. Seven datasets recorded indoors and outdoors under very varying conditions are used for this purpose. About 6800 snapshots of range data are processed in total. It is shown that drivability can be robustly detected with success rates ranging between 83% and 100%. Computation is extremely fast in the order of 5 to 50 ms. © 2008 Wiley Periodicals, Inc.

During the continual transitions from lab research to real-world applications of vision-based algorithms, there are significant challenges, e.g. the robustness to adapt to complex environments and the high demands of multi-task, multi-modal learning models. More specifically on visual simultaneous localization and mapping (visual SLAM) for mobile robots, two notable limitations are: (i) the drift issue during pose estimation, especially in dynamic environments, makes the positioning system unstable; (ii) the additional loop detection takes many computational resources for image retrieval and global feature geometry checking. This thesis explores the approaches of visual localization on unmanned ground vehicles by solving those limitations. The conventional feature extraction and matching pipeline for vision-based tasks involve three steps: (i) local feature detection and description, (ii) feature matching, and (iii) outlier rejection. The first contribution of the thesis is adding a feature selection and anticipation stage to reduce tracking drift. We explore the noise model of image features to select a subset of all the observed image features with the best ”contribution” during data association and pose estimation across multiple frames. Conventional SLAM algorithms take a strong assumption of scene rigidity, which limits the application under challenging environments. The second part of the thesis addresses the tough issue in dynamic environments with moving objects. We presented GMC, namely the motion clustering approach, a lightweight dynamic object filtering method. It can distinguish moving objects from static landmarks. Based on the theory of motion coherence within a particular image area, GMC could segment dynamic objects in 3D space. We can provide an efficient and robust correspondence algorithm that can extract dynamic objects from a static background with the method. In this way, we propose a dynamic SLAM system that is real-time and free from expensive GPU processors. In contrast to GMC, the thesis’s third part turns to an object-aware learning-based model for more general dynamic scenarios. We use object detection and tracking as points, lines, planes, etc. We utilize semantic information and extract sparse image features simultaneously to keep track of dynamic objects. The static background and different dynamic objects are jointly optimized in a newly developed bundle adjustment sliding window. The estimated 3D bounding boxes can provide more robust camera tracking and better scene understanding, and better map merging. The fourth part of the thesis leverages the emerging feature learning framework. It proposes a unified self-supervised model called LGDNet to generate both global and local image feature descriptors end-to-end. Global feature descriptors embed the whole image into a compact representation, leading to easier scene comparison. On the other hand, local features focus more on the local region similarities of some salient parts for structure from motion. Our proposed method can directly extract features together with descriptors that encode both local maximum responses and global context information, avoiding duplicate calculations based on different feature extraction criteria.

The detection of targets with complex backgrounds in aerial images is a challenging task. In this letter, we propose a robust insulator detection algorithm based on local features and spatial orders for aerial images. First, we detect local features and introduce a multiscale and multifeature descriptor to represent the local features. Then, we get several spatial orders features by training these local features, it improves the robustness of the algorithm. Finally, through a coarse-to-fine matching strategy, we eliminate background noise and determine the region of insulators. We test our method on a diverse aerial image set. The experimental results demonstrate the precision and robustness of our detection method, and indicate the possible use of our method in practical applications.

Person re-identification aims to match person of interest under non-overlapping camera views. Therefore, how to generate a robust and discriminative representation is crucial for person re-identification. Mining local clues from human body parts to describe pedestrians has been extensively studied in existing methods. However, existing methods locate human body parts coarsely and do not consider the relations among different local parts. To address the above problem, we propose a Part-based Structured Representation Learning (PSRL) for better exploiting local clues to improve the person representation. There are two important modules in our architecture: Local Semantic Feature Extraction and Structured Person Representation Learning. The Local Semantic Feature Extraction module is designed to extract local features from human body semantic regions. After obtaining the local features, the Structured Person Representation Learning is proposed to fuse the local features by considering the person structure. To model the underlying person structure, a graph convolutional network is employed to capture the relations of different semantic regions. The generated structured feature encodes underlying person structure information, and local semantic feature can solve the misalignment problem caused by pose variations in feature matching. By combining them together, we can improve the descriptive ability of the generated representation. Extensive evaluations on four standard benchmarks show that our proposed method achieves competitive performance against state-of-the-art methods.

This paper proposes an effective and robust fingertip detection method in 2D plane and applies it to a novel vision based human computer interaction system: finger writing character recognition system (FWCRS). The fingertip detection approach consists of two stages. First, based on the grid sampling and the analysis of sampled hand contour, the fingertip was detected roughly. Then, the location of fingertip was localized precisely based on circle feature matching. Experiments suggest that the proposed fingertip detection method is capable of detecting fingertip in a reliable manner even in a complex background under different light conditions without any markers. To demonstrate the strength of the method, the method was run on 5 sequences with varying light condition, different degrees of clutter background and different speeds of finger movement, experiment shows that the correct rate can reach 98.5%. The finger writing character recognition system in this paper is particularly advantageous for human-computer interaction (HCI) in that users can communicate with computers by their favorite mean: handwriting. At the same time, they can perform handwriting with only their finger directly.

Sentence similarity is widely used in various natural language tasks such as natural language inference, paraphrase identification, and question answering. However, a variety of linguistic expressions and ambiguities of words in sentences make it difficult to measure sentence similarity. Many studies show that using local features or global features of a sentence will produce satisfactory sentence representations that can be utilized to measure sentence similarity. Local features reflect the relationships of adjacent words for each sentence and the sequence information of a sentence are usually expressed by global features. However, local features lack abilities to capture sequence information while a small amount of extracted global features is not enough to produce sentence representations with good qualities. In this paper, we propose A Hybrid Model combining Local and Global features into a Siamese Network (HM-LGSN) for sentence similarity calculation. We first propose a new convolution neural network architecture called group convolution neural network to extract the most representative local features (or word semantic features). Then we combine these new features with pre-trained embeddings of words as input to the Bidirectional Gated Recurrent Units to extract global features of sentences. Finally, we select the global features to form sentence representations and calculate sentence similarity through Manhattan distance. The experimental results on SICK, MSRVID, STS-B datasets show that the accuracy of our proposed model is significantly improved by combining local features and global features.

Abstract. Focusing on the mismatching problems in line matching, this paper integrates the radiation information and the geometry information of the imagery as the multi-constraint conditions, and presents an improved line matching algorithm based on the improved homograph matrix constraint condition. This algorithm firstly obtains the homologous points by feature matching, and for each line to be matched, it calculates the homograph matrix with the homologous points in the neighbourhood of this line. And then it projects the line to be matched line in target image to the search image by the homograph matrix, and determines the candidate lines according to the distance between the central points of lines and the distance between two lines; In these candidate lines, this algorithm further determines the possible homologous lines according to the similarity constraints of the line angles, the distance from the origin of image to the lines, and the overlap of lines; Finally, the epipolar constraint is adopted to find out the overlap segments between homologous lines，and the real homologous line will be determined by the gray similarity constraint. This paper adopts the unmanned aerial vehicle images and UCX digital aerial images to carry on the experiments, and verifies the validity of the algorithm in this paper.

Robust object detection based on local similar structure statistical matching

The absence of local features and global shape constraints severely limits the performance of the hourglass network for facial landmark detection in unconstrained environments. Moreover, diverse feature types and scales may result in low accuracy. This paper proposes a probability‐guided hourglass network to enhance the shape constraints for robust facial landmark detection. Firstly, a multi‐scale pre‐processing module is designed to extract features at different scales. Secondly, based on the heatmaps generated by the stacked hourglass network, the coarse localizations are obtained, while the probability maps are generated with local features. Finally, a probability‐based boundary regression method is proposed and the hausdorff distance is modified as the loss function to constrain the feature shape. Adaptive weights are also added to the loss function, which can help relieve the data imbalance problem. Subjective and objective experimental results on the challenging datasets show that this method outperforms the state‐of‐the‐arts on unconstrained conditions.