ExemPoser

Hand is an important part of human in communicating with other persons and interacting with objects in the world. In this thesis, we will explore hand analysis, which includes hand detection, hand posture and hand shape analysis.
\n
\nFirst I will focus on hand detection. We start by exploring pixel-level hand detection. It is a challenging problem because the appearance of hands can vary greatly in images. We use a random forest framework in order to predict hand mask and hand part labels in an efficient and robust manner. Our approach exploits the inherent contextual information from structured hand labelling for pixel-level hand detection and hand part labelling. Having obtained a pixel-level hand probability map, we present an end-to-end system to detect hands in a bounding box fashion. Our model can not only detect and localize the hands but also provide detailed information within the bounding box by using a multi-task convolutional neural network. Experimental results show that hand detection can be improved by utilizing more information such as hand shape and hand landmark points than only using bounding boxes.
\n
\nIn the second part of my thesis, we will concentrate on two specific problems based on the good hand detection results. The first problem is to recognize the hand postures efficiently. We present a flexible method for hand posture recognition by fusing information from color and depth images using kernel descriptors. Existing methods usually focus on designing intuitive features for color and depth images. On the contrary, our method extracts common patch-level features, and combines them by means of kernel descriptors. Linear SVM is used to predict the class label efficiently.
\nThe second problem is related to hand shape. We present an interesting application of shape analysis to train a hand shape recognizer automatically from simple sketches, such as a “stick-figure” of a hand shape. We introduce the Hand Boltzmann Machine, a generative model built upon unsupervised learning, to represent the hand shape space of a binary image, and formulate the user provided sketches as an initial guidance for sampling to generate realistic hand shape samples. A hand shape recognizer is trained from these samples and used to classify the hand shapes intuitively.

Surface defects, which often occur during the production of aluminum profiles, can directly affect the quality of aluminum profiles, and should be monitored in real time. This paper proposes an effective, lightweight detection method for aluminum profiles to realize real-time surface defect detection with ensured detection accuracy. Based on the YOLOv5s framework, a lightweight network model is designed by adding the attention mechanism and depth-separable convolution for the detection of aluminum. The lightweight network model improves the limitations of the YOLOv5s framework regarding to its detection accuracy and detection speed. The backbone network GCANet is built based on the Ghost module, in which the Attention mechanism module is embedded in the AC3Ghost module. A compression of the backbone network is achieved, and more channel information is focused on. The model size is further reduced by compressing the Neck network using a deep separable convolution. The experimental results show that, compared to YOLOv5s, the proposed method improves the mAP by 1.76%, reduces the model size by 52.08%, and increases the detection speed by a factor of two. Furthermore, the detection speed can reach 17.4 FPS on Nvidia Jeston Nano’s edge test, which achieves real-time detection. It also provides the possibility of embedding devices for real-time industrial inspection.

Electrogoniometer error compensation, for hand position measurement during planar reaching movements

Hand posture recognition has tremendous potential in the field of natural user interactions. There were many advances in research in recent years but there are still limitations regarding its usage in unfavorable live situations where hand posture variation, illumination change or background complexity are an issue. In cases like these, recognizing the hand posture is a difficult task. As such, we considered reducing the difficulty of the task by using curriculum learning with intermediate information. We proceeded to divide the complex architecture of the hand posture recognition task into two easier ones: 1) Extraction of the hand shape under clutter background with illumination change, 2) Recognition of the hand posture from a binary image. In order to do so, we propose here a bottom-up structured deep convolutional neural network incorporating a special layer for binary image extraction. Our proposed method also employs state-of-the art techniques for deep learning to obtain generalization. As a result, we achieved better recognition performances of the hand posture under clutter background compared to the baseline method.

This paper presents a new algorithm in computer vision for the recognition of hand gestures. In the proposed system, Kinect sensor is used to track and segment hand in the clutter background and feature extracted by finger and an angle between them. Classify the hand posture using multi-class support vector machine. The hand gesture is recognised by stochastic context free grammar (SCFG). Stochastic context free grammar uses syntactic structure analysis and by this method, recognises hand gestures by set of production rules which consists of a combination of hand postures. The proposed algorithm is able to recognise various hand postures in real time with more than 97% accuracy.

Recently, with the increasing adoption of synthetic aperture radar (SAR) ship detection methods on mobile platforms, the lightweighting of detection methods has become a research focus. Despite certain achievements, there are still several limitations: 1) Existing studies have mainly focused on reducing model complexity through shallow network structures. However, this approach frequently results in performance degradation, as they neglected a thorough investigation into achieving a better balance point between inference speed and detection accuracy. 2) Under the lightweight network structure, the rich edge features contained in SAR images, which are crucial for distinguishing ship targets from complex backgrounds, are often underutilized. To address these issues, we propose a novel lightweight detection method based on edge feature aware and fusion. Specifically, to effectively extract edge feature, we introduce an Edge Feature-Aware (EFA) network that incorporates a multiscale channel attention module. Furthermore, a lightweight feature fusion network, Filter-Pruned Bi-directional Feature Pyramid Network (FP-BiFPN), is carefully designed, which can not only suppresses background information, but also accentuates ship targets. Finally, we propose a selective quantization algorithm based on a bit-width selection mechanism to reduce model memory usage without compromising performance. To validate the superiority of our proposed method, we conduct extensive experiments on multiple public datasets, achieving average accuracy scores of 94.2%, 97.6%, and 97.7% on the HRSID, SAR-Ship-Dataset, and SSDD, respectively, with a model parameter size of only 3.36 M, and the fastest processing time for a single frame is 7.2 ms.

The authors propose a modular architecture to learn hand posture kinematics during grasp from little knowledge about the task. The developed model is composed of two parts. The first part is dedicated to the learning of the finger inverse kinematics. This function is fulfilled by a modular architecture (called Fingers Configuration Neural Network, or FCNN) consisting of several neural networks that take into account the discontinuity of the inverse kinematics mapping of the fingers as well as the joint limits. Following the concept of direct associative learning, a second neural network based model is used to optimize the kinematic configuration of the hand according to an evaluative function based on the results of the FCNN. Simulation results show a good learning of grasping postures of various types of objects, with different numbers of fingers and contact sets.

Robust and comprehensive modelling of targets is the key to successful person re‐identification. However, some useful information may be ignored, since CNNs tend to learn from the most distinctive feature region of the human body. In the present study, a multi‐branch lightweight network structure that can enhance the ability of diverse feature retrieval is introduced. The proposed network consists of three branches. In the feature erasure branch, a drop block model is added to remove the horizontal region with the highest activation degree from feature vectors so as to allow the network to learn relatively low discrimination features. The global branch is used as an essential supplement to the feature erasure branch. A unified horizontal segmentation strategy is adopted in the local branch to avoid the influence of feature dislocation. Finally, diverse feature learning is achieved through the branch network structure. The proposed method can achieve state‐of‐the‐art results on Market‐1501, CUHK03 and DukeMTMC‐Reid data sets, thereby demonstrating the effectiveness of the method.

We propose a two-stage hybrid approach with neural networks as the new feature construction algorithms for bankcard response classifications. The hybrid model uses a very simple neural network structure as the new feature construction tool in the first stage, then the newly created features are used as the additional input variables in logistic regression in the second stage. The model is compared with the traditional one-stage model in credit customer response classification. It is observed that the proposed two-stage model outperforms the one-stage model in terms of accuracy, the area under ROC curve, and KS statistic. By creating new features with the neural network technique, the underlying nonlinear relationships between variables are identified. Furthermore, by using a very simple neural network structure, the model could overcome the drawbacks of neural networks in terms of its long training time, complex topology, and limited interpretability.

To perform many critical manipulation tasks successfully, human-robot mimicking systems should not only accurately copy the position of a human hand, but its orientation as well. Deep learning methods trained from pairs of corresponding human and robot poses offer one promising approach for constructing a human-robot mapping to accomplish this. However, ignoring the spatial and temporal structure of this mapping makes learning it less effective. We propose two different hierarchical architectures that leverage the structural and temporal human-robot mapping. We partially separate the robotic manipulator's end-effector position and orientation while considering the mutual coupling effects between them. This divides the main problem---making the robot match the human's hand position and mimic its orientation accurately along an unknown trajectory---into several sub-problems. We address these using different recurrent neural networks (RNNs) with Long-Short Term Memory (LSTM) that we combine and train hierarchically based on the coupling over the aspects of the robot that each controls. We evaluate our proposed architectures using a virtual reality system to track human table tennis motions and compare with single artificial neural network (ANN) and RNN models. We compare the benefits of using deep learning neural networks with and without our architectures and find smaller errors in position and orientation, along with increased flexibility in wrist movement are obtained by our proposed architectures. Also, we propose a hybrid approach to collect the training dataset. The hybrid training dataset is collected by two approaches when the robot mimics human motions (standard learn from demonstrator LfD) and when the human mimics robot motions (LfDr). We evaluate the hybrid training dataset and show that the performance of the machine learning system trained by the hybrid training dataset is better with less error and faster training time compared to using the collected dataset using standard LfD approach.

To address the issues of low efficiency and high omission rates in monitoring workers’ compliance with safety dress codes in intelligent factories, this paper proposes the SFA-YOLO network, an enhanced real-time detection model based on a Selective Feature Attention mechanism. This model enables inspection robots to automatically and accurately identify whether the workers’ attire meets the safety standards. First, this paper constructs a comprehensive dataset of safety attire, including images captured under various scenarios, personnel numbers, and operational conditions. All images are manually annotated to enhance the model’s generalization capability. The dataset contains 3966 images, covering four classes: vest, no-vest, helmet, and no-helmet. Second, the proposed model integrates the SFA mechanism to improve the YOLO architecture. This mechanism combines multi-scale feature fusion with a gated feature extraction module to improve detection accuracy, strengthening the model’s ability to detect occluded targets, partial images, and small objects. Additionally, a lightweight network structure is adopted to meet the inference speed requirements of real-time monitoring. The experimental results demonstrate that the SFA-YOLO model achieves a detection precision of 89.3% and a frame rate of 149 FPS in the safety attire detection task, effectively balancing precision and real-time performance. Compared to YOLOv5n, the proposed model achieves a 5.2% improvement in precision, an 11.5% increase in recall, a 13.1% gain in mAP@0.5, and a 12.5% improvement in mAP@0.5:0.95. Furthermore, the generalization experiment confirms the model’s robustness in various task environments. Compared with conventional YOLO models, the proposed method performs more stably in safety attire detection, offering a reliable technical foundation for safety management in intelligent factories.

Although skeleton-based action recognition has achieved great success in recent years, most of the existing methods may suffer from a large model size and slow execution speed. To alleviate this issue, we analyze skeleton sequence properties to propose a Double-feature Double-motion Network (DD-Net) for skeleton-based action recognition. By using a lightweight network structure (i.e., 0.15 million parameters), DD-Net can reach a super fast speed, as 3,500 FPS on an ordinary GPU (e.g., GTX 1080Ti), or, 2,000 FPS on an ordinary CPU (e.g., Intel E5-2620). By employing robust features, DD-Net achieves state-of-the-art performance on our experiment datasets: SHREC (i.e., hand actions) and JHMDB (i.e., body actions). Our code is on https://github.com/fandulu/DD-Net.

Three neural networks and three numerical methods are investigated, developed, and compared to solve a repetitive motion planning (RMP) scheme for remedying joint-drift problems of redundant robot manipulators. Three recurrent neural networks, i.e., a dual neural network, a linear variational inequality (LVI)-based primal-dual neural network, and a simplified LVI-based primal-dual neural network, are recurrent and real time, and they do not need to be trained in advance. Three numerical methods, i.e., the 94LVI method, the E47 method, and the M4 method, are time discrete and ready to conduct in digital computers. All these solutions have global linear convergence. Computer simulations and physical robot experiments verify that they are all effective to solve the RMP scheme. The comparisons show that neural networks are more accurate and faster than numerical methods on the same simulated condition under the majority normal circumstances. Furthermore, numerical methods are easy to be applied in digital computers since they are time discrete.

The effect of nursing care on intracranial pressure in head-injured patients : Hata K, Takarada M, Matsubara M, et al. Jpn J Acute Med 1986;10:1003–1008

This paper proposes a real-time visual hand tracking and posture estimation system to guide a robotic arm in gripping gestures. We present an original 26 degree-of-freedom model of the hand, for which forward and inverse kinematics formulations have been developed. The hand, wearing a dark glove marked with colored cues, is tracked in real time in the image while completing a grasping task. The position of the color markers in the image is used to reconstruct the hand's pose. Occlusions are handled by position prediction and validation. A robot gripper can then be guided by the different configurations of the hand with respect to the object to be grasped, provided the relationship between the hand and the gripper is known. Successful results for the teleoperated control of a robot manipulator in simulation are presented.