HAND GESTURE RECOGNITION FOR INDONESIAN SIGN LANGUAGE INTERPRETER SYSTEM WITH MYO ARMBAND USING SUPPORT VECTOR MACHINE

We create an API (Application Programming Interface) for Indonesian Sign Language (Sistem Isyarat Bahasa Indonesia/SIBI) which is called OpenSIBI. In this case study, we use the Myo Armband device to capture hand gesture data movement. It uses five sensors: Accelerometer, Gyroscope, Orientation, Orientation-Euler, and EMG. First, we record, convert and save those data into JSON dataset in the server as data learning. Then, every data request (trial data) from the client will compare them using k-NN Normalization process. OpenSIBI API works as the middleware which integrated to RabbitMQ as the queue request arranger. Every service request from the client will automatically spread to the server with the queue process. As the media observation, we create a client data request by SIBI Words and Alphabeth Game, which allows the user to answer several stages of puzzle-game with Indonesian Sign Language hand gesture. Game-player must use the Myo armband as an interactive device that reads the hand movement and its fingers for answering the questions given. Thus, the data will be classified and normalized by the k-NN algorithm, which will be processed on the server. In this process, data will pass OpenAPI SIBI (which connected to RabbitMQ) to queue every incoming data-request. So, the obtained data will be processed one by one and sent it back to the client as the answer.

MuViH: Multi-View Hand gesture dataset and recognition pipeline for human–robot interaction in a collaborative robotic finishing platform

Hand gesture is one of the method used in sign language for non-verbal communication. It is most commonly used by deaf & dumb people who have hearing or speech problems to communicate among themselves or with normal people. Various sign language systems has been developed by many makers around the world but they are neither flexible nor cost-effective for the end users. Hence in this paper introduced software which presents a system prototype that is able to automatically recognize sign language to help deaf and dumb people to communicate more effectively with each other or normal people. Pattern recognition and Gesture recognition are the developing fields of research. Being a significant part in nonverbal communication hand gestures are playing key role in our daily life. Hand Gesture recognition system provides us an innovative, natural, user friendly way of communication with the computer which is more familiar to the human beings. By considering in mind the similarities of human hand shape with four fingers and one thumb, the software aims to present a real time system for recognition of hand gesture on basis of detection of some shape based features like orientation, Centre of mass centroid, fingers status, thumb in positions of raised or folded fingers of hand

BISINDO is one of Indonesian sign language, which do not have many facilities to implement. Because it can cause deaf people have difficulty to live their daily life. Therefore, this research tries to offer an recognition or translation system of the BISINDO alphabet into a text. The system is expected to help deaf people to communicate in two directions. In this study the problems encountered is small datasets. Therefore this research will do the testing of hand gesture recognition, by comparing two model CNN algorithms, that is LeNet-5 and Alexnet. This test will look for which classification technique is better if the dataset conditions in an amount that does not reach 1000 images in each class. After testing, the results found that the CNN technique on the Alexnet architectural model is better to used, this is because when doing the testing process by using still-image and Alexnet model data which has been released in training process, Alexnet model data gives greater prediction results that is equal to 76%. While the LeNet model is only able to predict with the percentage of 19%. When that Alexnet data model used on the system offered, only able to predict correcly by 60%.
 
 Keywords: Sign language, BISINDO, Computer Vision, Hand Gesture Recognition, Skin Segmentation, CIELab, Deep Learning, CNN.

Myoelectric prostheses are a viable solution for people with amputations. The challenge in implementing a usable myoelectric prosthesis lies in accurately recognizing different hand gestures. The current myoelectric devices usually implement very few hand gestures. In order to approximate a real hand functionality, a myoelectric prosthesis should implement a large number of hand and finger gestures. However, increasing number of gestures can lead to a decrease in recognition accuracy. In this work a Myo arm band device is used to recognize fourteen gestures (five build in gestures of Myo armband in addition to nine new gestures). The data in this research is collected from three body-able subjects for a period of 7 seconds per gesture. The proposed method uses a pattern recognition technique based on Multi-Layer Perceptron Neural Network (MLPNN). The results show an average accuracy of 90.5% in recognizing the proposed fourteen gestures.

Abstract. Inability to use speech interfaces greatly limits the deaf and hearing impaired people in the possibility of human-machine interaction. To solve this problem and to increase the accuracy and reliability of the automatic Russian sign language recognition system it is proposed to use lip-reading in addition to hand gestures recognition. Deaf and hearing impaired people use sign language as the main way of communication in everyday life. Sign language is a structured form of hand gestures and lips movements involving visual motions and signs, which is used as a communication system. Since sign language includes not only hand gestures, but also lip movements that mimic vocalized pronunciation, it is of interest to investigate how accurately such a visual speech can be recognized by a lip-reading system, especially considering the fact that the visual speech of hearing impaired people is often characterized with hyper-articulation, which should potentially facilitate its recognition. For this purpose, thesaurus of Russian sign language (TheRusLan) collected in SPIIRAS in 2018–19 was used. The database consists of color optical FullHD video recordings of 13 native Russian sign language signers (11 females and 2 males) from “Pavlovsk boarding school for the hearing impaired”. Each of the signers demonstrated 164 phrases for 5 times. This work covers the initial stages of this research, including data collection, data labeling, region-of-interest detection and methods for informative features extraction. The results of this study can later be used to create assistive technologies for deaf or hearing impaired people.

Human hand recognition plays an important role in a wide range of applications ranging from sign language translators, gesture recognition, augmented reality, surveillance and medical image processing to various Human Computer Interaction (HCI) domains. Human hand is a complex articulated object consisting of many connected parts and joints. Therefore, for applications that involve HCI one can find many challenges to establish a system with high detection and recognition accuracy for hand posture and/or gesture. Hand posture is defined as a static hand configuration without any movement involved. Meanwhile, hand gesture is a sequence of hand postures connected by continuous motions. During the past decades, many approaches have been presented for hand posture and/or gesture recognition. In this paper, we provide a survey on approaches which are based on Hidden Markov Models (HMM) for hand posture and gesture recognition for HCI applications.

Sign Language is most accepted and purposeful way of communication for deaf and mute folks of the society. signing uses gestures, head/body movements and facial expressions for communication. it's a strong manner of Non-Verbal communication among humans. each country has its own developed signing. The language which is employed in Bharat is named as “Indian Sign Language (ISL)”. terribly lessresearch work has been carried out during this space as ISL has standardized recently. presently several researchers have primarily focused on gesture recognition that has been recorded underneath static hand gesture. solely few works have been reported for recognition of dynamic hand gesture. several ways are developed to recognize alphabets and numbers of ISL. The major steps concerned in planning such a system area unit gesture acquisition, trailing and segmentation, feature extraction and gesture recognition. This paper presents a survey on varied hand gesture recognition approaches for ISL. General Terms : Hand Gesture Recognition, Human computer interaction, YOLO3

Hand gesture recognition is an important area of research in the field of Human Computer Interaction (HCI). The geometric attributes of the hand play an important role in hand shape reconstruction and gesture recognition. That said, fingertips are one of the important attributes for the detection of hand gestures and can provide valuable information from hand images. Many methods are available in scientific literature for fingertips detection with an open hand but very poor results are available for fingertips detection when the hand is closed. This paper presents a new method for the detection of fingertips in a closed hand using the corner detection method and an advanced edge detection algorithm. It is important to note that the skin color segmentation methodology did not work for fingertips detection in a closed hand. Thus the proposed method applied Gabor filter techniques for the detection of edges and then applied the corner detection algorithm for the detection of fingertips through the edges. To check the accuracy of the method, this method was tested on a vast number of images taken with a webcam. The method resulted in a higher accuracy rate of detections from the images. The method was further implemented on video for testing its validity on real time image capturing. These closed hand fingertips detection would help in controlling an electro-mechanical robotic hand via hand gesture in a natural way.

Hand gesture recognition is a relevant study topic for a reason that sometimes we may not be in position to communicate verbally. There is need to design Hand gesture recognition systems in order to help people adopt to nonverbal communication mainly sign language. However, there is no clue to understand the meaning of gesture through the computers directly. So this calls for definitions that generalize models in a computer. That is why the machine-learning approaches are implemented in recognition systems. There are generally two types of hand gestures recognition systems which researches have concentrated on. These include static and dynamic Hand gesture recognition systems. However, in building Hand gesture recognition systems, various machine learning approaches have been used. For implementing the proposed system, MS Kinect depth sensor was used as a hardware. The Kinect depth sensor is composed of an infrared camera. This is an advantage to the systems that are designed basing on the depth sensing because factors like color, clothing and background have less effect on the performance. So Kinect based depth sensor systems have a high accuracy and performance making them relevant and applicable in our daily lives. In this paper, we propose a static hand gesture recognition system in real time using two machine learning methods namely Support Vector Machine and Artificial Neural Networks. We use of the newly launched Microsoft Kinect sensor for image extraction. The sensor helps us to extract the hand images. We implement the system on a Matlab platform for reasons that Matlab is widely used by researchers in different fields and that can handle complex computations. In the training of the model, we collect a hundred depth-based Histogram of Oriented Gradient features per alphabet from the hand gesture images which we trained, tested and validated using Artificial Neural Networks (ANN) and Support Vector Machine (SVM). From this dataset, we can generate the generalized gesture model for each alphabet image. For the proposed system, the classification with ANN proves a higher performance then SVM.

The interaction between human and machine can be achieved by implementing human gesture applications which are known as human computer interaction (HCI). It has been developed several years ago, and now we can easily find it in many applications in real world e.g. in virtual reality, games, robotics, sign language recognition, etc. Gesture recognition has the potential to offer a natural way of communication between humans and machines. The most popular methods in human gesture application are hand gesture recognition. Several of hand gesture recognition methods still have the same problem when trying to recognize hand gesture accurately because of the existence of the uncontrolled environment problem, especially complex background. This study offered a method that can handle complex background in hand gesture recognition by analyzing pattern in spatio temporal domain. The system could reduce irregular noise in complex background significantly that affected to recognize the hand gesture. The experiment had been conducted using 180 videos with a total of 15 gestures. The result showed that the proposed method could recognize 15 hand gestures with accuracy of 98.33%.

As Computer Science has grown into an encompassed field in various scientific areas, the need for developing a computer aided and artificially intelligent device has become more important especially in the medical field. Artificial Intelligence (AI) plays a vital role not only in accelerating and optimizing common tasks but also in performing tasks that humans are incapable of. This paper presents a Myo Armband Sign-Language Translator (MAST), which is a novel algorithm to translate a hand's gestures into medical sign language using a Myo armband sensor which collects muscles' electromyography signals and then to classify them using an enhanced version of a dynamic random forest. Our experimental results indicate that a systematic fine tuning of MAST parameters leads to an accuracy improvement of 13% over the state-of-the-art scheme such as SCIKIT's random forest. Other comparison results show an improvement of over 20% compared to a popular classification scheme such as Support Vector Machines (SVM) and a deep learning technique such as Convolutional Neural Network (CNN).

In this paper we have implemented a wireless vision based mobile robot control through hand gesture recognition based on perceptual color space such as HIS, HSV/HSB, HSL. Vision-based hand gesture recognition is an important problem in the field of human-computer interaction, since hand motions and gestures could potentially be used to interact with computers in more natural ways. The robot control was done purely based on the orientation histograms a simple and fast algorithm on the system which would recognize static hand gestures with HSV color spaces as major parameters. The wireless based mobile robot system using hand gestures is a new innovative user interface that resolves the complications of using numerous remote controls for various applications. Based on one unified set of hand gestures, this system interprets the user hand gestures into pre-defined commands to control the remote robot. The experimental results are very encouraging as the system produces real-time responses and highly accurate recognition towards various gestures under different lighting conditions.

Deaf people are one of those with disabilities who cannot speak and hear. Deaf people using sign language for communication who use sign language with hand gestures, body and convey facial expressions using sign language. One important component in sign language is the alphabetical finger or the manual alphabet needed to complete communication. The alphabet of the finger is done by spelling words on spoken language, by spelling letter by letter using a finger. This method is used to spell the name or mention the word. However, not everyone understands sign language, so tools needed to bridge communication between people who are deaf with normal people. One solution that will be offered is to use computer technology as a tool to recognize sign language. The technology is an automatic language translator system that process input images using by using the Convolutional Neural Network (CNN). On this research consist of 3 class such us A, assalamualaikum and hallo which get the accuracy respectively 100% for each class.

This paper aims to develop a real time system for deaf and dump people for achieving better way of communication by using hand gesture recognition. Existing way of communication is not much helpful for the deaf and dump people, since existing approach depends upon translator person, who can understand the sign language of the dump and they manually decode their signs and convey the message. The translator may commit some mistakes due to wrong recognition and that limitation could be significantly overcome by using an image acquisition device and processing unit to process the gestures. The personal computer will correctly display the message which was communicated.