Convolutional Neural Network Implementation in BISINDO Alphabet Sign Language Recognition System

Convolutional Neural Network (CNN) is a deep learning algorithm that is widely used to identify and classify a digital image object. In this study the Convolutional Neural Network (CNN) is used as an algorithm that functions to identify leaf types (certain plants) based on images obtained from a public dataset provider named Daun Jamu Indonesia. The existence of image characteristics causes the assistance process to require a more detailed feature selection process. Therefore the CNN method is used in order to solve the problem. The Convolutional Neural Network (CNN) method is capable of performing image recognition by minimizing feature extraction. CNN is also reliable in processing unstructured data because it uses a multi-layered structure of artificial reasoning networks. The image recognition process is carried out by looking for the shape of the model that matches the processed data in order to get the best results. In this study, the augmentation process was carried out on the training data and validation data so that overfitting does not occur in the Convolutional Neural Network (CNN). The results obtained in this study indicate that the Convolutional Neural Network (CNN) method can identify leaf types with a measured accuracy rate of 92% using the Confusion Matrix evaluation method. It is hoped that this research can be used as a reference for the use of the Convolutional Neural Network (CNN) method for image data, especially plant leaf types.

Over the last years, Convolutional Neural Networks (CNNs) have been widely used in remote sensing applications, such as marine surveillance, traffic management or road networks detection. However, since CNNs have extremely high computation, bandwith and memory requirements, the hardware implementation of a CNN on space-grade devices like FPGAs for the on-board processing of the acquired images has brought many challenges. In fact, FPGAs show great advantages in terms of reconfigurability and performance-power ratio, whichmakes them a suitable election to deploy CNNs, although the computational capabilities of this type of hardware devices available on-board is limited, so implementations have to be carefully planned. In this paper, the authors present their work towards the implementation of an efficient CNN onto a spacegrade FPGA in order to achieve the on-board processing of very-high resolution remotely sensed images as soon as the data are provided by the sensor. All this work has been conducted within the EU-funded VIDEO project. As it will be presented in this paper, the work includes the introduction of a methodology based on the project constraints, the evaluation of different state-of-the-art CNN architectures by means of a new efficiency measurement also proposed in this work, the introduction of a new efficient CNN architecture, and finally, its optimized hardware implementation by means of high-level synthesis tools.

Bahasa isyarat merupakan hal yang sangat penting bagi suatu kelompok masyarakat, yaitu masyarakat bisu atau tuli. Untuk dapat berkomunikasi dengan masyarakat bisu atau tuli, orang yang tidak bisu atau tuli memerlukan bahasa isyarat tersebut untuk dapat mengerti maksud atau pikiran mereka yang bisu atau tuli. Sebagian besar percakapan pada bahasa isyarat dilakukan dengan menggunakan tangan, dimana tangan beserta jari-jarinya digunakan untuk membentuk pose atau bentuk yang unik, sehingga dapat dikenali sebagai maksud tertentu. Penulis mengusulkan dikembangkan sistem rekognisi citra digital untuk dapat mengenali bahasa isyarat tersebut. Dengan menggunakan metode Convolutional Neural Network (CNN) yang merupakan bagian dari Deep Learning atau Machine Learning, sistem akan mengenali pose atau bentuk dari citra bahasa isyarat yang dimasukkan, dan memberikan luaran yang sesuai dengan maksud dari pose atau bentuk dari citra bahasa isyarat tersebut. Penelitian ini dimulai dengan pengumpulan data, baik data sekunder dari internet maupun data pribadi yang diambil secara manual. Data kemudian melalui pemrosesan awal dan diklasifikasikan dengan CNN, lalu didapatkan hasil untuk dianalisis. Apabila hasil memuaskan, model akan diekspor untuk dimasukkan ke dalam aplikasi berbasis web untuk digunakan secara real-time. Berdasarkan hasil pengujian, model yang terbaik untuk arsitektur adalah model EfficientNet B4 dengan menggunakan Hyperparameter optimizer Adam dan learning rate 0.001 beserta scheduler. Digunakan pretrained weights untuk meningkatkan akurasi tersebut, dan ditambahkan Spatial transformer untuk mencoba membuat model menjadi lebih kokoh. Ditambah dengan pretrained weights, model diekspor untuk digunakan secara real-time. Hasil pengujian real-time menunjukkan bahwa model mampu mendeteksi setidaknya 23 dari 26 alfabet pada latar belakang yang abstrak. Apabila diuji pada latar belakang polos seperti hitam atau putih, model mampu mendeteksi seluruh 26 alfabet dengan probabilitas yang hampir sempurna. Hal ini menunjukkan bahwa metode yang digunakan sudah mampu mengatasi masalah yang disampaikan. Abstract Sign language is very important for a group of people, namely the deaf or dumb. To be able to communicate with people who are mute or deaf, people who are not mute or deaf require sign language to be able to understand the intentions or thoughts of those who are mute or deaf. Most conversations in sign language are carried out using the hands, where the hands and their fingers are used to form unique poses or shapes, so that they can be recognized as having certain meanings. The author proposes to develop a digital image recognition system to be able to recognize sign language. By using the Convolutional Neural Network (CNN) method which is part of Deep Learning or Machine Learning, the system will recognize the pose or shape of the entered sign language image, and provide output that matches the meaning of the pose or shape of the sign language image. This research began with data collection, both secondary data from the internet and personal data taken manually. The data then goes through initial processing and is classified with CNN, then results are obtained for analysis. If the results are satisfactory, the model will be exported to be included in a web-based application for use in real-time. Based on the test results, the best model for the architecture is the EfficientNet B4 model with the Hyperparameter consisting of optimizer Adam and learning rate 0.001 along with the scheduler. Pretrained weights were used to improve accuracy, and Spatial transformers were added to try to make the model more robust. Coupled with pretrained weights, the model is exported for use in real-time. Real-time test results show that the model is able to detect at least 23 of the 26 alphabets on an abstract background. When tested on a plain background such as black or white, the model was able to detect all 26 alphabets with almost perfect probability. This shows that the method used is able to overcome the problem presented.

This research aims to implement deep learning techniques to determine fact and fake news in Indonesian language. The methods used are Convolutional Neural Network (CNN) and Long Short Term Memory (LSTM). The stages of the research consisted of collecting data, labeling data, preprocessing data, word embedding, splitting data, forming CNN and LSTM models, evaluating, testing new input data and comparing evaluations of the established CNN and LSTM models. The Data are collected from a fact and fake news provider site that is valid, namely TurnbackHoax.id. There are 1786 news used in this study, with 802 fact and 984 fake news. The results indicate that the CNN and LSTM methods were successfully applied to determine fact and fake news in Indonesian language properly. CNN has an accuracy test, precision and recall value of 0.88, while the LSTM model has an accuracy test and precision value of 0.84 and a recall of 0.83. In testing the new data input, all of the predictions obtained by CNN are correct, while the prediction results obtained by LSTM have 1 wrong prediction. Based on the evaluation results and the results of testing the new data input, the model produced by the CNN method is better than the model produced by the LSTM method.

Brain Computer Interface (BCI) is a system that connects the human brain with the outside world for people who have motor skills disability problems. One form of utilization is the P300 speller which is used for character recognition or detection by classifying the P300 signal. The Convolutional Neural Network (CNN) method is a deep learning method that can be used to handle signal problems with ID-CNN. At the initial stage the data signal will be transformed and followed by a duplication process using RandomOverSampling because the amount of data in each class is not balanced. The data will be divided into training, validation, and test data. After that, a training with CNN will be conducted and followed by an evaluation to find the best model. The test results from this study are a good-fitting CNN model with an evaluation value consisting of an accuracy of 94.27%, precision of 90.64%, sensitivity / recall of 98.30%, and f-measure of 94.31%. Based on the test, the CNN method can be used and implemented in authentication detection based on the P300 signal.

Aiming at the time-consuming problem of the full-wave (FW) simulation of the scattering characteristics of the traditional graphene reconfigurable reflectarray antenna, a fast prediction method of electromagnetic (EM) response based on deep learning is proposed. The convolutional neural network (CNN) method in deep learning is effectively used in the research of this paper. This method first discretizes the input vector (patch geometry, chemical potential, frequency, incident angle, etc.) of the graphene reflectarray antenna, and then preprocesses the data into a two-dimensional image suitable for CNN training, and finally uses CNN to train the model instead of extensive FW simulation calculations, the EM response of the reflectarray antenna is calculated. The training results of three algorithms of support vector regression (SVR), radial basis function network (RBFN) and CNN are comprehensively compared. The experimental results show that CNN method has good performance and accuracy in the EM response prediction of the graphene reconfigurable reflectarray antenna, with an accuracy of over 99%, and can also save at least 99% of time.

Aiming at the problem of objects in image recognition rectangle, but objects which are input into convolutional neural networks square, the object recognition model was put forward which was based on BING method to realize object estimate, used vectorization of convolutional neural networks to realize input square image in convolutional networks, therefore, built joint convolution neural networks, which achieve multiple size image input. Verified by experiments, the accuracy of multi-object image recognition was improved by 6.70% compared with single vectorization of convolutional neural networks. Therefore, image recognition method of joint convolutional neural networks can enhance the accuracy in image recognition, especially for target in rectangular shape.

Horticultural plants are widely cultivated in Indonesia, but failure because of the wrong cultivation technique is common. In this research, we focus on the broccoli plant because it is famous and watering is very important for this plant. To help the farmer, we try to develop the technology to control the water content and give the water to broccoli. In this case, we use object recognition to detect broccoli by using a webcam. The object detection method in the broccoli image is carried out using the Convolutional Neural Network (CNN) method with the You Only Look Once (YOLO) object recognition algorithm and the shape detection method as a detector for measuring floret area. This method is used to classify and measure floret area on broccoli plants via video or real-time webcam. From the results of the shape detection experiment, we succeed in classifying small florets which have a floret area with a range of (3479 -7787) pixels, while the area of medium florets has a range (of 14321 - 16822) pixels and large florets have a range (23316 - 36790) pixels. The average accuracy that we get from the implementation of the YOLO algorithm in this research reaches 99% accuracy.

Some researchers propose using the Convolutional Neural Network (CNN) method to classified batik images. It can extract features automatically without the need to define feature manually from the image. However, CNN’s weakness is that its accuracy is quite low, especially for small-sized datasets, compared to machine learning methods that use handcrafted feature extraction. In this research, an ensemble CNN method is proposed to improve the accuracy of the CNN method in classifying batik images. This method will train several CNN models at once, and then by voting and averaging techniques, the output label will be determined. Test results for two different datasets show this method can improve the accuracy of the CNN method and get an accuracy value of 100%. This method is also proven to extract features faster than the state-of-the-art method like MTCD+SVM, which is included in the hand-crafted feature extraction category.

Garbage is a unique problem in Indonesia. From ordinary waste to limited emergency plastic waste, Indonesia is the second-largest source of plastic waste in the world. Separate collection and disposal of waste is one way to reduce the waste generated by society and industry in Indonesia. Sorting out the types of waste is the first step before the recycling process. In the field of Computer Vision research, it is difficult to see the type and form of waste with a camera, therefore this study aims to overcome this problem by using Deep Learning technology which is expected to be implemented in the whole of Indonesia starting from some of the largest waste-producing cities. Deep Learning is a computer (AI) technique for learning like a human - with experiments being a Part of Machine Learning that can be used to classify images. The method used in this study uses the Convolutional Neural Network (CNN) method which can be used to detect and recognize objects in an image, which can be used to create an automatic waste classification system. Broadly speaking, CNN utilizes the convolution process by moving a convolution kernel (filter) of a certain size to an image, the computer gets new representative information from the results of multiplying that part of the image with the filter used. The test results show that the CNN method can classify inorganic waste with accuracy. 96% and organic waste with an accuracy of 62%.

Most people are not used to communicating via sign language because sign language is not a mandatory language to learn so very few people understand how to communicate using sign language. This problem for people with special needs to interact and communicate with other people, especially those who do not understand how to communicate using sign language. Convolutional Neural Network is the method that will be used in this research because this method is one of the deep learning methods that currently provides the best results in object detection. The Convolutional Neural Network method is able to imitate human abilities in the image recognition system of the human visual cortex, so that it is able to process visual information. This research used Tensorflow to develop various models as well as work related to other statistical analysis. The evaluation metrics results obtained from this research show precision of 82.99%, recall of 84.42% and f1-score of 83.68%. The value of average precision is 0.830% and average recall is 0.844%. There is also a loss value produced by this model of 0.039065. For accuracy results from real time sign language detection, the accuracy value was 94.64%.

Indonesian Sign Language or BISINDO is an alternative language used by people who suffer from disabilities, especially those who have hearing impairments. This language grew and developed from the deaf community, so its use is based on the visual aspect. This research aims to apply Residual Networks to detect objects in the context of Bisindo Letter Sign Language, with the hope of increasing accuracy and efficiency in letter recognition. Object detection goes through 2 stages, namely feature extraction and model training. ResNet is a type of Convolutional Neural Network (CNN) architecture that utilizes models that have been previously trained, so it can save the time required in the model development process. In this research, Residual Network (ResNet) was used for feature extraction to recognize important aspects in the Bisindo letter sign image, such as hand position, finger shape characteristics, and direction of movement. The research results show that the new dataset used as training data and test data has a fairly good ability to detect with a division of 70% train set, 20% valid set and 10% test set with size 640x640 with 300 epochs for the training model.

Image classification is one of the studies that is currently being developed. The details of the characteristics that must be captured make researchers compete to find the most suitable method for classifying. The Convolutional Neural Network (CNN) algorithm is one of the most superior algorithms in the field of object classification and identification today. With the help of several packages contained in Google Colab for classification, this algorithm is easier to use. In this study, the target case is the age of a person who will be classified using photos or images taken from the internet which are then stored in the form of Google Drive. The research data used is divided into 2 parts, namely for training data as many as 23.440 images, and 10,046 for testing data. Then to facilitate the extraction of features from the features to be identified, the researchers carried out the preprocessing stage, namely grayscale images, and data augmentation. The purpose of this study is to implement the concept of Deep Learning with Convolutional Neural Networks (CNN) in image classification and to determine the level of accuracy of the CNN model in classifying images. After the algorithm is run and the model has been formed, an accuracy of 78.5% is obtained. It can be concluded that the Convolutional Neural Network (CNN) method is good at classifying images

The hardware implementation of convolutional neural network has the problem of resource limitation, which can be solved by the design of convolutional neural network accelerator based on FPGA dynamic reconstruction. The whole parallel strategy and architecture of CNN accelerator are designed, and the functional modules of CNN are designed based on pipeline. The dynamic reconstruction technology is used to redesign the convolutional neural network accelerator, and the dynamic reconstruction region and division are established; BPI flash is selected to store the configuration file, and the configuration file is read internally to dynamically configure the dynamic reconstruction area. Finally, for lenet. 5 handwriting recognition, compared with the corresponding static design, the use of slice LUTS, slice registers and DSP resources of the accelerator based on dynamic reconstruction design is reduced by 46%, 25% and 68% respectively. Compared with the implementation based on software platform. The system execution time is greatly reduced. However, due to the bandwidth limitation of the internal configuration port, the reconfiguration configuration time prolongs the execution time of the whole convolutional network.

Leucorrhea routine test is one of the most widely used tests in gynecological examinations, and fungi inspection is vital for gynecological test because fungi is an important evidence for fungal vaginitis. In order to improve detection accuracy, an automatic identification of fungi in microscopic leucorrhea images based on convolutional neural network (CNN) and morphological method is proposed in this paper. First, we use the maximum inter-class variance method to segment original image and obtain possible fungi subimages. Then, a fully trained CNN is applied to recognize fungi. Finally, morphological method, such as template match method and concave point detection method, is used to further classify the selected candidate to improve recognize accuracy. In experiments, the method using CNN and morphological method achieved 93.26% accuracy.