SENTIMENT ANALYSIS USING LONG TERM MEMORY (LSTM) BOOK CASE STUDY: UNSOLICITED ADVICE FOR MURDERERS BY VERA WONG'S

Artificial neural networks (ANN) have been used in many applications such like handwriting recognition and speech recognition. It is well-known that learning rate is a crucial value in the training procedure for artificial neural networks. It is shown that the initial value of learning rate can confoundedly affect the final result and this value is always set manually in practice. A new parameter called beta stabilizer has been introduced to reduce the sensitivity of the initial learning rate. But this method has only been proposed for deep neural network (DNN) with sigmoid activation function. In this paper we extended beta stabilizer to long short-term memory (LSTM) and investigated the effects of beta stabilizer parameters on different models, including LSTM and DNN with relu activation function. It is concluded that beta stabilizer parameters can reduce the sensitivity of learning rate with almost the same performance on DNN with relu activation function and LSTM. However, it is shown that the effects of beta stabilizer on DNN with relu activation function and LSTM are fewer than the effects on DNN with sigmoid activation function.

Research on sentiment analysis in recent years has increased. However, in sentiment analysis research there are still few ideas about the handling of negation, one of which is in the Indonesian sentence. This results in sentences that contain elements of the word negation have not found the exact polarity.The purpose of this research is to analyze the effect of the negation word in Indonesian. Based on positive, neutral and negative classes, using attention-based Long Short Term Memory and word2vec feature extraction method with continuous bag-of-word (CBOW) architecture. The dataset used is data from Twitter. Model performance is seen in the accuracy value.The use of word2vec with CBOW architecture and the addition of layer attention to the Long Short Term Memory (LSTM) and Bidirectional Long Short Term Memory (BiLSTM) methods obtained an accuracy of 78.16% and for BiLSTM resulted in an accuracy of 79.68%. whereas in the FSW algorithm is 73.50% and FWL 73.79%. It can be concluded that attention based BiLSTM has the highest accuracy, but the addition of layer attention in the Long Short Term Memory method is not too significant for negation handling. because the addition of the attention layer cannot determine the words that you want to pay attention to.

Wind speed is a crucial parameter alongside coastal areas, especially Indonesia. Above average wind speed can cause harmful effects on human activities. This study uses wind speed data from Berakit Bay, Bintan Island is a potential location for coastal community settlement, fisheries, and tourist activities. The wind parameter then predicted using the Long Short-Term Memory or LSTM algorithm. This algorithm is able to study long-term dependencies by converting simple nervous system designs into specialized blocks containing cells. It is suitable to be applied to long-term wind predictions where the wind speed at this time is very influential with the wind speed in the future. In preparing the LSTM, the data preprocessing and the architecture used will determine the prediction results. In this study, four different architectures were made in order to determine the most optimal architecture. The results show that the LSTM architecture is able to obtain a relatively good RMSE value of 1.87 and an accuracy of 39.40% with the use of two LSTM layers, 256 units in the first layer and 128 in the second layer. The LSTM algorithm in predicting wind can also be applied to other areas in Indonesia.

Many human activities consist of physical gestures that tend to be performed in certain sequences. Wearable inertial sensor data have as a consequence been employed to automatically detect human activities, lately predominantly with deep learning methods. This article focuses on the necessity of recurrent layers—more specifically Long Short-Term Memory (LSTM) layers—in common Deep Learning architectures for Human Activity Recognition (HAR). Our experimental pipeline investigates the effects of employing none, one, or two LSTM layers, as well as different layers' sizes, within the popular DeepConvLSTM architecture. We evaluate the architecture's performance on five well-known activity recognition datasets and provide an in-depth analysis of the per-class results, showing trends which type of activities or datasets profit the most from the removal of LSTM layers. For 4 out of 5 datasets, an altered architecture with one LSTM layer produces the best prediction results. In our previous work we already investigated the impact of a 2-layered LSTM when dealing with sequential activity data. Extending upon this, we now propose a metric, rGP, which aims to measure the effectiveness of learned temporal patterns for a dataset and can be used as a decision metric whether to include recurrent layers into a network at all. Even for datasets including activities without explicit temporal processes, the rGP can be high, suggesting that temporal patterns were learned, and consequently convolutional networks are being outperformed by networks including recurrent layers. We conclude this article by putting forward the question to what degree popular HAR datasets contain unwanted temporal dependencies, which if not taken care of, can benefit networks in achieving high benchmark scores and give a false sense of overall generability to a real-world setting.

The case of inflation can influence monetary policy. Therefore, in assisting policy decision-making, inflation forecasts can be made. Inflation forecasting is a connecting bridge to determine the value of inflation for the coming period. Running inflation allows it to change from time to time, resulting in a nonlinear model that will provide a more accurate forecast of inflation. The neural network is a general function approach capable of mapping any nonlinear function. One part of the neural network method used in forecasting is the Long Short Term Memory (LSTM) method. This method has the advantage of storing information for a more extended period. However, the efficiency of the neural network method depends on the network structure of the number of hidden neurons and epochs in converging conditions. This study aims to obtain the best inflation forecasting model in Indonesia using the LSTM method. This method is a development network of the Recurrent Neural Network, which is composed of forget gate, input gate, cell state, and output gate. Based on the research results, the best LSTM model in predicting inflation in Indonesia has more than one hidden neuron with the optimum number of epochs. However, too many hidden neurons are used, and the use of epochs that are not optimized will make the root mean square error value and mean absolute error based on the sample out worse. This indicates that too many hidden neurons and epochs will lead to overfitting in Indonesia's inflation forecasting.

Human action recognition in videos is an important task with a broad range of applications. In this study, we improve the performance of recurrent attention convolutional neural network (RACNN) by proposing a novel model, “attention-again”. We consider the nature of video frames as sequences, which will cause the change of regions of interest in the frame, thus we cannot use an attention mechanism similar to that in images. “Attention-again” model is a variant from traditional attention model for recognizing human activities and is embedded in two long short-term memory (LSTM) layers. Different from hierarchal LSTM which change the LSTM structure to combine the hidden states from two LSTM layers, our proposals introduce “attention-again” model to avoid the change of LSTM structure. Furthermore, this model not only learns the relations in each frame, but also obtains the relations among all frames, and these relations instruct the next learning stage. Therefore, our proposed model outperform the baseline and is superior to methods with the same experimental conditions on three benchmark datasets: UCF-11, HMDB-51 and UCF-101. To understand how the model works, we also visualize the region of interest in the frame.

In recent years, supervised video summarization has achieved promising progress with various recurrent neural networks (RNNs) based methods, which treats video summarization as a sequence-to-sequence learning problem to exploit temporal dependency among video frames across variable ranges. However, RNN has limitations in modelling the long-term temporal dependency for summarizing videos with thousands of frames due to the restricted memory storage unit. Therefore, in this paper we propose a stacked memory network called SMN to explicitly model the long dependency among video frames so that redundancy could be minimized in the video summaries produced. Our proposed SMN consists of two key components: Long Short-Term Memory (LSTM) layer and memory layer, where each LSTM layer is augmented with an external memory layer. In particular, we stack multiple LSTM layers and memory layers hierarchically to integrate the learned representation from prior layers. By combining the hidden states of the LSTM layers and the read representations of the memory layers, our SMN is able to derive more accurate video summaries for individual video frames. Compared with the existing RNN based methods, our SMN is particularly good at capturing long temporal dependency among frames with few additional training parameters. Experimental results on two widely used public benchmark datasets: SumMe and TVsum, demonstrate that our proposed model is able to clearly outperform a number of state-of-the-art ones under various settings.

Technology social is the fastest and most up-to-date source of information. A model that can provide mapping will help in sorting out information more precisely and quickly. Public opinion in the mass media always develops quickly to talk about an issue in just a few days or even hours, so we do not know what the opinions of the people in the mass media are on the issue. In this study, the author applied topic modeling to the results of sentiment analysis on PPKM. The source of data in this study was obtained from twitter using SNScrape. The collected data was analyzed sentiment using the Long Short-term Memory (LSTM) method, so that public opinion was obtained with positive, negative, and neutral sentiments. The classification obtained from the results of the sentiment analysis process is continued with the topic modeling process using the Latent Dirichlet Allocation (LDA) method and visualized in the form of a wordcloud to find out the relationship between one topic and another. The sentiment analysis process produces a model with an accuracy rate of 90.8% and the topic modeling process successfully presents topics that are easy to interpret so that conclusions can be known about an issue.

Long-short term memory (LSTM) is a state-of-art and widely used model to forecast financial time series. However, primitive LSTM networks do not perform well due to over-fitting problems of the deep learning model and nonlinear and non-stationary characteristics of financial time series data. In addition, complete ensemble empirical mode decomposition with adaptive noise (CEEMDAN) is an outstanding data frequency decomposition technique that can decompose original time series into several intrinsic mode functions and a residue. Thus, this paper proposed a novel hybrid network CEEMDAN-LSTM-BN based on LSTM. Specifically, to avoid over-fitting, the modified LSTM-BN network consists of two LSTM layers, two Batch Normalization (BN) layers following each LSTM layer, and a dropout layer. Each of the intrinsic mode functions and the residue would be processed by CEEMDAN-LSTM-BN and the final prediction results are obtained by reconstructing each predictive series. The advantages of the proposed CEEMDANLSTM-BN networks are verified by comparing them to primitive LSTM, other hybrid models, and some famous machine learning models. Moreover, the robustness of the networks is assessed by numerical experiments on different stock indices datasets.

Every year the government always holds procurement of goods and services (tenders) which are informed through the Electronic Procurement Service (LPSE) or the General Procurement Plan Information System (SIRUP). The process of selecting the type of procurement is still manual, namely by selecting the package category so that it is possible for mistakes to occur such as the type of service procurement into the category of goods procurement type or vice versa. Therefore, this research proposes to use the Natural Language Processing (NLP) method that can classify these packages based on existing categories. The method used is Long Short-Term Memory (LSTM) by comparing existing classification methods such as naïve bayes, logistic regression, decision tree, XG Boost, Gradient Boost, Random Forest and Support Vector Machine. The results obtained by the LSTM method have a higher accuracy than other methods, with an accuracy of 90.25%. With a parameter configuration of 100 units in the LSTM layer, epoch 10, batch size 64 and validation step 5

Hybrid blockchain-based spectrum sharing algorithm for dynamic channel selection in cognitive radio

Over the years, well test analysis or pressure transient analysis (PTA) methods have progressed from straight lines via type curve analysis to pressure derivatives and deconvolution methods. Today, analysis of the log-log (pressure and its derivative) response is the most used method for PTA. Although these methods are widely available through commercial software, they are not fully automated, and human interaction is needed for their application. Furthermore, PTA is described as an inverse problem, whose solution in general is non-unique, and several models (well, reservoir and boundary) can be found applicable to similar pressure-derivative response. This tends to always bring about confusion in choosing the correct model using the conventional approach. This results in multiple iterations that are time consuming and requires constant human interaction. Our approach automates the process of PTA using a Siamese neural network (SNN) architecture comprised of Convolutional neural network (CNN) and Long Short-Term Memory (LSTM) layers. The SNN model is trained on simulated experimental data created using a design of experiments (DOE) approach involving most common 14 interpretation scenarios across well, reservoir, and boundary model types. Across each model type, parameters such as permeability, horizontal well length, skin factor, and distance to the boundary were sampled to compute 560 different pressure derivative responses. SNN is trained using a self-supervised training strategy where the positive and negative pairs are generated from the training data. We use transformations such as compression and expansion to generate positive pairs and negative pairs for the well test model responses. For a given well test model response, similarity scores are computed against the candidates in each model class, and the best match from each class is identified. These matches are then ranked according to the similarity scores to identify optimal candidates. Experimental analysis indicated that the true model class frequently appeared among the top ranked classes. The model achieves an accuracy of 93% for the top one model recommendations when tested on 70 samples from the 14 interpretation scenarios. Prior information on the top ranked probable well test models, significantly reduces the manual effort involved in the analysis. This machine learning (ML) approach can be integrated with any PTA software or function as a standalone application in the interpreter's system. Current work using SNN with LSTM layers can be used to speed up the process of detecting the pressure derivative response explained by a certain combination of well, reservoir and boundary models and produce models with less user interaction. This methodology will facilitate the interpretation engineer in making the model recognition faster for detailed integration with additional information from sources such as geophysics, geology, petrophysics, drilling, and production logging.

Packet loss may affect a wide range of applications that use voice over IP (VoIP), e.g. video conferencing. In this paper, we investigate a time-domain convolutional recurrent network (CRN) for online packet loss concealment. The CRN comprises a convolutional encoder-decoder structure and long short-term memory (LSTM) layers, which have been shown to be suitable for real-time speech enhancement applications. Moreover, we propose lookahead and masked training to further improve the performance of the CRN framework. Experimental results show that the proposed system outperforms a baseline system using only LSTM layers in terms of two objective metrics – perceptual evaluation of speech quality (PESQ) and short-term objective intelligibility (STOI); it also reduces the word error rate (WER) more than the baseline when used as a frontend for speech recognition. The advantage of the proposed system is also verified in a subjective evaluation by the mean opinion score (MOS).

Autism Spectrum Disorder (ASD) is a heterogeneous disorder in children, and the current clinical diagnosis is accomplished using behavioral, cognitive, developmental, and language metrics. These clinical metrics can be imperfect measures as they are subject to high test-retest variability, and are influenced by assessment factors such as environment, social structure, or comorbid disorders. Advances in neuroimaging coupled with machine-learning provides an opportunity to develop methods that are more quantifiable, and reliable than existing clinical techniques. In this paper, we design and develop a deep-learning model that operates on functional magnetic resonance imaging (fMRI) data, and can classify between ASD and neurotypical brains. We introduce a novel strategy to transform time-series data extracted from fMRI signals into Gramian Angular Field (GAF) while locking in the temporal and spatial patterns in the data. Our motivation is to design and develop a novel framework that could encode the time-series, acquired from fMRI data, into images that can be used by deep-learning architectures that have been successful in computer vision. In our proposed framework called ASD-GResTM, we used a Convolutional Neural Network (CNN) to extract useful features from GAF images. We then used a Long Short-Term Memory (LSTM) layer to learn the activities between the regions. Finally, the output representations of the last LSTM layer are applied to a single-layer perceptron (SPL) to get the final classification. Our extensive experimentation demonstrates high accuracy across 4 centers, and outperforms state-of-the-art models on two centers with an increase in the accuracy of 17.58% and 6.7%, respectively as compared to the state of the art. Our model achieved the maximum accuracy of 81.78% with high degree of sensitivity and specificity. All training, validation, and testing was accomplished using openly available ABIDE-I benchmarking dataset.

Long-time prediction of sea wave trains by LSTM machine learning method