Automatic Stock Price Prediction and Classification Based on Hybrid with AI Feature Selection Method

Prediction of stock prices has been the primary objective of an investor. Any future decision taken by the investor directly depends on the stock prices associated with a company. This work presents a hybrid approach for the prediction of intra-day stock prices by considering both time-series and sentiment analysis. Furthermore, it focuses on long short-term memory (LSTM) architecture for the time-series analysis of stock prices and Valence Aware Dictionary and sEntiment Reasoner (VADER) for sentiment analysis. LSTM is a modified recurrent neural network (RNN) architecture. It is efficient at extracting patterns over sequential time-series data, where the data spans over long sequences and also overcomes the gradient vanishing problem of RNN. VADER is a lexicon and rule-based sentiment analysis tool attuned to sentiments expressed in social media and news articles. The results of both techniques are combined to forecast the intra-day stock movement and hence the model named as LSTM-VDR. The model is first of its kind, a combination of LSTM and VADER to predict stock prices. The dataset contains closing prices of the stock and recent news articles combined from various online sources. This approach, when applied on the stock prices of Bombay Stock Exchange (BSE) listed companies, has shown improvements in comparison to prior studies.

As an indispensable part in financial industry, stock investment has gradually become more and more important in people's life.The major fluctuation in stock market directly affects the stability of financial markets and the healthy development of the economy.The successful stock price and trend prediction will help investors to yield profit and provide timely and reasonable guidance to market regulators.Therefore, how to forecast stock market is a crucial and worthy topic in financial area.Neural network simulated how brain functions, can automatically extract features from the historical data and related information to predicate future trend, with the primary characteristics of self-organization and adaption.Therefore, neural network is suitable for processing prediction in the field of nonlinear time series.This paper presents a novel data mining algorithm based on the BP neural network and its applications on stock price prediction.Taking improving the efficiency of the traditional back propagation network, we revise the perspective error, and modify incentive function and the adaptive learning rate in order to avoid the local minimum and decrease training time.Moreover, to eliminate the influences caused by noisy data, we combine the fuzzy rough set theory to pre-process the original data.The experimental result verifies feasibility of our model, without the local minimum and slow convergence speed, compared to the other state-of-the-art approaches.Therefore, our algorithm on the predications of the exact stock price value and future trend are applicable and satisfactory.

Stock price exhibits distinct features during different time scales due to the effects of complex factors. Analyzing these features can help delineate the mechanisms that determine the stock price and enhance the prediction accuracy of the stock price. By using singular spectrum analysis (SSA), this paper first decomposes the original price series into a trend component, a market fluctuation component and a noise component to analyze the stock price. The economic meanings of the three components are identified as a long-term trend, effects of significant events and short-term fluctuations caused by noise in the market. Then, to take into account the features of the above three components to the stock price prediction, a novel combined model that integrates SSA and support vector machine (SVM) (e.g., SSA–SVM) is proposed. Compared with SVM, adaptive network-based fuzzy inference system (ANFIS), ensemble empirical mode decomposition-ANFIS (EEMD–ANFIS), EEMD–SVM and SSA–ANFIS, SSA–SVM demonstrates the best prediction performance based on four criteria, indicating that the proposed model is a promising approach for stock price prediction.

Probability distribution function-based classification of structural MRI for the detection of Alzheimer’s disease

Forecasting stock prices is crucial in financial markets, with significant ramifications for investors, traders, and financial institutions. Making accurate predictions allows for well-informed decision-making and effective risk management. In recent years, there has been a notable increase in the utilization of machine learning methods within the financial sector to augment the accuracy of stock price forecasting. The expansion is driven by the growing accessibility of historical market data and the processing capacity necessary for its analysis. Using machine learning techniques inside financial markets has brought about a notable shift in the prevailing paradigm. Conventional approaches, such as time series analysis and fundamental analysis, have played a crucial role but had inherent limitations in comprehensively capturing the intricate dynamics of stock markets. Machine learning has promise in capturing complex patterns and correlations present in data, potentially enhancing the precision of predictive outcomes. The primary objective of this research study is to comprehensively evaluate the efficacy of machine learning models in predicting stock prices. The central research question is: "How effective are machine learning models in predicting the price of stocks?" This question is paramount as it directly impacts investment decisions and risk management strategies. Understanding the capabilities and limitations of machine learning models in this context is crucial for financial professionals and individual investors. This research will primarily consider publicly traded stocks in major stock markets (e.g., NYSE, NASDAQ) over the last decade to maintain a focused investigation. It will explore a range of machine learning techniques, including time series models like ARIMA, deep learning models such as LSTM, and ensemble methods like Random Forest. The analysis will encompass various evaluation metrics commonly used in forecasting, such as Mean Absolute Error (MAE) and Root Mean Square Error (RMSE). This paper is structured as follows: Section 2, provides a comprehensive literature review, delving into traditional stock price prediction methods, the application of machine learning in finance, and previous research relevant to the study. Section 3 analyzes the data collection process, feature selection, model selection, and the chosen evaluation metrics. The experimental data are reported in Section 4, followed by a comprehensive analysis and discussion of these results in Section 5. In conclusion, the present study culminates in Section 6, wherein it provides a comprehensive summary of the principal findings, deliberate on the potential ramifications, and provide valuable perspectives on the prospective advancements in the domain of machine learning as applied to the prediction of stock prices. This study illuminates the dynamic terrain of the banking industry, where the utilization of data-driven decision-making, helped by machine learning, assumes a more central position. Through thoroughly evaluating machine learning models, our research aims to enhance our comprehension of their efficacy and constraints in stock price prediction.

With the rapid development of artificial intelligence in recent years, the research on image processing, text mining, and genome informatics has gradually deepened, and the mining of large-scale databases has begun to receive more and more attention. The objects of data mining have also become more complex, and the data dimensions of mining objects have become higher and higher. Compared with the ultra-high data dimensions, the number of samples available for analysis is too small, resulting in the production of high-dimensional small sample data. High-dimensional small sample data will bring serious dimensional disasters to the mining process. Through feature selection, redundancy and noise features in high-dimensional small sample data can be effectively eliminated, avoiding dimensional disasters and improving the actual efficiency of mining algorithms. However, the existing feature selection methods emphasize the classification or clustering performance of the feature selection results and ignore the stability of the feature selection results, which will lead to unstable feature selection results, and it is difficult to obtain real and understandable features. Based on the traditional feature selection method, this paper proposes an ensemble feature selection method, Random Bits Forest Recursive Clustering Eliminate (RBF-RCE) feature selection method, combined with multiple sets of basic classifiers to carry out parallel learning and screen out the best feature classification results, optimizes the classification performance of traditional feature selection methods, and can also improve the stability of feature selection. Then, this paper analyzes the reasons for the instability of feature selection and introduces a feature selection stability measurement method, the Intersection Measurement (IM), to evaluate whether the feature selection process is stable. The effectiveness of the proposed method is verified by experiments on several groups of high-dimensional small sample data sets.

From past to present, the prediction of stock price in stock market has been a knotty problem. Many researchers have made various attempts and studies to predict stock prices. The prediction of stock price in stock market has been of concern to researchers in many disciplines, including economics, mathematics, physics, and computer science. This study intends to learn fluctuation of stock prices in stock market by using recently spotlighted techniques of deep learning to predict future stock price. In previous studies, we have used price-based input-features to measure performance changes in deep learning models. Results of this studies have revealed that the performance of stock price models would change according to varied input-features configured based on stock price. Therefore, we have concluded that more novel input-feature in deep learning model is needed to predict patterns of stock price fluctuation more precisely. In this paper, for predicting stock price fluctuation, we design deep learning model using 715 novel input-features configured on the basis of technical analyses. The performance of the prediction model was then compared to another model that employed simple price-based input-features. Also, rather than taking randomly collected set of stocks, stocks of a similar pattern of price fluctuation were filtered to identify the influence of filtering technique on the deep learning model. Finally, we compared and analyzed the performances of several models using different configuration of input-features and target-vectors.

QSAR analysis of diaryl COX-2 inhibitors: Comparison of feature selection and train-test data selection methods

Online social media is used to show the sentiments of different individuals about various subjects. Sentiment analysis or opinion mining has recently been considered as one of the highly dynamic research fields in natural language processing, Web mining, and machine learning. There has been a very limited amount of research that focuses on sentiment analysis in the Malay language. This study investigates how feature selection methods contribute to the improvement of Malay sentiment classification performance. Three supervised machine-learning classifiers and seven feature selection methods are used to conduct a series of experiments for the effective selection of the appropriate methods for the automatic sentiment classification of online Malay-written reviews. Findings show that the classifications of Malay sentiment improve using feature selections approaches. This work demonstrates that all feature reduction methods generally improve classifier performance. Support Vector Machine (SVM) approach provide the highest accuracy performance of features selection in order to classify Malay sentiment comparing with other classifications approaches such as PCA and CHI square. SVM records 87% as experimental accuracy result of feature selection.

Software defect prediction models are used for predicting high risk software components. Feature selection has significant impact on the prediction performance of the software defect prediction models since redundant and unimportant features make the prediction model more difficult to learn. Ensemble feature selection has recently emerged as a new methodology for enhancing feature selection performance. This paper proposes a new multi-criteria-decision-making (MCDM) based ensemble feature selection (EFS) method. This new method is termed as MCDM-EFS. The proposed method, MCDM-EFS, first generates the decision matrix signifying the feature’s importance score with respect to various existing feature selection methods. Next, the decision matrix is used as the input to well-known MCDM method TOPSIS for assigning a final rank to each feature. The proposed approach is validated by an experimental study for predicting software defects using two classifiers K-nearest neighbor (KNN) and naïve bayes (NB) over five open-source datasets. The predictive performance of the proposed approach is compared with existing feature selection algorithms. Two evaluation metrics – nMCC and G-measure are used to compare predictive performance. The experimental results show that the MCDM-EFS significantly improves the predictive performance of software defect prediction models against other feature selection methods in terms of nMCC as well as G-measure.

Prediction of the stock price has always been a challenging task due to irregular patterns of the market. Uncertainty has made researchers think of some new and robust predictive methods. Many studies are available in the literature, with many models to predict the stock price accurately. Statistical, machine learning, deep learning, and other related approaches can create a predictive model. ARIMA model is the most commonly used statistical model for time series prediction. But ensemble learning techniques have not been explored much to predict future stock price. So, the present study stresses comparing statistical methods with ensemble learning methods. This paper compares the ARIMA, Random Forest, and Extreme Gradient Boosting models based on root mean squared error (RMSE) and mean absolute percentage error (MAPE). The subject chosen is Google’s stocks, and the data used is from NASDAQ stock exchange. The analysis results show that the ARIMA model performed fairly well for short-term predictions but relatively high MAPE value. The extreme gradient boosting model gave the best performance with the lowest RMSE and MAPE value. Hence, it is evident that after proper hyperparameter tuning, ensemble learning techniques can be used to create robust stock price-prediction models.

Stock market prediction is a very noisy problem and the use of any additional information to increase accuracy is necessary. In this paper, for the stock daily return prediction problem, the set of features is expanded to include indicators not only for the stock to be predicted itself but also a set of other stocks and currencies. Afterwards, different feature selection and classification methods are utilized for prediction. The daily close returns of the 3 most traded stocks (GARAN, THYAO, and ISCTR) in Borsa Istanbul (BIST) are predicted using indicators computed on those stocks, indicators for all the other stocks listed in the BIST100 index, and indicators on the dollar-gold prices. Twenty-five different indicators on daily stock prices are computed to form feature vectors for each trading day. These feature vectors are assigned class labels according to the daily close returns. Expanding the feature space with BIST100 stocks features results in a high dimensional feature space, with possibly noisy or irrelevant features. Therefore, feature selection methods are utilized to select the most informative features. In order to determine relevance scores of features, fast filter-based methods, gain ratio and relief, are used. Experiments are performed based on individual stock features, dollar-gold features (DG), BIST100 stock features (BIST100), and a combination of BIST100 and DG with and without feature selection. Using the gain ratio feature selection with a gradient boosting machine (GBM), the movements of GARAN stock were predicted with an accuracy of 0.599 and an F-measure of 0.614. For THYAO, the relief feature selection with the GBM gave an accuracy of 0.558, and for ISCTR, the gain ratio feature selection with logistic regression achieved an accuracy of 0.581. It was found that using BIST100 stock features boosts classification results for all stocks in terms of accuracy.

In the era of artificial intelligence, machine learning methods are successfully used in various fields. Machine learning has attracted extensive attention from investors in the financial market, especially in stock price prediction. However, one argument for the machine learning methods used in stock price prediction is that they are black-box models which are difficult to interpret. In this paper, we focus on the future stock price prediction with the historical stock price by machine learning and deep learning methods, such as support vector machine (SVM), random forest (RF), Bayesian classifier (BC), decision tree (DT), multilayer perceptron (MLP), convolutional neural network (CNN), bi-directional long-short term memory (BiLSTM), the embedded CNN, and the embedded BiLSTM. Firstly, we manually design several financial time series where the future price correlates with the historical stock prices in pre-designed modes, namely the curve-shape-feature (CSF) and the non-curve-shape-feature (NCSF) modes. In the CSF mode, the future prices can be extracted from the curve shapes of the historical stock prices. Conversely, in the NCSF mode, they can’t. Secondly, we apply various algorithms to those pre-designed and real financial time series. We find that the existing machine learning and deep learning algorithms fail in stock price prediction because in the real financial time series, less information of future prices is contained in the CSF mode, and perhaps more information is contained in the NCSF. Various machine learning and deep learning algorithms are good at handling the CSF in historical data, which are successfully applied in image recognition and natural language processing. However, they are inappropriate for stock price prediction on account of the NCSF. Therefore, accurate stock price prediction is the key to successful investment, and new machine learning algorithms handling the NCSF series are needed.

In many real-world unsupervised learning applications, given data with balanced distribution, that is, there are an approximately equal number of instances in each class, we often need to construct a model to reveal such balance. However, in many data, especially the high-dimensional ones, the data in the original feature space often do not present such balance due to the redundant and noisy features. To tackle this problem, we apply an unsupervised spectral feature selection method to select some informative features, which can better reveal the balanced structure of data. Although spectral feature selection is one of the most popular unsupervised feature selection methods and has been widely studied, none of the existing spectral feature selection methods consider the balance property of data. To address this issue, in this article, we propose a novel balanced spectral feature selection (BSFS) method, which not only selects the discriminative features but also picks those to reveal the balanced structure of data. To the best of our knowledge, this is the first spectral feature selection method considering balance structure of data. By introducing a balanced regularization term, we integrate the balanced spectral clustering and feature selection into a unified framework seamlessly. At last, the experiments on benchmark datasets show that the proposed one outperforms the conventional feature selection methods in both clustering performance and balance, which demonstrates the effectiveness and efficiency of the proposed method.

Stock is a significant component of financial market so that prediction of stock prices has always been a hot topic in the field of financial research. Nowadays, traditional financial models for prediction are confronted with problems of low accuracy and stability. In order to solve the above problems, the article uses LSTM model to predict American AT&T company's stock closing prices and then utilizes attention mechanism to optimize LSTM model (LSTM-Attention). In addition, by comparing LSTM, Transformer and LSTM-Attention models, it is discovered that attention mechanism is superior than traditional deep leaning models in the aspect of predicting stock prices. The experiment shows that LSTM-Attention`s accuracy is higher than LSTM by 8.13% and its RMSE is lower than LSTM by 5.31 %. Meanwhile, Transformer's accuracy is higher than LSTM by 4.68% and its RMSE is lower than LSTM by 2.91 %. Therefore, models based on attention mechanism are better at prediction than LSTM and the difference between LSTM-Attention and Transformer is insignificant. The method in this article is supported to be with high effectiveness, which plays a crucial role in prediction of stock prices in the financial field.