Randomness in neural networks: an overview

Besides the feedforward neural networks, there are the recurrent networks, where the impulses can be transmitted in both directions due to some reaction connections in these networks. Recurrent Neural Networks (RNNs) are linear or nonlinear dynamic systems. The dynamic behavior presented by the recurrent neural networks can be described both in continuous time, by differential equations and at discrete times by the recurrence relations (difference equations). The distinction between recurrent (or dynamic) neural networks and static neural networks is due to recurrent connections both between the layers of neurons of these networks and within the same layer, too. The aim of this chapter is to describe a Recurrent Fuzzy Neural Network (RFNN) model, whose learning algorithm is based on the Improved Particle Swarm Optimization (IPSO) method.

With the huge amount of information available online, the World Wide Web is a fertile area for data mining research. The data mining research is at the cross road of research from several research communities, such as database, information retrieval, and within AI, especially the sub-areas of machine learning and data integrity. Every E-commerce and social website in World Wide Web uses the Classification is one of the data mining problems receiving great attention recently in the database community. Neural network is not suitable for data mining directly, because how the classifications were made is not explicitly stated as symbolic rules that are suitable for verification or interpretation by humans. Different concise symbolic rules with high accuracy can be extracted from a neural network with the proposed approach. The neural network is first trained to achieve the required accuracy in data mining. In this paper we are going to combine neural network with the three different algorithms which are commonly used in data mining to improve the data mining result. These three algorithms are CHARM Algorithm, Top K Rules mining and CM SPAM Algorithm. The different datasets of online e-commerce website filpkart and Amazon are used to train the neural network and to use in data mining. The results of all three data mining algorithm with neural network techniques then tested on the available datasets and result are compared by computational complexity of the algorithm.

Intelligent computing system based on pattern recognition and data mining algorithms

Large scale artificial neural networks (ANNs) have been widely used in data processing applications. The recurrent neural network (RNN) is a special type of neural network equipped with additional recurrent connections. Such a unique architecture enables the recurrent neural network to remember the past processed information and makes it an expressive model for nonlinear sequence processing tasks. However, the large computation complexity makes it difficult to effectively train a recurrent neural network and therefore significantly limits the research on the recurrent neural network in the last 20 years. In recent years, the use of graphics processing units (GPUs) becomes a significant advance to speed up the training process of large scale neural networks by taking advantage of the massive parallelism capabilities of GPUs. In this paper, we propose an efficient GPU implementation of the large scale recurrent neural network and demonstrate the power of scaling up the recurrent neural network with GPUs. We first explore the potential parallelism of the recurrent neural network and propose a fine-grained two-stage pipeline implementation. Experiment results show that the proposed GPU implementation can achieve 2 ~ 11 x speed-up compared with the basic CPU implementation with the Intel Math Kernel Library. We then use the proposed GPU implementation to scale up the recurrent neural network and improve its performance. The experiment results of the Microsoft Research Sentence Completion Challenge demonstrate that the large scale recurrent network without class layer is able to beat the traditional class-based modest-size recurrent network and achieve an accuracy of 47%, the best result achieved by a single recurrent neural network on the same dataset.

The thesis starts from the basic theory, introduces the method of physical data acquisition and the concept of data mining. Marketing evolves from traditional marketing schemes to data mining, which leads to the relationship between data mining technology and modern market management. It focuses on the method of obtaining user information and data mining technology in "cross-marketing", "integrated marketing", and "integrated marketing". Applications in "real-time marketing", "green marketing", "relationship marketing", and "network marketing". At the same time, combined with case analysis, the comprehensive application of data mining in "shopping basket analysis" is discussed and elucidated from the theoretical and technical level.

The Sixth International Symposium on Neural Networks (ISNN 2009)

With an onset of social network amount of data is increasing day by day. In order to analyze the data and extract useful information, the data mining technology can be used. Clustering is one of the popular method of data mining. Clustering can be used for visualizing and analyzing of data. We are discussing Kohonen SOM. We are using neural networks, as a data mining tool which provides statistical observation and layout from big data-sets. We determine how Self-Organizing Kohonen Maps, can be efficiently used for data mining purposes. The Self Organizing Map (SOM) unsupervised learning is an effective computational tool in data mining processes. Self-Organizing Maps (SOMs) used to visualize social network dataset. We used Self-Organizing Map for clustering and analyzing high-dimensional and complex social network datasets. This paper also visualizes SOM neighbor connection, SOM neighbor weight distance, SOM weight position. We perform self organizing map algorithm for social network dataset in matlab.

The traditional data mining method is featured by no analysis over the data distribution and incomplete derived association rule. As a result, the data mining results have the deficiencies of large redundancy probability, large root-mean-square error of approximation (RMSEA) and long consumption time. To handle these issues, this paper proposes a medical data stream distribution pattern association rule mining algorithm based on density estimation. This paper collects medical data, selects the distance method to detect abnormal orphan data in the data stream, detects the duplicate data in the data stream by the similar field matching degree, and eliminates the abnormal data and the duplicate data. Then, the data stream density is estimated based on the histogram estimation samples. According to the data density estimation results, this paper analyzes the distribution of medical data stream from perspectives of concentration, dispersion and morphological characteristics of data distribution. Afterwards, the data distribution pattern association rule mining model is constructed based on compound neural network, data distribution parameters are entered into model’s clustering layer, and in-depth training is conducted over the BP (Back Propagation) neural network at the model’s mining layer. Meanwhile, all rules under the combination of hidden layer’s neuron activity value and corresponding output value, and all rules under the combination of hidden layer’s neuron activity value and corresponding input value are derived, so as to complete association rule mining of medical data stream distribution pattern. The experimental results show that the proposed algorithm has a contour curve closest to the true probability density curve; the dispersion degree of medical data is within a reasonable range, and the medical data has high stability; the data redundancy probability is smaller; the mining result’s RMSEA is small; data mining takes less time.

Data-intensive science is a critical science paradigm that interferes with all other sciences. Data mining (DM) is a powerful and useful technology with wide potential users focusing on important meaningful patterns and discovers a new knowledge from a collected dataset. Any predictive task in DM uses some attribute to classify an unknown class. Classification algorithms are a class of prominent mathematical techniques in DM. Constructing a model is the core aspect of such algorithms. However, their performance highly depends on the algorithm behavior upon manipulating data. Focusing on binarazaition as an approach for preprocessing, this paper analysis and evaluates different classification algorithms when construct a model based on accuracy in the classification task. The Mixed National Institute of Standards and Technology (MNIST) handwritten digits dataset provided by Yann LeCun has been used in evaluation. The paper focuses on machine learning approaches for handwritten digits detection. Machine learning establishes classification methods, such as K-Nearest Neighbor(KNN), Decision Tree (DT), and Neural Networks (NN). Results showed that the knowledge-based method, i.e. NN algorithm, is more accurate in determining the digits as it reduces the error rate. The implication of this evaluation is providing essential insights for computer scientists and practitioners for choosing the suitable DM technique that fit with their data.

At present, deep learning is becoming one of the most popular approach to creation of the artificial intelligences systems such as speech recognition, natural language processing, computer vision and so on. The paper presents a historical overview of deep learning in neural networks. The model of the artificial neural network is described as well as the learning algorithms for neural networks including the error backpropagation algorithm, which is used to train deep neural networks. The development of neural networks architectures is presented including neocognitron, autoencoders, convolutional neural networks, restricted Boltzmann machine, deep belief networks, long short-term memory, gated recurrent neural networks, and residual networks. Training deep neural networks with many hidden layers is impeded by the vanishing gradient problem. The paper describes the approaches to solve this problem that provide the ability to train neural networks with more than hundred layers. An overview of popular deep learning libraries is presented. Nowadays, for computer vision tasks convolutional neural networks are utilized, while for sequence processing, including natural language processing, recurrent networks are preferred solution, primarily long short-term memory networks and gated recurrent neural networks.

This paper conducts a methodological research on the academic performance of online learners. This paper applies classification models commonly used in data mining algorithms, such as random forests based on decision trees, support vector machines, neural networks, K nearest neighbor algorithms, etc., combined with data mining tools S software and statistical analysis tools, and analyzes the University of Finance and Economics Tools scores of online learners in the class of 2019. It studies the important factors that affect the academic performance of college students' online learners, and uses these factors to predict students' academic performance. Based on the distance calculation method in mathematics, the article separately studied the application of Euclidean distance correlation analysis algorithm and correlation coefficient correlation algorithm in curriculum relevance, and compared several correlation algorithms. Experimental research results show that in the era of big data, learners will accumulate a large amount of structured and unstructured data during online learning. We can explore the influencing factors of online learners' academic performance through data mining technology, and we can also use machine learning to automatically learn from the data to the academic performance prediction model.

Decision making tasks are subject to limitation and also depends on human knowledge, experiences, judgments and preferences. In this case, Intelligent Decision System (IDS) technologies can be used to provide realistic and consistent decisions, besides to improve the effectiveness of decision making processes. Intelligent Decision Support System (IDSS) is a contributory of IDS technology to assist decision makers in high level phases of decision making by integrating human knowledge with modeling tools. Nowadays, data mining (DM) techniques is also can be used to support Knowledge Management (KM) tasks especially for knowledge discovery and knowledge engineering. DM is emerging data analysis tool and widely used in order to produce valuable knowledge for decision making as knowledge modeling task. In Human Resource (HR), managing talent is among the challenges of HR professionals which can be handle by using IDSS and data mining technologies. For that reason, in this article, we discussed the potential to uses IDSS approach for talent management using DM techniques by proposing IDSS architecture and a case study on talent classification. This study consists of three parts; the first part is to know the previous works on IDSS, talent management and DM. The second part is discussion on proposed IDSS architecture for talent management. Finally, the third part is a case study on the use of DM classification method for talent forecasting, especially for employee's job promotion.

As a confluence of data mining and World Wide Web technologies, it is now possible to perform data mining on Web log records collected from the Internet Web-page access history. The behaviour of Web page readers is imprinted in the Web server log files. Analyzing and exploring regularities in this behaviour can improve the system performance, enhance the quality and delivery of Internet information services to the end user, and identify populations of potential customers for electronic commerce. Thus, by observing people using collections of data, data mining can bring a considerable contribution to digital library designers. In a joint effort between the TeleLearning-NCE (Networks of Centres of Excellence) project on the Virtual University and the NCE-IRIS project on data mining, we have been developing a knowledge discovery tool, called WebLogMiner, for mining Web server log files. This paper presents the design of WebLogMiner, reports current progress and outlines future work in this direction.

The Outlier detection is very active area of research in data mining where outlier is a mismatched data in dataset with respect to the other available data.In existing approaches the outlier detection done only on numeric dataset.For outlier detection if we use clustering method , then they mainly focus on those elements as outliers which are lying outside the clusters but it may possible that some of the unknown elements with any possible reasons became the part of the cluster so we have to concentrate on that also.The Proposed method uses hybrid approach to reduce the number of outliers.The number of outlier can only reduce by improving the cluster formulation method.The proposed method uses two data mining techniques for cluster formulation i.e. weighted k-means and neural network where weighted kmeans is the clustering technique that can apply on text and date data set as well as numeric data set.Weighted kmeans assign the weights to each element in dataset.The output of weighted k-means becomes the input for neural network where the neural network is the classification and clustering technique of data mining.Training is provided to the neural network and according to that neurons performed the testing.The neural network test the cluster formulated by weighted k-means to ensure that the clusters formulated by weighted k-means are group accordingly.There is lots of outlier detection methods present in data mining.The proposed method use Integrating Semantic Knowledge (SOF) for outlier detection.This method detects the semantic outlier where the semantic outlier is a data point that behaves differently with other data points in the same class or cluster.The main motive of this research work is to reduce the number of outliers by improving the cluster formulation methods so that outlier rate reduces and also to decrease the mean square error and improve the accuracy.The simulation result clearly shows that proposed method works pretty well as it significantly reduces the outlier.

Recurrent neural networks are dynamical network structures which have the capabilities of processing and generating temporal information. To our knowledge the earliest neural network model that processed temporal information was that of MeCulloch and Pitts [McCulloch43]. Kleene [Kleene56] extended this work to show the equivalence of finite automata and McCulloch and Pitts' representation of nerve net activity. Minsky [Minsky67] showed that any hard-threshold neural network could represent a finite state automata and developed a method for actually constructing a neural network finite state automata. However, many different neural network models can be defined as recurrent; for example see [Grossberg82] and [Hopfield82]. Our focus is on discrete-time recurrent neural networks that dynamically process temporal information and follows in the tradition of dynamically (nonautonomous) recurrent network models defined by [Elman90, Jordan86, Narendra90, Pollack91,Tsoi94]. In particular this paper develops a new model, a neural network pushdown automaton (NNPDA), which is a hybrid system that couples a recurrent network to an external stack memory. More importantly, a NNPDA should be capable of learning and recognizing some class of context-free grammars. As such, this model is a significant extension of previous work where neural network finite state automata simulated and learned regular grammars. We explore the capabilities of such a model by inferring automata from sample strings the problem of grammatical inference. It is important to note that our focus is only on that of inference, not of prediction or translation. We will be concerned with problem of inferring an unknown system model based on observing sample strings and not on predicting the next string dement in a sequence. In some ways, our problem can be thought of as one of system identification [Ljung87].