Multi-label incremental learning applied to web page categorization

Neural network, a nonalgorithmic, nondigital, intensely parallel and distributive information processing system, is being used more and more everyday. The main interest in neural networks is rooted in the recognition that the human brain processes information in a different manner than conventional digital computers. Computers are extremely fast and precise at executing sequences of instructions that have been formulated for them. A human information processing system is composed of neurons switching at speeds about a million times slower than computer gates. Yet, humans are more efficient than computers at such computationally complex tasks as speech and other pattern-recognition problems. Artificial neural systems, or neural networks, are physical cellular systems that can acquire, store, and use experiential knowledge. The knowledge is in the form of stable states or mapping embedded in networks that can be recalled in response to the presentation of cues. In a typical neural data processing procedure, the database is divided into two separate portions called training and test sets. The training set is used to develop the desired network. In this process (depending on the paradigm that is being used), the desired output in the training set is used to help the network learn by adjusting the weights between its neurons or processing elements. Once the network has learned the information in the training set and has"converged," the test set is applied to the network for verification. It is important to note that although the user has the desired output of the test set, it has not been seen by the network. This ensures the integrity and robustness of the trained network. A handful of articles on the use of neural networks in the petroleum industry has appeared in SPE conferences, proceedings, and publications in the past 2 years. These articles can be divided into two categories: those that use neural networks to analyze formation lithology from well logs and those that use neural networks to pick a reservoir model to be used in conventional welltest interpretation studies. These tasks are usually done by log analysts and reservoir engineers, and their automation using a fault-tolerant process may prove valuable. Neural networks can help engineers and researchers by addressing some fundamental petroleum engineering problems as well as specific ones that conventional computing has been unable to solve. Petroleum engineers may benefit from neural networks on occasions when engineering data for design and interpretations are less than adequate. This is an especially common occurrence in the Appalachian basin, where some fields are quite old. Lack of adequate engineering data may also be encountered because of the high cost of coring, well testing, and so on. Neural networks have shown great potential for generating accurate analysis and results from large amounts of historical data that otherwise would seem not to be useful or relevant in the analysis. An example of such a problem was encountered by a gas company for a gas storage field in Ohio. In the absence of appropriate data, which normally would make engineering design and evaluation of the fracturing jobs virtually impossible, a carefully designed neural network was able to predict the performance of fracturing jobs with great accuracy. A linear plot of the actual fracturing job results (data never seen by the network during training) and network predictions resulted in a correlation coefficient of 0.98, where 1.00 is a perfect match. Neural networks have proved to be valuable pattern-recognition tools. They are capable of finding highly complex patterns within large amounts of data. A relevant example is well log interpretation. It is generally accepted that there is more information embedded in well logs than meets the eye. Determination, prediction, or estimation of formation permeability without actual laboratory measurement of the cores or interruption in production for well test data collection has been a fundamental problem for petroleum engineers. From geophysical well log data, it was possible to predict and/or estimate permeability of a highly heterogeneous formation in West Virginia, as shown in Fig 1. Reference Mohaghegh, S., Arefi, R., and Ameri, S.: "Design and Development of an Artificial Neural Network for Prediction of Formation Permeability,"paper SPE 28237 presented at the 1994 SPE Petroleum Computer Conference, July31-Aug. 3, Dallas.

With the development of the Internet, the Intrusion Detection has been gradually playing a more and more important role in Network Security. Radial Basis Function Neural Network are widely used in Intrusion Detection, especially Probabilistic Neural Network. However, the detection speed is a problem which impedes it to be applied to Real-time Intrusion Detection. In this paper, for increasing the Detection Speed, the Incremental Training Method replaces the Exact Training Method. The simulation experiment shows that the detection speed of Incremental Probabilistic Neural Network is much faster than that of Exact Probabilistic Neural Network. Therefore, the Incremental Probabilistic Neural Network is more suitable for real-time intrusion detection than Exact Probabilistic Neural Network.

A neural network is a massively parallel distributed processor made up of simple processing units known as neurons. These neurons are organized in layers and every neuron in each layer is connected to each neuron in the adjacent layers. This connection architecture makes for an enormous number of communication links between neurons This is a major issue when considering a hardware implementation of a neural network since communication links take up hardware space, and hardware space costs money. To overcome this space problem incremental communication for multilayer neural networks has been proposed. Incremental communication works by only communicating the change in value between neurons as opposed to the entire magnitude of the value. This allows for the numbers to be represented with a fewer number of bits, and thus can be communicated with narrower communication links. To validate the idea of incremental communication an incremental communication neural network was designed and implemented, and then compared to a traditional neural network. From the implementation it is seen that even though the incremental communication neural network saves design space through reduced communication links, the additional resources necessary to shape the data for transmission outweighs any design space savings when targeting a modern FPGA.

For the cloud computing based on software-defined networks (SDNs), a larger amount of data is collected to cloud for analysis, which will cause the larger amount of redundancy data and longer service response time due to the capacity-limited Internet. To solve this problem, a novel service orchestration and data aggregation framework (SODA) is proposed, which can orchestrate data as services and aggregate data packets to reduce data redundancy and service response delay. In SODA, the network is divided into three layers. 1) Data centers layer (DCL). Data centers (DCs) release software with a specific function to all devices in the network, devices orchestrate data as services and aggregate data packets using software to reduce service response delay. 2) Middle routing layer (MRL). The routing path of data packets in this layer is adjusted according to the correlation of data packets and routing distance. The correlation of data packets is higher and routing distance is short, the probability that data packets are transmitted along the same routing path is higher to reduce redundancy data. 3) Vehicle network layer (VNL). Mobile vehicles are used to transmit data packets and services among devices. A series of experiments and simulation is conducted. The results illustrate that the proposed scheme has better performance compared with the traditional scheme.

The research reported in this paper is part of a larger project on the automatic classification of Web pages by their genres, using a distance function classification model. In this paper, we investigate the effect of several commonly used data preprocessing steps, explore the use of byte and word n-grams, and test our classification model on three Web page data sets. Our approach is to represent each Web page by a profile that is composed of fixed-length n-grams and their normalized frequencies within the document. Similarly, each of the genres in a data set is represented by a profile that is constructed by combining the n-gram profiles for each exemplar Web page of that genre, forming a centroid profile for each Web page genre. We use a distance function approach to determine the similarity between two profiles, assigning each Web page the label of the genre profile to which its profile is most similar. Our results compare very favorably to those of other researchers.

Clustering techniques have been applied to categorize documents on Web and extract knowledge from Web. In this paper, we introduce a novel clustering method into Web page clustering, which is an extension of affinity propagation (AP). This method is called partition adaptive affinity propagation (PAAP), which can automatically rerun AP procedure to yield optimal clustering results and eliminate number oscillations if they occur. Experiments are carried out to compare PAAP with K-means and AP on ten different Web page data sets. The results verify that PAAP can find better clusters when compared with similar methods. And the results also demonstrate that PAAP is robust and effective when clustering Web pages.

Recent advances on deep neural networks, especially in combination with generative models, provide the means to deal with the large amounts of data and noisiness in the neuroimaging domain. This chapter provides an overview of state-of-the-art techniques to learn complex features from brain signal recordings that capture processes underlying auditory imagery. An introduction to important deep learning techniques and their application to auditory and neuroimaging data are given. Special attention is paid to generative models and reverse inference. These models allow to build new research paradigms, such as stimulus reconstruction from multiple sensory modalities. We discuss how models trained on sensory data can be used for inference about imagery processes, which are not directly associated with the auditory stimulus and thus make complex demands to machine learning models that traditionally require large amounts of annotated data. Deep neural networks generally require large amounts of training data, while the amount and size of available datasets are small. This motivates a description of requirements for new datasets in the field. Based on these considerations, a final section discusses possible future improvements for deep neural networks in the auditory imagery domain, such as biologically inspired computational models.

This article discusses how anomaly detection is an important problem in various areas of software products in the modern world, anomaly detection can be useful in cybersecurity, the Internet of Things, and financial transaction analysis. Above all, anomalies can signal the need to take some action to avoid negative consequences. In addition, the importance of anomaly detection for business intelligence and risk management is explored. Much attention is paid to the study of different types of anomalies, including point, contextual and collective, with examples in different contexts. The importance of using intelligent machine learning algorithms to detect anomalies in large amounts of data and quickly process information with warnings to staff is emphasized. Anomaly detection using machine learning is an urgent problem in the modern world when dealing with large amounts of data and ever-growing threats in the field of cybersecurity, financial fraud, medical diagnostics, industrial safety and other industries. With the proliferation of the Internet of Things (IoT) and the large amount of data it generates, detecting unusual, anomalous, or suspicious events is becoming increasingly challenging for traditional data processing methods. Machine learning automates the anomaly detection process by using algorithms to analyze and classify data. This improves the efficiency and speed of anomaly detection, reduces the cost of manual analysis, and facilitates a more accurate and rapid response to potential threats or issues. With the in-depth development of machine learning technologies such as neural networks, deep learning algorithms, and the constant growth of machine learning models, anomaly detection capabilities are becoming more accurate and diverse. This makes it possible to detect anomalies in real time and ensure a reliable level of security in various fields of activity, which is extremely important in today's digital world. There are three situations in which the algorithm can be applied: supervised learning, semi-supervised learning, and unsupervised learning. The classification is based on algorithmic access, including probabilistic methods, distance and density methods, clustering methods, activity-based methods, and reconstruction and spectral methods. To choose the best approach to anomaly detection, it is important to consider various factors. The article provides illustrative examples of anomaly detection algorithms based on real data.

<bold>Background</bold>Neural networks, with their powerful fitting capabilities, can learn the relationships between input and output variables based on large amounts of data, often serve as proxy models for physical programs in the field of engineering calculations, including nuclear engineering calculations. Neutron transport calculations, as one of the core links in neutronics simulations, are often suffer from lengthy computational times. However, this issue can also be addressed by utilizing neural network models. Nevertheless, neural network models have a series of hyperparameters that need to be set, but manually adjusting these hyperparameters is laborious, repetitive, and reliant only on experience. Moreover, these hyperparameters are not reusable when solving different problems.<bold>Purpose</bold>This study aims to propose a novel algorithm to improve computational efficiency of core neutronic parameter prediction, and provide reference for the application of artificial intelligence in core physics calculation.<bold>Methods</bold>The dataset was derived from the VITAS program's calculations of the TAKEDA1 and TAKEDA2 benchmark problems, consisting of 10 000 and 20 000 sets respectively. The core layout sequence was taken as input whilst the output included the effective multiplication factor and the regional integral flux. Thereafter, the core layout sequence was mapped into a one-dimensional vector, and the dataset was divided into training and validation sets in a 6:4 ratio. Then, the Bayesian optimization algorithm was employed to adjust fully connected neural network (FCNN) hyperparameters, and the adaptive learning rate decay and loss function optimization methods were combined to automatically search for the optimal combination of hyperparameters for datasets with different problems. Finally, critical core parameters, i.e., the effective multiplication factor and the regional integrated flux, were fitted to valid the generalization and accuracy of abovementioned algorithm, and a comparative experiment was conducted using manually set hyperparameters and hyperparameters output by Bayesian optimization as neural network training parameters.<bold>Results</bold>The results demonstrate that Bayesian optimization effectively enhances the neural network's accuracy. Specifically, the average error of the effective multiplication factor is within 1.50×10^-3, while that of the regional integral flux on the TAKEDA1 dataset is 1.72%, with a maximum error rate of 7.56%.<bold>Conclusions</bold>This approach can automatically search for the optimal combination of hyperparameters for different datasets to achieve the best performance, demonstrating high flexibility, efficiency, and strong generalization.

In this paper, a neural network is used to implement a generalized cost function for a genetic algorithm (GA). Traditional GAs are inefficient because a large amount of data which describes the problem space is discarded after each generation. Using the neural network enhanced genetic algorithm (NNEGA), this redundant information is fed back into the GAs' cost function via the neural network. The neural network learns the optimal weights of the cost function by identifying trends and compromising weights depending on the knowledge that it accumulates in-situ. The NNEGA is applied to an array antenna design problem for verification. To ensure convergence, the output of the neural network is only fed back to the cost function after a certain number of generations.

This paper presents some new results on the generalization of incremental learning radial basis function neural networks which are trained with selected significant samples from the input space. Our main focus is to show that we need to pick the right proportion of significant samples from the input space which not only generate an optimal size network but also ensure an acceptable generalization accuracy for an application. Experimental results on these data sets reveal that training with significant patterns of various proportions has greater influence on the generalization ability of the RBF networks.

The work is devoted to the use of artificial neural networks to solve the problem of recognition and isolation of objects (cells) in digital micrographs used in the practice of microbiological research. This task is relevant due to a combination of two factors: a large amount of data (hundreds and thousands of photographs, hundreds of objects on each) arising from such studies, as well as the high complexity of manual image processing and the risk of operator errors. To solve the problem of recognition and separation of objects (cells) in digital biomedical micrographs, artificial convolutional neural networks are used. However, in most cases, the use of artificial neural networks requires selection of the network architecture and its training for each task individually. Each time when solving the problem of training the network, it is necessary to collect huge training samples, the more complex the more complex the architecture. The task of cell isolation was solved using a convolutional neural network. The neural network architecture of Mask R-CNN Facebook Research was used. We used a pretrained neural network, retrained in digital micrographs obtained and marked out by the authors. The solution is implemented in Python using TensorFlow, an open source machine learning software library developed by Google. As a result of testing the system, on the available data, the correctness of cell recognition in microphotographs was more than 95%, despite the small size of the digital images used. The applied approach definitely showed its efficiency on the available experimental data and has development prospects in the direction of increasing the size of processed images, increasing recognition accuracy, expanding the composition of secreted objects, working not only with cells, but also with structures in tissues.

Wireless sensing has attracted considerable attention because it can sense the state of the targets by analyzing the surrounding wireless signals, which has become the key role of the artificial intelligence of things (AIoT). As the number of sensory nodes increases, large amounts of redundant data are exchanged between sensory terminals and the AI cloud. To process such large amounts of data efficiently and decrease power consumption, a machine-learning approach that operates close to or inside sensors must be developed. To this end, we present the radio-frequency neural network (RFNN), a physical neural network taking advantage of a group of transmissive intelligent surfaces (i.e., metasurfaces) to mimic the computations of a fully-connected neural network. The design is spurred by the capability of RFNNs to perform expensive multiplication and additions at the speed of light, with ultra-low power consumption. We prototype RFNN at 5 GHz for WiFi sensing regarding nine wireless sensing tasks. Extensive evaluations demonstrate the comparably equivalent inference ability as the conventional electronic neural networks while consuming less energy.

In recent years, ICSs have become increasingly commonplace in virtually every industry. The abbreviation “ICSs” refers to industrial control systems. These are specially designed computers used for monitoring, managing, and controlling procedures and tasks across a wide range of industries and vital infrastructure sectors. Production, power, disinfection of water, transport, and other sectors all greatly benefit from ICS use. The authors of this paper aim to detect ICS cyber hazards in industry. This article is the result of the writers’ extensive research on ICS programs and the impact of cyberattacks on them as well. The study narrowed its attention to just three ICS applications because there are simply too many to count: power plants, water reservoirs, and gas pipelines. The present paper focuses on the development and evaluation of neural networks for use in cyberattacks. An early form of neural network, the residual system, came first in the field. When a breach is detected in the ICS, the neural network sorts it into one of several categories. The produced datasets must not compromise users’ privacy or cause harm to the relevant industry if they fall into the wrong hands. An encoding device, decoder, pseudo-encoder, and critical model neural networks work together to generate random data. Finally, a set of trials is conducted in which a residual neural network is utilized to classify cyberattacks based on both the created and original datasets. Results from a series of studies indicate that using the created dataset is an effective technique to train high-quality neural networks for use in cybersecurity on a large amount of data without sacrificing the accuracy of the models. The Kullback-Leibler and Jensen-Shannon divergences also serve as the theoretical foundation and technique, respectively. In particular, the paper recommends operational and maintenance cybersecurity standards for ICS. This entails such things as secure password practices, patch management, and anti-malware defense. Physical safeguards for ICS is another topic that is covered.

Decision Trees (DTs) like LambdaMART have been one of the most effective types of learning-to-rank algorithms in the past decade. They typically work well with hand-crafted dense features (e.g., BM25 scores). Recently, Neural Networks (NNs) have shown impressive results in leveraging sparse and complex features (e.g., query and document keywords) directly when a large amount of training data is available. While there is a large body of work on how to use NNs for semantic matching between queries and documents, relatively less work has been conducted to compare NNs with DTs for general learning-to-rank tasks, where dense features are also available and DTs can achieve state-of-the-art performance. In this paper, we study how to combine DTs and NNs to effectively bring the benefits from both sides in the learning-to-rank setting. Specifically, we focus our study on personal search where clicks are used as the primary labels with unbiased learning-to-rank algorithms and a significantly large amount of training data is easily available. Our combination methods are based on ensemble learning. We design 12 variants and compare them based on two aspects, ranking effectiveness and ease-of-deployment, using two of the largest personal search services: Gmail search and Google Drive search. We show that direct application of existing ensemble methods can not achieve both aspects. We thus design a novel method that uses NNs to compensate DTs via boosting. We show that such a method is not only easier to deploy, but also gives comparable or better ranking accuracy.