Heterogeneous Multi-agents Learning Using Genetic Network Programming with Immune Adjustment Mechanism

In this paper, we propose a variant of Genetic Network Programming (GNP) which has multiple start nodes. GNP is a kind of evolutionary methods inspired from Genetic Programming (GP). While GP has tree architecture, GNP has a network architecture by which a single agent is controlled. In this paper, by increasing the number of start nodes in a network of GNP, we extend GNP to control multiple agents. We applied the extended GNP called Multi-Start Node GNP (MS-GNP) to a load transportation problem and a tile world problem.

Genetic Network Programming (GNP) is an evolutionary approach which can evolve itself and find the optimal solutions. It is based on the idea of Genetic Algorithm and uses the data structure of directed graphs. Many papers have demonstrated that GNP can deal with complex problems in the dynamic environments very efficiently and effectively. As a result, recently, GNP is getting more and more attentions and is being used in many different areas such as data mining, extracting trading rules of stock markets, elevator systems, etc and GNP has obtained some outstanding results. In order to improve GNP's performance further, this paper proposes a new method called GNP with Rules. The aim of the proposed method is to balance exploitation and exploration of GNP, that is, to strengthen exploitation ability by using the exploited information extensively during the evolution process of GNP. The proposed method consists of 4 steps: rule extraction, rule selection, individual reconstruction and individual replacement. These 4 steps are added to the conventional algorithm of GNP. In order to measure the performance of the proposed method, the tileworld was used as the simulation environment. The simulation results show some advantages of GNP with Rules over conventional GNPs.

Genetic Network Programming (GNP) is a newly proposed evolutionary approach which can evolve itself and find the optimal solutions. It is a novel method based on the idea of Genetic Algorithm (GA) and uses the data structure of directed graphs. As GNP has been developed for dealing with problems in dynamic environments, many papers have demonstrated that GNP can be applied to many areas such as data mining, forecasting stock markets, elevator control systems, etc. Focusing on GNP's distinguished expression ability of the graph structure, this paper proposes a method named Genetic Network Programming with Reconstructed Individuals (GNP with RI). In the proposed method, the worst individuals are reconstructed and enhanced by the elite information before undergoing genetic operations (mutation and crossover). The enhancement of worst individuals mimics the maturing phe nomenon in nature, where bad individuals can become smarter after receiving good education. GNP with RI has been applied to the the-world which is an excellent benchmark for evaluating the proposed architecture. The performance of GNP with RI is compared with conventional GNP demonstrating its superiority.

Among several methods of extracting association rules that have been reported, a new evolutionary method named Genetic Network Programming (GNP) has also shown its effectiveness for dense databases. However, the conventional GNP data mining method can not find comparative relations and hidden patterns among a large amount of data. In this paper, we present a method of comparative association rules mining using Genetic Network Programming (GNP) with attributes accumulation mechanism in order to uncover comparative association rules between different datasets. GNP is an evolutionary approach recently developed, which can evolve itself and find the optimal solutions. The objective of the comparative association rules mining is to check two or more databases instead of one, so as to find the hidden relations among them. The proposed method measures the importance of association rules by using the absolute values of the confidence differences of the rules obtained from different databases and can get a number of interesting rules. Association rules obtained by comparison can help us to find and analyze the explicit and implicit patterns among a large amount of data. On the other hand, the calculation is very time-consuming, when the conventional GNP based data mining is used for the large attributes case. So, we have proposed an attributes accumulation mechanism to improve the performances. Then, the comparative association rules mining using GNP has been applied to a complicated traffic system. By mining and analyzing the rules under different traffic situations, it was found that we can get interesting information of the traffic system.

Many methods of generating behavior sequences of agents by evolution have been reported. A new evolutionary computation method named Genetic Network Programming (GNP) has been also developed recently along with these trends. GNP has a directed graph structure and the search for obtaining optimal GNP becomes difficult when the scale of GNP is large. The aim of this paper is to find a well structured GNP considering Breadth and Depth of GNP searching. It has been shown that the proposed method is efficient compared with conventional GNPs from simulations using a garbage collector problem.

Many methods of generating behavior sequences of agents by evolution have been reported. A new evolutionary computation method named genetic network programming (GNP) has been also developed recently along with these trends. GNP has a directed graph structure and the search for obtaining optimal GNP becomes difficult when the scale of GNP is large. The aim of this paper is to find a well structured GNP considering Breadth and Depth of GNP searching. It has been shown that the proposed method is efficient compared with conventional GNPs from simulations using a garbage collector problem.

Computer systems are exposed to an increasing number and type of security threats due to the expanding of Internet in recent years. How to detect network intrusions effectively becomes an important techniques. This paper presents a novel fuzzy class association rule mining method based on Genetic Network Programming (GNP) for detecting network intrusions. GNP is an evolutionary optimization techniques, which uses directed graph structures as genes instead of strings (Genetic Algorithm) or trees (Genetic Programming), leading to creating compact programs and implicitly memorizing past action sequences. By combining fuzzy set theory with GNP, the proposed method can deal with the mixed database which contains both discrete and continuous attributes. And it can be flexibly applied to both misuse and anomaly detection in Network Intrusion Detection Problem. Experimental results with KDD99Cup and DAPRA98 databases from MIT Lincoln Laboratory show that the proposed method provides a competitively high detection rate compared with other machine learning techniques.

In this paper, the Genetic Network Programming (GNP) for creating trading rules on stocks is described. GNP is an evolutionary computation, which represents its solutions using graph structures and has some useful features inherently. It has been clarified that GNP works well especially in dynamic environments since GNP can create quite compact programs and has an implicit memory function. In this paper, GNP is applied to creating a stock trading model. There are three important points: The first important point is to combine GNP with Sarsa Learning which is one of the reinforcement learning algorithms. Evolution-based methods evolve their programs after task execution because they must calculate fitness values, while reinforcement learning can change programs during task execution, therefore the programs can be created efficiently. The second important point is that GNP uses candlestick chart and selects appropriate technical indices to judge the buying and selling timing of stocks. The third important point is that sub-nodes are used in each node to determine appropriate actions (buying/selling) and to select appropriate stock price information depending on the situation. In the simulations, the trading model is trained using the stock prices of 16 brands in 2001, 2002 and 2003. Then the generalization ability is tested using the stock prices in 2004. From the simulation results, it is clarified that the trading rules of the proposed method obtain much higher profits than Buy&Hold method and its effectiveness has been confirmed.

Genetic network programming (GNP) is an evolutionary approach which can evolve itself and find the optimal solutions. As many papers have demonstrated that GNP which has a directed graph structure can deal with dynamic environments very efficiently and effectively. It can be used in many areas such as data mining, forecasting stock markets, elevator system problems, etc. In order to improve GNPpsilas performance further, this paper proposes a method called GNP with Rules. The aim of the proposal method is to balance exploitation and exploration, that is, to strengthen exploitation ability by using the exploited information extensively during the evolution process of GNP. The proposal method consists of 4 steps: rule extraction, rule selection, individual reconstruction and individual replacement. Tile-world was used as a simulation environment. The simulation results show some advantages of GNP with Rules over conventional GNPs.

Genetic Network Programming (GNP) is one of the evolutionary algorithms. It adopts a directed graph structure to represent a solution to a given problem. Agents judge situations and execute actions sequentially following the node transitions in the graph. On one hand, GNP possesses an advantage of node reusability, which makes it possible to realize a compact graph structure that represents a solution. On the other hand, the compact structure suggests that any connection might play a significant role in the solution, i.e., a slight change to the connections could tremendously influence the performance of the agents for the given task. The conventional GNP, however, lacks an effective way to evaluate and to take advantage of the connections. This paper thus proposes a reinforcement learning approach to learn GNP's subgraphs that contain a relatively small number of connections, and further proposes a partial reconstruction approach to modify the solution with the obtained subgraphs. These two approaches are combined together to form a new evolutionary learning model named GNP with Evolution-oriented Reinforcement Learning (GNP-ERL). Some experiments are conducted on the Tileworld testbed to verify the effectiveness of GNP-ERL, and the simulation results demonstrate that it outperforms the conventional GNP in both training and testing phases.

In this paper, the Genetic Network Programming (GNP) for creating trading rules on stocks is described. GNP is an evolutionary computation, which represents its solutions using graph structures and has some useful features inherently. It has been clarified that GNP works well especially in dynamic environments since GNP can create quite compact programs and has an implicit memory function. In this paper, GNP is applied to creating a stock trading model. There are three important points: The first important point is to combine GNP with Sarsa Learning which is one of the reinforcement learning algorithms. Evolution-based methods evolve their programs after task execution because they must calculate fitness values, while reinforcement learning can change programs during task execution, therefore the programs can be created efficiently. The second important point is that GNP uses candlestick chart and selects appropriate technical indices to judge the timing of the buying and selling stocks. The third important point is that sub-nodes are used in each node to determine appropriate actions (buying/selling) and to select appropriate stock price information depending on the situation. In the simulations, the trading model is trained using the stock prices of 16 brands in 2001, 2002 and 2003. Then the generalization ability is tested using the stock prices in 2004. From the simulation results, it is clarified that the trading rules of the proposed method obtain much higher profits than Buy&Hold method and its effectiveness has been confirmed.

Genetic Network Programming(GNP) is extension of Genetic Algorithm(GA). The GNP is suitable for an agent programming. The GNP can evolve anoperation program of the agent. But, the GNP has the premature convergence problem as an evolution technique as well as GA. On the other hand, to avoid the initial convergence of the GA, Immune Alogorithm(IA) which is introduced the immune suppression to the GA had developed.Then if the GNP and IA are combined, a better algorithm for agent programming can be developed. Authors proposed Immune evolved Genetic Network Programming (IGNP). To compare the GNP and IGNP, the simmulation was done. As a result, IGNP is more exellent without the premature convergence than the GNP.In this paper, the authors explain the IGNP outline and a effectiveness of the IGNP is stated through the simulation result.

Genetic Network Programming (GNP) is an evolutionary algorithm which represents its solutions using graph structures. Since GNP can create quite compact programs and has an implicit memory function, GNP works well especially in dynamic environments. In addition, a study on creating trading rules on stock markets using GNP with Importance Index (GNP-IMX) has been done. IMX is one of the criterions for decision making. However, the values of IMXs must be deteminined by our experience/knowledge. Therefore in this paper, IMXs are adjusted appropriately during the stock trading in order to predict the rise and fall of the stocks. Moreover, newly defined flag nodes are introduced to GNP, which can appropriately judge the current situation of the stock prices, and also contributes to the use of many kinds of nodes in GNP program. In the simulation, programs are evolved using the stock prices of 20 companies. Then the generalization ability is tested and compared with GNP without flag nodes, GNP without IMX adjustment and Buy&Hold.

Double-deck elevator system (DDES) has been invented firstly as a solution to improve the transportation capacity of elevator group systems in the up-peak traffic pattern. The transportation capacity could be even doubled when DDES runs in a pure up-peak traffic pattern where two connected cages stop at every two floors in an elevator round trip. However, the specific features of DDES make the elevator system intractable when it runs in some other traffic patterns. Moreover, since almost all of the traffic flows vary continuously during a day, an optimized controller of DDES is required to adapt the varying traffic flow. In this paper, we have proposed a controller adaptive to traffic flows for DDES using genetic network programming (GNP) based on our past studies in this field, where the effectiveness of DDES controller using GNP has been verified in three typical traffic patterns. A traffic flow judgment part was introduced into the GNP framework of DDES controller, and the different parts of GNP were expected to be functionally localized by the evolutionary process to make the appropriate cage assignment in different traffic flow patterns. Simulation results show that the proposed method outperforms a conventional approach and two heuristic approaches in a varying traffic flow during the work time of a typical office building.

In this paper, the enhancement of trading rules on stock markets using genetic network programming (GNP) with Sarsa Learning is described. There are three important points in this paper: First, we use GNP with Sarsa learning as the basic algorithm while importance Indices and Candlestick Charts are introduced for efficient stock trading decision-making. Importance indices have been proposed to tell GNP the timing of buying and selling stocks. Second, to improve the performance of the proposed GNP-Sarsa algorithm, we develop a new method that can learn appropriate function describing the relation between the value of each technical index and the output of the importance index (IMX). This is an important point that devotes to the enhancement of the proposed GNP-Sarsa algorithm.Third, in order to create more efficient judgment functions to judge the current stock price appropriately, we develop a new way of classifying the candlestick chart body type. To confirm the effectiveness of the proposed method, we also compare the simulation results using GNP-Sarsa with other methods like traditional GNP and Buy&Hold method.