Learning behaviors for environmental modeling by genetic algorithm

Evolutionary behavior learning for action-based environment modeling by a mobile robot

For educational purposes there is a need to teach electrical and computer engineering students the basics of the design of state machines using programmable logic devices, and for students interested in mobile robots to teach them the basics of mobile robots' behaviors. At the same time one of the topics of interest in the mobile robot's community is how to generate their new behaviors, using state machines, automatically. This paper discusses how to create mobile robots' behaviors using genetic algorithms (GAs), implementing them in programmable logic devices (FPGAs) and how these behaviors can be programmed by students in a mobile robotics course. The behaviors are encoded as algorithm state machines and using feed-forward artificial neural networks (ANN). In the state-machine approach, each individual's chromosome represents, given a set of inputs coming from the sensors and the current state, both the next state and the outputs that control the robot's movements. In the ANN approach, whose weights are found also with GAs, a pipeline architecture was built to perform it, each pipeline executes a layer of the ANN, thus once the pipeline is full the execution speed of the ANN is one sensors' clock cycle. We evaluate the behaviors generated by the GA according to a fitness function that grades their performance for avoiding obstacles. The inputs to such a robot are infrared sensors to detect obstacles, the outputs are the velocities of its wheels. Our objectives are first, to prove that GAs are a good option as a method for finding behaviors for mobile robots' navigation, and second, that these behaviors can be implemented in an efficient way in FPGAs. We tested both behaviors in a small mobile robot, that is build in an electrical engineering course that teaches how to build mobile robots.

To design behaviors of a mobile robot for realizing given tasks, a designer has to make a set of rules which generate a proper action from a state of sensors. In general, however, it is difficult that the designer makes all set of rules since the number of rules is very large and the proper action for a state of sensors is not clear. Many methods have been proposed to solve such a problem using reinforcement learning genetic programming or genetic algorithm and so on. However, those methods are not sufficiently applied to the case in which the environment changes or is unknown. In this paper, new method which is efficiently applied to such a difficult case is proposed. The proposed method is constructed by genetic algorithm and action rule-base. The experimental results, using a mobile robot simulator, shows that the mobile robot can properly act in the unknown environments.

This paper had done a crowd evacuation simulation based on a hierarchical environmental model. An environment modeling software had been implemented which can be used to construct an environment model consisted of geometric level, topological level, path level and cognitive level. The environment model can provide many supports for visual effect, sensing, path planning, decision-making, and interaction et al, which turned the large scale crowd evacuation simulation into truth.

Establishing three different environment models (i.e., single reservoir model, common reservoir model and independent reservoir model), we investigate the evolutional characteristic of the entangled system initially in Bell-like state by the pseudomode method. Through comparing the entanglement decays under the three different environment models, we find that the entangled Bell-like state cos θ |ee> + sin θ |gg> will be kept in the single reservoir model for a time longer than in the other two models. However, the entanglement decay behavior of Bell-like state cos θ |eg> + sin θ |ge> is dependent on the specific environment model. Meanwhile, the comparison of entanglement decay between the above two Bell-like states at the same environment models is performed.

We present an evolutionary approach for reconstructing CT images; the algorithm reconstructs two-dimensional unknown images from four one-dimensional projections. A genetic algorithm works on a randomly generated population of strings each of which contains encodings of an image. The traditional, as well as new, genetic operators are applied on each generation. The mean square error between the projection data of the image encoded into a string and original projection data is used to estimate the string fitness. A Laplacian constraint term is included in the fitness function of the genetic algorithm for handling smooth images. Two new modified versions of the original genetic algorithm are presented. Results obtained by the original algorithm and the modified versions are compared to those obtained by the well-known algebraic reconstruction technique (ART), and it was found that the evolutionary method is more effective than ART in the particular case of limiting projection directions to four.

Average local image entropy can be seen as an effective measure of a training environments potential toward producing highly fit robots. As seen in Fig.6 the measure is sensitive to different window values but in our application a wide range of values could have been used. Our selection of neighborhood size was a reasonable as only very low values (around 10) show poor discriminancy. Fig. 7 shows how the average fitness obtained during testing is clearly dependent on the diversity of the training room used. A training environment with too much environmental diversity is as unsuitable as one with not enough diversity. Higher motivation diversity (H ( M)) caused lower average score values. This is possibly due to the different and conflicting (e.g. orthogonal) requirements of the different motivations upon the robotic neural brain. These scores are also negatively impacted by the fitness function used (4) given that it includes a motivation satisfaction product (mn × f n ) in each rule that further penalizes lower behavior fitnesses ( f n ). Even though obtained fitness was generally lower with more diverse motivations, the obtained behaviors demonstrated very good capability (e.g. room exploration, battery usage, etc) and were in agreement with the specified motivations. Because of this, motivation diversity results are somewhat counter intuitive in that by diversifying motivation values one could naively expect higher overall fitness but this is clearly not the case due to system constraints.

Purpose The purpose of this paper is to consider the smooth path planning problem for a mobile robot based on the genetic algorithm (GA) and the Bezier curve. Design/methodology/approach The workspace of a mobile robot is described by a new grid-based representation that facilitates the operations of the adopted GA. The chromosome of the GA is composed of a sequence of binary numbered grids (i.e. control points of the Bezier curve). Ordinary genetic operators including crossover and mutation are used to search the optimum chromosome where the optimization criterion is the length of a piecewise collision-free Bezier curve path determined by the control points. Findings This paper has proposed a new smooth path planning for a mobile robot by resorting to the GA and the Bezier curve. A new grid-based representation of the workspace has been presented, which makes it convenient to perform operations in the GA. The GA has been used to search the optimum control points that determine the Bezier curve-based smooth path. The effectiveness of the proposed approach has been verified by a numerical experiment, and some performances of the obtained method have also been analyzed. Research limitations/implications There still remain many interesting topics, for example, how to solve the specific smooth path planning problem by using the GA and how to promote the computational efficiency in the more grids case. These issues deserve further research. Originality/value The purpose of this paper is to improve the existing results by making the following three distinctive contributions: a rigorous mathematical formulation of the path planning optimization problem is formulated; a general grid-based representation (2n × 2n) is proposed to describe the workspace of the mobile robots to facilitate the implementation of the GA where n is chosen according to the trade-off between the accuracy and the computational burden; and the control points of the Bezier curve are directly linked to the optimization criteria so that the generated paths are guaranteed to be optimal without any need for smoothing afterwards.

In mobile robot research domain and especially in its path planning sub-domain, the optimization tools the most used are PSO (Particle Swarm Optimization) and GA (Genetic Algorithms). In fact, mobile robot path planning is a multi-objective optimization problem where at least these objectives are considered: the shortest and the safest path to the goal. Sometimes other objectives are added like travel duration. The subject of this paper is a comparison investigation concerning the efficiency of PSO and GA in mobile robot path planning optimization. We start with a comparison of evolutionary performances of PSO and GA. Then, we apply PSO and GA in the optimization of parameters of DVSF\(^{2}\) (Dynamic Variable Speed Force Field) path planning approach. Different environments of different types, statics and dynamics are considered in these simulations to decide which is better for mobile robot path planning optimization: PSO or GA.

We describe in this paper the application of evolutionary methods for the optimization of type-2 fuzzy logic controllers. These optimal type-2 fuzzy logic controllers are used for the trajectory tracking control of autonomous mobile robots. The evolutionary method to consider is the genetic algorithm, presenting simulations results in Simulink© of Matlab.

Purpose– The purpose of this paper is to explore a novel control approach for swift and accurate positioning and tracking of a mobile robot. Coordinated movement of the mobile robot-body and chameleon-inspired binocular “negative correlation” visual system (CIBNCVS) with neck has rarely been considered in conventional mobile robot design. However, it is vital in swift and accurate positioning and tracking of the target. Consequently, it is valuable to find an optimized method where the robot-body, the biomimetic eyes and neck could achieve optimal coordinated movement.Design/methodology/approach– Based on a wheeled mobile robot, a biomimetic dual Pan–Tilt–Zoom visual system with neck is constructed. The cameras can rely on the unique “negative correlation” mode of chameleon vision, and cooperate with neck, achieving swift search of the 160° scope in front of the robot. Genetic algorithm is used to obtain optimal rotation of the neck and robot-body. Variable resolution targeting is also applied for accurate aiming. Using these two approaches, we can achieve efficient targeting with low energy consumption. Particle filter algorithm is further utilized for real-time tracking.Findings– In the proposed approach, swift and accurate positioning and tracking of the target can be obtained. The rationality of the approach is verified by experiments on flat and sandy terrains with satisfactory results.Originality/value– This paper proposes a novel control approach for wheeled mobile robots, which achieves coordinated movement of the robot-body and CIBNCVS with neck concerning time and energy saving in the process of swift and accurate tracking.

This paper presents a localization algorithm for indoor environments. The environmental model is topological and the approach describes how a multimodal perception increases the reliability for the topological localization problem for mobile robots, by using the Bayesian Programming formalism. For the topological framework the fingerprint concept is used. This type of representation permits a reliable and distinctive environment modeling. Experimental results of a mobile robot equipped with a multi sensor system composed of two 180° laser range finders and an omni-directional camera are reported.

This paper describes a new method for indoor environment mapping and localization with stereo camera. For environmental modeling, we directly use the depth and color information in image pixels as visual features. Furthermore, only the depth and color information at horizontal centerline in image is used, where optical axis passes through. The usefulness of this method is that we can easily build a measure between modeling and sensing data only on the horizontal centerline. That is because vertical working volume between model and sensing data can be changed according to robot motion. Therefore, we can build a map about indoor environment as compact and efficient representation. Also, based on such nodes and sensing data, we suggest a method for estimating mobile robot positioning with random sampling stochastic algorithm. With basic real experiments, we show that the proposed method can be an effective visual navigation algorithm.

This work investigates the problem of finding near-optimal strategies for the bilateral negotiation with incomplete information between two competitive negotiation agents for resolving the differences in their objectives and preferences. While there are some former studies of finding negotiation strategies using evolutionary approaches, there are extremely few works on effectively locating the global optimal solution. In this study, we use a genetic algorithm (GA) to find near-optimal negotiation strategies for both negotiation agents conducting bilateral negotiation by coevolving both agents' strategies. Agents learn optimal strategies through trial-and-error, which is done by matching and negotiation of agents of one population with agents of the other population through random pairing in a one-to-one manner. However, a simple GA often cannot find the optimum results for both agents for the following reasons: (i) the premature convergence of a simple GA in itself, and (ii) both parties' failure to keep pace with each other's learning speed in the coevolution. To solve this problem, this paper proposes a GA which has a novel dynamic diversity controlling capability. The proposed method utilizes the accumulated frequency information of the occurrence of individuals in each band of the population for generations. The information is used in a diversification and refinement procedure of the proposed diversity controlling GA. Empirical results show that the proposed dynamic diversity controlling GA generally outperforms the traditional GA for finding near-optimal negotiation strategies.

The object of the study is a mobile walking robot with two or more movable articulated limbs. The concept of a "critical condition" is introduced, in which the mechanism balances on the verge of falling (but does not fall) or there is a possibility of damage to mechanical components due to the generation of unacceptable joint angles. The subject of the study is a general algorithm for the elimination of critical conditions, which provides the possibility of training an agent based on a deep reinforcement learning algorithm directly on a real robot, without the risk of damaging its mechanisms and interrupting the process of interaction with the environment to restore a stable state. The purpose of this work is to develop a general algorithm for the elimination of critical conditions in the context of adaptive control of a walking robot based on deep learning algorithms with reinforcement. A comparison was made between the proposed and standard methods of applying deep OP on a real robot. The experiments were conducted on 6,000 episodes, with a dimension of 300 steps each. The following quality metrics were selected for evaluation: the percentage of episodes without an actual fall, the percentage of fully completed episodes, and the maximum episode length. The algorithm is based on the concept of "critical condition" and uses the following principles and methods: the "trial and error" method, the feedback principle, holding the projection of the center of gravity point in the area of the polygon formed by the points of contact of the limbs with the work surface, which ensures the balancing of the structure and allows you to determine the boundary areas in which the robot is still stable. The scientific novelty of the work lies in the proposed approach, which allows an intelligent agent to control a physical robot "directly", without pre-configuration in a simulation environment with subsequent transfer implementation. The proposed algorithm is not aimed at improving the agent's performance, but is intended to provide greater autonomy in the learning process of the robot, directly in the hardware. The basic idea is to immediately respond to a critical condition in the form of the fastest sequential return to a certain number of steps back along the decision-making trajectory, ensuring that the agent remains in a stable and safe state at all times. The method of proximal policy optimization (PPO) was used as a method of deep reinforcement learning. As a result of the comparative analysis, the proposed algorithm demonstrated a hundredfold increase in the stability of the mechanism.