Efficient Numerical Optimization Algorithm Based on New Real-Coded Genetic Algorithm, AREX + JGG, and Application to the Inverse Problem in Systems Biology

In this paper, a novel and efficient real-coded genetic algorithm (RCGA) for process optimization is developed. The proposed RCGA is equipped with Ranking Selection (RS), Direction-Based Crossover (DBX) and Dynamic Random Mutation (DRM) operators. The RS operator is used to eliminate the bad solutions and reproduce good solutions, making the whole population to achieve a better average fitness. The DBX operator uses relative fitness information to direct the crossover toward a direction that significantly improves the objective fitness. The DRM operator prevents the premature convergence of RCGA and at the same time increases the precision of the searched solution. The effectiveness and application of the proposed RCGA are demonstrated through a variety of single objective optimization benchmark problems. For comparative study, other existing RCGAs with different evolution operators are also performed to the same problem set. Extensive experiment results reveal that the proposed RCGA provides a significantly faster convergence speed and much better search performance than comparative methods.

One of the challenging topics in inverse modeling is numerical efficiency and efficient optimization algorithms have been researched. In optimization algorithms, we need efficient perturbations to find a global minimum. Geostatistical methods combined with perturbation have an important role in generating input parameter fields. However, in cases where the given static sample data are not pertinent considering dynamic data, it is difficult to find a global minimum in the inverse modeling. To overcome this problem, we propose the concept of probabilistic sensitivity analysis. From the sensitivity analysis, cumulative probability distribution functions, which are used as indicators of perturbations, are constructed. Then, they are used for field generations in the gradual deformation method. By enforcing the restart of the optimization process at a local minimum state, the inverse modeling becomes stable and efficient. From the applications using various synthetic reservoirs, the proposed scheme works even in cases where reservoirs have hidden abnormal permeability values, which are not represented by sampling data.

As robots become more capable in terms of hardware, and more complex tasks are considered, optimality starts playing a more important role in the design of algorithms implemented in these systems. Optimality is a guiding principle that directs the computation of feasible and efficient solutions to different robotics tasks. In control theory, this principle is implemented online as a set of efficient numerical optimization algorithms, that in addition to solving the task, purports to save a suitably defined effort or energy term. This thesis investigates trajectory generation, learning and control for dynamic tasks from the unifying point of view of optimization. As an application, we focus on Table Tennis, a chal- lenging task where robots are yet to outperform humans. We believe that the required dexterity and accuracy for this dynamical task hinges on the developments in online optimization and efficient learning algorithms. We consider trajectory generation for table tennis in the first part of the thesis. In highly dynamic tasks like table tennis that involve moving targets, planning is necessary to figure out when, where and how to intercept the target. Motion planning can be very challenging in robotic table tennis in particular, due to time constraints, dimension of the search space and joint limits. Conventional planning algorithms often rely on a fixed virtual hitting plane to construct robot striking trajectories. These algorithms, however, generate restrictive strokes and can result in unnatural strategies when compared with human playing. In this thesis, we introduce a new trajectory generation framework for robotic table tennis that does not involve a fixed hitting plane. A free-time optimal control approach is used to derive two different trajectory optimizers. The resulting two algorithms, Focused Player and Defensive Player, encode two different play- styles. We evaluate their performance in simulation and in our robot table tennis platform with a high speed cable-driven seven DOF robot arm. The algorithms return the balls with a higher probability to the opponent’s court when compared with a virtual hitting plane based method. Moreover, both can be run online and the trajectories can be corrected with new ball observations. In the second part of the thesis, we look at how such trajectories, computed on the kine- matics level, can be tracked accurately with learning control based approaches. Highly dynamic tasks like table tennis require large accelerations and precise tracking for successful perfor- mance. To track desired trajectories well, such tasks usually rely on accurate models and/or high gain feedback. While kinematic optimization allows for efficient representation and online generation of hitting trajectories, learning to track such dynamic movements with inaccurate models remains an open problem. In particular, stability issues surrounding the learning per- formance, in the iteration domain, can prevent the successful implementation of model based learning approaches. To achieve accurate tracking for these tasks in a stable and efficient way, we propose a new adaptive Iterative Learning Control algorithm that is implemented efficiently using a recursive approach. Moreover, covariance estimates of model matrices are used to ex- ercise caution during learning. We evaluate the performance of the proposed approach in our robotic table tennis platform, where we show how the performance of two Barrett WAMs can be optimized. Our implementation on the table tennis platform compares favorably with two state-of-the-art approaches. Finally, we discuss an alternative learning from demonstrations approach, where we learn sparse representations from demonstrated movements. Learning from demonstrations is an easy and intuitive way to show examples of successful behavior to a robot. However, the fact that humans optimize or take advantage of their body and not of the robot, usually called the embodiment problem in robotics, often prevents industrial robots from executing the task in a straightforward way. The shown movements often do not or cannot utilize the degrees of freedom of the robot efficiently, and typically suffer from excessive execution errors. In the last chapter, we show a new approach that can alleviate some of these difficulties by learning sparse representations of movement. Moreover, the number of learned parameters are independent of the degrees of freedom of the robot. Sparsity is a desirable feature for policy search Rein- forcement Learning algorithms that adapt the parameters of these movement primitives. By ranking the learned parameters on the Elastic Net path in terms of importance, we note that our approach could be potentially useful to combat the curse of dimensionality in robot learning applications. We show preliminary results on the real robot setup, including a successful table tennis serve using our new movement primitive representation. Throughout the thesis, we present and analyze in detail new control and learning algo- rithms. Efficient online optimization approaches are presented that can be used to solve not just table tennis problems, but they can be adapted to solve different dynamic tasks.

This paper proposes using neural networks (NN) to implement a real coded genetic algorithm (GA) with the center of gravity crossover (CGX) and the minimal generation gap (MGG) model. With all genetic operations of GA including selection, crossover, mutation and evaluation implemented with NN modules, this approach can realize in parallel genetic operations on the whole chromosome to achieve the maximum parallel realization potential of the MGG model of the GA. At the same time expensive hardware for field programmable gate arrays (FPGA) and the high speed memory of hardware for GA can be avoided. The performance of our solution is validated with a suite of benchmark test functions. This paper suggests that implementing GA with NN is a promising research direction for greatly reducing the running time of GA.

The advancement of modern processors with many-core and large-cache may have little computational advantages if only serial computing is employed. In this study, several parallel computing approaches, using devices with multiple or many processor cores, and graphics processing units are applied and compared to illustrate the potential applications in fluid-film lubrication study. Two Reynolds equations and an air bearing optimum design are solved using three parallel computing paradigms, OpenMP, Compute Unified Device Architecture, and OpenACC, on standalone shared-memory computers. The newly developed processors with many-integrated-core are also using OpenMP to release the computing potential. The results show that the OpenACC computing can have a better performance than the OpenMP computing for the discretized Reynolds equation with a large gridwork. This is mainly due to larger sizes of available cache in the tested graphics processing units. The bearing design can benefit most when the system with many-integrated-core processor is being used. This is due to the many-integrated-core system can perform computation in the optimization-algorithm-level and using the many processor cores effectively. A proper combination of parallel computing devices and programming models can complement efficient numerical methods or optimization algorithms to accelerate many tribological simulations or engineering designs.

Spectrum monitoring is a key element of spectrum management process, in which spectrum sensors are considered the most crucial component. In this paper, we discuss the problem of optimal sensor deployment for spectrum monitoring applications. Optimal sensor deployment can significantly reduce the cost of sensor deployment and improve spectrum monitoring performance and efficiency. An optimal sensor deployment technique is developed, which provides the minimized number of sensors required, as well as their locations of deployment, for monitoring transmissions by desired stations in an area of interest. The problem of optimal sensor deployment is formulated using binary linear integer programming that is solved using efficient numerical optimization algorithms. The technique is scalable in the sense that it considers the effects of pre-installed or existing spectrum sensors in the area. Finally, we apply the technique to an Ottawa area for deploying sensor to monitor all down-link transmissions in the 800 MHz Land Mobile Radio (LMR) band to demonstrate its effectiveness and performance.

We present an efficient and novel numerical algorithm for inversion of transforms arising in imaging modalities such as ultrasound imaging, thermoacoustic and photoacoustic tomography, intravascular imaging, non-destructive testing, and radar imaging with circular acquisition geometry. Our algorithm is based on recently discovered explicit inversion formulas for circular and elliptical Radon transforms with radially partial data derived by Ambartsoumian, Gouia-Zarrad, Lewis and by Ambartsoumian and Krishnan. These inversion formulas hold when the support of the function lies on the inside (relevant in ultrasound imaging, thermoacoustic and photoacoustic tomography, non-destructive testing), outside (relevant in intravascular imaging), both inside and outside (relevant in radar imaging) of the acquisition circle. Given the importance of such inversion formulas in several new and emerging imaging modalities, an efficient numerical inversion algorithm is of tremendous topical interest. The novelty of our non-iterative numerical inversion approach is that the entire scheme can be pre-processed and used repeatedly in image reconstruction, leading to a very fast algorithm. Several numerical simulations are presented showing the robustness of our algorithm.

Solving optimization problems with sparse or low-rank optimal solutions has been an important topic since the recent emergence of compressed sensing and its matrix extensions such as the matrix rank minimization and robust principal component analysis problems. Compressed sensing enables one to recover a signal or image with fewer observations than the “length” of the signal or image, and thus provides potential breakthroughs in applications where data acquisition is costly. However, the potential impact of compressed sensing cannot be realized without efficient optimization algorithms that can handle extremely large-scale and dense data from real applications. Although the convex relaxations of these problems can be reformulated as either linear programming, second-order cone programming or semidefinite programming problems, the standard methods for solving these relaxations are not applicable because the problems are usually of huge size and contain dense data. In this dissertation, we give efficient algorithms for solving these “sparse” optimization problems and analyze the convergence and iteration complexity properties of these algorithms. Chapter 2 presents algorithms for solving the linearly constrained matrix rank minimization problem. The tightest convex relaxation of this problem is the linearly constrained nuclear norm minimization. Although the latter can be cast and solved as a semidefinite programming problem, such an approach is computationally expensive when the matrices are large. In Chapter 2, we propose fixed-point and Bregman iterative algorithms for solving the nuclear norm minimization problem and prove convergence of the first of these algorithms. By using a homotopy approach together with an approximate singular value decomposition procedure, we get a very fast, robust and powerful algorithm, which we call FPCA (Fixed Point Continuation with Approximate SVD), that can solve very large matrix rank minimization problems. Our numerical results on randomly generated and real matrix completion problems demonstrate that this algorithm is much faster and provides much better recoverability than semidefinite programming solvers such as SDPT3. For example, our algorithm can recover 1000 × 1000 matrices of rank 50 with a relative error of 10−5 in about 3 minutes by sampling only 20 percent of the elements. We know of no other method that achieves as good recoverability. Numerical experiments on online recommendation, DNA microarray data set and image inpainting problems demonstrate the effectiveness of our algorithms. In Chapter 3, we study the convergence/recoverability properties of the fixed-point continuation algorithm and its variants for matrix rank minimization. Heuristics for determining the rank of the matrix when its true rank is not known are also proposed. Some of these algorithms are closely related to greedy algorithms in compressed sensing. Numerical results for these algorithms for solving linearly constrained matrix rank minimization problems are reported. Chapters 4 and 5 considers alternating direction type methods for solving composite convex optimization problems. We present in Chapter 4 alternating linearization algorithms that are based on an alternating direction augmented Lagrangian approach for minimizing the sum of two convex functions. Our basic methods require at most O(1/e) iterations to obtain an e-optimal solution, while our accelerated (i.e., fast) versions require at most O(1/ 3 ) iterations, with little change in the computational effort required at each iteration. For more general problem, i.e., minimizing the sum of K convex functions, we propose multiple-splitting algorithms for solving them. We propose both basic and accelerated algorithms with O(1/e) and O(1/ 3 ) iteration complexity bounds for obtaining an e-optimal solution. To the best of our knowledge, the complexity results presented in these two chapters are the first ones of this type that have been given for splitting and alternating direction type methods. Numerical results on various applications in sparse and low-rank optimization, including compressed sensing, matrix completion, image deblurring, robust principal component analysis, are reported to demonstrate the efficiency of our methods.

A200 an important industrial equipment in the field of industrial production, the problem of operating efficiency of asynchronous motor has drawn wide attention with the increasing severity of environmental problems in recent years. Intended to this problem, this paper takes the retarder test bed motor control system as the research object, deduces the motor efficiency optimization control algorithm, and uses the Simulilnk module in Matlab to establish the asynchronous motor loss model. Based on the vector control of asynchronous motor, a simulation model of efficiency optimization control based on loss model is established, and the efficiency optimization control of induction motor based on loss model is realized. At the same time, this paper compares the efficiency optimization control algorithm with the traditional vector control to test the effect of the efficiency optimization algorithm. The experimental results show that the efficiency optimization algorithm used in this paper responds rapidly and the mutation of the load torque fluctuates slightly, but it can be quickly re-stabilized. At the same time, compared with the traditional vector control, efficiency optimization algorithm can significantly improve the test bed operating efficiency. The simulation experiment provides the theoretical support and reference for the future application of the motor control system on the retarder test bench.

This paper studies the frequency assignment problem (FAP) in TD-SCDMA network of mobile communications industry in China. The problem considers finding the optimal frequency allocation scheme for carriers with a limited frequency resource, such that the entire network interference is minimized. Besides, the allocation of frequencies needs to satisfy some constraints to avoid the effect of call interference within the same cell or adjacent cell. Given the formula for calculation of the network interference, we take the FAP as a constrained optimization problem and use a hyper-heuristic genetic algorithm (HHGA) to optimize the assignment of frequencies. We first define six low-level heuristics (LLHs) search strategies based on the computation of interference, and then use genetic algorithm (GA) at a high-level to find the best combination sequence of LLH strategies to reduce interferences of the overall network. GA uses two-point crossover, uniform mutation, and Minimal Generation Gap (MGG) as the generation alternation model. In order to speed up the search, we define a Tabu table to avoid repeat search of LLHs. Compared with scatter search as one of the meta-heuristic algorithm with best performance, our experimental results on real data sets of TD-SCDMA network have shown better result.

A Real-Coded Genetic Algorithm for Function Optimization Using the Unimodal Normal Distribution Crossover

This paper deals with the global optimization of the BMIEP (bilinear matrix inequalities eigenvalue problem) based on the UNDX (unimodal normal distribution crossover) GA. First, analyzing the structure of the BMIEP, the existence of the typical difficult structures is confirmed. Then, based on the results of the problem structures analysis and the consideration of the BMIEP characteristic properties, an efficient UNDX GA algorithm with LMI convex estimation is proposed. The effectiveness of the proposed algorithm is evaluated by several numerical examples.

This paper presents a new, more efficient numerical algorithm that has been developed to predict thermal and microstructure fields during quenching of steel rods. The present algorithm solves the full nonlinear heat conduction equation using a central finite-difference scheme coupled with a fourth-order Runge-Kutta nonlinear solver. Numerical results, obtained using the present algorithm, have been validated using experimental data and numerical results available in the literature. In addition to its accurate predictions, the present algorithm does not require iteration; hence, it is computationally more efficient than previous numerical algorithms.

Very recently, an efficient computational algorithm (DETQPT algorithm) for the determinant evaluation of general cyclic pentadiagonal Toeplitz matrices has been proposed by Y.L. Jiang and J.T. Jia (J. Math. Chem. 51: 2503-2513, 2013). In this paper, an explicit formula for the determinant of a cyclic pentadiagonal Toeplitz matrix is derived at first. Then, we present a more efficient numerical algorithm with the cost of 7n+O(logn)$7n+O(\log \ n)$ for evaluating n-th order cyclic pentadiagonal Toeplitz determinants. The algorithm is based on the use of a certain type of matrix reordering and matrix partition, and equalities involving the products of special kind of relevant matrices. Three numerical examples demonstrate the performance and effectiveness of the proposed algorithm and its competitiveness with other already existing algorithms.

In this work, we propose efficient and accurate numerical algorithms based on Difference Potentials Method for numerical solution of chemotaxis systems and related models in 3D. The developed algorithms handle 3D irregular geometry with the use of only Cartesian meshes and employ Fast Poisson Solvers. In addition, to further enhance computational efficiency of the methods, we design a Difference-Potentials-based domain decomposition approach which allows mesh adaptivity and easy parallelization of the algorithm in space. Extensive numerical experiments are presented to illustrate the accuracy, efficiency and robustness of the developed numerical algorithms.