An event-driven multithreaded dynamic optimization framework

Deep neural networks (DNNs) achieved great cognitive performance at the expense of a considerable computation workload. To relieve the computational burden, many optimization works are developed to reduce the model redundancy by identifying and removing insignificant model components, such as weight sparsity and filter pruning methods. However, these works only evaluate model components’ static significance with parameter information, ignoring their dynamic interaction with external inputs. Specifically, due to the difference in per-input features, the model components’ significance can dynamically change and, thus, the static methods can only achieve suboptimal performance. Focusing on this aspect, we propose a dynamic DNN optimization framework in this work. Based on the neural network attention mechanism, we propose a comprehensive dynamic optimization framework, including 1) testing-phase dynamic feature map pruning; 2) training-phase optimization by training with targeted dropout; and 3) deployment-phase one-for-all (OFA) model adaptability enhancement. By providing a holistic dynamic testing, training, and deployment co-optimization framework, our work has the following benefits: first, it can accurately identify and aggressively remove per-input feature redundancy by considering the model-input interaction and involving the channel/column-wise pruning flexibility; meanwhile, the training-testing co-optimization favors the dynamic pruning and helps maintain the model accuracy even with a very high feature pruning ratio. Finally, the deployment enhancement provides one unified OFA model to support full-spectrum feature sparsity ratios. The unified model can be dynamically reconfigured to meet different resource budgets without any retraining cost, and thus provide significant deployment flexibility. Extensive experiments show that our method could bring 37.4%–54.5% floating-point operations reduction with negligible accuracy drop on various test benchmarks. Meanwhile, the OFA deployment optimization enables us to use one model to support at most ten different resource constraints without any retraining cost.

Accurately predicting program behaviors (e.g., memory locality, method calling frequency) is fundamental for program optimizations and runtime adaptations. Despite decades of remarkable progress, prior studies have not systematically exploited the use of program inputs, a deciding factor of program behaviors, to help in program dynamic optimizations. Triggered by the strong and predictive correlations between program inputs and program behaviors that recent studies have uncovered, the dissertation work aims to bring program inputs into the focus of program behavior analysis and program dynamic optimization, cultivating a new paradigm named input-centric program behavior analysis and dynamic optimization.;The new optimization paradigm consists of three components, forming a three-layer pyramid. at the base is program input characterization, a component for resolving the complexity in program raw inputs and extracting important features. In the middle is input-behavior modeling, a component for recognizing and modeling the correlations between characterized input features and program behaviors. These two components constitute input-centric program behavior analysis, which (ideally) is able to predict the large-scope behaviors of a program's execution as soon as the execution starts. The top layer is input-centric adaptation, which capitalizes on the novel opportunities created by the first two components to facilitate proactive adaptation for program optimizations.;This dissertation aims to develop this paradigm in two stages. In the first stage, we concentrate on exploring the implications of program inputs for program behaviors and dynamic optimization. We construct the basic input-centric optimization framework based on of line training to realize the basic functionalities of the three major components of the paradigm. For the second stage, we focus on making the paradigm practical by addressing multi-facet issues in handling input complexities, transparent training data collection, predictive model evolvement across production runs. The techniques proposed in this stage together cultivate a lifelong continuous optimization scheme with cross-input adaptivity.;Fundamentally the new optimization paradigm provides a brand new solution for program dynamic optimization. The techniques proposed in the dissertation together resolve the adaptivity-proactivity dilemma that has been limiting the effectiveness of existing optimization techniques. its benefits are demonstrated through proactive dynamic optimizations in Jikes RVM and version selection using IBM XL C Compiler, yielding significant performance improvement on a set of Java and C/C++ programs. It may open new opportunities for a broad range of runtime optimizations and adaptations. The evaluation results on both Java and C/C++ applications demonstrate the new paradigm is promising in advancing the current state of program optimizations.

A dynamic optimizer is a runtime software system that groups a program's instruction sequences into traces, optimizes those traces, stores the optimized traces in a software-based code cache, and then executes the optimized code in the code cache. To maximize performance, the vast majority of the program's execution should occur in the code cache and not in the different aspects of the dynamic optimization system. In the past, designers of dynamic optimizers have used the SPEC2000 benchmark suite to justify their use of simple code cache management schemes. In this paper, we show that the problem and importance of code cache management changes dramatically as we move from SPEC2000, with its relatively small number of dynamically generated code traces, to large interactive Windows applications. We also propose and evaluate a new cache management algorithm based on generational code caches that results in an average miss rate reduction of 18% over a unified cache, which translates into 19% fewer instructions spent in the dynamic optimizer. The algorithm categorizes code traces based on their expected lifetimes and group traces with similar lifetimes together in separate storage areas. Using this algorithm, short-lived code traces can easily be removed from a code cache without introducing fragmentation and without suffering the performance penalties associated with evicting long-lived code traces.

A Tchebycheff-based multi-objective combined with a PSO–SQP dynamic real-time optimization framework for cycling energy systems

History matching is the process of modifying the uncertain attributes of a reservoir model to reproduce the real reservoir performance. It is a classical reservoir engineering problem and plays an important role in reservoir management since the resulting models are used to support decisions in other tasks such as economic analysis and production strategy. This work introduces a dynamic decision-making optimization framework for history matching problems in which new models are generated based on, and guided by, the dynamic analysis of the data of available solutions. The optimization framework follows a ‘learning-from-data’ approach, and includes two optimizer components that use machine learning techniques, such as unsupervised learning and statistical analysis, to uncover patterns of input attributes that lead to good output responses. These patterns are used to support the decision-making process while generating new, and better, history matched solutions. The proposed framework is applied to a benchmark model (UNISIM-I-H) based on the Namorado field in Brazil. Results show the potential the dynamic decision-making optimization framework has for improving the quality of history matching solutions using a substantial smaller number of simulations when compared with a previous work on the same benchmark.

The power systems of the future - smart grid - will see an increase in both renewable energy sources and load demand increasing the need for fast dynamic reconfiguration of system parameters. Advanced simulation capabilities are needed to support future planning and to implement real-time dynamic optimization. To achieve this, intelligent algorithms are needed for prediction and system monitoring along with advanced hardware to provide speed. FASTSIM capabilities allows for fast prediction of how a proposed reconfiguration will affect the system permitting implementation of radical reconfigurations without fear of disturbing vital system settings since the proposed solution will be checked before being implemented. Power systems will need to be continuously monitored for power losses and system contingencies. A combination of system states prediction, dynamic load flow, system optimization, and solution stability checking will allow for a more reliable, affordable and clean power grid. This paper presents a dynamic optimization method based on adaptive critic designs for the smart grid environment.

With the increasingly fierce market competition, optimizing the construction time-cost of deep and large foundation pit construction projects has become one of the key factors for construction enterprises to remain invincible. By dynamically optimizing the time-cost, the optimal time corresponding to the lowest engineering cost can be found. Based on a comprehensive transportation hub project, considering the time value of capital and the reward and punishment of construction time, a time-cost dynamic optimization model is constructed. Relying on BIM technology, the feasibility of construction plan is analyzed from both qualitative and quantitative perspectives, and the project parameters and resource information is obtained accurately and quickly. Using the MATLAB program, time-cost optimization based on genetic algorithm is carried out, and the static and dynamic optimization results are compared. The results show that the dynamic optimization scheme reduces the total cost by 1.68% while reducing the total construction time by 8.47%. The dynamic optimization scheme extends the construction period by 2 days while reducing the total cost by 89,500 yuan compared to static optimization. The peak value of the fund demand curve before and after optimization has decreased from 128,000 yuan to 127,000 yuan. The time-cost dynamic optimization, considering the time value of capital is more in line with engineering reality, and the optimization results are more reliable and accurate. BIM technology can accurately and quickly obtain project parameters and resource information, solving problems such as complex processes in super deep excavation and huge engineering data statistics. The genetic algorithm can efficiently and accurately search for the optimal solution within the global domain. This study combines BIM technology with the genetic algorithm to solve the dynamic optimization problem of construction period cost for deep and large foundation pits. The research results of this study provide a theoretical reference for the optimization of schedule cost for similar projects.

Dynamic program optimization offers performance improvements far beyond those possible with traditional compile-time optimization. These gains are due to the ability to exploit both architectural and input data set characteristics that are unknown prior to execution time. In this paper, we propose a novel framework for dynamic program optimization, ADAPT (Automated De-coupled Adaptive Program Transformation), that builds on the strengths of existing approaches. The key to our framework is the de-coupling of the dynamic compilation of new code variants from the dynamic selection of these variants at their points of use. This allows code generation to occur concurrently with program execution, removing dynamic compilation overheads from the critical path. We present a compilation system, based on the Polaris optimizing compiler, that automatically applies this framework to general plugged-in optimization techniques. We evaluate our system on three programs from the SPEC floating point benchmark suite by dynamically applying loop distribution, loop unrolling, loop tiling and automatic parallelization. We show that our techniques can improve performance by as much as 70% over statically optimized code.

Generating motion and capturing motion of an articulated body for computer animation is an expensive and time-consuming task. Conventionally, animators manually generate intermediate frames between key frames, but this task is very labor-intensive. This paper presents a model-based singularity-free automatic-initialization approach to capturing human motion from widely-available, static background monocular video sequences. A 3D human body model is built and projected on a 2D projection plane to find the best fit with the foreground image silhouette. We convert the human motion capture problem into two types of parameter optimization problems: static optimization and dynamic optimization. First, we determine each model body configuration using static optimizations for every input image. Then, to obtain better description of motion, the results from all static optimizations are fed into a dynamic optimization process where the entire sequence of motion is considered for the user-specified motion. The user-specified motion is defined by the user and the final form of the motion they want. A cost function for static optimization is used to estimate the degree of overlapping between the foreground input image silhouette and a projected 3D model body silhouette. The overlapping is computed using computational geometry by converting a set of pixels from the image domain to a polygon in the real projection plane domain. A cost function for dynamic optimization is the user-specified motion based on the static optimization results as well as image fitting. Our method is used to capture various human motions: walking, pushing, kicking, and handshaking.KeywordsInput ImageDynamic OptimizationModel BodyComputer AnimationParameter Optimization ProblemThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Aiming at the problems such as low computational efficiency and long process in dynamic topology optimization, combined with bi-directional evolutional structure optimization, the dynamic optimization problem is transformed into the static optimization problem in multiple load cases. Newmark method is used to analyze the structure dynamics load. The topology optimization model based on equivalent static load is established. On the basis of the equivalent static load, multi-objective dynamics evolutionary structure optimization is studied. And the mathematical model of the dynamic stiffness and natural frequency about the multi-objective optimization is established. The mathematical formula of the sensitivity is deduced. Then a complete optimization iterative process is given. The numerical example illustrates that it is feasible to solve the multi-objective dynamic topology optimization with equivalent static loads. Modal tracking technique is introduced to the multi-objective dynamic topology optimization. The numerical example shows that modal tracking technology can track the required modal in the optimization process.

This paper demonstrates that ant colony optimization can efficiently improve the traffic situation in an urban environment. A Dynamic Travel Path Optimization System (DTPOS) based on Ant Colony Optimization (ACO) is proposed for the prediction of the best path to a given destination. In DTPOS, traffic factors such as average travel speed, average waiting time of cars and number of stopped cars in queue are taken into consideration. The proposed method is modeled in Net Logo. The simulation results demonstrate that the DTPOS model can greatly reduce the average travel time of cars in urban cases and improves the mean travel time by 47 percent when compared to similar models where the cars select their path without ACO. It has also been shown that the results can be further improved by 56 percent when the Previous Path Replacement (PPR) method is applied to the DTPOS results.

Aiming at the vehicle power train, system dynamic optimization is profoundly studied based on the system torsional vibration characteristics analysis. First, based on the concentrated mass method, the general torsional vibration model of vehicle power train is established and solved after parameters’ (inertia, stiffness, and damping) matrix and mathematic constraint conditions are acquired. Furthermore, both free vibration and forced vibration characteristics are analyzed. Second, the effects of the coupling stiffness on the dynamics behaviors of power train are thoroughly analyzed. The sensitivity analysis procedure is explored. And sensitivity models of both free and forced vibration feature parameters are deduced. Finally, dynamic optimization theory model and program are constructed based on genetic algorithm. The optimization results indicate that the proposed optimization method could contribute to the sharp attenuation of system torsional vibration.

Under the dual pressures of global carbon emission reduction and production cost control, energy-intensive industrial enterprises are in urgent need of a balanced low-carbon operation strategy that reconciles economic benefits, environmental performance and production continuity. To address the limitations of existing methods in multi-dimensional objective balancing, this paper proposes a dynamic multi-objective optimization framework for industrial electricity consumption, integrating high-precision load forecasting and optimal scheduling. For load forecasting, an improved dual-gate optimization temporal attention long short-term memory (DGO-TA-LSTM) model is developed, which is modeled based on the one-year hourly electricity operation data (8760 samples) of a high-energy industrial enterprise in southern China, and its performance is verified via three standard metrics—the mean absolute error (MAE), root mean square error (RMSE) and mean absolute percentage error (MAPE)—compared with five mainstream baseline models. On this basis, when taking time-varying electricity-carbon factors and time-of-use electricity prices as dual guiding signals, a three-objective optimization model minimizing electricity cost, carbon emissions and load deviation is constructed, which is solved by the Non-Dominated Sorting Genetic Algorithm II (NSGA-II), with the Improved Gray Target Decision-Making (IGTD) method introduced to select the optimal compromise solution. Case study results show that the proposed scheme achieved a 1.9% reduction in electricity cost and a 30% reduction in carbon emissions compared with the unoptimized strategy, providing a feasible and scalable low-carbon operation path for industrial enterprises.

My dissertation presents several new algorithms incorporating non-parametric and deep learning approaches for computer vision and related tasks, including object localization, object tracking and model compression. With respect to object localization, I introduce a method to perform active localization by modeling spatial and other relationships between objects in a coherent "visual situation" using a set of probability distributions. I further refine this approach with the Multipole Density Estimation with Importance Clustering (MIC-Situate) algorithm. Next, I formulate active, "situation" object search as a Bayesian optimization problem using Gaussian Processes. Using my Gaussian Process Context Situation Learning (GP-CL) algorithm, I demonstrate improved efficiency for object localization over baseline procedures. In subsequent work, I expand this research to frame object tracking in video as a temporally-evolving, dynamic Bayesian optimization problem. Here I present the Siamese-Dynamic Bayesian Tracking Algorithm (SDBTA), the first integrated dynamic Bayesian optimization framework in combination with deep learning for video tracking. Through experiments, I show improved results for video tracking in comparison with baseline approaches. Finally, I propose a novel data compression algorithm, Regularized L21 Semi-NonNegative Matrix Factorization (L21 SNF) which serves as a general purpose, parts-based compression algorithm, applicable to deep model compression.

In satellite-denied environments, the inertial navigation system (INS) suffers from error accumulation, whereas integrated solutions often depend on costly external sensors and fail to exploit motion modes effectively. This paper reveals the mechanism of navigation enhancement under sparse motion modes, where effective states are intermittent but critical. We propose a task-driven dynamic decision optimization framework for self-constrained vehicle inertial navigation, combining deep learning with geometric state estimation. First, we establish multidimensional self-constraints from vehicle kinematics to suppress error propagation. Next, a Transformer-based network calculates motion mode confidences from inertial data, followed by a Bayesian optimization strategy to dynamically adjust thresholds, prioritizing false alarm suppression under a bounded missed detection tolerance. Further, a state persistence mechanism (SPM) leverages motion continuity to eliminate short-term misjudgments. Finally, motion constraints are fused with inertial data via a manifold-based invariant extended Kalman filter (IEKF), which embeds states on Lie groups to mitigate inconsistency in linearization. The experiments on the Urban Kaist dataset demonstrate that compared with the existing motion mode decision, the average absolute trajectory error of positioning is reduced by more than 8.19%. This verifies the effectiveness of the proposed method, which provides a new solution for high-precision autonomous navigation of vehicles.