Prompt Evolutionary Design Optimization with Generative Shape and Vision-Language models

Statistical Shape Models have been proven to be valuable tools for segmenting anatomical structures of arbitrary topology. Being based on the statistical description of representative shapes, an initial segmentation is required – preferably done by an expert. For this purpose, mostly manual segmentation methods followed by a mesh generation step are employed. A prerequisite for generating the training data based on these segmentations is the establishment of correspondences between all training meshes. While existing approaches decouple the expert segmentation from the correspondence establishment step, we propose in this work a segmentation approach that simultaneously establishes the landmark correspondences needed for the subsequent generation of shape models.Our approach uses a reference segmentation given as a regular mesh. After an initial placement of this reference mesh, it is manually deformed in order to best match the boundaries of the considered anatomical structure. This deformation is coupled with a real time optimization that preserves point correspondences and thus ensures that a pair of landmark points in two different data sets represents the same anatomical feature.We applied our new method to different anatomical structures: vertebra of the spinal chord, kidney, and cardiac left ventricle. In order to perform a visual evaluation of the degree of correspondence between different data sets, we have developed well adapted visualization methods. From our tests we conclude that the expected correspondences are established during the manual mesh deformation. Furthermore, our approach considerably speeds up the shape model generation, since there is no need for an independent correspondence establishment step. Finally, it allows the creation of shape models of arbitrary topology and removes potential error sources of landmark and correspondence optimization algorithms needed so far.KeywordsShape ModelSubdivision SurfaceMesh DeformationStatistical Shape ModelArbitrary TopologyThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Biological shape modeling is an essential task that is required for systems biology efforts to simulate complex cell behaviors. Statistical learning methods have been used to build generative shape models based on reconstructive shape parameters extracted from microscope image collections. However, such parametric modeling approaches are usually limited to simple shapes and easily-modeled parameter distributions. Moreover, to maximize the reconstruction accuracy, significant effort is required to design models for specific datasets or patterns. We have therefore developed an instance-based approach to model biological shapes within a shape space built upon diffeomorphic measurement. We also designed a recursive interpolation algorithm to probabilistically synthesize new shape instances using the shape space model and the original instances. The method is quite generalizable and therefore can be applied to most nuclear, cell and protein object shapes, in both 2D and 3D.

Phi-array: A novel method for fitness visualization and decision making in evolutionary design optimization

Approximate models have to be used in evolutionary optimization when the original fitness function is computationally very expensive. Unfortunately, the convergence property of the evolutionary algorithm is unclear when an approximate model is used for fitness evaluation because approximation errors are involved in the model. What is worse, the approximate model may introduce false optima that lead the evolutionary algorithm to a wrong solution. To address this problem, individual and generation based evolution control are introduced to ensure that the evolutionary algorithm using approximate fitness functions will converge correctly. A framework for managing approximate models in generation-based evolution control is proposed. This framework is well suited for parallel evolutionary optimization in which evaluation of the fitness function is time-consuming. Simulations on two benchmark problems and one example of aerodynamic design optimization demonstrate that the proposed algorithm is able to achieve a correct solution as well as a significantly reduced computation time.

This paper proposes a Bayesian treatment of a latent variable model for learning generative shape models of grid-structured representations, aka label maps, that relies on direct probabilistic formulation with a variational approach for deterministic model learning. Spatial coherency and sparsity priors are incorporated to lend stability to the optimization problem, thereby regularizing the solution space while avoiding overfitting in this high-dimensional, low-sample-size scenario. Hyperparameters are estimated in closed-form using type-II maximum likelihood to avoid grid searches. Further, a mixture formulation is proposed to capture nonlinear shape variations in a way that balances the model expressiveness with the efficiency of learning and inference. Experiments show that the proposed model outperforms state-of-the-art representations on real datasets w.r.t. generalization to unseen samples.

Rich geometric understanding of the world is an important component of many robotic applications such as planning and manipulation. In this paper, we present a modular pipeline for pose and shape estimation of objects from RGB-D images given their category. The core of our method is a generative shape model, which we integrate with a novel initialization network and a differentiable renderer to enable 6D pose and shape estimation from a single or multiple views. We investigate the use of discretized signed distance fields as an efficient shape representation for fast analysis-by-synthesis optimization. Our modular framework enables multi-view optimization and extensibility. We demonstrate the benefits of our approach over state-of-the-art methods in several experiments on both synthetic and real data. We open-source our approach at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/roym899/sdfest</uri> .

Statistical shape models (SSMs) are a standard generative shape modeling technique and they are still successfully employed in modern deep learning-based solutions for data augmentation purposes or as shape priors. However, with few training samples they often fail to represent local shape variations. Recently, a new state-of-the-art method has been proposed to alleviate this problem via a multi-level model localization scheme using distance-based covariance manipulations and Grassmannian-based level fusion during model training. This method significantly improves a SSMs performance, but heavily relies on costly eigendecompositions of large covariance matrices. In this paper, we derive a novel computationally-efficient formulation of the original method using ideas from kernel theory and randomized eigendecomposition. The proposed extension leads to a multi-level localization method for large-scale shape modeling problems that preserves the key characteristics of the original method while also improving its performance. Furthermore, our extensive evaluation on two publicly available data sets reveals the benefits of Grassmannian-based level fusion in contrast to a method derived from the popular Gaussian Process Morphable Models framework.

Chapter 13 - Robust and Scalable Shape Prior Modeling via Sparse Representation and Dictionary Learning

We introduce a 3D generative shape model based on the generalized autoencoder (GAE). GAEs learn a manifold latent space from data relations explicitly provided during training. In our work, we train a GAE for volumetric shape generation from data similarities derived from the Chamfer distance, and with a loss function which is the combination of the traditional autoencoder loss and the GAE loss. We show that this shape model is able to learn more meaningful structures for the latent manifolds of different categories of shapes, and provides better interpolations between shapes when compared to previous approaches such as autoencoders and variational autoencoders.

We optimize a six degrees of freedom hovering policy using reinforcement meta-learning. The policy maps flash LIDAR measurements directly to on/off spacecraft body-frame thrust commands, allowing hovering at a fixed position and attitude in the asteroid body-fixed reference frame. Importantly, the policy does not require position and velocity estimates, and can operate in environments with unknown dynamics, and without an asteroid shape model or navigation aids. Indeed, during optimization the agent is confronted with a new randomly generated asteroid for each episode, ensuring that it does not learn an asteroid’s shape, texture, or environmental dynamics. This allows the deployed policy to generalize well to novel asteroid characteristics, which we demonstrate in our experiments. The hovering controller has the potential to simplify mission planning by allowing asteroid body-fixed hovering immediately upon the spacecraft’s arrival to an asteroid. This, in turn, simplifies shape model generation and allows resource mapping via remote sensing immediately upon arrival at the target asteroid.

A dynamic industrial design optimization requires high-quality optimization algorithms as well as adaptive representations to find the global solution for a given problem. For adapting the representation to changing environments or to new input we utilize the concept of evolvability, which in our interpretation consists of three criteria: variability, regularity, and improvement potential, where regularity and improvement potential characterize conflicting goals between exploration and exploitation. Our goal is the efficient adaptation of the representation according to a given preference weight between regularity and improvement potential. We propose a combination of two heuristics, Lloyd sampling and orthogonal least squares sampling, to initialize the adaptation process for a given preference weight. We show that this initialization improves the convergence speed of the adaptation process as well as the resulting fitness. We then realize a stepwise design optimization procedure by alternating the adaptation of the representation with optimization of the design. During the design optimization process we extract information which we exploit in the next adaptation phase. We show that an intermediate preference weight, balancing between regularity and improvement potential, allows to exploit this information and is robust to erroneous initial information. Thereby, we increase the performance of the whole design optimization process.

Genetic engineering of recombinant antibodies (rAb) enables the selection and improvement of fundamental Ab properties such as affinity or thermal stability by rational and evolutionary design. Different ab gene sources are available for this purpose. The human combinatorial Ab library HuCAL® was applied as an example of a fully synthetic Ab repertoire. Specific clones were selected from the HuCAL® library by means of phage display for the detection of the herbicide glyphosate, as well as for food-borne pathogens. Depending on the germline Ab gene family individual Ab fragments exhibited extraordinarily high stability. As an example for a natural rAb library, murine Ab genes were cloned from immunoselected, s-triazine stimulated B cell repertoires. The most efficient strategy for the affinity improvement of rAbs for atrazine was found to be a combination of randomized point mutations and sequential recombination of variable domain encoding Ab gene repertoires. This strategy resulted in a shift of the equilibrium dissociation constant K D from 1.3 x 10 -8 M of the template Ab to 7.5 x 10 -10 M of the optimized variant. The applicability of the optimized Ab variant was confirmed by the determination of atrazine contamination in soil samples.

A new multi-directional search approach that aims at maximizing the flow entropy of water distribution systems is investigated. The aim is to develop an efficient and practical maximum entropy based approach. The resulting optimization problem has four objectives, and the merits of objective reduction in the computational solution of the problem are investigated also. The relationship between statistical flow entropy and hydraulic reliability/failure tolerance is not monotonic. Consequently, a large number of maximum flow entropy solutions must be investigated to strike a balance between cost and hydraulic reliability. A multi-objective evolutionary optimization model is developed that generates simultaneously a wide range of maximum entropy values along with clusters of maximum and near-maximum entropy solutions. Results for a benchmark network and a real network in the literature are included that demonstrate the effectiveness of the procedure.

This study has constructed a fully automated multidisciplinary and many-objective evolutionary design optimization system independent of computer environments to evaluate objective functions; the research applied it to a geometric design problem of a flyback booster for next-generation space transportation. In optimization involving objective functions to appraise the aero-/structural-dynamic performance with high fidelity, spatial discretization hinders the overall automation. This research has facilitated an efficient optimal design by wholly automating high-fidelity assessments, which designers had to implement manually, and has accomplished optimizations that directly contribute to real-world design problems. Moreover, this study would accumulate design knowledge for space transportation that the market is reviving. The total automated system yielded the embedding of geometric trait lines to ensure the discretization even for large curvature surfaces; the system innovated a robust automatic error-checking mechanism in the system’s preprocess. Consequently, the entirely automatized optimization procured nondominated solution sets for more precise data analyses in a pragmatic execution period. Design informatics, a framework combining optimization and data analysis, functioned usefully in real-world design on flyback-booster geometry by materializing smooth deriving and verifying a design hypothesis; eventually, the research gained a new design principle.

This work proposes a nodal‐based evolutionary design optimization algorithm to design frame structures whose edges are the Delaunay triangulation of homogeneously distributed nodes in the design domain. The remaining nodes can freely sway in the design domain except for the loading nodes and boundary nodes. As a result, it can extend the space of admissible solutions to this optimization problem and reduce the number of design variables. Then, the sensitivity of the objective function, namely, the sum of compliance and its volume, is input into the method of moving asymptotes to update the nodal coordinates and member thickness. The most inefficient node is deleted in each iteration based on the average nodal strain energy until its number reaches a prescribed limit. 2D numerical examples for the Michell arch, L‐shaped bracket, and Messerschmidt–Bölkow–Blohm beam show that the proposed algorithm can get the optimal structure within a few iterations. Compared with initial configurations in 3D, the optimal crane arm, transmission tower, and aquatic dome have less strain energy and fewer materials, showing a great potential of the proposed method to design actual space frames.