Optimizing CAD Models with Latent Space Manipulation

Incremental forming is a versatile and cost-effective sheet metal forming technique widely adopted in low-volume manufacturing and prototyping across various industries. Recent advancements in data-driven approaches, including machine vision, neural networks, and 3D reconstruction methods, have significantly enhanced the precision and efficiency of incremental forming processes. This study explores the integration of advanced data acquisition and processing techniques to improve the accuracy, automation, and defect detection capabilities in incremental forming. Key advancements such as robot-assisted forming, computer-controlled toolpath generation from CAD models, and real-time quality monitoring using machine vision are discussed. The potential of single- and multi-view 3D reconstruction methods for optimizing toolpath strategies and enhancing formability is also examined. The findings highlight opportunities for full automation in incremental forming, demonstrating its potential to revolutionize modern manufacturing by reducing costs, increasing customization, and improving product quality. These advancements could benefit industries such as aerospace, automotive, and medical device manufacturing, where precision and flexibility are critical.

Magnetic Resonance Imaging (MRI) is one of the most commonly used modalities in medical imaging, and one of the main limitations is its slow scanning and reconstruction speed. Recently, deep learning used for compressed sensing (CS) methods have been proposed to accelerate the acquisition by undersampling in the K-space and reconstruct images with neural networks. However, there are still some challenges remained: First, directly training networks based on L1/L2 distance to the target fully sampled images may lead to fuzzy reconstruction images because L1/L2 loss only enforces the overall image or patch similarity, but does not consider the local details such as anatomical sharpness. Second, Generative Adversarial Networks (GAN) can partially solve this problem. The undersampling image gets the latent space through the encoder, and the image is reconstructed by the decoder based on GAN loss, but it may generate unrealistic details by lacking of constraints in K-space domain. Third, most of the networks after training are fixed and have limited adaptation capability in the inference time, and the patient-specific information cannot be effectively used. To resolve these challenges, we proposed a new compressed sensing GAN reconstruction method, and there are two main contributions: (1) We proposed a encoder-decoder structure, which guided GAN optimization strategy data-consistency in latent space to improve the reconstruction quality such as preserving more local details and improving the anatomical sharpness while constraining GAN to follow the data distribution in the K-space to prevent the unrealistic details. (2) An online update strategy is used to find the best representation in the latent space for the underlying patient, and the reconstruct result can be further improved by incorporating the patient-specific information. Extensive experimental results show the effectiveness of our method. KeywordsMRI reconstructionData-consistencyLatent spaceOnline update

The objective of the proposal is to analyze what latent space is within a Deep-Learning system and how its visualization is capable of triggering a meaning-effect concerning the epistemology of big data. The latent space is the mathematical space that maps what a Neural Network has learned from the training dataset. It is the result of the compression of the input data and the step before the Neural Network’s output, a step that usually remains invisible to the human eye, rendering effective the promise of a transparent effect of reality generally promoted by Artificial Intelligence technologies. Precisely in contrast with this promise, the visualization of this complex spatiality makes accessible, and therefore intelligible, the epistemic and rhetorical relations inscribed within datasets, intended as archives that gather information. To achieve my objective, I will consider an artistic project realized by multimedia artist and coder Jake Elwes, Zizi-Queering the Dataset (2019), a multi-channel video where different facial portraits are shown in a morphing loop that visualizes what a Generative Adversarial Network has learned from the re-training of a dataset containing portraits with another one containing facial images of drag and non-binary individuals. This artistic gesture has led to a series of epistemic issues concerning big data and their situated and ideological meaning.

Generative models rely on the idea that data can be represented in terms of latent variables which are uncorrelated by definition. Lack of correlation among the latent variable support is important because it suggests that the latent-space manifold is simpler to understand and manipulate than the real-space representation. Many types of generative model are used in deep learning, e.g., variational autoencoders (VAEs) and generative adversarial networks (GANs). Based on the idea that the latent space behaves like a vector space Radford et al. (2015), we ask whether we can expand the latent space representation of our data elements in terms of an orthonormal basis set. Here we propose a method to build a set of linearly independent vectors in the latent space of a trained GAN, which we call quasi-eigenvectors. These quasi-eigenvectors have two key properties: i) They span the latent space, ii) A set of these quasi-eigenvectors map to each of the labeled features one-to-one. We show that in the case of the MNIST image data set, while the number of dimensions in latent space is large by design, 98% of the data in real space map to a sub-domain of latent space of dimensionality equal to the number of labels. We then show how the quasi-eigenvectors can be used for Latent Spectral Decomposition (LSD). We apply LSD to denoise MNIST images. Finally, using the quasi-eigenvectors, we construct rotation matrices in latent space which map to feature transformations in real space. Overall, from quasi-eigenvectors we gain insight regarding the latent space topology.

Most examinations of neural networks' learned latent spaces typically employ dimensionality reduction techniques such as t-distributed stochastic neighbor embedding (t-SNE) or uniform manifold approximation and projection (UMAP). These methods distort the local neighborhood in the visualization, making it hard to distinguish the structure of a subset of samples in the latent space. In response to this challenge, we introduce the k* distribution and its corresponding visualization technique. This method uses local neighborhood analysis to guarantee the preservation of the structure of sample distributions for individual classes within the subset of the learned latent space. This facilitates easy comparison of different k* distributions, enabling analysis of how various classes are processed by the same neural network. Our study reveals three distinct distributions of samples within the learned latent space subset: 1) fractured; 2) overlapped; and 3) clustered, providing a more profound understanding of the existing contemporary visualizations. Experiments show that the distribution of samples within the network's learned latent space significantly varies depending on the class. Furthermore, we illustrate that our analysis can be applied to explore the latent space of diverse neural network architectures, various layers within neural networks, transformations applied to input samples, and the distribution of training and testing data for neural networks. Thus, the k* distribution should aid in visualizing the structure inside neural networks and further foster their understanding.

A person’s gait is a behavioral trait that is uniquely associated with each individual and can be used to recognize the person. As information about the human gait can be captured by wearable devices, a few studies have led to the proposal of methods to process gait information for identification purposes. Despite recent advances in gait recognition, an open set gait recognition problem presents challenges to current approaches. To address the open set gait recognition problem, a system should be able to deal with unseen subjects who have not included in the training dataset. In this paper, we propose a system that learns a mapping from a multimodal time series collected using insole to a latent (embedding vector) space to address the open set gait recognition problem. The distance between two embedding vectors in the latent space corresponds to the similarity between two multimodal time series. Using the characteristics of the human gait pattern, multimodal time series are sliced into unit steps. The system maps unit steps to embedding vectors using an ensemble consisting of a convolutional neural network and a recurrent neural network. To recognize each individual, the system learns a decision function using a one-class support vector machine from a few embedding vectors of the person in the latent space, then the system determines whether an unknown unit step is recognized as belonging to a known individual. Our experiments demonstrate that the proposed framework recognizes individuals with high accuracy regardless they have been registered or not. If we could have an environment in which all people would be wearing the insole, the framework would be used for user verification widely.

Face-swapping technology has been widely used in people's life, and people also put forward higher requirements for it. Most of the current face-swapping methods are difficult to generate a high-definition face image. Through StyleGAN, we can generate high-definition face images. However, face-swapping with StyleGAN is still challenging. Firstly, we need to map the target image to the latent space of StyleGAN. Many tasks need to map the input image to a new latent space for face-swapping, because identity features are complex and challenging to map to specific latent space layers directly. So face-swapping is completed in the remapping process, which consumes excess computing resources for reconstruction. And the generated image is difficult to maintain the original image color, face attributes, background and other attributes. We propose a new method, which only edits the code of w+ latent space of StyleGAN to complete the face-swapping and generate high-definition face images. We propose the GAN inversion method to improve the effect of face swapping, which combines convolution networks' advantages in extracting texture features and the benefits of transformers in extracting structure features. In the latent space of StyleGAN, the low-level feature layer is dominated by structure information, and the high-level feature layer is overwhelmed by texture information. Furthermore, we propose latent space selection, through which the neural network can learn disentangled representations of identity information in the latent space. Finally, we improved the post-processing process of face swapping to keep the image's background. Our method can complete face-swapping by editing the w+ space. Thus, high-quality face image can be generated and a lot of computing resource is saved on image reconstruction. At the same time, our method can keep other attributes better in the face-swapping process.

Hydrogenerators are strategic assets for power utilities. Their reliability and availability can lead to significant benefits. For decades, monitoring and diagnosis of hydrogenerators have been at the core of maintenance strategies. The main cause of hydrogenerator breakdown comes from failure of its high voltage stator, which is a major component of hydrogenerators. In our previous study, it was shown that more than 85% of stator failure mechanisms indicate the presence of Partial Discharges (PD) activity. PD are minute sparks that occur within voids inside high voltage insulation or in the air around the insulating system. Each PD event does not cause immediate failure, but it will slowly erode the insulation system and will lead to breakdown in years to decades. PD signal can be detected from the lead of the hydrogenerator while it is running, thus allowing for on-line diagnosis. Hydro-Québec has been collecting more than 33.000 unlabeled PD measurement files over the last decades including two types of measurement instruments. One of the instruments used is the Partial Discharge Analyzer (PDA). PDs have several different sources and each source is characterized by a specific signature. It is therefore essential to be able to automatically recognize the nature of the DPs in order to anticipate possible degradation. Expert rules to characterize these PD signals exist but these rules cannot be used as an automatic classification tool. Indeed, for certain ambiguous cases (conflict between classes) or the presence of several signatures at the same time (multi-classes), the expert's judgement remains essential.
 A Variationel AutoEncoder (VAE) is used for dimension reduction and projection into a 2D latent space to analyze the training data and then classify it. The 2D latent space of the VAE allows the data space to be restructured and reorganized to anticipate the performance of the classifier. The problem is how to optimize this latent space and thus obtain the best distribution of the different sources of PDs to maximize the chances of good responses from the classifier? To our knowledge, there is currently no method in the literature that clearly answers this question. What is certain, however, is that the optimization of the VAE learning is directly related to the quality of the learning base, i.e. a large size and a perfectly balanced database (all the cases are present and sufficiently represented). The objective of this paper is to compare the quality of the latent space obtained from the experts' rules with a latent space obtained directly from the input signal in an « End-to-End » approach.
 The first method concerns an original unsupervised deep learning method for PD recognition. Instead of labelling the PD measurement files by the expert for a supervised learning process, we use the rules developed by the experts of Hydro-Québec to create a feature vector from recognizable PD signatures. Indeed, labelling a sufficient quantity of signatures for a supervised approach is very time-consuming and therefore cannot be implemented. This is a common problem in the industry where more and more operational data is available but cannot be labelled by experts, who are busy with other tasks. Expert knowledge is then injected into a characteristic and feature vector. In the second method, several Generative Adversarial Networks (GANs) associated with several types of PDs are thus used to generate artificial signals in order to increase the size of the learning base and especially to balance it. A new latent space is thus obtained from the learning of the VAE exclusively carried out on data generated by the GANs.
 Validation tests based on the 33,000 measurements with metrics for evaluating the performances of the various latent spaces are used.

Unlabeled-unlabeled (UU) learning was proposed to cope with the high cost of data annotation and some realistic cases, in which we cannot get labeled data. It allows us to train a classifier with only unlabeled data. State-of-the-art (SOTA) UU methods with good performance based on neural networks (NN) have been proposed; however, there is a lack of studies on boosting algorithms for learning from only unlabeled data, even though boosting algorithms sometimes perform very well with simple base classifiers. We propose a novel boosting algorithm for UU learning: Ada-UU, which compares against neural networks. The proposed method follows the general procedure of AdaBoost while the classification error is estimated with two sets of unlabeled (U) data. We empirically demonstrate that Ada-UU outperforms neural networks on several large-scale benchmark UU datasets and has comparable performance on a small-scale benchmark dataset.KeywordsBoostingWeakly supervised learningUU learning

Pose estimation is a widely explored problem, enabling many robotic tasks such as grasping and manipulation. In this paper, we tackle the problem of pose estimation for objects that exhibit rotational symmetry, which are common in man-made and industrial environments. In particular, our aim is to infer poses for objects not seen at training time, but for which their 3D CAD models are available at test time. Previous work has tackled this problem by learning to compare captured views of real objects with the rendered views of their 3D CAD models, by embedding them in a joint latent space using neural networks. We show that sidestepping the issue of symmetry in this scenario during training leads to poor performance at test time. We propose a model that reasons about rotational symmetry during training by having access to only a small set of symmetry-labeled objects, whereby exploiting a large collection of unlabeled CAD models. We demonstrate that our approach significantly outperforms a naively trained neural network on a new pose dataset containing images of tools and hardware.

In the field of deep learning, the generative model did not attract much attention until GANs (generative adversarial networks) appeared. In 2014, Google’s Ian Goodfellow proposed a generative model called GANs. GANs use different structures and objective functions from the existing generative model. For example, GANs use two neural networks: a generator that creates a realistic image, and a discriminator that distinguishes whether the input is real or synthetic. If there are no problems in the training process, GANs can generate images that are difficult even for experts to distinguish in terms of authenticity. Currently, GANs are the most researched subject in the field of computer vision, which deals with the technology of image style translation, synthesis, and generation, and various models have been unveiled. The issues raised are also improving one by one. In image synthesis, BEGAN (Boundary Equilibrium Generative Adversarial Network), which outperforms the previously announced GANs, learns the latent space of the image, while balancing the generator and discriminator. Nonetheless, BEGAN also has a mode collapse wherein the generator generates only a few images or a single one. Although BEGAN-CS (Boundary Equilibrium Generative Adversarial Network with Constrained Space), which was improved in terms of loss function, was introduced, it did not solve the mode collapse. The discriminator structure of BEGAN-CS is AE (AutoEncoder), which cannot create a particularly useful or structured latent space. Compression performance is not good either. In this paper, this characteristic of AE is considered to be related to the occurrence of mode collapse. Thus, we used VAE (Variational AutoEncoder), which added statistical techniques to AE. As a result of the experiment, the proposed model did not cause mode collapse but converged to a better state than BEGAN-CS.

Reverse engineering applications for recovery of broken or worn parts and re-manufacturing: Three case studies

The manufacture of total hip arthroplasty (THA) requires the control of the quality of free form surfaces. In fact, the polyethylene insert is deformed to fit the overall geometry of the femoral part, which has an impact on the quality of the contact. In this paper, we propose a method for evaluating the defects of complex forms. The originality of the approach is the use of artificial intelligence to position the cloud of measured points, obtained with a three-dimensional measuring machine equipped with a contactless sensor, with regard to the 3D CAD model of the THA. The artificial intelligence algorithm used is based on neural networks that are trained using a virtual positioning realized with 3D CAD software. Finally, the difference between the positioned point cloud and the CAD model allows us to evaluate the shape defect of the measured THA surface. We found that the error of the proposed method is at the vicinity of micron scale.

Generative adversarial network (GAN)-based models possess superior capability of high-fidelity image synthesis. There are a wide range of semantically meaningful directions in the latent representation space of well-trained GANs, and the corresponding latent space walks are meaningful for semantic controllability in the synthesized images. To explore the underlying organization of a latent space, we propose an unsupervised Density-Preserving Latent Semantics Exploration model (DP-LaSE). The important latent directions are determined by maximizing the variations in intermediate features, while the correlation between the directions is minimized. Considering that latent codes are sampled from a prior distribution, we adopt a density-preserving regularization approach to ensure latent space walks are maintained in iso-density regions, since moving to a higher/lower density region tends to cause unexpected transformations. To further refine semantics-specific transformations, we perform subspace learning over intermediate feature channels, such that the transformations are limited to the most relevant subspaces. Extensive experiments on a variety of benchmark datasets demonstrate that DP-LaSE is able to discover interpretable latent space walks, and specific properties of synthesized images can thus be precisely controlled.

Deep neural networks are widely used in computer vision for image classification, segmentation and generation. They are also often criticised as “black boxes” because their decision-making process is often not interpretable by humans. However, learning explainable representations that explicitly disentangle the underlying mechanisms that structure observational data is still a challenge. To further explore the latent space and achieve generic processing, we propose a pipeline for discovering the explainable directions in the latent space of generative models. Since the latent space contains semantically meaningful directions and can be explained, we propose a pipeline to fully resolve the representation of the latent space. It consists of a Dirichlet encoder, conditional deterministic diffusion, a group-swap and a latent traversal module. We believe that this study provides an insight into the advancement of research explaining the disentanglement of neural networks in the community.