Arm-Aware Guided Dexterous Grasp Generation with Arm-Agnostic Grasp Models

Previous studies are mainly focused on the works that depth image is treated as flat image, and then depth data tends to be mapped as gray values during the convolution processing and features extraction. To address this issue, an approach of 3D CNN hand pose estimation with end-to-end hierarchical model and physical constraints is proposed. After reconstruction of 3D space structure of hand from depth image, 3D model is converted into voxel grid for further hand pose estimation by 3D CNN. The 3D CNN method makes improvements by embedding end-to-end hierarchical model and constraints algorithm into the networks, resulting to train at fast convergence rate and avoid unrealistic hand pose. According to the experimental results, it reaches 87.98% of mean accuracy and 8.82 mm of mean absolute error (MAE) for all 21 joints within 24 ms at the inference time, which consistently outperforms several well-known gesture recognition algorithms.

Aerodynamic shape optimization of aircraft configurations often ignores stability considerations. To address this, a method for the computation of static, dynamic, and transient aircraft stability derivatives and their sensitivities for use in gradient-based optimization is introduced and evaluated. Computational fluid dynamics in the form of a three-dimensional structured-grid multiblock flow solver with both Euler and Reynolds-averaged Navier–Stokes equations is used. To compute the stability derivatives, a time-spectral formulation is used to compute an oscillating solution for the configuration of interest. From this oscillating solution, a series of linear regressions is performed to calculate the various stability derivatives. Because the solution is time dependent, it contains the information required to compute the transient, or “dot,” derivatives for the configuration. An adjoint method is used to compute the gradients of the stability derivatives of interest, enabling gradient-based stability-constrained aerodynamic shape optimization with respect to a large number of design variables. The computed stability derivatives are verified for an airfoil and validated for a generic unmanned combat aerial vehicle. The stability-constrained optimization of a wing demonstrates the viability and usefulness of the method for aircraft design optimization.

Articulated hand tracking systems have been commonly used in virtual reality applications, including systems with human-computer interaction or interaction with game consoles. However, building an effective real-time hand pose tracker remains challenging. In this paper, we present a simple and efficient methodology for tracking and reconstructing 3d hand poses using a markered optical motion capture system. Markers were positioned at strategic points, and an inverse kinematics solver was incorporated to fit the rest of the joints to the hand model. The model is highly constrained with rotational and orientational constraints, allowing motion only within a feasible set. The method is real-time implementable and the results are promising, even with a low frame rate.

For multivariate co-generation in scientific applications, we advocate pairwise block rather than joint modeling of all variables. This design mitigates the computational burden and data imbalance. To this end, we propose an Annealed Co-Generation (ACG) framework that replaces high-dimensional diffusion modeling with a low-dimensional diffusion model, which enables multivariate co-generation by composing pairwise variable generations. We first train an unconditional diffusion model over causal variables that are disentangled into pairs. At inference time, we recover the joint distribution by coupling these pairwise models through shared common variables, enabling coherent multivariate generation without any additional training. By employing a three-stage annealing process-Consensus, Heating, and Cooling-our method enforces consistency across shared common variables and progressively constrains each pairwise data distribution to lie on a learnable manifold, while maintaining high likelihood within each pair. We demonstrate the framework's flexibility and efficacy on two distinct scientific tasks: flow-field completion and antibody generation. All datasets and code will be made publicly available upon publication.

This study presents a comparative evaluation of three prominent deep learning architectures XceptionNet, VGG19, and InceptionV3 for Pneumonia detection using chest X-ray images, with a focus on their suitability for deployment on Android mobile devices via TensorFlow Lite. The analysis considers both model accuracy and practical deployment metrics such as size, training duration, and inference time. Among the models, XceptionNet demonstrated the highest test accuracy (98.08%) and the fastest inference time (0.1534 seconds), making it the most balanced and efficient choice for mobile deployment. VGG19, while being memory efficient at 80.6MB, incurred the longest training and inference times, reducing its practicality for mobile use. InceptionV3 showed signs of overfitting, achieving high training accuracy but the lowest test accuracy (96.74%) among the three, with moderate performance in mobile environments. These findings show the importance of evaluating both accuracy and deployment feasibility when selecting models for real-world healthcare applications on edge devices like mobile. phones.

Face sketch synthesis is a technique aimed at converting face photos into sketches. Existing face sketch synthesis research mainly relies on training with numerous photo–sketch sample pairs from existing datasets. However, these large-scale discriminative learning methods will have to face problems, such as data scarcity and high human labor costs. Once the training data become scarce, their generative performance significantly degrades. In this article, we propose a one-shot face sketch synthesis method based on diffusion models. We optimize text instructions on a diffusion model using face photo–sketch image pairs. Then, the instructions derived through gradient-based optimization are used for inference. To simulate real-world scenarios more accurately and evaluate method effectiveness more comprehensively, we introduce a new benchmark named One-shot Face Sketch Dataset (OS-Sketch). The benchmark consists of 400 pairs of face photo–sketch images, including sketches with different styles and photos with different backgrounds, ages, sexes, expressions, illumination, and so on. For a solid out-of-distribution evaluation, we select only one pair of images for training at each time, with the rest used for inference. Extensive experiments demonstrate that the proposed method can convert various photos into realistic and highly consistent sketches in a one-shot context. Compared to other methods, our approach offers greater convenience and broader applicability. The dataset will be available at: https://github.com/HanWu3125/OS-Sketch .

Hand pose estimation for a continuous sequence has been an important topic not only in computer vision but also human-computer-interaction. Exploring the feasibility to use hand gestures to replace input devices, e.g., mouse, keyboard, joy-stick and touch screen, has attracted increasing attention from academic and industrial researchers. The fast advancement of hand pose estimation techniques is complemented by the rapid development of smart sensors technology such as Kinect and Leap. We introduce a hand pose estimation multi-sensor system. Two tracking models are proposed based on Deep (Recurrent) Neural Network (DRNN) architecture. Data captured from different sensors are analyzed and fused to produce an optimal hand pose sequence. Experimental results show that our models outperform previous methods with better accuracy, meeting real-time application requirement. Performance comparisons between DNN and DRNN, spatial and spatial-temporal features, and single- and dual- sensors, are also presented.

Highly constrained multiobjective optimization problems (HCMOPs) refer to constrained multiobjective optimization problems (CMOPs) with complex constraints and small feasible regions, which are commonly encountered in many real-world applications. Current constraint-handling techniques will face two difficulties when dealing with HCMOPs: 1) feasible solution is hard to be found and too much search effort is spent in locating the feasible region and 2) since the total feasible region of an HCMOP can consist of several disconnected subregions, the search process might be stuck in the comparatively larger feasible subregion, which does not contain the whole Pareto front (PF). To address these two issues, an evolutionary algorithm with constraint relaxation strategy based on differential evolution algorithm, that is, CRS-DE, is proposed in this article. In each generation, the CRS-DE relaxes the constraints by dividing the infeasible solutions into two subpopulations based on total constraint violation, that is, the ``semifeasible'' subpopulation (SF) and ``infeasible'' subpopulation (IF), respectively. The SF provides information on the promising regions of finding the feasible solution and is the driving force for convergence toward the PF, while the IF focuses on global exploration for new promising regions. Corresponding reproduction and selection strategies are devised for the SF, IF, and feasible subpopulations, which create a clear division of labor with cooperation to facilitate the search for feasible solutions. To leverage the influence of CRS and prevent the population from premature convergence, a mobility restriction mechanism is developed to restrict the individuals in the SF and IF from entering the feasible subpopulation and enhance the diversity of the whole population. Comprehensive experiments on a series of benchmark test problems and a real-world CMOP demonstrate the competitiveness of our method compared with other representative algorithms in terms of effectiveness and reliability in finding a set of well-distributed optimal solutions for HCMOPs.

Planning smooth trajectories through a sequence of waypoints under nonconvex constraints is challenging due to the coupling between coefficient optimization and time allocation. Existing gradient-based spline trajectory optimization methods tend to be susceptible to local minima and poor initializations, or restrained by complicated gradient computations. We propose a sampling augmented bilevel optimization (SABO) approach that integrates gradient-based optimization with correlated spatio-temporal sampling for improved robustness and optimality. Through temporal normalization, the closed-form solution of coefficient optimization becomes an explicit function of segment durations, while the Hessian becomes linear in their powers, enabling analytic bilevel gradient computation without using finite differences or linearized constraints. Correlated mutations are subsequently performed around the gradient-induced solution to further explore the constrained spatio-temporal space, with sample projection and covariance matrix adaptation to guide sampling towards low-cost, feasible regions. Simulations show that SABO outperforms existing methods in terms of optimality and robustness. We validate SABO in flight experiments conducted on a quadrotor.

Pattern-based calibration of complex subsurface flow models against dynamic response data

Denoising diffusion models have demonstrated tremendous success in modeling data distributions and synthesizing high-quality samples. In the 2D image domain, they have become the state-of-the-art and are capable of generating photo-realistic images with high controllability. More recently, researchers have begun to explore how to utilize diffusion models to generate 3D data, as doing so has more potential in real-world applications. This requires careful design choices in two key ways: identifying a suitable 3D representation and determining how to apply the diffusion process. In this survey, we provide the first comprehensive review of diffusion models for manipulating 3D content, including 3D generation, reconstruction, and 3D-aware image synthesis. We classify existing methods into three major categories: 2D space diffusion with pretrained models, 2D space diffusion without pretrained models, and 3D space diffusion. We also summarize popular datasets used for 3D generation with diffusion models. Along with this survey, we maintain a repository https://github.com/cwchenwang/awesome-3d-diffusion to track the latest relevant papers and codebases. Finally, we pose current challenges for diffusion models for 3D generation, and suggest future research directions.

Model compression methods have become popular in recent years, which aim to alleviate the heavy load of deep neural networks (DNNs) in real-world applications. However, most of the existing compression methods have two limitations: 1) they usually adopt a cumbersome process, including pretraining, training with a sparsity constraint, pruning/decomposition, and fine-tuning. Moreover, the last three stages are usually iterated multiple times. 2) The models are pretrained under explicit sparsity or low-rank assumptions, which are difficult to guarantee wide appropriateness. In this article, we propose an efficient decomposition and pruning (EDP) scheme via constructing a compressed-aware block that can automatically minimize the rank of the weight matrix and identify the redundant channels. Specifically, we embed the compressed-aware block by decomposing one network layer into two layers: a new weight matrix layer and a coefficient matrix layer. By imposing regularizers on the coefficient matrix, the new weight matrix learns to become a low-rank basis weight, and its corresponding channels become sparse. In this way, the proposed compressed-aware block simultaneously achieves low-rank decomposition and channel pruning by only one single data-driven training stage. Moreover, the network of architecture is further compressed and optimized by a novel Pruning & Merging (PM) module which prunes redundant channels and merges redundant decomposed layers. Experimental results (17 competitors) on different data sets and networks demonstrate that the proposed EDP achieves a high compression ratio with acceptable accuracy degradation and outperforms state-of-the-arts on compression rate, accuracy, inference time, and run-time memory.

Image inpainting techniques have recently been developed leveraging deep neural networks and have seen many real-world applications. However, image inpainting networks, which are typically based on generative adversarial network (GAN), suffer from high parameter complexities and long inference time. While there are some efforts to compress image-to-image translation GAN, compressing image inpainting networks has rarely been explored. In this paper, we aim to create a small and efficient GAN-based inpainting model by compressing the generator of the inpainting model without sacrificing the quality of reconstructed images. We propose novel channel pruning and knowledge distillation techniques that are specialized for image inpainting models with mask information. Experimental results demonstrate that our compressed inpainting model with only one-tenth of the model size achieves similar performance to the full model.KeywordsImage inpaintingNetwork pruningKnowledge distillation

Symbols are a universal way to convey complex information in technical drawings since they can represent a wide range of elements, including components, materials, or relationships, in a concise and space-saving manner. Therefore, to enable a digital and automatic interpretation of pixel-based drawings, accurate detection of symbols is a crucial step. To enhance the efficiency of the digitization process, current research focuses on automating this symbol detection using deep learning models. However, the ever-increasing repertoire of model architectures poses a challenge for researchers and practitioners alike in retaining an overview of the latest advancements and selecting the most suitable model architecture for their respective use cases. To provide guidance, this contribution conducts a comparative study of prevalent and state-of-the-art model architectures for the task of symbol detection in pixel-based construction drawings. Therefore, this study evaluates six different object detection model architectures, including YOLOv5, YOLOv7, YOLOv8, Swin-Transformer, ConvNeXt, and Faster-RCNN. These models are trained and tested on two distinct datasets from the bridge and residential building domains, both representing substantial sub-sectors of the construction industry. Furthermore, the models are evaluated based on five criteria, i.e., detection accuracy, robustness to data scarcity, training time, inference time, and model size. In summary, our comparative study highlights the performance and capabilities of different deep learning models for symbol detection in construction drawings. Through the comprehensive evaluation and practical insights, this research facilitates the advancement of automated symbol detection by showing the strengths and weaknesses of the model architectures, thus providing users with valuable guidance in choosing the most appropriate model for their real-world applications

Lightweight single image super-resolution with attentive residual refinement network