Abstract Transient rendering simulates light in motion, measuring the time of flight from the light source to the camera. However, the stochastic nature of Monte Carlo is aggravated in transient rendering, since samples are now spread along the temporal domain. In our work, we propose to denoise transient Monte Carlo renders by exploiting the spatio‐temporal correlation of transient light transport, extending a recent statistical denoising formulation. By relying on statistics, we achieve a near‐optimal tradeoff between reduced variance and introduced bias. We efficiently collect per‐time‐bin statistics in the temporal domain while avoiding impractical memory requirements, and use these collected statistics to analyze the spatio‐temporal correlation and discriminate which time bins should be combined. Our statistics‐based transient denoiser does not hallucinate, guarantees convergence of the result, is efficient, does not require any training and naturally handles participating media. We believe that the generality of our method might pave the way for denoising time‐resolved Monte Carlo simulations in other domains, such as non‐line‐of‐sight imaging, acoustic rendering, or absorption microscopy.

Abstract We present an efficient and scalable method for the inverse shape design problem of elastic objects, with broad applicability to diverse materials and interactive editing. The core idea is to decouple material nonlinearity from geometry optimization by introducing the Cauchy stress tensor as an auxiliary variable. We design a three‐stage scheme that iteratively optimizes the stress tensors and the rest shape, with each stage being well‐posed and efficiently‐solvable. To address the lack of a theoretical convergence guarantee arising from the decoupled energy formulation, we incorporate a relaxation method that ensures robust stability in practice. As a result, our method achieves a 3 × speedup over the state‐of‐the‐art asymptotic method [Jia21] on a model with 40k vertices and 112k elements (Fig. 2), and exhibits near‐linear scalability to large systems (Fig. 8). We demonstrate applications including rest shape design for various materials (ranging from standard models to complex spline‐based materials [XSZB15]), interactive material and force editing, and elastic object reconstruction from images.

Abstract Sketching is a direct and inexpensive means of visual expression. Though image‐based sketching has been well studied, video‐based sketch animation generation is still very challenging due to the temporal coherence requirement. In this paper, we propose a novel end‐to‐end automatic generation approach for vector sketch animation. To solve the flickering issue, we introduce a Differentiable Motion Trajectory (DMT) representation that describes the frame‐wise movement of stroke control points using differentiable polynomial‐based trajectories. DMT enables global semantic gradient propagation across multiple frames, significantly improving the semantic consistency and temporal coherence, and producing high‐framerate output. DMT employs a Bernstein basis to balance the sensitivity of polynomial parameters, thus achieving more stable optimization. Instead of implicit fields, we introduce sparse track points for explicit spatial modeling, which improves efficiency and supports long‐duration video processing. Evaluations on DAVIS and LVOS datasets demonstrate the superiority of our approach over SOTA methods. Cross‐domain validation on 3D models and text‐to‐video data confirms the robustness and compatibility of our approach.

Abstract Virtual Reality (VR) applications aim to create an immersive virtual world, which demands a high level of visual realism. The analytical material models commonly used in VR often fall short of reproducing complex real‐world appearances. Recently, neural materials have emerged as a promising alternative, offering a compact yet effective representation of real‐world materials. Deploying neural materials on low‐power mobile VR devices poses significant challenges due to the computational complexity of neural networks and the high display resolution and frame rate requirements of VR devices (commonly 72+ frames per second). We address these challenges by leveraging texture‐space shading with spatiotemporal computation amortization, driven by a compact, coarse‐to‐fine neural material model of extremely low capacity. Thanks to our distillation training scheme, our compact neural materials achieve visual quality comparable to NeuMIP [KMX*21] at a much lower cost. Our method reaches over 90 FPS on a mobile VR device (Meta Quest 3) even under multiple light sources.

Abstract Recent advancements in Gaussian‐based human body reconstruction have achieved notable success in creating animatable avatars. However, there are ongoing challenges to fully exploit the SMPL model's prior knowledge and enhance the visual fidelity of these models to achieve more refined avatar reconstructions. In this paper, we introduce AniGaussian, which addresses the above issues with two insights. First, we propose an innovative pose‐guided deformation strategy that effectively constrains the dynamic Gaussian avatar using SMPL pose guidance, ensuring the reconstructed model not only captures detailed surface nuances but also maintains anatomical correctness across a wide range of motions. Second, we address the limitations of Gaussian models in representing dynamic human bodies in terms of expressiveness. We propose a novel canonical‐to‐observation rigidity prior to constrain the deformation field, which ensures geometric stability and significantly reduces unexpected artefacts during novel motions. Furthermore, we introduce a split‐with‐scale strategy that significantly improves geometry quality. The ablative study experiment demonstrates the effectiveness of our innovative model design. Through extensive comparisons with existing methods, AniGaussian demonstrates superior performance in both qualitative results and quantitative metrics.

Abstract As a task at the intersection of computer vision and natural language processing, image captioning offers significant application value in domains such as intelligent human–computer interaction, accessibility support and multimedia content retrieval. The primary objective is to generate natural language descriptions by interpreting visual features, traditionally relying on heterogeneous single‐stream grid features and region features. However, existing approaches face limitations: grid features struggle to balance global semantic perception with local detail analysis, and region features exhibit weakened spatial modelling efficacy due to sparse semantic correlations. Furthermore, fusing heterogeneous visual features often lacks effective control over complementarity and redundancy, leading to descriptions prone to semantic bias or detail omission. To address these challenges, we propose a novel Multi‐Gated Dual‐Stream Visual Feature Fusion (MGDSF) for Image Captioning. Our approach enhances the semantic accuracy and completeness of generated captions through dual‐stream feature extraction and a multi‐gated fusion (MGF) mechanism. First, we employ a Mamba‐like linear attention mechanism to construct a grid feature network with hierarchical positional awareness. This network achieves global modelling while maintaining local sensitivity by dynamically modulating information flow. Second, based on the Detection Transformer (DETR) framework, we design a region feature extractor to provide complementary local object visual information. Finally, we introduce a MGF module that balances the complementarity of dual‐stream visual features and suppresses cross‐modal information redundancy via multiple context‐aware gates, thereby achieving fine‐grained visual‐semantic alignment. Experiments on MS COCO demonstrate that MGDSF surpasses existing methods on multiple evaluation metrics, achieving METEOR, ROUGE‐L and CIDEr scores of 30.0%, 59.8% and 140.1%, respectively. These results validate the effectiveness of our proposed method and indicate its broad application potential.

Abstract Virtual try‐on is a highly active line of research, with increasing demand. Its task is replacing a piece of garment in an image with one from another while preserving person and garment characteristics as well as image fidelity. Current literature takes a supervised approach for the task, impairing generalisation and imposing heavy computation. In this paper, we present a novel zero‐shot training‐free method for embedding a clothing garment by reference. Our approach employs the prior of a diffusion model with no additional training, fully leveraging its native generalisation capabilities. The method employs extended attention to transfer image information from reference to target images, overcoming two significant challenges. We first initially warp the reference garment over the target human using deep features, alleviating ‘texture sticking.’ We then leverage the extended attention mechanism to encourage garment preservation while eliminating leakage of reference background and unwanted influence. Through a user study and a qualitative and quantitative comparison to state‐of‐the‐art approaches, we demonstrate superior image quality and garment preservation for unseen clothing pieces.

Abstract The parametric surface/surface intersection (SSI) computation serves as a fundamental component in geometric modelling kernels for computer‐aided design (CAD) systems. The geometric fidelity of intersection curves—particularly whether the computed intersection loci in the two parametric domains (‐curves) under the two surface maps agree with the true intersection curve in the modelling space—determines the watertightness of the surface trimming. Despite abundant research and industrial developments for SSI algorithms, ensuring the watertightness of the intersection remains challenging, which directly impacts the stability and reliability of the modelling systems. In this paper, we present a practical algorithm for computing parametric SSI with gap control between the maps in the modelling space of the two ‐curves. We first analyse the topology of the two ‐curves by solving lower‐dimensional systems of equations and build a graph in each domain representing the topology. Then we refine the graphs through adaptive edge subdivision and construct initial approximation of ‐curves by interpolation. A constrained optimization framework incorporating distance and tangential information is employed to improve accuracy and minimize gaps. We demonstrate the effectiveness of our algorithm through extensive experiments and comparisons with the intersection package in the open source software OCCT and the commercial engine ACIS.

Abstract In this paper, we propose a novel end‐to‐end method to model 3D objects with geometric details from a single‐view sketch input. Specifically, a novel deep learning‐based differentiable sketch renderer is introduced to establish the relationship between geometric features, represented by normal maps, and 2D sketch strokes. Then, building upon this renderer, we design algorithms to automatically create 3D models with geometric details from a single‐view sketch. With the aid of two introduced loss functions: one based on silhouette‐derived confidence maps and the other on regression similarities, our framework supports the gradient of loss functions calculated between the rendered sketch and input sketch back‐propagating through the whole architecture, thereby enhancing the geometric details on the frontal surface of the generated 3D object. Through comparisons with state‐of‐the‐art sketch‐based 3D modelling techniques, our approach demonstrates superior capability in generating plausible geometric shapes and details, without the necessity for semantic annotations within the input sketch.

Abstract Compressed voxel structures make it possible to store, interact with and display detailed and complex geometry. The Sparse Voxel DAG (directed acyclic graph) is such a compressed representation that supports real‐time rendering and interactive editing of high‐resolution voxel geometry. We present a novel encoding of the Sparse Voxel DAG that utilises the memory location of data to encode information about the structure of the voxel geometry. This encoding is not only more compact but also makes it possible to avoid memory accesses. In turn, this improves traversal performance, which we can observe in terms of increased ray tracing speed. Our new encoding retains compatibility with other existing methods. We demonstrate an adaption to the HashDAG data structure and show that our proposed encoding also results in better editing speed at similar memory requirements in this framework. Further, we demonstrate its compatibility with existing methods for storing voxel attributes such as colours.