Geometry-Guided Latent Diffusion Model for Static Point Cloud Color Attribute Denoising

Recent advances in text-to-video generation have demonstrated the substantial superiority of diffusion models. Nevertheless, generating high-resolution videos based on text description still faces a great challenge due to the enormous computation overhead for video diffusion model training. In this paper, we present a tuning-free video diffusion approach with Spatial-Temporal LAtent Grouping (ST-LAG), for highresolution video generation. ST-LAG exploits the prior knowledge of a pre-trained low-resolution video diffusion model for regionwise video latent denoising, and then combines all the denoised regions of video latent as a whole one to achieve global-wise spatial-temporal coherence. Specifically, ST-LAG denoises the whole video latents via two deliberately designed modules, e.g., Spatial Latent Grouping (SLG) and Temporal Latent Grouping (TLG), at spatial and temporal level, respectively. SLG spatially slices the latent of each frame into different local patches, and then feeds them into the low-resolution video diffusion model for local-region latent denoising. A text re-weighting scheme is devised in SLG to strength the cross-attention between features of text tokens and spatial regions to facilitate spatial-level finegrained details generation. TLG capitalizes on the segmentlevel latent grouping to match the length of each denoised local segment with the frame number in the training stage. The well aligned temporal receptive field facilitates better preservation of motion patterns. In each denoising step, all groups of video latent at spatial and temporal levels are fused together for highresolution video generation. Extensive experiments conducted on the ECTV-Prompt dataset demonstrate the effectiveness of our approach quantitatively and qualitatively.

Image denoising is a fundamental problem in computational photography, where achieving high perception with low distortion is highly demanding. Current methods either struggle with perceptual quality or suffer from significant distortion. Recently, the emerging diffusion model has achieved state-of-the-art performance in various tasks and demonstrates great potential for image denoising. However, stimulating diffusion models for image denoising is not straightforward and requires solving several critical problems. For one thing, the input inconsistency hinders the connection between diffusion models and image denoising. For another, the content inconsistency between the generated image and the desired denoised image introduces distortion. To tackle these problems, we present a novel strategy called the Diffusion Model for Image Denoising (DMID) by understanding and rethinking the diffusion model from a denoising perspective. Our DMID strategy includes an adaptive embedding method that embeds the noisy image into a pre-trained unconditional diffusion model and an adaptive ensembling method that reduces distortion in the denoised image. Our DMID strategy achieves state-of-the-art performance on both distortion-based and perception-based metrics, for both Gaussian and real-world image denoising.

Seismic waves contain information about the earthquake (EQ) source and many forms of noise deriving from the seismometer, anthropogenic effects, background noise associated with ocean waves, and microseismic noise. Separating the noise from the EQ signal is a critical first step in EQ physics and seismic waveform analysis. However, this is difficult because optimal parameters for filtering noise typically vary with time and may strongly alter the shape of the waveform. A few recent works have employed Deep Learning (DL) model for seismic denoising, among which we have taken as a benchmark Deep Denoiser and SEDENOSS. These models turn the noisy trace into a  2D signal (spectrograms) within the model to denoise the traces, making the process pretty heavy. We propose a novel DL-powered seismic denoising algorithm based on Diffusion Models (DMs), keeping the signal in 1D. DMs are the latest trend in Machine Learning (ML), having revolutionized the application fields of audio and image processing for denoising (DiffWave), synthesis (Stable Diffusion), and sequence modeling (STARS). The training of DMs proceeds by polluting a signal with noise until the signal has completely vanished into noise, then reversing the process by iterative denoising, conditioned on the latent signal representation. This makes DMs the ideal tool for seismic traces cleaning, as the model naturally learns from seismic sequences by denoising, which aligns the ML training procedure and the final task objective. In a preliminary evaluation, we used the Stanford Earthquake Dataset (STEAD); our proposed Diffusion-based Seismic Denoiser (DiffSD) outperforms the state-of-the-art DL methods on the Signal Noise Ratio (SNR),  Scale-Invariant Source to Distortion Ratio (SI-SDR), and Source to Distortion Ratio (SDR) metrics. DiffSD also yields qualitatively pleasing EQ traces out of visual inspection in time and frequency. Finally, DiffSD proceeds from regenerating clean EQ signals from noise, which opens the way to data-driven EQ sequence generations, potentially instrumental to further study and dataset augmentations.

High-throughput time-lapse imaging is a fundamental tool for efficient living cell profiling at single-cell resolution. Label-free phase-contrast video microscopy enables noninvasive, nontoxic, and long-term imaging. The tradeoff between speed and throughput, however, implies that despite the state-of-the-art autofocusing algorithms, out-of-focus cells are unavoidable due to the migratory nature of immune cells (velocities >10 μm/min). Here, we propose PostFocus to (i) identify out-of-focus images within time-lapse sequences with a classifier, and (ii) deploy a de-noising diffusion probabilistic model to yield reliable in-focus images. De-noising diffusion probabilistic model outperformed deep discriminative models with a superior performance on the whole image and around cell boundaries. In addition, PostFocus improves the accuracy of image analysis (cell and contact detection) and the yield of usable videos. Open-source code and sample data are available at: https://github.com/kwu14victor/PostFocus.

By formulating the data generation as a sequence procedure of denoising autoencoding, diffusion models have achieved superior in-painting performance on image data and beyond. Nevertheless, it is not trivial when capitalizing on diffusion models to generate missing 3D points. The difficulty originates from the intrinsic structure where 3D point cloud is a set of unordered and irregular coordinates. That motivates us to delve into the 3D structural information for designing point cloud encoder-decoder and shape latent generator, to precisely formulate the latent distribution of the complete point cloud and partial observation. In this paper, we propose Point cloud completion with Latent Diffusion Models (PointLDM), a new approach that leverages the conditional denoising diffusion probabilistic modeling (DDPM) in the 3D latent space for shape reconstruction. The architecture of PointLDM consists of a transformer-based variational auto-encoder (VAE) to model the complete shape latent, and a diffusion network for shape latent prediction. The encoder of VAE exploits both of global shape latent and local point features in shape distribution learning. With the learnt shape latent, the decoder first decodes the shape latent into coarse points, and then recovers the fine-grained details around each coarse point by deforming a 2D grid. To reconstruct the shape latent from partial observation, the diffusion network treats the partial observation as the conditional input and generates the shape latent via DDPM. Extensive experiments conducted on MVP, Completion3D, and KITTI quantitatively and qualitatively demonstrate the efficacy of PointLDM over the state-of-the-art shape completion approaches.

Training deep neural networks has become a common approach for addressing image restoration problems. An alternative for training a "task-specific" network for each observation model is to use pretrained deep denoisers for imposing only the signal's prior within iterative algorithms, without additional training. Recently, this approach has become increasingly popular with the rise of diffusion/score-based generative models, whose core is iterative denoising. Using denoisers for general purpose restoration requires guiding the iterations to ensure agreement of the signal with the observations. In low-noise settings, guidance that is based on back-projection (BP) has been shown to be a promising strategy (used recently in the context of diffusion models also under the names "pseudoinverse" or "range/null-space" guidance). However, the presence of noise in the observations hinders the gains from this approach. In this paper, we propose a novel guidance technique, based on preconditioning that allows traversing from BP-based guidance to least squares based guidance along the restoration scheme. The proposed approach is robust to noise while still having much simpler implementation than alternative methods (e.g., no SVD is required). We demonstrate its advantages for image deblurring and super-resolution.

Depth images are often accompanied by unavoidable and unpredictable noise. Depth image denoising algorithms mainly attempt to fill hole data and optimise edges. In this paper, we study in detail the problem of effectively filtering the data of depth images under noise interference. The classical filtering algorithm tends to blur edge and texture information, whereas the fractional integral operator can retain more edge and texture information. In this paper, the Grünwald–Letnikov-type fractional integral denoising operator is introduced into the depth image denoising process, and the convolution template of this operator is studied and improved upon to build a fractional integral denoising model and algorithm for depth image denoising. Depth images from the Redwood dataset were used to add noise, and the mask constructed by the fractional integral denoising operator was used to denoise the images by convolution. The experimental results show that the fractional integration order with the best denoising effect was −0.4 ≤ ν ≤ −0.3 and that the peak signal-to-noise ratio was improved by +3 to +6 dB. Under the same environment, median filter denoising had −15 to −30 dB distortion. The filtered depth image was converted to a point cloud image, from which the denoising effect was subjectively evaluated. Overall, the results prove that the fractional integral denoising operator can effectively handle noise in depth images while preserving their edge and texture information and thus has an excellent denoising effect.

Nonlocal orientation diffusion partial differential equation model for optics image denoising

The Gauss curvature-driven diffusion model for image denoising, which have been proposed by S H Lee and J K Seo, can preserve important structures of image. It uses the Gauss curvature as the conductance function that determines the amount of diffusion. However, the influence of image's gradient is not considered in the process of denoising. Therefore, the Gauss curvature-driven diffusion model would inevitably blur the edges where there are both high gradient and nonzero Gauss curvature. In this paper, we propose an improved method for image denoising by using a combination of the gradient and the Gauss curvature as the diffusion conductance. Some experiments show that the improved model outperforms the Gauss curvature-driven diffusion model in terms of both mean square error (MSE) and peak signal-to-noise ratio (PSNR).

The enrichment of the point clouds with colour images improves the visualisation of the data as well as the segmentation and recognition processes. Coloured point clouds are becoming increasingly common, however, the colour they display is not always as expected. Errors in the colouring of point clouds acquired with Mobile Laser Scanning are due to perspective in the camera image, different resolution or poor calibration between the LiDAR sensor and the image sensor. The consequences of these errors are noticeable in elements captured in images, but not in point clouds, such as the sky. This paper focuses on the correction of the sky-coloured points, without resorting to the images that were initially used to colour the whole point cloud. The proposed method consists of three stages. First the region of interest where the erroneously coloured points are accumulated, is selected. Second, the sky-coloured points are detected by calculating the colour distance in the Lab colour space to a sample of the sky-colour. And third, the colour of the sky-coloured detected points is restored from the colour of the nearby points. The method is tested in ten real case studies with their corresponding point clouds from urban and rural areas. In two case studies, sky-coloured points were assigned manually and the remaining eight case studies, the sky-coloured points are derived from the acquisition errors. The algorithm for sky-coloured points detection obtained an average F1-score of 94.7%. The results show a correct reassignment of colour, texture, and patterns, while improving the point cloud visualisation.

Abstract. Measuring the structure gauge of tunnels and other narrow passages has so far been the only way to evaluate whether large vehicles can pass through them. But especially for very long vehicles like train wagons and their cargo, the structure gauge is an insufficient measure because the center part of the vehicle between two bogies will inevitably leave the swept volume of its cross section when moving along any other trajectory than a straight line perpendicular to its cross section. In addition, the vehicle as well as the cargo must keep a minimum safety margin from the environment at all points of its trajectory. This paper explores an automated method to check for possible collisions of a model represented by a 3D point cloud moving through the 3D point cloud of an environment. We were given environment data of a train track through a narrow tunnel where simply relying on the structure gauge would indicate that a given wagon would pass through without any collision even though in reality, the train wagon would collide with the inner tunnel wall inside a sharp turn of the tracks. The k-d tree based collision detection method presented in this paper is able to correctly highlight these collisions and indicate the penetration depth of each colliding point of the environment into the model of the train wagon. It can be generalized for any setup where two static point clouds have to be tested for intersection along a trajectory.

With the increase of point cloud scale, the time required by traditional ICP-related point cloud registration methods increases dramatically, which cannot meet the registration requirements of large-scale point clouds. In this paper, a fast registration technique for large scale point clouds based on virtual viewpoint image generation is studied. Firstly, the projection image of color point cloud is generated by virtual viewpoint. Then, the feature is extracted based on ORB and the rotation and translation matrix is calculated. The experimental results show that the registration time of the proposed method is about 1s when the size of the point cloud is from 300,000 to 2 million, which is improved by 17–258 times compared with the traditional ICP registration method, and the registration error is reduced by 80% from ICP 5.0 to 1.0. This paper provides a new idea and method for large-scale color point cloud registration.

Synthesizing missing modalities in multi-parametric MRI (mpMRI) is vital for accurate tumor diagnosis, yet remains challenging due to incomplete acquisitions and modality heterogeneity. Diffusion models have shown strong generative capability, but conventional approaches typically operate in the image domain with high memory costs and often rely solely on noise-space supervision, which limits anatomical fidelity. Latent diffusion models (LDMs) improve efficiency by performing denoising in latent space, but standard LDMs lack explicit structural priors and struggle to integrate multiple modalities effectively. To address these limitations, we propose the anatomy-aware sketch-guided latent diffusion model (ASLDM), a novel LDM-based framework designed for flexible and structure-preserving MRI synthesis. ASLDM incorporates an anatomy-aware feature fusion module, which encodes tumor region masks and edge-based anatomical sketches via cross-attention to guide the denoising process with explicit structure priors. A modality synergistic reconstruction strategy enables the joint modeling of available and missing modalities, enhancing cross-modal consistency and supporting arbitrary missing scenarios. Additionally, we introduce image-level losses for pixel-space supervision using L1 and SSIM losses, overcoming the limitations of pure noise-based loss training and improving the anatomical accuracy of synthesized outputs. Extensive experiments on a five-modality orbital tumor mpMRI private dataset and a four-modality public BraTS2024 dataset demonstrate that ASLDM outperforms state-of-the-art methods in both synthesis quality and structural consistency, showing strong potential for clinically reliable multi-modal MRI completion. Our code is publicly available at: https://github.com/zltshadow/ASLDM.git.

Digital core technology is increasingly indispensable for the purpose of efficient development of sandstone reservoir in the Turgay basin central south Kazakhstan which could conduct multiple analyses and experiments without damaging the actual rock core, thereby enhancing efficiency, cutting costs, and minimizing resource wastage. The conventional methods for constructing the digital core mainly include high costs, while the construction cost of physical experimental methods is relatively high and the numerical reconstruction methods exhibit lower efficiency. An intelligent digital core construction framework combining multi-source experimental data with generative deep learning algorithms called MSED-GDL is proposed for quick and efficient reconstruction of the digital core. The framework begins with a U-net based autoencoder that encodes three-dimension (3D) images into a compact latent variable space, enhancing computational efficiency. This is followed by micro-seepage simulation and other processing on the 3D core database to gather multimodal data, including porosities, permeabilities, two-dimension (2D) thin section images, etc. A latent diffusion model is then employed to reconstruct high-fidelity 3D digital cores in seconds, capable of handling incomplete multimodal data inputs. To validate the MSED-GDL framework, 10000 digital core samples with the property of one typical sandstone reservoir in the Turgay basin central south Kazakhstan and a resolution of 256x256x256 were generated using Perlin noise and threshold filtering. Meanwhile, an autoencoder was adopted to compress these into a 16x16x16 tensor representation. Subsequently, the seepage simulations yielded multimodal data—comprising section images, mercury intrusion curves, phase permeability curves, permeabilities, etc. in various data forms (images, series, scalars). The latent diffusion model reconstructed the 3D digital cores using reverse diffusion and iterative denoising, achieving a reconstruction with minimal deviation from actual samples. The permeability, porosity, average throat radius, pore throat ratio, pore throat distribution, and relative permeability curve of the reconstructed digital core have strong consistency with multi-source experimental data indicating that the reconstructed three-dimensional digital core, in terms of geological, physical, and geometric, and other parameters that affect underground flow characteristics, has statistical significance similar to the original digital core and can serve as an approximate replacement for the original digital core.

The paper mainly focuses on the use of Latent Diffusion Models (LDMs) to address data scarcity, with an in-depth analysis of their practical performance in two important domains: medical imaging and modern industrial manufacturing. As a type of diffusion model, LDMs operate in low-dimensional latent spaces, avoiding the mode collapse issues of traditional Generative Adversarial Networks (GANs) while significantly reducing computational costs. In medical imaging, LDMs aid in generating high-quality, clinically relevant data while respecting privacy constraints; in industrial manufacturing, they support key tasks like enhancing defect detection by supplementing scarce defect samples. The paper further explores core challenges, including the lack of tailored evaluation criteria for LDM-generated images and risks to data privacy, like potential sensitive information leakage. Alongside this, the paper outlines future optimization directions, covering improvements to LDMs’ generalization capabilities and the development of more suitable assessment metrics for data-scarce scenarios so as to drive practical applications better.