Learning Reaction-Diffusion Models for Image Inpainting

Diffusion models have emerged as a popular method for 3D generation. However, it is still challenging for diffusion models to efficiently generate diverse and high‐quality 3D shapes. In this paper, we introduce OctFusion, which can generate 3D shapes with arbitrary resolutions in 2.5 seconds on a single Nvidia 4090 GPU, and the extracted meshes are guaranteed to be continuous and manifold. The key components of OctFusion are the octree‐based latent representation and the accompanying diffusion models. The representation combines the benefits of both implicit neural representations and explicit spatial octrees and is learned with an octree‐based variational autoencoder. The proposed diffusion model is a unified multi‐scale U‐Net that enables weights and computation sharing across different octree levels and avoids the complexity of widely used cascaded diffusion schemes. We verify the effectiveness of OctFusion on the ShapeNet and Objaverse datasets and achieve state‐of‐the‐art performances on shape generation tasks. We demonstrate that OctFusion is extendable and flexible by generating high‐quality color fields for textured mesh generation and high‐quality 3D shapes conditioned on text prompts, sketches, or category labels. Our code and pre‐trained models are available at https://github.com/octree‐nn/octfusion.

In the real world, forecasts would always be based on historical data with the assumption that the behaviour be the same for the future. But how do we forecast when there is no such data available? New product or new technologies normally has limited amount of data available. Knowing that forecasting is valuable for decision making, this paper presents forecasting of new product or new technologies using aggregate diffusion models and modified Bass Model. A newly launched Proton car and its penetration was chosen to demonstrate the possibility of forecasting sales demand where there is limited or no data available. The model was developed to forecast diffusion of new vehicle or an innovation in the Malaysian society. It is to represent the level of spread on the new vehicle among a given set of the society in terms of a simple mathematical function that elapsed since the introduction of the new product. This model will forecast the car sales volume. A procedure of the proposed diffusion model was designed and the parameters were estimated. Results obtained by applying the proposed diffusion model and numerical calculation shows that the model is robust and effective for forecasting demand of the new vehicle. The results reveal that newly developed modified Bass diffusion of demand function has significantly contributed for forecasting the diffusion of new Proton car or new product.

Traditional diffusivity based denoising models detect edges by the gradients of intensities, and thus are easily affected by noise. In this paper, we develop a nonlinear diffusion denoising method which adapts to the local context and thus preserves edges and diffuses more in the smooth regions. In the proposed diffusion model, the modulus of gradient in a diffusivity function is substituted by the modulus of a wavelet detail coefficient and the diffusion of wavelet coefficients is performed based on the local context. The local context is derived directly by analyzing the energy of transform across the scales and thus it performs efficiently in the real-time. The redundant representation of the stationary wavelet transform (SWT) and its shift-invariance lend themselves to edge detection and denoising applications. The proposed stationary wavelet context-based diffusivity (SWCD) model produces a better quality image compared to that attained by two high performance diffusion models, i.e. higher Peak Signal-to-Noise Ratio on average and lesser artifacts and blur are observed in a number of images representing texture, strong edges and smooth backgrounds.

The mechanism of carbon diffusion in the as-cast and heat-treated grey iron and 304 stainless steel bimetallic composites was investigated in this study. A simplistic empirical model for the interface carbon diffusion was proposed and has been validated against the experimental data. A liquid-solid compound casting technique was used to fabricate the bimetallic composites. For the graphitizing annealing heat treatment, the bimetallic composite specimens were heated to 940 °C and held for 120 min and 240 min in an electric heating furnace. The proposed empirical model is based on the presence of eutectic carbides chill zone near the interface that formed during solidification of the liquid grey iron above the solid stainless steel substrate. It was found that the calculated carbon concentration profiles based on the proposed diffusion model were in good agreement with the experimental results.

In the present study, boron diffusion in the surface layers of AISI M2 borided steels and the growth kinetics of the FeB/Fe2B layers were estimated. The boriding of AISI M2 steel was performed according to the powder-pack method and was conducted at 1173-1323 K and at various exposure times. As a result of the boriding process, the diffusion-controlled growth of the FeB/Fe2B layers was obtained at the surface of the high-alloy steel, and the kinetics of the growth process changed parabolically over time. The boron diffusion coefficients were estimated by solving two simultaneous equations based on the limits of the boron concentration in each layer, the boride incubation time, and the parabolic growth constant. With the proposed diffusion model, an expression, which describes the evolution of the FeB/Fe2B layers, was obtained. Moreover, the proposed model and diverse empirical models presented in the literature provided a good fit to the experimental data obtained for 10 h of exposure and different boriding temperatures.

Defect formation and annealing behaviors of fluorine-implanted, unintentionally doped GaN layers were studied by positron annihilation spectroscopy (PAS). Single Ga vacancies (VGa) were identified as the main vacancy-type defects detected by PAS after fluorine implantation at 180 keV with a dose of 1×1015 cm−2. Implantation-induced VGa tend to aggregate and form vacancy clusters after postimplantation annealing in N2 ambient at 600 °C. Fluorine ions tend to form F-vacancy complexes quickly after thermal annealing, which is consistent with the proposed diffusion model that predicts the behaviors of fluorine in GaN.

Accurate delineation of the clinical target volume (CTV) is essential in the radiotherapy treatment of soft tissue sarcomas. However, this process is subject to inter-reader variability due to the need for clinical assessment of risk and extent of potential microscopic spread. This can lead to inconsistencies in treatment planning, potentially impacting treatment outcomes. Most existing automatic CTV delineation methods do not account for this variability and can only generate a single CTV for each case. This study aims to develop a deep learning-based technique to generate multiple CTV contours for each case, simulating the inter-reader variability in the clinical practice. We employed a publicly available dataset consisting of fluorodeoxyglucose positron emission tomography (FDG-PET), x-ray computed tomography (CT), and pre-contrast T1-weighted magnetic resonance imaging (MRI) scans from 51 patients with soft tissue sarcoma, along with an independent validation set containing five additional patients. An experienced reader drew a contour of the gross tumor volume (GTV) for each patient based on multi-modality images. Subsequently, two additional readers, together with the first one, were responsible for contouring three CTVs in total based on the GTV. We developed a diffusion model-based deep learning method that is capable of generating arbitrary number of different and plausible CTVs to mimic the inter-reader variability in CTV delineation. The proposed model incorporates a separate encoder to extract features from the GTV masks, leveraging the critical role of GTV information in accurate CTVdelineation. The proposed diffusion model demonstrated superior performance with the highest Dice Index (0.902 compared to values below 0.881 for state-of-the-art models) and the best generalized energy distance (GED) (0.209 compared to values exceeding 0.221 for state-of-the-art models). It also achieved the second-highest recall and precision metrics among the compared ambiguous image segmentation models. Results from both datasets exhibited consistent trends, reinforcing the reliability of our findings. Additionally, ablation studies exploring different model structures and input configurations highlighted the significance of incorporating prior GTV information for accurate CTVdelineation. The proposed diffusion model successfully generates multiple plausible CTV contours for soft tissue sarcomas, effectively capturing inter-reader variability in CTVdelineation.

Online social networks have recently become an innovative and effective method for spreading information among people around the world. Information diffusion, rumour spreading and diseases infection are all instances of stochastic processes that occur over the edges of social networks. Many prior works have carried out empirical studies and diffusion models to understand how information propagates in online social networks; however they suffer from problems. In this paper, we propose an information diffusion model inspired by information propagation among people. Our proposed Social Behavioural Information Diffusion Model, abbreviated as SBIDM, considers the effect of mainstream media like TV and radio, as well as interaction with the neighbours. The advantages of our approach are four-fold. First, it models information diffusion in social networks inspired by social life, which considers the effect of aggregate social behaviour to diffuse information; second, it allows partial knowledge to be held in each individual; third, it considers the effects of social media in propagating information as well as the effects of interacting with neighbours; and last but not least, it is applicable to different types of data including synthetic and well-known real social networks like Facebook, Amazon, Epinions and DBLP. To explore the advantages of our approach, many experiments with different settings and specifications were conducted. The obtained results are very promising.

Image inpainting techniques have recently been developed leveraging deep neural networks and have seen many real-world applications. However, image inpainting networks, which are typically based on generative adversarial network (GAN), suffer from high parameter complexities and long inference time. While there are some efforts to compress image-to-image translation GAN, compressing image inpainting networks has rarely been explored. In this paper, we aim to create a small and efficient GAN-based inpainting model by compressing the generator of the inpainting model without sacrificing the quality of reconstructed images. We propose novel channel pruning and knowledge distillation techniques that are specialized for image inpainting models with mask information. Experimental results demonstrate that our compressed inpainting model with only one-tenth of the model size achieves similar performance to the full model.KeywordsImage inpaintingNetwork pruningKnowledge distillation

Lift3Dreamer: Boosting Text-Driven Novel View Synthesis via Lifted 3D Inpainting Model from Single Images.

This paper presents a multi-frame superresolution approach to reconstruct a high-resolution image from several low-resolution video frames. The proposed algorithm consists of three steps: i) definition of a local search region for the optimal patch using motion vectors, ii) adaptive selection of the optimum patch based on lowresolution image degradation model, and iii) combination of the optimum patch and reconstructed image. As a result, the proposed algorithm can remove interpolation artifacts using directionally adaptive patch selection based on the lowresolution image degradation model. Moreover, superresolved images without distortion between consecutive frames can be generated. The proposed method provides a significantly improved super-resolution performance over existing methods in the sense of both subjective and objective measures including peak-to-peak signal-to-noise ratio (PSNR), structural similarity measure (SSIM), and naturalness image quality evaluator (NIQE). The proposed multi-frame super-resolution algorithm is designed for realtime video processing hardware by reducing the search region for optimal patches, and suitable for consumer imaging devices including ultra-high-definition (UHD) digital televisions, surveillance systems, and medical imaging systems for image restoration and enhancement.

Estimating the standard error of cross-Validation-Based estimators of classifier performance

An important task in image processing is the process of lling in missing parts of damaged images based on the information obtained from the surrounding areas. It is called inpainting. The goals of inpainting are numerous such as removing scratches in old photographic image, removing text and logos, restoration of damaged paintings. In this paper we present a nonlinear diusion model for image inpainting based

This study aims to investigate the feasibility of a single general model to synthesize CT images across body sites, thorax, abdomen, and pelvis, to support treatment planning for MRI-only radiotherapy. A total of 157 patients who received MRI-guided radiation therapy in the thorax, abdomen, and pelvis on a 0.35T MRIdian Linac were included. A subset of 122 cases were used for model training and the remaining 35 cases were used for model validation. All patient datasets had semi-paired CT-simulation image and 0.35T MR image acquired using TrueFISP. A conditional generative adversarial network with a multi-planar method was used to generate synthetic CT images from 0.35T MR images. The effect of preprocessing methods (with and without bias field corrections) on the quality of synthetic CT was evaluated and found to be insignificant. The general models trained on all cases performed comparably to the site-specific models trained on individual body sites. For all models, the peak signal-to-noise ratios ranged from 31.7 to 34.9 and the structural index similarity measures ranged from 0.9547 to 0.9758. For the datasets with bias field corrections, the mean-absolute-errors in HU (general model vs. site-specific model) were 49.7±9.4 vs. 49.5±8.9, 48.7±7.6 vs. 43±7.8 and 32.8±5.5 vs 31.8±5.3 for the thorax, abdomen, and pelvis, respectively. When comparing plans between synthetic CTs and ground truth CTs, the dosimetric difference was on average less than 0.5% (0.2 Gy) for target coverage and less than 2.1% (0.4 Gy) for organ-at-risk metrics for all body sites with either the general or specific models. Synthetic CT plans showed good agreement with mean gamma pass rates of >94% and >99% for 1%/1 mm and 2%/2 mm, respectively. This study has demonstrated the feasibility of using a general model for multiple body sites and the potential of using synthetic CT to support an MRI-guided radiotherapy workflow.&#xD.

Diffusion models have gained significant attention in the field of generative modeling due to their capability to produce high-quality samples. However, recent studies show that applying a uniform treatment to all distributions during the training of diffusion models is sub-optimal. In this paper, we present a comprehensive theoretical analysis of the forward process in diffusion models. Our findings indicate that distribution variations are not uniform throughout the diffusion process, with the sharpest changes occurring during the initial stages. Moreover, we observe that the initial distribution converges to a Gaussian distribution at an exponential rate, indicating that different initial distributions rapidly become quite similar during the forward diffusion process. Consequently, employing a uniform timestep sampling strategy does not effectively capture these dynamics, potentially leading to sub-optimal training outcomes for diffusion models. To remedy this, we introduce the Bidirectional Beta-Tuned Diffusion Model (BB-TDM). The BB-TDM leverages the Beta distribution to design the timestep sampling distribution and enhance the separation between different initial distributions during the diffusion process. By selecting appropriate parameters, the BB-TDM ensures that the timestep sampling distribution is aligned with the properties of the forward diffusion process and moderates the convergence speed of different initial distributions. Extensive experiments across various benchmark datasets on different diffusion models confirm the efficacy of the proposed BB-TDM.