Enhancing one-step diffusion models using GANs with application to mental health mindfulness

In the field of deep learning, the generative model did not attract much attention until GANs (generative adversarial networks) appeared. In 2014, Google’s Ian Goodfellow proposed a generative model called GANs. GANs use different structures and objective functions from the existing generative model. For example, GANs use two neural networks: a generator that creates a realistic image, and a discriminator that distinguishes whether the input is real or synthetic. If there are no problems in the training process, GANs can generate images that are difficult even for experts to distinguish in terms of authenticity. Currently, GANs are the most researched subject in the field of computer vision, which deals with the technology of image style translation, synthesis, and generation, and various models have been unveiled. The issues raised are also improving one by one. In image synthesis, BEGAN (Boundary Equilibrium Generative Adversarial Network), which outperforms the previously announced GANs, learns the latent space of the image, while balancing the generator and discriminator. Nonetheless, BEGAN also has a mode collapse wherein the generator generates only a few images or a single one. Although BEGAN-CS (Boundary Equilibrium Generative Adversarial Network with Constrained Space), which was improved in terms of loss function, was introduced, it did not solve the mode collapse. The discriminator structure of BEGAN-CS is AE (AutoEncoder), which cannot create a particularly useful or structured latent space. Compression performance is not good either. In this paper, this characteristic of AE is considered to be related to the occurrence of mode collapse. Thus, we used VAE (Variational AutoEncoder), which added statistical techniques to AE. As a result of the experiment, the proposed model did not cause mode collapse but converged to a better state than BEGAN-CS.

Reliable pore-scale particle tracking is pivotal for understanding non-Fickian transport in dual-porosity media, yet the computational burden of deep-learning pipelines can limit their practical use. This study systematically quantifies how two fundamental hyper-parameters—image resolution and training batch size—jointly shape accuracy and efficiency in a state-of-the-art tracker that couples SimVP video prediction with Trackpy-derived ground truth. Fluorescence microscopy movies (1328 frames, 3672 × 4100 px) of colloid migration through a PDMS micromodel were down-scaled to 64–400 px and trained with batch sizes of 4 or 8. Point-wise errors (MSE, MAE, RMSE), structural fidelity (SSIM, PSNR), and perceptual quality (LPIPS) were evaluated on validated trajectories, 1000 unseen pairs, and a blind hold-out set. Increasing resolution from 64 to 300 px raises pixel-based errors (MSE × ≈700) and inference time (0.26 s → 5 s batch⁻ 1 ) but unlocks a 20-fold rise in detected particles, while perceptual metrics remain above high-quality thresholds (SSIM > 0.999). Reducing the batch from 8 to 4 consistently halves to tenfold improves all error metrics, boosts PSNR by ~ 8 dB, and lowers LPIPS by up to 85%, with minimal variance penalties. Recommended operating points are: batch 4 @ 300 px for maximum fidelity, batch 4 @ 400 px for the densest particle fields, and batch 8 @ 128–256 px where real-time throughput dominates. These results provide concrete guidelines for balancing accuracy, particle yield, and computational cost in next-generation pore-scale imaging studies.

In traditional time-series image prediction tasks, both accuracy and image quality tend to deteriorate as the prediction horizon extends. To address this challenge in Shiitake mushroom fruiting body growth prediction, this study selected Shiitake mushroom strain No. 509, cultivated by the Shanghai Academy of Agricultural Sciences, as the experimental subject and proposed an enhanced model, VMRNN-DMSA, based on the Vision Mamba RNN Depth architecture. This model integrates a skip-connection mechanism with a Max Feature Map module to effectively filter and fuse features, enhancing feature representation and prediction accuracy. Additionally, a Spatial Attention Mechanism was introduced to strengthen the perception of key regions and improve spatial modeling. Furthermore, an Adaptive Kernel Convolution module with irregular context convolution kernels was incorporated to extract fine-grained local features and enhance visual quality. A weighted loss function was used to balance pixel-level accuracy, structural fidelity, and perceptual quality. This function combines Mean Squared Error Loss, Multi-Scale Structural Similarity, and Perceptual Loss. Experimental results showed that the proposed method achieved an MSE of 39.4255, an SSIM of 0.8579, and a PSNR of 22.0774. Compared with baseline models, MSE decreased by 29.05%, while SSIM and PSNR increased by 19.34% and 14.52%, respectively. These results indicate that VMRNN-DMSA significantly improves both prediction accuracy and image quality in long-term forecasting tasks, providing a reliable reference for the growth prediction of other edible fungi.

Low-light Image Enhancement (LLIE) aims to restore images captured under insufficient illumination to a normally lit appearance with adequate brightness, clear structures, and natural colors. However, existing methods still perform poorly in extremely dark conditions, where structural distortion, loss of details, amplified noise, and color imbalance frequently occur, making it difficult to maintain both structural fidelity and perceptual quality. To address these challenges, this paper proposes CA-YUVNet, a cross-attention-based low-light enhancement model. The model converts input images from RGB to YUV space to decouple luminance and chrominance, thereby suppressing noise propagation and color distortion commonly observed in low-light scenes. Furthermore, a cross-branch crossattention mechanism is introduced in the U and V branches to mitigate color shifts and chromatic artifacts. To further improve detail restoration and color consistency, a Bipolar Channel Attention Fusion (BCAF) module is designed, which integrates multi-scale convolution and deep feature fusion to enhance noise suppression and structural refinement. Experimental results on the LOL-v1, LOL-v2-real, and LOL-v2-synthetic datasets demonstrate that CA-YUVNet outperforms existing methods, validating its effectiveness and robustness.

In modern high-demand applications, such as real-time video communication, cloud gaming, and high-definition live streaming, achieving both superior transmission speed and high visual fidelity is paramount. However, unstable networks and packet loss remain major bottlenecks, making accurate and low-latency video error concealment a critical challenge. Traditional error control strategies, such as Forward Error Correction (FEC) and Automatic Repeat Request (ARQ), often introduce excessive latency or bandwidth overhead. Meanwhile, receiver-side concealment methods struggle under high motion or significant packet loss, motivating the exploration of predictive models. SimVP-v2, with its efficient convolutional architecture and Gated Spatiotemporal Attention (GSTA) mechanism, provides a strong baseline by reducing complexity and achieving competitive prediction performance. Despite its merits, SimVP-v2’s reliance on 2D convolutions for implicit temporal aggregation limits its capacity to capture complex motion trajectories and long-term dependencies. This often results in artifacts such as motion blur, detail loss, and accumulated errors. Furthermore, its single-modality design ignores the complementary contextual cues embedded in the audio stream. To overcome these issues, we propose A3DSimVP (Audio- and 3D-Enhanced SimVP-v2), which integrates explicit spatio-temporal modeling with multimodal feature fusion. Architecturally, we replace the 2D depthwise separable convolutions within the GSTA module with their 3D counterparts, introducing a redesigned GSTA-3D module that significantly improves motion coherence across frames. Additionally, an efficient audio–visual fusion strategy supplements visual features with contextual audio guidance, thereby enhancing the model’s robustness and perceptual realism. We validate the effectiveness of A3DSimVP’s improvements through extensive experiments on the KTH dataset. Our model achieves a PSNR of 27.35 dB, surpassing the 27.04 of the SimVP-v2 baseline. Concurrently, our improved A3DSimVP model reduces the loss metrics on the KTH dataset, achieving an MSE of 43.82 and an MAE of 385.73, both lower than the baseline. Crucially, our LPIPS metric is substantially lowered to 0.22. These data tangibly confirm that A3DSimVP significantly enhances both structural fidelity and perceptual quality while maintaining high predictive accuracy. Notably, A3DSimVP attains faster inference speeds than the baseline with only a marginal increase in computational overhead. These results establish A3DSimVP as an efficient and robust solution for latency-critical video applications.

This paper studies the cooperative training of two generative models for image modeling and synthesis. Both models are parametrized by convolutional neural networks (ConvNets). The first model is a deep energy-based model, whose energy function is defined by a bottom-up ConvNet, which maps the observed image to the energy. We call it the descriptor network. The second model is a generator network, which is a non-linear version of factor analysis. It is defined by a top-down ConvNet, which maps the latent factors to the observed image. The maximum likelihood learning algorithms of both models involve MCMC sampling such as Langevin dynamics. We observe that the two learning algorithms can be seamlessly interwoven into a cooperative learning algorithm that can train both models simultaneously. Specifically, within each iteration of the cooperative learning algorithm, the generator model generates initial synthesized examples to initialize a finite-step MCMC that samples and trains the energy-based descriptor model. After that, the generator model learns from how the MCMC changes its synthesized examples. That is, the descriptor model teaches the generator model by MCMC, so that the generator model accumulates the MCMC transitions and reproduces them by direct ancestral sampling. We call this scheme MCMC teaching. We show that the cooperative algorithm can learn highly realistic generative models.

Dynamic 3D point clouds enable the immersive user experience and thus have become increasingly more popular in volumetric video streaming applications. When being streamed over best-effort networks, point cloud frames may suffer from lost or late packets, leading to non-trivial quality degradation. To solve this problem, we proposed the very first error concealment pipeline framework, which comprises five stages: pre-processing, matching, motion estimation, prediction, and post-processing. Alternative algorithms can be developed for each stage, while algorithms of different stages could be mixed and matched into pipelines for end-to-end performance evaluations. We discussed the design goal and proposed multiple algorithms for each stage. These algorithms were then quantitatively compared using dynamic 3D point cloud sequences with diverse characteristics. Based on the comparison results, we proposed four representative pipelines for: (i) diverse degrees of motion variance, i.e., minor versus significant, and (ii) different application requirements, i.e., high quality versus low overhead. Extensive end-to-end evaluations of our proposed pipelines demonstrated their superior concealed quality over the 3D frame-copy method in both: (i) 3D metrics, by up to 5.32 dB in GPSNR and 1.7 dB in CPSNR,and (ii) 2D metrics, by up to 2.22 dB in PSNR, 0.06 in SSIM, and 11.67 in VMAF. Adding to that, a user study with 15 subjects indicated that our best-performing pipeline achieved 100% preference winning rate over the state-of-the-art learning-based interpolation algorithms while consuming merely up to 8.55% of running time.

Existing low-light enhancement methods typically rely on fitting data mappings (pixel-wise mappings through fully supervised methods or distribution-wise mappings through weakly supervised or self-supervised methods). However, their performance is heavily dependent on specific scenes and fails to adequately model the intrinsic prior of natural images, resulting in poor generalization. To tackle this challenge, we leverage the strengths of powerful generative diffusion models, conditioned on a thoughtfully designed prior, and propose a novel zero-reference low-light enhancement framework that gets rid of dependence on the distribution of low-light images. In detail, we address the most fundamental core by proposing an illumination-invariant prior derived from the theory of physical light transfer, bridging the gap between normal and low-light domains, and enabling zero-shot enhancement without the need for low-light-specific training. A prior-to-image restoration framework is built upon generative diffusion models, pre-trained on normal-light data. During inference, the framework extracts the illumination-invariant prior from low-light inputs and maps them back to high-quality images, naturally for low-light enhancement. Additionally, such intrinsic properties of illumination-invariant prior open up opportunities for distilling diffusion models into compact CNN-based networks. We propose a novel prior-injected distillation paradigm incorporating intensity, frequency, and gradient domain-augmented regularization comprehensively. This distillation framework not only reduces computational costs but also maintains high fidelity and perceptual quality in enhanced outputs, making it more efficient and practical for real-world applications. The approach further extends seamlessly to handle over-exposure scenarios, demonstrating its versatility in addressing complex lighting conditions. Extensive experiments demonstrate the superiority of our framework in various scenarios, as well as its strong interpretability, robustness, and efficiency.

In agricultural images acquired under natural conditions, pomegranate fruits are often partially occluded by leaves and branches, resulting in missing structural information that compromises the accuracy of yield estimation and automated harvesting. To overcome the challenges of recovering structural integrity in occluded agricultural imagery, we propose the Conditional Segmentation-guided Diffusion Network (CSD-Net). CSD-Net is a lightweight, unified framework, representing the first conditional diffusion model specifically designed for the joint tasks of pomegranate image completion and segmentation. CSD-Net aims to address the structural fidelity limitations of traditional completion methods. It utilizes a shared encoder, a segmentation branch, and an RGB diffusion branch. Crucially, the network leverages the segmentation mask as a key structural prior condition to guide the diffusion generation process. This innovative conditional guidance mechanism ensures high-fidelity reconstruction of fruit structures while maintaining spatial and textural consistency. Experimental results demonstrate that CSD-Net substantially outperforms conventional methods across metrics, achieving 30.37 dB in PSNR and 0.9490 in SSIM. Furthermore, its model size is only 117 MB, striking an effective balance between high completion quality and inference efficiency. This study offers a novel and highly effective solution for mitigating occlusion issues in agricultural visual perception tasks. Upon acceptance of this paper, the source code will be made publicly available at https://github.com/zdkd/PCSN.

Images created using the Super Resolution method can generate more information compared to their low resolution counterparts. A super-resolved image, which is created using an original image captured by an imaging source is not only more meaningful to human perception but also has advantages on downstream tasks such as object detection and pattern recognition. In this work, we aim to apply the Super Resolution method to the Aerial Images captured for surveillance to enable more information about the original scenes. To achieve this Super Resolution Generative Adversarial Network (SRGAN), which is based on the Generative Adversarial Networks architecture is used. We also applied transfer learning methodology to achieve better image quality. Public xView and DOTA datasets which contain images mostly captured by satellites around the world are used to train a generative model via SRGAN architecture. Furthermore, DIV2K dataset is used to pre-train a generative model, and then the transfer learning technique is used to train separate models on xView and DOTA validation datasets. Perceptual Index (PI) and Root Mean Squared Error (RMSE) which are used on European Conference on Computer Vision -Perceptual Image Restoration and Manipulation Workshop 2018 are computed as the performance metrics. We have seen that the model which gives the best PI results, i.e. better perceptual quality, on xView and DOTA validation datasets is the one trained using the DIV2K dataset and the model which gives the best RMSE results, i.e. better reconstruction quality, is the one trained using the transfer learning technique.

Recently, manipulation of real-world images has been highly elaborated along with the development of Generative Adversarial Networks (GANs) and corresponding encoders, which embed real-world images into the latent space. However, designing encoders of GAN still remains a challenging task due to the trade-off between distortion and perception. In this paper, we point out that the existing encoders try to lower the distortion not only on the interest region, e.g., human facial region but also on the uninterest region, e.g., background patterns and obstacles. However, most uninterest regions in real-world images are located at out-of-distribution (OOD), which are infeasible to be ideally reconstructed by generative models. Moreover, we empirically find that the uninterest region overlapped with the interest region can mangle the original feature of the interest region, e.g., a microphone overlapped with a facial region is inverted into the white beard. As a result, lowering the distortion of the whole image while maintaining the perceptual quality is very challenging. To overcome this trade-off, we propose a simple yet effective encoder training scheme, coined IntereStyle, which facilitates encoding by focusing on the interest region. IntereStyle steers the encoder to disentangle the encodings of the interest and uninterest regions. To this end, we filter the information of the uninterest region iteratively to regulate the negative impact of the uninterest region. We demonstrate that IntereStyle achieves both lower distortion and higher perceptual quality compared to the existing state-of-the-art encoders. Especially, our model robustly conserves features of the original images, which shows the robust image editing and style mixing results. We will release our code with the pre-trained model after the review.KeywordsStyleGANRobust GAN inversionInterest regionInterest disentanglementUninterest filter

The existing image compression methods usually choose or optimize low-level representation manually. Actually, these methods struggle for the texture restoration at low bit rates. Recently, deep neural network (DNN)-based image compression methods have achieved impressive results. To achieve better perceptual quality, generative models are widely used, especially generative adversarial networks (GAN). However, training GAN is intractable, especially for high-resolution images, with the challenges of unconvincing reconstructions and unstable training. To overcome these problems, we propose a novel DNN-based image compression framework in this paper. The key point is decomposing an image into multi-scale sub-images using the proposed Laplacian pyramid based multi-scale networks. For each pyramid scale, we train a specific DNN to exploit the compressive representation. Meanwhile, each scale is optimized with different aspects, including pixel, semantics, distribution and entropy, for a good "rate-distortion-perception" trade-off. By independently optimizing each pyramid scale, we make each stage manageable and make each sub-image plausible. Experimental results demonstrate that our method achieves state-of-the-art performance, with advantages over existing methods in providing improved visual quality. Additionally, a better performance in the down-stream visual analysis tasks which are conducted on the reconstructed images, validates the excellent semantics-preserving ability of the proposed method.

Generative models able to synthesize layouts of different kinds (e.g. documents, user interfaces or furniture arrangements) are a useful tool to aid design processes and as a first step in the generation of synthetic data, among other tasks. We exploit the properties of self-attention layers to capture high level relationships between elements in a layout, and use these as the building blocks of the well-known Variational Autoencoder (VAE) formulation. Our proposed Variational Transformer Network (VTN) is capable of learning margins, alignments and other global design rules without explicit supervision. Layouts sampled from our model have a high degree of resemblance to the training data, while demonstrating appealing diversity. In an extensive evaluation on publicly available benchmarks for different layout types VTNs achieve state-of-the-art diversity and perceptual quality. Additionally, we show the capabilities of this method as part of a document layout detection pipeline.

We have witnessed rapid progress on 3D-aware image synthesis, leveraging recent advances in generative visual models and neural rendering. Existing approaches however fall short in two ways: first, they may lack an underlying 3D representation or rely on view-inconsistent rendering, hence synthesizing images that are not multi-view consistent; second, they often depend upon representation network architectures that are not expressive enough, and their results thus lack in image quality. We propose a novel generative model, named Periodic Implicit Generative Adversarial Networks ($π$-GAN or pi-GAN), for high-quality 3D-aware image synthesis. $π$-GAN leverages neural representations with periodic activation functions and volumetric rendering to represent scenes as view-consistent 3D representations with fine detail. The proposed approach obtains state-of-the-art results for 3D-aware image synthesis with multiple real and synthetic datasets.

With the emergence of online MRI radiotherapy treatments, MR-based workflows have increased in importance in the clinical workflow. However proper dose planning still requires CT images to calculate dose attenuation due to bony structures. In this paper, we present a novel deep image synthesis model that generates in an unsupervised manner CT images from diagnostic MRI for radiotherapy planning. The proposed model based on a generative adversarial network (GAN) consists of learning a new invariant representation to generate synthetic CT (sCT) images based on high frequency and appearance patterns. This new representation encodes each convolutional feature map of the convolutional GAN discriminator, leading the training of the proposed model to be particularly robust in terms of image synthesis quality. Our model includes an analysis of common histogram features in the training process, thus reinforcing the generator such that the output sCT image exhibits a histogram matching that of the ground-truth CT. This CT-matched histogram is embedded then in a multi-resolution framework by assessing the evaluation over all layers of the discriminator network, which then allows the model to robustly classify the output synthetic image. Experiments were conducted on head and neck images of 56 cancer patients with a wide range of shape sizes and spatial image resolutions. The obtained results confirm the efficiency of the proposed model compared to other generative models, where the mean absolute error yielded by our model was 26.44(0.62), with a Hounsfield unit error of 45.3(1.87), and an overall Dice coefficient of 0.74(0.05), demonstrating the potential of the synthesis model for radiotherapy planning applications.