Model-Based Deep Learning for Beam Prediction Based on a Channel Chart

Model-informed deep-learning photoacoustic reconstruction for low-element linear array

Magnetic particle imaging (MPI) offers exceptional contrast for magnetic nanoparticles (MNP) at high spatio-temporal resolution. A common procedure in MPI starts with a calibration scan to measure the system matrix (SM), which is then used to set up an inverse problem to reconstruct images of the MNP distribution during subsequent scans. This calibration enables the reconstruction to sensitively account for various system imperfections. Yet time-consuming SM measurements have to be repeated under notable changes in system properties. Here, we introduce a novel deep learning approach for accelerated MPI calibration based on Transformers for SM super-resolution (TranSMS). Low-resolution SM measurements are performed using large MNP samples for improved signal-to-noise ratio efficiency, and the high-resolution SM is super-resolved via model-based deep learning. TranSMS leverages a vision transformer module to capture contextual relationships in low-resolution input images, a dense convolutional module for localizing high-resolution image features, and a data-consistency module to ensure measurement fidelity. Demonstrations on simulated and experimental data indicate that TranSMS significantly improves SM recovery and MPI reconstruction for up to 64-fold acceleration in two-dimensional imaging.

Photon-counting detectors are greatly improving the resolution and image quality in computed tomography (CT) technology. The drawback is, however, that the reconstruction becomes more challenging. This is because there is a considerable increment of the processing data due to the multiple energy bins and materials in the reconstruction analysis, as well as improved resolution. Yet efficient material decomposition and reconstruction methods tend to generate noisy images that do not completely satisfy the expected image quality. Therefore, there is a need for efficient denoising of the resulting material images. We present a new and fast denoiser that is based on a linear minimum mean square error (LMMSE) estimator. The LMMSE is very fast to compute, but not commonly used for CT image denoising, probably due to its inability to adapt the amount of denoising to different parts of the image and the difficulty to derive accurate statistical properties from the CT data. To overcome these problems we propose a model-based deep learning strategy, that is, a deep neural network that preserves an LMMSE structure (model-based), providing more robustness unseen data, as well as good interpretability to the result. In this way, the solution adapts to the anatomy in every point of the image and noise properties at that particular location. In order to asses the performance of the new method, we compare it to both to a conventional LMMSE estimator and to a “black-box” CNN in a simulation study with anthropomorphic phantoms.

The use of photo-activated fluorescent molecules to create long sequences of low emitter-density diffraction-limited images enables high-precision emitter localization, but at the cost of low temporal resolution. We suggest combining SPARCOM, a recent high-performing classical method, with model-based deep learning, using the algorithm unfolding approach, to design a compact neural network incorporating domain knowledge. Our results show that we can obtain super-resolution imaging from a small number of high emitter density frames without knowledge of the optical system and across different test sets using the proposed learned SPARCOM (LSPARCOM) network. We believe LSPARCOM can pave the way to interpretable, efficient live-cell imaging in many settings, and find broad use in single molecule localization microscopy of biological structures.

Multiphase flow is ubiquitous in nature, industry and research, and accurate flow imaging is critical to understanding this complex phenomenon. Electrical tomography (ET) is a promising technique for multiphase flow visualization and characterization which provides a non-invasive and non-radiative way to unravel the internal physical properties at high temporal resolution. However, existing ET-based multiphase flow imaging methods are inadequate for quantitative imaging of multiphase flows due to inversion errors and limited ground truth data of fluid phases distribution. Here we report a digital twin (DT) framework of ET to address the challenges of real-time quantitative multiphase flow imaging. The proposed DT framework, building upon a synergistic integration of 3D field coupling simulation, model-based deep learning, and edge computing, allows ET to dynamically learn the flow features in the virtual space and implement the model in the physical system, thus providing excellent resolution and accuracy. The proposed DT framework is demonstrated using electrical capacitance tomography (ECT) of a gas-liquid two-phase flow. It can be readily extended to a broader range of tomography modalities, scenarios, and scales in biomedical, energy, and aerospace applications.

To complete complex tasks with space robots in the impedance environment, sliding mode control (SMC) has been proposed and plays an important role in the past decades. Learning-based control has also met with great success in robotics and has been closely followed in the control of space robots. However, as an interdisciplinary, only a few studies have combined deep learning and traditional controller in the space robot directly. This paper proposes a novel fractional-order fixed-time SMC for impedance control of space robots, taking advantage of a model-based deep learning approach to obtain the dynamics of space robots. The learning results are used to make predictions for the data-based robot dynamics. The fractional-order fixed-time SMC controller is added to that data-based model, and it ensures the finite closed-loop convergence time. Small-dataset training is more reasonable in the situation of space robots, which reduces training time and sampling size. Compared to traditional methods, combining deep learning and fractional-order fixed-time SMC provides better tracking performance and improves closed-loop robustness. It accomplishes the control target in the impedance model using the estimated data-driven parameters. Theoretical analysis proves the closed-loop behavior, and simulation results verify its effectiveness.

Direction of arrival (DOA) estimation is widely used in the field of array signal processing. The model-based algorithms rely on domain knowledge and assumptions, facing limitations in estimating coherent sources and running on few snapshots etc. In contrast, deep learning approaches can learn from data, offering a promising alternative for DOA estimation. In this paper, a novel end-to-end model-based deep learning DOA estimation architecture (MD-DOA) is proposed to estimate the DOAs of multiple narrowband signals captured by a uniform linear array. Specifically, the multi-branch convolutional recurrent neural network with a residual link (MBCR2net) is developed to extract multi-scale features and learn correlation in received temporal signals. Subsequently, the weighted noise subspace network (WNSnet) is proposed to learn a more representative noise subspace from the one obtained by eigenvalue decomposition (EVD), developing the more precise subspace division. The matrix reshape process (MRP) then generates the pseudo covariance matrix (PCM) and captures the correlation in the weighted noise subspace. Notably, EVD and MRP are the model-based modules to preserve the interpretability. Finally, the PCM-based DOA-finding network (PDFnet) estimates the desired DOAs. MD-DOA integrates the model-based and data-driven advantages. It inherits the overall framework of the subspace-based methods while using the network to augment the covariance matrix estimation, subspace division and peak-finding process. Our proposed architecture can operate successfully in the presence of array mismatch, low signal-to-noise ratios (SNRs), and few snapshots. It is also applicable to real-world measurements and demonstrates superior performance compared with other existing algorithms in this field.

Optimizing power allocation in cellular systems with deep learning enables real-time coordination of inter-cell interference. When channels are time-varying, the deep neural networks (DNNs) need to be re-trained frequently and the training samples need to be re-collected in a timely manner. To achieve higher sum rate with fewer training samples and lower training cost, domain knowledge should be resorted for designing DNNs. In this paper, we propose a DNN structure where the formula of data rate is used to facilitate the learning of power allocation policy, called data-rate based DNN (DRNN). Since such a model-based deep learning method does not exclude the use of prior knowledge for reducing the hypothesis space of a DNN, we further exploit a permutation equivariance prior by introducing parameter sharing into the DNN structure. By integrating the model and prior into DNN, simulations show that either sum rate is improved for given number of training samples or training complexity is reduced to achieve an expected performance.

We propose an image acquisition and reconstruction method based on segment-wise signal sampling and skipping. Unrolling model-based deep learning reconstruction is used to improve the quality of reconstructed images and reduce the reconstruction time. Simulation experiments show that the skipped signals were reconstructed in the iterative reconstruction based on a convolutional neural network and that the evaluation scores of reconstructed images were improved for unsegmented band-limited signals. The proposed method is applied to an experimentally obtained phase-scrambling Fourier transform signal to demonstrate its effectiveness.

Accurate delineation of the clinical target volume (CTV) is essential in the radiotherapy treatment of soft tissue sarcomas. However, this process is subject to inter-reader variability due to the need for clinical assessment of risk and extent of potential microscopic spread. This can lead to inconsistencies in treatment planning, potentially impacting treatment outcomes. Most existing automatic CTV delineation methods do not account for this variability and can only generate a single CTV for each case. This study aims to develop a deep learning-based technique to generate multiple CTV contours for each case, simulating the inter-reader variability in the clinical practice. We employed a publicly available dataset consisting of fluorodeoxyglucose positron emission tomography (FDG-PET), x-ray computed tomography (CT), and pre-contrast T1-weighted magnetic resonance imaging (MRI) scans from 51 patients with soft tissue sarcoma, along with an independent validation set containing five additional patients. An experienced reader drew a contour of the gross tumor volume (GTV) for each patient based on multi-modality images. Subsequently, two additional readers, together with the first one, were responsible for contouring three CTVs in total based on the GTV. We developed a diffusion model-based deep learning method that is capable of generating arbitrary number of different and plausible CTVs to mimic the inter-reader variability in CTV delineation. The proposed model incorporates a separate encoder to extract features from the GTV masks, leveraging the critical role of GTV information in accurate CTVdelineation. The proposed diffusion model demonstrated superior performance with the highest Dice Index (0.902 compared to values below 0.881 for state-of-the-art models) and the best generalized energy distance (GED) (0.209 compared to values exceeding 0.221 for state-of-the-art models). It also achieved the second-highest recall and precision metrics among the compared ambiguous image segmentation models. Results from both datasets exhibited consistent trends, reinforcing the reliability of our findings. Additionally, ablation studies exploring different model structures and input configurations highlighted the significance of incorporating prior GTV information for accurate CTVdelineation. The proposed diffusion model successfully generates multiple plausible CTV contours for soft tissue sarcomas, effectively capturing inter-reader variability in CTVdelineation.

We present two model-based neural network architectures purposed for sporadic user detection and channel estimation in massive machine-type communications. In the scenario under consideration, a base station assigns the users a set of pilot sequences that is linearly dependent, but because user activity is sporadic the detection/estimation problem is amenable to sparse recovery algorithms. Further, we consider a millimeter-wave wireless channel, so that the channel vectors are sparse in a known dictionary. We apply the deep unfolding framework to design custom neural network layers by unrolling two iterative optimization algorithms: (1) linearized alternating direction method of multipliers, which we apply to a constrained convex problem, and (2) vector approximate message passing featuring a novel denoiser based on the iterative shrinkage thresholding algorithm. The networks thus inherit domain knowledge as encapsulated by the signal model, and suitable operations as informed by the algorithms—in the same spirit as convolutional networks that exploit structure inherent in images and audio, except grounded in optimization and statistics. The networks, trained on synthetic data generated from the block-fading millimeter-wave multiple access channel model, offer improved complexity and accuracy relative to their iterative counterparts, and are potentially a boon to cell-free MIMO systems.

AID-DTI: Accelerating High-Fidelity Diffusion Tensor Imaging with Detail-Preserving Model-Based Deep Learning

We propose and experimentally demonstrate a model-based deep-learning framework that optimizes fiber-terahertz communication system end-to-end, while retaining the conventional single-car-rier transceiver architecture. Jointly learning of pulse shaping, pre-equalization, digital pre-distortion, and transceivers achieves a 1.8-dB sensitivity gain and 4-Gb/s capacity improvement at 209-GHz fiber-THz system.

Nowadays, Cardivascular Diseases (CVDs) are the most common cause of deaths worldwide. Some lesions connected to CVDs (such as Cardiomegaly, pleural effusion, pulmonary edema) can be detected on acquired Posterior Anterior (PA) chest X-ray projection images (one of the cheapest and most common modality in radiology). By developing robust methods for the segmentation of the heart as a computer aided diagnostic system (CADx) the automatic screening for such lesions becomes possible. In this paper we present a novel model-based deep learning approach by combining Active Shape Models (ASM), Deep Convolutional Neural Networks (CNN) and expert image processing knowledge through numerical optimization solved by Alternating Direction Method of Multipliers (ADMM) based optimization scheme. The proposed method is evaluated on various datasets against a purely neural network-based approach that is architecturally similar to the proposed solution. Results indicate that our model is more robust, provides a better generalization across datasets especially when segmentation is challenging due to lesions obscuring the target area. Comparison is done on the JSRT dataset as well as on a private dataset containing samples acquired in daily medical practice. Quantitatively, the proposed method achieved a Dices Score of <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$0.877\pm 0.05$</tex> and a Hausdorff distance of 17.35 mm <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\pm 4.52$</tex> mm,

The limited bandwidth of Wi-Fi severely confines the granularity (especially in differentiating multiple subjects) of Wi-Fi sensing, posing a significant challenge for its wide adoption. Though utilizing multiple channels to expand the effective bandwidth sounds plausible, continuous spectrum stitching towards ultra-wideband (UWB) is far from practical given various constraints (e.g., the runtime channel availability and inconsistent channel responses across a wide bandwidth). To this end, we propose UWB-Fi as a novel Wi-Fi sensing system with ultra-wide bandwidth, leveraging only discrete and irregular channel sampling. We first design a fast channel hopping scheme to perform arbitrary sampling across 4.7GHz (i.e., 2.4 to 7.1GHz) bandwidth on commodity Wi-Fi hardware without interrupting default communications. As no signal processing tool is available to handle such channel samples, we innovate in a model-based deep learning approach that translates discrete channel samples to high-dimensional spectral parameters; this method successfully avoids the bias-variance tradeoff in parameter estimation, while filtering out hardware-related offsets inherent to Wi-Fi. Through extensive evaluations, we demonstrate that UWB-Fi successfully achieves fine-granularity sensing, enabling centimeter-level resolution for indoor multi-person sensing.