Solving Fourier Phase Retrieval with a Reference Image as a Sequence of Linear Inverse Problems

An important aspect of 2D object recognition is matching. Based on a generalized distance transform method, Borgefors introduced the concept of the Chamfer scheme to 2D object matching. In Chamfer matching, a successful match is obtained when a scene in a reference image is correctly matched to the corresponding scene in the target image. The successful match corresponds to the lowest matching measure computed from the sum of Chamfer distances. However, if the target image has not been correctly corrected for distortions, there may not be an ideal best fit between the reference and target images. Instead, matching measures corresponding to local minima exist, for which it is difficult to choose the correct matching measure that corresponds to the best fit. The match fit corresponding to the global minimum measure may not necessarily be the best fit as there are many sub-optimal measures that could provide equally good fits between the target and reference images. This paper advocates that aside from the Chamfer matching measure, additional considerations need to be factored in to improve on finding the true global minimum measure which corresponds to the optimal fit. Int his paper, a new method has been designed to find the optimal fit between poorly-rectified target and reference images. The new approach uses simulated annealing to improve on the matching results. Simulated annealing is a powerful stochastic optimization technique for nonlinear problems. In our novel technique, the energy function for simulated annealing comprises two terms: a smoothness constraint term and the Chamfer matching measure term. The high computational burden of simulated annealing is reduced by using edge information for the matching process. Results are presented to illustrate the new matching technique.© (1996) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.

Magnetic resonance imaging plays a crucial role in clinical diagnosis due to its ability to provide information on soft tissue structure. At present, multi-contrast super-resolution (SR) methods for magnetic resonance (MR) images have been widely studied and have achieved good results. However, most studies overlook the impact of modal differences between reference and low-resolution (LR) image features on feature reconstruction, which may result in inaccurate reconstruction of detail features due to improper alignment of structural features. To address this issue, we propose a high-frequency feature transfer network for MR image SR task (HFTN-SR), which consists of two feature extraction branches and one high-frequency feature transfer branch. Considering the modal differences between the target and reference images, a high-low frequency decomposition method is used to decompose the reference image into high-frequency and low-frequency components, where the low-frequency components are used in the subsequent network to match LR target image features. A feature extraction block (FEB) is constructed to extract and integrate high-frequency and low-frequency features of the reference image, as well as features of the LR target image. In response to the modal differences between two image features, a feature transfer block (FTB) is designed to establish the correlation between the low-frequency features of the reference image and the target image features, and use the correlation matrix to transfer the high-frequency features of the reference image to the target image features. To further reduce the loss of shallow features caused by the increase in network depth, a standardized combined residual feature module (SCRFM) is constructed to supplement the shallow features of the target image into the final reconstructed features. Experiments on the public dataset FastMRI and the self-built dataset AXA show that the performance of HFTN-SR is superior to some state-of-the-art (SOTA) methods. Notably, HFTN-SR achieves the highest PSNR and SSIM scores across all tested datasets, with significant improvements in visual quality and detail reconstruction.

We present a novel approach to change detection between two brain MRI scans (reference and target.) The proposed method uses a single modality to find subtle changes; and does not require prior knowledge (learning) of the type of changes to be sought. The method is based on the computation of a local kernel from the reference image, which measures the likeness of a pixel to its surroundings. This kernel is then used as a feature and compared against analogous features from the target image. This comparison is made using cosine similarity. The overall algorithm yields a scalar dissimilarity map (DM), indicating the local statistical likelihood of dissimilarity between the reference and target images. DM values exceeding a threshold then identify meaningful and relevant changes. The proposed method is robust to various challenging conditions including unequal signal strength.

Two- and higher-dimensional bandlimited functions are almost always non-factorizable, meaning that their zeros form a single analytic curve. In principle one can use this property to separate the product of two bandlimited functions into its respective factors; this is important in Fourier phase retrieval and deconvolution problems. The intersection of this zero structure with the real plane is at points, closed curves or lines stretching to infinity. The location of zero points, curves or lines can only be estimated, in practice, from available noisy data. The estimated locations can be used to write a factorizable approximation to the original function and we explore the consequences of doing this. Of importance is the fact that point zeros can be used to represent a 2D bandlimited function. Hence from intensity data, point zero locations in the intensity can be used directly to estimate the complex spectrum and provide an approximate solution to the phase retrieval problem. Examples will be given and their importance for the general blind deconvolution problem discussed.

The reconstruction of a target image from its diffraction pattern is referred to as “diffractive imaging,” and this has been focused as a new microscopy. Fourier phase retrieval is required during the imaging process by means of numerical computations using computers. However, it is difficult to establish a complete convergence using an incomplete diffraction pattern such as experimental data in the retrieving process. A simple averaging procedure by weak phase-retrieved results using different initials is often used. Recently, the spherical shell structure of the retrieved images has been investigated and the center of the shell is to be a superior retrieved result. Therefore, it is important to construct such a structure by the retrieved images in order to reconstruct the target image. In this paper, a novel sequential on-line algorithm for effectively constructing the spherical shell structure of the images is introduced. A numerical demonstration is also presented using the Fourier intensities contaminated with Poisson noise.

The existing medical imaging technologies have little consideration on color information, thus most of medical images are gray. Classical hand-craft features-based methods have obtained unsatisfactory results in colorizing medical images. Moreover, these methods ignore the deep feature of medical images that represent pathology and color information. In this paper, we propose a novel method that iteratively colorizes grayscale medical images under preserving content in fine-tuned deep neural network. To the best of our knowledge, there is no currently work that attempts to colorize the medical image by using deep neural network. Specifically, we propose Y-loss which is defined as nonlinear combination of $\ell _{1}$ and $\ell _{2}$ norm to preserve content invariance between target and colorized medical image. Then, adaptive reference image search algorithm is introduced to code reference and target medical image with D-hash and search reference image in hash code automatically, which free the manual selection of the reference image. Extensive experiment results show that the proposed method can generate higher quality colored medical image than recent state-of-the-art methods, and can be approved by the doctor. The objective evaluation (PSNR and SSIM) outperform an average increment 24% and 47% than baseline method, respectively. Our code is available at: https://github.com/Tongshiyue/Adaptive-medical-image-deep-color-perception-algorithm .

The aim of this paper is to document the automatic reconstruction of clouds and their cast shadows for the generation of spontaneous cloud-free images from high-resolution multitemporal images. To apply the proposed technique, a cloud-free reference image, which has the same position as a target image acquired at a different time, is required. First, the cloud region in the target image is detected based on integration of thick and peripheral cloud candidate regions. Next, the detected cloud region is restored using the pixel values of the target image by considering their location relative to the reference images. Finally, the pixel values of the restored image are separately normalized to the values of the reference image to generate a natural-looking cloud-free image. Multitemporal KOMPSAT-2 high-resolution images are used to construct study sites for evaluation of the proposed method in diverse cloud-cover cases. The experimental results show that the proposed method can automatically generate cloud-free images from high-resolution multitemporal images with reasonable qualitative and quantitative performance.

We consider the problem of reconstructing two signals from the autocorrelation and cross-correlation measurements. This inverse problem is a fundamental one in signal processing, and arises in many applications, including phase retrieval and blind channel estimation. In a typical phase retrieval setup, only the autocorrelation measurements are obtainable. We show that, when the measurements are obtained using three simple “masks”, phase retrieval reduces to the aforementioned reconstruction problem. The classic solution to this problem is based on finding common factors between the $z$ -transforms of the autocorrelation and cross-correlation vectors. This solution has enjoyed limited practical success, mainly due to the fact that it is not sufficiently stable in the noisy setting. In this paper, inspired by the success of convex programming in provably and stably solving various quadratic constrained problems, we develop a semidefinite programming-based algorithm and provide theoretical guarantees. In particular, we show that almost all signals can be uniquely recovered by this algorithm (up to a global phase). Comparative numerical studies demonstrate that the proposed method significantly outperforms the classic method in the noisy setting.

Cloud-free remote sensing images are required for many applications, such as land cover classification, land surface temperature retrieval and agricultural-drought monitoring. Cloud cover in remote sensing images can be pervasive, dynamic and often unavoidable. Current techniques of cloud removal for the VNIR (visible and near-infrared) bands still encounters the problem of pixel values estimated for the cloudy area incomparable and inconsistent with the cloud-free region in the target image. In this paper, we proposed an efficient approach to remove thick clouds and their shadows in VNIR bands using multi-temporal images with good maintenance of DN (digital number) value consistency. We constructed the spectral similarity between the target image and reference one for DN value estimation of the cloudy pixels. The information reconstruction was done with 10 neighboring cloud-free pair-pixels with the highest similarity over a small window centering the cloudy pixel between target and reference images. Four Landsat5 TM images around Nanjing city of Jiangsu Province in Eastern China were used to validate the approach over four representative surface patterns (mountain, plain, water and city) for diverse sizes of cloud cover. Comparison with the conventional approaches indicates high accuracy of the approach in cloud removal for the VNIR bands. The approach was applied to the Landsat8 OLI (Operational Land Imager) image on 29 April 2016 in Nanjing area using two reference images. Very good consistency was achieved in the resulted images, which confirms that the proposed approach could be served as an alternative for cloud removal in the VNIR bands using multi-temporal images.

The Bayesian inference is widely used in many scientific and engineering problems, especially in the linear inverse problems in infinite-dimensional setting where the unknowns are functions. In such problems, choosing an appropriate prior distribution is an important task. Especially when the function to infer has much detail information, such as many sharp jumps, corners, and the discontinuous and nonsmooth oscillation, the so-called total variation-Gaussian (TG) prior is proposed in function space to address it. However, the TG prior is easy to lead the blocky (staircase) effect in numerical results. In this work, we present a fractional order-TG (FTG) hybrid prior to deal with such problems, where the fractional order total variation (FTV) term is used to capture the detail information of the unknowns and simultaneously uses the Gaussian measure to ensure that it results in a well-defined posterior measure. For the numerical implementations of linear inverse problems in function spaces, we also propose an efficient independence sampler based on a transport map, which uses a proposal distribution derived from a diagonal map, and the acceptance probability associated to the proposal is independent of discretization dimensionality. And in order to take full advantage of the transport map, the hierarchical Bayesian framework is applied to flexibly determine the regularization parameter. Finally we provide some numerical examples to demonstrate the performance of the FTG prior and the efficiency and robustness of the proposed independence sampler method.

Image registration is a vital problem in medical imaging. It has many potential applications in clinical diagnosis (e.g. diagnosis of cardiac, retinal, pelvic, renal, abdomen, liver and tissue disorders). In this paper a new technique has been proposed based on feature based image registration using SURF (Speeded up robust features) and Hessian matrix feature extraction techniques for detecting, extracting and matching corresponding points in the reference (fixed) and source image (deformed). The source image is aligned using SURF and Hessian feature extraction techniques to align with the reference image. The proposed technique is based on multimodality of images, feature based classification and non-rigid images as the images taken are from different sources like MRI, CT scan and X-ray. The proposed technique uses feature-based methods to find correspondence between image features such as points, lines and contours. The non-rigid images use fixed (source) image and also deformable (target) image of the subject and aligns target image according to source/reference image.

Fourier phase retrieval is a classical problem of restoring a signal only from the measured magnitude of its Fourier transform. Although Fienup-type algorithms, which use prior knowledge in both spatial and Fourier domains, have been widely used in practice, they can often stall in local minima. Convex relaxation methods such as PhaseLift and PhaseCut may offer performance guarantees, but these algorithms are usually computationally expensive for practical use. To address this problem, here we propose a novel unsupervised feed-forward neural network for Fourier phase retrieval which generates high quality reconstruction immediately. Unlike the existing deep learning approaches that use a neural network as a regularization term or an end-to-end blackbox model for supervised training, our algorithm is a feed-forward neural network implementation of physics-driven constraints in an unsupervised learning framework. Specifically, our network is composed of two generators: one for the phase estimation using PhaseCut loss, followed by another generator for image reconstruction, all of which are trained simultaneously without matched data. The link to the classical Fienup-type algorithms and the recent symmetry-breaking learning approach is also revealed. Extensive experiments demonstrate that the proposed method outperforms all existing approaches in Fourier phase retrieval problems.

Detection of unruptured aneurysms is a major task in magnetic resonance angiography (MRA). However, it is difficult for radiologists to detect small aneurysms on the maximum intensity projection (MIP) images because adjacent vessels may overlap with the aneurysms. Therefore, we proposed a method for making a new MIP image, the SelMIP image, with the interested vessels only, as opposed to all vessels, by manually selecting a cerebral artery from a list of cerebral arteries recognized automatically. By using our new SelMIP viewing technique, the selected vessel regions can also be observed from various directions and would further facilitate the radiologists in detecting small aneurysms. For automated classification of cerebral arteries, two 3D images, a target image and a reference image, are compared. Image registration is performed using the global matching and feature correspondence techniques. Segmentation of vessels in the target image is performed using the thresholding and region growing techniques. The segmented vessel regions were classified into eight cerebral arteries by calculating the Euclidean distance between a voxel in the target image and each of the voxels in the labeled eight vessel regions in the reference image. In applying the automated cerebral arteries recognization algorithm to thirteen MRA studies, results of 10 MRA studies were evaluated as clinically acceptable. Our new viewing technique would be useful in assisting radiologists for detection of aneurysms and for reducing the interpretation time.

As an expansion of coherent diffraction imaging, ptychographic iterative engine (PIE) not only inherits advantages such as ultra-high resolution and compact optical system, but also expands the field of view in quantitative imaging, thus PIE is widely used in short wavelength imaging such as X-ray and electron beam imaging, and then extended to visible light field. However, PIE requires coherent illumination for both phase and amplitude retrieval, while traditional X-ray or electron beam sources often cannot satisfy this strict coherent condition, which leads to poor-quality information retrieval with low signal-to-noise ratio. Though several proposed methods such as multiple wavelength and multi-mode algorithms can eliminate incoherency influence to some extent, various details such as quantitative spectrum of illuminating source should be obtained before information retrieval, which complicates computing procedures. In addition, it is hard to acquire the spectrum of the illuminating source in most cases. In order to acquire high-quality information based on PIE with partially coherent illumination, a newly designed enhanced phase retrieval method for weakly scattering samples in PIE with partially coherent illumination is presented in this paper, in which only the bright field of the diffraction patterns is used in the iterative procedures mimicking the coherent cases especially for weakly scattering samples without any prior illuminating details. The bright field area can be regarded as purely coherent diffraction patterns composed of a strengthened zeroth order beam and a weakened diffracted beam. While the dark-field area generated by interference of diffracted beams cannot satisfy the requirement for coherence, therefore, dark-field diffraction patterns should be excluded in sample information extraction and only the bright field is used for phase retrieval via iterative process. Compared with the proposed multiple wavelength and multi-mode algorithms, the proposed method can simplify sample reconstructing procedures due to needing no prior knowledge. Moreover, in order to enhance the information of weakly scattering samples in retrieval, high order iteration method is also introduced in the paper. Since the bright field can be regarded as purely coherent diffraction patterns composed of a strengthened zeroth order beam and a weakened diffracted beam. For weakly scattering sample, the weakened diffracted beam is much lower than zeroth order beam, thus it is difficult to acquire high-contrast information with classical PIE algorithms. Introducing high order iterative tactic, the contrast of weakly scattering sample is obviously improved and the details of weakly scattering sample can be retrieved clearly. Both theoretical analysis and numerical simulations are illustrated in detail, proving the robustness and availability of the designed method: high-contrast phase information can be obtained with the proposed method, while traditional phase retrieval algorithm almost loses all details of the sample. In order to mimic the real experimental situation, a 30 dB white noise is added into the simulation, the details of weakly scattering sample phase information can also be retrieved clearly by using the bright field of the diffraction patterns with high order iteration method. With the newly designed enhanced phase retrieval method for weakly scattering samples with partially coherent illumination, sample retrieval via PIE can not only use ordinary X-ray source or electron beam as illumination source, thereby avoiding the dependence on complete coherent source, but also obviously improve the retrieval quality of the sample characteristics, which widely expands the application fields of the PIE.

Motivated by the recent success of supervised and weakly supervised common object discovery, in this paper, we move forward one step further to tackle common object discovery in a fully unsupervised way. Generally, object co-localization aims at simultaneously localizing objects of the same class across a group of images. Traditional object localization/detection usually trains specific object detectors which require bounding box annotations of object instances, or at least image-level labels to indicate the presence/absence of objects in an image. Given a collection of images without any annotations, our proposed fully unsupervised method is to simultaneously discover images that contain common objects and also localize common objects in corresponding images. Without requiring to know the total number of common objects, we formulate this unsupervised object discovery as a sub-graph mining problem from a weighted graph of object proposals, where nodes correspond to object proposals, and edges represent the similarities between neighbouring proposals. The positive images and common objects are jointly discovered by finding sub-graphs of strongly connected nodes, with each sub-graph capturing one object pattern. The optimization problem can be efficiently solved by our proposed maximal-flow-based algorithm. Instead of assuming that each image contains only one common object, our proposed solution can better address wild images where each image may contain multiple common objects or even no common object. Moreover, our proposed method can be easily tailored to the task of image retrieval in which the nodes correspond to the similarity between query and reference images. Extensive experiments on PASCAL VOC 2007 and Object Discovery data sets demonstrate that even without any supervision, our approach can discover/localize common objects of various classes in the presence of scale, view point, appearance variation, and partial occlusions. We also conduct broad experiments on image retrieval benchmarks, Holidays and Oxford5k data sets, to show that our proposed method, which considers both the similarity between query and reference images and also similarities among reference images, can help to improve the retrieval results significantly.