HFTN-SR: High-Frequency Feature Transfer Network for MR Image Super-Resolution

An important aspect of 2D object recognition is matching. Based on a generalized distance transform method, Borgefors introduced the concept of the Chamfer scheme to 2D object matching. In Chamfer matching, a successful match is obtained when a scene in a reference image is correctly matched to the corresponding scene in the target image. The successful match corresponds to the lowest matching measure computed from the sum of Chamfer distances. However, if the target image has not been correctly corrected for distortions, there may not be an ideal best fit between the reference and target images. Instead, matching measures corresponding to local minima exist, for which it is difficult to choose the correct matching measure that corresponds to the best fit. The match fit corresponding to the global minimum measure may not necessarily be the best fit as there are many sub-optimal measures that could provide equally good fits between the target and reference images. This paper advocates that aside from the Chamfer matching measure, additional considerations need to be factored in to improve on finding the true global minimum measure which corresponds to the optimal fit. Int his paper, a new method has been designed to find the optimal fit between poorly-rectified target and reference images. The new approach uses simulated annealing to improve on the matching results. Simulated annealing is a powerful stochastic optimization technique for nonlinear problems. In our novel technique, the energy function for simulated annealing comprises two terms: a smoothness constraint term and the Chamfer matching measure term. The high computational burden of simulated annealing is reduced by using edge information for the matching process. Results are presented to illustrate the new matching technique.© (1996) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.

Image registration is a vital problem in medical imaging. It has many potential applications in clinical diagnosis (e.g. diagnosis of cardiac, retinal, pelvic, renal, abdomen, liver and tissue disorders). In this paper a new technique has been proposed based on feature based image registration using SURF (Speeded up robust features) and Hessian matrix feature extraction techniques for detecting, extracting and matching corresponding points in the reference (fixed) and source image (deformed). The source image is aligned using SURF and Hessian feature extraction techniques to align with the reference image. The proposed technique is based on multimodality of images, feature based classification and non-rigid images as the images taken are from different sources like MRI, CT scan and X-ray. The proposed technique uses feature-based methods to find correspondence between image features such as points, lines and contours. The non-rigid images use fixed (source) image and also deformable (target) image of the subject and aligns target image according to source/reference image.

Cloud-free remote sensing images are required for many applications, such as land cover classification, land surface temperature retrieval and agricultural-drought monitoring. Cloud cover in remote sensing images can be pervasive, dynamic and often unavoidable. Current techniques of cloud removal for the VNIR (visible and near-infrared) bands still encounters the problem of pixel values estimated for the cloudy area incomparable and inconsistent with the cloud-free region in the target image. In this paper, we proposed an efficient approach to remove thick clouds and their shadows in VNIR bands using multi-temporal images with good maintenance of DN (digital number) value consistency. We constructed the spectral similarity between the target image and reference one for DN value estimation of the cloudy pixels. The information reconstruction was done with 10 neighboring cloud-free pair-pixels with the highest similarity over a small window centering the cloudy pixel between target and reference images. Four Landsat5 TM images around Nanjing city of Jiangsu Province in Eastern China were used to validate the approach over four representative surface patterns (mountain, plain, water and city) for diverse sizes of cloud cover. Comparison with the conventional approaches indicates high accuracy of the approach in cloud removal for the VNIR bands. The approach was applied to the Landsat8 OLI (Operational Land Imager) image on 29 April 2016 in Nanjing area using two reference images. Very good consistency was achieved in the resulted images, which confirms that the proposed approach could be served as an alternative for cloud removal in the VNIR bands using multi-temporal images.

Fourier phase retrieval problem is equivalent to the recovery of a two-dimensional image from its autocorrelation measurements. This problem is generally nonlinear and nonconvex. Good initialization and prior information about the support or sparsity of the target image are often critical for a robust recovery. In this paper, we show that the presence of a known reference image can help us solve the nonlinear phase retrieval problem as a sequence of small linear inverse problems. Instead of recovering the entire image at once, our sequential method recovers a small number of rows or columns by solving a linear deconvolution problem at every step. Existing methods for the reference-based (holographic) phase retrieval either assume that the reference and target images are sufficiently separated so that the recovery problem is linear or recover the image via nonlinear optimization. In contrast, our proposed method does not require the separation condition. We performed an extensive set of simulations to demonstrate that our proposed method can successfully recover images from autocorrelation data under different settings of reference placement and noise.

We present a novel approach to change detection between two brain MRI scans (reference and target.) The proposed method uses a single modality to find subtle changes; and does not require prior knowledge (learning) of the type of changes to be sought. The method is based on the computation of a local kernel from the reference image, which measures the likeness of a pixel to its surroundings. This kernel is then used as a feature and compared against analogous features from the target image. This comparison is made using cosine similarity. The overall algorithm yields a scalar dissimilarity map (DM), indicating the local statistical likelihood of dissimilarity between the reference and target images. DM values exceeding a threshold then identify meaningful and relevant changes. The proposed method is robust to various challenging conditions including unequal signal strength.

Detection of unruptured aneurysms is a major task in magnetic resonance angiography (MRA). However, it is difficult for radiologists to detect small aneurysms on the maximum intensity projection (MIP) images because adjacent vessels may overlap with the aneurysms. Therefore, we proposed a method for making a new MIP image, the SelMIP image, with the interested vessels only, as opposed to all vessels, by manually selecting a cerebral artery from a list of cerebral arteries recognized automatically. By using our new SelMIP viewing technique, the selected vessel regions can also be observed from various directions and would further facilitate the radiologists in detecting small aneurysms. For automated classification of cerebral arteries, two 3D images, a target image and a reference image, are compared. Image registration is performed using the global matching and feature correspondence techniques. Segmentation of vessels in the target image is performed using the thresholding and region growing techniques. The segmented vessel regions were classified into eight cerebral arteries by calculating the Euclidean distance between a voxel in the target image and each of the voxels in the labeled eight vessel regions in the reference image. In applying the automated cerebral arteries recognization algorithm to thirteen MRA studies, results of 10 MRA studies were evaluated as clinically acceptable. Our new viewing technique would be useful in assisting radiologists for detection of aneurysms and for reducing the interpretation time.

The existing medical imaging technologies have little consideration on color information, thus most of medical images are gray. Classical hand-craft features-based methods have obtained unsatisfactory results in colorizing medical images. Moreover, these methods ignore the deep feature of medical images that represent pathology and color information. In this paper, we propose a novel method that iteratively colorizes grayscale medical images under preserving content in fine-tuned deep neural network. To the best of our knowledge, there is no currently work that attempts to colorize the medical image by using deep neural network. Specifically, we propose Y-loss which is defined as nonlinear combination of $\ell _{1}$ and $\ell _{2}$ norm to preserve content invariance between target and colorized medical image. Then, adaptive reference image search algorithm is introduced to code reference and target medical image with D-hash and search reference image in hash code automatically, which free the manual selection of the reference image. Extensive experiment results show that the proposed method can generate higher quality colored medical image than recent state-of-the-art methods, and can be approved by the doctor. The objective evaluation (PSNR and SSIM) outperform an average increment 24% and 47% than baseline method, respectively. Our code is available at: https://github.com/Tongshiyue/Adaptive-medical-image-deep-color-perception-algorithm .

We predicted molecular profiles in newly diagnosed glioblastoma patients using magnetic resonance (MR) imaging features and explored the associations between imaging features and major molecular alterations. This retrospective study included patients with newly diagnosed glioblastoma and available next-generation sequencing results. From preoperative MR imaging, Visually AcceSAble Rembrandt Images (VASARI) features, volumetric parameters, and apparent diffusion coefficient (ADC) values were obtained. First, univariate random forest was performed to identify gene abnormalities that could be predicted by imaging features with high accuracy and stability. Next, multivariate random forest was trained to predict the selected genes in the discovery cohort and was validated in the external cohort. Univariable logistic regression was performed to further explore the associations between imaging features and genes. Univariate random forest identified nine genes predicted by imaging features, with high accuracy and stability. The multivariate random forest model showed excellent performance in predicting IDH and PTPN11 mutations in the discovery cohort, which were validated in the external validation cohorts (areas under the receiver operator characteristiccurve [AUCs] of 0.855 for IDH and 0.88 for PTPN11). ATRX loss and EGFR mutation were predicted with AUCs of 0.753 and 0.739, respectively, whereas PTEN could not be reliably predicted. Based on univariable logistic regression analyses, IDH, ATRX, and TP53 were clustered according to their shared imaging features, whereas EGFR and CDKN2A/B were clustered in the opposite direction. MR imaging features are related to specific molecular alterations and can be used to predict molecular profiles in patients with newly diagnosed glioblastoma.

The conventional background-oriented schlieren (BOS) technique is an image-based technique that can calculate the density field in fluids using two static images [i.e., an undistorted background image (reference image) and a distorted background image due to the density change in fluids (target image)]. This paper proposes the smartphone BOS (SBOS) technique, which offers the measurement of the density gradient using the high-speed imaging feature of the smartphone being carried with a moving observer. The conventional BOS with a fixed camera visualizes the density gradient by comparing the reference image and the target image. In contrast, SBOS can obtain the time difference of the density gradient field. A reference image in SBOS is a target one at a previous time step. The movement of the smartphone is canceled using a registration technique for image accurate alignment. Three demonstrations are conducted to perform SBOS. First, in a static situation, the density field of heated air by a gas burner is visualized by comparing between SBOS and conventional BOS. Second, the local displacement of density field and the error displacement is estimated quantitatively when the smartphone is moving. Third, SBOS using an embossed wallpaper to visualize the density field is performed in the mobile condition. These achievements suggest that SBOS is an effective system to visualize the density field using only the smartphone, and is expected to be a useful tool such as a preliminary experiment in the laboratory and a teaching tool for general smartphone users.

The aim of this paper is to document the automatic reconstruction of clouds and their cast shadows for the generation of spontaneous cloud-free images from high-resolution multitemporal images. To apply the proposed technique, a cloud-free reference image, which has the same position as a target image acquired at a different time, is required. First, the cloud region in the target image is detected based on integration of thick and peripheral cloud candidate regions. Next, the detected cloud region is restored using the pixel values of the target image by considering their location relative to the reference images. Finally, the pixel values of the restored image are separately normalized to the values of the reference image to generate a natural-looking cloud-free image. Multitemporal KOMPSAT-2 high-resolution images are used to construct study sites for evaluation of the proposed method in diverse cloud-cover cases. The experimental results show that the proposed method can automatically generate cloud-free images from high-resolution multitemporal images with reasonable qualitative and quantitative performance.

Collaborative group: Composed image retrieval via consensus learning from noisy annotations

Single image super-resolution (SISR) with deep convolutional neural networks has recently attracted increasing attention due to its potentials to generate rich details. To obtain better fidelity and visual quality, most of existing methods are of heavy design with the depth of network. However, the lack of high-frequency information in the deep network and the limit of non-local context relations of pixels remain the core issue. In addition, there are many arbitrary-scale image super-resolution tasks, so it is a focus that provides an effective and efficient model for super resolution with arbitrary-scale. To this end, we propose a Feature Preserving and Enhancing Network (FPEN) based on implicit representation, which is aims to preserving high-frequency information in deep network and enhancing non-local contextual features of pixels to output more realistic images at arbitrary scales. In particular, our proposed High Frequency Preserving Block (HFPB) can divide the features into high-frequency components and low-frequency components, and allocate more operations to high-frequency components to ensure that high-frequency components can be preserved in the network. Since high-frequency information contains the details and texture of the image, it can restore the finer details of the image. Moreover, Pixel Continuity Attention (PiCA) module our proposed utilizes visual cues observed from pixels to adaptively recalibrate the pixel range that needs attention in the image to better achieve feature enhancement and generate smoother images. Extensive experiments conducted on benchmark SISR models and datasets show that Feature Preserving and Enhancing Network can be employed for various SISR tasks with arbitrary-scale to obtain the better visual quality than other state-of-the-art SR algorithms.

Compressive sensing magnetic resonance (MR) imaging is aimed at achieving high-quality MR image reconstruction by undersampling K-space data. It is crucial to explore prior information since compressive sensing MR imaging relies heavily on some prior assumptions, such as signal's sparse property. In this study, in order to explore the prior information fully, an improved MR image reconstruction model based on compressive sensing theory is proposed, named reference image MR imaging with adaptive tight frame. In the proposed model, an adaptive tight frame is involved to explore the sparse prior information adapt to MR images and the similarity prior information to the target image. Meanwhile, improved adaptive weighting parameters are used to trade off the sparsity between the regions with much similarity and that of little similarity. In addition, the smoothing-based fast iterative shrinkage-threshold algorithm is utilised to tackle the optimisation problem so as to speed up imaging. The experimental results demonstrate that the proposed MR image reconstruction method outperforms some state-of-the-art methods in terms of quantitative results.

In image-based visual hull (IBVH) rendering, highlight spots in reference images often cause undesired artifacts in the rendering results. In this paper, we propose a method that can automatically recognize and remove highlight spots from reference images, by utilizing the features of IBVH. First, highlight sub-images are extracted by histogram analyzing; then, their counterparts in other images are calculated. The highlight pixels can be retrieved from their corresponding pixels by pixel blending, and finally be seamlessly integrated back into the target image by resolving a Poisson equation.

Home automation is being implemented into more and more domiciles of the elderly and disabled in order to maintain their independence and safety. For that purpose, we propose and validate a surveillance video system, which detects various postures-based events. One of the novel points of this system is to use adapted Vander-Lugt correlator (VLC). Our system is focused on the goals of accuracy, convenience, and cost, which in addition does not require any devices attached to the subject. Our algorithm is doing faster face recognition using phase information of the image. After then, this algorithm will compare the target image with the reference image and then show us the comparison results, whether the target image is in the reference images or not.