An Algorithm Architecture for Radio Interferometric Data Processing

For the last 20 years, a huge observational effort has been devoted to the identification of all 3CR radio sources lying in directions away from the Galactic plane. The project is now more-or-less complete thanks to high-resolution radio observations with aperture synthesis radio telescopes, in particular the Cambridge 5-km telescope, and the use of a CCD camera on the Palomar 5-metre telescope (see e.g. Gunn et al 1980).

Fundamental scientific questions such as how the first stars were formed or how the universe came into existence and evolved to its present state drive us to observe weak radio signals impinging on the earth from the early days of the universe. During the last century, radio astronomy has been vastly advancing. Important discoveries on the formation of various celestial objects such as pulsars, neutron stars, black holes, radio galaxies and quasars are the result of radio astronomical observations. To study celestial objects and the astrophysical processes that are responsible for their radio emissions, images must be formed. This is done with the help of large radio telescope arrays. Next generation radio telescopes such as the Low Frequency Array Radio Telescope (LOFAR) [1] and the Square Kilometer Array (SKA) [2], bring about increasingly more observational evidence for the study of the radio sky by generating very high resolution and high fidelity images. In this dissertation, we study radio astronomical imaging as the problem of estimating the sky spatial intensity distribution over the field of view of the radio telescope array from the incomplete and noisy array data. The increased sensitivity, resolution and sky coverage of the new instruments pose additional challenges to the current radio astronomical imaging pipeline. Namely, the large amount of data captured by the radio telescopes cannot be stored and needs to be processed quasi-real time. Many pixel-based imaging algorithms, such as the widely-used CLEAN [3] algorithm, are not scalable to the size of the required images and perform very slow in high resolution scenarios. Therefore, there is an urgent need for new efficient imaging algorithms. Moreover, regardless of the amount of collected data, there is an inherent loss of information in the measurement process due to physical limitations. Therefore, to recover physically meaningful images additional information in the form of constraints and regularizing assumptions are necessary. The central objective of the current dissertation is to introduce advanced algebraic techniques together with custom-made regularization schemes to speed up the image formation pipeline of the next generation radio telescopes.

Legal Electronic Discovery (eDiscovery) is a business domain that utilizes large volumes of data, in a variety of structured and unstructured formats to discover evidence that may be pertinent to legal proceedings, compliance needs, litigation or other investigations. The eDiscovery practitioners are typically required to produce results with short turnaround times, utilizing mostly commodity solutions that are available at their disposal, while still conforming to legally defensible standards of data management. Therefore, they use optimal strategies for data analysis and management to meet the time and quality requirements of their business. In addition to using such optimal strategies, the time-to-results during eDiscovery can be further reduced by taking advantage of the High-Throughput Computing (HTC) paradigm on High Performance Computing (HPC) platforms.

Clusters of computers have emerged as mainstream parallel and distributed platforms for high‐performance, high‐throughput and high‐availability computing. To enable effective resource management on clusters, numerous cluster management systems and schedulers have been designed. However, their focus has essentially been on maximizing CPU performance, but not on improving the value of utility delivered to the user and quality of services. This paper presents a new computational economy driven scheduling system called Libra, which has been designed to support allocation of resources based on the users' quality of service requirements. It is intended to work as an add‐on to the existing queuing and resource management system. The first version has been implemented as a plugin scheduler to the Portable Batch System. The scheduler offers market‐based economy driven service for managing batch jobs on clusters by scheduling CPU time according to user‐perceived value (utility), determined by their budget and deadline rather than system performance considerations. The Libra scheduler has been simulated using the GridSim toolkit to carry out a detailed performance analysis. Results show that the deadline and budget based proportional resource allocation strategy improves the utility of the system and user satisfaction as compared with system‐centric scheduling strategies. Copyright © 2004 John Wiley & Sons, Ltd.

A real-time recursive back projection (BP) imaging algorithm is presented in this paper. Real time imaging is an intense demand but a challenging task in impulse surface penetrating radar (ImpSPR)'s application. Based on 'delay-sum' operation in time domain, BP imaging algorithm can precisely focus the scattering intensity and obtain high quality subsurface profile. But it's heavy computation burden restricts it's application in ImpSPR's real-time processing. By minutely analyzing it's procedure, a recursive model of BP imaging algorithm is established and real-time BP imaging algorithm is educed subsequently. The computation complexity of both non real time BP imaging algorithm and real time BP imaging algorithm are analyzed. Through processing the experimental data obtained by a ImpSPR system- RadarEye, the imaging algorithm validates its capability at the aspect of ImpSPR's real time processing.

Improved Rapid Corner Detection Algorithm in Medical Image

Context.Reconstructing an image from noisy, sparsely sampled Fourier data is an ill-posed inverse problem that occurs in a variety of subjects within science, including data analysis for Very Long Baseline Interferometry (VLBI) and the Spectrometer/Telescope for Imaging X-rays (STIX) with respect to solar observations. The need for high-resolution, high-fidelity imaging fosters the active development of a range of novel imaging algorithms in a variety of different algorithmic settings. However, despite these ongoing, parallel developments, such synergies remain unexplored.Aims.We study, for the first time, the synergies between the data analysis for the STIX instrument and VLBI. In particular, we compare the methodologies that have been developed in both fields and evaluate their potential. In this way, we identify key trends in the performance of several algorithmic ideas and draw recommendations for the future spending of resources in the study and implementation of novel imaging algorithms.Methods.To this end, we organized a semi-blind imaging challenge with data sets and source structures that are typical for sparse VLBI, specifically in the context of the Event Horizon Telescope (EHT) as well as STIX observations. We used 17 different algorithms from both communities, from six different imaging frameworks, in the challenge, making this work the largest scale code comparison for STIX and VLBI to date.Results.We identified strong synergies between the two communities, as proven by the success of the imaging methods proposed for STIX in imaging VLBI data sets and vice versa. Novel imaging methods outperform the standard CLEAN algorithm significantly in every test case. Improvements over the performance of CLEAN offer deeper updates to the inverse modeling pipeline necessary or, consequently, the possibility to replace inverse modeling with forward modeling. Entropy-based methods and Bayesian methods perform best on STIX data. The more complex imaging algorithms utilizing multiple regularization terms (recently proposed for VLBI) add little to no additional improvements for STIX. However, they do outperform the other methods on EHT data, which correspond to a larger number of angular scales.Conclusions.This work demonstrates the great synergy between the STIX and VLBI imaging efforts and the great potential for common developments. The comparison identifies key trends on the efficacy of specific algorithmic ideas for the VLBI and the STIX setting that may evolve into a roadmap for future developments.

In this paper we introduce a new algorithm for image and video texture synthesis. In our approach, patch regions from a sample image or video are transformed and copied to the output and then stitched together along optimal seams to generate a new (and typically larger) output. In contrast to other techniques, the size of the patch is not chosen a-priori , but instead a graph cut technique is used to determine the optimal patch region for any given offset between the input and output texture. Unlike dynamic programming, our graph cut technique for seam optimization is applicable in any dimension. We specifically explore it in 2D and 3D to perform video texture synthesis in addition to regular image synthesis. We present approximative offset search techniques that work well in conjunction with the presented patch size optimization. We show results for synthesizing regular, random, and natural images and videos. We also demonstrate how this method can be used to interactively merge different images to generate new scenes.

Interactive Parallelization Tool (IPT) is a semi-automatic tool that can be used by domain experts and students for transforming certain classes of existing applications into multiple parallel variants. An end-user of IPT provides existing application and high-level specifications for parallelization as input. On the basis of the specifications provided by the end-user, IPT carries out the code changes in the given existing application to generate parallel variants that can be run on different High Performance Computing (HPC) platforms. The parallel programming paradigms that are currently supported by IPT are MPI, OpenMP, and CUDA. The supported base languages are C and C++. Though IPT is still under active development, it has been recently made available on a web-enabled platform, named DiaGrid Hub, with the support from the XSEDE Extended Collaborative Support Service (ECSS). While the main goal of IPT is to make parallel programming easy for its end-users, the main goal of DiaGrid Hub is to enable the research community with instant access to HPC and High Throughput Computing platforms through a user-friendly web-interface. By deploying IPT on DiaGrid Hub, our goal is to enable the end-users to generate parallel versions of their existing applications without having to install IPT locally. They can also immediately compile and run the generated applications on Purdue and XSEDE resources that are available through DiaGrid Hub. Hence, the collaborative project that is reported in this paper lowers the entry-barriers to parallel programming and the usage of the national CyberInfrastructure (CI). In this paper, we present our ongoing work on deploying IPT over DiaGrid and testing the usability of IPT through a web-interface.

Autonomous multisensor tracking surveillance system

SMOS is the acronym for the Soil Moisture and Ocean Salinity mission by the European Space Agency (ESA) [1]. Its single payload, the Microwave Imaging Radiometer using Aperture Synthesis (MIRAS), was successfully launched in November 2009 to provide a continuous flow of fully polarimetric brightness temperature images. After more than five years of operation, SMOS has proven to be highly useful for a variety of scientific applications related to soil moisture over land, ocean salinity and winds over ocean, as well as specific studies over the ice covered surfaces [2][3]. Along these years of operation, SMOS calibration and imaging algorithms have undergone a continuous evolution to further improve the accuracy of the retrieved geophysical parameters. These improvements are particularly useful in Ocean Salinity applications where accuracy and stability requirements are very demanding and spatial and temporal averaging is required to achieve SMOS mission objectives. This paper analyzes the impact on ocean images of the so-called “floor error”, an image reconstruction artifact that, currently, is significant on the 3rd and 4th Stokes parameters. Error mitigation techniques are evaluated by computing spatial bias over the ocean and also by providing a 4th Stokes global error map over the ocean that takes into account SMOS full spatial and temporal averaging capabilities.

The growing computational power requirements of grand challenge applications has promoted the need for merging high throughput computing and grid computing principles to harness computational resources distributed across multiple organisations. This paper identifies the issues in resource management and scheduling in the emerging high throughput grid computing context. We also survey and study the performance of several space-sharing and time-sharing opportunistic scheduling policies that have been developed for high throughput computing.

This paper describes the design of a novel test-set for single-ridged waveguides in the 28.5 GHz band. The test-set composes of three main parts: a waveguide transformer with a 90° bend, a part of an antenna array (slots) represented by a ridged waveguide transmission line and a load. The transformer provides a single-ridged-waveguide-to-standard-WR-28 transition and vice versa. This transition forms the interface to the measurement equipment and parts of its design will also be applied to the original design of the antenna. The load can be tested by connecting it to the single-ridged output port of the adapter or to the waveguide transmission line. This saves precious development time and costs, and enables also an accurate verification of the adapter design. The waveguide transmission line can be used to test different slot configurations. Some simulations have been performed to investigate the tolerance values required for manufacturing the test-set.

This paper presents a study of the par alle lization possibilities of a Non-Linear Iterative Partial Least Squares algorithm and its adaptation to a Massively Parallel Processor Array manycore architecture, which assembles 256 cores distributed over 16 clusters. The aim of this work is twofold: first, to test the behavior of iterative, complex algorithms in a manycore architecture; and, secondly, to achieve real-time processing of hyperspectral images, which is fixed by the image capture rate of the hyperspectral sensor. Real-time is a challenging objective, as hyperspectral images are composed of extensive volumes of spectral information. This issue is usually addressed by reducing the image size prior to the processing phase itself. Consequently, this paper proposes an analysis of the intrinsic parallelism of the algorithm and its subsequent implementation on a manycore architecture. As a result, an average speedup of 13 has been achieved when compared to the sequential version. Additionally, this implementation has been compared with other state-of-the-art applications, outperforming them in terms of performance.

Presents a classification overview and a comparative study of despeckling algorithms. Almost all of the existing good speckle reduction algorithms show high computational complexity. Promising results giving a very good compromise between image enhancement and algorithm complexity are obtained using multiscale analysis techniques.