Experiences of commercial supercomputing in radio astronomy data processing

The Virtual Observatory (VO) is becoming the de-facto standard for astronomical data publication. However, the number of radio astronomical archives is still low in general, and even lower is the number of radio astronomical data available through the VO. In order to facilitate the building of new radio astronomical archives, easing at the same time their interoperability with VO framework, we have developed a VO-compliant data model which provides interoperable data semantics for radio data. That model, which we call the Radio Astronomical DAta Model for Single-dish (RADAMS) has been built using standards of (and recommendations from) the International Virtual Observatory Alliance (IVOA). This article describes the RADAMS and its components, including archived entities and their relationships to VO metadata. We show that by using IVOA principles and concepts, the effort needed for both the development of the archives and their VO compatibility has been lowered, and the joint development of two radio astronomical archives have been possible. We plan to adapt RADAMS to be able to deal with interferometry data in the future.

Among the techniques for atmospheric sounding, radio occultation enables an in depth investigation of vertical profiles from the ionosphere to the troposphere by measuring the radio frequency signal associated to the propagation medium. A precise characterization of the atmospheric layers requires a thorough processing of the raw radio tracking data to estimate the thermodynamic properties of the atmosphere and their related uncertainties. In this work, we present a method to retrieve refractivity, density, pressure, and temperature profiles with the associated uncertainties by analyzing a set of raw radio tracking data occulted by the atmosphere. This technique is also well suited to process two‐way Doppler measurements that are not acquired during dedicated occultation campaigns. The NASA mission Mars Reconnaissance Orbiter (MRO) provided a significant amount of radio occultation data that were not planned for atmospheric sounding, but were caused by the spacecraft orbit geometry. Our analysis of one of these occultation profiles with the proposed method allows indicating that MRO occultation datasets provide crucial information regarding Mars’ troposphere that can be used as input of general circulation models.

Solar radio bursts of spectral type II provide a prime diagnostic for the passage of shock waves, generated by solar flares, through the solar corona. In this investigation we have compared radio data on the shocks with computer simulations for the propagation of fast-mode MHD shocks through the solar corona. The radio data were recorded at the Harvard Radio Astronomy Station, Fort Davis, Texas. The computer simulations were carried out at NOAA, Boulder, Colorado.

view Abstract Citations (27) References (18) Co-Reads Similar Papers Volume Content Graphics Metrics Export Citation NASA/ADS Radio observations of IRAS extreme infrared galaxies. Antonucci, R. R. J. ; Olszewski, E. W. Abstract The authors present 6 cm VLA D-array observations of five IRAS "blank field" sources. Unresolved radio sources have been detected coincident with galaxies found in the fields by Aaronson and Olszewski (1984). Of the three candidate galaxy identifications for 0413+122 by Aaronson and Olszewski, only one is a radio source, and the authors suggest that it is the correct identification for the IRAS source. Also, based on their radio data and an improved IRAS position, the authors have determined the true identification of the infrared source 0404+101. For these galaxies, the correct νFν(80μ)/νFν(0.45μ) ratio ranges up to 250 with the far-infrared radiation presumably being thermal emission from dust grains. The primary energy sources could hypothetically be intense starbursts. Publication: The Astronomical Journal Pub Date: November 1985 DOI: 10.1086/113922 Bibcode: 1985AJ.....90.2203A Keywords: Infrared Astronomy Satellite; Radio Astronomy; Radio Galaxies; Astronomical Spectroscopy; Centimeter Waves; Cosmic Dust; Far Infrared Radiation; Stellar Evolution; Stellar Mass Ejection; Visible Spectrum; Astrophysics full text sources ADS | data products SIMBAD (17) NED (14)

In this paper, we review the file formats for data storage in radio astronomy. As a standard format used in astronomy, Flexible Image Transport System (FITS) is discussed firstly. Random Group FITS File (UVFITS) and FITS Interferometry Data Interchange (FITS-IDI) based on FITS are highlighted. And VOTable promoted by the development of Virtual Observatory (VO) is also briefly reviewed. We all know that the Common Astronomy Software Applications package (CASA) is a science data analysis package for the Atacama Large Millimeter/sub millimeter Array (ALMA). As the fundamental format of CASA, Measurement Set (MS) is emphatically reviewed. Also, a new Science Data Model (SDM) designed to be used in ALMA telescope during the observing process and offline in the ALMA science data archive is expected to play an important role in the format of radio astronomy data storage. In the end, we present a study about the imports and exports of the formats to show the advantage and disadvantage on storage space of each format. Meanwhile, a conclusion has been made.

Data collection for scientific applications is increasing exponentially and is forecasted to soon reach peta- and exabyte scales. Applications which process and analyze scientific data must be scalable and focus on execution performance to keep pace. In the field of radio astronomy, in addition to increasingly large datasets, tasks such as the identification of transient radio signals from extrasolar sources are computationally expensive. We present a scalable approach to radio pulsar detection written in Scala that parallelizes candidate identification to take advantage of in-memory task processing using Apache Spark on a YARN distributed system. Furthermore, we introduce a novel automated multiclass supervised machine learning technique that we combine with feature selection to reduce the time required for candidate classification. Experimental testing on a Beowulf cluster with 15 data nodes shows that the parallel implementation of the identification algorithm offers a speedup of up to 5X that of a similar multithreaded implementation. Further, we show that the combination of automated multiclass classification and feature selection speeds up the execution performance of the RandomForest machine learning algorithm by an average of 54% with less than a 2% average reduction in the algorithm's ability to correctly classify pulsars. The generalizability of these results is demonstrated by using two real-world radio astronomy data sets.

With the development and progress of astronomical observation technology and equipment, astronomy research has entered the era of big data. It's a huge challenge how massive astronomical data is stored, processed, accessed, and shared. It becomes more and more difficult how to access and utilize all astronomical data because of astronomical data with unique format, variety of structures, wide variety of species and global astronomical research trend. To solve this series of problems, this paper proposes to design a distributed prototype system-astronomical virtual telescope for astronomical information management and retrieval to achieve the storage, processing, and access of astronomical data and provide support for the research and application of astronomy.

We present the full panchromatic afterglow light-curve data of GW170817, including new radio data as well as archival optical and X-ray data, between 0.5 and 940 days post-merger. By compiling all archival data and reprocessing a subset of it, we have evaluated the impact of differences in data processing or flux determination methods used by different groups and attempted to mitigate these differences to provide a more uniform data set. Simple power-law fits to the uniform afterglow light curve indicate a t 0.86±0.04 rise, a t −1.92±0.12 decline, and a peak occurring at 155 ± 4 days. The afterglow is optically thin throughout its evolution, consistent with a single spectral index (−0.584 ± 0.002) across all epochs. This gives a precise and updated estimate of the electron power-law index, p = 2.168 ± 0.004. By studying the diffuse X-ray emission from the host galaxy, we place a conservative upper limit on the hot ionized interstellar medium density, <0.01 cm−3, consistent with previous afterglow studies. Using the late-time afterglow data we rule out any long-lived neutron star remnant having a magnetic field strength between 1010.4 and 1016 G. Our fits to the afterglow data using an analytical model that includes Very Long Baseline Interferometry proper motion from Mooley et al., and a structured jet model that ignores the proper motion, indicates that the proper-motion measurement needs to be considered when seeking an accurate estimate of the viewing angle.

ABSTRACTFlagging of Radio Frequency Interference (RFI) in time–frequency visibility data is an increasingly important challenge in radio astronomy. We present R-Net, a deep convolutional ResNet architecture that significantly outperforms existing algorithms – including the default MeerKAT RFI flagger, and deep U-Net architectures – across all metrics including AUC, F1-score, and MCC. We demonstrate the robustness of this improvement on both single dish and interferometric simulations and, using transfer learning, on real data. Our R-Net model’s precision is approximately $90{{\ \rm per\ cent}}$ better than the current MeerKAT flagger at $80{{\ \rm per\ cent}}$ recall and has a 35 per cent higher F1-score with no additional performance cost. We further highlight the effectiveness of transfer learning from a model initially trained on simulated MeerKAT data and fine-tuned on real, human-flagged, KAT-7 data. Despite the wide differences in the nature of the two telescope arrays, the model achieves an AUC of 0.91, while the best model without transfer learning only reaches an AUC of 0.67. We consider the use of phase information in our models but find that without calibration the phase adds almost no extra information relative to amplitude data only. Our results strongly suggest that deep learning on simulations, boosted by transfer learning on real data, will likely play a key role in the future of RFI flagging of radio astronomy data.

In radio interferometry imaging, the sparse sampling of the telescope array on the UV-plane results in a significant side lobe structure of the point spread function (PSF). The convolution of the real sky brightness and the PSF will introduce severe artifacts in the final imaging. In order to mitigate such artifacts and improve the quality of galaxy images, this paper explores the application of the Stripformer deep learning model to the reconstruction of galaxy images. The model innovatively uses the Strip Attention mechanism to re-weight the image features in the horizontal and vertical directions, effectively capture the long-distance directional dependence, so as to more accurately mitigate the artifacts caused by sparse sampling, and clearly restore the structure of the galaxy spiral arm and the fine details of the galaxy nucleus. The experiment in this paper uses simulated Galaxy Zoo galaxies observation data. The experimental results show that the average indexes of RMSE, PSNR and SSIM of the reconstructed image are 0.006,45.68 dB and 0.995, respectively, which proves the effectiveness of this method in galaxy structure recovery and artifacts suppression, providing a feasible technique for radio astronomy data processing.

The detection of close reflection peaks in ultrasonic pulse compression system has been studied by implementing the clean algorithm, which is popular in image processing of radio astronomy data. The algorithm has been adjusted so that it is applicable to ultrasonic data. It has been applied to some practical data with satisfactory results. It is found to be effective for the detection of close peaks and removing sidelobes. The algorithm has been implemented successfully in the detection of close holes in metal plates and some potential use of the method has been described.

\n Contains fulltext :\n 295725.pdf (Publisher’s version ) (Open Access)\n

Venus: Determination of atmospheric parameters from the microwave spectrum

In order to acquire high speed real-time astronomical image data and recognize the object planets, a kind of system based on FPGA and PCI bus is presented. In this system, logic design including Camera Link data interface, digital image data capture and data transmission to PC through PCI bus is completed in FPGA (Xilinx Virtex 4); and the final results can be verified in PC. The system can collect and process the digital image output from high speed CMOS camera which peak data rate is 660 MB/S, average rate is 200 MB/s, and frame ratio is 2500 fps.

The radio sky at frequencies below ∼30 MHz is virtually unobservable from Earth due to ionospheric disturbances and the opaqueness of the ionosphere below ∼10MHz, and also due to strong terrestrial radio interference. Deploying a radio observatory in space would open up this largely unexplored frequency band for science in astronomy, cosmology, geophysics, and space science. A Chinese-European team is proposing an ultra long wavelength (ULW) radio interferometer mission DSL (Discovering the Sky at the Longest Wavelengths). The proposed radio interferometer will be deployed in low-altitude lunar orbit, exploiting the radio quietness of the lunar far side. DSL will consist of a mother-spacecraft for data transport and control, plus eight small micro-satellites each equipped with three orthogonal dipoles. These satellites form a virtual distributed observatory with adjustable baselines, allowing different scientific observation strategies. The satellites are configured in a flexible quasi-linear array in nearly identical orbits, guaranteeing low relative drift rates. Short orbital periods and orbit precession ensure quick filling of the interferometric spatial frequency (u, v, w) space, enabling high quality imaging. The science themes considered for the DSL mission include pioneering studies of the unknown and exploratory science such as the search for signatures of the cosmological Dark Ages, complementing current (e.g. LOFAR) and future SKA telescope searches; full-sky continuum survey of discrete sources, including ultra-steep spectrum extragalactic sources, pulsars, and transients (galactic and extragalactic); full-sky map of continuum diffuse emission; solar-terrestrial physics, planetary sciences, and cosmic ray physics. The main frequency band covered is 1–30 MHz extending down to 0.1 MHz, and up to about 50 MHz for cross-referencing with ground-based instruments. DSL will support a variety of observational modes, including broad-band spectral analysis for Dark Ages, radio interferometric cross-correlations for imaging, and flexible raw data downlink capability. Data processing will be performed at radio astronomy science data centres in Europe and China.