Structural Analysis of Molecular Clouds: Dendrograms

Magnetic fields in galaxies exist on various spatial scales. Large‐scale magnetic fields are thought to be generated by the α − Ω dynamo. Small‐scale galactic magnetic fields (1 kpc and below) can be generated by tangling the large‐scale field. The analysis of field structures with the help of polarized radio continuum emission is hampered by the effect of Faraday dispersion (due to fluctuations in magnetic field and/or thermal electron density), which shifts the signals from large to small scales. At long observation wavelengths, large‐scale magnetic fields may become invisible, as in the case of the spectro‐polarimetric data cube of the spiral galaxy NGC 6946 observed with the Westerbork Synthesis Radio Telescope in the wavelength range 17–23 cm. The application of rotation measure (RM) synthesis alone does not overcome this problem. We propose to decompose the Faraday data cube into data cubes at different spatial scales by a wavelet transform. Signatures of the “magnetic arms” observed in NGC 6946 at shorter wavelengths then become visible. Our method allows the use of small‐scale structures as tracers of large‐scale magnetic field patterns in data cubes at long wavelengths, as provided by new‐generation radio telescopes.

A range query applies an aggregation operation over all selected cells of an OLAP data cube where the selection is specified by providing ranges of values for numeric dimensions. We present fast algorithms for range queries for two types of aggregation operations: SUM and MAX. These two operations cover techniques required for most popular aggregation operations, such as those supported by SQL.For range-sum queries, the essential idea is to precompute some auxiliary information (prefix sums) that is used to answer ad hoc queries at run-time. By maintaining auxiliary information which is of the same size as the data cube, all range queries for a given cube can be answered in constant time, irrespective of the size of the sub-cube circumscribed by a query. Alternatively, one can keep auxiliary information which is 1/bd of the size of the d-dimensional data cube. Response to a range query may now require access to some cells of the data cube in addition to the access to the auxiliary information, but the overall time complexity is typically reduced significantly. We also discuss how the precomputed information is incrementally updated by batching updates to the data cube. Finally, we present algorithms for choosing the subset of the data cube dimensions for which the auxiliary information is computed and the blocking factor to use for each such subset.Our approach to answering range-max queries is based on precomputed max over balanced hierarchical tree structures. We use a branch-and-bound-like procedure to speed up the finding of max in a region. We also show that with a branch-and-bound procedure, the average-case complexity is much smaller than the worst-case complexity.

<p>Earth Observation (EO) data availability is drastically increasing thanks to the Copernicus Sentinel missions. In 2014 Sentinel data volumes were approximately 200 TB (one operational mission) while in 2019 these volumes rose to 12 PB (nine operational missions) and will increase further with the planned launch of new Sentinel satellites. Dealing with this big data evolution has become an additional challenge in the development of downstream services next to algorithm development, product quality control, and data dissemination techniques.</p><p>The H2020 project ‘Data Cube Service for Copernicus (DCS4COP)’ addresses the downstream challenges of big data integrating Copernicus services in a data cube system. A data cube is typically a four-dimensions object, with a parameter dimension and three shared dimensions (time, latitude, longitude). The traditional geographical map data is transformed into a data cube based with user-defined spatial and temporal resolutions using tools such as mathematical operations, sub-setting, resampling, or gap filling to obtain a set of consistent parameters.</p><p>This work describes how different EO datasets are integrated in a data cube system to monitor the water quality in the Belgian Continental Shelf (BCS) for a period from 2017 to 2019. The EO data sources are divided in four groups: 1) high resolution data with low temporal coverage (i.e. Sentinel-2), 2) medium resolution data with daily coverage (i.e. Sentinel-3), 3) low resolution geostationary data with high coverage frequency (i.e. MSG-SEVIRI), and 4) merged EO data with different spatial and temporal information acquired from CMEMS. Each EO dataset from group 1 to 3 has its own thematic processor that is responsible for the acquisition of Level 1 data, the application of atmospheric corrections and a first quality control (QC) resulting in a Level 2 quality-controlled remote sensing reflectance (Rrs) product. The Level 2 Rrs is the main product used to generate other ocean colour products such as chlorophyll-a and suspended particulate matter. Each product generated from the Rrs passed by a second QC related to its characteristic and improvements (when applied) and organized in a common data format and structure to facilitate the direct integration into a product and sensor specific. At the end of the process, these products are defined as quality-controlled analysis ready data (ARD) and are ingested in the data cube system enabling fast and easy access to these big data volumes of multi-scale water quality products for further analysis (i.e. downstream service). The data cube system grants a fast and easy straightforward access converting netCDF data to Zarr and placing it on the server. In Zarr datasets, the object is divided into chunks and compressed while the metadata are stored in light weight .json files. Zarr works well on both local filesystems and cloud-based object stores which makes it possible to use through a variety of tools such as an interactive data viewer or jupyter notebooks.</p>

In high dimensional environments, the sheer size and volume of data poses a number of challenges in order to generate meaningful and informative data cubes. Data cube construction and exploration is a manual process in which analysts are required to visually explore the complex cube structure in order to find interesting information. Data cube construction and exploration has been dealt separately in the literature and in the past there has been very limited amount of work done which would guide the data warehouse designers and analysts to automatically construct and intelligently explore the data cubes. In the recent years, the combined use of data mining techniques and statistical methods has shown promising results in discovering knowledge from large and complex datasets. In this chapter, we propose a methodology that utilizes hierarchical clustering along with Principal Component Analysis (PCA) to generate informative data cubes at different levels of data abstraction. Moreover, automatically ranked cube navigational paths are provided by our proposed methods to enhance knowledge discovery from large data cubes. The methodology has been validated using real world dataset taken from UCI machine learning repository and the results show that the proposed approach assists in cube design and intelligent exploration of interesting cube regions.

Comparison of thresholding techniques for quantifying portland cement hydrates using synchrotron microtomography

The authors describe an approach to modeling large volumes of heterogeneous spatial data in the form of a hypercube based on discrete global grid systems. Bibliometric analysis and literature review of academic publications, mapping and explanation of the scientific landscape on the subject of big data and data cubes in Earth sciences are carried out. The mentioned phenomenon is interpreted in Earth sciences in the view of the spatial data life cycle. The results show that its transformative impact on cartography and geoinformatics is mutual, and the resulting methodological problem is their heterogeneity, not volume. To model them, it is proposed to use a data cube in which the spatial dimension is represented using discrete global grid systems with advantages over raster and vector models in application to that phenomenon. The content of the data cube is analysis-ready information.

We present new continuum and molecular line data from the ALMA Three-millimeter Observations of Massive Star-forming regions (ATOMS) survey for the two protoclusters, G12.42+0.50 and G19.88−0.53. The 3 mm continuum maps reveal seven cores in each of the two globally contracting protoclusters. These cores satisfy the radius–mass relation and the surface mass density criteria for high-mass star formation. Similar to their natal clumps, the virial analysis of the cores suggests that they are undergoing gravitational collapse ($\rm \alpha _{vir} \lt \lt 2$). The clump to core scale fragmentation is investigated and the derived core masses and separations are found to be consistent with thermal Jeans fragmentation. We detect large-scale filamentary structures with velocity gradients and multiple outflows in both regions. Dendrogram analysis of the H13CO+ map identifies several branch and leaf structures with sizes ∼ 0.1 and 0.03 pc, respectively. The supersonic gas motion displayed by the branch structures is in agreement with the Larson power law indicating that the gas kinematics at this spatial scale is driven by turbulence. The transition to transonic/subsonic gas motion is seen to occur at spatial scales of ∼0.1 pc indicating the dissipation of turbulence. In agreement with this, the leaf structures reveal gas motions that deviate from the slope of Larson’s law. From the large-scale converging filaments to the collapsing cores, the gas dynamics in G12.42+0.50 and G19.88−0.53 show scale-dependent dominance of turbulence and gravity and the combination of these two driving mechanisms needs to be invoked to explain massive star formation in the protoclusters.

\n Context. Cometary Globule 1 (CG 1) is the archetype Cometary Globule in the Gum nebula.\n Aims. We attempt to discover stars possibly embedded in the globule head of the Gum nebula and to map distribution of its ISM.\n Methods. We analyse C18O spectral line observations, NIR spectrosopy, narrow and broad band NIR imaging, and stellar photometry to determine the structure of the CG 1 head and the extinction of stars in its direction.\n Results. A young stellar object (YSO) associated with a bright NIR nebulosity and a molecular hydrogen object (MHO 1411, a probable obscured HH-object) were discovered in the globule. Molecular hydrogen and Brγ line emission is seen in the direction of the YSO. The observed maximum optical extinction in the globule head is 9.m2. The peak N(H2) column density and the total mass derived from the extinction are 9.0 × 1021 cm-2 and 16.7 M⊙ (d/300 pc)2. The C18O emission in the globule head is detected in a 1′̣5 by 4′ area with a sharp maximum SW of the YSO. Three regions can be discerned in C18O line velocity and excitation temperature. Because of variations in the C18O excitation temperature the integrated C18O line emission does not follow the optical extinction. It is argued that the changes in the C18Oexcitation temperatures are caused by radiative heating by NX Pup and interaction of the YSO with the parent cloud. No indication of a strong molecular outflow from the YSO is evident in the molecular line data. The IRAS point source 07178–4429 located in the CG 1 head resolves into two sources in the HIRES enhanced IRAS images. The 12 and 25 μm emission originates mainly in the star NX Puppis and the 60 and 100 μm emission in the YSO. \n

We report the results obtained from a multiwavelength study of the H ii region G18.148−0.283 using the upgraded Giant Metrewave Radio Telescope at 1350 MHz, along with other archival data. In addition to the radio continuum emission, we have detected the H169α and H170α radio recombination lines toward G18.148−0.283 using a correlator bandwidth of 100 MHz. The moment-1 map of the ionized gas reveals a velocity gradient of approximately 10 km s−1 across the radio continuum peaks. The 12CO (J = 3−2) molecular line data from the CO High-Resolution Survey (COHRS) also show the presence of two velocity components that are very close to the velocities detected in the ionized gas. The spectrum and position–velocity diagram from CO emission reveal molecular gas at an intermediate-velocity range bridging the velocity components. We see mid-infrared absorption and far-infrared emission establishing the presence of a filamentary infrared dark cloud, the extent of which includes the targeted H ii region. The magnetic field inferred from dust polarization is perpendicular to the filament within the H ii region. We have also identified two O9 stars and 30 young stellar objects toward the target using data from the Two Micron All Sky Survey (2MASS), UKIRT Infrared Deep Sky Survey (UKIDSS), and Galactic Legacy Infrared Midplane Survey Extraordinaire (GLIMPSE). Cumulatively, this suggests that the region is the site of a cloud–cloud collision that has triggered massive star formation and subsequent formation of an H ii region.

Usually data warehouses store a large volume of data, keeping in their structures all information decision-makers need on their daily activities. Usually, a significant amount of this information is processed and materialized in data cubes to be explored latter using the most flexible ways that on-line analytical processing platforms dispose to do that. However, this materialization process requires a lot of computational resources (processing time and storage), provoking as well serious performance bottlenecks in analytical systems. The distribution of such analytical structures across an organization seems to be quite appellative to attenuate such annoying effects, being in most of the cases a good alternative for improving system performance without great expense. Still, a distributed computational platform use to be less expensive than a centralized one, what give it some good advantages in any solution related to data cube distribution. Following this slope appears Grid environments. As we know these can use computational resources geographically dispersed, heterogeneously and non-dedicated as it was a single a unique environment. However, in order to reach higher performance levels, Grids need proper scheduling and data so they can be able to respond effectively to the jobs that were sent to them. In this paper, we integrate a performance prediction method in a data cube distribution strategy to define a more effective allocation of data cube views in a conventional Grid environment with the ability to receive them.

The total mass estimate of molecular clouds suffers from the uncertainty in the H2-CO conversion factor, the so-called XCO factor, which is used to convert the 12CO (1–0) integrated intensity to the H2 column density. We demonstrate the machine learning’s ability to predict the H2 column density from the 12CO, 13CO, and C18O (1–0) data set of four star-forming molecular clouds: Orion A, Orion B, Aquila, and M17. When the training is performed on a subset of each cloud, the overall distribution of the predicted column density is consistent with that of the Herschel column density. The total column density predicted and observed is consistent within 10 per cent, suggesting that the machine learning prediction provides a reasonable total mass estimate of each cloud. However, the distribution of the column density for values >∼2 × 1022 cm−2, which corresponds to the dense gas, could not be predicted well. This indicates that molecular line observations tracing the dense gas are required for the training. We also found a significant difference between the predicted and observed column density when we created the model after training the data on different clouds. This highlights the presence of different XCO factors between the clouds, and further training in various clouds is required to correct for these variations. We also demonstrated that this method could predict the column density towards the area not observed by Herschel if the molecular line and column density maps are available for the small portion, and the molecular line data are available for the larger areas.

In the context of ongoing climate change, extreme weather events are drawing increasing attention from the public and news media. A question often asked is how the likelihood of extremes might have changed by anthropogenic greenhouse-gas emissions. Answers to the question are strongly influenced by the model used, duration, spatial extent, and geographic location of the event—some of these factors often overlooked. Using output from four global climate models, we provide attribution statements characterised by a change in probability of occurrence due to anthropogenic greenhouse-gas emissions, for rainfall and temperature extremes occurring at seven discretised spatial scales and three temporal scales. An understanding of the sensitivity of attribution statements to a range of spatial and temporal scales of extremes allows for the scaling of attribution statements, rendering them relevant to other extremes having similar but non-identical characteristics. This is a procedure simple enough to approximate timely estimates of the anthropogenic contribution to the event probability. Furthermore, since real extremes do not have well-defined physical borders, scaling can help quantify uncertainty around attribution results due to uncertainty around the event definition. Results suggest that the sensitivity of attribution statements to spatial scale is similar across models and that the sensitivity of attribution statements to the model used is often greater than the sensitivity to a doubling or halving of the spatial scale of the event. The use of a range of spatial scales allows us to identify a nonlinear relationship between the spatial scale of the event studied and the attribution statement.

The volume of data in the field of Earth data observation has increased considerably, especially with the emergence of new generations of satellites and models providing much more precise measures and thus voluminous data and files. One of the most traditional and popular data formats used in scientific and education communities (reference) is the NetCDF format. However, it was designed before the development of cloud storage and parallel processing in big data architectures. Alternative solutions, under open source or under proprietary licences, appeared in the past few years (See Rasdaman, Opendatacube). These data cubes are managing the storage and the services for an easy access to the data but they are also altering the input information applying conversions and/or reprojections to homogenize their internal data structure, introducing a bias in the scientific value of the data. The consequence is that it drives the users in a closed infrastructure, made of customized storage and access services. The objective of this study is to propose a light new open source solution which is able to store gridded datasets into a native big data format and make data available for parallel processing, analytics or artificial intelligence learning. There is a demand for developing a unique storage solution that would be opened to different users: Scientists, setting up their prototypes and models in their customized environment and qualifying their data to publish as Copernicus datatsets for instance; Operational teams, in charge of the daily processing of data which can be run in another environment, to ingest the product in an archive and make it available to end-users for additional model and data science processing. Data ingestion and storage are key factors to study to ensure good performances in further subsetting access services and parallel processing. Through typical end users’ use cases, four storage and services implementations are compared through benchmarks: Unidata's THREDDS Data Server (TDS) which is a traditional NetCDF data access service solution built on the NetCDF-Java, an extension of the THREDDS Data Server using object store, pangeo/Dask/Python ecosystem, and the alternative Hadoop/Spark/Parquet solution, driven by CLS technical and business requirements.

We present the theoretical foundations for a prestack migration technique to image teleseismic P‐to‐S converted phases. The method builds on teleseismic P wave deconvolution, pseudostation stacking [Neal and Pavlis, 1999] and on the idea of using a plane wave decomposition for imaging as introduced by Treitel et al. [1982]. Deconvolution operators are constructed by pseudostation stacking of the array aligned to the incident P wave arrival times to produce a space‐variable deconvolution operator. The resulting data are then muted to remove the deconvolved direct P wave pulse and pseudostation stacked over a grid of feasible slowness vectors. The pseudostation stack interpolates the wave field onto a regular grid along Earth's surface producing a series (one per slowness vector) of uniformly sampled three‐dimensional data cubes (two space variables and time). The plane wave components can be propagated downward using a form of approximate ray tracing with a three‐dimensional Earth model. This yields a series of distorted cubes topologically equivalent to the original uniformly sampled data cubes. These data volumes are summed as a weighted stack with the weights derived from an integration formula for inverse scattering based on the generalized Radon transform. This allows an image of the subsurface to be constructed on an event by event basis beneath the array. We apply this technique to data from the Lodore array that was deployed in northwestern Colorado. The results suggest the presence of a major lithospheric‐scale discontinuity defined by a south dipping boundary.

Various compression schemes have been suggested for storage and distribution of hyperspectral remotely sensed data. Hyperspectral forestry applications that rely on the measurement of subtle variations in the spectral signature of the forest canopy can be affected by modification of the spectra induced by compression. As part of an experiment for the Canadian Space Agency (CSA), Hyperion data cubes acquired over the Greater Victoria Watershed District (GVWD) were compressed using the Successive Approximation Multi-stage Vector Quantization (SAMVQ) and Hierarchical Self-Organizing Cluster Vector Quantization (HSOCVQ) algorithms developed by CSA. The data were compressed using compression ratios 10:1 and 20:1 and were returned uncompressed. The data cubes were classified into forest species using the same supervised classification methodology as applied to the original data. The classification accuracies were compared. For some applications, one can achieve significant reductions in data volume through compression. Of the compression algorithms and ratios tested, SAMVQ 10:1 has the least overall effect but still reduces classification accuracies on difficult to separate classes. While uncompressed data are preferred, SAMVQ 10:1 compression may be suitable for forest inventory.