Parallel visualization on large clusters using MapReduce

In this paper we introduce a method for the display of isosurfaces extracted from unstructured tetrahedral grids. Our algorithm completely runs on the graphics hardware. The tetrahedra are streamed into a vertex program, which extracts the surface for the given isovalue and immediately renders it. The triangles are not stored explicitly but are computed during rendering time, so the user can modify the isovalue with immediate feedback. If the tetrahedra entirely fit into video memory, we achieve a throughput of more than nine million tetrahedra per second. Our performance can be further improved by using a hybrid method which pre-selects tetrahedra containing the isovalue. We compare our approach with a pure CPU based implementation which achieves about half the performance of our hardware accelerated method.

High resolution 3D range scanning as well as iso-surface extraction have introduced densely and uniformly sampled models that are difficult to render at an interactive rate. To remove excessive details and produce meshes of various resolutions for different kinds of applications, the study of fast and high quality polygonal mesh simplification algorithms has become important. In this paper, we propose a new linear time algorithm that can achieve fast and high quality mesh simplification. In the new algorithm, we pipeline the cost computation, optimization, and edge collapse, and use a small constant-sized replacement selection min-heap instead of a large greedy queue to effectively reduce the runtime complexity to linear complexity. Compared to previous works, our new algorithm has at least three advantages. First, the new algorithm is runtime efficient. Second, the new algorithm is memory efficient. Third, the algorithm is capable of generating competitive high quality outputs.

Big Data is a data analysis methodology made possible by current technological and architectural advancements. However, big data requires a significant investment in hardware and processing power, making it prohibitively expensive for small and medium-sized firms to implement. Small and medium-sized enterprises can benefit from cloud computing promise of big data implementation. The MapReduce programming paradigm is used to process large amounts of data. The MapReduce paradigm typically necessitates networked connected storage and parallel computation. The computing requirements of MapReduce programming are frequently beyond the capabilities of small and medium-sized businesses. Cloud computing is network access to computing resources on demand, given by a third party Hardware cost reduction, processing cost reduction, and the flexibility to verify the value of big data are three significant reasons for small and medium-sized organizations to embrace cloud computing for big data technology implementation. Security and loss of control are two main concerns with cloud computing. Key Words: Big data, MapReduce, cloud computing, private cloud, public cloud, and hybrid cloud.

HiePaCo: Scalable Hierarchical Exploration in Abstract Parallel Coordinates Under Budget Constraints

Nowadays, MapReduce becomes one of the most common computational paradigms that provide an efficient parallel processing of large-scale data especially in public clouds in order to enable users to process data without considering physical infrastructures and software installations. However, the deployment of MapReduce in such public environment needs to deal with many security threats attacks during networks communication. In this paper, we present a novel approach to test Access Control List (ACL) policies in MapReduce framework. The proposed system uses these ACLs to control which users can submit to certain queues as well as which users can administer a queue for example. To this end, we used the FSM formalism to write our system specification that takes into account these policies expressed in XACML language.

Machine learning for Big Data analytics in plants.

In this paper we focus on the implications of implementing generic algorithms on graphics hardware. As an example, we ported the dimensionality reduction algorithm FastMap to fragment programs and thus accelerated it by orders of magnitude, allowing for interactive tweaking and evaluating of the algorithm parameters for datasets of several hundred thousand points and tens of dimensions; even the animation of structural changes in relation to parameters is possible. This allows to complement the algorithmic heuristic used by FastMap by explorative results from human interaction. Such an approach can be considered a heuristic in itself, but has the advantage of being based on visual feedback, therefore allowing for iterative improvement of the results. Thus we demonstrate how to benefit from the high execution parallelism on commodity graphics hardware as an alternative to making use of other, more costly, multiprocessing techniques. We discuss performance and bandwidth aspects as well as accuracy problems since these results are of more general interest and can be applied to general processing on graphics hardware as a whole.

We consider the problem of isosurface extraction and rendering for large scale time varying data. Such datasets have been appearing at an increasing rate especially from physics-based simulations, and can range in size from hundreds of gigabytes to tens of terabytes. We develop a new simple indexing scheme, which makes use of the concepts of the interval tree and the span space data structures. The new scheme enables isosurface extraction and rendering in I/O optimal time, using more compact indexing structure and more effective bulk data movement than the previous schemes. Moreover, our indexing scheme can be easily extended to a multiprocessor environment in which each processor has access to its own local disk. The resulting parallel algorithm is provably efficient and scalable. That is, it achieves load balancing across the processors independent of the isovalue, with almost no overhead in the total amount of work relative to the sequential algorithm. We conduct a large number of experimental tests on the University of Maryland Visualization Cluster using the Richtmyer-Meshkov instability dataset, and obtain results that consistently validate the efficiency and the scalability of our algorithm.

An improved multi-channel graph convolutional network and its applications for rotating machinery diagnosis

Can virtual reality be useful for visualizing and analyzing molecular structures and three-dimensional (3D) microscopy? Uses we are exploring include studies of drug binding to proteins and the effects of mutations, building accurate atomic models in electron microscopy and x-ray density maps, understanding how immune system cells move using 3D light microscopy, and teaching schoolchildren about biomolecules that are the machinery of life. Virtual reality (VR) offers immersive display with a wide field of view and head tracking for better perception of molecular architectures and uses 6-degree-of-freedom hand controllers for simple manipulation of 3D data. Conventional computer displays with trackpad, mouse and keyboard excel at two-dimensional tasks such as writing and studying research literature, uses for which VR technology is at present far inferior. Adding VR to the conventional computing environment could improve 3D capabilities if new user-interface problems can be solved. We have developed three VR applications: ChimeraX for analyzing molecular structures and electron and light microscopy data, AltPDB for collaborative discussions around atomic models, and Molecular Zoo for teaching young students characteristics of biomolecules. Investigations over three decades have produced an extensive literature evaluating the potential of VR in research and education. Consumer VR headsets are now affordable to researchers and educators, allowing direct tests of whether the technology is valuable in these areas. We survey here advantages and disadvantages of VR for molecular biology in the context of affordable and dramatically more powerful VR and graphics hardware than has been available in the past.

Recently, a large amount of records, especially in electronic form, is generated every day. The traditional systems managing these records are no more able to support the indexing of these huge and fast mass of data. New technologies summarized as ‘big data’ have emerged such as MapReduce that allows parallel data processing. In this context the objective of this paper is to propose an implementation of MapReduce programming model in order to optimize Big Electronic Records semantic indexing.

Parallel data processing complicates the completion of string similarity joins because parallel data processing requires the use of a well designed data partitioning scheme. Moreover, efficient verification of string pairs is needed to speed up the entire string similarity join process. We propose a novel framework that addresses these requirements through the use of edit distance constraints. The Landmark-Join framework has two functions that reduce two kinds of search spaces. The first, q-bucket partitioning, reduces the number of verifications of dissimilar string pairs and lowers skewness among buckets. The second, local upper bound calculation, prunes the search space of edit distance to speed up each verification. Experimental results show that Landmark-Join has good parallel scalability and that the two proposed functions speed up the entire string similarity join process.

The ever-increasing need for the large-scale implementation of unmanned vehicles requires situational awareness of such vehicles. In this case two general directions are possible. The first: simplification and fixing situations with vehicles movements. The second: improving the onboard control system and its information component. We focus on the second direction in our study. Information support of vehicles, especially systems of technical vision, requires ever-increasing computing resources and intelligent technologies in algorithmic support. The experience of our research shows that reconfigurable high-performance heterogeneous resources are the effective software and hardware tool for information support of autonomous vehicles Such devices combine sequentially parallel processing of visual data. In the process of developing such resources, we use a heterogeneous computer network based on the PCI express bus. The report describes the general scheme of such a network, considers hardware and software configurations for solving application task, gives the results of experiments to solve the issues of collecting and processing visual data, and discusses the prospects for the development of this approach.

In this demonstration we will showcase CASPER, a novel tool that enables sequential data-intensive programs to automatically leverage the optimizations provided by parallel data processing frameworks. The goal of CASPER is to reduce the inertia against adaptation of new data processing frameworks---particularly for non-expert users---by automatically re-writing sequential programs written in general purpose languages to the high-level DSLs or APIs of these frameworks. Through CASPER's browser-based interface, users can enter the source code of their Java applications and have it automatically retargeted to execute on Apache Spark. In our interactive presentation, we will use CASPER to optimize sequential implementations of data visualization programs as well as image processing kernels. The optimized Spark implementations along with the original sequential implementations will then be executed simultaneously on the cloud to allow the demo to audience compare the runtime performances and outputs in real-time.

Three models of mind were proposed during the history of cognitive science: functionalist, psychological, and neural model. The functionalist model of mind is based on symbol manipulation (computation). This model has two versions: the conception of mind as an innate information processor and problem solver, and mind as innate cognitive and visual system. A psychological model of mind is the extension by “common sense”. The alternative is mind as a cognitive and visual system that is based mostly on manipulation of internal mental data. The neural model is the only model of embodied mind. It has two versions: non-pragmatic and pragmatic (autopoietic, distributed, and social mind).