Parallel Distributed Trajectory Pattern Mining Using Hierarchical Grid with MapReduce

SummaryAs a well‐known field of big data applications, smart city takes advantage of massive data analysis to achieve efficient management and sustainable development in the current worldwide urbanization process. An important problem in smart city is how to discover frequent trajectory sequence pattern and cluster trajectory. To solve this problem, this paper proposes a cloud‐based taxi trajectory pattern mining and trajectory clustering framework for smart city. Our work mainly includes (1) preprocessing raw Global Positioning System trace by calling the Baidu API Geocoding; (2) proposing a distributed trajectory pattern mining (DTPM) algorithm based onSpark; and (3) proposing a distributed trajectory clustering (DTC) algorithm based onSpark. The proposed DTPM algorithm and DTC algorithm can overcome the high input/output overhead and communication overhead by adopting in‐memory computation. In addition, the proposed DTPM algorithm can avoid generating redundant local trajectory patterns to significantly improve the overall performance. The proposed DTC algorithm can enhance the performance of trajectory similarity computation by transforming the trajectory similarity calculation into AND and OR operators. Experimental results indicate that DTPM algorithm and DTC algorithm can significantly improve the overall performance and scalability of trajectory pattern mining and trajectory clustering on massive taxi trace data. Copyright © 2016 John Wiley & Sons, Ltd.

The FP-growth algorithm is currently one of the fastest approaches to frequent item set mining. Fuzzy logic provides a mathematical framework where the entire range of the data lies in between 0 and 1. The PSO algorithm was developed from observations of the social behavior of animals, including bird flocking and fish schooling. It is easier to implement than evolutionary algorithms because it only involves a single operator for updating solutions. In contrast, evolutionary algorithms require a particular representation and specific methods for cross-over, mutation, and selection. Furthermore, PSO has been found to be very effective in a wide variety of applications, being able to produce good solutions at a very low computational cost. In this paper, we have considered the fuzzified dataset and have implemented various frequent pattern mining techniques. Out of the various frequent pattern mining techniques it was found that Frequent Pattern (FP) growth method yields us better results on a fuzzy dataset. Here, the frequent patterns obtained are considered as the set of initial population. For the selection criteria, we had considered the mean squared residue score rather using the threshold value. It was observed that out of the four fuzzy based frequent mining techniques, the PSO based fuzzy FP growth technique finds the best individual frequent patterns. Also, the run time of the algorithm and the number of frequent patterns generated is far better than the rest of the techniques used.

In this paper, from the point of view reducing the system calculation cost, a digital channelized receiver is proposed. The decimation factor of the proposed digital channelized receiver with low calculation cost is equal to the number of signal channels, which is acquired by changing the input signal of the embedded instantaneous frequency measurement receiver. Due to the increase of the system decimation factor, the data to be processed are decreased and the system processing speed is improved indirectly. In this paper, the architecture and working principle of the proposed digital channelized receiver with low calculation cost is firstly given; secondly, the overlapping channel division method used in the digital channelized receiver with low calculation cost is put forward. Meanwhile, the channel arbitration method of the unmodulated or modulated signal in the case of overlapping channel division is analyzed. Then, the frequency measurement principle of the embedded instantaneous frequency measurement receiver is deduced in detail. Through the use of the overlapping channel division method, the proposed digital channelized receiver with low calculation cost is able to avoid the existence of blind spots between channels and increases the interception probabilities of the input signals. And, the proposed digital channelized receiver can distinguish two signals coming simultaneously to the same channel by using the overlapping channel division method. In addition, the channel of the input signal truly residing can be simply determined by using the channel arbitration in the case of overlapping channel division and according to the output frequency of the embedded instantaneous frequency measurement receiver. Finally, the related simulation results verify the correctness and feasibility of the theories and method involved in this paper.

The rapid increase in the number of aerial and orbital Earth observation systems is generating a huge amount of remote sensing data that need to be readily transformed into useful information for policy and decision makers. This exposes an urgent demand for image interpretation tools that can deal efficiently with very large volumes of data. In this work, we introduce a set of methods that support distributed processing of georeferenced raster and vector data in a computer cluster, which may be a virtual cluster provided by cloud computing infrastructure services. The set of methods comprise a particular technique for indexing distributed georeferenced datasets, as well as strategies for distributing efficiently the processing of spatial context-aware operations. They provide the means for the development of scalable applications, capable of processing large volumes of geospatial data. We evaluated the proposed methods in a remote sensing image interpretation application, built on the MapReduce framework, and executed in a cloud computing infrastructure. The experimental results corroborate the capacity of the methods to support efficient handling of very large earth observation datasets.

Privacy preserving multi-party computation delegation for deep learning in cloud computing

Marine data acquisition and processing has generated new concepts in velocity analysis, amplitude analysis, and frequency analysis. This has been possible because of consistent surface conditions on marine data, development of direct detection of hydrocarbon concepts, and improved computing power. Exploitation of the correlation of time, amplitude, velocity, and frequency relations has been largely confined to time, amplitude, and velocity. Such basic seismic theories as dispersion, fluidity, and multiple energy indicate a strong relation between frequency and velocity. Experiments on synthetic data illustrates an additional interdependence between apparent velocity and frequency. Both the theoretical assumptions and the synthetic exercises are reenforced by experiments on seismic data. Overlays of continuous velocity analysis and continuous frequency analysis show a substantial correlation. Frequency analysis in a seismic section dimensionally compatible contour allows the correlation with velocity on large volumes of data. An example on Gulf Coast data indicates good potential for this type of analysis. INTRODUCTION The advent of marine seismic recording in conjunction with increased energy needs resulted in a very large volume of seismic data. It was also marine data which generated improved methods of rock velocity estimation. Direct detection of hydrocarbons motivated attention toward such neglected seismic observations as phase and frequency anomalies. Time relations continue to be the basis of interpretation of structure but stratigraphic exploration and direct detection of hydrocarbons brought amplitude to the fore as a meaningful and interpretable measure. Amplitude, while difficult on land data, is reasonably consistent on marine data due to the relatively steady surface conditions at sea. Analysis of both velocity and frequency began to receive more attention for the same reason. Interpretation techniques on seismic data broadened to include frequency and velocity considerations. Time, amplitude, velocity, and frequency are all directly related to the earth response. Anomalies in the seismic data usually will create anomalies in some of these basic parameters so that they are at least indirectly related to each other. Some formulas relating these measures are well known and much utilized. The best known such equation is perhaps the "moveout" equation directly relating time and velocity: (Available In Full Paper) The relation between frequency and velocity or frequency and amplitude is also direct but seldom used in data processing. However, any discussion of compressional wave theory must include frequency as a very basic parameter. Some existing seismic theory does relate velocity and frequency but actual measurement on real data is scant. It is proposed to consider their possible relationship in the context of geophysical theory and practical observation. VELOCITY DISPERSION One accepted theory, which directly relates frequency and velocity is dispersion. Residual attenuation of the seismic wavefront after distance and reflection factors are removed is proportional to frequency. The equation which governs this dependence of velocity on frequency according to O'Brien and Lucas, 1971, is: (Available In Full Paper) It is indicated by the equation that if the attenuation, Q, is known, the velocity at each frequency could be computed.

Chapter 21 - Big Data Integration

This paper presents a heart disease prediction model. Among the recent technology, internet of things-enabled healthcare plays a vital role. The medical sensors used in healthcare provide a huge volume of medical data in a continuous manner. The speed of data generation in IoT healthcare is high so the volume of data is also high. In order to overcome this problem, the proposed model is a novel three-step process to store and analyze the large volumes of data. The first step focuses on a collection of data from sensor devices. In Step 2, HBase has been used to store the large volume of medical sensor data from a wearable device to the cloud. Step 3 uses Mahout for devolving logistic regression-based prediction model. At last, ROC curve is used to find the parameters that cause heart disease.

Abstract. Uncertainties in an output of interest that depends on the solution of a complex system (e.g., of partial differential equations with random inputs) are often, if not nearly ubiquitously, determined in practice using Monte Carlo (MC) estimation. While simple to implement, MC estimation fails to provide reliable information about statistical quantities (such as the expected value of the output of interest) in application settings such as climate modeling, for which obtaining a single realization of the output of interest is a costly endeavor. Specifically, the dilemma encountered is that many samples of the output of interest have to be collected in order to obtain an MC estimator that has sufficient accuracy – so many, in fact, that the available computational budget is not large enough to effect the number of samples needed. To circumvent this dilemma, we consider using multifidelity Monte Carlo (MFMC) estimation which leverages the use of less costly and less accurate surrogate models (such as coarser grids, reduced-order models, simplified physics, and/or interpolants) to achieve, for the same computational budget, higher accuracy compared to that obtained by an MC estimator – or, looking at it another way, an MFMC estimator obtains the same accuracy as the MC estimator at lower computational cost. The key to the efficacy of MFMC estimation is the fact that most of the required computational budget is loaded onto the less costly surrogate models so that very few samples are taken of the more expensive model of interest. We first provide a more detailed discussion about the need to consider an alternative to MC estimation for uncertainty quantification. Subsequently, we present a review, in an abstract setting, of the MFMC approach along with its application to three climate-related benchmark problems as a proof-of-concept exercise.

Uncertainties in an output of interest that depends on the solution of a complex system (e.g., of partial differential equations with random inputs) are often, if not nearly ubiquitously, determined in practice using Monte Carlo (MC) estimation. While simple to implement, MC estimation fails to provide reliable information about statistical quantities (such as the expected value of the output of interest) in application settings such as climate modeling for which obtaining a single realization of the output of interest is a costly endeavor. Specifically, the dilemma encountered is that many samples of the output of interest have to be collected in order to obtain an MC estimator having sufficient accuracy; so many, in fact, that the available computational budget is not large enough to effect the number of samples needed. To circumvent this dilemma, we consider using multifidelity Monte Carlo (MFMC) estimation which leverages the use of less costly and less accurate surrogate models (such as coarser grids, reduced-order models, simplified physics, interpolants, etc.) to achieve, for the same computational budget, higher accuracy compared to that obtained by an MC estimator or, looking at it another way, an MFMC estimator obtains the same accuracy as the MC estimator at lower computational cost. The key to the efficacy of MFMC estimation is the fact that most of the required computational budget is loaded onto the less costly surrogate models, so that very few samples are taken of the more expensive model of interest. We first provide a more detailed discussion about the need to consider an alternate to MC estimation for uncertainty quantification. Subsequently, we present a review, in an abstract setting, of the MFMC approach along with its application to three climate-related benchmark problems as a proof-of-concept exercise.

BackgroundMany studies report about detection and functional characterization of cis-regulatory motifs in untranslated regions (UTRs) of mRNAs but little is known about the nature and functional role of their distribution. To address this issue we have developed a computational approach based on the use of data mining techniques. The idea is that of mining frequent combinations of translation regulatory motifs, since their significant co-occurrences could reveal functional relationships important for the post-transcriptional control of gene expression. The experimentation has been focused on targeted mitochondrial transcripts to elucidate the role of translational control in mitochondrial biogenesis and function.ResultsThe analysis is based on a two-stepped procedure using a sequential pattern mining algorithm. The first step searches for frequent patterns (FPs) of motifs without taking into account their spatial displacement. In the second step, frequent sequential patterns (FSPs) of spaced motifs are generated by taking into account the conservation of spacers between each ordered pair of co-occurring motifs. The algorithm makes no assumption on the relation among motifs and on the number of motifs involved in a pattern. Different FSPs can be found depending on different combinations of two parameters, i.e. the threshold of the minimum percentage of sequences supporting the pattern, and the granularity of spacer discretization. Results can be retrieved at the UTRminer web site: . The discovered FPs of motifs amount to 216 in the overall dataset and to 140 in the human subset. For each FP, the system provides information on the discovered FSPs, if any. A variety of search options help users in browsing the web resource. The list of sequence IDs supporting each pattern can be used for the retrieval of information from the UTRminer database.ConclusionComputational prediction of structural properties of regulatory sequences is not trivial. The presented data mining approach is able to overcome some limits observed in other competitive tools. Preliminary results on UTR sequences from nuclear transcripts targeting mitochondria are promising and lead us to be confident on the effectiveness of the approach for future developments.

Disentangled convolution for optimizing receptive field

Visualization of frequent temporal patterns in single or two populations

Most of the frequent pattern mining algorithms search for all frequent patterns. However, there are many real-life situations in which users are interested in only some tiny portions of the mined frequent patterns. For mining of constrained frequent patterns, several classes of user constraints---such as anti-monotone constraints---have been proposed and their properties have been exploited. In this paper, we introduce a new class of constraints called mixed monotone constraints. We exploit its property for effective mining of frequent patterns satisfying user constraints that sum both positive and negative numerical values.

Text classification in medical domain could result in an easier way of handling large volumes of medical data. They can be segregated depending on the type of diseases, which can be determined by extracting the decisive key texts from the original document. Due to various nuances present in understanding language in general, a requirement of large volumes of text-based data is required for algorithms to learn patterns properly. The problem with existing systems such as MedScape, MedLinePlus, Wrappin, and MedHunt is that they involve human interaction and high time consumption in handling a large volume of data. By employing automation in this proposed field, the large involvement of manpower could be removed which in turn speeds up the process of classification of the medical documents by which the shortage of medical technicians in third world countries are addressed.