Multimodal Video Database Modeling, Querying and Browsing

A wireless sensor network consists of many sensors that collect and transmit physical or environmental conditions at different locations to a server continuously. Many researches mainly focus on processing continuous queries on real-time data stream. However, they do not concern the problem of storing the historical data, which is mandatory to the historical queries. In this paper, we propose a system architecture for handling and storing sensor data stream in real-time to support the spatial and/or temporal queries besides continuous queries. We exploit a segment-based method to store the sensor data stream and reduce the managed tuples without any loss of information, which lead to the improvement of the accuracy of query results. In addition, we offer a method to reduce the cost of join operations in processing spatiotemporal queries by filtering out the list of irrelevant sensors from the query range before making the join operation. We then present a design of the system architecture for processing spatial and/or temporal queries. Finally, we implement a climate monitoring application system based on our proposed system architecture.

In this paper, we propose a system architecture for handling and storing sensor data stream in real-time to support the spatial and/or temporal queries besides continuous queries. We exploit a segment-based method to store the sensor data stream and reduce the managed tuples without any loss of information, which lead to the improvement of the accuracy of query results. In addition, we offer a method to reduce the cost of join operations in processing spatiotemporal queries by filtering out the list of irrelevant sensors from the query range before making the join operation. We then present a design of the system architecture for processing spatial and/or temporal queries. Finally, we implement a climate monitoring application system.KeywordsWireless Sensor NetworkSensor Data StreamSpatiotemporal Data StreamContinuous QueryHistorical Query

In video databases, each video contains temporal and spatial relationships between content objects. One of the well-known video indexing strategies is the 3D C-string strategy. However, it cannot deal with the condition that an object appears and then disappears for more than one time. To solve this problem, in this paper, we propose an indexing strategy, called temporal UID Matrix1. Based on the original 13 spatial relationships proposed by 2D C-string and our three new spatial relationships, we can derive the temporal relationships from the sequence of spatial relationships. Therefore, in our proposed strategy, although we build only index for spatial relationships, and we still can answer the video queries, i.e., spatial, temporal, and spatio-temporal queries. From our simulation study, we show that our proposed strategy is more efficient for video searching than the 3D C-string strategy.

Dynamic segmentation is viewed as one of the most important functions of geographic information systems for transportation applications. Although the road network and associated events (e.g., pavement material, traffic volume, incidents) can be referenced to both space and time, the spatial and temporal dimensions have not been well integrated. Modeling space-varying, time-varying, and space-time-varying events in dynamic segmentation by using an object database approach that is in line with the Object Database Management Group standard is explored. A mechanism called parametric polymorphism is used to lift conventional data types to spatial, temporal, and spatiotemporal types for maintaining knowledge about events that could change spatially, temporally, and spatiotemporally along linear features. An associated object query language, DS-OQL, was designed to support the formulation of spatial, temporal, and spatiotemporal queries on the road and event information.

Social networks, such as Facebook, Instagram, and Twitter, provide intuitive ways to share a variety of information including geotagged multimedia data within users' communities of interest (COI) or publicly in real-time. Real-time geotagged multimedia data can provide semantics to conventional spatial queries in order to enrich the user navigation experience. In this paper, we introduce a mechanism to process spatio-temporal queries by leveraging geotagged multimedia data such as images, audio, video, and text, in order to add semantics to the conventional queries. Our framework collects, stores, and spatially tags multimedia data shared by users through social networks or through our developed mobile application. The system then uses such data in order to enhance the conventional routing services by resolving existing usability issues and by providing semantics to the routes in terms of enriched points of interest while taking dynamic road conditions into account. A proof of concept of the system will be demonstrated with the following spatio-temporal queries on road networks: 1) multimedia-enhanced shortest path queries; 2) multimedia-enhanced k-nearest neighbor queries; and 3) multimedia-enhanced range queries. Finally, a novel technique for finding lost individuals using geotagged multimedia data is also introduced. The results are tailor-made to the users' smartphone bandwidth and resolution requirements

A number of algorithms of clustering spatial data for reducing the number of disk seeks required to process spatial queries have been developed. One of the algorithms is the scheme of page ordering, which is concerned with the order of pages in one-dimensional storage for storing two-dimensional spatial data. The space filling curves, especially the Hilbert curves, have been so far used to impose an order on all of pages. Page ordering based on the space filling curves, however, does not take into account the uneven distribution of spatial objects and the types of spatial queries. We will develop a cost model to define the page ordering problem based on performance measurement and then find out the method of page ordering for efficiently processing region queries in static databases. The experimental results will show that the newly proposed ordering method achieves better clustering than older methods.

Accurate, speedy and interoperable information exchange among the stakeholders achieve effective rescue and relief operations in an emergency. The current research work aims at location-based real time or near real time disaster data gathering and accumulation. The dynamic disaster data is integrated with the static geospatial data to facilitate spatial analytics and disseminate the integrated data through OGC web services to various stakeholders for further processing by different expert domain applications. The research work also facilitates spatio-temporal querying system through Geo-query, and OLAP operations on integrated disaster data with geospatial visualization. The design and implementation of the work is achieved through a mobile application integrated with a GIS based web portal by a centralized remote server. The entire architecture has been tested by implementing in an emergency situation and facilitated by an effective interoperable information exchange and spatio-temporal queries.

The video retrieval system refers to the task of retrieving the most relevant video collection, given a user query. By applying some feature extraction models the contents of the video can be extracted. With the exponential increase in video data in online and offline databases as well as a huge implementation of multiple applications in health, military, social media, and art, the Content-Based Video Retrieval (CBVR) system has emerged. The CBVR system takes the inner contents of the video frame and analyses features of each frame, through which similar videos are retrieved from the database. However, searching and retrieving the same clips from huge video collection is a hard job because of the presence of complex properties of visual data. Video clips have many frames and every frame has multiple properties that have many visual properties like color, shape, and texture. In this research, an efficient content-based video retrieval system using the AlexNet model of Convolutional Neural Network (CNN) on the keyframes system has been proposed. Firstly, select the keyframes from the video. Secondly, the color histogram is then calculated. Then the features of the color histogram are compared and analyzed for CBVR. The proposed system is based on the AlexNet model of CNN and color histogram, and extracted features from the frames are together to store in the feature vector. From MATLAB simulation results, the proposed method has been evaluated on benchmark dataset UCF101 which has 13320 videos from 101 action categories. The experiments of our system give a better performance as compared to the other state-of-the-art techniques. In contrast to the existing work, the proposed video retrieval system has shown a dramatic and outstanding performance by using accuracy and loss as performance evaluation parameters.

Conventional spatial keyword queries confront the difficulty of returning desired objects that are synonyms but morphologically different to query keywords. To overcome this flaw, this paper investigates the interactive spatial keyword querying with semantics. It aims to enhance the conventional queries by not only making sense of the query keywords, but also refining the understanding of query semantics through interactions. On top of the probabilistic topic model, a novel interactive strategy is proposed to precisely infer the latent query semantics by learning from user feedbacks. In each interaction, the returned objects are carefully selected to ensure effective inference of user intended query semantics. Query processing is carried out on a small candidate object set at each round of interaction, and the whole querying process terminates when the latent query semantics learned from user feedback becomes explicit enough. The experimental results on real check-in dataset demonstrates that the quality of results has been significantly improved through limited number of interactions.

Domain adaptation is an emerging research topic in computer vision. In some vision applications, the domain of interest (i.e., the target domain) contains very few or even no labelled samples, while an existing domain (i.e., the auxiliary/source domain) is often available with a large number of labelled examples. For example, millions of loosely labeled Flickr photos or YouTube videos can be readily obtained by using keywords (also called tags) based search. On the other hand, users may be interested in retrieving and organizing their own multimedia collections of images and videos at the semantic level, but may be reluctant to put forth the effort to annotate their photos and videos by themselves. This problem becomes more challenging because the feature distributions of training samples from the web domain and consumer domain may significantly differ in statistical properties. In

This article describes how public health information management is an interdisciplinary application which deals with cross linked application domains. Geospatial environment, place and meteorology parameters effect public health. Effective decision making plays a vital role and requires disease data analysis which in turn requires effective Public Health Knowledge Base (PHKB) and a strong efficient query engine. Ontologies enhance the performance of the retrieval system and achieve application interoperability. The current research aims at building PHKB through ontology based cross linked domain integration. It designs a dynamic GeoSPARQL query building from simple form based query composition. The spatial semantic multi criteria query engine is developed by identifying all possible query patterns considering the ontology elements and multi criteria from cross linked application domains. The research has adopted OGC, W3C, WHO and mHealth standards.

Relevance feedback and semantic retrieval have received extensive attention recently in the computer vision community. In this article, we present a semantic image query system with integrated feedback mechanism. Our system has two major components: the low-level feature space and the semantic space. In the low-level feature space, images are described by multidimensional vectors and are clustered based on the similarity of their contents. In the semantic space, the relationship among keywords is captured by a semantic hierarchy built by the aid of WordNet. Based on our system architecture, we propose a novel feedback solution for semantic retrieval called semantic feedback, which allows our system to interact with users directly at the semantic level. The short-term and long-term learning process of the semantic feedback substantially improves the image retrieval and annotation performance of our system. We demonstrate the effectiveness of our approach with experiments using 5,000 images from Corel database. The significant contribution of this article is in the scenario of having a relatively small training data set compared to the testing data set. Some of the previous work in the same direction has chosen a very large training set and a very small testing set. Clearly, the problem that we try to solve is more realistic and challenging.

This paper presents an approach to segmenting individual news stories in broadcast news programmes. The approach first performs shot boundary detection and keyframe extraction on the programme. Shots are then clustered into groups based on their colour and temporal similarity. The clustering process is controlled using the groups' statistics. After clustering, a set of criteria are applied and groups are successively eliminated in order to converge upon a set of anchorperson groups. The temporal locations of the shots in these anchorperson groups are then used to segment the programme in terms of individual news items. This work is carried out within the context of a complete video indexing, browsing and retrieval system.

We investigate the problem of processing historical queries on a sensor network. Since data is considered to have been already collected at the sensor nodes, the main issue is exploring the spatial component of the query in order to minimize its cost represented by the energy consumption. We assume queries can be issued at any network node, i.e., there is no central base station and all nodes have only local knowledge of the network. On the one hand, a globally optimum query processing plan is desirable but its construction is not possible due to the lack of global knowledge of the network. On the other hand, while a simple network flooding is feasible, it is not a practical choice from a cost perspective. To address this problem we propose a two-phase query processing strategy, where in the first phase a path from the query originator to the query region is found and in the second phase the query is processed within the query region itself. This strategy is supported by analytical models that are used to dynamically select the best processing strategy depending on the query specifics. Our extensive analytical and experimental results show that our analytical models are accurate and that the two-phase strategy is better suited for small to medium sized queries, being up to 10 times more cost effective than a typical network flooding. In addition, the dynamic selection of a query processing technique proved itself capable of always delivering at least as good performance as the most energy efficient strategy for all query sizes.

Existing approaches for privacy-aware mobility data sharing aim at publishing an anonymized version of the mobility dataset, operating under the assumption that most of the information in the original dataset can be disclosed without causing any privacy violations. In this paper, we assume that the majority of the information that exists in the mobility dataset must remain private and the data has to stay in-house to the hosting organization. To facilitate privacy-aware sharing of the mobility data we develop a trajectory query engine that allows subscribed users to gain restricted access to the database to accomplish various analysis tasks. The proposed engine (i) audits queries for trajectory data to block potential attacks to user privacy, (ii) supports range, distance, and k-nearest neighbors spatial and spatiotemporal queries, and (iii) preserves user anonymity in answers to queries by (a) augmenting the real trajectories with a set of carefully crafted, realistic fake trajectories, and (b) ensuring that no user-specific sensitive locations are reported as part of the returned trajectories.