Augmented transition networks as video browsing models for multimedia databases and multimedia information systems

Island parsing is a powerful technique for parsing with Augmented Transition Networks (ATNs) which was developed and successfully applied in the HWIM speech understanding project. The HWIM application grammar did not, however, exploit Woods' original full ATN specification. This paper describes an island parsing interpreter based on HWIM, but containing substantial and important extensions to enable it to interpret any grammar which conforms to that full specification of 1970. The most important contributions have been to eliminate the need for prior specification of scope clauses, to provide more power by implementing LIFTR and SENDR actions within the island parsing framework, and to improve the efficiency of the techniques used to merge together partially-built islands within the utterance.This paper also presents some observations about island parsing, based on the use of the parser described, and some suggestions for future directions for island parsing research.

Based on compressed video segment detection, the decompression step can be omitted, features can be directly extracted from the original video data stream, and the detection rate can be accelerated. This paper analyzes the characteristics of video data, video segmentation and key frame selection. Based on the analysis of the key technologies of video retrieval technology, the compressed video stream extraction method based on video segmentation and key frame is introduced. A method of extracting key frames from MPEG compressed video based on DC coefficients and motion vectors is proposed. Experiments show that the proposed method can reduce the computational burden, and can better represent the video content.

S.C. Chen et al. (1999) proposed a multimedia augmented transition network (ATN) model, together with its multimedia input strings, to model and structure video data. This multimedia ATN model was based on an ATN model that had been used within the artificial intelligence (AI) arena for natural-language understanding systems, and its inputs were modeled by multimedia input strings. The temporal and spatial relations of semantic objects were captured by an unsupervised video segmentation method called the SPCPE (simultaneous partitioning and class parameter estimation) algorithm, and they were modeled by the multimedia input strings. However, the segmentation method used was not able to identify objects that are overlapped together within video frames. The identification of overlapped objects is a great challenge. For this purpose, a backtrack-chain-update-split algorithm is developed in this paper that identifies the split segment (object) and uses this information in the current frame to update the previous frames in a backtrack-chain manner. The proposed split algorithm provides more accurate temporal and spatial information of the semantic objects for video indexing.

Segmentation, video data modeling, and annotation are indispensable operations necessary for creating and populating a video database. To support such video databases, annotation data can be collected as metadata for the database and subsequently used for indexing and query evaluation. In this paper we describe the design and development of a video annotation engine, called Vane, intended to solve this problem as a domain-independent video annotation application. Using the Vane tool, the annotation of raw video data is achieved through metadata collection. This process, which is performed semi-automatically, produces tailored SGML documents whose purpose is to describe information about the video content. These documents constitute the metadatabase component of the video database. The video data model which has been developed for the metadata, is as open as possible for multiple domain-specific applications. The tool is currently in use to annotate a video archive comprised of educational and news video content.

Key frame selection, which aims to choose a small number of frames that well represent the entire video contents, is one of the important research issues in video content analysis. Many existing methods developed so far use some prior information about the contents of video frames based on high-level semantic features, and thus they are applicable only in particular problems. Without any prior information, approaches based on low-level index features such as clusters, correlation, and consecutive frame difference are useful. Key frames found by these approaches are expected to be as much different from each other as possible. Therefore, accurately evaluating the difference among candidate key frames is a key challenge. However, a standard similarity metric such as the squared error and the cross correlation tends to provide redundant key frames. To cope with this problem, we propose to use quadratic mutual information (QMI) as an alternative similarity measure, which allows us to capture higher-order correlation for more than two variables simultaneously. Through experiments, we demonstrate the usefulness of the proposed QMI-based key frame selection method that uses an accurate and computationally efficient least-squares QMI estimator.

In this paper we present our new approach to video data preparation for a data compression enhancement named video data syntactic coding. Video data standard encoders MPEG, H. 26x use data preparation (preprocessing) based on either data spatial and temporal filtering or on video sequence complexity analysis (by auxiliary encoder use) that is exploited for video filtering. However, preprocessing efficiency is only 10% up to 15%. Our video data syntactic coding presented here is based on both video content complexity assessment in accordance with the named standard encoding algorithm (MC-DPCM) and data filtering adopted to the video complexity and a digital visual perception. The complexity is estimated by data entropy calculation based on averaging of histogram frame-to-frame changes. In addition, the histogram change is used for detection of new details appearance and their inframe position determination. The video data model is based on a detail visual perception, and the above video content analysis results provide effective video data pre-filtering. The developed video syntactic coding used together with H. 264 standard encoders presents up to 25% data compression enhancement.

Describes the design and implementation of the WVTDB (Web-based VideoText DataBase) system that demonstrates our research on video data modeling, semantic content-based video querying and video database system architectures. The video data model of WVTDB is based on multi-level video data abstractions and annotation layering, thus allowing dynamic and incremental video annotation and indexing, multi-user view sharing and video data reuse. Users can query, retrieve and browse video data based on their semantic content descriptions and temporal constraints on the video segments. WVTDB employs a modular system architecture that supports distributed video query processing and subquery caching. Several techniques, such as video wrappers and lazy delivery, are also proposed specifically to address the network bandwidth limitations for this kind of Web-based system. We also address adaptivity, data access control and user profile issues.

VRML —the Virtual Reality Modeling Language— has estab-lished as the standard description language for 3D worlds on theInternet. Although it recently has been extended by features inorder to support behaviors and user interactions, enablingauthors to realize interactive virtual worlds, there is still a lack ofsupport to share such worlds with other users.In this paper we will present our approach on supportinglarge-scaled shared virtual worlds based on VRML by an appro-priate network infrastructure as well as mechanisms to partitionthese worlds. We will show how virtual worlds can be populatedby world-wide distributed users and applications, and the re-quirements to share behaviors and interactions between those.CR Categories and Subject Descriptors: C.2.2 [ComputerCommunication Networks]: Network Protocols; C.2.4 [Com-puter Communication Networks]: Distributed Systems -Dis-tributed applications; H.5.1 [Information Interfaces and Pre-sentation] Multimedia Information Systems - ArtificialRealities; I.3.2 [Computer Graphics]: Graphics Systems -Dis-tributed/network graphics;I.3.6 [Computer Graphics]: Meth-odology and Techniques - Interaction Techniques; I.3.7. [Com-puter Graphics]: Three-Dimensional Graphics and Realism -Virtual Reality;Additional Keywords Virtual reality modeling language(VRML), multiuser environments, subdivision of shared virtualworlds, multicasting.

Due to video data exponential growth supported by quick advances in multimedia technology, it became difficult for the user to retrieve information from a large videos collection. Key frame extraction consists in providing an abstract of the entire video, containing the most informative frames. In this paper, we present an efficient key frame extraction method. This method is based on local interest points description and repeatability graph clustering via approaching modularity value. To minimize the data to be treated, the process will be applied only on a set candidate frames selected with a windowing rule. Indeed, the first step consists in extracting interest points on a set of candidate frames. After that, we compute repeatability values between each two frames from the candidate set. These values are represented by a repeatability direct graph. The selection of key frames is performed using graph clustering by approaching modularity principle. The experiments performed showed that the proposed method succeeds in extracting key frames that preserve the relevant video content without redundancy.

The advancement of multimedia technologies has enabled electronic processing of information to be recorded in formats that are different from the standard text format. These include image, audio and video formats. The video format is a rich and expressive form of media used in many areas of our everyday life, such as in education, medicine and engineering. The expressiveness of video documents is the main reason for their domination in future information systems. Therefore, effective and efficient access to video information that supports video-based applications has become a critical research area. This has led to the development of, for example, new digitizing and compression tools and technology, video data models and query languages, video data management systems and video analyzers. With applications of a vast amount of stored video data, such as news archives and digital television, video retrieval became, and still is, an active area of research.

The advancement of multimedia technologies has enabled electronic processing of information to be recorded in formats that are different from the standard text format. These include image, audio and video formats. The video format is a rich and expressive form of media used in many areas of our everyday life, such as in education, medicine and engineering. The expressiveness of video documents is the main reason for their domination in future information systems. Therefore, effective and efficient access to video information that supports video-based applications has become a critical research area. This has led to the development of, for example, new digitizing and compression tools and technology, video data models and query languages, video data management systems and video analyzers. With applications of a vast amount of stored video data, such as news archives and digital television, video retrieval became, and still is, an active area of research.

An intelligent annotation-based video data model called Smart VideoText is introduced. It utilizes the conceptual graph knowledge representation formalism to capture the semantic associations among the concepts described in text annotations of video data. The aim is to achieve more effective query, retrieval, and browsing capabilities based on the semantic content of video data. Finally, a generic and modular video database architecture based on the Smart VideoText data model is described.

Surface hardness monitoring of laser shock Peening: Acoustic emission and key frame selection

In this paper we carry out a survey on key frame extraction methods for Video Summary. We also discuss the summary evaluation criteria and compare the approaches based on the method, data set and the results. Video Summary is a process of presenting an abstract of entire video within a short period of time. It aims to provide a compact video representation, while preserving the essential activities of the original video. It is an essential task in video analysis and indexing applications. Most of the video summaries are based on selection of key frames within the shots of a video. Many of them use motion features and few use visual features for extracting the key frames. The video summary quality assessment methods are based more on subjective and less on objective measures. Tong wei Ren et al has provided a framework to assess the quality of the video against a given reference summary using both subjective and objective measures. Ciocca et al used the objective measures for evaluation of summary and most of them evaluate by taking the subjective opinion of experts. A framework for automatic evalution is needed based on both subjective and objective measures without the reference summary.

We describe the design of a multimedia database management system for a distributed news-on-demand multimedia information system. News-on-demand is an application that uses broadband network services to deliver news articles to subscribers in the form of multimedia documents. Different news providers insert articles into the database, which are then accessed by users remotely, over a broadband, asynchronous transfer-mode (ATM) network. The particulars of our design are an object-oriented approach and strict adherence to international standards, in particular the Standard Generalized Mark-up Language (SGML) and Hy-Time. The multimedia database system has a visual query facility, which is also described in this paper. The visual query interface provides three major facilities for end users: presentation, navigation, and querying of multimedia news documents. The main focus, however, is the querying of multimedia objects stored in the database.