Specification and implementation of reusable multimedia presentations

Synchronization is the major processing task during multimedia presentation, which defines the time relation between multimedia objects. There are many synchronization specification methods, and each has different advantages and disadvantages. Those methods work well for the simple and specific synchronization case, but when considering large complex multimedia presentation, there lacks an effective method to deal with the complex time relation. In this paper, a synchronization point based specification method (SPM) is proposed, and based on it, a static and dynamic validation mechanism for the synchronization relation is given. In SPM, the synchronization relation is defined in synchronization points (SP). Multimedia presentation control is based on SP's status during presentation. According to the order of SPs, SPM gives the static and dynamic validation algorithm, which can avoid the conflict relation between SPs. SPM gives a way to define the nondetermined synchronization relation between multimedia objects, and proposes static and dynamic validation mechanisms for complex synchronization control. This synchronization point based mechanism can extend the capability of synchronization definition and control in a multimedia system.

Given a very large multimedia database, how to process k-nearest-neighbor queries efficiently? While the sequential scan is one of the most obvious solutions for small-to-moderate multimedia databases, it becomes practically infeasible when the database size grows. Concomitant with the volume and velocity of data, multimedia databases are frequently endowed with a complex distance-based similarity model that supports content-based data access in an adjustable and adaptive manner. Typical for many state-of-the-art distance-based similarity models is an at least quadratic computation time complexity for a single distance evaluation between two multimedia objects. Thus the search for the most query-like multimedia objects is still one of the major challenges.In this paper, we address the problem of optimal k-nearest-neighbor query processing via multiple lower bound approximations in very large multimedia databases. To this end, we propose the concepts of filter-optimality and refinement-optimality and present the Cascading Multi-Step Algorithm and the Interleaved Multi-Step Algorithm for fast query processing. Besides the algorithms’ properties, we study their query processing performance with respect to the number of CPU and I/O operations on large-scale benchmark multimedia databases. Our performance analysis shows how to process k-nearest-neighbor queries in multimedia databases efficiently and provides a guide for further research.

The advantages of using a formal specification are in its precise definition of object properties and its declarative description of operations of a system. The Z notation is used widely in the literature of formal system designs and the research of language semantics. In line with the rapid growth of multimedia omputation research, we use the Z notation to describe a multimedia database that supports the reuse of multimedia presentations. The database consists of two layers: the frame layer and the resource layer. A frame group in the first layer is the basic presentation unit to be reused. A resource group in the second layer is a collection of resources to be used by a frame, or a frame group. In the hierarchy, we use four types of links to group frames and/or resources. The main contribution of this paper is in its reuse mechanism of multimedia objects in a multimedia database.

Multimedia applications require the handling of synchronization between media streams. We present tools for creating, editing and presenting synchronized multimedia objects. Specifying synchronization is supported by the graphical Synchronization Editor and execution of multimedia presentations is performed by the Synchronizer.Major topics cover requirements resulting from underlying distributed heterogeneous environments and the consideration of user interactions. The tools we have developed are not restricted to a fixed set of media, but support the inclusion of arbitrary user-defined media.The Synchronization Editor and the Synchronizer are part of the Mode project, that is used for the handling of distributed multimedia objects in NESTOR, an authoring-learning environment in development at the Universities of Karlsruhe and Kaiserslautern in cooperation with the CEC Karlsruhe, Digital Equipment.

Multimedia presentations are composed of objects belonging to different data types such as video, audio, text and image. An important aspect is that, quite often, the user defining a presentation needs to express sophisticated temporal and spatial constraints among the objects composing the presentation. We present a system (called MPGS-Multimedia Presentation Generator System) which supports the specification of constraints among multimedia objects and the generation of multimedia presentations according to the specified constraints. The constraint model provided by MPGS is very flexible and powerful in terms of the kinds of object constraints it can represent. A large number of innovative features are supported including: asynchronous and simultaneous spatial constraints; components of interest and priority levels; motion functions. Obviously, the flexibility provided to the users requires the development of nontrivial techniques to check constraint consistency and to generate a presentation satisfying the specified constraints. We illustrate the solutions we have devised in the framework of MPGS.

Querying multimedia presentations

A high-resolution and high-throughput synchrotron powder diffractometer has been automated for use with samples that are mailed in by Advanced Photon Source users. Implementation of a relational database with web interfaces for both outside users and beamline staff, which is integrated into the facility-wide proposal and safety system, allows all aspects of beamline management to be integrated. This system permits users to request kits for mounting samples, to provide sample safety information, to obtain their collected data and to provide usage information upon project completion in a quick and simple manner. Beamline staff use a separate interface to note receipt of samples, schedule and collect diffraction data, post-process and quality-check data, and dispose of samples. The design of the software and database are discussed in detail.

A multimedia database management system (MMDBMS) must provide facilities to store, model and query multimedia data. In the context of the STORM (Structural and Temporal Object-oRiented Multimedia database system) project, we are developing an object-oriented client-server MMDBMS which addresses these problems. We use an object-oriented approach to develop reusable components which can be easily integrated in the general framework of multimedia database applications. In this paper, we present the main functionalities of our system and we focus on video objects and their integration in multimedia presentations.

In today's fast-growing information age, multimedia data is becoming more and more common in daily applications, however this data is nearly useless if there is no computer-aided browsing, searching, and retrieving mechanism to obtain the desired contents. In this demonstration, we present a multimedia query and presentation management system for multimedia databases. It uses an unsupervised segmentation method for image and video feature extraction. A presentation design interface systems is provided that allows query results to be used to create multimedia presentations.

In this paper, we discuss some design principles that will aid in the design and querying of multimedia databases. We use an object-relational data model and argue that multimedia objects should normally have a special attribute called ‘core’ which stores the real object itself in contrast to the abstraction which is reflected in the rest of the attributes. We present an extension to the ER Diagram that takes advantage of the ‘core’ notion to facilitate design of multimedia databases. We discuss some desirable features in a query language for multimedia databases: simplifications like the use of path expressions and implicit use of functions (methods) as attributes, and explicit specification of the display layout and format either at the data definition level or query specification level. To materialize this last feature, we propose a display specification extension to SQL (SQL+D) that we have implemented.

Multimedia data mining refers to pattern discovery, rule extraction and knowledge acquisition from multimedia database. Two typical tasks in multimedia data mining are of visual data classification and clustering in terms of semantics. Usually performance of such classification or clustering systems may not be favorable due to the use of low-level features for image representation, and also some improper similarity metrics for measuring the closeness between multimedia objects as well. This paper considers a problem of modeling similarity for semantic image clustering. A collection of semantic images and feed-forward neural networks are used to approximate a characteristic function of equivalence classes, which is termed as a learning pseudo metric (LPM). Empirical criteria on evaluating the goodness of the LPM are established. A LPM based k-Mean rule is then employed for the semantic image clustering practice, where two impurity indices, classification performance and robustness are used for performance evaluation. An artificial image database with 11 semantics is employed for our simulation studies. Results demonstrate the merits and usefulness of our proposed techniques for multimedia data mining.

The theme of the MDMKDD'13 workshop is\mining large scale rich content in a networked society." The workshop brought together experts in computational analysis on digital media content and multimedia databases, as well as knowledge engineers and domain experts from different applied disciplines with potential in multimedia data mining. The new theme of this year's workshop highlights the integration of multimedia in people's daily lives. With the rapid growth of mobile devices and personalized data, we wanted the workshop to focus on mining large scale rich content which is abundant in today's networked societies. Vast amount of multimedia data are produced, shared, and accessed everyday in various social platforms. These multimedia objects (images, videos, texts, tags, etc.) represent rich, multifaceted recordings of human behavior in the networked society, which lead to a range of social applications such as consumer behavior forecasting, social driven advertising / business, local knowledge discovery (e.g., for tourism or shopping), detection of emergent news events and trends, and so on. In addition to techniques for mining single media items, all of these applications require new methods for extracting robust features and discovering stable relationships among different media modalities and the users, in a dynamic, social context rich, and likely noisy environment. Multimedia is designed to stimulate the human senses beyond text. Using the rich content in multiple media types, we can capture the full experience of an event. Mining the data within media rich platforms and immersive environments (such as online communities, blogs, social networks, and virtual worlds) integrates the digital and physical experiences and information. The aim of the MDMKDD'13 workshop is to explore the role that multimedia data mining can play to enhance virtual society experiences.

The paper describes the design concept and the implementation of the imaging database, Online Image Delivery System (OLIDS). We propose a new framework to support vertically parallel image processing in multimedia database systems. We present a generic and customizable framework for specifying an image query execution plan that permits the DBMS to optimize and execute parallel image processing. Moreover, it is important to provide the multimedia applications with the capability for customizing the information server according to specific requirements. This customization capability also provides both challenges and opportunities in building high performance multimedia and image systems. Applications such as Active hypermedia project or MODOS (Museum project between LRMF (France), NMWA and NACSIS (Japan)) demonstrate the usefulness of the customizable feature of the application oriented DBMS, Phasme, a microkernel database system, tested and extended inside NACSIS.

In recent years, the rapid expansion of multimedia applications, partly due to the exponential growth of the Internet, has proliferated over the daily life of computer users (Yang & Hurson, 2006). The integration of wireless communication, pervasive computing, and ubiquitous data processing with multimedia database systems has enabled the connection and fusion of distributed multimedia data sources. In addition, the emerging applications, such as smart classroom, digital library, habitat/environment surveillance, traffic monitoring, and battlefield sensing, have provided increasing motivation for conducting research on multimedia content representation, data delivery and dissemination, data fusion and analysis, and contentbased retrieval. Consequently, research on multimedia technologies is of increasing importance in computer society. In contrast with traditional text-based systems, multimedia applications usually incorporate much more powerful descriptions of human thought—video, audio, and images (Karpouzis, Raouzaiou, Tzouveli, Iaonnou, & Kollias, 2003; Liu, Bao, Yu, & Xu, 2005; Yang & Hurson, 2005). Moreover, the large collections of data in multimedia systems make it possible to resolve more complex data operations such as imprecise query or content-based retrieval. For instance, the image database systems may accept an example picture and return the most similar images of the example (Cox, Miller, & Minka, 2000; Hsu, Chua, & Pung, 2000; Huang, Chang, & Huang, 2003). However, the conveniences of multimedia applications come with challenges to the existing data management schemes: • Efficiency: Multimedia applications generally require more resources; however, the storage space and processing power are limited in many practical systems, for example, mobile devices and wireless networks (Yang & Hurson, 2005). Due to the large data volume and complicated operations of multimedia applications, new methods are needed to facilitate efficient representation, retrieval, and processing of multimedia data while considering the technical constraints. • Semantic Gap: There is a gap between user perception of multimedia entities and physical representation/access mechanism of multimedia data. Users often browse and desire to access multimedia data at the object level (“entities” such as human beings, animals, or buildings). However, the existing multimedia retrieval systems tend to access multimedia data based on their lower-level features (“characteristics” such as color patterns and textures), with little regard to combining these features into data objects. This representation gap often leads to higher processing cost and unexpected retrieval results. The representation of multimedia data according to human’s perspective is one of the focuses in recent research activities; however, few existing systems provide automated identification or classification of objects from general multimedia collections. • Heterogeneity: The collections of multimedia data are often diverse and poorly indexed. In a distributed environment, because of the autonomy and heterogeneity of data sources, multimedia data objects are often represented in heterogeneous formats. The difference in data formats further leads to the difficulty of incorporating multimedia data objects under a unique indexing framework. • Semantic Unawareness: The present research on content-based multimedia retrieval is based on feature vectors—features are extracted from audio/video streams or image pixels, empirically or heuristically, and combined into vectors according to the application criteria. Because of the application-specific multimedia formats, the feature-based paradigm lacks scalability and accuracy.

The design patterns from the widely used Gang of Four book provide flexible, reusable designs that can be employed in many disparate application domains. Each of these well-known patterns employs important object-oriented techniques, including encapsulation and inheritance, and polymorphism (both compile time and runtime), as well as object instantiation and deletion. Because these patterns employ the object-oriented techniques in standard ways, they can serve as a good mechanism for comparing different object-oriented languages and for comparing the compilers for these languages. The authors compare different modern object-oriented languages and compilers using the standard Gang of Four design patterns as benchmarks. The languages compared were C++ , Java, and the new .NET language, C#. The performances of C+ + and Java were examined on both Linux and Microsoft Windows XP, whereas the performance of C# (a Microsoft programming language) was examined only on Microsoft Windows XP. Of particular interest was the performance of C# , as it is the newest language of the three, as well as the performance of all three object-oriented programming languages in a standard PC-based environment, with the commonly used PC operating systems, Linux and Microsoft Windows. Statistical techniques were employed in order to achieve strong language comparisons.