Content-Based Retrieval in Digital Music Libraries

In recent years we have seen the development of several novel content-based retrieval (CBR) systems that have had success by focusing on a specific domain and exploiting domain-speci c information. CBR systems allow users to query a media database using item content as opposed submitting a text-based query. In many CBR applications, the input to the search process is a complicated object that may be composed of several constituent parts. The proposed approach performs CBR queries by decomposing a complex query into several heterogeneous queries. We have developed a multi-index, multi-object CBR framework for geospatial imagery retrieval that extracts features specifically developed for high-resolution commercial satellite imagery. The results of these queries will be spatially summarized for a user based on both retrieval score and spatial distance. This allows results to be presented in a logical manner to allow for more efficient interpretation by the user. Further, we propose to develop an additional search capability that allows for multi-object searches by spatial configuration rather than simply by object-to-object correspondence. Additionally, to confront situations where a user has determined that certain search results are not relevant, we will provide online and memory-based relevance feedback algorithms for use with multi-index, multi-object CBR systems. The experimental results demonstrate the efficiency and accuracy of the proposed methods; moreover, through the fusion of multi-index and multi-object search techniques, we are able to construct new, sophisticated query mechanisms.

This research investigated the strategic development of a large-scale transdisciplinary area, named intelligent infrastructure for human-centred communities, at Virginia Tech. Within such development, this study explored the future vision and anticipated scenarios of data infrastructure and digital libraries for smart community development. It draws upon the mixed-methods approach combining ethnographic participant observation, document analysis and semi-structured interviews. Grounded in socio-technical framework and rooted in empirical methods, this research produces results that augment design thinking and visioning practice for digital data libraries beyond traditional boundaries. The findings reveal the emerging scenarios around complex adaptive systems, intelligent data infrastructure and future digital libraries all in the context of building infrastructure for human-centred communities. Situated in this advancing reality, the results further discuss the next-generation data and information user experience, smart infrastructure data environment and future library capabilities. The article concludes that a smart library system, whether in its conceptual form of a ‘digital octopus’ or a ‘smart village data hub’ or an ‘intelligent virtual assistant’, will provide intelligence in data gathering, processing, summarising, communication and recommendations. By delivering unified and personalised data solutions, it will offer an end-to-end seamless experience for users throughout their journey of knowledge pursuit.

The design and implementation of Harbin Institute of Technology--Digital Music Library (HIT-DML) is presented in this paper. Firstly, a novel framework, a music data model, and a query language are proposed as the theoretical foundation of the library. Secondly, music computing algorithms used in the library for feature extracting and matching are described. In addition, indices are introduced for both mining themes of music objects and accelerating content-based information retrieval. Finally, experimental results on the indices and the current development of the library are provided. HIT-DML is distinguished by the following points. First, it is inherently based on database systems, and combines database technologies with multimedia technologies seamlessly. Musical data are structurally stored. Second, it has a solid theoretical foundation, from framework and data model to query language. Last, it can retrieve musical information based on content against different kinds of musical instruments. The indices used, also power the library.

As the size of multimedia database grows, it becomes impractical to manually annotate all contents and attributes of the media, and the difficulty in finding desired information increases. To copy with these challenges, content-based multimedia retrieval systems have been developed for various applications. The chapter not only provides a conceptual architecture for the design of content-based retrieval system, but also discusses essential components of retrieval system and their research issues, including feature extraction and representation, dimension reduction of feature vector, indexing, and query specifications. As content-based multimedia retrieval is a young research field and there exists many challenging research problems, this chapter also addresses several research issues for the future research.

Current Music Information Retrieval (MIR) systems focus on melody based retrieval with some tolerance for user errors in the melody specification. The system described here presents a novel method for theme retrieval: A theme is described as a list of musical events, containing melody and harmony features, which must be presented in a given order and within a given time frame. The system retrieves musical phrases that fit the description. A system of this type could serve musicians and listeners who wish to discover thematically similar phrases in music digital libraries. The prototype and underlying model have been tested on midi sequences of music by W.A. Mozart and have shown good performance results.

In recent years, the rapid expansion of multimedia applications, partly due to the exponential growth of the Internet, has proliferated over the daily life of computer users (Yang & Hurson, 2006). The integration of wireless communication, pervasive computing, and ubiquitous data processing with multimedia database systems has enabled the connection and fusion of distributed multimedia data sources. In addition, the emerging applications, such as smart classroom, digital library, habitat/environment surveillance, traffic monitoring, and battlefield sensing, have provided increasing motivation for conducting research on multimedia content representation, data delivery and dissemination, data fusion and analysis, and contentbased retrieval. Consequently, research on multimedia technologies is of increasing importance in computer society. In contrast with traditional text-based systems, multimedia applications usually incorporate much more powerful descriptions of human thought—video, audio, and images (Karpouzis, Raouzaiou, Tzouveli, Iaonnou, & Kollias, 2003; Liu, Bao, Yu, & Xu, 2005; Yang & Hurson, 2005). Moreover, the large collections of data in multimedia systems make it possible to resolve more complex data operations such as imprecise query or content-based retrieval. For instance, the image database systems may accept an example picture and return the most similar images of the example (Cox, Miller, & Minka, 2000; Hsu, Chua, & Pung, 2000; Huang, Chang, & Huang, 2003). However, the conveniences of multimedia applications come with challenges to the existing data management schemes: • Efficiency: Multimedia applications generally require more resources; however, the storage space and processing power are limited in many practical systems, for example, mobile devices and wireless networks (Yang & Hurson, 2005). Due to the large data volume and complicated operations of multimedia applications, new methods are needed to facilitate efficient representation, retrieval, and processing of multimedia data while considering the technical constraints. • Semantic Gap: There is a gap between user perception of multimedia entities and physical representation/access mechanism of multimedia data. Users often browse and desire to access multimedia data at the object level (“entities” such as human beings, animals, or buildings). However, the existing multimedia retrieval systems tend to access multimedia data based on their lower-level features (“characteristics” such as color patterns and textures), with little regard to combining these features into data objects. This representation gap often leads to higher processing cost and unexpected retrieval results. The representation of multimedia data according to human’s perspective is one of the focuses in recent research activities; however, few existing systems provide automated identification or classification of objects from general multimedia collections. • Heterogeneity: The collections of multimedia data are often diverse and poorly indexed. In a distributed environment, because of the autonomy and heterogeneity of data sources, multimedia data objects are often represented in heterogeneous formats. The difference in data formats further leads to the difficulty of incorporating multimedia data objects under a unique indexing framework. • Semantic Unawareness: The present research on content-based multimedia retrieval is based on feature vectors—features are extracted from audio/video streams or image pixels, empirically or heuristically, and combined into vectors according to the application criteria. Because of the application-specific multimedia formats, the feature-based paradigm lacks scalability and accuracy.

The Information Age is fully upon us. A recent article noted that there are perhaps 50 million people using the Internet on a regular basis, and That “the current growth rate is about 15% per month (!) and this could well continue until almost all of those in the ‘developed world’ are connected” [FM94, p. 30]. In addition, the digital domain consists not only of text but increasingly of other media representations, from graphics images to audio to motion video. As the amount of information and number of users exponentially escalate, more attention focuses on the basic problems of information management: How do you digitize information? How can you then visualize it and find what you need? How do you use and manipulate it effectively? How is it stored and managed? The proliferation of technical articles and special issues addressing these questions underscore their importance; see for example the special issue on Content-based retrieval [Nar95] or digital libraries [F+95]. This chapter will survey some of that work, especially that which relates to the treatment of video and the use of digital video libraries For education.

In this paper, we develop the concept of descriptors based on perceptual-level motion features such as time-to-collision, shot transition and temporal motion and it is shown that by including them the representational level of the video classes is significantly enhanced, e.g. violence could be detected. The temporal context cues, which had been largely neglected by present content-based retrieval (CBR) systems, are integrated into the framework. A dynamic Bayesian framework for the CBR systems which can learn the temporal structure through the fusion of all the features is designed The experimental results for more than 4 hours of videos are presented for a number of key applications like sequence identifier, highlight extraction for sports, and detecting climax or violence.

Digital libraries are a powerful tool for preserving and disseminating information in a technology-driven world. Providing options for housing non-traditional information, such as three-dimensional (3D) data, digital libraries also address barriers (e.g., geographic location and physical space) that affect traditional libraries. While there are certainly still barriers that may prevent users from accessing and utilizing digital content, digital libraries offer ways of preserving, housing, and disseminating non-traditional information that physical libraries cannot. Within the anthropological community, digital libraries can serve as a method of housing 3D fossil data. There are unique challenges and barriers associated with studying fossils that digital libraries help to break down. Through the preservation and dissemination of 3D fossil data, researchers across the globe can work with fossil data without the concerns associated with handling fragile fossils. However, such 3D digital libraries are not without challenges. While there are clear standards and metadata schemas (e.g., RDA and DCMI) for digital libraries containing traditional materials, no such standardization exists for digital libraries containing 3D data. As such, metadata requirements and elements for existing 3D digital libraries are inconsistent and may lack important details about the data, which can compromise functionality. This paper explores trends and challenges that digital libraries containing 3D data experience and highlights best-practice solutions for future 3D digital libraries to overcome these challenges.

Tsinghua University Architecture Digital Library (THADL) is constructing as a small but comprehensive prototype system. The aims of THADL are: 1) to study the theories, methods and key techniques in digital library, 2) to provide learning environment and mechanism for university education, and 3) to pave the way for constructing future large-scale digital library. In this paper, we propose the architecture for THADL that assures acquisition, organization, management, retrieval, and distribution of materials on Chinese ancient architecture. We discuss the information model of THADL and implement a Web-based courseware environment for university education via Internet. Finally, we establish the metadata specification--THADL Metadata for marking and describing the digital materials in THADL.

In this paper, the content-based retrieval with audio examples based on MP3 digital music archive for one singer is considered. In our proposed method, the Sub-Band Coefficients (SBC) in a MP3 frame is used for feature extraction. Both Quantization-Tree indexing (QT) approach and the Melody-Line Tracking (MLT) approach are proposed for indexing MP3 objects. Finally, a two-stage comparison method is used to measure the similarity between MP3 objects. We also employ the measurement of the similarity between query sample and database items for a singer. Evaluations on a content-based MP3 retrieval system are performed. Experimental results show that our proposed approach can achieve good performance and high accuracy.

Large image databases have been designed and used in various applications and in content-based retrieval systems. Medical Imaging modalities have been developed in the past each uniquely designed to specific imaging modality based on the image pathobiological medium. This paper presents an approach to improve the process of image storage and retrieval based on image data representation model that integrates both metadata and content description. The low level features and the labels associated with the salient objects extracted from the mages are stored in an ORDBMS (Object Relational Database Management systems) using Oracle. Our technique, which combines the image descriptors obtained by a unified approach phases of image segmentation and annotation, has a good time complexity and the experimental results on several image datasets.

This paper deals with the Music/Speech classification problem, starting from a set of features extracted directly from compressed audio data. The proposed classification system is able to label audio sequences stored as compressed MPEG layer III files. Decoding and analyzing in a unique stage is a fundamental tool for audio streaming applications, such as real time classification. Moreover, the techniques described herein provide useful tools in the management (data tagging, summarization, etc.) of a digital music library. The adopted set of short-time features are computed from the spectral information available in the decoding stage. In this paper, we show that for the classification problem at hand this set of features is redundant and can be dramatically pruned. To this aim we used an optimization strategy based on principal component analysis and genetic algorithms. The results show a very interesting classification accuracy using just one short-time feature.

Musicologists and musicians often would like to search by keys in a digital music library. In this paper, we introduce a new key-finding algorithm that can be applied to music in both symbolic and audio formats. The algorithm, which is based on a Hidden Markov Model (HMM), provides two stages of key-finding output; the first one referring to local keys and the second one to the global key. We describe the input, the two output stages, and the parameters of the model. In particular, we describe two configurable parameters, the transition probability distributions, which are based on a matrix of neighbouring keys, and the emission probability distributions, which make use of established key profiles. We discuss the local key-finding capabilities of the algorithm, presenting an example analysis of the Prelude Op. 28 No. 20 in C minor by Chopin, showing the local key regions obtained using different key profiles. We evaluate the global key-finding capabilities of the model, using an existing dataset and six well-known key profiles as different model parameters. Since different key profiles will tend to err or misclassify in different ways and across different pieces, we train an ensemble method with the predictions from all the key profiles (6) through our model. We show that the ensemble method achieves state-of-the-art performance for major and overall keys, however, it still underperforms the state-of-the-art for minor keys.

In this paper, a prototype system is described for the management and content-based retrieval of defect images in huge image databases. This is a real problem in surface inspection applications, since modern inspection systems may produce up to thousands of defect images in a day. The retrieval is based on shape and internal structure characteristics of defects, so neither manual labeling nor annotation needs to be made. We are using a noncommercial, generic content-based image retrieval (CBIR) system called PicSOM that is modified to fit the special requirements of our application. The results of experiments show that the prototype system works fast with good retrieval results.