Session details: AdMIRe'12 workshop 6

Purpose– A need to renew music-related information notions arises from both information-seeking models and literature of musical semiotics. The purpose of this paper is to create a music information typology, which aims at facilitating the examination of music information types at varying levels of abstraction in the context of information seeking.Design/methodology/approach– Literature of musical semiotics and information seeking are juxtaposed to develop a novel approach to music-related information. The grounding concepts are Bruner’s enactive, iconic and symbolic modes of representation. The modes of representation offer a universal scheme of knowledge that is applied to the domain of music by defining their content through Tarasti’sTheory of Musical Semiotics.Findings– This conceptual paper results in a music information typology ranging from the enactive music information representations to the abstract ones as follows. Music making as the first mode of enactive representations; music listening as the second mode of enactive representations; iconic representations of music; technological models of music as the first mode of symbolic representations; and ideological models of music as the second mode of symbolic representations.Originality/value– The present paper develops a music information typology that encompasses broadly different music information facets by categorizing music information sources according to their level of abstraction. When applied into empirical research, the typology opens a new window into the perceived roles of music information types in the context of information seeking.

A curious divide characterizes the usage of audio descriptors for timbre research in music information research (MIR) and music psychology. While MIR uses a multitude of audio descriptors for tasks such as automatic instrument classification, only a highly constrained set is used to describe the physical correlates of timbre perception in parts of music psychology. We argue that this gap is not coincidental and results from the differences in the two fields’ methodologies, their epistemic groundwork, and research goals. This paper lays out perspectives on the emergence of the divide and reviews studies in both fields with regards to divergences in research methods and goals. We discuss new representations for spectro-temporal modulations in MIR and psychology, and compare approaches to spectral envelope description in depth. Finally, we will propose that the interdisciplinary discourse on the computational modelling of music requires negotiations about the roles of scientific evaluation criteria.

This article comprehensively addresses the problem of similarity measurement between music artists via text-based features extracted from Web pages. To this end, we present a thorough evaluation of different term-weighting strategies, normalization methods, aggregation functions, and similarity measurement techniques. In large-scale genre classification experiments carried out on real-world artist collections, we analyze several thousand combinations of settings/parameters that influence the similarity calculation process, and investigate in which way they impact the quality of the similarity estimates. Accurate similarity measures for music are vital for many applications, such as automated playlist generation, music recommender systems, music information systems, or intelligent user interfaces to access music collections by means beyond text-based browsing. Therefore, by exhaustively analyzing the potential of text-based features derived from artist-related Web pages, this article constitutes an important contribution to context-based music information research.

Online learning has altered music education remarkable in the last decade. Large and increasing amount of music performing learners participate in online music learning courses due to the easy- accessibility and boundless of time-space constraints. However, online music learning cannot be extended to a large-scale unless there is an automatic system to provide assessment feedback for the student music performances. Singing can be considered the most basic form of music perform- ing. The critical role of singing played in music education cannot be overemphasized. Automatic singing voice assessment, as an important task in Music Information Research (MIR), aims to extract musically meaningful information and measure the quality of learners’ singing voice. Singing correctness and quality is culture-specific and its assessment requires culture-aware methodologies. Jingju (also known as Beijing opera) music is one of the representative music traditions in China and has spread to many places in the world where there are Chinese communities. The Chinese tonal languages and the strict conventions in oral transmission adopted by jingju singing training pose unique challenges that have not been addressed by the cur- rent MIR research, which motivates us to select it as the major mu- sic tradition for this dissertation. Our goal is to tackle unexplored automatic singing voice assessment problems in jingju music, to make the current eurogeneric assessment approaches more culture- aware, and in return, to develop new assessment approaches which can be generalized to other music traditions. This dissertation aims to develop data-driven audio signal processing and machine learning (deep learning) models for automatic singing voice assessment in audio collections of jingju music. We identify challenges and opportunities, and present several research tasks relevant to automatic singing voice assessment of jingju mu- sic. Data-driven computational approaches require well-organized data for model training and testing, and we report the process of curating the data collections (audio and editorial metadata) in detail. We then focus on the research topics of automatic syllable and phoneme segmentation, automatic mispronunciation detection and automatic pronunciation similarity measurement in jingju music. It is extremely demanding in jingju singing training that students have to pronounce each singing syllable correctly and to reproduce the teacher’s reference pronunciation quality. Automatic syllable and phoneme segmentation, as a preliminary step for the assessment, aims to divide the singing audio stream into finer granularities – syllable and phoneme. The proposed method adopts deep learning models to calculate syllable and phoneme onset probabilities, and achieves a state of the art segmentation accuracy by incorporating side information – syllable and phoneme durations estimated from musical scores, into the algorithm. Jingju singing uses a unique pronunciation system which is a mixture of several Chinese language dialects. This pronunciation system contains various special pronounced syllables which are not included in standard Mandarin. A crucial step in jingju singing training is to pronounce these special syllables correctly. We approach the problem of automatic mispronunciation detection for special pronunciation syllables using a deep learning-based classification method by which the student’s interpretation of a special pronounced syllable segment is assessed. The proposed method shows a great potential by comparing with the existing forced alignment-based approach, indicates its validity in pronunciation correctness assessment. The strict oral transmission convention in jingju singing teaching requires that students accurately reproduce the teacher’s reference pronunciation at phoneme level. Hence, the proposed assessment method needs to be able to measure the pronunciation similarity between teacher’s and student’s corresponding phonemes. Acous- tic phoneme embeddings learned by deep learning models can cap- ture the pronunciation nuance and convert variable-length phoneme segment into the fixed-length vector, and consequently to facilitate the pronunciation similarity measurement. The technologies developed from the work of this dissertation are a part of the comprehensive toolset within the CompMusic project, aimed at enriching the online learning experience for jingju music singing. The data and methodologies should also be contributed to computational musicology research and other MIR or speech tasks related to automatic voice assessment.

This paper aims at leveraging microblogs to address two challenges in music information retrieval (MIR), similarity estimation between music artists and inferring typical listening patterns at different granularity levels (city, country, global). From two collections of several million microblogs, which we gathered over ten months, music-related information is extracted and statistically analyzed. We propose and evaluate four co-occurrence-based methods to compute artist similarity scores. Moreover, we derive and analyze culture-specific music listening patterns to investigate the diversity of listening behavior around the world.

In the audio domain, the disciplines of signal processing, machine learning, psychoacoustics, information theory and library science have merged into the field of Music Information Retrieval (Music-IR). Music-IR researchers attempt to extract high level information from music like pitch, meter, genre, rhythm and timbre directly from audio signals as well as semantic meta-data over a wide variety of sources. This information is then used to organize and process data for large scale retrieval and novel interfaces. For creating musical content, access to hardware and software tools for producing music has become commonplace in the digital landscape. While the means to produce music have become widely available, significant time must be invested to attain professional results. Mixing multi-channel audio requires techniques and training far beyond the knowledge of the average music software user. As a result, there is significant growth and development in intelligent signal processing for audio, an emergent field combining audio signal processing and machine learning for producing music. This work focuses on methods for modeling and analyzing multi-timbral musical instrument mixtures and performing automated processing techniques to improve audio quality based on quantitative and qualitative measures. The main contributions of the work involve training models to predict mixing parameters for multi-channel audio sources and developing new methods to model the component interactions of individual timbres to an overall mixture. Linear dynamical systems (LDS) are shown to be capable of learning the relative contributions of individual instruments to re-create a commercial recording based on acoustic features extracted directly from audio. Variations in the model topology are explored to make it applicable to a more diverse range of input sources and improve performance. An exploration of relevant features for modeling timbre and identifying instruments is performed. Using various basis decomposition techniques, audio examples are reconstructed and analyzed in a perceptual listening test to evaluate their ability to capture salient aspects of timbre. These tests show that a 2-D decomposition is able to capture much more perceptually relevant information with regard to the temporal evolution of the frequency spectrum of a set of audio examples. The results indicate that joint modeling of frequencies and their evolution is essential for capturing higher level concepts in audio that we desire to leverage in automated systems.

GiantSteps is an EU-funded project that aims at developing the next generation of music composition tools for the creative industries by bridging the gap between music information research and end users' requirements. An important component of the project is the extraction of musical and application-targeted knowledge from social media and web resources. In this paper, we sketch potential ways to exploit social media and web data for the tasks of music analysis, creation, and algorithm evaluation.

Music genre prediction is one of the topics that digital music processing is interested in. In this study, acoustic features of music have been extracted by using digital signal processing techniques and then music genre classification and music recommendations have been made by using machine learning methods. In addition, convolutional neural networks, which are deep learning methods, were used for genre classification and music recommendation and performance comparison of the obtained results has been. In the study, GTZAN database has been used and the highest success was obtained with the SVM algorithm.

In music informatics, there has been increasing attention to relative similarity as it plays a central role in music retrieval, recommendation, and musicology. Most approaches for relative similarity are based on distance metric learning, in which similarity relationship is modelled by a parameterised distance function. Normally, these parameters can be learned by solving a constrained optimisation problem using kernel-based methods. In this paper, we study the use of restricted Boltzmann machines (RBMs) in similarity modelling. We take advantage of RBM as a probabilistic neural network to assign a true hypothesis “x is more similar to y than to z” with a higher probability. Such model can be trained by maximising the true hypotheses while, at the same time, minimising the false hypotheses using a stochastic method. Alternatively, we show that learning similarity relations can be done deterministically by minimising the free energy function of a bipolar RBM or using a classification approach. In the experiments, we evaluate our proposed approaches on music scripts extracted from MagnaTagATune dataset. The results show that an energy-based optimisation approach with bipolar RBM can achieve better performance than other methods, including support vector machine and machine learning rank which are the state-of-the-art for this dataset.

Music Information Retrieval (MIR) is an interdisciplinary research area that covers automated extraction of information from audio signals, music databases and services enabling the indexed information searching. In the early stages the primary focus of MIR was on music information through Query-by-Humming (QBH) applications, i.e. on identifying a piece of music by singing (singing/whistling), while more advanced implementations supporting Query-by-Example (QBE) searching resulted in names of audio tracks, song identification, etc. Both QBH and QBE required several steps, among others an optimized signal parametrization and the soft computing approach. Nowadays, MIR is associated with research based on the content analysis that is related to the retrieval of a musical style, genre or music referring to mood or emotions. Even though, this type of music retrieval called Query-by-Category still needs feature extraction and parametrization optimizing, but in this case search of global online music systems and services applications with their millions of users is based on statistical measures. The paper presents details concerning MIR background and answers a question concerning usage of soft computing versus statistics, namely: why and when each of them should be employed.

This paper proposes methods for information processing of audio streams using methods of information geometry. We lay the theoretical groundwork for a framework allowing the treatment of signal information as information entities, suitable for similarity and symbolic computing on audio signals. The theoretical basis of this paper is based on the information geometry of statistical structures representing audio spectrum features, and specifically through the bijection between the generic families of Bregman divergences and that of exponential distributions. The proposed framework, called Music Information Geometry, allows online segmentation of audio streams to metric balls where each ball represents a quasi-stationary continuous chunk of audio, and discusses methods to qualify and quantify information between entities for similarity computing. We define an information geometry that approximates a similarity metric space, redefine general notions in music information retrieval such as similarity between entities, and address methods for dealing with nonstationarity of audio signals. We demonstrate the framework on two sample applications for online audio structure discovery and audio matching.

In this paper we give an overview of the Foafing the Music system. The system uses the Friend of a Friend (FOAF) and RDF Site Summary (RSS) vocabularies for recommending music to a user, depending on the user’s musical tastes and listening habits. Music information (new album releases, podcast sessions, audio from MP3 blogs, related artists’ news and upcoming gigs) is gathered from thousands of RSS feeds. The presented system provides music discovery by means of: user profiling (defined in the user’s FOAF description), context based information (extracted from music related RSS feeds) and content based descriptions (extracted from the audio itself), based on a common ontology (OWL DL) that describes the music domain. The system is available at: http://foafing-the-music.iua.upf.edu

Foafing the Music: Bridging the Semantic Gap in Music Recommendation

Quantification of stylistic differences between musical artists is of academic interest to the music community, and is also useful for other applications such as music information retrieval and recommendation systems. Information about stylistic differences can be obtained by comparing the performances of different artists across common musical pieces. In this article, we develop a statistical methodology for identifying and quantifying systematic stylistic differences among artists that are consistent across audio recordings of a common set of pieces, in terms of several musical features. Our focus is on a comparison of ten different orchestras, based on data from audio recordings of the nine Beethoven symphonies. As generative or fully parametric models of raw audio data can be highly complex, and more complex than necessary for our goal of identifying differences between orchestras, we propose to reduce the data from a set of audio recordings down to pairwise distances between orchestras, based on different musical characteristics of the recordings, such as tempo, dynamics, and timbre. For each of these characteristics, we obtain multiple pairwise distance matrices, one for each movement of each symphony. We develop a hierarchical multidimensional scaling (HMDS) model to identify and quantify systematic differences between orchestras in terms of these three musical characteristics, and interpret the results in the context of known qualitative information about the orchestras. This methodology is able to recover several expected systematic similarities between orchestras, as well as to identify some more novel results. For example, we find that modern recordings exhibit a high degree of similarity to each other, as compared to older recordings.

In this research, the authors propose an intelligent system that can recommend songs to user according to his choice. They predict the next song a user might prefer to listen based on their previous listening patterns, currently played songs and similar music based on music data. To calculate music similarity the authors used a Matlab toolbox that considers audio signals. They used association rule mining to find users' listening patterns and predict the next song the user might prefer. As they propose a music discovery service as well, the authors use the information of music listening pattern and music data similarity to recommend a new song. Later in result section, they replaced the audio based similarity with last.fm api for similar song listing and analyzed the behaviour of their system with the new list of songs.