Music Information Retrieval: A Window into the Needs and Challenges

Classification of musical audio signals according to expressed mood or emotion has evident applications to content-based music retrieval in large databases. Wrapper selection is a dimension reduction method that has been proposed for improving classification performance. However, the technique is prone to lead to overfitting of the training data, which decreases the generalizability of the obtained results. We claim that previous attempts to apply wrapper selection in the field of music information retrieval (MIR) have led to disputable conclusions about the used methods due to inadequate analysis frameworks, indicative of overfitting, and biased results. This paper presents a framework based on cross-indexing for obtaining realistic performance estimate of wrapper selection by taking into account the simplicity and generalizability of the classification models. The framework is applied on sets of film soundtrack excerpts that are consensually associated with particular basic emotions, comparing Naive Bayes, k-NN, and SVM classifiers using both forward selection (FS) and backward elimination (BE). K-NN with BE yields the most promising results—56.5% accuracy with only four features. The most useful feature subset for k-NN contains mode majorness and key clarity, combined with dynamical, rhythmical, and structural features.

Melody extraction from polyphonic audio, which consists of number of different instruments, is one of the most challenging tasks in the field of Music Information Retrieval (MIR). This paper aims at implementation of extracting melody using two-way mismatch dual pitch tracking (TWMDPT) and harmonic cluster tracking (HCT) algorithms. In this work, melody of singing voice is estimated in presence of multi-accompaniment. In the first method temporal features of voice harmonics are used to find the voice pitch and it is based on salience function. The second method depends upon strong higher harmonics for robustness against distortion by the first harmonic due to low frequency accompaniments. Performance of these algorithms is evaluated on MIR-1k database and accuracy is estimated using Raw Pitch Accuracy (RPA), Raw Chroma Accuracy (RCA) and timbre feature.

A frequent problem when dealing with audio classification tasks is the scarcity of suitable training data. This work investigates ways of mitigating this problem by applying transfer learning techniques to neural network architectures for several classification tasks from the field of Music Information Retrieval (MIR). First, three state-of-the-art architectures are trained and evaluated with several datasets for the task of speech/music classification. Second, feature representations or embeddings are extracted from the trained networks to classify new tasks with unseen data. The effect of pre-training with respect to the similarity of the source and target tasks are investigated in the context of transfer learning, as well as different fine-tuning strategies.

There are two important research topics in the field of Music Information Retrieval (MIR). One is how to improve the robustness of features and the other is how to speed up the retrieval process. This paper improved the algorithms which proposed by Shazam company in these two aspects. We improve the robustness of the system by a new audio finger-printing extraction using computer graphics, and the system can recognize the recordings which get in complex environment accurately. On the other hand, we propose a recursive search algorithm based on the confidence measure to improve the retrieval speed. Quantitative analysis of the opposite experiment verifies the improvement in the retrieval speed and accuracy.

Technology has developed a lot over the last decades and has made a profound impact in almost every field. The field of Music Information Retrieval (MIR) has not been an exception to this as well, one of its most promising applications being Automatic Music Transcription (AMT). It is important to identify the active regions of various Instruments in a piece before transcription and the challenge elevates even more when the audio clips are contaminated with noise. MISNA (Musical Instrument Segregation from Noisy Clips) is a system proposed towards the identification of isolated Instruments from noisy clips which can aid towards AMT in noisy environments. The system works using statistical features (LPCC-S) derived from raw Linear Predictive Cepstral Coefficient values on very short clips of lengths 1 and 2 seconds. The system has been tested for various SNR scenarios and highest accuracies of 98.63% and 97.42% for Individual Instruments and Instrument Family identification has been obtained with the aid of Extreme Learning based classifier for a highest of 2626 clips.

Music is a complicated form of communication, where creators and cultures communicate and expose their individualities. Thanks to music digitalization, recommendation systems and other online services have become indispensable in the field of Music Information Retrieval (MIR). Classification of music is essential for music recommendation systems. In this paper, we propose an approach for finding similarity between music. Our approach is based on mid-level attributes like pitch, midi value, interval, contour, and duration, and applying text-based classification techniques. Performance evaluation has been done using the accuracy score of scikit-learn. As a preliminary study, our system first predicted jazz, metal, and ragtime for western music. The genre prediction system has been tested on 476 music files with a maximum accuracy of 95.8% across different n-grams. Then, we have analyzed and classified the Indian classical Carnatic music based on their raga. Our system has predicted Sankarabharam, Mohanam, and Sindhubhairavi ragas. The raga prediction system was tested on 68 music files with a maximum accuracy of 90.14% across different n-grams.

Microblogs are rich sources of information because they provide platforms for users to share their thoughts, news, information, activities, and so on. Twitter is one of the most popular microblogs. Twitter users often use hashtags to mark specific topics and to link them with related tweets. In this study, we investigate the relationship between the music listening behaviors of Twitter users and a popular music ranking service by comparing information extracted from tweets with music-related hashtags and the Billboard chart. We collect users' music listening behavior from Twitter using music-related hashtags (e.g., #nowplaying). We then build a predictive model to forecast the Billboard rankings and hit music. The results show that the numbers of daily tweets about a specific song and artist can be effectively used to predict Billboard rankings and hits. This research suggests that users' music listening behavior on Twitter is highly correlated with general music trends and could play an important role in understanding consumers' music consumption patterns. In addition, we believe that Twitter users' music listening behavior can be applied in the field of Music Information Retrieval (MIR).

In this paper, we propose a unified approach to fast index-based music recognition. As an important area within the field of music information retrieval (MIR), the goal of music recognition is, given a database of musical pieces and a query document, to locate all occurrences of that document within the database, up to certain possible errors. In particular, the identification of the query with regard to the database becomes possible. The approach presented in this paper is based on a general algorithmic framework for searching complex patterns of objects in large databases. We describe how this approach may be applied to two important music recognition tasks: The polyphonic (musical score-based) search in polyphonic score data and the identification of pulse-code modulation audio material from a given acoustic waveform. We give an overview on the various aspects of our technology including fault-tolerant search methods. Several areas of application are suggested. We describe several prototypic systems we have developed for those applications including the notify! and the audentify! systems for score- and waveform-based music recognition, respectively.

Genetic algorithms have been introduced to the field of media segmentation including image, video, and also music segmentation since segmentation problems usually have complex fitness landscapes. Music segmentation can provide insight into the structure of a music composition so it is an important task in music information retrieval (MIR). The authors have already presented the application of genetic algorithms for the music segmentation problem in an earlier paper. This paper focuses on the optimization of parameter settings for genetic algorithms in the field of MIR as well as on the comparison of their results.

Musical themes are essential elements in Western classical music. In this paper, we present the Musical Theme Dataset (MTD), a multimodal dataset inspired by “A Dictionary of Musical Themes” by Barlow and Morgenstern from 1948. For a subset of 2067 themes of the printed book, we created several digital representations of the musical themes. Beyond graphical sheet music, we provide symbolic music encodings, audio snippets of music recordings, alignments between the symbolic and audio representations, as well as detailed metadata on the composer, work, recording, and musical characteristics of the themes. In addition to the data, we also make several parsers and web-based interfaces available to access and explore the different modalities and their relations through visualizations and sonifications. These interfaces also include computational tools, bridging the gap between the original dictionary and music information retrieval (MIR) research. The dataset is of relevance for various subfields and tasks in MIR, such as cross-modal music retrieval, music alignment, optical music recognition, music transcription, and computational musicology.

Towards improving the performance in various music information processing tasks, recent studies exploit different modalities able to capture diverse aspects of music. Such modalities include audio recordings, symbolic music scores, mid-level representations, motion, and gestural data, video recordings, editorial or cultural tags, lyrics and album cover arts. This paper critically reviews the various approaches adopted in Music Information Processing and Retrieval and highlights how multimodal algorithms can help Music Computing applications. First, we categorize the related literature based on the application they address. Subsequently, we analyze existing information fusion approaches, and we conclude with the set of challenges that Music Information Retrieval and Sound and Music Computing research communities should focus in the next years.

Music Information Retrieval (MIR) is an interdisciplinary research area that covers automated extraction of information from audio signals, music databases and services enabling the indexed information searching. In the early stages the primary focus of MIR was on music information through Query-by-Humming (QBH) applications, i.e. on identifying a piece of music by singing (singing/whistling), while more advanced implementations supporting Query-by-Example (QBE) searching resulted in names of audio tracks, song identification, etc. Both QBH and QBE required several steps, among others an optimized signal parametrization and the soft computing approach. Nowadays, MIR is associated with research based on the content analysis that is related to the retrieval of a musical style, genre or music referring to mood or emotions. Even though, this type of music retrieval called Query-by-Category still needs feature extraction and parametrization optimizing, but in this case search of global online music systems and services applications with their millions of users is based on statistical measures. The paper presents details concerning MIR background and answers a question concerning usage of soft computing versus statistics, namely: why and when each of them should be employed.

As an important research project in the field of music information retrieval, music recognition has received high attention from researchers. Especially with the continuous progress of music art cause in China, more and more music resources are born. In this environment, people are very urgent to find a more efficient music recognition algorithm. In order to obtain a more ideal electronic music signal recognition as a result, this paper made analysis, this paper puts forward a new electronic music signal recognition system design and research, first of all human information and twang to build the corresponding model, system optimization and upgrade as the target, and made a rebuilt the twang of information, as well as for proper handling, in addition to the human voice, Experimental results show that compared with common chord recognition algorithms, the 3.5%-12.2% algorithm discussed in this paper has higher recognition accuracy and more application value overall.

In the audio domain, the disciplines of signal processing, machine learning, psychoacoustics, information theory and library science have merged into the field of Music Information Retrieval (Music-IR). Music-IR researchers attempt to extract high level information from music like pitch, meter, genre, rhythm and timbre directly from audio signals as well as semantic meta-data over a wide variety of sources. This information is then used to organize and process data for large scale retrieval and novel interfaces. For creating musical content, access to hardware and software tools for producing music has become commonplace in the digital landscape. While the means to produce music have become widely available, significant time must be invested to attain professional results. Mixing multi-channel audio requires techniques and training far beyond the knowledge of the average music software user. As a result, there is significant growth and development in intelligent signal processing for audio, an emergent field combining audio signal processing and machine learning for producing music. This work focuses on methods for modeling and analyzing multi-timbral musical instrument mixtures and performing automated processing techniques to improve audio quality based on quantitative and qualitative measures. The main contributions of the work involve training models to predict mixing parameters for multi-channel audio sources and developing new methods to model the component interactions of individual timbres to an overall mixture. Linear dynamical systems (LDS) are shown to be capable of learning the relative contributions of individual instruments to re-create a commercial recording based on acoustic features extracted directly from audio. Variations in the model topology are explored to make it applicable to a more diverse range of input sources and improve performance. An exploration of relevant features for modeling timbre and identifying instruments is performed. Using various basis decomposition techniques, audio examples are reconstructed and analyzed in a perceptual listening test to evaluate their ability to capture salient aspects of timbre. These tests show that a 2-D decomposition is able to capture much more perceptually relevant information with regard to the temporal evolution of the frequency spectrum of a set of audio examples. The results indicate that joint modeling of frequencies and their evolution is essential for capturing higher level concepts in audio that we desire to leverage in automated systems.

Music Genre Classification (MGC) automatically categorizes music into different genres based on various musical attributes and features in a small number of music files. This is a crucial problem in the field of music information retrieval as it provides a way to organize and analyse large amounts of music files. MGC can be performed using conventional machine learning algorithms such as SVM, k-nearest neighbours, Decision trees, and neural networks. These algorithms learn to recognize different musical features and attributes to categorize the music files into different genres. The literature shows that the performance of conventional machine learning algorithms is inferior to deep learning algorithms such as CNN, RNN, etc., in various applications. Hence, the CNN algorithm is adapted to implement the classification of music files. This aims to classify music genres using CNN deep learning techniques. The performance of the algorithms for MGC can be evaluated using metrics such as accuracy, precision, recall, and F1-score. Additionally, the impact of different features and algorithms on the performance of MGC can be studied and compared. It has applications in areas such as automated music recommendation systems, music education, and music production. An accuracy of 83% is achieved by using CNN to accomplish the task of MGC.