Research on personalized music education audio signal processing technology based on deep learning

Deep Learning has emerged as a powerful set of computational methods achieving great results in a variety of different tasks. Music signal processing, a field with rich commercial applications, seems to benefit too from this data-driven approach. In this paper a review of the state of the art Deep Learning methods applied on music signal processing is provided. A special focus is given in music information retrieval and music generation. In addition, possible future research directions are discussed.

The digitization, analysis, and processing technology of music signals are the core of digital music technology. There is generally a preprocessing process before the music signal processing. The preprocessing process usually includes antialiasing filtering, digitization, preemphasis, windowing, and framing. Songs in the popular wav format and MP3 format on the Internet are all songs that have been processed by digital technology and do not need to be digitalized. Preprocessing can affect the effectiveness and reliability of the feature parameter extraction of music signals. Since the music signal is a kind of voice signal, the processing of the voice is also applicable to the music signal. In the study of adaptive wave equation inversion, the traditional full-wave equation inversion uses the minimum mean square error between real data and simulated data as the objective function. The gradient direction is determined by the cross-correlation of the back propagation residual wave field and the forward simulation wave field with respect to the second derivative of time. When there is a big gap between the initial model and the formal model, the phenomenon of cycle jumping will inevitably appear. In this paper, adaptive wave equation inversion is used. This method adopts the idea of penalty function and introduces the Wiener filter to establish a dual objective function for the phase difference that appears in the inversion. This article discusses the calculation formulas of the accompanying source, gradient, and iteration step length and uses the conjugate gradient method to iteratively reduce the phase difference. In the test function group and the recorded music signal library, a large number of simulation experiments and comparative analysis of the music signal recognition experiment were performed on the extracted features, which verified the time-frequency analysis performance of the wave equation inversion and the improvement of the decomposition algorithm. The features extracted by the wave equation inversion have a higher recognition rate than the features extracted based on the standard decomposition algorithm, which verifies that the wave equation inversion has a better decomposition ability.

Stacked auto-encoders based visual features for speech/music classification

The following report presents some of the ongoing projects that are taking place in the group’s laboratory. One of the noteable characteristics of this group is the extensive research spectrum, the plurality of research areas that are being studied by it’s members, such as Music Information Retrieval, Signal Processing and New Interfaces for Musical Expression.

Since the age of the Vedas, devotion has remained a key component of Indian music through centuries of changes and foreign influences. The brightest example of this is the Bhakti tradition, a pan-Indian movement (7th-15th Century CE), which integrated poetry and music in the transmission of spiritual and social goals. Devotees often report perception of emotions like devotion, happiness, awe while listening to spiritual music of their own religion and culture. This paper aims to study the acoustical and neuro-cognitive correlates of these emotions for two Indian spiritual music traditions – (a) Sikh Gurbani/ Shabad Kirtan, (b) Bangla Vaishnav Kirtan, both of which emerged from the Bhakti tradition. 5 Punjabi and 5 Bengali speaking participants listened to 3-minute long four songs from these two spiritual music traditions while EEG signals were recorded from each participant along with their emotion responses. Acoustical time series of these music clips and their corresponding EEG signals were analysed using nonlinear DFA technique. DFA Scaling Exponent values were calculated for multiple 30-second EEG segments across the total duration of each music clip to understand the perception and induction process of devotion and happiness in human brain, which is a novel step in the domain of music cognition and signal processing.

Music signal processing is one of the active research area now a days. Identifying the musical instrument form the solo recordings is one of the applications under signal processing. Different types of algorithms have been proposed till now to identify the instrument. For this objective first of all we have to extract the features from sound samples and then we can use them for instrument identification. In this paper we are presenting one of efficient method to extract the features from sound sample that is Mel Frequency Cepsrtal Coefficient (MFCC). These MFCCs are closely related to behavior of human auditory system. While calculating these MFCCs we actually deal with spectral envelop. And hence it gives very distinctive features of the sound samples. So using these features for identification of musical instrument we can increase the accuracy of identification. In our work we have used five instruments and 30 samples of each instrument. In the training database total 90 samples (60% of total samples) and for testing database 60 (40%)samples are used. In the classification phase we have used K-Nearest Neighbor (K-NN) classifier. Proposed system gives the accuracy of 91.66% for Cello, Piano and Trumpet and accuracy of 83.33% for Flute and Violin.

Chroma features are a popular tool in musical signal processing and information retrieval tasks, providing a compact representation of the tonal content of a piece of music. A variety of approaches to chroma estimation have been proposed, most of which rely on the summation of related frequency partials. However, frequency partials may be incorrectly assigned due to the log/linear relationship of frequency and pitch. Variations of chroma employing overtone suppression strategies are found in the literature. We propose a compositional model of chroma, which considers a coarse modelling of the effects of overtones in the expected chroma vectors of single notes. Synthetic chord recognition experiments indicate the usefulness of the proposed approach.

In the last few years various musicological studies on the Byzantine Chant (BC) have been made via computer analysis. Due to the BC vocal nature and the pitch variability of the BC notes during the performance, the value of a music interval between two such notes in a certain BC mode - “echos”, is ambiguous. Because of that pitch variability a new method of BC intervals measurement is proposed, based on statistical processing of the pitch track that can be done after a proper pitch analysis of the BC recordings. The music scale of a particular BC echos is computed from the histogram of the pitch values by applying an algorithm that calculates a Gaussian Mixture Model approximating the music scale of that BC echos. This new method is applied, as an example, to ten BC recordings - performances of the Cherubic Hymn composed in the fourth echos (aghia) by Petros Lampadarios. The recordings come from the DAMASKINOS Corpus of University of Athens and the analysis results are in agreement with both the theoretical values of the Patriarchal Committee 1881 (approximation of up to 1 moria) and the findings in other similar studies. This method of analysis can be a base for both musicological analysis and music signal processing.

Detecting multiple pitches (F0s) and segregating musical instrument lines from monaural recordings of contrapuntal polyphonic music into separate tracks is a difficult problem in music signal processing. Applications include audio-to-MIDI conversion, automatic music transcription, and audio enhancement and transformation. Past attempts at separation have been limited to separating two harmonic signals in a contrapuntal duet (Maher, 1990) or several harmonic signals in a single chord (Virtanen and Klapuri, 2001, 2002). Several researchers have attempted polyphonic pitch detection (Klapuri, 2001; Eggink and Brown, 2004a), predominant melody extraction (Goto, 2001; Marolt, 2004; Eggink and Brown, 2004b), and instrument recognition (Eggink and Brown, 2003). Our solution assumes that each instrument is represented as a time-varying harmonic series and that errors can be corrected using prior knowledge of instrument spectra. Fundamental frequencies (F0s) for each time frame are estimated from input spectral data using an Expectation-Maximization (EM) based algorithm with Gaussian distributions used to represent the harmonic series. Collisions (i.e., overlaps) between instrument harmonics, which frequently occur, are predicted from the estimated F0s. The uncollided harmonics are matched to ones contained in a pre-stored spectrum library in order that each F0‘s harmonic series is assigned to the appropriate instrument. Corrupted harmonics are restored using data taken from the library. Finally, each voice is additively resynthesized to a separate track. This algorithm is demonstrated for a monaural signal containing three contrapuntal musical instrument voices with distinct timbres.KeywordsHarmonic SpectrumHarmonic SeriesMusic Information RetrievalPitch DetectionSeparate TrackThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

The traditional computer-aided design (CAD) software is often not realistic in rendering scenes, such as light, materials, shadows, and other details, which leads designers to spend a lot of energy on manual adjustment.The rendering algorithm based on deep learning (DL) can automatically learn the complex output, and the effect is usually better than traditional methods.In this article, a convolutional neural network (GA-CNN) based on genetic algorithm (GA) optimization is proposed, and it is combined with CAD data and applied to the rendering of environmental art design scenes.This method uses GA to optimize the parameters of CNN to improve its learning effect and generalization ability.From the overall trend, the response speed of GA-CNN algorithm and support vector machine (SVM).The response speed of the traditional CNN method is relatively slow, and the fluctuation range is large.This is because traditional CNN needs a lot of convolution operations and feature extraction when dealing with artistic scenes in complex environments, which leads to a long calculation time.In practical application, it is necessary to select appropriate algorithms and parameter settings according to specific task requirements to achieve the best results.

We develop a theoretical framework for representation and automated generation of South Indian classical music. The foundational part of the latter is based on symbolic dynamics and is implemented by translating the lexicographic rules for a rāga to constraints on a Markov chain whose state space is a layered graph. We analyze the statistical properties of this Markov chain from the point of view of information theory. We also develop several tools in music signal processing, such as, (a) a procedure for automated generation of gamakas or ornamental notes, unique to South Indian classical music, (b) rhythm synchronization, and (c) an algorithm for perceptual scale shifts. The Online Supplement has computer synthesized music from Śankarābharaṇa rāga, rhythm synchronized music for a Mōhana rāga composition, original and synthesized gamakas over madhyama for Begaḍa and Nīlāmbari rāgas, and perceptually scale shifted rāgas Hindoḷa, Madhyamāvati, Śuddha sāveri, and Udaya ravi chandrikā rāgas from the base Mōhana rāga.

Music technologies will open the future up to new ways of enjoying music both in terms of music appreciation and music creation. In this keynote speech, I introduce the frontiers of music technologies by showing some practical examples to demonstrate how end users can benefit from music signal processing, music understanding technologies, singing synthesis technologies, and music interfaces. For example, a web service for active music listening, "Songle" (http://songle.jp), has analyzed more than 830,000 songs on music- or video-sharing services and facilitates deeper understanding of music and music-synchronized control of robot dancers. A web service for large-scale music browsing, "Songrium" (http://songrium.jp), allows users to explore music while seeing and utilizing various relations among more than 680,000 music video clips on video-sharing services. Singing synthesis technologies open up new possibilities in music creation. I conclude by discussing grand challenges.

This paper describes the construction of a system called i-Ring that can generate a polyphonic ringtone based on a user's humming input. Algorithms used in the system for music transcription and chord generation include various techniques in speech and music signal processing, such as pitch tracking and dynamic programming, which are explained in the paper. Experimental results demonstrate the feasibility of i-Ring as a handy tool for computer assisted music composition for polyphonic ringtones

Non-negative Matrix Factorisation (NMF) is a popular tool in musical signal processing. However, problems using this methodology in the context of Automatic Music Transcription (AMT) have been noted resulting in the proposal of supervised and constrained variants of NMF for this purpose. Group sparsity has previously been seen to be effective for AMT when used with stepwise methods. In this paper group sparsity is introduced to supervised NMF decompositions and a dictionary tuning approach to AMT is proposed based upon group sparse NMF using the β-divergence. Experimental results are given showing improved AMT results over the state-of-the-art NMF-based AMT system.

Audio signal processing can be integrated with music domain in-order to retrieve contents within the music signal. Raga is often related to melodic framework in Indian classical music. Automatic Raga identification can be achieved through audio signal processing and Raga identification can be used for different applications including music signal synthesis, audio dataset management and music therapy. Classical methods for Raga analysis in time and frequency domain are less efficient and these algorithms are quite time consuming. The overall accuracy of Raga classification can be increased by incorporating artificial neural networks. In this paper, pitch contour of selected Ragas in Carnatic music are used as key features and model implementation is done using Convolutional Neural Network (CNN).