I-Ring: a system for humming transcription and chord generation

Deep Learning has emerged as a powerful set of computational methods achieving great results in a variety of different tasks. Music signal processing, a field with rich commercial applications, seems to benefit too from this data-driven approach. In this paper a review of the state of the art Deep Learning methods applied on music signal processing is provided. A special focus is given in music information retrieval and music generation. In addition, possible future research directions are discussed.

Introduction: In order to assess how well conventional and hybrid pitch detection techniques perform in speech processing applications, a comparative analysis of the two types of methods is conducted. Method: A proposed hybrid approach, Proposed PEF+CEP, is examined alongside five traditional algorithms, namely Normalized Correlation Function (NCF), Pitch Estimation Filter (PEF), Log- Harmonic Summation (LHS), Summation of Residual Harmonics (SRH) and Cepstrum Pitch Determination (CEP). The effectiveness is evaluated using performance metrics like accuracy, specificity, sensitivity, and Gross Pitch Error (GPE). Results and Discussion: Our findings show that the accuracy and specificity of the traditional methods are impressive; the accuracy and sensitivity of the suggested hybrid method surpass their performance, with an astounding 98.8% accuracy and 99.2% sensitivity. Conclusion: Furthermore, the Proposed PEF+CEP method is a promising solution for accurate and dependable pitch detection in speech processing applications because it strikes a strong balance between computational efficiency and robustness. These results open up new avenues for research in the field of speech processing and demonstrate the potential of hybrid approaches.

Stacked auto-encoders based visual features for speech/music classification

Speech processing is one of the required fields in digital signal processing that helps in processing the speech signals. The speech process is utilized in different fields such as emotion recognition, virtual assistants, voice identification, etc. Among the various applications, emotion recognition is one of the critical areas because it is used to recognize people’s exact emotions and eliminate physiological issues. Several researchers utilize signal processing and machine learning techniques together to find the exact human emotions. However, they fail to attain their feelings with less computational complexity and high accuracy. This paper introduces the intelligent computational technique called cat swarm optimized spiking neural network (CSSPNN). Initially, the emotional speech signal is collected from the Toronto emotional speech set (TESS) dataset, which is then processed by applying a wavelet approach to extract the features. The derived features are further examined using the defined classifier CSSPNN, which recognizes human emotions due to the effective training and learning process. Finally, the proficiency of the system is determined using experimental results and discussions. The proposed system recognizes the speech emotions up to 99.3% accuracy compared to recurrent neural networks (RNNs), deep neural networks (DNNs) and deep shallow neural networks (DSNNs).

Publication in the conference proceedings of EUSIPCO, Marrakech, Morocco, 2013

Detecting multiple pitches (F0s) and segregating musical instrument lines from monaural recordings of contrapuntal polyphonic music into separate tracks is a difficult problem in music signal processing. Applications include audio-to-MIDI conversion, automatic music transcription, and audio enhancement and transformation. Past attempts at separation have been limited to separating two harmonic signals in a contrapuntal duet (Maher, 1990) or several harmonic signals in a single chord (Virtanen and Klapuri, 2001, 2002). Several researchers have attempted polyphonic pitch detection (Klapuri, 2001; Eggink and Brown, 2004a), predominant melody extraction (Goto, 2001; Marolt, 2004; Eggink and Brown, 2004b), and instrument recognition (Eggink and Brown, 2003). Our solution assumes that each instrument is represented as a time-varying harmonic series and that errors can be corrected using prior knowledge of instrument spectra. Fundamental frequencies (F0s) for each time frame are estimated from input spectral data using an Expectation-Maximization (EM) based algorithm with Gaussian distributions used to represent the harmonic series. Collisions (i.e., overlaps) between instrument harmonics, which frequently occur, are predicted from the estimated F0s. The uncollided harmonics are matched to ones contained in a pre-stored spectrum library in order that each F0‘s harmonic series is assigned to the appropriate instrument. Corrupted harmonics are restored using data taken from the library. Finally, each voice is additively resynthesized to a separate track. This algorithm is demonstrated for a monaural signal containing three contrapuntal musical instrument voices with distinct timbres.KeywordsHarmonic SpectrumHarmonic SeriesMusic Information RetrievalPitch DetectionSeparate TrackThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

This chapter takes a new look at an old riddle by discussing putative parallelisms between speech and music processing, as well as music to speech transfer effects. Interestingly, both speech and music rely on comparable acoustic information (i.e. timing, pitch, and timbre) and share, at least partially, perceptive, cognitive, motivational, and motor components. Importantly, in this context, the chapter introduces a cortical framework of speech processing, as well as the perceptual and cognitive operations underlying speech learning mechanisms. It then reviews previous literature pinpointing the influence of music training on segmental and suprasegmental aspects of speech processing as well as on cognitive functions. Two complementary hypotheses underlying such transfer effects—cascade effects and multidimensional processes—are proposed. The issue of shared versus distinct neural networks for music and speech processing is also discussed. Finally, the chapter integrates the often observed perceptual and cognitive advantages of musicians in a holistic framework by considering their interrelations during an ecological, valid, word-learning task.

This dissertation addressed the problem of detecting and estimating the frequency components in a time varying signal. Specifically, we studied the problems of estimating the pitch frequency of a voiced speech signal and extracting the frequency components in a music signal. Our pitch estimation research clarified some relationships among the existing pitch determination algorithms (PDAs). It also resulted in a few new PDAs: Firstly, a multi-channel PDA was proposed for use with speech data recorded using a microphone array. This algorithm jointly estimates the pitch and signal's time delay at each microphone. Secondly, a class of variable frame PDAs was proposed. In addition, we presented a new ML formulation of the pitch estimation problem. This formulation removes the singularity in the previous formulation caused by the number of unknown model parameters increasing with the unknown pitch period. Two joint pitch and spectrum estimators were derived, respectively, for the white and autoregressive spectrum models. Finally, we generalized these two estimators such that they can use multi-channel data to jointly estimate pitch, spectrum and time delay of the signal. As for the problem of estimating the frequency components in a music tone signal, we proposed a phase interpolation algorithm for use in an existing music synthesis algorithm and a new analysis-based music analysis and synthesis algorithm. The proposed phase interpolation algorithm uses quadratic spline functions to model the phase track. With respect to the perceived sound quality, it is much better than the existing algorithm using the quadratic phase and is comparable to the one that uses a cubic phase. However, the latter requires more computation and storage space. The proposed analysis-based music analysis-and-synthesis algorithm aims to improve analysis accuracy and increase synthesis efficiency of the similar algorithms in the literature. The analysis accuracy is improved by using a global waveform fitting approach to estimating the model parameters, in contrast to the frame-based Fourier transform approach used in the existing algorithms. The synthesis efficiency in both computation time and storage space is increased by using a quadratic phase model rather than the cubic phase model used in the previous algorithms.

Pitch estimation plays an important role on speech processing such as speech coding, synthesis, recognition, and so on. Although current pitch estimation method performs well under clean condition, the performance deteriorates significantly in noisy environment. For this reason, robust pitch estimation against additive noise is required. Modified auto-correlation method is commonly used as pitch estimation in which LP residual is used to compute the auto-correlation. We have previously proposed pitch estimation methods based on Time-Varying Complex AR (TV-CAR) analysis whose criterion is the weighted correlation of the complex residual obtained by the TV-CAR analysis, sum of the harmonics for the complex residual spectrum, or so on. On the other hand, Azarov et al. have proposed an improved method of RAPT (Robust Algorithm for Pitch Tracking) using an instantaneous harmonics that is called IRAPT (Instantaneous RAPT). The IRAPT can perform better estimation than RAPT. Since IRAPT uses band-limited analytic signal to obtain harmonic frequencies, the complex residual signal obtained by the TV-CAR analysis can also be applied to the IRAPT. In this paper, novel pitch estimation method using the instantaneous frequency based on the robust WLP (Weighted Linear Prediction) TV-CAR residual is proposed and evaluated.

A comprehensive overview on signal processing and artificial intelligence techniques applications in classification of power quality disturbances

The design and implementation of a parallel-processing-based pitch detector is presented. Pitch information is extracted by performing pitch detection on four different waveforms derived from the speech signal. Pitch information from the four pitch-detection processes is then combined to determine a final pitch estimate. The performance of this pitch detector is evaluated on a large database and compared to other well-known pitch detection algorithms. It has been implemented in real time on a TMS32020 fixed-point digital signal processor as part of a 2.4 kb/s vocoder. A performance comparison of the real-time fixed-point implementation and a computer simulation are also given. The results show that the pitch detector performance is maintained in the real-time implementation mainly because the majority of the algorithm computations are integer arithmetic and logic-type operations. >

Non-negative Matrix Factorisation (NMF) is a popular tool in musical signal processing. However, problems using this methodology in the context of Automatic Music Transcription (AMT) have been noted resulting in the proposal of supervised and constrained variants of NMF for this purpose. Group sparsity has previously been seen to be effective for AMT when used with stepwise methods. In this paper group sparsity is introduced to supervised NMF decompositions and a dictionary tuning approach to AMT is proposed based upon group sparse NMF using the β-divergence. Experimental results are given showing improved AMT results over the state-of-the-art NMF-based AMT system.

Artificial intelligence-empowered music processing is a domain that involves the use of artificial intelligence technologies to enhance music analysis, understanding, and generation. This field encompasses a variety of tasks from music generation to music comprehension. In practical applications, the complexity of interwoven tasks, differences in data representation, scattered distribution of tool resources, and the threshold of professional music knowledge often become barriers that hinder developers from smoothly carrying out generative tasks. Therefore, it is essential to establish a system that can automatically analyze their needs and invoke appropriate tools to simplify the music processing workflow. Inspired by the recent success of Large Language Models (LLMs) in task automation, we have developed a system named LongMergent, which integrates numerous music-related tools and autonomous workflows to address user requirements. By granting users the freedom to effortlessly combine tools, this system provides a seamless and rich musical experience.

Chroma features are a popular tool in musical signal processing and information retrieval tasks, providing a compact representation of the tonal content of a piece of music. A variety of approaches to chroma estimation have been proposed, most of which rely on the summation of related frequency partials. However, frequency partials may be incorrectly assigned due to the log/linear relationship of frequency and pitch. Variations of chroma employing overtone suppression strategies are found in the literature. We propose a compositional model of chroma, which considers a coarse modelling of the effects of overtones in the expected chroma vectors of single notes. Synthetic chord recognition experiments indicate the usefulness of the proposed approach.

In the last few years various musicological studies on the Byzantine Chant (BC) have been made via computer analysis. Due to the BC vocal nature and the pitch variability of the BC notes during the performance, the value of a music interval between two such notes in a certain BC mode - “echos”, is ambiguous. Because of that pitch variability a new method of BC intervals measurement is proposed, based on statistical processing of the pitch track that can be done after a proper pitch analysis of the BC recordings. The music scale of a particular BC echos is computed from the histogram of the pitch values by applying an algorithm that calculates a Gaussian Mixture Model approximating the music scale of that BC echos. This new method is applied, as an example, to ten BC recordings - performances of the Cherubic Hymn composed in the fourth echos (aghia) by Petros Lampadarios. The recordings come from the DAMASKINOS Corpus of University of Athens and the analysis results are in agreement with both the theoretical values of the Patriarchal Committee 1881 (approximation of up to 1 moria) and the findings in other similar studies. This method of analysis can be a base for both musicological analysis and music signal processing.