Recent research in musical timbre perception.

Timbre spaces represent the organization of perceptual distances, as measured with dissimilarity ratings, among tones equated for pitch, loudness, and perceived duration. A number of potential acoustic correlates of timbre-space dimensions have been proposed in the psychoacoustic literature, including attack time, spectral centroid, spectral flux, and spectrum fine structure. The experiments reported here were designed as direct tests of the perceptual relevance of these acoustical parameters for timbre dissimilarity judgments. Listeners presented with carefully controlled synthetic tones use attack time, spectral centroid, and spectrum fine structure in dissimilarity rating experiments. These parameters thus appear as major determinants of timbre. However, spectral flux appears as a less salient timbre parameter, its salience depending on the number of other dimensions varying concurrently in the stimulus set. Dissimilarity ratings were analyzed with two different multidimensional scaling models (CLASCAL and CONSCAL), the latter providing psychophysical functions constrained by the physical parameters. Their complementarity is discussed.

Figure: © iStock/chaossCochlear implants (CIs) have allowed users to understand speech and interact with the auditory world. However, CI recipients still have difficulty perceiving nonspeech sounds, especially music. Music appreciation is important to evaluate in CI users because of its association with quality of life, personal expression, and social connectedness.Figure: Alice HeCochlear implant recipients have difficulty recognizing musical melodies, identifying instrument sounds, and differentiating pitch. However, perceptual accuracy is not the only important aspect in music listening. The way music sounds as a whole through the cochlear implant has clinical relevance, as it relates to the user's ability to appreciate and enjoy music in everyday life.Figure: Charles J. Limb, MDWhile one might anticipate a direct link between cochlear implant recipients’ ability to perceive music accurately and their degree of music appreciation, the early literature has not supported this idea. In fact, some studies suggest exactly the opposite—that there is little to no relationship between music perception and appraisal in CI users ( J Am Acad Audiol 2012;23[5]:350-365http://aaa.publisher.ingentaconnect.com/content/aaa/jaaa/2012/00000023/00000005/art00006). However, these previous results were obtained from small cohorts of cochlear implant recipients in controlled laboratory settings, with tests that require significant time to administer, limiting their applicability in clinical settings. In a new study, Ward R. Drennan, PhD, and colleagues evaluated the correlation between music perception and appraisal among a large cohort of CI users in clinical settings. The authors used shorter music tests to determine if these assessments could be administered in the clinic and whether findings would be consistent with previous results from smaller sample sizes. Clinical Evaluation of Music Perception, Appraisal, and Experience in Cochlear Implant Users Drennan WR, Oleson JJ, Gfeller K, et al Int J Audiol Sept. 1, 2014; e-pub ahead of printhttp://informahealthcare.com/doi/abs/10.3109/14992027.2014.948219 The Clinical Assessment of Music Perception (CAMP) test is commonly used in some laboratories to study CI users’ music perception ( Otol Neurotol 2008;29[2]:149-155http://journals.lww.com/otology-neurotology/Abstract/2008/02000/Clinical_Assessment_of_Music_Perception_in.10.aspx; Ear Hear 2009;30[4]:411-418http://journals.lww.com/ear-hearing/Abstract/2009/08000/Development_and_Validation_of_the_University_of.4.aspx). The CAMP includes three separate perception tests, each designed to evaluate a different element of music perception ability: pitch direction discrimination, familiar melody recognition, and timbre recognition. Since these three elements are relevant to music listening and enjoyment, Dr. Drennan et al used the CAMP test to study music perception. To assess music appraisal, the researchers used the Iowa Musical Background Questionnaire (IMBQ). In the IMBQ, CI recipients described the quality of music heard through their cochlear implant by rating six bipolar adjectives on a scale of zero to 10: clear versus fuzzy, easy to follow versus difficult to follow, like versus dislike, sounds unlike music versus sounds like music, unnatural versus natural, and unpleasant versus pleasant. Prior research identified these comparisons as relevant to music listening. The CAMP test and IMBQ were administered to 145 CI listeners across 14 clinical sites in the United States and Canada. It took 30 minutes to 35 minutes to administer the CAMP test, and another 15 minutes to 25 minutes to complete the IMBQ. These time frames are reasonable for administration in the clinic—a major advantage in terms of practical feasibility, the authors noted. WEAK RELATIONSHIP REPORTED In their group of cochlear implant users from several clinical settings, Dr. Drennan and colleagues found weak relationships between CAMP results and IMBQ musical sound quality scores. There was a statistically significant but slight correlation between the CAMP's timbre test component and music appraisal, suggesting a relatively weak association between the two. All other relationships were deemed insignificant. This lack of a strong association, which is consistent with the findings of previous studies ( J Am Acad Audiol 2012;23[5]:350-365http://aaa.publisher.ingentaconnect.com/content/aaa/jaaa/2012/00000023/00000005/art00006; J Am Acad Audiol 2008;19[2]:120-134http://aaa.publisher.ingentaconnect.com/content/aaa/jaaa/2008/00000019/00000002/art00003) means that in order to fully evaluate CI music functionality, it is not sufficient to test either music perception or music appraisal alone. Both measures need to be evaluated, as sufficiency or deficiency in one aspect does not correlate with the same result in the other aspect. These results also highlight the limitations of subjective musical sound quality assessment and the possible benefits of objective approaches ( Otol Neurotol 2012;33[3]:319-327http://journals.lww.com/otology-neurotology/Abstract/2012/04000/Assessment_of_Sound_Quality_Perception_in_Cochlear.7.aspx). In an objective approach, the goal is not to evaluate how pleasant or unpleasant music may sound, or whether the CI user enjoys it. Rather, the goal is to measure the capacity of a cochlear implant user to detect a deterioration of sound quality in response to experimental manipulation of any given sound parameter. MANY FACTORS AFFECT ENJOYMENT Dr. Drennan and colleagues noted limitations of their study design. For example, the CAMP was conducted without hearing aids, even if listeners used hearing aids regularly. While the IMBQ questionnaire asked about music appraisal based on real-world musical listening experiences, the CAMP used simple, controlled sound stimuli to assess basic music perception abilities instead of using real-world stimuli. To some extent, these study design decisions might have affected CI users’ music test scores. Despite these potential limitations, the authors make a compelling argument that perceptual acuity is not closely related to music appraisal ratings. In other words, cochlear implant users can have poor perceptual abilities and still feel that music sounds good or is highly enjoyable. Conversely, CI users can perceive various aspects of music well while feeling that music sounds poor or not enjoying what they are able to hear. The study by Dr. Drennan and colleagues underscores the notion that the musical experience is linked to many factors. Furthermore, these findings suggest that studying individual elements of music in isolation may not accurately reflect the perceptual salience of music when it's presented in its natural form—a composite of a wide range of components. In this study, the authors demonstrated that there is little to no relationship between music acuity and music appraisal. Both aspects are important clinical outcomes, as one addresses patient functionality and ability (acuity), while the other addresses patient feeling and overall satisfaction with music quality (appraisal). These findings strongly suggest the need for further research to overcome limitations in cochlear implant users’ ability to perceive music, and to understand the factors that limit enjoyment or satisfaction in music listening.

<p><strong>Objectives:</strong> Despite vast amounts of research examining the influence of hearing loss on speech perception, comparatively little is known about its influence on music perception. No standardized test exists to quantify music perception of hearing-impaired (HI) persons in a clinically practical manner. This study presents the Adaptive Music Perception (AMP) test as a tool to assess important aspects of music perception with hearing loss.</p> <p><strong>Design:</strong> A computer-driven test was developed to determine the discrimination thresholds of 10 low-level physical dimensions (e.g., duration, level) in the context of perceptual judgments about musical dimensions: meter, harmony, melody, and timbre. In the meter test, the listener is asked to judge whether a tone sequence is duple or triple in meter. The harmony test requires that the listener make judgments about the stability of the chord sequences. In the melody test, the listener must judge whether a comparison melody is the same as a standard melody when presented in transposition and in the context of a chordal accompaniment that serves as a mask. The timbre test requires that the listener determines which of two comparison tones is different in timbre from a standard tone (ABX design). Twenty-one HI participants and 19 normal-hearing (NH) participants were recruited to carry out the music tests. Participants were tested twice on separate occasions to evaluate test-retest reliability.</p> <p><strong>Results:</strong> The HI group had significantly higher discrimination thresholds than the NH group in 7 of the 10 low-level physical dimensions: frequency discrimination in the meter test, dissonance and intonation perception in the harmony test, melody-to-chord ratio for both melody types in the melody test, and the perception of brightness and spectral irregularity in the timbre test. Small but significant improvement between test and retest was observed in three dimensions: frequency discrimination (meter test), dissonance (harmony test), and attack length (timbre test). All other dimensions did not show a session effect. Test-retest reliability was poor (<0.6) for spectral irregularity (timbre test); acceptable (>0.6) for pitch and duration (meter test), dissonance and intonation (harmony test), and melody-to-chord ratio I and II (melody test); and excellent (>0.8) for level (meter test) and attack (timbre test).</p> <p><strong>Conclusion:</strong> The AMP test revealed differences in a wide range of music perceptual abilities between NH and HI listeners. The recognition of meter was more difficult for HI listeners when the listening task was based on frequency discrimination. The HI group was less sensitive to changes in harmony and had more difficulties with distinguishing melodies in a background of music. In addition, the thresholds to discriminate timbre were significantly higher for the HI group in brightness and spectral irregularity dimensions. The AMP test can be used as a research tool to further investigate music perception with hearing aids and compare the benefit of different music processing strategies for the HI listener. Future testing will involve larger samples with the inclusion of hearing aided conditions allowing for the establishment of norms so that the test might be appropriate for use in clinical practice.</p>

Introduction: The objective of our study was to evaluate musical perception and its relation to the quality of life in patients with bimodal binaural auditory stimulation.Materials and Methods: Nineteen adult patients with a cochlear implant (CI) for minimum 6 months, and moderate to severe contralateral hearing loss with a hearing aid (HA), and 21 normal hearing adults were included in this prospective, cross-sectional study. Pure-tone and speech audiometry, musical test evaluating sound perception characteristics and musical listening abilities, Munich questionnaire for musical habits, and the APHAB questionnaire were recoded. Performance in musical perception test with HA, CI, and HA + CI, and potential correlations between music test, audiometry and questionnaires were investigated.Results: Bimodal stimulation improved musical perception in several features (sound brightness, roughness, and clarity) in comparison to unimodal hearing, but CI did not add to HA performances in texture, polyphony or musical emotion and even appeared to interfere negatively in pitch perception with HA. Musical perception performances (sound clarity, instrument recognition) appeared to be correlated to hearing-related quality of life (APHAB RV and EC subdomains) but not with speech performances suggesting that the exploration of musical perception complements speech understanding evaluation to better describe every-day life hearing handicap.Conclusion: Testing musical sound perception provides important information on hearing performances as a complement to speech audiometry and appears to be related to hearing-related quality of life.

Ten sustained musical instrument tones (bassoon, cello, clarinet, flute, horn, oboe, recorder, alto saxophone, trumpet, and violin) were spectrally analyzed and then equalized for duration, attack and decay time, fundamental frequency, number of harmonics, average spectral centroid, and presentation loudness. The tones were resynthesized both with time-varying harmonic amplitudes and frequencies (dynamic case) and fixed amplitudes and frequencies (static case). Tone triads were presented to ten musically experienced listeners whose tasks were to specify the most dissimilar and most similar pairs in each triad. Based on the resulting dissimilarity matrix, multidimensional scaling (MDS) was used to position the instruments in two- and three-dimensional metric spaces. Two measures of instrument amplitude spectra were found to correlate strongly with MDS dimensions. For both the static- and dynamic-case 2-D solutions, the ratio of even-to-odd rms amplitudes correlated strongly with one of the dimensions. For the dynamic case, spectral centroid variation correlated strongly with the second dimension. Also, 2-D solution instrument groupings agreed well with groupings based on coefficients of the first two components of a principle components analysis representing 90% of the instruments’ spectral variance. [This work was supported by the Hong Kong Research Grants Council’s CERG Project 613505.]

This study aimed to investigate whether adults with cochlear implants benefit from a change of fine structure (FS) coding strategies regarding the discrimination of prosodic speech cues, timbre cues, and the identification of natural instruments. The FS processing (FSP) coding strategy was compared to 2 settings of the FS4 strategy. A longitudinal crossover, double-blinded study was conducted. This study consisted of 2 parts, with 14 participants in the first part and 12 participants in the second part. Each part lasted 3 months, in which participants were alternately fitted with either the established FSP strategy or 1 of the 2 newly developed FS4 settings. Participants had to complete an intonation identification test; a timbre discrimination test in which 1 of 2 isolated cues changed, either the spectral centroid or the spectral irregularity; and an instrument identification test. A significant effect was seen in the discrimination of spectral irregularity with 1 of the 2 FS4 settings. The improvement was seen in the FS4 setting in which the upper envelope channels had a low stimulation rate. This improvement was not seen with the FS4 setting that had a higher stimulation rate on the envelope channels. In general, the FSP strategy and the 2 settings of the FS4 strategy provided similar levels in the perception of prosody and timbre cues, as well as in the identification of instruments.

Previous research on musical timbre has suggested that onsets are particularly important for the perception of musical timbre; identification is poor if listeners do not hear onsets, and dimensions of MDS solutions tend to correspond to attack length. However, more recent research has demonstrated that the dynamic attributes that influence similarity judgments cannot be localized to onset transients [P. Iverson and C. L. Krumhansl, J. Acoust. Soc. Am. 94, 2595–2603 (1993)]. In separate experiments, subjects rated the similarity of complete tones, the onsets of those tones, and tones with the onsets removed (remainders). Multidimensional scaling solutions revealed that judgments on all three sets of tones were quite similar, demonstrating that the attributes that influence timbre in complete tones are present both at onsets and remainders. Results from acoustic analyses suggested that similarity judgments may rely on global acoustic comparisons rather than on the isolation of specific acoustic attributes. In contrast, identification judgments may rely on specific acoustic attributes present only at the onsets. This research will be reviewed in light of new research on musical timbre perception.

We have proposed the notion of timbre saliency as the attention-capturing quality of timbre. The definition of saliency requires an object to stand out with respect to its surroundings, implying dissimilarity between the object and its neighbors. What then might be the relationship between timbre saliency and timbre dissimilarity? A classic timbre dissimilarity experiment and a timbre saliency experiment were carried out with 20 participants on the same set of stimuli. Multidimensional scaling revealed a two-dimensional dissimilarity space. Using the features obtained from the Timbre Toolbox [Peeters et al. (2011), J. Acoust. Soc. Am., 130, 2902-2916], the first dimension shows a high correlation with spectral centroid [r(13)=.845, p<.0001] and spectral spread [r(13)=.855, p<.0001], both based on the ERB-FFT model spectrum, and the second with the attack time [r(13)=-.692, p=.004] and power spectral crest [r(13)=.732, p<.005]. This confirms spectral centroid and attack time as two major acoustic correlates of timbre dissimilarity. The saliency dimension shows a moderate correlation with the second dimension [r(13)=.578, p=.0241] but not with the first dimension [r(13)=.182, p=.517], suggesting that the saliency might be more related to the temporal characteristics of timbre.

In this paper, we report the influence that classification accuracies have in speech analysis from a clinical dataset by adding acoustic low-level descriptors (LLD) belonging to prosodic (i.e. pitch, formants, energy, jitter, shimmer) and spectral features (i.e. spectral flux, centroid, entropy and roll-off) along with their delta (d) and delta-delta (d-d) coefficients to two baseline features of Mel frequency cepstral coefficients and Teager energy critical-band based autocorrelation envelope. Extracted acoustic low-level descriptors (LLD) that display an increase in accuracy after being added to these baseline features were finally modeled together using Gaussian mixture models and tested. A clinical data set of speech from 139 adolescents, including 68 (49 girls and 19 boys) diagnosed as clinically depressed, was used in the classification experiments. For male subjects, the combination of (TEO-CB-Auto-Env + d + d-d) + F0 + (LogE + d + d-d) + (Shimmer + d) + Spectral Flux + Spectral Roll-off gave the highest classification rate of 77.82% while for the female subjects, using TEO-CB-Auto-Env gave an accuracy of 74.74%.

Acoustic classification of Australian frogs based on enhanced features and machine learning algorithms

This paper explains classification using Support Vector Machines (SVM) technique and clustering using K-means technique in identifying eight spoken dialects in Indonesian language. Dialect identification is important to build a better Automatic Speech Recognition system. The experiment in this research is divided into using three features of sound; Mel Frequency Cepstral Coefficient (MFCC), spectral flux, and spectral centroid, and compares it to model with MFCC features only. For methods, it uses one-against-one and all-at-once as comparison. The best result is from using SVM one-against-one with three features which gives 55%.

Investigation of acoustic and visual features for acoustic scene classification

We present a framework for selecting best audio features for audiovisual analysis and synthesis of dance figures. Dance figures are performed synchronously with the musical rhythm. They can be analyzed through the audio spectra using spectral and rhythmic musical features. In the proposed audio feature evaluation system, dance figures are manually labeled over the video stream. The music segments, which correspond to labeled dance figures, are used to train hidden Markov model (HMM) structures to learn temporal spectrum patterns for the dance figures. The dance figure recognition performances of the HMM models for various spectral feature sets are evaluated. Audio features, which are maximizing dance figure recognition performances, are selected as the best audio features for the analyzed audiovisual dance recordings. In our evaluations, mel-scale cepstral coefficients (MFCC) with their first and second derivatives, spectral centroid, spectral flux and spectral roll-off are used as candidate audio features. Selection of the best audio features can be used towards analysis and synthesis of audio-driven body animation.

This paper reports the result of Musical instrument recognition using fractional fourier transform (FRFT) based features. The FRFT features are computed by replacing conventional Fourier transform in Mel Frequecny Cepstral coefficient ( MFCC) with FRFT. The result of the system using FRFT is compared with the result of the system using Mel Frequency Cepstral Coefficients (MFCC), Wavelet and Timbrel features with different machine learning algorithms. The experimentation is performed on isolated musical sounds of 19 musical instruments covering four different instrument families. The system using FRFT features outperforms over MFCC, Wavelet and Timbrel features with 91.84% recognition accuracy for individual instruments. The system is tested on benchmarked McGill University musical sound database. The experimental result shows that musical sound signals can be better represented using FRFT.

Assessment of wood quality for Sape making: Vibroacoustic analysis and machine learning classification