Gaussian process data fusion for heterogeneous HRTF datasets

Purpose – The purpose of this paper is to present a system for customized binaural audio delivery based on the extraction of relevant features from a 2-D representation of the listener’s pinna. Design/methodology/approach – The most significant pinna contours are extracted by means of multi-flash imaging, and they provide values for the parameters of a structural head-related transfer function (HRTF) model. The HRTF model spatializes a given sound file according to the listener’s head orientation, tracked by sensor-equipped headphones, with respect to the virtual sound source. Findings – A preliminary localization test shows that the model is able to statically render the elevation of a virtual sound source better than non-individual HRTFs. Research limitations/implications – Results encourage a deeper analysis of the psychoacoustic impact that the individualized HRTF model has on perceived elevation of virtual sound sources. Practical implications – The model has low complexity and is suitable for implementation on mobile devices. The resulting hardware/software package will hopefully allow an easy and low-tech fruition of custom spatial audio to any user. Originality/value – The authors show that custom binaural audio can be successfully deployed without the need of cumbersome subjective measurements.

이 연구는 중앙면 상에서 주성분 분석법을 이용하여 시간 및 주파수 영역에서 머리전달함수의 모형화 기법들을 다룬다. 시간영역의 머리전달함수, 복소수 값의 머리전달함수, 확장된 머리전달함수, 로그 크기의 머리전달함수에 기반하여 각각 주성분 분석법을 수행하여 얻은 네 가지 머리전달함수 모형들에 대해서 최소자승오차 관점에서 모형화 성능을 비교하고, 모형들간의 이론적인 관계를 살펴보는 것이 이 연구의 목적이다. 모형화에 사용되는 기저함수의 수가 동일하다면, 시간영역에서의 머리전달함수 혹은 확장된 머리전달함수에 기반한 모형이 복소수 값의 머리전달함수에 기반한 모형보다 최소자승오차 관점에서 더 효율적인 모형화 성능을 지닌다. 시간영역에서의 머리전달함수에 기반한 모형과 확장된 머리전달함수에 기반한 모형은 이론적으로 동일한 모형이며 서로 푸리에 변환 관계가 있다. 로그 크기의 머리전달함수에 기반한 모형은 다른 모형들과 모형화 성능 및 이론적인 관계를 비교할 수가 없는데, 이는 로그 크기의 머리전달함수에 기반한 모형은 머리전달함수의 크기 정보만을 로그 크기로 다루는 반면에 다른 모형들은 선형 크기로 머리전달함수의 크기와 위상정보를 모두 다루기 때문이다. This study deals with modeling of head-related transfer functions(HRTFs) using principal components analysis(PCA) in the time and frequency domains. Four PCA models based on head-related impulse responses(HRIRs), complex-valued HRTFs, augmented HRTFs, and log-magnitudes of HRTFs are investigated. The objective of this study is to compare modeling performances of the PCA models in the least-squares sense and to show the theoretical relationship between the PCA models. In terms of the number of principal components needed for modeling, the PCA model based on HRIR or augmented HRTFs showed more efficient modeling performance than the PCA model based on complex-valued HRTFs. The PCA model based on HRIRs in the time domain and that based on augmented HRTFs in the frequency domain are shown to be theoretically equivalent. Modeling performance of the PCA model based on log-magnitudes of HRTFs cannot be compared with that of other PCA models because the PCA model deals with log-scaled magnitude components only, whereas the other PCA models consider both magnitude and phase components in linear scale.

Head-Related Transfer Function (HRTF) representation and interpolation is an important problem in spatial audio. We present a kernel regression method based on Gaussian process (GP) modeling of the joint spatial-frequency relationship between HRTF measurements and obtain a smooth non-linear representation based on data measured over both arbitrary and structured spherical measurement grids. This representation is further extended to the problem of extracting spectral extrema (notches and peaks). We perform HRTF interpolation and spectral extrema extraction using freely available CIPIC HRTF data. Experimental results are shown.

Head-related transfer function (HRTF) plays an important role in the construction of 3D auditory display. This paper presents an individual HRTF modeling method using deep neural networks based on spatial principal component analysis. The HRTFs are represented by a small set of spatial principal components combined with frequency and individual-dependent weights. By estimating the spatial principal components using deep neural networks and mapping the corresponding weights to a quantity of anthropometric parameters, we predict individual HRTFs in arbitrary spatial directions. The objective and subjective experiments evaluate the HRTFs generated by the proposed method, the principal component analysis (PCA) method, and the generic method. The results show that the HRTFs generated by the proposed method and PCA method perform better than the generic method. For most frequencies the spectral distortion of the proposed method is significantly smaller than the PCA method in the high frequencies but significantly larger in the low frequencies. The evaluation of the localization model shows the PCA method is better than the proposed method. The subjective localization experiments show that the PCA and the proposed methods have similar performances in most conditions. Both the objective and subjective experiments show that the proposed method can predict HRTFs in arbitrary spatial directions.

Four head-related transfer function (HRTF) data sets were compared to determine the effect of HRTF measurement methodology on the localization of spatialized auditory stimuli. Spatial audio interfaces typically require HRTF data sets to generate the spatialized auditory stimuli. HRTF measurement is accomplished using a variety of techniques that can require several nearly arbitrary decisions about methodology. The effects of these choices upon the resulting spatial audio interface are unclear. Sixteen participants completed a sound localization task that included real-world, broadband stimuli spatialized at eight locations on the horizontal plane. Four different HRTF data sets were utilized to spatialize the stimuli: two publicly available HRTF data sets and two data sets obtained using different in-house measurement systems. All HRTFs were obtained from the Knowles Electronics Mannequin for Acoustics Research. Unsigned localization error and proportion of front/back reversals did not differ significantly across HRTF data sets. Poorest accuracy was observed in locations near the medial (front/back) axis of the listener, mainly because of the relatively large proportions of reversals at these locations. This study suggests that the particular generalized HRTF data set chosen for spatialization is of minimal importance to the localizability of the resulting stimuli. This result will inform the design of many spatial audio interfaces that are based upon generalized HRTFs, including wayfaring devices, communication systems, and virtual reality systems.

Human beings have a remarkable ability to determine the direction of arrival of a sound and to separate sounds of interest. Replicating this ability is a challenging problem in audio signal processing. In this paper we present a model for the head related transfer function (HRTF) developed with the localisation objective in mind. This is achieved by splitting the 3D localisation cues in terms of two functions which can be independently evaluated. We illustrate the theory for calculating these functions and validate the results against actual HRTF data. We find the model to be a close match for a significant number of potential source locations.

It is widely acknowledged that individualized head-related transfer function (HRTF) measurements are needed to adequately capture all of the 3D spatial hearing cues. However, many perceptual studies have shown that localization accuracy in the lateral dimension is only minimally decreased by the use of non-individualized head-related transfer functions. This evidence supports the idea that the individualized components of an HRTF could be isolated from those that are more general in nature. In the present study we decomposed the HRTF at each location into average, lateral and intraconic spectral components, along with an ITD in an effort to isolate the sound localization cues that are responsible for the inter-individual differences in localization performance. HRTFs for a given listener were then reconstructed systematically with components that were both individualized and non-individualized in nature, and the effect of each modification was analyzed via a virtual localization test where brief 250 ms noise bursts were rendered with the modified HRTFs. Results indicate that the cues important for individualization of HRTFs are contained almost exclusively in the intraconic portion of the HRTF spectra and localization is only minimally affected by introducing non-individualized cues into the other HRTF components. These results provide new insights into what specific inter-individual differences in head-related acoustical features are most relevant to sound localization, and provide a framework for how future human-machine interfaces might be more effectively generalized and/or individualized.

Extensive head-related transfer function (HRTF) measurements show high HRTF repeatability, consequences of different measurement methods, and conditions covering the whole three-dimensional space [Riederer, J. Audio Eng. Soc. (Abstracts) 46, 1036 (1998), preprint 4846]. This study concentrates on specific effects on HRTFs carefully re-measured on the same Cortex dummy head applying Sennheiser KE4-211-2 microphones at its silicone putty blocked ear-canal entrances, employing 252 sound incidents including seven elevations. The effects of five different wigs (synthetic, natural, thick, thin, long and short hair) with varied hairstyles, four hats (cap, bicycle helmet, mens and womens trilby), clothes (alpaca pullover, bicycling drymax-jacket) and spectacles were investigated under 28 combinations. The influences are highly dependent on direction, frequency, and case. Clothes and eye-glasses affect minimally HRTF; hair has a stronger effect, depending on the actual hairdo (typically above 7 kHz). Hats alter intensively HRTFs (typically above 5 kHz), depending on the model. The measurements give deeper insight to the development of idiosyncratic features in binaural localization cues. The second part of the study addresses their perceptual effects [Riederer, J. Acoust. Soc. Am., this issue]. [Work supported by Graduate School of Electronics, Telecommunication and Automation; thanks to Finnish Broadcasting Company, Mr. Hellstrom; Mrs. Chen.]

Head related transfer function (HRTF) is a continuous function of spatial direction. In practice, only HRTFs at discrete directions can be measured. In order to obtain HRTFs at unmeasured directions, a continuous reconstruction (or interpolation) procedure is needed. In this paper, a method for continuous reconstruction of HRTF based on optimal-degree spherical harmonics expansion is developed. Expansion coefficients were derived by using the pseudo-inverse method. To address the ill-posed problem in inverse calculation, the expansion degree for each frequency bin was optimized by a heuristic method. This method searches the minimum-norm errors of the reconstructed HRTFs with respect to measured ones which were not used to derive the coefficients. The KEMAR HRTF at 710 directions and a human HRTF at 493 directions were employed in reconstruction. Measured HRTFs from KEMAR or human subject were dived into three groups according to directions, in which the first group of 2/5, the second group of 2/5, and the third group of 1/5 data were used to calculate the expansion coefficients, determine the optimal expansion degree, and evaluate the reconstruction performance, respectively. Compared with the expansion methods using a constant degree for all frequency, the reconstruction performance of the current method is prior.

Toward orthogonal non-individualised head-related transfer functions for forward and backward directional sound: cluster analysis and an experimental study

This paper provides a small set of essential parameters for a personalized and effective real-time auralization with headphones. An image-guided procedure with two 2D images of the user’s head guides the mixed structural modeling of head-related transfer function (HRTF), combining a spherical head model with ear displacement with the HRTF high-frequency magnitude selected from a database according to ear anthropometry. Room acoustics phenomena are simplified following the scattering delay network (SDN) approach which allows an accurate spatialization of first order reflections. Finally, statically significant improvements in localization performances within a virtual reality (VR) test allow to identify some benefits of the proposed customized auralization model compared to the widely used higher-order ambisonics (HOA) rendering with generic HRTFs.

In this paper we analyze several auditory localization neural models that are based on head related transfer functions (HRTFs). HRTFs represent the combined directional-spectral response of the pinnae head and torso. The role of HRTFs in such modeling has hitherto been underestimated despite substantial experimental evidence to its relevance in spatial hearing, especially in determining direction of highfrequency sound sources. In the first section we suggest a neural model that links the physiology of binaural processing to a neural network that extracts spectral ratios. These ratios correspond to HRTFs ratios and can provide auditory directional cues. Next, we compare several methods of matching HRTFs ratios using discriminative matching measure (DMM). We consider several solutions to the matching problem from a neural signal processing viewpoint. We compare correlation based approaches with DMM optimization approach and with a non-linear approach based on neural back-propagation algorithm. All three models can be implemented by neural networks. Finally, we include experimental results of simulations that are conducted using these methods. Experiments show that the back-propagation based neural network yields the best results in terms of DMM both for narrow-band and broad band excitation. The back-propagation neural network is also superior in matching noisy HRTF ratio vectors. [Neurol Res 2001; 23: 489-500]

"© 2017 IEEE. Personal use of this material is permitted. Permissíon from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertisíng or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works."

Applications in the creation of virtual auditory spaces (VAS) and sonification require individualized head related transfer functions (HRTFs) for perceptual fidelity. HRTFs exhibit significant variation from person to person due to differences between their pinnae, and their body sizes. We propose and preliminarily implement a simple HRTF customization based on the use of a published database of HRTFs (Algazi et al., 2001) that also contains geometrical measurements of subject pinnae. We measure some of these features via simple image processing, and select the HRTF that has features most closely corresponding to the individual's features. This selection procedure is implemented along with the virtual auditory system described in (Zotkin et al., 2002), and listener tests conducted comparing the customized HRTF and a fixed HRTF. Despite the simplicity of the method, tests reveal average improvement in localization accuracy of about 25 percent, though performance improvement varies with source location and individuals.

International audience