Apparatus for Implementing 3-Dimensional Virtual Sound and Method Thereof

The head-related transfer function(HRTF) is an essential part of spatial auditory display systems. In recent years, numerous HRTF databases are established to include human subjects' measurements, which enabled data-driven research projects such as HRTF prediction and personalization. However, in most cases, each HRTF database has its own unique measuring standard and source direction grid set. It's difficult to merge and efficiently represent the global HRTF information across multiple HRTF databases. Current presentation methods, such as principle component analysis (PCA) and spherical harmonics transform (SHT), are constrained by the source grid layout and regularization errors, which lays burdens for common feature extraction from different sets of HRTF measurements. In this work, we propose a novel approach for the global compact representation of HRTFs across different measuring standards, using hemispherical harmonics (HSH) and conformal mapping. The method takes into account the spatial domain covered by the typical HRTF measurements and proves to be less contained by the source direction grids. Both numerical and auditory model experiments are performed, to examine the representation error and consistency when including multiple HRTF databases with different measuring standards and grid layouts. In continuing work, we are using this method as a pre-processing approach for HRTF personalization utilizing multiple HRTF databases.

Head-related transfer function (HRTF) interpolation plays an important role for implementation of 3D sound system because it can not only reduce the number of measurements for HRTFs, but also reduce the data of HRTFs for seamless binaural synthesis. This paper addresses the problem of accurately realizing the interpolation of HRTF for synthesis of virtual auditory space, and proposes a HRTF interpolation method based on principal component analysis. Firstly, the HRTF is decomposed into principal components and corresponding principal component weights, where principal components are direction-independent and principal component weights are direction- dependent; then the directional variation of the principal component weight is multivariate polynomial fitted with a bivariate function of two spatial angulus (azimuth and elevation). Moreover, a sphere- partitioning optimization scheme is employed to improve the approximation precision. Experiment results demonstrate that HRTFs in the entire sphere surface can be interpolated by the proposed method with small distortion, and the proposed method performs better than conventional methods. Therefore the proposed method gives a promising way for HRTF interpolation.

Applications in the creation of virtual auditory spaces (VAS) and sonification require individualized head related transfer functions (HRTFs) for perceptual fidelity. HRTFs exhibit significant variation from person to person due to differences between their pinnae, and their body sizes. We propose and preliminarily implement a simple HRTF customization based on the use of a published database of HRTFs (Algazi et al., 2001) that also contains geometrical measurements of subject pinnae. We measure some of these features via simple image processing, and select the HRTF that has features most closely corresponding to the individual's features. This selection procedure is implemented along with the virtual auditory system described in (Zotkin et al., 2002), and listener tests conducted comparing the customized HRTF and a fixed HRTF. Despite the simplicity of the method, tests reveal average improvement in localization accuracy of about 25 percent, though performance improvement varies with source location and individuals.

BackgroundIn order to present virtual sound sources via headphones spatially, head-related transfer functions (HRTFs) can be applied to audio signals. In this so-called binaural virtual acoustics, the spatial perception may be degraded if the HRTFs deviate from the true HRTFs of the listener.ObjectiveIn this study, participants wearing virtual reality (VR) headsets performed a listening test on the 3D audio perception of virtual audiovisual scenes, thus enabling us to investigate the necessity and influence of the individualization of HRTFs. Two hypotheses were investigated: first, general HRTFs lead to limitations of 3D audio perception in VR and second, the localization model for stationary localization errors is transferable to nonindividualized HRTFs in more complex environments such as VR.MethodsFor the evaluation, 39 subjects rated individualized and nonindividualized HRTFs in an audiovisual virtual scene on the basis of 5 perceptual qualities: localizability, front-back position, externalization, tone color, and realism. The VR listening experiment consisted of 2 tests: in the first test, subjects evaluated their own and the general HRTF from the Massachusetts Institute of Technology Knowles Electronics Manikin for Acoustic Research database and in the second test, their own and 2 other nonindividualized HRTFs from the Acoustics Research Institute HRTF database. For the experiment, 2 subject-specific, nonindividualized HRTFs with a minimal and maximal localization error deviation were selected according to the localization model in sagittal planes.ResultsWith the Wilcoxon signed-rank test for the first test, analysis of variance for the second test, and a sample size of 78, the results were significant in all perceptual qualities, except for the front-back position between own and minimal deviant nonindividualized HRTF (P=.06).ConclusionsBoth hypotheses have been accepted. Sounds filtered by individualized HRTFs are considered easier to localize, easier to externalize, more natural in timbre, and thus more realistic compared to sounds filtered by nonindividualized HRTFs.

Spatial impression is an important acoustic quality of concert halls. An accepted objective measure for spatial impression is the interaural cross-correlation (IACC) coefficient. Recently, spherical microphone arrays have been studied for room acoustics analysis and music recordings. This study presents a theoretical formulation for the computation of IACC using spherical-harmonics representations of the sound field, as measured by spherical microphone arrays, and spherical-harmonics representation of head-related transfer functions (HRTFs), taken from HRTF databases. As spherical microphone arrays typically use a finite number of microphones, they may not be able to capture the complete spatial information in a sound field. Therefore, the effect of limited spherical-harmonics order on the accuracy of IACC approximation using the proposed method is studied using simulated and measured data. The method presented in this paper can be further used to study the effect of limited spatial information on the spatial perception of sound fields.

Interaural time delay (ITD) is the dominant cue to sound-source location. When one component of a complex sound changes rapidly but smoothly in ITD, it perceptually segregates from the complex. When different components were changed in ITD in succession, a recognizable melody was heard. Each note was more detectable from the transition than from a distinct ITD with respect to the complex. However, a transition relative to a coherently changing complex produced no segregation, whereas cyclic modulation of ITD, too rapid to be perceived as movement, did produce segregation. These two results suggest that the relevant cue is not movement per se but rather the lack of a well-defined ITD during the transition. Segregation largely disappeared when the transition in ITD was replaced by a temporal gap in the complex of the order of 100 ms. The effect seems similar to visual motion segregation but is best explained by the mechanism of binaural unmasking.

Head-related transfer functions (HRTFs) describe the acoustical transmission process from a point sound source to two ears in the free field. They are vital to the researches of binaural hearing and virtual auditory display. Measurement is the most important approach to obtain HRTFs, but it is time-consuming. To accurate HRTF measurement, a computer-controlling measurement system is designed. The system consists of multiple sound sources at different elevations with a finest interval of 5° and a horizontal turntable with an azimuthal resolution of 0.1°. It is able to work in non-anechoic environment and suitable for both mannequin and human subject measurement. A practical example indicates that the system allows for measuring far-field HRTFs at 493 source directions within half an hour. By using the system, a super high directional resolution HRTF database for KEMAR mannequin is established. The database includes 3889 pairs of head-related impulse responses (HRIRs, the time domain counterpart of HRTFs) at distance of 1.0 m, elevation interval of 5° (from -45° to 90° ), and azimuthal interval of 2.5°. Each HRIR is 1024-point length at 96 kHz sampling frequency and 24-bit quantization. The database is applicable to the research on binaural hearing and virtual auditory display.

While various methods for sound source localization have been developed, most of them utilize on the time difference of arrival (TDOA) between microphones or the measured head related transfer functions (HRTF). In case of a real robot implementation, the former has a merit of light computation load to estimate the sound direction but can not consider the effect of platform on TDOAs, while the latter can, because characteristics of robot platform are included in HRTF. However, the latter needs large resources for the HRTF database of a specific robot platform. We propose the compensation method which has the light computation load while the effect of platform on TDOA can be taken into account. The proposed method is used with spherical head related transfer function (SHRTF) on the assumption that robot platform, for example a robot head, installed microphones can be modeled to a sphere. We verify that the proposed method decreases the estimation error caused by the robot platform through the simulation.

Known errors exist in loudspeaker array processing techniques, often degrading source localization and timbre. The goal of the present study was to use virtual loudspeaker arrays to investigate how treatment of the interaural time delay (ITD) cue from each loudspeaker impacts these errors. Virtual loudspeaker arrays rendered over headphones using head-related impulse responses (HRIRs) allow flexible control of array size. Here, three HRIR delay treatment strategies were evaluated using minimum-phase loudspeaker HRIRs: reapplying the original HRIR delays, applying the relative ITD to the contralateral ear, or separately applying the HRIR delays prior to virtual array processing. Seven array sizes were simulated, and panning techniques were used to estimate HRIRs from 3000 directions using higher-order Ambisonics, vector-base amplitude panning, and the closest loudspeaker technique. Compared to a traditional, physical array, the prior HRIR delay treatment strategy produced similar errors with a 95% reduction in the required array size. When compared to direct spherical harmonic (SH) fitting of head-related transfer functions (HRTFs), the prior delays strategy reduced errors in reconstruction accuracy of timbral and directional psychoacoustic cues. This result suggests that delay optimization can greatly reduce the number of virtual loudspeakers required for accurate rendering of acoustic scenes without SH-based HRTF representation.

Binaural localization of speech sources in 3-D, using head-related transfer functions (HRTFs), always suffers elevation ambiguity due to the limited high frequency spectral information available at the receivers. This paper presents a method that overcomes this limitation by exploiting the interaural phase and magnitude features present in the HRTF. We (i) introduce a new feature vector that combines these two sets of features in a non-linear fashion, and (ii) propose a mechanism to extract this feature vector free from distortion by the speech spectra. The performance of the proposed method is evaluated and compared with a correlation-based HRTF database matching approach and a two-step localization technique for multiple source positions, HRTFs (individuals) and speech inputs. The results suggest that up to 20% improvement in localization performance can be achieved for moderate signal-to-noise ratios.

Head-related transfer functions (HRTFs) reflect the interaction between human anatomical structures and sound waves, and depend on individuals due to differences in anatomical structures among individuals. This work investigates inter-individual differences of spectral cues encoded in median-plane individual HRTFs using principle component analysis (PCA). First, magnitudes of individual HRTFs from the CIPIC database are decomposed and reduced dimensions by PCA. Then, individual expansion weights of PCA are extracted. Finally, the inter-individual differences in expansion weights are analyzed using standard deviation. Results show that the inter-individual spectral difference in HRTFs decreases with increasing expansion degree of PCA, and the spatial distribution of large inter-individual difference is always at low elevations. When the expansion degree is 1 and 2, the mean of individual expansion weights mainly reflects the inter-elevation difference. The same method is also applied to a combined HRTF database including 295 subjects.

We consider immersive rendering of a set of sound sources, apparently positioned at different locations in three-dimensional (3-D) space around a listener. Each sound source is associated with a subband-coded audio stream. A low-complexity rendering approach, based on reduced-rank models of subband-domain Head-related transfer functions (HRTFs), is proposed in this paper. HRTFs are approximated using a set of basis vectors in subband-domain. Immersive rendering of partially decoded audio stream is performed in subband-domain using the basis vectors. Perceptual objective error measurements and subjective test results indicate, despite low computational complexity, this technique is effective in rendering of coded audio streams in 3-D space.

Head-related transfer functions (HRTFs) measured from human subjects were approximated using infinite-impulse-response (IIR) filter models. Models were restricted to rational transfer functions (plus simple delays) so that specific models are characterized by the locations of poles and zeros in the complex plane. The all-pole case (with no nontrivial zeros) is treated first using the theory of linear prediction. Then the general pole-zero model is derived using a weighted-least-squares (WLS) formulation of the modified least-squares problem proposed by Kalman (1958). Both estimation algorithms are based on solutions of sets of linear equations and result in efficient computational schemes to find low-order model HRTFs. The validity of each of these two low-order models was assessed in psychophysical experiments. Specifically, a four-interval, two-alternative, forced-choice paradigm was used to test the discriminability of virtual stimuli constructed from empirical and model HRTFs for corresponding locations. For these experiments, the stimuli were 80 ms, noise tokens generated from a wideband noise generator. Results show that sounds synthesized through model HRTFs were indistinguishable from sounds synthesized from original HRTF measurements for the majority of positions tested. The advantages of the techniques described here are the computational efficiencies achieved for low-order IIR models. Properties of the all-pole and pole-zero estimators are discussed in the context of low-order HRTF representations, and implications for basic and applied contexts are considered.

Neural networks were trained with back propagation to localize ‘‘virtual’’ sounds that could originate from any of 24 azimuths (−165° to +180°) and 6 elevations (−36° to +54°). The sounds (clicks) were filtered with head related transfer functions and transformed into 22-point quarter-octave spectra. The networks were composed of 22 to 44 input nodes, followed by 50 hidden nodes and 30 output nodes (24 azimuth nodes and 6 elevation nodes). With the ‘‘binaural’’ configuration, the interaural delay and the interaural difference spectrum were provided as inputs to the network. With the ‘‘monaural’’ configuration, separate networks were trained with the left-ear and the right-ear spectra; a third, arbitrator, network learned to localize based on the output of these two monaural networks (i.e., the activation levels of their azimuth and elevation nodes). The monaural configuration did not allow for binaural interaction in the traditional sense. Both configurations achieved performance comparable to humans, suggesting that for these conditions either monaural cues or binaural cues are sufficient to explain human sound localization. [Work supported by NIH-DC-00786, AFOSR-91-0289, and AFOSR-Task 2313V3.]

A high voltage damped oscillating wave generator with variable frequency has been developed using power devices (IGBT), photo devices, a designed noncore transformer and a function-generator in order to diagnose insulating materials as power apparatus (transformer, cable), enameled wires, printed circuit board (PCB), wire-harness in automobiles. The generated maximum voltage (2.5 kV) and the frequency dependency (0.01-2 kHz) were confirmed in output tests.