Combined speech and audio coding with bit rate and bandwidth scalability

This paper describes the progress in frequency-domain linear prediction coding (LPC)-based audio coding schemes. Although LPC was originally used only for time-domain speech coders, it has been applied to frequency-domain coders since the late 1980s. With the progress in associated technologies, the frequency-domain LPC-based audio coding scheme has become more promising, and it has been used in speech/audio coding standards, such as MPEG-D unified speech and audio coding and 3GPP enhanced voice services since 2010. Three of the latest investigations on the representations of LPC envelopes in frequency-domain coders are shown. These are the harmonic model, frequency-resolution warping and the Powered All-Pole Spectral Envelope, all of which are aiming at further enhancement of the coding efficiency.

This paper addresses the problem of reconstructing wideband speech signals from observed narrowband speech signals. The goal of this work is to improve the perceived quality of speech signals which have been transmitted through narrowband channels or degraded during acquisition. We describe a system, based on linear predictive coding, for estimating wideband speech from narrowband. This system employs both previously identified and novel techniques. Experimental results are provided in order to illustrate the system's ability to improve speech quality. Both objective and subjective criteria are used to evaluate the quality of the processed speech signals.

Electronic speech coding had its origin in 1928 when Homer Dudley proposed to send speech signals over a new transatlantic telegraph cable with the (then) enormous bandwidth of 100 Hz. Dudley argued that speech was generated by slowly moving articulators and should therefore require only some 100-Hz total bandwidth for transmission. Since extraction of these parameters proved difficult, Dudley suggested, as an alternative, the transmission of the likewise slowly changing spectral information. The result was the channel vocoder, which (together with formant and voice-excited vocoders) dominated speech coding and that of musical sounds of single-pitch instruments until the mid 1960s. Unhappy with the vocoder's “electronic accent.” Bishnu Atal and the author, in 1967, advanced the idea of linear predictive coding (LPC), motivated by the desire to replace rigid vocoderlike coding by a more flexible method in which a “prediction residual” could represent deviations of real speech signals from an assumed all-pole model. In 1972, subjective error criteria were introduced, based on auditory models of masking and the possibility of “hiding” quantizing noise under the speech signal. The resulting code-excited linear predictive (CELP) coders based on statistical models have been operated at 1/4 bit per sample for the residual at virtually no audible degradation. With powerful digital signal processing chips, CELP has now become the method of choice for synthetic-voice applications. For musical sounds, masking models and linear predictive coding, which is equivalent to prewhitening of the spectrum, lead to a substantial reduction in the required number of bits per sample.

Speech technologies have been developed for decades as a typical signal processing area, while the last decade has brought a huge progress based on new machine learning paradigms. Owing not only to their intrinsic complexity but also to their relation with cognitive sciences, speech technologies are now viewed as a prime example of interdisciplinary knowledge area. This review article on speech signal analysis and processing, corresponding machine learning algorithms, and applied computational intelligence aims to give an insight into several fields, covering speech production and auditory perception, cognitive aspects of speech communication and language understanding, both speech recognition and text-to-speech synthesis in more details, and consequently the main directions in development of spoken dialogue systems. Additionally, the article discusses the concepts and recent advances in speech signal compression, coding, and transmission, including cognitive speech coding. To conclude, the main intention of this article is to highlight recent achievements and challenges based on new machine learning paradigms that, over the last decade, had an immense impact in the field of speech signal processing.

Sinusoidal representation has been widely applied to speech modification, low bit rate speech and audio coding. Usually, speech signal is analyzed and synthesized using the overlap-add algorithm or the peak-picking algorithm. But the overlap-add algorithm is well known for high computational complexity and the peak-picking algorithm cannot track the transient and syllabic variation well. In this letter, both algorithms are applied to speech analysis/synthesis. Peaks are picked in the curve of power spectral density for speech signal; the frequencies corresponding to these peaks are arranged according to the descending orders of their corresponding power spectral densities. These frequencies are regarded as the candidate frequencies to determine the corresponding amplitudes and initial phases according to the least mean square error criterion. The summation of the extracted sinusoidal components is used to successively approach the original speech signal. The results show that the proposed algorithm can track the transient and syllabic variation and can attain the good synthesized speech signal with low computational complexity.

For speech coders which fall within the class of waveform coders, the reconstructed signal approaches the original with increasing bit rate. In such coders, the distortion criterion generally operates on the speech signal or a signal obtained by adaptive linear filtering of the speech signal. To satisfy computational and delay constraints, the distortion criterion must be reduced to a very simple approximation of the auditory system. This drawback of conventional approaches motivates a new speech coding paradigm in which the coding is performed in a domain where the single-letter squared-error criterion forms an accurate representation of perception. The new paradigm requires a model of the auditory periphery which is accurate, can be be inverted with relatively low computational effort, and which represents the signal with relatively few parameters. We develop such a model of the auditory periphery and discuss its suitability for speech coding. The results indicate that the new paradigm in general and our auditory model in particular form a promising basis for the coding of both speech and audio at low bit rates.

Recent advances in speech coding algorithms and techniques based on the use of linear prediction now permit high quality voice reproduction at remarkably low bit rates. This paper reviews some of the main ideas underlying the algorithms of major interest today. The concept of removing redundancy by linear prediction is reviewed, first in the context of predictive quantization or DPCM. Then linear predictive coding, adaptive predictive coding, and vector quantization are discussed. The concepts of excitation coding via analysis-by-synthesis linear predictive coding is explained and some important enhancements such as vector sum excitations, and adaptive postfiltering are described. Low-delay coding by backward computation of LPC parameters is explained. The concept of phonetic segmentation of speech for closed-loop coding systems is also presented. Linear prediction is the key technique that underlies almost all of the important algorithms for speech coding of interest today. Finally, we discuss some recent work on nonlinear prediction of speech and its potential for the future of speech coding.

Audio coding involves balancing data rate and system complexity limitations against needs for high-quality audio. While audio quality is a fundamental concept in audio coding, it remains very difficult to describe it in objective terms. Traditional quality measurements such as the signal to noise ratio or the total block distortion provide simple, objective measures of audio quality but they ignore psychoacoustic effects that can lead to large differences in perceived quality. In contrast, perceptual objective measurement schemes, which rely upon specific models of hearing, are subject to the criticism that the predicted results do not correlate well with the perceived audio quality. While neither simple objective measures nor perceptual measures are considered fully satisfactory, audio coding has traditionally relied on formal listening tests to assess a system’s audio quality when a highly accurate assessment is needed. After all, human listeners are the ultimate judges of quality in any application.

This paper reviews some important technologies for speech and audio coding, contrasting time domain and frequency domain processing. These technologies have contributed to the standardization activities in ITU-T and ISO/IEC MPEG, respectively. It is interesting that the standardization targets for speech and audio have started to converge recently. The MPEG USAC (Unified Speech and Audio Coding) scheme, which is being actively developed, may make use of most of the important technologies for speech and audio that have been developed so far.

The aim of the study was to inspect the acoustic properties and sleep characteristics of a preapneic snoring sound. The feasibility of forecasting upcoming respiratory events by snoring sound was also investigated. Participants with habitual snoring or a heavy breathing sound during sleep were recruited consecutively. Polysomnography was conducted, and snoring-related breathing sound was recorded simultaneously. Acoustic features and sleep features were extracted from 30-second samples, and a machine learning algorithm was used to establish 2 prediction models. A total of 74 eligible participants were included. Model 1, tested by 5-fold cross-validation, achieved an accuracy of 0.92 and an area under the curve of 0.94 for respiratory event prediction. Model 2, with acoustic features and sleep information tested by Leave-One-Out cross-validation, had an accuracy of 0.78 and an area under the curve of 0.80. Sleep position was found to be the most important among all sleep features contributing to the performance of the 2 models. Preapneic sound presented unique acoustic characteristics, and snoring-related breathing sound could be deployed as a real-time apneic event predictor. The models, combined with sleep information, serve as a promising tool for an early warning system to forecast apneic events. Wang B, Yi X, Gao J, etal. Real-time prediction of upcoming respiratory events via machine learning using snoring sound signal. J Clin Sleep Med. 2021;17(9):1777-1784.

A very efficient speech coding scheme with a compression ratio of up to 120:1 is described. The difference between the fast Fourier transform (FFT) of two consecutive frames of speech is considered to be a two-dimensional image in the frequency domain. Using linear contractive transformation; this image can be reduced to a set of rules and coefficients that could either be transmitted or stored. The speech signal is reconstructed using these coefficients with minimal loss of signal quality. The output speech is superior in quality to that of linear predictive coding (LPC) speech. This algorithm can be applied equally to one-and two-dimensional signals, providing a possible solution for the hypermedia problem. >

This paper describes about speech recognition implemented on ATmega162 microcontroller. The word (voice command) in a speech signal is applied for controlling movement of a mobile robot. The mobile robot will move according to the voice command in the speech signal. There are five commands, which are Indonesian language, used to control the movement of mobile robot. They are “maju”, “mundur”, “kiri”, “kanan”, and “stop” used to command the mobile robot to move forward, move backward, turn left, turn right and stop respectively.The methods used in this research are Linear Predictive Coding (LPC) and Hidden Markov Model (HMM). LPC is used to extract word data from a speech signal. HMM is used to recognize the word pattern data, which are extracted from a speech signal. Sampling rate of the speech signal is 8 kHz and the speech signal is sampled for 0.5 seconds. Experiments were done in several variations of observation symbol number and number of sample. The experimental results show that the highest recognition rate that can be achieved by the system is 87%. The mobile robot can move in accordance with the voice command that is given to the robot.

Linear predictive coding (LPC) residual is an important component for analysis-by-synthesis speech coders. Using spectral envelope vector quantisation (SEVQ) for LPC residual signals, MPEG-4 harmonic vector excitation coding (HVXC) speech coder is able to change the tone and speed for decoding bitstreams. A modified version is proposed to the simplified hearing-based SEVQ (SSEVQ) approach proposed by Wang and Yang. In addition, the energy first search scheme for SEVQ (EFSEVQ) is proposed in order to reduce the computational complexity of SEVQ for MPEG-4 HVXC speech coder. The proposed EFSEVQ scheme also agrees with human hearing properties. Simulation results reveal that the proposed EFSEVQ search scheme not only reduces the computational complexity but also preserves the quality of encoded speech.

In this paper, we introduce the compressed HDTV over IPv6 system, applications, and its experiment in IPv6 network between Korea and China. We developed the compressed HDTV over IPv6 system which requires 20-25Mbps with MPEG2. The system use 720/30p or 1080i HDTV format and support IPv4/IPv6 and unicast/multicast. The compressed HDTV system provides high quality audio and video with relatively small bandwidth comparing with uncompressed HDTV system, but it suffers about 1 second delay for MPEG2 encoding and decoding. Its high quality video and audio enables applications such as remote lecture/seminar with limited interaction. Using compressed HDTV system, we had two-way experiment which requires 40-50Mbps bandwidth in Gigabit IPv6 network in dual stack mode. This paper contains brief experiment result including jitter and packet loss. We expect practical application of compressed HDTV system such as remote lecture and seminar between Korea and China.

본 논문에서는 음성/음악 통합 압축 기술(USAC)에 적용된 공간 오디오 부호화 기법인 MPEG 서라운드에서 채널 간 위상차(IPD) 파라미터를 효과적으로 생략하는 기법에 대해 다룬다. 기존의 방법에서는 파라미터 밴드의 IPD 파라미터가 모두 작은 경우에도 전체를 처리하여 전송한다. 이러한 경우 생략을 통해 비트 효율을 향상시킬 수 있다. IPD 파라미터의 심리음향적 민감도를 고려하여 청취 환경에서 인지하지 못하는 문턱 값을 추정하고, 문턱 값 아래의 위상차를 발생시키는 IPD 파라미터는 0으로 설정하고 전송하지 않는다. 제안하는 채널 간 위상차 파라미터 생략 기법을 적용하면 기존에 비해 38% 정도의 위상 파라미터 절감 효과를 얻을 수 있고, MUSHRA 청취 실험을 통해 복호화된 오디오의 음질 하락이 없음을 보인다. This paper deals with an efficient method for skipping inter-channel phase differences (IPD) in the MPEG surround of the unified speech and audio coding (USAC). Based on the psycho-acoustic sensitivity on the IPD, we estimate a threshold on IPD, below which we can not notice degradation in spatial cue. We propose an IPD skip method, in which any IPDs within the threshold are set to zero and are not transmitted. The proposed IPD skip method gives about 38% savings in terms of bit amount for IPD. Nevertheless, in the MUSHRA test, the proposed method does not show any noticeable degradation in the decoded audio quality.