Content-based audio classification and segmentation by using support vector machines

Identifying different audio segments in videos is the first step for many important tasks such as event detection and speech transcription. Approaches using Mel-Frequency Cepstral coefficients (MFCCs) with Gaussian mixture models (GMMs) and hidden Markov models (HMMs) perform reasonably well in stationary conditions but do not scale to a broad range of environmental conditions. This paper focuses on the audio segmentation in broadcast soccer videos into audio classes such as Silence, Speech Only, Speech Over Crowd, Crowd Only and Excited, with an alternative feature set which is simplistic as well as robust to changes in the environment conditions. Support Vector Machines (SVMs), Neural Networks and Random Forest are used for the classification. The accuracy achieved with SVMs, Neural Networks and Random Forest are 83.80%, 86.07%, and 88.35% respectively. The proposed features and Random Forest classifier are found to achieve better accuracy compared to the other classifiers.

In recent times, there have been increased rates at which researchers are searching for advanced ways of carrying out land-use land-cover (LULC) mapping, especially in developing countries. Four machine-learning algorithms, namely Random Forest (RF), Support Vector Machine (SVM), K-Nearest Neighbour (K-NN), and Gaussian Mixture Models (GMM) were examined. This study also attempted to validate the various models using the index-based validation method. Accuracy assessment was performed by using the Kappa coefficient. The results of the LULC showed that RF classified 23% of the study area as bare land, SVM has 24% of the study area classified as bare land, K-NN also allotted 24% to bare land, while that of GMM classifier was 30%. The overall accuracy of RF, SVM, K-NN and GMM were 0.9840, 0.9780, 0.9641 and 0.9421 respectively. The Kappa Coefficient of the various classifiers were RF (0.9695), SVM (0.9580), K-NN (0.9319) and GMM (0.8916). This study showed that though all the algorithms performed relatively very well, RF performed better than the other classifiers. It suffices to state that, there is a need for further studies since other extraneous environmental variables may be underpinning these conclusions.

For the safe and economical construction of embankment dams, the mechanical behaviour of the rockfill materials used in the dam’s shell must be analyzed. The characterization of rockfill materials with specified shear strength is difficult and expensive due to the presence of particles greater than 500 mm in diameter. This work investigates the feasibility of using an extreme gradient boosting (XGBoost) computing paradigm to estimate the shear strength of rockfill materials. To train and validate the proposed XGBoost model, a total of 165 databases obtained from the literature are chosen. The XGBoost model was compared against support vector machine (SVM), adaptive boosting (AdaBoost), random forest (RF), and K‐nearest neighbor (KNN) models described in the literature. XGBoost beats SVM, RF, AdaBoost, and KNN models in terms of performance evaluation metrics such as coefficient of determination ( R 2 ), Nash–Sutcliffe coefficient (NSE), and error in the root mean square ratio (RMSE) to the standard deviation of the measured data (RSR). The results demonstrated that the XGBoost model has the highest prediction performance with ( R 2 = 0.9707, NSE = 0.9701, and RSR = 0.1729), followed by the SVM model with ( R 2 = 0.9655, NSE = 0.9639, and RSR = 0.1899), RF ( R 2 = 0.9545, NSE = 0.9542, and RSR = 0.2140), the AdaBoost model with ( R 2 = 0.9390, NSE = 0.9388, and RSR = 0.2474) and the KNN model with ( R 2 = 0.6233, NSE = 0.6180, and RSR = 0.6181). A sensitivity analysis has been conducted to ascertain the impact of each investigated input parameter. This study demonstrates that the established XGBoost model for estimating the shear strength of rockfill materials is reliable.

Existing studies have attempted to determine the tool chipping condition using the indirect method of data capture and intelligent analysis techniques considering machine parameters, and tool conditions using signal processing techniques. Due to the obstructive nature of the machining operation, however, it is daunting to use signal capturing to intelligently capture the condition of the tool as well as that of the workpiece. This study aimed to apply some advanced signal processing techniques to the vibration signals captured experimentally during machining operation for the decision making and analysis of tool and workpiece conditions. Vibration signals were captured during turning operations while using four (4) classes of tools, based on their flank wear. The signals were first pre-processed and decomposed using the Empirical Mode Decomposition (EMD) method. The Hilbert–Huang transform (HHT) was applied to the resulting IMFs obtained to compute the feature vectors used to classify the condition of the tool and workpiece. A total of 12 features, consisting of instantaneous properties such as instantaneous energy, instantaneous frequencies, and amplitudes, were obtained for data training and classification of tool conditions. To optimize the classification process, feature selection was performed using a genetic algorithm (GA) to reduce the number of features from 12 to 4 for data training and classification. The feature vectors were first trained for tool classification with a neural network scaled conjugate gradient (SCG) algorithm. The result showed that the model classification error was 0.102. Two other machine learning models, support vector machine (SVM) and K-Nearest Neighbors (KNN), were also implemented for classifying the tool conditions, from the feature vector, to determine the model that most accurately predicted the condition of the tool. To avoid bias and reduce misclassification errors, the k-fold cross-validation technique was applied with ‘k’ taken as 5 and 10. The computed feature vectors were used as inputs to train the machine learning model using both SVM and KNN models to classify the tool and workpiece condition during machining. The error loss of each model was evaluated and plotted to review the performance. The average overall error loss of 0.5031 was observed for the SVM model with 5-fold cross-validation, whereas the error loss of 0.0318 was observed for the KNN model with 5-fold cross-validation. The average overall error loss of 0.5009 was observed for the SVM model with 10-fold cross-validation when trained using the features selected by a genetic algorithm (GA), while the average overall error loss of 0.0343 was observed for the KNN model. The optimal performance of the SVM model was obtained when all features were used for the training, whereas the KNN model performed better when feature selection was implemented. The error losses of the models were evaluated to be less in KNN models, compared to SVM and SCG. The obtained results also showed that the developed KNN models performed 10 times better than the SVM model in predicting the tool condition from the captured vibration signal during the machining process.

In this paper, we describe an automatic- speaker based- audio segmentation and identification system for broadcasted news indexation purposes. We specifically focus on speaker identification and audio scene detection. Speaker identification (SI) is based on the state of the art Gaussian mixture models, whereas scene change detection process uses the classical Bayesian Information Criteria (BIC) and the recently proposed DISTBIC algorithm. In this work, the effectiveness of Mel Frequency Cepstral coefficients MFCC, Linear Predictive Cepstral Coefficients LPCC, and Log Area Ratio LAR coefficients are compared for the purpose of text-independent speaker identification and speaker based audio segmentation. Both the Fisher Discrimination Ratio-feature analysis and performance evaluation in terms of correct identification rate on the TIMIT database showed that the LPCC outperforms the other features especially for low order coefficients. Our experiments on audio segmentation module showed that the DISTBIC segmentation technique is more accurate than the BIC procedure especially in the presence of short segments.

This paper proposes a security-monitoring instrument that can detect and classify the location and nature of different sounds in a room. The instrument is reliable and robust even in the presence of reverberation and in low signal to noise ratio conditions. This paper proposes a new algorithm for classifying first an audio segment as speech or nonspeech then classifies the nonspeech audio segment into its own audio type. The algorithm divides an audio segment into frames, estimates the presence of pitch in each frame, and calculates a pitch ratio parameter. This parameter is then used to classify the audio segment. The threshold used in calculating this parameter is adapted to accommodate different environments. Nonspeech audio segment has further classification using time delayed neural network to be classified into it is own type. The performance of the proposed algorithm is evaluated for different signal-to-noise ratios using a library of audio segments. The library includes speech segments and nonspeech segments such as windows breaking and footsteps. Using 0.4 second segments it is shown that the proposed algorithm can achieve an average correct decision for 94.5% of the reverberant library and 95.1% of the nonreverberant library

The rising machine learning (ML) models have become the preferred way for landslide detection based on remote sensing images, but the performance of these models in a sample-free area are rarely concerned in many studies. In this study, we used a cross-validation method (training model in one area and validation in another) to compare the model portability of trained ML models applied in an “off-site” area, as a consideration of the landslide detection ability of these models in sample-free areas. We integrate nighttime light imagery, multi-seasonal optical Landsat time-series and digital elevation data, and we employed support vector machines (SVM), artificial neural networks (ANN) and random forest (RF) models to classify the satellite imagery and identify landslides. Samples of two scenarios generated from two subareas of the Jiuzhaigou disaster-stricken region are used for the cross-application and accuracy evaluation of three ML models. The results revealed that when the trained models are applied in areas outside those in which they were developed, the landslide identification accuracy of these three models has declined. Especially for the SVM and ANN models, the accuracy is greatly reduced and there appears a seriously imbalanced user’s and producer’s accuracy. However, although the performance of the RF model is lower than that of SVM and ANN models in their local area, the RF model exhibits stable portability, and retains the original performance and achieves a satisfactory balance between overestimation and underestimation in “off-site” areas. An additional validation from a new area proved that the landslide detection performance of the RF model with stable portability is higher than that of the SVM and ANN models in “off-site” areas. The results suggest that evaluating the model portability through cross-application can be a useful way to determine the most suitable model for landslide detection in “off-site” areas with a similar geographic environment to model development areas, so as to maximize the accuracy of landslide detection based on limited samples.

In this paper, we propose a golf video browsing system using three stage audio segmentation. To increase the accuracy of audio segment boundaries for golf video browsing on digital video recorder, the proposed three-stage audio segmentation algorithm consists of Gaussian Mixture Model (GMM)-based classification, GMM-Linear Discriminant Analysis (LDA)-based classification, and occupation ratio-based segmentation. The detected audio segment is synchronized with the video frame for the extraction of thumbnails for use in the intelligent browsing of golf videos.

Classification model for blast furnace status based on multi-source information

We focus the attention on the area of generic and automatic audio classification and segmentation for audio-based multimedia indexing and retrieval applications. In particular, we present a fuzzy approach toward hierarchic audio classification and global segmentation framework based on automatic audio analysis providing robust, bi-modal, efficient and parameter invariant classification over global audio segments. The input audio is split into segments, which are classified as speech, music, fuzzy or silent. The proposed method minimizes critical errors of misclassification by fuzzy region modeling, thus increasing the efficiency of both pure and fuzzy classification. The experimental results show that the critical errors are minimized and the proposed framework significantly increases the efficiency and the accuracy of audio-based retrieval especially in large multimedia databases.

Many downscaling techniques have been developed in the past few years for projection of station‐scale hydrological variables from large‐scale atmospheric variables simulated by general circulation models (GCMs) to assess the hydrological impacts of climate change. This article compares the performances of three downscaling methods, viz. conditional random field (CRF), K‐nearest neighbour (KNN) and support vector machine (SVM) methods in downscaling precipitation in the Punjab region of India, belonging to the monsoon regime. The CRF model is a recently developed method for downscaling hydrological variables in a probabilistic framework, while the SVM model is a popular machine learning tool useful in terms of its ability to generalize and capture nonlinear relationships between predictors and predictand. The KNN model is an analogue‐type method that queries days similar to a given feature vector from the training data and classifies future days by random sampling from a weighted set of K closest training examples. The models are applied for downscaling monsoon (June to September) daily precipitation at six locations in Punjab. Model performances with respect to reproduction of various statistics such as dry and wet spell length distributions, daily rainfall distribution, and intersite correlations are examined. It is found that the CRF and KNN models perform slightly better than the SVM model in reproducing most daily rainfall statistics. These models are then used to project future precipitation at the six locations. Output from the Canadian global climate model (CGCM3) GCM for three scenarios, viz. A1B, A2, and B1 is used for projection of future precipitation. The projections show a change in probability density functions of daily rainfall amount and changes in the wet and dry spell distributions of daily precipitation. Copyright © 2011 John Wiley & Sons, Ltd.

An on-line audio classification and segmentation system is presented in this research, where audio recordings are classified and segmented into speech, music, several types of environmental sounds and silence based on audio content analysis. This is the first step of our continuing work towards a general content-based audio classification and retrieval system. The extracted audio features include temporal curves of the energy function,the average zero- crossing rate, the fundamental frequency of audio signals, as well as statistical and morphological features of these curves. The classification result is achieved through a threshold-based heuristic procedure. The audio database that we have built, details of feature extraction, classification and segmentation procedures, and experimental results are described. It is shown that, with the proposed new system, audio recordings can be automatically segmented and classified into basic types in real time with an accuracy of over 90 percent. Outlines of further classification of audio into finer types and a query-by-example audio retrieval system on top of the coarse classification are also introduced.

The results of directly comparing the prediction accuracy of optimized 3D Quantitative Structure-Activity Relationship (3D-QSAR) models and linear Support Vector Machine (SVM) classifiers to identify small molecule inhibitors of the BRAF-V600E and HIV Integrase targets are reported. Performance comparisons were carried out using 303 compounds (68 active) against BRAF-V600E and 204 compounds (159 active) against HIV Integrase. A SVM prediction accuracy of 95% (BRAF-V600E) and 100% (HIV Integrase) and 3D-QSAR prediction accuracy of 76% (BRAFV600E) and 82% (HIV Integrase) was observed. To help explain the better performance of SVM in the comparison reported here and to help assess the degree to which a SVM or 3D-QSAR model is likely to perform best for other targetligands of interest a new EPP (Expected Predictive Performance) metric is introduced. How EPP can be used to help predict future performance of SVM and 3D-QSAR models by quantifying the degree of similarity between candidate compounds and training data is also demonstrated. Results show that the EPP metric is capable of predicting future prediction accuracy of SVM and 3D-QSAr models within 7% of actual performance.

Improving sea level prediction in coastal areas using machine learning techniques

Recently, there has been a remarkable growth in Artificial Intelligence (AI) with the development of efficient AI models and high-power computational resources for processing complex datasets. There has been a growing number of applications of machine learning in satellite remote sensing image data processing. In this work, machine learning methods were applied for crop classification of temporal multi- spectral satellite image to achieve better prediction of crop-wise area statistics. In India, agriculture has a huge impact on the national economy and most of the critical decisions are dependent on agricultural statistics. Sentinel-2 satellite image data for the Guntur district region of Andhra Pradesh, India has been used as the study area. The main reason for selecting this region is the diversity of agricultural crops and the availability of ground truth. As a baseline, the performance of machine learning models like Support Vector Machine (SVM), Random Forest (RF), Decision Tree (DT), and K-Nearest Neighbor (KNN) on crop classification was evaluated on 2016 single time multi-spectral satellite image. SVM performed well with an F1 score of 0.94. Further, the performance of SVM, RF, 1D and 2D Convolutional Neural Network (CNN), Recurrent Neural Network (RNN) with Long Short-Term Memory (LSTM) and RNN with Gated Recurrent Unit (GRU) on 2018 Kharif season temporal multi-spectral satellite image has been evaluated for estimation of crop areas. The results show that SVM has the best F1 score of 0.99 and achieved a 95.9% agreement with the ground surveyed crop areas.