Audio contributions to semantic video search

Detection and recognition of loud sounds and characteristic noises can significantly increase the level of safety and ensure timely response to various emergency situations. Audio event detection is the first step in recognizing audio signals in a continuous audio input stream. This article presents a number of problems that are associated with the development of sound event detection systems, such as the deviation for each environment and each sound category, overlapping audio events, unreliable training data, etc. Both methods for detecting monophonic impulsive audio event and polyphonic sound event detection methods which are used in the state-of-the-art sound event detection systems are presented. Such systems are presented in Detection and Classification of Acoustic Scenes and Events (DCASE) challenges and workshops, which take place every year. Beside a majority of works focusing on the improving overall performance in terms of accuracy many other aspects have also been studied. Several systems presented at DCASE 2021 task 4 were considered, and based on their analysis, there was a conclusion about possible future for sound event detection systems. Also the actual directions in the development of modern audio analytics systems are presented, including the study and use of various architectures of neural networks, the use of several data augmentation techniques, such as universal sound separation, etc.

Non‐speech audio event detection (AED) could be used for low‐cost, spatially diffuse surveillance applications, e.g., monitoring of vehicle activity in a national park, or of footsteps in a hallway. Experiments have shown that non‐speech AED benefits from the dynamic inference strategies such as the hidden Markov model (HMM), but that the acoustic features useful for non‐speech events may not be the same as those useful for speech. One possible solution is a tandem HMM: an HMM whose observation vector is constructed from the output of an instantaneous discriminative classifier, e.g., a neural network. The use of tandem HMMs for non‐speech AED is hindered, however, by the relatively small size of most non‐speech‐audio training corpora. This talk will demonstrate that tandem HMMs can be trained to detect non‐speech audio events using a novel form of regularized training: Baum–Welch back‐propagation (as proposed by Bengio et al.), using the conjugate‐gradient adaptive form of the Baum–Welch auxiliary functio...

Optimized machine learning approaches for identifying vertical temperature gradient on ballastless track in natural environments

Performance optimization of wireless networks is typically complicated because of high computational complexity and dynamic channel conditions. Considering a specific case, the recent introduction of intelligent reflecting surface (IRS) can reshape the wireless channels by controlling the scattering elements' phase shifts, namely, passive beamforming. However, due to the large size of scattering elements, the IRS's beamforming optimization becomes intractable. In this article, we focus on machine learning (ML) approaches for complex optimization problems in wireless networks. ML approaches can provide flexibility and robustness against uncertain and dynamic systems. However, practical challenges still remain due to slow convergence in offline training or online learning. This motivated us to design a novel optimization-driven ML framework that exploits the efficiency of model-based optimization and the robustness of model-free ML approaches. Splitting the control variables into two parts allows one part to be updated by the outer loop ML approach while the other part is solved by the inner loop optimization. The case study in IRS-assisted wireless networks confirms that the optimization-driven ML framework can improve learning efficiency and the reward performance significantly compared to conventional model-free ML approaches.

From a personalized media point of view, many users favor a flexible tool to quickly browse the affective content in a video. Such affective content may cause audiences' strong reactions or special emotional experiences, such as anger, sadness, fear, joy and love. This paper attempts to extract affective content for digital videos by analyzing the subtitle files of DVD/DivX videos and utilize audio event to assist affective content detection. Firstly, videos are segmented by dialogue script partition. Compared to traditional video shot, video segmented by scripts is not affected by camera changes and shooting angles and easy to include video segments with compact content. Secondly, emotion-related vocabularies in video script are detected to locate affective video content. Using script to directly access video content avoids complex video analysis. Thirdly, audio event detection is utilized to assist affective content detection. Compared with traditional video semantic analysis, affective content analysis puts much more emphasis on the audience's reactions and emotions. Initial experiments are carried on sitcom videos because its simple video structure provides useful domain knowledge. The experimental results demonstrate that subtitle file analysis and audio event detection provides effective and efficient clues to determine the emotional content of the videos.

Efficient fertilizer application is vital for enhancing maize production and profitability in Sub-Saharan Africa, where soil fertility varies widely across regions. This study aimed to develop a machine learning approach for generating site-specific fertilizer recommendations for maize production in Ghana and to evaluate its performance against conventional and semi-mechanistic approaches. A random forest machine learning model was trained on 482 maize yield experiments, consisting of 3136 yield observations collected from 1991 to 2020, to predict maize yield response to different fertilizer rates. The model incorporated multiple explanatory variables, including soil properties, climate conditions, and management practices, to generate fertilizer response curves from which fertilizer recommendations were derived for 14 sites across three agro-ecological zones in Ghana where field validation experiments were conducted. On these sites, the recommendations were compared with recommendations derived from the Quantitative Evaluation of the Fertility of Tropical Soils (QUEFTS), Conventional Fertilizer Dose Response (CFDR), and Updated Conventional Fertilizer Dose Response (UCFDR) approaches and validated through field experiments. The machine learning approach generally recommended lower rates of phosphorus and potassium than the other approaches, while nitrogen recommendations were comparable. In the Guinea Savanna zone, the recommendations from the machine learning approach outperformed those from the other approaches, producing higher mean yields for three out of the four sites in the zone. In the Forest-Savanna Transition (FST) zone, the machine learning model recommendations led to higher mean yields at four sites, while the approaches based on QUEFTS and UCFDR performed best at two other sites. In the Semi-deciduous Forest zone, the recommendations of the QUEFTS approach resulted in the highest mean yields at three sites, and CFDR at one site. Despite high input prices during the period of experimentation, the machine learning approach-based recommendations demonstrated higher net profit margins in the FST zone, suggesting cost-effectiveness in this zone. These findings indicate that site-specific fertilizer recommendations are more efficient than blanket recommendations and that machine learning approaches offer a promising and innovative approach for generating cost-effective, site-specific fertilizer recommendations in tropical climates. • ML and QUEFTS approaches provided site-specific fertilizer recommendations. • ML and QUEFTS outperformed conventional fertilizer recommendation approaches. • CFDR and UCFDR were constrained by generalized, non-site-specific recommendations. • Machine learning supports sustainable fertilizer use.

Classification and prediction tasks are common in health research. With the increasing availability of vast health data repositories (e.g. electronic medical record databases) and advances in computing power, traditional statistical approaches are being augmented or replaced with machine learning (ML) approaches to classify and predict health outcomes. ML describes the automated process of identifying ("learning") patterns in data to perform tasks. Developing an ML model includes selecting between many ML models (e.g. decision trees, support vector machines, neural networks); model specifications such as hyperparameter tuning; and evaluation of model performance. This process is conducted repeatedly to find the model and corresponding specifications that optimize some measure of model performance. ML models can make more accurate classifications and predictions than their statistical counterparts and confer greater flexibility when modelling unstructured data or interactions between covariates; however, many ML models require larger sample sizes to achieve good classification or predictive performance and have been criticized as "black box" for their poor transparency and interpretability. ML holds potential in family medicine for risk profiling of patients' disease risk and clinical decision support to present additional information at times of uncertainty or high demand. In the future, ML approaches are positioned to become commonplace in family medicine. As such, it is important to understand the objectives that can be addressed using ML approaches and the associated techniques and limitations. This article provides a brief introduction into the use of ML approaches for classification and prediction tasks in family medicine.

A machine learning approach for hierarchical classification of software requirements

Passive non-invasive sensing signals from wearable devices and smartphones are typically collected continuously without user input. This passive and continuous data collection makes these signals suitable for moment-by-moment monitoring of health-related outcomes, disease diagnosis, and prediction modeling. A growing number of studies have utilized machine learning (ML) approaches to predict and analyze health indicators and diseases using passive non-invasive signals collected via wearable devices and smartphones. This systematic review identified peer-reviewed journal articles utilizing ML approaches for digital phenotyping and measuring digital biomarkers to analyze, screen, identify, and/or predict health-related outcomes using passive non-invasive signals collected from wearable devices or smartphones. PubMed, PubMed with Mesh, Web of Science, Scopus, and IEEE Xplore were searched for peer-reviewed journal articles published up to June 2024, identifying 66 papers. We reviewed the study populations used for data collection, data acquisition details, signal types, data preparation steps, ML approaches used, digital phenotypes and digital biomarkers, and health outcomes and diseases predicted using these ML techniques. Our findings highlight the promising potential for objective tracking of health outcomes and diseases using passive non-invasive signals collected from wearable devices and smartphones with ML approaches for characterization and prediction of a range of health outcomes and diseases, such as stress, seizure, fatigue, depression, and Parkinson’s disease. Future studies should focus on improving the quality of collected data, addressing missing data challenges, providing better documentation on study participants, and sharing the source code of the implemented methods and algorithms, along with their datasets and methods, for reproducibility purposes.

Machine learning simulation of one-dimensional deterministic water wave propagation

Regression analysis is one of the most widely utilized methods because of its adaptability and simplicity. Recently, the machine learning (ML) approach, which is one aspect of regression methods, has been gaining attention from researchers, including social science, but there are only a few studies that compared the traditional approaches with the ML approach. This study was conducted to explore the usefulness of the ML approach by comparing the ordinary least square estimate (OLS), the stochastic gradient descent algorithm (SGD), and the support vector regression (SVR) with a model predicting and explaining the tuberculosis screening intention. The optimized models were evaluated by four aspects: computational speed, effect and importance of individual predictor, and model performance. The result demonstrated that each model yielded a similar direction of effect and importance in each predictor, and the SVR with the radial kernel had the finest model performance compared to its computational speed. Finally, this study discussed the usefulness and attentive points of the ML approach when a researcher utilizes it in the field of communication.

To develop and evaluate the concept of a non-invasive machine learning (ML) approach for detecting hypoglycaemia based exclusively on combined driving (CAN) and eye tracking (ET) data. We first developed and tested our ML approach in pronounced hypoglycaemia, and then we applied it to mild hypoglycaemia to evaluate its early warning potential. For this, we conducted two consecutive, interventional studies in individuals with type 1 diabetes. In study 1 (n=18), we collected CAN and ET data in a driving simulator during euglycaemia and pronounced hypoglycaemia (blood glucose [BG] 2.0-2.5 mmol L-1 ). In study 2 (n=9), we collected CAN and ET data in the same simulator but in euglycaemia and mild hypoglycaemia (BG 3.0-3.5 mmol L-1 ). Here, we show that our ML approach detects pronounced and mild hypoglycaemia with high accuracy (area under the receiver operating characteristics curve 0.88 ± 0.10 and 0.83 ± 0.11, respectively). Our findings suggest that an ML approach based on CAN and ET data, exclusively, enables detection of hypoglycaemia while driving. This provides a promising concept for alternative and non-invasive detection of hypoglycaemia.

BackgroundMalaria is still a major global health burden, with more than 3.2 billion people in 91 countries remaining at risk of the disease. Accurately distinguishing malaria from other diseases, especially uncomplicated malaria (UM) from non-malarial infections (nMI), remains a challenge. Furthermore, the success of rapid diagnostic tests (RDTs) is threatened by Pfhrp2/3 deletions and decreased sensitivity at low parasitaemia. Analysis of haematological indices can be used to support the identification of possible malaria cases for further diagnosis, especially in travellers returning from endemic areas. As a new application for precision medicine, we aimed to evaluate machine learning (ML) approaches that can accurately classify nMI, UM, and severe malaria (SM) using haematological parameters.MethodsWe obtained haematological data from 2,207 participants collected in Ghana: nMI (n = 978), SM (n = 526), and UM (n = 703). Six different ML approaches were tested, to select the best approach. An artificial neural network (ANN) with three hidden layers was used for multi-classification of UM, SM, and uMI. Binary classifiers were developed to further identify the parameters that can distinguish UM or SM from nMI. Local interpretable model-agnostic explanations (LIME) were used to explain the binary classifiers.ResultsThe multi-classification model had greater than 85% training and testing accuracy to distinguish clinical malaria from nMI. To distinguish UM from nMI, our approach identified platelet counts, red blood cell (RBC) counts, lymphocyte counts, and percentages as the top classifiers of UM with 0.801 test accuracy (AUC = 0.866 and F1 score = 0.747). To distinguish SM from nMI, the classifier had a test accuracy of 0.96 (AUC = 0.983 and F1 score = 0.944) with mean platelet volume and mean cell volume being the unique classifiers of SM. Random forest was used to confirm the classifications, and it showed that platelet and RBC counts were the major classifiers of UM, regardless of possible confounders such as patient age and sampling location.ConclusionThe study provides proof of concept methods that classify UM and SM from nMI, showing that the ML approach is a feasible tool for clinical decision support. In the future, ML approaches could be incorporated into clinical decision-support algorithms for the diagnosis of acute febrile illness and monitoring response to acute SM treatment particularly in endemic settings.

In this paper, we present a noble method to segment and classify audio stream using a temporally weighted fuzzy c-means algorithm (TWFCM). The proposed algorithm is utilized to determine the boundaries between different kinds of sounds in an audio stream; and then classify the audio segments into five classes of sound such as music, speech, speech with music background, speech with noise background, and silence. This is an enhancement on conventional fuzzy c-means algorithm, applied in audio segmentation and classification domain, by addressing and reflecting the matter of temporal correlations between the audio signals in the current and previous time. A 3-elements feature vector is utilized in segmentation and a 5-elements feature vector is utilized in classification by using TWFCM. The audio-cuts can be detected accurately by this method, and mistakes caused by audio effects can be eliminated in segmentation. Improved classification performance is also achieved. The application of this method is demonstrated in segmenting and classifying real-world audio data such as television news, radio signals, etc. Experimental results indicate that the proposed method outperforms the conventional FCM.KeywordsAudio segmentation and classificationfuzzy c-means algorithmdatabase retrieval

Background: Adolescent depression is highly prevalent in low- and middle-income countries (LMICs). Identifying top key risk factors is necessary to inform effective prevention program design. Machine learning (ML) offers a powerful approach to analyze complex multidomain of data to identify the most relevant predictors and estimate risks for mental health problems. This paper applies ML to study risks for adolescent depression to enhance adolescent depression prevention efforts in LMICs. Methods: Six ML approaches (e.g., Explainable Boosting Machine, random forests, and XGBoost) were applied to study the risks of depression. Data were drawn from a digital health intervention study conducted in Kenya (year 2024–2025, n = 269). Multiple domains of childhood and adolescent adversity and stress experiences were used to predict adolescent depression (using PHQ9-A). Findings: We found that ML was a valuable approach in the early identification of adolescents at risk for depression. Among the six ML approaches applied, the random forest approach outperformed other ML approaches, especially when multiple domains of risks were included. We also found that childhood adversity or home adversity alone were not strong predictors for depression. Adding adolescent stress experiences and community school adversity experiences significantly improves the accuracy and predictability of depression. Using the top-15 and top-20 ranking factors, we achieved 74.8% and 75.1% accuracy in depression prediction, which was similar to the accuracy when all 49 adverse/stress factors were included in the predictive model (78.3%). Conclusions: Innovative ML and modern predictive modeling approaches have the potential to transform modern preventive mental health care by better utilizing multidomain data to identify individuals at risk for developing depression early and identify top risk factors (for targeted individuals and/or populations). Findings from ML can inform tailored intervention design to better mitigate risks in order to prevent depression problem development. They can also inform the better utilization of resources to target high-need cases and key determinants, which is particularly useful for LMICs and low-resource settings. This paper illustrates an example of how to move toward this direction. Future research is needed to validate the approach.