Classification of sleep-waking states using modular neural networks

A Universal Approximation Theorem for Gaussian-Gated Mixture of Experts Models

Traditionally, claim frequency and severity are assumed to be independent in insurance claims modeling. However, this assumption often proves to be inaccurate in the ratemaking process. In this study, we address this dependency by employing a logit-weighted reduced mixture of experts (MoE) model. Furthermore, previous studies have often overlooked the potential clustering effects of spatial covariates within the data. To address this, we introduce the spatially clustered mixture of experts (SCMoE) model, which incorporates a regularization term into the original framework. For inference, we propose a novel minorization–maximization (MM) algorithm, which integrates the alternating direction method of multipliers (ADMM) in the second M-step. The robustness of the SCMoE model, relative to the conventional MoE model, is demonstrated through two simulation studies. Finally, we apply our model to data from Chinese health insurance. The empirical analysis demonstrates the superiority of the MoE model over several alternatives. Furthermore, the model considering spatial clustering outperforms the one without it in terms of predictive performance.

Lithium-ion batteries (LIBs) experience degradation influenced by various operational and environmental factors, including temperature, depth of discharge (DoD), state of charge (SoC), and current rates. Among these, temperature is a critical driver, as elevated temperatures accelerate chemical reactions, induce thermal stress, and promote side reactions such as lithium plating. These mechanisms degrade battery components, reducing capacity and performance over time. Understanding and predicting capacity fade under such conditions is essential for improving LIB safety, efficiency, and longevity. Traditional physics-based models have been widely used for degradation prediction but face challenges in operational settings due to limited and non-standardized input data. Recent advancements in machine learning (ML), particularly low-data ML models, offer promising alternatives for accelerated degradation prediction under such constraints. This study presents a novel framework integrating two advanced low-data ML models—a cycle-based autoencoder (CD-Net) and a time-based mixture of experts (MoE) model—to address the challenges of predicting LIB degradation while accounting for thermal effects.The CD-Net autoencoder tracks capacity degradation across operational cycles by incorporating cell-specific chemistry information. It captures cycle-specific degradation patterns and correlates them with temperature variations, enabling accurate real-time state-of-health (SoH) predictions with minimal latency. This model is particularly suited for integrating battery management systems (BMS), offering insights into the interplay between thermal conditions and battery performance. Complementing this, the MoE model provides long-term capacity degradation forecasts using limited early-cycle data. Its ensemble architecture adapts to diverse cathode chemistries and thermal profiles, effectively generalizing even under extreme temperature conditions or data scarcity. Additionally, the MoE model generates high-fidelity synthetic datasets validated through statistical techniques such as Wasserstein Distance and Kolmogorov-Smirnov tests. These synthetic datasets enhance predictive accuracy and support the exploration of emerging chemistries and thermal scenarios.By combining these models, the framework delivers a comprehensive approach to LIB degradation analysis. The CD-Net excels in real-time monitoring, while the MoE model offers robust long-term forecasting and synthetic data generation capabilities. Together, they address challenges posed by extreme conditions such as rapid temperature fluctuations, overcharge/discharge scenarios, and high C-rate operations. This integrated methodology facilitates accelerated testing of new chemistries and operational profiles while reducing reliance on extensive experimental data.The framework provides actionable insights into thermal management system design by simulating temperature-driven degradation mechanisms. It supports safer and more efficient LIB systems in demanding applications like fast charging, low-temperature environments, and high-performance scenarios. This approach advances predictive modeling for LIBs, contributing to improved safety, energy storage efficiency, and the development of next-generation battery technologies.

EEG signal classification using wavelet feature extraction and a mixture of expert model

Robust mixture of experts modeling using the [formula omitted] distribution

Transcription Factor Binding Site (TFBS) prediction is crucial for understanding gene regulation and various biological processes. This study introduces a novel Mixture of Experts (MoE) approach for TFBS prediction, integrating multiple pre-trained Convolutional Neural Network (CNN) models, each specializing in different TFBS patterns. We evaluate the performance of our MoE model against individual expert models on both in-distribution and out-of-distribution (OOD) datasets, using six randomly selected transcription factors (TFs) for OOD testing. Our results demonstrate that the MoE model achieves competitive or superior performance across diverse TF binding sites, particularly excelling in OOD scenarios. The Analysis of Variance (ANOVA) statistical test confirms the significance of these performance differences. Additionally, we introduce ShiftSmooth, a novel attribution mapping technique that provides more robust model interpretability by considering small shifts in input sequences. Through comprehensive explainability analysis, we show that ShiftSmooth offers superior attribution for motif discovery and localization compared to traditional Vanilla Gradient methods. Our work presents an efficient, generalizable, and interpretable solution for TFBS prediction, potentially enabling new discoveries in genome biology and advancing our understanding of transcriptional regulation.

Recent mixture of experts (MoE)-based transformer models have achieved remarkable performance across various applications. However, as model size increases, the associated memory demand escalates dramatically, limiting their deployment in resource-constrained edge environments. To address this challenge, this study investigates the trade-off between performance and computational overhead under extreme compression conditions by applying ternary quantization and additional compression schemes to MoE models. Experimental results demonstrate that Huffman coding achieves a higher compression ratio but suffers from substantial decoding latency, while dictionary coding provides a lower compression ratio with significantly faster decoding. These findings highlight that efficient deployment of MoE models requires an integrated compression framework that jointly optimizes compression ratio and overhead. Furthermore, based on these observations, potential research directions are suggested, including hybrid coding formats and look-up table-based dedicated hardware accelerators for efficient MoE model compression.

Machine learning based digital twin for dynamical systems with multiple time-scales

This study compares single and hybrid soft computing models for estimating daily solar radiation flux for two scenarios. Scenario I developed single soft computing models, including multilayer perceptron (MLP), support vector machines (SVM), adaptive neuro-fuzzy inference system (ANFIS), and multivariate adaptive regression splines (MARS), for estimating daily solar radiation flux at two stations from the USA and South Korea. The MLP model was used to evaluate the effect of factors controlling daily solar radiation flux. Using different combinations of controlling factors as input, the MLP and SVM models, based on evaluation measures, were found to be superior to the ANFIS and MARS models at Big Bend station, USA. In addition, the MLP, SVM, and MARS models performed better than did the ANFIS model at Incheon station, South Korea. Scenario II combined the discrete wavelet transform (DWT) and single soft computing models (e.g., MLP and SVM) for improved performance using 4-input combination. The wavelet-based MLP (WMLP) and SVM (WSVM) models were superior to other single soft computing models (MLP, SVM, ANFIS, and MARS) at two stations. Taylor diagrams, violin plots and point density plots were also utilized to examine the similarity between the observed and estimated solar radiation flux values. Results showed that scenarios I and II can be alternatives for estimating daily solar radiation flux based on different meteorological information, such as peninsular and continental conditions.

We investigate techniques to combine multiple representations of a handwritten digit to increase classification accuracy without significantly increasing system complexity or recognition time. We compare multiexpert and multistage combination techniques and discuss in detail in a comparative manner methods for combining multiple learners: voting, mixture of experts, stacking, boosting and cascading. In pen based handwritten character recognition, the input is the dynamic movement of the pentip over the pressure sensitive tablet. There is also the image formed as a result of this movement. On a real world database, we notice that the two multi layer perceptron (MLP) neural network based classifiers using these representations separately make errors on different patterns, implying that a suitable combination of the two would lead to higher accuracy. Thus we implement and compare voting, mixture of experts, stacking and cascading. Combined classifiers have an error percentage less than individual ones. The final combined system of two MLPs has less complexity and memory requirement than a single k nearest neighbor using one of the representations.

This research investigated and optimized the separation of carbon dioxide (CO2) from natural gas in an adsorption column filled with grafted-beam nanofiber adsorbent. The main purpose of using ANN and RSM models in this manuscript is to compare these two methods in predicting the CO2 adsorption capacity. In other words, it was made to find a suitable model that has the highest agreement with the experimental data. Also, the other purpose of using the RSM model is to detect the optimized empirical conditions. Moreover, two common ANN models are applied in this work, including a multilayer perceptron (MLP) and radial basis function (RBF). The novelties of this work are explained as follows: (1) detecting the optimized synthesis factors of NF-PAN/PUGMA sorbent which possess the highest CO2 adsorption capacity with the help of response surface methodology (RSM), (2) studying the simultaneous interaction of synthesis parameters on the CO2 adsorption capacity with the help of both RSM and artificial neural networks (ANNs), (3) testing two types of ANN models including multilayer perceptron (MLP) and radial basis function (RBF) to predict the effect of monomer volume percentage and irradiation dose on the CO2 adsorption capacity. Indeed, finding the best model (ANN or RSM) can help engineers in practical applications predict CO2 adsorption capacity using NF-PAN/PUGMA under different conditions without incurring high-priced chemical materials, electricity, or human resources. The validation results were examined using correlation coefficients (R2) of RSM, RBF, and MLP models. The correlation coefficients for the RSM, RBF, and MLP models were 0.9910, 0.9949, and 0.9968, respectively. Additionally, the average absolute relative deviation (AARD) values for the RBF and MLP models were 0.00046512 and 0.00045511, respectively, indicating that the MLP model is better than the RBF model. To identify the optimal network structure, the trial-and-error method was conducted for MLP and RBF models. The number of neurons was found at 12 and 45 for MLP and RBF, respectively. The optimized effective parameters were obtained using RSM: 25.80% GMA, 66.45% amine, and an irradiation intensity of 28kGy.

Abstract: Mixture of experts (ME) is a modular neural network architecture for supervised learning. This paper illustrates the use of the ME network structure to guide model selection for classification of electrocardiogram (ECG) beats. The expectation maximization algorithm is used for training the ME so that the learning process is decoupled in a manner that fits well with the modular structure. The ECG signals were decomposed into time–frequency representations using discrete wavelet transforms and statistical features were calculated to depict their distribution. The ME network structure was implemented for ECG beats classification using the statistical features as inputs. To improve classification accuracy, the outputs of expert networks were combined by a gating network simultaneously trained in order to stochastically select the expert that is performing the best at solving the problem. Five types of ECG beats (normal beat, congestive heart failure beat, ventricular tachyarrhythmia beat, atrial fibrillation beat, partial epilepsy beat) obtained from the Physiobank database were classified with an accuracy of 96.89% by the ME network structure. The ME network structure achieved accuracy rates which were higher than those of the stand‐alone neural network models.

Graph convolutional network – Long short term memory neural network- multi layer perceptron- Gaussian progress regression model: A new deep learning model for predicting ozone concertation

Robust color image watermarking algorithm based on synchronization correction with multi-layer perceptron and Cauchy distribution model

This paper investigates the use of artificial neural networks (ANN) in risk estimation of asset returns. Istanbul Stock Exchange (ISE) index (XU100) is studied with a mixture of experts ANN architecture using daily data over a 12-year period. Results are compared to feed-forward neural networks, multilayer perceptron (MLP) and radial basis function (RBF) networks and recurrent neural networks (RNN). They are also compared to widely accepted Exponential Generalized Autoregressive Conditional Heteroskedasticity (EGARCH) volatility model. These results suggest that mixture of experts (MoE) have the strength to capture the volatility in index return series and prepares a valuable basis for financial decision making.KeywordsRoot Mean Square ErrorMean Square ErrorRadial Basis FunctionRecurrent Neural NetworkHide UnitThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.