Toward Efficient Deployment of Mixture of Experts Models: Quantization and Compression Analysis

Lithium-ion batteries (LIBs) experience degradation influenced by various operational and environmental factors, including temperature, depth of discharge (DoD), state of charge (SoC), and current rates. Among these, temperature is a critical driver, as elevated temperatures accelerate chemical reactions, induce thermal stress, and promote side reactions such as lithium plating. These mechanisms degrade battery components, reducing capacity and performance over time. Understanding and predicting capacity fade under such conditions is essential for improving LIB safety, efficiency, and longevity. Traditional physics-based models have been widely used for degradation prediction but face challenges in operational settings due to limited and non-standardized input data. Recent advancements in machine learning (ML), particularly low-data ML models, offer promising alternatives for accelerated degradation prediction under such constraints. This study presents a novel framework integrating two advanced low-data ML models—a cycle-based autoencoder (CD-Net) and a time-based mixture of experts (MoE) model—to address the challenges of predicting LIB degradation while accounting for thermal effects.The CD-Net autoencoder tracks capacity degradation across operational cycles by incorporating cell-specific chemistry information. It captures cycle-specific degradation patterns and correlates them with temperature variations, enabling accurate real-time state-of-health (SoH) predictions with minimal latency. This model is particularly suited for integrating battery management systems (BMS), offering insights into the interplay between thermal conditions and battery performance. Complementing this, the MoE model provides long-term capacity degradation forecasts using limited early-cycle data. Its ensemble architecture adapts to diverse cathode chemistries and thermal profiles, effectively generalizing even under extreme temperature conditions or data scarcity. Additionally, the MoE model generates high-fidelity synthetic datasets validated through statistical techniques such as Wasserstein Distance and Kolmogorov-Smirnov tests. These synthetic datasets enhance predictive accuracy and support the exploration of emerging chemistries and thermal scenarios.By combining these models, the framework delivers a comprehensive approach to LIB degradation analysis. The CD-Net excels in real-time monitoring, while the MoE model offers robust long-term forecasting and synthetic data generation capabilities. Together, they address challenges posed by extreme conditions such as rapid temperature fluctuations, overcharge/discharge scenarios, and high C-rate operations. This integrated methodology facilitates accelerated testing of new chemistries and operational profiles while reducing reliance on extensive experimental data.The framework provides actionable insights into thermal management system design by simulating temperature-driven degradation mechanisms. It supports safer and more efficient LIB systems in demanding applications like fast charging, low-temperature environments, and high-performance scenarios. This approach advances predictive modeling for LIBs, contributing to improved safety, energy storage efficiency, and the development of next-generation battery technologies.

A Universal Approximation Theorem for Gaussian-Gated Mixture of Experts Models

Traditionally, claim frequency and severity are assumed to be independent in insurance claims modeling. However, this assumption often proves to be inaccurate in the ratemaking process. In this study, we address this dependency by employing a logit-weighted reduced mixture of experts (MoE) model. Furthermore, previous studies have often overlooked the potential clustering effects of spatial covariates within the data. To address this, we introduce the spatially clustered mixture of experts (SCMoE) model, which incorporates a regularization term into the original framework. For inference, we propose a novel minorization–maximization (MM) algorithm, which integrates the alternating direction method of multipliers (ADMM) in the second M-step. The robustness of the SCMoE model, relative to the conventional MoE model, is demonstrated through two simulation studies. Finally, we apply our model to data from Chinese health insurance. The empirical analysis demonstrates the superiority of the MoE model over several alternatives. Furthermore, the model considering spatial clustering outperforms the one without it in terms of predictive performance.

EEG signal classification using wavelet feature extraction and a mixture of expert model

Robust mixture of experts modeling using the [formula omitted] distribution

Transcription Factor Binding Site (TFBS) prediction is crucial for understanding gene regulation and various biological processes. This study introduces a novel Mixture of Experts (MoE) approach for TFBS prediction, integrating multiple pre-trained Convolutional Neural Network (CNN) models, each specializing in different TFBS patterns. We evaluate the performance of our MoE model against individual expert models on both in-distribution and out-of-distribution (OOD) datasets, using six randomly selected transcription factors (TFs) for OOD testing. Our results demonstrate that the MoE model achieves competitive or superior performance across diverse TF binding sites, particularly excelling in OOD scenarios. The Analysis of Variance (ANOVA) statistical test confirms the significance of these performance differences. Additionally, we introduce ShiftSmooth, a novel attribution mapping technique that provides more robust model interpretability by considering small shifts in input sequences. Through comprehensive explainability analysis, we show that ShiftSmooth offers superior attribution for motif discovery and localization compared to traditional Vanilla Gradient methods. Our work presents an efficient, generalizable, and interpretable solution for TFBS prediction, potentially enabling new discoveries in genome biology and advancing our understanding of transcriptional regulation.

Machine learning based digital twin for dynamical systems with multiple time-scales

Applies modular neural network models to classify sleep-waking states in infants. The performances of three connectionist models are compared: (a) a multilayer perceptron (MLP), (b) a mixture of experts (ME) and (c) a fuzzy ganglionar lattice (FGL). We propose a new methodology for enhancing neural classifiers based on input variable selection and confusion error analysis using expert criteria. The ME model was more robust than the MLP and FGL models in the presence of inconsistent or noisy data. Input variable selection and confusion error analysis using expert criteria led to parsimonious models with less parameters and better classification rates.

Experimental investigation on combustion and emissions of a two-stroke DISI engine fueled with aviation kerosene at various compression ratios

The performance of diesel engine highly depends on atomization, vaporization and mixing of fuel with air. These factors are strongly influenced by various parameters e.g. injection pressure, injection timing, compression ratio, equivalence ratio, cylinder geometry, in cylinder air motion etc. In this study, a diesel engine has been investigated by employing a commercial CFD software (ANSYS Forte, version 18.1) especially developed for internal combustion engines (ICE) modeling; focusing primarily on the effects of equivalence ratio and compression ratio on combustion and emission characteristics. RNG k-ε model was employed as the turbulence model for analyzing the physical phenomena involved in the change of kinetic energy. In order to reduce the computational cost and time, a sector mesh of 45o angle with periodic boundary conditions applied at the periodic faces of the sector, is considered instead of using the whole engine geometry. Simulations are performed for a range of equivalence ratio varying from 0.6 to 1.2 and for three compression ratios namely, 15:1, 18:1 and 21:1. Results show that, improvement in combustion characteristics with higher compression ratio could be achieved for both lean and rich mixtures. Peak in-cylinder pressure and peak heat release nearer to TDC are achieved for compression ratio of 18:1 that could results in more engine torque. For compression ratio beyond 16:1, effects of fuel concentration on ignition delay is more pronounced. At lower compression ratio, in-cylinder temperature is not sufficiently high for atomization, vaporization, mixing of fuel with air, and preflame reactions to occur immediately after the fuel injection. NOx emission in diesel engine increases due to higher pressure and temperature inside the cylinder associated with relatively higher compression ratio. Rich mixture leads to more CO and unburnt hydrocarbon emission compared to lean mixture as result of incomplete combustion. Engine operation with too high compression ratio is detrimental as emission is a major concern.

Sparse Mixture-of-Experts (MoE) has been a successful approach for scaling multilingual translation models to billions of parameters without a proportional increase in training computation. However, MoE models are prohibitively large and practitioners often resort to methods such as distillation for serving. In this work, we investigate routing strategies at different granularity (token, sentence, task) in MoE models to bypass distillation. Experiments on WMT and a web-scale dataset suggest that task-level routing (task-MoE) enables us to extract smaller, ready-to-deploy sub-networks from large sparse models. On WMT, our task-MoE with 32 experts (533M parameters) outperforms the best performing token-level MoE model (token-MoE) by +1.0 BLEU on average across 30 language pairs. The peak inference throughput is also improved by a factor of 1.9x when we route by tasks instead of tokens. While distilling a token-MoE to a smaller dense model preserves only 32% of the BLEU gains, our sub-network task-MoE, by design, preserves all the gains with the same inference cost as the distilled student model. Finally, when scaling up to 200 language pairs, our 128-expert task-MoE (13B parameters) performs competitively with a token-level counterpart, while improving the peak inference throughput by a factor of 2.6x.

The Mixture-of-Experts (MoE) architecture has emerged as a promising paradigm for scaling large language models (LLMs) by activating only a sparse subset of experts per input. However, its massive parameter size remains a major obstacle to efficient deployment. Existing pruning methods often ignore two key aspects: the intricate structural dependencies among experts and the heterogeneous importance of different layers. To tackle these issues, we propose C-GNN-PRUNE, a unified and structure-aware compression framework tailored for MoE models. Our method introduces an EntropyGuided Allocation Module that dynamically assigns pruning budgets by leveraging expert activation entropy, enabling adaptive handling of inter-layer heterogeneity. To preserve structural collaboration patterns, we construct an expert interaction graph that fuses functional similarity and routing behavior, and employ a GNN-Based Embedding Module to learn structure-aware expert representations. These embeddings, along with co-activation patterns, are fed into a Community Detection Module to identify expert clusters for structured pruning. Finally, an Activation-Aware Selection Module retains the most critical experts in each community, balancing sparsity and expressiveness. Experiments on multiple open-source MoE models demonstrate that C-GNN-PRUNE consistently outperforms prior methods under various pruning ratios, achieving better trade-offs between compression and accuracy. This framework provides a modular and effective solution for structure-preserving compression of large-scale MoE models.

Low compression ratio (LCR) approach in diesel engines can reduce the oxides of nitrogen (NOx) and soot emissions simultaneously owing to lower temperatures and longer fuel-air premixing time. The present work investigates the effects of lowering the geometric compression ratio (CR) from 18:1 to 14:1 in a naturally aspirated (NA) single cylinder common rail direct injection (CRDI) diesel engine. Based on the investigations done across the entire speed and load range, significant benefits were observed in the NOx and soot emissions. However, lowering the compression ratio had adverse effects on brake specific fuel consumption (BSFC), unburned hydrocarbon (HC) and carbon monoxide (CO) emissions, especially at low-load and high-speed operating points. To overcome these limitations, novel strategies including split-cooling system (SCS) and secondary exhaust valve opening (SEVO) are proposed in the present work. While the fuel injection parameters optimization specific to LCR could help to improve the BSFC, HC and CO emissions penalty to a reasonable extent, the SCS concept can provide further benefits by reducing the heat loss to coolant and improving the engine component temperatures. Increasing the residual gas fraction using the optimized SEVO concept could further improve the charge temperature leading to a further reduction in the BSFC, HC and CO emissions. The net benefits of the optimized LCR approach are quantified for the modified Indian drive cycle (MIDC) using a one-dimensional simulation tool. The results obtained show a signification reduction of 22% and 74% in NOx and soot emissions respectively as compared to the base 18 CR engine results. Moreover, the penalty in HC and CO emissions could be contained to a large extent resulting in only a slight penalty of 23% and 20% respectively. Furthermore, the higher BSFC with the LCR approach could be successfully addressed and the final values were found to be better than the stock compression ratio by 1.5%. Overall, the strategies proposed in the present work are found to be beneficial to develop modern diesel engines in compliance with the future emission regulations which demand extreme control on NOx and soot emissions.

Impact of variable compression ratios on engine performance and unregulated HC emitted from a research single cylinder engine fueled with commercial gasoline

The effects of different parameters on the residual gas mass spectrograms of turbo-molecular pumps