Identifying optimal clustering structures for residential energy consumption patterns using competency questions

In this paper, seasonal autoregressive integrated moving average (SARIMA) and regression with SARIMA errors (regression-SARIMA) models are developed to predict daily peak electricity demand in South Africa using data for the period 1996 to 2009. The performance of the developed models is evaluated by comparing them with Winter’s triple exponential smoothing model. Empirical results from the study show that the SARIMA model produces more accurate short-term forecasts. The regression-SARIMA modelling framework captures important drivers of electricity demand. These results are important to decision makers, load forecasters and systems operators in load flow analysis and scheduling of electricity.

Aggregate demand for electricity in South Africa: An analysis using the bounds testing approach to cointegration

The evolution of price elasticity of electricity demand in South Africa: A Kalman filter application

Prediction of daily peak electricity demand in South Africa using volatility forecasting models

A variety of real-world applications such as complex industry process usually are lack of abundant training samples since the data acquiring process is time and labor consuming. Hence, it is important to utilize the limited training samples to build a sophisticated data-driven model, which may improve industry productivity. Recently, nonlinear learning models such as artificial neural networks and support vector machines have shown to be effective in modeling small-scale data by their strong modeling ability. However, these nonlinear learning models work as a black box and are often not human understandable and are difficult to be interpreted. In addition, in many applications, domain experts could provide us valuable expertise knowledge which may help further improve the modeling process. In this paper, we propose to integrate expertise knowledge to the nonlinear learning model to advance the data-driven modeling process in real-world applications. Experimental results on six benchmark datasets and a real-world industry application validate the effectiveness of the proposed model.

Assessing the societal impact of satellite remote sensing datasets is essential to understanding how these data influence decision-making and to identifying opportunities for further engagement. However, measuring such impacts remains challenging for missions serving diverse stakeholder communities. In this study, we evaluate the broader impact of NASA's Ice, Cloud, and land Elevation Satellite (ICESat) and its successor mission, ICESat-2 by adapting the scientific impact framework (SIF), originally developed to assess public health research, into an Earth science-specific version (e-SIF). This framework captures data dissemination, community awareness, data-driven actions, measurable changes, and future influence, moving beyond traditional academic metrics to assess the missions' reach and effectiveness. Our findings reveal extensive global usage of ICESat and ICESat-2 data, with applications including, but not limited to, shallow water bathymetry, climate mitigation strategies, and forest management. By comparing the prevalence of topical areas in academic literature to real-world applications, we found that althoughthe cryosphere is the most frequently studied domain, differences between research focus and practical use highlight potential areas where further research could better support stakeholders. We found that ICESat and ICESat-2 data are widely employed by national and international governmental and non-governmental organizations but found only limited use by private sector and local governments. We recommend that the ICESat-2 Applications Team expand outreach efforts to these sectors to enhance dissemination of mission data. Furthermore, numerous ICESat-2 applications benefit from long-term data continuity, reinforcing the need for a successor mission. This study demonstrates the feasibility to use e-SIF to evaluate the impact of Earth science missions. • This study involves an innovative application of the CDC Scientific Impact Framework. • ICESat and ICESat-2 provide global altimetry data that empowers diverse applications. • Application topics vary between the academic and broader stakeholder communities. • The study models methods of how to examine real-world outcomes in the Earth sciences. Open scientific data empowers diverse communities to generate knowledge and insights that support decision-making across public policy, economic planning, and environmental management. Through Earth-observing missions, NASA aims to improve scientific understanding of our planet's current state and possible future changes. The NASA missions Ice, Cloud, and land Elevation Satellite (ICESat) and ICESat-2 have contributed to this effort by providing highly accurate elevation data using laser altimetry. In this study, we examined the impact of these missions beyond the academic community to identify key stakeholders and assess how these missions influence decision-making. To do so, we modified the Centers for Disease Control and Prevention's (CDC) Scientific Impact Framework (SIF), originally designed to evaluate public health research, to create an Earth science relevant SIF (e-SIF) in order to discover and analyze the applications of Earth science missions beyond the academic sector. Our findings show that these data products have had a strong policy impact at the intergovernmental level but have been used less frequently by local governments and corporations. Moreover, we observed notable differences between the scientific topics emphasized in academic literature and those found in reports and real-world applications. This study highlights the importance of evaluating how scientific research translates into real-world impacts, including for technological advancements, policy changes, economic effects, and more. As ICESat-2 approaches the end of its mission, ensuring data continuity is critical to supporting current and future applications that rely on these observations.

Current approaches are unlikely to achieve the aggressive global tuberculosis (TB) control targets set for 2035 and beyond. Active case finding (ACF) may be an important tool for augmenting existing strategies, but the cost-effectiveness of ACF remains uncertain. Program evaluators can often measure the cost of ACF per TB case detected, but how this accessible measure translates into traditional metrics of cost-effectiveness, such as the cost per disability-adjusted life year (DALY), remains unclear. We constructed dynamic models of TB in India, China, and South Africa to explore the medium-term impact and cost-effectiveness of generic ACF activities, conceptualized separately as discrete (2-year) campaigns and as continuous activities integrated into ongoing TB control programs. Our primary outcome was the cost per DALY, measured in relationship to the cost per TB case actively detected and started on treatment. Discrete campaigns costing up to $1,200 (95% uncertainty range [UR] 850-2,043) per case actively detected and started on treatment in India, $3,800 (95% UR 2,706-6,392) in China, and $9,400 (95% UR 6,957-13,221) in South Africa were all highly cost-effective (cost per DALY averted less than per capita gross domestic product). Prolonged integration was even more effective and cost-effective. Short-term assessments of ACF dramatically underestimated potential longer term gains; for example, an assessment of an ACF program at 2 years might find a non-significant 11% reduction in prevalence, but a 10-year evaluation of that same intervention would show a 33% reduction. ACF can be a powerful and highly cost-effective tool in the fight against TB. Given that short-term assessments may dramatically underestimate medium-term effectiveness, current willingness to pay may be too low. ACF should receive strong consideration as a basic tool for TB control in most high-burden settings, even when it may cost over $1,000 to detect and initiate treatment for each extra case of active TB.

Assessing systemic risk presents a significant challenge in finance and insurance, where conditional risk measures are essential for capturing contagion effects. This paper introduces two novel systemic risk measures – conditional interval value-at-risk (CoIVaR) and conditional interval expected shortfall (CoIES) – which extend traditional metrics by incorporating interval-based uncertainty. A formal theoretical framework is developed for both measures, offering a detailed characterization of their key properties and risk contributions. We then propose a comprehensive comparison methodology for systemic risk assessment, leveraging stochastic orders, dependence structures, and marginal distributions to establish conditions for ranking risk vectors. Finally, through numerical experiments and real-world stock market applications, we demonstrate the practical utility of CoIVaR and CoIES in quantifying systemic risk under uncertainty. The findings provide valuable insights into systemic risk propagation and establish a robust foundation for risk management in interconnected financial systems.

Evaluating artificial intelligence (AI) models in clinical medicine requires more than conventional metrics such as accuracy, Area Under the Receiver Operating Characteristic (AUROC), or F1-score, which often overlook key considerations such as fairness, reliability, and real-world utility. We introduce CUES as a multiplicative composite score for clinical prediction models; it is defined as CUES=(C⋅U⋅E⋅S)1/4, where C represents calibration, U integrated clinical utility, E equity across patient subpopulations, and S sampling stability. We formally establish boundedness, monotonicity, and differentiability on the domain (0,1]4, derive first-order sensitivity relations, and provide asymptotic approximations for its sampling distribution via the delta method. To facilitate inference, we propose bootstrap procedures for constructing confidence intervals and for comparative model evaluation. Analytic examples illustrate how CUES can diverge from traditional metrics, capturing dimensions of predictive performance that are essential for clinical reliability but often missed by AUROC or F1-score alone. By integrating multiple facets of clinical utility and robustness, CUES provides a comprehensive tool for model evaluation, comparison, and selection in real-world medical applications.

Integrative clustering is used to identify groups of samples by jointly analysing multiple datasets describing the same set of biological samples, such as gene expression, copy number, methylation etc. Most existing algorithms for integrative clustering assume that there is a shared consistent set of clusters across all datasets, and most of the data samples follow this structure. However in practice, the structure across heterogeneous datasets can be more varied, with clusters being joined in some datasets and separated in others. In this paper, we present a probabilistic clustering method to identify groups across datasets that do not share the same cluster structure. The proposed algorithm, Clusternomics, identifies groups of samples that share their global behaviour across heterogeneous datasets. The algorithm models clusters on the level of individual datasets, while also extracting global structure that arises from the local cluster assignments. Clusters on both the local and the global level are modelled using a hierarchical Dirichlet mixture model to identify structure on both levels. We evaluated the model both on simulated and on real-world datasets. The simulated data exemplifies datasets with varying degrees of common structure. In such a setting Clusternomics outperforms existing algorithms for integrative and consensus clustering. In a real-world application, we used the algorithm for cancer subtyping, identifying subtypes of cancer from heterogeneous datasets. We applied the algorithm to TCGA breast cancer dataset, integrating gene expression, miRNA expression, DNA methylation and proteomics. The algorithm extracted clinically meaningful clusters with significantly different survival probabilities. We also evaluated the algorithm on lung and kidney cancer TCGA datasets with high dimensionality, again showing clinically significant results and scalability of the algorithm.

This paper presents a new approach to non-parametric cluster analysis called adaptive weights’ clustering. The method is fully adaptive and does not require to specify the number of clusters or their structure. The clustering results are not sensitive to noise and outliers, and the procedure is able to recover different clusters with sharp edges or manifold structure. The method is also scalable and computationally feasible. Our intensive numerical study shows a state-of-the-art performance of the method in various artificial examples and applications to text data. The idea of the method is to identify the clustering structure by checking at different points and for different scales on departure from local homogeneity. The proposed procedure describes the clustering structure in terms of weights $w_{ij}$ , and each of them measures the degree of local inhomogeneity for two neighbor local clusters using statistical tests of “no gap” between them. The procedure starts from very local scale, and then, the parameter of locality grows by some factor at each step. We also provide a rigorous theoretical study of the procedure and state its optimal sensitivity to deviations from local homogeneity.

DNA microarrays are used to measure the expression levels of thousands of genes simultaneously. In a time series experiment, the gene expressions are measured as a function of time. We present an application for integrated visualization of genome expression and network dynamics in both regulatory networks and metabolic pathways. Integration of these two levels of cellular processes is necessary, since it provides the link between the measurements at the transcriptional level (gene expression levels approximated from microarray data) and the phenotype (the observable characteristics of an organism) at the functional and behavioral level. The integration requires visualization approaches besides traditional clustering and statistical analysis methods. Our application can (i) visualize the data from time series experiments in the context of a regulatory network and KEGG metabolic pathways; (ii) identify and visualize active regulatory subnetworks from the gene expression data; (iii) perform a statistical test to identify and subsequently visualize pathways that are affected by differentially expressed genes. We present a case study, which demonstrates that our approach and application both facilitates and speeds up data analysis tremendously in comparison to a more traditional approach that involves many manual, laborious, and error‐prone steps.

Functional factorial [formula omitted]-means analysis

A cluster validity measure with a hybrid parameter search method for the support vector clustering algorithm

Fuzzy Cognitive Maps (FCMs) are well-suited techniques to model the behavior of complex systems. But, since they depend on the quality of human knowledge they are prone to subjective false estimations. Moreover, even domain experts can fully describe only less complex systems which, in turn, limits the effectiveness of FCMs in real world applications. Thus, a technique is needed that, on the one hand, eases the use and, on the other hand, optimizes the formal structure of an FCM. Against that background, this article elucidates the use of different bio-inspired algorithms to optimize the application of the FCMs: Evolutionary Strategies (ES), Particle Swarm Optimization (PSO), and Big Bang-Big Crunch (BB-BC). The proposed approaches are applied to a novel valuation technique using FCMs. Besides principal feasibility, we demonstrate its potential to facilitate the use of vague and incomplete expert knowledge in the valuation of complex investment opportunities such as, but not limited to, young High-Technology Ventures.