Performance Metrics for Multiobjective Optimization Under Noise

This research compares alternative performance metrics to those more commonly used in target detection system performance evaluation. The alternative performance metrics examined here include the Fisher ratio, a modification of the Dice similarity coefficient, and the Youden index. These metrics are compared to metrics that have been previously introduced for such target detection system performance evaluation: the receiver operating characteristic (ROC) curve (and the related summary area under the ROC curve (AUC) value), and the confidence error generation (CEG) curve (and the related summary root square deviation (RSD) value). The ROC curve is a discrimination metric that measures the ability of a target detection system to distinguish between target and non-target. The CEG curve quantifies detection system knowledge of its own performance. An approach is presented that combines such metrics; this combination may be dynamically adjusted and updated based on current and future evaluation requirements for particular target detection systems.

This paper examines the role of the control objective and the control time in determining fuel-optimal actuator placement for structural control. A general theory is developed that can be easily extended to include alternative performance metrics such as energy and timeoptimal control. The performance metric defines a convex admissible control set which leads to a max-min optimization problem expressing optimal location as a function of initial conditions and control time. A solution procedure based on a nested genetic algorithm is presented and applied to an example problem. Results indicate that the optimal placement varies widely as a function of both control time and disturbance location. An approximate fitness function is presented to alleviate the computational burden associated with finding exact solutions. This function is shown to accurately predict the optimal actuator locations for a 6th-order system, and is further demonstrated on a 12th-order system.

Thyroid nodules are common clinical entities, with a significant proportion being malignant. Early, accurate, and non-invasive tools to differentiate benign and malignant nodules can optimize patient management and reduce unnecessary surgery. This study aimed to evaluate the efficacy and accuracy of near-infrared spectroscopy (NIRS) in distinguishing benign from malignant thyroid nodules. A diffuse reflectance spectrum for a total of 20 thyroid nodule samples (10 samples as colloid goiter and 10 samples as thyroid cancer), were acquired in the wavelength range from 1000 to 2500 nm. Spectral data from NIRS were analyzed by means of principal component analysis (PCA), quadratic discriminant analysis (QDA), and linear discriminant analysis (LDA) to classify and differentiate thyroid nodule samples. The present study found that NIRS effectively distinguished colloid goiter and thyroid cancer using the first two principal components (PCs), explaining 90% and 10% of the variance, respectively. QDA discrimination plot displayed a clear separation between colloid goiter and thyroid cancer with minimal overlap, aligning with reported 95% accuracy. Additionally, applying LDA to seven PCs from PCA achieved a 100% accuracy rate in classifying colloid goiter and thyroid cancer from near-infrared spectral data. In conclusion, NIRS offers a promising, non-invasive complementing diagnostic tool for differentiating benign from malignant thyroid nodules with high accuracy. Future work should integrate these results into predictive model development, emphasizing external validation, alternative performance metrics, and protecting against potential overfitting translation of a machine learning model to a clinical setting.

This study attempts to examine the influence of key bank specific factors namely capital adequacy ratio (CAR), non-performing loans (NPL), management efficiency, bank size, and growth rate—on the financial performance of commercial banks in Nepal. Financial performance is measured using two widely recognized indicators: return on assets (ROA) and return on equity (ROE). Utilizing a balanced panel dataset of 12 commercial banks over the period 2016/17 to 2023/24, the analysis encompasses 96 observations. This study employs descriptive statistics, correlation analysis, and multiple regression models to assess the relationships between the selected variables and financial performance. The results indicate that non-performing loans and bank size exert a statistically significant negative impact on ROA, suggesting that higher loan defaults and larger scale operations may undermine asset profitability. In contrast, management efficiency demonstrates a significant positive effect on ROA, highlighting the importance of effective operational management. Surprisingly, capital adequacy ratio (CAR) is found to have no significant association with ROA, challenging conventional assumptions about its role in profitability. The study concludes that NPL levels and management efficiency are critical determinants of financial performance in Nepal’s commercial banking sector, while increased bank size may contribute to inefficiencies rather than economies of scale. These insights carry important implications for banking regulators, policymakers, and institutional managers seeking to strengthen the financial health and sustainability of banks in Nepal. For future research, the study recommends incorporating primary data, alternative performance metrics, and macroeconomic variables to provide a more comprehensive understanding of bank performance dynamics.

Traditional medical publication metrics, such as impact factor, have various well described limitations [1, 2]. Modern, alternative metrics, such as the Altmetric attention score (https://www.altmetric.com), provide scores for each publication using an automated algorithm representing a weighted count of the amount of attention for a research output from various sources. Sources include but are not limited to: news articles; blogs; policy documents; patents; engagements on social media platforms; and number of citations. We aimed to use the Altmetric attention score to derive a new performance metric comparable with impact factor, and we applied this metric to journals in peri-operative medicine, critical care and pain. This new metric, the Altmetric impact factor, encompasses the same denominator and timescale as impact factor, but uses the Altmetric attention score instead of number of citations as the numerator. Eighteen journals were selected based on their recent impact factor with six chosen from fields in peri-operative, critical care and pain medicine. Altmetric attention scores of all articles published in 17 journals during 2018 and 2019 were searched manually using the Altmetric website (www.altmetric.com) between April and June 2020. For the British Journal of Anaesthesia, Altmetric attention scores were retrieved from Dimensions (www.dimensions.ai) during January 2021. For the denominator, we included original articles, reviews of all types and guidelines. We excluded non-citable items such as editorials and correspondence. Normality of distribution of the cumulative Altmetric attention scores per article of all journals for 2018 and 2019 was tested using the Kolmogorov–Smirnov method. To correlate the Altmetric impact factor with the impact factor of the 18 journals, we used Spearman's rho test. The Altmetric impact factor of all journals was higher than the corresponding impact factor, except for the European Journal of Anaesthesiology. The mean (SD) Altmetric impact factor vs. impact factor of peri-operative, critical care and pain medicine journals was 27.813 (28.081) vs. 5.384 (1.387); 32.071 (19.945) vs. 9.197 (6.739); and 11.948 (5.955) vs. 3.94 (1.897), respectively. The overall mean (SD) Altmetric impact factor of the 18 journals was 23.944 (20.946), thus close to four-fold higher compared with the mean (SD) impact factor of 6.174 (4.493). Ranking the 18 journals by the Altmetric impact factor does not follow ranking by impact factor (Fig. 1). We found a positive correlation between Altmetric impact factor and impact factor (rs = 0.730, p = 0.001). Several studies report correlations between the Altmetric attention score or their sources (blogs, tweets or news etc.) with citations [3-5]. Comparisons of Altmetric attention score with impact factor are, however, very limited and, as yet, inconclusive [6, 7]. For journal level performance measurement based on social and mainstream news media, our novel performance metric is arguably more relevant than impact factor. Different rankings have previously been observed in peri-operative medicine journals by total and per article Altmetric attention score and by impact factor [3]. Altmetric attention score may be comparable to citations, but not to impact factor. Impact factor includes a denominator comprising the number of original articles, reviews and guidelines [1]. Altmetric impact factor similarly calculated could be compared with the traditional impact factor. Another issue is that the Altmetric attention score represents the attention for an output in real time but does not follow the increase rate of citations in the medium or long term [8]. Altmetric do not offer journal level metrics or ranking. This becomes feasible by implementing the Altmetric impact factor metric outlined here. Further studies are required in more journals and disciplines. In conclusion, the proposed Altmetric impact factor metric follows a different pattern of journal ranking and highlights the attention given to journals by the academic community from means other than citations in a 2-year period following publication alone.

This study evaluates the association between strategic management practices and financial performance in medium and large enterprises across multiple industries in emerging economies, integrating accounting-related practices and marketing strategies into the analysis. Utilizing data and an exploratory approach from 368 firms in six emerging economies, both survey-based measures and secondary financial data were employed. Hierarchical multiple regression models tested three hypotheses: the direct effect of strategic management, the regulating role of macroeconomic conditions, and the amplifying effect of marketing integration. Results indicate that strategic management practices significantly improve the financial results determined by ROA, ROE, and net return margin especially when combined with high-quality accounting practices and marketing integration. Positive macroeconomic environments strengthen these effects, while sectoral differences highlight the varying benefits of accounting versus marketing emphasis. Robustness checks using alternative performance metrics, lagged effects, and industry-fixed models confirm the stability of findings. The study advances the accounting–economics–marketing nexus literature, offering theoretical insights into capability alignment and practical guidance for managers in emerging markets to optimize strategic and functional integration under diverse economic conditions.

To find the strengths and weaknesses of a new multi-objective optimization algorithm, we need to compare its performance with the performances of the state-of-the-art algorithms. Such a comparison involves a selection of a performance metric, a set of benchmark problems, and a statistical test to ensure that the results are statistical significant. There are also studies in which instead of using one performance metric, a comparison is made using a set of performance metrics. All these studies assume that all involved performance metrics are equal. In this paper, we introduce a data-driven preference-based approach that is a combination of multiple criteria decision analysis with deep statistical rankings. The approach ranks the algorithms for each benchmark problem using the preference (the influence) of each performance metric that is estimated using its entropy. Experimental results show that this approach achieved similar rankings to a previously proposed method, which is based on the idea of the majority vote, where all performance metrics are assumed equal. However, as it will be shown, this approach can give different rankings because it is based not only on the idea of counting wins, but also includes information about the influence of each performance metric.

Abstract. The process of model evaluation is not only an integral part of model development and calibration but also of paramount importance when communicating modelling results to the scientific community and stakeholders. The modelling community has a large and well-tested toolbox of metrics to evaluate temporal model performance. In contrast, spatial performance evaluation does not correspond to the grand availability of spatial observations readily available and to the sophisticate model codes simulating the spatial variability of complex hydrological processes. This study makes a contribution towards advancing spatial-pattern-oriented model calibration by rigorously testing a multiple-component performance metric. The promoted SPAtial EFficiency (SPAEF) metric reflects three equally weighted components: correlation, coefficient of variation and histogram overlap. This multiple-component approach is found to be advantageous in order to achieve the complex task of comparing spatial patterns. SPAEF, its three components individually and two alternative spatial performance metrics, i.e. connectivity analysis and fractions skill score, are applied in a spatial-pattern-oriented model calibration of a catchment model in Denmark. Results suggest the importance of multiple-component metrics because stand-alone metrics tend to fail to provide holistic pattern information. The three SPAEF components are found to be independent, which allows them to complement each other in a meaningful way. In order to optimally exploit spatial observations made available by remote sensing platforms, this study suggests applying bias insensitive metrics which further allow for a comparison of variables which are related but may differ in unit. This study applies SPAEF in the hydrological context using the mesoscale Hydrologic Model (mHM; version 5.8), but we see great potential across disciplines related to spatially distributed earth system modelling.

Multiobjective multi-verse optimization algorithm to solve combined economic, heat and power emission dispatch problems

Testing of on-chip RF and microwave circuits has always been a challenge for test engineers. Since the emergence of systems-on-a-chip (SoCs), characterization and test development for SoCs has become a significant part of the RF IC manufacturing cost. In this paper, we propose a RF built-in-test methodology using on-chip sensors. Instead of measuring the device performance metrics (not possible for embedded RF circuits), the sensor output values (low frequency or DC) are used to estimate the performance metrics of the embedded device under test (DUT). The method has high coverage of defective DUTs for low-yielding processes. An algorithm for optimal placement of sensors on internal transceiver nodes is presented and two test cases are discussed. In this paper, we use the sensor outputs to obtain an estimate of the specification values of the system and the individual modules. Using this method, test specification values can be estimated with very high accuracy.

The problem of quickly detecting and locating a statistical change point using multiple sensors is considered. Specifically, a statistical change point occurs at a random time and at a random location in the coverage region (or the region of interest) of a multi-sensor network. Each sensor node in the network has a limited detection range hence the observations of each sensor would be affected by the change point only if the change point occurs in its vicinity. The region of interest is partitioned by the detection ranges of all the sensors in the network. One is required to detect the presence of the change and to locate the partition of the change based on the sequential observations from all sensor nodes. Three performance metrics, namely the average detection delay, the false alarm probability and the false location probability, are of interest. The goal is to find a stopping time τ, at which the decision maker stops taking more observations and claims the change has occurred, and a terminal decision rule δ, by which the decision maker locates the partition that the change occurs at, to minimize a weighted sum of these three performance metrics. We obtain the optimal solution by first converting the proposed problem to a Markovian stopping time problem and then solving the problem using the tools from the optimal stopping theory.

Aperture is an open, adaptable and extensible Web 2.0 visualization framework, designed to produce visualizations for analysts and decision makers in any common web browser. Aperture utilizes a novel layer based approach to visualization assembly, and a data mapping API that simplifies the process of adaptable transformation of data and analytic results into visual forms and properties. This common visual layer and data mapping API, combined with core elements such as contextually derivable color palettes, layout and symbol ontology services is designed to enable highly creative and expressive visual analytics, rapidly and with less effort. This paper introduces the Aperture framework, describing key features of the programming API and reference implementation, presents example use cases, and proposes an approach for measuring technical performance metrics for software development, and operational performance metrics for visualization support of analysis and decision making.

With growing energy supply concerns and rising costs, energy efficiency is a critical component of industrial energy resilience and competitiveness by directly reducing energy operating costs. Energy efficiency projects in manufacturing also yield valuable benefits to other key metrics, such as improved quality, reduced maintenance costs, improved safety, decreased pollution, and enhanced productivity. However, it is difficult to receive approval for energy efficiency projects, so implementation rates are low, even when meeting capital project payback period criteria. The inclusion and quantification of non-energy benefits (NEBs) in the decision-making process for energy efficiency projects can improve the overall financial payback period while demonstrating a positive impact on the firm's key performance metrics and business strategy. Despite their significant financial and strategic value, NEBs are rarely factored into decision-making due to lack of tools to effectively identify and quantify them. Therefore, a comprehensive and integrative approach is needed for the rapidly evolving energy landscape. To address these challenges, through funding from U.S. Department of Energy, our new assessment methodology integrates common continuous improvement six sigma concepts, such as the DMAIC process, and a protocol of guiding questions, into energy efficiency assessments to identify NEBs. We have also developed open-source software, JUSTIFI, to guide users through this process, data collection, and quantification. It is designed to be used concurrently with DOE energy system analysis software suite, MEASUR. Our methodology and tools inform energy assessors, firm engineering, decision makers, and workforce seeking to increase energy resilience and to maximize benefits aligned with performance metrics.

Interactive multiobjective optimization (IMO) aims at finding the most preferred solution of a decision maker with the guidance of his/her preferences which are provided progressively. During the process, the decision maker can adjust his/her preferences and explore only interested regions of the search space. In recent decades, IMO has gradually become a common interest of two distinct communities, namely, the multiple criteria decision making (MCDM) and the evolutionary multiobjective optimization (EMO). The IMO methods developed by the MCDM community usually use the mathematical programming methodology to search for a single preferred Pareto optimal solution, while those which are rooted in EMO often employ evolutionary algorithms to generate a representative set of solutions in the decision maker's preferred region. This paper aims to give a review of IMO research from both MCDM and EMO perspectives. Taking into account four classification criteria including the interaction pattern, preference information, preference model, and search engine (i.e., optimization algorithm), a taxonomy is established to identify important IMO factors and differentiate various IMO methods. According to the taxonomy, state-of-the-art IMO methods are categorized and reviewed and the design ideas behind them are summarized. A collection of important issues, e.g., the burdens, cognitive biases and preference inconsistency of decision makers, and the performance measures and metrics for evaluating IMO methods, are highlighted and discussed. Several promising directions worthy of future research are also presented.

In binary classification problems, many performance metrics evaluate the probability that some error exceeds a threshold. Nevertheless, they focus more on the probability and fail to capture the magnitude of the error, which evaluates how large this error exceeds the threshold. Capturing the magnitude of error is desired in many applications. For example, in detecting disease and predicting credit default, the magnitude of error illustrates the confidence in making the wrong prediction. We propose a novel metric, the Fragility Index (FI), to evaluate the performance of binary classifiers by capturing the magnitude of the error. FI alleviates the risk of misclassification by penalizing the large error greatly, which is seldom considered by standard metrics. Moreover, to strengthen the generalization ability and handle unseen samples, we adopt the framework of distributionally robust optimization and robust satisficing, which allows us to derive and control the maximum degree of fragility of the classifier when the distribution of samples shifts. We show that FI can be easily calculated and optimized for common probabilistic distance measures. Experiments with real datasets demonstrate the new insights brought by FI and the advantages of classifiers selected under FI, which always improve the robustness and reduce the risk of large errors as compared to classifiers selected by alternative metrics.