A comparison of uncertainty and sensitivity analysis results obtained with random and Latin hypercube sampling

Artificial intelligence (AI)-powered chatbots, such as Chat Generative Pretrained Transformer (ChatGPT), have shown promising results in healthcare settings. These tools can help patients obtain real-time responses to queries, ensuring immediate access to relevant information. The study aimed to explore the potential use of ChatGPT-generated medical Arabic responses for patients with metabolic dysfunction-associated steatotic liver disease (MASLD). An English patient questionnaire on MASLD was translated to Arabic. The Arabic questions were then entered into ChatGPT 3.5 on November 12, 2023. The responses were evaluated for accuracy, completeness, and comprehensibility by 10 Saudi MASLD experts who were native Arabic speakers. Likert scales were used to evaluate: 1) Accuracy, 2) Completeness, and 3) Comprehensibility. The questions were grouped into 3 domains: (1) Specialist referral, (2) Lifestyle, and (3) Physical activity. Accuracy mean score was 4.9 ± 0.94 on a 6-point Likert scale corresponding to "Nearly all correct." Kendall's coefficient of concordance (KCC) ranged from 0.025 to 0.649, with a mean of 0.28, indicating moderate agreement between all 10 experts. Mean completeness score was 2.4 ± 0.53 on a 3-point Likert scale corresponding to "Comprehensive" (KCC: 0.03-0.553; mean: 0.22). Comprehensibility mean score was 2.74 ± 0.52 on a 3-point Likert scale, which indicates the responses were "Easy to understand" (KCC: 0.00-0.447; mean: 0.25). MASLD experts found that ChatGPT responses were accurate, complete, and comprehensible. The results support the increasing trend of leveraging the power of AI chatbots to revolutionize the dissemination of information for patients with MASLD. However, many AI-powered chatbots require further enhancement of scientific content to avoid the risks of circulating medical misinformation.

This paper reviews five related types of analysis, namely (i) sensitivity or what-if analysis, (ii) uncertainty or risk analysis, (iii) screening, (iv) validation, and (v) optimization. The main questions are: when should which type of analysis be applied; which statistical techniques may then be used? This paper claims that the proper sequence to follow in the evaluation of simulation models is as follows. 1) Validation, in which the availability of data on the real system determines which type of statistical technique to use for validation. 2) Screening: in the simulation‘s pilot phase the really important inputs can be identified through a novel technique, called sequential bifurcation, which uses aggregation and sequential experimentation. 3) Sensitivity analysis: the really important inputs should be subjected to a more detailed analysis, which includes interactions between these inputs; relevant statistical techniques are design of experiments (DOE) and regression analysis. 4) Uncertainty analysis: the important environmental inputs may have values that are not precisely known, so the uncertainties of the model outputs that result from the uncertainties in these model inputs should be quantified; relevant techniques are the Monte Carlo method and Latin hypercube sampling. 5) Optimization: the policy variables should be controlled; a relevant technique is Response Surface Methodology (RSM), which combines DOE, regression analysis, and steepest-ascent hill-climbing. The recommended sequence implies that sensitivity analysis procede uncertainty analysis. Several case studies for each phase are briefly discussed in this paper.

In earth and environmental sciences applications, uncertainty analysis regarding the outputs of models whose parameters are spatially varying (or spatially distributed) is often performed in a Monte Carlo framework. In this context, alternative realizations of the spatial distribution of model inputs, typically conditioned to reproduce attribute values at locations where measurements are obtained, are generated via geostatistical simulation using simple random (SR) sampling. The environmental model under consideration is then evaluated using each of these realizations as a plausible input, in order to construct a distribution of plausible model outputs for uncertainty analysis purposes. In hydrogeological investigations, for example, conditional simulations of saturated hydraulic conductivity are used as input to physically-based simulators of flow and transport to evaluate the associated uncertainty in the spatial distribution of solute concentration. Realistic uncertainty analysis via SR sampling, however, requires a large number of simulated attribute realizations for the model inputs in order to yield a representative distribution of model outputs; this often hinders the application of uncertainty analysis due to the computational expense of evaluating complex environmental models. Stratified sampling methods, including variants of Latin hypercube sampling, constitute more efficient sampling aternatives, often resulting in a more representative distribution of model outputs (e.g., solute concentration) with fewer model input realizations (e.g., hydraulic conductivity), thus reducing the computational cost of uncertainty analysis. The application of stratified and Latin hypercube sampling in a geostatistical simulation context, however, is not widespread, and, apart from a few exceptions, has been limited to the unconditional simulation case. This paper proposes methodological modifications for adopting existing methods for stratified sampling (including Latin hypercube sampling), employed to date in an unconditional geostatistical simulation context, for the purpose of efficient conditional simulation of Gaussian random fields. The proposed conditional simulation methods are compared to traditional geostatistical simulation, based on SR sampling, in the context of a hydrogeological flow and transport model via a synthetic case study. The results indicate that stratified sampling methods (including Latin hypercube sampling) are more efficient than SR, overall reproducing to a similar extent statistics of the conductivity (and subsequently concentration) fields, yet with smaller sampling variability. These findings suggest that the proposed efficient conditional sampling methods could contribute to the wider application of uncertainty analysis in spatially distributed environmental models using geostatistical simulation.

Uncertainty and sensitivity analyses as a validation tool for BWR bundle thermal–hydraulic predictions

The practical value of oculomotor plant mathematical models (OPMMs) has been demonstrated across various domains, including biometrics and eye movement prediction. To further enhance their utilization, new optimization approaches are commonly developed and introduced within the research community. In this study, we demonstrate a new integration of a previously developed per-subject optimization procedure for an Enderle OPMM and introduce methods to evaluate the reliability of OPMM parameters using the intraclass correlation coefficient (ICC) and Kendall's Coefficient of Concordance (KCC). We evaluated two per-subject OPMM models, Bahill and Enderle, using the 'GazeBase' eye movement dataset. The models differed in accuracy, expressed as the per-sample error in degrees of visual angle, based on the differences between the actual and predicted saccade trajectories. We found that some of the parameter estimates for both models were quite unreliable. We discuss the importance of addressing low-reliability issues and suggest methods to modify the models to enhance reliability and performance.

Appropriate validation of the segmentation algorithms is important for clinical acceptance of those methods. Receiver operating characteristic (ROC) analysis provides the most comprehensive description of the accuracy performance of image segmentation. Total area under an ROC curve (AUC) is widely used as an index of ROC analysis of performance test. However, a large part of the ROC curve is in the clinically irrelevant range. The total area can be misleading in some clinical situation. In this paper, we proposed a partial area index of ROC curves, which measures the segmentation performance in a clinically relevant range decided by learning from subjective ratings. The boundary of the range is defined by a linear cost function of false positive fraction (FPF) and true positive fraction (TPF). The cost factors of FPF and TPF are learned by maximizing the Kendall's coefficient of concordance (KCC) between the partial areas and the subjective ratings. Experiment results show that our method gives a large cost factor on FPF and a small cost factor on TPF on a tumor data set. This is consistent with the fact that a large FPF is generally more difficult to be accepted in tumor segmentation. Our method is able to determine the optimal range for partial area index of ROC analysis, and this partial area index is more appropriate than AUC for evaluating segmentation performance.

A method for evaluating sensitivity and uncertainty analyses is presented here for use with chemical environmental exposure models. Sensitivity analysis allows one to assess the relationship between a set of potentially important model input variables and the model output. Uncertainty analysis allows one to assess the set of output values obtained and estimate various statistical properties such as the mean, median, standard deviation, and range. Absence of the ability to quantify parameter value uncertainty and model output uncertainty limits the validity of predictive model use. Use of the methods presented in this paper can increase the confidence in predictive environmental chemical fate model use. The present investigation employed the Latin hypercube sampling method developed by Iman and Shortencarier at Sandia National Laboratories, Albuquerque, NM. It requires that a set of input variables be identified, and the probability distribution of each be specified. The range of each input variable is divided into intervals having equal probabilities, and one value is randomly selected from each interval. Sets of model input variables are then generated which cover the range of each variable. Some model inputs may then be calculated as functions of these key input variables. The computer model is run for each set of input variables and the model output is obtained. Regression techniques can then be employed to examine the differential sensitivity of the model to the various input variables and functions thereof, for example, cross products of input variables. Since the model input values are chosen based on probabilities, the set of model outputs can also be interpreted on a probabilistic basis. The techniques discussed in this paper respond to the growing need by chemical exposure model users for better defined sensitivity analysis procedures which can aid in explaining and interpreting results. An example problem data set is presented which highlights the methodological techniques and their utility.

Use of replicated Latin hypercube sampling to estimate sampling variance in uncertainty and sensitivity analysis results for the geologic disposal of radioactive waste

Various traditions have investigated the relationship between emotion and language. For the basic emotions view, emotional prototypes are lexically sedimented in language, evidenced in cultural convergence in emotional recognition and expression tasks. For constructionist theories, conceptual knowledge supported by language is at the core of emotions. Understanding emotion words is embedded in various interrelated constructs such as emotional intelligence, emotion knowledge or emotion differentiation, and is related to, but different from, general vocabulary. A clear advantage of Emotion Vocabulary over most emotion-related constructs is that it can be measured objectively. In two successive corpus-based studies, we tested the predictions of concordance and absolute agreement on the frequency of use of a total of 100 Spanish emotion labels in the eight main Spanish-speaking areas: Spain, Mexico-Central America, River Plate, Continental Caribbean, Andean, Antilles, Chilean, and the United States. In both studies, the intraclass correlation coefficient was statistically different from the null and very large, over .95, as was the Kendall's concordance coefficient, indicating broad consensus among the Spanish linguistic areas. From an applied perspective, our results provide supporting evidence for the similarity in frequency, and therefore cross-cultural generalizability regarding familiarity of the 100 emotion labels as item stems or as experimental stimuli without going through a process of additional adaptation. On a broader scope, these results add evidence on the role of language for emotion theories. In this regard, countries and regions compared here share the same Spanish language, but differ in several aspects in history, culture, and socio-economic structure.

Fire safety design based on calculations: Uncertainty analysis and safety verification

Identification of the building parameters that influence heating and cooling energy loads for apartment buildings in hot-humid climates

Taking into account the farmer's perception of impediments, this study examined the impediments encountered in cyclone ravaged regions as well as their plausible suggestions. Odisha is one of the top 10 ranked climatic vulnerable state of India. 480 km of coastal boundary is exposed to Bay of Bengal and 06 coastal districts are highly vulnerable towards climatic events. As Odisha has an agrarian economy, farmers are severely affected by cyclones. So, the state was purposively selected for the study. Total of 160 respondent farmers were chosen randomly from 8 villages of 2 districts. 20 specific impediments (constraints) were further categorized into 4 broad dimensions of impediments viz., social, economic, environmental, and miscellaneous impediments. Impediments were ranked using Garrett's ranking method, while Kendall's coefficient of concordance approach was executed to rank 4 broad dimensions of impediments. Among 20 impediments, "rain and storms lead to flooding after cyclone" with Garrett's mean value 64.53 was perceived as the most important one. Kendall's coefficient of concordance was significant at 1% significance level (p=.000, <0.01) with the commendable agreement value of 0.588, as it ranges from 0 to 1. Where, Zero denotes nonexistent of agreement among judges, and one denotes perfect agreement with the ?2 value of 282.192 with 3 degrees of freedom. Kendall's coefficient of concordance values for 4 broad impediments were environmental impediment (3.53) was ranked first as constraint followed by economic impediment (2.85), social impediment (2.40) and miscellaneous impediment (1.22). The following suggestions were extracted from the grassroots level investigations viz., attending timely training and awareness programs regarding cyclones, constructions of pucca houses to protect from heavy wind and rain, cultivation of early verities of paddy crops and vegetables, constructing permanent drainage systems, reforestation and afforestation of salt-tolerant mangrove trees, chlorination of drinking water after the cyclone and use of battery operated gadgets to save life post cyclones.

<abstract> <bold><sc>Abstract. </sc></bold>Performing a comprehensive sensitivity/uncertainty analysis is a valuable step in understanding and using a predictive hydrologic/water quality (H/WQ) model. This article applies one-factor-at-a-time (OAT) sensitivity analysis (SA) and first-order error analysis (FOEA)/Monte Carlo simulation with Latin hypercube sampling (LHS) uncertainty analysis techniques for evaluation of a complex, process-based water erosion prediction tool, the USDA Water Erosion Prediction Project (WEPP) model (version 2010.1). Assessment of the WEPP hillslope profile model on a Midwestern U.S. Miami silt loam soil for three cropping/management scenarios and three erosion process cases (as defined by topography) is described. WEPP model runoff, soil loss, and corn (Zea mays L.) yield output responses in the form of expected values and error variances were determined to illustrate model prediction uncertainty. The OAT SA showed that WEPP runoff and soil loss output responses were most sensitive to changes in the baseline effective hydraulic conductivity (K_b) and sand content. WEPP model corn yield output response was most sensitive to crop input parameters affecting the simulation of biomass development. The FOEA showed that the largest contributions to runoff, soil loss, and corn yield total error variance came from K_b and sand/clay content, K_b and baseline soil erodibility factors, and the biomass energy ratio of a crop and harvest index, respectively. The FOEA total variances presented in this study for runoff and soil loss were considerably larger than the corresponding Monte Carlo LHS simulation total variances. The Monte Carlo LHS total variance results were reasonable, making Monte Carlo LHS appear to be a better alternative for quantifying WEPP output response error variance. The Monte Carlo LHS soil loss output responses were also compared to Universal Soil Loss Equation (USLE) soil loss predictions. The USLE soil loss estimates were within the Monte Carlo LHS 90% prediction intervals for six of the nine cropping/management and erosion process cases. Results of this study illustrate the usefulness of combining SA and Monte Carlo LHS for providing detailed uncertainty analysis information for complex, physically based models such as WEPP.

This paper describes the uncertainty analysis carried out for 10% Hot leg break LOCA of Large Scale Test Facility as a part of IAEA Coordinated Research Project on “Evaluation of Uncertainty in Best Estimate Accident Analysis”. The best estimate code used for this analysis is RELAP5/MOD3.2. Initially the nodalisation of the test facility for carrying out the analysis is qualified for both steady state and transient level by systematically applying the procedures lead by Uncertainty Methodology based on Accuracy Extrapolation developed at University of Pisa. Subsequently the uncertainty analysis is carried out using sampling based Monte Carlo approach, which involves the generation and extrapolation of a mapping from uncertain inputs to the uncertain analysis results. The major steps followed in this methodology mainly includes screening sensitivity analysis for input parameters, design matrix generation using Latin Hypercube Sampling, representation of uncertainty analysis results based on best estimate thermal hydraulic code runs and importance /sensitivity analysis using regression analysis. The steps followed have been described in details in this paper.

An optimization based sampling approach for multiple metrics uncertainty analysis using generalized likelihood uncertainty estimation