Case-Based Reasoning with Language Models for Classification of Logical Fallacies

SummaryAccurate prediction of the solubility of chemical substances in solvents remains a challenge. The sparsity of high-quality solubility data is recognized as the biggest hurdle in the development of robust data-driven methods for practical use. Nonetheless, the effects of the quality and quantity of data on aqueous solubility predictions have not yet been scrutinized. In this study, the roles of the size and the quality of data sets on the performances of the solubility prediction models are unraveled, and the concepts of actual and observed performances are introduced. In an effort to curtail the gap between actual and observed performances, a quality-oriented data selection method, which evaluates the quality of data and extracts the most accurate part of it through statistical validation, is designed. Applying this method on the largest publicly available solubility database and using a consensus machine learning approach, a top-performing solubility prediction model is achieved.

Argument(ation) Mining (AM) is the dimension of computational argumentation aiming at automatically processing natural language arguments and reason upon them. More precisely, argument mining aims at extracting, classifying and analysing natural language arguments and their relations from text, with the final goal of providing machine-processable structured data for computational models of argument. In this keynote talk, I will first introduce this research area, highlighting the main successful tasks and open issues in identifying argumentative structures from different kinds of texts (e.g., clinical trials, online user generated content, news articles). Then, I will present a key challenge which conjugate argument mining with formal computational models of argumentation, i.e., the assessment of the trustworthiness of natural language arguments, with a focus on fallacious argumentation, with the aim to show how these methods can be used to automatically identify fallacious arguments in political debates. Fallacies play a prominent role in argumentation since antiquity due to their contribution to argumentation in critical thinking education. Their role is even more crucial nowadays as contemporary argumentation technologies face challenging tasks as misleading and manipulative information detection in news articles and political discourse, and counter-narrative generation. I will conclude with some thoughts on the challenge of automatic generation of counter-arguments to fight online disinformation and hate speech.

To assess strategies for enhancing the generalizability of healthcare artificial intelligence models, we analyzed the impact of preprocessing approaches applied to medical free text, compared single- versus multiple-institution data models, and evaluated data divergence metrics. From 1,607,393 procedures across 44 U.S. institutions, deep neural network models were created to classify anesthesiology Current Procedural Terminology codes from medical free text. Three levels of text preprocessing were analyzed from minimal to automated (cSpell) with comprehensive physician review. Kullback–Leibler Divergence and k-medoid clustering were used to predict single- vs multiple-institutional model performances. Single-institution models showed a mean accuracy of 92.5% [2.8% SD] and 0.923 [0.029] F1 on internal data but generalized poorly on external data (− 22.4% [7.0%]; − 0.223 [0.081]). Free text preprocessing minimally altered performance (+ 0.51% [2.23]; + 0.004 [0.020]). An all-institution model performed worse on internal data (-4.88% [2.43%]; − 0.045 [0.020]), but improved generalizability to external data (+ 17.1% [8.7%]; + 0.182 [0.073]). Compared to vocabulary overlap and Jaccard similarity, Kullback–Leibler Divergence correlated with model performance (R2 of 0.41 vs 0.16 vs 0.08, respectively) and was successful clustering institutions and identifying outlier data. Overall, pre-processing medical free text showed limited utility improving generalization of machine learning models, single institution models performed best but generalized poorly, while combined data models improved generalization but never achieved performance of single-institutional models. Kullback–Leibler Divergence provided valuable insight as a reliable heuristic to evaluate generalizability. These results have important implications in developing broad use artificial intelligence healthcare applications, providing valuable insight into their development and evaluations.

Model-informed precision dosing (MIPD) optimizes drug doses based on pharmacokinetic (PK) model predictions, necessitating careful selection of models tailored to patient characteristics. This study evaluates the predictive performance of various vancomycin PK models across diverse age and BMI categories, drawing insights from a large multi-site database. Adults receiving vancomycin intravenous therapy at United States health systems between January 1, 2022, and December 31, 2023, were included. Patient demographics, vancomycin administration records, and therapeutic drug monitoring levels (TDMs) were collected from the InsightRX database. Age and body mass index (BMI)-based subgroups were formed to assess model performance, with predictions made iteratively. The optimal model for each age-BMI subgroup was chosen based on predefined criteria: models were filtered for mean percentage error (MPE) ≤ 20% and normalized root mean squared error (RMSE) < 8 mg/L, and then the most accurate among them was selected. A total of 384,876 treatment courses across 155 US health systems were analyzed, contributing 841,604 TDMs. Eleven models were compared, showing varying accuracy across age-BMI categories (41%-73%), with higher accuracy observed once TDMs were available for Bayesian estimates of individual PK parameters. Models performed more poorly in younger adults compared to older adults, and the optimal model differed depending on age-BMI categories and prediction methods. Notably, in the a priori period, the Colin model performed best in adults aged 18-64 years across most BMI categories; the Goti/Tong model performed best in the older, non-obese adults; and the Hughes model performed best in many of the obese categories. Our study identifies specific vancomycin PK models that demonstrate superior predictions across age-BMI categories in MIPD applications. Our findings underscore the importance of tailored model selection for vancomycin management, especially highlighting the need for improved models in younger adult patients. Further research into the clinical implications of model performance is warranted to enhance patient care outcomes.

Soil water retention curve (SWRC) is key hydraulic property for vadose zone hydrology. Selected 22 SWRC models with different structures were divided into four types: traditional, single‐segment, two‐segment, and three‐segment models. According to the accuracy and model selection criterion, these models were compared and evaluated using water retention data from saturation to oven dryness of 94 soil samples covering almost all soil types. Among the models, the EG model (exponential model combined with the Groenevelt–Grant model) shows the best performance, but the BC (Brooks–Corey) and VG (van Genuchten) models performed the worst. The relationship between model performance and complexity showed that traditional models with residual water content performed worse than segmental models from saturation to oven dryness, and the segmental models performed worse with an increase in the segment number. In general, models with more parameters perform better, but when the parameter number is greater than five, the model performance gradually worsens due to the equifinality of different parameters. The correlation analysis between soil properties on model performance showed that soil texture has a more significant effect on model performance than bulk density and organic matter content, because the models are primarily affected by their mathematical forms rather than the physical significance of the parameters. Each model picks its limitations and may apply to only one or particular group(s) of soil(s). Therefore, the complementarities of different models may be a great choice to improve SWRC fitting and simulation in arid soil.

Atmospheric nitrogen oxide and nitrogen dioxide (NO + NO2, together termed as NOX) estimates from annual photochemical simulations for years 2002–2016 are compared to surface network measurements of NOX and total gas-phase-oxidized reactive nitrogen (NOY) to evaluate the Community Multiscale Air Quality (CMAQ) modeling system performance by U.S. region, season, and time of day. In addition, aircraft measurements from 2011 Deriving Information on Surface Conditions from Column and Vertically Resolved Observations Relevant to Air Quality are used to evaluate how emissions, chemical mechanism, and measurement uncertainty each contribute to the overall model performance. We show distinct seasonal and time-of-day patterns in NOX performance. Summertime NOX is overpredicted with bimodal peaks in bias during early morning and evening hours and persisting overnight. The summertime morning NOX bias dropped from between 28% and 57% for earlier years (2002–2012) to between −2% and 7% for later years (2013–2016). Summer daytime NOX tends to be unbiased or underpredicted. In winter, the evening NOX overpredictions remain, but NOX is unbiased or underpredicted overnight, in the morning, and during the day. NOX overpredictions are most pronounced in the Midwestern and Southern United States with Western regions having more of a tendency toward model underpredictions of NOX. Modeled NOX performance has improved substantially over time, reflecting updates to the emission inputs and the CMAQ air quality model. Model performance improvements are largest for years simulated with CMAQv5.1 or later and for emission inventory years 2014 and later, coinciding with reduced onroad NOX emissions from vehicles with newer emission control technologies and improved treatment of chemistry, deposition, and vertical mixing in CMAQ. Our findings suggest that emissions temporalization of specific mobile source sectors have a small impact on model performance, while chemistry updates improve predictions of NOY but do not improve summertime NOX bias in the Baltimore/DC area. Sensitivity runs performed for different locations across the country suggest that the improvement in summer NOX performance can be attributed to updates in vertical mixing incorporated in CMAQv5.1.

Accurately determining sex using features like facial bone profiles and teeth is crucial for identifying unknown victims. Lateral cephalometric radiographs effectively depict the lateral cranial structure, aiding the development of computational identification models. This study develops and evaluates a sex prediction model using cephalometric radiographs with several convolutional neural network (CNN) architectures. The primary goal is to evaluate the model's performance on standardized radiographic data and real-world cranial photographs to simulate forensic applications. Six CNN architectures-VGG16, VGG19, MobileNetV2, ResNet50V2, InceptionV3, and InceptionResNetV2-were employed to train and validate 340 cephalometric images of Indonesian individuals aged 18 to 40 years. The data were divided into training (70%), validation (15%), and testing (15%) subsets. Data augmentation was implemented to mitigate class imbalance. Additionally, a set of 40 cranial images from anatomical specimens was employed to evaluate the model's generalizability. Model performance metrics included accuracy, precision, recall, and F1-score. CNN models were trained and evaluated on 340 cephalometric images (255 females and 85 males). VGG19 and ResNet50V2 achieved high F1-scores of 95% (females) and 83% (males), respectively, using cephalometric data, highlighting their strong class-specific performance. Although the overall accuracy exceeded 90%, the F1-score better reflected model performance in this imbalanced dataset. In contrast, performance notably decreased with cranial photographs, particularly when classifying female samples. That is, while InceptionResNetV2 achieved the highest F1-score for cranial photographs (62%), misclassification of females remained significant. Confusion matrices and per-class metrics further revealed persistent issues related to data imbalance and generalization across imaging modalities. Basic CNN models perform well on standardized cephalometric images but less effectively on photographic cranial images, indicating a domain shift between image types that limits generalizability. Improving real-world forensic performance will require further optimization and more diverse training data. Not applicable.

BackgroundManual assessment of bone marrow signal is time-consuming and requires meticulous standardisation to secure adequate precision of findings.ObjectiveWe examined the feasibility of using deep learning for automated segmentation of bone marrow signal in children and adolescents.Materials and methodsWe selected knee images from 95 whole-body MRI examinations of healthy individuals and of children with chronic non-bacterial osteomyelitis, ages 6–18 years, in a longitudinal prospective multi-centre study cohort. Bone marrow signal on T2-weighted Dixon water-only images was divided into three color-coded intensity-levels: 1 = slightly increased; 2 = mildly increased; 3 = moderately to highly increased, up to fluid-like signal. We trained a convolutional neural network on 85 examinations to perform bone marrow segmentation. Four readers manually segmented a test set of 10 examinations and calculated ground truth using simultaneous truth and performance level estimation (STAPLE). We evaluated model and rater performance through Dice similarity coefficient and in consensus.ResultsConsensus score of model performance showed acceptable results for all but one examination. Model performance and reader agreement had highest scores for level-1 signal (median Dice 0.68) and lowest scores for level-3 signal (median Dice 0.40), particularly in examinations where this signal was sparse.ConclusionIt is feasible to develop a deep-learning-based model for automated segmentation of bone marrow signal in children and adolescents. Our model performed poorest for the highest signal intensity in examinations where this signal was sparse. Further improvement requires training on larger and more balanced datasets and validation against ground truth, which should be established by radiologists from several institutions in consensus.

Prediction models are essential in clinical decision-making for estimating the probability of current (diagnosis, screening) or future (prognosis) outcomes. Network meta-analysis (NMA) serves as a powerful tool to compare the performance of multiple prediction models simultaneously. However, there is hardly any guidance on methods and reporting for studies employing NMA to evaluate prediction models. To provide an overview of NMAs assessing prediction model (external validation) performance, regardless of whether they use aggregate data (AD) or individual participant data (IPD). In addition, we offer recommendations for improving the reporting and conduct of NMAs in prediction model research. We searched PubMed and Embase up to September 1, 2025, to identify studies that addressed the evaluation of diagnostic or prognostic prediction model performance using NMA. We included articles that employed NMA to compare and assess at least three prediction models. We summarized the identified studies based on, eg, their application (diagnostic vs prognostic), data use (AD vs IPD), medical contexts in which the models were assessed, and evaluation metrics applied (eg, discrimination, calibration, and (re)classification). In addition, we examined the statistical approaches employed, the NMA assumptions (such as consistency, transitivity, and exchangeability), and the ranking methods used for model comparison. After screening 2436 articles, 28 were included. Twenty-six studies (92.9%) used AD, while two (7.1%) used IPD. Hospital care was the most common setting (n = 22; 78.6%), with respirology (n = 7; 25.0%) and cardiology (n = 5; 17.9%) as the most frequently studied clinical domains. Key NMA assumptions were addressed differently across the 28 NMAs: 14.3% (n = 4) discussed transitivity, similarity, or exchangeability, and 53.6% (n = 15) tested for consistency. The statistical approach also varied, with 60.7% of studies (n = 17) reporting Bayesian methods and 17.9% (n = 5) reporting frequentist approaches. Surface under the cumulative ranking was the predominant ranking method (n = 18). Most NMAs included 5 to 10 models in the network, with 5 NMAs analyzing more than 20 models. Performance metrics varied, with 39.3% of studies (n = 11) reporting discrimination measures, such as C statistics, while none reported calibration metrics. Sensitivity or specificity was provided in 64.3% of studies (n = 18), and no articles reported advanced decision-analytic metrics like the decision curve analysis. This scoping review highlights the limited and diverse use of NMA methods in evaluating prediction models, with a predominant reliance on aggregate rather than IPD, and inconsistent consideration of key NMA assumptions and model performance metrics. We provide recommendations for the reporting and conduct of an NMA of prediction model validation performance. Prediction models are tools used in medicine to estimate a patient's risk of developing a disease or experiencing a health outcome. Many different prediction models exist for the same condition, and it can be difficult for doctors and researchers to know which model performs best. One way to compare multiple models is through a statistical method called network meta-analysis (NMA), which is commonly used to compare treatments but has rarely been applied to prediction models. In our study, we reviewed all published NMAs that evaluated prediction models to see how they were conducted and reported. We looked at whether the studies reported important performance measures, such as how well the models could distinguish between patients with and without the outcome (discrimination), and how well the predictions matched actual outcomes (calibration). We also checked if key NMA assumptions were considered and how analyses were conducted. We found that most studies used summary data instead of patient-level data. Many did not report crucial performance measures and rarely checked NMA assumptions. There was also limited transparency in how the models were analyzed, making it difficult for others to reproduce the results. Our findings show that while NMA has great potential to help compare prediction models and identify the most reliable ones, current practice often lacks the detailed reporting needed to make these comparisons fully trustworthy. We recommend better reporting, sharing of data and analysis code, and careful checking of assumptions to help researchers and doctors choose the most reliable models, ultimately improving patient care.

Abstract. This study compares the performance of 12 regional chemical transport models (CTMs) from the third phase of the Model Inter-Comparison Study for Asia (MICS-Asia III) on simulating the particulate matter (PM) over East Asia (EA) in 2010. The participating models include the Weather Research and Forecasting model coupled with Community Multiscale Air Quality (WRF-CMAQ; v4.7.1 and v5.0.2), the Regional Atmospheric Modeling System coupled with CMAQ (RAMS-CMAQ; v4.7.1 and v5.0.2), the Weather Research and Forecasting model coupled with chemistry (WRF-Chem; v3.6.1 and v3.7.1), Goddard Earth Observing System coupled with chemistry (GEOS-Chem), a non-hydrostatic model coupled with chemistry (NHM-Chem), the Nested Air Quality Prediction Modeling System (NAQPMS) and the NASA-Unified WRF (NU-WRF). This study investigates three model processes as the possible reasons for different model performances on PM. (1) Models perform very differently in the gas–particle conversion of sulfur (S) and oxidized nitrogen (N). The model differences in sulfur oxidation ratio (50 %) are of the same magnitude as that in SO42- concentrations. The gas–particle conversion is one of the main reasons for different model performances on fine mode PM. (2) Models without dust emission modules can perform well on PM10 at non-dust-affected sites but largely underestimate (up to 50 %) the PM10 concentrations at dust sites. The implementation of dust emission modules in the models has largely improved the model accuracies at dust sites (reduce model bias to −20 %). However, both the magnitude and distribution of dust pollution are not fully captured. (3) The amounts of modeled depositions vary among models by 75 %, 39 %, 21 % and 38 % for S wet, S dry, N wet and N dry depositions, respectively. Large inter-model differences are found in the washout ratios of wet deposition (at most 170 % in India) and dry deposition velocities (generally 0.3–2 cm s−1 differences over inland regions).

Purpose: Numerous normal tissue complication probability (NTCP) models for radiation pneumonitis (RP) have been published in the literature, but minimal consideration is given to their predictive performance. Using several published model parameters, we quantified the range of model performance metrics to differentiate between and assess clinical utility of predictive RP models. Methods: Clinical, demographic, and dosimetric data was collected for 453 NSCLC patients treated with definitive photon radiotherapy. A series of logistic NTCP models were developed to assess overall performance. A baseline mean lung dose (MLD) model was considered in addition to our institutions best model, which incorporates MLD and smoking status. The model parameters from several publications were identified and incorporated in the analysis. The considered endpoint was CTCAEv3.0 grade&gt;=3. Metrics of model performance including area under the curve (AUC) and Spearmans rank correlation (rs) were calculated for each of the models to facilitate comparison. Results: The MLD plus smoking status model performed best in terms of observed AUC (0.685) and rs (0.337); however, model performance was similar to that achieved by all of the other tested models (AUC range=[0.663‐0.685] and rs range=[0.263‐0.337]). In all considered models, predictive performance was poor (i.e. AUC&lt;0.7 and rs&lt;0.4). Additionally, the narrow range of AUC/rs values indicates that there is marginal difference in the performance of any one model over another. Conclusion: The range of quantitative performance metrics indicates that there is minimal ability to differentiate between the selected models. The achieved values for AUC and rs indicate poor overall model accuracy, which limits the practical utility of NTCP models for RP prediction. Furthermore, model parameters from the literature cannot by applied to our dataset to improve RP prediction. Other model formulations and still unidentified patient factors should be considered to develop more applicable and clinically useful NTCP models for treatment personalization.

Power Quality Disturbances (PQD) has result in numerous failures and damage to electrical equipment. This paper utilized MATLAB Application to propose ways in detecting and classifying Voltage Sag, Swell and Transient. The proposal was divided into three parts which are detection, classification, and performance evaluation. The detection stage was done using Discrete Wavelet Transform in Wavelet Analyzer to obtain signal decomposition in different energy levels to be used in Energy Distribution Deviation (EDD) method. The classification stage was done in Classification Learner to check how good Multilayer Perceptron Neural Network able to trains, validates, and predicts as a classification model. The performance evaluation stage was done in Neural Net Fitting using Levenberg-Marquardt (LM) as training algorithm to see how well the model perform in term of Mean Square Error (MSE) and regression. This paper also discusses the effect of input ratio, activation function (Sigmoid, Tangent Hyperbolic, Rectified Linear Unit) and training algorithm (Levenberg-Marquardt, Bayesian Regularization, Scale Conjugate Gradient) towards accuracy in a Neural Network model. This study found that EDD was able to detect the difference in energy distribution of PQD properly. The Multilayer Perceptron model was observed to performed better and had higher accuracy when fed with more sample data, bigger layer size and activated using Tangent Hyperbolic (Tanh) activation function. Increasing layer size also resulted in slower prediction speed and longer training time. The model performance was evaluated with the lowest MSE and highest regression when Levenberg-Marquardt (LM) was implemented compared to Bayesian Regularization (BR) and Scale Conjugate Gradient (SCG).

Artificial intelligence (AI) has the potential to aid clinicians in assessing case difficulty in endodontics. The objectives of this study were to develop and validate deep learning models for the detection of clinically negotiable MB2 canals in periapical images of maxillary first and second molars, and to compare the performance of AI models with that of human clinicians. A total of 1504 pre-operative periapical images of maxillary first and second molars that were treated by endodontic specialists were collected with clinical data as to the presence or absence of a clinically negotiable MB2 canal. Six pretrained supervised convolutional neural networks (ResNet-18, ResNet-50, ResNeXt-101, VGG-16, DenseNet-121 and MobileNetV2) and three self-supervised models (DINO, SimCLR and BYOL) were fine-tuned using fivefold cross-validation. Model performance was evaluated on a hold-out test set using accuracy, precision, sensitivity, specificity, and F1-score with 95% confidence intervals. Three independent clinicians (an endodontist, an endodontic resident, and an oral and maxillofacial radiologist) also assessed the test set. In cross-validation, ResNet-50 achieved the highest mean accuracy (67.6%), while DINO was the top-performing self-supervised model (62.8%). ResNet-18, ResNet-50, ResNeXt-101, DenseNet-121 and DINO significantly outperformed BYOL (p < 0.01), while no significant differences were observed among the top-performing models. ResNet-18 achieved the highest accuracy at 66.0% (95% CI, 63.0-68.9) on the test set while human expert accuracy ranged from 53.6% to 61.4%. Stratified analysis showed a general trend for improved AI model performance in maxillary first molars and in teeth without full-crown restorations. There was no significant difference in the accuracy of the top-performing AI model and human experts (p > 0.05). Deep learning models performed similarly to clinician experts in identifying clinically negotiable MB2 canals in periapical images of maxillary first and second molars. These findings support the potential role of AI in endodontic case difficulty assessment.

We compared modeled and observed streamflow trends from 1984 to 2016 using five statistical transfer models and one deterministic, distributed-parameter, process-based model, for 26 flow metrics at 502 basins in the United States that are minimally influenced by development. We also looked at a measure of overall model fit and average bias. A higher percentage of basins, for all models, had relatively low trend differences between modeled and observed mean/median flows than for very high or low flows such as the annual 1-day high and 7-day low flows. Mean-flow metrics also had the largest percentage of basins with relatively good overall model fit and low bias. The five statistical transfer models performed better at more basins than the process-based model. The overall model fit for all models, for mean and/or high flows, was correlated with one or more measures of basin precipitation or aridity. Our study and previous studies generally observed good model performance for high flows up to 90th or 95th percentile flows. However, we found model performance was substantially worse for more extreme flows, including 99th percentile and annual 1-day high flows, indicating the importance of including more extreme high flows in analyses of model performance.

Accurate landslide susceptibility assessment is vital for disaster prevention, but current mapping lacks systematic analysis of the underlying mechanisms between multi-scale factors and model performance. Taking Zhenxiong County as an example, this paper combines the IV, WOE, LR models, and PCA to reveal the impact of methodological differences and scale selection on mapping results, and quantitatively evaluates them using ROC curves and landslide density statistics. Results show that: 1) The scale effect of influencing factors is significant. Natural factors such as topography, geological conditions, and rainfall play dominant roles at the regional scale, while the impacts of human activities, geological features, and soil erosion intensity are more pronounced at local and moderate scales. 2) The landslide susceptibility mapping results of the three models at different spatial scales show similar spatial distribution trends. As the spatial scale increases, high/very high susceptibility areas and low/very low susceptibility areas spread outward, while the spatial distribution of medium susceptibility areas shows a fragmented expansion outward first and then agglomeration and contraction inward. 3) Scale selection significantly affects the accuracy of landslide susceptibility mapping, and expanding the spatial scale appropriately improves mapping precision. The IV and WOE models show the highest AUC at the 600-m buffer, while the LR model peaks at 400 m. In terms of landslide identification accuracy, the IV model performs best at 400-m buffer, WOE at 600-m buffer, and LR at 100 -meter buffer. 4) Different methods have different mapping performances. Overall, the IV model performs best, followed by the WOE model, with the LR model lagging behind. In terms of high-risk area recognition, the LR model excels, followed by the IV model, while the WOE model performs relatively poorly. 5) Scale and method selection significantly impact landslide susceptibility mapping outcomes. The IV model excelled in global prediction at the 600-m buffer, whereas the LR model was effective in pinpointing high-risk areas at the 100-m buffer. This paper proposes a landslide susceptibility evaluation method that integrates model performance and scale effects, enhancing disaster assessment and prevention capabilities.