Berlin Center for Epidemiology and Health Research

Background/Objectives: Multimodal large language models (MLLMs) offer potential for automated fracture detection, yet their diagnostic stability under repeated inference remains underexplored. This study evaluates the diagnostic accuracy, stability, and intra-model consistency of four MLLMs in detecting hand fractures on plain radiographs. Methods: In total, images of hand radiographs of 65 adult patients with confirmed hand fractures (30 phalangeal, 30 metacarpal, 5 scaphoid) were evaluated by four models: GPT-5 Pro, Gemini 2.5 Pro, Claude Sonnet 4.5, and Mistral Medium 3.1. Each image was independently analyzed five times per model using identical zero-shot prompts (1300 total inferences). Diagnostic accuracy, inter-run reliability (Fleiss’ κ), case-level agreement profiles, subgroup performance, and exploratory demographic inference (age, sex) were assessed. Results: GPT-5 Pro achieved the highest accuracy (64.3%) and consistency (κ = 0.71), followed by Gemini 2.5 Pro (56.9%, κ = 0.57). Mistral Medium 3.1 exhibited high agreement (κ = 0.88) despite low accuracy (38.5%), indicating systematic error (“confident hallucination”). Claude Sonnet 4.5 showed low accuracy (33.8%) and consistency (κ = 0.33), reflecting instability. While phalangeal fractures were reliably detected by top models, scaphoid fractures remained challenging. Demographic analysis revealed poor capabilities, with age estimation errors exceeding 12 years and sex prediction accuracy near random chance. Conclusions: Diagnostic accuracy and consistency are distinct performance dimensions; high intra-model agreement does not imply correctness. While GPT-5 Pro demonstrated the most favorable balance of accuracy and stability, other models exhibited critical failure modes ranging from systematic bias to random instability. At present, MLLMs should be regarded as experimental diagnostic reasoning systems rather than reliable standalone tools for clinical fracture detection.

Health economic methods can support the development and evaluation of new healthcare interventions by generating data on the resources used and relating these to a defined benefit. However, the standard methodology of health economic evaluation that is usually used does not do justice to the high degree of complexity of interventions in healthcare. As a result, there is a lack of decision-relevant information, for example, on the preferences of the target group, on spillover effects on the part of carers, or on implementation costs and the role of different contexts in the implementation of interventions into routine care. The UK Medical Research Council's (MRC) standard-setting framework for complex interventions therefore emphasises the need to incorporate health economic aspects more strongly into all phases of the development and evaluation of complex interventions. To make this possible, the MRC's recommendations for expanding and adapting the standard methodology of health economic evaluation must be concretised and supplemented. Building on already established methodological procedures, recommendations should be developed and proposals for necessary further research formulated.

The Systemic Lupus International Collaborating Clinics/American College of Rheumatology Damage Index (SDI) captures organ damage (OD) in patients with systemic lupus erythematosus (SLE). However, SDI is not always documented in the clinic or medical claims data. This non-interventional retrospective cohort study (GSK Study 209523) used a claims-based algorithm to estimate annual damage scores and associated economic burden in patients with SLE in Germany. Patients were identified from the Betriebskrankenkassen German Sickness Fund Database based on the first recorded SLE diagnosis code (index). A claims-based algorithm incorporating conditions listed in the SDI was used to identify OD and estimate damage scores for each follow-up year. Annual costs were estimated from inpatient and outpatient resources and stratified by a damage score point increase. Patients with SLE (n=2121; pre-existing: n=1037; newly diagnosed: n=1084) and without SLE (n=6308) were included. At baseline, 60.5% (n=1283) of patients with SLE had OD, including 55.3% (n=599) newly diagnosed patients. In Year 1, the mean damage score was 1.82 among all patients with SLE; it was similar between patients with baseline OD and pre-existing (2.76) and newly diagnosed SLE (2.89). Among patients with SLE, during years with a 1-, 2- and ≥3-point increase in damage score, median annual costs were 1.94, 3.75 and 7.84 times those in years without damage score increase. Our findings suggest that real-world administrative claims data can effectively estimate damage scores in patients with SLE, and that the increase in scores correlates with a higher economic burden.

ObjectivesHospitals play a major role in generating clinical evidence on new medical technologies. Thus far, the extent of German hospitals’ contribution to the evidence base has not been sufficiently investigated. This study aims to: (1) examine the utilization of new medical technologies in German hospitals and its relationship to different hospital characteristics; (2) investigate the participation of German hospitals in research on these technologies and the association between hospital characteristics and research involvement; and (3) investigate the contribution of German hospitals to international research activities, including the levels of evidence of any studies conducted.MethodsUsing a systematically derived sample of 13 new medical technologies and various data sources, we retrospectively analyzed the utilization of and research activities by German hospitals between 2005 and 2017 and explored which hospital characteristics they were associated with. The data were analyzed descriptively and are expressed as bar plots, box plots, quartiles, and crude odds ratios (ORs).ResultsThe proportion of German hospitals using new technologies while also being involved in related clinical research was relatively low (ranging from 0.3% to 29.4%, except for transcatheter aortic valve implantation (TAVI), with 60.7%), particularly for prospective studies. Research involvement was positively associated with university hospital status, larger bed capacity, and public ownership. Overall, the research involving German hospitals predominantly consisted of single-arm studies and not randomized controlled trials (RCTs).ConclusionsOur study identified a gap between hospitals using new medical technologies and their involvement in evidence generation. This imbalance can contribute to uncertainty regarding the actual efficacy, effectiveness and safety of new medical technologies. To ensure evidence-based patient care, it is therefore essential to strengthen the link between research and practice, in both directions. A first step to achieve this could entail restricting the use of new medical technologies to specialized innovation centers (e.g., university hospitals, specialized hospitals) during the initial years of their utilization to ensure an adequate evidence base is generated before widespread implementation.