The Boston qualitative scoring system for the Rey-Osterrieth complex figure: Description and interrater reliability

PurposeThis study aimed to use three statistical methods to investigate the inter-rater and intra-rater reliability of the Chinese version of the Short Orientation-Memory-Concentration Test (C-SOMC) for people who have had a stroke.MethodsForty-four participants (31 men and 13 women) with a mean age of 59.05 ± 10.79 years who have had experienced a single episode of stroke were enrolled in this study. To determine the inter-rater reliability of the C-SOMC, the test was administered to each participant on the same day by two raters (A and B) who each had more than 1 year of work experience. To determine intra-rater reliability, rater B re-evaluated 36 of the 44 participants with the C-SOMC on the subsequent day. Intraclass correlation coefficients (ICCs), paired-samples t-tests, and Bland-Altman plots were used to analyze the inter-rater and intra-rater reliability.ResultsThe evaluation of inter-rater reliability for the total score and item 1, 4, 5, 6, and 3/7 showed respective ICCs of 0.959, 0.918, 1.000, 0.942, 0.905 and 0.913, indicating excellent inter-rater reliability for the C-SOMC. Item 2 had an ICC of 0.796, indicating moderate to good inter-rater reliability. The evaluation of intra-rater reliability showed an ICC of 0.978 for the total score, and respective ICCs of 1.000, 1.000, 1.000, 0.968, 0.973 and 0.929 for the individual items, indicating excellent intra-rater reliability for the C-SOMC. The paired-samples t-test for the C-SOMC showed no statistically significant differences (P > 0.05) between ratings made by two different raters or by the same rater on separate occasions. The minimal detectable change value at the 95% confidence threshold (MDC95) of the SOMC score was found to be 2.14. Bland-Altman plots showed a mean difference of 0.02 and 95% limits of agreement (95% LOA) ranging from −3.86 to 3.90 for the inter-rater measurements and a mean difference of 0.33 and 95% LOA of −2.71 to 3.37 for the intra-rater measurements.ConclusionsThe C-SOMC demonstrated excellent inter-rater and intra-rater reliability when administered to people who have had a stroke. The C-SOMC may be used to screen for cognitive impairment in people who have had a stroke.

Muscle length is a common component of the physical therapy examination, which may include the prone knee flexion (Ely) and active knee extension (AKE) tests. Clinicians using these tests should understand the clinimetric properties. To investigate the reliability and minimal detectable change (MDC95) of the Ely and AKE tests. Reliability analysis. Seventy-one asymptomatic adults (mean age 24.6 + /- 2.8 years) were recruited based on a convenience sample. Two examiners each performed the Ely and AKE test one time each in an intrasession design for the interrater reliability component, with one examiner repeating the tests one time 48 hours later to determine the intra-rater reliability. Results were recorded based on one trial per test and utilized a pelvic strap for the Ely test and an adjustable bolster for the AKE test. A separate researcher recorded measurements and results were blinded from the examiners. The Ely test had excellent intra-rater and inter-rater reliability with an intraclass correlation coefficient (ICC) (3,1) of 0.900 and ICC (2,1) of 0.914 respectively. The intra-rater and inter-rater reliability of the AKE test was good with ICC (3,1) of 0.882 and ICC (2,1) 0.886 respectively. The MDC95 indicated that a change greater than or equal to 8° and 12° is required to exceed the threshold of error for the Ely and AKE test respectively. The Ely and AKE tests have good to excellent inter-rater and intra-rater reliability for measuring rectus femoris and hamstring muscle length when stabilization of the pelvis and hip is accounted for. The MDC should be considered as a threshold for true change in the asymptomatic adult population. 2b.

Large language models, such as OpenAI's ChatGPT, have the potential to revolutionize patient education. These platforms allow for a vast collection, analysis, and organization of information largely unavailable to online users. Within medicine, these tools could help complement physicians in better educating patients on complex and routine medical information. Currently, limited literature exists on the reliability of such tools to provide high-quality information to patients inquiring about gender-affirming top surgery. Therefore, this study aimed to evaluate ChatGPT's performance when generating patient-level information on gender-affirming top surgery compared with the current online information provided by the American Society of Plastic Surgery (ASPS) using the Modified Ensuring Quality Information for Patients (mEQIP) tool. ChatGPT-4-generated patient-level education on transmasculine and transfeminine gender-affirming top surgery was compared against current online content provided by the ASPS. ChatGPT-4 patient content was generated by individually formatting standardized mEQIP content items to incorporate the topic of gender-affirming top surgery into ChatGPT-4, with responses recorded for each item. Four experts in gender-affirming top surgery independently rated both sources using a 36-item mEQIP tool. Paired t-tests comparing overall and content-specific mEQIP scores of the ChatGPT-4 and ASPS material were then estimated to measure the quality of the content. The effect size between the two groups was evaluated using Cohen's d. Lastly, Cronbach's alpha and ICC (Intraclass Correlation Coefficient) were calculated to measure internal consistency among raters and interrater agreement. When analyzing ChatGPT-4 and ASPS patient material, paired t-tests showed a statistically significant increase in overall mEQIP scores for ChatGPT with a mean difference of 7.50 (CI 6.75-8.25; p<0.001). For the mEQIP content-specific scores, a paired t-test revealed a similarly significant increase in ChatGPT scores with a mean difference of 9.75 (CI 9.26-10.24; p<0.001). When evaluating the effect size, a paired Cohen's d value of 13.00 was calculated, demonstrating a statistically significant difference in magnitude between the two groups. To measure internal consistency among raters and interrater agreement, an ICCand Cronbach's alpha were calculated for both ASPS and ChatGPT. The ASPS-mEQIP showed good internal consistency and excellent interrater reliability (ICC=0.89, Cronbach's α=0.84), while ChatGPT-mEQIP showed excellent internal consistency and excellent interrater reliability (ICC=0.96, Cronbach's α=0.96). These results demonstrate that ChatGPT-4-generated patient education on gender-affirming top surgery exceeded the current ASPS online content in both overall and content-specific scores, as measured by the mEQIP tool. ChatGPT achieved significantly higher scores across both domains with large effect sizes, and raters demonstrated excellent internal consistency and excellent interrater reliability. Going forward, commonly accessible artificial Intelligences (AIs), such as ChatGPT, may serve as a valuable complement to patient education and shared decision-making within plastic and reconstructive surgery, though future studies are warranted to evaluate freely generated responses to better reflect current AI use.

To establish outcome of brow suspension techniques, a reliable and clinically suitable measurement tool is required. Most of the published methods lack statistical validation, and in none of the reviewed studies, the standardized photograph protocols were validated. Fifteen subjects were independently photographed by 2 different professional medical photographers. Adobe Photoshop CS5 software (San Jose, Calif) was used to perform the measurements based on interpupillary distance and medial, midpupil, and lateral reference points on the brow. To establish intrarater reliability, the photos of both photographers of the same subject were analyzed by the same researcher. To establish interrater reliability, the photos of 1 photographer were analyzed by 1 researcher, whereas the photos of the second photographer were analyzed by a second researcher. Reliability was assessed using the intraclass correlation coefficients (ICCs) and the smallest detectable differences. The ICCs indicated good to excellent interrater and intrarater reliability. The ICCs of the interrater reliability were lower compared with those of the intrarater reliability, although they could still be categorized as good to excellent. We found that the lateral measurements had the highest reliability, and the medial measurements, the lowest reliability. The current study supports that the "en face" photographs in our database are truly standardized. The good to excellent interrater and intrarater reliability of the present method makes it a valid measuring tool.

Introduction: Respiratory sarcopenia, defined as reduced respiratory muscle mass and strength, and has been reported as a potential prognostic factor in lung cancer. In patients undergoing non-small cell lung cancer (NSCLC) surgery, preoperative chest computed tomography (CT) is routinely performed and offers an opportunity to assess respiratory muscle volume. However, the reliability of CT-based measurements for different respiratory muscles has not been established, and standardized protocols are lacking. This study aimed to evaluate the inter-rater reliability of respiratory muscle volume measurements using preoperative CT images in patients with lung cancer.Methods: Five patients who underwent lung cancer surgery at our hospital between July and August 2023 were selected. Four physical therapists with no prior experience in measuring muscle cross-sectional areas (CSAs) on chest CT images received a lecture on the measurement methods from a radiology technician. Respiratory muscle CSAs (pectoralis, intercostal, and latissimus dorsi muscles) were measured from preoperative chest CT images using an image analysis system. Measurements were performed four times on different days. Intra-class correlation coefficients (ICC) were used for intra- and inter-rater reliability assessments, applying a two-way mixed-effects model with absolute agreement: ICC (2, 1) for intra-rater and ICC (2, k) for inter-rater reliability. ICC ≥ 0.75 indicated excellent reliability.Results: Participants had a median age of 75.0 (72.0, 79.0) years and body mass index of 23.0 (20.0-24.0). Four participants (80%) were at tumor stage I. Excellent intra-assessor reliability was demonstrated for all the respiratory muscles (pectoralis CSA [ICC: 0.908 to 0.996], intercostal CSA [ICC: 0.941 to 0.985], and latissimus CSA [ICC: 0.885 to 0.984]). Excellent inter-rater reliability was demonstrated for the pectoralis and intercostal CSAs.Conclusions: This study confirmed excellent intra- and inter-rater reliability for measuring respiratory muscle CSAs using CT images in patients with preoperative lung cancer in the pectoral and intercostal muscles. This suggests that CT assessment of respiratory muscle volume may be widely used as a clinically useful indicator. Further standardization and simplification of this method will allow its use in broader clinical settings.

BackgroundOsteoarthritis (OA) of the hip affects the entire joint structure and function and leads joint capsular changes which result limitation in range of motion (ROM). Therefore, measuring ROM is an...

Remote administration of physical performance tests among persons with and without a cancer history: Establishing reliability and agreement with in-person assessment

A reliable method for assessing ichthyosis severity has not been uniformly agreed upon. The objective of our study was to develop and validate a tool to measure the clinical severity of the congenital ichthyoses, the Congenital Ichthyoses Severity Index. A prospective study was performed to determine reliability of three Likert scales to evaluate clinical severity of ichthyosis. Thirty-eight subjects recruited from the Foundation for Ichthyosis and Related Skin Types National Conference were evaluated separately by two blinded investigators using the Likert scales. Subjects were then asked to evaluate themselves using these scales. Inter-rater reliability was determined between ratings provided by all three raters. Test-retest validation was conducted with 21 subjects who completed follow-up surveys 4 weeks later. Our severity scales had excellent inter-rater and test-retest reliability as determined by intraclass correlation coefficients (ICC >0.7), with the exception of our hyperkeratosis scale, which demonstrated moderate test-retest reliability (ICC = 0.4). This pilot study provides a promising method for evaluating clinical severity of the congenital ichthyoses, one easily employed by both physicians and patients. Future epidemiologic studies may benefit from use of this instrument, as well as studies evaluating emerging therapies for ichthyosis.

Objective: This study aimed to determine the intra-rater and inter-rater reliability and validity of a hybrid device, combining virtual reality goggles, a magnetometer and an inclinometer application for smartphones, to measure craniocervical range. Summary of Background Data: Accurate evaluation of craniocervical range of motion is important for early detection of certain diseased conditions and monitoring the progress of interventions. The universal goniometer is widely used for the measurement but it requires experienced practitioners. Whether a combination of virtual reality goggles and smartphone applications can provide the same or better performance compared with the goniometer is still unknown. Methods: Forty-one healthy adults from the department of physical medicine and rehabilitation were recruited for craniocervical range examination (flexion, extension, side-bending to the right or left and rotating to the right or left) by using the hybrid device and universal goniometer. Using the hybrid device, repeated measurements were performed twice by a primary rater and once by a second rater. The primary rater also conducted a measurement using the universal goniometer in the same cohort. The intra-rater and inter-rater reliability (intra-class correlation coefficient (ICC)) were calculated using the two-way random effect model, whereas the validity was examined by the Pearson correlation coefficient and Bland-and-Altman plot. The interval between the first and second sessions of the measurement for intra-rater reliability was set at 30 min. Results: Excellent intra-rater (ICC ≥ 0.925) and inter-rater (ICC ≥ 0.880) reliability was noted for the hybrid device. The minimal detectable changes from intra-observer and inter-observer comparisons ranged between 4.12° and 7.42° in all six directions. The Bland-and-Altman plot revealed small mean differences (≤1.68°) between the hybrid device and universal goniometer. Both instruments had highly correlated measurements of craniocervical motion (r values ≥ 0.918). Conclusion: For healthy participants, excellent intra-rater and inter-rater reliability was noted for the hybrid device, and the measurements were consistent with the universal goniometer measurements. Future studies are needed to examine whether the device can perform similarly for patients with neck disorders.

1. To assess the Inter-rater reliability and test-retest reliability of FPI-6 total score and individual scores in static foot posture evaluation among elderly female patients with knee osteoarthritis (KOA), aiming to establish the reliability of the FPI-6 scale. 2. To investigate the disparity between dominant and non-dominant quadriceps characteristics in elderly female KOA patients, as well as explore the correlation between quadriceps characteristics and abnormal foot posture, thereby offering novel insights for the prevention and treatment of KOA. The study enrolled a total of 80 lower legs of 40 participants (all female) with unilateral or bilateral KOA, who were assessed by two raters at three different time points. The inter-rater and test-retest reliability of the FPI-6 was evaluated using the intra-class correlation coefficient (ICC), while the absolute reliability of FPI-6 was examined using the standard error of measurement (SEM), minimum detectable change (MDC), and Bland-Altman analysis. The internal consistency of FPI-6 was assessed using Spearman's correlation coefficient. Additionally, MyotonPRO was employed to assess quadriceps muscle tone and stiffness in all participants, and the association between quadriceps muscle tone/stiffness and the total score of FPI-6 was analyzed. Our study found excellent inter-rater and test-retest reliability (ICC values of 0.923 and 0.931, respectively) for the FPI-6 total score, as well as good to excellent reliability (ICC values ranging from 0.680 to 0.863 and 0.739-0.883) for individual items. The SEM and MDC values for the total score of FPI-6 among our study inter-rater were 0.78 and 2.15, respectively. and the SEM and MDC values for the test-retest total score of FPI-6 were found to be 0.76 and 2.11, respectively. Furthermore, the SEM and MDC values between inter-rater and test-retest across six individual items ranged from 0.30 to 0.56 and from 0.84 to 1.56. The Bland-Altman plots and respective 95% LOA showed no evidence of systematic bias. In terms of the mechanical properties of the quadriceps on both sides, the muscle tone and stiffness of rectus femoris (RF), vastus medialis (VM), and vastus lateralis (VL) were significantly higher in the non-dominant leg compared to the dominant leg. Additionally, in the non-dominant leg, there was a significant positive correlation between the muscle tone and stiffness of VM, VL, RF and the total score of FPI-6. However, in the dominant leg, only VM's muscle tone and stiffness showed a significant positive correlation with the total score of FPI-6. The reliability of the FPI-6 total score and its six individual items was good to excellent. Our findings offer a straightforward and dependable approach for researchers to assess foot posture in elderly female patients with KOA. Furthermore, we observed significantly greater quadriceps tension and stiffness in the non-dominant leg compared to the dominant leg. The FPI-6 total score exhibited a significant correlation with changes in quadriceps muscle performance among KOA patients. These observations regarding the relationship between changes in quadriceps muscle performance and foot posture in elderly female KOA patients may provide novel insights for disease prevention, treatment, and rehabilitation.

PurposeThe brief negative symptoms scale (BNSS) is a concise instrument used to assess negative symptoms of subjects with schizophrenia covering five domains of negative symptoms and is suitable for use in clinical, experimental, and epidemiological settings. The original and translated version of BNSS has thus far been shown to have adequate psychometric properties. This study aimed to examine internal consistency, inter-rater and test-retest reliability, discriminant and convergent validity, and factor structure of the Japanese version of BNSS.Patients and methodsThe assessment was performed by 11 raters using interview videos of nine subjects. Reliability was calculated with Cronbach’s alpha for internal consistency and intra class correlation coefficient (ICC) for inter-rater reliability. Pearson’s correlation coefficients were calculated to estimate the test-retest reliability. In addition to BNSS, Scale for assessment of negative symptoms (SANS) and scale for assessment of positive symptoms (SAPS) was obtained to assess the convergent and discriminant validity. Factor structure was assessed using principle factor analysis.ResultsThe Japanese BNSS showed excellent internal consistency (Cronbach’s alpha=0.95), inter-rater reliability (intra class correlation coefficient=0.97), and test-retest reliability (r=0.94, p<0.001). The convergent validity shown by correlation with SANS total score (r=0.87, p<0.001) and discriminant validity shown by correlation with SAPS total score (r=0.17, p=−0.68) were also good. Principal factor analysis revealed a two-factor structure of BNSS, although the loading of each item differed from that in the literature.ConclusionOur pilot study demonstrated that Japanese BNSS had good psychometric properties which were achieved with relatively brief training. Further studies with more subjects and raters with various backgrounds recruited from multiple sites are warranted.

Radiographic evaluation of lumbar intervertebral disc height index: An intra and inter-rater agreement and reliability study

Intrarater and Inter-rater Reliability of Maximal Voluntary Neck Muscle Strength Assessment Using a Handheld Dynamometer in Women With Headache and Healthy Women

Objectives: The primary objective of this study was to determine the inter-rater reliability of the FMS&#13;\nwhen used by novice NHS (National Health Service) Physiotherapists. The secondary objective was to&#13;\ndetermine whether clinical specialism has any impact on the inter-rater reliability of the FMS.&#13;\nDesign: Reliability study.&#13;\nMethods: Forty participants with no recent MSK injury were video recorded completing the 7 component&#13;\nFMS tests. Six NHS Physiotherapists with no previous experience using the FMS attended a 2 hour&#13;\ntraining programme delivered by a certified FMS practitioner. Raters then viewed and scored videos of the&#13;\n40 participants completing the FMS.&#13;\nResults: The inter-rater reliability of the FMS composite score was excellent (ICC of 0.82 (95% CI: 0.41-&#13;\n0.93)).Non-specialist rotational Physiotherapists demonstrated excellent inter-rater reliability (ICC =&#13;\n0.89, 95%CI 0.78-0.94), whereas the specialist musculoskeletal Physiotherapists demonstrated good interrater&#13;\nreliability (ICC = 0.79, 95%CI 0.19-0.92) for FMS composite score. The seven individual movement&#13;\ntests of the FMS demonstrated poor to excellent inter-rater reliability. The Hurdle Step was the least&#13;\nreliable of the movement tests (kw = 0.15, 95% CI: -0.09-0.38), whereas Shoulder Mobility was the most&#13;\nreliable of the movement tests (kw= 0.85, 95%CI: 0.72-0.97).The seven individual movement tests of&#13;\nthe FMS demonstrated moderate to excellent inter-rater reliability between non-specialist rotational&#13;\nPhysiotherapists. In contrast, the seven individual movement tests of the FMS demonstrated poor to&#13;\nexcellent inter-rater reliability between specialist musculoskeletal Physiotherapists.&#13;\nConclusion: The FMS represents a good attempt to objectify the subjective with the FMS composite score&#13;\ndemonstrating excellent inter-rater reliability Due to poor construct validity, it has been suggested only&#13;\ncomponent scores should be utilised.&#13;\nThe results from this study suggest that five of the seven individual movement tests do not demonstrate&#13;\nacceptable reliability for clinical use.&#13;\nWith the composite score lacking construct validity and the majority of the component scores lacking both&#13;\nintra- and inter-rater reliability, the continued use of FMS within clinical practice is not supported.

Inter-rater reliability of shoulder measurements in middle-aged women