Response Rates, Reliability, and Convergent Validity in Experience Sampling Data.

ObjectivesTo develop and validate a bilingual experience survey for use in any NHS healthcare setting, to support service improvement.DesignA prospective mixed-methods study.SettingAny healthcare setting in NHS Wales including primary, secondary, urgent and planned care.ParticipantsAn opportunistic sample of people with experience of using local healthcare services. Qualitative interviews and focus groups were held to develop a draft survey. These were followed by online data collection from a wide participant sample for statistical validation. The tool was translated and linguistically validated following recognised methods. Patient engagement leads were involved to ensure the tool met their needs.ResultsWe conducted and analysed five focus groups and four interviews, consisting of 33 people in total. 12 draft questions were developed related to key aspects of patient experience. A series of online surveys were conducted to test the draft questions, with 769 responses received. Data were analysed to assess completion rates, intra-rater reliability, internal consistency and convergent validity. One question had both sub-par intrarater reliability and poor convergent validity, and despite attempts to improve the wording, it failed to meet minimum requirements of validity and was subsequently removed. The final validated People’s Experience Survey (PES) was subsequently translated into Welsh and validated with service users.ConclusionsThe PES is a reliable and valid tool, suitable for use in any healthcare setting. The robust processes that have been undertaken ensure that the questions included are available bilingually to collect reliable, meaningful data to support service improvement work.

BackgroundThe aim of this study was to adapt, translate and validate the Rapid Estimate of Adult Literacy in Dentistry (MREALD-30) instrument for the Orang Asli population in Malaysia.MethodsAfter translation and cross-cultural adaptation, interviews were conducted with 326 participants of the Temuan tribe from village Kampung Tering in Johol, Negeri Sembilan, Malaysia. The instrument's validity was assessed using the scores of MREALD-30, which were compared based on occupation, monthly household income, educational attainment, general literacy, use of dental services, and three dental outcomes. A questionnaire containing socio-behavioral information and validated Malay Oral Health Impact Profile (M-OHIP-14) was also administered. The reliability of the MREALD-30 was assessed by re-administering it to 30 subjects after two weeks. Its correlations evaluated convergent and discriminative validity of MREALD-30 with the level of education and dental visiting habits, monthly household income, respectively. Predictive validity was assessed with M-OHIP-14, while construct validity was evaluated by exploratory factor analysis using the Rasch model.ResultsThe internal consistency of the MREALD-30 measured by Cronbach's alpha was 0.89. The test–retest reliability was excellent (ICC 0.95, k = 0.85). MREALD-30 exhibited good construct validity. Rasch analysis showed two factors, and infit mean-square statistics for MREALD-30 were all within the desired range of 0.50–2.0. The discriminant validity and predictive validity were statistically significant (p < 0.05).ConclusionsMREALD-30 showed very strong reliability, good construct, discriminant, and predictive validity, but poor convergent validity. Overall, it showed good psychometric properties and can be used in these community settings.

Makaracı, Y, Nas, K, Ruiz-Cárdenas, JD, Gündüz, K, Aydemir, M, and Orange, ST. Test-retest reliability and convergent validity of piezoelectric force plate measures of single-leg sit-to-stand performance in trained adults. J Strength Cond Res 37(12): 2373-2380, 2023-The single-leg sit-to-stand (STS) test has emerged as a promising method of assessing lower-limb functional strength and asymmetry. However, the reliability of its performance parameters on a force plate has not been explored. This study examined the test-retest reliability and convergent validity of the single-leg STS test performed on a piezoelectric-based force plate in trained subjects. Thirty trained male adults (age: 21.4 ± 1.7 years) performed 3 separate single-leg STS days of testing to assess both intraday and interday reliability. Performance parameters included STS time, ground reaction force (GRF), and center of pressure (CoP) sway velocity. The relationship between single-leg STS parameters and unilateral countermovement jump (CMJ) variables was assessed for convergent validity. Intraclass correlation coefficient (ICC) and coefficient of variation (CV) were calculated for reliability analyses, and convergent validity was assessed with Spearman's correlation coefficient (ρ). In the dominant leg, single-leg performance parameters showed moderate-to-excellent intraday reliability (ICC = 0.65-0.90, CV = 4.3-11.2%) and moderate interday reliability (ICC = 0.54-0.74, CV = 5.8-13.5%). In the nondominant leg, all single-leg STS performance parameters showed good intraday (ICC = 0.79-0.86, CV = 3.8-9.8%) and interday reliability (ICC = 0.75-0.82, CV = 4.6-9.7%). STS times in the dominant and nondominant legs were inversely related to unilateral CMJ velocity (ρ = -0.47 and -0.38, respectively). CoP sway velocity in the nondominant leg showed positive correlations with unilateral CMJ power and velocity (ρ = 0.38 and 0.54, respectively). In conclusion, the force plate-based single-leg STS test provides reliable measures of STS time, GRF, and CoP sway velocity in trained adults and could be used to assess lower-limb function and asymmetry.

PurposeThe brief negative symptoms scale (BNSS) is a concise instrument used to assess negative symptoms of subjects with schizophrenia covering five domains of negative symptoms and is suitable for use in clinical, experimental, and epidemiological settings. The original and translated version of BNSS has thus far been shown to have adequate psychometric properties. This study aimed to examine internal consistency, inter-rater and test-retest reliability, discriminant and convergent validity, and factor structure of the Japanese version of BNSS.Patients and methodsThe assessment was performed by 11 raters using interview videos of nine subjects. Reliability was calculated with Cronbach’s alpha for internal consistency and intra class correlation coefficient (ICC) for inter-rater reliability. Pearson’s correlation coefficients were calculated to estimate the test-retest reliability. In addition to BNSS, Scale for assessment of negative symptoms (SANS) and scale for assessment of positive symptoms (SAPS) was obtained to assess the convergent and discriminant validity. Factor structure was assessed using principle factor analysis.ResultsThe Japanese BNSS showed excellent internal consistency (Cronbach’s alpha=0.95), inter-rater reliability (intra class correlation coefficient=0.97), and test-retest reliability (r=0.94, p<0.001). The convergent validity shown by correlation with SANS total score (r=0.87, p<0.001) and discriminant validity shown by correlation with SAPS total score (r=0.17, p=−0.68) were also good. Principal factor analysis revealed a two-factor structure of BNSS, although the loading of each item differed from that in the literature.ConclusionOur pilot study demonstrated that Japanese BNSS had good psychometric properties which were achieved with relatively brief training. Further studies with more subjects and raters with various backgrounds recruited from multiple sites are warranted.

Summary This study tested the convergent and discriminant validities of various psychological techniques used for the measurement of the affects of anxiety, depression and hostility. The subjects were 29 psychiatric patients and 25 normal controls, all males. Subjects were interviewed and rated for the three affects and given an extensive battery of tests. Anxiety and depression measures from ratings, checklists, and questionnaires demonstrated good convergent validity but poor discriminant validity. The projective methods did not demonstrate much convergent validity. Hostility was separable from anxiety and depression but the tests showed poor convergent validity for hostility. In the normals, fantasy hostility (TAT) and admitted hostility (Multiple Affect Adjective Check List) showed some convergence, but in the patients there was no congruence between these two levels.

The study included neonates between 24 and 42 weeks of gestational age. Convergent construct validity was analyzed by comparing COMFORTneo-Brazil scores with those from the Échelle Douleur Inconfort Nouveau-Né (EDIN) scale. Two trained raters performed bedside evaluations and video assessments, separated by 10-14 days, to assess intra- and inter-rater reliability. Spearman's correlation, linear weighted kappa, intraclass correlation coefficient (ICC), Bland-Altman plots, and Cronbach's alpha were used in the analysis. One hundred and thirty-three evaluations were conducted. There was excellent convergent validity between COMFORTneo-Brazil and EDIN scores (rho = 0.929; p < 0.001). The COMFORTneo-Brazil demonstrated excellent intra- and inter-rater reliability (ICC > 0.9). Internal consistency was also high (Cronbach's alpha = 0.858). The COMFORTneo-Brazil is a valid and reliable tool for prolonged pain assessment in neonates.

ABSTRACTThe standardized Mensendieck test (SMT) was developed to quantify posture, movement, gait, and respiration. In the hands of an experienced therapist, the SMT is proven to be a reliable tool. It is unclear whether posture, movement, gait, and respiration are related to the degree of functional disability in patients with chronic pain. The objective of this study was to assess the reliability and convergent validity of the SMT in a heterogeneous sample of 50 patients with chronic pain. Methods: Internal consistency was determined by Cronbach’s α and interrater reliability by the intraclass correlation coefficient (ICC). Convergent validity was assessed by determining the Spearman rank correlation coefficient between the movement quality measured in the SMT and functional limitation measured on the disability rating index (DRI). Results: The internal consistency was Cronbach’s α 0.91. Substantial reliability was found for the items: movement (ICC = 0.68), gait (ICC = 0.69), sitting posture (ICC = 0.63), and respiration (ICC = 0.64). Insufficient reliability was found for standing posture (ICC = 0.23). A moderate correlation was found between average test score SMT and the DRI (r = −0.37) and respiration and DRI (r = −0.45). Discussion: The SMT is a reasonably reliable tool to assess movement, gait, sitting posture, and respiration. None of the items in the domain standing posture has sufficient reliability. A thorough study of this domain should be considered. The results show little evidence for convergent validity. Several items of the SMT correlated moderately with functional limitation with the DRI. These items were global movement, hip flexion, pelvis rotation, and all respiration items.

ABSTRACTObjective: To assess the suitability for use within economic evaluation of a widely used sleep-related instrument (the Epworth Sleepiness Scale [ESS]) by examining its convergent and discriminant validity with two widely used generic preference-based instruments (Short-Form 36 [SF-36] and the Assessment of Quality of Life 4 dimensions [AQoL-4D]). Methods: Data from a cross-section of 2,236 community-dwelling Australian men were analyzed. Convergent validity was investigated using Spearman’s correlation, intraclass correlation, and modified Bland-Altman plots, while discriminant validity was examined using Kruskal Wallis tests. Results: All instruments showed good discriminant validity. The ESS was weakly correlated to the Short Form 6 dimension, or SF-6D (derived from the SF-36) and AQoL-4D utilities (r = 0.20 and r = 0.19, respectively). Correlations between ESS and SF-36/AQoL-4D dimensions measuring the same construct were all in the hypothesized directions but also weak (range of absolute r = 0.00 to 0.18). The level of agreement between the ESS and AQoL-4D was the weakest, followed by that between the ESS and SF-6D. Moderate convergent validity was seen between the utilities. Conclusions: The lack of convergent validity between the ESS and the preference-based instruments shows that sleep-related constructs are not captured by the latter. The ESS has, however, demonstrated good discriminant validity comparable to that of the AQoL-4D and the SF-36/SF-6D and would therefore be equally useful for measuring subgroup differences within economic evaluation. We therefore recommend using the ESS within cost-effectiveness analysis as a complement to preference-based instruments in order to capture sleep-specific constructs not measured by the latter.

Mentalization, emotion regulation, and identity diffusion are theoretically and clinically important transdiagnostic psychological constructs that contribute to mental health. In order to advance meaningful empirical research on these constructs, we need measures that are well tested. In this study, we used confirmatory factor analysis to assess the reliability and construct validity of the Mentalization Questionnaire (MZQ), different versions of the Difficulties in Emotion Regulation Scale (DERS), and the Assessment of Identity Development and Identity Diffusion in Adolescence (AIDA) with data from a general population of Finnish adolescents (N = 360). For MZQ, the factor structure and validity of the subscales were not confirmed. For DERS, a short version, that did not include the lack of emotional awareness subscale was the most coherent and recommendable version of the measure, with a good degree of reliability and a reasonable indication of the convergent and discriminative validity between the subscales. For AIDA, the factor structure was confirmed, but when using this measure for research purposes, it should be taken into account that reverse coding items may affect the factor structure by creating a method factor. The reliability of the AIDA was acceptable, but some of the subscales showed poor convergent and discriminative validity.

Internalized stigma related to HIV is associated with poorer outcomes for people living with HIV (PLWH). However, little is known about the association between experiences of daily acts of discrimination by others and the activation of internalized stigma, including factors that may moderate this association. One hundred nine men living with HIV responded to experience sampling method (ESM) questions 3 times a day for 7 days via smart-phones. ESM questions included experiences of recent acts of discrimination, internalized HIV stigma, avoidance coping with HIV, and recent social support. We also administered several traditional questionnaire measures assessing psychosocial constructs. In Hierarchical Linear Modeling analyses controlling for age, race, socioeconomic status, and time on antiretroviral therapy, experiencing discrimination predicted internalized stigma within-persons. Individuals higher on attachment-related avoidance, attachment-related anxiety, avoidance coping, perceived community stigma, and helplessness, and individuals lower on social support, had stronger associations between discrimination and current internalized stigma. Similarly, results from two state moderator variables supported our trait analyses: State-level (ESM) social support and avoidance coping were significant moderators. Thus, when PLWH experience incidents of discrimination due to HIV, this may lead to increased feelings of internalized stigma. We extend the literature by demonstrating that the associations between experienced and internalized stigma are not just at the generalized trait level, but also occur at the state-level, accounting for within person variability. Results provide implications for interventions aiming to modify maladaptive interpersonal traits as well as interventions to increase social support to reduce the impact of discrimination on PLWH.

Test-retest reliability and convergent validity of measures of children's travel behaviours and independent mobility

Convergent Validity and Test-Retest Reliability of Multimodal Ultrasonography and Related Clinical Measures in People With Chronic Stroke

Episodic memory testing is fundamental for the diagnosis of Alzheimer's disease (AD). Although the Free and Cued Selective Reminding Test (FCSRT) is widely used for this purpose, it may not be sensitive enough for early detection of subtle decline in preclinical AD. The Memory Binding Test (MBT) intends to overcome this limitation. To analyze the test-retest reliability of the MBT and its convergent validity with the FCRST. 36 cognitively healthy participants of the ALFA Study, aged 45 to 65, were included for the test-retest study and 69 for the convergent analysis. They were visited twice in a period of 6 ± 2 weeks. Test-retest reliability was determined by the calculation of the intra-class correlation coefficient (ICC). Score differences were studied by computing the mean percentage of score variation between visits and visualized by Bland-Altman plots. Convergent validity was determined by Pearson's correlations. ICC values in the test-retest reliability analysis of the MBT direct scores ranged from 0.64 to 0.76. Subjects showed consistent practice effects, with mean amounts of score increasing between 10% and 26%. Pearson correlation between MBT and FCSRT direct scores showed r values between 0.40 and 0.53. The FCSRT displayed ceiling effects not observed in the MBT. The MBT shows adequate test-retest reliability and overall moderate convergent validity with the FCSRT. Unlike the FCSRT, the MBT does not have ceiling effects and it may therefore be especially useful in longitudinal studies, facilitating the measurement of subtle memory performance decline and the detection of very early AD.

&lt;p&gt;The current study examined the factor structure and psychometric properties of the Chinese version of the Revised Life Orientation Test (CLOT-R) in a sample of 342 community-dwelling older Chinese immigrants in Canada (mean age = 71.99, &lt;em&gt;SD&lt;/em&gt; = 5.62; 58.5% female). Confirmatory factor analysis demonstrated that the CLOT-R yields a two-factor model with one item cross-loading on two latent constructs of optimism and pessimism. Analysis further revealed poor internal consistency and convergent validity. Evidence for discriminant and convergent validity was found between optimism and perceived stress, as well as optimism and quality of life. Compared with the factor structure reported in previous Chinese-speaking samples, the modified two-factor structure found in the current group of older Chinese immigrants could be attributed to the heterogeneity of the sample and possible configural variance across culture and age. Overall, the current findings suggest that the CLOT-R may not be a reliable and valid measure to assess dispositional optimism and pessimism among older Chinese immigrants. Theoretical implications and suggestions for further scale development and research is discussed.&lt;/p&gt; &lt;p&gt; &lt;/p&gt;

Purpose To develop and cross-culturally adapt a Brazilian-Portuguese version of the Falls Behavioral Scale (FaB-Brazil) and to verify its psychometric properties. Material and Methods The translation and cross-cultural adaptation process of the scale followed standard guidelines. The FaB-Brazil scale was applied to 93 community-dwelling older people. Cronbach’s alpha was calculated to evaluate internal consistency and the intraclass correlation coefficient (ICC) to evaluate interrater and test–retest reliability. The standard error of measurement (SEM), minimal detectable change (MDC), ceiling and floor effects, convergent and discriminative validity were evaluated. A significance level of .05 was set for statistical analyses. Results Internal consistency was moderate (α = 0.73). An excellent inter-rater (ICC = 0.93; p < 0.001) and a good test–retest (ICC = 0.79; p < 0.001) reliability were found. The SEM was 0.27 and MDC was 0.53. Neither ceiling nor floor effects were found. Convergent validity was established by the positive correlations between the FaB-Brazil scale, age, and functional mobility, and by the negative correlations between the FaB-Brazil scale and balance confidence, community mobility and EuroQol-5D (p < 0.05). No significant differences were found between males and females and between non-fallers and fallers. Conclusions Our results offer evidence for the reliability and validity of the FaB-Brazil scale for community-dwelling older people. Implications for Rehabilitation Fall-related behaviors should be part of the fall risk assessment of community-dwelling older people. The Brazilian-Portuguese version of the Falls Behavioral Scale (FaB-Brazil) is reliable and valid for assessing fall-related behaviors in community-dwelling older people. The FaB-Brazil scale may be used to raise awareness about potential fall hazards and to guide fall prevention programs.