Inter- and intra-rater reliability of handpump functionality field tests

Background:The inter-rater reliability of the craniocervical flexion test (CCFT) has not been established.Objective:To investigate the intra-rater and inter-rater reliabilities of the CCFT in asymptomatic subjects.Methods:Sixty asymptomatic subjects were randomly selected for the study. The CCFT was measured on each subject by two testers for inter-rater reliability and by one of the testers after a gap of seven days for the intra-rater reliability. Before testing, the participants were trained for the movement and compensations were corrected.Results:The CCFT has high inter-rater reliability (intra-class correlation coefficient 0.907, standard error of mean 0.735) and high intra-rater reliability (intra-class correlation coefficient 0.986, standard error of mean 0.287). A Bland & Altman limits of agreement analysis has confirmed the high inter- and intra-rater reliabilities of the test.Conclusion:The CCFT has high inter-rater and intra-rater reliabilities in asymptomatic subjects.

Anatomists and radiologists use the Zaidi–Dayal and Richards–Jabbour scales to study the shape of the foramen magnum. Our aim is to measure the interrater and intrarater agreement and reliability of ratings made using the two scales. We invited 16 radiology residents to attend two sessions, four weeks apart. During each session, we asked the residents to classify the shape of the foramen magnum in 35 images using both scales. We used Fleiss’ κ to measure interrater reliability and Cohen’s κ to measure intrarater reliability. The interrater reliability of ratings made using the Zaidi–Dayal scale was 0.34 (0.26–0.46) for session one and 0.30 (0.24–0.39) for session two, and the intrarater reliability was 0.39 (0.34–0.44). The interrater reliability of ratings made using the Richards–Jabbour scale was 0.14 (0.10–0.19) for session one and 0.12 (0.09–0.17) for session two, and the intrarater reliability was 0.11 (0.07–0.15). In conclusion, the interrater and intrarater agreement and reliability of ratings made using the Zaidi–Dayal and Richards–Jabbour scales are inadequate. We recommend an objective method by Zdilla et al. to researchers interested in studying the shape of the foramen magnum.

Muscle weakness frequently impairs the ability to maintain upright sitting in individuals with spinal cord injury (SCI). The primary purpose of this study was to examine the intrarater and interrater reliability of hand-held dynamometry to assess postural muscle strength for maintaining upright sitting in individuals with SCI. We also assessed reliability of forces measured in four directions of force application and of measures obtained by experienced versus student physical therapist examiners. Twenty-nine individuals with SCI (mean age, 32.4 +/- 11.0 years; injury level C4-L1; American Spinal Injury Association Impairment Scale (AIS) classification A-D) participated in this study. The raters were two experienced physical therapists and two student physical therapists. Force was applied to the anterior, posterior, and right and left lateral trunk. Values were acquired in a group of participants who did not require upper extremity support for sitting (n = 22) and a group who did require upper extremity support (n = 7). Intrarater reliability was good to excellent (intraclass correlation coefficients, 0.80-0.98 [unsupported]; 0.79-0.99 [supported]) for all raters in the four directions of force application. Interrater reliability was excellent (intraclass correlation coefficients, 0.97-0.99 [unsupported]; 0.96-0.98 [supported]) for all directions. There were no significant differences among peak forces obtained among the four directions of force application or by experienced raters compared with student raters. The use of hand-held dynamometry to assess postural muscle strength for maintaining upright sitting in individuals with SCI has high intrarater and interrater reliability. The direction of force application and experience of the rater did not influence the level of reliability. Future research is needed to identify the minimum muscle strength required to maintain the seated posture and to understand how this measure relates to seated postural control and balance.

Original Article Purpose: Brain surface intensity model (BSIM)-based cortical thickness analysis does not require complicated 3D segmentation of brain gray/white matters. Instead, this technique uses the local intensity profile to compute cortical thickness. The aim of the present study was to evaluate intra-rater and inter-rater reliability of BSIMbased cortical thickness analysis using images from elderly participants. Materials and Methods: Fifteen healthy elderly participants (ages, 55-84 years) were included in this study. High-resolution 3D T1-spoiled gradient recalled-echo (SPGR) images were obtained using 3T MRI. BSIM-based processing steps included an inhomogeneity correction, intensity normalization, skull stripping, atlas registration, extraction of intensity profiles, and calculation of cortical thickness. Processing steps were automatic, with the exception of semiautomatic skull stripping. Individual cortical thicknesses were compared to a database indicating mean cortical thickness of healthy adults, in order to produce Z-score thinning maps. Intra-class correlation coefficients (ICCs) were calculated in order to evaluate inter-rater and intra-rater reliabilities. Results: ICCs for intra-rater reliability were excellent, ranging from 0.751-0.940 in brain regions except the right occipital, left anterior cingulate, and left and right cerebellum (ICCs = 0.65-0.741). Although ICCs for inter-rater reliability were fair to excellent in most regions, poor inter-rater correlations were observed for the cingulate and occipital regions. Processing time, including manual skull stripping, was 17.07 ± 3.43 min. Z-score maps for all participants indicated that cortical thicknesses were not significantly different from those in the comparison databases of healthy adults. Conclusion: BSIM-based cortical thickness measurements provide acceptable intrarater and inter-rater reliability. We therefore suggest BSIM-based cortical thickness analysis as an adjunct clinical tool to detect cortical atrophy.

Muscle length is a common component of the physical therapy examination, which may include the prone knee flexion (Ely) and active knee extension (AKE) tests. Clinicians using these tests should understand the clinimetric properties. To investigate the reliability and minimal detectable change (MDC95) of the Ely and AKE tests. Reliability analysis. Seventy-one asymptomatic adults (mean age 24.6 + /- 2.8 years) were recruited based on a convenience sample. Two examiners each performed the Ely and AKE test one time each in an intrasession design for the interrater reliability component, with one examiner repeating the tests one time 48 hours later to determine the intra-rater reliability. Results were recorded based on one trial per test and utilized a pelvic strap for the Ely test and an adjustable bolster for the AKE test. A separate researcher recorded measurements and results were blinded from the examiners. The Ely test had excellent intra-rater and inter-rater reliability with an intraclass correlation coefficient (ICC) (3,1) of 0.900 and ICC (2,1) of 0.914 respectively. The intra-rater and inter-rater reliability of the AKE test was good with ICC (3,1) of 0.882 and ICC (2,1) 0.886 respectively. The MDC95 indicated that a change greater than or equal to 8° and 12° is required to exceed the threshold of error for the Ely and AKE test respectively. The Ely and AKE tests have good to excellent inter-rater and intra-rater reliability for measuring rectus femoris and hamstring muscle length when stabilization of the pelvis and hip is accounted for. The MDC should be considered as a threshold for true change in the asymptomatic adult population. 2b.

Aims This paper examines: the intra-rater, inter-rater and test-retest reliability of the Five-Times-Sit-To-Stand (FTSTS) test administered by experienced and inexperienced assessors; and the correlation of FTSTS test times with lower limb muscle strength and balance performance in healthy older and young people. Method The cross-sectional study investigated reliability using test batteries including: FTSTS times; hand-held dynamometer measurements of hip flexor, knee flexor and extensor, and ankle dorsiflexor and plantarflexor muscle strength; Berg Balance Scale (BBS) scores; and dynamic posturography measurements of the limits of stability (LOS). A convenience sample of 12 healthy older and 12 younger participants was recruited in the local university-based rehabilitation centre. Results The FTSTS test showed excellent intra-rater reliability (intraclass correlation coefficient (ICC) range: 0.914–0.933) and moderate intra-rater reliability (ICC range: 0.644–0.879) in healthy older and young subject groups respectively. Inter-rater reliability between assessors yielded an ICC of 0.990. Test-retest reliability (ICC range: 0.988–0.995) was excellent. FTSTS times were not significantly associated with the older subjects' lower limb muscle strength, BBS scores or LOS results. Conclusion The FTSTS has good intra-rater, inter-rater and test-retest reliability. There is no significant correlation of FTSTS times with lower limb muscle strength or balance performance in healthy older people. The FTSTS test is a reliable measurement tool whether used by experienced or inexperienced assessors. A future study with a larger sample size is required for establishing the validity of FTSTS test times in the older population.

Background: Generalised joint hypermobility (GJH) is a common connective tissue disorder associated with a range of musculoskeletal complaints. An effective screening tool to assess GJH may influence our understanding and choice of management. Diagnosis is clinical, using tools such as the Beighton Hypermobility Score and the Contompasis Scoring System. The comparable reliability of these tools has not been previously reported. The aim of the present study was to compare the intra- and inter-rater reliability of the Beighton Score to the Contompasis Score to assess GJH. Methods: This was a randomised observational study assessing 36 pain-free participants: 27 females and nine males, with a mean age of 22.7 years ± 3.71 (range 18 to 32 years). Participants were assessed by two researchers over two sessions to allow for intra- and inter-rater analysis. Intraclass correlation coefficient (ICC) and weighted Kappa statistics were used to calculate the level of agreement. Results: The intra- (ICC=0.71–0.82) and inter-rater (ICC=0.72–0.80) reliability of the Beighton Score is substantial to almost perfect. The Contompasis Score displayed substantial to almost perfect intra-rater (ICC=0.73–0.82) reliability and moderate to substantial inter-rater (ICC=0.58–0.62) reliability. Conclusions: It is hypothesised that in clinical practice the Beighton Score is more appropriate due to greater inter-rater reliability, whereas in a research setting with experienced practitioners, the Contompasis Score would be more appropriate, due to almost perfect intra-rater reliability. Implications: The present study provides an indication of the measurement capabilities of the Beighton and Contompasis Scores, as well as providing a platform for further research.

PurposeThis study aimed to use three statistical methods to investigate the inter-rater and intra-rater reliability of the Chinese version of the Short Orientation-Memory-Concentration Test (C-SOMC) for people who have had a stroke.MethodsForty-four participants (31 men and 13 women) with a mean age of 59.05 ± 10.79 years who have had experienced a single episode of stroke were enrolled in this study. To determine the inter-rater reliability of the C-SOMC, the test was administered to each participant on the same day by two raters (A and B) who each had more than 1 year of work experience. To determine intra-rater reliability, rater B re-evaluated 36 of the 44 participants with the C-SOMC on the subsequent day. Intraclass correlation coefficients (ICCs), paired-samples t-tests, and Bland-Altman plots were used to analyze the inter-rater and intra-rater reliability.ResultsThe evaluation of inter-rater reliability for the total score and item 1, 4, 5, 6, and 3/7 showed respective ICCs of 0.959, 0.918, 1.000, 0.942, 0.905 and 0.913, indicating excellent inter-rater reliability for the C-SOMC. Item 2 had an ICC of 0.796, indicating moderate to good inter-rater reliability. The evaluation of intra-rater reliability showed an ICC of 0.978 for the total score, and respective ICCs of 1.000, 1.000, 1.000, 0.968, 0.973 and 0.929 for the individual items, indicating excellent intra-rater reliability for the C-SOMC. The paired-samples t-test for the C-SOMC showed no statistically significant differences (P > 0.05) between ratings made by two different raters or by the same rater on separate occasions. The minimal detectable change value at the 95% confidence threshold (MDC95) of the SOMC score was found to be 2.14. Bland-Altman plots showed a mean difference of 0.02 and 95% limits of agreement (95% LOA) ranging from −3.86 to 3.90 for the inter-rater measurements and a mean difference of 0.33 and 95% LOA of −2.71 to 3.37 for the intra-rater measurements.ConclusionsThe C-SOMC demonstrated excellent inter-rater and intra-rater reliability when administered to people who have had a stroke. The C-SOMC may be used to screen for cognitive impairment in people who have had a stroke.

The inter-rater and intra-rater reliability of passive physiological accessory movement assessment of lumbar spine in novice manual therapists

BackgroundOsteoarthritis (OA) of the hip affects the entire joint structure and function and leads joint capsular changes which result limitation in range of motion (ROM). Therefore, measuring ROM is an...

Intrarater and Inter-rater Reliability of Maximal Voluntary Neck Muscle Strength Assessment Using a Handheld Dynamometer in Women With Headache and Healthy Women

To establish the utility and feasibility of the International Spinal Cord Injury/Dysfunction (SCI/D) Bowel Function Basic Data Set Version 2.0 in pediatric SCI populations. This was a noninterventional, repeated measure design conducted in Pennsylvania, Maryland, Illinois, Kentucky, and South Carolina. The International Spinal Cord Injury/Dysfunction (SCI/D) Bowel Function Basic Data Set Version 2.0 was administered repeatedly, twice at the point of care and once over the phone. Time to complete the data set was recorded. Inter- and intrarater reliability was examined by intraclass correlation coefficients (ICC) with 95% confidence intervals (CI), and agreement between the bowel function basic data set variables and medical records was calculated using percentages. Intrarater reliability involved the same person administering the data once at the point of care and once over the phone. Forty-one children/youth ages 1 to 20 years participated in this study. Average time to complete the data set was 5.17 minutes. Interrater reliability was good to excellent (ICC ≥ 0.75) for most variables. Five variables had moderate interrater reliability (ICC = 0.05-0.74) and three had poor interrater reliability (ICC < 0.05). With the exception of one variable that had poor intrarater reliability (constipating agent, ICC = 0.00) and one that approached moderate reliability (digital evacuation, ICC = 0.74), intrarater reliability was good to strong for every bowel variable (ICC = 0.88-1.00). Only 12 (32%) medical records had explicit documentation of one or more of the variables on the Basic Bowel Function Basic Data Set V2.0. The results support future research with a larger and more diverse sample of children with SCI to build upon the psychometric work described herein.

Objective: This study aimed to determine the intra-rater and inter-rater reliability and validity of a hybrid device, combining virtual reality goggles, a magnetometer and an inclinometer application for smartphones, to measure craniocervical range. Summary of Background Data: Accurate evaluation of craniocervical range of motion is important for early detection of certain diseased conditions and monitoring the progress of interventions. The universal goniometer is widely used for the measurement but it requires experienced practitioners. Whether a combination of virtual reality goggles and smartphone applications can provide the same or better performance compared with the goniometer is still unknown. Methods: Forty-one healthy adults from the department of physical medicine and rehabilitation were recruited for craniocervical range examination (flexion, extension, side-bending to the right or left and rotating to the right or left) by using the hybrid device and universal goniometer. Using the hybrid device, repeated measurements were performed twice by a primary rater and once by a second rater. The primary rater also conducted a measurement using the universal goniometer in the same cohort. The intra-rater and inter-rater reliability (intra-class correlation coefficient (ICC)) were calculated using the two-way random effect model, whereas the validity was examined by the Pearson correlation coefficient and Bland-and-Altman plot. The interval between the first and second sessions of the measurement for intra-rater reliability was set at 30 min. Results: Excellent intra-rater (ICC ≥ 0.925) and inter-rater (ICC ≥ 0.880) reliability was noted for the hybrid device. The minimal detectable changes from intra-observer and inter-observer comparisons ranged between 4.12° and 7.42° in all six directions. The Bland-and-Altman plot revealed small mean differences (≤1.68°) between the hybrid device and universal goniometer. Both instruments had highly correlated measurements of craniocervical motion (r values ≥ 0.918). Conclusion: For healthy participants, excellent intra-rater and inter-rater reliability was noted for the hybrid device, and the measurements were consistent with the universal goniometer measurements. Future studies are needed to examine whether the device can perform similarly for patients with neck disorders.

Objective: To investigate the intra – rater and inter – rater reliability of cranio cervical flexion test in symptomatic subjects. Background of study: Neck flexor muscles endurance was negatively correlated with cervical pain and dysfunction. Cranio cervical flexion test protocol has been advocated to train cranio cervical flexor muscle performance at present there is no consensus to the most effective method. The CCFT is a clinical test of neuromotor control of deep flexor of the cervical spine. Methodology: This was an observational study design with test-retest type and conducted in the ACS medical college and hospital. 50 samples were selected based on inclusion criteria. The CCFT was measured by one tester twice for intra rater reliability and two testers for inter-rater reliability were again tested after 24 Hours (Next day) by keeping time and environment optimal. In this CCFT, the activation pressure score and highest-pressure score was measured using Pressure Biofeedback unit. Result: The mean score for Activation Pressure by intra-rater tester one was found to be 2.24 (1.69) and 2.64 (1.48) in retest. The ICC intrarater score for Highest pressure value by tester one was 23.92 (2.17) and 24.32 (1.99) in retest. The mean score for activation pressure by Inter – rater examiners was found to be 2.92(1.15) by examiner - 1 and 2.72(1.19) by examiner - 2. The ICC Inter – rater score for highest pressure value by examiner - 1 was 24.32(1.95) and 24.4(2.33) by examiner - 2. The ICC value of the study indicated high reliability of the activation pressure and highest-pressure value among the intra and inter-raters. Conclusion: The CCFT has high inter-rater and intra-rater reliabilities in symptomatic subjects. Bangladesh Journal of Medical Science Vol. 22 No. 03 July’23 Page : 617-622

This study evaluated the reliability of a standardized grading system for scoring the overall severity of cellulite on the posterior thigh. The study evaluated inter-rater and intra-rater (test/re-test) reliability of the method and engaged in an iterative process to develop a reliable method to evaluate changes in the appearance of cellulite. There were two stages in the validation process. The first stage was an open process without evaluator training. The second stage was a more controlled process with training given and moderator involvement to review grade selections. In the first stage, inter-rater reliability was examined across five evaluators who were asked to evaluate 24 photographs (right thighs) based on a cellulite graded severity chart. During the second stage, the same photographs were examined by paired evaluators who had received additional training. Scores were independently moderated by a third person. The inter-rater reliability and intra-rater reliability over a 4-week interval were evaluated using intraclass correlation coefficients (ICCs). Twenty-four female participants (18-51 years, mean 31.68 ± 9.03 years) with a mean BMI of 29.04 ± 6.52 participated in the trial. Five female evaluators completed the initial evaluations. In stage 1, the inter-rater reliability (ICC2,5 ) was 0.838 (95%CI:0.700-0.922) and test/retest ICC3,1 values ranged from 0.360-0.990. In stage 2, the inter-rater reliability for 2 evaluators improved to 0.978 (95%CI:0.948-0.991), and the test/retest reliability of the moderated scoring method improved to 0.993 (95%CI:0.983-0.997). The iterative process developed a simple and reliable method of rating cellulite severity, with excellent inter-and intra-rater reliability, based on evaluating images of cellulite against a standard set of graded severity images. A reliable method of assessing cellulite severity is essential for undertaking future clinical trials to evaluate cellulite treatments.