Quantifying sentence acceptability measures: Reliability, bias, and variability

Considerations of sample size in medical research

Cosine similarity as a sample size-free measure to quantify phase clustering within a single neurophysiological signal

This study investigated the attitude structure of schizophrenic patients with particular reference to rigidity. Zax, Gardiner and Lowy (1964) have shown that schizophrenics give more extreme responses to unstructured materials and other evidence has suggested that such patients display rigid and stereoryped responses, expressed, for example, by inabiliry to shift set or adopt flexible strategies (Shakow, 1963). Such a response style is often heightened in the presence of personally relevant and anxiety arousing material. This study examined this phenomenon in relation to attitudes about relevant and irrelevant topics in a group of schizophrenics attending a day treatment center in an outpatient clinic. (1) Schizophrenics were expected to give more extreme answers than normals in filling out attitude questionnaires; (2) all Ss to give more extreme answers in atticudinal areas that are personally relevant than those which are less so; (3) diagnosis and relevance would interact, such that schizophrenics would give more extreme responses in personally relevant areas than normals. Schizophrenics (n = 40) and 40 normal veterans who attended a medical service in the same clinic and were without known history of emotional disorder filled out a 60-item attitude questionnaire. There were two personally relevant topics, mental illness and work, and two personally irrelevant topics, teenage music and men's clothing styles. Individual items were answered by checking off a six-category Likert scale ranging from strongly agree to strongly disagvee. Topic relevance was established beforehand by asking 20 schizophrenic and 20 normals similar to chose in the study to choose from 30 topics 3 most and 3 least personally relevanr. Their choices were tallied and topics were selected from the

ABSTRACTThis study addresses equating issues with varying sample sizes using the Rasch model by examining how sample size affects the stability of item calibrations and person ability estimates. A resampling design was used to create 9 sample size conditions (200, 100, 50, 45, 40, 35, 30, 25, and 20), each replicated 10 times. Items were recalibrated using each of these 90 samples. The deviation of these calibrations from the full sample (N = 9,678) calibrations were then computed. The ability estimates for all 9,678 examinees were then recomputed 90 times using the item calibrations from each of the 90 different samples. The deviation of 90 sets of ability estimates from the original set of ability estimates was computed. This study found that less precision and item calibration instability occur with smaller sample sizes; however, the decreasing sample size has minimal effect on the person ability estimates.

The uncertainty in pesticide residue levels (UPRL) associated with sampling size was estimated using individual acetamiprid and cypermethrin residue data from preharvested apple, broccoli, cabbage, grape, and sweet pepper samples. The relative standard deviation from the mean of each sampling size (n = 2(x), where x = 1-6) of randomly selected samples was defined as the UPRL for each sampling size. The estimated UPRLs, which were calculated on the basis of the regulatory sampling size recommended by the OECD Guidelines on Crop Field Trials (weights from 1 to 5 kg, and commodity unit numbers from 12 to 24), ranged from 2.1% for cypermethrin in sweet peppers to 14.6% for cypermethrin in cabbage samples. The percentages of commodity exceeding the maximum residue limits (MRLs) specified by the Japanese Food Sanitation Law may be predicted from the equation derived from this study, which was based on samples of various size ranges with mean residue levels below the MRL. The estimated UPRLs have confirmed that sufficient sampling weight and numbers are required for analysis and/or re-examination of subsamples to provide accurate values of pesticide residue levels for the enforcement of MRLs. The equation derived from the present study would aid the estimation of more accurate residue levels even from small sampling sizes.

An evaluation of the effects of sample size on estimating length composition of catches from tuna longline fisheries using computer simulations

Bayesian methods allow borrowing of historical information through prior distributions. The concept of prior effective sample size (prior ESS) facilitates quantification and communication of such prior information by equating it to a sample size. Prior information can arise from historical observations; thus, the traditional approach identifies the ESS with such a historical sample size. However, this measure is independent of newly observed data, and thus would not capture an actual "loss of information" induced by the prior in case of prior-data conflict. We build on a recent work to relate prior impact to the number of (virtual) samples from the current data model and introduce the effective current sample size (ECSS) of a prior, tailored to the application in Bayesian clinical trial designs. Special emphasis is put on robust mixture, power, and commensurate priors. We apply the approach to an adaptive design in which the number of recruited patients is adjusted depending on the effective sample size at an interim analysis. We argue that the ECSS is the appropriate measure in this case, as the aim is to save current (as opposed to historical) patients from recruitment. Furthermore, the ECSS can help overcome lack of consensus in the ESS assessment of mixture priors and can, more broadly, provide further insights into the impact of priors. An R package accompanies the paper.

Top differentially expressed gene lists are often inconsistent between studies and it has been suggested that small sample sizes contribute to lack of reproducibility and poor prediction accuracy in discriminative models. We considered sex differences (69♂, 65♀) in 134 human skeletal muscle biopsies using DNA microarray. The full dataset and subsamples (n = 10 (5♂, 5♀) to n = 120 (60♂, 60♀)) thereof were used to assess the effect of sample size on the differential expression of single genes, gene rank order and prediction accuracy. Using our full dataset (n = 134), we identified 717 differentially expressed transcripts (p<0.0001) and we were able predict sex with ∼90% accuracy, both within our dataset and on external datasets. Both p-values and rank order of top differentially expressed genes became more variable using smaller subsamples. For example, at n = 10 (5♂, 5♀), no gene was considered differentially expressed at p<0.0001 and prediction accuracy was ∼50% (no better than chance). We found that sample size clearly affects microarray analysis results; small sample sizes result in unstable gene lists and poor prediction accuracy. We anticipate this will apply to other phenotypes, in addition to sex.

BackgroundEmpathy is a key aspect of the physician-patient interactions. The Jefferson Scale of Empathy (JSE) is one of the most used empathy measures of medical students. The development of cross-cultural empathy studies depends on valid and reliable translations of the JSE. This study sought to: (1) adapt and assess the psychometric properties in Spanish students of the Spanish JSE validated in Mexican students; (2) test a second order latent factor model.MethodsThe Spanish JSE was adapted from the Spanish JSE-S, resulting in a final version of the measure. A non-probabilistic sample of 1104 medical students of two Spanish medical schools completed a socio-demographic and the Spanish JSE-S. Descriptive statistics, along with a confirmatory factor analysis, the average variance extracted (AVE), Cronbach’s alphas and composite reliability (CR) coefficients were computed. An independent samples t-test was performed to access sex differences.ResultsThe Spanish JSE-S demonstrated acceptable to good sensitivity (individual items – except for item 2 – and JSE-S total score: −2.72 < Sk < 0.35 and −0.77 < Ku < 7.85), convergent validity (AVE: between 0.28 and 0.45) and reliability (Cronbach’s alphas: between 0.62 and 0.78; CR: between 0.62 and 0.87). The confirmatory factor analysis supported the three-factor solution and the second order latent factor model.ConclusionsThe findings provide support for the sensitivity, construct validity and reliability of the adapted Spanish JSE-S with Spanish medical students. Data confirm the hypothesized second order latent factor model. This version may be useful in future research examining empathy in Spanish medical students, as well as in cross-cultural studies.Electronic supplementary materialThe online version of this article (doi:10.1186/s12909-016-0763-5) contains supplementary material, which is available to authorized users.

294 Background: Economic evaluations in oncology aim to assess the value of new therapies in the long term based on clinical trial data that often have restricted follow-up times (&lt; 5 years) and small sample sizes (&lt; 500 patients). This requires the use of extrapolation assumptions on long-term survival that go beyond the observed data. In this analysis, differences in survival extrapolation methods are tested in samples of sizes and follow-up reflecting typical clinical trials against a background of known survival in prostate cancer from a US based cancer registry. Methods: Data from the National Cancer Institute's Surveillance Epidemiology and End Results (SEER) registry on long-term survival in patients with stage IV prostate cancer were employed. The data set comprised those patients diagnosed between 1988 and 2003, with follow-up data available until 2012. Additional survival for those who received surgery (compared to those who did not), was estimated based on extrapolations using standard parametric statistical models (exponential, Weibull, log-logistic, log-normal, Gamma) fitted to the observed data. Survival analyses were run for 5 sample size scenarios (n = 27,670, 1000, 500, 200, 50) and 6 follow-up scenarios (follow-up years = 25, 20, 10, 5, 2, 1) yielding 30 combination scenarios. Performance of the methods was tested relative to the maximum follow-up, maximum sample size scenario (i.e. reference case) from the SEER registry. Results: Log-logistic and log-normal models were associated with flat tails which led to inflated survival estimations. For scenarios with smaller sizes, gamma models often did not converge. Exponential models were the most frequently reported as best model fit (in approximately 50% of scenarios). Also, gains in OS were consistent when exponential models were selected, and closely matched gain in OS from the reference case. Conclusions: Since clinical trials in oncology are often associated with small patient sample sizes and restricted follow-up, selecting an exponential model may lead to the most consistent and stable results based on the experiment constructed here. Further research should confirm these results for other types of cancer.

Liang과 Zeger는 이산형 혹은 연속형 반복측정자료를 분석하기 위한 일반화 추정방정식 (GEE)을 제안하였다 GEE모형은 범주형 반복측정자료의 모형으로 확장될 수 있으며, 이 GEE추정량은 대표본인 경우 다변량 정규분포를 따른다. 그러나 GEE는 대표본근사이론에 기초한다. 본 논문에서는 소표본인 경우 반복 측정된 순서자료에 대한 GEE추정량의 성질을 연구한다. 우리는 두가지 방법을 사용하여 두그룹의 반복 측정된 순서자료를 생성하며 모의실험을 통하여 소표본인 경우 여러 개 범주를 갖는 순서반응 자료에 대하여 GEE추정량의 1종 오류율, 검정력, 상대효율, 두 그룹의 표본크기가 다를 경우 효과, 그리고 분산 추정량의 성질등을 연구한다. Liang and Zeger proposed generalized estimating equations(GEE) for analyzing repeated data which is discrete or continuous. GEE model can be extended to model for repeated categorical data and its estimator has asymptotic multivariate normal distribution in large sample sizes. But GEE is based on large sample asymptotic theory. In this paper, we study the properties of GEE estimators for repeated ordinal data in small sample sizes. We generate ordinal repeated measurements for two groups using two methods. Through Monte Carlo simulation studies we investigate the empirical type 1 error rates, powers, relative efficiencies of the GEE estimators, the effect of unequal sample size of two groups, and the performance of variance estimators for polytomous ordinal response variables, especially in small sample sizes.

This study investigates the effects of sample size and sample prevalence on cellular automata (CA) simulation of urban growth. We take the CA models based on an artificial neural network (ANN), logistic regression (LR), and support vector machine (SVM) as examples, to simulate the urban growth of Wuhan city in China and the Wuhan Metropolitan Area under different sampling schemes. The results of the CA models based on the ANN, LR, and SVM methods are generally consistent. The sampling scheme with a small sample size and a low sample prevalence should be discarded because of the high uncertainty. Sample size determines the robustness of a CA model, whereas sample prevalence affects the performance of a CA model when there are sufficient samples. In particular, the closer the sample prevalence is to the population prevalence, the higher the simulation accuracy and the lower the shape complexity and fragmentation of the simulated urban patterns. We suggest that the optimal sampling scheme has a sample rate of 1% and a sample prevalence that is the same as the population prevalence. The selection of the optimal sampling scheme is independent of the population sizes represented by different study areas.

The success of a long-term breeding program for outcrossing tree species depends not only on genetic gains through selection but also on the maintenance of genetic diversity over consecutive generations of plant breeding. To inform future breeding and deployment of Melaleuca alternifolia, this study benchmarked genetic variation in 114 selected individuals from a third-generation breeding population with 18 microsatellite markers. Five loci were used to compare diversity measures in the breeding population to values of genetic variation for natural populations in Australia. The expected heterozygosity in the breeding population was high (HE = 0.823) and comparable to the average heterozygosity for 427 individuals from 36 natural populations (HE = 0.844). The inbreeding coefficient FIS was 0.044 and 0.151 in breeding and natural populations, respectively. These favourable findings confirmed that objectives of past breeding practices were successful in maintaining diversity and preventing inbreeding. Sample size effects were evident, nonetheless, and the average number of alleles per locus (NA) was markedly reduced, especially those of low frequency. To study the impact deploying clones would have on genetic diversity, simulated populations of clones were created by repeatedly sampling the breeding population for subsets of 10, 15 or 20 individuals and generating diversity parameters for all 18 loci. Results for each number of individuals were compared to the diversity in the entire breeding population. Sample size had no effect on HE, but NA was significantly and progressively reduced with smaller sample size.

This study is motivated by the scarcity of research examining the consumers of halal-certified hotel restaurants. Lombok Island is the best halal tourism destination in Indonesia, making research on the island particularly intriguing. The objective of this study is to empirically investigate the influence of personal factors, religiosity, and social perception on the decision to visit halal-certified hotel restaurants on Lombok Island. This research employs multiple regression analysis with a sample size of 154 respondents, using a convenience sampling technique. The measurement scale utilized is a Likert scale (1-5), ranging from strongly disagree to strongly agree. The findings indicate that personal factors, religiosity, and social perception have a positive and significant effect on the decision to visit halal-certified hotel restaurants on Lombok Island. This study is limited by a small sample size; therefore, we recommend further research with a larger number of respondents and a more diverse range of regions.

Rationale: Training in the diagnosis and management of pulmonary hypertension (PH) is a core requirement of Pulmonary and Critical Care (PCCM) Fellowships. There is currently no recognized curriculum to ensure thorough and standardized PH-related learning during subspecialty training. To meet this need, our PH faculty created a novel curriculum as well as an assessment tool to evaluate the curriculum's impact on knowledge and confidence in the care of PH patients. Methods: Our Pulmonary Hypertension Association accredited Comprehensive Care Center developed a two-week curriculum focusing on PH classification, diagnosis, risk assessment, and management. The curriculum includes inpatient PH consultation, outpatient subspecialty clinic, right heart catheterizations, and asynchronous video didactics. The video didactics provide curriculum consistency independent of faculty or service census. This curriculum was successfully piloted with first year PCCM fellows (n=5), and continued with the subsequent class (n=4). Our evaluation instrument underwent rigorous protocols for survey design before being distributed. The final tool consisted of twenty-one single best answer multiple choice questions assessing PH clinical knowledge and eighteen 5-point Likert scale questions assessing confidence in the application of that knowledge to patient care. The post-curriculum survey included additional satisfaction questions aimed at curriculum improvement and was repeated at 6 months to assess retention. Given the small sample size (n=9), Wilcoxon Signed-Rank test and descriptive statistics were used to compare the pre- and post-tests. Results: All nine participants showed improvement in knowledge of PH after completing the curriculum (p = 0.004454). Pre-test knowledge scores ranged from 29% to 62% (median: 52%) and post-test knowledge scores ranged from 67% to 100% (median: 90%). In the first cohort (n=5), knowledge was retained at the 6 month evaluation. Confidence was assessed utilizing a Likert Scale of 0 (not confident) to 4 (extremely confident). The pre-test aggregate median of 0.97 (between not confident and slightly confident) improved to an aggregate median of 2.8 (quite confident) on the post-test. The fellows’ curriculum satisfaction responses averaged between 3.5 – 4.0 (extremely satisfied). Conclusions: This innovative PH curriculum confers a trend toward increased knowledge and confidence in the classification, diagnosis, risk assessment, and management of PH. The video didactic component ensures robust, consistent, sustainable, and transferable review of PH topics. With further increases in sample size via subsequent fellowship classes and expansion to Cardiology and outside institutions’ programs, we seek to confirm that this curriculum improves knowledge and confidence during subspecialty training.