Rate it again: Using the wisdom of many to improve performance evaluations

ABSTRACTThis article reports on a study of foreign students’ success in higher education in Norway and focuses specifically on the relation between academic success and students’ proficiency level of Norwegian as measured by a CEFR-based university entrance test. This study is distinguished from prior studies because it sets out to investigate not only the relation between entrance test score and future academic success but also what level of target language proficiency is a necessary minimum requirement for foreign students to manage the linguistic demands they face in higher education. It includes students who passed the entrance test in question as well as students who failed, hence overcoming a common limitation of predictive validity studies. The results of the study show a significant positive correlation between scores on the university entrance test and self-reported academic success. Moreover, the study shows that students with proficiency levels below B2 seem to lack the necessary language skills to master their studies, regardless of discipline and faculty. The results are of relevance to other countries and educational contexts where the CEFR scale is used as a reference for university entrance tests. Together with similar studies of the predictive validity of CEFR-based entrance tests, the study contributes to an ongoing validation of the CEFR, which has been called for since its publication.

The majority of cultivated strawberries (Fragaria × ananassa) in the northern United States (US) and Canadian provinces is grown in perennial matted rows across a range of soil types and microclimates. Management practices vary in fertilization rates, intensity of pesticide use, and the source of inputs depending on grower preferences. The objective of this study was to identify environmental and management factors that influence strawberry flavor attributes across a range of production systems. The cultivar Jewel was selected for its popularity in this region and reputation for excellent flavor. “Jewel” was sampled from regional farms and, concurrently, grown in a controlled field study with different inputs over three years. Soluble solids content (SSC) and titratable acidity (TA) across farms was found to be positively associated with the air temperature differential during fruit ripening. In controlled field studies, yield was correlated positively with total N in the form of synthetic urea, but not with the rate of applied organic nitrogen (N). Despite different levels of soil carbon inputs, N rates, pesticides, and microbial supplements, the fruit quality attributes, including SSC, TA, aromatic volatile concentration, and phenolics were not associated with treatment. A human sensory evaluation found no perceptible differences in flavor or aroma among contrasting treatments. Our study concludes that growers should invest in temperature management, rather than agricultural inputs, to influence SSC and TA of strawberry.

Pancreas disease (PD) caused by salmonid alphavirus (SAV) continues to negatively impact salmon farming. To assess the effect on growth and mortality of three vaccines against PD, two controlled field designs were employed: one controlled field study with individual marked fish (PIT tag) assessing three PD vaccines and three controls groups, and a second controlled field study with group marked fish (Maxilla) comparing two PD vaccines against controls. In addition, a descriptive study using whole cages compared fish immunized with two different PD vaccines against controls.The target populations experienced a natural PD outbreak where both SAV 2 and SAV 3 were identified. Only one of the PD vaccines provided statistically significant improvements in harvest weight of 0.43 kg (CI: 0.29–0.57) and 0.51 kg (CI: 0.36–0.65) compared with the control in the PIT tag and the Maxilla study, respectively. In the latter, a significant reduction in mortality of 1.31 (CI:0.8–1.8) per cent points was registered for the same vaccine compared with controls. These results aligned with the growth and PD‐specific mortality registered in the descriptive Cage study. The data in this study show a difference in the efficacy of PD vaccines in farmed Atlantic salmon.

Human-robot collaboration will increasingly take place in human social settings, including contexts where ethical and honest behavior is paramount. How might these robots affect human honesty? In this paper, we present first evidence of how a robot's presence affects people's ethical behavior in a controlled field study. We observed people passing by a food plate marked as "reserved", comparing three conditions: no observer, a human observer, and a robot observer. We found that a human observer elicits less attention than a robot, but evokes more of a socially normative presence causing people to act honestly. Conversely, we found that a robot observer elicits more attention, engagement, and a monitoring presence. But even though people were suspicious that they were being monitored, they still behaved dishonestly in the robot observer condition.

A comprehensive examination is an academic exercise that aims to determine students’performance relevant to their achievements in the previous years of their studies. In the oldcurriculum of the College of Engineering at Camarines Sur Polytechnic Colleges, Philippines,a comprehensive examination is given to second-year students of all engineering programs.With the change in curriculum due to the K-12 implementation, this comprehensive examinationis being planned to be given to first-year engineering students. This study aims to explorethe influence of students’ academic preparation and academic performance on the comprehensiveexamination of first-year engineering students. The researcher analyzed the datafrom 468 first-year engineering students for the academic year 2018–2019 to determine thedegree of correlation between their academic preparation, academic performance, and comprehensiveexamination. Findings showed that senior high school (SHS) tracks have no relationshipwith the academic performance of civil, electronics, and mechanical engineeringstudents. However, for electrical engineering students, there is a relationship between theSHS tracks and their academic performance. Also, the college entrance test (CET) and academicperformance showed a relationship with the comprehensive examination. Findingsrevealed that higher ratings in the CET and academic performance tend to go with higherratings in the comprehensive examination and vice versa. Thus, a comprehensive examinationis an effective way to determine the extent of achievement of students in engineeringmathematics and sciences. This study did not consider any other factors that may affect theresult of the comprehensive examination.

The concept of differential validity suggests that cognitive ability tests are associated with varying levels of validity across ethnic groups, such that validity is lower in certain ethnic subgroups than in others. A recent meta-analysis has revived the viability of this concept. Unfortunately, data were not available in this meta-analysis to correct for range restriction within ethnic groups. We reviewed the differential validity literature and conducted 4 studies. In Study 1, we empirically demonstrated that using a cognitive ability test with a common cutoff decreases variance in test scores of Black subgroup samples more than in White samples. In Study 2, we developed a simulation that examined the effects of range restriction on estimates of differential validity. Results demonstrated that different levels of range restriction for subgroups can explain the apparent observed differential validity results in employment and educational settings (but not military settings) when no differential validity exists in the population. In Study 3, we conducted a simulation in which we examined how one corrects for range restriction affects the accuracy of these corrections. Results suggest that the correction approach using a common range restriction ratio for various subgroups may create or perpetuate the illusion of differential validity and that corrections are most accurate when done within each subgroup. Finally, in Study 4, we conducted a simulation in which we assumed differential validity in the population. We found that range restriction artificially increased the size of observed differential validity estimates when the validity of cognitive ability tests was assumed to be higher among Whites. Overall, we suggest that the concept of differential validity may be largely artifactual and current data are not definitive enough to suggest such effects exist.

Microbial-root associations are important to help plants cope with abiotic and biotic stressors. Managing these interactions offers an opportunity for improving the efficiency and sustainability of agricultural production. By characterizing the bacterial and archaeal community (via 16S rRNA sequencing) associated with bulk and rhizosphere soil of sixteen strawberry cultivars in two controlled field studies, we explored the relationships between the soil microbiome and plant resistance to two soil-borne fungal pathogens (Verticillium dahliae and Macrophomina phaseolina). Overall, the plants had a distinctive and genotype-dependent rhizosphere microbiome with higher abundances of known beneficial bacteria such as Pseudomonads and Rhizobium. The rhizosphere microbiome played a significant role in the resistance to the two soil-borne pathogens as shown by the differences in microbiome between high and low resistance cultivars. Resistant cultivars were characterized by higher abundances of known biocontrol microorganisms including actinobacteria (Arthrobacter, Nocardioides and Gaiella) and unclassified acidobacteria (Gp6, Gp16 and Gp4), in both pathogen trials. Additionally, cultivars that were resistant to V. dahliae had higher rhizosphere abundances of Burkholderia and cultivars resistant to M. phaseolina had higher abundances of Pseudomonas. The mechanisms involved in these beneficial plant-microbial interactions and their plasticity in different environments should be studied further for the design of low-input disease management strategies.

Inference control is a modern topic in data usage management, especially in the context of data-centric business models. However, it is generally not well understood how protection mechanisms could be designed to protect the users. The contributions of this paper are threefold: firstly, it describes the inference problem and relate it to protection mechanisms; secondly, it reports on a simple mechanism to provide a-priori inference protection; thirdly, it discusses on the drawbacks of such a mechanism, as well as on the acceptance it had on a preliminary, controlled field study. In particular, the study shows that, contrary to our expectations, participants prefer an a-posteriori approach based upon audits to detect whether inferences happened.

In a controlled field study, twelve office employees with computer-intensive jobs were monitored during the redesign of their work environment. Before office redesign, they worked in closed offices with four walls and sitting height, non-adjustable workstations. Then they worked in more open offices with three walls and sit-stand adjustable VDT workstations. The effects of this office redesign were evaluated three months post-occupancy. During the three months, employees worked standing for two hours every day. The results suggest that change in the office layout, i.e. open versus closed, increased the interaction and communication between employees. However, it significantly decreased employees' perceived privacy, and increased the amount of visual and noise distractions. In the offices with sit-stand adjustable furniture, subjects felt more energetic and less tired by the end of the workday.

The rise of social media has sparked discussions about potential risks to health professionals’ reputations and employment. However, its impact on professional credibility and the health professional-client relationship remains underexplored. This study examines how personal disclosure on health professionals’ Instagram profiles influences their credibility and affects client perceptions across cultural contexts. Using a controlled field study, participants from Kuwait and the US reviewed randomly assigned mock Instagram profiles of health professionals, varying by gender, intensity of social media use, and level of personal disclosure. Participants rated credibility on a visual analog scale, with analyses testing its effect on their willingness to become clients. Results indicate that personal disclosure significantly influences credibility but in culturally distinct ways. In Kuwait, profiles with higher personal disclosure increased perceived credibility, while in the US, they lowered it. Overall, greater personal disclosure negatively impacted willingness to be a client, with credibility mediating this relationship. US participants were less likely to seek services from professionals they deemed less credible, while Kuwaiti participants showed the opposite trend. These findings highlight the role of cultural context in shaping digital reputation management, providing critical insights for healthcare professionals navigating social media use.

General vaccination with a combined measles, mumps, and rubella vaccine was introduced in Sweden in 1982. The immunisation schedule comprises two vaccine injections, given at 18 months and 12 years of age, respectively. A controlled field study was carried out in 150 children aged 18 months using two different batches of the vaccine. Seroconversion was seen in 96% against measles, 93% against mumps, and 99% against rubella--the same rates with both vaccine lots. Nevertheless, a difference was noted between the two batches with respect to postvaccination reactions. Fever and rash were recorded mainly five to 12 days after vaccination. Moderate fever (38.5-39.4 degrees C) was observed in 22 children, high fever (greater than or equal to 39.5 degrees C) in 33, and rash in 35. Preliminary results obtained by follow up of routinely vaccinated schoolchildren aged 12 indicated considerably lower rates of fever and rash during the postvaccination period, occurring in 3-10% of cases only. These findings show that complete eradication of measles, mumps, and rubella in Sweden is entirely practicable by the mass vaccination programme and that side effects of vaccination are likely to be few and mild.

One major intrinsic factor affecting recruitment of young into a population is infanticide, the killing of conspecific young by adult males. It occurs in most mammal species, like our study species the bank vole (Myodes glareolus), and is widely accepted as an adaptive behavior, which may increase male fitness via nutritional gain, decreased competition, or an increased access to mates. A turnover of males in a population increases the risk of infanticide owing to a disruption of social structures. In a controlled field study, we tested the effects of total male turnover and density on juvenile recruitment and female space use in experimental bank vole populations. Juvenile recruitment declined significantly in low‐density populations with male turnover, while growth of high‐density populations seemed unaffected. The females’ spatial behavior seemed unaffected by both male turnover and density. Our results suggest that male infanticide may considerably affect population growth and it is more likely to do so at lower densities.

There are many questions that must be raised, problems that must be resolved, and consequences that must be traced as results of the claims that William James makes in his “The Will to Believe.” I have discussed some of those questions and problems in Chapter IV; therefore, I will simply draw out here some of the presuppositions and consequences of James’s introduction of psychology into religion. The dispute between James and William Clifford serves to illustrate the fundamental epistemological difference, as it is usually drawn, that underlies the difference between theism and naturalism. Clifford and naturalists generally think that we are limited in terms of what can justify any belief in terms of adequate evidence based upon our sense experience of natural phenomena. Such evidence, naturalists claim, limits our beliefs to the natural world and the natural phenomena in that world. The restriction that naturalists place upon what is considered to be adequate evidence for beliefs leads to a naturalism that will not allow justification of beliefs that extend beyond the natural world. The evidence must be empirical evidence of natural phenomena, and the scope of our theories is determined by the limits of sense experience. Although he regarded himself as a strict empiricist, James clearly held that beliefs based upon and limited simply to natural phenomena do not exhaust all there is to the world or all that we are justified in claiming to know about the world. According to James, “tough-minded” naturalism only goes so far, and then we must admit that there is more to the world than simply that for which our sense experience provides evidence. Alternatively, one can understand James as broadening the field of naturalistic data by the introduction of psychological data, and one should keep in mind that James’s understanding of psychology was not nearly so empirical or scientific as is the contemporary understanding. In The Varieties of Religious Experience, James replies upon an “armchair” style of doing what is now called “humanistic psychology,” which would hardly pass as rigorous social science today. His “data” concerning religion, which he subjects to philosophical and psychological analysis, are the result of literary accounts, biographical reports, or second-hand information. It was James’s student, Edwin Starbuck, who was to become known for introducing more empirical and more scientific methods into the study of religious experience and religious phenomena that rely upon the now commonplace, controlled field studies. James, however, opened the door for the study of the psychology of religion and provided the clear juxtaposition of naturalism and theism for the social sciences.

Gastrointestinal (GI) complaints are frequently experienced during running. Sports drinks to prevent dehydration and hypoglycemia during exercise are generally used. The aim was to investigate the effect of 3 different drinks on GI complaints and performance during competitive running in a controlled field study. Ninety-eight well-trained subjects (90 M, 8 F, age 41 +/- 8 y) performed a competitive 18-km run three times within 8 days. The study was a controlled, standardized field experiment following a randomized, crossover design. Three different drinks were compared: water, a sports drink (CES), and a sports drink with added 150 mg/l caffeine (CAF). The incidence of GI complaints and the effect of the drinks on performance was studied. Each subject consumed 4 times 150 ml as follows: at the start, after 4.5 km, 9 km, and 13.5 km. Fluid intake was controlled. Incidence and intensity of GI complaints during the run were determined using a 10 points scale questionnaire. There were no significant differences in performance between the 3 drinks. Run time (18 km, mean +/- SD): WAT 1 : 18 : 03 +/- 08 : 30, CES 1 : 18 : 23 +/- 08 : 47, CAF 1 : 18 : 03 +/- 08 : 42. The use of carbohydrate-containing sports drinks led to higher incidences of all types of GI complaints compared to water. Significant differences (p < 0.05) were reached for flatulence; incidence: WAT 17.9 %, CES 28.6 %, CAF 30.6 %, and reflux; incidence: WAT 55.7 %, CES 78.6 %, CAF 72.5 %. There were no significant differences in intensity of the GI complaints. Addition of caffeine to CES had no effect on GI complaints, compared to CES alone. We conclude that sports drinks used during an 18-km run in cool environmental conditions do not support the performance better than mineral water. The use of sports drinks during an 18-km run leads to a higher incidence of both upper and lower GI complaints compared to water. Addition of caffeine to the sports drink has no effect on either running performance or GI complaints.

The capacity of forests to mitigate global climate change can be negatively influenced by tropospheric ozone that impairs both photosynthesis and stomatal control of plant transpiration, thus affecting ecosystem productivity and watershed hydrology. We have evaluated individual and interactive effects of ozone and climate on late season streamflow for six forested watersheds (38–970 000 ha) located in the Southeastern United States. Models were based on 18–26 year data records for each watershed and involved multivariate analysis of interannual variability of late season streamflow in response to physical and chemical climate during the growing season. In all cases, some combination of ozone variables significantly improved model performance over climate‐only models. Effects of ozone and ozone × climate interactions were also consistently negative and were proportional to variations in actual ozone exposures, both spatially across the region and over time. Conservative estimates of the influence of ozone on the variability (R2) of observed flow ranged from 7% in the area of lowest ozone exposure in West Virginia to 23% in the areas of highest exposure in Tennessee. Our results are supported by a controlled field study using free‐air concentration enrichment methodology which indicated progressive ozone‐induced loss of stomatal control over tree transpiration during the summer in mixed aspen‐birch stands. Despite the frequent assumption that ozone reduces tree water loss, our findings support increasing evidence that ozone at near ambient concentrations can reduce stomatal control of leaf transpiration, and increase water use. Increases in evapotranspiration and associated streamflow reductions in response to ambient ozone exposures are expected to episodically increase the frequency and severity of drought and affect flow‐dependent aquatic biota in forested watersheds. Regional and global models of hydrologic cycles and related ecosystem functions should consider potential interactions of ozone with climate under both current and future warmer and ozone‐enriched climatic conditions.