Evaluating Construct Convergence: A Canonical Correlation Analysis of Diagnostic Classification Model-Based SJT and IPIP-NEO-PI-120

Towards efficient state of charge estimation of lithium-ion batteries using canonical correlation analysis

Situational judgment tests: An overview of current research

Abstract. During the past 20 years, Situational Judgment Tests (SJTs) have developed into a viable tool in personnel selection. Despite their growing popularity, research examining the extent of measurement error is widely lacking. Using reliability generalization, the aim of this article was twofold: (1) establish an estimate for an average coefficient alpha of SJT scores across studies and (2) examine the influence of essential SJT features and selected study variables on score reliability. To handle potential dependent observations a three-level hierarchical linear model was used. The results indicate that the reliability of SJT scores is typically rather low and below recommended levels for high-stakes applications. Additionally, both SJT and study characteristics affect score reliability. Implications for practitioners and researchers are provided to guide an appropriate use of SJTs and to initiate future research.

The Situational judgment Test (SJT) is a psychological measurement instrument that promotes ideas and knowledge to enable decisions to be made by stakeholders and measures decisions based on the responses of test candidates. SJT is gaining popularity all over the world and is often used in various fields especially for Personnel Selection, promotion and professional development. However, limited practice and discussion has been identified mainly in terms of concepts and theories related to SJT. Therefore, this article discusses the definition of SJT, the history of SJT, and the components of SJT items. The ideas and discussions in this article can not only increase the theoretical value, but also the practical value in measuring the quality of test candidates, especially involving psychological measurements.

Reports an error in "Insights from an updated personnel selection meta-analytic matrix: Revisiting general mental ability tests' role in the validity-diversity trade-off" by Christopher M. Berry, Filip Lievens, Charlene Zhang and Paul R. Sackett (Journal of Applied Psychology, 2024[Oct], Vol 109[10], 1611-1634; see record 2024-80266-001). In the original article, the adverse impact ratios for the dotted lines in Figure 1 are incorrect. The error does not impact the conclusions of this research, and the corrected figure is shown in the erratum. (The following abstract of the original article appeared in record 2024-80266-001). General mental ability (GMA) tests have long been at the heart of the validity-diversity trade-off, with conventional wisdom being that reducing their weight in personnel selection can improve adverse impact, but that this results in steep costs to criterion-related validity. However, Sackett et al. (2022) revealed that the criterion-related validity of GMA tests has been considerably overestimated due to inappropriate range restriction corrections. Thus, we revisit the role of GMA tests in the validity-diversity trade-off using an updated meta-analytic correlation matrix of the relationships six selection methods (biodata, GMA tests, conscientiousness tests, structured interviews, integrity tests, and situational judgment tests) have with job performance, along with their Black-White mean differences. Our results lead to the conclusion that excluding GMA tests generally has little to no effect on validity, but substantially decreases adverse impact. Contrary to popular belief, GMA tests are not a driving factor in the validity-diversity trade-off. This does not fully resolve the validity-diversity trade-off, though: Our results show there is still some validity reduction required to get to an adverse impact ratio of .80, although the validity reduction is less than previously thought. Instead, it shows that the validity-diversity trade-off conversation should shift from the role of GMA tests to that of other selection methods. The present study also addresses which selection methods now emerge as most valid and whether composites of selection methods can result in validities similar to those expected prior to Sackett et al. (2022). (PsycInfo Database Record (c) 2025 APA, all rights reserved).

This paper gives three easily computed highly outlier resistant robust $\sqrt{n}$ consistent estimators of multivariate location and dispersion for elliptically contoured distributions with fourth moments. When the data is from a multivariate normal distribution, the dispersion estimators are also consistent estimators of the covariance matrix. Outlier detection and robust canonical correlation analysis are presented as applications.

Sports associations and companies share a common goal of identifying and selecting top talent, prompting a growing interest in refining their selection processes. Recent decades have witnessed increased attention to predictive valid constructs (e.g., motivation) and predictive assessment methods (e.g., questionnaires). Leveraging an extensive history and empirical evidence from personnel selection research in business, a cross-contextual knowledge transfer could offer potential benefits for both sports and business. To examine the utilization of information between sports and the business context, this study explores the connection between these contexts. Analyzing 20,492 articles from SCOPUS and Web of Science databases, we followed PRISMA guidelines and conducted a citation network analysis (n = 940 articles). Results reveal limited interconnectivity (3,728 links), with only six articles cited between contexts: three sports articles cited business articles and vice versa. Sports articles referred to business research for talent definition, skill assessment, and methodological concerns. Contrarily, the business domain primarily used sports research as examples and for individual aspects of talent development models. This study highlights the potential for points of connection, including requirement analysis, skill assessment, and research on predictive validity. Future research should systematically explore shared and underrepresented topics in both contexts, such as the use of, for example, situational judgement tests in sports to enhance talent selection in sports, drawing insights from the business domain.Systematic review registrationhttps://aspredicted.org/366x-7cfd.pdf.

: U.S. Army leadership recognizes first and foremost the importance of its people - Soldiers - to the effectiveness of transformation to the Future Force. Preparing for this future will affect all aspects of the Soldier management system - selection, job classification, training, and leader development. This research effort is concerned with Soldier accession and job classification and is titled New Predictors for Selecting and Assigning Future Force Soldiers (Select2l). The Select2l goal is to ensure the Army acquires Soldiers with the knowledge, skills, and attributes (KSAs) needed for performing the types of tasks envisioned in a transformed Army. The objectives of the project are to (a) identify Future Force job demands and the pre-enlistment KSAs required to meet them, (b) develop measures of job performance and critical KSAs, and (c) validate the experimental predictor (KSA) measures in a concurrent criterion- related validation. This report documents efforts to develop Select21 predictor and criterion measures. The predictor set includes measures of cognitive ability, temperament, psychomotor skills, values, expectations, and experience. Performance criteria include rating scales to be completed by supervisors and peers, technical knowledge tests, a situational judgment test, and indicators of personal environment fit (e.g., job satisfaction).

Computational psychometrics data and soft skills recognition have become prevalent means in personnel selection processes. Likewise, companies have shown a growing interest in using computational data and machine learning to predict employee behavior. With the aim to enhance selection strategies and applicant reactions simultaneously, the human resources population is researching and developing reliable and valid tools to fit the right person with the right job. In an innovative approach, gamified situational judgment tests have recently received positive results in behavior assessment in combining the acknowledged traditional situation judgment test approach with the advantages of gamification. To pursue previous work in the field and explore this new area of research, we proposed a novel approach to enhance the reliability and validity of gamified situational judgment test’s scoring system based on computational psychometrics data. Our approach has been tested and compared to the initial scoring system of an existing gamified situational judgment test intended to assess bank managers across four soft skills.

The International Classification of Diseases-11th edition (ICD-11) includes a dimensional model of personality disorder assessing five domains of maladaptive personality. To avoid unnecessary complexity, the ICD-11 model includes assessment of personality traits only at the domain level. A measure exists to assess the domains of the ICD-11 model (the Personality Inventory for ICD-11), yet a more rich and useful assessment of personality is provided at the facet level. We used items from the scales assessing the five-factor model of personality disorder to develop the Five-Factor Personality Inventory for ICD-11 (FFiCD), a new 121-item, 20-facet, self-report measure of the ICD-11 maladaptive personality domains at the facet level. Further, the FFiCD includes 47 short scales organized beneath the facets-at the "nuance" level. Items were selected and evaluated empirically across two independent data collections, and the resulting scales were further validated in a third data collection. Correlational and factor analytic results comparing the scales of the FFiCD to the five-factor model, Personality Inventory for ICD-11, and Personality Inventory for DSM-5 supported the validity of the theoretical structure of the FFiCD and the ICD-11 model. The FFiCD may be a useful instrument for clinicians and researchers interested in a more specific assessment of maladaptive personality according to the dimensional ICD-11 personality disorder model. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Canonical correlation analysis (CCA) is a popular method that has been extensively used in feature learning. In nature, the objective function of CCA is equivalent to minimizing the distance of the paired data, and L2-norm is used as the distance metric. We know that L2-norm-based objective function will emphasize the large distance pairs and de-emphasizes the small distance pairs. To alleviate the aforementioned problems of CCA, we propose an approach named CCA based on L1-norm minimization (CCA-L1) for feature learning. To optimize the objective function, we develop an algorithm that can get a global optimized value. To maintain the distribution and the nonlinear characteristic respectively, we proposed two extensions of CCA-L1. Further, all of the aforementioned three proposed algorithms are extended to deal with multifeature data. The experimental results on an artificial dataset, real-world crop leaf disease dataset, ORL face dataset, and PIE face dataset show that our methods outperform traditional CCA and its variants.

P. GREGORY IRVINGThe University of Western OntarioAbstractAlthough the use of personality as a predictor in personnel has not met with substantial success in the past, recent evidence has suggested that personality measures are related to performance criteria which are unrelated to cognitive ability when the traits measured are conceptually related to these criteria. It seems that personality measures may predict job performance dimensions which cannot be predicted by cognitive ability measures. The use of personality measures in personnel may be warranted when a careful job analysis is undertaken to determine which performance dimensions may be related to personality traits. As early as 1923, Freyd (cited in Guion, 1983) recognized that certain steps must be undertaken to ensure that the personnel procedures used by organizations are valid. These steps included conducting a job analysis to determine the characteristics which led to success or failure on the job, designating a single - measure criterion of success, developing an exhaustive list of abilities required for success, finding or devising a measure of these abilities, and statistically comparing the test scores with the criterion scores. Freyd's steps continue to represent sound personnel practises in major Industrial/Organizational psychology textbooks (e.g., Cascio, 1987; Landy, 1989).Despite the fact that Freyd outlined these steps some 70 years ago, researchers investigating the relationship between personality and job performance have tended to ignore his advice. Early attempts to use personality traits to predict various job criteria have generally used a shotgun approach in which a large number of scales were correlated with a large number of criteria. Such an approach has been used to predict accident rates of truck drivers (Parker, 1953) and job satisfaction of farmers (Brayfield & Marsh, 1957). The predictor in both studies was the clinical scales of the Minnesota Multiphasic Personality Inventory (MMPI). The problem with this empirical approach to predicting job - related criteria is that one may expect at least some relationships to be significant by chance alone and attempts at cross - validation would likely result in substantial shrinkage in the validity coefficients. The assessment of personality has been a controversial topic in personnel selection. Over the past several decades, a number of literature reviews have been conducted resulting in conflicting viewpoints regarding the predictability of job performance based on personality traits (Ghiselli & Barthol, 1953; Guion & Gottier, 1965; Schmitt, Gooding, Noe & Kirsch, 1984; Tett, Jackson & Rothstein, 1991). Although Guion and Gottier (1965) stated that there is no generalizable evidence that personality measures can be recommended as good or practical tools for employee selection (p. 159), they observed the importance of predicting job criteria which are unrelated to cognitive ability. Without exception, these reviews have cautioned against the shotgun approach to prediction which has plagued previous attempts to validate personality measures in personnel selection. Hollenbeck and Whitener (1988) have suggested that one of the reasons for the poor predictive ability of personality variables in previous studies is that many of the validation studies lacked statistical power (see Schmidt, Hunter, & Urry, 1976, for a discussion of the lack of power in validation studies). As evidence, they point to the fact that in Guion and Gottier's (1965) review, 62 of 100 validation studies had sample sizes of less than 84.Personality assessments have long been used as a part of the process for several professions including police officers (Burbeck & Furnham, 1985; Inwald & Shusman, 1984), flight attendants (Ferris, Bergin & Gilmore, 1986), and firefighters (Johnson, 1983). Much of the recent research involving personality measures in personnel selection, however, has employed such measures in screening for psychological problems (e. …

The Five Factor Model (FFM) is widely accepted as a valid descriptor of normal personality and commonly used as a framework for prediction of job performance. As an inventory that operationalizes the FFM, the NEO PI-R is often utilized in personnel selection. The California Commission on Peace Officer Standards and Training has identified ten dimensions that increasingly serve as a template for the screening of police officer applicants. These screening dimensions are based on the FFM. The NEO PI-R thus appears well suited to serve as an inventory used for screening police officer applicants. A literature review is provided and strengths and weaknesses of the inventory discussed.

Medical school selection is currently in the paradoxical situation in which selection tools may predict study outcomes, but which constructs are actually doing the predicting is unknown (the ‘black box of selection’). Therefore, our research focused on those constructs, answering the question: do the internal structures of the tests in an outcome-based selection procedure reflect the content that was intended to be measured? Downing’s validity framework was applied to organize evidence for construct validity, focusing on evidence related to content and internal structure. The applied selection procedure was a multi-tool, CanMEDS-based procedure comprised of a video-based situational judgement test (focused on (inter)personal competencies), and a written aptitude test (reflecting a broader array of CanMEDS competencies). First, we examined content-related evidence pertaining to the creation and application of the competency-based selection blueprint and found that the set-up of the selection procedure was a robust, transparent and replicable process. Second, the internal structure of the selection tests was investigated by connecting applicants’ performance on the selection tests to the predetermined blueprint using cognitive diagnostic modeling. The data indicate 89% overlap between the expected and measured constructs. Our results support the notion that the focus placed on creating the right content and following a competency-blueprint was effective in terms of internal structure: most items measured what they were intended to measure. This way of linking a predetermined blueprint to the applicants’ results sheds light into the ‘black box of selection’ and can be used to support the construct validity of selection procedures.

The aim of this transversal research was twofold: to check the metric characteristics of the measuring instruments, and to examine the interactions of the model of The Big Five personality traits – Big Five (as predictor) and dimensions of motivation (as criterion) on a sample of junior volleyball players. The pertinent sample included (N = 218) junior Serbian volleyball players of average age (Mage= 18.05, SD = 1.52). the following measuring instruments were used: Big Five Inventory (BFI), Sport Attitudes Inventory (SAI), and Task and ego orientation in Sport Questionnaire (TEOSQ). Cronbach's alpha reliability coefficient has shown that the values of internal consistency of the measuring instruments are satisfactory, which means that they are suitable to be used in the context of Serbian sports. Two canonical functions statistically significant on the level (p ≤ 0.03) were extracted using the canonical correlation analysis. The first pair of the isolated canonical factors, with 37% of the explained variance, is determined by personality traits: conscientiousness, extraversion, and openness, as well as by the dimensions of motivation: motive for achieving power, motive for achieving success, and task orientation. The second pair of the isolated canonical factors in the group of personality traits is defined by the variables: neuroticism, agreeableness, and conscientiousness (on the negative end), motive for avoiding failure, and goal orientation (on the positive end). That isolated canonical latent variable indicates that young athletes are prone to avoidance as a model of behavior. Empirical findings of this research can be used in player selection and managing sports training and competition of junior volleyball players. The obtained canonical structures can contribute to a better understanding of correlations between latent variables in adolescent volleyball population in Serbia, which have not been sufficiently examined nor scientifically explained.