A type 1 hybrid multi-site randomized controlled trial protocol for evaluating virtual interview training among autistic transition-age youth

Services are available to help support existing employment for individuals with psychiatric disabilities; however, there is a gap in services targeting job interview skills that can help obtain employment. We assessed the feasibility and efficacy of Virtual Reality Job Interview Training (VR-JIT) in a randomized controlled trial. Participants were randomized to VR-JIT (n = 25) or treatment-as-usual (TAU) (n = 12) groups. VR-JIT consisted of 10 hours of simulated job interviews with a virtual character and didactic online training. The participants attended 95% of laboratory-based training sessions and found VR-JIT easy to use and felt prepared for future interviews. The VR-JIT group improved their job interview role-play performance (p ≤ 0.05) and self-confidence (p ≤ 0.05) between baseline and follow-up as compared with the TAU group. VR-JIT performance scores increased over time (R = 0.65). VR-JIT demonstrated initial feasibility and efficacy at improving job interview skills and self-confidence. Future research may help clarify whether this intervention is efficacious in community-based settings.

Autistic transition age youth struggle with obtaining employment, and interviewing is a critical barrier to getting a job. We adapted an efficacious virtual reality job interview intervention to meet the needs of autistic transition age youth, called the Virtual Interview Training for Transition Age Youth. This study evaluated whether Virtual Interview Training for Transition Age Youth can be feasibly delivered in high school special education settings and whether Virtual Interview Training for Transition Age Youth improves job interview skills, job interview self-efficacy, job interview anxiety, and access to employment. Forty-eight autistic transition age youth received school-based pre-employment services as usual with Virtual Interview Training for Transition Age Youth, while 23 autistic transition age youth received services as usual only. Local teachers trained and supervised autistic transition age youth using Virtual Interview Training for Transition Age Youth. Participants reported Virtual Interview Training for Transition Age Youth was highly acceptable. Participants receiving services as usual and Virtual Interview Training for Transition Age Youth, compared to participants receiving services as usual only, had better job interview skills and lower job interview anxiety as well as greater access to jobs. Overall, Virtual Interview Training for Transition Age Youth appears to be effective at teaching job interview skills that are associated with accessing competitive jobs. Moreover, youth enjoyed Virtual Interview Training for Transition Age Youth and teachers feasibly implemented the tool within special education pre-employment transition services. Future research needs to better understand how autistic transition age youth from culturally diverse backgrounds and different social, behavioral, or mental health challenges may respond to Virtual Interview Training for Transition Age Youth.

Virtual Reality Job Interview Training (VR-JIT) is a computerized interview simulator with efficacy at enhancing interview skills and employment outcomes. A randomized controlled trial assessed VR-JIT effectiveness for participants in individual placement and support (IPS), in which approximately 55% of individuals with serious mental illness obtain employment. Ninety participants with serious mental illness were randomly assigned to IPS+VR-JIT (N=54) or IPS as usual (N=36), completing pretest-posttest assessments and an employment evaluation at 9 months. Intent-to-treat chi-square analysis, multivariable logistic regression, Cox proportional hazards models, and mixed-effects linear regressions were conducted. Fifty-one percent were IPS nonresponders (i.e., no employment within the first 90 days of IPS). IPS+VR-JIT participants did not have significantly higher employment rates, compared with IPS-as-usual participants (43% versus 28%). IPS nonresponders (N=46) in the IPS+VR-JIT group had greater odds of obtaining employment (odds ratio [OR]=5.82, p=0.014) and shorter time to employment (hazard ratio=2.70, p=0.044) compared with IPS nonresponders in the IPS-as-usual group. Intent-to-treat mixed-effects linear analyses indicated that IPS+VR-JIT, compared with IPS as usual, significantly improved interview skills (p=0.006), interview confidence (p=0.013), and interview anxiety (p=0.019). VR-JIT's potential benefits (increased employment in a shorter time) appeared to be specific to IPS nonresponders, whereas employment outcomes for recent IPS enrollees were not affected. VR-JIT could be a valuable resource for employment specialists to support IPS nonresponders, because 47% of participants engaged in mock interview training with their specialist. Future research should focus on evaluating the effectiveness and implementation of VR-JIT among IPS nonresponders.

Employment rates among individuals with serious mental illness may be improved by engagement in the individual placement and support (IPS) model of supported employment. Results from a recent randomized controlled trial (RCT) indicate that virtual reality job interview training (VR-JIT) improves employment rates among individuals with serious mental illness who have been actively engaged in IPS for at least 90 days. This study reports on an initial implementation evaluation of VR-JIT during the RCT in a community mental health agency. A sequential, complementary mixed-methods design included use of qualitative data to improve understanding of quantitative findings. Thirteen IPS staff trained to lead VR-JIT implementation completed VR-JIT acceptability, appropriateness, and feasibility surveys. Participants randomly assigned to IPS with VR-JIT completed acceptability (N=42) and usability (N=28) surveys after implementation. The authors also conducted five focus groups with IPS staff (N=11) and VR-JIT recipients (N=13) and semistructured interviews with IPS staff (N=9) and VR-JIT recipients (N=4), followed by an integrated analysis process. Quantitative results suggest that IPS staff found VR-JIT to be highly acceptable, appropriate for integration with IPS, and feasible for delivery. VR-JIT was highly acceptable to recipients. Qualitative results add important context to the quantitative findings, including benefits of VR-JIT for IPS staff as well as adaptations for delivering technology-based interventions to individuals with serious mental illness. These qualitative and quantitative findings are consistent with each other and were influenced by VR-JIT's adaptability and perceived benefits. Tailoring VR-JIT instruction and delivery to individuals with serious mental illness may help optimize VR-JIT implementation within IPS.

A pilot RCT of virtual reality job interview training in transition-age youth on the autism spectrum

Individual Placement and Support (IPS) is an evidence based strategy for facilitating employment among adults with severe mental illness (SMI) where staff may lead mock job interviews to prepare clients for real-world interviews (a method with limited scalability and cost effectiveness). A virtual reality job interview training program (VR-JIT)-delivered via the internet-has demonstrated efficacy for increasing employment among adults with SMI. Now, VR-JIT is being implemented with a community mental health agency (CMHA) and evaluated for its effectiveness within IPS. This study is a budget impact analysis, evaluating the costs of preparing a CMHA to implement VR-JIT. Implementation preparation occurred over 7 months from October 1, 2016 to April 30, 2017. CMHA staff (n = 15) and external research partners (n = 3) tracked their hours completing implementation preparation activities. Salaries plus a 28% fringe benefit rate were used to derive a per-hour salary amount for each individual and applied to each activity. Non-labor equipment costs were obtained from purchase receipts. A budget impact analysis evaluated the expenditures associated with preparing the CMHA to implement VRJIT. The total implementation preparation costs equaled $25,482. Labor costs equaled $22,882 and non-labor costs equaled $2,600. In total, 655 person-hours were spent preparing for VR-JIT implementation (e.g., preparing lab space, training, and supervising operations). This study presents an initial evaluation of the budget impact of preparing to implement VR-JIT in a CMHA. Cost considerations for future implementation preparation will be discussed. Given that the cost to prepare to implement an intervention can hinder its adoption, results provide an important analysis for decision-makers that may enhance uptake. Future work will determine the cost-effectiveness of VR-JIT implemented within IPS. This study is registered at http://clinicaltrials.gov, NCT = 03049813, "Virtual Reality Job Interview Training: An Enhancement to Supported Employment in Severe Mental Illness."

Enhancing individual placement and support (IPS) - Supported employment: A Type 1 hybrid design randomized controlled trial to evaluate virtual reality job interview training among adults with severe mental illness

Four randomized controlled trials revealed that virtual-reality job interview training (VR-JIT) improved interviewing skills and the odds of obtaining a job offer among trainees with severe mental illness or autism spectrum disorder. This study assessed whether postintervention interviewing skills mediated the relationship between completion of virtual interviews and receiving job offers by six-month follow-up. VR-JIT trainees (N=79) completed pre- and posttest mock interviews and a brief survey approximately six months later to assess whether they received a job offer. As hypothesized, analyses indicated that the number of completed virtual interviews predicted greater posttest interviewing skills (β=.20, 95% posterior credible interval [PCI]=.08-.33), which in turn predicted trainees' obtaining a job offer (β=.28, 95% PCI=.01-.53). VR-JIT may provide a mechanism of action that helps trainees with various psychiatric diagnoses obtain job offers in the community. Future research can evaluate the community-based effectiveness of this novel intervention.

BackgroundThe incidence of autism spectrum disorders (ASD) is on the rise. Currently, 1 in 59 children are identified with ASD in the United States. ASD refers to a range of neurological disorders that involve some degree of difficulty with communication and interpersonal relationships. The range of the spectrum for autism disorders is wide with those at the higher functioning end often able to lead relatively independent lives and complete academic programs even while demonstrating social awkwardness. Those at the lower functioning end of the autism spectrum often demonstrate physical limitations, may lack speech, and have the inability to relate socially with others. As persons with ASD age, options such as employment become increasingly important as a consideration for long‐term personal planning and quality of life. While many challenges exist for persons with ASD in obtaining and maintaining employment, some research shows that, with effective behavioral and social interventions, employment can occur. About 37% of individuals with ASD report having been employed for 12 months or more, 4 years after exiting high school. However, several studies show that individuals with ASD are more likely to lose their employment for behavioral and social interaction problems rather than their inability to perform assigned work tasks. Although Westbrook et al. (2012a, 2013, 2015) have reviewed the literature on interventions targeting employment for individuals with ASD, this review is outdated and does not account for recent developments in the field.ObjectivesThe objective of this review is to determine the effectiveness of employment interventions in securing and maintaining employment for adults and transition‐age youth with ASD, updating two reviews by Westbrook et al. (2012a, 2013).Search MethodsThe comprehensive search strategy used to identify relevant studies included a review of 28 relevant electronic databases. Search terminology for each of the electronic databases was developed from available database thesauri. Appropriate synonyms were used to maximize the database search output. Several international databases were included among the 28 databases searched. In addition, the authors identified and reviewed gray literature through analysis of reference lists of relevant studies. Unpublished dissertations and theses were also identified through database searches. The programs of conferences held by associations and organizations relevant to ASD and employment were also searched. In sum, the search strategy replicated and expanded the prior search methods used by Westbrook et al. (2012a, 2013).Selection CriteriaSelection criteria consisted of an intervention evaluation using a randomized controlled trial or quasi‐experimental design, an employment outcome, and a population of individuals with ASD.Data Collection and AnalysisWe updated the search from Westbrook et al., replicating and broadening the information retrieval processes. Our wide array of sources included electronic databases, gray literature, and conference and organization websites. Once all potentially relevant studies were located, pairs of coders evaluated the relevance of each title and abstract. Among the studies deemed potentially relevant, 278 were subjected to full‐text retrieval and screening by pairs of coders. Because many intervention studies did not include employment outcomes, only three studies met our inclusion criteria. Given the small number of included studies, meta‐analytic procedures were not used; rather, we opted to use more narrative and descriptive analysis to summarize the available evidence, including an assessment of risk of bias.ResultsThe systematic review update identified three studies that evaluated employment outcomes for interventions for individuals with ASD. All three studies identified in the review suggest that vocation‐focused programs may have positive impacts on the employment outcomes for individuals with ASD. Wehman et al. indicated that participants in Project SEARCH had higher employment rates than control participants at both 9‐month and 1‐year follow‐up time points. Adding autism spectrum disorder supports, Project SEARCH in Wehman et al.'s study also demonstrated higher employment rates for treatment participants than control participants at postgraduation, 3‐month follow‐up, and 12‐month follow‐up. Smith et al. found that virtual reality job interview training was able to increase the number of job offers treatment participants received compared to control participants.Authors' ConclusionsGiven that prior reviews did not identify interventions with actual employment outcomes, the more recent emergence of evaluations of such programs is encouraging. This suggests that there is a growing body of evidence regarding interventions to enhance the employment outcomes for individuals with ASD but also greater need to conduct rigorous trials of vocation‐based interventions for individuals with ASD that measure employment outcomes.

Virtual agents usually come in a virtual environment that can be designed in various ways which might affect users. This paper presents a study that examines whether the design of the virtual environment has an impact on the assessment of the virtual agent and the interaction. In a virtual job interview training, participants interacted with a virtual interviewer that behaved exactly the same, but the background and lighting conditions were manipulated. Our results indicate that the environmental design affects the assessment of the interviewer as well as the interview process.

Adults with serious mental illnesses (e.g., Autism Spectrum Disorder [ASD], schizophrenia, post-traumatic stress disorder [PTSD]) often have difficulties obtaining employment. The Job Interview Training System with Molly Porter, developed in collaboration with Yale and Northwestern Universities and vocational rehabilitation specialists with funding from The National Institutes of Health (R43/44MH080496), allows learners to practice job interviews on computers in a stress free environment. The system includes user-driven educational materials, an interactive job application, a practice simulation with a fictional interviewer (Molly Porter), and extensive feedback. SIMmersion's PeopleSIM™ technology allows each conversation with Molly to provide a unique interview experience, enabling users to gain confidence while building skills. The on-screen coach provides insight during the conversation, and a comprehensive after-action review provides learners with feedback on the entire interview. In a randomized control trial, the system was proven effective at improving participants' interview skills and confidence. Ninety-six (96) unemployed adults with ASD (n=26), schizophrenia/other (n=37) or PTSD (n=33) were recruited. Participants were randomized into control (n=32) and experimental (n=64) conditions. The control group was "wait-listed" to receive training, and the experimental group used the training system with Molly Porter. Both groups completed pre- and post-intervention role-play interviews and self-assessment questionnaires. Analyses of covariance showed that the simulation provided a highly significant training effect, with experimental group participants scoring better in the role-play interviews and self-assessing higher than control group participants. By increasing skills and confidence, this system may ultimately reduce the length of unemployment for adults with mental illnesses.

Purpose – One of the biggest problems in the e-learning distribution process is the lack of quality content and learners' discredit in e-learning content. In order to respond to the various demands of the corporate education field appropriately, it is necessary to search for directions of new e-learning models that are out of traditional e-learning contents. The purpose of this study is to identify recent trend issues related to corporate e-learning and to suggest directions for development. Research design, data, and methodology – Based on the literature review, trend issues that should be considered important in corporate e-learning were derived. Online survey was conducted to evaluate the importance-feasibility of each issue to 13 experts on e-learning and corporate education. The contents of the questionnaire are as follows: 1) recognition of importance and feasibility of trend issues to be considered important in the future corporate education field; 2) factors to be considered in developing future e-learning contents. Results – Six trends derived from a comprehensive literature review. The most important e-learning trends for corporate education field were 'mobile learning', 'micro learning', 'blended learning', 'social learning', 'adaptive learning', ‘engaged learning'. As a result of evaluating the importance and feasibility of each issue, experts point out that 'mobile learning' and 'micro learning' should be actively considered for introduction and utilization at present. In addition, 'social learning' and 'blended learning' need to be actively considered in the near future. On the other hand, experts recognized that 'adaptive learning' and 'engaged learning' need to be prepared from a long-term perspective. Conclusions – There are two main reasons for this result. First, in corporate e-learning, it is important to 1) be able to update on time, 2) the connection with the workplace is important. Second, it requires realistic verification of the expected performance of the learning model. To be considered part of the future are as follows: First, the value and effectiveness of the new e-learning type should be studied. Seconds, e-learning contents should be developed through adopting SAM or Agile methodology. Through this process, we would be able to enhance the quality in e-learning content.

Not minding the gap: Does ride-hailing serve transit deserts?

Job interviews are usually high-stakes social situations where professional and behavioral skills are required for a satisfactory outcome. In order to increase the chances of recruitment technological approaches have emerged to generate meaningful feedback for job candidates. We extended an interactive virtual job interview training system with a Generative Adversarial Network (GAN)-based approach that first detects behavioral weaknesses and subsequently generates personalized feedback. To evaluate the usefulness of the generated feedback, we conducted a mixed-methods pilot study using mock-ups from the job interview training system. The overall study results indicate that the GAN-based generated behavioral feedback is helpful. Moreover, participants assessed that the feedback would improve their job interview performance.KeywordsJob interview trainingGenerative adversarial networksCounterfactual explanationsEngagement

Interview skills are abilities or behaviours that enable a person to be more effective during the job interview process. Interview skills are traditional; many interview skills assist candidates in preparing for and participating in job interviews. A job/assignment interview is a selection process. It is a procedure that entails research into the industry, the position, and the candidate. Negotiations commence when a mutual study is completed. The ultimate goal of both the interviewer and the interviewee is to obtain accurate and valuable information in order to make a decision. Both will examine various questioning strategies as well as points in their favour. Job interviews are a necessary aspect of advancing in profession. They are the pinnacle of all of our hard work in honing our abilities, double-checking our résumé, and gathering references in order to stand out in a sea of applications. An interview is an attempt to obtain as much information as possible from an applicant about his or her suitability for the position in question.