ServiceNow (United States)

Large language models (LLMs) are a new asset class in the machine-learning landscape. Here we offer a primer on defining properties of these modeling techniques. We then reflect on new modes of investigation in which LLMs can be used to reframe classic neuroscience questions to deliver fresh answers. We reason that LLMs have the potential to (1) enrich neuroscience datasets by adding valuable meta-information, such as advanced text sentiment, (2) summarize vast information sources to overcome divides between siloed neuroscience communities, (3) enable previously unthinkable fusion of disparate information sources relevant to the brain, (4) help deconvolve which cognitive concepts most usefully grasp phenomena in the brain, and much more.

In the version of this article initially published, Sara R. Jordan's first name was misspelled ("Sarah"), and has now been corrected in the HTML and PDF versions of the article.

In a conversational question answering scenario, a questioner seeks to extract information about a topic through a series of interdependent questions and answers. As the conversation progresses, they may switch to related topics, a phenomenon commonly observed in information-seeking search sessions. However, current datasets for conversational question answering are limiting in two ways: 1) they do not contain topic switches; and 2) they assume the reference text for the conversation is given, i.e., the setting is not open-domain. We introduce TopiOCQA (pronounced Tapioca), an open-domain conversational dataset with topic switches based on Wikipedia. TopiOCQA contains 3,920 conversations with information-seeking questions and free-form answers. On average, a conversation in our dataset spans 13 question-answer turns and involves four topics (documents). TopiOCQA poses a challenging test-bed for models, where efficient retrieval is required on multiple turns of the same conversation, in conjunction with constructing valid responses using conversational history. We evaluate several baselines, by combining state-of-the-art document retrieval methods with neural reader models. Our best model achieves F1 of 55.8, falling short of human performance by 14.2 points, indicating the difficulty of our dataset. Our dataset and code is available at https://mcgill-nlp.github.io/topiocqa.

The advances in AI over the last decade has led to significantly better outcomes and improved experiences in consumer applications. Meanwhile, successful applications of AI in enterprises have been modest during this period. In this talk, I will provide an overview of the scope of Enterprise AI, how it is similar and different from AI for consumer applications, a few successful applications, open problems and challenges and the enormous opportunity to create better outcomes and transform the experiences for employees and customers of enterprises.

Natural Language Processing (NLP) models are found to exhibit discriminatory stereotypes across many social constructs, e.g. gender and race. In comparison to the progress made in reducing bias from static word embeddings, fairness in sentence-level text encoders received little consideration despite their wider applicability in contemporary NLP tasks. In this paper, we propose a debiasing method for pre-trained text encoders that both reduces social stereotypes, and inflicts next to no semantic damage. Unlike previous studies that directly manipulate the embeddings, we suggest to dive deeper into the operation of these encoders, and pay more attention to the way they pay attention to different social groups. We find that stereotypes are also encoded in the attention layer. Then, we work on model debiasing by redistributing the attention scores of a text encoder such that it forgets any preference to historically advantaged groups, and attends to all social classes with the same intensity. Our experiments confirm that reducing bias from attention effectively mitigates it from the model’s text representations.

Purpose: Oncofertility care at cancer diagnosis remains underimplemented across oncology and fertility care settings, with limited tools to scale up effective implementation strategies. Using implementation science theory, we systematically assessed factors that influence oncofertility care implementation and mapped scalable strategies, particularly electronic health record (EHR)-enabled ones, that fit adult and pediatric oncology care contexts. Methods: Using purposeful sampling, we recruited health care providers and female, reproductive-aged survivors of adolescent and young adult (AYA) cancers (AYA survivors) from a comprehensive cancer center and a freestanding children's hospital to semistructured interviews and focus groups. Using thematic analysis combining inductive codes with deductive codes using the Consolidated Framework for Implementation Research (CFIR), we characterized barriers and facilitators to care and designed responsive strategies. Two coders independently coded each transcript. Results: We recruited 19 oncology and fertility providers and 9 cancer survivors. We identified barriers and facilitators to oncofertility care in the CFIR domains of individual, inner setting, outer setting, and process, allowing us to conceptualize oncofertility care to encompass three core components (screening, referral, and fertility preservation counseling) and map five strategies to these components that fit an adult and a children's context and bridge oncology and fertility practices. The strategies were screening using a best practice advisory, referral order, telehealth fertility counseling, provider audit and feedback, and provider education. All but provider education were EHR tools with embedded efficiencies. Conclusion: An implementation science approach systematically assessed oncofertility care and mapped strategies to provide a theory-based approach and scalable EHR tools to support wider dissemination.

The COVID-19 pandemic has globally impacted the world with both near-term and long-term damage. At an individual level the impacts were financially, mentally, physically, etc. which resulted in drastic behavior shifts. The majority of working professionals were permitted by the government, local authorities & the employers continue working from home (remote work) to avoid mass gatherings & maintain social distancing. Considering the user research professionals, conducting in-person user research and other qualitative studies became challenging, which has also impacted the working methodology of a user researcher. The primary goals of the study were to understand the impact of the pandemic on user research activities and identifying the challenges faced in various research stages by a user researcher. In this study, 57 user researchers (work experience ranging from 0 to 10+ years) participated by responding an online survey. Key themes emerged from the study such as ‘increase in effort to co-ordinate’, ‘exploration & usage of digital tools’, ‘scope & time-lines’, ‘virtual cross team collaboration’, and ‘challenges with participant recruitment’ [1]. The current context of a global pandemic presents challenges to the continuation of longitudinal studies [2]. Approximately 67% participants reported facing challenges in working remotely & conducting research. ‘Running & Execution (57.89%)’, ‘Planning (55.26%)’ and ‘Preparation (47.37%) were identified as the most challenging stages of research.

Building conversational task-oriented bots requires large and diverse sets of annotated user utterances to learn mappings between natural language utterances and user intents. Given the complexity of human language as well as the recent advances on intent recognition (especially deep-learning-based approaches), bot developers now have faced a new challenge: efficiently and effectively collecting a large number of quality (e.g., diverse and unbiased) training samples. This article studies training user utterance acquisition along several important dimensions including cost and quality. We discuss state of the art techniques, identify open issues, and inform an outlook on future research directions.