Reference question analysis and search strategy development by man and machine

Background: In an era of explosive growth in biomedical evidence, improving systematic review (SR) search processes is increasingly critical. Text-mining tools (TMTs) are a potentially powerful resource to improve and streamline search strategy development. Two types of TMTs are especially of interest to searchers: word frequency (useful for identifying most used keyword terms, e.g., PubReminer) and clustering (visualizing common themes, e.g., Carrot2). Objectives: The objectives of this study were to compare the benefits and trade-offs of searches with and without the use of TMTs for evidence synthesis products in real world settings. Specific questions included: (1) Do TMTs decrease the time spent developing search strategies? (2) How do TMTs affect the sensitivity and yield of searches? (3) Do TMTs identify groups of records that can be safely excluded in the search evaluation step? (4) Does the complexity of a systematic review topic affect TMT performance? In addition to quantitative data, we collected librarians' comments on their experiences using TMTs to explore when and how these new tools may be useful in systematic review search¬¬ creation. Methods: In this prospective comparative study, we included seven SR projects, and classified them into simple or complex topics. The project librarian used conventional “usual practice” (UP) methods to create the MEDLINE search strategy, while a paired TMT librarian simultaneously and independently created a search strategy using a variety of TMTs. TMT librarians could choose one or more freely available TMTs per category from a pre-selected list in each of three categories: (1) keyword/phrase tools: AntConc, PubReMiner; (2) subject term tools: MeSH on Demand, PubReMiner, Yale MeSH Analyzer; and (3) strategy evaluation tools: Carrot2, VOSviewer. We collected results from both MEDLINE searches (with and without TMTs), coded every citation’s origin (UP or TMT respectively), deduplicated them, and then sent the citation library to the review team for screening. When the draft report was submitted, we used the final list of included citations to calculate the sensitivity, precision, and number-needed-to-read for each search (with and without TMTs). Separately, we tracked the time spent on various aspects of search creation by each librarian. Simple and complex topics were analyzed separately to provide insight into whether TMTs could be more useful for one type of topic or another. Results: Across all reviews, UP searches seemed to perform better than TMT, but because of the small sample size, none of these differences was statistically significant. UP searches were slightly more sensitive (92% [95% confidence intervals (CI) 85–99%]) than TMT searches (84.9% [95% CI 74.4–95.4%]). The mean number-needed-to-read was 83 (SD 34) for UP and 90 (SD 68) for TMT. Keyword and subject term development using TMTs generally took less time than those developed using UP alone. The average total time was 12 hours (SD 8) to create a complete search strategy by UP librarians, and 5 hours (SD 2) for the TMT librarians. TMTs neither affected search evaluation time nor improved identification of exclusion concepts (irrelevant records) that can be safely removed from the search set. Conclusion: Across all reviews but one, TMT searches were less sensitive than UP searches. For simple SR topics (i.e., single indication–single drug), TMT searches were slightly less sensitive, but reduced time spent in search design. For complex SR topics (e.g., multicomponent interventions), TMT searches were less sensitive than UP searches; nevertheless, in complex reviews, they identified unique eligible citations not found by the UP searches. TMT searches also reduced time spent in search strategy development. For all evidence synthesis types, TMT searches may be more efficient in reviews where comprehensiveness is not paramount, or as an adjunct to UP for evidence syntheses, because they can identify unique includable citations. If TMTs were easier to learn and use, their utility would be increased.

Search filters for systematic reviews and meta-analyses in sleep medicine.

In this chapter we present data on trends in innovation in climate mitigation technologies. The data have been generated using newly developed search strategies to extract data from a worldwide database of patent documents (PATSTAT). In comparison with previous reports, this chapter draws upon the more detailed European Classification System (ECLA) and, most significantly, draws upon the expertise of patent examiners at the European Patent Office (EPO) in the development of search strategies. We present data on invention, technology transfer and research co-operation for a broad cross-section of countries over the last three decades. We also review some of the relevant empirical evidence on the determinants of innovation in these fields.

Two experiments with 3‐ and 5‐year‐old children investigated the development of indirect search strategies. Two components of indirect search were evaluated: the acquisition of indirect search information and the utilization of that information to target further searches effectively. Even 3‐year‐olds showed some use of both components of indirect search, although they were more likely than 5‐year‐olds to search in places where previous indirect searches had indicated the target object could not be. An intriguing aspect of the results was that children were not necessarily any more efficient in their indirect searches than they would have been with a direct approach to the problem. This suggests that children actively strive to develop strategies for solving problems intelligently, even when those strategies do not produce any immediate improvement over direct or rote procedures.

A review of:
 
 Bradford, Jane T., Barbara Costello, and Robert Lenholt. “Reference Service in the Digital Age: An Analysis of Sources Used to Answer Reference Questions.” The Journal of Academic Librarianship 31.3 (May 2005): 263-72.
 
 Objective – To test the hypothesis that electronic sources are used by librarians more often than print sources to answer questions at the reference desk.
 
 Design – Use study.
 
 Setting – Small, private university in the United States.
 
 Subjects – Five full-time and two part-time librarians working at the reference desk for four months (two months in the fall of 2002, and two months in the spring of 2003).
 
 Methods – The study recorded every question asked by library patrons during the two study periods, with the exception of non-library related directional questions and hardware problems. For each question recorded, librarians, while working solo at the reference desk, paraphrased the question and recorded the source(s) used to answer each question. Although questions were recorded regardless of source – in person, via email, or by telephone – the medium in which each question was asked and answered was not recorded. For the first half of the study period (fall 2002 semester), librarians kept manual records that were subsequently transcribed into a spreadsheet by a student assistant. In the second half of the study period (spring 2003 semester), the librarians entered data directly into a spreadsheet. The study’s data monitor (who was also a study participant) was responsible for ensuring the integrity of the data and for assigning a category to each source. The source category ‘librarian’ was problematic in that it was unclear whether or not the actual source of the answer was the librarian or a source located by the librarian. After the first half of the study, the procedure was changed to require that a reference librarian was to label a source used to answer a question as ‘librarian’ only if the answer came from a librarian’s own knowledge, and if it did not require consultation of an outside source. Categories were generated on the fly, as the data monitor reviewed the recorded questions and sources. By the end of the study, 23 categories had emerged. While all sources for answers were categorized, questions themselves were not. During the second part of the study, the gender of the patron asking the question at the reference desk was also recorded.
 
 Main results –The results for the fall 2002 and spring 2003 semesters were similar. For the entire study period, librarians used a total of 3,487 sources to answer 2,491 questions. Sources fell into 23 different categories. The top 5 categories used to answer reference questions were databases (23.92%), librarians (23.6%), library catalogue (15.03%), internal Web page (12.27%), and reference books (9.38%). The top five categories accounted for 84.2% of all sources used. For 75% of the questions, librarians referred to a single source for an answer. Almost 60% of the sources used to answer questions were electronic. Of the internal Web pages used to answer questions, the library’s online journal title listings accounted for 76%. Reference books were used to answer questions in less than 10% of cases. Less than 2% of the library’s print reference collection (173 of 9,587 titles) was consulted to answer reference questions during the study period. The approximate 60:40 ratio of questions asked by female to male patrons corresponded to the university’s student body ratio.
 
 Conclusion – The results of this study confirm the researchers’ hypothesis that librarians use electronic sources with greater frequency than they use print sources to answer patrons’ reference questions. The surprising finding in this study is the proportion (approximately one quarter) of reference questions answered by the librarians themselves, without the need to consult an outside source, either print or electronic. The study suggests that a large proportion of the reference collection goes unused in answering patrons’ questions and that librarians often answer a question using only a single a source of information. A reshelving study analyzing use of the reference collection is underway to supplement the results of the current study.

Background and rationale:Obesity is a metabolic condition brought on by the interplay of hereditary and environmental factors, making it one of the most common diseases in the world. Insulin resistance (IR) and obesity have a close connection and can both be advantageous. One of the main methods of epigenetic regulation is DNA methylation modification. Studies have demonstrated over the past few years that DNA methylation is crucial to the emergence of obesity and DNA methylation can lead to IR. Adipose tissue participates in the physiopathological processes of obesity and IR and functions as an endocrine organ controlling the body’s balanced metabolism, thus, adipose tissue-associated gene DNA methylation affects the development of obesity and IR by influencing the function of adipose tissue. Hence, an explanation of current research on DNA methylation, IR, and obesity, following the most recent developments, exploring changes in DNA methylation in different types of adipose tissue in insulin-resistant patients and obese patients may enable the identification of novel targets in clinical obesity prevention and treatment.Method and analysis:The following electronic bibliographic databases will be searched from inception for peer-reviewed original research published: MEDLINE (through PubMed), Scopus, and EMBASE. Cochrane Library, Cochrane Clinical Trials Registry, the National Institutes for Health Clinical Trials Registry, and the WHO International Clinical Trials Registry Platform from inception to December 31, 2021 will be conducted. Systematic reviews will adhere to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses reporting guidelines. The development of search strategies will make use of medical issue phrases and keywords associated with DNA methylation, Adipose tissue DNA methylation, obesity, and IR. Identified citations will be independently reviewed by two authors to determine eligibility at the title and abstract level, and then at the full text and data extraction phases. Disagreements and conflicts will be resolved through discussion with a third author. Two authors will extract the necessary data from the included studies independently, and The Cochrane Risk of Bias Assessment Tool will be used to assess the bias of randomized controlled studies, and the Newcastle–Ottawa scale for nonrandomized controlled studies. If the interventions and outcomes evaluated are sufficiently homogeneous, results from subgroups of studies will be pooled together in a meta-analysis.

A brief review has been provided with regard to some approaches to information retrieval that are now under development in the United States. These developments are discussed in terms of the unit operations of documentation: acquisition; analysis (abstracting, indexing, classifying, extracting); terminology control; recording of results of analysis on searchable medium; storage of source documents; question analysis and development of search strategy;conductingofsearch;anddelivery of search results.At the present time efforts in this field are largely uncoordinated, with decisions on depth of analysis of documents and philosophy of coding system left largely to the desires and opinions of individuals or separate organizations.Two programs in medjcal documentation at Western Reserve University are discussed: one in communicable diseases; and the other in diabetes. Both programs involve the analysis of the literature in considerable detail, using roles and links, and the control of the meaning of terminology using the technique , of semantic factoring.

The main sources of citation injustice are the collective biases in the scientific community, including literature retrieval bias, which has direct effects on citation inequality. In this commentary I focus on index bias in literature databases, inherent/unconscious bias during search strategy development, and systematic bias of controlled vocabularies. Exemplar literature search strategies and/or retrieval analyses comparing Web of Science and OpenAlex are used to demonstrate these biases. Moreover, this commentary offers steps to consider during literature search preparation, search query building, and the search itself, which can lead to a more inclusive representation of the literature in the topic of interest and reduce citation inequity.

Suicide has become a serious public health concern and international research has shown that the majority of individuals who die by suicide had receive healthcare in the year prior to their death. This presents an opportunity for suicide prevention by strategically training healthcare students in suicide prevention knowledge and skills. The objective of this scoping review is to identify literature that describes the design, development, implementation and/or evaluation of suicide prevention training for healthcare and/or social care students in higher education settings. Studies will only be considered eligible for inclusion if they describe the design, development, implementation and/or evaluation of suicide prevention curricula being delivered to healthcare or social care degree students in higher education. Quantitative, qualitative, and mixed method studies published between 2011 and 2023 (inclusive) and in the English language will be considered eligible. This scoping review will be conducted according to the PRISMA guidelines for scoping reviews (PRISMA-ScR). The developed search strategy will be implemented across six databases: PubMed, ERIC (Education Resources Information Center), PsycINFO, Embase, CINAHL and Web of Science. Several grey literature sources will also be consulted. Further potential results will be located by hand-searching the reference lists of included articles. The search strategy will include variations of the terms: 'student', 'suicide prevention' and 'education'. The search will be limited to titles, abstracts, and keywords in databases that allow it. Two reviewers will complete the screening using the predefined inclusion criteria. A third reviewer will resolve any conflicts during the screening and eligibility appraisal processes. Results will be presented in the form of tabulated results and an accompanying narrative summary, describing key findings and context related to learning outcomes, methodologies employed and implementation of the identified programmes.

Objective To evaluate and synthesise current applications of large language models (LLMs) in systematic reviews and meta-analyses (SRMAs), identify key limitations and propose an enhanced theoretical framework to improve the efficiency, scalability and reliability of evidence synthesis. Methods and analysis We conducted a narrative review of recent studies applying LLMs across key SRMA stages. A total of 21 publications were analysed for model type, task application, accuracy metrics and workflow impact. Building on this evidence base, we designed a comprehensive LLM-enhanced SRMA framework that categorises LLM roles as consultants and assistants, integrates human-in-the-loop strategies and uses retrieval-augmented generation (RAG) and agent-based architectures to address critical challenges including hallucinations, bias and workflow inefficiency. Results The reviewed literature demonstrated that LLMs can support various SRMA tasks with reported accuracy ranging from 61% to 99%, showing particular promise in literature screening and data extraction. Our proposed framework conceptualises modular integration of LLMs across all six SRMA stages, with LLMs serving as consultants for research question formulation and search strategy development and as assistants for task automation including abstract screening and structured data extraction. The framework incorporates RAG technology to reduce hallucinations by grounding outputs in retrieved literature and employs agent-based orchestration for complex analytical workflows. Theoretical analysis suggests potential for significant efficiency gains while maintaining methodological rigour through strategic human oversight. Conclusion LLMs offer substantial theoretical potential to transform evidence synthesis by improving efficiency, scalability and consistency across SRMA workflows. The proposed LLM-enhanced framework provides a systematic, theoretically grounded approach for integrating advanced artificial intelligence capabilities into existing SRMA methodologies while preserving essential human oversight and analytical integrity. Future empirical studies are needed to validate the framework’s practical effectiveness, establish implementation protocols and demonstrate real-world benefits in evidence-based medicine.

BackgroundIn response to the high burden of disease among sex workers and their position as a population heavily affected by the HIV epidemic, there has been a growing body of literature investigating the prevalence and risk factors associated with HIV risk among sex workers. To contextualize and summarize the existing research evidence base, a systematic review was conducted to synthesize the epidemiological literature on sex workers in Uganda.MethodsDatabase selection and search strategy development followed the Cochrane Collaboration’s standards for conducting systematic review searches. All studies that included sex workers as the primary research participants were included in the review. The search was then geographically restricted to the country of Uganda. Items were identified from 18 databases (grey and peer-review) on March 10–11, 2015.ResultsA total of 484 articles were retrieved from the database search. After removal of duplicates, a total of 353 articles were screened for eligibility and 64 full-text articles were assessed. The final review included 24 studies with quantitative methodology conducted among sex workers in Uganda. The HIV prevalence among female sex workers ranged from 32.4–52.0 % and between 8.2–9.0 % had multiple HIV infections. Both multi-drug resistance to antiretroviral therapy (2.6 %) and antibiotics (83.1 %) were observed. Between 33.3–55.1 % reported inconsistent condom use in the past month. In the previous 6 months, over 80 % of sex workers experienced client-perpetrated violence and 18 % experienced intimate partner violence. Over 30 % had a history of extreme war-related trauma.ConclusionsThere was limited information on socio-structural factors that affect sex workers’ commercial working environments in Uganda, including the role of policing and criminalization, as well as the prevalence and factors associated with violence. The majority of the existing evidence is based in Kampala, highlighting a need for information on sex work in other regions of Uganda. Additionally, there is limited information on features of the non-commercial components of sex workers’ lives as well as the services needed to reduce risks outside of the sex industry.Electronic supplementary materialThe online version of this article (doi:10.1186/s12889-015-2553-0) contains supplementary material, which is available to authorized users.

This paper describes the search approach for preprints for a post COVID-19 condition (i.e., long COVID) scoping review, including source selection, search strategy development, challenges, and insights throughout a project life cycle. With the growth of medical preprints since the COVID-19 pandemic, information professionals and researchers should be aware that preprints are possible sources of evidence and be prepared to manage them in evidence reviews for COVID-19 topics and beyond. Preprints are not peer-reviewed but can include important evidence about emerging topics. Because of the importance of preprints to the scoping review, a preprint search of Europe PubMed Central (PMC) was added. Europe PMC and similar aggregators combine multiple preprint servers and often have Boolean search, but sometimes limited search functionalities or few export options. Strategy translation encountered challenges such as varying and inconsistent terminology for post-COVID-19 condition, a complex search, and negotiating large numbers of preprints with resource constraints. Europe PMC identified additional preprints for inclusion due to additional preprint server coverage. It was helpful to limit the preprint search to the title and abstract fields, and to run an extra Internet search for publication of included study preprints. Challenges and potential solutions are summarized to support those conducting preprint searches for COVID-19 and other topics.

To determine the extent to which post-hypoglycaemic patients with diabetes, who are prescribed oral hypoglycaemic agents (OHA) are at risk of repeat hypoglycaemic events (RHE) after being treated in the...

This article provides a comprehensive guide to conducting and documenting systematic reviews (SRs) in educational research. While SRs are increasingly recognized for their value in synthesizing and evaluating literature on specific research questions or topics, there remains a notable scarcity of research-based papers that guide their development within the field of education. Systematic reviews, distinguished from traditional literature reviews by their standardized processes—including systematic searching, selection, and critical appraisal of relevant studies—offer a more accurate and comprehensive understanding of the research landscape by integrating findings from multiple sources. This paper underscores the importance of adhering to established methodologies and guidelines to ensure the quality and reliability of SRs. Essential elements discussed include defining research questions, developing search strategies, applying inclusion and exclusion criteria, and synthesizing results. The paper also highlights the role of frameworks such as the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) in enhancing transparency and methodological rigor. By following this structured approach, researchers can produce systematic reviews that provide valuable insights into educational practices and policies, thereby supporting evidence-based decision-making and advancing the field of education.

This study examined the success rates of single immediate implants and their associated biological, hardware and aesthetic complications. Using a developed search strategy, randomized controlled trials (RCTs) on single-unit immediate implants with at least six human participants, a minimum follow-up time of 12 months and published between January 1999 and January 2021 were identified. Data was extracted independently using pre-designed data extraction forms. Information on success rates and associated biological, hardware and aesthetic complications were obtained and assessed. Out of 191 potentially eligible studies, 26 RCTs assessing 1270 patients with a total of 1326 single implants were included and further evaluated. In this review, success rate was reported to be 96.7–100% over a total of 9 studies. However, there was a lack of consensus on a universal success criterion between authors emphasizing the need for agreement. The average follow up was 29 months and most reported complications were aesthetic (63 cases, 4.7%), whilst there were relatively fewer biological, (20 cases, 1.5%), and hardware complications (24 cases, 1.8%). Success rate is an uncommon clinical outcome with 9 out of 26 of the selected RCTs reporting it. In these studies, single immediate implants showed a high success rate with low numbers of biological and hardware complications, and high patient satisfaction with aesthetics were reported in the short-term follow-up of one year.