Open science in a closing world

This paper gives the reader a chance to experience, or revisit, PHOS16: a conference on the History and Philosophy of Open Science. In the winter of 2016, we invited a varied international group to engage with these topics at the University of Helsinki, Finland. Our aim was a critical assessment of the defining features, underlying narratives, and overall objectives of the contemporary open science movement. The event brought together contemporary open science scholars, publishers, and advocates to discuss the philosophical foundations and historical roots of openness in academic research. The eight sessions combined historical views with more contemporary perspectives on topics such as transparency, reproducibility, collaboration, publishing, peer review, research ethics, as well as societal impact and engagement. We gathered together expert panelists and 15 invited speakers who have published extensively on these topics, which allowed us to engage in a thorough and multifaceted discussion. Together with our involved audience we charted the role and foundations of openness of research in our time, considered the accumulation and dissemination of scientific knowledge, and debated the various technical, legal, and ethical challenges of the past and present. In this article, we provide an overview of the topics covered at the conference as well as individual video interviews with each speaker. In addition to this, all the talks were recorded and they are offered here as an openly licensed community resource in both video and audio form.

Decoding methods for large language models (LLMs) usually struggle with the tradeoff between ensuring factuality and maintaining diversity. In this paper, we propose REAL (Residual Entropy from Asymptotic Line) sampling,1 which predicts the step-wise hallucination likelihood of an LLM. When an LLM is likely to hallucinate, REAL lowers the p threshold in nucleus sampling. Otherwise, REAL sampling increases the p threshold to boost the diversity. To predict the step-wise hallucination likelihood without supervision, we construct a THF (Token-level Hallucination Forecasting) model, which predicts the asymptotic entropy (i.e., inherent uncertainty) of the next token by extrapolating the next-token entropies of an infinitely large language model from a series of LLMs with different sizes. If an LLM’s entropy is higher than the asymptotic entropy (i.e., the LLM is more uncertain than it should be), the THF model predicts a high hallucination hazard, which leads to a lower p threshold in REAL sampling. In the FactualityPrompts benchmark (Lee et al., 2022), we demonstrate that REAL sampling based on a 70M THF model can substantially improve the factuality and diversity of 7B LLMs simultaneously. After combined with contrastive decoding, REAL sampling outperforms 13 sampling methods, and generates texts that are more factual than the greedy sampling and more diverse than the nucleus sampling with p = 0.5.

This landscape study aims to provide an understanding of Large Language Models (LLMs), especially with a view point of supporting scientific and community queries for climate resilience and preparedness. Typically, for queries related to climate resilience and preparedness, the data span multiple heterogeneous sources and are complex by nature. Scientists as well as communities seek answers to real-world questions from LLMs, yet the accuracy of the feedback is not always guaranteed to be consistent and free of hallucinations. Given the complex landscape and the number of LLM models available, the choice of a model becomes complex too. In addition, LLM ensemble models are also taking on an important role where high level of trustworthiness and accuracy is needed. To address this gap, we performed a review of the LLM research landscape, identifying common themes. This was followed by a deep dive into ensemble models. We evaluate the ensembles as well as a popular single LLM model (that is, ChatGPT) with two sample queries: (i) ‘Are there any linkages between sea ice retreat and ice shelf melt in the Antarctic?’, (ii) ‘What is the best crop for sandy soils?’. The responses were assessed for robustness and accuracy by the domain expert. Through our analysis we concluded that a single LLM provides a parameter free setup whereas the ensemble models needed fine tuning to arrive to robust outcomes. However, the paper does not address ensemble performance issues and neither does it diminish their importance. Instead, this indicates that a lot more research is needed to get ensembles to a more user-friendly and approachable state of use unlike single LLMs. We also outline criteria that an ensemble model selection will need to consider such as initial model selection, retraining, ensemble optimization, energy utilization.

<ns3:p>The 25th annual Bioinformatics Open Source Conference (BOSC 2024, open-bio.org/events/bosc-2024) was part of the 2024 conference on Intelligent Systems for Molecular Biology (ISMB 2024). Launched in 2000 and held yearly since, BOSC is the premier meeting covering open-source bioinformatics and open science. ISMB 2024 was held in Montréal, Canada, with an online participation option. A total of nearly 2000 people attended; about 200 people participated in BOSC sessions. Over the course of two days, BOSC covered a wide range of topics in open science and open source bioinformatics, including Data Analysis, Open Data, Visualization, Developer Tools and Libraries, Standards and Frameworks for Open Science, and Open AI/ML. Mélanie Courtot delivered an impactful first keynote with a perspective on how “The Data Shows We Need Better Data”. The second keynote speaker, Andrew Su, discussed “Open Data, Knowledge Graphs, and Large Language Models.” BOSC ended with a panel, “Open Source AI/ML: A Game Changer for Bioinformatics?,” in which Lawrence Hunter and Thomas Hervé Mboa Nkoudou joined BOSC’s keynote speakers as panelists. Immediately following BOSC, the CollaborationFest was held at Montréal’s University of Québec campus. First launched in 2010, CoFest is a collaborative work event held yearly around BOSC. This year’s CoFest included 42 participants who worked together on 10 projects.</ns3:p>

As Drumond Rennie put it, ‘Science does not come alive until it is shared publicly’ (1998), emphasising that the rapid advancement of scientific research requires its efficient and rigorous dissemination both to encourage the development of new strategies and to avoid duplication of effort and resources. The current model of scientific and technological research is facing a significant challenge: the cost associated with publishing its results. It is now increasingly common for publishers to impose fees on the scientific community to publish their results, generating debate about the impact of this practice on the fairness of scientific dissemination. Requiring researchers to pay publication costs would limit or exclude research from countries with limited resources, creating significant barriers to the dissemination of knowledge. It has been estimated that costs in some cases are up to ten times higher than what is considered adequate to cover the actual costs of publication (Grossmann & Brembs, 2021). This mismatch raises questions about the justification for such fees and the benefits to funders, given that the funds are often public, or even come from researchers' own pockets. Moreover, this current model supports a highly lucrative publishing industry, valued at approximately $10 billion (Global $10B Scientific & Technical Publishing Industry Report, 2019-2023), and is possibly one of the main causes of the emergence of so-called Predatory Journals. These journals, sometimes described as fraudulent, appear to be legitimate journals, but in reality distort certain publication practices, with a purely profit-driven ultimate purpose (Elmore & Weston, 2020), representing an ethical violation by exploiting the need of researchers for financial gain without providing true scientific value. Given the current landscape, it is crucial to reflect on how these practices affect equity in the dissemination of scientific knowledge?, and how publishing power can limit the dissemination of research on specific pathologies in emerging countries, restricting the right of communities to access crucial information and develop effective public health policies?. Undoubtedly, equity of access to and distribution of scientific knowledge is a fundamental principle that must be strongly defended. Paying high costs for scientific and technological publication not only jeopardises equal opportunities for researchers from regions with fewer resources, but also affects fairness in the distribution of knowledge. This economic barrier can result in a systematic exclusion of relevant perspectives and data from emerging countries, deepening inequalities and limiting global progress in critical areas such as public health. Consequently, there is a need to re-examine the current publication model to ensure that science continues to advance in a way that is fair and accessible to all, and to ensure that information vital to public health and scientific progress is not restricted by economic barriers, but is based on principles of fairness and transparency. Indeed, the scientific community must work together to promote the dissemination of knowledge in a manner that is accessible, equitable and free from harmful practices.

Hablar de la enseñanza de la metodología de la investigación científica en la formación profesional es hablar de la formación en y para la producción y difusión de conocimiento científico, a través de la investigación científica y la interacción social universitaria. Asumiendo que esta es la misión de la Universidad, desde la denominada sociedad del conocimiento, en Bolivia se nos remite al modelo educativo que se ha propuesto encarar como política estatal denominado Socio Comunitario Productivo (MEC, 2009) &#x0D; Lo dicho no resulta novedoso, puesto que el Sistema de la Universidad Boliviana, en su VIII Congreso Nacional (CEUB, 1995) resuelve asumir el paradigma productivo, como respuesta a una de las mayores preocupaciones de la sociedad con referencia a la calidad de la Educación Superior cual es la producción del conocimiento desde la investigación, manifestando con ello que es su responsabilidad crear las bases de un nuevo poder, que es el conocimiento y la racionalidad científica. Si bien esta resolución empieza a concretarse con la creación de Institutos de Investigación, la inclusión de la asignatura de Metodología de la Investigación en los Planes Curriculares de todas las Carreras y la implementación de modalidades de titulación que impliquen tanto la investigación cuanto la interacción social, no se ha avanzado en la difusión de trabajos de investigación realizados por los estudiantes durante su formación profesional. Esto quiere decir que tampoco se han desarrollado procesos de enseñanza aprendizaje orientados a la formación de competencias para la producción y difusión de conocimiento científico.

Background. Implementation of responsible research and innovation (RRI) practices in biosciences is important for better understanding of research by society. Public Engagement as the first RRI key can be implemented through Open Science. Strengthening interaction with the society is a significant aspect of Open Science and a prevalent challenge for biologists. Methods and Results. The project “RESponsible research and innovation grounding practices in BIOSciences (ResBios)” comprises interrelated research and implementation designed to achieve the specific objectives. The project runs for three years (January 2019 – December 2022) and focuses on biosciences at the crossroads of science and society. The activities organized in work packages include technical assistance, learning process on RRI-oriented structural change, monitoring and assessment, communication and dissemination of knowledge, and project management. Results and Discussion. This article is dedicated to the description of the expe­rience of the Department of Biochemistry team in engagement of different actors. The Department of Biochemistry of Ivan Franko National University of Lviv participates in the ResBios project and implements the RRI grounding practices, particularly Public Engagement through Open Science. Within the project framework a number of activities were held with a view to popularizing biosciences in society. Among others, a Workshop on pharmaceutical and house chemicals for schoolchildren and teachers of secondary schools and annual Summer School on Nutrition for undergraduate and graduate students of universities were organized. Besides, educational and popular-science videos were created, and the YouTube Chanel of the Department of Biochemistry was launched. Conclusion. Participation in the ResBios project enabled us to improve Public Engagement through Open Science in the activities organized by the team of the Department of Biochemistry. Taking into account the feedback obtained from the participants it could be concluded that these activities will have a prolonged effect on communication between biologists and citizens in Ukraine and abroad.

AI technology’s rapid advancements have led to its widespread adoption in diverse industries, such as support agents, healthcare, education, software development and finance because such AI technologies has the ability to generated human-like text to solve a complex user’s problem. Nevertheless, the effectiveness and success of AI models greatly depends on large language model that contain extensive, varied, and top-notch datasets and social network platforms for both training and evaluation purposes that is sourced from web and directly form user interaction that will also contain sensitive information. However, this power capabilities and progress have raised significant concerns related data security and privacy, particularly regarding protection of user data including social network during training and deployment phase. Large language models process and generate huge amount of data which have the potential to unintentionally memorise and repeat personal sensitive information like location data, social security number, and phone numbers which are shared by users that can lead to pose a risk of data leaking and degrade the accuracy of model training and information. This paper will perform a comprehensive and systematic review to examine the fundamentals principle of large language model with particularly concentration on data privacy concern associated with large language model during training data and potential biases. Furthermore, we analysis and evaluated the current level of vulnerabilities, privacy challenges, and examine new security and privacy attacks targeting LLMs that cause of misuse and information leakage. Additionally, we proposed a set of best practices aimed to protect user data. Recognising the significance of conducting such research on protecting privacy and improving security is crucial because it ensures the continuous progress and public trust in LLM technology.

In the rapidly evolving global shipping industry, accurate vessel trajectory prediction is essential for effective maritime traffic management. However, the inherent uncertainties in vessel voyages pose significant challenges for existing methods in achieving precise long-term predictions. Inspired by the success of large language models (LLMs) in natural language processing, we propose SEMINT—a novel framework that integrates SEMantic cognition and INTent context-awareness for long-term vessel trajectory prediction. SEMINT leverages LLMs to semantically interpret vessel navigation behaviors and infer multiple voyage intents. The final intent inference is derived from the consensus among multiple reasoning chains, which are then combined with historical preferences to form an intent context. Guided by this context, a Transformer-based trajectory prediction model captures the dependencies between critical waypoints and voyage intent, thereby achieving stable long-term numerical predictions. SEMINT combines the advantages of LLMs and task-specific deep learning models, transforming voyage uncertainties into high-level intent diversity and simplifying the learning process for long-term movement patterns. Experiments on real-world datasets show that SEMINT outperforms previous methods, reducing average and final displacement errors by 28.47% and 31.59%, respectively.

Systemic property dispossession from minority groups has often been carried out in the name of technological progress. In this paper, we identify evidence that the current paradigm of large language models (LLMs) likely continues this long history. Examining common LLM training datasets, we find that a disproportionate amount of content authored by Jewish Americans is used for training without their consent. The degree of over-representation ranges from around 2x to around 6.5x. Given that LLMs may substitute for the paid labor of those who produced their training data, they have the potential to cause even more substantial and disproportionate economic harm to Jewish Americans in the coming years. This paper focuses on Jewish Americans as a case study, but it is probable that other minority communities (e.g., Asian Americans, Hindu Americans) may be similarly affected and, most importantly, the results should likely be interpreted as a “canary in the coal mine” that highlights deep structural concerns about the current LLM paradigm whose harms could soon affect nearly everyone. We discuss the implications of these results for the policymakers thinking about how to regulate LLMs as well as for those in the AI field who are working to advance LLMs. Our findings stress the importance of working together towards alternative LLM paradigms that avoid both disparate impacts and widespread societal harms.

Democratizing access to information is an enabler for our digital future. It can transform how knowledge is created, preserved, and shared, and strengthen the connection between academics and the communities they serve. Yet, open scholarship is influenced by history and politics. This article explores the foundations underlying open scholarship as a quest for more just, equitable, and inclusive societies. It analyzes the origins of the open scholarship movement and explores how systemic factors have impacted equality and equity of knowledge access and production according to location, nationality, race, age, gender, and socio-economic circumstances. It highlights how the privileges of the global North permeate academic and technical standards, norms, and infrastructures. It also reviews how the collective design of more open and collaborative networks can engage a richer diversity of communities, enabling greater social inclusion, and presents key examples. By fostering dialogue with multiple stakeholders, more effective avenues for knowledge production and representation can be built based on approaches that are accessible, participatory, interactive, ethical, and transparent, and that reach a far broader public. This expansive vision of open science will lead to a more unified knowledge economy.

&lt;strong&gt; &lt;strong&gt; A value-led perspective on Open Science &lt;/strong&gt; &lt;/strong&gt; In 2021, UNESCO approved its Recommendation on Open Science (OS). By signing this recommendation, 193 countries made a commitment to support the development of OS with a vision of science as a global public good.

This article explores the pivotal role played by the National Consortium ORCID Ukraine in effectively implementing the Open Science Policy within the Ukrainian research landscape. . The article delves into the strategies employed by the consortium to promote ORCID iDs (Open Researcher and Contributor IDs) and their integration into various research processes to enhance research transparency, streamline collaboration, and establish a robust foundation for open science practices. Emphasizing the significance of ORCID iDs in uniquely identifying researchers, the discussion also highlights their impact on reducing authorship ambiguity and ensuring accurate attribution of scholarly contributions. Furthermore, the article explores the broader implications of promoting open science in Ukraine, emphasizing the positive correlation between research transparency and economic development. It is shown that by facilitating a more transparent and collaborative research environment, the National Consortium ORCID Ukraine generates high-quality research outputs, fosters innovation, and attracts investment. The experiences and successes of the consortium in advocating for open science principles, along with the challenges encountered and lessons learned, are detailed. Particularly, it highlighted the challenges and the technical complexity associated with using the ORCID Affiliation Manager tool by Ukrainian research institutes and ensuring compatibility and seamless data transfer between the digital repository system DSpace and the editorial workflows of the publishing platform OJS. The article concludes by underscoring the critical role that initiatives like the National Consortium ORCID Ukraine play in shaping a research ecosystem that not only adheres to open science principles but also stimulates economic growth through knowledge dissemination and collaboration.

Knowledge diffusion has prime importance for generation of new knowledge. The creation of new knowledge is not possible without referring or consulting the past work. Two very important potential flows related to knowledge diffusion can be observed in the common practice of researchers. First, in scientific research knowledge diffusion estimation using citation counts is generally used to establish the value of knowledge which inflates citations. Second the researchers use cited work to search the connected and related resources. Recently the social and collaborative phenomena termed as Web 2.0 has spurred new era of knowledge and information flow on the web. Its potential for the growth and diffusion of scientific knowledge has not been well explored. The emerging social and collaborative applications, such as tagging and bookmarking, are transforming the ways scientists and researchers organize their personal and collaborative information spaces. These bookmarking and tagging applications provide open data and rich metadata resources such as tags. Past research shows that the bookmarking and tagging can be used as a supplementary indicator for measuring research popularity and knowledge diffusion. However the current work exploits author keywords of scientific publications to link these resources with relevant tags extracted from a social bookmarking application such as CiteULike. This work compares, for a focus resource, the tags extracted from CiteULike based on author keywords with their corresponding tag cloud of CiteULike. The result shows that system extends the authors keyword set with social tags providing links to rich and focused resources in CiteULike. This also enhances the serendipitous discovery of emerging concepts related to the focused resource. Such a system may enhance the discovery of related and popular resources for researchers. This dataset has been made available publicly for scientific community.

Research data management (RDM) has become an important discipline that enables researchers to effectively organise, preserve and share their research results.<br> RDM is a new development that aims to prepare researchers for the future by building on the principles of open science. It utilises innovative approaches such as generative artificial intelligence (genAI), which is powered by large language models (LLMs), to complement traditional research methods.<br> As data-driven research becomes increasingly complex, researchers often have to spend a lot of time learning how to manage, analyse and interpret large amounts of information. Traditional data literacy training can be time-consuming and doesn't always keep pace with evolving technologies and methods of analysis.<br> Foundation models based on generative AI offer the potential to streamline this learning process. By automating data pre-processing, pattern recognition and even hypothesis generation, these models can lower the technical barriers to entry, allowing researchers to focus more on insights and discovery rather than spending excessive amounts of time mastering data skills.<br> The objective of this workshop is an exchange of perspectives regarding the implementation of novel RDM approaches using LLMs or not, both past and prospective, in research and practice. The presentations of the Large Language Models for Research Data Management?! workshop were as follows: Magnus Bender<br> <em>Aarhus University, Denmark</em><br> <strong>Welcome</strong> Ralf Möller<br> <em>University of Hamburg, Germany</em><br> Keynote: <strong>Advancing RDM: From Immersion to Argumentation in Science</strong><br> (download pdf) Jens Dörpinghaus^1,2,3, Michael Tiemann^1,2<br> <em>¹University of Koblenz, Germany; ²Federal Institute for Vocational Education and Training (BIBB), Germany; ³Linnaeus University, Sweden</em><br> <strong>Large Language Models in Labor Market Research Data Management: Potentials and Limitations</strong> Edyta Jurkiewicz-Rohrbacher^1,2, Thomas Asselborn²<br> <em>¹University of Regensburg, Germany; ²University of Hamburg, Germany</em><br> <strong>Challenges in Automatic Speech Recognition in the Research on Multilingualism</strong><br> (download pdf) Florian Marwitz, Marcel Gehrke<br> <em>University of Hamburg, Germany</em><br> <strong>Improving Accessibility and Reproducibility by Guiding Large Language Models</strong><br> (download pdf) Maximilian Plazotta, Meike Klettke<br> <em>University of Regensburg, Germany</em><br> <strong>Talk to your database: An open-source in-context learning approach to interact with relational databases through LLMs</strong><br> (download pdf) Thomas Asselborn, Magnus Bender, Florian Marwitz, Ralf Möller, Sylvia Melzer<br> <em>University of Hamburg, Germany</em><br> <strong>Verbalisation Process of a RAG-Based Chatbot to Support Tabular Data Evaluation for Humanities Researchers</strong><br> (download pdf) Magnus Bender<br> <em>Aarhus University, Denmark</em><br> <strong>Farewell</strong>