Metrics for scientific software

Persistent Identifiers and the Next Generation of Legal Scholarship

The Biodiversity Heritage Library (BHL) will soon upload its 60 millionth page of open access biodiversity literature onto the BHL website and the BHL's Internet Archive Collection. The BHL’s massive repository of free knowledge includes content that is available nowhere else online, as well as accessible versions of content that are locked behind paywalls elsewhere. If we are to continue to expand our understanding of life on Earth, we must ensure that the foundation of biodiversity knowledge provided by BHL is discoverable by the tools we rely on to navigate the ever-expanding internet. These tools – search engines and their algorithms – preferentially deliver (and rank) content with good metadata and persistent identifiers (PIDs). In modern online publishing, PID assignment and linking happens at the point of publication: DOIs (Digital Object Identifiers) for publications, ORCIDs (Open Researcher and Contributor IDs) for people, and RORs (Research Organization Registry IDs) for organisations. The DOI system provided by Crossref (the DOI registration agency for scholarly content) delivers reciprocal citations, enabling convenient clicking from article to article, and citation tracking, enabling authors and institutions to track the impact and reach of their research output. Publications that lack PIDs, which include the vast majority of legacy literature, are hard to find and sit outside the linked network of scholarly research. This makes it nearly impossible to determine whether they are being cited, let alone viewed, mentioned, shared or liked. At TDWG 2020, Page 2020, Kearney 2020, Richard 2020 (and 2019, Page 2019b, Page 2019a, Kearney 2019b, Kearney 2019a and 2018, Kearney 2018), we emphasised the need to bring the historic biodiversity literature into the modern linked network of scholarly research. In October 2020, BHL launched a new working group to do exactly this. The BHL Persistent Identifier Working Group (Team #RetroPID) brings together expertise from across BHL’s global community. Over the past year, we have worked tirelessly to make it easier to find, cite, link, share and track the content on BHL, adding article-level metadata to journals and retrospectively assigning DOIs (#RetroPIDs). Most importantly, we have developed the tools and documentation that will enable the entire BHL community to take contributed content from “just” accessible to persistently discoverable. This paper will detail our efforts to retrofit the historic literature (a square peg) into the modern PID system (a round hole) and will present both the achievements and the challenges of this important work.

Persistent identifiers (PIDs) are reference keys to pieces of digital information or digital objects (Meadows and Haak 2018). PIDs are long-lasting, trustworthy and ideally globally unique, allowing information to be unambigiously associated with a digital object. This allows, for example, collection objects to be annotated with data (e.g., improved geographic coordinates) from different web services and databases (Page 2008). In 2014, Norway began an initiative to provide all museum specimens at the University of Oslo's Natural History Museum (UiO NHM) with PIDs persistent and globally unique identifiers (PIDs) in the form of universally unique identifiers (UUIDs) prefixed by a persistent uniform resource locator (PURL) (Endresen and Svindseth 2014). This poster provides an update of progress made in this endeavor, and details the technical set up and workflow, as well as problems encountered and reflections on the process. So far, roughly 40% of UiO NHM's collections have PIDs entered in the Darwin Core materialSampleID field for the collections management system. The main technical problem has been in configuring the PURL service to set up redirects correctly. In the future, we plan to migrate from PURLs to the Handle system which provides identifiers for digital objects, or to using DOIs (Digital Object Identifiers), which are an implementation of the Handle system (Paskin 2017).

Existem várias formas de recuperação da informação periódica na Internet, mas não de maneira eficiente, como, por exemplo, através de um identificador persistente de objeto. Com essa preocupação, o Portal de Periódicos Eletrônicos Científicos (PPEC), vinculado ao Sistema de Bibliotecas da UNICAMP (SBU), manifestou, junto à Coordenação Geral da Universidade (CGU), o pedido de afiliação à CrossRef / PILA (Publisher International Linking Association) por meio da Associação Brasileira de Editores Científicos (ABEC), visando a aquisição do Digital Identifier Object (DOI), a ser atribuído aos periódicos vinculados e credenciados no PPEC. O presente trabalho tem como objetivo apresentar o processo de atribuição e controle do DOI nos periódicos do PPEC, a fim de garantir a sua identificação persistente na Internet para efeito da busca e recuperação dos metadados qualificados. Metodologicamente, os editores e assistentes da Universidade vinculados ao PPEC verificam, após a publicação do fascículo, se o mesmo está em conformidade com os metadados. Após essa verificação, é iniciado o processo de validação desse fascículo e seus artigos na CrossRef. A ferramenta que subsidia esse processo de atribuição é Open Journal System (OJS/SEER), que já possui integrado ao sistema plug-in de habilitação para o DOI, tanto para o fascículo quanto para os artigos dos periódicos. Optou-se pelo controle por artigos, pois ele facilita a validação de cada DOI junto à CrossRef, cujo pagamento é efetuado a cada três meses por meio da ABEC. Existem duas formas de realizar a validação do DOI: gerando um arquivo na linguagem XML ou selecionando os artigos e os registrando dentro do próprio sistema. A diferença é que enviando o XML, recebemos a comunicação de validação por e-mail. Já no registro dentro do sistema, a verificação da validação só é possível através da conexão à CrossRef. Após a realização da validação, é realizado um controle, em uma planilha Excel, para fins de pagamento e também do que está sendo enviado à CrossRef. Obtivemos de imediato, junto a CGU, a contemplação para atribuição de 17.000 DOIs a toda coleção de periódicos de 2000 a 2016 da UNICAMP, o que garantirá aos periódicos a internacionalização dos seus artigos. Concluímos, assim, que esse trabalho planejado, organizado e de total relevância para a Universidade aumentará cada vez mais o impacto na busca por seus periódicos, evidenciando, dessa forma, o quanto uma publicação qualificada e estruturada é capaz de promover a visibilidade e a aceitação junto à comunidade científica.

<p>GBIF—the Global Biodiversity Information Facility—and its network of more than 1,500 institutions maintain the world's largest index of biodiversity data (https://www.gbif.org), containing nearly 1.4 billion species occurrence records. This infrastructure offers a model of best practices, both technological and cultural, that other domains may wish to adapt or emulate to ensure that its users have free, FAIR and open access to data.</p><p>The availability of community-supported data and metadata standards in the biodiversity informatics community, combined with the adoption (in 2014) of open Creative Commons licensing for data shared with GBIF, established the necessary preconditions for the network's recent growth.</p><p>But GBIF's development of a data citation system based on the uses of DOIs—Digital Object Identifiers—has established an approach for using unique identifiers to establish direct links between scientific research and the underlying data on which it depends. The resulting state-of-the-art system tracks uses and reuses of data in research and credits data citations back to individual datasets and publishers, helping to ensure the transparency of biodiversity-related scientific analyses.</p><p>In 2015, GBIF began issuing a unique Digital Object Identifier (DOI) for every data download. This system resolves each download to a landing page containing 1) the taxonomic, geographic, temporal and other search parameters used to generate the download; 2) a quantitative map of the underlying datasets that contributed to the download; and 3) a simple citation to be included in works that rely on the data.</p><p>When authors cite these download DOIs, they in effect assert direct links between scientific papers and underlying data. Crossref registers these links through Event Data, enabling GBIF to track citation counts automatically for each download, dataset and publisher. These counts expand to display a bibliography of all research reuses of the data.This system improves the incentives for institutions to share open data by providing quantifiable measures demonstrating the value and impact of sharing data for others' research.</p><p>GBIF is a mature infrastructure that supports a wide pool of researchers publish two peer-reviewed journal articles that rely on this data every day. That said, the citation-tracking and -crediting system has room for improvement. At present, 21% of papers using GBIF-mediated data provide DOI citations—which represents a 30% increase over 2018. Through outreach to authors and collaboration with journals, GBIF aims to continue this trend.</p><p>In addition, members of the GBIF network are seeking to extend citation credits to individuals through tools like Bloodhound Tracker (https://www.bloodhound-tracker.net) using persistent identifiers from ORCID and Wikidata IDs. This approach provides a compelling model for the scientific and scholarly benefits of treating individual data records from specimens as micro- or nanopublications—first-class research objects that advancing both FAIR data and open science.</p>

Persistent identifiers (PIDs) such as Digital Object Identifiers (DOIs) provide a unique and persistent way to identify and cite digital objects such as publications, media content and research data. They are widely used by data producers to catalogue and publish digital assets and research data. Nowadays, research infrastructures (RIs) offer services not only for accessing and publishing data objects, but also for processing data based on user demands, e.g., via scientific workflows or third party virtual research environments. However, efficiently retrieving and sharing digital objects in a shared data processing environment requires knowledge of application access patterns as well as the underlying network level distribution. As the number and size of data objects increases, optimizing data discovery and access among distributed partners on shared infrastructure emerges as an important challenge for infrastructure operators to maintain quality of service and user experience. In this paper, we propose a novel approach that utilizes Information Centric Networking (ICN) to retrieve content based on PIDs while optimizing data access on shared infrastructure.

Data are the infrastructure of science and they serve as the groundwork for scientific pursuits. Data publication has emerged as a game-changing breakthrough in scholarly communication. Data form the outputs of research but also are a gateway to new hypotheses, enabling new scientific insights and driving innovation. And yet stakeholders across the scholarly ecosystem, including practitioners, institutions, and funders of scientific research are increasingly concerned about the lack of sharing and reuse of research data. Across disciplines and countries, researchers, funders, and publishers are pushing for a more effective research environment, minimizing the duplication of work and maximizing the interaction between researchers. Availability, discoverability, and reproducibility of research outputs are key factors to support data reuse and make possible this new environment of highly collaborative research. An interoperable e-infrastructure is imperative in order to develop new platforms and services for to data publication and reuse. DataCite has been working to establish and promote methods to locate, identify and share information about research data. Along with service development, DataCite supports and advocates for the standards behind persistent identifiers (in particular DOIs, Digital Object Identifiers) for data and other research outputs. Persistent identifiers allow different platforms to exchange information consistently and unambiguously and provide a reliable way to track citations and reuse. Because of this, data publication can become a reality from a technical standpoint, but the adoption of data publication and data citation as a practice by researchers is still in its early stages. Since 2009, DataCite has been developing a series of tools and services to foster the adoption of data publication and citation among the research community. Through the years, DataCite has worked in a close collaboration with interdisciplinary partners on these issues and we have gained insight into the development of data publication workflows. This paper describes the types of different actions and the lessons learned by DataCite.Â

I have variously talked about persistent identifiers on this blog. These largely take the form of DOIs (Digital object identifiers), and here they relate to either journal articles or datasets associated with either the article or the blog post or both. Other disciplines, particularly the earth sciences, have long used persistent identifiers (PIDs) to identify physical objects rather than digital ones.

In the academia it is “publish or perish”. At some stage, the number of international publications also counts. It may however sound ridiculous that referencing may hinder an author from being able to publish in some international journals. This is because such international journals request for the Digital Object Identifier (DOI) of references cited by authors. Regrettably, most authors have never heard about DOI before, let alone being familiar with it. DOI is a standard numbering system like ISBN/ISSN. It is a unique code preferred by publishers in the identification and exchange of the content of a digital object such as journal articles, web documents, or other intellectual property. This article is an eye opener to the features and benefits of DOI, anatomy of a DOI name, citing DOI in references and locating DOI of references.Key words: Digital object identifier (DOI), authors, referencing.

PurposeThe publisher of the International Neurourology Journal changed the text to English in 2010 to promote the journal as an international publication. Four years later, what has happened to this journal? This paper will use citation indicators to describe the degree of internationalization.MethodsCitation indicators such as impact factors, total citations from Web of Science, Science Journal Rankings (SJR), cites per documents (2 years), and Hirsch indexes (h-indexes) from Web of Science, digital object identifier (DOI)/CrossRef, ScimagoJR, or Scopus were calculated. In addition, the native countries of the authors and researchers citing the journal in Web of Science were analyzed.ResultsImpact factors in 2012 and 2013 were 0.645 and 0.857, respectively. Total citations in 2011, 2012, and 2013 from Web of Science were 15, 51, and 99, respectively, and the SJRs in 2011 and 2012 were 0.220 and 0.390, respectively. The h-indexes from DOI/CrossRef, Scopus, and Web of Science were 7, 8, and 6, respectively. Out of 153 unsolicited published papers, 27 (17.6%) were from outside of Korea. The researchers citing the journal in Web of Science and Scopus were primarily from the United States, Korea, China, the United Kingdom, and France. Funding agencies supported 39 of 101 original articles (38.6%).ConclusionsAfter changing the text to the English language, the citation indicators show that the International Neurourology Journal has been elevated to an international journal. Although the nationality of authors varies from year to year, the increase in the number of manuscripts from international authors is obvious.

The work required to collect, clean and publish biodiversity datasets is significant, and those who do it deserve recognition for their efforts. Researchers publish studies using open biodiversity data available from GBIF—the Global Biodiversity Information Facility—at a rate of about two papers a day. These studies cover areas such as macroecology, evolution, climate change, and invasive alien species, relying on data sharing by hundreds of publishing institutions and the curatorial work of thousands of individual contributors. With more than 90 per cent of these datasets licensed under Creative Commons Attribution licenses (CC BY and CC BY-NC), data users are required to credit the dataset providers. For GBIF, it is crucial to link these scientific uses to the underlying data as one means of demonstrating the value and impact of open science, while seeking to ensure attribution of individual, organizational and national contributions to the global pool of open data about biodiversity. Every single authenticated download of occurrence records from GBIF.org is issued a unique Digital Object Identifier (DOI). These DOIs each resolve to a landing page that contains details of the search parameters used to generate the download a quantitative map of the underlying datasets that contributed to the download a simple citation to be included in works that rely on the data the search parameters used to generate the download a quantitative map of the underlying datasets that contributed to the download a simple citation to be included in works that rely on the data When used properly by authors and deposited correctly by journals in the article metadata, the DOI citation establishes a direct link between a scientific paper and the underlying data. Crossref—the main DOI Registration Agency for academic literature— exposes such links in Event Data, which can be consumed programmatically to report direct use of individual datasets. GBIF also records these links, permanently preserving the download archives while exposing a citation count on download landing pages that is also summarized on the landing pages of each contributing datasets and publishers. The citation counts can be expanded to produce lists of all papers unambiguously linked to use of specific datasets. In 2018, just 15 per cent of papers based on GBIF-mediated data used DOIs to cite or acknowledge the datasets used in the studies. To promote crediting of data publishers and digital recognition of data sharing, the GBIF Secretariat has been reaching out systematically to authors and publishers since April 2018 whenever a paper fails to include a proper data citation. While publishing lags may hinder immediate effects, preliminary findings suggest that uptake is improving—as the number of papers with DOI data citations during the first part of 2019 is up more than 60 per cent compared to 2018. Focusing on the value of linking scientific publications and data, this presentation will explore the potential for establishing automatic linkage through DOI metadata while demonstrating efforts to improve metrics of data use and attribution of data providers through outreach campaigns to authors and journal publishers.

Among the entities involved in software citation, software source code\nrequires special attention, due to the role it plays in ensuring scientific\nreproducibility. To reference source code we need identifiers that are not only\nunique and persistent, but also support \\emph{integrity} checking\nintrinsically. Suitable identifiers must guarantee that denotedobjects will\nalways stay the same, without relying on external third parties and\nadministrative processes. We analyze the role of identifiers for digital\nobjects (IDOs), whose properties are different from, and complementary to,\nthose of the various digital identifiers of objects (DIOs) that are today\npopular building blocks of software and data citation toolchains.We argue that\nboth kinds of identifiers are needed and detail the syntax, semantics, and\npractical implementation of the persistent identifiers (PIDs) adopted by the\nSoftware Heritage project to reference billions of softwaresource code\nartifacts such as source code files, directories, and commits.\n

The operation of large US Department of Energy (DOE) research facilities, like the DIII-D National Fusion Facility, results in the collection of complex multi-dimensional scientific datasets, both experimental and model-generated. In the future, it is envisioned that integrated data analysis coupled with large-scale high performance computing (HPC) simulations will be used to improve experimental planning and operation. Practically, massive data sets from these simulations provide the physics basis for generation of both reduced semi-analytic and machine-learning-based models. Storage of both HPC simulation datasets (generated from US DOE leadership computing facilities) and experimental datasets presents significant challenges. In this paper, we present a vision for a DOE-wide data management workflow that integrates US DOE fusion facilities with leadership computing facilities. Data persistence and long-term availability beyond the length of allocated projects is essential, particularly for verification and recalibration of artificial intelligence and machine learning (AI/ML) models. Because these data sets are often generated and shared among hundreds of users across multiple leadership computing facility centers, they would benefit from cross-platform accessibility, persistent identifiers (e.g. DOI, or digital object identifier), and provenance tracking. The ability to handle different data access patterns suggests that a combination of low cost, high latency (e.g. for storing ML training sets) and high cost, low latency systems (e.g. for real-time, integrated machine control feedback) may be needed.KeywordsFusionBig dataHigh performance computingData management

Implementation of data citations and persistent identifiers at the ORNL DAAC

<sec> <title>BACKGROUND</title> The COVID-19 pandemic exposed significant limitations in existing data infrastructure, particularly the lack of systems for rapidly collecting, integrating, and analyzing data to support timely and evidence-based public health responses. These shortcomings hampered efforts to conduct comprehensive analyses and make rapid, data-driven decisions in response to emerging threats. To overcome these challenges, the US National Institutes of Health launched the Rapid Acceleration of Diagnostics (RADx) initiative. A key component of this initiative is the RADx Data Hub—a centralized, cloud-based platform designed to support data sharing, harmonization, and reuse across multiple COVID-19 research programs and data sources. </sec> <sec> <title>OBJECTIVE</title> We aim to present the design, implementation, and capabilities of the RADx Data Hub, a cloud-based platform developed to support findable, accessible, interoperable, reusable (FAIR) data practices and enable secondary analyses of the COVID-19–related data contributed by a nationwide network of researchers. </sec> <sec> <title>METHODS</title> The RADx Data Hub was developed on a scalable cloud infrastructure, grounded in the FAIR data principles. The platform integrates heterogeneous data types—including clinical data, diagnostic test results, behavioral data, and social determinants of health—submitted by over 100 research organizations across 46 US states and territories. The data pipeline includes automated and manual processes for deidentification, quality validation, expert curation, and harmonization. Metadata standards are enforced using tools such as the Center for Expanded Data Annotation and Retrieval (CEDAR) Workbench and BioPortal. Data files are structured using a unified specification to support consistent representation and machine-actionable metadata. </sec> <sec> <title>RESULTS</title> As of May 2025, the RADx Data Hub hosts 187 studies and over 1700 data files, spanning 4 RADx programs: RADx Underserved Populations (RADx-UP), RADx Radical (RADx-rad), RADx Tech, and RADx Digital Health Technologies (RADx DHT). The Study Explorer and Analytics Workbench components enable researchers to discover relevant studies, inspect rich metadata, and conduct analyses within a secure cloud-based environment. Harmonized data conforming to a core set of common data elements facilitate cross-study integration and support secondary use. The platform provides persistent identifiers (digital object identifiers) for each study and supports access to structured metadata that adhere to the CEDAR specification, available in both JSON and YAML formats for seamless integration into computational workflows. </sec> <sec> <title>CONCLUSIONS</title> The RADx Data Hub successfully addresses key data integration challenges by providing a centralized, FAIR-compliant platform for public health research. Its adaptable architecture and data management practices are designed to support secondary analyses and can be repurposed for other scientific disciplines, strengthening data infrastructure and enhancing preparedness for future health crises. </sec>