• Home
  • Search
  • DPDispatcher: Scalable HPC Task Scheduling for AI-Driven Science.
  • https://doi.org/10.1021/acs.jcim.5c02081Copy DOI Icon

DPDispatcher: Scalable HPC Task Scheduling for AI-Driven Science.

Show More
  • Abstract
  • Literature Map
  • References
  • Similar Papers
Abstract

Artificial intelligence (AI) is reshaping computational science, but AI-driven workflows routinely span heterogeneous tasks executed across diverse high-performance computing (HPC) systems. We introduce DPDispatcher, an open-source Python framework for scalable, fault-tolerant task scheduling in such environments with an emphasis on lightweight submission, automatic retries, and robust resumption. DPDispatcher separates connection and file-staging concerns from scheduler control, supports multiple HPC job managers, and provides both local and secure shell (SSH) backends. DPDispatcher has been adopted by more than ten scientific packages. Representative use cases include active learning for machine-learning potentials, free-energy and thermodynamic integration workflows, large-scale materials screening, and large language model (LLM)-driven agents that launch HPC computations. Across these settings, DPDispatcher reduces operational overhead and error rates while improving portability and automation for reliable, high-throughput scientific computing.

Similar Papers
  • Front Matter
  • Citations1

Editorial: Large language models in work and business.

  • Nov 29, 2024
  • Frontiers in artificial intelligence
  • Şadi Evren Şeker
  • Book Chapter

Code Modernization Tools for Assisting Users in Migrating to Future Generations of Supercomputers

  • Jan 01, 2017
  • Ritu Arora +1
  • PDF
  • Research Article
  • Citations15

Students are using large language models and AI detectors can often detect their use

  • Jun 07, 2024
  • Frontiers in Education
  • Timothy Paustian +1
  • Research Article

Integrating Large Language Models Into Trauma Education for Medical Students: Randomized Controlled Pilot Trial

  • Mar 17, 2026
  • JMIR Medical Education
  • Joona Gustafsson +2
  • Research Article
  • Citations63

The impact of AI in physics education: a comprehensive review from GCSE to university levels

  • Feb 06, 2024
  • Physics Education
  • Will Yeadon +1
  • Research Article
  • Citations7

Legal aspects of generative artificial intelligence and large language models in examinations and theses.

  • Jan 01, 2024
  • GMS journal for medical education
  • Maren März +3
  • Research Article

Evaluating gpt-4 for zero-shot classification of bleeding and clotting events: Can large language models serve as second reviewers?

  • Nov 03, 2025
  • Blood
  • Samantha Rizzo +5
  • Conference Article
  • Citations1

Node level Power Profiling and Thermal Management in HPC system

  • Feb 01, 2016
  • Sherin M A +4
  • Conference Article
  • Citations4

Simulation Framework for Studying Optical Cable Failures in Dragonfly Topologies

  • May 01, 2019
  • Tiffany A Connors +3
  • Research Article
  • Citations11

How do generative artificial intelligence (AI) tools and large language models (LLMs) influence language learners’ critical thinking in EFL education? A systematic review

  • Aug 04, 2025
  • Smart Learning Environments
  • Jing Liu +2
  • Research Article
  • Citations45

Custom Large Language Models Improve Accuracy: Comparing Retrieval Augmented Generation and Artificial Intelligence Agents to Non-Custom Models for Evidence-Based Medicine

  • Nov 07, 2024
  • Arthroscopy: The Journal of Arthroscopic and Related Surgery
  • Joshua J Woo +7
  • Research Article
  • Citations60

Harnessing artificial intelligence in bariatric surgery: comparative analysis of ChatGPT-4, Bing, and Bard in generating clinician-level bariatric surgery recommendations

  • Mar 24, 2024
  • Surgery for Obesity and Related Diseases
  • Yung Lee +11
  • Research Article
  • Citations6

Assessing the risk of takeover catastrophe from large language models.

  • Jun 30, 2024
  • Risk analysis : an official publication of the Society for Risk Analysis
  • Seth D Baum
  • Research Article
  • Citations1

GPT-4o and OpenAI o1 Performance on the 2024 Spanish Competitive Medical Specialty Access Examination: Cross-Sectional Quantitative Evaluation Study

  • Jan 12, 2026
  • JMIR Medical Education
  • Pau Benito +9
  • Research Article
  • Citations178

The ethics of disclosing the use of artificial intelligence tools in writing scholarly manuscripts

  • Jun 15, 2023
  • Research Ethics
  • Mohammad Hosseini +2
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.