• Home
  • Search
  • Active Learning for SAT Solver Benchmarking
  • https://doi.org/10.1007/s10817-025-09729-6Copy DOI Icon

Active Learning for SAT Solver Benchmarking

Show More
  • Abstract
  • PDF
  • Literature Map
  • References
  • Similar Papers
Abstract

Benchmarking is crucial for developing new algorithms. This also applies to solvers for the propositional satisfiability (SAT) problem. Benchmark selection is about choosing representative problem instances that reliably discriminate solvers based on their runtime. In this paper, we present a dynamic benchmark selection approach based on active learning. Our approach estimates the rank of a new solver among its competitors, striving to minimize benchmarking runtime but maximize ranking accuracy. Instead of using real-valued solver runtimes, our approach works with discretized runtime labels, which yielded better solver rank predictions. We evaluated this approach on the Anniversary Track dataset from the SAT Competition 2022. Our benchmark selection approach can predict the rank of a new solver after approximately 10 % of the time it would take to run the solver on all instances of this dataset, with a prediction accuracy of approximately 92 %. Additionally, we discuss the importance of instance families in the selection process. In conclusion, our tool offers a reliable method for solver engineers to assess a new solver’s performance efficiently.

Loading PDF

Similar Papers
  • Book Chapter
  • Citations25

Automatically Improving SAT Encoding of Constraint Problems Through Common Subexpression Elimination in Savile Row

  • Jan 01, 2015
  • Peter Nightingale +2
  • Conference Article

An analog SAT solver based on a deterministic dynamical system: (Invited paper)

  • Nov 01, 2017
  • Xunzhao Yin +2
  • Book Chapter
  • Citations172

Chapter 2 Satisfiability Solvers

  • Dec 07, 2007
  • Foundations of Artificial Intelligence
  • Carla P Gomes +3
  • Book Chapter
  • Citations262

PySAT: A Python Toolkit for Prototyping with SAT Oracles

  • Jan 01, 2018
  • Alexey Ignatiev +2
  • Book Chapter
  • Citations6

Writing Declarative Specifications for Clauses

  • Jan 01, 2016
  • Martin Gebser +4
  • Conference Article

Solving query-answering problems using all-solution satisfiability solvers: Algorithm and correctness

  • Dec 01, 2013
  • Kiyoshi Akama +1
  • Conference Article
  • Citations3

An integrated approach for combining BDD and SAT provers

  • Jan 01, 2006
  • R Drechsler +2
  • Conference Article
  • Citations1

SAT based model checking for MDG models

  • Jun 01, 2010
  • Khaza Anuarul Hoque +3
  • Research Article
  • Citations8

Heuristic-Based Backtracking Relaxation for Propositional Satisfiability

  • Oct 01, 2005
  • Journal of Automated Reasoning
  • Ateet Bhalla +3
  • Research Article
  • Citations36

Efficient Analog Circuits for Boolean Satisfiability

  • Jan 01, 2018
  • IEEE Transactions on Very Large Scale Integration (VLSI) Systems
  • Xunzhao Yin +5
  • Research Article
  • Citations36

Automatic construction of parallel portfolios via algorithm configuration

  • May 20, 2016
  • Artificial Intelligence
  • Marius Lindauer +3
  • Research Article
  • Citations16

Improving SAT using 2SAT

  • Jan 01, 2002
  • Lei Zheng +1
  • Conference Article
  • Citations9

Utilizing don't care states in SAT-based bounded sequential problems

  • Apr 17, 2005
  • Sean Safarpour +3
  • Research Article
  • Citations1

What we can learn from conflicts in propositional satisfiability

  • Oct 01, 2015
  • Annals of Operations Research
  • Youssef Hamadi +2
  • Research Article

Extracting Problem Structure with LLMs for Optimized SAT Local Search

  • Jul 19, 2025
  • Proceedings of the International Symposium on Combinatorial Search
  • André Schidler +1
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.