• Home
  • Search
  • TypeEvalPy: A Micro-benchmarking Framework for Python Type Inference Tools
  • Cite Icon1
  • https://doi.org/10.1145/3639478.3640033Copy DOI Icon

TypeEvalPy: A Micro-benchmarking Framework for Python Type Inference Tools

  • Apr 14, 2024
  • Ashwin Prasad Shivarpatna Venkatesh +5 more
Show More
  • Abstract
  • PDF
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

In light of the growing interest in type inference research for Python, both researchers and practitioners require a standardized process to assess the performance of various type inference techniques. This paper introduces TypeEvalPy, a comprehensive micro-benchmarking framework for evaluating type inference tools. TypeEvalPy contains 154 code snippets with 845 type annotations across 18 categories that target various Python features. The framework manages the execution of containerized tools, transforms inferred types into a standardized format, and produces meaningful metrics for assessment. Through our analysis, we compare the performance of six type inference tools, highlighting their strengths and limitations. Our findings provide a foundation for further research and optimization in the domain of Python type inference.

Loading PDF

Similar Papers
  • Research Article

From ML type inference to stratified type inference

  • Sep 12, 2005
  • ACM SIGPLAN Notices
  • François Pottier
  • Conference Article
  • Citations42

Type inference for COBOL systems

  • Oct 12, 1998
  • A Van Deursen +1
  • Research Article
  • Citations3

Deriving a complete type inference for Hindley–Milner and vector sizes using expansion

  • Mar 25, 2014
  • Science of Computer Programming
  • Axel Simon
  • Research Article
  • Citations49

The ins and outs of gradual type inference

  • Jan 18, 2012
  • ACM SIGPLAN Notices
  • Aseem Rastogi +2
  • Conference Article
  • Citations24

Improved type specialization for dynamic scripting languages

  • Oct 28, 2013
  • Madhukar N Kedlaya +4
  • Research Article
  • Citations18

Dynamic Type Inference to Support Object-Oriented Reenginerring in Smalltalk

  • Jul 20, 1998
  • Open Access CRIS of the University of Bern
  • Pascal Rapicault +3
  • Conference Article
  • Citations9

Exploiting Type Hints in Method Argument Names to Improve Lightweight Type Inference

  • May 01, 2017
  • Nevena Milojkovic +2
  • Conference Article
  • Citations123

A system of constructor classes

  • Jul 01, 1993
  • Mark P Jones
  • Research Article
  • Citations24

LHD 2.0: A text mining approach to typing entities in knowledge graphs

  • Jun 02, 2016
  • Journal of Web Semantics
  • Tomáš Kliegr +1
  • Book Chapter

Type Inferability and Decidability of the Security Problem Against Inference Attacks on Object-Oriented Databases

  • Jan 01, 2004
  • Yasunori Ishihara +2
  • Book Chapter

10 - Defining Facets

  • Jan 01, 2007
  • System Level Design with Rosetta
  • Perry Alexander
  • Research Article

Intermittent Hypoxic Exposure Improves Mean Reaction Time (MRT) Upon Rapid Ascent to High Altitude

  • May 01, 2008
  • Medicine & Science in Sports & Exercise
  • Julie Barnes +8
  • Research Article
  • Citations68

Types for atomicity

  • Jul 01, 2008
  • ACM Transactions on Programming Languages and Systems
  • Cormac Flanagan +3
  • Research Article

Healthcare Innovation Management in a New Era: A Conceptual Design

  • Jan 01, 2013
  • Academy of Management Proceedings
  • James R Smith +1
  • Book Chapter
  • Citations77

Type Inference for Parameterized Race-Free Java

  • Jan 01, 2004
  • Rahul Agarwal +1
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.