• Home
  • Search
  • Developer Prompts in Practice: An Empirical Study of Bias, Security, and Optimization
  • https://doi.org/10.1109/esem64174.2025.00067Copy DOI Icon

Developer Prompts in Practice: An Empirical Study of Bias, Security, and Optimization

  • Oct 2, 2025
  • Dhia Elhaq Rzig +4 more
Show More
  • Abstract
  • Literature Map
  • References
  • Similar Papers
Abstract

Background: Modern software increasingly relies on Developer Prompts (Dev Prompts)-snippets of natural language embedded directly in source code-to leverage the capabilities of Large Language Models (LLMs) for tasks like classification, summarization, and content generation. Yet, despite the rapid adoption of LLMs and Dev Prompts, it remains unclear to what extent these prompts unintentionally encode biases, invite injection attacks, or underperform due to sub-optimal phrasing. Aims: To address this gap, we present a large-scale empirical analysis of Dev Prompts found in real open-source software projects to assess the prevalence of bias, security vulnerabilities, and performance issues. Then, we propose and validate approaches to mitigate these issues, and demonstrate the practical feasibility of addressing them. Method: We systematically sampled 2,320 Dev Prompts from a set of 40,573 found in open-source software projects, to identify the prevalence of the aforementioned issues. We also implemented a lightweight tool that automatically rewrites flawed prompts. Results: We find evidence of easy-to-fix issues across multiple dimensions: 3.46% of prompts contain language likely to lead to biased model responses, while over 10.75% are vulnerable to straightforward injection attacks, and we posit that many more are amenable to performance improvement through minor adjustments. Our prototype successfully mitigated bias in 68.29% of cases, prevented injection vulnerabilities in 41.81%, and improved performance in 37.1% of tested prompts. Conclusions: Our findings highlight an urgent need for future research and dedicated tool-support to help software developers write safer, fairer, and more effective prompts. To facilitate ongoing work in this emerging area, we share our data and analysis infrastructure publicly. We encourage the community to further explore the implications of Dev Prompts in modern software.

Similar Papers
  • Research Article

Ukrainian Foreign and Security Policy: Theoretical and Comparative Perspectives by J. D. P. Moroney T. Kuzio M. Molchanov (review)

  • Jan 01, 2004
  • Slavonic and East European Review
  • Andrew Wilson
  • Research Article

Needs assessment for the establishment of a Masters of Arts program in international studies with a concentration in international security and a specialization in international terrorism

  • Apr 30, 2008
  • Educational Research Review
  • David H Gray
  • Conference Article
  • Citations3

How to Misuse SMTP over TLS: A Study of the (In) Security of Email Server Communication

  • Aug 01, 2015
  • Lars Baumgaertner +3
  • Conference Article
  • Citations5

An Empirical Study of Smart Contract Decompilers

  • Mar 01, 2023
  • Xia Liu +3
  • Research Article
  • Citations18

Психологическая безопасность личности педагога в условиях неопределенности

  • Jan 01, 2022
  • The Bulletin of Irkutsk State University. Series Psychology
  • E L Trofimova +1
  • Research Article

貿易自由化與國家安全:以《海峽兩岸服務貿易協議》為例

  • Sep 01, 2014
  • 國防雜誌
  • 譚偉恩
  • Conference Article
  • Citations3

Too Quiet in the Library: An Empirical Study of Security Updates in Android Apps' Native Code

  • May 01, 2021
  • Sumaya Almanee +3
  • Research Article
  • Citations3180

SYMPTOMATOLOGY AND MANAGEMENT OF ACUTE GRIEF

  • Sep 01, 1944
  • American Journal of Psychiatry
  • Erich Lindemann
  • Research Article
  • Citations2

Data Poisoning Vulnerabilities Across Health Care Artificial Intelligence Architectures: Analytical Security Framework and Defense Strategies.

  • Jan 23, 2026
  • Journal of medical Internet research
  • Farhad Abtahi +3
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.