• Home
  • Search
  • Improved Deep Recurrent Q-Network of POMDPs for Automated Penetration Testing
  • Cite Icon16
  • https://doi.org/10.3390/app122010339Copy DOI Icon

Improved Deep Recurrent Q-Network of POMDPs for Automated Penetration Testing

Show More
  • Abstract
  • PDF
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

With the development of technology, people’s daily lives are closely related to networks. The importance of cybersecurity protection draws global attention. Automated penetration testing is the novel method to protect the security of networks, which enhances efficiency and reduces costs compared with traditional manual penetration testing. Previous studies have provided many ways to obtain a better policy for penetration testing paths, but many studies are based on ideal penetration testing scenarios. In order to find potential vulnerabilities from the perspective of hackers in the real world, this paper models the process of black-box penetration testing as a Partially Observed Markov Decision Process (POMDP). In addition, we propose a new algorithm named ND3RQN, which is applied to the automated black-box penetration testing. In the POMDP model, an agent interacts with a network environment to choose a better policy without insider information about the target network, except for the start points. To handle this problem, we utilize a Long Short-Term Memory (LSTM) structure empowering agent to make decisions based on historical memory. In addition, this paper enhances the current algorithm using the structure of the neural network, the calculation method of the Q-value, and adding noise parameters to the neural network to advance the generalization and efficiency of this algorithm. In the last section, we conduct comparison experiments of the ND3RQN algorithm and other recent state-of-the-art (SOTA) algorithms. The experimental results vividly show that this novel algorithm is able to find a greater attack-path strategy for all vulnerable hosts in the automated black-box penetration testing. Additionally, the generalization and robustness of this algorithm are far superior to other SOTA algorithms in different size simulation scenarios based on the CyberBattleSim simulation developed by Microsoft.

Loading PDF

Similar Papers
  • Research Article

ShotFlex: A reinforcement learning-based cyber attack path generation method for cybersecurity evaluation

  • Jan 01, 2025
  • Security and Safety
  • Zhuocheng Yu +5
  • Research Article
  • Citations23

Deep Reinforcement Learning With Modulated Hebbian Plus Q-Network Architecture.

  • May 01, 2022
  • IEEE Transactions on Neural Networks and Learning Systems
  • Pawel Ladosz +7
  • Research Article
  • Citations30

Online leakage current classification using convolutional neural network long short-term memory for high voltage insulators on web-based service

  • Mar 01, 2023
  • Electric Power Systems Research
  • Phuong Nguyen Thanh +1
  • Book Chapter
  • Citations4

Dialogue Control by Pomdp Using Dialogue Data Statistics

  • Nov 01, 2010
  • Yasuhiro Minami +5
  • Conference Article
  • Citations3

Implementation of Distributed Attack Penetration Testing Automation Using Dynamic Infrastructure Framework Axiom on Web-Based Systems

  • Nov 07, 2023
  • Pratama Aji Prisadi +3
  • Conference Article
  • Citations17

Planning with POMDPs using a compact, logic-based representation

  • Jan 01, 2005
  • Chenggang Wang +1
  • Research Article
  • Citations74

POMDPs Make Better Hackers: Accounting for Uncertainty in Penetration Testing

  • Sep 20, 2021
  • Proceedings of the AAAI Conference on Artificial Intelligence
  • Carlos Sarraute +2
  • Supplementary Content
  • Citations13

HYBRID, METRIC - TOPOLOGICAL, MOBILE ROBOT NAVIGATION

  • Jan 01, 2001
  • Infoscience (Ecole Polytechnique Fédérale de Lausanne)
  • N Tomatis
  • PDF
  • Research Article
  • Citations27

Improving Dengue Forecasts by Using Geospatial Big Data Analysis in Google Earth Engine and the Historical Dengue Information-Aided Long Short Term Memory Modeling

  • Jan 21, 2022
  • Biology
  • Zhichao Li +4
  • Research Article
  • Citations39

An adaptive update model based on improved Long Short Term Memory for online prediction of vibration signal

  • Apr 01, 2020
  • Journal of Intelligent Manufacturing
  • Huixin Tian +3
  • Conference Article
  • Citations4

Long Short-Term Graph Memory Against Class-imbalanced Over-smoothing

  • Oct 26, 2023
  • Liang Yang +8
  • Conference Article
  • Citations14

Prediction of Sufficient Accuracy for Human Activity Recognition using Novel Long Short Term Memory in Compared with Decision Tree

  • Mar 23, 2023
  • Sai Charan +2
  • Research Article
  • Citations403

Long short-term memory - Fully connected (LSTM-FC) neural network for PM2.5 concentration prediction

  • Dec 21, 2018
  • Chemosphere
  • Jiachen Zhao +3
  • Research Article

Penetration testing of a quantum key distribution system as a black box.

  • Mar 19, 2026
  • National science review
  • Anqi Huang +10
  • Research Article

PREDICTING FUTURE ENROLLMENT TRENDS AT UNIVERSITAS LANCANG KUNING USING ARIMA AND LSTM MODELS

  • Jun 30, 2025
  • JURTEKSI (Jurnal Teknologi dan Sistem Informasi)
  • Sutejo Sutejo +3
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.