• Home
  • Search
  • Improving Tor Hidden Service Crawler Performance
  • Cite Icon16
  • https://doi.org/10.1109/desec.2018.8625103Copy DOI Icon

Improving Tor Hidden Service Crawler Performance

  • Dec 1, 2018
  • Jonghyeon Park +2 more
Show More
  • Abstract
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

Tor is a popular anonymous network that provides the dark web called the hidden service not accessible with traditional web browsers. The number of hidden services using Tor's anonymous network has been steadily increasing every year with the demand of user privacy concerns as well as illegal services that abuse Tor's anonymity. As a result, how to collect and analyze the contents of Tor hidden service is not yet in full swing. In addition, due to the slow access speed of the Tor browser, it is difficult to observe the dynamics of the hidden services. In this work, we present an efficient Tor hidden service crawling method that can monitor the status of hidden services for the analysis of authentic hidden service contents. We have designed our Tor crawler with the virtualization software, Docker, and parallel Tor browsers to improve the crawling performance. In order to maximize the utilization, we analyzed the collected Tor hidden services by a clustering technique and reduced the crawling targets. Crawling time was 76% improved with 5 Docker containers with 5 browsers. From clustering same hidden services, we can also reduce 39% of crawling the Tor hidden services. From January 1, 2018 to May 31, 2018, we collected 25,261 onion addresses and found 2,527 unique hidden services. We crawled this hidden services twice a day, collect 456,739 HTML pages. We classified 722 hidden services contents and analyzed the Tor hidden service dynamics through collected HTML pages and crawler's log data. As a result, we found that the percentage of illegal services is higher in hidden services, which is actually in service.

Similar Papers
  • Conference Article
  • Citations54

Traffic Analysis Attacks on Tor: A Survey

  • Feb 01, 2020
  • Lamiaa Basyoni +4
  • Research Article
  • Citations8

Subverting Surveillance or Accessing the Dark Web? Interest in the Tor Anonymity Network in U.S. States, 2006–2015

  • May 12, 2020
  • Social Currents
  • Andrew M Lindner +1
  • Conference Article
  • Citations8

SmarTor

  • Dec 03, 2018
  • Greubel Andre +2
  • Research Article
  • Citations29

LASTor: A Low-Latency AS-Aware Tor Client

  • Dec 01, 2014
  • IEEE/ACM Transactions on Networking
  • Masoud Akhoondi +2
  • PDF
  • Research Article
  • Citations1

DIGITAL INFORMATION AS AN OBJECT OF EXPERT RESEARCH IN THE CONTEXT OF DIGITALIZATION: PROBLEMS AND DEVELOPMENT PROSPECTS

  • Jul 03, 2023
  • Criminalistics and Forensics
  • Antonina Cheremnova +1
  • Research Article
  • Citations1

PSO-Enabled Privacy Preservation of Data Clustering

  • Mar 01, 2017
  • Indian Journal of Science and Technology
  • G Asha Kiran +2
  • Research Article
  • Citations2

CONTAINER-BASED DEFENSE: COMBATING DDOS ATTACKS IN SOFTWARE-DEFINED NETWORKS

  • Nov 16, 2025
  • International Journal of Applied Mathematics
  • Sambit Satpathy
  • Book Chapter
  • Citations14

Practical Vulnerabilities of the Tor Anonymity Network

  • Apr 03, 2013
  • Paul Syverson
  • Dissertation

Jurisdictional arbitrage : quantifying and counteracting the threat of government intelligence agencies against Tor

  • Jan 01, 2017
  • Cortes
  • Research Article
  • Citations8

Optimising node selection probabilities in multi‐hop M/D/1 queuing networks to reduce latency of Tor

  • Aug 01, 2014
  • Electronics Letters
  • S.J Herbert +2
  • Research Article
  • Citations25

Soft computing: tools for intelligent reservoir characterization (IRESC) and optimum well placement (OWP)

  • Apr 30, 2001
  • Journal of Petroleum Science and Engineering
  • Masoud Nikravesh +2
  • PDF
  • Research Article
  • Citations33

Employing Statistical Algorithms and Clustering Techniques to Assess Lithological Facies for Identifying Optimal Reservoir Rocks: A Case Study of the Mansouri Oilfields, SW Iran

  • Feb 25, 2024
  • Minerals
  • Seyedeh Hajar Eftekhari +7
  • Conference Article
  • Citations24

Trunked radio systems: traffic prediction based on user clusters

  • Sep 20, 2004
  • Hao Chen +1
  • Research Article

Analysing User Navigation Patterns through Association Rule Mining and Clustering: A Case Study on Web Server Logs

  • Apr 22, 2025
  • Journal of Information Systems Engineering and Management
  • Mukesh Kumar, Dharminder Kumar
  • Conference Article
  • Citations13

Conformance Checking and Classification of Manufacturing Log Data

  • Jul 01, 2019
  • Matthias Ehrendorfer +3
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.