gospider
Fast web spider written in Go
Platforms: Linux · Last verified September 6, 2026
This package contains a Fast web spider written in Go. The features are: - Fast web crawling - Brute force and parse sitemap.xml - Parse robots.txt - Generate and verify link from JavaScript files - Link Finder - Find AWS-S3 from response source - Find subdomains from response source - Get URLs from Wayback Machine, Common Crawl, Virus Total, Alien Vault - Format output easy to Grep - Support Burp input - Crawl multiple sites in parallel - Random mobile/web User-AgentShowcases
Only use this tool against systems you own or are explicitly authorized to test — see the disclaimer.
Getting started
Install on Kali Linux with `sudo apt install gospider`. See the official Kali tool page and upstream homepage linked above for full usage and configuration details.
