Was just messing around with subdomain enumeration and wanted to give a quick overview of a few tools I find useful: hakrawler, gau, theHarvester, sublist3r, gobuster, and recon-ng.
For the most part, they fall into two categories – passive and active enumeration. Passive enumeration will not interact with the target itself, instead finding information in places like web archives and search engine databases (gau, for example, uses AlienVault’s Open Threat Exchange, the Wayback Machine, Common Crawl, and URLScan). Active enumeration will interact with the target directly (hakrawler, for example, will find the URLs that respond to http(s) and crawl them all). For recon-ng, it can do both.
This is just a very quick overview of these tools – each of them is a lot more powerful than I can get into here. They were all run from the Kali Linux terminal.
Part 1: Docker based tools – these are run as a docker container from the command line.
For the two tools in this section below, hakrawler and gau, require docker to be set up before using. This is as easy as running: ‘sudo apt install docker.io’ (without the quotes).
In the commands below, you can see that there are two docker flags in each: ‘–rm’ is to automatically clean up the container and remove the file system when the container exits; ‘-i’ allows you to send commands to the container via standard input (“STDIN”).
hakrawler
Active Enumeration
Website: https://github.com/hakluke/hakrawler
Commands:
sudo docker run –rm -i hakluke/hakrawler –help
This will bring up the help file.
echo https://www.google.com | sudo docker run –rm -i hakluke/hakrawler -subs
A typical enumeration command for google. ‘-subs’ will include subdomains for crawling.
Notes: This is a really nice tool and works very well. Pros are: that it will find a lot of currently used URLs. Cons are: it will directly interact with the target.
gau
Passive Enumeration
Website: https://github.com/lc/gau
Commands:
sudo docker run –rm sxcurity/gau:latest –help
Brings up the help file
sudo docker run –rm -i sxcurity/gau google.com
A typical enumeration command for google.
Notes: works really well – might be my favorite tool for quickly gathering information passively.
Part 2: Command line based tools – these are run directly from the command line
theharvester
Passive Enumeration
Website: https://www.kali.org/tools/theharvester/ or https://github.com/laramies/theHarvester
Commands:
theHarvester -d indeed.com -l 1000 -b all
theHarvester -d indeed.com -l 1000 -b duckduckgo
‘-d’ is domain; ‘-l’ is limit; and ‘-b’ is source. Source can be ‘all’ which has a list of a couple dozen search engines, but also could be ‘google’, ‘duckduckgo’, or ‘virustotal’. There is a full list on the kali harvester page.
Notes: In addition to subdomain enumeration, this tool can be used for enumerating email addresses and mentions across the web. When running it, I had some issues with some sites because google blocked my IP address – you can get around this by using a different search engine, but this doesn’t always work.
gobuster
Passive Enumeration
Website: https://github.com/OJ/gobuster or https://www.kali.org/tools/gobuster/
Commands:
gobuster dns -q -r 8.8.8.8 -d example.com -w wordlists/Discovery/DNS/subdomains-top1million-5000.txt -t 4 –delay 1s -o results.txt
dns mode, -q –quiet, -r –resolver string, -d –domain string, -w –wordlist string (path to the wordlist), -t –threads, –delay (duration), -o –output string (output file to write results to)
Notes: Gobuster is a tool used to brute-force URIs including directories and files as well as DNS subdomains. For a lot of the enumeration, it will use a word list, external to the program – pros is that it’s easy to update, con is that you need it for gobuster to be most effective.
Good news is that one can be found here: https://github.com/danielmiessler/SecLists/blob/master/Discovery/DNS/subdomains-top1million-5000.txt
recon-ng
Both Passive and Active depending on how it’s used
Website: https://www.kali.org/tools/recon-ng/ or https://github.com/lanmaster53/recon-ng
Commands: This works a bit differently from the tools above. First, in the command line, run ‘recon-ng’ – this will pull up the program. Once in there, to quickly enumerate a domain run the following (without the numbers):
1. modules load hacker target
2. options set source google.com
3. info
4. run
5. show hosts
If you want to choose another source, just replace google.com with the target of your choosing.
Notes: It has the look and feel of Metasploit, so if you’re familiar with that, this will be second nature – if not, there is a learning curve to get it working. Some of the modules may not be installed initially, so they need to be added via the marketplace. This can be done by installing them individually or just running ‘marketplace install all’.
This is a very powerful tool – perhaps the most powerful of all here. It can run scripts, use API keys, interact with databases… it’s incredibly versatile, and subdomain enumeration is an incredibly small subset of what it can do. It’s worth looking more into it at the website above.
While recon-ng requires more in depth knowledge to get going, it can perhaps be the most rewarding.