Subdomain Enumeration

Was just messing around with subdomain enumeration and wanted to give a quick overview of a few tools I find useful: hakrawler, gau, theHarvester, sublist3r, gobuster, and recon-ng.

For the most part, they fall into two categories – passive and active enumeration. Passive enumeration will not interact with the target itself, instead finding information in places like web archives and search engine databases (gau, for example, uses AlienVault’s Open Threat Exchange, the Wayback Machine, Common Crawl, and URLScan). Active enumeration will interact with the target directly (hakrawler, for example, will find the URLs that respond to http(s) and crawl them all). For recon-ng, it can do both.

This is just a very quick overview of these tools – each of them is a lot more powerful than I can get into here. They were all run from the Kali Linux terminal.

Part 1: Docker based tools – these are run as a docker container from the command line.

For the two tools in this section below, hakrawler and gau, require docker to be set up before using. This is as easy as running: ‘sudo apt install docker.io’ (without the quotes). 

In the commands below, you can see that there are two docker flags in each: ‘–rm’ is to automatically clean up the container and remove the file system when the container exits; ‘-i’ allows you to send commands to the container via standard input (“STDIN”).

hakrawler

    Active Enumeration

    Website: https://github.com/hakluke/hakrawler

    Commands: 

        sudo docker run –rm -i hakluke/hakrawler –help

    This will bring up the help file.

        echo https://www.google.com | sudo docker run –rm -i hakluke/hakrawler -subs

            A typical enumeration command for google. ‘-subs’ will include subdomains for crawling.

    Notes: This is a really nice tool and works very well. Pros are: that it will find a lot of currently used URLs. Cons are: it will directly interact with the target.


gau

    Passive Enumeration

    Website: https://github.com/lc/gau

    Commands:

        sudo docker run –rm sxcurity/gau:latest –help

            Brings up the help file

        sudo docker run –rm -i sxcurity/gau google.com

            A typical enumeration command for google.

    Notes: works really well – might be my favorite tool for quickly gathering information passively.


Part 2: Command line based tools – these are run directly from the command line 

theharvester

    Passive Enumeration

    Website: https://www.kali.org/tools/theharvester/ or https://github.com/laramies/theHarvester

    Commands:

        theHarvester -d indeed.com -l 1000 -b all

        theHarvester -d indeed.com -l 1000 -b duckduckgo

            ‘-d’ is domain; ‘-l’ is limit; and ‘-b’ is source. Source can be ‘all’ which has a list of a couple dozen search engines, but also could be ‘google’, ‘duckduckgo’, or ‘virustotal’. There is a full list on the kali harvester page.

    Notes: In addition to subdomain enumeration, this tool can be used for enumerating email addresses and mentions across the web. When running it, I had some issues with some sites because google blocked my IP address – you can get around this by using a different search engine, but this doesn’t always work.


sublist3r

    Passive Enumeration

    Website: https://www.kali.org/tools/sublist3r/
    Commands:
        python3 sublist3r.py -d yahoo.com -b -t 50 -p 80,443,21,22 
        python3 sublist3r -v -d kali.org -t 5 -e bing 
        sublist3r -v -d kali.org -t 5 -e bing
    
            -d is domain, -b enables brute force mode, -t is the number of threads, -p is the ports, and -e is the engines
        
    Notes: This package contains a Python tool designed to enumerate subdomains of websites using OSINT. It helps penetration testers and bug hunters collect and gather subdomains for the domain they are targeting. Sublist3r enumerates subdomains using many search engines such as Google, Yahoo, Bing, Baidu, and Ask. Sublist3r also enumerates subdomains using Netcraft, Virustotal, ThreatCrowd, DNSdumpster, and ReverseDNS.

gobuster

    Passive Enumeration

    Website: https://github.com/OJ/gobuster or https://www.kali.org/tools/gobuster/

    Commands: 

        gobuster dns -q -r 8.8.8.8 -d example.com -w wordlists/Discovery/DNS/subdomains-top1million-5000.txt -t 4 –delay 1s -o results.txt

    dns mode, -q –quiet, -r –resolver string, -d –domain string, -w –wordlist string (path to the wordlist),  -t –threads, –delay (duration),  -o –output string (output file to write results to)

    Notes: Gobuster is a tool used to brute-force URIs including directories and files as well as DNS subdomains. For a lot of the enumeration, it will use a word list, external to the program – pros is that it’s easy to update, con is that you need it for gobuster to be most effective. 

Good news is that one can be found here: https://github.com/danielmiessler/SecLists/blob/master/Discovery/DNS/subdomains-top1million-5000.txt

Part 3: Program based tools – after running the initial command, it will bring up a program

recon-ng 

    Both Passive and Active depending on how it’s used

    Website: https://www.kali.org/tools/recon-ng/ or https://github.com/lanmaster53/recon-ng

    Commands: This works a bit differently from the tools above. First, in the command line, run ‘recon-ng’ – this will pull up the program. Once in there, to quickly enumerate a domain run the following (without the numbers):

    1. modules load hacker target

    2. options set source google.com

    3. info

    4. run

    5. show hosts

    If you want to choose another source, just replace google.com with the target of your choosing.

    Notes:  It has the look and feel of Metasploit, so if you’re familiar with that, this will be second nature – if not, there is a learning curve to get it working. Some of the modules may not be installed initially, so they need to be added via the marketplace. This can be done by installing them individually or just running ‘marketplace install all’.

This is a very powerful tool – perhaps the most powerful of all here. It can run scripts, use API keys, interact with databases… it’s incredibly versatile, and subdomain enumeration is an incredibly small subset of what it can do. It’s worth looking more into it at the website above.

While recon-ng requires more in depth knowledge to get going, it can perhaps be the most rewarding.

Leave a Reply

Your email address will not be published.