‍Hacking Hidden Gems: Content Discovery with Webseek

4 months ago 42

BOOK THIS SPACE FOR AD

ARTICLE AD

By Tahir Mujawar, Certified Ethical Hacker & Cyber Security Researcher

Hello everyone 👋, I’m back with another intriguing and unique story. I’m on a mission to write bash scripts for every task that requires automation in bug bounty hunting.

As many of you know, I’ve already completed work on subdomain enumeration, so I decided to move forward with content discovery. To tackle this, I began researching the methodologies used by various security researchers and bug bounty hunters for content discovery.

Content Discovery with webseek by Tahir Mujawar

After studying and observing a wealth of resources available online, I came up with the idea to create my own content discovery script, WebSeek. The most challenging part was deciding on the methodologies, tools, and workflows to adopt. Eventually, I outlined certain key points to clarify my requirements.

WebSeek is a content discovery script designed to find urls and parameters, perform JavaScript file enumeration, extract endpoints using gf patterns for well-known vulnerabilities, and utilize unfurl to extract paths, key pairs, keys, and parameter values.

Features

Comprehensive URL Discovery : Aggregates URLs from various tools including gospider, waybackurls, hakrawler, gau, and katana.Parameter Discovery : Extracts and identifies URL parameters using tools like ParamSpider, Parameth, and roboxtractor.JavaScript File Enumeration : Enumerates JavaScript files using LinkFinder, subjs, and katana.Unfurl URL Parsing : Decomposes URLs into paths, keys, values, and JSON data for detailed analysis.Sensitive Pattern Detection : Searches for sensitive patterns like passwords, admin panels, and email addresses within URLs.Vulnerability Pattern Search : Uses gf to detect common vulnerability patterns such as XSS, SQLi, SSRF, and more.Automated Directory Creation : Automatically sets up directories for organizing scan results.Customizable Output Directory : Allows specification of a custom output directory for organizing results.Parallel Processing : Scans multiple domains or subdomains in parallel for efficiency.Logging : Maintains logs of scanning activities and errors for troubleshooting and record-keeping.

Now, let’s dive into how to configure and use WebSeek 👨🏻‍💻

Prerequisites

Ensure the following tools are installed and accessible in your system’s PATH -

gospider - https://github.com/jaeles-project/gospider
waybackurls - https://github.com/tomnomnom/waybackurls
hakrawler - https://github.com/hakluke/hakrawler
gau (GetAllUrls) - https://github.com/lc/gau
waymore - https://github.com/xnl-h4ck3r/waymore
katana - https://github.com/projectdiscovery/katana
paramspider - https://github.com/devanshbatham/ParamSpider
parameth - https://github.com/maK-/parameth
roboxtractor - https://github.com/yoeelingbin/roboxtractor
github-endpoints - https://github.com/gwen001/github-search
subjs - https://github.com/lc/subjs
LinkFinder - https://github.com/GerbenJavado/LinkFinder
uro - https://github.com/s0md3v/uro
unfurl - https://github.com/tomnomnom/unfurl
gf (grep patterns) - https://github.com/tomnomnom/gf
mantra - https://github.com/0xPayne/Mantra
secretfinder - https://github.com/m4ll0k/SecretFinder

Installation 👨🏻‍💻

First, clone the repository from my GitHub profile.

Change directory by hitting command

cd webseek

Then, make the webseek.sh script executable by running the command

chmod +x webseek.sh

Note: You must have to configure Github Tokens in token.txt file.

You will need to configure your GitHub tokens in a file named token.txt using the following format: GITHUB_TOKEN=GITHUB_TOKEN_HERE and so on for github-endpoint discovery.Additionally, if you have your own parameter wordlist, you can modify medium.txt accordingly.

webseek by Tahir Mujawar

To run the script for single domain, use the following command:

./webseek.sh -d https://example.com -o output_directory

Ensure that the URL specified with the -d option includes https://

To run the script for subdomain list, use the following command:

./webseek.sh -sL subdomains.txt -o output_directory

That’s all from today, Researching takes considerable time, so if you found this helpful, please give it a like, share it with others, and follow for more updates. Your support is greatly appreciated and keeps the cyber journey strong!

Don’t hesitate to connect with me on ...

👨🏻‍💻 Github 🔗 LinkedIn 🐦 Twitter 🅾 thatfellowhacker

webseek

Happy hacking! Bye bye, hackers 👋

Read Entire Article