Grinder Framework helps overcome Shodan false negatives and blind spots

Casting light into the shadows

A utility that’s designed to overcome the limitations and constraints of security-related search engines such as Shodan was unveiled at Black Hat Asia this week.

The Grinder Framework – developed by security researchers Anton Nikolaev and Denis Kolegov of Bi.Zone – is designed to optimize the usefulness of search tools including Censys and ZoomEye as well as Shodan by minimizing blind spots and false negatives.

Overcoming these drawbacks can save security researchers a great deal of time in projects such as discovering vulnerable hosts and developing fingerprints for vulnerability scanners, among other applications.

INSIGHT Shodan founder John Matherly on IoT security and dual-purpose hacking tools

“The Grinder Framework is an open source security research toolkit adopted to Internet-wide surveys and allows you to use the full power of tools like Nmap, Shodan, Censys, Vulners, and TLS-attacker, and bringing the light through tailored scanning and threat intelligence approach,” the researchers explain in a preview for a presentation for an Arsenal session held during Black Hat Asia today (October 1).

The Grinder Framework automatically collects different hosts on the internet using various systems, including search engines, such as Shodan and Censys, for discovering hosts and retrieving information from them passively, and the Nmap network scanner for fingerprinting and specific active checks.

Aggregated threat intelligence

The framework was created as part of a project to explore software-defined wide-area network (SD-WAN) security on the internet, but has applications far beyond any one particular application.

Kolegov told The Daily Swig: “We started the SD-WAN New Hope project in 2017. The goals were to understand the prevalence of SD-WAN technologies and products on the internet and their security from a practical perspective.

Catch up on more of the latest news from Black Hat Asia 2020

“We created a database of SD-WAN fingerprints and implemented a bunch of Python scripts that initially request Shodan, collect all available information, and render the results in a map.”

SD-WAN allows corporates to optimize their use of network resources in the same way virtualization improves data center management.

Kolegov said: “We quickly realized that this approach can be extended and improved. We found that Shodan has many peculiarities not allowing us to discover SD-WAN hosts effectively. We came up with the following basic idea: let’s create something adopted to internet-wide surveys.”

Put simply, the Grinder framework helps to understand global exposure statistics on different devices that are collected by different search engines.

Nikolaev added: “We desired to provide the possibility of organizing different modules, tools, scripts, and heuristics in one orchestrator-like framework, which can provide different analytical methods easily – with plots, statistics, map, and other visual and statistical information in an easy-to-use form.”

RECOMMENDED BitLocker sleep mode vulnerability can bypass Windows’ full disk encryption