Blog
-

9 min read

Community-powered scanning with Nuclei

Nuclei & Templates
Community-powered scanning with Nuclei

Share

Nuclei is a new breed of scanners, that moves away from the traditional model of vulnerability scanners allowing complete extensibility with a very simple and easy to use templating syntax.

Introduction

At its basic level, Nuclei is a scanner - it sends HTTP/DNS requests and checks for some response to discover anomalies in the target. It works on top of YAML files allowing the users to define the behaviour they are looking for in a readable format.

Users can easily define Matchers which check the response for various types of patterns such as a word or a regex, Extractors which extracts some content from the response using a regex. Besides the powerful matching and extracting features, users can also completely customize the requests being sent, allowing full control over each and every part of the request/response process.

How is it different?

Instead of bundling the scanner with vulnerability checks, Nuclei approach is to focus on the core engine, leaving the task of creating vulnerability checks to the end user.

The core is designed from the start to be very extensible, allowing the user to customize everything as needed. Following are the core goals of the Nuclei project -

  1. Simplicity - Nuclei templates are defined in a straightforward to read YAML format, which allows Non-Developers to also easily create templates without going through the hassle of writing lengthy code and debugging it to make things work.
  2. Flexibility- Templates can be customized as the user pleases. There are very few things that can’t be done with the templates itself. Every part of the request is customizable as needed and works flawlessly.
  3. Portability - One of the core goals was to allow the users to share templates with each other. Each template is a single file, which can be run independently without any prerequisites. All you need is a nuclei installation. In this way, nuclei concept increases community sharing.

Features

Nuclei Templates

nuclei-templates is the main focus of the nuclei project. It is a community built collection of templates available for the nuclei scanner. These templates have been contributed by nuclei users and are carefully curated to weed out false-positives, malicious code, etc, which have been a problem with scanners for some time.

Currently, 46 people have contributed to the nuclei-templates with 130+ templates in total, some of which have been 1-Days published by the wonderful community.

Nuclei has integrated support for downloading and managing a local copy of the nuclei-templates repository. Users can simply run nuclei -update-templates to download the repository locally. In case an update is published, user is notified and can update their copy simply by running the same command again.

The templates can be ran without specifying the full path of the templates. If the user wishes to run the cves/ folder of nuclei-templates repositories, he can simply run the command nuclei -t cves/. Nuclei will first look in the current directory for the folder, and if the templates are installed locally, it will automatically pick up the cves/ directory from the local templates installation.

This makes it easy to run scans without having to remember the directory or doing manual git pull every time to fetch the latest changes.

HTTP & DNS Requests Support

HTTP and DNS protocols are supported by the scanner. HTTP requests can be built either by using declarative YAML language or by using the raw request functionality for HTTP requests. DNS requests can only be defined by using YAML language.

Raw requests allow infinite capabilities for customization. Requests can be copied/pasted directly from Burp Suite.

All the below are valid requests -

yaml

1
dns:
2
  - name: "{{FQDN}}"
3
    type: CNAME
4
    class: inet
5
    recursion: true
6
    retries: 3
7
    matchers:
8
      - type: word
9
        words:
10
          - "IN\tCNAME"
11
          
12
requests:
13
  - method: GET
14
    path:
15
      - "{{BaseURL}}/test.txt"
16
    matchers:
17
      - type: word
18
        words:
19
          - "This is test!"

For more information on the request syntax, checkout the documentation for the templates here.

Powerful Matching/Extracting Capabilities

Matchers can allow checking if some string or pattern was found in the response. Extractors allow the user to extract any piece of relevant information from the response as desired by the user.

Matchers can be of 5 types - size, word, regex, binary, dsl. word allows matching strings, size allows matching integers, regex allows matching a Regular Expression on the body, binary allows matching hexadecimal patterns in the response. dsl is the most powerful matcher which provides a unique expression language to allow the user to match as desired with multiple conditions, etc.

yaml

1
dsl:
2
      - "len(body)<1024 && status_code==200" # Body length less than 1024 and 200 status code
3
      - "contains(toupper(body), md5(cookie))" # Check if the MD5 sum of cookies is contained in the uppercase body

For most use cases, a word matcher or a regex matcher should suffice.

Currently, extractors are only present for Regular Expressions. Users can define a regex, which will be matched, and the found matches will be returned as results. This has allowed some really creative templates, including a template to search HTTP responses for Slack API Keys as well as a template to look for HTTP URLs with username and passsword in them.

Multiple matchers and extractors can be chained together with different conditions to achieve precise results.

Workflow support for conditional scanning

A recently added feature is the support of workflows which are conditions and order in which to execute requests to make the process more precise. The workflow condition support is provided by using tengo project which is a small portable language written Go. A workflow has two parts, variables and logic. Variables are locations to a template/templates which will be executed. The second part is Logic which defines how the variables should be run.

yaml

1
variables:
2
  detection_script: detect.yaml
3
  exploit: exploit.yaml
4
  exploit_directory: /root/templates/exploits/ 
5
logic:
6
    |
7
  if detection_script() {
8
    exploit()
9
    exploit_directory()
10
  }

Using this syntax, users can achieve precise control over the scanning process without having to deal with false positives or slow scans.

Use Cases

Given below are some examples use-cases for which you can easily write nuclei templates.

CVE Identification

Let’s say a new CVE is announced. We’ll take the example of the very famous CVE-2017-9506 which is a full SSRF in jira. A successful exploitation of the CVE returns the content of the target URL provided on the page. The following payload detects the exploit -

bash

1
{{BaseURL}}/plugins/servlet/oauth/users/icon-uri?consumerUri={{url_to_fetch}}

To identify this vulnerability, all we need to do is make a request to the following URL and find a static page that returns some unique string that can signal successful finding. We can use a hack and try to request https://ipinfo.io/json which always returns a fixed word ipinfo.io/missingauth in the response. Utilising this piece of info, we can simply use the following template.

yaml

1
id: CVE-2017-9506
2


3
info:
4
  name: Jira IconURIServlet SSRF
5
  author: Ice3man
6
  severity: high
7


8
requests:
9
  - method: GET
10
    path:
11
      - "{{BaseURL}}/plugins/servlet/oauth/users/icon-uri?consumerUri=https://ipinfo.io/json"
12
    matchers:
13
      - type: word
14
        words:
15
          - "ipinfo.io/missingauth"
16
        part: body

It requests the following URL and checks if the response contains the word we mentioned before. Simple as that!

Technology Detection

Technology fingerprinting is one of the most important part of any workflow. Gaining an idea of the stuff running on target can allow a hacker to decide how he wants to proceed with the target.

Nuclei makes it very easy to write templates to fingerprint the applications running on the target. Technology specific templates can be created to detect an applications, which can also allow you to go as far as to detect the versions installed too. Just find a fixed string present on the target application of your choice and/or a path that’s only present in that application and create a template.

For example, we know that Jira installations contain a path called /secure/Dashboard.jspa and the response to that path contains a fixed string - Project Management Software. Utilising these two pieces of information, we can write a template to detect running Jira servers as follows -

yaml

1
id: jira-detect
2


3
info:
4
  name: Detect Jira Issue Management Software
5
  author: bauthard
6
  severity: informative
7


8
requests:
9
  - method: GET
10
    path:
11
      - "{{BaseURL}}/secure/Dashboard.jspa"
12
      - "{{BaseURL}}/jira/secure/Dashboard.jspa"
13
    matchers:
14
      - type: word
15
        part: body
16
        words:
17
          - "Project Management Software"

Similarly, Jenkins Server has a header present always in the response called x-jenkins. Utilizing this information, we can also write a template to detect Jenkins servers very easily by searching the header part for the word.

yaml

1
id: jenkins-headers-detect
2


3
info:
4
  name: Jenkins Headers Based Detection
5
  author: ice3man
6
  severity: informative
7


8
requests:
9
  - method: GET
10
    path:
11
      - "{{BaseURL}}/"
12
    matchers:
13
      - type: word
14
        words:
15
          - "X-Jenkins"
16
        part: header

The same technique can be applied to create templates to detect any running piece of application you desire. All you need is a unique pattern present somewhere in the application, and you can detect it effortlessly.

Stack specific exploitation using workflows

The last two parts of the blog showed you how to write templates to detect applications and how to exploit vulnerabilities in them. This next section is about chaining both those together and achieving highly targeted exploitation using the Workflow feature of nuclei.

The idea here is to run Jira-specific exploitation templates like the CVE-2017-9506 only when a jira installation is detected and not on all URLs. This is where workflows are helpful. You can define a workflow to check for a Jira installation first and check for exploits only when the installation is detected.

yaml

1
id: jira-workflow
2


3
info:
4
  name: Jira Exploitation Workflow
5
  author: ice3man
6


7
variables:
8
  jira_detection: jira-detect.yaml
9
  cve_1: CVE-2017-9506.yaml # CVE-2017-9506
10
  cve_2: CVE-2018-20824.yaml # CVE-2018-2082
11
  jira_exploits: /root/templates/jira-exploits/ # Folder with jira exploits
12
logic:
13
    |
14
  if jira_detection() {
15
    cve_1()
16
    cve_2()
17
    jira_exploits()
18
  }

This template will run the exploits for jira only when the jira-detect.yaml file returns any results. This allows users to define highly specific scan flows which are targeted to each application stack.

DNS & HTTP Misconfiguration Detection

DNS Requests can also be sent, and the responses can be similarly checked for anomalies.

Azure Domains have an NXDOMAIN response if they have a CNAME of cloudapp.azure.com, cloudapp.net, etc and the DNS status code is NXDOMAIN. A nuclei template can be easily created to automate the detection of this flaw.

yaml

1
id: azure-takeover-detection
2


3
info:
4
  name: Azure takeover detection
5
  author: "pdnuclei - projectdiscovery.io"
6
  severity: high
7


8
dns:
9
  - name: "{{FQDN}}"
10
    type: CNAME
11
    class: inet
12
    recursion: true
13
    retries: 3
14
    matchers-condition: and
15
    matchers:
16
      - type: word
17
        words:
18
          - "cloudapp.azure.com"
19
          - "cloudapp.net" # More CNAMES can be added
20
      - type: word
21
        words:
22
          - "NXDOMAIN"

Similarly, common HTTP related vulnerabilities can also be automated in a simple manner. As an example, say the user wants to detect if the value of X-Forwarded-Host header is reflected in the page. A basic template can be created which sends this header and looks for reflection of the header value in the page.

yaml

1
id: host-header-injection
2


3
info:
4
  name: Host Header Injection (x-forwarded-host)
5
  author: melbadry9
6
  severity: low
7


8
requests:
9
  - method: GET
10
    headers:
11
      X-Forwarded-Host: "0021e78f48fe6525798294b7711c6f72.com"
12
    path:
13
      - "{{BaseURL}}/"
14
    matchers:
15
      - type: word
16
        words:
17
          - "0021e78f48fe6525798294b7711c6f72"

The above templates are very simple examples of what is possible with the power of templating in nuclei.

Takeover Detection

Subdomain Takeovers are one of the most common vulnerabilities found now-days with the increase in number of SaaS providers. Using nuclei, you can easily automate the detection of dangling webpages which may have been left behind by the developers.

Nuclei-Templates repository has a complete list of common subdomain takeover fingerprints built by the community here.

Creating a new check is as easy as detecting some unique string present on the vulnerable page and checking for it in the response. As an example, AWS S3 responds with the string The specified bucket does not exists on pages that are vulnerable. Using this bit of information, the detection can be easily automated using the below template.

yaml

1
id: detect-aws-s3-takeover
2


3
info:
4
  name: Subdomain takeover finder
5
  author: "melbadry9 & pxmme1337"
6
  severity: high
7


8
requests:
9
  - method: GET
10
    path:
11
      - "{{BaseURL}}/"
12
    matchers-condition: or
13


14
    matchers:
15
      - type: word
16
        name: aws-s3-bucket
17
        words:
18
          - "The specified bucket does not exist"

Bruteforcing HTTP Basic Authentication

Now we will look at a very interesting and powerful nuclei template for bruteforcing Tomcat HTTP basic authentication using raw requests and Burp Intruder like fuzzing capabilities.

Let’s say you have a file with Tomcat usernames called username.txt and a file with passwords called passwords.txt. There are 3 attack types available - sniper, clusterbomb and pitchfork. The names should be familiar to Burp Suite users from where they are inspired.

For HTTP Basic Authentication bruteforce, we want to send each username with all the passwords in the passwords.txt file, so the required attack type will be clusterbomb which will combine both in the desired pattern.

Using a raw request, the following can be sent in the request Authorization: Basic {{base64(username:password)}} which is a template syntax. What it does is combines username and password variables with a colon and Base64 encodes it. More details on these functions is available in the documentation.

The below script is a finished version of the Tomcat HTTP Basic Authentication bruteforce it.

yaml

1
id: http-raw-bruteforce
2
info:
3
  name: HTTP Basic authentication Bruteforce
4
  author: @pdteam
5


6
requests:
7
  - payloads:
8
      username: username.txt
9
      password: password.txt
10
    attack: clusterbomb 
11


12
    raw:
13
      - |
14
        GET /manager/html HTTP/1.1
15
        Host: {{Hostname}}
16
        Authorization: Basic {{base64(username:password)}}  
17
        Accept-Encoding: gzip, deflate        
18
        Accept-Language: en-US,en;q=0.9
19
        Connection: close        
20


21
    matchers:
22
      - type: status
23
        status:
24
          - 200

Integration In Workflows

Post Recon Fingerprinting and Exploitation

Nuclei is a tool primarily intended for Bug-Bounty hunters as well as Infosec people to Fingerprint as well as detect knows vulnerabilities in applications after the reconnaissance stage.

The scans may take some time depending on the scope; hence it’s best recommended keeping the scans running in the background while hacking manually on the target.

CI/CD Tests For Deployed Applications

An interesting idea we’ve come across from the community is deploying nuclei for scanning internal applications using known vulnerability templates to eliminate threats that may lead to exploitation of the company. This approach can be very helpful for DevOps teams, leading to identification of bugs as soon as they are introduced.

Future of the project

In the future, we’ll be adding the following new features to the project. Follow us to keep in touch with the progress.

  1. UI / Web Form to create a template with just a few clicks.
  2. A new documentation site for easy access to templating guide and docs.
  3. Test Server to validate nuclei templates at runtime.
  4. Notification module to send alerts on identified bugs.

Questions / Feedback

If you’re already a user of nuclei and would like to suggest some features or share some ideas, feel free to reach out. You can contact/tweet us on twitter @pdnuclei or @pdiscoveryio. You can also email us at contact@projectdiscovery.io. We’d love to hear from you.

You can follow the project on github https://github.com/projectdiscovery/nuclei as well as the related templates repository at https://github.com/projectdiscovery/nuclei-templates. Contributions of new templates as well as ideas are very welcome!