Table of Contents
Authors
The latest nuclei release has been a pretty big one with lots of new features added as well as bug fixes to existing code base. Significant new additions have been made in fuzzing as well templating capabilities of nuclei.
Non-RFC Compliant Requests
Earlier versions of nuclei used the base Go HTTP library. The requests were strictly validated and non-spec compliant requests were dropped. The new release comes with an unsafe attribute using our rawhttp library which allows sending any kind of malformed requests to detect interesting behaviour and allow unlimited control over the sent requests.
Some examples of the type of requests you can send with nuclei unsafe requests are-
- HTTP Smuggling Requests
- CRLF Requests with Malformed Characters
- Custom Host Header Requests
- Invalid Request Formats, non-standard HTTP Methods, etc.
These examples are just scratching the surface of what’s possible with rawhttp, think complete control over the requests!
HTTP Smuggling
HTTP Smuggling is a class of Web-Attacks recently made popular by Portswigger’s Research into the topic. For an in-depth overview, please visit the article linked above.
In the open source space, detecting http smuggling is difficult particularly due to the requests for detection being malformed by nature. Nuclei is able to reliably detect HTTP Smuggling vulnerabilities utilising the rawhttp engine.
The most basic example of a HTTP Smuggling vulnerability is CL.TE Smuggling. An example template to detect a CE.TL HTTP Smuggling vulnerability is provided below using the unsafe: true attribute for rawhttp based requests.
yaml
1id: CL.TE-http-smuggling
2
3info:
4 name: HTTP request smuggling, basic CL.TE vulnerability
5 author: pdteam
6 severity: info
7 lab: https://portswigger.net/web-security/request-smuggling/lab-basic-cl-te
8
9requests:
10 - raw:
11 - |
12 POST / HTTP/1.1
13 Host: {{Hostname}}
14 Connection: keep-alive
15 Content-Type: application/x-www-form-urlencoded
16 Content-Length: 6
17 Transfer-Encoding: chunked
18
19 0
20
21 G
22 - |
23 POST / HTTP/1.1
24 Host: {{Hostname}}
25 Connection: keep-alive
26 Content-Type: application/x-www-form-urlencoded
27 Content-Length: 6
28 Transfer-Encoding: chunked
29
30 0
31
32 G
33
34 unsafe: true
35 matchers:
36 - type: word
37 words:
38 - 'Unrecognized method GPOST'More examples are available in nuclei-docs for smuggling templates.
Other examples
Another example is Host header attacks. These can now be detected very reliably using nuclei with the new rawhttp addition. The example below shows a template for detecting a host header based SSRF from portswigger labs.
yaml
1id: host-header-ssrf
2
3info:
4 name: Flawed Request Parsing Host Header SSRF
5 author: pdteam
6 severity: info
7
8requests:
9 - raw:
10 - |
11 GET https://your-lab-id.web-security-academy.net/
12 Host: your-collaborator-id.burpcollaborator.net
13 unsafe: true
14...Race Condition
Race Conditions are another class of bugs not easily automated via traditional tooling. Burp Suite introduced a Gate mechanism to Turbo Intruder where all the bytes for all the requests are sent expect the last one at once which is only sent together for all requests synchronising the send event.
Below is an example template where the same request is repeated for 15 times using the gate logic.
yaml
1id: race-condition-testing
2
3info:
4 name: Race condition testing
5 author: pdteam
6 severity: info
7
8requests:
9 - raw:
10 - |
11 POST /coupons HTTP/1.1
12 Host: {{Hostname}}
13 Pragma: no-cache
14 Cache-Control: no-cache, no-transform
15 User-Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:47.0) Gecko/20100101 Firefox/47.0
16
17 promo_code=20OFF
18
19 race: true
20 race_count: 15
21
22 matchers:
23 - type: status
24 part: header
25 status:
26 - 200Now test for race conditions in web applications with as simple as this template.
Advanced Fuzzing Support
We’ve enriched nuclei to allow advanced fuzzing of web servers. Users can now use multiple options to tune HTTP fuzzing workflows.
HTTP Pipelining
HTTP Pipelining support has been added which allows multiple HTTP requests to be sent on the same connection inspired from [https://portswigger.net/research/http-desync-attacks-request-smuggling-reborn.
An example template demonstrating pipelining capabilities of nuclei has been provided below-
yaml
1id: pipeline-testing
2info:
3 name: pipeline testing
4 author: pdteam
5 severity: info
6
7requests:
8
9 - payloads:
10 path: path_wordlist.txt
11
12 attack: sniper
13 unsafe: true
14 pipeline: true
15 pipeline-max-connections: 40
16 pipeline-max-workers: 25000
17
18 raw:
19 - |
20 GET /§path§ HTTP/1.1
21 Host: {{Hostname}}
22 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:79.0) Gecko/20100101 Firefox/79.0
23 Accept: application/json, text/plain, */*
24 Accept-Language: en-US,en;q=0.5
25 Referer: {{BaseURL}}
26 Connection: keep-alive
27
28 matchers:
29 - type: status
30 part: header
31 status:
32 - 200HTTP Connection Pooling
While the earlier versions of nuclei did not do connection pooling, users can now configure templates to either use HTTP connection pooling or not. This allows for faster scanning based on requirement. An example template for this new capability-
yaml
1id: fuzzing-example
2info:
3 name: Connection pooling example
4 author: pdteam
5 severity: info
6
7requests:
8 - payloads:
9 password: password.txt
10
11 threads: 40
12 attack: sniper
13
14 raw:
15 - |
16 raw:
17 - |
18 GET /protected HTTP/1.1
19 Host: {{Hostname}}
20 Authorization: Basic {{base64('admin:§password§')}}
21 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0
22 Accept-Language: en-US,en;q=0.9
23 Connection: close
24
25 matchers-condition: and
26 matchers:
27 - type: status
28 status:
29 - 200
30
31 - type: word
32 words:
33 - "Unique string"
34 part: bodyNote:- Do not add ‘Connection: Close’ header when using the connection pooling template.
Simplified Workflow Syntax
We use tengo as a scripting engine for nuclei workflows which provided a very powerful and highly customizable engine for users to automate all their needs. However creating a workflows using tengo scripting syntax was something we always wanted to update for better user experience and to align with simple YAML format just like templates.
Workflows have been reworked to adopt simpler YAML based syntax which make creating complex workflows a breeze. Older workflows are still supported and will run flawlessly.
An example template which runs exploits if Spring Web Framework is detected using the new workflow syntax is provided below.
yaml
1workflows:
2 - template: security-misconfiguration/springboot-detect.yaml
3 subtemplates:
4 - template: cves/CVE-2018-1271.yaml
5 - template: cves/CVE-2018-1271.yaml
6 - template: cves/CVE-2020-5410.yamlMatcher names can also be checked and multiple conditions can be executed. Chained templates are also possible by specifying under the template block.
yaml
1workflows:
2- template: technologies/tech-detect.yaml
3 matchers:
4 - name: lotus-domino
5 subtemplates:
6 - template: technologies/lotus-domino-version.yaml
7 subtemplates:
8 - template: cves/CVE-2005-2428.yamlBurp Collaborator Support
We’ve also added support for Burp Collaborator based polling for Out-Of-Band and blind security testing. This allows you to create templates that report interactions based on DNS or HTTP events.
yaml
1id: collab-automation
2info:
3 name: Collab automation with nuclei
4 author: pdteam
5 severity: info
6
7requests:
8 - raw:
9 - |
10 POST /api/v1/proxy HTTP/1.1
11 Host: {{Hostname}}
12 Connection: close
13 Content-Length: 549
14 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) Safari/537.36
15 Content-Type: application/json
16
17 {'url':'http://<collab-id>.burpcollaborator.net/'}
18
19 matchers:
20 - type: dsl
21 dsl:
22 - "waitfor(6) && collab('')"Example of running template with collaborator:-
sh
1echo https://vul-target.com | nuclei -t collab-automation.yaml -burp-collaborator-biid xxxxxxxBy default nuclei polls every 5 seconds and keeps track of last 150 collaborator events. Provide it with a Burp Suite Collaborator BIID and you’re good to go!
Miscellaneous
Project Support
We’ve added Project File support which can be used to cache previous requests to targets as well as use the stored request later for re-verification purposes.
When project flag is used it will store the current scan requests in a temporary cache on Disk. These requests can be used later by the templates making the request to same paths. With the project-path path flag, the requests can also be stored to a custom directory.
Basic stats support
We’ve replaced the progress bar with a simpler stats line which is printed every 5 seconds. This was done to simplify the progress bar implementation causing issues on several platforms.
What’s Next?
In the next release, Network Protocol Requests like raw TCP, UDP, etc and Local Directory / Files will be supported.
The complete change-log of this release is available here.
Future of the project
In future, we’ll be adding the following new features to the project. Follow us to keep in touch with the progress.
- A new documentation site for easy access to templating guide and docs.
- Notification module to send alerts on identified bugs.
- UI / Web Form to create a template with just a few clicks.
- Test Server to validate nuclei templates at runtime.
Questions / Feedback
If you’re already a user of nuclei and would like to suggest some feature or share some ideas, feel free to reach out. You can contact/tweet us on twitter @pdnuclei/ @pdiscoveryio/ contact@projectdiscovery.io. We’d love to hear from you.
You can follow the Nuclei and Nuclei templates project on Github. Contributions of new templates as well as ideas are very welcome!