Table of Contents
- Nuclei YAML Syntax Documentation
- Environment & CLI Variables Support
- JSON and XPath Extractors
- Elasticsearch Exporter
- JSONSchema Support for Nuclei Syntax
- More features for HTTP Protocol
- New Global Variables
- Unified Attack mode support
- Stop At First Match
- Metadata attribute
- Scan optimization
- Template validation enhancement
Authors
Nuclei YAML Syntax Documentation
A common problem with YAML syntax is figuring out all the supported fields and their relevant documentation. With Nuclei, this has always been a barrier of entry for people new to the project.
This Automatically Generated Documentation for Nuclei YAML Syntax is a step in this direction, making the Nuclei Syntax easier for people to understand and get working with. The documentation is generated using yamldoc-go, which is based on the amazing work of talos-system people.
A link to the documentation is provided below. It contains all the fields in the syntax, along with descriptions and examples.
The documentation is generated from code on each release, so you can be sure that it will always be up-to-date with the latest engine features.
Environment & CLI Variables Support
This addition allows using values from Environment Variable inside a template with an optional flag -env-vars
. When enabled, all the values in env are available for use in a template.
yaml
1requests:
2 - method: GET
3 path:
4 - "{{BaseURL}}"
5 headers:
6 Content-Type: '{{ENV_TEST}}'
In the above example, the ENV_TEST
string will be replaced with the environment variable value of ENV_TEST.
It has also been made possible to specify these variables from CLI. This can be done using the -var key=value
flag syntax. The above example ENV_TEST
can be replicated from CLI with the -var ENV_TEST=example
flag.
This feature also allows us to create templates that conduct authenticated actions without hardcoding keys or credentials.
JSON and XPath Extractors
Two new extractor types have been added to nuclei - json
and xpath
.
JSON extractor allows using JQ-style syntax to extract items from json responses. An example of using JSON extractor to extract IDs from Gitlab project ids is provided below.
yaml
1extractors:
2 - type: json
3 part: body
4 json:
5 - '.[] | .id'
XPath extractor allows using XPath expressions to extract items from HTML responses. An optional attribute to extract can also be provided.
yaml
1extractors:
2 - type: xpath
3 part: body
4 attribute: href
5 xpath:
6 - "/html/body/div/p[2]/a"
Elasticsearch Exporter
It is now possible to export results directly from nuclei to Elasticsearch. This can be used for result visualization as well as data storage. The below-provided configuration file can be used with nuclei to export data to Elasticsearch.
yaml
1elasticsearch:
2 # IP for elasticsearch instance
3 ip: 127.0.0.1
4 # Port is the port of elasticsearch instance
5 port: 9200
6 # IndexName is the name of the elasticsearch index
7 index-name: nuclei
8 # SSL enables ssl for elasticsearch connection
9 # ssl: false
10 # SSLVerification disables SSL verification for elasticsearch
11 # ssl-verification: false
12 # Username for the elasticsearch instance
13 # username: test
14 # Pasword is the password for elasticsearch instance
15 # password: test
Running nuclei with nuclei -rc config.yaml -t <templates> -l <list>
sends the found results to Elasticsearch, which can now be visualized with Kibana and other solutions.
JSONSchema Support for Nuclei Syntax
An automatically generated json-schema for the Nuclei YAML Syntax has been added to the repository as well. The link is provided below -
This schema can be used as a starting point for Nuclei Intellisense support. A more detailed Blog Post regarding all Editor integrations will come soon; however, until then, you can follow the medium post here to set up Nuclei YAML JSONSchema in your editor. The link provided is for VSCode; however, it should be similar for other editors.
More features for HTTP Protocol
New Global Variables
A list of new variables has been added to Nuclei HTTP Module, generated from the provided URL, and can be used anywhere within the request block. An example is provided below, including new variables -
https://example.com:443/foo/bar.php
Variable | Value |
---|---|
{{BaseURL}} | https://example.com:443/foo/bar.php |
{{RootURL}} | https://example.com:443 |
{{Hostname}} | example.com:443 |
{{Host}} | example.com |
{{Port}} | 443 |
{{Path}} | /foo |
{{File}} | bar.php |
{{Scheme}} | https |
Unified Attack mode support
The nuclei Attack module where the engine tries a combination of payload values and helper functions is now available for use in the Base HTTP request. Earlier, these could only be used in conjunction with Raw HTTP requests.
yaml
1id: basic-http-payload-helpers-example
2
3info:
4 name: Test HTTP Payload Template
5 author: pdteam
6 severity: info
7
8requests:
9 - method: GET
10 path:
11 - "{{BaseURL}}/{{path}}"
12 headers:
13 hello: "{{md5('Hello')}}"
14 payloads:
15 path:
16 - abc
17 - dsa
18 attack: sniper
19
20 matchers:
21 - type: status
22 status:
23 - 200
Stop At First Match
stop-at-first-match
is now supported in the template for HTTP requests. What this means is, if there are multiple requests, nuclei will stop sending requests as soon as it gets a match. This is useful in cases like brute-forcing, etc., where you want to stop after finding the first match. Previously, it was supported by a CLI flag that applied to all templates, and now it can be defined within a template, allowing it to be used in particular cases.
yaml
1id: test-stop-at-first-match
2
3info:
4 name: test-stop-at-first-match
5 author: pdteam
6 severity: info
7
8requests:
9 - method: GET
10 path:
11 - "{{BaseURL}}/php.php"
12 - "{{BaseURL}}/phpinfo.php"
13 - "{{BaseURL}}/info.php"
14 - "{{BaseURL}}/infophp.php"
15 - "{{BaseURL}}/php_info.php"
16 - "{{BaseURL}}/test.php"
17 - "{{BaseURL}}/i.php"
18 - "{{BaseURL}}/asdf.php"
19
20 stop-at-first-match: true
21 matchers:
22 - type: word
23 words:
24 - "PHP Extension"
25 - "PHP Version"
26 condition: and
Metadata attribute
A new info
field has been added called metadata
which can be used to provide extra metadata information in the template. This also means dynamic key: value fields now can be defined within the metadata
block rather than outside of it, and the templates with additional fields in the earlier format will still work but will not be processed in the JSON output.
yaml
1info:
2 name: Name # static field
3 author: test # static field
4 severity: info # static field
5 description: Description # static optional field
6 remediation: Remediation # static optional field
7 reference: https://example.com # static optional field
8 tags: cve,xss # static optional field
9 metadata:
10 os: linux # dynamic optional field
11 vendor: jira # dynamic optional field
Scan optimization
Nuclei now track errors occurring for each Host, and if it exceeds a certain threshold defined by the max-host-error
flag, the host is skipped from the scan. This saves a lot of time by not trying further template scans on unresponsive or continuously failing hosts.
Template validation enhancement
Nuclei engine now has stronger validation for loaded templates and displays the number of templates loaded with the invalid format, including errors and warnings; also, the validate
flag has been upgraded to detect and report the error information to help with template fixing.
Numerous small crashes and issues have been fixed to make the overall experience smoother and better. The codebase has also been refactored to introduce new ideas to make it scalable and maintainable for the future.
GitHub issues closed in this release can be found here, and the complete changelog of the release is available on the GitHub release page.