Compare commits

...
149 Commits
Author SHA1 Message Date
Mzack9999 40d08ee493 Merge pull request #7980 from kamilikamil/request-body-budget
Bound request body memory held by concurrent HTTP requests
2026-11-07 19:27:15 +01:00
Mzack9999 27ff6b19b7 assign context 2026-11-07 18:55:07 +01:00
Mzack9999 361f3b1383 merge dev 2026-11-07 18:49:32 +01:00
Muhammad Munib NawazandMzack9999 8cda906559 feat(js): add MQTT JavaScript library (#8075)
* feat(js): add mqtt javascript library

* fix(mqtt): address CodeRabbit feedback on security, limits, and robustness

* fix(mqtt): handle in-flight publish in ping, password without username, and staticcheck

* feat(mqtt): add InsecureSkipVerify option to ConnectOptions

* fix subscribe

* fix lint

* require tls12

---------

Co-authored-by: Mzack9999 <[email protected]>
2026-11-07 00:12:10 +01:00
KVR VishnuandMzack9999 a2e403a1d8 fix(core): avoid nested workflow deadlock at template concurrency limit (#8076)
* fix(core): avoid nested workflow deadlock at template concurrency limit

Release a workflow step's AdaptiveWaitGroup slot before acquiring slots
for its children. Matching nested parents previously held a slot while
blocked on AddWithContext, so when their count reached TemplateThreads
the workflow hung forever.

Fixes #8074

* fix workflow

---------

Co-authored-by: Mzack9999 <[email protected]>
2026-11-06 23:44:28 +01:00
Mzack9999 bbebeba8b2 Merge pull request #7974 from TassioSales/fix/json-output-nested-metadata-maps
fix(output): sanitize nested metadata maps before JSON encoding (#7971)
2026-11-06 17:56:10 +01:00
Mzack9999 578d00829a Merge pull request #8071 from projectdiscovery/dwisiswant0/fix/js/make-sshclient-close-a-no-op-without-a-connection
fix(js): isolate JS objects and handle missing connections
2026-11-06 17:56:03 +01:00
Mzack9999 3be47b098f Merge pull request #8072 from AdallomRoy/dev
Upgrade github.com/alecthomas/chroma to latest
2026-11-06 17:34:41 +01:00
Mzack9999 b29ecc6907 Merge pull request #8050 from antcybersec/fix/matcher-status-keeps-response
fix(http): keep the response in -ms failure events
2026-11-06 17:34:33 +01:00
Mzack9999 29559cbd2a Merge pull request #8052 from antcybersec/fix/header-values-keep-hash
fix(cmd): keep "#" in -H header values
2026-11-06 17:34:26 +01:00
Roy Reznik 94f7148b7e Upgrade github.com/alecthomas/chroma to latest 2026-11-06 11:47:53 +00:00
Dwi Siswanto 1829fca65c fix(js): make NetConn safe to use without a connection
NetConn had the same flaw as SSHClient. new net.NetConn() creates an
object with no connection, and every method dereferenced it, so Close,
Send and Recv all aborted the template with a nil pointer panic. Close
also kept the closed connection.

Close now does nothing when no connection is open, and clears the
connection after closing it. The send and receive methods return an
error that says to open the connection with net.Open or net.OpenTLS,
which goja throws as a JS exception the template can catch.

Signed-off-by: Dwi Siswanto <[email protected]>
2026-11-06 09:40:26 +07:00
Dwi Siswanto accfe3551b fix(js): make SSHClient.Close a no-op without a connection
Calling Close on an SSHClient that never connected, or whose Connect
failed, dereferenced a nil *ssh.Client. goja turned the panic into a
request error, so the template stopped at Close and returned nothing.

Close also kept the closed client. On a client reused across login
attempts, a later Close then failed with "use of closed network
connection", and Run reported a transport error instead of "no
connection".

Close now returns true when no connection is open, and clears the
connection after closing it. It returns true rather than an error
because goja throws a returned error as a JS exception, which would
still abort the template.

Fixes: #8070
Signed-off-by: Dwi Siswanto <[email protected]>
2026-11-06 09:40:26 +07:00
Dwi Siswanto d6ec8407fb fix(js): stop JS class constructors sharing one object
GetClassConstructor handed the same Go value to every object it
constructed, in every runtime. All generated JS classes use it, so
every new ssh.SSHClient() in a scan was one client: one script's
Connect or Close changed the connection that scripts on other targets
were using, and those writes raced.

Each constructed object now starts as a shallow copy of the instance
passed in. The generated bindings all pass a zero value, so every new
object starts empty.

Signed-off-by: Dwi Siswanto <[email protected]>
2026-11-06 09:05:43 +07:00
Mzack9999 bb2da9bbea Use only template values for postgres conns (#7985)
* isolate credentials

* pin options
2026-11-06 03:45:59 +07:00
Mzack9999 6e2b59827c keep template screenshots in a nuclei dir (#8068)
* screenshot limits

* named path

* close comment
2026-11-06 03:30:17 +07:00
dw1 875b229313 fix(http): read annotations only before the request line (#8062)
Raw requests applied @Host, @tls-sni and @timeout from any matching
line of the rendered request, headers and body included. A response
value reused in a later request could carry an "@Host:" line and send
that request, with its body, to a host the template does not name.

The raw parser and the docs already define annotations as the "@"
lines before the request line. raw.SplitAnnotations now holds that
rule once, and the parser, the request annotations, @once and the
compile-time @timeout check all read from it.

Fixes: #8061

Signed-off-by: Dwi Siswanto <[email protected]>
2026-11-06 03:29:42 +07:00
Mzack9999 91f8bd1e4f send rdap requests through the scan dialer (#8066)
* rdap dialer

* assert redirect

* ipv6 literal
2026-11-06 03:29:04 +07:00
dw1 fb0eed5122 fix(http): flag truncated responses at the effective read limit (#8046)
The "(Truncated)" marker on dumped responses rebuilt the read-limit
rule from the template max-size and -rsr instead of using the limit
the body was read with. With neither set, the body is cut at the
10 MB default, but the dump was not marked truncated.

Comparing the body length with the effective limit keeps one rule for
both the read and the marker. c918d010c already made -rsr a ceiling
over a smaller template max-size; no release contains it yet, which
is why v3.8.0 still reads the full -rsr limit per response.

Fixes: #8045
Reported-by: @Lidar-Baruch

Signed-off-by: Dwi Siswanto <[email protected]>
2026-11-05 18:07:29 +07:00
Mzack9999 29833da25a send template doh queries through the scan dialer (#8064) 2026-11-05 18:06:41 +07:00
dw1 9b519d2264 fix(http): render self-contained raw requests once (#8059)
A self-contained raw request was rendered twice: once to read the URL
from its request line, and again with the variables that URL provides,
such as {{Hostname}}. The second pass evaluated values that the first
pass had inserted. A response body of {{secret}} reused in a later
request was sent as the template variable's value, and with -ev a
response body naming an environment variable was sent as its value.

Render the request line first and check its host. Then evaluate again
only the template variables and payloads whose template text names a
URL variable, and render the annotations and the rest of the request
once. Response data is now sent unchanged, as in templates that are
not self-contained. A value that changes which line is the request
line is rejected, so the host checked against the network policy is
the host that is sent.

Fixes: #8058

Signed-off-by: Dwi Siswanto <[email protected]>
2026-11-05 18:00:36 +07:00
dw1 4fed40f75a fix(fuzz): stop re-rendering evaluated template variables (#8057)
Fuzz rules rendered the template's variables once against request
data, then passed the output to the Interactsh render pass as template
text. A variable such as b: "{{body}}" therefore had the request body
substituted into it and was rendered a second time, so DSL-shaped bytes
in the incoming request were evaluated and the result was sent to the
target. With payloads, the same leak depended on map iteration order,
because that pass rendered each value against the map it was writing
to.

Treat rendered variable output as data, as the HTTP request builder
does, so only generator payload definitions remain template text.
Without payloads nothing is left to render, so the second pass is
dropped. Payload values now render against the unrendered input, which
makes the result independent of map iteration order. A variable that
references a payload now gets the payload's unrendered definition; it
only saw the rendered value through the second pass.

Fixes: #8054

Signed-off-by: Dwi Siswanto <[email protected]>
2026-11-05 18:00:12 +07:00
dw1 2f4ac3d9e5 fix(expressions): evaluate template expressions in place (#8056)
Evaluate substituted every placeholder into the whole text first, then
wrote each expression result over the first matching marker it found.
When a substituted value repeated an expression's text, such as a
response body of {{to_upper(seed)}}, the result went into that value
and the template's own expression was left unevaluated.

Replace had the same flaw in another form: its § pass ran over the
output of its {{}} pass, so a substituted value of §name§ resolved to
the value of name, and a value could pull in any other variable.

Both now make one left-to-right pass over the template text. Evaluate
locates each expression before anything is substituted and writes the
result at that position, and Replace never scans a value it inserted.
Replace also stops converting every value up front, so rendering is
faster and allocates less.

Fixes: #8053
Fixes: #8055
Refs: #7221

Signed-off-by: Dwi Siswanto <[email protected]>
2026-11-05 17:59:43 +07:00
Mzack9999 27ac1e8726 chore(deps): bump go-jose to v4.1.6 (#8037) 2026-11-05 13:37:53 +07:00
Mzack9999 5b499e0650 verify TLS and validate template_id for nuclei -ai (#8044) 2026-11-05 13:36:59 +07:00
anant 7748411200 fix(cmd): keep "#" in -H header values
Signed-off-by: anant <[email protected]>
2026-11-05 09:59:47 +05:30
anant 0642f05241 fix(http): keep the response in -ms failure events
Signed-off-by: anant <[email protected]>
2026-11-05 09:59:35 +05:30
Mzack9999 df707d924f Verify TLS for cloud uploads (#8048) 2026-11-05 06:45:55 +07:00
Mzack9999 5f5a40c2d4 Merge pull request #8040 from projectdiscovery/feat/js-ipmi
add ipmi javascript library
2026-11-04 22:12:54 +01:00
Mzack9999 82d5a8acc2 exporters: verify TLS certificates (#8042) 2026-11-04 16:04:54 +07:00
Dogan Can Bakir 9d3be8b22e verify bmc proofs and signed replies, match replies to requests in the ipmi library 2026-11-04 10:52:52 +03:00
Dogan Can Bakir 76dd059821 add ipmi javascript library 2026-11-04 09:26:50 +03:00
Mzack9999 46fbc92a76 Merge pull request #8023 from projectdiscovery/fix/dns-resolver-client
Give each scan its own DNS resolvers
2026-11-03 21:42:45 +01:00
Mzack9999 36d0adba18 Merge pull request #8024 from projectdiscovery/fix/headless-exclude-lock
Read headless excludes under the dialer lock
2026-11-03 20:47:14 +01:00
Mzack9999 049e6b629e Merge pull request #8018 from projectdiscovery/feat/jev-llm-provider
use jev for llm matchers when -llm-provider jev
2026-11-03 19:21:59 +01:00
Dogan Can Bakir 2c6b223bb4 Merge pull request #7983 from projectdiscovery/headless-host-canonical
Deny IPv4-mapped forms of denied hosts in the dialer
2026-11-03 20:45:05 +03:00
Mzack9999 0827b2779b preserve proxy 2026-11-03 18:34:18 +01:00
Mzack9999 0971fb83e2 skip extractors 2026-11-03 18:07:38 +01:00
Mzack9999 62dae92b3d refresh resolvers 2026-11-03 17:30:04 +01:00
Mzack9999 e44df6e947 lock excludes 2026-11-03 17:30:04 +01:00
Mzack9999 25f50f1036 Merge pull request #8012 from TassioSales/fix/clean-flow-js-runtime
fix(flow): prevent global state leakage in shared JS runtime pool (#7986)
2026-11-03 17:13:36 +01:00
Mzack9999 f9446e9290 Merge pull request #8016 from projectdiscovery/dwisiswant0/fix/headless/fail-requests-whose-url-net-url-cannot-parse
fix(headless): fail requests whose URL net/url cannot parse
2026-11-03 17:12:37 +01:00
Mzack9999 faaeaf30b1 mapped deny 2026-11-03 17:07:35 +01:00
Mzack9999 b7fe335285 merge dev 2026-11-03 17:02:03 +01:00
Mzack9999 7481b20d78 Merge pull request #8014 from TassioSales/fix/headless-ipv4-mapped-ipv6-exclude-hosts-7982
fix(headless): normalize IPv4-mapped IPv6 host forms for -exclude-hosts and -lna (#7982)
2026-11-03 13:39:20 +01:00
Mzack9999 48600a24de use canonical 2026-11-03 12:51:48 +01:00
Mzack9999 ec4036147e Merge pull request #8013 from 0xABCD01/fix/govaluate-concat-regression-test
test(expressions): add regression case for govaluate concat bug (follow-up to #7998)
2026-11-03 12:51:11 +01:00
Mzack9999 9b04153a3b merge dev 2026-11-03 12:45:38 +01:00
Dogan Can Bakir 23f3407af7 bump utils for the jev provider 2026-11-03 12:50:46 +03:00
Dwi Siswanto be40690a29 fix(headless): fail requests whose URL net/url cannot parse
With a request modification action (setheader, addheader, deleteheader,
setmethod or setbody), routingRuleHandler replays every browser request
through nuclei's Go HTTP client. go-rod builds that request with
url.Parse and drops the error, so a URL that Chrome accepts but net/url
rejects, such as a path ending in "100%.png", arrived with a nil URL.
The cookie jar sync then panicked, or httputil.DumpRequestOut did when
cookies were disabled, and the panic killed the whole scan.

Check the URL once on entry and fail such requests in the browser. The
Go client cannot send them, and continuing them in the browser would
drop the template's modification rules and, with -cdp-endpoint, skip
the dial-time address checks of nuclei's network policy.

Fixes: #8015
Reported-by: @l4rm4nd
Signed-off-by: Dwi Siswanto <[email protected]>
2026-11-03 16:13:37 +07:00
Dogan Can Bakir 64f94f3e66 use jev for llm matchers when -llm-provider jev 2026-11-03 12:04:47 +03:00
TassioSales b2b54a7ac4 fix(headless): preserve port in CanonicalizeHost and stabilize proxy test
Signed-off-by: TassioSales <[email protected]>
2026-11-03 01:28:54 -03:00
TassioSales 22aedebe5d fix(headless): normalize IPv4-mapped IPv6 host forms for -exclude-hosts and -lna (#7982)
Normalize hostnames and IPv4-mapped IPv6 addresses (e.g. ::ffff:127.0.0.1,
::ffff:7f00:1, and bracketed variants) before validating against NetworkPolicy
and ExcludeTargets in headless execution. Also expand CDP blocked patterns
in SetupHeadlessNetworkPolicy to match mapped IPv6 representations.

Signed-off-by: TassioSales <[email protected]>
2026-11-03 01:07:02 -03:00
Hafiar 5896509c84 test(expressions): add regression case for govaluate concat bug
base64(username + ':' + password) evaluated to base64 of "[admin:]admin"
on govaluate v0.0.0-20260729205521-4c61bf19382e, the pin #7998 started
from. The case was requested by @dogancanbakir in review of #7998 and did
not land before the merge.
2026-11-02 23:42:37 +00:00
Mzack9999 825dcfb10a lock runtime 2026-11-02 22:58:36 +01:00
Mzack9999 d465cc82a4 Merge pull request #7995 from KaemiaDeveloper/fix/ipv6-dual-stack-target-selection
Scan every address family with -sa
2026-11-02 22:36:42 +01:00
Mzack9999 48f36b4edc keep fallback 2026-11-02 21:47:23 +01:00
Mzack9999 0e7a56b0d7 prefer ipv6 2026-11-02 21:31:52 +01:00
TassioSales f8cf960022 fix(flow): prevent global state leakage in shared JS runtime pool (#7986)
Always replenish the JS runtime pool with a fresh, clean runtime after flow
template execution instead of reusing contaminated runtimes.

Signed-off-by: TassioSales <[email protected]>
2026-11-02 16:19:52 -03:00
Mzack9999 cbd2cb3881 Merge pull request #8010 from Ayham970/websocket-subprotocol-support
websocket: negotiate Sec-WebSocket-Protocol via the dialer
2026-11-02 19:53:04 +01:00
Mzack9999 1fc87e6b02 Merge pull request #7998 from 0xABCD01/fix/bump-panicking-deps
fix: bump govaluate and antchfx/xpath so malformed expressions error instead of panicking
2026-11-02 19:00:10 +01:00
Mzack9999 7a08c714f9 Merge pull request #8009 from projectdiscovery/dwisiswant0/fix/templates/keep-eid-exclusions-independent-of-etags
fix(templates): keep -eid exclusions independent of -etags
2026-11-02 18:59:46 +01:00
Ayham970andClaude Opus 4.8 6903f2d559 websocket: reject CR/LF in subprotocols; require negotiation in test
Address review feedback:
- Skip Sec-WebSocket-Protocol values containing CR or LF before negotiating
  them, since gobwas/ws writes subprotocols into the handshake request line and
  a line break could smuggle extra handshake headers.
- Tighten the test server to reject the upgrade unless the client offers the
  vite-hmr subprotocol, so the test fails if negotiation does not happen.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-11-02 17:22:43 +04:00
Ayham970andClaude Opus 4.8 27edbd6a64 websocket: negotiate Sec-WebSocket-Protocol via the dialer
The websocket protocol sent a Sec-WebSocket-Protocol header as a raw handshake
header but never set gobwas/ws Dialer.Protocols. gobwas validates the
server-selected subprotocol against Dialer.Protocols and rejects one that was
only sent as a header, so any server that requires and echoes a subprotocol
(e.g. a Vite HMR dev server advertising "vite-hmr") failed the handshake with
"unexpected protocol in Sec-WebSocket-Protocol header".

Route a Sec-WebSocket-Protocol request header into Dialer.Protocols (comma
splitting, trimming) instead of the raw header map, so the subprotocol is
offered in the handshake and the server's echo is accepted. Adds a regression
test with a server that only upgrades when "vite-hmr" is negotiated.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-11-02 17:12:17 +04:00
Dwi Siswanto be678d9d7c fix(templates): keep -eid exclusions independent of -etags
NewTagFilter added an -eid value to the excluded IDs only when -etags
did not list the same value. The check came from the tag loops in
#1448, which changed the map it writes but not the map it reads. With
-eid dup -etags dup, the filter excluded nothing by ID. Workflow steps
still go through this filter, so a workflow ran the template that a -t
scan with the same flags excluded.

Build each filter set from its own flag with one helper, so no set
reads another flag's values. The insert-if-absent checks go too:
writing an existing map key is a no-op, and the one check that changed
anything was the wrong one. -itags stops deleting from the -etags set,
because Match already lets an included tag override its exclusion.

Fixes: #8004
Signed-off-by: Dwi Siswanto <[email protected]>
2026-11-02 19:33:26 +07:00
dw1 515b22a35a Merge pull request #8006 from projectdiscovery/dwisiswant0/fix/workflow/keep-exclusions-from-silently-emptying-workflows
fix(workflow): keep exclusions from silently emptying workflows
2026-11-02 13:13:51 +01:00
github-actions[bot]andehsandeep cd0122aa85 chore: refresh security context (#7996)
Co-authored-by: ehsandeep <[email protected]>
2026-11-02 19:09:40 +07:00
Dwi Siswanto b7d22eb52b fix(workflow): warn when a step's subtemplates cannot run
A step that loads no templates never matches, so the subtemplates
loaded below it never run. Nothing said so when an exclusion emptied
the step: a tags step dropped the filter's error, and a template step
warned only for an excluded tag, naming the template but not the
subtemplates it held back, and only with -v, since warnings are
hidden by default. Such steps are now counted in a warning shown after
loading, and -v logs each one with its workflow.

Signed-off-by: Dwi Siswanto <[email protected]>
2026-11-02 18:33:15 +07:00
Dwi Siswanto c2cecb92a8 fix(workflow): skip -es for steps whose subtemplates loaded
A step whose subtemplates loaded applied every exclusion, -es included.
Most workflows start from a detection template with info severity, so
-es info, often used to hide low-value results, left them running
nothing, while -s low,medium,high,critical, which selects the same
results, still ran them. A step with subtemplates reports no results of
its own beyond late out-of-band matches, so excluding its severity
mostly keeps the checks below it from running. Such a step now skips
-es as it skips -s.

Signed-off-by: Dwi Siswanto <[email protected]>
2026-11-02 18:33:15 +07:00
Codess 9206560d1d style(test): align regression test provider fixture 2026-11-02 12:01:06 +01:00
Codess e94840ad56 test(input): initialize exclusion map and scope DNS fixtures to test hosts 2026-11-02 11:59:33 +01:00
Dwi Siswanto aec0b6eebf fix(workflow)!: load tags steps when a subtemplate passes the filters
With a template filter set, a workflow step selected by tags loaded only
the templates that had its tags and passed the filter. When the filter
selected one of the step's subtemplates instead, the step loaded
nothing, and since subtemplates run only after their step matches, the
selected subtemplate never ran. Steps selected by template already load
for a loaded subtemplate (#7965), but the tag lookup never got that
signal, and it matched the step's tags and the filters in a single call,
so it could not skip the filters without also dropping the tags.

The tag lookup now selects templates by the step's tags first and then
filters them. A step loaded for its subtemplate used to skip every
filter that rejected a template without an error, so -eid, -es and -ept
did not apply to it while -etags and -et did. Both lookups now apply
only the exclusion filters to such a step, so every exclusion holds, as
it does against -itags since #7536.

The tag filter also let -id lift -eid for the same template, so a
workflow ran a template that a scan with the same flags skips. -eid now
wins, as in the metadata index filter.

BREAKING-CHANGE: WorkflowLoader.GetTemplatePathsByTags takes noValidate
Fixes: #7966
Refs: #7534
Signed-off-by: Dwi Siswanto <[email protected]>
2026-11-02 17:24:37 +07:00
Mzack9999 b6dd16ffbf fix concat 2026-11-02 11:23:21 +01:00
Mzack9999 d510a9c2fb Merge pull request #8003 from projectdiscovery/feat/har-input
add har input mode
2026-11-02 11:11:44 +01:00
Mzack9999 de382b5de3 Merge pull request #7994 from projectdiscovery/feat/auto-calibrate
drop HTTP matches on a host's catch-all page with -auto-calibrate
2026-11-02 11:10:23 +01:00
Mzack9999 7ed324680f Merge pull request #8000 from projectdiscovery/fix/tl-jsonl
honour -jsonl in -tl and drop the listing banner from json output
2026-11-02 11:09:12 +01:00
Dogan Can Bakir 684d55eeed add har input mode 2026-11-02 09:31:10 +03:00
Doğan Can Bakır 4dcb37ad23 honour -jsonl in -tl and drop the listing banner from json output 2026-11-02 08:23:57 +03:00
Hafiar ab506a2bb6 fix: bump govaluate and antchfx/xpath so malformed expressions error instead of panicking 2026-11-02 12:18:58 +07:00
TassioSales 6863ab16e5 fix(output): sanitize nested metadata maps before JSON encoding (#7971)
Templates containing nested maps or lists of maps in info.metadata produce yaml.v2 map[interface{}]interface{} collections during decoding. When formatting JSON output (-j / -jsonl), JSON encoders (sonic / encoding/json) fail on interface{} map keys with 'json: unsupported type: interface {}', causing Nuclei to silently drop results and report 0 matches found.

Sanitize nested metadata maps recursively to map[string]interface{} prior to JSON serialization so all matching results are written.

Fixes #7971.

Signed-off-by: TassioSales <[email protected]>
2026-11-01 22:03:25 -03:00
Codess 7b44856ce8 fix(input): avoid duplicate scans for automatic dual-stack targets 2026-11-01 21:52:20 +01:00
Codess 288f657fca fix(input): preserve IPv4 fallback when IPv6 is unreachable 2026-11-01 21:38:50 +01:00
Codess 87a8c61932 fix(input): select IPv6-only and dual-stack targets without -iv 2026-11-01 21:25:39 +01:00
Doğan Can Bakır 9c26dc1d2a document -auto-calibrate 2026-11-01 18:54:08 +03:00
Doğan Can Bakır 613e581542 calibrate against the host root and cover the http hook 2026-11-01 18:53:48 +03:00
Doğan Can Bakır 5c9f02a4fc drop http matches on a host's catch-all page with -auto-calibrate 2026-11-01 18:35:12 +03:00
ghost df57ae6d29 chore(js): update memoized functions 🤖 2026-10-30 16:11:04 +00:00
Mzack9999 2a5392eaad Merge pull request #7988 from gnuletik/fix/js-redis-reply-scan
fix(js): check redis reply headers in linear time, skip payloads
2026-10-30 17:10:36 +01:00
Mzack9999 bdf7e983c5 Merge pull request #7990 from IBazsecI/fix/openapi-swagger-nil-panics
fix(openapi,swagger): skip null spec entries instead of panicking
2026-10-30 16:58:25 +01:00
bazsec 6a3300b13d fix(openapi,swagger): drop null response headers, reject only-null servers
Address review feedback:

- swagger: a null entry under a response's `headers:` map is also
  dereferenced by openapi2conv.ToV3Response, for both operation and
  top-level responses. Drop those along with null responses.
- openapi: a `servers:` list containing only null entries now returns
  the same "no servers found" error as an empty list, instead of
  succeeding without generating any request.
2026-10-30 15:18:06 +03:00
bazsec 266593d7a0 fix(openapi,swagger): skip null spec entries instead of panicking
A null value in an OpenAPI or Swagger document (an empty list item, or a
map key with no value) decodes to a nil pointer. Several of these were
dereferenced without a check, so loading such a spec with -im openapi or
-im swagger crashed nuclei with SIGSEGV.

openapi (generator.go):
- a null entry under `servers:` -> nil *Server
- a null media type under `content:` -> nil *MediaType

swagger (dereferenced inside kin-openapi's openapi2conv.ToV3):
- a null path item under `paths:` -> nil *PathItem
- a null item under path/operation `parameters:` or a null top-level
  parameter -> nil *Parameter
- a null operation or top-level response -> nil *Response

Skip the nil entries in the generator, and drop them from the decoded
Swagger document before converting it. Other null entries are already
rejected by kin-openapi with an error and are left alone.

Also add a go-fuzz harness for the OpenAPI parser with a seed corpus, so
the weekly fuzz workflow picks it up, and regression tests that panic on
the pre-fix code.
2026-10-30 14:54:30 +03:00
gnuletikandClaude Opus 5.5 7b0ca00fdf fix(js): check redis reply headers in linear time, skip payloads
The read cap scanned a reply line by line and copied what was left of
its buffer for every line, so a 1 MiB bulk reply of newlines allocated
17 GiB. It also checked lines inside bulk payloads as if they were
headers, so a Lua script returning "a\n*abc" failed with "reply exceeds
16777216 bytes".

Follow the RESP framing as bytes pass instead: skip a bulk payload by
its declared length and check only header lines. A rejected header's
read now ends before its "\n", so go-redis never gets a line to
allocate from, and nothing is buffered or copied.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-30 11:56:54 +01:00
Kamil Kamili f5c033162f Reserve request body budget before building parallel and pipeline requests 2026-10-30 02:43:17 +04:00
Mzack9999 2e258fc3f5 Merge pull request #7981 from kamilikamil/js-streaming-hash
Add JS modules for streaming hash and decompression
2026-10-29 22:06:48 +01:00
Mzack9999 b3596ac64e dedupe imports 2026-10-29 21:33:09 +01:00
Mzack9999 ac52212d17 canonicalize headless 2026-10-29 21:25:10 +01:00
Mzack9999 1f79f786da Merge pull request #7975 from antcybersec/fix/exclude-hosts-file-comments
fix(cmd): ignore comments in -exclude-hosts values
2026-10-29 17:17:04 +01:00
Mzack9999 3ab77ec3e7 Merge pull request #7972 from arththebird/fix/json-output-nested-yaml-maps
fix(output): encode nested template maps in JSON output
2026-10-29 16:34:01 +01:00
anant 58354693e0 fix(cmd): ignore comments in -exclude-hosts values 2026-10-29 21:02:44 +05:30
Mzack9999 dc817c8dbd Merge pull request #7970 from Eren-Akdag/fix/7969-httpx-probe-resolvers
fix(runner): use -resolvers for the httpx probe of non-URL inputs (#7969)
2026-10-29 16:30:54 +01:00
Kamil Kamili 71c268d59e Make -max-request-body-inflight opt-in
Default it to 0 so the budget changes nothing unless it is set.
2026-10-29 19:24:41 +04:00
Arthur LoiselleandClaude Opus 5.5 e55509d881 fix(output): reject metadata keys that collide after string conversion
Keys of different types can format to the same string (`1` and `1.0`), and
keeping either value would depend on map iteration order. `ToJSONValue` and
`ToJSONMap` now return an error on such a collision, which both
`MarshalJSON` methods propagate.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-29 11:04:42 -04:00
Mzack9999 998641728c exclude context 2026-10-29 15:57:32 +01:00
Mzack9999 4e3f78c2c0 exclude context 2026-10-29 15:57:32 +01:00
Mzack9999 7a9ec9c8da Merge pull request #7912 from gnuletik/fix/js-redis-dial-once
fix(js): dial a redis connection once and log its failures as warnings
2026-10-29 15:50:11 +01:00
Mzack9999 5d876b88e5 Merge pull request #7965 from antcybersec/fix/workflow-subtemplate-filters
fix(templates): load workflow steps only when they or a subtemplate pass the filters
2026-10-29 15:50:03 +01:00
Kamil Kamili 1d2a913241 Document -max-request-body-inflight in the README 2026-10-29 18:00:56 +04:00
Kamil Kamili 7d143c989a Trim comments in the nuclei/hash and nuclei/compress modules 2026-10-29 17:56:02 +04:00
Kamil Kamili 128ec95adb Trim comments in the request body budget change 2026-10-29 17:54:25 +04:00
Kamil Kamili aa45ef92f6 Test streamed hashes against the HTTP engine's decoded body
The streamed sha256 and mmh3 of a response read over nuclei/net must
equal sha256(body) and mmh3(body) on the body the HTTP engine's response
chain decodes, for each Content-Encoding, with and without chunked.
2026-10-29 17:29:34 +04:00
Kamil Kamili 7f57e92074 Decode chunked transfer coding in nuclei/compress
NewDecoder now takes a comma-separated list of codings in the order they
were applied, as HTTP headers list them, and accepts chunked. Finished
reports when a self-delimiting stream has ended, so a template reading a
chunked body knows when to stop calling Recv.
2026-10-29 17:29:33 +04:00
Arthur LoiselleandClaude Opus 5.5 3ae6ca8e3d fix(output): encode nested template maps in JSON output
Template YAML decodes nested maps as `map[interface{}]interface{}`, which
the JSON encoder rejects, so a result whose `info.metadata` (or javascript
args carried in `meta`) held a map was dropped from `-j`/`-jsonl` output.
`model.Info` and `output.ResultEvent` now encode their metadata with
string-keyed maps.

Closes #7971

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-29 09:21:08 -04:00
Kamil Kamili 6c615b76df Add nuclei/compress JS module for streaming decompression
A javascript template reading a compressed response with Recv cannot
decode it without first holding the whole body. compress.NewDecoder
takes a Content-Encoding the HTTP engine decodes (gzip, deflate, br,
zstd, or identity) and decodes the chunks written to it, handing back
up to about 1 MB per call so a highly compressed chunk never has to be
held whole.

Go's decompressors pull from a reader while templates push chunks, so
each decoder runs its decompressor on a goroutine that pauses whenever
it needs input or has 1 MB to hand back. The goroutine exits when the
script's context ends, so an abandoned decoder does not leak.
2026-10-29 16:00:39 +04:00
Kamil Kamili 681854444d Add nuclei/hash JS module for incremental hashing
A javascript template that reads a large response with Recv has to
hold the whole body to compare it, since only the DSL hash functions
exist and they take the full input. hash.New(algorithm) returns a
running hash: Write and WriteString add chunks and Sum returns the
value so far, in the same format as the DSL md5, sha1, sha256, sha512
and mmh3 functions. Memory does not grow with the data hashed.
2026-10-29 16:00:39 +04:00
Eren-Akdag 449fdd2c37 test(runner): clear proxy env in the httpx probe resolver test
The probe client uses http.ProxyFromEnvironment, so a proxy set in the
environment could route the probe away from the local test server when
the test runs on its own.
2026-10-29 14:58:05 +03:00
Eren-Akdag 9d7aa53226 fix(runner): use -resolvers for the httpx probe of non-URL inputs
The httpx client that probes non-URL inputs for http or https was built
from httpx.DefaultOptions without the configured resolvers, so the probe
resolved inputs through fastdialer's default resolvers (resolv.conf plus
1.1.1.1, 1.0.0.1, 8.8.8.8 and 8.8.4.4) even when -resolvers was set.

Pass InternalResolversList to httpxOptions.Resolvers in the runner and in
the SDK. httpx then uses only those resolvers. Without -resolvers the
probe behaves as before.

Closes #7969
2026-10-29 14:37:28 +03:00
Kamil Kamili f766893685 Reserve request body budget before the body is built
generator.Make builds the whole body from the template's DSL before
executeRequest takes it from the budget, and building costs about twice
the body. With 25 concurrent 23 MB builds outside the budget, it did
not lower peak memory.

ExecuteWithResults now reserves the template's largest body size seen
so far before calling Make. The size is only known after the DSL has
run, so a template's first build runs alone and records it. A body
larger than the reservation releases it and acquires its own size
again, instead of waiting while holding part of the budget, which could
deadlock two growing requests.
2026-10-29 15:31:42 +04:00
Kamil Kamili a4422d8962 Bound request-body bytes held by in-flight HTTP requests
Add -max-request-body-inflight (default 256 MB, 0 disables it).
executeRequest acquires its body size from a shared budget before it
dumps and sends, and releases it when it returns, so concurrent
requests with multi-MB bodies queue instead of all holding their
copies at once. Bodies under 1 MB skip the budget, since the semaphore
is FIFO and a queued heavy request would stall them. Race requests
skip it too, because they must all reach the open gate together.

The HTTP timeout in ExecuteWithResults now starts once the budget
admits the request, so time spent queued does not time it out.
2026-10-29 14:53:14 +04:00
Kamil Kamili efed3f2d8a Cut full-body copies from the post-send request dump
executeRequest dumps every request in full after sending it, so
matchers can read the whole body. For a 23 MB body that dump allocated
about 5x the body per request: cappedWriter grew its buffer by
doubling, the Accept-Encoding insertion copied the whole dump again,
and savedRequest copied it once more before capping it.

Size the buffer for the body up front, insert the header in place, and
let savedRequest cap the dump without converting it to bytes first. The
dump bytes are unchanged.
2026-10-29 14:53:14 +04:00
dependabot[bot] 3ee4f0f408 chore(deps): bump github.com/getkin/kin-openapi (#7968)
Bumps the go_modules group with 1 update in the / directory: [github.com/getkin/kin-openapi](https://github.com/getkin/kin-openapi).


Updates `github.com/getkin/kin-openapi` from 0.144.0 to 0.149.0
- [Release notes](https://github.com/getkin/kin-openapi/releases)
- [Commits](https://github.com/getkin/kin-openapi/compare/v0.144.0...v0.149.0)

---
updated-dependencies:
- dependency-name: github.com/getkin/kin-openapi
  dependency-version: 0.149.0
  dependency-type: direct:production
  dependency-group: go_modules
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-29 04:37:22 +00:00
ghost 091191f518 docs: update syntax & JSON schema 🤖 2026-10-29 04:05:50 +00:00
dependabot[bot] 827f4a9f88 chore(deps): bump the modules group with 2 updates (#7942)
Bumps the modules group with 2 updates: [github.com/projectdiscovery/dsl](https://github.com/projectdiscovery/dsl) and [github.com/projectdiscovery/wappalyzergo](https://github.com/projectdiscovery/wappalyzergo).


Updates `github.com/projectdiscovery/dsl` from 0.8.24 to 0.9.0
- [Release notes](https://github.com/projectdiscovery/dsl/releases)
- [Commits](https://github.com/projectdiscovery/dsl/compare/v0.8.24...v0.9.0)

Updates `github.com/projectdiscovery/wappalyzergo` from 0.3.5 to 0.3.6
- [Release notes](https://github.com/projectdiscovery/wappalyzergo/releases)
- [Commits](https://github.com/projectdiscovery/wappalyzergo/compare/v0.3.5...v0.3.6)

---
updated-dependencies:
- dependency-name: github.com/projectdiscovery/dsl
  dependency-version: 0.9.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: modules
- dependency-name: github.com/projectdiscovery/wappalyzergo
  dependency-version: 0.3.6
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: modules
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-29 11:00:17 +07:00
Mzack9999 bd1c4dd226 feat(headless): expose header/cookie DSL variables (#7573)
* header variables

* Fix review

* Browser cookies

* Reserve keys

* Host cookies
2026-10-29 10:59:17 +07:00
github-actions[bot]andMzack9999 f990fe4d5d chore: refresh security context (#7693)
Co-authored-by: Mzack9999 <[email protected]>
2026-10-29 10:57:39 +07:00
Mzack9999 00ed0552da feat(js): Apply network policy to mysql ConnectWithDSN (#7768)
* enforce policy

* Fix review
2026-10-29 10:57:00 +07:00
Mzack9999 099a037928 feat(headless): chrome sandbox root-only (#7951) 2026-10-29 10:55:52 +07:00
Mzack9999 12d6f03826 test: isolate resolvers (#7952) 2026-10-29 10:53:53 +07:00
anant 1644ba7e81 fix(templates): load workflow steps only when they or a subtemplate pass the filters 2026-10-29 04:40:36 +05:30
Mzack9999 a113cf0db8 Merge pull request #7962 from je-bugshell/feat/no-redact-flag
Add -no-redact to disable the built-in credential redaction
2026-10-28 18:12:43 +01:00
Mzack9999 97337e2b4a update readmes 2026-10-28 17:41:18 +01:00
Mzack9999 8d29c12eef fix redaction 2026-10-28 16:36:38 +01:00
John Eric von Rosen 4fc17dc284 Keep a semicolon inside a redacted query value
Stopping a key=value match at every ; left ?password=;s3cr3t unmasked, since
the value could not start. The value is now lazy and ends only at a cookie
delimiter (; name=), &, a quote or a line break.
2026-10-28 13:09:35 +01:00
John Eric von Rosen 035fc270f9 Keep neighbouring cookies and query parameters when redacting one key
A redacted value used to run to the end of the line, so -redact tracking on
Cookie: tracking=1; session=abc masked the session cookie too. A key=value
match now stops at ; or &; a Key: header is still masked whole. The test also
covers the matched URL.
2026-10-28 13:00:32 +01:00
John Eric von Rosen 3f341f0222 Add -no-redact to disable the built-in credential redaction
Since #7469 the output writer always masks Authorization, Proxy-Authorization,
Cookie, Set-Cookie and X-Api-Key, which leaves exported findings without the
data needed to reproduce them by hand. -no-redact turns the built-in list off;
keys given with -redact are still masked.
2026-10-28 12:35:18 +01:00
Mzack9999 26ce6d2ef2 Merge pull request #7959 from arxhr007/fix/7949-validate-missing-path
fix(loader): fail -validate when a template path can't be found
2026-10-27 18:13:16 +01:00
Mzack9999 0f34b1b6b8 Merge pull request #7960 from projectdiscovery/fix/7957-exclude-hosts-match
fix exclude-hosts matching unrelated hosts at dial time
2026-10-27 17:03:18 +01:00
Doğan Can Bakır 77ba7792f3 keep subdomain and regex exclude-hosts entries 2026-10-27 18:37:21 +03:00
Doğan Can Bakır 8caa65a839 match exclude-hosts entries exactly at dial time 2026-10-27 17:58:24 +03:00
arxhr007 a4afaa41b2 fix(loader): fail -validate when a template path can't be found
`nuclei -validate -t /no/such/path` logged "Could not find template" and
then still printed "All templates validated successfully" and exited 0,
so a mistyped path in a CI step passed. ValidateTemplates ignored the
paths that GetTemplatesPath could not resolve.

Count the paths that aren't handled by NotFoundCallback and return the
validation error when there is any. Paths the callback resolves are
still accepted.

Fixes #7949
2026-10-27 19:59:58 +05:30
Mzack9999 abcdcf19a4 Merge pull request #7940 from mazzz3r/feat/info-placeholders
feat(templates): fill placeholders in info description, impact and remediation
2026-10-27 15:21:09 +01:00
Michael Rodger 9dc004926d fix(trackers): gitea reporter pagination (#7945) 2026-10-27 14:11:01 +07:00
Mzack9999 3d2f411e1d Merge pull request #7925 from projectdiscovery/dwisiswant0/fix/offlinehttp/read-status-lines-by-position-not-text-search
fix(offlinehttp): read status lines by position, not text search
2026-10-26 21:38:57 +01:00
Mzack9999 763e490f42 merge dev 2026-10-26 21:10:10 +01:00
Mzack9999 421ec70686 Merge pull request #7947 from projectdiscovery/dwisiswant0/fix/trackers/follow-x-next-page-in-gitlab-duplicate-search
fix(trackers): follow X-Next-Page in gitlab duplicate search
2026-10-26 20:47:55 +01:00
gnuletikandClaude Opus 5.5 1eeb2f3979 fix(js): log go-redis messages as gologger warnings
Rather than disabling go-redis's logger, route its messages through
gologger at warning level. nuclei's CLI then shows them with -v or -debug
only, and a program embedding nuclei receives them through its gologger
writer, instead of go-redis writing to stderr on its own.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-26 20:05:37 +01:00
gnuletikandClaude Opus 5.5 ea108ee04f fix(js): dial a redis connection once and keep go-redis off stderr
go-redis v9.19 retries a failed dial 5 times, 100ms apart, from a goroutine
that dials on its own context rather than the caller's. Against a filtered
port, a JS call returned at its execution deadline while the pool kept
dialing for ~25s; against a closed port, one call made 20 dials and took
1.7s. Each exhausted dial also printed "connection pool: failed to dial after
5 attempts" to stderr.

The redis clients now dial once per command attempt, as go-redis v9.11 did,
and go-redis's logger is disabled, as the mysql lib does for its driver.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-26 20:05:37 +01:00
Dwi Siswanto aac1427cdc fix(trackers): follow X-Next-Page in gitlab duplicate search
The GitLab tracker ended the duplicate search at the first page holding
fewer issues than duplicate-issue-page-size. GitLab caps per_page at
100, so with a page size above 100 every page looked like the last one.
The tracker read only the first 100 search results and opened a new
issue when the match came later.

The client library already parses X-Next-Page into Response.NextPage.
Follow it instead of guessing from the page length. This also drops the
empty-page request the old check sent when the last page was full.

Fixes: #7946
Refs: #7944
Signed-off-by: Dwi Siswanto <[email protected]>
2026-10-26 21:10:51 +07:00
mazzz3r c4552c6011 feat(templates): fill placeholders in info description, impact and remediation
Description, impact and remediation can reference values of the finding
with the usual {{...}} syntax: named extractors, payload values of the
matching request, variables, DSL helpers, and the result fields host,
port, ip, url, scheme, path, matched-at, matcher-name and
extractor-name, which win over values of the same name. Name, severity,
tags and classification stay static, since filtering, grouping and
issue trackers key on them.

MakeDefaultResultEvent renders each result's copy of the info, so every
protocol gets it and the template never changes. Each field goes
through render.Render without scan options, like request and matcher
text, so nested placeholders and helpers behave the same and network
helpers stay denied. A field whose expression fails keeps the
template's text.

When placeholders change the text, the result keeps the written info in
TemplateInfo (json:"-" bson:"-"):
- the writer runs the -redact patterns over the changed fields, as it
  does over the raw request and response;
- SARIF builds the shared rule from the written info and puts each
  finding's filled text, capped at 4096 characters, in its result
  message.

Info is filled only from a template's own event. Templates with info
placeholders are not clustered, since a cluster's shared event holds the
first template's variables and extracted values, and a global matcher
fills its info from the result fields alone.
2026-10-25 19:37:42 +00:00
Dwi Siswanto 4331cd0f7a fix(offlinehttp): read status lines by position, not text search
Passive mode added a minor version to status lines such as "HTTP/2 200
OK" by rewriting the first "HTTP/<digit> " anywhere in the raw data. In
a valid HTTP/1.1 response that text could sit in a header, the reason
phrase, or the body, which was then altered: a Content-Length body came
out truncated and a chunked one failed to decode. Only the protocol
token at the start of the response is normalized now.

When the data starts with a request, the response is now read from
where the request body ends, with the same normalization. That path
used to reject HTTP/2 status lines and read a request body as the
status line, then fall back to the last "HTTP/" in the data, which
fails whenever the response body contains one. A dump that keeps a
request's Content-Length but leaves out its body is retried right after
the headers. Two exchanges back to back now yield the first response
for HTTP/2, as they already did for HTTP/1.1.

Up to five interim 1xx responses are skipped to reach the final one,
and an interim response is returned only when no final response is
found. A response read after a request carries that request. Data that
cannot be read this way still uses the last "HTTP/" search; httpx
-store-chain output, which puts a newline after the first request,
relies on it to return the chain's final response.

This supersedes #7922, which anchors the version pattern and so fixes

Fixes: #7921
Fixes: #7924
Reported-by: @MasterchiefGalaxy
Signed-off-by: Dwi Siswanto <[email protected]>
2026-10-25 13:16:36 +01:00
209 changed files with 14014 additions and 1579 deletions

No files matched your search

+1 -1
View File
@@ -36,7 +36,7 @@ It also specifies a `WorkflowLoader` interface that is used during workflow load
```go
type WorkflowLoader interface {
GetTemplatePathsByTags(tags []string) []string
GetTemplatePathsByTags(tags []string, noValidate bool) []string
GetTemplatePaths(templatesList []string, noValidate bool) []string
}
```
+6 -3
View File
@@ -145,12 +145,12 @@ TARGET:
-eh, -exclude-hosts string[] hosts to exclude to scan from the input list (ip, cidr, hostname)
-resume string resume scan from and save to specified file (clustering will be disabled)
-sa, -scan-all-ips scan all the IP's associated with dns record
-iv, -ip-version string[] IP version to scan of hostname (4,6) - (default 4)
-iv, -ip-version string[] IP version to scan of hostname (4,6); default uses dialer fallback, -sa scans all addresses
-dun, -disable-url-normalization keep targets exactly as supplied instead of collapsing urls that address the same resource
-mup, -max-urls-per-pattern int scan at most this many urls sharing one structural shape, e.g. /user/1 and /user/2 (0 to scan all)
TARGET-FORMAT:
-im, -input-mode string mode of input file (list, burp, jsonl, yaml, openapi, swagger, http) (default "list")
-im, -input-mode string mode of input file (list, burp, har, jsonl, yaml, openapi, swagger, http) (default "list")
-ro, -required-only use only required fields in input format when generating requests
-sfv, -skip-format-validation skip format validation (like missing vars) when parsing input file
-vtt, -vars-text-templating enable text templating for vars in input file (only for yaml input mode)
@@ -177,7 +177,7 @@ TEMPLATES:
-egm, -enable-global-matchers enable loading global matchers templates
-file enable loading file templates
-llm enable llm matchers and extractors (semantic matching)
-llm-provider string llm provider for semantic matching (openai, ollama, llamacpp, vllm, lmstudio, groq, openrouter, together, orcarouter) (default "openai")
-llm-provider string llm provider for semantic matching (openai, ollama, llamacpp, vllm, lmstudio, groq, openrouter, together, orcarouter, jev) (default "openai")
-llm-base-url string openai-compatible endpoint for semantic matching (overrides -llm-provider, e.g. a local model)
-llm-model string model used for semantic matching
-llm-timeout int time in seconds to wait for a single llm call (default 30)
@@ -221,6 +221,7 @@ OUTPUT:
-jle, -jsonl-export string file to export results in JSONL(ine) format
-pe, -pdf-export string file to export results in PDF format
-rd, -redact string[] redact given list of keys from query parameter, request header and body
-nrd, -no-redact disable the built-in redaction of credential headers (Authorization, Cookie, Set-Cookie, ...) in output; keys given with -redact are still redacted
CONFIGURATIONS:
-config string path to the nuclei configuration file
@@ -255,6 +256,7 @@ CONFIGURATIONS:
-rsr, -response-size-read int max response size to read in bytes
-rss, -response-size-save int max response size to read in bytes (default 1048576)
-rqss, -request-size-save int max request body size to save in bytes (0 to save the whole body) (default 32768)
-mrbi, -max-request-body-inflight int max request body bytes held by in-flight http requests; requests with bodies of 1MB or more queue past it (0 disables)
-reset reset removes all nuclei configuration and data files (including nuclei-templates)
-tlsi, -tls-impersonate enable experimental client hello (ja3) tls randomization
-hae, -http-api-endpoint string experimental http api endpoint
@@ -326,6 +328,7 @@ OPTIMIZATIONS:
-nh, -no-httpx disable httpx probing for non-url input
-preflight-portscan run preflight resolve + TCP portscan and filter targets before scanning (disabled by default)
-no-stdin disable stdin processing
-ac, -auto-calibrate drop HTTP matches on a host's catch-all page (answers missing paths with 2xx/3xx)
HEADLESS:
-headless enable templates that require headless browser support (root user on Linux will disable sandbox)
+3 -2
View File
@@ -148,7 +148,7 @@ TARGET:
-iv, -ip-version string[] IP version to scan of hostname (4,6) - (default 4)
TARGET-FORMAT:
-im, -input-mode string mode of input file (list, burp, jsonl, yaml, openapi, swagger, http) (default "list")
-im, -input-mode string mode of input file (list, burp, har, jsonl, yaml, openapi, swagger, http) (default "list")
-ro, -required-only use only required fields in input format when generating requests
-sfv, -skip-format-validation skip format validation (like missing vars) when parsing input file
-vtt, -vars-text-templating enable text templating for vars in input file (only for yaml input mode)
@@ -175,7 +175,7 @@ TEMPLATES:
-egm, -enable-global-matchers enable loading global matchers templates
-file enable loading file templates
-llm enable llm matchers and extractors (semantic matching)
-llm-provider string llm provider for semantic matching (openai, ollama, llamacpp, vllm, lmstudio, groq, openrouter, together, orcarouter) (default "openai")
-llm-provider string llm provider for semantic matching (openai, ollama, llamacpp, vllm, lmstudio, groq, openrouter, together, orcarouter, jev) (default "openai")
-llm-base-url string openai-compatible endpoint for semantic matching (overrides -llm-provider, e.g. a local model)
-llm-model string model used for semantic matching
-llm-timeout int time in seconds to wait for a single llm call (default 30)
@@ -219,6 +219,7 @@ OUTPUT:
-jle, -jsonl-export string file to export results in JSONL(ine) format
-pe, -pdf-export string file to export results in PDF format
-rd, -redact string[] redact given list of keys from query parameter, request header and body
-nrd, -no-redact disable the built-in redaction of credential headers (Authorization, Cookie, Set-Cookie, ...) in output; keys given with -redact are still redacted
CONFIGURATIONS:
-config string path to the nuclei configuration file
+3 -2
View File
@@ -148,7 +148,7 @@ TARGET:
-iv, -ip-version string[] IP version to scan of hostname (4,6) - (default 4)
TARGET-FORMAT:
-im, -input-mode string mode of input file (list, burp, jsonl, yaml, openapi, swagger, http) (default "list")
-im, -input-mode string mode of input file (list, burp, har, jsonl, yaml, openapi, swagger, http) (default "list")
-ro, -required-only use only required fields in input format when generating requests
-sfv, -skip-format-validation skip format validation (like missing vars) when parsing input file
-vtt, -vars-text-templating enable text templating for vars in input file (only for yaml input mode)
@@ -175,7 +175,7 @@ TEMPLATES:
-egm, -enable-global-matchers enable loading global matchers templates
-file enable loading file templates
-llm enable llm matchers and extractors (semantic matching)
-llm-provider string llm provider for semantic matching (openai, ollama, llamacpp, vllm, lmstudio, groq, openrouter, together, orcarouter) (default "openai")
-llm-provider string llm provider for semantic matching (openai, ollama, llamacpp, vllm, lmstudio, groq, openrouter, together, orcarouter, jev) (default "openai")
-llm-base-url string openai-compatible endpoint for semantic matching (overrides -llm-provider, e.g. a local model)
-llm-model string model used for semantic matching
-llm-timeout int time in seconds to wait for a single llm call (default 30)
@@ -219,6 +219,7 @@ OUTPUT:
-jle, -jsonl-export string file to export results in JSONL(ine) format
-pe, -pdf-export string file to export results in PDF format
-rd, -redact string[] redact given list of keys from query parameter, request header and body
-nrd, -no-redact disable the built-in redaction of credential headers (Authorization, Cookie, Set-Cookie, ...) in output; keys given with -redact are still redacted
CONFIGURATIONS:
-config string path to the nuclei configuration file
+3 -2
View File
@@ -165,7 +165,7 @@ TARGET:
-mup, -max-urls-per-pattern int scan at most this many urls sharing one structural shape, e.g. /user/1 and /user/2 (0 to scan all)
TARGET-FORMAT:
-im, -input-mode string mode of input file (list, burp, jsonl, yaml, openapi, swagger, http) (default "list")
-im, -input-mode string mode of input file (list, burp, har, jsonl, yaml, openapi, swagger, http) (default "list")
-ro, -required-only use only required fields in input format when generating requests
-sfv, -skip-format-validation skip format validation (like missing vars) when parsing input file
-vtt, -vars-text-templating enable text templating for vars in input file (only for yaml input mode)
@@ -192,7 +192,7 @@ TEMPLATES:
-egm, -enable-global-matchers enable loading global matchers templates
-file enable loading file templates
-llm enable llm matchers and extractors (semantic matching)
-llm-provider string llm provider for semantic matching (openai, ollama, llamacpp, vllm, lmstudio, groq, openrouter, together, orcarouter) (default "openai")
-llm-provider string llm provider for semantic matching (openai, ollama, llamacpp, vllm, lmstudio, groq, openrouter, together, orcarouter, jev) (default "openai")
-llm-base-url string openai-compatible endpoint for semantic matching (overrides -llm-provider, e.g. a local model)
-llm-model string model used for semantic matching
-llm-timeout int time in seconds to wait for a single llm call (default 30)
@@ -236,6 +236,7 @@ OUTPUT:
-jle, -jsonl-export string file to export results in JSONL(ine) format
-pe, -pdf-export string file to export results in PDF format
-rd, -redact string[] redact given list of keys from query parameter, request header and body
-nrd, -no-redact disable the built-in redaction of credential headers (Authorization, Cookie, Set-Cookie, ...) in output; keys given with -redact are still redacted
CONFIGURATIONS:
-config string path to the nuclei configuration file
+3 -2
View File
@@ -148,7 +148,7 @@ TARGET:
-iv, -ip-version string[] IP version to scan of hostname (4,6) - (default 4)
TARGET-FORMAT:
-im, -input-mode string mode of input file (list, burp, jsonl, yaml, openapi, swagger, http) (default "list")
-im, -input-mode string mode of input file (list, burp, har, jsonl, yaml, openapi, swagger, http) (default "list")
-ro, -required-only use only required fields in input format when generating requests
-sfv, -skip-format-validation skip format validation (like missing vars) when parsing input file
-vtt, -vars-text-templating enable text templating for vars in input file (only for yaml input mode)
@@ -175,7 +175,7 @@ TEMPLATES:
-egm, -enable-global-matchers enable loading global matchers templates
-file enable loading file templates
-llm enable llm matchers and extractors (semantic matching)
-llm-provider string llm provider for semantic matching (openai, ollama, llamacpp, vllm, lmstudio, groq, openrouter, together, orcarouter) (default "openai")
-llm-provider string llm provider for semantic matching (openai, ollama, llamacpp, vllm, lmstudio, groq, openrouter, together, orcarouter, jev) (default "openai")
-llm-base-url string openai-compatible endpoint for semantic matching (overrides -llm-provider, e.g. a local model)
-llm-model string model used for semantic matching
-llm-timeout int time in seconds to wait for a single llm call (default 30)
@@ -219,6 +219,7 @@ OUTPUT:
-jle, -jsonl-export string file to export results in JSONL(ine) format
-pe, -pdf-export string file to export results in PDF format
-rd, -redact string[] redact given list of keys from query parameter, request header and body
-nrd, -no-redact disable the built-in redaction of credential headers (Authorization, Cookie, Set-Cookie, ...) in output; keys given with -redact are still redacted
CONFIGURATIONS:
-config string path to the nuclei configuration file
+3 -2
View File
@@ -148,7 +148,7 @@ TARGET:
-iv, -ip-version string[] IP version to scan of hostname (4,6) - (default 4)
TARGET-FORMAT:
-im, -input-mode string mode of input file (list, burp, jsonl, yaml, openapi, swagger, http) (default "list")
-im, -input-mode string mode of input file (list, burp, har, jsonl, yaml, openapi, swagger, http) (default "list")
-ro, -required-only use only required fields in input format when generating requests
-sfv, -skip-format-validation skip format validation (like missing vars) when parsing input file
-vtt, -vars-text-templating enable text templating for vars in input file (only for yaml input mode)
@@ -175,7 +175,7 @@ TEMPLATES:
-egm, -enable-global-matchers enable loading global matchers templates
-file enable loading file templates
-llm enable llm matchers and extractors (semantic matching)
-llm-provider string llm provider for semantic matching (openai, ollama, llamacpp, vllm, lmstudio, groq, openrouter, together, orcarouter) (default "openai")
-llm-provider string llm provider for semantic matching (openai, ollama, llamacpp, vllm, lmstudio, groq, openrouter, together, orcarouter, jev) (default "openai")
-llm-base-url string openai-compatible endpoint for semantic matching (overrides -llm-provider, e.g. a local model)
-llm-model string model used for semantic matching
-llm-timeout int time in seconds to wait for a single llm call (default 30)
@@ -219,6 +219,7 @@ OUTPUT:
-jle, -jsonl-export string file to export results in JSONL(ine) format
-pe, -pdf-export string file to export results in PDF format
-rd, -redact string[] redact given list of keys from query parameter, request header and body
-nrd, -no-redact disable the built-in redaction of credential headers (Authorization, Cookie, Set-Cookie, ...) in output; keys given with -redact are still redacted
CONFIGURATIONS:
-config string path to the nuclei configuration file
+3 -2
View File
@@ -148,7 +148,7 @@ TARGET:
-iv, -ip-version string[] IP version to scan of hostname (4,6) - (default 4)
TARGET-FORMAT:
-im, -input-mode string mode of input file (list, burp, jsonl, yaml, openapi, swagger, http) (default "list")
-im, -input-mode string mode of input file (list, burp, har, jsonl, yaml, openapi, swagger, http) (default "list")
-ro, -required-only use only required fields in input format when generating requests
-sfv, -skip-format-validation skip format validation (like missing vars) when parsing input file
-vtt, -vars-text-templating enable text templating for vars in input file (only for yaml input mode)
@@ -175,7 +175,7 @@ TEMPLATES:
-egm, -enable-global-matchers enable loading global matchers templates
-file enable loading file templates
-llm enable llm matchers and extractors (semantic matching)
-llm-provider string llm provider for semantic matching (openai, ollama, llamacpp, vllm, lmstudio, groq, openrouter, together, orcarouter) (default "openai")
-llm-provider string llm provider for semantic matching (openai, ollama, llamacpp, vllm, lmstudio, groq, openrouter, together, orcarouter, jev) (default "openai")
-llm-base-url string openai-compatible endpoint for semantic matching (overrides -llm-provider, e.g. a local model)
-llm-model string model used for semantic matching
-llm-timeout int time in seconds to wait for a single llm call (default 30)
@@ -219,6 +219,7 @@ OUTPUT:
-jle, -jsonl-export string file to export results in JSONL(ine) format
-pe, -pdf-export string file to export results in PDF format
-rd, -redact string[] redact given list of keys from query parameter, request header and body
-nrd, -no-redact disable the built-in redaction of credential headers (Authorization, Cookie, Set-Cookie, ...) in output; keys given with -redact are still redacted
CONFIGURATIONS:
-config string path to the nuclei configuration file
+3 -2
View File
@@ -148,7 +148,7 @@ TARGET:
-iv, -ip-version string[] IP version to scan of hostname (4,6) - (default 4)
TARGET-FORMAT:
-im, -input-mode string mode of input file (list, burp, jsonl, yaml, openapi, swagger, http) (default "list")
-im, -input-mode string mode of input file (list, burp, har, jsonl, yaml, openapi, swagger, http) (default "list")
-ro, -required-only use only required fields in input format when generating requests
-sfv, -skip-format-validation skip format validation (like missing vars) when parsing input file
-vtt, -vars-text-templating enable text templating for vars in input file (only for yaml input mode)
@@ -175,7 +175,7 @@ TEMPLATES:
-egm, -enable-global-matchers enable loading global matchers templates
-file enable loading file templates
-llm enable llm matchers and extractors (semantic matching)
-llm-provider string llm provider for semantic matching (openai, ollama, llamacpp, vllm, lmstudio, groq, openrouter, together, orcarouter) (default "openai")
-llm-provider string llm provider for semantic matching (openai, ollama, llamacpp, vllm, lmstudio, groq, openrouter, together, orcarouter, jev) (default "openai")
-llm-base-url string openai-compatible endpoint for semantic matching (overrides -llm-provider, e.g. a local model)
-llm-model string model used for semantic matching
-llm-timeout int time in seconds to wait for a single llm call (default 30)
@@ -219,6 +219,7 @@ OUTPUT:
-jle, -jsonl-export string file to export results in JSONL(ine) format
-pe, -pdf-export string file to export results in PDF format
-rd, -redact string[] redact given list of keys from query parameter, request header and body
-nrd, -no-redact disable the built-in redaction of credential headers (Authorization, Cookie, Set-Cookie, ...) in output; keys given with -redact are still redacted
CONFIGURATIONS:
-config string path to the nuclei configuration file
+3 -2
View File
@@ -148,7 +148,7 @@ TARGET:
-iv, -ip-version string[] IP version to scan of hostname (4,6) - (default 4)
TARGET-FORMAT:
-im, -input-mode string mode of input file (list, burp, jsonl, yaml, openapi, swagger, http) (default "list")
-im, -input-mode string mode of input file (list, burp, har, jsonl, yaml, openapi, swagger, http) (default "list")
-ro, -required-only use only required fields in input format when generating requests
-sfv, -skip-format-validation skip format validation (like missing vars) when parsing input file
-vtt, -vars-text-templating enable text templating for vars in input file (only for yaml input mode)
@@ -175,7 +175,7 @@ TEMPLATES:
-egm, -enable-global-matchers enable loading global matchers templates
-file enable loading file templates
-llm enable llm matchers and extractors (semantic matching)
-llm-provider string llm provider for semantic matching (openai, ollama, llamacpp, vllm, lmstudio, groq, openrouter, together, orcarouter) (default "openai")
-llm-provider string llm provider for semantic matching (openai, ollama, llamacpp, vllm, lmstudio, groq, openrouter, together, orcarouter, jev) (default "openai")
-llm-base-url string openai-compatible endpoint for semantic matching (overrides -llm-provider, e.g. a local model)
-llm-model string model used for semantic matching
-llm-timeout int time in seconds to wait for a single llm call (default 30)
@@ -219,6 +219,7 @@ OUTPUT:
-jle, -jsonl-export string file to export results in JSONL(ine) format
-pe, -pdf-export string file to export results in PDF format
-rd, -redact string[] redact given list of keys from query parameter, request header and body
-nrd, -no-redact disable the built-in redaction of credential headers (Authorization, Cookie, Set-Cookie, ...) in output; keys given with -redact are still redacted
CONFIGURATIONS:
-config string path to the nuclei configuration file
+103 -238
View File
@@ -1,255 +1,120 @@
<!-- Generated by securitycontext.dev. Source HEAD: 3e1e0e81f091c7817bbf7af64bb8eec6e8c2bac1. -->
<!-- Generated by securitycontext.dev. Source HEAD: df57ae6d2955864fd92a1bd2565fe9e4e9a7592b. -->
# Security Context: projectdiscovery/nuclei @ dev
> Reference context for editing projectdiscovery/nuclei. Before changing any component below, **preserve the listed guards and do not reintroduce these fixed issues**. Each fingerprint is a regression check, and each shared attack surface is a guard that must keep holding on every path.
_Generated 2026-09-01 16:22 UTC by secc_
_Analyzed commit: 3e1e0e81f091c7817bbf7af64bb8eec6e8c2bac1_
**Remediation coverage: 56.7%**
_17 of 30 historical fixes have no remaining unguarded variant; 368 open remediation leads to verify. Measures fix consistency, not a vulnerability verdict._
- Commits analyzed: **4814**
- Commits flagged for review: **235**
- Confirmed historical security fixes: **32**
- Known published CVEs: **0**
## Summary
Nuclei's historical security posture is dominated by sandbox escapes and code execution via unsigned or unverified templates, with recurring SSRF, path traversal, and auth bypass issues across JS libraries, loaders, and workflow paths. The most critical theme is inconsistent enforcement of signature verification and local-file-access (-lfa) guards across direct, workflow, DAST, and JS-sandbox entry points, enabling variant attacks. A second theme is network policy (host allowlist) gaps in LDAP dialing and proxy handling. The security context also includes dependency upgrades (Alpine, go-git) and output file permission hardening, but the highest-value focus is on template trust boundaries and sandbox enforcement parity.
_Analyzed [df57ae6d29][c-df57ae6d29] on `dev` · 5174 commits, 548 security-relevant examined · generated 2026-11-02_
## Recurring Security Patterns
**15.6% of classified fixes repeat a previously fixed pattern** (5 recurring, 27 first-seen). A recurring pattern means this repository fixed a similar issue before — not that the code is currently vulnerable.
- **Code Execution:** [1c440e755a](https://github.com/projectdiscovery/nuclei/commit/1c440e755a97471c56fb0276ee8a8c4132974645) → [9de96e4dda](https://github.com/projectdiscovery/nuclei/commit/9de96e4dda5a03da963b9ae6582f03ea55791a76) — matched on `class:code execution`, `component:pkg/catalog/loader/loader.go`
- **Code Execution via Unsigned Template:** [fa56800fcc](https://github.com/projectdiscovery/nuclei/commit/fa56800fcca3c8c8fe8e8dec28584210ca1527de) → [fac0b7196d](https://github.com/projectdiscovery/nuclei/commit/fac0b7196d2fbca56c8af8ff8e1899c111041a37) — matched on `class:code execution via unsigned template`, `CWE-347`, `sink:loadtemplates`
- **SSRF:** [a167e6c57b](https://github.com/projectdiscovery/nuclei/commit/a167e6c57b80f5dc7be231231b62d7c594ee2770) → [9def9af7e3](https://github.com/projectdiscovery/nuclei/commit/9def9af7e3f7ebcf5cd561c37b623b9999ee7572) — matched on `class:ssrf`, `CWE-918`, `component:pkg/js/libs/ldap/ldap.go`
- **Dependency Vulnerability:** [d93b4a01df](https://github.com/projectdiscovery/nuclei/commit/d93b4a01df9c218a36dd2fccfb1c2c1f96b181fe) → [fcd5c6b111](https://github.com/projectdiscovery/nuclei/commit/fcd5c6b111ddcb746525f88b7d0bf4b4abbc2529) — matched on `class:dependency vulnerability`, `CWE-1104`, `component:dockerfile`
- **Sandbox Escape:** [96049a32fa](https://github.com/projectdiscovery/nuclei/commit/96049a32fa70aef0483d4180d386a57b3c2d973a) → [cf2a98b03d](https://github.com/projectdiscovery/nuclei/commit/cf2a98b03d76327c20fb0183b7f1c849eb692c91) — matched on `class:sandbox escape`, `CWE-284`, `component:v2/internal/runner/processor.go`, `sink:stdlib allmodulenames`, `sink:stdlib getmodulemap`
**7.1% of classified fixes repeat a previously fixed pattern** (1 recurring, 13 first-seen). A recurring pattern means this repository fixed a similar issue before — not that the code is currently vulnerable.
## Top Risks
### 1. Sandbox Escape: `pkg/js/compiler/pool.go, v2/pkg/protocols/javascript/js.go, v2/pkg/protocols/headless/engine/engine.go, v2/internal/runner/processor.go` (high)
Components with the strongest concentration of past fixes for one class.
Multiple sandbox escapes across JS require(), eval/ReadFile, headless browser flags, and tengo imports, indicating weak default sandbox boundaries that require consistent -lfa and safe-module enforcement.
Evidence: 6f2ade6a9b427c284c15a43445f9c7f055e60e5d, c35162c8ef061a96dd74b54c8f022c1a41cf5833, df55f7a2eb591a7949b5c66a9c54db54aa28f450, cf2a98b03d76327c20fb0183b7f1c849eb692c91, 96049a32fa70aef0483d4180d386a57b3c2d973a
### 2. Code Execution via Unsigned Template: `pkg/catalog/loader/loader.go, pkg/templates/workflows.go, pkg/templates/templates.go` (critical)
Unsigned templates can execute code through direct loading, workflows, and DAST branches; signature verification was bypassed via cache poisoning and missing guards on specific paths.
Evidence: 9de96e4dda5a03da963b9ae6582f03ea55791a76, 1c440e755a97471c56fb0276ee8a8c4132974645, fac0b7196d2fbca56c8af8ff8e1899c111041a37, e86f38299765b82ad724fdb701557e0eaff3884d, fa56800fcca3c8c8fe8e8dec28584210ca1527de
### 3. Auth Bypass: `pkg/templates/compile.go, pkg/templates/signer/tmpl_signer.go, pkg/catalog/loader/loader.go, pkg/protocols/http/request.go` (high)
Signature verification can be bypassed via cached metadata, improper signature extraction, or missing checks on specific load paths, and cookie jar bypass via disabled cookie handling.
Evidence: 4534e9cb3034ebe5cc7b0cecabe399b6fb9ce0e4, 0da993afe6d41b4b1b814e8fad23a2acba13c60a, 9bd4db3f74aedd067af568df4f9198a8a090f68b, a976ca8d3f6a18c9ca746c8ce73f28fdcd518642
### 4. SSRF: `pkg/js/libs/ldap/ldap.go, v2/internal/runner/proxy.go` (high)
SSRF via LDAP dialing to internal hosts and via proxy configuration; host allowlist checks were missing or applied to the wrong target (domain vs controller).
Evidence: 9def9af7e3f7ebcf5cd561c37b623b9999ee7572, a167e6c57b80f5dc7be231231b62d7c594ee2770, 0e46d3e0410f89e4af41099f5cd38fa7f54ae54f
### 5. Path Traversal: `pkg/catalog/config/nucleiconfig.go, v2/internal/runner/update.go, v2/pkg/protocols/common/protocolstate/headless.go` (high)
Path traversal via lexical prefix checks, zip-slip in template extraction, and file:// scheme in headless mode; canonicalization and containment checks are required.
Evidence: 20cbf0bd73893c1270582c9778b6b26a5cb11206, f9c214a66f37a00fc29fe64232b690e6285ec59c, d3928e080d819f25306bfbaecc399f03516db5aa
### 6. Code Execution: `pkg/catalog/loader/loader.go, pkg/templates/workflows.go` (high)
Code execution via cache poisoning and unverified code templates in workflows; requires -code flag and verified templates.
Evidence: 9de96e4dda5a03da963b9ae6582f03ea55791a76, e86f38299765b82ad724fdb701557e0eaff3884d
## Shared Attack Surfaces
Capabilities guarded by more than one past fix on different code paths. A guard enforced on one entry point is often missing on a sibling; verify each alternate path before assuming it's covered.
### 1. Template signature verification and unsigned-template opt-in enforcement across all load paths
- Guard that should hold everywhere: signature verification (template.Verified, Verify), DisableUnsignedTemplates flag, -code flag for code templates, cache invalidation on verification status/fingerprint/digest
- Entry points: `pkg/catalog/loader/loader.go (LoadTemplatesWithTags, DAST branch)`, `pkg/templates/workflows.go (loadTemplates, workflowExecutor)`, `pkg/templates/templates.go (loadTemplates)`, `pkg/templates/compile.go (template.Verified)`, `pkg/templates/signer/tmpl_signer.go (Verify)`
- Check: Inspect the DAST and workflow sub-loaders: they must enforce the same DisableUnsignedTemplates and -code checks as the direct loader, and must not rely on cached verification results without re-checking signatures.
- Evidence: 9de96e4dda5a03da963b9ae6582f03ea55791a76, 1c440e755a97471c56fb0276ee8a8c4132974645, fac0b7196d2fbca56c8af8ff8e1899c111041a37, e86f38299765b82ad724fdb701557e0eaff3884d, fa56800fcca3c8c8fe8e8dec28584210ca1527de, 4534e9cb3034ebe5cc7b0cecabe399b6fb9ce0e4, 0da993afe6d41b4b1b814e8fad23a2acba13c60a, 9bd4db3f74aedd067af568df4f9198a8a090f68b
### 2. Local file access (read/write) outside the sandbox, gated by -lfa flag
- Guard that should hold everywhere: -lfa flag enforcement, path canonicalization and containment (sandbox directory), file:// scheme blocking, allowAllFiles stripping, require() registry rebuild
- Entry points: `pkg/js/libs/krbforge/krbforge.go (CreateTicket)`, `pkg/js/libs/mysql/mysql_private.go (sql.Open)`, `pkg/js/compiler/pool.go (require)`, `v2/pkg/protocols/javascript/js.go (ReadFile, eval)`, `v2/pkg/protocols/common/protocolstate/headless.go (file:)`, `v2/pkg/protocols/common/generators/load.go (loadPayloadsFromFile)`
- Check: Verify that every JS library and protocol helper (krbforge, mysql, require, headless file://, payload loaders) enforces the -lfa flag and path containment; a sibling path that skips the -lfa check is a sandbox escape.
- Evidence: 3a8b65cb45f121fcdf418f60e5177ed9f250c122, 9611926a9a1ca99e8ed554d1b26fbeff60e06368, 6f2ade6a9b427c284c15a43445f9c7f055e60e5d, c35162c8ef061a96dd74b54c8f022c1a41cf5833, d3928e080d819f25306bfbaecc399f03516db5aa, e7fb40a41308f6c41267e858d1781eb67f7e0572
### 3. Network policy (host allowlist) enforcement before outbound connections
- Guard that should hold everywhere: IsHostAllowed check on the actual target host (controller, not domain), proxy validation and connectivity check, ProxyInternal gate for HTTP_PROXY env
- Entry points: `pkg/js/libs/ldap/ldap.go (Dial, newLdapSession)`, `v2/internal/runner/proxy.go (loadProxyServers, os.Setenv HTTP_PROXY)`
- Check: Inspect other dialing paths (e.g., HTTP, headless) to ensure they apply the same host allowlist check on the resolved IP/host, not just a user-supplied domain, and that proxy settings cannot redirect traffic to internal hosts.
- Evidence: 9def9af7e3f7ebcf5cd561c37b623b9999ee7572, a167e6c57b80f5dc7be231231b62d7c594ee2770, 0e46d3e0410f89e4af41099f5cd38fa7f54ae54f, 3f5c055aca1499642caf4f40e8cf55ad32bd4da9
### 4. Path traversal in file extraction and template path trust boundaries
- Guard that should hold everywhere: canonical path containment check (filepath.Rel, prefix), zip-slip validation against template directory, sandbox path prefix validation
- Entry points: `pkg/catalog/config/nucleiconfig.go (IsCustomTemplate)`, `v2/internal/runner/update.go (os.OpenFile)`, `v2/pkg/protocols/common/generators/load.go (loadPayloadsFromFile)`
- Check: Audit all file-write and file-read paths that accept template-controlled paths (e.g., template extraction, payload loading) to ensure they use canonical containment, not lexical prefix checks, and apply the same sandbox directory restriction.
- Evidence: 20cbf0bd73893c1270582c9778b6b26a5cb11206, f9c214a66f37a00fc29fe64232b690e6285ec59c, e7fb40a41308f6c41267e858d1781eb67f7e0572
## Historical Vulnerability Classes
| Class | Count |
|---|---|
| Sandbox Escape | 7 |
| Auth Bypass | 4 |
| Code Execution | 3 |
| Dependency Vulnerability | 3 |
| Path Traversal | 3 |
| SSRF | 3 |
| Code Execution via Unsigned Template | 2 |
| Arbitrary File Read | 1 |
| Cryptographic Weakness | 1 |
| Deserialization of Untrusted Data | 1 |
| Information Disclosure | 1 |
| Insecure File Permissions | 1 |
| Race Condition | 1 |
| Security Misconfiguration | 1 |
| # | Class | Component | Severity | Fixes | Evidence |
|---|---|---|---|---|---|
| 1 | SSRF | `pkg/protocols/common/protocolstate/headless.go` | high | 2 | [d43c1f547f][c-d43c1f547f], [d9dabbb4a1][c-d9dabbb4a1] |
| 2 | Arbitrary File Access | `v2/pkg/protocols/common/generators/load.go` | high | 0 (1 component gone, not counted) | [e5154d362a][c-e5154d362a] |
| 3 | Arbitrary File Access | `pkg/js/libs/mysql/mysql.go` | high | 1 | [634cecbb28][c-634cecbb28] |
| 4 | Code Execution | `pkg/fuzz/execute.go` | high | 1 | [ccbfb12bd0][c-ccbfb12bd0] |
| 5 | Code Execution | `pkg/protocols/common/expressions/expressions.go` | high | 1 | [d221732016][c-d221732016] |
| 6 | Information Disclosure | `internal/httpapi/apiendpoint.go` | high | 1 | [a20aa26024][c-a20aa26024] |
## Fixed-Issue Fingerprints
These are real security fixes mined from git history. Use them as regression checks.
Real security fixes mined from git history; use them as regression checks. `verified` fixes are linked to a published advisory, `inferred` fixes were confirmed from the diff evidence alone. Each is identified by its short commit SHA, which the variant-leads report uses to reference it back to this history.
- **Auth Bypass** (high) [CWE-287]: Honors disable-cookie by preventing the shared cookie jar from being applied when cookie handling is disabled, closing a path where cookies could be reused across requests.
- sink: `SetCookieJar(input.CookieJar)`
- fix: Adds !request.DisableCookie check before applying shared cookie jar
- commit: https://github.com/projectdiscovery/nuclei/commit/a976ca8d3f6a18c9ca746c8ce73f28fdcd518642 (2026-08-25)
- **Code Execution** (high) [CWE-347]: This commit fixes a cache poisoning vulnerability where unsigned templates could bypass signature verification and execute malicious code during compilation or request execution.
- sink: `template compilation and execution of JavaScript`
- fix: cache invalidation based on verification status, verifier fingerprint, and content digest
- commit: https://github.com/projectdiscovery/nuclei/commit/9de96e4dda5a03da963b9ae6582f03ea55791a76 (2026-08-04)
- **Code Execution via Unsigned Template** (high) [CWE-347]: Enforces signature verification on JavaScript templates, preventing execution of unsigned templates that could lead to arbitrary code execution.
- sink: `loading unsigned javascript templates into execution paths`
- fix: Require signature verification for javascript templates in direct loading and workflows when signing is enforced
- commit: https://github.com/projectdiscovery/nuclei/commit/fac0b7196d2fbca56c8af8ff8e1899c111041a37 (2026-07-03)
- **Sandbox Escape** (high) [CWE-22]: Enforces nuclei's local file access policy on krbforge output paths, preventing arbitrary file writes outside the sandbox.
- sink: `gpkrb.CreateTicket(cfg) with cfg.OutputFile`
- fix: Path normalization into sandbox and denial of outside paths unless -lfa
- commit: https://github.com/projectdiscovery/nuclei/commit/3a8b65cb45f121fcdf418f60e5177ed9f250c122 (2026-06-24)
- **SSRF** (high) [CWE-918]: Adds network policy host allowlist checks before LDAP dialing, preventing SSRF to restricted internal hosts.
- sink: `dialers.Fastdialer.Dial`
- fix: host allowlist check via IsHostAllowed before dialing
- commit: https://github.com/projectdiscovery/nuclei/commit/9def9af7e3f7ebcf5cd561c37b623b9999ee7572 (2026-06-24)
- **Arbitrary File Read** (high) [CWE-22]: Gates MySQL's allowAllFiles option behind the -lfa flag, preventing arbitrary local file reads via LOAD DATA LOCAL INFILE.
- sink: `sql.Open("mysql", dsn) with AllowAllFiles=true`
- fix: DSN sandbox that strips allowAllFiles unless -lfa is set
- commit: https://github.com/projectdiscovery/nuclei/commit/9611926a9a1ca99e8ed554d1b26fbeff60e06368 (2026-06-19)
- **Code Execution** (critical) [CWE-94]: Moves the unsigned code-template guard ahead of the DAST branch, preventing unsigned code templates from being loaded and executed under DAST, which previously bypassed the signature check.
- sink: `Template loading and execution`
- fix: Hoisting unsigned code-template guard before DAST branch to ensure it applies on all load paths
- commit: https://github.com/projectdiscovery/nuclei/commit/1c440e755a97471c56fb0276ee8a8c4132974645 (2026-06-19)
- **Path Traversal** (high) [CWE-22]: This commit replaces lexical prefix checks with canonical path containment checks across multiple filesystem trust boundaries, preventing path traversal where sibling paths were incorrectly treated as trusted.
- sink: `IsCustomTemplate`
- fix: path canonicalization and containment check
- commit: https://github.com/projectdiscovery/nuclei/commit/20cbf0bd73893c1270582c9778b6b26a5cb11206 (2026-06-04)
- **Information Disclosure** (medium) [CWE-200]: This commit fixes a data leakage issue in workflows where sibling state could leak across concurrent executions, and race requests ignored dynamic values, by syncing context and cloning inputs.
- sink: `syncWorkflowInputToTemplateCtx`
- fix: context synchronization and input cloning
- commit: https://github.com/projectdiscovery/nuclei/commit/805bcfe76f3629dbe71021795d4ce865406c1555 (2026-05-08)
- **Sandbox Escape** (high) [CWE-284]: Fixes sandbox escape where require() bypassed local file access restrictions and reused privileged module cache across executions.
- sink: `require() using default host filesystem loader`
- fix: Rebuild require registry per execution and enforce allow-local-file-access sandbox rules
- commit: https://github.com/projectdiscovery/nuclei/commit/6f2ade6a9b427c284c15a43445f9c7f055e60e5d (2026-04-10)
- **Cryptographic Weakness** (medium) [CWE-338]: Replaces predictable math/rand PRNG with cryptographically secure crypto/rand in JS helper functions.
- sink: `rand.Intn(255) / rand.Int63()`
- fix: Replace predictable math/rand with crypto/rand
- commit: https://github.com/projectdiscovery/nuclei/commit/44ef9ce34ffcfdfca3a96626abc73d99dd0e83ab (2026-03-15)
- **Auth Bypass** (high) [CWE-347]: Caching signature verification results without cryptographic re-check allows a metadata index compromise to bypass template signature verification.
- sink: `template.Verified, template.TemplateVerifier`
- fix: cached verification data reuse
- commit: https://github.com/projectdiscovery/nuclei/commit/4534e9cb3034ebe5cc7b0cecabe399b6fb9ce0e4 (2026-01-21)
- **Race Condition** (medium) [CWE-362]: Fixes a race condition in workflow execution caused by shared context callbacks, which could lead to data corruption or crashes; the primary impact is reliability, not a direct security breach.
- sink: `shared context callbacks in workflow execution`
- fix: fix race condition by making workflow execution sequential
- commit: https://github.com/projectdiscovery/nuclei/commit/24d1f58682027e23cb21a299a170abd2bdf20dc2 (2025-12-05)
- **Auth Bypass** (high) [CWE-347]: Fixes template signature verification by centralizing signature extraction and requiring the signature to be at the end of the template, preventing bypass.
- sink: `Verify`
- fix: signature extraction and verification
- commit: https://github.com/projectdiscovery/nuclei/commit/0da993afe6d41b4b1b814e8fad23a2acba13c60a (2024-08-19)
- **Dependency Vulnerability** (medium) [CWE-1104]: Upgrades Alpine base image from 3.18.5 to 3.18.6 to fix known security vulnerabilities in bundled packages.
- sink: `FROM alpine:3.18.5`
- fix: Upgrade base image to patched version
- commit: https://github.com/projectdiscovery/nuclei/commit/fcd5c6b111ddcb746525f88b7d0bf4b4abbc2529 (2024-03-15)
- **Dependency Vulnerability** (medium) [CWE-1104]: Upgrades Alpine base image from 3.18.2 to 3.18.5 to fix known security vulnerabilities in bundled packages.
- sink: `FROM alpine:3.18.2`
- fix: Upgrade base image to patched version
- commit: https://github.com/projectdiscovery/nuclei/commit/d93b4a01df9c218a36dd2fccfb1c2c1f96b181fe (2024-03-15)
- **Dependency Vulnerability** (high) [CWE-1035]: Updates go-git to fix known vulnerabilities (CVE-2023-49569, CVE-2023-49568), which could be exploited when processing untrusted git data.
- sink: `go-git library usage`
- fix: dependency upgrade to patched version
- commit: https://github.com/projectdiscovery/nuclei/commit/1700cdf4e23128a358066364062246b58c1c013f (2024-03-14)
- **Code Execution via Unsigned Template** (high) [CWE-347]: Introduces a flag to disable unsigned templates, preventing execution of unverified templates that could lead to arbitrary code execution.
- sink: `loading unsigned templates when -dut flag is set`
- fix: Adds -disable-unsigned-templates flag to skip unsigned templates and report them separately
- commit: https://github.com/projectdiscovery/nuclei/commit/fa56800fcca3c8c8fe8e8dec28584210ca1527de (2024-03-14)
- **Auth Bypass** (medium) [CWE-347]: This commit introduces a flag to skip unsigned templates, closing a path where an attacker could execute untrusted templates by bypassing signature verification.
- sink: `parsed.Verified`
- fix: skip unverified templates when DisableUnsignedTemplates is set
- commit: https://github.com/projectdiscovery/nuclei/commit/9bd4db3f74aedd067af568df4f9198a8a090f68b (2024-03-10)
- **Code Execution** (high) [CWE-94]: Prevents execution of unverified code templates in workflows, mitigating code execution risk.
- sink: `template.RequestsCode`
- fix: require -code flag and verified templates for code templates in workflows
- commit: https://github.com/projectdiscovery/nuclei/commit/e86f38299765b82ad724fdb701557e0eaff3884d (2024-03-01)
- **SSRF** (high) [CWE-918]: Fixes SSRF by validating the actual LDAP controller host against the network policy instead of the domain name.
- sink: `newLdapSession`
- fix: host allowlist check on controller instead of domain
- commit: https://github.com/projectdiscovery/nuclei/commit/a167e6c57b80f5dc7be231231b62d7c594ee2770 (2024-01-20)
- **Sandbox Escape** (high) [CWE-94]: Disables eval and tightens file read validation in the JavaScript sandbox, preventing template-driven code execution and file access outside the sandbox.
- sink: `eval function and file read in sandbox mode`
- fix: Disable eval function and rewrite file validation in sandbox mode
- commit: https://github.com/projectdiscovery/nuclei/commit/c35162c8ef061a96dd74b54c8f022c1a41cf5833 (2023-10-13)
- **Path Traversal** (high) [CWE-22]: Adds network policy and blocks file:// protocol in headless mode unless -lfa is set, preventing local file disclosure.
- sink: `file:// protocol access in headless browser`
- fix: Network policy validation and file:// scheme blocking when local file access is disabled
- commit: https://github.com/projectdiscovery/nuclei/commit/d3928e080d819f25306bfbaecc399f03516db5aa (2023-08-25)
- **Deserialization of Untrusted Data** (medium) [CWE-502]: Fixes JSON deserialization by adding explicit UnmarshalJSON methods and correcting tags, preventing type confusion and potential logic bypasses.
- sink: `UnmarshalJSON on user-controlled fields like Type and Name`
- fix: Adds proper UnmarshalJSON implementations and corrects json tags to prevent type confusion
- commit: https://github.com/projectdiscovery/nuclei/commit/85090b75317977bb9301444c9dfd8e4c2dba667a (2023-02-07)
- **Sandbox Escape** (medium) [CWE-22]: The commit adds sandbox mode that restricts payload file paths to the template directory, preventing arbitrary file reads when sandbox is enabled.
- sink: `loadPayloadsFromFile`
- fix: path prefix validation against template directory and template path directory
- commit: https://github.com/projectdiscovery/nuclei/commit/e7fb40a41308f6c41267e858d1781eb67f7e0572 (2022-11-16)
- **Insecure File Permissions** (low) [CWE-732]: Fixes overly permissive file permissions on debug output files by changing from os.ModePerm (0777) to 0644.
- sink: `os.OpenFile`
- fix: restrict file mode to 0644
- commit: https://github.com/projectdiscovery/nuclei/commit/af4854f90de728bb08689d5cae3dec4acfdb6b9a (2022-06-20)
- **Security Misconfiguration** (medium) [CWE-16]: Prevents setting HTTP_PROXY environment variable unless explicitly enabled, avoiding potential proxy hijacking.
- sink: `os.Setenv(types.HTTP_PROXY_ENV, proxyURL.String())`
- fix: Added check for options.ProxyInternal before setting HTTP_PROXY environment variable
- commit: https://github.com/projectdiscovery/nuclei/commit/3f5c055aca1499642caf4f40e8cf55ad32bd4da9 (2022-03-09)
- **Sandbox Escape** (high) [CWE-250]: The commit restricts the use of the --no-sandbox flag to Linux root users, preventing sandbox escape vulnerabilities in other environments.
- sink: `NoSandbox(true)`
- fix: Conditionally disable sandbox only on Linux as root
- commit: https://github.com/projectdiscovery/nuclei/commit/df55f7a2eb591a7949b5c66a9c54db54aa28f450 (2021-12-05)
- **Path Traversal** (high) [CWE-22]: This commit fixes a zip-slip path traversal vulnerability by validating the extracted file path against the configured template directory.
- sink: `os.OpenFile`
- fix: path canonicalization and prefix check
- commit: https://github.com/projectdiscovery/nuclei/commit/f9c214a66f37a00fc29fe64232b690e6285ec59c (2021-11-25)
- **SSRF** (high) [CWE-918]: Adds validation and connectivity checks for SOCKS5 proxies before launching a scan, preventing SSRF through proxy configuration.
- sink: `Proxy connection validation and loading`
- fix: Proxy server validation and connectivity check before scan launch
- commit: https://github.com/projectdiscovery/nuclei/commit/0e46d3e0410f89e4af41099f5cd38fa7f54ae54f (2021-11-10)
- **Sandbox Escape** (high) [CWE-284]: This commit adds a sandbox flag that, when enabled, restricts the tengo scripting engine to safe modules, preventing sandbox escape and arbitrary command execution.
- sink: `stdlib.GetModuleMap(stdlib.AllModuleNames()...)`
- fix: Added sandbox flag to restrict imports to safe modules when sandbox is enabled
- commit: https://github.com/projectdiscovery/nuclei/commit/cf2a98b03d76327c20fb0183b7f1c849eb692c91 (2020-12-13)
- **Sandbox Escape** (high) [CWE-284]: This commit restricts the tengo scripting engine to a safe module list, preventing sandbox escape and arbitrary command execution via workflow templates.
- sink: `stdlib.GetModuleMap(stdlib.AllModuleNames()...)`
- fix: Restricted tengo script imports to a safe module list
- commit: https://github.com/projectdiscovery/nuclei/commit/96049a32fa70aef0483d4180d386a57b3c2d973a (2020-12-13)
**By component** (directories with the most fixes):
- `pkg/protocols/common/protocolstate` — 2 fixes: SSRF (2)
- [1c0f77d28e][c-1c0f77d28e] **fix(js/net): enforce network policy on proxied target in JS net dial** — SSRF, high [CWE-918] · inferred · 2026-10-18
`pkg/js/libs/net/net.go` · sink `dialHTTPProxy` · fix: input validation
- [d43c1f547f][c-d43c1f547f] **fix(headless): canonicalize IPv4-mapped IPv6 literals and re-validate dial targets** — SSRF, high [CWE-918] · inferred · 2026-10-13
`pkg/protocols/common/protocolstate/headless.go` · sink `net.DialTimeout` · fix: canonicalization before validation
- [d9dabbb4a1][c-d9dabbb4a1] **fix(headless): enforce network policy on ws and wss schemes (#7867)** — SSRF, high [CWE-918] · inferred · 2026-10-12
`pkg/protocols/common/protocolstate/headless.go` · sink `isValidHost` · fix: canonicalization before validation
- [a20aa26024][c-a20aa26024] **Fix pprof/auth exposure on http-api-endpoint and harden file permissions** — Information Disclosure, high [CWE-200] · inferred · 2026-08-03
`internal/httpapi/apiendpoint.go` · sink `http.ListenAndServe` · fix: authentication verification
- [fac0b7196d][c-fac0b7196d] **feat(templates): require signatures for javascript templates (#7514)** — Signature Verification Bypass, high [CWE-347] · verified (CVE-2026-76819, GHSA-vxg7-f2jj-jmqm) · 2026-07-03
`pkg/catalog/loader/loader.go` · sink `t.HasJavascriptRequest` · fix: cryptographic verification
- [ccbfb12bd0][c-ccbfb12bd0] **refactor(protocols): template rendering boundary (#7499)** — Code Execution, high [CWE-94] · verified (CVE-2026-76805, GHSA-jpvm-9frm-hjcq) · 2026-06-30
`pkg/fuzz/execute.go` · sink `expressions.Evaluate` · fix: sandbox isolation
- [3a8b65cb45][c-3a8b65cb45] **fix(js): reject unsandboxed ccache writes in krbforge (#7482)** — Sandbox Escape, high [CWE-693] · inferred · 2026-06-24
`pkg/js/libs/krbforge/krbforge.go` · sink `OutputFile` · fix: sandbox isolation
- [68daef44a7][c-68daef44a7] **fix(js): escape dbname in lib/pq URL (#7479)** — SQL Injection, high [CWE-89] · inferred · 2026-06-23
`pkg/js/libs/postgres/postgres.go` · sink `fmt.Sprintf` (library call — the operation around it is the sink) · fix: context-aware output escaping
- [634cecbb28][c-634cecbb28] **gate mysql allowAllFiles behind -lfa** — Arbitrary File Access, high [CWE-22] · verified (CVE-2026-76803, GHSA-xhmx-w2j4-rw3q) · 2026-06-19
`pkg/js/libs/mysql/mysql.go` · sink `sql.Open` · fix: sandbox isolation
- [20cbf0bd73][c-20cbf0bd73] **refactor: use path-aware filesystem containment checks (#7420)** — Path Traversal, high [CWE-22] · inferred · 2026-06-04
`pkg/installer/template.go` · sink `filepath.Join` · fix: canonicalization before validation
- [6f2ade6a9b][c-6f2ade6a9b] **fix(js): respect \`allow-local-file-access\` in \`require\` (#7332)** — Sandbox Escape, high [CWE-693] · inferred · 2026-04-10
`pkg/js/compiler/pool.go` · sink `require.DefaultSourceLoader` · fix: sandbox isolation
- [d221732016][c-d221732016] **fix(expressions): only eval template-authored expressions (#7221)** — Code Execution, high [CWE-94] · verified (CVE-2026-76805, GHSA-jpvm-9frm-hjcq) · 2026-03-16
`pkg/protocols/common/expressions/expressions.go` · sink `FindExpressions` · fix: allowlist enforcement
- [0da993afe6][c-0da993afe6] **Merge commit from fork** — Signature Verification Bypass, medium [CWE-347] · verified (CVE-2024-43405, GHSA-7h5p-mmpp-hgmm) · 2024-08-19
`pkg/templates/signer/tmpl_signer.go` · sink `os.ReadFile` · fix: cryptographic verification
- [e5154d362a][c-e5154d362a] **fixing payload load (#3927)** — Arbitrary File Access, high [CWE-22] · verified (CVE-2023-37896, GHSA-2xx4-jj5v-6mff) · 2023-07-14
`v2/pkg/protocols/common/generators/load.go` · sink `loadPayloads` · fix: sandbox isolation
## Historical Vulnerability Classes
| Class | Fixes | Verified | Most recent fix |
|---|---|---|---|
| SSRF | 3 | 0 | [2026-10-18][c-1c0f77d28e] 1c0f77d28e |
| Arbitrary File Access | 1 (1 component gone, not counted) | 2 | [2026-06-19][c-634cecbb28] 634cecbb28 |
| Code Execution | 2 | 2 | [2026-06-30][c-ccbfb12bd0] ccbfb12bd0 |
| Sandbox Escape | 2 | 0 | [2026-06-24][c-3a8b65cb45] 3a8b65cb45 |
| Signature Verification Bypass | 2 | 2 | [2026-07-03][c-fac0b7196d] fac0b7196d |
| Information Disclosure | 1 | 0 | [2026-08-03][c-a20aa26024] a20aa26024 |
| Path Traversal | 1 | 0 | [2026-06-04][c-20cbf0bd73] 20cbf0bd73 |
| SQL Injection | 1 | 0 | [2026-06-23][c-68daef44a7] 68daef44a7 |
## Known Advisories
### Linked to a fix commit in this history (5)
- **CVE-2026-76805** / GHSA-jpvm-9frm-hjcq (medium 5.3) CWE-200 — fixed in [ccbfb12bd0][c-ccbfb12bd0], [d221732016][c-d221732016]
- Environment Variable Disclosure via Response-Derived Data in DAST/Fuzz Mode. A vulnerability in Nuclei's DAST/fuzz expression evaluation path allows a malicious target server to trigger disclosure of ...
- **CVE-2026-76803** / GHSA-xhmx-w2j4-rw3q (medium 5.3) CWE-284 — fixed in [634cecbb28][c-634cecbb28]
- Local File Read via MySQL Client Sandbox Bypass. A vulnerability in Nuclei's JavaScript MySQL client library allows arbitrary local file reads that bypass the `-allow-local-file-access` (`-lfa`) sandb...
- **CVE-2026-76819** / GHSA-vxg7-f2jj-jmqm (high 8.6) CWE-94,CWE-787 — fixed in [fac0b7196d][c-fac0b7196d]
- Arbitrary Code Execution via Goja JavaScript Engine Vulnerability. A vulnerability in the Goja JavaScript engine used by Nuclei's `javascript:` protocol allows arbitrary native code execution on the s...
- **CVE-2024-43405** / GHSA-7h5p-mmpp-hgmm (high 7.4) CWE-78 — fixed in [0da993afe6][c-0da993afe6]
- Nuclei Template Signature Verification Bypass. ## Summary A vulnerability has been identified in Nuclei's template signature verification system that could allow an attacker to bypass the signature ch...
- **CVE-2023-37896** / GHSA-2xx4-jj5v-6mff (medium 5.9) CWE-22 — fixed in [e5154d362a][c-e5154d362a]
- Nuclei < 2.9.9 - Sandbox restriction bypass. ## Overview Affected: < 2.9.9 (patched 2.9.9).
### Without a linked fix commit (6)
Published elsewhere, fixed before the history window, or not yet linked; full descriptions are in the JSON context.
| Advisory | Severity | Published | CWE | Summary |
|---|---|---|---|---|
| CVE-2026-76804 / GHSA-qgw5-7j4f-fg97 | medium 5.5 | 2026-07-16 | CWE-284 | Local File Read via Workflow File-Protocol Gate Bypass. A vulnerability in Nuclei's workfl... |
| CVE-2026-76802 / GHSA-jpf4-98qj-qr67 | medium 4.7 | 2026-07-16 | CWE-78 | Arbitrary Command Execution via DAST Code Signature Bypass. A vulnerability in Nuclei's DA... |
| CVE-2026-41645 / GHSA-jm34-66cf-qpvr | medium 5.3 | 2026-04-18 | CWE-94 | Environment variable disclosure via Response-Derived DSL Expressions. A vulnerability in N... |
| CVE-2026-41646 / GHSA-29rg-wmcw-hpf4 | medium 5.5 | 2026-04-18 | CWE-284 | Local File Read via require() Module Loader Bypass. A vulnerability in Nuclei's JavaScript... |
| CVE-2024-40641 / GHSA-c3q9-c27p-cw9h | high 7.4 | 2024-07-17 | CWE-78 | Unsigned code template execution through workflows in projectdiscovery/nuclei. ### Summary... |
| CVE-2024-27920 / GHSA-w5wx-6g2r-r78q | high 7.4 | 2024-03-15 | CWE-78 | Unsigned code template execution through workflows in projectdiscovery/nuclei. ### Overvie... |
[c-df57ae6d29]: https://github.com/projectdiscovery/nuclei/commit/df57ae6d2955864fd92a1bd2565fe9e4e9a7592b
[c-d43c1f547f]: https://github.com/projectdiscovery/nuclei/commit/d43c1f547f98ac56654f4335ad20af0cca4c229c
[c-d9dabbb4a1]: https://github.com/projectdiscovery/nuclei/commit/d9dabbb4a19e897a6bc12334a030a43035b27b73
[c-e5154d362a]: https://github.com/projectdiscovery/nuclei/commit/e5154d362af900e83b30fb79028bfab1a7f0c648
[c-634cecbb28]: https://github.com/projectdiscovery/nuclei/commit/634cecbb289db494247c04a9c4ec6a113aef3cc0
[c-ccbfb12bd0]: https://github.com/projectdiscovery/nuclei/commit/ccbfb12bd01447ba25f6e755dfb2bee677736da6
[c-d221732016]: https://github.com/projectdiscovery/nuclei/commit/d2217320162d5782ca7cb95bef9dda17063818f3
[c-a20aa26024]: https://github.com/projectdiscovery/nuclei/commit/a20aa260245e5db889c5a421d4a65aeb2a64bf46
[c-1c0f77d28e]: https://github.com/projectdiscovery/nuclei/commit/1c0f77d28ed639a34c2036fa23c7503ffac0bf27
[c-fac0b7196d]: https://github.com/projectdiscovery/nuclei/commit/fac0b7196d2fbca56c8af8ff8e1899c111041a37
[c-3a8b65cb45]: https://github.com/projectdiscovery/nuclei/commit/3a8b65cb45f121fcdf418f60e5177ed9f250c122
[c-68daef44a7]: https://github.com/projectdiscovery/nuclei/commit/68daef44a7eafd41c39690924620e418762909ef
[c-20cbf0bd73]: https://github.com/projectdiscovery/nuclei/commit/20cbf0bd73893c1270582c9778b6b26a5cb11206
[c-6f2ade6a9b]: https://github.com/projectdiscovery/nuclei/commit/6f2ade6a9b427c284c15a43445f9c7f055e60e5d
[c-0da993afe6]: https://github.com/projectdiscovery/nuclei/commit/0da993afe6d41b4b1b814e8fad23a2acba13c60a
---
_Fixed-issue fingerprints and CVE matches are derived from real data; LLM-written prose is advisory and should be verified against the cited commits._
_Fixed-issue fingerprints and advisory matches are derived from git history and published advisories; the risk tables and pattern descriptions above are derived from them and should be verified against the cited commits._
+20
View File
@@ -530,6 +530,16 @@ Description of the template.
You can go in-depth here on what the template actually does.
Description, impact and remediation may hold {{placeholders}} filled for
each finding: named extractors, payloads, variables and DSL helpers
(network helpers excluded), and the result fields host, port, ip, url,
scheme, path, matched-at, matcher-name and extractor-name, which win over
values of the same name. Use join(", ", name) for an extractor with
several values, and brackets for a hyphenated name inside a helper:
to_upper([matcher-name]). Unknown placeholders stay as written, and so
does a field whose expression fails. Global matcher templates match on
other templates' responses, so only the result fields fill their text.
Examples:
@@ -543,6 +553,10 @@ description: Bower is a package manager which stores package information in the
description: Subversion ALM for the enterprise before 8.8.2 allows reflected XSS at multiple locations
```
```yaml
description: Redis {{version}} on {{host}}:{{port}} accepts the default password
```
</div>
@@ -559,6 +573,8 @@ Impact of the template.
You can go in-depth here on impact of the template.
Supports the same placeholders as description.
Examples:
@@ -665,6 +681,8 @@ Remediation steps for the template.
You can go in-depth here on how to mitigate the problem found by this template.
Supports the same placeholders as description.
Examples:
@@ -3757,6 +3775,8 @@ Part Definitions:
- <code>req</code> - Headless request made from the client
- <code>duration</code> - Latest measured operation duration in seconds
- <code>resp,body,data</code> - Headless response received from client (default)
- <code>header,all_headers</code> - Headless response headers
- <code>status_code</code> - Status code received from the server
<hr />
+2
View File
@@ -20,6 +20,8 @@ extend-exclude = [
"vendor/",
# Go checksum file
"go.sum",
# Generated history. Commit SHAs are not words.
"SECURITY_CONTEXT.md",
]
[default.extend-identifiers]
+57
View File
@@ -0,0 +1,57 @@
package main
import (
"fmt"
"os"
"os/exec"
"path/filepath"
"strings"
"testing"
"github.com/projectdiscovery/gologger"
"github.com/projectdiscovery/nuclei/v3/pkg/types"
"github.com/stretchr/testify/require"
)
const headerChildEnv = "NUCLEI_HEADER_CHILD_ARGS"
// -H values can contain "#", on the command line, in a header file and in the
// config file, while lines of a header file that start with "#" are skipped as
// comments.
func TestHeaderValuesKeepHash(t *testing.T) {
if args := os.Getenv(headerChildEnv); args != "" {
options = &types.Options{Logger: gologger.DefaultLogger}
os.Args = append([]string{os.Args[0]}, strings.Split(args, "\n")...)
readConfig()
fmt.Printf("headers=%s\n", strings.Join(options.CustomHeaders, "|"))
os.Exit(0)
}
root := t.TempDir()
headerFile := filepath.Join(root, "headers.txt")
require.NoError(t, os.WriteFile(headerFile, []byte("# auth headers\nX-Token: abc#def\nReferer: https://app.example.com/#/dashboard\n"), 0o600))
configFile := filepath.Join(root, "config.yaml")
require.NoError(t, os.WriteFile(configFile, []byte("header:\n - \"X-Config: cfg#value\"\n"), 0o600))
tests := []struct {
name string
args []string
want string
}{
{"cli and header file", []string{"-H", "Cookie: sid=1; color=#ff0000", "-H", headerFile}, "Cookie: sid=1; color=#ff0000|X-Token: abc#def|Referer: https://app.example.com/#/dashboard"},
{"config file", []string{"-config", configFile}, "X-Config: cfg#value"},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
tempRoot := filepath.Join(t.TempDir(), "tmp")
require.NoError(t, os.Mkdir(tempRoot, 0o700))
args := append(tt.args, "-disable-update-check")
cmd := exec.Command(os.Args[0], "-test.run=^TestHeaderValuesKeepHash$", "-test.count=1")
cmd.Env = isolatedNucleiStorageEnv(root, tempRoot, headerChildEnv+"="+strings.Join(args, "\n"))
output, err := cmd.CombinedOutput()
require.NoError(t, err, string(output))
require.Contains(t, string(output), "headers="+tt.want+"\n")
})
}
}
+2 -1
View File
@@ -57,7 +57,8 @@
# severity: low
# # duplicate-issue-check (optional) flag to enable duplicate tracking issue check
# duplicate-issue-check: false
# # duplicate-issue-page-size (optional) controls how many issues to fetch per page when searching for duplicates
# # duplicate-issue-page-size (optional) sets the number of issues to request per page when searching for duplicates
# # (GitLab returns at most 100 issues per page)
# duplicate-issue-page-size: 100
# # duplicate-issue-max-pages (optional) limits how many pages to fetch when searching for duplicates (0 = no limit)
# duplicate-issue-max-pages: 0
+23 -4
View File
@@ -55,6 +55,17 @@ var (
inlineSecretsTempMu sync.Mutex
)
// normalizeHeaderValue trims a -H value and drops it when the whole line is a
// "#" comment. The goflags default strips inline comments, which cut values
// such as "Cookie: color=#fff" at the "#".
func normalizeHeaderValue(s string) string {
s = strings.TrimSpace(s)
if strings.HasPrefix(s, "#") {
return ""
}
return s
}
func main() {
options.Logger = gologger.DefaultLogger
@@ -255,6 +266,11 @@ func readConfig() *goflags.FlagSet {
var pdcpAuth string
var fuzzFlag bool
// FileStringSliceOptions as set up by runner.ConfigureOptions, but keeping
// "#" inside -H values
headerSliceOptions := goflags.FileStringSliceOptions
headerSliceOptions.Normalize = normalizeHeaderValue
flagSet := goflags.NewFlagSet()
flagSet.CaseSensitive = true
flagSet.SetDescription(`Nuclei is a fast, template based vulnerability scanner focusing
@@ -271,10 +287,10 @@ on extensive configurability, massive extensibility and ease of use.`)
flagSet.StringSliceVarP(&options.Targets, "target", "u", nil, "target URLs/hosts to scan", goflags.CommaSeparatedStringSliceOptions),
flagSet.StringVarP(&options.TargetsFilePath, "list", "l", "", "path to file containing a list of target URLs/hosts to scan (one per line)"),
flagSet.StringVarP(&options.InlineTargetsList, "targets-inline", "", "", "inline multiline target list (for use in template profiles)"),
flagSet.StringSliceVarP(&options.ExcludeTargets, "exclude-hosts", "eh", nil, "hosts to exclude to scan from the input list (ip, cidr, hostname)", goflags.FileCommaSeparatedStringSliceOptions),
flagSet.StringSliceVarP(&options.ExcludeTargets, "exclude-hosts", "eh", nil, "hosts to exclude to scan from the input list (ip, cidr, hostname)", goflags.FileNormalizedOriginalStringSliceOptions),
flagSet.StringVar(&options.Resume, "resume", "", "resume scan from and save to specified file (clustering will be disabled)"),
flagSet.BoolVarP(&options.ScanAllIPs, "scan-all-ips", "sa", false, "scan all the IP's associated with dns record"),
flagSet.StringSliceVarP(&options.IPVersion, "ip-version", "iv", nil, "IP version to scan of hostname (4,6) - (default 4)", goflags.CommaSeparatedStringSliceOptions),
flagSet.StringSliceVarP(&options.IPVersion, "ip-version", "iv", nil, "IP version to scan of hostname (4,6); default uses dialer fallback, -sa scans all addresses", goflags.CommaSeparatedStringSliceOptions),
flagSet.BoolVarP(&options.DisableURLNormalization, "disable-url-normalization", "dun", false, "keep targets exactly as supplied instead of collapsing urls that address the same resource"),
flagSet.IntVarP(&options.MaxURLsPerPattern, "max-urls-per-pattern", "mup", 0, "scan at most this many urls sharing one structural shape, e.g. /user/1 and /user/2 (0 to scan all)"),
)
@@ -309,7 +325,7 @@ on extensive configurability, massive extensibility and ease of use.`)
flagSet.BoolVarP(&options.EnableGlobalMatchersTemplates, "enable-global-matchers", "egm", false, "enable loading global matchers templates"),
flagSet.BoolVar(&options.EnableFileTemplates, "file", false, "enable loading file templates"),
flagSet.BoolVar(&options.EnableLLM, "llm", false, "enable llm matchers and extractors (semantic matching)"),
flagSet.StringVar(&options.LLMProvider, "llm-provider", "openai", "llm provider for semantic matching (openai, ollama, llamacpp, vllm, lmstudio, groq, openrouter, together, orcarouter)"),
flagSet.StringVar(&options.LLMProvider, "llm-provider", "openai", "llm provider for semantic matching (openai, ollama, llamacpp, vllm, lmstudio, groq, openrouter, together, orcarouter, jev)"),
flagSet.StringVar(&options.LLMBaseURL, "llm-base-url", "", "openai-compatible endpoint for semantic matching (overrides -llm-provider, e.g. a local model)"),
flagSet.StringVar(&options.LLMModel, "llm-model", "", "model used for semantic matching"),
flagSet.IntVar(&options.LLMTimeout, "llm-timeout", 30, "time in seconds to wait for a single llm call"),
@@ -355,6 +371,7 @@ on extensive configurability, massive extensibility and ease of use.`)
flagSet.StringVarP(&options.JSONLExport, "jsonl-export", "jle", "", "file to export results in JSONL(ine) format"),
flagSet.StringVarP(&options.PDFExport, "pdf-export", "pe", "", "file to export results in PDF format"),
flagSet.StringSliceVarP(&options.Redact, "redact", "rd", nil, "redact given list of keys from query parameter, request header and body", goflags.CommaSeparatedStringSliceOptions),
flagSet.BoolVarP(&options.NoRedact, "no-redact", "nrd", false, "disable the built-in redaction of credential headers (Authorization, Cookie, Set-Cookie, ...) in output; keys given with -redact are still redacted"),
)
flagSet.CreateGroup("configs", "Configurations",
@@ -366,7 +383,7 @@ on extensive configurability, massive extensibility and ease of use.`)
flagSet.IntVarP(&options.MaxRedirects, "max-redirects", "mr", 10, "max number of redirects to follow for http templates"),
flagSet.BoolVarP(&options.DisableRedirects, "disable-redirects", "dr", false, "disable redirects for http templates"),
flagSet.StringVarP(&options.ReportingConfig, "report-config", "rc", "", "nuclei reporting module configuration file"), // TODO merge into the config file or rename to issue-tracking
flagSet.StringSliceVarP(&options.CustomHeaders, "header", "H", nil, "custom header/cookie to include in all http request in header:value format (cli, file)", goflags.FileStringSliceOptions),
flagSet.StringSliceVarP(&options.CustomHeaders, "header", "H", nil, "custom header/cookie to include in all http request in header:value format (cli, file)", headerSliceOptions),
flagSet.RuntimeMapVarP(&options.Vars, "var", "V", nil, "custom vars in key=value format"),
flagSet.StringVarP(&options.ResolversFile, "resolvers", "r", "", "file containing resolver list for nuclei"),
flagSet.BoolVarP(&options.SystemResolvers, "system-resolvers", "sr", false, "use system DNS resolving as error fallback"),
@@ -390,6 +407,7 @@ on extensive configurability, massive extensibility and ease of use.`)
flagSet.IntVarP(&options.ResponseReadSize, "response-size-read", "rsr", 0, "max response size to read in bytes"),
flagSet.IntVarP(&options.ResponseSaveSize, "response-size-save", "rss", unitutils.Mega, "max response size to read in bytes"),
flagSet.IntVarP(&options.RequestSaveSize, "request-size-save", "rqss", 32*unitutils.Kilo, "max request body size to save in bytes (0 to save the whole body)"),
flagSet.IntVarP(&options.MaxRequestBodyInFlight, "max-request-body-inflight", "mrbi", 0, "max request body bytes held by in-flight http requests; requests with bodies of 1MB or more queue past it (0 disables)"),
flagSet.CallbackVar(func() { resetRequested = true }, "reset", "reset removes all nuclei configuration and data files (including nuclei-templates)"),
flagSet.BoolVarP(&options.TlsImpersonate, "tls-impersonate", "tlsi", false, "enable experimental client hello (ja3) tls randomization"),
flagSet.StringVarP(&options.HttpApiEndpoint, "http-api-endpoint", "hae", "", "experimental http api endpoint"),
@@ -470,6 +488,7 @@ on extensive configurability, massive extensibility and ease of use.`)
flagSet.BoolVar(&options.PreflightPortScan, "preflight-portscan", false, "run preflight resolve + TCP portscan and filter targets before scanning (disabled by default)"),
flagSet.BoolVar(&options.DisableStdin, "no-stdin", false, "disable stdin processing"),
flagSet.BoolVar(&options.EnableHTTPCache, "http-cache", false, "enable HTTP cache (RFC 9111) for HTTP requests"),
flagSet.BoolVarP(&options.AutoCalibrate, "auto-calibrate", "ac", false, "drop HTTP matches on a host's catch-all page (answers missing paths with 2xx/3xx)"),
)
flagSet.CreateGroup("headless", "Headless",
+15 -17
View File
@@ -15,13 +15,14 @@ require (
github.com/Mzack9999/go-rsync v0.0.0-20250821180103-81ffa574ef4d
github.com/RedTeamPentesting/adauth v0.5.4-0.20260511073005-3d18e8a5a687
github.com/adrg/xdg v0.5.3
github.com/alecthomas/chroma v0.10.0
github.com/alecthomas/chroma/v2 v2.27.0
github.com/alexsnet/go-vnc v0.1.0
github.com/alitto/pond v1.9.2
github.com/andybalholm/brotli v1.2.0
github.com/andygrunwald/go-jira v1.16.1
github.com/antchfx/htmlquery v1.3.5
github.com/antchfx/htmlquery v1.3.7
github.com/antchfx/xmlquery v1.4.4
github.com/antchfx/xpath v1.3.6
github.com/antchfx/xpath v1.3.9
github.com/aws/aws-sdk-go-v2 v1.41.5
github.com/aws/aws-sdk-go-v2/config v1.31.16
github.com/aws/aws-sdk-go-v2/credentials v1.18.20
@@ -33,10 +34,11 @@ require (
github.com/clbanning/mxj/v2 v2.7.0
github.com/ditashi/jsbeautifier-go v0.0.0-20141206144643-2520a8026a9c
github.com/docker/go-units v0.5.0
github.com/dsnet/compress v0.0.2-0.20230904184137-39efe44ab707
github.com/elazarl/goproxy v1.7.2
github.com/fatih/structs v1.1.0
github.com/fullstorydev/grpcurl v1.9.3
github.com/getkin/kin-openapi v0.144.0
github.com/getkin/kin-openapi v0.149.0
github.com/getsops/sops/v3 v3.9.2
github.com/go-echarts/go-echarts/v2 v2.6.0
github.com/go-git/go-git/v5 v5.19.2
@@ -61,6 +63,7 @@ require (
github.com/jhump/protoreflect v1.17.0
github.com/julienschmidt/httprouter v1.3.0
github.com/kitabisa/go-ci v1.0.3
github.com/klauspost/compress v1.18.7
github.com/leslie-qiwa/flat v0.0.0-20230424180412-f9d1cf014baa
github.com/lib/pq v1.12.3
github.com/logrusorgru/aurora/v4 v4.0.0
@@ -71,11 +74,12 @@ require (
github.com/miekg/dns v1.1.73
github.com/oiweiwei/go-msrpc v1.2.12
github.com/olekukonko/tablewriter v1.0.8
github.com/openrdap/rdap v0.9.1
github.com/ory/dockertest/v3 v3.12.0
github.com/pkg/errors v0.9.1
github.com/praetorian-inc/fingerprintx v1.1.15
github.com/projectdiscovery/clistats v0.1.7
github.com/projectdiscovery/dsl v0.8.24
github.com/projectdiscovery/dsl v0.9.0
github.com/projectdiscovery/fastdialer v0.6.0
github.com/projectdiscovery/fasttemplate v0.0.2
github.com/projectdiscovery/gcache v0.0.0-20241015120333-12546c6e3f4c
@@ -85,7 +89,7 @@ require (
github.com/projectdiscovery/goja_nodejs v0.0.0-20260618132410-8519f75f703d
github.com/projectdiscovery/gologger v1.1.73
github.com/projectdiscovery/gostruct v0.0.2
github.com/projectdiscovery/govaluate v0.0.0-20260615100919-5ee2581bbf7e
github.com/projectdiscovery/govaluate v0.0.0-20261102102118-9294133b15a2
github.com/projectdiscovery/gozero v0.1.1-0.20260530071156-fa1dad563d76
github.com/projectdiscovery/hmap v0.0.102
github.com/projectdiscovery/httpx v1.12.0
@@ -102,8 +106,8 @@ require (
github.com/projectdiscovery/tlsx v1.4.3-0.20261020052910-acb80c2c7c75
github.com/projectdiscovery/uncover v1.2.1
github.com/projectdiscovery/useragent v0.0.109
github.com/projectdiscovery/utils v0.11.12-0.20261025172519-64b3c221cfe5
github.com/projectdiscovery/wappalyzergo v0.3.5
github.com/projectdiscovery/utils v0.11.13-0.20261103094816-5c0b20e35350
github.com/projectdiscovery/wappalyzergo v0.3.6
github.com/projectdiscovery/yamldoc-go v1.0.7
github.com/redis/go-redis/v9 v9.19.0
github.com/remeh/sizedwaitgroup v1.0.0
@@ -113,6 +117,7 @@ require (
github.com/segmentio/ksuid v1.0.4
github.com/shurcooL/graphql v0.0.0-20230722043721-ed46e5a46466
github.com/sijms/go-ora/v2 v2.9.0
github.com/spaolacci/murmur3 v1.1.0
github.com/spf13/cast v1.10.0
github.com/stretchr/testify v1.12.1
github.com/syndtr/goleveldb v1.0.0
@@ -171,11 +176,9 @@ require (
github.com/STARRY-S/zip v0.2.3 // indirect
github.com/VividCortex/ewma v1.2.0 // indirect
github.com/akrylysov/pogreb v0.10.2 // indirect
github.com/alecthomas/chroma/v2 v2.20.0 // indirect
github.com/alecthomas/kingpin/v2 v2.4.0 // indirect
github.com/alecthomas/template v0.0.0-20190718012654-fb15b899a751 // indirect
github.com/alecthomas/units v0.0.0-20211218093645-b94a6e3cc137 // indirect
github.com/andybalholm/brotli v1.2.0 // indirect
github.com/andybalholm/cascadia v1.3.4 // indirect
github.com/asaskevich/govalidator v0.0.0-20230301143203-a9d515a09cc2 // indirect
github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.7.8 // indirect
@@ -234,12 +237,10 @@ require (
github.com/dimchansky/utfbom v1.1.1 // indirect
github.com/distribution/reference v0.6.0 // indirect
github.com/djherbis/times v1.6.0 // indirect
github.com/dlclark/regexp2 v1.11.5 // indirect
github.com/dlclark/regexp2/v2 v2.2.1 // indirect
github.com/docker/cli v29.2.0+incompatible // indirect
github.com/docker/go-connections v0.7.0 // indirect
github.com/dop251/goja v0.0.0-20260311135729-065cd970411c // indirect
github.com/dsnet/compress v0.0.2-0.20230904184137-39efe44ab707 // indirect
github.com/dustin/go-humanize v1.0.1 // indirect
github.com/ebitengine/purego v0.10.0 // indirect
github.com/emersion/go-sasl v0.0.0-20241020182733-b788ff22d5a6 // indirect
@@ -262,7 +263,7 @@ require (
github.com/go-fed/httpsig v1.1.0 // indirect
github.com/go-git/gcfg v1.5.1-0.20230307220236-3a3c6141e376 // indirect
github.com/go-git/go-billy/v5 v5.9.0 // indirect
github.com/go-jose/go-jose/v4 v4.1.4 // indirect
github.com/go-jose/go-jose/v4 v4.1.6 // indirect
github.com/go-logr/logr v1.4.4 // indirect
github.com/go-logr/stdr v1.2.2 // indirect
github.com/go-ole/go-ole v1.3.0 // indirect
@@ -323,7 +324,6 @@ require (
github.com/kaiakz/ubuffer v0.0.0-20200803053910-dd1083087166 // indirect
github.com/kataras/jwt v0.1.10 // indirect
github.com/kevinburke/ssh_config v1.2.0 // indirect
github.com/klauspost/compress v1.18.7 // indirect
github.com/klauspost/cpuid/v2 v2.3.0 // indirect
github.com/klauspost/pgzip v1.2.6 // indirect
github.com/kylelemons/godebug v1.1.0 // indirect
@@ -366,7 +366,6 @@ require (
github.com/opencontainers/go-digest v1.0.0 // indirect
github.com/opencontainers/image-spec v1.1.1 // indirect
github.com/opencontainers/runc v1.3.6 // indirect
github.com/openrdap/rdap v0.9.1 // indirect
github.com/pelletier/go-toml/v2 v2.0.8 // indirect
github.com/pierrec/lz4/v4 v4.1.23 // indirect
github.com/pjbgf/sha1cd v0.6.0 // indirect
@@ -385,14 +384,13 @@ require (
github.com/russross/blackfriday/v2 v2.1.0 // indirect
github.com/ryanuber/go-glob v1.0.0 // indirect
github.com/saintfish/chardet v0.0.0-20230101081208-5e3ef4b5456d // indirect
github.com/santhosh-tekuri/jsonschema/v6 v6.0.2 // indirect
github.com/santhosh-tekuri/jsonschema/v6 v6.0.3 // indirect
github.com/sergi/go-diff v1.4.0 // indirect
github.com/shirou/gopsutil v3.21.11+incompatible // indirect
github.com/shirou/gopsutil/v4 v4.26.3 // indirect
github.com/sirupsen/logrus v1.10.2 // indirect
github.com/skeema/knownhosts v1.3.1 // indirect
github.com/sorairolake/lzip-go v0.3.8 // indirect
github.com/spaolacci/murmur3 v1.1.0 // indirect
github.com/spf13/afero v1.15.0 // indirect
github.com/spf13/pflag v1.0.10 // indirect
github.com/spiffe/go-spiffe/v2 v2.7.0 // indirect
+22 -26
View File
@@ -147,15 +147,13 @@ github.com/akrylysov/pogreb v0.10.2 h1:e6PxmeyEhWyi2AKOBIJzAEi4HkiC+lKyCocRGlnDi
github.com/akrylysov/pogreb v0.10.2/go.mod h1:pNs6QmpQ1UlTJKDezuRWmaqkgUE2TuU0YTWyqJZ7+lI=
github.com/alecthomas/assert/v2 v2.11.0 h1:2Q9r3ki8+JYXvGsDyBXwH3LcJ+WK5D0gc5E8vS6K3D0=
github.com/alecthomas/assert/v2 v2.11.0/go.mod h1:Bze95FyfUr7x34QZrjL+XP+0qgp/zg8yS+TtBj1WA3k=
github.com/alecthomas/chroma v0.10.0 h1:7XDcGkCQopCNKjZHfYrNLraA+M7e0fMiJ/Mfikbfjek=
github.com/alecthomas/chroma v0.10.0/go.mod h1:jtJATyUxlIORhUOFNA9NZDWGAQ8wpxQQqNSB4rjA/1s=
github.com/alecthomas/chroma/v2 v2.20.0 h1:sfIHpxPyR07/Oylvmcai3X/exDlE8+FA820NTz+9sGw=
github.com/alecthomas/chroma/v2 v2.20.0/go.mod h1:e7tViK0xh/Nf4BYHl00ycY6rV7b8iXBksI9E359yNmA=
github.com/alecthomas/chroma/v2 v2.27.0 h1:FodwmyOBgJULFYmDqibcp9pvfDLWdtPRh9v/r5BXYZs=
github.com/alecthomas/chroma/v2 v2.27.0/go.mod h1:NjJ3ciIgrqBNeIkWZ4e46nseoLDslxU1LmfCoL+wcY8=
github.com/alecthomas/kingpin/v2 v2.3.1/go.mod h1:oYL5vtsvEHZGHxU7DMp32Dvx+qL+ptGn6lWaot2vCNE=
github.com/alecthomas/kingpin/v2 v2.4.0 h1:f48lwail6p8zpO1bC4TxtqACaGqHYA22qkHjHpqDjYY=
github.com/alecthomas/kingpin/v2 v2.4.0/go.mod h1:0gyi0zQnjuFk8xrkNKamJoyUo382HRL7ATRpFZCw6tE=
github.com/alecthomas/repr v0.5.1 h1:E3G4t2QbHTSNpPKBgMTln5KLkZHLOcU7r37J4pXBuIg=
github.com/alecthomas/repr v0.5.1/go.mod h1:Fr0507jx4eOXV7AlPV6AVZLYrLIuIeSOWtW57eE/O/4=
github.com/alecthomas/repr v0.5.2 h1:SU73FTI9D1P5UNtvseffFSGmdNci/O6RsqzeXJtP0Qs=
github.com/alecthomas/repr v0.5.2/go.mod h1:Fr0507jx4eOXV7AlPV6AVZLYrLIuIeSOWtW57eE/O/4=
github.com/alecthomas/template v0.0.0-20160405071501-a0175ee3bccc/go.mod h1:LOuyumcjzFXgccqObfd/Ljyb9UuFJ6TxHnclSeseNhc=
github.com/alecthomas/template v0.0.0-20190718012654-fb15b899a751 h1:JYp7IbQjafoB+tBA3gMyHYHrpOtNuDiK/uB5uXxq5wM=
github.com/alecthomas/template v0.0.0-20190718012654-fb15b899a751/go.mod h1:LOuyumcjzFXgccqObfd/Ljyb9UuFJ6TxHnclSeseNhc=
@@ -180,14 +178,13 @@ github.com/andygrunwald/go-jira v1.16.1 h1:WoQEar5XoDRAibOgKzTFELlPNlKAtnfWr296R
github.com/andygrunwald/go-jira v1.16.1/go.mod h1:UQH4IBVxIYWbgagc0LF/k9FRs9xjIiQ8hIcC6HfLwFU=
github.com/anmitsu/go-shlex v0.0.0-20200514113438-38f4b401e2be h1:9AeTilPcZAjCFIImctFaOjnTIavg87rW78vTPkQqLI8=
github.com/anmitsu/go-shlex v0.0.0-20200514113438-38f4b401e2be/go.mod h1:ySMOLuWl6zY27l47sB3qLNK6tF2fkHG55UZxx8oIVo4=
github.com/antchfx/htmlquery v1.3.5 h1:aYthDDClnG2a2xePf6tys/UyyM/kRcsFRm+ifhFKoU0=
github.com/antchfx/htmlquery v1.3.5/go.mod h1:5oyIPIa3ovYGtLqMPNjBF2Uf25NPCKsMjCnQ8lvjaoA=
github.com/antchfx/htmlquery v1.3.7 h1:KCUklE1EO/xm6iS3CWx6iatFlcnENBk4I4UYa2pBmLU=
github.com/antchfx/htmlquery v1.3.7/go.mod h1:/uBTMnFEl6LaBlIgw3KZ+c061tAQdlfbCXObZoxwQf4=
github.com/antchfx/xmlquery v1.4.4 h1:mxMEkdYP3pjKSftxss4nUHfjBhnMk4imGoR96FRY2dg=
github.com/antchfx/xmlquery v1.4.4/go.mod h1:AEPEEPYE9GnA2mj5Ur2L5Q5/2PycJ0N9Fusrx9b12fc=
github.com/antchfx/xpath v1.3.3/go.mod h1:i54GszH55fYfBmoZXapTHN8T8tkcHfRgLyVwwqzXNcs=
github.com/antchfx/xpath v1.3.5/go.mod h1:i54GszH55fYfBmoZXapTHN8T8tkcHfRgLyVwwqzXNcs=
github.com/antchfx/xpath v1.3.6 h1:s0y+ElRRtTQdfHP609qFu0+c6bglDv20pqOViQjjdPI=
github.com/antchfx/xpath v1.3.6/go.mod h1:i54GszH55fYfBmoZXapTHN8T8tkcHfRgLyVwwqzXNcs=
github.com/antchfx/xpath v1.3.9 h1:muXJSod6jRzhsiUjvduNSxsRSptFZ2GOJ1oMD1QkPSA=
github.com/antchfx/xpath v1.3.9/go.mod h1:i54GszH55fYfBmoZXapTHN8T8tkcHfRgLyVwwqzXNcs=
github.com/armon/go-socks5 v0.0.0-20160902184237-e75332964ef5 h1:0CwZNZbxp69SHPdPJAN/hZIm0C4OItdklCFmMRWYpio=
github.com/armon/go-socks5 v0.0.0-20160902184237-e75332964ef5/go.mod h1:wHh0iHkYZB8zMSxRWpUBQtwG5a7fFgvEO+odwuTv2gs=
github.com/asaskevich/govalidator v0.0.0-20230301143203-a9d515a09cc2 h1:DklsrG3dyBCFEj5IhUbnKptjxatkF07cF2ak3yi77so=
@@ -359,7 +356,6 @@ github.com/ditashi/jsbeautifier-go v0.0.0-20141206144643-2520a8026a9c h1:+Zo5Ca9
github.com/ditashi/jsbeautifier-go v0.0.0-20141206144643-2520a8026a9c/go.mod h1:HJGU9ULdREjOcVGZVPB5s6zYmHi1RxzT71l2wQyLmnE=
github.com/djherbis/times v1.6.0 h1:w2ctJ92J8fBvWPxugmXIv7Nz7Q3iDMKNx9v5ocVH20c=
github.com/djherbis/times v1.6.0/go.mod h1:gOHeRAz2h+VJNZ5Gmc/o7iD9k4wW7NMVqieYCY99oc0=
github.com/dlclark/regexp2 v1.4.0/go.mod h1:2pZnwuY/m+8K6iRw6wQdMtk+rH5tNGR1i55kozfMjCc=
github.com/dlclark/regexp2 v1.11.5 h1:Q/sSnsKerHeCkc/jSTNq1oCm7KiVgUMZRDUoRu0JQZQ=
github.com/dlclark/regexp2 v1.11.5/go.mod h1:DHkYz0B9wPfa6wondMfaivmHpzrQ3v9q8cnmRbL6yW8=
github.com/dlclark/regexp2/v2 v2.2.1 h1:mf4KkFUj0gJuarK8P+LgiS+Lit7m9N1yAwEfPbee7R0=
@@ -427,8 +423,8 @@ github.com/gaissmai/bart v0.30.0 h1:K1oTSQ6Nf76bhrgxeTqzZa4hE0M9x4xDfL6n6UG67Sc=
github.com/gaissmai/bart v0.30.0/go.mod h1:GREWQfTLRWz/c5FTOsIw+KkscuFkIV5t8Rp7Nd1Td5c=
github.com/geoffgarside/ber v1.2.0 h1:/loowoRcs/MWLYmGX9QtIAbA+V/FrnVLsMMPhwiRm64=
github.com/geoffgarside/ber v1.2.0/go.mod h1:jVPKeCbj6MvQZhwLYsGwaGI52oUorHoHKNecGT85ZCc=
github.com/getkin/kin-openapi v0.144.0 h1:hIRcTH+KjLfkLpYU6bSSfdFpi0fZi1fp+hSPi4aQu9Y=
github.com/getkin/kin-openapi v0.144.0/go.mod h1:3BH9M9XDe/y9M5DSvEocVYAYq1w0qrhJHjC/vZi0AaY=
github.com/getkin/kin-openapi v0.149.0 h1:ZbhmVJ4yq5RZDUsyP8lcBcGMsjsaTqXEFt6isdtMDfA=
github.com/getkin/kin-openapi v0.149.0/go.mod h1:1+BHDzstro+P5CKtPy1X4PfofnFgmRe6uvMy9+r9fKY=
github.com/getsops/gopgagent v0.0.0-20240527072608-0c14999532fe h1:QKe/kmAYbndxwu91TcjHERsnMh5SgOB1x/qicvOdUJ8=
github.com/getsops/gopgagent v0.0.0-20240527072608-0c14999532fe/go.mod h1:awFzISqLJoZLm+i9QQ4SgMNHDqljH6jWV0B36V5MrUM=
github.com/getsops/sops/v3 v3.9.2 h1:Cuuahc/UGu7W3+NXd+mrl+2oXGxxj/jIPsw6Ah2CbxA=
@@ -456,8 +452,8 @@ github.com/go-git/go-git/v5 v5.19.2/go.mod h1:QqCBE1EFN5ddFmrliLQ3/ntRCUjZU3EJuw
github.com/go-gl/glfw v0.0.0-20190409004039-e6da0acd62b1/go.mod h1:vR7hzQXu2zJy9AVAgeJqvqgH9Q5CA+iKCZ2gyEVpxRU=
github.com/go-gl/glfw/v3.3/glfw v0.0.0-20191125211704-12ad95a8df72/go.mod h1:tQ2UAYgL5IevRw8kRxooKSPJfGvJ9fJQFa0TUsXzTg8=
github.com/go-gl/glfw/v3.3/glfw v0.0.0-20200222043503-6f7a984d4dc4/go.mod h1:tQ2UAYgL5IevRw8kRxooKSPJfGvJ9fJQFa0TUsXzTg8=
github.com/go-jose/go-jose/v4 v4.1.4 h1:moDMcTHmvE6Groj34emNPLs/qtYXRVcd6S7NHbHz3kA=
github.com/go-jose/go-jose/v4 v4.1.4/go.mod h1:x4oUasVrzR7071A4TnHLGSPpNOm2a21K9Kf04k1rs08=
github.com/go-jose/go-jose/v4 v4.1.6 h1:sNRkftxFJxwsILiQukqp5z83QBSmntVwmgRY7AqBDO4=
github.com/go-jose/go-jose/v4 v4.1.6/go.mod h1:x4oUasVrzR7071A4TnHLGSPpNOm2a21K9Kf04k1rs08=
github.com/go-kit/kit v0.8.0/go.mod h1:xBxKIO96dXMWWy0MnWVtmwkA9/13aqxPnvrjFYMA2as=
github.com/go-kit/kit v0.9.0/go.mod h1:xBxKIO96dXMWWy0MnWVtmwkA9/13aqxPnvrjFYMA2as=
github.com/go-kit/log v0.1.0/go.mod h1:zbhenjAZHb184qTLMA9ZjW7ThYL0H2mk7Q6pNt4vbaY=
@@ -935,8 +931,8 @@ github.com/projectdiscovery/cdncheck v1.3.2 h1:I453ZWNBeG0WWIy3wi9412l4PTzorLgeb
github.com/projectdiscovery/cdncheck v1.3.2/go.mod h1:5IkUMMlcGaeg1DWW5oJVd3r9X3EpJ88j5HxBeF4Me9M=
github.com/projectdiscovery/clistats v0.1.7 h1:NXf4lUzx1QcbJd6QycNwwYHTv2FsW9MchQ2LsgSL6C0=
github.com/projectdiscovery/clistats v0.1.7/go.mod h1:RxRGxhMzkT344VTTvZWIrojurUXdQQRN1yfwgg/88I8=
github.com/projectdiscovery/dsl v0.8.24 h1:rN2w6LSIjA16qKGRjyj70oxdwXGaspe6sAjEkP7F2fk=
github.com/projectdiscovery/dsl v0.8.24/go.mod h1:V+EludvS5PS19UNMlM3CuBwZ+sFnMaQhCFGxysDUanQ=
github.com/projectdiscovery/dsl v0.9.0 h1:i1KyCbKJyKfHkgdbfh0JkWdd14APUjpEYf+GaYANlCg=
github.com/projectdiscovery/dsl v0.9.0/go.mod h1:ntNFkkiYCTEosGFTbApA61RMrvJTCjnakPxnWj56MyY=
github.com/projectdiscovery/fastdialer v0.6.0 h1:4ZeFQ6LyRH4D8e3qO/yzqIP4Cf9Fr0DduaVpJD6ENnE=
github.com/projectdiscovery/fastdialer v0.6.0/go.mod h1:FpHwa9S5PUgfPbh+irOm+cdU6AO631XidxkhPwPv7yY=
github.com/projectdiscovery/fasttemplate v0.0.2 h1:h2cISk5xDhlJEinlBQS6RRx0vOlOirB2y3Yu4PJzpiA=
@@ -957,8 +953,8 @@ github.com/projectdiscovery/gologger v1.1.73 h1:WX2nzq29cL7HdvOHqIttacoid/5lMwap
github.com/projectdiscovery/gologger v1.1.73/go.mod h1:mJwODZcFDg70ihINpOvZevmBtgvpP8H9/l8Y+OPhZPY=
github.com/projectdiscovery/gostruct v0.0.2 h1:s8gP8ApugGM4go1pA+sVlPDXaWqNP5BBDDSv7VEdG1M=
github.com/projectdiscovery/gostruct v0.0.2/go.mod h1:H86peL4HKwMXcQQtEa6lmC8FuD9XFt6gkNR0B/Mu5PE=
github.com/projectdiscovery/govaluate v0.0.0-20260615100919-5ee2581bbf7e h1:vxzgQlz2Cy/YvizYDQx9OhucBcmBotfDhbQ4yCY2vfA=
github.com/projectdiscovery/govaluate v0.0.0-20260615100919-5ee2581bbf7e/go.mod h1:xH7bPwHxUlz1yx9UlVeTF+UVCUaKhTnZgaxHb5z362E=
github.com/projectdiscovery/govaluate v0.0.0-20261102102118-9294133b15a2 h1:aSLbDvrJQPlp4HvojU+bTU5XV8xdAVCjoc5gX1Wfbw4=
github.com/projectdiscovery/govaluate v0.0.0-20261102102118-9294133b15a2/go.mod h1:Z3VjpUJV1pnXDwvAddA8aZ4e839SoeftB7CSdZntAVI=
github.com/projectdiscovery/gozero v0.1.1-0.20260530071156-fa1dad563d76 h1:AN70bbi6BBs7KpIM9w0LxygUN7uzT/oH+owDIQ+Fz/k=
github.com/projectdiscovery/gozero v0.1.1-0.20260530071156-fa1dad563d76/go.mod h1:cWHYnRXoYWHtTpOYyAp5laGYX8GH8ITUhgQaP8G/8FA=
github.com/projectdiscovery/hmap v0.0.102 h1:ybPuf7UQvuOLiHLOUbVMvd+PSlItQU5iQpP1Erl8kaU=
@@ -999,10 +995,10 @@ github.com/projectdiscovery/uncover v1.2.1 h1:8U46T/96CLT7BPoXBgkTvWqB06lOyeTSLv
github.com/projectdiscovery/uncover v1.2.1/go.mod h1:0p8onrWxfpXQEYs90ZDzTSpu1107gWmodX1NWqu/+z4=
github.com/projectdiscovery/useragent v0.0.109 h1:b86BPdZvmgbJousa04jAugDRY4Y+rAvEamE0TFCPmeE=
github.com/projectdiscovery/useragent v0.0.109/go.mod h1:sRyBqDAcD31RKYrO8fyXGowLd36fNwY1g0MNW9mp7zk=
github.com/projectdiscovery/utils v0.11.12-0.20261025172519-64b3c221cfe5 h1:AOspf7ScbwZr8A2YnjCFd4ZJAMSFqzfEaIJIbJPuu10=
github.com/projectdiscovery/utils v0.11.12-0.20261025172519-64b3c221cfe5/go.mod h1:TJgsyVjB02CavwZVFzUzPg6YOSJXQxLhL808JCiQKlU=
github.com/projectdiscovery/wappalyzergo v0.3.5 h1:UZradYF1GIcruSIV5wwo/Coph9/VnM17oajLD9eXR2U=
github.com/projectdiscovery/wappalyzergo v0.3.5/go.mod h1:4E6+fh/ckaY3wo5tlM7lNYSkhuTYRHzu2YmGIRraXP0=
github.com/projectdiscovery/utils v0.11.13-0.20261103094816-5c0b20e35350 h1:uXqpYa6XKV9dCvoti9PbSgIWIsBKPu5YKMywB9G4q0Q=
github.com/projectdiscovery/utils v0.11.13-0.20261103094816-5c0b20e35350/go.mod h1:TJgsyVjB02CavwZVFzUzPg6YOSJXQxLhL808JCiQKlU=
github.com/projectdiscovery/wappalyzergo v0.3.6 h1:1FXCMeCa1xVu2zaW+eeBcbVxtlMStqGOlKcMLyh2pzI=
github.com/projectdiscovery/wappalyzergo v0.3.6/go.mod h1:4E6+fh/ckaY3wo5tlM7lNYSkhuTYRHzu2YmGIRraXP0=
github.com/projectdiscovery/yamldoc-go v1.0.7 h1:Odmjd1o4EkMff2F6QHi1dIYPIQPZUFy24YqmQCprmTk=
github.com/projectdiscovery/yamldoc-go v1.0.7/go.mod h1:nqD1iPMcofpRA1DbXavPByJhWURrshT+l8WN9XjyQag=
github.com/prometheus/client_golang v0.9.1/go.mod h1:7SWBe2y4D6OKWSNQJUaRYU/AaXPKyh/dDVn+NZz0KFw=
@@ -1055,8 +1051,8 @@ github.com/saintfish/chardet v0.0.0-20230101081208-5e3ef4b5456d h1:hrujxIzL1woJ7
github.com/saintfish/chardet v0.0.0-20230101081208-5e3ef4b5456d/go.mod h1:uugorj2VCxiV1x+LzaIdVa9b4S4qGAcH6cbhh4qVxOU=
github.com/sandrolain/httpcache v1.4.0 h1:Jf4Vx62X2ybvNPSpPvI1kT3xvMdDG1AsApQjOQKO9E0=
github.com/sandrolain/httpcache v1.4.0/go.mod h1:kHBuXveitSn39SNPBhdf/ybG272X706HJ2RJqOQ+Em0=
github.com/santhosh-tekuri/jsonschema/v6 v6.0.2 h1:KRzFb2m7YtdldCEkzs6KqmJw4nqEVZGK7IN2kJkjTuQ=
github.com/santhosh-tekuri/jsonschema/v6 v6.0.2/go.mod h1:JXeL+ps8p7/KNMjDQk3TCwPpBy0wYklyWTfbkIzdIFU=
github.com/santhosh-tekuri/jsonschema/v6 v6.0.3 h1:1EYB5IzjZawrrnELUi78f9fPu57HuXjmddZPjrls/28=
github.com/santhosh-tekuri/jsonschema/v6 v6.0.3/go.mod h1:JXeL+ps8p7/KNMjDQk3TCwPpBy0wYklyWTfbkIzdIFU=
github.com/segmentio/ksuid v1.0.4 h1:sBo2BdShXjmcugAMwjugoGUdUV0pcxY5mW4xKRn3v4c=
github.com/segmentio/ksuid v1.0.4/go.mod h1:/XUiZBD3kVx5SmUOl55voK5yeAbBNNIed+2O73XgrPE=
github.com/sergi/go-diff v1.4.0 h1:n/SP9D5ad1fORl+llWyN+D6qoUETXNZARKjyY2/KVCw=
+29 -5
View File
@@ -4,6 +4,8 @@ import (
"bufio"
"bytes"
"context"
"crypto/tls"
"errors"
"fmt"
"io"
"net/http"
@@ -87,11 +89,7 @@ func NewUploadWriter(ctx context.Context, logger *gologger.Logger, creds *pdcpau
tmp.Update()
u.uploadURL = tmp.URL
// create http client
opts := retryablehttp.DefaultOptionsSingle
opts.NoAdjustTimeout = true
opts.Timeout = time.Duration(3) * time.Minute
u.client = retryablehttp.NewClient(opts)
u.client = newUploadClient()
// create context
ctx, u.cancel = context.WithCancel(ctx)
@@ -101,6 +99,32 @@ func NewUploadWriter(ctx context.Context, logger *gologger.Logger, creds *pdcpau
return u, nil
}
// newUploadClient returns the client for cloud uploads. The upload sends the
// API key and the scan results, so the client verifies the server certificate.
// The retryablehttp default clients skip verification.
func newUploadClient() *retryablehttp.Client {
transport := retryablehttp.DefaultReusePooledTransport()
// fastdialer DialTLS does not verify certificates, so net/http does the
// TLS handshake over the fastdialer connection.
transport.DialTLSContext = nil
transport.TLSClientConfig = &tls.Config{MinVersion: tls.VersionTLS12}
opts := retryablehttp.DefaultOptionsSingle
opts.NoAdjustTimeout = true
opts.Timeout = time.Duration(3) * time.Minute
opts.HttpClient = &http.Client{Transport: transport}
// a certificate error does not go away on retry
defaultRetry := retryablehttp.DefaultRetryPolicy()
opts.CheckRetry = func(ctx context.Context, resp *http.Response, err error) (bool, error) {
var certErr *tls.CertificateVerificationError
if errors.As(err, &certErr) {
return false, nil
}
return defaultRetry(ctx, resp, err)
}
return retryablehttp.NewClient(opts)
}
// SetScanID sets the scan id for the upload writer
func (u *UploadWriter) SetScanID(id string) error {
if !xidRegex.MatchString(id) {
+55
View File
@@ -0,0 +1,55 @@
package pdcp
import (
"context"
"crypto/tls"
"crypto/x509"
"net/http"
"net/http/httptest"
"sync/atomic"
"testing"
"github.com/projectdiscovery/gologger"
pdcpauth "github.com/projectdiscovery/utils/auth/pdcp"
"github.com/stretchr/testify/require"
)
func newTestUploadWriter(t *testing.T, gotKey *atomic.Value) (*UploadWriter, *httptest.Server) {
t.Helper()
srv := httptest.NewTLSServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
gotKey.Store(r.Header.Get(pdcpauth.ApiKeyHeaderName))
_, _ = w.Write([]byte(`{"id":"abcdefghij0123456789"}`))
}))
t.Cleanup(srv.Close)
creds := &pdcpauth.PDCPCredentials{APIKey: "test-key", Server: srv.URL}
u, err := NewUploadWriter(context.Background(), gologger.DefaultLogger, creds)
require.NoError(t, err)
t.Cleanup(u.Close)
return u, srv
}
func TestUploadRejectsUntrustedCertificate(t *testing.T) {
var gotKey atomic.Value
u, _ := newTestUploadWriter(t, &gotKey)
err := u.upload([]byte("{}\n"))
require.Error(t, err)
var unknownAuthority x509.UnknownAuthorityError
require.ErrorAs(t, err, &unknownAuthority)
require.Nil(t, gotKey.Load(), "API key must not reach a server with an untrusted certificate")
}
func TestUploadTrustedCertificate(t *testing.T) {
var gotKey atomic.Value
u, srv := newTestUploadWriter(t, &gotKey)
pool := x509.NewCertPool()
pool.AddCert(srv.Certificate())
transport, ok := u.client.HTTPClient.Transport.(*http.Transport)
require.True(t, ok)
transport.TLSClientConfig = &tls.Config{RootCAs: pool, MinVersion: tls.VersionTLS12}
require.NoError(t, u.upload([]byte("{}\n")))
require.Equal(t, "test-key", gotKey.Load())
require.Equal(t, "abcdefghij0123456789", u.scanID)
}
+3
View File
@@ -39,6 +39,9 @@ func (r *Runner) initializeTemplatesHTTPInput() (*hybrid.HybridMap, error) {
}
httpxOptions.RetryMax = r.options.Retries
httpxOptions.Timeout = time.Duration(r.options.Timeout) * time.Second
if len(r.options.InternalResolversList) > 0 {
httpxOptions.Resolvers = r.options.InternalResolversList
}
dialers := protocolstate.GetDialersWithId(r.options.ExecutionId)
if dialers == nil {
+35 -1
View File
@@ -2,8 +2,12 @@ package runner
import (
"context"
"errors"
"sync"
"time"
"github.com/projectdiscovery/gologger"
llmclient "github.com/projectdiscovery/nuclei/v3/pkg/operators/common/llm"
"github.com/projectdiscovery/nuclei/v3/pkg/types"
utilsllm "github.com/projectdiscovery/utils/llm"
@@ -28,6 +32,32 @@ func (a *llmMatcherClient) Complete(ctx context.Context, prompt string, asJSON b
})
}
// llmJudgeClient is an llmMatcherClient whose provider classifies natively, so
// llm matchers ask it through Judge instead of Complete.
type llmJudgeClient struct {
*llmMatcherClient
warnComplete sync.Once
}
func (a *llmJudgeClient) Judge(ctx context.Context, question string, labels, inputs []string) (llmclient.Verdict, error) {
verdict, err := a.client.Judge(ctx, utilsllm.JudgeRequest{Question: question, Labels: labels, Inputs: inputs})
if err != nil {
return llmclient.Verdict{}, err
}
return llmclient.Verdict{Label: verdict.Label, Confidence: verdict.Confidence, Model: verdict.Model}, nil
}
// Complete is only reached by llm extractors, which need free text a
// classifier cannot produce. Returning here spends no call budget and
// makes no provider request. Extractors turn the error into an empty
// result, and the warning says why.
func (a *llmJudgeClient) Complete(context.Context, string, bool) (string, error) {
a.warnComplete.Do(func() {
gologger.Warning().Msgf("llm provider %q cannot run llm extractors, they return nothing", utilsllm.ProviderJev)
})
return "", errors.New("llm provider cannot run extractors")
}
// configureLLM builds the scan's llm client from options. It returns nil unless
// -llm is set, so nothing reaches a model until the user opts in. The API key is
// read from the environment by the provider layer, never from options.
@@ -52,5 +82,9 @@ func configureLLM(options *types.Options) (llmclient.Client, error) {
return nil, err
}
return &llmMatcherClient{client: client, model: options.LLMModel}, nil
matcherClient := &llmMatcherClient{client: client, model: options.LLMModel}
if client.CanJudge() {
return &llmJudgeClient{llmMatcherClient: matcherClient}, nil
}
return matcherClient, nil
}
+65
View File
@@ -0,0 +1,65 @@
package runner
import (
"context"
"net/http"
"net/http/httptest"
"sync/atomic"
"testing"
llmclient "github.com/projectdiscovery/nuclei/v3/pkg/operators/common/llm"
"github.com/projectdiscovery/nuclei/v3/pkg/types"
"github.com/stretchr/testify/require"
)
func TestConfigureLLMExposesJudgeOnlyForClassifiers(t *testing.T) {
t.Setenv("LLM_API_KEY", "key")
client, err := configureLLM(&types.Options{EnableLLM: true, LLMProvider: "jev", LLMModel: "jev-latest"})
require.NoError(t, err)
require.Implements(t, (*llmclient.Judge)(nil), client)
client, err = configureLLM(&types.Options{EnableLLM: true, LLMProvider: "ollama", LLMModel: "llama3.2"})
require.NoError(t, err)
_, isJudge := client.(llmclient.Judge)
require.False(t, isJudge, "chat providers keep the Complete path")
}
func TestJevExtractorDoesNotSpendCallBudget(t *testing.T) {
t.Setenv("LLM_API_KEY", "key")
// ProxyFromEnvironment caches the first request in the process. Match the
// proxy the interactsh client test expects, and keep the local server direct.
for _, key := range []string{"HTTP_PROXY", "http_proxy", "ALL_PROXY", "all_proxy", "https_proxy", "no_proxy"} {
t.Setenv(key, "")
}
t.Setenv("HTTPS_PROXY", "http://127.0.0.1:9")
t.Setenv("NO_PROXY", "127.0.0.1,localhost")
var calls atomic.Int32
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
calls.Add(1)
_, _ = w.Write([]byte(`{"model":"jev-1.14.0","answers":{"q":{"type":"noul","noul":0.9}}}`))
}))
t.Cleanup(server.Close)
client, err := configureLLM(&types.Options{
EnableLLM: true,
LLMProvider: "jev",
LLMBaseURL: server.URL + "/v1",
LLMModel: "jev-latest",
LLMMaxCalls: 1,
})
require.NoError(t, err)
_, err = client.Complete(context.Background(), "extract the title", true)
require.ErrorContains(t, err, "cannot run extractors")
_, err = client.Complete(context.Background(), "extract another", true)
require.Error(t, err)
require.Zero(t, calls.Load())
judge := client.(llmclient.Judge)
verdict, err := judge.Judge(context.Background(), "login page?", nil, []string{"<form>"})
require.NoError(t, err)
require.Equal(t, "yes", verdict.Label)
require.EqualValues(t, 1, calls.Load())
}
+4 -15
View File
@@ -228,25 +228,14 @@ func ValidateOptions(options *types.Options) error {
}
}
// verify that a valid ip version type was selected (4, 6)
if len(options.IPVersion) == 0 {
// add ipv4 as default
options.IPVersion = append(options.IPVersion, "4")
}
var useIPV4, useIPV6 bool
// An empty -iv leaves one unpinned target so the dialer can fall back
// between address families. -sa resolves and scans every selected address.
// Explicit -iv 4, -iv 6 and -iv 4,6 retain their usual selection semantics.
for _, ipv := range options.IPVersion {
switch ipv {
case "4":
useIPV4 = true
case "6":
useIPV6 = true
default:
if ipv != "4" && ipv != "6" {
return fmt.Errorf("unsupported ip version: %s", ipv)
}
}
if !useIPV4 && !useIPV6 {
return errors.New("ipv4 and/or ipv6 must be selected")
}
return nil
}
+12
View File
@@ -90,3 +90,15 @@ func TestReadsTargetsFromStdin(t *testing.T) {
})
}
}
// The default must remain empty so the provider can distinguish automatic
// IPv6 preference from explicit user-supplied -iv selections.
func TestValidateOptionsIPVersionPreference(t *testing.T) {
automatic := &types.Options{}
require.NoError(t, ValidateOptions(automatic))
require.Empty(t, automatic.IPVersion)
for _, versions := range []goflags.StringSlice{{"4"}, {"6"}, {"4", "6"}} {
require.NoError(t, ValidateOptions(&types.Options{IPVersion: versions}))
}
require.ErrorContains(t, ValidateOptions(&types.Options{IPVersion: goflags.StringSlice{"7"}}), "unsupported ip version")
}
+7
View File
@@ -67,11 +67,13 @@ import (
"github.com/projectdiscovery/nuclei/v3/pkg/protocols/common/utils/excludematchers"
"github.com/projectdiscovery/nuclei/v3/pkg/protocols/headless/engine"
httpProtocol "github.com/projectdiscovery/nuclei/v3/pkg/protocols/http"
"github.com/projectdiscovery/nuclei/v3/pkg/protocols/http/catchall"
"github.com/projectdiscovery/nuclei/v3/pkg/protocols/http/httpclientpool"
"github.com/projectdiscovery/nuclei/v3/pkg/reporting"
"github.com/projectdiscovery/nuclei/v3/pkg/templates"
"github.com/projectdiscovery/nuclei/v3/pkg/types"
"github.com/projectdiscovery/nuclei/v3/pkg/utils"
"github.com/projectdiscovery/nuclei/v3/pkg/utils/httpbody"
"github.com/projectdiscovery/nuclei/v3/pkg/utils/stats"
"github.com/projectdiscovery/nuclei/v3/pkg/utils/yaml"
"github.com/projectdiscovery/retryablehttp-go"
@@ -675,6 +677,7 @@ func (r *Runner) RunEnumeration() error {
Catalog: r.catalog,
IssuesClient: r.issuesClient,
RateLimiter: r.rateLimiter,
RequestBodyBudget: httpbody.NewBudget(int64(r.options.MaxRequestBodyInFlight)),
Interactsh: r.interactsh,
ProjectFile: r.projectFile,
Browser: r.browser,
@@ -734,6 +737,9 @@ func (r *Runner) RunEnumeration() error {
if r.options.HostBackoff {
executorOpts.HostBackoff = hostbackoff.New(hostbackoff.Config{Max: r.options.HostBackoffMax})
}
if r.options.AutoCalibrate {
executorOpts.CatchAll = catchall.New(0)
}
executorEngine := core.New(r.options)
executorEngine.SetExecuterOptions(executorOpts)
@@ -1072,6 +1078,7 @@ func (r *Runner) displayExecutionInfo(store *loader.Store) {
stats.DisplayAsWarning(httpProtocol.SetThreadToCountZero)
stats.ForceDisplayWarning(templates.SkippedUnverifiedTemplateStats)
stats.ForceDisplayWarning(templates.SkippedRequestSignatureTemplateStats)
stats.ForceDisplayWarning(templates.SkippedWorkflowStepStats)
cfg := config.DefaultConfig
+66
View File
@@ -6,11 +6,17 @@ import (
"net/http/httptest"
"os"
"path/filepath"
"strconv"
"sync/atomic"
"testing"
"time"
"github.com/miekg/dns"
"github.com/projectdiscovery/gologger"
"github.com/stretchr/testify/require"
"github.com/projectdiscovery/nuclei/v3/pkg/input"
"github.com/projectdiscovery/nuclei/v3/pkg/input/provider"
"github.com/projectdiscovery/nuclei/v3/pkg/model/types/severity"
"github.com/projectdiscovery/nuclei/v3/pkg/protocols/common/protocolstate"
"github.com/projectdiscovery/nuclei/v3/pkg/types"
@@ -197,3 +203,63 @@ func TestNewInteractshHTTPClient(t *testing.T) {
require.NotNil(t, proxyURL)
require.Equal(t, "http://127.0.0.1:9", proxyURL.String())
}
func TestInitializeTemplatesHTTPInputUsesResolvers(t *testing.T) {
for _, key := range []string{"HTTP_PROXY", "http_proxy", "HTTPS_PROXY", "https_proxy"} {
t.Setenv(key, "")
}
listener, err := net.ListenPacket("udp", "127.0.0.1:0")
require.NoError(t, err)
queries := new(atomic.Int32)
server := &dns.Server{PacketConn: listener, Handler: dns.HandlerFunc(func(writer dns.ResponseWriter, request *dns.Msg) {
queries.Add(1)
response := new(dns.Msg).SetReply(request)
for _, question := range request.Question {
if question.Qtype == dns.TypeA {
response.Answer = append(response.Answer, &dns.A{Hdr: dns.RR_Header{Name: question.Name, Rrtype: dns.TypeA, Class: dns.ClassINET}, A: net.ParseIP("127.0.0.1")})
}
}
_ = writer.WriteMsg(response)
})}
started := make(chan struct{})
server.NotifyStartedFunc = func() { close(started) }
done := make(chan error, 1)
go func() { done <- server.ActivateAndServe() }()
<-started
t.Cleanup(func() { require.NoError(t, server.Shutdown()); require.NoError(t, <-done) })
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
w.WriteHeader(http.StatusOK)
}))
t.Cleanup(srv.Close)
resolverFile := filepath.Join(t.TempDir(), "resolv.conf")
require.NoError(t, os.WriteFile(resolverFile, nil, 0o600))
t.Setenv("RESOLVERS_PATH", resolverFile)
options := &types.Options{
ExecutionId: t.Name(),
Timeout: 5,
BulkSize: 1,
InternalResolversList: []string{listener.LocalAddr().String()},
}
require.NoError(t, protocolstate.Init(options))
t.Cleanup(func() { protocolstate.Close(options.ExecutionId) })
target := "probe.nuclei.test:" + strconv.Itoa(srv.Listener.Addr().(*net.TCPAddr).Port)
r := &Runner{
options: options,
inputProvider: provider.NewSimpleInputProviderWithUrls(options.ExecutionId, target),
Logger: &gologger.Logger{},
}
hm, err := r.initializeTemplatesHTTPInput()
require.NoError(t, err)
t.Cleanup(func() { _ = hm.Close() })
probed, ok := hm.Get(input.ProbeKey(target, ""))
require.True(t, ok, "the probe should resolve the target through the -r resolver")
require.Equal(t, "http://"+target, string(probed))
require.Positive(t, queries.Load())
}
+43 -8
View File
@@ -6,13 +6,15 @@ import (
"sort"
"strings"
"github.com/alecthomas/chroma/quick"
"github.com/alecthomas/chroma/v2/quick"
"github.com/logrusorgru/aurora/v4"
"github.com/projectdiscovery/nuclei/v3/pkg/catalog/config"
"github.com/projectdiscovery/nuclei/v3/pkg/catalog/loader"
"github.com/projectdiscovery/nuclei/v3/pkg/model"
"github.com/projectdiscovery/nuclei/v3/pkg/templates"
"github.com/projectdiscovery/nuclei/v3/pkg/types"
"github.com/projectdiscovery/nuclei/v3/pkg/utils"
"github.com/projectdiscovery/nuclei/v3/pkg/utils/json"
)
@@ -38,12 +40,17 @@ func (r *Runner) verboseTemplate(tpl *templates.Template) {
tpl.Info.SeverityHolder.Severity))
}
// listedTemplate names its fields like a result event, so a -jsonl listing
// joins with scan results on the same keys.
type listedTemplate struct {
Template string `json:"template,omitempty"`
TemplateID string `json:"template-id"`
TemplatePath string `json:"template-path"`
Info model.Info `json:"info"`
}
func (r *Runner) listAvailableStoreTemplates(store *loader.Store) {
r.Logger.Print().Msgf(
"\nListing available %v nuclei templates for %v",
config.DefaultConfig.TemplateVersion,
config.DefaultConfig.TemplatesDirectory,
)
r.listingHeader("templates")
// order templates alphabetically by path
templates := store.Templates()
sort.Slice(templates, func(i, j int) bool {
@@ -51,6 +58,10 @@ func (r *Runner) listAvailableStoreTemplates(store *loader.Store) {
})
for _, tpl := range templates {
if r.options.JSONL && !r.options.TemplateDisplay {
r.listTemplateJSON(tpl)
continue
}
if hasExtraFlags(r.options) {
if r.options.TemplateDisplay {
colorize := !r.options.NoColor
@@ -78,12 +89,36 @@ func (r *Runner) listAvailableStoreTemplates(store *loader.Store) {
}
}
func (r *Runner) listAvailableTags(tagsMap map[string]int) {
func (r *Runner) listTemplateJSON(tpl *templates.Template) {
path, _ := utils.TemplatePathURL(tpl.Path, tpl.ID, "")
marshalled, err := json.Marshal(listedTemplate{
Template: path,
TemplateID: tpl.ID,
TemplatePath: tpl.Path,
Info: tpl.Info,
})
if err != nil {
r.Logger.Error().Msgf("Could not marshal the template %s: %s", tpl.Path, err)
return
}
r.Logger.Print().Msgf("%s", marshalled)
}
// listingHeader is skipped for -jsonl, where every line must parse as JSON.
func (r *Runner) listingHeader(what string) {
if r.options.JSONL {
return
}
r.Logger.Print().Msgf(
"\nListing available %v nuclei tags for %v",
"\nListing available %v nuclei %s for %v",
config.DefaultConfig.TemplateVersion,
what,
config.DefaultConfig.TemplatesDirectory,
)
}
func (r *Runner) listAvailableTags(tagsMap map[string]int) {
r.listingHeader("tags")
type kv struct {
Key string `json:"tag"`
+76
View File
@@ -0,0 +1,76 @@
package runner
import (
"os"
"path/filepath"
"strings"
"testing"
"github.com/projectdiscovery/gologger"
"github.com/projectdiscovery/gologger/formatter"
"github.com/projectdiscovery/gologger/levels"
"github.com/projectdiscovery/nuclei/v3/internal/tests/testutils"
"github.com/projectdiscovery/nuclei/v3/pkg/catalog/config"
"github.com/projectdiscovery/nuclei/v3/pkg/catalog/disk"
"github.com/projectdiscovery/nuclei/v3/pkg/catalog/loader"
"github.com/projectdiscovery/nuclei/v3/pkg/templates"
"github.com/projectdiscovery/nuclei/v3/pkg/utils/json"
"github.com/stretchr/testify/require"
)
const listedTemplateYAML = `id: listed-template
info:
name: Listed Template
author: pdteam
severity: info
tags: listing
http:
- method: GET
path:
- "{{BaseURL}}"
`
func TestTemplateListWritesJSONLines(t *testing.T) {
root := t.TempDir()
cfg := config.DefaultConfig
oldRoot := cfg.TemplatesDirectory
t.Cleanup(func() { cfg.SetTemplatesDir(oldRoot) })
cfg.SetTemplatesDir(root)
path := filepath.Join(root, "http", "listed-template.yaml")
require.NoError(t, os.MkdirAll(filepath.Dir(path), 0o755))
require.NoError(t, os.WriteFile(path, []byte(listedTemplateYAML), 0o600))
writer := &ignoreFileLogWriter{}
logger := &gologger.Logger{}
logger.SetFormatter(formatter.NewCLI(true))
logger.SetWriter(writer)
logger.SetMaxLevel(levels.LevelInfo)
options := testutils.DefaultOptions.Copy()
options.ExecutionId = t.Name()
options.Logger = logger
options.Templates = []string{path}
options.TemplateList = true
options.JSONL = true
testutils.Init(options)
t.Cleanup(func() { testutils.Cleanup(options) })
executerOpts := testutils.NewMockExecuterOptions(options, nil)
executerOpts.Parser = templates.NewParser()
store, err := loader.New(loader.NewConfig(options, disk.NewCatalog(root), executerOpts))
require.NoError(t, err)
require.NoError(t, store.LoadTemplatesOnlyMetadata())
require.Len(t, store.Templates(), 1)
r := &Runner{options: options, Logger: logger}
r.listAvailableStoreTemplates(store)
require.Len(t, writer.records, 1, "JSONL output carries no banner line")
var listed map[string]any
require.NoError(t, json.Unmarshal([]byte(strings.TrimSpace(writer.records[0].message)), &listed))
require.Equal(t, "listed-template", listed["template-id"])
require.Equal(t, filepath.Join("http", "listed-template.yaml"), listed["template"])
require.Equal(t, "Listed Template", listed["info"].(map[string]any)["name"])
}
+4
View File
@@ -33,6 +33,7 @@ import (
"github.com/projectdiscovery/nuclei/v3/pkg/protocols/common/interactsh"
"github.com/projectdiscovery/nuclei/v3/pkg/protocols/common/utils/excludematchers"
browserEngine "github.com/projectdiscovery/nuclei/v3/pkg/protocols/headless/engine"
"github.com/projectdiscovery/nuclei/v3/pkg/protocols/http/catchall"
"github.com/projectdiscovery/nuclei/v3/pkg/reporting"
"github.com/projectdiscovery/nuclei/v3/pkg/templates"
"github.com/projectdiscovery/nuclei/v3/pkg/types"
@@ -110,6 +111,9 @@ func newNucleiExecutor(opts *NucleiExecutorOptions) (*nucleiExecutor, error) {
if opts.Options.HostBackoff {
executorOpts.HostBackoff = hostbackoff.New(hostbackoff.Config{Max: opts.Options.HostBackoffMax})
}
if opts.Options.AutoCalibrate {
executorOpts.CatchAll = catchall.New(0)
}
executorEngine := core.New(opts.Options)
executorEngine.SetExecuterOptions(executorOpts)
+57
View File
@@ -102,6 +102,7 @@ var httpTestcases = []integrationCase{
{Path: "protocols/http/raw-unsafe-path-single-slash.yaml", TestCase: &httpRawUnsafePathSingleSlash{}},
{Path: "protocols/http/disable-http-cache.yaml", TestCase: &httpDisableCache{}},
{Path: "protocols/http/http-cache.yaml", TestCase: &httpCache{}},
{Path: "protocols/http/get.yaml", TestCase: &httpExcludeHostsFileComments{}},
}
type httpMultiVarSharing struct{}
@@ -1983,3 +1984,59 @@ func (h *httpDisableCache) Execute(filePath string) error {
return nil
}
// ensure that comments in an -exclude-hosts file are ignored
type httpExcludeHostsFileComments struct{}
// Execute executes a test case and returns an error if occurred
func (h *httpExcludeHostsFileComments) Execute(filePath string) error {
router := httprouter.New()
router.GET("/", func(w http.ResponseWriter, r *http.Request, _ httprouter.Params) {
_, _ = fmt.Fprintf(w, "This is test matcher text")
})
ts := httptest.NewServer(router)
defer ts.Close()
excludeHosts := func(content string) ([]string, error) {
file, err := os.CreateTemp("", "exclude-hosts-*.txt")
if err != nil {
return nil, err
}
defer func() { _ = os.Remove(file.Name()) }()
_, err = file.WriteString(content)
_ = file.Close()
if err != nil {
return nil, err
}
return testutils.RunNucleiTemplateAndGetResults(filePath, ts.URL, debug, "-eh", file.Name())
}
// a comment line is not a host, even when it reads as an invalid regex
results, err := excludeHosts("# hosts out of scope (see ticket\n203.0.113.10\n")
if err != nil {
return fmt.Errorf("comment line: %w", err)
}
if err := expectResultsCount(results, 1); err != nil {
return fmt.Errorf("comment line: %w", err)
}
// a trailing comment does not stop the host from being excluded
results, err = excludeHosts("127.0.0.1 # test server\n")
if err != nil {
return fmt.Errorf("trailing comment: %w", err)
}
if err := expectResultsCount(results, 0); err != nil {
return fmt.Errorf("trailing comment: %w", err)
}
// comma separated hosts are still split when passed on the command line
results, err = testutils.RunNucleiTemplateAndGetResults(filePath, ts.URL, debug, "-eh", "203.0.113.10, 127.0.0.1")
if err != nil {
return fmt.Errorf("comma separated: %w", err)
}
if err := expectResultsCount(results, 0); err != nil {
return fmt.Errorf("comma separated: %w", err)
}
return nil
}
@@ -0,0 +1,89 @@
//go:build integration
// +build integration
package integration_test
import (
"fmt"
"net"
"time"
"github.com/ory/dockertest/v3"
)
// ipmiSimSetup runs OpenIPMI's BMC simulator with an anonymous user,
// ADMIN/ADMIN and root/calvin, and cipher suite 0 enabled.
const ipmiSimSetup = `apt-get update -qq && apt-get install -y -qq --no-install-recommends openipmi >/dev/null && mkdir -p /etc/ipmi && cat > /etc/ipmi/lan.conf <<'EOF'
name "bmc"
set_working_mc 0x20
startlan 1
addr 0.0.0.0 623
priv_limit admin
allowed_auths_callback none md2 md5 straight
allowed_auths_user none md2 md5 straight
allowed_auths_operator none md2 md5 straight
allowed_auths_admin none md2 md5 straight
guid a123456789abcdefa123456789abcdef
endlan
user 1 true "" "" user 10 none md2 md5 straight
user 2 true "ADMIN" "ADMIN" admin 10 none md2 md5 straight
user 3 true "root" "calvin" admin 10 none md2 md5 straight
EOF
printf 'mc_setbmc 0x20\nmc_add 0x20 0 no-device-sdrs 0x23 9 8 0x9f 0x1291 0xf02 persist_sdr\nmc_enable 0x20\n' > /etc/ipmi/emu.cmd
exec ipmi_sim -c /etc/ipmi/lan.conf -f /etc/ipmi/emu.cmd -n`
type javascriptIPMI struct{}
func (j *javascriptIPMI) Execute(filePath string) error {
pool, err := dockertest.NewPool("")
if err != nil {
return fmt.Errorf("could not create docker pool: %w", err)
}
if err := pool.Client.Ping(); err != nil {
return fmt.Errorf("could not connect to Docker: %w", err)
}
resource, err := pool.RunWithOptions(&dockertest.RunOptions{
Repository: "debian",
Tag: "bookworm-slim",
Platform: "linux/amd64",
Cmd: []string{"sh", "-c", ipmiSimSetup},
ExposedPorts: []string{"623/udp"},
})
if err != nil {
return fmt.Errorf("could not start ipmi simulator: %w", err)
}
defer purge(pool, resource)
address := net.JoinHostPort("127.0.0.1", resource.GetPort("623/udp"))
if err := waitForServiceCheck(address, 3*time.Minute, rmcpPingCheck); err != nil {
return err
}
results, err := runSignedNucleiTemplateAndGetResults(filePath, address, debug)
if err != nil {
return err
}
return expectResultsCount(results, 1)
}
// rmcpPingCheck waits for an answer to an RMCP presence ping, since the
// port is udp and cannot be probed with a connect.
func rmcpPingCheck(address string) error {
conn, err := net.Dial("udp", address)
if err != nil {
return err
}
defer func() { _ = conn.Close() }()
if _, err := conn.Write([]byte{0x06, 0x00, 0xff, 0x06, 0x00, 0x00, 0x11, 0xbe, 0x80, 0x00, 0x00, 0x00}); err != nil {
return err
}
_ = conn.SetReadDeadline(time.Now().Add(time.Second))
reply := make([]byte, 64)
n, err := conn.Read(reply)
if err != nil {
return err
}
if n < 9 || reply[8] != 0x40 {
return fmt.Errorf("unexpected rmcp reply")
}
return nil
}
@@ -46,6 +46,7 @@ var jsTestcases = []integrationCase{
{Path: "protocols/javascript/http-get.yaml", TestCase: &javascriptHTTPGet{}},
{Path: "protocols/javascript/http-client-flow.yaml", TestCase: &javascriptHTTPClientFlow{}},
{Path: "protocols/javascript/http-denied.yaml", TestCase: &javascriptHTTPDenied{}},
{Path: "protocols/javascript/ipmi.yaml", TestCase: &javascriptIPMI{}, DisableOn: javascriptDockerDisabled, Serial: true},
{Path: "protocols/javascript/rsync-test.yaml", TestCase: &javascriptRsyncTest{}, DisableOn: javascriptDockerDisabled, Serial: true},
{Path: "protocols/javascript/vnc-pass-brute.yaml", TestCase: &javascriptVncPassBrute{}, DisableOn: javascriptDockerDisabled, Serial: true},
{Path: "protocols/javascript/postgres-pass-brute.yaml", TestCase: &javascriptPostgresPassBrute{}, DisableOn: javascriptDockerDisabled, Serial: true},
@@ -0,0 +1,46 @@
id: ipmi-checks
info:
name: IPMI Checks
author: pdteam
severity: info
javascript:
- code: |
const ipmi = require('nuclei/ipmi');
const caps = ipmi.GetAuthCapabilities(Host, Port);
const root = ipmi.GetRAKPHash(Host, Port, 'root');
// cipher suite 0 lets a wrong password in
const anyPassword = new ipmi.Client();
const cipherZero = anyPassword.Login(Host, Port, 'ADMIN', 'wrong', 0);
anyPassword.Close();
// an encrypted, signed session, then Get Device ID and admin privilege
const client = new ipmi.Client();
client.Login(Host, Port, 'ADMIN', 'ADMIN', 3);
const device = client.Command(0x06, 0x01, []);
const admin = client.Command(0x06, 0x3b, [0x04]);
client.Close();
const manufacturer = device.Data[6] | device.Data[7] << 8 | device.Data[8] << 16;
let wrongRejected = false;
try { new ipmi.Client().Login(Host, Port, 'ADMIN', 'wrong', 3); } catch (e) { wrongRejected = true; }
ipmi.Ping(Host, Port) && caps.IPMI20 && caps.AnonymousLogin &&
ipmi.GetCipherSuites(Host, Port).includes(0) &&
ipmi.GetSystemGUID(Host, Port).length == 32 &&
ipmi.FindRAKPPassword(root, ['admin', 'calvin']) == 'calvin' &&
!ipmi.CheckRAKPPassword(root, 'admin') &&
cipherZero && wrongRejected &&
device.CompletionCode == 0 && manufacturer == 4753 &&
admin.CompletionCode == 0;
args:
Host: "{{Host}}"
Port: "623"
matchers:
- type: dsl
dsl:
- "success == true"
+8 -6
View File
@@ -39,12 +39,14 @@ func createEphemeralObjects(ctx context.Context, base *NucleiEngine, opts *types
Catalog: base.catalog,
IssuesClient: base.rc,
RateLimiter: base.rateLimiter,
Interactsh: base.interactshClient,
Colorizer: aurora.New(aurora.WithColors(true)),
ResumeCfg: types.NewResumeCfg(),
Parser: base.parser,
Browser: base.browserInstance,
DoNotCache: opts.DoNotCacheTemplates,
RequestBodyBudget: base.executerOpts.RequestBodyBudget,
Interactsh: base.interactshClient,
Colorizer: aurora.New(aurora.WithColors(true)),
ResumeCfg: types.NewResumeCfg(),
Parser: base.parser,
Browser: base.browserInstance,
DoNotCache: opts.DoNotCacheTemplates,
}
if base.interactshClient != nil {
u.executerOpts.InteractshScope = base.interactshClient.NewRequestScope()
+5
View File
@@ -36,6 +36,7 @@ import (
"github.com/projectdiscovery/nuclei/v3/pkg/templates"
"github.com/projectdiscovery/nuclei/v3/pkg/types"
nucleiUtils "github.com/projectdiscovery/nuclei/v3/pkg/utils"
"github.com/projectdiscovery/nuclei/v3/pkg/utils/httpbody"
"github.com/projectdiscovery/ratelimit"
pdcpauth "github.com/projectdiscovery/utils/auth/pdcp"
)
@@ -279,6 +280,7 @@ func (e *NucleiEngine) init(ctx context.Context) error {
Catalog: e.catalog,
IssuesClient: e.rc,
RateLimiter: e.rateLimiter,
RequestBodyBudget: httpbody.NewBudget(int64(e.opts.MaxRequestBodyInFlight)),
Interactsh: e.interactshClient,
Colorizer: aurora.New(aurora.WithColors(true)),
ResumeCfg: types.NewResumeCfg(),
@@ -358,6 +360,9 @@ func (e *NucleiEngine) init(ctx context.Context) error {
httpxOptions := httpx.DefaultOptions
httpxOptions.Timeout = 5 * time.Second
if len(e.opts.InternalResolversList) > 0 {
httpxOptions.Resolvers = e.opts.InternalResolversList
}
if client, err := httpx.New(&httpxOptions); err != nil {
return err
} else {
+12 -4
View File
@@ -7,18 +7,22 @@ import (
"net/http"
"os"
"path/filepath"
"regexp"
"strings"
"github.com/alecthomas/chroma/quick"
"github.com/alecthomas/chroma/v2/quick"
"github.com/projectdiscovery/nuclei/v3/pkg/catalog/config"
"github.com/projectdiscovery/nuclei/v3/pkg/types"
"github.com/projectdiscovery/retryablehttp-go"
pdcpauth "github.com/projectdiscovery/utils/auth/pdcp"
"github.com/projectdiscovery/utils/errkit"
)
const (
var (
aiTemplateGeneratorAPIEndpoint = "https://api.projectdiscovery.io/v1/template/ai"
// aiTemplateHTTPClient verifies TLS, since the request carries the PDCP API key.
aiTemplateHTTPClient = &http.Client{}
// aiTemplateIDRegex keeps the template file name inside the pdcp directory.
aiTemplateIDRegex = regexp.MustCompile(`^[A-Za-z0-9_-]+$`)
)
type AITemplateResponse struct {
@@ -112,7 +116,7 @@ func generateAITemplate(prompt string) (string, string, error) {
req.Header.Set("Content-Type", "application/json")
req.Header.Set(pdcpauth.ApiKeyHeaderName, creds.APIKey)
resp, err := retryablehttp.DefaultClient().Do(req)
resp, err := aiTemplateHTTPClient.Do(req)
if err != nil {
return "", "", errkit.Newf("Failed to send HTTP request: %v", err)
}
@@ -138,5 +142,9 @@ func generateAITemplate(prompt string) (string, string, error) {
return "", "", errkit.Newf("Failed to generate template")
}
if !aiTemplateIDRegex.MatchString(result.TemplateID) {
return "", "", errkit.Newf("Invalid template ID in API response: %q", result.TemplateID)
}
return result.Completion, result.TemplateID, nil
}
+55
View File
@@ -0,0 +1,55 @@
package loader
import (
"encoding/json"
"net/http"
"net/http/httptest"
"sync/atomic"
"testing"
"github.com/stretchr/testify/require"
)
func newAITemplateServer(t *testing.T, templateID string, gotKey *atomic.Value) {
srv := httptest.NewTLSServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
gotKey.Store(r.Header.Get("X-Api-Key"))
_ = json.NewEncoder(w).Encode(map[string]string{
"completion": "id: test\n",
"template_id": templateID,
})
}))
t.Cleanup(srv.Close)
oldEndpoint, oldClient := aiTemplateGeneratorAPIEndpoint, aiTemplateHTTPClient
t.Cleanup(func() { aiTemplateGeneratorAPIEndpoint, aiTemplateHTTPClient = oldEndpoint, oldClient })
aiTemplateGeneratorAPIEndpoint = srv.URL
t.Setenv("PDCP_API_KEY", "test-key")
if templateID != "" {
// Trust the test certificate, to check the template_id only.
aiTemplateHTTPClient = srv.Client()
}
}
func TestAITemplateVerifiesTLS(t *testing.T) {
var gotKey atomic.Value
newAITemplateServer(t, "", &gotKey)
_, _, err := generateAITemplate("check for exposed admin panels")
require.ErrorContains(t, err, "certificate")
require.Nil(t, gotKey.Load(), "the API key went to an untrusted server")
}
func TestAITemplateRejectsUnsafeID(t *testing.T) {
for _, id := range []string{"../../.config/nuclei/config", "a/b", `a\b`, "a.b", "a b"} {
var gotKey atomic.Value
newAITemplateServer(t, id, &gotKey)
_, _, err := generateAITemplate("check for exposed admin panels")
require.Error(t, err, id)
}
var gotKey atomic.Value
newAITemplateServer(t, "Abc-123_x", &gotKey)
_, templateID, err := generateAITemplate("check for exposed admin panels")
require.NoError(t, err)
require.Equal(t, "Abc-123_x", templateID)
}
+10 -4
View File
@@ -631,10 +631,10 @@ func (store *Store) noteMissingCapabilities(templatePath string, missingCaps []t
// erroring out on discovering any faulty templates.
func (store *Store) ValidateTemplates() error {
templatePaths, errs := store.config.Catalog.GetTemplatesPath(store.finalTemplates)
store.logErroredTemplates(errs)
notFound := store.logErroredTemplates(errs)
workflowPaths, errs := store.config.Catalog.GetTemplatesPath(store.finalWorkflows)
store.logErroredTemplates(errs)
notFound += store.logErroredTemplates(errs)
templatePathsMap := make(map[string]struct{}, len(templatePaths))
for _, path := range templatePaths {
@@ -646,7 +646,8 @@ func (store *Store) ValidateTemplates() error {
workflowPathsMap[path] = struct{}{}
}
if store.areTemplatesValid(templatePathsMap) && store.areWorkflowsValid(workflowPathsMap) {
// a path that can't be resolved is not validated, so it can't count as valid
if store.areTemplatesValid(templatePathsMap) && store.areWorkflowsValid(workflowPathsMap) && notFound == 0 {
return nil
}
@@ -1056,10 +1057,15 @@ func workflowContainsProtocol(workflow []*workflows.WorkflowTemplate) bool {
return false
}
func (s *Store) logErroredTemplates(erred map[string]error) {
// logErroredTemplates logs the templates that could not be found and returns
// how many of them were not handled by the NotFoundCallback.
func (s *Store) logErroredTemplates(erred map[string]error) int {
notFound := 0
for template, err := range erred {
if s.NotFoundCallback == nil || !s.NotFoundCallback(template) {
s.logger.Error().Msgf("Could not find template '%s': %s", template, err)
notFound++
}
}
return notFound
}
+43 -20
View File
@@ -176,25 +176,8 @@ func TestLoadTemplatesOnlyMetadataLogsCachedTemplateParseErrors(t *testing.T) {
require.Contains(t, output.String(), "Could not load template")
}
func TestValidateTemplatesRejectsRuntimeCompilationError(t *testing.T) {
templatePath := filepath.Join(t.TempDir(), "invalid-regex-group.yaml")
require.NoError(t, os.WriteFile(templatePath, []byte(`id: invalid-regex-group
info:
name: Invalid Regex Group
author: pdteam
severity: info
http:
- method: GET
path:
- "{{BaseURL}}"
extractors:
- type: regex
group: -1
regex:
- "(a)(b)"
`), 0o600))
func newValidateStore(t *testing.T, executionID, templatePath string) (*Store, *bytes.Buffer) {
t.Helper()
options := testutils.DefaultOptions.Copy()
var output bytes.Buffer
@@ -203,7 +186,7 @@ http:
logger.SetWriter(&utils.CaptureWriter{Buffer: &output})
logger.SetMaxLevel(levels.LevelDebug)
options.Logger = logger
options.ExecutionId = "loader-invalid-regex-group"
options.ExecutionId = executionID
options.Templates = []string{templatePath}
options.Validate = true
options.TemplateLoadingConcurrency = 1
@@ -226,10 +209,50 @@ http:
store, err := New(NewConfig(options, catalog, executerOpts))
require.NoError(t, err)
return store, &output
}
func TestValidateTemplatesRejectsRuntimeCompilationError(t *testing.T) {
templatePath := filepath.Join(t.TempDir(), "invalid-regex-group.yaml")
require.NoError(t, os.WriteFile(templatePath, []byte(`id: invalid-regex-group
info:
name: Invalid Regex Group
author: pdteam
severity: info
http:
- method: GET
path:
- "{{BaseURL}}"
extractors:
- type: regex
group: -1
regex:
- "(a)(b)"
`), 0o600))
store, output := newValidateStore(t, "loader-invalid-regex-group", templatePath)
require.Error(t, store.ValidateTemplates())
require.Contains(t, output.String(), "regex extractor group must be >= 0")
}
func TestValidateTemplatesRejectsMissingTemplatePath(t *testing.T) {
templatePath := filepath.Join(t.TempDir(), "no-such-templates")
store, output := newValidateStore(t, "loader-missing-template-path", templatePath)
require.Error(t, store.ValidateTemplates())
require.Contains(t, output.String(), "Could not find template")
}
func TestValidateTemplatesAcceptsMissingPathHandledByCallback(t *testing.T) {
templatePath := filepath.Join(t.TempDir(), "no-such-templates")
store, _ := newValidateStore(t, "loader-missing-path-callback", templatePath)
store.NotFoundCallback = func(string) bool { return true }
require.NoError(t, store.ValidateTemplates())
}
func TestRemoteTemplates(t *testing.T) {
catalog := disk.NewCatalog("")
+47 -7
View File
@@ -3,6 +3,7 @@ package core
import (
"fmt"
"net/http/cookiejar"
"sync"
"sync/atomic"
"github.com/projectdiscovery/gologger"
@@ -32,22 +33,45 @@ func (e *Engine) executeWorkflow(ctx *scan.ScanContext, w *workflows.Workflow) b
templateThreads++
}
swg, _ := syncutil.New(syncutil.WithSize(templateThreads))
// pending tracks every started step until it returns. swg only limits
// concurrency and its slots can be released early, so completion is
// tracked here instead.
var pending sync.WaitGroup
for _, template := range w.Workflows {
newCtx := scan.NewScanContext(ctx.Context(), ctx.Input.Clone())
if err := e.runWorkflowStep(template, newCtx, results, swg, w); err != nil {
// Root steps do not own a waitgroup slot.
if err := e.runWorkflowStep(template, newCtx, results, swg, &pending, w, nil); err != nil {
gologger.Warning().Msgf(workflowStepExecutionError, template.Template, err)
}
}
swg.Wait()
pending.Wait()
return results.Load()
}
// slotRelease returns a func that gives back one swg slot. It is safe to call
// from several goroutines and several times; only the first call releases.
func slotRelease(swg *syncutil.AdaptiveWaitGroup) func() {
var once sync.Once
return func() {
once.Do(swg.Done)
}
}
// runWorkflowStep runs a workflow step for the workflow. It executes the workflow
// in a recursive manner running all subtemplates and matchers.
func (e *Engine) runWorkflowStep(template *workflows.WorkflowTemplate, ctx *scan.ScanContext, results *atomic.Bool, swg *syncutil.AdaptiveWaitGroup, w *workflows.Workflow) error {
//
// releaseSlot, when non-nil, releases the AdaptiveWaitGroup slot held by the
// goroutine that is running this step. It must be called before this step
// blocks on swg.AddWithContext for child steps; otherwise every in-flight
// parent can hold a slot while waiting for another, which deadlocks when the
// number of matching nested steps reaches TemplateThreads (see #8074).
//
// pending is incremented for every child step before it starts, and each child
// calls Done when it returns, so the root can wait for the whole tree.
func (e *Engine) runWorkflowStep(template *workflows.WorkflowTemplate, ctx *scan.ScanContext, results *atomic.Bool, swg *syncutil.AdaptiveWaitGroup, pending *sync.WaitGroup, w *workflows.Workflow, releaseSlot func()) error {
var firstMatched bool
var err error
var mainErr error
@@ -131,18 +155,25 @@ func (e *Engine) runWorkflowStep(template *workflows.WorkflowTemplate, ctx *scan
continue
}
// Free this step's slot before blocking on child acquires.
if releaseSlot != nil {
releaseSlot()
}
for _, subtemplate := range matcher.Subtemplates {
if err := swg.AddWithContext(ctx.Context()); err != nil {
return
}
pending.Add(1)
go func(subtemplate *workflows.WorkflowTemplate) {
defer swg.Done()
defer pending.Done()
release := slotRelease(swg)
defer release()
// create a new context with the same input but with unset callbacks
// clone the Input so that other parallel executions won't overwrite the shared variables when subsequent templates are running
subCtx := scan.NewScanContext(ctx.Context(), ctx.Input.Clone())
if err := e.runWorkflowStep(subtemplate, subCtx, results, swg, w); err != nil {
if err := e.runWorkflowStep(subtemplate, subCtx, results, swg, pending, w, release); err != nil {
gologger.Warning().Msgf(workflowStepExecutionError, subtemplate.Template, err)
}
}(subtemplate)
@@ -162,18 +193,27 @@ func (e *Engine) runWorkflowStep(template *workflows.WorkflowTemplate, ctx *scan
return mainErr
}
if len(template.Subtemplates) > 0 && firstMatched {
// Free this step's slot before blocking on child acquires so nested
// matching steps cannot exhaust TemplateThreads and deadlock (#8074).
if releaseSlot != nil {
releaseSlot()
}
for _, subtemplate := range template.Subtemplates {
if err := swg.AddWithContext(ctx.Context()); err != nil {
return err
}
pending.Add(1)
go func(template *workflows.WorkflowTemplate) {
defer pending.Done()
release := slotRelease(swg)
defer release()
// create a new context with the same input but with unset callbacks
subCtx := scan.NewScanContext(ctx.Context(), ctx.Input.Clone())
if err := e.runWorkflowStep(template, subCtx, results, swg, w); err != nil {
if err := e.runWorkflowStep(template, subCtx, results, swg, pending, w, release); err != nil {
gologger.Warning().Msgf(workflowStepExecutionError, template.Template, err)
}
swg.Done()
}(subtemplate)
}
}
+83
View File
@@ -303,6 +303,89 @@ func TestWorkflowsSameStepExecutersRefreshTemplateContext(t *testing.T) {
require.Equal(t, "foo", secondWhoami, "could not refresh workflow context into later executers in the same step")
}
func TestWorkflowNestedSubtemplatesDoNotDeadlockAtConcurrencyLimit(t *testing.T) {
// Repro for #8074: with TemplateThreads equal to the number of matching
// nested parents, each parent used to hold a waitgroup slot while calling
// AddWithContext for its children, so every slot was occupied by a parent
// waiting for another slot and the workflow never finished.
progressBar, _ := progress.NewStatsTicker(0, false, false, false, 0)
matchOutputs := []*output.InternalWrappedEvent{{
OperatorsResult: &operators.Result{},
Results: []*output.ResultEvent{{}},
}}
const parents = 3
var entered atomic.Int32
allHolding := make(chan struct{})
releaseParents := make(chan struct{})
leafRuns := &atomic.Int32{}
leaf := &mockExecuter{result: true, executeScanHook: func(ctx *scan.ScanContext) {
leafRuns.Add(1)
}}
parentExec := &mockExecuter{
result: true,
outputs: matchOutputs,
executeScanHook: func(ctx *scan.ScanContext) {
if entered.Add(1) == parents {
close(allHolding)
}
// Stay inside ExecuteWithResults so the parent still owns its
// waitgroup slot while every peer also holds one.
<-releaseParents
},
}
mkParent := func() *workflows.WorkflowTemplate {
return &workflows.WorkflowTemplate{
Executers: []*workflows.ProtocolExecuterPair{{
Executer: parentExec,
Options: &protocols.ExecutorOptions{Progress: progressBar},
}},
Subtemplates: []*workflows.WorkflowTemplate{{
Executers: []*workflows.ProtocolExecuterPair{{
Executer: leaf,
Options: &protocols.ExecutorOptions{Progress: progressBar},
}},
}},
}
}
workflow := &workflows.Workflow{
Options: &protocols.ExecutorOptions{Options: &types.Options{TemplateThreads: parents}},
Workflows: []*workflows.WorkflowTemplate{{
Executers: []*workflows.ProtocolExecuterPair{{
Executer: &mockExecuter{result: true, outputs: matchOutputs},
Options: &protocols.ExecutorOptions{Progress: progressBar},
}},
Subtemplates: []*workflows.WorkflowTemplate{mkParent(), mkParent(), mkParent()},
}},
}
engine := &Engine{}
input := contextargs.NewWithInput(context.Background(), "https://test.com")
ctx := scan.NewScanContext(context.Background(), input)
done := make(chan bool, 1)
go func() {
done <- engine.executeWorkflow(ctx, workflow)
}()
select {
case <-allHolding:
case <-time.After(2 * time.Second):
t.Fatal("nested parents did not all acquire TemplateThreads slots")
}
close(releaseParents)
select {
case matched := <-done:
require.True(t, matched, "nested workflow should report a match")
require.EqualValues(t, parents, leafRuns.Load(), "each nested parent should run its leaf")
case <-time.After(3 * time.Second):
t.Fatal("nested workflow deadlocked at TemplateThreads == matching nested parents")
}
}
func TestWorkflowSubtemplatesUnblockCancelledAcquire(t *testing.T) {
progressBar, _ := progress.NewStatsTicker(0, false, false, false, 0)
var started atomic.Int32
+14 -18
View File
@@ -155,23 +155,14 @@ func (rule *Rule) Execute(input *ExecuteRuleInput) (err error) {
baseInteractURLs := append([]string(nil), input.InteractURLs...)
if rule.generator == nil {
for _, component := range finalComponentList {
var dataKeys map[string]struct{}
optionVars := rule.options.Options.Vars.AsMap()
constants := rule.options.Constants
dataValues := rule.options.NewVariablesScope(baseValues, optionVars, constants)
// get vars from variables while replacing interactsh urls
// Render variables once, allocating their interactsh URLs. The
// output may contain runtime data, so it is never rendered again.
evaluation := rule.options.Variables.EvaluateWithInteractshScope(dataValues, rule.options.Interactsh)
evaluatedValues, interactURLs := evaluation.Values, evaluation.InteractURLs
input.Values, dataKeys = mergeFuzzValueLayers(
fuzzValueLayer{values: evaluatedValues, kind: fuzzValueTemplate},
fuzzValueLayer{values: baseValues, kind: fuzzValueData},
fuzzValueLayer{values: optionVars, kind: fuzzValueData},
fuzzValueLayer{values: constants, kind: fuzzValueData},
)
// Replace template-text interactsh markers without evaluating runtime values.
input.Values, interactURLs = rule.evaluateVarsWithInteractsh(input.Values, interactURLs, dataKeys)
input.InteractURLs = mergeInteractURLs(baseInteractURLs, interactURLs)
input.Values = generators.MergeMaps(evaluation.Values, baseValues, optionVars, constants)
input.InteractURLs = mergeInteractURLs(baseInteractURLs, evaluation.InteractURLs)
err := rule.executeRuleValues(input, component)
if err != nil {
@@ -240,15 +231,17 @@ func (rule *Rule) prepareGeneratorValues(values, baseValues map[string]interface
evaluation := rule.options.Variables.EvaluateWithInteractshScope(evaluationValues, rule.options.Interactsh)
evaluatedValues, interactURLs := evaluation.Values, evaluation.InteractURLs
inputValues, dataKeys := mergeFuzzValueLayers(
fuzzValueLayer{values: evaluatedValues, kind: fuzzValueData},
fuzzValueLayer{values: values, kind: fuzzValueTemplate},
fuzzValueLayer{values: evaluatedValues, kind: fuzzValueTemplate},
fuzzValueLayer{values: baseValues, kind: fuzzValueData},
fuzzValueLayer{values: optionVars, kind: fuzzValueData},
fuzzValueLayer{values: constants, kind: fuzzValueData},
)
// Generator values are template-authored payload definitions until this
// render. Runtime/base, CLI option, and constant values remain data.
// render. Rendered variables, runtime/base, CLI option, and constant
// values remain data. Payloads layer over variables so a payload that
// shares a variable's name is still rendered.
return rule.evaluateVarsWithInteractsh(inputValues, interactURLs, dataKeys)
}
@@ -326,7 +319,7 @@ func (rule *Rule) evaluateVars(input string) (string, error) {
func (rule *Rule) evaluateVarsWithInteractsh(data map[string]interface{}, interactshUrls []string, dataKeys map[string]struct{}) (map[string]interface{}, []string) {
// Check if Interactsh options are configured
if rule.options.Interactsh != nil {
data = maps.Clone(data)
rendered := maps.Clone(data)
interactshUrlsMap := make(map[string]struct{})
for _, url := range interactshUrls {
@@ -335,7 +328,8 @@ func (rule *Rule) evaluateVarsWithInteractsh(data map[string]interface{}, intera
interactshUrls = mapsutil.GetKeys(interactshUrlsMap)
// Iterate through template-text data to replace Interactsh URL markers.
// Render each template-text value against the unrendered input, so
// the result does not depend on map iteration order.
for k, v := range data {
if _, ok := dataKeys[k]; ok {
continue
@@ -349,7 +343,7 @@ func (rule *Rule) evaluateVarsWithInteractsh(data map[string]interface{}, intera
InteractURLs: interactshUrls,
})
if err == nil {
data[k] = got.Text
rendered[k] = got.Text
for _, url := range got.InteractURLs {
if _, ok := interactshUrlsMap[url]; ok {
continue
@@ -360,6 +354,8 @@ func (rule *Rule) evaluateVarsWithInteractsh(data map[string]interface{}, intera
}
}
}
data = rendered
}
return data, interactshUrls
+224
View File
@@ -0,0 +1,224 @@
package fuzz
import (
"context"
"fmt"
"io"
"net/http"
"strings"
"testing"
"github.com/projectdiscovery/nuclei/v3/pkg/catalog/disk"
"github.com/projectdiscovery/nuclei/v3/pkg/fuzz/component"
"github.com/projectdiscovery/nuclei/v3/pkg/protocols"
"github.com/projectdiscovery/nuclei/v3/pkg/protocols/common/contextargs"
"github.com/projectdiscovery/nuclei/v3/pkg/protocols/common/generators"
"github.com/projectdiscovery/nuclei/v3/pkg/protocols/common/interactsh"
"github.com/projectdiscovery/nuclei/v3/pkg/protocols/common/variables"
"github.com/projectdiscovery/nuclei/v3/pkg/types"
"github.com/projectdiscovery/retryablehttp-go"
"github.com/stretchr/testify/require"
)
const (
// requestBodyWithDSL is request data that looks like a DSL expression.
requestBodyWithDSL = `PLAIN-{{to_upper("abc")}}-END`
queryRuleTarget = "http://127.0.0.1/echo?p=SEED"
)
// fakeInteractshClient hands out numbered URLs without contacting a server.
// Fuzz rules only allocate URLs through NewURLWithData, so the embedded nil
// IClient is never called.
type fakeInteractshClient struct {
interactsh.IClient
issued int
}
func (c *fakeInteractshClient) NewURLWithData(string) (string, error) {
c.issued++
return fmt.Sprintf("oast%d.example", c.issued), nil
}
// newQueryRule compiles a single-mode rule that replaces the p query value
// with fuzzValue.
func newQueryRule(
t *testing.T,
fuzzValue string,
generator *generators.PayloadGenerator,
templateVars variables.Variable,
client interactsh.IClient,
) *Rule {
t.Helper()
rule := &Rule{
Part: component.RequestQueryComponent,
Type: "replace",
Mode: "single",
Fuzz: SliceOrMapSlice{Value: []string{fuzzValue}},
}
err := rule.Compile(generator, &protocols.ExecutorOptions{
Variables: templateVars,
Options: &types.Options{},
Interactsh: client,
})
require.NoError(t, err)
return rule
}
// executeQueryRule runs rule against queryRuleTarget and returns the generated
// requests.
func executeQueryRule(
t *testing.T,
rule *Rule,
method string,
body io.Reader,
values map[string]any,
) []GeneratedRequest {
t.Helper()
req, err := retryablehttp.NewRequest(method, queryRuleTarget, body)
require.NoError(t, err)
var got []GeneratedRequest
err = rule.Execute(&ExecuteRuleInput{
Input: contextargs.NewWithInput(context.Background(), queryRuleTarget),
BaseRequest: req,
Values: values,
Callback: func(gr GeneratedRequest) bool {
got = append(got, gr)
return true
},
})
require.NoError(t, err)
return got
}
// newRequestDataVariableRule returns a query rule whose template declares
// b: "{{body}}", so b captures request data.
func newRequestDataVariableRule(t *testing.T, fuzzValue string, payloads map[string]any) *Rule {
t.Helper()
templateVars := newFuzzTestVariables()
templateVars.Set("b", "{{body}}")
var generator *generators.PayloadGenerator
if payloads != nil {
var err error
generator, err = generators.New(
payloads, generators.BatteringRamAttack, "", disk.NewCatalog(""), "", types.DefaultOptions(),
)
require.NoError(t, err)
}
// The CLI creates an Interactsh client even with -no-interactsh.
return newQueryRule(t, fuzzValue, generator, templateVars, &interactsh.Client{})
}
// fuzzedQueryValues runs rule against a POST request whose body is
// requestBodyWithDSL and returns the fuzzed p query value of each generated
// request.
func fuzzedQueryValues(t *testing.T, rule *Rule) []string {
t.Helper()
generated := executeQueryRule(t, rule, http.MethodPost, strings.NewReader(requestBodyWithDSL), map[string]any{
"body": requestBodyWithDSL,
})
got := make([]string, 0, len(generated))
for _, gr := range generated {
got = append(got, gr.Request.URL.Query().Get("p"))
}
return got
}
func TestExecuteKeepsRequestDataInTemplateVariableLiteral(t *testing.T) {
t.Parallel()
t.Run("fuzz value", func(t *testing.T) {
t.Parallel()
rule := newRequestDataVariableRule(t, "{{b}}", nil)
require.Equal(t, []string{requestBodyWithDSL}, fuzzedQueryValues(t, rule))
})
t.Run("payload value", func(t *testing.T) {
t.Parallel()
rule := newRequestDataVariableRule(t, "{{inj}}", map[string]any{
"inj": []any{"{{b}}"},
})
// Each run iterates a fresh value map. Repeat so the result
// cannot depend on map iteration order.
for range 256 {
require.Equal(t, []string{requestBodyWithDSL}, fuzzedQueryValues(t, rule))
}
})
}
func TestExecuteRendersTemplateVariableInteractshMarker(t *testing.T) {
t.Parallel()
templateVars := newFuzzTestVariables()
templateVars.Set("marker", "{{interactsh-url}}")
rule := newQueryRule(t, "{{marker}}", nil, templateVars, &fakeInteractshClient{})
got := executeQueryRule(t, rule, http.MethodGet, nil, nil)
require.Len(t, got, 1)
require.Equal(t, []string{"oast1.example"}, got[0].InteractURLs)
require.Equal(t, "oast1.example", got[0].Request.URL.Query().Get("p"))
}
func TestPrepareGeneratorValuesRendersPayloadSharingVariableName(t *testing.T) {
t.Parallel()
templateVars := newFuzzTestVariables()
templateVars.Set("inj", "variable")
rule := &Rule{
options: &protocols.ExecutorOptions{
Variables: templateVars,
Options: &types.Options{},
Interactsh: &fakeInteractshClient{},
},
}
values, _ := rule.prepareGeneratorValues(map[string]any{
"inj": `{{to_upper("payload")}}`,
}, nil)
require.Equal(t, "PAYLOAD", values["inj"])
}
func TestPrepareGeneratorValuesRendersPayloadsAgainstUnrenderedInput(t *testing.T) {
t.Parallel()
rule := &Rule{
options: &protocols.ExecutorOptions{
Variables: newFuzzTestVariables(),
Options: &types.Options{},
Interactsh: &interactsh.Client{},
},
}
for range 256 {
values, _ := rule.prepareGeneratorValues(map[string]any{
"upper": `{{to_upper("x")}}`,
"ref": "{{upper}}",
}, nil)
require.Equal(t, "X", values["upper"])
// ref sees the unrendered definition of upper as data.
require.Equal(t, `{{to_upper("x")}}`, values["ref"])
}
}
+1
View File
@@ -5,6 +5,7 @@ Formats implements support for passing a number of request source as input provi
Currently the following formats are implemented -
- Burp Suite XML Request/Response file
- HAR (HTTP Archive) file exported by browsers and proxies
- Proxify JSONL output file
- OpenAPI Specification file
- Postman Collection file
+137
View File
@@ -0,0 +1,137 @@
package har
import (
"encoding/base64"
"io"
"net/http"
"strconv"
"strings"
"github.com/pkg/errors"
"github.com/projectdiscovery/gologger"
"github.com/projectdiscovery/nuclei/v3/pkg/input/formats"
"github.com/projectdiscovery/nuclei/v3/pkg/input/types"
"github.com/projectdiscovery/nuclei/v3/pkg/utils/json"
)
// HARFormat is an HTTP Archive (HAR 1.2) parser, the format browsers and
// proxies export captured traffic in.
type HARFormat struct {
opts formats.InputFormatOptions
}
// New creates a new HAR parser
func New() *HARFormat {
return &HARFormat{}
}
var _ formats.Format = &HARFormat{}
type archive struct {
Log struct {
Entries []entry `json:"entries"`
} `json:"log"`
}
type entry struct {
Request struct {
Method string `json:"method"`
URL string `json:"url"`
Headers []header `json:"headers"`
PostData *struct {
Text string `json:"text"`
} `json:"postData"`
} `json:"request"`
Response struct {
Status int `json:"status"`
StatusText string `json:"statusText"`
Headers []header `json:"headers"`
Content struct {
Text string `json:"text"`
Encoding string `json:"encoding"`
} `json:"content"`
} `json:"response"`
}
type header struct {
Name string `json:"name"`
Value string `json:"value"`
}
// Name returns the name of the format
func (h *HARFormat) Name() string {
return "har"
}
func (h *HARFormat) SetOptions(options formats.InputFormatOptions) {
h.opts = options
}
// Parse parses the input and calls the provided callback
// function for each RawRequest it discovers.
func (h *HARFormat) Parse(input io.Reader, resultsCb formats.ParseReqRespCallback, filePath string) error {
var har archive
if err := json.NewDecoder(input).Decode(&har); err != nil {
return errors.Wrap(err, "could not decode har file")
}
for _, e := range har.Log.Entries {
// Browsers also log data:, blob: and extension URLs, which have no
// host to send a request to.
if !strings.HasPrefix(e.Request.URL, "http://") && !strings.HasPrefix(e.Request.URL, "https://") {
continue
}
rawRequest, err := types.ParseRawRequestWithURL(rawRequestOf(e), e.Request.URL)
if err != nil {
gologger.Warning().Msgf("har: Could not parse request %s: %s\n", e.Request.URL, err)
continue
}
// Status 0 is a request that was blocked or never answered.
if e.Response.Status != 0 {
rawRequest.Response = &types.HttpResponse{Raw: rawResponseOf(e)}
}
resultsCb(rawRequest)
}
return nil
}
// rawRequestOf rebuilds the request as HTTP/1.1 with an absolute target, which
// carries the authority that HTTP/2 captures only keep in pseudo-headers.
func rawRequestOf(e entry) string {
var b strings.Builder
b.WriteString(e.Request.Method + " " + e.Request.URL + " HTTP/1.1\r\n")
writeHeaders(&b, e.Request.Headers)
if e.Request.PostData != nil {
b.WriteString(e.Request.PostData.Text)
}
return b.String()
}
func rawResponseOf(e entry) string {
statusText := e.Response.StatusText
if statusText == "" {
statusText = http.StatusText(e.Response.Status)
}
var b strings.Builder
b.WriteString("HTTP/1.1 " + strconv.Itoa(e.Response.Status) + " " + statusText + "\r\n")
writeHeaders(&b, e.Response.Headers)
body := e.Response.Content.Text
if e.Response.Content.Encoding == "base64" {
if decoded, err := base64.StdEncoding.DecodeString(body); err == nil {
body = string(decoded)
}
}
b.WriteString(body)
return b.String()
}
func writeHeaders(b *strings.Builder, headers []header) {
for _, h := range headers {
// HTTP/2 pseudo-headers (:authority, :path, ...) are not headers.
if strings.HasPrefix(h.Name, ":") {
continue
}
b.WriteString(h.Name + ": " + h.Value + "\r\n")
}
b.WriteString("\r\n")
}
+71
View File
@@ -0,0 +1,71 @@
package har
import (
"os"
"strings"
"testing"
"github.com/projectdiscovery/nuclei/v3/pkg/input/types"
"github.com/stretchr/testify/require"
)
const chromeHAR = "../testdata/chrome.har"
func parse(t *testing.T, input string) []*types.RequestResponse {
t.Helper()
file, err := os.Open(input)
require.NoError(t, err)
defer func() { _ = file.Close() }()
var got []*types.RequestResponse
require.NoError(t, New().Parse(file, func(rr *types.RequestResponse) bool {
got = append(got, rr)
return false
}, input))
return got
}
func TestHARParse(t *testing.T) {
got := parse(t, chromeHAR)
urls := make([]string, 0, len(got))
for _, rr := range got {
urls = append(urls, rr.URL.String())
}
require.Equal(t, []string{
"https://ginandjuice.shop/catalog?category=Juice",
"https://ginandjuice.shop/login",
"https://ginandjuice.shop/blocked.js",
}, urls, "non-http entries such as data: URLs are skipped")
}
func TestHARParseHTTP2Request(t *testing.T) {
rr := parse(t, chromeHAR)[0]
require.Equal(t, "GET", rr.Request.Method)
cookie, _ := rr.Request.Headers.Get("cookie")
require.Equal(t, "session=abc123", cookie)
for _, name := range rr.Request.Headers.GetKeys() {
require.False(t, strings.HasPrefix(name, ":"), "pseudo-header %s is not a real header", name)
}
req, err := rr.BuildRequest()
require.NoError(t, err)
require.Equal(t, "ginandjuice.shop", req.Host)
}
func TestHARParseBodyAndResponse(t *testing.T) {
got := parse(t, chromeHAR)
login := got[1]
require.Equal(t, "POST", login.Request.Method)
require.Equal(t, "username=carlos&password=hunter2", login.Request.Body)
require.NotNil(t, login.Response)
require.Contains(t, login.Response.Raw, "302 Found")
require.Contains(t, login.Response.Raw, "Location: /my-account")
require.NotNil(t, got[0].Response)
require.Contains(t, got[0].Response.Raw, "<h1>Juice</h1>", "base64 content is decoded")
require.Nil(t, got[2].Response, "a request that never got an answer has no response")
}
+20
View File
@@ -0,0 +1,20 @@
//go:build gofuzz
// +build gofuzz
package openapi
// Fuzz exercises OpenAPI 3.0 schema parsing and request generation. The input
// bytes are treated as a raw schema document (YAML or JSON) and fed through the
// format's Parse, which loads the schema and generates requests from it.
func Fuzz(data []byte) int {
if len(data) == 0 {
return 0
}
if len(data) > fuzzMaxInputSize {
return -1
}
if !fuzzOpenAPIParsing(data) {
return 0
}
return 1
}
+32
View File
@@ -0,0 +1,32 @@
package openapi
import (
"bytes"
"github.com/projectdiscovery/nuclei/v3/pkg/input/formats"
"github.com/projectdiscovery/nuclei/v3/pkg/input/types"
)
const fuzzMaxInputSize = 64 << 10
// fuzzOpenAPIParsing feeds data to the OpenAPI format parser as a raw schema
// document and drives request generation. It must never panic for any input;
// it returns true when the schema parsed and at least one request was produced.
func fuzzOpenAPIParsing(data []byte) bool {
if len(data) == 0 || len(data) > fuzzMaxInputSize {
return false
}
format := New()
format.SetOptions(formats.InputFormatOptions{})
produced := false
cb := func(rr *types.RequestResponse) bool {
produced = true
return true
}
if err := format.Parse(bytes.NewReader(data), cb, "fuzz.openapi"); err != nil {
return false
}
return produced
}
@@ -0,0 +1,84 @@
package openapi
import (
"os"
"path/filepath"
"strings"
"testing"
"github.com/projectdiscovery/nuclei/v3/pkg/input/types"
"github.com/stretchr/testify/require"
)
func TestFuzzOpenAPIParsingSeedCorpus(t *testing.T) {
entries, err := os.ReadDir("testdata/gofuzz-corpus")
require.NoError(t, err)
require.NotEmpty(t, entries)
for _, entry := range entries {
if entry.IsDir() {
continue
}
data, err := os.ReadFile(filepath.Join("testdata/gofuzz-corpus", entry.Name()))
require.NoError(t, err)
require.Truef(t, fuzzOpenAPIParsing(data), "seed %s should produce at least one request", entry.Name())
}
}
func TestFuzzOpenAPIParsingRejectsOversizeInput(t *testing.T) {
require.False(t, fuzzOpenAPIParsing(make([]byte, fuzzMaxInputSize+1)))
}
// Null entries decode to nil pointers and used to panic during request
// generation. The remaining valid entries must still produce requests.
func TestFuzzOpenAPIParsingNullEntries(t *testing.T) {
specs := map[string]string{
"null server": `
openapi: 3.0.0
info: {title: t, version: "1"}
servers:
-
- {url: "http://example.com"}
paths:
/users:
get:
responses: {"200": {description: ok}}
`,
"null media type": `
openapi: 3.0.0
info: {title: t, version: "1"}
servers: [{url: "http://example.com"}]
paths:
/users:
post:
requestBody:
content:
application/xml:
application/json:
schema: {type: object, properties: {name: {type: string}}}
responses: {"200": {description: ok}}
`,
}
for name, spec := range specs {
t.Run(name, func(t *testing.T) {
require.True(t, fuzzOpenAPIParsing([]byte(spec)))
})
}
}
func TestGenerateRequestsOnlyNullServers(t *testing.T) {
spec := `
openapi: 3.0.0
info: {title: t, version: "1"}
servers:
-
paths:
/users:
get:
responses: {"200": {description: ok}}
`
err := New().Parse(strings.NewReader(spec), func(*types.RequestResponse) bool { return false }, "spec.yaml")
require.ErrorContains(t, err, "no servers found in openapi schema")
}
+14 -1
View File
@@ -9,6 +9,7 @@ import (
"net/http/httputil"
"net/url"
"os"
"slices"
"strings"
"github.com/clbanning/mxj/v2"
@@ -33,7 +34,9 @@ const (
// GenerateRequestsFromSchema generates http requests from an OpenAPI 3.0 document object
func GenerateRequestsFromSchema(schema *openapi3.T, opts formats.InputFormatOptions, callback formats.ParseReqRespCallback) error {
if len(schema.Servers) == 0 {
// Null entries in `servers:` are skipped below, so a list of only null
// entries has no usable server, the same as an empty list.
if !slices.ContainsFunc(schema.Servers, func(s *openapi3.Server) bool { return s != nil }) {
return errors.New("no servers found in openapi schema")
}
globalParams, globalSecurityErr := selectSecurityParameters(schema, &schema.Security, opts.Variables)
@@ -49,6 +52,11 @@ func GenerateRequestsFromSchema(schema *openapi3.T, opts formats.InputFormatOpti
}
for _, serverURL := range schema.Servers {
// A null entry in the `servers:` list decodes to a nil *Server, so skip
// it rather than dereferencing it.
if serverURL == nil {
continue
}
pathURL := serverURL.URL
// Split the server URL into baseURL and serverPath
u, err := url.Parse(pathURL)
@@ -268,6 +276,11 @@ func generateRequestsFromOp(opts *generateReqOptions) error {
if opts.op.RequestBody != nil {
for content, value := range opts.op.RequestBody.Value.Content {
// A null media-type entry (e.g. `content: {application/json: }`)
// decodes to a nil *MediaType, so skip it before dereferencing.
if value == nil {
continue
}
cloned := req.Clone(req.Context())
var val interface{}
@@ -0,0 +1 @@
{"openapi":"3.0.0","info":{"title":"t","version":"1"},"servers":[{"url":"http://example.com"}],"paths":{"/a":{"post":{"requestBody":{"content":{"application/json":{"schema":{"type":"object","properties":{"k":{"type":"string"}}}}}},"responses":{"200":{"description":"ok"}}}}}}
@@ -0,0 +1,7 @@
openapi: 3.0.0
info: {title: t, version: "1"}
servers: [{url: "http://example.com"}]
paths:
/users:
get:
responses: {"200": {description: ok}}
@@ -0,0 +1,12 @@
openapi: 3.0.0
info: {title: t, version: "1"}
servers: [{url: "http://example.com"}]
paths:
/users:
post:
requestBody:
content:
application/xml:
application/json:
schema: {type: object, properties: {name: {type: string}}}
responses: {"200": {description: ok}}
@@ -0,0 +1,9 @@
openapi: 3.0.0
info: {title: t, version: "1"}
servers:
-
- {url: "http://example.com"}
paths:
/users:
get:
responses: {"200": {description: ok}}
+29
View File
@@ -3,7 +3,9 @@ package swagger
import (
"fmt"
"io"
"maps"
"path"
"slices"
"github.com/getkin/kin-openapi/openapi2"
"github.com/getkin/kin-openapi/openapi2conv"
@@ -55,6 +57,7 @@ func (j *SwaggerFormat) Parse(input io.Reader, resultsCb formats.ParseReqRespCal
if err != nil {
return errors.Wrap(err, "could not decode openapi 2.0 schema")
}
dropNullEntries(schemav2)
schema, err := openapi2conv.ToV3(schemav2)
if err != nil {
return errors.Wrap(err, "could not convert openapi 2.0 schema to 3.0")
@@ -67,6 +70,32 @@ func (j *SwaggerFormat) Parse(input io.Reader, resultsCb formats.ParseReqRespCal
return openapi.GenerateRequestsFromSchema(schema, j.opts, resultsCb)
}
// dropNullEntries removes null entries from the maps and lists of a decoded
// OpenAPI 2.0 document. A null value such as `paths: {/x: }` or an empty list
// item under `parameters:` decodes to a nil pointer, which openapi2conv.ToV3
// dereferences without checking.
func dropNullEntries(doc *openapi2.T) {
maps.DeleteFunc(doc.Paths, func(_ string, v *openapi2.PathItem) bool { return v == nil })
maps.DeleteFunc(doc.Parameters, func(_ string, v *openapi2.Parameter) bool { return v == nil })
dropNullResponses(doc.Responses)
for _, pathItem := range doc.Paths {
pathItem.Parameters = slices.DeleteFunc(pathItem.Parameters, func(v *openapi2.Parameter) bool { return v == nil })
for _, op := range pathItem.Operations() {
op.Parameters = slices.DeleteFunc(op.Parameters, func(v *openapi2.Parameter) bool { return v == nil })
dropNullResponses(op.Responses)
}
}
}
// dropNullResponses removes null responses, and null headers of the remaining
// responses, which openapi2conv.ToV3Response also dereferences.
func dropNullResponses(responses map[string]*openapi2.Response) {
maps.DeleteFunc(responses, func(_ string, v *openapi2.Response) bool { return v == nil })
for _, response := range responses {
maps.DeleteFunc(response.Headers, func(_ string, v *openapi2.Header) bool { return v == nil })
}
}
func decodeYAML(data []byte, target interface{}) error {
var value interface{}
if err := yaml.Unmarshal(data, &value); err != nil {
+67
View File
@@ -2,6 +2,7 @@ package swagger
import (
"os"
"strings"
"testing"
"github.com/projectdiscovery/nuclei/v3/pkg/input/types"
@@ -39,3 +40,69 @@ func TestSwaggerAPIParser(t *testing.T) {
}
require.ElementsMatch(t, gotMethodsToURLs, expectedURLs, "could not get swagger urls")
}
func TestSwaggerParseNullEntries(t *testing.T) {
// Each spec has a null map value or list item that decodes to a nil
// pointer and used to panic inside openapi2conv.ToV3.
specs := map[string]string{
"null-path-item": `
swagger: "2.0"
info: {title: t, version: "1"}
host: localhost
paths:
/x:
`,
"null-parameter": `
swagger: "2.0"
info: {title: t, version: "1"}
host: localhost
paths:
/x:
get:
parameters:
-
responses: {"200": {description: ok}}
`,
"null-response": `
swagger: "2.0"
info: {title: t, version: "1"}
host: localhost
paths:
/x:
get:
responses: {"200": }
`,
"null-response-header": `
swagger: "2.0"
info: {title: t, version: "1"}
host: localhost
responses:
Shared:
description: ok
headers:
X-Shared:
paths:
/x:
get:
responses:
"200":
description: ok
headers:
X-Rate-Limit:
`,
}
for name, spec := range specs {
t.Run(name, func(t *testing.T) {
var count int
err := New().Parse(strings.NewReader(spec), func(request *types.RequestResponse) bool {
count++
return false
}, "spec.yaml")
require.NoError(t, err)
if name != "null-path-item" {
require.Equal(t, 1, count, "the operation should still produce a request")
}
})
}
}
+71
View File
@@ -0,0 +1,71 @@
{
"log": {
"version": "1.2",
"creator": { "name": "WebInspector", "version": "537.36" },
"entries": [
{
"request": {
"method": "GET",
"url": "https://ginandjuice.shop/catalog?category=Juice",
"httpVersion": "http/2.0",
"headers": [
{ "name": ":authority", "value": "ginandjuice.shop" },
{ "name": ":method", "value": "GET" },
{ "name": ":path", "value": "/catalog?category=Juice" },
{ "name": ":scheme", "value": "https" },
{ "name": "accept", "value": "text/html" },
{ "name": "cookie", "value": "session=abc123" }
],
"queryString": [{ "name": "category", "value": "Juice" }],
"cookies": [],
"headersSize": -1,
"bodySize": 0
},
"response": {
"status": 200,
"statusText": "",
"httpVersion": "http/2.0",
"headers": [{ "name": "content-type", "value": "text/html; charset=utf-8" }],
"content": { "size": 19, "mimeType": "text/html", "text": "PGgxPkp1aWNlPC9oMT4=", "encoding": "base64" }
}
},
{
"request": {
"method": "POST",
"url": "https://ginandjuice.shop/login",
"httpVersion": "HTTP/1.1",
"headers": [
{ "name": "Host", "value": "ginandjuice.shop" },
{ "name": "Content-Type", "value": "application/x-www-form-urlencoded" }
],
"postData": { "mimeType": "application/x-www-form-urlencoded", "text": "username=carlos&password=hunter2" }
},
"response": {
"status": 302,
"statusText": "Found",
"httpVersion": "HTTP/1.1",
"headers": [{ "name": "Location", "value": "/my-account" }],
"content": { "size": 0, "mimeType": "" }
}
},
{
"request": {
"method": "GET",
"url": "data:image/png;base64,iVBORw0KGgo=",
"httpVersion": "",
"headers": []
},
"response": { "status": 200, "statusText": "", "httpVersion": "", "headers": [], "content": { "size": 0 } }
},
{
"request": {
"method": "GET",
"url": "https://ginandjuice.shop/blocked.js",
"httpVersion": "",
"headers": []
},
"response": { "status": 0, "statusText": "", "httpVersion": "", "headers": [], "content": { "size": 0 } }
}
]
}
}
+2
View File
@@ -10,6 +10,7 @@ import (
"github.com/projectdiscovery/gologger"
"github.com/projectdiscovery/nuclei/v3/pkg/input/formats"
"github.com/projectdiscovery/nuclei/v3/pkg/input/formats/burp"
"github.com/projectdiscovery/nuclei/v3/pkg/input/formats/har"
"github.com/projectdiscovery/nuclei/v3/pkg/input/formats/json"
"github.com/projectdiscovery/nuclei/v3/pkg/input/formats/openapi"
"github.com/projectdiscovery/nuclei/v3/pkg/input/formats/raw"
@@ -142,6 +143,7 @@ func (i *HttpInputProvider) Close() {}
// Supported Providers
var providersList = []formats.Format{
burp.New(),
har.New(),
json.New(),
yaml.New(),
openapi.New(),
+66 -138
View File
@@ -89,8 +89,9 @@ func New(opts *Options) (*ListInputProvider, error) {
patterns: normalize.NewFingerprinter(options.MaxURLsPerPattern),
ipOptions: &ipOptions{
ScanAllIPs: options.ScanAllIPs,
IPV4: sliceutil.Contains(options.IPVersion, "4"),
IPV6: sliceutil.Contains(options.IPVersion, "6"),
IPV4: len(options.IPVersion) == 0 || sliceutil.Contains(options.IPVersion, "4"),
IPV6: len(options.IPVersion) == 0 || sliceutil.Contains(options.IPVersion, "6"),
Auto: len(options.IPVersion) == 0,
},
excludedHosts: make(map[string]struct{}),
profiles: targetprofile.NewRegistry(configuration.TemplatesDirectory(options, config.DefaultConfig)),
@@ -168,6 +169,61 @@ func (i *ListInputProvider) TargetProfiles() *targetprofile.Registry {
return i.profiles
}
// selectedIPs returns address pins for a hostname. An empty pin delegates
// DNS and connection fallback to fastdialer, which races IPv4 and IPv6.
// Set and Del share this selection. With -iv omitted and without -sa, one
// unpinned target is returned. -sa returns every selected address.
func (i *ListInputProvider) selectedIPs(executionId, hostname string) []string {
if i.ipOptions == nil || (!i.ipOptions.ScanAllIPs && (i.ipOptions.Auto || !i.ipOptions.IPV6)) {
return []string{""}
}
dialers := protocolstate.GetDialersWithId(executionId)
if dialers == nil {
gologger.Warning().Msgf("cannot resolve %s: dialer %s not found", hostname, executionId)
return nil
}
dnsData, err := dialers.Fastdialer.GetDNSData(hostname)
if err != nil {
gologger.Warning().Msgf("DNS resolution failed for %s: %v", hostname, err)
if i.ipOptions.IPV4 {
return []string{""}
}
return nil
}
if len(dnsData.A) == 0 && len(dnsData.AAAA) == 0 {
gologger.Warning().Msgf("DNS resolution returned no IP addresses for %s", hostname)
if i.ipOptions.IPV4 {
return []string{""}
}
return nil
}
if i.ipOptions.ScanAllIPs {
var ips []string
if i.ipOptions.IPV6 {
ips = append(ips, dnsData.AAAA...)
}
if i.ipOptions.IPV4 {
ips = append(ips, dnsData.A...)
}
if len(ips) == 0 {
gologger.Warning().Msgf("no DNS addresses matching -iv selection for %s", hostname)
}
return ips
}
var ips []string
if i.ipOptions.IPV6 && len(dnsData.AAAA) > 0 {
ips = append(ips, dnsData.AAAA[0])
}
// Explicit -iv 4,6 scans the first IPv6 address and one unpinned target.
if i.ipOptions.IPV4 {
ips = append(ips, "")
}
if len(ips) == 0 {
gologger.Warning().Msgf("no IPv6 DNS address found for %s", hostname)
}
return ips
}
// Set normalizes and stores passed input values
func (i *ListInputProvider) Set(executionId string, value string) {
i.set(executionId, value, nil)
@@ -202,75 +258,11 @@ func (i *ListInputProvider) set(executionId string, value string, selection *tar
return
}
if i.ipOptions.ScanAllIPs {
// scan all ips
dialers := protocolstate.GetDialersWithId(executionId)
if dialers == nil {
gologger.Error().Msgf("dialers with executionId %s not found", executionId)
return
}
dnsData, err := dialers.Fastdialer.GetDNSData(urlx.Hostname())
if err == nil {
if (len(dnsData.A) + len(dnsData.AAAA)) > 0 {
var ips []string
if i.ipOptions.IPV4 {
ips = append(ips, dnsData.A...)
}
if i.ipOptions.IPV6 {
ips = append(ips, dnsData.AAAA...)
}
for _, ip := range ips {
if ip == "" {
continue
}
metaInput := contextargs.NewMetaInput()
metaInput.Input = URL
metaInput.CustomIP = ip
i.setItem(metaInput, selection)
}
return
} else {
gologger.Debug().Msgf("scanAllIps: no ip's found reverting to default")
}
} else {
// failed to scanallips falling back to defaults
gologger.Debug().Msgf("scanAllIps: dns resolution failed: %v", err)
}
}
ips := []string{}
// only scan the target but ipv6 if it has one
if i.ipOptions.IPV6 {
dialers := protocolstate.GetDialersWithId(executionId)
if dialers == nil {
gologger.Error().Msgf("dialers with executionId %s not found", executionId)
return
}
dnsData, err := dialers.Fastdialer.GetDNSData(urlx.Hostname())
if err == nil && len(dnsData.AAAA) > 0 {
// pick/ prefer 1st
ips = append(ips, dnsData.AAAA[0])
} else {
gologger.Warning().Msgf("target does not have ipv6 address falling back to ipv4 %v\n", err)
}
}
if i.ipOptions.IPV4 {
// if IPV4 is enabled do not specify ip let dialer handle it
ips = append(ips, "")
}
for _, ip := range ips {
for _, ip := range i.selectedIPs(executionId, urlx.Hostname()) {
metaInput := contextargs.NewMetaInput()
if ip != "" {
metaInput.Input = URL
metaInput.CustomIP = ip
i.setItem(metaInput, selection)
} else {
metaInput.Input = URL
i.setItem(metaInput, selection)
}
metaInput.Input = URL
metaInput.CustomIP = ip
i.setItem(metaInput, selection)
}
}
@@ -493,75 +485,11 @@ func (i *ListInputProvider) Del(executionId string, value string) {
return
}
if i.ipOptions.ScanAllIPs {
// scan all ips
dialers := protocolstate.GetDialersWithId(executionId)
if dialers == nil {
gologger.Error().Msgf("dialers with executionId %s not found", executionId)
return
}
dnsData, err := dialers.Fastdialer.GetDNSData(urlx.Hostname())
if err == nil {
if (len(dnsData.A) + len(dnsData.AAAA)) > 0 {
var ips []string
if i.ipOptions.IPV4 {
ips = append(ips, dnsData.A...)
}
if i.ipOptions.IPV6 {
ips = append(ips, dnsData.AAAA...)
}
for _, ip := range ips {
if ip == "" {
continue
}
metaInput := contextargs.NewMetaInput()
metaInput.Input = value
metaInput.CustomIP = ip
i.delItem(metaInput)
}
return
} else {
gologger.Debug().Msgf("scanAllIps: no ip's found reverting to default")
}
} else {
// failed to scanallips falling back to defaults
gologger.Debug().Msgf("scanAllIps: dns resolution failed: %v", err)
}
}
ips := []string{}
// only scan the target but ipv6 if it has one
if i.ipOptions.IPV6 {
dialers := protocolstate.GetDialersWithId(executionId)
if dialers == nil {
gologger.Error().Msgf("dialers with executionId %s not found", executionId)
return
}
dnsData, err := dialers.Fastdialer.GetDNSData(urlx.Hostname())
if err == nil && len(dnsData.AAAA) > 0 {
// pick/ prefer 1st
ips = append(ips, dnsData.AAAA[0])
} else {
gologger.Warning().Msgf("target does not have ipv6 address falling back to ipv4 %v\n", err)
}
}
if i.ipOptions.IPV4 {
// if IPV4 is enabled do not specify ip let dialer handle it
ips = append(ips, "")
}
for _, ip := range ips {
for _, ip := range i.selectedIPs(executionId, urlx.Hostname()) {
metaInput := contextargs.NewMetaInput()
if ip != "" {
metaInput.Input = URL
metaInput.CustomIP = ip
i.delItem(metaInput)
} else {
metaInput.Input = URL
i.delItem(metaInput)
}
metaInput.Input = URL
metaInput.CustomIP = ip
i.delItem(metaInput)
}
}
+106
View File
@@ -0,0 +1,106 @@
package list
import (
"net"
"strings"
"testing"
"time"
"github.com/miekg/dns"
"github.com/projectdiscovery/hmap/store/hybrid"
"github.com/projectdiscovery/nuclei/v3/pkg/protocols/common/contextargs"
"github.com/projectdiscovery/nuclei/v3/pkg/protocols/common/protocolstate"
"github.com/projectdiscovery/nuclei/v3/pkg/types"
"github.com/stretchr/testify/require"
)
// A local deterministic DNS responder for dual-stack, A-only and AAAA-only hosts.
type ipVersionDNSHandler struct{}
func (ipVersionDNSHandler) ServeDNS(w dns.ResponseWriter, r *dns.Msg) {
reply := new(dns.Msg)
reply.SetReply(r)
if len(r.Question) == 0 {
_ = w.WriteMsg(reply)
return
}
question := r.Question[0]
// Some resolvers try search suffixes before the absolute name. Match the
// test host prefix rather than answering unrelated names as dual-stack.
host := strings.ToLower(question.Name)
if strings.HasPrefix(host, "empty.") {
_ = w.WriteMsg(reply)
return
}
isDual := strings.HasPrefix(host, "dual.")
isV4 := strings.HasPrefix(host, "v4only.")
isV6 := strings.HasPrefix(host, "v6only.")
if question.Qtype == dns.TypeA && (isDual || isV4) {
reply.Answer = append(reply.Answer, &dns.A{Hdr: dns.RR_Header{Name: question.Name, Rrtype: dns.TypeA, Class: dns.ClassINET, Ttl: 60}, A: net.ParseIP("192.0.2.10")})
}
if question.Qtype == dns.TypeAAAA && (isDual || isV6) {
reply.Answer = append(reply.Answer, &dns.AAAA{Hdr: dns.RR_Header{Name: question.Name, Rrtype: dns.TypeAAAA, Class: dns.ClassINET, Ttl: 60}, AAAA: net.ParseIP("2001:db8::10")})
}
_ = w.WriteMsg(reply)
}
func TestIPVersionsDefaultAndExplicitSelection(t *testing.T) {
conn, err := net.ListenPacket("udp", "127.0.0.1:0")
require.NoError(t, err)
started := make(chan struct{})
server := &dns.Server{PacketConn: conn, Handler: ipVersionDNSHandler{}, NotifyStartedFunc: func() { close(started) }}
go func() { _ = server.ActivateAndServe() }()
t.Cleanup(func() { _ = server.Shutdown() })
select {
case <-started:
case <-time.After(5 * time.Second):
t.Fatal("DNS test resolver did not start")
}
opts := types.DefaultOptions()
opts.ExecutionId = "ip-version-regression"
opts.InternalResolversList = []string{conn.LocalAddr().String()}
require.NoError(t, protocolstate.Init(opts))
t.Cleanup(func() { protocolstate.Close(opts.ExecutionId) })
for _, tc := range []struct {
name, hostname string
scanAll, ipv4, ipv6, auto bool
want []string
}{
{"automatic dual-stack keeps dialer fallback", "dual.test", false, true, true, true, []string{""}},
{"automatic IPv6-only keeps dialer fallback", "v6only.test", false, true, true, true, []string{""}},
{"auto IPv4 fallback", "v4only.test", false, true, true, true, []string{""}},
{"auto empty DNS response fallback", "empty.test", false, true, true, true, []string{""}},
{"all empty DNS response fallback", "empty.test", true, true, true, true, []string{""}},
{"all dual", "dual.test", true, true, true, true, []string{"192.0.2.10", "2001:db8::10"}},
{"all IPv6 only", "v6only.test", true, true, true, true, []string{"2001:db8::10"}},
{"all IPv4 only", "v4only.test", true, true, true, true, []string{"192.0.2.10"}},
{"explicit IPv4 only", "dual.test", false, true, false, false, []string{""}},
{"explicit IPv6 only", "dual.test", false, false, true, false, []string{"2001:db8::10"}},
{"explicit both", "dual.test", false, true, true, false, []string{"2001:db8::10", ""}},
{"all explicit IPv6", "dual.test", true, false, true, false, []string{"2001:db8::10"}},
{"all explicit IPv4", "dual.test", true, true, false, false, []string{"192.0.2.10"}},
{"IPv6 restricted on IPv4-only", "v4only.test", true, false, true, false, nil},
} {
t.Run(tc.name, func(t *testing.T) {
hm, err := hybrid.New(hybrid.DefaultDiskOptions)
require.NoError(t, err)
input := &ListInputProvider{
hostMap: hm,
excludedHosts: make(map[string]struct{}),
ipOptions: &ipOptions{ScanAllIPs: tc.scanAll, IPV4: tc.ipv4, IPV6: tc.ipv6, Auto: tc.auto},
}
t.Cleanup(input.Close)
input.Set(opts.ExecutionId, "https://"+tc.hostname)
var got []string
input.Iterate(func(value *contextargs.MetaInput) bool {
got = append(got, value.CustomIP)
return true
})
require.ElementsMatch(t, tc.want, got)
if tc.scanAll && len(tc.want) > 0 {
input.Del(opts.ExecutionId, "https://"+tc.hostname)
require.EqualValues(t, 0, input.Count(), "Del must remove the same IPs added by Set")
}
})
}
}
+1
View File
@@ -4,4 +4,5 @@ type ipOptions struct {
ScanAllIPs bool
IPV4 bool
IPV6 bool
Auto bool // -iv omitted: one unpinned target unless -sa scans every address
}
+53
View File
@@ -0,0 +1,53 @@
package compiler_test
//nolint:depguard // No depguard rules are configured, so its default list allows only the standard library.
import (
"testing"
"github.com/projectdiscovery/nuclei/v3/pkg/js/compiler"
"github.com/projectdiscovery/nuclei/v3/pkg/types"
)
// TestCloseWithoutConnection calls Close on clients that never connected.
// The script must keep running past the call.
func TestCloseWithoutConnection(t *testing.T) {
t.Parallel()
scripts := map[string]string{
"ssh": `
const ssh = require('nuclei/ssh');
ExportAs('closed', new ssh.SSHClient().Close());
true;
`,
"net": `
const net = require('nuclei/net');
new net.NetConn().Close();
ExportAs('closed', true);
true;
`,
}
for name, script := range scripts {
t.Run(name, func(t *testing.T) {
t.Parallel()
program, err := compiler.SourceAutoMode(script, false)
if err != nil {
t.Fatal(err)
}
opts := new(compiler.ExecuteOptions)
opts.ExecutionId = t.Name()
opts.Source = &script // ExportAs is defined only for sources that use it.
opts.TimeoutVariants = types.NewTimeoutVariant(5)
result, err := compiler.New().ExecuteWithOptions(t.Context(), program, compiler.NewExecuteArgs(), opts)
if err != nil {
t.Fatal(err)
}
if got := result["closed"]; got != "true" {
t.Errorf(`result["closed"] = %v, want "true"`, got)
}
})
}
}
+4
View File
@@ -16,18 +16,22 @@ import (
syncutil "github.com/projectdiscovery/utils/sync"
_ "github.com/projectdiscovery/nuclei/v3/pkg/js/generated/go/libbytes"
_ "github.com/projectdiscovery/nuclei/v3/pkg/js/generated/go/libcompress"
_ "github.com/projectdiscovery/nuclei/v3/pkg/js/generated/go/libdcerpc"
_ "github.com/projectdiscovery/nuclei/v3/pkg/js/generated/go/libdcom"
_ "github.com/projectdiscovery/nuclei/v3/pkg/js/generated/go/libfs"
_ "github.com/projectdiscovery/nuclei/v3/pkg/js/generated/go/libgrpc"
_ "github.com/projectdiscovery/nuclei/v3/pkg/js/generated/go/libhash"
_ "github.com/projectdiscovery/nuclei/v3/pkg/js/generated/go/libhttp"
_ "github.com/projectdiscovery/nuclei/v3/pkg/js/generated/go/libikev2"
_ "github.com/projectdiscovery/nuclei/v3/pkg/js/generated/go/libipmi"
_ "github.com/projectdiscovery/nuclei/v3/pkg/js/generated/go/libkerberos"
_ "github.com/projectdiscovery/nuclei/v3/pkg/js/generated/go/libkrbforge"
_ "github.com/projectdiscovery/nuclei/v3/pkg/js/generated/go/libkrbroast"
_ "github.com/projectdiscovery/nuclei/v3/pkg/js/generated/go/libldap"
_ "github.com/projectdiscovery/nuclei/v3/pkg/js/generated/go/libmssql"
_ "github.com/projectdiscovery/nuclei/v3/pkg/js/generated/go/libmysql"
_ "github.com/projectdiscovery/nuclei/v3/pkg/js/generated/go/libmqtt"
_ "github.com/projectdiscovery/nuclei/v3/pkg/js/generated/go/libnet"
_ "github.com/projectdiscovery/nuclei/v3/pkg/js/generated/go/liboracle"
_ "github.com/projectdiscovery/nuclei/v3/pkg/js/generated/go/libpop3"
+158
View File
@@ -0,0 +1,158 @@
package compiler
import (
"bytes"
"fmt"
"io"
"math/rand"
"net/http"
"net/http/httptest"
"net/url"
"strconv"
"testing"
"time"
"github.com/andybalholm/brotli"
"github.com/klauspost/compress/gzip"
"github.com/klauspost/compress/zlib"
"github.com/klauspost/compress/zstd"
"github.com/projectdiscovery/dsl"
"github.com/projectdiscovery/nuclei/v3/pkg/protocols/common/protocolstate"
"github.com/projectdiscovery/nuclei/v3/pkg/types"
httputil "github.com/projectdiscovery/utils/http"
"github.com/stretchr/testify/require"
)
const streamHashScript = `
const net = require('nuclei/net');
const compress = require('nuclei/compress');
const hash = require('nuclei/hash');
const conn = net.Open('tcp', %q);
conn.Send('GET / HTTP/1.1\r\nHost: test\r\nAccept-Encoding: gzip, deflate, br, zstd\r\n\r\n');
let head = '';
while (!head.endsWith('\r\n\r\n')) head += conn.RecvString(1);
const header = (name) => (head.match(new RegExp('\r\n' + name + ': ([^\r]+)', 'i')) || [])[1];
const length = header('Content-Length');
const coding = [header('Content-Encoding'), header('Transfer-Encoding')].filter(Boolean).join(', ');
const d = compress.NewDecoder(coding);
const sha = hash.New('sha256'), mmh3 = hash.New('mmh3');
const take = (out) => { sha.Write(out); mmh3.Write(out); };
if (length !== undefined) {
for (let read = 0; read < parseInt(length); ) {
const chunk = conn.Recv(65536);
read += chunk.length;
for (let out = d.Write(chunk); out.length > 0; out = d.Read()) take(out);
}
for (let out = d.Close(); out.length > 0; out = d.Read()) take(out);
} else {
while (!d.Finished()) {
for (let out = d.Write(conn.Recv(65536)); out.length > 0; out = d.Read()) take(out);
}
}
ExportAs('sha256', sha.Sum());
ExportAs('mmh3', mmh3.Sum());
true;
`
func encodeBody(t *testing.T, encoding string, data []byte) []byte {
t.Helper()
var buf bytes.Buffer
var w io.WriteCloser
switch encoding {
case "gzip":
w = gzip.NewWriter(&buf)
case "deflate":
w = zlib.NewWriter(&buf)
case "br":
w = brotli.NewWriter(&buf)
case "zstd":
zw, err := zstd.NewWriter(&buf)
require.NoError(t, err)
w = zw
case "":
return data
}
_, err := w.Write(data)
require.NoError(t, err)
require.NoError(t, w.Close())
return buf.Bytes()
}
func TestStreamingHashMatchesHTTPEngine(t *testing.T) {
rng := rand.New(rand.NewSource(1))
var body []byte
for len(body) < 6<<20 {
body = append(body, "large compressed response body "...)
body = append(body, byte(rng.Intn(256)))
}
executionID := t.Name()
require.NoError(t, protocolstate.Init(&types.Options{ExecutionId: executionID}))
t.Cleanup(func() { protocolstate.Close(executionID) })
for _, tc := range []struct {
encoding string
chunked bool
}{
{"gzip", false},
{"gzip", true},
{"deflate", false},
{"br", true},
{"zstd", true},
{"", true},
} {
t.Run(fmt.Sprintf("%s chunked=%v", tc.encoding, tc.chunked), func(t *testing.T) {
encoded := encodeBody(t, tc.encoding, body)
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if tc.encoding != "" {
w.Header().Set("Content-Encoding", tc.encoding)
}
if !tc.chunked {
w.Header().Set("Content-Length", strconv.Itoa(len(encoded)))
_, _ = w.Write(encoded)
return
}
// Flushing before the handler returns makes the body chunked.
for rest := encoded; len(rest) > 0; {
n := min(len(rest), 50000)
_, _ = w.Write(rest[:n])
w.(http.Flusher).Flush()
rest = rest[n:]
}
}))
defer server.Close()
client := &http.Client{Transport: &http.Transport{DisableCompression: true}}
resp, err := client.Get(server.URL)
require.NoError(t, err)
if tc.chunked {
require.Equal(t, []string{"chunked"}, resp.TransferEncoding)
}
chain := httputil.NewResponseChain(resp, 64<<20)
defer chain.Close()
require.NoError(t, chain.Fill())
require.Equal(t, len(body), chain.Body().Len(), "the engine must decode the whole body")
engineBody := map[string]interface{}{"body": chain.BodyString()}
wantSha, err := dsl.EvalExpr("sha256(body)", engineBody)
require.NoError(t, err)
wantMmh3, err := dsl.EvalExpr("mmh3(body)", engineBody)
require.NoError(t, err)
serverURL, err := url.Parse(server.URL)
require.NoError(t, err)
script := fmt.Sprintf(streamHashScript, serverURL.Host)
compiled, err := SourceAutoMode(script, false)
require.NoError(t, err)
result, err := New().ExecuteWithOptions(t.Context(), compiled, NewExecuteArgs(), &ExecuteOptions{
ExecutionId: executionID,
Source: &script,
TimeoutVariants: &types.Timeouts{JsCompilerExecutionTimeout: 20 * time.Second},
})
require.NoError(t, err)
require.Equal(t, fmt.Sprint(wantSha), result["sha256"])
require.Equal(t, fmt.Sprint(wantMmh3), result["mmh3"])
})
}
}
@@ -0,0 +1,30 @@
package compress
import (
lib_compress "github.com/projectdiscovery/nuclei/v3/pkg/js/libs/compress"
"github.com/projectdiscovery/goja"
"github.com/projectdiscovery/nuclei/v3/pkg/js/gojs"
)
var (
module = gojs.NewGojaModule("nuclei/compress")
)
func init() {
module.Set(
gojs.Objects{
// Functions
"NewDecoder": lib_compress.NewDecoder,
// Var and consts
// Objects / Classes
"Decoder": gojs.GetClassConstructor[lib_compress.Decoder](&lib_compress.Decoder{}),
},
).Register()
}
func Enable(runtime *goja.Runtime) {
module.Enable(runtime)
}
+30
View File
@@ -0,0 +1,30 @@
package hash
import (
lib_hash "github.com/projectdiscovery/nuclei/v3/pkg/js/libs/hash"
"github.com/projectdiscovery/goja"
"github.com/projectdiscovery/nuclei/v3/pkg/js/gojs"
)
var (
module = gojs.NewGojaModule("nuclei/hash")
)
func init() {
module.Set(
gojs.Objects{
// Functions
"New": lib_hash.New,
// Var and consts
// Objects / Classes
"Hash": gojs.GetClassConstructor[lib_hash.Hash](&lib_hash.Hash{}),
},
).Register()
}
func Enable(runtime *goja.Runtime) {
module.Enable(runtime)
}
+39
View File
@@ -0,0 +1,39 @@
package ipmi
import (
lib_ipmi "github.com/projectdiscovery/nuclei/v3/pkg/js/libs/ipmi"
"github.com/projectdiscovery/goja"
"github.com/projectdiscovery/nuclei/v3/pkg/js/gojs"
)
var (
module = gojs.NewGojaModule("nuclei/ipmi")
)
func init() {
module.Set(
gojs.Objects{
// Functions
"CheckRAKPPassword": lib_ipmi.CheckRAKPPassword,
"FindRAKPPassword": lib_ipmi.FindRAKPPassword,
"GetAuthCapabilities": lib_ipmi.GetAuthCapabilities,
"GetCipherSuites": lib_ipmi.GetCipherSuites,
"GetRAKPHash": lib_ipmi.GetRAKPHash,
"GetSystemGUID": lib_ipmi.GetSystemGUID,
"Ping": lib_ipmi.Ping,
// Var and consts
// Objects / Classes
"AuthCapabilities": gojs.GetClassConstructor[lib_ipmi.AuthCapabilities](&lib_ipmi.AuthCapabilities{}),
"Client": gojs.GetClassConstructor[lib_ipmi.Client](&lib_ipmi.Client{}),
"CommandResponse": gojs.GetClassConstructor[lib_ipmi.CommandResponse](&lib_ipmi.CommandResponse{}),
"RAKPHash": gojs.GetClassConstructor[lib_ipmi.RAKPHash](&lib_ipmi.RAKPHash{}),
},
).Register()
}
func Enable(runtime *goja.Runtime) {
module.Enable(runtime)
}
+34
View File
@@ -0,0 +1,34 @@
package mqtt
import (
lib_mqtt "github.com/projectdiscovery/nuclei/v3/pkg/js/libs/mqtt"
"github.com/projectdiscovery/goja"
"github.com/projectdiscovery/nuclei/v3/pkg/js/gojs"
)
var (
module = gojs.NewGojaModule("nuclei/mqtt")
)
func init() {
module.Set(
gojs.Objects{
// Functions
"IsMQTT": lib_mqtt.IsMQTT,
"IsAnonymousAllowed": lib_mqtt.IsAnonymousAllowed,
// Var and consts
// Objects / Classes
"Client": gojs.GetClassConstructor[lib_mqtt.Client](&lib_mqtt.Client{}),
"ConnectOptions": gojs.GetClassConstructor[lib_mqtt.ConnectOptions](&lib_mqtt.ConnectOptions{}),
"ConnectResult": gojs.GetClassConstructor[lib_mqtt.ConnectResult](&lib_mqtt.ConnectResult{}),
"Message": gojs.GetClassConstructor[lib_mqtt.Message](&lib_mqtt.Message{}),
},
).Register()
}
func Enable(runtime *goja.Runtime) {
module.Enable(runtime)
}
+102
View File
@@ -0,0 +1,102 @@
/**
* NewDecoder returns a decoder for gzip, deflate, br, zstd, chunked or
* identity. List several comma-separated, in the order they were applied.
* @example
* ```javascript
* const compress = require('nuclei/compress');
* const d = compress.NewDecoder('br');
* // Content-Encoding: gzip sent with Transfer-Encoding: chunked
* const c = compress.NewDecoder('gzip, chunked');
* ```
*/
export function NewDecoder(encoding: string): Decoder | null {
return null;
}
/**
* Decoder decodes a compressed stream written to it in chunks. Each call
* returns up to about 1 MB; call Read until it returns nothing before the
* next Write.
* @example
* ```javascript
* const compress = require('nuclei/compress');
* const hash = require('nuclei/hash');
* const d = compress.NewDecoder('gzip');
* const h = hash.New('sha256');
* for (const chunk of chunks) {
* for (let out = d.Write(chunk); out.length > 0; out = d.Read()) h.Write(out);
* }
* for (let out = d.Close(); out.length > 0; out = d.Read()) h.Write(out);
* log(h.Sum());
* ```
*/
export class Decoder {
// Constructor of Decoder
constructor() {}
/**
* Write decodes the next chunk and returns up to about 1 MB of output.
* @example
* ```javascript
* const compress = require('nuclei/compress');
* const d = compress.NewDecoder('gzip');
* const out = d.Write(chunk);
* ```
*/
public Write(chunk: Uint8Array): Uint8Array | null {
return null;
}
/**
* Read returns more decoded bytes, or none once the decoder needs input.
* @example
* ```javascript
* const compress = require('nuclei/compress');
* const d = compress.NewDecoder('gzip');
* for (let out = d.Write(chunk); out.length > 0; out = d.Read()) log(out.length);
* ```
*/
public Read(): Uint8Array | null {
return null;
}
/**
* Close ends the input and returns the last decoded bytes. A truncated
* stream errors once the bytes before the cut have been returned.
* @example
* ```javascript
* const compress = require('nuclei/compress');
* const d = compress.NewDecoder('gzip');
* d.Write(chunk);
* for (let out = d.Close(); out.length > 0; out = d.Read()) log(out.length);
* ```
*/
public Close(): Uint8Array | null {
return null;
}
/**
* Finished reports whether a stream that marks its own end, such as a
* chunked body, has ended and all its output was read.
* @example
* ```javascript
* const compress = require('nuclei/compress');
* const d = compress.NewDecoder('gzip, chunked');
* while (!d.Finished()) for (let out = d.Write(conn.Recv(65536)); out.length > 0; out = d.Read()) log(out.length);
* ```
*/
public Finished(): boolean {
return false;
}
}
+77
View File
@@ -0,0 +1,77 @@
/**
* New returns a running hash for md5, sha1, sha256, sha512 or mmh3. Sum
* matches the DSL function of the same name.
* @example
* ```javascript
* const hash = require('nuclei/hash');
* const h = hash.New('sha256');
* ```
*/
export function New(algorithm: string): Hash | null {
return null;
}
/**
* Hash is a running hash.
* @example
* ```javascript
* const hash = require('nuclei/hash');
* const h = hash.New('sha256');
* h.WriteString('hello');
* log(h.Sum());
* ```
*/
export class Hash {
// Constructor of Hash
constructor() {}
/**
* Write adds data to the hash.
* @example
* ```javascript
* const hash = require('nuclei/hash');
* const h = hash.New('sha256');
* h.Write([104, 105]);
* ```
*/
public Write(data: Uint8Array): void {
return;
}
/**
* WriteString adds the string data to the hash.
* @example
* ```javascript
* const hash = require('nuclei/hash');
* const h = hash.New('sha256');
* h.WriteString('hi');
* ```
*/
public WriteString(data: string): void {
return;
}
/**
* Sum returns the hash of the data written so far, formatted as the DSL does.
* @example
* ```javascript
* const hash = require('nuclei/hash');
* const h = hash.New('mmh3');
* h.WriteString('hello');
* log(h.Sum());
* ```
*/
public Sum(): string | null {
return null;
}
}
+4
View File
@@ -1,17 +1,21 @@
export * as bytes from './bytes';
export * as compress from './compress';
export * as dcom from './dcom';
export * as dcerpc from './dcerpc';
export * as fs from './fs';
export * as goconsole from './goconsole';
export * as grpc from './grpc';
export * as hash from './hash';
export * as http from './http';
export * as ikev2 from './ikev2';
export * as ipmi from './ipmi';
export * as kerberos from './kerberos';
export * as krbforge from './krbforge';
export * as krbroast from './krbroast';
export * as ldap from './ldap';
export * as mssql from './mssql';
export * as mysql from './mysql';
export * as mqtt from './mqtt';
export * as net from './net';
export * as oracle from './oracle';
export * as pop3 from './pop3';
+304
View File
@@ -0,0 +1,304 @@
/**
* CheckRAKPPassword reports whether password produced hash. It runs offline,
* so it can test any number of passwords without touching the BMC.
* @example
* ```javascript
* const ipmi = require('nuclei/ipmi');
* const hash = ipmi.GetRAKPHash('acme.com', 623, 'ADMIN');
* log(ipmi.CheckRAKPPassword(hash, 'ADMIN'));
* ```
*/
export function CheckRAKPPassword(hash: RAKPHash, password: string): boolean {
return false;
}
/**
* FindRAKPPassword returns the first of passwords that produced hash, or an
* empty string.
* @example
* ```javascript
* const ipmi = require('nuclei/ipmi');
* const hash = ipmi.GetRAKPHash('acme.com', 623, 'root');
* log(ipmi.FindRAKPPassword(hash, ['calvin', 'root', 'admin']));
* ```
*/
export function FindRAKPPassword(hash: RAKPHash, passwords: string[]): string {
return "";
}
/**
* GetAuthCapabilities returns the authentication capabilities the BMC
* advertises on the current channel.
* @example
* ```javascript
* const ipmi = require('nuclei/ipmi');
* const caps = ipmi.GetAuthCapabilities('acme.com', 623);
* log(caps.AnonymousLogin);
* ```
*/
export function GetAuthCapabilities(ctx: any, host: string, port: number): AuthCapabilities | null {
return null;
}
/**
* GetCipherSuites returns the IDs of the RMCP+ cipher suites the BMC
* offers. Cipher suite 0 authenticates nothing, so any password logs in.
* @example
* ```javascript
* const ipmi = require('nuclei/ipmi');
* const suites = ipmi.GetCipherSuites('acme.com', 623);
* log(suites.includes(0));
* ```
*/
export function GetCipherSuites(ctx: any, host: string, port: number): number[] | null {
return null;
}
/**
* GetRAKPHash starts an RMCP+ session as username and returns the HMAC the
* BMC answers with. The session is never completed, so no login is
* attempted; the BMC drops the half-open session after its timeout. It fails
* when the BMC rejects the username.
* @example
* ```javascript
* const ipmi = require('nuclei/ipmi');
* const hash = ipmi.GetRAKPHash('acme.com', 623, 'ADMIN');
* log(ipmi.CheckRAKPPassword(hash, 'ADMIN'));
* ```
*/
export function GetRAKPHash(ctx: any, host: string, port: number, username: string): RAKPHash | null {
return null;
}
/**
* GetSystemGUID returns the BMC system GUID, which vendors derive from
* serial numbers or MAC addresses.
* @example
* ```javascript
* const ipmi = require('nuclei/ipmi');
* const guid = ipmi.GetSystemGUID('acme.com', 623);
* log(guid);
* ```
*/
export function GetSystemGUID(ctx: any, host: string, port: number): string | null {
return null;
}
/**
* Ping reports whether the host answers an RMCP presence ping and advertises
* IPMI support.
* @example
* ```javascript
* const ipmi = require('nuclei/ipmi');
* const isIPMI = ipmi.Ping('acme.com', 623);
* log(isIPMI);
* ```
*/
export function Ping(ctx: any, host: string, port: number): boolean | null {
return null;
}
/**
* Client holds one RMCP+ session, for commands that need a login.
* Close it when done: BMCs allow only a few sessions at a time.
* @example
* ```javascript
* const ipmi = require('nuclei/ipmi');
* const client = new ipmi.Client();
* client.Login('acme.com', 623, 'ADMIN', 'ADMIN', 3);
* const id = client.Command(0x06, 0x01, []);
* client.Close();
* ```
*/
export class Client {
// Constructor of Client
constructor() {}
/**
* Login opens an RMCP+ session as username with the given cipher suite
* (0, 1, 2, 3 or 6). Cipher suite 0 skips authentication, so a BMC
* that allows it accepts any password. A rejected login is an error.
* @example
* ```javascript
* const ipmi = require('nuclei/ipmi');
* const client = new ipmi.Client();
* client.Login('acme.com', 623, 'ADMIN', 'not-the-password', 0);
* ```
*/
public Login(ctx: any, host: string, port: number, username: string, password: string, cipherSuite: number): boolean | null {
return null;
}
/**
* Command sends an IPMI request in the session, such as Get Device ID
* (netFn 0x06, cmd 0x01) or Set Session Privilege Level (0x06, 0x3b).
* @example
* ```javascript
* const ipmi = require('nuclei/ipmi');
* const client = new ipmi.Client();
* client.Login('acme.com', 623, 'ADMIN', 'ADMIN', 3);
* const admin = client.Command(0x06, 0x3b, [0x04]).CompletionCode == 0;
* client.Close();
* ```
*/
public Command(netFn: number, cmd: number, data: Uint8Array): CommandResponse | null {
return null;
}
/**
* Close ends the session on the BMC, freeing its slot.
* @example
* ```javascript
* const ipmi = require('nuclei/ipmi');
* const client = new ipmi.Client();
* client.Login('acme.com', 623, 'ADMIN', 'ADMIN', 3);
* client.Close();
* ```
*/
public Close(): void {
return;
}
}
/**
* AuthCapabilities is what a BMC advertises before any login, from Get
* Channel Authentication Capabilities.
* @example
* ```javascript
* const ipmi = require('nuclei/ipmi');
* const caps = ipmi.GetAuthCapabilities('acme.com', 623);
* log(toJSON(caps));
* ```
*/
export interface AuthCapabilities {
/**
* IPMI20 is true when the BMC supports IPMI 2.0 (RMCP+).
*/
IPMI20?: boolean,
/**
* AuthTypes are the IPMI 1.5 authentication types the channel
* accepts: none, md2, md5, straight (cleartext password), oem.
*/
AuthTypes?: string[],
/**
* AnonymousLogin is true when a null username with a null password
* is accepted.
*/
AnonymousLogin?: boolean,
/**
* NullUsernames is true when accounts with an empty name are enabled.
*/
NullUsernames?: boolean,
/**
* NonNullUsernames is true when named accounts are enabled.
*/
NonNullUsernames?: boolean,
/**
* PerMessageAuth is false when the BMC skips authenticating
* messages after login.
*/
PerMessageAuth?: boolean,
/**
* UserLevelAuth is false when user-level commands skip
* authentication.
*/
UserLevelAuth?: boolean,
/**
* OEMID is the IANA enterprise number of the BMC vendor, or 0.
*/
OEMID?: number,
}
/**
* CommandResponse is the reply to an IPMI command.
*/
export interface CommandResponse {
/**
* CompletionCode is 0 on success.
*/
CompletionCode?: number,
Data?: Uint8Array,
}
/**
* RAKPHash is the HMAC a BMC returns for a user during RMCP+ session
* setup (CVE-2013-4786). It can be checked offline against candidate
* passwords, which tests credentials without a login attempt.
* @example
* ```javascript
* const ipmi = require('nuclei/ipmi');
* const hash = ipmi.GetRAKPHash('acme.com', 623, 'ADMIN');
* log(hash.Hashcat);
* ```
*/
export interface RAKPHash {
Username?: string,
/**
* Algorithm is the HMAC the BMC used: sha1 or md5.
*/
Algorithm?: string,
/**
* Salt and HMAC are hex encoded; Salt is everything the HMAC covers.
*/
Salt?: string,
HMAC?: string,
/**
* Hashcat is the hash in hashcat mode 7300 format, for sha1 only.
*/
Hashcat?: string,
}
+190
View File
@@ -0,0 +1,190 @@
/**
* IsMQTT checks whether the target host:port is running an MQTT broker.
* @example
* ```javascript
* const mqtt = require('nuclei/mqtt');
* const isMqtt = mqtt.IsMQTT('acme.com', 1883);
* log(isMqtt);
* ```
*/
export function IsMQTT(host: string, port: number): boolean | null {
return null;
}
/**
* IsAnonymousAllowed checks whether an MQTT broker permits unauthenticated / anonymous login.
* @example
* ```javascript
* const mqtt = require('nuclei/mqtt');
* const allowed = mqtt.IsAnonymousAllowed('acme.com', 1883);
* log(allowed);
* ```
*/
export function IsAnonymousAllowed(host: string, port: number): boolean | null {
return null;
}
/**
* Message represents an MQTT published message.
* @example
* ```javascript
* const mqtt = require('nuclei/mqtt');
* const client = new mqtt.Client();
* client.Connect('broker.hivemq.com', 1883);
* const msgs = client.Subscribe('test/topic', 2);
* log(msgs[0].Topic, msgs[0].Payload);
* ```
*/
export interface Message {
Topic?: string;
Payload?: string;
QoS?: number;
Retain?: boolean;
}
/**
* ConnectOptions contains connection options for an MQTT broker.
* @example
* ```javascript
* const mqtt = require('nuclei/mqtt');
* const opts = new mqtt.ConnectOptions();
* opts.Username = 'admin';
* opts.Password = 'admin';
* opts.CleanSession = true;
* ```
*/
export interface ConnectOptions {
Username?: string;
Password?: string;
ClientID?: string;
CleanSession?: boolean;
KeepAlive?: number;
TLS?: boolean;
InsecureSkipVerify?: boolean;
Timeout?: number;
}
/**
* ConnectResult is the response returned by an MQTT connection attempt.
* @example
* ```javascript
* const mqtt = require('nuclei/mqtt');
* const client = new mqtt.Client();
* const res = client.Connect('acme.com', 1883);
* log(res.Success, res.ReturnCode, res.Reason);
* ```
*/
export interface ConnectResult {
Success?: boolean;
ReturnCode?: number;
Reason?: string;
SessionPresent?: boolean;
}
/**
* Client is an MQTT client for testing MQTT broker misconfigurations and authentication.
* @example
* ```javascript
* const mqtt = require('nuclei/mqtt');
* const client = new mqtt.Client();
* const res = client.Connect('acme.com', 1883);
* if (res.Success) {
* const msgs = client.Subscribe('#', 2);
* client.Publish('nuclei/probe', 'test');
* }
* client.Close();
* ```
*/
export class Client {
constructor() {}
/**
* Connect connects to an MQTT broker using optional credentials and settings.
* @example
* ```javascript
* const mqtt = require('nuclei/mqtt');
* const client = new mqtt.Client();
* const res = client.Connect('acme.com', 1883);
* log(res.Success);
* ```
*/
public Connect(host: string, port: number, options?: ConnectOptions): ConnectResult | null {
return null;
}
/**
* ConnectAuth connects to an MQTT broker with a username and password.
* @example
* ```javascript
* const mqtt = require('nuclei/mqtt');
* const client = new mqtt.Client();
* const res = client.ConnectAuth('acme.com', 1883, 'admin', 'admin');
* log(res.Success);
* ```
*/
public ConnectAuth(host: string, port: number, username: string, password: string): ConnectResult | null {
return null;
}
/**
* Subscribe subscribes to a topic filter and collects received messages up to timeoutSec.
* @example
* ```javascript
* const mqtt = require('nuclei/mqtt');
* const client = new mqtt.Client();
* client.Connect('acme.com', 1883);
* const msgs = client.Subscribe('#', 3);
* for (const m of msgs) {
* log(m.Topic, m.Payload);
* }
* ```
*/
public Subscribe(topic: string, timeoutSec?: number): Message[] | null {
return null;
}
/**
* Publish publishes a message payload to a topic.
* @example
* ```javascript
* const mqtt = require('nuclei/mqtt');
* const client = new mqtt.Client();
* client.Connect('acme.com', 1883);
* const ok = client.Publish('nuclei/test', 'probe payload');
* log(ok);
* ```
*/
public Publish(topic: string, payload: string): boolean | null {
return null;
}
/**
* Ping sends an MQTT PINGREQ and awaits PINGRESP.
* @example
* ```javascript
* const mqtt = require('nuclei/mqtt');
* const client = new mqtt.Client();
* client.Connect('acme.com', 1883);
* const alive = client.Ping();
* log(alive);
* ```
*/
public Ping(): boolean | null {
return null;
}
/**
* Close cleanly disconnects and closes the connection.
* @example
* ```javascript
* const mqtt = require('nuclei/mqtt');
* const client = new mqtt.Client();
* client.Connect('acme.com', 1883);
* client.Close();
* ```
*/
public Close(): error | null {
return null;
}
}
+1
View File
@@ -46,6 +46,7 @@ export class NetConn {
constructor() {}
/**
* Close closes the connection.
* Close does nothing if no connection is open.
* @example
* ```javascript
* const net = require('nuclei/net');
+1 -1
View File
@@ -2,7 +2,7 @@
/**
* PGClient is a client for Postgres database.
* Internally client uses go-pg/pg driver.
* Internally client uses lib/pq driver.
* @example
* ```javascript
* const postgres = require('nuclei/postgres');
+2 -1
View File
@@ -106,7 +106,8 @@ export class SSHClient {
/**
* Close closes the SSH connection and destroys the client
* Returns the success state and error. If error is not nil,
* state will be false
* state will be false. Close does nothing and returns true
* if no connection is open.
* @example
* ```javascript
* const ssh = require('nuclei/ssh');
+6 -2
View File
@@ -128,9 +128,13 @@ func (p *GojaModule) Register() Module {
return p
}
// GetClassConstructor returns a constructor for any given go struct type for goja runtime
// GetClassConstructor returns a constructor for any given go struct type for goja runtime.
// Each object it constructs starts as a shallow copy of instance, so objects
// do not share state.
func GetClassConstructor[T any](instance *T) func(call goja.ConstructorCall, runtime *goja.Runtime) *goja.Object {
return func(call goja.ConstructorCall, runtime *goja.Runtime) *goja.Object {
return utils.LinkConstructor[*T](call, runtime, instance)
obj := *instance
return utils.LinkConstructor(call, runtime, &obj)
}
}
+46
View File
@@ -0,0 +1,46 @@
package gojs_test
//nolint:depguard // No depguard rules are configured, so its default list allows only the standard library.
import (
"testing"
"github.com/projectdiscovery/goja"
"github.com/projectdiscovery/nuclei/v3/pkg/js/gojs"
)
type counter struct{ n int }
func (c *counter) Inc() int {
c.n++
return c.n
}
func TestGetClassConstructorReturnsNewObjects(t *testing.T) {
t.Parallel()
prototype := new(counter)
runtime := goja.New()
err := runtime.Set("Counter", gojs.GetClassConstructor(prototype))
if err != nil {
t.Fatal(err)
}
value, err := runtime.RunString(`
const first = new Counter();
first.Inc();
new Counter().Inc();
`)
if err != nil {
t.Fatal(err)
}
if got := value.ToInteger(); got != 1 {
t.Errorf("new Counter().Inc() = %d, want 1", got)
}
if prototype.n != 0 {
t.Errorf("prototype.n = %d, want 0", prototype.n)
}
}
+303
View File
@@ -0,0 +1,303 @@
// Package compress exposes streaming decompression to nuclei JavaScript templates.
package compress
import (
"bytes"
"context"
"errors"
"fmt"
"io"
"net/http/httputil"
"slices"
"strings"
"github.com/dsnet/compress/brotli"
"github.com/klauspost/compress/gzip"
"github.com/klauspost/compress/zlib"
"github.com/klauspost/compress/zstd"
)
// maxDecodedChunk caps the decoded bytes one call hands back.
const maxDecodedChunk = 1 << 20
type pauseReason int
const (
needInput pauseReason = iota
outputFull
finished
)
type (
// Decoder decodes a compressed stream written to it in chunks. Each call
// returns up to about 1 MB; call Read until it returns nothing before the
// next Write.
// @example
// ```javascript
// const compress = require('nuclei/compress');
// const hash = require('nuclei/hash');
// const d = compress.NewDecoder('gzip');
// const h = hash.New('sha256');
// for (const chunk of chunks) {
// for (let out = d.Write(chunk); out.length > 0; out = d.Read()) h.Write(out);
// }
// for (let out = d.Close(); out.length > 0; out = d.Read()) h.Write(out);
// log(h.Sum());
// ```
Decoder struct {
ctx context.Context
// The decode goroutine and the template's calls take turns over these
// channels, so no field is accessed by both at once.
resume chan []byte
paused chan pauseReason
done chan struct{}
out []byte
state pauseReason
inputEnded bool
pending []byte
eof bool
err error
}
)
// NewDecoder returns a decoder for gzip, deflate, br, zstd, chunked or
// identity. List several comma-separated, in the order they were applied.
// @example
// ```javascript
// const compress = require('nuclei/compress');
// const d = compress.NewDecoder('br');
// // Content-Encoding: gzip sent with Transfer-Encoding: chunked
// const c = compress.NewDecoder('gzip, chunked');
// ```
func NewDecoder(ctx context.Context, encoding string) (*Decoder, error) {
var opens []func(io.Reader) (io.Reader, func(), error)
for name := range strings.SplitSeq(encoding, ",") {
switch name = strings.ToLower(strings.TrimSpace(name)); name {
case "gzip":
opens = append(opens, func(r io.Reader) (io.Reader, func(), error) {
zr, err := gzip.NewReader(r)
return zr, func() {}, err
})
case "deflate":
opens = append(opens, func(r io.Reader) (io.Reader, func(), error) {
zr, err := zlib.NewReader(r)
return zr, func() {}, err
})
case "br":
opens = append(opens, func(r io.Reader) (io.Reader, func(), error) {
br, err := brotli.NewReader(r, nil)
return br, func() {}, err
})
case "zstd":
opens = append(opens, func(r io.Reader) (io.Reader, func(), error) {
// One goroutine so no background workers outlive the stream.
zr, err := zstd.NewReader(r, zstd.WithDecoderConcurrency(1))
if err != nil {
return nil, nil, err
}
return zr, zr.Close, nil
})
case "chunked":
opens = append(opens, func(r io.Reader) (io.Reader, func(), error) {
return httputil.NewChunkedReader(r), func() {}, nil
})
case "", "identity":
default:
return nil, fmt.Errorf("unsupported encoding %q, expected gzip, deflate, br, zstd, chunked or identity", name)
}
}
slices.Reverse(opens)
open := func(r io.Reader) (io.Reader, func(), error) {
var closers []func()
closeAll := func() {
for _, c := range slices.Backward(closers) {
c()
}
}
for _, o := range opens {
next, c, err := o(r)
if err != nil {
closeAll()
return nil, nil, err
}
r = next
closers = append(closers, c)
}
return r, closeAll, nil
}
d := &Decoder{
ctx: ctx,
resume: make(chan []byte),
paused: make(chan pauseReason),
done: make(chan struct{}),
}
go d.run(open)
d.wait()
return d, nil
}
// Write decodes the next chunk and returns up to about 1 MB of output.
// @example
// ```javascript
// const compress = require('nuclei/compress');
// const d = compress.NewDecoder('gzip');
// const out = d.Write(chunk);
// ```
func (d *Decoder) Write(chunk []byte) ([]byte, error) {
if d.done == nil {
return nil, errNotCreatedByNewDecoder
}
if d.inputEnded {
return nil, errors.New("decoder is closed")
}
if d.state == outputFull {
return nil, errors.New("decoded bytes are waiting: call Read until it returns no bytes before writing more")
}
if d.state == needInput && len(chunk) > 0 {
// The decode goroutine may still hold the chunk after Write returns.
d.resumeWith(bytes.Clone(chunk))
}
return d.result()
}
// Read returns more decoded bytes, or none once the decoder needs input.
// @example
// ```javascript
// const compress = require('nuclei/compress');
// const d = compress.NewDecoder('gzip');
// for (let out = d.Write(chunk); out.length > 0; out = d.Read()) log(out.length);
// ```
func (d *Decoder) Read() ([]byte, error) {
if d.done == nil {
return nil, errNotCreatedByNewDecoder
}
d.advance()
return d.result()
}
// Close ends the input and returns the last decoded bytes. A truncated
// stream errors once the bytes before the cut have been returned.
// @example
// ```javascript
// const compress = require('nuclei/compress');
// const d = compress.NewDecoder('gzip');
// d.Write(chunk);
// for (let out = d.Close(); out.length > 0; out = d.Read()) log(out.length);
// ```
func (d *Decoder) Close() ([]byte, error) {
if d.done == nil {
return nil, errNotCreatedByNewDecoder
}
d.inputEnded = true
d.advance()
return d.result()
}
// Finished reports whether a stream that marks its own end, such as a
// chunked body, has ended and all its output was read.
// @example
// ```javascript
// const compress = require('nuclei/compress');
// const d = compress.NewDecoder('gzip, chunked');
// while (!d.Finished()) for (let out = d.Write(conn.Recv(65536)); out.length > 0; out = d.Read()) log(out.length);
// ```
func (d *Decoder) Finished() bool {
return d.state == finished && len(d.out) == 0
}
var errNotCreatedByNewDecoder = errors.New("decoder was not created with compress.NewDecoder")
func (d *Decoder) advance() {
for len(d.out) == 0 && (d.state == outputFull || d.state == needInput && d.inputEnded) {
d.resumeWith(nil)
}
}
func (d *Decoder) result() ([]byte, error) {
out := d.out
d.out = nil
if len(out) == 0 && d.state == finished {
return nil, d.err
}
return out, nil
}
func (d *Decoder) resumeWith(chunk []byte) {
select {
case d.resume <- chunk:
d.wait()
case <-d.done:
d.state = finished
}
}
func (d *Decoder) wait() {
select {
case d.state = <-d.paused:
case <-d.done:
d.state = finished
}
}
func (d *Decoder) run(open func(io.Reader) (io.Reader, func(), error)) {
defer close(d.done)
r, closeReader, err := open(decoderInput{d})
if err != nil {
d.err = err
return
}
defer closeReader()
_, err = io.Copy(decoderOutput{d}, r)
d.err = err
}
func (d *Decoder) pause(reason pauseReason) ([]byte, error) {
select {
case d.paused <- reason:
case <-d.ctx.Done():
return nil, context.Cause(d.ctx)
}
select {
case chunk := <-d.resume:
return chunk, nil
case <-d.ctx.Done():
return nil, context.Cause(d.ctx)
}
}
type decoderInput struct{ d *Decoder }
func (in decoderInput) Read(p []byte) (int, error) {
d := in.d
for len(d.pending) == 0 {
if d.eof {
return 0, io.EOF
}
chunk, err := d.pause(needInput)
if err != nil {
return 0, err
}
if chunk == nil {
d.eof = true
}
d.pending = chunk
}
n := copy(p, d.pending)
d.pending = d.pending[n:]
return n, nil
}
type decoderOutput struct{ d *Decoder }
func (out decoderOutput) Write(p []byte) (int, error) {
d := out.d
d.out = append(d.out, p...)
if len(d.out) >= maxDecodedChunk {
if _, err := d.pause(outputFull); err != nil {
return 0, err
}
}
return len(p), nil
}
+192
View File
@@ -0,0 +1,192 @@
package compress
import (
"bytes"
"context"
"io"
"math/rand"
"net/http/httputil"
"testing"
"time"
"github.com/andybalholm/brotli"
"github.com/klauspost/compress/gzip"
"github.com/klauspost/compress/zlib"
"github.com/klauspost/compress/zstd"
"github.com/stretchr/testify/require"
)
func compressed(t *testing.T, encoding string, data []byte) []byte {
t.Helper()
var buf bytes.Buffer
var w io.WriteCloser
switch encoding {
case "gzip":
w = gzip.NewWriter(&buf)
case "deflate":
w = zlib.NewWriter(&buf)
case "br":
w = brotli.NewWriter(&buf)
case "zstd":
zw, err := zstd.NewWriter(&buf)
require.NoError(t, err)
w = zw
case "identity":
return data
}
_, err := w.Write(data)
require.NoError(t, err)
require.NoError(t, w.Close())
return buf.Bytes()
}
func decodeInChunks(t *testing.T, d *Decoder, input []byte) ([]byte, error) {
t.Helper()
rng := rand.New(rand.NewSource(1))
var decoded []byte
collect := func(out []byte, err error) bool {
require.LessOrEqual(t, len(out), maxDecodedChunk+64<<10)
decoded = append(decoded, out...)
return err == nil && len(out) > 0
}
drain := func(out []byte, err error) error {
for collect(out, err) {
out, err = d.Read()
}
return err
}
for len(input) > 0 {
n := min(len(input), 1+rng.Intn(70000))
out, err := d.Write(input[:n])
if err := drain(out, err); err != nil {
return decoded, err
}
input = input[n:]
}
out, err := d.Close()
return decoded, drain(out, err)
}
func TestDecoderMatchesWholeBodyDecode(t *testing.T) {
data := bytes.Repeat([]byte("nuclei streaming decoder "), 400000)
for _, encoding := range []string{"gzip", "deflate", "br", "zstd", "identity"} {
t.Run(encoding, func(t *testing.T) {
d, err := NewDecoder(t.Context(), encoding)
require.NoError(t, err)
decoded, err := decodeInChunks(t, d, compressed(t, encoding, data))
require.NoError(t, err)
require.Equal(t, data, decoded)
})
}
}
func chunked(t *testing.T, data []byte) []byte {
t.Helper()
var buf bytes.Buffer
w := httputil.NewChunkedWriter(&buf)
for len(data) > 0 {
n := min(len(data), 8000)
_, err := w.Write(data[:n])
require.NoError(t, err)
data = data[n:]
}
require.NoError(t, w.Close())
buf.WriteString("\r\n")
return buf.Bytes()
}
func TestDecoderDecodesChunkedTransfer(t *testing.T) {
data := bytes.Repeat([]byte("nuclei chunked decoder "), 400000)
for _, encoding := range []string{"gzip", "deflate", "br", "zstd", "identity"} {
t.Run(encoding, func(t *testing.T) {
d, err := NewDecoder(t.Context(), encoding+", chunked")
require.NoError(t, err)
decoded, err := decodeInChunks(t, d, chunked(t, compressed(t, encoding, data)))
require.NoError(t, err)
require.Equal(t, data, decoded)
})
}
}
func TestDecoderFinishesAtLastChunk(t *testing.T) {
data := bytes.Repeat([]byte("finished "), 1000)
d, err := NewDecoder(t.Context(), "gzip, chunked")
require.NoError(t, err)
input := chunked(t, compressed(t, "gzip", data))
half := len(input) / 2
_, err = d.Write(input[:half])
require.NoError(t, err)
require.False(t, d.Finished())
out, err := d.Write(input[half:])
require.NoError(t, err)
var decoded []byte
for ; len(out) > 0; out, err = d.Read() {
decoded = append(decoded, out...)
}
require.NoError(t, err)
require.Equal(t, data, decoded)
require.True(t, d.Finished(), "decoder must finish without Close once the last chunk arrives")
}
func TestDecoderAppliesCodingsInReverse(t *testing.T) {
data := bytes.Repeat([]byte("two codings "), 100000)
input := compressed(t, "br", compressed(t, "gzip", data))
d, err := NewDecoder(t.Context(), "gzip, br")
require.NoError(t, err)
decoded, err := decodeInChunks(t, d, input)
require.NoError(t, err)
require.Equal(t, data, decoded)
}
func TestDecoderReportsTruncatedStream(t *testing.T) {
data := bytes.Repeat([]byte("truncated "), 200000)
for _, encoding := range []string{"gzip", "deflate", "br", "zstd", "identity, chunked"} {
t.Run(encoding, func(t *testing.T) {
var input []byte
if encoding == "identity, chunked" {
input = chunked(t, data)
} else {
input = compressed(t, encoding, data)
}
d, err := NewDecoder(t.Context(), encoding)
require.NoError(t, err)
decoded, err := decodeInChunks(t, d, input[:len(input)/2])
require.Error(t, err)
require.True(t, bytes.HasPrefix(data, decoded), "bytes before the cut must still be handed back")
})
}
}
func TestDecoderStopsWhenContextEnds(t *testing.T) {
ctx, cancel := context.WithCancel(t.Context())
d, err := NewDecoder(ctx, "gzip")
require.NoError(t, err)
_, err = d.Write(compressed(t, "gzip", []byte("abandoned"))[:5])
require.NoError(t, err)
cancel()
select {
case <-d.done:
case <-time.After(time.Second):
t.Fatal("decode goroutine outlived its context")
}
}
func TestNewDecoderRejectsUnknownEncoding(t *testing.T) {
_, err := NewDecoder(t.Context(), "compress")
require.Error(t, err)
_, err = NewDecoder(t.Context(), "gzip, compress")
require.Error(t, err)
}
func TestDecoderNotCreatedByNewDecoderErrors(t *testing.T) {
var d Decoder
_, err := d.Write([]byte("x"))
require.ErrorIs(t, err, errNotCreatedByNewDecoder)
_, err = d.Read()
require.ErrorIs(t, err, errNotCreatedByNewDecoder)
_, err = d.Close()
require.ErrorIs(t, err, errNotCreatedByNewDecoder)
}
+100
View File
@@ -0,0 +1,100 @@
// Package hash exposes incremental hashes to nuclei JavaScript templates.
package hash
import (
"crypto/md5"
"crypto/sha1"
"crypto/sha256"
"crypto/sha512"
"encoding/hex"
"errors"
"fmt"
"hash"
"strings"
"github.com/spaolacci/murmur3"
)
type (
// Hash is a running hash.
// @example
// ```javascript
// const hash = require('nuclei/hash');
// const h = hash.New('sha256');
// h.WriteString('hello');
// log(h.Sum());
// ```
Hash struct {
h hash.Hash
mmh3 bool
}
)
// New returns a running hash for md5, sha1, sha256, sha512 or mmh3. Sum
// matches the DSL function of the same name.
// @example
// ```javascript
// const hash = require('nuclei/hash');
// const h = hash.New('sha256');
// ```
func New(algorithm string) (*Hash, error) {
switch strings.ToLower(algorithm) {
case "md5":
return &Hash{h: md5.New()}, nil
case "sha1":
return &Hash{h: sha1.New()}, nil
case "sha256":
return &Hash{h: sha256.New()}, nil
case "sha512":
return &Hash{h: sha512.New()}, nil
case "mmh3":
return &Hash{h: murmur3.New32WithSeed(0), mmh3: true}, nil
}
return nil, fmt.Errorf("unsupported hash algorithm %q, expected md5, sha1, sha256, sha512 or mmh3", algorithm)
}
// Write adds data to the hash.
// @example
// ```javascript
// const hash = require('nuclei/hash');
// const h = hash.New('sha256');
// h.Write([104, 105]);
// ```
func (h *Hash) Write(data []byte) error {
if h.h == nil {
return errNotCreatedByNew
}
_, _ = h.h.Write(data)
return nil
}
// WriteString adds the string data to the hash.
// @example
// ```javascript
// const hash = require('nuclei/hash');
// const h = hash.New('sha256');
// h.WriteString('hi');
// ```
func (h *Hash) WriteString(data string) error {
return h.Write([]byte(data))
}
// Sum returns the hash of the data written so far, formatted as the DSL does.
// @example
// ```javascript
// const hash = require('nuclei/hash');
// const h = hash.New('mmh3');
// h.WriteString('hello');
// log(h.Sum());
// ```
func (h *Hash) Sum() (string, error) {
if h.h == nil {
return "", errNotCreatedByNew
}
if h.mmh3 {
return fmt.Sprintf("%d", int32(h.h.(hash.Hash32).Sum32())), nil
}
return hex.EncodeToString(h.h.Sum(nil)), nil
}
var errNotCreatedByNew = errors.New("hash was not created with hash.New")
+43
View File
@@ -0,0 +1,43 @@
package hash
import (
"testing"
"github.com/projectdiscovery/dsl"
"github.com/stretchr/testify/require"
)
func TestSumMatchesDSLAcrossChunks(t *testing.T) {
data := make([]byte, 3<<20+7)
for i := range data {
data[i] = byte(i * 31)
}
for _, algorithm := range []string{"md5", "sha1", "sha256", "sha512", "mmh3"} {
t.Run(algorithm, func(t *testing.T) {
h, err := New(algorithm)
require.NoError(t, err)
for chunk := data; len(chunk) > 0; {
n := min(len(chunk), 65521)
require.NoError(t, h.Write(chunk[:n]))
chunk = chunk[n:]
}
want, err := dsl.EvalExpr(algorithm+"(body)", map[string]interface{}{"body": string(data)})
require.NoError(t, err)
sum, err := h.Sum()
require.NoError(t, err)
require.Equal(t, want, sum)
})
}
}
func TestNewRejectsUnknownAlgorithm(t *testing.T) {
_, err := New("crc32")
require.Error(t, err)
}
func TestHashNotCreatedByNewErrors(t *testing.T) {
var h Hash
require.ErrorIs(t, h.WriteString("x"), errNotCreatedByNew)
_, err := h.Sum()
require.ErrorIs(t, err, errNotCreatedByNew)
}
+394
View File
@@ -0,0 +1,394 @@
package ipmi
import (
"context"
"crypto/hmac"
"crypto/md5"
"crypto/sha1"
"encoding/hex"
"errors"
"fmt"
)
type (
// AuthCapabilities is what a BMC advertises before any login, from Get
// Channel Authentication Capabilities.
// @example
// ```javascript
// const ipmi = require('nuclei/ipmi');
// const caps = ipmi.GetAuthCapabilities('acme.com', 623);
// log(toJSON(caps));
// ```
AuthCapabilities struct {
// IPMI20 is true when the BMC supports IPMI 2.0 (RMCP+).
IPMI20 bool
// AuthTypes are the IPMI 1.5 authentication types the channel
// accepts: none, md2, md5, straight (cleartext password), oem.
AuthTypes []string
// AnonymousLogin is true when a null username with a null password
// is accepted.
AnonymousLogin bool
// NullUsernames is true when accounts with an empty name are enabled.
NullUsernames bool
// NonNullUsernames is true when named accounts are enabled.
NonNullUsernames bool
// PerMessageAuth is false when the BMC skips authenticating
// messages after login.
PerMessageAuth bool
// UserLevelAuth is false when user-level commands skip
// authentication.
UserLevelAuth bool
// OEMID is the IANA enterprise number of the BMC vendor, or 0.
OEMID int
}
// RAKPHash is the HMAC a BMC returns for a user during RMCP+ session
// setup (CVE-2013-4786). It can be checked offline against candidate
// passwords, which tests credentials without a login attempt.
// @example
// ```javascript
// const ipmi = require('nuclei/ipmi');
// const hash = ipmi.GetRAKPHash('acme.com', 623, 'ADMIN');
// log(hash.Hashcat);
// ```
RAKPHash struct {
Username string
// Algorithm is the HMAC the BMC used: sha1 or md5.
Algorithm string
// Salt and HMAC are hex encoded; Salt is everything the HMAC covers.
Salt string
HMAC string
// Hashcat is the hash in hashcat mode 7300 format, for sha1 only.
Hashcat string
}
// Client holds one RMCP+ session, for commands that need a login.
// Close it when done: BMCs allow only a few sessions at a time.
// @example
// ```javascript
// const ipmi = require('nuclei/ipmi');
// const client = new ipmi.Client();
// client.Login('acme.com', 623, 'ADMIN', 'ADMIN', 3);
// const id = client.Command(0x06, 0x01, []);
// client.Close();
// ```
Client struct {
session *established
}
// CommandResponse is the reply to an IPMI command.
CommandResponse struct {
// CompletionCode is 0 on success.
CompletionCode int
Data []byte
}
)
var authTypes = []struct {
bit byte
name string
}{{0x01, "none"}, {0x02, "md2"}, {0x04, "md5"}, {0x10, "straight"}, {0x20, "oem"}}
// Ping reports whether the host answers an RMCP presence ping and advertises
// IPMI support.
// @example
// ```javascript
// const ipmi = require('nuclei/ipmi');
// const isIPMI = ipmi.Ping('acme.com', 623);
// log(isIPMI);
// ```
func Ping(ctx context.Context, host string, port int) (bool, error) {
s, err := dial(ctx, host, port)
if err != nil {
return false, err
}
defer s.close()
return s.ping()
}
// GetAuthCapabilities returns the authentication capabilities the BMC
// advertises on the current channel.
// @example
// ```javascript
// const ipmi = require('nuclei/ipmi');
// const caps = ipmi.GetAuthCapabilities('acme.com', 623);
// log(caps.AnonymousLogin);
// ```
func GetAuthCapabilities(ctx context.Context, host string, port int) (*AuthCapabilities, error) {
s, err := dial(ctx, host, port)
if err != nil {
return nil, err
}
defer s.close()
// bit 7 asks for the IPMI 2.0 extended data; an IPMI 1.5 only BMC
// rejects it, so ask again without
data, err := s.command(cmdGetChannelAuthCaps, []byte{0x80 | currentChannel, privAdmin})
if err != nil {
if data, err = s.command(cmdGetChannelAuthCaps, []byte{currentChannel, privAdmin}); err != nil {
return nil, err
}
}
return parseAuthCapabilities(data)
}
// parseAuthCapabilities decodes the Get Channel Authentication Capabilities
// response data (IPMI v2.0 table 22-15).
func parseAuthCapabilities(data []byte) (*AuthCapabilities, error) {
if len(data) < 8 {
return nil, fmt.Errorf("short channel authentication capabilities response")
}
caps := &AuthCapabilities{
IPMI20: data[1]&0x80 != 0 && data[3]&0x02 != 0,
AnonymousLogin: data[2]&0x01 != 0,
NullUsernames: data[2]&0x02 != 0,
NonNullUsernames: data[2]&0x04 != 0,
UserLevelAuth: data[2]&0x08 == 0,
PerMessageAuth: data[2]&0x10 == 0,
OEMID: int(data[4]) | int(data[5])<<8 | int(data[6])<<16,
}
for _, authType := range authTypes {
if data[1]&authType.bit != 0 {
caps.AuthTypes = append(caps.AuthTypes, authType.name)
}
}
return caps, nil
}
// GetSystemGUID returns the BMC system GUID, which vendors derive from
// serial numbers or MAC addresses.
// @example
// ```javascript
// const ipmi = require('nuclei/ipmi');
// const guid = ipmi.GetSystemGUID('acme.com', 623);
// log(guid);
// ```
func GetSystemGUID(ctx context.Context, host string, port int) (string, error) {
s, err := dial(ctx, host, port)
if err != nil {
return "", err
}
defer s.close()
data, err := s.command(cmdGetSystemGUID, nil)
if err != nil {
return "", err
}
if len(data) < 16 {
return "", fmt.Errorf("short system guid response")
}
return hex.EncodeToString(data[:16]), nil
}
// GetCipherSuites returns the IDs of the RMCP+ cipher suites the BMC
// offers. Cipher suite 0 authenticates nothing, so any password logs in.
// @example
// ```javascript
// const ipmi = require('nuclei/ipmi');
// const suites = ipmi.GetCipherSuites('acme.com', 623);
// log(suites.includes(0));
// ```
func GetCipherSuites(ctx context.Context, host string, port int) ([]int, error) {
s, err := dial(ctx, host, port)
if err != nil {
return nil, err
}
defer s.close()
// the records come in pages of 16 bytes; a shorter page is the last
var records []byte
for index := byte(0); index < 64; index++ {
data, err := s.command(cmdGetCipherSuites, []byte{currentChannel, 0x00, 0x80 | index})
if err != nil {
return nil, err
}
if len(data) > 0 {
records = append(records, data[1:]...)
}
if len(data) < 17 {
break
}
}
return parseCipherSuites(records), nil
}
// parseCipherSuites returns the suite IDs in Get Channel Cipher Suites
// record data. A record starts with 0xc0 (standard) or 0xc1 (oem, followed
// by a 3 byte IANA); algorithm bytes are tagged below 0xc0, so they never
// look like a record start.
func parseCipherSuites(records []byte) []int {
suites := []int{}
for i := 0; i < len(records); i++ {
switch records[i] {
case 0xc0:
if i+1 < len(records) {
suites = append(suites, int(records[i+1]))
i++
}
case 0xc1:
if i+1 < len(records) {
suites = append(suites, int(records[i+1]))
i += 4
}
}
}
return suites
}
// GetRAKPHash starts an RMCP+ session as username and returns the HMAC the
// BMC answers with. The session is never completed, so no login is
// attempted; the BMC drops the half-open session after its timeout. It fails
// when the BMC rejects the username.
// @example
// ```javascript
// const ipmi = require('nuclei/ipmi');
// const hash = ipmi.GetRAKPHash('acme.com', 623, 'ADMIN');
// log(ipmi.CheckRAKPPassword(hash, 'ADMIN'));
// ```
func GetRAKPHash(ctx context.Context, host string, port int, username string) (*RAKPHash, error) {
s, err := dial(ctx, host, port)
if err != nil {
return nil, err
}
defer s.close()
// the session is never completed, so no login is attempted; md5 is for
// BMCs that refuse sha1
var h *handshake
for _, auth := range []byte{authHMACSHA1, authHMACMD5} {
if h, err = s.startSession(algorithms{auth: auth}, username, privAdmin); err == nil {
break
}
var refused *rakpError
if !errors.As(err, &refused) || refused.step != "the session" {
return nil, err
}
}
if err != nil {
return nil, err
}
hash := &RAKPHash{Username: username, Algorithm: "sha1", Salt: hex.EncodeToString(h.rakp2Salt()), HMAC: hex.EncodeToString(h.bmcMAC)}
if h.auth == authHMACMD5 {
hash.Algorithm = "md5"
} else {
hash.Hashcat = hash.Salt + ":" + hash.HMAC
}
return hash, nil
}
// CheckRAKPPassword reports whether password produced hash. It runs offline,
// so it can test any number of passwords without touching the BMC.
// @example
// ```javascript
// const ipmi = require('nuclei/ipmi');
// const hash = ipmi.GetRAKPHash('acme.com', 623, 'ADMIN');
// log(ipmi.CheckRAKPPassword(hash, 'ADMIN'));
// ```
func CheckRAKPPassword(hash *RAKPHash, password string) bool {
if hash == nil {
return false
}
salt, err := hex.DecodeString(hash.Salt)
if err != nil {
return false
}
want, err := hex.DecodeString(hash.HMAC)
if err != nil {
return false
}
newHash := sha1.New
if hash.Algorithm == "md5" {
newHash = md5.New
}
return hmac.Equal(hmacOf(newHash, []byte(password), salt), want)
}
// FindRAKPPassword returns the first of passwords that produced hash, or an
// empty string.
// @example
// ```javascript
// const ipmi = require('nuclei/ipmi');
// const hash = ipmi.GetRAKPHash('acme.com', 623, 'root');
// log(ipmi.FindRAKPPassword(hash, ['calvin', 'root', 'admin']));
// ```
func FindRAKPPassword(hash *RAKPHash, passwords []string) string {
for _, password := range passwords {
if CheckRAKPPassword(hash, password) {
return password
}
}
return ""
}
// Login opens an RMCP+ session as username with the given cipher suite
// (0, 1, 2, 3 or 6). Cipher suite 0 skips authentication, so a BMC
// that allows it accepts any password. A rejected login is an error.
// @example
// ```javascript
// const ipmi = require('nuclei/ipmi');
// const client = new ipmi.Client();
// client.Login('acme.com', 623, 'ADMIN', 'not-the-password', 0);
// ```
func (c *Client) Login(ctx context.Context, host string, port int, username string, password string, cipherSuite int) (bool, error) {
alg, ok := cipherSuites[cipherSuite]
if !ok {
return false, fmt.Errorf("unsupported cipher suite %d", cipherSuite)
}
c.Close()
s, err := dial(ctx, host, port)
if err != nil {
return false, err
}
h, err := s.startSession(alg, username, privAdmin)
if err != nil {
s.close()
return false, err
}
k, err := s.finishSession(h, password)
if err != nil {
s.close()
return false, err
}
c.session = &established{session: s, algorithms: alg, keys: k, bmcSessionID: h.bmcSessionID, consoleSessionID: h.consoleSessionID}
return true, nil
}
// Command sends an IPMI request in the session, such as Get Device ID
// (netFn 0x06, cmd 0x01) or Set Session Privilege Level (0x06, 0x3b).
// @example
// ```javascript
// const ipmi = require('nuclei/ipmi');
// const client = new ipmi.Client();
// client.Login('acme.com', 623, 'ADMIN', 'ADMIN', 3);
// const admin = client.Command(0x06, 0x3b, [0x04]).CompletionCode == 0;
// client.Close();
// ```
func (c *Client) Command(netFn int, cmd int, data []byte) (*CommandResponse, error) {
if c.session == nil {
return nil, fmt.Errorf("not logged in")
}
code, reply, err := c.session.command(byte(netFn), byte(cmd), data)
if err != nil {
return nil, err
}
return &CommandResponse{CompletionCode: int(code), Data: reply}, nil
}
// Close ends the session on the BMC, freeing its slot.
// @example
// ```javascript
// const ipmi = require('nuclei/ipmi');
// const client = new ipmi.Client();
// client.Login('acme.com', 623, 'ADMIN', 'ADMIN', 3);
// client.Close();
// ```
func (c *Client) Close() {
if c.session == nil {
return
}
_, _, _ = c.session.command(netFnApp, cmdCloseSession, c.session.bmcSessionID)
c.session.close()
c.session = nil
}
+178
View File
@@ -0,0 +1,178 @@
package ipmi
import (
"bytes"
"context"
"crypto/sha1"
"encoding/hex"
"net"
"testing"
"time"
"github.com/projectdiscovery/nuclei/v3/pkg/protocols/common/protocolstate"
"github.com/projectdiscovery/nuclei/v3/pkg/types"
"github.com/stretchr/testify/require"
)
// The fixtures were captured from OpenIPMI's ipmi_sim with an enabled user
// with an empty name and password, ADMIN/ADMIN and root/calvin.
func mustHex(t *testing.T, s string) []byte {
t.Helper()
data, err := hex.DecodeString(s)
require.NoError(t, err)
return data
}
func TestParseAuthCapabilities(t *testing.T) {
caps, err := parseAuthCapabilities(mustHex(t, "0197050300000000"))
require.NoError(t, err)
require.Equal(t, &AuthCapabilities{
IPMI20: true,
AuthTypes: []string{"none", "md2", "md5", "straight"},
AnonymousLogin: true,
NonNullUsernames: true,
PerMessageAuth: true,
UserLevelAuth: true,
}, caps)
_, err = parseAuthCapabilities(mustHex(t, "0197"))
require.Error(t, err)
}
func TestParseCipherSuites(t *testing.T) {
records := mustHex(t, "c000004080c001014080c002014180c003014181c004014182c005014183c006024080c007024280c008024281c009024282c00a024283c00b024380c00c024381c00d024382c00e024383")
require.Equal(t, []int{0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14}, parseCipherSuites(records))
// an oem record carries a 3 byte IANA before its algorithms
require.Equal(t, []int{1, 7}, parseCipherSuites(mustHex(t, "c0010140c107d70200014080")))
}
func TestRAKPPasswordChecksAreOffline(t *testing.T) {
// hashcat -m 7300 cracks this hash to calvin as well
hash := &RAKPHash{
Username: "root",
Salt: "988200cd0402000069b30c7a5e3deb30504c5b76f7b1840d5e36439998d9e0f59ae49b09119a10b2a123456789abcdefa123456789abcdef1404726f6f74",
HMAC: "23c711e5c7a737c3c0beb9f4680ef31fa6aad398",
}
require.True(t, CheckRAKPPassword(hash, "calvin"))
require.False(t, CheckRAKPPassword(hash, "ADMIN"))
require.Equal(t, "calvin", FindRAKPPassword(hash, []string{"admin", "root", "calvin"}))
require.Empty(t, FindRAKPPassword(hash, []string{"admin"}))
require.False(t, CheckRAKPPassword(nil, "calvin"))
require.False(t, CheckRAKPPassword(&RAKPHash{Salt: "zz", HMAC: "00"}, "calvin"))
}
func TestRequestsHonourNetworkPolicy(t *testing.T) {
execID := "ipmi-deny"
require.NoError(t, protocolstate.Init(&types.Options{ExecutionId: execID, ExcludeTargets: []string{"203.0.113.50"}}))
t.Cleanup(func() { protocolstate.Close(execID) })
ctx := context.WithValue(context.Background(), "executionId", execID) //nolint:staticcheck
_, err := GetRAKPHash(ctx, "203.0.113.50", 623, "ADMIN")
require.ErrorContains(t, err, "denied")
require.NotErrorIs(t, err, errNoResponse, "denied before sending, not timed out")
_, err = Ping(ctx, "203.0.113.50", 623)
require.ErrorContains(t, err, "denied")
}
func TestPayloadEncryptionRoundTrips(t *testing.T) {
key := mustHex(t, "000102030405060708090a0b0c0d0e0f")
for size := range 40 {
payload := make([]byte, size)
for i := range payload {
payload[i] = byte(i)
}
sealed, err := encryptPayload(key, payload)
require.NoError(t, err)
require.Zero(t, len(sealed)%16, "iv and padded blocks")
opened, err := decryptPayload(key, sealed)
require.NoError(t, err)
require.Equal(t, payload, opened)
}
_, err := decryptPayload(key, []byte{1, 2, 3})
require.Error(t, err)
}
func TestSignAlignsTheIntegrityPad(t *testing.T) {
e := &established{algorithms: cipherSuites[3], keys: &keys{integrity: make([]byte, 20)}}
for size := range 8 {
packet := append(rmcpHeader(rmcpClassIPMI), make([]byte, 12+size)...)
signed := e.sign(packet)
// everything from the auth type through the next header is a
// multiple of 4, then 12 bytes of HMAC-SHA1-96
require.Zero(t, (len(signed)-12-4)%4)
require.Equal(t, byte(0x07), signed[len(signed)-13])
}
}
func TestLoginRejectsUnsupportedCipherSuites(t *testing.T) {
_, err := (&Client{}).Login(context.Background(), "127.0.0.1", 623, "ADMIN", "ADMIN", 17)
require.ErrorContains(t, err, "unsupported cipher suite 17")
_, err = (&Client{}).Command(0x06, 0x01, nil)
require.ErrorContains(t, err, "not logged in")
}
// capturedHandshake is a cipher suite 3 session setup with ipmi_sim for
// ADMIN/ADMIN, up to and including RAKP message 4.
func capturedHandshake(t *testing.T) (*handshake, []byte) {
return &handshake{
algorithms: cipherSuites[3],
consoleSessionID: mustHex(t, "244a5ac8"),
bmcSessionID: mustHex(t, "04170000"),
consoleRandom: mustHex(t, "de862025bb0c5ee6c30a58d383d44cb5"),
bmcRandom: mustHex(t, "b56fef2ee0fcab6664a019a4c88df4f6"),
bmcGUID: mustHex(t, "a123456789abcdefa123456789abcdef"),
role: 0x14,
username: "ADMIN",
bmcMAC: mustHex(t, "7fac6f776553c58df508b9ff656b5eacb6c2def9"),
}, mustHex(t, "00000000244a5ac8231219c5fc212cdd58c09424")
}
func TestBMCProofsMatchOnlyTheRealPassword(t *testing.T) {
h, rakp4 := capturedHandshake(t)
sik := func(password string) []byte {
return hmacOf(sha1.New, []byte(password), h.consoleRandom, h.bmcRandom, []byte{h.role, byte(len(h.username))}, []byte(h.username))
}
require.Equal(t, h.bmcMAC, hmacOf(sha1.New, []byte("ADMIN"), h.rakp2Salt()), "rakp 2 code")
require.NotEqual(t, h.bmcMAC, hmacOf(sha1.New, []byte("wrong"), h.rakp2Salt()))
require.True(t, rakp4Valid(sha1.New, sik("ADMIN"), h, rakp4))
require.False(t, rakp4Valid(sha1.New, sik("wrong"), h, rakp4))
tampered := bytes.Clone(rakp4)
tampered[len(tampered)-1] ^= 0xff
require.False(t, rakp4Valid(sha1.New, sik("ADMIN"), h, tampered), "a forged success must not pass")
require.False(t, rakp4Valid(sha1.New, sik("ADMIN"), h, rakp4[:10]))
}
func TestShortCommandReplyIsRejectedNotPanicked(t *testing.T) {
execID := "ipmi-short"
require.NoError(t, protocolstate.Init(&types.Options{ExecutionId: execID}))
t.Cleanup(func() { protocolstate.Close(execID) })
ctx := context.WithValue(context.Background(), "executionId", execID) //nolint:staticcheck
// a reply that ends at the completion code, without the checksum
bmc, err := net.ListenPacket("udp", "127.0.0.1:0")
require.NoError(t, err)
t.Cleanup(func() { _ = bmc.Close() })
go func() {
buf := make([]byte, 512)
for {
n, addr, err := bmc.ReadFrom(buf)
if err != nil {
return
}
reply := bytes.Clone(buf[:21])
reply[19] = buf[n-2] // echo the command
reply[20] = 0x00
_, _ = bmc.WriteTo(reply, addr)
}
}()
port := bmc.LocalAddr().(*net.UDPAddr).Port
ctx, cancel := context.WithTimeout(ctx, 2*time.Second)
defer cancel()
_, err = GetSystemGUID(ctx, "127.0.0.1", port)
require.ErrorIs(t, err, errNoResponse)
}
+173
View File
@@ -0,0 +1,173 @@
package ipmi
import (
"bytes"
"context"
"encoding/binary"
"errors"
"fmt"
"net"
"time"
libnet "github.com/projectdiscovery/nuclei/v3/pkg/js/libs/net"
)
// Wire constants from the IPMI v2.0 specification and DSP0136 (ASF).
const (
rmcpVersion = 0x06
rmcpNoAck = 0xff
rmcpClassASF = 0x06
rmcpClassIPMI = 0x07
asfIANA = 0x000011be
asfTypePing = 0x80
asfTypePong = 0x40
asfEntitiesIPMI = 0x80
authTypeNone = 0x00
authTypeRMCPP = 0x06
netFnApp = 0x06
cmdGetSystemGUID = 0x37
cmdGetChannelAuthCaps = 0x38
cmdGetCipherSuites = 0x54
bmcAddress = 0x20
consoleAddress = 0x81
currentChannel = 0x0e
privAdmin = 0x04
payloadOpenSessionRequest = 0x10
payloadOpenSessionResponse = 0x11
payloadRAKP1 = 0x12
payloadRAKP2 = 0x13
payloadRAKP3 = 0x14
payloadRAKP4 = 0x15
// rakpNameOnlyLookup asks the BMC to find the user by name alone, so the
// role does not have to match the one the account was created with.
rakpNameOnlyLookup = 0x10
attempts = 2
)
var errNoResponse = errors.New("no ipmi response")
// session is one UDP exchange with a BMC. UDP is lossy, so each request is
// sent up to attempts times.
type session struct {
conn net.Conn
timeout time.Duration
}
func dial(ctx context.Context, host string, port int) (*session, error) {
conn, err := libnet.Dial(ctx, "udp", net.JoinHostPort(host, fmt.Sprint(port)))
if err != nil {
return nil, err
}
timeout := 3 * time.Second
if deadline, ok := ctx.Deadline(); ok {
timeout = min(timeout, time.Until(deadline)/attempts)
}
return &session{conn: conn, timeout: timeout}, nil
}
func (s *session) close() { _ = s.conn.Close() }
// exchange sends packet and returns the first reply accepted by match.
func (s *session) exchange(packet []byte, match func([]byte) bool) ([]byte, error) {
buf := make([]byte, 1024)
for range attempts {
if _, err := s.conn.Write(packet); err != nil {
return nil, err
}
_ = s.conn.SetReadDeadline(time.Now().Add(s.timeout))
for {
n, err := s.conn.Read(buf)
if err != nil {
break
}
if reply := buf[:n]; match(reply) {
return bytes.Clone(reply), nil
}
}
}
return nil, errNoResponse
}
func rmcpHeader(class byte) []byte {
return []byte{rmcpVersion, 0x00, rmcpNoAck, class}
}
// ping sends an ASF presence ping and reports whether the pong advertises
// IPMI support.
func (s *session) ping() (bool, error) {
packet := append(rmcpHeader(rmcpClassASF), 0, 0, 0, 0, asfTypePing, 0x00, 0x00, 0x00)
binary.BigEndian.PutUint32(packet[4:8], asfIANA)
reply, err := s.exchange(packet, func(reply []byte) bool {
return len(reply) >= 12 && reply[3] == rmcpClassASF && reply[8] == asfTypePong
})
if err != nil {
return false, err
}
// pong data: IANA (4), OEM (4), supported entities (1), interactions (1)
if len(reply) < 21 {
return false, nil
}
return reply[20]&asfEntitiesIPMI != 0, nil
}
func checksum(data []byte) byte {
var sum byte
for _, b := range data {
sum += b
}
return -sum
}
// command sends a session-less IPMI 1.5 request and returns the response
// data after a successful completion code.
func (s *session) command(cmd byte, data []byte) ([]byte, error) {
header := []byte{bmcAddress, netFnApp << 2}
body := append([]byte{consoleAddress, 0x00, cmd}, data...)
message := append(append(header, checksum(header)), body...)
message = append(message, checksum(body))
packet := append(rmcpHeader(rmcpClassIPMI), authTypeNone, 0, 0, 0, 0, 0, 0, 0, 0, byte(len(message)))
packet = append(packet, message...)
reply, err := s.exchange(packet, func(reply []byte) bool {
// rmcp (4), session header (10), then rqAddr, netFn, checksum,
// rsAddr, seq, cmd, completion code
// ... completion code, then at least the trailing checksum
return len(reply) >= 22 && reply[3] == rmcpClassIPMI && reply[4] == authTypeNone && reply[19] == cmd
})
if err != nil {
return nil, err
}
if code := reply[20]; code != 0 {
return nil, fmt.Errorf("ipmi command 0x%02x failed with completion code 0x%02x", cmd, code)
}
// drop the trailing checksum
return reply[21 : len(reply)-1], nil
}
// rmcpPlus sends an IPMI 2.0 session-setup payload and returns the reply
// payload of the expected type.
func (s *session) rmcpPlus(payloadType byte, payload []byte, replyType byte) ([]byte, error) {
packet := append(rmcpHeader(rmcpClassIPMI), authTypeRMCPP, payloadType, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0)
binary.LittleEndian.PutUint16(packet[14:16], uint16(len(payload)))
packet = append(packet, payload...)
reply, err := s.exchange(packet, func(reply []byte) bool {
return len(reply) >= 16 && reply[3] == rmcpClassIPMI && reply[4] == authTypeRMCPP && reply[5]&0x3f == replyType
})
if err != nil {
return nil, err
}
length := int(binary.LittleEndian.Uint16(reply[14:16]))
if len(reply) < 16+length {
return nil, fmt.Errorf("truncated ipmi payload")
}
return reply[16 : 16+length], nil
}
+398
View File
@@ -0,0 +1,398 @@
package ipmi
import (
"bytes"
"crypto/aes"
"crypto/cipher"
"crypto/hmac"
"crypto/md5"
"crypto/rand"
"crypto/sha1"
"encoding/binary"
"errors"
"fmt"
"hash"
)
// RMCP+ algorithm numbers (IPMI v2.0 tables 13-17 to 13-19).
const (
authNone = 0x00
authHMACSHA1 = 0x01
authHMACMD5 = 0x02
integrityNone = 0x00
integrityHMACSHA196 = 0x01
confidentialityNone = 0x00
confidentialityAESCBC128 = 0x01
payloadIPMI = 0x00
payloadEncrypted = 0x80
payloadSigned = 0x40
cmdCloseSession = 0x3c
// rakp status codes worth naming in errors
rakpInvalidRole = 0x09
rakpUnauthorizedName = 0x0d
)
type algorithms struct {
auth, integrity, confidentiality byte
}
// cipherSuites lists the suites a session can be opened with. Suites with
// SHA-256 or MD5 message integrity are left out until they can be checked
// against a BMC that implements them correctly.
var cipherSuites = map[int]algorithms{
0: {authNone, integrityNone, confidentialityNone},
1: {authHMACSHA1, integrityNone, confidentialityNone},
2: {authHMACSHA1, integrityHMACSHA196, confidentialityNone},
3: {authHMACSHA1, integrityHMACSHA196, confidentialityAESCBC128},
6: {authHMACMD5, integrityNone, confidentialityNone},
}
func authHash(auth byte) func() hash.Hash {
switch auth {
case authHMACSHA1:
return sha1.New
case authHMACMD5:
return md5.New
}
return nil
}
func hmacOf(newHash func() hash.Hash, key []byte, parts ...[]byte) []byte {
mac := hmac.New(newHash, key)
for _, part := range parts {
_, _ = mac.Write(part)
}
return mac.Sum(nil)
}
// handshake is the state of an RMCP+ session setup up to RAKP message 2.
type handshake struct {
algorithms
consoleSessionID, bmcSessionID []byte
consoleRandom, bmcRandom []byte
bmcGUID []byte
role byte
username string
// bmcMAC is the RAKP message 2 key exchange code, empty without
// authentication.
bmcMAC []byte
}
// rakpError is a non-zero status in an RMCP+ session setup reply.
type rakpError struct {
step string
status byte
}
func (e *rakpError) Error() string {
switch e.status {
case rakpUnauthorizedName:
return fmt.Sprintf("bmc rejected the username in %s (status 0x%02x)", e.step, e.status)
case rakpInvalidRole:
return fmt.Sprintf("bmc rejected the requested privilege in %s (status 0x%02x)", e.step, e.status)
}
return fmt.Sprintf("bmc refused %s with status 0x%02x", e.step, e.status)
}
// startSession opens an RMCP+ session and runs RAKP messages 1 and 2.
func (s *session) startSession(alg algorithms, username string, privilege byte) (*handshake, error) {
if len(username) > 16 {
return nil, fmt.Errorf("ipmi usernames are at most 16 bytes")
}
h := &handshake{
algorithms: alg,
consoleSessionID: make([]byte, 4),
consoleRandom: make([]byte, 16),
role: privilege | rakpNameOnlyLookup,
username: username,
}
if _, err := rand.Read(h.consoleSessionID); err != nil {
return nil, err
}
if _, err := rand.Read(h.consoleRandom); err != nil {
return nil, err
}
open := []byte{0x00, privilege, 0x00, 0x00}
open = append(open, h.consoleSessionID...)
open = append(open,
0x00, 0x00, 0x00, 0x08, alg.auth, 0x00, 0x00, 0x00,
0x01, 0x00, 0x00, 0x08, alg.integrity, 0x00, 0x00, 0x00,
0x02, 0x00, 0x00, 0x08, alg.confidentiality, 0x00, 0x00, 0x00,
)
opened, err := s.rmcpPlus(payloadOpenSessionRequest, open, payloadOpenSessionResponse)
if err != nil {
return nil, err
}
if len(opened) < 2 {
return nil, fmt.Errorf("short open session response")
}
if opened[1] != 0 {
return nil, &rakpError{step: "the session", status: opened[1]}
}
if len(opened) < 12 {
return nil, fmt.Errorf("short open session response")
}
h.bmcSessionID = bytes.Clone(opened[8:12])
rakp1 := []byte{0x00, 0x00, 0x00, 0x00}
rakp1 = append(rakp1, h.bmcSessionID...)
rakp1 = append(rakp1, h.consoleRandom...)
rakp1 = append(rakp1, h.role, 0x00, 0x00, byte(len(username)))
rakp1 = append(rakp1, username...)
rakp2, err := s.rmcpPlus(payloadRAKP1, rakp1, payloadRAKP2)
if err != nil {
return nil, err
}
if len(rakp2) < 2 {
return nil, fmt.Errorf("short rakp message 2")
}
if rakp2[1] != 0 {
return nil, &rakpError{step: "rakp message 1", status: rakp2[1]}
}
if len(rakp2) < 40 {
return nil, fmt.Errorf("short rakp message 2")
}
h.bmcRandom = bytes.Clone(rakp2[8:24])
h.bmcGUID = bytes.Clone(rakp2[24:40])
h.bmcMAC = bytes.Clone(rakp2[40:])
return h, nil
}
// rakp2Salt is everything the RAKP message 2 code covers.
func (h *handshake) rakp2Salt() []byte {
salt := make([]byte, 0, 58+len(h.username))
salt = append(salt, h.consoleSessionID...)
salt = append(salt, h.bmcSessionID...)
salt = append(salt, h.consoleRandom...)
salt = append(salt, h.bmcRandom...)
salt = append(salt, h.bmcGUID...)
salt = append(salt, h.role, byte(len(h.username)))
return append(salt, h.username...)
}
// keys are the session keys derived from the session integrity key.
type keys struct {
integrity []byte // K1
aes []byte // first 16 bytes of K2
}
// finishSession runs RAKP messages 3 and 4, which is where a wrong password
// is rejected, and derives the session keys. Both directions are proven:
// the BMC's RAKP message 2 code and RAKP message 4 integrity value must
// match the password, so a peer that only answers with success statuses
// cannot pass for a login.
func (s *session) finishSession(h *handshake, password string) (*keys, error) {
if len(password) > 20 {
return nil, fmt.Errorf("ipmi passwords are at most 20 bytes")
}
newHash := authHash(h.auth)
if newHash != nil && !hmac.Equal(h.bmcMAC, hmacOf(newHash, []byte(password), h.rakp2Salt())) {
return nil, errBMCProof
}
rakp3 := []byte{0x00, 0x00, 0x00, 0x00}
rakp3 = append(rakp3, h.bmcSessionID...)
if newHash != nil {
rakp3 = append(rakp3, hmacOf(newHash, []byte(password),
h.bmcRandom, h.consoleSessionID, []byte{h.role, byte(len(h.username))}, []byte(h.username))...)
}
rakp4, err := s.rmcpPlus(payloadRAKP3, rakp3, payloadRAKP4)
if err != nil {
return nil, err
}
if len(rakp4) < 2 {
return nil, fmt.Errorf("short rakp message 4")
}
if rakp4[1] != 0 {
return nil, &rakpError{step: "the login", status: rakp4[1]}
}
if newHash == nil {
return &keys{}, nil
}
sik := hmacOf(newHash, []byte(password),
h.consoleRandom, h.bmcRandom, []byte{h.role, byte(len(h.username))}, []byte(h.username))
if !rakp4Valid(newHash, sik, h, rakp4) {
return nil, errBMCProof
}
return &keys{
integrity: hmacOf(newHash, sik, bytes.Repeat([]byte{0x01}, 20)),
aes: hmacOf(newHash, sik, bytes.Repeat([]byte{0x02}, 20))[:16],
}, nil
}
// errBMCProof means the peer's authentication code does not match the
// password: a wrong password, or something other than the BMC answering.
var errBMCProof = errors.New("bmc authentication code does not match the password")
// rakp4Valid checks the RAKP message 4 integrity value, the session key's
// HMAC over the console random, BMC session ID and BMC GUID, truncated to
// 12 bytes for sha1 and 16 for md5.
func rakp4Valid(newHash func() hash.Hash, sik []byte, h *handshake, rakp4 []byte) bool {
size := 12
if h.auth == authHMACMD5 {
size = 16
}
if len(rakp4) < 8+size {
return false
}
want := hmacOf(newHash, sik, h.consoleRandom, h.bmcSessionID, h.bmcGUID)[:size]
return hmac.Equal(rakp4[8:8+size], want)
}
// established is an authenticated RMCP+ session.
type established struct {
*session
algorithms
keys *keys
bmcSessionID []byte
consoleSessionID []byte
sequence uint32
requestSeq byte
}
// command sends an IPMI request inside the session and returns its
// completion code and response data.
func (e *established) command(netFn, cmd byte, data []byte) (byte, []byte, error) {
// the request sequence is 6 bits on the wire
e.requestSeq = (e.requestSeq + 1) & 0x3f
header := []byte{bmcAddress, netFn << 2}
body := append([]byte{consoleAddress, e.requestSeq << 2, cmd}, data...)
message := append(append(header, checksum(header)), body...)
message = append(message, checksum(body))
payloadType := byte(payloadIPMI)
if e.confidentiality == confidentialityAESCBC128 {
payloadType |= payloadEncrypted
var err error
if message, err = encryptPayload(e.keys.aes, message); err != nil {
return 0, nil, err
}
}
if e.integrity != integrityNone {
payloadType |= payloadSigned
}
e.sequence++
packet := append(rmcpHeader(rmcpClassIPMI), authTypeRMCPP, payloadType)
packet = append(packet, e.bmcSessionID...)
packet = binary.LittleEndian.AppendUint32(packet, e.sequence)
packet = binary.LittleEndian.AppendUint16(packet, uint16(len(message)))
packet = append(packet, message...)
if e.integrity != integrityNone {
packet = e.sign(packet)
}
// a reply must be for this session, pass the integrity check when the
// session is signed, and answer this request, not an earlier one that
// arrived late
var payload []byte
_, err := e.exchange(packet, func(reply []byte) bool {
payload = e.open(reply)
return len(payload) >= 8 && payload[4]>>2 == e.requestSeq && payload[5] == cmd
})
if err != nil {
return 0, nil, err
}
// rqAddr, netFn, checksum, rsAddr, seq, cmd, completion code, data,
// checksum
return payload[6], bytes.Clone(payload[7 : len(payload)-1]), nil
}
// open returns the IPMI message in a session reply, or nil when the reply
// is not for this session or fails its integrity check.
func (e *established) open(reply []byte) []byte {
if len(reply) < 16 || reply[3] != rmcpClassIPMI || reply[4] != authTypeRMCPP || reply[5]&0x3f != payloadIPMI {
return nil
}
if !bytes.Equal(reply[6:10], e.consoleSessionID) {
return nil
}
length := int(binary.LittleEndian.Uint16(reply[14:16]))
if len(reply) < 16+length {
return nil
}
if e.integrity != integrityNone {
if reply[5]&payloadSigned == 0 || !e.verify(reply, 16+length) {
return nil
}
}
payload := reply[16 : 16+length]
if reply[5]&payloadEncrypted != 0 {
var err error
if payload, err = decryptPayload(e.keys.aes, payload); err != nil {
return nil
}
}
return payload
}
// verify checks a reply's HMAC-SHA1-96 trailer: the pad after the payload,
// pad length and next header, then the code over everything from the auth
// type on.
func (e *established) verify(reply []byte, payloadEnd int) bool {
const codeSize = 12
if len(reply) < payloadEnd+2+codeSize {
return false
}
codeStart := len(reply) - codeSize
if reply[codeStart-1] != 0x07 || int(reply[codeStart-2]) != codeStart-2-payloadEnd {
return false
}
want := hmacOf(sha1.New, e.keys.integrity, reply[4:codeStart])[:codeSize]
return hmac.Equal(reply[codeStart:], want)
}
// sign appends the HMAC-SHA1-96 integrity trailer: pad to a multiple of 4 from the auth
// type on, pad length, next header, and the truncated HMAC over it all.
func (e *established) sign(packet []byte) []byte {
pad := (4 - (len(packet)-4+2)%4) % 4
packet = append(packet, bytes.Repeat([]byte{0xff}, pad)...)
packet = append(packet, byte(pad), 0x07)
return append(packet, hmacOf(sha1.New, e.keys.integrity, packet[4:])[:12]...)
}
// encryptPayload is AES-CBC-128 with a random IV in front and the
// confidentiality pad (1, 2, 3, ... then its length) at the end.
func encryptPayload(key, payload []byte) ([]byte, error) {
block, err := aes.NewCipher(key)
if err != nil {
return nil, err
}
pad := (aes.BlockSize - (len(payload)+1)%aes.BlockSize) % aes.BlockSize
plain := bytes.Clone(payload)
for i := 1; i <= pad; i++ {
plain = append(plain, byte(i))
}
plain = append(plain, byte(pad))
out := make([]byte, aes.BlockSize+len(plain))
if _, err := rand.Read(out[:aes.BlockSize]); err != nil {
return nil, err
}
cipher.NewCBCEncrypter(block, out[:aes.BlockSize]).CryptBlocks(out[aes.BlockSize:], plain)
return out, nil
}
func decryptPayload(key, payload []byte) ([]byte, error) {
if len(payload) < 2*aes.BlockSize || len(payload)%aes.BlockSize != 0 {
return nil, fmt.Errorf("malformed encrypted payload")
}
block, err := aes.NewCipher(key)
if err != nil {
return nil, err
}
plain := make([]byte, len(payload)-aes.BlockSize)
cipher.NewCBCDecrypter(block, payload[:aes.BlockSize]).CryptBlocks(plain, payload[aes.BlockSize:])
pad := int(plain[len(plain)-1])
if pad+1 > len(plain) {
return nil, fmt.Errorf("malformed encrypted payload")
}
return plain[:len(plain)-pad-1], nil
}
+596
View File
@@ -0,0 +1,596 @@
package mqtt
import (
"context"
"crypto/tls"
"errors"
"fmt"
"net"
"strconv"
"sync"
"time"
libnet "github.com/projectdiscovery/nuclei/v3/pkg/js/libs/net"
)
var (
defaultTimeout = 10 * time.Second
)
const (
maxSubscribedMessages = 1000
maxSubscribedBytes = 10 * 1024 * 1024 // 10MB
)
type (
// Message represents an MQTT published message.
// @example
// ```javascript
// const mqtt = require('nuclei/mqtt');
// const client = new mqtt.Client();
// client.Connect('broker.hivemq.com', 1883);
// const msgs = client.Subscribe('test/topic', 2);
// log(msgs[0].Topic, msgs[0].Payload);
// ```
Message struct {
Topic string `json:"topic"`
Payload string `json:"payload"`
QoS int `json:"qos"`
Retain bool `json:"retain"`
}
// ConnectOptions contains connection options for an MQTT broker.
// @example
// ```javascript
// const mqtt = require('nuclei/mqtt');
// const opts = new mqtt.ConnectOptions();
// opts.Username = 'admin';
// opts.Password = 'admin';
// ```
ConnectOptions struct {
Username string `json:"username,omitempty"`
Password string `json:"password,omitempty"`
ClientID string `json:"client_id,omitempty"`
CleanSession *bool `json:"clean_session,omitempty"`
KeepAlive int `json:"keep_alive,omitempty"`
TLS bool `json:"tls,omitempty"`
InsecureSkipVerify *bool `json:"insecure_skip_verify,omitempty"`
Timeout int `json:"timeout,omitempty"`
}
// ConnectResult is the response returned by an MQTT connection attempt.
// @example
// ```javascript
// const mqtt = require('nuclei/mqtt');
// const client = new mqtt.Client();
// const res = client.Connect('acme.com', 1883);
// log(res.Success, res.ReturnCode, res.Reason);
// ```
ConnectResult struct {
Success bool `json:"success"`
ReturnCode int `json:"return_code"`
Reason string `json:"reason"`
SessionPresent bool `json:"session_present"`
}
// Client is an MQTT client for testing MQTT broker misconfigurations and authentication.
// @example
// ```javascript
// const mqtt = require('nuclei/mqtt');
// const client = new mqtt.Client();
// const res = client.Connect('acme.com', 1883);
// if (res.Success) {
// const msgs = client.Subscribe('#', 2);
// client.Publish('nuclei/probe', 'test');
// }
// client.Close();
// ```
Client struct {
conn net.Conn
packetID uint16
mu sync.Mutex
timeout time.Duration
}
)
// dial opens a network connection to host:port respecting protocolstate policy and context.
func dial(ctx context.Context, host string, port int, useTLS bool, insecureSkipVerify bool, timeout time.Duration) (net.Conn, error) {
if host == "" || port <= 0 {
return nil, fmt.Errorf("invalid host or port")
}
addr := net.JoinHostPort(host, strconv.Itoa(port))
conn, err := libnet.Dial(ctx, "tcp", addr)
if err != nil {
return nil, err
}
_ = conn.SetDeadline(time.Now().Add(timeout))
if useTLS {
tlsConn := tls.Client(conn, &tls.Config{
InsecureSkipVerify: insecureSkipVerify,
MinVersion: tls.VersionTLS12,
ServerName: host,
})
if err := tlsConn.HandshakeContext(ctx); err != nil {
_ = conn.Close()
return nil, fmt.Errorf("TLS handshake failed: %w", err)
}
conn = tlsConn
}
_ = conn.SetDeadline(time.Now().Add(timeout))
return conn, nil
}
// IsMQTT checks whether the target host:port is running an MQTT broker.
// @example
// ```javascript
// const mqtt = require('nuclei/mqtt');
// const isMqtt = mqtt.IsMQTT('acme.com', 1883);
// log(isMqtt);
// ```
func IsMQTT(ctx context.Context, host string, port int) (bool, error) {
conn, err := dial(ctx, host, port, false, true, defaultTimeout)
if err != nil {
return false, err
}
defer func() { _ = conn.Close() }()
pkt, err := buildConnectPacket("nuclei-probe", "", "", true, 60)
if err != nil {
return false, err
}
_ = conn.SetWriteDeadline(time.Now().Add(defaultTimeout))
if _, err := conn.Write(pkt); err != nil {
return false, err
}
header, payload, err := readPacket(conn)
if err != nil {
return false, nil
}
// Any valid CONNACK indicates an active MQTT broker
if (header >> 4) == PacketConnack {
_, _, err := parseConnack(payload)
return err == nil, nil
}
return false, nil
}
// IsAnonymousAllowed checks whether an MQTT broker permits unauthenticated / anonymous login.
// @example
// ```javascript
// const mqtt = require('nuclei/mqtt');
// const allowed = mqtt.IsAnonymousAllowed('acme.com', 1883);
// log(allowed);
// ```
func IsAnonymousAllowed(ctx context.Context, host string, port int) (bool, error) {
conn, err := dial(ctx, host, port, false, true, defaultTimeout)
if err != nil {
return false, err
}
defer func() { _ = conn.Close() }()
pkt, err := buildConnectPacket("", "", "", true, 60)
if err != nil {
return false, err
}
_ = conn.SetWriteDeadline(time.Now().Add(defaultTimeout))
if _, err := conn.Write(pkt); err != nil {
return false, err
}
header, payload, err := readPacket(conn)
if err != nil {
return false, err
}
if (header >> 4) != PacketConnack {
return false, fmt.Errorf("expected CONNACK, got packet type %d", header>>4)
}
_, returnCode, err := parseConnack(payload)
if err != nil {
return false, err
}
return returnCode == ConnAccepted, nil
}
// Connect connects to an MQTT broker using optional credentials and settings.
// @example
// ```javascript
// const mqtt = require('nuclei/mqtt');
// const client = new mqtt.Client();
// const res = client.Connect('acme.com', 1883);
// log(res.Success);
// ```
func (c *Client) Connect(ctx context.Context, host string, port int, options *ConnectOptions) (*ConnectResult, error) {
c.mu.Lock()
defer c.mu.Unlock()
if c.conn != nil {
_ = c.conn.Close()
c.conn = nil
}
timeout := defaultTimeout
cleanSession := true
keepAlive := uint16(60)
clientID := "nuclei"
var username, password string
useTLS := false
insecureSkipVerify := true
if options != nil {
if options.Timeout > 0 {
timeout = time.Duration(options.Timeout) * time.Second
}
if options.KeepAlive > 0 {
if options.KeepAlive > 65535 {
keepAlive = 65535
} else {
keepAlive = uint16(options.KeepAlive)
}
}
if options.ClientID != "" {
clientID = options.ClientID
}
if options.CleanSession != nil {
cleanSession = *options.CleanSession
}
if options.InsecureSkipVerify != nil {
insecureSkipVerify = *options.InsecureSkipVerify
}
username = options.Username
password = options.Password
useTLS = options.TLS
}
c.timeout = timeout
conn, err := dial(ctx, host, port, useTLS, insecureSkipVerify, timeout)
if err != nil {
return &ConnectResult{
Success: false,
ReturnCode: -1,
Reason: err.Error(),
}, err
}
pkt, err := buildConnectPacket(clientID, username, password, cleanSession, keepAlive)
if err != nil {
_ = conn.Close()
return &ConnectResult{
Success: false,
ReturnCode: -1,
Reason: err.Error(),
}, err
}
_ = conn.SetWriteDeadline(time.Now().Add(timeout))
if _, err := conn.Write(pkt); err != nil {
_ = conn.Close()
return &ConnectResult{
Success: false,
ReturnCode: -1,
Reason: fmt.Sprintf("failed to send CONNECT: %v", err),
}, err
}
header, payload, err := readPacket(conn)
if err != nil {
_ = conn.Close()
return &ConnectResult{
Success: false,
ReturnCode: -1,
Reason: fmt.Sprintf("failed to read CONNACK: %v", err),
}, err
}
if (header >> 4) != PacketConnack {
_ = conn.Close()
return &ConnectResult{
Success: false,
ReturnCode: -1,
Reason: fmt.Sprintf("unexpected packet %d (expected CONNACK)", header>>4),
}, fmt.Errorf("unexpected packet %d", header>>4)
}
sessPresent, returnCode, err := parseConnack(payload)
if err != nil {
_ = conn.Close()
return &ConnectResult{
Success: false,
ReturnCode: int(returnCode),
Reason: err.Error(),
}, err
}
res := &ConnectResult{
Success: returnCode == ConnAccepted,
ReturnCode: int(returnCode),
Reason: ConnackReason(returnCode),
SessionPresent: sessPresent,
}
if res.Success {
c.conn = conn
} else {
_ = conn.Close()
}
return res, nil
}
// ConnectAuth connects to an MQTT broker with a username and password.
// @example
// ```javascript
// const mqtt = require('nuclei/mqtt');
// const client = new mqtt.Client();
// const res = client.ConnectAuth('acme.com', 1883, 'admin', 'admin');
// log(res.Success);
// ```
func (c *Client) ConnectAuth(ctx context.Context, host string, port int, username, password string) (*ConnectResult, error) {
clean := true
return c.Connect(ctx, host, port, &ConnectOptions{
Username: username,
Password: password,
CleanSession: &clean,
})
}
// Subscribe subscribes to a topic filter and collects received messages up to timeoutSec.
// @example
// ```javascript
// const mqtt = require('nuclei/mqtt');
// const client = new mqtt.Client();
// client.Connect('acme.com', 1883);
// const msgs = client.Subscribe('#', 3);
//
// for (const m of msgs) {
// log(m.Topic, m.Payload);
// }
//
// ```
func (c *Client) Subscribe(ctx context.Context, topic string, timeoutSec int) ([]*Message, error) {
c.mu.Lock()
defer c.mu.Unlock()
if c.conn == nil {
return nil, fmt.Errorf("not connected to MQTT broker")
}
c.packetID++
if c.packetID == 0 {
c.packetID = 1
}
pktID := c.packetID
subPkt, err := buildSubscribePacket(pktID, topic, 0)
if err != nil {
return nil, fmt.Errorf("failed to build SUBSCRIBE: %w", err)
}
_ = c.conn.SetWriteDeadline(time.Now().Add(c.timeout))
if _, err := c.conn.Write(subPkt); err != nil {
_ = c.conn.Close()
c.conn = nil
return nil, fmt.Errorf("failed to send SUBSCRIBE: %w", err)
}
waitDuration := 2 * time.Second
if timeoutSec > 0 {
if timeoutSec > 60 {
timeoutSec = 60
}
waitDuration = time.Duration(timeoutSec) * time.Second
}
if deadline, ok := ctx.Deadline(); ok {
if remaining := time.Until(deadline); remaining < waitDuration {
waitDuration = remaining
}
}
readDeadline := time.Now().Add(waitDuration)
_ = c.conn.SetDeadline(readDeadline)
defer func() {
if c.conn != nil {
_ = c.conn.SetDeadline(time.Time{})
}
}()
var messages []*Message
var totalBytes int
subackReceived := false
for {
select {
case <-ctx.Done():
_ = c.conn.Close()
c.conn = nil
return messages, ctx.Err()
default:
}
header, payload, err := readPacket(c.conn)
if err != nil {
if errors.Is(err, errBrokenStream) {
// The window closed in the middle of a packet. The stream cannot be
// read in sync again, so the connection is dropped.
_ = c.conn.Close()
c.conn = nil
if !subackReceived {
return nil, fmt.Errorf("failed waiting for SUBACK: %w", err)
}
break
}
var netErr net.Error
if errors.As(err, &netErr) && netErr.Timeout() {
// Window finished
break
}
if !subackReceived {
_ = c.conn.Close()
c.conn = nil
return nil, fmt.Errorf("failed waiting for SUBACK: %w", err)
}
break
}
packetType := header >> 4
switch packetType {
case PacketSuback:
subackPID, grantedQoS, err := parseSuback(payload)
if err != nil {
_ = c.conn.Close()
c.conn = nil
return nil, err
}
if subackPID != pktID {
continue // Ignore stale/mismatched SUBACK from an earlier subscription
}
for _, qos := range grantedQoS {
if qos == 0x80 {
return nil, fmt.Errorf("subscription rejected by broker for topic %q", topic)
}
}
subackReceived = true
case PacketPublish:
msg, err := parsePublish(header, payload)
if err == nil {
messages = append(messages, msg)
totalBytes += len(msg.Topic) + len(msg.Payload)
if len(messages) >= maxSubscribedMessages || totalBytes >= maxSubscribedBytes {
return messages, nil
}
}
}
if time.Now().After(readDeadline) {
break
}
}
if !subackReceived {
_ = c.conn.Close()
c.conn = nil
return nil, fmt.Errorf("subscription timed out waiting for SUBACK")
}
return messages, nil
}
// Publish publishes a message payload to a topic.
// @example
// ```javascript
// const mqtt = require('nuclei/mqtt');
// const client = new mqtt.Client();
// client.Connect('acme.com', 1883);
// const ok = client.Publish('nuclei/test', 'probe payload');
// log(ok);
// ```
func (c *Client) Publish(ctx context.Context, topic string, payload string) (bool, error) {
c.mu.Lock()
defer c.mu.Unlock()
if c.conn == nil {
return false, fmt.Errorf("not connected to MQTT broker")
}
select {
case <-ctx.Done():
return false, ctx.Err()
default:
}
pkt, err := buildPublishPacket(topic, []byte(payload), 0, false, 0)
if err != nil {
return false, fmt.Errorf("failed to build PUBLISH: %w", err)
}
_ = c.conn.SetWriteDeadline(time.Now().Add(5 * time.Second))
if _, err := c.conn.Write(pkt); err != nil {
_ = c.conn.Close()
c.conn = nil
return false, fmt.Errorf("failed to send PUBLISH: %w", err)
}
return true, nil
}
// Ping sends an MQTT PINGREQ and awaits PINGRESP.
// @example
// ```javascript
// const mqtt = require('nuclei/mqtt');
// const client = new mqtt.Client();
// client.Connect('acme.com', 1883);
// const alive = client.Ping();
// log(alive);
// ```
func (c *Client) Ping(ctx context.Context) (bool, error) {
c.mu.Lock()
defer c.mu.Unlock()
if c.conn == nil {
return false, fmt.Errorf("not connected to MQTT broker")
}
select {
case <-ctx.Done():
return false, ctx.Err()
default:
}
_ = c.conn.SetDeadline(time.Now().Add(5 * time.Second))
_ = c.conn.SetWriteDeadline(time.Now().Add(5 * time.Second))
if _, err := c.conn.Write(buildPingreqPacket()); err != nil {
_ = c.conn.Close()
c.conn = nil
return false, fmt.Errorf("failed to send PINGREQ: %w", err)
}
for {
header, _, err := readPacket(c.conn)
if err != nil {
_ = c.conn.Close()
c.conn = nil
return false, fmt.Errorf("failed to read PINGRESP: %w", err)
}
packetType := header >> 4
if packetType == PacketPingresp {
return true, nil
}
if packetType == PacketPublish {
// Broker delivered an incoming publish message while ping was in flight; continue waiting for PINGRESP
continue
}
return false, fmt.Errorf("unexpected ping response packet: %d", packetType)
}
}
// Close cleanly disconnects and closes the connection.
// @example
// ```javascript
// const mqtt = require('nuclei/mqtt');
// const client = new mqtt.Client();
// client.Connect('acme.com', 1883);
// client.Close();
// ```
func (c *Client) Close() error {
c.mu.Lock()
defer c.mu.Unlock()
if c.conn != nil {
_ = c.conn.SetWriteDeadline(time.Now().Add(3 * time.Second))
_, _ = c.conn.Write(buildDisconnectPacket())
err := c.conn.Close()
c.conn = nil
return err
}
return nil
}
+395
View File
@@ -0,0 +1,395 @@
package mqtt
import (
"bytes"
"context"
"encoding/binary"
"net"
"strings"
"testing"
"github.com/projectdiscovery/nuclei/v3/pkg/protocols/common/protocolstate"
"github.com/projectdiscovery/nuclei/v3/pkg/types"
"github.com/stretchr/testify/require"
)
func TestVarLengthEncoding(t *testing.T) {
testCases := []int{0, 1, 64, 127, 128, 16383, 16384, 2097151}
for _, tc := range testCases {
encoded, err := encodeVarLength(tc)
require.NoError(t, err)
decoded, err := decodeVarLength(bytes.NewReader(encoded))
require.NoError(t, err)
require.Equal(t, tc, decoded)
}
// Test negative or overflow
_, err := encodeVarLength(-1)
require.Error(t, err)
_, err = encodeVarLength(MaxRemainingLength + 1)
require.Error(t, err)
}
func TestOversizedStringsAndPackets(t *testing.T) {
hugeString := strings.Repeat("A", MaxStringLength+1)
_, err := encodeString(hugeString)
require.Error(t, err)
_, err = buildConnectPacket("id", hugeString, "pass", true, 60)
require.Error(t, err)
_, err = buildSubscribePacket(1, hugeString, 0)
require.Error(t, err)
_, err = buildPublishPacket(hugeString, []byte("ok"), 0, false, 0)
require.Error(t, err)
// Oversized packet reading
oversizedLen, err := encodeVarLength(MaxPacketSize + 100)
require.NoError(t, err)
buf := append([]byte{PacketPublish << 4}, oversizedLen...)
_, _, err = readPacket(bytes.NewReader(buf))
require.ErrorContains(t, err, "exceeds maximum allowed")
}
func TestPacketConnect(t *testing.T) {
pkt, err := buildConnectPacket("test-client", "user1", "pass1", true, 60)
require.NoError(t, err)
require.NotEmpty(t, pkt)
require.Equal(t, byte(PacketConnect<<4), pkt[0])
r := bytes.NewReader(pkt)
header, payload, err := readPacket(r)
require.NoError(t, err)
require.Equal(t, byte(PacketConnect<<4), header)
// Verify protocol name "MQTT"
protoLen := binary.BigEndian.Uint16(payload[0:2])
require.Equal(t, uint16(4), protoLen)
require.Equal(t, "MQTT", string(payload[2:6]))
require.Equal(t, byte(0x04), payload[6]) // Protocol level 4 (MQTT 3.1.1)
// CleanSession (0x02) | User (0x80) | Pass (0x40) = 0xC2
require.Equal(t, byte(0xC2), payload[7])
}
func TestPacketPublishAndParse(t *testing.T) {
pkt, err := buildPublishPacket("sensors/temperature", []byte("23.5C"), 0, true, 0)
require.NoError(t, err)
require.NotEmpty(t, pkt)
r := bytes.NewReader(pkt)
header, payload, err := readPacket(r)
require.NoError(t, err)
msg, err := parsePublish(header, payload)
require.NoError(t, err)
require.Equal(t, "sensors/temperature", msg.Topic)
require.Equal(t, "23.5C", msg.Payload)
require.Equal(t, 0, msg.QoS)
require.True(t, msg.Retain)
}
func TestPacketSubscribe(t *testing.T) {
pkt, err := buildSubscribePacket(42, "devices/#", 0)
require.NoError(t, err)
require.NotEmpty(t, pkt)
require.Equal(t, byte((PacketSubscribe<<4)|0x02), pkt[0])
r := bytes.NewReader(pkt)
header, payload, err := readPacket(r)
require.NoError(t, err)
require.Equal(t, byte((PacketSubscribe<<4)|0x02), header)
pid := binary.BigEndian.Uint16(payload[0:2])
require.Equal(t, uint16(42), pid)
subackPayload := []byte{0x00, 0x2A, 0x00} // PID 42, Success QoS 0
subackPid, granted, err := parseSuback(subackPayload)
require.NoError(t, err)
require.Equal(t, uint16(42), subackPid)
require.Equal(t, []byte{0x00}, granted)
}
func TestConnackReasons(t *testing.T) {
require.Equal(t, "connection accepted", ConnackReason(ConnAccepted))
require.Equal(t, "connection refused: bad user name or password", ConnackReason(ConnRefusedBadAuth))
require.Equal(t, "connection refused: not authorized", ConnackReason(ConnRefusedNotAuthorized))
require.Contains(t, ConnackReason(99), "connection refused: return code 99")
}
func TestRequestsHonourNetworkPolicy(t *testing.T) {
execID := "mqtt-deny"
require.NoError(t, protocolstate.Init(&types.Options{ExecutionId: execID, ExcludeTargets: []string{"203.0.113.50"}}))
t.Cleanup(func() { protocolstate.Close(execID) })
ctx := context.WithValue(context.Background(), "executionId", execID) //nolint:staticcheck
_, err := IsMQTT(ctx, "203.0.113.50", 1883)
require.Error(t, err)
client := &Client{}
_, err = client.Connect(ctx, "203.0.113.50", 1883, nil)
require.Error(t, err)
}
func startMockMQTTBroker(t *testing.T) (net.Listener, int) {
l, err := net.Listen("tcp", "127.0.0.1:0")
require.NoError(t, err)
port := l.Addr().(*net.TCPAddr).Port
go func() {
for {
conn, err := l.Accept()
if err != nil {
return
}
go handleMockClient(conn)
}
}()
return l, port
}
func handleMockClient(conn net.Conn) {
defer func() { _ = conn.Close() }()
for {
header, payload, err := readPacket(conn)
if err != nil {
return
}
packetType := header >> 4
switch packetType {
case PacketConnect:
// Check if username is present (flags byte at offset 7)
if len(payload) >= 8 {
flags := payload[7]
if (flags & 0x80) != 0 {
// Check username
offset := 10 // after proto(6) + level(1) + flags(1) + keepalive(2)
if len(payload) >= offset+2 {
clientLen := int(binary.BigEndian.Uint16(payload[offset : offset+2]))
offset += 2 + clientLen
if len(payload) >= offset+2 {
userLen := int(binary.BigEndian.Uint16(payload[offset : offset+2]))
user := string(payload[offset+2 : offset+2+userLen])
if user == "baduser" {
// Send CONNACK Refused Bad Auth
_, _ = conn.Write([]byte{0x20, 0x02, 0x00, ConnRefusedBadAuth})
return
}
}
}
}
}
// Send CONNACK Accepted
_, _ = conn.Write([]byte{0x20, 0x02, 0x00, ConnAccepted})
case PacketSubscribe:
// Send SUBACK
pid := binary.BigEndian.Uint16(payload[0:2])
suback := []byte{0x90, 0x03, byte(pid >> 8), byte(pid & 0xFF), 0x00}
_, _ = conn.Write(suback)
// Publish test message
pubPkt, err := buildPublishPacket("nuclei/alert", []byte("vuln-detected"), 0, false, 0)
if err == nil {
_, _ = conn.Write(pubPkt)
}
case PacketPublish:
// QoS 0, no ack needed
case PacketPingreq:
// Send PINGRESP
_, _ = conn.Write([]byte{PacketPingresp << 4, 0x00})
case PacketDisconnect:
return
}
}
}
func TestMockBrokerInteractions(t *testing.T) {
execID := "mqtt-mock-test"
require.NoError(t, protocolstate.Init(&types.Options{ExecutionId: execID}))
t.Cleanup(func() { protocolstate.Close(execID) })
ctx := context.WithValue(context.Background(), "executionId", execID) //nolint:staticcheck
l, port := startMockMQTTBroker(t)
defer func() { _ = l.Close() }()
// 1. Test IsMQTT
isMQTT, err := IsMQTT(ctx, "127.0.0.1", port)
require.NoError(t, err)
require.True(t, isMQTT)
// 2. Test IsAnonymousAllowed
anonAllowed, err := IsAnonymousAllowed(ctx, "127.0.0.1", port)
require.NoError(t, err)
require.True(t, anonAllowed)
// 3. Test Client Connect Auth Failure
badClient := &Client{}
badRes, _ := badClient.ConnectAuth(ctx, "127.0.0.1", port, "baduser", "wrongpass")
require.False(t, badRes.Success)
require.Equal(t, int(ConnRefusedBadAuth), badRes.ReturnCode)
// 4. Test Client Connect Success with default CleanSession preservation
client := &Client{}
res, err := client.Connect(ctx, "127.0.0.1", port, &ConnectOptions{KeepAlive: 70000})
require.NoError(t, err)
require.True(t, res.Success)
require.Equal(t, int(ConnAccepted), res.ReturnCode)
// 5. Test Ping
alive, err := client.Ping(ctx)
require.NoError(t, err)
require.True(t, alive)
// 6. Test Subscribe and Receive Message
msgs, err := client.Subscribe(ctx, "nuclei/alert", 1)
require.NoError(t, err)
require.NotEmpty(t, msgs)
require.Equal(t, "nuclei/alert", msgs[0].Topic)
require.Equal(t, "vuln-detected", msgs[0].Payload)
// 7. Test Publish
pubOk, err := client.Publish(ctx, "nuclei/probe", "test payload")
require.NoError(t, err)
require.True(t, pubOk)
// 8. Test Close and Double Close
require.NoError(t, client.Close())
require.NoError(t, client.Close())
}
func TestPasswordWithoutUsername(t *testing.T) {
// MQTT 3.1.1 (MQTT-3.1.2-22): If Password flag is set, User Name flag MUST be set.
pkt, err := buildConnectPacket("token-client", "", "my-secret-token", true, 60)
require.NoError(t, err)
r := bytes.NewReader(pkt)
_, payload, err := readPacket(r)
require.NoError(t, err)
// CleanSession (0x02) | User (0x80) | Pass (0x40) = 0xC2
require.Equal(t, byte(0xC2), payload[7])
// The client ID comes right after the keep alive at payload[8:10], and the
// username length follows it.
clientIDLen := int(binary.BigEndian.Uint16(payload[10:12]))
off := 12 + clientIDLen
// Username length should be 0 (empty string)
userLen := binary.BigEndian.Uint16(payload[off : off+2])
require.Equal(t, uint16(0), userLen)
off += 2
// Password follows the username
passLen := binary.BigEndian.Uint16(payload[off : off+2])
require.Equal(t, uint16(len("my-secret-token")), passLen)
off += 2
require.Equal(t, "my-secret-token", string(payload[off:off+int(passLen)]))
}
// A window that closes in the middle of a PUBLISH must drop the connection.
// Keeping it would let the next read start inside that packet.
func TestSubscribePartialPacketClosesConnection(t *testing.T) {
execID := "mqtt-partial-subscribe"
require.NoError(t, protocolstate.Init(&types.Options{ExecutionId: execID}))
t.Cleanup(func() { protocolstate.Close(execID) })
ctx := context.WithValue(context.Background(), "executionId", execID) //nolint:staticcheck
l, err := net.Listen("tcp", "127.0.0.1:0")
require.NoError(t, err)
defer func() { _ = l.Close() }()
port := l.Addr().(*net.TCPAddr).Port
go func() {
conn, err := l.Accept()
if err != nil {
return
}
defer func() { _ = conn.Close() }()
for {
header, payload, err := readPacket(conn)
if err != nil {
return
}
switch header >> 4 {
case PacketConnect:
_, _ = conn.Write([]byte{PacketConnack << 4, 0x02, 0x00, ConnAccepted})
case PacketSubscribe:
// SUBACK for the packet id, then a PUBLISH that promises 10 bytes
// and sends 5. The client's window closes in the middle of it.
_, _ = conn.Write([]byte{PacketSuback << 4, 0x03, payload[0], payload[1], 0x00})
_, _ = conn.Write([]byte{PacketPublish << 4, 0x0a, 0x00, 0x03, 'a', '/', 'b'})
}
}
}()
client := &Client{}
res, err := client.Connect(ctx, "127.0.0.1", port, nil)
require.NoError(t, err)
require.True(t, res.Success)
msgs, err := client.Subscribe(ctx, "a/b", 1)
require.NoError(t, err)
require.Empty(t, msgs)
require.Nil(t, client.conn, "a stream cut in the middle of a packet must not be reused")
}
func TestPingWithInFlightPublish(t *testing.T) {
execID := "mqtt-ping-inflight"
require.NoError(t, protocolstate.Init(&types.Options{ExecutionId: execID}))
t.Cleanup(func() { protocolstate.Close(execID) })
ctx := context.WithValue(context.Background(), "executionId", execID) //nolint:staticcheck
l, err := net.Listen("tcp", "127.0.0.1:0")
require.NoError(t, err)
defer func() { _ = l.Close() }()
port := l.Addr().(*net.TCPAddr).Port
go func() {
conn, err := l.Accept()
if err != nil {
return
}
defer func() { _ = conn.Close() }()
for {
header, _, err := readPacket(conn)
if err != nil {
return
}
packetType := header >> 4
switch packetType {
case PacketConnect:
_, _ = conn.Write([]byte{0x20, 0x02, 0x00, ConnAccepted})
case PacketPingreq:
// Send a PUBLISH packet first before PINGRESP
pubPkt, _ := buildPublishPacket("sensors/temp", []byte("25.0"), 0, false, 0)
_, _ = conn.Write(pubPkt)
// Then send PINGRESP
_, _ = conn.Write([]byte{PacketPingresp << 4, 0x00})
case PacketDisconnect:
return
}
}
}()
client := &Client{}
res, err := client.Connect(ctx, "127.0.0.1", port, nil)
require.NoError(t, err)
require.True(t, res.Success)
// Ping should skip the in-flight PUBLISH and successfully receive PINGRESP
alive, err := client.Ping(ctx)
require.NoError(t, err)
require.True(t, alive)
_ = client.Close()
}
+339
View File
@@ -0,0 +1,339 @@
package mqtt
import (
"encoding/binary"
"errors"
"fmt"
"io"
)
// Limits to prevent DoS and buffer overflows
const (
MaxStringLength = 65535
MaxRemainingLength = 268435455
MaxPacketSize = 10 * 1024 * 1024 // 10MB limit on incoming packets
)
// MQTT 3.1.1 Control Packet Types
const (
PacketConnect byte = 1
PacketConnack byte = 2
PacketPublish byte = 3
PacketPuback byte = 4
PacketPubrec byte = 5
PacketPubrel byte = 6
PacketPubcomp byte = 7
PacketSubscribe byte = 8
PacketSuback byte = 9
PacketUnsub byte = 10
PacketUnsuback byte = 11
PacketPingreq byte = 12
PacketPingresp byte = 13
PacketDisconnect byte = 14
)
// CONNACK Return Codes (MQTT 3.1.1 table 3.1)
const (
ConnAccepted byte = 0
ConnRefusedUnacceptableProtocol byte = 1
ConnRefusedIdentifierRejected byte = 2
ConnRefusedServerUnavailable byte = 3
ConnRefusedBadAuth byte = 4
ConnRefusedNotAuthorized byte = 5
)
// ConnackReason returns a human-readable description for CONNACK return codes.
func ConnackReason(code byte) string {
switch code {
case ConnAccepted:
return "connection accepted"
case ConnRefusedUnacceptableProtocol:
return "connection refused: unacceptable protocol version"
case ConnRefusedIdentifierRejected:
return "connection refused: identifier rejected"
case ConnRefusedServerUnavailable:
return "connection refused: server unavailable"
case ConnRefusedBadAuth:
return "connection refused: bad user name or password"
case ConnRefusedNotAuthorized:
return "connection refused: not authorized"
default:
return fmt.Sprintf("connection refused: return code %d", code)
}
}
// encodeVarLength encodes a length integer as an MQTT variable byte integer.
func encodeVarLength(length int) ([]byte, error) {
if length < 0 || length > MaxRemainingLength {
return nil, fmt.Errorf("remaining length %d exceeds MQTT max %d", length, MaxRemainingLength)
}
var out []byte
for {
encodedByte := byte(length % 128)
length /= 128
if length > 0 {
encodedByte |= 128
}
out = append(out, encodedByte)
if length == 0 {
break
}
}
return out, nil
}
// decodeVarLength decodes an MQTT variable byte integer from reader.
func decodeVarLength(r io.Reader) (int, error) {
multiplier := 1
value := 0
buf := make([]byte, 1)
for i := 0; i < 4; i++ {
if _, err := io.ReadFull(r, buf); err != nil {
return 0, err
}
encodedByte := buf[0]
value += int(encodedByte&127) * multiplier
if multiplier > 128*128*128 {
return 0, errors.New("malformed remaining length")
}
multiplier *= 128
if encodedByte&128 == 0 {
return value, nil
}
}
return 0, errors.New("malformed remaining length")
}
// encodeString encodes a UTF-8 string with a 2-byte big-endian length prefix.
func encodeString(s string) ([]byte, error) {
b := []byte(s)
if len(b) > MaxStringLength {
return nil, fmt.Errorf("string length %d exceeds MQTT max %d", len(b), MaxStringLength)
}
out := make([]byte, 2+len(b))
binary.BigEndian.PutUint16(out[0:2], uint16(len(b)))
copy(out[2:], b)
return out, nil
}
// buildConnectPacket constructs an MQTT 3.1.1 CONNECT packet.
func buildConnectPacket(clientID, username, password string, cleanSession bool, keepAlive uint16) ([]byte, error) {
var varHeader []byte
protoName, err := encodeString("MQTT")
if err != nil {
return nil, err
}
varHeader = append(varHeader, protoName...)
varHeader = append(varHeader, 0x04) // Protocol Level: 4 (MQTT 3.1.1)
// Connect Flags
var flags byte
if cleanSession {
flags |= 0x02
}
// Per MQTT 3.1.1 spec (MQTT-3.1.2-22): If the User Name Flag is 0, the Password Flag MUST be 0.
// If a password is provided, the User Name Flag MUST be set (encoding an empty username if omitted).
hasUsername := username != "" || password != ""
if hasUsername {
flags |= 0x80
}
if password != "" {
flags |= 0x40
}
varHeader = append(varHeader, flags)
// Keep Alive (2 bytes big endian)
ka := make([]byte, 2)
binary.BigEndian.PutUint16(ka, keepAlive)
varHeader = append(varHeader, ka...)
// Payload
var payload []byte
cidBytes, err := encodeString(clientID)
if err != nil {
return nil, fmt.Errorf("invalid clientID: %w", err)
}
payload = append(payload, cidBytes...)
if hasUsername {
userBytes, err := encodeString(username)
if err != nil {
return nil, fmt.Errorf("invalid username: %w", err)
}
payload = append(payload, userBytes...)
}
if password != "" {
passBytes, err := encodeString(password)
if err != nil {
return nil, fmt.Errorf("invalid password: %w", err)
}
payload = append(payload, passBytes...)
}
remainingLength := len(varHeader) + len(payload)
lenBytes, err := encodeVarLength(remainingLength)
if err != nil {
return nil, err
}
var pkt []byte
pkt = append(pkt, PacketConnect<<4)
pkt = append(pkt, lenBytes...)
pkt = append(pkt, varHeader...)
pkt = append(pkt, payload...)
return pkt, nil
}
// buildSubscribePacket constructs an MQTT 3.1.1 SUBSCRIBE packet for a single topic.
func buildSubscribePacket(packetID uint16, topic string, qos byte) ([]byte, error) {
var varHeader [2]byte
binary.BigEndian.PutUint16(varHeader[:], packetID)
topicBytes, err := encodeString(topic)
if err != nil {
return nil, fmt.Errorf("invalid topic: %w", err)
}
var payload []byte
payload = append(payload, topicBytes...)
payload = append(payload, qos&0x03)
remainingLength := len(varHeader) + len(payload)
lenBytes, err := encodeVarLength(remainingLength)
if err != nil {
return nil, err
}
var pkt []byte
// SUBSCRIBE fixed header flags must be 0010 (0x02)
pkt = append(pkt, (PacketSubscribe<<4)|0x02)
pkt = append(pkt, lenBytes...)
pkt = append(pkt, varHeader[:]...)
pkt = append(pkt, payload...)
return pkt, nil
}
// buildPublishPacket constructs an MQTT 3.1.1 PUBLISH packet.
func buildPublishPacket(topic string, payload []byte, qos byte, retain bool, packetID uint16) ([]byte, error) {
headerByte := (PacketPublish << 4)
if retain {
headerByte |= 0x01
}
headerByte |= (qos & 0x03) << 1
topicBytes, err := encodeString(topic)
if err != nil {
return nil, fmt.Errorf("invalid topic: %w", err)
}
var varHeader []byte
varHeader = append(varHeader, topicBytes...)
if qos > 0 {
pid := make([]byte, 2)
binary.BigEndian.PutUint16(pid, packetID)
varHeader = append(varHeader, pid...)
}
remainingLength := len(varHeader) + len(payload)
lenBytes, err := encodeVarLength(remainingLength)
if err != nil {
return nil, err
}
var pkt []byte
pkt = append(pkt, headerByte)
pkt = append(pkt, lenBytes...)
pkt = append(pkt, varHeader...)
pkt = append(pkt, payload...)
return pkt, nil
}
// buildPingreqPacket constructs an MQTT PINGREQ packet.
func buildPingreqPacket() []byte {
return []byte{PacketPingreq << 4, 0x00}
}
// buildDisconnectPacket constructs an MQTT DISCONNECT packet.
func buildDisconnectPacket() []byte {
return []byte{PacketDisconnect << 4, 0x00}
}
// errBrokenStream means a read failed after part of a packet was consumed, or
// a packet was refused, so the next read would start in the middle of a packet.
// The connection must not be used again.
var errBrokenStream = errors.New("MQTT stream is out of sync")
// readPacket reads a full MQTT packet from reader, rejecting packets over MaxPacketSize.
// Failures before the first byte is read are returned as they are. Failures after
// it wrap errBrokenStream.
func readPacket(r io.Reader) (headerByte byte, payload []byte, err error) {
var header [1]byte
if _, err := io.ReadFull(r, header[:]); err != nil {
return 0, nil, err
}
length, err := decodeVarLength(r)
if err != nil {
return 0, nil, fmt.Errorf("%w: %w", errBrokenStream, err)
}
if length > MaxPacketSize {
return 0, nil, fmt.Errorf("%w: MQTT packet length %d exceeds maximum allowed %d", errBrokenStream, length, MaxPacketSize)
}
buf := make([]byte, length)
if length > 0 {
if _, err := io.ReadFull(r, buf); err != nil {
return 0, nil, fmt.Errorf("%w: %w", errBrokenStream, err)
}
}
return header[0], buf, nil
}
// parseConnack decodes a CONNACK variable header.
func parseConnack(payload []byte) (sessionPresent bool, returnCode byte, err error) {
if len(payload) < 2 {
return false, 0, errors.New("short CONNACK response")
}
sessionPresent = (payload[0] & 0x01) != 0
returnCode = payload[1]
return sessionPresent, returnCode, nil
}
// parseSuback decodes a SUBACK packet payload.
func parseSuback(payload []byte) (packetID uint16, grantedQoS []byte, err error) {
if len(payload) < 3 {
return 0, nil, errors.New("short SUBACK response")
}
packetID = binary.BigEndian.Uint16(payload[0:2])
grantedQoS = payload[2:]
return packetID, grantedQoS, nil
}
// parsePublish decodes a PUBLISH packet.
func parsePublish(headerByte byte, body []byte) (*Message, error) {
qos := (headerByte >> 1) & 0x03
retain := (headerByte & 0x01) != 0
if len(body) < 2 {
return nil, errors.New("short PUBLISH packet")
}
topicLen := int(binary.BigEndian.Uint16(body[0:2]))
if len(body) < 2+topicLen {
return nil, errors.New("short PUBLISH topic")
}
topic := string(body[2 : 2+topicLen])
offset := 2 + topicLen
if qos > 0 {
if len(body) < offset+2 {
return nil, errors.New("short PUBLISH packet ID")
}
offset += 2
}
payload := string(body[offset:])
return &Message{
Topic: topic,
Payload: payload,
QoS: int(qos),
Retain: retain,
}, nil
}
@@ -2,7 +2,11 @@ package mysql
import (
"context"
"fmt"
"net"
"os"
"path/filepath"
"runtime"
"strings"
"sync/atomic"
"testing"
@@ -169,6 +173,84 @@ func TestFingerprintMySQLDeniedByNetworkPolicy(t *testing.T) {
require.Contains(t, err.Error(), "127.0.0.1")
}
func TestConnectWithDSNDeniedByNetworkPolicy(t *testing.T) {
t.Parallel()
executionID := "mysql-dsn-deny-" + strings.NewReplacer("/", "-", " ", "-").Replace(t.Name())
require.NoError(t, protocolstate.Init(&types.Options{
ExecutionId: executionID,
RestrictLocalNetworkAccess: true,
}))
t.Cleanup(func() { protocolstate.Close(executionID) })
ctx := context.WithValue(context.Background(), "executionId", executionID) //nolint:staticcheck
var accepted atomic.Int32
ln, err := net.Listen("tcp", "127.0.0.1:0")
require.NoError(t, err)
t.Cleanup(func() { _ = ln.Close() })
go func() {
for {
conn, err := ln.Accept()
if err != nil {
return
}
accepted.Add(1)
_ = conn.Close()
}
}()
port := ln.Addr().(*net.TCPAddr).Port
dsn := fmt.Sprintf("vuser:vpass@tcp(127.0.0.1:%d)/vdb", port)
ok, err := (&MySQLClient{}).ConnectWithDSN(ctx, dsn)
require.Error(t, err)
require.False(t, ok)
require.Contains(t, err.Error(), "127.0.0.1")
require.Zero(t, accepted.Load())
}
func TestConnectWithDSNUnixDeniedByNetworkPolicy(t *testing.T) {
t.Parallel()
executionID := "mysql-dsn-unix-deny-" + strings.NewReplacer("/", "-", " ", "-").Replace(t.Name())
require.NoError(t, protocolstate.Init(&types.Options{
ExecutionId: executionID,
RestrictLocalNetworkAccess: true,
}))
t.Cleanup(func() { protocolstate.Close(executionID) })
ctx := context.WithValue(context.Background(), "executionId", executionID) //nolint:staticcheck
// t.TempDir can exceed the unix socket path limit on macOS, so use a short dir.
dir, err := os.MkdirTemp("", "nuclei-mysql")
require.NoError(t, err)
t.Cleanup(func() { _ = os.RemoveAll(dir) })
sock := filepath.Join(dir, "lna.sock")
ln, err := net.Listen("unix", sock)
if err != nil && runtime.GOOS == "windows" {
t.Skipf("unix sockets not available: %v", err)
}
require.NoError(t, err)
t.Cleanup(func() { _ = ln.Close() })
var accepted atomic.Int32
go func() {
for {
conn, err := ln.Accept()
if err != nil {
return
}
accepted.Add(1)
_ = conn.Close()
}
}()
dsn := fmt.Sprintf("vuser:vpass@unix(%s)/vdb", sock)
ok, err := (&MySQLClient{}).ConnectWithDSN(ctx, dsn)
require.Error(t, err)
require.False(t, ok)
require.Contains(t, err.Error(), "127.0.0.1")
require.Zero(t, accepted.Load())
}
func TestFingerprintMySQLVersionNeverEmptyOnHandshakeFixtures(t *testing.T) {
t.Parallel()
+40 -7
View File
@@ -76,24 +76,57 @@ func BuildDSN(opts MySQLOptions) (string, error) {
return dsn.String(), nil
}
// sandboxDSN enforces the local file access sandbox on a MySQL DSN. The
// driver's allowAllFiles option lets a malicious server read any local file
// off the host via LOAD DATA LOCAL INFILE, so it is only honored when -lfa is
// enabled, mirroring the fs.ReadFile restriction.
func sandboxDSN(dsn string, lfaAllowed bool) (string, error) {
// sandboxDSN enforces local file and network policy on a MySQL DSN.
// allowAllFiles is only honored when -lfa is enabled. tcp, tcp4 and tcp6
// DSNs are rewritten to nucleitcp so the driver uses the fastdialer
// registered in protocolstate. Unix sockets are treated as loopback for -lna.
// Other networks are rejected.
func sandboxDSN(executionId, dsn string) (string, error) {
cfg, err := mysql.ParseDSN(dsn)
if err != nil {
return "", err
}
if cfg.AllowAllFiles && !lfaAllowed {
opts := &types.Options{ExecutionId: executionId}
if cfg.AllowAllFiles && !protocolstate.IsLfaAllowed(opts) {
cfg.AllowAllFiles = false
}
netName := cfg.Net
if netName == "" {
netName = "tcp"
}
switch netName {
case "unix":
// IP-based policy never sees a socket path; gate on loopback like ldapi.
const unixPolicyHost = "127.0.0.1"
if !protocolstate.IsHostAllowed(executionId, unixPolicyHost) {
return "", protocolstate.ErrHostDenied.Msgf(unixPolicyHost)
}
case "tcp", "tcp4", "tcp6", "nucleitcp":
// Only nucleitcp has a registered dialer; the others would use the
// driver's plain net.Dialer and skip the dial-time policy check.
cfg.Net = "nucleitcp"
host := dsnPolicyHost(cfg.Addr)
if !protocolstate.IsHostAllowed(executionId, host) {
return "", protocolstate.ErrHostDenied.Msgf(host)
}
default:
return "", fmt.Errorf("unsupported mysql network %q", netName)
}
return cfg.FormatDSN(), nil
}
func dsnPolicyHost(addr string) string {
host, _, err := net.SplitHostPort(addr)
if err == nil && host != "" {
return host
}
return addr
}
// openDB opens a sandboxed MySQL connection from dsn.
func openDB(executionId, dsn string) (*sql.DB, error) {
dsn, err := sandboxDSN(dsn, protocolstate.IsLfaAllowed(&types.Options{ExecutionId: executionId}))
dsn, err := sandboxDSN(executionId, dsn)
if err != nil {
return nil, err
}
+86 -4
View File
@@ -5,11 +5,14 @@ import (
"github.com/go-sql-driver/mysql"
"github.com/stretchr/testify/require"
"github.com/projectdiscovery/nuclei/v3/pkg/protocols/common/protocolstate"
"github.com/projectdiscovery/nuclei/v3/pkg/types"
)
func TestSandboxDSN(t *testing.T) {
t.Run("strips allowAllFiles when lfa disabled", func(t *testing.T) {
got, err := sandboxDSN("root:x@nucleitcp(127.0.0.1:3306)/?allowAllFiles=true", false)
got, err := sandboxDSN("", "root:x@nucleitcp(127.0.0.1:3306)/?allowAllFiles=true")
require.NoError(t, err)
cfg, err := mysql.ParseDSN(got)
@@ -18,7 +21,14 @@ func TestSandboxDSN(t *testing.T) {
})
t.Run("keeps allowAllFiles when lfa enabled", func(t *testing.T) {
got, err := sandboxDSN("root:x@nucleitcp(127.0.0.1:3306)/?allowAllFiles=true", true)
executionID := "mysql-lfa-" + t.Name()
require.NoError(t, protocolstate.Init(&types.Options{
ExecutionId: executionID,
AllowLocalFileAccess: true,
}))
t.Cleanup(func() { protocolstate.Close(executionID) })
got, err := sandboxDSN(executionID, "root:x@nucleitcp(127.0.0.1:3306)/?allowAllFiles=true")
require.NoError(t, err)
cfg, err := mysql.ParseDSN(got)
@@ -27,7 +37,7 @@ func TestSandboxDSN(t *testing.T) {
})
t.Run("leaves dsn without allowAllFiles untouched", func(t *testing.T) {
got, err := sandboxDSN("root:x@nucleitcp(127.0.0.1:3306)/", false)
got, err := sandboxDSN("", "root:x@nucleitcp(127.0.0.1:3306)/")
require.NoError(t, err)
cfg, err := mysql.ParseDSN(got)
@@ -35,8 +45,80 @@ func TestSandboxDSN(t *testing.T) {
require.False(t, cfg.AllowAllFiles)
})
t.Run("rewrites tcp to nucleitcp", func(t *testing.T) {
got, err := sandboxDSN("", "root:x@tcp(acme.com:3306)/")
require.NoError(t, err)
cfg, err := mysql.ParseDSN(got)
require.NoError(t, err)
require.Equal(t, "nucleitcp", cfg.Net)
require.Equal(t, "acme.com:3306", cfg.Addr)
})
t.Run("rewrites tcp4 and tcp6 to nucleitcp", func(t *testing.T) {
for _, netName := range []string{"tcp4", "tcp6"} {
got, err := sandboxDSN("", "root:x@"+netName+"(acme.com:3306)/")
require.NoError(t, err)
cfg, err := mysql.ParseDSN(got)
require.NoError(t, err)
require.Equal(t, "nucleitcp", cfg.Net)
require.Equal(t, "acme.com:3306", cfg.Addr)
}
})
t.Run("rejects unsupported network", func(t *testing.T) {
_, err := sandboxDSN("", "root:x@udp(acme.com:3306)/")
require.Error(t, err)
require.Contains(t, err.Error(), "udp")
})
t.Run("errors on invalid dsn", func(t *testing.T) {
_, err := sandboxDSN("::not-a-dsn::", false)
_, err := sandboxDSN("", "::not-a-dsn::")
require.Error(t, err)
})
t.Run("denies loopback tcp when lna enabled", func(t *testing.T) {
executionID := "mysql-lna-tcp-" + t.Name()
require.NoError(t, protocolstate.Init(&types.Options{
ExecutionId: executionID,
RestrictLocalNetworkAccess: true,
}))
t.Cleanup(func() { protocolstate.Close(executionID) })
_, err := sandboxDSN(executionID, "vuser:vpass@tcp(127.0.0.1:3306)/vdb")
require.Error(t, err)
require.Contains(t, err.Error(), "127.0.0.1")
})
t.Run("denies loopback variants when lna enabled", func(t *testing.T) {
executionID := "mysql-lna-variants-" + t.Name()
require.NoError(t, protocolstate.Init(&types.Options{
ExecutionId: executionID,
RestrictLocalNetworkAccess: true,
}))
t.Cleanup(func() { protocolstate.Close(executionID) })
for _, dsn := range []string{
"vuser:vpass@tcp4(127.0.0.1:3306)/vdb",
"vuser:vpass@tcp6([::1]:3306)/vdb",
"vuser:vpass@tcp([::ffff:127.0.0.1]:3306)/vdb",
} {
_, err := sandboxDSN(executionID, dsn)
require.Error(t, err, dsn)
}
})
t.Run("denies unix socket when lna enabled", func(t *testing.T) {
executionID := "mysql-lna-unix-" + t.Name()
require.NoError(t, protocolstate.Init(&types.Options{
ExecutionId: executionID,
RestrictLocalNetworkAccess: true,
}))
t.Cleanup(func() { protocolstate.Close(executionID) })
_, err := sandboxDSN(executionID, "vuser:vpass@unix(/tmp/nuclei-mysql-lna.sock)/vdb")
require.Error(t, err)
require.Contains(t, err.Error(), "127.0.0.1")
})
}
+31
View File
@@ -6,6 +6,7 @@ import (
"crypto/tls"
"encoding/base64"
"encoding/hex"
"errors"
"fmt"
"net"
"net/http"
@@ -20,6 +21,8 @@ import (
var (
defaultTimeout = time.Duration(5) * time.Second
errNoConnection = errors.New("no connection: open one with net.Open or net.OpenTLS")
)
// getTimeoutFromContext returns the configured TCP timeout or defaultTimeout.
@@ -271,12 +274,14 @@ type (
// const conn = net.Open('tcp', 'acme.com:80');
// ```
NetConn struct {
// conn is nil when no connection is open.
conn net.Conn
timeout time.Duration
}
)
// Close closes the connection.
// Close does nothing if no connection is open.
// @example
// ```javascript
// const net = require('nuclei/net');
@@ -284,7 +289,13 @@ type (
// conn.Close();
// ```
func (c *NetConn) Close() error {
if c.conn == nil {
return nil
}
err := c.conn.Close()
c.conn = nil
return err
}
@@ -321,6 +332,10 @@ func (c *NetConn) unsetDeadLine() {
// conn.SendArray(['hello', 'world']);
// ```
func (c *NetConn) SendArray(data []interface{}) error {
if c.conn == nil {
return errNoConnection
}
c.setDeadLine()
defer c.unsetDeadLine()
input := types.ToByteSlice(data)
@@ -342,6 +357,10 @@ func (c *NetConn) SendArray(data []interface{}) error {
// conn.SendHex('68656c6c6f');
// ```
func (c *NetConn) SendHex(data string) error {
if c.conn == nil {
return errNoConnection
}
c.setDeadLine()
defer c.unsetDeadLine()
bin, err := hex.DecodeString(data)
@@ -366,6 +385,10 @@ func (c *NetConn) SendHex(data string) error {
// conn.Send('hello');
// ```
func (c *NetConn) Send(data string) error {
if c.conn == nil {
return errNoConnection
}
c.setDeadLine()
defer c.unsetDeadLine()
bin := []byte(data)
@@ -389,6 +412,10 @@ func (c *NetConn) Send(data string) error {
// const data = conn.RecvFull(1024);
// ```
func (c *NetConn) RecvFull(N int) ([]byte, error) {
if c.conn == nil {
return []byte{}, errNoConnection
}
c.setDeadLine()
defer c.unsetDeadLine()
if N == 0 {
@@ -414,6 +441,10 @@ func (c *NetConn) RecvFull(N int) ([]byte, error) {
// log(`Received ${data.length} bytes from the server`)
// ```
func (c *NetConn) Recv(N int) ([]byte, error) {
if c.conn == nil {
return []byte{}, errNoConnection
}
c.setDeadLine()
defer c.unsetDeadLine()
if N == 0 {
+38
View File
@@ -406,3 +406,41 @@ func (f *fakeConn) RemoteAddr() net.Addr { return nil }
func (f *fakeConn) SetDeadline(time.Time) error { return nil }
func (f *fakeConn) SetReadDeadline(time.Time) error { return nil }
func (f *fakeConn) SetWriteDeadline(time.Time) error { return nil }
func TestNetConnWithoutConnection(t *testing.T) {
t.Parallel()
tests := map[string]func(t *testing.T) *NetConn{
"never opened": func(*testing.T) *NetConn { return new(NetConn) },
"already closed": func(t *testing.T) *NetConn {
t.Helper()
local, remote := net.Pipe()
t.Cleanup(func() { _ = remote.Close() })
conn := new(NetConn)
conn.conn = local
require.NoError(t, conn.Close())
return conn
},
}
for name, newConn := range tests {
t.Run(name, func(t *testing.T) {
t.Parallel()
conn := newConn(t)
require.NoError(t, conn.Close())
require.ErrorContains(t, conn.Send("x"), "no connection")
require.ErrorContains(t, conn.SendHex("78"), "no connection")
require.ErrorContains(t, conn.SendArray([]any{120}), "no connection")
_, err := conn.Recv(1)
require.ErrorContains(t, err, "no connection")
_, err = conn.RecvFull(1)
require.ErrorContains(t, err, "no connection")
})
}
}
+36 -37
View File
@@ -3,12 +3,12 @@ package postgres
import (
"context"
"fmt"
"net"
"net/url"
"os"
"path/filepath"
"strconv"
"strings"
"time"
"github.com/go-pg/pg/v10"
"github.com/praetorian-inc/fingerprintx/pkg/plugins"
postgres "github.com/praetorian-inc/fingerprintx/pkg/plugins/services/postgresql"
utils "github.com/projectdiscovery/nuclei/v3/pkg/js/utils"
@@ -18,7 +18,7 @@ import (
type (
// PGClient is a client for Postgres database.
// Internally client uses go-pg/pg driver.
// Internally client uses lib/pq driver.
// @example
// ```javascript
// const postgres = require('nuclei/postgres');
@@ -124,9 +124,7 @@ func executeQuery(ctx context.Context, executionId string, host string, port int
return nil, protocolstate.ErrHostDenied.Msgf(host)
}
target := net.JoinHostPort(host, fmt.Sprintf("%d", port))
connStr := buildPostgresConnURL(username, password, target, dbName, executionId)
connStr := buildPostgresDSN(host, port, username, password, dbName)
db, err := pgwrap.OpenDB(ctx, executionId, connStr)
if err != nil {
return nil, err
@@ -146,17 +144,30 @@ func executeQuery(ctx context.Context, executionId string, host string, port int
return resp, nil
}
func buildPostgresConnURL(username, password, target, dbName, executionId string) string {
values := url.Values{}
values.Set("sslmode", "disable")
values.Set("executionId", executionId)
return fmt.Sprintf("postgres://%s@%s/%s?%s",
url.UserPassword(username, password).String(),
target,
url.PathEscape(dbName),
values.Encode(),
)
// buildPostgresDSN returns a lib/pq key=value connection string. user, password, dbname, service and passfile
// are always set, also when empty: lib/pq fills unset values from the scanner's environment (PGUSER, PGPASSWORD,
// PGDATABASE), a service file (PGSERVICE) and ~/.pgpass (PGPASSFILE), which sends the operator's credentials to the target.
func buildPostgresDSN(host string, port int, username, password, dbName string) string {
quote := strings.NewReplacer(`\`, `\\`, `'`, `\'`)
params := []struct{ key, value string }{
{"host", host},
{"port", strconv.Itoa(port)},
{"user", username},
{"password", password},
{"dbname", dbName},
{"sslmode", "disable"},
{"connect_timeout", "10"},
{"service", ""},
{"options", ""},
{"application_name", ""},
// a path below a file never exists, so no password file is read
{"passfile", filepath.Join(os.DevNull, "pgpass")},
}
kvs := make([]string, 0, len(params))
for _, param := range params {
kvs = append(kvs, param.key+"='"+quote.Replace(param.value)+"'")
}
return strings.Join(kvs, " ")
}
// ConnectWithDB connects to Postgres database using given credentials and database name.
@@ -194,32 +205,20 @@ func connect(ctx context.Context, executionId string, host string, port int, use
return false, protocolstate.ErrHostDenied.Msgf(host)
}
target := net.JoinHostPort(host, fmt.Sprintf("%d", port))
execCtx, cancel := context.WithCancel(ctx)
execCtx, cancel := context.WithTimeout(ctx, 10*time.Second)
defer cancel()
dialer := protocolstate.GetDialersWithId(executionId)
if dialer == nil {
return false, fmt.Errorf("dialers not initialized for %s", executionId)
// lib/pq with every credential set, as go-pg replaces an empty user, password or database
// with PGUSER, PGPASSWORD or PGDATABASE from the scanner's environment
db, err := pgwrap.OpenDB(execCtx, executionId, buildPostgresDSN(host, port, username, password, dbName))
if err != nil {
return false, err
}
db := pg.Connect(&pg.Options{
Addr: target,
User: username,
Password: password,
Database: dbName,
Dialer: func(dialCtx context.Context, network, addr string) (net.Conn, error) {
return dialer.Fastdialer.Dial(dialCtx, network, addr)
},
IdleCheckFrequency: -1,
}).WithTimeout(10 * time.Second)
defer func() {
_ = db.Close()
}()
_, err := db.ExecContext(execCtx, "select 1")
_, err = db.ExecContext(execCtx, "select 1")
if err != nil {
switch true {
case strings.Contains(err.Error(), "connect: connection refused"):
+151 -60
View File
@@ -1,74 +1,165 @@
package postgres
import (
"net/url"
"context"
"encoding/binary"
"io"
"net"
"os"
"path/filepath"
"strings"
"testing"
"github.com/lib/pq"
"github.com/projectdiscovery/nuclei/v3/pkg/protocols/common/protocolstate"
"github.com/projectdiscovery/nuclei/v3/pkg/types"
"github.com/stretchr/testify/require"
)
func TestBuildPostgresConnectionURLDoesNotAllowDBNameQueryInjection(t *testing.T) {
dbName := "testdb?sslrootcert=/etc/passwd&sslmode=verify-ca&junk="
connStr := buildPostgresConnURL("postgres", "x", "127.0.0.1:5432", dbName, "exec-1")
func TestBuildPostgresDSNEscapesDBName(t *testing.T) {
dbName := "testdb' sslrootcert='/etc/passwd' sslmode='verify-ca"
cfg, err := pq.NewConfig(buildPostgresDSN("127.0.0.1", 5432, "postgres", "x", dbName))
require.NoError(t, err)
u, err := url.Parse(connStr)
if err != nil {
t.Fatalf("parse connection URL: %v", err)
}
if got := strings.TrimPrefix(u.Path, "/"); got != dbName {
t.Fatalf("database name = %q, want %q", got, dbName)
}
values := u.Query()
if got := values.Get("sslmode"); got != "disable" {
t.Fatalf("sslmode = %q, want disable", got)
}
if got := values.Get("executionId"); got != "exec-1" {
t.Fatalf("executionId = %q, want exec-1", got)
}
deniedParams := []string{"sslrootcert", "sslcert", "sslkey", "service", "junk"}
for _, denied := range deniedParams {
if got := values.Get(denied); got != "" {
t.Fatalf("%s was injected with value %q", denied, got)
}
}
pqDSN, err := pq.ParseURL(connStr) //nolint:staticcheck // validates lib/pq URL parsing of the generated DSN.
if err != nil {
t.Fatalf("parse connection URL as lib/pq DSN: %v", err)
}
if !strings.Contains(pqDSN, "dbname='"+dbName+"'") {
t.Fatalf("lib/pq DSN = %q, want dbName preserved as dbname", pqDSN)
}
for _, denied := range deniedParams {
if strings.Contains(pqDSN, " "+denied+"=") {
t.Fatalf("%s was injected into lib/pq DSN %q", denied, pqDSN)
}
}
require.Equal(t, dbName, cfg.Database)
require.Equal(t, pq.SSLModeDisable, cfg.SSLMode)
require.Empty(t, cfg.SSLRootCert)
require.Empty(t, cfg.Runtime)
}
func TestBuildPostgresConnectionURLEscapesCredentials(t *testing.T) {
username := "user:[email protected]"
password := "pa:ss@word?x"
connStr := buildPostgresConnURL(username, password, "127.0.0.1:5432", "postgres", "exec-1")
func TestBuildPostgresDSNEscapesCredentials(t *testing.T) {
username := `user:[email protected]' host='other`
password := `pa:ss@word?x\' sslmode='require`
cfg, err := pq.NewConfig(buildPostgresDSN("127.0.0.1", 5432, username, password, "postgres"))
require.NoError(t, err)
require.Equal(t, username, cfg.User)
require.Equal(t, password, cfg.Password)
require.Equal(t, "127.0.0.1", cfg.Host)
require.Equal(t, uint16(5432), cfg.Port)
require.Equal(t, "postgres", cfg.Database)
require.Equal(t, pq.SSLModeDisable, cfg.SSLMode)
}
// setOperatorPostgresEnv sets every source lib/pq and go-pg read credentials from
func setOperatorPostgresEnv(t *testing.T) {
home := t.TempDir()
pgpass := filepath.Join(home, ".pgpass")
require.NoError(t, os.WriteFile(pgpass, []byte("*:*:*:*:OPERATOR-PGPASS-SECRET\n"), 0600))
serviceFile := filepath.Join(home, ".pg_service.conf")
require.NoError(t, os.WriteFile(serviceFile, []byte("[operator]\npassword=OPERATOR-SERVICE-SECRET\n"), 0600))
t.Setenv("HOME", home)
t.Setenv("PGPASSFILE", pgpass)
t.Setenv("PGSERVICEFILE", serviceFile)
t.Setenv("PGSERVICE", "operator")
t.Setenv("PGPASSWORD", "OPERATOR-ENV-PGPASSWORD")
t.Setenv("PGUSER", "operator-user")
t.Setenv("PGDATABASE", "operator-db")
t.Setenv("PGOPTIONS", "-c search_path=operator")
t.Setenv("PGAPPNAME", "operator-app")
}
func TestBuildPostgresDSNIgnoresOperatorEnvironment(t *testing.T) {
setOperatorPostgresEnv(t)
cfg, err := pq.NewConfig(buildPostgresDSN("127.0.0.1", 5432, "", "", ""))
require.NoError(t, err)
require.Empty(t, cfg.User)
require.Empty(t, cfg.Password)
require.Empty(t, cfg.Database)
require.Empty(t, cfg.Service)
require.Empty(t, cfg.Options)
require.Empty(t, cfg.ApplicationName)
}
type startup struct {
user, database, password string
}
// fakePostgres asks every client for a cleartext password, records it and rejects the login
func fakePostgres(t *testing.T) (int, <-chan startup) {
listener, err := net.Listen("tcp", "127.0.0.1:0")
require.NoError(t, err)
t.Cleanup(func() { _ = listener.Close() })
logins := make(chan startup, 16)
go func() {
for {
conn, err := listener.Accept()
if err != nil {
return
}
go func(conn net.Conn) {
defer func() { _ = conn.Close() }()
readMessage := func(typed bool) []byte {
header := make([]byte, 4)
if typed {
header = make([]byte, 5)
}
if _, err := io.ReadFull(conn, header); err != nil {
return nil
}
body := make([]byte, binary.BigEndian.Uint32(header[len(header)-4:])-4)
if _, err := io.ReadFull(conn, body); err != nil {
return nil
}
return body
}
params := readMessage(false)
if len(params) < 4 {
return
}
fields := strings.Split(string(params[4:]), "\x00")
var login startup
for i := 0; i+1 < len(fields); i += 2 {
switch fields[i] {
case "user":
login.user = fields[i+1]
case "database":
login.database = fields[i+1]
}
}
// AuthenticationCleartextPassword
if _, err := conn.Write([]byte{'R', 0, 0, 0, 8, 0, 0, 0, 3}); err != nil {
return
}
password := readMessage(true)
login.password = strings.TrimSuffix(string(password), "\x00")
logins <- login
errorBody := "SFATAL\x00C28P01\x00Mpassword authentication failed\x00\x00"
header := []byte{'E', 0, 0, 0, 0}
binary.BigEndian.PutUint32(header[1:], uint32(len(errorBody)+4))
_, _ = conn.Write(append(header, errorBody...))
}(conn)
}
}()
return listener.Addr().(*net.TCPAddr).Port, logins
}
func TestPostgresEmptyCredentialsAreNotFilledFromEnvironment(t *testing.T) {
setOperatorPostgresEnv(t)
executionID := t.Name()
require.NoError(t, protocolstate.Init(&types.Options{ExecutionId: executionID}))
t.Cleanup(func() { protocolstate.Close(executionID) })
port, logins := fakePostgres(t)
ctx := context.Background()
_, err := executeQuery(ctx, executionID, "127.0.0.1", port, "postgres", "", "postgres", "select 1")
require.Error(t, err)
require.Equal(t, startup{user: "postgres", database: "postgres"}, <-logins)
connected, err := connect(ctx, executionID, "127.0.0.1", port, "postgres", "", "postgres")
require.NoError(t, err)
require.False(t, connected)
require.Equal(t, startup{user: "postgres", database: "postgres"}, <-logins)
_, _ = connect(ctx, executionID, "127.0.0.1", port, "", "", "")
require.Equal(t, startup{}, <-logins)
_, _ = executeQuery(ctx, executionID, "127.0.0.1", port, "admin", "secret", "db", "select 1")
require.Equal(t, startup{user: "admin", database: "db", password: "secret"}, <-logins)
u, err := url.Parse(connStr)
if err != nil {
t.Fatalf("parse connection URL: %v", err)
}
if got := u.User.Username(); got != username {
t.Fatalf("username = %q, want %q", got, username)
}
if got, _ := u.User.Password(); got != password {
t.Fatalf("password = %q, want %q", got, password)
}
if got := u.Host; got != "127.0.0.1:5432" {
t.Fatalf("host = %q, want 127.0.0.1:5432", got)
}
if got := strings.TrimPrefix(u.Path, "/"); got != "postgres" {
t.Fatalf("database name = %q, want postgres", got)
}
}
-1
View File
@@ -3,7 +3,6 @@ package redis
import (
"context"
"errors"
"fmt"
+113 -117
View File
@@ -3,13 +3,12 @@ package redis
import (
"bytes"
"context"
"errors"
"fmt"
"io"
"net"
"strconv"
"time"
"github.com/projectdiscovery/gologger"
"github.com/projectdiscovery/nuclei/v3/pkg/protocols/common/protocolstate"
"github.com/redis/go-redis/v9"
@@ -269,9 +268,10 @@ var errReadLimit = fmt.Errorf("redis: reply exceeds %d bytes", maxReadBytes)
// maxReadBytes.
func newClient(addr, password string) *redis.Client {
opts := &redis.Options{
Addr: addr,
Password: password,
DB: 0,
Addr: addr,
Password: password,
DB: 0,
DialerRetries: dialAttempts,
}
dial := redis.NewDialer(opts)
opts.Dialer = func(ctx context.Context, network, addr string) (net.Conn, error) {
@@ -285,138 +285,134 @@ func newClient(addr, password string) *redis.Client {
return redis.NewClient(opts)
}
// limitedConn fails a read once remaining is spent, and withholds a RESP length line whose
// declared size is over the cap so go-redis never allocates it.
// maxHeaderLen fits a type byte, any int64, and "\r". Redis never sends a longer length header.
const maxHeaderLen = 32
// limitedConn fails a read once remaining is spent, and ends a read before the "\n" of a RESP
// length header whose declared size is over the cap, so go-redis never gets the line to allocate
// from.
//
// It follows the reply's framing as the bytes pass: a bulk payload is skipped by its declared
// length, so only header lines are checked. A payload line such as "*abc" is data.
type limitedConn struct {
net.Conn
remaining int64
pending []byte
partial []byte
err error
scratch [32 << 10]byte
// payload is how many bytes of the current bulk payload, "\r\n" included, are still to come.
payload int64
// header is the start of the current header line, without its "\n".
header []byte
overlong bool
err error
}
func (c *limitedConn) Read(b []byte) (int, error) {
if len(b) == 0 {
if c.err != nil {
return 0, c.err
}
if c.remaining <= 0 {
return 0, errReadLimit
}
if int64(len(b)) > c.remaining {
b = b[:c.remaining]
}
n, err := c.Conn.Read(b)
c.remaining -= int64(n)
// Errors from the connection, deadlines included, are returned as they come and not kept: a
// read deadline belongs to one call. go-redis peeks for push notifications with a few
// microseconds of deadline, and keeping that timeout would fail the command that follows.
kept, scanErr := c.scan(b[:n])
if scanErr == nil {
return n, err
}
c.err = scanErr
if kept == 0 {
return 0, scanErr
}
return kept, nil
}
// scan follows the RESP framing of p and returns how many of its bytes may be returned.
func (c *limitedConn) scan(p []byte) (int, error) {
for i := 0; i < len(p); {
if c.payload > 0 {
skip := min(c.payload, int64(len(p)-i))
c.payload -= skip
i += int(skip)
continue
}
end := bytes.IndexByte(p[i:], '\n')
if end < 0 {
c.keepHeader(p[i:])
return len(p), nil
}
c.keepHeader(p[i : i+end])
payload, err := declaredPayload(c.header, c.overlong)
c.header, c.overlong = c.header[:0], false
if err != nil {
return i + end, err
}
c.payload = payload
i += end + 1
}
return len(p), nil
}
func (c *limitedConn) keepHeader(p []byte) {
if room := maxHeaderLen - len(c.header); len(p) > room {
p = p[:room]
c.overlong = true
}
c.header = append(c.header, p...)
}
// declaredPayload returns how many payload bytes, "\r\n" included, follow a RESP header line
// given without its "\n". It returns errReadLimit when go-redis would allocate more than the cap
// from the line: the declared byte length for bulk strings, one interface per declared element
// for arrays, sets, maps, attributes, and pushes.
func declaredPayload(line []byte, overlong bool) (int64, error) {
if len(line) == 0 {
return 0, nil
}
if len(c.pending) == 0 {
if err := c.fill(); err != nil {
return 0, err
}
}
n := copy(b, c.pending)
c.pending = c.pending[n:]
return n, nil
}
func (c *limitedConn) fill() error {
if len(c.pending) > 0 {
return nil
}
if c.err != nil {
return c.err
}
for len(c.pending) == 0 {
if c.remaining <= 0 {
c.err = errReadLimit
return c.err
}
want := int64(len(c.scratch))
if want > c.remaining {
want = c.remaining
}
n, err := c.Conn.Read(c.scratch[:want])
if n > 0 {
c.remaining -= int64(n)
if aerr := c.accept(c.scratch[:n]); aerr != nil {
c.err = aerr
if len(c.pending) > 0 {
return nil
}
return aerr
}
}
if err != nil {
// A read deadline belongs to one call. go-redis peeks for push notifications
// with a few microseconds of deadline, and on Windows that peek always runs.
// Keeping the timeout fails the command that follows.
if timeoutErr(err) {
if len(c.pending) > 0 {
return nil
}
return err
}
c.err = err
if len(c.pending) > 0 {
return nil
}
return err
}
if n == 0 {
c.err = io.ErrUnexpectedEOF
return c.err
}
}
return nil
}
func (c *limitedConn) accept(p []byte) error {
c.partial = append(c.partial, p...)
for {
i := bytes.IndexByte(c.partial, '\n')
if i < 0 {
if int64(len(c.partial)) > maxReadBytes {
c.partial = nil
return errReadLimit
}
return nil
}
line := c.partial[:i+1]
if lengthTooLarge(line) {
c.partial = nil
return errReadLimit
}
copied := make([]byte, len(line))
copy(copied, line)
c.pending = append(c.pending, copied...)
rest := make([]byte, len(c.partial)-(i+1))
copy(rest, c.partial[i+1:])
c.partial = rest
}
}
// lengthTooLarge reports whether line is a RESP length header go-redis would allocate from.
// Bulk strings allocate the declared byte length. Arrays, sets, maps, attributes, and pushes
// allocate one interface per declared element.
func lengthTooLarge(line []byte) bool {
if len(line) < 3 {
return false
}
var limit int64
bulk := false
switch line[0] {
case '$', '=', '!':
limit = maxReadBytes
limit, bulk = maxReadBytes, true
case '*', '~', '>', '%', '|':
limit = maxAggregate
default:
return false
return 0, nil
}
body := line[1 : len(line)-1]
if len(body) > 0 && body[len(body)-1] == '\r' {
body = body[:len(body)-1]
if overlong {
return 0, errReadLimit
}
body := bytes.TrimSuffix(line[1:], []byte("\r"))
if len(body) == 0 || body[0] == '-' {
return false
return 0, nil
}
n, err := strconv.ParseInt(string(body), 10, 64)
if err != nil {
return true
if err != nil || n > limit {
return 0, errReadLimit
}
return n > limit
if !bulk {
return 0, nil
}
return n + 2, nil
}
func timeoutErr(err error) bool {
var netErr net.Error
return errors.As(err, &netErr) && netErr.Timeout()
// dialAttempts replaces go-redis's default of 5. The extra attempts run in a goroutine that
// ignores the caller's context, so they kept dialing a filtered port for ~25s after the
// template had given up on it.
const dialAttempts = 1
func init() {
redis.SetLogger(warningLogger{})
}
// warningLogger hands go-redis's logs to gologger as warnings. Its default logger writes them
// straight to stderr, where a failed dial against the scanned host reads as an error.
type warningLogger struct{}
func (warningLogger) Printf(_ context.Context, format string, v ...interface{}) {
gologger.Warning().Msgf(format, v...)
}
+81
View File
@@ -0,0 +1,81 @@
package redis
import (
"context"
"net"
"runtime"
"strings"
"sync"
"testing"
"time"
"github.com/projectdiscovery/gologger"
"github.com/projectdiscovery/gologger/levels"
"github.com/projectdiscovery/gologger/writer"
"github.com/stretchr/testify/require"
)
// go-redis retries a failed dial 4 times, 100ms apart, on each of a command's 4 attempts: a closed
// port took at least 1.6s to fail.
func TestGetServerInfoFailsFastOnAClosedPort(t *testing.T) {
if runtime.GOOS == "windows" {
t.Skip("windows takes about a second to report a refused localhost dial")
}
listener, err := net.Listen("tcp", "127.0.0.1:0")
require.NoError(t, err)
addr, ok := listener.Addr().(*net.TCPAddr)
require.True(t, ok)
require.NoError(t, listener.Close())
start := time.Now()
_, err = getServerInfo(context.Background(), "redis-closed-port", "127.0.0.1", addr.Port)
require.Error(t, err)
require.Less(t, time.Since(start), time.Second)
}
// go-redis's default logger wrote each failed dial straight to stderr, past nuclei's logger and
// its level.
func TestGoRedisLogsAreGologgerWarnings(t *testing.T) {
logs := &recordingWriter{}
gologger.DefaultLogger.SetWriter(logs)
gologger.DefaultLogger.SetMaxLevel(levels.LevelWarning)
t.Cleanup(func() {
// gologger's own defaults: it has no getter to save the previous ones.
gologger.DefaultLogger.SetWriter(writer.NewCLI())
gologger.DefaultLogger.SetMaxLevel(levels.LevelInfo)
})
listener, err := net.Listen("tcp", "127.0.0.1:0")
require.NoError(t, err)
addr, ok := listener.Addr().(*net.TCPAddr)
require.True(t, ok)
require.NoError(t, listener.Close())
_, err = getServerInfo(context.Background(), "redis-log-level", "127.0.0.1", addr.Port)
require.Error(t, err)
warnings := logs.at(levels.LevelWarning)
require.NotEmpty(t, warnings)
require.Contains(t, warnings[0], "failed to dial")
}
type recordingWriter struct {
mu sync.Mutex
lines map[levels.Level][]string
}
func (w *recordingWriter) Write(data []byte, level levels.Level) {
w.mu.Lock()
defer w.mu.Unlock()
if w.lines == nil {
w.lines = make(map[levels.Level][]string)
}
w.lines[level] = append(w.lines[level], strings.TrimSpace(string(data)))
}
func (w *recordingWriter) at(level levels.Level) []string {
w.mu.Lock()
defer w.mu.Unlock()
return w.lines[level]
}
Loaded 100 of 209 files, more files were not shown because too many files have changed in this diff. Show more