Compare commits

..
Author SHA1 Message Date
Mzack9999 fe24f8638b review fixes 2026-11-04 05:09:15 +01:00
Mzack9999 8b7cbf44e6 all trackers 2026-11-04 03:26:07 +01:00
Mzack9999 59ae4cac04 tls dialer 2026-11-04 03:00:58 +01:00
Mzack9999 64965ba776 tls verify 2026-11-04 00:54:48 +01:00
Mzack9999 15271926ad tracker client 2026-11-03 23:28:48 +01:00
Mzack9999 46fbc92a76 Merge pull request #8023 from projectdiscovery/fix/dns-resolver-client
Give each scan its own DNS resolvers
2026-11-03 21:42:45 +01:00
Mzack9999 36d0adba18 Merge pull request #8024 from projectdiscovery/fix/headless-exclude-lock
Read headless excludes under the dialer lock
2026-11-03 20:47:14 +01:00
Mzack9999 049e6b629e Merge pull request #8018 from projectdiscovery/feat/jev-llm-provider
use jev for llm matchers when -llm-provider jev
2026-11-03 19:21:59 +01:00
Dogan Can Bakir 2c6b223bb4 Merge pull request #7983 from projectdiscovery/headless-host-canonical
Deny IPv4-mapped forms of denied hosts in the dialer
2026-11-03 20:45:05 +03:00
Mzack9999 0827b2779b preserve proxy 2026-11-03 18:34:18 +01:00
Mzack9999 0971fb83e2 skip extractors 2026-11-03 18:07:38 +01:00
Mzack9999 62dae92b3d refresh resolvers 2026-11-03 17:30:04 +01:00
Mzack9999 e44df6e947 lock excludes 2026-11-03 17:30:04 +01:00
Mzack9999 25f50f1036 Merge pull request #8012 from TassioSales/fix/clean-flow-js-runtime
fix(flow): prevent global state leakage in shared JS runtime pool (#7986)
2026-11-03 17:13:36 +01:00
Mzack9999 f9446e9290 Merge pull request #8016 from projectdiscovery/dwisiswant0/fix/headless/fail-requests-whose-url-net-url-cannot-parse
fix(headless): fail requests whose URL net/url cannot parse
2026-11-03 17:12:37 +01:00
Mzack9999 faaeaf30b1 mapped deny 2026-11-03 17:07:35 +01:00
Mzack9999 b7fe335285 merge dev 2026-11-03 17:02:03 +01:00
Mzack9999 7481b20d78 Merge pull request #8014 from TassioSales/fix/headless-ipv4-mapped-ipv6-exclude-hosts-7982
fix(headless): normalize IPv4-mapped IPv6 host forms for -exclude-hosts and -lna (#7982)
2026-11-03 13:39:20 +01:00
Mzack9999 48600a24de use canonical 2026-11-03 12:51:48 +01:00
Mzack9999 ec4036147e Merge pull request #8013 from 0xABCD01/fix/govaluate-concat-regression-test
test(expressions): add regression case for govaluate concat bug (follow-up to #7998)
2026-11-03 12:51:11 +01:00
Mzack9999 9b04153a3b merge dev 2026-11-03 12:45:38 +01:00
Dogan Can Bakir 23f3407af7 bump utils for the jev provider 2026-11-03 12:50:46 +03:00
Dwi Siswanto be40690a29 fix(headless): fail requests whose URL net/url cannot parse
With a request modification action (setheader, addheader, deleteheader,
setmethod or setbody), routingRuleHandler replays every browser request
through nuclei's Go HTTP client. go-rod builds that request with
url.Parse and drops the error, so a URL that Chrome accepts but net/url
rejects, such as a path ending in "100%.png", arrived with a nil URL.
The cookie jar sync then panicked, or httputil.DumpRequestOut did when
cookies were disabled, and the panic killed the whole scan.

Check the URL once on entry and fail such requests in the browser. The
Go client cannot send them, and continuing them in the browser would
drop the template's modification rules and, with -cdp-endpoint, skip
the dial-time address checks of nuclei's network policy.

Fixes: #8015
Reported-by: @l4rm4nd
Signed-off-by: Dwi Siswanto <[email protected]>
2026-11-03 16:13:37 +07:00
Dogan Can Bakir 64f94f3e66 use jev for llm matchers when -llm-provider jev 2026-11-03 12:04:47 +03:00
TassioSales b2b54a7ac4 fix(headless): preserve port in CanonicalizeHost and stabilize proxy test
Signed-off-by: TassioSales <[email protected]>
2026-11-03 01:28:54 -03:00
TassioSales 22aedebe5d fix(headless): normalize IPv4-mapped IPv6 host forms for -exclude-hosts and -lna (#7982)
Normalize hostnames and IPv4-mapped IPv6 addresses (e.g. ::ffff:127.0.0.1,
::ffff:7f00:1, and bracketed variants) before validating against NetworkPolicy
and ExcludeTargets in headless execution. Also expand CDP blocked patterns
in SetupHeadlessNetworkPolicy to match mapped IPv6 representations.

Signed-off-by: TassioSales <[email protected]>
2026-11-03 01:07:02 -03:00
Hafiar 5896509c84 test(expressions): add regression case for govaluate concat bug
base64(username + ':' + password) evaluated to base64 of "[admin:]admin"
on govaluate v0.0.0-20260729205521-4c61bf19382e, the pin #7998 started
from. The case was requested by @dogancanbakir in review of #7998 and did
not land before the merge.
2026-11-02 23:42:37 +00:00
Mzack9999 825dcfb10a lock runtime 2026-11-02 22:58:36 +01:00
Mzack9999 d465cc82a4 Merge pull request #7995 from KaemiaDeveloper/fix/ipv6-dual-stack-target-selection
Scan every address family with -sa
2026-11-02 22:36:42 +01:00
Mzack9999 48f36b4edc keep fallback 2026-11-02 21:47:23 +01:00
Mzack9999 0e7a56b0d7 prefer ipv6 2026-11-02 21:31:52 +01:00
TassioSales f8cf960022 fix(flow): prevent global state leakage in shared JS runtime pool (#7986)
Always replenish the JS runtime pool with a fresh, clean runtime after flow
template execution instead of reusing contaminated runtimes.

Signed-off-by: TassioSales <[email protected]>
2026-11-02 16:19:52 -03:00
Mzack9999 cbd2cb3881 Merge pull request #8010 from Ayham970/websocket-subprotocol-support
websocket: negotiate Sec-WebSocket-Protocol via the dialer
2026-11-02 19:53:04 +01:00
Mzack9999 1fc87e6b02 Merge pull request #7998 from 0xABCD01/fix/bump-panicking-deps
fix: bump govaluate and antchfx/xpath so malformed expressions error instead of panicking
2026-11-02 19:00:10 +01:00
Mzack9999 7a08c714f9 Merge pull request #8009 from projectdiscovery/dwisiswant0/fix/templates/keep-eid-exclusions-independent-of-etags
fix(templates): keep -eid exclusions independent of -etags
2026-11-02 18:59:46 +01:00
Ayham970andClaude Opus 4.8 6903f2d559 websocket: reject CR/LF in subprotocols; require negotiation in test
Address review feedback:
- Skip Sec-WebSocket-Protocol values containing CR or LF before negotiating
  them, since gobwas/ws writes subprotocols into the handshake request line and
  a line break could smuggle extra handshake headers.
- Tighten the test server to reject the upgrade unless the client offers the
  vite-hmr subprotocol, so the test fails if negotiation does not happen.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-11-02 17:22:43 +04:00
Ayham970andClaude Opus 4.8 27edbd6a64 websocket: negotiate Sec-WebSocket-Protocol via the dialer
The websocket protocol sent a Sec-WebSocket-Protocol header as a raw handshake
header but never set gobwas/ws Dialer.Protocols. gobwas validates the
server-selected subprotocol against Dialer.Protocols and rejects one that was
only sent as a header, so any server that requires and echoes a subprotocol
(e.g. a Vite HMR dev server advertising "vite-hmr") failed the handshake with
"unexpected protocol in Sec-WebSocket-Protocol header".

Route a Sec-WebSocket-Protocol request header into Dialer.Protocols (comma
splitting, trimming) instead of the raw header map, so the subprotocol is
offered in the handshake and the server's echo is accepted. Adds a regression
test with a server that only upgrades when "vite-hmr" is negotiated.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-11-02 17:12:17 +04:00
Dwi Siswanto be678d9d7c fix(templates): keep -eid exclusions independent of -etags
NewTagFilter added an -eid value to the excluded IDs only when -etags
did not list the same value. The check came from the tag loops in
#1448, which changed the map it writes but not the map it reads. With
-eid dup -etags dup, the filter excluded nothing by ID. Workflow steps
still go through this filter, so a workflow ran the template that a -t
scan with the same flags excluded.

Build each filter set from its own flag with one helper, so no set
reads another flag's values. The insert-if-absent checks go too:
writing an existing map key is a no-op, and the one check that changed
anything was the wrong one. -itags stops deleting from the -etags set,
because Match already lets an included tag override its exclusion.

Fixes: #8004
Signed-off-by: Dwi Siswanto <[email protected]>
2026-11-02 19:33:26 +07:00
dw1 515b22a35a Merge pull request #8006 from projectdiscovery/dwisiswant0/fix/workflow/keep-exclusions-from-silently-emptying-workflows
fix(workflow): keep exclusions from silently emptying workflows
2026-11-02 13:13:51 +01:00
github-actions[bot]andehsandeep cd0122aa85 chore: refresh security context (#7996)
Co-authored-by: ehsandeep <[email protected]>
2026-11-02 19:09:40 +07:00
Dwi Siswanto b7d22eb52b fix(workflow): warn when a step's subtemplates cannot run
A step that loads no templates never matches, so the subtemplates
loaded below it never run. Nothing said so when an exclusion emptied
the step: a tags step dropped the filter's error, and a template step
warned only for an excluded tag, naming the template but not the
subtemplates it held back, and only with -v, since warnings are
hidden by default. Such steps are now counted in a warning shown after
loading, and -v logs each one with its workflow.

Signed-off-by: Dwi Siswanto <[email protected]>
2026-11-02 18:33:15 +07:00
Dwi Siswanto c2cecb92a8 fix(workflow): skip -es for steps whose subtemplates loaded
A step whose subtemplates loaded applied every exclusion, -es included.
Most workflows start from a detection template with info severity, so
-es info, often used to hide low-value results, left them running
nothing, while -s low,medium,high,critical, which selects the same
results, still ran them. A step with subtemplates reports no results of
its own beyond late out-of-band matches, so excluding its severity
mostly keeps the checks below it from running. Such a step now skips
-es as it skips -s.

Signed-off-by: Dwi Siswanto <[email protected]>
2026-11-02 18:33:15 +07:00
Codess 9206560d1d style(test): align regression test provider fixture 2026-11-02 12:01:06 +01:00
Codess e94840ad56 test(input): initialize exclusion map and scope DNS fixtures to test hosts 2026-11-02 11:59:33 +01:00
Dwi Siswanto aec0b6eebf fix(workflow)!: load tags steps when a subtemplate passes the filters
With a template filter set, a workflow step selected by tags loaded only
the templates that had its tags and passed the filter. When the filter
selected one of the step's subtemplates instead, the step loaded
nothing, and since subtemplates run only after their step matches, the
selected subtemplate never ran. Steps selected by template already load
for a loaded subtemplate (#7965), but the tag lookup never got that
signal, and it matched the step's tags and the filters in a single call,
so it could not skip the filters without also dropping the tags.

The tag lookup now selects templates by the step's tags first and then
filters them. A step loaded for its subtemplate used to skip every
filter that rejected a template without an error, so -eid, -es and -ept
did not apply to it while -etags and -et did. Both lookups now apply
only the exclusion filters to such a step, so every exclusion holds, as
it does against -itags since #7536.

The tag filter also let -id lift -eid for the same template, so a
workflow ran a template that a scan with the same flags skips. -eid now
wins, as in the metadata index filter.

BREAKING-CHANGE: WorkflowLoader.GetTemplatePathsByTags takes noValidate
Fixes: #7966
Refs: #7534
Signed-off-by: Dwi Siswanto <[email protected]>
2026-11-02 17:24:37 +07:00
Mzack9999 b6dd16ffbf fix concat 2026-11-02 11:23:21 +01:00
Mzack9999 d510a9c2fb Merge pull request #8003 from projectdiscovery/feat/har-input
add har input mode
2026-11-02 11:11:44 +01:00
Mzack9999 de382b5de3 Merge pull request #7994 from projectdiscovery/feat/auto-calibrate
drop HTTP matches on a host's catch-all page with -auto-calibrate
2026-11-02 11:10:23 +01:00
Mzack9999 7ed324680f Merge pull request #8000 from projectdiscovery/fix/tl-jsonl
honour -jsonl in -tl and drop the listing banner from json output
2026-11-02 11:09:12 +01:00
Dogan Can Bakir 684d55eeed add har input mode 2026-11-02 09:31:10 +03:00
Doğan Can Bakır 4dcb37ad23 honour -jsonl in -tl and drop the listing banner from json output 2026-11-02 08:23:57 +03:00
Hafiar ab506a2bb6 fix: bump govaluate and antchfx/xpath so malformed expressions error instead of panicking 2026-11-02 12:18:58 +07:00
Codess 7b44856ce8 fix(input): avoid duplicate scans for automatic dual-stack targets 2026-11-01 21:52:20 +01:00
Codess 288f657fca fix(input): preserve IPv4 fallback when IPv6 is unreachable 2026-11-01 21:38:50 +01:00
Codess 87a8c61932 fix(input): select IPv6-only and dual-stack targets without -iv 2026-11-01 21:25:39 +01:00
Doğan Can Bakır 9c26dc1d2a document -auto-calibrate 2026-11-01 18:54:08 +03:00
Doğan Can Bakır 613e581542 calibrate against the host root and cover the http hook 2026-11-01 18:53:48 +03:00
Doğan Can Bakır 5c9f02a4fc drop http matches on a host's catch-all page with -auto-calibrate 2026-11-01 18:35:12 +03:00
ghost df57ae6d29 chore(js): update memoized functions 🤖 2026-10-30 16:11:04 +00:00
Mzack9999 2a5392eaad Merge pull request #7988 from gnuletik/fix/js-redis-reply-scan
fix(js): check redis reply headers in linear time, skip payloads
2026-10-30 17:10:36 +01:00
Mzack9999 bdf7e983c5 Merge pull request #7990 from IBazsecI/fix/openapi-swagger-nil-panics
fix(openapi,swagger): skip null spec entries instead of panicking
2026-10-30 16:58:25 +01:00
bazsec 6a3300b13d fix(openapi,swagger): drop null response headers, reject only-null servers
Address review feedback:

- swagger: a null entry under a response's `headers:` map is also
  dereferenced by openapi2conv.ToV3Response, for both operation and
  top-level responses. Drop those along with null responses.
- openapi: a `servers:` list containing only null entries now returns
  the same "no servers found" error as an empty list, instead of
  succeeding without generating any request.
2026-10-30 15:18:06 +03:00
bazsec 266593d7a0 fix(openapi,swagger): skip null spec entries instead of panicking
A null value in an OpenAPI or Swagger document (an empty list item, or a
map key with no value) decodes to a nil pointer. Several of these were
dereferenced without a check, so loading such a spec with -im openapi or
-im swagger crashed nuclei with SIGSEGV.

openapi (generator.go):
- a null entry under `servers:` -> nil *Server
- a null media type under `content:` -> nil *MediaType

swagger (dereferenced inside kin-openapi's openapi2conv.ToV3):
- a null path item under `paths:` -> nil *PathItem
- a null item under path/operation `parameters:` or a null top-level
  parameter -> nil *Parameter
- a null operation or top-level response -> nil *Response

Skip the nil entries in the generator, and drop them from the decoded
Swagger document before converting it. Other null entries are already
rejected by kin-openapi with an error and are left alone.

Also add a go-fuzz harness for the OpenAPI parser with a seed corpus, so
the weekly fuzz workflow picks it up, and regression tests that panic on
the pre-fix code.
2026-10-30 14:54:30 +03:00
gnuletikandClaude Opus 5.5 7b0ca00fdf fix(js): check redis reply headers in linear time, skip payloads
The read cap scanned a reply line by line and copied what was left of
its buffer for every line, so a 1 MiB bulk reply of newlines allocated
17 GiB. It also checked lines inside bulk payloads as if they were
headers, so a Lua script returning "a\n*abc" failed with "reply exceeds
16777216 bytes".

Follow the RESP framing as bytes pass instead: skip a bulk payload by
its declared length and check only header lines. A rejected header's
read now ends before its "\n", so go-redis never gets a line to
allocate from, and nothing is buffered or copied.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-30 11:56:54 +01:00
Mzack9999 2e258fc3f5 Merge pull request #7981 from kamilikamil/js-streaming-hash
Add JS modules for streaming hash and decompression
2026-10-29 22:06:48 +01:00
Mzack9999 b3596ac64e dedupe imports 2026-10-29 21:33:09 +01:00
Mzack9999 ac52212d17 canonicalize headless 2026-10-29 21:25:10 +01:00
Mzack9999 1f79f786da Merge pull request #7975 from antcybersec/fix/exclude-hosts-file-comments
fix(cmd): ignore comments in -exclude-hosts values
2026-10-29 17:17:04 +01:00
Mzack9999 3ab77ec3e7 Merge pull request #7972 from arththebird/fix/json-output-nested-yaml-maps
fix(output): encode nested template maps in JSON output
2026-10-29 16:34:01 +01:00
anant 58354693e0 fix(cmd): ignore comments in -exclude-hosts values 2026-10-29 21:02:44 +05:30
Mzack9999 dc817c8dbd Merge pull request #7970 from Eren-Akdag/fix/7969-httpx-probe-resolvers
fix(runner): use -resolvers for the httpx probe of non-URL inputs (#7969)
2026-10-29 16:30:54 +01:00
Arthur LoiselleandClaude Opus 5.5 e55509d881 fix(output): reject metadata keys that collide after string conversion
Keys of different types can format to the same string (`1` and `1.0`), and
keeping either value would depend on map iteration order. `ToJSONValue` and
`ToJSONMap` now return an error on such a collision, which both
`MarshalJSON` methods propagate.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-29 11:04:42 -04:00
Mzack9999 998641728c exclude context 2026-10-29 15:57:32 +01:00
Mzack9999 4e3f78c2c0 exclude context 2026-10-29 15:57:32 +01:00
Mzack9999 7a9ec9c8da Merge pull request #7912 from gnuletik/fix/js-redis-dial-once
fix(js): dial a redis connection once and log its failures as warnings
2026-10-29 15:50:11 +01:00
Mzack9999 5d876b88e5 Merge pull request #7965 from antcybersec/fix/workflow-subtemplate-filters
fix(templates): load workflow steps only when they or a subtemplate pass the filters
2026-10-29 15:50:03 +01:00
Kamil Kamili 7d143c989a Trim comments in the nuclei/hash and nuclei/compress modules 2026-10-29 17:56:02 +04:00
Kamil Kamili aa45ef92f6 Test streamed hashes against the HTTP engine's decoded body
The streamed sha256 and mmh3 of a response read over nuclei/net must
equal sha256(body) and mmh3(body) on the body the HTTP engine's response
chain decodes, for each Content-Encoding, with and without chunked.
2026-10-29 17:29:34 +04:00
Kamil Kamili 7f57e92074 Decode chunked transfer coding in nuclei/compress
NewDecoder now takes a comma-separated list of codings in the order they
were applied, as HTTP headers list them, and accepts chunked. Finished
reports when a self-delimiting stream has ended, so a template reading a
chunked body knows when to stop calling Recv.
2026-10-29 17:29:33 +04:00
Arthur LoiselleandClaude Opus 5.5 3ae6ca8e3d fix(output): encode nested template maps in JSON output
Template YAML decodes nested maps as `map[interface{}]interface{}`, which
the JSON encoder rejects, so a result whose `info.metadata` (or javascript
args carried in `meta`) held a map was dropped from `-j`/`-jsonl` output.
`model.Info` and `output.ResultEvent` now encode their metadata with
string-keyed maps.

Closes #7971

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-29 09:21:08 -04:00
Kamil Kamili 6c615b76df Add nuclei/compress JS module for streaming decompression
A javascript template reading a compressed response with Recv cannot
decode it without first holding the whole body. compress.NewDecoder
takes a Content-Encoding the HTTP engine decodes (gzip, deflate, br,
zstd, or identity) and decodes the chunks written to it, handing back
up to about 1 MB per call so a highly compressed chunk never has to be
held whole.

Go's decompressors pull from a reader while templates push chunks, so
each decoder runs its decompressor on a goroutine that pauses whenever
it needs input or has 1 MB to hand back. The goroutine exits when the
script's context ends, so an abandoned decoder does not leak.
2026-10-29 16:00:39 +04:00
Kamil Kamili 681854444d Add nuclei/hash JS module for incremental hashing
A javascript template that reads a large response with Recv has to
hold the whole body to compare it, since only the DSL hash functions
exist and they take the full input. hash.New(algorithm) returns a
running hash: Write and WriteString add chunks and Sum returns the
value so far, in the same format as the DSL md5, sha1, sha256, sha512
and mmh3 functions. Memory does not grow with the data hashed.
2026-10-29 16:00:39 +04:00
Eren-Akdag 449fdd2c37 test(runner): clear proxy env in the httpx probe resolver test
The probe client uses http.ProxyFromEnvironment, so a proxy set in the
environment could route the probe away from the local test server when
the test runs on its own.
2026-10-29 14:58:05 +03:00
Eren-Akdag 9d7aa53226 fix(runner): use -resolvers for the httpx probe of non-URL inputs
The httpx client that probes non-URL inputs for http or https was built
from httpx.DefaultOptions without the configured resolvers, so the probe
resolved inputs through fastdialer's default resolvers (resolv.conf plus
1.1.1.1, 1.0.0.1, 8.8.8.8 and 8.8.4.4) even when -resolvers was set.

Pass InternalResolversList to httpxOptions.Resolvers in the runner and in
the SDK. httpx then uses only those resolvers. Without -resolvers the
probe behaves as before.

Closes #7969
2026-10-29 14:37:28 +03:00
dependabot[bot] 3ee4f0f408 chore(deps): bump github.com/getkin/kin-openapi (#7968)
Bumps the go_modules group with 1 update in the / directory: [github.com/getkin/kin-openapi](https://github.com/getkin/kin-openapi).


Updates `github.com/getkin/kin-openapi` from 0.144.0 to 0.149.0
- [Release notes](https://github.com/getkin/kin-openapi/releases)
- [Commits](https://github.com/getkin/kin-openapi/compare/v0.144.0...v0.149.0)

---
updated-dependencies:
- dependency-name: github.com/getkin/kin-openapi
  dependency-version: 0.149.0
  dependency-type: direct:production
  dependency-group: go_modules
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-29 04:37:22 +00:00
ghost 091191f518 docs: update syntax & JSON schema 🤖 2026-10-29 04:05:50 +00:00
dependabot[bot] 827f4a9f88 chore(deps): bump the modules group with 2 updates (#7942)
Bumps the modules group with 2 updates: [github.com/projectdiscovery/dsl](https://github.com/projectdiscovery/dsl) and [github.com/projectdiscovery/wappalyzergo](https://github.com/projectdiscovery/wappalyzergo).


Updates `github.com/projectdiscovery/dsl` from 0.8.24 to 0.9.0
- [Release notes](https://github.com/projectdiscovery/dsl/releases)
- [Commits](https://github.com/projectdiscovery/dsl/compare/v0.8.24...v0.9.0)

Updates `github.com/projectdiscovery/wappalyzergo` from 0.3.5 to 0.3.6
- [Release notes](https://github.com/projectdiscovery/wappalyzergo/releases)
- [Commits](https://github.com/projectdiscovery/wappalyzergo/compare/v0.3.5...v0.3.6)

---
updated-dependencies:
- dependency-name: github.com/projectdiscovery/dsl
  dependency-version: 0.9.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: modules
- dependency-name: github.com/projectdiscovery/wappalyzergo
  dependency-version: 0.3.6
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: modules
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-29 11:00:17 +07:00
Mzack9999 bd1c4dd226 feat(headless): expose header/cookie DSL variables (#7573)
* header variables

* Fix review

* Browser cookies

* Reserve keys

* Host cookies
2026-10-29 10:59:17 +07:00
github-actions[bot]andMzack9999 f990fe4d5d chore: refresh security context (#7693)
Co-authored-by: Mzack9999 <[email protected]>
2026-10-29 10:57:39 +07:00
Mzack9999 00ed0552da feat(js): Apply network policy to mysql ConnectWithDSN (#7768)
* enforce policy

* Fix review
2026-10-29 10:57:00 +07:00
Mzack9999 099a037928 feat(headless): chrome sandbox root-only (#7951) 2026-10-29 10:55:52 +07:00
Mzack9999 12d6f03826 test: isolate resolvers (#7952) 2026-10-29 10:53:53 +07:00
anant 1644ba7e81 fix(templates): load workflow steps only when they or a subtemplate pass the filters 2026-10-29 04:40:36 +05:30
Mzack9999 a113cf0db8 Merge pull request #7962 from je-bugshell/feat/no-redact-flag
Add -no-redact to disable the built-in credential redaction
2026-10-28 18:12:43 +01:00
Mzack9999 97337e2b4a update readmes 2026-10-28 17:41:18 +01:00
Mzack9999 8d29c12eef fix redaction 2026-10-28 16:36:38 +01:00
John Eric von Rosen 4fc17dc284 Keep a semicolon inside a redacted query value
Stopping a key=value match at every ; left ?password=;s3cr3t unmasked, since
the value could not start. The value is now lazy and ends only at a cookie
delimiter (; name=), &, a quote or a line break.
2026-10-28 13:09:35 +01:00
John Eric von Rosen 035fc270f9 Keep neighbouring cookies and query parameters when redacting one key
A redacted value used to run to the end of the line, so -redact tracking on
Cookie: tracking=1; session=abc masked the session cookie too. A key=value
match now stops at ; or &; a Key: header is still masked whole. The test also
covers the matched URL.
2026-10-28 13:00:32 +01:00
John Eric von Rosen 3f341f0222 Add -no-redact to disable the built-in credential redaction
Since #7469 the output writer always masks Authorization, Proxy-Authorization,
Cookie, Set-Cookie and X-Api-Key, which leaves exported findings without the
data needed to reproduce them by hand. -no-redact turns the built-in list off;
keys given with -redact are still masked.
2026-10-28 12:35:18 +01:00
Mzack9999 26ce6d2ef2 Merge pull request #7959 from arxhr007/fix/7949-validate-missing-path
fix(loader): fail -validate when a template path can't be found
2026-10-27 18:13:16 +01:00
Mzack9999 0f34b1b6b8 Merge pull request #7960 from projectdiscovery/fix/7957-exclude-hosts-match
fix exclude-hosts matching unrelated hosts at dial time
2026-10-27 17:03:18 +01:00
Doğan Can Bakır 77ba7792f3 keep subdomain and regex exclude-hosts entries 2026-10-27 18:37:21 +03:00
Doğan Can Bakır 8caa65a839 match exclude-hosts entries exactly at dial time 2026-10-27 17:58:24 +03:00
arxhr007 a4afaa41b2 fix(loader): fail -validate when a template path can't be found
`nuclei -validate -t /no/such/path` logged "Could not find template" and
then still printed "All templates validated successfully" and exited 0,
so a mistyped path in a CI step passed. ValidateTemplates ignored the
paths that GetTemplatesPath could not resolve.

Count the paths that aren't handled by NotFoundCallback and return the
validation error when there is any. Paths the callback resolves are
still accepted.

Fixes #7949
2026-10-27 19:59:58 +05:30
Mzack9999 abcdcf19a4 Merge pull request #7940 from mazzz3r/feat/info-placeholders
feat(templates): fill placeholders in info description, impact and remediation
2026-10-27 15:21:09 +01:00
Michael Rodger 9dc004926d fix(trackers): gitea reporter pagination (#7945) 2026-10-27 14:11:01 +07:00
Mzack9999 3d2f411e1d Merge pull request #7925 from projectdiscovery/dwisiswant0/fix/offlinehttp/read-status-lines-by-position-not-text-search
fix(offlinehttp): read status lines by position, not text search
2026-10-26 21:38:57 +01:00
Mzack9999 763e490f42 merge dev 2026-10-26 21:10:10 +01:00
Mzack9999 421ec70686 Merge pull request #7947 from projectdiscovery/dwisiswant0/fix/trackers/follow-x-next-page-in-gitlab-duplicate-search
fix(trackers): follow X-Next-Page in gitlab duplicate search
2026-10-26 20:47:55 +01:00
gnuletikandClaude Opus 5.5 1eeb2f3979 fix(js): log go-redis messages as gologger warnings
Rather than disabling go-redis's logger, route its messages through
gologger at warning level. nuclei's CLI then shows them with -v or -debug
only, and a program embedding nuclei receives them through its gologger
writer, instead of go-redis writing to stderr on its own.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-26 20:05:37 +01:00
gnuletikandClaude Opus 5.5 ea108ee04f fix(js): dial a redis connection once and keep go-redis off stderr
go-redis v9.19 retries a failed dial 5 times, 100ms apart, from a goroutine
that dials on its own context rather than the caller's. Against a filtered
port, a JS call returned at its execution deadline while the pool kept
dialing for ~25s; against a closed port, one call made 20 dials and took
1.7s. Each exhausted dial also printed "connection pool: failed to dial after
5 attempts" to stderr.

The redis clients now dial once per command attempt, as go-redis v9.11 did,
and go-redis's logger is disabled, as the mysql lib does for its driver.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-26 20:05:37 +01:00
Dwi Siswanto aac1427cdc fix(trackers): follow X-Next-Page in gitlab duplicate search
The GitLab tracker ended the duplicate search at the first page holding
fewer issues than duplicate-issue-page-size. GitLab caps per_page at
100, so with a page size above 100 every page looked like the last one.
The tracker read only the first 100 search results and opened a new
issue when the match came later.

The client library already parses X-Next-Page into Response.NextPage.
Follow it instead of guessing from the page length. This also drops the
empty-page request the old check sent when the last page was full.

Fixes: #7946
Refs: #7944
Signed-off-by: Dwi Siswanto <[email protected]>
2026-10-26 21:10:51 +07:00
mazzz3r c4552c6011 feat(templates): fill placeholders in info description, impact and remediation
Description, impact and remediation can reference values of the finding
with the usual {{...}} syntax: named extractors, payload values of the
matching request, variables, DSL helpers, and the result fields host,
port, ip, url, scheme, path, matched-at, matcher-name and
extractor-name, which win over values of the same name. Name, severity,
tags and classification stay static, since filtering, grouping and
issue trackers key on them.

MakeDefaultResultEvent renders each result's copy of the info, so every
protocol gets it and the template never changes. Each field goes
through render.Render without scan options, like request and matcher
text, so nested placeholders and helpers behave the same and network
helpers stay denied. A field whose expression fails keeps the
template's text.

When placeholders change the text, the result keeps the written info in
TemplateInfo (json:"-" bson:"-"):
- the writer runs the -redact patterns over the changed fields, as it
  does over the raw request and response;
- SARIF builds the shared rule from the written info and puts each
  finding's filled text, capped at 4096 characters, in its result
  message.

Info is filled only from a template's own event. Templates with info
placeholders are not clustered, since a cluster's shared event holds the
first template's variables and extracted values, and a global matcher
fills its info from the result fields alone.
2026-10-25 19:37:42 +00:00
Dwi Siswanto 4331cd0f7a fix(offlinehttp): read status lines by position, not text search
Passive mode added a minor version to status lines such as "HTTP/2 200
OK" by rewriting the first "HTTP/<digit> " anywhere in the raw data. In
a valid HTTP/1.1 response that text could sit in a header, the reason
phrase, or the body, which was then altered: a Content-Length body came
out truncated and a chunked one failed to decode. Only the protocol
token at the start of the response is normalized now.

When the data starts with a request, the response is now read from
where the request body ends, with the same normalization. That path
used to reject HTTP/2 status lines and read a request body as the
status line, then fall back to the last "HTTP/" in the data, which
fails whenever the response body contains one. A dump that keeps a
request's Content-Length but leaves out its body is retried right after
the headers. Two exchanges back to back now yield the first response
for HTTP/2, as they already did for HTTP/1.1.

Up to five interim 1xx responses are skipped to reach the final one,
and an interim response is returned only when no final response is
found. A response read after a request carries that request. Data that
cannot be read this way still uses the last "HTTP/" search; httpx
-store-chain output, which puts a newline after the first request,
relies on it to return the chain's final response.

This supersedes #7922, which anchors the version pattern and so fixes

Fixes: #7921
Fixes: #7924
Reported-by: @MasterchiefGalaxy
Signed-off-by: Dwi Siswanto <[email protected]>
2026-10-25 13:16:36 +01:00
190 changed files with 7352 additions and 9711 deletions

No files matched your search

+1 -1
View File
@@ -36,7 +36,7 @@ It also specifies a `WorkflowLoader` interface that is used during workflow load
```go
type WorkflowLoader interface {
GetTemplatePathsByTags(tags []string) []string
GetTemplatePathsByTags(tags []string, noValidate bool) []string
GetTemplatePaths(templatesList []string, noValidate bool) []string
}
```
+7 -15
View File
@@ -145,12 +145,12 @@ TARGET:
-eh, -exclude-hosts string[] hosts to exclude to scan from the input list (ip, cidr, hostname)
-resume string resume scan from and save to specified file (clustering will be disabled)
-sa, -scan-all-ips scan all the IP's associated with dns record
-iv, -ip-version string[] IP version to scan of hostname (4,6) - (default 4)
-iv, -ip-version string[] IP version to scan of hostname (4,6); default uses dialer fallback, -sa scans all addresses
-dun, -disable-url-normalization keep targets exactly as supplied instead of collapsing urls that address the same resource
-mup, -max-urls-per-pattern int scan at most this many urls sharing one structural shape, e.g. /user/1 and /user/2 (0 to scan all)
TARGET-FORMAT:
-im, -input-mode string mode of input file (list, burp, jsonl, yaml, openapi, swagger, http) (default "list")
-im, -input-mode string mode of input file (list, burp, har, jsonl, yaml, openapi, swagger, http) (default "list")
-ro, -required-only use only required fields in input format when generating requests
-sfv, -skip-format-validation skip format validation (like missing vars) when parsing input file
-vtt, -vars-text-templating enable text templating for vars in input file (only for yaml input mode)
@@ -177,7 +177,7 @@ TEMPLATES:
-egm, -enable-global-matchers enable loading global matchers templates
-file enable loading file templates
-llm enable llm matchers and extractors (semantic matching)
-llm-provider string llm provider for semantic matching (openai, ollama, llamacpp, vllm, lmstudio, groq, openrouter, together, orcarouter) (default "openai")
-llm-provider string llm provider for semantic matching (openai, ollama, llamacpp, vllm, lmstudio, groq, openrouter, together, orcarouter, jev) (default "openai")
-llm-base-url string openai-compatible endpoint for semantic matching (overrides -llm-provider, e.g. a local model)
-llm-model string model used for semantic matching
-llm-timeout int time in seconds to wait for a single llm call (default 30)
@@ -221,6 +221,7 @@ OUTPUT:
-jle, -jsonl-export string file to export results in JSONL(ine) format
-pe, -pdf-export string file to export results in PDF format
-rd, -redact string[] redact given list of keys from query parameter, request header and body
-nrd, -no-redact disable the built-in redaction of credential headers (Authorization, Cookie, Set-Cookie, ...) in output; keys given with -redact are still redacted
CONFIGURATIONS:
-config string path to the nuclei configuration file
@@ -326,6 +327,7 @@ OPTIMIZATIONS:
-nh, -no-httpx disable httpx probing for non-url input
-preflight-portscan run preflight resolve + TCP portscan and filter targets before scanning (disabled by default)
-no-stdin disable stdin processing
-ac, -auto-calibrate drop HTTP matches on a host's catch-all page (answers missing paths with 2xx/3xx)
HEADLESS:
-headless enable templates that require headless browser support (root user on Linux will disable sandbox)
@@ -386,18 +388,8 @@ CLOUD:
-pdu, -dashboard-upload string upload / view nuclei results file (jsonl) in projectdiscovery cloud (pdcp) UI dashboard
AUTHENTICATION:
-sf, -secret-file string[] path to config file containing secrets for nuclei authenticated scan
-ps, -prefetch-secrets prefetch secrets from the secrets file
-alu, -auth-login-url string login page url for turnkey auto-login authenticated scan
-au, -auth-username string username for auto-login (-auth-login-url)
-apw, -auth-password string password for auto-login (-auth-login-url); prefer NUCLEI_AUTH_PASSWORD to avoid argv exposure
-auf, -auth-username-field string override the detected username form field name for auto-login
-apf, -auth-password-field string override the detected password form field name for auto-login
-ah, -auth-headless drive auto-login through a real browser (JS/SPA/SSO login pages)
-ar, -auth-recording string path to a chrome devtools recorder json login flow to replay for auto-login
-ac, -auth-capture open a browser to log in manually once, then capture the session for the scan
-arsc, -auth-reauth-status-codes string comma-separated status codes (e.g. 401,403) that trigger an auto-login re-authentication during the scan
-ari, -auth-refresh-interval string re-run the auto-login once the captured session is older than this duration (e.g. 15m)
-sf, -secret-file string[] path to config file containing secrets for nuclei authenticated scan
-ps, -prefetch-secrets prefetch secrets from the secrets file
# NOTE: Headers in secrets files preserve exact casing (useful for case-sensitive APIs)
+5 -14
View File
@@ -148,7 +148,7 @@ TARGET:
-iv, -ip-version string[] IP version to scan of hostname (4,6) - (default 4)
TARGET-FORMAT:
-im, -input-mode string mode of input file (list, burp, jsonl, yaml, openapi, swagger, http) (default "list")
-im, -input-mode string mode of input file (list, burp, har, jsonl, yaml, openapi, swagger, http) (default "list")
-ro, -required-only use only required fields in input format when generating requests
-sfv, -skip-format-validation skip format validation (like missing vars) when parsing input file
-vtt, -vars-text-templating enable text templating for vars in input file (only for yaml input mode)
@@ -175,7 +175,7 @@ TEMPLATES:
-egm, -enable-global-matchers enable loading global matchers templates
-file enable loading file templates
-llm enable llm matchers and extractors (semantic matching)
-llm-provider string llm provider for semantic matching (openai, ollama, llamacpp, vllm, lmstudio, groq, openrouter, together, orcarouter) (default "openai")
-llm-provider string llm provider for semantic matching (openai, ollama, llamacpp, vllm, lmstudio, groq, openrouter, together, orcarouter, jev) (default "openai")
-llm-base-url string openai-compatible endpoint for semantic matching (overrides -llm-provider, e.g. a local model)
-llm-model string model used for semantic matching
-llm-timeout int time in seconds to wait for a single llm call (default 30)
@@ -219,6 +219,7 @@ OUTPUT:
-jle, -jsonl-export string file to export results in JSONL(ine) format
-pe, -pdf-export string file to export results in PDF format
-rd, -redact string[] redact given list of keys from query parameter, request header and body
-nrd, -no-redact disable the built-in redaction of credential headers (Authorization, Cookie, Set-Cookie, ...) in output; keys given with -redact are still redacted
CONFIGURATIONS:
-config string path to the nuclei configuration file
@@ -380,18 +381,8 @@ CLOUD:
-pdu, -dashboard-upload string upload / view nuclei results file (jsonl) in projectdiscovery cloud (pdcp) UI dashboard
AUTHENTICATION:
-sf, -secret-file string[] path to config file containing secrets for nuclei authenticated scan
-ps, -prefetch-secrets prefetch secrets from the secrets file
-alu, -auth-login-url string login page url for turnkey auto-login authenticated scan
-au, -auth-username string username for auto-login (-auth-login-url)
-apw, -auth-password string password for auto-login (-auth-login-url); prefer NUCLEI_AUTH_PASSWORD to avoid argv exposure
-auf, -auth-username-field string override the detected username form field name for auto-login
-apf, -auth-password-field string override the detected password form field name for auto-login
-ah, -auth-headless drive auto-login through a real browser (JS/SPA/SSO login pages)
-ar, -auth-recording string path to a chrome devtools recorder json login flow to replay for auto-login
-ac, -auth-capture open a browser to log in manually once, then capture the session for the scan
-arsc, -auth-reauth-status-codes string comma-separated status codes (e.g. 401,403) that trigger an auto-login re-authentication during the scan
-ari, -auth-refresh-interval string re-run the auto-login once the captured session is older than this duration (e.g. 15m)
-sf, -secret-file string[] path to config file containing secrets for nuclei authenticated scan
-ps, -prefetch-secrets prefetch secrets from the secrets file
# NOTE: Headers in secrets files preserve exact casing (useful for case-sensitive APIs)
+5 -14
View File
@@ -148,7 +148,7 @@ TARGET:
-iv, -ip-version string[] IP version to scan of hostname (4,6) - (default 4)
TARGET-FORMAT:
-im, -input-mode string mode of input file (list, burp, jsonl, yaml, openapi, swagger, http) (default "list")
-im, -input-mode string mode of input file (list, burp, har, jsonl, yaml, openapi, swagger, http) (default "list")
-ro, -required-only use only required fields in input format when generating requests
-sfv, -skip-format-validation skip format validation (like missing vars) when parsing input file
-vtt, -vars-text-templating enable text templating for vars in input file (only for yaml input mode)
@@ -175,7 +175,7 @@ TEMPLATES:
-egm, -enable-global-matchers enable loading global matchers templates
-file enable loading file templates
-llm enable llm matchers and extractors (semantic matching)
-llm-provider string llm provider for semantic matching (openai, ollama, llamacpp, vllm, lmstudio, groq, openrouter, together, orcarouter) (default "openai")
-llm-provider string llm provider for semantic matching (openai, ollama, llamacpp, vllm, lmstudio, groq, openrouter, together, orcarouter, jev) (default "openai")
-llm-base-url string openai-compatible endpoint for semantic matching (overrides -llm-provider, e.g. a local model)
-llm-model string model used for semantic matching
-llm-timeout int time in seconds to wait for a single llm call (default 30)
@@ -219,6 +219,7 @@ OUTPUT:
-jle, -jsonl-export string file to export results in JSONL(ine) format
-pe, -pdf-export string file to export results in PDF format
-rd, -redact string[] redact given list of keys from query parameter, request header and body
-nrd, -no-redact disable the built-in redaction of credential headers (Authorization, Cookie, Set-Cookie, ...) in output; keys given with -redact are still redacted
CONFIGURATIONS:
-config string path to the nuclei configuration file
@@ -380,18 +381,8 @@ CLOUD:
-pdu, -dashboard-upload string upload / view nuclei results file (jsonl) in projectdiscovery cloud (pdcp) UI dashboard
AUTHENTICATION:
-sf, -secret-file string[] path to config file containing secrets for nuclei authenticated scan
-ps, -prefetch-secrets prefetch secrets from the secrets file
-alu, -auth-login-url string login page url for turnkey auto-login authenticated scan
-au, -auth-username string username for auto-login (-auth-login-url)
-apw, -auth-password string password for auto-login (-auth-login-url); prefer NUCLEI_AUTH_PASSWORD to avoid argv exposure
-auf, -auth-username-field string override the detected username form field name for auto-login
-apf, -auth-password-field string override the detected password form field name for auto-login
-ah, -auth-headless drive auto-login through a real browser (JS/SPA/SSO login pages)
-ar, -auth-recording string path to a chrome devtools recorder json login flow to replay for auto-login
-ac, -auth-capture open a browser to log in manually once, then capture the session for the scan
-arsc, -auth-reauth-status-codes string comma-separated status codes (e.g. 401,403) that trigger an auto-login re-authentication during the scan
-ari, -auth-refresh-interval string re-run the auto-login once the captured session is older than this duration (e.g. 15m)
-sf, -secret-file string[] path to config file containing secrets for nuclei authenticated scan
-ps, -prefetch-secrets prefetch secrets from the secrets file
# NOTE: Headers in secrets files preserve exact casing (useful for case-sensitive APIs)
+5 -14
View File
@@ -165,7 +165,7 @@ TARGET:
-mup, -max-urls-per-pattern int scan at most this many urls sharing one structural shape, e.g. /user/1 and /user/2 (0 to scan all)
TARGET-FORMAT:
-im, -input-mode string mode of input file (list, burp, jsonl, yaml, openapi, swagger, http) (default "list")
-im, -input-mode string mode of input file (list, burp, har, jsonl, yaml, openapi, swagger, http) (default "list")
-ro, -required-only use only required fields in input format when generating requests
-sfv, -skip-format-validation skip format validation (like missing vars) when parsing input file
-vtt, -vars-text-templating enable text templating for vars in input file (only for yaml input mode)
@@ -192,7 +192,7 @@ TEMPLATES:
-egm, -enable-global-matchers enable loading global matchers templates
-file enable loading file templates
-llm enable llm matchers and extractors (semantic matching)
-llm-provider string llm provider for semantic matching (openai, ollama, llamacpp, vllm, lmstudio, groq, openrouter, together, orcarouter) (default "openai")
-llm-provider string llm provider for semantic matching (openai, ollama, llamacpp, vllm, lmstudio, groq, openrouter, together, orcarouter, jev) (default "openai")
-llm-base-url string openai-compatible endpoint for semantic matching (overrides -llm-provider, e.g. a local model)
-llm-model string model used for semantic matching
-llm-timeout int time in seconds to wait for a single llm call (default 30)
@@ -236,6 +236,7 @@ OUTPUT:
-jle, -jsonl-export string file to export results in JSONL(ine) format
-pe, -pdf-export string file to export results in PDF format
-rd, -redact string[] redact given list of keys from query parameter, request header and body
-nrd, -no-redact disable the built-in redaction of credential headers (Authorization, Cookie, Set-Cookie, ...) in output; keys given with -redact are still redacted
CONFIGURATIONS:
-config string path to the nuclei configuration file
@@ -401,18 +402,8 @@ CLOUD:
-pdu, -dashboard-upload string upload / view nuclei results file (jsonl) in projectdiscovery cloud (pdcp) UI dashboard
AUTHENTICATION:
-sf, -secret-file string[] path to config file containing secrets for nuclei authenticated scan
-ps, -prefetch-secrets prefetch secrets from the secrets file
-alu, -auth-login-url string login page url for turnkey auto-login authenticated scan
-au, -auth-username string username for auto-login (-auth-login-url)
-apw, -auth-password string password for auto-login (-auth-login-url); prefer NUCLEI_AUTH_PASSWORD to avoid argv exposure
-auf, -auth-username-field string override the detected username form field name for auto-login
-apf, -auth-password-field string override the detected password form field name for auto-login
-ah, -auth-headless drive auto-login through a real browser (JS/SPA/SSO login pages)
-ar, -auth-recording string path to a chrome devtools recorder json login flow to replay for auto-login
-ac, -auth-capture open a browser to log in manually once, then capture the session for the scan
-arsc, -auth-reauth-status-codes string comma-separated status codes (e.g. 401,403) that trigger an auto-login re-authentication during the scan
-ari, -auth-refresh-interval string re-run the auto-login once the captured session is older than this duration (e.g. 15m)
-sf, -secret-file string[] path to config file containing secrets for nuclei authenticated scan
-ps, -prefetch-secrets prefetch secrets from the secrets file
# NOTE: Headers in secrets files preserve exact casing (useful for case-sensitive APIs)
+5 -14
View File
@@ -148,7 +148,7 @@ TARGET:
-iv, -ip-version string[] IP version to scan of hostname (4,6) - (default 4)
TARGET-FORMAT:
-im, -input-mode string mode of input file (list, burp, jsonl, yaml, openapi, swagger, http) (default "list")
-im, -input-mode string mode of input file (list, burp, har, jsonl, yaml, openapi, swagger, http) (default "list")
-ro, -required-only use only required fields in input format when generating requests
-sfv, -skip-format-validation skip format validation (like missing vars) when parsing input file
-vtt, -vars-text-templating enable text templating for vars in input file (only for yaml input mode)
@@ -175,7 +175,7 @@ TEMPLATES:
-egm, -enable-global-matchers enable loading global matchers templates
-file enable loading file templates
-llm enable llm matchers and extractors (semantic matching)
-llm-provider string llm provider for semantic matching (openai, ollama, llamacpp, vllm, lmstudio, groq, openrouter, together, orcarouter) (default "openai")
-llm-provider string llm provider for semantic matching (openai, ollama, llamacpp, vllm, lmstudio, groq, openrouter, together, orcarouter, jev) (default "openai")
-llm-base-url string openai-compatible endpoint for semantic matching (overrides -llm-provider, e.g. a local model)
-llm-model string model used for semantic matching
-llm-timeout int time in seconds to wait for a single llm call (default 30)
@@ -219,6 +219,7 @@ OUTPUT:
-jle, -jsonl-export string file to export results in JSONL(ine) format
-pe, -pdf-export string file to export results in PDF format
-rd, -redact string[] redact given list of keys from query parameter, request header and body
-nrd, -no-redact disable the built-in redaction of credential headers (Authorization, Cookie, Set-Cookie, ...) in output; keys given with -redact are still redacted
CONFIGURATIONS:
-config string path to the nuclei configuration file
@@ -380,18 +381,8 @@ CLOUD:
-pdu, -dashboard-upload string upload / view nuclei results file (jsonl) in projectdiscovery cloud (pdcp) UI dashboard
AUTHENTICATION:
-sf, -secret-file string[] path to config file containing secrets for nuclei authenticated scan
-ps, -prefetch-secrets prefetch secrets from the secrets file
-alu, -auth-login-url string login page url for turnkey auto-login authenticated scan
-au, -auth-username string username for auto-login (-auth-login-url)
-apw, -auth-password string password for auto-login (-auth-login-url); prefer NUCLEI_AUTH_PASSWORD to avoid argv exposure
-auf, -auth-username-field string override the detected username form field name for auto-login
-apf, -auth-password-field string override the detected password form field name for auto-login
-ah, -auth-headless drive auto-login through a real browser (JS/SPA/SSO login pages)
-ar, -auth-recording string path to a chrome devtools recorder json login flow to replay for auto-login
-ac, -auth-capture open a browser to log in manually once, then capture the session for the scan
-arsc, -auth-reauth-status-codes string comma-separated status codes (e.g. 401,403) that trigger an auto-login re-authentication during the scan
-ari, -auth-refresh-interval string re-run the auto-login once the captured session is older than this duration (e.g. 15m)
-sf, -secret-file string[] path to config file containing secrets for nuclei authenticated scan
-ps, -prefetch-secrets prefetch secrets from the secrets file
# NOTE: Headers in secrets files preserve exact casing (useful for case-sensitive APIs)
+5 -14
View File
@@ -148,7 +148,7 @@ TARGET:
-iv, -ip-version string[] IP version to scan of hostname (4,6) - (default 4)
TARGET-FORMAT:
-im, -input-mode string mode of input file (list, burp, jsonl, yaml, openapi, swagger, http) (default "list")
-im, -input-mode string mode of input file (list, burp, har, jsonl, yaml, openapi, swagger, http) (default "list")
-ro, -required-only use only required fields in input format when generating requests
-sfv, -skip-format-validation skip format validation (like missing vars) when parsing input file
-vtt, -vars-text-templating enable text templating for vars in input file (only for yaml input mode)
@@ -175,7 +175,7 @@ TEMPLATES:
-egm, -enable-global-matchers enable loading global matchers templates
-file enable loading file templates
-llm enable llm matchers and extractors (semantic matching)
-llm-provider string llm provider for semantic matching (openai, ollama, llamacpp, vllm, lmstudio, groq, openrouter, together, orcarouter) (default "openai")
-llm-provider string llm provider for semantic matching (openai, ollama, llamacpp, vllm, lmstudio, groq, openrouter, together, orcarouter, jev) (default "openai")
-llm-base-url string openai-compatible endpoint for semantic matching (overrides -llm-provider, e.g. a local model)
-llm-model string model used for semantic matching
-llm-timeout int time in seconds to wait for a single llm call (default 30)
@@ -219,6 +219,7 @@ OUTPUT:
-jle, -jsonl-export string file to export results in JSONL(ine) format
-pe, -pdf-export string file to export results in PDF format
-rd, -redact string[] redact given list of keys from query parameter, request header and body
-nrd, -no-redact disable the built-in redaction of credential headers (Authorization, Cookie, Set-Cookie, ...) in output; keys given with -redact are still redacted
CONFIGURATIONS:
-config string path to the nuclei configuration file
@@ -380,18 +381,8 @@ CLOUD:
-pdu, -dashboard-upload string upload / view nuclei results file (jsonl) in projectdiscovery cloud (pdcp) UI dashboard
AUTHENTICATION:
-sf, -secret-file string[] path to config file containing secrets for nuclei authenticated scan
-ps, -prefetch-secrets prefetch secrets from the secrets file
-alu, -auth-login-url string login page url for turnkey auto-login authenticated scan
-au, -auth-username string username for auto-login (-auth-login-url)
-apw, -auth-password string password for auto-login (-auth-login-url); prefer NUCLEI_AUTH_PASSWORD to avoid argv exposure
-auf, -auth-username-field string override the detected username form field name for auto-login
-apf, -auth-password-field string override the detected password form field name for auto-login
-ah, -auth-headless drive auto-login through a real browser (JS/SPA/SSO login pages)
-ar, -auth-recording string path to a chrome devtools recorder json login flow to replay for auto-login
-ac, -auth-capture open a browser to log in manually once, then capture the session for the scan
-arsc, -auth-reauth-status-codes string comma-separated status codes (e.g. 401,403) that trigger an auto-login re-authentication during the scan
-ari, -auth-refresh-interval string re-run the auto-login once the captured session is older than this duration (e.g. 15m)
-sf, -secret-file string[] path to config file containing secrets for nuclei authenticated scan
-ps, -prefetch-secrets prefetch secrets from the secrets file
# NOTE: Headers in secrets files preserve exact casing (useful for case-sensitive APIs)
+5 -14
View File
@@ -148,7 +148,7 @@ TARGET:
-iv, -ip-version string[] IP version to scan of hostname (4,6) - (default 4)
TARGET-FORMAT:
-im, -input-mode string mode of input file (list, burp, jsonl, yaml, openapi, swagger, http) (default "list")
-im, -input-mode string mode of input file (list, burp, har, jsonl, yaml, openapi, swagger, http) (default "list")
-ro, -required-only use only required fields in input format when generating requests
-sfv, -skip-format-validation skip format validation (like missing vars) when parsing input file
-vtt, -vars-text-templating enable text templating for vars in input file (only for yaml input mode)
@@ -175,7 +175,7 @@ TEMPLATES:
-egm, -enable-global-matchers enable loading global matchers templates
-file enable loading file templates
-llm enable llm matchers and extractors (semantic matching)
-llm-provider string llm provider for semantic matching (openai, ollama, llamacpp, vllm, lmstudio, groq, openrouter, together, orcarouter) (default "openai")
-llm-provider string llm provider for semantic matching (openai, ollama, llamacpp, vllm, lmstudio, groq, openrouter, together, orcarouter, jev) (default "openai")
-llm-base-url string openai-compatible endpoint for semantic matching (overrides -llm-provider, e.g. a local model)
-llm-model string model used for semantic matching
-llm-timeout int time in seconds to wait for a single llm call (default 30)
@@ -219,6 +219,7 @@ OUTPUT:
-jle, -jsonl-export string file to export results in JSONL(ine) format
-pe, -pdf-export string file to export results in PDF format
-rd, -redact string[] redact given list of keys from query parameter, request header and body
-nrd, -no-redact disable the built-in redaction of credential headers (Authorization, Cookie, Set-Cookie, ...) in output; keys given with -redact are still redacted
CONFIGURATIONS:
-config string path to the nuclei configuration file
@@ -380,18 +381,8 @@ CLOUD:
-pdu, -dashboard-upload string upload / view nuclei results file (jsonl) in projectdiscovery cloud (pdcp) UI dashboard
AUTHENTICATION:
-sf, -secret-file string[] path to config file containing secrets for nuclei authenticated scan
-ps, -prefetch-secrets prefetch secrets from the secrets file
-alu, -auth-login-url string login page url for turnkey auto-login authenticated scan
-au, -auth-username string username for auto-login (-auth-login-url)
-apw, -auth-password string password for auto-login (-auth-login-url); prefer NUCLEI_AUTH_PASSWORD to avoid argv exposure
-auf, -auth-username-field string override the detected username form field name for auto-login
-apf, -auth-password-field string override the detected password form field name for auto-login
-ah, -auth-headless drive auto-login through a real browser (JS/SPA/SSO login pages)
-ar, -auth-recording string path to a chrome devtools recorder json login flow to replay for auto-login
-ac, -auth-capture open a browser to log in manually once, then capture the session for the scan
-arsc, -auth-reauth-status-codes string comma-separated status codes (e.g. 401,403) that trigger an auto-login re-authentication during the scan
-ari, -auth-refresh-interval string re-run the auto-login once the captured session is older than this duration (e.g. 15m)
-sf, -secret-file string[] path to config file containing secrets for nuclei authenticated scan
-ps, -prefetch-secrets prefetch secrets from the secrets file
# NOTE: Headers in secrets files preserve exact casing (useful for case-sensitive APIs)
+5 -14
View File
@@ -148,7 +148,7 @@ TARGET:
-iv, -ip-version string[] IP version to scan of hostname (4,6) - (default 4)
TARGET-FORMAT:
-im, -input-mode string mode of input file (list, burp, jsonl, yaml, openapi, swagger, http) (default "list")
-im, -input-mode string mode of input file (list, burp, har, jsonl, yaml, openapi, swagger, http) (default "list")
-ro, -required-only use only required fields in input format when generating requests
-sfv, -skip-format-validation skip format validation (like missing vars) when parsing input file
-vtt, -vars-text-templating enable text templating for vars in input file (only for yaml input mode)
@@ -175,7 +175,7 @@ TEMPLATES:
-egm, -enable-global-matchers enable loading global matchers templates
-file enable loading file templates
-llm enable llm matchers and extractors (semantic matching)
-llm-provider string llm provider for semantic matching (openai, ollama, llamacpp, vllm, lmstudio, groq, openrouter, together, orcarouter) (default "openai")
-llm-provider string llm provider for semantic matching (openai, ollama, llamacpp, vllm, lmstudio, groq, openrouter, together, orcarouter, jev) (default "openai")
-llm-base-url string openai-compatible endpoint for semantic matching (overrides -llm-provider, e.g. a local model)
-llm-model string model used for semantic matching
-llm-timeout int time in seconds to wait for a single llm call (default 30)
@@ -219,6 +219,7 @@ OUTPUT:
-jle, -jsonl-export string file to export results in JSONL(ine) format
-pe, -pdf-export string file to export results in PDF format
-rd, -redact string[] redact given list of keys from query parameter, request header and body
-nrd, -no-redact disable the built-in redaction of credential headers (Authorization, Cookie, Set-Cookie, ...) in output; keys given with -redact are still redacted
CONFIGURATIONS:
-config string path to the nuclei configuration file
@@ -380,18 +381,8 @@ CLOUD:
-pdu, -dashboard-upload string upload / view nuclei results file (jsonl) in projectdiscovery cloud (pdcp) UI dashboard
AUTHENTICATION:
-sf, -secret-file string[] path to config file containing secrets for nuclei authenticated scan
-ps, -prefetch-secrets prefetch secrets from the secrets file
-alu, -auth-login-url string login page url for turnkey auto-login authenticated scan
-au, -auth-username string username for auto-login (-auth-login-url)
-apw, -auth-password string password for auto-login (-auth-login-url); prefer NUCLEI_AUTH_PASSWORD to avoid argv exposure
-auf, -auth-username-field string override the detected username form field name for auto-login
-apf, -auth-password-field string override the detected password form field name for auto-login
-ah, -auth-headless drive auto-login through a real browser (JS/SPA/SSO login pages)
-ar, -auth-recording string path to a chrome devtools recorder json login flow to replay for auto-login
-ac, -auth-capture open a browser to log in manually once, then capture the session for the scan
-arsc, -auth-reauth-status-codes string comma-separated status codes (e.g. 401,403) that trigger an auto-login re-authentication during the scan
-ari, -auth-refresh-interval string re-run the auto-login once the captured session is older than this duration (e.g. 15m)
-sf, -secret-file string[] path to config file containing secrets for nuclei authenticated scan
-ps, -prefetch-secrets prefetch secrets from the secrets file
# NOTE: Headers in secrets files preserve exact casing (useful for case-sensitive APIs)
+5 -14
View File
@@ -148,7 +148,7 @@ TARGET:
-iv, -ip-version string[] IP version to scan of hostname (4,6) - (default 4)
TARGET-FORMAT:
-im, -input-mode string mode of input file (list, burp, jsonl, yaml, openapi, swagger, http) (default "list")
-im, -input-mode string mode of input file (list, burp, har, jsonl, yaml, openapi, swagger, http) (default "list")
-ro, -required-only use only required fields in input format when generating requests
-sfv, -skip-format-validation skip format validation (like missing vars) when parsing input file
-vtt, -vars-text-templating enable text templating for vars in input file (only for yaml input mode)
@@ -175,7 +175,7 @@ TEMPLATES:
-egm, -enable-global-matchers enable loading global matchers templates
-file enable loading file templates
-llm enable llm matchers and extractors (semantic matching)
-llm-provider string llm provider for semantic matching (openai, ollama, llamacpp, vllm, lmstudio, groq, openrouter, together, orcarouter) (default "openai")
-llm-provider string llm provider for semantic matching (openai, ollama, llamacpp, vllm, lmstudio, groq, openrouter, together, orcarouter, jev) (default "openai")
-llm-base-url string openai-compatible endpoint for semantic matching (overrides -llm-provider, e.g. a local model)
-llm-model string model used for semantic matching
-llm-timeout int time in seconds to wait for a single llm call (default 30)
@@ -219,6 +219,7 @@ OUTPUT:
-jle, -jsonl-export string file to export results in JSONL(ine) format
-pe, -pdf-export string file to export results in PDF format
-rd, -redact string[] redact given list of keys from query parameter, request header and body
-nrd, -no-redact disable the built-in redaction of credential headers (Authorization, Cookie, Set-Cookie, ...) in output; keys given with -redact are still redacted
CONFIGURATIONS:
-config string path to the nuclei configuration file
@@ -380,18 +381,8 @@ CLOUD:
-pdu, -dashboard-upload string upload / view nuclei results file (jsonl) in projectdiscovery cloud (pdcp) UI dashboard
AUTHENTICATION:
-sf, -secret-file string[] path to config file containing secrets for nuclei authenticated scan
-ps, -prefetch-secrets prefetch secrets from the secrets file
-alu, -auth-login-url string login page url for turnkey auto-login authenticated scan
-au, -auth-username string username for auto-login (-auth-login-url)
-apw, -auth-password string password for auto-login (-auth-login-url); prefer NUCLEI_AUTH_PASSWORD to avoid argv exposure
-auf, -auth-username-field string override the detected username form field name for auto-login
-apf, -auth-password-field string override the detected password form field name for auto-login
-ah, -auth-headless drive auto-login through a real browser (JS/SPA/SSO login pages)
-ar, -auth-recording string path to a chrome devtools recorder json login flow to replay for auto-login
-ac, -auth-capture open a browser to log in manually once, then capture the session for the scan
-arsc, -auth-reauth-status-codes string comma-separated status codes (e.g. 401,403) that trigger an auto-login re-authentication during the scan
-ari, -auth-refresh-interval string re-run the auto-login once the captured session is older than this duration (e.g. 15m)
-sf, -secret-file string[] path to config file containing secrets for nuclei authenticated scan
-ps, -prefetch-secrets prefetch secrets from the secrets file
# NOTE: Headers in secrets files preserve exact casing (useful for case-sensitive APIs)
+103 -238
View File
@@ -1,255 +1,120 @@
<!-- Generated by securitycontext.dev. Source HEAD: 3e1e0e81f091c7817bbf7af64bb8eec6e8c2bac1. -->
<!-- Generated by securitycontext.dev. Source HEAD: df57ae6d2955864fd92a1bd2565fe9e4e9a7592b. -->
# Security Context: projectdiscovery/nuclei @ dev
> Reference context for editing projectdiscovery/nuclei. Before changing any component below, **preserve the listed guards and do not reintroduce these fixed issues**. Each fingerprint is a regression check, and each shared attack surface is a guard that must keep holding on every path.
_Generated 2026-09-01 16:22 UTC by secc_
_Analyzed commit: 3e1e0e81f091c7817bbf7af64bb8eec6e8c2bac1_
**Remediation coverage: 56.7%**
_17 of 30 historical fixes have no remaining unguarded variant; 368 open remediation leads to verify. Measures fix consistency, not a vulnerability verdict._
- Commits analyzed: **4814**
- Commits flagged for review: **235**
- Confirmed historical security fixes: **32**
- Known published CVEs: **0**
## Summary
Nuclei's historical security posture is dominated by sandbox escapes and code execution via unsigned or unverified templates, with recurring SSRF, path traversal, and auth bypass issues across JS libraries, loaders, and workflow paths. The most critical theme is inconsistent enforcement of signature verification and local-file-access (-lfa) guards across direct, workflow, DAST, and JS-sandbox entry points, enabling variant attacks. A second theme is network policy (host allowlist) gaps in LDAP dialing and proxy handling. The security context also includes dependency upgrades (Alpine, go-git) and output file permission hardening, but the highest-value focus is on template trust boundaries and sandbox enforcement parity.
_Analyzed [df57ae6d29][c-df57ae6d29] on `dev` · 5174 commits, 548 security-relevant examined · generated 2026-11-02_
## Recurring Security Patterns
**15.6% of classified fixes repeat a previously fixed pattern** (5 recurring, 27 first-seen). A recurring pattern means this repository fixed a similar issue before — not that the code is currently vulnerable.
- **Code Execution:** [1c440e755a](https://github.com/projectdiscovery/nuclei/commit/1c440e755a97471c56fb0276ee8a8c4132974645) → [9de96e4dda](https://github.com/projectdiscovery/nuclei/commit/9de96e4dda5a03da963b9ae6582f03ea55791a76) — matched on `class:code execution`, `component:pkg/catalog/loader/loader.go`
- **Code Execution via Unsigned Template:** [fa56800fcc](https://github.com/projectdiscovery/nuclei/commit/fa56800fcca3c8c8fe8e8dec28584210ca1527de) → [fac0b7196d](https://github.com/projectdiscovery/nuclei/commit/fac0b7196d2fbca56c8af8ff8e1899c111041a37) — matched on `class:code execution via unsigned template`, `CWE-347`, `sink:loadtemplates`
- **SSRF:** [a167e6c57b](https://github.com/projectdiscovery/nuclei/commit/a167e6c57b80f5dc7be231231b62d7c594ee2770) → [9def9af7e3](https://github.com/projectdiscovery/nuclei/commit/9def9af7e3f7ebcf5cd561c37b623b9999ee7572) — matched on `class:ssrf`, `CWE-918`, `component:pkg/js/libs/ldap/ldap.go`
- **Dependency Vulnerability:** [d93b4a01df](https://github.com/projectdiscovery/nuclei/commit/d93b4a01df9c218a36dd2fccfb1c2c1f96b181fe) → [fcd5c6b111](https://github.com/projectdiscovery/nuclei/commit/fcd5c6b111ddcb746525f88b7d0bf4b4abbc2529) — matched on `class:dependency vulnerability`, `CWE-1104`, `component:dockerfile`
- **Sandbox Escape:** [96049a32fa](https://github.com/projectdiscovery/nuclei/commit/96049a32fa70aef0483d4180d386a57b3c2d973a) → [cf2a98b03d](https://github.com/projectdiscovery/nuclei/commit/cf2a98b03d76327c20fb0183b7f1c849eb692c91) — matched on `class:sandbox escape`, `CWE-284`, `component:v2/internal/runner/processor.go`, `sink:stdlib allmodulenames`, `sink:stdlib getmodulemap`
**7.1% of classified fixes repeat a previously fixed pattern** (1 recurring, 13 first-seen). A recurring pattern means this repository fixed a similar issue before — not that the code is currently vulnerable.
## Top Risks
### 1. Sandbox Escape: `pkg/js/compiler/pool.go, v2/pkg/protocols/javascript/js.go, v2/pkg/protocols/headless/engine/engine.go, v2/internal/runner/processor.go` (high)
Components with the strongest concentration of past fixes for one class.
Multiple sandbox escapes across JS require(), eval/ReadFile, headless browser flags, and tengo imports, indicating weak default sandbox boundaries that require consistent -lfa and safe-module enforcement.
Evidence: 6f2ade6a9b427c284c15a43445f9c7f055e60e5d, c35162c8ef061a96dd74b54c8f022c1a41cf5833, df55f7a2eb591a7949b5c66a9c54db54aa28f450, cf2a98b03d76327c20fb0183b7f1c849eb692c91, 96049a32fa70aef0483d4180d386a57b3c2d973a
### 2. Code Execution via Unsigned Template: `pkg/catalog/loader/loader.go, pkg/templates/workflows.go, pkg/templates/templates.go` (critical)
Unsigned templates can execute code through direct loading, workflows, and DAST branches; signature verification was bypassed via cache poisoning and missing guards on specific paths.
Evidence: 9de96e4dda5a03da963b9ae6582f03ea55791a76, 1c440e755a97471c56fb0276ee8a8c4132974645, fac0b7196d2fbca56c8af8ff8e1899c111041a37, e86f38299765b82ad724fdb701557e0eaff3884d, fa56800fcca3c8c8fe8e8dec28584210ca1527de
### 3. Auth Bypass: `pkg/templates/compile.go, pkg/templates/signer/tmpl_signer.go, pkg/catalog/loader/loader.go, pkg/protocols/http/request.go` (high)
Signature verification can be bypassed via cached metadata, improper signature extraction, or missing checks on specific load paths, and cookie jar bypass via disabled cookie handling.
Evidence: 4534e9cb3034ebe5cc7b0cecabe399b6fb9ce0e4, 0da993afe6d41b4b1b814e8fad23a2acba13c60a, 9bd4db3f74aedd067af568df4f9198a8a090f68b, a976ca8d3f6a18c9ca746c8ce73f28fdcd518642
### 4. SSRF: `pkg/js/libs/ldap/ldap.go, v2/internal/runner/proxy.go` (high)
SSRF via LDAP dialing to internal hosts and via proxy configuration; host allowlist checks were missing or applied to the wrong target (domain vs controller).
Evidence: 9def9af7e3f7ebcf5cd561c37b623b9999ee7572, a167e6c57b80f5dc7be231231b62d7c594ee2770, 0e46d3e0410f89e4af41099f5cd38fa7f54ae54f
### 5. Path Traversal: `pkg/catalog/config/nucleiconfig.go, v2/internal/runner/update.go, v2/pkg/protocols/common/protocolstate/headless.go` (high)
Path traversal via lexical prefix checks, zip-slip in template extraction, and file:// scheme in headless mode; canonicalization and containment checks are required.
Evidence: 20cbf0bd73893c1270582c9778b6b26a5cb11206, f9c214a66f37a00fc29fe64232b690e6285ec59c, d3928e080d819f25306bfbaecc399f03516db5aa
### 6. Code Execution: `pkg/catalog/loader/loader.go, pkg/templates/workflows.go` (high)
Code execution via cache poisoning and unverified code templates in workflows; requires -code flag and verified templates.
Evidence: 9de96e4dda5a03da963b9ae6582f03ea55791a76, e86f38299765b82ad724fdb701557e0eaff3884d
## Shared Attack Surfaces
Capabilities guarded by more than one past fix on different code paths. A guard enforced on one entry point is often missing on a sibling; verify each alternate path before assuming it's covered.
### 1. Template signature verification and unsigned-template opt-in enforcement across all load paths
- Guard that should hold everywhere: signature verification (template.Verified, Verify), DisableUnsignedTemplates flag, -code flag for code templates, cache invalidation on verification status/fingerprint/digest
- Entry points: `pkg/catalog/loader/loader.go (LoadTemplatesWithTags, DAST branch)`, `pkg/templates/workflows.go (loadTemplates, workflowExecutor)`, `pkg/templates/templates.go (loadTemplates)`, `pkg/templates/compile.go (template.Verified)`, `pkg/templates/signer/tmpl_signer.go (Verify)`
- Check: Inspect the DAST and workflow sub-loaders: they must enforce the same DisableUnsignedTemplates and -code checks as the direct loader, and must not rely on cached verification results without re-checking signatures.
- Evidence: 9de96e4dda5a03da963b9ae6582f03ea55791a76, 1c440e755a97471c56fb0276ee8a8c4132974645, fac0b7196d2fbca56c8af8ff8e1899c111041a37, e86f38299765b82ad724fdb701557e0eaff3884d, fa56800fcca3c8c8fe8e8dec28584210ca1527de, 4534e9cb3034ebe5cc7b0cecabe399b6fb9ce0e4, 0da993afe6d41b4b1b814e8fad23a2acba13c60a, 9bd4db3f74aedd067af568df4f9198a8a090f68b
### 2. Local file access (read/write) outside the sandbox, gated by -lfa flag
- Guard that should hold everywhere: -lfa flag enforcement, path canonicalization and containment (sandbox directory), file:// scheme blocking, allowAllFiles stripping, require() registry rebuild
- Entry points: `pkg/js/libs/krbforge/krbforge.go (CreateTicket)`, `pkg/js/libs/mysql/mysql_private.go (sql.Open)`, `pkg/js/compiler/pool.go (require)`, `v2/pkg/protocols/javascript/js.go (ReadFile, eval)`, `v2/pkg/protocols/common/protocolstate/headless.go (file:)`, `v2/pkg/protocols/common/generators/load.go (loadPayloadsFromFile)`
- Check: Verify that every JS library and protocol helper (krbforge, mysql, require, headless file://, payload loaders) enforces the -lfa flag and path containment; a sibling path that skips the -lfa check is a sandbox escape.
- Evidence: 3a8b65cb45f121fcdf418f60e5177ed9f250c122, 9611926a9a1ca99e8ed554d1b26fbeff60e06368, 6f2ade6a9b427c284c15a43445f9c7f055e60e5d, c35162c8ef061a96dd74b54c8f022c1a41cf5833, d3928e080d819f25306bfbaecc399f03516db5aa, e7fb40a41308f6c41267e858d1781eb67f7e0572
### 3. Network policy (host allowlist) enforcement before outbound connections
- Guard that should hold everywhere: IsHostAllowed check on the actual target host (controller, not domain), proxy validation and connectivity check, ProxyInternal gate for HTTP_PROXY env
- Entry points: `pkg/js/libs/ldap/ldap.go (Dial, newLdapSession)`, `v2/internal/runner/proxy.go (loadProxyServers, os.Setenv HTTP_PROXY)`
- Check: Inspect other dialing paths (e.g., HTTP, headless) to ensure they apply the same host allowlist check on the resolved IP/host, not just a user-supplied domain, and that proxy settings cannot redirect traffic to internal hosts.
- Evidence: 9def9af7e3f7ebcf5cd561c37b623b9999ee7572, a167e6c57b80f5dc7be231231b62d7c594ee2770, 0e46d3e0410f89e4af41099f5cd38fa7f54ae54f, 3f5c055aca1499642caf4f40e8cf55ad32bd4da9
### 4. Path traversal in file extraction and template path trust boundaries
- Guard that should hold everywhere: canonical path containment check (filepath.Rel, prefix), zip-slip validation against template directory, sandbox path prefix validation
- Entry points: `pkg/catalog/config/nucleiconfig.go (IsCustomTemplate)`, `v2/internal/runner/update.go (os.OpenFile)`, `v2/pkg/protocols/common/generators/load.go (loadPayloadsFromFile)`
- Check: Audit all file-write and file-read paths that accept template-controlled paths (e.g., template extraction, payload loading) to ensure they use canonical containment, not lexical prefix checks, and apply the same sandbox directory restriction.
- Evidence: 20cbf0bd73893c1270582c9778b6b26a5cb11206, f9c214a66f37a00fc29fe64232b690e6285ec59c, e7fb40a41308f6c41267e858d1781eb67f7e0572
## Historical Vulnerability Classes
| Class | Count |
|---|---|
| Sandbox Escape | 7 |
| Auth Bypass | 4 |
| Code Execution | 3 |
| Dependency Vulnerability | 3 |
| Path Traversal | 3 |
| SSRF | 3 |
| Code Execution via Unsigned Template | 2 |
| Arbitrary File Read | 1 |
| Cryptographic Weakness | 1 |
| Deserialization of Untrusted Data | 1 |
| Information Disclosure | 1 |
| Insecure File Permissions | 1 |
| Race Condition | 1 |
| Security Misconfiguration | 1 |
| # | Class | Component | Severity | Fixes | Evidence |
|---|---|---|---|---|---|
| 1 | SSRF | `pkg/protocols/common/protocolstate/headless.go` | high | 2 | [d43c1f547f][c-d43c1f547f], [d9dabbb4a1][c-d9dabbb4a1] |
| 2 | Arbitrary File Access | `v2/pkg/protocols/common/generators/load.go` | high | 0 (1 component gone, not counted) | [e5154d362a][c-e5154d362a] |
| 3 | Arbitrary File Access | `pkg/js/libs/mysql/mysql.go` | high | 1 | [634cecbb28][c-634cecbb28] |
| 4 | Code Execution | `pkg/fuzz/execute.go` | high | 1 | [ccbfb12bd0][c-ccbfb12bd0] |
| 5 | Code Execution | `pkg/protocols/common/expressions/expressions.go` | high | 1 | [d221732016][c-d221732016] |
| 6 | Information Disclosure | `internal/httpapi/apiendpoint.go` | high | 1 | [a20aa26024][c-a20aa26024] |
## Fixed-Issue Fingerprints
These are real security fixes mined from git history. Use them as regression checks.
Real security fixes mined from git history; use them as regression checks. `verified` fixes are linked to a published advisory, `inferred` fixes were confirmed from the diff evidence alone. Each is identified by its short commit SHA, which the variant-leads report uses to reference it back to this history.
- **Auth Bypass** (high) [CWE-287]: Honors disable-cookie by preventing the shared cookie jar from being applied when cookie handling is disabled, closing a path where cookies could be reused across requests.
- sink: `SetCookieJar(input.CookieJar)`
- fix: Adds !request.DisableCookie check before applying shared cookie jar
- commit: https://github.com/projectdiscovery/nuclei/commit/a976ca8d3f6a18c9ca746c8ce73f28fdcd518642 (2026-08-25)
- **Code Execution** (high) [CWE-347]: This commit fixes a cache poisoning vulnerability where unsigned templates could bypass signature verification and execute malicious code during compilation or request execution.
- sink: `template compilation and execution of JavaScript`
- fix: cache invalidation based on verification status, verifier fingerprint, and content digest
- commit: https://github.com/projectdiscovery/nuclei/commit/9de96e4dda5a03da963b9ae6582f03ea55791a76 (2026-08-04)
- **Code Execution via Unsigned Template** (high) [CWE-347]: Enforces signature verification on JavaScript templates, preventing execution of unsigned templates that could lead to arbitrary code execution.
- sink: `loading unsigned javascript templates into execution paths`
- fix: Require signature verification for javascript templates in direct loading and workflows when signing is enforced
- commit: https://github.com/projectdiscovery/nuclei/commit/fac0b7196d2fbca56c8af8ff8e1899c111041a37 (2026-07-03)
- **Sandbox Escape** (high) [CWE-22]: Enforces nuclei's local file access policy on krbforge output paths, preventing arbitrary file writes outside the sandbox.
- sink: `gpkrb.CreateTicket(cfg) with cfg.OutputFile`
- fix: Path normalization into sandbox and denial of outside paths unless -lfa
- commit: https://github.com/projectdiscovery/nuclei/commit/3a8b65cb45f121fcdf418f60e5177ed9f250c122 (2026-06-24)
- **SSRF** (high) [CWE-918]: Adds network policy host allowlist checks before LDAP dialing, preventing SSRF to restricted internal hosts.
- sink: `dialers.Fastdialer.Dial`
- fix: host allowlist check via IsHostAllowed before dialing
- commit: https://github.com/projectdiscovery/nuclei/commit/9def9af7e3f7ebcf5cd561c37b623b9999ee7572 (2026-06-24)
- **Arbitrary File Read** (high) [CWE-22]: Gates MySQL's allowAllFiles option behind the -lfa flag, preventing arbitrary local file reads via LOAD DATA LOCAL INFILE.
- sink: `sql.Open("mysql", dsn) with AllowAllFiles=true`
- fix: DSN sandbox that strips allowAllFiles unless -lfa is set
- commit: https://github.com/projectdiscovery/nuclei/commit/9611926a9a1ca99e8ed554d1b26fbeff60e06368 (2026-06-19)
- **Code Execution** (critical) [CWE-94]: Moves the unsigned code-template guard ahead of the DAST branch, preventing unsigned code templates from being loaded and executed under DAST, which previously bypassed the signature check.
- sink: `Template loading and execution`
- fix: Hoisting unsigned code-template guard before DAST branch to ensure it applies on all load paths
- commit: https://github.com/projectdiscovery/nuclei/commit/1c440e755a97471c56fb0276ee8a8c4132974645 (2026-06-19)
- **Path Traversal** (high) [CWE-22]: This commit replaces lexical prefix checks with canonical path containment checks across multiple filesystem trust boundaries, preventing path traversal where sibling paths were incorrectly treated as trusted.
- sink: `IsCustomTemplate`
- fix: path canonicalization and containment check
- commit: https://github.com/projectdiscovery/nuclei/commit/20cbf0bd73893c1270582c9778b6b26a5cb11206 (2026-06-04)
- **Information Disclosure** (medium) [CWE-200]: This commit fixes a data leakage issue in workflows where sibling state could leak across concurrent executions, and race requests ignored dynamic values, by syncing context and cloning inputs.
- sink: `syncWorkflowInputToTemplateCtx`
- fix: context synchronization and input cloning
- commit: https://github.com/projectdiscovery/nuclei/commit/805bcfe76f3629dbe71021795d4ce865406c1555 (2026-05-08)
- **Sandbox Escape** (high) [CWE-284]: Fixes sandbox escape where require() bypassed local file access restrictions and reused privileged module cache across executions.
- sink: `require() using default host filesystem loader`
- fix: Rebuild require registry per execution and enforce allow-local-file-access sandbox rules
- commit: https://github.com/projectdiscovery/nuclei/commit/6f2ade6a9b427c284c15a43445f9c7f055e60e5d (2026-04-10)
- **Cryptographic Weakness** (medium) [CWE-338]: Replaces predictable math/rand PRNG with cryptographically secure crypto/rand in JS helper functions.
- sink: `rand.Intn(255) / rand.Int63()`
- fix: Replace predictable math/rand with crypto/rand
- commit: https://github.com/projectdiscovery/nuclei/commit/44ef9ce34ffcfdfca3a96626abc73d99dd0e83ab (2026-03-15)
- **Auth Bypass** (high) [CWE-347]: Caching signature verification results without cryptographic re-check allows a metadata index compromise to bypass template signature verification.
- sink: `template.Verified, template.TemplateVerifier`
- fix: cached verification data reuse
- commit: https://github.com/projectdiscovery/nuclei/commit/4534e9cb3034ebe5cc7b0cecabe399b6fb9ce0e4 (2026-01-21)
- **Race Condition** (medium) [CWE-362]: Fixes a race condition in workflow execution caused by shared context callbacks, which could lead to data corruption or crashes; the primary impact is reliability, not a direct security breach.
- sink: `shared context callbacks in workflow execution`
- fix: fix race condition by making workflow execution sequential
- commit: https://github.com/projectdiscovery/nuclei/commit/24d1f58682027e23cb21a299a170abd2bdf20dc2 (2025-12-05)
- **Auth Bypass** (high) [CWE-347]: Fixes template signature verification by centralizing signature extraction and requiring the signature to be at the end of the template, preventing bypass.
- sink: `Verify`
- fix: signature extraction and verification
- commit: https://github.com/projectdiscovery/nuclei/commit/0da993afe6d41b4b1b814e8fad23a2acba13c60a (2024-08-19)
- **Dependency Vulnerability** (medium) [CWE-1104]: Upgrades Alpine base image from 3.18.5 to 3.18.6 to fix known security vulnerabilities in bundled packages.
- sink: `FROM alpine:3.18.5`
- fix: Upgrade base image to patched version
- commit: https://github.com/projectdiscovery/nuclei/commit/fcd5c6b111ddcb746525f88b7d0bf4b4abbc2529 (2024-03-15)
- **Dependency Vulnerability** (medium) [CWE-1104]: Upgrades Alpine base image from 3.18.2 to 3.18.5 to fix known security vulnerabilities in bundled packages.
- sink: `FROM alpine:3.18.2`
- fix: Upgrade base image to patched version
- commit: https://github.com/projectdiscovery/nuclei/commit/d93b4a01df9c218a36dd2fccfb1c2c1f96b181fe (2024-03-15)
- **Dependency Vulnerability** (high) [CWE-1035]: Updates go-git to fix known vulnerabilities (CVE-2023-49569, CVE-2023-49568), which could be exploited when processing untrusted git data.
- sink: `go-git library usage`
- fix: dependency upgrade to patched version
- commit: https://github.com/projectdiscovery/nuclei/commit/1700cdf4e23128a358066364062246b58c1c013f (2024-03-14)
- **Code Execution via Unsigned Template** (high) [CWE-347]: Introduces a flag to disable unsigned templates, preventing execution of unverified templates that could lead to arbitrary code execution.
- sink: `loading unsigned templates when -dut flag is set`
- fix: Adds -disable-unsigned-templates flag to skip unsigned templates and report them separately
- commit: https://github.com/projectdiscovery/nuclei/commit/fa56800fcca3c8c8fe8e8dec28584210ca1527de (2024-03-14)
- **Auth Bypass** (medium) [CWE-347]: This commit introduces a flag to skip unsigned templates, closing a path where an attacker could execute untrusted templates by bypassing signature verification.
- sink: `parsed.Verified`
- fix: skip unverified templates when DisableUnsignedTemplates is set
- commit: https://github.com/projectdiscovery/nuclei/commit/9bd4db3f74aedd067af568df4f9198a8a090f68b (2024-03-10)
- **Code Execution** (high) [CWE-94]: Prevents execution of unverified code templates in workflows, mitigating code execution risk.
- sink: `template.RequestsCode`
- fix: require -code flag and verified templates for code templates in workflows
- commit: https://github.com/projectdiscovery/nuclei/commit/e86f38299765b82ad724fdb701557e0eaff3884d (2024-03-01)
- **SSRF** (high) [CWE-918]: Fixes SSRF by validating the actual LDAP controller host against the network policy instead of the domain name.
- sink: `newLdapSession`
- fix: host allowlist check on controller instead of domain
- commit: https://github.com/projectdiscovery/nuclei/commit/a167e6c57b80f5dc7be231231b62d7c594ee2770 (2024-01-20)
- **Sandbox Escape** (high) [CWE-94]: Disables eval and tightens file read validation in the JavaScript sandbox, preventing template-driven code execution and file access outside the sandbox.
- sink: `eval function and file read in sandbox mode`
- fix: Disable eval function and rewrite file validation in sandbox mode
- commit: https://github.com/projectdiscovery/nuclei/commit/c35162c8ef061a96dd74b54c8f022c1a41cf5833 (2023-10-13)
- **Path Traversal** (high) [CWE-22]: Adds network policy and blocks file:// protocol in headless mode unless -lfa is set, preventing local file disclosure.
- sink: `file:// protocol access in headless browser`
- fix: Network policy validation and file:// scheme blocking when local file access is disabled
- commit: https://github.com/projectdiscovery/nuclei/commit/d3928e080d819f25306bfbaecc399f03516db5aa (2023-08-25)
- **Deserialization of Untrusted Data** (medium) [CWE-502]: Fixes JSON deserialization by adding explicit UnmarshalJSON methods and correcting tags, preventing type confusion and potential logic bypasses.
- sink: `UnmarshalJSON on user-controlled fields like Type and Name`
- fix: Adds proper UnmarshalJSON implementations and corrects json tags to prevent type confusion
- commit: https://github.com/projectdiscovery/nuclei/commit/85090b75317977bb9301444c9dfd8e4c2dba667a (2023-02-07)
- **Sandbox Escape** (medium) [CWE-22]: The commit adds sandbox mode that restricts payload file paths to the template directory, preventing arbitrary file reads when sandbox is enabled.
- sink: `loadPayloadsFromFile`
- fix: path prefix validation against template directory and template path directory
- commit: https://github.com/projectdiscovery/nuclei/commit/e7fb40a41308f6c41267e858d1781eb67f7e0572 (2022-11-16)
- **Insecure File Permissions** (low) [CWE-732]: Fixes overly permissive file permissions on debug output files by changing from os.ModePerm (0777) to 0644.
- sink: `os.OpenFile`
- fix: restrict file mode to 0644
- commit: https://github.com/projectdiscovery/nuclei/commit/af4854f90de728bb08689d5cae3dec4acfdb6b9a (2022-06-20)
- **Security Misconfiguration** (medium) [CWE-16]: Prevents setting HTTP_PROXY environment variable unless explicitly enabled, avoiding potential proxy hijacking.
- sink: `os.Setenv(types.HTTP_PROXY_ENV, proxyURL.String())`
- fix: Added check for options.ProxyInternal before setting HTTP_PROXY environment variable
- commit: https://github.com/projectdiscovery/nuclei/commit/3f5c055aca1499642caf4f40e8cf55ad32bd4da9 (2022-03-09)
- **Sandbox Escape** (high) [CWE-250]: The commit restricts the use of the --no-sandbox flag to Linux root users, preventing sandbox escape vulnerabilities in other environments.
- sink: `NoSandbox(true)`
- fix: Conditionally disable sandbox only on Linux as root
- commit: https://github.com/projectdiscovery/nuclei/commit/df55f7a2eb591a7949b5c66a9c54db54aa28f450 (2021-12-05)
- **Path Traversal** (high) [CWE-22]: This commit fixes a zip-slip path traversal vulnerability by validating the extracted file path against the configured template directory.
- sink: `os.OpenFile`
- fix: path canonicalization and prefix check
- commit: https://github.com/projectdiscovery/nuclei/commit/f9c214a66f37a00fc29fe64232b690e6285ec59c (2021-11-25)
- **SSRF** (high) [CWE-918]: Adds validation and connectivity checks for SOCKS5 proxies before launching a scan, preventing SSRF through proxy configuration.
- sink: `Proxy connection validation and loading`
- fix: Proxy server validation and connectivity check before scan launch
- commit: https://github.com/projectdiscovery/nuclei/commit/0e46d3e0410f89e4af41099f5cd38fa7f54ae54f (2021-11-10)
- **Sandbox Escape** (high) [CWE-284]: This commit adds a sandbox flag that, when enabled, restricts the tengo scripting engine to safe modules, preventing sandbox escape and arbitrary command execution.
- sink: `stdlib.GetModuleMap(stdlib.AllModuleNames()...)`
- fix: Added sandbox flag to restrict imports to safe modules when sandbox is enabled
- commit: https://github.com/projectdiscovery/nuclei/commit/cf2a98b03d76327c20fb0183b7f1c849eb692c91 (2020-12-13)
- **Sandbox Escape** (high) [CWE-284]: This commit restricts the tengo scripting engine to a safe module list, preventing sandbox escape and arbitrary command execution via workflow templates.
- sink: `stdlib.GetModuleMap(stdlib.AllModuleNames()...)`
- fix: Restricted tengo script imports to a safe module list
- commit: https://github.com/projectdiscovery/nuclei/commit/96049a32fa70aef0483d4180d386a57b3c2d973a (2020-12-13)
**By component** (directories with the most fixes):
- `pkg/protocols/common/protocolstate` — 2 fixes: SSRF (2)
- [1c0f77d28e][c-1c0f77d28e] **fix(js/net): enforce network policy on proxied target in JS net dial** — SSRF, high [CWE-918] · inferred · 2026-10-18
`pkg/js/libs/net/net.go` · sink `dialHTTPProxy` · fix: input validation
- [d43c1f547f][c-d43c1f547f] **fix(headless): canonicalize IPv4-mapped IPv6 literals and re-validate dial targets** — SSRF, high [CWE-918] · inferred · 2026-10-13
`pkg/protocols/common/protocolstate/headless.go` · sink `net.DialTimeout` · fix: canonicalization before validation
- [d9dabbb4a1][c-d9dabbb4a1] **fix(headless): enforce network policy on ws and wss schemes (#7867)** — SSRF, high [CWE-918] · inferred · 2026-10-12
`pkg/protocols/common/protocolstate/headless.go` · sink `isValidHost` · fix: canonicalization before validation
- [a20aa26024][c-a20aa26024] **Fix pprof/auth exposure on http-api-endpoint and harden file permissions** — Information Disclosure, high [CWE-200] · inferred · 2026-08-03
`internal/httpapi/apiendpoint.go` · sink `http.ListenAndServe` · fix: authentication verification
- [fac0b7196d][c-fac0b7196d] **feat(templates): require signatures for javascript templates (#7514)** — Signature Verification Bypass, high [CWE-347] · verified (CVE-2026-76819, GHSA-vxg7-f2jj-jmqm) · 2026-07-03
`pkg/catalog/loader/loader.go` · sink `t.HasJavascriptRequest` · fix: cryptographic verification
- [ccbfb12bd0][c-ccbfb12bd0] **refactor(protocols): template rendering boundary (#7499)** — Code Execution, high [CWE-94] · verified (CVE-2026-76805, GHSA-jpvm-9frm-hjcq) · 2026-06-30
`pkg/fuzz/execute.go` · sink `expressions.Evaluate` · fix: sandbox isolation
- [3a8b65cb45][c-3a8b65cb45] **fix(js): reject unsandboxed ccache writes in krbforge (#7482)** — Sandbox Escape, high [CWE-693] · inferred · 2026-06-24
`pkg/js/libs/krbforge/krbforge.go` · sink `OutputFile` · fix: sandbox isolation
- [68daef44a7][c-68daef44a7] **fix(js): escape dbname in lib/pq URL (#7479)** — SQL Injection, high [CWE-89] · inferred · 2026-06-23
`pkg/js/libs/postgres/postgres.go` · sink `fmt.Sprintf` (library call — the operation around it is the sink) · fix: context-aware output escaping
- [634cecbb28][c-634cecbb28] **gate mysql allowAllFiles behind -lfa** — Arbitrary File Access, high [CWE-22] · verified (CVE-2026-76803, GHSA-xhmx-w2j4-rw3q) · 2026-06-19
`pkg/js/libs/mysql/mysql.go` · sink `sql.Open` · fix: sandbox isolation
- [20cbf0bd73][c-20cbf0bd73] **refactor: use path-aware filesystem containment checks (#7420)** — Path Traversal, high [CWE-22] · inferred · 2026-06-04
`pkg/installer/template.go` · sink `filepath.Join` · fix: canonicalization before validation
- [6f2ade6a9b][c-6f2ade6a9b] **fix(js): respect \`allow-local-file-access\` in \`require\` (#7332)** — Sandbox Escape, high [CWE-693] · inferred · 2026-04-10
`pkg/js/compiler/pool.go` · sink `require.DefaultSourceLoader` · fix: sandbox isolation
- [d221732016][c-d221732016] **fix(expressions): only eval template-authored expressions (#7221)** — Code Execution, high [CWE-94] · verified (CVE-2026-76805, GHSA-jpvm-9frm-hjcq) · 2026-03-16
`pkg/protocols/common/expressions/expressions.go` · sink `FindExpressions` · fix: allowlist enforcement
- [0da993afe6][c-0da993afe6] **Merge commit from fork** — Signature Verification Bypass, medium [CWE-347] · verified (CVE-2024-43405, GHSA-7h5p-mmpp-hgmm) · 2024-08-19
`pkg/templates/signer/tmpl_signer.go` · sink `os.ReadFile` · fix: cryptographic verification
- [e5154d362a][c-e5154d362a] **fixing payload load (#3927)** — Arbitrary File Access, high [CWE-22] · verified (CVE-2023-37896, GHSA-2xx4-jj5v-6mff) · 2023-07-14
`v2/pkg/protocols/common/generators/load.go` · sink `loadPayloads` · fix: sandbox isolation
## Historical Vulnerability Classes
| Class | Fixes | Verified | Most recent fix |
|---|---|---|---|
| SSRF | 3 | 0 | [2026-10-18][c-1c0f77d28e] 1c0f77d28e |
| Arbitrary File Access | 1 (1 component gone, not counted) | 2 | [2026-06-19][c-634cecbb28] 634cecbb28 |
| Code Execution | 2 | 2 | [2026-06-30][c-ccbfb12bd0] ccbfb12bd0 |
| Sandbox Escape | 2 | 0 | [2026-06-24][c-3a8b65cb45] 3a8b65cb45 |
| Signature Verification Bypass | 2 | 2 | [2026-07-03][c-fac0b7196d] fac0b7196d |
| Information Disclosure | 1 | 0 | [2026-08-03][c-a20aa26024] a20aa26024 |
| Path Traversal | 1 | 0 | [2026-06-04][c-20cbf0bd73] 20cbf0bd73 |
| SQL Injection | 1 | 0 | [2026-06-23][c-68daef44a7] 68daef44a7 |
## Known Advisories
### Linked to a fix commit in this history (5)
- **CVE-2026-76805** / GHSA-jpvm-9frm-hjcq (medium 5.3) CWE-200 — fixed in [ccbfb12bd0][c-ccbfb12bd0], [d221732016][c-d221732016]
- Environment Variable Disclosure via Response-Derived Data in DAST/Fuzz Mode. A vulnerability in Nuclei's DAST/fuzz expression evaluation path allows a malicious target server to trigger disclosure of ...
- **CVE-2026-76803** / GHSA-xhmx-w2j4-rw3q (medium 5.3) CWE-284 — fixed in [634cecbb28][c-634cecbb28]
- Local File Read via MySQL Client Sandbox Bypass. A vulnerability in Nuclei's JavaScript MySQL client library allows arbitrary local file reads that bypass the `-allow-local-file-access` (`-lfa`) sandb...
- **CVE-2026-76819** / GHSA-vxg7-f2jj-jmqm (high 8.6) CWE-94,CWE-787 — fixed in [fac0b7196d][c-fac0b7196d]
- Arbitrary Code Execution via Goja JavaScript Engine Vulnerability. A vulnerability in the Goja JavaScript engine used by Nuclei's `javascript:` protocol allows arbitrary native code execution on the s...
- **CVE-2024-43405** / GHSA-7h5p-mmpp-hgmm (high 7.4) CWE-78 — fixed in [0da993afe6][c-0da993afe6]
- Nuclei Template Signature Verification Bypass. ## Summary A vulnerability has been identified in Nuclei's template signature verification system that could allow an attacker to bypass the signature ch...
- **CVE-2023-37896** / GHSA-2xx4-jj5v-6mff (medium 5.9) CWE-22 — fixed in [e5154d362a][c-e5154d362a]
- Nuclei < 2.9.9 - Sandbox restriction bypass. ## Overview Affected: < 2.9.9 (patched 2.9.9).
### Without a linked fix commit (6)
Published elsewhere, fixed before the history window, or not yet linked; full descriptions are in the JSON context.
| Advisory | Severity | Published | CWE | Summary |
|---|---|---|---|---|
| CVE-2026-76804 / GHSA-qgw5-7j4f-fg97 | medium 5.5 | 2026-07-16 | CWE-284 | Local File Read via Workflow File-Protocol Gate Bypass. A vulnerability in Nuclei's workfl... |
| CVE-2026-76802 / GHSA-jpf4-98qj-qr67 | medium 4.7 | 2026-07-16 | CWE-78 | Arbitrary Command Execution via DAST Code Signature Bypass. A vulnerability in Nuclei's DA... |
| CVE-2026-41645 / GHSA-jm34-66cf-qpvr | medium 5.3 | 2026-04-18 | CWE-94 | Environment variable disclosure via Response-Derived DSL Expressions. A vulnerability in N... |
| CVE-2026-41646 / GHSA-29rg-wmcw-hpf4 | medium 5.5 | 2026-04-18 | CWE-284 | Local File Read via require() Module Loader Bypass. A vulnerability in Nuclei's JavaScript... |
| CVE-2024-40641 / GHSA-c3q9-c27p-cw9h | high 7.4 | 2024-07-17 | CWE-78 | Unsigned code template execution through workflows in projectdiscovery/nuclei. ### Summary... |
| CVE-2024-27920 / GHSA-w5wx-6g2r-r78q | high 7.4 | 2024-03-15 | CWE-78 | Unsigned code template execution through workflows in projectdiscovery/nuclei. ### Overvie... |
[c-df57ae6d29]: https://github.com/projectdiscovery/nuclei/commit/df57ae6d2955864fd92a1bd2565fe9e4e9a7592b
[c-d43c1f547f]: https://github.com/projectdiscovery/nuclei/commit/d43c1f547f98ac56654f4335ad20af0cca4c229c
[c-d9dabbb4a1]: https://github.com/projectdiscovery/nuclei/commit/d9dabbb4a19e897a6bc12334a030a43035b27b73
[c-e5154d362a]: https://github.com/projectdiscovery/nuclei/commit/e5154d362af900e83b30fb79028bfab1a7f0c648
[c-634cecbb28]: https://github.com/projectdiscovery/nuclei/commit/634cecbb289db494247c04a9c4ec6a113aef3cc0
[c-ccbfb12bd0]: https://github.com/projectdiscovery/nuclei/commit/ccbfb12bd01447ba25f6e755dfb2bee677736da6
[c-d221732016]: https://github.com/projectdiscovery/nuclei/commit/d2217320162d5782ca7cb95bef9dda17063818f3
[c-a20aa26024]: https://github.com/projectdiscovery/nuclei/commit/a20aa260245e5db889c5a421d4a65aeb2a64bf46
[c-1c0f77d28e]: https://github.com/projectdiscovery/nuclei/commit/1c0f77d28ed639a34c2036fa23c7503ffac0bf27
[c-fac0b7196d]: https://github.com/projectdiscovery/nuclei/commit/fac0b7196d2fbca56c8af8ff8e1899c111041a37
[c-3a8b65cb45]: https://github.com/projectdiscovery/nuclei/commit/3a8b65cb45f121fcdf418f60e5177ed9f250c122
[c-68daef44a7]: https://github.com/projectdiscovery/nuclei/commit/68daef44a7eafd41c39690924620e418762909ef
[c-20cbf0bd73]: https://github.com/projectdiscovery/nuclei/commit/20cbf0bd73893c1270582c9778b6b26a5cb11206
[c-6f2ade6a9b]: https://github.com/projectdiscovery/nuclei/commit/6f2ade6a9b427c284c15a43445f9c7f055e60e5d
[c-0da993afe6]: https://github.com/projectdiscovery/nuclei/commit/0da993afe6d41b4b1b814e8fad23a2acba13c60a
---
_Fixed-issue fingerprints and CVE matches are derived from real data; LLM-written prose is advisory and should be verified against the cited commits._
_Fixed-issue fingerprints and advisory matches are derived from git history and published advisories; the risk tables and pattern descriptions above are derived from them and should be verified against the cited commits._
+20
View File
@@ -530,6 +530,16 @@ Description of the template.
You can go in-depth here on what the template actually does.
Description, impact and remediation may hold {{placeholders}} filled for
each finding: named extractors, payloads, variables and DSL helpers
(network helpers excluded), and the result fields host, port, ip, url,
scheme, path, matched-at, matcher-name and extractor-name, which win over
values of the same name. Use join(", ", name) for an extractor with
several values, and brackets for a hyphenated name inside a helper:
to_upper([matcher-name]). Unknown placeholders stay as written, and so
does a field whose expression fails. Global matcher templates match on
other templates' responses, so only the result fields fill their text.
Examples:
@@ -543,6 +553,10 @@ description: Bower is a package manager which stores package information in the
description: Subversion ALM for the enterprise before 8.8.2 allows reflected XSS at multiple locations
```
```yaml
description: Redis {{version}} on {{host}}:{{port}} accepts the default password
```
</div>
@@ -559,6 +573,8 @@ Impact of the template.
You can go in-depth here on impact of the template.
Supports the same placeholders as description.
Examples:
@@ -665,6 +681,8 @@ Remediation steps for the template.
You can go in-depth here on how to mitigate the problem found by this template.
Supports the same placeholders as description.
Examples:
@@ -3757,6 +3775,8 @@ Part Definitions:
- <code>req</code> - Headless request made from the client
- <code>duration</code> - Latest measured operation duration in seconds
- <code>resp,body,data</code> - Headless response received from client (default)
- <code>header,all_headers</code> - Headless response headers
- <code>status_code</code> - Status code received from the server
<hr />
+2
View File
@@ -20,6 +20,8 @@ extend-exclude = [
"vendor/",
# Go checksum file
"go.sum",
# Generated history. Commit SHAs are not words.
"SECURITY_CONTEXT.md",
]
[default.extend-identifiers]
+2 -1
View File
@@ -57,7 +57,8 @@
# severity: low
# # duplicate-issue-check (optional) flag to enable duplicate tracking issue check
# duplicate-issue-check: false
# # duplicate-issue-page-size (optional) controls how many issues to fetch per page when searching for duplicates
# # duplicate-issue-page-size (optional) sets the number of issues to request per page when searching for duplicates
# # (GitLab returns at most 100 issues per page)
# duplicate-issue-page-size: 100
# # duplicate-issue-max-pages (optional) limits how many pages to fetch when searching for duplicates (0 = no limit)
# duplicate-issue-max-pages: 0
+5 -13
View File
@@ -271,10 +271,10 @@ on extensive configurability, massive extensibility and ease of use.`)
flagSet.StringSliceVarP(&options.Targets, "target", "u", nil, "target URLs/hosts to scan", goflags.CommaSeparatedStringSliceOptions),
flagSet.StringVarP(&options.TargetsFilePath, "list", "l", "", "path to file containing a list of target URLs/hosts to scan (one per line)"),
flagSet.StringVarP(&options.InlineTargetsList, "targets-inline", "", "", "inline multiline target list (for use in template profiles)"),
flagSet.StringSliceVarP(&options.ExcludeTargets, "exclude-hosts", "eh", nil, "hosts to exclude to scan from the input list (ip, cidr, hostname)", goflags.FileCommaSeparatedStringSliceOptions),
flagSet.StringSliceVarP(&options.ExcludeTargets, "exclude-hosts", "eh", nil, "hosts to exclude to scan from the input list (ip, cidr, hostname)", goflags.FileNormalizedOriginalStringSliceOptions),
flagSet.StringVar(&options.Resume, "resume", "", "resume scan from and save to specified file (clustering will be disabled)"),
flagSet.BoolVarP(&options.ScanAllIPs, "scan-all-ips", "sa", false, "scan all the IP's associated with dns record"),
flagSet.StringSliceVarP(&options.IPVersion, "ip-version", "iv", nil, "IP version to scan of hostname (4,6) - (default 4)", goflags.CommaSeparatedStringSliceOptions),
flagSet.StringSliceVarP(&options.IPVersion, "ip-version", "iv", nil, "IP version to scan of hostname (4,6); default uses dialer fallback, -sa scans all addresses", goflags.CommaSeparatedStringSliceOptions),
flagSet.BoolVarP(&options.DisableURLNormalization, "disable-url-normalization", "dun", false, "keep targets exactly as supplied instead of collapsing urls that address the same resource"),
flagSet.IntVarP(&options.MaxURLsPerPattern, "max-urls-per-pattern", "mup", 0, "scan at most this many urls sharing one structural shape, e.g. /user/1 and /user/2 (0 to scan all)"),
)
@@ -309,7 +309,7 @@ on extensive configurability, massive extensibility and ease of use.`)
flagSet.BoolVarP(&options.EnableGlobalMatchersTemplates, "enable-global-matchers", "egm", false, "enable loading global matchers templates"),
flagSet.BoolVar(&options.EnableFileTemplates, "file", false, "enable loading file templates"),
flagSet.BoolVar(&options.EnableLLM, "llm", false, "enable llm matchers and extractors (semantic matching)"),
flagSet.StringVar(&options.LLMProvider, "llm-provider", "openai", "llm provider for semantic matching (openai, ollama, llamacpp, vllm, lmstudio, groq, openrouter, together, orcarouter)"),
flagSet.StringVar(&options.LLMProvider, "llm-provider", "openai", "llm provider for semantic matching (openai, ollama, llamacpp, vllm, lmstudio, groq, openrouter, together, orcarouter, jev)"),
flagSet.StringVar(&options.LLMBaseURL, "llm-base-url", "", "openai-compatible endpoint for semantic matching (overrides -llm-provider, e.g. a local model)"),
flagSet.StringVar(&options.LLMModel, "llm-model", "", "model used for semantic matching"),
flagSet.IntVar(&options.LLMTimeout, "llm-timeout", 30, "time in seconds to wait for a single llm call"),
@@ -355,6 +355,7 @@ on extensive configurability, massive extensibility and ease of use.`)
flagSet.StringVarP(&options.JSONLExport, "jsonl-export", "jle", "", "file to export results in JSONL(ine) format"),
flagSet.StringVarP(&options.PDFExport, "pdf-export", "pe", "", "file to export results in PDF format"),
flagSet.StringSliceVarP(&options.Redact, "redact", "rd", nil, "redact given list of keys from query parameter, request header and body", goflags.CommaSeparatedStringSliceOptions),
flagSet.BoolVarP(&options.NoRedact, "no-redact", "nrd", false, "disable the built-in redaction of credential headers (Authorization, Cookie, Set-Cookie, ...) in output; keys given with -redact are still redacted"),
)
flagSet.CreateGroup("configs", "Configurations",
@@ -470,6 +471,7 @@ on extensive configurability, massive extensibility and ease of use.`)
flagSet.BoolVar(&options.PreflightPortScan, "preflight-portscan", false, "run preflight resolve + TCP portscan and filter targets before scanning (disabled by default)"),
flagSet.BoolVar(&options.DisableStdin, "no-stdin", false, "disable stdin processing"),
flagSet.BoolVar(&options.EnableHTTPCache, "http-cache", false, "enable HTTP cache (RFC 9111) for HTTP requests"),
flagSet.BoolVarP(&options.AutoCalibrate, "auto-calibrate", "ac", false, "drop HTTP matches on a host's catch-all page (answers missing paths with 2xx/3xx)"),
)
flagSet.CreateGroup("headless", "Headless",
@@ -539,16 +541,6 @@ on extensive configurability, massive extensibility and ease of use.`)
flagSet.CreateGroup("Authentication", "Authentication",
flagSet.StringSliceVarP(&options.SecretsFile, "secret-file", "sf", nil, "path to config file containing secrets for nuclei authenticated scan", goflags.CommaSeparatedStringSliceOptions),
flagSet.BoolVarP(&options.PreFetchSecrets, "prefetch-secrets", "ps", false, "prefetch secrets from the secrets file"),
flagSet.StringVarP(&options.AuthLoginURL, "auth-login-url", "alu", "", "login page url for turnkey auto-login authenticated scan"),
flagSet.StringVarP(&options.AuthUsername, "auth-username", "au", "", "username for auto-login (-auth-login-url)"),
flagSet.StringVarP(&options.AuthPassword, "auth-password", "apw", "", "password for auto-login (-auth-login-url); prefer NUCLEI_AUTH_PASSWORD to avoid argv exposure"),
flagSet.StringVarP(&options.AuthUsernameField, "auth-username-field", "auf", "", "override the detected username form field name for auto-login"),
flagSet.StringVarP(&options.AuthPasswordField, "auth-password-field", "apf", "", "override the detected password form field name for auto-login"),
flagSet.BoolVarP(&options.AuthHeadless, "auth-headless", "ah", false, "drive auto-login through a real browser (JS/SPA/SSO login pages)"),
flagSet.StringVarP(&options.AuthRecording, "auth-recording", "ar", "", "path to a chrome devtools recorder json login flow to replay for auto-login"),
flagSet.BoolVarP(&options.AuthCapture, "auth-capture", "ac", false, "open a browser to log in manually once, then capture the session for the scan"),
flagSet.StringVarP(&options.AuthReauthStatusCodes, "auth-reauth-status-codes", "arsc", "", "comma-separated status codes (e.g. 401,403) that trigger an auto-login re-authentication during the scan"),
flagSet.StringVarP(&options.AuthRefreshInterval, "auth-refresh-interval", "ari", "", "re-run the auto-login once the captured session is older than this duration (e.g. 15m)"),
)
flagSet.SetCustomHelpText(`EXAMPLES:
+12 -12
View File
@@ -18,10 +18,11 @@ require (
github.com/alecthomas/chroma v0.10.0
github.com/alexsnet/go-vnc v0.1.0
github.com/alitto/pond v1.9.2
github.com/andybalholm/brotli v1.2.0
github.com/andygrunwald/go-jira v1.16.1
github.com/antchfx/htmlquery v1.3.5
github.com/antchfx/htmlquery v1.3.7
github.com/antchfx/xmlquery v1.4.4
github.com/antchfx/xpath v1.3.6
github.com/antchfx/xpath v1.3.9
github.com/aws/aws-sdk-go-v2 v1.41.5
github.com/aws/aws-sdk-go-v2/config v1.31.16
github.com/aws/aws-sdk-go-v2/credentials v1.18.20
@@ -33,10 +34,11 @@ require (
github.com/clbanning/mxj/v2 v2.7.0
github.com/ditashi/jsbeautifier-go v0.0.0-20141206144643-2520a8026a9c
github.com/docker/go-units v0.5.0
github.com/dsnet/compress v0.0.2-0.20230904184137-39efe44ab707
github.com/elazarl/goproxy v1.7.2
github.com/fatih/structs v1.1.0
github.com/fullstorydev/grpcurl v1.9.3
github.com/getkin/kin-openapi v0.144.0
github.com/getkin/kin-openapi v0.149.0
github.com/getsops/sops/v3 v3.9.2
github.com/go-echarts/go-echarts/v2 v2.6.0
github.com/go-git/go-git/v5 v5.19.2
@@ -61,6 +63,7 @@ require (
github.com/jhump/protoreflect v1.17.0
github.com/julienschmidt/httprouter v1.3.0
github.com/kitabisa/go-ci v1.0.3
github.com/klauspost/compress v1.18.7
github.com/leslie-qiwa/flat v0.0.0-20230424180412-f9d1cf014baa
github.com/lib/pq v1.12.3
github.com/logrusorgru/aurora/v4 v4.0.0
@@ -75,7 +78,7 @@ require (
github.com/pkg/errors v0.9.1
github.com/praetorian-inc/fingerprintx v1.1.15
github.com/projectdiscovery/clistats v0.1.7
github.com/projectdiscovery/dsl v0.8.24
github.com/projectdiscovery/dsl v0.9.0
github.com/projectdiscovery/fastdialer v0.6.0
github.com/projectdiscovery/fasttemplate v0.0.2
github.com/projectdiscovery/gcache v0.0.0-20241015120333-12546c6e3f4c
@@ -85,7 +88,7 @@ require (
github.com/projectdiscovery/goja_nodejs v0.0.0-20260618132410-8519f75f703d
github.com/projectdiscovery/gologger v1.1.73
github.com/projectdiscovery/gostruct v0.0.2
github.com/projectdiscovery/govaluate v0.0.0-20260615100919-5ee2581bbf7e
github.com/projectdiscovery/govaluate v0.0.0-20261102102118-9294133b15a2
github.com/projectdiscovery/gozero v0.1.1-0.20260530071156-fa1dad563d76
github.com/projectdiscovery/hmap v0.0.102
github.com/projectdiscovery/httpx v1.12.0
@@ -102,8 +105,8 @@ require (
github.com/projectdiscovery/tlsx v1.4.3-0.20261020052910-acb80c2c7c75
github.com/projectdiscovery/uncover v1.2.1
github.com/projectdiscovery/useragent v0.0.109
github.com/projectdiscovery/utils v0.11.12-0.20261025172519-64b3c221cfe5
github.com/projectdiscovery/wappalyzergo v0.3.5
github.com/projectdiscovery/utils v0.11.13-0.20261103094816-5c0b20e35350
github.com/projectdiscovery/wappalyzergo v0.3.6
github.com/projectdiscovery/yamldoc-go v1.0.7
github.com/redis/go-redis/v9 v9.19.0
github.com/remeh/sizedwaitgroup v1.0.0
@@ -113,6 +116,7 @@ require (
github.com/segmentio/ksuid v1.0.4
github.com/shurcooL/graphql v0.0.0-20230722043721-ed46e5a46466
github.com/sijms/go-ora/v2 v2.9.0
github.com/spaolacci/murmur3 v1.1.0
github.com/spf13/cast v1.10.0
github.com/stretchr/testify v1.12.1
github.com/syndtr/goleveldb v1.0.0
@@ -175,7 +179,6 @@ require (
github.com/alecthomas/kingpin/v2 v2.4.0 // indirect
github.com/alecthomas/template v0.0.0-20190718012654-fb15b899a751 // indirect
github.com/alecthomas/units v0.0.0-20211218093645-b94a6e3cc137 // indirect
github.com/andybalholm/brotli v1.2.0 // indirect
github.com/andybalholm/cascadia v1.3.4 // indirect
github.com/asaskevich/govalidator v0.0.0-20230301143203-a9d515a09cc2 // indirect
github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.7.8 // indirect
@@ -239,7 +242,6 @@ require (
github.com/docker/cli v29.2.0+incompatible // indirect
github.com/docker/go-connections v0.7.0 // indirect
github.com/dop251/goja v0.0.0-20260311135729-065cd970411c // indirect
github.com/dsnet/compress v0.0.2-0.20230904184137-39efe44ab707 // indirect
github.com/dustin/go-humanize v1.0.1 // indirect
github.com/ebitengine/purego v0.10.0 // indirect
github.com/emersion/go-sasl v0.0.0-20241020182733-b788ff22d5a6 // indirect
@@ -323,7 +325,6 @@ require (
github.com/kaiakz/ubuffer v0.0.0-20200803053910-dd1083087166 // indirect
github.com/kataras/jwt v0.1.10 // indirect
github.com/kevinburke/ssh_config v1.2.0 // indirect
github.com/klauspost/compress v1.18.7 // indirect
github.com/klauspost/cpuid/v2 v2.3.0 // indirect
github.com/klauspost/pgzip v1.2.6 // indirect
github.com/kylelemons/godebug v1.1.0 // indirect
@@ -385,14 +386,13 @@ require (
github.com/russross/blackfriday/v2 v2.1.0 // indirect
github.com/ryanuber/go-glob v1.0.0 // indirect
github.com/saintfish/chardet v0.0.0-20230101081208-5e3ef4b5456d // indirect
github.com/santhosh-tekuri/jsonschema/v6 v6.0.2 // indirect
github.com/santhosh-tekuri/jsonschema/v6 v6.0.3 // indirect
github.com/sergi/go-diff v1.4.0 // indirect
github.com/shirou/gopsutil v3.21.11+incompatible // indirect
github.com/shirou/gopsutil/v4 v4.26.3 // indirect
github.com/sirupsen/logrus v1.10.2 // indirect
github.com/skeema/knownhosts v1.3.1 // indirect
github.com/sorairolake/lzip-go v0.3.8 // indirect
github.com/spaolacci/murmur3 v1.1.0 // indirect
github.com/spf13/afero v1.15.0 // indirect
github.com/spf13/pflag v1.0.10 // indirect
github.com/spiffe/go-spiffe/v2 v2.7.0 // indirect
+16 -17
View File
@@ -180,14 +180,13 @@ github.com/andygrunwald/go-jira v1.16.1 h1:WoQEar5XoDRAibOgKzTFELlPNlKAtnfWr296R
github.com/andygrunwald/go-jira v1.16.1/go.mod h1:UQH4IBVxIYWbgagc0LF/k9FRs9xjIiQ8hIcC6HfLwFU=
github.com/anmitsu/go-shlex v0.0.0-20200514113438-38f4b401e2be h1:9AeTilPcZAjCFIImctFaOjnTIavg87rW78vTPkQqLI8=
github.com/anmitsu/go-shlex v0.0.0-20200514113438-38f4b401e2be/go.mod h1:ySMOLuWl6zY27l47sB3qLNK6tF2fkHG55UZxx8oIVo4=
github.com/antchfx/htmlquery v1.3.5 h1:aYthDDClnG2a2xePf6tys/UyyM/kRcsFRm+ifhFKoU0=
github.com/antchfx/htmlquery v1.3.5/go.mod h1:5oyIPIa3ovYGtLqMPNjBF2Uf25NPCKsMjCnQ8lvjaoA=
github.com/antchfx/htmlquery v1.3.7 h1:KCUklE1EO/xm6iS3CWx6iatFlcnENBk4I4UYa2pBmLU=
github.com/antchfx/htmlquery v1.3.7/go.mod h1:/uBTMnFEl6LaBlIgw3KZ+c061tAQdlfbCXObZoxwQf4=
github.com/antchfx/xmlquery v1.4.4 h1:mxMEkdYP3pjKSftxss4nUHfjBhnMk4imGoR96FRY2dg=
github.com/antchfx/xmlquery v1.4.4/go.mod h1:AEPEEPYE9GnA2mj5Ur2L5Q5/2PycJ0N9Fusrx9b12fc=
github.com/antchfx/xpath v1.3.3/go.mod h1:i54GszH55fYfBmoZXapTHN8T8tkcHfRgLyVwwqzXNcs=
github.com/antchfx/xpath v1.3.5/go.mod h1:i54GszH55fYfBmoZXapTHN8T8tkcHfRgLyVwwqzXNcs=
github.com/antchfx/xpath v1.3.6 h1:s0y+ElRRtTQdfHP609qFu0+c6bglDv20pqOViQjjdPI=
github.com/antchfx/xpath v1.3.6/go.mod h1:i54GszH55fYfBmoZXapTHN8T8tkcHfRgLyVwwqzXNcs=
github.com/antchfx/xpath v1.3.9 h1:muXJSod6jRzhsiUjvduNSxsRSptFZ2GOJ1oMD1QkPSA=
github.com/antchfx/xpath v1.3.9/go.mod h1:i54GszH55fYfBmoZXapTHN8T8tkcHfRgLyVwwqzXNcs=
github.com/armon/go-socks5 v0.0.0-20160902184237-e75332964ef5 h1:0CwZNZbxp69SHPdPJAN/hZIm0C4OItdklCFmMRWYpio=
github.com/armon/go-socks5 v0.0.0-20160902184237-e75332964ef5/go.mod h1:wHh0iHkYZB8zMSxRWpUBQtwG5a7fFgvEO+odwuTv2gs=
github.com/asaskevich/govalidator v0.0.0-20230301143203-a9d515a09cc2 h1:DklsrG3dyBCFEj5IhUbnKptjxatkF07cF2ak3yi77so=
@@ -427,8 +426,8 @@ github.com/gaissmai/bart v0.30.0 h1:K1oTSQ6Nf76bhrgxeTqzZa4hE0M9x4xDfL6n6UG67Sc=
github.com/gaissmai/bart v0.30.0/go.mod h1:GREWQfTLRWz/c5FTOsIw+KkscuFkIV5t8Rp7Nd1Td5c=
github.com/geoffgarside/ber v1.2.0 h1:/loowoRcs/MWLYmGX9QtIAbA+V/FrnVLsMMPhwiRm64=
github.com/geoffgarside/ber v1.2.0/go.mod h1:jVPKeCbj6MvQZhwLYsGwaGI52oUorHoHKNecGT85ZCc=
github.com/getkin/kin-openapi v0.144.0 h1:hIRcTH+KjLfkLpYU6bSSfdFpi0fZi1fp+hSPi4aQu9Y=
github.com/getkin/kin-openapi v0.144.0/go.mod h1:3BH9M9XDe/y9M5DSvEocVYAYq1w0qrhJHjC/vZi0AaY=
github.com/getkin/kin-openapi v0.149.0 h1:ZbhmVJ4yq5RZDUsyP8lcBcGMsjsaTqXEFt6isdtMDfA=
github.com/getkin/kin-openapi v0.149.0/go.mod h1:1+BHDzstro+P5CKtPy1X4PfofnFgmRe6uvMy9+r9fKY=
github.com/getsops/gopgagent v0.0.0-20240527072608-0c14999532fe h1:QKe/kmAYbndxwu91TcjHERsnMh5SgOB1x/qicvOdUJ8=
github.com/getsops/gopgagent v0.0.0-20240527072608-0c14999532fe/go.mod h1:awFzISqLJoZLm+i9QQ4SgMNHDqljH6jWV0B36V5MrUM=
github.com/getsops/sops/v3 v3.9.2 h1:Cuuahc/UGu7W3+NXd+mrl+2oXGxxj/jIPsw6Ah2CbxA=
@@ -935,8 +934,8 @@ github.com/projectdiscovery/cdncheck v1.3.2 h1:I453ZWNBeG0WWIy3wi9412l4PTzorLgeb
github.com/projectdiscovery/cdncheck v1.3.2/go.mod h1:5IkUMMlcGaeg1DWW5oJVd3r9X3EpJ88j5HxBeF4Me9M=
github.com/projectdiscovery/clistats v0.1.7 h1:NXf4lUzx1QcbJd6QycNwwYHTv2FsW9MchQ2LsgSL6C0=
github.com/projectdiscovery/clistats v0.1.7/go.mod h1:RxRGxhMzkT344VTTvZWIrojurUXdQQRN1yfwgg/88I8=
github.com/projectdiscovery/dsl v0.8.24 h1:rN2w6LSIjA16qKGRjyj70oxdwXGaspe6sAjEkP7F2fk=
github.com/projectdiscovery/dsl v0.8.24/go.mod h1:V+EludvS5PS19UNMlM3CuBwZ+sFnMaQhCFGxysDUanQ=
github.com/projectdiscovery/dsl v0.9.0 h1:i1KyCbKJyKfHkgdbfh0JkWdd14APUjpEYf+GaYANlCg=
github.com/projectdiscovery/dsl v0.9.0/go.mod h1:ntNFkkiYCTEosGFTbApA61RMrvJTCjnakPxnWj56MyY=
github.com/projectdiscovery/fastdialer v0.6.0 h1:4ZeFQ6LyRH4D8e3qO/yzqIP4Cf9Fr0DduaVpJD6ENnE=
github.com/projectdiscovery/fastdialer v0.6.0/go.mod h1:FpHwa9S5PUgfPbh+irOm+cdU6AO631XidxkhPwPv7yY=
github.com/projectdiscovery/fasttemplate v0.0.2 h1:h2cISk5xDhlJEinlBQS6RRx0vOlOirB2y3Yu4PJzpiA=
@@ -957,8 +956,8 @@ github.com/projectdiscovery/gologger v1.1.73 h1:WX2nzq29cL7HdvOHqIttacoid/5lMwap
github.com/projectdiscovery/gologger v1.1.73/go.mod h1:mJwODZcFDg70ihINpOvZevmBtgvpP8H9/l8Y+OPhZPY=
github.com/projectdiscovery/gostruct v0.0.2 h1:s8gP8ApugGM4go1pA+sVlPDXaWqNP5BBDDSv7VEdG1M=
github.com/projectdiscovery/gostruct v0.0.2/go.mod h1:H86peL4HKwMXcQQtEa6lmC8FuD9XFt6gkNR0B/Mu5PE=
github.com/projectdiscovery/govaluate v0.0.0-20260615100919-5ee2581bbf7e h1:vxzgQlz2Cy/YvizYDQx9OhucBcmBotfDhbQ4yCY2vfA=
github.com/projectdiscovery/govaluate v0.0.0-20260615100919-5ee2581bbf7e/go.mod h1:xH7bPwHxUlz1yx9UlVeTF+UVCUaKhTnZgaxHb5z362E=
github.com/projectdiscovery/govaluate v0.0.0-20261102102118-9294133b15a2 h1:aSLbDvrJQPlp4HvojU+bTU5XV8xdAVCjoc5gX1Wfbw4=
github.com/projectdiscovery/govaluate v0.0.0-20261102102118-9294133b15a2/go.mod h1:Z3VjpUJV1pnXDwvAddA8aZ4e839SoeftB7CSdZntAVI=
github.com/projectdiscovery/gozero v0.1.1-0.20260530071156-fa1dad563d76 h1:AN70bbi6BBs7KpIM9w0LxygUN7uzT/oH+owDIQ+Fz/k=
github.com/projectdiscovery/gozero v0.1.1-0.20260530071156-fa1dad563d76/go.mod h1:cWHYnRXoYWHtTpOYyAp5laGYX8GH8ITUhgQaP8G/8FA=
github.com/projectdiscovery/hmap v0.0.102 h1:ybPuf7UQvuOLiHLOUbVMvd+PSlItQU5iQpP1Erl8kaU=
@@ -999,10 +998,10 @@ github.com/projectdiscovery/uncover v1.2.1 h1:8U46T/96CLT7BPoXBgkTvWqB06lOyeTSLv
github.com/projectdiscovery/uncover v1.2.1/go.mod h1:0p8onrWxfpXQEYs90ZDzTSpu1107gWmodX1NWqu/+z4=
github.com/projectdiscovery/useragent v0.0.109 h1:b86BPdZvmgbJousa04jAugDRY4Y+rAvEamE0TFCPmeE=
github.com/projectdiscovery/useragent v0.0.109/go.mod h1:sRyBqDAcD31RKYrO8fyXGowLd36fNwY1g0MNW9mp7zk=
github.com/projectdiscovery/utils v0.11.12-0.20261025172519-64b3c221cfe5 h1:AOspf7ScbwZr8A2YnjCFd4ZJAMSFqzfEaIJIbJPuu10=
github.com/projectdiscovery/utils v0.11.12-0.20261025172519-64b3c221cfe5/go.mod h1:TJgsyVjB02CavwZVFzUzPg6YOSJXQxLhL808JCiQKlU=
github.com/projectdiscovery/wappalyzergo v0.3.5 h1:UZradYF1GIcruSIV5wwo/Coph9/VnM17oajLD9eXR2U=
github.com/projectdiscovery/wappalyzergo v0.3.5/go.mod h1:4E6+fh/ckaY3wo5tlM7lNYSkhuTYRHzu2YmGIRraXP0=
github.com/projectdiscovery/utils v0.11.13-0.20261103094816-5c0b20e35350 h1:uXqpYa6XKV9dCvoti9PbSgIWIsBKPu5YKMywB9G4q0Q=
github.com/projectdiscovery/utils v0.11.13-0.20261103094816-5c0b20e35350/go.mod h1:TJgsyVjB02CavwZVFzUzPg6YOSJXQxLhL808JCiQKlU=
github.com/projectdiscovery/wappalyzergo v0.3.6 h1:1FXCMeCa1xVu2zaW+eeBcbVxtlMStqGOlKcMLyh2pzI=
github.com/projectdiscovery/wappalyzergo v0.3.6/go.mod h1:4E6+fh/ckaY3wo5tlM7lNYSkhuTYRHzu2YmGIRraXP0=
github.com/projectdiscovery/yamldoc-go v1.0.7 h1:Odmjd1o4EkMff2F6QHi1dIYPIQPZUFy24YqmQCprmTk=
github.com/projectdiscovery/yamldoc-go v1.0.7/go.mod h1:nqD1iPMcofpRA1DbXavPByJhWURrshT+l8WN9XjyQag=
github.com/prometheus/client_golang v0.9.1/go.mod h1:7SWBe2y4D6OKWSNQJUaRYU/AaXPKyh/dDVn+NZz0KFw=
@@ -1055,8 +1054,8 @@ github.com/saintfish/chardet v0.0.0-20230101081208-5e3ef4b5456d h1:hrujxIzL1woJ7
github.com/saintfish/chardet v0.0.0-20230101081208-5e3ef4b5456d/go.mod h1:uugorj2VCxiV1x+LzaIdVa9b4S4qGAcH6cbhh4qVxOU=
github.com/sandrolain/httpcache v1.4.0 h1:Jf4Vx62X2ybvNPSpPvI1kT3xvMdDG1AsApQjOQKO9E0=
github.com/sandrolain/httpcache v1.4.0/go.mod h1:kHBuXveitSn39SNPBhdf/ybG272X706HJ2RJqOQ+Em0=
github.com/santhosh-tekuri/jsonschema/v6 v6.0.2 h1:KRzFb2m7YtdldCEkzs6KqmJw4nqEVZGK7IN2kJkjTuQ=
github.com/santhosh-tekuri/jsonschema/v6 v6.0.2/go.mod h1:JXeL+ps8p7/KNMjDQk3TCwPpBy0wYklyWTfbkIzdIFU=
github.com/santhosh-tekuri/jsonschema/v6 v6.0.3 h1:1EYB5IzjZawrrnELUi78f9fPu57HuXjmddZPjrls/28=
github.com/santhosh-tekuri/jsonschema/v6 v6.0.3/go.mod h1:JXeL+ps8p7/KNMjDQk3TCwPpBy0wYklyWTfbkIzdIFU=
github.com/segmentio/ksuid v1.0.4 h1:sBo2BdShXjmcugAMwjugoGUdUV0pcxY5mW4xKRn3v4c=
github.com/segmentio/ksuid v1.0.4/go.mod h1:/XUiZBD3kVx5SmUOl55voK5yeAbBNNIed+2O73XgrPE=
github.com/sergi/go-diff v1.4.0 h1:n/SP9D5ad1fORl+llWyN+D6qoUETXNZARKjyY2/KVCw=
-556
View File
@@ -1,556 +0,0 @@
package fuzzplayground
import (
"crypto/rand"
"encoding/hex"
"encoding/json"
"fmt"
"html"
"net/http"
"strings"
"sync"
)
// This file turns the playground into a small but realistic "modern app" auth
// test bench. It serves a range of login styles seen in the wild — classic
// server-rendered forms, JS/SPA-rendered forms, username-first multi-step flows,
// SPA logins that stash a JWT in web storage, CSRF-protected forms, a
// disabled-until-valid submit button (a real regression class), and an
// OAuth-style redirect dance — plus protected endpoints (whoami) and an
// authenticated, fuzzable endpoint. It lets us exercise the auto-login /
// recording / capture engine and authenticated scanning entirely in-repo,
// without external apps.
//
// Credentials are fixed and shared across every style.
const (
AuthUsername = "[email protected]"
AuthPassword = "nuclei-rocks"
authSessionCookie = "PSESSION"
)
// authStore is a tiny in-memory session/CSRF store for the auth playground.
type authStore struct {
mu sync.Mutex
sessions map[string]string // session token -> username
csrf map[string]struct{}
}
func newAuthStore() *authStore {
// Pre-seed the SPA-token style's fixed session so the JWT it stashes in web
// storage also authenticates protected APIs (bearer path).
return &authStore{
sessions: map[string]string{"spa-static-session": AuthUsername},
csrf: map[string]struct{}{},
}
}
func randToken() string {
b := make([]byte, 16)
_, _ = rand.Read(b)
return hex.EncodeToString(b)
}
// jwtFor returns a JWT-shaped token (eyJ...) so token-regex extraction has a
// realistic target. It is not a real signed JWT; the playground treats the
// session token embedded in it as the source of truth.
func jwtFor(session string) string {
// header.payload.signature where payload carries the opaque session id
return "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9." +
hex.EncodeToString([]byte(session)) + ".c2lnbmF0dXJl"
}
func (s *authStore) issue(username string) (session, jwt string) {
session = randToken()
s.mu.Lock()
s.sessions[session] = username
s.mu.Unlock()
return session, jwtFor(session)
}
func (s *authStore) userFor(session string) (string, bool) {
s.mu.Lock()
defer s.mu.Unlock()
u, ok := s.sessions[session]
return u, ok
}
func (s *authStore) issueCSRF() string {
tok := randToken()
s.mu.Lock()
s.csrf[tok] = struct{}{}
s.mu.Unlock()
return tok
}
func (s *authStore) consumeCSRF(tok string) bool {
s.mu.Lock()
defer s.mu.Unlock()
if _, ok := s.csrf[tok]; ok {
delete(s.csrf, tok)
return true
}
return false
}
// authenticatedUser resolves the caller's identity from either the session
// cookie or a bearer token (the JWT carries the hex-encoded session id).
func (s *authStore) authenticatedUser(r *http.Request) (string, bool) {
if c, err := r.Cookie(authSessionCookie); err == nil && c.Value != "" {
if u, ok := s.userFor(c.Value); ok {
return u, true
}
}
auth := r.Header.Get("Authorization")
if strings.HasPrefix(auth, "Bearer ") {
if sess := sessionFromJWT(strings.TrimPrefix(auth, "Bearer ")); sess != "" {
if u, ok := s.userFor(sess); ok {
return u, true
}
}
}
return "", false
}
// sessionFromJWT extracts the opaque session id embedded in a playground JWT.
func sessionFromJWT(token string) string {
parts := strings.Split(token, ".")
if len(parts) != 3 {
return ""
}
raw, err := hex.DecodeString(parts[1])
if err != nil {
return ""
}
return string(raw)
}
func setSessionCookie(w http.ResponseWriter, session string) {
http.SetCookie(w, &http.Cookie{
Name: authSessionCookie,
Value: session,
Path: "/",
HttpOnly: true,
})
}
// registerAuthRoutes wires every login style and the protected endpoints onto a
// per-server auth store.
func registerAuthRoutes(mux *http.ServeMux) {
st := newAuthStore()
// 1. Classic server-rendered form login (works with the static HTTP engine).
mux.HandleFunc("GET /auth/form-login", func(w http.ResponseWriter, _ *http.Request) {
writeHTML(w, http.StatusOK, formLoginPage("", ""))
})
mux.HandleFunc("POST /auth/form-login", func(w http.ResponseWriter, r *http.Request) {
if r.FormValue("username") == AuthUsername && r.FormValue("password") == AuthPassword {
session, _ := st.issue(AuthUsername)
setSessionCookie(w, session)
http.Redirect(w, r, "/auth/dashboard", http.StatusFound)
return
}
writeHTML(w, http.StatusOK, formLoginPage(r.FormValue("username"), "Invalid credentials"))
})
// 2. CSRF-protected server-rendered form: a hidden token must be echoed back.
mux.HandleFunc("GET /auth/csrf-login", func(w http.ResponseWriter, _ *http.Request) {
writeHTML(w, http.StatusOK, csrfLoginPage(st.issueCSRF(), ""))
})
mux.HandleFunc("POST /auth/csrf-login", func(w http.ResponseWriter, r *http.Request) {
if !st.consumeCSRF(r.FormValue("csrf_token")) {
writeHTML(w, http.StatusForbidden, csrfLoginPage(st.issueCSRF(), "Invalid CSRF token"))
return
}
if r.FormValue("username") == AuthUsername && r.FormValue("password") == AuthPassword {
session, _ := st.issue(AuthUsername)
setSessionCookie(w, session)
http.Redirect(w, r, "/auth/dashboard", http.StatusFound)
return
}
writeHTML(w, http.StatusOK, csrfLoginPage(st.issueCSRF(), "Invalid credentials"))
})
// 3. JS/SPA-rendered form: the raw HTML has no <form>; JS builds it and posts
// via fetch. Only a real browser can log in here.
mux.HandleFunc("GET /auth/spa-login", func(w http.ResponseWriter, _ *http.Request) {
writeHTML(w, http.StatusOK, spaLoginPage)
})
mux.HandleFunc("POST /auth/api/login", func(w http.ResponseWriter, r *http.Request) {
var body struct {
Username string `json:"username"`
Password string `json:"password"`
}
if err := json.NewDecoder(r.Body).Decode(&body); err != nil {
writeJSON(w, http.StatusBadRequest, map[string]string{"error": "bad request"})
return
}
if body.Username == AuthUsername && body.Password == AuthPassword {
session, jwt := st.issue(AuthUsername)
setSessionCookie(w, session)
writeJSON(w, http.StatusOK, map[string]string{"token": jwt})
return
}
writeJSON(w, http.StatusUnauthorized, map[string]string{"error": "invalid credentials"})
})
// 4. Username-first multi-step flow (Google/Microsoft-style): password field
// is revealed only after the "Next" button.
mux.HandleFunc("GET /auth/multistep-login", func(w http.ResponseWriter, _ *http.Request) {
writeHTML(w, http.StatusOK, multiStepLoginPage)
})
// 5. Disabled-until-valid submit button — a real regression class: the submit
// stays disabled until JS marks the form valid, so a naive immediate click
// hangs. The engine must settle/fall back to Enter.
mux.HandleFunc("GET /auth/strict-login", func(w http.ResponseWriter, _ *http.Request) {
writeHTML(w, http.StatusOK, strictLoginPage)
})
// 6. SPA login that stores a JWT in localStorage (no cookie at all).
mux.HandleFunc("GET /auth/spa-token-login", func(w http.ResponseWriter, _ *http.Request) {
writeHTML(w, http.StatusOK, spaTokenLoginPage)
})
// 7. OAuth-style redirect flow: authorize -> (auto consent) -> callback sets
// the session and redirects to the dashboard.
mux.HandleFunc("GET /auth/oauth/authorize", func(w http.ResponseWriter, r *http.Request) {
http.Redirect(w, r, "/auth/oauth/callback?code="+randToken(), http.StatusFound)
})
mux.HandleFunc("GET /auth/oauth/callback", func(w http.ResponseWriter, r *http.Request) {
if r.URL.Query().Get("code") == "" {
writeString(w, http.StatusBadRequest, "missing code")
return
}
session, _ := st.issue(AuthUsername)
setSessionCookie(w, session)
http.Redirect(w, r, "/auth/dashboard", http.StatusFound)
})
// 8. Multi-form page: a decoy search form (no password) precedes the real
// login form. The detector must score forms and pick the credential one
// rather than blindly taking the first <form>.
mux.HandleFunc("GET /auth/multiform-login", func(w http.ResponseWriter, _ *http.Request) {
writeHTML(w, http.StatusOK, multiFormLoginPage)
})
// 9. JS-set cookie: login succeeds via XHR that returns the session id in the
// body (no Set-Cookie), and client JS writes document.cookie. The headless
// engine must capture cookies from the browser jar, not just Set-Cookie.
mux.HandleFunc("GET /auth/jscookie-login", func(w http.ResponseWriter, _ *http.Request) {
writeHTML(w, http.StatusOK, jsCookieLoginPage)
})
mux.HandleFunc("POST /auth/api/login-jsbody", func(w http.ResponseWriter, r *http.Request) {
var body struct {
Username string `json:"username"`
Password string `json:"password"`
}
if err := json.NewDecoder(r.Body).Decode(&body); err != nil {
writeJSON(w, http.StatusBadRequest, map[string]string{"error": "bad request"})
return
}
if body.Username == AuthUsername && body.Password == AuthPassword {
session, _ := st.issue(AuthUsername)
// Deliberately no Set-Cookie: the client sets the cookie via JS.
writeJSON(w, http.StatusOK, map[string]string{"session": session})
return
}
writeJSON(w, http.StatusUnauthorized, map[string]string{"error": "invalid credentials"})
})
// 10. Delayed render: the login form is injected after an async tick, so an
// engine that probes for the password field immediately would miss it and
// must wait for it to appear.
mux.HandleFunc("GET /auth/delayed-login", func(w http.ResponseWriter, _ *http.Request) {
writeHTML(w, http.StatusOK, delayedLoginPage)
})
// 11. Header-token login: a server-rendered form whose POST returns the
// session token in a response header (no cookie), the token-in-header
// API style. The engine must read the token from the header.
mux.HandleFunc("GET /auth/header-token-login", func(w http.ResponseWriter, _ *http.Request) {
writeHTML(w, http.StatusOK, headerTokenLoginPage(""))
})
mux.HandleFunc("POST /auth/header-token-login", func(w http.ResponseWriter, r *http.Request) {
if r.FormValue("username") == AuthUsername && r.FormValue("password") == AuthPassword {
_, jwt := st.issue(AuthUsername)
// Token in a header, deliberately no Set-Cookie.
w.Header().Set("X-Auth-Token", jwt)
writeHTML(w, http.StatusOK, `<html><head><title>Dashboard</title></head><body><h1>Welcome, signed in</h1></body></html>`)
return
}
writeHTML(w, http.StatusOK, headerTokenLoginPage("Invalid credentials"))
})
// 12. SPA whose XHR login returns the token only in a response header (no
// cookie, no body/storage token). Exercises the headless engine's
// passive response-header interception.
mux.HandleFunc("GET /auth/spa-header-token-login", func(w http.ResponseWriter, _ *http.Request) {
writeHTML(w, http.StatusOK, spaHeaderTokenLoginPage)
})
mux.HandleFunc("POST /auth/api/login-header", func(w http.ResponseWriter, r *http.Request) {
var body struct {
Username string `json:"username"`
Password string `json:"password"`
}
if err := json.NewDecoder(r.Body).Decode(&body); err != nil {
writeJSON(w, http.StatusBadRequest, map[string]string{"error": "bad request"})
return
}
if body.Username == AuthUsername && body.Password == AuthPassword {
_, jwt := st.issue(AuthUsername)
w.Header().Set("X-Auth-Token", jwt)
writeJSON(w, http.StatusOK, map[string]bool{"ok": true})
return
}
writeJSON(w, http.StatusUnauthorized, map[string]string{"error": "invalid credentials"})
})
// Protected landing page: the login-success heuristic relies on the final
// page having no password field, which this page satisfies.
mux.HandleFunc("GET /auth/dashboard", func(w http.ResponseWriter, r *http.Request) {
user, ok := st.authenticatedUser(r)
if !ok {
http.Redirect(w, r, "/auth/form-login", http.StatusFound)
return
}
writeHTML(w, http.StatusOK, fmt.Sprintf(`<html><head><title>Dashboard</title></head>
<body><h1>Welcome, %s</h1><a href="/auth/logout">Logout</a></body></html>`, html.EscapeString(user)))
})
// whoami: protected API used to assert a captured session authenticates.
mux.HandleFunc("GET /auth/whoami", func(w http.ResponseWriter, r *http.Request) {
user, ok := st.authenticatedUser(r)
if !ok {
writeJSON(w, http.StatusUnauthorized, map[string]string{"error": "unauthorized"})
return
}
writeJSON(w, http.StatusOK, map[string]string{"user": user})
})
// Authenticated, fuzzable endpoint: only reachable with a valid session, and
// reflects q without sanitization (reflected XSS) so authenticated fuzzing
// has a genuine target behind the login wall.
mux.HandleFunc("GET /auth/api/search", func(w http.ResponseWriter, r *http.Request) {
if _, ok := st.authenticatedUser(r); !ok {
writeJSON(w, http.StatusUnauthorized, map[string]string{"error": "unauthorized"})
return
}
q := r.URL.Query().Get("q")
writeHTML(w, http.StatusOK, fmt.Sprintf("<html><body><div>results for: %s</div></body></html>", q))
})
mux.HandleFunc("GET /auth/logout", func(w http.ResponseWriter, r *http.Request) {
if ck, err := r.Cookie(authSessionCookie); err == nil {
st.mu.Lock()
delete(st.sessions, ck.Value)
st.mu.Unlock()
}
setSessionCookie(w, "")
http.Redirect(w, r, "/auth/form-login", http.StatusFound)
})
}
// --- Login page templates --------------------------------------------------
func formLoginPage(username, errMsg string) string {
var banner string
if errMsg != "" {
banner = fmt.Sprintf(`<p class="error">%s</p>`, html.EscapeString(errMsg))
}
return fmt.Sprintf(`<html><head><title>Login</title></head><body>
<h1>Sign in</h1>%s
<form method="post" action="/auth/form-login">
<input type="text" name="username" placeholder="Email" value="%s">
<input type="password" name="password" placeholder="Password">
<button type="submit">Sign in</button>
</form></body></html>`, banner, html.EscapeString(username))
}
func csrfLoginPage(csrf, errMsg string) string {
var banner string
if errMsg != "" {
banner = fmt.Sprintf(`<p class="error">%s</p>`, html.EscapeString(errMsg))
}
return fmt.Sprintf(`<html><head><title>Login</title></head><body>
<h1>Sign in</h1>%s
<form method="post" action="/auth/csrf-login">
<input type="hidden" name="csrf_token" value="%s">
<input type="text" name="username" placeholder="Email">
<input type="password" name="password" placeholder="Password">
<button type="submit">Sign in</button>
</form></body></html>`, banner, html.EscapeString(csrf))
}
// spaLoginPage builds the form entirely in JS (no <form> in the raw HTML) and
// submits it via fetch, then redirects on success.
const spaLoginPage = `<html><head><title>SPA Login</title></head><body>
<div id="app"></div>
<script>
var app = document.getElementById('app');
var email = document.createElement('input'); email.id = 'email'; email.type = 'email';
var pass = document.createElement('input'); pass.id = 'password'; pass.type = 'password';
var btn = document.createElement('button'); btn.id = 'submit'; btn.textContent = 'Sign in';
app.appendChild(email); app.appendChild(pass); app.appendChild(btn);
btn.addEventListener('click', function () {
fetch('/auth/api/login', {
method: 'POST',
headers: {'Content-Type': 'application/json'},
body: JSON.stringify({username: email.value, password: pass.value})
}).then(function (r) { return r.json(); }).then(function (d) {
if (d.token) { document.body.innerHTML = '<h1>Welcome, signed in</h1>'; }
else { document.body.innerHTML += '<p>login failed</p>'; }
});
});
</script></body></html>`
// multiStepLoginPage hides the password field until "Next" is clicked.
const multiStepLoginPage = `<html><head><title>Multi-step Login</title></head><body>
<form id="f" method="post" action="/auth/form-login">
<input type="text" name="username" id="username" placeholder="Email">
<button type="button" id="next">Next</button>
<div id="pwwrap" style="display:none">
<input type="password" name="password" id="password" placeholder="Password">
<button type="submit" id="submit">Sign in</button>
</div>
</form>
<script>
document.getElementById('next').addEventListener('click', function () {
document.getElementById('pwwrap').style.display = 'block';
});
</script></body></html>`
// strictLoginPage keeps the submit button disabled until both fields are
// non-empty, reproducing the disabled-until-valid regression class.
const strictLoginPage = `<html><head><title>Strict Login</title></head><body>
<form method="post" action="/auth/form-login">
<input type="text" name="username" id="username" placeholder="Email">
<input type="password" name="password" id="password" placeholder="Password">
<button type="submit" id="submit" disabled>Sign in</button>
</form>
<script>
var u = document.getElementById('username'), p = document.getElementById('password'), b = document.getElementById('submit');
function validate() { b.disabled = !(u.value.length > 0 && p.value.length > 0); }
u.addEventListener('input', validate); p.addEventListener('input', validate);
</script></body></html>`
// spaTokenLoginPage validates client-side and stores a JWT in localStorage with
// no cookie, the canonical token-in-web-storage case.
var spaTokenLoginPage = fmt.Sprintf(`<html><head><title>SPA Token Login</title></head><body>
<form id="f">
<input type="email" name="username" id="email" placeholder="Email">
<input type="password" name="password" id="password" placeholder="Password">
<button type="submit" id="submit">Sign in</button>
</form>
<script>
document.getElementById('f').addEventListener('submit', function (e) {
e.preventDefault();
var u = document.getElementById('email').value;
var p = document.getElementById('password').value;
if (u === %q && p === %q) {
window.localStorage.setItem('access_token', %q);
window.sessionStorage.setItem('uid', '1');
document.body.innerHTML = '<h1>Welcome, signed in</h1>';
} else {
document.body.innerHTML += '<p>login failed</p>';
}
});
</script></body></html>`, AuthUsername, AuthPassword, jwtFor("spa-static-session"))
// multiFormLoginPage places a decoy search form (no password field) before the
// real login form to exercise best-form selection over first-form selection.
const multiFormLoginPage = `<html><head><title>Multi-form Login</title></head><body>
<form id="search" method="get" action="/auth/api/search">
<input type="text" name="q" placeholder="Search">
<button type="submit">Search</button>
</form>
<hr>
<form id="login" method="post" action="/auth/form-login">
<input type="text" name="username" placeholder="Email">
<input type="password" name="password" placeholder="Password">
<button type="submit">Sign in</button>
</form></body></html>`
// jsCookieLoginPage logs in via XHR and sets the session cookie from JS
// (document.cookie) rather than relying on a Set-Cookie response header.
const jsCookieLoginPage = `<html><head><title>JS Cookie Login</title></head><body>
<form id="f">
<input type="email" name="username" id="email" placeholder="Email">
<input type="password" name="password" id="password" placeholder="Password">
<button type="submit" id="submit">Sign in</button>
</form>
<script>
document.getElementById('f').addEventListener('submit', function (e) {
e.preventDefault();
fetch('/auth/api/login-jsbody', {
method: 'POST',
headers: {'Content-Type': 'application/json'},
body: JSON.stringify({
username: document.getElementById('email').value,
password: document.getElementById('password').value
})
}).then(function (r) { return r.json(); }).then(function (d) {
if (d.session) {
document.cookie = 'PSESSION=' + d.session + '; path=/';
document.body.innerHTML = '<h1>Welcome, signed in</h1>';
} else {
document.body.innerHTML += '<p>login failed</p>';
}
});
});
</script></body></html>`
// headerTokenLoginPage is a classic server-rendered form whose POST returns the
// session token in a response header rather than a cookie or body.
func headerTokenLoginPage(errMsg string) string {
var banner string
if errMsg != "" {
banner = fmt.Sprintf(`<p class="error">%s</p>`, html.EscapeString(errMsg))
}
return fmt.Sprintf(`<html><head><title>Header Token Login</title></head><body>
<h1>Sign in</h1>%s
<form method="post" action="/auth/header-token-login">
<input type="text" name="username" placeholder="Email">
<input type="password" name="password" placeholder="Password">
<button type="submit">Sign in</button>
</form></body></html>`, banner)
}
// spaHeaderTokenLoginPage logs in via XHR; the token comes back only in a
// response header (the page never reads or stores it), so the session is
// recoverable only by observing response headers.
const spaHeaderTokenLoginPage = `<html><head><title>SPA Header Token Login</title></head><body>
<form id="f">
<input type="email" name="username" id="email" placeholder="Email">
<input type="password" name="password" id="password" placeholder="Password">
<button type="submit" id="submit">Sign in</button>
</form>
<script>
document.getElementById('f').addEventListener('submit', function (e) {
e.preventDefault();
fetch('/auth/api/login-header', {
method: 'POST',
headers: {'Content-Type': 'application/json'},
body: JSON.stringify({
username: document.getElementById('email').value,
password: document.getElementById('password').value
})
}).then(function (r) { return r.json(); }).then(function (d) {
if (d.ok) { document.body.innerHTML = '<h1>Welcome, signed in</h1>'; }
else { document.body.innerHTML += '<p>login failed</p>'; }
});
});
</script></body></html>`
// delayedLoginPage injects the login form only after an async tick, simulating a
// SPA that renders its form after bootstrapping.
const delayedLoginPage = `<html><head><title>Delayed Login</title></head><body>
<div id="app"><p>loading...</p></div>
<script>
setTimeout(function () {
document.getElementById('app').innerHTML =
'<form method="post" action="/auth/form-login">' +
'<input type="text" name="username" placeholder="Email">' +
'<input type="password" name="password" placeholder="Password">' +
'<button type="submit">Sign in</button></form>';
}, 700);
</script></body></html>`
-8
View File
@@ -39,8 +39,6 @@ func GetPlaygroundServer() *PlaygroundServer {
mux.HandleFunc("POST /user", patchUnsanitizedUserHandler)
mux.HandleFunc("GET /blog/posts", getPostsHandler)
registerAuthRoutes(mux)
handler := recoverPlaygroundRequest(logPlaygroundRequest(mux))
return &PlaygroundServer{handler: handler}
}
@@ -286,12 +284,6 @@ func writeJSON(w http.ResponseWriter, statusCode int, value interface{}) {
_ = json.NewEncoder(w).Encode(value)
}
func writeString(w http.ResponseWriter, statusCode int, value string) {
w.Header().Set("Content-Type", "text/plain; charset=UTF-8")
w.WriteHeader(statusCode)
_, _ = io.WriteString(w, value)
}
func recoverPlaygroundRequest(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
defer func() {
+3
View File
@@ -39,6 +39,9 @@ func (r *Runner) initializeTemplatesHTTPInput() (*hybrid.HybridMap, error) {
}
httpxOptions.RetryMax = r.options.Retries
httpxOptions.Timeout = time.Duration(r.options.Timeout) * time.Second
if len(r.options.InternalResolversList) > 0 {
httpxOptions.Resolvers = r.options.InternalResolversList
}
dialers := protocolstate.GetDialersWithId(r.options.ExecutionId)
if dialers == nil {
-266
View File
@@ -1,17 +1,11 @@
package runner
import (
"bufio"
"context"
"fmt"
"net/http"
"net/url"
"os"
"strconv"
"strings"
"github.com/projectdiscovery/nuclei/v3/pkg/authprovider/authx"
"github.com/projectdiscovery/nuclei/v3/pkg/authprovider/autologin"
"github.com/projectdiscovery/nuclei/v3/pkg/catalog"
"github.com/projectdiscovery/nuclei/v3/pkg/catalog/disk"
"github.com/projectdiscovery/nuclei/v3/pkg/catalog/loader"
@@ -22,7 +16,6 @@ import (
"github.com/projectdiscovery/nuclei/v3/pkg/protocols/common/helpers/writer"
"github.com/projectdiscovery/nuclei/v3/pkg/protocols/common/replacer"
httpprotocol "github.com/projectdiscovery/nuclei/v3/pkg/protocols/http"
"github.com/projectdiscovery/nuclei/v3/pkg/protocols/http/httpclientpool"
"github.com/projectdiscovery/nuclei/v3/pkg/scan"
"github.com/projectdiscovery/nuclei/v3/pkg/templates"
"github.com/projectdiscovery/nuclei/v3/pkg/types"
@@ -78,265 +71,6 @@ func GetAuthTmplStore(opts *types.Options, catalog catalog.Catalog, execOpts *pr
return store, nil
}
// captureOnceStoreFromOptions launches a visible browser at -auth-login-url,
// lets the user log in manually (pressing Enter in the terminal when done) and
// builds an in-memory static auth store from the captured session (cookies and
// an optional token), scoped to the login host. Unlike auto-login this captures
// a point-in-time session with no automated re-authentication.
func captureOnceStoreFromOptions(opts *types.Options) (*authx.Authx, error) {
u, err := url.Parse(opts.AuthLoginURL)
if err != nil {
return nil, errkit.Wrap(err, "invalid -auth-login-url")
}
if u.Host == "" {
return nil, errkit.New("capture-once: -auth-login-url is required and must include a host")
}
rt := buildAutoLoginRuntimeOptions(opts)
cfg := autologin.Config{
LoginURL: opts.AuthLoginURL,
Proxy: rt.Proxy,
CDPEndpoint: rt.CDPEndpoint,
UseInstalledChrome: rt.UseInstalledChrome,
UserAgent: rt.UserAgent,
CustomHeaders: rt.CustomHeaders,
}
ready := func(ctx context.Context) error {
fmt.Fprintf(os.Stderr, "[auth-capture] Log in to %s in the opened browser window, then press Enter here to capture the session...", opts.AuthLoginURL)
reader := bufio.NewReader(os.Stdin)
done := make(chan error, 1)
go func() {
_, rerr := reader.ReadString('\n')
done <- rerr
}()
select {
case <-ctx.Done():
return ctx.Err()
case rerr := <-done:
return rerr
}
}
session, err := autologin.CaptureOnce(context.Background(), cfg, ready)
if err != nil {
return nil, err
}
return captureSessionToStore(session, u.Host)
}
// captureSessionToStore converts a captured login session into a static auth
// store scoped to host: cookies as a Cookie secret, an extracted token as a
// BearerToken secret, and any web storage as a WebStorage secret (replayed into
// headless scan pages). It is split out from captureOnceStoreFromOptions so the
// session->store mapping is unit-testable without driving a browser.
func captureSessionToStore(session *autologin.Session, host string) (*authx.Authx, error) {
if session == nil {
return nil, errkit.New("capture-once: nil session")
}
var secrets []authx.Secret
if len(session.Cookies) > 0 {
cookies := make([]authx.Cookie, 0, len(session.Cookies))
domains := []string{host}
for _, c := range session.Cookies {
cookies = append(cookies, authx.Cookie{
Key: c.Name,
Value: c.Value,
Domain: c.Domain,
Path: c.Path,
Secure: c.Secure,
})
if d := strings.TrimPrefix(strings.TrimSpace(c.Domain), "."); d != "" && !strings.EqualFold(d, host) {
found := false
for _, existing := range domains {
if strings.EqualFold(existing, d) {
found = true
break
}
}
if !found {
domains = append(domains, d)
}
}
}
secrets = append(secrets, authx.Secret{
Type: string(authx.CookiesAuth),
Domains: domains,
Cookies: cookies,
})
}
if session.Token != "" {
secrets = append(secrets, authx.Secret{
Type: string(authx.BearerTokenAuth),
Domains: []string{host},
Token: session.Token,
})
}
if len(session.LocalStorage) > 0 || len(session.SessionStorage) > 0 {
// Carry captured web storage so a token-in-web-storage SPA session
// replays into headless scan pages (HTTP scans ignore this secret).
secrets = append(secrets, authx.Secret{
Type: string(authx.WebStorageAuth),
Domains: []string{host},
LocalStorage: session.LocalStorage,
SessionStorage: session.SessionStorage,
})
}
if len(secrets) == 0 {
return nil, errkit.New("capture-once: captured session had no replayable cookies, token or web storage")
}
for i := range secrets {
if err := secrets[i].Validate(); err != nil {
return nil, errkit.Wrap(err, "capture-once: invalid captured secret")
}
}
return &authx.Authx{ID: "cli-capture-once", Secrets: secrets}, nil
}
// buildAutoLoginRuntimeOptions maps scan-level options into the auto-login
// runtime options so a (headless) auto-login uses the same identity and network
// path as the scan: user-agent and custom headers (-H), proxy, CDP endpoint and
// Chrome settings.
func buildAutoLoginRuntimeOptions(opts *types.Options) *authx.AutoLoginRuntimeOptions {
// Mirror the proxy precedence used everywhere else in the codebase (HTTP
// proxy first, SOCKS as fallback) so a SOCKS-proxied scan does not silently
// bypass the proxy during auto-login.
proxy := opts.AliveHttpProxy
if proxy == "" {
proxy = opts.AliveSocksProxy
}
rt := &authx.AutoLoginRuntimeOptions{
Proxy: proxy,
CDPEndpoint: opts.CDPEndpoint,
UseInstalledChrome: opts.UseInstalledChrome,
ShowBrowser: opts.ShowBrowser,
}
for _, header := range opts.CustomHeaders {
parts := strings.SplitN(header, ":", 2)
if len(parts) != 2 {
continue
}
key := strings.TrimSpace(parts[0])
value := strings.TrimSpace(parts[1])
if key == "" || value == "" {
continue
}
if strings.EqualFold(key, "User-Agent") {
rt.UserAgent = value
continue
}
if rt.CustomHeaders == nil {
rt.CustomHeaders = map[string]string{}
}
rt.CustomHeaders[key] = value
}
return rt
}
// autoLoginHTTPClient returns the nuclei HTTP client of the scan's execution
// id. The HTTP auto-login reuses its transport, so the login requests go
// through nuclei's dialer like every scan request: network policy (-lna,
// excluded hosts), resolvers and proxy. autologin.Login still uses its own
// cookie jar and cross-origin redirect guard on top of that transport.
func autoLoginHTTPClient(opts *types.Options) (*http.Client, error) {
client, err := httpclientpool.Get(opts, &httpclientpool.Configuration{}, "")
if err != nil {
return nil, err
}
if client == nil || client.HTTPClient == nil {
return nil, errkit.New("auto-login: nuclei http client is not available")
}
return client.HTTPClient, nil
}
// autoLoginStoreFromOptions assembles an in-memory Authx store containing a
// single auto-login dynamic secret built from the -auth-login-url flag set. The
// captured session is scoped to the login URL's host.
func autoLoginStoreFromOptions(opts *types.Options) (*authx.Authx, error) {
username, password := authCredentialsFromOptions(opts)
// The login URL may be supplied directly (-auth-login-url) or derived from a
// recording's first navigate step; resolve the host scope from whichever is
// available.
loginURL := opts.AuthLoginURL
if loginURL == "" && opts.AuthRecording != "" {
steps, err := autologin.StepsFromRecordingFile(opts.AuthRecording, username, password)
if err != nil {
return nil, err
}
loginURL = autologin.FirstNavigateURL(steps)
}
u, err := url.Parse(loginURL)
if err != nil {
return nil, errkit.Wrap(err, "invalid auto-login url")
}
if u.Host == "" {
return nil, errkit.New("auto-login: could not determine host (set -auth-login-url or provide a recording with a navigate step)")
}
reauthCodes, err := parseStatusCodes(opts.AuthReauthStatusCodes)
if err != nil {
return nil, err
}
return &authx.Authx{
ID: "cli-auto-login",
Dynamic: []authx.Dynamic{
{
Secret: &authx.Secret{Domains: []string{u.Host}},
// Session lifecycle: re-run the login on the configured expiry
// status codes and/or once the session ages past the interval, so
// a long scan does not silently continue with a dead session.
RefreshInterval: opts.AuthRefreshInterval,
ReauthStatusCodes: reauthCodes,
AutoLogin: &authx.AutoLoginConfig{
LoginURL: loginURL,
Username: username,
Password: password,
UsernameField: opts.AuthUsernameField,
PasswordField: opts.AuthPasswordField,
Headless: opts.AuthHeadless,
Recording: opts.AuthRecording,
},
},
},
}, nil
}
// authCredentialsFromOptions resolves CLI credentials, preferring flag values
// and falling back to NUCLEI_AUTH_PASSWORD so the password need not appear on
// the process argv / shell history.
func authCredentialsFromOptions(opts *types.Options) (username, password string) {
username = opts.AuthUsername
password = opts.AuthPassword
if password == "" {
password = os.Getenv("NUCLEI_AUTH_PASSWORD")
}
return username, password
}
// parseStatusCodes parses a comma-separated list of HTTP status codes (e.g.
// "401,403") into a slice of ints. An empty string yields a nil slice.
func parseStatusCodes(raw string) ([]int, error) {
raw = strings.TrimSpace(raw)
if raw == "" {
return nil, nil
}
var codes []int
for _, part := range strings.Split(raw, ",") {
part = strings.TrimSpace(part)
if part == "" {
continue
}
code, err := strconv.Atoi(part)
if err != nil {
return nil, errkit.Wrapf(err, "invalid -auth-reauth-status-codes value %q", part)
}
if code < 100 || code > 599 {
return nil, errkit.Newf("invalid -auth-reauth-status-codes value %q: must be a valid HTTP status code", part)
}
codes = append(codes, code)
}
return codes, nil
}
// GetLazyAuthFetchCallback returns a lazy fetch callback for auth secrets
func GetLazyAuthFetchCallback(opts *AuthLazyFetchOptions) authx.LazyFetchSecret {
return func(d *authx.Dynamic) error {
-233
View File
@@ -1,233 +0,0 @@
package runner
import (
"context"
"net/http"
"net/http/httptest"
"os"
"path/filepath"
"sync/atomic"
"testing"
"github.com/projectdiscovery/nuclei/v3/pkg/authprovider/authx"
"github.com/projectdiscovery/nuclei/v3/pkg/authprovider/autologin"
"github.com/projectdiscovery/nuclei/v3/pkg/protocols/common/protocolstate"
"github.com/projectdiscovery/nuclei/v3/pkg/types"
"github.com/stretchr/testify/require"
)
func TestCaptureSessionToStore(t *testing.T) {
session := &autologin.Session{
Cookies: []*http.Cookie{{Name: "session", Value: "sess-dave"}},
Token: "eyJ.payload.sig",
LocalStorage: map[string]string{"jwt": "eyJ.payload.sig"},
SessionStorage: map[string]string{"csrf": "tok"},
}
store, err := captureSessionToStore(session, "app.example.com")
require.NoError(t, err)
require.Len(t, store.Secrets, 3, "cookies, token and web storage must each map to a secret")
byType := map[string]authx.Secret{}
for _, s := range store.Secrets {
byType[s.Type] = s
require.Equal(t, []string{"app.example.com"}, s.Domains, "every secret is scoped to the login host")
require.NoError(t, s.Validate())
}
require.Equal(t, "session", byType[string(authx.CookiesAuth)].Cookies[0].Key)
require.Equal(t, "eyJ.payload.sig", byType[string(authx.BearerTokenAuth)].Token)
ws := byType[string(authx.WebStorageAuth)]
require.Equal(t, "eyJ.payload.sig", ws.LocalStorage["jwt"])
require.Equal(t, "tok", ws.SessionStorage["csrf"])
// The web storage secret must surface as a BrowserStorageProvider so the
// headless engine can seed it into scanned pages.
bsp, ok := ws.GetStrategy().(authx.BrowserStorageProvider)
require.True(t, ok)
local, sess := bsp.WebStorage()
require.Equal(t, "eyJ.payload.sig", local["jwt"])
require.Equal(t, "tok", sess["csrf"])
}
func TestCaptureSessionToStore_StorageOnly(t *testing.T) {
// A pure token-in-localStorage SPA: no cookies, no extracted token, only
// web storage — it must still produce a usable store.
session := &autologin.Session{LocalStorage: map[string]string{"jwt": "x"}}
store, err := captureSessionToStore(session, "spa.example.com")
require.NoError(t, err)
require.Len(t, store.Secrets, 1)
require.Equal(t, string(authx.WebStorageAuth), store.Secrets[0].Type)
}
func TestCaptureSessionToStore_Empty(t *testing.T) {
_, err := captureSessionToStore(&autologin.Session{}, "app.example.com")
require.Error(t, err, "an empty session must not produce a store")
}
func TestAutoLoginStoreFromOptions_RecordingDerivesHost(t *testing.T) {
recording := `{"steps": [
{"type": "navigate", "url": "https://recorded.example.com/signin"},
{"type": "change", "value": "bob", "selectors": [["#user"]]}
]}`
path := filepath.Join(t.TempDir(), "flow.json")
require.NoError(t, os.WriteFile(path, []byte(recording), 0o600))
// No -auth-login-url: the host scope must come from the recording.
opts := &types.Options{AuthRecording: path, AuthUsername: "bob", AuthPassword: "p"}
store, err := autoLoginStoreFromOptions(opts)
require.NoError(t, err)
require.Len(t, store.Dynamic, 1)
require.Equal(t, []string{"recorded.example.com"}, store.Dynamic[0].Domains)
require.Equal(t, path, store.Dynamic[0].AutoLogin.Recording)
}
func TestAutoLoginStoreFromOptions(t *testing.T) {
opts := &types.Options{
AuthLoginURL: "https://app.example.com:8443/login",
AuthUsername: "alice",
AuthPassword: "s3cr3t",
AuthUsernameField: "email",
AuthHeadless: true,
}
store, err := autoLoginStoreFromOptions(opts)
require.NoError(t, err)
require.Len(t, store.Dynamic, 1)
dyn := store.Dynamic[0]
require.NotNil(t, dyn.Secret)
require.Equal(t, []string{"app.example.com:8443"}, dyn.Domains, "session should be scoped to the login host")
require.NotNil(t, dyn.AutoLogin)
require.Equal(t, "https://app.example.com:8443/login", dyn.AutoLogin.LoginURL)
require.Equal(t, "alice", dyn.AutoLogin.Username)
require.Equal(t, "email", dyn.AutoLogin.UsernameField)
require.True(t, dyn.AutoLogin.Headless)
// The assembled store must validate as a real auto-login dynamic.
require.NoError(t, dyn.Validate())
}
func TestAutoLoginStoreFromOptions_InvalidURL(t *testing.T) {
_, err := autoLoginStoreFromOptions(&types.Options{AuthLoginURL: "://nohost"})
require.Error(t, err)
}
func TestAutoLoginStoreFromOptions_SessionLifecycle(t *testing.T) {
opts := &types.Options{
AuthLoginURL: "https://app.example.com/login",
AuthUsername: "alice",
AuthPassword: "s3cr3t",
AuthReauthStatusCodes: "401, 403",
AuthRefreshInterval: "15m",
}
store, err := autoLoginStoreFromOptions(opts)
require.NoError(t, err)
dyn := store.Dynamic[0]
require.Equal(t, []int{401, 403}, dyn.ReauthStatusCodes, "reauth status codes must be threaded onto the dynamic")
require.Equal(t, "15m", dyn.RefreshInterval)
// The lifecycle-configured store must validate (refresh-interval is parsed).
require.NoError(t, dyn.Validate())
}
func TestAutoLoginStoreFromOptions_InvalidReauthCodes(t *testing.T) {
_, err := autoLoginStoreFromOptions(&types.Options{
AuthLoginURL: "https://app.example.com/login",
AuthPassword: "s3cr3t",
AuthReauthStatusCodes: "401,nope",
})
require.Error(t, err, "non-numeric reauth status code must be rejected")
}
func TestParseStatusCodes(t *testing.T) {
cases := []struct {
in string
want []int
wantErr bool
}{
{"", nil, false},
{" ", nil, false},
{"401", []int{401}, false},
{"401,403, 419", []int{401, 403, 419}, false},
{"401,,403", []int{401, 403}, false},
{"abc", nil, true},
{"99", nil, true},
{"600", nil, true},
}
for _, tc := range cases {
got, err := parseStatusCodes(tc.in)
if tc.wantErr {
require.Error(t, err, "input %q", tc.in)
continue
}
require.NoError(t, err, "input %q", tc.in)
require.Equal(t, tc.want, got, "input %q", tc.in)
}
}
func TestBuildAutoLoginRuntimeOptions(t *testing.T) {
opts := &types.Options{
CustomHeaders: []string{"User-Agent: NucleiScan/1.0", "X-Env: staging", "malformed-header"},
AliveHttpProxy: "http://127.0.0.1:8080",
CDPEndpoint: "ws://127.0.0.1:9222",
UseInstalledChrome: true,
ShowBrowser: true,
}
rt := buildAutoLoginRuntimeOptions(opts)
require.Equal(t, "NucleiScan/1.0", rt.UserAgent, "User-Agent should be split out of custom headers")
require.Equal(t, "staging", rt.CustomHeaders["X-Env"])
require.NotContains(t, rt.CustomHeaders, "User-Agent", "UA must not be duplicated into custom headers")
require.Equal(t, "http://127.0.0.1:8080", rt.Proxy)
require.Equal(t, "ws://127.0.0.1:9222", rt.CDPEndpoint)
require.True(t, rt.UseInstalledChrome)
require.True(t, rt.ShowBrowser)
}
func TestBuildAutoLoginRuntimeOptions_SocksProxyFallback(t *testing.T) {
// With only a SOCKS proxy configured, the auto-login must still pick it up
// (regression: previously only AliveHttpProxy was threaded).
rt := buildAutoLoginRuntimeOptions(&types.Options{
AliveSocksProxy: "socks5://127.0.0.1:1080",
})
require.Equal(t, "socks5://127.0.0.1:1080", rt.Proxy)
// HTTP proxy takes precedence when both are set.
rt = buildAutoLoginRuntimeOptions(&types.Options{
AliveHttpProxy: "http://127.0.0.1:8080",
AliveSocksProxy: "socks5://127.0.0.1:1080",
})
require.Equal(t, "http://127.0.0.1:8080", rt.Proxy)
}
// The HTTP auto-login must go through nuclei's dialer, so -lna refuses a login
// page on the loopback network and the server never sees a request.
func TestAutoLoginHTTPClientHonorsNetworkPolicy(t *testing.T) {
var hits atomic.Int32
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
hits.Add(1)
_, _ = w.Write([]byte(`<form method="post" action="/login"><input name="user"><input type="password" name="pass"></form>`))
}))
t.Cleanup(srv.Close)
options := &types.Options{
ExecutionId: t.Name(),
Timeout: 5,
RestrictLocalNetworkAccess: true,
}
require.NoError(t, protocolstate.Init(options))
t.Cleanup(func() { protocolstate.Close(options.ExecutionId) })
client, err := autoLoginHTTPClient(options)
require.NoError(t, err)
_, err = autologin.Login(context.Background(), client, autologin.Config{
LoginURL: srv.URL + "/login",
Username: "admin",
Password: "secret",
})
require.Error(t, err, "the login page on loopback must be refused under -lna")
require.Zero(t, hits.Load(), "the local server must get no request")
}
func TestAutoLoginHTTPClientNeedsDialers(t *testing.T) {
_, err := autoLoginHTTPClient(&types.Options{ExecutionId: t.Name()})
require.Error(t, err, "without nuclei dialers there is no policy-bound client")
}
+35 -1
View File
@@ -2,8 +2,12 @@ package runner
import (
"context"
"errors"
"sync"
"time"
"github.com/projectdiscovery/gologger"
llmclient "github.com/projectdiscovery/nuclei/v3/pkg/operators/common/llm"
"github.com/projectdiscovery/nuclei/v3/pkg/types"
utilsllm "github.com/projectdiscovery/utils/llm"
@@ -28,6 +32,32 @@ func (a *llmMatcherClient) Complete(ctx context.Context, prompt string, asJSON b
})
}
// llmJudgeClient is an llmMatcherClient whose provider classifies natively, so
// llm matchers ask it through Judge instead of Complete.
type llmJudgeClient struct {
*llmMatcherClient
warnComplete sync.Once
}
func (a *llmJudgeClient) Judge(ctx context.Context, question string, labels, inputs []string) (llmclient.Verdict, error) {
verdict, err := a.client.Judge(ctx, utilsllm.JudgeRequest{Question: question, Labels: labels, Inputs: inputs})
if err != nil {
return llmclient.Verdict{}, err
}
return llmclient.Verdict{Label: verdict.Label, Confidence: verdict.Confidence, Model: verdict.Model}, nil
}
// Complete is only reached by llm extractors, which need free text a
// classifier cannot produce. Returning here spends no call budget and
// makes no provider request. Extractors turn the error into an empty
// result, and the warning says why.
func (a *llmJudgeClient) Complete(context.Context, string, bool) (string, error) {
a.warnComplete.Do(func() {
gologger.Warning().Msgf("llm provider %q cannot run llm extractors, they return nothing", utilsllm.ProviderJev)
})
return "", errors.New("llm provider cannot run extractors")
}
// configureLLM builds the scan's llm client from options. It returns nil unless
// -llm is set, so nothing reaches a model until the user opts in. The API key is
// read from the environment by the provider layer, never from options.
@@ -52,5 +82,9 @@ func configureLLM(options *types.Options) (llmclient.Client, error) {
return nil, err
}
return &llmMatcherClient{client: client, model: options.LLMModel}, nil
matcherClient := &llmMatcherClient{client: client, model: options.LLMModel}
if client.CanJudge() {
return &llmJudgeClient{llmMatcherClient: matcherClient}, nil
}
return matcherClient, nil
}
+65
View File
@@ -0,0 +1,65 @@
package runner
import (
"context"
"net/http"
"net/http/httptest"
"sync/atomic"
"testing"
llmclient "github.com/projectdiscovery/nuclei/v3/pkg/operators/common/llm"
"github.com/projectdiscovery/nuclei/v3/pkg/types"
"github.com/stretchr/testify/require"
)
func TestConfigureLLMExposesJudgeOnlyForClassifiers(t *testing.T) {
t.Setenv("LLM_API_KEY", "key")
client, err := configureLLM(&types.Options{EnableLLM: true, LLMProvider: "jev", LLMModel: "jev-latest"})
require.NoError(t, err)
require.Implements(t, (*llmclient.Judge)(nil), client)
client, err = configureLLM(&types.Options{EnableLLM: true, LLMProvider: "ollama", LLMModel: "llama3.2"})
require.NoError(t, err)
_, isJudge := client.(llmclient.Judge)
require.False(t, isJudge, "chat providers keep the Complete path")
}
func TestJevExtractorDoesNotSpendCallBudget(t *testing.T) {
t.Setenv("LLM_API_KEY", "key")
// ProxyFromEnvironment caches the first request in the process. Match the
// proxy the interactsh client test expects, and keep the local server direct.
for _, key := range []string{"HTTP_PROXY", "http_proxy", "ALL_PROXY", "all_proxy", "https_proxy", "no_proxy"} {
t.Setenv(key, "")
}
t.Setenv("HTTPS_PROXY", "http://127.0.0.1:9")
t.Setenv("NO_PROXY", "127.0.0.1,localhost")
var calls atomic.Int32
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
calls.Add(1)
_, _ = w.Write([]byte(`{"model":"jev-1.14.0","answers":{"q":{"type":"noul","noul":0.9}}}`))
}))
t.Cleanup(server.Close)
client, err := configureLLM(&types.Options{
EnableLLM: true,
LLMProvider: "jev",
LLMBaseURL: server.URL + "/v1",
LLMModel: "jev-latest",
LLMMaxCalls: 1,
})
require.NoError(t, err)
_, err = client.Complete(context.Background(), "extract the title", true)
require.ErrorContains(t, err, "cannot run extractors")
_, err = client.Complete(context.Background(), "extract another", true)
require.Error(t, err)
require.Zero(t, calls.Load())
judge := client.(llmclient.Judge)
verdict, err := judge.Judge(context.Background(), "login page?", nil, []string{"<form>"})
require.NoError(t, err)
require.Equal(t, "yes", verdict.Label)
require.EqualValues(t, 1, calls.Load())
}
+4 -15
View File
@@ -228,25 +228,14 @@ func ValidateOptions(options *types.Options) error {
}
}
// verify that a valid ip version type was selected (4, 6)
if len(options.IPVersion) == 0 {
// add ipv4 as default
options.IPVersion = append(options.IPVersion, "4")
}
var useIPV4, useIPV6 bool
// An empty -iv leaves one unpinned target so the dialer can fall back
// between address families. -sa resolves and scans every selected address.
// Explicit -iv 4, -iv 6 and -iv 4,6 retain their usual selection semantics.
for _, ipv := range options.IPVersion {
switch ipv {
case "4":
useIPV4 = true
case "6":
useIPV6 = true
default:
if ipv != "4" && ipv != "6" {
return fmt.Errorf("unsupported ip version: %s", ipv)
}
}
if !useIPV4 && !useIPV6 {
return errors.New("ipv4 and/or ipv6 must be selected")
}
return nil
}
+12
View File
@@ -90,3 +90,15 @@ func TestReadsTargetsFromStdin(t *testing.T) {
})
}
}
// The default must remain empty so the provider can distinguish automatic
// IPv6 preference from explicit user-supplied -iv selections.
func TestValidateOptionsIPVersionPreference(t *testing.T) {
automatic := &types.Options{}
require.NoError(t, ValidateOptions(automatic))
require.Empty(t, automatic.IPVersion)
for _, versions := range []goflags.StringSlice{{"4"}, {"6"}, {"4", "6"}} {
require.NoError(t, ValidateOptions(&types.Options{IPVersion: versions}))
}
require.ErrorContains(t, ValidateOptions(&types.Options{IPVersion: goflags.StringSlice{"7"}}), "unsupported ip version")
}
+20 -83
View File
@@ -67,6 +67,7 @@ import (
"github.com/projectdiscovery/nuclei/v3/pkg/protocols/common/utils/excludematchers"
"github.com/projectdiscovery/nuclei/v3/pkg/protocols/headless/engine"
httpProtocol "github.com/projectdiscovery/nuclei/v3/pkg/protocols/http"
"github.com/projectdiscovery/nuclei/v3/pkg/protocols/http/catchall"
"github.com/projectdiscovery/nuclei/v3/pkg/protocols/http/httpclientpool"
"github.com/projectdiscovery/nuclei/v3/pkg/reporting"
"github.com/projectdiscovery/nuclei/v3/pkg/templates"
@@ -114,7 +115,6 @@ type Runner struct {
httpApiEndpoint *httpapi.Server
fuzzStats *fuzzStats.Tracker
dastServer *server.DASTServer
authProvider authprovider.AuthProvider
llmClient llmclient.Client
}
@@ -536,11 +536,6 @@ func (r *Runner) Close() {
if r.browser != nil {
r.browser.Close()
}
// Drop any in-memory auto-login session so nuclei does not retain live
// session material after the scan (server-side sessions expire on their own).
if r.authProvider != nil {
r.authProvider.Close()
}
if r.tmpDir != "" {
_ = os.RemoveAll(r.tmpDir)
}
@@ -701,86 +696,24 @@ func (r *Runner) RunEnumeration() error {
executorOpts.ExportReqURLPattern = true
}
if (len(r.options.SecretsFile) > 0 || r.options.AuthLoginURL != "" || r.options.AuthRecording != "" || r.options.AuthCapture) && !r.options.Validate {
autoLoginOpts := buildAutoLoginRuntimeOptions(r.options)
autoLoginClient, err := autoLoginHTTPClient(r.options)
if len(r.options.SecretsFile) > 0 && !r.options.Validate {
// Clone options so GetAuthTmplStore can modify them without affecting the original
authOptions := r.options.Copy()
authTmplStore, err := GetAuthTmplStore(authOptions, r.catalog, executorOpts)
if err != nil {
return errors.Wrap(err, "could not create auto-login http client")
return errors.Wrap(err, "failed to load dynamic auth templates")
}
autoLoginOpts.HTTPClient = autoLoginClient
var providers []authprovider.AuthProvider
if len(r.options.SecretsFile) > 0 {
// Clone options so GetAuthTmplStore can modify them without affecting the original
authOptions := r.options.Copy()
authTmplStore, err := GetAuthTmplStore(authOptions, r.catalog, executorOpts)
if err != nil {
return errors.Wrap(err, "failed to load dynamic auth templates")
}
authOpts := &authprovider.AuthProviderOptions{SecretsFiles: r.options.SecretsFile}
authOpts.LazyFetchSecret = GetLazyAuthFetchCallback(&AuthLazyFetchOptions{
TemplateStore: authTmplStore,
ExecOpts: executorOpts,
})
authOpts.AutoLoginOptions = autoLoginOpts
fileProvider, err := authprovider.NewAuthProvider(authOpts)
if err != nil {
return errors.Wrap(err, "could not create auth provider")
}
providers = append(providers, fileProvider)
authOpts := &authprovider.AuthProviderOptions{SecretsFiles: r.options.SecretsFile}
authOpts.LazyFetchSecret = GetLazyAuthFetchCallback(&AuthLazyFetchOptions{
TemplateStore: authTmplStore,
ExecOpts: executorOpts,
})
// initialize auth provider
provider, err := authprovider.NewAuthProvider(authOpts)
if err != nil {
return errors.Wrap(err, "could not create auth provider")
}
// Turnkey auto-login from CLI flags (-auth-login-url): build an in-memory
// auto-login secret scoped to the login host, no secrets file needed.
// Capture-once also requires -auth-login-url, so skip the automated
// provider there to avoid overlapping sessions/logins.
if !r.options.AuthCapture && (r.options.AuthLoginURL != "" || r.options.AuthRecording != "") {
engine := "http"
if r.options.AuthHeadless || r.options.AuthRecording != "" {
engine = "headless"
}
target := r.options.AuthLoginURL
if target == "" {
target = r.options.AuthRecording
}
r.Logger.Info().Msgf("Auto-login enabled (%s engine) for %s", engine, target)
store, err := autoLoginStoreFromOptions(r.options)
if err != nil {
return errors.Wrap(err, "could not build auto-login auth provider")
}
cliProvider, err := authprovider.NewStoreAuthProvider(store, nil, autoLoginOpts)
if err != nil {
return errors.Wrap(err, "could not create auto-login auth provider")
}
providers = append(providers, cliProvider)
}
// Capture-once: open a visible browser for a one-time manual login and
// build a static session store from the captured cookies/token.
if r.options.AuthCapture {
if r.options.AuthLoginURL == "" {
return errors.New("-auth-capture requires -auth-login-url (the page to open for manual login)")
}
r.Logger.Info().Msgf("Capture-once login enabled for %s", r.options.AuthLoginURL)
store, err := captureOnceStoreFromOptions(r.options)
if err != nil {
return errors.Wrap(err, "could not capture login session")
}
captureProvider, err := authprovider.NewStoreAuthProvider(store, nil, autoLoginOpts)
if err != nil {
return errors.Wrap(err, "could not create capture-once auth provider")
}
providers = append(providers, captureProvider)
}
switch len(providers) {
case 1:
executorOpts.AuthProvider = providers[0]
default:
executorOpts.AuthProvider = authprovider.NewMultiAuthProvider(providers...)
}
// Retained so Runner.Close can drop captured session material on exit.
r.authProvider = executorOpts.AuthProvider
executorOpts.AuthProvider = provider
}
if r.options.ShouldUseHostError() {
@@ -802,6 +735,9 @@ func (r *Runner) RunEnumeration() error {
if r.options.HostBackoff {
executorOpts.HostBackoff = hostbackoff.New(hostbackoff.Config{Max: r.options.HostBackoffMax})
}
if r.options.AutoCalibrate {
executorOpts.CatchAll = catchall.New(0)
}
executorEngine := core.New(r.options)
executorEngine.SetExecuterOptions(executorOpts)
@@ -1140,6 +1076,7 @@ func (r *Runner) displayExecutionInfo(store *loader.Store) {
stats.DisplayAsWarning(httpProtocol.SetThreadToCountZero)
stats.ForceDisplayWarning(templates.SkippedUnverifiedTemplateStats)
stats.ForceDisplayWarning(templates.SkippedRequestSignatureTemplateStats)
stats.ForceDisplayWarning(templates.SkippedWorkflowStepStats)
cfg := config.DefaultConfig
+66
View File
@@ -6,11 +6,17 @@ import (
"net/http/httptest"
"os"
"path/filepath"
"strconv"
"sync/atomic"
"testing"
"time"
"github.com/miekg/dns"
"github.com/projectdiscovery/gologger"
"github.com/stretchr/testify/require"
"github.com/projectdiscovery/nuclei/v3/pkg/input"
"github.com/projectdiscovery/nuclei/v3/pkg/input/provider"
"github.com/projectdiscovery/nuclei/v3/pkg/model/types/severity"
"github.com/projectdiscovery/nuclei/v3/pkg/protocols/common/protocolstate"
"github.com/projectdiscovery/nuclei/v3/pkg/types"
@@ -197,3 +203,63 @@ func TestNewInteractshHTTPClient(t *testing.T) {
require.NotNil(t, proxyURL)
require.Equal(t, "http://127.0.0.1:9", proxyURL.String())
}
func TestInitializeTemplatesHTTPInputUsesResolvers(t *testing.T) {
for _, key := range []string{"HTTP_PROXY", "http_proxy", "HTTPS_PROXY", "https_proxy"} {
t.Setenv(key, "")
}
listener, err := net.ListenPacket("udp", "127.0.0.1:0")
require.NoError(t, err)
queries := new(atomic.Int32)
server := &dns.Server{PacketConn: listener, Handler: dns.HandlerFunc(func(writer dns.ResponseWriter, request *dns.Msg) {
queries.Add(1)
response := new(dns.Msg).SetReply(request)
for _, question := range request.Question {
if question.Qtype == dns.TypeA {
response.Answer = append(response.Answer, &dns.A{Hdr: dns.RR_Header{Name: question.Name, Rrtype: dns.TypeA, Class: dns.ClassINET}, A: net.ParseIP("127.0.0.1")})
}
}
_ = writer.WriteMsg(response)
})}
started := make(chan struct{})
server.NotifyStartedFunc = func() { close(started) }
done := make(chan error, 1)
go func() { done <- server.ActivateAndServe() }()
<-started
t.Cleanup(func() { require.NoError(t, server.Shutdown()); require.NoError(t, <-done) })
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
w.WriteHeader(http.StatusOK)
}))
t.Cleanup(srv.Close)
resolverFile := filepath.Join(t.TempDir(), "resolv.conf")
require.NoError(t, os.WriteFile(resolverFile, nil, 0o600))
t.Setenv("RESOLVERS_PATH", resolverFile)
options := &types.Options{
ExecutionId: t.Name(),
Timeout: 5,
BulkSize: 1,
InternalResolversList: []string{listener.LocalAddr().String()},
}
require.NoError(t, protocolstate.Init(options))
t.Cleanup(func() { protocolstate.Close(options.ExecutionId) })
target := "probe.nuclei.test:" + strconv.Itoa(srv.Listener.Addr().(*net.TCPAddr).Port)
r := &Runner{
options: options,
inputProvider: provider.NewSimpleInputProviderWithUrls(options.ExecutionId, target),
Logger: &gologger.Logger{},
}
hm, err := r.initializeTemplatesHTTPInput()
require.NoError(t, err)
t.Cleanup(func() { _ = hm.Close() })
probed, ok := hm.Get(input.ProbeKey(target, ""))
require.True(t, ok, "the probe should resolve the target through the -r resolver")
require.Equal(t, "http://"+target, string(probed))
require.Positive(t, queries.Load())
}
+42 -7
View File
@@ -10,9 +10,11 @@ import (
"github.com/logrusorgru/aurora/v4"
"github.com/projectdiscovery/nuclei/v3/pkg/catalog/config"
"github.com/projectdiscovery/nuclei/v3/pkg/catalog/loader"
"github.com/projectdiscovery/nuclei/v3/pkg/model"
"github.com/projectdiscovery/nuclei/v3/pkg/templates"
"github.com/projectdiscovery/nuclei/v3/pkg/types"
"github.com/projectdiscovery/nuclei/v3/pkg/utils"
"github.com/projectdiscovery/nuclei/v3/pkg/utils/json"
)
@@ -38,12 +40,17 @@ func (r *Runner) verboseTemplate(tpl *templates.Template) {
tpl.Info.SeverityHolder.Severity))
}
// listedTemplate names its fields like a result event, so a -jsonl listing
// joins with scan results on the same keys.
type listedTemplate struct {
Template string `json:"template,omitempty"`
TemplateID string `json:"template-id"`
TemplatePath string `json:"template-path"`
Info model.Info `json:"info"`
}
func (r *Runner) listAvailableStoreTemplates(store *loader.Store) {
r.Logger.Print().Msgf(
"\nListing available %v nuclei templates for %v",
config.DefaultConfig.TemplateVersion,
config.DefaultConfig.TemplatesDirectory,
)
r.listingHeader("templates")
// order templates alphabetically by path
templates := store.Templates()
sort.Slice(templates, func(i, j int) bool {
@@ -51,6 +58,10 @@ func (r *Runner) listAvailableStoreTemplates(store *loader.Store) {
})
for _, tpl := range templates {
if r.options.JSONL && !r.options.TemplateDisplay {
r.listTemplateJSON(tpl)
continue
}
if hasExtraFlags(r.options) {
if r.options.TemplateDisplay {
colorize := !r.options.NoColor
@@ -78,12 +89,36 @@ func (r *Runner) listAvailableStoreTemplates(store *loader.Store) {
}
}
func (r *Runner) listAvailableTags(tagsMap map[string]int) {
func (r *Runner) listTemplateJSON(tpl *templates.Template) {
path, _ := utils.TemplatePathURL(tpl.Path, tpl.ID, "")
marshalled, err := json.Marshal(listedTemplate{
Template: path,
TemplateID: tpl.ID,
TemplatePath: tpl.Path,
Info: tpl.Info,
})
if err != nil {
r.Logger.Error().Msgf("Could not marshal the template %s: %s", tpl.Path, err)
return
}
r.Logger.Print().Msgf("%s", marshalled)
}
// listingHeader is skipped for -jsonl, where every line must parse as JSON.
func (r *Runner) listingHeader(what string) {
if r.options.JSONL {
return
}
r.Logger.Print().Msgf(
"\nListing available %v nuclei tags for %v",
"\nListing available %v nuclei %s for %v",
config.DefaultConfig.TemplateVersion,
what,
config.DefaultConfig.TemplatesDirectory,
)
}
func (r *Runner) listAvailableTags(tagsMap map[string]int) {
r.listingHeader("tags")
type kv struct {
Key string `json:"tag"`
+76
View File
@@ -0,0 +1,76 @@
package runner
import (
"os"
"path/filepath"
"strings"
"testing"
"github.com/projectdiscovery/gologger"
"github.com/projectdiscovery/gologger/formatter"
"github.com/projectdiscovery/gologger/levels"
"github.com/projectdiscovery/nuclei/v3/internal/tests/testutils"
"github.com/projectdiscovery/nuclei/v3/pkg/catalog/config"
"github.com/projectdiscovery/nuclei/v3/pkg/catalog/disk"
"github.com/projectdiscovery/nuclei/v3/pkg/catalog/loader"
"github.com/projectdiscovery/nuclei/v3/pkg/templates"
"github.com/projectdiscovery/nuclei/v3/pkg/utils/json"
"github.com/stretchr/testify/require"
)
const listedTemplateYAML = `id: listed-template
info:
name: Listed Template
author: pdteam
severity: info
tags: listing
http:
- method: GET
path:
- "{{BaseURL}}"
`
func TestTemplateListWritesJSONLines(t *testing.T) {
root := t.TempDir()
cfg := config.DefaultConfig
oldRoot := cfg.TemplatesDirectory
t.Cleanup(func() { cfg.SetTemplatesDir(oldRoot) })
cfg.SetTemplatesDir(root)
path := filepath.Join(root, "http", "listed-template.yaml")
require.NoError(t, os.MkdirAll(filepath.Dir(path), 0o755))
require.NoError(t, os.WriteFile(path, []byte(listedTemplateYAML), 0o600))
writer := &ignoreFileLogWriter{}
logger := &gologger.Logger{}
logger.SetFormatter(formatter.NewCLI(true))
logger.SetWriter(writer)
logger.SetMaxLevel(levels.LevelInfo)
options := testutils.DefaultOptions.Copy()
options.ExecutionId = t.Name()
options.Logger = logger
options.Templates = []string{path}
options.TemplateList = true
options.JSONL = true
testutils.Init(options)
t.Cleanup(func() { testutils.Cleanup(options) })
executerOpts := testutils.NewMockExecuterOptions(options, nil)
executerOpts.Parser = templates.NewParser()
store, err := loader.New(loader.NewConfig(options, disk.NewCatalog(root), executerOpts))
require.NoError(t, err)
require.NoError(t, store.LoadTemplatesOnlyMetadata())
require.Len(t, store.Templates(), 1)
r := &Runner{options: options, Logger: logger}
r.listAvailableStoreTemplates(store)
require.Len(t, writer.records, 1, "JSONL output carries no banner line")
var listed map[string]any
require.NoError(t, json.Unmarshal([]byte(strings.TrimSpace(writer.records[0].message)), &listed))
require.Equal(t, "listed-template", listed["template-id"])
require.Equal(t, filepath.Join("http", "listed-template.yaml"), listed["template"])
require.Equal(t, "Listed Template", listed["info"].(map[string]any)["name"])
}
+4
View File
@@ -33,6 +33,7 @@ import (
"github.com/projectdiscovery/nuclei/v3/pkg/protocols/common/interactsh"
"github.com/projectdiscovery/nuclei/v3/pkg/protocols/common/utils/excludematchers"
browserEngine "github.com/projectdiscovery/nuclei/v3/pkg/protocols/headless/engine"
"github.com/projectdiscovery/nuclei/v3/pkg/protocols/http/catchall"
"github.com/projectdiscovery/nuclei/v3/pkg/reporting"
"github.com/projectdiscovery/nuclei/v3/pkg/templates"
"github.com/projectdiscovery/nuclei/v3/pkg/types"
@@ -110,6 +111,9 @@ func newNucleiExecutor(opts *NucleiExecutorOptions) (*nucleiExecutor, error) {
if opts.Options.HostBackoff {
executorOpts.HostBackoff = hostbackoff.New(hostbackoff.Config{Max: opts.Options.HostBackoffMax})
}
if opts.Options.AutoCalibrate {
executorOpts.CatchAll = catchall.New(0)
}
executorEngine := core.New(opts.Options)
executorEngine.SetExecuterOptions(executorOpts)
+57
View File
@@ -102,6 +102,7 @@ var httpTestcases = []integrationCase{
{Path: "protocols/http/raw-unsafe-path-single-slash.yaml", TestCase: &httpRawUnsafePathSingleSlash{}},
{Path: "protocols/http/disable-http-cache.yaml", TestCase: &httpDisableCache{}},
{Path: "protocols/http/http-cache.yaml", TestCase: &httpCache{}},
{Path: "protocols/http/get.yaml", TestCase: &httpExcludeHostsFileComments{}},
}
type httpMultiVarSharing struct{}
@@ -1983,3 +1984,59 @@ func (h *httpDisableCache) Execute(filePath string) error {
return nil
}
// ensure that comments in an -exclude-hosts file are ignored
type httpExcludeHostsFileComments struct{}
// Execute executes a test case and returns an error if occurred
func (h *httpExcludeHostsFileComments) Execute(filePath string) error {
router := httprouter.New()
router.GET("/", func(w http.ResponseWriter, r *http.Request, _ httprouter.Params) {
_, _ = fmt.Fprintf(w, "This is test matcher text")
})
ts := httptest.NewServer(router)
defer ts.Close()
excludeHosts := func(content string) ([]string, error) {
file, err := os.CreateTemp("", "exclude-hosts-*.txt")
if err != nil {
return nil, err
}
defer func() { _ = os.Remove(file.Name()) }()
_, err = file.WriteString(content)
_ = file.Close()
if err != nil {
return nil, err
}
return testutils.RunNucleiTemplateAndGetResults(filePath, ts.URL, debug, "-eh", file.Name())
}
// a comment line is not a host, even when it reads as an invalid regex
results, err := excludeHosts("# hosts out of scope (see ticket\n203.0.113.10\n")
if err != nil {
return fmt.Errorf("comment line: %w", err)
}
if err := expectResultsCount(results, 1); err != nil {
return fmt.Errorf("comment line: %w", err)
}
// a trailing comment does not stop the host from being excluded
results, err = excludeHosts("127.0.0.1 # test server\n")
if err != nil {
return fmt.Errorf("trailing comment: %w", err)
}
if err := expectResultsCount(results, 0); err != nil {
return fmt.Errorf("trailing comment: %w", err)
}
// comma separated hosts are still split when passed on the command line
results, err = testutils.RunNucleiTemplateAndGetResults(filePath, ts.URL, debug, "-eh", "203.0.113.10, 127.0.0.1")
if err != nil {
return fmt.Errorf("comma separated: %w", err)
}
if err := expectResultsCount(results, 0); err != nil {
return fmt.Errorf("comma separated: %w", err)
}
return nil
}
+3
View File
@@ -358,6 +358,9 @@ func (e *NucleiEngine) init(ctx context.Context) error {
httpxOptions := httpx.DefaultOptions
httpxOptions.Timeout = 5 * time.Second
if len(e.opts.InternalResolversList) > 0 {
httpxOptions.Resolvers = e.opts.InternalResolversList
}
if client, err := httpx.New(&httpxOptions); err != nil {
return err
} else {
-324
View File
@@ -1,324 +0,0 @@
package authx
import (
"context"
"fmt"
"net/http"
"strings"
"github.com/projectdiscovery/gologger"
"github.com/projectdiscovery/nuclei/v3/pkg/authprovider/autologin"
"github.com/projectdiscovery/utils/errkit"
)
// AutoLoginConfig declares a turnkey, template-free authenticated-scanning
// login. Instead of pointing at a hand-written auth template (Dynamic.TemplatePath),
// the user supplies only a login URL and credentials; nuclei fetches the login
// page, auto-detects the form (username/password fields plus hidden CSRF tokens)
// and submits it, capturing the resulting session. The captured session is fed
// into the same Dynamic state machine, so refresh-interval and
// reauth-status-codes re-run the login automatically.
type AutoLoginConfig struct {
// LoginURL is the page serving the login form (required).
LoginURL string `json:"login-url" yaml:"login-url"`
// Username / Password are the credentials to submit. Username may be empty
// for password-only forms.
Username string `json:"username" yaml:"username"`
Password string `json:"password" yaml:"password"`
// UsernameField / PasswordField force the form field names when
// auto-detection would guess wrong (optional).
UsernameField string `json:"username-field" yaml:"username-field"`
PasswordField string `json:"password-field" yaml:"password-field"`
// ExtraFields are additional form values submitted verbatim (e.g. a tenant
// id), overriding detected fields of the same name.
ExtraFields []KV `json:"extra-fields" yaml:"extra-fields"`
// TokenRegex, when set, extracts a bearer token (first capture group) from
// the final response body for token-based logins that don't set a cookie.
TokenRegex string `json:"token-regex" yaml:"token-regex"`
// AllowCrossOriginAction lets the HTTP engine submit the credentials to a
// login form whose action is on a different origin than LoginURL (e.g. an
// SSO provider). Such a form is refused by default.
AllowCrossOriginAction bool `json:"allow-cross-origin-action" yaml:"allow-cross-origin-action"`
// Headless drives a real browser for the login instead of the HTTP engine,
// enabling JS-rendered / SPA / multi-step / SSO login pages.
Headless bool `json:"headless" yaml:"headless"`
// ShowBrowser runs the headless browser headful (debugging only).
ShowBrowser bool `json:"show-browser" yaml:"show-browser"`
// UseInstalledChrome forces a system-installed Chrome for the headless login.
UseInstalledChrome bool `json:"use-installed-chrome" yaml:"use-installed-chrome"`
// Proxy routes the headless login's browser traffic through this proxy URL.
Proxy string `json:"proxy" yaml:"proxy"`
// Steps, when set, drives an explicit multi-step headless login flow
// (username-first / SSO / consent screens) instead of single-shot detection.
Steps []autologin.LoginStep `json:"steps" yaml:"steps"`
// Recording is a path to a Chrome DevTools Recorder / @puppeteer/replay JSON
// export of a real login. It is compiled into Steps at validation time, so a
// complex login can be authored by recording in a browser instead of writing
// steps by hand. Implies a headless login. Recorded credential literals are
// replaced with {{username}}/{{password}} placeholders.
Recording string `json:"recording" yaml:"recording"`
}
// Validate validates the auto-login configuration.
func (a *AutoLoginConfig) Validate() error {
// A recording compiles into Steps and drives a headless login. Load it first
// so a missing/invalid recording fails fast and so it can supply the login
// URL when one isn't given explicitly.
if a.Recording != "" {
if len(a.Steps) > 0 {
return errkit.New("recording and steps are mutually exclusive for auto-login")
}
steps, err := autologin.StepsFromRecordingFile(a.Recording, a.Username, a.Password)
if err != nil {
return err
}
a.Steps = steps
a.Headless = true
if a.LoginURL == "" {
a.LoginURL = autologin.FirstNavigateURL(steps)
}
}
if a.LoginURL == "" {
return errkit.New("login-url is required for auto-login dynamic secret")
}
if a.Password == "" {
return errkit.New("password is required for auto-login dynamic secret")
}
return nil
}
// AutoLoginRuntimeOptions carries scan-level browser configuration (from
// types.Options) into the auto-login engine so a headless login uses the same
// identity (user-agent, headers), network path (proxy, CDP endpoint) and Chrome
// settings as the scan. Per-secret YAML fields take precedence over these.
type AutoLoginRuntimeOptions struct {
HTTPClient *http.Client
UserAgent string
CustomHeaders map[string]string
Proxy string
CDPEndpoint string
UseInstalledChrome bool
ShowBrowser bool
}
// SetAutoLoginCallback installs the fetch callback that performs the form-based
// auto-login and renders the captured session into the secret. Unlike
// SetLazyFetchCallback, it does not run the template-substitution wrapper: the
// applied secret is built directly from the captured cookies/token, so it is
// simply overwritten on each (re-)authentication.
//
// rt carries optional scan-level browser/runtime options (user-agent, headers,
// proxy, CDP endpoint, HTTP client); pass nil for defaults.
func (d *Dynamic) SetAutoLoginCallback(rt *AutoLoginRuntimeOptions) {
if rt == nil {
rt = &AutoLoginRuntimeOptions{}
}
d.autoLoginClient = rt.HTTPClient
d.fetchCallback = func(d *Dynamic) error {
if d.AutoLogin == nil {
return errkit.New("auto-login callback invoked without auto-login config")
}
cfg := autologin.Config{
LoginURL: d.AutoLogin.LoginURL,
Username: d.AutoLogin.Username,
Password: d.AutoLogin.Password,
UsernameField: d.AutoLogin.UsernameField,
PasswordField: d.AutoLogin.PasswordField,
TokenRegex: d.AutoLogin.TokenRegex,
ExtraFields: kvSliceToMap(d.AutoLogin.ExtraFields),
AllowCrossOriginAction: d.AutoLogin.AllowCrossOriginAction,
Headless: d.AutoLogin.Headless,
ShowBrowser: d.AutoLogin.ShowBrowser || rt.ShowBrowser,
UseInstalledChrome: d.AutoLogin.UseInstalledChrome || rt.UseInstalledChrome,
Proxy: firstNonEmpty(d.AutoLogin.Proxy, rt.Proxy),
CDPEndpoint: rt.CDPEndpoint,
UserAgent: rt.UserAgent,
CustomHeaders: rt.CustomHeaders,
Steps: d.AutoLogin.Steps,
}
var (
session *autologin.Session
err error
)
engine := autoLoginEngineName(d.AutoLogin.Headless)
if d.AutoLogin.Headless {
// The browser engine runs JS, so it handles SPA / multi-step / SSO
// login pages the HTTP form submitter cannot.
session, err = autologin.LoginHeadless(context.Background(), cfg)
} else {
session, err = autologin.Login(context.Background(), d.autoLoginClient, cfg)
}
if err != nil {
// Surface a clear, actionable failure at scan start (prefetch) and on
// every lazy re-authentication.
gologger.Warning().Msgf("auto-login (%s) failed for %s: %s", engine, cfg.LoginURL, err)
return errkit.Wrapf(err, "auto-login (%s) failed for %s", engine, cfg.LoginURL)
}
// Expose extracted values for observability / downstream templating.
d.Extracted = map[string]interface{}{}
if session.CookieHeader != "" {
d.Extracted["cookie"] = session.CookieHeader
}
for _, c := range session.Cookies {
d.Extracted[c.Name] = c.Value
}
if session.Token != "" {
d.Extracted["token"] = session.Token
}
if applyErr := d.applyAutoLoginSession(session); applyErr != nil {
gologger.Warning().Msgf("auto-login (%s) for %s: %s", engine, cfg.LoginURL, applyErr)
return errkit.Wrapf(applyErr, "auto-login (%s) for %s", engine, cfg.LoginURL)
}
gologger.Info().Msgf("auto-login (%s) succeeded for %s: %s", engine, cfg.LoginURL, summarizeSession(session))
return nil
}
}
// autoLoginEngineName returns a human-readable engine label for log messages.
func autoLoginEngineName(headless bool) string {
if headless {
return "headless"
}
return "http"
}
// summarizeSession produces a concise human-readable summary of what an
// auto-login captured, for the success log line.
func summarizeSession(session *autologin.Session) string {
if session == nil {
return "no session"
}
var parts []string
if n := len(session.Cookies); n > 0 {
parts = append(parts, fmt.Sprintf("%d cookie(s)", n))
}
if session.Token != "" {
parts = append(parts, "bearer token")
}
if n := len(session.LocalStorage); n > 0 {
parts = append(parts, fmt.Sprintf("%d localStorage item(s)", n))
}
if n := len(session.SessionStorage); n > 0 {
parts = append(parts, fmt.Sprintf("%d sessionStorage item(s)", n))
}
if len(parts) == 0 {
return "no usable session material"
}
return strings.Join(parts, ", ")
}
// applyAutoLoginSession builds the concrete applied secret(s) from a captured
// session. Cookies are applied as a cookie secret; an extracted token is applied
// as a bearer header. When both are present (common for SPAs that set a session
// cookie *and* a localStorage JWT) both are applied. The applied fields are
// reset first so a re-authentication fully replaces the previous session.
func (d *Dynamic) applyAutoLoginSession(session *autologin.Session) error {
if d.Secret == nil {
d.Secret = &Secret{}
}
// Reset previously applied auth so re-auth replaces (not appends to) it.
d.Headers = nil
d.Cookies = nil
d.Token = ""
d.Secrets = nil
// Stash captured web storage (headless logins only) and any extra derived
// secrets on the shared fetchState so every domain-scoped value-copy and a
// re-authentication observe the same, freshly-captured session. Reset both
// each re-auth.
if d.fetchState != nil {
d.fetchState.webStorageLocal = session.LocalStorage
d.fetchState.webStorageSession = session.SessionStorage
d.fetchState.autoLoginSecrets = nil
}
hasCookies := len(session.Cookies) > 0
hasToken := session.Token != ""
hasStorage := len(session.LocalStorage) > 0 || len(session.SessionStorage) > 0
if !hasCookies && !hasToken && !hasStorage {
return errkit.New("auto-login produced no applicable session (no cookies, token or web storage)")
}
if !hasCookies && !hasToken {
// Storage-only session (e.g. a pure localStorage-JWT SPA): there is no
// HTTP-applicable secret, but the headless engine will replay the storage.
// Leave the secret type empty so it yields no HTTP strategy.
d.Type = ""
return nil
}
switch {
case hasCookies:
d.Type = string(CookiesAuth)
for _, c := range session.Cookies {
d.Cookies = append(d.Cookies, Cookie{
Key: c.Name,
Value: c.Value,
Domain: c.Domain,
Path: c.Path,
Secure: c.Secure,
})
// Expand secret domain scope from captured cookie domains so SSO
// cookies are not silently dropped when LookupURL uses Domains, while
// Apply still filters by each cookie's Domain/Path/Secure.
if host := cookieDomainHost(c.Domain); host != "" && !domainListContains(d.Domains, host) {
d.Domains = append(d.Domains, host)
}
}
if hasToken && d.fetchState != nil {
// Apply the token as an additional bearer header secret on the shared
// fetchState so it reaches every domain the secret is scoped to.
d.fetchState.autoLoginSecrets = append(d.fetchState.autoLoginSecrets, &Secret{
Type: string(BearerTokenAuth),
Domains: d.Domains,
Token: session.Token,
})
}
default: // token only
d.Type = string(BearerTokenAuth)
d.Token = session.Token
}
return nil
}
// firstNonEmpty returns the first non-empty string argument.
func firstNonEmpty(vals ...string) string {
for _, v := range vals {
if v != "" {
return v
}
}
return ""
}
// cookieDomainHost normalizes a Set-Cookie Domain attribute to a hostname
// suitable for Secret.Domains lookup (strips a leading dot).
func cookieDomainHost(domain string) string {
domain = strings.TrimSpace(domain)
domain = strings.TrimPrefix(domain, ".")
return domain
}
func domainListContains(domains []string, host string) bool {
host = strings.ToLower(host)
for _, d := range domains {
if strings.EqualFold(d, host) {
return true
}
}
return false
}
// kvSliceToMap converts a slice of KV pairs to a map for the autologin engine.
func kvSliceToMap(kvs []KV) map[string]string {
if len(kvs) == 0 {
return nil
}
m := make(map[string]string, len(kvs))
for _, kv := range kvs {
m[kv.Key] = kv.Value
}
return m
}
-319
View File
@@ -1,319 +0,0 @@
package authx
import (
"fmt"
"net/http"
"os"
"path/filepath"
"sync"
"sync/atomic"
"testing"
"github.com/projectdiscovery/nuclei/v3/pkg/authprovider/autologin"
"github.com/stretchr/testify/require"
)
func TestAutoLoginConfig_Validate_Recording(t *testing.T) {
recording := `{"steps": [
{"type": "navigate", "url": "https://app.example.com/login"},
{"type": "change", "value": "dave", "selectors": [["#user"]]},
{"type": "change", "value": "secret", "selectors": [["#pass"]]},
{"type": "click", "selectors": [["#submit"]]}
]}`
path := filepath.Join(t.TempDir(), "login.flow.json")
require.NoError(t, os.WriteFile(path, []byte(recording), 0o600))
// LoginURL intentionally omitted: it must be derived from the recording.
cfg := &AutoLoginConfig{Recording: path, Username: "dave", Password: "secret"}
require.NoError(t, cfg.Validate())
require.True(t, cfg.Headless, "a recording must force a headless login")
require.Equal(t, "https://app.example.com/login", cfg.LoginURL, "login URL derived from first navigate step")
require.Equal(t, []autologin.LoginStep{
{Action: "navigate", Value: "https://app.example.com/login"},
{Action: "fill", Selector: "#user", Value: "{{username}}"},
{Action: "fill", Selector: "#pass", Value: "{{password}}"},
{Action: "click", Selector: "#submit"},
}, cfg.Steps, "recording must compile to placeholder-parameterized steps")
}
func TestAutoLoginConfig_Validate_RecordingMissingFile(t *testing.T) {
cfg := &AutoLoginConfig{Recording: filepath.Join(t.TempDir(), "nope.json"), Password: "x"}
require.Error(t, cfg.Validate(), "a missing recording file must fail validation")
}
func TestAutoLoginConfig_Validate_RecordingAndStepsExclusive(t *testing.T) {
path := filepath.Join(t.TempDir(), "login.flow.json")
require.NoError(t, os.WriteFile(path, []byte(`{"steps":[{"type":"navigate","url":"https://example.com/login"}]}`), 0o600))
cfg := &AutoLoginConfig{
Recording: path,
Password: "x",
Steps: []autologin.LoginStep{{Action: "click", Selector: "#x"}},
}
require.Error(t, cfg.Validate(), "recording and steps must be mutually exclusive")
}
func newAutoLoginDynamic() *Dynamic {
return &Dynamic{
Secret: &Secret{Domains: []string{"app.example.com"}},
AutoLogin: &AutoLoginConfig{LoginURL: "https://app.example.com/login", Password: "x"},
}
}
func TestApplyAutoLoginSession_CookiesOnly(t *testing.T) {
d := newAutoLoginDynamic()
err := d.applyAutoLoginSession(&autologin.Session{
Cookies: []*http.Cookie{{Name: "session", Value: "abc"}},
})
require.NoError(t, err)
require.Equal(t, string(CookiesAuth), d.Type)
require.Len(t, d.Cookies, 1)
require.Equal(t, "session", d.Cookies[0].Key)
require.Empty(t, d.Secrets, "no extra bearer secret when there is no token")
}
func TestApplyAutoLoginSession_TokenOnly(t *testing.T) {
d := newAutoLoginDynamic()
err := d.applyAutoLoginSession(&autologin.Session{Token: "jwt-123"})
require.NoError(t, err)
require.Equal(t, string(BearerTokenAuth), d.Type)
require.Equal(t, "jwt-123", d.Token)
require.Empty(t, d.Secrets)
}
func TestApplyAutoLoginSession_CookiesAndToken(t *testing.T) {
d := newAutoLoginDynamic()
require.NoError(t, d.Validate())
err := d.applyAutoLoginSession(&autologin.Session{
Cookies: []*http.Cookie{{Name: "session", Value: "abc"}},
Token: "jwt-123",
})
require.NoError(t, err)
require.Equal(t, string(CookiesAuth), d.Type)
// The bearer lives on the shared fetchState (so it reaches every scoped
// domain), not on the per-copy Secrets slice.
require.Empty(t, d.Secrets, "bearer must not live on the per-copy Secrets slice")
require.Len(t, d.fetchState.autoLoginSecrets, 1, "token should be applied as an additional bearer secret")
require.Equal(t, string(BearerTokenAuth), d.fetchState.autoLoginSecrets[0].Type)
require.Equal(t, "jwt-123", d.fetchState.autoLoginSecrets[0].Token)
require.Equal(t, []string{"app.example.com"}, d.fetchState.autoLoginSecrets[0].Domains)
}
func TestApplyAutoLoginSession_Empty(t *testing.T) {
d := newAutoLoginDynamic()
err := d.applyAutoLoginSession(&autologin.Session{})
require.Error(t, err)
}
func TestSummarizeSession(t *testing.T) {
require.Equal(t, "no session", summarizeSession(nil))
require.Equal(t, "no usable session material", summarizeSession(&autologin.Session{}))
require.Equal(t, "2 cookie(s)", summarizeSession(&autologin.Session{
Cookies: []*http.Cookie{{Name: "a"}, {Name: "b"}},
}))
require.Equal(t, "1 cookie(s), bearer token, 2 localStorage item(s)", summarizeSession(&autologin.Session{
Cookies: []*http.Cookie{{Name: "a"}},
Token: "jwt",
LocalStorage: map[string]string{"x": "1", "y": "2"},
}))
}
func TestAutoLoginEngineName(t *testing.T) {
require.Equal(t, "headless", autoLoginEngineName(true))
require.Equal(t, "http", autoLoginEngineName(false))
}
// TestDynamic_WebStorage_ConcurrentReauth exercises the locking contract that
// storage replay relies on: readers (the headless engine) call WebStorage while
// re-authentication concurrently rewrites the captured storage on the shared
// fetchState. Must be run under -race.
func TestDynamic_WebStorage_ConcurrentReauth(t *testing.T) {
d := newAutoLoginDynamic()
d.ReauthStatusCodes = []int{401}
require.NoError(t, d.Validate())
var gen atomic.Int64
// Each (re)auth captures a fresh storage map, simulating a new session.
d.fetchCallback = func(dyn *Dynamic) error {
n := gen.Add(1)
// Cookie + token + storage so every shared-state write path (Secret,
// fetchState.autoLoginSecrets, webStorage) is exercised under -race.
return dyn.applyAutoLoginSession(&autologin.Session{
Cookies: []*http.Cookie{{Name: "session", Value: fmt.Sprintf("v%d", n)}},
Token: fmt.Sprintf("token-%d", n),
LocalStorage: map[string]string{"jwt": fmt.Sprintf("token-%d", n)},
})
}
// Share the session across value-copies, as the real provider does.
strategy := &DynamicAuthStrategy{Dynamic: *d}
var wg sync.WaitGroup
const workers = 8
const iterations = 200
for w := 0; w < workers; w++ {
wg.Add(1)
go func(id int) {
defer wg.Done()
for i := 0; i < iterations; i++ {
switch id % 4 {
case 0:
// Force a re-auth (rewrites the storage map under the write lock).
strategy.Dynamic.MarkStale()
_ = strategy.Dynamic.Fetch(false)
case 1:
// Read + iterate storage the way resolveBrowserStorage does.
local, session := strategy.WebStorage()
for k, v := range local {
_ = k + v
}
for k, v := range session {
_ = k + v
}
// Also materialize strategies (reads shared autoLoginSecrets).
_ = strategy.Dynamic.GetStrategies()
case 2:
strategy.Dynamic.ApplyStrategies(func(AuthStrategy) {})
default:
_ = strategy.Dynamic.NotifyResponse(401, 0)
}
}
}(w)
}
wg.Wait()
// After the storm, a fresh fetch must still yield a consistent session.
local, _ := strategy.WebStorage()
require.NotEmpty(t, local["jwt"], "storage must remain populated after concurrent re-auth")
}
// TestDynamicClose_WipesCapturedSession verifies that Close drops the in-memory
// session captured by an auto-login (cookies, bearer token, web storage and
// extracted values) without contacting the target.
func TestDynamicClose_WipesCapturedSession(t *testing.T) {
d := newAutoLoginDynamic()
require.NoError(t, d.Validate())
d.fetchCallback = func(dyn *Dynamic) error {
return dyn.applyAutoLoginSession(&autologin.Session{
Cookies: []*http.Cookie{{Name: "session", Value: "abc"}},
Token: "jwt-123",
LocalStorage: map[string]string{"jwt": "jwt-123"},
})
}
require.NoError(t, d.Fetch(false))
d.Extracted = map[string]interface{}{"token": "jwt-123"}
// Sanity: the session is populated before Close.
require.NotEmpty(t, d.Cookies)
local, _ := d.WebStorage()
require.NotEmpty(t, local["jwt"])
d.Close()
require.Empty(t, d.Cookies, "cookies must be wiped")
require.Empty(t, d.Token, "token must be wiped")
require.Empty(t, d.Headers, "headers must be wiped")
require.Nil(t, d.Extracted, "extracted values must be cleared")
require.False(t, d.IsExpired(), "a closed session reports not-expired (unfetched)")
require.Equal(t, []string{"app.example.com"}, d.Domains, "scoping domains must be preserved")
localAfter, sessionAfter := d.fetchState.webStorageLocal, d.fetchState.webStorageSession
require.Nil(t, localAfter, "captured local storage must be cleared")
require.Nil(t, sessionAfter, "captured session storage must be cleared")
}
// TestAutoLogin_MultiDomain_BearerSharedAcrossCopies reproduces a sharing bug:
// the auto-login session (cookie + bearer token) must be applied on *every*
// domain the secret is scoped to, not just the one whose DynamicAuthStrategy
// copy happened to trigger the login. init() creates one value-copy per domain,
// all sharing *Secret and *fetchState; the extra bearer secret must be visible
// to sibling copies too.
func TestAutoLogin_MultiDomain_BearerSharedAcrossCopies(t *testing.T) {
d := &Dynamic{
Secret: &Secret{Domains: []string{"app.example.com", "api.example.com"}},
AutoLogin: &AutoLoginConfig{LoginURL: "https://app.example.com/login", Password: "x"},
}
require.NoError(t, d.Validate())
d.fetchCallback = func(dyn *Dynamic) error {
return dyn.applyAutoLoginSession(&autologin.Session{
Cookies: []*http.Cookie{{Name: "session", Value: "abc"}},
Token: "jwt-123",
})
}
// Mirror init(): one copy per domain, sharing the pointers.
copyApp := &DynamicAuthStrategy{Dynamic: *d}
copyAPI := &DynamicAuthStrategy{Dynamic: *d}
// The app-domain copy triggers the login.
appStrategies := copyApp.Dynamic.GetStrategies()
require.Len(t, appStrategies, 2, "app domain must get cookie + bearer")
// The api-domain copy (login already done, shared fetchState) must ALSO see
// both the cookie and the bearer token.
apiStrategies := copyAPI.Dynamic.GetStrategies()
require.Len(t, apiStrategies, 2, "api domain must also get cookie + bearer (shared session)")
// Concretely, the bearer must be applied to a request on the api domain.
req, _ := http.NewRequest(http.MethodGet, "https://api.example.com/data", nil)
copyAPI.Apply(req)
require.Equal(t, "Bearer jwt-123", req.Header.Get("Authorization"), "bearer token must reach the api domain")
require.NotEmpty(t, req.Header.Get("Cookie"), "session cookie must reach the api domain")
}
func TestApplyAutoLoginSession_WebStorageCaptured(t *testing.T) {
d := newAutoLoginDynamic()
require.NoError(t, d.Validate())
d.SetAutoLoginCallback(nil)
// Stub the fetch to bypass the network and simulate a headless capture.
d.fetchCallback = func(dyn *Dynamic) error {
return dyn.applyAutoLoginSession(&autologin.Session{
Cookies: []*http.Cookie{{Name: "session", Value: "abc"}},
LocalStorage: map[string]string{"jwt": "eyJ..."},
SessionStorage: map[string]string{"csrf": "tok"},
})
}
local, session := d.WebStorage()
require.Equal(t, map[string]string{"jwt": "eyJ..."}, local)
require.Equal(t, map[string]string{"csrf": "tok"}, session)
// And the DynamicAuthStrategy must surface the same storage to the engine.
strategy := &DynamicAuthStrategy{Dynamic: *d}
l2, s2 := strategy.WebStorage()
require.Equal(t, "eyJ...", l2["jwt"])
require.Equal(t, "tok", s2["csrf"])
}
func TestApplyAutoLoginSession_StorageOnly(t *testing.T) {
d := newAutoLoginDynamic()
require.NoError(t, d.Validate())
// Storage-only session (pure localStorage-JWT SPA): no HTTP secret, but the
// session is still valid because storage will be replayed by the engine.
err := d.applyAutoLoginSession(&autologin.Session{
LocalStorage: map[string]string{"jwt": "x"},
})
require.NoError(t, err)
require.Empty(t, d.Type, "storage-only session yields no HTTP strategy")
require.Equal(t, map[string]string{"jwt": "x"}, d.fetchState.webStorageLocal)
}
func TestApplyAutoLoginSession_ReauthReplaces(t *testing.T) {
d := newAutoLoginDynamic()
require.NoError(t, d.Validate())
// First auth: cookies + token (creates an extra bearer secret).
require.NoError(t, d.applyAutoLoginSession(&autologin.Session{
Cookies: []*http.Cookie{{Name: "session", Value: "v1"}},
Token: "jwt-1",
}))
require.Len(t, d.fetchState.autoLoginSecrets, 1)
// Re-auth: cookies only -> the previous bearer secret and cookies must be
// fully replaced, not appended.
require.NoError(t, d.applyAutoLoginSession(&autologin.Session{
Cookies: []*http.Cookie{{Name: "session", Value: "v2"}},
}))
require.Empty(t, d.fetchState.autoLoginSecrets, "stale bearer secret must be cleared on re-auth")
require.Len(t, d.Cookies, 1)
require.Equal(t, "v2", d.Cookies[0].Value, "cookie value must be refreshed")
}
+5 -76
View File
@@ -2,9 +2,7 @@ package authx
import (
"net/http"
"net/url"
"slices"
"strings"
"github.com/projectdiscovery/retryablehttp-go"
)
@@ -27,15 +25,9 @@ func NewCookiesAuthStrategy(data *Secret) *CookiesAuthStrategy {
func (s *CookiesAuthStrategy) Apply(req *http.Request) {
var names []string
for _, cookie := range s.Data.Cookies {
if !cookieAppliesToRequest(cookie, req.URL) {
continue
}
c := &http.Cookie{
Name: cookie.Key,
Value: cookie.Value,
Domain: cookie.Domain,
Path: cookie.Path,
Secure: cookie.Secure,
Name: cookie.Key,
Value: cookie.Value,
}
req.AddCookie(c)
names = append(names, cookie.Key)
@@ -46,15 +38,8 @@ func (s *CookiesAuthStrategy) Apply(req *http.Request) {
// ApplyOnRR applies the cookies auth strategy to the retryable request
func (s *CookiesAuthStrategy) ApplyOnRR(req *retryablehttp.Request) {
existingCookies := req.Cookies()
var reqURL *url.URL
if req.URL != nil {
reqURL = req.URL.URL
}
for _, newCookie := range s.Data.Cookies {
if !cookieAppliesToRequest(newCookie, reqURL) {
continue
}
for i, existing := range existingCookies {
if existing.Name == newCookie.Key {
existingCookies = slices.Delete(existingCookies, i, i+1)
@@ -68,70 +53,14 @@ func (s *CookiesAuthStrategy) ApplyOnRR(req *retryablehttp.Request) {
for _, cookie := range existingCookies {
req.AddCookie(cookie)
}
// Add new cookies that match this request's scope
// Add new cookies
var names []string
for _, cookie := range s.Data.Cookies {
if !cookieAppliesToRequest(cookie, reqURL) {
continue
}
req.AddCookie(&http.Cookie{
Name: cookie.Key,
Value: cookie.Value,
Domain: cookie.Domain,
Path: cookie.Path,
Secure: cookie.Secure,
Name: cookie.Key,
Value: cookie.Value,
})
names = append(names, cookie.Key)
}
markCookieAuth(req.Request, names)
}
// cookieAppliesToRequest reports whether a captured/configured cookie should be
// attached to reqURL. Unscoped cookies (empty Domain) always apply, preserving
// backward compatibility for hand-written secrets.
func cookieAppliesToRequest(cookie Cookie, reqURL *url.URL) bool {
if reqURL == nil {
return cookie.Domain == ""
}
if cookie.Secure && !strings.EqualFold(reqURL.Scheme, "https") {
return false
}
if cookie.Domain != "" && !cookieDomainMatchesHost(cookie.Domain, reqURL.Hostname()) {
return false
}
path := cookie.Path
if path == "" {
path = "/"
}
reqPath := reqURL.EscapedPath()
if reqPath == "" {
reqPath = "/"
}
return cookiePathMatches(path, reqPath)
}
// cookieDomainMatchesHost implements a conservative Domain attribute match:
// exact host equality, or a leading-dot domain matching a subdomain suffix.
func cookieDomainMatchesHost(domain, host string) bool {
domain = strings.TrimSpace(strings.ToLower(domain))
host = strings.TrimSpace(strings.ToLower(host))
if domain == "" || host == "" {
return false
}
domain = strings.TrimPrefix(domain, ".")
if host == domain {
return true
}
return strings.HasSuffix(host, "."+domain)
}
// cookiePathMatches implements the RFC 6265 path-match algorithm.
func cookiePathMatches(cookiePath, reqPath string) bool {
if !strings.HasPrefix(reqPath, cookiePath) {
return false
}
if len(reqPath) == len(cookiePath) || strings.HasSuffix(cookiePath, "/") {
return true
}
return reqPath[len(cookiePath)] == '/'
}
@@ -1,60 +0,0 @@
package authx
import (
"net/http"
"testing"
"github.com/stretchr/testify/require"
)
func TestCookieAppliesToRequest_Scope(t *testing.T) {
t.Run("unscoped cookies always apply", func(t *testing.T) {
req, _ := http.NewRequest(http.MethodGet, "https://app.example.com/dashboard", nil)
require.True(t, cookieAppliesToRequest(Cookie{Key: "s", Value: "1"}, req.URL))
})
t.Run("domain and path filter", func(t *testing.T) {
c := Cookie{Key: "s", Value: "1", Domain: ".example.com", Path: "/app"}
ok, _ := http.NewRequest(http.MethodGet, "https://api.example.com/app/x", nil)
badHost, _ := http.NewRequest(http.MethodGet, "https://evil.com/app", nil)
badPath, _ := http.NewRequest(http.MethodGet, "https://api.example.com/other", nil)
require.True(t, cookieAppliesToRequest(c, ok.URL))
require.False(t, cookieAppliesToRequest(c, badHost.URL))
require.False(t, cookieAppliesToRequest(c, badPath.URL))
})
t.Run("secure cookies skip http", func(t *testing.T) {
c := Cookie{Key: "s", Value: "1", Secure: true}
httpsReq, _ := http.NewRequest(http.MethodGet, "https://app.example.com/", nil)
httpReq, _ := http.NewRequest(http.MethodGet, "http://app.example.com/", nil)
require.True(t, cookieAppliesToRequest(c, httpsReq.URL))
require.False(t, cookieAppliesToRequest(c, httpReq.URL))
})
}
func TestCookiesAuthStrategy_PreservesScopedCookies(t *testing.T) {
s := NewCookiesAuthStrategy(&Secret{
Cookies: []Cookie{
{Key: "session", Value: "app", Domain: "app.example.com", Path: "/"},
{Key: "session", Value: "idp", Domain: "login.example.com", Path: "/"},
{Key: "csrf", Value: "x", Domain: "app.example.com", Path: "/login"},
},
})
appReq, _ := http.NewRequest(http.MethodGet, "https://app.example.com/dashboard", nil)
s.Apply(appReq)
got := map[string]string{}
for _, c := range appReq.Cookies() {
got[c.Name] = c.Value
}
require.Equal(t, "app", got["session"], "app-scoped session must apply")
require.NotContains(t, got, "csrf", "login-path csrf must not apply to /dashboard")
idpReq, _ := http.NewRequest(http.MethodGet, "https://login.example.com/", nil)
s.Apply(idpReq)
got = map[string]string{}
for _, c := range idpReq.Cookies() {
got[c.Name] = c.Value
}
require.Equal(t, "idp", got["session"], "idp-scoped session must apply on idp host")
}
+29 -501
View File
@@ -3,13 +3,11 @@ package authx
import (
"bytes"
"fmt"
"net/http"
"runtime"
"strconv"
"strings"
"sync"
"sync/atomic"
"time"
"github.com/projectdiscovery/gologger"
"github.com/projectdiscovery/nuclei/v3/pkg/protocols/common/replacer"
@@ -20,59 +18,16 @@ import (
type LazyFetchSecret func(d *Dynamic) error
// fetchState holds the session state for a dynamic secret and makes it safe
// for concurrent use. It is stored as a pointer in Dynamic so that value copies
// (e.g. the one held inside DynamicAuthStrategy) share the same session.
//
// Unlike a plain sync.Once, this state machine supports re-authentication: the
// login flow can be re-run when the session goes stale (explicitly marked,
// detected from a response status, or after a configured refresh interval).
//
// Locking contract:
// - Fetch/Refresh take the write lock while (re)running the login callback.
// - The dynamic apply path (ApplyStrategies) holds the read lock while reading
// the rendered secret, so request-time reads never race a concurrent re-auth.
// fetchState holds the sync.Once and error for thread-safe fetching.
// This is stored as a pointer in Dynamic so that value copies share the same state.
type fetchState struct {
mu sync.RWMutex
fetched bool // whether the login callback has completed at least once
attempted bool // whether the login callback has run, even if it failed
stale bool // whether the session must be re-authenticated on next fetch
fetchedAt time.Time // time of the last successful/attempted fetch
err error // error from the most recent fetch attempt
// generation increments on every successful login so a late 401 from an
// older session can be ignored after a concurrent re-authentication.
generation uint64
// webStorageLocal/webStorageSession hold browser web storage captured by a
// headless auto-login, to be replayed into headless scan pages. Stored here
// (on the shared fetchState pointer) so all Dynamic value-copies and a
// re-authentication observe the same, freshly-captured storage.
webStorageLocal map[string]string
webStorageSession map[string]string
// autoLoginSecrets holds extra applied secrets derived from an auto-login
// session (e.g. a bearer token captured alongside a session cookie). They
// live on the shared fetchState rather than the per-copy Dynamic.Secrets
// slice so every domain-scoped value-copy applies the same captured session.
autoLoginSecrets []*Secret
once sync.Once
err error
// owner is the goroutine running the fetch callback. Nested Fetch/GetStrategies
// on that goroutine must not wait on the write lock (it would deadlock).
// on that goroutine must not wait on once (sync.Once deadlocks on re-entry).
owner atomic.Int64
// answered is set once a response that is not a reauth status code is seen
// for the current session. It is reset on every successful login.
answered atomic.Bool
// statusReauths counts consecutive status-triggered re-authentications whose
// session never got a response other than a reauth status code. Past
// maxStatusReauths the session is kept, so an app that rejects every
// request does not get one login per request (account lockout, WAF).
statusReauths int
// reauthCapLogged is set once the cap warning was printed for the current
// streak, so it is printed once and not on every response.
reauthCapLogged bool
}
// maxStatusReauths caps consecutive re-authentications triggered by reauth
// status codes when the fresh session still gets only reauth status codes.
const maxStatusReauths = 2
var (
_ json.Unmarshaler = &Dynamic{}
)
@@ -82,44 +37,16 @@ var (
// ex: username and password are dynamic secrets, the actual secret is the token obtained
// after authenticating with the username and password
type Dynamic struct {
*Secret `yaml:",inline"` // this is a static secret that will be generated after the dynamic secret is resolved
Secrets []*Secret `yaml:"secrets"`
TemplatePath string `json:"template" yaml:"template"`
Variables []KV `json:"variables" yaml:"variables"`
Input string `json:"input" yaml:"input"` // (optional) target for the dynamic secret
Extracted map[string]interface{} `json:"-" yaml:"-"` // extracted values from the dynamic secret
// AutoLogin, when set, replaces the template-based login (TemplatePath) with
// a turnkey form-based auto-login: nuclei fetches the login page, detects the
// form and submits the supplied credentials, capturing the session
// automatically. It is mutually exclusive with TemplatePath.
AutoLogin *AutoLoginConfig `json:"auto-login" yaml:"auto-login"`
// RefreshInterval, when set (e.g. "15m"), causes the session to be
// re-authenticated automatically once the rendered secret is older than the
// interval. When empty/zero the session is fetched once and never expires by time.
RefreshInterval string `json:"refresh-interval" yaml:"refresh-interval"`
// ReauthStatusCodes is the set of HTTP response status codes that indicate
// the session has expired (e.g. [401, 403]). When a matching response is
// observed, the session is marked stale and re-authenticated before the next
// request. When empty, response-triggered re-authentication is disabled.
ReauthStatusCodes []int `json:"reauth-status-codes" yaml:"reauth-status-codes"`
fetchCallback LazyFetchSecret `json:"-" yaml:"-"`
*Secret `yaml:",inline"` // this is a static secret that will be generated after the dynamic secret is resolved
Secrets []*Secret `yaml:"secrets"`
TemplatePath string `json:"template" yaml:"template"`
Variables []KV `json:"variables" yaml:"variables"`
Input string `json:"input" yaml:"input"` // (optional) target for the dynamic secret
Extracted map[string]interface{} `json:"-" yaml:"-"` // extracted values from the dynamic secret
fetchCallback LazyFetchSecret `json:"-" yaml:"-"`
// fetchState is shared across value-copies of Dynamic (e.g., inside DynamicAuthStrategy).
// It must be initialized via Validate() before calling Fetch().
fetchState *fetchState `json:"-" yaml:"-"`
// refreshInterval is the parsed form of RefreshInterval.
refreshInterval time.Duration `json:"-" yaml:"-"`
// autoLoginClient is an optional template HTTP client (proxy/TLS) reused for
// auto-login requests. nil means use the autologin engine's defaults.
autoLoginClient *http.Client `json:"-" yaml:"-"`
// origSecret/origSecrets hold pristine copies of the secret templates (with
// their {{var}} placeholders intact) captured before the first fetch. They are
// used to re-render the secret on each re-authentication, since substitution
// is destructive (placeholders are replaced with concrete values in place).
origSecret *Secret `json:"-" yaml:"-"`
origSecrets []*Secret `json:"-" yaml:"-"`
}
func (d *Dynamic) GetDomainAndDomainRegex() ([]string, []string) {
@@ -160,43 +87,12 @@ func (d *Dynamic) Validate() error {
// NOTE: Validate() must not be called concurrently with Fetch()/GetStrategies().
// Re-validating resets fetch state and allows re-fetching.
d.fetchState = &fetchState{}
d.refreshInterval = 0
// Auto-login is a template-free alternative to TemplatePath. When set, it is
// validated on its own and the template/variables requirements do not apply;
// domains (used for scoping the captured session) are still required.
if d.AutoLogin != nil {
if d.TemplatePath != "" {
return errkit.New("auto-login and template are mutually exclusive for dynamic secret")
}
if err := d.AutoLogin.Validate(); err != nil {
return err
}
domains, domainsRegex := d.GetDomainAndDomainRegex()
if len(domains) == 0 && len(domainsRegex) == 0 {
return errkit.New("domains or domains-regex are required for auto-login dynamic secret")
}
if err := d.parseRefreshInterval(); err != nil {
return err
}
// The applied secret is built from the captured session at fetch time, so
// skip the (pre-fetch) Secret.Validate which would reject the still-empty
// cookies/token. We only need its domains, captured above.
if d.Secret != nil {
d.skipCookieParse = true
}
return nil
}
if d.TemplatePath == "" {
return errkit.New(" template-path is required for dynamic secret")
}
if len(d.Variables) == 0 {
return errkit.New("variables are required for dynamic secret")
}
if err := d.parseRefreshInterval(); err != nil {
return err
}
if d.Secret != nil {
d.skipCookieParse = true // skip cookie parsing in dynamic secrets during validation
@@ -213,31 +109,8 @@ func (d *Dynamic) Validate() error {
return nil
}
// parseRefreshInterval parses RefreshInterval into refreshInterval.
func (d *Dynamic) parseRefreshInterval() error {
if d.RefreshInterval == "" {
return nil
}
dur, err := time.ParseDuration(d.RefreshInterval)
if err != nil {
return errkit.Newf("invalid refresh-interval %q for dynamic secret: %s", d.RefreshInterval, err)
}
if dur < 0 {
return errkit.New("refresh-interval cannot be negative for dynamic secret")
}
d.refreshInterval = dur
return nil
}
// SetLazyFetchCallback sets the lazy fetch callback for the dynamic secret.
//
// The provided callback is responsible for running the login flow and populating
// d.Extracted. The wrapper here renders the secret templates with the freshly
// extracted values. It snapshots the pristine secret templates on first use so
// that re-authentication can re-render them with new values (template
// substitution is destructive and would otherwise consume the placeholders).
// SetLazyFetchCallback sets the lazy fetch callback for the dynamic secret
func (d *Dynamic) SetLazyFetchCallback(callback LazyFetchSecret) {
d.snapshotTemplates()
d.fetchCallback = func(d *Dynamic) error {
err := callback(d)
if err != nil {
@@ -248,16 +121,12 @@ func (d *Dynamic) SetLazyFetchCallback(callback LazyFetchSecret) {
}
if d.Secret != nil {
restoreSecretTemplate(d.Secret, d.origSecret)
if err := d.applyValuesToSecret(d.Secret); err != nil {
return err
}
}
for i, secret := range d.Secrets {
if i < len(d.origSecrets) {
restoreSecretTemplate(secret, d.origSecrets[i])
}
for _, secret := range d.Secrets {
if err := d.applyValuesToSecret(secret); err != nil {
return err
}
@@ -266,47 +135,6 @@ func (d *Dynamic) SetLazyFetchCallback(callback LazyFetchSecret) {
}
}
// snapshotTemplates captures pristine copies of the secret templates (with their
// {{var}} placeholders intact) so they can be re-rendered on re-authentication.
func (d *Dynamic) snapshotTemplates() {
if d.Secret != nil {
d.origSecret = cloneSecretTemplate(d.Secret)
}
if len(d.Secrets) > 0 {
d.origSecrets = make([]*Secret, len(d.Secrets))
for i, secret := range d.Secrets {
d.origSecrets[i] = cloneSecretTemplate(secret)
}
}
}
// cloneSecretTemplate deep-copies the templated fields of a secret so the
// original placeholders survive destructive in-place substitution.
func cloneSecretTemplate(s *Secret) *Secret {
if s == nil {
return nil
}
clone := *s
clone.Headers = append([]KV(nil), s.Headers...)
clone.Cookies = append([]Cookie(nil), s.Cookies...)
clone.Params = append([]KV(nil), s.Params...)
return &clone
}
// restoreSecretTemplate resets the templated fields of live back to the pristine
// template captured in snap, so the next substitution starts from placeholders.
func restoreSecretTemplate(live, snap *Secret) {
if live == nil || snap == nil {
return
}
live.Headers = append([]KV(nil), snap.Headers...)
live.Cookies = append([]Cookie(nil), snap.Cookies...)
live.Params = append([]KV(nil), snap.Params...)
live.Username = snap.Username
live.Password = snap.Password
live.Token = snap.Token
}
func (d *Dynamic) applyValuesToSecret(secret *Secret) error {
// evaluate headers
for i, header := range secret.Headers {
@@ -380,89 +208,20 @@ func (d *Dynamic) GetStrategies() []AuthStrategy {
// does not terminate the entire scan process.
_ = d.Fetch(false)
if d.fetchState == nil {
return nil
}
d.fetchState.mu.RLock()
defer d.fetchState.mu.RUnlock()
if d.fetchState.err != nil {
if d.fetchState != nil && d.fetchState.err != nil {
return nil
}
var strategies []AuthStrategy
if d.Secret != nil {
if s := d.GetStrategy(); s != nil {
strategies = append(strategies, s)
}
strategies = append(strategies, d.GetStrategy())
}
for _, secret := range d.Secrets {
if s := secret.GetStrategy(); s != nil {
strategies = append(strategies, s)
}
}
for _, secret := range d.fetchState.autoLoginSecrets {
if s := secret.GetStrategy(); s != nil {
strategies = append(strategies, s)
}
strategies = append(strategies, secret.GetStrategy())
}
return strategies
}
// ApplyStrategies fetches (or re-authenticates) the session if needed and then
// applies each resolved auth strategy via the provided apply func while holding
// the read lock. This guarantees the rendered secret is never read at request
// time while a concurrent re-authentication is rewriting it.
// The returned generation identifies the session that was applied (0 if none).
func (d *Dynamic) ApplyStrategies(apply func(AuthStrategy)) uint64 {
// A login callback can issue requests that pass through auth lookup again.
// Do not apply this still-unresolved dynamic secret or try to take its read
// lock while the same goroutine owns the fetch write lock.
if d.fetchingOnThisGoroutine() {
return 0
}
// Fetch (re)authenticates under the write lock if the session is missing or stale.
_ = d.Fetch(false)
if d.fetchState == nil {
return 0
}
d.fetchState.mu.RLock()
defer d.fetchState.mu.RUnlock()
if d.fetchState.err != nil {
return 0
}
if d.Secret != nil {
if s := d.GetStrategy(); s != nil {
apply(s)
}
}
for _, secret := range d.Secrets {
if s := secret.GetStrategy(); s != nil {
apply(s)
}
}
for _, secret := range d.fetchState.autoLoginSecrets {
if s := secret.GetStrategy(); s != nil {
apply(s)
}
}
return d.fetchState.generation
}
// SessionGeneration returns the current successful-login generation, or 0.
func (d *Dynamic) SessionGeneration() uint64 {
if d.fetchState == nil {
return 0
}
d.fetchState.mu.RLock()
defer d.fetchState.mu.RUnlock()
if !d.fetchState.fetched {
return 0
}
return d.fetchState.generation
}
// Fetch fetches the dynamic secret, (re)running the login flow when the session
// has not been fetched yet or has gone stale (explicitly marked, expired by
// refresh-interval, or invalidated by an observed response status code).
// Fetch fetches the dynamic secret
// if isFatal is true, it will stop the execution if the secret could not be fetched
func (d *Dynamic) Fetch(isFatal bool) error {
if d.fetchState == nil {
@@ -476,205 +235,20 @@ func (d *Dynamic) Fetch(isFatal bool) error {
return d.fetchState.err
}
d.fetchState.mu.Lock()
if d.needsFetchLocked() {
d.runFetchLocked()
}
err := d.fetchState.err
d.fetchState.mu.Unlock()
if err != nil && isFatal {
gologger.Fatal().Msgf("Could not fetch dynamic secret: %s\n", err)
}
return err
}
// needsFetchLocked reports whether the login callback must run. A failed login
// is not retried on every request, as that would send one login per scan
// request (account lockout, WAF bans). It is retried only once the refresh
// interval has passed since the failed attempt. The caller must hold the lock.
func (d *Dynamic) needsFetchLocked() bool {
if !d.fetchState.fetched {
if !d.fetchState.attempted {
return true
d.fetchState.once.Do(func() {
d.fetchState.owner.Store(goroutineID())
defer d.fetchState.owner.Store(0)
if d.fetchCallback == nil {
d.fetchState.err = errkit.New("dynamic secret fetch callback not set: call SetLazyFetchCallback() before Fetch()")
return
}
return d.refreshInterval > 0 && time.Since(d.fetchState.fetchedAt) > d.refreshInterval
}
return d.isExpiredLocked()
}
d.fetchState.err = d.fetchCallback(d)
})
// Refresh forces an immediate re-authentication, re-running the login flow
// regardless of the current session state.
func (d *Dynamic) Refresh(isFatal bool) error {
if d.fetchState == nil {
if isFatal {
gologger.Fatal().Msgf("Could not refresh dynamic secret: Validate() must be called before Refresh()")
}
return errkit.New("dynamic secret not validated: call Validate() before Refresh()")
if d.fetchState.err != nil && isFatal {
gologger.Fatal().Msgf("Could not fetch dynamic secret: %s\n", d.fetchState.err)
}
d.fetchState.mu.Lock()
d.fetchState.fetched = false
d.fetchState.stale = false
d.runFetchLocked()
err := d.fetchState.err
d.fetchState.mu.Unlock()
if err != nil && isFatal {
gologger.Fatal().Msgf("Could not refresh dynamic secret: %s\n", err)
}
return err
}
// MarkStale marks the session for re-authentication on the next fetch. It is
// cheap and does not perform any network I/O; the actual re-auth happens on the
// next Fetch/ApplyStrategies call so it never runs in a response-handling path.
func (d *Dynamic) MarkStale() {
if d.fetchState == nil {
return
}
d.fetchState.mu.Lock()
d.fetchState.stale = true
d.fetchState.mu.Unlock()
}
// NotifyResponse inspects a response and, if its status code is configured as a
// session-expiry signal, marks the session stale so it is re-authenticated
// before the next request. It returns true if re-authentication was triggered.
//
// generation identifies the session that authenticated the request. When
// non-zero and different from the current session generation, the signal is
// ignored (a late 401 from an older session must not invalidate a fresher one).
// Pass 0 when the generation is unknown (legacy callers).
//
// Re-authentication is capped: when maxStatusReauths consecutive fresh
// sessions got only reauth status codes, the login evidently does not change
// the response, so the session is kept until a request gets another status.
func (d *Dynamic) NotifyResponse(statusCode int, generation uint64) bool {
if d.fetchState == nil || len(d.ReauthStatusCodes) == 0 {
return false
}
if !d.shouldReauthOnStatus(statusCode) {
d.recordAnswer(generation)
return false
}
d.fetchState.mu.Lock()
defer d.fetchState.mu.Unlock()
if !d.fetchState.fetched {
return false
}
if generation != 0 && generation != d.fetchState.generation {
return false
}
if d.fetchState.stale {
return true
}
if d.fetchState.answered.Load() {
d.fetchState.statusReauths = 0
d.fetchState.reauthCapLogged = false
}
if d.fetchState.statusReauths >= maxStatusReauths {
if !d.fetchState.reauthCapLogged {
d.fetchState.reauthCapLogged = true
gologger.Warning().Msgf("dynamic secret: %d re-authentications in a row did not change the %d response; not re-authenticating again until a request gets another status", maxStatusReauths, statusCode)
}
return false
}
d.fetchState.statusReauths++
d.fetchState.stale = true
return true
}
// recordAnswer notes that the session for generation got a response that is
// not a reauth status code, which re-arms status-triggered re-authentication.
func (d *Dynamic) recordAnswer(generation uint64) {
if d.fetchState.answered.Load() {
return
}
// stored under the read lock so a concurrent login cannot reset it in between
d.fetchState.mu.RLock()
defer d.fetchState.mu.RUnlock()
if d.fetchState.fetched && (generation == 0 || generation == d.fetchState.generation) {
d.fetchState.answered.Store(true)
}
}
// WebStorage returns the browser web storage (localStorage/sessionStorage)
// captured by a headless auto-login, fetching the session first if needed. It
// returns nil maps when no storage was captured (e.g. HTTP auto-login or a
// fetch error).
func (d *Dynamic) WebStorage() (map[string]string, map[string]string) {
_ = d.Fetch(false)
if d.fetchState == nil {
return nil, nil
}
d.fetchState.mu.RLock()
defer d.fetchState.mu.RUnlock()
if d.fetchState.err != nil {
return nil, nil
}
return d.fetchState.webStorageLocal, d.fetchState.webStorageSession
}
// IsExpired reports whether the session is currently considered expired/stale.
func (d *Dynamic) IsExpired() bool {
if d.fetchState == nil {
return false
}
d.fetchState.mu.RLock()
defer d.fetchState.mu.RUnlock()
return d.isExpiredLocked()
}
// runFetchLocked runs the login callback. The caller must hold the write lock.
func (d *Dynamic) runFetchLocked() {
d.fetchState.owner.Store(goroutineID())
defer d.fetchState.owner.Store(0)
if d.fetchCallback == nil {
d.fetchState.err = errkit.New("dynamic secret fetch callback not set: call SetLazyFetchCallback() before Fetch()")
return
}
d.fetchState.attempted = true
d.fetchState.fetchedAt = time.Now()
d.fetchState.err = d.fetchCallback(d)
if d.fetchState.err != nil {
// A failed (re-)login drops the session: requests go out without it
// and the login is retried only as needsFetchLocked allows.
d.fetchState.fetched = false
d.fetchState.stale = false
return
}
d.fetchState.fetched = true
d.fetchState.stale = false
d.fetchState.fetchedAt = time.Now()
d.fetchState.generation++
d.fetchState.answered.Store(false)
}
// isExpiredLocked reports whether the session needs re-authentication. The
// caller must hold at least the read lock.
func (d *Dynamic) isExpiredLocked() bool {
if !d.fetchState.fetched {
return false
}
if d.fetchState.stale {
return true
}
if d.refreshInterval > 0 && time.Since(d.fetchState.fetchedAt) > d.refreshInterval {
return true
}
return false
}
// shouldReauthOnStatus reports whether the given status code is configured as a
// session-expiry signal.
func (d *Dynamic) shouldReauthOnStatus(statusCode int) bool {
for _, code := range d.ReauthStatusCodes {
if code == statusCode {
return true
}
}
return false
return d.fetchState.err
}
// Error returns the error if any
@@ -682,55 +256,9 @@ func (d *Dynamic) Error() error {
if d.fetchState == nil {
return nil
}
d.fetchState.mu.RLock()
defer d.fetchState.mu.RUnlock()
return d.fetchState.err
}
// Close drops the in-memory session captured for this dynamic secret: the
// applied cookies/token/headers, any captured web storage and derived secrets,
// and the extracted values. It does not contact the server (sessions are left
// to expire on their own); it simply ensures nuclei does not retain live
// session material after the scan, which matters when nuclei is embedded as a
// library and the process outlives the scan. The session is also marked
// unfetched so a later reuse re-authenticates from scratch.
func (d *Dynamic) Close() {
if d.fetchState != nil {
d.fetchState.mu.Lock()
d.fetchState.webStorageLocal = nil
d.fetchState.webStorageSession = nil
for _, s := range d.fetchState.autoLoginSecrets {
wipeSecretSession(s)
}
d.fetchState.autoLoginSecrets = nil
d.fetchState.fetched = false
d.fetchState.attempted = false
d.fetchState.stale = false
d.fetchState.err = nil
d.fetchState.mu.Unlock()
}
wipeSecretSession(d.Secret)
for _, s := range d.Secrets {
wipeSecretSession(s)
}
d.Extracted = nil
}
// wipeSecretSession zeroes the session-bearing fields of a secret (cookies,
// token, headers, query params and web storage), leaving non-sensitive scoping
// fields (type/domains) intact. It is a no-op on nil.
func wipeSecretSession(s *Secret) {
if s == nil {
return
}
s.Headers = nil
s.Cookies = nil
s.Params = nil
s.Token = ""
s.LocalStorage = nil
s.SessionStorage = nil
}
func (d *Dynamic) fetchingOnThisGoroutine() bool {
if d == nil || d.fetchState == nil {
return false
+3 -65
View File
@@ -98,8 +98,8 @@ func TestDynamicUnmarshalJSON(t *testing.T) {
require.Equal(t, "HeadersAuth", d.Type)
require.Equal(t, []string{"api.test.com"}, d.Domains)
require.Len(t, d.Headers, 1)
require.Equal(t, "X-API-Key", d.Headers[0].Key)
require.Equal(t, "secret-key", d.Headers[0].Value)
require.Equal(t, "X-API-Key", d.Secret.Headers[0].Key)
require.Equal(t, "secret-key", d.Secret.Headers[0].Value)
// Dynamic fields
require.Equal(t, "test-template.yaml", d.TemplatePath)
@@ -142,8 +142,6 @@ func TestDynamicFetchConcurrent(t *testing.T) {
}
require.NoError(t, d.Validate())
d.SetLazyFetchCallback(func(_ *Dynamic) error {
// a failed login is not retried by the waiting requests: a second
// call would close fetchStarted again and panic
close(fetchStarted)
<-fetchUnblock
return wantErr
@@ -603,7 +601,7 @@ func TestSetLazyFetchCallbackAppliesValues(t *testing.T) {
err := d.Fetch(false)
require.NoError(t, err)
require.Equal(t, "Bearer jwt-secret-123", d.Headers[0].Value)
require.Equal(t, "Bearer jwt-secret-123", d.Secret.Headers[0].Value)
})
t.Run("applies to secrets array", func(t *testing.T) {
@@ -828,63 +826,3 @@ func TestGetDomainAndDomainRegex(t *testing.T) {
require.Nil(t, regexes)
})
}
// A failed login must not be retried on every request: each retry is one more
// login attempt against the target (account lockout, WAF bans).
func TestDynamicFailedLoginIsNotRetriedPerRequest(t *testing.T) {
newDynamic := func(refresh string, fail *atomic.Bool, calls *atomic.Int32) *Dynamic {
d := &Dynamic{
TemplatePath: "test-template.yaml",
Variables: []KV{{Key: "username", Value: "test"}},
RefreshInterval: refresh,
}
require.NoError(t, d.Validate())
d.SetLazyFetchCallback(func(d *Dynamic) error {
calls.Add(1)
if fail.Load() {
return errors.New("bad credentials")
}
d.Extracted = map[string]interface{}{"token": "x"}
return nil
})
return d
}
t.Run("first login fails", func(t *testing.T) {
var fail atomic.Bool
var calls atomic.Int32
fail.Store(true)
d := newDynamic("", &fail, &calls)
for i := 0; i < 5; i++ {
require.Error(t, d.Fetch(false))
}
require.Equal(t, int32(1), calls.Load())
})
t.Run("re-login after stale fails", func(t *testing.T) {
var fail atomic.Bool
var calls atomic.Int32
d := newDynamic("", &fail, &calls)
require.NoError(t, d.Fetch(false))
fail.Store(true)
d.MarkStale()
for i := 0; i < 5; i++ {
require.Error(t, d.Fetch(false))
}
require.Equal(t, int32(2), calls.Load())
})
t.Run("retried after refresh interval", func(t *testing.T) {
var fail atomic.Bool
var calls atomic.Int32
fail.Store(true)
d := newDynamic("50ms", &fail, &calls)
require.Error(t, d.Fetch(false))
require.Error(t, d.Fetch(false))
require.Equal(t, int32(1), calls.Load())
fail.Store(false)
time.Sleep(80 * time.Millisecond)
require.NoError(t, d.Fetch(false))
require.Equal(t, int32(2), calls.Load())
})
}
+10 -38
View File
@@ -25,11 +25,6 @@ const (
HeadersAuth AuthType = "Header"
CookiesAuth AuthType = "Cookie"
QueryAuth AuthType = "Query"
// WebStorageAuth carries browser localStorage/sessionStorage to be seeded
// into headless scan pages. It has no HTTP representation (its Apply is a
// no-op) and is used to replay token-in-web-storage sessions (e.g. captured
// via auth-capture) for SPA scans.
WebStorageAuth AuthType = "WebStorage"
)
// SupportedAuthTypes returns the supported auth types
@@ -40,7 +35,6 @@ func SupportedAuthTypes() []string {
string(HeadersAuth),
string(CookiesAuth),
string(QueryAuth),
string(WebStorageAuth),
}
}
@@ -61,20 +55,16 @@ type AuthFileInfo struct {
// Secret is a struct for secret or credential
type Secret struct {
Type string `json:"type" yaml:"type"`
Domains []string `json:"domains" yaml:"domains"`
DomainsRegex []string `json:"domains-regex" yaml:"domains-regex"`
Headers []KV `json:"headers" yaml:"headers"` // Headers preserve exact casing (useful for case-sensitive APIs)
Cookies []Cookie `json:"cookies" yaml:"cookies"`
Params []KV `json:"params" yaml:"params"`
Username string `json:"username" yaml:"username"` // can be either email or username
Password string `json:"password" yaml:"password"`
Token string `json:"token" yaml:"token"` // Bearer Auth token
// LocalStorage / SessionStorage carry browser web storage for WebStorageAuth
// secrets, seeded into headless scan pages before page scripts run.
LocalStorage map[string]string `json:"local-storage" yaml:"local-storage"`
SessionStorage map[string]string `json:"session-storage" yaml:"session-storage"`
skipCookieParse bool `json:"-" yaml:"-"` // temporary flag to skip cookie parsing (used in dynamic secrets)
Type string `json:"type" yaml:"type"`
Domains []string `json:"domains" yaml:"domains"`
DomainsRegex []string `json:"domains-regex" yaml:"domains-regex"`
Headers []KV `json:"headers" yaml:"headers"` // Headers preserve exact casing (useful for case-sensitive APIs)
Cookies []Cookie `json:"cookies" yaml:"cookies"`
Params []KV `json:"params" yaml:"params"`
Username string `json:"username" yaml:"username"` // can be either email or username
Password string `json:"password" yaml:"password"`
Token string `json:"token" yaml:"token"` // Bearer Auth token
skipCookieParse bool `json:"-" yaml:"-"` // temporary flag to skip cookie parsing (used in dynamic secrets)
}
// GetStrategy returns the auth strategy for the secret
@@ -90,8 +80,6 @@ func (s *Secret) GetStrategy() AuthStrategy {
return NewCookiesAuthStrategy(s)
case strings.EqualFold(s.Type, string(QueryAuth)):
return NewQueryAuthStrategy(s)
case strings.EqualFold(s.Type, string(WebStorageAuth)):
return NewWebStorageAuthStrategy(s.LocalStorage, s.SessionStorage)
}
return nil
}
@@ -147,10 +135,6 @@ func (s *Secret) Validate() error {
return fmt.Errorf("invalid cookie in cookiesAuth: %s", err)
}
}
case strings.EqualFold(s.Type, string(WebStorageAuth)):
if len(s.LocalStorage) == 0 && len(s.SessionStorage) == 0 {
return fmt.Errorf("local-storage or session-storage cannot be empty in web storage auth")
}
case strings.EqualFold(s.Type, string(QueryAuth)):
if len(s.Params) == 0 {
return fmt.Errorf("query cannot be empty in query auth")
@@ -185,13 +169,6 @@ type Cookie struct {
Key string `json:"key" yaml:"key"`
Value string `json:"value" yaml:"value"`
Raw string `json:"raw" yaml:"raw"`
// Domain / Path / Secure preserve the cookie's protocol scope when captured
// (e.g. from a headless login). Empty Domain/Path means "unscoped": the
// cookie is attached to every request the secret's domains match, matching
// historical nuclei behaviour for manually authored secrets.
Domain string `json:"domain,omitempty" yaml:"domain,omitempty"`
Path string `json:"path,omitempty" yaml:"path,omitempty"`
Secure bool `json:"secure,omitempty" yaml:"secure,omitempty"`
}
func (c *Cookie) Validate() error {
@@ -265,11 +242,6 @@ func GetTemplatePathsFromSecretFile(file string) ([]string, error) {
}
var paths []string
for _, dynamic := range auth.Dynamic {
// Auto-login dynamics have no template path; skip them so we don't try to
// load an empty template.
if dynamic.TemplatePath == "" {
continue
}
paths = append(paths, dynamic.TemplatePath)
}
return paths, nil
-138
View File
@@ -1,138 +0,0 @@
package authx
import (
"encoding/json"
"fmt"
"io"
"net/http"
"net/http/httptest"
"sync"
"sync/atomic"
"testing"
"github.com/stretchr/testify/require"
)
// fakeAuthServer simulates a login + protected-resource backend with
// server-side session expiry, to exercise the dynamic session lifecycle end to
// end over real HTTP:
// - GET /login issues a fresh incrementing bearer token and marks it as the
// only currently-valid session.
// - GET /api returns 200 when the presented bearer matches the valid session,
// and 401 once the session has been expired server-side.
type fakeAuthServer struct {
mu sync.Mutex
counter int
valid string // currently valid token; empty means "no valid session"
logins atomic.Int32
}
func (s *fakeAuthServer) handler() http.Handler {
mux := http.NewServeMux()
mux.HandleFunc("/login", func(w http.ResponseWriter, r *http.Request) {
s.mu.Lock()
s.counter++
s.valid = fmt.Sprintf("sess-%d", s.counter)
tok := s.valid
s.mu.Unlock()
s.logins.Add(1)
_ = json.NewEncoder(w).Encode(map[string]string{"token": tok})
})
mux.HandleFunc("/api", func(w http.ResponseWriter, r *http.Request) {
s.mu.Lock()
valid := s.valid
s.mu.Unlock()
if valid != "" && r.Header.Get("Authorization") == "Bearer "+valid {
w.WriteHeader(http.StatusOK)
_, _ = io.WriteString(w, "secret-data")
return
}
w.WriteHeader(http.StatusUnauthorized)
_, _ = io.WriteString(w, "session expired")
})
return mux
}
// expire invalidates the current session server-side without issuing a new one,
// so the next /api call with the old token returns 401.
func (s *fakeAuthServer) expire() {
s.mu.Lock()
s.valid = ""
s.mu.Unlock()
}
// TestSessionLifecycle_E2E drives the whole dynamic-session state machine against
// a real HTTP server: it performs a real login to mint a token, uses it on a
// protected endpoint, lets the server expire the session, and verifies that a
// 401 (surfaced through NotifyResponse) triggers a real re-authentication whose
// fresh token restores access — all through the production Dynamic/strategy code.
func TestSessionLifecycle_E2E(t *testing.T) {
backend := &fakeAuthServer{}
srv := httptest.NewServer(backend.handler())
defer srv.Close()
httpClient := srv.Client()
d := &Dynamic{
Secret: &Secret{
Type: "Header",
Domains: []string{"127.0.0.1", "localhost"},
Headers: []KV{{Key: "Authorization", Value: "Bearer {{token}}"}},
},
TemplatePath: "auth.yaml",
Variables: []KV{{Key: "user", Value: "admin"}},
ReauthStatusCodes: []int{401},
}
require.NoError(t, d.Validate())
// The login callback performs a REAL HTTP login and extracts the token, the
// same shape the template-driven fetch produces.
d.SetLazyFetchCallback(func(dyn *Dynamic) error {
resp, err := httpClient.Get(srv.URL + "/login")
if err != nil {
return err
}
defer func() { _ = resp.Body.Close() }()
var payload struct {
Token string `json:"token"`
}
if err := json.NewDecoder(resp.Body).Decode(&payload); err != nil {
return err
}
dyn.Extracted = map[string]interface{}{"token": payload.Token}
return nil
})
strategy := &DynamicAuthStrategy{Dynamic: *d}
inspector, ok := AuthStrategy(strategy).(ResponseInspector)
require.True(t, ok, "DynamicAuthStrategy must implement ResponseInspector")
// doAPI applies the (possibly re-authenticated) session and calls /api.
doAPI := func() int {
req, err := http.NewRequest(http.MethodGet, srv.URL+"/api", nil)
require.NoError(t, err)
strategy.Apply(req)
resp, err := httpClient.Do(req)
require.NoError(t, err)
_ = resp.Body.Close()
// surface the response to the auth strategy, like the http protocol does
inspector.OnResponse(resp.StatusCode, 0)
return resp.StatusCode
}
// 1) First call logs in and succeeds.
require.Equal(t, http.StatusOK, doAPI(), "first authenticated request should succeed")
require.Equal(t, int32(1), backend.logins.Load(), "exactly one login so far")
require.False(t, d.IsExpired())
// 2) Server expires the session out from under us → next call sees 401 and,
// via NotifyResponse, marks the session stale.
backend.expire()
require.Equal(t, http.StatusUnauthorized, doAPI(), "request with the now-expired token should 401")
require.True(t, d.IsExpired(), "401 must mark the session stale for re-authentication")
// 3) Next call re-authenticates (a second real login) and succeeds again.
require.Equal(t, http.StatusOK, doAPI(), "request after re-auth should succeed with the fresh token")
require.Equal(t, int32(2), backend.logins.Load(), "a second login should have occurred")
require.False(t, d.IsExpired(), "session should be fresh again after re-auth")
}
-297
View File
@@ -1,297 +0,0 @@
package authx
import (
"net/http"
"sync"
"sync/atomic"
"testing"
"time"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
// newDynamicWithToken builds a validated dynamic secret whose login callback
// returns an incrementing token each time it runs, so re-authentication is observable.
func newDynamicWithToken(t *testing.T, calls *atomic.Int32) *Dynamic {
t.Helper()
d := &Dynamic{
Secret: &Secret{
Type: "Header",
Domains: []string{"example.com"},
Headers: []KV{{Key: "Authorization", Value: "Bearer {{token}}"}},
},
TemplatePath: "auth.yaml",
Variables: []KV{{Key: "user", Value: "admin"}},
}
require.NoError(t, d.Validate())
d.SetLazyFetchCallback(func(dyn *Dynamic) error {
n := calls.Add(1)
dyn.Extracted = map[string]interface{}{"token": tokenForCall(n)}
return nil
})
return d
}
func tokenForCall(n int32) string {
switch n {
case 1:
return "token-1"
case 2:
return "token-2"
case 3:
return "token-3"
default:
return "token-n"
}
}
func applyHeader(t *testing.T, d *Dynamic) string {
t.Helper()
strategy := &DynamicAuthStrategy{Dynamic: *d}
req, _ := http.NewRequest("GET", "https://example.com", nil)
strategy.Apply(req)
return req.Header.Get("Authorization")
}
func TestSessionRefreshReRendersTemplate(t *testing.T) {
var calls atomic.Int32
d := newDynamicWithToken(t, &calls)
require.Equal(t, "Bearer token-1", applyHeader(t, d), "first fetch should render initial token")
require.Equal(t, "Bearer token-1", applyHeader(t, d), "second apply should reuse cached session")
require.Equal(t, int32(1), calls.Load(), "login should run once before refresh")
require.NoError(t, d.Refresh(false))
require.Equal(t, int32(2), calls.Load(), "refresh should re-run the login flow")
require.Equal(t, "Bearer token-2", applyHeader(t, d), "refresh must re-render template with the new token")
}
func TestSessionMarkStaleTriggersReauth(t *testing.T) {
var calls atomic.Int32
d := newDynamicWithToken(t, &calls)
require.Equal(t, "Bearer token-1", applyHeader(t, d))
require.Equal(t, int32(1), calls.Load())
d.MarkStale()
require.True(t, d.IsExpired(), "session should report expired after MarkStale")
require.Equal(t, "Bearer token-2", applyHeader(t, d), "stale session should re-authenticate on next apply")
require.Equal(t, int32(2), calls.Load())
require.False(t, d.IsExpired(), "session should be fresh again after re-auth")
}
func TestSessionRefreshIntervalExpiry(t *testing.T) {
var calls atomic.Int32
d := &Dynamic{
Secret: &Secret{
Type: "Header",
Domains: []string{"example.com"},
Headers: []KV{{Key: "Authorization", Value: "Bearer {{token}}"}},
},
TemplatePath: "auth.yaml",
Variables: []KV{{Key: "user", Value: "admin"}},
RefreshInterval: "40ms",
}
require.NoError(t, d.Validate())
d.SetLazyFetchCallback(func(dyn *Dynamic) error {
n := calls.Add(1)
dyn.Extracted = map[string]interface{}{"token": tokenForCall(n)}
return nil
})
require.Equal(t, "Bearer token-1", applyHeader(t, d))
require.Equal(t, "Bearer token-1", applyHeader(t, d), "within interval the session is reused")
require.Equal(t, int32(1), calls.Load())
time.Sleep(60 * time.Millisecond)
require.True(t, d.IsExpired(), "session should expire after the refresh interval")
require.Equal(t, "Bearer token-2", applyHeader(t, d), "expired session should re-authenticate")
require.Equal(t, int32(2), calls.Load())
}
func TestSessionNotifyResponseReauth(t *testing.T) {
t.Run("configured status marks stale", func(t *testing.T) {
var calls atomic.Int32
d := newDynamicWithToken(t, &calls)
d.ReauthStatusCodes = []int{401, 403}
require.Equal(t, "Bearer token-1", applyHeader(t, d))
require.False(t, d.NotifyResponse(200, 0), "2xx must not trigger re-auth")
require.False(t, d.IsExpired())
require.True(t, d.NotifyResponse(401, 0), "401 must trigger re-auth for established session")
require.True(t, d.IsExpired())
require.Equal(t, "Bearer token-2", applyHeader(t, d))
require.Equal(t, int32(2), calls.Load())
})
t.Run("no reauth before first fetch", func(t *testing.T) {
var calls atomic.Int32
d := newDynamicWithToken(t, &calls)
d.ReauthStatusCodes = []int{401}
require.False(t, d.NotifyResponse(401, 0), "should not mark stale before a session exists")
require.Equal(t, int32(0), calls.Load())
})
t.Run("unconfigured status code is ignored", func(t *testing.T) {
var calls atomic.Int32
d := newDynamicWithToken(t, &calls)
// ReauthStatusCodes left empty => response-triggered reauth disabled
require.Equal(t, "Bearer token-1", applyHeader(t, d))
require.False(t, d.NotifyResponse(401, 0))
require.False(t, d.IsExpired())
})
t.Run("stale generation is ignored", func(t *testing.T) {
var calls atomic.Int32
d := newDynamicWithToken(t, &calls)
d.ReauthStatusCodes = []int{401}
req, _ := http.NewRequest("GET", "https://example.com", nil)
(&DynamicAuthStrategy{Dynamic: *d}).Apply(req)
gen1 := SessionGenerationFromRequest(req)
require.NotZero(t, gen1)
require.True(t, d.NotifyResponse(401, gen1))
require.True(t, d.IsExpired())
req2, _ := http.NewRequest("GET", "https://example.com", nil)
(&DynamicAuthStrategy{Dynamic: *d}).Apply(req2)
gen2 := SessionGenerationFromRequest(req2)
require.NotEqual(t, gen1, gen2)
// A late 401 from the previous generation must not invalidate the new session.
require.False(t, d.NotifyResponse(401, gen1))
require.False(t, d.IsExpired())
require.Equal(t, "Bearer token-2", req2.Header.Get("Authorization"))
})
}
func TestSessionResponseInspectorInterface(t *testing.T) {
var calls atomic.Int32
d := newDynamicWithToken(t, &calls)
d.ReauthStatusCodes = []int{401}
var strategy AuthStrategy = &DynamicAuthStrategy{Dynamic: *d}
inspector, ok := strategy.(ResponseInspector)
require.True(t, ok, "DynamicAuthStrategy must implement ResponseInspector")
// establish session via the same shared state
req, _ := http.NewRequest("GET", "https://example.com", nil)
strategy.Apply(req)
require.Equal(t, "Bearer token-1", req.Header.Get("Authorization"))
require.True(t, inspector.OnResponse(401, 0))
req2, _ := http.NewRequest("GET", "https://example.com", nil)
strategy.Apply(req2)
require.Equal(t, "Bearer token-2", req2.Header.Get("Authorization"))
}
func TestSessionValidateRejectsBadRefreshInterval(t *testing.T) {
d := &Dynamic{
TemplatePath: "auth.yaml",
Variables: []KV{{Key: "user", Value: "admin"}},
RefreshInterval: "not-a-duration",
}
err := d.Validate()
require.Error(t, err)
require.Contains(t, err.Error(), "invalid refresh-interval")
}
// TestSessionConcurrentApplyDuringReauth exercises the locking contract: request
// threads applying the strategy must never race a concurrent re-authentication.
// Intended to be run with -race.
func TestSessionConcurrentApplyDuringReauth(t *testing.T) {
var calls atomic.Int32
d := &Dynamic{
Secret: &Secret{
Type: "Header",
Domains: []string{"example.com"},
Headers: []KV{{Key: "Authorization", Value: "Bearer {{token}}"}},
},
TemplatePath: "auth.yaml",
Variables: []KV{{Key: "user", Value: "admin"}},
}
require.NoError(t, d.Validate())
d.SetLazyFetchCallback(func(dyn *Dynamic) error {
calls.Add(1)
// simulate some work during login so writers and readers overlap
time.Sleep(time.Millisecond)
dyn.Extracted = map[string]interface{}{"token": "tok"}
return nil
})
strategy := &DynamicAuthStrategy{Dynamic: *d}
const workers = 16
var wg sync.WaitGroup
stop := make(chan struct{})
// readers continuously apply the strategy
for i := 0; i < workers; i++ {
wg.Add(1)
go func() {
defer wg.Done()
for {
select {
case <-stop:
return
default:
req, _ := http.NewRequest("GET", "https://example.com", nil)
strategy.Apply(req)
// assert (not require): a failed check in a worker must not
// call FailNow from a non-test goroutine.
assert.Equal(t, "Bearer tok", req.Header.Get("Authorization"))
}
}
}()
}
// writers continuously invalidate the session, forcing re-auth
for i := 0; i < 4; i++ {
wg.Add(1)
go func() {
defer wg.Done()
for {
select {
case <-stop:
return
default:
d.MarkStale()
time.Sleep(time.Millisecond)
}
}
}()
}
time.Sleep(200 * time.Millisecond)
close(stop)
wg.Wait()
require.Greater(t, calls.Load(), int32(1), "re-authentication should have happened multiple times")
}
func TestSessionNotifyResponseCapsUnproductiveReauth(t *testing.T) {
var calls atomic.Int32
d := newDynamicWithToken(t, &calls)
d.ReauthStatusCodes = []int{401}
// the login succeeds but every request still gets 401
for range 10 {
applyHeader(t, d)
d.NotifyResponse(401, 0)
}
require.Equal(t, int32(1+maxStatusReauths), calls.Load(), "re-authentication must stop when it does not change the response")
require.False(t, d.IsExpired())
// a request that gets another status re-arms status-triggered re-auth
require.False(t, d.NotifyResponse(200, 0))
require.True(t, d.NotifyResponse(401, 0))
applyHeader(t, d)
require.Equal(t, int32(2+maxStatusReauths), calls.Load())
}
+10 -65
View File
@@ -1,7 +1,6 @@
package authx
import (
"context"
"net/http"
"github.com/projectdiscovery/retryablehttp-go"
@@ -16,46 +15,6 @@ type AuthStrategy interface {
ApplyOnRR(*retryablehttp.Request)
}
// ResponseInspector is an optional interface that an AuthStrategy may implement
// to observe responses to authenticated requests. It is used to detect session
// expiry (e.g. a 401 after a previously valid session) and trigger
// re-authentication for subsequent requests.
type ResponseInspector interface {
// OnResponse is called with the status code of a response to a request the
// strategy authenticated. sessionGeneration identifies the session that was
// applied to that request (0 when unknown). It returns true if
// re-authentication was triggered.
OnResponse(statusCode int, sessionGeneration uint64) bool
}
var (
_ AuthStrategy = &DynamicAuthStrategy{}
_ ResponseInspector = &DynamicAuthStrategy{}
_ BrowserStorageProvider = &DynamicAuthStrategy{}
)
type authSessionGenerationKey struct{}
// WithSessionGeneration stores the dynamic-session generation on the request
// context so a later response can be correlated with the session that
// authenticated it.
func WithSessionGeneration(req *http.Request, generation uint64) *http.Request {
if req == nil || generation == 0 {
return req
}
return req.WithContext(context.WithValue(req.Context(), authSessionGenerationKey{}, generation))
}
// SessionGenerationFromRequest returns the session generation stamped onto req,
// or 0 when absent.
func SessionGenerationFromRequest(req *http.Request) uint64 {
if req == nil {
return 0
}
gen, _ := req.Context().Value(authSessionGenerationKey{}).(uint64)
return gen
}
// DynamicAuthStrategy is an auth strategy for dynamic secrets
// it implements the AuthStrategy interface
type DynamicAuthStrategy struct {
@@ -65,36 +24,22 @@ type DynamicAuthStrategy struct {
// Apply applies the strategy to the request
func (d *DynamicAuthStrategy) Apply(req *http.Request) {
gen := d.Dynamic.ApplyStrategies(func(s AuthStrategy) {
strategies := d.Dynamic.GetStrategies()
if strategies == nil {
return
}
for _, s := range strategies {
if s == nil {
continue
}
s.Apply(req)
})
if req != nil && gen != 0 {
*req = *WithSessionGeneration(req, gen)
}
}
// ApplyOnRR applies the strategy to the retryable request
func (d *DynamicAuthStrategy) ApplyOnRR(req *retryablehttp.Request) {
if req == nil {
return
}
gen := d.Dynamic.ApplyStrategies(func(s AuthStrategy) {
strategy := d.Dynamic.GetStrategies()
for _, s := range strategy {
s.ApplyOnRR(req)
})
if gen != 0 && req.Request != nil {
req.Request = WithSessionGeneration(req.Request, gen)
}
}
// OnResponse inspects a response status code and marks the dynamic session for
// re-authentication when the code signals an expired session for the given
// generation.
func (d *DynamicAuthStrategy) OnResponse(statusCode int, sessionGeneration uint64) bool {
return d.Dynamic.NotifyResponse(statusCode, sessionGeneration)
}
// WebStorage exposes the browser web storage captured by a headless auto-login
// so the headless engine can replay it into scan pages.
func (d *DynamicAuthStrategy) WebStorage() (map[string]string, map[string]string) {
return d.Dynamic.WebStorage()
}
@@ -1,47 +0,0 @@
package authx
import (
"net/http"
"github.com/projectdiscovery/retryablehttp-go"
)
var (
_ AuthStrategy = &WebStorageAuthStrategy{}
_ BrowserStorageProvider = &WebStorageAuthStrategy{}
)
// BrowserStorageProvider is an optional interface implemented by auth strategies
// that carry browser web storage (localStorage/sessionStorage) to be replayed
// into headless scan pages. HTTP-only strategies do not implement it, so the
// headless engine type-asserts for it before seeding storage.
type BrowserStorageProvider interface {
// WebStorage returns the localStorage and sessionStorage items to seed.
WebStorage() (local map[string]string, session map[string]string)
}
// WebStorageAuthStrategy carries captured web storage. Its Apply methods are
// intentional no-ops: web storage cannot be expressed on an HTTP request, so it
// only takes effect in the headless engine, which reads it via the
// BrowserStorageProvider interface and seeds it into the page before page
// scripts run.
type WebStorageAuthStrategy struct {
Local map[string]string
Session map[string]string
}
// NewWebStorageAuthStrategy creates a new web storage auth strategy.
func NewWebStorageAuthStrategy(local, session map[string]string) *WebStorageAuthStrategy {
return &WebStorageAuthStrategy{Local: local, Session: session}
}
// Apply is a no-op (web storage is not an HTTP request artifact).
func (s *WebStorageAuthStrategy) Apply(_ *http.Request) {}
// ApplyOnRR is a no-op (web storage is not an HTTP request artifact).
func (s *WebStorageAuthStrategy) ApplyOnRR(_ *retryablehttp.Request) {}
// WebStorage returns the carried storage items.
func (s *WebStorageAuthStrategy) WebStorage() (map[string]string, map[string]string) {
return s.Local, s.Session
}
@@ -1,40 +0,0 @@
package authx
import (
"testing"
"github.com/stretchr/testify/require"
)
func TestWebStorageSecret_GetStrategy(t *testing.T) {
s := &Secret{
Type: string(WebStorageAuth),
Domains: []string{"app.example.com"},
LocalStorage: map[string]string{"jwt": "eyJ.payload.sig"},
SessionStorage: map[string]string{"csrf": "tok"},
}
require.NoError(t, s.Validate())
strategy := s.GetStrategy()
require.NotNil(t, strategy)
bsp, ok := strategy.(BrowserStorageProvider)
require.True(t, ok, "web storage secret must yield a BrowserStorageProvider")
local, session := bsp.WebStorage()
require.Equal(t, "eyJ.payload.sig", local["jwt"])
require.Equal(t, "tok", session["csrf"])
}
func TestWebStorageSecret_ValidateEmpty(t *testing.T) {
s := &Secret{Type: string(WebStorageAuth), Domains: []string{"app.example.com"}}
require.Error(t, s.Validate(), "web storage secret with no storage must be invalid")
}
func TestWebStorageSecret_SessionOnlyValid(t *testing.T) {
s := &Secret{
Type: string(WebStorageAuth),
Domains: []string{"app.example.com"},
SessionStorage: map[string]string{"csrf": "tok"},
}
require.NoError(t, s.Validate(), "session-storage alone should satisfy validation")
}
-124
View File
@@ -1,124 +0,0 @@
package autologin
import (
"context"
"net/url"
"regexp"
"strings"
"github.com/go-rod/rod/lib/proto"
"github.com/projectdiscovery/utils/errkit"
)
// CaptureOnce launches a visible browser at cfg.LoginURL and lets the user log
// in manually, then — once ready(ctx) returns — captures the resulting session
// (cookies, web storage and an optional token). It performs no automated form
// filling.
//
// This is the "log in yourself, I'll grab the session" mode for one-off
// authenticated scans of flows that are impractical to record/replay
// (hardware-key MFA, CAPTCHA, interactive SSO). Because the captured session is
// a point-in-time snapshot there is no automated re-authentication; ready
// blocks until the caller signals login completion (e.g. the user pressing
// Enter in the terminal) and should honor ctx cancellation.
func CaptureOnce(ctx context.Context, cfg Config, ready func(context.Context) error) (*Session, error) {
if ready == nil {
return nil, errkit.New("auto-login(capture): a ready signal is required")
}
if strings.TrimSpace(cfg.LoginURL) == "" {
return nil, errkit.New("auto-login(capture): login-url is required")
}
if _, err := url.Parse(cfg.LoginURL); err != nil {
return nil, errkit.Wrap(err, "auto-login(capture): invalid login-url")
}
var tokenRe *regexp.Regexp
if cfg.TokenRegex != "" {
var err error
if tokenRe, err = regexp.Compile(cfg.TokenRegex); err != nil {
return nil, errkit.Wrap(err, "auto-login(capture): invalid token-regex")
}
}
// Force a visible browser so the user can interact with the login page.
cfg.ShowBrowser = true
browser, cleanup, err := launchBrowser(cfg)
if err != nil {
return nil, errkit.Wrap(err, "auto-login(capture): failed to launch browser")
}
defer cleanup()
page, err := browser.Page(proto.TargetCreateTarget{})
if err != nil {
return nil, errkit.Wrap(err, "auto-login(capture): failed to open page")
}
defer func() { _ = page.Close() }()
page = page.Context(ctx)
if cfg.UserAgent != "" {
if uaErr := page.SetUserAgent(&proto.NetworkSetUserAgentOverride{UserAgent: cfg.UserAgent}); uaErr != nil {
return nil, errkit.Wrap(uaErr, "auto-login(capture): failed to set user-agent")
}
}
if len(cfg.CustomHeaders) > 0 {
pairs := make([]string, 0, len(cfg.CustomHeaders)*2)
for k, v := range cfg.CustomHeaders {
pairs = append(pairs, k, v)
}
if _, hErr := page.SetExtraHeaders(pairs); hErr != nil {
return nil, errkit.Wrap(hErr, "auto-login(capture): failed to set custom headers")
}
}
if err := page.Navigate(cfg.LoginURL); err != nil {
return nil, errkit.Wrap(err, "auto-login(capture): failed to navigate to login page")
}
_ = page.WaitLoad()
// Block until the user signals that the manual login is complete.
if err := ready(ctx); err != nil {
return nil, errkit.Wrap(err, "auto-login(capture): aborted before capture")
}
if err := ctx.Err(); err != nil {
return nil, err
}
session := &Session{}
if info, ierr := page.Info(); ierr == nil {
session.FinalURL = info.URL
}
session.Cookies = capturePageCookies(page, cfg.LoginURL, session.FinalURL)
session.CookieHeader = renderCookieHeader(session.Cookies)
session.LocalStorage = readStorage(page, "localStorage")
session.SessionStorage = readStorage(page, "sessionStorage")
if tokenRe != nil {
var haystack strings.Builder
if html, herr := page.HTML(); herr == nil {
haystack.WriteString(html)
}
for _, store := range []map[string]string{session.LocalStorage, session.SessionStorage} {
for k, v := range store {
haystack.WriteString("\n")
haystack.WriteString(k)
haystack.WriteString("=")
haystack.WriteString(v)
}
}
if m := tokenRe.FindStringSubmatch(haystack.String()); len(m) > 1 {
session.Token = m[1]
}
}
// When no explicit token-regex matched, surface a JWT-shaped value from web
// storage so a captured SPA session also authenticates HTTP scan requests.
if session.Token == "" {
session.Token = detectStorageJWT(session.LocalStorage, session.SessionStorage)
}
if len(session.Cookies) == 0 && session.Token == "" && len(session.LocalStorage) == 0 && len(session.SessionStorage) == 0 {
return nil, errkit.New("auto-login(capture): no session captured (no cookies, token or web storage found)")
}
return session, nil
}
@@ -1,97 +0,0 @@
package autologin
import (
"context"
"errors"
"fmt"
"net/http"
"net/http/httptest"
"os"
"runtime"
"testing"
"time"
"github.com/stretchr/testify/require"
)
// requireDisplay skips headful (visible-browser) tests when no display server is
// available, e.g. headless Linux CI. macOS and Windows always provide one, but
// capture-once forces a visible browser which cannot start without an X/Wayland
// display.
func requireDisplay(t *testing.T) {
t.Helper()
if runtime.GOOS == "linux" && os.Getenv("DISPLAY") == "" && os.Getenv("WAYLAND_DISPLAY") == "" {
t.Skip("no display server ($DISPLAY/$WAYLAND_DISPLAY) available; skipping headful capture-once test")
}
}
// capturePage already "logs the user in" on GET (sets a session cookie and seeds
// web storage via JS), so a test can drive CaptureOnce without simulating manual
// interaction: the ready() callback simply returns once the page has loaded.
const capturePage = `<html><head><title>App</title></head><body>
<h1>welcome dave</h1>
<script>
window.localStorage.setItem('auth_token', 'eyJhbGciOiJI.payload.sig');
window.sessionStorage.setItem('csrf', 'abc123');
</script></body></html>`
func TestCaptureOnce_E2E(t *testing.T) {
requireChrome(t)
requireDisplay(t)
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
http.SetCookie(w, &http.Cookie{Name: "session", Value: "sess-dave", Path: "/"})
w.Header().Set("Content-Type", "text/html")
_, _ = fmt.Fprint(w, capturePage)
}))
defer srv.Close()
ctx, cancel := context.WithTimeout(context.Background(), 60*time.Second)
defer cancel()
var readyCalled bool
session, err := CaptureOnce(ctx, Config{
LoginURL: srv.URL + "/login",
TokenRegex: `(eyJ[A-Za-z0-9._-]+)`,
}, func(context.Context) error {
readyCalled = true // stands in for the user pressing Enter after logging in
return nil
})
require.NoError(t, err)
require.True(t, readyCalled, "ready callback must be invoked before capture")
names := map[string]string{}
for _, c := range session.Cookies {
names[c.Name] = c.Value
}
require.Equal(t, "sess-dave", names["session"], "session cookie must be captured")
require.Equal(t, "eyJhbGciOiJI.payload.sig", session.LocalStorage["auth_token"], "localStorage must be captured")
require.Equal(t, "abc123", session.SessionStorage["csrf"], "sessionStorage must be captured")
require.Equal(t, "eyJhbGciOiJI.payload.sig", session.Token, "token must be extracted")
}
func TestCaptureOnce_ReadyErrorAborts(t *testing.T) {
requireChrome(t)
requireDisplay(t)
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
_, _ = fmt.Fprint(w, "<html><body>login</body></html>")
}))
defer srv.Close()
ctx, cancel := context.WithTimeout(context.Background(), 60*time.Second)
defer cancel()
_, err := CaptureOnce(ctx, Config{LoginURL: srv.URL + "/login"}, func(context.Context) error {
return errors.New("user cancelled")
})
require.Error(t, err, "a ready() error must abort the capture")
}
func TestCaptureOnce_Validation(t *testing.T) {
_, err := CaptureOnce(context.Background(), Config{LoginURL: "https://x"}, nil)
require.Error(t, err, "nil ready must error")
_, err = CaptureOnce(context.Background(), Config{}, func(context.Context) error { return nil })
require.Error(t, err, "missing login-url must error")
}
-311
View File
@@ -1,311 +0,0 @@
// Package autologin implements turnkey form-based authentication for nuclei's
// authenticated scanning. Instead of requiring a hand-written auth template that
// performs the login and extracts a token, the caller supplies only a login URL
// and credentials; this package fetches the login page, auto-detects the login
// form (username/password fields plus any hidden CSRF tokens), submits the
// credentials and captures the resulting session.
//
// detect.go contains the pure, network-free form-detection logic so it can be
// unit-tested in isolation from the HTTP engine in login.go.
package autologin
import (
"net/url"
"strings"
"github.com/projectdiscovery/utils/errkit"
"golang.org/x/net/html"
)
var (
// ErrNoLoginForm is returned when the page contains no form with a password
// field (the minimal signal that a form is a credential-login form).
ErrNoLoginForm = errkit.New("no login form (password field) found on page")
)
// LoginForm describes a detected HTML login form and everything needed to
// submit it.
type LoginForm struct {
// Action is the absolute URL the form submits to, resolved against the page URL.
Action string
// Method is the upper-cased HTTP method ("POST" or "GET").
Method string
// EncType is the form encoding ("application/x-www-form-urlencoded" or
// "multipart/form-data"); empty means the urlencoded default.
EncType string
// UsernameField is the name of the input that should receive the username.
// It may be empty when no plausible username field is present (e.g. a
// password-only or token form), in which case the caller may still submit.
UsernameField string
// PasswordField is the name of the password input. Always non-empty on a
// successfully detected form.
PasswordField string
// Fields holds extra form values that must be submitted verbatim — most
// importantly hidden inputs carrying CSRF tokens, plus any pre-filled
// defaults and a named submit button. Username/password are NOT included
// here; the engine fills those into UsernameField/PasswordField.
Fields map[string]string
}
// usernameHintTokens are substrings (matched case-insensitively against an
// input's name/id/autocomplete) that strongly suggest a username/identifier
// field. Ordered roughly by specificity.
var usernameHintTokens = []string{"username", "user", "email", "e-mail", "login", "account", "userid", "uid", "identifier", "phone"}
// DetectLoginForm parses the given HTML page body and returns the best
// credential-login form found on it. pageURL is the absolute URL the body was
// fetched from and is used to resolve relative form actions. It returns
// ErrNoLoginForm if no form containing a password field exists.
func DetectLoginForm(body string, pageURL *url.URL) (*LoginForm, error) {
doc, err := html.Parse(strings.NewReader(body))
if err != nil {
return nil, errkit.Wrap(err, "failed to parse html")
}
var forms []*html.Node
collectForms(doc, &forms)
var best *LoginForm
bestScore := -1
for _, formNode := range forms {
candidate, score := buildLoginForm(formNode, pageURL)
if candidate == nil {
continue
}
// Prefer the highest-scoring form, keeping the first on ties so document
// order acts as the tie-breaker.
if score > bestScore {
best = candidate
bestScore = score
}
}
if best == nil {
return nil, ErrNoLoginForm
}
return best, nil
}
// collectForms walks the parse tree depth-first and appends every <form> node.
func collectForms(n *html.Node, out *[]*html.Node) {
if n.Type == html.ElementNode && n.Data == "form" {
*out = append(*out, n)
}
for c := n.FirstChild; c != nil; c = c.NextSibling {
collectForms(c, out)
}
}
// fieldNode is a flattened view of a form control relevant to login detection.
type fieldNode struct {
tag string // input, select, textarea, button
typ string // lower-cased type attribute (inputs)
name string
id string
value string
autoComp string // autocomplete attribute
}
// buildLoginForm inspects a single <form> node and, if it looks like a
// credential-login form (contains exactly one usable password field), returns a
// populated LoginForm together with a heuristic score used to pick the best
// form on a page. A nil form means "not a login form".
func buildLoginForm(formNode *html.Node, pageURL *url.URL) (*LoginForm, int) {
var fields []fieldNode
collectFields(formNode, &fields)
// Locate the password field. A form with no password field is not a
// credential-login form; a form with several password fields is almost
// always a registration / change-password form, so we skip it.
passwordIdx := -1
passwordCount := 0
for i, f := range fields {
if f.tag == "input" && f.typ == "password" {
passwordCount++
if passwordIdx == -1 {
passwordIdx = i
}
}
}
if passwordIdx == -1 || passwordCount > 1 {
return nil, 0
}
password := fields[passwordIdx]
if password.name == "" {
// A password field with no name cannot be submitted.
return nil, 0
}
form := &LoginForm{
Method: resolveMethod(formNode),
EncType: strings.ToLower(strings.TrimSpace(getAttr(formNode, "enctype"))),
PasswordField: password.name,
Fields: map[string]string{},
}
form.Action = resolveAction(formNode, pageURL)
usernameIdx := pickUsernameField(fields, passwordIdx)
if usernameIdx >= 0 {
form.UsernameField = fields[usernameIdx].name
}
// Carry along hidden inputs (CSRF tokens etc.), pre-filled defaults and a
// single named submit button, but never the username/password controls.
submitAdded := false
for i, f := range fields {
if i == passwordIdx || i == usernameIdx || f.name == "" {
continue
}
switch {
case f.tag == "input" && f.typ == "hidden":
form.Fields[f.name] = f.value
case f.tag == "input" && (f.typ == "submit" || f.typ == "image"):
// Many backends key off the submit button name; include the first one.
if !submitAdded && f.value != "" {
form.Fields[f.name] = f.value
submitAdded = true
}
case f.tag == "button" && (f.typ == "submit" || f.typ == ""):
if !submitAdded && f.value != "" {
form.Fields[f.name] = f.value
submitAdded = true
}
case f.value != "" && f.typ != "checkbox" && f.typ != "radio":
// Preserve pre-filled defaults for other text-like inputs so we don't
// drop values the server expects.
form.Fields[f.name] = f.value
}
}
// Score: a real login form usually has both a username and a password field;
// CSRF/hidden tokens raise confidence further. This lets DetectLoginForm
// prefer the actual login form over, say, a newsletter box that happens to
// share the page.
score := 1
if form.UsernameField != "" {
score += 2
}
if len(form.Fields) > 0 {
score++
}
return form, score
}
// collectFields flattens the form controls within a form node in document order.
func collectFields(n *html.Node, out *[]fieldNode) {
for c := n.FirstChild; c != nil; c = c.NextSibling {
if c.Type == html.ElementNode {
switch c.Data {
case "input", "select", "textarea", "button":
*out = append(*out, fieldNode{
tag: c.Data,
typ: strings.ToLower(strings.TrimSpace(getAttr(c, "type"))),
name: getAttr(c, "name"),
id: getAttr(c, "id"),
value: getAttr(c, "value"),
autoComp: strings.ToLower(strings.TrimSpace(getAttr(c, "autocomplete"))),
})
}
}
collectFields(c, out)
}
}
// pickUsernameField selects the most likely username/identifier field. It
// considers only text-like inputs appearing before the password field and
// prefers ones whose name/id/autocomplete match a known hint token; otherwise
// it falls back to the closest text-like input preceding the password.
func pickUsernameField(fields []fieldNode, passwordIdx int) int {
isCandidate := func(f fieldNode) bool {
if f.tag != "input" {
return false
}
switch f.typ {
case "", "text", "email", "tel", "search":
return f.name != ""
default:
return false
}
}
// First pass: hint-token match among candidates preceding the password field.
bestHint := -1
bestHintRank := len(usernameHintTokens)
for i := 0; i < passwordIdx; i++ {
f := fields[i]
if !isCandidate(f) {
continue
}
hay := strings.ToLower(f.name + " " + f.id + " " + f.autoComp)
for rank, tok := range usernameHintTokens {
if strings.Contains(hay, tok) && rank < bestHintRank {
bestHint = i
bestHintRank = rank
}
}
}
if bestHint >= 0 {
return bestHint
}
// Second pass: closest text-like input before the password field.
for i := passwordIdx - 1; i >= 0; i-- {
if isCandidate(fields[i]) {
return i
}
}
// Last resort: first text-like input anywhere in the form (covers oddly
// ordered markup where the username sits after the password).
for i := range fields {
if i == passwordIdx {
continue
}
if isCandidate(fields[i]) {
return i
}
}
return -1
}
// resolveMethod returns the upper-cased form method, defaulting to POST. The
// HTML default is GET, but credential-login forms virtually always POST, and a
// GET login would leak credentials into the URL/query — so for an autonomous
// login submitter POST is the safer default when unspecified.
func resolveMethod(formNode *html.Node) string {
m := strings.ToUpper(strings.TrimSpace(getAttr(formNode, "method")))
if m == "" {
return "POST"
}
return m
}
// resolveAction resolves the form action against the page URL. An empty/missing
// action submits back to the page itself.
func resolveAction(formNode *html.Node, pageURL *url.URL) string {
action := strings.TrimSpace(getAttr(formNode, "action"))
if action == "" {
if pageURL != nil {
return pageURL.String()
}
return ""
}
if pageURL == nil {
return action
}
ref, err := url.Parse(action)
if err != nil {
return action
}
return pageURL.ResolveReference(ref).String()
}
// getAttr returns the value of the named attribute (case-insensitive) on an
// element node, or "" if absent.
func getAttr(n *html.Node, key string) string {
for _, a := range n.Attr {
if strings.EqualFold(a.Key, key) {
return a.Val
}
}
return ""
}
-177
View File
@@ -1,177 +0,0 @@
package autologin
import (
"net/url"
"testing"
"github.com/stretchr/testify/require"
)
func mustURL(t *testing.T, raw string) *url.URL {
t.Helper()
u, err := url.Parse(raw)
require.NoError(t, err)
return u
}
func TestDetectLoginForm_Simple(t *testing.T) {
body := `<html><body>
<form action="/login" method="post">
<input type="text" name="username">
<input type="password" name="password">
<button type="submit">Log in</button>
</form>
</body></html>`
form, err := DetectLoginForm(body, mustURL(t, "https://app.example.com/auth"))
require.NoError(t, err)
require.Equal(t, "POST", form.Method)
require.Equal(t, "https://app.example.com/login", form.Action)
require.Equal(t, "username", form.UsernameField)
require.Equal(t, "password", form.PasswordField)
}
func TestDetectLoginForm_EmailAndCSRF(t *testing.T) {
body := `<html><body>
<form action="https://id.example.com/sessions" method="POST">
<input type="hidden" name="csrf_token" value="abc123">
<input type="email" name="email" autocomplete="username">
<input type="password" name="pass">
<input type="submit" name="commit" value="Sign in">
</form>
</body></html>`
form, err := DetectLoginForm(body, mustURL(t, "https://id.example.com/login"))
require.NoError(t, err)
require.Equal(t, "https://id.example.com/sessions", form.Action)
require.Equal(t, "email", form.UsernameField)
require.Equal(t, "pass", form.PasswordField)
require.Equal(t, "abc123", form.Fields["csrf_token"], "CSRF hidden input must be carried")
require.Equal(t, "Sign in", form.Fields["commit"], "named submit button should be carried")
require.NotContains(t, form.Fields, "email", "username field must not be in extra fields")
require.NotContains(t, form.Fields, "pass", "password field must not be in extra fields")
}
func TestDetectLoginForm_RelativeActionResolution(t *testing.T) {
body := `<form action="../session" method="post">
<input type="text" name="user">
<input type="password" name="pwd">
</form>`
form, err := DetectLoginForm(body, mustURL(t, "https://example.com/app/login"))
require.NoError(t, err)
require.Equal(t, "https://example.com/session", form.Action)
}
func TestDetectLoginForm_MissingActionUsesPageURL(t *testing.T) {
body := `<form method="post">
<input type="text" name="user">
<input type="password" name="pwd">
</form>`
form, err := DetectLoginForm(body, mustURL(t, "https://example.com/login?next=/home"))
require.NoError(t, err)
require.Equal(t, "https://example.com/login?next=/home", form.Action)
}
func TestDetectLoginForm_DefaultMethodIsPost(t *testing.T) {
body := `<form action="/login">
<input type="text" name="user">
<input type="password" name="pwd">
</form>`
form, err := DetectLoginForm(body, mustURL(t, "https://example.com/login"))
require.NoError(t, err)
require.Equal(t, "POST", form.Method)
}
func TestDetectLoginForm_PicksLoginFormAmongMany(t *testing.T) {
body := `<html><body>
<form action="/search" method="get">
<input type="text" name="q">
<input type="submit" value="Search">
</form>
<form action="/newsletter" method="post">
<input type="email" name="newsletter_email">
<input type="submit" value="Subscribe">
</form>
<form action="/login" method="post">
<input type="hidden" name="_token" value="xyz">
<input type="text" name="login" autocomplete="username">
<input type="password" name="password">
<input type="submit" value="Sign in">
</form>
</body></html>`
form, err := DetectLoginForm(body, mustURL(t, "https://example.com/"))
require.NoError(t, err)
require.Equal(t, "https://example.com/login", form.Action)
require.Equal(t, "login", form.UsernameField)
require.Equal(t, "password", form.PasswordField)
require.Equal(t, "xyz", form.Fields["_token"])
}
func TestDetectLoginForm_UsernameHintBeatsOrder(t *testing.T) {
// Two text inputs precede the password; the one with a username hint should
// win even though it is not the closest to the password field.
body := `<form action="/login" method="post">
<input type="text" name="email" autocomplete="username">
<input type="text" name="captcha">
<input type="password" name="password">
</form>`
form, err := DetectLoginForm(body, mustURL(t, "https://example.com/login"))
require.NoError(t, err)
require.Equal(t, "email", form.UsernameField)
require.NotContains(t, form.Fields, "captcha", "valueless non-username text input should not be injected")
}
func TestDetectLoginForm_NoPasswordField(t *testing.T) {
body := `<form action="/search" method="get">
<input type="text" name="q">
<input type="submit" value="Search">
</form>`
_, err := DetectLoginForm(body, mustURL(t, "https://example.com/"))
require.ErrorIs(t, err, ErrNoLoginForm)
}
func TestDetectLoginForm_SkipsMultiPasswordForms(t *testing.T) {
// A registration / change-password form (two password fields) must not be
// treated as a login form.
body := `<form action="/register" method="post">
<input type="text" name="username">
<input type="password" name="password">
<input type="password" name="password_confirm">
</form>`
_, err := DetectLoginForm(body, mustURL(t, "https://example.com/"))
require.ErrorIs(t, err, ErrNoLoginForm)
}
func TestDetectLoginForm_PasswordOnlyForm(t *testing.T) {
// A password-only form (no username) is still a valid login form.
body := `<form action="/unlock" method="post">
<input type="hidden" name="token" value="t">
<input type="password" name="password">
</form>`
form, err := DetectLoginForm(body, mustURL(t, "https://example.com/unlock"))
require.NoError(t, err)
require.Equal(t, "", form.UsernameField)
require.Equal(t, "password", form.PasswordField)
require.Equal(t, "t", form.Fields["token"])
}
func TestDetectLoginForm_DefaultTypeIsText(t *testing.T) {
// An input with no type attribute defaults to text and should be eligible as
// the username field.
body := `<form action="/login" method="post">
<input name="user">
<input type="password" name="pwd">
</form>`
form, err := DetectLoginForm(body, mustURL(t, "https://example.com/login"))
require.NoError(t, err)
require.Equal(t, "user", form.UsernameField)
}
-753
View File
@@ -1,753 +0,0 @@
package autologin
import (
"context"
"encoding/json"
"fmt"
"net/http"
"net/url"
"os"
"regexp"
"sort"
"strings"
"sync"
"time"
"github.com/go-rod/rod"
"github.com/go-rod/rod/lib/input"
"github.com/go-rod/rod/lib/launcher"
"github.com/go-rod/rod/lib/proto"
"github.com/projectdiscovery/gologger"
"github.com/projectdiscovery/utils/errkit"
fileutil "github.com/projectdiscovery/utils/file"
osutils "github.com/projectdiscovery/utils/os"
)
// LoginHeadless performs a browser-driven auto-login using go-rod. Unlike the
// HTTP engine (Login), it executes JavaScript, so it works on SPA/JS-rendered
// login pages, client-side form construction and multi-step / SSO redirect
// flows that the static HTTP form submitter cannot handle. It navigates to the
// login page in a real (headless) browser, fills the detected credential
// fields, submits, waits for the page to settle and captures the resulting
// session (cookies, plus an optional token from the body or localStorage).
//
// It intentionally drives go-rod directly rather than nuclei's headless engine
// package: that package imports authprovider, and autologin is (transitively)
// imported by authprovider, so depending on it would create an import cycle.
func LoginHeadless(ctx context.Context, cfg Config) (*Session, error) {
if strings.TrimSpace(cfg.LoginURL) == "" {
return nil, errkit.New("auto-login(headless): login-url is required")
}
loginURL, err := url.Parse(cfg.LoginURL)
if err != nil {
return nil, errkit.Wrap(err, "auto-login(headless): invalid login-url")
}
var tokenRe *regexp.Regexp
if cfg.TokenRegex != "" {
if tokenRe, err = regexp.Compile(cfg.TokenRegex); err != nil {
return nil, errkit.Wrap(err, "auto-login(headless): invalid token-regex")
}
}
timeout := cfg.Timeout
if timeout == 0 {
timeout = 45 * time.Second
}
settle := cfg.SettleTime
if settle == 0 {
settle = 5 * time.Second
}
browser, cleanup, err := launchBrowser(cfg)
if err != nil {
return nil, errkit.Wrap(err, "auto-login(headless): failed to launch browser")
}
defer cleanup()
// Use an incognito context for a clean, isolated cookie jar per login.
incognito, err := browser.Incognito()
if err != nil {
return nil, errkit.Wrap(err, "auto-login(headless): failed to create incognito context")
}
page, err := incognito.Page(proto.TargetCreateTarget{})
if err != nil {
return nil, errkit.Wrap(err, "auto-login(headless): failed to open page")
}
defer func() { _ = page.Close() }()
page = page.Context(ctx).Timeout(timeout)
// Passively observe response headers so a token delivered via a response
// header (Authorization, X-Auth-Token, ...) during an XHR login is captured,
// mirroring the HTTP engine. This is observation-only (no request hijacking)
// to avoid interfering with the login flow. The listener stops when the page
// is closed (deferred above).
//
// Two safeguards keep this from latching onto the wrong token: we only honor
// responses served by the login origin (a page loads many third-party
// requests — CDNs, analytics — whose headers must not be mistaken for the
// app's session token), and we keep the first match (first-party auth
// responses come early; a later beacon must not clobber the real token).
var (
headerTokenMu sync.Mutex
headerToken string
)
if err := (proto.NetworkEnable{}).Call(page); err == nil {
go page.EachEvent(func(e *proto.NetworkResponseReceived) {
if e.Response == nil {
return
}
if !sameLoginOrigin(loginURL, e.Response.URL) {
return
}
hdr := make(http.Header, len(e.Response.Headers))
for k, v := range e.Response.Headers {
hdr.Set(k, v.Str())
}
if tok := detectHeaderToken(hdr, tokenRe); tok != "" {
headerTokenMu.Lock()
if headerToken == "" {
headerToken = tok
}
headerTokenMu.Unlock()
}
})()
}
// Block a 307/308 replay of the login form submit (which carries the
// password) to another origin, as the HTTP engine does. It must be in place
// before the first navigation.
var replayGuard *redirectReplayGuard
if !cfg.AllowCrossOriginAction {
if replayGuard, err = guardRedirectReplay(page); err != nil {
return nil, errkit.Wrap(err, "auto-login(headless): failed to enable request interception")
}
}
// Apply UA / custom headers before navigating so the login request carries
// the same identity as the scan.
if cfg.UserAgent != "" {
if uaErr := page.SetUserAgent(&proto.NetworkSetUserAgentOverride{UserAgent: cfg.UserAgent}); uaErr != nil {
return nil, errkit.Wrap(uaErr, "auto-login(headless): failed to set user-agent")
}
}
if len(cfg.CustomHeaders) > 0 {
pairs := make([]string, 0, len(cfg.CustomHeaders)*2)
for k, v := range cfg.CustomHeaders {
pairs = append(pairs, k, v)
}
if _, hErr := page.SetExtraHeaders(pairs); hErr != nil {
return nil, errkit.Wrap(hErr, "auto-login(headless): failed to set custom headers")
}
}
if err := page.Navigate(cfg.LoginURL); err != nil {
return nil, errkit.Wrap(err, "auto-login(headless): failed to navigate to login page")
}
if err := page.WaitLoad(); err != nil {
return nil, errkit.Wrap(err, "auto-login(headless): login page failed to load")
}
// Give client-side scripts a chance to render the form.
_ = rod.Try(func() { page.Timeout(settle).MustWaitStable() })
if len(cfg.Steps) > 0 {
// Explicit multi-step flow (username-first / SSO / consent screens).
if err := runLoginSteps(ctx, page, cfg, settle); err != nil {
return nil, err
}
} else if err := autoFillAndSubmit(ctx, page, cfg, loginURL, settle); err != nil {
return nil, err
}
// Wait for the post-submit navigation / SPA state to settle.
_ = rod.Try(func() { page.Timeout(settle).MustWaitStable() })
_ = page.WaitLoad()
if blocked := replayGuard.blockedURL(); blocked != "" {
return nil, errkit.Newf("auto-login(headless): refused to replay the login form to a different origin (%s); set allow-cross-origin-action to send the credentials there", blocked)
}
session := &Session{}
if info, ierr := page.Info(); ierr == nil {
session.FinalURL = info.URL
}
// Capture cookies and web storage from the browser for the login + final hosts.
session.Cookies = capturePageCookies(page, cfg.LoginURL, session.FinalURL)
session.CookieHeader = renderCookieHeader(session.Cookies)
session.LocalStorage = readStorage(page, "localStorage")
session.SessionStorage = readStorage(page, "sessionStorage")
// Token extraction: search the rendered body and web storage.
if tokenRe != nil {
var haystack strings.Builder
if html, herr := page.HTML(); herr == nil {
haystack.WriteString(html)
}
for _, store := range []map[string]string{session.LocalStorage, session.SessionStorage} {
for k, v := range store {
haystack.WriteString("\n")
haystack.WriteString(k)
haystack.WriteString("=")
haystack.WriteString(v)
}
}
if m := tokenRe.FindStringSubmatch(haystack.String()); len(m) > 1 {
session.Token = m[1]
}
}
// Fallback: many SPAs keep their bearer token in web storage (e.g.
// localStorage["token"]) rather than a cookie. When no explicit token-regex
// matched, surface a JWT-shaped value from web storage as the session token
// so HTTP scan requests carry an Authorization header without the user
// having to specify a token-regex.
if session.Token == "" {
session.Token = detectStorageJWT(session.LocalStorage, session.SessionStorage)
}
// Last resort: a token observed in a response header during the login (e.g.
// an XHR auth call that returned Authorization / X-Auth-Token).
if session.Token == "" {
headerTokenMu.Lock()
session.Token = headerToken
headerTokenMu.Unlock()
}
// Success heuristic: being re-prompted (final page still presents a login
// form) with no token means the login failed.
if session.Token == "" {
if html, herr := page.HTML(); herr == nil {
var finalParsed *url.URL
if session.FinalURL != "" {
finalParsed, _ = url.Parse(session.FinalURL)
}
if _, derr := DetectLoginForm(html, finalParsed); derr == nil {
return nil, errkit.Wrapf(ErrLoginFailed, "still presented a login form at %s", session.FinalURL)
}
}
}
if len(session.Cookies) == 0 && session.Token == "" && len(session.LocalStorage) == 0 && len(session.SessionStorage) == 0 {
return nil, errkit.Wrapf(ErrLoginFailed, "no cookies, token or web storage captured at %s", session.FinalURL)
}
return session, nil
}
// redirectReplayGuard fails redirected requests that would replay a request
// body to another origin. It sees every request of the login page at the
// request stage (CDP Fetch), so it also covers the browser's own redirects.
type redirectReplayGuard struct {
mu sync.Mutex
urls map[proto.FetchRequestID]string
blocked string
}
// guardRedirectReplay enables request-stage interception on page. Each paused
// request continues unchanged, except a redirected request that keeps a body
// (a 307/308 of a POST, such as the login form submit with the password) to a
// different origin than the request that was redirected: that one fails before
// it leaves the browser. Interception is only used for the auto-login page.
func guardRedirectReplay(page *rod.Page) (*redirectReplayGuard, error) {
err := proto.FetchEnable{Patterns: []*proto.FetchRequestPattern{{
URLPattern: "*",
RequestStage: proto.FetchRequestStageRequest,
}}}.Call(page)
if err != nil {
return nil, err
}
g := &redirectReplayGuard{urls: map[proto.FetchRequestID]string{}}
go page.EachEvent(func(e *proto.FetchRequestPaused) {
if e.Request == nil {
_ = proto.FetchContinueRequest{RequestID: e.RequestID}.Call(page)
return
}
g.mu.Lock()
g.urls[e.RequestID] = e.Request.URL
from, known := g.urls[e.RedirectedRequestID]
g.mu.Unlock()
if e.RedirectedRequestID != "" && blockRedirectReplay(e.Request.Method, from, known, e.Request.URL) {
g.mu.Lock()
if g.blocked == "" {
g.blocked = redactURL(e.Request.URL)
}
g.mu.Unlock()
gologger.Warning().Msgf("auto-login(headless): blocked a %s redirect of the login form to a different origin (%s)", e.Request.Method, redactURL(e.Request.URL))
_ = proto.FetchFailRequest{RequestID: e.RequestID, ErrorReason: proto.NetworkErrorReasonBlockedByClient}.Call(page)
return
}
_ = proto.FetchContinueRequest{RequestID: e.RequestID}.Call(page)
})()
return g, nil
}
// blockedURL returns the first blocked redirect target, or "" when nothing was
// blocked (or the guard is off).
func (g *redirectReplayGuard) blockedURL() string {
if g == nil {
return ""
}
g.mu.Lock()
defer g.mu.Unlock()
return g.blocked
}
// blockRedirectReplay decides the headless counterpart of checkLoginRedirect:
// a redirect that turns into GET/HEAD drops the body and is allowed; one that
// keeps another method (307/308) is allowed only to the same origin as the
// request that redirected. An unknown source fails closed.
func blockRedirectReplay(method, from string, known bool, to string) bool {
if method == http.MethodGet || method == http.MethodHead {
return false
}
if !known {
return true
}
fromURL, err := url.Parse(from)
if err != nil {
return true
}
toURL, err := url.Parse(to)
if err != nil {
return true
}
return !sameOrigin(fromURL, toURL)
}
func redactURL(raw string) string {
if u, err := url.Parse(raw); err == nil {
return u.Redacted()
}
return raw
}
// sameLoginOrigin reports whether respURL shares the login page's origin
// (scheme + host[:port]). It is used to ignore third-party (CDN/analytics)
// responses when passively scavenging a session token from response headers.
// An unparsable respURL is treated as not matching (fail closed).
func sameLoginOrigin(loginURL *url.URL, respURL string) bool {
if loginURL == nil || respURL == "" {
return false
}
u, err := url.Parse(respURL)
if err != nil {
return false
}
return strings.EqualFold(u.Scheme, loginURL.Scheme) && strings.EqualFold(u.Host, loginURL.Host)
}
// jwtValueRe matches a JWT-shaped value (header.payload.signature in base64url).
// It is intentionally not anchored so it also matches tokens wrapped in quotes
// or embedded in a small JSON blob stored in web storage.
var jwtValueRe = regexp.MustCompile(`eyJ[A-Za-z0-9_-]{6,}\.[A-Za-z0-9_-]{6,}\.[A-Za-z0-9_-]{4,}`)
// detectStorageJWT returns the first JWT-shaped value found across the supplied
// web storage maps, or "" when none is present. Keys that conventionally hold an
// auth token are inspected first (and otherwise keys are visited in sorted order)
// so the result is deterministic across runs.
func detectStorageJWT(stores ...map[string]string) string {
preferredKeys := []string{"token", "access_token", "accesstoken", "jwt", "auth_token", "authtoken", "id_token", "idtoken", "bearer"}
for _, store := range stores {
if len(store) == 0 {
continue
}
for _, want := range preferredKeys {
for k, v := range store {
if strings.EqualFold(k, want) {
if m := jwtValueRe.FindString(v); m != "" {
return m
}
}
}
}
}
for _, store := range stores {
if len(store) == 0 {
continue
}
keys := make([]string, 0, len(store))
for k := range store {
keys = append(keys, k)
}
sort.Strings(keys)
for _, k := range keys {
if m := jwtValueRe.FindString(store[k]); m != "" {
return m
}
}
}
return ""
}
// launchBrowser launches (or connects to) a browser and returns it with a
// cleanup func. A system-installed Chrome is preferred when available to avoid
// triggering a managed-browser download.
func launchBrowser(cfg Config) (*rod.Browser, func(), error) {
if cfg.CDPEndpoint != "" {
b := rod.New().ControlURL(cfg.CDPEndpoint)
if err := b.Connect(); err != nil {
return nil, nil, err
}
return b, func() { _ = b.Close() }, nil
}
l := launcher.New().
Set("disable-gpu").
Set("disable-notifications").
Set("ignore-certificate-errors").
Set("ignore-ssl-errors").
Set("incognito").
Headless(!cfg.ShowBrowser).
Leakless(false)
if osutils.IsLinux() {
l = l.NoSandbox(true)
}
if cfg.Proxy != "" {
l = l.Proxy(cfg.Proxy)
}
// Mirror nuclei's headless engine browser selection: on musl (most likely
// Alpine) the managed Chromium download cannot run, so a system Chrome is
// required — as it is when the user forces it. Otherwise prefer a system
// Chrome when present and fall back to go-rod's managed browser, logging so a
// one-time download is not surprising in a server/CI context.
executablePath, _ := os.Executable()
useMusl, _ := fileutil.UseMusl(executablePath)
if path, ok := launcher.LookPath(); ok {
l = l.Bin(path)
} else if cfg.UseInstalledChrome || useMusl {
return nil, nil, errkit.New("auto-login(headless): no chrome/chromium binary found (a system install is required here)")
} else {
gologger.Info().Msgf("auto-login(headless): no system chrome found, using go-rod managed browser (a one-time download may occur)")
}
controlURL, err := l.Launch()
if err != nil {
return nil, nil, err
}
b := rod.New().ControlURL(controlURL)
if err := b.Connect(); err != nil {
l.Kill()
return nil, nil, err
}
return b, func() { _ = b.Close(); l.Kill() }, nil
}
// autoFillAndSubmit runs the default single-shot login: detect the password
// (and username) field, type the credentials and submit.
func autoFillAndSubmit(ctx context.Context, page *rod.Page, cfg Config, loginURL *url.URL, settle time.Duration) error {
// SPA forms are frequently injected after an async tick, so the password
// field may not exist immediately after the initial settle. Wait (bounded by
// settle) for it to appear before detecting, rather than failing outright.
_ = waitVisible(ctx, page, `input[type="password"]`, settle)
// Prefer field names detected from the rendered DOM; fall back to type-based
// selectors for SPA inputs that carry no name attribute.
var detected *LoginForm
if html, herr := page.HTML(); herr == nil {
detected, _ = DetectLoginForm(html, loginURL)
}
passEl := locatePasswordField(page, cfg, detected)
if passEl == nil {
return errkit.Wrap(ErrNoLoginForm, "auto-login(headless): no password field found in rendered page")
}
if userEl := locateUsernameField(page, cfg, detected); userEl != nil && cfg.Username != "" {
if err := typeInto(userEl, cfg.Username); err != nil {
return errkit.Wrap(err, "auto-login(headless): failed to type username")
}
}
if err := typeInto(passEl, cfg.Password); err != nil {
return errkit.Wrap(err, "auto-login(headless): failed to type password")
}
// Let the framework register the typed input and validate the form (which
// often gates the submit button) before attempting to submit.
_ = rod.Try(func() { page.Timeout(settle).MustWaitStable() })
return submitForm(page, passEl, settle)
}
// submitForm submits a login form. It prefers clicking a submit control, but
// frameworks frequently keep that control disabled until the form validates, so
// the click is bounded by interactable-wait: if the button never becomes
// interactable (or the click fails) it falls back to pressing Enter in the
// given field, which reliably triggers native/framework form submission.
func submitForm(page *rod.Page, fallbackField *rod.Element, settle time.Duration) error {
clickTimeout := settle
if clickTimeout <= 0 {
clickTimeout = 5 * time.Second
}
if btn := findVisible(page, `button[type="submit"]`, `input[type="submit"]`, `button:not([type])`, `[role="button"]`); btn != nil {
// Bound the interactable-wait so a permanently-disabled button can't hang
// the whole login; on failure we fall through to Enter-to-submit.
clickErr := rod.Try(func() {
b := btn.Timeout(clickTimeout)
b.MustWaitInteractable()
_ = b.ScrollIntoView()
b.MustClick()
})
if clickErr == nil {
return nil
}
if fallbackField == nil {
return errkit.Wrapf(ErrLoginFailed, "auto-login(headless): submit button never became interactable: %s", clickErr)
}
// fall through to Enter-to-submit
}
if fallbackField != nil {
if err := fallbackField.Type(input.Enter); err != nil {
return errkit.Wrap(err, "auto-login(headless): failed to submit form")
}
return nil
}
return errkit.Wrap(ErrLoginFailed, "auto-login(headless): no submit control found and no field to submit")
}
// runLoginSteps executes an explicit multi-step login flow.
func runLoginSteps(ctx context.Context, page *rod.Page, cfg Config, settle time.Duration) error {
for i, step := range cfg.Steps {
action := strings.ToLower(strings.TrimSpace(step.Action))
switch action {
case "navigate":
if err := page.Navigate(step.Value); err != nil {
return errkit.Wrapf(err, "login step %d (navigate): failed", i)
}
_ = page.WaitLoad()
case "fill", "input", "type":
el := findVisible(page, byName(step.Selector), step.Selector)
if el == nil {
return errkit.Newf("login step %d (fill): element not found: %s", i, step.Selector)
}
if err := typeInto(el, expandCredentials(step.Value, cfg)); err != nil {
return errkit.Wrapf(err, "login step %d (fill): failed", i)
}
case "click":
el := findVisible(page, step.Selector)
if el == nil {
return errkit.Newf("login step %d (click): element not found: %s", i, step.Selector)
}
_ = el.ScrollIntoView()
if err := el.Click(proto.InputMouseButtonLeft, 1); err != nil {
return errkit.Wrapf(err, "login step %d (click): failed", i)
}
case "waitvisible":
if err := waitVisible(ctx, page, step.Selector, settle); err != nil {
return errkit.Wrapf(err, "login step %d (waitvisible)", i)
}
case "wait":
d := settle
if step.Value != "" {
if pd, perr := time.ParseDuration(step.Value); perr == nil {
d = pd
}
}
select {
case <-time.After(d):
case <-ctx.Done():
return ctx.Err()
}
case "press":
key, kerr := keyFromName(step.Value)
if kerr != nil {
return errkit.Wrapf(kerr, "login step %d (press)", i)
}
if step.Selector != "" {
el := findVisible(page, byName(step.Selector), step.Selector)
if el == nil {
return errkit.Newf("login step %d (press): element not found: %s", i, step.Selector)
}
if err := el.Type(key); err != nil {
return errkit.Wrapf(err, "login step %d (press): failed", i)
}
} else if err := page.Keyboard.Type(key); err != nil {
return errkit.Wrapf(err, "login step %d (press): failed", i)
}
case "submit":
if err := submitForm(page, findVisible(page, `input[type="password"]`), settle); err != nil {
return errkit.Wrapf(err, "login step %d (submit)", i)
}
default:
return errkit.Newf("login step %d: unknown action %q", i, step.Action)
}
// Let the page settle between steps so the next selector is present.
_ = rod.Try(func() { page.Timeout(settle).MustWaitStable() })
}
return nil
}
// expandCredentials substitutes the {{username}}/{{password}} placeholders in a
// step value with the configured credentials.
func expandCredentials(value string, cfg Config) string {
value = strings.ReplaceAll(value, "{{username}}", cfg.Username)
value = strings.ReplaceAll(value, "{{password}}", cfg.Password)
return value
}
// keyFromName maps a friendly key name to a go-rod input key.
func keyFromName(name string) (input.Key, error) {
switch strings.ToLower(strings.TrimSpace(name)) {
case "", "enter", "return":
return input.Enter, nil
case "tab":
return input.Tab, nil
case "escape", "esc":
return input.Escape, nil
case "space":
return input.Space, nil
default:
return 0, errkit.Newf("unsupported key %q (supported: enter, tab, escape, space)", name)
}
}
// waitVisible polls until the selector becomes visible or the timeout elapses.
func waitVisible(ctx context.Context, page *rod.Page, selector string, timeout time.Duration) error {
deadline := time.Now().Add(timeout)
for time.Now().Before(deadline) {
if findVisible(page, selector) != nil {
return nil
}
select {
case <-time.After(150 * time.Millisecond):
case <-ctx.Done():
return ctx.Err()
}
}
return errkit.Newf("timeout waiting for element to become visible: %s", selector)
}
// locatePasswordField finds the password input, honoring an explicit override
// (treated as a field name first, then as a raw CSS selector), then the
// detected name, then a generic selector.
func locatePasswordField(page *rod.Page, cfg Config, detected *LoginForm) *rod.Element {
if cfg.PasswordField != "" {
if el := findVisible(page, byName(cfg.PasswordField), cfg.PasswordField); el != nil {
return el
}
}
if detected != nil && detected.PasswordField != "" {
if el := findVisible(page, byName(detected.PasswordField)); el != nil {
return el
}
}
return findVisible(page, `input[type="password"]`)
}
// locateUsernameField finds the username/identifier input using the same
// override/detected/heuristic precedence as the password field.
func locateUsernameField(page *rod.Page, cfg Config, detected *LoginForm) *rod.Element {
if cfg.UsernameField != "" {
if el := findVisible(page, byName(cfg.UsernameField), cfg.UsernameField); el != nil {
return el
}
}
if detected != nil && detected.UsernameField != "" {
if el := findVisible(page, byName(detected.UsernameField)); el != nil {
return el
}
}
return findVisible(page,
`input[type="email"]`,
`input[autocomplete="username"]`,
`input[name*="user" i]`,
`input[name*="email" i]`,
`input[name*="login" i]`,
`input[type="text"]`,
)
}
// byName builds a CSS attribute selector matching an input by its name.
func byName(name string) string {
return fmt.Sprintf(`input[name=%q]`, name)
}
// findVisible returns the first visible element matching any of the selectors
// in priority order, or nil if none match.
func findVisible(page *rod.Page, selectors ...string) *rod.Element {
for _, sel := range selectors {
els, err := queryElements(page, sel)
if err != nil {
continue
}
for _, el := range els {
if vis, verr := el.Visible(); verr == nil && vis {
return el
}
}
}
return nil
}
// queryElements resolves a selector that may be CSS (default) or XPath. XPath
// selectors are prefixed with "xpath=" (emitted by the recording importer) or
// written as a raw "//"/"(//" expression. This lets recorded login flows, whose
// most stable selector is often an XPath, replay reliably.
func queryElements(page *rod.Page, sel string) (rod.Elements, error) {
switch {
case strings.HasPrefix(sel, "xpath="):
return page.ElementsX(strings.TrimPrefix(sel, "xpath="))
case strings.HasPrefix(sel, "//"), strings.HasPrefix(sel, "(//"):
return page.ElementsX(sel)
default:
return page.Elements(sel)
}
}
// typeInto focuses the element, clears any existing value and types the text so
// that JS frameworks observing input events (e.g. React controlled inputs) pick
// up the change.
func typeInto(el *rod.Element, text string) error {
_ = el.ScrollIntoView()
if err := el.Focus(); err != nil {
return err
}
// Clear any pre-filled value by selecting all then overwriting.
_ = el.SelectAllText()
return el.Input(text)
}
// capturePageCookies pulls cookies the browser holds for the given URLs and
// converts them to net/http cookies, de-duplicated by name+domain+path.
func capturePageCookies(page *rod.Page, urls ...string) []*http.Cookie {
filtered := urls[:0]
for _, u := range urls {
if u != "" {
filtered = append(filtered, u)
}
}
cookies, err := page.Cookies(filtered)
if err != nil {
return nil
}
seen := map[string]bool{}
var out []*http.Cookie
for _, c := range cookies {
if c == nil {
continue
}
key := c.Name + "\x00" + c.Domain + "\x00" + c.Path
if seen[key] {
continue
}
seen[key] = true
out = append(out, &http.Cookie{Name: c.Name, Value: c.Value, Domain: c.Domain, Path: c.Path, HttpOnly: c.HTTPOnly, Secure: c.Secure})
}
return out
}
// readStorage returns the given web storage area (localStorage/sessionStorage)
// as a map, or nil on failure.
func readStorage(page *rod.Page, area string) map[string]string {
obj, err := page.Eval(fmt.Sprintf(`() => JSON.stringify(window.%s)`, area))
if err != nil || obj == nil {
return nil
}
raw := obj.Value.Str()
if raw == "" {
return nil
}
out := map[string]string{}
if err := json.Unmarshal([]byte(raw), &out); err != nil {
return nil
}
if len(out) == 0 {
return nil
}
return out
}
@@ -1,108 +0,0 @@
package autologin
import (
"context"
"net/http"
"net/http/httptest"
"sync/atomic"
"testing"
"time"
"github.com/stretchr/testify/require"
)
func TestBlockRedirectReplay(t *testing.T) {
const a, b = "http://127.0.0.1:8001/do-login", "http://127.0.0.1:8002/steal"
require.True(t, blockRedirectReplay(http.MethodPost, a, true, b), "cross-origin 307/308 POST must be blocked")
require.False(t, blockRedirectReplay(http.MethodPost, a, true, "http://127.0.0.1:8001/next"), "same-origin 307/308 POST is allowed")
require.False(t, blockRedirectReplay(http.MethodGet, a, true, b), "a redirect that became GET carries no body")
require.False(t, blockRedirectReplay(http.MethodHead, a, true, b))
require.True(t, blockRedirectReplay(http.MethodPut, a, true, b))
require.True(t, blockRedirectReplay(http.MethodPost, "", false, b), "unknown redirect source fails closed")
require.True(t, blockRedirectReplay(http.MethodPost, "http://127.0.0.1:8001/x", true, "https://127.0.0.1:8001/x"), "scheme change is a different origin")
}
// redirectApps starts a login origin whose form submit answers 307 to target
// (resolved lazily), and a second origin that counts POSTs it receives.
func redirectApps(t *testing.T, sameOriginTarget bool) (login *httptest.Server, other *httptest.Server, posts *atomic.Int32) {
t.Helper()
posts = &atomic.Int32{}
other = httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.Method == http.MethodPost {
posts.Add(1)
}
http.SetCookie(w, &http.Cookie{Name: "session", Value: "other", Path: "/"})
_, _ = w.Write([]byte("<html><body>welcome</body></html>"))
}))
t.Cleanup(other.Close)
login = httptest.NewServer(nil)
login.Config.Handler = http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
switch r.URL.Path {
case "/login":
_, _ = w.Write([]byte(`<html><body><form method="post" action="/do-login"><input name="username"><input type="password" name="password"><button type="submit">Sign in</button></form></body></html>`))
case "/do-login":
target := other.URL + "/steal"
if sameOriginTarget {
target = login.URL + "/final"
}
http.Redirect(w, r, target, http.StatusTemporaryRedirect)
case "/final":
http.SetCookie(w, &http.Cookie{Name: "session", Value: "ok", Path: "/"})
_, _ = w.Write([]byte("<html><body>welcome</body></html>"))
default:
http.NotFound(w, r)
}
})
t.Cleanup(login.Close)
return login, other, posts
}
func TestLoginHeadless_BlocksCrossOriginRedirectReplay(t *testing.T) {
requireChrome(t)
login, _, posts := redirectApps(t, false)
ctx, cancel := context.WithTimeout(context.Background(), 60*time.Second)
defer cancel()
_, err := LoginHeadless(ctx, Config{
LoginURL: login.URL + "/login",
Username: "admin",
Password: "secret",
SettleTime: 1500 * time.Millisecond,
})
require.Error(t, err)
require.Contains(t, err.Error(), "different origin")
require.Zero(t, posts.Load(), "the login POST must not reach the other origin")
}
func TestLoginHeadless_AllowsSameOriginRedirectReplay(t *testing.T) {
requireChrome(t)
login, _, posts := redirectApps(t, true)
ctx, cancel := context.WithTimeout(context.Background(), 60*time.Second)
defer cancel()
session, err := LoginHeadless(ctx, Config{
LoginURL: login.URL + "/login",
Username: "admin",
Password: "secret",
SettleTime: 1500 * time.Millisecond,
})
require.NoError(t, err)
require.NotEmpty(t, session.Cookies)
require.Zero(t, posts.Load())
}
func TestLoginHeadless_AllowCrossOriginActionPermitsReplay(t *testing.T) {
requireChrome(t)
login, _, posts := redirectApps(t, false)
ctx, cancel := context.WithTimeout(context.Background(), 60*time.Second)
defer cancel()
_, _ = LoginHeadless(ctx, Config{
LoginURL: login.URL + "/login",
Username: "admin",
Password: "secret",
AllowCrossOriginAction: true,
SettleTime: 1500 * time.Millisecond,
})
require.Equal(t, int32(1), posts.Load(), "with allow-cross-origin-action the browser follows the 307")
}
-478
View File
@@ -1,478 +0,0 @@
package autologin
import (
"context"
"fmt"
"net/http"
"net/http/httptest"
"sync"
"testing"
"time"
"github.com/go-rod/rod/lib/launcher"
"github.com/stretchr/testify/require"
)
// requireChrome skips the test unless a system Chrome/Chromium is installed, so
// CI without a managed-browser download does not fail or hang.
func requireChrome(t *testing.T) {
t.Helper()
if _, ok := launcher.LookPath(); !ok {
t.Skip("no system chrome/chromium found; skipping headless auto-login test")
}
}
// jsLoginApp serves a login form that is constructed entirely by JavaScript:
// the raw HTML has no <form>, so the static HTTP engine cannot detect or submit
// it — only a real browser that executes JS can. This is the canonical case
// that motivates headless login.
type jsLoginApp struct {
mu sync.Mutex
sessions map[string]bool
}
func newJSLoginApp() *jsLoginApp { return &jsLoginApp{sessions: map[string]bool{}} }
func (a *jsLoginApp) server() *httptest.Server {
mux := http.NewServeMux()
mux.HandleFunc("/login", a.handleLogin)
mux.HandleFunc("/dashboard", a.handleDashboard)
return httptest.NewServer(mux)
}
const jsLoginPage = `<html><head><title>Login</title></head><body>
<div id="app"></div>
<script>
var f = document.createElement('form');
f.setAttribute('action', '/login');
f.setAttribute('method', 'post');
f.innerHTML = '<input type="email" name="email" autocomplete="username">' +
'<input type="password" name="password">' +
'<button type="submit">Sign in</button>';
document.getElementById('app').appendChild(f);
</script></body></html>`
func (a *jsLoginApp) handleLogin(w http.ResponseWriter, r *http.Request) {
if r.Method == http.MethodGet {
w.Header().Set("Content-Type", "text/html")
_, _ = fmt.Fprint(w, jsLoginPage)
return
}
_ = r.ParseForm()
if r.PostFormValue("email") != "[email protected]" || r.PostFormValue("password") != "p@ss" {
w.Header().Set("Content-Type", "text/html")
_, _ = fmt.Fprint(w, jsLoginPage) // re-render: still a (JS) login page
return
}
token := "sess-dave"
a.mu.Lock()
a.sessions[token] = true
a.mu.Unlock()
http.SetCookie(w, &http.Cookie{Name: "session", Value: token, Path: "/"})
http.Redirect(w, r, "/dashboard", http.StatusFound)
}
func (a *jsLoginApp) handleDashboard(w http.ResponseWriter, r *http.Request) {
c, err := r.Cookie("session")
if err != nil {
http.Error(w, "unauthorized", http.StatusUnauthorized)
return
}
a.mu.Lock()
ok := a.sessions[c.Value]
a.mu.Unlock()
if !ok {
http.Error(w, "unauthorized", http.StatusUnauthorized)
return
}
_, _ = fmt.Fprint(w, "<html><body><h1>welcome dave</h1></body></html>")
}
func TestLoginHeadless_JSRenderedForm(t *testing.T) {
requireChrome(t)
app := newJSLoginApp()
srv := app.server()
defer srv.Close()
// Sanity: the HTTP engine cannot handle this page because the form only
// exists after JS runs — the raw HTML has no <form>.
_, httpErr := Login(context.Background(), nil, Config{
LoginURL: srv.URL + "/login",
Username: "[email protected]",
Password: "p@ss",
})
require.ErrorIs(t, httpErr, ErrNoLoginForm, "HTTP engine should fail on a JS-rendered form")
// Headless engine renders the JS, fills and submits the form, and captures
// the session.
ctx, cancel := context.WithTimeout(context.Background(), 60*time.Second)
defer cancel()
session, err := LoginHeadless(ctx, Config{
LoginURL: srv.URL + "/login",
Username: "[email protected]",
Password: "p@ss",
SettleTime: 1500 * time.Millisecond,
})
require.NoError(t, err)
names := map[string]string{}
for _, c := range session.Cookies {
names[c.Name] = c.Value
}
require.Contains(t, names, "session", "headless login should capture the session cookie")
// The captured session must authenticate against a protected route.
req, _ := http.NewRequest(http.MethodGet, srv.URL+"/dashboard", nil)
req.Header.Set("Cookie", session.CookieHeader)
resp, err := http.DefaultClient.Do(req)
require.NoError(t, err)
defer func() { _ = resp.Body.Close() }()
require.Equal(t, http.StatusOK, resp.StatusCode)
}
// twoStepLoginPage is a username-first flow: the password field is hidden until
// the "Next" button is clicked, mimicking Google/Microsoft-style logins.
const twoStepLoginPage = `<html><head><title>Login</title></head><body>
<form id="f" action="/login" method="post">
<input type="email" name="email" id="email">
<button type="button" id="next">Next</button>
<div id="pwwrap" style="display:none">
<input type="password" name="password" id="password">
<button type="submit" id="submit">Sign in</button>
</div>
</form>
<script>
document.getElementById('next').addEventListener('click', function () {
document.getElementById('pwwrap').style.display = 'block';
});
</script></body></html>`
func TestLoginHeadless_MultiStep(t *testing.T) {
requireChrome(t)
app := newJSLoginApp()
srv := httptest.NewServer(func() http.Handler {
mux := http.NewServeMux()
mux.HandleFunc("/login", func(w http.ResponseWriter, r *http.Request) {
if r.Method == http.MethodGet {
w.Header().Set("Content-Type", "text/html")
_, _ = fmt.Fprint(w, twoStepLoginPage)
return
}
_ = r.ParseForm()
if r.PostFormValue("email") != "[email protected]" || r.PostFormValue("password") != "p@ss" {
w.Header().Set("Content-Type", "text/html")
_, _ = fmt.Fprint(w, twoStepLoginPage)
return
}
http.SetCookie(w, &http.Cookie{Name: "session", Value: "sess-dave", Path: "/"})
app.mu.Lock()
app.sessions["sess-dave"] = true
app.mu.Unlock()
http.Redirect(w, r, "/dashboard", http.StatusFound)
})
mux.HandleFunc("/dashboard", app.handleDashboard)
return mux
}())
defer srv.Close()
ctx, cancel := context.WithTimeout(context.Background(), 60*time.Second)
defer cancel()
session, err := LoginHeadless(ctx, Config{
LoginURL: srv.URL + "/login",
Username: "[email protected]",
Password: "p@ss",
SettleTime: 1 * time.Second,
Steps: []LoginStep{
{Action: "fill", Selector: "#email", Value: "{{username}}"},
{Action: "click", Selector: "#next"},
{Action: "waitvisible", Selector: "#password"},
{Action: "fill", Selector: "#password", Value: "{{password}}"},
{Action: "click", Selector: "#submit"},
},
})
require.NoError(t, err)
names := map[string]string{}
for _, c := range session.Cookies {
names[c.Name] = c.Value
}
require.Contains(t, names, "session", "multi-step login should capture the session cookie")
}
// spaTokenPage is a pure-SPA login: on submit JS validates the credentials
// client-side, stores a JWT in localStorage (no cookie at all) and swaps in a
// logged-in view. This is the canonical "token lives in web storage" case that
// motivates storage capture/replay.
const spaTokenPage = `<html><head><title>SPA</title></head><body>
<form id="f">
<input type="email" name="email" id="email">
<input type="password" name="password" id="password">
<button type="submit" id="submit">Sign in</button>
</form>
<script>
document.getElementById('f').addEventListener('submit', function (e) {
e.preventDefault();
var email = document.getElementById('email').value;
var pw = document.getElementById('password').value;
if (email === 'dave@example.com' && pw === 'p@ss') {
window.localStorage.setItem('auth_token', 'eyJhbGciOiJI.payload.sig');
window.sessionStorage.setItem('csrf', 'abc123');
document.body.innerHTML = '<h1>welcome dave</h1>';
} else {
document.body.innerHTML += '<p>bad creds</p>';
}
});
</script></body></html>`
func TestLoginHeadless_LocalStorageToken(t *testing.T) {
requireChrome(t)
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "text/html")
_, _ = fmt.Fprint(w, spaTokenPage)
}))
defer srv.Close()
ctx, cancel := context.WithTimeout(context.Background(), 60*time.Second)
defer cancel()
session, err := LoginHeadless(ctx, Config{
LoginURL: srv.URL + "/login",
Username: "[email protected]",
Password: "p@ss",
TokenRegex: `(eyJ[A-Za-z0-9._-]+)`,
SettleTime: 1500 * time.Millisecond,
})
require.NoError(t, err)
require.Empty(t, session.Cookies, "SPA login sets no cookie")
require.Equal(t, "eyJhbGciOiJI.payload.sig", session.Token, "token must be extracted from localStorage")
require.Equal(t, "eyJhbGciOiJI.payload.sig", session.LocalStorage["auth_token"], "localStorage must be captured")
require.Equal(t, "abc123", session.SessionStorage["csrf"], "sessionStorage must be captured")
}
func TestLoginHeadless_WrongPasswordFails(t *testing.T) {
requireChrome(t)
app := newJSLoginApp()
srv := app.server()
defer srv.Close()
ctx, cancel := context.WithTimeout(context.Background(), 60*time.Second)
defer cancel()
_, err := LoginHeadless(ctx, Config{
LoginURL: srv.URL + "/login",
Username: "[email protected]",
Password: "wrong",
SettleTime: 1500 * time.Millisecond,
})
require.ErrorIs(t, err, ErrLoginFailed)
}
// disabledSubmitPage mimics a framework-validated form (e.g. Angular/Juice
// Shop): the submit button starts disabled and is only enabled once both fields
// have received input events. A naive click-immediately submitter hangs on the
// disabled button; the engine must settle (so the field input enables the
// button) and wait for the button to become interactable before clicking.
const disabledSubmitPage = `<html><head><title>Login</title></head><body>
<form id="f" action="/login" method="post">
<input type="email" name="email" id="email">
<input type="password" name="password" id="password">
<button type="submit" id="submit" disabled>Sign in</button>
</form>
<script>
var email = document.getElementById('email');
var pw = document.getElementById('password');
var btn = document.getElementById('submit');
function check() { btn.disabled = !(email.value && pw.value); }
email.addEventListener('input', check);
pw.addEventListener('input', check);
</script></body></html>`
func TestLoginHeadless_SubmitDisabledUntilValid(t *testing.T) {
requireChrome(t)
app := newJSLoginApp()
srv := httptest.NewServer(func() http.Handler {
mux := http.NewServeMux()
mux.HandleFunc("/login", func(w http.ResponseWriter, r *http.Request) {
if r.Method == http.MethodGet {
w.Header().Set("Content-Type", "text/html")
_, _ = fmt.Fprint(w, disabledSubmitPage)
return
}
_ = r.ParseForm()
if r.PostFormValue("email") != "[email protected]" || r.PostFormValue("password") != "p@ss" {
w.Header().Set("Content-Type", "text/html")
_, _ = fmt.Fprint(w, disabledSubmitPage)
return
}
http.SetCookie(w, &http.Cookie{Name: "session", Value: "sess-dave", Path: "/"})
app.mu.Lock()
app.sessions["sess-dave"] = true
app.mu.Unlock()
http.Redirect(w, r, "/dashboard", http.StatusFound)
})
mux.HandleFunc("/dashboard", app.handleDashboard)
return mux
}())
defer srv.Close()
ctx, cancel := context.WithTimeout(context.Background(), 60*time.Second)
defer cancel()
session, err := LoginHeadless(ctx, Config{
LoginURL: srv.URL + "/login",
Username: "[email protected]",
Password: "p@ss",
SettleTime: 1500 * time.Millisecond,
})
require.NoError(t, err, "engine must wait for the submit button to become interactable")
names := map[string]string{}
for _, c := range session.Cookies {
names[c.Name] = c.Value
}
require.Contains(t, names, "session", "login should succeed once the button is enabled")
}
// enterSubmitPage has no clickable submit control and submits only when Enter is
// pressed in the password field (common in minimal SPA forms). It exercises the
// engine's Enter-to-submit fallback when no submit button is found.
const enterSubmitPage = `<html><head><title>Login</title></head><body>
<form id="f" action="/login" method="post">
<input type="email" name="email" id="email">
<input type="password" name="password" id="password">
</form>
<script>
document.getElementById('password').addEventListener('keydown', function (e) {
if (e.key === 'Enter') { document.getElementById('f').submit(); }
});
</script></body></html>`
func TestLoginHeadless_EnterSubmitFallback(t *testing.T) {
requireChrome(t)
app := newJSLoginApp()
srv := httptest.NewServer(func() http.Handler {
mux := http.NewServeMux()
mux.HandleFunc("/login", func(w http.ResponseWriter, r *http.Request) {
if r.Method == http.MethodGet {
w.Header().Set("Content-Type", "text/html")
_, _ = fmt.Fprint(w, enterSubmitPage)
return
}
_ = r.ParseForm()
if r.PostFormValue("email") != "[email protected]" || r.PostFormValue("password") != "p@ss" {
w.Header().Set("Content-Type", "text/html")
_, _ = fmt.Fprint(w, enterSubmitPage)
return
}
http.SetCookie(w, &http.Cookie{Name: "session", Value: "sess-dave", Path: "/"})
app.mu.Lock()
app.sessions["sess-dave"] = true
app.mu.Unlock()
http.Redirect(w, r, "/dashboard", http.StatusFound)
})
mux.HandleFunc("/dashboard", app.handleDashboard)
return mux
}())
defer srv.Close()
ctx, cancel := context.WithTimeout(context.Background(), 60*time.Second)
defer cancel()
session, err := LoginHeadless(ctx, Config{
LoginURL: srv.URL + "/login",
Username: "[email protected]",
Password: "p@ss",
SettleTime: 1500 * time.Millisecond,
})
require.NoError(t, err, "engine must fall back to Enter-to-submit when no submit button exists")
names := map[string]string{}
for _, c := range session.Cookies {
names[c.Name] = c.Value
}
require.Contains(t, names, "session", "Enter-to-submit fallback should log in")
}
func TestDetectStorageJWT(t *testing.T) {
const jwt = "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiJkYXZlIn0.SflKxwRJSMeKKF2QT4fwpMeJf36POk6yJV_adQssw5c"
t.Run("preferred key wins over other JWT-shaped values", func(t *testing.T) {
other := "eyJraWQiOiJvdGhlciJ9.eyJub3RpdCI6dHJ1ZX0.zzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzz"
got := detectStorageJWT(map[string]string{"random_blob": other, "token": jwt})
require.Equal(t, jwt, got, "the value under a preferred key should be chosen")
})
t.Run("extracts JWT embedded in JSON", func(t *testing.T) {
got := detectStorageJWT(map[string]string{"auth": `{"access_token":"` + jwt + `","exp":123}`})
require.Equal(t, jwt, got)
})
t.Run("extracts JWT wrapped in quotes", func(t *testing.T) {
got := detectStorageJWT(map[string]string{"id_token": `"` + jwt + `"`})
require.Equal(t, jwt, got)
})
t.Run("falls back to sessionStorage when localStorage has none", func(t *testing.T) {
got := detectStorageJWT(map[string]string{"theme": "dark"}, map[string]string{"jwt": jwt})
require.Equal(t, jwt, got)
})
t.Run("returns empty when no JWT present", func(t *testing.T) {
require.Empty(t, detectStorageJWT(map[string]string{"theme": "dark", "lang": "en"}))
require.Empty(t, detectStorageJWT(nil, nil))
})
t.Run("ignores non-JWT values that merely start with eyJ", func(t *testing.T) {
require.Empty(t, detectStorageJWT(map[string]string{"x": "eyJustabase64ishstring"}))
})
}
// spaAutoTokenPage stores a realistic JWT in localStorage under a conventional
// key and sets no cookie, mirroring a token-in-web-storage SPA. It is used to
// verify that the engine surfaces the token even without an explicit TokenRegex.
const spaAutoTokenPage = `<html><head><title>SPA</title></head><body>
<form id="f">
<input type="email" name="email" id="email">
<input type="password" name="password" id="password">
<button type="submit" id="submit">Sign in</button>
</form>
<script>
document.getElementById('f').addEventListener('submit', function (e) {
e.preventDefault();
var email = document.getElementById('email').value;
var pw = document.getElementById('password').value;
if (email === 'dave@example.com' && pw === 'p@ss') {
window.localStorage.setItem('token', 'eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiJkYXZlIn0.SflKxwRJSMeKKF2QT4fwpMeJf36POk6yJV_adQssw5c');
document.body.innerHTML = '<h1>welcome dave</h1>';
} else {
document.body.innerHTML += '<p>bad creds</p>';
}
});
</script></body></html>`
func TestLoginHeadless_AutoDetectStorageJWTWithoutRegex(t *testing.T) {
requireChrome(t)
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "text/html")
_, _ = fmt.Fprint(w, spaAutoTokenPage)
}))
defer srv.Close()
ctx, cancel := context.WithTimeout(context.Background(), 60*time.Second)
defer cancel()
session, err := LoginHeadless(ctx, Config{
LoginURL: srv.URL + "/login",
Username: "[email protected]",
Password: "p@ss",
SettleTime: 1500 * time.Millisecond,
})
require.NoError(t, err)
require.Empty(t, session.Cookies, "SPA login sets no cookie")
require.Equal(t,
"eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiJkYXZlIn0.SflKxwRJSMeKKF2QT4fwpMeJf36POk6yJV_adQssw5c",
session.Token,
"JWT in localStorage must be auto-detected as the session token without a TokenRegex")
}
-501
View File
@@ -1,501 +0,0 @@
package autologin
import (
"context"
"io"
"net"
"net/http"
"net/http/cookiejar"
"net/url"
"regexp"
"sort"
"strings"
"time"
"github.com/projectdiscovery/gologger"
"github.com/projectdiscovery/utils/errkit"
"golang.org/x/net/proxy"
)
// maxBodyRead caps how much of the login/landing page we read into memory for
// form detection and success checks.
const maxBodyRead = 4 << 20 // 4MB
var (
// ErrLoginFailed is returned when the login flow completed but produced no
// usable session (no cookies, no token) — typically wrong credentials.
ErrLoginFailed = errkit.New("auto-login produced no session (check credentials)")
)
// Config describes a single auto-login attempt.
type Config struct {
// LoginURL is the page that serves the login form (required).
LoginURL string
// Username / Password are the credentials to submit. Username may be empty
// for password-only forms.
Username string
Password string
// The following are optional overrides for cases where auto-detection would
// guess wrong.
// UsernameField forces the form field name that receives the username.
UsernameField string
// PasswordField forces the form field name that receives the password.
PasswordField string
// ExtraFields are additional form values submitted verbatim; they override
// any detected field of the same name (e.g. a fixed TOTP or tenant id).
ExtraFields map[string]string
// AllowCrossOriginAction lets the HTTP engine submit the credentials to a
// form action on a different origin than LoginURL (e.g. an SSO provider).
// By default such a form is refused: the login page belongs to a scan
// target and could send the credentials anywhere.
AllowCrossOriginAction bool
// TokenRegex, when set, is applied to the final response body (and, for the
// HTTP engine, response headers) to extract a bearer/session token (first
// capture group). This enables token-based logins that return the token in
// the response body rather than a cookie. For headless logins it is also
// matched against window.localStorage.
TokenRegex string
// Timeout bounds the whole login flow. Defaults to 30s when zero.
Timeout time.Duration
// The following fields apply only to the headless engine (LoginHeadless).
// Headless drives a real browser (go-rod) instead of the HTTP engine,
// enabling JS-rendered / SPA / multi-step login pages.
Headless bool
// ShowBrowser runs the browser headful (useful for debugging).
ShowBrowser bool
// UseInstalledChrome forces using a system-installed Chrome binary.
UseInstalledChrome bool
// Proxy routes browser traffic through the given proxy URL.
Proxy string
// CDPEndpoint connects to an existing browser over the Chrome DevTools
// Protocol instead of launching one.
CDPEndpoint string
// UserAgent overrides the browser's User-Agent for the login navigation.
UserAgent string
// CustomHeaders are extra headers sent with the headless login requests
// (e.g. global -H headers from the scan).
CustomHeaders map[string]string
// SettleTime is how long to wait for the post-submit navigation / SPA to
// settle before capturing the session. Defaults to 5s.
SettleTime time.Duration
// Steps, when non-empty, drives an explicit multi-step headless login
// (e.g. username-first / SSO consent flows) instead of the single-shot
// auto-detect-and-submit path. Ignored by the HTTP engine.
Steps []LoginStep
}
// LoginStep is a single action in an explicit multi-step headless login flow.
// In Value, the placeholders {{username}} and {{password}} are substituted with
// the configured credentials so secrets stay out of the step list.
type LoginStep struct {
// Action is one of: navigate, fill, click, waitvisible, wait, press, submit.
Action string `json:"action" yaml:"action"`
// Selector is a CSS selector (or, for fill/press, an input name) the action
// targets. Unused by navigate/wait.
Selector string `json:"selector" yaml:"selector"`
// Value is the action argument: a URL (navigate), text to type (fill), a key
// name such as "enter"/"tab" (press) or a duration like "2s" (wait).
Value string `json:"value" yaml:"value"`
}
// Session is the captured result of a successful auto-login.
type Session struct {
// Cookies are the session cookies captured after login (name/value only).
Cookies []*http.Cookie
// CookieHeader is the cookies rendered as a single "k=v; k2=v2" header value.
CookieHeader string
// Token is a token extracted via Config.TokenRegex, if any.
Token string
// LocalStorage / SessionStorage hold the browser storage captured after a
// headless login (always empty for the HTTP engine). They let token-based
// SPAs surface their session even when it lives in web storage.
LocalStorage map[string]string
SessionStorage map[string]string
// FinalURL / FinalStatus describe where the login flow landed.
FinalURL string
FinalStatus int
}
// Login performs a form-based auto-login: it fetches the login page, detects the
// login form, submits the supplied credentials (carrying any hidden CSRF tokens
// and the cookie jar), follows redirects and captures the resulting session.
//
// base is an optional template client whose Transport is reused (for proxy/TLS
// settings); its cookie jar and redirect policy are intentionally NOT reused so
// the login runs with a clean, dedicated jar. Pass nil to use defaults.
func Login(ctx context.Context, base *http.Client, cfg Config) (*Session, error) {
if strings.TrimSpace(cfg.LoginURL) == "" {
return nil, errkit.New("auto-login: login-url is required")
}
loginURL, err := url.Parse(cfg.LoginURL)
if err != nil {
return nil, errkit.Wrap(err, "auto-login: invalid login-url")
}
var tokenRe *regexp.Regexp
if cfg.TokenRegex != "" {
tokenRe, err = regexp.Compile(cfg.TokenRegex)
if err != nil {
return nil, errkit.Wrap(err, "auto-login: invalid token-regex")
}
}
jar, err := cookiejar.New(nil)
if err != nil {
return nil, errkit.Wrap(err, "auto-login: failed to create cookie jar")
}
timeout := cfg.Timeout
if timeout == 0 {
timeout = 30 * time.Second
}
client := &http.Client{Jar: jar, Timeout: timeout, CheckRedirect: checkLoginRedirect(cfg.AllowCrossOriginAction)}
switch {
case base != nil && base.Transport != nil:
// Reuse the template client's transport (proxy/TLS) when provided.
client.Transport = base.Transport
case cfg.Proxy != "":
// Otherwise honor an explicitly configured proxy so a proxied scan does
// not bypass the proxy during HTTP auto-login.
tr, perr := proxyTransport(cfg.Proxy)
if perr != nil {
return nil, perr
}
client.Transport = tr
}
// 1. Fetch the login page (this also seeds the jar with any pre-login CSRF
// cookie the server sets).
pageBody, err := doRequest(ctx, client, http.MethodGet, loginURL.String(), "", "")
if err != nil {
return nil, errkit.Wrap(err, "auto-login: failed to fetch login page")
}
// 2. Detect the login form.
form, err := DetectLoginForm(pageBody, loginURL)
if err != nil {
return nil, err
}
if actionURL, perr := url.Parse(form.Action); perr == nil && !sameOrigin(loginURL, actionURL) {
if !cfg.AllowCrossOriginAction {
return nil, errkit.Newf("auto-login: login form at %s submits to a different origin (%s); set allow-cross-origin-action to send the credentials there", loginURL.Redacted(), actionURL.Redacted())
}
gologger.Warning().Msgf("auto-login: login form at %s submits the credentials to a different origin (%s)", loginURL.Redacted(), actionURL.Redacted())
}
// 3. Build the submission values: detected hidden/default fields first, then
// the credentials, then caller overrides.
values := url.Values{}
for k, v := range form.Fields {
values.Set(k, v)
}
userField := form.UsernameField
if cfg.UsernameField != "" {
userField = cfg.UsernameField
}
passField := form.PasswordField
if cfg.PasswordField != "" {
passField = cfg.PasswordField
}
if userField != "" {
values.Set(userField, cfg.Username)
}
values.Set(passField, cfg.Password)
for k, v := range cfg.ExtraFields {
values.Set(k, v)
}
// 4. Submit the form.
var finalBody, finalURLStr string
var finalStatus int
var finalHeaders http.Header
if form.Method == http.MethodGet {
// A GET login places the password in the query string, where it can leak
// into server/proxy logs and browser history. We still honor it (some
// apps genuinely use GET), but warn so the operator is aware.
if cfg.Password != "" {
gologger.Warning().Msgf("auto-login: login form at %s uses GET; credentials will be sent in the URL query string", form.Action)
}
actionURL, perr := url.Parse(form.Action)
if perr != nil {
return nil, errkit.Wrap(perr, "auto-login: invalid form action")
}
q := actionURL.Query()
for k, vs := range values {
for _, v := range vs {
q.Set(k, v)
}
}
actionURL.RawQuery = q.Encode()
finalBody, finalURLStr, finalStatus, finalHeaders, err = doRequestFull(ctx, client, http.MethodGet, actionURL.String(), "", "")
} else {
finalBody, finalURLStr, finalStatus, finalHeaders, err = doRequestFull(ctx, client, form.Method, form.Action, "application/x-www-form-urlencoded", values.Encode())
}
if err != nil {
return nil, errkit.Wrap(err, "auto-login: failed to submit login form")
}
// 5. Capture the session from the jar across the login/action/final hosts.
session := &Session{FinalURL: finalURLStr, FinalStatus: finalStatus}
session.Cookies = collectCookies(jar, loginURL, form.Action, finalURLStr)
session.CookieHeader = renderCookieHeader(session.Cookies)
// Token extraction: the configured regex against the body first, then the
// response headers (regex + well-known auth headers). Many token-based APIs
// return the session token in a header (Authorization, X-Auth-Token, ...)
// rather than the body or a cookie.
if tokenRe != nil {
if m := tokenRe.FindStringSubmatch(finalBody); len(m) > 1 {
session.Token = m[1]
}
}
if session.Token == "" {
session.Token = detectHeaderToken(finalHeaders, tokenRe)
}
// 6. Decide success.
//
// Auth failure statuses with a pre-login CSRF cookie must not count as a
// successful session — reject 401/403 before treating cookies as material.
if finalStatus == http.StatusUnauthorized || finalStatus == http.StatusForbidden {
return nil, errkit.Wrapf(ErrLoginFailed, "final status %d at %s", finalStatus, finalURLStr)
}
// "Captured a cookie" is NOT a sufficient signal: servers commonly set a
// pre-login CSRF cookie on the GET that survives a failed POST. The strongest
// generic failure signal is being re-prompted — i.e. the page we land on
// after submitting still presents a login form (a password field). When that
// happens and we did not extract a token, the login failed (e.g. wrong
// credentials). NOTE: this can misfire on apps that render a login form on
// every page even when authenticated; such cases can use TokenRegex or field
// overrides to disambiguate.
if session.Token == "" {
var finalURLParsed *url.URL
if u, perr := url.Parse(finalURLStr); perr == nil {
finalURLParsed = u
}
if _, derr := DetectLoginForm(finalBody, finalURLParsed); derr == nil {
return nil, errkit.Wrapf(ErrLoginFailed, "still presented a login form (final status %d at %s)", finalStatus, finalURLStr)
}
}
// A session must ultimately carry at least one cookie or an extracted token.
if len(session.Cookies) == 0 && session.Token == "" {
return nil, errkit.Wrapf(ErrLoginFailed, "final status %d at %s", finalStatus, finalURLStr)
}
return session, nil
}
// proxyTransport builds an http.Transport that routes through the given proxy.
// It supports http/https proxies (via CONNECT) and socks5/socks5h proxies.
func proxyTransport(rawProxy string) (*http.Transport, error) {
u, err := url.Parse(rawProxy)
if err != nil {
return nil, errkit.Wrap(err, "auto-login: invalid proxy url")
}
switch strings.ToLower(u.Scheme) {
case "http", "https":
return &http.Transport{Proxy: http.ProxyURL(u)}, nil
case "socks5", "socks5h":
var auth *proxy.Auth
if u.User != nil {
password, _ := u.User.Password()
auth = &proxy.Auth{User: u.User.Username(), Password: password}
}
dialer, derr := proxy.SOCKS5("tcp", u.Host, auth, proxy.Direct)
if derr != nil {
return nil, errkit.Wrap(derr, "auto-login: invalid socks5 proxy")
}
tr := &http.Transport{}
if ctxDialer, ok := dialer.(proxy.ContextDialer); ok {
tr.DialContext = ctxDialer.DialContext
} else {
tr.DialContext = func(_ context.Context, network, addr string) (net.Conn, error) {
return dialer.Dial(network, addr)
}
}
return tr, nil
default:
return nil, errkit.Newf("auto-login: unsupported proxy scheme %q", u.Scheme)
}
}
// collectCookies gathers cookies the jar holds for the login/action/final URLs.
// Cookies with Domain/Path are de-duplicated by name+domain+path. Jar cookies
// that lack scope metadata are de-duplicated by name, with later URLs winning so
// a post-login session cookie replaces a same-name pre-login CSRF cookie.
func collectCookies(jar http.CookieJar, loginURL *url.URL, action, final string) []*http.Cookie {
urls := []*url.URL{loginURL}
if u, err := url.Parse(action); err == nil {
urls = append(urls, u)
}
if u, err := url.Parse(final); err == nil {
urls = append(urls, u)
}
seen := map[string]int{}
var out []*http.Cookie
for _, u := range urls {
if u == nil {
continue
}
for _, c := range jar.Cookies(u) {
key := cookieDedupeKey(c)
if idx, ok := seen[key]; ok {
out[idx] = c
continue
}
seen[key] = len(out)
out = append(out, c)
}
}
return out
}
func cookieDedupeKey(c *http.Cookie) string {
if c.Domain == "" && c.Path == "" {
return c.Name
}
return c.Name + "\x00" + c.Domain + "\x00" + c.Path
}
// renderCookieHeader renders cookies as a single Cookie header value.
func renderCookieHeader(cookies []*http.Cookie) string {
parts := make([]string, 0, len(cookies))
for _, c := range cookies {
parts = append(parts, c.Name+"="+c.Value)
}
return strings.Join(parts, "; ")
}
// checkLoginRedirect keeps the default limit of 10 redirects and refuses a
// 307/308 redirect that would replay the submitted form body (with the
// password) to another origin. Without it a login endpoint could bounce the
// credentials anywhere, which the form action origin check alone cannot stop.
func checkLoginRedirect(allowCrossOrigin bool) func(*http.Request, []*http.Request) error {
return func(req *http.Request, via []*http.Request) error {
if len(via) >= 10 {
return errkit.New("auto-login: stopped after 10 redirects")
}
if allowCrossOrigin || req.Method == http.MethodGet || req.Method == http.MethodHead {
return nil
}
if !sameOrigin(via[len(via)-1].URL, req.URL) {
return errkit.Newf("auto-login: refusing to replay the login form to a different origin (%s); set allow-cross-origin-action to send the credentials there", req.URL.Redacted())
}
return nil
}
}
// sameOrigin reports whether a and b share scheme, host and port, with the
// default port of the scheme applied when one is missing.
func sameOrigin(a, b *url.URL) bool {
if a == nil || b == nil {
return false
}
return strings.EqualFold(a.Scheme, b.Scheme) &&
strings.EqualFold(a.Hostname(), b.Hostname()) &&
originPort(a) == originPort(b)
}
func originPort(u *url.URL) string {
if port := u.Port(); port != "" {
return port
}
switch strings.ToLower(u.Scheme) {
case "https":
return "443"
case "http":
return "80"
}
return ""
}
// doRequest issues a request and returns only the body (used for the GET page).
func doRequest(ctx context.Context, client *http.Client, method, rawURL, contentType, body string) (string, error) {
b, _, _, _, err := doRequestFull(ctx, client, method, rawURL, contentType, body)
return b, err
}
// doRequestFull issues a request and returns the (capped) body, the final URL
// after redirects, the final status code and the final response headers.
func doRequestFull(ctx context.Context, client *http.Client, method, rawURL, contentType, body string) (string, string, int, http.Header, error) {
var reader io.Reader
if body != "" {
reader = strings.NewReader(body)
}
req, err := http.NewRequestWithContext(ctx, method, rawURL, reader)
if err != nil {
return "", "", 0, nil, err
}
if contentType != "" {
req.Header.Set("Content-Type", contentType)
}
resp, err := client.Do(req)
if err != nil {
return "", "", 0, nil, err
}
defer func() { _ = resp.Body.Close() }()
data, err := io.ReadAll(io.LimitReader(resp.Body, maxBodyRead))
if err != nil {
return "", "", 0, nil, err
}
finalURL := rawURL
if resp.Request != nil && resp.Request.URL != nil {
finalURL = resp.Request.URL.String()
}
return string(data), finalURL, resp.StatusCode, resp.Header, nil
}
// tokenHeaderNames are response headers commonly used to deliver a session /
// bearer token, checked in priority order.
var tokenHeaderNames = []string{
"Authorization",
"X-Auth-Token",
"X-Access-Token",
"Access-Token",
"X-Subject-Token",
"X-Session-Token",
"X-Api-Token",
}
// detectHeaderToken extracts a session/bearer token from response headers. A
// configured token-regex is tried first (so a custom header can be targeted),
// then the well-known token headers in priority order. A leading "Bearer "
// scheme is stripped from the value.
func detectHeaderToken(h http.Header, tokenRe *regexp.Regexp) string {
if len(h) == 0 {
return ""
}
if tokenRe != nil {
names := make([]string, 0, len(h))
for name := range h {
names = append(names, name)
}
sort.Strings(names)
for _, name := range names {
for _, v := range h[name] {
if m := tokenRe.FindStringSubmatch(v); len(m) > 1 {
return m[1]
}
}
}
}
for _, name := range tokenHeaderNames {
v := strings.TrimSpace(h.Get(name))
if v == "" {
continue
}
// Strip a leading auth scheme (e.g. "Bearer <token>").
if i := strings.IndexByte(v, ' '); i > 0 && strings.EqualFold(v[:i], "bearer") {
v = strings.TrimSpace(v[i+1:])
}
if v != "" {
return v
}
}
return ""
}
-454
View File
@@ -1,454 +0,0 @@
package autologin
import (
"context"
"fmt"
"net/http"
"net/http/httptest"
"net/url"
"regexp"
"sync"
"sync/atomic"
"testing"
"github.com/stretchr/testify/require"
)
// loginApp is a hermetic web app with a realistic, CSRF-protected, cookie-based
// login flow used to exercise the auto-login engine end to end:
// - GET /login sets a per-visitor csrf cookie and renders a form echoing it in
// a hidden input.
// - POST /login validates the csrf (hidden field must equal the cookie),
// checks the credentials, and on success sets a `session` cookie and 302s to
// /dashboard.
// - GET /dashboard returns 200 only when a valid session cookie is presented.
type loginApp struct {
mu sync.Mutex
sessions map[string]bool // valid session tokens
user string
pass string
}
func newLoginApp(user, pass string) *loginApp {
return &loginApp{sessions: map[string]bool{}, user: user, pass: pass}
}
func (a *loginApp) server() *httptest.Server {
mux := http.NewServeMux()
mux.HandleFunc("/login", a.handleLogin)
mux.HandleFunc("/dashboard", a.handleDashboard)
return httptest.NewServer(mux)
}
func (a *loginApp) handleLogin(w http.ResponseWriter, r *http.Request) {
if r.Method == http.MethodGet {
// Seed a CSRF cookie and echo it into the form.
csrf := "csrf-" + fmt.Sprint(len(a.sessions)) + "-tok"
http.SetCookie(w, &http.Cookie{Name: "csrftoken", Value: csrf, Path: "/"})
w.Header().Set("Content-Type", "text/html")
_, _ = fmt.Fprintf(w, `<html><body>
<form action="/login" method="post">
<input type="hidden" name="csrf" value="%s">
<input type="text" name="email" autocomplete="username">
<input type="password" name="password">
<input type="submit" name="commit" value="Sign in">
</form>
</body></html>`, csrf)
return
}
// POST: validate CSRF against the cookie, then credentials.
_ = r.ParseForm()
csrfCookie, err := r.Cookie("csrftoken")
if err != nil || r.PostFormValue("csrf") != csrfCookie.Value {
http.Error(w, "csrf mismatch", http.StatusForbidden)
return
}
if r.PostFormValue("email") != a.user || r.PostFormValue("password") != a.pass {
// Re-render the login form on failure (no session set).
w.Header().Set("Content-Type", "text/html")
w.WriteHeader(http.StatusOK)
_, _ = fmt.Fprint(w, `<html><body><p>Invalid credentials</p>
<form action="/login" method="post">
<input type="text" name="email">
<input type="password" name="password">
</form></body></html>`)
return
}
token := "sess-" + r.PostFormValue("email")
a.mu.Lock()
a.sessions[token] = true
a.mu.Unlock()
http.SetCookie(w, &http.Cookie{Name: "session", Value: token, Path: "/"})
http.Redirect(w, r, "/dashboard", http.StatusFound)
}
func (a *loginApp) handleDashboard(w http.ResponseWriter, r *http.Request) {
c, err := r.Cookie("session")
if err != nil {
http.Error(w, "unauthorized", http.StatusUnauthorized)
return
}
a.mu.Lock()
ok := a.sessions[c.Value]
a.mu.Unlock()
if !ok {
http.Error(w, "unauthorized", http.StatusUnauthorized)
return
}
_, _ = fmt.Fprint(w, "welcome to your dashboard")
}
func TestLogin_CSRFFormAndSessionCookie(t *testing.T) {
app := newLoginApp("[email protected]", "s3cr3t")
srv := app.server()
defer srv.Close()
session, err := Login(context.Background(), nil, Config{
LoginURL: srv.URL + "/login",
Username: "[email protected]",
Password: "s3cr3t",
})
require.NoError(t, err)
require.NotEmpty(t, session.Cookies, "should capture session cookie")
// The captured session must actually authenticate against a protected route.
names := map[string]string{}
for _, c := range session.Cookies {
names[c.Name] = c.Value
}
require.Contains(t, names, "session", "session cookie must be captured")
req, _ := http.NewRequest(http.MethodGet, srv.URL+"/dashboard", nil)
req.Header.Set("Cookie", session.CookieHeader)
resp, err := http.DefaultClient.Do(req)
require.NoError(t, err)
defer func() { _ = resp.Body.Close() }()
require.Equal(t, http.StatusOK, resp.StatusCode, "captured session should grant access to /dashboard")
}
func TestLogin_WrongPasswordFails(t *testing.T) {
app := newLoginApp("[email protected]", "s3cr3t")
srv := app.server()
defer srv.Close()
_, err := Login(context.Background(), nil, Config{
LoginURL: srv.URL + "/login",
Username: "[email protected]",
Password: "wrong-password",
})
require.ErrorIs(t, err, ErrLoginFailed)
}
func TestLogin_FieldOverride(t *testing.T) {
// Force the username field name via override; detection would pick "email"
// anyway, but this verifies the override path submits correctly.
app := newLoginApp("[email protected]", "hunter2")
srv := app.server()
defer srv.Close()
session, err := Login(context.Background(), nil, Config{
LoginURL: srv.URL + "/login",
Username: "[email protected]",
Password: "hunter2",
UsernameField: "email",
})
require.NoError(t, err)
require.NotEmpty(t, session.Cookies)
}
// tokenApp returns a JSON token in the login response body instead of setting a
// cookie, to exercise TokenRegex-based capture.
func tokenApp() *httptest.Server {
mux := http.NewServeMux()
mux.HandleFunc("/login", func(w http.ResponseWriter, r *http.Request) {
if r.Method == http.MethodGet {
w.Header().Set("Content-Type", "text/html")
_, _ = fmt.Fprint(w, `<form action="/api/login" method="post">
<input type="text" name="user">
<input type="password" name="password">
</form>`)
return
}
})
mux.HandleFunc("/api/login", func(w http.ResponseWriter, r *http.Request) {
_ = r.ParseForm()
if r.PostFormValue("user") == "carol" && r.PostFormValue("password") == "pw" {
w.Header().Set("Content-Type", "application/json")
_, _ = fmt.Fprint(w, `{"access_token":"eyJhbGciOiJI.payload.sig","expires_in":3600}`)
return
}
http.Error(w, "bad creds", http.StatusUnauthorized)
})
return httptest.NewServer(mux)
}
func TestLogin_TokenExtraction(t *testing.T) {
srv := tokenApp()
defer srv.Close()
session, err := Login(context.Background(), nil, Config{
LoginURL: srv.URL + "/login",
Username: "carol",
Password: "pw",
TokenRegex: `"access_token":"([^"]+)"`,
})
require.NoError(t, err)
require.Equal(t, "eyJhbGciOiJI.payload.sig", session.Token)
}
// getFormApp serves a GET-method login form (credentials travel in the query
// string). It exercises the GET submission branch of Login.
func getFormApp(user, pass string) *httptest.Server {
mux := http.NewServeMux()
mux.HandleFunc("/login", func(w http.ResponseWriter, r *http.Request) {
q := r.URL.Query()
// A submit is any GET carrying the password param.
if q.Get("password") != "" {
if q.Get("user") == user && q.Get("password") == pass && q.Get("realm") == "corp" {
http.SetCookie(w, &http.Cookie{Name: "session", Value: "ok", Path: "/"})
w.Header().Set("Content-Type", "text/html")
_, _ = fmt.Fprint(w, `<html><body>welcome, no form here</body></html>`)
return
}
w.Header().Set("Content-Type", "text/html")
_, _ = fmt.Fprint(w, `<html><body><form method="get" action="/login">
<input type="text" name="user"><input type="password" name="password"></form></body></html>`)
return
}
w.Header().Set("Content-Type", "text/html")
_, _ = fmt.Fprint(w, `<html><body><form method="get" action="/login">
<input type="hidden" name="realm" value="corp">
<input type="text" name="user" autocomplete="username">
<input type="password" name="password">
</form></body></html>`)
})
return httptest.NewServer(mux)
}
func TestLogin_GetMethodForm(t *testing.T) {
srv := getFormApp("dave", "pw123")
defer srv.Close()
session, err := Login(context.Background(), nil, Config{
LoginURL: srv.URL + "/login",
Username: "dave",
Password: "pw123",
})
require.NoError(t, err)
require.NotEmpty(t, session.Cookies, "GET-form login should still capture the session cookie")
// The hidden 'realm' default must have been carried into the query string.
require.Contains(t, session.FinalURL, "realm=corp")
}
// extraFieldApp requires an additional non-form field (e.g. a tenant id) that is
// not present in the markup, so it must be supplied via Config.ExtraFields.
func extraFieldApp(user, pass string) *httptest.Server {
mux := http.NewServeMux()
mux.HandleFunc("/login", func(w http.ResponseWriter, r *http.Request) {
if r.Method == http.MethodGet {
w.Header().Set("Content-Type", "text/html")
_, _ = fmt.Fprint(w, `<form action="/login" method="post">
<input type="text" name="user"><input type="password" name="password"></form>`)
return
}
_ = r.ParseForm()
if r.PostFormValue("user") == user && r.PostFormValue("password") == pass && r.PostFormValue("tenant") == "acme" {
http.SetCookie(w, &http.Cookie{Name: "session", Value: "ok", Path: "/"})
_, _ = fmt.Fprint(w, "<html><body>signed in</body></html>")
return
}
w.Header().Set("Content-Type", "text/html")
_, _ = fmt.Fprint(w, `<form action="/login" method="post"><input type="password" name="password"></form>`)
})
return httptest.NewServer(mux)
}
func TestLogin_ExtraFields(t *testing.T) {
srv := extraFieldApp("erin", "pw")
defer srv.Close()
// Without the extra field the login must fail (re-prompted).
_, err := Login(context.Background(), nil, Config{
LoginURL: srv.URL + "/login", Username: "erin", Password: "pw",
})
require.ErrorIs(t, err, ErrLoginFailed)
// Supplying the tenant via ExtraFields must succeed.
session, err := Login(context.Background(), nil, Config{
LoginURL: srv.URL + "/login",
Username: "erin",
Password: "pw",
ExtraFields: map[string]string{"tenant": "acme"},
})
require.NoError(t, err)
require.NotEmpty(t, session.Cookies)
}
func TestProxyTransport(t *testing.T) {
tr, err := proxyTransport("http://127.0.0.1:8080")
require.NoError(t, err)
require.NotNil(t, tr.Proxy, "http proxy must set Transport.Proxy")
tr, err = proxyTransport("socks5://127.0.0.1:1080")
require.NoError(t, err)
require.NotNil(t, tr.DialContext, "socks5 proxy must set Transport.DialContext")
_, err = proxyTransport("ftp://127.0.0.1:21")
require.Error(t, err, "unsupported proxy scheme must error")
}
// TestLogin_HTTPProxy proves the HTTP engine actually routes through cfg.Proxy:
// the login target host is unresolvable, so the flow can only succeed if every
// request is sent to the proxy (which serves the login app by path).
func TestLogin_HTTPProxy(t *testing.T) {
var proxyHits int32
mux := http.NewServeMux()
mux.HandleFunc("/login", func(w http.ResponseWriter, r *http.Request) {
atomic.AddInt32(&proxyHits, 1)
if r.Method == http.MethodGet {
w.Header().Set("Content-Type", "text/html")
_, _ = fmt.Fprint(w, `<form action="/login" method="post">
<input type="text" name="user"><input type="password" name="password"></form>`)
return
}
_ = r.ParseForm()
if r.PostFormValue("user") == "frank" && r.PostFormValue("password") == "pw" {
http.SetCookie(w, &http.Cookie{Name: "session", Value: "ok", Path: "/"})
_, _ = fmt.Fprint(w, "<html><body>signed in</body></html>")
return
}
_, _ = fmt.Fprint(w, `<form action="/login" method="post"><input type="password" name="password"></form>`)
})
proxySrv := httptest.NewServer(mux)
defer proxySrv.Close()
session, err := Login(context.Background(), nil, Config{
// Unresolvable host: only reachable if the proxy is honored.
LoginURL: "http://login.invalid.test/login",
Username: "frank",
Password: "pw",
Proxy: proxySrv.URL,
})
require.NoError(t, err, "login should succeed via the proxy")
require.NotEmpty(t, session.Cookies)
require.GreaterOrEqual(t, atomic.LoadInt32(&proxyHits), int32(2), "both GET and POST must traverse the proxy")
}
func TestLogin_NoFormIsError(t *testing.T) {
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
_, _ = fmt.Fprint(w, "<html><body>no form here</body></html>")
}))
defer srv.Close()
_, err := Login(context.Background(), nil, Config{
LoginURL: srv.URL + "/login",
Username: "x",
Password: "y",
})
require.ErrorIs(t, err, ErrNoLoginForm)
}
func TestDetectHeaderToken(t *testing.T) {
t.Run("strips bearer scheme from Authorization", func(t *testing.T) {
h := http.Header{"Authorization": {"Bearer abc.def.ghi"}}
require.Equal(t, "abc.def.ghi", detectHeaderToken(h, nil))
})
t.Run("reads well-known token header", func(t *testing.T) {
h := http.Header{"X-Auth-Token": {"tok-123"}}
require.Equal(t, "tok-123", detectHeaderToken(h, nil))
})
t.Run("priority order prefers Authorization over X-Auth-Token", func(t *testing.T) {
h := http.Header{"Authorization": {"Bearer first"}, "X-Auth-Token": {"second"}}
require.Equal(t, "first", detectHeaderToken(h, nil))
})
t.Run("token-regex targets a custom header", func(t *testing.T) {
h := http.Header{"X-Custom": {"session=zzz999;"}}
re := regexp.MustCompile(`session=([^;]+)`)
require.Equal(t, "zzz999", detectHeaderToken(h, re))
})
t.Run("no token headers yields empty", func(t *testing.T) {
require.Empty(t, detectHeaderToken(http.Header{"Content-Type": {"text/html"}}, nil))
require.Empty(t, detectHeaderToken(nil, nil))
})
}
func TestLogin_CrossOriginActionRefused(t *testing.T) {
var collected atomic.Int32
collector := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
collected.Add(1)
http.SetCookie(w, &http.Cookie{Name: "session", Value: "stolen", Path: "/"})
w.WriteHeader(http.StatusOK)
}))
defer collector.Close()
page := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
w.Header().Set("Content-Type", "text/html")
_, _ = fmt.Fprintf(w, `<html><body>
<form action="%s/collect" method="post">
<input type="text" name="username">
<input type="password" name="password">
</form></body></html>`, collector.URL)
}))
defer page.Close()
cfg := Config{LoginURL: page.URL + "/login", Username: "admin", Password: "s3cret"}
_, err := Login(context.Background(), nil, cfg)
require.ErrorContains(t, err, "different origin")
require.Zero(t, collected.Load(), "credentials must not be sent to another origin by default")
cfg.AllowCrossOriginAction = true
session, err := Login(context.Background(), nil, cfg)
require.NoError(t, err)
require.NotNil(t, session)
require.Equal(t, int32(1), collected.Load())
}
// A same-origin form action that answers with a 307/308 would make the client
// replay the form body, password included, to the redirect target.
func TestLogin_CrossOriginRedirectReplayRefused(t *testing.T) {
var bodies atomic.Int32
collector := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.Method == http.MethodPost {
bodies.Add(1)
}
http.SetCookie(w, &http.Cookie{Name: "session", Value: "stolen", Path: "/"})
w.WriteHeader(http.StatusOK)
}))
defer collector.Close()
page := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.Method == http.MethodPost {
http.Redirect(w, r, collector.URL+"/collect", http.StatusTemporaryRedirect)
return
}
w.Header().Set("Content-Type", "text/html")
_, _ = fmt.Fprint(w, `<html><body>
<form action="/login" method="post">
<input type="text" name="username">
<input type="password" name="password">
</form></body></html>`)
}))
defer page.Close()
cfg := Config{LoginURL: page.URL + "/login", Username: "admin", Password: "s3cret"}
_, err := Login(context.Background(), nil, cfg)
require.ErrorContains(t, err, "different origin")
require.Zero(t, bodies.Load(), "the password must not be replayed to another origin")
cfg.AllowCrossOriginAction = true
_, err = Login(context.Background(), nil, cfg)
require.NoError(t, err)
require.Equal(t, int32(1), bodies.Load())
}
func TestSameOrigin(t *testing.T) {
parse := func(raw string) *url.URL {
u, err := url.Parse(raw)
require.NoError(t, err)
return u
}
require.True(t, sameOrigin(parse("https://Example.com/login"), parse("https://example.com:443/session")))
require.True(t, sameOrigin(parse("http://example.com:8080/a"), parse("http://example.com:8080/b")))
require.False(t, sameOrigin(parse("https://example.com/login"), parse("http://example.com/session")))
require.False(t, sameOrigin(parse("https://example.com/login"), parse("https://evil.example/collect")))
require.False(t, sameOrigin(parse("http://example.com/login"), parse("http://example.com:8080/session")))
}
@@ -1,401 +0,0 @@
package autologin
import (
"context"
"io"
"net/http"
"net/http/httptest"
"os"
"path/filepath"
"strings"
"testing"
"time"
"github.com/projectdiscovery/nuclei/v3/internal/fuzzplayground"
"github.com/stretchr/testify/require"
)
// playgroundServer starts the in-repo auth playground and returns its base URL.
func playgroundServer(t *testing.T) string {
t.Helper()
srv := httptest.NewServer(fuzzplayground.GetPlaygroundServer())
t.Cleanup(srv.Close)
return srv.URL
}
// assertSessionAuthenticates verifies a captured session actually authenticates
// against the playground's protected /auth/whoami endpoint, trying both the
// cookie and bearer-token representations.
func assertSessionAuthenticates(t *testing.T, base string, s *Session) {
t.Helper()
apply := func(req *http.Request) {
if s.CookieHeader != "" {
req.Header.Set("Cookie", s.CookieHeader)
}
if s.Token != "" {
req.Header.Set("Authorization", "Bearer "+s.Token)
}
}
req, _ := http.NewRequest(http.MethodGet, base+"/auth/whoami", nil)
apply(req)
resp, err := http.DefaultClient.Do(req)
require.NoError(t, err)
defer func() { _ = resp.Body.Close() }()
body, _ := io.ReadAll(resp.Body)
require.Equal(t, http.StatusOK, resp.StatusCode, "whoami should accept the captured session; body=%s", body)
require.Contains(t, string(body), fuzzplayground.AuthUsername, "whoami should identify the logged-in user")
}
func cookieNames(s *Session) map[string]string {
out := map[string]string{}
for _, c := range s.Cookies {
out[c.Name] = c.Value
}
return out
}
// --- HTTP engine styles ----------------------------------------------------
func TestPlayground_HTTPFormLogin(t *testing.T) {
base := playgroundServer(t)
session, err := Login(context.Background(), nil, Config{
LoginURL: base + "/auth/form-login",
Username: fuzzplayground.AuthUsername,
Password: fuzzplayground.AuthPassword,
})
require.NoError(t, err)
require.Contains(t, cookieNames(session), "PSESSION", "form login should set a session cookie")
assertSessionAuthenticates(t, base, session)
}
func TestPlayground_HTTPCSRFLogin(t *testing.T) {
base := playgroundServer(t)
session, err := Login(context.Background(), nil, Config{
LoginURL: base + "/auth/csrf-login",
Username: fuzzplayground.AuthUsername,
Password: fuzzplayground.AuthPassword,
})
require.NoError(t, err, "HTTP engine must capture and replay the hidden CSRF token")
require.Contains(t, cookieNames(session), "PSESSION")
assertSessionAuthenticates(t, base, session)
}
func TestPlayground_HTTPWrongPasswordFails(t *testing.T) {
base := playgroundServer(t)
_, err := Login(context.Background(), nil, Config{
LoginURL: base + "/auth/form-login",
Username: fuzzplayground.AuthUsername,
Password: "wrong",
})
require.ErrorIs(t, err, ErrLoginFailed)
}
// TestPlayground_HTTPHeaderTokenLogin proves the HTTP engine extracts a session
// token delivered in a response header (no cookie, no body token).
func TestPlayground_HTTPHeaderTokenLogin(t *testing.T) {
base := playgroundServer(t)
session, err := Login(context.Background(), nil, Config{
LoginURL: base + "/auth/header-token-login",
Username: fuzzplayground.AuthUsername,
Password: fuzzplayground.AuthPassword,
})
require.NoError(t, err, "the engine must extract the token from the response header")
require.Empty(t, session.Cookies, "header-token login sets no cookie")
require.NotEmpty(t, session.Token, "token must be read from the X-Auth-Token header")
assertSessionAuthenticates(t, base, session)
}
// --- Headless styles -------------------------------------------------------
func TestPlayground_HeadlessSPALogin(t *testing.T) {
requireChrome(t)
base := playgroundServer(t)
ctx, cancel := context.WithTimeout(context.Background(), 60*time.Second)
defer cancel()
// The static HTTP engine cannot log in here (the form is built by JS).
_, httpErr := Login(context.Background(), nil, Config{
LoginURL: base + "/auth/spa-login",
Username: fuzzplayground.AuthUsername,
Password: fuzzplayground.AuthPassword,
})
require.ErrorIs(t, httpErr, ErrNoLoginForm)
session, err := LoginHeadless(ctx, Config{
LoginURL: base + "/auth/spa-login",
Username: fuzzplayground.AuthUsername,
Password: fuzzplayground.AuthPassword,
SettleTime: 1500 * time.Millisecond,
})
require.NoError(t, err)
require.Contains(t, cookieNames(session), "PSESSION")
assertSessionAuthenticates(t, base, session)
}
func TestPlayground_HeadlessMultiStep(t *testing.T) {
requireChrome(t)
base := playgroundServer(t)
ctx, cancel := context.WithTimeout(context.Background(), 60*time.Second)
defer cancel()
session, err := LoginHeadless(ctx, Config{
LoginURL: base + "/auth/multistep-login",
Username: fuzzplayground.AuthUsername,
Password: fuzzplayground.AuthPassword,
SettleTime: 1 * time.Second,
Steps: []LoginStep{
{Action: "fill", Selector: "#username", Value: "{{username}}"},
{Action: "click", Selector: "#next"},
{Action: "waitvisible", Selector: "#password"},
{Action: "fill", Selector: "#password", Value: "{{password}}"},
{Action: "click", Selector: "#submit"},
},
})
require.NoError(t, err)
require.Contains(t, cookieNames(session), "PSESSION")
assertSessionAuthenticates(t, base, session)
}
// TestPlayground_HeadlessStrictSubmit is the regression for the disabled-until-
// valid submit button: the engine must settle (so JS enables the button) and/or
// fall back to Enter rather than hang on the disabled control.
func TestPlayground_HeadlessStrictSubmit(t *testing.T) {
requireChrome(t)
base := playgroundServer(t)
ctx, cancel := context.WithTimeout(context.Background(), 60*time.Second)
defer cancel()
session, err := LoginHeadless(ctx, Config{
LoginURL: base + "/auth/strict-login",
Username: fuzzplayground.AuthUsername,
Password: fuzzplayground.AuthPassword,
SettleTime: 1500 * time.Millisecond,
})
require.NoError(t, err)
require.Contains(t, cookieNames(session), "PSESSION")
assertSessionAuthenticates(t, base, session)
}
func TestPlayground_HeadlessSPAToken(t *testing.T) {
requireChrome(t)
base := playgroundServer(t)
ctx, cancel := context.WithTimeout(context.Background(), 60*time.Second)
defer cancel()
session, err := LoginHeadless(ctx, Config{
LoginURL: base + "/auth/spa-token-login",
Username: fuzzplayground.AuthUsername,
Password: fuzzplayground.AuthPassword,
TokenRegex: `(eyJ[A-Za-z0-9._-]+\.[A-Za-z0-9._-]+\.[A-Za-z0-9._-]+)`,
SettleTime: 1500 * time.Millisecond,
})
require.NoError(t, err)
require.Empty(t, session.Cookies, "SPA-token login sets no cookie")
require.NotEmpty(t, session.Token, "token must be extracted from localStorage")
require.NotEmpty(t, session.LocalStorage["access_token"], "localStorage must be captured")
// The captured bearer token must authenticate the protected API.
assertSessionAuthenticates(t, base, session)
}
// TestPlayground_HeadlessSPATokenAutoDetect proves the engine surfaces a JWT
// kept in web storage as the session token even when no token-regex is given
// (the JWT is stored under the conventional "access_token" key).
func TestPlayground_HeadlessSPATokenAutoDetect(t *testing.T) {
requireChrome(t)
base := playgroundServer(t)
ctx, cancel := context.WithTimeout(context.Background(), 60*time.Second)
defer cancel()
session, err := LoginHeadless(ctx, Config{
LoginURL: base + "/auth/spa-token-login",
Username: fuzzplayground.AuthUsername,
Password: fuzzplayground.AuthPassword,
SettleTime: 1500 * time.Millisecond,
// Intentionally no TokenRegex: rely on web-storage JWT auto-detection.
})
require.NoError(t, err)
require.NotEmpty(t, session.Token, "JWT in localStorage must be auto-detected without a token-regex")
require.NotEmpty(t, session.LocalStorage["access_token"], "localStorage must be captured")
assertSessionAuthenticates(t, base, session)
}
// TestPlayground_HTTPMultiFormPicksLogin proves the form detector selects the
// credential form on a page that also contains a decoy search form (no
// password), rather than blindly taking the first <form>.
func TestPlayground_HTTPMultiFormPicksLogin(t *testing.T) {
base := playgroundServer(t)
session, err := Login(context.Background(), nil, Config{
LoginURL: base + "/auth/multiform-login",
Username: fuzzplayground.AuthUsername,
Password: fuzzplayground.AuthPassword,
})
require.NoError(t, err, "detector must pick the login form over the decoy search form")
require.Contains(t, cookieNames(session), "PSESSION")
assertSessionAuthenticates(t, base, session)
}
// TestPlayground_HeadlessJSCookieLogin proves cookies set client-side via
// document.cookie (no Set-Cookie header) are captured from the browser jar.
func TestPlayground_HeadlessJSCookieLogin(t *testing.T) {
requireChrome(t)
base := playgroundServer(t)
ctx, cancel := context.WithTimeout(context.Background(), 60*time.Second)
defer cancel()
session, err := LoginHeadless(ctx, Config{
LoginURL: base + "/auth/jscookie-login",
Username: fuzzplayground.AuthUsername,
Password: fuzzplayground.AuthPassword,
SettleTime: 1500 * time.Millisecond,
})
require.NoError(t, err)
require.Contains(t, cookieNames(session), "PSESSION", "JS-set cookie must be captured from the browser")
assertSessionAuthenticates(t, base, session)
}
// TestPlayground_HeadlessDelayedFormLogin proves the engine waits for a login
// form that is injected asynchronously instead of failing immediately.
func TestPlayground_HeadlessDelayedFormLogin(t *testing.T) {
requireChrome(t)
base := playgroundServer(t)
ctx, cancel := context.WithTimeout(context.Background(), 60*time.Second)
defer cancel()
session, err := LoginHeadless(ctx, Config{
LoginURL: base + "/auth/delayed-login",
Username: fuzzplayground.AuthUsername,
Password: fuzzplayground.AuthPassword,
SettleTime: 2 * time.Second,
})
require.NoError(t, err, "engine must wait for the asynchronously-rendered form")
require.Contains(t, cookieNames(session), "PSESSION")
assertSessionAuthenticates(t, base, session)
}
// TestPlayground_HeadlessHeaderTokenLogin proves the headless engine recovers a
// token that is only ever delivered in an XHR response header (never stored in
// a cookie or web storage) via passive response-header interception.
func TestPlayground_HeadlessHeaderTokenLogin(t *testing.T) {
requireChrome(t)
base := playgroundServer(t)
ctx, cancel := context.WithTimeout(context.Background(), 60*time.Second)
defer cancel()
session, err := LoginHeadless(ctx, Config{
LoginURL: base + "/auth/spa-header-token-login",
Username: fuzzplayground.AuthUsername,
Password: fuzzplayground.AuthPassword,
SettleTime: 1500 * time.Millisecond,
})
require.NoError(t, err)
require.Empty(t, session.Cookies, "header-token SPA sets no cookie")
require.Empty(t, session.LocalStorage["token"], "the SPA does not store the token")
require.NotEmpty(t, session.Token, "token must be captured from the XHR response header")
assertSessionAuthenticates(t, base, session)
}
func TestPlayground_HeadlessOAuthRedirect(t *testing.T) {
requireChrome(t)
base := playgroundServer(t)
ctx, cancel := context.WithTimeout(context.Background(), 60*time.Second)
defer cancel()
// Pure redirect SSO: visiting authorize logs the user in; there is no form,
// so a single settle step lets the redirect chain complete before capture.
session, err := LoginHeadless(ctx, Config{
LoginURL: base + "/auth/oauth/authorize",
SettleTime: 1 * time.Second,
Steps: []LoginStep{{Action: "wait", Value: "1s"}},
})
require.NoError(t, err)
require.Contains(t, cookieNames(session), "PSESSION")
assertSessionAuthenticates(t, base, session)
}
// TestPlayground_RecordingReplay imports a Chrome Recorder export (with the base
// URL templated in), verifies the recorded credential literals are parameterized
// out, and replays the compiled steps headlessly to obtain a real session.
func TestPlayground_RecordingReplay(t *testing.T) {
requireChrome(t)
base := playgroundServer(t)
raw, err := os.ReadFile(filepath.Join("testdata", "recordings", "spa_login.json"))
require.NoError(t, err)
rec := strings.ReplaceAll(string(raw), "__BASE__", base)
recFile := filepath.Join(t.TempDir(), "spa_login.json")
require.NoError(t, os.WriteFile(recFile, []byte(rec), 0o600))
steps, err := StepsFromRecordingFile(recFile, fuzzplayground.AuthUsername, fuzzplayground.AuthPassword)
require.NoError(t, err)
// Credentials must never survive in the compiled steps.
for _, s := range steps {
require.NotContains(t, s.Value, fuzzplayground.AuthPassword, "password literal leaked into steps")
require.NotContains(t, s.Value, fuzzplayground.AuthUsername, "username literal leaked into steps")
}
require.Contains(t, stepValues(steps), "{{username}}")
require.Contains(t, stepValues(steps), "{{password}}")
// An xpath-only selector must compile to the engine's xpath= form.
require.Contains(t, stepSelectors(steps), "xpath=//input[@id='email']")
ctx, cancel := context.WithTimeout(context.Background(), 60*time.Second)
defer cancel()
session, err := LoginHeadless(ctx, Config{
LoginURL: FirstNavigateURL(steps),
Username: fuzzplayground.AuthUsername,
Password: fuzzplayground.AuthPassword,
Steps: steps,
SettleTime: 1500 * time.Millisecond,
})
require.NoError(t, err)
require.Contains(t, cookieNames(session), "PSESSION")
assertSessionAuthenticates(t, base, session)
}
func stepValues(steps []LoginStep) []string {
var out []string
for _, s := range steps {
out = append(out, s.Value)
}
return out
}
func stepSelectors(steps []LoginStep) []string {
var out []string
for _, s := range steps {
out = append(out, s.Selector)
}
return out
}
// --- Authenticated fuzzing target -----------------------------------------
// TestPlayground_AuthenticatedEndpointBehindLogin proves the authenticated,
// fuzzable endpoint is only reachable with a captured session and reflects input
// (a genuine target behind the login wall for authenticated scanning).
func TestPlayground_AuthenticatedEndpointBehindLogin(t *testing.T) {
base := playgroundServer(t)
session, err := Login(context.Background(), nil, Config{
LoginURL: base + "/auth/form-login",
Username: fuzzplayground.AuthUsername,
Password: fuzzplayground.AuthPassword,
})
require.NoError(t, err)
// Unauthenticated: blocked.
resp, err := http.Get(base + "/auth/api/search?q=hello")
require.NoError(t, err)
_ = resp.Body.Close()
require.Equal(t, http.StatusUnauthorized, resp.StatusCode, "search must require auth")
// Authenticated: reachable and reflects the payload unsanitized.
req, _ := http.NewRequest(http.MethodGet, base+"/auth/api/search?q="+payloadXSS, nil)
req.Header.Set("Cookie", session.CookieHeader)
resp2, err := http.DefaultClient.Do(req)
require.NoError(t, err)
defer func() { _ = resp2.Body.Close() }()
body, _ := io.ReadAll(resp2.Body)
require.Equal(t, http.StatusOK, resp2.StatusCode)
require.True(t, strings.Contains(string(body), payloadXSS), "authenticated endpoint should reflect the payload")
}
const payloadXSS = "<script>alert(1)</script>"
-138
View File
@@ -1,138 +0,0 @@
package autologin
import (
"context"
"encoding/json"
"os"
"testing"
"time"
"github.com/stretchr/testify/require"
)
// Real-application harness. These tests are skipped unless NUCLEI_REALAPP_LOGIN_URL
// is set, so normal CI is unaffected. They drive the headless engine against a
// live app to flush out engine bugs that synthetic httptest servers miss
// (heavy SPA bootstrap, XHR logins, hash routes, token-in-web-storage, etc.).
//
// Example (OWASP Juice Shop):
//
// docker run -d -p 3000:3000 bkimminich/juice-shop
// NUCLEI_REALAPP_LOGIN_URL='http://localhost:3000/#/login' \
// NUCLEI_REALAPP_USERNAME='[email protected]' \
// NUCLEI_REALAPP_PASSWORD='admin123' \
// NUCLEI_REALAPP_TOKEN_REGEX='(eyJ[A-Za-z0-9._-]+\.[A-Za-z0-9._-]+\.[A-Za-z0-9._-]+)' \
// NUCLEI_REALAPP_EXPECT_STORAGE_KEY='token' \
// go test ./pkg/authprovider/autologin/ -run TestRealApp_HeadlessLogin -v
func realAppConfig(t *testing.T) (Config, bool) {
t.Helper()
loginURL := os.Getenv("NUCLEI_REALAPP_LOGIN_URL")
if loginURL == "" {
t.Skip("set NUCLEI_REALAPP_LOGIN_URL to run the real-app login harness")
return Config{}, false
}
cfg := Config{
LoginURL: loginURL,
Username: os.Getenv("NUCLEI_REALAPP_USERNAME"),
Password: os.Getenv("NUCLEI_REALAPP_PASSWORD"),
UsernameField: os.Getenv("NUCLEI_REALAPP_USERNAME_FIELD"),
PasswordField: os.Getenv("NUCLEI_REALAPP_PASSWORD_FIELD"),
TokenRegex: os.Getenv("NUCLEI_REALAPP_TOKEN_REGEX"),
Proxy: os.Getenv("NUCLEI_REALAPP_PROXY"),
SettleTime: 8 * time.Second,
Timeout: 90 * time.Second,
}
if steps := os.Getenv("NUCLEI_REALAPP_STEPS"); steps != "" {
require.NoError(t, json.Unmarshal([]byte(steps), &cfg.Steps), "NUCLEI_REALAPP_STEPS must be a JSON array of LoginStep")
}
return cfg, true
}
func logSession(t *testing.T, s *Session) {
t.Helper()
t.Logf("final url: %s", s.FinalURL)
t.Logf("cookies (%d):", len(s.Cookies))
for _, c := range s.Cookies {
t.Logf(" %s", c.Name)
}
if s.Token != "" {
t.Logf("token: present (len=%d)", len(s.Token))
}
t.Logf("localStorage keys: %d, sessionStorage keys: %d", len(s.LocalStorage), len(s.SessionStorage))
for k := range s.LocalStorage {
t.Logf(" local[%s]", k)
}
for k := range s.SessionStorage {
t.Logf(" session[%s]", k)
}
}
func TestRealApp_HeadlessLogin(t *testing.T) {
requireChrome(t)
cfg, ok := realAppConfig(t)
if !ok {
return
}
ctx, cancel := context.WithTimeout(context.Background(), 2*time.Minute)
defer cancel()
session, err := LoginHeadless(ctx, cfg)
require.NoError(t, err, "headless login against real app failed")
logSession(t, session)
require.True(t, len(session.Cookies) > 0 || session.Token != "" || len(session.LocalStorage) > 0 || len(session.SessionStorage) > 0,
"expected to capture a session (cookie, token or web storage)")
if key := os.Getenv("NUCLEI_REALAPP_EXPECT_STORAGE_KEY"); key != "" {
_, inLocal := session.LocalStorage[key]
_, inSession := session.SessionStorage[key]
require.True(t, inLocal || inSession, "expected web storage key %q to be captured", key)
}
if name := os.Getenv("NUCLEI_REALAPP_EXPECT_COOKIE"); name != "" {
found := false
for _, c := range session.Cookies {
if c.Name == name {
found = true
break
}
}
require.True(t, found, "expected cookie %q to be captured", name)
}
}
// TestRealApp_RecordingReplay drives a recorder-JSON login flow (compiled to
// steps, with credential parameterization) against the live app, exercising the
// full record->replay path end to end. Set NUCLEI_REALAPP_RECORDING to a Chrome
// DevTools Recorder export to run it.
func TestRealApp_RecordingReplay(t *testing.T) {
requireChrome(t)
cfg, ok := realAppConfig(t)
if !ok {
return
}
recording := os.Getenv("NUCLEI_REALAPP_RECORDING")
if recording == "" {
t.Skip("set NUCLEI_REALAPP_RECORDING to a recorder json to run the replay harness")
}
steps, err := StepsFromRecordingFile(recording, cfg.Username, cfg.Password)
require.NoError(t, err, "failed to compile recording")
t.Logf("compiled %d steps from recording", len(steps))
for _, s := range steps {
t.Logf(" %s selector=%q", s.Action, s.Selector)
}
cfg.Steps = steps
if cfg.LoginURL == "" {
cfg.LoginURL = FirstNavigateURL(steps)
}
ctx, cancel := context.WithTimeout(context.Background(), 2*time.Minute)
defer cancel()
session, err := LoginHeadless(ctx, cfg)
require.NoError(t, err, "recorded login replay against real app failed")
logSession(t, session)
require.True(t, len(session.Cookies) > 0 || session.Token != "" || len(session.LocalStorage) > 0 || len(session.SessionStorage) > 0,
"expected recorded login to capture a session")
}
-221
View File
@@ -1,221 +0,0 @@
package autologin
import (
"encoding/json"
"fmt"
"os"
"strings"
"github.com/projectdiscovery/utils/errkit"
)
// recordingFile is the Chrome DevTools Recorder / @puppeteer/replay export
// schema (the two are identical). A user records their real login in Chrome
// (DevTools -> Recorder) and exports it as JSON; we compile it down to the same
// LoginStep list the headless engine already replays, so complex SSO/MFA logins
// can be authored by recording instead of hand-writing steps or templates.
type recordingFile struct {
Title string `json:"title"`
Steps []recordingStep `json:"steps"`
}
// recordingStep is a single recorded user action. Selectors is a list of
// alternative selector strategies for the target element, each itself a list
// (Chrome groups shadow/frame piercing selectors); we use the first entry of
// each group as a candidate.
type recordingStep struct {
Type string `json:"type"`
URL string `json:"url"`
Selectors [][]string `json:"selectors"`
Value string `json:"value"`
Key string `json:"key"`
Target string `json:"target"`
Expression string `json:"expression"`
}
// StepsFromRecordingFile reads and converts a recorder export file into
// LoginSteps. username/password are used to parameterize the recording so the
// captured credential literals are replaced with {{username}}/{{password}}
// placeholders (kept out of the committed flow file).
func StepsFromRecordingFile(path, username, password string) ([]LoginStep, error) {
data, err := os.ReadFile(path)
if err != nil {
return nil, errkit.Wrap(err, "auto-login: failed to read recording file")
}
return StepsFromRecording(data, username, password)
}
// StepsFromRecording converts a recorder export (JSON) into LoginSteps.
func StepsFromRecording(data []byte, username, password string) ([]LoginStep, error) {
var rec recordingFile
if err := json.Unmarshal(data, &rec); err != nil {
return nil, errkit.Wrap(err, "auto-login: invalid recording json")
}
if len(rec.Steps) == 0 {
return nil, errkit.New("auto-login: recording contains no steps")
}
var steps []LoginStep
for _, rs := range rec.Steps {
switch strings.ToLower(strings.TrimSpace(rs.Type)) {
case "navigate":
if rs.URL != "" {
steps = append(steps, LoginStep{Action: "navigate", Value: rs.URL})
}
case "click":
sel := pickSelector(rs.Selectors)
if sel == "" {
continue
}
steps = append(steps, LoginStep{Action: "click", Selector: sel})
case "change":
sel := pickSelector(rs.Selectors)
if sel == "" {
continue
}
steps = append(steps, LoginStep{
Action: "fill",
Selector: sel,
Value: parameterizeValue(rs.Value, rs.Selectors, username, password),
})
case "keydown":
// Only emit actionable keys; character keys are captured by `change`,
// and the paired keyUp event is ignored to avoid double submission.
if key := normalizeKey(rs.Key); key != "" {
steps = append(steps, LoginStep{Action: "press", Value: key, Selector: pickSelector(rs.Selectors)})
}
case "waitforelement":
sel := pickSelector(rs.Selectors)
if sel == "" {
continue
}
steps = append(steps, LoginStep{Action: "waitvisible", Selector: sel})
case "setviewport", "keyup", "scroll", "close", "emulatenetworkconditions", "hover", "":
// Not relevant to (or not supported for) headless login replay.
continue
default:
return nil, errkit.Newf("auto-login: unsupported recording action %q", rs.Type)
}
}
if len(steps) == 0 {
return nil, errkit.New("auto-login: recording produced no replayable steps")
}
return steps, nil
}
// FirstNavigateURL returns the URL of the first navigate step in a recording, or
// "" if none. It lets the caller derive the login URL from the recording.
func FirstNavigateURL(steps []LoginStep) string {
for _, s := range steps {
if strings.EqualFold(s.Action, "navigate") && s.Value != "" {
return s.Value
}
}
return ""
}
// normalizeKey maps recorder key names to the engine's supported press keys.
// Character keys (length 1) return "" because the text they produce is already
// captured by `change` steps.
func normalizeKey(key string) string {
switch strings.ToLower(strings.TrimSpace(key)) {
case "enter", "return", "numpadenter":
return "enter"
case "tab":
return "tab"
case "escape", "esc":
return "escape"
case "space":
return "space"
default:
return ""
}
}
// parameterizeValue replaces a recorded credential literal with a placeholder so
// secrets never live in the recording file. It masks by exact match against the
// configured credentials and, defensively, masks any value typed into a field
// whose selector (any alternative) looks like a password input.
func parameterizeValue(value string, selectors [][]string, username, password string) string {
if password != "" && value == password {
return "{{password}}"
}
if username != "" && value == username {
return "{{username}}"
}
for _, group := range selectors {
for _, selector := range group {
if looksLikePasswordSelector(selector) {
return "{{password}}"
}
}
}
return value
}
func looksLikePasswordSelector(selector string) bool {
s := strings.ToLower(selector)
return strings.Contains(s, "password") || strings.Contains(s, "passwd") || strings.Contains(s, "type=\"password\"") || strings.Contains(s, "type=password")
}
// pickSelector chooses the most engine-friendly selector from the recorder's
// alternatives, preferring native CSS, then XPath, then an aria-label CSS
// approximation, then a text-content XPath. Shadow-piercing (pierce/) selectors
// are skipped because the engine cannot resolve them.
func pickSelector(groups [][]string) string {
var css, xpath, aria, text string
for _, group := range groups {
for _, s := range group {
s = strings.TrimSpace(s)
switch {
case s == "":
continue
case strings.HasPrefix(s, "xpath/"):
if xpath == "" {
xpath = "xpath=" + strings.TrimPrefix(s, "xpath/")
}
case strings.HasPrefix(s, "aria/"):
if aria == "" {
aria = ariaToCSS(strings.TrimPrefix(s, "aria/"))
}
case strings.HasPrefix(s, "text/"):
if text == "" {
text = textToXPath(strings.TrimPrefix(s, "text/"))
}
case strings.HasPrefix(s, "pierce/"):
continue
default:
if css == "" {
css = s
}
}
}
}
for _, candidate := range []string{css, xpath, aria, text} {
if candidate != "" {
return candidate
}
}
return ""
}
// ariaToCSS approximates an accessibility-name selector as an aria-label CSS
// attribute selector. It is best-effort; recordings usually also carry a CSS or
// XPath selector that is preferred over this.
func ariaToCSS(name string) string {
name = strings.TrimSpace(name)
if name == "" || strings.Contains(name, `"`) {
return ""
}
return fmt.Sprintf(`[aria-label="%s"]`, name)
}
// textToXPath converts a text-content selector into an XPath the engine can run.
func textToXPath(t string) string {
t = strings.TrimSpace(t)
if t == "" || strings.Contains(t, `"`) {
return ""
}
return fmt.Sprintf(`xpath=//*[contains(normalize-space(.), "%s")]`, t)
}
@@ -1,174 +0,0 @@
package autologin
import (
"context"
"fmt"
"net/http"
"net/http/httptest"
"testing"
"time"
"github.com/stretchr/testify/require"
)
// chromeRecording is a representative Chrome DevTools Recorder export of a
// username-first login, including the noise (setViewport, paired keyUp) and the
// multiple selector strategies Chrome emits per step.
const chromeRecording = `{
"title": "login",
"steps": [
{"type": "setViewport", "width": 1280, "height": 720},
{"type": "navigate", "url": "https://app.example.com/login"},
{"type": "change", "value": "[email protected]", "selectors": [["#email"], ["aria/Email"], ["xpath///*[@id=\"email\"]"]]},
{"type": "click", "selectors": [["#next"], ["aria/Next"]]},
{"type": "waitForElement", "selectors": [["#password"]]},
{"type": "change", "value": "p@ss", "selectors": [["#password"]]},
{"type": "keyDown", "key": "Enter"},
{"type": "keyUp", "key": "Enter"},
{"type": "click", "selectors": [["#submit"]]}
]
}`
func TestStepsFromRecording_MapsAndParameterizes(t *testing.T) {
steps, err := StepsFromRecording([]byte(chromeRecording), "[email protected]", "p@ss")
require.NoError(t, err)
require.Equal(t, []LoginStep{
{Action: "navigate", Value: "https://app.example.com/login"},
{Action: "fill", Selector: "#email", Value: "{{username}}"},
{Action: "click", Selector: "#next"},
{Action: "waitvisible", Selector: "#password"},
{Action: "fill", Selector: "#password", Value: "{{password}}"},
{Action: "press", Value: "enter"},
{Action: "click", Selector: "#submit"},
}, steps, "recording should map to placeholder-parameterized steps (setViewport/keyUp dropped)")
}
func TestStepsFromRecording_PasswordSelectorMaskedWithoutCredMatch(t *testing.T) {
// Even if the recorded literal doesn't match configured creds, a value typed
// into a password-looking field must never survive in the steps.
rec := `{"steps": [
{"type": "change", "value": "literal-secret", "selectors": [["input[type=password]"]]}
]}`
steps, err := StepsFromRecording([]byte(rec), "", "")
require.NoError(t, err)
require.Len(t, steps, 1)
require.Equal(t, "{{password}}", steps[0].Value, "value typed into a password field must be masked")
// CSS can win over an aria/Password alternative; still mask via any selector.
rec = `{"steps": [
{"type": "change", "value": "literal-secret", "selectors": [["#input-2"], ["aria/Password"]]}
]}`
steps, err = StepsFromRecording([]byte(rec), "", "")
require.NoError(t, err)
require.Len(t, steps, 1)
require.Equal(t, "#input-2", steps[0].Selector)
require.Equal(t, "{{password}}", steps[0].Value, "password-looking alternate selectors must still mask")
}
func TestStepsFromRecording_SelectorPriority(t *testing.T) {
// CSS preferred over aria/xpath; xpath/ prefix becomes an engine xpath= form;
// aria-only falls back to an aria-label CSS approximation; pierce/ is ignored.
rec := `{"steps": [
{"type": "click", "selectors": [["aria/Save"], ["xpath///button[1]"], ["#real"]]},
{"type": "click", "selectors": [["pierce/#shadow"], ["xpath///div[2]"]]},
{"type": "click", "selectors": [["aria/Submit"]]}
]}`
steps, err := StepsFromRecording([]byte(rec), "", "")
require.NoError(t, err)
require.Equal(t, "#real", steps[0].Selector, "CSS should win over aria/xpath")
require.Equal(t, "xpath=//div[2]", steps[1].Selector, "xpath used when no CSS, pierce ignored")
require.Equal(t, `[aria-label="Submit"]`, steps[2].Selector, "aria falls back to aria-label CSS")
}
func TestStepsFromRecording_Errors(t *testing.T) {
_, err := StepsFromRecording([]byte(`not json`), "", "")
require.Error(t, err)
_, err = StepsFromRecording([]byte(`{"steps": []}`), "", "")
require.Error(t, err, "empty recording must error")
_, err = StepsFromRecording([]byte(`{"steps": [{"type":"setViewport"}]}`), "", "")
require.Error(t, err, "recording with no replayable steps must error")
_, err = StepsFromRecording([]byte(`{"steps": [{"type":"doubleClick","selectors":[["#x"]]}]}`), "", "")
require.Error(t, err, "unsupported actions must fail the import")
require.Contains(t, err.Error(), "unsupported recording action")
_, err = StepsFromRecording([]byte(`{"steps": [{"type":"waitForExpression","expression":"1==1"}]}`), "", "")
require.Error(t, err, "expression waits must fail rather than become fixed waits")
}
func TestFirstNavigateURL(t *testing.T) {
steps := []LoginStep{
{Action: "fill", Selector: "#x"},
{Action: "navigate", Value: "https://app.example.com/login"},
{Action: "navigate", Value: "https://second"},
}
require.Equal(t, "https://app.example.com/login", FirstNavigateURL(steps))
require.Equal(t, "", FirstNavigateURL([]LoginStep{{Action: "click"}}))
}
// TestLoginHeadless_FromRecording proves an end-to-end recorded login: a Chrome
// recording is compiled to steps (one of them an XPath selector) and replayed by
// the headless engine against a real browser to acquire the session.
func TestLoginHeadless_FromRecording(t *testing.T) {
requireChrome(t)
app := newJSLoginApp()
srv := httptest.NewServer(func() http.Handler {
mux := http.NewServeMux()
mux.HandleFunc("/login", func(w http.ResponseWriter, r *http.Request) {
if r.Method == http.MethodGet {
w.Header().Set("Content-Type", "text/html")
_, _ = fmt.Fprint(w, twoStepLoginPage)
return
}
_ = r.ParseForm()
if r.PostFormValue("email") != "[email protected]" || r.PostFormValue("password") != "p@ss" {
w.Header().Set("Content-Type", "text/html")
_, _ = fmt.Fprint(w, twoStepLoginPage)
return
}
http.SetCookie(w, &http.Cookie{Name: "session", Value: "sess-dave", Path: "/"})
app.mu.Lock()
app.sessions["sess-dave"] = true
app.mu.Unlock()
http.Redirect(w, r, "/dashboard", http.StatusFound)
})
mux.HandleFunc("/dashboard", app.handleDashboard)
return mux
}())
defer srv.Close()
// The "next" click uses an XPath selector to exercise the engine's XPath path.
recording := fmt.Sprintf(`{"steps": [
{"type": "navigate", "url": %q},
{"type": "change", "value": "[email protected]", "selectors": [["#email"]]},
{"type": "click", "selectors": [["xpath///*[@id=\"next\"]"]]},
{"type": "waitForElement", "selectors": [["#password"]]},
{"type": "change", "value": "p@ss", "selectors": [["#password"]]},
{"type": "click", "selectors": [["#submit"]]}
]}`, srv.URL+"/login")
steps, err := StepsFromRecording([]byte(recording), "[email protected]", "p@ss")
require.NoError(t, err)
require.Equal(t, "xpath=//*[@id=\"next\"]", steps[2].Selector, "recorded xpath selector should be preserved")
ctx, cancel := context.WithTimeout(context.Background(), 60*time.Second)
defer cancel()
session, err := LoginHeadless(ctx, Config{
LoginURL: FirstNavigateURL(steps),
Username: "[email protected]",
Password: "p@ss",
SettleTime: 1 * time.Second,
Steps: steps,
})
require.NoError(t, err)
names := map[string]string{}
for _, c := range session.Cookies {
names[c.Name] = c.Value
}
require.Contains(t, names, "session", "recorded login should capture the session cookie")
}
@@ -1,41 +0,0 @@
{
"title": "playground spa login",
"steps": [
{
"type": "setViewport",
"width": 1280,
"height": 720
},
{
"type": "navigate",
"url": "__BASE__/auth/spa-login"
},
{
"type": "change",
"value": "[email protected]",
"selectors": [
["xpath///input[@id='email']"]
]
},
{
"type": "change",
"value": "nuclei-rocks",
"selectors": [
["#password"],
["aria/Password"]
]
},
{
"type": "click",
"selectors": [
["aria/Sign in"],
["#submit"],
["xpath///button[@id='submit']"]
]
},
{
"type": "keyUp",
"key": "Enter"
}
]
}
-175
View File
@@ -1,175 +0,0 @@
package authprovider
import (
"fmt"
"net/http"
"net/http/httptest"
"os"
"path/filepath"
"sync"
"testing"
"github.com/projectdiscovery/nuclei/v3/pkg/authprovider/authx"
"github.com/stretchr/testify/require"
)
// autoLoginE2EApp is a hermetic, CSRF-protected, cookie-session web app used to
// exercise the whole auto-login autonomy path through the real FileAuthProvider:
// secrets-file parse -> dynamic validate -> auto-login callback -> form detect +
// submit -> captured session applied as a cookie strategy -> reauth-on-401.
//
// Each successful login mints a *fresh* session token (sess-N), so a
// re-authentication is observable as a changed applied cookie value.
type autoLoginE2EApp struct {
mu sync.Mutex
loginCount int
validToken string // the currently valid session token; "" means expired
}
func (a *autoLoginE2EApp) server() *httptest.Server {
mux := http.NewServeMux()
mux.HandleFunc("/login", a.handleLogin)
mux.HandleFunc("/dashboard", a.handleDashboard)
return httptest.NewServer(mux)
}
func (a *autoLoginE2EApp) handleLogin(w http.ResponseWriter, r *http.Request) {
if r.Method == http.MethodGet {
http.SetCookie(w, &http.Cookie{Name: "csrftoken", Value: "csrf-secret", Path: "/"})
w.Header().Set("Content-Type", "text/html")
_, _ = fmt.Fprint(w, `<html><body>
<form action="/login" method="post">
<input type="hidden" name="csrf" value="csrf-secret">
<input type="email" name="email" autocomplete="username">
<input type="password" name="password">
<input type="submit" name="commit" value="Sign in">
</form></body></html>`)
return
}
_ = r.ParseForm()
csrf, err := r.Cookie("csrftoken")
if err != nil || r.PostFormValue("csrf") != csrf.Value {
http.Error(w, "csrf mismatch", http.StatusForbidden)
return
}
if r.PostFormValue("email") != "[email protected]" || r.PostFormValue("password") != "s3cr3t" {
w.Header().Set("Content-Type", "text/html")
_, _ = fmt.Fprint(w, `<form action="/login" method="post"><input type="password" name="password"></form>`)
return
}
a.mu.Lock()
a.loginCount++
a.validToken = fmt.Sprintf("sess-%d", a.loginCount)
token := a.validToken
a.mu.Unlock()
http.SetCookie(w, &http.Cookie{Name: "session", Value: token, Path: "/"})
http.Redirect(w, r, "/dashboard", http.StatusFound)
}
func (a *autoLoginE2EApp) handleDashboard(w http.ResponseWriter, r *http.Request) {
c, err := r.Cookie("session")
a.mu.Lock()
valid := a.validToken
a.mu.Unlock()
if err != nil || valid == "" || c.Value != valid {
http.Error(w, "unauthorized", http.StatusUnauthorized)
return
}
_, _ = fmt.Fprint(w, "welcome")
}
func (a *autoLoginE2EApp) expire() {
a.mu.Lock()
a.validToken = ""
a.mu.Unlock()
}
// appliedSessionCookie applies the strategies to a fresh request and returns the
// value of the "session" cookie they set (or "").
func appliedSessionCookie(strategies []authx.AuthStrategy, target string) string {
req, _ := http.NewRequest(http.MethodGet, target, nil)
for _, s := range strategies {
s.Apply(req)
}
if c, err := req.Cookie("session"); err == nil {
return c.Value
}
return ""
}
func TestAutoLogin_FullLifecycle_E2E(t *testing.T) {
app := &autoLoginE2EApp{}
srv := app.server()
defer srv.Close()
host := srv.Listener.Addr().String()
secrets := fmt.Sprintf(`id: auto-login-e2e
info:
name: auto-login-e2e
dynamic:
- domains:
- "%s"
auto-login:
login-url: "%s/login"
username: "[email protected]"
password: "s3cr3t"
reauth-status-codes:
- 401
`, host, srv.URL)
dir := t.TempDir()
secretsPath := filepath.Join(dir, "auth.yaml")
require.NoError(t, os.WriteFile(secretsPath, []byte(secrets), 0o644))
// callback is nil: auto-login dynamics need no template store.
provider, err := NewFileAuthProvider(secretsPath, nil, nil)
require.NoError(t, err)
// Pre-fetch performs the real form login.
require.NoError(t, provider.PreFetchSecrets())
strategies := provider.LookupAddr(host)
require.NotEmpty(t, strategies, "auto-login secret should be scoped to the app host")
// 1. The captured session authenticates against the protected route.
cookie1 := appliedSessionCookie(strategies, srv.URL+"/dashboard")
require.Equal(t, "sess-1", cookie1, "first login should apply session token sess-1")
requireDashboard(t, srv.URL, cookie1, http.StatusOK)
// 2. Server-side session expiry: the applied cookie now 401s.
app.expire()
requireDashboard(t, srv.URL, cookie1, http.StatusUnauthorized)
// 3. Surface the 401 to the strategy (as the scan engine does), which marks
// the dynamic session stale for re-authentication.
reauthTriggered := false
for _, s := range strategies {
if insp, ok := s.(authx.ResponseInspector); ok {
if insp.OnResponse(http.StatusUnauthorized, 0) {
reauthTriggered = true
}
}
}
require.True(t, reauthTriggered, "401 should trigger re-authentication on the dynamic strategy")
// 4. The next application re-runs the form login and applies a *fresh*
// session token, restoring access.
cookie2 := appliedSessionCookie(strategies, srv.URL+"/dashboard")
require.Equal(t, "sess-2", cookie2, "re-auth should apply the freshly minted session token")
requireDashboard(t, srv.URL, cookie2, http.StatusOK)
require.Equal(t, 2, app.loginCount, "login form should have been submitted exactly twice")
}
func requireDashboard(t *testing.T, baseURL, sessionCookie string, wantStatus int) {
t.Helper()
req, _ := http.NewRequest(http.MethodGet, baseURL+"/dashboard", nil)
if sessionCookie != "" {
req.AddCookie(&http.Cookie{Name: "session", Value: sessionCookie})
}
resp, err := http.DefaultClient.Do(req)
require.NoError(t, err)
defer func() { _ = resp.Body.Close() }()
require.Equal(t, wantStatus, resp.StatusCode)
}
+5 -38
View File
@@ -21,34 +21,16 @@ type FileAuthProvider struct {
}
// NewFileAuthProvider creates a new file based auth provider
func NewFileAuthProvider(path string, callback authx.LazyFetchSecret, autoLoginOpts *authx.AutoLoginRuntimeOptions) (AuthProvider, error) {
func NewFileAuthProvider(path string, callback authx.LazyFetchSecret) (AuthProvider, error) {
store, err := authx.GetAuthDataFromFile(path)
if err != nil {
return nil, err
}
return newProviderFromStore(path, store, callback, autoLoginOpts)
}
// NewStoreAuthProvider builds an auth provider from an already-constructed Authx
// store (e.g. one assembled from CLI flags rather than read from a file).
func NewStoreAuthProvider(store *authx.Authx, callback authx.LazyFetchSecret, autoLoginOpts *authx.AutoLoginRuntimeOptions) (AuthProvider, error) {
return newProviderFromStore("", store, callback, autoLoginOpts)
}
// newProviderFromStore validates the store, installs fetch callbacks and builds
// the domain lookup indexes shared by the file- and store-based constructors.
func newProviderFromStore(path string, store *authx.Authx, callback authx.LazyFetchSecret, autoLoginOpts *authx.AutoLoginRuntimeOptions) (AuthProvider, error) {
if store == nil || (len(store.Secrets) == 0 && len(store.Dynamic) == 0) {
if len(store.Secrets) == 0 && len(store.Dynamic) == 0 {
return nil, ErrNoSecrets
}
if callback == nil {
// The lazy fetch callback is only required for template-based dynamic
// secrets; auto-login dynamics resolve their session without a template.
for _, dynamic := range store.Dynamic {
if dynamic.AutoLogin == nil {
return nil, errkit.New("lazy fetch callback is required for dynamic secrets")
}
}
if len(store.Dynamic) > 0 && callback == nil {
return nil, errkit.New("lazy fetch callback is required for dynamic secrets")
}
for _, secret := range store.Secrets {
if err := secret.Validate(); err != nil {
@@ -63,13 +45,7 @@ func newProviderFromStore(path string, store *authx.Authx, callback authx.LazyFe
errorErr.Msgf("invalid dynamic in file: %s", path)
return nil, errorErr
}
if dynamic.AutoLogin != nil {
// Auto-login resolves its session via the form-login engine rather
// than a template, so it uses a dedicated callback.
dynamic.SetAutoLoginCallback(autoLoginOpts)
} else {
dynamic.SetLazyFetchCallback(callback)
}
dynamic.SetLazyFetchCallback(callback)
store.Dynamic[i] = dynamic
}
f := &FileAuthProvider{Path: path, store: store}
@@ -216,12 +192,3 @@ func (f *FileAuthProvider) PreFetchSecrets() error {
}
return nil
}
// Close drops any in-memory session captured by dynamic (auto-login) secrets so
// nuclei does not retain live session material after the scan. It is a no-op for
// static secrets and never contacts the target.
func (f *FileAuthProvider) Close() {
for i := range f.store.Dynamic {
f.store.Dynamic[i].Close()
}
}
+1 -1
View File
@@ -41,7 +41,7 @@ dynamic:
time.Sleep(75 * time.Millisecond)
dynamic.Extracted = map[string]interface{}{"token": "session-token"}
return nil
}, nil)
})
require.NoError(t, err)
const workers = 20
+1 -9
View File
@@ -35,11 +35,6 @@ type AuthProvider interface {
// PreFetchSecrets pre-fetches the secrets from the auth provider
// instead of lazy fetching
PreFetchSecrets() error
// Close releases any in-memory session material captured during the scan
// (e.g. cookies/tokens/web storage obtained by an auto-login dynamic
// secret). It must be safe to call once at scan teardown and a no-op for
// providers that hold only static secrets. It never contacts the target.
Close()
}
// AuthProviderOptions contains options for the auth provider
@@ -48,16 +43,13 @@ type AuthProviderOptions struct {
SecretsFiles []string
// LazyFetchSecret is a callback for lazy fetching of dynamic secrets
LazyFetchSecret authx.LazyFetchSecret
// AutoLoginOptions carries scan-level browser/runtime configuration into
// auto-login dynamic secrets (user-agent, headers, proxy, CDP, etc.). May be nil.
AutoLoginOptions *authx.AutoLoginRuntimeOptions
}
// NewAuthProvider creates a new auth provider from the given options
func NewAuthProvider(options *AuthProviderOptions) (AuthProvider, error) {
var providers []AuthProvider
for _, file := range options.SecretsFiles {
provider, err := NewFileAuthProvider(file, options.LazyFetchSecret, options.AutoLoginOptions)
provider, err := NewFileAuthProvider(file, options.LazyFetchSecret)
if err != nil {
return nil, err
}
-8
View File
@@ -65,11 +65,3 @@ func (m *MultiAuthProvider) PreFetchSecrets() error {
}
return nil
}
// Close releases any in-memory session held by the wrapped providers (e.g.
// auto-login dynamic secrets).
func (m *MultiAuthProvider) Close() {
for _, provider := range m.Providers {
provider.Close()
}
}
+10 -4
View File
@@ -631,10 +631,10 @@ func (store *Store) noteMissingCapabilities(templatePath string, missingCaps []t
// erroring out on discovering any faulty templates.
func (store *Store) ValidateTemplates() error {
templatePaths, errs := store.config.Catalog.GetTemplatesPath(store.finalTemplates)
store.logErroredTemplates(errs)
notFound := store.logErroredTemplates(errs)
workflowPaths, errs := store.config.Catalog.GetTemplatesPath(store.finalWorkflows)
store.logErroredTemplates(errs)
notFound += store.logErroredTemplates(errs)
templatePathsMap := make(map[string]struct{}, len(templatePaths))
for _, path := range templatePaths {
@@ -646,7 +646,8 @@ func (store *Store) ValidateTemplates() error {
workflowPathsMap[path] = struct{}{}
}
if store.areTemplatesValid(templatePathsMap) && store.areWorkflowsValid(workflowPathsMap) {
// a path that can't be resolved is not validated, so it can't count as valid
if store.areTemplatesValid(templatePathsMap) && store.areWorkflowsValid(workflowPathsMap) && notFound == 0 {
return nil
}
@@ -1056,10 +1057,15 @@ func workflowContainsProtocol(workflow []*workflows.WorkflowTemplate) bool {
return false
}
func (s *Store) logErroredTemplates(erred map[string]error) {
// logErroredTemplates logs the templates that could not be found and returns
// how many of them were not handled by the NotFoundCallback.
func (s *Store) logErroredTemplates(erred map[string]error) int {
notFound := 0
for template, err := range erred {
if s.NotFoundCallback == nil || !s.NotFoundCallback(template) {
s.logger.Error().Msgf("Could not find template '%s': %s", template, err)
notFound++
}
}
return notFound
}
+43 -20
View File
@@ -176,25 +176,8 @@ func TestLoadTemplatesOnlyMetadataLogsCachedTemplateParseErrors(t *testing.T) {
require.Contains(t, output.String(), "Could not load template")
}
func TestValidateTemplatesRejectsRuntimeCompilationError(t *testing.T) {
templatePath := filepath.Join(t.TempDir(), "invalid-regex-group.yaml")
require.NoError(t, os.WriteFile(templatePath, []byte(`id: invalid-regex-group
info:
name: Invalid Regex Group
author: pdteam
severity: info
http:
- method: GET
path:
- "{{BaseURL}}"
extractors:
- type: regex
group: -1
regex:
- "(a)(b)"
`), 0o600))
func newValidateStore(t *testing.T, executionID, templatePath string) (*Store, *bytes.Buffer) {
t.Helper()
options := testutils.DefaultOptions.Copy()
var output bytes.Buffer
@@ -203,7 +186,7 @@ http:
logger.SetWriter(&utils.CaptureWriter{Buffer: &output})
logger.SetMaxLevel(levels.LevelDebug)
options.Logger = logger
options.ExecutionId = "loader-invalid-regex-group"
options.ExecutionId = executionID
options.Templates = []string{templatePath}
options.Validate = true
options.TemplateLoadingConcurrency = 1
@@ -226,10 +209,50 @@ http:
store, err := New(NewConfig(options, catalog, executerOpts))
require.NoError(t, err)
return store, &output
}
func TestValidateTemplatesRejectsRuntimeCompilationError(t *testing.T) {
templatePath := filepath.Join(t.TempDir(), "invalid-regex-group.yaml")
require.NoError(t, os.WriteFile(templatePath, []byte(`id: invalid-regex-group
info:
name: Invalid Regex Group
author: pdteam
severity: info
http:
- method: GET
path:
- "{{BaseURL}}"
extractors:
- type: regex
group: -1
regex:
- "(a)(b)"
`), 0o600))
store, output := newValidateStore(t, "loader-invalid-regex-group", templatePath)
require.Error(t, store.ValidateTemplates())
require.Contains(t, output.String(), "regex extractor group must be >= 0")
}
func TestValidateTemplatesRejectsMissingTemplatePath(t *testing.T) {
templatePath := filepath.Join(t.TempDir(), "no-such-templates")
store, output := newValidateStore(t, "loader-missing-template-path", templatePath)
require.Error(t, store.ValidateTemplates())
require.Contains(t, output.String(), "Could not find template")
}
func TestValidateTemplatesAcceptsMissingPathHandledByCallback(t *testing.T) {
templatePath := filepath.Join(t.TempDir(), "no-such-templates")
store, _ := newValidateStore(t, "loader-missing-path-callback", templatePath)
store.NotFoundCallback = func(string) bool { return true }
require.NoError(t, store.ValidateTemplates())
}
func TestRemoteTemplates(t *testing.T) {
catalog := disk.NewCatalog("")
+1
View File
@@ -5,6 +5,7 @@ Formats implements support for passing a number of request source as input provi
Currently the following formats are implemented -
- Burp Suite XML Request/Response file
- HAR (HTTP Archive) file exported by browsers and proxies
- Proxify JSONL output file
- OpenAPI Specification file
- Postman Collection file
+137
View File
@@ -0,0 +1,137 @@
package har
import (
"encoding/base64"
"io"
"net/http"
"strconv"
"strings"
"github.com/pkg/errors"
"github.com/projectdiscovery/gologger"
"github.com/projectdiscovery/nuclei/v3/pkg/input/formats"
"github.com/projectdiscovery/nuclei/v3/pkg/input/types"
"github.com/projectdiscovery/nuclei/v3/pkg/utils/json"
)
// HARFormat is an HTTP Archive (HAR 1.2) parser, the format browsers and
// proxies export captured traffic in.
type HARFormat struct {
opts formats.InputFormatOptions
}
// New creates a new HAR parser
func New() *HARFormat {
return &HARFormat{}
}
var _ formats.Format = &HARFormat{}
type archive struct {
Log struct {
Entries []entry `json:"entries"`
} `json:"log"`
}
type entry struct {
Request struct {
Method string `json:"method"`
URL string `json:"url"`
Headers []header `json:"headers"`
PostData *struct {
Text string `json:"text"`
} `json:"postData"`
} `json:"request"`
Response struct {
Status int `json:"status"`
StatusText string `json:"statusText"`
Headers []header `json:"headers"`
Content struct {
Text string `json:"text"`
Encoding string `json:"encoding"`
} `json:"content"`
} `json:"response"`
}
type header struct {
Name string `json:"name"`
Value string `json:"value"`
}
// Name returns the name of the format
func (h *HARFormat) Name() string {
return "har"
}
func (h *HARFormat) SetOptions(options formats.InputFormatOptions) {
h.opts = options
}
// Parse parses the input and calls the provided callback
// function for each RawRequest it discovers.
func (h *HARFormat) Parse(input io.Reader, resultsCb formats.ParseReqRespCallback, filePath string) error {
var har archive
if err := json.NewDecoder(input).Decode(&har); err != nil {
return errors.Wrap(err, "could not decode har file")
}
for _, e := range har.Log.Entries {
// Browsers also log data:, blob: and extension URLs, which have no
// host to send a request to.
if !strings.HasPrefix(e.Request.URL, "http://") && !strings.HasPrefix(e.Request.URL, "https://") {
continue
}
rawRequest, err := types.ParseRawRequestWithURL(rawRequestOf(e), e.Request.URL)
if err != nil {
gologger.Warning().Msgf("har: Could not parse request %s: %s\n", e.Request.URL, err)
continue
}
// Status 0 is a request that was blocked or never answered.
if e.Response.Status != 0 {
rawRequest.Response = &types.HttpResponse{Raw: rawResponseOf(e)}
}
resultsCb(rawRequest)
}
return nil
}
// rawRequestOf rebuilds the request as HTTP/1.1 with an absolute target, which
// carries the authority that HTTP/2 captures only keep in pseudo-headers.
func rawRequestOf(e entry) string {
var b strings.Builder
b.WriteString(e.Request.Method + " " + e.Request.URL + " HTTP/1.1\r\n")
writeHeaders(&b, e.Request.Headers)
if e.Request.PostData != nil {
b.WriteString(e.Request.PostData.Text)
}
return b.String()
}
func rawResponseOf(e entry) string {
statusText := e.Response.StatusText
if statusText == "" {
statusText = http.StatusText(e.Response.Status)
}
var b strings.Builder
b.WriteString("HTTP/1.1 " + strconv.Itoa(e.Response.Status) + " " + statusText + "\r\n")
writeHeaders(&b, e.Response.Headers)
body := e.Response.Content.Text
if e.Response.Content.Encoding == "base64" {
if decoded, err := base64.StdEncoding.DecodeString(body); err == nil {
body = string(decoded)
}
}
b.WriteString(body)
return b.String()
}
func writeHeaders(b *strings.Builder, headers []header) {
for _, h := range headers {
// HTTP/2 pseudo-headers (:authority, :path, ...) are not headers.
if strings.HasPrefix(h.Name, ":") {
continue
}
b.WriteString(h.Name + ": " + h.Value + "\r\n")
}
b.WriteString("\r\n")
}
+71
View File
@@ -0,0 +1,71 @@
package har
import (
"os"
"strings"
"testing"
"github.com/projectdiscovery/nuclei/v3/pkg/input/types"
"github.com/stretchr/testify/require"
)
const chromeHAR = "../testdata/chrome.har"
func parse(t *testing.T, input string) []*types.RequestResponse {
t.Helper()
file, err := os.Open(input)
require.NoError(t, err)
defer func() { _ = file.Close() }()
var got []*types.RequestResponse
require.NoError(t, New().Parse(file, func(rr *types.RequestResponse) bool {
got = append(got, rr)
return false
}, input))
return got
}
func TestHARParse(t *testing.T) {
got := parse(t, chromeHAR)
urls := make([]string, 0, len(got))
for _, rr := range got {
urls = append(urls, rr.URL.String())
}
require.Equal(t, []string{
"https://ginandjuice.shop/catalog?category=Juice",
"https://ginandjuice.shop/login",
"https://ginandjuice.shop/blocked.js",
}, urls, "non-http entries such as data: URLs are skipped")
}
func TestHARParseHTTP2Request(t *testing.T) {
rr := parse(t, chromeHAR)[0]
require.Equal(t, "GET", rr.Request.Method)
cookie, _ := rr.Request.Headers.Get("cookie")
require.Equal(t, "session=abc123", cookie)
for _, name := range rr.Request.Headers.GetKeys() {
require.False(t, strings.HasPrefix(name, ":"), "pseudo-header %s is not a real header", name)
}
req, err := rr.BuildRequest()
require.NoError(t, err)
require.Equal(t, "ginandjuice.shop", req.Host)
}
func TestHARParseBodyAndResponse(t *testing.T) {
got := parse(t, chromeHAR)
login := got[1]
require.Equal(t, "POST", login.Request.Method)
require.Equal(t, "username=carlos&password=hunter2", login.Request.Body)
require.NotNil(t, login.Response)
require.Contains(t, login.Response.Raw, "302 Found")
require.Contains(t, login.Response.Raw, "Location: /my-account")
require.NotNil(t, got[0].Response)
require.Contains(t, got[0].Response.Raw, "<h1>Juice</h1>", "base64 content is decoded")
require.Nil(t, got[2].Response, "a request that never got an answer has no response")
}
+20
View File
@@ -0,0 +1,20 @@
//go:build gofuzz
// +build gofuzz
package openapi
// Fuzz exercises OpenAPI 3.0 schema parsing and request generation. The input
// bytes are treated as a raw schema document (YAML or JSON) and fed through the
// format's Parse, which loads the schema and generates requests from it.
func Fuzz(data []byte) int {
if len(data) == 0 {
return 0
}
if len(data) > fuzzMaxInputSize {
return -1
}
if !fuzzOpenAPIParsing(data) {
return 0
}
return 1
}
+32
View File
@@ -0,0 +1,32 @@
package openapi
import (
"bytes"
"github.com/projectdiscovery/nuclei/v3/pkg/input/formats"
"github.com/projectdiscovery/nuclei/v3/pkg/input/types"
)
const fuzzMaxInputSize = 64 << 10
// fuzzOpenAPIParsing feeds data to the OpenAPI format parser as a raw schema
// document and drives request generation. It must never panic for any input;
// it returns true when the schema parsed and at least one request was produced.
func fuzzOpenAPIParsing(data []byte) bool {
if len(data) == 0 || len(data) > fuzzMaxInputSize {
return false
}
format := New()
format.SetOptions(formats.InputFormatOptions{})
produced := false
cb := func(rr *types.RequestResponse) bool {
produced = true
return true
}
if err := format.Parse(bytes.NewReader(data), cb, "fuzz.openapi"); err != nil {
return false
}
return produced
}
@@ -0,0 +1,84 @@
package openapi
import (
"os"
"path/filepath"
"strings"
"testing"
"github.com/projectdiscovery/nuclei/v3/pkg/input/types"
"github.com/stretchr/testify/require"
)
func TestFuzzOpenAPIParsingSeedCorpus(t *testing.T) {
entries, err := os.ReadDir("testdata/gofuzz-corpus")
require.NoError(t, err)
require.NotEmpty(t, entries)
for _, entry := range entries {
if entry.IsDir() {
continue
}
data, err := os.ReadFile(filepath.Join("testdata/gofuzz-corpus", entry.Name()))
require.NoError(t, err)
require.Truef(t, fuzzOpenAPIParsing(data), "seed %s should produce at least one request", entry.Name())
}
}
func TestFuzzOpenAPIParsingRejectsOversizeInput(t *testing.T) {
require.False(t, fuzzOpenAPIParsing(make([]byte, fuzzMaxInputSize+1)))
}
// Null entries decode to nil pointers and used to panic during request
// generation. The remaining valid entries must still produce requests.
func TestFuzzOpenAPIParsingNullEntries(t *testing.T) {
specs := map[string]string{
"null server": `
openapi: 3.0.0
info: {title: t, version: "1"}
servers:
-
- {url: "http://example.com"}
paths:
/users:
get:
responses: {"200": {description: ok}}
`,
"null media type": `
openapi: 3.0.0
info: {title: t, version: "1"}
servers: [{url: "http://example.com"}]
paths:
/users:
post:
requestBody:
content:
application/xml:
application/json:
schema: {type: object, properties: {name: {type: string}}}
responses: {"200": {description: ok}}
`,
}
for name, spec := range specs {
t.Run(name, func(t *testing.T) {
require.True(t, fuzzOpenAPIParsing([]byte(spec)))
})
}
}
func TestGenerateRequestsOnlyNullServers(t *testing.T) {
spec := `
openapi: 3.0.0
info: {title: t, version: "1"}
servers:
-
paths:
/users:
get:
responses: {"200": {description: ok}}
`
err := New().Parse(strings.NewReader(spec), func(*types.RequestResponse) bool { return false }, "spec.yaml")
require.ErrorContains(t, err, "no servers found in openapi schema")
}
+14 -1
View File
@@ -9,6 +9,7 @@ import (
"net/http/httputil"
"net/url"
"os"
"slices"
"strings"
"github.com/clbanning/mxj/v2"
@@ -33,7 +34,9 @@ const (
// GenerateRequestsFromSchema generates http requests from an OpenAPI 3.0 document object
func GenerateRequestsFromSchema(schema *openapi3.T, opts formats.InputFormatOptions, callback formats.ParseReqRespCallback) error {
if len(schema.Servers) == 0 {
// Null entries in `servers:` are skipped below, so a list of only null
// entries has no usable server, the same as an empty list.
if !slices.ContainsFunc(schema.Servers, func(s *openapi3.Server) bool { return s != nil }) {
return errors.New("no servers found in openapi schema")
}
globalParams, globalSecurityErr := selectSecurityParameters(schema, &schema.Security, opts.Variables)
@@ -49,6 +52,11 @@ func GenerateRequestsFromSchema(schema *openapi3.T, opts formats.InputFormatOpti
}
for _, serverURL := range schema.Servers {
// A null entry in the `servers:` list decodes to a nil *Server, so skip
// it rather than dereferencing it.
if serverURL == nil {
continue
}
pathURL := serverURL.URL
// Split the server URL into baseURL and serverPath
u, err := url.Parse(pathURL)
@@ -268,6 +276,11 @@ func generateRequestsFromOp(opts *generateReqOptions) error {
if opts.op.RequestBody != nil {
for content, value := range opts.op.RequestBody.Value.Content {
// A null media-type entry (e.g. `content: {application/json: }`)
// decodes to a nil *MediaType, so skip it before dereferencing.
if value == nil {
continue
}
cloned := req.Clone(req.Context())
var val interface{}
@@ -0,0 +1 @@
{"openapi":"3.0.0","info":{"title":"t","version":"1"},"servers":[{"url":"http://example.com"}],"paths":{"/a":{"post":{"requestBody":{"content":{"application/json":{"schema":{"type":"object","properties":{"k":{"type":"string"}}}}}},"responses":{"200":{"description":"ok"}}}}}}
@@ -0,0 +1,7 @@
openapi: 3.0.0
info: {title: t, version: "1"}
servers: [{url: "http://example.com"}]
paths:
/users:
get:
responses: {"200": {description: ok}}
@@ -0,0 +1,12 @@
openapi: 3.0.0
info: {title: t, version: "1"}
servers: [{url: "http://example.com"}]
paths:
/users:
post:
requestBody:
content:
application/xml:
application/json:
schema: {type: object, properties: {name: {type: string}}}
responses: {"200": {description: ok}}
@@ -0,0 +1,9 @@
openapi: 3.0.0
info: {title: t, version: "1"}
servers:
-
- {url: "http://example.com"}
paths:
/users:
get:
responses: {"200": {description: ok}}
+29
View File
@@ -3,7 +3,9 @@ package swagger
import (
"fmt"
"io"
"maps"
"path"
"slices"
"github.com/getkin/kin-openapi/openapi2"
"github.com/getkin/kin-openapi/openapi2conv"
@@ -55,6 +57,7 @@ func (j *SwaggerFormat) Parse(input io.Reader, resultsCb formats.ParseReqRespCal
if err != nil {
return errors.Wrap(err, "could not decode openapi 2.0 schema")
}
dropNullEntries(schemav2)
schema, err := openapi2conv.ToV3(schemav2)
if err != nil {
return errors.Wrap(err, "could not convert openapi 2.0 schema to 3.0")
@@ -67,6 +70,32 @@ func (j *SwaggerFormat) Parse(input io.Reader, resultsCb formats.ParseReqRespCal
return openapi.GenerateRequestsFromSchema(schema, j.opts, resultsCb)
}
// dropNullEntries removes null entries from the maps and lists of a decoded
// OpenAPI 2.0 document. A null value such as `paths: {/x: }` or an empty list
// item under `parameters:` decodes to a nil pointer, which openapi2conv.ToV3
// dereferences without checking.
func dropNullEntries(doc *openapi2.T) {
maps.DeleteFunc(doc.Paths, func(_ string, v *openapi2.PathItem) bool { return v == nil })
maps.DeleteFunc(doc.Parameters, func(_ string, v *openapi2.Parameter) bool { return v == nil })
dropNullResponses(doc.Responses)
for _, pathItem := range doc.Paths {
pathItem.Parameters = slices.DeleteFunc(pathItem.Parameters, func(v *openapi2.Parameter) bool { return v == nil })
for _, op := range pathItem.Operations() {
op.Parameters = slices.DeleteFunc(op.Parameters, func(v *openapi2.Parameter) bool { return v == nil })
dropNullResponses(op.Responses)
}
}
}
// dropNullResponses removes null responses, and null headers of the remaining
// responses, which openapi2conv.ToV3Response also dereferences.
func dropNullResponses(responses map[string]*openapi2.Response) {
maps.DeleteFunc(responses, func(_ string, v *openapi2.Response) bool { return v == nil })
for _, response := range responses {
maps.DeleteFunc(response.Headers, func(_ string, v *openapi2.Header) bool { return v == nil })
}
}
func decodeYAML(data []byte, target interface{}) error {
var value interface{}
if err := yaml.Unmarshal(data, &value); err != nil {
+67
View File
@@ -2,6 +2,7 @@ package swagger
import (
"os"
"strings"
"testing"
"github.com/projectdiscovery/nuclei/v3/pkg/input/types"
@@ -39,3 +40,69 @@ func TestSwaggerAPIParser(t *testing.T) {
}
require.ElementsMatch(t, gotMethodsToURLs, expectedURLs, "could not get swagger urls")
}
func TestSwaggerParseNullEntries(t *testing.T) {
// Each spec has a null map value or list item that decodes to a nil
// pointer and used to panic inside openapi2conv.ToV3.
specs := map[string]string{
"null-path-item": `
swagger: "2.0"
info: {title: t, version: "1"}
host: localhost
paths:
/x:
`,
"null-parameter": `
swagger: "2.0"
info: {title: t, version: "1"}
host: localhost
paths:
/x:
get:
parameters:
-
responses: {"200": {description: ok}}
`,
"null-response": `
swagger: "2.0"
info: {title: t, version: "1"}
host: localhost
paths:
/x:
get:
responses: {"200": }
`,
"null-response-header": `
swagger: "2.0"
info: {title: t, version: "1"}
host: localhost
responses:
Shared:
description: ok
headers:
X-Shared:
paths:
/x:
get:
responses:
"200":
description: ok
headers:
X-Rate-Limit:
`,
}
for name, spec := range specs {
t.Run(name, func(t *testing.T) {
var count int
err := New().Parse(strings.NewReader(spec), func(request *types.RequestResponse) bool {
count++
return false
}, "spec.yaml")
require.NoError(t, err)
if name != "null-path-item" {
require.Equal(t, 1, count, "the operation should still produce a request")
}
})
}
}
+71
View File
@@ -0,0 +1,71 @@
{
"log": {
"version": "1.2",
"creator": { "name": "WebInspector", "version": "537.36" },
"entries": [
{
"request": {
"method": "GET",
"url": "https://ginandjuice.shop/catalog?category=Juice",
"httpVersion": "http/2.0",
"headers": [
{ "name": ":authority", "value": "ginandjuice.shop" },
{ "name": ":method", "value": "GET" },
{ "name": ":path", "value": "/catalog?category=Juice" },
{ "name": ":scheme", "value": "https" },
{ "name": "accept", "value": "text/html" },
{ "name": "cookie", "value": "session=abc123" }
],
"queryString": [{ "name": "category", "value": "Juice" }],
"cookies": [],
"headersSize": -1,
"bodySize": 0
},
"response": {
"status": 200,
"statusText": "",
"httpVersion": "http/2.0",
"headers": [{ "name": "content-type", "value": "text/html; charset=utf-8" }],
"content": { "size": 19, "mimeType": "text/html", "text": "PGgxPkp1aWNlPC9oMT4=", "encoding": "base64" }
}
},
{
"request": {
"method": "POST",
"url": "https://ginandjuice.shop/login",
"httpVersion": "HTTP/1.1",
"headers": [
{ "name": "Host", "value": "ginandjuice.shop" },
{ "name": "Content-Type", "value": "application/x-www-form-urlencoded" }
],
"postData": { "mimeType": "application/x-www-form-urlencoded", "text": "username=carlos&password=hunter2" }
},
"response": {
"status": 302,
"statusText": "Found",
"httpVersion": "HTTP/1.1",
"headers": [{ "name": "Location", "value": "/my-account" }],
"content": { "size": 0, "mimeType": "" }
}
},
{
"request": {
"method": "GET",
"url": "data:image/png;base64,iVBORw0KGgo=",
"httpVersion": "",
"headers": []
},
"response": { "status": 200, "statusText": "", "httpVersion": "", "headers": [], "content": { "size": 0 } }
},
{
"request": {
"method": "GET",
"url": "https://ginandjuice.shop/blocked.js",
"httpVersion": "",
"headers": []
},
"response": { "status": 0, "statusText": "", "httpVersion": "", "headers": [], "content": { "size": 0 } }
}
]
}
}
+2
View File
@@ -10,6 +10,7 @@ import (
"github.com/projectdiscovery/gologger"
"github.com/projectdiscovery/nuclei/v3/pkg/input/formats"
"github.com/projectdiscovery/nuclei/v3/pkg/input/formats/burp"
"github.com/projectdiscovery/nuclei/v3/pkg/input/formats/har"
"github.com/projectdiscovery/nuclei/v3/pkg/input/formats/json"
"github.com/projectdiscovery/nuclei/v3/pkg/input/formats/openapi"
"github.com/projectdiscovery/nuclei/v3/pkg/input/formats/raw"
@@ -142,6 +143,7 @@ func (i *HttpInputProvider) Close() {}
// Supported Providers
var providersList = []formats.Format{
burp.New(),
har.New(),
json.New(),
yaml.New(),
openapi.New(),
+66 -138
View File
@@ -89,8 +89,9 @@ func New(opts *Options) (*ListInputProvider, error) {
patterns: normalize.NewFingerprinter(options.MaxURLsPerPattern),
ipOptions: &ipOptions{
ScanAllIPs: options.ScanAllIPs,
IPV4: sliceutil.Contains(options.IPVersion, "4"),
IPV6: sliceutil.Contains(options.IPVersion, "6"),
IPV4: len(options.IPVersion) == 0 || sliceutil.Contains(options.IPVersion, "4"),
IPV6: len(options.IPVersion) == 0 || sliceutil.Contains(options.IPVersion, "6"),
Auto: len(options.IPVersion) == 0,
},
excludedHosts: make(map[string]struct{}),
profiles: targetprofile.NewRegistry(configuration.TemplatesDirectory(options, config.DefaultConfig)),
@@ -168,6 +169,61 @@ func (i *ListInputProvider) TargetProfiles() *targetprofile.Registry {
return i.profiles
}
// selectedIPs returns address pins for a hostname. An empty pin delegates
// DNS and connection fallback to fastdialer, which races IPv4 and IPv6.
// Set and Del share this selection. With -iv omitted and without -sa, one
// unpinned target is returned. -sa returns every selected address.
func (i *ListInputProvider) selectedIPs(executionId, hostname string) []string {
if i.ipOptions == nil || (!i.ipOptions.ScanAllIPs && (i.ipOptions.Auto || !i.ipOptions.IPV6)) {
return []string{""}
}
dialers := protocolstate.GetDialersWithId(executionId)
if dialers == nil {
gologger.Warning().Msgf("cannot resolve %s: dialer %s not found", hostname, executionId)
return nil
}
dnsData, err := dialers.Fastdialer.GetDNSData(hostname)
if err != nil {
gologger.Warning().Msgf("DNS resolution failed for %s: %v", hostname, err)
if i.ipOptions.IPV4 {
return []string{""}
}
return nil
}
if len(dnsData.A) == 0 && len(dnsData.AAAA) == 0 {
gologger.Warning().Msgf("DNS resolution returned no IP addresses for %s", hostname)
if i.ipOptions.IPV4 {
return []string{""}
}
return nil
}
if i.ipOptions.ScanAllIPs {
var ips []string
if i.ipOptions.IPV6 {
ips = append(ips, dnsData.AAAA...)
}
if i.ipOptions.IPV4 {
ips = append(ips, dnsData.A...)
}
if len(ips) == 0 {
gologger.Warning().Msgf("no DNS addresses matching -iv selection for %s", hostname)
}
return ips
}
var ips []string
if i.ipOptions.IPV6 && len(dnsData.AAAA) > 0 {
ips = append(ips, dnsData.AAAA[0])
}
// Explicit -iv 4,6 scans the first IPv6 address and one unpinned target.
if i.ipOptions.IPV4 {
ips = append(ips, "")
}
if len(ips) == 0 {
gologger.Warning().Msgf("no IPv6 DNS address found for %s", hostname)
}
return ips
}
// Set normalizes and stores passed input values
func (i *ListInputProvider) Set(executionId string, value string) {
i.set(executionId, value, nil)
@@ -202,75 +258,11 @@ func (i *ListInputProvider) set(executionId string, value string, selection *tar
return
}
if i.ipOptions.ScanAllIPs {
// scan all ips
dialers := protocolstate.GetDialersWithId(executionId)
if dialers == nil {
gologger.Error().Msgf("dialers with executionId %s not found", executionId)
return
}
dnsData, err := dialers.Fastdialer.GetDNSData(urlx.Hostname())
if err == nil {
if (len(dnsData.A) + len(dnsData.AAAA)) > 0 {
var ips []string
if i.ipOptions.IPV4 {
ips = append(ips, dnsData.A...)
}
if i.ipOptions.IPV6 {
ips = append(ips, dnsData.AAAA...)
}
for _, ip := range ips {
if ip == "" {
continue
}
metaInput := contextargs.NewMetaInput()
metaInput.Input = URL
metaInput.CustomIP = ip
i.setItem(metaInput, selection)
}
return
} else {
gologger.Debug().Msgf("scanAllIps: no ip's found reverting to default")
}
} else {
// failed to scanallips falling back to defaults
gologger.Debug().Msgf("scanAllIps: dns resolution failed: %v", err)
}
}
ips := []string{}
// only scan the target but ipv6 if it has one
if i.ipOptions.IPV6 {
dialers := protocolstate.GetDialersWithId(executionId)
if dialers == nil {
gologger.Error().Msgf("dialers with executionId %s not found", executionId)
return
}
dnsData, err := dialers.Fastdialer.GetDNSData(urlx.Hostname())
if err == nil && len(dnsData.AAAA) > 0 {
// pick/ prefer 1st
ips = append(ips, dnsData.AAAA[0])
} else {
gologger.Warning().Msgf("target does not have ipv6 address falling back to ipv4 %v\n", err)
}
}
if i.ipOptions.IPV4 {
// if IPV4 is enabled do not specify ip let dialer handle it
ips = append(ips, "")
}
for _, ip := range ips {
for _, ip := range i.selectedIPs(executionId, urlx.Hostname()) {
metaInput := contextargs.NewMetaInput()
if ip != "" {
metaInput.Input = URL
metaInput.CustomIP = ip
i.setItem(metaInput, selection)
} else {
metaInput.Input = URL
i.setItem(metaInput, selection)
}
metaInput.Input = URL
metaInput.CustomIP = ip
i.setItem(metaInput, selection)
}
}
@@ -493,75 +485,11 @@ func (i *ListInputProvider) Del(executionId string, value string) {
return
}
if i.ipOptions.ScanAllIPs {
// scan all ips
dialers := protocolstate.GetDialersWithId(executionId)
if dialers == nil {
gologger.Error().Msgf("dialers with executionId %s not found", executionId)
return
}
dnsData, err := dialers.Fastdialer.GetDNSData(urlx.Hostname())
if err == nil {
if (len(dnsData.A) + len(dnsData.AAAA)) > 0 {
var ips []string
if i.ipOptions.IPV4 {
ips = append(ips, dnsData.A...)
}
if i.ipOptions.IPV6 {
ips = append(ips, dnsData.AAAA...)
}
for _, ip := range ips {
if ip == "" {
continue
}
metaInput := contextargs.NewMetaInput()
metaInput.Input = value
metaInput.CustomIP = ip
i.delItem(metaInput)
}
return
} else {
gologger.Debug().Msgf("scanAllIps: no ip's found reverting to default")
}
} else {
// failed to scanallips falling back to defaults
gologger.Debug().Msgf("scanAllIps: dns resolution failed: %v", err)
}
}
ips := []string{}
// only scan the target but ipv6 if it has one
if i.ipOptions.IPV6 {
dialers := protocolstate.GetDialersWithId(executionId)
if dialers == nil {
gologger.Error().Msgf("dialers with executionId %s not found", executionId)
return
}
dnsData, err := dialers.Fastdialer.GetDNSData(urlx.Hostname())
if err == nil && len(dnsData.AAAA) > 0 {
// pick/ prefer 1st
ips = append(ips, dnsData.AAAA[0])
} else {
gologger.Warning().Msgf("target does not have ipv6 address falling back to ipv4 %v\n", err)
}
}
if i.ipOptions.IPV4 {
// if IPV4 is enabled do not specify ip let dialer handle it
ips = append(ips, "")
}
for _, ip := range ips {
for _, ip := range i.selectedIPs(executionId, urlx.Hostname()) {
metaInput := contextargs.NewMetaInput()
if ip != "" {
metaInput.Input = URL
metaInput.CustomIP = ip
i.delItem(metaInput)
} else {
metaInput.Input = URL
i.delItem(metaInput)
}
metaInput.Input = URL
metaInput.CustomIP = ip
i.delItem(metaInput)
}
}
+106
View File
@@ -0,0 +1,106 @@
package list
import (
"net"
"strings"
"testing"
"time"
"github.com/miekg/dns"
"github.com/projectdiscovery/hmap/store/hybrid"
"github.com/projectdiscovery/nuclei/v3/pkg/protocols/common/contextargs"
"github.com/projectdiscovery/nuclei/v3/pkg/protocols/common/protocolstate"
"github.com/projectdiscovery/nuclei/v3/pkg/types"
"github.com/stretchr/testify/require"
)
// A local deterministic DNS responder for dual-stack, A-only and AAAA-only hosts.
type ipVersionDNSHandler struct{}
func (ipVersionDNSHandler) ServeDNS(w dns.ResponseWriter, r *dns.Msg) {
reply := new(dns.Msg)
reply.SetReply(r)
if len(r.Question) == 0 {
_ = w.WriteMsg(reply)
return
}
question := r.Question[0]
// Some resolvers try search suffixes before the absolute name. Match the
// test host prefix rather than answering unrelated names as dual-stack.
host := strings.ToLower(question.Name)
if strings.HasPrefix(host, "empty.") {
_ = w.WriteMsg(reply)
return
}
isDual := strings.HasPrefix(host, "dual.")
isV4 := strings.HasPrefix(host, "v4only.")
isV6 := strings.HasPrefix(host, "v6only.")
if question.Qtype == dns.TypeA && (isDual || isV4) {
reply.Answer = append(reply.Answer, &dns.A{Hdr: dns.RR_Header{Name: question.Name, Rrtype: dns.TypeA, Class: dns.ClassINET, Ttl: 60}, A: net.ParseIP("192.0.2.10")})
}
if question.Qtype == dns.TypeAAAA && (isDual || isV6) {
reply.Answer = append(reply.Answer, &dns.AAAA{Hdr: dns.RR_Header{Name: question.Name, Rrtype: dns.TypeAAAA, Class: dns.ClassINET, Ttl: 60}, AAAA: net.ParseIP("2001:db8::10")})
}
_ = w.WriteMsg(reply)
}
func TestIPVersionsDefaultAndExplicitSelection(t *testing.T) {
conn, err := net.ListenPacket("udp", "127.0.0.1:0")
require.NoError(t, err)
started := make(chan struct{})
server := &dns.Server{PacketConn: conn, Handler: ipVersionDNSHandler{}, NotifyStartedFunc: func() { close(started) }}
go func() { _ = server.ActivateAndServe() }()
t.Cleanup(func() { _ = server.Shutdown() })
select {
case <-started:
case <-time.After(5 * time.Second):
t.Fatal("DNS test resolver did not start")
}
opts := types.DefaultOptions()
opts.ExecutionId = "ip-version-regression"
opts.InternalResolversList = []string{conn.LocalAddr().String()}
require.NoError(t, protocolstate.Init(opts))
t.Cleanup(func() { protocolstate.Close(opts.ExecutionId) })
for _, tc := range []struct {
name, hostname string
scanAll, ipv4, ipv6, auto bool
want []string
}{
{"automatic dual-stack keeps dialer fallback", "dual.test", false, true, true, true, []string{""}},
{"automatic IPv6-only keeps dialer fallback", "v6only.test", false, true, true, true, []string{""}},
{"auto IPv4 fallback", "v4only.test", false, true, true, true, []string{""}},
{"auto empty DNS response fallback", "empty.test", false, true, true, true, []string{""}},
{"all empty DNS response fallback", "empty.test", true, true, true, true, []string{""}},
{"all dual", "dual.test", true, true, true, true, []string{"192.0.2.10", "2001:db8::10"}},
{"all IPv6 only", "v6only.test", true, true, true, true, []string{"2001:db8::10"}},
{"all IPv4 only", "v4only.test", true, true, true, true, []string{"192.0.2.10"}},
{"explicit IPv4 only", "dual.test", false, true, false, false, []string{""}},
{"explicit IPv6 only", "dual.test", false, false, true, false, []string{"2001:db8::10"}},
{"explicit both", "dual.test", false, true, true, false, []string{"2001:db8::10", ""}},
{"all explicit IPv6", "dual.test", true, false, true, false, []string{"2001:db8::10"}},
{"all explicit IPv4", "dual.test", true, true, false, false, []string{"192.0.2.10"}},
{"IPv6 restricted on IPv4-only", "v4only.test", true, false, true, false, nil},
} {
t.Run(tc.name, func(t *testing.T) {
hm, err := hybrid.New(hybrid.DefaultDiskOptions)
require.NoError(t, err)
input := &ListInputProvider{
hostMap: hm,
excludedHosts: make(map[string]struct{}),
ipOptions: &ipOptions{ScanAllIPs: tc.scanAll, IPV4: tc.ipv4, IPV6: tc.ipv6, Auto: tc.auto},
}
t.Cleanup(input.Close)
input.Set(opts.ExecutionId, "https://"+tc.hostname)
var got []string
input.Iterate(func(value *contextargs.MetaInput) bool {
got = append(got, value.CustomIP)
return true
})
require.ElementsMatch(t, tc.want, got)
if tc.scanAll && len(tc.want) > 0 {
input.Del(opts.ExecutionId, "https://"+tc.hostname)
require.EqualValues(t, 0, input.Count(), "Del must remove the same IPs added by Set")
}
})
}
}
+1
View File
@@ -4,4 +4,5 @@ type ipOptions struct {
ScanAllIPs bool
IPV4 bool
IPV6 bool
Auto bool // -iv omitted: one unpinned target unless -sa scans every address
}
+2
View File
@@ -16,10 +16,12 @@ import (
syncutil "github.com/projectdiscovery/utils/sync"
_ "github.com/projectdiscovery/nuclei/v3/pkg/js/generated/go/libbytes"
_ "github.com/projectdiscovery/nuclei/v3/pkg/js/generated/go/libcompress"
_ "github.com/projectdiscovery/nuclei/v3/pkg/js/generated/go/libdcerpc"
_ "github.com/projectdiscovery/nuclei/v3/pkg/js/generated/go/libdcom"
_ "github.com/projectdiscovery/nuclei/v3/pkg/js/generated/go/libfs"
_ "github.com/projectdiscovery/nuclei/v3/pkg/js/generated/go/libgrpc"
_ "github.com/projectdiscovery/nuclei/v3/pkg/js/generated/go/libhash"
_ "github.com/projectdiscovery/nuclei/v3/pkg/js/generated/go/libhttp"
_ "github.com/projectdiscovery/nuclei/v3/pkg/js/generated/go/libikev2"
_ "github.com/projectdiscovery/nuclei/v3/pkg/js/generated/go/libkerberos"
+158
View File
@@ -0,0 +1,158 @@
package compiler
import (
"bytes"
"fmt"
"io"
"math/rand"
"net/http"
"net/http/httptest"
"net/url"
"strconv"
"testing"
"time"
"github.com/andybalholm/brotli"
"github.com/klauspost/compress/gzip"
"github.com/klauspost/compress/zlib"
"github.com/klauspost/compress/zstd"
"github.com/projectdiscovery/dsl"
"github.com/projectdiscovery/nuclei/v3/pkg/protocols/common/protocolstate"
"github.com/projectdiscovery/nuclei/v3/pkg/types"
httputil "github.com/projectdiscovery/utils/http"
"github.com/stretchr/testify/require"
)
const streamHashScript = `
const net = require('nuclei/net');
const compress = require('nuclei/compress');
const hash = require('nuclei/hash');
const conn = net.Open('tcp', %q);
conn.Send('GET / HTTP/1.1\r\nHost: test\r\nAccept-Encoding: gzip, deflate, br, zstd\r\n\r\n');
let head = '';
while (!head.endsWith('\r\n\r\n')) head += conn.RecvString(1);
const header = (name) => (head.match(new RegExp('\r\n' + name + ': ([^\r]+)', 'i')) || [])[1];
const length = header('Content-Length');
const coding = [header('Content-Encoding'), header('Transfer-Encoding')].filter(Boolean).join(', ');
const d = compress.NewDecoder(coding);
const sha = hash.New('sha256'), mmh3 = hash.New('mmh3');
const take = (out) => { sha.Write(out); mmh3.Write(out); };
if (length !== undefined) {
for (let read = 0; read < parseInt(length); ) {
const chunk = conn.Recv(65536);
read += chunk.length;
for (let out = d.Write(chunk); out.length > 0; out = d.Read()) take(out);
}
for (let out = d.Close(); out.length > 0; out = d.Read()) take(out);
} else {
while (!d.Finished()) {
for (let out = d.Write(conn.Recv(65536)); out.length > 0; out = d.Read()) take(out);
}
}
ExportAs('sha256', sha.Sum());
ExportAs('mmh3', mmh3.Sum());
true;
`
func encodeBody(t *testing.T, encoding string, data []byte) []byte {
t.Helper()
var buf bytes.Buffer
var w io.WriteCloser
switch encoding {
case "gzip":
w = gzip.NewWriter(&buf)
case "deflate":
w = zlib.NewWriter(&buf)
case "br":
w = brotli.NewWriter(&buf)
case "zstd":
zw, err := zstd.NewWriter(&buf)
require.NoError(t, err)
w = zw
case "":
return data
}
_, err := w.Write(data)
require.NoError(t, err)
require.NoError(t, w.Close())
return buf.Bytes()
}
func TestStreamingHashMatchesHTTPEngine(t *testing.T) {
rng := rand.New(rand.NewSource(1))
var body []byte
for len(body) < 6<<20 {
body = append(body, "large compressed response body "...)
body = append(body, byte(rng.Intn(256)))
}
executionID := t.Name()
require.NoError(t, protocolstate.Init(&types.Options{ExecutionId: executionID}))
t.Cleanup(func() { protocolstate.Close(executionID) })
for _, tc := range []struct {
encoding string
chunked bool
}{
{"gzip", false},
{"gzip", true},
{"deflate", false},
{"br", true},
{"zstd", true},
{"", true},
} {
t.Run(fmt.Sprintf("%s chunked=%v", tc.encoding, tc.chunked), func(t *testing.T) {
encoded := encodeBody(t, tc.encoding, body)
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if tc.encoding != "" {
w.Header().Set("Content-Encoding", tc.encoding)
}
if !tc.chunked {
w.Header().Set("Content-Length", strconv.Itoa(len(encoded)))
_, _ = w.Write(encoded)
return
}
// Flushing before the handler returns makes the body chunked.
for rest := encoded; len(rest) > 0; {
n := min(len(rest), 50000)
_, _ = w.Write(rest[:n])
w.(http.Flusher).Flush()
rest = rest[n:]
}
}))
defer server.Close()
client := &http.Client{Transport: &http.Transport{DisableCompression: true}}
resp, err := client.Get(server.URL)
require.NoError(t, err)
if tc.chunked {
require.Equal(t, []string{"chunked"}, resp.TransferEncoding)
}
chain := httputil.NewResponseChain(resp, 64<<20)
defer chain.Close()
require.NoError(t, chain.Fill())
require.Equal(t, len(body), chain.Body().Len(), "the engine must decode the whole body")
engineBody := map[string]interface{}{"body": chain.BodyString()}
wantSha, err := dsl.EvalExpr("sha256(body)", engineBody)
require.NoError(t, err)
wantMmh3, err := dsl.EvalExpr("mmh3(body)", engineBody)
require.NoError(t, err)
serverURL, err := url.Parse(server.URL)
require.NoError(t, err)
script := fmt.Sprintf(streamHashScript, serverURL.Host)
compiled, err := SourceAutoMode(script, false)
require.NoError(t, err)
result, err := New().ExecuteWithOptions(t.Context(), compiled, NewExecuteArgs(), &ExecuteOptions{
ExecutionId: executionID,
Source: &script,
TimeoutVariants: &types.Timeouts{JsCompilerExecutionTimeout: 20 * time.Second},
})
require.NoError(t, err)
require.Equal(t, fmt.Sprint(wantSha), result["sha256"])
require.Equal(t, fmt.Sprint(wantMmh3), result["mmh3"])
})
}
}
@@ -0,0 +1,30 @@
package compress
import (
lib_compress "github.com/projectdiscovery/nuclei/v3/pkg/js/libs/compress"
"github.com/projectdiscovery/goja"
"github.com/projectdiscovery/nuclei/v3/pkg/js/gojs"
)
var (
module = gojs.NewGojaModule("nuclei/compress")
)
func init() {
module.Set(
gojs.Objects{
// Functions
"NewDecoder": lib_compress.NewDecoder,
// Var and consts
// Objects / Classes
"Decoder": gojs.GetClassConstructor[lib_compress.Decoder](&lib_compress.Decoder{}),
},
).Register()
}
func Enable(runtime *goja.Runtime) {
module.Enable(runtime)
}
+30
View File
@@ -0,0 +1,30 @@
package hash
import (
lib_hash "github.com/projectdiscovery/nuclei/v3/pkg/js/libs/hash"
"github.com/projectdiscovery/goja"
"github.com/projectdiscovery/nuclei/v3/pkg/js/gojs"
)
var (
module = gojs.NewGojaModule("nuclei/hash")
)
func init() {
module.Set(
gojs.Objects{
// Functions
"New": lib_hash.New,
// Var and consts
// Objects / Classes
"Hash": gojs.GetClassConstructor[lib_hash.Hash](&lib_hash.Hash{}),
},
).Register()
}
func Enable(runtime *goja.Runtime) {
module.Enable(runtime)
}
+102
View File
@@ -0,0 +1,102 @@
/**
* NewDecoder returns a decoder for gzip, deflate, br, zstd, chunked or
* identity. List several comma-separated, in the order they were applied.
* @example
* ```javascript
* const compress = require('nuclei/compress');
* const d = compress.NewDecoder('br');
* // Content-Encoding: gzip sent with Transfer-Encoding: chunked
* const c = compress.NewDecoder('gzip, chunked');
* ```
*/
export function NewDecoder(encoding: string): Decoder | null {
return null;
}
/**
* Decoder decodes a compressed stream written to it in chunks. Each call
* returns up to about 1 MB; call Read until it returns nothing before the
* next Write.
* @example
* ```javascript
* const compress = require('nuclei/compress');
* const hash = require('nuclei/hash');
* const d = compress.NewDecoder('gzip');
* const h = hash.New('sha256');
* for (const chunk of chunks) {
* for (let out = d.Write(chunk); out.length > 0; out = d.Read()) h.Write(out);
* }
* for (let out = d.Close(); out.length > 0; out = d.Read()) h.Write(out);
* log(h.Sum());
* ```
*/
export class Decoder {
// Constructor of Decoder
constructor() {}
/**
* Write decodes the next chunk and returns up to about 1 MB of output.
* @example
* ```javascript
* const compress = require('nuclei/compress');
* const d = compress.NewDecoder('gzip');
* const out = d.Write(chunk);
* ```
*/
public Write(chunk: Uint8Array): Uint8Array | null {
return null;
}
/**
* Read returns more decoded bytes, or none once the decoder needs input.
* @example
* ```javascript
* const compress = require('nuclei/compress');
* const d = compress.NewDecoder('gzip');
* for (let out = d.Write(chunk); out.length > 0; out = d.Read()) log(out.length);
* ```
*/
public Read(): Uint8Array | null {
return null;
}
/**
* Close ends the input and returns the last decoded bytes. A truncated
* stream errors once the bytes before the cut have been returned.
* @example
* ```javascript
* const compress = require('nuclei/compress');
* const d = compress.NewDecoder('gzip');
* d.Write(chunk);
* for (let out = d.Close(); out.length > 0; out = d.Read()) log(out.length);
* ```
*/
public Close(): Uint8Array | null {
return null;
}
/**
* Finished reports whether a stream that marks its own end, such as a
* chunked body, has ended and all its output was read.
* @example
* ```javascript
* const compress = require('nuclei/compress');
* const d = compress.NewDecoder('gzip, chunked');
* while (!d.Finished()) for (let out = d.Write(conn.Recv(65536)); out.length > 0; out = d.Read()) log(out.length);
* ```
*/
public Finished(): boolean {
return false;
}
}
+77
View File
@@ -0,0 +1,77 @@
/**
* New returns a running hash for md5, sha1, sha256, sha512 or mmh3. Sum
* matches the DSL function of the same name.
* @example
* ```javascript
* const hash = require('nuclei/hash');
* const h = hash.New('sha256');
* ```
*/
export function New(algorithm: string): Hash | null {
return null;
}
/**
* Hash is a running hash.
* @example
* ```javascript
* const hash = require('nuclei/hash');
* const h = hash.New('sha256');
* h.WriteString('hello');
* log(h.Sum());
* ```
*/
export class Hash {
// Constructor of Hash
constructor() {}
/**
* Write adds data to the hash.
* @example
* ```javascript
* const hash = require('nuclei/hash');
* const h = hash.New('sha256');
* h.Write([104, 105]);
* ```
*/
public Write(data: Uint8Array): void {
return;
}
/**
* WriteString adds the string data to the hash.
* @example
* ```javascript
* const hash = require('nuclei/hash');
* const h = hash.New('sha256');
* h.WriteString('hi');
* ```
*/
public WriteString(data: string): void {
return;
}
/**
* Sum returns the hash of the data written so far, formatted as the DSL does.
* @example
* ```javascript
* const hash = require('nuclei/hash');
* const h = hash.New('mmh3');
* h.WriteString('hello');
* log(h.Sum());
* ```
*/
public Sum(): string | null {
return null;
}
}
+2
View File
@@ -1,9 +1,11 @@
export * as bytes from './bytes';
export * as compress from './compress';
export * as dcom from './dcom';
export * as dcerpc from './dcerpc';
export * as fs from './fs';
export * as goconsole from './goconsole';
export * as grpc from './grpc';
export * as hash from './hash';
export * as http from './http';
export * as ikev2 from './ikev2';
export * as kerberos from './kerberos';
+303
View File
@@ -0,0 +1,303 @@
// Package compress exposes streaming decompression to nuclei JavaScript templates.
package compress
import (
"bytes"
"context"
"errors"
"fmt"
"io"
"net/http/httputil"
"slices"
"strings"
"github.com/dsnet/compress/brotli"
"github.com/klauspost/compress/gzip"
"github.com/klauspost/compress/zlib"
"github.com/klauspost/compress/zstd"
)
// maxDecodedChunk caps the decoded bytes one call hands back.
const maxDecodedChunk = 1 << 20
type pauseReason int
const (
needInput pauseReason = iota
outputFull
finished
)
type (
// Decoder decodes a compressed stream written to it in chunks. Each call
// returns up to about 1 MB; call Read until it returns nothing before the
// next Write.
// @example
// ```javascript
// const compress = require('nuclei/compress');
// const hash = require('nuclei/hash');
// const d = compress.NewDecoder('gzip');
// const h = hash.New('sha256');
// for (const chunk of chunks) {
// for (let out = d.Write(chunk); out.length > 0; out = d.Read()) h.Write(out);
// }
// for (let out = d.Close(); out.length > 0; out = d.Read()) h.Write(out);
// log(h.Sum());
// ```
Decoder struct {
ctx context.Context
// The decode goroutine and the template's calls take turns over these
// channels, so no field is accessed by both at once.
resume chan []byte
paused chan pauseReason
done chan struct{}
out []byte
state pauseReason
inputEnded bool
pending []byte
eof bool
err error
}
)
// NewDecoder returns a decoder for gzip, deflate, br, zstd, chunked or
// identity. List several comma-separated, in the order they were applied.
// @example
// ```javascript
// const compress = require('nuclei/compress');
// const d = compress.NewDecoder('br');
// // Content-Encoding: gzip sent with Transfer-Encoding: chunked
// const c = compress.NewDecoder('gzip, chunked');
// ```
func NewDecoder(ctx context.Context, encoding string) (*Decoder, error) {
var opens []func(io.Reader) (io.Reader, func(), error)
for name := range strings.SplitSeq(encoding, ",") {
switch name = strings.ToLower(strings.TrimSpace(name)); name {
case "gzip":
opens = append(opens, func(r io.Reader) (io.Reader, func(), error) {
zr, err := gzip.NewReader(r)
return zr, func() {}, err
})
case "deflate":
opens = append(opens, func(r io.Reader) (io.Reader, func(), error) {
zr, err := zlib.NewReader(r)
return zr, func() {}, err
})
case "br":
opens = append(opens, func(r io.Reader) (io.Reader, func(), error) {
br, err := brotli.NewReader(r, nil)
return br, func() {}, err
})
case "zstd":
opens = append(opens, func(r io.Reader) (io.Reader, func(), error) {
// One goroutine so no background workers outlive the stream.
zr, err := zstd.NewReader(r, zstd.WithDecoderConcurrency(1))
if err != nil {
return nil, nil, err
}
return zr, zr.Close, nil
})
case "chunked":
opens = append(opens, func(r io.Reader) (io.Reader, func(), error) {
return httputil.NewChunkedReader(r), func() {}, nil
})
case "", "identity":
default:
return nil, fmt.Errorf("unsupported encoding %q, expected gzip, deflate, br, zstd, chunked or identity", name)
}
}
slices.Reverse(opens)
open := func(r io.Reader) (io.Reader, func(), error) {
var closers []func()
closeAll := func() {
for _, c := range slices.Backward(closers) {
c()
}
}
for _, o := range opens {
next, c, err := o(r)
if err != nil {
closeAll()
return nil, nil, err
}
r = next
closers = append(closers, c)
}
return r, closeAll, nil
}
d := &Decoder{
ctx: ctx,
resume: make(chan []byte),
paused: make(chan pauseReason),
done: make(chan struct{}),
}
go d.run(open)
d.wait()
return d, nil
}
// Write decodes the next chunk and returns up to about 1 MB of output.
// @example
// ```javascript
// const compress = require('nuclei/compress');
// const d = compress.NewDecoder('gzip');
// const out = d.Write(chunk);
// ```
func (d *Decoder) Write(chunk []byte) ([]byte, error) {
if d.done == nil {
return nil, errNotCreatedByNewDecoder
}
if d.inputEnded {
return nil, errors.New("decoder is closed")
}
if d.state == outputFull {
return nil, errors.New("decoded bytes are waiting: call Read until it returns no bytes before writing more")
}
if d.state == needInput && len(chunk) > 0 {
// The decode goroutine may still hold the chunk after Write returns.
d.resumeWith(bytes.Clone(chunk))
}
return d.result()
}
// Read returns more decoded bytes, or none once the decoder needs input.
// @example
// ```javascript
// const compress = require('nuclei/compress');
// const d = compress.NewDecoder('gzip');
// for (let out = d.Write(chunk); out.length > 0; out = d.Read()) log(out.length);
// ```
func (d *Decoder) Read() ([]byte, error) {
if d.done == nil {
return nil, errNotCreatedByNewDecoder
}
d.advance()
return d.result()
}
// Close ends the input and returns the last decoded bytes. A truncated
// stream errors once the bytes before the cut have been returned.
// @example
// ```javascript
// const compress = require('nuclei/compress');
// const d = compress.NewDecoder('gzip');
// d.Write(chunk);
// for (let out = d.Close(); out.length > 0; out = d.Read()) log(out.length);
// ```
func (d *Decoder) Close() ([]byte, error) {
if d.done == nil {
return nil, errNotCreatedByNewDecoder
}
d.inputEnded = true
d.advance()
return d.result()
}
// Finished reports whether a stream that marks its own end, such as a
// chunked body, has ended and all its output was read.
// @example
// ```javascript
// const compress = require('nuclei/compress');
// const d = compress.NewDecoder('gzip, chunked');
// while (!d.Finished()) for (let out = d.Write(conn.Recv(65536)); out.length > 0; out = d.Read()) log(out.length);
// ```
func (d *Decoder) Finished() bool {
return d.state == finished && len(d.out) == 0
}
var errNotCreatedByNewDecoder = errors.New("decoder was not created with compress.NewDecoder")
func (d *Decoder) advance() {
for len(d.out) == 0 && (d.state == outputFull || d.state == needInput && d.inputEnded) {
d.resumeWith(nil)
}
}
func (d *Decoder) result() ([]byte, error) {
out := d.out
d.out = nil
if len(out) == 0 && d.state == finished {
return nil, d.err
}
return out, nil
}
func (d *Decoder) resumeWith(chunk []byte) {
select {
case d.resume <- chunk:
d.wait()
case <-d.done:
d.state = finished
}
}
func (d *Decoder) wait() {
select {
case d.state = <-d.paused:
case <-d.done:
d.state = finished
}
}
func (d *Decoder) run(open func(io.Reader) (io.Reader, func(), error)) {
defer close(d.done)
r, closeReader, err := open(decoderInput{d})
if err != nil {
d.err = err
return
}
defer closeReader()
_, err = io.Copy(decoderOutput{d}, r)
d.err = err
}
func (d *Decoder) pause(reason pauseReason) ([]byte, error) {
select {
case d.paused <- reason:
case <-d.ctx.Done():
return nil, context.Cause(d.ctx)
}
select {
case chunk := <-d.resume:
return chunk, nil
case <-d.ctx.Done():
return nil, context.Cause(d.ctx)
}
}
type decoderInput struct{ d *Decoder }
func (in decoderInput) Read(p []byte) (int, error) {
d := in.d
for len(d.pending) == 0 {
if d.eof {
return 0, io.EOF
}
chunk, err := d.pause(needInput)
if err != nil {
return 0, err
}
if chunk == nil {
d.eof = true
}
d.pending = chunk
}
n := copy(p, d.pending)
d.pending = d.pending[n:]
return n, nil
}
type decoderOutput struct{ d *Decoder }
func (out decoderOutput) Write(p []byte) (int, error) {
d := out.d
d.out = append(d.out, p...)
if len(d.out) >= maxDecodedChunk {
if _, err := d.pause(outputFull); err != nil {
return 0, err
}
}
return len(p), nil
}
+192
View File
@@ -0,0 +1,192 @@
package compress
import (
"bytes"
"context"
"io"
"math/rand"
"net/http/httputil"
"testing"
"time"
"github.com/andybalholm/brotli"
"github.com/klauspost/compress/gzip"
"github.com/klauspost/compress/zlib"
"github.com/klauspost/compress/zstd"
"github.com/stretchr/testify/require"
)
func compressed(t *testing.T, encoding string, data []byte) []byte {
t.Helper()
var buf bytes.Buffer
var w io.WriteCloser
switch encoding {
case "gzip":
w = gzip.NewWriter(&buf)
case "deflate":
w = zlib.NewWriter(&buf)
case "br":
w = brotli.NewWriter(&buf)
case "zstd":
zw, err := zstd.NewWriter(&buf)
require.NoError(t, err)
w = zw
case "identity":
return data
}
_, err := w.Write(data)
require.NoError(t, err)
require.NoError(t, w.Close())
return buf.Bytes()
}
func decodeInChunks(t *testing.T, d *Decoder, input []byte) ([]byte, error) {
t.Helper()
rng := rand.New(rand.NewSource(1))
var decoded []byte
collect := func(out []byte, err error) bool {
require.LessOrEqual(t, len(out), maxDecodedChunk+64<<10)
decoded = append(decoded, out...)
return err == nil && len(out) > 0
}
drain := func(out []byte, err error) error {
for collect(out, err) {
out, err = d.Read()
}
return err
}
for len(input) > 0 {
n := min(len(input), 1+rng.Intn(70000))
out, err := d.Write(input[:n])
if err := drain(out, err); err != nil {
return decoded, err
}
input = input[n:]
}
out, err := d.Close()
return decoded, drain(out, err)
}
func TestDecoderMatchesWholeBodyDecode(t *testing.T) {
data := bytes.Repeat([]byte("nuclei streaming decoder "), 400000)
for _, encoding := range []string{"gzip", "deflate", "br", "zstd", "identity"} {
t.Run(encoding, func(t *testing.T) {
d, err := NewDecoder(t.Context(), encoding)
require.NoError(t, err)
decoded, err := decodeInChunks(t, d, compressed(t, encoding, data))
require.NoError(t, err)
require.Equal(t, data, decoded)
})
}
}
func chunked(t *testing.T, data []byte) []byte {
t.Helper()
var buf bytes.Buffer
w := httputil.NewChunkedWriter(&buf)
for len(data) > 0 {
n := min(len(data), 8000)
_, err := w.Write(data[:n])
require.NoError(t, err)
data = data[n:]
}
require.NoError(t, w.Close())
buf.WriteString("\r\n")
return buf.Bytes()
}
func TestDecoderDecodesChunkedTransfer(t *testing.T) {
data := bytes.Repeat([]byte("nuclei chunked decoder "), 400000)
for _, encoding := range []string{"gzip", "deflate", "br", "zstd", "identity"} {
t.Run(encoding, func(t *testing.T) {
d, err := NewDecoder(t.Context(), encoding+", chunked")
require.NoError(t, err)
decoded, err := decodeInChunks(t, d, chunked(t, compressed(t, encoding, data)))
require.NoError(t, err)
require.Equal(t, data, decoded)
})
}
}
func TestDecoderFinishesAtLastChunk(t *testing.T) {
data := bytes.Repeat([]byte("finished "), 1000)
d, err := NewDecoder(t.Context(), "gzip, chunked")
require.NoError(t, err)
input := chunked(t, compressed(t, "gzip", data))
half := len(input) / 2
_, err = d.Write(input[:half])
require.NoError(t, err)
require.False(t, d.Finished())
out, err := d.Write(input[half:])
require.NoError(t, err)
var decoded []byte
for ; len(out) > 0; out, err = d.Read() {
decoded = append(decoded, out...)
}
require.NoError(t, err)
require.Equal(t, data, decoded)
require.True(t, d.Finished(), "decoder must finish without Close once the last chunk arrives")
}
func TestDecoderAppliesCodingsInReverse(t *testing.T) {
data := bytes.Repeat([]byte("two codings "), 100000)
input := compressed(t, "br", compressed(t, "gzip", data))
d, err := NewDecoder(t.Context(), "gzip, br")
require.NoError(t, err)
decoded, err := decodeInChunks(t, d, input)
require.NoError(t, err)
require.Equal(t, data, decoded)
}
func TestDecoderReportsTruncatedStream(t *testing.T) {
data := bytes.Repeat([]byte("truncated "), 200000)
for _, encoding := range []string{"gzip", "deflate", "br", "zstd", "identity, chunked"} {
t.Run(encoding, func(t *testing.T) {
var input []byte
if encoding == "identity, chunked" {
input = chunked(t, data)
} else {
input = compressed(t, encoding, data)
}
d, err := NewDecoder(t.Context(), encoding)
require.NoError(t, err)
decoded, err := decodeInChunks(t, d, input[:len(input)/2])
require.Error(t, err)
require.True(t, bytes.HasPrefix(data, decoded), "bytes before the cut must still be handed back")
})
}
}
func TestDecoderStopsWhenContextEnds(t *testing.T) {
ctx, cancel := context.WithCancel(t.Context())
d, err := NewDecoder(ctx, "gzip")
require.NoError(t, err)
_, err = d.Write(compressed(t, "gzip", []byte("abandoned"))[:5])
require.NoError(t, err)
cancel()
select {
case <-d.done:
case <-time.After(time.Second):
t.Fatal("decode goroutine outlived its context")
}
}
func TestNewDecoderRejectsUnknownEncoding(t *testing.T) {
_, err := NewDecoder(t.Context(), "compress")
require.Error(t, err)
_, err = NewDecoder(t.Context(), "gzip, compress")
require.Error(t, err)
}
func TestDecoderNotCreatedByNewDecoderErrors(t *testing.T) {
var d Decoder
_, err := d.Write([]byte("x"))
require.ErrorIs(t, err, errNotCreatedByNewDecoder)
_, err = d.Read()
require.ErrorIs(t, err, errNotCreatedByNewDecoder)
_, err = d.Close()
require.ErrorIs(t, err, errNotCreatedByNewDecoder)
}
+100
View File
@@ -0,0 +1,100 @@
// Package hash exposes incremental hashes to nuclei JavaScript templates.
package hash
import (
"crypto/md5"
"crypto/sha1"
"crypto/sha256"
"crypto/sha512"
"encoding/hex"
"errors"
"fmt"
"hash"
"strings"
"github.com/spaolacci/murmur3"
)
type (
// Hash is a running hash.
// @example
// ```javascript
// const hash = require('nuclei/hash');
// const h = hash.New('sha256');
// h.WriteString('hello');
// log(h.Sum());
// ```
Hash struct {
h hash.Hash
mmh3 bool
}
)
// New returns a running hash for md5, sha1, sha256, sha512 or mmh3. Sum
// matches the DSL function of the same name.
// @example
// ```javascript
// const hash = require('nuclei/hash');
// const h = hash.New('sha256');
// ```
func New(algorithm string) (*Hash, error) {
switch strings.ToLower(algorithm) {
case "md5":
return &Hash{h: md5.New()}, nil
case "sha1":
return &Hash{h: sha1.New()}, nil
case "sha256":
return &Hash{h: sha256.New()}, nil
case "sha512":
return &Hash{h: sha512.New()}, nil
case "mmh3":
return &Hash{h: murmur3.New32WithSeed(0), mmh3: true}, nil
}
return nil, fmt.Errorf("unsupported hash algorithm %q, expected md5, sha1, sha256, sha512 or mmh3", algorithm)
}
// Write adds data to the hash.
// @example
// ```javascript
// const hash = require('nuclei/hash');
// const h = hash.New('sha256');
// h.Write([104, 105]);
// ```
func (h *Hash) Write(data []byte) error {
if h.h == nil {
return errNotCreatedByNew
}
_, _ = h.h.Write(data)
return nil
}
// WriteString adds the string data to the hash.
// @example
// ```javascript
// const hash = require('nuclei/hash');
// const h = hash.New('sha256');
// h.WriteString('hi');
// ```
func (h *Hash) WriteString(data string) error {
return h.Write([]byte(data))
}
// Sum returns the hash of the data written so far, formatted as the DSL does.
// @example
// ```javascript
// const hash = require('nuclei/hash');
// const h = hash.New('mmh3');
// h.WriteString('hello');
// log(h.Sum());
// ```
func (h *Hash) Sum() (string, error) {
if h.h == nil {
return "", errNotCreatedByNew
}
if h.mmh3 {
return fmt.Sprintf("%d", int32(h.h.(hash.Hash32).Sum32())), nil
}
return hex.EncodeToString(h.h.Sum(nil)), nil
}
var errNotCreatedByNew = errors.New("hash was not created with hash.New")
+43
View File
@@ -0,0 +1,43 @@
package hash
import (
"testing"
"github.com/projectdiscovery/dsl"
"github.com/stretchr/testify/require"
)
func TestSumMatchesDSLAcrossChunks(t *testing.T) {
data := make([]byte, 3<<20+7)
for i := range data {
data[i] = byte(i * 31)
}
for _, algorithm := range []string{"md5", "sha1", "sha256", "sha512", "mmh3"} {
t.Run(algorithm, func(t *testing.T) {
h, err := New(algorithm)
require.NoError(t, err)
for chunk := data; len(chunk) > 0; {
n := min(len(chunk), 65521)
require.NoError(t, h.Write(chunk[:n]))
chunk = chunk[n:]
}
want, err := dsl.EvalExpr(algorithm+"(body)", map[string]interface{}{"body": string(data)})
require.NoError(t, err)
sum, err := h.Sum()
require.NoError(t, err)
require.Equal(t, want, sum)
})
}
}
func TestNewRejectsUnknownAlgorithm(t *testing.T) {
_, err := New("crc32")
require.Error(t, err)
}
func TestHashNotCreatedByNewErrors(t *testing.T) {
var h Hash
require.ErrorIs(t, h.WriteString("x"), errNotCreatedByNew)
_, err := h.Sum()
require.ErrorIs(t, err, errNotCreatedByNew)
}
@@ -2,7 +2,11 @@ package mysql
import (
"context"
"fmt"
"net"
"os"
"path/filepath"
"runtime"
"strings"
"sync/atomic"
"testing"
@@ -169,6 +173,84 @@ func TestFingerprintMySQLDeniedByNetworkPolicy(t *testing.T) {
require.Contains(t, err.Error(), "127.0.0.1")
}
func TestConnectWithDSNDeniedByNetworkPolicy(t *testing.T) {
t.Parallel()
executionID := "mysql-dsn-deny-" + strings.NewReplacer("/", "-", " ", "-").Replace(t.Name())
require.NoError(t, protocolstate.Init(&types.Options{
ExecutionId: executionID,
RestrictLocalNetworkAccess: true,
}))
t.Cleanup(func() { protocolstate.Close(executionID) })
ctx := context.WithValue(context.Background(), "executionId", executionID) //nolint:staticcheck
var accepted atomic.Int32
ln, err := net.Listen("tcp", "127.0.0.1:0")
require.NoError(t, err)
t.Cleanup(func() { _ = ln.Close() })
go func() {
for {
conn, err := ln.Accept()
if err != nil {
return
}
accepted.Add(1)
_ = conn.Close()
}
}()
port := ln.Addr().(*net.TCPAddr).Port
dsn := fmt.Sprintf("vuser:vpass@tcp(127.0.0.1:%d)/vdb", port)
ok, err := (&MySQLClient{}).ConnectWithDSN(ctx, dsn)
require.Error(t, err)
require.False(t, ok)
require.Contains(t, err.Error(), "127.0.0.1")
require.Zero(t, accepted.Load())
}
func TestConnectWithDSNUnixDeniedByNetworkPolicy(t *testing.T) {
t.Parallel()
executionID := "mysql-dsn-unix-deny-" + strings.NewReplacer("/", "-", " ", "-").Replace(t.Name())
require.NoError(t, protocolstate.Init(&types.Options{
ExecutionId: executionID,
RestrictLocalNetworkAccess: true,
}))
t.Cleanup(func() { protocolstate.Close(executionID) })
ctx := context.WithValue(context.Background(), "executionId", executionID) //nolint:staticcheck
// t.TempDir can exceed the unix socket path limit on macOS, so use a short dir.
dir, err := os.MkdirTemp("", "nuclei-mysql")
require.NoError(t, err)
t.Cleanup(func() { _ = os.RemoveAll(dir) })
sock := filepath.Join(dir, "lna.sock")
ln, err := net.Listen("unix", sock)
if err != nil && runtime.GOOS == "windows" {
t.Skipf("unix sockets not available: %v", err)
}
require.NoError(t, err)
t.Cleanup(func() { _ = ln.Close() })
var accepted atomic.Int32
go func() {
for {
conn, err := ln.Accept()
if err != nil {
return
}
accepted.Add(1)
_ = conn.Close()
}
}()
dsn := fmt.Sprintf("vuser:vpass@unix(%s)/vdb", sock)
ok, err := (&MySQLClient{}).ConnectWithDSN(ctx, dsn)
require.Error(t, err)
require.False(t, ok)
require.Contains(t, err.Error(), "127.0.0.1")
require.Zero(t, accepted.Load())
}
func TestFingerprintMySQLVersionNeverEmptyOnHandshakeFixtures(t *testing.T) {
t.Parallel()
+40 -7
View File
@@ -76,24 +76,57 @@ func BuildDSN(opts MySQLOptions) (string, error) {
return dsn.String(), nil
}
// sandboxDSN enforces the local file access sandbox on a MySQL DSN. The
// driver's allowAllFiles option lets a malicious server read any local file
// off the host via LOAD DATA LOCAL INFILE, so it is only honored when -lfa is
// enabled, mirroring the fs.ReadFile restriction.
func sandboxDSN(dsn string, lfaAllowed bool) (string, error) {
// sandboxDSN enforces local file and network policy on a MySQL DSN.
// allowAllFiles is only honored when -lfa is enabled. tcp, tcp4 and tcp6
// DSNs are rewritten to nucleitcp so the driver uses the fastdialer
// registered in protocolstate. Unix sockets are treated as loopback for -lna.
// Other networks are rejected.
func sandboxDSN(executionId, dsn string) (string, error) {
cfg, err := mysql.ParseDSN(dsn)
if err != nil {
return "", err
}
if cfg.AllowAllFiles && !lfaAllowed {
opts := &types.Options{ExecutionId: executionId}
if cfg.AllowAllFiles && !protocolstate.IsLfaAllowed(opts) {
cfg.AllowAllFiles = false
}
netName := cfg.Net
if netName == "" {
netName = "tcp"
}
switch netName {
case "unix":
// IP-based policy never sees a socket path; gate on loopback like ldapi.
const unixPolicyHost = "127.0.0.1"
if !protocolstate.IsHostAllowed(executionId, unixPolicyHost) {
return "", protocolstate.ErrHostDenied.Msgf(unixPolicyHost)
}
case "tcp", "tcp4", "tcp6", "nucleitcp":
// Only nucleitcp has a registered dialer; the others would use the
// driver's plain net.Dialer and skip the dial-time policy check.
cfg.Net = "nucleitcp"
host := dsnPolicyHost(cfg.Addr)
if !protocolstate.IsHostAllowed(executionId, host) {
return "", protocolstate.ErrHostDenied.Msgf(host)
}
default:
return "", fmt.Errorf("unsupported mysql network %q", netName)
}
return cfg.FormatDSN(), nil
}
func dsnPolicyHost(addr string) string {
host, _, err := net.SplitHostPort(addr)
if err == nil && host != "" {
return host
}
return addr
}
// openDB opens a sandboxed MySQL connection from dsn.
func openDB(executionId, dsn string) (*sql.DB, error) {
dsn, err := sandboxDSN(dsn, protocolstate.IsLfaAllowed(&types.Options{ExecutionId: executionId}))
dsn, err := sandboxDSN(executionId, dsn)
if err != nil {
return nil, err
}
+86 -4
View File
@@ -5,11 +5,14 @@ import (
"github.com/go-sql-driver/mysql"
"github.com/stretchr/testify/require"
"github.com/projectdiscovery/nuclei/v3/pkg/protocols/common/protocolstate"
"github.com/projectdiscovery/nuclei/v3/pkg/types"
)
func TestSandboxDSN(t *testing.T) {
t.Run("strips allowAllFiles when lfa disabled", func(t *testing.T) {
got, err := sandboxDSN("root:x@nucleitcp(127.0.0.1:3306)/?allowAllFiles=true", false)
got, err := sandboxDSN("", "root:x@nucleitcp(127.0.0.1:3306)/?allowAllFiles=true")
require.NoError(t, err)
cfg, err := mysql.ParseDSN(got)
@@ -18,7 +21,14 @@ func TestSandboxDSN(t *testing.T) {
})
t.Run("keeps allowAllFiles when lfa enabled", func(t *testing.T) {
got, err := sandboxDSN("root:x@nucleitcp(127.0.0.1:3306)/?allowAllFiles=true", true)
executionID := "mysql-lfa-" + t.Name()
require.NoError(t, protocolstate.Init(&types.Options{
ExecutionId: executionID,
AllowLocalFileAccess: true,
}))
t.Cleanup(func() { protocolstate.Close(executionID) })
got, err := sandboxDSN(executionID, "root:x@nucleitcp(127.0.0.1:3306)/?allowAllFiles=true")
require.NoError(t, err)
cfg, err := mysql.ParseDSN(got)
@@ -27,7 +37,7 @@ func TestSandboxDSN(t *testing.T) {
})
t.Run("leaves dsn without allowAllFiles untouched", func(t *testing.T) {
got, err := sandboxDSN("root:x@nucleitcp(127.0.0.1:3306)/", false)
got, err := sandboxDSN("", "root:x@nucleitcp(127.0.0.1:3306)/")
require.NoError(t, err)
cfg, err := mysql.ParseDSN(got)
@@ -35,8 +45,80 @@ func TestSandboxDSN(t *testing.T) {
require.False(t, cfg.AllowAllFiles)
})
t.Run("rewrites tcp to nucleitcp", func(t *testing.T) {
got, err := sandboxDSN("", "root:x@tcp(acme.com:3306)/")
require.NoError(t, err)
cfg, err := mysql.ParseDSN(got)
require.NoError(t, err)
require.Equal(t, "nucleitcp", cfg.Net)
require.Equal(t, "acme.com:3306", cfg.Addr)
})
t.Run("rewrites tcp4 and tcp6 to nucleitcp", func(t *testing.T) {
for _, netName := range []string{"tcp4", "tcp6"} {
got, err := sandboxDSN("", "root:x@"+netName+"(acme.com:3306)/")
require.NoError(t, err)
cfg, err := mysql.ParseDSN(got)
require.NoError(t, err)
require.Equal(t, "nucleitcp", cfg.Net)
require.Equal(t, "acme.com:3306", cfg.Addr)
}
})
t.Run("rejects unsupported network", func(t *testing.T) {
_, err := sandboxDSN("", "root:x@udp(acme.com:3306)/")
require.Error(t, err)
require.Contains(t, err.Error(), "udp")
})
t.Run("errors on invalid dsn", func(t *testing.T) {
_, err := sandboxDSN("::not-a-dsn::", false)
_, err := sandboxDSN("", "::not-a-dsn::")
require.Error(t, err)
})
t.Run("denies loopback tcp when lna enabled", func(t *testing.T) {
executionID := "mysql-lna-tcp-" + t.Name()
require.NoError(t, protocolstate.Init(&types.Options{
ExecutionId: executionID,
RestrictLocalNetworkAccess: true,
}))
t.Cleanup(func() { protocolstate.Close(executionID) })
_, err := sandboxDSN(executionID, "vuser:vpass@tcp(127.0.0.1:3306)/vdb")
require.Error(t, err)
require.Contains(t, err.Error(), "127.0.0.1")
})
t.Run("denies loopback variants when lna enabled", func(t *testing.T) {
executionID := "mysql-lna-variants-" + t.Name()
require.NoError(t, protocolstate.Init(&types.Options{
ExecutionId: executionID,
RestrictLocalNetworkAccess: true,
}))
t.Cleanup(func() { protocolstate.Close(executionID) })
for _, dsn := range []string{
"vuser:vpass@tcp4(127.0.0.1:3306)/vdb",
"vuser:vpass@tcp6([::1]:3306)/vdb",
"vuser:vpass@tcp([::ffff:127.0.0.1]:3306)/vdb",
} {
_, err := sandboxDSN(executionID, dsn)
require.Error(t, err, dsn)
}
})
t.Run("denies unix socket when lna enabled", func(t *testing.T) {
executionID := "mysql-lna-unix-" + t.Name()
require.NoError(t, protocolstate.Init(&types.Options{
ExecutionId: executionID,
RestrictLocalNetworkAccess: true,
}))
t.Cleanup(func() { protocolstate.Close(executionID) })
_, err := sandboxDSN(executionID, "vuser:vpass@unix(/tmp/nuclei-mysql-lna.sock)/vdb")
require.Error(t, err)
require.Contains(t, err.Error(), "127.0.0.1")
})
}
-1
View File
@@ -3,7 +3,6 @@ package redis
import (
"context"
"errors"
"fmt"
+113 -117
View File
@@ -3,13 +3,12 @@ package redis
import (
"bytes"
"context"
"errors"
"fmt"
"io"
"net"
"strconv"
"time"
"github.com/projectdiscovery/gologger"
"github.com/projectdiscovery/nuclei/v3/pkg/protocols/common/protocolstate"
"github.com/redis/go-redis/v9"
@@ -269,9 +268,10 @@ var errReadLimit = fmt.Errorf("redis: reply exceeds %d bytes", maxReadBytes)
// maxReadBytes.
func newClient(addr, password string) *redis.Client {
opts := &redis.Options{
Addr: addr,
Password: password,
DB: 0,
Addr: addr,
Password: password,
DB: 0,
DialerRetries: dialAttempts,
}
dial := redis.NewDialer(opts)
opts.Dialer = func(ctx context.Context, network, addr string) (net.Conn, error) {
@@ -285,138 +285,134 @@ func newClient(addr, password string) *redis.Client {
return redis.NewClient(opts)
}
// limitedConn fails a read once remaining is spent, and withholds a RESP length line whose
// declared size is over the cap so go-redis never allocates it.
// maxHeaderLen fits a type byte, any int64, and "\r". Redis never sends a longer length header.
const maxHeaderLen = 32
// limitedConn fails a read once remaining is spent, and ends a read before the "\n" of a RESP
// length header whose declared size is over the cap, so go-redis never gets the line to allocate
// from.
//
// It follows the reply's framing as the bytes pass: a bulk payload is skipped by its declared
// length, so only header lines are checked. A payload line such as "*abc" is data.
type limitedConn struct {
net.Conn
remaining int64
pending []byte
partial []byte
err error
scratch [32 << 10]byte
// payload is how many bytes of the current bulk payload, "\r\n" included, are still to come.
payload int64
// header is the start of the current header line, without its "\n".
header []byte
overlong bool
err error
}
func (c *limitedConn) Read(b []byte) (int, error) {
if len(b) == 0 {
if c.err != nil {
return 0, c.err
}
if c.remaining <= 0 {
return 0, errReadLimit
}
if int64(len(b)) > c.remaining {
b = b[:c.remaining]
}
n, err := c.Conn.Read(b)
c.remaining -= int64(n)
// Errors from the connection, deadlines included, are returned as they come and not kept: a
// read deadline belongs to one call. go-redis peeks for push notifications with a few
// microseconds of deadline, and keeping that timeout would fail the command that follows.
kept, scanErr := c.scan(b[:n])
if scanErr == nil {
return n, err
}
c.err = scanErr
if kept == 0 {
return 0, scanErr
}
return kept, nil
}
// scan follows the RESP framing of p and returns how many of its bytes may be returned.
func (c *limitedConn) scan(p []byte) (int, error) {
for i := 0; i < len(p); {
if c.payload > 0 {
skip := min(c.payload, int64(len(p)-i))
c.payload -= skip
i += int(skip)
continue
}
end := bytes.IndexByte(p[i:], '\n')
if end < 0 {
c.keepHeader(p[i:])
return len(p), nil
}
c.keepHeader(p[i : i+end])
payload, err := declaredPayload(c.header, c.overlong)
c.header, c.overlong = c.header[:0], false
if err != nil {
return i + end, err
}
c.payload = payload
i += end + 1
}
return len(p), nil
}
func (c *limitedConn) keepHeader(p []byte) {
if room := maxHeaderLen - len(c.header); len(p) > room {
p = p[:room]
c.overlong = true
}
c.header = append(c.header, p...)
}
// declaredPayload returns how many payload bytes, "\r\n" included, follow a RESP header line
// given without its "\n". It returns errReadLimit when go-redis would allocate more than the cap
// from the line: the declared byte length for bulk strings, one interface per declared element
// for arrays, sets, maps, attributes, and pushes.
func declaredPayload(line []byte, overlong bool) (int64, error) {
if len(line) == 0 {
return 0, nil
}
if len(c.pending) == 0 {
if err := c.fill(); err != nil {
return 0, err
}
}
n := copy(b, c.pending)
c.pending = c.pending[n:]
return n, nil
}
func (c *limitedConn) fill() error {
if len(c.pending) > 0 {
return nil
}
if c.err != nil {
return c.err
}
for len(c.pending) == 0 {
if c.remaining <= 0 {
c.err = errReadLimit
return c.err
}
want := int64(len(c.scratch))
if want > c.remaining {
want = c.remaining
}
n, err := c.Conn.Read(c.scratch[:want])
if n > 0 {
c.remaining -= int64(n)
if aerr := c.accept(c.scratch[:n]); aerr != nil {
c.err = aerr
if len(c.pending) > 0 {
return nil
}
return aerr
}
}
if err != nil {
// A read deadline belongs to one call. go-redis peeks for push notifications
// with a few microseconds of deadline, and on Windows that peek always runs.
// Keeping the timeout fails the command that follows.
if timeoutErr(err) {
if len(c.pending) > 0 {
return nil
}
return err
}
c.err = err
if len(c.pending) > 0 {
return nil
}
return err
}
if n == 0 {
c.err = io.ErrUnexpectedEOF
return c.err
}
}
return nil
}
func (c *limitedConn) accept(p []byte) error {
c.partial = append(c.partial, p...)
for {
i := bytes.IndexByte(c.partial, '\n')
if i < 0 {
if int64(len(c.partial)) > maxReadBytes {
c.partial = nil
return errReadLimit
}
return nil
}
line := c.partial[:i+1]
if lengthTooLarge(line) {
c.partial = nil
return errReadLimit
}
copied := make([]byte, len(line))
copy(copied, line)
c.pending = append(c.pending, copied...)
rest := make([]byte, len(c.partial)-(i+1))
copy(rest, c.partial[i+1:])
c.partial = rest
}
}
// lengthTooLarge reports whether line is a RESP length header go-redis would allocate from.
// Bulk strings allocate the declared byte length. Arrays, sets, maps, attributes, and pushes
// allocate one interface per declared element.
func lengthTooLarge(line []byte) bool {
if len(line) < 3 {
return false
}
var limit int64
bulk := false
switch line[0] {
case '$', '=', '!':
limit = maxReadBytes
limit, bulk = maxReadBytes, true
case '*', '~', '>', '%', '|':
limit = maxAggregate
default:
return false
return 0, nil
}
body := line[1 : len(line)-1]
if len(body) > 0 && body[len(body)-1] == '\r' {
body = body[:len(body)-1]
if overlong {
return 0, errReadLimit
}
body := bytes.TrimSuffix(line[1:], []byte("\r"))
if len(body) == 0 || body[0] == '-' {
return false
return 0, nil
}
n, err := strconv.ParseInt(string(body), 10, 64)
if err != nil {
return true
if err != nil || n > limit {
return 0, errReadLimit
}
return n > limit
if !bulk {
return 0, nil
}
return n + 2, nil
}
func timeoutErr(err error) bool {
var netErr net.Error
return errors.As(err, &netErr) && netErr.Timeout()
// dialAttempts replaces go-redis's default of 5. The extra attempts run in a goroutine that
// ignores the caller's context, so they kept dialing a filtered port for ~25s after the
// template had given up on it.
const dialAttempts = 1
func init() {
redis.SetLogger(warningLogger{})
}
// warningLogger hands go-redis's logs to gologger as warnings. Its default logger writes them
// straight to stderr, where a failed dial against the scanned host reads as an error.
type warningLogger struct{}
func (warningLogger) Printf(_ context.Context, format string, v ...interface{}) {
gologger.Warning().Msgf(format, v...)
}
Loaded 100 of 190 files, more files were not shown because too many files have changed in this diff. Show more