Compare commits

...
Author SHA1 Message Date
Nakul Bharti a99715a1e9 test(flow): fix compatibility imports 2026-10-13 22:19:43 +05:30
Nakul Bharti 86d437fd84 fix(flow): bound runtime execution 2026-10-13 22:16:40 +05:30
Nakul Bharti 588ebb3e22 fix: address PR review comments 2026-10-05 04:02:24 +05:30
Nakul Bharti e8c7aa174d fix(headless): bound browser setup by request context 2026-10-05 03:58:27 +05:30
Nakul Bharti 12189e8f90 test: keep request coverage with request tests 2026-10-03 01:10:14 +05:30
Nakul Bharti d53dd155b4 fix: address PR review comments 2026-10-03 00:20:18 +05:30
Nakul Bharti cd15ec4ca8 fix(http): cancel per-host rate admission 2026-10-03 00:02:42 +05:30
Nakul Bharti 66068dbec2 perf: improve embedded scan runtime efficiency (#7717)
* feat(templates): add execution-local parser lifecycle

* fix(network): include ports and payloads in request count

* fix(headless): include payloads in request count

* feat(core): expose template execution lifecycle

* fix(core): report context-interrupted template executions

* perf(interactsh): allocate caches on first use

* perf(stats): reuse compiled waf detector

* perf(hosterrors): release cache storage on close

* fix(deps): use published tlsx release

* perf(templates): coalesce concurrent parses

* perf(stats): prefilter waf response regexes

* perf(stats): scan waf literals once per response

* fix(sdk): isolate shared interactsh executions

* fix(protocols): preserve interactsh request scope

* fix(interactsh): drain in-flight scoped callbacks

* perf(hosterrors): allocate storage on first failure

* fix(interactsh): recognize raw SNI callbacks

* fix(interactsh): guard non-fuzz callback accounting

* perf(jarm): integrate bounded shared fingerprints

* fix(jarm): honor configured scan proxy

* perf(core): support adaptive template concurrency

* perf(core): enforce shared template concurrency budget

* fix(http): smooth low per-host request limits

* perf(http): remove per-host limiter goroutines

* fix(http): retain host budgets for large embedded scans

* test: consolidate parser and headless coverage

* fix: address PR review comments

* fix(headless): honor scan cancellation
2026-09-15 02:35:21 +05:30
Dogan Can Bakir 5347be3255 Merge pull request #7725 from projectdiscovery/dogancanbakir/feat/4953-dast-proxy
add dast proxy mode to fuzz live traffic
2026-09-14 13:41:07 +03:00
dependabot[bot] 59f4635e8c chore(deps): bump the modules group with 11 updates (#7737)
Bumps the modules group with 11 updates:

| Package | From | To |
| --- | --- | --- |
| [github.com/projectdiscovery/clistats](https://github.com/projectdiscovery/clistats) | `0.1.6` | `0.1.7` |
| [github.com/projectdiscovery/fastdialer](https://github.com/projectdiscovery/fastdialer) | `0.5.19` | `0.5.20` |
| [github.com/projectdiscovery/retryablehttp-go](https://github.com/projectdiscovery/retryablehttp-go) | `1.3.25` | `1.3.26` |
| [github.com/projectdiscovery/goflags](https://github.com/projectdiscovery/goflags) | `0.2.0` | `0.2.1` |
| [github.com/projectdiscovery/gologger](https://github.com/projectdiscovery/gologger) | `1.1.72` | `1.1.73` |
| [github.com/projectdiscovery/httpx](https://github.com/projectdiscovery/httpx) | `1.11.0` | `1.12.0` |
| [github.com/projectdiscovery/networkpolicy](https://github.com/projectdiscovery/networkpolicy) | `0.1.49` | `0.1.50` |
| [github.com/projectdiscovery/ratelimit](https://github.com/projectdiscovery/ratelimit) | `0.0.88` | `0.0.89` |
| [github.com/projectdiscovery/tlsx](https://github.com/projectdiscovery/tlsx) | `1.3.2` | `1.4.0` |
| [github.com/projectdiscovery/wappalyzergo](https://github.com/projectdiscovery/wappalyzergo) | `0.2.96` | `0.3.0` |
| [github.com/projectdiscovery/cdncheck](https://github.com/projectdiscovery/cdncheck) | `1.2.51` | `1.3.0` |


Updates `github.com/projectdiscovery/clistats` from 0.1.6 to 0.1.7
- [Release notes](https://github.com/projectdiscovery/clistats/releases)
- [Commits](https://github.com/projectdiscovery/clistats/compare/v0.1.6...v0.1.7)

Updates `github.com/projectdiscovery/fastdialer` from 0.5.19 to 0.5.20
- [Release notes](https://github.com/projectdiscovery/fastdialer/releases)
- [Commits](https://github.com/projectdiscovery/fastdialer/compare/v0.5.19...v0.5.20)

Updates `github.com/projectdiscovery/retryablehttp-go` from 1.3.25 to 1.3.26
- [Release notes](https://github.com/projectdiscovery/retryablehttp-go/releases)
- [Commits](https://github.com/projectdiscovery/retryablehttp-go/compare/v1.3.25...v1.3.26)

Updates `github.com/projectdiscovery/goflags` from 0.2.0 to 0.2.1
- [Release notes](https://github.com/projectdiscovery/goflags/releases)
- [Commits](https://github.com/projectdiscovery/goflags/compare/v0.2.0...v0.2.1)

Updates `github.com/projectdiscovery/gologger` from 1.1.72 to 1.1.73
- [Release notes](https://github.com/projectdiscovery/gologger/releases)
- [Commits](https://github.com/projectdiscovery/gologger/compare/v1.1.72...v1.1.73)

Updates `github.com/projectdiscovery/httpx` from 1.11.0 to 1.12.0
- [Release notes](https://github.com/projectdiscovery/httpx/releases)
- [Commits](https://github.com/projectdiscovery/httpx/compare/v1.11.0...v1.12.0)

Updates `github.com/projectdiscovery/networkpolicy` from 0.1.49 to 0.1.50
- [Release notes](https://github.com/projectdiscovery/networkpolicy/releases)
- [Commits](https://github.com/projectdiscovery/networkpolicy/compare/v0.1.49...v0.1.50)

Updates `github.com/projectdiscovery/ratelimit` from 0.0.88 to 0.0.89
- [Release notes](https://github.com/projectdiscovery/ratelimit/releases)
- [Commits](https://github.com/projectdiscovery/ratelimit/compare/v0.0.88...v0.0.89)

Updates `github.com/projectdiscovery/tlsx` from 1.3.2 to 1.4.0
- [Release notes](https://github.com/projectdiscovery/tlsx/releases)
- [Commits](https://github.com/projectdiscovery/tlsx/commits/v1.4.0)

Updates `github.com/projectdiscovery/wappalyzergo` from 0.2.96 to 0.3.0
- [Release notes](https://github.com/projectdiscovery/wappalyzergo/releases)
- [Commits](https://github.com/projectdiscovery/wappalyzergo/compare/v0.2.96...v0.3.0)

Updates `github.com/projectdiscovery/cdncheck` from 1.2.51 to 1.3.0
- [Release notes](https://github.com/projectdiscovery/cdncheck/releases)
- [Commits](https://github.com/projectdiscovery/cdncheck/compare/v1.2.51...v1.3.0)

---
updated-dependencies:
- dependency-name: github.com/projectdiscovery/clistats
  dependency-version: 0.1.7
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: modules
- dependency-name: github.com/projectdiscovery/fastdialer
  dependency-version: 0.5.20
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: modules
- dependency-name: github.com/projectdiscovery/retryablehttp-go
  dependency-version: 1.3.26
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: modules
- dependency-name: github.com/projectdiscovery/goflags
  dependency-version: 0.2.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: modules
- dependency-name: github.com/projectdiscovery/gologger
  dependency-version: 1.1.73
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: modules
- dependency-name: github.com/projectdiscovery/httpx
  dependency-version: 1.12.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: modules
- dependency-name: github.com/projectdiscovery/networkpolicy
  dependency-version: 0.1.50
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: modules
- dependency-name: github.com/projectdiscovery/ratelimit
  dependency-version: 0.0.89
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: modules
- dependency-name: github.com/projectdiscovery/tlsx
  dependency-version: 1.4.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: modules
- dependency-name: github.com/projectdiscovery/wappalyzergo
  dependency-version: 0.3.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: modules
- dependency-name: github.com/projectdiscovery/cdncheck
  dependency-version: 1.3.0
  dependency-type: indirect
  update-type: version-update:semver-minor
  dependency-group: modules
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-14 06:22:51 +00:00
Mzack9999 8f4c1fff90 raise timeout 2026-09-11 20:37:46 +02:00
Mzack9999 6138a17677 bound tests 2026-09-11 18:23:14 +02:00
Mzack9999 666dde153a fix hex 2026-09-10 21:52:42 +02:00
Mzack9999 6dfd955715 harden proxy 2026-09-10 19:06:48 +02:00
Doğan Can Bakır 7b14d3db3c recover dast proxy CA from a partial write 2026-09-10 18:37:49 +03:00
Doğan Can Bakır b69bf17ce1 add dast proxy mode to fuzz live traffic 2026-09-10 14:16:32 +03:00
dependabot[bot] a04aba1022 chore(deps): bump actions/checkout in the workflows group (#7713)
Bumps the workflows group with 1 update: [actions/checkout](https://github.com/actions/checkout).


Updates `actions/checkout` from 6.0.2 to 7.0.1
- [Release notes](https://github.com/actions/checkout/releases)
- [Commits](https://github.com/actions/checkout/compare/v6.0.2...v7.0.1)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-version: 7.0.1
  dependency-type: direct:production
  update-type: version-update:semver-major
  dependency-group: workflows
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-10 09:09:43 +07:00
Mzack9999 f91f5bb405 Merge pull request #7721 from projectdiscovery/dwisiswant0/chore/deps/bump-github-com-bytedance-sonic-v1-15-3
chore(deps): bump github.com/bytedance/[email protected].3
2026-09-09 23:30:27 +02:00
Mzack9999 1cbdf4f4c7 Merge pull request #7724 from projectdiscovery/dogancanbakir/fix/7722-stats-percent
fix(stats): invalid percent on automatic scan
2026-09-09 23:21:11 +02:00
Mzack9999 97f24a1c3e Merge pull request #7723 from projectdiscovery/dogancanbakir/docs/lfa-scope
docs(cli): clarify -lfa scope
2026-09-09 22:37:31 +02:00
Mzack9999 e39543e57b guard percent 2026-09-09 22:13:38 +02:00
Doğan Can Bakır c12b7f21c3 fix(stats): count automatic scan requests in progress 2026-09-09 17:09:29 +03:00
Doğan Can Bakır 2d1914d808 fix(stats): guard percent divide by zero 2026-09-09 17:09:22 +03:00
Doğan Can Bakır 7c4d90f9d3 docs(cli): clarify -lfa scope 2026-09-09 16:49:38 +03:00
Dwi Siswanto 88f78206b1 chore(deps): bump github.com/bytedance/[email protected]
Signed-off-by: Dwi Siswanto <[email protected]>
2026-09-09 18:03:59 +07:00
dependabot[bot] 12aa925acc chore(deps): bump filippo.io/age (#7719)
Bumps the go_modules group with 1 update in the / directory: [filippo.io/age](https://github.com/FiloSottile/age).


Updates `filippo.io/age` from 1.2.0 to 1.2.1
- [Release notes](https://github.com/FiloSottile/age/releases)
- [Commits](https://github.com/FiloSottile/age/compare/v1.2.0...v1.2.1)

---
updated-dependencies:
- dependency-name: filippo.io/age
  dependency-version: 1.2.1
  dependency-type: direct:production
  dependency-group: go_modules
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-09 01:39:38 +00:00
Mzack9999 f2e1f2c830 Merge pull request #7711 from projectdiscovery/dwisiswant0/fix/operators/reject-matchers-extractors-w-o-values
fix(operators): reject matchers & extractors w/o values
2026-09-08 17:06:07 +04:00
dependabot[bot] f6abdcf2f2 chore(deps): bump the modules group with 9 updates (#7712)
Bumps the modules group with 9 updates:

| Package | From | To |
| --- | --- | --- |
| [github.com/projectdiscovery/fastdialer](https://github.com/projectdiscovery/fastdialer) | `0.5.18` | `0.5.19` |
| [github.com/projectdiscovery/rawhttp](https://github.com/projectdiscovery/rawhttp) | `0.1.91` | `0.1.92` |
| [github.com/projectdiscovery/retryablehttp-go](https://github.com/projectdiscovery/retryablehttp-go) | `1.3.24` | `1.3.25` |
| [github.com/projectdiscovery/yamldoc-go](https://github.com/projectdiscovery/yamldoc-go) | `1.0.6` | `1.0.7` |
| [github.com/projectdiscovery/httpx](https://github.com/projectdiscovery/httpx) | `1.10.0` | `1.11.0` |
| [github.com/projectdiscovery/networkpolicy](https://github.com/projectdiscovery/networkpolicy) | `0.1.48` | `0.1.49` |
| [github.com/projectdiscovery/utils](https://github.com/projectdiscovery/utils) | `0.11.2` | `0.11.3` |
| [github.com/projectdiscovery/wappalyzergo](https://github.com/projectdiscovery/wappalyzergo) | `0.2.95` | `0.2.96` |
| [github.com/projectdiscovery/cdncheck](https://github.com/projectdiscovery/cdncheck) | `1.2.50` | `1.2.51` |


Updates `github.com/projectdiscovery/fastdialer` from 0.5.18 to 0.5.19
- [Release notes](https://github.com/projectdiscovery/fastdialer/releases)
- [Commits](https://github.com/projectdiscovery/fastdialer/compare/v0.5.18...v0.5.19)

Updates `github.com/projectdiscovery/rawhttp` from 0.1.91 to 0.1.92
- [Release notes](https://github.com/projectdiscovery/rawhttp/releases)
- [Commits](https://github.com/projectdiscovery/rawhttp/compare/v0.1.91...v0.1.92)

Updates `github.com/projectdiscovery/retryablehttp-go` from 1.3.24 to 1.3.25
- [Release notes](https://github.com/projectdiscovery/retryablehttp-go/releases)
- [Commits](https://github.com/projectdiscovery/retryablehttp-go/compare/v1.3.24...v1.3.25)

Updates `github.com/projectdiscovery/yamldoc-go` from 1.0.6 to 1.0.7
- [Release notes](https://github.com/projectdiscovery/yamldoc-go/releases)
- [Commits](https://github.com/projectdiscovery/yamldoc-go/compare/v1.0.6...v1.0.7)

Updates `github.com/projectdiscovery/httpx` from 1.10.0 to 1.11.0
- [Release notes](https://github.com/projectdiscovery/httpx/releases)
- [Commits](https://github.com/projectdiscovery/httpx/compare/v1.10.0...v1.11.0)

Updates `github.com/projectdiscovery/networkpolicy` from 0.1.48 to 0.1.49
- [Release notes](https://github.com/projectdiscovery/networkpolicy/releases)
- [Commits](https://github.com/projectdiscovery/networkpolicy/compare/v0.1.48...v0.1.49)

Updates `github.com/projectdiscovery/utils` from 0.11.2 to 0.11.3
- [Release notes](https://github.com/projectdiscovery/utils/releases)
- [Changelog](https://github.com/projectdiscovery/utils/blob/main/CHANGELOG.md)
- [Commits](https://github.com/projectdiscovery/utils/compare/v0.11.2...v0.11.3)

Updates `github.com/projectdiscovery/wappalyzergo` from 0.2.95 to 0.2.96
- [Release notes](https://github.com/projectdiscovery/wappalyzergo/releases)
- [Commits](https://github.com/projectdiscovery/wappalyzergo/compare/v0.2.95...v0.2.96)

Updates `github.com/projectdiscovery/cdncheck` from 1.2.50 to 1.2.51
- [Release notes](https://github.com/projectdiscovery/cdncheck/releases)
- [Commits](https://github.com/projectdiscovery/cdncheck/compare/v1.2.50...v1.2.51)

---
updated-dependencies:
- dependency-name: github.com/projectdiscovery/fastdialer
  dependency-version: 0.5.19
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: modules
- dependency-name: github.com/projectdiscovery/rawhttp
  dependency-version: 0.1.92
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: modules
- dependency-name: github.com/projectdiscovery/retryablehttp-go
  dependency-version: 1.3.25
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: modules
- dependency-name: github.com/projectdiscovery/yamldoc-go
  dependency-version: 1.0.7
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: modules
- dependency-name: github.com/projectdiscovery/httpx
  dependency-version: 1.11.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: modules
- dependency-name: github.com/projectdiscovery/networkpolicy
  dependency-version: 0.1.49
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: modules
- dependency-name: github.com/projectdiscovery/utils
  dependency-version: 0.11.3
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: modules
- dependency-name: github.com/projectdiscovery/wappalyzergo
  dependency-version: 0.2.96
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: modules
- dependency-name: github.com/projectdiscovery/cdncheck
  dependency-version: 1.2.51
  dependency-type: indirect
  update-type: version-update:semver-patch
  dependency-group: modules
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-07 06:19:19 +00:00
Dwi Siswanto 37076fdc95 test(file): add word value to mock matcher
The shared fixture used an empty word matcher,
causing four tests to fail after operator
compilation began rejecting missing values.

Signed-off-by: Dwi Siswanto <[email protected]>
2026-09-07 10:12:45 +07:00
Dwi Siswanto 87cd437d67 fix(operators): reject matchers & extractors w/o values
Require at least one value for the selected
operator type during compilation. Empty lists
previously compiled without an error and produced
no results.

Fixes #7604

Signed-off-by: Dwi Siswanto <[email protected]>
2026-09-07 09:26:31 +07:00
f5d0ef81a5 feat(http): add RFC 9111 cache w/ per-request disable (#6746)
* feat(http): add RFC-9111 cache w/ per-request disable

* wrap HTTP transports with sandrolain/httpcache
  and expose global `--no-http-cache` flag.
* allow templates to set `disable-http-cache`.
* bump Go to 1.25.7.

Signed-off-by: Dwi Siswanto <[email protected]>

* ci: pin go version by go.mod

Signed-off-by: Dwi Siswanto <[email protected]>

* build: set GODEBUG=tlssha1=1

Signed-off-by: Dwi Siswanto <[email protected]>

* build(make): add more `GO*` envs for `go-build`

Signed-off-by: Dwi Siswanto <[email protected]>

* chore(cmd): fix typo

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>

* fix(types): add `DisableHTTPCache` to `Copy` method

Signed-off-by: Dwi Siswanto <[email protected]>

* test: add HTTP cache integration tests

Signed-off-by: Dwi Siswanto <[email protected]>

* refactor: inverse HTTP cache (RFC 9111) logic (opt-in)

Signed-off-by: Dwi Siswanto <[email protected]>

* chore(deps): go mod tidy

Signed-off-by: Dwi Siswanto <[email protected]>

* test(integration): add -http-cache to both HTTP cache test cases

Signed-off-by: Dwi Siswanto <[email protected]>

* filter cache

* bound cache

---------

Signed-off-by: Dwi Siswanto <[email protected]>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Co-authored-by: Mzack9999 <[email protected]>
2026-09-07 03:55:36 +07:00
Mzack9999 911c2303fe Merge pull request #7658 from lucasonline0/fix/7651-ignore-file-validation
fix(installer): validate nuclei ignore file downloads
2026-09-04 18:17:38 +04:00
Madureira dd2374604b test(installer): fix ignore file path assertion on Windows 2026-09-04 10:17:13 -03:00
Mzack9999 4fb9d22b68 validate ignorefile 2026-09-04 15:47:23 +04:00
Mzack9999 38dd02f13b merge dev 2026-09-04 15:42:17 +04:00
dependabot[bot] fe7b1df128 chore(deps): bump google.golang.org/grpc (#7699)
Bumps the go_modules group with 1 update in the / directory: [google.golang.org/grpc](https://github.com/grpc/grpc-go).


Updates `google.golang.org/grpc` from 1.82.1 to 1.83.1
- [Release notes](https://github.com/grpc/grpc-go/releases)
- [Commits](https://github.com/grpc/grpc-go/compare/v1.82.1...v1.83.1)

---
updated-dependencies:
- dependency-name: google.golang.org/grpc
  dependency-version: 1.83.1
  dependency-type: direct:production
  dependency-group: go_modules
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-09-04 08:38:06 +07:00
dw1andMzack9999 33d71f9156 refactor(installer): preserve local templates during updates (#7584)
* refactor(installer): preserve local templates during updates

Track files installed by each official release and
remove only retired templates whose contents still
match the recorded digest. Bootstrap ownership for
existing installations so upgrades keep unknown
and locally modified templates intact.

Write templates, ownership state, and config thru
synced temporary files. Recover interrupted
cleanup before another update and record the new
version only after metadata finalization succeeds.

Signed-off-by: Dwi Siswanto <[email protected]>

* Harden restore

* test(installer): compare restored permissions with source

Use the quarantined file's reported mode as the
expected value so the test works on cross-plats.

Signed-off-by: Dwi Siswanto <[email protected]>

* chore(installer): rm deprecated UpdateNucleiIgnoreHash call

Signed-off-by: Dwi Siswanto <[email protected]>

---------

Signed-off-by: Dwi Siswanto <[email protected]>
Co-authored-by: Mzack9999 <[email protected]>
2026-09-04 04:33:32 +07:00
Mzack9999 c126e2abcc Merge pull request #7698 from guardian360/fix/sdk-per-scan-filters-keep-ignore-excludes
fix(sdk): keep baseline ExcludeTags across per-execution filters
2026-09-04 00:12:52 +04:00
Mzack9999 d00461a2cc fix exclude 2026-09-03 23:18:36 +04:00
Niek den Breeje 09c1bbf51b fix(sdk): keep baseline ExcludeTags across per-execution filters
applyRequiredDefaults installs the .nuclei-ignore deny-list into
Options.ExcludeTags once, while the engine is built.
ExecuteNucleiWithOptsCtx applies per-execution options to a COPY of
those options afterwards, and WithTemplateFilters assigns the whole
filter set — so a literal naming only Tags cleared ExcludeTags.

Nothing re-reads the ignore file after construction, so every execution
that varied tags per call ran with an empty exclude list and executed
templates tagged dos, local, fuzz, bruteforce and txt-service. The
failure was silent. Varying filters across concurrent executions is the
documented purpose of the thread-safe engine, so this was reachable
through intended use.

Restore the engine's baseline exclusions after per-execution options are
applied: a per-execution filter can now add exclusions but not silently
discard them. IncludeTags is untouched and remains the explicit per-tag
override for callers who do want an ignored template to run.

No extra I/O — the deny-list is already on the base engine's options —
and only missing entries are appended, so the helper is idempotent
across executions against a long-lived engine.

Fixes #7695
2026-09-03 23:01:40 +04:00
Mzack9999 d7d7cf2b2c Merge pull request #7697 from guardian360/fix/sdk-ignore-file-files-section
fix(sdk): apply the files section of .nuclei-ignore
2026-09-03 22:50:09 +04:00
Mzack9999 689026b756 fix ignore 2026-09-03 20:55:15 +04:00
Mzack9999 d0e6a9d869 Merge pull request #7687 from ateto3/fix-dynamic-fetch-deadlock-7639
fix: perform request dump before auth/post-processors to avoid signat…
2026-09-03 19:14:42 +04:00
Mzack9999 80f1839aa3 Merge pull request #7281 from eyupcanakman/fix/js-template-flags-6645
fix(js): propagate proxy, timeout, and custom headers to JS templates
2026-09-03 18:40:52 +04:00
Mzack9999 fcbaa8f4b2 Merge pull request #7692 from projectdiscovery/dwisiswant0/fix/config/reject-corrupt-active-ignore-files
fix(config): reject corrupt active ignore files
2026-09-03 18:03:05 +04:00
Mzack9999 4138b68134 Merge pull request #7694 from projectdiscovery/dwisiswant0/feat/authx/sops-integration
feat(authx): SOPS integration
2026-09-03 17:33:28 +04:00
Niek den Breeje f66a832d8e fix(sdk): apply the files section of .nuclei-ignore
ReadIgnoreFile returns both `tags` and `files`, but applyRequiredDefaults
applied only `tags`, leaving ExcludedTemplates empty. The CLI runner
applies both, so the `files` section was inert for every SDK consumer.

That section exists to suppress templates known to have weak matchers,
so library users received false positives the CLI would never report.

Populate ExcludedTemplates from ignoreFile.Files, matching
internal/runner. ExcludedTemplates already flows to the loader as
Config.ExcludeTemplates, so no new plumbing is needed.

Fixes #7696
2026-09-03 11:27:12 +02:00
Mzack9999 dc5f464282 fix sops 2026-09-03 12:34:22 +04:00
Dwi Siswanto 2e22d4f612 chore: rm golangci config
Signed-off-by: Dwi Siswanto <[email protected]>
2026-09-03 14:54:34 +07:00
Dwi Siswanto 7ebe19d60b test(lib): add go.opencensus.io/stats/view to knownLeaks
Signed-off-by: Dwi Siswanto <[email protected]>
2026-09-03 14:20:28 +07:00
Dwi Siswanto 2ba82f722f chore: add golangci-lint confing
Signed-off-by: Dwi Siswanto <[email protected]>
2026-09-03 14:19:32 +07:00
Dwi Siswanto 82071fe0c9 chore: add SOPS creation rules file
Signed-off-by: Dwi Siswanto <[email protected]>
2026-09-03 14:18:58 +07:00
Dwi Siswanto 550b92da42 chore: tidy up mod
Signed-off-by: Dwi Siswanto <[email protected]>
2026-09-03 14:18:57 +07:00
dw1andMzack9999 a34f810ffa refactor(exporters): improve exported sarif rule and result metadata (#7368)
* refactor(exporters): improve exported sarif rule and result metadata

Normalize rule ids and descriptions, attach help
text, references and GitHub security properties,
and enrich result messages with target, matcher,
extractor, extracted values and reproduction
details.

Keep rule indices stable, generate better artifact
locations, mark the invocation successful, and
export with normalization and empty-object pruning
enabled.

Also adapt the exporter to the updated sarif API.

Signed-off-by: Dwi Siswanto <[email protected]>

* adding tests

---------

Signed-off-by: Dwi Siswanto <[email protected]>
Co-authored-by: Mzack9999 <[email protected]>
2026-09-03 13:52:49 +07:00
Dwi Siswanto 6d4d1be80c fix(config): reject corrupt active ignore files
Return .nuclei-ignore read and parse failures to
callers instead of continuing with an empty
exclusion set.

Treat only a missing active ignore file as
recoverable and log it as a warning. Apply this
behavior to the CLI, SDK, and loader paths.

Fixes #7691

Signed-off-by: Dwi Siswanto <[email protected]>
2026-09-03 11:02:09 +07:00
ateto3 46ea619bc6 Merge branch 'projectdiscovery:dev' into fix-dynamic-fetch-deadlock-7639 2026-09-02 23:03:11 +02:00
Mzack9999 87fbc2d045 fix deadlock 2026-09-02 22:44:23 +04:00
Mzack9999 6ec0e9e52c js flags 2026-09-02 22:14:05 +04:00
Eyüp Can Akman 91ffec51a1 fix lint 2026-09-02 22:00:47 +04:00
Eyüp Can Akman 24a101fbdf fix(js): support https proxies in net helpers 2026-09-02 22:00:47 +04:00
Eyüp Can Akman eb10c9f160 fix(js): propagate proxy, timeout, and custom headers to JS templates
Fixes #6645
2026-09-02 22:00:47 +04:00
Lucas Madureira e908de14fa fix(installer): validate nuclei ignore file downloads 2026-08-14 09:10:47 -03:00
158 changed files with 12494 additions and 1259 deletions

No files matched your search

+1 -2
View File
@@ -13,8 +13,7 @@ jobs:
permissions:
contents: write
steps:
- uses: actions/checkout@v7
- uses: actions/checkout@v7.0.1
- uses: projectdiscovery/actions/setup/go/compat-checks@v1
with:
go-version: "stable"
release-test: true
+1 -1
View File
@@ -12,7 +12,7 @@ jobs:
TARGET_URL: "http://honey.scanme.sh/-/?foo=bar"
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
- uses: actions/checkout@v7.0.1
- uses: projectdiscovery/actions/setup/go@v1
- uses: projectdiscovery/nuclei-action/cache@v3
- run: make build
+2 -2
View File
@@ -17,7 +17,7 @@ jobs:
outputs:
matrix: ${{ steps.discover.outputs.matrix }}
steps:
- uses: actions/checkout@v7
- uses: actions/checkout@v7.0.1
- uses: projectdiscovery/actions/setup/go@v1
with:
go-version-file: go.tool.mod
@@ -37,7 +37,7 @@ jobs:
GOFUZZ_PACKAGE: ${{ matrix.pkg }}
FUZZ_DURATION: 15m
steps:
- uses: actions/checkout@v7
- uses: actions/checkout@v7.0.1
- uses: projectdiscovery/actions/setup/go@v1
with:
go-version-file: go.tool.mod
+1 -1
View File
@@ -11,7 +11,7 @@ jobs:
if: ${{ !endsWith(github.actor, '[bot]') }}
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
- uses: actions/checkout@v7.0.1
- uses: projectdiscovery/actions/setup/go@v1
- uses: projectdiscovery/actions/setup/git@v1
- run: make syntax-docs
+1 -1
View File
@@ -18,7 +18,7 @@ jobs:
TARGET_LIST: "/tmp/targets.txt"
PROFILE_MEM: "/tmp/nuclei-profile"
steps:
- uses: actions/checkout@v7
- uses: actions/checkout@v7.0.1
- uses: projectdiscovery/actions/setup/go@v1
- uses: projectdiscovery/nuclei-action/cache@v3
- run: |
+1 -1
View File
@@ -16,7 +16,7 @@ jobs:
env:
OUTPUT: "/tmp/results.sarif"
steps:
- uses: actions/checkout@v7
- uses: actions/checkout@v7.0.1
- uses: projectdiscovery/actions/setup/go@v1
with:
go-version-file: go.tool.mod
+1 -1
View File
@@ -14,7 +14,7 @@ jobs:
if: ${{ github.repository == 'projectdiscovery/nuclei' && !endsWith(github.actor, '[bot]') }}
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
- uses: actions/checkout@v7.0.1
- uses: projectdiscovery/actions/setup/go@v1
- uses: projectdiscovery/actions/setup/git@v1
- run: make memogen
+1 -1
View File
@@ -11,7 +11,7 @@ jobs:
env:
BENCH_OUT: "/tmp/bench.out"
steps:
- uses: actions/checkout@v7
- uses: actions/checkout@v7.0.1
- uses: projectdiscovery/actions/setup/go@v1
- uses: projectdiscovery/nuclei-action/cache@v3
- run: make build-test
+1 -1
View File
@@ -16,7 +16,7 @@ jobs:
needs: ["pgo"]
runs-on: ubuntu-latest-16-cores
steps:
- uses: actions/checkout@v7
- uses: actions/checkout@v7.0.1
with:
fetch-depth: 0
- uses: actions/download-artifact@v8
+1 -1
View File
@@ -23,7 +23,7 @@ jobs:
# A workflow can initially run from the branch that adds this file. The
# API analyzes the repository default branch, so check out that same
# branch before calculating the HEAD we wait for.
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
ref: ${{ github.event.repository.default_branch }}
+8 -10
View File
@@ -22,7 +22,7 @@ jobs:
if: ${{ !endsWith(github.actor, '[bot]') }}
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
- uses: actions/checkout@v7.0.1
- uses: projectdiscovery/actions/setup/go@v1
- uses: projectdiscovery/nuclei-action/cache@v3
- uses: projectdiscovery/actions/golangci-lint/v2@v1
@@ -42,7 +42,7 @@ jobs:
os: [ubuntu-latest, windows-latest, macOS-latest]
runs-on: "${{ matrix.os }}"
steps:
- uses: actions/checkout@v7
- uses: actions/checkout@v7.0.1
- uses: projectdiscovery/actions/setup/go@v1
- uses: projectdiscovery/nuclei-action/cache@v3
- uses: projectdiscovery/actions/free-disk-space@v1
@@ -90,7 +90,7 @@ jobs:
env:
GITHUB_TOKEN: "${{ github.token }}"
steps:
- uses: actions/checkout@v7
- uses: actions/checkout@v7.0.1
- uses: projectdiscovery/actions/setup/go@v1
- uses: projectdiscovery/nuclei-action/cache@v3
- name: "Simple"
@@ -115,7 +115,7 @@ jobs:
os: [ubuntu-latest, windows-latest, macOS-latest]
runs-on: ${{ matrix.os }}
steps:
- uses: actions/checkout@v7
- uses: actions/checkout@v7.0.1
- uses: projectdiscovery/actions/setup/python@v1
- uses: projectdiscovery/actions/setup/go@v1
- uses: projectdiscovery/nuclei-action/cache@v3
@@ -135,7 +135,7 @@ jobs:
os: [ubuntu-latest, windows-latest, macOS-latest]
runs-on: ${{ matrix.os }}
steps:
- uses: actions/checkout@v7
- uses: actions/checkout@v7.0.1
- uses: projectdiscovery/actions/setup/python@v1
- uses: projectdiscovery/actions/setup/go@v1
- uses: projectdiscovery/nuclei-action/cache@v3
@@ -152,7 +152,7 @@ jobs:
env:
GITHUB_TOKEN: "${{ github.token }}"
steps:
- uses: actions/checkout@v7
- uses: actions/checkout@v7.0.1
- uses: projectdiscovery/actions/setup/go@v1
- uses: projectdiscovery/nuclei-action/cache@v3
- run: make template-validate
@@ -166,7 +166,7 @@ jobs:
contents: read
security-events: write
steps:
- uses: actions/checkout@v7
- uses: actions/checkout@v7.0.1
- uses: github/codeql-action/init@v4
with:
languages: 'go'
@@ -204,10 +204,8 @@ jobs:
needs: ["tests"]
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
- uses: actions/checkout@v7.0.1
- uses: projectdiscovery/actions/setup/go@v1
with:
go-version: "stable"
- uses: projectdiscovery/actions/goreleaser@v1
flamegraph:
+2
View File
@@ -33,6 +33,8 @@
# Templates
/*.yaml
!/.sops.yaml
!/.golangci.yaml
# Generated docs
/dsl.md
+1
View File
@@ -12,6 +12,7 @@ builds:
env:
- CGO_ENABLED=0
- GOEXPERIMENT=greenteagc
- GODEBUG=tlssha1=1
goos: [windows, linux, darwin]
goarch: [amd64, '386', arm, arm64]
ignore:
+4
View File
@@ -0,0 +1,4 @@
# SOPS Creation Rules for Secret File (ref: https://docs.projectdiscovery.io/tools/nuclei/authenticated-scans)
creation_rules:
- encrypted_regex: ^(password|username|token|value|key|raw)$
+7 -4
View File
@@ -33,8 +33,9 @@ clean:
go-build: clean
go-build:
CGO_ENABLED=0 $(GOBUILD) -trimpath $(GOFLAGS) -ldflags '${LDFLAGS}' $(GOBUILD_ADDITIONAL_ARGS) \
-o '${GOBUILD_OUTPUT}${GOBUILD_OUTPUT_EXT}' $(GOBUILD_PACKAGES)
CGO_ENABLED=0 GOEXPERIMENT=greenteagc GODEBUG=tlssha1=1 $(GOBUILD) \
-trimpath $(GOFLAGS) -ldflags '${LDFLAGS}' $(GOBUILD_ADDITIONAL_ARGS) \
-o '${GOBUILD_OUTPUT}${GOBUILD_OUTPUT_EXT}' $(GOBUILD_PACKAGES)
build: GOFLAGS = -pgo=auto
build: GOBUILD_OUTPUT = ./bin/nuclei
@@ -90,8 +91,10 @@ test: GOFLAGS = $(RACE) -v -timeout 1h -count 1
test:
$(GOTEST) $(GOFLAGS) ./...
# Keep this under the GitHub Actions step timeout (50m) so a hung nuclei process
# is reported by go test instead of the runner killing the job with no test name.
integration:
$(GOTEST) -tags=integration -timeout 1h ./internal/tests/integration
$(GOTEST) -tags=integration -timeout 40m ./internal/tests/integration
integration-debug:
$(GOTEST) -tags=integration ./internal/tests/integration -v $(GO_TEST_ARGS) -args $(INTEGRATION_ARGS)
@@ -297,4 +300,4 @@ template-validate:
./bin/nuclei -validate \
-w workflows \
-et http/technologies \
-ept code
-ept code
+4 -1
View File
@@ -235,7 +235,7 @@ CONFIGURATIONS:
-ztls use ztls library with autofallback to standard one for tls13 [Deprecated] autofallback to ztls is enabled by default
-sni string tls sni hostname to use (default: input domain name)
-dka, -dialer-keep-alive value keep-alive duration for network requests.
-lfa, -allow-local-file-access allows file (payload) access anywhere on the system
-lfa, -allow-local-file-access allow all templates in the run to read any file on the system (not just payloads)
-lna, -restrict-local-network-access blocks connections to the local / private network
-i, -interface string network interface to use for network scan
-at, -attack-type string type of payload combinations to perform (batteringram,pitchfork,clusterbomb)
@@ -264,6 +264,9 @@ FUZZING:
-dtr, -dast-report write dast scan report to file
-dtst, -dast-server-token string dast server token (optional)
-dtsa, -dast-server-address string dast server address (default "localhost:9055")
-dtp, -dast-proxy enable dast proxy mode (fuzz live traffic proxied through nuclei)
-dtpa, -dast-proxy-address string dast proxy listen address (default "127.0.0.1:9056")
-dtpau, -dast-proxy-auth string dast proxy basic auth in user:pass format (required for non-loopback address)
-dfp, -display-fuzz-points display fuzz points in the output for debugging
-fuzz-param-frequency int frequency of uninteresting parameters for fuzzing before skipping (default 10)
-fa, -fuzz-aggression string fuzzing aggression level controls payload count for fuzz (low, medium, high) (default "low")
+4 -1
View File
@@ -235,7 +235,7 @@ CONFIGURATIONS:
-ztls use ztls library with autofallback to standard one for tls13 [Deprecated] autofallback to ztls is enabled by default
-sni string tls sni hostname to use (default: input domain name)
-dka, -dialer-keep-alive value keep-alive duration for network requests.
-lfa, -allow-local-file-access allows file (payload) access anywhere on the system
-lfa, -allow-local-file-access allow all templates in the run to read any file on the system (not just payloads)
-lna, -restrict-local-network-access blocks connections to the local / private network
-i, -interface string network interface to use for network scan
-at, -attack-type string type of payload combinations to perform (batteringram,pitchfork,clusterbomb)
@@ -264,6 +264,9 @@ FUZZING:
-dtr, -dast-report write dast scan report to file
-dtst, -dast-server-token string dast server token (optional)
-dtsa, -dast-server-address string dast server address (default "localhost:9055")
-dtp, -dast-proxy enable dast proxy mode (fuzz live traffic proxied through nuclei)
-dtpa, -dast-proxy-address string dast proxy listen address (default "127.0.0.1:9056")
-dtpau, -dast-proxy-auth string dast proxy basic auth in user:pass format (required for non-loopback address)
-dfp, -display-fuzz-points display fuzz points in the output for debugging
-fuzz-param-frequency int frequency of uninteresting parameters for fuzzing before skipping (default 10)
-fa, -fuzz-aggression string fuzzing aggression level controls payload count for fuzz (low, medium, high) (default "low")
+4 -1
View File
@@ -235,7 +235,7 @@ CONFIGURATIONS:
-ztls use ztls library with autofallback to standard one for tls13 [Deprecated] autofallback to ztls is enabled by default
-sni string tls sni hostname to use (default: input domain name)
-dka, -dialer-keep-alive value keep-alive duration for network requests.
-lfa, -allow-local-file-access allows file (payload) access anywhere on the system
-lfa, -allow-local-file-access allow all templates in the run to read any file on the system (not just payloads)
-lna, -restrict-local-network-access blocks connections to the local / private network
-i, -interface string network interface to use for network scan
-at, -attack-type string type of payload combinations to perform (batteringram,pitchfork,clusterbomb)
@@ -264,6 +264,9 @@ FUZZING:
-dtr, -dast-report write dast scan report to file
-dtst, -dast-server-token string dast server token (optional)
-dtsa, -dast-server-address string dast server address (default "localhost:9055")
-dtp, -dast-proxy enable dast proxy mode (fuzz live traffic proxied through nuclei)
-dtpa, -dast-proxy-address string dast proxy listen address (default "127.0.0.1:9056")
-dtpau, -dast-proxy-auth string dast proxy basic auth in user:pass format (required for non-loopback address)
-dfp, -display-fuzz-points display fuzz points in the output for debugging
-fuzz-param-frequency int frequency of uninteresting parameters for fuzzing before skipping (default 10)
-fa, -fuzz-aggression string fuzzing aggression level controls payload count for fuzz (low, medium, high) (default "low")
+4 -1
View File
@@ -235,7 +235,7 @@ CONFIGURATIONS:
-ztls use ztls library with autofallback to standard one for tls13 [Deprecated] autofallback to ztls is enabled by default
-sni string tls sni hostname to use (default: input domain name)
-dka, -dialer-keep-alive value keep-alive duration for network requests.
-lfa, -allow-local-file-access allows file (payload) access anywhere on the system
-lfa, -allow-local-file-access allow all templates in the run to read any file on the system (not just payloads)
-lna, -restrict-local-network-access blocks connections to the local / private network
-i, -interface string network interface to use for network scan
-at, -attack-type string type of payload combinations to perform (batteringram,pitchfork,clusterbomb)
@@ -264,6 +264,9 @@ FUZZING:
-dtr, -dast-report write dast scan report to file
-dtst, -dast-server-token string dast server token (optional)
-dtsa, -dast-server-address string dast server address (default "localhost:9055")
-dtp, -dast-proxy enable dast proxy mode (fuzz live traffic proxied through nuclei)
-dtpa, -dast-proxy-address string dast proxy listen address (default "127.0.0.1:9056")
-dtpau, -dast-proxy-auth string dast proxy basic auth in user:pass format (required for non-loopback address)
-dfp, -display-fuzz-points display fuzz points in the output for debugging
-fuzz-param-frequency int frequency of uninteresting parameters for fuzzing before skipping (default 10)
-fa, -fuzz-aggression string fuzzing aggression level controls payload count for fuzz (low, medium, high) (default "low")
+4 -1
View File
@@ -235,7 +235,7 @@ CONFIGURATIONS:
-ztls use ztls library with autofallback to standard one for tls13 [Deprecated] autofallback to ztls is enabled by default
-sni string tls sni hostname to use (default: input domain name)
-dka, -dialer-keep-alive value keep-alive duration for network requests.
-lfa, -allow-local-file-access allows file (payload) access anywhere on the system
-lfa, -allow-local-file-access allow all templates in the run to read any file on the system (not just payloads)
-lna, -restrict-local-network-access blocks connections to the local / private network
-i, -interface string network interface to use for network scan
-at, -attack-type string type of payload combinations to perform (batteringram,pitchfork,clusterbomb)
@@ -264,6 +264,9 @@ FUZZING:
-dtr, -dast-report write dast scan report to file
-dtst, -dast-server-token string dast server token (optional)
-dtsa, -dast-server-address string dast server address (default "localhost:9055")
-dtp, -dast-proxy enable dast proxy mode (fuzz live traffic proxied through nuclei)
-dtpa, -dast-proxy-address string dast proxy listen address (default "127.0.0.1:9056")
-dtpau, -dast-proxy-auth string dast proxy basic auth in user:pass format (required for non-loopback address)
-dfp, -display-fuzz-points display fuzz points in the output for debugging
-fuzz-param-frequency int frequency of uninteresting parameters for fuzzing before skipping (default 10)
-fa, -fuzz-aggression string fuzzing aggression level controls payload count for fuzz (low, medium, high) (default "low")
+4 -1
View File
@@ -235,7 +235,7 @@ CONFIGURATIONS:
-ztls use ztls library with autofallback to standard one for tls13 [Deprecated] autofallback to ztls is enabled by default
-sni string tls sni hostname to use (default: input domain name)
-dka, -dialer-keep-alive value keep-alive duration for network requests.
-lfa, -allow-local-file-access allows file (payload) access anywhere on the system
-lfa, -allow-local-file-access allow all templates in the run to read any file on the system (not just payloads)
-lna, -restrict-local-network-access blocks connections to the local / private network
-i, -interface string network interface to use for network scan
-at, -attack-type string type of payload combinations to perform (batteringram,pitchfork,clusterbomb)
@@ -264,6 +264,9 @@ FUZZING:
-dtr, -dast-report write dast scan report to file
-dtst, -dast-server-token string dast server token (optional)
-dtsa, -dast-server-address string dast server address (default "localhost:9055")
-dtp, -dast-proxy enable dast proxy mode (fuzz live traffic proxied through nuclei)
-dtpa, -dast-proxy-address string dast proxy listen address (default "127.0.0.1:9056")
-dtpau, -dast-proxy-auth string dast proxy basic auth in user:pass format (required for non-loopback address)
-dfp, -display-fuzz-points display fuzz points in the output for debugging
-fuzz-param-frequency int frequency of uninteresting parameters for fuzzing before skipping (default 10)
-fa, -fuzz-aggression string fuzzing aggression level controls payload count for fuzz (low, medium, high) (default "low")
+4 -1
View File
@@ -235,7 +235,7 @@ CONFIGURATIONS:
-ztls use ztls library with autofallback to standard one for tls13 [Deprecated] autofallback to ztls is enabled by default
-sni string tls sni hostname to use (default: input domain name)
-dka, -dialer-keep-alive value keep-alive duration for network requests.
-lfa, -allow-local-file-access allows file (payload) access anywhere on the system
-lfa, -allow-local-file-access allow all templates in the run to read any file on the system (not just payloads)
-lna, -restrict-local-network-access blocks connections to the local / private network
-i, -interface string network interface to use for network scan
-at, -attack-type string type of payload combinations to perform (batteringram,pitchfork,clusterbomb)
@@ -264,6 +264,9 @@ FUZZING:
-dtr, -dast-report write dast scan report to file
-dtst, -dast-server-token string dast server token (optional)
-dtsa, -dast-server-address string dast server address (default "localhost:9055")
-dtp, -dast-proxy enable dast proxy mode (fuzz live traffic proxied through nuclei)
-dtpa, -dast-proxy-address string dast proxy listen address (default "127.0.0.1:9056")
-dtpau, -dast-proxy-auth string dast proxy basic auth in user:pass format (required for non-loopback address)
-dfp, -display-fuzz-points display fuzz points in the output for debugging
-fuzz-param-frequency int frequency of uninteresting parameters for fuzzing before skipping (default 10)
-fa, -fuzz-aggression string fuzzing aggression level controls payload count for fuzz (low, medium, high) (default "low")
+4 -1
View File
@@ -235,7 +235,7 @@ CONFIGURATIONS:
-ztls use ztls library with autofallback to standard one for tls13 [Deprecated] autofallback to ztls is enabled by default
-sni string tls sni hostname to use (default: input domain name)
-dka, -dialer-keep-alive value keep-alive duration for network requests.
-lfa, -allow-local-file-access allows file (payload) access anywhere on the system
-lfa, -allow-local-file-access allow all templates in the run to read any file on the system (not just payloads)
-lna, -restrict-local-network-access blocks connections to the local / private network
-i, -interface string network interface to use for network scan
-at, -attack-type string type of payload combinations to perform (batteringram,pitchfork,clusterbomb)
@@ -264,6 +264,9 @@ FUZZING:
-dtr, -dast-report write dast scan report to file
-dtst, -dast-server-token string dast server token (optional)
-dtsa, -dast-server-address string dast server address (default "localhost:9055")
-dtp, -dast-proxy enable dast proxy mode (fuzz live traffic proxied through nuclei)
-dtpa, -dast-proxy-address string dast proxy listen address (default "127.0.0.1:9056")
-dtpau, -dast-proxy-auth string dast proxy basic auth in user:pass format (required for non-loopback address)
-dfp, -display-fuzz-points display fuzz points in the output for debugging
-fuzz-param-frequency int frequency of uninteresting parameters for fuzzing before skipping (default 10)
-fa, -fuzz-aggression string fuzzing aggression level controls payload count for fuzz (low, medium, high) (default "low")
+13
View File
@@ -1477,6 +1477,19 @@ DisableCookie is an optional setting that disables cookie reuse
<div class="dd">
<code>disable-http-cache</code> <i>bool</i>
</div>
<div class="dt">
DisableHTTPCache turns off HTTP caching for this request. It cannot turn caching on when -http-cache is unset.
</div>
<hr />
<div class="dd">
<code>read-all</code> <i>bool</i>
</div>
+5 -1
View File
@@ -369,7 +369,7 @@ on extensive configurability, massive extensibility and ease of use.`)
flagSet.BoolVar(&options.ZTLS, "ztls", false, "use ztls library with autofallback to standard one for tls13 [Deprecated] autofallback to ztls is enabled by default"), //nolint:all
flagSet.StringVar(&options.SNI, "sni", "", "tls sni hostname to use (default: input domain name)"),
flagSet.DurationVarP(&options.DialerKeepAlive, "dialer-keep-alive", "dka", 0, "keep-alive duration for network requests."),
flagSet.BoolVarP(&options.AllowLocalFileAccess, "allow-local-file-access", "lfa", false, "allows file (payload) access anywhere on the system"),
flagSet.BoolVarP(&options.AllowLocalFileAccess, "allow-local-file-access", "lfa", false, "allow all templates in the run to read any file on the system (not just payloads)"),
flagSet.BoolVarP(&options.RestrictLocalNetworkAccess, "restrict-local-network-access", "lna", false, "blocks connections to the local / private network"),
flagSet.StringVarP(&options.Interface, "interface", "i", "", "network interface to use for network scan"),
flagSet.StringVarP(&options.AttackType, "attack-type", "at", "", "type of payload combinations to perform (batteringram,pitchfork,clusterbomb)"),
@@ -400,6 +400,9 @@ on extensive configurability, massive extensibility and ease of use.`)
flagSet.BoolVarP(&options.DASTReport, "dast-report", "dtr", false, "write dast scan report to file"),
flagSet.StringVarP(&options.DASTServerToken, "dast-server-token", "dtst", "", "dast server token (optional)"),
flagSet.StringVarP(&options.DASTServerAddress, "dast-server-address", "dtsa", "localhost:9055", "dast server address"),
flagSet.BoolVarP(&options.DASTProxy, "dast-proxy", "dtp", false, "enable intercepting dast proxy (CA/stats on -dtsa; POST /fuzz needs -dast-server)"),
flagSet.StringVarP(&options.DASTProxyAddress, "dast-proxy-address", "dtpa", "127.0.0.1:9056", "dast proxy listen address"),
flagSet.StringVarP(&options.DASTProxyAuth, "dast-proxy-auth", "dtpau", "", "dast proxy basic auth in user:pass format (required for non-loopback address)"),
flagSet.BoolVarP(&options.DisplayFuzzPoints, "display-fuzz-points", "dfp", false, "display fuzz points in the output for debugging"),
flagSet.IntVar(&options.FuzzParamFrequency, "fuzz-param-frequency", 10, "frequency of uninteresting parameters for fuzzing before skipping"),
flagSet.StringVarP(&options.FuzzAggressionLevel, "fuzz-aggression", "fa", "low", "fuzzing aggression level controls payload count for fuzz (low, medium, high)"),
@@ -450,6 +453,7 @@ on extensive configurability, massive extensibility and ease of use.`)
flagSet.BoolVarP(&options.DisableHTTPProbe, "no-httpx", "nh", false, "disable httpx probing for non-url input"),
flagSet.BoolVar(&options.PreflightPortScan, "preflight-portscan", false, "run preflight resolve + TCP portscan and filter targets before scanning (disabled by default)"),
flagSet.BoolVar(&options.DisableStdin, "no-stdin", false, "disable stdin processing"),
flagSet.BoolVar(&options.EnableHTTPCache, "http-cache", false, "enable HTTP cache (RFC 9111) for HTTP requests"),
)
flagSet.CreateGroup("headless", "Headless",
+96 -46
View File
@@ -1,6 +1,6 @@
module github.com/projectdiscovery/nuclei/v3
go 1.26
go 1.26.0
require (
github.com/andygrunwald/go-jira v1.16.1
@@ -13,17 +13,17 @@ require (
github.com/json-iterator/go v1.1.12 // indirect
github.com/julienschmidt/httprouter v1.3.0
github.com/logrusorgru/aurora v2.0.3+incompatible // indirect
github.com/miekg/dns v1.1.72
github.com/miekg/dns v1.1.73
github.com/olekukonko/tablewriter v1.0.8
github.com/pkg/errors v0.9.1
github.com/projectdiscovery/clistats v0.1.6
github.com/projectdiscovery/fastdialer v0.5.18
github.com/projectdiscovery/clistats v0.1.7
github.com/projectdiscovery/fastdialer v0.5.20
github.com/projectdiscovery/hmap v0.0.102
github.com/projectdiscovery/interactsh v1.3.1
github.com/projectdiscovery/rawhttp v0.1.91
github.com/projectdiscovery/rawhttp v0.1.92
github.com/projectdiscovery/retryabledns v1.0.116
github.com/projectdiscovery/retryablehttp-go v1.3.24
github.com/projectdiscovery/yamldoc-go v1.0.6
github.com/projectdiscovery/retryablehttp-go v1.3.26
github.com/projectdiscovery/yamldoc-go v1.0.7
github.com/remeh/sizedwaitgroup v1.0.0
github.com/rs/xid v1.6.0
github.com/segmentio/ksuid v1.0.4
@@ -32,14 +32,16 @@ require (
github.com/syndtr/goleveldb v1.0.0
github.com/weppos/publicsuffix-go v0.50.3 // indirect
go.uber.org/multierr v1.11.0
golang.org/x/net v0.57.0
golang.org/x/net v0.58.0
golang.org/x/oauth2 v0.36.0
golang.org/x/text v0.40.0
golang.org/x/text v0.41.0
golang.org/x/time v0.15.0
)
require (
carvel.dev/ytt v0.52.0
code.gitea.io/sdk/gitea v0.17.0
filippo.io/age v1.2.1
github.com/Azure/azure-sdk-for-go/sdk/azidentity v1.10.1
github.com/Azure/azure-sdk-for-go/sdk/storage/azblob v1.1.0
github.com/Azure/go-ntlmssp v0.1.1
@@ -55,19 +57,21 @@ require (
github.com/antchfx/xmlquery v1.4.4
github.com/antchfx/xpath v1.3.6
github.com/aws/aws-sdk-go-v2 v1.41.5
github.com/aws/aws-sdk-go-v2/config v1.29.17
github.com/aws/aws-sdk-go-v2/credentials v1.17.70
github.com/aws/aws-sdk-go-v2/feature/s3/manager v1.17.82
github.com/aws/aws-sdk-go-v2/config v1.31.16
github.com/aws/aws-sdk-go-v2/credentials v1.18.20
github.com/aws/aws-sdk-go-v2/feature/s3/manager v1.20.2
github.com/aws/aws-sdk-go-v2/service/s3 v1.99.0
github.com/bytedance/sonic v1.15.0
github.com/bytedance/sonic v1.15.3
github.com/cespare/xxhash v1.1.0
github.com/charmbracelet/glamour v0.10.0
github.com/clbanning/mxj/v2 v2.7.0
github.com/ditashi/jsbeautifier-go v0.0.0-20141206144643-2520a8026a9c
github.com/docker/go-units v0.5.0
github.com/elazarl/goproxy v1.7.2
github.com/fatih/structs v1.1.0
github.com/fullstorydev/grpcurl v1.9.3
github.com/getkin/kin-openapi v0.144.0
github.com/getsops/sops/v3 v3.9.2
github.com/go-git/go-git/v5 v5.19.2
github.com/go-ldap/ldap/v3 v3.4.12
github.com/go-pdf/fpdf v0.9.0
@@ -92,30 +96,31 @@ require (
github.com/oiweiwei/go-msrpc v1.2.12
github.com/ory/dockertest/v3 v3.12.0
github.com/praetorian-inc/fingerprintx v1.1.15
github.com/projectdiscovery/dsl v0.8.22
github.com/projectdiscovery/dsl v0.8.23-0.20260910174914-8e1207b367b8
github.com/projectdiscovery/fasttemplate v0.0.2
github.com/projectdiscovery/gcache v0.0.0-20241015120333-12546c6e3f4c
github.com/projectdiscovery/goflags v0.2.0
github.com/projectdiscovery/goflags v0.2.1
github.com/projectdiscovery/goja v0.0.0-20260618133720-acb73e419534
github.com/projectdiscovery/goja_nodejs v0.0.0-20260618132410-8519f75f703d
github.com/projectdiscovery/gologger v1.1.72
github.com/projectdiscovery/gologger v1.1.73
github.com/projectdiscovery/gostruct v0.0.2
github.com/projectdiscovery/govaluate v0.0.0-20260615100919-5ee2581bbf7e
github.com/projectdiscovery/gozero v0.1.1-0.20260530071156-fa1dad563d76
github.com/projectdiscovery/httpx v1.10.0
github.com/projectdiscovery/httpx v1.12.0
github.com/projectdiscovery/mapcidr v1.1.97
github.com/projectdiscovery/n3iwf v0.0.0-20230523120440-b8cd232ff1f5
github.com/projectdiscovery/networkpolicy v0.1.48
github.com/projectdiscovery/ratelimit v0.0.88
github.com/projectdiscovery/networkpolicy v0.1.50
github.com/projectdiscovery/ratelimit v0.0.90-0.20260912164306-aeeca5645294
github.com/projectdiscovery/rdap v0.9.0
github.com/projectdiscovery/sarif v0.1.0
github.com/projectdiscovery/tlsx v1.3.2
github.com/projectdiscovery/tlsx v1.4.0
github.com/projectdiscovery/uncover v1.2.1
github.com/projectdiscovery/useragent v0.0.109
github.com/projectdiscovery/utils v0.11.2
github.com/projectdiscovery/wappalyzergo v0.2.95
github.com/projectdiscovery/utils v0.11.4-0.20260910160917-83d6b6e62668
github.com/projectdiscovery/wappalyzergo v0.3.0
github.com/redis/go-redis/v9 v9.11.0
github.com/rs/zerolog v1.34.0
github.com/sandrolain/httpcache v1.4.0
github.com/shurcooL/graphql v0.0.0-20230722043721-ed46e5a46466
github.com/sijms/go-ora/v2 v2.9.0
github.com/stretchr/testify v1.12.1
@@ -126,27 +131,41 @@ require (
go.mongodb.org/mongo-driver v1.17.9
golang.org/x/sync v0.22.0
golang.org/x/term v0.45.0
google.golang.org/grpc v1.82.1
google.golang.org/grpc v1.83.1
gopkg.in/yaml.v3 v3.0.1
moul.io/http2curl v1.0.0
)
require (
aead.dev/minisign v0.3.0 // indirect
cel.dev/expr v0.25.2 // indirect
cloud.google.com/go v0.123.0 // indirect
cloud.google.com/go/auth v0.18.2 // indirect
cloud.google.com/go/auth/oauth2adapt v0.2.8 // indirect
cloud.google.com/go/compute/metadata v0.9.0 // indirect
cloud.google.com/go/iam v1.5.3 // indirect
cloud.google.com/go/kms v1.25.0 // indirect
cloud.google.com/go/longrunning v0.8.0 // indirect
cloud.google.com/go/monitoring v1.24.3 // indirect
cloud.google.com/go/storage v1.56.0 // indirect
dario.cat/mergo v1.0.2 // indirect
filippo.io/edwards25519 v1.2.0 // indirect
git.mills.io/prologic/smtpd v0.0.0-20210710122116-a525b76c287a // indirect
github.com/Azure/azure-sdk-for-go/sdk/azcore v1.18.0 // indirect
github.com/Azure/azure-sdk-for-go/sdk/internal v1.11.1 // indirect
github.com/Azure/azure-sdk-for-go/sdk/security/keyvault/azkeys v1.3.1 // indirect
github.com/Azure/azure-sdk-for-go/sdk/security/keyvault/internal v1.1.1 // indirect
github.com/Azure/go-ansiterm v0.0.0-20250102033503-faa5f7b0171c // indirect
github.com/AzureAD/microsoft-authentication-library-for-go v1.4.2 // indirect
github.com/BurntSushi/toml v1.4.0 // indirect
github.com/GoogleCloudPlatform/opentelemetry-operations-go/detectors/gcp v1.33.0 // indirect
github.com/GoogleCloudPlatform/opentelemetry-operations-go/exporter/metric v0.53.0 // indirect
github.com/GoogleCloudPlatform/opentelemetry-operations-go/internal/resourcemapping v0.53.0 // indirect
github.com/Microsoft/go-winio v0.6.2 // indirect
github.com/Mzack9999/gcache v0.0.0-20230410081825-519e28eab057 // indirect
github.com/Mzack9999/go-http-digest-auth-client v0.6.1-0.20220414142836-eb8883508809 // indirect
github.com/Nvveen/Gotty v0.0.0-20120604004816-cd527374f1e5 // indirect
github.com/ProtonMail/go-crypto v1.1.6 // indirect
github.com/PuerkitoBio/goquery v1.12.0 // indirect
github.com/PuerkitoBio/goquery v1.13.0 // indirect
github.com/STARRY-S/zip v0.2.3 // indirect
github.com/VividCortex/ewma v1.2.0 // indirect
github.com/akrylysov/pogreb v0.10.2 // indirect
@@ -155,27 +174,29 @@ require (
github.com/alecthomas/template v0.0.0-20190718012654-fb15b899a751 // indirect
github.com/alecthomas/units v0.0.0-20211218093645-b94a6e3cc137 // indirect
github.com/andybalholm/brotli v1.2.0 // indirect
github.com/andybalholm/cascadia v1.3.3 // indirect
github.com/andybalholm/cascadia v1.3.4 // indirect
github.com/asaskevich/govalidator v0.0.0-20230301143203-a9d515a09cc2 // indirect
github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.7.8 // indirect
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.16.32 // indirect
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.18.12 // indirect
github.com/aws/aws-sdk-go-v2/internal/configsources v1.4.21 // indirect
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.7.21 // indirect
github.com/aws/aws-sdk-go-v2/internal/ini v1.8.3 // indirect
github.com/aws/aws-sdk-go-v2/internal/ini v1.8.4 // indirect
github.com/aws/aws-sdk-go-v2/internal/v4a v1.4.22 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.13.7 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.9.13 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.13.21 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.19.21 // indirect
github.com/aws/aws-sdk-go-v2/service/sso v1.25.5 // indirect
github.com/aws/aws-sdk-go-v2/service/ssooidc v1.30.3 // indirect
github.com/aws/aws-sdk-go-v2/service/sts v1.34.0 // indirect
github.com/aws/aws-sdk-go-v2/service/kms v1.37.7 // indirect
github.com/aws/aws-sdk-go-v2/service/sso v1.30.0 // indirect
github.com/aws/aws-sdk-go-v2/service/ssooidc v1.35.4 // indirect
github.com/aws/aws-sdk-go-v2/service/sts v1.39.0 // indirect
github.com/aws/smithy-go v1.24.2 // indirect
github.com/aymanbagabas/go-osc52/v2 v2.0.1 // indirect
github.com/aymerick/douceur v0.2.0 // indirect
github.com/bahlo/generic-list-go v0.2.0 // indirect
github.com/bits-and-blooms/bitset v1.22.0 // indirect
github.com/bits-and-blooms/bloom/v3 v3.5.0 // indirect
github.com/blang/semver v3.5.1+incompatible // indirect
github.com/bodgit/plumbing v1.3.0 // indirect
github.com/bodgit/sevenzip v1.6.1 // indirect
github.com/bodgit/windows v1.0.1 // indirect
@@ -183,7 +204,7 @@ require (
github.com/bufbuild/protocompile v0.14.1 // indirect
github.com/buger/jsonparser v1.1.2 // indirect
github.com/bytedance/gopkg v0.1.3 // indirect
github.com/bytedance/sonic/loader v0.5.0 // indirect
github.com/bytedance/sonic/loader v0.5.2 // indirect
github.com/caddyserver/certmagic v0.25.0 // indirect
github.com/caddyserver/zerossl v0.1.3 // indirect
github.com/cenkalti/backoff/v4 v4.3.0 // indirect
@@ -196,7 +217,7 @@ require (
github.com/charmbracelet/x/exp/slice v0.0.0-20250908092851-c2208eb08494 // indirect
github.com/charmbracelet/x/term v0.2.1 // indirect
github.com/cheggaaa/pb/v3 v3.1.7 // indirect
github.com/cloudflare/cfssl v1.6.4 // indirect
github.com/cloudflare/cfssl v1.6.5 // indirect
github.com/cloudflare/circl v1.6.3 // indirect
github.com/cloudwego/base64x v0.1.6 // indirect
github.com/cncf/xds/go v0.0.0-20260202195803-dba9d589def2 // indirect
@@ -204,6 +225,7 @@ require (
github.com/containerd/continuity v0.4.5 // indirect
github.com/containerd/errdefs v1.0.0 // indirect
github.com/containerd/errdefs/pkg v0.3.0 // indirect
github.com/cpuguy83/go-md2man/v2 v2.0.5 // indirect
github.com/cyphar/filepath-securejoin v0.6.1 // indirect
github.com/davidmz/go-pageant v1.0.2 // indirect
github.com/dgryski/go-rendezvous v0.0.0-20200823014737-9f7001d12a5f // indirect
@@ -213,6 +235,7 @@ require (
github.com/dlclark/regexp2/v2 v2.2.1 // indirect
github.com/docker/cli v29.2.0+incompatible // indirect
github.com/docker/go-connections v0.7.0 // indirect
github.com/dop251/goja v0.0.0-20260311135729-065cd970411c // indirect
github.com/dsnet/compress v0.0.2-0.20230904184137-39efe44ab707 // indirect
github.com/dustin/go-humanize v1.0.1 // indirect
github.com/ebitengine/purego v0.10.0 // indirect
@@ -227,6 +250,7 @@ require (
github.com/gabriel-vasile/mimetype v1.4.8 // indirect
github.com/gaissmai/bart v0.29.0 // indirect
github.com/geoffgarside/ber v1.2.0 // indirect
github.com/getsops/gopgagent v0.0.0-20240527072608-0c14999532fe // indirect
github.com/gin-contrib/sse v0.1.0 // indirect
github.com/gin-gonic/gin v1.9.1 // indirect
github.com/go-asn1-ber/asn1-ber v1.5.8-0.20250403174932-29230038a667 // indirect
@@ -250,15 +274,28 @@ require (
github.com/golang/groupcache v0.0.0-20241129210726-2c02b8208cf8 // indirect
github.com/golang/protobuf v1.5.4 // indirect
github.com/golang/snappy v1.0.0 // indirect
github.com/google/certificate-transparency-go v1.3.2 // indirect
github.com/google/certificate-transparency-go v1.3.3 // indirect
github.com/google/go-cmp v0.7.0 // indirect
github.com/google/pprof v0.0.0-20240727154555-813a5fbdbec8 // indirect
github.com/google/s2a-go v0.1.9 // indirect
github.com/googleapis/enterprise-certificate-proxy v0.3.11 // indirect
github.com/googleapis/gax-go/v2 v2.17.0 // indirect
github.com/gorilla/css v1.0.1 // indirect
github.com/gosimple/slug v1.15.0 // indirect
github.com/gosimple/unidecode v1.0.1 // indirect
github.com/goware/prefixer v0.0.0-20160118172347-395022866408 // indirect
github.com/hashicorp/errwrap v1.1.0 // indirect
github.com/hashicorp/go-cleanhttp v0.5.2 // indirect
github.com/hashicorp/go-multierror v1.1.1 // indirect
github.com/hashicorp/go-retryablehttp v0.7.8 // indirect
github.com/hashicorp/go-rootcerts v1.0.2 // indirect
github.com/hashicorp/go-secure-stdlib/parseutil v0.1.8 // indirect
github.com/hashicorp/go-secure-stdlib/strutil v0.1.2 // indirect
github.com/hashicorp/go-sockaddr v1.0.7 // indirect
github.com/hashicorp/go-uuid v1.0.3 // indirect
github.com/hashicorp/go-version v1.8.0 // indirect
github.com/hashicorp/go-version v1.9.0 // indirect
github.com/hashicorp/hcl v1.0.0 // indirect
github.com/hashicorp/vault/api v1.15.0 // indirect
github.com/hbakhtiyor/strsim v0.0.0-20190107154042-4d2bbb273edf // indirect
github.com/hdm/jarm-go v0.0.8 // indirect
github.com/iangcarroll/cookiemonster v1.6.0 // indirect
@@ -284,7 +321,7 @@ require (
github.com/libdns/libdns v1.1.1 // indirect
github.com/lor00x/goldap v0.0.0-20240304151906-8d785c64d1c8 // indirect
github.com/lucasb-eyer/go-colorful v1.3.0 // indirect
github.com/lufia/plan9stats v0.0.0-20250827001030-24949be3fa54 // indirect
github.com/lufia/plan9stats v0.0.0-20251013123823-9fd1530e3ec3 // indirect
github.com/mackerelio/go-osstat v0.2.6 // indirect
github.com/mailru/easyjson v0.7.7 // indirect
github.com/mattn/go-colorable v0.1.14 // indirect
@@ -296,6 +333,8 @@ require (
github.com/minio/minlz v1.0.1 // indirect
github.com/minio/selfupdate v0.6.1-0.20230907112617-f11e74f84ca7 // indirect
github.com/mitchellh/go-homedir v1.1.0 // indirect
github.com/mitchellh/go-wordwrap v1.0.1 // indirect
github.com/mitchellh/mapstructure v1.5.0 // indirect
github.com/moby/docker-image-spec v1.3.1 // indirect
github.com/moby/moby/api v1.54.2 // indirect
github.com/moby/moby/client v0.4.1 // indirect
@@ -325,14 +364,17 @@ require (
github.com/power-devops/perfstat v0.0.0-20240221224432-82ca36839d55 // indirect
github.com/projectdiscovery/asnmap v1.1.1 // indirect
github.com/projectdiscovery/blackrock v0.0.2 // indirect
github.com/projectdiscovery/cdncheck v1.2.50 // indirect
github.com/projectdiscovery/cdncheck v1.3.0 // indirect
github.com/projectdiscovery/freeport v0.0.7 // indirect
github.com/projectdiscovery/jarm-go v0.0.0-20260910160638-430e2bae86f6 // indirect
github.com/projectdiscovery/ldapserver v1.0.2-0.20240219154113-dcc758ebc0cb // indirect
github.com/projectdiscovery/machineid v0.0.0-20250715113114-c77eb3567582 // indirect
github.com/refraction-networking/utls v1.8.2 // indirect
github.com/russross/blackfriday/v2 v2.1.0 // indirect
github.com/ryanuber/go-glob v1.0.0 // indirect
github.com/santhosh-tekuri/jsonschema/v6 v6.0.2 // indirect
github.com/sashabaranov/go-openai v1.37.0 // indirect
github.com/sergi/go-diff v1.3.2-0.20230802210424-5b0b94c5c0d3 // indirect
github.com/sergi/go-diff v1.4.0 // indirect
github.com/shirou/gopsutil v3.21.11+incompatible // indirect
github.com/shirou/gopsutil/v4 v4.26.3 // indirect
github.com/sirupsen/logrus v1.9.4 // indirect
@@ -340,7 +382,7 @@ require (
github.com/sorairolake/lzip-go v0.3.8 // indirect
github.com/spf13/afero v1.15.0 // indirect
github.com/spf13/pflag v1.0.10 // indirect
github.com/spiffe/go-spiffe/v2 v2.6.0 // indirect
github.com/spiffe/go-spiffe/v2 v2.7.0 // indirect
github.com/tidwall/btree v1.8.1 // indirect
github.com/tidwall/buntdb v1.3.2 // indirect
github.com/tidwall/gjson v1.18.0 // indirect
@@ -356,6 +398,7 @@ require (
github.com/twitchyliquid64/golang-asm v0.15.1 // indirect
github.com/ugorji/go/codec v1.2.11 // indirect
github.com/ulikunitz/xz v0.5.15 // indirect
github.com/urfave/cli v1.22.16 // indirect
github.com/valyala/bytebufferpool v1.0.0 // indirect
github.com/vmihailenco/bufpool v0.1.11 // indirect
github.com/vmihailenco/msgpack/v5 v5.3.4 // indirect
@@ -379,15 +422,23 @@ require (
github.com/zcalusic/sysinfo v1.1.3 // indirect
github.com/zeebo/blake3 v0.2.4 // indirect
go.opentelemetry.io/auto/sdk v1.2.1 // indirect
go.opentelemetry.io/contrib/detectors/gcp v1.44.0 // indirect
go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.61.0 // indirect
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.68.0 // indirect
go.opentelemetry.io/otel v1.44.0 // indirect
go.opentelemetry.io/otel/metric v1.44.0 // indirect
go.opentelemetry.io/otel/sdk v1.44.0 // indirect
go.opentelemetry.io/otel/sdk/metric v1.44.0 // indirect
go.opentelemetry.io/otel/trace v1.44.0 // indirect
go.uber.org/zap/exp v0.3.0 // indirect
go.yaml.in/yaml/v3 v3.0.5 // indirect
go4.org v0.0.0-20230225012048-214862532bf5 // indirect
golang.org/x/arch v0.3.0 // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20260414002931-afd174a4e478 // indirect
google.golang.org/api v0.264.0 // indirect
google.golang.org/genproto v0.0.0-20260128011058-8636f8732409 // indirect
google.golang.org/genproto/googleapis/api v0.0.0-20260526163538-3dc84a4a5aaa // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20260526163538-3dc84a4a5aaa // indirect
gopkg.in/ini.v1 v1.67.0 // indirect
mellium.im/sasl v0.3.2 // indirect
software.sslmate.com/src/go-pkcs12 v0.7.2 // indirect
)
@@ -398,7 +449,7 @@ require (
github.com/gobwas/httphead v0.1.0 // indirect
github.com/gobwas/pool v0.2.1 // indirect
github.com/golang-jwt/jwt/v4 v4.5.2 // indirect
github.com/google/go-querystring v1.1.0 // indirect
github.com/google/go-querystring v1.2.0 // indirect
github.com/rivo/uniseg v0.4.7 // indirect
github.com/saintfish/chardet v0.0.0-20230101081208-5e3ef4b5456d // indirect
github.com/trivago/tgo v1.0.7
@@ -411,12 +462,11 @@ require (
go.etcd.io/bbolt v1.4.3 // indirect
go.uber.org/zap v1.27.0 // indirect
goftp.io/server/v2 v2.0.1 // indirect
golang.org/x/crypto v0.54.0 // indirect
golang.org/x/crypto v0.55.0 // indirect
golang.org/x/exp v0.0.0-20260527015227-08cc5374adb3
golang.org/x/mod v0.37.0 // indirect
golang.org/x/sys v0.47.0 // indirect
golang.org/x/time v0.14.0 // indirect
golang.org/x/tools v0.47.0
golang.org/x/mod v0.39.0 // indirect
golang.org/x/sys v0.47.0
golang.org/x/tools v0.49.0
google.golang.org/protobuf v1.36.11
gopkg.in/alecthomas/kingpin.v2 v2.2.6 // indirect
gopkg.in/corvus-ch/zbase32.v1 v1.0.0 // indirect
+203 -89
View File
@@ -1,8 +1,12 @@
aead.dev/minisign v0.2.0/go.mod h1:zdq6LdSd9TbuSxchxwhpA9zEb9YXcVGoE8JakuiGaIQ=
aead.dev/minisign v0.3.0 h1:8Xafzy5PEVZqYDNP60yJHARlW1eOQtsKNp/Ph2c0vRA=
aead.dev/minisign v0.3.0/go.mod h1:NLvG3Uoq3skkRMDuc3YHpWUTMTrSExqm+Ij73W13F6Y=
c2sp.org/CCTV/age v0.0.0-20240306222714-3ec4d716e805 h1:u2qwJeEvnypw+OCPUHmoZE3IqwfuN5kgDfo5MLzpNM0=
c2sp.org/CCTV/age v0.0.0-20240306222714-3ec4d716e805/go.mod h1:FomMrUJ2Lxt5jCLmZkG3FHa72zUprnhd3v/Z18Snm4w=
carvel.dev/ytt v0.52.0 h1:tkJPL8Gun5snVfypNXbmMKwnbwMyspcTi3Ypyso3nRY=
carvel.dev/ytt v0.52.0/go.mod h1:QgmuU7E15EXW1r2wxTt7zExVz14IHwEG4WNMmaFBkJo=
cel.dev/expr v0.25.2 h1:K6j46C81hXtZQfuX60cVWQFBJahKSE2gfRbNuvr5bFs=
cel.dev/expr v0.25.2/go.mod h1:hrXvqGP6G6gyx8UAHSHJ5RGk//1Oj5nXQ2NI02Nrsg4=
cloud.google.com/go v0.26.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw=
cloud.google.com/go v0.34.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw=
cloud.google.com/go v0.38.0/go.mod h1:990N+gfupTy94rShfmMCWGDn0LpTmnzTp2qbd1dvSRU=
@@ -18,6 +22,12 @@ cloud.google.com/go v0.56.0/go.mod h1:jr7tqZxxKOVYizybht9+26Z/gUq7tiRzu+ACVAMbKV
cloud.google.com/go v0.57.0/go.mod h1:oXiQ6Rzq3RAkkY7N6t3TcE6jE+CIBBbA36lwQ1JyzZs=
cloud.google.com/go v0.62.0/go.mod h1:jmCYTdRCQuc1PHIIJ/maLInMho30T/Y0M4hTdTShOYc=
cloud.google.com/go v0.65.0/go.mod h1:O5N8zS7uWy9vkA9vayVHs65eM1ubvY4h553ofrNHObY=
cloud.google.com/go v0.123.0 h1:2NAUJwPR47q+E35uaJeYoNhuNEM9kM8SjgRgdeOJUSE=
cloud.google.com/go v0.123.0/go.mod h1:xBoMV08QcqUGuPW65Qfm1o9Y4zKZBpGS+7bImXLTAZU=
cloud.google.com/go/auth v0.18.2 h1:+Nbt5Ev0xEqxlNjd6c+yYUeosQ5TtEUaNcN/3FozlaM=
cloud.google.com/go/auth v0.18.2/go.mod h1:xD+oY7gcahcu7G2SG2DsBerfFxgPAJz17zz2joOFF3M=
cloud.google.com/go/auth/oauth2adapt v0.2.8 h1:keo8NaayQZ6wimpNSmW5OPc283g65QNIiLpZnkHRbnc=
cloud.google.com/go/auth/oauth2adapt v0.2.8/go.mod h1:XQ9y31RkqZCcwJWNSx2Xvric3RrU88hAYYbjDWYDL+c=
cloud.google.com/go/bigquery v1.0.1/go.mod h1:i/xbL2UlR5RvWAURpBYZTtm/cXjCha9lbfbpx4poX+o=
cloud.google.com/go/bigquery v1.3.0/go.mod h1:PjpwJnslEMmckchkHFfq+HTD2DmtT67aNFKH1/VBDHE=
cloud.google.com/go/bigquery v1.4.0/go.mod h1:S8dzgnTigyfTmLBfrtrhyYhwRxG72rYxvftPBK2Dvzc=
@@ -25,8 +35,20 @@ cloud.google.com/go/bigquery v1.5.0/go.mod h1:snEHRnqQbz117VIFhE8bmtwIDY80NLUZUM
cloud.google.com/go/bigquery v1.7.0/go.mod h1://okPTzCYNXSlb24MZs83e2Do+h+VXtc4gLoIoXIAPc=
cloud.google.com/go/bigquery v1.8.0/go.mod h1:J5hqkt3O0uAFnINi6JXValWIb1v0goeZM77hZzJN/fQ=
cloud.google.com/go/compute/metadata v0.2.0/go.mod h1:zFmK7XCadkQkj6TtorcaGlCW1hT1fIilQDwofLpJ20k=
cloud.google.com/go/compute/metadata v0.9.0 h1:pDUj4QMoPejqq20dK0Pg2N4yG9zIkYGdBtwLoEkH9Zs=
cloud.google.com/go/compute/metadata v0.9.0/go.mod h1:E0bWwX5wTnLPedCKqk3pJmVgCBSM6qQI1yTBdEb3C10=
cloud.google.com/go/datastore v1.0.0/go.mod h1:LXYbyblFSglQ5pkeyhO+Qmw7ukd3C+pD7TKLgZqpHYE=
cloud.google.com/go/datastore v1.1.0/go.mod h1:umbIZjpQpHh4hmRpGhH4tLFup+FVzqBi1b3c64qFpCk=
cloud.google.com/go/iam v1.5.3 h1:+vMINPiDF2ognBJ97ABAYYwRgsaqxPbQDlMnbHMjolc=
cloud.google.com/go/iam v1.5.3/go.mod h1:MR3v9oLkZCTlaqljW6Eb2d3HGDGK5/bDv93jhfISFvU=
cloud.google.com/go/kms v1.25.0 h1:gVqvGGUmz0nYCmtoxWmdc1wli2L1apgP8U4fghPGSbQ=
cloud.google.com/go/kms v1.25.0/go.mod h1:XIdHkzfj0bUO3E+LvwPg+oc7s58/Ns8Nd8Sdtljihbk=
cloud.google.com/go/logging v1.13.1 h1:O7LvmO0kGLaHY/gq8cV7T0dyp6zJhYAOtZPX4TF3QtY=
cloud.google.com/go/logging v1.13.1/go.mod h1:XAQkfkMBxQRjQek96WLPNze7vsOmay9H5PqfsNYDqvw=
cloud.google.com/go/longrunning v0.8.0 h1:LiKK77J3bx5gDLi4SMViHixjD2ohlkwBi+mKA7EhfW8=
cloud.google.com/go/longrunning v0.8.0/go.mod h1:UmErU2Onzi+fKDg2gR7dusz11Pe26aknR4kHmJJqIfk=
cloud.google.com/go/monitoring v1.24.3 h1:dde+gMNc0UhPZD1Azu6at2e79bfdztVDS5lvhOdsgaE=
cloud.google.com/go/monitoring v1.24.3/go.mod h1:nYP6W0tm3N9H/bOw8am7t62YTzZY+zUeQ+Bi6+2eonI=
cloud.google.com/go/pubsub v1.0.1/go.mod h1:R0Gpsv3s54REJCy4fxDixWD93lHJMoZTyQ2kNxGRt3I=
cloud.google.com/go/pubsub v1.1.0/go.mod h1:EwwdRX2sKPjnvnqCa270oGRyludottCI76h+R3AArQw=
cloud.google.com/go/pubsub v1.2.0/go.mod h1:jhfEVHT8odbXTkndysNHCcx0awwzvfOlguIAii9o8iA=
@@ -36,11 +58,17 @@ cloud.google.com/go/storage v1.5.0/go.mod h1:tpKbwo567HUNpVclU5sGELwQWBDZ8gh0Zeo
cloud.google.com/go/storage v1.6.0/go.mod h1:N7U0C8pVQ/+NIKOBQyamJIeKQKkZ+mxpohlUTyfDhBk=
cloud.google.com/go/storage v1.8.0/go.mod h1:Wv1Oy7z6Yz3DshWRJFhqM/UCfaWIRTdp0RXyy7KQOVs=
cloud.google.com/go/storage v1.10.0/go.mod h1:FLPqc6j+Ki4BU591ie1oL6qBQGu2Bl/tZ9ullr3+Kg0=
cloud.google.com/go/storage v1.56.0 h1:iixmq2Fse2tqxMbWhLWC9HfBj1qdxqAmiK8/eqtsLxI=
cloud.google.com/go/storage v1.56.0/go.mod h1:Tpuj6t4NweCLzlNbw9Z9iwxEkrSem20AetIeH/shgVU=
cloud.google.com/go/trace v1.11.7 h1:kDNDX8JkaAG3R2nq1lIdkb7FCSi1rCmsEtKVsty7p+U=
cloud.google.com/go/trace v1.11.7/go.mod h1:TNn9d5V3fQVf6s4SCveVMIBS2LJUqo73GACmq/Tky0s=
code.gitea.io/sdk/gitea v0.17.0 h1:8JPBss4+Jf7AE1YcfyiGrngTXE8dFSG3si/bypsTH34=
code.gitea.io/sdk/gitea v0.17.0/go.mod h1:ndkDk99BnfiUCCYEUhpNzi0lpmApXlwRFqClBlOlEBg=
dario.cat/mergo v1.0.2 h1:85+piFYR1tMbRrLcDwR18y4UKJ3aH1Tbzi24VRW1TK8=
dario.cat/mergo v1.0.2/go.mod h1:E/hbnu0NxMFBjpMIE34DRGLWqDy0g5FuKDhCb31ngxA=
dmitri.shuralyov.com/gpu/mtl v0.0.0-20190408044501-666a987793e9/go.mod h1:H6x//7gZCb22OMCxBHrMx7a5I7Hp++hsVxbQ4BYO7hU=
filippo.io/age v1.2.1 h1:X0TZjehAZylOIj4DubWYU1vWQxv9bJpo+Uu2/LGhi1o=
filippo.io/age v1.2.1/go.mod h1:JL9ew2lTN+Pyft4RiNGguFfOpewKwSHm5ayKD/A4004=
filippo.io/edwards25519 v1.2.0 h1:crnVqOiS4jqYleHd9vaKZ+HKtHfllngJIiOpNpoJsjo=
filippo.io/edwards25519 v1.2.0/go.mod h1:xzAOLCNug/yB62zG1bQ8uziwrIqIuxhctzJT18Q77mc=
git.mills.io/prologic/smtpd v0.0.0-20210710122116-a525b76c287a h1:3i+FJ7IpSZHL+VAjtpQeZCRhrpP0odl5XfoLBY4fxJ8=
@@ -77,6 +105,14 @@ github.com/DataDog/gostackparse v0.7.0 h1:i7dLkXHvYzHV308hnkvVGDL3BR4FWl7IsXNPz/
github.com/DataDog/gostackparse v0.7.0/go.mod h1:lTfqcJKqS9KnXQGnyQMCugq3u1FP6UZMfWR0aitKFMM=
github.com/FalconOpsLLC/goexec v0.3.0 h1:ryLMkrGT6asnkqdc5rFMNOSTYdMH/iCfyEuwu0D6ZhA=
github.com/FalconOpsLLC/goexec v0.3.0/go.mod h1:kiyxVbmFCGbbwXRyZmOSKlOy7PiK+JH2gq07Ztag/k8=
github.com/GoogleCloudPlatform/opentelemetry-operations-go/detectors/gcp v1.33.0 h1:l7+6kwRMJNwdCvYdDl7Eax+wzEYHSnNY7zrrfbhDdTA=
github.com/GoogleCloudPlatform/opentelemetry-operations-go/detectors/gcp v1.33.0/go.mod h1:pJTkW8hEUIIi3Pf65lPZOnn4Y81yCllX6IWk2jNXdkM=
github.com/GoogleCloudPlatform/opentelemetry-operations-go/exporter/metric v0.53.0 h1:owcC2UnmsZycprQ5RfRgjydWhuoxg71LUfyiQdijZuM=
github.com/GoogleCloudPlatform/opentelemetry-operations-go/exporter/metric v0.53.0/go.mod h1:ZPpqegjbE99EPKsu3iUWV22A04wzGPcAY/ziSIQEEgs=
github.com/GoogleCloudPlatform/opentelemetry-operations-go/internal/cloudmock v0.53.0 h1:4LP6hvB4I5ouTbGgWtixJhgED6xdf67twf9PoY96Tbg=
github.com/GoogleCloudPlatform/opentelemetry-operations-go/internal/cloudmock v0.53.0/go.mod h1:jUZ5LYlw40WMd07qxcQJD5M40aUxrfwqQX1g7zxYnrQ=
github.com/GoogleCloudPlatform/opentelemetry-operations-go/internal/resourcemapping v0.53.0 h1:Ron4zCA/yk6U7WOBXhTJcDpsUBG9npumK6xw2auFltQ=
github.com/GoogleCloudPlatform/opentelemetry-operations-go/internal/resourcemapping v0.53.0/go.mod h1:cSgYe11MCNYunTnRXrKiR/tHc0eoKjICUuWpNZoVCOo=
github.com/Masterminds/semver/v3 v3.5.0 h1:kQceYJfbupGfZOKZQg0kou0DgAKhzDg2NZPAwZ/2OOE=
github.com/Masterminds/semver/v3 v3.5.0/go.mod h1:4V+yj/TJE1HU9XfppCwVMZq3I84lprf4nC11bSS5beM=
github.com/Microsoft/go-winio v0.5.2/go.mod h1:WpS1mjBmmwHBEWmogvA2mj8546UReBk4v8QkMxJ6pZY=
@@ -97,8 +133,8 @@ github.com/OneOfOne/xxhash v1.2.2/go.mod h1:HSdplMjZKSmBqAxg5vPj2TmRDmfkzw+cTzAE
github.com/ProtonMail/go-crypto v0.0.0-20230217124315-7d5c6f04bbb8/go.mod h1:I0gYDMZ6Z5GRU7l58bNFSkPTFN6Yl12dsUlAZ8xy98g=
github.com/ProtonMail/go-crypto v1.1.6 h1:ZcV+Ropw6Qn0AX9brlQLAUXfqLBc7Bl+f/DmNxpLfdw=
github.com/ProtonMail/go-crypto v1.1.6/go.mod h1:rA3QumHc/FZ8pAHreoekgiAbzpNsfQAosU5td4SnOrE=
github.com/PuerkitoBio/goquery v1.12.0 h1:pAcL4g3WRXekcB9AU/y1mbKez2dbY2AajVhtkO8RIBo=
github.com/PuerkitoBio/goquery v1.12.0/go.mod h1:802ej+gV2y7bbIhOIoPY5sT183ZW0YFofScC4q/hIpQ=
github.com/PuerkitoBio/goquery v1.13.0 h1:mqHbjD7Jmnul4DTR24LKTjo1uUmHUh072kteGV+xpFM=
github.com/PuerkitoBio/goquery v1.13.0/go.mod h1:Hip5mdBL8K2wEGKJdr27sRaNwIdDajmCwB/ExUPwW+g=
github.com/RedTeamPentesting/adauth v0.5.4-0.20260511073005-3d18e8a5a687 h1:HHkHNwakgAFX3qlMm88c090vIYNFWiO5+4WyAr0xJuM=
github.com/RedTeamPentesting/adauth v0.5.4-0.20260511073005-3d18e8a5a687/go.mod h1:l6FRaC2TliS/JMNWskO3J1tmrcKJyOaMFhWC6hHeWno=
github.com/RumbleDiscovery/rumble-tools v0.0.0-20201105153123-f2adbb3244d2/go.mod h1:jD2+mU+E2SZUuAOHZvZj4xP4frlOo+N/YrXDvASFhkE=
@@ -137,8 +173,8 @@ github.com/alitto/pond v1.9.2 h1:9Qb75z/scEZVCoSU+osVmQ0I0JOeLfdTDafrbcJ8CLs=
github.com/alitto/pond v1.9.2/go.mod h1:xQn3P/sHTYcU/1BR3i86IGIrilcrGC2LiS+E2+CJWsI=
github.com/andybalholm/brotli v1.2.0 h1:ukwgCxwYrmACq68yiUqwIWnGY0cTPox/M94sVwToPjQ=
github.com/andybalholm/brotli v1.2.0/go.mod h1:rzTDkvFWvIrjDXZHkuS16NPggd91W3kUSvPlQ1pLaKY=
github.com/andybalholm/cascadia v1.3.3 h1:AG2YHrzJIm4BZ19iwJ/DAua6Btl3IwJX+VI4kktS1LM=
github.com/andybalholm/cascadia v1.3.3/go.mod h1:xNd9bqTn98Ln4DwST8/nG+H0yuB8Hmgu1YHNnWw0GeA=
github.com/andybalholm/cascadia v1.3.4 h1:vM2lgh0Vru9Vwyfm4cQqWP2HHMW0u0+2PAW7Q38Qufg=
github.com/andybalholm/cascadia v1.3.4/go.mod h1:BLRmbRjpEtNKieZOCCvYj4RqN+KRA41GBe/5O+G93kM=
github.com/andygrunwald/go-jira v1.16.1 h1:WoQEar5XoDRAibOgKzTFELlPNlKAtnfWr296R9zdFLA=
github.com/andygrunwald/go-jira v1.16.1/go.mod h1:UQH4IBVxIYWbgagc0LF/k9FRs9xjIiQ8hIcC6HfLwFU=
github.com/anmitsu/go-shlex v0.0.0-20200514113438-38f4b401e2be h1:9AeTilPcZAjCFIImctFaOjnTIavg87rW78vTPkQqLI8=
@@ -159,20 +195,20 @@ github.com/aws/aws-sdk-go-v2 v1.41.5 h1:dj5kopbwUsVUVFgO4Fi5BIT3t4WyqIDjGKCangnV
github.com/aws/aws-sdk-go-v2 v1.41.5/go.mod h1:mwsPRE8ceUUpiTgF7QmQIJ7lgsKUPQOUl3o72QBrE1o=
github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.7.8 h1:eBMB84YGghSocM7PsjmmPffTa+1FBUeNvGvFou6V/4o=
github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.7.8/go.mod h1:lyw7GFp3qENLh7kwzf7iMzAxDn+NzjXEAGjKS2UOKqI=
github.com/aws/aws-sdk-go-v2/config v1.29.17 h1:jSuiQ5jEe4SAMH6lLRMY9OVC+TqJLP5655pBGjmnjr0=
github.com/aws/aws-sdk-go-v2/config v1.29.17/go.mod h1:9P4wwACpbeXs9Pm9w1QTh6BwWwJjwYvJ1iCt5QbCXh8=
github.com/aws/aws-sdk-go-v2/credentials v1.17.70 h1:ONnH5CM16RTXRkS8Z1qg7/s2eDOhHhaXVd72mmyv4/0=
github.com/aws/aws-sdk-go-v2/credentials v1.17.70/go.mod h1:M+lWhhmomVGgtuPOhO85u4pEa3SmssPTdcYpP/5J/xc=
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.16.32 h1:KAXP9JSHO1vKGCr5f4O6WmlVKLFFXgWYAGoJosorxzU=
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.16.32/go.mod h1:h4Sg6FQdexC1yYG9RDnOvLbW1a/P986++/Y/a+GyEM8=
github.com/aws/aws-sdk-go-v2/feature/s3/manager v1.17.82 h1:EO13QJTCD1Ig2IrQnoHTRrn981H9mB7afXsZ89WptI4=
github.com/aws/aws-sdk-go-v2/feature/s3/manager v1.17.82/go.mod h1:AGh1NCg0SH+uyJamiJA5tTQcql4MMRDXGRdMmCxCXzY=
github.com/aws/aws-sdk-go-v2/config v1.31.16 h1:E4Tz+tJiPc7kGnXwIfCyUj6xHJNpENlY11oKpRTgsjc=
github.com/aws/aws-sdk-go-v2/config v1.31.16/go.mod h1:2S9hBElpCyGMifv14WxQ7EfPumgoeCPZUpuPX8VtW34=
github.com/aws/aws-sdk-go-v2/credentials v1.18.20 h1:KFndAnHd9NUuzikHjQ8D5CfFVO+bgELkmcGY8yAw98Q=
github.com/aws/aws-sdk-go-v2/credentials v1.18.20/go.mod h1:9mCi28a+fmBHSQ0UM79omkz6JtN+PEsvLrnG36uoUv0=
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.18.12 h1:VO3FIM2TDbm0kqp6sFNR0PbioXJb/HzCDW6NtIZpIWE=
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.18.12/go.mod h1:6C39gB8kg82tx3r72muZSrNhHia9rjGkX7ORaS2GKNE=
github.com/aws/aws-sdk-go-v2/feature/s3/manager v1.20.2 h1:9/HxDeIgA7DcKK6e6ZaP5PQiXugYbNERx3Z5u30mN+k=
github.com/aws/aws-sdk-go-v2/feature/s3/manager v1.20.2/go.mod h1:3N1RoxKNcVHmbOKVMMw8pvMs5TUhGYPQP/aq1zmAWqo=
github.com/aws/aws-sdk-go-v2/internal/configsources v1.4.21 h1:Rgg6wvjjtX8bNHcvi9OnXWwcE0a2vGpbwmtICOsvcf4=
github.com/aws/aws-sdk-go-v2/internal/configsources v1.4.21/go.mod h1:A/kJFst/nm//cyqonihbdpQZwiUhhzpqTsdbhDdRF9c=
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.7.21 h1:PEgGVtPoB6NTpPrBgqSE5hE/o47Ij9qk/SEZFbUOe9A=
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.7.21/go.mod h1:p+hz+PRAYlY3zcpJhPwXlLC4C+kqn70WIHwnzAfs6ps=
github.com/aws/aws-sdk-go-v2/internal/ini v1.8.3 h1:bIqFDwgGXXN1Kpp99pDOdKMTTb5d2KyU5X/BZxjOkRo=
github.com/aws/aws-sdk-go-v2/internal/ini v1.8.3/go.mod h1:H5O/EsxDWyU+LP/V8i5sm8cxoZgc2fdNR9bxlOFrQTo=
github.com/aws/aws-sdk-go-v2/internal/ini v1.8.4 h1:WKuaxf++XKWlHWu9ECbMlha8WOEGm0OUEZqm4K/Gcfk=
github.com/aws/aws-sdk-go-v2/internal/ini v1.8.4/go.mod h1:ZWy7j6v1vWGmPReu0iSGvRiise4YI5SkR3OHKTZ6Wuc=
github.com/aws/aws-sdk-go-v2/internal/v4a v1.4.22 h1:rWyie/PxDRIdhNf4DzRk0lvjVOqFJuNnO8WwaIRVxzQ=
github.com/aws/aws-sdk-go-v2/internal/v4a v1.4.22/go.mod h1:zd/JsJ4P7oGfUhXn1VyLqaRZwPmZwg44Jf2dS84Dm3Y=
github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.13.7 h1:5EniKhLZe4xzL7a+fU3C2tfUN4nWIqlLesfrjkuPFTY=
@@ -183,14 +219,16 @@ github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.13.21 h1:c31//R3x
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.13.21/go.mod h1:r6+pf23ouCB718FUxaqzZdbpYFyDtehyZcmP5KL9FkA=
github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.19.21 h1:ZlvrNcHSFFWURB8avufQq9gFsheUgjVD9536obIknfM=
github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.19.21/go.mod h1:cv3TNhVrssKR0O/xxLJVRfd2oazSnZnkUeTf6ctUwfQ=
github.com/aws/aws-sdk-go-v2/service/kms v1.37.7 h1:dZmNIRtPUvtvUIIDVNpvtnJQ8N8Iqm7SQAxf18htZYw=
github.com/aws/aws-sdk-go-v2/service/kms v1.37.7/go.mod h1:vj8PlfJH9mnGeIzd6uMLPi5VgiqzGG7AZoe1kf1uTXM=
github.com/aws/aws-sdk-go-v2/service/s3 v1.99.0 h1:hlSuz394kV0vhv9drL5lhuEFbEOEP1VyQpy15qWh1Pk=
github.com/aws/aws-sdk-go-v2/service/s3 v1.99.0/go.mod h1:uoA43SdFwacedBfSgfFSjjCvYe8aYBS7EnU5GZ/YKMM=
github.com/aws/aws-sdk-go-v2/service/sso v1.25.5 h1:AIRJ3lfb2w/1/8wOOSqYb9fUKGwQbtysJ2H1MofRUPg=
github.com/aws/aws-sdk-go-v2/service/sso v1.25.5/go.mod h1:b7SiVprpU+iGazDUqvRSLf5XmCdn+JtT1on7uNL6Ipc=
github.com/aws/aws-sdk-go-v2/service/ssooidc v1.30.3 h1:BpOxT3yhLwSJ77qIY3DoHAQjZsc4HEGfMCE4NGy3uFg=
github.com/aws/aws-sdk-go-v2/service/ssooidc v1.30.3/go.mod h1:vq/GQR1gOFLquZMSrxUK/cpvKCNVYibNyJ1m7JrU88E=
github.com/aws/aws-sdk-go-v2/service/sts v1.34.0 h1:NFOJ/NXEGV4Rq//71Hs1jC/NvPs1ezajK+yQmkwnPV0=
github.com/aws/aws-sdk-go-v2/service/sts v1.34.0/go.mod h1:7ph2tGpfQvwzgistp2+zga9f+bCjlQJPkPUmMgDSD7w=
github.com/aws/aws-sdk-go-v2/service/sso v1.30.0 h1:xHXvxst78wBpJFgDW07xllOx0IAzbryrSdM4nMVQ4Dw=
github.com/aws/aws-sdk-go-v2/service/sso v1.30.0/go.mod h1:/e8m+AO6HNPPqMyfKRtzZ9+mBF5/x1Wk8QiDva4m07I=
github.com/aws/aws-sdk-go-v2/service/ssooidc v1.35.4 h1:tBw2Qhf0kj4ZwtsVpDiVRU3zKLvjvjgIjHMKirxXg8M=
github.com/aws/aws-sdk-go-v2/service/ssooidc v1.35.4/go.mod h1:Deq4B7sRM6Awq/xyOBlxBdgW8/Z926KYNNaGMW2lrkA=
github.com/aws/aws-sdk-go-v2/service/sts v1.39.0 h1:C+BRMnasSYFcgDw8o9H5hzehKzXyAb9GY5v/8bP9DUY=
github.com/aws/aws-sdk-go-v2/service/sts v1.39.0/go.mod h1:4EjU+4mIx6+JqKQkruye+CaigV7alL3thVPfDd9VlMs=
github.com/aws/smithy-go v1.24.2 h1:FzA3bu/nt/vDvmnkg+R8Xl46gmzEDam6mZ1hzmwXFng=
github.com/aws/smithy-go v1.24.2/go.mod h1:YE2RhdIuDbA5E5bTdciG9KrW3+TiEONeUWCqxX9i1Fc=
github.com/aymanbagabas/go-osc52/v2 v2.0.1 h1:HwpRHbFMcZLEVr42D4p7XBqjyuxQH5SMiErDT4WkJ2k=
@@ -209,6 +247,8 @@ github.com/bits-and-blooms/bitset v1.22.0 h1:Tquv9S8+SGaS3EhyA+up3FXzmkhxPGjQQCk
github.com/bits-and-blooms/bitset v1.22.0/go.mod h1:7hO7Gc7Pp1vODcmWvKMRA9BNmbv6a/7QIWpPxHddWR8=
github.com/bits-and-blooms/bloom/v3 v3.5.0 h1:AKDvi1V3xJCmSR6QhcBfHbCN4Vf8FfxeWkMNQfmAGhY=
github.com/bits-and-blooms/bloom/v3 v3.5.0/go.mod h1:Y8vrn7nk1tPIlmLtW2ZPV+W7StdVMor6bC1xgpjMZFs=
github.com/blang/semver v3.5.1+incompatible h1:cQNTCjp13qL8KC3Nbxr/y2Bqb63oX6wdnnjpJbkM4JQ=
github.com/blang/semver v3.5.1+incompatible/go.mod h1:kRBLl5iJ+tD4TcOOxsy/0fnwebNt5EWlYSAyrTnjyyk=
github.com/bodgit/plumbing v1.3.0 h1:pf9Itz1JOQgn7vEOE7v7nlEfBykYqvUYioC61TwWCFU=
github.com/bodgit/plumbing v1.3.0/go.mod h1:JOTb4XiRu5xfnmdnDJo6GmSbSbtSyufrsyZFByMtKEs=
github.com/bodgit/sevenzip v1.6.1 h1:kikg2pUMYC9ljU7W9SaqHXhym5HyKm8/M/jd31fYan4=
@@ -228,10 +268,10 @@ github.com/buger/jsonparser v1.1.2/go.mod h1:6RYKKt7H4d4+iWqouImQ9R2FZql3VbhNgx2
github.com/bwesterb/go-ristretto v1.2.0/go.mod h1:fUIoIZaG73pV5biE2Blr2xEzDoMj7NFEuV9ekS419A0=
github.com/bytedance/gopkg v0.1.3 h1:TPBSwH8RsouGCBcMBktLt1AymVo2TVsBVCY4b6TnZ/M=
github.com/bytedance/gopkg v0.1.3/go.mod h1:576VvJ+eJgyCzdjS+c4+77QF3p7ubbtiKARP3TxducM=
github.com/bytedance/sonic v1.15.0 h1:/PXeWFaR5ElNcVE84U0dOHjiMHQOwNIx3K4ymzh/uSE=
github.com/bytedance/sonic v1.15.0/go.mod h1:tFkWrPz0/CUCLEF4ri4UkHekCIcdnkqXw9VduqpJh0k=
github.com/bytedance/sonic/loader v0.5.0 h1:gXH3KVnatgY7loH5/TkeVyXPfESoqSBSBEiDd5VjlgE=
github.com/bytedance/sonic/loader v0.5.0/go.mod h1:AR4NYCk5DdzZizZ5djGqQ92eEhCCcdf5x77udYiSJRo=
github.com/bytedance/sonic v1.15.3 h1:P3akjLPBtV/i6bHC6LbcLjY3KuoOvfiqF8wFHeP5IhY=
github.com/bytedance/sonic v1.15.3/go.mod h1:8e51yTPdY8M6t+vvGL1c2Y1xL9i+frEeIAQAEl75NUc=
github.com/bytedance/sonic/loader v0.5.2 h1:0QtP1gevc1OZ6/H8Lb9BRZiCXd1Ftjd3OKuj1T1lBIo=
github.com/bytedance/sonic/loader v0.5.2/go.mod h1:AR4NYCk5DdzZizZ5djGqQ92eEhCCcdf5x77udYiSJRo=
github.com/caddyserver/certmagic v0.25.0 h1:VMleO/XA48gEWes5l+Fh6tRWo9bHkhwAEhx63i+F5ic=
github.com/caddyserver/certmagic v0.25.0/go.mod h1:m9yB7Mud24OQbPHOiipAoyKPn9pKHhpSJxXR1jydBxA=
github.com/caddyserver/zerossl v0.1.3 h1:onS+pxp3M8HnHpN5MMbOMyNjmTheJyWRaZYwn+YTAyA=
@@ -278,8 +318,8 @@ github.com/chzyer/test v1.0.0/go.mod h1:2JlltgoNkt4TW/z9V/IzDdFaMTM2JPIi26O1pF38
github.com/clbanning/mxj/v2 v2.7.0 h1:WA/La7UGCanFe5NpHF0Q3DNtnCsVoxbPKuyBNHWRyME=
github.com/clbanning/mxj/v2 v2.7.0/go.mod h1:hNiWqW14h+kc+MdF9C6/YoRfjEJoR3ou6tn/Qo+ve2s=
github.com/client9/misspell v0.3.4/go.mod h1:qj6jICC3Q7zFZvVWo7KLAzC3yx5G7kyvSDkc90ppPyw=
github.com/cloudflare/cfssl v1.6.4 h1:NMOvfrEjFfC63K3SGXgAnFdsgkmiq4kATme5BfcqrO8=
github.com/cloudflare/cfssl v1.6.4/go.mod h1:8b3CQMxfWPAeom3zBnGJ6sd+G1NkL5TXqmDXacb+1J0=
github.com/cloudflare/cfssl v1.6.5 h1:46zpNkm6dlNkMZH/wMW22ejih6gIaJbzL2du6vD7ZeI=
github.com/cloudflare/cfssl v1.6.5/go.mod h1:Bk1si7sq8h2+yVEDrFJiz3d7Aw+pfjjJSZVaD+Taky4=
github.com/cloudflare/circl v1.1.0/go.mod h1:prBCrKB9DV4poKZY1l9zBXg2QJY7mvgRvtMxxK7fi4I=
github.com/cloudflare/circl v1.6.3 h1:9GPOhQGF9MCYUeXyMYlqTR6a5gTrgR/fBLXvUgtVcg8=
github.com/cloudflare/circl v1.6.3/go.mod h1:2eXP6Qfat4O/Yhh8BznvKnJ+uzEoTQ6jVKJRn81BiS4=
@@ -297,6 +337,8 @@ github.com/containerd/errdefs v1.0.0/go.mod h1:+YBYIdtsnF4Iw6nWZhJcqGSg/dwvV7tyJ
github.com/containerd/errdefs/pkg v0.3.0 h1:9IKJ06FvyNlexW690DXuQNx2KA2cUJXx151Xdx3ZPPE=
github.com/containerd/errdefs/pkg v0.3.0/go.mod h1:NJw6s9HwNuRhnjJhM7pylWwMyAkmCQvQ4GpJHEqRLVk=
github.com/coreos/go-systemd/v22 v22.5.0/go.mod h1:Y58oyj3AT4RCenI/lSvhwexgC+NSVTIJ3seZv2GcEnc=
github.com/cpuguy83/go-md2man/v2 v2.0.5 h1:ZtcqGrnekaHpVLArFSe4HK5DoKx1T0rq2DwVB0alcyc=
github.com/cpuguy83/go-md2man/v2 v2.0.5/go.mod h1:tgQtvFlXSQOSOSIRvRPT7W67SCa46tRHOmNcaadrF8o=
github.com/creack/pty v1.1.24 h1:bJrF4RRfyJnbTJqzRLHzcGaZK1NeM5kTC9jGgovnR1s=
github.com/creack/pty v1.1.24/go.mod h1:08sCNb52WyoAwi2QDyzUCTgcvVFhUzewun7wtTfvcwE=
github.com/cyphar/filepath-securejoin v0.6.1 h1:5CeZ1jPXEiYt3+Z6zqprSAgSWiggmpVyciv8syjIpVE=
@@ -328,6 +370,8 @@ github.com/docker/go-connections v0.7.0 h1:6SsRfJddP22WMrCkj19x9WKjEDTB+ahsdiGYf
github.com/docker/go-connections v0.7.0/go.mod h1:no1qkHdjq7kLMGUXYAduOhYPSJxxvgWBh7ogVvptn3Q=
github.com/docker/go-units v0.5.0 h1:69rxXcBk27SvSaaxTtLh/8llcHD8vYHT7WSdRZ/jvr4=
github.com/docker/go-units v0.5.0/go.mod h1:fgPhTUdO+D/Jk86RDLlptpiXQzgHJF7gydDDbaIK4Dk=
github.com/dop251/goja v0.0.0-20260311135729-065cd970411c h1:OcLmPfx1T1RmZVHHFwWMPaZDdRf0DBMZOFMVWJa7Pdk=
github.com/dop251/goja v0.0.0-20260311135729-065cd970411c/go.mod h1:MxLav0peU43GgvwVgNbLAj1s/bSGboKkhuULvq/7hx4=
github.com/dsnet/compress v0.0.2-0.20230904184137-39efe44ab707 h1:2tV76y6Q9BB+NEBasnqvs7e49aEBFI8ejC89PSnWH+4=
github.com/dsnet/compress v0.0.2-0.20230904184137-39efe44ab707/go.mod h1:qssHWj60/X5sZFNxpG4HBPDHVqxNm4DfnCKgrbZOT+s=
github.com/dsnet/golib v0.0.0-20171103203638-1ea166775780/go.mod h1:Lj+Z9rebOhdfkVLjJ8T6VcRQv3SXugXy999NBtR9aFY=
@@ -343,8 +387,12 @@ github.com/emirpasic/gods v1.18.1/go.mod h1:8tpGGwCnJ5H4r6BWwaV6OrWmMoPhUl5jm/FM
github.com/envoyproxy/go-control-plane v0.9.0/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4=
github.com/envoyproxy/go-control-plane v0.9.1-0.20191026205805-5f8ba28d4473/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4=
github.com/envoyproxy/go-control-plane v0.9.4/go.mod h1:6rpuAdCZL397s3pYoYcLgu1mIlRU8Am5FuJP05cCM98=
github.com/envoyproxy/go-control-plane v0.14.0 h1:hbG2kr4RuFj222B6+7T83thSPqLjwBIfQawTkC++2HA=
github.com/envoyproxy/go-control-plane v0.14.0/go.mod h1:NcS5X47pLl/hfqxU70yPwL9ZMkUlwlKxtAohpi2wBEU=
github.com/envoyproxy/go-control-plane/envoy v1.37.0 h1:u3riX6BoYRfF4Dr7dwSOroNfdSbEPe9Yyl09/B6wBrQ=
github.com/envoyproxy/go-control-plane/envoy v1.37.0/go.mod h1:DReE9MMrmecPy+YvQOAOHNYMALuowAnbjjEMkkWOi6A=
github.com/envoyproxy/go-control-plane/ratelimit v0.1.0 h1:/G9QYbddjL25KvtKTv3an9lx6VBE2cnb8wp1vEGNYGI=
github.com/envoyproxy/go-control-plane/ratelimit v0.1.0/go.mod h1:Wk+tMFAFbCXaJPzVVHnPgRKdUdwW/KdbRt94AzgRee4=
github.com/envoyproxy/protoc-gen-validate v0.1.0/go.mod h1:iSmxcyjqTsJpI2R4NaDN7+kN2VEUnK/pcBlmesArF7c=
github.com/envoyproxy/protoc-gen-validate v1.3.3 h1:MVQghNeW+LZcmXe7SY1V36Z+WFMDjpqGAGacLe2T0ds=
github.com/envoyproxy/protoc-gen-validate v1.3.3/go.mod h1:TsndJ/ngyIdQRhMcVVGDDHINPLWB7C82oDArY51KfB0=
@@ -377,6 +425,10 @@ github.com/geoffgarside/ber v1.2.0 h1:/loowoRcs/MWLYmGX9QtIAbA+V/FrnVLsMMPhwiRm6
github.com/geoffgarside/ber v1.2.0/go.mod h1:jVPKeCbj6MvQZhwLYsGwaGI52oUorHoHKNecGT85ZCc=
github.com/getkin/kin-openapi v0.144.0 h1:hIRcTH+KjLfkLpYU6bSSfdFpi0fZi1fp+hSPi4aQu9Y=
github.com/getkin/kin-openapi v0.144.0/go.mod h1:3BH9M9XDe/y9M5DSvEocVYAYq1w0qrhJHjC/vZi0AaY=
github.com/getsops/gopgagent v0.0.0-20240527072608-0c14999532fe h1:QKe/kmAYbndxwu91TcjHERsnMh5SgOB1x/qicvOdUJ8=
github.com/getsops/gopgagent v0.0.0-20240527072608-0c14999532fe/go.mod h1:awFzISqLJoZLm+i9QQ4SgMNHDqljH6jWV0B36V5MrUM=
github.com/getsops/sops/v3 v3.9.2 h1:Cuuahc/UGu7W3+NXd+mrl+2oXGxxj/jIPsw6Ah2CbxA=
github.com/getsops/sops/v3 v3.9.2/go.mod h1:J8CMEdmPWT8gxFaNmPBSrc5NFd4uNndf8aFQwhD/ZR8=
github.com/gin-contrib/sse v0.1.0 h1:Y/yl/+YNO8GZSjAhjMsSuLt29uWRFHdHYUb5lYOV9qE=
github.com/gin-contrib/sse v0.1.0/go.mod h1:RHrZQHXnP2xjPF+u1gW/2HnVO7nvIa9PG3Gm+fLHvGI=
github.com/gin-gonic/gin v1.9.1 h1:4idEAncQnU5cB7BeOkPtxjfCSye0AAm1R0RVIqJ+Jmg=
@@ -448,6 +500,8 @@ github.com/go-sourcemap/sourcemap v2.1.4+incompatible/go.mod h1:F8jJfvm2KbVjc5Nq
github.com/go-sql-driver/mysql v1.10.0 h1:Q+1LV8DkHJvSYAdR83XzuhDaTykuDx0l6fkXxoWCWfw=
github.com/go-sql-driver/mysql v1.10.0/go.mod h1:M+cqaI7+xxXGG9swrdeUIoPG3Y3KCkF0pZej+SK+nWk=
github.com/go-stack/stack v1.8.0/go.mod h1:v0f6uXyyMGvRgIKkXu+yp6POWl0qKG85gN/melR3HDY=
github.com/go-test/deep v1.0.2 h1:onZX1rnHT3Wv6cqNgYyFOOlgVKJrksuCMCRvJStbMYw=
github.com/go-test/deep v1.0.2/go.mod h1:wGDj63lr65AM2AQyKZd/NYHGb0R+1RLqB8NKt3aSFNA=
github.com/go-viper/mapstructure/v2 v2.5.0 h1:vM5IJoUAy3d7zRSVtIwQgBj7BiWtMPfmPEgAXnvj1Ro=
github.com/go-viper/mapstructure/v2 v2.5.0/go.mod h1:oJDH3BJKyqBA2TXFhDsKDGDTlndYOZ6rGS0BRZIxGhM=
github.com/goburrow/cache v0.1.4 h1:As4KzO3hgmzPlnaMniZU9+VmoNYseUhuELbxy9mRBfw=
@@ -513,8 +567,8 @@ github.com/golang/snappy v1.0.0 h1:Oy607GVXHs7RtbggtPBnr2RmDArIsAefDwvrdWvRhGs=
github.com/golang/snappy v1.0.0/go.mod h1:/XxbfmMg8lxefKM7IXC3fBNl/7bRcc72aCRzEWrmP2Q=
github.com/google/btree v0.0.0-20180813153112-4030bb1f1f0c/go.mod h1:lNA+9X1NB3Zf8V7Ke586lFgjr2dZNuvo3lPJSGZ5JPQ=
github.com/google/btree v1.0.0/go.mod h1:lNA+9X1NB3Zf8V7Ke586lFgjr2dZNuvo3lPJSGZ5JPQ=
github.com/google/certificate-transparency-go v1.3.2 h1:9ahSNZF2o7SYMaKaXhAumVEzXB2QaayzII9C8rv7v+A=
github.com/google/certificate-transparency-go v1.3.2/go.mod h1:H5FpMUaGa5Ab2+KCYsxg6sELw3Flkl7pGZzWdBoYLXs=
github.com/google/certificate-transparency-go v1.3.3 h1:hq/rSxztSkXN2tx/3jQqF6Xc0O565UQPdHrOWvZwybo=
github.com/google/certificate-transparency-go v1.3.3/go.mod h1:iR17ZgSaXRzSa5qvjFl8TnVD5h8ky2JMVio+dzoKMgA=
github.com/google/go-cmp v0.2.0/go.mod h1:oXzfMopK8JAjlY9xF4vHSVASa0yLyX7SntLO5aqRK0M=
github.com/google/go-cmp v0.3.0/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=
github.com/google/go-cmp v0.3.1/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=
@@ -535,13 +589,17 @@ github.com/google/go-github/v30 v30.1.0 h1:VLDx+UolQICEOKu2m4uAoMti1SxuEBAl7RSEG
github.com/google/go-github/v30 v30.1.0/go.mod h1:n8jBpHl45a/rlBUtRJMOG4GhNADUQFEufcolZ95JfU8=
github.com/google/go-github/v50 v50.2.0/go.mod h1:VBY8FB6yPIjrtKhozXv4FQupxKLS6H4m6xFZlT43q8Q=
github.com/google/go-querystring v1.0.0/go.mod h1:odCYkC5MyYFN7vkCjXpyrEuKhc/BUO6wN/zVPAxq5ck=
github.com/google/go-querystring v1.1.0 h1:AnCroh3fv4ZBgVIf1Iwtovgjaw/GiKJo8M8yD/fhyJ8=
github.com/google/go-querystring v1.1.0/go.mod h1:Kcdr2DB4koayq7X8pmAG4sNG59So17icRSOU623lUBU=
github.com/google/go-querystring v1.2.0 h1:yhqkPbu2/OH+V9BfpCVPZkNmUXhb2gBxJArfhIxNtP0=
github.com/google/go-querystring v1.2.0/go.mod h1:8IFJqpSRITyJ8QhQ13bmbeMBDfmeEJZD5A0egEOmkqU=
github.com/google/gofuzz v1.0.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg=
github.com/google/gofuzz v1.2.0 h1:xRy4A+RhZaiKjJ1bPfwQ8sedCA+YS2YcCHW6ec7JMi0=
github.com/google/gofuzz v1.2.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg=
github.com/google/martian v2.1.0+incompatible h1:/CP5g8u/VJHijgedC/Legn3BAbAaWPgecwXBIDzw5no=
github.com/google/martian v2.1.0+incompatible/go.mod h1:9I4somxYTbIHy5NJKHRl3wXiIaQGbYVAs8BPL6v8lEs=
github.com/google/martian/v3 v3.0.0/go.mod h1:y5Zk1BBys9G+gd6Jrk0W3cC1+ELVxBWuIGO+w/tUAp0=
github.com/google/martian/v3 v3.3.3 h1:DIhPTQrbPkgs2yJYdXU/eNACCG5DVQjySNRNlflZ9Fc=
github.com/google/martian/v3 v3.3.3/go.mod h1:iEPrYcgCF7jA9OtScMFQyAlZZ4YXTKEtJ1E6RWzmBA0=
github.com/google/pprof v0.0.0-20181206194817-3ea8567a2e57/go.mod h1:zfwlbNMJ+OItoe0UupaVj+oy1omPYYDuagoSzA8v9mc=
github.com/google/pprof v0.0.0-20190515194954-54271f7e092f/go.mod h1:zfwlbNMJ+OItoe0UupaVj+oy1omPYYDuagoSzA8v9mc=
github.com/google/pprof v0.0.0-20191218002539-d4f498aebedc/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM=
@@ -553,12 +611,18 @@ github.com/google/pprof v0.0.0-20240227163752-401108e1b7e7/go.mod h1:czg5+yv1E0Z
github.com/google/pprof v0.0.0-20240727154555-813a5fbdbec8 h1:FKHo8hFI3A+7w0aUQuYXQ+6EN5stWmeY/AZqtM8xk9k=
github.com/google/pprof v0.0.0-20240727154555-813a5fbdbec8/go.mod h1:K1liHPHnj73Fdn/EKuT8nrFqBihUSKXoLYU0BuatOYo=
github.com/google/renameio v0.1.0/go.mod h1:KWCgfxg9yswjAJkECMjeO8J8rahYeXnNhOm40UhjYkI=
github.com/google/s2a-go v0.1.9 h1:LGD7gtMgezd8a/Xak7mEWL0PjoTQFvpRudN895yqKW0=
github.com/google/s2a-go v0.1.9/go.mod h1:YA0Ei2ZQL3acow2O62kdp9UlnvMmU7kA6Eutn0dXayM=
github.com/google/shlex v0.0.0-20191202100458-e7afc7fbc510 h1:El6M4kTTCOh6aBiKaUGG7oYTSPP8MxqL4YI3kZKwcP4=
github.com/google/shlex v0.0.0-20191202100458-e7afc7fbc510/go.mod h1:pupxD2MaaD3pAXIBCelhxNneeOaAeabZDe5s4K6zSpQ=
github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/googleapis/enterprise-certificate-proxy v0.3.11 h1:vAe81Msw+8tKUxi2Dqh/NZMz7475yUvmRIkXr4oN2ao=
github.com/googleapis/enterprise-certificate-proxy v0.3.11/go.mod h1:RFV7MUdlb7AgEq2v7FmMCfeSMCllAzWxFgRdusoGks8=
github.com/googleapis/gax-go/v2 v2.0.4/go.mod h1:0Wqv26UfaUD9n4G6kQubkQ+KchISgw+vpHVxEJEs9eg=
github.com/googleapis/gax-go/v2 v2.0.5/go.mod h1:DWXyrwAJ9X0FpwwEdw+IPEYBICEFu5mhpdKc/us6bOk=
github.com/googleapis/gax-go/v2 v2.17.0 h1:RksgfBpxqff0EZkDWYuz9q/uWsTVz+kf43LsZ1J6SMc=
github.com/googleapis/gax-go/v2 v2.17.0/go.mod h1:mzaqghpQp4JDh3HvADwrat+6M3MOIDp5YKHhb9PAgDY=
github.com/gopherjs/gopherjs v0.0.0-20181017120253-0766667cb4d1 h1:EGx4pi6eqNxGaHF6qqu48+N2wcFQ5qg5FXgOdqsJ5d8=
github.com/gopherjs/gopherjs v0.0.0-20181017120253-0766667cb4d1/go.mod h1:wJfORRmW1u3UXTncJ5qlYoELFm8eSnnEO6hX4iZ3EWY=
github.com/gorilla/css v1.0.1 h1:ntNaBIghp6JmvWnxbZKANoLyuXTPZ4cAMlo6RyhlbO8=
@@ -573,24 +637,43 @@ github.com/gosimple/slug v1.15.0 h1:wRZHsRrRcs6b0XnxMUBM6WK1U1Vg5B0R7VkIf1Xzobo=
github.com/gosimple/slug v1.15.0/go.mod h1:UiRaFH+GEilHstLUmcBgWcI42viBN7mAb818JrYOeFQ=
github.com/gosimple/unidecode v1.0.1 h1:hZzFTMMqSswvf0LBJZCZgThIZrpDHFXux9KeGmn6T/o=
github.com/gosimple/unidecode v1.0.1/go.mod h1:CP0Cr1Y1kogOtx0bJblKzsVWrqYaqfNOnHzpgWw4Awc=
github.com/goware/prefixer v0.0.0-20160118172347-395022866408 h1:Y9iQJfEqnN3/Nce9cOegemcy/9Ai5k3huT6E80F3zaw=
github.com/goware/prefixer v0.0.0-20160118172347-395022866408/go.mod h1:PE1ycukgRPJ7bJ9a1fdfQ9j8i/cEcRAoLZzbxYpNB/s=
github.com/h2non/filetype v1.1.3 h1:FKkx9QbD7HR/zjK1Ia5XiBsq9zdLi5Kf3zGyFTAFkGg=
github.com/h2non/filetype v1.1.3/go.mod h1:319b3zT68BvV+WRj7cwy856M2ehB3HqNOt6sy1HndBY=
github.com/hashicorp/errwrap v1.0.0/go.mod h1:YH+1FKiLXxHSkmPseP+kNlulaMuP3n2brvKWEqk/Jc4=
github.com/hashicorp/errwrap v1.1.0 h1:OxrOeh75EUXMY8TBjag2fzXGZ40LB6IKw45YeGUDY2I=
github.com/hashicorp/errwrap v1.1.0/go.mod h1:YH+1FKiLXxHSkmPseP+kNlulaMuP3n2brvKWEqk/Jc4=
github.com/hashicorp/go-cleanhttp v0.5.2 h1:035FKYIWjmULyFRBKPs8TBQoi0x6d9G4xc9neXJWAZQ=
github.com/hashicorp/go-cleanhttp v0.5.2/go.mod h1:kO/YDlP8L1346E6Sodw+PrpBSV4/SoxCXGY6BqNFT48=
github.com/hashicorp/go-hclog v1.6.3 h1:Qr2kF+eVWjTiYmU7Y31tYlP1h0q/X3Nl3tPGdaB11/k=
github.com/hashicorp/go-hclog v1.6.3/go.mod h1:W4Qnvbt70Wk/zYJryRzDRU/4r0kIg0PVHBcfoyhpF5M=
github.com/hashicorp/go-multierror v1.1.1 h1:H5DkEtf6CXdFp0N0Em5UCwQpXMWke8IA0+lD48awMYo=
github.com/hashicorp/go-multierror v1.1.1/go.mod h1:iw975J/qwKPdAO1clOe2L8331t/9/fmwbPZ6JB6eMoM=
github.com/hashicorp/go-retryablehttp v0.7.8 h1:ylXZWnqa7Lhqpk0L1P1LzDtGcCR0rPVUrx/c8Unxc48=
github.com/hashicorp/go-retryablehttp v0.7.8/go.mod h1:rjiScheydd+CxvumBsIrFKlx3iS0jrZ7LvzFGFmuKbw=
github.com/hashicorp/go-rootcerts v1.0.2 h1:jzhAVGtqPKbwpyCPELlgNWhE1znq+qwJtW5Oi2viEzc=
github.com/hashicorp/go-rootcerts v1.0.2/go.mod h1:pqUvnprVnM5bf7AOirdbb01K4ccR319Vf4pU3K5EGc8=
github.com/hashicorp/go-secure-stdlib/parseutil v0.1.8 h1:iBt4Ew4XEGLfh6/bPk4rSYmuZJGizr6/x/AEizP0CQc=
github.com/hashicorp/go-secure-stdlib/parseutil v0.1.8/go.mod h1:aiJI+PIApBRQG7FZTEBx5GiiX+HbOHilUdNxUZi4eV0=
github.com/hashicorp/go-secure-stdlib/strutil v0.1.2 h1:kes8mmyCpxJsI7FTwtzRqEy9CdjCtrXrXGuOpxEA7Ts=
github.com/hashicorp/go-secure-stdlib/strutil v0.1.2/go.mod h1:Gou2R9+il93BqX25LAKCLuM+y9U2T4hlwvT1yprcna4=
github.com/hashicorp/go-sockaddr v1.0.7 h1:G+pTkSO01HpR5qCxg7lxfsFEZaG+C0VssTy/9dbT+Fw=
github.com/hashicorp/go-sockaddr v1.0.7/go.mod h1:FZQbEYa1pxkQ7WLpyXJ6cbjpT8q0YgQaK/JakXqGyWw=
github.com/hashicorp/go-uuid v1.0.2/go.mod h1:6SBZvOh/SIDV7/2o3Jml5SYk/TvGqwFJ/bN7x4byOro=
github.com/hashicorp/go-uuid v1.0.3 h1:2gKiV6YVmrJ1i2CKKa9obLvRieoRGviZFL26PcT/Co8=
github.com/hashicorp/go-uuid v1.0.3/go.mod h1:6SBZvOh/SIDV7/2o3Jml5SYk/TvGqwFJ/bN7x4byOro=
github.com/hashicorp/go-version v1.5.0/go.mod h1:fltr4n8CU8Ke44wwGCBoEymUuxUHl09ZGVZPK5anwXA=
github.com/hashicorp/go-version v1.8.0 h1:KAkNb1HAiZd1ukkxDFGmokVZe1Xy9HG6NUp+bPle2i4=
github.com/hashicorp/go-version v1.8.0/go.mod h1:fltr4n8CU8Ke44wwGCBoEymUuxUHl09ZGVZPK5anwXA=
github.com/hashicorp/go-version v1.9.0 h1:CeOIz6k+LoN3qX9Z0tyQrPtiB1DFYRPfCIBtaXPSCnA=
github.com/hashicorp/go-version v1.9.0/go.mod h1:fltr4n8CU8Ke44wwGCBoEymUuxUHl09ZGVZPK5anwXA=
github.com/hashicorp/golang-lru v0.5.0/go.mod h1:/m3WP610KZHVQ1SGc6re/UDhFvYD7pJ4Ao+sR/qLZy8=
github.com/hashicorp/golang-lru v0.5.1/go.mod h1:/m3WP610KZHVQ1SGc6re/UDhFvYD7pJ4Ao+sR/qLZy8=
github.com/hashicorp/golang-lru/v2 v2.0.7 h1:a+bsQ5rvGLjzHuww6tVxozPZFVghXaHOwFs4luLUK2k=
github.com/hashicorp/golang-lru/v2 v2.0.7/go.mod h1:QeFd9opnmA6QUJc5vARoKUSoFhyfM2/ZepoAG6RGpeM=
github.com/hashicorp/hcl v1.0.0 h1:0Anlzjpi4vEasTeNFn2mLJgTSwt0+6sfsiTG8qcWGx4=
github.com/hashicorp/hcl v1.0.0/go.mod h1:E5yfLk+7swimpb2L/Alb/PJmXilQ/rhwaUYs4T20WEQ=
github.com/hashicorp/vault/api v1.15.0 h1:O24FYQCWwhwKnF7CuSqP30S51rTV7vz1iACXE/pj5DA=
github.com/hashicorp/vault/api v1.15.0/go.mod h1:+5YTO09JGn0u+b6ySD/LLVf8WkJCPLAL2Vkmrn2+CM8=
github.com/hbakhtiyor/strsim v0.0.0-20190107154042-4d2bbb273edf h1:umfGUaWdFP2s6457fz1+xXYIWDxdGc7HdkLS9aJ1skk=
github.com/hbakhtiyor/strsim v0.0.0-20190107154042-4d2bbb273edf/go.mod h1:V99KdStnMHZsvVOwIvhfcUzYgYkRZeQWUtumtL+SKxA=
github.com/hdm/jarm-go v0.0.7/go.mod h1:kinGoS0+Sdn1Rr54OtanET5E5n7AlD6T6CrJAKDjJSQ=
@@ -703,8 +786,8 @@ github.com/lor00x/goldap v0.0.0-20240304151906-8d785c64d1c8 h1:z9RDOBcFcf3f2hSfK
github.com/lor00x/goldap v0.0.0-20240304151906-8d785c64d1c8/go.mod h1:37YR9jabpiIxsb8X9VCIx8qFOjTDIIrIHHODa8C4gz0=
github.com/lucasb-eyer/go-colorful v1.3.0 h1:2/yBRLdWBZKrf7gB40FoiKfAWYQ0lqNcbuQwVHXptag=
github.com/lucasb-eyer/go-colorful v1.3.0/go.mod h1:R4dSotOR9KMtayYi1e77YzuveK+i7ruzyGqttikkLy0=
github.com/lufia/plan9stats v0.0.0-20250827001030-24949be3fa54 h1:mFWunSatvkQQDhpdyuFAYwyAan3hzCuma+Pz8sqvOfg=
github.com/lufia/plan9stats v0.0.0-20250827001030-24949be3fa54/go.mod h1:autxFIvghDt3jPTLoqZ9OZ7s9qTGNAWmYCjVFWPX/zg=
github.com/lufia/plan9stats v0.0.0-20251013123823-9fd1530e3ec3 h1:PwQumkgq4/acIiZhtifTV5OUqqiP82UAl0h87xj/l9k=
github.com/lufia/plan9stats v0.0.0-20251013123823-9fd1530e3ec3/go.mod h1:autxFIvghDt3jPTLoqZ9OZ7s9qTGNAWmYCjVFWPX/zg=
github.com/mackerelio/go-osstat v0.2.6 h1:gs4U8BZeS1tjrL08tt5VUliVvSWP26Ai2Ob8Lr7f2i0=
github.com/mackerelio/go-osstat v0.2.6/go.mod h1:lRy8V9ZuHpuRVZh+vyTkODeDPl3/d5MgXHtLSaqG8bA=
github.com/mailru/easyjson v0.7.7 h1:UGYAvKxe3sBsEDzO8ZeWOSlIQfWFlxbzLZe7hwFURr0=
@@ -734,8 +817,8 @@ github.com/microcosm-cc/bluemonday v1.0.27/go.mod h1:jFi9vgW+H7c3V0lb6nR74Ib/DIB
github.com/microsoft/go-mssqldb v1.9.2 h1:nY8TmFMQOHpm2qVWo6y4I2mAmVdZqlGiMGAYt64Ibbs=
github.com/microsoft/go-mssqldb v1.9.2/go.mod h1:GBbW9ASTiDC+mpgWDGKdm3FnFLTUsLYN3iFL90lQ+PA=
github.com/miekg/dns v1.1.35/go.mod h1:KNUDUusw/aVsxyTYZM1oqvCicbwhgbNgztCETuNZ7xM=
github.com/miekg/dns v1.1.72 h1:vhmr+TF2A3tuoGNkLDFK9zi36F2LS+hKTRW0Uf8kbzI=
github.com/miekg/dns v1.1.72/go.mod h1:+EuEPhdHOsfk6Wk5TT2CzssZdqkmFhf8r+aVyDEToIs=
github.com/miekg/dns v1.1.73 h1:uhT8nJxmTrPJYClxVxTCX+CVn6qnzSiybRk72Z6DgrE=
github.com/miekg/dns v1.1.73/go.mod h1:RW2Obtfd5NZHvOFe3zYG0W8koWOQtAzyHaLo8vASBuQ=
github.com/mikelolasagasti/xz v1.0.1 h1:Q2F2jX0RYJUG3+WsM+FJknv+6eVjsjXNDV0KJXZzkD0=
github.com/mikelolasagasti/xz v1.0.1/go.mod h1:muAirjiOUxPRXwm9HdDtB3uoRPrGnL85XHtokL9Hcgc=
github.com/minio/minio-go/v6 v6.0.46/go.mod h1:qD0lajrGW49lKZLtXKtCB4X/qkMf0a5tBvN2PaZg7Gg=
@@ -746,6 +829,10 @@ github.com/minio/selfupdate v0.6.1-0.20230907112617-f11e74f84ca7/go.mod h1:bO02G
github.com/minio/sha256-simd v0.1.1/go.mod h1:B5e1o+1/KgNmWrSQK08Y6Z1Vb5pwIktudl0J58iy0KM=
github.com/mitchellh/go-homedir v1.1.0 h1:lukF9ziXFxDFPkA1vsr5zpc1XuPDn/wFntq5mG+4E0Y=
github.com/mitchellh/go-homedir v1.1.0/go.mod h1:SfyaCUpYCn1Vlf4IUYiD9fPX4A5wJrkLzIz1N1q0pr0=
github.com/mitchellh/go-wordwrap v1.0.1 h1:TLuKupo69TCn6TQSyGxwI1EblZZEsQ0vMlAFQflz0v0=
github.com/mitchellh/go-wordwrap v1.0.1/go.mod h1:R62XHJLzvMFRBbcrT7m7WgmE1eOyTSsCt+hzestvNj0=
github.com/mitchellh/mapstructure v1.5.0 h1:jeMsZIYE/09sWLaz43PL7Gy6RuMjD2eJVyuac5Z2hdY=
github.com/mitchellh/mapstructure v1.5.0/go.mod h1:bFUtVrKA4DC2yAKiSyO/QUcy7e+RRV2QTWOzhPopBRo=
github.com/moby/docker-image-spec v1.3.1 h1:jMKff3w6PgbfSa69GfNg+zN/XLhfXJGnEx3Nl2EsFP0=
github.com/moby/docker-image-spec v1.3.1/go.mod h1:eKmb5VW8vQEh/BAr2yvVNvuiJuY6UIocYsFu/DxxRpo=
github.com/moby/moby/api v1.54.2 h1:wiat9QAhnDQjA7wk1kh/TqHz2I1uUA7M7t9SAl/JNXg=
@@ -835,28 +922,28 @@ github.com/projectdiscovery/asnmap v1.1.1 h1:ImJiKIaACOT7HPx4Pabb5dksolzaFYsD1kI
github.com/projectdiscovery/asnmap v1.1.1/go.mod h1:QT7jt9nQanj+Ucjr9BqGr1Q2veCCKSAVyUzLXfEcQ60=
github.com/projectdiscovery/blackrock v0.0.2 h1:mxXdu0uM8P2L2Qi210COlU8QiICPFW/Rxk5QUhlPO2k=
github.com/projectdiscovery/blackrock v0.0.2/go.mod h1:ANUtjDfaVrqB453bzToU+YB4cUbvBRpLvEwoWIwlTss=
github.com/projectdiscovery/cdncheck v1.2.50 h1:KZBYT4jKNvlmlDKJqGMJ7r6XtwAim82oEZLaI+NAWWk=
github.com/projectdiscovery/cdncheck v1.2.50/go.mod h1:9oE9KKxCSHNvUf0UaMeqqUwWpC38FkNaTll0ScIBT3w=
github.com/projectdiscovery/clistats v0.1.6 h1:e2Hc7cVWKJ4sCbyQ7th5zuBvgqsZcLwRq1qY0vmQXY4=
github.com/projectdiscovery/clistats v0.1.6/go.mod h1:RxRGxhMzkT344VTTvZWIrojurUXdQQRN1yfwgg/88I8=
github.com/projectdiscovery/dsl v0.8.22 h1:ZfPunLaaYUJTBffE1ShuAAV9hkSH2thvlszK48FlZS0=
github.com/projectdiscovery/dsl v0.8.22/go.mod h1:fr8WMU381As89e4Z+SQ8tYQ1NudniWAS6NLPHrhe+Vs=
github.com/projectdiscovery/fastdialer v0.5.18 h1:bBGJdni/xruoUISBug86KwRpuewLg1lEUfQLTg3p0G0=
github.com/projectdiscovery/fastdialer v0.5.18/go.mod h1:fH4CDmC7Dk7Shm4v3Rsql7ql3OhlKeZfOdAlc/muP1I=
github.com/projectdiscovery/cdncheck v1.3.0 h1:/bWdNXIU1P5ASZuW8QA+tAw1FNHhmWlWXk/qiUHyx2c=
github.com/projectdiscovery/cdncheck v1.3.0/go.mod h1:GIAA0SqjDfxzDQjHR/X+bK05BGmUGfsVplP5Zdv/kHA=
github.com/projectdiscovery/clistats v0.1.7 h1:NXf4lUzx1QcbJd6QycNwwYHTv2FsW9MchQ2LsgSL6C0=
github.com/projectdiscovery/clistats v0.1.7/go.mod h1:RxRGxhMzkT344VTTvZWIrojurUXdQQRN1yfwgg/88I8=
github.com/projectdiscovery/dsl v0.8.23-0.20260910174914-8e1207b367b8 h1:zaAfFHjyLPq9Aoii1mXPzXkITvXyPJTSZo6nNhzC1xI=
github.com/projectdiscovery/dsl v0.8.23-0.20260910174914-8e1207b367b8/go.mod h1:TYqDNtD4hU8bGFgVxPqhYQ9K1z5rMLV/YFyvQgcU5TM=
github.com/projectdiscovery/fastdialer v0.5.20 h1:MlXoVXQERJY4ILqMNYfuoTKVR4Ix4XonJLkFaoHzgM0=
github.com/projectdiscovery/fastdialer v0.5.20/go.mod h1:zdtIbBt5ELNygMEPDSyQ9Nr9KpP9FuAvJwzNIrZZoJA=
github.com/projectdiscovery/fasttemplate v0.0.2 h1:h2cISk5xDhlJEinlBQS6RRx0vOlOirB2y3Yu4PJzpiA=
github.com/projectdiscovery/fasttemplate v0.0.2/go.mod h1:XYWWVMxnItd+r0GbjA1GCsUopMw1/XusuQxdyAIHMCw=
github.com/projectdiscovery/freeport v0.0.7 h1:Q6uXo/j8SaV/GlAHkEYQi8WQoPXyJWxyspx+aFmz9Qk=
github.com/projectdiscovery/freeport v0.0.7/go.mod h1:cOhWKvNBe9xM6dFJ3RrrLvJ5vXx2NQ36SecuwjenV2k=
github.com/projectdiscovery/gcache v0.0.0-20241015120333-12546c6e3f4c h1:s+lLAlrOrgwlPZQ9DFqNw+kia2nteKnJZ2Ek313yoUc=
github.com/projectdiscovery/gcache v0.0.0-20241015120333-12546c6e3f4c/go.mod h1:rN35/D3lVx2YDeENFFz06uj8j3XIqK1Ym9XcISF5fzg=
github.com/projectdiscovery/goflags v0.2.0 h1:zj16hnTmLmlnVYm02hfOrCY6oYTu/AQDBE99yZkS+s8=
github.com/projectdiscovery/goflags v0.2.0/go.mod h1:7nAP1r2Dqgn/rwmOE3EWbZWUCEJKNIhVSBGpuzJAIns=
github.com/projectdiscovery/goflags v0.2.1 h1:XnSYvsCjUwHbZaaJUZ3YznCJPoXRk/IhTFP9Gecmx6M=
github.com/projectdiscovery/goflags v0.2.1/go.mod h1:7nAP1r2Dqgn/rwmOE3EWbZWUCEJKNIhVSBGpuzJAIns=
github.com/projectdiscovery/goja v0.0.0-20260618133720-acb73e419534 h1:hYd1zQA/dxO2ASyQ6Re73TcJkW1LjLQvt4+86Hxefz8=
github.com/projectdiscovery/goja v0.0.0-20260618133720-acb73e419534/go.mod h1:SO0AP+uKfYeeoR6jyVH/PKRYJE/f5gJrPMAM00iGEMc=
github.com/projectdiscovery/goja_nodejs v0.0.0-20260618132410-8519f75f703d h1:fqqH9LHpN2WDz9QuxFrhKNxXSRtzk+Sa6jAhbB7tXcQ=
github.com/projectdiscovery/goja_nodejs v0.0.0-20260618132410-8519f75f703d/go.mod h1:Ezmbgdaw4EunGGBU4MQViLoGMJc37LA3ip55YV3KeRI=
github.com/projectdiscovery/gologger v1.1.72 h1:PKk+aSx3jYOCBPi+FD+nTnhe+Vxx9u3K7Ega0GVlMCI=
github.com/projectdiscovery/gologger v1.1.72/go.mod h1:mJwODZcFDg70ihINpOvZevmBtgvpP8H9/l8Y+OPhZPY=
github.com/projectdiscovery/gologger v1.1.73 h1:WX2nzq29cL7HdvOHqIttacoid/5lMwapyM7ukGU1jFY=
github.com/projectdiscovery/gologger v1.1.73/go.mod h1:mJwODZcFDg70ihINpOvZevmBtgvpP8H9/l8Y+OPhZPY=
github.com/projectdiscovery/gostruct v0.0.2 h1:s8gP8ApugGM4go1pA+sVlPDXaWqNP5BBDDSv7VEdG1M=
github.com/projectdiscovery/gostruct v0.0.2/go.mod h1:H86peL4HKwMXcQQtEa6lmC8FuD9XFt6gkNR0B/Mu5PE=
github.com/projectdiscovery/govaluate v0.0.0-20260615100919-5ee2581bbf7e h1:vxzgQlz2Cy/YvizYDQx9OhucBcmBotfDhbQ4yCY2vfA=
@@ -865,10 +952,12 @@ github.com/projectdiscovery/gozero v0.1.1-0.20260530071156-fa1dad563d76 h1:AN70b
github.com/projectdiscovery/gozero v0.1.1-0.20260530071156-fa1dad563d76/go.mod h1:cWHYnRXoYWHtTpOYyAp5laGYX8GH8ITUhgQaP8G/8FA=
github.com/projectdiscovery/hmap v0.0.102 h1:ybPuf7UQvuOLiHLOUbVMvd+PSlItQU5iQpP1Erl8kaU=
github.com/projectdiscovery/hmap v0.0.102/go.mod h1:IZ92wXFmxYpdn0euZ7VFmypzOBNDwq8c3yRyIIol6XE=
github.com/projectdiscovery/httpx v1.10.0 h1:sZvfeJDeX4eJfB5Oh29My4gOHmLfvpCt2wCqhargJrE=
github.com/projectdiscovery/httpx v1.10.0/go.mod h1:gfdNnFznLLlAgAgSQlYjUeEVvYzKrNV0jO5Qyo5w+FI=
github.com/projectdiscovery/httpx v1.12.0 h1:NN5iDrw7NjPBDlACdjac4lePTQmZ5aOmgOzptMpP44c=
github.com/projectdiscovery/httpx v1.12.0/go.mod h1:OgtyibNGdu9BrJxRdNKm280Davw1Z4s2FAhfPNNtLB4=
github.com/projectdiscovery/interactsh v1.3.1 h1:5HzeVGVCAX/cjTguJ+7ClOmML5r97Ty7op9s+/F7BiM=
github.com/projectdiscovery/interactsh v1.3.1/go.mod h1:MXQ11EoBPROb4bEw+WP9e4DX4fMhrpS6EwfMfZomBsw=
github.com/projectdiscovery/jarm-go v0.0.0-20260910160638-430e2bae86f6 h1:GUHQYLllWhoca+C4UAT0+TTB1PrUT05klj39V6/KpK8=
github.com/projectdiscovery/jarm-go v0.0.0-20260910160638-430e2bae86f6/go.mod h1:f8P5X5+jEkUqK9z3oNWtSH1g23jMfw3nuEqmtA1jXA4=
github.com/projectdiscovery/ldapserver v1.0.2-0.20240219154113-dcc758ebc0cb h1:MGtI4oE12ruWv11ZlPXXd7hl/uAaQZrFvrIDYDeVMd8=
github.com/projectdiscovery/ldapserver v1.0.2-0.20240219154113-dcc758ebc0cb/go.mod h1:vmgC0DTFCfoCLp0RAfsfYTZZan0QMVs+cmTbH6blfjk=
github.com/projectdiscovery/machineid v0.0.0-20250715113114-c77eb3567582 h1:eR+0HE//Ciyfwy3HC7fjRyKShSJHYoX2Pv7pPshjK/Q=
@@ -877,34 +966,34 @@ github.com/projectdiscovery/mapcidr v1.1.97 h1:7FkxNNVXp+m1rIu5Nv/2SrF9k4+LwP8Qu
github.com/projectdiscovery/mapcidr v1.1.97/go.mod h1:9dgTJh1SP02gYZdpzMjm6vtYFkEHQHoTyaVNvaeJ7lA=
github.com/projectdiscovery/n3iwf v0.0.0-20230523120440-b8cd232ff1f5 h1:L/e8z8yw1pfT6bg35NiN7yd1XKtJap5Nk6lMwQ0RNi8=
github.com/projectdiscovery/n3iwf v0.0.0-20230523120440-b8cd232ff1f5/go.mod h1:pGW2ncnTxTxHtP9wzcIJAB+3/NMp6IiuQWd2NK7K+oc=
github.com/projectdiscovery/networkpolicy v0.1.48 h1:s/d0nEyfkHYVBeZI/bgRiOCS66KDxv3+s7Hb+AvepHY=
github.com/projectdiscovery/networkpolicy v0.1.48/go.mod h1:bb2ZkDnOeVuR9Ok8BdCdoBEz8S9Y9EFvFFcn5FTUV2U=
github.com/projectdiscovery/ratelimit v0.0.88 h1:AcurW9aLRzlEyPe9kSjnOpr3XzLMWTpiWAlW/w73ALU=
github.com/projectdiscovery/ratelimit v0.0.88/go.mod h1:CU1s+68UUG2mctSl2wi32/DHLJA6TMg+4rxgP59LfVk=
github.com/projectdiscovery/rawhttp v0.1.91 h1:6EIhZxCkBn27kJAl47FieCFCe4H2ZdbhoBVj0J6ZCVY=
github.com/projectdiscovery/rawhttp v0.1.91/go.mod h1:VZYAM25UI/wVB3URZ95ZaftgOnsbphxyAw/XnQRRz4Y=
github.com/projectdiscovery/networkpolicy v0.1.50 h1:z/ZMuNAtPH6Q7BlMZ4DLA1ytm+7TYPAdBCNLjNqaV0U=
github.com/projectdiscovery/networkpolicy v0.1.50/go.mod h1:QAAMLSm1YsbGeZTFNGBMtyob28B1VUVblNT56wPhDJc=
github.com/projectdiscovery/ratelimit v0.0.90-0.20260912164306-aeeca5645294 h1:d5fkf/st46mg+QpOkzOuFBBCYfV8ljO3pmi8CChwIyE=
github.com/projectdiscovery/ratelimit v0.0.90-0.20260912164306-aeeca5645294/go.mod h1:pQg2urKomS0WkMmLCF5HhWyuresBXosh1kIfn1Tlwug=
github.com/projectdiscovery/rawhttp v0.1.92 h1:38GRsVLNllES/dZcEcA6s6P+rYBp34H9mDphMJMDvvg=
github.com/projectdiscovery/rawhttp v0.1.92/go.mod h1:spfXTPjxCj+bfWLynMVB7uOb+LS3PDNAI9HexMEV1Zg=
github.com/projectdiscovery/rdap v0.9.0 h1:wPhHx5pQ2QI+WGhyNb2PjhTl0NtB39Nk7YFZ9cp8ZGA=
github.com/projectdiscovery/rdap v0.9.0/go.mod h1:zk4yrJFQ2Hy36Aqk+DvotYQxYAeALaCJ5ORySkff36Q=
github.com/projectdiscovery/retryabledns v1.0.116 h1:tK28PJ4tLO7bgM89RpIFjkfTPMSbRegwvg9mN/q6z78=
github.com/projectdiscovery/retryabledns v1.0.116/go.mod h1:r3vH0AKYtJ9fculcqRFrdVYe2ABcHwZeR4C6j3rqCbY=
github.com/projectdiscovery/retryablehttp-go v1.3.24 h1:bODdeUEka/U5SS9wfnhoAjBRg1wq/AJdwb9CUNIOR88=
github.com/projectdiscovery/retryablehttp-go v1.3.24/go.mod h1:OL5zZEBd59GXEMaDjEHW/bfGlFsV441ADz64hWajgxc=
github.com/projectdiscovery/retryablehttp-go v1.3.26 h1:BkVqMUIVPhx+3vOoo5CxYGNnMqQQYXOedu89XQB1KTg=
github.com/projectdiscovery/retryablehttp-go v1.3.26/go.mod h1:poffUOX+jvS0XbaPCYess221jXvSO7R/QlE7zhs3+9Q=
github.com/projectdiscovery/sarif v0.1.0 h1:O541T+a448nSJsmIMnXXSOeDQEzpnCAYvRfe0eG5h74=
github.com/projectdiscovery/sarif v0.1.0/go.mod h1:LBC+reM3bkI3qIIhE0rZaINaYX6VG+En6u2hHa5mA7E=
github.com/projectdiscovery/stringsutil v0.0.2 h1:uzmw3IVLJSMW1kEg8eCStG/cGbYYZAja8BH3LqqJXMA=
github.com/projectdiscovery/stringsutil v0.0.2/go.mod h1:EJ3w6bC5fBYjVou6ryzodQq37D5c6qbAYQpGmAy+DC0=
github.com/projectdiscovery/tlsx v1.3.2 h1:1Lh2ith79o8R+rOOSBZiMq8EiZ8j4D3T/8ENO+gAqXA=
github.com/projectdiscovery/tlsx v1.3.2/go.mod h1:2wgTGC/sourHvoR+RX8cAok0Sa8YK6QbEP3xo5SDzPI=
github.com/projectdiscovery/tlsx v1.4.0 h1:OU2mNK6TOB/YiFX3jhmgwOepFWxPoj5rn8npWiCCz74=
github.com/projectdiscovery/tlsx v1.4.0/go.mod h1:WCrUXAHjdKbDNjsu5EbVBS7qvzap3fxi/E85v/Flyx8=
github.com/projectdiscovery/uncover v1.2.1 h1:8U46T/96CLT7BPoXBgkTvWqB06lOyeTSLvh5+UjzATE=
github.com/projectdiscovery/uncover v1.2.1/go.mod h1:0p8onrWxfpXQEYs90ZDzTSpu1107gWmodX1NWqu/+z4=
github.com/projectdiscovery/useragent v0.0.109 h1:b86BPdZvmgbJousa04jAugDRY4Y+rAvEamE0TFCPmeE=
github.com/projectdiscovery/useragent v0.0.109/go.mod h1:sRyBqDAcD31RKYrO8fyXGowLd36fNwY1g0MNW9mp7zk=
github.com/projectdiscovery/utils v0.11.2 h1:gfvXFBEHN4DgX8HBRzsoO1s2wH/2R+cH17cLpC865x4=
github.com/projectdiscovery/utils v0.11.2/go.mod h1:HMxhxLigsAr+M9Oa8n9Z0ROZcs8wAJHGsgR7UGb+oUE=
github.com/projectdiscovery/wappalyzergo v0.2.95 h1:XHjbQVrrxusezFq7ciWmtznPKWhpr/srWp7Yk2noxqg=
github.com/projectdiscovery/wappalyzergo v0.2.95/go.mod h1:E2p8L90ysTTUkU5FUOgGoCQ7ucEUgqJ/tvYgiGGAlEM=
github.com/projectdiscovery/yamldoc-go v1.0.6 h1:GCEdIRlQjDux28xTXKszM7n3jlMf152d5nqVpVoetas=
github.com/projectdiscovery/yamldoc-go v1.0.6/go.mod h1:R5lWrNzP+7Oyn77NDVPnBsxx2/FyQZBBkIAaSaCQFxw=
github.com/projectdiscovery/utils v0.11.4-0.20260910160917-83d6b6e62668 h1:umwLd2H4q5hfHPdAHSQWdrmBvECZs2qsF9hskvo1jw8=
github.com/projectdiscovery/utils v0.11.4-0.20260910160917-83d6b6e62668/go.mod h1:DzaUDLQ+Tqsn2nOL47M1mRBZSVyCmwsAlR2E9dxHB4Y=
github.com/projectdiscovery/wappalyzergo v0.3.0 h1:2AsEkjnKgFZK4tjaiNhwC0zfj04rpckSvIOZINeTGyQ=
github.com/projectdiscovery/wappalyzergo v0.3.0/go.mod h1:6xTRCUQhX8m+L1eqJDQIALJlRv2Bnhrmku80YU9bBXE=
github.com/projectdiscovery/yamldoc-go v1.0.7 h1:Odmjd1o4EkMff2F6QHi1dIYPIQPZUFy24YqmQCprmTk=
github.com/projectdiscovery/yamldoc-go v1.0.7/go.mod h1:nqD1iPMcofpRA1DbXavPByJhWURrshT+l8WN9XjyQag=
github.com/prometheus/client_golang v0.9.1/go.mod h1:7SWBe2y4D6OKWSNQJUaRYU/AaXPKyh/dDVn+NZz0KFw=
github.com/prometheus/client_golang v1.0.0/go.mod h1:db9x61etRT2tGnBNRi70OPL5FsnadC4Ky3P0J6CfImo=
github.com/prometheus/client_golang v1.7.1/go.mod h1:PY5Wy2awLA44sXw4AOSfFBetzPP4j5+D6mVACh+pe2M=
@@ -946,17 +1035,23 @@ github.com/rs/xid v1.6.0 h1:fV591PaemRlL6JfRxGDEPl69wICngIQ3shQtzfy2gxU=
github.com/rs/xid v1.6.0/go.mod h1:7XoLgs4eV+QndskICGsho+ADou8ySMSjJKDIan90Nz0=
github.com/rs/zerolog v1.34.0 h1:k43nTLIwcTVQAncfCw4KZ2VY6ukYoZaBPNOE8txlOeY=
github.com/rs/zerolog v1.34.0/go.mod h1:bJsvje4Z08ROH4Nhs5iH600c3IkWhwp44iRc54W6wYQ=
github.com/russross/blackfriday/v2 v2.1.0 h1:JIOH55/0cWyOuilr9/qlrm0BSXldqnqwMsf35Ld67mk=
github.com/russross/blackfriday/v2 v2.1.0/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM=
github.com/rwcarlsen/goexif v0.0.0-20190401172101-9e8deecbddbd/go.mod h1:hPqNNc0+uJM6H+SuU8sEs5K5IQeKccPqeSjfgcKGgPk=
github.com/ryanuber/go-glob v1.0.0 h1:iQh3xXAumdQ+4Ufa5b25cRpC5TYKlno6hsv6Cb3pkBk=
github.com/ryanuber/go-glob v1.0.0/go.mod h1:807d1WSdnB0XRJzKNil9Om6lcp/3a0v4qIHxIXzX/Yc=
github.com/saintfish/chardet v0.0.0-20230101081208-5e3ef4b5456d h1:hrujxIzL1woJ7AwssoOcM/tq5JjjG2yYOc8odClEiXA=
github.com/saintfish/chardet v0.0.0-20230101081208-5e3ef4b5456d/go.mod h1:uugorj2VCxiV1x+LzaIdVa9b4S4qGAcH6cbhh4qVxOU=
github.com/sandrolain/httpcache v1.4.0 h1:Jf4Vx62X2ybvNPSpPvI1kT3xvMdDG1AsApQjOQKO9E0=
github.com/sandrolain/httpcache v1.4.0/go.mod h1:kHBuXveitSn39SNPBhdf/ybG272X706HJ2RJqOQ+Em0=
github.com/santhosh-tekuri/jsonschema/v6 v6.0.2 h1:KRzFb2m7YtdldCEkzs6KqmJw4nqEVZGK7IN2kJkjTuQ=
github.com/santhosh-tekuri/jsonschema/v6 v6.0.2/go.mod h1:JXeL+ps8p7/KNMjDQk3TCwPpBy0wYklyWTfbkIzdIFU=
github.com/sashabaranov/go-openai v1.37.0 h1:hQQowgYm4OXJ1Z/wTrE+XZaO20BYsL0R3uRPSpfNZkY=
github.com/sashabaranov/go-openai v1.37.0/go.mod h1:lj5b/K+zjTSFxVLijLSTDZuP7adOgerWeFyZLUhAKRg=
github.com/segmentio/ksuid v1.0.4 h1:sBo2BdShXjmcugAMwjugoGUdUV0pcxY5mW4xKRn3v4c=
github.com/segmentio/ksuid v1.0.4/go.mod h1:/XUiZBD3kVx5SmUOl55voK5yeAbBNNIed+2O73XgrPE=
github.com/sergi/go-diff v1.3.2-0.20230802210424-5b0b94c5c0d3 h1:n661drycOFuPLCN3Uc8sB6B/s6Z4t2xvBgU1htSHuq8=
github.com/sergi/go-diff v1.3.2-0.20230802210424-5b0b94c5c0d3/go.mod h1:A0bzQcvG0E7Rwjx0REVgAGH58e96+X0MeOfepqsbeW4=
github.com/sergi/go-diff v1.4.0 h1:n/SP9D5ad1fORl+llWyN+D6qoUETXNZARKjyY2/KVCw=
github.com/sergi/go-diff v1.4.0/go.mod h1:A0bzQcvG0E7Rwjx0REVgAGH58e96+X0MeOfepqsbeW4=
github.com/shirou/gopsutil v3.21.11+incompatible h1:+1+c1VGhc88SSonWP6foOcLhvnKlUeu/erjjvaPEYiI=
github.com/shirou/gopsutil v3.21.11+incompatible/go.mod h1:5b4v6he4MtMOwMlS0TUMTu2PcXUg8+E1lC7eC3UO/RA=
github.com/shirou/gopsutil/v4 v4.26.3 h1:2ESdQt90yU3oXF/CdOlRCJxrP+Am1aBYubTMTfxJ1qc=
@@ -992,8 +1087,8 @@ github.com/spf13/cast v1.10.0 h1:h2x0u2shc1QuLHfxi+cTJvs30+ZAHOGRic8uyGTDWxY=
github.com/spf13/cast v1.10.0/go.mod h1:jNfB8QC9IA6ZuY2ZjDp0KtFO2LZZlg4S/7bzP6qqeHo=
github.com/spf13/pflag v1.0.10 h1:4EBh2KAYBwaONj6b2Ye1GiHfwjqyROoF4RwYO+vPwFk=
github.com/spf13/pflag v1.0.10/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
github.com/spiffe/go-spiffe/v2 v2.6.0 h1:l+DolpxNWYgruGQVV0xsfeya3CsC7m8iBzDnMpsbLuo=
github.com/spiffe/go-spiffe/v2 v2.6.0/go.mod h1:gm2SeUoMZEtpnzPNs2Csc0D/gX33k1xIx7lEzqblHEs=
github.com/spiffe/go-spiffe/v2 v2.7.0 h1:uXe1MflJoHw58wAUvxVlcM7WpKtijWG7I1UidcGh6g4=
github.com/spiffe/go-spiffe/v2 v2.7.0/go.mod h1:47Q0Q9/AqGha8QLHp+kxpH4Wca7X7EnOtlIJy3mxZ3U=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/objx v0.1.1/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSSt89Yw=
@@ -1012,6 +1107,7 @@ github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO
github.com/stretchr/testify v1.8.1/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4=
github.com/stretchr/testify v1.8.3/go.mod h1:sz/lmYIOXD/1dqDmKjjqLyZ2RngseejIcXlSw2iwfAo=
github.com/stretchr/testify v1.8.4/go.mod h1:sz/lmYIOXD/1dqDmKjjqLyZ2RngseejIcXlSw2iwfAo=
github.com/stretchr/testify v1.9.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY=
github.com/stretchr/testify v1.10.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY=
github.com/stretchr/testify v1.12.1 h1:EuwCh5fleGS7H32xRwO3wRGT7DxrDhLAT6FF8MpWDWE=
github.com/stretchr/testify v1.12.1/go.mod h1:MDEgiDPPsNp5cuIrHPPCyornHKgEVbtFUmoNlxoYthg=
@@ -1061,6 +1157,8 @@ github.com/ugorji/go/codec v1.2.11/go.mod h1:UNopzCgEMSXjBc6AOMqYvWC1ktqTAfzJZUZ
github.com/ulikunitz/xz v0.5.8/go.mod h1:nbz6k7qbPmH4IRqmfOplQw/tblSgqTqBwxkY0oWt/14=
github.com/ulikunitz/xz v0.5.15 h1:9DNdB5s+SgV3bQ2ApL10xRc35ck0DuIX/isZvIk+ubY=
github.com/ulikunitz/xz v0.5.15/go.mod h1:nbz6k7qbPmH4IRqmfOplQw/tblSgqTqBwxkY0oWt/14=
github.com/urfave/cli v1.22.16 h1:MH0k6uJxdwdeWQTwhSO42Pwr4YLrNLwBtg1MRgTqPdQ=
github.com/urfave/cli v1.22.16/go.mod h1:EeJR6BKodywf4zciqrdw6hpCPk68JO9z5LazXZMn5Po=
github.com/valyala/bytebufferpool v1.0.0 h1:GqA5TC/0021Y/b9FG4Oi9Mr3q7XYx6KllzawFIhcdPw=
github.com/valyala/bytebufferpool v1.0.0/go.mod h1:6bBcMArwyJ5K/AmCkWv1jt77kVWyCJ6HpOuEn7z0Csc=
github.com/vmihailenco/bufpool v0.1.11 h1:gOq2WmBrq0i2yW5QJ16ykccQ4wH9UyEsgLm6czKAd94=
@@ -1170,16 +1268,24 @@ go.opencensus.io v0.22.3/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw=
go.opencensus.io v0.22.4/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw=
go.opentelemetry.io/auto/sdk v1.2.1 h1:jXsnJ4Lmnqd11kwkBV2LgLoFMZKizbCi5fNZ/ipaZ64=
go.opentelemetry.io/auto/sdk v1.2.1/go.mod h1:KRTj+aOaElaLi+wW1kO/DZRXwkF4C5xPbEe3ZiIhN7Y=
go.opentelemetry.io/contrib/detectors/gcp v1.44.0 h1:NmLfL734pJhM0JKaYd2Y28+nY9dPRWYAAbxhRCrKXPw=
go.opentelemetry.io/contrib/detectors/gcp v1.44.0/go.mod h1:tNAsgd8avTGke1+MndXlU5Cru4PQ9Ai/cCNWQv/ZJ/s=
go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.61.0 h1:q4XOmH/0opmeuJtPsbFNivyl7bCt7yRBbeEm2sC/XtQ=
go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.61.0/go.mod h1:snMWehoOh2wsEwnvvwtDyFCxVeDAODenXHtn5vzrKjo=
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.68.0 h1:CqXxU8VOmDefoh0+ztfGaymYbhdB/tT3zs79QaZTNGY=
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.68.0/go.mod h1:BuhAPThV8PBHBvg8ZzZ/Ok3idOdhWIodywz2xEcRbJo=
go.opentelemetry.io/otel v1.44.0 h1:JjwHmHpA4iZ3wBxluu2fbbE7j4kqlE8jXyAyPXH7HqU=
go.opentelemetry.io/otel v1.44.0/go.mod h1:BMgjTHL9WPRlRjL2oZCBTL4whCGtXch2H4BhOPIAyYc=
go.opentelemetry.io/otel/exporters/stdout/stdoutmetric v1.36.0 h1:rixTyDGXFxRy1xzhKrotaHy3/KXdPhlWARrCgK+eqUY=
go.opentelemetry.io/otel/exporters/stdout/stdoutmetric v1.36.0/go.mod h1:dowW6UsM9MKbJq5JTz2AMVp3/5iW5I/TStsk8S+CfHw=
go.opentelemetry.io/otel/metric v1.44.0 h1:1w0gILTcHdr3YI+ixLyjemwrVnsMURbTZFrSYCdDdmc=
go.opentelemetry.io/otel/metric v1.44.0/go.mod h1:8O7hanEPBNgEMmybD3s2VBKcgWOCsA6tzHBPODAiquo=
go.opentelemetry.io/otel/sdk v1.43.0 h1:pi5mE86i5rTeLXqoF/hhiBtUNcrAGHLKQdhg4h4V9Dg=
go.opentelemetry.io/otel/sdk v1.43.0/go.mod h1:P+IkVU3iWukmiit/Yf9AWvpyRDlUeBaRg6Y+C58QHzg=
go.opentelemetry.io/otel/sdk/metric v1.43.0 h1:S88dyqXjJkuBNLeMcVPRFXpRw2fuwdvfCGLEo89fDkw=
go.opentelemetry.io/otel/sdk/metric v1.43.0/go.mod h1:C/RJtwSEJ5hzTiUz5pXF1kILHStzb9zFlIEe85bhj6A=
go.opentelemetry.io/otel/metric/x v0.66.0 h1:YkCrx1zLOChi9ZcZ6euupOcsgzbVlec7D/xoEU1+cTA=
go.opentelemetry.io/otel/metric/x v0.66.0/go.mod h1:d1+BDj9t96do0/1LoU1ayfCv79ZgNE41qbhBvnMOBZk=
go.opentelemetry.io/otel/sdk v1.44.0 h1:nHYwb9lK+fJPU/dnT6s7W7Z8itMWyqrnVfbheVYrZ58=
go.opentelemetry.io/otel/sdk v1.44.0/go.mod h1:Osuydd3Se74nqjAKxid74N5eC+jfEqfTegHRnq58oK0=
go.opentelemetry.io/otel/sdk/metric v1.44.0 h1:3LlKgI+VjbVsjNRFZJZAJ30WjXC5VkNRks6si09iEfI=
go.opentelemetry.io/otel/sdk/metric v1.44.0/go.mod h1:5B5pMARnXxKhltooO4xUuCBorl65a4EpnTalObqOigA=
go.opentelemetry.io/otel/trace v1.44.0 h1:jxF5CsGYCe74MCRx2X4g7WsY/VBKRqqpNvXlX/6gtIk=
go.opentelemetry.io/otel/trace v1.44.0/go.mod h1:oLl1jrMQAVo6v3GAggN+1VH9VIz9iUSvW53sW1Q8PIE=
go.uber.org/goleak v1.3.0 h1:2K3zAYmnTNqV73imy9J1T3WC+gmCePx2hEGkimedGto=
@@ -1222,8 +1328,8 @@ golang.org/x/crypto v0.19.0/go.mod h1:Iy9bg/ha4yyC70EfRS8jz+B6ybOBKMaSxLj6P6oBDf
golang.org/x/crypto v0.23.0/go.mod h1:CKFgDieR+mRhux2Lsu27y0fO304Db0wZe70UKqHu0v8=
golang.org/x/crypto v0.25.0/go.mod h1:T+wALwcMOSE0kXgUAnPAHqTLW+XHgcELELW8VaDgm/M=
golang.org/x/crypto v0.31.0/go.mod h1:kDsLvtWBEx7MV9tJOj9bnXsPbxwJQ6csT/x4KIN4Ssk=
golang.org/x/crypto v0.54.0 h1:YLIA59K4fiNzHzjnZt2tUJQjQtUWfWbeHBqKtk3eScw=
golang.org/x/crypto v0.54.0/go.mod h1:KWL8ny2AZdGR2cWmzeHrp2azQPGogOv+HeQaVEXC2dk=
golang.org/x/crypto v0.55.0 h1:+KWHjbgOaAQ66dh/YlkZKHlz9ZUlq61AFirAR9ntP8M=
golang.org/x/crypto v0.55.0/go.mod h1:uq0V9dE/fzQuJtbnL+2EhWOE63vo164FY8xqEnV9xis=
golang.org/x/exp v0.0.0-20190121172915-509febef88a4/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
golang.org/x/exp v0.0.0-20190306152737-a1d7652674e8/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
golang.org/x/exp v0.0.0-20190510132918-efd6b22b2522/go.mod h1:ZjyILWgesfNpC6sMxTJOJm9Kp84zZh5NQWvqDGG3Qr8=
@@ -1261,8 +1367,8 @@ golang.org/x/mod v0.8.0/go.mod h1:iBbtSCu2XBx23ZKBPSOrRkjjQPZFPuis4dIYUhu/chs=
golang.org/x/mod v0.12.0/go.mod h1:iBbtSCu2XBx23ZKBPSOrRkjjQPZFPuis4dIYUhu/chs=
golang.org/x/mod v0.15.0/go.mod h1:hTbmBsO62+eylJbnUtE2MGJUyE7QWk4xUqPFrRgJ+7c=
golang.org/x/mod v0.17.0/go.mod h1:hTbmBsO62+eylJbnUtE2MGJUyE7QWk4xUqPFrRgJ+7c=
golang.org/x/mod v0.37.0 h1:vF1DjpVEshcIqoEaauuHebaLk1O1forxjxBaVn884JQ=
golang.org/x/mod v0.37.0/go.mod h1:m8S8VeM9r4dzDwjrKO0a1sZP3YjeMamRRlD+fmR2Q/0=
golang.org/x/mod v0.39.0 h1:UF5zwQdCRRUpHfyPwr7d4UrGiVeldIsogtzWVnczL74=
golang.org/x/mod v0.39.0/go.mod h1:bvIbwjQ0HUFFf5AKukeeYQG4ZBUG9yxQbR9aEweIwYY=
golang.org/x/net v0.0.0-20180724234803-3673e40ba225/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20180826012351-8a410e7b638d/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20180906233101-161cd47e91fd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
@@ -1312,8 +1418,8 @@ golang.org/x/net v0.21.0/go.mod h1:bIjVDfnllIU7BJ2DNgfnXvpSvtn8VRwhlsaeUTyUS44=
golang.org/x/net v0.25.0/go.mod h1:JkAGAh7GEvH74S6FOH42FLoXpXbE/aqXSrIQjXgsiwM=
golang.org/x/net v0.27.0/go.mod h1:dDi0PyhWNoiUOrAS8uXv/vnScO4wnHQO4mj9fn/RytE=
golang.org/x/net v0.33.0/go.mod h1:HXLR5J+9DxmrqMwG9qjGCxZ+zKXxBru04zlTvWlWuN4=
golang.org/x/net v0.57.0 h1:K5+3DljvIuDG9/Jv9rvyMywYNFCQ9RSUY6OOTTkT+tE=
golang.org/x/net v0.57.0/go.mod h1:KpXc8iv+r3XplLAG/f7Jsf9RPszJzdR0f58q9vGOuEU=
golang.org/x/net v0.58.0 h1:ynWG7rqYi4ccpTEuPZ2QGWHktVEM9DMCj9yzDE0Q7To=
golang.org/x/net v0.58.0/go.mod h1:YwCddHnFlT7eLQqVprV19OnhLGtc5xOKgE0RyqgfWAU=
golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
golang.org/x/oauth2 v0.0.0-20190226205417-e64efc72b421/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
golang.org/x/oauth2 v0.0.0-20190604053449-0f29369cfe45/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
@@ -1443,13 +1549,13 @@ golang.org/x/text v0.14.0/go.mod h1:18ZOQIKpY8NJVqYksKHtTdi31H5itFRjB5/qKTNYzSU=
golang.org/x/text v0.15.0/go.mod h1:18ZOQIKpY8NJVqYksKHtTdi31H5itFRjB5/qKTNYzSU=
golang.org/x/text v0.16.0/go.mod h1:GhwF1Be+LQoKShO3cGOHzqOgRrGaYc9AvblQOmPVHnI=
golang.org/x/text v0.21.0/go.mod h1:4IBbMaMmOPCJ8SecivzSH54+73PCFmPWxNTLm+vZkEQ=
golang.org/x/text v0.40.0 h1:Ub2Z6/xjgF1WrYQz2nuITOEegKFtiIy+rieRJ5lHZKs=
golang.org/x/text v0.40.0/go.mod h1:hpnzDAfGV753zIKo+wk3u1bVKCGPbrnF7+7LBF/UHVY=
golang.org/x/text v0.41.0 h1:vz/seA0lnX87Othu2f/0L24RcgrXD9/YFTSuGjj3rH8=
golang.org/x/text v0.41.0/go.mod h1:jvf1O8ajNzZqhSrQBPbutR/EB83Cc0CFrezNQIwbb5M=
golang.org/x/time v0.0.0-20181108054448-85acf8d2951c/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/time v0.0.0-20190308202827-9d24e82272b4/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/time v0.0.0-20191024005414-555d28b269f0/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/time v0.14.0 h1:MRx4UaLrDotUKUdCIqzPC48t1Y9hANFKIRpNx+Te8PI=
golang.org/x/time v0.14.0/go.mod h1:eL/Oa2bBBK0TkX57Fyni+NgnyQQN4LitPmob2Hjnqw4=
golang.org/x/time v0.15.0 h1:bbrp8t3bGUeFOx08pvsMYRTCVSMk89u4tKbNOZbp88U=
golang.org/x/time v0.15.0/go.mod h1:Y4YMaQmXwGQZoFaVFk4YpCt4FLQMYKZe9oeV/f4MSno=
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20190114222345-bf090417da8b/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20190226205152-f727befe758c/go.mod h1:9Yl7xja0Znq3iFh3HoIrodX9oNMXvdceNzlUR8zjMvY=
@@ -1496,8 +1602,8 @@ golang.org/x/tools v0.1.12/go.mod h1:hNGJHUnrk76NpqgfD5Aqm5Crs+Hm0VOH/i9J2+nxYbc
golang.org/x/tools v0.6.0/go.mod h1:Xwgl3UAJ/d3gWutnCtw505GrjyAbvKui8lOU390QaIU=
golang.org/x/tools v0.13.0/go.mod h1:HvlwmtVNQAhOuCjW7xxvovg8wbNq7LwfXh/k7wXUl58=
golang.org/x/tools v0.21.1-0.20240508182429-e35e4ccd0d2d/go.mod h1:aiJjzUbINMkxbQROHiO6hDPo2LHcIPhhQsa9DLh0yGk=
golang.org/x/tools v0.47.0 h1:7Kn5x/d1svx/PzryTsqeoZN4TZwqeH5pGWjefhLi/1Q=
golang.org/x/tools v0.47.0/go.mod h1:dFHnyTvFWY212G+h7ZY4Vsp/K3U4/7W9TyVaAul8uCA=
golang.org/x/tools v0.49.0 h1:3NI7VXzL9+1WZD52Dx2ttoPwD5DWrFGpl9mFZDlmisI=
golang.org/x/tools v0.49.0/go.mod h1:SJNXV9DBKT0UbdttsQjbfJlAE/q+y36++zo3uL3N0Oo=
golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
@@ -1520,6 +1626,8 @@ google.golang.org/api v0.24.0/go.mod h1:lIXQywCXRcnZPGlsd8NbLnOjtAoL6em04bJ9+z0M
google.golang.org/api v0.28.0/go.mod h1:lIXQywCXRcnZPGlsd8NbLnOjtAoL6em04bJ9+z0MncE=
google.golang.org/api v0.29.0/go.mod h1:Lcubydp8VUV7KeIHD9z2Bys/sm/vGKnG1UHuDBSrHWM=
google.golang.org/api v0.30.0/go.mod h1:QGmEvQ87FHZNiUVJkT14jQNYJ4ZJjdRF23ZXz5138Fc=
google.golang.org/api v0.264.0 h1:+Fo3DQXBK8gLdf8rFZ3uLu39JpOnhvzJrLMQSoSYZJM=
google.golang.org/api v0.264.0/go.mod h1:fAU1xtNNisHgOF5JooAs8rRaTkl2rT3uaoNGo9NS3R8=
google.golang.org/appengine v1.1.0/go.mod h1:EbEs0AVv82hx2wNQdGPgUI5lhzA/G0D9YwlJXL52JkM=
google.golang.org/appengine v1.4.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4=
google.golang.org/appengine v1.5.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4=
@@ -1556,8 +1664,12 @@ google.golang.org/genproto v0.0.0-20200618031413-b414f8b61790/go.mod h1:jDfRM7Fc
google.golang.org/genproto v0.0.0-20200729003335-053ba62fc06f/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
google.golang.org/genproto v0.0.0-20200804131852-c06518451d9c/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
google.golang.org/genproto v0.0.0-20200825200019-8632dd797987/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
google.golang.org/genproto/googleapis/rpc v0.0.0-20260414002931-afd174a4e478 h1:RmoJA1ujG+/lRGNfUnOMfhCy5EipVMyvUE+KNbPbTlw=
google.golang.org/genproto/googleapis/rpc v0.0.0-20260414002931-afd174a4e478/go.mod h1:4Hqkh8ycfw05ld/3BWL7rJOSfebL2Q+DVDeRgYgxUU8=
google.golang.org/genproto v0.0.0-20260128011058-8636f8732409 h1:VQZ/yAbAtjkHgH80teYd2em3xtIkkHd7ZhqfH2N9CsM=
google.golang.org/genproto v0.0.0-20260128011058-8636f8732409/go.mod h1:rxKD3IEILWEu3P44seeNOAwZN4SaoKaQ/2eTg4mM6EM=
google.golang.org/genproto/googleapis/api v0.0.0-20260526163538-3dc84a4a5aaa h1:Kjn0N0tCrDgiAFW+lGO4JZ3ck44CehvJQMAwj9QF0G8=
google.golang.org/genproto/googleapis/api v0.0.0-20260526163538-3dc84a4a5aaa/go.mod h1:q4lMZS6kskjT5HvCPrnnypcDPVJqT/f4nfxmkE7gryY=
google.golang.org/genproto/googleapis/rpc v0.0.0-20260526163538-3dc84a4a5aaa h1:mZHHdPZl0dbGHCflZgAq/Q468DWVFcU2whhB2KAo8fk=
google.golang.org/genproto/googleapis/rpc v0.0.0-20260526163538-3dc84a4a5aaa/go.mod h1:4Hqkh8ycfw05ld/3BWL7rJOSfebL2Q+DVDeRgYgxUU8=
google.golang.org/grpc v1.19.0/go.mod h1:mqu4LbDTu4XGKhr4mRzUsmM4RtVoemTSY81AxZiDr8c=
google.golang.org/grpc v1.20.1/go.mod h1:10oTOabMzJvdu6/UiuZezV6QK5dSlG84ov/aaiqXj38=
google.golang.org/grpc v1.21.1/go.mod h1:oYelfM1adQP15Ek0mdvEgi9Df8B9CZIaU1084ijfRaM=
@@ -1570,8 +1682,8 @@ google.golang.org/grpc v1.28.0/go.mod h1:rpkK4SK4GF4Ach/+MFLZUBavHOvF2JJB5uozKKa
google.golang.org/grpc v1.29.1/go.mod h1:itym6AZVZYACWQqET3MqgPpjcuV5QH3BxFS3IjizoKk=
google.golang.org/grpc v1.30.0/go.mod h1:N36X2cJ7JwdamYAgDz+s+rVMFjt3numwzf/HckM8pak=
google.golang.org/grpc v1.31.0/go.mod h1:N36X2cJ7JwdamYAgDz+s+rVMFjt3numwzf/HckM8pak=
google.golang.org/grpc v1.82.1 h1:NnAxzGRA0677vCa4BUkOAnO5+FfQqVl9iUXeD0IqcGE=
google.golang.org/grpc v1.82.1/go.mod h1:yzTZ1TB1Z3SG+LIYaI+WiE8D5+PZ3ArnrSp8zF3+/ZA=
google.golang.org/grpc v1.83.1 h1:HIO0+BEtBP6soyqvqC8sNUjZ7bTs+0hFQuFF+RAy++Y=
google.golang.org/grpc v1.83.1/go.mod h1:kDyl6SKsiHKt0uylY5gtn5cEjkrIOhQOGDgIc4JGwzQ=
google.golang.org/protobuf v0.0.0-20200109180630-ec00e32a8dfd/go.mod h1:DFci5gLYBciE7Vtevhsrf46CRTquxDuWsQurQQe4oz8=
google.golang.org/protobuf v0.0.0-20200221191635-4d8936d0db64/go.mod h1:kwYJMbMJ01Woi6D6+Kah6886xMZcty6N08ah7+eCXa0=
google.golang.org/protobuf v0.0.0-20200228230310-ab0ca4ff8a60/go.mod h1:cfTl7dwQJ+fmap5saPgwCLgHXTUD7jkjRqWcaiX5VyM=
@@ -1602,6 +1714,8 @@ gopkg.in/corvus-ch/zbase32.v1 v1.0.0/go.mod h1:T3oKkPOm4AV/bNXCNFUxRmlE9RUyBz/DS
gopkg.in/errgo.v2 v2.1.0/go.mod h1:hNsd1EY+bozCKY1Ytp96fpM3vjJbqLJn88ws8XvfDNI=
gopkg.in/fsnotify.v1 v1.4.7/go.mod h1:Tz8NjZHkW78fSQdbUxIjBTcgA1z1m8ZHf0WmKUhAMys=
gopkg.in/ini.v1 v1.42.0/go.mod h1:pNLf8WUiyNEtQjuu5G5vTm06TEv9tsIgeAvK8hOrP4k=
gopkg.in/ini.v1 v1.67.0 h1:Dgnx+6+nfE+IfzjUEISNeydPJh9AXNNsWbGP9KzCsOA=
gopkg.in/ini.v1 v1.67.0/go.mod h1:pNLf8WUiyNEtQjuu5G5vTm06TEv9tsIgeAvK8hOrP4k=
gopkg.in/mgo.v2 v2.0.0-20190816093944-a6b53ec6cb22/go.mod h1:yeKp02qBN3iKW1OzL3MGk2IdtZzaj7SFntXj72NppTA=
gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7 h1:uRGJdciOHaEIrze2W8Q3AKkepLTh2hOroT7a+7czfdQ=
gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7/go.mod h1:dt/ZhP58zS4L8KSrWDmTeBkI65Dw0HsyUHuEVlX15mw=
+73
View File
@@ -0,0 +1,73 @@
package runner
import (
"os"
"path/filepath"
"strconv"
"testing"
"github.com/projectdiscovery/gologger"
"github.com/projectdiscovery/gologger/formatter"
"github.com/projectdiscovery/gologger/levels"
"github.com/projectdiscovery/nuclei/v3/pkg/catalog/config"
"github.com/projectdiscovery/nuclei/v3/pkg/types"
"github.com/stretchr/testify/require"
)
type ignoreFileLogRecord struct {
message string
level levels.Level
}
type ignoreFileLogWriter struct {
records []ignoreFileLogRecord
}
func (w *ignoreFileLogWriter) Write(data []byte, level levels.Level) {
w.records = append(w.records, ignoreFileLogRecord{message: string(data), level: level})
}
func newIgnoreFileTestRunner(t *testing.T, root string, writer *ignoreFileLogWriter) *Runner {
t.Helper()
cfg := config.DefaultConfig
oldRoot := cfg.TemplatesDirectory
t.Cleanup(func() { cfg.SetTemplatesDir(oldRoot) })
cfg.SetTemplatesDir(root)
logger := &gologger.Logger{}
logger.SetFormatter(formatter.NewCLI(false))
logger.SetWriter(writer)
logger.SetMaxLevel(levels.LevelWarning)
options := types.DefaultOptions()
options.Logger = logger
return &Runner{options: options, Logger: logger}
}
func TestLoadIgnoreFileWarnsAndContinuesWhenActiveFileIsMissing(t *testing.T) {
root := t.TempDir()
writer := &ignoreFileLogWriter{}
runner := newIgnoreFileTestRunner(t, root, writer)
require.NoError(t, runner.loadIgnoreFile())
require.Empty(t, runner.options.ExcludeTags)
require.Empty(t, runner.options.ExcludedTemplates)
require.Len(t, writer.records, 1)
require.Equal(t, levels.LevelWarning, writer.records[0].level)
require.Contains(t, writer.records[0].message, strconv.Quote(filepath.Join(root, config.NucleiIgnoreFileName)))
require.Contains(t, writer.records[0].message, "continuing without ignore exclusions")
}
func TestLoadIgnoreFileRejectsCorruptActiveFile(t *testing.T) {
root := t.TempDir()
path := filepath.Join(root, config.NucleiIgnoreFileName)
require.NoError(t, os.WriteFile(path, []byte("tags: ["), 0o600))
writer := &ignoreFileLogWriter{}
runner := newIgnoreFileTestRunner(t, root, writer)
err := runner.loadIgnoreFile()
require.ErrorContains(t, err, "error parsing")
require.ErrorContains(t, err, strconv.Quote(path))
require.Empty(t, writer.records)
}
+47
View File
@@ -15,7 +15,9 @@ import (
"github.com/projectdiscovery/nuclei/v3/pkg/protocols/common/generators"
"github.com/projectdiscovery/nuclei/v3/pkg/protocols/common/helpers/writer"
"github.com/projectdiscovery/nuclei/v3/pkg/protocols/common/replacer"
httpprotocol "github.com/projectdiscovery/nuclei/v3/pkg/protocols/http"
"github.com/projectdiscovery/nuclei/v3/pkg/scan"
"github.com/projectdiscovery/nuclei/v3/pkg/templates"
"github.com/projectdiscovery/nuclei/v3/pkg/types"
"github.com/projectdiscovery/utils/env"
"github.com/projectdiscovery/utils/errkit"
@@ -79,6 +81,8 @@ func GetLazyAuthFetchCallback(opts *AuthLazyFetchOptions) authx.LazyFetchSecret
}
data := map[string]interface{}{}
tmpl := tmpls[0]
restore := skipSecretFileOnAuthTemplate(tmpl)
defer restore()
// add args to tmpl here
vars := map[string]interface{}{}
mainCtx := context.Background()
@@ -149,3 +153,46 @@ func GetLazyAuthFetchCallback(opts *AuthLazyFetchOptions) authx.LazyFetchSecret
return finalErr
}
}
// skipSecretFileOnAuthTemplate disables secret-file auth on the login template for
// the duration of the fetch. Login requests that hit the same domain as the
// secret would otherwise ApplyAuth → Fetch and deadlock on sync.Once (#7639).
// Previous SkipSecretFile values are restored so a cached template is unchanged.
func skipSecretFileOnAuthTemplate(tmpl *templates.Template) func() {
if tmpl == nil {
return func() {}
}
var reqs []*httpprotocol.Request
seen := make(map[*httpprotocol.Request]struct{})
add := func(r *httpprotocol.Request) {
if r == nil {
return
}
if _, ok := seen[r]; ok {
return
}
seen[r] = struct{}{}
reqs = append(reqs, r)
}
for _, r := range tmpl.RequestsHTTP {
add(r)
}
for _, r := range tmpl.RequestsWithHTTP {
add(r)
}
for _, req := range tmpl.RequestsQueue {
if r, ok := req.(*httpprotocol.Request); ok {
add(r)
}
}
prev := make([]bool, len(reqs))
for i, r := range reqs {
prev[i] = r.SkipSecretFile
r.SkipSecretFile = true
}
return func() {
for i, r := range reqs {
r.SkipSecretFile = prev[i]
}
}
}
+27
View File
@@ -6,6 +6,9 @@ import (
"github.com/projectdiscovery/nuclei/v3/pkg/operators"
"github.com/projectdiscovery/nuclei/v3/pkg/output"
"github.com/projectdiscovery/nuclei/v3/pkg/protocols"
httpprotocol "github.com/projectdiscovery/nuclei/v3/pkg/protocols/http"
"github.com/projectdiscovery/nuclei/v3/pkg/templates"
"github.com/stretchr/testify/require"
)
@@ -137,3 +140,27 @@ func TestCallbackDynamicValues(t *testing.T) {
require.NoError(t, err)
require.Equal(t, "sess-xyz-123", data["session"], "should pick the longest value")
}
func TestSkipSecretFileOnAuthTemplate(t *testing.T) {
t.Run("nil template is a no-op", func(t *testing.T) {
restore := skipSecretFileOnAuthTemplate(nil)
require.NotPanics(t, restore)
})
t.Run("sets skip during fetch and restores previous values", func(t *testing.T) {
alreadySkipped := &httpprotocol.Request{SkipSecretFile: true}
needsSkip := &httpprotocol.Request{}
tmpl := &templates.Template{
RequestsHTTP: []*httpprotocol.Request{alreadySkipped, needsSkip},
}
tmpl.RequestsQueue = []protocols.Request{alreadySkipped, needsSkip}
restore := skipSecretFileOnAuthTemplate(tmpl)
require.True(t, alreadySkipped.SkipSecretFile)
require.True(t, needsSkip.SkipSecretFile)
restore()
require.True(t, alreadySkipped.SkipSecretFile, "templates that already skipped secret-file must stay skipped")
require.False(t, needsSkip.SkipSecretFile, "scan-time ApplyAuth must remain enabled after login fetch")
})
}
+6
View File
@@ -18,6 +18,7 @@ import (
"github.com/projectdiscovery/gologger"
"github.com/projectdiscovery/gologger/formatter"
"github.com/projectdiscovery/gologger/levels"
"github.com/projectdiscovery/nuclei/v3/internal/server/proxy"
"github.com/projectdiscovery/nuclei/v3/pkg/catalog/config"
"github.com/projectdiscovery/nuclei/v3/pkg/protocols/common/protocolinit"
"github.com/projectdiscovery/nuclei/v3/pkg/protocols/common/utils/vardump"
@@ -296,6 +297,11 @@ func validateDASTOptions(options *types.Options) error {
if len(options.DASTServerToken) > 0 && len(options.DASTServerToken) < 16 {
return fmt.Errorf("DAST server token must be at least 16 characters long")
}
if options.DASTProxy {
if _, _, err := proxy.ParseAuth(options.DASTProxyAuth); err != nil {
return err
}
}
return nil
}
+47 -9
View File
@@ -14,6 +14,7 @@ import (
"github.com/projectdiscovery/gologger"
"github.com/projectdiscovery/nuclei/v3/internal/pdcp"
"github.com/projectdiscovery/nuclei/v3/internal/server"
"github.com/projectdiscovery/nuclei/v3/internal/server/proxy"
"github.com/projectdiscovery/nuclei/v3/pkg/authprovider"
"github.com/projectdiscovery/nuclei/v3/pkg/fuzz/frequency"
"github.com/projectdiscovery/nuclei/v3/pkg/input/provider"
@@ -359,14 +360,14 @@ func New(options *types.Options) (*Runner, error) {
runner.resumeCfg = resumeCfg
if options.DASTReport || options.DASTServer {
if options.DASTReport || options.DASTServer || options.DASTProxy {
var err error
runner.fuzzStats, err = fuzzStats.NewTracker()
if err != nil {
return nil, errors.Wrap(err, "could not create fuzz stats db")
}
if !options.DASTServer {
if !options.DASTServer && !options.DASTProxy {
dastServer, err := server.NewStatsServer(runner.fuzzStats)
if err != nil {
return nil, errors.Wrap(err, "could not create dast server")
@@ -577,7 +578,7 @@ func (r *Runner) RunEnumeration() error {
// If the user has asked for DAST server mode, run the live
// DAST fuzzing server.
if r.options.DASTServer {
if r.options.DASTServer || r.options.DASTProxy {
execurOpts := &server.NucleiExecutorOptions{
Options: r.options,
Output: r.output,
@@ -594,7 +595,7 @@ func (r *Runner) RunEnumeration() error {
FuzzStatsDB: r.fuzzStats,
Logger: r.Logger,
}
dastServer, err := server.New(&server.Options{
serverOptions := &server.Options{
Address: r.options.DASTServerAddress,
Templates: r.options.Templates,
OutputWriter: r.output,
@@ -603,11 +604,28 @@ func (r *Runner) RunEnumeration() error {
InScope: r.options.Scope,
OutScope: r.options.OutOfScope,
NucleiExecutorOptions: execurOpts,
})
EnableFuzzAPI: r.options.DASTServer,
ForwardProxy: r.options.AliveHttpProxy,
}
if r.options.DASTProxy {
username, password, err := proxy.ParseAuth(r.options.DASTProxyAuth)
if err != nil {
return err
}
serverOptions.ProxyAddress = r.options.DASTProxyAddress
serverOptions.ProxyCADir = config.DefaultConfig.GetConfigDir()
serverOptions.ProxyUsername = username
serverOptions.ProxyPassword = password
}
dastServer, err := server.New(serverOptions)
if err != nil {
return err
}
r.dastServer = dastServer
return dastServer.Start()
}
@@ -617,16 +635,18 @@ func (r *Runner) RunEnumeration() error {
r.options.Templates = append(r.options.Templates, arr...)
}
}
if len(r.options.NewTemplatesWithVersion) > 0 {
if arr := installer.GetNewTemplatesInVersions(r.options.NewTemplatesWithVersion...); len(arr) > 0 {
r.options.Templates = append(r.options.Templates, arr...)
}
}
// Exclude ignored file for validation
// Apply the active ignore policy only when templates can execute.
if !r.options.Validate {
ignoreFile := config.ReadIgnoreFile()
r.options.ExcludeTags = append(r.options.ExcludeTags, ignoreFile.Tags...)
r.options.ExcludedTemplates = append(r.options.ExcludedTemplates, ignoreFile.Files...)
if err := r.loadIgnoreFile(); err != nil {
return err
}
}
fuzzFreqCache := frequency.New(frequency.DefaultMaxTrackCount, r.options.FuzzParamFrequency)
@@ -886,6 +906,24 @@ func (r *Runner) RunEnumeration() error {
return err
}
func (r *Runner) loadIgnoreFile() error {
ignoreFile, err := config.ReadIgnoreFile()
if errors.Is(err, os.ErrNotExist) {
r.Logger.Warning().Msgf("Could not read active .nuclei-ignore file: %s; continuing without ignore exclusions", err)
return nil
}
if err != nil {
return err
}
r.options.ExcludeTags = append(r.options.ExcludeTags, ignoreFile.Tags...)
r.options.ExcludedTemplates = append(r.options.ExcludedTemplates, ignoreFile.Files...)
return nil
}
func shortDur(d time.Duration) string {
if d < time.Minute {
return d.String()
+1 -1
View File
@@ -117,7 +117,7 @@ func newNucleiExecutor(opts *NucleiExecutorOptions) (*nucleiExecutor, error) {
// If using input-file flags, only load http fuzzing based templates.
loaderConfig := loader.NewConfig(opts.Options, opts.Catalog, executorOpts)
if (!strings.EqualFold(opts.Options.InputFileMode, "list") || opts.Options.DAST || opts.Options.DASTServer) && !opts.Options.OfflineHTTP {
if (!strings.EqualFold(opts.Options.InputFileMode, "list") || opts.Options.DAST || opts.Options.DASTServer || opts.Options.DASTProxy) && !opts.Options.OfflineHTTP {
// if input type is not list (implicitly enable fuzzing), unless passive/offlinehttp
opts.Options.DAST = true
}
+194
View File
@@ -0,0 +1,194 @@
package proxy
import (
"crypto/ecdsa"
"crypto/elliptic"
"crypto/rand"
"crypto/tls"
"crypto/x509"
"crypto/x509/pkix"
"encoding/pem"
"math/big"
"os"
"path/filepath"
"runtime"
"time"
"github.com/pkg/errors"
fileutil "github.com/projectdiscovery/utils/file"
)
const (
caCertFileName = "dast-proxy-ca.pem"
caKeyFileName = "dast-proxy-ca.key"
caValidity = 2 * 365 * 24 * time.Hour
// The key can mint a certificate that the user's browser trusts for any
// host, so it must never be readable by other accounts on the machine.
caKeyFileMode os.FileMode = 0o600
caCertFileMode os.FileMode = 0o644
)
// CA is the certificate authority used to sign intercepted TLS connections.
type CA struct {
Certificate *tls.Certificate
CertPath string
CertPEM []byte
}
// LoadOrCreateCA loads the interception CA from dir, generating and persisting
// a new one on first use.
//
// goproxy ships a built-in CA whose private key is published with the library;
// it is deliberately never used here. A per-installation key is the only thing
// stopping a third party from minting certificates the user already trusts.
func LoadOrCreateCA(dir string) (*CA, error) {
certPath := filepath.Join(dir, caCertFileName)
keyPath := filepath.Join(dir, caKeyFileName)
certExists, keyExists := fileutil.FileExists(certPath), fileutil.FileExists(keyPath)
if certExists && keyExists {
return loadCA(certPath, keyPath)
}
// Half a keypair is unusable, and regenerating over it would invalidate a
// certificate the user may already have installed and trusted. Say which
// file is orphaned instead of guessing.
if certExists != keyExists {
orphan := certPath
if keyExists {
orphan = keyPath
}
return nil, errors.Errorf("dast proxy CA is incomplete, remove %s to generate a new keypair", orphan)
}
return createCA(dir, certPath, keyPath)
}
func loadCA(certPath, keyPath string) (*CA, error) {
if err := verifyKeyPermissions(keyPath); err != nil {
return nil, err
}
certPEM, err := os.ReadFile(certPath)
if err != nil {
return nil, errors.Wrap(err, "could not read dast proxy CA certificate")
}
keyPEM, err := os.ReadFile(keyPath)
if err != nil {
return nil, errors.Wrap(err, "could not read dast proxy CA key")
}
certificate, err := tls.X509KeyPair(certPEM, keyPEM)
if err != nil {
return nil, errors.Wrap(err, "could not parse dast proxy CA keypair")
}
leaf, err := x509.ParseCertificate(certificate.Certificate[0])
if err != nil {
return nil, errors.Wrap(err, "could not parse dast proxy CA certificate")
}
if !leaf.IsCA {
return nil, errors.Errorf("%s is not a certificate authority", certPath)
}
// Regenerating silently would swap the CA out from under a certificate the
// user has already installed and trusted, so this is left to the user.
if time.Now().After(leaf.NotAfter) {
return nil, errors.Errorf("dast proxy CA expired on %s, remove %s and %s to generate a new one",
leaf.NotAfter.Format(time.RFC3339), certPath, keyPath)
}
certificate.Leaf = leaf
return &CA{Certificate: &certificate, CertPath: certPath, CertPEM: certPEM}, nil
}
func createCA(dir, certPath, keyPath string) (*CA, error) {
if err := os.MkdirAll(dir, 0o755); err != nil {
return nil, errors.Wrap(err, "could not create dast proxy CA directory")
}
privateKey, err := ecdsa.GenerateKey(elliptic.P256(), rand.Reader)
if err != nil {
return nil, errors.Wrap(err, "could not generate dast proxy CA key")
}
serialNumber, err := rand.Int(rand.Reader, new(big.Int).Lsh(big.NewInt(1), 128))
if err != nil {
return nil, errors.Wrap(err, "could not generate dast proxy CA serial number")
}
now := time.Now()
template := &x509.Certificate{
SerialNumber: serialNumber,
Subject: pkix.Name{
CommonName: "Nuclei DAST Proxy CA",
Organization: []string{"ProjectDiscovery"},
},
NotBefore: now.Add(-1 * time.Hour), // tolerate clock skew on the client
NotAfter: now.Add(caValidity),
KeyUsage: x509.KeyUsageCertSign | x509.KeyUsageCRLSign | x509.KeyUsageDigitalSignature,
BasicConstraintsValid: true,
IsCA: true,
// Sign leaf certificates only: a CA reachable through this one must not
// be able to issue further authorities.
MaxPathLen: 0,
MaxPathLenZero: true,
}
der, err := x509.CreateCertificate(rand.Reader, template, template, &privateKey.PublicKey, privateKey)
if err != nil {
return nil, errors.Wrap(err, "could not create dast proxy CA certificate")
}
keyDER, err := x509.MarshalPKCS8PrivateKey(privateKey)
if err != nil {
return nil, errors.Wrap(err, "could not marshal dast proxy CA key")
}
certPEM := pem.EncodeToMemory(&pem.Block{Type: "CERTIFICATE", Bytes: der})
keyPEM := pem.EncodeToMemory(&pem.Block{Type: "PRIVATE KEY", Bytes: keyDER})
// O_EXCL so a pre-existing path, including a symlink planted by another
// user, can never be written through.
if err := writeNewFile(keyPath, keyPEM, caKeyFileMode); err != nil {
return nil, errors.Wrap(err, "could not write dast proxy CA key")
}
if err := writeNewFile(certPath, certPEM, caCertFileMode); err != nil {
// Leaving the key behind would wedge every later start on the
// incomplete-keypair check above.
_ = os.Remove(keyPath)
return nil, errors.Wrap(err, "could not write dast proxy CA certificate")
}
leaf, err := x509.ParseCertificate(der)
if err != nil {
return nil, errors.Wrap(err, "could not parse generated dast proxy CA certificate")
}
return &CA{
Certificate: &tls.Certificate{Certificate: [][]byte{der}, PrivateKey: privateKey, Leaf: leaf},
CertPath: certPath,
CertPEM: certPEM,
}, nil
}
func writeNewFile(path string, contents []byte, mode os.FileMode) error {
file, err := os.OpenFile(path, os.O_WRONLY|os.O_CREATE|os.O_EXCL, mode)
if err != nil {
return err
}
if _, err := file.Write(contents); err != nil {
_ = file.Close()
return err
}
return file.Close()
}
func verifyKeyPermissions(keyPath string) error {
if runtime.GOOS == "windows" {
return nil
}
info, err := os.Stat(keyPath)
if err != nil {
return errors.Wrap(err, "could not stat dast proxy CA key")
}
if mode := info.Mode().Perm(); mode&0o077 != 0 {
return errors.Errorf("dast proxy CA key %s is readable by other users (%#o), run: chmod 600 %s",
keyPath, mode, keyPath)
}
return nil
}
+90
View File
@@ -0,0 +1,90 @@
package proxy
import (
"os"
"path/filepath"
"runtime"
"testing"
"github.com/stretchr/testify/require"
)
func TestLoadOrCreateCAGeneratesUsableAuthority(t *testing.T) {
dir := t.TempDir()
ca, err := LoadOrCreateCA(dir)
require.NoError(t, err)
require.NotNil(t, ca.Certificate.Leaf)
require.True(t, ca.Certificate.Leaf.IsCA)
require.True(t, ca.Certificate.Leaf.MaxPathLenZero, "CA must not be able to issue sub authorities")
require.Contains(t, string(ca.CertPEM), "BEGIN CERTIFICATE")
require.NotContains(t, string(ca.CertPEM), "PRIVATE KEY")
require.Equal(t, filepath.Join(dir, caCertFileName), ca.CertPath)
}
func TestLoadOrCreateCAKeyIsOwnerReadableOnly(t *testing.T) {
if runtime.GOOS == "windows" {
t.Skip("unix file modes only")
}
dir := t.TempDir()
_, err := LoadOrCreateCA(dir)
require.NoError(t, err)
info, err := os.Stat(filepath.Join(dir, caKeyFileName))
require.NoError(t, err)
require.Equal(t, caKeyFileMode, info.Mode().Perm())
}
func TestLoadOrCreateCAReusesExistingAuthority(t *testing.T) {
dir := t.TempDir()
first, err := LoadOrCreateCA(dir)
require.NoError(t, err)
second, err := LoadOrCreateCA(dir)
require.NoError(t, err)
require.Equal(t, first.Certificate.Leaf.SerialNumber, second.Certificate.Leaf.SerialNumber)
}
func TestLoadOrCreateCARejectsGroupReadableKey(t *testing.T) {
if runtime.GOOS == "windows" {
t.Skip("unix file modes only")
}
dir := t.TempDir()
_, err := LoadOrCreateCA(dir)
require.NoError(t, err)
require.NoError(t, os.Chmod(filepath.Join(dir, caKeyFileName), 0o644))
_, err = LoadOrCreateCA(dir)
require.ErrorContains(t, err, "readable by other users")
}
func TestLoadOrCreateCARejectsIncompleteKeypair(t *testing.T) {
for _, orphan := range []string{caCertFileName, caKeyFileName} {
dir := t.TempDir()
require.NoError(t, os.WriteFile(filepath.Join(dir, orphan), []byte("x"), 0o600))
_, err := LoadOrCreateCA(dir)
require.ErrorContains(t, err, "incomplete", orphan)
require.ErrorContains(t, err, orphan)
}
}
// A failed certificate write must not leave the key behind, or the
// incomplete-keypair check would wedge every later start.
func TestLoadOrCreateCACleansUpKeyWhenCertificateWriteFails(t *testing.T) {
dir := t.TempDir()
// A directory at the certificate path makes the write fail.
require.NoError(t, os.Mkdir(filepath.Join(dir, caCertFileName), 0o755))
_, err := LoadOrCreateCA(dir)
require.Error(t, err)
require.NoFileExists(t, filepath.Join(dir, caKeyFileName), "orphaned key must be removed")
require.NoError(t, os.Remove(filepath.Join(dir, caCertFileName)))
ca, err := LoadOrCreateCA(dir)
require.NoError(t, err, "startup must recover once the blocker is gone")
require.True(t, ca.Certificate.Leaf.IsCA)
}
+452
View File
@@ -0,0 +1,452 @@
// Package proxy implements an intercepting HTTP proxy that feeds live traffic
// into the nuclei DAST scanner. Requests are forwarded untouched; a copy of
// each intercepted request is handed to the scanner out of band.
package proxy
import (
"bytes"
"crypto/subtle"
"crypto/tls"
"encoding/base64"
"io"
"net"
"net/http"
"net/http/httputil"
"net/url"
"strings"
"sync"
"sync/atomic"
"time"
"github.com/elazarl/goproxy"
"github.com/pkg/errors"
"github.com/projectdiscovery/gologger"
)
const (
// Requests with a larger body are not mirrored at all: see captureRequest.
maxCapturedBodySize = 4 << 20
readHeaderTimeout = 30 * time.Second
)
// hopByHopHeaders are stripped from mirrored requests: they belong to the
// client-to-proxy leg and must not be replayed at a target.
var hopByHopHeaders = []string{
"Connection",
"Keep-Alive",
"Proxy-Authorization",
"Proxy-Authenticate",
"Proxy-Connection",
"TE",
"Trailer",
"Transfer-Encoding",
"Upgrade",
}
// Options configures the intercepting proxy.
type Options struct {
// Address is the host:port to listen on.
Address string
// CADir is the directory holding the interception CA.
CADir string
// Username and Password gate the proxy with basic authentication. They are
// mandatory when Address is not a loopback address.
Username string
Password string
// Verbose enables goproxy's own request logging.
Verbose bool
// Intercept reports whether the TLS connection to host should be decrypted.
// Hosts it rejects are tunnelled through without inspection. Optional;
// every host is intercepted when nil.
Intercept func(host string) bool
// Submit hands a captured request to the scanner. It must not block, and
// returns false when the request could not be queued.
Submit func(rawHTTP, targetURL string) bool
// ForwardProxy is an optional HTTP(S) proxy URL for the forwarding leg.
// Ambient HTTP_PROXY / HTTPS_PROXY are ignored so a leftover environment
// variable cannot silently chain this listener.
ForwardProxy string
}
// Proxy is an intercepting proxy that mirrors traffic into the scanner.
type Proxy struct {
options *Options
ca *CA
server *http.Server
intercepted atomic.Int64
tunnelsPassedThrough atomic.Int64
dropped atomic.Int64
}
// requestState is what a proxied request carries between the goproxy hooks.
// A CONNECT tunnel reuses one instance for every request inside it.
type requestState struct {
tunnelAuthorized bool
rawHTTP string
}
// Stats reports what the proxy has done with the traffic it has seen.
type Stats struct {
Intercepted int64
// TunnelsPassedThrough counts CONNECT tunnels left encrypted, not requests:
// their contents are never seen.
TunnelsPassedThrough int64
Dropped int64
}
// New returns a proxy ready to be started.
func New(options *Options) (*Proxy, error) {
if options.Submit == nil {
return nil, errors.New("proxy requires a submit callback")
}
if err := validateBinding(options); err != nil {
return nil, err
}
if _, err := parseForwardProxy(options.ForwardProxy); err != nil {
return nil, err
}
ca, err := LoadOrCreateCA(options.CADir)
if err != nil {
return nil, err
}
proxy := &Proxy{options: options, ca: ca}
proxy.server = &http.Server{
Handler: proxy.handler(),
// Read and write timeouts would break long lived CONNECT tunnels, so
// only the header read is bounded.
ReadHeaderTimeout: readHeaderTimeout,
}
return proxy, nil
}
// CertPath is the on-disk path of the CA certificate users need to trust.
func (p *Proxy) CertPath() string { return p.ca.CertPath }
// CertPEM is the PEM encoded CA certificate. It never contains the key.
func (p *Proxy) CertPEM() []byte { return p.ca.CertPEM }
// Stats returns the current proxy counters.
func (p *Proxy) Stats() Stats {
return Stats{
Intercepted: p.intercepted.Load(),
TunnelsPassedThrough: p.tunnelsPassedThrough.Load(),
Dropped: p.dropped.Load(),
}
}
// Start listens and serves until Close is called.
func (p *Proxy) Start() error {
listener, err := net.Listen("tcp", p.options.Address)
if err != nil {
return errors.Wrap(err, "could not listen on dast proxy address")
}
if err := p.server.Serve(listener); err != nil && !errors.Is(err, http.ErrServerClosed) {
return err
}
return nil
}
// Close shuts the proxy listener down.
func (p *Proxy) Close() error {
if p.server == nil {
return nil
}
if err := p.server.Close(); err != nil && !errors.Is(err, http.ErrServerClosed) {
return err
}
return nil
}
func (p *Proxy) handler() http.Handler {
handler := goproxy.NewProxyHttpServer()
handler.Verbose = p.options.Verbose
handler.Logger = proxyLogger{}
handler.CertStore = newCertCache()
// Stated rather than inherited from goproxy's default, because it is a
// deliberate choice: the forwarding leg does not verify target certificates,
// matching the rest of nuclei (see httpclientpool), since DAST targets
// routinely serve self-signed ones. Interception inherently replaces the
// browser's own validation, so this is reported at startup.
forward, _ := parseForwardProxy(p.options.ForwardProxy)
handler.Tr = &http.Transport{
TLSClientConfig: &tls.Config{InsecureSkipVerify: true}, // #nosec G402
Proxy: forward,
}
// Every action carries our own CA. goproxy's package level OkConnect and
// MitmConnect close over its built-in CA, whose private key ships with the
// library, so none of them are reused here.
signWithOurCA := goproxy.TLSConfigFromCA(p.ca.Certificate)
mitm := &goproxy.ConnectAction{Action: goproxy.ConnectMitm, TLSConfig: signWithOurCA}
tunnel := &goproxy.ConnectAction{Action: goproxy.ConnectAccept, TLSConfig: signWithOurCA}
// Answering CONNECT with a challenge rather than dropping the connection is
// what lets a browser prompt for the proxy credentials.
challenge := &goproxy.ConnectAction{
Action: goproxy.ConnectHijack,
Hijack: func(req *http.Request, client net.Conn, _ *goproxy.ProxyCtx) {
defer func() { _ = client.Close() }()
if err := authRequiredResponse(req).Write(client); err != nil {
gologger.Debug().Msgf("Could not write proxy auth challenge: %s", err)
}
},
TLSConfig: signWithOurCA,
}
handler.OnRequest().HandleConnectFunc(func(host string, ctx *goproxy.ProxyCtx) (*goproxy.ConnectAction, string) {
if !p.authorized(ctx.Req) {
return challenge, host
}
// A client only sends Proxy-Authorization on CONNECT, so the successful
// check is recorded on the context that every request inside the tunnel
// goes on to share, rather than being repeated per request.
ctx.UserData = &requestState{tunnelAuthorized: true}
if p.options.Intercept != nil && !p.options.Intercept(host) {
p.tunnelsPassedThrough.Add(1)
return tunnel, host
}
return mitm, host
})
handler.OnRequest().DoFunc(func(req *http.Request, ctx *goproxy.ProxyCtx) (*http.Request, *http.Response) {
state, _ := ctx.UserData.(*requestState)
if state == nil || !state.tunnelAuthorized {
if !p.authorized(req) {
return req, authRequiredResponse(req)
}
state = &requestState{}
ctx.UserData = state
}
// The body has to be captured before the transport consumes it, but the
// response is only available later, so the raw request rides along on
// the context until then.
if raw, ok := captureRequest(req); ok {
state.rawHTTP = raw
}
return req, nil
})
handler.OnResponse().DoFunc(func(resp *http.Response, ctx *goproxy.ProxyCtx) *http.Response {
p.submit(ctx)
return resp
})
return handler
}
// submit mirrors an intercepted request into the scanner. It runs while the
// client is still waiting for its response, so it must stay non-blocking.
func (p *Proxy) submit(ctx *goproxy.ProxyCtx) {
if ctx == nil || ctx.Req == nil {
return
}
state, ok := ctx.UserData.(*requestState)
if !ok || state.rawHTTP == "" {
return
}
raw := state.rawHTTP
// Cleared so a tunnel that reuses this state cannot mirror it twice.
state.rawHTTP = ""
p.intercepted.Add(1)
if !p.options.Submit(raw, ctx.Req.URL.String()) {
p.dropped.Add(1)
}
}
func (p *Proxy) authorized(req *http.Request) bool {
if p.options.Username == "" && p.options.Password == "" {
return true
}
if req == nil {
return false
}
username, password, ok := parseProxyAuth(req.Header.Get("Proxy-Authorization"))
if !ok {
return false
}
// Both comparisons always run so the result does not leak which half failed.
userMatch := subtle.ConstantTimeCompare([]byte(username), []byte(p.options.Username))
passMatch := subtle.ConstantTimeCompare([]byte(password), []byte(p.options.Password))
return userMatch&passMatch == 1
}
func parseProxyAuth(header string) (username, password string, ok bool) {
const prefix = "Basic "
if len(header) < len(prefix) || !strings.EqualFold(header[:len(prefix)], prefix) {
return "", "", false
}
decoded, err := base64.StdEncoding.DecodeString(strings.TrimSpace(header[len(prefix):]))
if err != nil {
return "", "", false
}
username, password, found := strings.Cut(string(decoded), ":")
if !found {
return "", "", false
}
return username, password, true
}
func authRequiredResponse(req *http.Request) *http.Response {
resp := goproxy.NewResponse(req, "text/plain", http.StatusProxyAuthRequired, "proxy authentication required\n")
resp.Header.Set("Proxy-Authenticate", `Basic realm="nuclei dast proxy"`)
// goproxy leaves the protocol version unset, which writes a "HTTP/0.0"
// status line that strict clients reject when this is hijacked onto a raw
// connection to answer CONNECT.
resp.Proto, resp.ProtoMajor, resp.ProtoMinor = "HTTP/1.1", 1, 1
resp.Close = true
return resp
}
// captureRequest renders the request as raw HTTP for the scanner, leaving the
// request that gets forwarded untouched.
func captureRequest(req *http.Request) (string, bool) {
if req == nil || req.URL == nil {
return "", false
}
if req.URL.Scheme != "http" && req.URL.Scheme != "https" {
return "", false
}
// A body with no declared length, or one above the cap, is not mirrored at
// all: fuzzing a large upload is not useful, buffering it would let proxied
// traffic drive the scanner's memory use, and capturing only the headers
// would hand the scanner a request that contradicts its own Content-Length.
if hasBody(req) && (req.ContentLength < 0 || req.ContentLength > maxCapturedBodySize) {
return "", false
}
body, err := bufferBody(req)
if err != nil {
gologger.Debug().Msgf("Could not buffer proxied request body: %s", err)
return "", false
}
capture := req.Clone(req.Context())
capture.Body = io.NopCloser(bytes.NewReader(body))
capture.ContentLength = int64(len(body))
// The proxy's own absolute-form request URI is not what the target sees.
capture.RequestURI = ""
// goproxy only strips hop-by-hop headers after this hook has run, so the
// copy is sanitised here. The proxy credential in particular must never be
// replayed to a target or written into scan output.
stripHopByHopHeaders(capture.Header)
dumped, err := httputil.DumpRequest(capture, true)
if err != nil {
gologger.Debug().Msgf("Could not capture proxied request: %s", err)
return "", false
}
return string(dumped), true
}
func stripHopByHopHeaders(header http.Header) {
for _, extra := range header.Values("Connection") {
for _, name := range strings.Split(extra, ",") {
if name = strings.TrimSpace(name); name != "" {
header.Del(name)
}
}
}
for _, name := range hopByHopHeaders {
header.Del(name)
}
}
func parseForwardProxy(raw string) (func(*http.Request) (*url.URL, error), error) {
if raw == "" {
return nil, nil
}
parsed, err := url.Parse(raw)
if err != nil || parsed.Host == "" {
return nil, errors.Errorf("invalid dast proxy forward proxy %q", raw)
}
return http.ProxyURL(parsed), nil
}
func hasBody(req *http.Request) bool {
return req.Body != nil && req.Body != http.NoBody
}
// bufferBody reads the body and restores it on req so the request can still be
// forwarded after being mirrored.
func bufferBody(req *http.Request) ([]byte, error) {
if !hasBody(req) {
return nil, nil
}
body, err := io.ReadAll(io.LimitReader(req.Body, maxCapturedBodySize))
if err != nil {
return nil, err
}
req.Body = io.NopCloser(bytes.NewReader(body))
return body, nil
}
func validateBinding(options *Options) error {
host, _, err := net.SplitHostPort(options.Address)
if err != nil {
return errors.Wrapf(err, "invalid dast proxy address %q", options.Address)
}
if (options.Username == "") != (options.Password == "") {
return errors.New("dast proxy auth needs both a username and a password")
}
if options.Username != "" || isLoopback(host) {
return nil
}
// An unauthenticated proxy on a routable address is an open relay for
// anyone who can reach it.
return errors.Errorf("refusing to bind dast proxy to %q without authentication, pass -dast-proxy-auth user:pass or bind to localhost", options.Address)
}
func isLoopback(host string) bool {
if host == "" {
// A bare port binds every interface.
return false
}
if host == "localhost" {
return true
}
if ip := net.ParseIP(host); ip != nil {
return ip.IsLoopback()
}
return false
}
// certCache memoises the leaf certificate minted for each intercepted host.
type certCache struct {
certs sync.Map
}
func newCertCache() *certCache { return &certCache{} }
func (c *certCache) Fetch(hostname string, gen func() (*tls.Certificate, error)) (*tls.Certificate, error) {
if cached, ok := c.certs.Load(hostname); ok {
return cached.(*tls.Certificate), nil
}
cert, err := gen()
if err != nil {
return nil, err
}
actual, _ := c.certs.LoadOrStore(hostname, cert)
return actual.(*tls.Certificate), nil
}
type proxyLogger struct{}
func (proxyLogger) Printf(format string, v ...any) {
gologger.Debug().Msgf(format, v...)
}
// ParseAuth splits a user:pass credential pair as accepted on the command line.
func ParseAuth(value string) (username, password string, err error) {
if value == "" {
return "", "", nil
}
username, password, found := strings.Cut(value, ":")
if !found || username == "" || password == "" {
return "", "", errors.New("dast proxy auth must be in user:pass format")
}
return username, password, nil
}
+349
View File
@@ -0,0 +1,349 @@
package proxy
import (
"bufio"
"crypto/tls"
"crypto/x509"
"io"
"net"
"net/http"
"net/http/httptest"
"net/url"
"strings"
"testing"
"github.com/projectdiscovery/nuclei/v3/pkg/input/types"
"github.com/stretchr/testify/require"
)
type captured struct {
rawHTTP string
targetURL string
}
// newTestProxy returns a proxy served over a local listener along with the
// requests it mirrors into the scanner.
func newTestProxy(t *testing.T, options *Options) (*Proxy, *url.URL, <-chan captured) {
t.Helper()
mirrored := make(chan captured, 8)
options.CADir = t.TempDir()
options.Submit = func(rawHTTP, targetURL string) bool {
mirrored <- captured{rawHTTP: rawHTTP, targetURL: targetURL}
return true
}
if options.Address == "" {
options.Address = "127.0.0.1:0"
}
instance, err := New(options)
require.NoError(t, err)
frontend := httptest.NewServer(instance.server.Handler)
t.Cleanup(frontend.Close)
proxyURL, err := url.Parse(frontend.URL)
require.NoError(t, err)
return instance, proxyURL, mirrored
}
// skipInstance drops the proxy handle for tests that only drive it over HTTP.
func skipInstance(_ *Proxy, proxyURL *url.URL, mirrored <-chan captured) (*url.URL, <-chan captured) {
return proxyURL, mirrored
}
func clientThrough(proxyURL *url.URL) *http.Client {
return &http.Client{Transport: &http.Transport{Proxy: http.ProxyURL(proxyURL)}}
}
func TestProxyForwardsAndMirrorsRequest(t *testing.T) {
upstream := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
_, _ = io.WriteString(w, "upstream-body")
}))
defer upstream.Close()
proxyURL, mirrored := skipInstance(newTestProxy(t, &Options{}))
resp, err := clientThrough(proxyURL).Post(upstream.URL+"/login?next=/admin", "application/json", strings.NewReader(`{"user":"admin"}`))
require.NoError(t, err)
defer func() { _ = resp.Body.Close() }()
body, err := io.ReadAll(resp.Body)
require.NoError(t, err)
require.Equal(t, http.StatusOK, resp.StatusCode)
require.Equal(t, "upstream-body", string(body), "proxy must forward the upstream response untouched")
got := <-mirrored
require.Equal(t, upstream.URL+"/login?next=/admin", got.targetURL)
// The scanner consumes the mirrored traffic through this parser, so the
// capture is only useful if it round-trips.
parsed, err := types.ParseRawRequestWithURL(got.rawHTTP, got.targetURL)
require.NoError(t, err)
require.Equal(t, http.MethodPost, parsed.Request.Method)
require.Equal(t, `{"user":"admin"}`, parsed.Request.Body)
require.Equal(t, "/login", parsed.URL.Path)
}
func TestProxyRequiresAuthWhenConfigured(t *testing.T) {
upstream := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
_, _ = io.WriteString(w, "upstream-body")
}))
defer upstream.Close()
proxyURL, mirrored := skipInstance(newTestProxy(t, &Options{Username: "nuclei", Password: "s3cret"}))
resp, err := clientThrough(proxyURL).Get(upstream.URL + "/")
require.NoError(t, err)
_ = resp.Body.Close()
require.Equal(t, http.StatusProxyAuthRequired, resp.StatusCode)
require.Contains(t, resp.Header.Get("Proxy-Authenticate"), "Basic")
require.Empty(t, mirrored, "an unauthenticated request must not reach the scanner")
authenticated := proxyURL
authenticated.User = url.UserPassword("nuclei", "s3cret")
resp, err = clientThrough(authenticated).Get(upstream.URL + "/")
require.NoError(t, err)
_ = resp.Body.Close()
require.Equal(t, http.StatusOK, resp.StatusCode)
require.Equal(t, upstream.URL+"/", (<-mirrored).targetURL)
}
func TestProxyRejectsWrongCredentials(t *testing.T) {
upstream := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {}))
defer upstream.Close()
proxyURL, _ := skipInstance(newTestProxy(t, &Options{Username: "nuclei", Password: "s3cret"}))
proxyURL.User = url.UserPassword("nuclei", "wrong")
resp, err := clientThrough(proxyURL).Get(upstream.URL + "/")
require.NoError(t, err)
_ = resp.Body.Close()
require.Equal(t, http.StatusProxyAuthRequired, resp.StatusCode)
}
func TestNewRefusesUnauthenticatedNonLoopbackBind(t *testing.T) {
for _, address := range []string{"0.0.0.0:9056", "192.0.2.10:9056", ":9056"} {
_, err := New(&Options{
Address: address,
CADir: t.TempDir(),
Submit: func(string, string) bool { return true },
})
require.ErrorContains(t, err, "refusing to bind", address)
}
}
func TestNewAllowsLoopbackAndAuthenticatedBinds(t *testing.T) {
for _, options := range []*Options{
{Address: "127.0.0.1:9056"},
{Address: "localhost:9056"},
{Address: "[::1]:9056"},
{Address: "0.0.0.0:9056", Username: "nuclei", Password: "s3cret"},
} {
options.CADir = t.TempDir()
options.Submit = func(string, string) bool { return true }
_, err := New(options)
require.NoError(t, err, options.Address)
}
}
func TestNewRequiresBothAuthHalves(t *testing.T) {
_, err := New(&Options{
Address: "127.0.0.1:9056",
CADir: t.TempDir(),
Username: "nuclei",
Submit: func(string, string) bool { return true },
})
require.ErrorContains(t, err, "both a username and a password")
}
func TestParseAuth(t *testing.T) {
username, password, err := ParseAuth("nuclei:s3cret")
require.NoError(t, err)
require.Equal(t, "nuclei", username)
require.Equal(t, "s3cret", password)
username, password, err = ParseAuth("")
require.NoError(t, err)
require.Empty(t, username)
require.Empty(t, password)
for _, invalid := range []string{"nuclei", ":s3cret", "nuclei:"} {
_, _, err := ParseAuth(invalid)
require.ErrorContains(t, err, "user:pass", invalid)
}
}
func TestCaptureRequestSkipsUnboundedBody(t *testing.T) {
request, err := http.NewRequest(http.MethodPost, "http://example.com/upload", strings.NewReader("payload"))
require.NoError(t, err)
request.ContentLength = maxCapturedBodySize + 1
_, ok := captureRequest(request)
require.False(t, ok, "an oversized body must not be buffered")
request.ContentLength = int64(len("payload"))
raw, ok := captureRequest(request)
require.True(t, ok)
require.Contains(t, raw, "payload")
}
func TestCaptureRequestLeavesForwardedRequestIntact(t *testing.T) {
request, err := http.NewRequest(http.MethodPost, "http://example.com/login", strings.NewReader("user=admin"))
require.NoError(t, err)
raw, ok := captureRequest(request)
require.True(t, ok)
require.Contains(t, raw, "user=admin")
// The forwarded request still has to be readable after being mirrored.
forwarded, err := io.ReadAll(request.Body)
require.NoError(t, err)
require.Equal(t, "user=admin", string(forwarded))
}
func TestCaptureRequestDropsProxyCredentials(t *testing.T) {
request, err := http.NewRequest(http.MethodGet, "http://example.com/", nil)
require.NoError(t, err)
request.RequestURI = "http://example.com/" // absolute form, as a proxy receives it
request.Header.Set("Proxy-Authorization", "Basic bnVjbGVpOnMzY3JldA==")
request.Header.Set("Proxy-Connection", "keep-alive")
request.Header.Set("Authorization", "Bearer target-token")
raw, ok := captureRequest(request)
require.True(t, ok)
require.NotContains(t, raw, "Proxy-Authorization", "the proxy credential must never reach the scanner")
require.NotContains(t, raw, "bnVjbGVpOnMzY3JldA==")
require.NotContains(t, raw, "Proxy-Connection")
require.Contains(t, raw, "Bearer target-token", "the target's own credentials must be preserved")
require.True(t, strings.HasPrefix(raw, "GET / HTTP/1.1"), "request line must be origin form, got %q", raw)
}
func TestCaptureRequestDropsHopByHopHeaders(t *testing.T) {
request, err := http.NewRequest(http.MethodGet, "http://example.com/", nil)
require.NoError(t, err)
request.Header.Set("Connection", "close, X-Proxy-Debug")
request.Header.Set("X-Proxy-Debug", "1")
request.Header.Set("Keep-Alive", "timeout=5")
request.Header.Set("TE", "trailers")
request.Header.Set("X-Custom", "keep")
raw, ok := captureRequest(request)
require.True(t, ok)
require.NotContains(t, raw, "X-Proxy-Debug")
require.NotContains(t, raw, "Keep-Alive")
require.NotContains(t, raw, "TE:")
require.NotContains(t, raw, "Connection:")
require.Contains(t, raw, "X-Custom: keep")
}
func TestNewRejectsInvalidForwardProxy(t *testing.T) {
_, err := New(&Options{
Address: "127.0.0.1:9056",
CADir: t.TempDir(),
ForwardProxy: "://bad",
Submit: func(string, string) bool { return true },
})
require.ErrorContains(t, err, "forward proxy")
}
func TestCaptureRequestRejectsNonHTTPScheme(t *testing.T) {
request, err := http.NewRequest(http.MethodGet, "http://example.com/", nil)
require.NoError(t, err)
request.URL.Scheme = "ftp"
_, ok := captureRequest(request)
require.False(t, ok)
}
// A client only sends Proxy-Authorization on CONNECT, so requests inside an
// established tunnel must not be challenged again.
func TestProxyAuthorizesTunnelOnceOnConnect(t *testing.T) {
upstream := httptest.NewTLSServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
_, _ = io.WriteString(w, "upstream-body")
}))
defer upstream.Close()
instance, proxyURL, mirrored := newTestProxy(t, &Options{Username: "nuclei", Password: "s3cret"})
proxyURL.User = url.UserPassword("nuclei", "s3cret")
roots := x509.NewCertPool()
require.True(t, roots.AppendCertsFromPEM(instance.CertPEM()))
client := &http.Client{Transport: &http.Transport{
Proxy: http.ProxyURL(proxyURL),
TLSClientConfig: &tls.Config{RootCAs: roots, MinVersion: tls.VersionTLS12},
}}
resp, err := client.Get(upstream.URL + "/admin?id=1")
require.NoError(t, err)
defer func() { _ = resp.Body.Close() }()
body, err := io.ReadAll(resp.Body)
require.NoError(t, err)
require.Equal(t, http.StatusOK, resp.StatusCode, "a request inside an authorized tunnel must not be challenged")
require.Equal(t, "upstream-body", string(body))
got := <-mirrored
require.Equal(t, upstream.URL+"/admin?id=1", got.targetURL)
require.True(t, strings.HasPrefix(got.rawHTTP, "GET /admin?id=1 HTTP/1.1"), "got %q", got.rawHTTP)
}
func TestProxyChallengesUnauthenticatedConnect(t *testing.T) {
_, proxyURL, _ := newTestProxy(t, &Options{Username: "nuclei", Password: "s3cret"})
conn, err := net.Dial("tcp", proxyURL.Host)
require.NoError(t, err)
defer func() { _ = conn.Close() }()
_, err = io.WriteString(conn, "CONNECT example.com:443 HTTP/1.1\r\nHost: example.com:443\r\n\r\n")
require.NoError(t, err)
resp, err := http.ReadResponse(bufio.NewReader(conn), nil)
require.NoError(t, err)
defer func() { _ = resp.Body.Close() }()
// A malformed status line leaves a browser unable to prompt for credentials.
require.Equal(t, http.StatusProxyAuthRequired, resp.StatusCode)
require.Equal(t, 1, resp.ProtoMajor)
require.Equal(t, 1, resp.ProtoMinor)
require.Contains(t, resp.Header.Get("Proxy-Authenticate"), "Basic")
}
func TestProxyPassesThroughExcludedHostsWithoutMITM(t *testing.T) {
upstream := httptest.NewTLSServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
_, _ = io.WriteString(w, "upstream-body")
}))
defer upstream.Close()
upstreamURL, err := url.Parse(upstream.URL)
require.NoError(t, err)
instance, proxyURL, mirrored := newTestProxy(t, &Options{
Intercept: func(host string) bool {
return host != upstreamURL.Host
},
})
// Trust only the upstream certificate. An intercepted connection is
// signed by the nuclei CA and would fail this handshake.
upstreamRoots := x509.NewCertPool()
upstreamRoots.AddCert(upstream.Certificate())
client := &http.Client{Transport: &http.Transport{
Proxy: http.ProxyURL(proxyURL),
TLSClientConfig: &tls.Config{RootCAs: upstreamRoots, MinVersion: tls.VersionTLS12},
}}
resp, err := client.Get(upstream.URL + "/secret")
require.NoError(t, err)
defer func() { _ = resp.Body.Close() }()
body, err := io.ReadAll(resp.Body)
require.NoError(t, err)
require.Equal(t, http.StatusOK, resp.StatusCode)
require.Equal(t, "upstream-body", string(body))
require.True(t, resp.TLS.PeerCertificates[0].Equal(upstream.Certificate()),
"pass-through must present the target certificate, not a nuclei-minted one")
require.Equal(t, int64(1), instance.Stats().TunnelsPassedThrough)
require.Zero(t, instance.Stats().Intercepted)
require.Empty(t, mirrored)
}
+12
View File
@@ -75,3 +75,15 @@ func (m *Manager) validateURL(URL string) (bool, error) {
}
return inScopeMatched, nil
}
// IsExplicitlyOutOfScope reports whether the URL matches an out-of-scope rule.
// It exists for decisions that can only see a host, such as whether to decrypt
// a CONNECT tunnel, where in-scope rules carrying a path can never match.
func (m *Manager) IsExplicitlyOutOfScope(URL *url.URL) bool {
for _, item := range m.outOfScope {
if item.MatchString(URL.String()) {
return true
}
}
return false
}
+18
View File
@@ -1,6 +1,7 @@
package scope
import (
"net/url"
"testing"
urlutil "github.com/projectdiscovery/utils/url"
@@ -24,3 +25,20 @@ func TestManagerValidate(t *testing.T) {
})
}
func TestManagerIsExplicitlyOutOfScope(t *testing.T) {
manager, err := NewManager(
[]string{`^https://app\.example\.com/api/`},
[]string{`^https://telemetry\.example\.com/`},
)
require.NoError(t, err)
excluded, err := url.Parse("https://telemetry.example.com/")
require.NoError(t, err)
require.True(t, manager.IsExplicitlyOutOfScope(excluded))
// Not matching an in-scope rule is not the same as being excluded.
unlisted, err := url.Parse("https://app.example.com/")
require.NoError(t, err)
require.False(t, manager.IsExplicitlyOutOfScope(unlisted))
}
+148 -10
View File
@@ -3,8 +3,10 @@ package server
import (
_ "embed"
"encoding/json"
"errors"
"fmt"
"html/template"
"net"
"net/http"
"net/url"
"strings"
@@ -13,6 +15,7 @@ import (
"github.com/alitto/pond"
"github.com/projectdiscovery/gologger"
"github.com/projectdiscovery/nuclei/v3/internal/server/proxy"
"github.com/projectdiscovery/nuclei/v3/internal/server/scope"
"github.com/projectdiscovery/nuclei/v3/pkg/catalog/config"
"github.com/projectdiscovery/nuclei/v3/pkg/fuzz/stats"
@@ -29,6 +32,7 @@ type DASTServer struct {
tasksPool *pond.WorkerPool
deduplicator *requestDeduplicator
scopeManager *scope.Manager
proxy *proxy.Proxy
startTime time.Time
// metrics
@@ -49,6 +53,21 @@ type Options struct {
// Verbose is a flag that controls verbose output
Verbose bool
// ProxyAddress enables the intercepting proxy front-end on this address
ProxyAddress string
// ProxyCADir is the directory holding the proxy interception CA
ProxyCADir string
// ProxyUsername and ProxyPassword gate the intercepting proxy with basic
// authentication. Mandatory when ProxyAddress is not a loopback address.
ProxyUsername string
ProxyPassword string
// EnableFuzzAPI exposes POST /fuzz. Proxy-only mode leaves it off so the
// stats/CA listener cannot be used to inject arbitrary raw HTTP.
EnableFuzzAPI bool
// ForwardProxy is an optional HTTP(S) proxy URL for the intercepting
// proxy's forwarding leg. Ambient HTTP_PROXY is not used.
ForwardProxy string
// Scope fields for fuzzer
InScope []string
OutScope []string
@@ -97,17 +116,70 @@ func New(options *Options) (*DASTServer, error) {
}
server.scopeManager = scopeManager
if options.ProxyAddress != "" {
if err := server.setupProxy(); err != nil {
return nil, err
}
}
var builder strings.Builder
gologger.Debug().Msgf("Using %d parallel tasks with %d buffer", maxWorkers, bufferSize)
if options.Token != "" {
builder.WriteString(" (with token)")
}
gologger.Info().Msgf("DAST Server API: %s", server.buildURL("/fuzz"))
if options.EnableFuzzAPI {
gologger.Info().Msgf("DAST Server API: %s", server.buildURL("/fuzz"))
}
gologger.Info().Msgf("DAST Server Stats URL: %s", server.buildURL("/stats"))
return server, nil
}
// setupProxy wires the intercepting proxy front-end into the same queue the
// HTTP API feeds.
func (s *DASTServer) setupProxy() error {
interceptingProxy, err := proxy.New(&proxy.Options{
Address: s.options.ProxyAddress,
CADir: s.options.ProxyCADir,
Username: s.options.ProxyUsername,
Password: s.options.ProxyPassword,
Verbose: s.options.Verbose,
Intercept: s.shouldIntercept,
Submit: s.Submit,
ForwardProxy: s.options.ForwardProxy,
})
if err != nil {
return err
}
s.proxy = interceptingProxy
gologger.Info().Msgf("DAST Proxy: http://%s", s.options.ProxyAddress)
gologger.Info().Msgf("DAST Proxy CA certificate: %s (install it to intercept https)", interceptingProxy.CertPath())
gologger.Info().Msgf("DAST Proxy CA download: %s", s.buildURL("/ca"))
// Info, not Warning: gologger orders LevelWarning above LevelInfo, so
// warnings are filtered out at default verbosity and these two notices
// describe what interception does to the user's traffic.
gologger.Info().Msgf("DAST Proxy: intercepted https is re-originated by nuclei, target certificates are not verified")
if len(s.options.InScope) == 0 && len(s.options.OutScope) == 0 {
gologger.Info().Msgf("DAST Proxy: no scope set, every proxied https host will be decrypted (narrow it with -fuzz-scope / -fuzz-out-scope)")
}
return nil
}
// shouldIntercept decides whether a CONNECT tunnel is decrypted. Only an
// explicit out-of-scope match leaves a host encrypted, because in-scope
// patterns may carry a path that a CONNECT host can never match against.
func (s *DASTServer) shouldIntercept(hostPort string) bool {
host := hostPort
// Only the default port is dropped, so the synthesised URL has the same
// shape as the request URLs these rules are applied to once a request has
// actually been captured.
if parsedHost, port, err := net.SplitHostPort(hostPort); err == nil && port == "443" {
host = parsedHost
}
return !s.scopeManager.IsExplicitlyOutOfScope(&url.URL{Scheme: "https", Host: host, Path: "/"})
}
func NewStatsServer(fuzzStatsDB *stats.Tracker) (*DASTServer, error) {
server := &DASTServer{
options: &Options{},
@@ -124,6 +196,11 @@ func NewStatsServer(fuzzStatsDB *stats.Tracker) (*DASTServer, error) {
}
func (s *DASTServer) Close() {
if s.proxy != nil {
if err := s.proxy.Close(); err != nil {
gologger.Warning().Msgf("Could not close dast proxy: %s", err)
}
}
if s.nucleiExecutor != nil {
s.nucleiExecutor.Close()
}
@@ -160,16 +237,18 @@ func (s *DASTServer) optionsOrDefault() *Options {
}
func (s *DASTServer) setupHandlers(onlyStats bool) {
opts := s.optionsOrDefault()
mux := http.NewServeMux()
// POST /fuzz - Queue a request for fuzzing
if !onlyStats {
if !onlyStats && opts.EnableFuzzAPI {
mux.HandleFunc("POST /fuzz", s.handleRequest)
}
mux.HandleFunc("GET /stats", s.handleStats)
mux.HandleFunc("GET /stats.json", s.handleStatsJSON)
if !onlyStats && opts.ProxyAddress != "" {
mux.HandleFunc("GET /ca", s.handleProxyCA)
}
handler := http.Handler(mux)
opts := s.optionsOrDefault()
if opts.Token != "" {
handler = s.tokenAuthMiddleware(handler)
}
@@ -187,12 +266,37 @@ func (s *DASTServer) Start() error {
s.setupHandlers(false)
}
s.httpServer.Addr = s.optionsOrDefault().Address
if err := s.httpServer.ListenAndServe(); err != nil && err != http.ErrServerClosed {
if s.proxy == nil {
return s.serveAPI()
}
// Both front-ends feed the same queue; whichever fails first ends the run.
errs := make(chan error, 2)
go func() { errs <- s.serveAPI() }()
go func() { errs <- s.proxy.Start() }()
return <-errs
}
func (s *DASTServer) serveAPI() error {
if err := s.httpServer.ListenAndServe(); err != nil && !errors.Is(err, http.ErrServerClosed) {
return err
}
return nil
}
// handleProxyCA serves the interception CA certificate so it can be installed.
// Only the certificate is ever served, never the private key.
func (s *DASTServer) handleProxyCA(w http.ResponseWriter, _ *http.Request) {
if s.proxy == nil {
writeServerJSON(w, http.StatusNotFound, map[string]string{"error": "proxy is not enabled"})
return
}
w.Header().Set("Content-Type", "application/x-pem-file")
w.Header().Set("Content-Disposition", `attachment; filename="nuclei-dast-proxy-ca.pem"`)
_, _ = w.Write(s.proxy.CertPEM())
}
// PostRequestsHandlerRequest is the request body for the /fuzz POST handler.
type PostRequestsHandlerRequest struct {
RawHTTP string `json:"raw_http"`
@@ -214,13 +318,27 @@ func (s *DASTServer) handleRequest(w http.ResponseWriter, r *http.Request) {
return
}
s.endpointsInQueue.Add(1)
s.tasksPool.Submit(func() {
s.consumeTaskRequest(req)
})
if !s.Submit(req.RawHTTP, req.URL) {
writeServerJSON(w, http.StatusServiceUnavailable, map[string]string{"error": "scan queue is full"})
return
}
w.WriteHeader(http.StatusOK)
}
// Submit queues a captured request for fuzzing and reports whether it was
// accepted. It never blocks: a saturated scanner must not stall the live
// traffic flowing through the proxy, so excess requests are dropped instead.
func (s *DASTServer) Submit(rawHTTP, targetURL string) bool {
req := PostRequestsHandlerRequest{RawHTTP: rawHTTP, URL: targetURL}
s.endpointsInQueue.Add(1)
if !s.tasksPool.TrySubmit(func() { s.consumeTaskRequest(req) }) {
s.endpointsInQueue.Add(-1)
return false
}
return true
}
type StatsResponse struct {
DASTServerInfo DASTServerInfo `json:"dast_server_info"`
DASTScanStatistics DASTScanStatistics `json:"dast_scan_statistics"`
@@ -235,6 +353,8 @@ type DASTServerInfo struct {
NucleiTemplateVersion string `json:"nuclei_template_version"`
NucleiDastServerAPI string `json:"nuclei_dast_server_api"`
ServerAuthEnabled bool `json:"sever_auth_enabled"`
NucleiDastProxyAddr string `json:"nuclei_dast_proxy_address,omitempty"`
ProxyAuthEnabled bool `json:"proxy_auth_enabled"`
}
type DASTScanStatistics struct {
@@ -247,17 +367,29 @@ type DASTScanStatistics struct {
TotalEndpointsTested int64 `json:"total_endpoints_tested"`
TotalFuzzedRequests int64 `json:"total_fuzzed_requests"`
TotalErroredRequests int64 `json:"total_errored_requests"`
// Proxy counters, populated only in proxy mode. A rising drop count means
// the scanner cannot keep up with the proxied traffic.
ProxyRequestsIntercepted int64 `json:"proxy_requests_intercepted"`
ProxyTunnelsPassedThrough int64 `json:"proxy_tunnels_passed_through"`
ProxyRequestsDropped int64 `json:"proxy_requests_dropped"`
}
func (s *DASTServer) getStats() (StatsResponse, error) {
cfg := config.DefaultConfig
api := ""
if s.options.EnableFuzzAPI {
api = s.buildURL("/fuzz")
}
resp := StatsResponse{
DASTServerInfo: DASTServerInfo{
NucleiVersion: config.Version,
NucleiTemplateVersion: cfg.TemplateVersion,
NucleiDastServerAPI: s.buildURL("/fuzz"),
NucleiDastServerAPI: api,
ServerAuthEnabled: s.options.Token != "",
NucleiDastProxyAddr: s.options.ProxyAddress,
ProxyAuthEnabled: s.options.ProxyUsername != "",
},
DASTScanStartTime: s.startTime,
DASTScanStatistics: DASTScanStatistics{
@@ -266,6 +398,12 @@ func (s *DASTServer) getStats() (StatsResponse, error) {
TotalTemplatesLoaded: int64(len(s.nucleiExecutor.store.Templates())),
},
}
if s.proxy != nil {
proxyStats := s.proxy.Stats()
resp.DASTScanStatistics.ProxyRequestsIntercepted = proxyStats.Intercepted
resp.DASTScanStatistics.ProxyTunnelsPassedThrough = proxyStats.TunnelsPassedThrough
resp.DASTScanStatistics.ProxyRequestsDropped = proxyStats.Dropped
}
if s.nucleiExecutor.executorOpts.FuzzStatsDB != nil {
fuzzStats := s.nucleiExecutor.executorOpts.FuzzStatsDB.GetStats()
resp.DASTScanSeverityBreakdown = fuzzStats.SeverityCounts
+69
View File
@@ -3,8 +3,11 @@ package server
import (
"net/http"
"net/http/httptest"
"strings"
"testing"
"github.com/alitto/pond"
"github.com/projectdiscovery/nuclei/v3/internal/server/scope"
"github.com/stretchr/testify/require"
)
@@ -64,3 +67,69 @@ func TestStatsServerCloseDoesNotRequireFullExecutorOptions(t *testing.T) {
require.NotPanics(t, server.Close)
}
func TestShouldInterceptOnlySkipsExplicitlyExcludedHosts(t *testing.T) {
scopeManager, err := scope.NewManager(
// An in-scope rule carrying a path can never match a CONNECT host, so
// it must not be what decides whether the tunnel is decrypted.
[]string{`^https://app\.example\.com/api/`},
[]string{`^https://telemetry\.example\.com/`},
)
require.NoError(t, err)
server := &DASTServer{options: &Options{}, scopeManager: scopeManager}
require.True(t, server.shouldIntercept("app.example.com:443"))
require.True(t, server.shouldIntercept("unrelated.example.com:443"))
require.False(t, server.shouldIntercept("telemetry.example.com:443"))
// A non-default port stays in the URL, matching what the same rules see
// once a request off that connection is captured.
staging, err := scope.NewManager(nil, []string{`^https://staging\.example\.com:8443/`})
require.NoError(t, err)
server.scopeManager = staging
require.False(t, server.shouldIntercept("staging.example.com:8443"))
require.True(t, server.shouldIntercept("staging.example.com:443"))
}
func TestSubmitDropsInsteadOfBlockingWhenQueueIsFull(t *testing.T) {
occupied, release := make(chan struct{}), make(chan struct{})
// One worker and no queue slot: once the worker is busy there is nowhere
// left for a task to go.
server := &DASTServer{options: &Options{}, tasksPool: pond.New(1, 0)}
server.tasksPool.Submit(func() {
close(occupied)
<-release
})
<-occupied
defer close(release)
require.False(t, server.Submit("GET / HTTP/1.1\r\nHost: example.com\r\n\r\n", "http://example.com/"),
"a saturated queue must drop rather than block live proxy traffic")
require.Zero(t, server.endpointsInQueue.Load(), "a dropped request must not stay counted as queued")
}
func TestProxyOnlyDoesNotExposeFuzzAPI(t *testing.T) {
server := &DASTServer{options: &Options{ProxyAddress: "127.0.0.1:9056"}}
server.setupHandlers(false)
request := httptest.NewRequest(http.MethodPost, "/fuzz", strings.NewReader(`{"raw_http":"GET / HTTP/1.1\r\n\r\n","url":"http://example.com/"}`))
response := httptest.NewRecorder()
server.httpServer.Handler.ServeHTTP(response, request)
require.Equal(t, http.StatusNotFound, response.Code)
request = httptest.NewRequest(http.MethodGet, "/ca", nil)
response = httptest.NewRecorder()
server.httpServer.Handler.ServeHTTP(response, request)
require.Equal(t, http.StatusNotFound, response.Code, "CA download needs a running proxy instance")
}
func TestFuzzAPIEnabledWhenRequested(t *testing.T) {
server := &DASTServer{options: &Options{EnableFuzzAPI: true}}
server.setupHandlers(false)
request := httptest.NewRequest(http.MethodPost, "/fuzz", strings.NewReader(`{"raw_http":"","url":""}`))
response := httptest.NewRecorder()
server.httpServer.Handler.ServeHTTP(response, request)
require.Equal(t, http.StatusBadRequest, response.Code)
}
+13
View File
@@ -241,6 +241,10 @@ Dynamic Application Security Testing (DAST) API Server
<div class="terminal-line"><span class="key">Template Version</span><span class="value">{{.DASTServerInfo.NucleiTemplateVersion}}</span></div>
<div class="terminal-line"><span class="key">DAST Server API</span><span class="value">{{.DASTServerInfo.NucleiDastServerAPI}}</span></div>
<div class="terminal-line"><span class="key">Auth Status</span><span class="value">{{if .DASTServerInfo.ServerAuthEnabled}}ENABLED{{else}}DISABLED{{end}}</span></div>
{{if .DASTServerInfo.NucleiDastProxyAddr}}
<div class="terminal-line"><span class="key">DAST Proxy</span><span class="value">{{.DASTServerInfo.NucleiDastProxyAddr}}</span></div>
<div class="terminal-line"><span class="key">Proxy Auth Status</span><span class="value">{{if .DASTServerInfo.ProxyAuthEnabled}}ENABLED{{else}}DISABLED{{end}}</span></div>
{{end}}
</div>
<div class="section">
@@ -256,6 +260,15 @@ Dynamic Application Security Testing (DAST) API Server
<div class="terminal-line"><span class="key">Total Errors</span><span class="value">{{.DASTScanStatistics.TotalErroredRequests}}</span></div>
</div>
{{if .DASTServerInfo.NucleiDastProxyAddr}}
<div class="section">
<div class="section-header">[~] Proxy Traffic</div>
<div class="terminal-line"><span class="key">Requests Intercepted</span><span class="value">{{.DASTScanStatistics.ProxyRequestsIntercepted}}</span></div>
<div class="terminal-line"><span class="key">Tunnels Passed Through</span><span class="value">{{.DASTScanStatistics.ProxyTunnelsPassedThrough}}</span></div>
<div class="terminal-line"><span class="key">Requests Dropped</span><span class="value">{{.DASTScanStatistics.ProxyRequestsDropped}}</span></div>
</div>
{{end}}
<div class="section">
<div class="section-header">[!] Security Findings</div>
<div class="grid">
+67
View File
@@ -16,6 +16,7 @@ import (
"strconv"
"strings"
"sync"
"sync/atomic"
"time"
"github.com/julienschmidt/httprouter"
@@ -96,6 +97,8 @@ var httpTestcases = []integrationCase{
{Path: "protocols/http/response-data-literal-reuse.yaml", TestCase: &httpResponseDataLiteralReuse{}},
{Path: "protocols/http/raw-path-single-slash.yaml", TestCase: &httpRawPathSingleSlash{}},
{Path: "protocols/http/raw-unsafe-path-single-slash.yaml", TestCase: &httpRawUnsafePathSingleSlash{}},
{Path: "protocols/http/disable-http-cache.yaml", TestCase: &httpDisableCache{}},
{Path: "protocols/http/http-cache.yaml", TestCase: &httpCache{}},
}
type httpMultiVarSharing struct{}
@@ -1849,3 +1852,67 @@ func (h *httpRawUnsafePathSingleSlash) Execute(filepath string) error {
}
return nil
}
type httpCache struct{}
func (h *httpCache) Execute(filePath string) error {
router := httprouter.New()
var requestCount int32
router.GET("/", func(w http.ResponseWriter, r *http.Request, _ httprouter.Params) {
atomic.AddInt32(&requestCount, 1)
w.Header().Set("Cache-Control", "max-age=2")
w.WriteHeader(http.StatusOK)
_, _ = fmt.Fprint(w, requestCount)
})
ts := httptest.NewServer(router)
defer ts.Close()
results, err := testutils.RunNucleiTemplateAndGetResults(filePath, ts.URL, debug, "-http-cache")
if err != nil {
return err
}
// We expect 2 results because we made 2 requests and both should match
if err := expectResultsCount(results, 2); err != nil {
return err
}
// We expect only 1 actual request to the server because of caching
if count := atomic.LoadInt32(&requestCount); count != 1 {
return fmt.Errorf("expected 1 request to server, got %d", count)
}
return nil
}
type httpDisableCache struct{}
func (h *httpDisableCache) Execute(filePath string) error {
var requestCount int32
router := httprouter.New()
router.GET("/", func(w http.ResponseWriter, r *http.Request, _ httprouter.Params) {
atomic.AddInt32(&requestCount, 1)
w.Header().Set("Cache-Control", "max-age=2")
w.WriteHeader(http.StatusOK)
_, _ = fmt.Fprint(w, requestCount)
})
ts := httptest.NewServer(router)
defer ts.Close()
results, err := testutils.RunNucleiTemplateAndGetResults(filePath, ts.URL, debug, "-http-cache")
if err != nil {
return err
}
// We expect 2 results because we made 2 requests and both should match
if err := expectResultsCount(results, 2); err != nil {
return err
}
// We expect 2 actual requests to the server because cache is disabled
if count := atomic.LoadInt32(&requestCount); count != 2 {
return fmt.Errorf("expected 2 requests to server, got %d", count)
}
return nil
}
+9 -1
View File
@@ -9,6 +9,7 @@ import (
"log"
"net/http"
"net/http/httptest"
"os"
"path/filepath"
"strings"
"time"
@@ -92,7 +93,14 @@ func executeNucleiAsLibrary(templatePath, templateURL string) ([]string, error)
actualTemplatePath = fixturePath(actualTemplatePath)
}
defaultOpts.Templates = goflags.StringSlice{actualTemplatePath}
defaultOpts.ExcludeTags = config.ReadIgnoreFile().Tags
ignoreFile, err := config.ReadIgnoreFile()
if errors.Is(err, os.ErrNotExist) {
defaultOpts.Logger.Warning().Msgf("%s; continuing without ignore exclusions\n", err)
} else if err != nil {
return nil, err
} else {
defaultOpts.ExcludeTags = ignoreFile.Tags
}
outputWriter := testutils.NewMockOutputWriter(defaultOpts.OmitTemplate)
var results []string
@@ -4,6 +4,7 @@
package integration_test
import (
"errors"
"flag"
"fmt"
"os"
@@ -174,6 +175,11 @@ func executeIntegrationCase(testCase integrationCase) error {
if err == nil {
return nil
}
// a killed nuclei is not a flake; retrying it just spends the package
// timeout that every remaining case shares
if errors.Is(err, testutils.ErrCommandTimeout) {
return err
}
}
if retries > 1 {
return fmt.Errorf("test %s failed after %d attempts: %w", testCase.Path, retries, err)
@@ -0,0 +1,23 @@
id: disable-http-cache
info:
name: Disable HTTP Cache Test
author: dwisiswant0
severity: info
description: Tests if disable-http-cache works as expected
tags: test
requests:
- raw:
- |
GET / HTTP/1.1
Host: {{Hostname}}
- |
GET / HTTP/1.1
Host: {{Hostname}}
disable-http-cache: true
matchers:
- type: dsl
dsl:
- body_1 == "1"
- body_2 == "2"
@@ -0,0 +1,23 @@
id: http-cache
info:
name: HTTP Cache Test
author: dwisiswant0
severity: info
description: Tests if HTTP cache (RFC 9111) works as expected
tags: test
requests:
- raw:
- |
GET / HTTP/1.1
Host: {{Hostname}}
- |
GET / HTTP/1.1
Host: {{Hostname}}
disable-http-cache: false
matchers:
- type: dsl
dsl:
- body_1 == "1"
- body_2 == "1"
@@ -11,7 +11,6 @@ network:
inputs:
- data: "50494e47"
type: hex
- data: "\r\n"
read-size: 4
matchers:
+99 -59
View File
@@ -1,6 +1,7 @@
package testutils
import (
"context"
"crypto/tls"
"errors"
"fmt"
@@ -11,12 +12,29 @@ import (
"os/exec"
"regexp"
"strings"
"time"
"github.com/gobwas/ws"
"github.com/julienschmidt/httprouter"
"github.com/projectdiscovery/utils/conversion"
)
// defaultCommandTimeout bounds a single nuclei process in integration tests.
// The CI job is 50m; without this a hung scan leaves the suite nameless until
// the runner kills the step. It must stay well above the slowest legitimate
// case: raw-unsafe-path scans scanme.sh over the network and can take minutes.
const defaultCommandTimeout = 10 * time.Minute
// commandWaitDelay bounds the wait for output pipes after the process is
// killed. A grandchild that inherited stdout/stderr keeps the pipe open, and
// without this Output/CombinedOutput blocks forever despite the killed process.
const commandWaitDelay = 10 * time.Second
// ErrCommandTimeout reports that nuclei was killed by commandTimeout. Callers
// treat it as fatal rather than flaky: retrying a hang only burns the budget
// the whole suite shares.
var ErrCommandTimeout = errors.New("nuclei timed out")
type Runner struct {
BinaryPath string
WorkingDir string
@@ -26,6 +44,7 @@ type Runner struct {
AllowLocalFileAccess bool
InteractionsPollDuration string
InteractionsCooldownPeriod string
CommandTimeout time.Duration
}
type RunnerOption func(*Runner)
@@ -38,6 +57,7 @@ func NewRunner(options ...RunnerOption) *Runner {
AllowLocalFileAccess: true,
InteractionsPollDuration: "1",
InteractionsCooldownPeriod: "10",
CommandTimeout: defaultCommandTimeout,
}
for _, option := range options {
option(runner)
@@ -56,6 +76,7 @@ func (r *Runner) Clone(options ...RunnerOption) *Runner {
AllowLocalFileAccess: r.AllowLocalFileAccess,
InteractionsPollDuration: r.InteractionsPollDuration,
InteractionsCooldownPeriod: r.InteractionsCooldownPeriod,
CommandTimeout: r.CommandTimeout,
}
for _, option := range options {
option(clone)
@@ -106,7 +127,14 @@ func SetDefaultRunner(runner *Runner) {
defaultRunner = runner.Clone()
}
func (r *Runner) command(binaryPath string, args ...string) *exec.Cmd {
func (r *Runner) commandTimeout() time.Duration {
if r != nil && r.CommandTimeout > 0 {
return r.CommandTimeout
}
return defaultCommandTimeout
}
func (r *Runner) command(ctx context.Context, binaryPath string, args ...string) *exec.Cmd {
resolvedBinary := strings.TrimSpace(binaryPath)
if resolvedBinary == "" {
resolvedBinary = strings.TrimSpace(r.BinaryPath)
@@ -116,7 +144,8 @@ func (r *Runner) command(binaryPath string, args ...string) *exec.Cmd {
resolvedBinary = "nuclei"
}
cmd := exec.Command(resolvedBinary, args...)
cmd := exec.CommandContext(ctx, resolvedBinary, args...)
cmd.WaitDelay = commandWaitDelay
if r.WorkingDir != "" {
cmd.Dir = r.WorkingDir
}
@@ -124,6 +153,28 @@ func (r *Runner) command(binaryPath string, args ...string) *exec.Cmd {
return cmd
}
func (r *Runner) runCommand(combined bool, binaryPath string, setup func(cmd *exec.Cmd)) ([]byte, error) {
ctx, cancel := context.WithTimeout(context.Background(), r.commandTimeout())
defer cancel()
cmd := r.command(ctx, binaryPath)
setup(cmd)
var (
output []byte
err error
)
if combined {
output, err = cmd.CombinedOutput()
} else {
output, err = cmd.Output()
}
if errors.Is(ctx.Err(), context.DeadlineExceeded) {
return output, fmt.Errorf("%w after %s: %v", ErrCommandTimeout, r.commandTimeout(), err)
}
return output, err
}
func (r *Runner) buildArgs(args []string, pollDuration string) []string {
builtArgs := append([]string{}, args...)
@@ -188,20 +239,17 @@ func (r *Runner) WorkflowResults(workflow, url string, debug bool, extra ...stri
}
func (r *Runner) BareResults(debug bool, env []string, args ...string) ([]string, error) {
cmd := r.command("")
cmd.Args = append(cmd.Args[:1], r.buildArgs(args, r.InteractionsPollDuration)...)
cmd.Env = r.buildEnv(env)
if debug {
cmd.Args = append(cmd.Args, "-debug")
cmd.Stderr = os.Stderr
fmt.Println(cmd.String())
} else {
cmd.Args = append(cmd.Args, "-silent")
}
output, err := cmd.Output()
output, err := r.runCommand(false, "", func(cmd *exec.Cmd) {
cmd.Args = append(cmd.Args[:1], r.buildArgs(args, r.InteractionsPollDuration)...)
cmd.Env = r.buildEnv(env)
if debug {
cmd.Args = append(cmd.Args, "-debug")
cmd.Stderr = os.Stderr
fmt.Println(cmd.String())
} else {
cmd.Args = append(cmd.Args, "-silent")
}
})
if debug && len(output) > 0 {
fmt.Println(strings.TrimSpace(conversion.String(output)))
}
@@ -210,19 +258,17 @@ func (r *Runner) BareResults(debug bool, env []string, args ...string) ([]string
}
func (r *Runner) ArgsResults(debug bool, args ...string) ([]string, error) {
cmd := r.command("", append(append([]string{}, args...), r.ExtraArgs...)...)
cmd.Env = r.buildEnv(nil)
if debug {
cmd.Args = append(cmd.Args, "-debug")
cmd.Stderr = os.Stderr
fmt.Println(cmd.String())
} else {
cmd.Args = append(cmd.Args, "-silent")
}
output, err := cmd.Output()
output, err := r.runCommand(false, "", func(cmd *exec.Cmd) {
cmd.Args = append(cmd.Args[:1], append(append([]string{}, args...), r.ExtraArgs...)...)
cmd.Env = r.buildEnv(nil)
if debug {
cmd.Args = append(cmd.Args, "-debug")
cmd.Stderr = os.Stderr
fmt.Println(cmd.String())
} else {
cmd.Args = append(cmd.Args, "-silent")
}
})
if debug && len(output) > 0 {
fmt.Println(strings.TrimSpace(conversion.String(output)))
}
@@ -231,12 +277,11 @@ func (r *Runner) ArgsResults(debug bool, args ...string) ([]string, error) {
}
func (r *Runner) ArgsErrors(debug bool, env []string, args ...string) ([]string, error) {
cmd := r.command("")
cmd.Args = append(cmd.Args[:1], r.buildArgs(args, r.InteractionsPollDuration)...)
cmd.Args = append(cmd.Args, "-nc")
cmd.Env = r.buildEnv(env)
output, err := cmd.CombinedOutput()
output, err := r.runCommand(true, "", func(cmd *exec.Cmd) {
cmd.Args = append(cmd.Args[:1], r.buildArgs(args, r.InteractionsPollDuration)...)
cmd.Args = append(cmd.Args, "-nc")
cmd.Env = r.buildEnv(env)
})
if debug && len(output) > 0 {
fmt.Println(string(output))
}
@@ -263,20 +308,17 @@ func (r *Runner) ArgsResultsWithEnv(debug bool, env []string, args ...string) ([
}
func (r *Runner) LoadedTemplates(binaryPath string, debug bool, args []string) (string, error) {
cmd := r.command(binaryPath, append(append([]string{}, args...), r.ExtraArgs...)...)
cmd.Env = r.buildEnv(nil)
if r.DisableAutoUpdate {
cmd.Args = append(cmd.Args, "-duc")
}
if debug {
cmd.Args = append(cmd.Args, "-debug")
fmt.Println(cmd.String())
}
data, err := cmd.CombinedOutput()
data, err := r.runCommand(true, binaryPath, func(cmd *exec.Cmd) {
cmd.Args = append(cmd.Args[:1], append(append([]string{}, args...), r.ExtraArgs...)...)
cmd.Env = r.buildEnv(nil)
if r.DisableAutoUpdate {
cmd.Args = append(cmd.Args, "-duc")
}
if debug {
cmd.Args = append(cmd.Args, "-debug")
fmt.Println(cmd.String())
}
})
if debug && len(data) > 0 {
fmt.Println(string(data))
}
@@ -295,16 +337,14 @@ func (r *Runner) LoadedTemplates(binaryPath string, debug bool, args []string) (
func (r *Runner) CombinedOutput(debug bool, args []string) (string, error) {
builtArgs := r.buildArgs(args, r.InteractionsPollDuration)
cmd := r.command("", builtArgs...)
cmd.Env = r.buildEnv(nil)
if debug {
cmd.Args = append(cmd.Args, "-debug")
fmt.Println(cmd.String())
}
data, err := cmd.CombinedOutput()
data, err := r.runCommand(true, "", func(cmd *exec.Cmd) {
cmd.Args = append(cmd.Args[:1], builtArgs...)
cmd.Env = r.buildEnv(nil)
if debug {
cmd.Args = append(cmd.Args, "-debug")
fmt.Println(cmd.String())
}
})
if debug && len(data) > 0 {
fmt.Println(string(data))
}
+28
View File
@@ -0,0 +1,28 @@
package nuclei
import (
"context"
"os"
"path/filepath"
"strconv"
"testing"
"github.com/projectdiscovery/nuclei/v3/pkg/catalog/config"
"github.com/stretchr/testify/require"
)
func TestNewNucleiEngineRejectsCorruptActiveIgnoreFile(t *testing.T) {
root := t.TempDir()
path := filepath.Join(root, config.NucleiIgnoreFileName)
require.NoError(t, os.WriteFile(path, []byte("tags: ["), 0o600))
cfg := config.DefaultConfig
oldRoot := cfg.TemplatesDirectory
t.Cleanup(func() { cfg.SetTemplatesDir(oldRoot) })
cfg.SetTemplatesDir(root)
engine, err := NewNucleiEngineCtx(context.Background())
require.Nil(t, engine)
require.ErrorContains(t, err, "error parsing")
require.ErrorContains(t, err, strconv.Quote(path))
}
+45
View File
@@ -50,6 +50,9 @@ func createEphemeralObjects(ctx context.Context, base *NucleiEngine, opts *types
// would otherwise route all findings to whichever call last won the race.
DoNotCache: true,
}
if base.interactshClient != nil {
u.executerOpts.InteractshScope = base.interactshClient.NewRequestScope()
}
if opts.ShouldUseHostError() && base.hostErrCache != nil {
u.executerOpts.HostErrorsCache = base.hostErrCache
}
@@ -64,6 +67,32 @@ func createEphemeralObjects(ctx context.Context, base *NucleiEngine, opts *types
return u, nil
}
// restoreBaseExcludeTags re-adds any of the engine's baseline excluded tags that
// a per-execution option dropped, appending only what is missing so the result
// is order-stable and repeated application is a no-op.
//
// The current ExcludeTags slice is cloned before merging so a caller-owned
// backing array (e.g. a reused WithTemplateFilters ExcludeTags value with spare
// capacity) is never mutated or shared across concurrent executions.
func restoreBaseExcludeTags(base []string, opts *types.Options) {
if len(base) == 0 {
return
}
merged := append([]string(nil), opts.ExcludeTags...)
present := make(map[string]struct{}, len(merged)+len(base))
for _, tag := range merged {
present[tag] = struct{}{}
}
for _, tag := range base {
if _, ok := present[tag]; ok {
continue
}
merged = append(merged, tag)
present[tag] = struct{}{}
}
opts.ExcludeTags = merged
}
// resolveEphemeralOutput combines the base/global writer with any per-call result callbacks.
// Global callbacks keep working; per-call callbacks are isolated to this execution.
func resolveEphemeralOutput(base, call *NucleiEngine) output.Writer {
@@ -90,6 +119,9 @@ func resolveEphemeralOutput(base, call *NucleiEngine) output.Writer {
// closeEphemeralObjects closes all resources used by ephemeral nuclei objects/instances/types
func closeEphemeralObjects(u *unsafeOptions) {
if u.executerOpts.InteractshScope != nil {
u.executerOpts.InteractshScope.Close()
}
if u.executerOpts.RateLimiter != nil {
u.executerOpts.RateLimiter.Stop()
}
@@ -98,6 +130,7 @@ func closeEphemeralObjects(u *unsafeOptions) {
u.executerOpts.Output = nil
u.executerOpts.IssuesClient = nil
u.executerOpts.Interactsh = nil
u.executerOpts.InteractshScope = nil
u.executerOpts.HostErrorsCache = nil
u.executerOpts.Progress = nil
u.executerOpts.Catalog = nil
@@ -161,6 +194,18 @@ func (e *ThreadSafeNucleiEngine) ExecuteNucleiWithOptsCtx(ctx context.Context, t
return err
}
}
// Per-execution options land on a COPY of the engine's options, so an option
// that assigns a whole filter set clears every field it does not name —
// WithTemplateFilters does exactly that. ExcludeTags is one of those fields,
// and it carries the .nuclei-ignore deny-list that applyRequiredDefaults
// installed at construction; nothing re-reads the ignore file afterwards, so
// the exclusions would be lost for this execution and templates tagged dos,
// bruteforce, fuzz, local or txt-service would run.
//
// Restore the engine's baseline exclusions so a per-execution filter can add
// to them but never silently discard them. IncludeTags remains the explicit
// per-tag override for callers who do want an ignored template to run.
restoreBaseExcludeTags(e.eng.opts.ExcludeTags, tmpEngine.opts)
// create ephemeral nuclei objects/instances/types using base nuclei engine
unsafeOpts, err := createEphemeralObjects(ctx, e.eng, tmpEngine.opts, resolveEphemeralOutput(e.eng, tmpEngine))
+161
View File
@@ -0,0 +1,161 @@
package nuclei_test
import (
"context"
"net/http"
"net/http/httptest"
"os"
"path/filepath"
"sync/atomic"
"testing"
nuclei "github.com/projectdiscovery/nuclei/v3/lib"
"github.com/projectdiscovery/nuclei/v3/pkg/catalog/config"
"github.com/projectdiscovery/nuclei/v3/pkg/output"
"github.com/projectdiscovery/nuclei/v3/pkg/types"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
// TestExecuteNucleiWithOptsCtxKeepsBaseExcludeTags covers the case where a
// per-execution filter that names only Tags used to clear the engine's
// ExcludeTags — which carry the .nuclei-ignore deny-list installed at
// construction — letting templates tagged dos (and local, fuzz, bruteforce,
// txt-service) execute against the target.
//
// The engine's construction-time ExcludeTags stand in for that deny-list, and
// the per-execution filter deliberately asks for the excluded tag: exclusion
// must still win, because IncludeTags is the documented way to override it.
func TestExecuteNucleiWithOptsCtxKeepsBaseExcludeTags(t *testing.T) {
templatePath := writeSDKDosTaggedTemplate(t)
var reached atomic.Bool
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
reached.Store(true)
_, _ = w.Write([]byte("token-dos"))
}))
t.Cleanup(srv.Close)
ne, err := nuclei.NewThreadSafeNucleiEngineCtx(context.Background(),
nuclei.WithTemplateFilters(nuclei.TemplateFilters{ExcludeTags: []string{"dos"}}),
)
require.NoError(t, err, "could not create thread-safe nuclei engine")
t.Cleanup(ne.Close)
var matches atomic.Int64
ne.GlobalResultCallback(func(*output.ResultEvent) { matches.Add(1) })
err = ne.ExecuteNucleiWithOptsCtx(context.Background(), []string{srv.URL},
nuclei.WithTemplatesOrWorkflows(nuclei.TemplateSources{Templates: []string{templatePath}}),
nuclei.WithTemplateFilters(nuclei.TemplateFilters{Tags: []string{"dos"}}),
)
require.ErrorIs(t, err, nuclei.ErrNoTemplatesAvailable,
"the dos-tagged template must stay excluded, leaving nothing to run")
assert.Zero(t, matches.Load(), "an excluded template must not report results")
assert.False(t, reached.Load(), "an excluded template must not reach the target")
}
// TestExecuteNucleiWithOptsCtxHonoursIncludeTags is the other half of the
// contract: restoring the baseline exclusions must not break the documented
// per-tag override, or callers lose the only way to run an ignored template.
func TestExecuteNucleiWithOptsCtxHonoursIncludeTags(t *testing.T) {
templatePath := writeSDKDosTaggedTemplate(t)
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
_, _ = w.Write([]byte("token-dos"))
}))
t.Cleanup(srv.Close)
ne, err := nuclei.NewThreadSafeNucleiEngineCtx(context.Background(),
nuclei.WithTemplateFilters(nuclei.TemplateFilters{ExcludeTags: []string{"dos"}}),
)
require.NoError(t, err, "could not create thread-safe nuclei engine")
t.Cleanup(ne.Close)
var matches atomic.Int64
ne.GlobalResultCallback(func(*output.ResultEvent) { matches.Add(1) })
err = ne.ExecuteNucleiWithOptsCtx(context.Background(), []string{srv.URL},
nuclei.WithTemplatesOrWorkflows(nuclei.TemplateSources{Templates: []string{templatePath}}),
nuclei.WithTemplateFilters(nuclei.TemplateFilters{IncludeTags: []string{"dos"}}),
)
require.NoError(t, err)
assert.NotZero(t, matches.Load(), "IncludeTags should still override the exclusion")
}
func writeSDKDosTaggedTemplate(t *testing.T) string {
t.Helper()
dir := t.TempDir()
path := filepath.Join(dir, "sdk-dos-tagged.yaml")
content := `id: sdk-dos-tagged
info:
name: SDK dos tagged
author: nuclei-sdk-test
severity: info
tags: dos
http:
- method: GET
path:
- "{{BaseURL}}/"
matchers:
- type: word
words:
- "token-"
`
require.NoError(t, os.WriteFile(path, []byte(content), 0o600))
return path
}
// TestExecuteNucleiWithOptsCtxKeepsIgnoreFileExcludeTags is the end-to-end path
// for #7695 once #7697 is on the tree: tags from a real .nuclei-ignore must
// survive a per-execution WithTemplateFilters that only sets Tags.
func TestExecuteNucleiWithOptsCtxKeepsIgnoreFileExcludeTags(t *testing.T) {
previousConfigDir := config.DefaultConfig.GetConfigDir()
previousTemplatesDir := config.DefaultConfig.TemplatesDirectory
previousStateDir := config.DefaultConfig.GetStateDir()
t.Cleanup(func() {
config.DefaultConfig.SetConfigDir(previousConfigDir)
config.DefaultConfig.SetStateDir(previousStateDir)
config.DefaultConfig.SetTemplatesDir(previousTemplatesDir)
})
dir := t.TempDir()
config.DefaultConfig.SetConfigDir(dir)
config.DefaultConfig.SetTemplatesDir(dir)
require.NoError(t, os.WriteFile(
config.DefaultConfig.GetActiveIgnoreFilePath(),
[]byte("tags:\n - dos\n"),
0o600,
))
templatePath := writeSDKDosTaggedTemplate(t)
var reached atomic.Bool
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
reached.Store(true)
_, _ = w.Write([]byte("token-dos"))
}))
t.Cleanup(srv.Close)
ne, err := nuclei.NewThreadSafeNucleiEngineCtx(context.Background(),
nuclei.WithOptions(types.DefaultOptions()),
nuclei.DisableUpdateCheck(),
)
require.NoError(t, err, "could not create thread-safe nuclei engine")
t.Cleanup(ne.Close)
var matches atomic.Int64
ne.GlobalResultCallback(func(*output.ResultEvent) { matches.Add(1) })
err = ne.ExecuteNucleiWithOptsCtx(context.Background(), []string{srv.URL},
nuclei.WithTemplatesOrWorkflows(nuclei.TemplateSources{Templates: []string{templatePath}}),
nuclei.WithTemplateFilters(nuclei.TemplateFilters{Tags: []string{"dos"}}),
)
require.ErrorIs(t, err, nuclei.ErrNoTemplatesAvailable,
"ignore-file dos exclusion must survive a Tags-only per-execution filter")
assert.Zero(t, matches.Load())
assert.False(t, reached.Load())
}
+88
View File
@@ -0,0 +1,88 @@
package nuclei
import (
"sync"
"testing"
"github.com/projectdiscovery/nuclei/v3/pkg/types"
"github.com/stretchr/testify/assert"
)
func TestRestoreBaseExcludeTags(t *testing.T) {
tests := []struct {
name string
base []string
current []string
expected []string
}{
{
name: "per-execution filter cleared the deny-list",
base: []string{"dos", "fuzz"},
current: nil,
expected: []string{"dos", "fuzz"},
},
{
name: "per-execution exclusions are kept alongside the baseline",
base: []string{"dos"},
current: []string{"custom"},
expected: []string{"custom", "dos"},
},
{
name: "entries already present are not duplicated",
base: []string{"dos", "fuzz"},
current: []string{"fuzz"},
expected: []string{"fuzz", "dos"},
},
{
name: "no baseline is a no-op",
base: nil,
current: []string{"custom"},
expected: []string{"custom"},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
opts := &types.Options{ExcludeTags: tt.current}
restoreBaseExcludeTags(tt.base, opts)
assert.Equal(t, tt.expected, []string(opts.ExcludeTags))
})
}
}
// TestRestoreBaseExcludeTagsIsIdempotent matters because the helper runs on
// every execution against a long-lived engine.
func TestRestoreBaseExcludeTagsIsIdempotent(t *testing.T) {
opts := &types.Options{}
restoreBaseExcludeTags([]string{"dos", "fuzz"}, opts)
restoreBaseExcludeTags([]string{"dos", "fuzz"}, opts)
assert.Equal(t, []string{"dos", "fuzz"}, []string(opts.ExcludeTags))
}
// TestRestoreBaseExcludeTagsDoesNotMutateCallerSlice guards the concurrent
// ThreadSafeNucleiEngine case where WithTemplateFilters assigns a caller-owned
// ExcludeTags slice with spare capacity: a naive append would race and mutate
// the caller's backing array.
func TestRestoreBaseExcludeTagsDoesNotMutateCallerSlice(t *testing.T) {
shared := make([]string, 1, 8)
shared[0] = "custom"
base := []string{"dos", "fuzz"}
var wg sync.WaitGroup
for i := 0; i < 32; i++ {
wg.Add(1)
go func() {
defer wg.Done()
opts := &types.Options{ExcludeTags: shared}
restoreBaseExcludeTags(base, opts)
if got := []string(opts.ExcludeTags); len(got) != 3 || got[0] != "custom" || got[1] != "dos" || got[2] != "fuzz" {
panic(got)
}
}()
}
wg.Wait()
assert.Equal(t, []string{"custom"}, shared[:1])
assert.Equal(t, make([]string, 7), shared[1:cap(shared)])
}
+27 -4
View File
@@ -135,13 +135,32 @@ func (e *NucleiEngine) applyRequiredDefaults(ctx context.Context) {
e.opts.ExcludeTags = []string{}
}
// these templates are known to have weak matchers
// and idea is to disable them to avoid false positives
e.opts.ExcludeTags = append(e.opts.ExcludeTags, config.ReadIgnoreFile().Tags...)
e.inputProvider = provider.NewSimpleInputProvider()
}
func (e *NucleiEngine) loadIgnoreFile() error {
ignoreFile, err := config.ReadIgnoreFile()
if errors.Is(err, os.ErrNotExist) {
e.Logger.Warning().Msgf("Could not read active .nuclei-ignore file: %s; continuing without ignore exclusions", err)
return nil
}
if err != nil {
return err
}
// .nuclei-ignore blocks templates that should not run by default, by tag
// (dos, fuzz, ...) and by path (templates known to have weak matchers, which
// would otherwise generate false positives). Both sections are applied, as
// the CLI runner does — applying only the tags leaves the `files` section
// inert for every SDK consumer.
e.opts.ExcludeTags = append(e.opts.ExcludeTags, ignoreFile.Tags...)
e.opts.ExcludedTemplates = append(e.opts.ExcludedTemplates, ignoreFile.Files...)
return nil
}
// init
func (e *NucleiEngine) init(ctx context.Context) error {
if err := config.DefaultConfig.InitializationError(); err != nil {
@@ -165,6 +184,10 @@ func (e *NucleiEngine) init(ctx context.Context) error {
return err
}
if err := e.loadIgnoreFile(); err != nil {
return err
}
if e.opts.Parser != nil {
if op, ok := e.opts.Parser.(*templates.Parser); ok {
e.parser = op
+35
View File
@@ -8,6 +8,7 @@ import (
"time"
nuclei "github.com/projectdiscovery/nuclei/v3/lib"
"github.com/projectdiscovery/nuclei/v3/pkg/catalog/config"
"github.com/projectdiscovery/nuclei/v3/pkg/types"
"github.com/stretchr/testify/require"
)
@@ -120,3 +121,37 @@ func TestWithOptionsRateLimitSetsRuntimeLimiter(t *testing.T) {
require.NotNil(t, execOpts.RateLimiter, "rate limiter should be initialized")
require.Equal(t, opts.RateLimit, int(execOpts.RateLimiter.GetLimit()), "runtime limiter should match rate limit from WithOptions")
}
// TestIgnoreFileFilesSectionIsApplied verifies the SDK applies BOTH sections of
// .nuclei-ignore, matching the CLI runner. The files section suppresses
// templates known to have weak matchers, so skipping it hands SDK consumers
// false positives the CLI would never report.
func TestIgnoreFileFilesSectionIsApplied(t *testing.T) {
previousConfigDir := config.DefaultConfig.GetConfigDir()
previousTemplatesDir := config.DefaultConfig.TemplatesDirectory
previousStateDir := config.DefaultConfig.GetStateDir()
t.Cleanup(func() {
config.DefaultConfig.SetConfigDir(previousConfigDir)
config.DefaultConfig.SetStateDir(previousStateDir)
config.DefaultConfig.SetTemplatesDir(previousTemplatesDir)
})
dir := t.TempDir()
config.DefaultConfig.SetConfigDir(dir)
config.DefaultConfig.SetTemplatesDir(dir)
ignoreFile := "tags:\n - dos\nfiles:\n - http/fuzzing/wordpress-themes-detect.yaml\n"
require.NoError(t, os.WriteFile(config.DefaultConfig.GetActiveIgnoreFilePath(), []byte(ignoreFile), 0o600))
ne, err := nuclei.NewNucleiEngineCtx(context.Background(),
nuclei.WithOptions(types.DefaultOptions()),
nuclei.DisableUpdateCheck(),
)
require.NoError(t, err, "could not create nuclei engine")
defer ne.Close()
require.Contains(t, ne.Options().ExcludeTags, "dos",
"tags section should reach ExcludeTags")
require.Contains(t, ne.Options().ExcludedTemplates, "http/fuzzing/wordpress-themes-detect.yaml",
"files section should reach ExcludedTemplates")
}
+3
View File
@@ -28,6 +28,9 @@ var knownLeaks = []goleak.Option{
goleak.IgnoreAnyContainingPkg("github.com/syndtr/goleveldb"),
goleak.IgnoreAnyContainingPkg("github.com/projectdiscovery/ratelimit"),
goleak.IgnoreAnyFunction("github.com/projectdiscovery/nuclei/v3/pkg/protocols/common/protocolstate.StartActiveMemGuardian.func1"),
// NOTE(dwisiswant0): This is an indirect call and not used anywhere, not
// even within SOPS (#5841). See `go mod why go.opencensus.io/stats/view`.
goleak.IgnoreAnyContainingPkg("go.opencensus.io/stats/view"),
}
func TestSimpleNuclei(t *testing.T) {
+5
View File
@@ -880,6 +880,11 @@
"title": "optional disable cookie reuse",
"description": "Optional setting that disables cookie reuse"
},
"disable-http-cache": {
"type": "boolean",
"title": "disable HTTP cache",
"description": "Turns off HTTP caching for this request; cannot enable cache when -http-cache is unset"
},
"read-all": {
"type": "boolean",
"title": "force read all body",
+46
View File
@@ -1,9 +1,13 @@
package authx
import (
"bytes"
"fmt"
"runtime"
"strconv"
"strings"
"sync"
"sync/atomic"
"github.com/projectdiscovery/gologger"
"github.com/projectdiscovery/nuclei/v3/pkg/protocols/common/replacer"
@@ -19,6 +23,9 @@ type LazyFetchSecret func(d *Dynamic) error
type fetchState struct {
once sync.Once
err error
// owner is the goroutine running the fetch callback. Nested Fetch/GetStrategies
// on that goroutine must not wait on once (sync.Once deadlocks on re-entry).
owner atomic.Int64
}
var (
@@ -191,6 +198,11 @@ func (d *Dynamic) applyValuesToSecret(secret *Secret) error {
// GetStrategies returns the auth strategies for the dynamic secret
func (d *Dynamic) GetStrategies() []AuthStrategy {
// Nested lookup from inside the fetch callback (login template ApplyAuth)
// must not wait on Fetch and must not apply still-unresolved secrets.
if d.fetchingOnThisGoroutine() {
return nil
}
// Ensure fetch has completed before returning strategies.
// Fetch errors are treated as non-fatal here so a failed dynamic auth fetch
// does not terminate the entire scan process.
@@ -219,7 +231,13 @@ func (d *Dynamic) Fetch(isFatal bool) error {
return errkit.New("dynamic secret not validated: call Validate() before Fetch()")
}
if d.fetchingOnThisGoroutine() {
return d.fetchState.err
}
d.fetchState.once.Do(func() {
d.fetchState.owner.Store(goroutineID())
defer d.fetchState.owner.Store(0)
if d.fetchCallback == nil {
d.fetchState.err = errkit.New("dynamic secret fetch callback not set: call SetLazyFetchCallback() before Fetch()")
return
@@ -240,3 +258,31 @@ func (d *Dynamic) Error() error {
}
return d.fetchState.err
}
func (d *Dynamic) fetchingOnThisGoroutine() bool {
if d == nil || d.fetchState == nil {
return false
}
owner := d.fetchState.owner.Load()
return owner != 0 && owner == goroutineID()
}
func goroutineID() int64 {
var buf [64]byte
n := runtime.Stack(buf[:], false)
s := buf[:n]
const prefix = "goroutine "
if !bytes.HasPrefix(s, []byte(prefix)) {
return 0
}
s = s[len(prefix):]
i := bytes.IndexByte(s, ' ')
if i <= 0 {
return 0
}
id, err := strconv.ParseInt(string(s[:i]), 10, 64)
if err != nil {
return 0
}
return id
}
+72
View File
@@ -226,6 +226,78 @@ func TestDynamicFetchConcurrent(t *testing.T) {
})
}
func TestDynamicFetchReentry(t *testing.T) {
t.Run("nested Fetch from callback does not deadlock", func(t *testing.T) {
d := &Dynamic{
TemplatePath: "auth.yaml",
Variables: []KV{{Key: "user", Value: "admin"}},
}
require.NoError(t, d.Validate())
var nestedErr error
d.SetLazyFetchCallback(func(dynamic *Dynamic) error {
nestedErr = dynamic.Fetch(false)
dynamic.Extracted = map[string]interface{}{"token": "abc"}
return nil
})
done := make(chan error, 1)
go func() {
done <- d.Fetch(false)
}()
select {
case err := <-done:
require.NoError(t, err)
require.NoError(t, nestedErr)
case <-time.After(2 * time.Second):
t.Fatal("Fetch deadlocked on re-entry")
}
})
t.Run("nested GetStrategies does not apply unresolved secrets", func(t *testing.T) {
d := &Dynamic{
Secret: &Secret{
Type: "Header",
Domains: []string{"example.com"},
Headers: []KV{{Key: "X-Token", Value: "{{token}}"}},
},
TemplatePath: "auth.yaml",
Variables: []KV{{Key: "user", Value: "admin"}},
}
require.NoError(t, d.Validate())
d.SetLazyFetchCallback(func(dynamic *Dynamic) error {
req, err := http.NewRequest(http.MethodGet, "https://example.com", nil)
require.NoError(t, err)
strategy := &DynamicAuthStrategy{Dynamic: *dynamic}
strategy.Apply(req)
require.Empty(t, req.Header.Get("X-Token"), "login request must not receive secret-file auth during fetch")
require.Nil(t, dynamic.GetStrategies())
dynamic.Extracted = map[string]interface{}{"token": "abc"}
return nil
})
done := make(chan error, 1)
go func() {
done <- d.Fetch(false)
}()
select {
case err := <-done:
require.NoError(t, err)
case <-time.After(2 * time.Second):
t.Fatal("GetStrategies deadlocked on re-entry")
}
req, err := http.NewRequest(http.MethodGet, "https://example.com", nil)
require.NoError(t, err)
strategy := &DynamicAuthStrategy{Dynamic: *d}
strategy.Apply(req)
require.Equal(t, "abc", req.Header.Get("X-Token"), "scan requests must still receive secrets after fetch")
})
}
func TestDynamicValidate(t *testing.T) {
t.Run("missing template path", func(t *testing.T) {
d := &Dynamic{
+23 -8
View File
@@ -1,12 +1,15 @@
package authx
import (
"errors"
"fmt"
"os"
"path/filepath"
"regexp"
"strings"
"github.com/getsops/sops/v3"
"github.com/getsops/sops/v3/decrypt"
"github.com/projectdiscovery/nuclei/v3/pkg/utils/json"
"github.com/projectdiscovery/utils/errkit"
"github.com/projectdiscovery/utils/generic"
@@ -205,18 +208,30 @@ func (c *Cookie) Parse() error {
// GetAuthDataFromFile reads the auth data from file
func GetAuthDataFromFile(file string) (*Authx, error) {
ext := filepath.Ext(file)
if !generic.EqualsAny(ext, ".yml", ".yaml", ".json") {
return nil, fmt.Errorf("invalid file extension: supported extensions are .yml,.yaml and .json got %s", ext)
var data []byte
ext := strings.TrimPrefix(filepath.Ext(file), ".")
if !generic.EqualsAny(ext, "yml", "yaml", "json") {
return nil, fmt.Errorf(`unsupported extensions: valid extensions are "yml", "yaml", and "json"; got %q`, ext)
}
bin, err := os.ReadFile(file)
data, err := decrypt.File(file, ext)
if err != nil {
return nil, err
if errors.Is(err, sops.MetadataNotFound) {
data, err = os.ReadFile(file)
if err != nil {
return nil, fmt.Errorf("could not read secret file: %w", err)
}
} else {
return nil, fmt.Errorf("could not decrypt file: %w", err)
}
}
if ext == ".yml" || ext == ".yaml" {
return GetAuthDataFromYAML(bin)
if ext == "yml" || ext == "yaml" {
return GetAuthDataFromYAML(data)
}
return GetAuthDataFromJSON(bin)
return GetAuthDataFromJSON(data)
}
// GetTemplatePathsFromSecretFile reads the template IDs from the secret file
+85 -5
View File
@@ -1,20 +1,100 @@
package authx
import (
"crypto/rand"
"os"
"path/filepath"
"testing"
"time"
"filippo.io/age"
"github.com/getsops/sops/v3"
"github.com/getsops/sops/v3/aes"
sopsage "github.com/getsops/sops/v3/age"
"github.com/getsops/sops/v3/config"
"github.com/getsops/sops/v3/stores/yaml"
"github.com/getsops/sops/v3/version"
"github.com/stretchr/testify/require"
)
func TestSecretsUnmarshal(t *testing.T) {
loc := "testData/example-auth.yaml"
data, err := GetAuthDataFromFile(loc)
require.Nil(t, err, "could not read secrets file")
data, err := GetAuthDataFromFile("testData/example-auth.yaml")
require.NoError(t, err, "could not read secrets file")
require.NotNil(t, data, "could not read secrets file")
t.Run("encrypted", func(t *testing.T) {
loc := encryptAuthFile(t, "testData/example-auth.yaml")
decryptedData, err := GetAuthDataFromFile(loc)
require.NoError(t, err, "could not read encrypted secrets file")
require.NotNil(t, decryptedData, "could not read secrets file")
require.Equal(t, data, decryptedData, "decrypted data should match original data")
for _, s := range decryptedData.Secrets {
require.NoError(t, s.Validate(), "could not validate secret")
}
for _, d := range decryptedData.Dynamic {
require.NoError(t, d.Validate(), "could not validate dynamic")
}
})
for _, s := range data.Secrets {
require.Nil(t, s.Validate(), "could not validate secret")
require.NoError(t, s.Validate(), "could not validate secret")
}
for _, d := range data.Dynamic {
require.Nil(t, d.Validate(), "could not validate dynamic")
require.NoError(t, d.Validate(), "could not validate dynamic")
}
}
func encryptAuthFile(t *testing.T, source string) string {
t.Helper()
plaintext, err := os.ReadFile(source)
require.NoError(t, err)
identity, err := age.GenerateX25519Identity()
require.NoError(t, err)
t.Setenv(sopsage.SopsAgeKeyEnv, identity.String())
masterKey, err := sopsage.MasterKeyFromRecipient(identity.Recipient().String())
require.NoError(t, err)
dataKey := make([]byte, 32)
_, err = rand.Read(dataKey)
require.NoError(t, err)
require.NoError(t, masterKey.Encrypt(dataKey))
store := yaml.NewStore(&config.YAMLStoreConfig{})
branches, err := store.LoadPlainFile(plaintext)
require.NoError(t, err)
tree := sops.Tree{
Branches: branches,
Metadata: sops.Metadata{
EncryptedRegex: `^(password|username|token|value|key|raw)$`,
KeyGroups: []sops.KeyGroup{{masterKey}},
Version: version.Version,
},
}
cipher := aes.NewCipher()
mac, err := tree.Encrypt(dataKey, cipher)
require.NoError(t, err)
tree.Metadata.LastModified = time.Now().UTC()
tree.Metadata.MessageAuthenticationCode, err = cipher.Encrypt(
mac,
dataKey,
tree.Metadata.LastModified.Format(time.RFC3339),
)
require.NoError(t, err)
encrypted, err := store.EmitEncryptedFile(tree)
require.NoError(t, err)
require.Contains(t, string(encrypted), "sops:")
require.NotContains(t, string(encrypted), "1a2b3c4d5e6f7g8h9i0j")
path := filepath.Join(t.TempDir(), "example-auth.yaml")
require.NoError(t, os.WriteFile(path, encrypted, 0600))
return path
}
+102 -11
View File
@@ -1,50 +1,141 @@
package config
import (
"bytes"
"crypto/md5"
"errors"
"fmt"
"io"
"os"
"reflect"
"slices"
"strings"
"github.com/projectdiscovery/gologger"
"github.com/projectdiscovery/nuclei/v3/pkg/utils/yaml"
)
// maxIgnoreFileBytes caps untrusted ignore-file payloads (PDTM downloads,
// template archives) before YAML decode.
const maxIgnoreFileBytes = 1 << 20
var errMultipleIgnoreFileDocuments = errors.New("multiple YAML documents are not supported")
// IgnoreFile is an internal nuclei template blocking configuration file
type IgnoreFile struct {
Tags []string `yaml:"tags"`
Files []string `yaml:"files"`
}
// ReadIgnoreFile reads the .nuclei-ignore file returning blocked tags and paths.
func ReadIgnoreFile() IgnoreFile {
// ReadIgnoreFile reads the active .nuclei-ignore file and returns blocked tags
// and paths. Callers must decide whether a read error is recoverable. Malformed
// content is returned as an error to prevent a fail-open scan.
func ReadIgnoreFile() (IgnoreFile, error) {
path := DefaultConfig.GetActiveIgnoreFilePath()
file, err := os.Open(path)
if err != nil {
gologger.Error().Msgf("Could not read .nuclei-ignore file %q: %s\n", path, err)
return IgnoreFile{}
return IgnoreFile{}, fmt.Errorf("error opening %q: %w", path, err)
}
defer func() {
_ = file.Close()
}()
ignore := IgnoreFile{}
if err := yaml.NewDecoder(file).Decode(&ignore); err != nil {
gologger.Error().Msgf("Could not parse .nuclei-ignore file %q: %s\n", path, err)
return IgnoreFile{}
ignore, err := decodeIgnoreFile(file, false)
if err != nil {
return IgnoreFile{}, fmt.Errorf("error parsing %q: %w", path, err)
}
return ignore
return ignore, nil
}
// WriteActiveIgnoreFile atomically replaces the active root ignore file.
func (c *Config) WriteActiveIgnoreFile(data []byte) error {
if err := atomicWriteFile(c.GetActiveIgnoreFilePath(), data, 0o600); err != nil {
path := c.GetActiveIgnoreFilePath()
if err := validateIgnoreFileContents(data); err != nil {
return fmt.Errorf("invalid active .nuclei-ignore file %q: %w", path, err)
}
if err := atomicWriteFile(path, data, 0o600); err != nil {
return fmt.Errorf("write active .nuclei-ignore file: %w", err)
}
return nil
}
func validateIgnoreFileContents(data []byte) error {
if len(data) > maxIgnoreFileBytes {
return fmt.Errorf("payload exceeds %d bytes", maxIgnoreFileBytes)
}
if len(bytes.TrimSpace(data)) == 0 {
return errors.New("empty document")
}
if _, err := decodeIgnoreFile(bytes.NewReader(data), true); err != nil {
if errors.Is(err, io.EOF) {
return errors.New("empty document")
}
return err
}
decoder := yaml.NewDecoder(bytes.NewReader(data))
var raw map[string]any
if err := decoder.Decode(&raw); err != nil {
return err
}
if raw == nil {
return errors.New("document must be a YAML mapping")
}
allowed := ignoreFileFieldNames()
found := false
for key := range raw {
if !slices.Contains(allowed, key) {
return fmt.Errorf("unknown field %q", key)
}
found = true
}
if !found {
return fmt.Errorf("document must define at least one of %s", strings.Join(allowed, ", "))
}
return nil
}
func decodeIgnoreFile(r io.Reader, knownFields bool) (IgnoreFile, error) {
decoder := yaml.NewDecoder(r)
decoder.KnownFields(knownFields)
ignore := IgnoreFile{}
if err := decoder.Decode(&ignore); err != nil {
return IgnoreFile{}, err
}
var additionalDocument any
if err := decoder.Decode(&additionalDocument); !errors.Is(err, io.EOF) {
if err != nil {
return IgnoreFile{}, err
}
return IgnoreFile{}, errMultipleIgnoreFileDocuments
}
return ignore, nil
}
func ignoreFileFieldNames() []string {
typ := reflect.TypeOf(IgnoreFile{})
names := make([]string, 0, typ.NumField())
for i := 0; i < typ.NumField(); i++ {
name, _, _ := strings.Cut(typ.Field(i).Tag.Get("yaml"), ",")
if name == "" || name == "-" {
continue
}
names = append(names, name)
}
slices.Sort(names)
return names
}
// IgnoreFileNeedsUpdate reports whether the active ignore file is
// missing or differs from the latest known hash.
func (c *Config) IgnoreFileNeedsUpdate(latestHash string) bool {
+150 -6
View File
@@ -1,33 +1,106 @@
package config
import (
"bytes"
"errors"
"os"
"path/filepath"
"strconv"
"strings"
"testing"
)
func TestReadIgnoreFileAllowsMissingOrInvalidActiveFile(t *testing.T) {
func setActiveIgnoreFileRoot(t *testing.T) string {
t.Helper()
root := t.TempDir()
cfg := DefaultConfig
oldRoot := cfg.TemplatesDirectory
t.Cleanup(func() { cfg.SetTemplatesDir(oldRoot) })
cfg.SetTemplatesDir(root)
return root
}
if ignore := ReadIgnoreFile(); len(ignore.Tags) != 0 || len(ignore.Files) != 0 {
func TestReadIgnoreFileReportsMissingActiveFile(t *testing.T) {
root := setActiveIgnoreFileRoot(t)
path := filepath.Join(root, NucleiIgnoreFileName)
ignore, err := ReadIgnoreFile()
if !errors.Is(err, os.ErrNotExist) {
t.Fatalf("read missing ignore file error = %v, want os.ErrNotExist", err)
}
if len(ignore.Tags) != 0 || len(ignore.Files) != 0 {
t.Fatalf("missing ignore file = %+v, want empty", ignore)
}
if message := err.Error(); !strings.Contains(message, strconv.Quote(path)) || !strings.Contains(message, "error opening") {
t.Fatalf("missing ignore file error = %q, want open failure and active path", message)
}
}
if err := os.WriteFile(filepath.Join(root, NucleiIgnoreFileName), []byte("tags: ["), 0o600); err != nil {
func TestReadIgnoreFileRejectsCorruptActiveFile(t *testing.T) {
root := setActiveIgnoreFileRoot(t)
path := filepath.Join(root, NucleiIgnoreFileName)
if err := os.WriteFile(path, []byte("tags: ["), 0o600); err != nil {
t.Fatalf("write invalid ignore file: %v", err)
}
if ignore := ReadIgnoreFile(); len(ignore.Tags) != 0 || len(ignore.Files) != 0 {
t.Fatalf("invalid ignore file = %+v, want empty", ignore)
ignore, err := ReadIgnoreFile()
if err == nil {
t.Fatalf("read corrupt ignore file = %+v, want error", ignore)
}
if message := err.Error(); !strings.Contains(message, strconv.Quote(path)) || !strings.Contains(message, "error parsing") {
t.Fatalf("corrupt ignore file error = %q, want parse failure and active path", message)
}
}
func TestReadIgnoreFileRejectsMalformedTrailingDocument(t *testing.T) {
root := setActiveIgnoreFileRoot(t)
path := filepath.Join(root, NucleiIgnoreFileName)
if err := os.WriteFile(path, []byte("tags: [weak]\n---\ntags: ["), 0o600); err != nil {
t.Fatalf("write ignore file: %v", err)
}
ignore, err := ReadIgnoreFile()
if err == nil {
t.Fatalf("read ignore file = %+v, want trailing parse error", ignore)
}
if len(ignore.Tags) != 0 || len(ignore.Files) != 0 {
t.Fatalf("ignore file after trailing parse error = %+v, want empty", ignore)
}
if message := err.Error(); !strings.Contains(message, strconv.Quote(path)) || !strings.Contains(message, "error parsing") {
t.Fatalf("trailing parse error = %q, want parse failure and active path", message)
}
}
func TestReadIgnoreFileRejectsAdditionalDocument(t *testing.T) {
root := setActiveIgnoreFileRoot(t)
path := filepath.Join(root, NucleiIgnoreFileName)
if err := os.WriteFile(path, []byte("tags: [weak]\n---\nfiles: [blocked.yaml]\n"), 0o600); err != nil {
t.Fatalf("write ignore file: %v", err)
}
ignore, err := ReadIgnoreFile()
if err == nil {
t.Fatalf("read ignore file = %+v, want multiple-document error", ignore)
}
if len(ignore.Tags) != 0 || len(ignore.Files) != 0 {
t.Fatalf("ignore file with additional document = %+v, want empty", ignore)
}
if message := err.Error(); !strings.Contains(message, strconv.Quote(path)) || !strings.Contains(message, "multiple YAML documents") {
t.Fatalf("multiple-document error = %q, want active path and document count failure", message)
}
}
func TestReadIgnoreFileParsesActiveFile(t *testing.T) {
root := setActiveIgnoreFileRoot(t)
if err := os.WriteFile(filepath.Join(root, NucleiIgnoreFileName), []byte("tags: [weak]\nfiles: [blocked.yaml]\n"), 0o600); err != nil {
t.Fatalf("write ignore file: %v", err)
}
ignore := ReadIgnoreFile()
ignore, err := ReadIgnoreFile()
if err != nil {
t.Fatalf("read ignore file: %v", err)
}
if len(ignore.Tags) != 1 || ignore.Tags[0] != "weak" {
t.Fatalf("ignore tags = %v, want [weak]", ignore.Tags)
}
@@ -36,6 +109,77 @@ func TestReadIgnoreFileAllowsMissingOrInvalidActiveFile(t *testing.T) {
}
}
func TestWriteActiveIgnoreFileRejectsInvalidPayloads(t *testing.T) {
root := setActiveIgnoreFileRoot(t)
cfg := DefaultConfig
path := filepath.Join(root, NucleiIgnoreFileName)
valid := []byte("tags: [weak]\nfiles: [blocked.yaml]\n")
if err := os.WriteFile(path, valid, 0o600); err != nil {
t.Fatalf("seed ignore file: %v", err)
}
cases := map[string][]byte{
"proxy html": []byte("<!DOCTYPE HTML>\n<html>\n<head>\n<meta name=\"description\" content=\"Zscaler: blocked\">\n</head>\n</html>\n"),
"unrelated yaml": []byte("error: access denied\nmessage: blocked by proxy\n"),
"empty": []byte(" \n"),
"empty mapping": []byte("{}\n"),
"null document": []byte("null\n"),
"template yaml": []byte("id: example\ninfo:\n name: example\n"),
"additional document": []byte("tags: [weak]\n---\nfiles: [blocked.yaml]\n"),
"malformed yaml": []byte("tags: ["),
"oversized": bytes.Repeat([]byte("a"), maxIgnoreFileBytes+1),
}
for name, payload := range cases {
t.Run(name, func(t *testing.T) {
err := cfg.WriteActiveIgnoreFile(payload)
if err == nil {
t.Fatal("write invalid ignore file succeeded")
}
if message := err.Error(); !strings.Contains(message, strconv.Quote(path)) {
t.Fatalf("invalid write error = %q, want active path", message)
}
got, readErr := os.ReadFile(path)
if readErr != nil {
t.Fatalf("read ignore file: %v", readErr)
}
if !bytes.Equal(got, valid) {
t.Fatalf("invalid payload overwrote ignore file: got %q", got)
}
})
}
}
func TestWriteActiveIgnoreFileAcceptsKnownShapes(t *testing.T) {
root := setActiveIgnoreFileRoot(t)
cfg := DefaultConfig
path := filepath.Join(root, NucleiIgnoreFileName)
cases := map[string][]byte{
"tags and files": []byte("tags: [weak]\nfiles: [blocked.yaml]\n"),
"tags only": []byte("tags: []\n"),
"files only": []byte("files: [blocked.yaml]\n"),
}
for name, payload := range cases {
t.Run(name, func(t *testing.T) {
if err := cfg.WriteActiveIgnoreFile(payload); err != nil {
t.Fatalf("write ignore file: %v", err)
}
got, err := os.ReadFile(path)
if err != nil {
t.Fatalf("read ignore file: %v", err)
}
if !bytes.Equal(got, payload) {
t.Fatalf("ignore file = %q, want %q", got, payload)
}
if _, err := ReadIgnoreFile(); err != nil {
t.Fatalf("read back written ignore file: %v", err)
}
})
}
}
func TestSetTemplatesDirDoesNotCopyLegacyIgnoreFile(t *testing.T) {
dir := t.TempDir()
configDir := filepath.Join(dir, "config")
+1 -1
View File
@@ -315,6 +315,7 @@ func (c *Config) WriteTemplatesIndex(index map[string]string) error {
indexFile := c.GetTemplateIndexFilePath()
var buff bytes.Buffer
for k, v := range index {
_, _ = buff.WriteString(k + "," + v + "\n")
}
@@ -329,7 +330,6 @@ func (c *Config) createConfigDirIfNotExists() error {
return errkit.Newf("could not create nuclei config directory at %s: %v", c.configDir, err)
}
}
return nil
}
@@ -0,0 +1,9 @@
//go:build !windows
package config
import "os"
func replaceTemplatesConfigFile(temporaryPath, configFilePath string) error {
return os.Rename(temporaryPath, configFilePath)
}
@@ -0,0 +1,48 @@
package config
import (
"errors"
"os"
"unsafe"
"golang.org/x/sys/windows"
)
var replaceFile = windows.NewLazySystemDLL("kernel32.dll").NewProc("ReplaceFileW")
func replaceTemplatesConfigFile(temporaryPath, configFilePath string) error {
temporary, err := windows.UTF16PtrFromString(temporaryPath)
if err != nil {
return err
}
target, err := windows.UTF16PtrFromString(configFilePath)
if err != nil {
return err
}
if _, err := os.Lstat(configFilePath); err != nil {
if !os.IsNotExist(err) {
return err
}
return windows.MoveFileEx(temporary, target, windows.MOVEFILE_WRITE_THROUGH)
}
replaced, _, callErr := replaceFile.Call(
uintptr(unsafe.Pointer(target)),
uintptr(unsafe.Pointer(temporary)),
0,
0,
0,
0,
)
if replaced != 0 {
return nil
}
if errors.Is(callErr, windows.ERROR_FILE_NOT_FOUND) {
return windows.MoveFileEx(temporary, target, windows.MOVEFILE_WRITE_THROUGH)
}
return callErr
}
@@ -0,0 +1,40 @@
package config
import (
"os"
"path/filepath"
"testing"
)
func TestReplaceTemplatesConfigFile(t *testing.T) {
for _, targetExists := range []bool{false, true} {
name := "creates"
if targetExists {
name = "replaces"
}
t.Run(name, func(t *testing.T) {
directory := t.TempDir()
target := filepath.Join(directory, "config.json")
if targetExists {
if err := os.WriteFile(target, []byte("previous"), 0o600); err != nil {
t.Fatal(err)
}
}
temporary := filepath.Join(directory, "config.tmp")
if err := os.WriteFile(temporary, []byte("current"), 0o600); err != nil {
t.Fatal(err)
}
if err := replaceTemplatesConfigFile(temporary, target); err != nil {
t.Fatal(err)
}
contents, err := os.ReadFile(target)
if err != nil {
t.Fatal(err)
}
if string(contents) != "current" {
t.Fatalf("expected replacement contents, got %q", contents)
}
})
}
}
+1 -1
View File
@@ -134,7 +134,7 @@ func atomicWriteFile(path string, data []byte, mode os.FileMode) error {
temp = nil
if err := os.Rename(tempPath, path); err != nil {
if err := replaceTemplatesConfigFile(tempPath, path); err != nil {
return fmt.Errorf("replace %q: %w", path, err)
}
+8 -1
View File
@@ -823,7 +823,14 @@ func (store *Store) LoadTemplatesWithTags(templatesList, tags []string) ([]*temp
// tags, so it must be matched against the ignore-file tags specifically:
// otherwise user-requested -exclude-tags drops would be mislabeled as
// .nuclei-ignore exclusions.
ignoreFileTags := config.ReadIgnoreFile().Tags
ignoreFile, err := config.ReadIgnoreFile()
if errors.Is(err, os.ErrNotExist) {
store.logger.Warning().Msgf("Could not read active .nuclei-ignore file: %s; continuing without ignore exclusions", err)
} else if err != nil {
return nil, err
}
ignoreFileTags := ignoreFile.Tags
noteExcludedByTag := func(templatePath string, metadata *index.Metadata) {
if len(ignoreFileTags) == 0 || !slices.ContainsFunc(ignoreFileTags, metadata.HasTag) {
return
+32
View File
@@ -5,6 +5,7 @@ import (
"os"
"path/filepath"
"reflect"
"strconv"
"testing"
"time"
@@ -60,6 +61,37 @@ func TestLoadTemplates(t *testing.T) {
})
}
func TestLoadTemplatesWithTagsRejectsCorruptIgnoreFileInValidationMode(t *testing.T) {
root := t.TempDir()
ignoreFilePath := filepath.Join(root, config.NucleiIgnoreFileName)
require.NoError(t, os.WriteFile(ignoreFilePath, []byte("tags: ["), 0o600))
cfg := config.DefaultConfig
oldRoot := cfg.TemplatesDirectory
t.Cleanup(func() { cfg.SetTemplatesDir(oldRoot) })
cfg.SetTemplatesDir(root)
options := testutils.DefaultOptions.Copy()
options.Validate = true
metadataIndex, err := metadataindex.NewIndex(t.TempDir())
require.NoError(t, err)
store, err := New(&Config{
Catalog: disk.NewCatalog(""),
ExecutorOptions: &protocols.ExecutorOptions{
Options: options,
Parser: templates.NewParser(),
},
Logger: options.Logger,
MetadataIndex: metadataIndex,
})
require.NoError(t, err)
_, err = store.LoadTemplatesWithTags(nil, nil)
require.ErrorContains(t, err, "error parsing")
require.ErrorContains(t, err, strconv.Quote(ignoreFilePath))
}
func TestNewUsesConfiguredMetadataIndex(t *testing.T) {
indexDir := t.TempDir()
metadataIndex, err := metadataindex.NewIndex(indexDir)
+54 -6
View File
@@ -4,9 +4,33 @@ import (
"github.com/projectdiscovery/gologger"
"github.com/projectdiscovery/nuclei/v3/pkg/output"
"github.com/projectdiscovery/nuclei/v3/pkg/protocols"
"github.com/projectdiscovery/nuclei/v3/pkg/templates"
"github.com/projectdiscovery/nuclei/v3/pkg/types"
)
// TemplateExecutionState identifies a template execution lifecycle transition.
type TemplateExecutionState uint8
const (
// TemplateExecutionStarted is emitted immediately before a template starts on a target.
TemplateExecutionStarted TemplateExecutionState = iota + 1
// TemplateExecutionFinished is emitted after that template execution returns.
TemplateExecutionFinished
)
// TemplateExecutionEvent identifies one template execution on one target.
type TemplateExecutionEvent struct {
TemplateID string
TemplatePath string
Target string
State TemplateExecutionState
ContextErr error
}
// TemplateExecutionCallback observes template execution lifecycle transitions.
// Nuclei can invoke the callback concurrently; implementations must be concurrency-safe.
type TemplateExecutionCallback func(TemplateExecutionEvent)
// Engine is an executer for running Nuclei Templates/Workflows.
//
// The engine contains multiple thread pools which allow using different
@@ -16,11 +40,12 @@ import (
// templates to leading to the final execution by the work pool, it is
// handled by the engine.
type Engine struct {
workPool *WorkPool
options *types.Options
executerOpts *protocols.ExecutorOptions
Callback func(*output.ResultEvent) // Executed on results
Logger *gologger.Logger
workPool *WorkPool
options *types.Options
executerOpts *protocols.ExecutorOptions
Callback func(*output.ResultEvent) // Executed on results
templateExecutionCallback TemplateExecutionCallback
Logger *gologger.Logger
}
// New returns a new Engine instance
@@ -36,7 +61,7 @@ func New(options *types.Options) *Engine {
func (e *Engine) GetWorkPoolConfig() WorkPoolConfig {
config := WorkPoolConfig{
InputConcurrency: e.options.BulkSize,
TypeConcurrency: e.options.TemplateThreads,
TypeConcurrency: e.options.CurrentTemplateThreads(),
HeadlessInputConcurrency: e.options.HeadlessBulkSize,
HeadlessTypeConcurrency: e.options.HeadlessTemplateThreads,
}
@@ -54,6 +79,29 @@ func (e *Engine) SetExecuterOptions(options *protocols.ExecutorOptions) {
e.executerOpts = options
}
// SetTemplateExecutionCallback registers a callback for template execution lifecycle events.
func (e *Engine) SetTemplateExecutionCallback(callback TemplateExecutionCallback) {
e.templateExecutionCallback = callback
}
func (e *Engine) templateExecutionStarted(template *templates.Template, target string) func(error) {
if e.templateExecutionCallback == nil {
return func(error) {}
}
event := TemplateExecutionEvent{
TemplateID: template.ID,
TemplatePath: template.Path,
Target: target,
State: TemplateExecutionStarted,
}
e.templateExecutionCallback(event)
return func(contextErr error) {
event.State = TemplateExecutionFinished
event.ContextErr = contextErr
e.templateExecutionCallback(event)
}
}
// ExecuterOptions returns protocols.ExecutorOptions for nuclei engine.
func (e *Engine) ExecuterOptions() *protocols.ExecutorOptions {
return e.executerOpts
+15 -3
View File
@@ -132,14 +132,26 @@ func (e *Engine) executeTemplateSpray(ctx context.Context, templatesList []*temp
wg = wp.Default
}
wg.Add()
go func(tpl *templates.Template) {
if err := wg.AddWithContext(ctx); err != nil {
return results
}
usesSharedTemplateBudget := templateType != types.HeadlessProtocol
if usesSharedTemplateBudget {
if err := e.options.AcquireTemplateThread(ctx); err != nil {
wg.Done()
return results
}
}
go func(tpl *templates.Template, sharedBudget bool) {
defer wg.Done()
if sharedBudget {
defer e.options.ReleaseTemplateThread()
}
// All other request types are executed here
// Note: executeTemplateWithTargets creates goroutines and blocks
// given template is executed on all targets
e.executeTemplateWithTargets(ctx, tpl, target, results)
}(template)
}(template, usesSharedTemplateBudget)
}
return results
}
+30 -5
View File
@@ -21,9 +21,20 @@ import (
// executeAllSelfContained executes all self contained templates that do not use `target`
func (e *Engine) executeAllSelfContained(ctx context.Context, alltemplates []*templates.Template, results *atomic.Bool, sg *sync.WaitGroup) {
for _, v := range alltemplates {
usesSharedTemplateBudget := v.Type() != types.HeadlessProtocol
if usesSharedTemplateBudget {
if err := e.options.AcquireTemplateThread(ctx); err != nil {
return
}
}
sg.Add(1)
go func(template *templates.Template) {
go func(template *templates.Template, sharedBudget bool) {
defer sg.Done()
if sharedBudget {
defer e.options.ReleaseTemplateThread()
}
finished := e.templateExecutionStarted(template, "")
defer func() { finished(ctx.Err()) }()
var err error
var match bool
ctx := scan.NewScanContext(ctx, contextargs.New(ctx))
@@ -42,7 +53,7 @@ func (e *Engine) executeAllSelfContained(ctx context.Context, alltemplates []*te
e.options.Logger.Warning().Msgf("[%s] Could not execute step (self-contained): %s\n", e.executerOpts.Colorizer.BrightBlue(template.ID), err)
}
results.CompareAndSwap(false, match)
}(v)
}(v, usesSharedTemplateBudget)
}
}
@@ -230,21 +241,35 @@ func (e *Engine) executeTemplatesOnTarget(ctx context.Context, alltemplates []*t
} else {
sg = wp.Default
}
sg.Add()
go func(template *templates.Template, value *contextargs.MetaInput, wg *syncutil.AdaptiveWaitGroup) {
if err := sg.AddWithContext(ctx); err != nil {
return
}
usesSharedTemplateBudget := tpl.Type() != types.HeadlessProtocol
if usesSharedTemplateBudget {
if err := e.options.AcquireTemplateThread(ctx); err != nil {
sg.Done()
return
}
}
go func(template *templates.Template, value *contextargs.MetaInput, wg *syncutil.AdaptiveWaitGroup, sharedBudget bool) {
defer wg.Done()
if sharedBudget {
defer e.options.ReleaseTemplateThread()
}
match, err := e.executeTemplateOnInput(ctx, template, value)
if err != nil {
e.options.Logger.Warning().Msgf("[%s] Could not execute step on %s: %s\n", template.ID, value.Input, err)
}
results.CompareAndSwap(false, match)
}(tpl, target, sg)
}(tpl, target, sg, usesSharedTemplateBudget)
}
}
// executeTemplateOnInput performs template execution for a single input and returns match status and error
func (e *Engine) executeTemplateOnInput(ctx context.Context, template *templates.Template, value *contextargs.MetaInput) (bool, error) {
finished := e.templateExecutionStarted(template, value.Input)
defer func() { finished(ctx.Err()) }()
ctxArgs := contextargs.New(ctx)
ctxArgs.MetaInput = value
scanCtx := scan.NewScanContext(ctx, ctxArgs)
+224
View File
@@ -3,6 +3,8 @@ package core
import (
"context"
"fmt"
"reflect"
"sync"
"sync/atomic"
"testing"
"time"
@@ -95,6 +97,49 @@ func Test_executeTemplateOnInput_CallbackErrorPropagates(t *testing.T) {
}
}
func TestExecuteTemplateOnInputReportsLifecycle(t *testing.T) {
e := newTestEngine()
var events []TemplateExecutionEvent
e.SetTemplateExecutionCallback(func(event TemplateExecutionEvent) {
events = append(events, event)
})
tpl := &templates.Template{ID: "template-id", Path: "http/example.yaml"}
tpl.Executer = &fakeExecuter{withResults: false}
_, err := e.executeTemplateOnInput(context.Background(), tpl, &contextargs.MetaInput{Input: "https://example.com"})
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
want := []TemplateExecutionEvent{
{TemplateID: "template-id", TemplatePath: "http/example.yaml", Target: "https://example.com", State: TemplateExecutionStarted},
{TemplateID: "template-id", TemplatePath: "http/example.yaml", Target: "https://example.com", State: TemplateExecutionFinished},
}
if !reflect.DeepEqual(events, want) {
t.Fatalf("unexpected lifecycle events: got %#v want %#v", events, want)
}
}
func TestExecuteTemplateOnInputReportsCancellationOnFinish(t *testing.T) {
e := newTestEngine()
var finished TemplateExecutionEvent
e.SetTemplateExecutionCallback(func(event TemplateExecutionEvent) {
if event.State == TemplateExecutionFinished {
finished = event
}
})
tpl := &templates.Template{ID: "interrupted"}
tpl.Executer = &fakeExecuter{withResults: false}
ctx, cancel := context.WithCancel(context.Background())
cancel()
_, _ = e.executeTemplateOnInput(ctx, tpl, &contextargs.MetaInput{Input: "target"})
if finished.ContextErr != context.Canceled {
t.Fatalf("got finish context error %v, want context canceled", finished.ContextErr)
}
}
type fakeTargetProvider struct {
values []*contextargs.MetaInput
}
@@ -127,6 +172,185 @@ func (s *slowExecuter) Execute(ctx *scan.ScanContext) (bool, error) {
return true, nil
}
}
type adaptiveConcurrencyExecuter struct {
delay time.Duration
started atomic.Int32
completed atomic.Int32
inFlight atomic.Int32
maxInFlight atomic.Int32
onStart func(int32)
}
func (s *adaptiveConcurrencyExecuter) Compile() error { return nil }
func (s *adaptiveConcurrencyExecuter) Requests() int { return 1 }
func (s *adaptiveConcurrencyExecuter) Execute(ctx *scan.ScanContext) (bool, error) {
started := s.started.Add(1)
if s.onStart != nil {
s.onStart(started)
}
inFlight := s.inFlight.Add(1)
for {
maximum := s.maxInFlight.Load()
if inFlight <= maximum || s.maxInFlight.CompareAndSwap(maximum, inFlight) {
break
}
}
defer s.inFlight.Add(-1)
select {
case <-ctx.Context().Done():
return false, ctx.Context().Err()
case <-time.After(s.delay):
s.completed.Add(1)
return true, nil
}
}
func (s *adaptiveConcurrencyExecuter) ExecuteWithResults(*scan.ScanContext) ([]*output.ResultEvent, error) {
return nil, nil
}
func TestExecuteTemplateSprayRefreshesDynamicTemplateConcurrency(t *testing.T) {
const (
templateCount = 8
initialConcurrency = 2
finalConcurrency = 8
)
var currentConcurrency atomic.Int32
currentConcurrency.Store(initialConcurrency)
var expandOnce sync.Once
executer := &adaptiveConcurrencyExecuter{
delay: 100 * time.Millisecond,
onStart: func(started int32) {
if started == initialConcurrency {
expandOnce.Do(func() { currentConcurrency.Store(finalConcurrency) })
}
},
}
options := &types.Options{
BulkSize: 1,
TemplateThreads: initialConcurrency,
HeadlessBulkSize: 1,
HeadlessTemplateThreads: 1,
}
options.SetTemplateThreadsProvider(func() int { return int(currentConcurrency.Load()) })
if got := options.Copy().CurrentTemplateThreads(); got != initialConcurrency {
t.Fatalf("copied options returned concurrency %d, expected %d", got, initialConcurrency)
}
engine := New(options)
engine.SetExecuterOptions(&protocols.ExecutorOptions{
Logger: engine.Logger,
Options: options,
ResumeCfg: types.NewResumeCfg(),
ProtocolType: tmpltypes.NetworkProtocol,
})
templatesList := make([]*templates.Template, 0, templateCount)
for index := range templateCount {
templatesList = append(templatesList, &templates.Template{ID: fmt.Sprintf("adaptive-%d", index), Executer: executer})
}
targets := &fakeTargetProvider{values: []*contextargs.MetaInput{{Input: "slow-target"}}}
started := time.Now()
engine.ExecuteScanWithOpts(context.Background(), templatesList, targets, true)
elapsed := time.Since(started)
if got := executer.completed.Load(); got != templateCount {
t.Fatalf("completed %d templates, expected %d", got, templateCount)
}
if got := executer.maxInFlight.Load(); got <= initialConcurrency {
t.Fatalf("maximum concurrency stayed at %d; expected it to expand beyond %d", got, initialConcurrency)
}
if elapsed >= 350*time.Millisecond {
t.Fatalf("dynamic execution took %s; fixed concurrency would take about 400ms", elapsed)
}
t.Logf("dynamic execution completed all %d templates in %s with peak concurrency %d", templateCount, elapsed, executer.maxInFlight.Load())
}
func TestExecuteTemplateSprayHonorsSharedConcurrencyLimiter(t *testing.T) {
const (
templateCount = 8
sharedBudget = 2
)
tokens := make(chan struct{}, sharedBudget)
executer := &adaptiveConcurrencyExecuter{delay: 50 * time.Millisecond}
options := &types.Options{
BulkSize: 1,
TemplateThreads: templateCount,
HeadlessBulkSize: 1,
HeadlessTemplateThreads: 1,
}
options.SetTemplateThreadsLimiter(func(ctx context.Context) error {
select {
case tokens <- struct{}{}:
return nil
case <-ctx.Done():
return ctx.Err()
}
}, func() {
<-tokens
})
engine := New(options)
engine.SetExecuterOptions(&protocols.ExecutorOptions{
Logger: engine.Logger,
Options: options,
ResumeCfg: types.NewResumeCfg(),
ProtocolType: tmpltypes.NetworkProtocol,
})
templatesList := make([]*templates.Template, 0, templateCount)
for index := range templateCount {
templatesList = append(templatesList, &templates.Template{ID: fmt.Sprintf("shared-limit-%d", index), Executer: executer})
}
engine.ExecuteScanWithOpts(context.Background(), templatesList,
&fakeTargetProvider{values: []*contextargs.MetaInput{{Input: "slow-target"}}}, true)
if got := executer.completed.Load(); got != templateCount {
t.Fatalf("completed %d templates, expected %d", got, templateCount)
}
if got := executer.maxInFlight.Load(); got > sharedBudget {
t.Fatalf("maximum concurrency was %d, shared budget is %d", got, sharedBudget)
}
}
func TestExecuteAllSelfContainedHonorsSharedConcurrencyLimiter(t *testing.T) {
const (
templateCount = 8
sharedBudget = 2
)
tokens := make(chan struct{}, sharedBudget)
executer := &adaptiveConcurrencyExecuter{delay: 20 * time.Millisecond}
options := &types.Options{}
options.SetTemplateThreadsLimiter(func(ctx context.Context) error {
select {
case tokens <- struct{}{}:
return nil
case <-ctx.Done():
return ctx.Err()
}
}, func() {
<-tokens
})
engine := New(options)
templatesList := make([]*templates.Template, 0, templateCount)
for index := range templateCount {
templatesList = append(templatesList, &templates.Template{ID: fmt.Sprintf("self-contained-%d", index), Executer: executer})
}
var results atomic.Bool
var waitGroup sync.WaitGroup
engine.executeAllSelfContained(context.Background(), templatesList, &results, &waitGroup)
waitGroup.Wait()
if got := executer.completed.Load(); got != templateCount {
t.Fatalf("completed %d templates, expected %d", got, templateCount)
}
if got := executer.maxInFlight.Load(); got > sharedBudget {
t.Fatalf("maximum concurrency was %d, shared budget is %d", got, sharedBudget)
}
}
func (s *slowExecuter) ExecuteWithResults(ctx *scan.ScanContext) ([]*output.ResultEvent, error) {
return nil, nil
}
+431 -240
View File
@@ -4,15 +4,19 @@ import (
"bytes"
"context"
"crypto/md5"
"crypto/rand"
"errors"
"fmt"
"io"
"io/fs"
"net/http"
"os"
"path/filepath"
"strconv"
"strings"
"github.com/charmbracelet/glamour"
"github.com/google/go-github/v30/github"
"github.com/olekukonko/tablewriter"
"github.com/projectdiscovery/gologger"
"github.com/projectdiscovery/nuclei/v3/pkg/catalog/config"
@@ -20,13 +24,13 @@ import (
filepathutil "github.com/projectdiscovery/nuclei/v3/pkg/utils/filepath"
"github.com/projectdiscovery/utils/errkit"
fileutil "github.com/projectdiscovery/utils/file"
mapsutil "github.com/projectdiscovery/utils/maps"
stringsutil "github.com/projectdiscovery/utils/strings"
updateutils "github.com/projectdiscovery/utils/update"
)
const (
checkSumFilePerm = 0644
checkSumFilePerm = 0644
templateOutputTemporaryPrefix = ".nuclei-write-"
)
var (
@@ -104,6 +108,10 @@ func (t *TemplateManager) updateIfOutdatedLocked() error {
return t.FreshInstallIfNotExists()
}
if err := recoverTemplateOwnership(config.DefaultConfig.TemplatesDirectory); err != nil {
return errkit.Wrapf(err, "failed to recover template ownership at %s", config.DefaultConfig.TemplatesDirectory)
}
needsUpdate := config.DefaultConfig.NeedsTemplateUpdate()
// NOTE(dwisiswant0): if PDTM API data is not available
@@ -139,15 +147,19 @@ func (t *TemplateManager) installTemplatesAt(dir string) error {
return nil
}
if err := recoverTemplateOwnership(dir); err != nil {
return errkit.Wrapf(err, "failed to recover template ownership at %s", dir)
}
ghrd, err := updateutils.NewghReleaseDownloader(config.OfficialNucleiTemplatesRepoName)
if err != nil {
return errkit.Wrapf(err, "failed to install templates at %s", dir)
}
// write templates to disk
_, err = t.writeTemplatesToDisk(ghrd, dir)
if err != nil {
return errkit.Wrapf(err, "failed to write templates to disk at %s", dir)
writtenOutputs, writeErr := t.writeTemplatesToDisk(ghrd, dir)
if _, finalizeErr := t.finalizeTemplateWrite(config.DefaultConfig, writtenOutputs, ghrd.Latest.GetTagName(), writeErr); finalizeErr != nil {
return errkit.Wrapf(finalizeErr, "failed to finalize template installation at %s", dir)
}
gologger.Info().Msgf("Successfully installed nuclei-templates at %s", dir)
@@ -155,13 +167,204 @@ func (t *TemplateManager) installTemplatesAt(dir string) error {
return nil
}
type templateArchiveFetcher func(version string) (*bytes.Reader, error)
func commitTemplateVersion(cfg *config.Config, version string) error {
previousVersion := cfg.TemplateVersion
if err := cfg.SetTemplatesVersion(version); err != nil {
cfg.TemplateVersion = previousVersion
return err
}
return nil
}
func (t *TemplateManager) finalizeTemplateRelease(cfg *config.Config, writtenOutputs map[string]string, version string) (map[string]string, error) {
dir := cfg.TemplatesDirectory
var finalizationErr error
if err := reconcileTemplateOwnership(dir, writtenOutputs); err != nil {
finalizationErr = errors.Join(finalizationErr, fmt.Errorf("reconcile template ownership: %w", err))
}
checksums, err := t.regenerateTemplateMetadata(cfg)
if err != nil {
finalizationErr = errors.Join(finalizationErr, fmt.Errorf("regenerate template metadata: %w", err))
}
if finalizationErr != nil {
return nil, finalizationErr
}
if err := commitTemplateVersion(cfg, version); err != nil {
return nil, errkit.Wrap(err, "failed to update templates version")
}
return checksums, nil
}
func (t *TemplateManager) finalizeTemplateWrite(cfg *config.Config, writtenOutputs map[string]string, version string, writeErr error) (map[string]string, error) {
if writeErr != nil {
if ownershipErr := recordPartialTemplateOwnership(cfg.TemplatesDirectory, writtenOutputs); ownershipErr != nil {
writeErr = errors.Join(writeErr, fmt.Errorf("record ownership for partially written templates: %w", ownershipErr))
}
return nil, writeErr
}
return t.finalizeTemplateRelease(cfg, writtenOutputs, version)
}
func (t *TemplateManager) bootstrapTemplateOwnership(dir, version string, fetchArchive templateArchiveFetcher) error {
if _, err := loadTemplateOwnership(dir); err == nil {
return nil
} else if errors.Is(err, errTemplateOwnershipInvalid) {
quarantines, recoveryErr := listTemplateOwnershipQuarantines(dir)
if recoveryErr != nil {
return errors.Join(err, recoveryErr)
}
restoreStates, recoveryErr := listTemplateOwnershipRestoreStates(dir)
if recoveryErr != nil {
return errors.Join(err, recoveryErr)
}
if len(quarantines) > 0 || len(restoreStates) > 0 {
return err
}
gologger.Verbose().Msgf("Replacing invalid template ownership metadata without retiring prior templates: %s", err)
return nil
} else if !errors.Is(err, errTemplateOwnershipMissing) {
return err
}
version = strings.TrimSpace(version)
if version == "" {
return nil
}
archive, err := fetchArchive(version)
if err != nil {
gologger.Verbose().Msgf("Continuing template update without prior ownership for %q: %s", version, err)
return nil
}
if err := t.bootstrapTemplateOwnershipFromArchive(dir, archive); err != nil {
if errors.Is(err, errTemplateOwnershipArchiveInvalid) {
gologger.Verbose().Msgf("Continuing template update without prior ownership for %q: %s", version, err)
return nil
}
return fmt.Errorf("build ownership from prior template release %q: %w", version, err)
}
return nil
}
func fetchTemplateReleaseArchive(version string) (*bytes.Reader, error) {
ctx := context.Background()
httpClient := &http.Client{Timeout: updateutils.DownloadUpdateTimeout}
client := github.NewClient(httpClient)
archiveURL, _, err := client.Repositories.GetArchiveLink(
ctx,
updateutils.Organization,
config.OfficialNucleiTemplatesRepoName,
github.Zipball,
&github.RepositoryContentGetOptions{Ref: version},
true,
)
if err != nil {
return nil, fmt.Errorf("resolve prior template release %q archive: %w", version, err)
}
request, err := http.NewRequestWithContext(ctx, http.MethodGet, archiveURL.String(), nil)
if err != nil {
return nil, fmt.Errorf("create prior template release %q archive request: %w", version, err)
}
var contents bytes.Buffer
response, err := client.Do(ctx, request, &contents)
if err != nil {
return nil, fmt.Errorf("download prior template release %q: %w", version, err)
}
if response.StatusCode != http.StatusOK {
return nil, fmt.Errorf("download prior template release %q: unexpected HTTP status %s", version, response.Status)
}
return bytes.NewReader(contents.Bytes()), nil
}
func (t *TemplateManager) bootstrapTemplateOwnershipFromArchive(dir string, archive *bytes.Reader) error {
absDir, err := filepath.Abs(dir)
if err != nil {
return fmt.Errorf("get absolute templates directory: %w", err)
}
manifest := &templateOwnershipManifest{
Version: templateOwnershipVersion,
Files: make(map[string]string),
}
callback := func(uri string, fileInfo fs.FileInfo, reader io.Reader) error {
if fileInfo.IsDir() {
return nil
}
_, writePath := t.getTemplateOutputLocation(absDir, uri, fileInfo)
if writePath == "" {
return nil
}
if !config.IsTemplate(writePath) {
return nil
}
relativePath, err := filepath.Rel(absDir, writePath)
if err != nil {
return fmt.Errorf("make prior template path %q relative to %q: %w", writePath, absDir, err)
}
relativePath = filepath.ToSlash(relativePath)
if !config.IsTemplate(relativePath) {
return nil
}
if err := validateTemplateOwnershipPath(relativePath); err != nil {
return err
}
contents, err := io.ReadAll(reader)
if err != nil {
return fmt.Errorf("read prior template %q: %w", uri, err)
}
manifest.Files[relativePath] = templateDigest(contents)
return nil
}
if err := updateutils.UnpackAssetWithCallback(updateutils.Zip, archive, callback); err != nil {
return fmt.Errorf("%w: unpack prior template release: %v", errTemplateOwnershipArchiveInvalid, err)
}
return writeTemplateOwnership(absDir, manifest)
}
// updateTemplatesAt updates templates at given directory
func (t *TemplateManager) updateTemplatesAt(dir string) error {
if t.DisablePublicTemplates {
gologger.Info().Msgf("Skipping update of public nuclei-templates")
return nil
}
if err := t.bootstrapTemplateOwnership(dir, config.DefaultConfig.TemplateVersion, fetchTemplateReleaseArchive); err != nil {
return errkit.Wrapf(err, "failed to migrate template ownership at %s", dir)
}
// firstly, read checksums from .checksum file these are used to generate stats
oldchecksums, err := t.getChecksumFromDir(dir)
if err != nil {
@@ -184,49 +387,15 @@ func (t *TemplateManager) updateTemplatesAt(dir string) error {
}
// write templates to disk
writtenPaths, err := t.writeTemplatesToDisk(ghrd, dir)
if err != nil {
return err
}
// cleanup orphaned templates that exist locally but weren't in the new release
if err := t.cleanupOrphanedTemplates(dir, writtenPaths); err != nil {
// log warning but don't fail the update
gologger.Warning().Msgf("failed to cleanup orphaned templates: %s", err)
} else {
// Regenerate metadata (index and checksum) after successful cleanup to ensure
// metadata accurately reflects the cleaned template tree. This prevents stale
// index entries and checksum entries for deleted templates.
if err := t.regenerateTemplateMetadata(dir); err != nil {
// Log warning but don't fail the update - metadata will be out of sync
// but templates are cleaned up correctly
gologger.Warning().Msgf("failed to regenerate template metadata after cleanup: %s", err)
}
}
// get checksums from new templates
newchecksums, err := t.getChecksumFromDir(dir)
if err != nil {
// unlikely this case will happen
return errkit.Wrapf(err, "failed to get checksums from %s after update", dir)
writtenOutputs, writeErr := t.writeTemplatesToDisk(ghrd, dir)
newchecksums, finalizeErr := t.finalizeTemplateWrite(config.DefaultConfig, writtenOutputs, latestVersion, writeErr)
if finalizeErr != nil {
return errkit.Wrapf(finalizeErr, "failed to finalize template update at %s", dir)
}
// summarize all changes
results := t.summarizeChanges(oldchecksums, newchecksums)
// remove deleted templates
for _, deletion := range results.deletions {
// the deletion list comes from the on-disk .checksum file; only remove
// paths inside the templates directory.
if !filepathutil.IsPathWithinDirectory(deletion, dir) {
gologger.Warning().Msgf("skipping deletion of %s: path is outside templates directory %s", deletion, dir)
continue
}
if err := os.Remove(deletion); err != nil && !os.IsNotExist(err) {
gologger.Warning().Msgf("failed to remove deleted template %s: %s", deletion, err)
}
}
// print summary
if results.totalCount > 0 {
gologger.Info().Msgf("Successfully updated nuclei-templates (%v) to %s. GoodLuck!", ghrd.Latest.GetTagName(), dir)
@@ -238,6 +407,7 @@ func (t *TemplateManager) updateTemplatesAt(dir string) error {
} else {
gologger.Info().Msgf("Successfully updated nuclei-templates (%v) to %s. GoodLuck!", ghrd.Latest.GetTagName(), dir)
}
return nil
}
@@ -253,26 +423,30 @@ func (t *TemplateManager) summarizeChanges(old, new map[string]string) *template
results.additions = append(results.additions, k)
}
}
for k := range old {
if _, ok := new[k]; !ok {
results.deletions = append(results.deletions, k)
}
}
results.totalCount = len(results.additions) + len(results.deletions) + len(results.modifications)
return results
}
// getAbsoluteFilePath returns an absolute path where a file should be written based on given uri(i.e., files in zip)
// if a returned path is empty, it means that file should not be written and skipped
func (t *TemplateManager) getAbsoluteFilePath(templateDir, uri string, f fs.FileInfo) string {
// getTemplateOutputLocation returns the safe root and absolute output path for an archive entry.
// An empty output path means the entry should be skipped.
func (t *TemplateManager) getTemplateOutputLocation(templateDir, uri string, f fs.FileInfo) (string, string) {
// overwrite .nuclei-ignore every time nuclei-templates are downloaded
if f.Name() == config.NucleiIgnoreFileName {
return config.DefaultConfig.GetActiveIgnoreFilePath()
return config.DefaultConfig.TemplatesDirectory, config.DefaultConfig.GetActiveIgnoreFilePath()
}
// skip all meta files
if !strings.EqualFold(f.Name(), config.NewTemplateAdditionsFileName) {
if strings.TrimSpace(f.Name()) == "" || strings.HasPrefix(f.Name(), ".") || strings.EqualFold(f.Name(), "README.md") {
return ""
return "", ""
}
}
@@ -290,9 +464,9 @@ func (t *TemplateManager) getAbsoluteFilePath(templateDir, uri string, f fs.File
// to outside the configured templates directory.
fallbackPath := filepath.Clean(filepath.Join(templateDir, uri))
if !filepathutil.IsPathWithinDirectory(fallbackPath, templateDir) {
return ""
return "", ""
}
return fallbackPath
return templateDir, fallbackPath
}
// separator is also included in rootDir
rootDirectory := uri[:index+1]
@@ -300,47 +474,41 @@ func (t *TemplateManager) getAbsoluteFilePath(templateDir, uri string, f fs.File
// if it is a github meta directory skip it
if stringsutil.HasPrefixAny(relPath, ".github", ".git") {
return ""
return "", ""
}
newPath := filepath.Clean(filepath.Join(templateDir, relPath))
if !filepathutil.IsPathWithinDirectory(newPath, templateDir) || !filepathutil.IsPathWithinDirectory(filepath.Dir(newPath), templateDir) {
// we don't allow LFI
return ""
return "", ""
}
if filepath.Clean(newPath) == filepath.Clean(templateDir) {
// skip writing the folder itself since it already exists
return ""
return "", ""
}
if relPath != "" && f.IsDir() {
// if uri is a directory, create it
if err := fileutil.CreateFolder(newPath); err != nil {
gologger.Warning().Msgf("uri %v: got %s while installing templates", uri, err)
}
return ""
}
return newPath
return templateDir, newPath
}
// writeTemplatesToDisk writes all templates to disk and returns a map of written file paths
// The returned map contains absolute paths of all template files that were successfully written
func (t *TemplateManager) writeTemplatesToDisk(ghrd *updateutils.GHReleaseDownloader, dir string) (*mapsutil.SyncLockMap[string, struct{}], error) {
localTemplatesIndex, err := config.GetNucleiTemplatesIndex()
if err != nil {
gologger.Warning().Msgf("failed to get local nuclei-templates index: %s", err)
if localTemplatesIndex == nil {
localTemplatesIndex = map[string]string{} // no-op
}
}
func isActiveIgnoreFilePath(writePath string) bool {
return filepath.Clean(writePath) == filepath.Clean(config.DefaultConfig.GetActiveIgnoreFilePath())
}
// Track all paths that are successfully written during this update
writtenPaths := mapsutil.NewSyncLockMap[string, struct{}]()
// writeTemplatesToDisk writes release outputs to disk and returns their digests.
// The returned map includes every successfully written output; ownership
// reconciliation filters it to official template paths.
func (t *TemplateManager) writeTemplatesToDisk(ghrd *updateutils.GHReleaseDownloader, dir string) (map[string]string, error) {
writtenOutputs := make(map[string]string)
touchedDirectories := make(map[string]struct{})
callbackFunc := func(uri string, f fs.FileInfo, r io.Reader) error {
writePath := t.getAbsoluteFilePath(dir, uri, f)
if f.IsDir() {
return nil
}
rootDir, writePath := t.getTemplateOutputLocation(dir, uri, f)
if writePath == "" {
// skip writing file
return nil
@@ -351,64 +519,33 @@ func (t *TemplateManager) writeTemplatesToDisk(ghrd *updateutils.GHReleaseDownlo
// if error occurs, iteration also stops
return errkit.Wrapf(err, "failed to read file %s", uri)
}
// TODO: It might be better to just download index file from nuclei templates repo
// instead of creating it from scratch
id, _ := config.GetTemplateIDFromReader(bytes.NewReader(bin), uri)
if id != "" {
// based on template id, check if we are updating a path of official nuclei template
if oldPath, ok := localTemplatesIndex[id]; ok {
if oldPath != writePath {
// write new template at a new path and delete old template
if err := os.WriteFile(writePath, bin, f.Mode()); err != nil {
return errkit.Wrapf(err, "failed to write file %s", uri)
}
// Track the new path as written
_ = writtenPaths.Set(writePath, struct{}{})
// after successful write, remove old template. oldPath comes
// from the on-disk .templates-index; only remove paths inside
// the templates directory.
if !filepathutil.IsPathWithinDirectory(oldPath, dir) {
gologger.Warning().Msgf("skipping removal of old template %s: path is outside templates directory %s", oldPath, dir)
} else if err := os.Remove(oldPath); err != nil {
gologger.Warning().Msgf("failed to remove old template %s: %s", oldPath, err)
}
return nil
}
}
outputResult, outputErr := writeTemplateOutput(rootDir, writePath, bin, f.Mode())
for _, directory := range outputResult.touchedDirectories {
touchedDirectories[directory] = struct{}{}
}
// no change in template Path of official templates
if err := os.WriteFile(writePath, bin, f.Mode()); err != nil {
return errkit.Wrapf(err, "failed to write file %s", uri)
if outputErr != nil {
return errkit.Wrapf(outputErr, "failed to write file %s", uri)
}
// Track successfully written paths
_ = writtenPaths.Set(writePath, struct{}{})
writtenOutputs[writePath] = templateDigest(bin)
return nil
}
err = ghrd.DownloadSourceWithCallback(!HideProgressBar, callbackFunc)
if err != nil {
return nil, errkit.Wrap(err, "failed to download templates")
var writeErr error
if err := ghrd.DownloadSourceWithCallback(!HideProgressBar, callbackFunc); err != nil {
writeErr = errkit.Wrap(err, "failed to download templates")
}
if err := config.DefaultConfig.WriteTemplatesConfig(); err != nil {
return nil, errkit.Wrap(err, "failed to write templates config")
}
// update templates version in config file
if err := config.DefaultConfig.SetTemplatesVersion(ghrd.Latest.GetTagName()); err != nil {
return nil, errkit.Wrap(err, "failed to update templates version")
if err := syncTemplateOutputDirectories(touchedDirectories, syncTemplateOwnershipDirectory); err != nil {
writeErr = errors.Join(writeErr, errkit.Wrap(err, "failed to sync template output directories"))
}
PurgeEmptyDirectories(dir)
// generate index of all templates
_ = os.Remove(config.DefaultConfig.GetTemplateIndexFilePath())
index, err := config.GetNucleiTemplatesIndex()
if err != nil {
return nil, errkit.Wrap(err, "failed to get nuclei templates index")
}
if err = config.DefaultConfig.WriteTemplatesIndex(index); err != nil {
return nil, errkit.Wrap(err, "failed to write nuclei templates index")
if writeErr != nil {
return writtenOutputs, writeErr
}
if !HideReleaseNotes {
@@ -418,148 +555,162 @@ func (t *TemplateManager) writeTemplatesToDisk(ghrd *updateutils.GHReleaseDownlo
if err != nil {
gologger.Error().Msgf("markdown rendering not supported: %v", err)
}
if rendered, err := r.Render(output); err == nil {
output = rendered
} else {
gologger.Error().Msg(err.Error())
}
gologger.Print().Msgf("\n%v\n\n", output)
}
// after installation, create and write checksums to .checksum file
if err := t.writeChecksumFileInDir(dir); err != nil {
return nil, err
}
return writtenPaths, nil
return writtenOutputs, nil
}
// cleanupOrphanedTemplates removes template files that exist locally but were not part of the new release
// It scans the templates directory for template files and deletes those that are not in the writtenPaths set
// This function handles empty directories gracefully - if the directory is empty, no orphaned files will be found
func (t *TemplateManager) cleanupOrphanedTemplates(dir string, writtenPaths *mapsutil.SyncLockMap[string, struct{}]) error {
absDir, err := filepath.Abs(dir)
if err != nil {
return errkit.Wrapf(err, "failed to get absolute path of templates directory")
}
// Use Clean to normalize the path consistently (handles Windows paths better)
absDir = filepath.Clean(absDir)
func syncTemplateOutputDirectories(directories map[string]struct{}, syncDirectory func(*os.Root, string) error) error {
var syncErrors error
// If directory doesn't exist, there's nothing to clean up
if !fileutil.FolderExists(absDir) {
return nil
}
// Normalize all written paths to absolute paths for comparison
normalizedWrittenPaths := mapsutil.NewSyncLockMap[string, struct{}]()
for path := range writtenPaths.GetAll() {
absPath, err := filepath.Abs(path)
if err == nil {
// Use Clean to normalize the path consistently (handles Windows paths better)
absPath = filepath.Clean(absPath)
_ = normalizedWrittenPaths.Set(absPath, struct{}{})
}
}
// Get custom template directories to exclude
customDirs := config.DefaultConfig.GetAllCustomTemplateDirs()
customDirAbs := make([]string, 0, len(customDirs))
for _, customDir := range customDirs {
if absCustomDir, err := filepath.Abs(customDir); err == nil {
// Use Clean to normalize the path consistently (handles Windows paths better)
absCustomDir = filepath.Clean(absCustomDir)
customDirAbs = append(customDirAbs, absCustomDir)
}
}
var orphanedFiles []string
// Walk the templates directory to find all template files
err = filepath.WalkDir(absDir, func(path string, d fs.DirEntry, err error) error {
for directory := range directories {
root, err := os.OpenRoot(directory)
if err != nil {
// Log but continue walking
gologger.Debug().Msgf("error accessing path %s during orphan cleanup: %s", path, err)
return nil
syncErrors = errors.Join(syncErrors, fmt.Errorf("open output directory %q: %w", directory, err))
continue
}
// Skip directories
if d.IsDir() {
return nil
syncErr := syncDirectory(root, ".")
closeErr := root.Close()
if syncErr != nil {
syncErrors = errors.Join(syncErrors, fmt.Errorf("sync output directory %q: %w", directory, syncErr))
}
absPath, err := filepath.Abs(path)
if err != nil {
return nil
}
// Use Clean to normalize the path consistently (handles Windows paths better)
absPath = filepath.Clean(absPath)
// Skip custom template directories
if filepathutil.IsPathWithinAnyDirectory(absPath, customDirAbs...) {
return nil
}
// Only process template files
if !config.IsTemplate(absPath) {
return nil
}
// Skip if this file was written in the new release
if normalizedWrittenPaths.Has(absPath) {
return nil
}
// This is an orphaned template file
orphanedFiles = append(orphanedFiles, absPath)
return nil
})
if err != nil {
return errkit.Wrapf(err, "failed to walk templates directory for orphan cleanup")
}
// Delete orphaned files
for _, orphanPath := range orphanedFiles {
if err := os.Remove(orphanPath); err != nil {
if !os.IsNotExist(err) {
gologger.Warning().Msgf("failed to remove orphaned template %s: %s", orphanPath, err)
}
} else {
gologger.Debug().Msgf("removed orphaned template: %s", orphanPath)
if closeErr != nil {
syncErrors = errors.Join(syncErrors, fmt.Errorf("close output directory %q: %w", directory, closeErr))
}
}
if len(orphanedFiles) > 0 {
gologger.Info().Msgf("cleaned up %d orphaned template file(s)", len(orphanedFiles))
}
return nil
return syncErrors
}
// regenerateTemplateMetadata regenerates template index and checksum files after cleanup operations.
// This ensures the metadata accurately reflects the current state of template files on disk.
func (t *TemplateManager) regenerateTemplateMetadata(dir string) error {
// Purge empty directories that may have been left after cleanup
// templateOutputWriteResult records filesystem state that must be finalized
// even when the associated write returns an error.
type templateOutputWriteResult struct {
touchedDirectories []string
}
func writeTemplateOutput(rootDir, writePath string, contents []byte, mode fs.FileMode) (templateOutputWriteResult, error) {
var result templateOutputWriteResult
if isActiveIgnoreFilePath(writePath) {
if err := config.DefaultConfig.WriteActiveIgnoreFile(contents); err != nil {
return result, err
}
result.touchedDirectories = append(result.touchedDirectories, filepath.Dir(writePath))
return result, nil
}
relativePath, err := filepath.Rel(rootDir, writePath)
if err != nil {
return result, err
}
if relativePath == ".." || strings.HasPrefix(relativePath, ".."+string(os.PathSeparator)) || filepath.IsAbs(relativePath) {
return result, fmt.Errorf("output path %q escapes root %q", writePath, rootDir)
}
root, err := os.OpenRoot(rootDir)
if err != nil {
return result, fmt.Errorf("open output root %q: %w", rootDir, err)
}
defer func() { _ = root.Close() }()
parent := filepath.Dir(relativePath)
if parent != "." {
parentsToSync, err := createTemplateDirectories(root, parent)
for _, parentToSync := range parentsToSync {
result.touchedDirectories = append(result.touchedDirectories, filepath.Clean(filepath.Join(rootDir, parentToSync)))
}
if err != nil {
return result, fmt.Errorf("create output parent %q: %w", parent, err)
}
}
result.touchedDirectories = append(result.touchedDirectories, filepath.Dir(writePath))
randomSuffix := make([]byte, 16)
if _, err := rand.Read(randomSuffix); err != nil {
return result, fmt.Errorf("generate temporary output name: %w", err)
}
temporaryPath := filepath.Join(parent, fmt.Sprintf("%s%x", templateOutputTemporaryPrefix, randomSuffix))
temporaryMode := mode.Perm()
preserveMode := false
if info, err := root.Lstat(relativePath); err == nil && info.Mode().IsRegular() {
temporaryMode = 0o600
mode = info.Mode()
preserveMode = true
} else if err != nil && !os.IsNotExist(err) {
return result, fmt.Errorf("inspect existing output %q: %w", relativePath, err)
}
temporary, err := root.OpenFile(temporaryPath, os.O_WRONLY|os.O_CREATE|os.O_EXCL, temporaryMode)
if err != nil {
return result, fmt.Errorf("create temporary output %q: %w", temporaryPath, err)
}
defer func() {
_ = temporary.Close()
_ = root.Remove(temporaryPath)
}()
if _, err := temporary.Write(contents); err != nil {
return result, fmt.Errorf("write temporary output %q: %w", temporaryPath, err)
}
if preserveMode {
if err := temporary.Chmod(mode.Perm()); err != nil {
return result, fmt.Errorf("preserve output mode %q: %w", temporaryPath, err)
}
}
// A release contains thousands of files, so syncing every temporary file
// makes installation latency scale with storage flush latency. Closing and
// renaming keeps replacement atomic; touched directories are synced once
// after extraction and before ownership or version finalization.
if err := temporary.Close(); err != nil {
return result, fmt.Errorf("close temporary output %q: %w", temporaryPath, err)
}
if err := root.Rename(temporaryPath, relativePath); err != nil {
return result, fmt.Errorf("replace output %q: %w", relativePath, err)
}
return result, nil
}
// regenerateTemplateMetadata rebuilds the index and checksums from the finalized on-disk tree.
func (t *TemplateManager) regenerateTemplateMetadata(cfg *config.Config) (map[string]string, error) {
dir := cfg.TemplatesDirectory
// Purge empty directories before rebuilding metadata.
PurgeEmptyDirectories(dir)
// Ensure templates directory exists (it may have been purged if empty)
// Ensure the templates directory exists if the finalized tree is empty.
if !fileutil.FolderExists(dir) {
if err := os.MkdirAll(dir, 0755); err != nil {
return errkit.Wrapf(err, "failed to recreate templates directory %s after purge", dir)
return nil, errkit.Wrapf(err, "failed to recreate templates directory %s after purge", dir)
}
}
// Remove old index file and regenerate it from current templates on disk
indexFilePath := config.DefaultConfig.GetTemplateIndexFilePath()
indexFilePath := cfg.GetTemplateIndexFilePath()
if err := os.Remove(indexFilePath); err != nil && !os.IsNotExist(err) {
return errkit.Wrapf(err, "failed to remove old index file %s", indexFilePath)
return nil, errkit.Wrapf(err, "failed to remove old index file %s", indexFilePath)
}
// Force regeneration by ensuring the file doesn't exist (handles Windows file handle issues)
// GetNucleiTemplatesIndex will scan the directory if the file doesn't exist
index, err := config.GetNucleiTemplatesIndex()
if err != nil {
return errkit.Wrap(err, "failed to regenerate nuclei templates index after cleanup")
return nil, errkit.Wrap(err, "failed to regenerate nuclei templates index")
}
// Filter out any entries that don't actually exist on disk (Windows file deletion timing issues)
@@ -570,16 +721,20 @@ func (t *TemplateManager) regenerateTemplateMetadata(dir string) error {
}
}
if err = config.DefaultConfig.WriteTemplatesIndex(filteredIndex); err != nil {
return errkit.Wrap(err, "failed to write nuclei templates index after cleanup")
if err = cfg.WriteTemplatesIndex(filteredIndex); err != nil {
return nil, errkit.Wrap(err, "failed to write regenerated nuclei templates index")
}
// Regenerate checksum file to reflect current templates on disk
if err := t.writeChecksumFileInDir(dir); err != nil {
return errkit.Wrap(err, "failed to regenerate checksum file after cleanup")
checksumMap, err := t.calculateChecksumMap(dir)
if err != nil {
return nil, errkit.Wrap(err, "failed to regenerate checksum map")
}
return nil
if err := writeChecksumMap(cfg, checksumMap); err != nil {
return nil, errkit.Wrap(err, "failed to write regenerated checksum file")
}
return checksumMap, nil
}
// getChecksumFromDir returns a map containing checksums (md5 hash) of all yaml files (with .yaml extension)
@@ -598,23 +753,31 @@ func (t *TemplateManager) getChecksumFromDir(dir string) (map[string]string, err
if len(tmparr) != 2 {
continue
}
allChecksums[tmparr[0]] = tmparr[1]
}
return allChecksums, nil
}
}
return t.calculateChecksumMap(dir)
}
// writeChecksumFileInDir creates checksums of all yaml files in given directory
// and writes them to a file named .checksum
// writeChecksumFileInDir creates checksums of all YAML files in dir and writes
// them to the configured checksum file.
func (t *TemplateManager) writeChecksumFileInDir(dir string) error {
checksumMap, err := t.calculateChecksumMap(dir)
if err != nil {
return err
}
return writeChecksumMap(config.DefaultConfig, checksumMap)
}
func writeChecksumMap(cfg *config.Config, checksumMap map[string]string) error {
var buff bytes.Buffer
for k, v := range checksumMap {
buff.WriteString(k)
buff.WriteString(",")
@@ -622,7 +785,19 @@ func (t *TemplateManager) writeChecksumFileInDir(dir string) error {
buff.WriteString(";")
}
return os.WriteFile(config.DefaultConfig.GetChecksumFilePath(), buff.Bytes(), checkSumFilePerm)
return os.WriteFile(cfg.GetChecksumFilePath(), buff.Bytes(), checkSumFilePerm)
}
func isTemplateOutputTemporary(name string) bool {
suffix, found := strings.CutPrefix(name, templateOutputTemporaryPrefix)
if !found {
return false
}
if isLowerHex(suffix, 32) {
return true
}
pathDigest, token, found := strings.Cut(suffix, "-")
return found && isLowerHex(pathDigest, 32) && isLowerHex(token, 32)
}
// getChecksumMap returns a map containing checksums (md5 hash) of all yaml files (with .yaml extension)
@@ -638,6 +813,7 @@ func (t *TemplateManager) calculateChecksumMap(dir string) (map[string]string, e
if err != nil {
return "", err
}
return fmt.Sprintf("%x", md5.Sum(bin)), nil
}
@@ -645,6 +821,18 @@ func (t *TemplateManager) calculateChecksumMap(dir string) (map[string]string, e
if err != nil {
return err
}
if !d.IsDir() && isTemplateOutputTemporary(filepath.Base(path)) {
return nil
}
if filepath.Dir(path) == filepath.Clean(dir) {
name := filepath.Base(path)
if name == templateOwnershipFileName || strings.HasPrefix(name, templateOwnershipTemporaryPrefix) || strings.HasPrefix(name, templateOwnershipRetiredPrefix) || strings.HasPrefix(name, templateOwnershipRestorePrefix) {
return nil
}
}
// skip checksums of custom templates i.e github and s3
if filepathutil.IsPathWithinAnyDirectory(path, config.DefaultConfig.GetAllCustomTemplateDirs()...) {
return nil
@@ -656,9 +844,12 @@ func (t *TemplateManager) calculateChecksumMap(dir string) (map[string]string, e
if err != nil {
return err
}
checksumMap[path] = checksum
}
return nil
})
return checksumMap, errkit.Wrap(err, "failed to calculate checksums of templates")
}
@@ -0,0 +1,42 @@
//go:build !windows
package installer
import (
"os"
"path/filepath"
"syscall"
"testing"
"github.com/stretchr/testify/require"
)
func TestWriteTemplateOutputHonorsUmask(t *testing.T) {
previousUmask := syscall.Umask(0o077)
t.Cleanup(func() { syscall.Umask(previousUmask) })
outputPath := filepath.Join(t.TempDir(), "official.yaml")
_, err := writeTemplateOutput(filepath.Dir(outputPath), outputPath, []byte("release"), 0o666)
require.NoError(t, err)
info, err := os.Stat(outputPath)
require.NoError(t, err)
require.Equal(t, os.FileMode(0o600), info.Mode().Perm())
}
func TestCopyQuarantinedTemplatePreservesMode(t *testing.T) {
templatesDir := t.TempDir()
quarantinePath := templateOwnershipRetiredPrefix + "mode"
require.NoError(t, os.WriteFile(filepath.Join(templatesDir, quarantinePath), []byte("modified"), 0o644))
previousUmask := syscall.Umask(0o077)
t.Cleanup(func() { syscall.Umask(previousUmask) })
root, err := os.OpenRoot(templatesDir)
require.NoError(t, err)
t.Cleanup(func() { require.NoError(t, root.Close()) })
require.NoError(t, copyQuarantinedTemplate(root, "retired.yaml", quarantinePath))
info, err := root.Stat("retired.yaml")
require.NoError(t, err)
require.Equal(t, os.FileMode(0o644), info.Mode().Perm())
}
File diff suppressed because it is too large. Load diff
@@ -0,0 +1,42 @@
//go:build darwin
package installer
import (
"errors"
"fmt"
"os"
"path/filepath"
"golang.org/x/sys/unix"
)
func renameTemplateRestoreNoReplace(root *os.Root, temporaryPath, retiredPath string) error {
sourceDirectory, err := root.Open(filepath.Dir(temporaryPath))
if err != nil {
return err
}
defer func() { _ = sourceDirectory.Close() }()
destinationDirectory, err := root.Open(filepath.Dir(retiredPath))
if err != nil {
return err
}
defer func() { _ = destinationDirectory.Close() }()
err = unix.RenameatxNp(int(sourceDirectory.Fd()), filepath.Base(temporaryPath), int(destinationDirectory.Fd()), filepath.Base(retiredPath), unix.RENAME_EXCL)
return mapRenameNoReplaceError(err)
}
func mapRenameNoReplaceError(err error) error {
if err == nil {
return nil
}
switch {
case errors.Is(err, unix.ENOTSUP), errors.Is(err, unix.ENOSYS), errors.Is(err, unix.EINVAL):
return fmt.Errorf("%w: %w", errors.ErrUnsupported, err)
default:
return err
}
}
@@ -0,0 +1,42 @@
//go:build linux
package installer
import (
"errors"
"fmt"
"os"
"path/filepath"
"golang.org/x/sys/unix"
)
func renameTemplateRestoreNoReplace(root *os.Root, temporaryPath, retiredPath string) error {
sourceDirectory, err := root.Open(filepath.Dir(temporaryPath))
if err != nil {
return err
}
defer func() { _ = sourceDirectory.Close() }()
destinationDirectory, err := root.Open(filepath.Dir(retiredPath))
if err != nil {
return err
}
defer func() { _ = destinationDirectory.Close() }()
err = unix.Renameat2(int(sourceDirectory.Fd()), filepath.Base(temporaryPath), int(destinationDirectory.Fd()), filepath.Base(retiredPath), unix.RENAME_NOREPLACE)
return mapRenameNoReplaceError(err)
}
func mapRenameNoReplaceError(err error) error {
if err == nil {
return nil
}
switch {
case errors.Is(err, unix.ENOSYS), errors.Is(err, unix.EINVAL), errors.Is(err, unix.EOPNOTSUPP):
return fmt.Errorf("%w: %w", errors.ErrUnsupported, err)
default:
return err
}
}
@@ -0,0 +1,67 @@
//go:build linux
package installer
import (
"errors"
"os"
"testing"
"github.com/stretchr/testify/require"
)
func TestCopyQuarantinedTemplateWithoutHardLinks(t *testing.T) {
root, err := os.OpenRoot(t.TempDir())
require.NoError(t, err)
t.Cleanup(func() { require.NoError(t, root.Close()) })
quarantinePath := templateOwnershipRetiredPrefix + "no-hard-links"
contents := []byte("locally modified contents")
require.NoError(t, root.WriteFile(quarantinePath, contents, 0o640))
openQuarantine, err := root.OpenFile(quarantinePath, os.O_RDWR, 0)
require.NoError(t, err)
t.Cleanup(func() { require.NoError(t, openQuarantine.Close()) })
err = restoreQuarantinedTemplateWithLink(root, "retired.yaml", quarantinePath, func(string, string) error {
return errors.ErrUnsupported
})
require.NoError(t, err)
require.NoError(t, openQuarantine.Truncate(0))
_, err = openQuarantine.WriteAt([]byte("late local write"), 0)
require.NoError(t, err)
restored, err := root.ReadFile("retired.yaml")
require.NoError(t, err)
require.Equal(t, []byte("late local write"), restored)
info, err := root.Stat("retired.yaml")
require.NoError(t, err)
require.Equal(t, os.FileMode(0o640), info.Mode().Perm())
}
func TestResumeTemplateRestoreStateWithoutHardLinks(t *testing.T) {
root, err := os.OpenRoot(t.TempDir())
require.NoError(t, err)
t.Cleanup(func() { require.NoError(t, root.Close()) })
retiredPath := "retired.yaml"
quarantinePath := templateOwnershipRetiredPrefix + "pre-publication"
contents := []byte("locally modified contents")
require.NoError(t, root.WriteFile(quarantinePath, contents, 0o600))
state, err := newTemplateRestoreState(retiredPath)
require.NoError(t, err)
state.kind = templateRestoreCopy
state.digest = templateDigest(contents)
require.NoError(t, root.WriteFile(state.temporaryPath, contents, 0o600))
require.NoError(t, createTemplateRestoreState(root, state))
completed, err := resumeTemplateRestoreState(root, state, quarantinePath, func(string, string) error {
return errors.ErrUnsupported
})
require.NoError(t, err)
require.False(t, completed)
require.NoError(t, restoreQuarantinedTemplate(root, retiredPath, quarantinePath))
restored, err := root.ReadFile(retiredPath)
require.NoError(t, err)
require.Equal(t, contents, restored)
_, err = root.Lstat(state.statePath)
require.ErrorIs(t, err, os.ErrNotExist)
}
@@ -0,0 +1,12 @@
//go:build !linux && !darwin && !windows
package installer
import (
"errors"
"os"
)
func renameTemplateRestoreNoReplace(_ *os.Root, _, _ string) error {
return errors.ErrUnsupported
}
@@ -0,0 +1,81 @@
//go:build windows
package installer
import (
"os"
"path/filepath"
"unsafe"
"golang.org/x/sys/windows"
)
type templateRestoreRenameInformation struct {
replaceIfExists uint32
rootDirectory windows.Handle
fileNameLength uint32
fileName [1]uint16
}
func renameTemplateRestoreNoReplace(root *os.Root, temporaryPath, retiredPath string) error {
sourceDirectory, err := root.Open(filepath.Dir(temporaryPath))
if err != nil {
return err
}
defer func() { _ = sourceDirectory.Close() }()
destinationDirectory, err := root.Open(filepath.Dir(retiredPath))
if err != nil {
return err
}
defer func() { _ = destinationDirectory.Close() }()
objectName, err := windows.NewNTUnicodeString(filepath.Base(temporaryPath))
if err != nil {
return err
}
attributes := &windows.OBJECT_ATTRIBUTES{
Length: uint32(unsafe.Sizeof(windows.OBJECT_ATTRIBUTES{})),
RootDirectory: windows.Handle(sourceDirectory.Fd()),
ObjectName: objectName,
Attributes: windows.OBJ_CASE_INSENSITIVE,
}
var source windows.Handle
var status windows.IO_STATUS_BLOCK
if err := windows.NtCreateFile(
&source,
windows.SYNCHRONIZE|windows.DELETE,
attributes,
&status,
nil,
0,
windows.FILE_SHARE_READ|windows.FILE_SHARE_WRITE|windows.FILE_SHARE_DELETE,
windows.FILE_OPEN,
windows.FILE_OPEN_REPARSE_POINT|windows.FILE_OPEN_FOR_BACKUP_INTENT|windows.FILE_SYNCHRONOUS_IO_NONALERT,
0,
0,
); err != nil {
return err
}
defer windows.CloseHandle(source)
targetName, err := windows.UTF16FromString(filepath.Base(retiredPath))
if err != nil {
return err
}
targetNameLength := (len(targetName) - 1) * 2
var header templateRestoreRenameInformation
buffer := make([]byte, int(unsafe.Offsetof(header.fileName))+targetNameLength)
info := (*templateRestoreRenameInformation)(unsafe.Pointer(&buffer[0]))
info.rootDirectory = windows.Handle(destinationDirectory.Fd())
info.fileNameLength = uint32(targetNameLength)
copy((*[windows.MAX_LONG_PATH]uint16)(unsafe.Pointer(&info.fileName[0]))[:targetNameLength/2:targetNameLength/2], targetName)
return windows.NtSetInformationFile(source, &status, &buffer[0], uint32(len(buffer)), windows.FileRenameInformation)
}
@@ -0,0 +1,27 @@
//go:build !windows
package installer
import (
"os"
"path/filepath"
"strings"
"syscall"
"testing"
"github.com/stretchr/testify/require"
)
func TestLoadTemplateRestoreStateRejectsFIFO(t *testing.T) {
templatesDir := t.TempDir()
root, err := os.OpenRoot(templatesDir)
require.NoError(t, err)
t.Cleanup(func() { require.NoError(t, root.Close()) })
state := templateRestoreState{
retiredPath: "retired.yaml",
statePath: templateRestoreStatePrefix("retired.yaml") + strings.Repeat("a", 32),
}
require.NoError(t, syscall.Mkfifo(filepath.Join(templatesDir, state.statePath), 0o600))
require.ErrorContains(t, loadTemplateRestoreState(root, &state), "is not a regular file")
}
+34
View File
@@ -0,0 +1,34 @@
//go:build !windows
package installer
import (
"errors"
"io/fs"
"os"
)
func syncTemplateOwnershipFile(root *os.Root, relativePath string, mode fs.FileMode) error {
file, err := root.Open(relativePath)
if err != nil {
return err
}
if err := file.Chmod(mode.Perm()); err != nil {
_ = file.Close()
return err
}
return errors.Join(file.Sync(), file.Close())
}
func syncTemplateOwnershipDirectory(root *os.Root, relativePath string) error {
directory, err := root.Open(relativePath)
if err != nil {
return err
}
defer func() { _ = directory.Close() }()
return directory.Sync()
}
@@ -0,0 +1,35 @@
package installer
import (
"errors"
"io/fs"
"os"
)
func syncTemplateOwnershipFile(root *os.Root, relativePath string, mode fs.FileMode) error {
return syncTemplateOwnershipFileWithOpen(root, relativePath, mode, func() (*os.File, error) {
return root.OpenFile(relativePath, os.O_RDWR, 0)
})
}
func syncTemplateOwnershipFileWithOpen(root *os.Root, relativePath string, mode fs.FileMode, open func() (*os.File, error)) error {
if err := root.Chmod(relativePath, 0o600); err != nil {
return err
}
file, err := open()
if err != nil {
return errors.Join(err, root.Chmod(relativePath, mode.Perm()))
}
if err := file.Chmod(mode.Perm()); err != nil {
return errors.Join(err, file.Close(), root.Chmod(relativePath, mode.Perm()))
}
return errors.Join(file.Sync(), file.Close())
}
// Windows cannot flush the read-only directory handle exposed by os.Root.
func syncTemplateOwnershipDirectory(_ *os.Root, _ string) error {
return nil
}
@@ -0,0 +1,73 @@
package installer
import (
"errors"
"os"
"path/filepath"
"testing"
"github.com/stretchr/testify/require"
)
func TestSyncTemplateOwnershipDirectory(t *testing.T) {
root, err := os.OpenRoot(t.TempDir())
require.NoError(t, err)
t.Cleanup(func() { require.NoError(t, root.Close()) })
require.NoError(t, syncTemplateOwnershipDirectory(root, "."))
}
func TestSyncTemplateOwnershipFileRestoresModeAfterOpenFailure(t *testing.T) {
root, err := os.OpenRoot(t.TempDir())
require.NoError(t, err)
t.Cleanup(func() { require.NoError(t, root.Close()) })
require.NoError(t, root.WriteFile("restored.yaml", []byte("restored"), 0o444))
originalInfo, err := root.Stat("restored.yaml")
require.NoError(t, err)
openErr := errors.New("open failed")
err = syncTemplateOwnershipFileWithOpen(root, "restored.yaml", originalInfo.Mode(), func() (*os.File, error) {
return nil, openErr
})
require.ErrorIs(t, err, openErr)
info, err := root.Stat("restored.yaml")
require.NoError(t, err)
require.Equal(t, originalInfo.Mode().Perm(), info.Mode().Perm())
}
func TestRenameTemplateRestoreNoReplaceUsesOpenedRoot(t *testing.T) {
parent := t.TempDir()
openedParent := filepath.Join(parent, "opened")
currentParent := filepath.Join(parent, "current")
rootPath := filepath.Join(openedParent, "templates")
currentRootPath := filepath.Join(currentParent, "templates")
require.NoError(t, os.MkdirAll(filepath.Join(rootPath, "source"), 0o755))
require.NoError(t, os.Mkdir(filepath.Join(rootPath, "destination"), 0o755))
require.NoError(t, os.MkdirAll(filepath.Join(currentRootPath, "source"), 0o755))
require.NoError(t, os.Mkdir(filepath.Join(currentRootPath, "destination"), 0o755))
require.NoError(t, os.WriteFile(filepath.Join(currentRootPath, "source", "temporary.yaml"), []byte("wrong root"), 0o600))
t.Chdir(openedParent)
root, err := os.OpenRoot("templates")
require.NoError(t, err)
t.Cleanup(func() { require.NoError(t, root.Close()) })
require.NoError(t, root.WriteFile(filepath.Join("source", "temporary.yaml"), []byte("restored"), 0o600))
t.Chdir(currentParent)
require.NoError(t, renameTemplateRestoreNoReplace(root, filepath.Join("source", "temporary.yaml"), filepath.Join("destination", "retired.yaml")))
require.FileExists(t, filepath.Join(rootPath, "destination", "retired.yaml"))
require.NoFileExists(t, filepath.Join(currentRootPath, "destination", "retired.yaml"))
require.NoError(t, os.Rename(filepath.Join(rootPath, "source"), filepath.Join(rootPath, "moved-source")))
require.NoError(t, os.Rename(filepath.Join(rootPath, "destination"), filepath.Join(rootPath, "moved-destination")))
}
func TestSyncTemplateOwnershipFile(t *testing.T) {
root, err := os.OpenRoot(t.TempDir())
require.NoError(t, err)
t.Cleanup(func() { require.NoError(t, root.Close()) })
require.NoError(t, root.WriteFile("restored.yaml", []byte("restored"), 0o444))
require.NoError(t, syncTemplateOwnershipFile(root, "restored.yaml", 0o444))
info, err := root.Stat("restored.yaml")
require.NoError(t, err)
require.Equal(t, os.FileMode(0o444), info.Mode().Perm())
}
File diff suppressed because it is too large. Load diff
+347 -390
View File
@@ -1,14 +1,16 @@
package installer
import (
"errors"
"fmt"
"os"
"path/filepath"
"strings"
"testing"
"github.com/projectdiscovery/gologger"
"github.com/projectdiscovery/nuclei/v3/pkg/catalog/config"
fileutil "github.com/projectdiscovery/utils/file"
mapsutil "github.com/projectdiscovery/utils/maps"
"github.com/stretchr/testify/require"
)
@@ -59,6 +61,9 @@ func TestTemplateInstallation(t *testing.T) {
require.Greater(t, counter, 1000)
// every time we install templates, it should override the ignore file with latest one
require.FileExists(t, config.DefaultConfig.GetActiveIgnoreFilePath())
ownership, err := loadTemplateOwnership(templatesTempDir)
require.NoError(t, err)
require.NotEmpty(t, ownership.Files, "fresh installation should record official template ownership")
t.Logf("Installed %d templates", counter)
}
@@ -101,277 +106,262 @@ func TestIsOutdatedVersion(t *testing.T) {
}
}
func TestCleanupOrphanedTemplates(t *testing.T) {
HideProgressBar = true
func TestWriteTemplateOutput(t *testing.T) {
t.Run("replaces a final symlink without modifying its target", func(t *testing.T) {
templatesDir := t.TempDir()
externalPath := filepath.Join(t.TempDir(), "external.yaml")
require.NoError(t, os.WriteFile(externalPath, []byte("external"), 0o644))
outputPath := filepath.Join(templatesDir, "official.yaml")
if err := os.Symlink(externalPath, outputPath); err != nil {
t.Skipf("symlinks are unavailable: %v", err)
}
_, err := writeTemplateOutput(templatesDir, outputPath, []byte("release"), 0o644)
require.NoError(t, err)
require.FileExists(t, outputPath)
info, err := os.Lstat(outputPath)
require.NoError(t, err)
require.Zero(t, info.Mode()&os.ModeSymlink)
contents, err := os.ReadFile(outputPath)
require.NoError(t, err)
require.Equal(t, "release", string(contents))
require.FileExists(t, externalPath)
contents, err = os.ReadFile(externalPath)
require.NoError(t, err)
require.Equal(t, "external", string(contents))
})
t.Run("rejects a parent symlink outside the templates directory", func(t *testing.T) {
templatesDir := t.TempDir()
externalDir := t.TempDir()
linkedParent := filepath.Join(templatesDir, "nested")
if err := os.Symlink(externalDir, linkedParent); err != nil {
t.Skipf("symlinks are unavailable: %v", err)
}
_, err := writeTemplateOutput(templatesDir, filepath.Join(linkedParent, "official.yaml"), []byte("release"), 0o644)
require.Error(t, err)
require.NoFileExists(t, filepath.Join(externalDir, "official.yaml"))
})
t.Run("preserves permissions on an existing regular file", func(t *testing.T) {
templatesDir := t.TempDir()
outputPath := filepath.Join(templatesDir, "official.yaml")
require.NoError(t, os.WriteFile(outputPath, []byte("previous"), 0o600))
previousInfo, err := os.Stat(outputPath)
require.NoError(t, err)
_, err = writeTemplateOutput(templatesDir, outputPath, []byte("release"), 0o777)
require.NoError(t, err)
info, err := os.Stat(outputPath)
require.NoError(t, err)
require.Equal(t, previousInfo.Mode().Perm(), info.Mode().Perm())
})
t.Run("reports every directory touched by nested output creation", func(t *testing.T) {
templatesDir := t.TempDir()
firstParent := filepath.Join(templatesDir, "first")
outputParent := filepath.Join(firstParent, "second")
outputPath := filepath.Join(outputParent, "official.yaml")
result, err := writeTemplateOutput(templatesDir, outputPath, []byte("release"), 0o644)
require.NoError(t, err)
require.ElementsMatch(t, []string{templatesDir, firstParent, outputParent}, result.touchedDirectories)
})
t.Run("reports the parent touched by a failed replacement", func(t *testing.T) {
templatesDir := t.TempDir()
outputPath := filepath.Join(templatesDir, "existing-directory")
require.NoError(t, os.Mkdir(outputPath, 0o755))
result, err := writeTemplateOutput(templatesDir, outputPath, []byte("release"), 0o644)
require.Error(t, err)
require.Equal(t, []string{templatesDir}, result.touchedDirectories)
})
}
func TestSyncTemplateOutputDirectories(t *testing.T) {
t.Run("syncs each distinct directory once", func(t *testing.T) {
directories := map[string]struct{}{
t.TempDir(): {},
t.TempDir(): {},
}
syncCalls := 0
err := syncTemplateOutputDirectories(directories, func(*os.Root, string) error {
syncCalls++
return nil
})
require.NoError(t, err)
require.Equal(t, len(directories), syncCalls)
})
t.Run("continues after a sync failure", func(t *testing.T) {
directories := map[string]struct{}{
t.TempDir(): {},
t.TempDir(): {},
}
syncErr := errors.New("sync failed")
syncCalls := 0
err := syncTemplateOutputDirectories(directories, func(*os.Root, string) error {
syncCalls++
if syncCalls == 1 {
return syncErr
}
return nil
})
require.ErrorIs(t, err, syncErr)
require.Equal(t, len(directories), syncCalls)
})
}
func BenchmarkWriteTemplateOutputs(b *testing.B) {
b.Run("batched", func(b *testing.B) {
benchmarkWriteTemplateOutputs(b, false)
})
b.Run("per-output-sync", func(b *testing.B) {
benchmarkWriteTemplateOutputs(b, true)
})
}
func benchmarkWriteTemplateOutputs(b *testing.B, syncEachOutput bool) {
rootDir := b.TempDir()
contents := []byte("id: benchmark\ninfo:\n name: Benchmark\n author: test\n severity: info\n")
touchedDirectories := make(map[string]struct{})
b.ReportAllocs()
b.ResetTimer()
for index := 0; index < b.N; index++ {
writePath := filepath.Join(rootDir, fmt.Sprintf("group-%d", index%16), fmt.Sprintf("template-%d.yaml", index))
outputResult, err := writeTemplateOutput(rootDir, writePath, contents, 0o644)
if err != nil {
b.Fatal(err)
}
for _, directory := range outputResult.touchedDirectories {
touchedDirectories[directory] = struct{}{}
}
if syncEachOutput {
output, err := os.OpenFile(writePath, os.O_RDWR, 0)
if err != nil {
b.Fatal(err)
}
syncErr := output.Sync()
closeErr := output.Close()
if err := errors.Join(syncErr, closeErr); err != nil {
b.Fatal(err)
}
}
}
if err := syncTemplateOutputDirectories(touchedDirectories, syncTemplateOwnershipDirectory); err != nil {
b.Fatal(err)
}
}
func TestGetTemplateOutputLocationRoutesIgnoreFile(t *testing.T) {
previousConfig := config.DefaultConfig
templatesDir := t.TempDir()
cfg := &config.Config{}
cfg.SetTemplatesDir(templatesDir)
config.DefaultConfig = cfg
t.Cleanup(func() { config.DefaultConfig = previousConfig })
ignoreEntry := filepath.Join(t.TempDir(), config.NucleiIgnoreFileName)
require.NoError(t, os.WriteFile(ignoreEntry, nil, 0o600))
info, err := os.Stat(ignoreEntry)
require.NoError(t, err)
rootDir, writePath := (&TemplateManager{}).getTemplateOutputLocation(templatesDir, "nuclei-templates/"+config.NucleiIgnoreFileName, info)
require.Equal(t, templatesDir, rootDir)
require.Equal(t, cfg.GetActiveIgnoreFilePath(), writePath)
}
func TestWriteTemplateOutputValidatesIgnoreFile(t *testing.T) {
previousConfig := config.DefaultConfig
templatesDir := t.TempDir()
cfg := &config.Config{}
cfg.SetTemplatesDir(templatesDir)
config.DefaultConfig = cfg
t.Cleanup(func() { config.DefaultConfig = previousConfig })
path := cfg.GetActiveIgnoreFilePath()
valid := []byte("tags: [weak]\n")
require.NoError(t, os.WriteFile(path, valid, 0o600))
_, err := writeTemplateOutput(templatesDir, path, []byte("<html>blocked</html>\n"), 0o644)
require.Error(t, err)
got, err := os.ReadFile(path)
require.NoError(t, err)
require.Equal(t, valid, got)
payload := []byte("files: [blocked.yaml]\n")
_, err = writeTemplateOutput(templatesDir, path, payload, 0o644)
require.NoError(t, err)
got, err = os.ReadFile(path)
require.NoError(t, err)
require.Equal(t, payload, got)
}
func TestCommitTemplateVersionRestoresMemoryOnWriteFailure(t *testing.T) {
stateDir := filepath.Join(t.TempDir(), "state")
require.NoError(t, os.WriteFile(stateDir, nil, 0o600))
cfg := &config.Config{TemplateVersion: "v1.2.3"}
cfg.SetStateDir(stateDir)
err := commitTemplateVersion(cfg, "v2.0.0")
require.Error(t, err)
require.Equal(t, "v1.2.3", cfg.TemplateVersion)
}
func TestFinalizeTemplateReleaseCommitsVersionLast(t *testing.T) {
newConfig := func(t *testing.T) *config.Config {
t.Helper()
configDir := t.TempDir()
templatesDir := t.TempDir()
cfg := &config.Config{Logger: gologger.DefaultLogger}
cfg.SetConfigDir(configDir)
cfg.SetTemplatesDir(templatesDir)
require.NoError(t, os.WriteFile(cfg.GetActiveIgnoreFilePath(), []byte("ignore contents"), 0o600))
cfg.TemplateVersion = "v1.0.0"
previousConfig := config.DefaultConfig
config.DefaultConfig = cfg
t.Cleanup(func() { config.DefaultConfig = previousConfig })
return cfg
}
t.Run("commits after successful finalization", func(t *testing.T) {
cfg := newConfig(t)
templatePath := filepath.Join(cfg.TemplatesDirectory, "official.yaml")
require.NoError(t, os.WriteFile(templatePath, []byte("id: official\ninfo:\n name: Official\n author: test\n severity: info\n"), 0644))
tm := &TemplateManager{}
_, err := tm.finalizeTemplateRelease(cfg, newWrittenTemplates(t, templatePath), "v2.0.0")
require.NoError(t, err)
require.Equal(t, "v2.0.0", cfg.TemplateVersion)
})
t.Run("preserves version when metadata finalization fails", func(t *testing.T) {
cfg := newConfig(t)
templatePath := filepath.Join(cfg.TemplatesDirectory, "official.yaml")
require.NoError(t, os.WriteFile(templatePath, []byte("id: official\ninfo:\n name: Official\n author: test\n severity: info\n"), 0644))
require.NoError(t, os.Mkdir(cfg.GetTemplateIndexFilePath(), 0755))
require.NoError(t, os.WriteFile(filepath.Join(cfg.GetTemplateIndexFilePath(), "keep"), []byte("keep"), 0600))
tm := &TemplateManager{}
_, err := tm.finalizeTemplateRelease(cfg, newWrittenTemplates(t, templatePath), "v2.0.0")
require.Error(t, err)
require.Equal(t, "v1.0.0", cfg.TemplateVersion)
})
}
func TestCalculateChecksumMap(t *testing.T) {
tm := &TemplateManager{}
t.Run("removes orphaned templates", func(t *testing.T) {
// Create temporary directories
tmpDir, err := os.MkdirTemp("", "nuclei-cleanup-test-*")
require.NoError(t, err)
defer func() {
_ = os.RemoveAll(tmpDir)
}()
cfgdir, err := os.MkdirTemp("", "nuclei-config-*")
require.NoError(t, err)
defer func() {
_ = os.RemoveAll(cfgdir)
}()
config.DefaultConfig.SetConfigDir(cfgdir)
config.DefaultConfig.SetTemplatesDir(tmpDir)
// Create subdirectories for templates
templatesDir1 := filepath.Join(tmpDir, "cves", "2023")
templatesDir2 := filepath.Join(tmpDir, "exposures", "configs")
require.NoError(t, os.MkdirAll(templatesDir1, 0755))
require.NoError(t, os.MkdirAll(templatesDir2, 0755))
// Create template files
template1 := filepath.Join(templatesDir1, "CVE-2023-1234.yaml")
template2 := filepath.Join(templatesDir1, "CVE-2023-5678.yaml")
template3 := filepath.Join(templatesDir2, "git-config-exposure.yaml")
orphanedTemplate1 := filepath.Join(templatesDir1, "old-template.yaml")
orphanedTemplate2 := filepath.Join(templatesDir2, "removed-template.yaml")
// Write valid template files
templateContent := `id: test-template
info:
name: Test Template
author: test
severity: info`
require.NoError(t, os.WriteFile(template1, []byte(templateContent), 0644))
require.NoError(t, os.WriteFile(template2, []byte(templateContent), 0644))
require.NoError(t, os.WriteFile(template3, []byte(templateContent), 0644))
require.NoError(t, os.WriteFile(orphanedTemplate1, []byte(templateContent), 0644))
require.NoError(t, os.WriteFile(orphanedTemplate2, []byte(templateContent), 0644))
// Simulate written paths from new release (only template1, template2, template3)
writtenPaths := mapsutil.NewSyncLockMap[string, struct{}]()
absTemplate1, _ := filepath.Abs(template1)
absTemplate2, _ := filepath.Abs(template2)
absTemplate3, _ := filepath.Abs(template3)
// Normalize paths consistently (same as cleanupOrphanedTemplates does)
absTemplate1 = filepath.Clean(absTemplate1)
absTemplate2 = filepath.Clean(absTemplate2)
absTemplate3 = filepath.Clean(absTemplate3)
_ = writtenPaths.Set(absTemplate1, struct{}{})
_ = writtenPaths.Set(absTemplate2, struct{}{})
_ = writtenPaths.Set(absTemplate3, struct{}{})
// Run cleanup
err = tm.cleanupOrphanedTemplates(tmpDir, writtenPaths)
require.NoError(t, err)
// Verify orphaned templates were removed
require.NoFileExists(t, orphanedTemplate1, "orphaned template should be removed")
require.NoFileExists(t, orphanedTemplate2, "orphaned template should be removed")
// Verify non-orphaned templates still exist
require.FileExists(t, template1, "template from new release should exist")
require.FileExists(t, template2, "template from new release should exist")
require.FileExists(t, template3, "template from new release should exist")
})
t.Run("preserves custom templates", func(t *testing.T) {
// Create temporary directories
tmpDir, err := os.MkdirTemp("", "nuclei-cleanup-custom-test-*")
require.NoError(t, err)
defer func() {
_ = os.RemoveAll(tmpDir)
}()
cfgdir, err := os.MkdirTemp("", "nuclei-config-*")
require.NoError(t, err)
defer func() {
_ = os.RemoveAll(cfgdir)
}()
config.DefaultConfig.SetConfigDir(cfgdir)
config.DefaultConfig.SetTemplatesDir(tmpDir)
// Create custom template directory
customGitHubDir := filepath.Join(tmpDir, "github", "owner", "repo")
require.NoError(t, os.MkdirAll(customGitHubDir, 0755))
// Create custom template file
customTemplate := filepath.Join(customGitHubDir, "custom-template.yaml")
templateContent := `id: custom-template
info:
name: Custom Template
author: test
severity: info`
require.NoError(t, os.WriteFile(customTemplate, []byte(templateContent), 0644))
// Empty written paths (simulating no custom templates in new release)
writtenPaths := mapsutil.NewSyncLockMap[string, struct{}]()
// Run cleanup
err = tm.cleanupOrphanedTemplates(tmpDir, writtenPaths)
require.NoError(t, err)
// Verify custom template was NOT removed
require.FileExists(t, customTemplate, "custom template should be preserved")
})
t.Run("removes orphaned templates from custom dir sibling prefixes", func(t *testing.T) {
tmpDir, err := os.MkdirTemp("", "nuclei-cleanup-custom-prefix-test-*")
require.NoError(t, err)
defer func() {
_ = os.RemoveAll(tmpDir)
}()
cfgdir, err := os.MkdirTemp("", "nuclei-config-*")
require.NoError(t, err)
defer func() {
_ = os.RemoveAll(cfgdir)
}()
config.DefaultConfig.SetConfigDir(cfgdir)
config.DefaultConfig.SetTemplatesDir(tmpDir)
customGitHubDir := filepath.Join(tmpDir, "github", "owner", "repo")
require.NoError(t, os.MkdirAll(customGitHubDir, 0755))
customTemplate := filepath.Join(customGitHubDir, "custom-template.yaml")
require.NoError(t, os.WriteFile(customTemplate, []byte(`id: custom-template
info:
name: Custom Template
author: test
severity: info`), 0644))
siblingDir := filepath.Join(tmpDir, "github-evil")
require.NoError(t, os.MkdirAll(siblingDir, 0755))
siblingTemplate := filepath.Join(siblingDir, "orphaned-template.yaml")
require.NoError(t, os.WriteFile(siblingTemplate, []byte(`id: orphaned-template
info:
name: Orphaned Template
author: test
severity: info`), 0644))
writtenPaths := mapsutil.NewSyncLockMap[string, struct{}]()
err = tm.cleanupOrphanedTemplates(tmpDir, writtenPaths)
require.NoError(t, err)
require.FileExists(t, customTemplate, "custom template should be preserved")
require.NoFileExists(t, siblingTemplate, "custom directory sibling prefix should not be preserved")
})
t.Run("skips non-template files", func(t *testing.T) {
// Create temporary directories
tmpDir, err := os.MkdirTemp("", "nuclei-cleanup-nontemplate-test-*")
require.NoError(t, err)
defer func() {
_ = os.RemoveAll(tmpDir)
}()
cfgdir, err := os.MkdirTemp("", "nuclei-config-*")
require.NoError(t, err)
defer func() {
_ = os.RemoveAll(cfgdir)
}()
config.DefaultConfig.SetConfigDir(cfgdir)
config.DefaultConfig.SetTemplatesDir(tmpDir)
// Create non-template files
readmeFile := filepath.Join(tmpDir, "README.md")
configFile := filepath.Join(tmpDir, "cves.json")
checksumFile := filepath.Join(tmpDir, ".checksum")
require.NoError(t, os.WriteFile(readmeFile, []byte("# Templates"), 0644))
require.NoError(t, os.WriteFile(configFile, []byte("{}"), 0644))
require.NoError(t, os.WriteFile(checksumFile, []byte(""), 0644))
// Empty written paths
writtenPaths := mapsutil.NewSyncLockMap[string, struct{}]()
// Run cleanup
err = tm.cleanupOrphanedTemplates(tmpDir, writtenPaths)
require.NoError(t, err)
// Verify non-template files were NOT removed
require.FileExists(t, readmeFile, "README.md should be preserved")
require.FileExists(t, configFile, "config file should be preserved")
require.FileExists(t, checksumFile, "checksum file should be preserved")
})
t.Run("handles empty written paths", func(t *testing.T) {
// Create temporary directories
tmpDir, err := os.MkdirTemp("", "nuclei-cleanup-empty-test-*")
require.NoError(t, err)
defer func() {
_ = os.RemoveAll(tmpDir)
}()
cfgdir, err := os.MkdirTemp("", "nuclei-config-*")
require.NoError(t, err)
defer func() {
_ = os.RemoveAll(cfgdir)
}()
config.DefaultConfig.SetConfigDir(cfgdir)
config.DefaultConfig.SetTemplatesDir(tmpDir)
// Create template files
template1 := filepath.Join(tmpDir, "template1.yaml")
templateContent := `id: test-template
info:
name: Test Template
author: test
severity: info`
require.NoError(t, os.WriteFile(template1, []byte(templateContent), 0644))
// Empty written paths
writtenPaths := mapsutil.NewSyncLockMap[string, struct{}]()
// Run cleanup
err = tm.cleanupOrphanedTemplates(tmpDir, writtenPaths)
require.NoError(t, err)
// Verify template was removed (since it's not in written paths)
require.NoFileExists(t, template1, "template should be removed when not in written paths")
})
t.Run("handles relative and absolute paths correctly", func(t *testing.T) {
// Create temporary directories
tmpDir, err := os.MkdirTemp("", "nuclei-cleanup-path-test-*")
require.NoError(t, err)
defer func() {
_ = os.RemoveAll(tmpDir)
}()
cfgdir, err := os.MkdirTemp("", "nuclei-config-*")
require.NoError(t, err)
defer func() {
_ = os.RemoveAll(cfgdir)
}()
config.DefaultConfig.SetConfigDir(cfgdir)
config.DefaultConfig.SetTemplatesDir(tmpDir)
// Create template file
template1 := filepath.Join(tmpDir, "template1.yaml")
templateContent := `id: test-template
info:
name: Test Template
author: test
severity: info`
require.NoError(t, os.WriteFile(template1, []byte(templateContent), 0644))
// Use relative path in written paths
writtenPaths := mapsutil.NewSyncLockMap[string, struct{}]()
_ = writtenPaths.Set(template1, struct{}{}) // relative path
// Run cleanup - should normalize paths correctly
err = tm.cleanupOrphanedTemplates(tmpDir, writtenPaths)
require.NoError(t, err)
// Verify template was NOT removed (it was in written paths)
require.FileExists(t, template1, "template should be preserved when in written paths")
})
previousTemplatesDir := config.DefaultConfig.TemplatesDirectory
t.Cleanup(func() { config.DefaultConfig.SetTemplatesDir(previousTemplatesDir) })
t.Run("checksums custom dir sibling prefixes", func(t *testing.T) {
tmpDir, err := os.MkdirTemp("", "nuclei-checksum-custom-prefix-test-*")
@@ -380,13 +370,6 @@ info:
_ = os.RemoveAll(tmpDir)
}()
cfgdir, err := os.MkdirTemp("", "nuclei-config-*")
require.NoError(t, err)
defer func() {
_ = os.RemoveAll(cfgdir)
}()
config.DefaultConfig.SetConfigDir(cfgdir)
config.DefaultConfig.SetTemplatesDir(tmpDir)
customGitHubDir := filepath.Join(tmpDir, "github")
@@ -413,50 +396,39 @@ info:
require.Contains(t, checksums, siblingTemplate, "custom directory sibling prefix should be checksummed")
})
t.Run("handles empty templates directory", func(t *testing.T) {
// Create temporary directories
tmpDir, err := os.MkdirTemp("", "nuclei-cleanup-empty-dir-test-*")
t.Run("excludes internal temporary files from checksums", func(t *testing.T) {
templatesDir := t.TempDir()
config.DefaultConfig.SetTemplatesDir(templatesDir)
templateFile := filepath.Join(templatesDir, "template.yaml")
ownershipTemporary := filepath.Join(templatesDir, templateOwnershipFileName+".tmp-interrupted")
retiredQuarantine := filepath.Join(templatesDir, templateOwnershipRetiredPrefix+"interrupted")
nestedDir := filepath.Join(templatesDir, "nested")
nestedOwnershipTemporary := filepath.Join(nestedDir, templateOwnershipFileName+".tmp-user")
outputTemporary := filepath.Join(nestedDir, templateOutputTemporaryPrefix+strings.Repeat("a", 32))
restoreTemporary := filepath.Join(nestedDir, templateOutputTemporaryPrefix+strings.Repeat("b", 32)+"-"+strings.Repeat("c", 32))
restoreState := filepath.Join(templatesDir, templateOwnershipRestorePrefix+strings.Repeat("d", 64)+"-"+strings.Repeat("e", 32))
userFileWithSimilarName := filepath.Join(nestedDir, templateOutputTemporaryPrefix+"user.yaml")
require.NoError(t, os.Mkdir(nestedDir, 0755))
require.NoError(t, os.WriteFile(templateFile, []byte("template"), 0644))
require.NoError(t, os.WriteFile(ownershipTemporary, []byte("partial manifest"), 0600))
require.NoError(t, os.WriteFile(retiredQuarantine, []byte("retired template"), 0600))
require.NoError(t, os.WriteFile(nestedOwnershipTemporary, []byte("user file"), 0644))
require.NoError(t, os.WriteFile(outputTemporary, []byte("interrupted write"), 0600))
require.NoError(t, os.WriteFile(restoreTemporary, []byte("interrupted restore"), 0600))
require.NoError(t, os.WriteFile(restoreState, nil, 0600))
require.NoError(t, os.WriteFile(userFileWithSimilarName, []byte("user file"), 0644))
checksums, err := tm.calculateChecksumMap(templatesDir)
require.NoError(t, err)
defer func() {
_ = os.RemoveAll(tmpDir)
}()
cfgdir, err := os.MkdirTemp("", "nuclei-config-*")
require.NoError(t, err)
defer func() {
_ = os.RemoveAll(cfgdir)
}()
config.DefaultConfig.SetConfigDir(cfgdir)
config.DefaultConfig.SetTemplatesDir(tmpDir)
// Directory exists but is empty (user deleted all templates)
require.True(t, fileutil.FolderExists(tmpDir), "templates directory should exist")
// Written paths from new release
writtenPaths := mapsutil.NewSyncLockMap[string, struct{}]()
// Run cleanup - should handle empty directory gracefully
err = tm.cleanupOrphanedTemplates(tmpDir, writtenPaths)
require.NoError(t, err, "cleanup should handle empty directory without error")
// Directory should still exist after cleanup
require.True(t, fileutil.FolderExists(tmpDir), "templates directory should still exist")
})
t.Run("handles non-existent directory gracefully", func(t *testing.T) {
// Use a non-existent directory path
nonExistentDir := "/tmp/nuclei-test-non-existent-dir-12345"
// Ensure it doesn't exist
_ = os.RemoveAll(nonExistentDir)
require.False(t, fileutil.FolderExists(nonExistentDir), "directory should not exist")
writtenPaths := mapsutil.NewSyncLockMap[string, struct{}]()
// Run cleanup - should handle non-existent directory gracefully
err := tm.cleanupOrphanedTemplates(nonExistentDir, writtenPaths)
require.NoError(t, err, "cleanup should handle non-existent directory without error")
require.Contains(t, checksums, templateFile)
require.NotContains(t, checksums, ownershipTemporary)
require.NotContains(t, checksums, retiredQuarantine)
require.NotContains(t, checksums, outputTemporary)
require.NotContains(t, checksums, restoreTemporary)
require.NotContains(t, checksums, restoreState)
require.Contains(t, checksums, nestedOwnershipTemporary, "only root-level ownership artifacts are internal metadata")
require.Contains(t, checksums, userFileWithSimilarName, "only names matching the complete internal write pattern are excluded")
})
}
@@ -500,7 +472,7 @@ info:
require.NoError(t, os.WriteFile(template2, []byte(template2Content), 0644))
// Regenerate metadata
err = tm.regenerateTemplateMetadata(tmpDir)
_, err = tm.regenerateTemplateMetadata(config.DefaultConfig)
require.NoError(t, err)
// Verify index file was created
@@ -524,7 +496,7 @@ info:
require.Contains(t, checksums, template2, "checksum should contain template2")
})
t.Run("excludes deleted templates from index after cleanup", func(t *testing.T) {
t.Run("excludes retired templates from the regenerated index", func(t *testing.T) {
tmpDir, err := os.MkdirTemp("", "nuclei-metadata-cleanup-test-*")
require.NoError(t, err)
defer func() {
@@ -543,7 +515,7 @@ info:
// Create template files
template1 := filepath.Join(tmpDir, "kept-template.yaml")
template2 := filepath.Join(tmpDir, "deleted-template.yaml")
orphanedTemplate := filepath.Join(tmpDir, "orphaned-template.yaml")
retiredTemplate := filepath.Join(tmpDir, "retired-template.yaml")
template1Content := `id: test-template-1
info:
@@ -555,21 +527,21 @@ info:
name: Test Template 2
author: test
severity: info`
orphanedContent := `id: test-template-orphaned
retiredContent := `id: test-template-retired
info:
name: Test Template Orphaned
name: Test Template Retired
author: test
severity: info`
require.NoError(t, os.WriteFile(template1, []byte(template1Content), 0644))
require.NoError(t, os.WriteFile(template2, []byte(template2Content), 0644))
require.NoError(t, os.WriteFile(orphanedTemplate, []byte(orphanedContent), 0644))
require.NoError(t, os.WriteFile(retiredTemplate, []byte(retiredContent), 0644))
// Create initial index with all templates (simulating state before cleanup)
// Create initial index with all previously owned templates.
initialIndex := map[string]string{
"test-template-1": template1,
"test-template-2": template2,
"test-template-orphaned": orphanedTemplate,
"test-template-1": template1,
"test-template-2": template2,
"test-template-retired": retiredTemplate,
}
err = config.DefaultConfig.WriteTemplatesIndex(initialIndex)
require.NoError(t, err)
@@ -577,33 +549,25 @@ info:
// Verify initial index contains all templates
index, err := config.GetNucleiTemplatesIndex()
require.NoError(t, err)
require.Contains(t, index, "test-template-orphaned", "initial index should contain orphaned template")
require.Contains(t, index, "test-template-retired", "initial index should contain the retired template")
// Simulate cleanup: remove orphaned template
writtenPaths := mapsutil.NewSyncLockMap[string, struct{}]()
absTemplate1, _ := filepath.Abs(template1)
// Normalize path consistently (same as cleanupOrphanedTemplates does)
absTemplate1 = filepath.Clean(absTemplate1)
_ = writtenPaths.Set(absTemplate1, struct{}{})
// Arrange the post-reconciliation tree directly; reconciliation has dedicated coverage.
require.NoError(t, os.Remove(template2))
require.NoError(t, os.Remove(retiredTemplate))
err = tm.cleanupOrphanedTemplates(tmpDir, writtenPaths)
require.NoError(t, err)
require.NoFileExists(t, orphanedTemplate, "orphaned template should be deleted")
require.NoFileExists(t, template2, "template2 should be deleted since it's not in writtenPaths")
// Regenerate metadata after cleanup
err = tm.regenerateTemplateMetadata(tmpDir)
// Regenerate metadata from the arranged post-reconciliation tree.
_, err = tm.regenerateTemplateMetadata(config.DefaultConfig)
require.NoError(t, err)
// Verify index no longer contains deleted template
// The regenerated index excludes templates retired by the release.
index, err = config.GetNucleiTemplatesIndex()
require.NoError(t, err)
require.NotContains(t, index, "test-template-orphaned", "index should not contain deleted orphaned template")
require.NotContains(t, index, "test-template-retired", "index should not contain a retired template")
require.Contains(t, index, "test-template-1", "index should still contain kept template")
require.NotContains(t, index, "test-template-2", "index should not contain template that was deleted but not cleaned")
})
t.Run("excludes deleted templates from checksum after cleanup", func(t *testing.T) {
t.Run("excludes retired templates from the regenerated checksum", func(t *testing.T) {
tmpDir, err := os.MkdirTemp("", "nuclei-checksum-cleanup-test-*")
require.NoError(t, err)
defer func() {
@@ -621,7 +585,7 @@ info:
// Create template files
keptTemplate := filepath.Join(tmpDir, "kept.yaml")
orphanedTemplate := filepath.Join(tmpDir, "orphaned.yaml")
retiredTemplate := filepath.Join(tmpDir, "retired.yaml")
templateContent := `id: test-template
info:
@@ -630,40 +594,34 @@ info:
severity: info`
require.NoError(t, os.WriteFile(keptTemplate, []byte(templateContent), 0644))
require.NoError(t, os.WriteFile(orphanedTemplate, []byte(templateContent), 0644))
require.NoError(t, os.WriteFile(retiredTemplate, []byte(templateContent), 0644))
// Create initial checksum with both templates
err = tm.writeChecksumFileInDir(tmpDir)
checksumMap, err := tm.calculateChecksumMap(tmpDir)
require.NoError(t, err)
err = writeChecksumMap(config.DefaultConfig, checksumMap)
require.NoError(t, err)
// Verify initial checksum contains both templates
initialChecksums, err := tm.getChecksumFromDir(tmpDir)
require.NoError(t, err)
require.Contains(t, initialChecksums, orphanedTemplate, "initial checksum should contain orphaned template")
require.Contains(t, initialChecksums, retiredTemplate, "initial checksum should contain the retired template")
// Simulate cleanup: remove orphaned template
writtenPaths := mapsutil.NewSyncLockMap[string, struct{}]()
absKept, _ := filepath.Abs(keptTemplate)
// Normalize path consistently (same as cleanupOrphanedTemplates does)
absKept = filepath.Clean(absKept)
_ = writtenPaths.Set(absKept, struct{}{})
// Arrange the post-reconciliation tree directly; reconciliation has dedicated coverage.
require.NoError(t, os.Remove(retiredTemplate))
err = tm.cleanupOrphanedTemplates(tmpDir, writtenPaths)
require.NoError(t, err)
require.NoFileExists(t, orphanedTemplate, "orphaned template should be deleted")
// Regenerate metadata after cleanup
err = tm.regenerateTemplateMetadata(tmpDir)
// Regenerate metadata from the arranged post-reconciliation tree.
_, err = tm.regenerateTemplateMetadata(config.DefaultConfig)
require.NoError(t, err)
// Verify checksum no longer contains deleted template
// The regenerated checksum excludes templates retired by the release.
checksums, err := tm.getChecksumFromDir(tmpDir)
require.NoError(t, err)
require.NotContains(t, checksums, orphanedTemplate, "checksum should not contain deleted orphaned template")
require.NotContains(t, checksums, retiredTemplate, "checksum should not contain a retired template")
require.Contains(t, checksums, keptTemplate, "checksum should still contain kept template")
})
t.Run("cleanup and metadata regeneration integration", func(t *testing.T) {
t.Run("reconciles ownership before metadata regeneration", func(t *testing.T) {
tmpDir, err := os.MkdirTemp("", "nuclei-integration-test-*")
require.NoError(t, err)
defer func() {
@@ -682,12 +640,12 @@ info:
// Create multiple templates
template1 := filepath.Join(tmpDir, "cves", "2023", "cve1.yaml")
template2 := filepath.Join(tmpDir, "cves", "2023", "cve2.yaml")
orphaned1 := filepath.Join(tmpDir, "cves", "2022", "old-cve.yaml")
orphaned2 := filepath.Join(tmpDir, "exposures", "old-exposure.yaml")
retired1 := filepath.Join(tmpDir, "cves", "2022", "old-cve.yaml")
retired2 := filepath.Join(tmpDir, "exposures", "old-exposure.yaml")
require.NoError(t, os.MkdirAll(filepath.Dir(template1), 0755))
require.NoError(t, os.MkdirAll(filepath.Dir(orphaned1), 0755))
require.NoError(t, os.MkdirAll(filepath.Dir(orphaned2), 0755))
require.NoError(t, os.MkdirAll(filepath.Dir(retired1), 0755))
require.NoError(t, os.MkdirAll(filepath.Dir(retired2), 0755))
template1Content := `id: cve1
info:
@@ -699,12 +657,12 @@ info:
name: CVE2
author: test
severity: info`
orphaned1Content := `id: old-cve
retired1Content := `id: old-cve
info:
name: Old CVE
author: test
severity: info`
orphaned2Content := `id: old-exposure
retired2Content := `id: old-exposure
info:
name: Old Exposure
author: test
@@ -712,27 +670,26 @@ info:
require.NoError(t, os.WriteFile(template1, []byte(template1Content), 0644))
require.NoError(t, os.WriteFile(template2, []byte(template2Content), 0644))
require.NoError(t, os.WriteFile(orphaned1, []byte(orphaned1Content), 0644))
require.NoError(t, os.WriteFile(orphaned2, []byte(orphaned2Content), 0644))
require.NoError(t, os.WriteFile(retired1, []byte(retired1Content), 0644))
require.NoError(t, os.WriteFile(retired2, []byte(retired2Content), 0644))
seedTemplateOwnership(t, tmpDir, template1, template2, retired1, retired2)
// Simulate written paths from new release
writtenPaths := mapsutil.NewSyncLockMap[string, struct{}]()
// The current release retains template1 and template2.
absTemplate1, _ := filepath.Abs(template1)
absTemplate2, _ := filepath.Abs(template2)
// Normalize paths consistently (same as cleanupOrphanedTemplates does)
// Normalize paths consistently with ownership reconciliation.
absTemplate1 = filepath.Clean(absTemplate1)
absTemplate2 = filepath.Clean(absTemplate2)
_ = writtenPaths.Set(absTemplate1, struct{}{})
_ = writtenPaths.Set(absTemplate2, struct{}{})
writtenTemplates := newWrittenTemplates(t, absTemplate1, absTemplate2)
// Perform cleanup
err = tm.cleanupOrphanedTemplates(tmpDir, writtenPaths)
// Reconcile prior ownership with the current release.
err = reconcileTemplateOwnership(tmpDir, writtenTemplates)
require.NoError(t, err)
require.NoFileExists(t, orphaned1, "orphaned template 1 should be deleted")
require.NoFileExists(t, orphaned2, "orphaned template 2 should be deleted")
require.NoFileExists(t, retired1, "retired template 1 should be deleted")
require.NoFileExists(t, retired2, "retired template 2 should be deleted")
// Regenerate metadata (simulating what updateTemplatesAt does)
err = tm.regenerateTemplateMetadata(tmpDir)
_, err = tm.regenerateTemplateMetadata(config.DefaultConfig)
require.NoError(t, err)
// Verify index only contains kept templates
@@ -748,12 +705,12 @@ info:
require.NoError(t, err)
require.Contains(t, checksums, template1, "checksum should contain kept template1")
require.Contains(t, checksums, template2, "checksum should contain kept template2")
require.NotContains(t, checksums, orphaned1, "checksum should not contain deleted template")
require.NotContains(t, checksums, orphaned2, "checksum should not contain deleted template")
require.NotContains(t, checksums, retired1, "checksum should not contain deleted template")
require.NotContains(t, checksums, retired2, "checksum should not contain deleted template")
// Verify empty directories are purged
require.False(t, fileutil.FolderExists(filepath.Dir(orphaned1)), "empty directory should be purged")
require.False(t, fileutil.FolderExists(filepath.Dir(orphaned2)), "empty directory should be purged")
require.False(t, fileutil.FolderExists(filepath.Dir(retired1)), "empty directory should be purged")
require.False(t, fileutil.FolderExists(filepath.Dir(retired2)), "empty directory should be purged")
})
t.Run("handles empty templates directory", func(t *testing.T) {
@@ -776,7 +733,7 @@ info:
require.NoError(t, os.MkdirAll(tmpDir, 0755))
// Regenerate metadata on empty directory
err = tm.regenerateTemplateMetadata(tmpDir)
_, err = tm.regenerateTemplateMetadata(config.DefaultConfig)
require.NoError(t, err, "should handle empty directory without error")
// Index should exist but be empty or minimal
@@ -820,7 +777,7 @@ info:
severity: info`), 0644))
// Regenerate metadata (should purge empty directories)
err = tm.regenerateTemplateMetadata(tmpDir)
_, err = tm.regenerateTemplateMetadata(config.DefaultConfig)
require.NoError(t, err)
// Verify empty directories were purged
+95
View File
@@ -1,13 +1,25 @@
package installer
import (
"fmt"
"io"
"net/http"
"os"
"strings"
"testing"
"github.com/projectdiscovery/nuclei/v3/pkg/catalog/config"
"github.com/projectdiscovery/retryablehttp-go"
"github.com/projectdiscovery/utils/generic"
"github.com/stretchr/testify/require"
)
type roundTripFunc func(*http.Request) (*http.Response, error)
func (f roundTripFunc) RoundTrip(req *http.Request) (*http.Response, error) {
return f(req)
}
func TestVersionCheck(t *testing.T) {
ignoreHash, err := NucleiVersionCheck()
require.Nil(t, err)
@@ -17,3 +29,86 @@ func TestVersionCheck(t *testing.T) {
t.Errorf("something went wrong got empty response nuclei-version=%v templates-version=%v ignore-hash=%v", cfg.LatestNucleiVersion, cfg.LatestNucleiTemplatesVersion, ignoreHash)
}
}
func TestUpdateIgnoreFileRejectsHTMLAndPreservesExisting(t *testing.T) {
html := []byte("<!DOCTYPE HTML>\n<html>\n<head>\n<meta name=\"description\" content=\"Zscaler: blocked\">\n</head>\n</html>\n")
withIgnoreDownload(t, http.StatusOK, html)
cfg := config.DefaultConfig
root := t.TempDir()
previous := cfg.TemplatesDirectory
t.Cleanup(func() { cfg.SetTemplatesDir(previous) })
cfg.SetTemplatesDir(root)
valid := []byte("tags: [weak]\nfiles: [blocked.yaml]\n")
path := cfg.GetActiveIgnoreFilePath()
require.NoError(t, os.WriteFile(path, valid, 0o600))
err := UpdateIgnoreFile()
require.Error(t, err)
require.Contains(t, err.Error(), fmt.Sprintf("%q", path))
got, err := os.ReadFile(path)
require.NoError(t, err)
require.Equal(t, valid, got)
}
func TestUpdateIgnoreFileWritesValidContents(t *testing.T) {
payload := []byte("tags: [dos]\nfiles: [http/test.yaml]\n")
withIgnoreDownload(t, http.StatusOK, payload)
cfg := config.DefaultConfig
root := t.TempDir()
previous := cfg.TemplatesDirectory
t.Cleanup(func() { cfg.SetTemplatesDir(previous) })
cfg.SetTemplatesDir(root)
require.NoError(t, UpdateIgnoreFile())
got, err := os.ReadFile(cfg.GetActiveIgnoreFilePath())
require.NoError(t, err)
require.Equal(t, payload, got)
}
func TestUpdateIgnoreFileRejectsUnexpectedStatus(t *testing.T) {
withIgnoreDownload(t, http.StatusForbidden, []byte("tags: [weak]\n"))
cfg := config.DefaultConfig
root := t.TempDir()
previous := cfg.TemplatesDirectory
t.Cleanup(func() { cfg.SetTemplatesDir(previous) })
cfg.SetTemplatesDir(root)
err := UpdateIgnoreFile()
require.Error(t, err)
require.Contains(t, err.Error(), "unexpected HTTP status")
_, statErr := os.Stat(cfg.GetActiveIgnoreFilePath())
require.ErrorIs(t, statErr, os.ErrNotExist)
}
func withIgnoreDownload(t *testing.T, status int, body []byte) {
t.Helper()
previous := retryableHttpClient
t.Cleanup(func() { retryableHttpClient = previous })
retryableHttpClient = retryablehttp.NewClient(retryablehttp.Options{
RetryMax: 0,
HttpClient: &http.Client{
Transport: roundTripFunc(func(req *http.Request) (*http.Response, error) {
header := make(http.Header)
header.Set("Content-Type", "text/plain")
return &http.Response{
StatusCode: status,
Status: http.StatusText(status),
Body: io.NopCloser(strings.NewReader(string(body))),
Header: header,
Request: req,
Proto: "HTTP/1.1",
ProtoMajor: 1,
ProtoMinor: 1,
}, nil
}),
},
})
}
+8 -8
View File
@@ -64,19 +64,19 @@ func TestZipSlip(t *testing.T) {
for _, filePathFromZip := range filePathsFromZip {
var tmp fs.FileInfo = &tempFileInfo{name: filePathFromZip}
writePath := tm.getAbsoluteFilePath(configuredTemplateDirectory, filePathFromZip, tmp)
_, writePath := tm.getTemplateOutputLocation(configuredTemplateDirectory, filePathFromZip, tmp)
require.Equal(t, "", writePath, filePathFromZip)
}
})
t.Run("positive no-slash fallback", func(t *testing.T) {
// Entry names with no slash exercise the fallback branch in
// getAbsoluteFilePath. Legitimate single-name entries must still be
// getTemplateOutputLocation. Legitimate single-name entries must still be
// written into templateDir (regression test for the containment check
// added to that branch).
tm := TemplateManager{}
var tmp fs.FileInfo = &tempFileInfo{name: "single-file.yaml"}
writePath := tm.getAbsoluteFilePath(configuredTemplateDirectory, "single-file.yaml", tmp)
_, writePath := tm.getTemplateOutputLocation(configuredTemplateDirectory, "single-file.yaml", tmp)
require.Equal(t, filepath.Join(configuredTemplateDirectory, "single-file.yaml"), writePath)
})
@@ -89,7 +89,7 @@ func TestZipSlip(t *testing.T) {
for filePathFromZip, expectedWritePath := range filePathsFromZip {
var tmp fs.FileInfo = &tempFileInfo{name: filePathFromZip}
writePath := tm.getAbsoluteFilePath(configuredTemplateDirectory, filePathFromZip, tmp)
_, writePath := tm.getTemplateOutputLocation(configuredTemplateDirectory, filePathFromZip, tmp)
require.Equal(t, expectedWritePath, writePath, filePathFromZip)
}
})
@@ -101,7 +101,7 @@ func TestZipSlip(t *testing.T) {
tm := TemplateManager{}
var tmp fs.FileInfo = &tempFileInfo{name: "nuclei-templates/cves/test.yaml"}
writePath := tm.getAbsoluteFilePath(aliasDir, "nuclei-templates/cves/test.yaml", tmp)
_, writePath := tm.getTemplateOutputLocation(aliasDir, "nuclei-templates/cves/test.yaml", tmp)
require.Equal(t, filepath.Join(aliasDir, "cves", "test.yaml"), writePath)
})
@@ -109,7 +109,7 @@ func TestZipSlip(t *testing.T) {
// path component is itself a symlink that points outside the configured
// templates directory. Because both the entry's leaf and its parent on
// disk may be missing, a naive lexical containment check could miss the
// escape. canonicalizePath inside IsPathWithinDirectory walks up to the
// escape. canonicalizePath inside IsPathWithinDirectory walks up to the
// nearest existing ancestor (the symlink) and resolves it, which is what
// makes this rejection sound. This test pins that behavior.
t.Run("negative symlinked ancestor escapes templateDir", func(t *testing.T) {
@@ -117,7 +117,7 @@ func TestZipSlip(t *testing.T) {
outsideDir := t.TempDir()
// Plant a symlink "evil" inside templateDir that points outside.
// A malicious zip entry that traverses through it must be rejected
// before it ever reaches WriteFile / CreateFolder.
// before it reaches the root-confined write boundary.
require.NoError(t, os.Symlink(outsideDir, filepath.Join(templateDir, "evil")))
tm := TemplateManager{}
@@ -128,7 +128,7 @@ func TestZipSlip(t *testing.T) {
}
for _, entry := range entries {
var tmp fs.FileInfo = &tempFileInfo{name: entry}
writePath := tm.getAbsoluteFilePath(templateDir, entry, tmp)
_, writePath := tm.getTemplateOutputLocation(templateDir, entry, tmp)
require.Equal(t, "", writePath, entry)
}
})
+9
View File
@@ -45,6 +45,15 @@ type ExecuteOptions struct {
TimeoutVariants *types.Timeouts
// ProxyURL is the HTTP(S) proxy URL for JS net dials via HTTP CONNECT.
// SOCKS proxies are handled by fastdialer and must not be set here.
ProxyURL string
// CustomHeaders contains global CLI headers for the nuclei/http module.
// Raw nuclei/net connections construct their own wire payloads and do not
// consume these headers.
CustomHeaders []string
// Manually exported objects
exports map[string]interface{}
}
+18 -1
View File
@@ -137,8 +137,22 @@ func (s *session) prepareCommon() {
_ = s.config.runtime.Set(k, v)
}
executionCtx := s.config.ctx
if s.config.opts.TimeoutVariants != nil {
s.config.runtime.SetContextValue("timeoutVariants", s.config.opts.TimeoutVariants)
executionCtx = context.WithValue(executionCtx, "timeoutVariants", s.config.opts.TimeoutVariants) //nolint:staticcheck
}
if s.config.opts.ProxyURL != "" {
s.config.runtime.SetContextValue("proxyURL", s.config.opts.ProxyURL)
executionCtx = context.WithValue(executionCtx, "proxyURL", s.config.opts.ProxyURL) //nolint:staticcheck
}
if len(s.config.opts.CustomHeaders) > 0 {
headers := append([]string(nil), s.config.opts.CustomHeaders...)
s.config.runtime.SetContextValue("customHeaders", headers)
executionCtx = context.WithValue(executionCtx, "customHeaders", headers) //nolint:staticcheck
}
s.config.runtime.SetContextValue("executionId", s.config.opts.ExecutionId)
s.config.runtime.SetContextValue("ctx", s.config.ctx)
s.config.runtime.SetContextValue("ctx", executionCtx)
enableRequire(s.config.runtime)
}
@@ -207,6 +221,9 @@ func (s *session) cleanupCommon() {
}
s.config.runtime.RemoveContextValue("executionId")
s.config.runtime.RemoveContextValue("ctx")
s.config.runtime.RemoveContextValue("timeoutVariants")
s.config.runtime.RemoveContextValue("proxyURL")
s.config.runtime.RemoveContextValue("customHeaders")
}
func (s *session) cleanupPath() {
+107
View File
@@ -0,0 +1,107 @@
package compiler
import (
"context"
"io"
"net/http"
"net/http/httptest"
"testing"
"time"
"github.com/projectdiscovery/goja"
"github.com/projectdiscovery/nuclei/v3/pkg/protocols/common/protocolstate"
"github.com/projectdiscovery/nuclei/v3/pkg/types"
"github.com/stretchr/testify/require"
)
func TestSessionInjectsProxyAndTimeoutContext(t *testing.T) {
runtime := goja.New()
program, err := goja.Compile("test.js", `
function run() {
return { success: true };
}
run();
`, false)
require.NoError(t, err)
var sawProxy string
var sawTimeout time.Duration
var sawHeaders []string
proxyURL := "http://127.0.0.1:8080"
customHeaders := []string{"X-Global: value"}
timeouts := &types.Timeouts{
TcpReadTimeout: 12 * time.Second,
JsCompilerExecutionTimeout: 5 * time.Second,
}
opts := &ExecuteOptions{
ExecutionId: "session-proxy-test",
ProxyURL: proxyURL,
CustomHeaders: customHeaders,
TimeoutVariants: timeouts,
Callback: func(rt *goja.Runtime) error {
if v, ok := rt.GetContextValue("proxyURL"); ok {
sawProxy, _ = v.(string)
}
if v, ok := rt.GetContextValue("timeoutVariants"); ok {
if tv, ok := v.(*types.Timeouts); ok && tv != nil {
sawTimeout = tv.TcpReadTimeout
}
}
if v, ok := rt.GetContextValue("customHeaders"); ok {
sawHeaders, _ = v.([]string)
}
if v, ok := rt.GetContextValue("ctx"); ok {
executionCtx, _ := v.(context.Context)
require.Equal(t, proxyURL, executionCtx.Value("proxyURL"))
require.Equal(t, timeouts, executionCtx.Value("timeoutVariants"))
require.Equal(t, customHeaders, executionCtx.Value("customHeaders"))
}
return nil
},
}
_, err = executeWithRuntime(context.Background(), runtime, program, NewExecuteArgs(), opts, nil)
require.NoError(t, err)
require.Equal(t, "http://127.0.0.1:8080", sawProxy)
require.Equal(t, 12*time.Second, sawTimeout)
require.Equal(t, customHeaders, sawHeaders)
_, ok := runtime.GetContextValue("proxyURL")
require.False(t, ok, "proxyURL should be cleaned up after execution")
_, ok = runtime.GetContextValue("timeoutVariants")
require.False(t, ok, "timeoutVariants should be cleaned up after execution")
_, ok = runtime.GetContextValue("customHeaders")
require.False(t, ok, "customHeaders should be cleaned up after execution")
}
func TestSessionAppliesOptionsToJSHTTP(t *testing.T) {
proxy := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
require.Equal(t, "example.com", r.Host)
_, _ = io.WriteString(w, r.Header.Get("X-Global"))
}))
t.Cleanup(proxy.Close)
executionID := "session-js-http"
require.NoError(t, protocolstate.Init(&types.Options{ExecutionId: executionID}))
t.Cleanup(func() { protocolstate.Close(executionID) })
source := `
const http = require("nuclei/http");
http.Get("http://example.com/through-proxy").Body;
`
program, err := goja.Compile("test.js", source, false)
require.NoError(t, err)
result, err := executeWithRuntime(t.Context(), createNewRuntime(), program, NewExecuteArgs(), &ExecuteOptions{
ExecutionId: executionID,
ProxyURL: proxy.URL,
CustomHeaders: []string{"X-Global: inherited"},
TimeoutVariants: &types.Timeouts{
HttpTimeout: 2 * time.Second,
JsCompilerExecutionTimeout: 5 * time.Second,
},
}, nil)
require.NoError(t, err)
require.Equal(t, "inherited", result.String())
}
+2 -7
View File
@@ -14,6 +14,7 @@ import (
"github.com/logrusorgru/aurora/v4"
"github.com/projectdiscovery/gologger"
"github.com/projectdiscovery/nuclei/v3/pkg/js/gojs"
jsnet "github.com/projectdiscovery/nuclei/v3/pkg/js/libs/net"
"github.com/projectdiscovery/nuclei/v3/pkg/protocols/common/protocolstate"
"github.com/projectdiscovery/nuclei/v3/pkg/protocols/common/utils/vardump"
"github.com/projectdiscovery/nuclei/v3/pkg/types"
@@ -127,13 +128,7 @@ func initBuiltInFunc(runtime *goja.Runtime) {
return false, errkit.New("isPortOpen: host or port is empty")
}
executionId := ctx.Value("executionId").(string)
dialer := protocolstate.GetDialersWithId(executionId)
if dialer == nil {
panic("dialers with executionId " + executionId + " not found")
}
conn, err := dialer.Fastdialer.Dial(ctx, "tcp", net.JoinHostPort(host, port))
conn, err := jsnet.Dial(ctx, "tcp", net.JoinHostPort(host, port))
if err != nil {
return false, err
}
+90 -8
View File
@@ -22,6 +22,7 @@ import (
"github.com/projectdiscovery/goja"
"github.com/projectdiscovery/nuclei/v3/pkg/js/utils"
"github.com/projectdiscovery/nuclei/v3/pkg/protocols/common/protocolstate"
"github.com/projectdiscovery/nuclei/v3/pkg/types"
"golang.org/x/net/publicsuffix"
)
@@ -87,9 +88,9 @@ func NewClient(call goja.ConstructorCall, runtime *goja.Runtime) *goja.Object {
nj: utils.NewNucleiJS(runtime),
FollowRedirects: true,
MaxRedirects: defaultMaxRedirects,
TimeoutSeconds: defaultTimeoutSeconds,
TimeoutSeconds: timeoutSecondsFromRuntime(runtime),
MaxBodyBytes: defaultMaxBodyBytes,
headers: make(http.Header),
headers: customHeadersFromRuntime(runtime),
}
c.nj.ObjectSig = "Client(options?)"
@@ -230,7 +231,7 @@ func Post(ctx context.Context, rawURL, body string) (*Response, error) {
// @example
// ```javascript
// const http = require('nuclei/http');
// const resp = http.Request('OPTIONS', 'https://example.com', '');
// const resp = http.Request("OPTIONS", "https://example.com", "");
// ```
func Request(ctx context.Context, method, rawURL, body string) (*Response, error) {
return defaultClient().Request(ctx, method, rawURL, body)
@@ -240,13 +241,14 @@ func defaultClient() *Client {
return &Client{
FollowRedirects: true,
MaxRedirects: defaultMaxRedirects,
TimeoutSeconds: defaultTimeoutSeconds,
TimeoutSeconds: 0,
MaxBodyBytes: defaultMaxBodyBytes,
headers: make(http.Header),
}
}
func (c *Client) do(ctx context.Context, method, rawURL, body string) (*Response, error) {
timeout := requestTimeout(ctx, c.TimeoutSeconds)
c.init()
executionID := executionIDFrom(ctx, c)
@@ -281,8 +283,12 @@ func (c *Client) do(ctx context.Context, method, rawURL, body string) (*Response
tlsConfig.ServerName = host
}
proxy, err := proxyFromContext(ctx)
if err != nil {
return nil, err
}
transport := &http.Transport{
Proxy: http.ProxyFromEnvironment,
Proxy: proxy,
DialContext: func(ctx context.Context, network, addr string) (net.Conn, error) {
return dialers.Fastdialer.Dial(ctx, network, addr)
},
@@ -294,12 +300,13 @@ func (c *Client) do(ctx context.Context, method, rawURL, body string) (*Response
MaxIdleConns: 4,
MaxIdleConnsPerHost: 4,
IdleConnTimeout: 30 * time.Second,
ResponseHeaderTimeout: time.Duration(c.TimeoutSeconds) * time.Second,
ResponseHeaderTimeout: timeout,
}
defer transport.CloseIdleConnections()
httpClient := &http.Client{
Transport: transport,
Timeout: time.Duration(c.TimeoutSeconds) * time.Second,
Timeout: timeout,
}
if c.jar != nil {
httpClient.Jar = c.jar
@@ -330,9 +337,10 @@ func (c *Client) do(ctx context.Context, method, rawURL, body string) (*Response
if err != nil {
return nil, err
}
applyCustomHeaders(req.Header, customHeadersFromContext(ctx))
for k, vals := range c.headers {
for _, v := range vals {
req.Header.Add(k, v)
req.Header.Set(k, v)
}
}
if req.Header.Get("User-Agent") == "" {
@@ -366,6 +374,80 @@ func (c *Client) do(ctx context.Context, method, rawURL, body string) (*Response
return out, nil
}
func timeoutSecondsFromRuntime(runtime *goja.Runtime) int {
if runtime != nil {
if value, ok := runtime.GetContextValue("timeoutVariants"); ok {
if timeouts, ok := value.(*types.Timeouts); ok && timeouts != nil && timeouts.HttpTimeout > 0 {
return int(timeouts.HttpTimeout / time.Second)
}
}
}
return defaultTimeoutSeconds
}
func requestTimeout(ctx context.Context, seconds int) time.Duration {
if seconds > 0 {
return time.Duration(seconds) * time.Second
}
if ctx != nil {
if timeouts, ok := ctx.Value("timeoutVariants").(*types.Timeouts); ok && timeouts != nil && timeouts.HttpTimeout > 0 {
return timeouts.HttpTimeout
}
}
return defaultTimeoutSeconds * time.Second
}
func proxyFromContext(ctx context.Context) (func(*http.Request) (*url.URL, error), error) {
if ctx == nil {
return http.ProxyFromEnvironment, nil
}
raw, _ := ctx.Value("proxyURL").(string)
if raw == "" {
return http.ProxyFromEnvironment, nil
}
proxyURL, err := url.Parse(raw)
if err != nil {
return nil, fmt.Errorf("http: invalid proxy URL: %w", err)
}
if proxyURL.Host == "" || (proxyURL.Scheme != "http" && proxyURL.Scheme != "https") {
return nil, fmt.Errorf("http: proxy URL must use http or https")
}
return http.ProxyURL(proxyURL), nil
}
func customHeadersFromRuntime(runtime *goja.Runtime) http.Header {
headers := make(http.Header)
if runtime == nil {
return headers
}
if value, ok := runtime.GetContextValue("customHeaders"); ok {
if raw, ok := value.([]string); ok {
applyCustomHeaders(headers, raw)
}
}
return headers
}
func customHeadersFromContext(ctx context.Context) []string {
if ctx == nil {
return nil
}
headers, _ := ctx.Value("customHeaders").([]string)
return headers
}
func applyCustomHeaders(headers http.Header, rawHeaders []string) {
for _, raw := range rawHeaders {
key, value, ok := strings.Cut(raw, ":")
key = strings.TrimSpace(key)
value = strings.TrimSpace(value)
if !ok || key == "" || value == "" {
continue
}
headers.Set(key, value)
}
}
func executionIDFrom(ctx context.Context, c *Client) string {
if c != nil && c.nj != nil {
if id := c.nj.ExecutionId(); id != "" {
+108
View File
@@ -9,6 +9,7 @@ import (
"net/http/httptest"
"strings"
"testing"
"time"
"github.com/projectdiscovery/goja"
"github.com/projectdiscovery/nuclei/v3/pkg/protocols/common/protocolstate"
@@ -192,6 +193,113 @@ func TestClientSetHeader(t *testing.T) {
require.Equal(t, "from-client", resp.Body)
}
func TestClientInheritsGlobalHeaders(t *testing.T) {
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
_, _ = io.WriteString(w, r.Header.Get("X-Global")+"|"+r.Header.Get("X-Override"))
}))
t.Cleanup(srv.Close)
executionID := initExec(t, &types.Options{})
client := newRuntimeClient(t, executionID, Options{})
client.SetHeader("X-Override", "client")
ctx := withExec(executionID)
ctx = context.WithValue(ctx, "customHeaders", []string{ //nolint:staticcheck
"X-Global: from-cli",
"X-Override: global",
"invalid",
})
resp, err := client.Get(ctx, srv.URL)
require.NoError(t, err)
require.Equal(t, "from-cli|client", resp.Body)
}
func TestClientUsesConfiguredProxy(t *testing.T) {
proxy := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
require.Equal(t, "example.com", r.Host)
require.Equal(t, "/through-proxy", r.URL.Path)
_, _ = io.WriteString(w, "proxied")
}))
t.Cleanup(proxy.Close)
executionID := initExec(t, &types.Options{})
client := newRuntimeClient(t, executionID, Options{})
ctx := withExec(executionID)
ctx = context.WithValue(ctx, "proxyURL", proxy.URL) //nolint:staticcheck
resp, err := client.Get(ctx, "http://example.com/through-proxy")
require.NoError(t, err)
require.Equal(t, "proxied", resp.Body)
}
func TestClientUsesConfiguredProxyForHTTPS(t *testing.T) {
target := httptest.NewTLSServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
_, _ = io.WriteString(w, "secure-proxied")
}))
t.Cleanup(target.Close)
proxy := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
require.Equal(t, http.MethodConnect, r.Method)
upstream, err := net.Dial("tcp", r.Host)
require.NoError(t, err)
defer func() { _ = upstream.Close() }()
hijacker, ok := w.(http.Hijacker)
require.True(t, ok)
client, rw, err := hijacker.Hijack()
require.NoError(t, err)
defer func() { _ = client.Close() }()
_, err = rw.WriteString("HTTP/1.1 200 Connection Established\r\n\r\n")
require.NoError(t, err)
require.NoError(t, rw.Flush())
done := make(chan struct{})
go func() {
_, _ = io.Copy(upstream, client)
close(done)
}()
_, _ = io.Copy(client, upstream)
<-done
}))
t.Cleanup(proxy.Close)
executionID := initExec(t, &types.Options{})
client := newRuntimeClient(t, executionID, Options{})
ctx := withExec(executionID)
ctx = context.WithValue(ctx, "proxyURL", proxy.URL) //nolint:staticcheck
resp, err := client.Get(ctx, target.URL)
require.NoError(t, err)
require.Equal(t, "secure-proxied", resp.Body)
}
func TestClientRejectsInvalidConfiguredProxy(t *testing.T) {
executionID := initExec(t, &types.Options{})
client := newRuntimeClient(t, executionID, Options{})
ctx := withExec(executionID)
ctx = context.WithValue(ctx, "proxyURL", "socks5://127.0.0.1:1080") //nolint:staticcheck
_, err := client.Get(ctx, "http://example.com/")
require.ErrorContains(t, err, "must use http or https")
}
func TestClientUsesConfiguredHTTPTimeout(t *testing.T) {
timeouts := &types.Timeouts{HttpTimeout: 7 * time.Second}
ctx := context.WithValue(context.Background(), "timeoutVariants", timeouts) //nolint:staticcheck
require.Equal(t, 7*time.Second, requestTimeout(ctx, 0))
require.Equal(t, 2*time.Second, requestTimeout(ctx, 2), "client option must override the global timeout")
runtime := goja.New()
runtime.SetContextValue("executionId", "timeout-test")
runtime.SetContextValue("timeoutVariants", timeouts)
obj, err := runtime.New(runtime.ToValue(NewClient))
require.NoError(t, err)
client, ok := obj.Export().(*Client)
require.True(t, ok)
require.Equal(t, 7, client.TimeoutSeconds)
}
func TestClientHead(t *testing.T) {
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
require.Equal(t, http.MethodHead, r.Method)
+184 -11
View File
@@ -1,11 +1,15 @@
package net
import (
"bufio"
"context"
"crypto/tls"
"encoding/base64"
"encoding/hex"
"fmt"
"net"
"net/http"
"net/url"
"time"
"github.com/projectdiscovery/nuclei/v3/pkg/protocols/common/protocolstate"
@@ -18,6 +22,162 @@ var (
defaultTimeout = time.Duration(5) * time.Second
)
// getTimeoutFromContext returns the configured TCP timeout or defaultTimeout.
func getTimeoutFromContext(ctx context.Context) time.Duration {
if tv, ok := ctx.Value("timeoutVariants").(*types.Timeouts); ok && tv != nil {
if tv.TcpReadTimeout > 0 {
return tv.TcpReadTimeout
}
}
return defaultTimeout
}
func proxyURLFromContext(ctx context.Context) string {
if proxyURL, ok := ctx.Value("proxyURL").(string); ok {
return proxyURL
}
return ""
}
func executionIDFromContext(ctx context.Context) (string, error) {
executionId, _ := ctx.Value("executionId").(string)
if executionId == "" {
return "", fmt.Errorf("executionId missing from context")
}
return executionId, nil
}
type dialFunc func(ctx context.Context, network, address string) (net.Conn, error)
// dialHTTPProxy establishes a TCP connection through an HTTP CONNECT proxy.
func dialHTTPProxy(ctx context.Context, dial dialFunc, proxyURL string, address string, timeout time.Duration) (net.Conn, error) {
parsed, err := url.Parse(proxyURL)
if err != nil {
return nil, fmt.Errorf("invalid proxy URL: %w", err)
}
proxyAddr := parsed.Host
if _, _, splitErr := net.SplitHostPort(proxyAddr); splitErr != nil {
defaultPort := "8080"
if parsed.Scheme == "https" {
defaultPort = "443"
}
proxyAddr = net.JoinHostPort(proxyAddr, defaultPort)
}
proxyConn, err := dial(ctx, "tcp", proxyAddr)
if err != nil {
return nil, fmt.Errorf("could not connect to proxy %s: %w", proxyAddr, err)
}
if parsed.Scheme == "https" {
config := &tls.Config{InsecureSkipVerify: true, MinVersion: tls.VersionTLS10}
if hostname := parsed.Hostname(); hostname != "" {
config.ServerName = hostname
}
tlsConn := tls.Client(proxyConn, config)
if err := tlsConn.HandshakeContext(ctx); err != nil {
_ = proxyConn.Close()
return nil, fmt.Errorf("could not establish TLS connection to proxy %s: %w", proxyAddr, err)
}
proxyConn = tlsConn
}
connectReq := &http.Request{
Method: http.MethodConnect,
URL: &url.URL{Opaque: address},
Host: address,
Header: make(http.Header),
}
if parsed.User != nil {
connectReq.Header.Set("Proxy-Authorization", "Basic "+basicAuth(parsed.User))
}
_ = proxyConn.SetDeadline(time.Now().Add(timeout))
if err := connectReq.Write(proxyConn); err != nil {
_ = proxyConn.Close()
return nil, fmt.Errorf("could not send CONNECT request: %w", err)
}
br := bufio.NewReader(proxyConn)
resp, err := http.ReadResponse(br, connectReq)
if err != nil {
_ = proxyConn.Close()
return nil, fmt.Errorf("could not read CONNECT response: %w", err)
}
_ = resp.Body.Close()
if resp.StatusCode != http.StatusOK {
_ = proxyConn.Close()
return nil, fmt.Errorf("proxy CONNECT returned status %d", resp.StatusCode)
}
// reset deadline after successful CONNECT
_ = proxyConn.SetDeadline(time.Time{})
// preserve any bytes the buffered reader consumed past the CONNECT response
if br.Buffered() > 0 {
return &bufferedConn{Conn: proxyConn, reader: br}, nil
}
return proxyConn, nil
}
// basicAuth encodes proxy user credentials.
func basicAuth(user *url.Userinfo) string {
username := user.Username()
password, _ := user.Password()
return base64.StdEncoding.EncodeToString([]byte(username + ":" + password))
}
func redactProxyURL(proxyURL string) string {
parsed, err := url.Parse(proxyURL)
if err != nil {
return "<invalid proxy URL>"
}
parsed.User = nil
return parsed.String()
}
// bufferedConn preserves bytes buffered during the CONNECT handshake.
type bufferedConn struct {
net.Conn
reader *bufio.Reader
}
func (c *bufferedConn) Read(b []byte) (int, error) {
return c.reader.Read(b)
}
// Dial opens a connection using the execution dialer.
// When an HTTP(S) proxy is present in context, TCP dials use HTTP CONNECT.
// Misconfigured proxies return an error instead of silently dialing direct.
func Dial(ctx context.Context, protocol, address string) (net.Conn, error) {
executionId, err := executionIDFromContext(ctx)
if err != nil {
return nil, err
}
dialer := protocolstate.GetDialersWithId(executionId)
if dialer == nil {
return nil, fmt.Errorf("dialers not initialized for %s", executionId)
}
if proxyURL := proxyURLFromContext(ctx); proxyURL != "" {
if protocol != "tcp" {
return nil, fmt.Errorf("http proxy does not support protocol %q", protocol)
}
parsed, err := url.Parse(proxyURL)
if err != nil {
return nil, fmt.Errorf("invalid proxy URL %s: %w", redactProxyURL(proxyURL), err)
}
switch parsed.Scheme {
case "http", "https":
return dialHTTPProxy(ctx, dialer.Fastdialer.Dial, proxyURL, address, getTimeoutFromContext(ctx))
default:
return nil, fmt.Errorf("unsupported proxy scheme %q in %s (expected http or https)", parsed.Scheme, redactProxyURL(proxyURL))
}
}
return dialer.Fastdialer.Dial(ctx, protocol, address)
}
// Open opens a new connection to the address with a timeout.
// supported protocols: tcp, udp
// @example
@@ -26,19 +186,15 @@ var (
// const conn = net.Open('tcp', 'acme.com:80');
// ```
func Open(ctx context.Context, protocol, address string) (*NetConn, error) {
executionId := ctx.Value("executionId").(string)
dialer := protocolstate.GetDialersWithId(executionId)
if dialer == nil {
return nil, fmt.Errorf("dialers not initialized for %s", executionId)
}
conn, err := dialer.Fastdialer.Dial(ctx, protocol, address)
timeout := getTimeoutFromContext(ctx)
conn, err := Dial(ctx, protocol, address)
if err != nil {
return nil, err
}
return &NetConn{conn: conn, timeout: defaultTimeout}, nil
return &NetConn{conn: conn, timeout: timeout}, nil
}
// Open opens a new connection to the address with a timeout.
// OpenTLS opens a new TLS connection to the address with a timeout.
// supported protocols: tcp, udp
// @example
// ```javascript
@@ -46,6 +202,7 @@ func Open(ctx context.Context, protocol, address string) (*NetConn, error) {
// const conn = net.OpenTLS('tcp', 'acme.com:443');
// ```
func OpenTLS(ctx context.Context, protocol, address string) (*NetConn, error) {
timeout := getTimeoutFromContext(ctx)
config := &tls.Config{InsecureSkipVerify: true, MinVersion: tls.VersionTLS10}
host, _, _ := net.SplitHostPort(address)
if host != "" {
@@ -53,17 +210,33 @@ func OpenTLS(ctx context.Context, protocol, address string) (*NetConn, error) {
c.ServerName = host
config = c
}
executionId := ctx.Value("executionId").(string)
if proxyURLFromContext(ctx) != "" {
conn, err := Dial(ctx, protocol, address)
if err != nil {
return nil, err
}
tlsConn := tls.Client(conn, config)
if err := tlsConn.HandshakeContext(ctx); err != nil {
_ = conn.Close()
return nil, fmt.Errorf("TLS handshake failed: %w", err)
}
return &NetConn{conn: tlsConn, timeout: timeout}, nil
}
executionId, err := executionIDFromContext(ctx)
if err != nil {
return nil, err
}
dialer := protocolstate.GetDialersWithId(executionId)
if dialer == nil {
return nil, fmt.Errorf("dialers not initialized for %s", executionId)
}
conn, err := dialer.Fastdialer.DialTLSWithConfig(ctx, protocol, address, config)
if err != nil {
return nil, err
}
return &NetConn{conn: conn, timeout: defaultTimeout}, nil
return &NetConn{conn: conn, timeout: timeout}, nil
}
type (
+333
View File
@@ -0,0 +1,333 @@
package net
import (
"bufio"
"bytes"
"context"
"fmt"
"io"
"net"
"net/http"
"net/http/httptest"
"net/url"
"testing"
"time"
"github.com/projectdiscovery/nuclei/v3/pkg/protocols/common/protocolstate"
"github.com/projectdiscovery/nuclei/v3/pkg/types"
"github.com/stretchr/testify/require"
)
func TestGetTimeoutFromContext(t *testing.T) {
t.Run("default when missing", func(t *testing.T) {
require.Equal(t, defaultTimeout, getTimeoutFromContext(context.Background()))
})
t.Run("reads TcpReadTimeout", func(t *testing.T) {
expected := 15 * time.Second
tv := &types.Timeouts{TcpReadTimeout: expected}
ctx := context.WithValue(context.Background(), "timeoutVariants", tv) // nolint: staticcheck
require.Equal(t, expected, getTimeoutFromContext(ctx))
})
}
func TestBufferedConn(t *testing.T) {
t.Run("read drains buffered bytes first", func(t *testing.T) {
underlying := &bytes.Buffer{}
underlying.WriteString("hello world")
br := bufio.NewReader(underlying)
_, _ = br.Peek(5)
conn := &bufferedConn{
Conn: &fakeConn{},
reader: br,
}
buf := make([]byte, 64)
n, err := conn.Read(buf)
require.NoError(t, err)
require.Equal(t, "hello world", string(buf[:n]))
})
t.Run("read returns EOF after buffer is drained", func(t *testing.T) {
underlying := bytes.NewBufferString("data")
br := bufio.NewReader(underlying)
conn := &bufferedConn{Conn: &fakeConn{}, reader: br}
buf := make([]byte, 64)
n, err := conn.Read(buf)
require.NoError(t, err)
require.Equal(t, "data", string(buf[:n]))
_, err = conn.Read(buf)
require.ErrorIs(t, err, io.EOF)
})
}
// directDial is a plain net.Dialer for tests (no fastdialer dependency).
var directDial = (&net.Dialer{Timeout: 5 * time.Second}).DialContext
func TestDialHTTPProxy(t *testing.T) {
t.Run("successful CONNECT tunnel", func(t *testing.T) {
ln, err := net.Listen("tcp", "127.0.0.1:0")
require.NoError(t, err)
defer func() { _ = ln.Close() }()
targetPayload := "hello from target"
go func() {
conn, err := ln.Accept()
if err != nil {
return
}
defer func() { _ = conn.Close() }()
req, err := http.ReadRequest(bufio.NewReader(conn))
if err != nil {
return
}
_ = req.Body.Close()
_, _ = conn.Write([]byte("HTTP/1.1 200 Connection Established\r\n\r\n"))
_, _ = conn.Write([]byte(targetPayload))
}()
ctx := context.Background()
proxyURL := fmt.Sprintf("http://%s", ln.Addr().String())
conn, err := dialHTTPProxy(ctx, directDial, proxyURL, "example.com:443", 5*time.Second)
require.NoError(t, err)
defer func() { _ = conn.Close() }()
buf := make([]byte, 64)
n, err := conn.Read(buf)
require.NoError(t, err)
require.Equal(t, targetPayload, string(buf[:n]))
})
t.Run("proxy returns non-200 status", func(t *testing.T) {
ln, err := net.Listen("tcp", "127.0.0.1:0")
require.NoError(t, err)
defer func() { _ = ln.Close() }()
go func() {
conn, err := ln.Accept()
if err != nil {
return
}
defer func() { _ = conn.Close() }()
req, err := http.ReadRequest(bufio.NewReader(conn))
if err != nil {
return
}
_ = req.Body.Close()
_, _ = conn.Write([]byte("HTTP/1.1 403 Forbidden\r\n\r\n"))
}()
ctx := context.Background()
proxyURL := fmt.Sprintf("http://%s", ln.Addr().String())
_, err = dialHTTPProxy(ctx, directDial, proxyURL, "example.com:443", 5*time.Second)
require.Error(t, err)
require.Contains(t, err.Error(), "403")
})
t.Run("invalid proxy URL", func(t *testing.T) {
ctx := context.Background()
_, err := dialHTTPProxy(ctx, directDial, "://bad-url", "example.com:443", 5*time.Second)
require.Error(t, err)
})
t.Run("successful HTTPS CONNECT tunnel", func(t *testing.T) {
targetPayload := "hello from secure proxy"
proxy := httptest.NewTLSServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.Method != http.MethodConnect {
http.Error(w, "expected CONNECT", http.StatusMethodNotAllowed)
return
}
hijacker, ok := w.(http.Hijacker)
if !ok {
http.Error(w, "hijacking not supported", http.StatusInternalServerError)
return
}
conn, rw, err := hijacker.Hijack()
if err != nil {
return
}
defer func() { _ = conn.Close() }()
_, _ = rw.WriteString("HTTP/1.1 200 Connection Established\r\n\r\n")
_, _ = rw.WriteString(targetPayload)
_ = rw.Flush()
}))
defer proxy.Close()
ctx := context.Background()
conn, err := dialHTTPProxy(ctx, directDial, proxy.URL, "example.com:443", 5*time.Second)
require.NoError(t, err)
defer func() { _ = conn.Close() }()
buf := make([]byte, 64)
n, err := conn.Read(buf)
require.NoError(t, err)
require.Equal(t, targetPayload, string(buf[:n]))
})
t.Run("unreachable proxy", func(t *testing.T) {
ctx := context.Background()
_, err := dialHTTPProxy(ctx, directDial, "http://127.0.0.1:1", "example.com:443", 1*time.Second)
require.Error(t, err)
})
t.Run("proxy auth header", func(t *testing.T) {
ln, err := net.Listen("tcp", "127.0.0.1:0")
require.NoError(t, err)
defer func() { _ = ln.Close() }()
gotAuth := make(chan string, 1)
go func() {
conn, err := ln.Accept()
if err != nil {
return
}
defer func() { _ = conn.Close() }()
req, err := http.ReadRequest(bufio.NewReader(conn))
if err != nil {
return
}
_ = req.Body.Close()
gotAuth <- req.Header.Get("Proxy-Authorization")
_, _ = conn.Write([]byte("HTTP/1.1 200 Connection Established\r\n\r\n"))
}()
proxyURL := fmt.Sprintf("http://user:secret@%s", ln.Addr().String())
conn, err := dialHTTPProxy(context.Background(), directDial, proxyURL, "example.com:443", 5*time.Second)
require.NoError(t, err)
defer func() { _ = conn.Close() }()
require.Equal(t, "Basic "+basicAuth(url.UserPassword("user", "secret")), <-gotAuth)
})
}
func TestDial(t *testing.T) {
t.Run("missing execution id", func(t *testing.T) {
_, err := Dial(context.Background(), "tcp", "127.0.0.1:1")
require.Error(t, err)
require.Contains(t, err.Error(), "executionId")
})
t.Run("unsupported proxy scheme fails hard", func(t *testing.T) {
opts := &types.Options{ExecutionId: "net-dial-scheme-" + t.Name()}
require.NoError(t, protocolstate.Init(opts))
t.Cleanup(func() { protocolstate.Close(opts.ExecutionId) })
ctx := context.WithValue(context.Background(), "executionId", opts.ExecutionId) // nolint: staticcheck
ctx = context.WithValue(ctx, "proxyURL", "socks5://127.0.0.1:1080") // nolint: staticcheck
_, err := Dial(ctx, "tcp", "example.com:80")
require.Error(t, err)
require.Contains(t, err.Error(), "unsupported proxy scheme")
})
t.Run("invalid proxy url fails hard", func(t *testing.T) {
opts := &types.Options{ExecutionId: "net-dial-invalid-" + t.Name()}
require.NoError(t, protocolstate.Init(opts))
t.Cleanup(func() { protocolstate.Close(opts.ExecutionId) })
ctx := context.WithValue(context.Background(), "executionId", opts.ExecutionId) // nolint: staticcheck
ctx = context.WithValue(ctx, "proxyURL", "://bad") // nolint: staticcheck
_, err := Dial(ctx, "tcp", "example.com:80")
require.Error(t, err)
require.Contains(t, err.Error(), "invalid proxy URL")
})
t.Run("udp with http proxy fails hard", func(t *testing.T) {
opts := &types.Options{ExecutionId: "net-dial-udp-" + t.Name()}
require.NoError(t, protocolstate.Init(opts))
t.Cleanup(func() { protocolstate.Close(opts.ExecutionId) })
ctx := context.WithValue(context.Background(), "executionId", opts.ExecutionId) // nolint: staticcheck
ctx = context.WithValue(ctx, "proxyURL", "http://127.0.0.1:8080") // nolint: staticcheck
_, err := Dial(ctx, "udp", "example.com:53")
require.Error(t, err)
require.Contains(t, err.Error(), "does not support protocol")
})
t.Run("open through local http proxy", func(t *testing.T) {
target, err := net.Listen("tcp", "127.0.0.1:0")
require.NoError(t, err)
defer func() { _ = target.Close() }()
go func() {
conn, err := target.Accept()
if err != nil {
return
}
defer func() { _ = conn.Close() }()
_, _ = conn.Write([]byte("pong"))
}()
proxyLn, err := net.Listen("tcp", "127.0.0.1:0")
require.NoError(t, err)
defer func() { _ = proxyLn.Close() }()
go func() {
conn, err := proxyLn.Accept()
if err != nil {
return
}
defer func() { _ = conn.Close() }()
req, err := http.ReadRequest(bufio.NewReader(conn))
if err != nil {
return
}
_ = req.Body.Close()
require.Equal(t, http.MethodConnect, req.Method)
require.Equal(t, target.Addr().String(), req.Host)
upstream, err := net.Dial("tcp", target.Addr().String())
if err != nil {
return
}
defer func() { _ = upstream.Close() }()
_, _ = conn.Write([]byte("HTTP/1.1 200 Connection Established\r\n\r\n"))
go func() { _, _ = io.Copy(upstream, conn) }()
_, _ = io.Copy(conn, upstream)
}()
opts := &types.Options{ExecutionId: "net-dial-proxy-" + t.Name()}
require.NoError(t, protocolstate.Init(opts))
t.Cleanup(func() { protocolstate.Close(opts.ExecutionId) })
ctx := context.WithValue(context.Background(), "executionId", opts.ExecutionId) // nolint: staticcheck
ctx = context.WithValue(ctx, "proxyURL", "http://"+proxyLn.Addr().String()) // nolint: staticcheck
ctx = context.WithValue(ctx, "timeoutVariants", &types.Timeouts{TcpReadTimeout: 5 * time.Second}) // nolint: staticcheck
nconn, err := Open(ctx, "tcp", target.Addr().String())
require.NoError(t, err)
defer func() { _ = nconn.Close() }()
buf := make([]byte, 16)
n, err := nconn.conn.Read(buf)
require.NoError(t, err)
require.Equal(t, "pong", string(buf[:n]))
})
}
func TestRedactProxyURL(t *testing.T) {
require.Equal(t, "http://proxy.example.com:8080", redactProxyURL("http://user:[email protected]:8080"))
require.Equal(t, "http://proxy.example.com:8080", redactProxyURL("http://proxy.example.com:8080"))
require.Equal(t, "<invalid proxy URL>", redactProxyURL("://bad-url"))
}
// fakeConn is a minimal net.Conn for testing bufferedConn.
type fakeConn struct{ net.Conn }
func (f *fakeConn) Read([]byte) (int, error) { return 0, io.EOF }
func (f *fakeConn) Write([]byte) (int, error) { return 0, nil }
func (f *fakeConn) Close() error { return nil }
func (f *fakeConn) LocalAddr() net.Addr { return nil }
func (f *fakeConn) RemoteAddr() net.Addr { return nil }
func (f *fakeConn) SetDeadline(time.Time) error { return nil }
func (f *fakeConn) SetReadDeadline(time.Time) error { return nil }
func (f *fakeConn) SetWriteDeadline(time.Time) error { return nil }
+18
View File
@@ -24,6 +24,24 @@ func (e *Extractor) CompileExtractors() error {
return fmt.Errorf("regex extractor group must be >= 0, got %d", e.RegexGroup)
}
var requiredField string
var valueCount int
switch e.extractorType {
case RegexExtractor:
requiredField, valueCount = "regex", len(e.Regex)
case KValExtractor:
requiredField, valueCount = "kval", len(e.KVal)
case JSONExtractor:
requiredField, valueCount = "json", len(e.JSON)
case XPathExtractor:
requiredField, valueCount = "xpath", len(e.XPath)
case DSLExtractor:
requiredField, valueCount = "dsl", len(e.DSL)
}
if valueCount == 0 {
return fmt.Errorf("%s extractor requires at least one %s value", e.extractorType, requiredField)
}
// Compile the regexes
for _, regex := range e.Regex {
if cached, err := cache.Regex().GetIFPresent(regex); err == nil && cached != nil {
+51
View File
@@ -0,0 +1,51 @@
package extractors
import (
"fmt"
"testing"
"github.com/stretchr/testify/require"
"gopkg.in/yaml.v3"
)
func TestCompileExtractorsRequiresValues(t *testing.T) {
tests := []struct {
typ string
field string
values string
}{
{"regex", "regex", `["hello"]`},
{"kval", "kval", `["header"]`},
{"json", "json", `[".value"]`},
{"xpath", "xpath", `["//a"]`},
{"dsl", "dsl", `["body"]`},
}
for _, tt := range tests {
t.Run(tt.typ, func(t *testing.T) {
for _, suffix := range []struct{ name, yaml string }{
{"omitted", ""},
{"null", tt.field + ": null\n"},
{"empty", tt.field + ": []\n"},
} {
t.Run(suffix.name, func(t *testing.T) {
var operator Extractor
require.NoError(t, yaml.Unmarshal([]byte("type: "+tt.typ+"\n"+suffix.yaml), &operator))
require.ErrorContains(t, operator.CompileExtractors(), fmt.Sprintf("%s extractor requires at least one %s value", tt.typ, tt.field))
})
}
t.Run("valid", func(t *testing.T) {
var operator Extractor
require.NoError(t, yaml.Unmarshal([]byte("type: "+tt.typ+"\n"+tt.field+": "+tt.values), &operator))
require.NoError(t, operator.CompileExtractors())
})
})
}
}
func TestCompileExtractorsRequiresValuesForSelectedType(t *testing.T) {
e := &Extractor{
Type: ExtractorTypeHolder{ExtractorType: RegexExtractor},
KVal: []string{"header"},
}
require.ErrorContains(t, e.CompileExtractors(), "regex extractor requires at least one regex value")
}
Loaded 100 of 158 files, more files were not shown because too many files have changed in this diff. Show more