Dogan Can Bakir
cb1f0757d4
send identical requests once per scan and share the response across templates
2026-11-10 00:44:19 +03:00
Mzack9999
36d0adba18
Merge pull request #8024 from projectdiscovery/fix/headless-exclude-lock
...
Read headless excludes under the dialer lock
2026-11-03 20:47:14 +01:00
Mzack9999
e44df6e947
lock excludes
2026-11-03 17:30:04 +01:00
Mzack9999
faaeaf30b1
mapped deny
2026-11-03 17:07:35 +01:00
Mzack9999
48600a24de
use canonical
2026-11-03 12:51:48 +01:00
TassioSales
b2b54a7ac4
fix(headless): preserve port in CanonicalizeHost and stabilize proxy test
...
Signed-off-by: TassioSales <[email protected]
2026-11-03 01:28:54 -03:00
TassioSales
22aedebe5d
fix(headless): normalize IPv4-mapped IPv6 host forms for -exclude-hosts and -lna ( #7982 )
...
Normalize hostnames and IPv4-mapped IPv6 addresses (e.g. ::ffff:127.0.0.1,
::ffff:7f00:1, and bracketed variants) before validating against NetworkPolicy
and ExcludeTargets in headless execution. Also expand CDP blocked patterns
in SetupHeadlessNetworkPolicy to match mapped IPv6 representations.
Signed-off-by: TassioSales <[email protected]
2026-11-03 01:07:02 -03:00
Doğan Can Bakır
77ba7792f3
keep subdomain and regex exclude-hosts entries
2026-10-27 18:37:21 +03:00
Doğan Can Bakır
8caa65a839
match exclude-hosts entries exactly at dial time
2026-10-27 17:58:24 +03:00
Mzack9999
87afa2fd4e
Merge pull request #7926 from projectdiscovery/feat/7894-js-backoff
...
apply host backoff to javascript template dials
2026-10-25 15:46:57 +01:00
Mzack9999
9301048bc4
merge dev
2026-10-25 13:37:52 +01:00
Doğan Can Bakır
bf53db167d
apply host backoff to javascript template dials
2026-10-24 23:22:24 +03:00
Mzack9999
308044e234
merge dev
2026-10-24 01:01:45 +02:00
kleonyuan
6a42a7bf52
chore: fix some comments ( #7830 )
...
Signed-off-by: kleonyuan <[email protected]
2026-10-18 05:00:43 +05:30
TassioSales
8b440a5a77
fix(headless): address errcheck linter warnings for unchecked returns in proxy and tests
2026-10-14 00:34:33 -03:00
TassioSales
d43c1f547f
fix(headless): canonicalize IPv4-mapped IPv6 literals and re-validate dial targets
2026-10-13 15:01:50 -03:00
TassioSales
506e4fb0fc
fix(headless): enforce network policy on WebSocket handshakes via loopback proxy
2026-10-13 14:16:53 -03:00
TassioSales
52960db710
fix(headless): harden blocked URL staging, URL target parsing, and wsMap cleanup
2026-10-13 02:37:33 -03:00
TassioSales
ca3e27c5f5
fix(headless): accumulate dynamic blocked URL patterns across denied WebSockets
2026-10-13 02:22:21 -03:00
TassioSales
a94aa7f00d
fix(headless): propagate NetworkEnable errors during policy initialization
2026-10-13 02:12:01 -03:00
TassioSales
c14335d351
fix(headless): address code review feedback on page lifecycle and CIDR handling
2026-10-13 02:02:26 -03:00
TassioSales
c5a528cc60
fix(headless): enforce network policy and blocked URLs for WebSocket connections via CDP
2026-10-13 01:45:12 -03:00
TassioSales
d9dabbb4a1
fix(headless): enforce network policy on ws and wss schemes ( #7867 )
...
Previously, isValidHost only checked targets with http: and https:
prefixes, allowing ws: and wss: connections in headless mode to bypass
configured network policies (e.g. -exclude-hosts or local network restrictions).
This commit normalizes the scheme check and enforces host validation
against the network policy for ws: and wss: protocols as well.
2026-10-12 23:50:25 -03:00
Doğan Can Bakır
1b080a7d19
fix: apply per-host rate limiting to every protocol
2026-10-07 12:45:14 +03:00
Mzack9999
213d3badfd
merge dev
2026-09-23 20:23:30 +02:00
f5d0ef81a5
feat(http): add RFC 9111 cache w/ per-request disable ( #6746 )
...
* feat(http): add RFC-9111 cache w/ per-request disable
* wrap HTTP transports with sandrolain/httpcache
and expose global `--no-http-cache` flag.
* allow templates to set `disable-http-cache`.
* bump Go to 1.25.7.
Signed-off-by: Dwi Siswanto <[email protected] [email protected] [email protected] [email protected] [email protected] [email protected] [email protected] [email protected] [email protected] [email protected] [email protected]
2026-09-07 03:55:36 +07:00
Mzack9999
034cf04d47
quarantine hosts
2026-08-26 10:37:40 +04:00
Mzack9999
6d6bfd7950
fix lint
2026-08-24 12:49:10 +04:00
Mzack9999
1427b35cdb
General maintenance and cleanup ( #7469 )
...
* general maintenance
* address review
2026-07-03 05:48:17 +05:30
Mzack9999
f82673434c
fix leaks ( #7502 )
...
* bound cache
* release limiters
* purge templates
* address review
2026-06-29 12:35:22 +07:00
Mzack9999
1fe6025b96
bump goja ( #7467 )
2026-06-20 22:13:23 +05:30
Mzack9999 and Ice3man
c041bf2922
Fix connection reuse + port pre-flight ( #6715 )
...
* .
* .
* adding pre-flight
* add conn reuse
* making pre-flight optional
* adding per-host httpclient + ratelimit
* add conn reuse stats
* ignoring leak in lru
* adding global connection pooling with sharding
* lint
* addressing comments
* fix lint: disambiguate retryablehttp URL access
retryablehttp.Request embeds *urlutil.URL, so .URL.String() resolved to
the wrong type and tripped staticcheck QF1008. Use .Request.URL to read
the underlying http.Request URL for rate limiting and pooling keys.
* fix race
* regression harness
* ci tuning
* address review comments
- preflight: resolve via MetaInput.Target()/CustomIP, gate banner on Silent
- http: drop dead Priority 4 branch in AnalyzeConnectionReuse
* add tests
* host stats
* reuse timing
---------
Co-authored-by: Ice3man <[email protected]
2026-06-19 15:22:16 +05:30
Mzack9999
63124eff2c
per-host http client pool ( #7301 )
...
* attempting per-host http client pool
* close idle conns
* fixing idle conn leak
* lint
* ignore http internal routines
* fixing tests
* lowering flaky http requirement
* fixing open gate issue
* adding perf tests
* pool rework
2026-06-16 20:21:49 +05:30
Dwi Siswanto and Mzack9999
20cbf0bd73
refactor: use path-aware filesystem containment checks ( #7420 )
...
* feat(filepath): add IsPathWithinAnyDirectory func
Signed-off-by: Dwi Siswanto <[email protected] [email protected] [email protected] [email protected]
2026-06-04 21:03:51 +05:30
Dwi Siswanto
6f2ade6a9b
fix(js): respect allow-local-file-access in require ( #7332 )
...
* fix(js): respect `allow-local-file-access` in `require`
The goja `require() `function used the default
host filesystem loader which let JavaScript
templates import any local files even when
`allow-local-file-access` was disabled.
Pooled runtimes kept `require()` state around so
a module loaded during a privileged execution
could remain cached for a later restricted one.
Rebuild the require registry per execution after
setting the execution context, and route file-
backed module loads to preserve native modules
while enforcing the same sandbox rules (as
`nuclei/fs`).
Signed-off-by: Dwi Siswanto <[email protected] [email protected] [email protected]
2026-04-10 08:06:39 +07:00
Dwi Siswanto
0eb87c2621
fix(js): mysql panic due to missing executionId in ctx
...
The `connectWithDSN` func used `db.Exec()` which
implicitly uses `context.Background()`[1]. This
caused the registered "nucleitcp" dialer
callback to receive a ctx missing the
`executionId`, leading to a panic during type
assertion.
Refactor `connectWithDSN` to accept `executionId`
explicitly and use it to create a `context` for
`db.PingContext()` (yeah, instead of `db.Exec()`).
And, add a defensive check in the dialer callback
to handle nil values gracefully.
Fixes #6733 regression introduced in #6296 .
[1]: "Exec uses `context.Background` internally" -
https://pkg.go.dev/database/sql#DB.Exec .
Signed-off-by: Dwi Siswanto <[email protected]
2025-12-25 17:16:49 +07:00
dependabot[bot] and Dwi Siswanto
7e151de8a3
chore: bump PD modules & update httputil calls ( #6629 )
...
* chore(deps): bump the modules group across 1 directory with 11 updates
Bumps the modules group with 11 updates in the / directory:
| Package | From | To |
| --- | --- | --- |
| [github.com/projectdiscovery/fastdialer](https://github.com/projectdiscovery/fastdialer ) | `0.4.16` | `0.4.17` |
| [github.com/projectdiscovery/hmap](https://github.com/projectdiscovery/hmap ) | `0.0.95` | `0.0.96` |
| [github.com/projectdiscovery/retryabledns](https://github.com/projectdiscovery/retryabledns ) | `1.0.108` | `1.0.109` |
| [github.com/projectdiscovery/retryablehttp-go](https://github.com/projectdiscovery/retryablehttp-go ) | `1.0.131` | `1.0.132` |
| [github.com/projectdiscovery/gologger](https://github.com/projectdiscovery/gologger ) | `1.1.60` | `1.1.61` |
| [github.com/projectdiscovery/networkpolicy](https://github.com/projectdiscovery/networkpolicy ) | `0.1.28` | `0.1.29` |
| [github.com/projectdiscovery/tlsx](https://github.com/projectdiscovery/tlsx ) | `1.2.1` | `1.2.2` |
| [github.com/projectdiscovery/useragent](https://github.com/projectdiscovery/useragent ) | `0.0.102` | `0.0.103` |
| [github.com/projectdiscovery/utils](https://github.com/projectdiscovery/utils ) | `0.6.1` | `0.7.1` |
| [github.com/projectdiscovery/wappalyzergo](https://github.com/projectdiscovery/wappalyzergo ) | `0.2.55` | `0.2.56` |
| [github.com/projectdiscovery/cdncheck](https://github.com/projectdiscovery/cdncheck ) | `1.2.10` | `1.2.11` |
Updates `github.com/projectdiscovery/fastdialer` from 0.4.16 to 0.4.17
- [Release notes](https://github.com/projectdiscovery/fastdialer/releases )
- [Commits](https://github.com/projectdiscovery/fastdialer/compare/v0.4.16...v0.4.17 )
Updates `github.com/projectdiscovery/hmap` from 0.0.95 to 0.0.96
- [Release notes](https://github.com/projectdiscovery/hmap/releases )
- [Commits](https://github.com/projectdiscovery/hmap/compare/v0.0.95...v0.0.96 )
Updates `github.com/projectdiscovery/retryabledns` from 1.0.108 to 1.0.109
- [Release notes](https://github.com/projectdiscovery/retryabledns/releases )
- [Commits](https://github.com/projectdiscovery/retryabledns/compare/v1.0.108...v1.0.109 )
Updates `github.com/projectdiscovery/retryablehttp-go` from 1.0.131 to 1.0.132
- [Release notes](https://github.com/projectdiscovery/retryablehttp-go/releases )
- [Commits](https://github.com/projectdiscovery/retryablehttp-go/compare/v1.0.131...v1.0.132 )
Updates `github.com/projectdiscovery/gologger` from 1.1.60 to 1.1.61
- [Release notes](https://github.com/projectdiscovery/gologger/releases )
- [Commits](https://github.com/projectdiscovery/gologger/compare/v1.1.60...v1.1.61 )
Updates `github.com/projectdiscovery/networkpolicy` from 0.1.28 to 0.1.29
- [Release notes](https://github.com/projectdiscovery/networkpolicy/releases )
- [Commits](https://github.com/projectdiscovery/networkpolicy/compare/v0.1.28...v0.1.29 )
Updates `github.com/projectdiscovery/tlsx` from 1.2.1 to 1.2.2
- [Release notes](https://github.com/projectdiscovery/tlsx/releases )
- [Changelog](https://github.com/projectdiscovery/tlsx/blob/main/.goreleaser.yml )
- [Commits](https://github.com/projectdiscovery/tlsx/compare/v1.2.1...v1.2.2 )
Updates `github.com/projectdiscovery/useragent` from 0.0.102 to 0.0.103
- [Release notes](https://github.com/projectdiscovery/useragent/releases )
- [Commits](https://github.com/projectdiscovery/useragent/compare/v0.0.102...v0.0.103 )
Updates `github.com/projectdiscovery/utils` from 0.6.1 to 0.7.1
- [Release notes](https://github.com/projectdiscovery/utils/releases )
- [Changelog](https://github.com/projectdiscovery/utils/blob/main/CHANGELOG.md )
- [Commits](https://github.com/projectdiscovery/utils/compare/v0.6.1...v0.7.1 )
Updates `github.com/projectdiscovery/wappalyzergo` from 0.2.55 to 0.2.56
- [Release notes](https://github.com/projectdiscovery/wappalyzergo/releases )
- [Commits](https://github.com/projectdiscovery/wappalyzergo/compare/v0.2.55...v0.2.56 )
Updates `github.com/projectdiscovery/cdncheck` from 1.2.10 to 1.2.11
- [Release notes](https://github.com/projectdiscovery/cdncheck/releases )
- [Changelog](https://github.com/projectdiscovery/cdncheck/blob/main/.goreleaser.yaml )
- [Commits](https://github.com/projectdiscovery/cdncheck/compare/v1.2.10...v1.2.11 )
---
updated-dependencies:
- dependency-name: github.com/projectdiscovery/fastdialer
dependency-version: 0.4.17
dependency-type: direct:production
update-type: version-update:semver-patch
dependency-group: modules
- dependency-name: github.com/projectdiscovery/hmap
dependency-version: 0.0.96
dependency-type: direct:production
update-type: version-update:semver-patch
dependency-group: modules
- dependency-name: github.com/projectdiscovery/retryabledns
dependency-version: 1.0.109
dependency-type: direct:production
update-type: version-update:semver-patch
dependency-group: modules
- dependency-name: github.com/projectdiscovery/retryablehttp-go
dependency-version: 1.0.132
dependency-type: direct:production
update-type: version-update:semver-patch
dependency-group: modules
- dependency-name: github.com/projectdiscovery/gologger
dependency-version: 1.1.61
dependency-type: direct:production
update-type: version-update:semver-patch
dependency-group: modules
- dependency-name: github.com/projectdiscovery/networkpolicy
dependency-version: 0.1.29
dependency-type: direct:production
update-type: version-update:semver-patch
dependency-group: modules
- dependency-name: github.com/projectdiscovery/tlsx
dependency-version: 1.2.2
dependency-type: direct:production
update-type: version-update:semver-patch
dependency-group: modules
- dependency-name: github.com/projectdiscovery/useragent
dependency-version: 0.0.103
dependency-type: direct:production
update-type: version-update:semver-patch
dependency-group: modules
- dependency-name: github.com/projectdiscovery/utils
dependency-version: 0.7.1
dependency-type: direct:production
update-type: version-update:semver-minor
dependency-group: modules
- dependency-name: github.com/projectdiscovery/wappalyzergo
dependency-version: 0.2.56
dependency-type: direct:production
update-type: version-update:semver-patch
dependency-group: modules
- dependency-name: github.com/projectdiscovery/cdncheck
dependency-version: 1.2.11
dependency-type: indirect
update-type: version-update:semver-patch
dependency-group: modules
...
Signed-off-by: dependabot[bot] <[email protected] [email protected] [email protected] [email protected] [email protected] [email protected]
2025-12-01 02:07:25 +07:00
Mzack9999
b9ce0c2226
fixing syntax
2025-10-22 19:11:26 +04:00
Mzack9999
5557f4eae3
fixing lint
2025-10-21 16:57:26 +04:00
Mzack9999
f6efa9da6c
clean up pools after 24hours inactivity
2025-10-21 16:41:26 +04:00
Nakul Bharti
93be3b8291
fix: improve cleanup in parallel execution ( #6490 )
2025-09-24 01:12:43 +05:30
Tarun Koyalwar
19247ae74b
Path-Based Fuzzing SQL fix ( #6400 )
...
* setup claude
* migrate to using errkit
* fix unused imports + lint errors
* update settings.json
* fix url encoding issue
* fix lint error
* fix the path fuzzing component
* fix lint error
2025-08-25 13:36:58 +05:30
Sandeep Singh and Mzack9999
b4644af80a
Lint + test fixes after utils dep update ( #6393 )
...
* fix: remove undefined errorutil.ShowStackTrace
* feat: add make lint support and integrate with test
* refactor: migrate errorutil to errkit across codebase
- Replace deprecated errorutil with modern errkit
- Convert error declarations from var to func for better compatibility
- Fix all SA1019 deprecation warnings
- Maintain error chain support and stack traces
* fix: improve DNS test reliability using Google DNS
- Configure test to use Google DNS (8.8.8.8) for stability
- Fix nil pointer issue in DNS client initialization
- Keep production defaults unchanged
* fixing logic
* removing unwanted branches in makefile
---------
Co-authored-by: Mzack9999 <[email protected]
2025-08-20 05:28:23 +05:30
5b89811b90
Support concurrent Nuclei engines in the same process ( #6322 )
...
* support for concurrent nuclei engines
* clarify LfaAllowed race
* remove unused mutex
* update LfaAllowed logic to prevent races until it can be reworked for per-execution ID
* Update pkg/templates/parser.go
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
* debug tests
* debug gh action
* fixig gh template test
* using atomic
* using synclockmap
* restore tests concurrency
* lint
* wiring executionId in js fs
---------
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Co-authored-by: Mzack9999 <[email protected]
2025-07-19 00:10:58 +05:30
HD Moore
875941ce8d
avoid data races using mutex for memguardian
2025-07-15 02:34:47 -05:00
f26996cb89
Remove singletons from Nuclei engine (continuation of #6210 ) ( #6296 )
...
* introducing execution id
* wip
* .
* adding separate execution context id
* lint
* vet
* fixing pg dialers
* test ignore
* fixing loader FD limit
* test
* fd fix
* wip: remove CloseProcesses() from dev merge
* wip: fix merge issue
* protocolstate: stop memguarding on last dialer delete
* avoid data race in dialers.RawHTTPClient
* use shared logger and avoid race conditions
* use shared logger and avoid race conditions
* go mod
* patch executionId into compiled template cache
* clean up comment in Parse
* go mod update
* bump echarts
* address merge issues
* fix use of gologger
* switch cmd/nuclei to options.Logger
* address merge issues with go.mod
* go vet: address copy of lock with new Copy function
* fixing tests
* disable speed control
* fix nil ExecuterOptions
* removing deprecated code
* fixing result print
* default logger
* cli default logger
* filter warning from results
* fix performance test
* hardcoding path
* disable upload
* refactor(runner): uses `Warning` instead of `Print` for `pdcpUploadErrMsg`
Signed-off-by: Dwi Siswanto <[email protected] 114fbe6663cf12ca800e[email protected] [email protected] [email protected] [email protected]
2025-07-10 01:17:26 +05:30
Douglas Danger Manley
e4a0b8ec60
Do not clobber the "tcp" dialer for MySQL ( #5681 )
...
This introduces a "nucleitcp" protocol that Nuclei will use when
making MySQL connections as part of its templates.
Previously, this would register (and de-register!) a custom "tcp"
dialer, and that applied globally, so any piece of software that
used a MySQL database and included nuclei in SDK mode would have
its database connections ripped out from under it due to the dialer
hijacking.
By using "nucleitcp" as the protocol, we are free to do whatever
we want with the dialer and not impact any other packages.
Within our `BuildDSN` function, we quietly replace the protocol to
"nucleitcp" if it was "tcp", so nuclei developers don't have to do
anything special to use this functionality; it will always do it.
2025-01-15 00:29:30 +05:30
Shubham Rasal
be1f634eae
Add Alive Proxy into Options ( #5903 )
...
* Move proxy variable from global to options
- Provides ability to pass diff proxy in single nuclei instance using sdk
* add type check (resolve comments)
2024-12-13 04:23:27 +05:30
Dwi Siswanto and Ice3man
13af7ccd49
fix: data race at protocolstate, contextargs & some outdated test cases ( #5820 )
...
* test(flow): update outdated test cases
Signed-off-by: Dwi Siswanto <[email protected] [email protected] [email protected] [email protected] [email protected] [email protected] [email protected]
2024-11-19 11:08:25 +05:30
Danny Shemesh
888a732fbc
Unlock memguard global change mutex only when locked ( #5714 )
2024-10-14 14:18:59 +05:30
Dogan Can Bakir