A step that loads no templates never matches, so the subtemplates
loaded below it never run. Nothing said so when an exclusion emptied
the step: a tags step dropped the filter's error, and a template step
warned only for an excluded tag, naming the template but not the
subtemplates it held back, and only with -v, since warnings are
hidden by default. Such steps are now counted in a warning shown after
loading, and -v logs each one with its workflow.
Signed-off-by: Dwi Siswanto <[email protected]>
The probe client uses http.ProxyFromEnvironment, so a proxy set in the
environment could route the probe away from the local test server when
the test runs on its own.
The httpx client that probes non-URL inputs for http or https was built
from httpx.DefaultOptions without the configured resolvers, so the probe
resolved inputs through fastdialer's default resolvers (resolv.conf plus
1.1.1.1, 1.0.0.1, 8.8.8.8 and 8.8.4.4) even when -resolvers was set.
Pass InternalResolversList to httpxOptions.Resolvers in the runner and in
the SDK. httpx then uses only those resolvers. Without -resolvers the
probe behaves as before.
Closes#7969
Add -max-request-body-inflight (default 256 MB, 0 disables it).
executeRequest acquires its body size from a shared budget before it
dumps and sends, and releases it when it returns, so concurrent
requests with multi-MB bodies queue instead of all holding their
copies at once. Bodies under 1 MB skip the budget, since the semaphore
is FIFO and a queued heavy request would stall them. Race requests
skip it too, because they must all reach the open gate together.
The HTTP timeout in ExecuteWithResults now starts once the budget
admits the request, so time spent queued does not time it out.
A target without a scheme is probed through its pinned ip, but the
result was stored under the target alone, so the last probe of a host
pinned to several ips set the scheme for all of them, and a server that
answered only on the other scheme was not scanned. The result is now
stored under the target and its ip, and TransformPinned looks it up for
a pinned input.
-replay-proxy re-sends the requests behind a finding through a proxy,
which resolves the host itself, so the replay of a pinned target would
not reach the pinned ip. The pinned-target checks move into
pinnedTargetConflict with a table test, and the input README lists
-replay-proxy next to -proxy and -automatic-scan.
dev requires fastdialer v0.6.0, which has WithResolveOverride and the
dial cache fix, so go.mod and go.sum take dev's versions and the
fastdialer replace is dropped.
Fixes#7875
When options.ProxyInternal was enabled, only HTTP_PROXY was set in the environment.
Because Go standard library http.ProxyFromEnvironment inspects HTTPS_PROXY for https URLs,
internal HTTPS requests (such as template updates via GitHub API / releases, pdtm, etc.)
failed to route through the specified proxy in restricted or proxy-required environments.
This sets HTTP_PROXY, HTTPS_PROXY, ALL_PROXY (and lowercase counterparts) when ProxyInternal
is enabled, and adds unit test coverage. Also fixes operator precedence for ProxyInternal
checks in runner.go and sdk_private.go.
The archive link points to a different host than the API, and the github
client sends the token to every request it makes. The archive is public,
so only the API call needs the token now, and the archive url has to be
https.
Also adds doc comments to the new functions and tests.
Adds -utc / -update-templates-channel (stable, latest). stable keeps the
current behaviour. latest installs the head of the nuclei-templates main
branch, so templates merged after the last release are available without
waiting for the next one.
The version for the latest channel is the last release plus the short
commit as semver build metadata (e.g. v10.5.0+3de629d). Build metadata is
ignored when comparing versions, so switching back to stable doesn't
downgrade, and only a newer release makes it outdated there. Since the
version check only knows about releases, the latest channel compares the
installed version against the current head to decide when to update.
The channel can also be set in the config file like the other options.
Closes#5608
Dogan Can Bakir