Address review feedback:
- swagger: a null entry under a response's `headers:` map is also
dereferenced by openapi2conv.ToV3Response, for both operation and
top-level responses. Drop those along with null responses.
- openapi: a `servers:` list containing only null entries now returns
the same "no servers found" error as an empty list, instead of
succeeding without generating any request.
A null value in an OpenAPI or Swagger document (an empty list item, or a
map key with no value) decodes to a nil pointer. Several of these were
dereferenced without a check, so loading such a spec with -im openapi or
-im swagger crashed nuclei with SIGSEGV.
openapi (generator.go):
- a null entry under `servers:` -> nil *Server
- a null media type under `content:` -> nil *MediaType
swagger (dereferenced inside kin-openapi's openapi2conv.ToV3):
- a null path item under `paths:` -> nil *PathItem
- a null item under path/operation `parameters:` or a null top-level
parameter -> nil *Parameter
- a null operation or top-level response -> nil *Response
Skip the nil entries in the generator, and drop them from the decoded
Swagger document before converting it. Other null entries are already
rejected by kin-openapi with an error and are left alone.
Also add a go-fuzz harness for the OpenAPI parser with a seed corpus, so
the weekly fuzz workflow picks it up, and regression tests that panic on
the pre-fix code.
executeTurboHTTP dialed pipelined connections with context.Background(),
so a pipelined request of a pinned target resolved the host instead of
connecting to its ip. It now dials with the input context, which carries
the pin. protocols/http/target-line-ip-pipeline.yaml runs the pinned
target test with pipeline: true, and gets no result without the change.
ProbeKey joins the input and the ip with a NUL byte, which no input
holds, instead of "@", which a scheme-less input may hold itself.
A pinned input whose ip had no probe result fell back to Transform,
which takes the result probed for the input without an ip, so the
pinned target could get the scheme of another server. It now gets the
conversion of an input that was not probed.
A target without a scheme is probed through its pinned ip, but the
result was stored under the target alone, so the last probe of a host
pinned to several ips set the scheme for all of them, and a server that
answered only on the other scheme was not scanned. The result is now
stored under the target and its ip, and TransformPinned looks it up for
a pinned input.
-replay-proxy re-sends the requests behind a finding through a proxy,
which resolves the host itself, so the replay of a pinned target would
not reach the pinned ip. The pinned-target checks move into
pinnedTargetConflict with a table test, and the input README lists
-replay-proxy next to -proxy and -automatic-scan.
dev requires fastdialer v0.6.0, which has WithResolveOverride and the
dial cache fix, so go.mod and go.sum take dev's versions and the
fastdialer replace is dropped.
Scanning a vhost on a given server, or an origin behind a CDN, needs the
hostname in the Host header, SNI and template variables while the
connection goes to another ip, and one host often has to be scanned on
several ips. -sa scans the ips DNS returns, and a target of the ip with
a Host header sends no SNI and sets {{Hostname}} to the ip.
A target line can now pin its target to an ip, like curl --resolve:
{"target": "https://app.example", "ip": "192.0.2.10"}
Each target and ip pair is a separate target with its own template
selection. The ip is validated and canonicalized, the target must have
a hostname, -sa and -iv don't resolve a pinned target, and an ip the
network policy denies is skipped. Pinned targets are refused with
-proxy and -automatic-scan.
The engine pins the host of every target with a CustomIP, from a target
line or -sa, with fastdialer.WithResolveOverride on the context each
template runs with. HTTP, network, websocket and javascript templates
then connect to the ip, while redirects to other hosts and the proxy
dial are no longer pinned. HTTP keeps a client per pinned ip, network
and websocket results report the ip, and scheme-less targets are probed
through it.
The host errors cache keys a failed dial of a pinned target by its ip,
also for IDN hosts, and -mup counts the urls of each ip apart, so a url
listed with several ips, or resolved to several with -sa, is kept on
each of them.
go.mod points fastdialer at a fork commit with WithResolveOverride until
a fastdialer release includes it.
The octet-stream branch compared the schema format against "bindary",
so a requestBody with format: binary went out with Content-Type:
text/plain and only format: byte kept application/octet-stream.
RequestResponse.MarshalJSON stored the pre-marshaled request/response bytes as []byte inside map[string]interface{}, which encoding/json (and sonic) encode as base64 strings. The type's own UnmarshalJSON expects JSON objects under those keys, so the marshal -> unmarshal round-trip always failed when Request != nil, breaking the json.JSONCodec contract asserted at pkg/input/types/http.go.
MetaInput.ReqResp serializes under the raw-request key with this codec (MetaInput.MarshalString -> MetaInput.Unmarshal), used by host-map scans and exclusion processing in pkg/input/provider/list/hmap.go and internal/runner/preflight_portscan.go: a captured request attached to a MetaInput produced an encoded key that could not be decoded.
Wrap the pre-marshaled bytes as json.Message so they are emitted verbatim, and omit the keys for nil Request/Response instead of encoding base64 null, mirroring UnmarshalJSON's optional-key behavior.
Fixes#7795
An XML array example has no single root to encode, and the request
still went out with no body or Content-Type. Skip it after the warning,
like the default branch does for content types it cannot build.
When a request body had no schema, the generator used the schema that
generateEmptySchemaValue returned as the body value itself. An
application/xml body then hit the unchecked map assertion and nuclei
panicked while loading the spec, a JSON body was sent as
{"type":"object"}, and a text/plain body as a dump of the schema struct.
Generate an example from the empty schema instead, and let the XML
branch take a string example as the body and warn on types it cannot
encode instead of panicking. The empty schema for
application/octet-stream has no example, so that body now gets the
placeholder the octet-stream branch already has for an empty value.
Mzack9999