Commit Graph
3 Commits
Author SHA1 Message Date
Kamil Kamili 128ec95adb Trim comments in the request body budget change 2026-10-29 17:54:25 +04:00
Kamil Kamili a4422d8962 Bound request-body bytes held by in-flight HTTP requests
Add -max-request-body-inflight (default 256 MB, 0 disables it).
executeRequest acquires its body size from a shared budget before it
dumps and sends, and releases it when it returns, so concurrent
requests with multi-MB bodies queue instead of all holding their
copies at once. Bodies under 1 MB skip the budget, since the semaphore
is FIFO and a queued heavy request would stall them. Race requests
skip it too, because they must all reach the open gate together.

The HTTP timeout in ExecuteWithResults now starts once the budget
admits the request, so time spent queued does not time it out.
2026-10-29 14:53:14 +04:00
Ice3manandClaude Opus 5.5 2954b81c85 http: stop copying large request bodies
Templates that send multi-MB bodies (e.g. CVE-2026-109618 sends 1, 8 and
20 MB) kept 8-10 copies of each body alive per request. With 25 hosts in
flight this reached ~10 GB RSS and OOM-killed scan VMs.

- add pkg/utils/httpbody: a zero-copy request body that rewinds on EOF;
  GetBody hands out new cursors over the same slice instead of cloning
- build raw, model and fuzzed request bodies with httpbody instead of
  retryablehttp's ReusableReadCloser, which tees every read into a second
  growing buffer and clones on each GetBody
- dump requests with http.Request.Write into a capped writer instead of
  retryablehttp Dump/httputil.DumpRequestOut; output is unchanged for
  bodies under the cap
- keep only the first 32 KiB of a body in request dumps (new
  -rqss/-request-size-save flag, 0 keeps all) and leave bodies over 4 KiB
  out of curl commands
- build the project cache key only when a project file is used, hashing
  the body when the dump is truncated
- replay sends the full body when the dump was truncated
- slice the raw request body from the request instead of reading it out
- pass replacer values to fasttemplate as []byte so writes don't copy them

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-25 18:48:04 +05:30