Compare commits

..
Author SHA1 Message Date
QianEricandQianEric b46f982b9b scripts: report a missing script program before scanning (#965)
Scripts are run through `sh -c`, so when the program a `call_format`
starts with is missing, the only feedback is whatever the shell prints
once the scan has already finished:

    Open 127.0.0.1:18080
    [~] Starting Script(s)
    [!] Error No such file or directory (os error 2)

which neither names the program nor arrives before a long scan has been
wasted.

Check the first word of every selected script format against PATH while
the scripts are initialised, and fail there with the program named. Name
resolution follows what the shell would do: PATH entries in order, plus
PATHEXT on Windows.

Formats that do not start with a bare program name are left alone, since
their first word is not something PATH can answer for:

- `FOO=bar cmd ...` (leading assignment)
- `/opt/custom/nmap ...` (explicit path)
- `{{script}} ...` (placeholder filled in at run time)

With nmap absent, on the same scan:

    before: scans, then "[!] Error No such file or directory (os error 2)"
    after:  "[!] Initiating scripts failed!
             The script `default` runs `nmap`, which is not installed or
             not in PATH. Install it, or skip scripts with `--scripts none`."

With nmap present the scan and the script phase run as before.

Implemented with std/target-specific helpers rather than a new
dependency: `which` pulls in a newer windows-sys and rewrites ~110 lines
of Cargo.lock for a 15-line check.

Refs #691

Co-authored-by: QianEric <[email protected]>
2026-11-04 09:29:26 +00:00
Rudy Mizrahi Celekli 521f8bf720 fix(config): accept documented lowercase script modes (#964) 2026-11-04 09:29:20 +00:00
Rudy Mizrahi Celekli ab1e69f1af fix(scripts): preserve literal hashes in metadata (#962) 2026-11-04 09:29:08 +00:00
demrom 29d85a77df fix(main): print the no-open-ports hint once, only when nothing was found (#960)
A subnet sweep for one port printed the batch-size hint for every host without that port. When any host answered with an open port, or a closed one under --closed, replies got through at this batch size and timeout, so batch size and timeout are unlikely to be the cause and the hint is skipped. When nothing was found it is printed once for all targets.

Fixes #795
2026-11-04 09:29:01 +00:00
dependabot[bot] a79a471111 build(deps): bump actions-rust-lang/setup-rust-toolchain (#953)
Bumps [actions-rust-lang/setup-rust-toolchain](https://github.com/actions-rust-lang/setup-rust-toolchain) from 1.17.0 to 2.0.0.
- [Release notes](https://github.com/actions-rust-lang/setup-rust-toolchain/releases)
- [Changelog](https://github.com/actions-rust-lang/setup-rust-toolchain/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions-rust-lang/setup-rust-toolchain/compare/166cdcfd11aee3cb47222f9ddb555ce30ddb9659...ecabd13d1c56bd1345c230e542e9144811ad706f)

---
updated-dependencies:
- dependency-name: actions-rust-lang/setup-rust-toolchain
  dependency-version: 2.0.0
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-11-04 09:28:55 +00:00
dependabot[bot] f4192fd367 build(deps): bump windows-sys from 0.59.0 to 0.61.2 (#956)
Bumps [windows-sys](https://github.com/microsoft/windows-rs) from 0.59.0 to 0.61.2.
- [Release notes](https://github.com/microsoft/windows-rs/releases)
- [Commits](https://github.com/microsoft/windows-rs/commits)

---
updated-dependencies:
- dependency-name: windows-sys
  dependency-version: 0.61.2
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-11-04 09:28:48 +00:00
dependabot[bot] e9a4972e1f build(deps): bump toml from 1.1.6+spec-1.1.0 to 1.1.8+spec-1.1.0 (#957)
Bumps [toml](https://github.com/toml-rs/toml) from 1.1.6+spec-1.1.0 to 1.1.8+spec-1.1.0.
- [Commits](https://github.com/toml-rs/toml/compare/toml-v1.1.6...toml-v1.1.8)

---
updated-dependencies:
- dependency-name: toml
  dependency-version: 1.1.8+spec-1.1.0
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-11-04 09:28:42 +00:00
dependabot[bot] 2f81acf45b build(deps): bump libc from 0.2.181 to 0.2.191 (#959)
Bumps [libc](https://github.com/rust-lang/libc) from 0.2.181 to 0.2.191.
- [Release notes](https://github.com/rust-lang/libc/releases)
- [Changelog](https://github.com/rust-lang/libc/blob/0.2.191/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/libc/compare/0.2.181...0.2.191)

---
updated-dependencies:
- dependency-name: libc
  dependency-version: 0.2.191
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-11-04 09:28:35 +00:00
dependabot[bot] 8f8a51adf2 build(deps): bump log from 0.4.34 to 0.4.35 (#966)
Bumps [log](https://github.com/rust-lang/log) from 0.4.34 to 0.4.35.
- [Release notes](https://github.com/rust-lang/log/releases)
- [Changelog](https://github.com/rust-lang/log/blob/main/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/log/compare/0.4.34...v0.4.35)

---
updated-dependencies:
- dependency-name: log
  dependency-version: 0.4.35
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-11-04 09:28:28 +00:00
Autumn (Bee)andbee-san 1dda392d1c ci: Dependabot monthly instead of daily (cost) (#958)
Co-authored-by: bee-san <[email protected]>
2026-11-04 09:28:21 +00:00
Autumn (Bee)andAutumn Skerritt 937903340a Avoid formatting discarded scan errors (#950)
* perf(scanner): defer TCP error diagnostics

* bench(scanner): check open-only TCP and UDP paths

* fix(scanner): Import HashSet in tests after main merge

---------

Co-authored-by: Autumn Skerritt <[email protected]>
2026-10-05 19:17:26 +09:00
dependabot[bot] f33752812c build(deps): bump tokio from 1.44.2 to 1.50.0 (#952)
Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.44.2 to 1.50.0.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.44.2...tokio-1.50.0)

---
updated-dependencies:
- dependency-name: tokio
  dependency-version: 1.50.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-05 18:39:29 +09:00
Autumn (Bee) 9b48f03ecd fix(config): let command-line options override the config file (#947)
`Opts::merge` copied every value present in the config file over the
parsed command line, so e.g. `-r 8600-8650` was silently replaced by the
config's `range` and the requested ports were never scanned (`-p` only
survived because `PortStrategy::pick` prefers `ports` over `range`).
The same happened to `-a`, `-b`, `-t`, `--tries`, `-g`, `--scripts`,
`--scan-order`, the trailing nmap command and every other merged option.
This contradicts the documented behaviour: "Any arguments passed
directly to the binary will override the default provided in this
config file".

`Opts::parse()` cannot tell a value typed by the user from a clap
default, so `Opts::read()` now parses through `ArgMatches` and records
the ids of the arguments whose `value_source` is `CommandLine` in a new
skipped `cli_args` field. `merge_required`/`merge_optional` leave those
fields alone; options not given on the command line are still filled in
from the config file. `--top` and `--no-config` behave as before.

Add pure parsing tests (no sockets) covering both directions.

Fixes #722
2026-10-03 06:41:48 +01:00
Autumn (Bee) 32d7d2cb17 Defer fallback DNS initialization for literal targets (#949)
* Initialize the fallback DNS resolver only when needed

* Record lazy DNS startup benchmark results

* docs(perf): remeasure lazy DNS against merged Tokio master

* docs(perf): verify lazy DNS on current master
2026-10-02 20:45:12 +09:00
Autumn (Bee) 67c08a3310 Speed up port exclusions and avoid redundant port copies (#954)
* test(scanner): preserve port filtering order and boundaries

* perf(scanner): reuse port lists and index large exclusions

* perf(scanner): use bitmap for dense exclusions

* bench(scanner): document production port preparation speedups

* ci(bench): diagnose repeated macOS listener misses

* ci(bench): capture socket errors immediately after listener misses

* perf(scanner): keep bitmap storage out of repeated async polls

* ci(bench): provision macOS TCP memory for repeated scans
2026-10-02 20:14:56 +09:00
Autumn (Bee)andVrtgs b7119f7f3e Port to tokio (supersedes #856, closes #944) (#951)
* refactor(scanner): run the scanner on tokio instead of async-std

async-std is discontinued upstream (RUSTSEC-2025-0052). tokio is already in
the dependency tree through hickory-resolver, so switching adds no crate and
removes 28 from Cargo.lock (async-std, async-io, polling,
async-global-executor, blocking, futures-executor, ...).

This is a like-for-like port. The scanner keeps its architecture and the
library API keeps its signatures:

- a single `FuturesUnordered` keeps at most `batch_size` sockets in flight
  from one future; nothing is spawned per socket;
- connect and UDP receive timeouts use `tokio::time::timeout` and report the
  same `TimedOut` error that async-std's `io::timeout` did;
- open TCP sockets are still shut down in both directions before they are
  closed (through `into_std()`, because tokio only has a write-side shutdown);
- `--interval` sleeps with `tokio::time::sleep`.

The binary runs the scan on a current-thread runtime that exists only for
the scan. Address resolution (blocking DNS lookups and file reads) runs
before the runtime is created and scripts run after it is dropped, so neither
can stall the scan, and hickory's internal runtime is never nested in ours.

Library users now have to poll the scan futures from a tokio runtime with the
I/O and time drivers enabled. The crate and `Scanner` docs say so, and the
crate example shows how.

Builds on the earlier port in #856 by @vrtgs.

Co-authored-by: Vrtgs <[email protected]>

* perf(scanner): opt the scan out of tokio's cooperative budget

The scan polls every in-flight socket from one future through a
`FuturesUnordered`. tokio's cooperative scheduling gives that future a budget
of 128 units per poll. Once it is spent, every tokio resource returns
`Pending` and defers its wake-up, but `FuturesUnordered` still polls all the
other ready sockets in that same poll. When thousands of connections finish
together (a burst of refusals, or a batch of timeouts firing at once), the
scan keeps re-polling sockets only to park them again, roughly 128 at a time.

async-std had no budget. `tokio::task::unconstrained` restores that
behaviour: one poll handles every ready socket. The future still returns
`Pending`, and so lets the runtime poll I/O and timers, as soon as no socket
is ready.

* perf(scanner): send UDP probes before registering the socket with tokio

tokio's socket I/O is readiness based: `UdpSocket::send` and `recv` first
wait until the reactor has reported the socket ready and only then make the
system call. async-io, which async-std used, made the call first and only
waited after `WouldBlock`.

With a straight port, every UDP probe therefore made an extra trip through
the event loop before its send. Probes to closed ports on the local host
also no longer finished in their first poll: there the ICMP "port
unreachable" is usually queued before `send` returns, so async-std's first
`recv` already saw the refusal.

Bind a non-blocking std socket instead, send the probe and try the first
receive straight away. The socket is only handed to tokio, and registered
with its reactor, when there is really something to wait for. Errors and
timeouts are reported exactly as before.

* ci: add an opt-in scan benchmark comparing a change with its base

The test suite deliberately never opens sockets (see #942), so nothing in CI
showed whether a change to the scanner makes RustScan slower, hungrier or
wrong. The tokio port (#944) needs exactly that evidence.

The "Scan benchmark" workflow runs for pull requests that touch the scanner,
its runtime or its dependencies, and on demand. It builds the change and its
base in release mode on Linux x64/arm64, macOS and Windows runners. It then
runs .github/scan-benchmark/scan_bench.py, which:

- opens 315 TCP listeners and 32 UDP responders on 127.0.0.1 inside the
  runner. Nothing outside the runner is scanned. Ports stay out of the
  ephemeral range, so a scan can never connect a socket to itself;
- scans them with both builds, alternating the build order between runs:
  full TCP sweeps with batch sizes 500/4500/10000, 8 loopback hosts, one
  port, a UDP sweep, and on Linux ports that iptables drops (filtered TCP and
  UDP, and a mixed sweep), so timeouts are measured too;
- fails unless every run finds every listener and no build reports ports the
  base never found, or misses ports the base always found;
- reports medians of throughput (from RustScan's own scan timer), wall time,
  time to the first open port, peak RSS, CPU time, and sampled peak
  descriptors and threads. The table goes to the job summary, the raw data
  to an artifact;
- fails if the change's median throughput or wall time is more than 15%
  worse, or its peak RSS more than 25% worse, in any scenario.

contributing.md documents the workflow as the place where scanning is
measured, next to the rule that tests must not scan.

* ci(scan-benchmark): temporarily compare the port's intermediate builds (to be reverted)

* Revert "ci(scan-benchmark): temporarily compare the port's intermediate builds (to be reverted)"

This reverts commit 30ac107c55.

* fix(scanner): let the runtime poll I/O while a batch is started

The scan drives every socket from one future on a current-thread runtime,
so the runtime only polls for I/O and fires timers between two polls of
that future. A poll only ended once no socket was ready, which could take
a long time:

- starting a whole batch at once: 8,191 sockets took longer than a 500 ms
  timeout on the Windows runner, so connections that had succeeded during
  the fill were only seen after their timers fired. The benchmark found no
  open port at all with `-b 10000 -t 500` there;
- UDP probes to closed ports on the local host finish straight away, so a
  whole sweep ran in one poll, and the replies from open ports waited until
  the end: the first open UDP port was reported 2.6x later than with
  async-std.

`scan_sockets` now starts and finishes sockets in rounds of at most 128
(`WORK_PER_TURN`, both counted together) and calls `yield_now()` between
rounds, which defers its wake-up until the runtime has polled the OS
selector without blocking. Each round still keeps up to `batch_size`
sockets in flight; the batch now fills over several rounds instead of in
one go. The cost is one extra non-blocking `epoll_wait` (or equivalent)
per 128 sockets.

* fix(scanner): see ICMP errors on UDP sockets that wait in tokio

A UDP probe that gets no answer straight away is handed to tokio to wait for
the reply. On Linux, an ICMP "port unreachable" that arrives later only
raises EPOLLERR on the socket. tokio does not count that as readable, so
`UdpSocket::recv` slept until the timeout instead of returning the refusal.
async-io treated EPOLLERR as readable.

The straight port (before the eager probe) showed the effect: the UDP sweep
of the local host took 4.1 s instead of 1.0 s, because every closed port
waited for the 500 ms timeout. With the eager probe it only hit closed ports
whose ICMP arrived after the first receive. That is rare on the local host
but the common case for any remote host, where it turned every closed port
into a timeout.

`recv_or_error` waits for `READABLE | ERROR` readiness and hands out the
queued error (`take_error`), so a refused probe ends as soon as the ICMP
arrives, as before.

* ci(scan-benchmark): add a delayed host, fit Windows, compare misses

What the first runs showed:

- Only answers that arrive while the probe is still being sent were
  measured: on the loopback interface, refusals and ICMP errors are queued
  before connect()/send() returns. A remote host answers a round trip
  later, which is the path where tokio's UDP receive missed ICMP errors.
  On Linux the workflow now also sets up a host 5 ms away (a network
  namespace behind a netem-delayed veth pair, with ICMP rate limiting off,
  running its own listeners), and the benchmark sweeps it over TCP and UDP.
- Windows retries a refused connection for about a second. With
  `-t 500`, 8,191 concurrent attempts made even the base miss listeners.
  Windows now uses the default timeout and sweeps 8,191 ports (2,047 for
  the 500-socket batch).
- A build now fails the check only if it misses more listeners than the
  base does (any miss at all if the base misses none). Misses by the base
  are reported as a warning in the summary.
- The macOS runner has no `net.inet.icmp.icmplim`, which failed the
  "describe" step, and caps descriptors at 10240, so the 65536 limit
  could not be set. The describe step can no longer fail the job, and
  10240 is tried as well.

* docs(address): say that parse_addresses must not run inside async code

hickory's synchronous resolver runs its own tokio runtime, and tokio panics
when a runtime is dropped inside an asynchronous context. That was already
true before, but library users now have to run the scanner on tokio, so
they are more likely to call `parse_addresses` from async code. Document
the two safe ways: call it before starting the runtime (as the binary does)
or through `spawn_blocking`.

---------

Co-authored-by: Vrtgs <[email protected]>
2026-10-02 17:45:31 +09:00
18 changed files with 6075 additions and 224 deletions

No files matched your search

+3 -3
View File
@@ -3,15 +3,15 @@ updates:
- package-ecosystem: "github-actions"
directory: "/"
schedule:
interval: "daily"
interval: "monthly"
- package-ecosystem: "docker"
directory: "/"
schedule:
interval: "daily"
interval: "monthly"
- package-ecosystem: cargo
directory: "/"
schedule:
interval: daily
interval: monthly
open-pull-requests-limit: 10
ignore:
- dependency-name: anyhow
@@ -0,0 +1,57 @@
"""Provision TCP memory for repeated scans on a disposable macOS CI runner."""
import ctypes
import json
import os
import subprocess
class Statistics(ctypes.Structure):
# XNU's bsd/sys/mem_acct_private.h, struct memacct_statistics.
_fields_ = [("peak", ctypes.c_uint64), ("allocated", ctypes.c_int64),
("softlimit", ctypes.c_uint64), ("hardlimit", ctypes.c_uint64),
("name", ctypes.c_char * 16)]
def tcp_memory(new_limit: int | None = None) -> dict:
"""Read the TCP accounting object, optionally raising its hard limit."""
libc = ctypes.CDLL(None, use_errno=True)
libc.sysctlnametomib.argtypes = [ctypes.c_char_p, ctypes.POINTER(ctypes.c_int),
ctypes.POINTER(ctypes.c_size_t)]
libc.sysctl.argtypes = [ctypes.POINTER(ctypes.c_int), ctypes.c_uint,
ctypes.c_void_p, ctypes.POINTER(ctypes.c_size_t),
ctypes.c_void_p, ctypes.c_size_t]
mib = (ctypes.c_int * 24)()
count = ctypes.c_size_t(22)
if libc.sysctlnametomib(b"kern.memacct", mib, ctypes.byref(count)):
raise OSError(ctypes.get_errno(), os.strerror(ctypes.get_errno()))
def query(operation: int, index: int, output, replacement=None):
mib[count.value], mib[count.value + 1] = operation, index
length = ctypes.c_size_t(ctypes.sizeof(output))
if libc.sysctl(mib, count.value + 2, ctypes.byref(output), ctypes.byref(length),
ctypes.byref(replacement) if replacement is not None else None,
ctypes.sizeof(replacement) if replacement is not None else 0):
raise OSError(ctypes.get_errno(), os.strerror(ctypes.get_errno()))
return length.value
names = ctypes.create_string_buffer(8 * 16)
length = query(5, 0, names) # MEM_ACCT_SUBSYSTEMS
index = next(i for i in range(length // 16)
if names.raw[i * 16:(i + 1) * 16].rstrip(b"\0") == b"TCP")
stats = Statistics()
query(6, index, stats) # MEM_ACCT_ALL_SUBSYSTEM_STATISTICS
previous_limit = stats.hardlimit
if new_limit is not None and 0 < previous_limit < new_limit:
query(3, index, ctypes.c_uint64(), ctypes.c_uint64(new_limit))
query(6, index, stats)
return {"previous_hardlimit": previous_limit,
**{name: getattr(stats, name) for name in ("peak", "allocated", "softlimit", "hardlimit")}}
if __name__ == "__main__":
# The default TCP budget is 1/32 of physical memory. Repeated 10,000-socket
# batches hit ENOBUFS on Darwin 25.6 even on master. Give both builds 1/8
# before timing; retain every listener check and performance threshold.
memory = int(subprocess.check_output(["sysctl", "-n", "hw.memsize"], text=True))
print(json.dumps(tcp_memory(memory // 8)))
+67 -2
View File
@@ -142,11 +142,13 @@ class Scenario:
timeout_ms: int = 1500
udp: bool = False
repeat: int = 1 # extra samples for very short scenarios
excluded_ports: tuple[int, ...] = ()
@property
def sockets(self) -> int:
span = sum(end - start + 1 for start, end in self.ranges)
return len(self.addresses) * (len(self.ports) + span)
excluded = sum(self.covers(port) for port in set(self.excluded_ports))
return len(self.addresses) * (len(self.ports) + span - excluded)
def covers(self, port: int) -> bool:
return port in self.ports or any(s <= port <= e for s, e in self.ranges)
@@ -160,6 +162,8 @@ class Scenario:
args += ["-b", str(self.batch), "-t", str(self.timeout_ms)]
if self.udp:
args.append("--udp")
if self.excluded_ports:
args += ["--exclude-ports", ",".join(map(str, self.excluded_ports))]
return args
def expected(self, listeners: dict) -> set[str]:
@@ -168,7 +172,7 @@ class Scenario:
f"{address}:{port}"
for address in self.addresses
for port in by_address.get(address, [])
if self.covers(port)
if self.covers(port) and port not in self.excluded_ports
}
def describe(self) -> str:
@@ -192,7 +196,15 @@ def build_scenarios(
scenarios = [
Scenario("tcp-1-port", "TCP, 1 open port", lo, ports=(TCP_PORTS[0],), repeat=3),
Scenario("tcp-open", "TCP, open listeners only", lo, ports=TCP_PORTS, repeat=3),
Scenario("tcp-sweep", f"TCP, {ports(sweep)}, default batch", lo, sweep),
Scenario(
"tcp-sweep-excluded",
f"TCP, {ports(sweep)}, 1,024 excluded ports",
lo,
sweep,
excluded_ports=tuple(range(1024, 2048)),
),
Scenario(
"tcp-sweep-b500", f"TCP, {ports(small_sweep)}, small batch", lo, small_sweep, batch=500
),
@@ -225,6 +237,12 @@ def build_scenarios(
scenarios.append(
Scenario("udp-sweep", f"UDP, {ports(sweep)}", lo, sweep, timeout_ms=500, udp=True)
)
scenarios.append(
Scenario(
"udp-open", "UDP, echo responders only", lo, ports=UDP_PORTS,
timeout_ms=500, udp=True, repeat=9,
)
)
if delayed:
scenarios.append(
Scenario(
@@ -546,6 +564,41 @@ def ephemeral_range() -> tuple[int, int] | None:
return (49152, 65535) if system == "Windows" else None
def diagnose_tcp(binary: str, scenario: Scenario, listeners: dict) -> dict:
"""Untimed failure capture on a disposable macOS runner, using debug logs."""
from macos_tcp_memory import tcp_memory
def states() -> dict[str, int]:
snapshot = subprocess.run(
["netstat", "-an", "-p", "tcp"], capture_output=True, text=True, check=True
)
counts: dict[str, int] = {}
for line in snapshot.stdout.splitlines():
if line.startswith("tcp"):
state = line.split()[-1]
counts[state] = counts.get(state, 0) + 1
return counts
before = states()
memory_before = tcp_memory()
probe = subprocess.run(
[binary, *scenario.args(), "--scripts", "none", "--accessible", "--no-banner", "--no-config"],
env=dict(os.environ, RUST_LOG="rustscan=info,rustscan::scanner=debug"),
capture_output=True, text=True, check=True,
)
opened = {
match.group(1) for line in probe.stdout.splitlines()
if (match := OPEN_LINE.match(line))
}
return {
"before": before, "after": states(),
"memory_before": memory_before, "memory_after": tcp_memory(),
"missing": sorted(scenario.expected(listeners) - opened),
"errors": [line for line in probe.stderr.splitlines()
if "Typical socket connection errors" in line],
}
def environment() -> dict[str, object]:
system = platform.system()
info: dict[str, object] = {
@@ -818,6 +871,7 @@ def run(args: argparse.Namespace) -> int:
names = list(builds)
results: Results = {sc.name: {name: [] for name in names} for sc in scenarios}
diagnostics = []
try:
warmup = next((sc for sc in scenarios if sc.name == "tcp-sweep"), scenarios[0])
for name in names:
@@ -842,6 +896,16 @@ def run(args: argparse.Namespace) -> int:
)
if not res.ok:
print(f" {res.error}", flush=True)
if platform.system() == "Darwin" and not diagnostics and any(
sc.expected(listeners) - set(results[sc.name][name][-1].open) for name in names
):
# A failed pair cannot support a performance claim. Capture
# its OS errors immediately, outside both measured scans.
for name in names:
record = {"build": name, "scenario": sc.name,
**diagnose_tcp(builds[name], sc, listeners)}
diagnostics.append(record)
print("TCP diagnosis: " + json.dumps(record), flush=True)
finally:
server.kill()
server.wait()
@@ -861,6 +925,7 @@ def run(args: argparse.Namespace) -> int:
"failures": failures,
"mismatches": mismatches,
"notes": notes,
"diagnostics": diagnostics,
}
markdown = render_markdown(report, results, scenarios)
if args.json:
+11 -2
View File
@@ -75,7 +75,7 @@ jobs:
run: python -m py_compile .github/scan-benchmark/scan_bench.py
- name: Setup Rust
uses: actions-rust-lang/setup-rust-toolchain@166cdcfd11aee3cb47222f9ddb555ce30ddb9659 # v1.17.0
uses: actions-rust-lang/setup-rust-toolchain@ecabd13d1c56bd1345c230e542e9144811ad706f # v2.0.0
with:
toolchain: stable
# build.rs formats the generated payload table with rustfmt.
@@ -184,6 +184,14 @@ jobs:
head -c 200 "$RUNNER_TEMP/far.json"; echo
echo "DELAYED_OK=1" >> "$GITHUB_ENV"
- name: Provision macOS TCP memory for repeated socket batches
if: runner.os == 'macOS'
run: |
# XNU's TCP accounting hard limit otherwise makes both builds miss
# listeners with ENOBUFS during repeated sweeps. Apply one budget to
# both builds before timing; keep all input sizes and gates intact.
sudo "$(command -v python)" .github/scan-benchmark/macos_tcp_memory.py
- name: Describe the runner
continue-on-error: true
run: |
@@ -195,7 +203,8 @@ jobs:
ss -Hltnu | sort -k5 | head -n 50 || true
elif [ "$RUNNER_OS" = macOS ]; then
for oid in hw.ncpu machdep.cpu.brand_string kern.maxfilesperproc \
net.inet.ip.portrange.first net.inet.ip.portrange.last; do
net.inet.ip.portrange.first net.inet.ip.portrange.last \
kern.ipc.maxsockbuf kern.ipc.nmbclusters; do
sysctl "$oid" || true
done
fi
+1 -1
View File
@@ -26,7 +26,7 @@ jobs:
uses: actions/checkout@v7
- name: Setup Rust
uses: actions-rust-lang/setup-rust-toolchain@166cdcfd11aee3cb47222f9ddb555ce30ddb9659 # v1.17.0
uses: actions-rust-lang/setup-rust-toolchain@ecabd13d1c56bd1345c230e542e9144811ad706f # v2.0.0
with:
toolchain: ${{ matrix.rust }}
components: clippy, rustfmt
Generated
+41 -155
View File
@@ -2,21 +2,6 @@
# It is not intended for manual editing.
version = 4
[[package]]
name = "addr2line"
version = "0.24.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "dfbe277e56a376000877090da837660b4427aad530e3028d44e0bffe4f89a1c1"
dependencies = [
"gimli",
]
[[package]]
name = "adler2"
version = "2.0.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "512761e0bb2578dd7380c6baaa0f4ce03e84f95e960231d1dec8bf4d7d6e2627"
[[package]]
name = "ahash"
version = "0.8.11"
@@ -135,21 +120,6 @@ version = "1.4.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ace50bade8e6234aa140d9a2f552bbee1db4d353f69b8217bc503490fc1a9f26"
[[package]]
name = "backtrace"
version = "0.3.74"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8d82cb332cdfaed17ae235a638438ac4d4839913cc2af585c3c6746e8f8bee1a"
dependencies = [
"addr2line",
"cfg-if",
"libc",
"miniz_oxide",
"object",
"rustc-demangle",
"windows-targets 0.52.6",
]
[[package]]
name = "base64"
version = "0.21.7"
@@ -309,7 +279,7 @@ version = "3.1.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "faf9468729b8cbcea668e36183cb69d317348c2e08e994829fb56ebfdfbaac34"
dependencies = [
"windows-sys 0.60.2",
"windows-sys 0.61.2",
]
[[package]]
@@ -417,7 +387,7 @@ dependencies = [
"libc",
"option-ext",
"redox_users",
"windows-sys 0.60.2",
"windows-sys 0.61.2",
]
[[package]]
@@ -485,7 +455,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "33d852cb9b869c2a9b3df2f71a3074817f01e1844f839a144f5fcef059a4eb5d"
dependencies = [
"libc",
"windows-sys 0.59.0",
"windows-sys 0.52.0",
]
[[package]]
@@ -599,12 +569,6 @@ dependencies = [
"wasip3",
]
[[package]]
name = "gimli"
version = "0.31.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "07e28edb80900c19c28f1072f2e8aeca7fa06b23cd4169cefe1af5aa3260783f"
[[package]]
name = "half"
version = "2.4.1"
@@ -885,7 +849,7 @@ version = "0.3.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b58db92f96b720de98181bbbe63c831e87005ab460c1bf306eb2622b4707997f"
dependencies = [
"socket2",
"socket2 0.5.8",
"widestring",
"windows-sys 0.48.0",
"winreg",
@@ -969,9 +933,9 @@ checksum = "09edd9e8b54e49e587e4f6295a7d29c3ea94d469cb40ab8ca70b288248a81db2"
[[package]]
name = "libc"
version = "0.2.181"
version = "0.2.191"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "459427e2af2b9c839b132acb702a1c654d95e10f8c326bfc2ad11310e458b1c5"
checksum = "65cd87c9a6d60536580b316ad985726104730a5151c99845f2490428ef91240a"
[[package]]
name = "libredox"
@@ -1013,9 +977,9 @@ dependencies = [
[[package]]
name = "log"
version = "0.4.34"
version = "0.4.35"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f9f8bd3e56ce4dfc153cf470fffbfa98c7620958b312ca5c3a4b8d5181fd13c6"
checksum = "5d15f86d12da44cd8d4ec3c73984537eb7f4cadcb9776a575a0a8a05ca11005c"
[[package]]
name = "lru-cache"
@@ -1038,15 +1002,6 @@ version = "2.7.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "78ca9ab1a0babb1e7d5695e3530886289c18cf2f87ec19a575a0abdce112e3a3"
[[package]]
name = "miniz_oxide"
version = "0.8.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8e3e04debbb59698c15bacbb6d93584a8c0ca9cc3213cb423d31f760d8843ce5"
dependencies = [
"adler2",
]
[[package]]
name = "mio"
version = "1.0.3"
@@ -1086,15 +1041,6 @@ dependencies = [
"autocfg",
]
[[package]]
name = "object"
version = "0.36.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "62948e14d923ea95ea2c7c86c71013138b66525b86bdc08d2dcc262bdb497b87"
dependencies = [
"memchr",
]
[[package]]
name = "once_cell"
version = "1.21.4"
@@ -1426,12 +1372,6 @@ dependencies = [
"libc",
]
[[package]]
name = "rustc-demangle"
version = "0.1.24"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "719b953e2095829ee67db738b3bfa9fa368c94900df327b3f07fe6e794d2fe1f"
[[package]]
name = "rustix"
version = "1.1.3"
@@ -1442,7 +1382,7 @@ dependencies = [
"errno",
"libc",
"linux-raw-sys",
"windows-sys 0.60.2",
"windows-sys 0.61.2",
]
[[package]]
@@ -1494,6 +1434,7 @@ dependencies = [
"gcd",
"hickory-resolver",
"itertools 0.15.0",
"libc",
"log",
"once_cell",
"parameterized",
@@ -1504,6 +1445,7 @@ dependencies = [
"text_placeholder",
"tokio",
"toml",
"windows-sys 0.61.2",
]
[[package]]
@@ -1593,9 +1535,9 @@ dependencies = [
[[package]]
name = "serde_spanned"
version = "1.1.1"
version = "1.1.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6662b5879511e06e8999a8a235d848113e942c9124f211511b16466ee2995f26"
checksum = "4e7523beb55eece201a2356bee0bbca0d1ab466c14c07703b2e0ee6d42cb0c2c"
dependencies = [
"serde_core",
]
@@ -1631,6 +1573,16 @@ dependencies = [
"windows-sys 0.52.0",
]
[[package]]
name = "socket2"
version = "0.6.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c3d1e2c7f27f8d4cb10542a02c49005dbd6e93095799d6f3be745fae9f8fedd4"
dependencies = [
"libc",
"windows-sys 0.61.2",
]
[[package]]
name = "stable_deref_trait"
version = "1.2.0"
@@ -1683,7 +1635,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "230a1b821ccbd75b185820a1f1ff7b14d21da1e442e22c0863ea5f08771a8874"
dependencies = [
"rustix",
"windows-sys 0.60.2",
"windows-sys 0.61.2",
]
[[package]]
@@ -1774,17 +1726,16 @@ checksum = "1f3ccbac311fea05f86f61904b462b55fb3df8837a366dfc601a0161d0532f20"
[[package]]
name = "tokio"
version = "1.44.2"
version = "1.50.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e6b88822cbe49de4185e3a4cbf8321dd487cf5fe0c5c65695fef6346371e9c48"
checksum = "27ad5e34374e03cfffefc301becb44e9dc3c17584f414349ebe29ed26661822d"
dependencies = [
"backtrace",
"bytes",
"libc",
"mio",
"pin-project-lite",
"socket2",
"windows-sys 0.52.0",
"socket2 0.6.5",
"windows-sys 0.61.2",
]
[[package]]
@@ -1799,9 +1750,9 @@ dependencies = [
[[package]]
name = "toml"
version = "1.1.6+spec-1.1.0"
version = "1.1.8+spec-1.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "920602543f0911ab71da12c50d59701da54c196d1a2bf5cb4b75667f137a406a"
checksum = "20489e00e4d8741d6be680764cc12e270655e375a20d1011e844a9c3379e678d"
dependencies = [
"indexmap 2.13.0",
"serde_core",
@@ -1814,27 +1765,27 @@ dependencies = [
[[package]]
name = "toml_datetime"
version = "1.1.1+spec-1.1.0"
version = "1.1.2+spec-1.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3165f65f62e28e0115a00b2ebdd37eb6f3b641855f9d636d3cd4103767159ad7"
checksum = "2b86d767906c6c42421dcba507eb9d203e779497710a47782a224bb871653053"
dependencies = [
"serde_core",
]
[[package]]
name = "toml_parser"
version = "1.1.3+spec-1.1.0"
version = "1.1.5+spec-1.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1d38ac1cf9b95face32296c0a3ede1fdc270627c9d9c02a7274dd6d960dc4d56"
checksum = "baa693a8032d7e1cada7d0041e96126df243179ff061456783ac7f12bda4744c"
dependencies = [
"winnow",
]
[[package]]
name = "toml_writer"
version = "1.1.2+spec-1.1.0"
version = "1.1.3+spec-1.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7d56353a2a665ad0f41a421187180aab746c8c325620617ad883a99a1cbe66d2"
checksum = "06bdbd8cfc056b8d2e2e85f29b56a3bdbecb527cef81eb39e3e7b98af4652770"
[[package]]
name = "tracing"
@@ -2084,7 +2035,7 @@ version = "0.1.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "cf221c93e13a30d793f7645a0e7762c55d169dbb0a49671918a2319d289b10bb"
dependencies = [
"windows-sys 0.59.0",
"windows-sys 0.48.0",
]
[[package]]
@@ -2128,11 +2079,11 @@ dependencies = [
[[package]]
name = "windows-sys"
version = "0.60.2"
version = "0.61.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f2f500e4d28234f72040990ec9d39e3a6b950f9f22d3dba18416c35882612bcb"
checksum = "ae137229bcbd6cdf0f7b80a31df61766145077ddf49416a728b02cb3921ff3fc"
dependencies = [
"windows-targets 0.53.5",
"windows-link",
]
[[package]]
@@ -2159,30 +2110,13 @@ dependencies = [
"windows_aarch64_gnullvm 0.52.6",
"windows_aarch64_msvc 0.52.6",
"windows_i686_gnu 0.52.6",
"windows_i686_gnullvm 0.52.6",
"windows_i686_gnullvm",
"windows_i686_msvc 0.52.6",
"windows_x86_64_gnu 0.52.6",
"windows_x86_64_gnullvm 0.52.6",
"windows_x86_64_msvc 0.52.6",
]
[[package]]
name = "windows-targets"
version = "0.53.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4945f9f551b88e0d65f3db0bc25c33b8acea4d9e41163edf90dcd0b19f9069f3"
dependencies = [
"windows-link",
"windows_aarch64_gnullvm 0.53.1",
"windows_aarch64_msvc 0.53.1",
"windows_i686_gnu 0.53.1",
"windows_i686_gnullvm 0.53.1",
"windows_i686_msvc 0.53.1",
"windows_x86_64_gnu 0.53.1",
"windows_x86_64_gnullvm 0.53.1",
"windows_x86_64_msvc 0.53.1",
]
[[package]]
name = "windows_aarch64_gnullvm"
version = "0.48.5"
@@ -2195,12 +2129,6 @@ version = "0.52.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "32a4622180e7a0ec044bb555404c800bc9fd9ec262ec147edd5989ccd0c02cd3"
[[package]]
name = "windows_aarch64_gnullvm"
version = "0.53.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a9d8416fa8b42f5c947f8482c43e7d89e73a173cead56d044f6a56104a6d1b53"
[[package]]
name = "windows_aarch64_msvc"
version = "0.48.5"
@@ -2213,12 +2141,6 @@ version = "0.52.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "09ec2a7bb152e2252b53fa7803150007879548bc709c039df7627cabbd05d469"
[[package]]
name = "windows_aarch64_msvc"
version = "0.53.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b9d782e804c2f632e395708e99a94275910eb9100b2114651e04744e9b125006"
[[package]]
name = "windows_i686_gnu"
version = "0.48.5"
@@ -2231,24 +2153,12 @@ version = "0.52.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8e9b5ad5ab802e97eb8e295ac6720e509ee4c243f69d781394014ebfe8bbfa0b"
[[package]]
name = "windows_i686_gnu"
version = "0.53.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "960e6da069d81e09becb0ca57a65220ddff016ff2d6af6a223cf372a506593a3"
[[package]]
name = "windows_i686_gnullvm"
version = "0.52.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0eee52d38c090b3caa76c563b86c3a4bd71ef1a819287c19d586d7334ae8ed66"
[[package]]
name = "windows_i686_gnullvm"
version = "0.53.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "fa7359d10048f68ab8b09fa71c3daccfb0e9b559aed648a8f95469c27057180c"
[[package]]
name = "windows_i686_msvc"
version = "0.48.5"
@@ -2261,12 +2171,6 @@ version = "0.52.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "240948bc05c5e7c6dabba28bf89d89ffce3e303022809e73deaefe4f6ec56c66"
[[package]]
name = "windows_i686_msvc"
version = "0.53.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1e7ac75179f18232fe9c285163565a57ef8d3c89254a30685b57d83a38d326c2"
[[package]]
name = "windows_x86_64_gnu"
version = "0.48.5"
@@ -2279,12 +2183,6 @@ version = "0.52.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "147a5c80aabfbf0c7d901cb5895d1de30ef2907eb21fbbab29ca94c5b08b1a78"
[[package]]
name = "windows_x86_64_gnu"
version = "0.53.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9c3842cdd74a865a8066ab39c8a7a473c0778a3f29370b5fd6b4b9aa7df4a499"
[[package]]
name = "windows_x86_64_gnullvm"
version = "0.48.5"
@@ -2297,12 +2195,6 @@ version = "0.52.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "24d5b23dc417412679681396f2b49f3de8c1473deb516bd34410872eff51ed0d"
[[package]]
name = "windows_x86_64_gnullvm"
version = "0.53.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0ffa179e2d07eee8ad8f57493436566c7cc30ac536a3379fdf008f47f6bb7ae1"
[[package]]
name = "windows_x86_64_msvc"
version = "0.48.5"
@@ -2315,12 +2207,6 @@ version = "0.52.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "589f6da84c646204747d1270a2a5661ea66ed1cced2631d546fdfb155959f9ec"
[[package]]
name = "windows_x86_64_msvc"
version = "0.53.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d6bbff5f0aada427a1e5a6da5f1f98158182f26556f345ac9e04d36d0ebed650"
[[package]]
name = "winnow"
version = "1.0.0"
+6
View File
@@ -43,6 +43,12 @@ anyhow = "1.0.40"
text_placeholder = { version = "0.5", features = ["struct_context"] }
once_cell = "1.21.4"
[target.'cfg(unix)'.dependencies]
libc = "0.2"
[target.'cfg(windows)'.dependencies]
windows-sys = { version = "0.61", features = ["Win32_Networking_WinSock"] }
[dev-dependencies]
parameterized = "2.0.0"
criterion = { version = "0.8", features = ["html_reports"] }
+45 -1
View File
@@ -2,7 +2,7 @@ use criterion::{criterion_group, criterion_main, Criterion};
use rustscan::generated::get_parsed_data;
use rustscan::input::{Opts, PortRanges, ScanOrder};
use rustscan::port_strategy::PortStrategy;
use rustscan::scanner::build_udp_payload_lookup;
use rustscan::scanner::{build_udp_payload_lookup, Scanner};
use std::collections::BTreeMap;
use std::hint::black_box;
use std::net::IpAddr;
@@ -60,6 +60,50 @@ fn criterion_benchmark(c: &mut Criterion) {
});
address_group.finish();
// Exercise production port preparation without opening a socket. The
// scanner has no target addresses, and runtime construction is not timed.
let runtime = tokio::runtime::Builder::new_current_thread()
.enable_all()
.build()
.unwrap();
let mut preparation = c.benchmark_group("port preparation");
preparation.sample_size(20);
preparation.warm_up_time(Duration::from_millis(500));
preparation.measurement_time(Duration::from_secs(2));
for (port_count, excluded_count) in [
(1, 1),
(16, 4),
(64, 1),
(64, 4096),
(4096, 0),
(4096, 64),
(4096, 4096),
(65535, 0),
(65535, 4),
(65535, 64),
(65535, 1024),
(65535, 4096),
] {
let scanner = Scanner::new(
&[],
500,
Duration::from_millis(100),
1,
true,
PortStrategy::Manual((1..=port_count).collect()),
true,
(0..excluded_count)
.map(|i| (i * 13 % 65536) as u16)
.collect(),
false,
);
preparation.bench_function(
format!("{port_count} ports, {excluded_count} exclusions"),
|b| b.iter(|| black_box(runtime.block_on(black_box(&scanner).run_with_status()))),
);
}
preparation.finish();
// UDP payload lookup micro-benchmark: compares the old linear scan of the
// payload map with the precomputed port -> payload lookup. No sockets.
let udp_map = get_parsed_data();
+151
View File
@@ -0,0 +1,151 @@
{
"metadata": {
"base": "67c08a33100362c54b56fde784b0fd234f70fa11",
"candidate": "a05592a8a88b77ad85d961137943dfadfe98b612",
"environment": "Linux-7.2.6-1-cachyos-x86_64-with-glibc2.44",
"rust": "rustc 1.98.1 (48a229cea 2026-09-01) (Arch Linux rust 1:1.98.1-1.1)",
"hosts_bytes": 2760051,
"profile": "release with LTO",
"network_namespace": "unshare --user --map-root-user --net",
"common_args": [
"--no-config",
"--config-path",
"/dev/null",
"--no-banner",
"--scripts",
"none",
"--greppable",
"--ports",
"80",
"--exclude-ports",
"80"
],
"workloads": {
"literal-ipv4": [
"-a",
"127.0.0.1"
],
"mixed-cidrs": [
"-a",
"127.0.0.1,192.0.2.0/30,2001:db8::/126",
"--exclude-addresses",
"192.0.2.1,2001:db8::1"
]
},
"warmups_each": 2,
"samples_each": 20,
"binary_cpus": "0-3",
"harness_cpus": "8-9",
"binary_sha256": {
"base": "4d6b090218ff36249ed95f14ec1a82626e8ea33854f3823f19a6de247b1524b8",
"dns": "a9d6bf949f56c53d18cc85bc9b53b21bd85a63bcdd04ad78992d392c675c77ac"
}
},
"summary": {
"literal-ipv4": {
"base_ms": 168.23068849999999,
"dns_ms": 2.248762,
"samples_each": 20
},
"mixed-cidrs": {
"base_ms": 178.0855925,
"dns_ms": 2.4539600000000004,
"samples_each": 20
}
},
"samples": {
"literal-ipv4": {
"base": [
143.391226,
146.562804,
149.615525,
151.188597,
152.269193,
159.499601,
164.926958,
164.347529,
167.208931,
165.433184,
172.246939,
169.252446,
172.777899,
170.729249,
170.092681,
171.863255,
169.612842,
170.857396,
173.117403,
175.206345
],
"dns": [
2.241793,
1.817341,
2.39956,
2.048986,
3.123401,
1.820017,
3.615094,
2.165312,
2.675535,
2.103261,
2.531392,
2.215141,
2.572939,
2.063803,
2.671431,
2.059846,
2.598858,
2.230916,
2.678259,
2.255731
]
},
"mixed-cidrs": {
"base": [
175.504767,
178.797842,
186.183728,
176.885958,
177.989816,
180.028568,
177.66299,
174.757221,
175.468485,
175.328023,
177.639639,
175.637593,
177.922435,
185.394396,
182.587204,
180.58311,
178.961137,
178.293186,
180.482419,
178.181369
],
"dns": [
3.275193,
2.114074,
2.700771,
2.183317,
2.606538,
2.368046,
2.599702,
2.238105,
2.683892,
2.292249,
2.528366,
2.114478,
2.668921,
2.103271,
3.275514,
2.27974,
3.103311,
2.265298,
2.580743,
2.379554
]
}
},
"invocations": 88
}
+48
View File
@@ -0,0 +1,48 @@
# Lazy DNS initialization benchmark
Compared current Tokio master `67c08a33100362c54b56fde784b0fd234f70fa11` with the lazy-resolver implementation
at `a05592a8a88b77ad85d961137943dfadfe98b612` on 2026-10-02.
## Method
Both binaries are release builds with LTO, using Rust 1.98.1 on an Intel Core
Ultra 7 165U running Linux 7.2.6-1-cachyos. Each workload used twenty measured
runs and two warmups per binary, alternating A/B and B/A order. Binaries used
CPUs 0-3; the harness used CPUs 8-9. No builds ran during measurement.
The harness ran in a private network namespace. Every invocation excluded
its only requested port, so no scan traffic was sent. Configuration and
scripts were disabled. Timing spans process launch through exit, including
the same `taskset` affinity launcher for both builds.
Common arguments:
```sh
--no-config --config-path /dev/null --no-banner --scripts none --greppable \
--ports 80 --exclude-ports 80
```
The literal case adds `-a 127.0.0.1`. The mixed case adds
`-a 127.0.0.1,192.0.2.0/30,2001:db8::/126` and
`--exclude-addresses 192.0.2.1,2001:db8::1`. Both use `RUST_LOG=rustscan=info`.
## Results
| Workload | Master median | Lazy DNS median | Time saved |
| --- | ---: | ---: | ---: |
| Literal IPv4 | 168.23 ms | 2.25 ms | 165.98 ms (98.66%) |
| IPv4/IPv6 CIDRs and exclusions | 178.09 ms | 2.45 ms | 175.63 ms (98.62%) |
All 88 invocations exited successfully with the expected empty scan output.
A separate syscall trace of the optimized literal-IP invocation recorded no
reads of `/etc/hosts` or `/etc/resolv.conf` and no network connects or sends.
This host has a 2,760,051-byte hosts file. Eager Hickory initialization parses
that file even for numeric targets. Systems with smaller hosts files will
save less; this measures CLI startup, not socket throughput. Hostnames that
need fallback DNS still pay resolver initialization once, and parsing order,
file handling, deduplication and exclusions keep their existing behavior.
[Raw samples, setup and binary hashes](lazy-dns-benchmark-results.json) are
included. Network-free regression tests verify that literal IPv4/IPv6 targets,
CIDRs, exclusions, duplicates and empty inputs do not construct the resolver.
File diff suppressed because it is too large. Load diff
+114
View File
@@ -0,0 +1,114 @@
# Port exclusion preparation benchmark
Compared Tokio master `b7119f7f3eb1601e81126ef3572699ed7b3b7c0b` with
implementation `270561d553688acbd99f61c460cdfa7c76c1423e` on 2026-10-02.
## Method
Release builds with LTO, Rust 1.98.1, Intel Core Ultra 7 165U, Linux
7.2.6-1-cachyos. Both versions use the same lockfile and identical
`benches/benchmark_helpers.rs`; copy the final benchmark harness into the
baseline worktree before building it. The benchmark runs the production
`Scanner::run_with_status` with **zero target addresses**, so it opens no
sockets and does no DNS or scripting. It includes generating/cloning the
port list, filtering and empty scan setup. Scanner construction and the
current-thread Tokio runtime construction are outside the timer.
Manual ports are `1..=port_count`; exclusions are
`(i * 13 % 65536) as u16` for `i` in `0..excluded_count`. Each case has two
independent twenty-sample Criterion runs per build, a 500 ms warmup and a
2 s target measurement. Cases run in A/B then B/A order on CPU 0. No builds
or other local checks ran during timing. CPU frequency was not fixed on
this shared laptop, so the table retains both repetitions rather than
selecting one result. Values are Criterion's mean point estimates.
Build command for each worktree:
```sh
cargo bench --locked --bench benchmark_helpers --no-run
```
The saved benchmark executables are then run serially, for each case:
```sh
CARGO_TARGET_DIR=/tmp/preparation-results taskset -c 0 /path/to/benchmark_helpers \
--bench 'port preparation/65535 ports, 4096 exclusions' --save-baseline base-a
```
Repeat with the candidate binary as `head-a`, then the candidate as
`head-b` and the baseline as `base-b`. To run the group normally without
comparing builds:
```sh
cargo bench --locked --bench benchmark_helpers -- 'port preparation'
```
## Results
Times are microseconds; A and B are separate matched repetitions.
| Ports | Exclusions | Master µs A / B | Optimized µs A / B | Time saved A / B |
| ---: | ---: | ---: | ---: | ---: |
| 1 | 1 | 0.289 / 0.286 | 0.261 / 0.271 | 9.7% / 5.2% |
| 16 | 4 | 0.260 / 0.268 | 0.232 / 0.237 | 10.6% / 11.6% |
| 64 | 1 | 0.433 / 0.437 | 0.237 / 0.239 | 45.2% / 45.4% |
| 64 | 4,096 | 6.231 / 10.093 | 2.484 / 3.038 | 60.1% / 69.9% |
| 4,096 | 0 | 10.889 / 10.037 | 0.335 / 0.397 | 96.9% / 96.0% |
| 4,096 | 64 | 20.680 / 20.363 | 2.930 / 3.064 | 85.8% / 85.0% |
| 4,096 | 4,096 | 524.223 / 519.670 | 6.805 / 6.710 | 98.7% / 98.7% |
| 65,535 | 0 | 166.113 / 115.686 | 3.655 / 2.333 | 97.8% / 98.0% |
| 65,535 | 4 | 175.717 / 181.343 | 29.769 / 30.230 | 83.1% / 83.3% |
| 65,535 | 64 | 207.617 / 206.910 | 31.595 / 31.462 | 84.8% / 84.8% |
| 65,535 | 1,024 | 2018.463 / 3271.564 | 33.403 / 33.520 | 98.3% / 99.0% |
| 65,535 | 4,096 | 8811.234 / 8469.706 | 59.853 / 63.211 | 99.3% / 99.3% |
All twelve cases improved in both repetitions. This measures **preparation,
not end-to-end network scan speed**: many scans are dominated by socket I/O,
timeouts and reporting. In particular, the default no-exclusion saving is
only a fraction of a millisecond.
## Implementation and correctness
With no exclusions, keep the already generated vector. For fewer than 64
ports, filter in place with linear membership checks; this avoids bitmap
setup and a second allocation. Larger lists use an 8 KiB bitmap covering
all 65,536 values representable by `u16`, then retain ports in place. This
changes the large-list filtering cost from ports × exclusions to ports +
exclusions, while preserving order, duplicate ports and boundary values.
The helper is kept out of line: release assembly showed an inlined bitmap
reserved 8,424 bytes in every async poll, whereas the final poll frame uses
344 bytes and the helper reserves its bitmap only during preparation.
Exploration found that in-place linear membership slows large cases. A
first production candidate also slowed 64 ports with 4,096 exclusions;
using the bitmap for that case corrected the regression before the final
matched runs. The cutoff is a performance choice, not a limit on input.
One focused network-free regression test compares the implementation with
an independent membership oracle for empty inputs, endpoints 0/65535,
duplicate ports/exclusions, all ports, and a frozen random order.
`cargo test --locked` passes 74 tests with one pre-existing ignored doctest.
All-target Clippy, formatting, documentation and diff checks pass. The
CI-only scan harness adds an excluded-port sweep using the existing
listeners, checking expected results and using the actual attempted socket
count for throughput.
[Raw Criterion samples, confidence intervals, revisions and binary hashes](port-exclusion-benchmark-results.json)
are included. Hosted correctness/performance results are linked in the PR.
## macOS runner setup
Repeated hosted sweeps on Darwin 25.6 returned `ENOBUFS` (error 55) on
both master and the candidate, missing fixture listeners. Such runs are
invalid for performance claims. The workflow provisions the TCP memory
budget once before either build is measured, raising it from the default
1/32 to 1/8 of physical memory when necessary, and prints the actual limit
and allocation counters. It retains every scenario, repeat, listener
comparison and performance threshold. An immediate untimed diagnostic
scan records errors and kernel counters if a measured pair misses a
listener; that failed dataset is not used to claim a speedup.
This uses XNU's [TCP budget initialization](https://github.com/apple-oss-distributions/xnu/blob/main/bsd/netinet/tcp_subr.c)
and [memory accounting interface](https://github.com/apple-oss-distributions/xnu/blob/main/bsd/sys/mem_acct_private.h).
It changes the disposable benchmark environment, not RustScan's behavior
or the operating system configuration on users' machines.
+82 -11
View File
@@ -1,4 +1,5 @@
//! Provides functions to parse input IP addresses, CIDRs or files.
use std::cell::LazyCell;
use std::collections::BTreeSet;
use std::fs::{self, File};
use std::io::{prelude::*, BufReader};
@@ -39,12 +40,23 @@ use crate::warning;
/// start your runtime (as the `rustscan` binary does) or from
/// `tokio::task::spawn_blocking`, not directly from async code.
pub fn parse_addresses(input: &Opts) -> Vec<IpAddr> {
parse_addresses_with_resolver(input, || get_resolver(&input.resolver))
}
fn parse_addresses_with_resolver(
input: &Opts,
create_resolver: impl FnOnce() -> Resolver,
) -> Vec<IpAddr> {
let mut ips: Vec<IpAddr> = Vec::new();
let mut unresolved_addresses: Vec<&str> = Vec::new();
let backup_resolver = get_resolver(&input.resolver);
// Most scans use literal IPs or CIDRs. Defer the resolver's runtime and
// hosts-file loading until a DNS fallback actually needs them, then reuse
// that resolver for the rest of the targets and exclusions.
let backup_resolver = LazyCell::new(create_resolver);
let resolver = || &*backup_resolver;
for address in &input.addresses {
let parsed_ips = parse_address(address, &backup_resolver);
let parsed_ips = parse_address_with_resolver(address, &resolver);
if !parsed_ips.is_empty() {
ips.extend(parsed_ips);
} else {
@@ -66,7 +78,7 @@ pub fn parse_addresses(input: &Opts) -> Vec<IpAddr> {
continue;
}
if let Ok(x) = read_ips_from_file(file_path, &backup_resolver) {
if let Ok(x) = read_ips_from_file(file_path, &resolver) {
ips.extend(x);
} else {
warning!(
@@ -77,7 +89,7 @@ pub fn parse_addresses(input: &Opts) -> Vec<IpAddr> {
}
}
let excluded_cidrs = parse_excluded_networks(&input.exclude_addresses, &backup_resolver);
let excluded_cidrs = parse_excluded_networks_with_resolver(&input.exclude_addresses, &resolver);
// Remove duplicated/excluded IPs.
let mut seen = BTreeSet::new();
@@ -100,6 +112,13 @@ pub fn parse_addresses(input: &Opts) -> Vec<IpAddr> {
/// let ips = parse_address("127.0.0.1", &Resolver::default().unwrap());
/// ```
pub fn parse_address(address: &str, resolver: &Resolver) -> Vec<IpAddr> {
parse_address_with_resolver(address, &|| resolver)
}
fn parse_address_with_resolver<'a>(
address: &str,
resolver: &impl Fn() -> &'a Resolver,
) -> Vec<IpAddr> {
if let Ok(addr) = IpAddr::from_str(address) {
// `address` is an IP string
vec![addr]
@@ -118,14 +137,17 @@ pub fn parse_address(address: &str, resolver: &Resolver) -> Vec<IpAddr> {
}
/// Uses DNS to get the IPS associated with host
fn resolve_ips_from_host(source: &str, backup_resolver: &Resolver) -> Vec<IpAddr> {
fn resolve_ips_from_host<'a>(
source: &str,
backup_resolver: &impl Fn() -> &'a Resolver,
) -> Vec<IpAddr> {
let mut ips: Vec<IpAddr> = Vec::new();
if let Ok(addrs) = source.to_socket_addrs() {
for ip in addrs {
ips.push(ip.ip());
}
} else if let Ok(addrs) = backup_resolver.lookup_ip(source) {
} else if let Ok(addrs) = backup_resolver().lookup_ip(source) {
ips.extend(addrs.iter());
}
@@ -148,6 +170,13 @@ fn resolve_ips_from_host(source: &str, backup_resolver: &Resolver) -> Vec<IpAddr
pub fn parse_excluded_networks(
exclude_addresses: &Option<Vec<String>>,
resolver: &Resolver,
) -> Vec<IpCidr> {
parse_excluded_networks_with_resolver(exclude_addresses, &|| resolver)
}
fn parse_excluded_networks_with_resolver<'a>(
exclude_addresses: &Option<Vec<String>>,
resolver: &impl Fn() -> &'a Resolver,
) -> Vec<IpCidr> {
exclude_addresses
.iter()
@@ -157,7 +186,10 @@ pub fn parse_excluded_networks(
}
/// Parses a single address into an IpCidr, handling CIDR notation, IP addresses, and hostnames.
fn parse_single_excluded_address(addr: &str, resolver: &Resolver) -> Vec<IpCidr> {
fn parse_single_excluded_address<'a>(
addr: &str,
resolver: &impl Fn() -> &'a Resolver,
) -> Vec<IpCidr> {
if let Ok(cidr) = IpCidr::from_str(addr) {
return vec![cidr];
}
@@ -261,9 +293,9 @@ fn read_resolver_from_file(path: &str) -> Result<Vec<IpAddr>, std::io::Error> {
#[cfg(not(tarpaulin_include))]
/// Parses an input file of IPs and uses those
fn read_ips_from_file(
fn read_ips_from_file<'a>(
ips: &std::path::Path,
backup_resolver: &Resolver,
backup_resolver: &impl Fn() -> &'a Resolver,
) -> Result<Vec<IpAddr>, std::io::Error> {
let file = File::open(ips)?;
let reader = BufReader::new(file);
@@ -272,7 +304,7 @@ fn read_ips_from_file(
for address_line in reader.lines() {
if let Ok(address) = address_line {
ips.extend(parse_address(&address, backup_resolver));
ips.extend(parse_address_with_resolver(&address, backup_resolver));
} else {
debug!("Line in file is not valid");
}
@@ -283,9 +315,48 @@ fn read_ips_from_file(
#[cfg(test)]
mod tests {
use super::{parse_addresses, Opts};
use super::{parse_addresses, parse_addresses_with_resolver, Opts};
use std::net::{IpAddr, Ipv4Addr};
#[test]
fn literal_targets_and_exclusions_do_not_initialize_a_resolver() {
let opts = Opts {
addresses: vec![
"192.0.2.0/30".to_owned(),
"192.0.2.2".to_owned(),
"2001:db8::/126".to_owned(),
"2001:db8::3".to_owned(),
],
exclude_addresses: Some(vec![
"192.0.2.0/31".to_owned(),
"192.0.2.3".to_owned(),
"2001:db8::/127".to_owned(),
"2001:db8::3".to_owned(),
]),
..Default::default()
};
let ips = parse_addresses_with_resolver(&opts, || {
panic!("literal targets and exclusions must not initialize DNS")
});
assert_eq!(
ips,
[
"192.0.2.2".parse::<IpAddr>().unwrap(),
"2001:db8::2".parse::<IpAddr>().unwrap(),
]
);
}
#[test]
fn empty_targets_do_not_initialize_a_resolver() {
assert!(parse_addresses_with_resolver(&Opts::default(), || {
panic!("empty targets must not initialize DNS")
})
.is_empty());
}
#[test]
fn parse_correct_addresses() {
let opts = Opts {
+181 -6
View File
@@ -1,6 +1,9 @@
//! Provides a means to read, parse and hold configuration options for scans.
use clap::{Parser, ValueEnum};
use clap::parser::ValueSource;
use clap::{CommandFactory, FromArgMatches, Parser, ValueEnum};
use serde_derive::Deserialize;
use std::collections::HashSet;
use std::ffi::OsString;
use std::fs;
use std::path::PathBuf;
@@ -22,8 +25,11 @@ pub enum ScanOrder {
/// - custom will read the ScriptConfig file and the available scripts in the predefined folders
#[derive(Deserialize, Debug, ValueEnum, Clone, PartialEq, Eq, Copy)]
pub enum ScriptsRequired {
#[serde(alias = "none")]
None,
#[serde(alias = "default")]
Default,
#[serde(alias = "custom")]
Custom,
}
@@ -231,18 +237,48 @@ pub struct Opts {
/// scanning the next one, for slow, low-noise scans. 0 disables the delay.
#[arg(long, default_value = "0", value_name = "MS")]
pub interval: u64,
/// Ids of the options given explicitly on the command line. The
/// configuration file never overrides these (see [`Opts::merge`]).
#[arg(skip)]
pub cli_args: HashSet<String>,
}
#[cfg(not(tarpaulin_include))]
impl Opts {
pub fn read() -> Self {
let mut opts = Opts::parse();
Self::try_read_from(std::env::args_os()).unwrap_or_else(|e| e.exit())
}
/// Parses `args` like [`Opts::read`] does, returning an error instead of
/// exiting.
fn try_read_from<I, T>(args: I) -> Result<Self, clap::Error>
where
I: IntoIterator<Item = T>,
T: Into<OsString> + Clone,
{
let mut command = Self::command();
let mut matches = command.try_get_matches_from_mut(args)?;
// clap fills in default values, so the parsed struct alone cannot
// tell `-b 4500` apart from no `-b` at all. Record what the user
// actually typed so that `merge` lets it win over the config file.
let cli_args = command
.get_arguments()
.map(|arg| arg.get_id().as_str())
.filter(|id| matches.value_source(id) == Some(ValueSource::CommandLine))
.map(str::to_owned)
.collect();
let mut opts =
Self::from_arg_matches_mut(&mut matches).map_err(|e| e.format(&mut command))?;
opts.cli_args = cli_args;
if opts.ports.is_none() && opts.range.is_none() {
opts.range = Some(PortRanges(vec![(LOWEST_PORT_NUMBER, TOP_PORT_NUMBER)]));
}
opts
Ok(opts)
}
/// Validates options whose availability or semantics depend on the
@@ -267,7 +303,8 @@ impl Opts {
}
/// Reads the command line arguments into an Opts struct and merge
/// values found within the user configuration file.
/// values found within the user configuration file. Options given on the
/// command line take precedence over the configuration file.
pub fn merge(&mut self, config: &Config) {
if !self.no_config {
self.merge_required(config);
@@ -280,7 +317,9 @@ impl Opts {
($($field: ident),+) => {
$(
if let Some(e) = &config.$field {
self.$field = e.clone();
if !self.cli_args.contains(stringify!($field)) {
self.$field = e.clone();
}
}
)+
}
@@ -296,7 +335,7 @@ impl Opts {
macro_rules! merge_optional {
($($field: ident),+) => {
$(
if config.$field.is_some() {
if config.$field.is_some() && !self.cli_args.contains(stringify!($field)) {
self.$field = config.$field.clone();
}
)+
@@ -337,6 +376,7 @@ impl Default for Opts {
udp: false,
closed: false,
interval: 0,
cli_args: HashSet::new(),
}
}
}
@@ -459,6 +499,7 @@ pub fn old_default_config_path() -> PathBuf {
mod tests {
use clap::{CommandFactory, Parser};
use parameterized::parameterized;
use std::collections::HashSet;
use super::{Config, Opts, PortRanges, ScanOrder, ScriptsRequired};
@@ -488,6 +529,55 @@ mod tests {
}
}
fn assert_config_script_mode(value: &str, expected: ScriptsRequired) {
let config: Config = toml::from_str(&format!("scripts = \"{value}\""))
.expect("documented script mode must deserialize");
assert_eq!(config.scripts, Some(expected));
}
#[test]
fn config_scripts_lowercase_none() {
assert_config_script_mode("none", ScriptsRequired::None);
}
#[test]
fn config_scripts_lowercase_default() {
assert_config_script_mode("default", ScriptsRequired::Default);
}
#[test]
fn config_scripts_lowercase_custom() {
assert_config_script_mode("custom", ScriptsRequired::Custom);
}
#[test]
fn config_scripts_legacy_and_unknown_controls() {
assert_config_script_mode("None", ScriptsRequired::None);
assert_config_script_mode("Default", ScriptsRequired::Default);
assert_config_script_mode("Custom", ScriptsRequired::Custom);
for invalid in ["disabled", "unknown", ""] {
assert!(toml::from_str::<Config>(&format!("scripts = \"{invalid}\"")).is_err());
}
}
#[test]
fn config_read_accepts_documented_lowercase_script_modes() {
for (value, expected) in [
("none", ScriptsRequired::None),
("default", ScriptsRequired::Default),
("custom", ScriptsRequired::Custom),
] {
let fixture = std::env::temp_dir().join(format!(
"rustscan-config-scripts-{}-{value}.toml",
std::process::id()
));
std::fs::write(&fixture, format!("scripts = \"{value}\"\n")).unwrap();
let config = Config::read(Some(fixture.clone()));
std::fs::remove_file(fixture).unwrap();
assert_eq!(config.scripts, Some(expected));
}
}
#[test]
fn verify_cli() {
Opts::command().debug_assert();
@@ -635,6 +725,91 @@ mod tests {
assert_eq!(opts.resolver, config.resolver);
}
#[test]
fn cli_arguments_override_config() {
// Regression test for #722: the config file used to silently replace
// options given on the command line, e.g. `-r` with its `range`.
let mut opts = Opts::try_read_from([
"rustscan",
"-a",
"192.168.0.1",
"-r",
"8600-8650",
"-b",
"100",
// Passing clap's default value explicitly must still win.
"-t",
"1500",
"--tries",
"3",
"--scan-order",
"serial",
"--scripts",
"none",
"-g",
"--",
"-sV",
])
.unwrap();
let config: Config = toml::from_str(
r#"
addresses = ["127.0.0.1"]
range = { start = 1, end = 100 }
batch_size = 25000
timeout = 1000
tries = 1
scan_order = "Random"
scripts = "Custom"
greppable = false
command = ["-A"]
"#,
)
.unwrap();
opts.merge(&config);
assert_eq!(opts.addresses, vec!["192.168.0.1".to_owned()]);
assert_eq!(opts.range, Some(PortRanges(vec![(8_600, 8_650)])));
assert_eq!(opts.batch_size, 100);
assert_eq!(opts.timeout, 1_500);
assert_eq!(opts.tries, 3);
assert_eq!(opts.scan_order, ScanOrder::Serial);
assert_eq!(opts.scripts, ScriptsRequired::None);
assert!(opts.greppable);
assert_eq!(opts.command, vec!["-sV".to_owned()]);
}
#[test]
fn config_fills_arguments_not_given_on_cli() {
let mut opts = Opts::try_read_from(["rustscan", "-a", "192.168.0.1"]).unwrap();
let config: Config = toml::from_str(
r#"
addresses = ["127.0.0.1"]
range = { start = 1, end = 100 }
batch_size = 25000
timeout = 1000
tries = 3
scan_order = "Random"
greppable = true
command = ["-A"]
"#,
)
.unwrap();
opts.merge(&config);
// Only `-a` was typed; clap defaults such as `-b 4500` must not count.
assert_eq!(opts.cli_args, HashSet::from(["addresses".to_owned()]));
assert_eq!(opts.addresses, vec!["192.168.0.1".to_owned()]);
assert_eq!(opts.range, Some(PortRanges(vec![(1, 100)])));
assert_eq!(opts.batch_size, 25_000);
assert_eq!(opts.timeout, 1_000);
assert_eq!(opts.tries, 3);
assert_eq!(opts.scan_order, ScanOrder::Random);
assert!(opts.greppable);
assert_eq!(opts.command, vec!["-A".to_owned()]);
}
#[test]
fn parses_comma_separated_ranges() {
let opts = Opts::parse_from([
+72 -15
View File
@@ -128,20 +128,7 @@ fn main() {
map.entry(socket.ip()).or_default().push(socket.port());
}
for ip in ips {
if ports_per_ip.contains_key(&ip) {
continue;
}
// If we got here it means the IP was not found within the HashMap, this
// means the scan couldn't find any open ports for it.
let x = format!("Looks like I didn't find any open ports for {:?}. This is usually caused by a high batch size.
\n*I used {} batch size, consider lowering it with {} or a comfortable number for your system.
\n Alternatively, increase the timeout if your ping is high. Rustscan -t 2000 for 2000 milliseconds (2s) timeout.\n",
ip,
batch_size,
"'rustscan -b <batch_size> -a <ip address>'");
if let Some(x) = no_open_ports_warning(&ips, &ports_per_ip, &closed_ports_per_ip, batch_size) {
warning!(x, opts.greppable, opts.accessible);
}
@@ -224,6 +211,35 @@ fn main() {
info!("{}", benchmarks.summary());
}
/// Builds the hint printed when the scan found no open ports.
///
/// A host without open ports is normal when another host answered (a subnet
/// sweep for one port): replies got through at this batch size and timeout,
/// so batch size and timeout are unlikely to be the cause. The hint is printed
/// once, and only when no host answered with an open or, under `--closed`, a
/// closed port.
fn no_open_ports_warning(
ips: &[IpAddr],
ports_per_ip: &HashMap<IpAddr, Vec<u16>>,
closed_ports_per_ip: &HashMap<IpAddr, Vec<u16>>,
batch_size: usize,
) -> Option<String> {
if !ports_per_ip.is_empty() || !closed_ports_per_ip.is_empty() {
return None;
}
let target = match ips {
[ip] => format!("for {ip:?}"),
_ => format!("on any of the {} addresses", ips.len()),
};
Some(format!("Looks like I didn't find any open ports {}. This is usually caused by a high batch size.
\n*I used {} batch size, consider lowering it with {} or a comfortable number for your system.
\n Alternatively, increase the timeout if your ping is high. Rustscan -t 2000 for 2000 milliseconds (2s) timeout.\n",
target,
batch_size,
"'rustscan -b <batch_size> -a <ip address>'"))
}
/// Determines the actual batch size used by the scanner.
///
/// Unix systems may reduce the requested batch size according to the process
@@ -373,7 +389,48 @@ mod tests {
use super::effective_batch_size;
#[cfg(unix)]
use super::{adjust_ulimit_size, infer_batch_size};
use super::{print_opening, Opts};
use super::{no_open_ports_warning, print_opening, Opts};
use std::collections::HashMap;
use std::net::IpAddr;
fn hosts(n: u8) -> Vec<IpAddr> {
(1..=n).map(|i| IpAddr::from([192, 0, 2, i])).collect()
}
#[test]
fn no_open_ports_silent_when_another_host_answered() {
// Sweeping a subnet for one port: one host has it open, the rest do not.
let ips = hosts(254);
let ports_per_ip = HashMap::from([(ips[0], vec![22])]);
assert!(no_open_ports_warning(&ips, &ports_per_ip, &HashMap::new(), 4500).is_none());
}
#[test]
fn no_open_ports_silent_when_a_host_answered_closed() {
// With --closed, a refused connection also proves replies get through.
let ips = hosts(254);
let closed_ports_per_ip = HashMap::from([(ips[0], vec![443])]);
assert!(no_open_ports_warning(&ips, &HashMap::new(), &closed_ports_per_ip, 4500).is_none());
}
#[test]
fn no_open_ports_warns_once_when_nothing_found() {
let warning = no_open_ports_warning(&hosts(254), &HashMap::new(), &HashMap::new(), 4500);
let warning = warning.expect("one hint when nothing was found");
assert!(warning.contains("on any of the 254 addresses."));
assert!(warning.contains("4500 batch size"));
}
#[test]
fn no_open_ports_names_the_single_host() {
let warning = no_open_ports_warning(&hosts(1), &HashMap::new(), &HashMap::new(), 4500);
let warning = warning.expect("one hint for the single host");
assert!(warning.contains("for 192.0.2.1."));
}
#[test]
#[cfg(unix)]
+163
View File
@@ -0,0 +1,163 @@
use std::{collections::HashSet, fmt, io, net::IpAddr};
/// Keep the OS error and its target intact until diagnostics need formatting.
#[derive(Debug)]
struct ScanError {
ip: IpAddr,
error: io::Error,
}
impl fmt::Display for ScanError {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
write!(f, "{} {}", self.error, self.ip)
}
}
impl std::error::Error for ScanError {}
/// Preserve the normal I/O result layout; only diagnostics need target context.
pub(super) fn diagnostic_error(error: io::Error, ip: IpAddr, enabled: bool) -> io::Error {
if enabled {
io::Error::other(ScanError { ip, error })
} else {
error
}
}
pub(super) struct ScanErrors {
enabled: bool,
limit: usize,
messages: HashSet<String>,
}
impl ScanErrors {
pub(super) fn new(enabled: bool, limit: usize) -> Self {
Self {
enabled,
limit,
messages: HashSet::new(),
}
}
pub(super) fn record(&mut self, failure: io::Error) {
if self.enabled && self.messages.len() < self.limit {
self.messages.insert(failure.to_string());
}
}
pub(super) fn messages(&self) -> &HashSet<String> {
&self.messages
}
}
/// Detect descriptor exhaustion without allocating or interpreting OS messages.
#[cfg(unix)]
pub(super) fn is_descriptor_exhaustion(error: &io::Error) -> bool {
matches!(error.raw_os_error(), Some(libc::EMFILE | libc::ENFILE))
}
#[cfg(windows)]
pub(super) fn is_descriptor_exhaustion(error: &io::Error) -> bool {
error.raw_os_error() == Some(windows_sys::Win32::Networking::WinSock::WSAEMFILE)
}
// Retain the previous fallback for targets without Unix or Winsock error codes.
#[cfg(not(any(unix, windows)))]
pub(super) fn is_descriptor_exhaustion(error: &io::Error) -> bool {
error
.to_string()
.to_lowercase()
.contains("too many open files")
}
#[cfg(test)]
mod tests {
use super::*;
use std::{fmt, net::SocketAddr};
#[derive(Debug)]
struct MustNotFormat;
impl fmt::Display for MustNotFormat {
fn fmt(&self, _: &mut fmt::Formatter<'_>) -> fmt::Result {
panic!("discarded diagnostics must not format errors")
}
}
impl std::error::Error for MustNotFormat {}
fn failure(error: io::Error) -> io::Error {
diagnostic_error(error, "127.0.0.1".parse().unwrap(), true)
}
#[test]
fn disabled_diagnostics_do_not_format_errors() {
let mut errors = ScanErrors::new(false, 1000);
errors.record(diagnostic_error(
io::Error::other(MustNotFormat),
"127.0.0.1".parse().unwrap(),
false,
));
assert!(errors.messages().is_empty());
}
#[test]
fn disabled_diagnostics_preserve_the_original_os_error() {
let original = io::Error::from_raw_os_error(123);
let error = diagnostic_error(original, "127.0.0.1".parse().unwrap(), false);
assert_eq!(error.raw_os_error(), Some(123));
}
#[test]
fn full_diagnostics_do_not_format_more_errors() {
let mut errors = ScanErrors::new(true, 1);
errors.record(failure(io::ErrorKind::ConnectionRefused.into()));
errors.record(failure(io::Error::other(MustNotFormat)));
assert_eq!(errors.messages().len(), 1);
}
#[test]
fn enabled_diagnostics_keep_error_details_and_deduplicate_per_ip() {
let mut errors = ScanErrors::new(true, 1000);
for port in [80, 443] {
errors.record(diagnostic_error(
io::Error::other("test socket error"),
SocketAddr::from(([127, 0, 0, 1], port)).ip(),
true,
));
}
assert_eq!(errors.messages().len(), 1);
assert!(errors.messages().contains("test socket error 127.0.0.1"));
}
#[cfg(unix)]
#[test]
fn detects_unix_descriptor_limits_by_code() {
for code in [libc::EMFILE, libc::ENFILE] {
assert!(is_descriptor_exhaustion(&io::Error::from_raw_os_error(
code
)));
}
assert!(!is_descriptor_exhaustion(&io::Error::from_raw_os_error(
libc::ECONNREFUSED
)));
}
#[cfg(windows)]
#[test]
fn detects_winsock_descriptor_limits_by_code() {
use windows_sys::Win32::Networking::WinSock::{WSAECONNREFUSED, WSAEMFILE};
assert!(is_descriptor_exhaustion(&io::Error::from_raw_os_error(
WSAEMFILE
)));
assert!(!is_descriptor_exhaustion(&io::Error::from_raw_os_error(
WSAECONNREFUSED
)));
}
#[cfg(any(unix, windows))]
#[test]
fn descriptor_check_never_formats_an_error() {
assert!(!is_descriptor_exhaustion(&io::Error::other(MustNotFormat)));
}
}
+72 -26
View File
@@ -7,13 +7,16 @@ use log::debug;
mod socket_iterator;
use socket_iterator::SocketIterator;
mod errors;
use errors::{diagnostic_error, is_descriptor_exhaustion, ScanErrors};
use colored::Colorize;
use futures::stream::{FuturesUnordered, StreamExt};
use std::collections::BTreeMap;
use std::future::poll_fn;
use std::task::Poll;
use std::{
collections::{HashMap, HashSet},
collections::HashMap,
io,
net::{IpAddr, Ipv4Addr, Ipv6Addr, Shutdown, SocketAddr},
num::NonZeroU8,
@@ -32,6 +35,29 @@ use tokio::time::{sleep, timeout};
/// polls to about a millisecond on Linux.
const WORK_PER_TURN: usize = 128;
/// Keeps the generated order, including duplicate ports, while removing exclusions.
// Keep the bitmap out of the async polling frame, which runs on every wake.
#[inline(never)]
fn filter_excluded_ports(mut ports: Vec<u16>, excluded: &[u16]) -> Vec<u16> {
if excluded.is_empty() {
return ports;
}
// Bitmap setup costs more than membership checks on short port lists.
if ports.len() < 64 {
ports.retain(|port| !excluded.contains(port));
return ports;
}
// The complete u16 port space fits in an 8 KiB bitmap.
let mut excluded_bits = [0_u64; 1024];
for &port in excluded {
excluded_bits[usize::from(port) / 64] |= 1 << (port % 64);
}
ports.retain(|&port| excluded_bits[usize::from(port) / 64] & (1 << (port % 64)) == 0);
ports
}
/// UDP payload lookup: port -> payload bytes
///
/// `get_parsed_data()` returns a `&'static BTreeMap<...>`, so we can store
@@ -190,15 +216,10 @@ impl Scanner {
}
async fn scan(&self) -> Vec<PortStatus> {
let ports: Vec<u16> = self
.port_strategy
.order()
.iter()
.filter(|&port| !self.exclude_ports.contains(port))
.copied()
.collect();
let ports = filter_excluded_ports(self.port_strategy.order(), &self.exclude_ports);
let mut found_sockets: Vec<PortStatus> = Vec::new();
let mut errors: HashSet<String> = HashSet::new();
let mut errors =
ScanErrors::new(log::log_enabled!(log::Level::Debug), self.ips.len() * 1000);
// Build UDP payload lookup once (only if we are scanning UDP).
// This avoids cloning a big map into every spawned future and turns
@@ -233,7 +254,7 @@ impl Scanner {
}
}
debug!("Typical socket connection errors {errors:?}");
debug!("Typical socket connection errors {:?}", errors.messages());
debug!("Sockets found: {:?}", found_sockets);
found_sockets
}
@@ -254,7 +275,7 @@ impl Scanner {
mut sockets: SocketIterator<'_>,
udp_payloads: &Option<Arc<UdpPayloadLookup>>,
found_sockets: &mut Vec<PortStatus>,
errors: &mut HashSet<String>,
errors: &mut ScanErrors,
) {
let mut ftrs = FuturesUnordered::new();
@@ -302,24 +323,19 @@ impl Scanner {
&self,
result: io::Result<PortStatus>,
found_sockets: &mut Vec<PortStatus>,
errors: &mut HashSet<String>,
errors: &mut ScanErrors,
) {
match result {
Ok(status) => found_sockets.push(status),
Err(e) => {
let error_string = e.to_string();
if errors.len() < self.ips.len() * 1000 {
errors.insert(error_string);
}
}
Err(error) => errors.record(error),
}
}
/// Given a socket, scan it self.tries times.
/// Turns the address into a SocketAddr
/// Deals with the `<result>` type
/// If it experiences error ErrorKind::Other then too many files are open and it Panics!
/// Else any other error, it returns the error in Result as a string
/// Panics if the OS reports descriptor exhaustion.
/// Other failures retain their I/O error, with target context for debug diagnostics.
/// If no errors occur, it returns the port number in Result to signify the port is open.
/// This function mainly deals with the logic of Results handling.
/// # Example
@@ -359,14 +375,14 @@ impl Scanner {
return Ok(PortStatus::Closed(socket));
}
let mut error_string = e.to_string();
assert!(!error_string.to_lowercase().contains("too many open files"), "Too many open files. Please reduce batch size. The default is 5000. Try -b 2500.");
assert!(!is_descriptor_exhaustion(&e), "Too many open files. Please reduce batch size. The default is 5000. Try -b 2500.");
if nr_try == tries {
error_string.push(' ');
error_string.push_str(&socket.ip().to_string());
return Err(io::Error::other(error_string));
return Err(diagnostic_error(
e,
socket.ip(),
log::log_enabled!(log::Level::Debug),
));
}
}
};
@@ -589,6 +605,7 @@ mod tests {
use super::*;
use crate::input::{PortRanges, ScanOrder};
use std::collections::HashSet;
// These tests never open sockets: they only build a `Scanner` (and its
// futures), run a scan that has no socket to scan, or inspect the payload
@@ -610,6 +627,35 @@ mod tests {
)
}
#[test]
fn port_exclusions_preserve_order_duplicates_and_boundaries() {
let inputs = [
Vec::new(),
vec![65535, 0, 80, 443, 80, 65535, 1],
(0..=65535).collect(),
PortStrategy::pick(&Some(PortRanges(vec![(0, 1023)])), None, ScanOrder::Random).order(),
];
let exclusions = [
Vec::new(),
vec![0],
vec![0, 65535, 0, 443],
(0..1024).collect(),
(0..=65535).collect(),
];
for ports in inputs {
for excluded in &exclusions {
let excluded_set: HashSet<_> = excluded.iter().copied().collect();
let expected: Vec<u16> = ports
.iter()
.filter(|&port| !excluded_set.contains(port))
.copied()
.collect();
assert_eq!(filter_excluded_ports(ports.clone(), excluded), expected);
}
}
}
#[test]
fn library_scanner_is_quiet_by_default() {
let scanner = test_scanner();
+121 -2
View File
@@ -143,9 +143,78 @@ pub fn init_scripts(scripts: &ScriptsRequired) -> Result<Vec<ScriptFile>> {
}
}
verify_script_programs(&scripts_to_run)?;
Ok(scripts_to_run)
}
/// Whether `path/program` is something the shell could run.
#[cfg(not(windows))]
fn program_exists_in(dir: &std::path::Path, program: &str) -> bool {
dir.join(program).is_file()
}
/// Whether `path/program` is something `cmd.exe` could run, which appends
/// entries from `PATHEXT` to a bare name.
#[cfg(windows)]
fn program_exists_in(dir: &std::path::Path, program: &str) -> bool {
if dir.join(program).is_file() {
return true;
}
let extensions = std::env::var("PATHEXT").unwrap_or_default();
extensions
.split(';')
.filter(|extension| !extension.is_empty())
.any(|extension| dir.join(format!("{program}{extension}")).is_file())
}
fn program_is_available(program: &str) -> bool {
match std::env::var_os("PATH") {
Some(path) => std::env::split_paths(&path).any(|dir| program_exists_in(&dir, program)),
None => false,
}
}
/// Check that the program each selected script starts with is available.
///
/// Scripts run through `sh -c`, so a missing binary surfaces only as a bare
/// "Exit code = 127" once the scan has already finished. Report it up front
/// instead, before a long scan has been wasted.
fn verify_script_programs(scripts_to_run: &[ScriptFile]) -> Result<()> {
for script in scripts_to_run {
let call_format = match &script.call_format {
Some(call_format) => call_format,
None => continue,
};
let program = match call_format.split_whitespace().next() {
Some(program) => program,
None => continue,
};
// A leading assignment (`FOO=bar nmap ...`) or an absolute path is not
// a bare program name, and `{{...}}` placeholders are only filled in
// when the script actually runs, so leave those formats alone.
if program.contains('=')
|| program.contains('/')
|| program.contains('$')
|| program.contains("{{")
{
continue;
}
if !program_is_available(program) {
return Err(anyhow!(
"The script `{}` runs `{}`, which is not installed or not in PATH. \
Install it, or skip scripts with `--scripts none`.",
script
.path
.as_ref()
.map_or_else(|| "default".to_string(), |path| path.display().to_string()),
program
));
}
}
Ok(())
}
pub fn parse_scripts(scripts: Vec<PathBuf>) -> Vec<ScriptFile> {
let mut parsed_scripts: Vec<ScriptFile> = Vec::with_capacity(scripts.len());
for script in scripts {
@@ -348,8 +417,7 @@ impl ScriptFile {
if let Ok(file) = File::open(script) {
for mut line in io::BufReader::new(file).lines().skip(1).flatten() {
if line.starts_with('#') {
line.retain(|c| c != '#');
line = line.trim().to_string();
line = line[1..].trim().to_string();
line.push('\n');
lines_buf.push_str(&line);
} else {
@@ -403,6 +471,21 @@ impl ScriptConfig {
mod tests {
use super::*;
#[test]
fn script_headers_preserve_literal_hash_characters() {
let fixture =
std::env::temp_dir().join(format!("rustscan-literal-hash-{}.txt", std::process::id()));
fs::write(&fixture, "#!/usr/bin/env python3\n#tags = [\"C#\"]\n#call_format = \"fixture {{ip}} {{port}} https://example.invalid/#section\"\n\n").unwrap();
let parsed = ScriptFile::new(fixture.clone());
fs::remove_file(fixture).unwrap();
let parsed = parsed.expect("valid TOML metadata");
assert_eq!(parsed.tags, Some(vec!["C#".to_owned()]));
assert_eq!(
parsed.call_format.as_deref(),
Some("fixture {{ip}} {{port}} https://example.invalid/#section")
);
}
#[test]
fn find_and_parse_scripts() {
let scripts = find_scripts("fixtures/.rustscan_scripts".into()).unwrap();
@@ -495,4 +578,40 @@ mod tests {
assert_eq!(script_dir_base, dirs::home_dir().unwrap());
}
fn script_with_call_format(call_format: &str) -> ScriptFile {
ScriptFile {
path: Some(PathBuf::from("/tmp/script.txt")),
tags: None,
developer: None,
port: None,
ports_separator: None,
call_format: Some(call_format.to_string()),
}
}
#[test]
fn missing_script_program_is_reported() {
let scripts = vec![script_with_call_format("definitely-not-a-real-binary {{ip}}")];
let err = verify_script_programs(&scripts).unwrap_err();
assert!(err.to_string().contains("definitely-not-a-real-binary"));
}
#[test]
fn available_script_program_is_accepted() {
// `sh` is what the scripts are actually run through, so it is present
// on every platform that can execute them.
let scripts = vec![script_with_call_format("sh -c true {{ip}}")];
assert!(verify_script_programs(&scripts).is_ok());
}
#[test]
fn placeholder_and_assignment_prefixes_are_skipped() {
let scripts = vec![
script_with_call_format("{{script}} {{ip}} {{port}}"),
script_with_call_format("FOO=bar definitely-not-a-real-binary"),
script_with_call_format("/opt/custom/nmap {{ip}}"),
];
assert!(verify_script_programs(&scripts).is_ok());
}
}