Files
0620b24004 fix(engine+eval): 11 verified correctness/safety fixes (failed-hunt, TLS, auth-sweep FP, +8) (#50)
* fix(engine): 3 verified council findings — F7 (silent failed-hunt), F32 (TLS), F43 (auth-sweep FP)

F7 — engine/agent.py + engine/engine.py: agent hard errors (timeout / rate-limit /
  parse-fail / missing 'claude' CLI) were recorded as 'tested, found nothing' and never
  retried; sequential mode also crashed on a missing CLI. Now: catch FileNotFoundError/
  OSError in run_agent; _hunt_agent flags errored/no-verdict results; hunt() leaves those
  items un-tested (stay on worklist for retry) instead of marking them tested-negative.

F32 — cbh/cli.py: an ambient corporate HTTPS_PROXY/HTTP_PROXY silently set ssl.CERT_NONE
  (TLS verification off). Now CERT_NONE is gated behind an explicit Burp/insecure opt-in
  (--burp/--proxy or CBH_BURP_PROXY); ambient proxy env keeps verification ON.

F43 — engine/recon.py: a global OpenAPI 'security: []' (= public API) was treated as
  secured, producing false-positive 'broken function-level auth' findings. Now mirrors the
  op-level check: glob = spec.get('security') not in (None, []).

Verified: all files compile; agent.py/state.py self-tests pass; --mock dry-run unchanged;
added behavioral checks for each fix (errored-not-tested, ambient-proxy-TLS-on, security:[]-not-swept).

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>

* fix(engine+eval): 8 more verified council findings (F8/F9/F41/F24/F23/F12/F14/F20)

Core engine:
- F8  cbh/cli.py: crt.sh outage/rate-limit/JSON-error now warns loudly instead of
      silently returning empty (a failed lookup was indistinguishable from '0 subdomains').
- F9  engine/recon.py: prune_inert_params distinguishes a FAILED probe (code 0, network/
      curl error -> keep the endpoint) from a genuine HTTP >=400 dead endpoint; logs the
      unprobeable count. Transient failures no longer silently drop live surface.
- F41 engine/recon.py: mined relative routes now carry their source-document base
      (found_rel stores (base, rel)); each is resolved against the doc it came from, not the
      first seed's origin — fixes wrong-host endpoints on multi-seed runs.

Eval / labs / scripts:
- F24 docs/verification/phase3-playwright/harness.py: OAuth @-userinfo bypass payload was
      malformed (@ landed in the PATH -> browser stayed on localhost, refuting a real bug).
      Corrected to http://localhost:58020@<evil>/... (userinfo=localhost:58020, host=evil),
      which passes the server's startswith prefix AND swaps host. Docstring corrected.
- F23 eval/run_fp.py: validate --conditions against {baseline,skills}; a typo'd condition
      previously ran as baseline under the wrong label and invalidated the ablation.
- F12 eval/run_eval_ps.py: a post-run oracle read-failure (expired instance) was scored as
      solved=False (a miss). Now captures the ok flag, records solved=None + oracle_ok, and
      excludes oracle-unreadable runs from the solve-rate denominator (summary + per-class).
- F14 secret_scan.py: missing/unreadable and >10MB files were silently skipped (false
      'clean'); now warns to stderr on every skip.
- F20 eval/fp_app.py: the 'safe' /go open-redirect trap accepted /\evil.com (browsers
      normalize \ -> / -> protocol-relative external redirect), so its ground-truth 'safe'
      label was wrong. Now rejects backslashes and control chars.

Verified: all files compile; recon.py/agent.py/state.py self-tests pass; --mock flow
unchanged; added behavioral checks for F9/F41/F20/F24/F14/F23/F12.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>

---------

Co-authored-by: Sachin Sharma <[email protected]>
Co-authored-by: Claude Opus 4.8 (1M context) <[email protected]>
2026-07-27 23:55:30 +05:30
..