John SmithandClaude 87825b7bf5 feat(overhaul): AblationRunner, agent-engine adapter, bwrap sandbox, lsp_enabled field ...

Rectifies all gaps from the 2026-08-22 onboarding assessment.

Phase 0 — data model:
- Add ExecutionResult.lsp_enabled / plugins_loaded / mcp_loaded fields.
- MetricCollector.infer_tier now uses the explicit field (not bool(diagnostics)).

Phase 1 — LSP feedback loop:
- New evaluation/lsp_feedback.py with wrap_prompt_for_lsp() and
  run_lsp_iteration_loop() (max 3 rounds).

Phase 2 — AblationRunner:
- New evaluation/ablation_runner.py drives 5 explicit tier cells via
  BenchmarkRunner._run_cell; --ablation CLI flag; full Markdown proof
  matrix renders at the end of the run.

Phase 3 — Agent engine integration:
- Move scripts/agent_engine.py -> agents/agent_engine.py (importable).
- New agents/agent_engine_adapter.py registered as 'agent-engine'.
- BaseAgentAdapter._should_fallback_to_engine + _execute_with_engine_fallback
  guarded by HARNESS_BENCH_FALLBACK_ENGINE=1.

Phase 4 — Bubblewrap sandbox:
- BaseAgentAdapter._wrap_with_bwrap() opt-in via HARNESS_BENCH_USE_BWRAP=1.
  Falls back to plain cmd when bwrap is missing.

Phase 5 — Cleanup & docs:
- Delete reporting/junit.py; rewire tests/test_core_and_reporting.py to
  use the canonical metrics/junit_exporter.py.
- README: ## Prerequisites section + illustrative banners above the
  empirical tables + reproduction recipe.
- CLAUDE.md / AGENTS.md: match the new env toggles table + prerequisites.
- scripts/install_harness.sh: copy agent_engine.py from agents/ (not scripts/).

Phase 6 — verification:
- 86 tests pass (up from 55). ruff check + format clean.
- Smoke + ablation + bwrap + fallback integration paths verified locally.

Co-Authored-By: Claude <[email protected]>

2026-08-22 00:31:13 -04:00

..

install_harness.sh

feat(overhaul): AblationRunner, agent-engine adapter, bwrap sandbox, lsp_enabled field

2026-08-22 00:31:13 -04:00