Commit Graph
6 Commits
Author SHA1 Message Date
Brent Vatne 6394778533 Merge the second round of flows fixes from #74: strategy.ts session inputs wait for the session's platform on a flows run
runSessionExploration now chooses its strategy before it knows the platform, so the #74 change
is ported onto strategyInputs: a flows run without --platform reads its secrets once the
session reports its platform.
2026-10-24 11:59:55 -07:00
Brent Vatne d944e7d6d8 Flows: keep a checkpoint a budget stops, cap a described step at 40 calls, and skip flows with nothing to compare
From a second review of #74:

- A checkpoint was marked attempted before the explorer answered, so a screen, turn or time
  budget that stopped the run during the call dropped it from exploration.flows. It is now
  marked once the call returns, and recorded as "The run stopped before it." otherwise.
- A described step's 40-call limit relied on the session's maxTurns, which a Codex session
  does not have: one step could spend the whole run's budgets. runGoal now counts the calls
  and fails the step at the limit.
- A checkpoint on a screen that withholds its picture (a secret on it, a blocked screenshot)
  is recorded without one and its flow goes on, as the code did; the README said it ended the
  flow. The docs now say so, and the run's summary lists such checkpoints.
- A flow with no checkpoint on the platform being run is skipped, and its steps need no model
  or secrets.
- explore:session --flows without --platform required every platform's secrets. It now checks
  them once the session reports its platform, still before any device work.
- A test runs a described step through runBoundedSession, which rejects with AGENT_CANCELLED
  when step_done ends the step, as the real sessions do.
2026-10-24 11:58:05 -07:00
Brent Vatne 66393ddbc9 Merge the flows fixes from #74: strategy.ts filters secrets and model access by platform, counts turns, and hands the explorer the checkpoints a flows run never attempted 2026-10-23 23:01:36 -07:00
Brent Vatne e66fc6c690 Flows: record every planned flow and checkpoint, and fix what a relaunch left stale
From a review of #74 and an adversarial review of its fixes:

- A checkpoint reused the picture taken on arriving at a screen even after a relaunch, or after
  the model looked again. The arrival picture now lasts only until the next look at the app.
- Flows and checkpoints went missing from exploration.flows: a flow whose relaunch failed, a
  checkpoint whose observation threw (its flow still said completed), and on a budget stop,
  everything not reached. start_flow now creates the record before relaunching; a checkpoint
  that throws is recorded as missing and fails its flow; and when the run ends, the explorer
  asks the strategy which checkpoints it never attempted and records them as missing. A run
  blocked at launch records every flow as failed.
- explore:session's first flow started wherever the app was. The first open relaunches when
  the run follows flows, before the check that the app stays open.
- A flow's journey stopped at the first screen change that was not a transition. The walk now
  goes on as another journey of the flow (journeyIds).
- A launch-URL error, at the start or during a relaunch, was a plain error that ended the run
  without its secrets removed. Both are wrapped through failureText. After two relaunches fail
  in a row, the flows stop.
- A described step or a secret limited to another platform no longer needs a model or a value.
- assertFlowsFit counts the tool calls the flows take, as assertReplayFits does.
2026-10-23 22:58:54 -07:00
Brent Vatne 1b0a0a1d96 Explorations: one place chooses and starts the strategy that drives the explorer
binary/strategy.ts holds what explore:binary and explore:session each branched on: chooseStrategy
checks the options before any device starts (a model, a replay plan or a flows file),
strategyInputs reads the {{secret:NAME}} inputs they need, and startStrategy returns the session
that drives the explorer's tools, with how the report names it. A new strategy is one more case
there. assertAppUrl moves to binary/app-url.ts so strategy.ts does not import run.ts.
2026-10-23 22:03:29 -07:00
Brent Vatne 064d8be4e4 Flows: follow named flows through a build, with named checkpoints that two runs pair up by
explore:binary and explore:session take --flows FILE: named flows, each a list of exact actions
(the --replay format), steps described in words ({"do": …}) and named checkpoints. Each flow after
the first starts from a relaunched app. A model carries out each described step on its own, with
only observe and act, so the explorer still records and blocks what it always does.

The explorer offers start_flow, checkpoint and end_flow to a flows run. A checkpoint is always
recorded, reusing the picture taken on arrival when nothing has happened since. A flow becomes a
journey of the map, named after the flow, and the exploration report lists each flow with its
status and its checkpoints in order (exploration.flows, optional). A failed step ends its flow
and names the checkpoints it did not reach; the next flow still runs.
2026-10-23 21:42:44 -07:00