Files
João MouraandClaude Opus 5.5 b3c4409bc5 feat(cli): record why an evaluation stopped before its verdict (#7963)
* feat(cli): record why an evaluation stopped before its verdict

`cli_usage:evaluate` counts presses of Evaluate and `cli_usage:eval` counts
evaluations that started, but about four in five presses never become an
evaluation and nothing said why. Every stop now counts once as
`cli_usage:eval_stopped:<reason>` from a closed list: untraced, declined,
no_project, login_unreadable, network_error, amp_401/403/404/4xx/5xx,
invalid_response, unexpected. The reason rides on EvaluationStoppedError from
where it is raised; messages, exit codes and screens are unchanged.

A run that `crewai eval` hands to the run app and that turns out untraced is
counted by the command only, so one stop is never counted twice.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* fix(cli): count each eval stop once, with the reason shown

- the run app answers for a delegated run it could not start, so the waiting crewai eval does not start it again and count the stop twice
- an unreadable saved login on the unattended path counts as login_unreadable, not untraced
- an evaluation AMP itself failed counts as evaluation_failed, in the CLI and the app; --models unchanged

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

---------

Co-authored-by: Claude Opus 5.5 (1M context) <[email protected]>
2026-10-09 04:50:12 -03:00
..