mirror of
https://github.com/crewAIInc/crewAI.git
synced 2026-10-12 02:12:31 +00:00
main
145
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Lorenze Jay
|
d522c31339 | feat: bump versions to 1.15.27 (#8001) | ||
|
João MouraandClaude Opus 5.5
|
b3c4409bc5 |
feat(cli): record why an evaluation stopped before its verdict (#7963)
* feat(cli): record why an evaluation stopped before its verdict `cli_usage:evaluate` counts presses of Evaluate and `cli_usage:eval` counts evaluations that started, but about four in five presses never become an evaluation and nothing said why. Every stop now counts once as `cli_usage:eval_stopped:<reason>` from a closed list: untraced, declined, no_project, login_unreadable, network_error, amp_401/403/404/4xx/5xx, invalid_response, unexpected. The reason rides on EvaluationStoppedError from where it is raised; messages, exit codes and screens are unchanged. A run that `crewai eval` hands to the run app and that turns out untraced is counted by the command only, so one stop is never counted twice. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * fix(cli): count each eval stop once, with the reason shown - the run app answers for a delegated run it could not start, so the waiting crewai eval does not start it again and count the stop twice - an unreadable saved login on the unattended path counts as login_unreadable, not untraced - an evaluation AMP itself failed counts as evaluation_failed, in the CLI and the app; --models unchanged Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> --------- Co-authored-by: Claude Opus 5.5 (1M context) <[email protected]> |
||
|
João MouraandClaude Opus 5.5
|
ed71a6f4d6 |
crewai eval --models: per-run cost and time, and no repeated progress lines (#7965)
* crewai eval --models: per-run cost and time, and no repeated progress lines The comparison table printed each setup's cost and time summed over all its runs, while the brief and the report say them per run. It now prints cost_per_run / seconds_per_run under 'cost / run' and 'time / run' (the totals, headed 'cost' / 'time', only when the service sends no per-run figure; a column never mixes the two). Progress deduped only against the previous line, so parallel setups made polls alternate and the same lines printed again and again. Each line is now said once per wait. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * Keep the per-run unit when a model is unpriced, and explain its dash Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> --------- Co-authored-by: Claude Opus 5.5 (1M context) <[email protected]> |
||
|
João MouraandClaude Opus 5.5
|
24a1ddf23f |
feat(cli): record what the run app's Deploy met before an attempt (#7964)
About half the presses of Deploy in the run app never reach a deployment attempt, and nothing said why. With no saved login the press raises before the attempt is counted and prompts for one. That prompt now counts once as `cli_usage:deploy_login_prompted`, and a press that still never reaches an attempt counts as `cli_usage:deploy_stopped:<reason>`: login_still_required, login_abandoned or login_failed. Counting re-raises; what the user sees is unchanged. Co-authored-by: Claude Opus 5.5 (1M context) <[email protected]> |
||
|
João Moura
|
1eff1d6bc9 | feat: bump versions to 1.15.26 (#7961) | ||
|
Sathyajith MCandVidit Ostwal
|
b90fa6e822 |
fix(cli): show actual task output in log-tasks-outputs (#7936)
* fix(cli): show actual task output in log-tasks-outputs * fix(cli): show task result instead of storage blob in log-tasks-outputs --------- Co-authored-by: Vidit Ostwal <[email protected]> |
||
|
Lorenze Jay
|
7ed27d764a | feat: bump versions to 1.15.25 (#7945) | ||
|
Lorenze Jayandgithub-actions[bot]
|
82d09de711 |
feat: bump versions to 1.15.24 (#7941)
* feat: bump versions to 1.15.24 * chore: update tool specifications --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> |
||
|
João MouraandClaude Opus 5.5
|
00e2ff98e9 |
feat(cli): crewai eval prints a markdown brief when run by an agent (#7907)
* feat(cli): crewai eval prints a markdown brief when run by an agent When stdout is not a terminal (or CREWAI_DMN is on), `crewai eval` and `crewai eval --models` print the poll response's `brief_markdown` after the verdict and the report link — written straight to stdout, never through Rich markup — or "Brief (markdown): <brief_url>" when only the link came. An older AMP sends neither and nothing extra is printed. In a terminal nothing changes. Exit codes unchanged. `crewai eval --help` and the `--models` help say so, and the AGENTS.md the crew, flow and declarative-flow templates scaffold gains an "Evaluating" section telling a coding agent to run `crewai eval` after a change and act on the brief. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * docs(cli): an agent re-runs before it re-evaluates; each command's exit rule; the brief may be a link Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * fix(cli): keep the eval guidance out of tool projects; the brief is conditional in --help `crewai create tool` copies the shared AGENTS.md too, but a tool project has no `crewai run` and no traced run for `crewai eval`. The evaluation blocks are marked in the template and dropped for a tool (`evaluation=False`); crew, flow and JSON-crew projects keep them, markers stripped. --help now says the brief prints when AMP provides it, else its link, else nothing more. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> --------- Co-authored-by: Claude Opus 5.5 (1M context) <[email protected]> |
||
|
Sathyajith MCandVidit Ostwal
|
e836a191cf |
fix(cli): correct train filename error message (#7903)
* fix(cli): correct train filename error message * fix(cli): fail train on invalid input with nonzero exit --------- Co-authored-by: Vidit Ostwal <[email protected]> |
||
|
João MouraandClaude Opus 5.5
|
1b9bbccbed |
feat(cli): crewai eval --models, and llm_overlay swaps models as well as roles (#7811)
* feat(llm): llm_overlay maps models too, not only agent roles A key `model:<provider/model>` maps every LLM built from that model string inside the block, and `model:*` every LLM built from any model string. That reaches what a role cannot: a flow step's own `LLM(...).call()`. The key is read in `LLM.__new__` (and so `create_llm` and an agent's `llm="..."`), and on an agent's declared llm built outside the block; a role key still wins for that agent. The caller's settings follow the mapped model by the rule a declared llm's do. `MODEL_KEY_PREFIX` is public so another package can feature-detect the form. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * feat(telemetry): allow-list cli_usage:eval_models attributes The span `crewai eval --models` records carries whether the caller was logged in, the models compared (provider/model names) and how many, in both telemetry classes. Nothing about the run, its output or the organization. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * feat(cli): crewai eval --models compares models on the deployment `crewai eval --models "a,b"` (and `--deployment UUID` when AMP cannot tell the deployment from the project id) asks AMP for a models evaluation: the project id, the models from one comma-separated list (stripped, repeats dropped, provider/model required, 1-5) and the project's eval.jsonc. It needs the login, prints and opens the report link, says where the comparison is while it runs, then prints one row per model (goal, tasks, agents, tools, cost, time; the deployed one marked, the best per column starred) and the top three suggestions. Exit 0 when it finished, 1 when it failed or could not start. Everything from the wire is printed as Text. The comparison is counted as `cli_usage:eval_models` once AMP accepts it. Mode 1 now sends the project id with `create_evaluation` too, so every evaluation is filed under the project it is about. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * feat(cli): crewai deploy create and push send the project id Every create (git or ZIP) and every push (redeploy by name or uuid, or a ZIP update) carries `[tool.crewai].project_id` when the project has one, so AMP knows which project a deployment runs and `crewai eval --models` can find it. The id is read, never minted: a deploy does not rewrite pyproject.toml. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * fix(llm): a model key maps an LLM whose declared SDK is not installed Nothing is built on the declared model, so a missing SDK for it is no reason to fail the mapped build; its credentials cannot be matched to the new provider, so none are carried. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * feat(cli): crewai eval --models leaves its criteria behind as eval.jsonc When the comparison's answer carries `eval_config` and the project has no eval.jsonc, it is written through `write_eval_config` and the same one line Mode 1 prints says so. An existing file is never overwritten. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * fix(llm): a model key compares providers by provider, not by class `_same_route_provider` now follows `_same_provider`'s rule case for case: a LiteLLM-routed `LLM(...)` is on the same provider as a native route when its provider names that native class, so its key and endpoint follow a model key to another model of the same provider. Another provider still gets none. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * chore(llm): say why mark_mapped may skip an object, and one import form in its tests Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * fix(cli): usage stats name a compared model only when it is a public one `cli_usage:eval_models` carried the `--models` strings as typed, and those can identify a customer: a fine-tune id, an Azure deployment name, a self-hosted model or host. A model is now sent by name only when it is an exact entry of crewAI's model catalog (`LLM_CONTEXT_WINDOW_SIZES`), else as `<provider>/other`, and a provider crewAI does not route as `other/other`. An `ft:` id is never sent. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * test(cli): a refused comparison's test no longer keeps a None return Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * test(llms): re-importing OpenAI's completion module restores its package attribute `test_openai_completion_module_is_imported` deletes the module from sys.modules and lets `LLM(...)` import it again, which binds the new module on `crewai.llms.providers.openai` as well. monkeypatch restored only the sys.modules entry, so every later test on that worker saw two module objects for one name: `patch("crewai.llms.providers.openai.completion.X")` patched the attribute's module while `from ... import X` read sys.modules'. On Python 3.10 that made tests/llms/azure/test_azure_responses.py fail whenever it shared a worker with test_openai.py and test_crew_loader.py, which the new tests' even split now did. The attribute is restored too. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * fix(llm): model keys strip only a model's own prefix; a role builds from the declaration Two review findings on the model-for-model overlay: - `overlay_model_for_model` took the text after the first slash as another name for the model, so an aggregator's route (`openrouter/openai/gpt-4o`) matched a `model:openai/gpt-4o` or `model:gpt-4o` key meant for the native model. A prefix is now stripped only when it is a native provider's own and what remains has no slash; an instance on an aggregator is compared as `<provider>/<its model id>`. - With a role key and a model key both active, the model key mapped the declared llm first and the role's model was built like that mapped instance, losing the caller's key and endpoint. The overlay now remembers what the caller declared for every llm it built, and a role key is built from that declaration. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * fix(llm): model keys read a provider's other names; a LiteLLM choice survives a missing SDK - The router's provider aliases are now one table, `crewai.llm.PROVIDER_ALIASES`, and `llm_overlay` compares model names through it: `google/x` and `gemini/x` are one model whether the llm is built in the block or was built before it (a prebuilt instance records `gemini`). An exact key still wins, and an aggregator's route is still compared whole. - When the declared model's SDK is missing, a caller's explicit `is_litellm=True` is kept for the mapped model instead of dropping to a native route. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * fix(deploy): a git create sends the project id once, at the top level AMP reads project_id at the top level on every deploy route, so the git create no longer repeats it inside `deploy`. The redeploy docstring records why its optional body is safe on an AMP that predates project ids. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> --------- Co-authored-by: Claude Opus 5.5 (1M context) <[email protected]> |
||
|
João MouraandClaude Opus 5.5
|
19d2ebcc8c |
fix(cli): crewai eval exits 1 unless the gate passed, and says to log in when nothing was traced unattended (#7806)
* fix(cli): `crewai eval` exits 1 unless the gate passed, and says to log in when nothing was traced unattended The exit code is what a CI job reads, and it said only whether the evaluation finished: a run whose goal gate FAILED exited 0, so a pipeline gating on `crewai eval` waved it through. It is the gate's now — 0 only for PASSED; a failed gate, one without a verdict, and an evaluation that stopped are 1. The command's help says so. With no terminal and no login, an anonymous run's trace stays on the machine even with tracing on (nobody is there to approve the upload). `crewai eval` then told the user to turn tracing on, which they had — the same loop again. When tracing is on and nobody is logged in, it now says what traces an unattended run: `crewai login`. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * fix(cli): an unreadable login is the reason given when nothing was traced unattended Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * fix(cli): print the unattended reason as text, never markup Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> --------- Co-authored-by: Claude Opus 5.5 (1M context) <[email protected]> |
||
|
Lorenze Jay
|
c018da6546 | feat: bump versions to 1.15.23 (#7804) | ||
|
João MouraandClaude Opus 5.5
|
4dcd19aa2e |
fix(tracing): turning tracing on is the answer, and an Evaluate button in the TUI (#7765)
* fix(tracing): turning tracing on is the answer; stop asking again
Two things a user who ran `crewai eval` sees today.
`crewai eval` offers to turn tracing on and run the crew. The run finishes —
and the ephemeral buffer asks "Share this execution trace with CrewAI?", which
is the same question again. Turning tracing on IS consent: `tracing_asked_for()`
is true when `CREWAI_TRACING_ENABLED` says so or when `tracing=True` was passed
in this context, and the buffer takes it as the answer. First-time
auto-collection still asks, because nobody asked for that one.
And the offer named the mechanism rather than the effect — "Turn tracing on
(CREWAI_TRACING_ENABLED=true stays in .env) and run the crew now?", then
"Tracing is on for this project (CREWAI_TRACING_ENABLED=true in .env)." The
variable is how it works, not what the user is agreeing to; both lines now say
what happens and leave the plumbing to the .env file it is written in.
491 telemetry tests and 62 eval-CLI tests green.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
* fix(tracing): the TUI's modal is a prompt too, and say where traces go
Two follow-ups from running it.
The terminal UI installs its own consent callback, so turning tracing on still
bought a modal at the end of the run — the callback is consulted before the
default prompt, and the check only guarded the prompt. A host's callback stays
authoritative, because an embedder may answer no for reasons of its own; what
changed is that the TUI's callback, which exists to ASK, now answers for itself
when tracing was asked for.
And having taken the variable out of the offer, the offer should say what saying
yes means: "Turn tracing on and run the crew now? The run's trace is sent to
CrewAI AMP", and afterwards "Tracing is on for this project — its runs are
traced to CrewAI AMP."
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
* feat(tui): an Evaluate button beside View Traces and Deploy
A run finishes in the terminal UI and the next question is usually "was that
any good?" — which `crewai eval` answers, from the command line, about the run
that just happened. The button is there now, with `e` bound to it.
It behaves like Deploy rather than like View Traces: it leaves the UI and then
runs the command, because `crewai eval` talks — it prints the link to follow,
waits for the verdict and prints that — and none of it belongs inside a
full-screen app. It calls `eval_crew()` itself rather than reimplementing any
of it, so the button and the command can never say different things; a clean
SystemExit is the command deciding, and a failure is reported without failing
the run that already succeeded.
Declarative flows get it too — same button, same tail.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
* feat(telemetry): count `crewai eval`, not only the button that runs it
The TUI's buttons each record a `cli_usage:` feature span, so the new Evaluate
button was counted — but the command it runs was not, and most people will type
`crewai eval` rather than press a button. Every evaluation now records
`cli_usage:eval`; the button keeps its own `cli_usage:evaluate`, so the
difference between the two is how many were started from the TUI.
It counts the command and nothing about the run: no id, no project, no verdict.
And it is wrapped like every other telemetry call here — a missing tracer or a
refused span never stops an evaluation.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
* feat(telemetry): say which run was evaluated, and by whom
`project_id` already rides every span. What only this command knows is which
run is being graded, whether the caller was logged in, and — when they are —
which organization they are logged in to. A feature span can carry attributes
now, and `cli_usage:eval` carries those three.
What it does not carry is anything about the work: no inputs, no output, no
verdict, no criteria. The execution id is the join key; what was found lives in
AMP, where the evaluation itself is.
An anonymous caller reports `authenticated=false` and an empty organization, and
is not asked for one.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
* Address the review: a terminal, a thread, and the right run
Three findings worth acting on, one already fixed, and ruff.
**The security review is right.** `prompt_user_for_trace_viewing` fails closed
where no one can answer, so an anonymous run in CI has never uploaded. Treating
the switch as consent everywhere would have changed that: a project `.env`
copied into CI carries the variable, not the person. `tracing_asked_for()` now
answers only where a prompt could have been shown — the double-ask this PR is
for happens in a terminal, and nothing else moves.
**The ContextVar never crosses the worker.** `set_tracing_enabled` runs in the
crew's validator, on the thread the crew was BUILT on, and a flow never sets it
at all — so `tracing=True` without the env var still opened the modal. The TUI
reads the declaration off the object as well: the same yes, wherever written.
**The button now names the run it watched.** `_chain_eval()` called `crewai eval`
with no id, which looks up the last recorded run — a PREVIOUS run when this one
was not traced, and an offer to run the crew again when there is no record at
all. The TUI remembers what was recorded before the run, hands the new id to the
command, and says "this run was not traced" rather than grading someone else's.
**Already fixed**: telemetry is recorded after `get_or_create_project_id()`, not
before — that moved when the span gained its attributes.
492 telemetry tests, 187 CLI tests. Ruff format and check clean.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
* Close all three doors the prompt closes, not just the terminal
The security review named three places `prompt_user_for_trace_viewing` fails
closed — no terminal, a suite under test, and a host that suppressed tracing
messages — and the first pass only honoured the terminal. A run under test, or
inside a host that asked for silence, could still have uploaded on the strength
of a variable.
The three conditions are one function now, `_prompt_can_be_shown()`, used by the
prompt itself and by `tracing_asked_for`, so the rule and the prompt it answers
cannot drift apart. Where the question could not have been put to anybody, there
is nothing to answer and nothing leaves the machine.
The terminal UI decides for itself, and says why: it silences crewAI's console
to keep it out of its own layout, which is a reason a PROMPT cannot be shown,
not a reason to ask a second time — it has a screen and somebody in front of it.
494 telemetry tests, 88 TUI tests. Ruff clean.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
* Tests: a real session for the consent rule, and a laid-out screen for a click
The rule that a declared CREWAI_TRACING_ENABLED needs no modal only holds
where a prompt could have been shown, and a test suite is one of the places
it could not — so the three cases about a real session say a person is here.
The consent click waited for the screen, which is not the same as waiting for
its buttons to have a region; it now waits for the layout it clicks on.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
* A run started for an evaluation closes itself, even in a child process
A project's crew and flow are run through `uv run …`, so the app lives in
another process and the ContextVar saying "an evaluation is waiting" never
reaches it: the screen stayed open and `crewai eval` waited behind it, which
is the trap the handshake was meant to remove. The environment is the one
thing that crosses into that process, so the command sets a variable there and
clears it after; the id travels back the way it already did, through the
project's last-run record. A run that FAILED leaves too — the command behind
the screen has its own way of saying there is nothing to grade, and it cannot
say it while somebody has to quit a screen first.
And the spans a run exports reach AMP a moment after the run ends, sometimes
later than that, so a command that just watched the run would ask for a trace
still in flight and be told, correctly and uselessly, that there is nothing
there. A run we know is fresh now waits up to two minutes and says it is
waiting; an id somebody typed fails at once, as it always has.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
* The evaluation happens in the app, on the screen the reader is already on
Closing the app the moment the run ended and leaving somebody in a bare
terminal, waiting some seconds for a link to appear and a browser to open, is
worse than the screen it replaced. So the evaluation runs where the run does:
`Evaluating this run` under the run's own line, the report link beside it, the
browser opened from there, and `Goal gate PASSED · goal 5/5 · …` when it comes
back. Nothing closes; the reader leaves when they are done reading, and the
link is printed once more on the way out so it outlives the screen.
The Evaluate button does the same thing now instead of quitting the app and
chaining into the command, so there is one implementation of one experience —
`_chain_eval` and `_want_eval` are gone. Pressing it again opens the report it
already made rather than paying for a second evaluation.
Underneath, `crewai eval`'s two steps became callable as values: they raise
`EvaluationStoppedError` with the sentence to show instead of printing and
exiting, and `evaluate_run()` is the app's one door — the url as soon as AMP
gives it, every unfinished answer while it waits. The command keeps the
terminal wording it had; when it ran the crew itself, it now has nothing left
to say and steps aside for the app it opened.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
* The button says which report it opens
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
* The button works while it waits: it spins, and it stays pressable
A dimmed label that never moves reads as a screen that has stopped. The
Evaluate button now carries the same spinner the run's own line does, the
clock goes back up to eight frames a second while the evaluation runs, and the
button is no longer disabled — the report page is where the progress is, so
pressing it while it waits is the useful thing to do.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
* View Traces opens the traces
The button never had a link. crewAI grants one for every exported run and
prints it after a run — but never under a TUI, where its console is kept out
of the layout — so the app had nothing to open and said so, which read as a
feature that had stopped working.
The link is now recorded beside the rest of what crewAI writes when a run's
spans reach AMP, and the button reads it off that record, matched to THIS
app's execution rather than to whatever ran last in the project. When there is
no link the notice says which of the two reasons it is: a run nobody traced,
or a traced run AMP granted no viewer for — only one of those is the reader's
to fix.
Verified against a real anonymous run: pressing `t` opens
app.crewai.com/crewai_plus/ephemeral_traces/<id> and shows it.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
* The review's five: a trust boundary, a fallback, a retry, an exit and a count
**A project's `.env` cannot lend itself the login.** `evaluate_run()` built its
AMP client after the project's `.env` had been loaded with override, so a
`CREWAI_PLUS_URL` a project put there landed in the set of origins trusted with
the saved `crewai login` bearer. The command reads that variable BEFORE the
`.env` — a shell export is this machine speaking — but inside the app there is
no such "before", so the variable is left out of the set entirely. The request
still follows it; the credential does not.
**A run no app evaluated is still evaluated.** The command returned as soon as
a trace existed, assuming the app had graded it — but a conversational session
never ends the way a crew does and a flow with human feedback takes the
terminal instead, so `crewai eval` could exit 0 having graded nothing. The app
now leaves a marker naming the run it started evaluating, and the command
grades anything that marker does not name.
**An evaluation that stopped can be tried again.** The button re-enabled itself
after a failure but the action still treated any existing evaluation as
finished, so a 502 could only be escaped by re-running the crew.
**A login that exits is shown on screen.** `saved_login()` fails by exiting,
which is not an `Exception` and so slipped past the worker's hands: the button
could sit on "Evaluating…" forever with the reason gone with the exit.
**An evaluation the app runs is counted.** `cli_usage:eval` is the count of
evaluations that begin, and the ones that begin in the app were missing from
it.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
* A stop is a sentence, and the app's own word settles what it graded
**"1" is not a reason.** A credential that exists but cannot be read stopped
the evaluation by printing its sentence and exiting — under the app the print
lands beneath the layout and the exit carries only a code, so the screen said
`1`. `saved_login()` now raises the stop like everything else that stops an
evaluation, the command catches it where it builds the client and prints it as
before, and the app shows it. The test that claimed to cover this raised a
`SystemExit` carrying a sentence, which is not what `_fail` raises; it now
breaks the credential store and reads what reaches the screen. The worker
still catches an exit, and says that the reason did not survive rather than
echoing its code.
**The record is the project's last finish, not this run.** Another run
finishing in the same project replaces it while the app is open, so "is the
marker about the id in the record" could send the command off to evaluate a
different execution. The marker is now the app's own word about the run it
watched — the id it evaluated, or null when it looked and found nothing to
grade — and the command trusts it over the record, matching on time rather
than on an id that may already have been overwritten. No marker still means no
app got here, which is the conversational session and the flow that takes the
terminal, and the command grades those itself.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
* A project can say what good means, and `crewai eval` carries it
An evaluation grades a run against the crew's own `expected_output`s — which
is the prompt that produced the output, so it catches a crew that ignored its
own contract and nothing else. What discriminates lives in criteria somebody
writes, and there was nowhere to write them.
Now there is. After the first evaluation the criteria it used are written to
`eval.jsonc` in the project — once, never over a file that exists, because the
one thing worse than no criteria is criteria that vanish whenever somebody
evaluates — and the line that says so points at what to do with it. From then
on the file travels with every evaluation, as it was written, comments and
all: what a criterion means is the grader's to read, not this command's.
Both doors do it, since most evaluations now happen inside the run app: the
app writes the file and names it in the line that outlives its screen, and
`crewai eval` in a terminal says it there.
The file is bounded at the same 64KB AMP holds the field to, so one that would
be refused is never uploaded and the run is graded anyway.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
* The review's asks: anonymous eval stats, a handshake a project cannot forge
Usage stats are anonymous, so the `crewai eval` count says whether the caller
was logged in and nothing else — no execution id, no organization (the product
decision on this PR). Which runs were evaluated, and by whom, is AMP's record.
A feature span now keeps only the keys its own feature lists, in both
telemetry classes; everything else — the dimensions every span carries
included — is dropped.
The run app's "an evaluation is waiting" handshake was an environment variable
any project's `.env` could set, which would start an evaluation with the
machine's login on a plain `crewai run`. Its value is now a random token that
counts only while the command's file of that name exists in the user's crewAI
data directory; a `.env` cannot create a file.
And three smaller ones: when no app evaluated the run, the command grades the
project's record only if it was written after this run began; the "Wrote
eval.jsonc" line after the app closes now appears (the key it read was never
set); an oversized eval.jsonc is said once, through the caller's note, rather
than printed under the TUI on every retry.
uv.lock is back to main's: the change was re-resolution drift, nothing this PR
needs.
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
* A report link is a web address, and an anonymous request names no organization
Two from the security review of
|
||
|
Vidit Ostwal
|
0e6440b397 |
docs(cli): guide assistants to platform tools (#7581)
* feat(cli): prioritize platform integrations * docs(cli): guide assistants to platform tools * test(platform): update missing token assertion * fix(platform): retain integration token environment key * chore: ignore Codex workspace files * feat(platform): expand integration catalog * feat(platform): show integration action counts * feat(platform): select agent actions separately * feat(platform): group integration applications * fix(cli): keep platform picker navigation in place * fix(cli): validate platform apps once * fix(platform): use one integration token variable * test(platform): update integration token assertion * fix(cli): clarify platform token validation * fix(cli): show platform token prompts * refactor(platform): expose application catalog * refactor(platform): export catalog from Python * docs(platform): document catalog regeneration * fix(platform): resolve catalog type export |
||
|
Vidit Ostwal
|
a2ea278210 |
feat(cli): prioritize popular platform integrations (#7573)
* feat(cli): prioritize platform integrations * test(platform): update missing token assertion * chore: ignore Codex workspace files * feat(platform): expand integration catalog * feat(platform): show integration action counts * feat(platform): select agent actions separately * feat(platform): group integration applications * fix(cli): keep platform picker navigation in place * fix(cli): validate platform apps once * fix(platform): use one integration token variable * test(platform): update integration token assertion * fix(cli): clarify platform token validation * fix(cli): show platform token prompts * refactor(platform): expose application catalog * refactor(platform): export catalog from Python * docs(platform): document catalog regeneration * fix(platform): resolve catalog type export |
||
|
João MouraandClaude Opus 5
|
bdd1bc6200 |
chore(cli): move crewai eval under experimental (#7707)
Nothing about the command changes. It is still `crewai eval`, still listed in `--help`, still takes `--run`, still prints the same lines. Only the module moved, from `crewai_cli/eval_crew.py` to `crewai_cli/experimental/eval_crew.py`, with its tests alongside. The package says what the command's location could not: its output and its options may change between releases without the deprecation cycle a settled command gets. That is honest about where it stands — most of what it depends on lives outside this repository, in an AMP endpoint and the evaluator behind it, and both are new enough that the report's shape is still moving. The command is already built to absorb some of that, printing whatever the evaluation graded rather than a list of its own, but the promise it makes to a user should match. `crewai/experimental/` has the same role on the library side, so this is the convention the repository already has rather than a new one. The CLI had no experimental package before; it does now, and `crewai eval` is its first tenant. Co-authored-by: Claude Opus 5 (1M context) <[email protected]> |
||
|
João Moura
|
9de80df7f3 |
fix(cli): print whatever areas the evaluation graded (#7701)
`crewai eval` printed a fixed tuple of area names. The areas belong to the evaluation, not to the client, so a fixed list does two wrong things the moment the evaluator's vocabulary moves ahead of an installed CLI: it drops every area it has not heard of, and it invents "not measured" for ones that no longer exist. A user would see one real grade and three phantom blanks, with no sign that anything had been dropped. It now prints the areas it was sent, in the order they arrived. The verdict is still validated exactly as before — a grade is an int in 1..5 or null — so a malformed payload is still a protocol error rather than something printed. This lands before the evaluator's own change so that an installed CLI keeps working through the rollout rather than after it. |
||
|
João MouraandClaude Fable 5.1
|
0a3b8912b3 |
feat(cli): crewai eval evaluates the last traced run through AMP (#7649)
* feat(cli): crewai eval evaluates the last traced run through AMP `crewai eval` reads `.crewai/last_run.json` — the record crewAI writes when a traced run's spans reach Wharf — and asks AMP to evaluate that run: POST /crewai_plus/api/v1/tracing/evaluations with the execution id, sending the saved `crewai login` when there is one and nothing otherwise. AMP answers with an evaluation id and a URL; the command prints the URL, opens it, waits for the verdict and prints it (goal gate and the four grades), exit 1 only when the evaluation itself failed. `--run EXECUTION_ID` evaluates another run. With no traced run recorded it offers to turn tracing on for the project (`CREWAI_TRACING_ENABLED=true` in .env, set_key so nothing else in the file moves) and run the crew now with `crewai run`; without a terminal, or declined, it prints the three steps instead. A run that leaves no record behind is explained, never guessed at. AMP's refusals are printed in its own words: a run that needs an account (401 account_required), a refused credential (then `crewai login`), a run AMP does not hold (404), rate limiting (429 with Retry-After), and any other status with AMP's message. The two AMP calls live on the CLI's PlusAPI subclass, so no crewai-core release is needed. Who may evaluate what is AMP's decision, not the command's: an anonymous run once without an account, then it needs one; a run traced while logged in for that organization's members; a deployment execution for members who may see its traces. Co-Authored-By: Claude Fable 5.1 <[email protected]> * fix(cli): crewai eval — guard the project, survive AMP blips, name the right subject, read the whole record Review findings, each reproduced before the fix: - The run-it-now offer wrote CREWAI_TRACING_ENABLED into ./.env before checking the directory is a crewAI project, then run_crew() died on a missing pyproject.toml with a traceback. Now: no pyproject.toml → one sentence, exit 1, nothing written. - httpx errors (AMP unreachable, a timeout) surfaced as tracebacks. Now the start says "Could not reach AMP to start the evaluation: …"; while waiting, an unreachable AMP or a 5xx is retried up to POLL_RETRIES consecutive times, then reported with the URL — the evaluation keeps running server-side either way. - The POST now carries a 120 s timeout (AMP reads the run's spans inside it), the poll 30 s. - A 200 whose body has no known status (a non-dict, no status, a status outside queued/running/done/failed) polled forever. Now it stops with the status it saw and the URL. - A 404 without a JSON message read "AMP holds no run <evaluation id>" while polling. _refused takes "run <id>" / "evaluation <id>" and says "AMP answered 404 for <subject>" — AMP's own message still wins. - The record's amp_base_url was ignored; the CLI now evaluates the run at the AMP it was traced to. --run keeps the configured AMP. - The post-run explanation names the third cause: a crewai older than the version that records the last run. Tests for each, plus the previously untested paths: Ctrl-C exits 130, a 2xx without an id, a refusal mid-poll, DMN opens no browser, --run skips the offer. 51 passed in lib/cli/tests (eval + plus_api). Co-Authored-By: Claude Fable 5.1 <[email protected]> * fix(cli): crewai eval — the credential goes only to the configured AMP; an explicit yes before running the crew; a done answer needs a well-formed verdict Review findings (CodeRabbit, the PR gate): - The record's amp_base_url was handed to PlusAPI beside the saved login, so a modified .crewai/last_run.json could send the token to any origin. The client is now built from the configured AMP only (CREWAI_PLUS_URL, the saved settings, app.crewai.com); the project's .env is loaded first, as `crewai run` loads it, so the configured AMP is the one the run was traced to. A record naming another address gets a one-line note and no credential. - The offer to turn tracing on and run the crew defaults to no and says the .env change stays; Enter no longer spends a crew run. - A `done` answer whose verdict is missing or malformed (no gate, grades not an object, a grade not an int or null) is a protocol error, exit 1, instead of an INCONCLUSIVE line with exit 0 or an AttributeError. Tests for each: a foreign origin in the record with a saved token, the .env-loaded same-AMP case, seven malformed verdicts, the prompt's text and default. 59 passed (eval + plus_api); ruff and mypy clean. Co-Authored-By: Claude Fable 5.1 <[email protected]> * fix(cli): crewai eval — Enter accepts the offer to run the crew (y/n, Y default; the prompt names both effects) João's call (2026-09-20): the confirm is y/n with Y as the default. The prompt still says tracing stays on in .env and that the crew runs now. Co-Authored-By: Claude Fable 5.1 <[email protected]> --------- Co-authored-by: Claude Fable 5.1 <[email protected]> |
||
|
João MouraandClaude Opus 5
|
6e1fa8cf2a |
feat(tracing): record the last traced run for crewai eval instead of printing it (#7648)
* feat(tracing): record the last traced run for crewai eval instead of printing it After a traced run, crewAI printed a panel with the execution id and a viewer link. The id is internal, and the panel exposed it for no purpose a user has. Nothing is printed now. Instead, once the run's spans have reached Wharf, crewAI writes `.crewai/last_run.json` in the project: the execution id, when the run started and ended, whether it was traced anonymously or under an account, and the AMP base url. `crewai eval` reads it back, so the user evaluates their last run without pasting anything. GrantSpanExporter.record_export replaces show_trace_summary at the two finish points (an authenticated run's shutdown, an anonymous run's share). Only a run whose every export succeeded is recorded — a partial export would name a run the grader could not read whole. Recording is silent in the TUI and under message suppression too, off under the test suite, and inert inside a deployment, where the platform binds the execution before crewAI's own tracing starts. The record is written atomically and a write failure never fails the run. The crew, flow and json_crew scaffolds now ignore `.crewai/` like the declarative flow scaffold already did. Co-Authored-By: Claude Fable 5.1 <[email protected]> * fix(tracing): never record a truncated ephemeral run; resolve the record path inside the guard; pin the window and the once - An ephemeral buffer that dropped spans at its cap still uploaded the rest and recorded the run. The grader could not read that run whole, so it is no longer recorded (the upload still happens). Test added. - last_run_path() ran before the try in record_last_run; a cwd that vanished mid-run would have raised out of a function documented never to raise. It is inside now, with the same debug log and None. - Tests: the "recorded once" check counted an unchanged mtime, which proves nothing on a coarse filesystem — it now counts calls to record_last_run; the first-start/last-end fold across export batches had no value test — batches now arrive out of order and the ISO window is asserted. - Docs: the conversational-flows comment said "one trace link" for output that no longer exists; tracing.mdx now names .crewai/last_run.json, what it holds, who reads it, and when it is not written. en, ar, ko, pt-BR. 85 passed (test_session_trace_export + test_last_run); ruff and mypy clean. Co-Authored-By: Claude Fable 5.1 <[email protected]> * fix(tracing): a temporary file of its own per writer; the docs name deployments - record_last_run wrote every record through one `.tmp` name, so two crews finishing together in one project could replace each other's temporary file and one record was lost. Each write now goes through tempfile.mkstemp in the same directory, then os.replace; a failed write removes its own temporary file. Tests: two writers use two names and leave only last_run.json; a failed replace leaves nothing behind. - tracing.mdx (en, ar, ko, pt-BR): nothing is recorded inside a deployment, where the platform owns the trace. 87 passed (test_last_run + test_session_trace_export); ruff and mypy clean. Co-Authored-By: Claude Fable 5.1 <[email protected]> * fix(tracing): the run that finished last stays recorded, whichever writer comes last Two crews finishing together in one project: the older run's writer could reach the file after the newer one and leave the older run as "the last". record_last_run now compares finished_at (recorded_at when absent) with the record already there and keeps the later-finished run; the same run recorded again (a refreshed grant) and a record without a comparable time never block the run just finished. No lock file: the file is a convenience pointer and the compare closes the ordering, leaving a microsecond window between read and replace that two runs finishing in the same instant could hit — either is then a fair "last run". Test: the older writer replaces after the newer one; the newer stays. 88 passed. Co-Authored-By: Claude Fable 5.1 <[email protected]> * fix(tracing): a same-millisecond tie keeps the record already there; nothing is recorded inside a deployment - The times in the record are stored to the millisecond; two runs finishing in the same millisecond compared equal and the later writer won, which could leave the older run recorded. A tie now keeps what is there. No finer order is worth a field in the file. - recording_enabled() is false when the platform's integration token is present: a deployment normally never reaches this code (the host binds the trace before crewAI would start its own), but a container that did not would have written a file the platform never reads, against what the docs promise. Test added. 89 passed; ruff and mypy clean. Co-Authored-By: Claude Fable 5.1 <[email protected]> * fix(tracing): only a run that provably finished later holds the record My previous commit compared `finished_at` and fell back to `recorded_at`, which broke two existing tests on CI: two records written in the same millisecond with no completion time compared equal, so the second write was dropped and the older run stayed. Locally the two writes happened to land in different milliseconds, so the suite passed — a timing-dependent bug, not a CI quirk. The rule is now narrow: a write stands unless the record already there is a DIFFERENT run with a later `finished_at`. No completion time on either side, the same run recorded again, or a tie to the millisecond all leave the write to stand, so "the last run recorded" stays what a reader gets. The case the rule exists for is unchanged: the older-finished writer arriving last does not replace the newer one. 485 passed in lib/crewai/tests/telemetry (test_last_run run five times for timing); ruff and mypy clean. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> * fix(tracing): serialise the write across processes; drop the platform-token guard, which was not a deployment test - Reading the record, deciding and replacing it are now one step, held under a lock file beside the record, so a writer can no longer decide against a record another process has already replaced. Where the platform has no flock the write goes ahead unserialised: the record is a convenience pointer for `crewai eval`, never worth failing a run over. The regression drives three writers at once and fails without the lock. - The deployment guard I added yesterday read CREWAI_PLATFORM_INTEGRATION_TOKEN, which is not a deployment marker: `crewai create crew` writes that token into a project's own .env for platform tools, and crew_base loads it, so the guard would have stopped recording for every developer who uses them. It is also unnecessary — a deployment kicks off with tracing off, so crewAI never builds the exporter and never reaches this module. Guard removed, with a test that a project carrying that token is still recorded. - The module docstring claimed the platform "binds the execution before crewAI would start its own tracing". It does not; it sets tracing off at kickoff. Docstring and the four tracing docs now say the real reason. 487 passed in lib/crewai/tests/telemetry; ruff and mypy clean. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> * fix(tracing): a refused lock still records the run; the platform-token test exercises the real guard Two Cursor findings on the locking commit. - If flock itself failed — some network mounts refuse it — the OSError escaped _exclusive, the outer handler deleted the temporary file and record_last_run returned None, so the run was not recorded at all. A refused lock is now logged and the write goes ahead unserialised, matching what already happened when the lock file could not be opened or flock does not exist. Unserialised beats not recorded. - test_a_project_using_platform_tools_is_still_recorded used the `project` fixture, which stubs recording_enabled — the very thing under test — so the regression it guards could have come back green. It now stubs only project_dir and asserts the real guard. 13 in test_last_run, 489 across lib/crewai/tests/telemetry. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> --------- Co-authored-by: Claude Fable 5.1 <[email protected]> |
||
|
Vidit Ostwal
|
977aa0c6ec |
feat(cli): improve platform integration setup UX (#7453)
* fix(cli): silence tool import warnings during platform setup * feat(cli): validate platform integrations concurrently * fix(cli): persist platform token during crew setup * fix(core): make settings write probe concurrency-safe * test(cli): simplify event loop fallback stub |
||
|
|
b9629cfb8f |
feat: add native Gemini 3.8 Flash support (#7284)
* feat(llm): add Gemini 3.8 Flash support - Register Gemini 3.8 Flash in CrewAI and CLI model catalogs - Add 1,048,576-token context window mapping - Add native Gemini provider support - Add Gemini 3.8 Flash to the CLI model picker - Add native model detection coverage for gemini/ and google/ formats - Add context window regression coverage Closes #7241 * test: add VCR cassette for gemini-3.8-flash * chore: stop tracking .env.test * Bringing back .env.test --------- Co-authored-by: Vidit Ostwal <[email protected]> Co-authored-by: ViditOstwal <[email protected]> |
||
|
Lorenze Jay
|
b0343eb75f | feat: bump versions to 1.15.22 (#7520) | ||
|
|
5990ead5ec |
feat(cli): record why a deployment create failed (#7451)
* feat(cli): record why a deployment create failed `crewai deploy create` counts every attempt (`Create Crew Deployment`) and every success (`Crew Deployment Created`), but the gap between them carried no cause: among clients able to emit the success span, the CLI succeeds 96.7% of the time and the run TUI 36.4%, and nothing said why. A third span, `Crew Deployment Failed`, now fires for every failure after the attempt is counted, with a closed vocabulary `reason` (api_4xx, api_5xx, invalid_response, network_error, zip_error, user_declined, unexpected), the HTTP `status_code` when the API answered, and the existing `source`. Never the error message. The request path is factored into `_request_crew_creation`; every exception is classified, reported and re-raised unchanged, so CLI and TUI behaviour is the same as before. HTTP failures are classified before `_validate_response`, which still prints and exits as it did. Co-Authored-By: Claude Fable 5.1 <[email protected]> * fix(cli): classify deploy create failures by status and by stage Review fixes on the failure span. Check the HTTP class before the body so a gateway's HTML page counts as api_4xx / api_5xx with its code. Treat a 2xx whose body is not a JSON object carrying uuid and status as invalid_response and exit cleanly, instead of emitting a success span and crashing in the display step. Recognise archive failures by a dedicated ArchiveError (a ValueError) raised from create_project_zip, so the git helpers' own ValueErrors no longer read as zip_error; a failed ZIP write is wrapped and its partial file removed. Co-Authored-By: Claude Fable 5.1 <[email protected]> * fix(cli): treat staging and temp-file failures as archive errors `create_project_zip` only wrapped the ZIP write, so an `OSError` while staging files or creating the temporary archive escaped as a bare `OSError` and the deploy command recorded it as `unexpected` instead of `zip_error`. The archive boundary now covers staging, temp-file creation and the write; the staging directory is removed on every path, and no partial archive is left behind. Co-Authored-By: Claude Fable 5.1 <[email protected]> * feat(cli): tell a non-JSON 2xx apart from a non-creation 2xx A proxy's 200 HTML page and a JSON body missing the creation fields were both recorded as `invalid_response`. They are different failures, one in the network path and one in the API contract, so the deploy failure span now records `invalid_json` for the first and `invalid_creation_response` for the second. Requested in review. Co-Authored-By: Claude Fable 5.1 <[email protected]> --------- Co-authored-by: Claude Fable 5.1 <[email protected]> Co-authored-by: Vidit Ostwal <[email protected]> |
||
|
João MouraandClaude Fable 5.1
|
6b6a39d503 |
fix(events): Improve coding agents instructions (#7447)
* feat: scaffold project with assistant instruction files - Updated project creation to include `CLAUDE.md` and `GEMINI.md` that import `AGENTS.md`, ensuring consistent guidance across coding assistants. - Implemented utility functions to copy assistant instruction files during project setup. - Enhanced documentation in `AGENTS.md` to emphasize the importance of keeping telemetry enabled for optimal performance. - Added tests to verify the correct scaffolding of assistant instruction files and their contents. * fix(cli): neutral observability guidance in scaffolded AGENTS.md - State the observability rule as the user's decision, never a fix for console warnings, speed, or a "clean" configuration - Rewrite the AMP section as built-in capabilities: no "free", "proactively", "sales pitch", or scripted pitches - Turn the research mandate into a list of sources to consult when version details matter - Retarget the scaffold tests to the new wording Co-Authored-By: Claude Fable 5.1 <[email protected]> * fix(cli): scaffold assistant files for JSON crews and harden telemetry tests - create_json_crew, the default `crewai create crew` path, now copies AGENTS.md, CLAUDE.md and GEMINI.md; AGENTS.md documents the JSON layout - span helper no longer depends on OTEL_SDK_DISABLED being popped by an earlier test; thread-scope test stops its worker before leaving the mock - single import style in the shutdown test Co-Authored-By: Claude Fable 5.1 <[email protected]> --------- Co-authored-by: Claude Fable 5.1 <[email protected]> |
||
|
Lorenze Jay
|
993a96c4e0 |
feat(tracing): port trace events sessions to OSS (#7464)
* feat(tracing): port enterprise event sessions to OSS * fix(tracing): address review findings and verify concurrent exports * fix(tracing): keep redactor ownership in enterprise integrations * test(tracing): isolate intentional failures from cleanup assertions |
||
|
Sharoon Sharif
|
756d8d33c8 |
fix(cli): read json checkpoints as utf-8 (#7491)
JsonProvider writes checkpoints with encoding="utf-8" and the runtime serialises non-ASCII text verbatim, but the checkpoint CLI still opened them with the platform default encoding. On Windows (cp1252) `crewai checkpoint info` raised UnicodeDecodeError and `list` showed a 0-byte entry for any checkpoint containing non-ASCII text. Open the files as UTF-8 and add regression tests for the three readers. |
||
|
Sharoon SharifandVidit Ostwal
|
0e2fa7ec24 |
fix(cli): overwrite stale poetry.lock backup on windows (#7463)
* fix(cli): overwrite stale poetry.lock backup on windows os.rename raises FileExistsError on Windows when poetry-old.lock already exists from a previous run, so a second `crewai update` crashed there while POSIX silently replaced the file. Use os.replace, which overwrites on every platform, and add a regression test. * test(cli): add docstrings to update_crew tests --------- Co-authored-by: Vidit Ostwal <[email protected]> |
||
|
theaterandVidit Ostwal
|
a225c1b373 |
fix(cli): don't crash the run TUI when streamed output contains a literal [...] (#7435)
* fix(cli): reject non-serializable literal_eval results in TUI JSON formatting _try_parse_structured() accepted any dict/list coming out of ast.literal_eval(), including values json.dumps() cannot encode such as [Ellipsis] from a literal [...]. _format_json_in_text() then raised TypeError: Object of type ellipsis is not JSON serializable, which propagated through _tick and cancelled the whole crew run. Validate the parsed object with json.dumps() inside _try_parse_structured() so only JSON-serializable dict/list values are returned; anything else falls back to the original text. Fixes #7434. * fix(cli): contain RecursionError in the TUI JSON formatting boundary A streamed structure nested deeper than the JSON backend can walk could raise RecursionError out of _try_parse_structured (from json.loads) or out of the render-path dumps, escaping _tick and losing the TUI update. Catch RecursionError when loading, and validate literal_eval results with the exact kwargs the render path uses, so any structure the render cannot encode is rejected at the boundary and the raw text renders instead. --------- Co-authored-by: Vidit Ostwal <[email protected]> |
||
|
Vidit Ostwal
|
93ad4d67c4 |
feat: validate platform integrations during crew setup (#7385)
* feat: validate platform integrations during crew setup * fix: clarify platform integration validation * refactor: split platform authentication workflow * feat: list connected platform integrations |
||
|
Vidit Ostwal
|
d20845f0a3 |
feat: add platform tools to JSON crew wizard (#7384)
* feat: support platform tools in JSON crews * style: clarify platform integration labels * fix: avoid repeating tool picker title * fix(platform): surface JSON tool discovery errors |
||
|
Lorenze Jay
|
d469e9fb2b | feat: bump versions to 1.15.21 (#7362) | ||
|
Vidit Ostwal
|
4ed4aba6cd |
fix(cli): keep deploy push on the AMP create source (#7345)
Push was choosing ZIP vs git from a local origin remote, so adding origin later rebuilt the last ZIP with no files. Prefer AMP zip_deployment from status, and fall back to the old origin heuristic when that field is missing. |
||
|
Lorenze JayandVidit Ostwal
|
b0fd0a9fa3 |
feat(telemetry): track checkpoint runtime and CLI usage (#7348)
* feat(cli): track checkpoint command and TUI usage * feat(telemetry): track runtime checkpoint operations * fix(telemetry): count prune usage after argument validation * style(cli): format checkpoint prune command --------- Co-authored-by: Vidit Ostwal <[email protected]> |
||
|
João Moura
|
1457740528 | feat: bump versions to 1.15.20 (#7270) | ||
|
João Moura
|
227844ef86 | feat: bump versions to 1.15.19 (#7265) | ||
|
Lorenze JayandVidit Ostwal
|
917b9df6d7 |
Validate JSON crews in project environments (#7171)
* Validate JSON crews in project environments * fix(cli): address standalone deploy review feedback --------- Co-authored-by: Vidit Ostwal <[email protected]> |
||
|
Lorenze Jay
|
5fd7c47669 | feat: bump versions to 1.15.18 (#7136) | ||
|
João MouraandClaude Opus 5
|
fcdeb3d98d |
feat(events): record a created deployment with the uuid it was given (#7115)
`Create Crew Deployment` fires before the API call that creates the deployment, so it counts creation ATTEMPTS and cannot carry the uuid - the call that creates the deployment is the call that returns it. Live effect: `create_deployment` reads 76,015 events with 0 carrying a uuid (0.000%), so deployments cannot be joined to anything. Moving the existing span after the response would fix the uuid and silently redefine the metric, turning attempts into successes; the deployment churn figures are built on attempts. So this adds a second span rather than moving the first. `Crew Deployment Created` fires after `_validate_response`, which raises SystemExit on failure - a failed create therefore still counts as an attempt and reports no creation. Both creation paths, git remote and zip upload, converge on that line and both return the uuid. Emits no `deploy:created` feature count: the attempt span already does, and a second emit would double the deployment count that origin-independent aggregation depends on. Tests cover the emitter (uuid carried, distinct span name, no second feature count, absent-vs-empty uuid) and the call site across both creation paths plus the failure path. The warehouse consumer must be widened BEFORE this merges or it delivers nothing: `mv_span_fanout_forward` filters on a hard-coded 13-name allowlist, and `mv_fanout_deployment_spans`'s `multiIf` ends in a catch-all `remove_crew` arm that would mislabel the new span. Runbook prepared separately. Co-authored-by: Claude Opus 5 (1M context) <[email protected]> |
||
|
Vidit OstwalandLorenze Jay
|
039f6ff5f6 |
fix(llms): raise Anthropic default max_tokens so large tool calls survive (#7077)
* fix(llms): raise Anthropic default max_tokens so large tool calls survive * fix(llms): default Anthropic to sonnet-4-6 and drop retired models --------- Co-authored-by: Lorenze Jay <[email protected]> |
||
|
Vidit Ostwal
|
d0e9208627 |
[OSS-129] Map GPT-5.6 family to the official 1.05M context window (#7012)
* fix(core): map GPT-5.6 to the official 1.05M context window LiteLLM fallback treated Sol, Terra, Luna, and the gpt-5.6 alias as unknown and used the 8k default. * fix(openai): give GPT-5.6 its own 1.05M window Native lookup matched gpt-5 first, so Sol, Terra, and Luna inherited 1,047,576. Longest-prefix matching keeps gpt-5 and gpt-5.4-mini on their own sizes. * fix(azure): map GPT-5.6 deployments to the official 1.05M window Azure had no gpt-5 / gpt-5.6 entry, so Sol, Terra, and Luna fell back to 8k. * feat(cli): list GPT-5.6 Sol, Terra, and Luna in curated catalogs The family is generally available; keep gpt-5.5 as the offline default. * refactor: keep context-window tables in longest-prefix order Drop the runtime sort and document that new keys must be inserted longest-first so startswith matching stays correct. * fix(core): resolve prefixed LiteLLM models to the GPT-5.6 window openai/gpt-5.6-luna kept its provider prefix on self.model, so startswith matching missed the 1.05M mapping. Strip recognized prefixes for lookup and leave unknown ones intact. |
||
|
|
9e9a8577be |
feat(events): report project creation with the id minted for it (#7074)
* feat(telemetry): report project creation with the id minted for it Acquisition was only observable from a project's first run. That misses every project created and never run, and dates the rest to the wrong day. All three scaffolding paths already mint a project_id into the new pyproject.toml. None of them reported it, and two of them - create_crew and create_json_crew, which is the default `crewai create crew` path - emitted no telemetry at all. `Project Created` carries the kind (crew, json_crew, flow) and the id that was just minted, and is emitted after the mint so it can carry it. The attribute is `created_project_id`, not `project_id`, because those are two different things. CommonAttributesSpanProcessor stamps `project_id` on every span from get_project_id(), which reads the current working directory and is cached for the life of the process - during `crewai create` that describes the directory the command was run from, not the project being created. Reusing the name would have given one column two meanings depending on span type. Nothing is emitted for `create_crew(parent_folder=...)`: that adds a crew to a project which already exists, mints no id, and is not an acquisition. The existing `Flow Creation` span is left exactly as it is. Note for whoever reads it: it is emitted from two places with two different meanings - CLI scaffolding (create_flow.py) and runtime flow construction (event_listener.py on FlowCreatedEvent) - so it cannot separate acquisition from usage. Not changed here because it is a live series and renaming it would break continuity. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01RfV2uMqWRcdfufMvtdCVoN * test(telemetry): stop the creation-span tests exporting to the real collector The recorded_spans fixture has to enable telemetry for its assertions to mean anything - a disabled Telemetry never builds self.provider, so every assertion would pass vacuously against a non-recording span. But enabling it is exactly what makes __init__ wire a BatchSpanProcessor around the real SafeOTLPSpanExporter, pointed at the production collector. Measured, with a spy on both SafeOTLPSpanExporter classes reporting at interpreter exit: before this change the file made 3 real export calls, handed 3 synthetic Project Created spans with invented created_project_id values to the production exporter, and completed 3 connects to the collector. After: 0, 0, 0. Sampling at pytest_sessionfinish reports 0 either way and is how this was missed - BatchSpanProcessor flushes on a background timer, and with no provider.shutdown() the flush lands in the atexit handler, which runs after sessionfinish. --block-network does not prevent it: it is function-scoped and only swaps socket.connect, which a background batch thread outlives. Follows telemetry_with_exporter in tests/telemetry/test_tracer_isolation.py: _NullExporter swapped in before construction, _register_shutdown_handlers suppressed so no atexit hook is left behind, and provider.shutdown() in finally. Patch target is crewai_core because that is where this Telemetry comes from. Also disambiguates the docs rows: the minted ID belongs to the new project, not to the directory the command ran in, and the two can differ. Reworded in all four languages rather than renaming the token to created_project_id - this table documents data, never span-attribute keys (kind and crewai_version on the same row are unnamed), and `project_id` is already the page's name for the pyproject.toml key. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01RfV2uMqWRcdfufMvtdCVoN * docs(ar): use tanwin fath on the letter, not on the alif مشروعًا / جديدًا rather than مشروعاً / جديداً, in the row added by this PR. Both forms appear in docs/edge/ar (3 each), so this is not a house convention being broken either way; the corrected form is the more standard one and the text is mine. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01RfV2uMqWRcdfufMvtdCVoN * docs: stop the creation row implying the span's project_id holds the minted value CodeRabbit re-raised this as Major after I rejected the first version, and it was right to. My rejection argued the page never names wire keys, so introducing created_project_id would be its only one. That part still holds -- grep finds no attribute key anywhere in the four files. But it was the wrong conclusion: the row still used the token `project_id` for a value the span does NOT carry under that key, while the same span's real project_id holds the cwd-derived value. The row also already exposes literal wire values (`crew`, `json_crew`, `flow` are the actual kind values), so "this page has no wire detail" was overstated. Dropping the token resolves the ambiguity without adding the page's only key name: the row now says "the project ID minted for that new project", and names `project_id` only to say the minted value is recorded separately from it. All four languages. No docs/v*/ touched. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01RfV2uMqWRcdfufMvtdCVoN * docs: use an em dash in the creation row, matching the rest of the page The two ASCII `--` occurrences in these files were both mine, introduced by this PR: the page otherwise uses em dashes throughout (en 4, ar 4, ko 2, pt-BR 4). CodeRabbit flagged pt-BR; the same slip was in en, so both are fixed. Now zero ASCII `--` across all four language files. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01RfV2uMqWRcdfufMvtdCVoN --------- Co-authored-by: Claude Opus 5 (1M context) <[email protected]> Co-authored-by: Vidit Ostwal <[email protected]> |
||
|
João MouraandClaude Opus 5
|
4718b190d1 |
fix(cli): open the conversational TUI for a declarative chat flow (#7060)
* fix(cli): open the conversational TUI for a declarative chat flow `crewai run` refused a declarative conversational flow and told the user to drive it from Python. That was wrong: the conversational TUI already exists and already does this job. `CrewRunApp(conversational=True)` renders a chat pane and drives `handle_turn` per message (crew_run_tui.py:833-935), and `kickoff_flow._run_conversational_flow_tui` launches it for a Python conversational Flow. A declaration-built flow satisfies everything that TUI needs -- `handle_turn`, a settable `defer_trace_finalization`, and `finalize_session_traces()` -- so it now routes there instead of exiting. A chat loop still needs a terminal. A headless run (`is_interactive()` false, which folds in CREWAI_DMN) says what it would have needed rather than kicking off a single turn and presenting that as the whole conversation. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> * fix(cli): do not send a human-feedback chat flow to the Textual TUI A declaration can carry both a `conversational:` block and a method with `human_feedback:` -- verified, both predicates return True on the same flow. Routing it to the chat TUI hangs: the runtime collects feedback with a blocking `input()` (flow/runtime/__init__.py:3719) that Textual cannot service, so the prompt is never shown. The STEPS TUI already declines these for exactly this reason. Such a flow now falls back to the terminal REPL, which can prompt. Also updates the guide in en/ar/ko/pt-BR: it still said `crewai run` has no chat loop and exits, which is now the opposite of what the CLI does. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> * fix(cli): reject --inputs on a conversational flow instead of dropping it The conversational branch returns before `_resolve_flow_inputs`, and the TUI calls `handle_turn(message)` -- which owns the kickoff inputs, passing `{"id": session_id}` itself. So any `--inputs` value was silently discarded and the conversation ran as if it had been applied. It now errors, and says that resuming a session by id is not wired up yet rather than implying it worked. Also corrects the Arabic guide: `مُوجّه محجوز` reads as "reserved router", not the blocking prompt it describes. Both found by CodeRabbit on #7060. * fix(cli): document the conversational routing exceptions Three review follow-ups: - The Arabic guide read خدمته (masculine) against the feminine مُطالبة introduced by the last fix. - Both docstrings described a routing path that now has exceptions: a conversational declaration rejects --inputs and skips state-schema resolution, and a human-feedback one uses the terminal REPL. - The --inputs rejection test accepted SystemExit(0); it now pins code 1. Found by CodeRabbit on #7060. * fix(cli): reject --inputs on a chat flow even when it parses empty parse_inputs_json returns {} both when the option is absent and when the user passes --inputs "{}", so the falsy check started the TUI for the second case while the docs said it was unsupported. The conversational path now takes whether the option was supplied, not what it parsed to. Documents the restriction in en, ar, ko and pt-BR. Found by CodeRabbit on #7060. * test(cli): pin the headless conversational exit status pytest.raises(SystemExit) also accepts SystemExit(0), so the error path could regress to a successful exit unnoticed. Found by CodeRabbit on #7060. --------- Co-authored-by: Claude Opus 5 (1M context) <[email protected]> |
||
|
João MouraandClaude Opus 5
|
0c2bcb510c |
feat(cli): backfill project_id from every user-invoked project command (#7057)
* feat(cli): backfill project_id from every user-invoked project command `crewai run` has always backfilled: a project declaring [tool.crewai] without a project_id gets one minted the first time it runs. No other command did, so a project driven entirely through `crewai test`, `crewai deploy` or `crewai traces enable` never acquired an id and every one of its runs stayed unattributable - which is the denominator problem, not a cosmetic gap. Adds the same call to train, replay, test, login, deploy create, deploy push, flow add-crew, enterprise configure and traces enable. Every one is an action the user explicitly invoked, which is the condition run_crew already relies on, so this is the existing principle applied evenly rather than a new policy. It is still never called from the SDK during kickoff, and get_or_create_project_id still refuses to create the [tool.crewai] table, so an unrelated directory is never rewritten. `crewai flow kickoff` is deliberately untouched: it delegates to run_crew and already inherits the backfill. A test pins that so the delegation is not accidentally duplicated. There is no `crewai evaluate` command - `crewai test` is that path. The call is the first statement in each command so a command that later fails still leaves the project with an id. The tests patch the backfill to raise, which proves the call happened and guarantees nothing after it runs, so no test touches user settings, spawns a subprocess or reaches the network. Verified they fail against the unpatched module: 9 command tests fail, the 2 guard tests still pass. Tests live under lib/crewai/tests/cli/ because that is the path the required CI job runs; nothing runs lib/cli/tests/. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01RfV2uMqWRcdfufMvtdCVoN * test(cli): assert the backfill at runtime instead of reading source text Addresses CodeRabbit and github-code-quality on #7057. The two guard tests grepped module source for a call string, which asserts on formatting rather than behavior: a reformat would break them and a real regression could slip past. They now invoke the commands in an isolated project and assert on observed calls. The flow-kickoff test patches the two distinct import sites separately and asserts run_crew's is called exactly once while cli's is not called at all, which is what makes 'delegates' and 'duplicates' distinguishable at runtime rather than by reading the file. Verified both catch what they claim: injecting a duplicate call into flow_run fails the delegation test, and removing run_crew's own call fails the run test. This also drops the module-level 'import crewai_cli.cli as cli_module' that mixed import styles with the existing 'from crewai_cli.cli import crewai', which is the code-quality finding - the rewrite removes the need for it entirely. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01RfV2uMqWRcdfufMvtdCVoN * test(cli): make the exact-once assertion observable Addresses CodeRabbit on #7057, and the finding was correct: with side_effect=_BackfillReached the mock raised on first use, so call_count == 1 was guaranteed by the mock rather than by the code. A second backfill call inside the same run_crew execution could never have been observed. Both backfill mocks now return normally and execution is stopped at the first call AFTER the backfill (configured_project_json_crew), so the recorded count is real. Verified the difference this makes: injecting a duplicate get_or_create_project_id() INSIDE run_crew now fails both tests, which the previous version could not detect at all. The flow_run duplicate case is still caught. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01RfV2uMqWRcdfufMvtdCVoN * test(cli): assert flow kickoff reaches the post-backfill boundary Addresses CodeRabbit on #7057, and the finding was right: the flow-kickoff test discarded the runner.invoke() result, so if the path returned or raised after one backfill call but before configured_project_json_crew, both call-count assertions would still have passed - for the wrong reason. test_run_still_backfills already asserted the boundary; this makes the pair consistent. Verified it earns its place: injecting an early return after the backfill and before the boundary now fails both tests, and previously would have failed neither. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01RfV2uMqWRcdfufMvtdCVoN * test(cli): pin that the backfill precedes command-specific work Addresses CodeRabbit on #7057. The finding is valid: the parametrized test proves the backfill is reached, not that nothing ran before it, so its assertion message claimed more than the test established. Fixed in two parts rather than as proposed. The message now states what the test actually proves, and a new test pins the ordering on login: , whose first action goes through a module-level name that can be patched without reaching into the command. Deliberately not parameterized across all nine commands, which is what the finding suggested: that would mean naming each command's current first action, and those change as commands evolve, so the suite would end up tracking their internals rather than this ordering property. One representative command establishes it, and placement is visible in the diff for the rest. Verified it catches the regression: swapping login's first two statements so its own work runs before the backfill fails the new test. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01RfV2uMqWRcdfufMvtdCVoN --------- Co-authored-by: Claude Opus 5 (1M context) <[email protected]> |
||
|
João Moura
|
5ddf62ae7b | feat: bump versions to 1.15.17 (#7054) | ||
|
|
4dfd074fae |
docs(flow): document declarative conversational flows (#7035)
* test(flow): unskip the conversational end-to-end suite The `conversational_graph_broken` marker parked 21 end-to-end conversational tests with the reason "the definition-first start migration intentionally stopped scanning inherited methods, so that graph no longer registers". That is no longer true: `_iter_flow_methods` walks the MRO for `__conversational_only__` methods (dsl/_utils.py:406-420), so a `conversational = True` subclass does register `route_conversation`, `converse_turn`, `end_conversation` and `answer_from_history_turn` — which `test_flow_definition.py:391-407` already asserts. Removing the marker takes the file from 47 passed / 21 skipped to 68 passed. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> * feat(flow): make conversational opt-in unmistakable Opting a Flow into chat took two statements, and forgetting one failed silently. With `@ConversationConfig(...)` but no `conversational = True`: `FlowDefinition.conversational` came back `None`, the built-in graph never registered, and `handle_turn()` returned `None` without appending a message or raising — while `chat()` reported "only available on conversational flows" on a class that was literally decorated with a conversational config. Three changes: - `ConversationConfig.__call__` now also sets `conversational = True`. Every field on the config is consumed only by the conversational graph, so a decorated non-conversational Flow could only ever discard it. - `FlowConversationalDefinition.enabled` defaults to True. The block is absent on non-conversational flows, so declaring it is the opt-in; `enabled: false` remains an explicit opt-out. - `handle_turn()` raises like `chat()` and `stream_turn()` already do instead of silently returning `None`. Setting `conversational = True` by hand still works and is still the way to opt in without a config. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> * feat(flow): let a declaration drive conversational mode `FlowDefinition.conversational` was written by the DSL projection and read by nothing: every conversational gate resolved through `type(self).flow_definition()` — the class projection — instead of `self._definition`, the declaration a flow was actually built from. So `Flow.from_declaration()` on a definition with a conversational block produced a flow that reported itself non-conversational, dropped the user message, and never registered its declared routes. Resolution rules, applied consistently: - Structure (enabled, methods, route labels, builtin/internal routes) comes from `self._definition`, which is the loaded declaration for a declarative flow and the class projection otherwise. Both paths now agree. - Behavior (`conversational_config`) still prefers the class attribute, which can hold live objects — a configured LLM, a custom BaseLLM, a response_format model class — that the serializable definition degrades to a config dict or a `module:qualname` ref. Reading the definition first would silently downgrade every decorated Python flow. A declaration-built flow has no class config, so `_config_from_definition` supplies one, cached for stable identity. - A declared `state:` block is never replaced. `_create_default_extension_state` is consulted before `_create_definition_state`, so returning `ConversationState` there discarded every field the declaration asked for. It now yields to a declared state and only supplies the default when nothing else does. The class-scoped `_is_conversational` / `_conversational_definition` classmethods are gone; the existing instance-scoped `_is_conversational_enabled` is the single gate. A router `response_format` that survived serialization as a ref or schema dict is dropped with a warning rather than handed to `llm.call()`, which needs a real class. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> * feat(flow): synthesize the built-in conversational methods for declarations A declaration carrying `conversational: {}` loaded clean and then ran zero methods and returned `None`, because the four built-in graph handlers are inherited from `_ConversationalMixin` and a declaration has nothing to inherit from. Authors had to name `crewai.experimental.conversational_mixin:_Conversational Mixin.route_conversation` and three siblings by hand. `Flow._extend_definition` is a new runtime extension hook, called once `_definition` is resolved and before methods are bound. The conversational mixin overrides it to fill in `route_conversation`, `converse_turn`, `end_conversation` and `answer_from_history_turn` when they are missing, using the same code refs the DSL projection already emits so a declaration and a class projection of the same flow produce identical method definitions. Synthesis is deliberately a runtime concern, not a contract one: `FlowDefinition` stays independent of the authoring layer and of the engine, as `test_flow_definition_contract_is_dsl_agnostic` requires, and a loaded declaration still serializes back to exactly what its author wrote. Route descriptions are now carried by the contract. The DSL projects a handler docstring's first line into `FlowMethodDefinition.description`, and the router catalog reads that before falling back to the live docstring. This also fixes a real defect: for a declarative flow `getattr(type(self), handler_name, None)` is `None`, and the old code read `None.__doc__` — so the router LLM was told a route's description was "The type of the None singleton." Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> * fix(flow): let an agent or crew handler reply in a conversation (#7034) `handle_turn` promotes a handler's return value to the assistant message when the handler did not append one itself, but the check required `isinstance(result, str)`. Declarative `agent` and `crew` actions return `LiteAgentOutput` and `CrewOutput`, whose text lives on `.raw` — so the most natural declarative handler was exactly the one whose reply never reached the transcript. `_is_public_turn_result` now unwraps `.raw` before deciding, matching `_stringify_result`, which already did. The routing-artefact guards are applied to the unwrapped text, so an output echoing a route label or this turn's intent is still not promoted. Co-authored-by: Claude Opus 5 (1M context) <[email protected]> * fix(flow): keep privately recorded agent results out of the transcript Unwrapping `.raw` in `_is_public_turn_result` made the end-of-turn fallback promote `LiteAgentOutput` / `CrewOutput` objects that the handler had already recorded via `append_agent_result` with the default private visibility. That call does not set `_assistant_reply_appended`, so the fallback republished the very object the handler asked to keep private — defeating `visible_agent_outputs`. Reproduced against `main` for contrast: no leak before the unwrap, leak after. `append_agent_result` now remembers the object it recorded for the duration of the turn, and the fallback skips anything already routed that way. The check is identity-based on purpose: a handler that records scratch work privately and then returns a user-facing summary still gets that summary promoted, which a simple "handler already handled it" flag would have broken. Found by Cursor Bugbot on #7033. * feat(flow): mark a declarative chat flow conversational on the instance A declaration enables chat through `conversational.enabled`, without the `conversational = True` class attribute. Callers outside this package capability-check that attribute -- the AG-UI serving guide states it as a requirement -- so it disagreed with `_is_conversational_enabled()` and a declarative conversational flow looked non-conversational from outside. `_extend_definition` now sets it on the instance when the definition enables chat. Instance-only on purpose: the DSL projection reads the attribute off the *class* to decide whether to emit a conversational block, so setting it there would make every later subclass look conversational. Verified on a real declarative flow: `conversational` and `stream_turn` both now satisfy the documented capability check, while `Flow.conversational` and any later subclass stay False. * refactor(flow): derive routing-artefact labels from the effective routes `_is_public_turn_result` matched a literal set of route labels, duplicating knowledge that `_effective_builtin_routes()` already owns. A declaration that adds a builtin route was not covered, so a handler echoing that label could be promoted into the transcript -- the same class of divergence already fixed for `route_turn`. Verified the derived set is byte-identical to the old literal one for a class-based flow, so this is a pure generalization: `conversation` and `route_to_flow` stay explicit because neither is a route. Also replaces a tuple-index lambda in the chat REPL test with a named `input_fn`; it relied on tuple evaluation order and on the list being mutated before its length was read. Both found by CodeRabbit on #7033. * docs(flow): document declarative conversational flows The authoring skill told LLM authors "use top-level `conversational` only when the user asks for a chat flow" while documenting none of its 19 fields — there was no ModelSpec for either conversational model, so the API reference appendix skipped them entirely. - Adds both conversational models to the skill reference, with field descriptions, and registers them under the existing `conversational` skip so `skills(skips=["conversational"])` still suppresses the whole block. - Adds authoring rules: do not declare the built-in graph, do not name a handler after the route it listens to, do not declare state unless it needs extra fields, and give every route handler a description. - Documents the declarative form in the conversational-flows guide across en, ar, ko and pt-BR, including what is supplied automatically, how to run it, and what a declaration cannot express (live LLM objects, a response_format class, route_turn overrides). - `crewai run` on a conversational declaration now says it has no chat loop and points at handle_turn/chat, instead of quietly running a single turn and exiting. It fails closed: a flow that cannot be inspected runs normally. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> * fix(flow): render the conversational router section in the skill Both conversational models shared one `Conversational` section, and the template renders only the first model of a non-union section. The router's fields were therefore dropped from the API reference and the generated link to them pointed at a heading that did not exist. Also softens the built-in-handler rule: `_extend_definition` keeps an author-supplied entry and the guide documents that override, so the skill should say to omit those handlers by default rather than never declare them. Adds regression tests for both sections rendering, for every field of both models appearing, and for `skips=["conversational"]` suppressing both. Both found by CodeRabbit on #7035. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> * docs(flow): correct the route-description rule in the authoring skill The rule said every route handler must define `description`, but `conversational.router.route_descriptions` is the higher-precedence source -- `_build_route_catalog` checks the overrides before falling back to the method description. Either one describes a route; the rule now says so, and says what happens when a route has neither. --------- Co-authored-by: Claude Opus 5 (1M context) <[email protected]> Co-authored-by: ViditOstwal <[email protected]> Co-authored-by: Vidit Ostwal <[email protected]> |
||
|
Lorenze Jay
|
852916c8bf | feat: bump versions to 1.15.16 (#6990) | ||
|
João MouraandClaude Opus 5
|
93d7a07422 |
feat(telemetry): count deployments from any origin and record where they started (#6974)
Deployments were only countable through two span types that no aggregation reads, and cli_usage:deploy counted the TUI button rather than deployments. Emit deploy:created and deploy:pushed alongside the existing spans so a deployment is countable from the feature-usage aggregation no matter how it was started, and tag Create Crew Deployment / Start Deployment with source=cli|tui so the two origins stay distinguishable. Separately, the TUI's `t` and `d` key bindings dispatch straight to action_view_traces / action_deploy_crew, which never recorded anything - only on_button_pressed did. Every keyboard-driven trace view and deploy was therefore invisible. Move the recording into the actions, which both input paths funnel through, and past the completed guard so a mid-run keypress that does nothing is not counted as usage. Claude-Session: https://claude.ai/code/session_01ASfWmW3RGy4qAQm6s8U9jH Co-authored-by: Claude Opus 5 (1M context) <[email protected]> |
||
|
João MouraandClaude Opus 5
|
7d2437cf85 |
feat: bump versions to 1.15.15 (#6962)
* feat(flow): report flow outcome and human-in-the-loop signals A flow reported only that it started. FlowFinishedEvent, FlowFailedEvent, MethodExecutionFailedEvent, MethodExecutionPausedEvent and FlowPausedEvent all reached the console formatter and stopped there, and FlowInputRequestedEvent, FlowInputReceivedEvent and ConversationTurnFailedEvent had no listener at all - so success rate, failure rate and every HITL pause were unmeasurable. Adds flow:completed, flow:failed, flow:method_failed, flow:paused, flow:hitl_paused, flow:input_requested, flow:input_received and flow:conversation_turn_failed as feature-usage spans, which the existing feature-usage aggregation already reads. Deliberately does not hold the Flow Execution span open to measure duration: flow_executions_daily_target counts those spans at start, so a run that never finishes would disappear from the count entirely. Duration needs its own span. Counts only - flow names, method names, error text and flow state are never recorded. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01ASfWmW3RGy4qAQm6s8U9jH * feat(flow): record how long a flow ran Adds a Flow Completed span carrying flow_name, duration_ms and outcome, emitted when a flow finishes or fails. Elapsed time comes from a monotonic stamp taken at flow start and cleared on use. Kept separate from the Flow Execution span rather than holding that one open: it is emitted and closed at start and the daily aggregate counts it, so holding it would drop every run that is killed or crashes from the execution count. A killed run now simply has no Flow Completed row, and the count is unaffected. Elapsed time is an explicit duration_ms attribute rather than the span's own duration, which the ingestion pipeline stores as a suffixed string ("0.0000184s") that downstream aggregation parses to zero. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01ASfWmW3RGy4qAQm6s8U9jH * feat(flow): tag flow origin and report resumed runs Two gaps found while testing the pause/resume path end to end. Resumed runs were invisible. There is no resume event: a restored run re-enters through kickoff(), so it looked identical to a fresh start. flow:resumed is derived from _is_execution_resuming at flow start, which makes flow:paused - flow:resumed the abandonment rate. Flow counts are dominated by CrewAI's own AgentExecutor, which is itself a Flow and runs once per agent execution - it is the top flow in the warehouse by a wide margin. Nothing distinguished it from a user's flows except guessing at the name. Both Flow Execution and Flow Completed now carry origin: "internal" when the flow class is defined under crewai.*, "user" otherwise. Tagging only the new span would have left the existing daily count unsplittable. Both span methods take origin with a default, so their signatures stay backward compatible. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01ASfWmW3RGy4qAQm6s8U9jH * fix(flow): scope outcome and resume signals to user flows Two findings from review, both confirmed against the code. Outcome features counted CrewAI's own flows. The agent executor, memory encoding and memory recall are all Flows and all set suppress_flow_events; they run far more often than anything a user wrote, so flow:completed, flow:failed and flow:method_failed were mostly bookkeeping. Those three are now emitted only for flows the caller wrote. Internal outcomes are still recorded on the Flow Completed span, which carries origin. flow:resumed counted checkpoint restores. _is_execution_resuming is set both by from_pending (a human pause) and by a checkpoint restore that never paused for anyone, so resumes could exceed pauses and the abandonment rate was unusable. Keyed off _pending_feedback_context instead, which only from_pending sets. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01ASfWmW3RGy4qAQm6s8U9jH * fix(flow): declare internal flows instead of inferring them Three findings from review, all confirmed against the code. Gating on suppress_flow_events was wrong. That flag asks for console quiet and is a public field, so a caller who set it on their own flow silently lost flow:completed, flow:failed and flow:method_failed. Deciding origin from the defining module was also wrong. Flow.from_declaration() returns a Flow typed in crewai.flow.flow, so a caller's declarative flow was reported as one of CrewAI's own - the inversion this split exists to prevent. Both had the same root cause: the discriminator was inferred. Flow now declares is_crewai_internal, set on the agent executor and the memory encoding/recall flows, and one helper serves both origin and the outcome gate. A failed conversational session was reported as completed. Its session closes with FlowFinishedEvent whatever happened, so a failed turn produced flow:conversation_turn_failed and flow:completed together. The turn failure is now recorded on the flow and read back when the session finishes. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01ASfWmW3RGy4qAQm6s8U9jH * refactor(flow): report flow lifecycle as spans, not feature usage Flow start, completion, pause and method failure are lifecycle facts, and the lifecycle is reported as spans everywhere else. Reporting them through feature usage put them in a table that aggregates on the feature string alone - it cannot carry origin, duration or outcome, so those signals could never be split between a user's flows and the ones CrewAI runs for itself. Adds Flow Paused and Flow Method Failed spans, and a resumed marker on Flow Execution so a run restored from a pause is not counted as a second fresh start. Removes the duplicate feature rows for completed, failed, method_failed, paused and resumed - every one of those facts is now on a span, with more attached to it than the feature row ever carried. Feature usage keeps only genuine adoption signals: flow:hitl_paused, flow:input_requested, flow:input_received and flow:conversation_turn_failed. Also clears the conversational turn-failure flag on every terminal path. A turn that failed without deferred finalization ends via FlowFailedEvent, and the flag left set there marked the next run on that instance as failed. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01ASfWmW3RGy4qAQm6s8U9jH * test(flow): update the flow_execution_span caller for the resumed argument Adding the resumed marker changed a signature that tests/utilities/test_events.py asserts on exactly, and that assertion was not re-run before pushing. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01ASfWmW3RGy4qAQm6s8U9jH * test(flow): make the checkpoint-restore guard actually guard The test asserted that flow:resumed was absent from feature usage, but that signal moved onto the Flow Execution span. The assertion could no longer fail, so a regression that mis-tagged checkpoint restores as resumes would have gone unnoticed. Now asserts the resumed attribute, and waits for the handlers: the manual emit dispatches asynchronously, so the previous shape also read its result before the listener had run. Confirmed it discriminates - keying resumed off _is_execution_resuming again fails it with [('RestoredFlow', True)] == [('RestoredFlow', False)]. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01ASfWmW3RGy4qAQm6s8U9jH * fix(telemetry): record the resumed marker as a string Verified end to end against the live collector and ClickHouse: the pipeline encodes a boolean attribute as the presence of a vBool key, so false arrives as the key simply being absent. That is invisible in the schema and easy to read wrongly - crew_memory is extracted as "the attribute exists" and consequently reports 1 for 99.8% of crews against a field that defaults to False. A string leaves nothing to infer. Confirmed in the warehouse: the emitted span reads resumed = "false". Adds direct coverage for the attributes each flow span records, including both resumed values, and resets the Telemetry singleton in the helper so more than one span method can be exercised per session. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01ASfWmW3RGy4qAQm6s8U9jH * feat: bump versions to 1.15.15 --------- Co-authored-by: Claude Opus 5 (1M context) <[email protected]> |
||
|
Vidit Ostwal
|
11890e6701 |
Standardize CLI flags to kebab-case (#6880)
* Standardize CLI flags to kebab-case with deprecated snake_case aliases. Unify active long-option naming across create, train, test, and replay while keeping hidden backward-compatible aliases and documenting the migration in edge docs and AGENTS.md. * Emit deprecation warnings when snake_case CLI flag aliases are used. Route hidden legacy flags through separate internal params so warnings fire only when the alias is supplied, and extend CLI tests for create, replay, and --help coverage. * Merge CLI deprecation warn helpers into warn_deprecated(kind=...). Replace warn_deprecated_command and warn_deprecated_flag with one helper that accepts kind="command" or kind="flag". |