Compare commits

...
Author SHA1 Message Date
MervinPraisonandClaude Opus 5.5 69d9906752 fix(browser): assert start_automation on send_json; run test_server.py in CI
#5818 moved start_automation delivery from send_text to send_json, but
test_ws_no_origin_cli_start_session still asserted send_text, and a
not-awaited check on send_text became vacuous. CI never ran test_server.py
(only test_protocol_compliance.py), so the break went unseen.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
2026-10-26 12:11:33 +00:00
Mervin PraisonandClaude Opus 5.5 e10129677f fix(ci): indent heredoc so the sync-to-praisonai-js workflow parses (#6009)
The PR body heredoc started at column 0, which ends the run: | block, so the
file was invalid YAML: every push recorded a failed zero-job run and the
workflow could not be dispatched. YAML strips the block indentation, so the
shell still sees the same text.

Co-authored-by: Claude Opus 5.5 (1M context) <[email protected]>
2026-10-26 13:07:13 +01:00
5ab53c48b9 fix(ci): reinstall litellm after unclecode-litellm clobbers it (#6008)
* fix(ci): reinstall litellm after unclecode-litellm clobbers it

crawl4ai depends on unclecode-litellm==1.81.13, which installs into the same
litellm/ directory as litellm. uv installs both in parallel, so whichever lands
last wins and test shards fail at random with
'cannot import name get_env_int_in_range from litellm...env_utils'.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* fix(ci): use version-agnostic litellm import check after reinstall

get_env_int_in_range does not exist at the litellm>=1.83.14 floor, so the
check would fail on a valid lower resolution. 'import litellm' loads
constants.py, which is where the clobbered import surfaced.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

---------

Co-authored-by: Claude Opus 5.5 (1M context) <[email protected]>
Co-authored-by: MervinPraison <[email protected]>
2026-10-26 13:05:26 +01:00
praisonai-triage-agent[bot] 649ccf2af9 Merge pull request #6000 from MervinPraison/claude/issue-5997-20261026
fix: implement shared-session observers in the gateway server
2026-10-26 11:58:58 +00:00
MervinPraison 790350da9b docs: auto-update feature parity trackers [skip ci] 2026-10-26 11:52:37 +00:00
2bbe6dbc4a fix: route LLM retry, token budgets and state saves through shared helpers (#5841)
* fix: route LLM retry, token budgets and state saves through shared helpers (fixes #5838)

- classify_error_kind: status code / exception type (error_classifier) is
  authoritative; message text only refines within that category. A 400
  containing "500" is no longer retried as overloaded, 401/403 map to auth,
  404 to model_not_found, a bare 503 to overloaded.
- _parse_retry_delay accepts the exception so Retry-After headers are honoured.
- rag.budget.estimate_tokens, the max_budget pre-call cost guard and
  RouterAgent use context.tokens.estimate_tokens_heuristic (CJK-aware).
- FileTracker, PermissionAllowlist and FileIndex saves use atomic_write_json;
  a corrupt allowlist is quarantined and loads as empty (fail closed).

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix: truncate RAG context by estimated tokens, not 4 chars/token, so CJK stays within budget

Co-Authored-By: Claude Opus 5.5 <[email protected]>

---------

Co-authored-by: MervinPraison <[email protected]>
Co-authored-by: Claude Opus 5.5 <[email protected]>
2026-10-26 12:50:51 +01:00
a5b6a28f1e fix: ImageAgent uses an image-model default instead of the chat default (#5948)
* fix: ImageAgent uses an image-model default instead of the chat default (fixes #5944)

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix: ImageAgent honours an explicit model= as the image model

Co-Authored-By: Claude Opus 5.5 <[email protected]>

---------

Co-authored-by: MervinPraison <[email protected]>
Co-authored-by: Claude Opus 5.5 <[email protected]>
2026-10-26 12:49:56 +01:00
6e0d559418 fix: shut down live_server runtime HTTP server on fixture teardown (fixes #5938) (#5940)
Co-authored-by: MervinPraison <[email protected]>
Co-authored-by: Claude Opus 5.5 <[email protected]>
2026-10-26 12:49:52 +01:00
708f4012cf fix(browser): extension start_session without start_automation echo (Windows) (#5818)
* fix(browser): skip start_automation echo for extension WebSocket (Windows)

* fix: update browser server tests for extension start_session contract

- Delivery tests mock send_json (matches server's send_json path)
- Add coverage for extension-originated start_session skipping start_automation echo
- Drop now-unused json_mod import

Co-Authored-By: Claude Opus 5.5 <[email protected]>

---------

Co-authored-by: Dhivya-Bharathy <[email protected]>
Co-authored-by: MervinPraison <[email protected]>
Co-authored-by: Claude Opus 5.5 <[email protected]>
2026-10-26 12:49:47 +01:00
4f52d950ef fix: A2A cancel is real and final, shell timeout keeps partial output, async EventBus errors are logged (#5991)
* fix: make A2A cancel real and final, keep shell output on timeout, log async EventBus subscriber errors (fixes #5990)

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix: drop late A2A results after cancel, cap store on completion, keep unterminated shell output on timeout

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix: keep in-flight sync A2A tasks from eviction so a cancelled send can still respond

Co-Authored-By: Claude Opus 5.5 <[email protected]>

---------

Co-authored-by: MervinPraison <[email protected]>
Co-authored-by: Claude Opus 5.5 <[email protected]>
Co-authored-by: Mervin Praison <[email protected]>
2026-10-26 12:47:59 +01:00
MervinPraison ce8cb991c4 docs: auto-update feature parity trackers [skip ci] 2026-10-26 11:46:26 +00:00
953196e4ed fix: per-turn loop guard; caller errors no longer open tool circuit breaker (#5861)
* fix: per-turn loop guard and caller errors no longer open tool circuit breaker (fixes #5860)

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix: caller errors are neutral breaker outcomes; streams get their own loop guard

Caller errors (bad input) were returned through the circuit breaker and
counted as successes, so they could close a half-open breaker while the
dependency was still down. They are now recorded as neither success nor
failure. start(stream=True)/iter_stream() now bind a per-turn loop guard
like chat()/achat(), so overlapping streams neither exhaust nor inherit
each other's limits.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

---------

Co-authored-by: MervinPraison <[email protected]>
Co-authored-by: Claude Opus 5.5 <[email protected]>
2026-10-26 12:45:42 +01:00
1ebbdf0a0f fix: add device-identity handshake contract to gateway hello (#6001)
* fix: add device-identity handshake contract to gateway hello (fixes #5998)

Additive, stdlib-only core contract: DeviceAuth wire shape, optional
HelloParams.device, decode-time validation (device_id must equal the
public-key fingerprint), canonical signing payload bound to nonce/agent,
and a DeviceIdentityProtocol seam for the wrapper gateway to implement.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix: canonical device-key fingerprint, overflow-safe signed_at, pinned wire vectors (#5998)

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix: make DeviceAuth.signed_at integer seconds so signed bytes are language-neutral

repr(float) is Python-specific (1700000000 -> '1700000000.0', 1e16 -> '1e+16'),
so a JS/TS client could not reproduce signing_payload bytes and every
signature would fail verification.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

---------

Co-authored-by: MervinPraison <[email protected]>
Co-authored-by: Claude Opus 5.5 <[email protected]>
Co-authored-by: Mervin Praison <[email protected]>
2026-10-26 12:45:37 +01:00
5140e24913 fix: release gateway admission slot while a turn awaits human approval (#5917)
* fix: release gateway admission slot while a turn awaits human approval (fixes #5912)

The approval registry enters an optional host-bound human-wait scope around
the backend wait; the gateway AdmissionGate binds one per admitted turn that
lends the run slot back and re-acquires it (bypassing the policy) when the
decision arrives. Sync agents now receive the turn context in their worker
thread so the hook reaches them. No hook bound -> unchanged behaviour.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix: never block a resumed human wait on a slot; borrow and repay instead

Addresses Greptile P1s on #5917:
- sync approvals parked in pool workers could deadlock against a turn holding
  the slot but waiting for a worker
- with parallel approvals, an early-answered branch resumed without a slot

Co-Authored-By: Claude Opus 5.5 <[email protected]>

---------

Co-authored-by: MervinPraison <[email protected]>
Co-authored-by: Claude Opus 5.5 <[email protected]>
2026-10-26 12:45:33 +01:00
c560fe5ae3 fix: show the run's effective model and its source in config provenance (#5907)
* fix: surface the model a run actually dispatches in config provenance (fixes #5906)

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix: report configured agent.model as run.model in config provenance (#5906)

run prefers config agent.model over resolve_default_model(), so provenance
must too; previously a recent/state model was reported instead.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix: keep provider notice/source on model-resolver exception fallback (#5906)

Co-Authored-By: Claude Opus 5.5 <[email protected]>

---------

Co-authored-by: MervinPraison <[email protected]>
Co-authored-by: Claude Opus 5.5 <[email protected]>
Co-authored-by: Mervin Praison <[email protected]>
2026-10-26 12:45:28 +01:00
777c4f1cce fix: single-source YAML tool-name keys for ToolResolver build and validate (#5754)
* fix: single-source YAML tool-name keys for ToolResolver build and validate (fixes #5753)

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix: validate_yaml_tools accepts the same local tool sources the build merges

Co-Authored-By: Claude Opus 5.5 <[email protected]>

---------

Co-authored-by: MervinPraison <[email protected]>
Co-authored-by: Claude Opus 5.5 <[email protected]>
Co-authored-by: Mervin Praison <[email protected]>
2026-10-26 12:45:23 +01:00
3c53f43767 fix: use canonical_preset_key in param_resolver instead of inline duplicate (fixes #5148) (#5226)
Route preset resolution through parse_utils.canonical_preset_key(), the
documented single source of truth for the preset spelling rule, instead of
re-implementing its body inline. Keeps validation and resolution from drifting
apart. Behaviour-preserving.

Co-authored-by: praisonai-triage-agent[bot] <272766704+praisonai-triage-agent[bot]@users.noreply.github.com>
Co-authored-by: Mervin Praison <[email protected]>
Co-authored-by: Mervin Praison <[email protected]>
2026-10-26 12:45:18 +01:00
MervinPraisonandClaude Opus 5.5 734770ac37 fix: attached observers cannot rebind via hello/join; replay records only the recipient's own session
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-26 11:42:31 +00:00
praisonai-triage-agent[bot] 3a76737e49 Merge pull request #5947 from MervinPraison/claude/issue-5945-20261022
fix: screen resolved api_base addresses and scope ComputeManagedAgent files per call
2026-10-26 11:21:59 +00:00
praisonai-triage-agent[bot] 4d3d96ba3e Merge pull request #5901 from MervinPraison/claude/issue-5899-20261019
fix: FileScheduleStore loses run history across processes and hides save failures
2026-10-26 11:19:28 +00:00
5b48056781 fix: transcribe inbound voice notes on Discord and Signal (fixes #5929) (#5930)
* fix: transcribe inbound voice notes on Discord and Signal via shared STT helper (fixes #5929)

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix: transcribe voice notes only after access gates, skip download when STT off, read full Signal body

Co-Authored-By: Claude Opus 5.5 <[email protected]>

---------

Co-authored-by: MervinPraison <[email protected]>
Co-authored-by: Claude Opus 5.5 <[email protected]>
Co-authored-by: Mervin Praison <[email protected]>
2026-10-26 12:18:12 +01:00
praisonai-triage-agent[bot] 17aac57b9b Merge pull request #5970 from MervinPraison/claude/issue-5967-20261024
fix: close SSRF gaps in MCP OAuth discovery and web_crawl urllib fallback
2026-10-26 11:14:53 +00:00
MervinPraisonandClaude Opus 5.5 23bb2af225 fix: member-driven shared-session turns reach the owner and its replay buffer
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-26 11:07:10 +00:00
Mervin Praison 1fccb5739f Merge branch 'main' into claude/issue-5997-20261026 2026-10-26 12:01:34 +01:00
Mervin Praison 66e6868d49 Merge branch 'main' into claude/issue-5945-20261022 2026-10-26 12:01:28 +01:00
Mervin Praison 546cb79a7f Merge branch 'main' into claude/issue-5899-20261019 2026-10-26 12:01:22 +01:00
MervinPraisonandClaude Opus 5.5 82a062316a fix: close redirect responses in web_crawl urllib fallback
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-26 10:42:24 +00:00
Mervin PraisonandClaude Opus 5.5 0be88481af fix(ci): outsider flagging counts only what the run changed; writers can clear a wrong flag (#6007)
* fix(ci): outsider flagging counts only what the run changed; writers can clear a wrong flag

Every automation acts as the App, so 'the App touched it during an outsider run' also caught routine pipeline label syncs on unrelated PRs (e.g. #5901/#5907 flagged by run 39229895860, whose transcript never mentions them), and the sticky rule then cascaded flags through references (#5945/#5947 -> #5949/#5950). touchedByRun now counts only content-changing App events (comments, reviews, pushes, title/body edits, creating the item, adding the claude label) inside the run window; replaying the real run flags only its own PR. A person with write access can clear a wrong flag by removing the label; bots cannot.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* fix(ci): count gate-loosening label changes and bound the outsider window by the Claude job's end

Addresses review on #6007: removing no-auto-merge or a manual-only label (or adding an
override) is a touch; the flag step passes claude-response's completed_at as the upper bound.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* fix(ci): match reusable-workflow job names in jobCompletedAt

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

---------

Co-authored-by: Claude Opus 5.5 (1M context) <[email protected]>
2026-10-26 11:40:43 +01:00
Mervin Praison 04694dd1f9 Merge branch 'main' into claude/issue-5967-20261024 2026-10-26 11:37:15 +01:00
2bc462f71a fix: gateway mcp_servers block with one shared MCP client per downstream server (#6006)
* fix: gateway mcp_servers block shares one pooled MCP client across routed agents (fixes #6004)

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix: prefix pooled MCP tools by server, open servers off-loop on reload, retire unused clients

Co-Authored-By: Claude Opus 5.5 <[email protected]>

---------

Co-authored-by: MervinPraison <[email protected]>
Co-authored-by: Claude Opus 5.5 <[email protected]>
2026-10-26 11:22:36 +01:00
374701f529 fix: fail CI when TS GatewayEventType drifts from Python EventType (#6005)
* fix: lockstep test keeping TS GatewayEventType in sync with Python EventType (fixes #6003)

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix: fail lockstep test on unparseable TS GatewayEventType entries

Co-Authored-By: Claude Opus 5.5 <[email protected]>

---------

Co-authored-by: MervinPraison <[email protected]>
Co-authored-by: Claude Opus 5.5 <[email protected]>
2026-10-26 11:17:36 +01:00
fc4f83cde3 fix: don't silently replace stored key on auth login (#5950)
* fix: confirm before auth login replaces an existing stored key (fixes #5949)

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix: report replaced key in auth login JSON output

Co-Authored-By: Claude Opus 5.5 <[email protected]>

---------

Co-authored-by: MervinPraison <[email protected]>
Co-authored-by: Claude Opus 5.5 <[email protected]>
2026-10-26 11:17:32 +01:00
a2c72c6a16 chore(benchmarks): date README perf claim, archive superseded results, fix scripts, re-measure (#5836)
* chore(benchmarks): date README perf claim, archive superseded results, fix benchmark scripts, re-measure (fixes #5824)

- Re-measured on a quiet runner: Agent instantiation median 20.3us (1.6.60)
  -> 47.1us (main) on identical hardware; gradual creep, recorded in
  INSTANTIATION_RCA.md
- performance_benchmark.py: lazy framework imports, rank-based 'fastest'
  label, ratios vs the fastest, min/median/std + versions + machine info,
  no README rewrite; BENCHMARK_RESULTS.md regenerated in one run
- simple_benchmark.py: --save writes SIMPLE_BENCHMARK_RESULTS.md; README
  updater removed
- Archive superseded result files with a 'Superseded' header
- benchmark.yml: PR gate via simple_benchmark --json --fail-over-threshold;
  parse failures now fail instead of passing as 0; threshold 450 -> 150us
- Delete dead nested src/praisonai-agents/.github workflow
- README: dated 46us figure linking BENCHMARK_RESULTS.md

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* docs(benchmarks): link follow-up issue #5835 in RCA

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(benchmarks): absolute README link, validate --iterations/--warmup, accurate reproduce command

Co-Authored-By: Claude Opus 5.5 <[email protected]>

---------

Co-authored-by: MervinPraison <[email protected]>
Co-authored-by: Claude Opus 5.5 <[email protected]>
2026-10-26 11:17:27 +01:00
2c65365389 fix: isolate display-callback errors, atomic state writes, deadlines on remote git/HTTP (#5871)
* fix: isolate display-callback errors, write durable state atomically, add deadlines to remote git/HTTP calls (fixes #5870)

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix: keep symlink target and file mode when saving config; assert cached fallback on git timeout

Co-Authored-By: Claude Opus 5.5 <[email protected]>

---------

Co-authored-by: MervinPraison <[email protected]>
Co-authored-by: Claude Opus 5.5 <[email protected]>
2026-10-26 11:17:20 +01:00
9a6ae6ea4b fix: adapter YAML parity, owner-scoped jobs reconcile, safe-by-default agent_file root (#5881)
* fix: honour crewai/autogen YAML config, owner-scoped jobs reconcile, cwd-default agent_file root (fixes #5880)

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix: pid-reuse-safe job owner, strict owner migration, keep manager_llm model name (#5880)

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix: treat unreadable owner start time as unknown; ps-based start time off /proc (#5880)

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix: cap litellm <1.106 (requires mcp>=2.2, conflicts with mcp<2 pin) (#5880)

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix: owner-migration test injects failure at ALTER TABLE, not PRAGMA (#5880)

Co-Authored-By: Claude Opus 5.5 <[email protected]>

---------

Co-authored-by: MervinPraison <[email protected]>
Co-authored-by: Claude Opus 5.5 <[email protected]>
2026-10-26 11:17:15 +01:00
MervinPraisonandClaude Opus 5.5 dc828259ab fix: validate visibility before creating a session, persist it in the initial snapshot, and revoke observer access when visibility is restricted
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-26 09:39:35 +00:00
MervinPraisonandClaude Opus 5.5 d220e92f15 fix: implement shared-session observers (attach/detach/observers) in gateway server (fixes #5997)
WebSocketGateway now implements core's SessionObserverProtocol honouring
SessionVisibility/SessionSharingRole: private sessions stay 1:1 (default),
read_only admits viewers, shared admits members who co-drive via the normal
message path. The live stream and final frame fan out to attached observers
gated by their OperatorScope, and an observer leaving/disconnecting detaches
instead of closing the owner's session.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-26 09:31:03 +00:00
MervinPraisonandClaude Opus 5.5 ae0c5f2636 test: mock urllib opener in web_crawl size-cap test after SSRF redirect walk
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-24 07:46:15 +00:00
MervinPraisonandClaude Opus 5.5 d03321ac4e fix: reject non-2xx OAuth credential responses before saving tokens
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-24 07:45:22 +00:00
MervinPraisonandClaude Opus 5.5 e638d18b5b fix: route MCP OAuth and web_crawl urllib fetches through the SSRF guard (fixes #5967)
- oauth_discovery._require_https now also rejects private/link-local/
  ambiguous hosts via tools.url_safety.is_safe_http_url (loopback still
  allowed for local testing)
- _get_json follows redirects manually, validating each hop before it is
  requested
- oauth_registration reuses the single _require_https; credential POSTs
  (token exchange, refresh, DCR) no longer follow redirects
- web_crawl urllib fallback walks redirects by hand with SSRF checks per hop

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-24 07:45:22 +00:00
MervinPraisonandClaude Opus 5.5 83554d8077 test: align #5945 tests with main's api_base screener after rebase
Main (#5960) replaced the passthrough api_base check with the shared
resolve-and-pin screener; drop the duplicate passthrough tests and keep the
ComputeManagedAgent per-call path test plus an IPv4-mapped literal check.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-23 08:34:41 +00:00
MervinPraisonandClaude Opus 5.5 6f69043f59 fix: run api_base DNS validation off the event loop in apassthrough
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-23 08:34:10 +00:00
MervinPraisonandClaude Opus 5.5 f599e91dc8 fix: screen resolved api_base addresses and scope ComputeManagedAgent files per call (fixes #5945)
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-23 08:34:07 +00:00
MervinPraisonandClaude Opus 5.5 7afb2265ee fix: keep run history newest-first when retrying unsaved records
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-19 08:07:16 +00:00
MervinPraisonandClaude Opus 5.5 9030155857 fix: keep history on read/write errors and contain store write errors in mark_run
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-19 07:32:50 +00:00
MervinPraisonandClaude Opus 5.5 49aee2d618 fix: FileScheduleStore cross-process run history and fail-loud persistence (fixes #5899)
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-19 07:23:47 +00:00
107 changed files with 4602 additions and 1233 deletions

No files matched your search

@@ -94,3 +94,17 @@ runs:
cd "$WRAPPER_PKG"
pip install -e "${WRAPPER_SUFFIX}"
fi
# crawl4ai depends on unclecode-litellm, a fork that writes into the same
# litellm/ directory. Installed together, the two overwrite each other's
# files in whatever order they land, so imports of litellm fail at random.
# Reinstall the resolved litellm last so its own files win.
if python -m pip show unclecode-litellm >/dev/null 2>&1; then
LITELLM_VERSION="$(python -m pip show litellm | awk '/^Version:/{print $2}')"
if [ "${{ inputs.use-uv }}" = "true" ]; then
uv pip install --system --reinstall --no-deps "litellm==${LITELLM_VERSION}"
else
pip install --force-reinstall --no-deps "litellm==${LITELLM_VERSION}"
fi
python -c "import litellm"
fi
@@ -178,13 +178,43 @@ assert('no source issue', o.sourceIssueNumbers({ head: { ref: 'feat/x' }, body:
await o.touchedByRun(runGh([{ event: 'committed', committer: { date: '2026-10-17T10:07:00Z' } }]), 'o', 'r', item(), SINCE, APP));
assert('App editing the body during the run is touched (GraphQL login has no [bot])',
await o.touchedByRun(runGh([], [{ editedAt: '2026-10-17T10:08:00Z', editor: { login: 'praisonai-triage-agent' } }]), 'o', 'r', item(), SINCE, APP));
assert('routine label sync by the App during the run is not touched (shared App identity)',
!(await o.touchedByRun(runGh([{ event: 'unlabeled', actor: { login: 'praisonai-triage-agent[bot]' }, label: { name: 'pipeline/blocked:ci' }, created_at: '2026-10-17T10:06:00Z' }]), 'o', 'r', item(), SINCE, APP)));
assert('the App adding the claude label during the run is touched (it starts a Claude run)',
await o.touchedByRun(runGh([{ event: 'labeled', actor: { login: 'praisonai-triage-agent[bot]' }, label: { name: 'claude' }, created_at: '2026-10-17T10:06:00Z' }]), 'o', 'r', item(), SINCE, APP));
const appLabel = (event, name) => runGh([{ event, actor: { login: 'praisonai-triage-agent[bot]' }, label: { name }, created_at: '2026-10-17T10:06:00Z' }]);
assert('the App removing no-auto-merge during the run is touched (it loosens the gate)',
await o.touchedByRun(appLabel('unlabeled', 'no-auto-merge'), 'o', 'r', item(), SINCE, APP));
assert('the App removing a manual-only label during the run is touched',
await o.touchedByRun(appLabel('unlabeled', 'needs-manual-review'), 'o', 'r', item(), SINCE, APP));
assert('the App adding a maintainer-accept override during the run is touched',
await o.touchedByRun(appLabel('labeled', 'maintainer-accept-core-tools'), 'o', 'r', item(), SINCE, APP));
assert('the App adding no-auto-merge (tightening) is not touched',
!(await o.touchedByRun(appLabel('labeled', 'no-auto-merge'), 'o', 'r', item(), SINCE, APP)));
assert('the gate releasing its own lock label is not touched',
!(await o.touchedByRun(appLabel('unlabeled', 'claude-merge-gate-active'), 'o', 'r', item(), SINCE, APP)));
assert('the App force-pushing during the run is touched',
await o.touchedByRun(runGh([{ event: 'head_ref_force_pushed', actor: { login: 'praisonai-triage-agent[bot]' }, created_at: '2026-10-17T10:06:00Z' }]), 'o', 'r', item(), SINCE, APP));
assert('a maintainer comment during the run is not touched',
!(await o.touchedByRun(runGh([{ event: 'commented', user: { login: 'MervinPraison' }, created_at: '2026-10-17T10:06:00Z' }]), 'o', 'r', item(), SINCE, APP)));
assert('App activity after the run ended is not touched',
!(await o.touchedByRun(runGh([{ event: 'commented', user: { login: 'praisonai-triage-agent[bot]' }, created_at: '2026-10-20T10:00:00Z' }]), 'o', 'r', item(), SINCE, APP, '2026-10-17T10:30:00Z')));
assert('App activity before the run is not touched',
!(await o.touchedByRun(runGh([{ event: 'labeled', actor: { login: 'praisonai-triage-agent[bot]' }, created_at: '2026-10-17T09:00:00Z' }]), 'o', 'r', item(), SINCE, APP)));
const editFail = await (async () => { const g = runGh([]); g.graphql = async () => { throw new Error('rate'); }; return o.touchedByRun(g, 'o', 'r', item(), SINCE, APP).then(() => null, (e) => e); })();
assert('an edit-history lookup failure fails closed (throws)', editFail instanceof Error);
{
const jobs = [{ name: 'issue-triage', completed_at: '2026-10-17T10:00:01Z' }, { name: 'claude-response', completed_at: '2026-10-17T10:20:00Z' }];
const g = { rest: { actions: { listJobsForWorkflowRun: 'jobs' } }, paginate: async () => jobs };
assert('the Claude job end time bounds the run window', (await o.jobCompletedAt(g, 'o', 'r', 1, 'claude-response')) === '2026-10-17T10:20:00Z');
const hub = { rest: g.rest, paginate: async () => [{ name: 'claude / claude-response', completed_at: '2026-10-17T10:21:00Z' }] };
assert('a reusable workflow job (caller / job) is found', (await o.jobCompletedAt(hub, 'o', 'r', 1, 'claude-response')) === '2026-10-17T10:21:00Z');
assert('an unfinished or missing job gives no bound', (await o.jobCompletedAt(g, 'o', 'r', 1, 'nope')) === null);
const bad = { rest: { actions: { listJobsForWorkflowRun: 'jobs' } }, paginate: async () => { throw new Error('403'); } };
assert('a jobs lookup failure gives no bound (caller counts up to now)', (await o.jobCompletedAt(bad, 'o', 'r', 1, 'claude-response')) === null);
}
// A request mentioning another repo's numbers as bare #N (Docs issues cite PraisonAI PRs this way)
// must not make the run outsider: a number that names nothing here carries no content.
r = await o.referencedIssueReasons(fakeGithub({ issueError: 404 }), 'o', 'r', 'see upstream #5987 and #6366');
@@ -211,6 +241,21 @@ assert('no source issue', o.sourceIssueNumbers({ head: { ref: 'feat/x' }, body:
r = await o.outsiderOriginReasons(fakeGithub({ issueError: 404 }), 'o', 'r', internalPr(), []);
assert('a missing SOURCE issue (404) still fails closed', r.some((x) => x.includes('unavailable (HTTP 404)')));
// A person with write access can clear a wrong flag; bots cannot.
const lab = (event, login, t) => ({ event, label: { name: o.OUTSIDER_ORIGIN_LABEL }, actor: { login }, created_at: t });
r = await o.outsiderOriginReasons(fakeGithub({ issue: maintainerIssue, timeline: [
lab('labeled', 'praisonai-triage-agent[bot]', '2026-10-19T09:30:00Z'), lab('unlabeled', 'MervinPraison', '2026-10-26T10:00:00Z')] }), 'o', 'r', internalPr(), []);
assert('a writer removing the flag clears it', !r.some((x) => x.includes('flagged')));
r = await o.outsiderOriginReasons(fakeGithub({ issue: maintainerIssue, timeline: [
lab('labeled', 'praisonai-triage-agent[bot]', '2026-10-19T09:30:00Z'), lab('unlabeled', 'praisonai-triage-agent[bot]', '2026-10-26T10:00:00Z')] }), 'o', 'r', internalPr(), []);
assert('a bot removing the flag does not clear it', r.some((x) => x.includes('flagged')));
r = await o.outsiderOriginReasons(fakeGithub({ issue: maintainerIssue, timeline: [
lab('labeled', 'praisonai-triage-agent[bot]', '2026-10-19T09:30:00Z'), lab('unlabeled', 'stranger', '2026-10-26T10:00:00Z')] }), 'o', 'r', internalPr(), []);
assert('a non-writer removing the flag does not clear it', r.some((x) => x.includes('flagged')));
r = await o.outsiderOriginReasons(fakeGithub({ issue: maintainerIssue, timeline: [
lab('labeled', 'x[bot]', '2026-10-19T09:30:00Z'), lab('unlabeled', 'MervinPraison', '2026-10-20T10:00:00Z'), lab('labeled', 'x[bot]', '2026-10-21T10:00:00Z')] }), 'o', 'r', internalPr(), []);
assert('flagged again after a clear is flagged', r.some((x) => x.includes('flagged')));
r = await o.outsiderOriginReasons(fakeGithub({ issueError: 410 }), 'o', 'r', internalPr(), []);
assert('a deleted or transferred source issue (410) counts as unknown, so outsider', r.some((x) => x.includes('unavailable (HTTP 410)')));
+48 -6
View File
@@ -26,6 +26,15 @@ const TRUSTED_AUTOMATION = new Set(['Copilot']);
const SECURITY_STATUS_CONTEXT = 'security/outsider-review';
const SECURITY_STATUS_CREATOR = 'github-actions[bot]';
const OUTSIDER_ORIGIN_LABEL = 'origin/outsider';
// Timeline events that change content Claude or the merge gate reads.
const CONTENT_EVENTS = new Set(['commented', 'reviewed', 'line-commented', 'commit-commented', 'head_ref_force_pushed', 'renamed']);
// Label changes that loosen what Claude or the merge gate does (see merge-gate.js):
// adding a trigger or an override, or removing a stop or manual-only label.
// Pipeline status labels and the gate's own lock are routine and do not count.
const LOOSENING_LABELS = {
labeled: new Set(['claude', 'merge-gate-ci-only', 'maintainer-accept-example', 'maintainer-accept-core-tools']),
unlabeled: new Set(['no-auto-merge', 'claude-conflict-pending', 'auto-merged-by-gate', 'security', 'breaking-change', 'needs-manual-review', 'release']),
};
// Build, dependency, CI and install-time surface. Code here runs on developer
// machines, in CI with secrets, or at install time, so an outsider-origin
@@ -164,12 +173,24 @@ async function outsiderOriginReasons(github, owner, repo, pr, comments = []) {
return reasons;
}
/**
* Was this ever flagged? Timeline events cannot be edited, so deleting the
* outsider content does not clear it. Only a person with write access can clear
* a wrong flag, by removing the label after it was last added (bots cannot:
* every Claude run acts as the App).
*/
async function wasLabelled(github, owner, repo, number, label) {
try {
const events = await github.paginate(github.rest.issues.listEventsForTimeline, {
const events = (await github.paginate(github.rest.issues.listEventsForTimeline, {
owner, repo, issue_number: number, per_page: 100,
});
return events.some((e) => e.event === 'labeled' && e.label?.name === label);
})).filter((e) => (e.event === 'labeled' || e.event === 'unlabeled') && e.label?.name === label);
const lastAdded = events.map((e) => e.event).lastIndexOf('labeled');
if (lastAdded < 0) return false;
const isWriter = makePermissionLookup(github, owner, repo);
for (const e of events.slice(lastAdded + 1)) {
if (e.event === 'unlabeled' && !isAutomation(e.actor?.login) && (await isWriter(e.actor.login))) return false;
}
return true;
} catch (err) {
if (err.status === 404 || err.status === 410) return false;
throw err;
@@ -233,16 +254,20 @@ async function issueReasons(github, owner, repo, issueNumber, isWriter, label =
* commits to it, or edited its text. Unrelated activity by people does not count,
* so flagging what an outsider run touched does not sweep up other work.
*/
async function touchedByRun(github, owner, repo, item, since, appLogins) {
async function touchedByRun(github, owner, repo, item, since, appLogins, until = new Date().toISOString()) {
const t = new Date(since).getTime();
const after = (d) => !!d && new Date(d).getTime() >= t;
const end = new Date(until).getTime();
const after = (d) => !!d && new Date(d).getTime() >= t && new Date(d).getTime() <= end;
const isApp = (login) => !!login && (appLogins.has(login) || appLogins.has(`${login}[bot]`));
if (isApp(item.user?.login) && after(item.created_at)) return true;
const events = await github.paginate(github.rest.issues.listEventsForTimeline, {
owner, repo, issue_number: item.number, per_page: 100,
});
for (const e of events) {
if (isApp(e.actor?.login || e.user?.login) && after(e.created_at || e.submitted_at || e.updated_at)) return true;
// Only actions that change what Claude or the gate reads. Label syncing by the
// pipeline also runs as the App and must not count as the outsider run.
const content = CONTENT_EVENTS.has(e.event) || !!LOOSENING_LABELS[e.event]?.has(e.label?.name);
if (content && isApp(e.actor?.login || e.user?.login) && after(e.created_at || e.submitted_at || e.updated_at)) return true;
if (e.event === 'committed' && after(e.committer?.date)) return true;
}
let data;
@@ -265,6 +290,22 @@ async function touchedByRun(github, owner, repo, item, since, appLogins) {
return edits.some((n) => after(n.editedAt) && isApp(n.editor?.login));
}
/**
* When a job of this workflow run finished (null if unknown). It bounds what counts
* as that job's activity, so work by later runs is not attributed to it.
*/
async function jobCompletedAt(github, owner, repo, runId, jobName) {
try {
const jobs = await github.paginate(github.rest.actions.listJobsForWorkflowRun, {
owner, repo, run_id: runId, per_page: 100,
});
// A reusable workflow's jobs are named '<caller job> / <job>'.
return jobs.find((j) => j.name === jobName || j.name.endsWith(` / ${jobName}`))?.completed_at || null;
} catch {
return null;
}
}
/**
* Why an issue Claude is asked to work on is outsider-influenced ([] when not):
* the issue itself, anything it was flagged as before, and every issue the
@@ -441,6 +482,7 @@ module.exports = {
referencedIssueNumbers,
referencedIssueReasons,
touchedByRun,
jobCompletedAt,
outsiderSensitivePaths,
riskFlags,
formatRiskFlags,
+52 -8
View File
@@ -11,9 +11,52 @@ on:
description: 'Fail if performance regresses'
required: false
default: 'true'
pull_request:
paths:
- 'src/praisonai-agents/praisonaiagents/agent/**'
- 'src/praisonai-agents/praisonaiagents/llm/**'
- 'src/praisonai-agents/benchmarks/simple_benchmark.py'
- 'src/praisonai-agents/benchmarks/thresholds.json'
- '.github/workflows/benchmark.yml'
permissions:
contents: read
jobs:
instantiation-gate:
# Fails the PR when Agent instantiation exceeds benchmarks/thresholds.json.
runs-on: ubuntu-latest
timeout-minutes: 15
steps:
- name: Checkout repository
uses: actions/checkout@v4
with:
persist-credentials: false
- name: Set up Python 3.12
uses: actions/setup-python@v5
with:
python-version: '3.12'
cache: 'pip'
- name: Install praisonaiagents
run: |
python -m pip install --upgrade pip
pip install -e src/praisonai-agents
- name: Check instantiation threshold
run: |
cd src/praisonai-agents
python benchmarks/simple_benchmark.py --skip-crewai --json --fail-over-threshold > simple.json
python - <<'PY' >> "$GITHUB_STEP_SUMMARY"
import json
data = json.load(open('simple.json'))
budget = json.load(open('benchmarks/thresholds.json'))['praisonai_instantiation_us_max']
print(f"PraisonAI instantiation: {data['results']['PraisonAI']:.2f} us (budget {budget} us)")
PY
benchmark:
if: github.event_name == 'workflow_dispatch'
runs-on: ubuntu-latest
timeout-minutes: 30
@@ -44,7 +87,8 @@ jobs:
python benchmarks/import_time.py 2>&1 | tee import_time_output.txt
# Extract import time
IMPORT_TIME=$(grep -oP 'Median:\s+\K[\d.]+' import_time_output.txt || echo "0")
IMPORT_TIME=$(grep -oP 'Median:\s+\K[\d.]+' import_time_output.txt | head -1)
[ -n "$IMPORT_TIME" ] || { echo "::error::Could not parse import time"; exit 1; }
echo "import_time=$IMPORT_TIME" >> $GITHUB_OUTPUT
# Check threshold (200ms)
@@ -66,7 +110,8 @@ jobs:
python benchmarks/memory_usage.py 2>&1 | tee memory_output.txt
# Extract peak memory
PEAK_MEMORY=$(grep -oP 'Peak:\s+\K[\d.]+' memory_output.txt || echo "0")
PEAK_MEMORY=$(grep -oP 'Peak:\s+\K[\d.]+' memory_output.txt | head -1)
[ -n "$PEAK_MEMORY" ] || { echo "::error::Could not parse peak memory"; exit 1; }
echo "peak_memory=$PEAK_MEMORY" >> $GITHUB_OUTPUT
# Check threshold (30MB)
@@ -83,10 +128,8 @@ jobs:
id: simple
run: |
cd src/praisonai-agents
python benchmarks/simple_benchmark.py 2>&1 | tee simple_output.txt
# Extract PraisonAI time
PRAISON_TIME=$(grep -oP 'PraisonAI\s+\K[\d.]+' simple_output.txt | head -1 || echo "0")
python benchmarks/simple_benchmark.py --json > simple.json
PRAISON_TIME=$(python -c "import json; print(f\"{json.load(open('simple.json'))['results']['PraisonAI']:.2f}\")")
echo "praison_time=$PRAISON_TIME" >> $GITHUB_OUTPUT
- name: Run Tools Benchmark
@@ -96,7 +139,8 @@ jobs:
python benchmarks/tools_benchmark.py 2>&1 | tee tools_output.txt
# Extract PraisonAI time with tools
PRAISON_TOOLS_TIME=$(grep -oP 'PraisonAI\s+\K[\d.]+' tools_output.txt | head -1 || echo "0")
PRAISON_TOOLS_TIME=$(grep -oP 'PraisonAI\s+\K[\d.]+' tools_output.txt | head -1)
[ -n "$PRAISON_TOOLS_TIME" ] || { echo "::error::Could not parse tools benchmark"; exit 1; }
echo "praison_tools_time=$PRAISON_TOOLS_TIME" >> $GITHUB_OUTPUT
- name: Run Parameter Impact Benchmark
@@ -133,7 +177,7 @@ jobs:
echo "|--------|-------|-----------|--------|" >> $GITHUB_STEP_SUMMARY
echo "| Import Time | ${{ steps.import_time.outputs.import_time }}ms | <200ms | ${{ steps.import_time.outputs.import_passed == 'true' && '✅' || '❌' }} |" >> $GITHUB_STEP_SUMMARY
echo "| Peak Memory | ${{ steps.memory.outputs.peak_memory }}MB | <30MB | ${{ steps.memory.outputs.memory_passed == 'true' && '✅' || '❌' }} |" >> $GITHUB_STEP_SUMMARY
echo "| Agent Instantiation | ${{ steps.simple.outputs.praison_time }}μs | - | ℹ️ |" >> $GITHUB_STEP_SUMMARY
echo "| Agent Instantiation | ${{ steps.simple.outputs.praison_time }}μs | see thresholds.json | ℹ️ |" >> $GITHUB_STEP_SUMMARY
echo "| Agent with Tools | ${{ steps.tools.outputs.praison_tools_time }}μs | - | ℹ️ |" >> $GITHUB_STEP_SUMMARY
echo "" >> $GITHUB_STEP_SUMMARY
echo "### Lazy Import Check" >> $GITHUB_STEP_SUMMARY
+5 -1
View File
@@ -558,13 +558,17 @@ jobs:
const o = require(path.join(process.env.GITHUB_WORKSPACE, '.github/scripts/outsider-review.js'));
const { owner, repo } = context.repo;
const since = process.env.STARTED_AT || new Date(0).toISOString();
// Claude's job is over (its token revoked, its processes killed); later
// activity belongs to other runs. Unknown: count everything up to now.
const until = (await o.jobCompletedAt(github, owner, repo, context.runId, 'claude-response')) || new Date().toISOString();
core.info(`Outsider run window: ${since} .. ${until}`);
// Issues and PRs created, edited, commented on or pushed to since then.
const touched = await github.paginate(github.rest.issues.listForRepo, { owner, repo, state: 'all', since, per_page: 100 });
const app = new Set(['praisonai-triage-agent[bot]']);
for (const item of touched) {
let flag = true;
try {
flag = await o.touchedByRun(github, owner, repo, item, since, app);
flag = await o.touchedByRun(github, owner, repo, item, since, app, until);
} catch (err) {
core.warning(`#${item.number}: ${err.message}; flagging it to be safe`);
}
+5 -5
View File
@@ -96,11 +96,11 @@ jobs:
--head "$branch" \
--title "$msg" \
--body "$(cat <<EOF
Automated sync from [\`MervinPraison/PraisonAI\`](https://github.com/MervinPraison/PraisonAI) \`${SOURCE_PATH}/\` into the npm mirror repo.
Automated sync from [\`MervinPraison/PraisonAI\`](https://github.com/MervinPraison/PraisonAI) \`${SOURCE_PATH}/\` into the npm mirror repo.
Canonical development is in the monorepo; this PR updates the public \`praisonai-js\` checkout.
Canonical development is in the monorepo; this PR updates the public \`praisonai-js\` checkout.
Excludes from rsync source: \`node_modules/\`, \`dist/\`, \`coverage/\`. \`.github/\` on praisonai-js is unchanged.
EOF
)"
Excludes from rsync source: \`node_modules/\`, \`dist/\`, \`coverage/\`. \`.github/\` on praisonai-js is unchanged.
EOF
)"
echo "Opened PR on ${TARGET_REPO} (${branch} → ${TARGET_BRANCH})."
+1 -1
View File
@@ -256,7 +256,7 @@ jobs:
"
/tmp/praisonai-browser-standalone/bin/praisonai-browser --help
bash "$GITHUB_WORKSPACE/scripts/check_c11_browser_imports.sh"
python -m pytest src/praisonai-browser/tests/browser/test_protocol_compliance.py -q -m "not network"
python -m pytest src/praisonai-browser/tests/browser/test_protocol_compliance.py src/praisonai-browser/tests/browser/test_server.py -q -m "not network"
- name: Standalone praisonai-mcp smoke (C12)
run: |
+6 -4
View File
@@ -1002,11 +1002,13 @@ agent.start('Write a movie script about a robot in Mars');
## ⚡ Performance
PraisonAI is built for speed, with agent instantiation in around 14μs. This reduces overhead, improves responsiveness, and helps multi-agent systems scale efficiently in real-world production workloads.
PraisonAI keeps framework overhead microsecond-class: creating an `Agent` (one tool, no LLM call) averaged **46 μs** in our 2026-10-15 run, so per-agent setup is negligible next to model latency in multi-agent workloads.
| Performance Metric | PraisonAI |
|--------------------|-----------|
| Avg Instantiation Time | **14 μs** |
| Performance Metric | PraisonAI | Measured |
|--------------------|-----------|----------|
| Avg Agent instantiation (1 tool, 1000 iterations) | **46 μs** | 2026-10-15 · praisonaiagents 1.7.12 |
Methodology, comparisons and how to reproduce: [benchmarks/BENCHMARK_RESULTS.md](https://github.com/MervinPraison/PraisonAI/blob/main/src/praisonai-agents/benchmarks/BENCHMARK_RESULTS.md) · [docs](https://docs.praison.ai/docs/developers/benchmarks)
---
@@ -5,7 +5,6 @@ FastContext provides rapid code search capabilities.
Use it directly from the context module for code search.
FastContext provides:
- 10-20x faster code search than traditional methods
- Parallel execution (8 concurrent searches)
- Automatic caching for repeated queries
+1 -1
View File
@@ -2,7 +2,7 @@
Example 1: Basic Fast Context Usage
This example demonstrates the basic usage of Fast Context for rapid code search.
Fast Context is 10-20x faster than traditional sequential search because it:
Fast Context speeds up code search because it:
- Executes up to 8 tool calls in parallel
- Limits search to 4 turns maximum
- Uses a lightweight model optimized for search
-280
View File
@@ -1,280 +0,0 @@
# Performance Benchmark CI
# Runs import time and instantiation benchmarks on push/PR
# Fails if import time exceeds 200ms threshold
name: Performance Benchmark
on:
push:
branches: [main, develop]
paths:
- 'praisonaiagents/**'
- 'benchmarks/**'
- '.github/workflows/benchmark.yml'
pull_request:
branches: [main, develop]
paths:
- 'praisonaiagents/**'
- 'benchmarks/**'
- '.github/workflows/benchmark.yml'
workflow_dispatch: # Allow manual trigger
jobs:
benchmark:
runs-on: ubuntu-latest
timeout-minutes: 10
steps:
- uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: '3.11'
cache: 'pip'
- name: Install package
run: |
pip install -e .
pip install pytest
- name: Measure Import Time
id: import_time
run: |
# Measure import time (full Agent import)
IMPORT_TIME=$(python -c "
import time
import sys
# Clear any cached modules
for key in list(sys.modules.keys()):
if 'praisonai' in key:
del sys.modules[key]
start = time.perf_counter()
from praisonaiagents import Agent
elapsed = (time.perf_counter() - start) * 1000
print(f'{elapsed:.1f}')
")
echo "import_time=$IMPORT_TIME" >> $GITHUB_OUTPUT
echo "Import time: ${IMPORT_TIME}ms"
- name: Check Import Time Threshold
run: |
IMPORT_TIME=${{ steps.import_time.outputs.import_time }}
THRESHOLD=200
echo "Import time: ${IMPORT_TIME}ms"
echo "Threshold: ${THRESHOLD}ms"
# Compare using bc for floating point
if (( $(echo "$IMPORT_TIME > $THRESHOLD" | bc -l) )); then
echo "❌ FAIL: Import time ${IMPORT_TIME}ms exceeds threshold ${THRESHOLD}ms"
exit 1
else
echo "✅ PASS: Import time ${IMPORT_TIME}ms is within threshold ${THRESHOLD}ms"
fi
- name: Measure Instantiation Time
run: |
python -c "
import time
from praisonaiagents import Agent
# Warmup
for _ in range(5):
Agent(name='Test', output='silent')
# Measure
times = []
for _ in range(50):
start = time.perf_counter()
Agent(name='Test', output='silent')
times.append((time.perf_counter() - start) * 1e6)
avg = sum(times) / len(times)
min_t = min(times)
max_t = max(times)
print(f'Instantiation time:')
print(f' Average: {avg:.2f}μs')
print(f' Min: {min_t:.2f}μs')
print(f' Max: {max_t:.2f}μs')
"
- name: Verify Lazy Imports
run: |
python -c "
import sys
# Clear any cached modules
for key in list(sys.modules.keys()):
if 'praisonai' in key or 'litellm' in key:
del sys.modules[key]
# Import package
import praisonaiagents
# Check heavy deps are NOT loaded
heavy_deps = ['litellm', 'chromadb', 'mem0', 'rich']
loaded = []
for dep in heavy_deps:
if dep in sys.modules:
loaded.append(dep)
if loaded:
print(f'⚠️ WARNING: Heavy deps loaded eagerly: {loaded}')
# Don't fail, just warn - some deps may be needed
else:
print('✅ All heavy deps are lazy loaded')
"
- name: Run Quick Benchmark (if available)
continue-on-error: true
run: |
if [ -f "benchmarks/quick_benchmark.py" ]; then
python benchmarks/quick_benchmark.py --fast || true
else
echo "Quick benchmark not found, skipping"
fi
# Real API benchmark - only runs when API keys are available
# Triggered manually or on main branch with secrets
real-api-benchmark:
runs-on: ubuntu-latest
timeout-minutes: 15
if: github.event_name == 'workflow_dispatch' || (github.ref == 'refs/heads/main' && github.event_name == 'push')
steps:
- uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: '3.11'
cache: 'pip'
- name: Install package
run: |
pip install -e .
pip install pytest
- name: Check API Key Availability
id: check_keys
run: |
if [ -n "${{ secrets.OPENAI_API_KEY }}" ]; then
echo "has_openai=true" >> $GITHUB_OUTPUT
echo "✅ OpenAI API key available"
else
echo "has_openai=false" >> $GITHUB_OUTPUT
echo "⚠️ OpenAI API key not available, skipping real API tests"
fi
- name: Run Real API Benchmark
if: steps.check_keys.outputs.has_openai == 'true'
env:
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
run: |
python -c "
import time
import os
# Verify API key is set
if not os.environ.get('OPENAI_API_KEY'):
print('❌ OPENAI_API_KEY not set')
exit(1)
from praisonaiagents import Agent
# Create a simple agent
agent = Agent(
name='BenchmarkAgent',
instructions='You are a helpful assistant. Be concise.',
llm='gpt-4o-mini',
output='silent'
)
# Warmup run
print('Running warmup...')
agent.chat('Say hello in one word')
# Benchmark runs
print('Running benchmark (3 iterations)...')
times = []
for i in range(3):
start = time.perf_counter()
response = agent.chat('What is 2+2? Answer with just the number.')
elapsed = time.perf_counter() - start
times.append(elapsed)
print(f' Run {i+1}: {elapsed:.2f}s')
avg = sum(times) / len(times)
print(f'')
print(f'Real API Benchmark Results:')
print(f' Average response time: {avg:.2f}s')
print(f' Min: {min(times):.2f}s')
print(f' Max: {max(times):.2f}s')
# Verify response quality
if '4' in response:
print('✅ Response quality check passed')
else:
print(f'⚠️ Unexpected response: {response}')
"
- name: Run Multi-Agent Benchmark
if: steps.check_keys.outputs.has_openai == 'true'
env:
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
run: |
python -c "
import time
import os
from praisonaiagents import Agent, Agents, Task
# Create agents
researcher = Agent(
name='Researcher',
instructions='You research topics. Be brief.',
llm='gpt-4o-mini',
output='silent'
)
writer = Agent(
name='Writer',
instructions='You write summaries. Be brief.',
llm='gpt-4o-mini',
output='silent'
)
# Create tasks
task1 = Task(
description='List 2 facts about Python programming',
agent=researcher,
expected_output='2 brief facts'
)
task2 = Task(
description='Summarize the facts in one sentence',
agent=writer,
expected_output='One sentence summary'
)
# Run workflow
print('Running multi-agent workflow...')
start = time.perf_counter()
agents = Agents(agents=[researcher, writer], tasks=[task1, task2])
result = agents.start()
elapsed = time.perf_counter() - start
print(f'')
print(f'Multi-Agent Benchmark Results:')
print(f' Total workflow time: {elapsed:.2f}s')
print(f' Agents: 2')
print(f' Tasks: 2')
print('✅ Multi-agent workflow completed successfully')
"
@@ -1,70 +1,47 @@
# PraisonAI Agents - Benchmark Results
**Generated:** 2026-06-17
**Methodology:** 10 warmup runs + 1000 measured iterations (`performance_benchmark.py` config)
**Config:** `Agent(name=..., instructions=..., model="gpt-4o-mini", tools=[get_weather], output="silent")`
**Generated:** 2026-10-15 (single run of `performance_benchmark.py`)
**Methodology:** 10 warmup runs + 1000 measured iterations per framework
**Setup (every framework):** one agent, model `gpt-4o-mini`, one `get_weather` tool, no LLM call. Tool wrapping (`function_tool`, `@tool`, ...) happens inside the timed call. Frameworks that were not installed are omitted.
## Agent Instantiation Time
| Framework | Avg Time (μs) | Min (μs) | Median (μs) | Relative |
|-----------|---------------|----------|-------------|----------|
| OpenAI Agents SDK | 5.05 | — | — | 1.00x (fastest) |
| Agno | 5.27 | — | — | 1.04x |
| **PraisonAI** | **13.62** | **12.58** | **12.96** | **2.69x** |
| PraisonAI (LiteLLM) | 14.49 | 13.37 | 13.71 | 2.86x |
### PraisonAI detail (1000 iter)
| Variant | Avg (μs) | Min (μs) | Median (μs) | Std (μs) |
|---------|----------|----------|-------------|----------|
| With tools (`gpt-4o-mini`) | 13.62 | 12.58 | 12.96 | 2.73 |
| With tools (`openai/gpt-4o-mini`) | 14.49 | 13.37 | 13.71 | 4.14 |
| Minimal, no tools | 13.40 | — | — | — |
## Historical baseline
| Source | Config | Avg (μs) | Notes |
|--------|--------|----------|-------|
| `performance_benchmark.py` (Dec 2025) | tools + 1000 iter | **3.20** | Pre-init-bloat baseline |
| `simple_benchmark.py` (Dec 2025) | no tools, 100 iter | **3.77** | Saved in prior `BENCHMARK_RESULTS.md` |
| Current (Jun 2026) | tools + 1000 iter | **13.62** | After lazy-init optimisations (was ~31 μs pre-fix) |
| Framework | Avg (μs) | Min (μs) | Median (μs) | Std (μs) | Avg memory (KiB) | Relative |
|-----------|----------|----------|-------------|----------|------------------|----------|
| Agno | 9.51 | 9.16 | 9.33 | 1.00 | 6.83 | 1.00x (fastest) |
| PraisonAI | 45.55 | 42.63 | 44.26 | 4.11 | 20.55 | 4.79x |
| PraisonAI (LiteLLM) | 46.94 | 44.65 | 45.82 | 3.26 | 20.81 | 4.94x |
| OpenAI Agents SDK | 591.02 | 529.26 | 566.69 | 60.62 | 60.19 | 62.18x |
| PydanticAI | 1,021.99 | 882.02 | 935.63 | 1,674.03 | 51.22 | 108x |
| LangGraph | 3,232.91 | 2,836.27 | 3,183.02 | 233.32 | 89.66 | 340x |
## Package Versions
| Package | Version |
|---------|---------|
| PraisonAI | 1.6.60 |
| Agno | 2.4.2 |
| OpenAI Agents SDK | 0.7.0 |
| praisonaiagents | 1.7.12 |
| agno | 3.1.3 |
| pydantic-ai-slim | 2.56.0 |
| openai-agents | 0.23.1 |
| langgraph | 1.2.15 |
## Machine
| Field | Value |
|-------|-------|
| Platform | Linux-6.17.0-1022-azure-x86_64-with-glibc2.39 |
| Processor | x86_64 |
| CPUs | 4 |
| Python | 3.12.3 |
| Load avg (1 min) at start | 0.85 |
## How to Reproduce
PraisonAI-only (fast; avoids CrewAI/LangGraph import hang):
```bash
cd praisonai-agents
PYTHONWARNINGS='ignore::DeprecationWarning' python -c "
import time, statistics
from typing import Literal
from praisonaiagents import Agent
def get_weather(city: Literal['nyc', 'sf']):
return 'cloudy' if city == 'nyc' else 'sunny'
def bench(fn, n=1000, w=10):
for _ in range(w): fn()
times = []
for _ in range(n):
s = time.perf_counter(); fn(); times.append(time.perf_counter()-s)
print(f'avg={statistics.mean(times)*1e6:.2f} μs')
bench(lambda: Agent(name='Test', instructions='Hi', model='gpt-4o-mini', tools=[get_weather], output='silent'))
"
cd src/praisonai-agents
pip install -e . agno openai-agents "pydantic-ai-slim[openai]" langgraph langchain-openai crewai
# OpenAI-backed constructors need a key to exist; no request is made
OPENAI_API_KEY=sk-dummy PYTHONWARNINGS='ignore::DeprecationWarning' python benchmarks/performance_benchmark.py --save --iterations 1000 --warmup 10 --only praisonai praisonai_litellm agno pydantic openai_agents langgraph
```
Full multi-framework benchmark:
```bash
cd praisonai-agents
PYTHONWARNINGS='ignore::DeprecationWarning' python benchmarks/performance_benchmark.py --save
```
Older, superseded result files live in [`archive/`](archive/).
@@ -6,7 +6,7 @@
## Summary
Instantiation regressed from **~3.2 μs** (Dec 2025) to **~31 μs** (~10×). After fixes, steady-state is **~16 μs** (~2× historical). PraisonAI is competitive with Agno (~5 μs) on warmed benchmarks using `model=`.
Instantiation regressed from **~3.2 μs** (Dec 2025) to **~31 μs** (~10×). After fixes, steady-state is **~16 μs** (~2× historical). Agno measured ~5 μs on the same machine at the time, so PraisonAI was still ~3× slower; see `BENCHMARK_RESULTS.md` for current numbers.
## Before vs After Fixes
@@ -53,10 +53,45 @@ Instantiation regressed from **~3.2 μs** (Dec 2025) to **~31 μs** (~10×). Aft
The residual ~13 μs is primarily **`Agent.__init__` size** (~1200 lines of consolidated config resolution added since v0.1.5). Further optimisation requires a dedicated slim fast-path for `Agent(name=..., output="silent")` without full config object churn.
## Re-measure (2026-10-15)
Quiet machine (GitHub-hosted `ubuntu` x86_64 runner, 4 vCPU, load 0.77, Python 3.12.3). Same config as the
June run (`performance_benchmark.py`: one tool, 10–20 warmup + 1000 iterations). Both versions were run on the
same machine, so the two columns are directly comparable. The June numbers in `BENCHMARK_RESULTS.md` came from a
faster machine.
| Code | Avg (μs) | Min (μs) | Median (μs) |
|------|----------|----------|-------------|
| praisonaiagents 1.6.60 (`709a03878`) | 20.65 | 19.85 | 20.30 |
| `main` @ `a84e42aa9` (1.7.12 dev) | 47.92 | 45.71 | 47.11 |
The slowdown is real: **~2.3×** on identical hardware. Median, sampled along `main` first-parent history:
| Date | Commit | Median (μs) |
|------|--------|-------------|
| 2026-06-17 | `709a03878` (1.6.60) | 20.3 |
| 2026-06-24 | `bb2d35ba6` | 24.2 |
| 2026-07-01 | `f45c49b32` | 26.7 |
| 2026-07-15 | `b1086c275` | 30.5 |
| 2026-08-05 | `aadb62acf` | 29.7 |
| 2026-08-19 | `d32fe29b4` | 34.0 |
| 2026-08-26 | `7746520df` | **3,063** (transient ms-scale regression, since fixed) |
| 2026-09-03 | `17d8726c3` | **3,012** |
| 2026-09-10 | `8d2b4cdbc` | 48.3 |
| 2026-10-02 | `a2b52f669` | 60.2 |
| 2026-10-14 | `a84e42aa9` | 47.2 |
`git bisect` with a 33 μs cut-off has no single culprit: it lands on a wrapper-only commit, and neighbouring
commits differ by less than the noise. The cost is gradual creep. cProfile on `main` puts ~43% of construction
time in the body of `Agent.__init__`, and no callee costs more than ~10%. The fix is the slim construction path
described above, tracked in #5835. A ~3 ms regression shipped for two weeks unnoticed; the PR gate
in `.github/workflows/benchmark.yml` (`simple_benchmark.py --json --fail-over-threshold`, 150 μs budget) now
catches regressions of that size.
## How to reproduce
```bash
cd praisonai-agents
cd src/praisonai-agents
PYTHONWARNINGS='ignore::DeprecationWarning' python benchmarks/simple_benchmark.py
PYTHONWARNINGS='ignore::DeprecationWarning' python benchmarks/performance_benchmark.py
PYTHONWARNINGS='ignore::DeprecationWarning' python benchmarks/performance_benchmark.py --only praisonai
```
@@ -1,3 +1,5 @@
> **Superseded.** Historical result from 2026-01-13, kept for reference only. Package versions and methodology differ from current runs; do not cite. Current results: [../BENCHMARK_RESULTS.md](../BENCHMARK_RESULTS.md).
# PraisonAI Agents - Real Execution Benchmark
**Generated:** 2026-01-13 06:07:28
@@ -1,3 +1,5 @@
> **Superseded.** Historical result from 2026-01-13, kept for reference only. Package versions and methodology differ from current runs; do not cite. Current results: [../BENCHMARK_RESULTS.md](../BENCHMARK_RESULTS.md).
# PraisonAI Agents - Real-World Execution Benchmark
**Generated:** 2026-01-13 14:42:59
@@ -1,3 +1,5 @@
> **Superseded.** Historical result from 2025-12-18, kept for reference only. Package versions and methodology differ from current runs; do not cite. Current results: [../BENCHMARK_RESULTS.md](../BENCHMARK_RESULTS.md).
# PraisonAI Agents - Benchmark Results
**Generated:** 2025-12-18 13:23:43
@@ -1,3 +1,5 @@
> **Superseded.** Historical result from 2026-01-23, kept for reference only. Package versions and methodology differ from current runs; do not cite. Current results: [../BENCHMARK_RESULTS.md](../BENCHMARK_RESULTS.md).
# PraisonAI Agents - Tools Benchmark Results
**Generated:** 2026-01-23 11:50:18
@@ -6,6 +6,8 @@ multiple AI agent frameworks using industry-standard methodology.
Usage:
python benchmarks/performance_benchmark.py
python benchmarks/performance_benchmark.py --save # rewrite BENCHMARK_RESULTS.md
python benchmarks/performance_benchmark.py --only praisonai
"""
import gc
@@ -134,39 +136,14 @@ def run_benchmark(func: Callable, name: str, iterations: int = 1000, warmup: int
# ============================================================================
# Framework Imports
# Framework factories
#
# Each builder imports its framework lazily and returns a zero-arg callable
# that creates one agent with one tool. A missing framework raises ImportError
# and is skipped, so the script runs with only praisonaiagents installed.
# ============================================================================
from praisonaiagents import Agent as PraisonAgent
from agno.agent import Agent as AgnoAgent
from agno.models.openai import OpenAIChat
# Additional frameworks for comparison
try:
from pydantic_ai import Agent as PydanticAgent
PYDANTIC_AVAILABLE = True
except ImportError:
PYDANTIC_AVAILABLE = False
try:
from crewai.agent import Agent as CrewAgent
from crewai.tools import tool as crewai_tool
CREWAI_AVAILABLE = True
except ImportError:
CREWAI_AVAILABLE = False
try:
from langchain_core.tools import tool as langchain_tool
from langchain_openai import ChatOpenAI
from langgraph.prebuilt import create_react_agent
LANGGRAPH_AVAILABLE = True
except ImportError:
LANGGRAPH_AVAILABLE = False
try:
from agents import Agent as OpenAIAgent, function_tool
OPENAI_AGENTS_AVAILABLE = True
except ImportError:
OPENAI_AGENTS_AVAILABLE = False
INSTRUCTIONS = "Be concise, reply with one sentence."
def get_weather(city: Literal["nyc", "sf"]):
@@ -180,195 +157,181 @@ def get_weather(city: Literal["nyc", "sf"]):
tools = [get_weather]
def instantiate_praisonai_agent():
"""Instantiate a PraisonAI agent with one tool."""
return PraisonAgent(
name="Test Agent",
instructions="Be concise, reply with one sentence.",
model="gpt-4o-mini",
tools=tools,
output="silent"
def build_praisonai():
from praisonaiagents import Agent as PraisonAgent
return lambda: PraisonAgent(
name="Test Agent", instructions=INSTRUCTIONS, model="gpt-4o-mini",
tools=tools, output="silent",
)
def instantiate_praisonai_litellm_agent():
"""Instantiate a PraisonAI agent with LiteLLM backend."""
return PraisonAgent(
name="Test Agent",
instructions="Be concise, reply with one sentence.",
model="openai/gpt-4o-mini",
tools=tools,
output="silent"
def build_praisonai_litellm():
from praisonaiagents import Agent as PraisonAgent
return lambda: PraisonAgent(
name="Test Agent", instructions=INSTRUCTIONS, model="openai/gpt-4o-mini",
tools=tools, output="silent",
)
def instantiate_agno_agent():
"""Instantiate an Agno agent with one tool."""
return AgnoAgent(model=OpenAIChat(id="gpt-4o-mini"), tools=tools)
def build_agno():
from agno.agent import Agent as AgnoAgent
from agno.models.openai import OpenAIChat
return lambda: AgnoAgent(model=OpenAIChat(id="gpt-4o-mini"), instructions=INSTRUCTIONS, tools=tools)
def instantiate_pydantic_agent():
"""Instantiate a PydanticAI agent with one tool."""
agent = PydanticAgent("openai:gpt-4o-mini", system_prompt="Be concise, reply with one sentence.")
@agent.tool_plain
def get_weather_pydantic(city: Literal["nyc", "sf"]):
"""Use this to get weather information."""
if city == "nyc":
return "It might be cloudy in nyc"
elif city == "sf":
return "It's always sunny in sf"
return agent
def build_pydantic():
from pydantic_ai import Agent as PydanticAgent
def create():
agent = PydanticAgent("openai:gpt-4o-mini", system_prompt=INSTRUCTIONS)
agent.tool_plain(get_weather)
return agent
return create
def instantiate_openai_agents():
"""Instantiate an OpenAI Agents SDK agent with one tool."""
return OpenAIAgent(
name="Test agent",
instructions="Be concise, reply with one sentence.",
model="gpt-4o-mini",
def build_openai_agents():
from agents import Agent as OpenAIAgent, function_tool
return lambda: OpenAIAgent(
name="Test agent", instructions=INSTRUCTIONS, model="gpt-4o-mini",
tools=[function_tool(get_weather)],
)
def instantiate_langgraph_agent():
"""Instantiate a LangGraph agent with one tool."""
@langchain_tool
def get_weather_lg(city: Literal["nyc", "sf"]):
"""Use this to get weather information."""
if city == "nyc":
return "It might be cloudy in nyc"
elif city == "sf":
return "It's always sunny in sf"
return create_react_agent(model=ChatOpenAI(model="gpt-4o-mini"), tools=[get_weather_lg])
def instantiate_crewai_agent():
"""Instantiate a CrewAI agent with one tool."""
@crewai_tool("Weather Tool")
def get_weather_crew(city: Literal["nyc", "sf"]):
"""Use this to get weather information."""
if city == "nyc":
return "It might be cloudy in nyc"
elif city == "sf":
return "It's always sunny in sf"
return CrewAgent(
llm="gpt-4o-mini",
role="Test Agent",
goal="Be concise, reply with one sentence.",
tools=[get_weather_crew],
backstory="Test",
def build_langgraph():
from langchain_core.tools import tool as langchain_tool
from langchain_openai import ChatOpenAI
from langgraph.prebuilt import create_react_agent
return lambda: create_react_agent(
model=ChatOpenAI(model="gpt-4o-mini"), tools=[langchain_tool(get_weather)],
)
def get_package_versions():
"""Get version numbers for all benchmarked packages."""
def build_crewai():
from crewai.agent import Agent as CrewAgent
from crewai.tools import tool as crewai_tool
def create():
return CrewAgent(
llm="gpt-4o-mini", role="Test Agent", goal=INSTRUCTIONS,
tools=[crewai_tool("Weather Tool")(get_weather)], backstory="Test",
)
return create
# key -> (display name, PyPI package, builder)
FRAMEWORKS = {
'praisonai': ('PraisonAI', 'praisonaiagents', build_praisonai),
'praisonai_litellm': ('PraisonAI (LiteLLM)', 'praisonaiagents', build_praisonai_litellm),
'agno': ('Agno', 'agno', build_agno),
'pydantic': ('PydanticAI', 'pydantic-ai-slim', build_pydantic),
'openai_agents': ('OpenAI Agents SDK', 'openai-agents', build_openai_agents),
'langgraph': ('LangGraph', 'langgraph', build_langgraph),
'crewai': ('CrewAI', 'crewai', build_crewai),
}
def get_package_versions(keys=None):
"""Get version numbers for benchmarked packages."""
from importlib.metadata import version as get_version
packages = {
'PraisonAI': 'praisonaiagents',
'Agno': 'agno',
'PydanticAI': 'pydantic-ai',
'OpenAI Agents SDK': 'openai-agents',
'LangGraph': 'langgraph',
'CrewAI': 'crewai'
}
versions = {}
for display_name, pkg_name in packages.items():
for key in keys or FRAMEWORKS:
display_name, pkg_name, _ = FRAMEWORKS[key]
if pkg_name in versions:
continue
try:
versions[display_name] = get_version(pkg_name)
versions[pkg_name] = get_version(pkg_name)
except Exception:
versions[display_name] = 'not installed'
versions[pkg_name] = 'not installed'
return versions
def save_benchmark_results(results: dict, baseline):
"""Save benchmark results to BENCHMARK_RESULTS.md and update README.md"""
def get_machine_info() -> dict:
"""Describe the machine so results are traceable."""
import os
import re
from datetime import datetime
benchmarks_dir = os.path.dirname(__file__)
filepath = os.path.join(benchmarks_dir, 'BENCHMARK_RESULTS.md')
# Get package versions
versions = get_package_versions()
display_names = {
'agno': 'Agno',
'praisonai': 'PraisonAI',
'praisonai_litellm': 'PraisonAI (LiteLLM)',
'pydantic': 'PydanticAI',
'openai_agents': 'OpenAI Agents SDK',
'langgraph': 'LangGraph',
'crewai': 'CrewAI'
import platform
load = os.getloadavg()[0] if hasattr(os, 'getloadavg') else None
return {
'Platform': platform.platform(),
'Processor': platform.processor() or platform.machine(),
'CPUs': os.cpu_count(),
'Python': platform.python_version(),
'Load avg (1 min) at start': f'{load:.2f}' if load is not None else 'n/a',
}
# Build table rows
table_rows = []
sorted_results = sorted(results.items(), key=lambda x: x[1].avg_run_time)
for name, result in sorted_results:
display_name = display_names.get(name, name)
avg_us = result.avg_run_time * 1e6
ratio = result.avg_run_time / baseline.avg_run_time if baseline.avg_run_time > 0 else 0
if name == 'praisonai':
table_rows.append(f'| **{display_name}** | **{avg_us:.2f}** | **1.00x (fastest)** |')
else:
avg_str = f'{avg_us:,.2f}' if avg_us >= 1000 else f'{avg_us:.2f}'
ratio_str = f'{ratio:,.0f}x' if ratio >= 100 else f'{ratio:.2f}x'
table_rows.append(f'| {display_name} | {avg_str} | {ratio_str} |')
# Save to BENCHMARK_RESULTS.md
def rank_results(results: dict) -> list:
"""Return ``(key, result, ratio)`` sorted fastest first, ratio vs the fastest."""
ranked = sorted(results.items(), key=lambda x: x[1].avg_run_time)
fastest = ranked[0][1].avg_run_time if ranked else 0
return [(k, r, r.avg_run_time / fastest if fastest > 0 else 0) for k, r in ranked]
def save_benchmark_results(results: dict, machine: dict, iterations: int, warmup: int,
filename: str = 'BENCHMARK_RESULTS.md') -> str:
"""Write every framework's results from this single run to ``BENCHMARK_RESULTS.md``."""
import os
from datetime import datetime
filepath = os.path.join(os.path.dirname(__file__), filename)
versions = get_package_versions(results.keys())
def us(seconds):
value = seconds * 1e6
return f'{value:,.2f}' if value >= 1000 else f'{value:.2f}'
lines = [
'# PraisonAI Agents - Benchmark Results', '',
f'**Generated:** {datetime.now().strftime("%Y-%m-%d")} (single run of `performance_benchmark.py`)',
f'**Methodology:** {warmup} warmup runs + {iterations} measured iterations per framework',
f'**Setup (every framework):** one agent, model `gpt-4o-mini`, one `get_weather` tool, '
'no LLM call. Tool wrapping (`function_tool`, `@tool`, ...) happens inside the timed call. '
'Frameworks that were not installed are omitted.',
'', '## Agent Instantiation Time', '',
'| Framework | Avg (μs) | Min (μs) | Median (μs) | Std (μs) | Avg memory (KiB) | Relative |',
'|-----------|----------|----------|-------------|----------|------------------|----------|',
]
for rank, (key, r, ratio) in enumerate(rank_results(results)):
relative = '1.00x (fastest)' if rank == 0 else (f'{ratio:,.0f}x' if ratio >= 100 else f'{ratio:.2f}x')
lines.append(
f'| {FRAMEWORKS[key][0]} | {us(r.avg_run_time)} | {us(r.min_run_time)} | '
f'{us(r.median_run_time)} | {us(r.std_dev_run_time)} | '
f'{r.avg_memory_usage * 1024:.2f} | {relative} |'
)
lines += ['', '## Package Versions', '', '| Package | Version |', '|---------|---------|']
lines += [f'| {pkg} | {ver} |' for pkg, ver in versions.items()]
lines += ['', '## Machine', '', '| Field | Value |', '|-------|-------|']
lines += [f'| {k} | {v} |' for k, v in machine.items()]
lines += [
'', '## How to Reproduce', '', '```bash', 'cd src/praisonai-agents',
'pip install -e . agno openai-agents "pydantic-ai-slim[openai]" langgraph langchain-openai crewai',
"# OpenAI-backed constructors need a key to exist; no request is made",
"OPENAI_API_KEY=sk-dummy PYTHONWARNINGS='ignore::DeprecationWarning' "
f"python benchmarks/performance_benchmark.py --save --iterations {iterations} "
f"--warmup {warmup} --only {' '.join(results)}",
'```', '',
'Older, superseded result files live in [`archive/`](archive/).', '',
]
with open(filepath, 'w') as f:
f.write('# PraisonAI Agents - Benchmark Results\n\n')
f.write(f'**Generated:** {datetime.now().strftime("%Y-%m-%d %H:%M:%S")}\n')
f.write('**Methodology:** 10 warmup runs + 1000 measured iterations\n\n')
f.write('## Agent Instantiation Time\n\n')
f.write('| Framework | Avg Time (μs) | Relative |\n')
f.write('|-----------|---------------|----------|\n')
f.write('\n'.join(table_rows) + '\n')
f.write('\n## Package Versions\n\n')
f.write('| Package | Version |\n')
f.write('|---------|--------|\n')
for pkg, ver in versions.items():
f.write(f'| {pkg} | {ver} |\n')
f.write('\n## How to Reproduce\n\n')
f.write('```bash\n')
f.write('cd praisonai-agents\n')
f.write('python benchmarks/performance_benchmark.py\n')
f.write('```\n')
f.write('\n'.join(lines))
print(f'\nResults saved to: {filepath}')
# Update README.md
readme_path = os.path.join(benchmarks_dir, '..', '..', '..', 'README.md')
if os.path.exists(readme_path):
with open(readme_path, 'r') as f:
content = f.read()
new_table = '''| Framework | Avg Time (μs) | Relative |
|-----------|---------------|----------|
''' + '\n'.join(table_rows)
pattern = r'(\| Framework \| Avg Time \(μs\) \| Relative \|\n\|[-|]+\|\n)(\|[^\n]+\|\n)+'
if re.search(pattern, content):
content = re.sub(pattern, new_table + '\n', content)
with open(readme_path, 'w') as f:
f.write(content)
print(f'README.md updated: {readme_path}')
return filepath
if __name__ == "__main__":
def main():
parser = argparse.ArgumentParser(description='PraisonAI Agents - Comprehensive Performance Benchmark')
parser.add_argument('--save', action='store_true', help='Save results to file')
parser.add_argument('--save', action='store_true', help='Save results to BENCHMARK_RESULTS.md')
parser.add_argument('--iterations', type=int, default=1000, help='Measured iterations (default: 1000)')
parser.add_argument('--warmup', type=int, default=10, help='Warmup runs (default: 10)')
parser.add_argument('--only', nargs='+', choices=list(FRAMEWORKS), help='Benchmark only these frameworks')
args = parser.parse_args()
if args.iterations <= 0 or args.warmup < 0:
parser.error('--iterations must be > 0 and --warmup must be >= 0')
machine = get_machine_info()
print("="*70)
print("PraisonAI Agents - Comprehensive Performance Benchmark")
print("="*70)
@@ -376,156 +339,48 @@ if __name__ == "__main__":
print(" 1. Agent instantiation time (how fast an agent object is created)")
print(" 2. Memory footprint (how much memory an agent uses)")
print("\nMethodology:")
print(" - 10 warmup runs (not measured)")
print(" - 1000 measured iterations")
print(f" - {args.warmup} warmup runs (not measured)")
print(f" - {args.iterations} measured iterations")
print(" - tracemalloc for memory measurement")
print(" - Baseline subtraction for accuracy")
print("\nFrameworks available:")
print(" - PraisonAI: ✅")
print(f" - Agno: ✅")
print(f" - PydanticAI: {'✅' if PYDANTIC_AVAILABLE else '❌'}")
print(f" - OpenAI Agents SDK: {'✅' if OPENAI_AGENTS_AVAILABLE else '❌'}")
print(f" - LangGraph: {'✅' if LANGGRAPH_AVAILABLE else '❌'}")
print(f" - CrewAI: {'✅' if CREWAI_AVAILABLE else '❌'}")
print("\nMachine:")
for k, v in machine.items():
print(f" - {k}: {v}")
results = {}
# Test PraisonAI first
try:
print("\n" + "="*70)
print("Testing PraisonAI Agents")
print("="*70)
results['praisonai'] = run_benchmark(
instantiate_praisonai_agent,
"PraisonAI",
iterations=1000,
warmup=10
)
except Exception as e:
print(f"Error testing PraisonAI: {e}")
try:
print("\n" + "="*70)
print("Testing PraisonAI Agents (LiteLLM)")
print("="*70)
results['praisonai_litellm'] = run_benchmark(
instantiate_praisonai_litellm_agent,
"PraisonAI (LiteLLM)",
iterations=1000,
warmup=10
)
except Exception as e:
print(f"Error testing PraisonAI (LiteLLM): {e}")
# Test Agno
try:
print("\n" + "="*70)
print("Testing Agno")
print("="*70)
results['agno'] = run_benchmark(
instantiate_agno_agent,
"Agno",
iterations=1000,
warmup=10
)
except Exception as e:
print(f"Error testing Agno: {e}")
# Test PydanticAI
if PYDANTIC_AVAILABLE:
for key in args.only or FRAMEWORKS:
display_name, _, builder = FRAMEWORKS[key]
try:
print("\n" + "="*70)
print("Testing PydanticAI")
print("="*70)
results['pydantic'] = run_benchmark(
instantiate_pydantic_agent,
"PydanticAI Agent Instantiation",
iterations=1000,
warmup=10
)
except Exception as e:
print(f"Error testing PydanticAI: {e}")
# Test OpenAI Agents SDK
if OPENAI_AGENTS_AVAILABLE:
create = builder()
except ImportError:
print(f"\nSkipping {display_name}: not installed")
continue
try:
print("\n" + "="*70)
print("Testing OpenAI Agents SDK")
print("="*70)
results['openai_agents'] = run_benchmark(
instantiate_openai_agents,
"OpenAI Agents SDK Instantiation",
iterations=1000,
warmup=10
)
results[key] = run_benchmark(create, display_name, iterations=args.iterations, warmup=args.warmup)
except Exception as e:
print(f"Error testing OpenAI Agents SDK: {e}")
# Test LangGraph
if LANGGRAPH_AVAILABLE:
try:
print("\n" + "="*70)
print("Testing LangGraph")
print("="*70)
results['langgraph'] = run_benchmark(
instantiate_langgraph_agent,
"LangGraph Agent Instantiation",
iterations=1000,
warmup=10
)
except Exception as e:
print(f"Error testing LangGraph: {e}")
# Test CrewAI
if CREWAI_AVAILABLE:
try:
print("\n" + "="*70)
print("Testing CrewAI")
print("="*70)
results['crewai'] = run_benchmark(
instantiate_crewai_agent,
"CrewAI Agent Instantiation",
iterations=1000,
warmup=10
)
except Exception as e:
print(f"Error testing CrewAI: {e}")
print(f"Error testing {display_name}: {e}")
if not results:
print("\nNo framework could be benchmarked.")
return 1
# Final Comparison Table
print("\n" + "="*70)
print("RESULTS SUMMARY")
print("="*70)
if 'praisonai' in results:
baseline = results['praisonai']
print(f"\n{'Framework':<25} {'Avg Time (μs)':<15} {'Avg Memory (KiB)':<18} {'Relative':<15}")
print("-" * 73)
# Sort by time
sorted_results = sorted(results.items(), key=lambda x: x[1].avg_run_time)
for name, result in sorted_results:
time_ratio = result.avg_run_time / baseline.avg_run_time if baseline.avg_run_time > 0 else 0
time_str = f"{time_ratio:.2f}x"
display_name = {
'agno': 'Agno',
'praisonai': 'PraisonAI',
'praisonai_litellm': 'PraisonAI (LiteLLM)',
'pydantic': 'PydanticAI',
'openai_agents': 'OpenAI Agents SDK',
'langgraph': 'LangGraph',
'crewai': 'CrewAI'
}.get(name, name)
print(f"{display_name:<25} {result.avg_run_time*1e6:<15.2f} {result.avg_memory_usage*1024:<18.2f} {time_str:<15}")
print("\n" + "="*70)
# Save results to files
if args.save:
save_benchmark_results(results, baseline)
else:
print('\nResults not saved (use --save flag to save results to file)')
print(f"\n{'Framework':<25} {'Avg Time (μs)':<15} {'Median (μs)':<13} {'Avg Memory (KiB)':<18} {'Relative':<15}")
print("-" * 86)
for key, result, ratio in rank_results(results):
print(f"{FRAMEWORKS[key][0]:<25} {result.avg_run_time*1e6:<15.2f} {result.median_run_time*1e6:<13.2f} "
f"{result.avg_memory_usage*1024:<18.2f} {ratio:.2f}x")
print("\n" + "="*70)
if args.save:
save_benchmark_results(results, machine, args.iterations, args.warmup)
else:
print('\nResults not saved (use --save flag to save results to file)')
return 0
if __name__ == "__main__":
raise SystemExit(main())
@@ -188,15 +188,17 @@ def get_package_versions():
return versions
def save_results(results: dict, filename: str = 'BENCHMARK_RESULTS.md', *,
def save_results(results: dict, filename: str = 'SIMPLE_BENCHMARK_RESULTS.md', *,
iterations: int = ITERATIONS, log=print):
"""Save benchmark results to a markdown file and update README.
"""Save benchmark results to a markdown file.
Writes ``SIMPLE_BENCHMARK_RESULTS.md`` by default so the curated
``BENCHMARK_RESULTS.md`` (from ``performance_benchmark.py``) is never
overwritten.
Status messages are emitted via ``log`` (defaults to ``print``). Pass a
stderr logger when stdout must stay machine-readable (e.g. ``--json``).
"""
import os
import re
from datetime import datetime
filepath = os.path.join(os.path.dirname(__file__), filename)
@@ -217,7 +219,6 @@ def save_results(results: dict, filename: str = 'BENCHMARK_RESULTS.md', *,
ratio_str = f'{ratio:,.0f}x' if ratio >= 100 else f'{ratio:.2f}x'
table_rows.append(f'| {name} | {avg_str} | {ratio_str} |')
# Save to BENCHMARK_RESULTS.md
with open(filepath, 'w') as f:
f.write('# PraisonAI Agents - Benchmark Results\n\n')
f.write(f'**Generated:** {datetime.now().strftime("%Y-%m-%d %H:%M:%S")}\n')
@@ -239,39 +240,9 @@ def save_results(results: dict, filename: str = 'BENCHMARK_RESULTS.md', *,
f.write('```\n')
log(f'\nResults saved to: {filepath}')
# Also update the main README.md
readme_path = os.path.join(os.path.dirname(__file__), '..', '..', '..', 'README.md')
if os.path.exists(readme_path):
update_readme(readme_path, table_rows, log=log)
return filepath
def update_readme(readme_path: str, table_rows: list, *, log=print):
"""Update the performance section in README.md with latest results."""
import re
with open(readme_path, 'r') as f:
content = f.read()
# Build the new table
new_table = '''| Framework | Avg Time (μs) | Relative |
|-----------|---------------|----------|
''' + '\n'.join(table_rows)
# Pattern to match the performance table
pattern = r'(\| Framework \| Avg Time \(μs\) \| Relative \|\n\|[-|]+\|\n)(\|[^\n]+\|\n)+'
if re.search(pattern, content):
content = re.sub(pattern, new_table + '\n', content)
with open(readme_path, 'w') as f:
f.write(content)
log(f'README.md updated: {readme_path}')
def main():
parser = argparse.ArgumentParser(description='PraisonAI Agents - Performance Benchmark')
parser.add_argument('--save', action='store_true', help='Save results to file')
@@ -1,5 +1,5 @@
{
"praisonai_instantiation_us_max": 450,
"praisonai_instantiation_us_max": 150,
"iterations": 100,
"warmup": 10
}
@@ -457,11 +457,16 @@ class Agent(GoalLoopMixin, SteeringMixin, SandboxMixin, SkillReviewMixin, Unifie
self._wire_deferred_history_callback()
def _ensure_loop_guard(self):
"""Lazy-create loop guard on first tool execution."""
"""Lazy-create loop guard on first tool execution.
Inside a chat turn, returns that turn's own guard (same config) so
concurrent turns on a shared agent do not share per-turn counters."""
if self._loop_guard is None:
from ..escalation.loop_guard import LoopGuard, LoopGuardConfig
self._loop_guard = LoopGuard(LoopGuardConfig(enabled=True))
return self._loop_guard
from .chat_mixin import _current_turn_loop_guard
turn_guard = _current_turn_loop_guard(self)
return turn_guard if turn_guard is not None else self._loop_guard
def _ensure_loop_detector(self):
"""Lazy-create the result-aware tool-loop detector on first tool call.
@@ -6524,7 +6529,8 @@ Summary:"""
# In async context - prefer async hook if available
if hasattr(self._memory_instance, 'aon_memory_write'):
# Schedule async hook without blocking (fire and forget)
asyncio.create_task(self._memory_instance.aon_memory_write(action, memory_type, content, metadata))
from ..utils.async_bridge import spawn_background
spawn_background(self._memory_instance.aon_memory_write(action, memory_type, content, metadata), "aon_memory_write")
elif hasattr(self._memory_instance, 'on_memory_write'):
self._memory_instance.on_memory_write(action, memory_type, content, metadata)
except RuntimeError:
@@ -6,6 +6,7 @@ and response processing. Extracted from agent.py for maintainability.
"""
import asyncio
import contextvars
import os
import re
import time
@@ -24,12 +25,17 @@ logger = logging.getLogger(__name__)
_turns_lock = threading.Lock()
# Per-turn loop guard: ``[agent, LoopGuard | None]`` bound by ``_enter_turn``. Copied
# into tool worker threads with the context, so each turn has its own counters and
# overlapping turns on a shared agent never exhaust each other's limits.
_turn_loop_guard = contextvars.ContextVar("praisonai_turn_loop_guard", default=None)
def _enter_turn(agent, cancel):
"""Mark a ``chat``/``achat`` turn in flight and bind its cancel token to this context."""
with _turns_lock:
agent._turns_in_flight = getattr(agent, "_turns_in_flight", 0) + 1
return _active_turn_cancel.set((agent, cancel))
return _active_turn_cancel.set((agent, cancel)), _turn_loop_guard.set([agent, None])
def _exit_turn(agent, ctx_token):
@@ -38,18 +44,36 @@ def _exit_turn(agent, ctx_token):
return
with _turns_lock:
agent._turns_in_flight = max(getattr(agent, "_turns_in_flight", 1) - 1, 0)
try:
_active_turn_cancel.reset(ctx_token)
except (ValueError, LookupError):
_active_turn_cancel.set(None)
cancel_token, guard_token = ctx_token
for var, token in ((_active_turn_cancel, cancel_token), (_turn_loop_guard, guard_token)):
try:
var.reset(token)
except (ValueError, LookupError):
var.set(None)
def _current_turn_loop_guard(agent):
"""Return this turn's loop guard for ``agent`` (lazily created, sharing the agent
guard's config), or ``None`` when not inside one of ``agent``'s chat turns."""
slot = _turn_loop_guard.get()
if slot is None or slot[0] is not agent:
return None
with _turns_lock:
if slot[1] is None:
from ..escalation.loop_guard import LoopGuard
slot[1] = LoopGuard(agent._loop_guard.config)
return slot[1]
def _reset_loop_guard_for_turn(agent):
"""Reset per-turn loop-guard limits unless another turn on this agent is still
running (the guard is shared; resetting it would restart that turn's limits).
Never force-creates the guard."""
"""Start fresh per-turn loop-guard limits. Inside a chat turn this drops the turn's
own guard; outside one it resets the agent-level guard. Never force-creates a guard."""
slot = _turn_loop_guard.get()
if slot is not None and slot[0] is agent:
slot[1] = None
return
guard = getattr(agent, "_loop_guard", None)
if guard is not None and getattr(agent, "_turns_in_flight", 0) <= 1:
if guard is not None:
guard.reset_turn()
@@ -5291,6 +5315,19 @@ Output MUST be JSON with 'reflection' and 'satisfactory'.
def _start_stream(self, prompt: str, **kwargs) -> Generator[str, None, None]:
"""Own the durable context for the complete streaming generator."""
# A stream is a turn too: give it its own loop guard (as chat/achat get
# via _enter_turn) so overlapping streams on a shared agent neither
# exhaust nor inherit each other's per-turn limits.
guard_token = _turn_loop_guard.set([self, None])
try:
yield from self._start_stream_turn(prompt, **kwargs)
finally:
try:
_turn_loop_guard.reset(guard_token)
except (ValueError, LookupError):
_turn_loop_guard.set(None)
def _start_stream_turn(self, prompt: str, **kwargs) -> Generator[str, None, None]:
# Tools go to their sandbox before the first model call, once per
# agent. Placed on chat/achat/_start_stream rather than run/start
# because AgentTeam and AgentFlow call these three directly -- an
@@ -1777,7 +1777,7 @@ Focus on creating clear, structured metadata that will help with intelligent fil
request = urllib.request.Request(api_url, headers=headers)
with urllib.request.urlopen(request) as response:
with urllib.request.urlopen(request, timeout=15) as response:
file_data = json.loads(response.read())
if file_data.get('content') and file_data.get('encoding') == 'base64':
@@ -2350,7 +2350,7 @@ This PRP provides:
import urllib.request
import json
with urllib.request.urlopen(api_url) as response:
with urllib.request.urlopen(api_url, timeout=15) as response:
data = json.loads(response.read())
file_structure = []
@@ -1970,11 +1970,13 @@ Write the complete compiled report:"""
# (a never-raised local class never matches a real exception).
class _CircuitBreakerException(Exception):
pass
_NeutralOutcome = _CircuitBreakerException
try:
from ..tools.circuit_breaker import (
get_circuit_breaker,
CircuitBreakerConfig,
CircuitBreakerException as _CircuitBreakerException,
_NeutralOutcome,
)
breaker_name = f"tool_{id(self)}_{function_name}"
breaker = get_circuit_breaker(
@@ -2002,7 +2004,17 @@ Write the complete compiled report:"""
super().__init__(error_dict.get("error", "Tool execution failed"))
async def _invoke_for_breaker():
r = await _invoke()
# Caller errors (bad input) say nothing about the tool's health:
# recorded as neither failure nor success (sync parity), so they
# can neither open the breaker for other callers nor close a
# half-open one. Unwrapped in _invoke_guarded.
try:
r = await _invoke()
except (ValueError, TypeError, AttributeError) as e:
raise _NeutralOutcome(e)
if isinstance(r, dict) and r.get("error") and \
r.get("_praison_retryable") is False:
raise _NeutralOutcome(r)
if isinstance(r, dict) and r.get("error") and \
not r.get("approval_denied") and \
not r.get("permission_denied") and \
@@ -2026,6 +2038,11 @@ Write the complete compiled report:"""
except _ToolFailure as tf:
# Failure was counted by the breaker; return the original dict.
return tf.error_dict
except _NeutralOutcome as n:
neutral = n.value
if isinstance(neutral, BaseException):
raise neutral
return neutral
def _circuit_open_result():
# Record the rejection as a failure so repeated open-circuit
@@ -70,6 +70,9 @@ class ImageAgent(Agent):
api_version=api_version
)
# Only an explicit llm=/model= names an image model; the base Agent's chat default must not leak into image calls
self._image_llm_set = llm is not None or kwargs.get("model") is not None
# Lazy load litellm
self._litellm = None
# Configure logging based on verbose level
@@ -124,15 +127,21 @@ class ImageAgent(Agent):
"Please install it with: pip install litellm"
)
return self._litellm
def _image_model(self, default: str) -> str:
"""Return the explicit ``llm=`` model name, else an image-capable ``default``."""
if not self._image_llm_set:
return default
model_info = self.llm_model
return model_info.model if hasattr(model_info, 'model') else str(model_info)
def generate_image(self, prompt: str, **kwargs) -> Dict[str, Any]:
"""Generate an image based on the provided prompt."""
# Merge default config with any provided kwargs
config = self.image_config.dict(exclude_none=True)
config.update(kwargs)
# Get the model name robustly from the parent Agent's property
model_info = self.llm_model
model_name = model_info.model if hasattr(model_info, 'model') else str(model_info)
model_name = self._image_model("dall-e-3")
# Use the model name in config
config['model'] = model_name
@@ -264,7 +273,7 @@ class ImageAgent(Agent):
import litellm
litellm.telemetry = False
model_name = self.llm or self.model or "dall-e-2"
model_name = self._image_model("dall-e-2")
config = {
"model": model_name,
@@ -339,7 +348,7 @@ class ImageAgent(Agent):
import litellm
litellm.telemetry = False
model_name = self.llm or self.model or "dall-e-2"
model_name = self._image_model("dall-e-2")
config = {
"model": model_name,
@@ -301,10 +301,9 @@ class RouterAgent(Agent):
Returns:
Task execution result
"""
# Estimate context size in tokens (rough estimate: ~4 chars per token)
# This is a simplified heuristic; actual tokenization varies by model
text_length = len(task_description) + (len(context) if context else 0)
context_size = text_length // 4 # Approximate token count
# Estimate context size in tokens (canonical heuristic, CJK-aware)
from ..context.tokens import estimate_tokens_heuristic
context_size = estimate_tokens_heuristic(task_description + (context or ""))
# Select the best model for this task
selected_model = self._select_model_for_task(
@@ -2174,22 +2174,25 @@ class ToolExecutionMixin:
Returns:
Estimated minimum cost in USD for the call.
"""
prompt_chars = 0
texts = []
if messages:
for _msg in messages:
_content = _msg.get("content") if isinstance(_msg, dict) else None
if isinstance(_content, str):
prompt_chars += len(_content)
texts.append(_content)
elif isinstance(_content, list):
for _part in _content:
if isinstance(_part, dict):
_text = _part.get("text")
if isinstance(_text, str):
prompt_chars += len(_text)
texts.append(_text)
elif _content is not None:
prompt_chars += len(str(_content))
texts.append(str(_content))
prompt_tokens = prompt_chars // 4
from ..context.tokens import estimate_tokens_heuristic
prompt_text = "".join(texts)
prompt_chars = len(prompt_text)
prompt_tokens = estimate_tokens_heuristic(prompt_text)
if max_tokens:
completion_tokens = int(max_tokens)
elif prompt_chars > 0:
@@ -3298,7 +3301,7 @@ class ToolExecutionMixin:
"""
# Import circuit breaker components first (lazy import for performance)
try:
from ..tools.circuit_breaker import get_circuit_breaker, CircuitBreakerConfig, CircuitBreakerException
from ..tools.circuit_breaker import get_circuit_breaker, CircuitBreakerConfig, CircuitBreakerException, _NeutralOutcome
except ImportError:
# Circuit breaker not available - fallback to direct execution
logging.debug("Circuit breaker not available, falling back to direct tool execution")
@@ -3327,6 +3330,13 @@ class ToolExecutionMixin:
# Execute tool through circuit breaker with failure detection wrapper
def _tool_wrapper():
result = self._execute_tool_impl(function_name, arguments)
# Caller errors (bad input: ValueError/TypeError/AttributeError,
# tagged non-retryable) say nothing about the tool's health: record
# them as neither failure nor success, so they can neither open the
# breaker for other callers nor close a half-open one.
if isinstance(result, dict) and result.get("error") and \
result.get("_praison_retryable") is False:
raise _NeutralOutcome(result)
# Convert error dicts to exceptions so circuit breaker can detect failures
# Don't treat approval/permission denials as circuit breaker failures
if isinstance(result, dict) and result.get("error") and \
@@ -3345,6 +3355,8 @@ class ToolExecutionMixin:
try:
return breaker.call(_tool_wrapper)
except _NeutralOutcome as n:
return n.value
except Exception as e:
# Check if this is our sentinel exception
if hasattr(e, 'error_dict'):
@@ -336,17 +336,27 @@ class PermissionAllowlist:
if not perm.session_only
}
}
from ..utils.atomic_io import atomic_write_json
os.makedirs(os.path.dirname(filepath) or ".", exist_ok=True)
with open(filepath, "w") as f:
json.dump(data, f, indent=2)
atomic_write_json(filepath, data, indent=2)
@classmethod
def load(cls, filepath: str) -> "PermissionAllowlist":
allowlist = cls()
if not os.path.exists(filepath):
return allowlist
with open(filepath, "r") as f:
data = json.load(f)
try:
with open(filepath, "r") as f:
data = json.load(f)
except json.JSONDecodeError:
# Fail closed: a corrupt allowlist grants nothing.
from ..utils.atomic_io import quarantine_corrupt_file
try:
dest = quarantine_corrupt_file(filepath)
logger.warning("Corrupt permission allowlist %s moved to %s; starting empty", filepath, dest)
except OSError:
logger.warning("Corrupt permission allowlist %s; ignoring it", filepath)
return allowlist
for tool_name, tool_data in data.get("tools", {}).items():
allowlist.add_tool(tool_name, paths=tool_data.get("allowed_paths", []), session_only=tool_data.get("session_only", False))
return allowlist
@@ -91,6 +91,24 @@ PERMISSION_PRESETS = {
"off": frozenset(),
}
# Optional host hook (Issue #5912): a runner that bounds concurrent work (e.g.
# the gateway's admission gate) binds a zero-arg factory here for the duration
# of an admitted turn. The registry enters ``factory()`` — a sync *and* async
# context manager — around the backend's human wait so a turn parked on a
# person does not occupy a run slot. Unbound (the default) is a no-op.
HUMAN_WAIT_SCOPE: contextvars.ContextVar[Optional[Callable[[], object]]] = contextvars.ContextVar(
"praisonai_approval_human_wait_scope", default=None
)
def _human_wait_scope() -> object:
factory = HUMAN_WAIT_SCOPE.get()
if factory is None:
import contextlib
return contextlib.nullcontext()
return factory()
class ApprovalRegistry:
"""Per-agent approval configuration.
@@ -671,16 +689,17 @@ class ApprovalRegistry:
liveness=liveness,
)
# Prefer sync method if available
if hasattr(backend, "request_approval_sync"):
decision = backend.request_approval_sync(request)
else:
# Use shared utility for consistent async-to-sync bridging
from .utils import run_coroutine_safely
decision = run_coroutine_safely(
backend.request_approval(request),
timeout=self.timeout
)
with _human_wait_scope():
# Prefer sync method if available
if hasattr(backend, "request_approval_sync"):
decision = backend.request_approval_sync(request)
else:
# Use shared utility for consistent async-to-sync bridging
from .utils import run_coroutine_safely
decision = run_coroutine_safely(
backend.request_approval(request),
timeout=self.timeout
)
# Live-authority gate: a resolution that lands after the originating
# turn was stopped/superseded is dropped fail-closed — the tool does not
@@ -746,10 +765,11 @@ class ApprovalRegistry:
)
try:
decision = await asyncio.wait_for(
backend.request_approval(request),
timeout=self.timeout,
)
async with _human_wait_scope():
decision = await asyncio.wait_for(
backend.request_approval(request),
timeout=self.timeout,
)
except asyncio.TimeoutError:
decision = ApprovalDecision(approved=False, reason="Approval timed out")
@@ -8,7 +8,7 @@ import asyncio
from praisonaiagents._logging import get_logger
import threading
from typing import Any, Callable, Dict, List, Optional, Set, Union
from ..utils.async_bridge import run_coroutine_from_any_context
from ..utils.async_bridge import run_coroutine_from_any_context, spawn_background
from dataclasses import dataclass, field
from .event import Event, EventType
@@ -235,11 +235,14 @@ class EventBus:
if sub.is_async:
# Schedule async callback safely
try:
loop = asyncio.get_running_loop()
loop.create_task(sub.callback(event))
asyncio.get_running_loop()
except RuntimeError:
# No running loop, use safe async bridge
run_coroutine_from_any_context(sub.callback(event))
else:
# Keep a strong ref and log failures under this
# subscriber's id, like the sync path below.
spawn_background(sub.callback(event), f"subscriber {sub.id}")
else:
sub.callback(event)
except Exception as e:
@@ -708,12 +708,17 @@ def _config_write_target() -> Path:
def _dump_config_file(path: Path, data: Dict[str, Any]) -> None:
"""Write ``data`` to ``path`` as YAML or TOML based on its suffix."""
from ..utils.atomic_io import atomic_write_text
# Write through a symlinked config so the shared target gets the update,
# and keep an existing file's mode (the atomic temp file is owner-only).
path = path.resolve()
mode = path.stat().st_mode & 0o7777 if path.exists() else None
path.parent.mkdir(parents=True, exist_ok=True)
if path.suffix.lower() in (".yaml", ".yml"):
import yaml
with open(path, "w", encoding="utf-8") as f:
yaml.safe_dump(data, f, sort_keys=False, default_flow_style=False)
text = yaml.safe_dump(data, sort_keys=False, default_flow_style=False)
else:
try:
import tomli_w
@@ -727,8 +732,10 @@ def _dump_config_file(path: Path, data: Dict[str, Any]) -> None:
"Install tomli-w (pip install tomli-w), or convert the config "
"to .praisonai/config.yaml."
) from None
with open(path, "wb") as f:
tomli_w.dump(data, f)
text = tomli_w.dumps(data)
atomic_write_text(str(path), text)
if mode is not None:
os.chmod(path, mode)
def set_plugin_enabled(name: str, enabled: bool) -> Path:
@@ -16,6 +16,7 @@ from typing import Any, Callable, Dict, Optional, Set, Type, Union
import inspect
from .parse_utils import (
canonical_preset_key,
detect_url_scheme,
is_path_like,
make_preset_error,
@@ -345,12 +346,12 @@ def _resolve_string(
if presets:
# Case-insensitive lookup, whitespace-tolerant and treating -/_ as
# interchangeable so the accepted spellings match the closed-set guard
# in validate_preset_string (which uses the same normalization). Without
# this, a value that passed construction-time validation (e.g.
# " sliding_window " or "sliding-window") would be rejected here.
value_norm = value.strip().lower().replace("-", "_")
# in validate_preset_string. Both go through canonical_preset_key, the
# single source of truth for the spelling rule, so validation and
# resolution cannot disagree about what counts as the same preset.
value_norm = canonical_preset_key(value)
for preset_key in presets:
if preset_key.strip().lower().replace("-", "_") == value_norm:
if canonical_preset_key(preset_key) == value_norm:
return _apply_preset(preset_key, presets, config_class)
# Not a valid preset - raise helpful error
@@ -157,8 +157,8 @@ class FileIndex:
"entries": {k: v.to_dict() for k, v in self.entries.items()}
}
with open(self.index_path, 'w', encoding='utf-8') as f:
json.dump(data, f, indent=2)
from praisonaiagents.utils.atomic_io import atomic_write_json
atomic_write_json(self.index_path, data, indent=2)
return True
except Exception as e:
@@ -228,6 +228,10 @@ _LAZY_IMPORTS = {
"StopResult": ("praisonaiagents.gateway.protocols", "StopResult"),
# Schema-validated inbound frame codec (Issue #2831)
"HelloParams": ("praisonaiagents.gateway.protocols", "HelloParams"),
# Device-identity handshake contract (Issue #5998)
"DeviceAuth": ("praisonaiagents.gateway.protocols", "DeviceAuth"),
"DeviceIdentityProtocol": ("praisonaiagents.gateway.protocols", "DeviceIdentityProtocol"),
"device_id_from_public_key": ("praisonaiagents.gateway.protocols", "device_id_from_public_key"),
"HelloResult": ("praisonaiagents.gateway.protocols", "HelloResult"),
"HelloError": ("praisonaiagents.gateway.protocols", "HelloError"),
"ConnectErrorCode": ("praisonaiagents.gateway.protocols", "ConnectErrorCode"),
@@ -586,6 +590,9 @@ __all__ = [
"StopResult",
# Schema-validated inbound frame codec (Issue #2831)
"HelloParams",
"DeviceAuth",
"DeviceIdentityProtocol",
"device_id_from_public_key",
"HelloResult",
"HelloError",
"ConnectErrorCode",
@@ -11,6 +11,8 @@ The ``praisonai`` wrapper provides backward-compatible shims.
from __future__ import annotations
import base64
import hashlib
import math
import time
import uuid
@@ -347,10 +349,88 @@ class SessionObserverProtocol(Protocol):
...
# ---------------------------------------------------------------------------
# Device identity (Issue #5998)
#
# A per-device keypair signs a server-issued challenge nonce so the gateway can
# derive a stable ``device_id`` and approve/revoke devices individually instead
# of sharing one gateway-wide secret. Core only fixes the wire shape, the
# fingerprint and the exact bytes that are signed; signature verification,
# device storage and per-device tokens live in the wrapper gateway.
# ---------------------------------------------------------------------------
def device_id_from_public_key(public_key: str) -> str:
"""Return the stable device id: hex SHA-256 of the raw public-key bytes.
``public_key`` must be standard base64 (RFC 4648, padded, no whitespace) of
the raw key bytes. Hashing the decoded bytes keeps the id independent of
how a client spells the encoding. Raises ``ValueError`` on invalid base64.
"""
return hashlib.sha256(base64.b64decode(public_key, validate=True)).hexdigest()
@dataclass
class DeviceAuth:
"""Device-identity proof carried in a ``hello`` frame.
Attributes:
device_id: Fingerprint of ``public_key`` (see :func:`device_id_from_public_key`).
public_key: Standard base64 (padded) of the device's raw public key.
signature: Standard base64 (padded) signature over :meth:`signing_payload`.
signed_at: Integer Unix seconds at which the device signed the challenge
(an integer so every client language renders identical signed bytes).
nonce: The server-issued challenge nonce being answered.
"""
device_id: str
public_key: str
signature: str
signed_at: int
nonce: str
def signing_payload(self, agent_id: str) -> bytes:
"""Canonical bytes the device signs and the gateway verifies.
Binds the nonce to this device, the signing time and the target agent so
a captured proof cannot be replayed for another device or agent.
"""
return "\n".join((
"praisonai-device-v1",
self.device_id,
self.nonce,
str(int(self.signed_at)),
agent_id,
)).encode("utf-8")
@runtime_checkable
class DeviceIdentityProtocol(Protocol):
"""Seam for per-device challenge/response auth (implemented in the wrapper).
Implementations must approve only on an exact device-key match — never on
network reachability alone — and record how each device was approved.
"""
def issue_challenge(self) -> str:
"""Return a fresh single-use challenge nonce for a connecting client."""
...
def verify(self, device: DeviceAuth, *, agent_id: str) -> Optional[str]:
"""Verify ``device`` against an issued nonce; return its role, or ``None`` if rejected."""
...
def list_devices(self) -> List[Dict[str, Any]]:
"""List known devices (id, role, approval path, last seen)."""
...
def revoke(self, device_id: str) -> bool:
"""Revoke one device; return ``False`` if it was unknown."""
...
@dataclass
class HelloParams:
"""Parameters for initiating a versioned handshake.
Attributes:
agent_id: The agent to connect to
protocol_min: Minimum protocol version the client supports
@@ -358,6 +438,7 @@ class HelloParams:
capabilities: Optional list of capability tokens the client supports
session_id: Optional session to resume
since: Optional cursor for event replay
device: Optional device-identity proof (Issue #5998)
"""
agent_id: str
protocol_min: int
@@ -365,6 +446,7 @@ class HelloParams:
capabilities: List[str] = field(default_factory=list)
session_id: Optional[str] = None
since: Optional[int] = None
device: Optional[DeviceAuth] = None
type: str = field(default="hello", init=False)
@@ -897,6 +979,9 @@ def _decode_hello(data: Dict[str, Any]) -> "HelloParams":
if since is not None:
since = _coerce_int(since, field_name="since")
device_data = data.get("device")
device = _decode_device(device_data) if device_data is not None else None
return HelloParams(
agent_id=agent_id,
protocol_min=client_min,
@@ -904,6 +989,60 @@ def _decode_hello(data: Dict[str, Any]) -> "HelloParams":
capabilities=capabilities,
session_id=_as_opt_str(data.get("session_id")),
since=since,
device=device,
)
def _decode_device(data: Any) -> DeviceAuth:
"""Validate a ``hello.device`` block into :class:`DeviceAuth`.
Shape errors are configuration errors; a ``device_id`` that is not the
fingerprint of ``public_key`` is rejected as unauthorized so a client can
never claim another device's id.
"""
if not isinstance(data, dict):
raise FrameDecodeError(
HelloError(
code=ConnectErrorCode.CONFIGURATION_ERROR,
message="Field 'device' must be an object",
next_step=ConnectRecoveryStep.DO_NOT_RETRY,
)
)
public_key = _require_str(data.get("public_key"), field_name="device.public_key")
device_id = _require_str(data.get("device_id"), field_name="device.device_id")
signed_at = data.get("signed_at")
if isinstance(signed_at, bool) or not isinstance(signed_at, int):
raise FrameDecodeError(
HelloError(
code=ConnectErrorCode.CONFIGURATION_ERROR,
message="Field 'device.signed_at' must be an integer (Unix seconds)",
next_step=ConnectRecoveryStep.DO_NOT_RETRY,
)
)
try:
fingerprint = device_id_from_public_key(public_key)
except ValueError:
raise FrameDecodeError(
HelloError(
code=ConnectErrorCode.CONFIGURATION_ERROR,
message="Field 'device.public_key' must be padded standard base64",
next_step=ConnectRecoveryStep.DO_NOT_RETRY,
)
) from None
if device_id != fingerprint:
raise FrameDecodeError(
HelloError(
code=ConnectErrorCode.AUTH_UNAUTHORIZED,
message="device.device_id does not match the public key fingerprint",
next_step=ConnectRecoveryStep.REAUTHENTICATE,
)
)
return DeviceAuth(
device_id=device_id,
public_key=public_key,
signature=_require_str(data.get("signature"), field_name="device.signature"),
signed_at=signed_at,
nonce=_require_str(data.get("nonce"), field_name="device.nonce"),
)
@@ -417,9 +417,9 @@ class FileTracker:
return
try:
os.makedirs(os.path.dirname(self._state_file), exist_ok=True)
with open(self._state_file, "w", encoding="utf-8") as f:
json.dump(self._tracked, f, indent=2)
os.makedirs(os.path.dirname(self._state_file) or ".", exist_ok=True)
from ..utils.atomic_io import atomic_write_json
atomic_write_json(self._state_file, self._tracked, indent=2)
except (OSError, IOError):
pass
+45 -10
View File
@@ -242,6 +242,19 @@ class TokenUsage:
'audio_output_tokens': self.audio_output_tokens,
}
# error_classifier category -> (text-derived kinds that may refine it, default kind).
_CATEGORY_KINDS = {
"rate_limit": ({"rate_limit", "billing"}, "rate_limit"),
"auth": ({"auth", "auth_permanent"}, "auth"),
"context_limit": ({"context_overflow"}, "context_overflow"),
"invalid_request": (
{"context_overflow", "model_not_found", "auth", "auth_permanent", "billing", "format_error"},
"format_error",
),
"transient": ({"overloaded", "idle_timeout"}, "overloaded"),
"permanent": ({"billing"}, "billing"),
}
class LLM:
"""
Easy to use wrapper for language models. Supports multiple providers like OpenAI,
@@ -894,11 +907,12 @@ Respond with ONLY a valid JSON tool call in this format:
else:
return False
def _parse_retry_delay(self, error_message: str) -> float:
"""Parse retry delay from rate limit error message.
def _parse_retry_delay(self, error_message: Union[Exception, str]) -> float:
"""Parse retry delay from a rate limit error.
Args:
error_message: The error message from the API
error_message: The provider exception (preferred, so the structured
``Retry-After`` response header is honoured) or its message
Returns:
Retry delay in seconds, or default if not found
@@ -910,7 +924,8 @@ Respond with ONLY a valid JSON tool call in this format:
from .error_classifier import extract_retry_after
delay = extract_retry_after(Exception(error_message), cap_seconds=max_delay)
error = error_message if isinstance(error_message, BaseException) else Exception(error_message)
delay = extract_retry_after(error, cap_seconds=max_delay)
return self._retry_delay if delay is None else delay
def _is_rate_limit_error(self, error: Exception) -> bool:
@@ -947,15 +962,35 @@ Respond with ONLY a valid JSON tool call in this format:
def classify_error_kind(self, error: Exception) -> AgentErrorKind:
"""
Classify error into typed AgentErrorKind instead of freeform strings.
Replaces scattered regex checks with systematic classification.
The HTTP status code / exception type (via ``error_classifier``) is
authoritative; message text only refines the kind within that category,
so e.g. a 400 whose body happens to contain "500" is never retried as
an overload.
Args:
error: Exception to classify
Returns:
AgentErrorKind: Typed error classification
"""
text_kind = self._classify_error_kind_from_text(error)
from .error_classifier import _classify_by_type_and_code, _extract_status_code
category = _classify_by_type_and_code(error)
if category is None:
return text_kind
allowed, default = _CATEGORY_KINDS[category.value]
if text_kind in allowed:
return text_kind
# A 404 from a completion endpoint means the model/deployment is missing.
if _extract_status_code(error) == 404 or type(error).__name__ == "NotFoundError":
return "model_not_found"
if default == "overloaded" and "timeout" in type(error).__name__.lower():
return "idle_timeout"
return default
def _classify_error_kind_from_text(self, error: Exception) -> AgentErrorKind:
"""Message-substring fallback used when no status code / type is known."""
error_str = str(error).lower()
error_type = type(error).__name__.lower()
@@ -977,7 +1012,7 @@ Respond with ONLY a valid JSON tool call in this format:
# Billing/quota issues (must be checked before generic 429/rate-limit)
if any(indicator in error_str for indicator in [
"insufficient quota", "quota exceeded", "billing", "credit",
"insufficient quota", "quota exceeded", "billing", "credit balance", "insufficient credit",
"payment required", "subscription required", "subscription expired",
"plan limit"
]):
@@ -1109,7 +1144,7 @@ Respond with ONLY a valid JSON tool call in this format:
# Rate limiting - extract retry delay
if error_kind == "rate_limit":
backoff = self._parse_retry_delay(str(error)) # Returns seconds
backoff = self._parse_retry_delay(error) # Returns seconds
if backoff == 0: # No specific delay found, use exponential backoff
backoff = min(2 ** (attempt - 1), 60) # 1s, 2s, 4s, ... cap at 60s
return FailoverDecision(
+25 -40
View File
@@ -154,10 +154,19 @@ def register_approval_callback(callback_fn):
add_display_callback = register_display_callback
add_approval_callback = register_approval_callback
def _filter_callback_kwargs(callback, kwargs):
"""Filter kwargs to what the callback accepts to maintain backward compatibility."""
import inspect
sig = inspect.signature(callback)
if any(p.kind == inspect.Parameter.VAR_KEYWORD for p in sig.parameters.values()):
return kwargs
return {k: v for k, v in kwargs.items() if k in sig.parameters}
def execute_sync_callback(display_type: str, **kwargs):
"""Execute synchronous callback for a given display type without displaying anything.
This function is used to trigger callbacks even when verbose=False.
Callback errors are logged and suppressed so observers cannot abort the run.
Args:
display_type (str): Type of display event
@@ -167,61 +176,37 @@ def execute_sync_callback(display_type: str, **kwargs):
callback = sync_display_callbacks.get(display_type)
if callback:
import inspect
sig = inspect.signature(callback)
# Filter kwargs to what the callback accepts to maintain backward compatibility
if any(p.kind == inspect.Parameter.VAR_KEYWORD for p in sig.parameters.values()):
# Callback accepts **kwargs, so pass all arguments
supported_kwargs = kwargs
else:
# Only pass arguments that the callback signature supports
supported_kwargs = {k: v for k, v in kwargs.items() if k in sig.parameters}
callback(**supported_kwargs)
try:
callback(**_filter_callback_kwargs(callback, kwargs))
except Exception:
logging.exception("Display callback for %r failed; ignored", display_type)
async def execute_callback(display_type: str, **kwargs):
"""Execute both sync and async callbacks for a given display type.
Callback errors are logged and suppressed so observers cannot abort the run.
Args:
display_type (str): Type of display event
**kwargs: Arguments to pass to the callback functions
"""
import inspect
# Execute synchronous callback if registered
with _callbacks_lock:
sync_callback = sync_display_callbacks.get(display_type)
async_callback = async_display_callbacks.get(display_type)
if sync_callback:
sig = inspect.signature(sync_callback)
# Filter kwargs to what the callback accepts to maintain backward compatibility
if any(p.kind == inspect.Parameter.VAR_KEYWORD for p in sig.parameters.values()):
# Callback accepts **kwargs, so pass all arguments
supported_kwargs = kwargs
else:
# Only pass arguments that the callback signature supports
supported_kwargs = {k: v for k, v in kwargs.items() if k in sig.parameters}
loop = asyncio.get_event_loop()
await loop.run_in_executor(None, lambda: sync_callback(**supported_kwargs))
try:
supported_kwargs = _filter_callback_kwargs(sync_callback, kwargs)
loop = asyncio.get_running_loop()
await loop.run_in_executor(None, lambda: sync_callback(**supported_kwargs))
except Exception:
logging.exception("Display callback for %r failed; ignored", display_type)
# Execute asynchronous callback if registered
if async_callback:
callback = async_callback
sig = inspect.signature(callback)
# Filter kwargs to what the callback accepts to maintain backward compatibility
if any(p.kind == inspect.Parameter.VAR_KEYWORD for p in sig.parameters.values()):
# Callback accepts **kwargs, so pass all arguments
supported_kwargs = kwargs
else:
# Only pass arguments that the callback signature supports
supported_kwargs = {k: v for k, v in kwargs.items() if k in sig.parameters}
await callback(**supported_kwargs)
try:
await async_callback(**_filter_callback_kwargs(async_callback, kwargs))
except Exception:
logging.exception("Async display callback for %r failed; ignored", display_type)
def _clean_display_content(content: str, max_length: int = 20000) -> str:
"""Helper function to clean and truncate content for display."""
@@ -28,15 +28,35 @@ _LOCALHOST_HOSTS = {"localhost", "127.0.0.1", "::1"}
def _require_https(url: str) -> None:
"""Reject non-HTTPS metadata URLs (loopback HTTP is allowed for testing)."""
"""Reject OAuth URLs that are not HTTPS or that point at internal hosts.
Loopback hosts are allowed (over HTTP too) for local testing. Any other
host must pass :func:`~praisonaiagents.tools.url_safety.is_safe_http_url`,
so private, link-local (cloud metadata) and ambiguous numeric hosts are
refused even over HTTPS.
"""
parsed = urlparse(url)
if parsed.scheme == "https":
if parsed.scheme in ("http", "https") and (parsed.hostname or "") in _LOCALHOST_HOSTS:
return
if parsed.scheme == "http" and (parsed.hostname or "") in _LOCALHOST_HOSTS:
return
raise ValueError(
f"Refusing to fetch OAuth metadata over insecure transport: {url!r}"
)
if parsed.scheme != "https":
raise ValueError(
f"Refusing OAuth request over insecure transport: {url!r}"
)
from ..tools.url_safety import is_safe_http_url
if not is_safe_http_url(url):
raise ValueError(
f"Refusing OAuth request to a private/internal address: {url!r}"
)
def _require_success(resp: Any) -> None:
"""Raise unless ``resp`` is 2xx (credential POSTs don't follow redirects)."""
resp.raise_for_status()
if not 200 <= resp.status_code < 300:
raise ValueError(
f"Unexpected HTTP {resp.status_code} from OAuth endpoint {resp.url!r}"
)
def parse_www_authenticate(header: Optional[str]) -> Dict[str, str]:
@@ -62,30 +82,31 @@ def parse_www_authenticate(header: Optional[str]) -> Dict[str, str]:
return params
def _get_json(url: str, timeout: float = 10.0) -> Dict[str, Any]:
def _get_json(url: str, timeout: float = 10.0, max_redirects: int = 3) -> Dict[str, Any]:
"""Fetch and parse a JSON metadata document over HTTPS.
Redirects are followed but every hop — including the final destination — is
re-validated with :func:`_require_https` so a redirect cannot escape the
HTTPS/loopback guard to an internal or plaintext endpoint (SSRF-safe).
Redirects are followed manually and every hop is validated with
:func:`_require_https` *before* it is requested, so a redirect cannot
reach an internal or plaintext endpoint (SSRF-safe).
"""
_require_https(url)
import requests # lazy import — optional dependency
resp = requests.get(
url,
headers={"Accept": "application/json"},
timeout=timeout,
allow_redirects=True,
)
# Re-validate every redirect hop and the final URL: allow_redirects only
# checks the initial URL, so a 30x to http:// or an internal host would
# otherwise bypass the guard.
for previous in getattr(resp, "history", None) or []:
_require_https(previous.url)
_require_https(resp.url)
resp.raise_for_status()
return resp.json()
current = url
for _ in range(max_redirects + 1):
_require_https(current)
resp = requests.get(
current,
headers={"Accept": "application/json"},
timeout=timeout,
allow_redirects=False,
)
location = resp.headers.get("Location")
if resp.status_code in (301, 302, 303, 307, 308) and location:
current = urljoin(current, location)
continue
resp.raise_for_status()
return resp.json()
raise ValueError(f"Too many redirects fetching OAuth metadata: {url!r}")
def _wellknown_candidates(server_url: str) -> List[str]:
@@ -268,8 +268,11 @@ class MCPOAuthProvider:
data=data,
headers={"Accept": "application/json"},
timeout=timeout,
allow_redirects=False, # never re-send credentials to a redirect target
)
resp.raise_for_status()
from .oauth_discovery import _require_success
_require_success(resp)
return self._normalize_tokens(
resp.json(), token_endpoint=metadata["token_endpoint"]
)
@@ -285,7 +288,7 @@ class MCPOAuthProvider:
return False
try:
from .oauth_discovery import _require_https
from .oauth_discovery import _require_https, _require_success
# Defense in depth: never POST a refresh token / client secret to a
# persisted endpoint that isn't HTTPS (or loopback).
@@ -306,8 +309,9 @@ class MCPOAuthProvider:
data=data,
headers={"Accept": "application/json"},
timeout=10.0,
allow_redirects=False, # never re-send credentials to a redirect target
)
resp.raise_for_status()
_require_success(resp)
new_tokens = self._normalize_tokens(
resp.json(), token_endpoint=token_endpoint
)
@@ -10,26 +10,13 @@ returned for persistence via :meth:`MCPAuthStorage.set_client_info`.
"""
from typing import Any, Dict, List, Optional
from urllib.parse import urlparse
from praisonaiagents._logging import get_logger
from .oauth_discovery import _require_https, _require_success
logger = get_logger("mcp-oauth-registration")
_LOCALHOST_HOSTS = {"localhost", "127.0.0.1", "::1"}
def _require_https(url: str) -> None:
"""Reject non-HTTPS registration URLs (loopback HTTP allowed for testing)."""
parsed = urlparse(url)
if parsed.scheme == "https":
return
if parsed.scheme == "http" and (parsed.hostname or "") in _LOCALHOST_HOSTS:
return
raise ValueError(
f"Refusing to register client over insecure transport: {url!r}"
)
def register_client(
registration_endpoint: str,
@@ -75,8 +62,9 @@ def register_client(
json=payload,
headers={"Content-Type": "application/json", "Accept": "application/json"},
timeout=timeout,
allow_redirects=False, # never re-send credentials to a redirect target
)
resp.raise_for_status()
_require_success(resp)
client_info = resp.json()
if not client_info.get("client_id"):
@@ -115,11 +115,8 @@ class PolicyConfig:
rules_dir = os.path.dirname(self.rules_file)
os.makedirs(rules_dir, exist_ok=True)
# Atomic write: tmp file + rename
tmp_file = self.rules_file + ".tmp"
with open(tmp_file, "w") as f:
json.dump({"rules": existing}, f, indent=2)
os.replace(tmp_file, self.rules_file)
from ..utils.atomic_io import atomic_write_json
atomic_write_json(self.rules_file, {"rules": existing}, indent=2)
logger.debug(f"Saved learned rule: {resource} -> {action}")
return True
@@ -11,6 +11,7 @@ from dataclasses import dataclass
from typing import Any, Dict, List, Optional, Protocol, runtime_checkable
from ..context.budgeter import MODEL_LIMITS as _CANONICAL_LIMITS
from ..context.tokens import estimate_tokens_heuristic
# RAG-specific context windows not present in the canonical budgeter table
@@ -285,11 +286,28 @@ def estimate_tokens(text: str) -> int:
Returns:
Estimated token count
"""
if not text:
return 0
# Average of 4 characters per token for English
# This is a conservative estimate
return len(text) // 4 + 1
# Delegate to the canonical estimator so dense (CJK/non-ASCII) text is not
# under-counted ~5x and over-filled past the budget.
return estimate_tokens_heuristic(text)
def _truncate_to_tokens(text: str, max_tokens: int, estimator=None) -> str:
"""Return the longest prefix of ``text`` whose estimate fits ``max_tokens``.
Binary-searches on the (monotonic) estimator instead of assuming
~4 chars/token, which overshoots the budget for dense (CJK) text.
"""
estimator = estimator or estimate_tokens
if max_tokens <= 0 or not text:
return ""
lo, hi = 0, len(text)
while lo < hi:
mid = (lo + hi + 1) // 2
if estimator(text[:mid]) <= max_tokens:
lo = mid
else:
hi = mid - 1
return text[:lo]
class DefaultBudgetEnforcer:
@@ -340,9 +358,12 @@ class DefaultBudgetEnforcer:
# Try to fit partial chunk if there's remaining space
remaining = available - used_tokens
if remaining > 100: # Only if meaningful space remains
# Truncate text to fit
chars_to_keep = remaining * 4 # Approximate
truncated_text = text[:chars_to_keep]
# Truncate text to fit, leaving room for the "..." suffix
truncated_text = _truncate_to_tokens(
text,
remaining - self._token_estimator("..."),
self._token_estimator,
)
if truncated_text:
truncated_chunk = {
**chunk,
@@ -12,7 +12,7 @@ No heavy imports at module level.
from dataclasses import dataclass, field
from typing import Any, Dict, List, Optional, Protocol, runtime_checkable
from .budget import estimate_tokens
from .budget import _truncate_to_tokens, estimate_tokens
@runtime_checkable
@@ -219,8 +219,9 @@ class ContextCompressor:
# Partial fit
remaining = target_tokens - used_tokens
if remaining > 50: # Only if meaningful space
chars_to_keep = remaining * 4
truncated_text = text[:chars_to_keep] + "..."
truncated_text = _truncate_to_tokens(
text, remaining - estimate_tokens("...")
) + "..."
result.append({
**chunk,
'text': truncated_text,
@@ -8,7 +8,7 @@ No heavy imports - only stdlib.
import hashlib
from typing import TYPE_CHECKING, Any, Dict, List, Optional, Set, Tuple, Union
from .budget import estimate_tokens as _budget_estimate_tokens
from .budget import _truncate_to_tokens, estimate_tokens as _budget_estimate_tokens
if TYPE_CHECKING:
from ..knowledge.models import SearchResultItem
@@ -156,10 +156,10 @@ def truncate_context(
# Calculate target character count
suffix_tokens = _estimate_tokens(suffix)
target_tokens = max_tokens - suffix_tokens
target_chars = target_tokens * 4
truncated = _truncate_to_tokens(text, target_tokens, _estimate_tokens)
target_chars = len(truncated)
# Truncate at word boundary
truncated = text[:target_chars]
last_space = truncated.rfind(" ")
if last_space > target_chars * 0.8:
truncated = truncated[:last_space]
@@ -119,10 +119,15 @@ class ScheduleRunner:
job_name=job.name,
)
if job.should_retire(job.last_run_at):
self._store.remove(job.id)
else:
self._store.update(job)
# The run already happened (its claim was persisted), so a failed write
# here is reported, not raised: one bad write must not stop the loop.
try:
if job.should_retire(job.last_run_at):
self._store.remove(job.id)
else:
self._store.update(job)
except OSError as e:
logger.error("Failed to persist run state for job %r: %s", job.id, e)
# ── job-to-job context chaining ───────────────────────────────────
@@ -16,6 +16,7 @@ from typing import Dict, Iterator, List, Optional
from .models import ScheduleJob, RunRecord
from .due import is_due as _is_due, is_misfire as _is_misfire
from .hook_emit import emit_schedule_add, emit_schedule_remove
from ..utils.atomic_io import atomic_write_json
logger = get_logger(__name__)
@@ -41,6 +42,8 @@ class FileScheduleStore:
self._lock = threading.RLock()
self._jobs: Dict[str, ScheduleJob] = {}
self._history: List[RunRecord] = []
# Run records whose history write failed; merged into the next save.
self._unsaved: List[RunRecord] = []
# In-memory record of leases we currently hold, so ``complete`` can
# release them. Cross-process leases live in the persisted job
# (``lease_until`` / ``lease_owner``) which is the source of truth.
@@ -60,7 +63,7 @@ class FileScheduleStore:
if job.id in self._jobs:
raise ValueError(f"Job '{job.id}' already exists")
self._jobs[job.id] = job
self._save()
self._save_or_rollback(job.id)
# Emit SCHEDULE_ADD outside the store locks: a hook must never be able
# to deadlock the cross-process file lock by calling back in.
emit_schedule_add(job)
@@ -120,7 +123,7 @@ class FileScheduleStore:
job._lease_until = getattr(existing, "_lease_until", 0.0) or 0.0
job._lease_owner = getattr(existing, "_lease_owner", None)
self._jobs[job.id] = job
self._save()
self._save_or_rollback(job.id)
def remove(self, job_id: str) -> bool:
removed = None
@@ -128,7 +131,7 @@ class FileScheduleStore:
self._reload_locked()
if job_id in self._jobs:
removed = self._jobs.pop(job_id)
self._save()
self._save_or_rollback(job_id)
if removed is None:
return False
emit_schedule_remove(removed)
@@ -150,7 +153,7 @@ class FileScheduleStore:
if principal is not None and job.principal != principal:
continue
removed = self._jobs.pop(jid)
self._save()
self._save_or_rollback(jid)
break
if removed is None:
return False
@@ -255,7 +258,7 @@ class FileScheduleStore:
emit_schedule_remove(job)
# Record suppressed occurrences so a missed run is visible in history
# rather than silently dropped. Logged outside the file lock (log_run
# takes only the thread lock) to keep the critical section short.
# takes it itself; flock is not re-entrant across file handles).
for job_id, job_name in missed:
self.log_run(
job_id=job_id,
@@ -301,12 +304,26 @@ class FileScheduleStore:
delivered=delivered,
timestamp=time.time(),
)
with self._lock:
self._history.insert(0, record)
# Prune to max_history
if len(self._history) > self._max_history:
self._history = self._history[:self._max_history]
self._save_history()
with self._lock, self._file_lock():
# Reload under the cross-process lock so concurrent writers (e.g.
# the scheduler daemon and a CLI process) don't drop each other's
# records (last-writer-wins). Records whose earlier save failed are
# carried forward so they are retried, not discarded.
pending = ([record] + self._unsaved)[:self._max_history]
on_disk = self._read_history()
if on_disk is None:
# Unreadable history: never overwrite saved records with a
# partial view. Keep the new record in memory for a retry.
self._history = ([record] + self._history)[:self._max_history]
self._unsaved = pending
return
# Retried records may be older than ones other processes wrote
# since; sort so history stays newest-first before trimming.
merged = sorted(
pending + on_disk, key=lambda r: r.timestamp, reverse=True
)
self._history = merged[:self._max_history]
self._unsaved = [] if self._save_history() else pending
def get_history(
self,
@@ -392,6 +409,17 @@ class FileScheduleStore:
except Exception as e:
logger.warning("Failed to load schedule store from %s: %s", self._path, e)
def _save_or_rollback(self, job_id: str) -> None:
"""Persist jobs or restore on-disk state and raise (file lock held).
Keeps memory consistent with disk so a failed write never reports
success (or fires lifecycle hooks) for a change that will vanish on
restart.
"""
if not self._save():
self._reload_locked()
raise OSError(f"Failed to persist schedule job '{job_id}'")
def _save(self) -> bool:
"""Persist jobs to disk.
@@ -401,10 +429,7 @@ class FileScheduleStore:
try:
os.makedirs(self._dir, exist_ok=True)
data = [j.to_dict() for j in self._jobs.values()]
tmp = self._path + ".tmp"
with open(tmp, "w") as f:
json.dump(data, f, indent=2)
os.replace(tmp, self._path)
atomic_write_json(self._path, data, indent=2)
return True
except Exception as e:
logger.warning("Failed to save schedule store to %s: %s", self._path, e)
@@ -412,25 +437,27 @@ class FileScheduleStore:
def _load_history(self) -> None:
"""Load execution history from history.json."""
self._history = self._read_history() or []
def _read_history(self) -> Optional[List[RunRecord]]:
"""Read history.json; ``[]`` if absent, ``None`` if unreadable."""
if not os.path.exists(self._history_path):
return
return []
try:
with open(self._history_path, "r") as f:
data = json.load(f)
for d in data:
record = RunRecord.from_dict(d)
self._history.append(record)
return [RunRecord.from_dict(d) for d in data]
except Exception as e:
logger.warning("Failed to load history from %s: %s", self._history_path, e)
return None
def _save_history(self) -> None:
"""Save execution history to history.json."""
def _save_history(self) -> bool:
"""Save execution history to history.json; ``True`` on success."""
try:
os.makedirs(self._dir, exist_ok=True)
data = [r.to_dict() for r in self._history]
tmp = self._history_path + ".tmp"
with open(tmp, "w") as f:
json.dump(data, f, indent=2)
os.replace(tmp, self._history_path)
atomic_write_json(self._history_path, data, indent=2)
return True
except Exception as e:
logger.warning("Failed to save history to %s: %s", self._history_path, e)
return False
@@ -23,6 +23,8 @@ logger = logging.getLogger(__name__)
CACHE_SUBDIR = "remote-skills"
_CURRENT_LINK = "current"
# Deadline for git calls so a stalled remote falls back to the cached copy.
_GIT_TIMEOUT = 120
def get_remote_cache_root() -> Path:
@@ -89,7 +91,10 @@ class GitRemoteSkillSource:
if self.ref:
cmd += ["--branch", self.ref]
cmd += [self.url, str(clone_path)]
proc = subprocess.run(cmd, capture_output=True, text=True)
proc = subprocess.run(
cmd, capture_output=True, text=True, timeout=_GIT_TIMEOUT,
env={**os.environ, "GIT_TERMINAL_PROMPT": "0"},
)
if proc.returncode != 0:
logger.warning(
"Remote skill sync failed for %s: %s; using cached copy.",
@@ -123,7 +128,7 @@ class GitRemoteSkillSource:
def _resolve_version(self, clone_path: Path) -> str:
proc = subprocess.run(
["git", "-C", str(clone_path), "rev-parse", "HEAD"],
capture_output=True, text=True,
capture_output=True, text=True, timeout=_GIT_TIMEOUT,
)
if proc.returncode == 0 and proc.stdout.strip():
return proc.stdout.strip()[:16]
@@ -120,6 +120,17 @@ class CircuitBreakerException(Exception):
super().__init__(f"Circuit breaker open for service '{service_name}', state: {state.value}")
class _NeutralOutcome(Exception):
"""Raised by a guarded callable whose outcome says nothing about the service's
health (e.g. a caller's bad input): recorded as neither success nor failure,
so it can neither open the circuit nor close a half-open one. Carries the
real outcome in ``value`` for the caller to unwrap."""
def __init__(self, value):
super().__init__("neutral outcome")
self.value = value
class CircuitBreaker:
"""Lightweight circuit breaker implementation.
@@ -226,6 +237,8 @@ class CircuitBreaker:
result = func(*args, **kwargs)
self._on_success()
return result
except _NeutralOutcome:
raise
except Exception as e:
self._on_failure()
raise
@@ -269,6 +282,8 @@ class CircuitBreaker:
result = func(*args, **kwargs)
self._on_success()
return result
except _NeutralOutcome:
raise
except Exception as e:
self._on_failure()
raise
@@ -298,43 +298,38 @@ class ShellTools:
result['stderr_artifact'] = stderr_path
return result
except _CommandCancelled:
except _CommandCancelled as e:
# A cooperative interrupt (/stop, Ctrl-C) fired while the child
# was still running. Kill the whole process group so descendants
# do not keep mutating state, then report an interrupted outcome.
self._kill_process_tree(process)
return {
'stdout': '',
'stderr': 'Command interrupted by user',
'exit_code': -1,
'success': False,
'interrupted': True,
'execution_time': time.time() - start_time
}
result = self._partial_result(
e, 'Command interrupted by user', max_output_size, spill, spill_dir
)
result.update(interrupted=True, execution_time=time.time() - start_time)
return result
except _StreamDrainTimeout:
except _StreamDrainTimeout as e:
# The direct child already exited; only an inherited pipe kept a
# reader blocked. Report a timeout but do NOT kill — the direct
# child's PID is gone (and may have been recycled), so killing it
# would be unsafe and misleading.
return {
'stdout': '',
'stderr': f'Command output streaming timed out after {timeout} seconds',
'exit_code': -1,
'success': False,
'execution_time': timeout
}
result = self._partial_result(
e, f'Command output streaming timed out after {timeout} seconds',
max_output_size, spill, spill_dir,
)
result.update(timed_out=True, execution_time=timeout)
return result
except subprocess.TimeoutExpired:
except subprocess.TimeoutExpired as e:
# Kill process on timeout
self._kill_process_tree(process)
return {
'stdout': '',
'stderr': f'Command timed out after {timeout} seconds',
'exit_code': -1,
'success': False,
'execution_time': timeout
}
result = self._partial_result(
e, f'Command timed out after {timeout} seconds',
max_output_size, spill, spill_dir,
)
result.update(timed_out=True, execution_time=timeout)
return result
except Exception as e:
# Name the exception type and log the traceback. Some exceptions
@@ -356,6 +351,28 @@ class ShellTools:
'execution_time': 0
}
def _partial_result(self, exc, reason, max_output_size, spill, spill_dir):
"""Build a failed result that keeps output captured before ``exc``."""
collect = getattr(exc, 'collect_output', None)
stdout, stderr = collect() if collect else ('', '')
stdout, stdout_path = self._handle_overflow(
stdout, "stdout", max_output_size, spill, spill_dir
)
stderr, stderr_path = self._handle_overflow(
stderr, "stderr", max_output_size, spill, spill_dir
)
result: Dict[str, Union[str, int, bool]] = {
'stdout': stdout,
'stderr': f"{stderr.rstrip()}\n{reason}" if stderr.strip() else reason,
'exit_code': -1,
'success': False,
}
if stdout_path:
result['stdout_artifact'] = stdout_path
if stderr_path:
result['stderr_artifact'] = stderr_path
return result
def _start_background(self, command, cwd, process_env):
"""Start *command* detached, writing stdout+stderr to a log file."""
import tempfile
@@ -602,18 +619,31 @@ class ShellTools:
t_out.start()
t_err.start()
def _with_partial(exc, drain=True):
# Keep whatever the command already printed: on a timeout/interrupt
# the real error usually lives in the output produced so far. The
# caller collects only AFTER killing the child, so the pipes close
# and readline() returns trailing text written without a newline.
def _collect():
if drain:
for reader in (t_out, t_err):
reader.join(timeout=_DRAIN_GRACE_SECONDS)
return "".join(stdout_chunks), "".join(stderr_chunks)
exc.collect_output = _collect
return exc
deadline = None if timeout is None else time.monotonic() + timeout
if cancel_event is None:
try:
process.wait(timeout=timeout)
except subprocess.TimeoutExpired:
except subprocess.TimeoutExpired as exc:
# The direct child is still running. Disable any further emission
# and re-raise TimeoutExpired so the caller's timeout path KILLS
# the process before returning — a recorded read error must not
# short-circuit that cleanup (otherwise the still-running child
# leaks with its pipes open).
stop_emitting.set()
raise
raise _with_partial(exc)
else:
# Poll in short slices so a cooperative interrupt aborts the child
# promptly (within the poll interval) rather than only on timeout.
@@ -621,12 +651,12 @@ class ShellTools:
while True:
if cancel_event.is_set():
stop_emitting.set()
raise _CommandCancelled()
raise _with_partial(_CommandCancelled())
if deadline is not None:
remaining = deadline - time.monotonic()
if remaining <= 0:
stop_emitting.set()
raise subprocess.TimeoutExpired(process.args, timeout)
raise _with_partial(subprocess.TimeoutExpired(process.args, timeout))
wait_slice = min(_POLL, remaining)
else:
wait_slice = _POLL
@@ -658,7 +688,7 @@ class ShellTools:
# Stop emitting and signal a drain-timeout so the caller reports a
# timeout WITHOUT trying to kill an already-exited process.
stop_emitting.set()
raise _StreamDrainTimeout()
raise _with_partial(_StreamDrainTimeout(), drain=False)
if read_errors:
raise read_errors[0]
@@ -443,12 +443,33 @@ def _crawl_with_httpx(urls: List[str]) -> List[Dict[str, Any]]:
else:
raise ValueError("Too many redirects")
except ImportError:
# Fallback to urllib
# Fallback to urllib, walking redirects by hand so every hop
# is SSRF-validated before it is requested (as above).
import urllib.error
import urllib.request
if not _is_safe_crawl_url(url):
raise ValueError("URL blocked by SSRF policy")
with urllib.request.urlopen(url, timeout=30) as response:
content = response.read(_MAX_CRAWL_BYTES).decode('utf-8', errors='ignore')
class _NoRedirect(urllib.request.HTTPRedirectHandler):
def redirect_request(self, *args, **kwargs):
return None # surface 30x as HTTPError
opener = urllib.request.build_opener(_NoRedirect)
current = url
for _ in range(max_redirects + 1):
if not _is_safe_crawl_url(current):
raise ValueError("Redirect target failed SSRF validation")
try:
with opener.open(current, timeout=30) as response:
content = response.read(_MAX_CRAWL_BYTES).decode('utf-8', errors='ignore')
break
except urllib.error.HTTPError as e:
location = e.headers.get("Location")
if e.code not in redirect_statuses or not location:
raise
e.close() # release the redirect response's socket
current = urllib.parse.urljoin(current, location)
else:
raise ValueError("Too many redirects")
base_url = current
# Convert HTML to structured Markdown using the standard library so
# headings, lists, links, and code blocks survive on this path.
@@ -6,11 +6,11 @@ Exposes PraisonAI Agents via the A2A (Agent2Agent) protocol.
import asyncio
from praisonaiagents._logging import get_logger
from typing import List, Optional, TYPE_CHECKING
from typing import Dict, List, Optional, TYPE_CHECKING
from praisonaiagents.ui.a2a.types import AgentCard
from praisonaiagents.ui.a2a.agent_card import generate_agent_card
from praisonaiagents.ui.a2a.task_store import TaskStore
from praisonaiagents.ui.a2a.task_store import TaskStore, TERMINAL_STATES
if TYPE_CHECKING:
from praisonaiagents import Agent, AgentTeam
@@ -98,6 +98,8 @@ class A2A:
# Initialize task store
self.task_store = TaskStore()
# Strong refs to background tasks so they aren't GC'd and can be cancelled
self._running: Dict[str, asyncio.Task] = {}
# Router cache
self._router: Optional["APIRouter"] = None
@@ -290,12 +292,17 @@ class A2A:
self.task_store.add_to_history(task_id, response_msg)
self.task_store.add_artifact(task_id, artifact)
self.task_store.update_status(task_id, TaskState.COMPLETED)
except asyncio.CancelledError:
self.task_store.update_status(task_id, TaskState.CANCELLED)
raise
except Exception as e:
logger.error(f"Background agent error: {e}")
self.task_store.update_status(task_id, TaskState.FAILED)
user_input = extract_user_input([message])
asyncio.create_task(_background_process(task.id, user_input))
bg = asyncio.create_task(_background_process(task.id, user_input))
self._running[task.id] = bg
bg.add_done_callback(lambda _t, tid=task.id: self._running.pop(tid, None))
return JSONResponse(content={
"jsonrpc": "2.0",
@@ -305,6 +312,9 @@ class A2A:
# Update to working
self.task_store.update_status(task.id, TaskState.WORKING)
# Keep the task while we wait on the agent: a cancel can make it
# terminal (and evictable) before we build the response.
self.task_store._pinned.add(task.id)
try:
# Extract user input text
@@ -333,6 +343,8 @@ class A2A:
except Exception as e:
logger.error(f"Agent error in message/send: {e}")
updated_task = self.task_store.update_status(task.id, TaskState.FAILED)
finally:
self.task_store._pinned.discard(task.id)
# Return JSON-RPC response
return JSONResponse(content={
@@ -410,9 +422,19 @@ class A2A:
if not task_id:
return _jsonrpc_error(request_id, -32602, "Invalid params: 'id' required")
task = self.task_store.cancel_task(task_id)
if not task:
existing = self.task_store.get_task(task_id)
if not existing:
return _jsonrpc_error(request_id, -32000, f"Task not found: {task_id}")
if existing.status.state in TERMINAL_STATES:
return _jsonrpc_error(
request_id, -32002,
f"Task not cancelable: already {existing.status.state.value}",
)
task = self.task_store.cancel_task(task_id)
running = self._running.get(task_id)
if running:
running.cancel()
return JSONResponse(content={
"jsonrpc": "2.0",
@@ -6,7 +6,7 @@ In-memory storage for A2A Task lifecycle management.
import uuid
from datetime import datetime, timezone
from typing import Dict, List, Optional
from typing import Dict, List, Optional, Set
from praisonaiagents.ui.a2a.types import (
Task,
@@ -16,6 +16,12 @@ from praisonaiagents.ui.a2a.types import (
Artifact,
)
# Terminal states are final: once reached, status can no longer change.
TERMINAL_STATES = frozenset({TaskState.COMPLETED, TaskState.FAILED, TaskState.CANCELLED})
# Upper bound on stored tasks; oldest terminal tasks are evicted beyond this.
_MAX_TASKS = 10_000
class TaskStore:
"""
@@ -28,6 +34,8 @@ class TaskStore:
def __init__(self):
"""Initialize empty task store."""
self._tasks: Dict[str, Task] = {}
# Ids of tasks a request is still waiting on; never evicted.
self._pinned: Set[str] = set()
def create_task(self, message: Message, context_id: Optional[str] = None) -> Task:
"""
@@ -59,8 +67,22 @@ class TaskStore:
)
self._tasks[task_id] = task
self._evict()
return task
def _evict(self, keep: Optional[str] = None):
"""Drop the oldest terminal tasks (except *keep*) once over the cap."""
excess = len(self._tasks) - _MAX_TASKS
if excess <= 0:
return
old = [
t.id for t in self._tasks.values()
if t.status.state in TERMINAL_STATES
and t.id != keep and t.id not in self._pinned
]
for tid in old[:excess]:
del self._tasks[tid]
def get_task(self, task_id: str) -> Optional[Task]:
"""
Retrieve a task by its ID.
@@ -88,11 +110,14 @@ class TaskStore:
message: Optional status message
Returns:
Updated Task if found, None otherwise
Updated Task if found, None otherwise. A task already in a
terminal state is returned unchanged.
"""
task = self._tasks.get(task_id)
if not task:
return None
if task.status.state in TERMINAL_STATES:
return task
# Create new status
new_status = TaskStatus(
@@ -112,6 +137,8 @@ class TaskStore:
)
self._tasks[task_id] = updated
if state in TERMINAL_STATES:
self._evict(keep=task_id)
return updated
def add_artifact(self, task_id: str, artifact: Artifact) -> Optional[Task]:
@@ -123,11 +150,14 @@ class TaskStore:
artifact: Artifact to add
Returns:
Updated Task if found, None otherwise
Updated Task if found, None otherwise. A task in a terminal
state is returned unchanged.
"""
task = self._tasks.get(task_id)
if not task:
return None
if task.status.state in TERMINAL_STATES:
return task
# Get existing artifacts or create new list
artifacts = list(task.artifacts) if task.artifacts else []
@@ -155,11 +185,14 @@ class TaskStore:
message: Message to add to history
Returns:
Updated Task if found, None otherwise
Updated Task if found, None otherwise. A task in a terminal
state is returned unchanged.
"""
task = self._tasks.get(task_id)
if not task:
return None
if task.status.state in TERMINAL_STATES:
return task
# Get existing history or create new list
history = list(task.history) if task.history else []
@@ -120,4 +120,28 @@ async def run_sync_in_executor(func, *args, **kwargs) -> Any:
return func(*args, **kwargs)
return await loop.run_in_executor(None, wrapper)
else:
return await loop.run_in_executor(None, func, *args)
return await loop.run_in_executor(None, func, *args)
# Strong refs to fire-and-forget tasks: the event loop only keeps weak refs,
# so an unreferenced task can be garbage-collected before it finishes.
_background_tasks: "set[asyncio.Task]" = set()
def spawn_background(coro: Awaitable[Any], label: str = "background task") -> "asyncio.Task":
"""
Schedule *coro* on the running loop without awaiting it.
Keeps a strong reference until the task finishes and logs any exception
with *label*, instead of leaving it to asyncio's "Task exception was never
retrieved" warning at GC time. Must be called with a running loop.
"""
task = asyncio.get_running_loop().create_task(coro)
_background_tasks.add(task)
def _done(t: "asyncio.Task") -> None:
_background_tasks.discard(t)
if not t.cancelled() and t.exception() is not None:
logger.error("Error in %s: %s", label, t.exception(), exc_info=t.exception())
task.add_done_callback(_done)
return task
+2 -2
View File
@@ -51,7 +51,7 @@ parallel-search = [
memory = [
"chromadb>=1.0.0",
"litellm>=1.83.14",
"litellm>=1.83.14,<1.106", # 1.106 requires mcp>=2.2; mcp is pinned <2
]
knowledge = [
@@ -69,7 +69,7 @@ graph = [
# Add LLM dependencies
llm = [
"litellm>=1.83.14",
"litellm>=1.83.14,<1.106", # 1.106 requires mcp>=2.2; mcp is pinned <2
"pydantic>=2.10.0"
]
+9 -3
View File
@@ -388,7 +388,7 @@ class TestA2AJsonRpc:
assert data["error"]["code"] == -32000
def test_post_a2a_tasks_cancel(self):
"""tasks/cancel → cancels existing task."""
"""tasks/cancel on a completed task → -32002 TaskNotCancelable."""
from fastapi.testclient import TestClient
app, a2a = _make_app()
@@ -421,8 +421,14 @@ class TestA2AJsonRpc:
assert cancel_resp.status_code == 200
data = cancel_resp.json()
assert "result" in data
assert data["result"]["status"]["state"] == "cancelled"
assert data["error"]["code"] == -32002
# The completed task keeps its terminal state
get_resp = client.post("/a2a", json={
"jsonrpc": "2.0", "method": "tasks/get", "id": "3",
"params": {"id": task_id},
})
assert get_resp.json()["result"]["status"]["state"] == "completed"
def test_post_a2a_message_stream(self):
"""message/stream → returns SSE streaming response."""
@@ -0,0 +1,139 @@
"""tasks/cancel must stop background work and stay final (issue #5990)."""
import asyncio
import threading
import pytest
from praisonaiagents.ui.a2a.task_store import TaskStore
from praisonaiagents.ui.a2a.types import Message, Role, TextPart, TaskState
def _msg():
return Message(message_id="m1", role=Role.USER, parts=[TextPart(text="hi")])
def test_terminal_states_are_final():
store = TaskStore()
t = store.create_task(_msg())
store.update_status(t.id, TaskState.WORKING)
store.cancel_task(t.id)
store.update_status(t.id, TaskState.COMPLETED)
assert store.get_task(t.id).status.state == TaskState.CANCELLED
t2 = store.create_task(_msg())
store.update_status(t2.id, TaskState.COMPLETED)
store.cancel_task(t2.id)
assert store.get_task(t2.id).status.state == TaskState.COMPLETED
def test_store_evicts_oldest_terminal_tasks(monkeypatch):
from praisonaiagents.ui.a2a import task_store as ts
monkeypatch.setattr(ts, "_MAX_TASKS", 3)
store = TaskStore()
done = store.create_task(_msg())
store.update_status(done.id, TaskState.COMPLETED)
live = [store.create_task(_msg()) for _ in range(3)]
assert store.get_task(done.id) is None
assert all(store.get_task(t.id) for t in live)
@pytest.mark.asyncio
async def test_cancel_stops_background_task_and_is_not_overwritten():
from praisonaiagents import Agent
from praisonaiagents.ui.a2a import A2A
release = threading.Event()
agent = Agent(name="Cancel Test", role="Tester", goal="Test")
agent.chat = lambda _text: (release.wait(5), "late result")[1]
a2a = A2A(agent=agent, url="http://localhost:8000/a2a")
resp = await a2a._handle_message_send("1", {
"message": {"messageId": "m1", "role": "user", "parts": [{"text": "Hi"}]},
"configuration": {"returnImmediately": True},
})
import json
task_id = json.loads(resp.body)["result"]["id"]
await asyncio.sleep(0.05)
running = a2a._running[task_id]
a2a._handle_tasks_cancel("2", {"id": task_id})
release.set()
with pytest.raises(asyncio.CancelledError):
await running
task = a2a.task_store.get_task(task_id)
assert task.status.state == TaskState.CANCELLED
assert not task.artifacts
assert task_id not in a2a._running
def test_store_evicts_when_tasks_finish(monkeypatch):
from praisonaiagents.ui.a2a import task_store as ts
monkeypatch.setattr(ts, "_MAX_TASKS", 2)
store = TaskStore()
live = [store.create_task(_msg()) for _ in range(4)]
for t in live:
store.update_status(t.id, TaskState.COMPLETED)
assert len(store.list_tasks()) == 2
# The task that just finished is kept for the caller's response.
assert store.get_task(live[-1].id) is not None
@pytest.mark.asyncio
async def test_cancel_during_sync_send_discards_late_result():
import json
from praisonaiagents import Agent
from praisonaiagents.ui.a2a import A2A
started, release = threading.Event(), threading.Event()
agent = Agent(name="Cancel Test", role="Tester", goal="Test")
agent.chat = lambda _text: (started.set(), release.wait(5), "late result")[2]
a2a = A2A(agent=agent, url="http://localhost:8000/a2a")
send = asyncio.create_task(a2a._handle_message_send("1", {
"message": {"messageId": "m1", "role": "user", "parts": [{"text": "Hi"}]},
}))
await asyncio.to_thread(started.wait, 5)
task_id = a2a.task_store.list_tasks()[0].id
a2a._handle_tasks_cancel("2", {"id": task_id})
release.set()
result = json.loads((await send).body)["result"]
task = a2a.task_store.get_task(task_id)
assert result["status"]["state"] == TaskState.CANCELLED.value
assert task.status.state == TaskState.CANCELLED
assert not task.artifacts
assert all(m.role == Role.USER for m in task.history or [])
@pytest.mark.asyncio
async def test_cancelled_sync_send_survives_eviction(monkeypatch):
import json
from praisonaiagents import Agent
from praisonaiagents.ui.a2a import A2A
from praisonaiagents.ui.a2a import task_store as ts
monkeypatch.setattr(ts, "_MAX_TASKS", 2)
started, release = threading.Event(), threading.Event()
agent = Agent(name="Cancel Test", role="Tester", goal="Test")
agent.chat = lambda _text: (started.set(), release.wait(5), "late result")[2]
a2a = A2A(agent=agent, url="http://localhost:8000/a2a")
send = asyncio.create_task(a2a._handle_message_send("1", {
"message": {"messageId": "m1", "role": "user", "parts": [{"text": "Hi"}]},
}))
await asyncio.to_thread(started.wait, 5)
task_id = a2a.task_store.list_tasks()[0].id
a2a._handle_tasks_cancel("2", {"id": task_id})
# Other tasks finish and push the store over the cap while we wait.
for _ in range(3):
other = a2a.task_store.create_task(_msg())
a2a.task_store.update_status(other.id, TaskState.COMPLETED)
assert a2a.task_store.get_task(task_id) is not None
release.set()
result = json.loads((await send).body)["result"]
assert result["id"] == task_id
assert result["status"]["state"] == TaskState.CANCELLED.value
assert task_id not in a2a.task_store._pinned
@@ -0,0 +1,159 @@
"""Tool guards on a shared Agent must not punish the wrong caller (issue #5860).
1. Each chat turn gets its own loop guard, so overlapping turns never exhaust
each other's per-turn limits.
2. Caller errors (ValueError/TypeError/AttributeError from bad input) do not
open the tool circuit breaker for every other caller; real failures still do.
"""
import asyncio
import threading
import pytest
@pytest.fixture
def agent_factory(monkeypatch):
from praisonaiagents import Agent
monkeypatch.setenv("OPENAI_API_KEY", "test-key")
def make(tools):
return Agent(name="shared", instructions="x", tools=tools, llm="gpt-4o-mini")
return make
def _lookup(path: str) -> str:
"""Read something (distinct path each call)."""
return f"read {path}"
def test_overlapping_turns_get_independent_loop_guards(agent_factory):
from praisonaiagents.agent.chat_mixin import _enter_turn, _exit_turn
agent = agent_factory([_lookup])
agent_guard = agent._ensure_loop_guard()
outer = _enter_turn(agent, None) # a long-running turn stays in flight
try:
outer_guard = agent._ensure_loop_guard()
assert outer_guard is not agent_guard
assert outer_guard.config is agent_guard.config
outer_guard.record("lookup", {"q": "x"}, True, "same")
seen = {}
def other_turn():
tok = _enter_turn(agent, None)
try:
seen["guard"] = agent._ensure_loop_guard()
finally:
_exit_turn(agent, tok)
th = threading.Thread(target=other_turn)
th.start()
th.join(5)
assert seen["guard"] is not outer_guard
assert seen["guard"]._tool_counts == {}
assert outer_guard._tool_counts == {"lookup": 1}
finally:
_exit_turn(agent, outer)
# Outside a turn the agent-level guard is used, as before.
assert agent._ensure_loop_guard() is agent_guard
def test_many_overlapping_turns_are_not_halted(agent_factory):
from praisonaiagents.agent.chat_mixin import _enter_turn, _exit_turn
agent = agent_factory([_lookup])
agent._ensure_loop_guard()
tokens = []
try:
for i in range(20): # 20 concurrent turns, one tool call each
tokens.append(_enter_turn(agent, None))
result = agent.execute_tool("_lookup", {"path": f"/tmp/{i}"})
assert result == f"read /tmp/{i}"
finally:
for tok in reversed(tokens):
_exit_turn(agent, tok)
def _get_order(order_id: str) -> str:
"""Look up an order. Raises for unknown ids."""
if not order_id.startswith("A"):
raise ValueError(f"unknown order {order_id}")
return f"order {order_id}: shipped"
def _down(x: str) -> str:
"""A tool whose dependency is down."""
raise RuntimeError("dependency down")
def _sync_run(agent, name, args):
try:
return agent.execute_tool(name, args)
except Exception as e: # surfaced ToolExecutionError
return str(e)
def _async_run(agent, name, args):
try:
return asyncio.run(agent.execute_tool_async(name, args))
except Exception as e:
return str(e)
@pytest.mark.parametrize("run", [_sync_run, _async_run], ids=["sync", "async"])
def test_bad_input_does_not_open_circuit_breaker(agent_factory, run):
agent = agent_factory([_get_order])
agent._ensure_loop_guard().config.enabled = False
for i in range(6):
run(agent, "_get_order", {"order_id": f"bad{i}"})
assert run(agent, "_get_order", {"order_id": "A100"}) == "order A100: shipped"
@pytest.mark.parametrize("run", [_sync_run, _async_run], ids=["sync", "async"])
def test_real_failures_still_open_circuit_breaker(agent_factory, run):
agent = agent_factory([_down])
agent._ensure_loop_guard().config.enabled = False
for _ in range(6):
result = run(agent, "_down", {"x": "1"})
assert "circuit breaker open" in str(result)
@pytest.mark.parametrize("run", [_sync_run, _async_run], ids=["sync", "async"])
def test_bad_input_does_not_close_half_open_breaker(agent_factory, run):
from praisonaiagents.tools.circuit_breaker import CircuitState, _get_global_registry
agent = agent_factory([_get_order])
agent._ensure_loop_guard().config.enabled = False
run(agent, "_get_order", {"order_id": "A1"}) # create the breaker
breaker = _get_global_registry().get(f"tool_{id(agent)}__get_order")
with breaker._lock:
breaker._stats.state = CircuitState.HALF_OPEN
breaker._stats.success_count = 0
for i in range(breaker.config.success_threshold + 1):
run(agent, "_get_order", {"order_id": f"bad{i}"})
assert breaker.state == CircuitState.HALF_OPEN
assert breaker.stats.success_count == 0
def test_streams_get_their_own_loop_guard(agent_factory, monkeypatch):
agent = agent_factory([_lookup])
agent_guard = agent._ensure_loop_guard()
seen = []
def fake_impl(prompt, **kwargs):
guard = agent._ensure_loop_guard()
guard.record("lookup", {"q": prompt}, True, "same")
seen.append(guard)
yield prompt
monkeypatch.setattr(agent, "_start_stream_impl", fake_impl)
assert list(agent._start_stream("a")) == ["a"]
assert list(agent._start_stream("b")) == ["b"]
assert seen[0] is not seen[1] and agent_guard not in seen
assert seen[1]._tool_counts == {"lookup": 1}
assert agent._ensure_loop_guard() is agent_guard
assert agent_guard._tool_counts == {}
@@ -497,3 +497,22 @@ class TestEventType:
bus.publish(EventType.SESSION_CREATED, {})
assert len(received) == 1
class TestAsyncSubscriberErrors:
"""In-loop async subscriber failures are logged by the bus (issue #5990)."""
@pytest.mark.asyncio
async def test_async_subscriber_error_is_logged(self, caplog):
import logging
bus = EventBus()
async def boom(event):
raise ValueError("boom")
sub_id = bus.subscribe(boom)
with caplog.at_level(logging.ERROR):
bus.publish("test.event", {})
for _ in range(5):
await asyncio.sleep(0)
assert any(sub_id in r.getMessage() and "boom" in r.getMessage() for r in caplog.records)
@@ -7,6 +7,14 @@ do not require live network access or a browser.
import pytest
@pytest.fixture
def public_dns(monkeypatch):
"""Treat example.com hosts as public without needing live DNS."""
monkeypatch.setattr(
"praisonaiagents.tools.url_safety.is_safe_http_url", lambda url: True
)
class TestParseWWWAuthenticate:
def test_none_returns_empty(self):
from praisonaiagents.mcp.oauth_discovery import parse_www_authenticate
@@ -31,7 +39,7 @@ class TestHttpsGuard:
with pytest.raises(ValueError):
_require_https("http://evil.example.com/.well-known/x")
def test_allows_https(self):
def test_allows_https(self, public_dns):
from praisonaiagents.mcp.oauth_discovery import _require_https
_require_https("https://api.example.com/.well-known/x")
@@ -50,7 +58,7 @@ class TestWellKnownCandidates:
class TestEndpointHttpsValidation:
def test_rejects_plaintext_advertised_endpoints(self, monkeypatch):
def test_rejects_plaintext_advertised_endpoints(self, monkeypatch, public_dns):
from praisonaiagents.mcp import oauth_discovery
monkeypatch.setattr(
@@ -64,10 +72,10 @@ class TestEndpointHttpsValidation:
"token_endpoint": "http://auth.example.com/token",
},
)
with pytest.raises(ValueError):
with pytest.raises(ValueError, match="insecure transport"):
oauth_discovery.discover_oauth_metadata("https://api.example.com/mcp")
def test_accepts_all_https_endpoints(self, monkeypatch):
def test_accepts_all_https_endpoints(self, monkeypatch, public_dns):
from praisonaiagents.mcp import oauth_discovery
monkeypatch.setattr(
@@ -87,32 +95,52 @@ class TestEndpointHttpsValidation:
class TestRedirectRevalidation:
def test_rejects_redirect_to_plaintext(self, monkeypatch):
def test_rejects_redirect_before_requesting_it(self, monkeypatch):
from praisonaiagents.mcp import oauth_discovery
class _Hop:
url = "http://internal.local/.well-known/x"
requested = []
class _Resp:
url = "http://internal.local/.well-known/x"
history = [_Hop()]
def raise_for_status(self):
pass
def json(self):
return {}
status_code = 302
headers = {"Location": "http://internal.local/.well-known/x"}
class _Requests:
@staticmethod
def get(*a, **k):
def get(url, **k):
requested.append(url)
assert k["allow_redirects"] is False
return _Resp()
monkeypatch.setitem(
__import__("sys").modules, "requests", _Requests()
)
with pytest.raises(ValueError):
oauth_discovery._get_json("https://api.example.com/.well-known/x")
oauth_discovery._get_json("http://127.0.0.1:8080/.well-known/x")
# The redirect target was refused without ever being fetched.
assert requested == ["http://127.0.0.1:8080/.well-known/x"]
class TestPrivateHostGuard:
@pytest.mark.parametrize("url", [
"https://169.254.169.254/latest/meta-data/",
"https://10.0.0.5/admin",
"https://[::ffff:169.254.169.254]/",
"https://0177.0.0.1/",
])
def test_rejects_internal_https_hosts(self, url, monkeypatch):
from praisonaiagents.mcp.oauth_discovery import _require_https
monkeypatch.delenv("ALLOW_LOCAL_CRAWL", raising=False)
with pytest.raises(ValueError):
_require_https(url)
def test_registration_rejects_internal_endpoint(self, monkeypatch):
from praisonaiagents.mcp.oauth_registration import register_client
monkeypatch.delenv("ALLOW_LOCAL_CRAWL", raising=False)
with pytest.raises(ValueError):
register_client(
"https://10.0.0.5/register",
redirect_uris=["http://127.0.0.1:19876/mcp/oauth/callback"],
)
class TestRegistrationGuard:
@@ -145,6 +173,29 @@ class TestProviderTokenReuse:
assert provider.get_valid_token() == "tok123"
assert provider.ensure_authenticated() == "tok123"
def test_refresh_redirect_keeps_saved_tokens(self, storage, monkeypatch, public_dns):
import requests
from praisonaiagents.mcp.oauth_provider import MCPOAuthProvider
url = "https://api.example.com/mcp"
tokens = {
"access_token": "old",
"refresh_token": "r1",
"token_endpoint": "https://auth.example.com/token",
}
storage.set_tokens("example", tokens, server_url=url)
storage.set_client_info("example", {"client_id": "c1"}, server_url=url)
resp = requests.Response()
resp.status_code = 307
resp.url = tokens["token_endpoint"]
resp._content = b'{"detail": "redirect"}'
monkeypatch.setattr(requests, "post", lambda *a, **k: resp)
provider = MCPOAuthProvider("example", url, storage=storage)
assert provider._try_refresh(storage.get_for_url("example", url)) is False
assert storage.get_for_url("example", url)["tokens"]["access_token"] == "old"
def test_expired_token_without_refresh_returns_none(self, storage):
import time
from praisonaiagents.mcp.oauth_provider import MCPOAuthProvider
@@ -100,5 +100,5 @@ class TestEstimateTokens:
from praisonaiagents.rag.compressor import estimate_tokens
assert estimate_tokens("") == 0
assert estimate_tokens("hello") == 2 # 5/4 + 1 = 2
assert estimate_tokens("a" * 100) == 26 # 100/4 + 1 = 26
assert estimate_tokens("hello") == 1 # canonical: 5 * 0.25, min 1
assert estimate_tokens("a" * 100) == 25
@@ -136,4 +136,4 @@ class TestEstimateTokensSummarizer:
from praisonaiagents.rag.summarizer import estimate_tokens
assert estimate_tokens("") == 0
assert estimate_tokens("test") == 2
assert estimate_tokens("test") == 1
@@ -0,0 +1,122 @@
"""Device-identity handshake contract (Issue #5998)."""
import base64
import pytest
from praisonaiagents.gateway import (
ConnectErrorCode,
DeviceAuth,
DeviceIdentityProtocol,
FrameDecodeError,
HelloParams,
decode_client_frame,
device_id_from_public_key,
)
PUB = "dGVzdC1wdWJsaWMta2V5"
def _hello(**device):
block = {
"device_id": device_id_from_public_key(PUB),
"public_key": PUB,
"signature": "sig",
"signed_at": 1700000000,
"nonce": "n-1",
}
block.update(device)
return {"type": "hello", "agent_id": "a", "device": block}
def test_hello_without_device_is_unchanged():
frame = decode_client_frame({"type": "hello", "agent_id": "a"})
assert isinstance(frame, HelloParams)
assert frame.device is None
def test_hello_decodes_device():
frame = decode_client_frame(_hello())
assert isinstance(frame.device, DeviceAuth)
assert frame.device.device_id == device_id_from_public_key(PUB)
assert frame.device.nonce == "n-1"
assert frame.device.signed_at == 1700000000
def test_spoofed_device_id_is_unauthorized():
with pytest.raises(FrameDecodeError) as exc:
decode_client_frame(_hello(device_id="0" * 64))
assert exc.value.error.code == ConnectErrorCode.AUTH_UNAUTHORIZED
@pytest.mark.parametrize("bad", [
{"signature": ""},
{"nonce": None},
{"signed_at": True},
{"signed_at": "now"},
{"signed_at": float("nan")},
{"signed_at": 1700000000.5},
{"public_key": "not base64!"},
{"public_key": "dGVzdC1wdWJsaWMta2V5\n"},
])
def test_malformed_device_rejected(bad):
with pytest.raises(FrameDecodeError) as exc:
decode_client_frame(_hello(**bad))
assert exc.value.error.code == ConnectErrorCode.CONFIGURATION_ERROR
def test_device_must_be_object():
with pytest.raises(FrameDecodeError):
decode_client_frame({"type": "hello", "agent_id": "a", "device": "x"})
def test_wire_format_is_pinned():
# Literal vectors: changing the fingerprint or signed bytes breaks clients.
assert device_id_from_public_key(PUB) == (
"c88e80202a4d650841e48649b4c3f553e48131b415db680476fd0d65632ff2b0"
)
d = DeviceAuth(device_id_from_public_key(PUB), PUB, "sig", 1700000000, "n-1")
assert d.signing_payload("a") == (
b"praisonai-device-v1\n"
b"c88e80202a4d650841e48649b4c3f553e48131b415db680476fd0d65632ff2b0\n"
b"n-1\n1700000000\na"
)
def test_signing_payload_binds_agent_and_nonce():
d = decode_client_frame(_hello()).device
assert d.signing_payload("a") != d.signing_payload("b")
other = DeviceAuth(d.device_id, d.public_key, d.signature, d.signed_at, "n-2")
assert d.signing_payload("a") != other.signing_payload("a")
def test_ed25519_round_trip():
ed = pytest.importorskip("cryptography.hazmat.primitives.asymmetric.ed25519")
from cryptography.hazmat.primitives import serialization
key = ed.Ed25519PrivateKey.generate()
raw = key.public_key().public_bytes(
serialization.Encoding.Raw, serialization.PublicFormat.Raw
)
pub = base64.b64encode(raw).decode()
unsigned = DeviceAuth(device_id_from_public_key(pub), pub, "", 1700000000, "nonce")
sig = base64.b64encode(key.sign(unsigned.signing_payload("a"))).decode()
frame = decode_client_frame({
"type": "hello", "agent_id": "a",
"device": {**unsigned.__dict__, "signature": sig},
})
ed.Ed25519PublicKey.from_public_bytes(base64.b64decode(frame.device.public_key)).verify(
base64.b64decode(frame.device.signature),
frame.device.signing_payload(frame.agent_id),
)
def test_protocol_is_runtime_checkable():
class Store:
def issue_challenge(self): return "n"
def verify(self, device, *, agent_id): return None
def list_devices(self): return []
def revoke(self, device_id): return False
assert isinstance(Store(), DeviceIdentityProtocol)
@@ -0,0 +1,78 @@
"""The TypeScript client's ``GatewayEventType`` must mirror Python's ``EventType``.
The gateway wire event vocabulary is defined once in
``praisonaiagents/gateway/protocols.py`` and re-declared in
``praisonai-ts/src/gateway/index.ts``. Nothing else ties the two together, so an
event type added (or renamed) on the Python side would otherwise reach the TS
client only as silently ignored or misrouted frames at runtime.
This is a lockstep test: it asserts that the two independently-maintained lists
agree member-for-member (name and wire value), so drift fails CI instead.
"""
import re
from pathlib import Path
import pytest
from praisonaiagents.gateway.protocols import EventType
_TS_GATEWAY = (
Path(__file__).resolve().parents[3]
/ "praisonai-ts"
/ "src"
/ "gateway"
/ "index.ts"
)
_TS_MEMBER = re.compile(r"""([A-Za-z_$][\w$]*)\s*=\s*(['"])(.*?)\2""", re.S)
def _parse_ts_enum(source: str) -> dict[str, str]:
"""Parse ``export enum GatewayEventType { NAME = 'value', ... }`` from TS source.
Every enum member must be a ``NAME = 'value'`` (or double-quoted) pair; any
other entry fails loudly rather than being silently skipped.
"""
body = re.search(r"export enum GatewayEventType\s*\{(.*?)\n\}", source, re.S)
assert body, "GatewayEventType enum not found"
text = re.sub(r"/\*.*?\*/|//[^\n]*", "", body.group(1), flags=re.S)
members = {}
for entry in filter(None, (e.strip() for e in text.split(","))):
match = _TS_MEMBER.fullmatch(entry)
assert match, f"Unparseable GatewayEventType entry: {entry!r}"
members[match.group(1)] = match.group(3)
return members
def _ts_event_types() -> dict[str, str]:
return _parse_ts_enum(_TS_GATEWAY.read_text(encoding="utf-8"))
def test_parse_ts_enum_rejects_or_captures_every_entry():
src = """export enum GatewayEventType {
// comment
A = 'a',
MESSAGE_V2 = 'message_v2',
B = "b",
}"""
assert _parse_ts_enum(src) == {"A": "a", "MESSAGE_V2": "message_v2", "B": "b"}
with pytest.raises(AssertionError, match="Unparseable"):
_parse_ts_enum("export enum GatewayEventType {\n A = 'a',\n C,\n}")
@pytest.mark.skipif(not _TS_GATEWAY.exists(), reason="TypeScript SDK not present")
def test_ts_gateway_event_type_matches_python():
python = {member.name: member.value for member in EventType}
ts = _ts_event_types()
missing_in_ts = sorted(set(python) - set(ts))
extra_in_ts = sorted(set(ts) - set(python))
mismatched = sorted(n for n in set(python) & set(ts) if python[n] != ts[n])
assert not (missing_in_ts or extra_in_ts or mismatched), (
"praisonai-ts GatewayEventType drifted from praisonaiagents EventType: "
f"missing in TS={missing_in_ts}, extra in TS={extra_in_ts}, "
f"value mismatch={mismatched}. Update src/praisonai-ts/src/gateway/index.ts."
)
@@ -0,0 +1,39 @@
"""ImageAgent must not send the base Agent's chat default model to image APIs (#5944)."""
import sys
import types
import pytest
from praisonaiagents import ImageAgent
@pytest.mark.parametrize("model_kwargs,expected", [
({}, {"generate": "dall-e-3", "edit": "dall-e-2", "variation": "dall-e-2"}),
({"llm": "openai/gpt-image-1"}, dict.fromkeys(("generate", "edit", "variation"), "openai/gpt-image-1")),
({"model": "openai/gpt-image-1"}, dict.fromkeys(("generate", "edit", "variation"), "openai/gpt-image-1")),
])
def test_image_model_resolution(monkeypatch, tmp_path, model_kwargs, expected):
calls = {}
def record(name):
def call(**kwargs):
calls[name] = kwargs
return {"data": []}
return call
agent = ImageAgent(instructions="Create images", verbose=False, **model_kwargs)
monkeypatch.setattr(agent, "_litellm", record("generate"))
monkeypatch.setitem(sys.modules, "litellm", types.SimpleNamespace(
image_edit=record("edit"),
image_variation=record("variation"),
get_llm_provider=lambda model: (model, "openai", None, None),
))
image = tmp_path / "source.png"
image.write_bytes(b"png")
agent.generate("A red circle")
agent.edit(str(image), "Add clouds")
agent.variation(str(image))
assert {name: kwargs["model"] for name, kwargs in calls.items()} == expected
@@ -0,0 +1,125 @@
"""Hot paths use the shared error classifier, token estimator and atomic writer (#5838)."""
import json
import os
import pytest
class _E(Exception):
def __init__(self, msg, status_code=None, headers=None):
super().__init__(msg)
self.status_code = status_code
class _R:
pass
self.response = _R()
self.response.headers = headers or {}
@pytest.fixture
def llm():
from praisonaiagents.llm.llm import LLM
inst = LLM.__new__(LLM)
inst._rate_limiter = None
inst._retry_delay = 1.0
inst._max_retries = 3
return inst
@pytest.mark.parametrize("error, kind", [
(_E("max_tokens must be <= 4096, got 8500 (request req_5003abc)", 400), "format_error"),
(_E("Error code: 401 - Incorrect credentials", 401), "auth"),
(_E("Error code: 403 - region not supported", 403), "auth"),
(_E("Service Error", 503), "overloaded"),
(_E("Error code: 404 - The model `gpt-9` does not exist", 404), "model_not_found"),
(_E("Error code: 400 - invalid image: credit card detected", 400), "format_error"),
(_E("You exceeded your quota: insufficient quota", 429), "billing"),
(_E("Your credit balance is too low", 400), "billing"),
(_E("Rate limit reached for API key sk-...", 429), "rate_limit"),
(Exception("503 service unavailable"), "overloaded"),
])
def test_status_code_drives_error_kind(llm, error, kind):
assert llm.classify_error_kind(error) == kind
def test_retry_after_header_is_honoured(llm):
err = _E("Error code: 429", 429, {"retry-after": "37"})
assert llm._parse_retry_delay(err) == 37.0
decision = llm.resolve_failover_decision(err, {"attempt": 1, "max_retries": 3})
assert decision.action == "retry"
assert decision.backoff_ms == 37000
def test_budget_estimators_count_cjk_densely():
from praisonaiagents.context.tokens import estimate_tokens_heuristic
from praisonaiagents.rag.budget import estimate_tokens
text = "今天天气很好,我们一起去公园散步。" * 100
assert estimate_tokens(text) == estimate_tokens_heuristic(text)
assert estimate_tokens(text) > len(text)
def test_file_tracker_failed_save_keeps_previous_state(tmp_path):
from praisonaiagents.knowledge.indexing import FileTracker
state = str(tmp_path / "tracker.json")
tracker = FileTracker(state)
tracker._tracked = {"a.md": {"hash": "1", "mtime": 1, "memory_ids": ["m1"]}}
tracker.save()
tracker._tracked["b.md"] = {"hash": "2", "mtime": 2, "memory_ids": {"m3"}}
with pytest.raises(TypeError):
tracker.save()
reloaded = FileTracker(state)
reloaded.load()
assert list(reloaded._tracked) == ["a.md"]
def test_permission_allowlist_failed_save_and_corrupt_load(tmp_path):
from praisonaiagents.approval import PermissionAllowlist
path = str(tmp_path / "perm.json")
allowlist = PermissionAllowlist()
allowlist.add_tool("shell", paths=["/tmp"])
allowlist.save(path)
allowlist._tools["shell"].allowed_paths = [object()]
with pytest.raises(TypeError):
allowlist.save(path)
assert PermissionAllowlist.load(path).list_tools() == ["shell"]
with open(path, "w") as f:
f.write("{truncated")
assert PermissionAllowlist.load(path).is_empty()
assert not os.path.exists(path)
def test_file_index_save_is_atomic(tmp_path):
from praisonaiagents.context.fast.index_manager import FileIndex
index = FileIndex(workspace_path=str(tmp_path))
assert index.save()
with open(index.index_path) as f:
assert json.load(f)["workspace"] == str(tmp_path)
def test_cjk_truncation_stays_within_token_budget():
from praisonaiagents.rag.budget import DefaultBudgetEnforcer, TokenBudget, estimate_tokens
from praisonaiagents.rag.compressor import ContextCompressor
from praisonaiagents.rag.context import truncate_context
cjk = "汉" * 1000
out = truncate_context(cjk, 200)
assert out.endswith("[Context truncated...]")
assert estimate_tokens(out) <= 200
budget = TokenBudget(model_max_tokens=1200, reserved_response_tokens=0)
available = budget.dynamic_budget()
kept = DefaultBudgetEnforcer().enforce([{"text": cjk}, {"text": cjk}], budget)
assert sum(estimate_tokens(c["text"]) for c in kept) <= available
assert kept[-1]["metadata"]["_truncated"]
kept = ContextCompressor()._truncate_to_budget([{"text": cjk}], 200)
assert estimate_tokens(kept[0]["text"]) <= 200
@@ -0,0 +1,104 @@
"""Regression tests for issue #5870: observer isolation, atomic state writes, git deadlines."""
import asyncio
import json
import subprocess
import pytest
from praisonaiagents import main as _main
from praisonaiagents.approval import PermissionAllowlist
def test_raising_sync_callback_does_not_propagate():
def boom(**kw):
raise RuntimeError("callback bug")
_main.register_display_callback("interaction", boom)
_main.execute_sync_callback("interaction", message="hi") # must not raise
def test_raising_callbacks_do_not_propagate_async():
seen = []
def boom(**kw):
raise RuntimeError("sync bug")
async def aboom(message):
seen.append(message)
raise RuntimeError("async bug")
_main.register_display_callback("interaction", boom)
_main.register_display_callback("interaction", aboom, is_async=True)
asyncio.run(_main.execute_callback("interaction", message="hi", extra=1))
assert seen == ["hi"]
def test_allowlist_failed_save_keeps_previous_file(tmp_path):
p = str(tmp_path / "allow.json")
a = PermissionAllowlist()
a.add_tool("read_file", paths=["/ok"])
a.save(p)
good = open(p).read()
a._tools["read_file"].allowed_paths = ["/ok", object()]
with pytest.raises(TypeError):
a.save(p)
assert open(p).read() == good
assert PermissionAllowlist.load(p).list_tools() == ["read_file"]
def test_allowlist_load_quarantines_corrupt_file(tmp_path):
p = tmp_path / "allow.json"
p.write_text('{"tools": {')
assert PermissionAllowlist.load(str(p)).list_tools() == []
assert not p.exists()
assert list(tmp_path.glob("allow.json.*.corrupt"))
def _timeout_run(calls):
def fake_run(cmd, **kwargs):
calls.append(kwargs)
raise subprocess.TimeoutExpired(cmd, kwargs.get("timeout"))
return fake_run
def test_git_fetch_timeout_without_cache_returns_empty(tmp_path, monkeypatch):
from praisonaiagents.skills import remote
calls = []
monkeypatch.setattr(remote.subprocess, "run", _timeout_run(calls))
src = remote.GitRemoteSkillSource("http://127.0.0.1:1/org/skills.git")
assert src.fetch(tmp_path) == []
assert calls[0]["timeout"] == remote._GIT_TIMEOUT
assert calls[0]["env"]["GIT_TERMINAL_PROMPT"] == "0"
def test_git_fetch_timeout_falls_back_to_cache(tmp_path, monkeypatch):
from praisonaiagents.skills import remote
src = remote.GitRemoteSkillSource("http://127.0.0.1:1/org/skills.git")
skill = src._cache_base(tmp_path) / remote._CURRENT_LINK / "my-skill"
skill.mkdir(parents=True)
(skill / "SKILL.md").write_text("---\nname: my-skill\n---\n")
monkeypatch.setattr(remote.subprocess, "run", _timeout_run([]))
result = src.fetch(tmp_path)
assert len(result) == 1
assert (result[0] / "my-skill" / "SKILL.md").exists()
def test_config_dump_preserves_symlink_and_mode(tmp_path):
import os
from praisonaiagents.config.loader import _dump_config_file
shared = tmp_path / "shared.yaml"
shared.write_text("plugins: {}\n")
os.chmod(shared, 0o644)
link = tmp_path / "proj" / "config.yaml"
link.parent.mkdir()
link.symlink_to(shared)
_dump_config_file(link, {"plugins": {"enabled": ["x"]}})
assert link.is_symlink()
assert "x" in shared.read_text()
assert shared.stat().st_mode & 0o777 == 0o644
@@ -0,0 +1,154 @@
"""
Regression tests for FileScheduleStore persistence (issue #5899).
- Run history from two stores on the same dir is merged, not last-writer-wins.
- A failed save from add/update/remove raises, keeps memory consistent with
disk, and does not fire lifecycle hooks.
"""
import os
from unittest import mock
import pytest
from praisonaiagents.scheduler import store as store_mod
from praisonaiagents.scheduler.models import Schedule, ScheduleJob
from praisonaiagents.scheduler.store import FileScheduleStore
def _make_job(name="job"):
return ScheduleJob(
name=name,
schedule=Schedule(kind="every", every_seconds=60),
message="hello",
)
def test_history_not_lost_across_stores(tmp_path):
d = str(tmp_path)
a = FileScheduleStore(store_dir=d)
b = FileScheduleStore(store_dir=d)
a.log_run("j1", "succeeded", job_name="n1")
b.log_run("j2", "succeeded", job_name="n2")
ids = [r.job_id for r in FileScheduleStore(store_dir=d).get_history()]
assert ids == ["j2", "j1"]
def test_history_respects_max_after_reload(tmp_path):
d = str(tmp_path)
a = FileScheduleStore(store_dir=d, max_history=2)
b = FileScheduleStore(store_dir=d, max_history=2)
a.log_run("j1", "succeeded")
b.log_run("j2", "succeeded")
a.log_run("j3", "succeeded")
assert [r.job_id for r in a.get_history()] == ["j3", "j2"]
def _fail_replace(*args, **kwargs):
raise OSError("disk full")
def test_add_failed_save_raises_and_rolls_back(tmp_path):
d = str(tmp_path)
s = FileScheduleStore(store_dir=d)
with mock.patch("praisonaiagents.utils.atomic_io.os.replace", _fail_replace), \
mock.patch.object(store_mod, "emit_schedule_add") as emit:
with pytest.raises(OSError):
s.add(_make_job())
emit.assert_not_called()
assert s.list() == []
assert FileScheduleStore(store_dir=d).list() == []
assert not [f for f in os.listdir(d) if f.endswith(".tmp")]
def test_update_failed_save_restores_previous(tmp_path):
d = str(tmp_path)
s = FileScheduleStore(store_dir=d)
job = _make_job()
s.add(job)
changed = ScheduleJob.from_dict(job.to_dict())
changed.message = "changed"
with mock.patch("praisonaiagents.utils.atomic_io.os.replace", _fail_replace):
with pytest.raises(OSError):
s.update(changed)
assert s.get(job.id).message == "hello"
def test_remove_failed_save_keeps_job(tmp_path):
d = str(tmp_path)
s = FileScheduleStore(store_dir=d)
job = _make_job()
s.add(job)
with mock.patch("praisonaiagents.utils.atomic_io.os.replace", _fail_replace), \
mock.patch.object(store_mod, "emit_schedule_remove") as emit:
with pytest.raises(OSError):
s.remove(job.id)
with pytest.raises(OSError):
s.remove_by_name(job.name)
emit.assert_not_called()
assert s.get(job.id) is not None
def _log_many(d, prefix, n):
s = FileScheduleStore(store_dir=d, max_history=1000)
for i in range(n):
s.log_run(f"{prefix}{i}", "succeeded")
def test_history_overlapping_processes_keep_all_records(tmp_path):
import multiprocessing as mp
d = str(tmp_path)
ctx = mp.get_context("spawn")
procs = [ctx.Process(target=_log_many, args=(d, p, 20)) for p in ("a", "b")]
for p in procs:
p.start()
for p in procs:
p.join(60)
assert p.exitcode == 0
ids = {r.job_id for r in FileScheduleStore(store_dir=d, max_history=1000).get_history()}
assert ids == {f"{p}{i}" for p in ("a", "b") for i in range(20)}
def test_unreadable_history_is_not_overwritten(tmp_path):
d = str(tmp_path)
s = FileScheduleStore(store_dir=d)
s.log_run("j1", "succeeded")
with mock.patch.object(FileScheduleStore, "_read_history", return_value=None):
s.log_run("j2", "succeeded")
assert [r.job_id for r in FileScheduleStore(store_dir=d).get_history()] == ["j1"]
# The unsaved record is retried once history is readable again.
s.log_run("j3", "succeeded")
assert [r.job_id for r in FileScheduleStore(store_dir=d).get_history()] == ["j3", "j2", "j1"]
def test_failed_history_save_is_retried(tmp_path):
d = str(tmp_path)
s = FileScheduleStore(store_dir=d)
with mock.patch("praisonaiagents.utils.atomic_io.os.replace", _fail_replace):
s.log_run("j1", "succeeded")
s.log_run("j2", "succeeded")
assert [r.job_id for r in FileScheduleStore(store_dir=d).get_history()] == ["j2", "j1"]
def test_retried_record_does_not_hide_newer_records(tmp_path):
d = str(tmp_path)
a = FileScheduleStore(store_dir=d, max_history=2)
b = FileScheduleStore(store_dir=d, max_history=2)
with mock.patch.object(store_mod.time, "time", return_value=100.0):
with mock.patch("praisonaiagents.utils.atomic_io.os.replace", _fail_replace):
a.log_run("old", "succeeded")
with mock.patch.object(store_mod.time, "time", return_value=200.0):
b.log_run("newer", "succeeded")
with mock.patch.object(store_mod.time, "time", return_value=300.0):
a.log_run("newest", "succeeded")
ids = [r.job_id for r in FileScheduleStore(store_dir=d).get_history()]
assert ids == ["newest", "newer"]
def test_mark_run_contains_store_write_error(tmp_path):
from praisonaiagents.scheduler.runner import ScheduleRunner
s = FileScheduleStore(store_dir=str(tmp_path))
job = _make_job()
s.add(job)
with mock.patch("praisonaiagents.utils.atomic_io.os.replace", _fail_replace):
ScheduleRunner(s).mark_run(job) # must not raise
@@ -33,6 +33,54 @@ def test_web_crawl_rejects_redirect_to_loopback():
assert _is_safe_crawl_url("http://example.com/") is True
def test_web_crawl_urllib_fallback_does_not_follow_redirect_to_internal(monkeypatch):
import http.server
import sys
import threading
from praisonaiagents.tools import web_crawl_tools as w
monkeypatch.setitem(sys.modules, "httpx", None) # force the urllib fallback
hits = []
class Internal(http.server.BaseHTTPRequestHandler):
def do_GET(self):
hits.append(self.path)
self.send_response(200)
self.end_headers()
self.wfile.write(b"<html>internal-metadata</html>")
def log_message(self, *a):
pass
internal = http.server.HTTPServer(("127.0.0.1", 0), Internal)
class Redir(http.server.BaseHTTPRequestHandler):
def do_GET(self):
self.send_response(302)
self.send_header("Location", f"http://127.0.0.1:{internal.server_port}/meta")
self.end_headers()
def log_message(self, *a):
pass
redir = http.server.HTTPServer(("127.0.0.1", 0), Redir)
for server in (internal, redir):
threading.Thread(target=server.serve_forever, daemon=True).start()
start = f"http://127.0.0.1:{redir.server_port}/"
real = w._is_safe_crawl_url
monkeypatch.setattr(w, "_is_safe_crawl_url", lambda u: u == start or real(u))
try:
result = w._crawl_with_httpx([start])
finally:
internal.shutdown()
redir.shutdown()
assert hits == []
assert "internal-metadata" not in result[0]["content"]
assert "SSRF" in result[0]["error"]
def test_searxng_may_use_loopback_but_the_crawler_may_not():
"""One predicate, two callers, opposite requirements. Pin both."""
from praisonaiagents.tools.url_safety import validate_searxng_url
@@ -0,0 +1,51 @@
"""Timeout/interrupt keep output printed before the cut-off (issue #5990)."""
import sys
import threading
import pytest
from praisonaiagents.tools.shell_tools import ShellTools
@pytest.fixture(autouse=True)
def _auto_approve(monkeypatch):
# execute_command is approval-gated (critical); auto-approve for the test.
monkeypatch.setenv("PRAISONAI_AUTO_APPROVE", "true")
_SCRIPT = (
"import time,sys;print('step1 ok',flush=True);"
"print('warn: x',file=sys.stderr,flush=True);time.sleep(30)"
)
def _cmd():
return f'{sys.executable} -c "{_SCRIPT}"'
def test_timeout_keeps_partial_output():
result = ShellTools().execute_command(_cmd(), timeout=2)
assert result["success"] is False
assert result["timed_out"] is True
assert "step1 ok" in result["stdout"]
assert "warn: x" in result["stderr"]
assert result["stderr"].endswith("Command timed out after 2 seconds")
def test_interrupt_keeps_partial_output(monkeypatch):
cancel = threading.Event()
tools = ShellTools()
monkeypatch.setattr(tools, "_current_cancel_event", lambda: cancel)
threading.Timer(1.5, cancel.set).start()
result = tools.execute_command(_cmd(), timeout=20)
assert result["interrupted"] is True
assert "step1 ok" in result["stdout"]
assert result["stderr"].endswith("Command interrupted by user")
def test_timeout_keeps_unterminated_line():
# Flushed text without a trailing newline must survive the timeout too.
script = "import time,sys;sys.stdout.write('progress');sys.stdout.flush();time.sleep(30)"
result = ShellTools().execute_command(f'{sys.executable} -c "{script}"', timeout=2)
assert result["timed_out"] is True
assert "progress" in result["stdout"]
@@ -72,10 +72,13 @@ def test_urllib_fallback_reads_only_up_to_cap(monkeypatch):
monkeypatch.setitem(sys.modules, "httpx", None) # force ImportError
monkeypatch.setattr(w, "_is_safe_crawl_url", lambda url: True)
monkeypatch.setattr(
urllib.request, "urlopen",
lambda url, timeout=None: Resp(b"<html><h1>Hi</h1>" + b"a" * (w._MAX_CRAWL_BYTES * 2) + b"</html>"),
)
body = b"<html><h1>Hi</h1>" + b"a" * (w._MAX_CRAWL_BYTES * 2) + b"</html>"
class Opener:
def open(self, url, timeout=None):
return Resp(body)
monkeypatch.setattr(urllib.request, "build_opener", lambda *handlers: Opener())
result = w._crawl_with_httpx(["http://example.com/"])[0]
@@ -111,6 +111,113 @@ class AdmissionRejected(Exception):
self.message = message
class _TurnSlot:
"""The run slot held by one admitted turn, lent back during human waits.
Issue #5912: a turn blocked on a human approval consumes no compute, so
while it waits its slot is released to other turns. Resuming never blocks:
the turn takes a free slot or, when none is free, borrows one that the next
released slot repays. A sync approval parked in a pool worker therefore
cannot deadlock against a turn waiting for that worker, and every parallel
approval branch returns already holding the slot. All state transitions run
on the gate's event loop and never suspend; a sync approval in a worker
thread hops onto it.
"""
def __init__(self, gate: "AdmissionGate", sem: asyncio.Semaphore, scope: str):
self._gate = gate
self._sem = sem
self._scope = scope
self._loop = asyncio.get_running_loop()
self._waits = 0
self.held = True
self.closed = False
def human_wait(self) -> "_HumanWait":
return _HumanWait(self)
async def _suspend(self) -> None:
self._waits += 1
if self._waits == 1 and self.held and not self.closed:
self.held = False
self._gate.awaiting_human += 1
self._gate._give_slot(self._scope)
self._gate._release(self._sem)
async def _resume(self) -> None:
self._waits -= 1
if self.held or self.closed:
return
if self._sem.locked():
self._gate._debt += 1 # borrow; repaid by the next released slot
else:
await self._sem.acquire() # a permit is free: returns without suspending
self.held = True
self._gate.awaiting_human -= 1
self._gate._take_slot(self._scope)
def close(self) -> None:
"""End the turn: free the slot if held, else drop the human-wait count."""
self.closed = True
if self.held:
self.held = False
self._gate._give_slot(self._scope)
self._gate._release(self._sem)
else:
self._gate.awaiting_human -= 1
def _run_blocking(self, coro) -> bool:
"""Run ``coro`` on the gate loop from another thread; ``False`` if unsafe."""
try:
running = asyncio.get_running_loop()
except RuntimeError:
running = None
if running is self._loop or self._loop.is_closed():
# On the gate loop itself we cannot block; leave the slot held.
coro.close()
return False
try:
asyncio.run_coroutine_threadsafe(coro, self._loop).result()
return True
except Exception as e: # pragma: no cover — never break the approval
logger.debug("AdmissionGate: human-wait slot transfer failed: %s", e)
return False
class _HumanWait:
"""Sync + async context manager entered around one human approval wait."""
def __init__(self, slot: _TurnSlot):
self._slot = slot
self._entered = False
def __enter__(self) -> "_HumanWait":
self._entered = self._slot._run_blocking(self._slot._suspend())
return self
def __exit__(self, *exc) -> None:
if self._entered:
self._slot._run_blocking(self._slot._resume())
async def __aenter__(self) -> "_HumanWait":
slot = self._slot
if asyncio.get_running_loop() is slot._loop:
await slot._suspend()
self._entered = True
else:
self._entered = slot._run_blocking(slot._suspend())
return self
async def __aexit__(self, *exc) -> None:
if not self._entered:
return
slot = self._slot
if asyncio.get_running_loop() is slot._loop:
await slot._resume()
else:
slot._run_blocking(slot._resume())
class AdmissionGate:
"""Gateway-wide admission gate enforcing a concurrency ceiling + wait queue.
@@ -169,6 +276,10 @@ class AdmissionGate:
# Live counters.
self._in_flight = 0
self._queued = 0
# Admitted turns parked on a human approval, not holding a slot (#5912).
self.awaiting_human = 0
# Slots borrowed by resumed turns when none was free; repaid first (#5912).
self._debt = 0
# Live per-scope counters (only populated when the sub-limit is active),
# keyed by the caller-supplied scope (the session_id at the call sites).
self._scope_in_flight: "Dict[str, int]" = {}
@@ -222,6 +333,7 @@ class AdmissionGate:
"queue_depth": self._queue_depth,
"in_flight": self._in_flight,
"queued": self._queued,
"awaiting_human": self.awaiting_human,
"admitted": self.admitted,
"rejected": self.rejected,
"shed": self.shed,
@@ -339,16 +451,43 @@ class AdmissionGate:
if scope:
self._dec_scope(self._scope_queued, scope)
self._in_flight += 1
if scope:
self._scope_in_flight[scope] = self._scope_in_flight.get(scope, 0) + 1
self._take_slot(scope)
self.admitted += 1
# Issue #5912: lend the slot back while the turn is parked on a human
# approval, so pending approvals cannot starve the ceiling.
slot = _TurnSlot(self, sem, scope)
token = None
try:
from praisonaiagents.approval.registry import HUMAN_WAIT_SCOPE
except ImportError: # pragma: no cover — older core without the hook
HUMAN_WAIT_SCOPE = None
if HUMAN_WAIT_SCOPE is not None:
token = HUMAN_WAIT_SCOPE.set(slot.human_wait)
try:
yield
finally:
self._in_flight -= 1
if scope:
self._dec_scope(self._scope_in_flight, scope)
if token is not None:
try:
HUMAN_WAIT_SCOPE.reset(token)
except ValueError: # pragma: no cover — exited in another context
pass
slot.close()
def _take_slot(self, scope: str) -> None:
self._in_flight += 1
if scope:
self._scope_in_flight[scope] = self._scope_in_flight.get(scope, 0) + 1
def _give_slot(self, scope: str) -> None:
self._in_flight -= 1
if scope:
self._dec_scope(self._scope_in_flight, scope)
def _release(self, sem: asyncio.Semaphore) -> None:
"""Return a slot permit, first repaying any slot borrowed on resume."""
if self._debt:
self._debt -= 1
else:
sem.release()
@staticmethod
@@ -1106,6 +1106,11 @@ class GatewayConfigSchema(BaseModel):
# core ``RoomConfig.from_dict`` and bound like an agent via ``room:<name>``.
rooms: Optional[Dict[str, Dict[str, Any]]] = None
# Downstream MCP tool servers (Issue #6004). Each ``mcp_servers.<name>``
# (``command``/``args``/``env`` or ``url``/``headers``) is opened once as a
# shared ``MCP`` client and handed to agents that list ``<name>`` in tools.
mcp_servers: Optional[Dict[str, Union[Dict[str, Any], str]]] = None
# Gateway server settings (host/port, drain_timeout, admission control,
# etc.) and inbound trigger hooks. Kept as dicts/lists on this model so
# downstream consumers (``gateway/server.py`` reads them via ``.get(...)``)
@@ -163,9 +163,58 @@ def transcribe_media_path(
return None
async def transcribe_audio_bytes(
data: bytes,
config: Any,
*,
filename: Optional[str] = None,
) -> str:
"""Cache, transcribe and clean up downloaded inbound audio (Issue #5929).
One call for adapters that already hold the audio bytes (Discord, Signal),
so they share the size cap, STT policy and graceful degradation with
Telegram/Slack/WhatsApp. Always returns agent-consumable text: the
transcript on success, otherwise :data:`DEFAULT_VOICE_PLACEHOLDER` so the
voice note is never silently dropped.
"""
import asyncio
import os
from ._media import cache_inbound_media, resolve_max_inbound_media_bytes
stt_cfg = resolve_stt_config(config)
max_bytes = resolve_max_inbound_media_bytes(config)
if not stt_cfg.enabled or not data or not max_bytes or len(data) > max_bytes:
return DEFAULT_VOICE_PLACEHOLDER
try:
path = cache_inbound_media(
data, kind="audio", max_bytes=max_bytes, filename=filename
)
except Exception as e:
logger.warning("Could not cache inbound audio: %s", e)
return DEFAULT_VOICE_PLACEHOLDER
try:
text = await asyncio.to_thread(
transcribe_media_path,
path,
language=stt_cfg.language,
model=stt_cfg.model,
)
finally:
try:
os.remove(path)
except OSError:
pass
if not text:
return DEFAULT_VOICE_PLACEHOLDER
return f"[Voice message]: {text}" if stt_cfg.echo_transcripts else text
__all__ = [
"SttConfig",
"DEFAULT_VOICE_PLACEHOLDER",
"resolve_stt_config",
"transcribe_media_path",
"transcribe_audio_bytes",
]
@@ -320,6 +320,14 @@ class DiscordBot(OutboundResilienceMixin, ChatCommandMixin, MessageHookMixin):
user_allowed = await UnknownUserHandler.handle(bot_message, self._bot_context)
if not user_allowed:
return
# Issue #5929: transcribe an inbound voice note so the turn reaches
# the agent (placeholder on disable/failure, never dropped). Runs
# after the access gates so rejected senders trigger no STT work.
if not (bot_message.content or "").strip():
transcript = await self._transcribe_audio(message)
if transcript:
bot_message.content = transcript
if bot_message.is_command:
command = bot_message.command
@@ -1095,6 +1103,39 @@ class DiscordBot(OutboundResilienceMixin, ChatCommandMixin, MessageHookMixin):
extra = {cmd: "Custom command" for cmd in self._command_handlers}
return format_help(self._agent, self.platform, extra)
async def _transcribe_audio(self, message) -> Optional[str]:
"""Transcribe the first audio attachment of a Discord message.
Returns ``None`` when the message carries no audio; otherwise the
transcript or the shared voice placeholder (Issue #5929).
"""
audio = next(
(
a for a in (getattr(message, "attachments", None) or [])
if (getattr(a, "content_type", None) or "").lower().startswith("audio/")
),
None,
)
if audio is None:
return None
from ._media import resolve_max_inbound_media_bytes
from ._stt import DEFAULT_VOICE_PLACEHOLDER, resolve_stt_config, transcribe_audio_bytes
if not resolve_stt_config(self.config).enabled:
return DEFAULT_VOICE_PLACEHOLDER
max_bytes = resolve_max_inbound_media_bytes(self.config)
if not max_bytes or max_bytes <= 0 or (getattr(audio, "size", 0) or 0) > max_bytes:
return DEFAULT_VOICE_PLACEHOLDER
try:
data = await audio.read()
except Exception as e:
logger.warning("Discord audio download failed: %s", e)
return DEFAULT_VOICE_PLACEHOLDER
return await transcribe_audio_bytes(
data, self.config, filename=getattr(audio, "filename", None)
)
def _convert_message(self, message) -> BotMessage:
"""Convert Discord Message to BotMessage."""
sender = BotUser(
@@ -2106,6 +2106,29 @@
"default": null,
"title": "Rooms"
},
"mcp_servers": {
"anyOf": [
{
"additionalProperties": {
"anyOf": [
{
"additionalProperties": true,
"type": "object"
},
{
"type": "string"
}
]
},
"type": "object"
},
{
"type": "null"
}
],
"default": null,
"title": "Mcp Servers"
},
"gateway": {
"anyOf": [
{
+57 -1
View File
@@ -29,6 +29,7 @@ import logging
import os
import time
from typing import Any, Callable, Dict, List, Optional, TYPE_CHECKING, Union
from urllib.parse import quote
if TYPE_CHECKING:
from praisonaiagents import Agent
@@ -321,12 +322,60 @@ class SignalBot(OutboundResilienceMixin, ChatCommandMixin, MessageHookMixin):
except Exception as e: # noqa: BLE001 — one bad message must not stop the batch
logger.warning("Signal message processing error: %s", e)
async def _transcribe_audio(self, data_msg: Dict[str, Any]) -> Optional[str]:
"""Transcribe the first audio attachment of a signal-cli data message.
Returns ``None`` when there is no audio; otherwise the transcript or
the shared voice placeholder (Issue #5929).
"""
audio = next(
(
a for a in (data_msg.get("attachments") or [])
if isinstance(a, dict)
and str(a.get("contentType") or "").lower().startswith("audio/")
and a.get("id")
),
None,
)
if audio is None:
return None
from ._media import resolve_max_inbound_media_bytes
from ._stt import DEFAULT_VOICE_PLACEHOLDER, resolve_stt_config, transcribe_audio_bytes
if not resolve_stt_config(self.config).enabled:
return DEFAULT_VOICE_PLACEHOLDER
max_bytes = resolve_max_inbound_media_bytes(self.config)
if not max_bytes or max_bytes <= 0 or self._http_session is None:
return DEFAULT_VOICE_PLACEHOLDER
# The bridge is operator-configured, so no SSRF guard is needed here.
url = f"{self._bridge_url}/v1/attachments/{quote(str(audio['id']), safe='')}"
try:
async with self._http_session.get(url) as resp:
if resp.status != 200:
logger.warning("Signal attachment download failed: HTTP %s", resp.status)
return DEFAULT_VOICE_PLACEHOLDER
# read(n) may return a partial chunk; loop to EOF or the cap.
data = b""
while len(data) <= max_bytes:
chunk = await resp.content.read(max_bytes + 1 - len(data))
if not chunk:
break
data += chunk
except Exception as e: # noqa: BLE001
logger.warning("Signal attachment download failed: %s", e)
return DEFAULT_VOICE_PLACEHOLDER
return await transcribe_audio_bytes(
data, self.config, filename=audio.get("filename")
)
async def _handle_envelope(self, item: Dict[str, Any]) -> None:
"""Parse one signal-cli envelope and route it to the agent."""
envelope = item.get("envelope", item) if isinstance(item, dict) else {}
data_msg = envelope.get("dataMessage") or {}
content = data_msg.get("message")
if not content:
has_attachments = bool(data_msg.get("attachments"))
if not content and not has_attachments:
return # receipts, typing, empty — nothing to answer
source = str(envelope.get("source") or envelope.get("sourceNumber") or "")
@@ -349,6 +398,13 @@ class SignalBot(OutboundResilienceMixin, ChatCommandMixin, MessageHookMixin):
logger.debug("Signal: sender %s not in allowlist", source)
return
# Issue #5929: transcribe only after the allowlist gate so rejected
# senders trigger no download or STT work.
if not content:
content = await self._transcribe_audio(data_msg)
if not content:
return
sender = BotUser(
user_id=source,
username=source or source_name,
+335 -32
View File
@@ -547,6 +547,9 @@ class GatewaySession:
# interrupted by a gateway restart. ``None`` for direct-client sessions,
# which resume on reconnect and need no server-initiated channel notice.
_channel_target: Optional[str] = None
# Issue #5997: ``SessionVisibility`` value. ``private`` keeps today's 1:1
# owner binding; ``shared``/``read_only`` let other clients attach.
_visibility: str = "private"
@property
def session_id(self) -> str:
@@ -701,6 +704,7 @@ class GatewaySession:
"pending_inbox": pending_inbox,
"is_executing": self._is_executing,
"channel_target": self._channel_target,
"visibility": self._visibility,
}
@classmethod
@@ -751,6 +755,7 @@ class GatewaySession:
# Issue #3379: restore the channel origin so a boot-time resume can
# notify the originating channel about an interrupted turn.
session._channel_target = data.get("channel_target")
session._visibility = data.get("visibility", "private")
return session
@@ -1203,6 +1208,9 @@ class WebSocketGateway:
self._reaped_connections = 0
self._agents: Dict[str, "Agent"] = {}
# Issue #6004: downstream MCP servers from YAML ``mcp_servers:``, one
# shared ``MCP`` client per (name, spec) reused across agents/reloads.
self._mcp_pool: Dict[str, Any] = {}
self._sessions: Dict[str, GatewaySession] = {}
self._clients: Dict[str, Any] = {} # WebSocket connections
self._client_conns: Dict[str, _ClientConn] = {} # client_id -> bounded outbound conn
@@ -1214,6 +1222,9 @@ class WebSocketGateway:
# refreshed on every inbound frame, dropped on teardown.
self._client_last_seen: Dict[str, float] = {} # client_id -> last activity ts
self._client_sessions: Dict[str, str] = {} # client_id -> session_id
# Issue #5997: non-owner observers per shared session (session_id ->
# {client_id: SessionSharingRole}). The owner stays ``session.client_id``.
self._session_observers: Dict[str, Dict[str, Any]] = {}
self._client_scopes: Dict[str, List[str]] = {} # client_id -> operator scopes
# Issue #3467: in-flight turn registry so a running turn can be aborted
# (by a WS ``abort`` frame or a portable ``/stop`` chat command) and so
@@ -2008,9 +2019,7 @@ class WebSocketGateway:
await self._teardown_client_conn(client_id)
self._client_scopes.pop(client_id, None)
self._client_auth_generation.pop(client_id, None)
session_id = self._client_sessions.pop(client_id, None)
if session_id:
self.close_session(session_id)
self._release_client_session(client_id)
await self.emit(GatewayEvent(
type=EventType.DISCONNECT,
@@ -3843,6 +3852,14 @@ class WebSocketGateway:
except Exception:
logger.debug("Failed to mark clean exit in lifecycle ledger")
# Issue #6004: close pooled downstream MCP clients.
for mcp_client in self._mcp_pool.values():
try:
mcp_client.shutdown()
except Exception:
logger.debug("Failed to shut down pooled MCP client")
self._mcp_pool.clear()
# Release PID lock
if hasattr(self, '_pid_lock') and self._pid_lock:
self._pid_lock.release_lock()
@@ -3984,6 +4001,19 @@ class WebSocketGateway:
# Resume or create session
session_id = data.get("session_id")
since_cursor = data.get("since")
if isinstance(session_id, str) and client_id in self._session_observers.get(
session_id, {}
):
# Issue #5997: an attached observer keeps its sharing role; it
# must not rebind as the session's owner.
error = HelloError(
code=ConnectErrorCode.AUTH_UNAUTHORIZED,
message="Attached observers cannot take over the session",
next_step=ConnectRecoveryStep.DO_NOT_RETRY,
next_action="start_new_session",
)
await self._send_to_client(client_id, error.to_dict())
return
session, replay_events = self.resume_or_create_session(
session_id=session_id,
agent_id=agent_id,
@@ -4165,7 +4195,19 @@ class WebSocketGateway:
"message": "Invalid 'since' cursor. Must be an integer.",
})
return
if isinstance(session_id, str) and client_id in self._session_observers.get(
session_id, {}
):
# Issue #5997: an attached observer keeps its sharing
# role; it must not rebind as the session's owner.
await self._send_to_client(client_id, {
"type": "error",
"code": "unauthorized",
"message": "Attached observers cannot take over the session",
})
return
# Resume or create session
session, replay_events = self.resume_or_create_session(
session_id=session_id,
@@ -4502,10 +4544,10 @@ class WebSocketGateway:
})
elif msg_type == "leave":
session_id = self._client_sessions.pop(client_id, None)
# Close an owned session (kept persisted for resumption); an
# attached observer only detaches from the shared session.
session_id = self._release_client_session(client_id)
if session_id:
# Close session but keep it persisted for resumption
self.close_session(session_id, persist=True)
await self._send_to_client(client_id, {
"type": "left",
"session_id": session_id,
@@ -4632,7 +4674,12 @@ class WebSocketGateway:
_snapshot()
return result
return await loop.run_in_executor(None, _call_sync)
# Carry the turn's context into the worker so the admission gate's
# human-wait hook (#5912) reaches a sync agent's approval wait.
import contextvars
return await loop.run_in_executor(
None, contextvars.copy_context().run, _call_sync
)
def _abort_active_turn(self, session_id: str, reason: str = "user") -> bool:
"""Signal (and, if needed, cancel) the in-flight turn for ``session_id``.
@@ -4875,7 +4922,9 @@ class WebSocketGateway:
relay_callback = None
relay_futures: List[Any] = []
emitter = getattr(agent, 'stream_emitter', None)
if emitter is not None and client_id:
if emitter is not None and (
client_id or self._session_observers.get(session.session_id)
):
relay_callback = self._make_stream_relay(
client_id, session, relay_futures
)
@@ -4974,13 +5023,18 @@ class WebSocketGateway:
outcome: Dict[str, Any] = {"status": outcome_status}
if turn_error is not None:
outcome["error"] = turn_error
await self._send_to_client(client_id, {
final_frame = {
"type": "response",
"status": "final",
"content": response,
"outcome": outcome,
"session_id": session.session_id,
})
}
await self._send_to_client(client_id, final_frame)
for observer_id in self._observer_targets(
session.session_id, OperatorScope.READ, exclude=client_id
):
await self._send_to_client(observer_id, dict(final_frame))
finally:
session.mark_executing(False)
session._inflight_unstarted = False
@@ -5082,13 +5136,20 @@ class WebSocketGateway:
target=client_id,
)
# No get_event_loop() in the threaded callback.
fut = asyncio.run_coroutine_threadsafe(
gateway._send_to_client(client_id, gw_event.to_dict()),
loop,
# Issue #5997: fan out to the owner plus every attached
# observer whose scopes allow this event class.
targets = [client_id] if client_id else []
targets += gateway._observer_targets(
sid, gateway._event_required_scope(gw_event), exclude=client_id
)
if pending is not None:
pending.append(fut)
for target in targets:
# No get_event_loop() in the threaded callback.
fut = asyncio.run_coroutine_threadsafe(
gateway._send_to_client(target, gw_event.to_dict()),
loop,
)
if pending is not None:
pending.append(fut)
except Exception:
logger.warning("Stream relay error (non-fatal)", exc_info=True)
@@ -5135,9 +5196,7 @@ class WebSocketGateway:
# stale scope/session entries that accumulate until shutdown.
self._client_scopes.pop(client_id, None)
self._client_auth_generation.pop(client_id, None)
session_id = self._client_sessions.pop(client_id, None)
if session_id:
self.close_session(session_id)
self._release_client_session(client_id)
if ws is not None:
try:
await ws.close(
@@ -5207,9 +5266,7 @@ class WebSocketGateway:
await self._teardown_client_conn(client_id)
self._client_scopes.pop(client_id, None)
self._client_auth_generation.pop(client_id, None)
session_id = self._client_sessions.pop(client_id, None)
if session_id:
self.close_session(session_id)
self._release_client_session(client_id)
if ws is not None:
try:
await ws.close(
@@ -5261,9 +5318,7 @@ class WebSocketGateway:
await self._teardown_client_conn(client_id)
self._client_scopes.pop(client_id, None)
self._client_auth_generation.pop(client_id, None)
session_id = self._client_sessions.pop(client_id, None)
if session_id:
self.close_session(session_id)
self._release_client_session(client_id)
self._reaped_connections += 1
if ws is not None:
try:
@@ -5514,7 +5569,18 @@ class WebSocketGateway:
"tool_result_stream",
]:
session_id = self._client_sessions.get(client_id)
if session_id:
# Issue #5997: record only frames of the recipient's own
# session, and never an attached observer's fan-out copy
# (the owner's copy is the one recorded).
inner = data.get("data")
frame_sid = data.get("session_id") or (
inner.get("session_id") if isinstance(inner, dict) else None
)
if (
session_id
and frame_sid in (None, session_id)
and client_id not in self._session_observers.get(session_id, {})
):
session = self._sessions.get(session_id)
if session:
event = GatewayEvent(
@@ -6322,8 +6388,32 @@ class WebSocketGateway:
agent_id: str,
client_id: Optional[str] = None,
session_id: Optional[str] = None,
visibility: Optional[str] = None,
) -> GatewaySession:
"""Create a new session or resume an existing one.
``visibility`` (a ``SessionVisibility`` value) opts the session into
sharing; ``None`` keeps the existing/default ``private`` setting.
"""
if visibility is not None:
# Validate before anything is created or persisted.
from praisonaiagents.gateway.protocols import SessionVisibility
visibility = SessionVisibility(visibility).value
session = self._create_session(agent_id, client_id, session_id, visibility)
if visibility is not None and session._visibility != visibility:
# Resumed/active session: restricting visibility must also revoke
# access the new setting no longer allows.
session._visibility = visibility
self._enforce_session_visibility(session)
return session
def _create_session(
self,
agent_id: str,
client_id: Optional[str],
session_id: Optional[str],
visibility: Optional[str] = None,
) -> GatewaySession:
"""Create a new session or resume an existing one."""
sid = session_id or str(uuid.uuid4())
# Check if session exists in memory first
@@ -6385,9 +6475,10 @@ class WebSocketGateway:
_agent_id=agent_id,
_client_id=client_id,
_max_messages=self.config.session_config.max_messages,
_visibility=visibility or "private",
)
self._sessions[sid] = session
# Persist initial session state if store is configured
if self._session_store:
try:
@@ -6443,10 +6534,135 @@ class WebSocketGateway:
# Remove from active sessions
self._sessions.pop(session_id, None)
self._drop_session_observers(session_id)
logger.info(f"Session closed: {session_id}")
return True
# ── Shared-session observers (SessionObserverProtocol, issue #5997) ──
def attach(self, session_id: str, client_id: str, role: Any = "viewer") -> None:
"""Attach ``client_id`` to a shared session as ``member`` or ``viewer``.
Honours the session's ``SessionVisibility``: ``private`` sessions
refuse observers, ``read_only`` admits viewers only. A member is bound
to the session so its ``message`` frames co-drive it (still gated by
the ``write`` scope) and stay sender-attributed.
"""
from praisonaiagents.gateway.protocols import (
SessionSharingRole,
SessionVisibility,
)
session = self._sessions.get(session_id)
if session is None:
raise KeyError(f"Unknown session: {session_id}")
role = SessionSharingRole(role)
if role is SessionSharingRole.OWNER or client_id == session.client_id:
raise ValueError("The owner is bound at session creation, not attached")
visibility = SessionVisibility(session._visibility)
if visibility is SessionVisibility.PRIVATE:
raise PermissionError(f"Session {session_id} is private")
if visibility is SessionVisibility.READ_ONLY and role is not SessionSharingRole.VIEWER:
raise PermissionError(f"Session {session_id} is read-only; attach as viewer")
bound = self._client_sessions.get(client_id)
if bound is not None and bound != session_id:
raise ValueError(f"Client {client_id} is already bound to session {bound}")
self._session_observers.setdefault(session_id, {})[client_id] = role
if role is SessionSharingRole.MEMBER:
self._client_sessions[client_id] = session_id
elif bound == session_id:
# Demoted member -> viewer: it may no longer send.
self._client_sessions.pop(client_id, None)
def detach(self, session_id: str, client_id: str) -> None:
"""Stop ``client_id`` observing ``session_id`` (no-op if not attached)."""
roster = self._session_observers.get(session_id)
if not roster or roster.pop(client_id, None) is None:
return
if not roster:
self._session_observers.pop(session_id, None)
if self._client_sessions.get(client_id) == session_id:
self._client_sessions.pop(client_id, None)
def observers(self, session_id: str) -> List[Tuple[str, Any]]:
"""Return ``(client_id, role)`` pairs for the owner and attached observers."""
from praisonaiagents.gateway.protocols import SessionSharingRole
session = self._sessions.get(session_id)
roster: List[Tuple[str, Any]] = []
if session is not None and session.client_id:
roster.append((session.client_id, SessionSharingRole.OWNER))
roster.extend(self._session_observers.get(session_id, {}).items())
return roster
def _observer_targets(
self,
session_id: str,
scope: OperatorScope,
exclude: Optional[str] = None,
) -> List[str]:
"""Connected fan-out targets of ``session_id`` other than ``exclude``.
The owner is always included (as for its own turns), so a turn a
member drives still reaches the owner and its replay buffer;
observers additionally need ``scope``.
"""
session = self._sessions.get(session_id)
owner = session.client_id if session is not None else None
targets = [owner] if owner and owner != exclude and owner in self._clients else []
targets += [
cid
for cid in list(self._session_observers.get(session_id, {}))
if cid != exclude and cid in self._clients and self._client_has_scope(cid, scope)
]
return targets
def _enforce_session_visibility(self, session: GatewaySession) -> None:
"""Revoke observer access the session's visibility no longer allows.
``private`` drops every observer; ``read_only`` demotes members to
viewers (unbinding them so they can no longer send).
"""
from praisonaiagents.gateway.protocols import (
SessionSharingRole,
SessionVisibility,
)
sid = session.session_id
visibility = SessionVisibility(session._visibility)
if visibility is SessionVisibility.PRIVATE:
self._drop_session_observers(sid)
elif visibility is SessionVisibility.READ_ONLY:
roster = self._session_observers.get(sid, {})
for cid, role in list(roster.items()):
if role is not SessionSharingRole.VIEWER:
roster[cid] = SessionSharingRole.VIEWER
if self._client_sessions.get(cid) == sid:
self._client_sessions.pop(cid, None)
def _drop_session_observers(self, session_id: str) -> None:
"""Forget every observer of a closed session (unbinding members)."""
for cid in self._session_observers.pop(session_id, {}):
if self._client_sessions.get(cid) == session_id:
self._client_sessions.pop(cid, None)
def _release_client_session(self, client_id: str) -> Optional[str]:
"""Unbind a departing client; close its session only if it owns it.
Detaches the client from every shared session it observes, so a
member or viewer leaving never tears down someone else's session.
Returns the session the client was bound to (if any).
"""
bound = self._client_sessions.get(client_id)
for sid in [s for s, r in self._session_observers.items() if client_id in r]:
self.detach(sid, client_id)
session_id = self._client_sessions.pop(client_id, None)
if session_id:
self.close_session(session_id)
return bound
def list_sessions(self, agent_id: Optional[str] = None) -> List[str]:
"""List session IDs, optionally filtered by agent."""
if agent_id:
@@ -8242,6 +8458,67 @@ class WebSocketGateway:
self._agents[route_id] = room
logger.info(f"Room registered: {route_id} roster={room.config.roster}")
@staticmethod
def _mcp_pool_key(name: str, spec: Any) -> Tuple[Optional[str], Any]:
"""Normalise a ``mcp_servers`` entry; return ``(pool_key, spec)``."""
import json
if isinstance(spec, str):
spec = {"command": spec}
if not isinstance(spec, dict):
return None, spec
return f"{name}:{json.dumps(spec, sort_keys=True, default=str)}", spec
def _get_pooled_mcp(self, name: str, spec: Any) -> Any:
"""Return the shared ``MCP`` client for a ``mcp_servers`` entry.
One client per (name, spec) is opened and reused by every agent that
lists it — and across hot reloads while the spec is unchanged — so N
agents do not open N connections. The core ``MCP`` client already
restarts a dead stdio server on the next call. Tool names are prefixed
with the server name (``<name>_<tool>``) so two servers exposing the
same tool cannot shadow each other. Returns ``None`` (with a warning)
when the server cannot be started (Issue #6004).
"""
key, spec = self._mcp_pool_key(name, spec)
if key is None:
logger.warning("mcp_servers.%s must be a mapping or string; skipping", name)
return None
client = self._mcp_pool.get(key)
if client is not None:
return client
opts = dict(spec)
target = opts.pop("url", None) or opts.pop("command", None)
if not target:
logger.warning("mcp_servers.%s needs a 'command' or 'url'; skipping", name)
return None
try:
from praisonaiagents import MCP
client = MCP(target, **opts).with_tool_prefix(name)
except Exception as exc:
logger.warning("Failed to start MCP server '%s': %s", name, exc)
return None
self._mcp_pool[key] = client
return client
def _warm_mcp_pool(self, mcp_servers_cfg: Optional[Dict[str, Any]]) -> None:
"""Open every configured MCP client. Blocking; reloads run it off-loop."""
for name, spec in (mcp_servers_cfg or {}).items():
self._get_pooled_mcp(name, spec)
def _prune_mcp_pool(self, mcp_servers_cfg: Optional[Dict[str, Any]]) -> None:
"""Shut down pooled clients the live config no longer references."""
live = {
self._mcp_pool_key(name, spec)[0]
for name, spec in (mcp_servers_cfg or {}).items()
}
for key in [k for k in self._mcp_pool if k not in live]:
try:
self._mcp_pool.pop(key).shutdown()
except Exception:
logger.debug("Failed to shut down retired MCP client")
def _create_agents_from_config(
self,
agents_cfg: Dict[str, Dict[str, Any]],
@@ -8249,6 +8526,7 @@ class WebSocketGateway:
default_model: Optional[str] = None,
guardrails_cfg: Optional[Dict[str, Any]] = None,
durable_runs: bool = False,
mcp_servers_cfg: Optional[Dict[str, Any]] = None,
) -> None:
"""Create and register Agent instances from the agents section of config.
@@ -8280,6 +8558,9 @@ class WebSocketGateway:
store) the execution hot path is unchanged (zero-overhead).
Per-agent ``durable`` in YAML overrides this gateway-wide
default.
mcp_servers_cfg: Optional top-level ``mcp_servers`` dict. An agent
``tools:`` entry naming a server gets that server's
shared, pooled ``MCP`` client (Issue #6004).
"""
from praisonaiagents import Agent
@@ -8333,12 +8614,17 @@ class WebSocketGateway:
agent_tools = []
yaml_tool_names = agent_def.get("tools", [])
if yaml_tool_names and tool_resolver:
if yaml_tool_names and (tool_resolver or mcp_servers_cfg):
for tool_name in yaml_tool_names:
if not tool_name or not isinstance(tool_name, str):
continue
tool_name = tool_name.strip()
resolved = tool_resolver.resolve(tool_name)
if mcp_servers_cfg and tool_name in mcp_servers_cfg:
mcp_client = self._get_pooled_mcp(tool_name, mcp_servers_cfg[tool_name])
if mcp_client is not None:
agent_tools.append(mcp_client)
continue
resolved = tool_resolver.resolve(tool_name) if tool_resolver else None
if resolved:
agent_tools.append(resolved)
logger.debug(f"Resolved tool '{tool_name}' for agent '{agent_id}'")
@@ -10513,7 +10799,13 @@ class WebSocketGateway:
if self._reload_lock is None:
self._reload_lock = asyncio.Lock()
async with self._reload_lock:
await self._reload_config_locked(config_path)
try:
await self._reload_config_locked(config_path)
finally:
# Issue #6004: retire MCP clients that neither the committed
# config nor a restored previous one references (removed or
# re-specced servers, or ones opened for a rejected candidate).
self._prune_mcp_pool((self._loaded_config or {}).get("mcp_servers"))
def _record_reload_status(
self,
@@ -10600,6 +10892,7 @@ class WebSocketGateway:
default_model=default_model,
guardrails_cfg=guardrails_cfg,
durable_runs=durable_runs,
mcp_servers_cfg=new_cfg.get("mcp_servers"),
)
self._register_rooms_from_config(new_cfg.get("rooms"))
candidate = {
@@ -10672,6 +10965,7 @@ class WebSocketGateway:
default_model=default_model,
guardrails_cfg=guardrails_cfg,
durable_runs=durable_runs,
mcp_servers_cfg=prev_cfg.get("mcp_servers"),
)
self._register_rooms_from_config(prev_cfg.get("rooms"))
prev_channels_cfg = prev_cfg.get("channels", {})
@@ -10702,6 +10996,12 @@ class WebSocketGateway:
self._record_reload_status("failed", error=str(e))
return
# Issue #6004: opening downstream MCP servers waits for their startup
# handshake, so do it off the serving loop; the agent builders below
# then reuse the pooled clients without blocking.
if new_cfg.get("mcp_servers"):
await asyncio.to_thread(self._warm_mcp_pool, new_cfg.get("mcp_servers"))
# Issue #3020: reconcile the cross-platform identity resolver from the
# (possibly changed) top-level ``identity:`` block *before* any channel
# is (re)started below, so freshly created bots are stamped with the
@@ -10734,6 +11034,7 @@ class WebSocketGateway:
default_model=default_model,
guardrails_cfg=guardrails_cfg,
durable_runs=durable_runs,
mcp_servers_cfg=new_cfg.get("mcp_servers"),
)
self._register_rooms_from_config(new_cfg.get("rooms"))
@@ -10853,6 +11154,7 @@ class WebSocketGateway:
default_model=default_model,
guardrails_cfg=guardrails_cfg,
durable_runs=durable_runs,
mcp_servers_cfg=new_cfg.get("mcp_servers"),
)
self._register_rooms_from_config(new_cfg.get("rooms"))
logger.info("Agents reloaded")
@@ -11553,6 +11855,7 @@ class WebSocketGateway:
default_model=default_model,
guardrails_cfg=guardrails_cfg,
durable_runs=durable_runs,
mcp_servers_cfg=cfg.get("mcp_servers"),
)
self._register_rooms_from_config(cfg.get("rooms"))
@@ -0,0 +1,129 @@
"""Inbound voice notes reach the agent on Discord and Signal (Issue #5929)."""
import asyncio
from types import SimpleNamespace
from unittest.mock import patch
from praisonaiagents.bots import BotConfig
from praisonai_bot.bots import _stt
from praisonai_bot.bots._stt import DEFAULT_VOICE_PLACEHOLDER, transcribe_audio_bytes
OGG = b"OggS" + b"\x00" * 64
def _run(coro):
return asyncio.run(coro)
def test_transcribe_audio_bytes_returns_transcript():
with patch.object(_stt, "transcribe_media_path", return_value="hello there"):
assert _run(transcribe_audio_bytes(OGG, BotConfig(token="x"))) == "hello there"
def test_transcribe_audio_bytes_degrades_to_placeholder():
cfg = BotConfig(token="x")
with patch.object(_stt, "transcribe_media_path", return_value=None):
assert _run(transcribe_audio_bytes(OGG, cfg)) == DEFAULT_VOICE_PLACEHOLDER
# Invalid audio bytes fail magic-byte validation -> placeholder, no raise.
assert _run(transcribe_audio_bytes(b"not audio", cfg)) == DEFAULT_VOICE_PLACEHOLDER
# STT disabled -> placeholder without transcribing.
off = BotConfig(token="x", metadata={"stt": False})
with patch.object(_stt, "transcribe_media_path") as tm:
assert _run(transcribe_audio_bytes(OGG, off)) == DEFAULT_VOICE_PLACEHOLDER
tm.assert_not_called()
def test_discord_transcribes_audio_attachment():
from praisonai_bot.bots.discord import DiscordBot
bot = DiscordBot(token="x")
async def _read():
return OGG
att = SimpleNamespace(content_type="audio/ogg", size=len(OGG), filename="voice.ogg", read=_read)
with patch.object(_stt, "transcribe_media_path", return_value="reschedule the 3pm"):
assert _run(bot._transcribe_audio(SimpleNamespace(attachments=[att]))) == "reschedule the 3pm"
img = SimpleNamespace(content_type="image/png", size=10, filename="a.png", read=_read)
assert _run(bot._transcribe_audio(SimpleNamespace(attachments=[img]))) is None
class _Resp:
status = 200
def __init__(self, data, chunk=16):
self.content = SimpleNamespace(read=self._read)
self._data = data
self._chunk = chunk
async def _read(self, n):
# Deliver in small chunks, like a streamed body, to guard partial reads.
out, self._data = self._data[: min(n, self._chunk)], self._data[min(n, self._chunk):]
return out
async def __aenter__(self):
return self
async def __aexit__(self, *a):
return False
def test_signal_transcribes_audio_attachment():
from praisonai_bot.bots.signal import SignalBot
bot = SignalBot(account="+15550000000", bridge_url="http://bridge:8080")
urls = []
def _get(url):
urls.append(url)
return _Resp(OGG)
bot._http_session = SimpleNamespace(get=_get)
data_msg = {"attachments": [{"id": "abc", "contentType": "audio/aac", "filename": "v.m4a"}]}
seen = []
async def _capture(data, config, filename=None):
seen.append(data)
return "call me back"
with patch.object(_stt, "transcribe_audio_bytes", _capture):
assert _run(bot._transcribe_audio(data_msg)) == "call me back"
# The whole multi-chunk body reached transcription, not just the first chunk.
assert seen == [OGG]
assert urls == ["http://bridge:8080/v1/attachments/abc"]
assert _run(bot._transcribe_audio({"attachments": []})) is None
def test_signal_rejected_sender_triggers_no_download():
from praisonai_bot.bots.signal import SignalBot
bot = SignalBot(
account="+15550000000", bridge_url="http://bridge:8080", allowed_users=["+15551111111"]
)
def _get(url):
raise AssertionError("rejected sender must not trigger a download")
bot._http_session = SimpleNamespace(get=_get)
envelope = {
"envelope": {
"source": "+15559999999",
"dataMessage": {"attachments": [{"id": "abc", "contentType": "audio/aac"}]},
}
}
_run(bot._handle_envelope(envelope))
def test_stt_disabled_skips_download():
from praisonai_bot.bots.signal import SignalBot
bot = SignalBot(account="+15550000000", bridge_url="http://bridge:8080")
bot.config.metadata = {"stt": False}
def _get(url):
raise AssertionError("STT off must not download audio")
bot._http_session = SimpleNamespace(get=_get)
data_msg = {"attachments": [{"id": "abc", "contentType": "audio/aac"}]}
assert _run(bot._transcribe_audio(data_msg)) == DEFAULT_VOICE_PLACEHOLDER
@@ -0,0 +1,91 @@
"""Gateway ``mcp_servers:`` shares one pooled MCP client across agents (Issue #6004)."""
import praisonaiagents
from praisonai_bot.gateway import WebSocketGateway
def _fake_mcp_factory(calls, shutdowns=None):
def fake_mcp(target, **opts):
calls.append((target, opts))
def downstream_tool(query: str) -> str:
"""Fake downstream MCP tool."""
return query
def with_tool_prefix(prefix):
downstream_tool.prefix = prefix
return downstream_tool
downstream_tool.with_tool_prefix = with_tool_prefix
downstream_tool.shutdown = lambda: (shutdowns if shutdowns is not None else []).append(target)
return downstream_tool
return fake_mcp
def test_mcp_server_opened_once_and_shared(monkeypatch):
calls = []
monkeypatch.setattr(praisonaiagents, "MCP", _fake_mcp_factory(calls), raising=False)
gw = WebSocketGateway()
mcp_cfg = {"github": {"command": "npx", "args": ["-y", "server-github"]}}
agents_cfg = {
"a": {"instructions": "x", "model": "gpt-4o-mini", "tools": ["github"]},
"b": {"instructions": "y", "model": "gpt-4o-mini", "tools": ["github"]},
"c": {"instructions": "z", "model": "gpt-4o-mini"},
}
gw._create_agents_from_config(agents_cfg, mcp_servers_cfg=mcp_cfg)
# A reload with the same spec reuses the pooled client.
gw._create_agents_from_config(agents_cfg, mcp_servers_cfg=mcp_cfg)
assert calls == [("npx", {"args": ["-y", "server-github"]})]
shared = next(iter(gw._mcp_pool.values()))
assert shared in gw.get_agent("a").tools
assert shared in gw.get_agent("b").tools
assert shared not in (gw.get_agent("c").tools or [])
# Tools are namespaced by server name so two servers cannot collide.
assert shared.prefix == "github"
def test_mcp_server_spec_change_opens_new_client(monkeypatch):
calls = []
monkeypatch.setattr(praisonaiagents, "MCP", _fake_mcp_factory(calls), raising=False)
gw = WebSocketGateway()
agents_cfg = {"a": {"instructions": "x", "model": "gpt-4o-mini", "tools": ["search"]}}
gw._create_agents_from_config(agents_cfg, mcp_servers_cfg={"search": {"url": "http://h/sse"}})
gw._create_agents_from_config(agents_cfg, mcp_servers_cfg={"search": {"url": "http://h2/sse"}})
assert [c[0] for c in calls] == ["http://h/sse", "http://h2/sse"]
def test_bad_mcp_server_is_skipped(monkeypatch):
def boom(target, **opts):
raise RuntimeError("cannot start")
monkeypatch.setattr(praisonaiagents, "MCP", boom, raising=False)
gw = WebSocketGateway()
agents_cfg = {"a": {"instructions": "x", "model": "gpt-4o-mini", "tools": ["github", "nourl"]}}
gw._create_agents_from_config(
agents_cfg, mcp_servers_cfg={"github": {"command": "npx"}, "nourl": {}}
)
assert gw.get_agent("a") is not None
assert gw._mcp_pool == {}
def test_prune_retires_unreferenced_clients(monkeypatch):
calls, shutdowns = [], []
monkeypatch.setattr(praisonaiagents, "MCP", _fake_mcp_factory(calls, shutdowns), raising=False)
gw = WebSocketGateway()
gw._warm_mcp_pool({"a": {"url": "http://a/sse"}, "b": {"url": "http://b/sse"}})
gw._prune_mcp_pool({"a": {"url": "http://a/sse"}})
assert shutdowns == ["http://b/sse"]
assert len(gw._mcp_pool) == 1
gw._prune_mcp_pool(None)
assert shutdowns == ["http://b/sse", "http://a/sse"]
assert gw._mcp_pool == {}
@@ -0,0 +1,277 @@
"""Shared-session observers on the gateway server (Issue #5997).
The wrapper gateway implements core's ``SessionObserverProtocol`` honouring
``SessionVisibility`` / ``SessionSharingRole``: private sessions stay 1:1,
attached observers receive the live turn stream gated by their scopes, and a
member/viewer leaving never closes the owner's session.
"""
import asyncio
import pytest
from praisonaiagents.gateway import GatewayConfig, OperatorScope
from praisonaiagents.gateway.protocols import (
SessionObserverProtocol,
SessionSharingRole,
)
from praisonai_bot.gateway.server import WebSocketGateway
class _WS:
def __init__(self):
self.sent = []
async def send_json(self, data):
self.sent.append(data)
async def close(self, code=None, reason=None):
pass
class _EmitterAgent:
def __init__(self):
from praisonaiagents.streaming.events import StreamEventEmitter
self.stream_emitter = StreamEventEmitter()
async def achat(self, content):
from praisonaiagents.streaming.events import StreamEvent, StreamEventType
self.stream_emitter.emit(
StreamEvent(type=StreamEventType.DELTA_TEXT, content="tok")
)
await asyncio.sleep(0)
return f"echo:{content}"
def _gateway():
return WebSocketGateway(config=GatewayConfig(auth_token="tmp"))
def _client(gw, cid, scopes=None):
# Direct registration (no bounded conn) so sends land synchronously.
ws = _WS()
gw._clients[cid] = ws
if scopes is not None:
gw._client_scopes[cid] = scopes
return ws
def _owned(gw, visibility=None):
session = gw.create_session("agent", client_id="owner", visibility=visibility)
gw._client_sessions["owner"] = session.session_id
return session
def test_gateway_satisfies_observer_protocol():
assert isinstance(_gateway(), SessionObserverProtocol)
def test_private_session_refuses_observers_by_default():
gw = _gateway()
session = _owned(gw)
assert session._visibility == "private"
with pytest.raises(PermissionError):
gw.attach(session.session_id, "bob")
assert gw.observers(session.session_id) == [("owner", SessionSharingRole.OWNER)]
def test_read_only_admits_viewers_only():
gw = _gateway()
session = _owned(gw, visibility="read_only")
with pytest.raises(PermissionError):
gw.attach(session.session_id, "ann", role="member")
gw.attach(session.session_id, "bob", role="viewer")
assert ("bob", SessionSharingRole.VIEWER) in gw.observers(session.session_id)
# A viewer is not bound to the session, so it cannot send.
assert "bob" not in gw._client_sessions
def test_shared_attach_detach_roster():
gw = _gateway()
session = _owned(gw, visibility="shared")
sid = session.session_id
gw.attach(sid, "bob", role="viewer")
gw.attach(sid, "ann", role="member")
assert gw.observers(sid) == [
("owner", SessionSharingRole.OWNER),
("bob", SessionSharingRole.VIEWER),
("ann", SessionSharingRole.MEMBER),
]
# A member co-drives through the normal ``message`` path.
assert gw._client_sessions["ann"] == sid
with pytest.raises(ValueError):
gw.attach(sid, "owner", role="member")
gw.detach(sid, "ann")
assert "ann" not in gw._client_sessions
assert gw.observers(sid) == [
("owner", SessionSharingRole.OWNER),
("bob", SessionSharingRole.VIEWER),
]
def test_visibility_survives_persistence_round_trip():
gw = _gateway()
session = _owned(gw, visibility="shared")
restored = type(session).from_dict(session.to_dict())
assert restored._visibility == "shared"
class _Store:
def __init__(self):
self.saved = []
def add_message(self, session_id, role, content, metadata=None):
self.saved.append(metadata["session_data"])
def test_initial_stored_snapshot_keeps_visibility():
gw = _gateway()
gw._session_store = _Store()
gw.create_session("agent", client_id="owner", visibility="shared")
assert gw._session_store.saved[0]["visibility"] == "shared"
def test_invalid_visibility_creates_nothing():
gw = _gateway()
gw._session_store = _Store()
with pytest.raises(ValueError):
gw.create_session("agent", client_id="owner", session_id="s1", visibility="bogus")
assert gw.get_session("s1") is None
assert gw._session_store.saved == []
def test_restricting_visibility_revokes_access():
gw = _gateway()
session = _owned(gw, visibility="shared")
sid = session.session_id
gw.attach(sid, "ann", role="member")
gw.attach(sid, "bob", role="viewer")
# shared -> read_only: the member is demoted and can no longer send.
gw.create_session("agent", client_id="owner", session_id=sid, visibility="read_only")
assert ("ann", SessionSharingRole.VIEWER) in gw.observers(sid)
assert "ann" not in gw._client_sessions
# read_only -> private: every observer is dropped.
gw.create_session("agent", client_id="owner", session_id=sid, visibility="private")
assert gw.observers(sid) == [("owner", SessionSharingRole.OWNER)]
assert gw._observer_targets(sid, OperatorScope.READ) == []
def test_observer_leaving_does_not_close_owner_session():
gw = _gateway()
session = _owned(gw, visibility="shared")
sid = session.session_id
gw.attach(sid, "ann", role="member")
gw.attach(sid, "bob", role="viewer")
assert gw._release_client_session("ann") == sid
gw._release_client_session("bob")
assert gw.get_session(sid) is session
assert gw.observers(sid) == [("owner", SessionSharingRole.OWNER)]
# The owner leaving closes the session and drops any remaining observers.
gw.attach(sid, "ann", role="member")
gw._release_client_session("owner")
assert gw.get_session(sid) is None
assert "ann" not in gw._client_sessions
assert sid not in gw._session_observers
def test_live_turn_fans_out_to_observers_gated_by_scope():
async def _run():
gw = _gateway()
owner_ws = _client(gw, "owner")
viewer_ws = _client(gw, "bob", scopes=["read"])
blind_ws = _client(gw, "eve", scopes=["write"]) # no READ -> nothing
session = _owned(gw, visibility="shared")
gw.attach(session.session_id, "bob", role="viewer")
gw.attach(session.session_id, "eve", role="member")
agent = _EmitterAgent()
session.mark_executing(True)
await session.queue_message("hi")
await gw._run_session_queue(session, agent, "owner")
return owner_ws, viewer_ws, blind_ws
owner_ws, viewer_ws, blind_ws = asyncio.run(_run())
def _shape(ws):
return [(f["type"], f.get("content") or f.get("data", {}).get("content")) for f in ws.sent]
expected_tail = ("response", "echo:hi")
assert ("token_stream", "tok") in _shape(owner_ws)
assert _shape(owner_ws)[-1] == expected_tail
assert ("token_stream", "tok") in _shape(viewer_ws)
assert _shape(viewer_ws)[-1] == expected_tail
assert blind_ws.sent == []
def test_member_driven_turn_reaches_owner_and_replay():
async def _run():
gw = _gateway()
owner_ws = _client(gw, "owner")
member_ws = _client(gw, "ann", scopes=["read", "write"])
session = _owned(gw, visibility="shared")
gw.attach(session.session_id, "ann", role="member")
agent = _EmitterAgent()
session.mark_executing(True)
await session.queue_message("hi")
await gw._run_session_queue(session, agent, "ann")
return owner_ws, member_ws, session
owner_ws, member_ws, session = asyncio.run(_run())
for ws in (owner_ws, member_ws):
types = [f["type"] for f in ws.sent]
assert "token_stream" in types
assert ws.sent[-1]["content"] == "echo:hi"
# Recorded once (via the owner's copy), not zero or twice.
assert [e.type for e in session._events].count("response") == 1
@pytest.mark.parametrize("frame", ["hello", "join"])
def test_attached_viewer_cannot_rebind_via_handshake(frame):
async def _run():
gw = _gateway()
gw._agents["agent"] = object()
_client(gw, "owner")
viewer_ws = _client(gw, "bob", scopes=["read", "write"])
session = _owned(gw, visibility="read_only")
gw.attach(session.session_id, "bob", role="viewer")
await gw._handle_client_message(
"bob",
{"type": frame, "agent_id": "agent", "session_id": session.session_id},
)
return gw, session, viewer_ws
gw, session, viewer_ws = asyncio.run(_run())
assert session.client_id == "owner"
assert "bob" not in gw._client_sessions
assert gw.observers(session.session_id)[1] == ("bob", SessionSharingRole.VIEWER)
assert viewer_ws.sent and "observer" in str(viewer_ws.sent[-1]).lower()
def test_observer_copy_not_recorded_in_its_own_session_replay():
async def _run():
gw = _gateway()
_client(gw, "owner")
_client(gw, "bob", scopes=["read"])
session_a = _owned(gw, visibility="shared")
gw.attach(session_a.session_id, "bob", role="viewer")
# The viewer also owns an unrelated session B.
session_b = gw.create_session("agent", client_id="bob")
gw._client_sessions["bob"] = session_b.session_id
session_a.mark_executing(True)
await session_a.queue_message("hi")
await gw._run_session_queue(session_a, _EmitterAgent(), "owner")
return session_a, session_b
session_a, session_b = asyncio.run(_run())
assert [e.type for e in session_a._events].count("response") == 1
assert list(session_b._events) == []
@@ -0,0 +1,257 @@
"""Issue #5912: a turn parked on a human approval must not hold a run slot.
With ``max_concurrent_runs=1``, one turn waiting on a human approval used to
wedge every other turn. The admission gate now lends the slot back for the
duration of the approval wait; on resume the turn takes a free slot or
borrows one that the next released slot repays.
"""
import asyncio
import contextvars
import threading
import pytest
from praisonai_bot.bots._admission import AdmissionGate
from praisonaiagents.approval.protocols import ApprovalDecision
from praisonaiagents.approval.registry import ApprovalRegistry
from praisonaiagents.gateway import ConcurrencyLimitPolicy
def _gate(max_runs=1):
return AdmissionGate(
ConcurrencyLimitPolicy(
max_concurrent_runs=max_runs, queue_depth=4, overflow_policy="reject"
)
)
class _AsyncHuman:
def __init__(self):
self.parked = asyncio.Event()
self.decide = asyncio.Event()
async def request_approval(self, request):
self.parked.set()
await self.decide.wait()
return ApprovalDecision(approved=True, reason="ok")
class _SyncHuman:
def __init__(self):
self.parked = threading.Event()
self.decide = threading.Event()
def request_approval_sync(self, request):
self.parked.set()
self.decide.wait(5)
return ApprovalDecision(approved=True, reason="ok")
async def request_approval(self, request): # pragma: no cover - unused
raise AssertionError("sync path expected")
def _registry(backend):
reg = ApprovalRegistry()
reg.set_backend(backend)
return reg
@pytest.mark.asyncio
async def test_async_approval_wait_frees_slot():
gate = _gate()
human = _AsyncHuman()
reg = _registry(human)
async def parked_turn():
async with gate.admit(session_id="a"):
return await reg.approve_async("bot", "tool", {}, force=True)
a = asyncio.ensure_future(parked_turn())
await asyncio.wait_for(human.parked.wait(), 2)
assert gate.in_flight == 0
assert gate.stats()["awaiting_human"] == 1
# Another session runs immediately — no queueing behind the human.
ran = []
async with gate.admit(session_id="b"):
ran.append(gate.in_flight)
assert ran == [1]
human.decide.set()
decision = await asyncio.wait_for(a, 2)
assert decision.approved
assert gate.in_flight == 0 and gate.awaiting_human == 0
assert gate._sem._value == 1
@pytest.mark.asyncio
async def test_resumed_turn_borrows_slot_and_repays_it():
gate = _gate()
human = _AsyncHuman()
reg = _registry(human)
resumed = asyncio.Event()
finish_a = asyncio.Event()
async def parked_turn():
async with gate.admit(session_id="a"):
await reg.approve_async("bot", "tool", {}, force=True)
resumed.set()
await finish_a.wait()
a = asyncio.ensure_future(parked_turn())
await asyncio.wait_for(human.parked.wait(), 2)
release_b = asyncio.Event()
async def busy_turn():
async with gate.admit(session_id="b"):
await release_b.wait()
b = asyncio.ensure_future(busy_turn())
await asyncio.sleep(0)
human.decide.set()
# The approved turn resumes at once, holding a borrowed slot.
await asyncio.wait_for(resumed.wait(), 2)
assert gate.in_flight == 2 and gate.awaiting_human == 0
# B's freed slot repays the borrow, so the ceiling is restored for new turns.
release_b.set()
await asyncio.wait_for(b, 2)
assert gate.in_flight == 1 and gate._sem._value == 0
finish_a.set()
await asyncio.wait_for(a, 2)
assert gate.in_flight == 0 and gate._sem._value == 1
@pytest.mark.asyncio
async def test_parallel_approvals_each_resume_holding_the_slot():
gate = _gate()
first, second = _AsyncHuman(), _AsyncHuman()
reg1, reg2 = _registry(first), _registry(second)
in_flight_after = []
async def branch(reg):
await reg.approve_async("bot", "tool", {}, force=True)
in_flight_after.append(gate.in_flight)
async def parked_turn():
async with gate.admit(session_id="a"):
await asyncio.gather(branch(reg1), branch(reg2))
a = asyncio.ensure_future(parked_turn())
await asyncio.wait_for(asyncio.gather(first.parked.wait(), second.parked.wait()), 2)
assert gate.in_flight == 0 and gate.awaiting_human == 1
release_b = asyncio.Event()
async def busy_turn():
async with gate.admit(session_id="b"):
await release_b.wait()
b = asyncio.ensure_future(busy_turn())
await asyncio.sleep(0)
first.decide.set()
await asyncio.sleep(0.05)
# The first answered branch resumed counted against the ceiling.
assert in_flight_after == [2] and gate.awaiting_human == 0
second.decide.set()
release_b.set()
await asyncio.wait_for(asyncio.gather(a, b), 2)
assert gate.in_flight == 0 and gate._sem._value == 1
@pytest.mark.asyncio
async def test_sync_approvals_filling_worker_pool_do_not_deadlock():
from concurrent.futures import ThreadPoolExecutor
gate = _gate()
human = _SyncHuman()
reg = _registry(human)
loop = asyncio.get_running_loop()
pool = ThreadPoolExecutor(max_workers=1)
async def run_sync(fn):
return await loop.run_in_executor(pool, contextvars.copy_context().run, fn)
async def parked_turn():
async with gate.admit(session_id="a"):
return await run_sync(lambda: reg.approve_sync("bot", "tool", {}, force=True))
try:
a = asyncio.ensure_future(parked_turn())
await loop.run_in_executor(None, human.parked.wait, 2)
await asyncio.sleep(0.05)
# B takes the freed slot and queues behind A's parked worker.
b = asyncio.ensure_future(_admitted(gate, "b", lambda: run_sync(lambda: "b")))
await asyncio.sleep(0.05)
human.decide.set()
results = await asyncio.wait_for(asyncio.gather(a, b), 2)
assert results[0].approved and results[1] == "b"
assert gate.in_flight == 0 and gate._sem._value == 1
finally:
pool.shutdown(wait=False)
async def _admitted(gate, session_id, body):
async with gate.admit(session_id=session_id):
return await body()
@pytest.mark.asyncio
async def test_sync_approval_in_worker_thread_frees_slot():
gate = _gate()
human = _SyncHuman()
reg = _registry(human)
loop = asyncio.get_running_loop()
async def parked_turn():
async with gate.admit(session_id="a"):
return await loop.run_in_executor(
None,
contextvars.copy_context().run,
lambda: reg.approve_sync("bot", "tool", {}, force=True),
)
a = asyncio.ensure_future(parked_turn())
await loop.run_in_executor(None, human.parked.wait, 2)
await asyncio.sleep(0.05)
assert gate.in_flight == 0 and gate.awaiting_human == 1
async with gate.admit(session_id="b"):
assert gate.in_flight == 1
human.decide.set()
assert (await asyncio.wait_for(a, 2)).approved
assert gate.in_flight == 0 and gate.awaiting_human == 0
assert gate._sem._value == 1
@pytest.mark.asyncio
async def test_cancelled_while_parked_does_not_over_release():
gate = _gate()
human = _AsyncHuman()
reg = _registry(human)
async def parked_turn():
async with gate.admit(session_id="a"):
await reg.approve_async("bot", "tool", {}, force=True)
a = asyncio.ensure_future(parked_turn())
await asyncio.wait_for(human.parked.wait(), 2)
a.cancel()
with pytest.raises(asyncio.CancelledError):
await a
assert gate.in_flight == 0 and gate.awaiting_human == 0
assert gate._sem._value == 1
@pytest.mark.asyncio
async def test_no_gate_bound_is_noop():
human = _AsyncHuman()
human.decide.set()
reg = _registry(human)
assert (await reg.approve_async("bot", "tool", {}, force=True)).approved
@@ -441,6 +441,23 @@ class BrowserServer:
self._agents[session_id] = agent
logger.info(f"[SERVER][DATA] _handle_start_session:server.py session_id={session_id}, agent_model={agent.model}")
# Side panel already runs CDP + observation after start_session. Echoing
# start_automation on the same WebSocket fails on some Windows stacks
# (OSError: [Errno 22]) and duplicates the automation loop.
if conn.is_extension:
conn.session_id = session_id
logger.info(
"[SERVER][START] Extension-originated start_session; "
"skipping start_automation echo"
)
return {
"type": "status",
"status": "running",
"session_id": session_id,
"message": f"Session started with goal: {goal}",
"start_automation_sent": False,
}
# Send start_automation to ONE extension client (first available without a session)
# Only send to ONE to prevent duplicate debugger attachment
@@ -453,8 +470,6 @@ class BrowserServer:
logger.debug(f"[SERVER][SCAN] _handle_start_session:server.py checking {len(extension_conns)} extension connection(s)")
import json as json_mod
# Deliver to ONE idle extension connection only (never a CLI client) to
# avoid duplicate debugger attachment. Extension availability was already
# validated above via extension_conns, which excludes CLI clients and
@@ -468,7 +483,7 @@ class BrowserServer:
client_conn.session_id = session_id
try:
logger.info("[SERVER][START] Sending start_automation to extension")
await client_conn.websocket.send_text(json_mod.dumps(start_msg))
await client_conn.websocket.send_json(start_msg)
logger.info(f"[SERVER][START] start_automation sent, session={session_id[:8]}")
sent_to_extension = True
break # Only send to ONE extension
@@ -487,7 +502,7 @@ class BrowserServer:
continue
client_conn.session_id = session_id
try:
await client_conn.websocket.send_text(json_mod.dumps(start_msg))
await client_conn.websocket.send_json(start_msg)
logger.info("[SERVER][START] Retry: start_automation delivered")
sent_to_extension = True
break
@@ -303,7 +303,7 @@ class TestStartSession:
server = BrowserServer()
conn = ClientConnection(websocket=Mock(), is_cli=True)
ext_ws = Mock()
ext_ws.send_text = AsyncMock()
ext_ws.send_json = AsyncMock()
ext = ClientConnection(websocket=ext_ws, is_extension=True)
server._connections["cli"] = conn
server._connections["ext"] = ext
@@ -314,7 +314,7 @@ class TestStartSession:
)
assert response["start_automation_sent"] is True
ext_ws.send_text.assert_awaited()
ext_ws.send_json.assert_awaited()
if server._sessions:
server._sessions.close()
@@ -339,7 +339,32 @@ class TestStartSession:
assert response.get("start_automation_sent") is True
# The delivery loop claims the extension connection.
assert ext.session_id is not None
ext.websocket.send_text.assert_awaited()
ext.websocket.send_json.assert_awaited()
if server._sessions:
server._sessions.close()
@pytest.mark.asyncio
async def test_extension_start_session_skips_start_automation_echo(self):
"""Extension-originated start_session must not echo start_automation."""
from praisonai_browser.server import BrowserServer, ClientConnection
server = BrowserServer()
ext = ClientConnection(websocket=AsyncMock(), is_extension=True)
server._connections["ext"] = ext
response = await server._handle_start_session(
{"type": "start_session", "goal": "Test"},
ext
)
assert response["type"] == "status"
assert response["status"] == "running"
assert response["start_automation_sent"] is False
assert ext.session_id == response["session_id"]
assert ext.session_id in server._agents
ext.websocket.send_json.assert_not_awaited()
ext.websocket.send_text.assert_not_awaited()
if server._sessions:
server._sessions.close()
@@ -358,7 +383,7 @@ class TestStartSession:
peer_ws.send_json = AsyncMock()
peer = ClientConnection(websocket=peer_ws, is_extension=False, is_cli=True)
ext_ws = Mock()
ext_ws.send_text = AsyncMock()
ext_ws.send_json = AsyncMock()
ext = ClientConnection(websocket=ext_ws, is_extension=True)
server._connections["cli"] = conn
server._connections["peer"] = peer
@@ -370,7 +395,7 @@ class TestStartSession:
)
assert response["start_automation_sent"] is True
ext_ws.send_text.assert_awaited()
ext_ws.send_json.assert_awaited()
peer_ws.send_text.assert_not_awaited()
peer_ws.send_json.assert_not_awaited()
@@ -528,7 +553,7 @@ class TestRegisterRole:
{"type": "start_session", "goal": "Test"}, conn
)
assert response["code"] == "NO_EXTENSION"
conn.websocket.send_text.assert_not_awaited()
conn.websocket.send_json.assert_not_awaited()
@pytest.mark.asyncio
async def test_ws_no_origin_cli_start_session(self):
@@ -546,8 +571,8 @@ class TestRegisterRole:
{"type": "start_session", "goal": "Test"}, cli
)
assert response["status"] == "running"
ext.websocket.send_text.assert_awaited_once()
cli.websocket.send_text.assert_not_awaited()
ext.websocket.send_json.assert_awaited_once()
cli.websocket.send_json.assert_not_awaited()
if server._sessions:
server._sessions.close()
@@ -406,6 +406,23 @@ def auth_login(
else:
output.print_success(f"API key validated: {live_msg}")
# The store holds one credential per provider, so never replace an
# existing, different key silently: confirm when the key was typed at the
# prompt, and always report what was replaced.
try:
previous = CredentialStore().get_credential(provider)
except Exception:
previous = None
replaced = None
if previous and previous.api_key != api_key:
if not key and not key_stdin and not typer.confirm(
f"Replace existing {provider} key {redact_key(previous.api_key)}?",
default=True,
):
raise typer.Exit(1)
replaced = redact_key(previous.api_key)
output.print_info(f"Replacing existing {provider} key {replaced}")
# Store credential
try:
store = CredentialStore()
@@ -419,11 +436,14 @@ def auth_login(
output.print_success(f"Stored credentials for {provider}")
if output.is_json_mode:
output.print_json({
result = {
"provider": provider,
"status": "stored",
"key_redacted": redact_key(api_key)
})
}
if replaced:
result["replaced_key_redacted"] = replaced
output.print_json(result)
else:
output.print_info(f"Key stored: {redact_key(api_key)}")
@@ -681,6 +681,23 @@ def config_provenance(
resolver = get_resolver()
provenance = resolver.resolve_with_provenance()
# The TOML layers above are not what picks the model for a bare
# run/chat/code invocation; surface the model the run will actually
# dispatch, tagged with the step of the resolution chain that won.
# A configured agent.model wins over the resolver, as in `run`.
if key in (None, "run.model"):
configured = provenance.get("agent.model") or {}
if configured.get("value"):
provenance["run.model"] = dict(configured)
else:
from ..configuration.model_resolver import explain_default_model
model, model_source = explain_default_model()
provenance["run.model"] = {
"value": model,
"layer": "model-resolver",
"source": model_source,
}
if key:
entry = provenance.get(key)
if entry is None:
@@ -19,7 +19,7 @@ helper that maps present credentials to a default.
import json
import os
from pathlib import Path
from typing import Optional
from typing import Optional, Tuple
from .paths import get_state_dir
@@ -134,50 +134,29 @@ def _provider_for_model(model: str) -> Optional[str]:
return None
def resolve_default_model(
explicit: Optional[str] = None,
*,
persist: bool = True,
notify: bool = True,
) -> str:
"""Resolve the default model for a zero-config run.
def explain_default_model(explicit: Optional[str] = None) -> Tuple[str, str]:
"""Return ``(model, source)`` for the model a run will actually dispatch.
Precedence: ``explicit`` > recent (persisted) > ``MODEL_NAME`` /
``OPENAI_MODEL_NAME`` > provider-aware best > the terminal fallback
(:data:`praisonai_code.llm.env.DEFAULT_FALLBACK_MODEL`).
Only *user-chosen* models (``explicit`` and the env overrides) are
persisted as the recent model. Provider-*inferred* defaults and the
terminal fallback are intentionally **not** persisted: persisting them
would let a stale inferred default short-circuit credential-based
inference on a later run after the user's available providers have changed.
Args:
explicit: An explicitly configured model (``--model``/config/YAML).
When given it wins and is persisted as the recent model.
persist: Whether to remember user-chosen models for next time.
notify: Whether to emit a one-line transparency notice the first time a
provider-aware default is inferred (no explicit, no recent model).
Returns:
The resolved model id.
Side-effect free (no persistence, no notice) so it can back introspection
such as ``praisonai config provenance``. ``source`` is one of
``"explicit"``, ``"state:<path>"``, ``"env:MODEL_NAME"`` /
``"env:OPENAI_MODEL_NAME"``, ``"local"``, ``"provider:<ENV_VAR>"`` or
``"fallback"``. :func:`resolve_default_model` is built on top of this so the
reported source can never drift from the resolution itself.
"""
if explicit:
if persist:
set_recent_model(explicit)
return explicit
return explicit, "explicit"
recent = get_recent_model()
if recent:
return recent
return recent, f"state:{_model_state_path()}"
# Honour OPENAI_MODEL_NAME / MODEL_NAME for backward compatibility before
# falling back to credential-based inference.
env_model = os.environ.get("MODEL_NAME") or os.environ.get("OPENAI_MODEL_NAME")
if env_model:
if persist:
set_recent_model(env_model)
return env_model
for var in ("MODEL_NAME", "OPENAI_MODEL_NAME"):
env_model = os.environ.get(var)
if env_model:
return env_model, f"env:{var}"
try:
from praisonai_code.llm.env import (
@@ -195,30 +174,62 @@ def resolve_default_model(
except Exception:
local = None
if local:
if notify:
try:
import typer
typer.echo(
f"No cloud key found; using local model "
f"{local.model}. Run `praisonai setup` to add a "
f"hosted provider."
)
except Exception:
pass
return local.model
return local.model, "local"
except Exception:
model = _fallback_model()
if notify:
provider = _provider_for_model(model)
if provider and os.environ.get(provider):
try:
import typer
typer.echo(
f"No model set; using {model} because {provider} is present."
)
except Exception:
pass
provider = _provider_for_model(model)
if provider and os.environ.get(provider):
return model, f"provider:{provider}"
return model, "fallback"
def resolve_default_model(
explicit: Optional[str] = None,
*,
persist: bool = True,
notify: bool = True,
) -> str:
"""Resolve the default model for a zero-config run.
Precedence: ``explicit`` > recent (persisted) > ``MODEL_NAME`` /
``OPENAI_MODEL_NAME`` > provider-aware best > the terminal fallback
(:data:`praisonai_code.llm.env.DEFAULT_FALLBACK_MODEL`). See
:func:`explain_default_model` for the source of the chosen model.
Only *user-chosen* models (``explicit`` and the env overrides) are
persisted as the recent model. Provider-*inferred* defaults and the
terminal fallback are intentionally **not** persisted: persisting them
would let a stale inferred default short-circuit credential-based
inference on a later run after the user's available providers have changed.
Args:
explicit: An explicitly configured model (``--model``/config/YAML).
When given it wins and is persisted as the recent model.
persist: Whether to remember user-chosen models for next time.
notify: Whether to emit a one-line transparency notice the first time a
provider-aware default is inferred (no explicit, no recent model).
Returns:
The resolved model id.
"""
model, source = explain_default_model(explicit)
if source == "explicit" or source.startswith("env:"):
if persist:
set_recent_model(model)
return model
if notify and source == "local":
_echo(
f"No cloud key found; using local model {model}. Run "
f"`praisonai setup` to add a hosted provider."
)
elif notify and source.startswith("provider:"):
_echo(
f"No model set; using {model} because "
f"{source.split(':', 1)[1]} is present."
)
# NOTE: provider-inferred defaults and the terminal fallback are
# deliberately NOT persisted. Persisting them would let a stale inferred
@@ -226,3 +237,11 @@ def resolve_default_model(
# user's available providers have changed, re-introducing the very
# zero-config failure this resolver exists to prevent.
return model
def _echo(message: str) -> None:
try:
import typer
typer.echo(message)
except Exception:
pass
@@ -226,6 +226,32 @@ def _coerce_source(obj: Any) -> "ToolSource":
)
def _yaml_role_configs(yaml_config: Dict[str, Any]) -> List[Dict[str, Any]]:
"""Agent configs from the legacy ``roles`` or canonical ``agents`` shape."""
roles = yaml_config.get('roles') or yaml_config.get('agents') or {}
return [cfg for cfg in roles.values() if isinstance(cfg, dict)]
def _yaml_tool_names(yaml_config: Dict[str, Any]) -> set:
"""Every tool name referenced in a parsed YAML config.
Single owner of which keys carry tool names (``roles[*].tools`` and
``roles[*].tasks[*].tools``) so building and validating cannot drift (#5753).
"""
names: set = set()
for role_cfg in _yaml_role_configs(yaml_config):
tasks = role_cfg.get('tasks') or {}
# Handle both dict and list formats for tasks
task_list = tasks.values() if isinstance(tasks, dict) else tasks if isinstance(tasks, list) else []
sources = [role_cfg] + [t for t in task_list if isinstance(t, dict)]
for cfg in sources:
tools = cfg.get('tools') or []
if not isinstance(tools, list):
continue
names.update(t.strip() for t in tools if isinstance(t, str) and t.strip())
return names
class ToolResolver:
"""Resolves tool names to callables from multiple sources.
@@ -817,26 +843,15 @@ class ToolResolver:
Returns:
List of tool/toolset names that could not be resolved (empty if all valid)
"""
missing = []
roles = yaml_config.get('roles', {})
# Also support 'agents' key for canonical format
if not roles:
roles = yaml_config.get('agents', {})
for role_name, role_config in roles.items():
if not isinstance(role_config, dict):
continue
# Validate tools
tools = role_config.get('tools', [])
if isinstance(tools, list):
for tool_name in tools:
if not tool_name or not isinstance(tool_name, str):
continue
if not self.has_tool(tool_name.strip()):
missing.append(tool_name)
missing = [name for name in _yaml_tool_names(yaml_config) if not self.has_tool(name)]
if missing:
# Accept the same local sources the build merges in, so a tool that
# only lives in tools.py / tools/ / .praisonai/tools/ is not
# falsely reported missing. Only scanned when something is unresolved.
local_names = set().union(*self._yaml_local_tool_sources())
missing = [name for name in missing if name not in local_names]
for role_config in _yaml_role_configs(yaml_config):
# Validate toolsets
toolsets = role_config.get('toolsets', [])
if toolsets:
@@ -1068,49 +1083,17 @@ class ToolResolver:
tools/ contents. Returns a {name: callable} dict ready for the adapter.
"""
tools_dict: Dict[str, Callable] = {}
needed: set[str] = set()
# Accept both the legacy ``roles`` shape and the canonical ``agents``
# shape (mirrors validate_yaml_tools) so tools resolve consistently.
role_configs = yaml_config.get('roles') or yaml_config.get('agents') or {}
for role_cfg in role_configs.values():
if not isinstance(role_cfg, dict):
continue
for t in role_cfg.get('tools') or []:
if isinstance(t, str) and t.strip():
needed.add(t.strip())
tasks = role_cfg.get('tasks') or {}
# Handle both dict and list formats for tasks
if isinstance(tasks, dict):
task_list = tasks.values()
elif isinstance(tasks, list):
task_list = tasks
else:
task_list = []
for task_cfg in task_list:
if not isinstance(task_cfg, dict):
continue
for t in task_cfg.get('tools') or []:
if isinstance(t, str) and t.strip():
needed.add(t.strip())
for name in needed:
for name in _yaml_tool_names(yaml_config):
resolved = self.resolve(name)
if resolved is None:
logger.warning("Tool %r not found", name)
continue
tools_dict[name] = resolved() if inspect.isclass(resolved) else resolved
# Restore original mutual exclusion: tools.py OR tools/ directory, not both.
# Honor this resolver's bound tools.py path so explicit/custom locations
# are respected instead of always assuming CWD.
tools_py_path = Path(self._tools_py_path)
tools_dir = tools_py_path.parent / 'tools'
# Local class-based tools (path B semantics) are layered on top of the
# chain result. Surface any name collision so a silent override of a
# chain-resolved tool by a same-named local class is debuggable.
def _merge_local(local_tools: Dict[str, Any]) -> None:
for local_tools in self._yaml_local_tool_sources():
for name in local_tools:
if name in tools_dict:
logger.warning(
@@ -1118,16 +1101,28 @@ class ToolResolver:
"the resolution chain", name,
)
tools_dict.update(local_tools)
return tools_dict
def _yaml_local_tool_sources(self) -> List[Dict[str, Any]]:
"""Local tool dicts merged into YAML builds, in precedence order.
Shared by :meth:`resolve_all_from_yaml` and :meth:`validate_yaml_tools`
so validation accepts exactly the local tools the build supplies.
"""
# Restore original mutual exclusion: tools.py OR tools/ directory, not both.
# Honor this resolver's bound tools.py path so explicit/custom locations
# are respected instead of always assuming CWD.
tools_py_path = Path(self._tools_py_path)
tools_dir = tools_py_path.parent / 'tools'
sources: List[Dict[str, Any]] = []
# Mutual exclusion: prefer tools.py when it exists, otherwise tools/.
if tools_py_path.is_file():
local_tools = self.get_local_tool_classes()
if local_tools:
_merge_local(local_tools)
sources.append(self.get_local_tool_classes())
logger.debug("%s exists. Loading tools.py and skipping tools folder.", tools_py_path)
# Otherwise load from tools/ directory if it exists
elif tools_dir.is_dir():
_merge_local(self.get_local_tool_classes_from_dir(tools_dir))
sources.append(self.get_local_tool_classes_from_dir(tools_dir))
logger.debug("tools folder exists in the root directory")
# Additively merge project-local .praisonai/tools/*.py @tool functions,
@@ -1136,8 +1131,8 @@ class ToolResolver:
# tools.py / tools/ paths above rather than replacing them, so a
# @tool-decorated function dropped into .praisonai/tools/ is available
# to YAML-defined agents by its tool name.
_merge_local(self._discover_praisonai_dir_tools())
return tools_dict
sources.append(self._discover_praisonai_dir_tools())
return sources
def _discover_praisonai_dir_tools(self) -> Dict[str, Any]:
"""Discover @tool functions from the ``.praisonai/tools/`` convention.
@@ -0,0 +1,53 @@
"""Re-running `auth login` must not silently replace a stored key (#5949)."""
from typer.testing import CliRunner
from praisonai_code.cli.commands import auth
from praisonai_code.cli.configuration.credentials import CredentialStore
OLD = "sk-old-aaaaaaaaaaaaaaaaaaaaaaaa1111"
NEW = "sk-new-bbbbbbbbbbbbbbbbbbbbbbbb2222"
def _setup(tmp_path, monkeypatch):
path = tmp_path / "creds.json"
CredentialStore(path).store_credential("openai", OLD)
monkeypatch.setattr(auth, "CredentialStore", lambda: CredentialStore(path))
return path
def test_prompted_login_declined_keeps_existing_key(tmp_path, monkeypatch):
path = _setup(tmp_path, monkeypatch)
result = CliRunner().invoke(
auth.app, ["login", "openai", "--skip-validation"], input=f"{NEW}\nn\n"
)
assert result.exit_code == 1
assert CredentialStore(path).get_credential("openai").api_key == OLD
def test_scripted_login_replaces_and_reports(tmp_path, monkeypatch):
path = _setup(tmp_path, monkeypatch)
result = CliRunner().invoke(
auth.app, ["login", "openai", "--key", NEW, "--skip-validation"]
)
assert result.exit_code == 0, result.output
assert "Replacing existing openai key" in result.output
assert CredentialStore(path).get_credential("openai").api_key == NEW
def test_json_login_reports_replaced_key(tmp_path, monkeypatch):
import json
from praisonai_code.cli.output.console import OutputController, OutputMode
_setup(tmp_path, monkeypatch)
monkeypatch.setattr(
auth, "get_output_controller", lambda: OutputController(mode=OutputMode.JSON)
)
result = CliRunner().invoke(
auth.app, ["login", "openai", "--key", NEW, "--skip-validation"]
)
assert result.exit_code == 0, result.output
payload = json.loads(result.output[result.output.rindex("{\n"):])
assert payload["replaced_key_redacted"].endswith("1111")
assert payload["key_redacted"].endswith("2222")
@@ -0,0 +1,67 @@
"""`config provenance` reports the model a run actually dispatches (#5906)."""
import json
import pytest
from typer.testing import CliRunner
from praisonai_code.cli.configuration import model_resolver
from praisonai_code.cli.configuration.model_resolver import (
explain_default_model,
resolve_default_model,
)
@pytest.fixture(autouse=True)
def _isolated_state(tmp_path, monkeypatch):
monkeypatch.setattr(model_resolver, "_model_state_path", lambda: tmp_path / "model.json")
monkeypatch.setattr(model_resolver, "_state_dir", lambda: tmp_path)
for var in ("MODEL_NAME", "OPENAI_MODEL_NAME"):
monkeypatch.delenv(var, raising=False)
return tmp_path
def test_explicit_wins():
assert explain_default_model("gpt-x") == ("gpt-x", "explicit")
def test_recent_state_reports_path(_isolated_state):
model_resolver.set_recent_model("claude-recent")
model, source = explain_default_model()
assert model == "claude-recent"
assert source == f"state:{_isolated_state / 'model.json'}"
def test_env_reports_var(monkeypatch):
monkeypatch.setenv("OPENAI_MODEL_NAME", "gpt-env")
assert explain_default_model() == ("gpt-env", "env:OPENAI_MODEL_NAME")
def test_explain_matches_resolve_and_has_no_side_effects(monkeypatch, _isolated_state):
monkeypatch.setenv("MODEL_NAME", "gpt-env")
explain_default_model()
assert not (_isolated_state / "model.json").exists()
assert resolve_default_model(notify=False) == explain_default_model()[0]
def test_config_provenance_includes_run_model(monkeypatch):
from praisonai_code.cli.commands.config import app
monkeypatch.setenv("MODEL_NAME", "gpt-env")
result = CliRunner().invoke(app, ["provenance", "run.model"])
assert result.exit_code == 0, result.output
# MODEL_NAME feeds agent.model via the config env layer, as `run` sees it.
assert "run.model = gpt-env" in result.output
assert "layer: environment" in result.output
def test_config_provenance_prefers_configured_agent_model(monkeypatch):
from praisonai_code.cli.commands.config import app
model_resolver.set_recent_model("claude-recent")
monkeypatch.setenv("PRAISONAI_CONFIG_CONTENT", '{"agent": {"model": "gpt-configured"}}')
result = CliRunner().invoke(app, ["provenance", "run.model"])
assert result.exit_code == 0, result.output
assert "run.model = gpt-configured" in result.output
assert "layer: env-config" in result.output
assert "claude-recent" not in result.output
@@ -465,3 +465,46 @@ class TestResolveAllFromYamlDiscovery:
lambda: tmp_path,
)
assert self._resolver().resolve_all_from_yaml({}) == {}
class TestYamlToolKeysShared:
"""Building and validating walk the same tool-bearing YAML keys (#5753)."""
CONFIG = {
"agents": {
"researcher": {
"tools": ["agent_tool"],
"tasks": {"t1": {"tools": [" task_tool "]}},
}
}
}
def test_validate_reports_task_level_tools(self, tmp_path, monkeypatch):
from praisonai_code.tool_resolver import ToolResolver
monkeypatch.chdir(tmp_path)
resolver = ToolResolver()
monkeypatch.setattr(resolver, "has_tool", lambda name: name == "agent_tool")
assert resolver.validate_yaml_tools(self.CONFIG) == ["task_tool"]
def test_build_resolves_agent_and_task_tools(self, tmp_path, monkeypatch):
from praisonai_code.tool_resolver import ToolResolver
monkeypatch.chdir(tmp_path)
resolver = ToolResolver()
requested = []
monkeypatch.setattr(resolver, "resolve", lambda name: requested.append(name))
resolver.resolve_all_from_yaml(self.CONFIG)
assert sorted(requested) == ["agent_tool", "task_tool"]
def test_validate_accepts_task_only_local_tool(self, tmp_path, monkeypatch):
"""A task tool the build supplies from local sources is not 'missing'."""
from praisonai_code.tool_resolver import ToolResolver
monkeypatch.chdir(tmp_path)
resolver = ToolResolver()
monkeypatch.setattr(resolver, "has_tool", lambda name: name == "agent_tool")
monkeypatch.setattr(
resolver, "_discover_praisonai_dir_tools", lambda: {"task_tool": len}
)
assert resolver.validate_yaml_tools(self.CONFIG) == []
@@ -143,6 +143,17 @@ def live_server(tmp_path, monkeypatch):
import praisonaiagents
monkeypatch.setattr(praisonaiagents, "Agent", _LiveAgent, raising=False)
# Capture the server so teardown can stop serve_forever(); removing the
# lockfile alone leaves a listening server thread behind (Issue #5938).
servers = []
class _TrackedServer(server_mod.ThreadingHTTPServer):
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
servers.append(self)
monkeypatch.setattr(server_mod, "ThreadingHTTPServer", _TrackedServer)
t = threading.Thread(
target=server_mod.serve_runtime,
kwargs={"host": "127.0.0.1", "port": 0, "idle_timeout": 0},
@@ -157,6 +168,10 @@ def live_server(tmp_path, monkeypatch):
time.sleep(0.05)
assert descriptor is not None, "runtime did not start"
yield descriptor
for httpd in servers:
httpd.shutdown()
t.join(5)
assert not t.is_alive(), "runtime server did not shut down"
RuntimeDescriptor.remove()
+3 -3
View File
@@ -46,7 +46,7 @@ This complements `PARITY.md`, which only tracks whether an export exists.
### `Agent.__init__`
- Python: `src/praisonai-agents/praisonaiagents/agent/agent.py:619`
- Python: `src/praisonai-agents/praisonaiagents/agent/agent.py:624`
- TypeScript: `src/praisonai-ts/src/agent/simple.ts:197` (ctor `src/praisonai-ts/src/agent/simple.ts:1019`)
- Counts: 42 python params: 29 exact, 8 camelCase, 2 alias, 3 flattened, 0 missing; 8 mismatches; 8 waived; 14 TS-only of 60
@@ -219,7 +219,7 @@ TS-only members: `previousResult`?, `onToken`?, `signal`?, `onEvent`?, `options`
### `Agent.chat`
- Python: `src/praisonai-agents/praisonaiagents/agent/chat_mixin.py:3345`
- Python: `src/praisonai-agents/praisonaiagents/agent/chat_mixin.py:3369`
- TypeScript: `src/praisonai-ts/src/agent/simple.ts:3592`
- Counts: 17 python params: 7 exact, 9 camelCase, 1 alias, 0 flattened, 0 missing; 0 mismatches; 0 waived; 2 TS-only of 19
@@ -279,7 +279,7 @@ TS-only members: `condition`?, `contextPolicy`?, `maxContextTokens`?, `maxContex
### `LLM.__init__`
- Python: `src/praisonai-agents/praisonaiagents/llm/llm.py:501`
- Python: `src/praisonai-agents/praisonaiagents/llm/llm.py:514`
- TypeScript: `src/praisonai-ts/src/llm/index.ts:236` (ctor `src/praisonai-ts/src/llm/index.ts:416`)
- Counts: 25 python params: 8 exact, 16 camelCase, 1 alias, 0 flattened, 0 missing; 1 mismatches; 1 waived; 1 TS-only of 26
+3 -3
View File
@@ -1185,7 +1185,7 @@
}
],
"python": {
"location": "src/praisonai-agents/praisonaiagents/agent/agent.py:619",
"location": "src/praisonai-agents/praisonaiagents/agent/agent.py:624",
"resolved_class": "Agent"
},
"ts_only": [
@@ -1797,7 +1797,7 @@
}
],
"python": {
"location": "src/praisonai-agents/praisonaiagents/agent/chat_mixin.py:3345",
"location": "src/praisonai-agents/praisonaiagents/agent/chat_mixin.py:3369",
"resolved_class": "ChatMixin"
},
"ts_only": [
@@ -4659,7 +4659,7 @@
}
],
"python": {
"location": "src/praisonai-agents/praisonaiagents/llm/llm.py:501",
"location": "src/praisonai-agents/praisonaiagents/llm/llm.py:514",
"resolved_class": "LLM"
},
"ts_only": [
+2 -1
View File
@@ -117,7 +117,8 @@ export interface GatewayEvent {
/**
* Standard gateway event types.
* Python parity: `EventType(str, Enum)` in praisonaiagents/gateway/protocols.py,
* exported from the package as `GatewayEventType`.
* exported from the package as `GatewayEventType`. Kept in lockstep by
* praisonai-agents/tests/unit/test_gateway_ts_event_type_lockstep.py.
*/
export enum GatewayEventType {
// Connection events
+6 -4
View File
@@ -1002,11 +1002,13 @@ agent.start('Write a movie script about a robot in Mars');
## ⚡ Performance
PraisonAI is built for speed, with agent instantiation in around 14μs. This reduces overhead, improves responsiveness, and helps multi-agent systems scale efficiently in real-world production workloads.
PraisonAI keeps framework overhead microsecond-class: creating an `Agent` (one tool, no LLM call) averaged **46 μs** in our 2026-10-15 run, so per-agent setup is negligible next to model latency in multi-agent workloads.
| Performance Metric | PraisonAI |
|--------------------|-----------|
| Avg Instantiation Time | **14 μs** |
| Performance Metric | PraisonAI | Measured |
|--------------------|-----------|----------|
| Avg Agent instantiation (1 tool, 1000 iterations) | **46 μs** | 2026-10-15 · praisonaiagents 1.7.12 |
Methodology, comparisons and how to reproduce: [benchmarks/BENCHMARK_RESULTS.md](https://github.com/MervinPraison/PraisonAI/blob/main/src/praisonai-agents/benchmarks/BENCHMARK_RESULTS.md) · [docs](https://docs.praison.ai/docs/developers/benchmarks)
---
@@ -1,3 +1,5 @@
> **Superseded.** Historical result from 2025-12-19, kept for reference only. Package versions and methodology differ from current runs; do not cite. Current results: [praisonai-agents/benchmarks/BENCHMARK_RESULTS.md](../../../praisonai-agents/benchmarks/BENCHMARK_RESULTS.md).
# PraisonAI CLI - Benchmark Results
**Generated:** 2025-12-19 16:04:40
@@ -12,8 +12,9 @@ from .base import BaseFrameworkAdapter, warn_unsupported_fields
logger = logging.getLogger(__name__)
# Extended YAML fields every AutoGen-family backend consumes. The rest are
# dropped and surfaced via warn_unsupported_fields.
_AUTOGEN_SUPPORTED_FIELDS = frozenset({"llm", "function_calling_llm"})
# dropped and surfaced via warn_unsupported_fields. AutoGen has no separate
# function-calling model, so ``function_calling_llm`` is deliberately absent.
_AUTOGEN_SUPPORTED_FIELDS = frozenset({"llm"})
def _owning_registry(adapter):
@@ -260,10 +261,20 @@ class AutoGenAdapter(BaseFrameworkAdapter):
# Surface any extended YAML fields this backend silently drops
warn_unsupported_fields(self, spec.extras)
# Per-agent ``llm`` (model string or dict) overlays the shared entry
agent_llm = spec.extras.get("llm")
if isinstance(agent_llm, str) and agent_llm.strip():
agent_llm = {"model": agent_llm.strip()}
if isinstance(agent_llm, dict) and agent_llm:
shared = dict(llm_config[0]) if llm_config else {}
agent_llm_config = {"config_list": [{**shared, **agent_llm}]}
else:
agent_llm_config = llm_config_dict
# Create AutoGen assistant agent
agents[spec.key] = autogen.AssistantAgent(
name=spec.role,
llm_config=llm_config_dict,
llm_config=agent_llm_config,
system_message=spec.backstory +
". Must Reply \"TERMINATE\" in the end when everything is done.",
)
@@ -72,17 +72,25 @@ class BaseFrameworkAdapter(_CoreBaseFrameworkAdapter):
"""Build a provider model object from spec and shared llm_config."""
from ..inc import PraisonAIModel
model, base, key = self._llm_settings(spec, llm_config)
return PraisonAIModel(model=model, base_url=base, api_key=key).get_model()
def _llm_settings(self, spec: Any, llm_config: Optional[List[Dict]]):
"""Return ``(model, base_url, api_key)`` for spec over shared llm_config."""
# Delegate model-name precedence to core (single source of truth); core
# returns the model string and discards base/key by design, so we derive
# them locally and upgrade to a provider object.
# them locally.
model = super()._resolve_llm(spec, llm_config)
base = key = None
# Per-agent base_url/api_key win over the shared llm_config so a role
# pointed at a different endpoint/tenant is not silently re-routed.
spec_llm = spec if isinstance(spec, dict) else {}
base = spec_llm.get("base_url")
key = spec_llm.get("api_key")
if llm_config and len(llm_config) > 0:
base = llm_config[0].get("base_url")
key = llm_config[0].get("api_key")
return PraisonAIModel(model=model, base_url=base, api_key=key).get_model()
base = base or llm_config[0].get("base_url")
key = key or llm_config[0].get("api_key")
return model, base, key
# Guards the legacy class-level patch path (below). Only used as a fallback when
@@ -209,11 +209,23 @@ class CrewAIAdapter(BaseFrameworkAdapter):
# Create the crew (kickoff is driven by run/arun so the same
# crew can be executed synchronously or asynchronously).
crew = Crew(
agents=list(agents.values()),
tasks=tasks,
verbose=True
)
# Honour the top-level memory/process/manager_llm/verbose keys
# CrewAI supports natively instead of silently dropping them.
crew_kwargs = {
"agents": list(agents.values()),
"tasks": tasks,
"verbose": config.get('verbose', True),
}
if config.get('memory'):
crew_kwargs["memory"] = True
if config.get('process') == 'hierarchical':
from crewai import LLM, Process
crew_kwargs["process"] = Process.hierarchical
# crewai.LLM keeps the model name (a bare SDK client would
# drop it and CrewAI would fall back to its default model).
model, base, key = self._llm_settings(config.get('manager_llm'), llm_config)
crew_kwargs["manager_llm"] = LLM(model=model, base_url=base, api_key=key)
crew = Crew(**crew_kwargs)
logger.debug("Final Crew Configuration:")
logger.debug(f"Agents: {crew.agents}")
Loaded 100 of 107 files, more files were not shown because too many files have changed in this diff. Show more