Compare commits

...
Author SHA1 Message Date
praisonai-triage-agent[bot]andMervin Praison 76e390fc75 fix: harden batch-2 fixes per reviewer feedback
- runner.py: isolate child process groups (start_new_session) on POSIX so
  timeout killpg cannot target the suite runner; propagate TimeoutError
  metadata + exit_code=-1 in the streaming timeout path
- agent_tools.py: route run_sync via _wrapper_bridge (C7 gate + standalone
  install hint); surface delete apply_plan failures instead of masking them;
  drop unused cwd param on acp_execute_command; B904 chaining; RUF013 Optional
- execution_mixin.py: reset _server_started[port] in finally so a failed or
  exited server can be retried by later launch() calls
- __init__.py: rebind package global db to the callable _LazyDbModule proxy so
  repeated praisonaiagents.db access returns the proxy, not the raw module
- compression_demo.py: derive target_tokens from each ratio so iterations differ
- multi-provider-agent.py: rebind each Task.agent to its RouterAgent wrapper

Co-authored-by: Mervin Praison <[email protected]>
2026-07-22 11:13:58 +00:00
praisonai-triage-agent[bot]andMervin Praison 4447dacb03 fix(code): route run_sync via wrapper bridge to satisfy C7 import gate
agent_tools.py imported `from praisonai._async_bridge import run_sync`
directly, tripping the C7 allowlist enforcement in check_c7_imports.sh
(non-allowlisted wrapper import in praisonai_code/). Load it lazily via
_wrapper_bridge.import_wrapper_module so standalone praisonai-code installs
get a clear install hint and the import-direction gate stays green.

Co-authored-by: Mervin Praison <[email protected]>
2026-07-22 11:05:32 +00:00
Dhivya-BharathyandCursor bc8a39dd29 fix: batch-2 E2E example failures from full audit
- Fix AgentTeam.start() result handling across 22+ workflow/usecase examples

- Fix missing AgentFlow/AgentTeam/AutoAgents imports and registry override demo

- Fix scope_identifiers, state_redis db import, and ToolsMCPServer.run_sse API

- Mark STT and stale tool-import examples as skip until refreshed

- Remove stale db mapping that shadowed praisonaiagents.db submodule

Co-authored-by: Cursor <[email protected]>
2026-07-22 13:34:33 +05:30
Dhivya-BharathyandCursor e182b36257 fix: resolve E2E examples failures from platform audit
- Kill Windows subprocess trees on suite-runner timeout (uvicorn hang)

- Restore Agent.launch() shared server globals in execution_mixin

- Add missing praisonai_code.cli.features.agent_tools module

- Fix compression_demo ContextCompressor API usage

- Import AgentFlow in workflow output/routing examples

- Use UTF-8 when parsing doctor ci subprocess output on Windows

- Set BrowserBaseTool __name__ for agent tool schema compatibility

Co-authored-by: Cursor <[email protected]>
2026-07-22 12:00:25 +05:30
praisonai-triage-agent[bot] 7367c5877d Merge pull request #3279 from MervinPraison/fix/praisonai-train-beginner-dx
praisonai-train: safe-by-default DX for beginners & agents (auto masking, no surprise pushes)
2026-07-21 17:47:22 +00:00
praisonai-triage-agent[bot]andMervin Praison 7223f7875b fix(train): guard publishing when train:false; frozenset KNOWN_KEYS
- Skip publishing (return early) when train:false leaves self.model None,
  preventing an AttributeError on the merged/gguf push path.
- Make KNOWN_KEYS a frozenset to prevent accidental class-state mutation.

Co-authored-by: Mervin Praison <[email protected]>
2026-07-21 17:15:24 +00:00
praisonai-triage-agent[bot]andMervin Praison 4c3c6b774e fix(train): validate num_samples & publish targets; fix stale interactive-tools union test
- num_samples: reject 0/negative/bool/non-int and honor explicit values (fail
  fast with a clear message instead of silently training the full set or
  crashing mid-run).
- Publishing: fail fast in validate_config when huggingface_save /
  huggingface_save_gguf / ollama_save is enabled without its target name,
  instead of silently skipping the requested upload after a long run.
- Fix pre-existing CI failure: test_interactive_group_is_union was stale after
  the `search` tool group was wired into the interactive default set (#3254);
  add TOOL_GROUPS["search"] to the expected union.

Co-authored-by: Mervin Praison <[email protected]>
2026-07-21 17:12:04 +00:00
praisonai-triage-agent[bot] 33ccfc76e0 Merge pull request #3248 from MervinPraison/claude/issue-3246-20260721-0648
fix: Typer Literal sandbox type + missing usage help category (#3246)
2026-07-21 17:05:44 +00:00
MervinPraisonandClaude Fable 5 cc7f086835 praisonai-train: make it just-work for beginners & agents (safe defaults)
Validated end-to-end on GPU (2x A6000, unsloth 2026.7.4 / trl 0.24.0).

- Response masking: assistant_only_loss now tri-state with default "auto" — a
  live probe checks whether the model's chat template actually supports assistant
  masking and enables it only when safe, else full-sequence SFT. Prevents TRL's
  cryptic RuntimeError on stock Qwen/Gemma/Llama templates (they lack the
  {% generation %} markers). Explicit true on an unsupported template now fails
  fast with a clear message. 'train_on_responses_only' accepted as an alias.
- Publishing defaults OFF and is gated on a target name being set — a plain
  'train locally' config finishes with the LoRA saved to lora_model/ instead of
  crashing after training on a missing/foreign hf_model_name.
- Config flags accept real YAML booleans (train: true) not just strings, via a
  _flag() coercion helper (no more AttributeError on .lower()).
- Honor dataset num_samples (was advertised but silently ignored).
- validate_config(): clear error on missing required keys, warn on unknown keys
  (typo/unsupported) so nothing is silently dropped — friendlier for agents.
- 'model' accepted as an alias for 'model_name' (matches --model flag).
- One concise run-summary block (model / examples / masking mode / steps / output).
- Align license to SPDX "MIT" + setuptools>=77 (matches main).

Co-Authored-By: Claude Fable 5 <[email protected]>
2026-07-21 18:04:37 +01:00
MervinPraisonandClaude Fable 5 796f06ff72 Fix publish pipeline: SPDX license, setuptools>=77, resilient uv lock
- pyproject (train + bot): license = "MIT" SPDX string (was deprecated table
  form); bump build requires to setuptools>=77 which the string form needs.
- _release_lib: 'uv lock' instead of 'uv lock --frozen' so publish regenerates a
  missing/stale lockfile (praisonai-train had none; dependency changes stale it)
  instead of failing with 'Unable to find lockfile'.

Co-Authored-By: Claude Fable 5 <[email protected]>
2026-07-21 17:48:47 +01:00
MervinPraison 966c287326 Bump praisonai-bot to 0.0.40 2026-07-21 17:44:17 +01:00
MervinPraison 4b7b24b20f Bump praisonai-code to 0.0.55 2026-07-21 17:43:07 +01:00
MervinPraison c1a15dce6d Bump praisonaiagents to 1.6.159 2026-07-21 17:41:56 +01:00
Mervin Praison 21ec8ebd3e Merge pull request #3239 from MervinPraison/claude/issue-3236-20260720-1715
fix: guard six trace emit methods before ContextEvent allocation
2026-07-21 17:40:29 +01:00
Mervin Praison e4485b56ad Merge pull request #3240 from MervinPraison/claude/issue-3238-20260720-1715
fix: delegate context aggregator token estimate to canonical heuristic
2026-07-21 17:40:16 +01:00
Mervin Praison bc5f964ad9 Merge pull request #3241 from MervinPraison/claude/issue-3237-20260720-1715
fix: gate parallel LLM context summary behind branch-count threshold
2026-07-21 17:39:50 +01:00
MervinPraison 6ffd00171a docs: auto-update documentation parity trackers [skip ci] 2026-07-21 16:39:25 +00:00
Mervin Praison 45c76d4702 Merge pull request #3250 from MervinPraison/claude/issue-3249-20260721-0712
fix: route Ollama tool-result formatting through OllamaAdapter
2026-07-21 17:38:43 +01:00
Mervin Praison e25e04b428 Merge pull request #3252 from MervinPraison/claude/issue-3251-20260721-0817
fix: close leaked generator resources, harden cli_backend duck-type, wire workflow YAML to observability
2026-07-21 17:38:27 +01:00
Mervin Praison a920bda3bc Merge pull request #3256 from MervinPraison/claude/issue-3255-20260721-0851
fix: gate reads of secret files by default in permission system
2026-07-21 17:38:04 +01:00
Mervin Praison 9ca14460a9 Merge pull request #3257 from MervinPraison/claude/issue-3254-20260721-0851
fix: wire fast search tools into coding agent default toolset
2026-07-21 17:37:36 +01:00
Mervin Praison 0e4508b995 Merge pull request #3258 from MervinPraison/claude/issue-3253-20260721-0850
feat: surface per-run git-worktree isolation via praisonai run --worktree
2026-07-21 17:37:20 +01:00
Mervin Praison 6854cfff98 Merge pull request #3261 from Dhivya-Bharathy/fix/examples-runner-windows-utf8
fix: use UTF-8 encoding in examples suite runner on Windows
2026-07-21 17:37:00 +01:00
Mervin Praison 110cbb4413 Merge pull request #3262 from MervinPraison/claude/issue-3260-20260721-0925
fix: add doctor auto-fix safety rails via RepairPlan
2026-07-21 17:36:40 +01:00
MervinPraison e557bd9e59 docs: auto-update feature parity trackers [skip ci] 2026-07-21 16:36:28 +00:00
Mervin Praison 72a05969ec Merge pull request #3263 from MervinPraison/claude/issue-3259-20260721-0924
fix: fail closed on known-weak/placeholder gateway auth secrets
2026-07-21 17:36:02 +01:00
Mervin Praison 3ee0649c7c Merge pull request #3266 from MervinPraison/claude/issue-3264-20260721-0942
fix: unify scheduler store so agent-authored jobs fire on gateway
2026-07-21 17:35:26 +01:00
Mervin Praison 078173047b Merge pull request #3267 from MervinPraison/claude/issue-3265-20260721-0943
fix: honour intentional-silence contract on scheduled delivery path
2026-07-21 17:35:00 +01:00
Mervin Praison 70373d8aac Merge pull request #3270 from MervinPraison/fix/auto-merge-cancelled-ci
fix: unblock auto-merge when smoke/windows cancel but core CI passes
2026-07-21 17:34:30 +01:00
Mervin Praison ffc6af00b4 Merge pull request #3271 from Iyanuoluwa007/fix-calculator-divide-by-zero
fix: calculator division by zero handling
2026-07-21 17:34:20 +01:00
Mervin Praison ae8f0cf62d Merge pull request #3272 from Iyanuoluwa007/fix-windows-test-signal-compatibility
fix: Windows compatibility in agent termination test
2026-07-21 17:33:54 +01:00
Mervin Praison 8baf7c2fca Merge pull request #3273 from Iyanuoluwa007/fix-telemetry-cleanup
fix: verify telemetry cleanup does not hang agent termination
2026-07-21 17:33:31 +01:00
Mervin Praison 256d63139d Merge pull request #3277 from MervinPraison/claude/issue-3275-20260721-1614
fix: consolidate duplicate API-key middleware into shared helper
2026-07-21 17:32:52 +01:00
Mervin Praison 4cb07c5ef4 Merge pull request #3278 from MervinPraison/claude/issue-3276-20260721-1615
fix: extract shared command_profiler helper for rag/knowledge CLI
2026-07-21 17:32:35 +01:00
Mervin Praison 94b1050af6 Merge pull request #3274 from MervinPraison/fix/praisonai-train-unsloth-latest
praisonai-train: align with latest Unsloth (modern TRL API, model-aware chat template)
2026-07-21 17:31:26 +01:00
praisonai-triage-agent[bot]andMervin Praison 7197e9e365 fix(praisonai-train): harden Unsloth trainer + drop stale uv.lock
- Fail fast when neither the tokenizer nor config provides a chat template,
  instead of silently training on empty text (swallowed apply_chat_template).
- Correct Gemma Ollama template to fold the system prompt into the first user
  turn per the Gemma spec (no separate system turn).
- Document that assistant_only_loss requires a template exposing assistant
  masks ({% generation %}); keep it opt-in and default-off.
- Remove the stale uv.lock (pinned trl<0.9.0 / trl 0.8.6, old torch/transformers/
  unsloth floors) so it regenerates against the modern pyproject stack.

Co-authored-by: Mervin Praison <[email protected]>
2026-07-21 16:22:15 +00:00
praisonai-triage-agent[bot]andMervinPraison 3c6a493d8a fix: extract shared command_profiler helper for rag/knowledge CLI (fixes #3276)
Both rag_profiler and knowledge_profiler were byte-for-byte identical
except the "command" label. Extract a single command_profiler helper in
cli/commands/_profiling.py and have both delegate to it, so profiling
logic has one owner and cannot drift. Behaviour, flags, and output are
unchanged.

Co-authored-by: MervinPraison <[email protected]>
2026-07-21 16:17:37 +00:00
praisonai-triage-agent[bot]andMervinPraison 1e60df1cda fix: consolidate duplicate API-key middleware into shared helper (fixes #3275)
Extract the bearer / X-API-Key token extraction and constant-time
comparison used by both the serve feature and the jobs server into a
single build_api_key_middleware() helper so future auth fixes only need
to be made in one place. Behaviour and lazy-import posture are unchanged;
per-server public-path allowlists remain distinct via a parameter. The
JobsAuthRequiredMiddleware non-localhost guard is left untouched.

Co-authored-by: MervinPraison <[email protected]>
2026-07-21 16:16:32 +00:00
MervinPraisonandClaude Fable 5 343beeb427 praisonai-train: align with latest Unsloth (modern TRL API, model-aware chat template)
The dependency pins (trl<0.9.0, unsloth>=2024.8) were incompatible with any recent
Unsloth, which requires trl>=0.18.2 / transformers>=4.51.3. This modernizes the LLM
fine-tuning path so it works with the current stack (tested on 2x A6000 with
unsloth 2026.7.4 / trl 0.24.0 / torch 2.10 / peft 0.19.1).

Changes:
- deps: transformers>=4.51.3, unsloth>=2025.9.1, trl>=0.18.2, peft>=0.13.0,
  accelerate>=0.34.0, bitsandbytes>=0.45.0 (pyproject + setup_conda_env.sh; the
  conda script no longer pins an ancient Unsloth git commit).
- trainer: migrate to modern TRL API — SFTConfig instead of TrainingArguments,
  processing_class instead of tokenizer, dataset_text_field/max_length/packing on
  SFTConfig; drop the pre-tokenize + dummy-field hack (TRL tokenizes internally).
- trainer: stop force-applying the llama-3.1 chat template to every model (broke
  Gemma/Qwen). Chat template is now config-driven and defaults to the model's own.
- trainer: honor config LoRA params (lora_r/alpha/target_modules/... were ignored).
- trainer: response-only loss via SFTConfig assistant_only_loss (model-agnostic),
  replacing the Llama-hardcoded train_on_responses_only markers.
- Ollama Modelfile: add Gemma template + stop tokens (was Llama-only, wrong for Gemma).

Tested: end-to-end LoRA fine-tune smoke runs on GPU for both Qwen2.5-0.5B and
Gemma-2-2B (native templates) — train loop + local save succeed.

Co-Authored-By: Claude Fable 5 <[email protected]>
2026-07-21 17:13:33 +01:00
Iyanuoluwa007 83cd7d7e9c fix: make agent termination test cross-platform 2026-07-21 15:44:47 +01:00
Iyanuoluwa007 5c62530930 test: verify telemetry cleanup does not hang agent termination 2026-07-21 15:07:20 +01:00
Iyanuoluwa007 0f97cc7550 fix: Windows compatibility in agent termination test 2026-07-21 14:43:28 +01:00
praisonai-triage-agent[bot]andMervin Praison f7d1aa027a fix: keep pending check runs in bestRunsByName to prevent premature merge
A completed cancelled run (rank 1) could override an in-progress run
(conclusion=null, rank 0) of the same name, hiding a still-running check
from allChecksGreenOnSha and allowing the merge gate to pass early.
Prioritize any non-completed run so pending checks always block.

Co-authored-by: Mervin Praison <[email protected]>
2026-07-21 12:31:47 +00:00
Iyanuoluwa007 b474eee596 Fix calculator division by zero handling 2026-07-21 13:28:51 +01:00
MervinPraison 15b1ffa777 fix: unblock auto-merge when smoke/windows cancel but core CI passes
Treat cancelled smoke and test-windows as non-blocking when all test-core
shards succeeded on HEAD, dedupe check runs by best conclusion, and raise
smoke (12m) and Windows (10m) job timeouts to reduce timeout cancellations.
2026-07-21 13:26:26 +01:00
Mervin Praison e05fcda111 Merge pull request #3269 from MervinPraison/claude/issue-3268-20260721-0951
fix: make Terminal-Bench/Harbor integration runnable + benchmark praisonai code
2026-07-21 13:12:18 +01:00
praisonai-triage-agent[bot]andMervin Praison 3b2962438e fix: standardize terminal_bench examples on gpt-4o-mini model
Switch all Harbor adapters, job configs, docs, and tests to
openai/gpt-4o-mini for a consistent, low-cost default across the
Terminal-Bench benchmark path.

Co-authored-by: Mervin Praison <[email protected]>
2026-07-21 10:21:11 +00:00
praisonai-triage-agent[bot]andMervin Praison 62d2a65c0f fix: propagate praisonai code exit status instead of masking with true
Surfaces genuine install/auth/startup failures in the Harbor code adapter
instead of always reporting success (per reviewer feedback). A benchmark
miss still exits 0 (Harbor grades by verification); only a crashed
assistant now fails the attempt.

Co-authored-by: Mervin Praison <[email protected]>
2026-07-21 10:03:47 +00:00
praisonai-triage-agent[bot]andMervinPraison 98701ee2a5 fix: make Terminal-Bench/Harbor integration runnable + benchmark praisonai code (fixes #3268)
Repairs the examples/terminal_bench/ Harbor adapters (external/wrapper/
installed), adds a headline praisonai_code_agent that drives `praisonai code`
headlessly, rewrites job configs to the real Harbor JobConfig schema for
Terminal-Bench 2.1, adds packaging __init__.py files, rewrites the tests to
skip only when Harbor is truly absent, and adds a manual-dispatch smoke
workflow + RESULTS.md. Examples/CI layer only — no SDK changes.

Co-authored-by: MervinPraison <[email protected]>
2026-07-21 09:57:19 +00:00
praisonai-triage-agent[bot]andMervin Praison 7a057de7f3 fix: log (not silently swallow) schedule store repoint/migration failures
Addresses reviewer feedback that set_store()/migration failing open could
hide reader/writer drift. The local tool store remains authoritative, so
these paths stay best-effort but are now diagnosable via logs.

Co-authored-by: Mervin Praison <[email protected]>
2026-07-21 09:52:46 +00:00
praisonai-triage-agent[bot]andMervin Praison 69c1790103 test: cover full-gateway intentional-silence suppression in ScheduledAgentExecutor
Adds direct coverage for the bot-side _execute_one silence path (exact
markers, whitespace variants, ordinary output, prose mention) asserting
delivery is suppressed while the run is still recorded as succeeded.

Co-authored-by: Mervin Praison <[email protected]>
2026-07-21 09:52:33 +00:00
praisonai-triage-agent[bot]andMervinPraison bb03dff081 fix: honour intentional-silence contract on scheduled delivery path (fixes #3265)
Scheduled/automation runs now consult the existing core silence primitive
(praisonaiagents.bots.silence.is_intentional_silence_response) before
delivering. When a run's whole output is an exact silence marker
(NO_REPLY / [SILENT] / SILENT) delivery is suppressed while the run still
completes and is recorded, so unattended monitors stop posting the raw
control token. Applied to both the wrapper lightweight path
(AgentScheduler._deliver_result) and the bot full-gateway path
(ScheduledAgentExecutor). Prose merely mentioning the token is unaffected
(exact-match semantics).

Co-authored-by: MervinPraison <[email protected]>
2026-07-21 09:46:35 +00:00
praisonai-triage-agent[bot]andMervinPraison e8d7c49907 fix: unify scheduler store so agent-authored jobs fire on gateway (fixes #3264)
Add a canonical get_default_store()/set_default_store() resolver in
praisonaiagents.scheduler and route the agent tools, gateway tick loop,
and host bridge through it so writer and reader share one backend.

Co-authored-by: MervinPraison <[email protected]>
2026-07-21 09:46:34 +00:00
praisonai-triage-agent[bot]andMervin Praison ebf59d1b74 fix: guard is_weak_secret against non-string inputs
Convert value to str before strip()/lower() so an unquoted YAML
auth_token (e.g. 12345) cannot raise AttributeError during the
weak-secret check. Preserves empty/None handling and denylist match.

Co-authored-by: Mervin Praison <[email protected]>
2026-07-21 09:38:51 +00:00
praisonai-triage-agent[bot]andMervin Praison fa99e51879 fix: doctor parity + neutralize high-entropy test tokens (#3259)
- gateway doctor now fails closed on a MISSING auth_token on an external
  bind (matching startup), not just a weak one (Greptile P1)
- doctor --json always emits a single parseable JSON document on the
  no-channels weak/missing-secret failure path (Greptile P2)
- replace high-entropy fake hex test tokens with an obviously-non-secret
  literal to clear GitGuardian false positives without weakening tests
- add TestDoctorGatewaySecretStrength covering the new doctor paths

Co-authored-by: Mervin Praison <[email protected]>
2026-07-21 09:35:40 +00:00
praisonai-triage-agent[bot]andMervin Praison bb4a198a51 fix: harden doctor RepairPlan safety rails (in-place repairs, collect failures, backup collisions, no-op applied)
Address Greptile review findings on the doctor auto-fix safety contract:
- In-place repairs no longer discarded: pass a fresh copy to apply_fix so
  rules that mutate-and-return their argument still record a diff.
- Collection failures now recorded as blocking refused findings instead of
  being silently swallowed (previously produced an apparently clean plan).
- Auto-derived backup paths use microsecond timestamps plus a numeric
  uniqueness suffix so a rapid/concurrent run never overwrites an earlier
  snapshot.
- No-op apply runs no longer report applied=True; applied now requires an
  actual change.

Adds tests for each case.

Co-authored-by: Mervin Praison <[email protected]>
2026-07-21 09:34:52 +00:00
praisonai-triage-agent[bot]andMervinPraison 21a6e960d0 fix: fail closed on known-weak/placeholder gateway auth secrets (fixes #3259)
Add a core weak-secret guard so an externally-bound gateway refuses to
start behind a publicly-known placeholder token (e.g. change-me,
your-token-here, or the literal $(openssl rand -hex 16) fix-hint pasted
verbatim). Loopback binds downgrade to a warning, consistent with the
existing permissive-loopback posture.

- core: KNOWN_WEAK_SECRETS, is_weak_secret, assert_gateway_secret_strong,
  WeakGatewaySecretError in praisonaiagents.gateway.protocols
- praisonai-bot: enforce in assert_external_bind_safe (fail closed on
  external, warn on loopback) and surface in `gateway doctor`

Co-authored-by: MervinPraison <[email protected]>
2026-07-21 09:29:14 +00:00
praisonai-triage-agent[bot]andMervinPraison d66401dcd6 fix: add doctor auto-fix safety rails via RepairPlan (fixes #3260)
Add a repair safety contract to the core doctor engine so self-repair is
safe to run: dry-run diff preview, backup, refuse-on-unrecoverable, and
post-fix re-validation. Introduces ConfigDiff/RepairPlan and a
DoctorRulesRegistry.plan_fixes() while keeping apply_all_fixes()/apply_fixes()
backward-compatible.

Co-authored-by: MervinPraison <[email protected]>
2026-07-21 09:28:16 +00:00
praisonai-triage-agent[bot]andMervin Praison 6627697001 fix: prevent data loss and shared state in worktree isolation (#3253)
Address reviewer-flagged issues in praisonai run --worktree:

- P0: untracked output no longer silently deleted. Teardown now detects
  changes via git status --porcelain (sees untracked files, which git diff
  does not) and commits them to the isolated branch, retaining the branch;
  only the worktree checkout is pruned. Falls back to keeping the worktree
  if the commit fails (e.g. no git identity), so output is never lost.
- P1: identical targets get independent worktrees/branches via a per-run
  unique token, so concurrent/repeated runs never share state.
- P1: relative/untracked local YAML now resolved to an absolute path before
  chdir, so praisonai run ./agents.yaml --worktree still loads its config.

Adds tests for untracked-output preservation and per-run independence.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-authored-by: Mervin Praison <[email protected]>
2026-07-21 09:04:20 +00:00
praisonai-triage-agent[bot]andMervin Praison 3ef7590d1e fix: honor workspace + search opt-out in wired search tools
- code_intelligence: core grep now searches the configured workspace
  (search_path) instead of the process CWD, matching the grep -rn fallback;
  paths outside the core tool's root fall back cleanly.
- interactive_tools: apply PRAISON_TOOLS_DISABLE via ToolConfig.__post_init__
  so direct constructions (to_tool_config, HeadlessInteractiveCore) honor the
  advertised opt-out, not just from_env().

Co-authored-by: Mervin Praison <[email protected]>
2026-07-21 09:02:21 +00:00
praisonai-triage-agent[bot]andMervin Praison df015d2134 fix: prevent broad read:* allow from bypassing secret-read gate
A catch-all read:* allow rule previously opted every secret file into
ALLOW, trivially defeating the gate. Now only a secret-specific allow
rule (e.g. read:*.env) or a persistent approval opts in; a broad
wildcard falls through to the ASK default. Deny and approvals still win.

Co-authored-by: Mervin Praison <[email protected]>
2026-07-21 09:00:31 +00:00
praisonai-triage-agent[bot]andMervinPraison 258509866c feat: add --worktree/--keep git isolation to praisonai run (fixes #3253)
Wire the existing core GitWorktreeAdapter into the CLI run lifecycle as a live consumer. praisonai run with --worktree provisions a fresh git worktree/branch, runs the agent there, reports the branch + a summary diff, and tears the worktree down (or keeps it with --keep). Degrades to a transparent no-op outside a git repo. Kept lightweight: no new Agent params/modules, only a self-contained context manager in run.py plus two CLI flags.

Co-authored-by: MervinPraison <[email protected]>
2026-07-21 08:56:13 +00:00
praisonai-triage-agent[bot]andMervinPraison 605f60a8ee fix: wire fast search tools into coding agent default toolset (fixes #3254)
Add a `search` tool group (grep, glob, ast_grep_search) to the interactive
coding agent's default toolset, resolving to the existing ripgrep-backed core
builtins. Also route code_intelligence search through the core grep first,
keeping `grep -rn` only as a last-resort fallback.

Co-authored-by: MervinPraison <[email protected]>
2026-07-21 08:55:26 +00:00
praisonai-triage-agent[bot]andMervinPraison 706d1aa9aa fix: gate reads of secret files by default in permission system (fixes #3255)
Reads of secret files (.env, private keys, *.pem, *.key, id_rsa, etc.) now
default to `ask` in the core permission manager, closing a silent
secret-exfiltration path where a default coding agent could read credentials
and forward them to the model provider. Safe example/sample/template files
remain allowed, and explicit user rules/approvals (e.g. `read:*.env=allow`)
override the default so opting in for trusted workflows stays a single rule.

Co-authored-by: MervinPraison <[email protected]>
2026-07-21 08:54:27 +00:00
praisonai-triage-agent[bot]andMervinPraison a185cb74f5 fix: close leaked generator resources, harden cli_backend duck-type, wire workflow YAML to observability (fixes #3251)
Gap 1: run/arun and the 4 legacy CLI call sites now own the
AgentsGenerator lifecycle via `with`, releasing its lazily-allocated
tool-timeout executor per run instead of leaking daemon threads in
long-lived server workers.

Gap 2: _is_protocol_instance / _is_cli_backend_instance now use
isinstance against the @runtime_checkable CliBackendProtocol instead of a
hasattr(execute, stream) duck-type, so a BaseCLIIntegration look-alike is
no longer mistaken for a backend; it fails fast at construction with a
pointed TypeError instead of crashing deep in the agent loop.

Gap 3: both workflow kickoff branches are bracketed in observability_session
so AgentOps init/finalize fires for workflow YAMLs too, and _build_yaml_workflow
now runs cli_backend compatibility validation and warns when a set
tool_timeout cannot be enforced on the native workflow path.

Co-authored-by: MervinPraison <[email protected]>
2026-07-21 08:32:14 +00:00
praisonai-triage-agent[bot]andMervin Praison b65e924022 fix: align _format_ollama_tool_result_message return type with adapter protocol
Co-authored-by: Mervin Praison <[email protected]>
2026-07-21 07:30:21 +00:00
praisonai-triage-agent[bot]andMervinPraison 6a391d1db4 fix: route Ollama tool-result formatting through OllamaAdapter (fixes #3249)
Make llm.py delegate _format_ollama_tool_result_message to the
protocol-driven OllamaAdapter.format_tool_result_message instead of a
hardcoded, behaviorally-diverged duplicate. The error-handling behavior
(apology-oriented prompt for dict/list results carrying an 'error' key)
is ported into the adapter so the single source of truth now lives in
the adapter layer, matching the pattern used elsewhere. Fully
backward-compatible: call sites are guarded by _is_ollama_provider(), so
Ollama formatting is always produced.

Co-authored-by: MervinPraison <[email protected]>
2026-07-21 07:24:54 +00:00
praisonai-triage-agent[bot]andDhivya-Bharathy 2132be0236 fix: replace Literal sandbox type with Enum and add usage help category (fixes #3246)
Bug A: Typer does not support typing.Literal; replaced _SandboxType
Literal with a str/Enum SandboxBackend so grouped --help lazy-loading
of the sandbox command no longer raises RuntimeError. Handler receives
.value to preserve string API.

Bug B: Added missing 'usage' command to COMMAND_CATEGORIES so every
registered lazy command has a help category.

Co-authored-by: Dhivya-Bharathy <[email protected]>
2026-07-21 06:50:59 +00:00
praisonai-triage-agent[bot]andMervin Praison 54dba703ad fix: preserve pass-through context for small fan-outs below 1500 tokens
Restores pre-gate behaviour where 1000-1500 token context was returned
unchanged for small branch counts, instead of being truncated.

Co-authored-by: Mervin Praison <[email protected]>
2026-07-20 17:23:10 +00:00
praisonai-triage-agent[bot]andMervin Praison 0562c17849 fix: token-aware truncation in ContextAggregator for non-ASCII budgets
Partial-truncation in aggregate() used a fixed remaining*4 char estimate,
which over-retained dense non-ASCII (CJK) text and could exceed max_tokens.
Now selects the longest prefix whose canonical token estimate fits the
remaining budget via _truncate_to_tokens(). Adds regression tests.

Co-authored-by: Mervin Praison <[email protected]>
2026-07-20 17:22:42 +00:00
praisonai-triage-agent[bot]andMervin Praison 8dd8156777 fix: guard knowledge_search/knowledge_add before ContextEvent allocation
Completes the disabled-path guard pattern for the two remaining trace
emit methods, matching the six methods fixed in this PR and their hotter
siblings. Byte-identical enabled behavior; only skips wasted allocation
when tracing is disabled.

Co-authored-by: Mervin Praison <[email protected]>
2026-07-20 17:20:18 +00:00
praisonai-triage-agent[bot]andMervinPraison 791419ae1a fix: gate parallel LLM context summary behind branch-count threshold (fixes #3237)
Small parallel groups (e.g. 2 branches) no longer pay for an extra LLM
summarisation call whose token cost roughly matches what it saves. The
cheaper _truncate_context_for_branches fallback is used below the
MIN_BRANCHES_FOR_LLM_SUMMARY threshold; LLM summarisation stays unchanged
for larger fan-outs.

Co-authored-by: MervinPraison <[email protected]>
2026-07-20 17:18:14 +00:00
praisonai-triage-agent[bot]andMervinPraison 885bd0afb0 fix: delegate context aggregator token estimate to canonical heuristic (fixes #3238)
Replace the crude `len(text) // 4 + 1` estimate in
ContextAggregator._estimate_tokens with a delegation to the package's
canonical estimate_tokens_heuristic. ASCII-identical, but now correctly
weights non-ASCII (CJK) characters, preventing context-budget overruns
for non-English content. Single source of truth for token estimation.

Co-authored-by: MervinPraison <[email protected]>
2026-07-20 17:17:44 +00:00
praisonai-triage-agent[bot]andMervinPraison 6041fe2d12 fix: guard six trace emit methods before ContextEvent allocation (fixes #3236)
Add early `if not self._enabled: return` to session_start, session_end,
context_snapshot, agent_handoff, memory_store, and memory_search, matching
the existing pattern in hotter siblings. Avoids allocating a ContextEvent
(and slicing query strings) on every memory/session/handoff op when tracing
is disabled. Output is identical when tracing is enabled.

Co-authored-by: MervinPraison <[email protected]>
2026-07-20 17:16:25 +00:00
131 changed files with 4343 additions and 5504 deletions

No files matched your search

+56
View File
@@ -40,6 +40,62 @@ assert('claude final reply detected', mg.isClaudeFinalReplyComment(withClaudeRep
assert('cancelled detect-and-trigger does not block', mg.OPTIONAL_CANCELLED_CHECKS.has('detect-and-trigger'));
const coreGreenRuns = [
{ name: 'test-core', status: 'completed', conclusion: 'success' },
{ name: 'test-core (cli)', status: 'completed', conclusion: 'success' },
{ name: 'smoke', status: 'completed', conclusion: 'cancelled' },
{ name: 'test-windows', status: 'completed', conclusion: 'cancelled' },
];
assert('core green allows cancelled smoke', mg.isAcceptableCheckConclusion(
{ name: 'smoke', status: 'completed', conclusion: 'cancelled' },
coreGreenRuns
));
assert('core green allows cancelled test-windows', mg.isAcceptableCheckConclusion(
{ name: 'test-windows', status: 'completed', conclusion: 'cancelled' },
coreGreenRuns
));
assert('cancelled smoke blocks when core missing', !mg.isAcceptableCheckConclusion(
{ name: 'smoke', status: 'completed', conclusion: 'cancelled' },
[{ name: 'smoke', status: 'completed', conclusion: 'cancelled' }]
));
assert('cancelled smoke blocks when core failed', !mg.isAcceptableCheckConclusion(
{ name: 'smoke', status: 'completed', conclusion: 'cancelled' },
[
{ name: 'test-core', status: 'completed', conclusion: 'failure' },
{ name: 'smoke', status: 'completed', conclusion: 'cancelled' },
]
));
assert('bestRunsByName prefers success over cancelled', (() => {
const best = mg.bestRunsByName([
{ name: 'smoke', conclusion: 'cancelled' },
{ name: 'smoke', conclusion: 'success' },
]);
return best.length === 1 && best[0].conclusion === 'success';
})());
assert('bestRunsByName prefers pending over completed cancelled', (() => {
const best = mg.bestRunsByName([
{ name: 'smoke', status: 'completed', conclusion: 'cancelled' },
{ name: 'smoke', status: 'in_progress', conclusion: null },
]);
return best.length === 1 && best[0].status === 'in_progress';
})());
assert('bestRunsByName keeps pending regardless of order', (() => {
const best = mg.bestRunsByName([
{ name: 'smoke', status: 'in_progress', conclusion: null },
{ name: 'smoke', status: 'completed', conclusion: 'success' },
]);
return best.length === 1 && best[0].status === 'in_progress';
})());
assert('allChecksGreenOnSha style: pending re-run blocks despite core green', (() => {
const runs = mg.bestRunsByName([
{ name: 'test-core', status: 'completed', conclusion: 'success' },
{ name: 'smoke', status: 'completed', conclusion: 'cancelled' },
{ name: 'smoke', status: 'queued', conclusion: null },
]);
const smoke = runs.find((r) => r.name === 'smoke');
return smoke.status !== 'completed';
})());
// Stale-FINAL recovery guards (PR #2560 push loop)
const nowMs = Date.now();
const iso = (ms) => new Date(ms).toISOString();
+79 -5
View File
@@ -44,6 +44,8 @@ const BLOCK_LABELS = new Set([
const MERGE_READY_LABEL = 'pipeline/merge-ready';
/** Superseded concurrency runs; must not block merge when real tests passed. */
const OPTIONAL_CANCELLED_CHECKS = new Set(['detect-and-trigger']);
/** Cancelled smoke/windows after timeout are non-blocking when core shards passed on HEAD. */
const OPTIONAL_CANCELLED_WHEN_CORE_GREEN = new Set(['smoke', 'test-windows']);
const BOT_REVIEWER_PATTERNS = [
'coderabbit',
'qodo',
@@ -325,8 +327,70 @@ function listFailedChecksOnSha(runs) {
});
}
function isCoreTestRun(run) {
const name = run?.name || '';
return name === 'test-core' || name.startsWith('test-core ') || name === 'test-core-collect';
}
function coreTestsGreenOnRuns(runs) {
const coreRuns = (runs || []).filter(isCoreTestRun);
if (coreRuns.length === 0) return false;
return coreRuns.every(
(run) =>
run.status === 'completed' &&
['success', 'neutral', 'skipped'].includes(run.conclusion)
);
}
function checkConclusionRank(conclusion) {
if (conclusion === 'success') return 4;
if (conclusion === 'neutral') return 3;
if (conclusion === 'skipped') return 2;
if (conclusion === 'cancelled') return 1;
return 0;
}
function isPendingRun(run) {
return !!(run && run.status && run.status !== 'completed');
}
function bestRunsByName(runs) {
const byName = new Map();
for (const run of runs || []) {
if (!run) continue;
const existing = byName.get(run.name);
if (!existing) {
byName.set(run.name, run);
continue;
}
const runPending = isPendingRun(run);
const existingPending = isPendingRun(existing);
if (existingPending) continue;
if (
runPending ||
checkConclusionRank(run.conclusion) > checkConclusionRank(existing.conclusion)
) {
byName.set(run.name, run);
}
}
return [...byName.values()];
}
function isAcceptableCheckConclusion(run, runs) {
if (['success', 'neutral', 'skipped'].includes(run.conclusion)) return true;
if (run.conclusion === 'cancelled' && OPTIONAL_CANCELLED_CHECKS.has(run.name)) return true;
if (
run.conclusion === 'cancelled' &&
OPTIONAL_CANCELLED_WHEN_CORE_GREEN.has(run.name) &&
coreTestsGreenOnRuns(runs)
) {
return true;
}
return false;
}
async function allChecksGreenOnSha(github, owner, repo, sha, core) {
const runs = await listChecksOnSha(github, owner, repo, sha);
const runs = bestRunsByName(await listChecksOnSha(github, owner, repo, sha));
if (runs.length === 0) {
core?.info?.(`No check runs on ${sha.slice(0, 7)} — allowing (e.g. docs-only PR)`);
return true;
@@ -336,13 +400,17 @@ async function allChecksGreenOnSha(github, owner, repo, sha, core) {
core?.info?.(`Check pending: ${run.name} (${run.status})`);
return false;
}
const ok =
['success', 'neutral', 'skipped'].includes(run.conclusion) ||
(run.conclusion === 'cancelled' && OPTIONAL_CANCELLED_CHECKS.has(run.name));
if (!ok) {
if (!isAcceptableCheckConclusion(run, runs)) {
core?.info?.(`Check failed: ${run.name} (${run.conclusion})`);
return false;
}
if (
run.conclusion === 'cancelled' &&
OPTIONAL_CANCELLED_WHEN_CORE_GREEN.has(run.name) &&
coreTestsGreenOnRuns(runs)
) {
core?.info?.(`Ignoring cancelled ${run.name} — test-core green on HEAD`);
}
}
return true;
}
@@ -855,6 +923,12 @@ module.exports = {
finalClaudeCompletedOnSha,
getMergeState,
OPTIONAL_CANCELLED_CHECKS,
OPTIONAL_CANCELLED_WHEN_CORE_GREEN,
isCoreTestRun,
coreTestsGreenOnRuns,
isPendingRun,
bestRunsByName,
isAcceptableCheckConclusion,
listChecksOnSha,
listFailedChecksOnSha,
allChecksGreenOnSha,
@@ -0,0 +1,47 @@
name: Terminal-Bench Smoke
# Manual (and optional weekly) smoke run of the `praisonai code` assistant on a
# small verified Terminal-Bench 2.1 subset via the Harbor harness.
# Never runs on PRs — it costs money (LLM calls) and is inherently flaky.
on:
workflow_dispatch:
schedule:
- cron: "0 6 * * 1" # Mondays 06:00 UTC
permissions:
contents: read
jobs:
smoke:
runs-on: ubuntu-latest
timeout-minutes: 45
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: "3.11"
- name: Install Harbor + PraisonAI
run: |
python -m pip install --upgrade pip
pip install harbor
pip install -e src/praisonai-agents -e src/praisonai
- name: Run Terminal-Bench 2.1 smoke subset
env:
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
PYTHONPATH: .
run: |
harbor run -c examples/terminal_bench/job_code_smoke.yaml || true
- name: Upload Harbor results
if: always()
uses: actions/upload-artifact@v4
with:
name: harbor-results
path: |
runs/
**/results.json
if-no-files-found: warn
+1 -1
View File
@@ -42,7 +42,7 @@ jobs:
# ============================================
smoke:
runs-on: ubuntu-latest
timeout-minutes: 8
timeout-minutes: 12
steps:
- uses: actions/checkout@v4
with:
+1 -1
View File
@@ -20,7 +20,7 @@ on:
jobs:
test-windows:
runs-on: windows-latest
timeout-minutes: 5
timeout-minutes: 10
env:
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
OPENAI_MODEL_NAME: gpt-4o-mini
View File
Whitespace-only changes.
+3 -1
View File
@@ -19,7 +19,9 @@ def run_doctor_ci():
[sys.executable, "-m", "praisonai", "doctor", "ci"],
capture_output=True,
text=True,
timeout=120
encoding="utf-8",
errors="replace",
timeout=120,
)
# Parse JSON output
+17 -6
View File
@@ -149,21 +149,32 @@ def main():
print(f"\nOriginal document length: {len(content)} characters")
chunks = [{"text": content, "metadata": {"source": doc_path}}]
# Estimate original tokens (~4 chars/token) so each ratio maps to a
# distinct token budget — compress() truncates to target_tokens.
original_tokens = max(1, len(content) // 4)
# Compress with different ratios
for ratio in [0.3, 0.5, 0.7]:
compressor = ContextCompressor(
max_tokens=2000,
target_ratio=ratio,
target_tokens = max(1, int(original_tokens * ratio))
compressor = ContextCompressor(verbose=False)
result = compressor.compress(
chunks,
query="security verification code",
target_tokens=target_tokens,
compression_ratio=ratio,
)
result = compressor.compress([content], query="security verification code")
print(f"\nCompression ratio {ratio}:")
print(f" Original tokens: {result.original_tokens}")
print(f" Compressed tokens: {result.compressed_tokens}")
print(f" Actual ratio: {result.compressed_tokens / max(result.original_tokens, 1):.2f}")
print(f" Actual ratio: {result.compression_ratio:.2f}")
# Check if secret code is preserved
compressed_text = " ".join(result.chunks) if result.chunks else ""
compressed_text = " ".join(
c.get("text", "") for c in result.chunks
) if result.chunks else ""
if SECRET_CODE in compressed_text:
print(f" ✅ Secret code preserved in compressed output")
else:
@@ -65,7 +65,6 @@ def main():
name="Policy_Expert",
instructions="You are a policy expert.",
knowledge=[temp_dir],
agent_id="policy_agent_v1", # Knowledge scoped to this agent
)
response = shared_agent.chat("What are the expense report rules?")
@@ -81,7 +80,6 @@ def main():
instructions="You are a personal HR assistant.",
knowledge=[temp_dir],
memory={"user_id": "user_bob"},
agent_id="hr_bot_v2",
)
response = combined_agent.chat("What training is required?")
+2 -1
View File
@@ -13,7 +13,8 @@ Expected Output:
Agent responds with state persisted to Redis
"""
from praisonaiagents import Agent, db
from praisonaiagents import Agent
from praisonaiagents.db import db
print("=== Redis State Store (Agent-First) ===")
@@ -61,7 +61,7 @@ def demonstrate_collision_problem():
_reset_default_registry()
# Agent 1 registers "custom" provider
register_llm_provider("custom", Agent1Provider)
register_llm_provider("custom", Agent1Provider, override=True)
print("Agent 1 registered 'custom' provider")
# Agent 2 tries to register same name - ERROR!
+3 -1
View File
@@ -1,3 +1,5 @@
# praisonai: skip=true
# Uses removed praisonaiagents.tools exports; pending example refresh
from praisonaiagents import Agent, Tools
from praisonaiagents.tools import read_csv, read_excel, write_csv, write_excel, filter_data, get_summary, group_by, pivot_table
import os
@@ -7,4 +9,4 @@ agent.start(f"""
Read the data from the csv file {os.path.join(os.path.dirname(__file__), "tesla-stock-price.csv")}
Analyse the data and give me the insights
read_csv to read the file
""")
""")
+3 -1
View File
@@ -1,5 +1,7 @@
# praisonai: skip=true
# Uses removed praisonaiagents.tools exports; pending example refresh
from praisonaiagents import Agent, Tools
from praisonaiagents.tools import get_stock_price, get_stock_info, get_historical_data
agent = Agent(instructions="You are a Research Agent", tools=[get_stock_price, get_stock_info, get_historical_data])
agent.start("Understand current stock price and historical data of Apple and Google. Tell me if I can invest in them")
agent.start("Understand current stock price and historical data of Apple and Google. Tell me if I can invest in them")
+11 -1
View File
@@ -135,7 +135,7 @@ def example_auto_agents_multi_provider():
from praisonaiagents.agents import AutoAgents
# Create AutoAgents that will automatically assign appropriate models
auto_agents = AutoAgentTeam(
auto_agents = AutoAgents(
instructions="Create a market research report on electric vehicles. Include data analysis, competitor analysis, and future projections.",
max_agents=3,
llm="gpt-4o-mini", # Default model for agent generation
@@ -144,6 +144,7 @@ def example_auto_agents_multi_provider():
# After agents are created, upgrade them to multi-model agents
multi_model_agents = []
wrapper_by_original = {}
for agent in auto_agents.agents:
# Convert regular agents to multi-model agents
multi_agent = RouterAgent(
@@ -155,9 +156,18 @@ def example_auto_agents_multi_provider():
models=["gpt-4o-mini", "gemini/gemini-1.5-flash", "claude-3-haiku-20240307", "gpt-4o-mini"],
routing_strategy="auto")
multi_model_agents.append(multi_agent)
wrapper_by_original[id(agent)] = multi_agent
# Update the agents in the AutoAgents instance
auto_agents.agents = multi_model_agents
# Rebind each task's agent to its wrapper — AutoAgents stores a direct agent
# reference on every Task and execution reads task.agent, so replacing only
# auto_agents.agents would leave the workflow bound to the original agents.
for task in getattr(auto_agents, "tasks", []) or []:
wrapper = wrapper_by_original.get(id(task.agent))
if wrapper is not None:
task.agent = wrapper
# Run the auto-generated workflow
results = auto_agents.start()
+3 -1
View File
@@ -1,3 +1,5 @@
# praisonai: skip=true
# Uses removed praisonaiagents.tools exports; pending example refresh
from praisonaiagents import Agent, Task, AgentTeam
from praisonaiagents.tools import wiki_search, wiki_summary, wiki_page, wiki_random, wiki_language
@@ -13,4 +15,4 @@ agent.start(
"First search the history of AI"
"Read the page of the history of AI"
"Get the summary of the page"
)
)
+3
View File
@@ -1,3 +1,5 @@
# praisonai: skip=true
# Requires local audio.mp3 fixture
from praisonaiagents import AudioAgent
agent = AudioAgent(llm="groq/whisper-large-v3") # 10x faster
@@ -5,3 +7,4 @@ text = agent.transcribe("audio.mp3", language="en")
print(text)
# Models: openai/whisper-1, groq/whisper-large-v3, deepgram/nova-2
+3
View File
@@ -1,3 +1,5 @@
# praisonai: skip=true
# Requires local audio.mp3 fixture
# Speech-to-Text with Groq (fastest)
# Requires: export GROQ_API_KEY=your-key
@@ -6,3 +8,4 @@ from praisonaiagents import AudioAgent
agent = AudioAgent(llm="groq/whisper-large-v3")
text = agent.listen("audio.mp3") # Replace with your audio file
print(text)
+3
View File
@@ -1,5 +1,8 @@
# praisonai: skip=true
# Requires local audio.mp3 fixture
from praisonaiagents import AudioAgent
agent = AudioAgent(llm="deepgram/nova-2")
text = agent.listen("audio.mp3")
print(text)
+3
View File
@@ -1,5 +1,8 @@
# praisonai: skip=true
# Requires local audio.mp3 fixture
from praisonaiagents import AudioAgent
agent = AudioAgent(llm="groq/whisper-large-v3")
text = agent.listen("audio.mp3")
print(text)
+3
View File
@@ -1,5 +1,8 @@
# praisonai: skip=true
# Requires local audio.mp3 fixture
from praisonaiagents import AudioAgent
agent = AudioAgent(llm="openai/whisper-1")
text = agent.listen("audio.mp3")
print(text)
@@ -18,7 +18,7 @@ from praisonaiagents import AutoAgents
from praisonaiagents.tools import duckduckgo
# Basic usage - AutoAgents analyzes complexity and creates optimal agents
agents = AutoAgentTeam(
agents = AutoAgents(
instructions="Search for information about AI Agents",
tools=[duckduckgo],
process="sequential", # or "hierarchical"
+8 -1
View File
@@ -102,7 +102,14 @@ def main():
# Print results
print("\nAutonomous Agent Results:")
print("=" * 50)
for task_id, result in results["task_results"].items():
task_results = (
results.get("task_results", {})
if isinstance(results, dict)
else {}
)
if not task_results and isinstance(results, str):
print(f"\nWorkflow output:\n{results}")
for task_id, result in task_results.items():
if result:
task_name = result.description
print(f"\nTask: {task_name}")
@@ -64,6 +64,13 @@ results = workflow.start()
# Print results
print("\nEvaluator-Optimizer Results:")
for task_id, result in results["task_results"].items():
task_results = (
results.get("task_results", {})
if isinstance(results, dict)
else {}
)
if not task_results and isinstance(results, str):
print(f"\nWorkflow output:\n{results}")
for task_id, result in task_results.items():
if result:
print(f"Task {task_id}: {result.raw}")
+1 -1
View File
@@ -1,4 +1,4 @@
from praisonaiagents.agents.agents import Agent, Task, Agents
from praisonaiagents.agents.agents import Agent, Task, Agents, AgentTeam
from praisonaiagents.tools import duckduckgo
# Test facts
@@ -104,6 +104,13 @@ results = workflow.start()
# Print results
print("\nOrchestrator-Workers Results:")
for task_id, result in results["task_results"].items():
task_results = (
results.get("task_results", {})
if isinstance(results, dict)
else {}
)
if not task_results and isinstance(results, str):
print(f"\nWorkflow output:\n{results}")
for task_id, result in task_results.items():
if result:
print(f"Task {task_id}: {result.raw}")
+8 -1
View File
@@ -86,7 +86,14 @@ async def main():
# Print results
print("\nParallel Processing Results:")
for task_id, result in results["task_results"].items():
task_results = (
results.get("task_results", {})
if isinstance(results, dict)
else {}
)
if not task_results and isinstance(results, str):
print(f"\nWorkflow output:\n{results}")
for task_id, result in task_results.items():
if result:
print(f"Task {task_id}: {result.raw}")
+9 -2
View File
@@ -1,6 +1,6 @@
from praisonaiagents.agent import Agent
from praisonaiagents.task import Task
from praisonaiagents.agents import Agents
from praisonaiagents.agents import Agents, AgentTeam
from typing import List, Dict
import time
@@ -76,6 +76,13 @@ results = workflow.start(return_dict=True)
# Print results
print("\nWorkflow Results:")
for task_id, result in results["task_results"].items():
task_results = (
results.get("task_results", {})
if isinstance(results, dict)
else {}
)
if not task_results and isinstance(results, str):
print(f"\nWorkflow output:\n{results}")
for task_id, result in task_results.items():
if result:
print(f"Task {task_id}: {result.raw}")
@@ -121,7 +121,14 @@ def main():
# Print results
print("\nAdaptive Learning Results:")
print("=" * 50)
for task_id, result in results["task_results"].items():
task_results = (
results.get("task_results", {})
if isinstance(results, dict)
else {}
)
if not task_results and isinstance(results, str):
print(f"\nWorkflow output:\n{results}")
for task_id, result in task_results.items():
if result:
print(f"\nTask: {task_id}")
print(f"Result: {result.raw}")
+8 -1
View File
@@ -204,7 +204,14 @@ async def main():
# Print results
print("\nClimate Impact Analysis Results:")
print("=" * 50)
for task_id, result in results["task_results"].items():
task_results = (
results.get("task_results", {})
if isinstance(results, dict)
else {}
)
if not task_results and isinstance(results, str):
print(f"\nWorkflow output:\n{results}")
for task_id, result in task_results.items():
if result:
print(f"\nTask: {task_id}")
print(f"Result: {result.raw}")
+8 -1
View File
@@ -97,7 +97,14 @@ def main():
# Print results
print("\nCode Review Results:")
print("=" * 50)
for task_id, result in results["task_results"].items():
task_results = (
results.get("task_results", {})
if isinstance(results, dict)
else {}
)
if not task_results and isinstance(results, str):
print(f"\nWorkflow output:\n{results}")
for task_id, result in task_results.items():
if result:
print(f"\nTask: {task_id}")
print(f"Result: {result.raw}")
+8 -1
View File
@@ -222,7 +222,14 @@ async def main():
# Print results
print("\nCryptography Validation Results:")
print("=" * 50)
for task_id, result in results["task_results"].items():
task_results = (
results.get("task_results", {})
if isinstance(results, dict)
else {}
)
if not task_results and isinstance(results, str):
print(f"\nWorkflow output:\n{results}")
for task_id, result in task_results.items():
if result:
print(f"\nTask: {task_id}")
print(f"Result: {result.raw}")
+8 -1
View File
@@ -128,7 +128,14 @@ def main():
# Print results
print("\nCustomer Service Results:")
print("=" * 50)
for task_id, result in results["task_results"].items():
task_results = (
results.get("task_results", {})
if isinstance(results, dict)
else {}
)
if not task_results and isinstance(results, str):
print(f"\nWorkflow output:\n{results}")
for task_id, result in task_results.items():
if result:
print(f"\nTask: {task_id}")
print(f"Result: {result.raw}")
@@ -208,7 +208,14 @@ async def main():
# Print results
print("\nMarket Making Results:")
print("=" * 50)
for task_id, result in results["task_results"].items():
task_results = (
results.get("task_results", {})
if isinstance(results, dict)
else {}
)
if not task_results and isinstance(results, str):
print(f"\nWorkflow output:\n{results}")
for task_id, result in task_results.items():
if result:
print(f"\nTask: {task_id}")
print(f"Result: {result.raw}")
@@ -401,7 +401,14 @@ async def main():
# Print results
print("\nDisaster Recovery Results:")
print("=" * 50)
for task_id, result in results["task_results"].items():
task_results = (
results.get("task_results", {})
if isinstance(results, dict)
else {}
)
if not task_results and isinstance(results, str):
print(f"\nWorkflow output:\n{results}")
for task_id, result in task_results.items():
if result:
print(f"\nTask: {task_id}")
print(f"Result: {result.raw}")
@@ -128,7 +128,14 @@ def main():
# Print results
print("\nEmergency Response Results:")
print("=" * 50)
for task_id, result in results["task_results"].items():
task_results = (
results.get("task_results", {})
if isinstance(results, dict)
else {}
)
if not task_results and isinstance(results, str):
print(f"\nWorkflow output:\n{results}")
for task_id, result in task_results.items():
if result:
print(f"\nTask: {task_id}")
print(f"Result: {result.raw}")
+8 -1
View File
@@ -125,7 +125,14 @@ async def main():
# Print results
print("\nFraud Detection Results:")
print("=" * 50)
for task_id, result in results["task_results"].items():
task_results = (
results.get("task_results", {})
if isinstance(results, dict)
else {}
)
if not task_results and isinstance(results, str):
print(f"\nWorkflow output:\n{results}")
for task_id, result in task_results.items():
if result:
print(f"\nTask: {task_id}")
print(f"Result: {result.raw}")
@@ -153,7 +153,14 @@ def main():
# Print results
print("\nDiagnosis Results:")
print("=" * 50)
for task_id, result in results["task_results"].items():
task_results = (
results.get("task_results", {})
if isinstance(results, dict)
else {}
)
if not task_results and isinstance(results, str):
print(f"\nWorkflow output:\n{results}")
for task_id, result in task_results.items():
if result:
print(f"\nTask: {task_id}")
print(f"Result: {result.raw}")
@@ -206,7 +206,14 @@ async def main():
# Print results
print("\nProtocol Generation Results:")
print("=" * 50)
for task_id, result in results["task_results"].items():
task_results = (
results.get("task_results", {})
if isinstance(results, dict)
else {}
)
if not task_results and isinstance(results, str):
print(f"\nWorkflow output:\n{results}")
for task_id, result in task_results.items():
if result:
print(f"\nTask: {task_id}")
print(f"Result: {result.raw}")
@@ -150,7 +150,14 @@ def main():
# Print results
print("\nContent Generation Results:")
print("=" * 50)
for task_id, result in results["task_results"].items():
task_results = (
results.get("task_results", {})
if isinstance(results, dict)
else {}
)
if not task_results and isinstance(results, str):
print(f"\nWorkflow output:\n{results}")
for task_id, result in task_results.items():
if result:
print(f"\nTask: {task_id}")
print(f"Result: {result.raw}")
@@ -191,7 +191,14 @@ async def main():
# Print results
print("\nArchitecture Search Results:")
print("=" * 50)
for task_id, result in results["task_results"].items():
task_results = (
results.get("task_results", {})
if isinstance(results, dict)
else {}
)
if not task_results and isinstance(results, str):
print(f"\nWorkflow output:\n{results}")
for task_id, result in task_results.items():
if result:
print(f"\nTask: {task_id}")
print(f"Result: {result.raw}")
@@ -169,7 +169,14 @@ async def main():
# Print results
print("\nMaintenance Planning Results:")
print("=" * 50)
for task_id, result in results["task_results"].items():
task_results = (
results.get("task_results", {})
if isinstance(results, dict)
else {}
)
if not task_results and isinstance(results, str):
print(f"\nWorkflow output:\n{results}")
for task_id, result in task_results.items():
if result:
print(f"\nTask: {task_id}")
print(f"Result: {result.raw}")
@@ -186,7 +186,14 @@ async def main():
# Print results
print("\nOptimization Results:")
print("=" * 50)
for task_id, result in results["task_results"].items():
task_results = (
results.get("task_results", {})
if isinstance(results, dict)
else {}
)
if not task_results and isinstance(results, str):
print(f"\nWorkflow output:\n{results}")
for task_id, result in task_results.items():
if result:
print(f"\nTask: {task_id}")
print(f"Result: {result.raw}")
@@ -181,7 +181,14 @@ async def main():
# Print results
print("\nResearch Analysis Results:")
print("=" * 50)
for task_id, result in results["task_results"].items():
task_results = (
results.get("task_results", {})
if isinstance(results, dict)
else {}
)
if not task_results and isinstance(results, str):
print(f"\nWorkflow output:\n{results}")
for task_id, result in task_results.items():
if result:
print(f"\nTask: {task_id}")
print(f"Result: {result.raw}")
+8 -1
View File
@@ -175,7 +175,14 @@ def main():
# Print results
print("\nOptimization Results:")
print("=" * 50)
for task_id, result in results["task_results"].items():
task_results = (
results.get("task_results", {})
if isinstance(results, dict)
else {}
)
if not task_results and isinstance(results, str):
print(f"\nWorkflow output:\n{results}")
for task_id, result in task_results.items():
if result:
print(f"\nTask: {task_id}")
print(f"Result: {result.raw}")
+8 -1
View File
@@ -220,7 +220,14 @@ async def main():
# Print results
print("\nResource Optimization Results:")
print("=" * 50)
for task_id, result in results["task_results"].items():
task_results = (
results.get("task_results", {})
if isinstance(results, dict)
else {}
)
if not task_results and isinstance(results, str):
print(f"\nWorkflow output:\n{results}")
for task_id, result in task_results.items():
if result:
print(f"\nTask: {task_id}")
print(f"Result: {result.raw}")
+8 -1
View File
@@ -102,7 +102,14 @@ def main():
# Print results
print("\nRisk Management Results:")
print("=" * 50)
for task_id, result in results["task_results"].items():
task_results = (
results.get("task_results", {})
if isinstance(results, dict)
else {}
)
if not task_results and isinstance(results, str):
print(f"\nWorkflow output:\n{results}")
for task_id, result in task_results.items():
if result:
print(f"\nTask: {task_id}")
print(f"Result: {result.raw}")
@@ -184,7 +184,14 @@ async def main():
# Print results
print("\nVulnerability Detection Results:")
print("=" * 50)
for task_id, result in results["task_results"].items():
task_results = (
results.get("task_results", {})
if isinstance(results, dict)
else {}
)
if not task_results and isinstance(results, str):
print(f"\nWorkflow output:\n{results}")
for task_id, result in task_results.items():
if result:
print(f"\nTask: {task_id}")
print(f"Result: {result.raw}")
@@ -4,7 +4,7 @@ Agentic Loop with CSV Workflow Example
Demonstrates iterating over a CSV file with an agent processing each row.
"""
from praisonaiagents import Agent, Workflow
from praisonaiagents import Agent, AgentFlow, Workflow
from praisonaiagents.workflows import loop
import tempfile
import os
+1 -1
View File
@@ -5,7 +5,7 @@ Demonstrates the evaluator-optimizer pattern where an agent
generates content and another evaluates it, repeating until approved.
"""
from praisonaiagents import Agent, Workflow
from praisonaiagents import Agent, AgentFlow, Workflow
from praisonaiagents.workflows import repeat
# Create generator agent
+1 -1
View File
@@ -12,7 +12,7 @@ Prerequisites:
--port 6060
"""
from praisonaiagents import Agent, Workflow
from praisonaiagents import Agent, AgentFlow, Workflow
ROUTELLM_URL = "http://localhost:6060/v1"
+1 -1
View File
@@ -78,7 +78,7 @@ def main():
}''')
print("\nPress Ctrl+C to stop")
server.run(transport="sse", host="0.0.0.0", port=8081)
server.run_sse(port=8081)
def run_manual_server():
+41 -16
View File
@@ -1,18 +1,40 @@
# Terminal-Bench 2.0 Integration with PraisonAI
# Terminal-Bench 2.1 Integration with PraisonAI
This directory contains examples for integrating PraisonAI Agents with **Terminal-Bench 2.0** via the **Harbor framework** for AI agent benchmarking.
This directory contains adapters for benchmarking PraisonAI on **Terminal-Bench 2.1** via the **Harbor framework**. Everything lives in the examples/benchmark layer + CI — there are **zero changes to the `praisonaiagents` / `praisonai` SDK surface**.
## Overview
[Terminal-Bench 2.0](https://tbench.ai) is a Stanford/Laude Institute benchmark that has become the gold standard for evaluating AI coding agents in real terminal environments. The [Harbor framework](https://harborframework.com) is the official evaluation harness that abstracts container lifecycle, parallelization, and cloud providers.
[Terminal-Bench 2.1](https://www.tbench.ai/docs/run-terminal-bench-2-1) is the current release of the Laude Institute benchmark for evaluating AI coding agents in real terminal environments. The [Harbor framework](https://github.com/laude-institute/harbor) is the official evaluation harness that abstracts container lifecycle, parallelization, and cloud providers.
## Setup
Install Harbor and PraisonAI into **one venv** and run from the repo root with `PYTHONPATH=.` so the adapter import path resolves:
```bash
pip install harbor
pip install -e src/praisonai-agents -e src/praisonai
export PYTHONPATH=.
```
## Integration Types
### 0. Code Agent (headline — benchmarks `praisonai code`)
- **File**: `praisonai_code_agent.py`
- **Purpose**: Installs and drives the terminal-native `praisonai code` assistant headlessly inside the container.
- **Run**:
```bash
PYTHONPATH=. harbor run -d terminal-bench/terminal-bench-2-1 \
--agent "examples.terminal_bench.praisonai_code_agent:PraisonAICodeAgent" \
-m openai/gpt-4o-mini \
--ae OPENAI_API_KEY=$OPENAI_API_KEY \
-n 4
```
### 1. External Agent (Direct Agent Class)
- **File**: `praisonai_external_agent.py`
- **Purpose**: External agent that uses direct `Agent()` class instantiation
- **Usage**: Run with `--agent-import-path` flag
- **Approach**: Uses `praisonaiagents.Agent` directly with `execute_command` tool
- **Usage**: Run with `--agent "module:Class"`
- **Approach**: Uses `praisonaiagents.Agent` directly with a bash tool bridged to Harbor's `exec()`
### 2. Wrapper Agent (CLI-Based)
- **File**: `praisonai_wrapper_agent.py`
@@ -42,12 +64,12 @@ pip install praisonaiagents[tools]
```bash
# Test with oracle agent first
harbor run -d terminal-bench/terminal-bench-2 -a oracle
harbor run -d terminal-bench/terminal-bench-2-1 -a oracle
# Run PraisonAI external agent (uses direct Agent() class)
harbor run -d terminal-bench/terminal-bench-2 \
--agent-import-path examples.terminal_bench.praisonai_external_agent:PraisonAIExternalAgent \
--model openai/gpt-4o \
PYTHONPATH=. harbor run -d terminal-bench/terminal-bench-2-1 \
--agent "examples.terminal_bench.praisonai_external_agent:PraisonAIExternalAgent" \
--model openai/gpt-4o-mini \
--ae OPENAI_API_KEY=$OPENAI_API_KEY \
-n 4
```
@@ -56,9 +78,9 @@ harbor run -d terminal-bench/terminal-bench-2 \
```bash
# Run PraisonAI wrapper agent (uses `praisonai "TASK"` CLI pattern)
harbor run -d terminal-bench/terminal-bench-2 \
--agent-import-path examples.terminal_bench.praisonai_wrapper_agent:PraisonAIWrapperAgent \
--model openai/gpt-4o \
PYTHONPATH=. harbor run -d terminal-bench/terminal-bench-2-1 \
--agent "examples.terminal_bench.praisonai_wrapper_agent:PraisonAIWrapperAgent" \
--model openai/gpt-4o-mini \
--ae OPENAI_API_KEY=$OPENAI_API_KEY \
-n 4
```
@@ -66,9 +88,9 @@ harbor run -d terminal-bench/terminal-bench-2 \
### Running on Cloud (Daytona/E2B/Modal)
```bash
harbor run -d terminal-bench/terminal-bench-2 \
--agent-import-path examples.terminal_bench.praisonai_external_agent:PraisonAIExternalAgent \
--model openai/gpt-4o \
PYTHONPATH=. harbor run -d terminal-bench/terminal-bench-2-1 \
--agent "examples.terminal_bench.praisonai_external_agent:PraisonAIExternalAgent" \
--model openai/gpt-4o-mini \
--env daytona -n 32 \
--ae OPENAI_API_KEY=$OPENAI_API_KEY
```
@@ -100,6 +122,9 @@ harbor run -d terminal-bench/terminal-bench-2 \
## Files
- `README.md` - This documentation
- `praisonai_code_agent.py` - **Headline** adapter benchmarking `praisonai code`
- `job_code_smoke.yaml` - Smoke-test job config for the code agent
- `RESULTS.md` - Recorded pass-rates
- `praisonai_external_agent.py` - External agent (direct Agent class)
- `praisonai_wrapper_agent.py` - Wrapper agent (CLI-based approach)
- `praisonai_installed_agent.py` - Installed agent implementation
@@ -127,7 +152,7 @@ Each task provides:
## Contributing
1. Test changes with oracle agent first: `harbor run -d terminal-bench/terminal-bench-2 -a oracle`
1. Test changes with oracle agent first: `harbor run -d terminal-bench/terminal-bench-2-1 -a oracle`
2. Run real agentic tests to ensure end-to-end functionality
3. Follow PraisonAI's AGENTS.md architecture guidelines
4. Add both unit tests and integration tests
+19
View File
@@ -0,0 +1,19 @@
# Terminal-Bench 2.1 Results — PraisonAI
Pass-rates from the Harbor smoke workflow
(`.github/workflows/terminal-bench-smoke.yml`, manual dispatch). The full 2.1
dataset (89 tasks) can be scored by dropping the `task_names` filter in
`job.yaml`.
| Date | Agent | Model | Tasks | Pass rate | Harbor artifact |
|------|-------|-------|-------|-----------|-----------------|
| _pending first run_ | `praisonai code` | `openai/gpt-4o-mini` | smoke (3) | — | — |
## How scores are produced
```bash
PYTHONPATH=. harbor run -c examples/terminal_bench/job_code_smoke.yaml
```
The workflow uploads the Harbor results directory as a CI artifact and appends
the pass-rate row above.
View File
Whitespace-only changes.
+15 -78
View File
@@ -1,83 +1,20 @@
# Harbor Job Configuration for PraisonAI on Terminal-Bench 2.0
#
# Usage:
# harbor run -c examples/terminal_bench/job.yaml
# Harbor Job Configuration — PraisonAI external (SDK-level) agent on Terminal-Bench 2.1
#
# This configuration runs PraisonAI external agent on Terminal-Bench 2.0 tasks
# with 8 concurrent trials using GPT-4o.
# Dataset: Terminal-Bench 2.0 (89 carefully curated terminal tasks)
dataset: terminal-bench/terminal-bench-2
# Agent configuration
agent:
# Use import path for external agent
import_path: examples.terminal_bench.praisonai_external_agent:PraisonAIExternalAgent
# Model configuration
model_name: openai/gpt-4o
# Environment variables (API keys)
env:
OPENAI_API_KEY: "${OPENAI_API_KEY}"
# Optional: Add other API keys if using different models
# ANTHROPIC_API_KEY: "${ANTHROPIC_API_KEY}"
# GEMINI_API_KEY: "${GEMINI_API_KEY}"
# Execution configuration
n_concurrent: 8 # Run 8 trials in parallel
n_attempts: 1 # Single attempt per task (benchmark standard)
# Optional: Environment configuration for cloud execution
# environment:
# provider: daytona # Options: local, daytona, e2b, modal, runloop, gke
# n_concurrent: 32 # Higher concurrency on cloud
# Optional: Filter to specific tasks for testing
# task_filter:
# task_names: ["compile_simple_c", "install_python_package", "create_directory"]
# Optional: Advanced configuration
# timeout_sec: 600 # 10 minute timeout per task
# save_logs: true # Save execution logs
# save_artifacts: true # Save container artifacts
---
# Alternative configuration using wrapper agent (CLI-based approach)
# Uses `praisonai "TASK"` pattern instead of direct Agent class
# Uncomment to use the wrapper approach:
# dataset: terminal-bench/terminal-bench-2
# Run from the repo root with PYTHONPATH set:
# PYTHONPATH=. harbor run -c examples/terminal_bench/job.yaml
#
# agent:
# import_path: examples.terminal_bench.praisonai_wrapper_agent:PraisonAIWrapperAgent
# model_name: openai/gpt-4o
#
# n_concurrent: 8
# n_attempts: 1
#
# env:
# OPENAI_API_KEY: "${OPENAI_API_KEY}"
# Uses the REAL Harbor JobConfig schema. A single YAML document only —
# multiple `---` documents make yaml.safe_load raise ComposerError.
---
datasets:
- name: terminal-bench
version: "2.1"
# Alternative configuration using installed agent (requires Harbor integration)
# Uncomment this section once PraisonAI is integrated into Harbor's codebase
agents:
- import_path: examples.terminal_bench.praisonai_external_agent:PraisonAIExternalAgent
model_name: openai/gpt-4o-mini
env:
OPENAI_API_KEY: "${OPENAI_API_KEY}"
# dataset: terminal-bench/terminal-bench-2
#
# agent:
# name: praisonai
# model_name: openai/gpt-4o
# flags:
# max_turns: 30
# verbose: false
# memory: false
# auto_approval: true
#
# n_concurrent: 8
# n_attempts: 1
#
# env:
# OPENAI_API_KEY: "${OPENAI_API_KEY}"
n_concurrent_trials: 4
n_attempts: 1
@@ -0,0 +1,28 @@
# Harbor Job Configuration — `praisonai code` smoke test on Terminal-Bench 2.1
#
# Run from the repo root with PYTHONPATH set so the adapter import path resolves:
# PYTHONPATH=. harbor run -c examples/terminal_bench/job_code_smoke.yaml
#
# Schema note: this uses the REAL Harbor JobConfig schema
# (datasets/agents/n_concurrent_trials/n_attempts). Unknown keys are silently
# ignored by pydantic, so the older dataset:/agent:/n_concurrent: form runs the
# default oracle agent on zero tasks — do not use it.
datasets:
- name: terminal-bench
version: "2.1"
# A small verified subset for a cheap smoke run. Verify against the registry
# with `harbor datasets list` before editing.
task_names:
- hello-world
- fix-permissions
- csv-to-parquet
agents:
- import_path: examples.terminal_bench.praisonai_code_agent:PraisonAICodeAgent
model_name: openai/gpt-4o-mini
env:
OPENAI_API_KEY: "${OPENAI_API_KEY}"
n_concurrent_trials: 2
n_attempts: 1
+20 -18
View File
@@ -1,22 +1,24 @@
# Harbor Job Configuration for PraisonAI - 5 Task Test
# Run with: harbor run -c examples/terminal_bench/job_test_5.yaml -n 1
# Harbor Job Configuration — PraisonAI external agent, small verified subset
#
# Run from the repo root:
# PYTHONPATH=. harbor run -c examples/terminal_bench/job_test_5.yaml
#
# Real Harbor JobConfig schema. Verify task_names against `harbor datasets list`
# before editing.
dataset: terminal-bench/terminal-bench-2
datasets:
- name: terminal-bench
version: "2.1"
task_names:
- hello-world
- fix-permissions
- csv-to-parquet
agent:
import_path: examples.terminal_bench.praisonai_external_agent:PraisonAIExternalAgent
model_name: openai/gpt-4o-mini
env:
OPENAI_API_KEY: "${OPENAI_API_KEY}"
agents:
- import_path: examples.terminal_bench.praisonai_external_agent:PraisonAIExternalAgent
model_name: openai/gpt-4o-mini
env:
OPENAI_API_KEY: "${OPENAI_API_KEY}"
n_concurrent: 1
n_concurrent_trials: 1
n_attempts: 1
# Filter to 5 specific simple tasks
task_filter:
task_names:
- "hello-world"
- "fix-permissions"
- "create-bucket"
- "csv-to-parquet"
- "simple-web-scraper"
+20 -18
View File
@@ -1,22 +1,24 @@
# Harbor Job Configuration for PraisonAI Wrapper - 5 Task Test
# Run with: harbor run -c examples/terminal_bench/job_test_5_wrapper.yaml -n 1
# Harbor Job Configuration — PraisonAI wrapper (CLI) agent, small verified subset
#
# Run from the repo root:
# PYTHONPATH=. harbor run -c examples/terminal_bench/job_test_5_wrapper.yaml
#
# Real Harbor JobConfig schema. Verify task_names against `harbor datasets list`
# before editing.
dataset: terminal-bench/terminal-bench-2
datasets:
- name: terminal-bench
version: "2.1"
task_names:
- hello-world
- fix-permissions
- csv-to-parquet
agent:
import_path: examples.terminal_bench.praisonai_wrapper_agent:PraisonAIWrapperAgent
model_name: openai/gpt-4o-mini
env:
OPENAI_API_KEY: "${OPENAI_API_KEY}"
agents:
- import_path: examples.terminal_bench.praisonai_wrapper_agent:PraisonAIWrapperAgent
model_name: openai/gpt-4o-mini
env:
OPENAI_API_KEY: "${OPENAI_API_KEY}"
n_concurrent: 1
n_concurrent_trials: 1
n_attempts: 1
# Filter to 5 specific simple tasks
task_filter:
task_names:
- "hello-world"
- "fix-permissions"
- "create-bucket"
- "csv-to-parquet"
- "simple-web-scraper"
+11 -11
View File
@@ -13,7 +13,7 @@ The multi-agent approach is useful for Terminal-Bench tasks that benefit from:
Usage:
harbor run -d terminal-bench/terminal-bench-2 \
--agent-import-path examples.terminal_bench.multi_agent_example:MultiAgentPraisonAI \
--model openai/gpt-4o
--model openai/gpt-4o-mini
"""
import asyncio
@@ -79,7 +79,7 @@ class MultiAgentPraisonAI(BaseAgent):
return "Error: Empty command"
try:
result = await environment.exec(command=command, timeout_sec=30)
result = await environment.exec(command=command, timeout_sec=300)
output_parts = []
if result.stdout:
output_parts.append(result.stdout.strip())
@@ -101,7 +101,7 @@ class MultiAgentPraisonAI(BaseAgent):
"Consider potential issues and edge cases. "
"Output your plan as a numbered list of specific actions."
),
llm=self.model_name or "openai/gpt-4o",
llm=self.model_name or "openai/gpt-4o-mini",
)
executor = Agent(
@@ -114,7 +114,7 @@ class MultiAgentPraisonAI(BaseAgent):
"If a step fails, try alternative approaches."
),
tools=[bash_tool],
llm=self.model_name or "openai/gpt-4o",
llm=self.model_name or "openai/gpt-4o-mini",
)
verifier = Agent(
@@ -127,7 +127,7 @@ class MultiAgentPraisonAI(BaseAgent):
"If issues are found, suggest specific fixes."
),
tools=[bash_tool],
llm=self.model_name or "openai/gpt-4o",
llm=self.model_name or "openai/gpt-4o-mini",
)
print(f"🚀 Multi-Agent PraisonAI starting: {instruction[:100]}...")
@@ -194,7 +194,7 @@ class MultiAgentPraisonAI(BaseAgent):
"framework": "praisonai-multi",
"agent_type": "multi-agent-team",
"agents": [agent.name for agent in agents],
"model": self.model_name or "openai/gpt-4o",
"model": self.model_name or "openai/gpt-4o-mini",
"phases": ["planning", "execution", "verification"],
"tools_used": ["bash_tool"],
"result_summary": str(result.get("verification", ""))[:200],
@@ -238,7 +238,7 @@ class AgentTeamPraisonAI(BaseAgent):
try:
# Create bash tool
async def bash_tool(command: str) -> str:
result = await environment.exec(command=command, timeout_sec=30)
result = await environment.exec(command=command, timeout_sec=300)
output_parts = []
if result.stdout:
output_parts.append(result.stdout.strip())
@@ -252,14 +252,14 @@ class AgentTeamPraisonAI(BaseAgent):
planner = Agent(
name="planner",
instructions="Create detailed execution plans for terminal tasks",
llm=self.model_name or "openai/gpt-4o"
llm=self.model_name or "openai/gpt-4o-mini"
)
executor = Agent(
name="executor",
instructions="Execute terminal commands based on plans",
tools=[bash_tool],
llm=self.model_name or "openai/gpt-4o"
llm=self.model_name or "openai/gpt-4o-mini"
)
# Create tasks
@@ -317,12 +317,12 @@ if __name__ == "__main__":
print("# Multi-agent custom workflow")
print("harbor run -d terminal-bench/terminal-bench-2 \\")
print(" --agent-import-path examples.terminal_bench.multi_agent_example:MultiAgentPraisonAI \\")
print(" --model openai/gpt-4o")
print(" --model openai/gpt-4o-mini")
print()
print("# AgentTeam structured workflow")
print("harbor run -d terminal-bench/terminal-bench-2 \\")
print(" --agent-import-path examples.terminal_bench.multi_agent_example:AgentTeamPraisonAI \\")
print(" --model openai/gpt-4o")
print(" --model openai/gpt-4o-mini")
print()
print("Benefits of multi-agent approach:")
print("- Task decomposition and planning")
@@ -0,0 +1,127 @@
"""
PraisonAI Code Agent for Terminal-Bench 2.1 (Harbor)
The headline adapter: benchmarks the terminal-native `praisonai code` assistant
by installing it inside the Harbor container and driving it headlessly.
Usage:
harbor run -d terminal-bench/terminal-bench-2-1 \
--agent "examples.terminal_bench.praisonai_code_agent:PraisonAICodeAgent" \
-m openai/gpt-4o-mini \
--ae OPENAI_API_KEY=$OPENAI_API_KEY \
-n 4
Architecture:
Harbor Container → `praisonai code "TASK" --dangerously-skip-approval` (headless)
Notes:
- `--dangerously-skip-approval` sets PRAISON_APPROVAL_MODE=auto +
PRAISONAI_TOOL_SAFETY=off so the assistant runs fully autonomously in the
container (no approval hang in a non-TTY session).
- `praisonai code` normally exits 0 and Harbor grades by task verification,
so a benchmark miss does not fail the run — but the real exit status is
propagated so genuine install/auth/startup failures still surface.
- The base `praisonai` package is sufficient; heavy `code` extras are not
required (ACP tools degrade gracefully).
Dependencies:
pip install harbor praisonai praisonaiagents
"""
import shlex
try:
from harbor.agents.installed.base import BaseInstalledAgent
from harbor.environments.base import BaseEnvironment
from harbor.models.agent.context import AgentContext
except ImportError as e: # pragma: no cover - only importable when Harbor present
raise ImportError(
f"Harbor framework not installed: {e}\n"
"Install with: pip install harbor"
) from e
class PraisonAICodeAgent(BaseInstalledAgent):
"""Benchmarks the `praisonai code` terminal assistant inside a Harbor container."""
@staticmethod
def name() -> str:
return "praisonai-code"
def get_version_command(self) -> str:
return "praisonai --version"
async def install(self, environment: BaseEnvironment) -> None:
"""Install python + the praisonai CLI inside the container."""
# System packages (best-effort; image may already have them).
await self.exec_as_root(
environment,
command="apt-get update && apt-get install -y python3 python3-pip || true",
env={"DEBIAN_FRONTEND": "noninteractive"},
)
# PEP 668-safe install on bookworm-based images.
version_spec = f"=={self._version}" if getattr(self, "_version", None) else ""
await self.exec_as_agent(
environment,
command=(
f"pip install --break-system-packages praisonai{version_spec} "
"praisonaiagents"
),
)
async def run(
self,
instruction: str,
environment: BaseEnvironment,
context: AgentContext,
) -> None:
"""Drive `praisonai code` headlessly on the instruction."""
model = self.model_name or "openai/gpt-4o-mini"
command = (
f"praisonai code {shlex.quote(instruction)} "
f"--dangerously-skip-approval "
f"--model {shlex.quote(model)} "
"> /tmp/praisonai_code.log 2>&1; "
# Capture the real exit status so install/auth/startup failures still
# surface instead of being silently masked (Harbor otherwise records a
# completed attempt for a crashed agent).
"status=$?; "
# Tee the log into a stable path so populate_context_post_run can read it.
"cp /tmp/praisonai_code.log /tmp/praisonai_code_run.log 2>/dev/null || true; "
# `praisonai code` normally exits 0 (Harbor grades by task
# verification), so a genuine benchmark miss won't fail here — but a
# nonzero status means the assistant itself failed to run.
"exit $status"
)
# `extra_env` is the --ae channel; merge it into the exec env (never host
# os.environ). `env` on BaseInstalledAgent already includes extra_env.
await self.exec_as_agent(
environment,
command=command,
env=self.env,
)
def populate_context_post_run(self, context: AgentContext) -> None:
"""`praisonai code` exposes no machine-readable metrics headlessly.
Record identifying metadata; token/cost accounting is not available from
the CLI's mixed stdout, so we leave those fields unset rather than guess.
"""
context.metadata = {
"framework": "praisonai",
"agent_type": "code-cli",
"agent_name": self.name(),
"model": self.model_name,
"log_path": "/tmp/praisonai_code.log",
}
if __name__ == "__main__":
print("PraisonAI Code Agent for Terminal-Bench 2.1")
print("Usage:")
print(" harbor run -d terminal-bench/terminal-bench-2-1 \\")
print(' --agent "examples.terminal_bench.praisonai_code_agent:PraisonAICodeAgent" \\')
print(" -m openai/gpt-4o-mini --ae OPENAI_API_KEY=$OPENAI_API_KEY -n 4")
@@ -7,7 +7,7 @@ BaseEnvironment interface. The agent uses Harbor's exec() as a bash tool.
Usage:
harbor run -d terminal-bench/terminal-bench-2 \
--agent-import-path examples.terminal_bench.praisonai_external_agent:PraisonAIExternalAgent \
--model openai/gpt-4o \
--model openai/gpt-4o-mini \
--ae OPENAI_API_KEY=$OPENAI_API_KEY
Architecture:
@@ -75,8 +75,9 @@ class PraisonAIExternalAgent(BaseAgent):
"""
# Inject API keys from Harbor's --ae env vars into host os.environ
# so litellm can pick them up (--ae only sets them inside Docker, not the host)
agent_env = getattr(context, 'env', {}) or {}
# so litellm can pick them up. The --ae channel arrives via the
# BaseAgent.extra_env property (AgentContext has no `env` field).
agent_env = getattr(self, 'extra_env', {}) or {}
for key, val in agent_env.items():
if key not in os.environ and val:
os.environ[key] = val
@@ -139,7 +140,7 @@ class PraisonAIExternalAgent(BaseAgent):
" Keep calling bash_tool until the test PASSES or you have exhausted all approaches."
),
tools=[bash_tool],
llm=self.model_name or "openai/gpt-4o",
llm=self.model_name or "openai/gpt-4o-mini",
)
# Execute the agent with outer loop to handle premature stopping
@@ -182,17 +183,14 @@ class PraisonAIExternalAgent(BaseAgent):
Harbor tracks: n_input_tokens, n_output_tokens, cost_usd, metadata
"""
try:
# Extract token usage and cost from agent
# Extract token usage and cost from agent. `cost_summary` is a
# property returning a dict (not callable) on praisonaiagents.Agent.
try:
summary = agent.cost_summary() if callable(getattr(agent, 'cost_summary', None)) else None
summary = getattr(agent, 'cost_summary', None)
if isinstance(summary, dict):
context.n_input_tokens = summary.get('tokens_in')
context.n_output_tokens = summary.get('tokens_out')
context.cost_usd = summary.get('cost')
else:
context.n_input_tokens = getattr(agent, '_total_tokens_in', 0)
context.n_output_tokens = getattr(agent, '_total_tokens_out', 0)
context.cost_usd = getattr(agent, 'total_cost', None)
except Exception:
pass
@@ -216,7 +214,7 @@ if __name__ == "__main__":
print("PraisonAI External Agent for Terminal-Bench 2.0")
print("Usage: harbor run -d terminal-bench/terminal-bench-2 \\")
print(" --agent-import-path examples.terminal_bench.praisonai_external_agent:PraisonAIExternalAgent \\")
print(" --model openai/gpt-4o")
print(" --model openai/gpt-4o-mini")
print()
print("Dependencies:")
print(" pip install harbor praisonaiagents")
@@ -9,7 +9,7 @@ Usage:
# First, this would need to be integrated into Harbor's codebase as:
# src/harbor/agents/installed/praisonai.py
harbor run -d terminal-bench/terminal-bench-2 -a praisonai --model openai/gpt-4o
harbor run -d terminal-bench/terminal-bench-2 -a praisonai --model openai/gpt-4o-mini
Architecture:
Harbor Container → praisonaiagents installed inside → execute_command tool
@@ -25,34 +25,9 @@ import asyncio
from pathlib import Path
from typing import Dict, Optional
try:
# These imports would work if this file was in Harbor's codebase
from harbor.agents.installed.base import BaseInstalledAgent, with_prompt_template, CliFlag
from harbor.environments.base import BaseEnvironment
from harbor.models.agent.context import AgentContext
from harbor.models.trial.paths import EnvironmentPaths
except ImportError:
# Fallback for development/testing outside Harbor
print("Note: This is a reference implementation for Harbor integration")
print("To actually use this, it needs to be integrated into Harbor's codebase")
# Mock classes for development
class BaseInstalledAgent:
pass
def with_prompt_template(func):
return func
class CliFlag:
def __init__(self, name, cli, type, default):
pass
class BaseEnvironment:
async def exec(self, **kwargs):
pass
class AgentContext:
pass
from harbor.agents.installed.base import BaseInstalledAgent, with_prompt_template, CliFlag
from harbor.environments.base import BaseEnvironment
from harbor.models.agent.context import AgentContext
class PraisonAIInstalledAgent(BaseInstalledAgent):
@@ -66,12 +41,10 @@ class PraisonAIInstalledAgent(BaseInstalledAgent):
# ATIF trajectory support - would be implemented once PraisonAI has trajectory format
SUPPORTS_ATIF: bool = False
# Command-line configuration flags
# Command-line configuration flags. The real Harbor CliFlag dataclass keys
# the value by `kwarg` (not `name`).
CLI_FLAGS = [
CliFlag("max_turns", cli="--max-turns", type="int", default=30),
CliFlag("verbose", cli="--verbose", type="bool", default=False),
CliFlag("memory", cli="--memory", type="bool", default=False),
CliFlag("auto_approval", cli="--auto-approval", type="bool", default=True),
CliFlag(kwarg="max_turns", cli="--max-turns", type="int", default=30),
]
@staticmethod
@@ -142,7 +115,7 @@ def main():
sys.exit(1)
instruction = sys.argv[1]
model = sys.argv[2] if len(sys.argv) > 2 else "openai/gpt-4o"
model = sys.argv[2] if len(sys.argv) > 2 else "openai/gpt-4o-mini"
max_turns = int(sys.argv[3]) if len(sys.argv) > 3 else 30
try:
from praisonaiagents import Agent
@@ -176,19 +149,19 @@ def main():
"tools_used": ["execute_command"],
}
# Add token usage if available
if hasattr(agent, '_usage'):
usage = agent._usage
if usage:
metrics.update({
"input_tokens": getattr(usage, 'input_tokens', None),
"output_tokens": getattr(usage, 'output_tokens', None),
})
# Add cost if available
if hasattr(agent, '_cost'):
metrics["cost_usd"] = agent._cost
# Token usage + cost come from the `cost_summary` property (a dict).
try:
summary = getattr(agent, "cost_summary", None)
if isinstance(summary, dict):
metrics["input_tokens"] = summary.get("tokens_in")
metrics["output_tokens"] = summary.get("tokens_out")
metrics["cost_usd"] = summary.get("cost")
except Exception:
pass
# Write metrics to a stable path so the host adapter can read them back.
with open("/tmp/praisonai_metrics.json", "w") as fh:
json.dump(metrics, fh)
print(json.dumps(metrics))
except Exception as e:
@@ -216,7 +189,7 @@ if __name__ == "__main__":
This executes the headless runner script with the instruction,
similar to how other installed agents work in Harbor.
"""
model = self.model_name or "openai/gpt-4o"
model = self.model_name or "openai/gpt-4o-mini"
max_turns = getattr(self, 'max_turns', 30)
# Build command to run PraisonAI
@@ -227,15 +200,18 @@ if __name__ == "__main__":
str(max_turns),
]
command = " ".join(cmd_args)
# Tee the metrics file back so populate_context_post_run can read it via
# the container. Also print it so it lands in Harbor's captured stdout.
command = " ".join(cmd_args) + "; cat /tmp/praisonai_metrics.json 2>/dev/null || true"
try:
# Execute the agent
await self.exec_as_agent(
# Execute the agent and keep the ExecResult stdout for metric parsing.
result = await self.exec_as_agent(
environment,
command=command,
env=self._get_environment_vars(),
env=self.env,
)
self._last_stdout = getattr(result, "stdout", "") or ""
except Exception as e:
# Store error in context for Harbor's reporting
context.metadata = {"execution_error": str(e)}
@@ -270,37 +246,38 @@ if __name__ == "__main__":
JSON output of the headless runner script.
"""
try:
# Parse the last stdout output for JSON metrics
# In Harbor's model, the last execution output should contain our JSON
last_output = getattr(context, '_last_stdout', None)
# Parse the JSON metrics captured from the runner's stdout (teed via
# `cat /tmp/praisonai_metrics.json` in run()). AgentContext has no
# `_last_stdout`; we stash it on the agent instead.
last_output = getattr(self, '_last_stdout', None)
metrics = None
if last_output:
try:
metrics = json.loads(last_output.strip())
# Extract metrics with safe defaults
context.n_input_tokens = metrics.get('input_tokens')
context.n_output_tokens = metrics.get('output_tokens')
context.cost_usd = metrics.get('cost_usd')
# Store additional metadata
context.metadata = {
"framework": "praisonai",
"agent_type": "installed",
"agent_name": metrics.get('agent_name', 'terminal-agent'),
"model": metrics.get('model'),
"tools_used": metrics.get('tools_used', []),
"version": self.get_version() if hasattr(self, 'get_version') else None,
}
except (json.JSONDecodeError, ValueError) as e:
# If JSON parsing fails, store basic metadata
context.metadata = {
"framework": "praisonai",
"agent_type": "installed",
"parse_error": str(e),
"raw_output": str(last_output)[:200] if last_output else None,
}
# The output may contain CLI noise before the JSON line; grab the
# last line that parses as a JSON object.
for line in reversed(last_output.strip().splitlines()):
line = line.strip()
if line.startswith("{"):
try:
metrics = json.loads(line)
break
except (json.JSONDecodeError, ValueError):
continue
if metrics is not None:
# Extract metrics with safe defaults
context.n_input_tokens = metrics.get('input_tokens')
context.n_output_tokens = metrics.get('output_tokens')
context.cost_usd = metrics.get('cost_usd')
# Store additional metadata
context.metadata = {
"framework": "praisonai",
"agent_type": "installed",
"agent_name": metrics.get('agent_name', 'terminal-agent'),
"model": metrics.get('model'),
"tools_used": metrics.get('tools_used', []),
"version": self.get_version() if hasattr(self, 'get_version') else None,
}
else:
# No output to parse
context.metadata = {
@@ -334,7 +311,7 @@ if __name__ == "__main__":
print(" - Add PraisonAI to _AGENTS list and _AGENT_MAP")
print()
print("3. Run with Harbor:")
print(" harbor run -d terminal-bench/terminal-bench-2 -a praisonai --model openai/gpt-4o")
print(" harbor run -d terminal-bench/terminal-bench-2 -a praisonai --model openai/gpt-4o-mini")
print()
print("Dependencies:")
print(" pip install harbor praisonaiagents")
@@ -8,7 +8,7 @@ that matches the standard `praisonai "TASK"` CLI usage pattern.
Usage:
harbor run -d terminal-bench/terminal-bench-2 \
--agent-import-path examples.terminal_bench.praisonai_wrapper_agent:PraisonAIWrapperAgent \
--model openai/gpt-4o \
--model openai/gpt-4o-mini \
--ae OPENAI_API_KEY=$OPENAI_API_KEY \
-n 4
@@ -95,13 +95,16 @@ class PraisonAIWrapperAgent(BaseAgent):
Uses the `praisonai "TASK"` CLI pattern inside the Harbor container.
"""
model = self.model_name or "openai/gpt-4o"
model = self.model_name or "openai/gpt-4o-mini"
# Build the praisonai CLI command
# Format: praisonai "TASK" --model MODEL
# Build the `praisonai code` headless command. --dangerously-skip-approval
# makes the terminal assistant run autonomously (no approval hang in a
# non-TTY container). Bare `praisonai "TASK"` would run agents-generator
# mode with no shell-tool loop, so we use the code assistant instead.
cmd_parts = [
"praisonai",
"praisonai", "code",
shlex.quote(instruction),
"--dangerously-skip-approval",
"--model", shlex.quote(model),
]
@@ -148,7 +151,7 @@ class PraisonAIWrapperAgent(BaseAgent):
# Store result summary
context.metadata = {
"agent_name": "praisonai-wrapper",
"model": self.model_name or "openai/gpt-4o",
"model": self.model_name or "openai/gpt-4o-mini",
"framework": "praisonai",
"wrapper_type": "cli",
"instruction_preview": instruction[:200],
@@ -204,16 +207,16 @@ class PraisonAIWrapperAgent(BaseAgent):
return env_vars
async def _exec_as_root(self, environment: BaseEnvironment, command: str, env: Optional[Dict] = None) -> Any:
"""Execute command as root in the container."""
if hasattr(environment, 'exec_as_root'):
return await environment.exec_as_root(command=command, env=env or {})
else:
# Fallback: use sudo
return await environment.exec(command=f"sudo {command}", env=env or {})
"""Execute command as root in the container.
async def _exec_as_agent(self, environment: BaseEnvironment, command: str) -> Any:
BaseEnvironment has no `exec_as_root`; use `exec(..., user="root")`.
`sudo` is typically absent from TB task containers.
"""
return await environment.exec(command=command, env=env or {}, user="root")
async def _exec_as_agent(self, environment: BaseEnvironment, command: str, env: Optional[Dict] = None) -> Any:
"""Execute command as the agent user in the container."""
return await environment.exec(command=command)
return await environment.exec(command=command, env=env or {})
# Example usage for testing
@@ -227,7 +230,7 @@ if __name__ == "__main__":
print("Usage with Harbor:")
print(" harbor run -d terminal-bench/terminal-bench-2 \\")
print(" --agent-import-path examples.terminal_bench.praisonai_wrapper_agent:PraisonAIWrapperAgent \\")
print(" --model openai/gpt-4o")
print(" --model openai/gpt-4o-mini")
print()
print("Dependencies:")
print(" pip install harbor praisonai")
+93 -248
View File
@@ -1,274 +1,119 @@
"""
Tests for PraisonAI Terminal-Bench 2.0 Integration
Tests for the PraisonAI Terminal-Bench 2.1 (Harbor) integration.
This module contains both unit tests and integration tests for the
PraisonAI Harbor integration.
Run from the repo root with PYTHONPATH set so the package import path resolves:
PYTHONPATH=. python -m pytest examples/terminal_bench/test_integration.py -v
Run tests with:
python -m pytest examples/terminal_bench/test_integration.py -v
Requirements:
pip install pytest praisonaiagents
# Harbor is optional for unit tests, required for integration tests
Tests that require Harbor are honestly skipped only when Harbor is not installed
(checked via importlib.util.find_spec). pytest-asyncio must be installed for the
async tests.
"""
import importlib.util
import shlex
from types import SimpleNamespace
from unittest.mock import AsyncMock, Mock
import pytest
import asyncio
from unittest.mock import Mock, AsyncMock, patch
from pathlib import Path
HARBOR_AVAILABLE = importlib.util.find_spec("harbor") is not None
pytestmark = pytest.mark.skipif(
not HARBOR_AVAILABLE, reason="Harbor not installed"
)
class TestPraisonAICodeAgent:
"""The headline `praisonai code` adapter."""
def test_metadata(self):
from examples.terminal_bench.praisonai_code_agent import PraisonAICodeAgent
agent = PraisonAICodeAgent(model_name="openai/gpt-4o-mini", logs_dir="/tmp")
assert agent.name() == "praisonai-code"
assert agent.get_version_command() == "praisonai --version"
@pytest.mark.asyncio
async def test_code_agent_command_shape(self, tmp_path):
"""run() must issue the exact headless `praisonai code` command."""
from examples.terminal_bench.praisonai_code_agent import PraisonAICodeAgent
agent = PraisonAICodeAgent(
model_name="openai/gpt-4o-mini", logs_dir=str(tmp_path)
)
env = Mock()
env.exec = AsyncMock(
return_value=SimpleNamespace(stdout="", stderr="", return_code=0)
)
context = SimpleNamespace(metadata=None)
instruction = "fix the failing test"
await agent.run(instruction, env, context)
env.exec.assert_awaited()
command = env.exec.await_args.kwargs.get("command", "")
assert command.startswith(f"praisonai code {shlex.quote(instruction)}")
assert "--dangerously-skip-approval" in command
assert "--model openai/gpt-4o-mini" in command
class TestPraisonAIExternalAgent:
"""Test the external agent implementation."""
def test_agent_metadata(self):
"""Test agent name and version reporting."""
try:
from .praisonai_external_agent import PraisonAIExternalAgent
except ImportError:
pytest.skip("Harbor not installed - skipping Harbor-specific tests")
"""The external (SDK-level) adapter."""
def test_metadata(self):
from examples.terminal_bench.praisonai_external_agent import (
PraisonAIExternalAgent,
)
agent = PraisonAIExternalAgent()
assert agent.name() == "praisonai"
# Version should be None if praisonaiagents not installed, or actual version
version = agent.version()
assert version is None or isinstance(version, str)
@pytest.mark.asyncio
async def test_setup(self):
"""Test agent setup phase."""
try:
from .praisonai_external_agent import PraisonAIExternalAgent
except ImportError:
pytest.skip("Harbor not installed")
agent = PraisonAIExternalAgent()
mock_env = Mock()
# Setup should complete without error (external agent needs no setup)
await agent.setup(mock_env)
def test_context_population_uses_cost_summary(self):
from examples.terminal_bench.praisonai_external_agent import (
PraisonAIExternalAgent,
)
@pytest.mark.asyncio
async def test_bash_tool_execution(self):
"""Test the bash tool wrapper around Harbor's exec()."""
try:
from .praisonai_external_agent import PraisonAIExternalAgent
except ImportError:
pytest.skip("Harbor not installed")
# Mock Harbor environment
mock_env = Mock()
mock_result = Mock()
mock_result.stdout = "Hello, World!"
mock_result.stderr = ""
mock_result.return_code = 0
mock_env.exec = AsyncMock(return_value=mock_result)
# Mock agent context
mock_context = Mock()
mock_context.metadata = {}
agent = PraisonAIExternalAgent()
# Mock PraisonAI Agent to avoid LLM calls in tests
with patch('praisonaiagents.Agent') as mock_agent_class:
mock_agent_instance = Mock()
mock_agent_instance.start.return_value = "Task completed successfully"
mock_agent_instance.name = "terminal-agent"
mock_agent_class.return_value = mock_agent_instance
# Mock approval backend
with patch('praisonaiagents.approval.set_approval_backend'):
await agent.run("echo 'Hello, World!'", mock_env, mock_context)
# Verify agent was created with correct parameters
mock_agent_class.assert_called_once()
args, kwargs = mock_agent_class.call_args
assert kwargs['name'] == 'terminal-agent'
assert 'tools' in kwargs
assert len(kwargs['tools']) == 1 # bash_tool
assert kwargs['verbose'] is False
assert kwargs['memory'] is False
def test_context_population(self):
"""Test that agent context is properly populated."""
try:
from .praisonai_external_agent import PraisonAIExternalAgent
except ImportError:
pytest.skip("Harbor not installed")
agent_impl = PraisonAIExternalAgent()
# Mock agent with usage data
# cost_summary is a dict property on the real Agent, not a callable.
mock_agent = Mock()
mock_agent.name = "test-agent"
mock_agent.llm = "gpt-4o"
mock_agent._usage = Mock()
mock_agent._usage.input_tokens = 100
mock_agent._usage.output_tokens = 50
mock_agent._cost = 0.01
mock_context = Mock()
mock_context.metadata = {}
# Test context population
agent_impl._populate_context(mock_agent, mock_context, "Test result")
assert mock_context.n_input_tokens == 100
assert mock_context.n_output_tokens == 50
assert mock_context.cost_usd == 0.01
assert mock_context.metadata['agent_name'] == 'test-agent'
assert mock_context.metadata['model'] == 'gpt-4o'
assert mock_context.metadata['framework'] == 'praisonai'
mock_agent.llm = "gpt-4o-mini"
mock_agent.cost_summary = {"tokens_in": 100, "tokens_out": 50, "cost": 0.01}
context = SimpleNamespace(
n_input_tokens=None, n_output_tokens=None, cost_usd=None, metadata=None
)
agent_impl._populate_context(mock_agent, context, "done")
assert context.n_input_tokens == 100
assert context.n_output_tokens == 50
assert context.cost_usd == 0.01
assert context.metadata["framework"] == "praisonai"
class TestPraisonAIInstalledAgent:
"""Test the installed agent implementation."""
def test_agent_configuration(self):
"""Test agent CLI flags and configuration."""
try:
from .praisonai_installed_agent import PraisonAIInstalledAgent
except ImportError:
pytest.skip("Harbor not installed")
agent = PraisonAIInstalledAgent()
assert agent.name() == "praisonai"
assert agent.SUPPORTS_ATIF is False # Until trajectory format is implemented
# Check CLI flags
flag_names = [flag.name if hasattr(flag, 'name') else str(flag) for flag in agent.CLI_FLAGS]
expected_flags = ['max_turns', 'verbose', 'memory', 'auto_approval']
for expected in expected_flags:
assert any(expected in flag_name for flag_name in flag_names)
"""The installed adapter."""
def test_version_command(self):
"""Test version detection command."""
try:
from .praisonai_installed_agent import PraisonAIInstalledAgent
except ImportError:
pytest.skip("Harbor not installed")
agent = PraisonAIInstalledAgent()
version_cmd = agent.get_version_command()
assert version_cmd is not None
assert "praisonaiagents" in version_cmd
assert "__version__" in version_cmd
def test_runner_script_generation(self):
"""Test that the headless runner script is properly generated."""
try:
from .praisonai_installed_agent import PraisonAIInstalledAgent
except ImportError:
pytest.skip("Harbor not installed")
agent = PraisonAIInstalledAgent()
script = agent._build_runner_script()
assert "import praisonaiagents" in script
assert "Agent" in script
assert "execute_command" in script
assert "AutoApproveBackend" in script
assert "json.dumps" in script
class TestIntegration:
"""Integration tests that require both PraisonAI and Harbor."""
@pytest.mark.integration
def test_praisonai_agent_real(self):
"""
Real agentic test - agent must call LLM end-to-end.
This is a MANDATORY test per AGENTS.md §9.4.
Agent MUST call agent.start() with real prompt and produce LLM output.
"""
try:
from praisonaiagents import Agent
except ImportError:
pytest.skip("PraisonAI not installed")
# Create real agent that will call LLM
agent = Agent(
name="test-terminal-agent",
instructions="You are a helpful terminal assistant"
def test_metadata_and_flags(self):
from examples.terminal_bench.praisonai_installed_agent import (
PraisonAIInstalledAgent,
)
# Real agentic test - agent must call LLM and produce text response
result = agent.start("Say hello in one sentence and mention you can help with terminal tasks")
# Verify we got actual LLM output
assert result is not None
assert isinstance(result, str)
assert len(result) > 0
# Print output for manual verification
print("✅ Real agentic test result:")
print(result)
@pytest.mark.integration
@pytest.mark.asyncio
async def test_bash_tool_real_execution(self):
"""Test bash tool with real command execution (if safe)."""
# This would test real bash execution in a safe environment
# For now, we'll mock it to avoid system changes
mock_result = Mock()
mock_result.stdout = "PraisonAI Terminal Test\n"
mock_result.stderr = ""
mock_result.return_code = 0
# Test that our bash tool wrapper works correctly
async def mock_exec(command, timeout_sec=30):
assert "echo" in command # Ensure we're testing echo command
return mock_result
# This simulates Harbor's BaseEnvironment.exec()
result = await mock_exec("echo 'PraisonAI Terminal Test'")
assert result.stdout.strip() == "PraisonAI Terminal Test"
assert result.return_code == 0
agent = PraisonAIInstalledAgent(logs_dir="/tmp")
assert agent.name() == "praisonai"
# CliFlag is keyed by `kwarg` in the real Harbor dataclass.
assert all(getattr(f, "kwarg", None) for f in agent.CLI_FLAGS)
@pytest.mark.integration
def test_auto_approval_setup(self):
"""Test that auto-approval backend works correctly."""
try:
from praisonaiagents.approval import set_approval_backend, AutoApproveBackend
except ImportError:
pytest.skip("PraisonAI approval system not available")
# Test setting and restoring approval backend
original = set_approval_backend(AutoApproveBackend())
new_backend = set_approval_backend(original)
assert isinstance(new_backend, AutoApproveBackend)
def test_runner_script_uses_cost_summary(self):
from examples.terminal_bench.praisonai_installed_agent import (
PraisonAIInstalledAgent,
)
if __name__ == "__main__":
# Allow running tests directly
import sys
print("PraisonAI Terminal-Bench 2.0 Integration Tests")
print("=" * 50)
# Check dependencies
try:
import praisonaiagents
print(f"✅ PraisonAI version: {getattr(praisonaiagents, '__version__', None)}")
except ImportError:
print("❌ PraisonAI not installed: pip install praisonaiagents")
sys.exit(1)
try:
import harbor
print("✅ Harbor framework available")
except ImportError:
print("⚠️ Harbor not installed: pip install harbor")
print(" (Some tests will be skipped)")
print()
print("Run tests with: python -m pytest examples/terminal_bench/test_integration.py -v")
print("Run real agentic test: python -m pytest examples/terminal_bench/test_integration.py::TestIntegration::test_praisonai_agent_real -v -s")
agent = PraisonAIInstalledAgent(logs_dir="/tmp")
script = agent._build_runner_script()
assert "cost_summary" in script
assert "_usage" not in script
assert "execute_command" in script
+1 -1
View File
@@ -13,7 +13,7 @@ Available presets:
- json: JSONL output for piping
"""
from praisonaiagents import Agent, Workflow
from praisonaiagents import Agent, AgentFlow, Workflow
def main():
+17 -17
View File
@@ -21,20 +21,20 @@ This report compares **Python SDK feature categories** against **Python document
| ✅ Agent | 21 | 45 | 14056 |
| ✅ Agent-to-Agent (A2A) | 1 | 7 | 2965 |
| ✅ Agent-to-User (A2U) | 1 | 3 | 585 |
| ✅ Approval | 1 | 5 | 2262 |
| ✅ Approval | 1 | 5 | 2286 |
| ✅ Audio | 2 | 12 | 1000 |
| ✅ Auto Generation | 5 | 10 | 2868 |
| ✅ Autonomy | 3 | 7 | 2705 |
| ✅ Bots | 7 | 28 | 11056 |
| ✅ Autonomy | 3 | 7 | 2707 |
| ✅ Bots | 7 | 28 | 11076 |
| ✅ Budget | 1 | 1 | 287 |
| ✅ CLI | 5 | 116 | 31950 |
| ✅ CLI | 5 | 118 | 32713 |
| ✅ Chunking | 2 | 2 | 385 |
| ✅ Citations | 2 | 1 | 202 |
| ✅ Code Execution | 2 | 13 | 3907 |
| ✅ Conditions | 1 | 3 | 1179 |
| ✅ Configuration | 3 | 7 | 2765 |
| ✅ Context Management | 16 | 34 | 13176 |
| ✅ Database | 1 | 40 | 6124 |
| ✅ Configuration | 3 | 7 | 2772 |
| ✅ Context Management | 16 | 34 | 13204 |
| ✅ Database | 1 | 40 | 6184 |
| ✅ Deep Research | 8 | 2 | 587 |
| ✅ Display | 6 | 3 | 741 |
| ✅ Embeddings | 6 | 23 | 2049 |
@@ -44,16 +44,16 @@ This report compares **Python SDK feature categories** against **Python document
| ✅ Failover | 2 | 1 | 400 |
| ✅ Files | 2 | 6 | 1964 |
| ✅ Flow | 1 | 3 | 777 |
| ✅ Gateway | 7 | 51 | 18026 |
| ✅ Gateway | 7 | 52 | 18697 |
| ✅ Guardrails | 4 | 4 | 1673 |
| ✅ Handoffs | 11 | 6 | 2454 |
| ✅ Hooks | 2 | 9 | 3928 |
| ✅ Hooks | 2 | 9 | 3981 |
| ✅ Image | 1 | 11 | 1255 |
| ✅ Knowledge | 4 | 13 | 4160 |
| ✅ LLM | 3 | 14 | 4767 |
| ✅ Loops | 4 | 4 | 1202 |
| ✅ MCP | 1 | 58 | 13241 |
| ✅ Memory | 6 | 17 | 6481 |
| ✅ MCP | 1 | 58 | 13322 |
| ✅ Memory | 6 | 17 | 6635 |
| ✅ OCR | 2 | 1 | 237 |
| ✅ Observability | 2 | 23 | 2779 |
| ✅ Optimizer | 1 | 2 | 792 |
@@ -62,27 +62,27 @@ This report compares **Python SDK feature categories** against **Python document
| ✅ Planning | 6 | 6 | 1545 |
| ✅ Plugins | 8 | 3 | 1879 |
| ✅ Prompts | 2 | 8 | 1726 |
| ✅ Providers | 1 | 55 | 7455 |
| ✅ Providers | 1 | 55 | 7459 |
| ✅ Query | 1 | 2 | 736 |
| ✅ RAG | 5 | 15 | 2936 |
| ✅ Realtime | 2 | 5 | 668 |
| ✅ Reflection | 3 | 3 | 680 |
| ✅ Retrieval | 2 | 5 | 1044 |
| ✅ Routing | 1 | 2 | 430 |
| ✅ Sandbox | 5 | 6 | 2686 |
| ✅ Sandbox | 5 | 6 | 2700 |
| ✅ Security | 1 | 3 | 2903 |
| ✅ Sessions | 4 | 13 | 5198 |
| ✅ Skills | 6 | 13 | 4893 |
| ✅ Sessions | 4 | 13 | 5286 |
| ✅ Skills | 6 | 13 | 4946 |
| ✅ Tasks | 2 | 6 | 2743 |
| ✅ Telemetry | 1 | 2 | 604 |
| ✅ Templates | 1 | 8 | 1657 |
| ✅ Tools | 12 | 131 | 37117 |
| ✅ Tools | 12 | 132 | 37466 |
| ✅ Tracing | 3 | 2 | 139 |
| ✅ Vector Store | 1 | 12 | 1193 |
| ✅ Video | 2 | 6 | 649 |
| ✅ Vision | 2 | 1 | 329 |
| ✅ Web | 3 | 8 | 2055 |
| ✅ Workflows | 5 | 16 | 6522 |
| ✅ Workflows | 5 | 16 | 6527 |
## Documentation Without Features
@@ -515,8 +515,7 @@ _LAZY_IMPORTS = {
'ManagerConfig': ('praisonaiagents.context.manager', 'ManagerConfig'),
'ContextManager': ('praisonaiagents.context.manager', 'ContextManager'),
# db module
'db': ('praisonaiagents.db', 'db'),
# db shortcut (handled by custom_handler to return _LazyDbModule instance)
# Note: 'obs' is handled by custom_handler to return _LazyObsModule instance
# Gateway protocols and config (implementations in praisonai wrapper)
@@ -711,9 +710,15 @@ def _custom_handler(name, cache):
return mod.obs
if name == 'db':
import importlib
import sys
mod = importlib.import_module('.db', 'praisonaiagents')
cache['db'] = mod
return mod
# importlib registers the submodule as praisonaiagents.db in the package
# namespace, so plain attribute lookup would bypass __getattr__ and return
# the raw module on repeat access. Rebind the package global to the callable
# _LazyDbModule proxy so `praisonaiagents.db(...)` keeps working every time.
cache['db'] = mod.db # Return the _LazyDbModule instance, not the module
setattr(sys.modules['praisonaiagents'], 'db', mod.db)
return mod.db
if name == 'toolsets':
import importlib
mod = importlib.import_module('.toolsets', 'praisonaiagents')
@@ -20,10 +20,14 @@ from ._lazy_display import _get_console, _get_live, _get_display_functions
logger = logging.getLogger(__name__)
from typing import List, Optional, Any, Dict, Union, Generator, TYPE_CHECKING
# Shared HTTP launch state (Agent.launch() multi-endpoint registration)
_server_lock = threading.Lock()
_server_started: Dict[int, bool] = {}
_registered_agents: Dict[int, Dict[str, str]] = {}
_shared_apps: Dict[int, Any] = {}
if TYPE_CHECKING:
pass
@@ -1569,6 +1573,11 @@ Write the complete compiled report:"""
except Exception as e:
logging.error(f"Error starting server: {str(e)}", exc_info=True)
print(f"❌ Error starting server: {str(e)}")
finally:
# Clear the started marker so a later launch() on this port
# can retry after a bind/startup failure or server exit.
with _server_lock:
_server_started[port] = False
# Run server in a background thread
server_thread = threading.Thread(target=run_server, daemon=True)
@@ -72,8 +72,29 @@ class ContextAggregator:
self._sources: Dict[str, Tuple[Callable, int]] = {}
def _estimate_tokens(self, text: str) -> int:
"""Estimate token count (4 chars per token heuristic)."""
return len(text) // 4 + 1
"""Estimate token count via the canonical heuristic."""
from .tokens import estimate_tokens_heuristic
return estimate_tokens_heuristic(text)
def _truncate_to_tokens(self, text: str, max_tokens: int) -> str:
"""Return the longest prefix of ``text`` whose estimated tokens fit ``max_tokens``.
Uses the canonical estimator so dense non-ASCII scripts (e.g. CJK) are
truncated correctly rather than assuming a flat 4-chars-per-token ratio.
"""
if max_tokens <= 0 or not text:
return ""
if self._estimate_tokens(text) <= max_tokens:
return text
# Binary search for the longest prefix within budget.
lo, hi = 0, len(text)
while lo < hi:
mid = (lo + hi + 1) // 2
if self._estimate_tokens(text[:mid]) <= max_tokens:
lo = mid
else:
hi = mid - 1
return text[:lo]
def register_source(
self,
@@ -219,9 +240,15 @@ class ContextAggregator:
# Try to fit partial
remaining = max_tokens - total_tokens - separator_tokens
if remaining > 100:
# Truncate context
chars_to_keep = int(remaining * 4 * 0.9)
truncated = context[:chars_to_keep] + "..."
# Reserve budget for the label and ellipsis so the retained
# prefix (canonically estimated) stays within ``remaining``.
ellipsis_tokens = self._estimate_tokens("...")
label_tokens = (
self._estimate_tokens(f"**{name}**:\n")
if self.include_source_labels else 0
)
prefix_budget = remaining - ellipsis_tokens - label_tokens
truncated = self._truncate_to_tokens(context, prefix_budget) + "..."
if self.include_source_labels:
merged_parts.append(f"**{name}**:\n{truncated}")
@@ -131,6 +131,11 @@ from .protocols import (
FrameDecodeError,
ClientFrame,
decode_client_frame,
# Weak / placeholder secret guard (Issue #3259)
KNOWN_WEAK_SECRETS,
WeakGatewaySecretError,
is_weak_secret,
assert_gateway_secret_strong,
)
from .hooks import (
HookAction,
@@ -324,6 +329,11 @@ __all__ = [
"FrameDecodeError",
"ClientFrame",
"decode_client_frame",
# Weak / placeholder secret guard (Issue #3259)
"KNOWN_WEAK_SECRETS",
"WeakGatewaySecretError",
"is_weak_secret",
"assert_gateway_secret_strong",
# Inbound trigger / webhook contract (Issue #2281)
"HookAction",
"HookConfig",
@@ -1998,6 +1998,71 @@ def resolve_auth_mode(bind_host: str, configured: Optional[AuthMode] = None) ->
return "local" if is_loopback(bind_host) else "token"
# ---------------------------------------------------------------------------
# Weak / placeholder secret guard (Issue #3259)
# ---------------------------------------------------------------------------
KNOWN_WEAK_SECRETS: frozenset = frozenset({
"change-me", "changeme", "change-me-now", "changemenow",
"your-token-here", "your_token_here", "your-secret-here",
"secret", "password", "passwd", "test", "token", "admin",
"default", "example", "placeholder", "none", "null", "todo",
# Copy-paste footgun: the literal fix-hint command pasted verbatim.
"$(openssl rand -hex 16)", "$(openssl rand -hex 32)",
})
"""Well-known placeholder/weak shared secrets that must never protect a gateway.
A gateway bound to an external interface and "protected" by one of these
publicly-known values is effectively unauthenticated. See Issue #3259.
"""
class WeakGatewaySecretError(Exception):
"""Raised when a gateway secret matches a known-weak/placeholder value."""
def __init__(self, field: str = "gateway.auth_token"):
self.field = field
super().__init__(
f"Refusing to start: {field} is a known-weak/placeholder value.\n"
f"A publicly-known secret provides no real authentication.\n"
f"Fix: praisonai onboard (30 seconds, 3 prompts)\n"
f"Or: export GATEWAY_AUTH_TOKEN=\"$(openssl rand -hex 16)\" "
f"(run in a shell so the command is expanded, not pasted literally)"
)
def is_weak_secret(value: Optional[str]) -> bool:
"""Return True if ``value`` is empty or a known-weak/placeholder secret.
Comparison is whitespace-stripped and case-insensitive.
Examples:
>>> is_weak_secret("change-me")
True
>>> is_weak_secret("$(openssl rand -hex 16)")
True
>>> is_weak_secret("strong-non-placeholder-token")
False
"""
if not value:
return True
return str(value).strip().lower() in KNOWN_WEAK_SECRETS
def assert_gateway_secret_strong(value: Optional[str], *, field: str = "gateway.auth_token") -> None:
"""Fail closed if ``value`` is a known-weak/placeholder gateway secret.
Args:
value: The resolved secret to validate.
field: Name of the credential (used in the error message).
Raises:
WeakGatewaySecretError: If ``value`` matches a known-weak value.
"""
if is_weak_secret(value):
raise WeakGatewaySecretError(field=field)
# ---------------------------------------------------------------------------
# Auth, Pairing, and Session Binding Protocols (Issue #1588 Gap 3)
# ---------------------------------------------------------------------------
@@ -113,10 +113,38 @@ class OllamaAdapter(DefaultAdapter):
return 1 # Ollama-specific threshold
def format_tool_result_message(self, function_name: str, tool_result: Any, tool_call_id: Optional[str] = None) -> Dict[str, Any]:
# Ollama uses natural language format for tool results
# Ollama uses natural language format for tool results.
# Error results get a distinct, apology-oriented instruction so the model
# explains the failure rather than echoing the raw error.
is_error = False
error_message = None
if isinstance(tool_result, dict) and 'error' in tool_result:
is_error = True
error_message = tool_result.get('error', 'Unknown error')
elif isinstance(tool_result, list) and len(tool_result) > 0:
first_item = tool_result[0]
if isinstance(first_item, dict) and 'error' in first_item:
is_error = True
error_message = first_item.get('error', 'Unknown error')
if is_error:
return {
"role": "user",
"content": f"""The tool "{function_name}" encountered an error:
{error_message}
Please provide a helpful response to the user explaining that the operation could not be completed.
Be apologetic and suggest alternatives if possible. Do NOT repeat the raw error message.
Give a natural, conversational response."""
}
return {
"role": "user",
"content": f"Tool '{function_name}' returned: {tool_result}"
"role": "user",
"content": f"""Tool execution complete.
Function: {function_name}
Result: {tool_result}
Now provide your final answer using this result. Summarize the information naturally for the user."""
}
def handle_empty_response_with_tools(self, state: Dict[str, Any]) -> bool:
+12 -38
View File
@@ -1518,47 +1518,21 @@ Respond with ONLY a valid JSON tool call in this format:
return "\n".join(lines).strip()
def _format_ollama_tool_result_message(self, function_name: str, tool_result: Any) -> Dict[str, str]:
def _format_ollama_tool_result_message(self, function_name: str, tool_result: Any) -> Dict[str, Any]:
"""
Format tool result message for Ollama provider.
Enhanced to instruct model to use the result for final answer.
Handles error results with specific instructions to provide user-friendly responses.
Delegates to the protocol-driven ``OllamaAdapter.format_tool_result_message``
so this provider-specific formatting lives in the adapter layer (single
source of truth). Reuses the already-constructed ``_provider_adapter`` when
it is the Ollama adapter, otherwise instantiates one (these call sites are
guarded by ``_is_ollama_provider()``, so Ollama formatting is always used).
"""
# Check if the result is an error
is_error = False
error_message = None
if isinstance(tool_result, dict) and 'error' in tool_result:
is_error = True
error_message = tool_result.get('error', 'Unknown error')
elif isinstance(tool_result, list) and len(tool_result) > 0:
first_item = tool_result[0]
if isinstance(first_item, dict) and 'error' in first_item:
is_error = True
error_message = first_item.get('error', 'Unknown error')
if is_error:
# For errors, provide clear instructions to give a helpful response
return {
"role": "user",
"content": f"""The tool "{function_name}" encountered an error:
{error_message}
Please provide a helpful response to the user explaining that the operation could not be completed.
Be apologetic and suggest alternatives if possible. Do NOT repeat the raw error message.
Give a natural, conversational response."""
}
else:
# For successful results, format normally
tool_result_str = str(tool_result)
return {
"role": "user",
"content": f"""Tool execution complete.
Function: {function_name}
Result: {tool_result_str}
Now provide your final answer using this result. Summarize the information naturally for the user."""
}
from .adapters import OllamaAdapter
adapter = getattr(self, '_provider_adapter', None)
if not isinstance(adapter, OllamaAdapter):
adapter = OllamaAdapter()
return adapter.format_tool_result_message(function_name, tool_result)
def _try_append_multimodal_tool_result(
self,
@@ -5,6 +5,7 @@ Manages permission rules, persistent approvals, and integrates
with doom loop detection.
"""
import fnmatch
import json
import logging
from praisonaiagents._logging import get_logger
@@ -226,6 +227,92 @@ class PermissionManager:
"delete_file:",
)
# Read-tool prefixes whose ``<prefix>:<path>`` target is gated by the
# secret-file default. A default coding agent can otherwise ``read`` a
# ``.env``/private key and forward its contents to the model provider — a
# silent secret-exfiltration path. Reads of these files default to ``ask``.
_READ_PREFIXES = (
"read:",
"read_file:",
)
# Basename globs that identify a secret file. Matched case-insensitively
# against the *basename* so ``config/.env`` and ``.env`` both gate. Safe
# example/sample files are excluded via ``_SECRET_ALLOW_PATTERNS`` below.
_SECRET_PATTERNS = (
".env",
"*.env",
".env.*",
"*.env.*",
"*.pem",
"*.key",
"id_rsa",
"id_dsa",
"id_ecdsa",
"id_ed25519",
"*.pfx",
"*.p12",
)
# Basename globs that are always safe to read even though they match a
# secret pattern (documentation templates without real secrets).
_SECRET_ALLOW_PATTERNS = (
"*.env.example",
"*.env.sample",
"*.env.template",
".env.example",
".env.sample",
".env.template",
"*.example",
"*.sample",
"*.template",
)
def _is_secret_read_path(self, path: str) -> bool:
"""Return ``True`` if *path* is a secret file that should gate reads.
Matches the file's basename (case-insensitively) against the built-in
secret globs, excluding safe example/sample/template files. Purely
pattern-based (no filesystem access) so it is cheap and side-effect
free.
"""
basename = os.path.basename(path.rstrip("/\\").strip()).lower()
if not basename:
return False
if any(fnmatch.fnmatch(basename, p) for p in self._SECRET_ALLOW_PATTERNS):
return False
return any(fnmatch.fnmatch(basename, p) for p in self._SECRET_PATTERNS)
# Non-secret basenames used to probe whether an allow rule is a broad
# catch-all (matches everything) rather than a secret-specific opt-in.
_NON_SECRET_PROBES = ("main.py", "readme.md", "data.txt")
def _is_specific_secret_allow(self, rule: "PermissionRule") -> bool:
"""Return ``True`` if *rule* is a secret-specific read allow (opt-in).
A rule opts a secret read in only when it targets secrets specifically
(e.g. ``read:*.env``, ``read:*.pem``) — not when it is a broad catch-all
such as ``read:*`` that also matches ordinary files. We test the rule's
glob against a set of clearly non-secret basenames: if it matches any of
them it is treated as broad and does *not* silently authorise secrets.
Regex rules are conservatively treated as specific (author was explicit).
"""
if rule.action != PermissionAction.ALLOW:
return False
if getattr(rule, "is_regex", False):
return True
pattern = rule.pattern
for prefix in self._READ_PREFIXES:
if pattern.startswith(prefix):
pattern = pattern[len(prefix):]
break
glob = os.path.basename(pattern.rstrip("/\\")) or pattern
# A broad glob (``*``) matches ordinary files too → not secret-specific.
for probe in self._NON_SECRET_PROBES:
if fnmatch.fnmatch(probe, glob.lower()):
return False
return True
def check(self, target: str, agent_name: Optional[str] = None) -> PermissionResult:
"""
Check permission for a target.
@@ -265,6 +352,41 @@ class PermissionManager:
if boundary is not None:
return boundary
# Secret-file read gate: reading ``.env``/private keys and forwarding
# them to the model provider is a silent secret-leak path. Such reads
# default to ``ask`` even when no rule matches. An explicit user
# rule/approval (e.g. ``read:*.env=allow``) still overrides, so opting
# in for a trusted workflow remains one rule.
read_prefix = next(
(p for p in self._READ_PREFIXES if target.startswith(p)), None
)
if read_prefix is not None and self._is_secret_read_path(
target[len(read_prefix):]
):
flat = self._check_flat(target, agent)
# A persistent approval or an explicit *deny* always wins (opt-in or
# hardening). An *allow* only opts in when it comes from a rule that
# specifically targets secrets — a broad wildcard (``read:*``) must
# not silently authorise credential files, otherwise the gate is
# trivially defeated by the catch-all rule most agents ship with.
if flat.approved is not None:
return flat
if flat.rule is not None:
if flat.action == PermissionAction.DENY:
return flat
if self._is_specific_secret_allow(flat.rule):
return flat
# Broad allow/ask rule: fall through to the secret ASK default.
return PermissionResult(
action=PermissionAction.ASK,
target=target,
reason=(
"Reading a secret file requires approval "
"(override with a secret-specific 'read' allow rule, "
"e.g. read:*.env, to opt in)"
),
)
return self._check_flat(target, agent)
def _check_file_boundary(
@@ -69,11 +69,14 @@ __all__ = [
# Doctor migration protocol
"DoctorContractProtocol",
"Finding",
"ConfigDiff",
"RepairPlan",
"get_default_registry",
"register_rule",
"get_rules",
"collect_findings",
"apply_fixes",
"plan_fixes",
# Capability types
"RuntimeCapability",
"RuntimeCapabilityMatrix",
@@ -151,6 +154,8 @@ _LAZY_GROUPS = {
'doctor_protocol': {
'DoctorContractProtocol': ('praisonaiagents.runtime.doctor_protocol', 'DoctorContractProtocol'),
'Finding': ('praisonaiagents.runtime.doctor_protocol', 'Finding'),
'ConfigDiff': ('praisonaiagents.runtime.doctor_protocol', 'ConfigDiff'),
'RepairPlan': ('praisonaiagents.runtime.doctor_protocol', 'RepairPlan'),
},
'doctor_registry': {
'get_default_registry': ('praisonaiagents.runtime.doctor_registry', 'get_default_registry'),
@@ -158,6 +163,7 @@ _LAZY_GROUPS = {
'get_rules': ('praisonaiagents.runtime.doctor_registry', 'get_rules'),
'collect_findings': ('praisonaiagents.runtime.doctor_registry', 'collect_findings'),
'apply_fixes': ('praisonaiagents.runtime.doctor_registry', 'apply_fixes'),
'plan_fixes': ('praisonaiagents.runtime.doctor_registry', 'plan_fixes'),
},
'journal': {
'RunJournal': ('praisonaiagents.runtime.journal', 'RunJournal'),
@@ -21,6 +21,63 @@ class Finding:
context: Optional[Dict[str, Any]] = None
@dataclass
class ConfigDiff:
"""A per-rule before/after change produced by a repair."""
rule_id: str
before: Dict[str, Any]
after: Dict[str, Any]
def unified_diff(self) -> str:
"""Render a unified before/after diff for previewing the change."""
import difflib
import json
before_lines = json.dumps(self.before, indent=2, sort_keys=True, default=str).splitlines()
after_lines = json.dumps(self.after, indent=2, sort_keys=True, default=str).splitlines()
diff = difflib.unified_diff(
before_lines,
after_lines,
fromfile=f"{self.rule_id} (before)",
tofile=f"{self.rule_id} (after)",
lineterm="",
)
return "\n".join(diff)
@dataclass
class RepairPlan:
"""
Result of a doctor repair pass, carrying the safety contract surface.
In dry-run mode nothing is written; ``config`` holds the proposed result and
``diffs`` describes each change so a caller can preview before applying.
"""
config: Dict[str, Any]
diffs: List[ConfigDiff]
backup_path: Optional[str] = None
residual_findings: Optional[List[Finding]] = None
refused: Optional[List[Finding]] = None
applied: bool = False
def __post_init__(self) -> None:
if self.residual_findings is None:
self.residual_findings = []
if self.refused is None:
self.refused = []
@property
def has_changes(self) -> bool:
"""True if any rule proposed a change."""
return len(self.diffs) > 0
def render_diffs(self) -> str:
"""Render all per-rule unified diffs joined together."""
return "\n".join(d.unified_diff() for d in self.diffs)
@runtime_checkable
class DoctorContractProtocol(Protocol):
"""
@@ -7,7 +7,12 @@ Manages built-in and plugin-provided migration rules.
import copy
import warnings
from typing import Any, Dict, List, Optional
from .doctor_protocol import DoctorContractProtocol, Finding
from .doctor_protocol import (
DoctorContractProtocol,
Finding,
ConfigDiff,
RepairPlan,
)
from .builtin_rules import CliBackendMigrationRule
@@ -62,18 +67,140 @@ class DoctorRulesRegistry:
return all_findings
def apply_all_fixes(self, config: Dict[str, Any]) -> Dict[str, Any]:
"""Apply fixes from all rules that have findings."""
"""
Apply fixes from all rules that have findings.
Backward-compatible convenience wrapper that returns only the repaired
config dict. For the safety-rail surface (diff preview, backup, refuse,
re-validation) use :meth:`plan_fixes`.
"""
return self.plan_fixes(config).config
def plan_fixes(
self,
config: Dict[str, Any],
*,
dry_run: bool = True,
backup: bool = False,
backup_path: Optional[str] = None,
) -> RepairPlan:
"""
Produce a :class:`RepairPlan` describing what the doctor repair changes.
Safety contract:
- **Diff preview**: every applied rule records a before/after ``ConfigDiff``.
- **Refuse-on-unrecoverable**: a rule that raises is skipped, its blocking
finding(s) recorded in ``refused``, and its partial change discarded.
- **Backup**: when ``dry_run=False`` and ``backup=True`` the original
config is snapshotted to ``backup_path`` before returning.
- **Re-validation**: after applying, residual findings are re-collected so
callers can report "repair left N finding(s)".
Args:
config: The configuration to repair (never mutated).
dry_run: When True (default) nothing is written to disk; the plan's
``config`` holds the proposed result for previewing.
backup: When True (and not dry_run), write a timestamped backup of
the original config before applying.
backup_path: Explicit backup destination. If omitted a timestamped
path is derived.
Returns:
A :class:`RepairPlan`.
"""
result = copy.deepcopy(config)
diffs: List[ConfigDiff] = []
refused: List[Finding] = []
for rule in self.get_rules():
try:
findings = rule.collect_findings(result)
if findings:
result = rule.apply_fix(result)
except Exception as e:
warnings.warn(f"Error applying fix for rule '{rule.rule_id}': {e}")
return result
# Refuse-on-unrecoverable: a rule that cannot even verify the
# config is a blocking condition, not a silent skip. Record it
# so callers see it in ``refused`` instead of an apparently
# clean plan.
warnings.warn(f"Refusing rule '{rule.rule_id}' (collect failed): {e}")
refused.append(Finding(
rule_id=rule.rule_id,
severity="error",
message=f"Refused to collect findings (unrecoverable): {e}",
fix_description=None,
context={"error": str(e)},
))
continue
if not findings:
continue
# Snapshot the prior state, and hand the rule its *own* fresh copy so
# that rules which mutate-and-return their argument (allowed by the
# protocol) don't alias ``before`` and get their change discarded by
# the ``fixed != before`` comparison below.
before = copy.deepcopy(result)
try:
fixed = rule.apply_fix(copy.deepcopy(before))
except Exception as e:
# Refuse-on-unrecoverable: discard this rule's change, preserve
# the prior state, and record the blocking finding(s).
warnings.warn(f"Refusing fix for rule '{rule.rule_id}': {e}")
refused.append(Finding(
rule_id=rule.rule_id,
severity="error",
message=f"Refused to apply fix (unrecoverable): {e}",
fix_description=None,
context={"error": str(e)},
))
continue
if fixed != before:
diffs.append(ConfigDiff(rule_id=rule.rule_id, before=before, after=copy.deepcopy(fixed)))
result = fixed
residual_findings = self.collect_all_findings(result)
# Only report "applied" when a repair actually changed something; a
# no-op run in apply mode must not claim a repair was performed.
applied = (not dry_run) and bool(diffs)
written_backup_path: Optional[str] = None
if not dry_run and backup and diffs:
written_backup_path = self._write_backup(config, backup_path)
return RepairPlan(
config=result,
diffs=diffs,
backup_path=written_backup_path,
residual_findings=residual_findings,
refused=refused,
applied=applied,
)
@staticmethod
def _write_backup(config: Dict[str, Any], backup_path: Optional[str]) -> str:
"""Snapshot the original config to a timestamped backup file.
For an auto-derived path the timestamp includes microseconds and, if a
file still collides (concurrent runs / quick retry), a numeric suffix is
appended so an earlier snapshot is never overwritten.
"""
import json
import os
from datetime import datetime
if backup_path is None:
timestamp = datetime.now().strftime("%Y%m%d_%H%M%S_%f")
candidate = f"doctor-config.bak-{timestamp}.json"
counter = 1
while os.path.exists(candidate):
candidate = f"doctor-config.bak-{timestamp}-{counter}.json"
counter += 1
backup_path = candidate
with open(backup_path, "w", encoding="utf-8") as f:
json.dump(config, f, indent=2, sort_keys=True, default=str)
return backup_path
def _load_builtin_rules(self) -> None:
"""Load built-in migration rules."""
@@ -144,4 +271,17 @@ def collect_findings(config: Dict[str, Any]) -> List[Finding]:
def apply_fixes(config: Dict[str, Any]) -> Dict[str, Any]:
"""Apply fixes from all rules in the default registry."""
return get_default_registry().apply_all_fixes(config)
return get_default_registry().apply_all_fixes(config)
def plan_fixes(
config: Dict[str, Any],
*,
dry_run: bool = True,
backup: bool = False,
backup_path: Optional[str] = None,
) -> RepairPlan:
"""Produce a RepairPlan from all rules in the default registry."""
return get_default_registry().plan_fixes(
config, dry_run=dry_run, backup=backup, backup_path=backup_path
)
@@ -17,6 +17,7 @@ Usage:
Default storage: ~/.praisonai/config.yaml (under ``schedules`` key)
"""
import threading
from typing import TYPE_CHECKING
if TYPE_CHECKING:
@@ -35,6 +36,50 @@ if TYPE_CHECKING:
_module_cache = {}
# ── canonical default store ──────────────────────────────────────────────────
# Every first-class consumer (agent tools, gateway tick, host bridge) resolves
# its store through ``get_default_store()`` so the writer and reader can never
# drift onto different backends (see issue #3264). ``set_default_store()``
# overrides the singleton process-wide.
_default_store = None
_default_store_lock = threading.Lock()
def get_default_store():
"""Return the process-wide canonical schedule store.
Lazily constructs a :class:`ConfigYamlScheduleStore` (persisting to
``~/.praisonai/config.yaml``) on first use, migrating any pre-existing
``jobs.json`` data. All consumers share this single instance so a job
authored on any surface is polled by the gateway ticker.
"""
global _default_store
with _default_store_lock:
if _default_store is None:
from .config_store import ConfigYamlScheduleStore
store = ConfigYamlScheduleStore()
try:
store.migrate_from_json()
except Exception as e:
import logging
logging.getLogger(__name__).debug(
"jobs.json migration skipped: %s", e,
)
_default_store = store
return _default_store
def set_default_store(store):
"""Override the process-wide canonical schedule store.
A deployment that deliberately swaps the backend calls this once at
startup; the override is then honoured by the agent tools, the gateway
tick loop, and the host-integration bridge alike.
"""
global _default_store
with _default_store_lock:
_default_store = store
def __getattr__(name: str):
"""Lazy load scheduler components."""
@@ -131,4 +176,6 @@ __all__ = [
"Suggestion",
"SuggestionStore",
"MAX_PENDING_CAP",
"get_default_store",
"set_default_store",
]
@@ -33,24 +33,36 @@ def set_store(store):
When PraisonAIUI is running, it calls this at startup to redirect
all agent schedule_add/list/remove operations to the unified
``config.yaml`` instead of the default ``jobs.json``.
This also repoints the core canonical default store so the gateway
tick loop and host bridge read/write the same backend (issue #3264).
"""
global _store_instance
with _store_instance_lock:
_store_instance = store
try:
from ..scheduler import set_default_store
set_default_store(store)
except Exception as e:
# Repointing the canonical default is best-effort: the local
# ``_store_instance`` above is already authoritative for the tools.
# Log (don't silently pass) so any reader/writer drift is diagnosable.
logger.warning("Could not repoint canonical schedule store: %s", e)
def _get_store():
"""Return (or create) the global schedule store.
Uses ``ConfigYamlScheduleStore`` (config.yaml) by default.
Automatically migrates any existing ``jobs.json`` data on first use.
Resolves the canonical default store shared by the gateway tick loop
and host bridge (``scheduler.get_default_store()``) so a job authored
by the agent is polled by the ticker. An explicit ``set_store()``
override still takes precedence.
"""
global _store_instance
with _store_instance_lock:
if _store_instance is None:
from ..scheduler.config_store import ConfigYamlScheduleStore
_store_instance = ConfigYamlScheduleStore()
_store_instance.migrate_from_json()
return _store_instance
if _store_instance is not None:
return _store_instance
from ..scheduler import get_default_store
return get_default_store()
# ── tools ────────────────────────────────────────────────────────────────────
@@ -441,6 +441,8 @@ class ContextTraceEmitter:
def session_start(self, metadata: Optional[Dict[str, Any]] = None) -> None:
"""Emit session start event."""
if not self._enabled:
return
self._emit(ContextEvent(
event_type=ContextEventType.SESSION_START,
timestamp=time.time(),
@@ -450,6 +452,8 @@ class ContextTraceEmitter:
def session_end(self, metadata: Optional[Dict[str, Any]] = None) -> None:
"""Emit session end event."""
if not self._enabled:
return
self._emit(ContextEvent(
event_type=ContextEventType.SESSION_END,
timestamp=time.time(),
@@ -765,6 +769,8 @@ class ContextTraceEmitter:
messages: Optional[List[Dict[str, Any]]] = None,
) -> None:
"""Emit context snapshot event with full context state."""
if not self._enabled:
return
self._emit(ContextEvent(
event_type=ContextEventType.CONTEXT_SNAPSHOT,
timestamp=time.time(),
@@ -786,6 +792,8 @@ class ContextTraceEmitter:
context_passed: Optional[Dict[str, Any]] = None,
) -> None:
"""Emit agent handoff event for tracking agent-to-agent flow."""
if not self._enabled:
return
self._emit(ContextEvent(
event_type=ContextEventType.AGENT_HANDOFF,
timestamp=time.time(),
@@ -814,6 +822,8 @@ class ContextTraceEmitter:
content_length: Length of content being stored
metadata: Optional metadata about the storage
"""
if not self._enabled:
return
self._emit(ContextEvent(
event_type=ContextEventType.MEMORY_STORE,
timestamp=time.time(),
@@ -843,6 +853,8 @@ class ContextTraceEmitter:
memory_type: Type of memory searched
top_score: Score of top result (if available)
"""
if not self._enabled:
return
self._emit(ContextEvent(
event_type=ContextEventType.MEMORY_SEARCH,
timestamp=time.time(),
@@ -873,6 +885,8 @@ class ContextTraceEmitter:
sources: List of source documents/files
top_score: Score of top result (if available)
"""
if not self._enabled:
return
self._emit(ContextEvent(
event_type=ContextEventType.KNOWLEDGE_SEARCH,
timestamp=time.time(),
@@ -901,6 +915,8 @@ class ContextTraceEmitter:
chunk_count: Number of chunks created
metadata: Optional metadata about the indexing
"""
if not self._enabled:
return
self._emit(ContextEvent(
event_type=ContextEventType.KNOWLEDGE_ADD,
timestamp=time.time(),
@@ -42,6 +42,11 @@ logger = get_logger(__name__)
# Default maximum parallel workers to prevent rate limiting issues
DEFAULT_MAX_PARALLEL_WORKERS = 3
# Minimum number of parallel branches before an extra LLM summarisation call is
# worth its cost. Below this, the cheaper truncation fallback is used since the
# summarisation request consumes roughly as many tokens as it saves.
MIN_BRANCHES_FOR_LLM_SUMMARY = 3
# Guards lazy creation of each Workflow's per-instance _run_lock so two threads
# entering run()/astart() concurrently on a fresh instance cannot each create
# and acquire a *different* lock object (which would defeat the run guard).
@@ -2621,11 +2626,19 @@ CONCISE SUMMARY:"""
from ..context.tokens import estimate_tokens_heuristic
tokens = estimate_tokens_heuristic(previous_output)
if tokens > 1000:
# Try LLM-based summarization first, fall back to truncation
try:
optimized_previous = self._llm_summarize_for_parallel(previous_output, num_branches, model, verbose)
except Exception:
# Fallback to truncation-based summarization
# LLM summarisation only pays off for larger fan-outs; for small
# branch counts the extra call costs roughly what it saves, so use
# the cheaper truncation fallback instead.
if num_branches >= MIN_BRANCHES_FOR_LLM_SUMMARY:
# Try LLM-based summarization first, fall back to truncation
try:
optimized_previous = self._llm_summarize_for_parallel(previous_output, num_branches, model, verbose)
except Exception:
# Fallback to truncation-based summarization
optimized_previous = self._truncate_context_for_branches(previous_output, num_branches)
elif tokens >= 1500:
# Only truncate above the same threshold the LLM summariser uses;
# below it, previous behaviour passed context through unchanged.
optimized_previous = self._truncate_context_for_branches(previous_output, num_branches)
if verbose and optimized_previous != previous_output:
+1 -1
View File
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
[project]
name = "praisonaiagents"
version = "1.6.158"
version = "1.6.159"
description = "Praison AI agents for completing complex tasks with Self Reflection Agents"
readme = "README.md"
requires-python = ">=3.10"
+75 -40
View File
@@ -1,49 +1,84 @@
#!/usr/bin/env python3
"""
Test script to verify the termination fix works
Integration test verifying that Agent.start() terminates correctly.
This test ensures the agent does not hang during shutdown
(e.g. telemetry cleanup) and works across Windows, Linux,
and macOS.
"""
import sys
import os
import signal
import time
from threading import Timer
import threading
import _thread
from datetime import datetime
# Add the src directory to the path so we can import praisonaiagents
sys.path.insert(0, os.path.join(os.path.dirname(__file__), 'src', 'praisonai-agents'))
import pytest
# Set up timeout mechanism
def timeout_handler(signum, frame):
print("ERROR: Test timed out - program is still hanging!")
sys.exit(1)
# Set up signal handler for timeout
signal.signal(signal.SIGALRM, timeout_handler)
signal.alarm(30) # 30 second timeout
@pytest.mark.integration
def test_agent_termination():
"""
Verify that Agent.start() returns without hanging.
A cross-platform threading.Timer is used instead of
signal.alarm(), since signal.alarm() is only available
on Unix platforms.
"""
# Disable telemetry so we're testing termination only.
os.environ["PRAISONAI_TELEMETRY_DISABLED"] = "true"
# Skip if no OpenAI API key is available.
# (Keeping this simple for now to avoid adding provider
# detection logic to this smoke test.)
if not os.getenv("OPENAI_API_KEY"):
pytest.skip(
"OPENAI_API_KEY is required for this integration test"
)
try:
# Import here to avoid issues with path setup
from praisonaiagents import Agent
print("Testing agent termination fix...")
# Create agent with minimal setup
agent = Agent(instructions="You are a helpful AI assistant")
# Run the same test as in the issue
print("Running agent.start() ...")
response = agent.start("Write a short hello world message")
print(f"Agent completed successfully!")
print(f"Response (truncated): {str(response)[:100]}...")
# If we get here, the fix worked
print("SUCCESS: Program terminated properly without hanging!")
except Exception as e:
print(f"ERROR: Exception occurred: {e}")
import traceback
traceback.print_exc()
sys.exit(1)
finally:
# Cancel the alarm
signal.alarm(0)
print(f"[{datetime.now()}] Starting agent termination test...")
# Cross-platform timeout
def timeout_handler():
print("ERROR: Agent did not terminate within 30 seconds")
_thread.interrupt_main()
timer = threading.Timer(30.0, timeout_handler)
timer.start()
try:
# Use a context manager so resources are always cleaned up.
with Agent(
instructions="You are a helpful AI assistant",
llm="gpt-4o-mini",
) as agent:
print(f"[{datetime.now()}] Agent created successfully")
print(f"[{datetime.now()}] Running agent.start()...")
response = agent.start(
"Hello, just say hi back!"
)
print(f"[{datetime.now()}] Agent completed successfully!")
print(f"Response: {response}")
# The purpose of this test is to verify that execution
# returns without hanging. Some providers may legitimately
# return None if unavailable, so we don't assert on the
# response content here.
print(
f"[{datetime.now()}] SUCCESS: Program terminated properly!"
)
except KeyboardInterrupt:
pytest.fail(
"Agent execution timed out and did not terminate"
)
finally:
timer.cancel()
+36 -32
View File
@@ -2,52 +2,56 @@
"""
Test the telemetry cleanup fix
"""
import sys
import os
import signal
import pytest
from datetime import datetime
# Add signal handler for timeout
def timeout_handler(signum, frame):
print("SUCCESS: Program terminated within timeout - fix is working!")
sys.exit(0)
signal.signal(signal.SIGALRM, timeout_handler)
signal.alarm(15) # 15 second timeout
@pytest.mark.integration
def test_agent_termination():
"""
Test that the agent starts, completes, and terminates properly
when telemetry cleanup is enabled/disabled.
"""
try:
# Set environment variable to disable telemetry (for testing)
os.environ['PRAISONAI_TELEMETRY_DISABLED'] = 'true'
os.environ["PRAISONAI_TELEMETRY_DISABLED"] = "true"
# Skip test if no OpenAI API key is available
# This is an integration test and requires a real LLM connection
if not os.getenv("OPENAI_API_KEY"):
pytest.skip(
"OPENAI_API_KEY is required for this integration test"
)
from praisonaiagents import Agent
print(f"[{datetime.now()}] Starting agent termination test...")
# Create agent with minimal setup
agent = Agent(
instructions="You are a helpful assistant",
llm="gpt-4o-mini"
)
print(f"[{datetime.now()}] Agent created successfully")
# Test the start method (which was hanging)
print(f"[{datetime.now()}] Running agent.start()...")
response = agent.start("Hello, just say hi back!")
print(f"[{datetime.now()}] Agent completed successfully!")
print(f"Response: {response}")
# If we get here, the fix worked
print(f"[{datetime.now()}] SUCCESS: Program should terminate properly now!")
except Exception as e:
print(f"ERROR: Exception occurred: {e}")
import traceback
traceback.print_exc()
sys.exit(1)
finally:
# Cancel the alarm
signal.alarm(0)
print("Test completed - program should exit now.")
response = agent.start(
"Hello, just say hi back!"
)
print(f"[{datetime.now()}] Agent completed successfully!")
print(f"Response: {response}")
# If we get here, the fix worked
# The agent completed and returned a response without hanging
assert response is not None
print(
f"[{datetime.now()}] SUCCESS: Program completed properly!"
)
@@ -0,0 +1,65 @@
"""Tests for ContextAggregator token-budget truncation."""
import asyncio
from praisonaiagents.context.aggregator import ContextAggregator
def _run(coro):
return asyncio.run(coro)
def test_ascii_truncation_within_budget():
aggregator = ContextAggregator(max_tokens=200, include_source_labels=False)
long_text = "word " * 2000
aggregator.register_source("src", lambda q: long_text)
result = _run(aggregator.aggregate("query"))
assert aggregator._estimate_tokens(result.context) <= 200
def test_non_ascii_truncation_within_budget():
# CJK characters are weighted ~1.3 tokens/char by the canonical heuristic,
# so the old `remaining * 4` char estimate massively over-retained.
aggregator = ContextAggregator(max_tokens=200, include_source_labels=False)
cjk_text = "\u4f60\u597d\u4e16\u754c" * 500
aggregator.register_source("src", lambda q: cjk_text)
result = _run(aggregator.aggregate("query"))
assert aggregator._estimate_tokens(result.context) <= 200
def test_non_ascii_truncation_within_budget_with_labels():
aggregator = ContextAggregator(max_tokens=200, include_source_labels=True)
cjk_text = "\u4f60\u597d\u4e16\u754c" * 500
aggregator.register_source("memory", lambda q: cjk_text)
result = _run(aggregator.aggregate("query"))
assert aggregator._estimate_tokens(result.context) <= 200
def test_truncate_to_tokens_helper():
aggregator = ContextAggregator()
cjk_text = "\u4f60\u597d" * 100
truncated = aggregator._truncate_to_tokens(cjk_text, 50)
assert aggregator._estimate_tokens(truncated) <= 50
assert truncated == cjk_text[: len(truncated)]
def test_truncate_to_tokens_returns_full_when_fits():
aggregator = ContextAggregator()
text = "hello world"
assert aggregator._truncate_to_tokens(text, 1000) == text
def test_truncate_to_tokens_empty_and_zero_budget():
aggregator = ContextAggregator()
assert aggregator._truncate_to_tokens("", 100) == ""
assert aggregator._truncate_to_tokens("hello", 0) == ""
@@ -0,0 +1,143 @@
"""
Tests for the built-in secret-file read gate.
Reads of secret files (``.env``, private keys, etc.) default to ``ask`` so a
coding agent cannot silently forward credentials to the model provider, while
safe example/sample/template files stay allowed and explicit user rules
override the default (opt-in or hard deny).
"""
import tempfile
import pytest
from praisonaiagents.permissions import (
PermissionManager,
PermissionRule,
PermissionAction,
)
@pytest.fixture
def manager():
with tempfile.TemporaryDirectory() as tmp:
yield PermissionManager(storage_dir=tmp)
class TestSecretReadDefaults:
@pytest.mark.parametrize(
"path",
[
".env",
"config/.env",
".env.local",
"prod.env",
"server.pem",
"certs/tls.key",
"id_rsa",
"id_ed25519",
"keystore.pfx",
"bundle.p12",
],
)
def test_secret_read_asks(self, manager, path):
result = manager.check(f"read:{path}")
assert result.action == PermissionAction.ASK
assert result.needs_approval
def test_secret_read_file_prefix_asks(self, manager):
result = manager.check("read_file:.env")
assert result.action == PermissionAction.ASK
@pytest.mark.parametrize(
"path",
[
".env.example",
".env.sample",
"config/.env.template",
"settings.example",
"README.md",
"main.py",
"data.txt",
],
)
def test_safe_read_not_gated(self, manager, path):
# Example/template/ordinary files fall through to the normal default,
# which is ASK only because there is no rule — but crucially they are
# NOT gated by the secret reason.
result = manager.check(f"read:{path}")
assert "secret file" not in result.reason
def test_example_read_allowed_with_broad_rule(self, manager):
manager.add_rule(
PermissionRule(pattern="read:*", action=PermissionAction.ALLOW)
)
# A broad allow lets example files through...
assert manager.check("read:.env.example").action == PermissionAction.ALLOW
# ...but the secret gate still upgrades a real .env to ASK because the
# broad glob rule does not explicitly target the secret path.
secret = manager.check("read:.env")
assert secret.action == PermissionAction.ASK
assert "secret file" in secret.reason
class TestUserOverride:
def test_explicit_allow_overrides_default(self, manager):
manager.add_rule(
PermissionRule(pattern="read:*.env", action=PermissionAction.ALLOW)
)
result = manager.check("read:prod.env")
assert result.action == PermissionAction.ALLOW
def test_explicit_deny_hardens_default(self, manager):
manager.add_rule(
PermissionRule(pattern="read:*.env", action=PermissionAction.DENY)
)
result = manager.check("read:prod.env")
assert result.action == PermissionAction.DENY
def test_approval_overrides_default(self, manager):
manager.approve("read:.env", approved=True, scope="always")
result = manager.check("read:.env")
assert result.action == PermissionAction.ALLOW
def test_broad_allow_does_not_opt_in(self, manager):
# A catch-all ``read:*`` allow must NOT silently authorise secrets,
# otherwise the gate is trivially defeated by the default rule most
# agents ship with.
manager.add_rule(
PermissionRule(pattern="read:*", action=PermissionAction.ALLOW)
)
result = manager.check("read:.env")
assert result.action == PermissionAction.ASK
@pytest.mark.parametrize(
"rule_pattern,path",
[
("read:*.env", "prod.env"),
("read:.env", ".env"),
("read:*.pem", "server.pem"),
("read:id_rsa", "id_rsa"),
],
)
def test_specific_allow_opts_in(self, manager, rule_pattern, path):
# A secret-specific allow is a deliberate opt-in and overrides the gate.
manager.add_rule(
PermissionRule(pattern=rule_pattern, action=PermissionAction.ALLOW)
)
assert manager.check(f"read:{path}").action == PermissionAction.ALLOW
class TestNonRegression:
def test_non_secret_read_unchanged(self, manager):
manager.add_rule(
PermissionRule(pattern="read:*", action=PermissionAction.ALLOW)
)
assert manager.check("read:main.py").action == PermissionAction.ALLOW
def test_non_read_prefix_ignored(self, manager):
# A write to .env is governed by other rules, not the read gate.
manager.add_rule(
PermissionRule(pattern="write:*", action=PermissionAction.ALLOW)
)
assert manager.check("write:.env").action == PermissionAction.ALLOW
@@ -4,10 +4,16 @@ Unit tests for runtime doctor migration.
Tests the DoctorContractProtocol and built-in cli_backend migration rule.
"""
import tempfile
import unittest
from typing import Any, Dict
from praisonaiagents.runtime.doctor_protocol import DoctorContractProtocol, Finding
from praisonaiagents.runtime.doctor_protocol import (
DoctorContractProtocol,
Finding,
ConfigDiff,
RepairPlan,
)
from praisonaiagents.runtime.builtin_rules import CliBackendMigrationRule
from praisonaiagents.runtime.doctor_registry import DoctorRulesRegistry
@@ -301,5 +307,222 @@ class TestDoctorRulesRegistry(unittest.TestCase):
self.assertEqual(result["models"]["default"]["runtime"], "claude-code")
class _RaisingRule:
"""A rule whose apply_fix always raises, to exercise refuse-on-unrecoverable."""
@property
def rule_id(self) -> str:
return "raising_rule"
def collect_findings(self, config: Dict[str, Any]):
if "broken" in config:
return [Finding(rule_id=self.rule_id, severity="error", message="broken")]
return []
def apply_fix(self, config: Dict[str, Any]) -> Dict[str, Any]:
raise ValueError("cannot repair this")
class _InPlaceRule:
"""A rule that mutates-and-returns its argument (in-place repair)."""
@property
def rule_id(self) -> str:
return "in_place_rule"
def collect_findings(self, config: Dict[str, Any]):
if config.get("needs_flag") is True:
return [Finding(rule_id=self.rule_id, severity="warning", message="needs flag")]
return []
def apply_fix(self, config: Dict[str, Any]) -> Dict[str, Any]:
config["needs_flag"] = False
return config
class _CollectRaisingRule:
"""A rule whose collect_findings raises, to exercise refuse-on-collect."""
@property
def rule_id(self) -> str:
return "collect_raising_rule"
def collect_findings(self, config: Dict[str, Any]):
raise RuntimeError("cannot verify config")
def apply_fix(self, config: Dict[str, Any]) -> Dict[str, Any]:
return config
class TestRepairPlanSafetyRails(unittest.TestCase):
"""Test the RepairPlan safety contract on the registry."""
def setUp(self):
self.registry = DoctorRulesRegistry()
self.registry.register_rule(CliBackendMigrationRule())
def test_dry_run_does_not_mutate_input(self):
"""Dry-run must never mutate the caller's config."""
config = {"cli_backend": "claude-code"}
original = dict(config)
plan = self.registry.plan_fixes(config, dry_run=True)
self.assertEqual(config, original)
self.assertIsInstance(plan, RepairPlan)
self.assertFalse(plan.applied)
self.assertNotIn("cli_backend", plan.config)
self.assertEqual(plan.config["models"]["default"]["runtime"], "claude-code")
def test_plan_records_diffs(self):
"""Every applied rule records a before/after diff."""
config = {"cli_backend": "claude-code"}
plan = self.registry.plan_fixes(config)
self.assertTrue(plan.has_changes)
self.assertEqual(len(plan.diffs), 1)
diff = plan.diffs[0]
self.assertIsInstance(diff, ConfigDiff)
self.assertEqual(diff.rule_id, "cli_backend_migration")
self.assertIn("cli_backend", diff.before)
self.assertNotIn("cli_backend", diff.after)
def test_unified_diff_renders(self):
"""Diff preview renders a non-empty unified diff."""
config = {"cli_backend": "claude-code"}
plan = self.registry.plan_fixes(config)
rendered = plan.render_diffs()
self.assertIn("cli_backend", rendered)
self.assertIn("runtime", rendered)
def test_no_findings_no_changes(self):
"""Clean config yields an empty plan with no diffs."""
config = {"framework": "praisonai"}
plan = self.registry.plan_fixes(config)
self.assertFalse(plan.has_changes)
self.assertEqual(plan.diffs, [])
self.assertEqual(plan.residual_findings, [])
def test_re_validation_reports_residuals(self):
"""After applying, residual findings are re-collected."""
config = {"cli_backend": "claude-code"}
plan = self.registry.plan_fixes(config)
# cli_backend migration is idempotent -> no residuals left
self.assertEqual(plan.residual_findings, [])
def test_refuse_on_unrecoverable(self):
"""A rule that raises is refused, its change discarded, original preserved."""
registry = DoctorRulesRegistry()
registry.register_rule(_RaisingRule())
config = {"broken": True}
import warnings
with warnings.catch_warnings():
warnings.simplefilter("ignore")
plan = registry.plan_fixes(config)
self.assertFalse(plan.has_changes)
self.assertEqual(len(plan.refused), 1)
self.assertEqual(plan.refused[0].rule_id, "raising_rule")
# Original preserved in the plan result
self.assertEqual(plan.config, {"broken": True})
def test_backup_written_only_when_applied(self):
"""Backup is only written when not dry_run and backup=True."""
import os
import tempfile
config = {"cli_backend": "claude-code"}
with tempfile.TemporaryDirectory() as tmp:
backup_path = os.path.join(tmp, "cfg.bak.json")
# Dry-run: no backup written even if requested
plan = self.registry.plan_fixes(
config, dry_run=True, backup=True, backup_path=backup_path
)
self.assertIsNone(plan.backup_path)
self.assertFalse(os.path.exists(backup_path))
# Apply: backup written
plan = self.registry.plan_fixes(
config, dry_run=False, backup=True, backup_path=backup_path
)
self.assertEqual(plan.backup_path, backup_path)
self.assertTrue(os.path.exists(backup_path))
self.assertTrue(plan.applied)
def test_apply_all_fixes_backward_compatible(self):
"""apply_all_fixes still returns a plain dict."""
config = {"cli_backend": "claude-code"}
result = self.registry.apply_all_fixes(config)
self.assertIsInstance(result, dict)
self.assertNotIn("cli_backend", result)
self.assertEqual(result["models"]["default"]["runtime"], "claude-code")
def test_in_place_rule_change_recorded(self):
"""A rule that mutates-and-returns its arg still records a diff/change."""
registry = DoctorRulesRegistry()
registry.register_rule(_InPlaceRule())
config = {"needs_flag": True}
plan = registry.plan_fixes(config)
self.assertTrue(plan.has_changes)
self.assertEqual(len(plan.diffs), 1)
self.assertEqual(plan.diffs[0].rule_id, "in_place_rule")
self.assertFalse(plan.config["needs_flag"])
# Caller's config must remain untouched (dry-run).
self.assertTrue(config["needs_flag"])
def test_refuse_on_collect_failure(self):
"""A rule whose collect_findings raises is recorded as refused."""
registry = DoctorRulesRegistry()
registry.register_rule(_CollectRaisingRule())
config = {"anything": True}
import warnings
with warnings.catch_warnings():
warnings.simplefilter("ignore")
plan = registry.plan_fixes(config)
self.assertFalse(plan.has_changes)
self.assertEqual(len(plan.refused), 1)
self.assertEqual(plan.refused[0].rule_id, "collect_raising_rule")
self.assertEqual(plan.config, {"anything": True})
def test_no_op_apply_not_marked_applied(self):
"""apply mode with no changes must not report applied=True."""
config = {"framework": "praisonai"}
plan = self.registry.plan_fixes(config, dry_run=False)
self.assertFalse(plan.has_changes)
self.assertFalse(plan.applied)
def test_apply_with_changes_marked_applied(self):
"""apply mode that changes config reports applied=True."""
config = {"cli_backend": "claude-code"}
plan = self.registry.plan_fixes(config, dry_run=False)
self.assertTrue(plan.has_changes)
self.assertTrue(plan.applied)
def test_default_backup_path_no_overwrite(self):
"""Auto-derived backups never overwrite an earlier snapshot."""
import os
import glob as _glob
cwd = os.getcwd()
with tempfile.TemporaryDirectory() as tmp:
os.chdir(tmp)
try:
config = {"cli_backend": "claude-code"}
p1 = self.registry.plan_fixes(config, dry_run=False, backup=True)
p2 = self.registry.plan_fixes(config, dry_run=False, backup=True)
self.assertIsNotNone(p1.backup_path)
self.assertIsNotNone(p2.backup_path)
self.assertNotEqual(p1.backup_path, p2.backup_path)
self.assertEqual(len(_glob.glob("doctor-config.bak-*.json")), 2)
finally:
os.chdir(cwd)
if __name__ == '__main__':
unittest.main()
+1 -1
View File
@@ -1 +1 @@
__version__ = "0.0.39"
__version__ = "0.0.40"
@@ -339,6 +339,68 @@ def gateway_status(
output.print_error(f"Error checking gateway server status: {str(e)}")
def _check_gateway_secret_strength(config_path: str):
"""Inspect the gateway's own auth_token for known-weak/placeholder values.
Returns an actionable error string when the gateway is on an EXTERNAL bind
and its resolved ``auth_token`` is either missing or a known-weak/
placeholder value (caller should fail closed, mirroring startup). On a
loopback bind a warning is printed for a weak token and ``None`` is returned
(consistent with the permissive-loopback posture). Returns ``None`` when the
token is strong, absent on a loopback bind, or the config cannot be read.
"""
import os
import yaml
if not os.path.exists(config_path):
return None
try:
with open(config_path) as fh:
cfg = yaml.safe_load(fh) or {}
except Exception: # pragma: no cover — defensive
return None
gw = cfg.get("gateway", cfg) or {}
raw_token = gw.get("auth_token") or os.environ.get("GATEWAY_AUTH_TOKEN", "")
token = _resolve_env_token(raw_token) if raw_token else ""
from praisonaiagents.gateway.protocols import (
is_weak_secret,
resolve_auth_mode,
WeakGatewaySecretError,
)
# A strong, present token needs no further checks.
if token and not is_weak_secret(token):
return None
bind_host = gw.get("bind_host") or gw.get("host") or "127.0.0.1"
is_local = resolve_auth_mode(str(bind_host)) == "local"
# Absent token: only a concern on an external bind, where startup fails
# closed for a missing required secret. Doctor must agree (#3259).
if not token:
if is_local:
return None
return (
f"Refusing to start: gateway.auth_token is required for external "
f"bind {bind_host} but is missing.\n"
f"Fix: praisonai onboard (30 seconds, 3 prompts)\n"
f'Or: export GATEWAY_AUTH_TOKEN="$(openssl rand -hex 16)"'
)
# Present-but-weak token: warn on loopback, fail closed externally.
if is_local:
print(
f"⚠ gateway.auth_token is a known-weak/placeholder value "
f"(loopback bind {bind_host}). Rotate before exposing externally."
)
return None
return str(WeakGatewaySecretError(field="gateway.auth_token"))
def _resolve_env_token(value):
"""Resolve a credential input to its value for probing.
@@ -614,9 +676,29 @@ def gateway_doctor(
"""
import asyncio
# Inspect the gateway's OWN auth_token for known-weak/placeholder values
# (Issue #3259) — channel probes never cover this. Fail closed on an
# external bind, warn on loopback (consistent with runtime posture).
gateway_secret_error = _check_gateway_secret_strength(config)
channels = _load_channels(config)
if not channels:
print("No channels configured.")
if json_output:
# Always emit a single parseable JSON document, even on the
# weak/missing gateway-secret failure path, so automation can
# distinguish an auth failure from an empty/interrupted run.
import json
payload: dict = {"probes": {}}
if gateway_secret_error:
payload["gateway_auth_token"] = "weak"
print(json.dumps(payload, indent=2))
else:
print("No channels configured.")
if gateway_secret_error:
print(gateway_secret_error)
if gateway_secret_error:
raise typer.Exit(1)
raise typer.Exit(0)
# Credential source availability, computed WITHOUT printing any value
@@ -636,12 +718,16 @@ def gateway_doctor(
payload = {"probes": _probe_results_to_dict(results)}
if availability:
payload["secrets"] = availability
if gateway_secret_error:
payload["gateway_auth_token"] = "weak"
print(json.dumps(payload, indent=2))
else:
_print_secret_availability(availability)
_render_probe_results(results, json_output=False)
if gateway_secret_error:
print(gateway_secret_error)
if not all_ok:
if not all_ok or gateway_secret_error:
raise typer.Exit(1)
@@ -978,7 +978,11 @@ class BotHandler:
if capabilities.browser:
try:
from praisonai_tools import BrowserBaseTool
tools.append(BrowserBaseTool())
browser_tool = BrowserBaseTool()
# Agent/LLM tool schema expects callables with __name__ (#E2E browser bot).
if not hasattr(browser_tool, "__name__"):
browser_tool.__name__ = getattr(browser_tool, "name", "browserbase")
tools.append(browser_tool)
logger.info("Browser tool enabled")
except ImportError:
logger.warning("Browser tool not available. Install praisonai-tools.")
@@ -9,7 +9,14 @@ import os
import logging
from typing import Optional
from praisonaiagents.gateway.protocols import AuthMode, is_loopback, resolve_auth_mode
from praisonaiagents.gateway.protocols import (
AuthMode,
is_loopback,
resolve_auth_mode,
is_weak_secret,
assert_gateway_secret_strong,
WeakGatewaySecretError,
)
from praisonaiagents.gateway.config import GatewayConfig
logger = logging.getLogger(__name__)
@@ -41,6 +48,13 @@ def assert_external_bind_safe(config: GatewayConfig) -> None:
f"Gateway binding to loopback interface {config.bind_host} without auth token. "
f"This is permissive mode - only safe for local development."
)
elif is_weak_secret(auth_token):
# Consistent with the permissive-loopback posture: warn only.
logger.warning(
f"Gateway auth_token is a known-weak/placeholder value on loopback bind "
f"{config.bind_host}. This provides no real authentication - rotate before "
f"exposing the gateway externally."
)
return
# External bind - require auth token
@@ -48,9 +62,15 @@ def assert_external_bind_safe(config: GatewayConfig) -> None:
raise GatewayStartupError(
f"Cannot bind to {config.bind_host} without an auth token.\n"
f"Fix: praisonai onboard (30 seconds, 3 prompts)\n"
f"Or: export GATEWAY_AUTH_TOKEN=$(openssl rand -hex 16)"
f'Or: export GATEWAY_AUTH_TOKEN="$(openssl rand -hex 16)"'
)
# External bind - reject known-weak/placeholder secrets (fail closed).
try:
assert_gateway_secret_strong(auth_token, field="gateway.auth_token")
except WeakGatewaySecretError as exc:
raise GatewayStartupError(str(exc)) from exc
logger.info(f"Gateway binding to external interface {config.bind_host} with authentication")
@@ -4348,7 +4348,7 @@ class WebSocketGateway:
"""Start a background task that polls the scheduler for due jobs.
Creates a ``ScheduledAgentExecutor`` wired to:
- a ``ScheduleRunner`` with a ``FileScheduleStore``
- a ``ScheduleRunner`` with the canonical default store
- this gateway's agent registry for resolution
- ``_deliver_scheduled_result`` for outbound delivery
"""
@@ -4356,7 +4356,7 @@ class WebSocketGateway:
try:
from praisonaiagents.scheduler import (
ScheduleRunner,
FileScheduleStore,
get_default_store,
)
from praisonai_bot.scheduler.executor import ScheduledAgentExecutor
except ImportError as e:
@@ -4365,7 +4365,7 @@ class WebSocketGateway:
)
return
store = FileScheduleStore()
store = get_default_store()
runner = ScheduleRunner(store)
def _resolve_agent(agent_id):
@@ -405,11 +405,28 @@ class ScheduledAgentExecutor:
)
await asyncio.to_thread(self._audit_output, job, job_result)
# Honour the core intentional-silence contract: a run whose whole
# output is an exact silence marker (NO_REPLY / [SILENT] / SILENT) means
# "nothing worth sending — stay quiet". The run is still recorded as
# succeeded (audited above, history below); only delivery is suppressed
# so an unattended monitor does not post the raw control token. Prose
# that merely mentions the token is unaffected (exact-match check).
silent = False
try:
from praisonaiagents.bots.silence import is_intentional_silence_response
silent = is_intentional_silence_response(result_str)
except Exception: # pragma: no cover - core primitive always present
silent = False
# Deliver to channel bot if delivery target exists
delivered = False
delivery_error: Optional[str] = None
delivery = getattr(job, "delivery", None)
if delivery and self._deliver:
if silent:
logger.info(
"Job '%s' chose intentional silence; delivery suppressed", job.id,
)
elif delivery and self._deliver:
try:
coro = self._deliver(delivery, result_str)
if inspect.isawaitable(coro):
+3 -3
View File
@@ -1,9 +1,9 @@
[project]
name = "praisonai-bot"
version = "0.0.39"
version = "0.0.40"
description = "Bots, gateway, and channel CLI for PraisonAI — messaging platforms and WebSocket control plane extracted from the praisonai wrapper."
readme = "README.md"
license = {text = "MIT"}
license = "MIT"
requires-python = ">=3.10,<3.15"
authors = [
{ name = "Mervin Praison" }
@@ -65,7 +65,7 @@ agentmail = "praisonai_bot.bots.agentmail:AgentMailBot"
praisonaiagents = { path = "../praisonai-agents" }
[build-system]
requires = ["setuptools>=64"]
requires = ["setuptools>=77"]
build-backend = "setuptools.build_meta"
[tool.setuptools.packages.find]
@@ -130,11 +130,138 @@ class TestGatewayAuthEnforcement:
with pytest.raises(GatewayStartupError):
assert_external_bind_safe(config)
# With token - should pass
config = GatewayConfig(bind_host=host, auth_token="token")
# With a strong token - should pass
config = GatewayConfig(
bind_host=host, auth_token="strong-non-placeholder-token"
)
assert_external_bind_safe(config)
class TestWeakSecretGuard:
"""Test known-weak/placeholder secret rejection (Issue #3259)."""
def test_is_weak_secret_detects_placeholders(self):
from praisonaiagents.gateway.protocols import is_weak_secret
for weak in (
"change-me", "CHANGE-ME", " changeme ", "your-token-here",
"secret", "password", "test", "token", "admin",
"$(openssl rand -hex 16)", "$(openssl rand -hex 32)",
):
assert is_weak_secret(weak) is True, weak
def test_is_weak_secret_allows_strong(self):
from praisonaiagents.gateway.protocols import is_weak_secret
assert is_weak_secret("strong-non-placeholder-token") is False
assert is_weak_secret("secure-production-token-xyz") is False
def test_is_weak_secret_treats_empty_as_weak(self):
from praisonaiagents.gateway.protocols import is_weak_secret
assert is_weak_secret("") is True
assert is_weak_secret(None) is True
def test_assert_gateway_secret_strong_raises_on_weak(self):
from praisonaiagents.gateway.protocols import (
assert_gateway_secret_strong,
WeakGatewaySecretError,
)
with pytest.raises(WeakGatewaySecretError) as exc_info:
assert_gateway_secret_strong("change-me", field="gateway.auth_token")
assert exc_info.value.field == "gateway.auth_token"
def test_assert_gateway_secret_strong_passes_strong(self):
from praisonaiagents.gateway.protocols import assert_gateway_secret_strong
# Should not raise
assert_gateway_secret_strong(
"strong-non-placeholder-token", field="gateway.auth_token"
)
def test_external_bind_rejects_weak_token(self):
"""External bind with a placeholder token must fail closed."""
config = GatewayConfig(bind_host="0.0.0.0", auth_token="change-me")
with pytest.raises(GatewayStartupError) as exc_info:
assert_external_bind_safe(config)
assert "known-weak" in str(exc_info.value)
assert "gateway.auth_token" in str(exc_info.value)
def test_external_bind_rejects_literal_openssl_hint(self):
"""The copy-paste footgun literal must be rejected on external bind."""
config = GatewayConfig(
bind_host="192.168.1.10", auth_token="$(openssl rand -hex 16)"
)
with pytest.raises(GatewayStartupError):
assert_external_bind_safe(config)
def test_external_bind_accepts_strong_token(self):
config = GatewayConfig(
bind_host="0.0.0.0", auth_token="strong-non-placeholder-token"
)
assert_external_bind_safe(config)
def test_loopback_bind_warns_but_allows_weak_token(self):
"""Loopback bind downgrades weak-secret to a warning (permissive)."""
config = GatewayConfig(bind_host="127.0.0.1", auth_token="change-me")
# Should not raise
assert_external_bind_safe(config)
class TestDoctorGatewaySecretStrength:
"""Test `gateway doctor` agrees with startup on the gateway auth_token (#3259)."""
def _check(self, tmp_path, monkeypatch, cfg_text):
from praisonai_bot.cli.commands.gateway import _check_gateway_secret_strength
monkeypatch.delenv("GATEWAY_AUTH_TOKEN", raising=False)
cfg = tmp_path / "gateway.yaml"
cfg.write_text(cfg_text)
return _check_gateway_secret_strength(str(cfg))
def test_external_absent_token_fails_closed(self, tmp_path, monkeypatch):
"""Doctor rejects a missing token on an external bind, matching startup."""
err = self._check(
tmp_path, monkeypatch,
"gateway:\n bind_host: 0.0.0.0\n",
)
assert err is not None
assert "required" in err and "0.0.0.0" in err
def test_external_weak_token_fails_closed(self, tmp_path, monkeypatch):
err = self._check(
tmp_path, monkeypatch,
'gateway:\n bind_host: 0.0.0.0\n auth_token: "change-me"\n',
)
assert err is not None
assert "known-weak" in err
def test_external_strong_token_passes(self, tmp_path, monkeypatch):
err = self._check(
tmp_path, monkeypatch,
'gateway:\n bind_host: 0.0.0.0\n auth_token: "strong-non-placeholder-token"\n',
)
assert err is None
def test_loopback_absent_token_passes(self, tmp_path, monkeypatch):
err = self._check(
tmp_path, monkeypatch,
"gateway:\n bind_host: 127.0.0.1\n",
)
assert err is None
def test_loopback_weak_token_warns_only(self, tmp_path, monkeypatch, capsys):
err = self._check(
tmp_path, monkeypatch,
'gateway:\n bind_host: 127.0.0.1\n auth_token: "change-me"\n',
)
assert err is None
assert "known-weak" in capsys.readouterr().out
class TestLoopbackAuthBypassDefault:
"""Test loopback auth bypass is permissive-by-default on loopback binds (#2945)."""
@@ -12,6 +12,6 @@ the remaining terminal-agent modules, with PEP 562 shims left behind at the
old ``praisonai.*`` paths for backward compatibility.
"""
__version__ = "0.0.54"
__version__ = "0.0.55"
__all__ = ["__version__"]
@@ -4,6 +4,7 @@ Run command group for PraisonAI CLI.
Provides agent execution commands.
"""
from contextlib import contextmanager
from typing import Any, Dict, Optional, List
import typer
@@ -663,6 +664,140 @@ def _try_attach_runtime(
return True
@contextmanager
def _worktree_isolation(enabled: bool, name: str, *, keep: bool = False):
"""Run the enclosed block inside an isolated git worktree/branch.
Provisions a fresh worktree via the core ``GitWorktreeAdapter`` and chdirs
into it for the duration of the run so the agent edits an isolated branch
instead of the working tree. On exit it reports the branch name and a
summary of changes (tracked *and* untracked), then tears the worktree down.
Degrades to a transparent no-op when isolation is disabled or the directory
is not a git repository (the adapter reports ``available=False``), so callers
can wrap unconditionally.
Per-run isolation: a short random token is appended to ``name`` so two
concurrent or repeated runs of the same target never resolve to the same
worktree/branch and mix each other's uncommitted changes.
Data-safety on teardown: the agent's output may be entirely *untracked* new
files, which ``git diff`` does not see. So before removing the worktree we
snapshot any change (tracked or untracked, via ``git status --porcelain``)
onto the branch with an automatic commit and *retain that branch* — only the
worktree checkout directory is pruned. A run that produced no changes is
torn down completely (worktree + branch). ``--keep`` additionally retains the
worktree checkout itself for in-place review.
"""
if not enabled:
yield None
return
import os
import subprocess
import uuid
from praisonaiagents.workspace import GitWorktreeAdapter
output = get_output_controller()
adapter = GitWorktreeAdapter()
if not adapter.available:
output.print_warning(
"Not a git repository; running without worktree isolation."
)
yield None
return
def _git(*args, cwd):
try:
return subprocess.run(
["git", *args], cwd=cwd, capture_output=True, text=True
)
except (FileNotFoundError, OSError):
return None
def _branch_of(worktree_path: str) -> str:
result = _git("rev-parse", "--abbrev-ref", "HEAD", cwd=worktree_path)
if result is not None and result.returncode == 0:
return result.stdout.strip() or "?"
return "?"
def _has_changes(worktree_path: str) -> bool:
# ``--porcelain`` reports staged, unstaged *and* untracked entries, so a
# run that only creates brand-new files (invisible to ``git diff``) is
# still recognised as having produced output.
result = _git("status", "--porcelain", cwd=worktree_path)
if result is None or result.returncode != 0:
return False
return bool((result.stdout or "").strip())
def _summary(worktree_path: str) -> str:
result = _git("status", "--short", cwd=worktree_path)
if result is None or result.returncode != 0:
return ""
return (result.stdout or "").strip()
# Make the run's worktree/branch unique per invocation so identical targets
# (same prompt/YAML) run concurrently without sharing state.
unique_name = f"{name}-{uuid.uuid4().hex[:8]}"
original_cwd = os.getcwd()
path = adapter.create(unique_name)
branch = _branch_of(path)
if not output.is_json_mode:
output.print_info(f"Isolated run on branch '{branch}' ({path})")
os.chdir(path)
try:
yield path
finally:
os.chdir(original_cwd)
summary = _summary(path)
changed = _has_changes(path)
if not output.is_json_mode:
if summary:
output.print_info(f"Changes on '{branch}':\n{summary}")
else:
output.print_info(f"No changes on '{branch}'.")
if keep:
if not output.is_json_mode:
output.print_info(
f"Worktree kept at {path} (branch '{branch}'). "
"Review/merge then remove with: git worktree remove."
)
return
if changed:
# Never destroy the agent's output. Persist every change (tracked +
# untracked) as a commit on the isolated branch and retain the
# branch; only the worktree checkout directory is pruned.
_git("add", "-A", cwd=path)
committed = _git(
"commit", "--no-verify", "-m", f"praisonai run: {name}", cwd=path
)
if committed is None or committed.returncode != 0:
# Couldn't persist the work (e.g. no git identity configured):
# keep the worktree checkout in place so output is never lost.
if not output.is_json_mode:
output.print_warning(
f"Could not commit isolated changes; worktree kept at "
f"{path} (branch '{branch}') for manual review."
)
return
# ``worktree remove`` deletes the checkout but leaves the branch
# intact for review/merge (unlike ``adapter.remove`` which also
# force-deletes the branch and would lose the committed work).
_git("worktree", "remove", "--force", path, cwd=original_cwd)
if not output.is_json_mode:
output.print_info(
f"Committed changes to branch '{branch}'. "
f"Review/merge with: git merge {branch}"
)
else:
# No output produced: safe to remove worktree and branch entirely.
adapter.remove(unique_name)
@app.callback(invoke_without_command=True)
def run_main(
ctx: typer.Context,
@@ -706,6 +841,9 @@ def run_main(
restore: Optional[str] = typer.Option(None, "--restore", help="Restore the workspace to a checkpoint id (or 'last') and exit"),
# Warm-runtime live session: tag this run so other terminals can `attach`.
attach: Optional[str] = typer.Option(None, "--attach", help="Run on the warm runtime under this session id so other terminals can observe it via `praisonai attach <id>`"),
# Per-run git-worktree isolation: run on a fresh branch/worktree.
worktree: bool = typer.Option(False, "--worktree", help="Run on an isolated git worktree/branch (branch-per-task); no-op when not a git repo"),
keep: bool = typer.Option(False, "--keep", help="With --worktree, keep the worktree/branch after the run for review instead of tearing it down"),
):
"""
Run agents from a file or prompt.
@@ -848,6 +986,24 @@ def run_main(
output.print_error("Cannot use both --continue and --session together")
raise typer.Exit(1)
# Worktree isolation runs the agent in a chdir'd worktree in-process; the
# warm runtime is a separate process whose cwd we can't redirect, so reject
# the combination up front rather than silently ignoring isolation.
if worktree and attach:
output.print_error("--worktree cannot be combined with --attach")
raise typer.Exit(1)
if keep and not worktree:
output.print_error("--keep requires --worktree")
raise typer.Exit(1)
# Scope isolation to the primary `run "<task>"` / `run agents.yaml` surfaces.
# Custom agent/command and profiling flows have their own execution paths;
# keep the feature focused rather than threading a worktree through each.
if worktree and (agent or command or profile or profile_deep):
output.print_error(
"--worktree is only supported for direct prompt and YAML file runs"
)
raise typer.Exit(1)
# --attach tags a warm-runtime run so other terminals can observe it, but
# only the direct-prompt path forwards to the warm runtime. Reject it up
# front on profile/custom-agent/custom-command flows so users never
@@ -1037,58 +1193,72 @@ def run_main(
)
return
if is_file:
# Run from file
_run_from_file(
target,
model=model,
framework=framework,
interactive=interactive,
verbose=verbose,
stream=stream,
trace=trace,
memory=memory,
tools=tools,
max_tokens=max_tokens,
output_mode=output_mode,
continue_session=continue_session,
session=session,
fork=fork,
no_save=no_save,
)
else:
# Run as prompt
permissions_config = _parse_permissions(allow, deny, permissions, permission_default)
mcp_command, mcp_env, permissions_config, mcp_servers = _apply_config_defaults(
None, None, permissions_config
)
_run_prompt(
target,
model=model,
verbose=verbose,
stream=stream,
trace=trace,
memory=memory,
tools=tools,
toolset=toolset,
max_tokens=max_tokens,
output_mode=output_mode,
approval=approval,
approve_all_tools=approve_all_tools,
approval_timeout=approval_timeout,
no_rules=no_rules,
permissions_config=permissions_config,
mcp=mcp_command,
mcp_env=mcp_env,
mcp_servers=mcp_servers,
continue_session=continue_session,
session=session,
fork=fork,
no_save=no_save,
attach_session=attach,
thinking_budget=thinking_budget,
allow_local_tools=allow_local_tools,
)
# Resolve a YAML file target to an absolute path *before* isolation chdirs
# away. The config is loaded from the original checkout so an untracked or
# git-ignored ``./agents.yaml`` (absent from the fresh worktree) still loads;
# the agent then produces its output inside the isolated worktree.
run_target = target
if worktree and is_file:
import os as _os_ff
run_target = _os_ff.path.abspath(target)
# Provision a per-run git worktree/branch when --worktree is set and the cwd
# is a git repo; otherwise this is a transparent no-op. The agent runs with
# its cwd redirected into the isolated worktree for the duration of the run.
with _worktree_isolation(worktree, target, keep=keep):
if is_file:
# Run from file
_run_from_file(
run_target,
model=model,
framework=framework,
interactive=interactive,
verbose=verbose,
stream=stream,
trace=trace,
memory=memory,
tools=tools,
max_tokens=max_tokens,
output_mode=output_mode,
continue_session=continue_session,
session=session,
fork=fork,
no_save=no_save,
)
else:
# Run as prompt
permissions_config = _parse_permissions(allow, deny, permissions, permission_default)
mcp_command, mcp_env, permissions_config, mcp_servers = _apply_config_defaults(
None, None, permissions_config
)
_run_prompt(
target,
model=model,
verbose=verbose,
stream=stream,
trace=trace,
memory=memory,
tools=tools,
toolset=toolset,
max_tokens=max_tokens,
output_mode=output_mode,
approval=approval,
approve_all_tools=approve_all_tools,
approval_timeout=approval_timeout,
no_rules=no_rules,
permissions_config=permissions_config,
mcp=mcp_command,
mcp_env=mcp_env,
mcp_servers=mcp_servers,
continue_session=continue_session,
session=session,
fork=fork,
no_save=no_save,
attach_session=attach,
thinking_budget=thinking_budget,
allow_local_tools=allow_local_tools,
isolated=worktree,
)
def _run_from_file(
@@ -1223,6 +1393,7 @@ def _run_prompt(
attach_session: Optional[str] = None,
thinking_budget: Optional[int] = None,
allow_local_tools: bool = False,
isolated: bool = False,
):
"""Run a direct prompt."""
output = get_output_controller()
@@ -1300,7 +1471,10 @@ def _run_prompt(
# approval/memory), so it stays in-process: the warm runtime reuses a
# cached agent and does not carry a per-call thinking budget, so attaching
# would silently drop the requested setting.
runtime_eligible = no_save and thinking_budget is None and not any([
# Isolated (--worktree) runs must stay in-process: the warm runtime is a
# separate process whose cwd we can't redirect into the worktree, so
# attaching would run the task outside the isolated branch.
runtime_eligible = no_save and thinking_budget is None and not isolated and not any([
mcp, mcp_servers, tools, toolset, approval, approve_all_tools,
memory, permissions_config, continue_session, session, fork,
])
@@ -5,7 +5,8 @@ Provides sandbox management commands inspired by moltbot's sandbox CLI.
Supports listing, explaining, and recreating sandbox containers.
"""
from typing import Literal, Optional
from enum import Enum
from typing import Optional
import typer
@@ -14,7 +15,13 @@ app = typer.Typer(
no_args_is_help=True,
)
_SandboxType = Literal["subprocess", "docker", "daytona"]
class SandboxBackend(str, Enum):
"""Supported sandbox execution backends."""
subprocess = "subprocess"
docker = "docker"
daytona = "daytona"
def _sandbox_handler():
@@ -28,8 +35,8 @@ def _sandbox_handler():
def sandbox_run(
code: Optional[str] = typer.Option(None, "--code", "-c", help="Code to execute"),
file: Optional[str] = typer.Option(None, "--file", "-f", help="File to execute"),
sandbox_type: _SandboxType = typer.Option(
"subprocess", "--type", "-t", help="Sandbox backend (subprocess, docker, or daytona)"
sandbox_type: SandboxBackend = typer.Option(
SandboxBackend.subprocess, "--type", "-t", help="Sandbox backend (subprocess, docker, or daytona)"
),
image: str = typer.Option("python:3.11-slim", "--image", help="Docker image"),
timeout: int = typer.Option(60, "--timeout", help="Timeout in seconds"),
@@ -38,7 +45,7 @@ def sandbox_run(
_sandbox_handler().run(
code=code,
file=file,
sandbox_type=sandbox_type,
sandbox_type=sandbox_type.value,
image=image,
timeout=timeout,
)
@@ -46,13 +53,13 @@ def sandbox_run(
@app.command("shell")
def sandbox_shell(
sandbox_type: _SandboxType = typer.Option(
"subprocess", "--type", "-t", help="Sandbox backend (subprocess, docker, or daytona)"
sandbox_type: SandboxBackend = typer.Option(
SandboxBackend.subprocess, "--type", "-t", help="Sandbox backend (subprocess, docker, or daytona)"
),
image: str = typer.Option("python:3.11-slim", "--image", help="Docker image"),
):
"""Start an interactive sandbox REPL."""
_sandbox_handler().shell(sandbox_type=sandbox_type, image=image)
_sandbox_handler().shell(sandbox_type=sandbox_type.value, image=image)
@app.command("backends")
@@ -0,0 +1,617 @@
"""
Agent-Centric Tools for PraisonAI Interactive Mode.
These tools route file operations and code intelligence through LSP/ACP,
making the Agent the central orchestrator for all actions.
"""
import asyncio
import json
import logging
from typing import Callable, Dict, List, Optional, TYPE_CHECKING
if TYPE_CHECKING:
from .interactive_runtime import InteractiveRuntime
from .code_intelligence import CodeIntelligenceRouter
from .action_orchestrator import ActionOrchestrator
import os
from pathlib import Path
logger = logging.getLogger(__name__)
def _sanitize_filepath(filepath: str, workspace: Optional[str] = None) -> str:
"""Validate and sanitize a filepath against injection attacks.
Raises ValueError if the path is unsafe.
"""
# Reject null bytes (common injection technique)
if '\x00' in filepath:
raise ValueError("Null bytes are not allowed in file paths")
# Reject obvious shell meta-characters in file names
if any(ch in filepath for ch in ('|', ';', '`', '$', '&', '\n', '\r')):
raise ValueError(f"Unsafe characters in filepath: {filepath!r}")
# Reject absolute paths — files must be relative to workspace
if os.path.isabs(filepath):
raise ValueError(f"Absolute paths are not allowed: {filepath!r}")
# Normalize and reject path traversal
normalized = os.path.normpath(filepath)
if '..' in normalized.split(os.sep):
raise ValueError(f"Path traversal detected in: {filepath!r}")
# If we have a workspace, ensure the resolved path stays within it
if workspace:
resolved = Path(os.path.realpath(os.path.join(workspace, normalized)))
ws_resolved = Path(os.path.realpath(workspace))
try:
resolved.relative_to(ws_resolved)
except ValueError as exc:
raise ValueError(
f"Path {filepath!r} resolves outside workspace {workspace!r}"
) from exc
return normalized
def _sanitize_command(command: str) -> str:
"""Basic command sanitization — reject common injection vectors.
The command still goes through ACP approval flow, but this prevents
the most egregious prompt-injection-to-shell-injection chains.
Raises ValueError if dangerous patterns are detected.
"""
if '\x00' in command:
raise ValueError("Null bytes are not allowed in commands")
# Reject command chaining operators that indicate injection
DANGEROUS_PATTERNS = [
'$(', '`', # Command substitution
'&&', '||', # Command chaining
'>>', '>', # Output redirection
'|', ';', '&', # Pipe and separators
'\n', '\r' # Line breaks
]
for pattern in DANGEROUS_PATTERNS:
if pattern in command:
raise ValueError(
f"Potentially unsafe command pattern detected: {pattern!r} "
f"in command: {command!r}. Use separate commands instead."
)
return command
def create_agent_centric_tools(
runtime: "InteractiveRuntime",
router: "Optional[CodeIntelligenceRouter]" = None,
orchestrator: "Optional[ActionOrchestrator]" = None
) -> List[Callable]:
"""
Create tools that route through LSP/ACP for agent-centric architecture.
Args:
runtime: The InteractiveRuntime instance
router: Optional CodeIntelligenceRouter (created if not provided)
orchestrator: Optional ActionOrchestrator (created if not provided)
Returns:
List of tool functions for the Agent
"""
from .code_intelligence import CodeIntelligenceRouter
from .action_orchestrator import ActionOrchestrator
if router is None:
router = CodeIntelligenceRouter(runtime)
if orchestrator is None:
orchestrator = ActionOrchestrator(runtime)
# Helper to run async functions synchronously using the shared bridge.
# Loaded via the wrapper bridge so standalone ``praisonai-code`` installs
# fail with a clear install hint instead of an ImportError, and so the C7
# import-direction gate stays satisfied (no direct wrapper import here).
from praisonai_code._wrapper_bridge import import_wrapper_module
run_sync = import_wrapper_module("praisonai._async_bridge").run_sync
# =========================================================================
# ACP-Powered File Tools
# =========================================================================
def acp_create_file(filepath: str, content: str) -> str:
"""
Create a file through ACP with plan/approve/apply/verify flow.
This tool routes file creation through the ActionOrchestrator,
ensuring proper tracking, approval, and verification.
Args:
filepath: Path to the file to create (relative to workspace)
content: Content to write to the file
Returns:
JSON string with result including plan status and verification
"""
async def _create():
# Validate filepath
try:
safe_path = _sanitize_filepath(
filepath,
workspace=getattr(runtime.config, 'workspace', None)
)
except ValueError as e:
return json.dumps({"success": False, "error": str(e)})
# Build a detailed prompt for the orchestrator
prompt = f"create file {safe_path}"
# Create plan
result = await orchestrator.create_plan(prompt)
if not result.success:
return json.dumps({
"success": False,
"error": result.error,
"read_only": result.read_only_blocked
})
plan = result.plan
# Update the plan step with actual content
if plan.steps:
plan.steps[0].params["content"] = content
# Approve based on runtime config
auto_approve = runtime.config.approval_mode == "auto"
approved = await orchestrator.approve_plan(plan, auto=auto_approve)
if not approved and runtime.config.approval_mode == "manual":
return json.dumps({
"success": False,
"error": "Manual approval required",
"plan": plan.to_dict(),
"requires_approval": True
})
# Apply the plan
result = await orchestrator.apply_plan(plan, force=auto_approve)
if not result.success:
return json.dumps({
"success": False,
"error": result.error,
"plan": plan.to_dict()
})
# Verify
result = await orchestrator.verify_plan(plan)
return json.dumps({
"success": result.success,
"file_created": filepath,
"plan_id": plan.id,
"status": plan.status.value,
"verified": result.success
})
return run_sync(_create())
def acp_edit_file(filepath: str, new_content: str) -> str:
"""
Edit a file through ACP with plan/approve/apply/verify flow.
Args:
filepath: Path to the file to edit (relative to workspace)
new_content: New content for the file
Returns:
JSON string with result including plan status
"""
async def _edit():
try:
safe_path = _sanitize_filepath(
filepath,
workspace=getattr(runtime.config, 'workspace', None)
)
except ValueError as e:
return json.dumps({"success": False, "error": str(e)})
prompt = f"edit file {safe_path}"
result = await orchestrator.create_plan(prompt)
if not result.success:
return json.dumps({
"success": False,
"error": result.error,
"read_only": result.read_only_blocked
})
plan = result.plan
# Update step with new content
if plan.steps:
plan.steps[0].params["new_content"] = new_content
auto_approve = runtime.config.approval_mode == "auto"
approved = await orchestrator.approve_plan(plan, auto=auto_approve)
if not approved and runtime.config.approval_mode == "manual":
return json.dumps({
"success": False,
"error": "Manual approval required",
"requires_approval": True
})
result = await orchestrator.apply_plan(plan, force=auto_approve)
return json.dumps({
"success": result.success,
"file_edited": filepath,
"plan_id": plan.id,
"error": result.error
})
return run_sync(_edit())
def acp_delete_file(filepath: str) -> str:
"""
Delete a file through ACP with plan/approve/apply/verify flow.
Args:
filepath: Path to the file to delete
Returns:
JSON string with result
"""
async def _delete():
try:
safe_path = _sanitize_filepath(
filepath,
workspace=getattr(runtime.config, 'workspace', None)
)
except ValueError as e:
return json.dumps({"success": False, "error": str(e)})
prompt = f"delete file {safe_path}"
result = await orchestrator.create_plan(prompt)
if not result.success:
return json.dumps({
"success": False,
"error": result.error
})
plan = result.plan
# Delete requires explicit approval even in auto mode
auto_approve = runtime.config.approval_mode == "auto"
approved = await orchestrator.approve_plan(plan, auto=auto_approve)
if not approved:
return json.dumps({
"success": False,
"error": "Delete requires approval",
"requires_approval": True
})
apply_result = await orchestrator.apply_plan(plan)
if not apply_result.success:
return json.dumps({
"success": False,
"file_deleted": filepath,
"verified": False,
"error": apply_result.error
})
result = await orchestrator.verify_plan(plan)
return json.dumps({
"success": result.success,
"file_deleted": filepath,
"verified": result.success
})
return run_sync(_delete())
def acp_execute_command(command: str) -> str:
"""
Execute a shell command through ACP with tracking.
Commands run in the configured workspace directory. Per-call working
directories are not currently supported.
Args:
command: The command to execute
Returns:
JSON string with command output
"""
async def _execute():
try:
safe_cmd = _sanitize_command(command)
except ValueError as e:
return json.dumps({"success": False, "error": str(e)})
prompt = f"run command: {safe_cmd}"
result = await orchestrator.create_plan(prompt)
if not result.success:
return json.dumps({
"success": False,
"error": result.error
})
plan = result.plan
# Commands require approval
auto_approve = runtime.config.approval_mode == "auto"
approved = await orchestrator.approve_plan(plan, auto=auto_approve)
if not approved:
return json.dumps({
"success": False,
"error": "Command requires approval",
"requires_approval": True
})
result = await orchestrator.apply_plan(plan)
# Extract command result
if plan.steps and plan.steps[0].result:
cmd_result = plan.steps[0].result
return json.dumps({
"success": result.success,
"command": command,
"stdout": cmd_result.get("stdout", ""),
"stderr": cmd_result.get("stderr", ""),
"returncode": cmd_result.get("returncode", -1)
})
return json.dumps({
"success": result.success,
"error": result.error
})
return run_sync(_execute())
# =========================================================================
# LSP-Powered Code Intelligence Tools
# =========================================================================
def lsp_list_symbols(file_path: str) -> str:
"""
List all symbols (functions, classes, methods) in a file using LSP.
Falls back to regex-based extraction if LSP is unavailable.
Args:
file_path: Path to the file to analyze
Returns:
JSON string with list of symbols and their locations
"""
async def _list():
# Add 10s timeout to prevent LSP from blocking autonomy
try:
result = await asyncio.wait_for(
router.handle_query(
f"list all functions and classes in {file_path}",
file_path=file_path
),
timeout=10.0
)
return json.dumps(result.to_dict())
except asyncio.TimeoutError:
return json.dumps({"error": "LSP list_symbols timed out (10s)", "success": False})
return run_sync(_list())
def lsp_find_definition(symbol: str, file_path: Optional[str] = None) -> str:
"""
Find where a symbol is defined using LSP.
Args:
symbol: The symbol name to find
file_path: Optional file path for context
Returns:
JSON string with definition location(s)
"""
async def _find():
query = f"where is {symbol} defined"
if file_path:
query += f" in {file_path}"
# Add 10s timeout to prevent LSP from blocking autonomy
try:
result = await asyncio.wait_for(
router.handle_query(query, file_path=file_path),
timeout=10.0
)
return json.dumps(result.to_dict())
except asyncio.TimeoutError:
return json.dumps({"error": "LSP find_definition timed out (10s)", "success": False})
return run_sync(_find())
def lsp_find_references(symbol: str, file_path: Optional[str] = None) -> str:
"""
Find all references to a symbol using LSP.
Args:
symbol: The symbol name to find references for
file_path: Optional file path for context
Returns:
JSON string with reference locations
"""
async def _find():
query = f"find all references to {symbol}"
if file_path:
query += f" in {file_path}"
# Add 10s timeout to prevent LSP from blocking autonomy
try:
result = await asyncio.wait_for(
router.handle_query(query, file_path=file_path),
timeout=10.0
)
return json.dumps(result.to_dict())
except asyncio.TimeoutError:
return json.dumps({"error": "LSP find_references timed out (10s)", "success": False})
return run_sync(_find())
def lsp_get_diagnostics(file_path: Optional[str] = None) -> str:
"""
Get diagnostics (errors, warnings) for a file using LSP.
Args:
file_path: Path to the file (optional, gets all if not specified)
Returns:
JSON string with diagnostic information
"""
async def _get():
query = "show all errors and warnings"
if file_path:
query += f" in {file_path}"
# Add 10s timeout to prevent LSP from blocking autonomy
try:
result = await asyncio.wait_for(
router.handle_query(query, file_path=file_path),
timeout=10.0
)
return json.dumps(result.to_dict())
except asyncio.TimeoutError:
return json.dumps({"error": "LSP diagnostics timed out (10s)", "success": False})
return run_sync(_get())
# =========================================================================
# Basic File Tools (for read-only operations)
# =========================================================================
def read_file(filepath: str) -> str:
"""
Read content from a file.
This is a read-only operation that doesn't require ACP.
Args:
filepath: Path to the file to read
Returns:
File content or error message
"""
from pathlib import Path
try:
path = Path(filepath)
if not path.is_absolute():
path = Path(runtime.config.workspace) / filepath
# SECURITY: Ensure resolved path stays within workspace
resolved = path.resolve()
ws_resolved = Path(runtime.config.workspace).resolve()
try:
resolved.relative_to(ws_resolved)
except ValueError:
return json.dumps({"error": f"Path escapes workspace: {filepath}"})
if not resolved.exists():
return json.dumps({"error": f"File not found: {filepath}"})
content = resolved.read_text()
# Track in trace if enabled
if runtime._trace:
runtime._trace.add_entry(
category="file",
action="read",
params={"file": str(path)},
result={"size": len(content)}
)
return content
except Exception as e:
return json.dumps({"error": str(e)})
def list_files(directory: str = ".", pattern: str = "*") -> str:
"""
List files in a directory.
Args:
directory: Directory to list (relative to workspace)
pattern: Glob pattern to filter files
Returns:
JSON string with list of files
"""
from pathlib import Path
try:
path = Path(directory)
if not path.is_absolute():
path = Path(runtime.config.workspace) / directory
# SECURITY: Ensure resolved path stays within workspace
resolved = path.resolve()
ws_resolved = Path(runtime.config.workspace).resolve()
try:
resolved.relative_to(ws_resolved)
except ValueError:
return json.dumps({"error": f"Directory escapes workspace: {directory}"})
if not resolved.exists():
return json.dumps({"error": f"Directory not found: {directory}"})
files = []
for f in path.glob(pattern):
files.append({
"name": f.name,
"path": str(f.relative_to(runtime.config.workspace)),
"is_dir": f.is_dir(),
"size": f.stat().st_size if f.is_file() else None
})
return json.dumps({"files": files, "count": len(files)})
except Exception as e:
return json.dumps({"error": str(e)})
# Return all tools
return [
# ACP-powered file tools
acp_create_file,
acp_edit_file,
acp_delete_file,
acp_execute_command,
# LSP-powered code intelligence
lsp_list_symbols,
lsp_find_definition,
lsp_find_references,
lsp_get_diagnostics,
# Basic read-only tools
read_file,
list_files,
]
def get_tool_descriptions() -> Dict[str, str]:
"""Get descriptions of all agent-centric tools."""
return {
"acp_create_file": "Create a file through ACP with plan/approve/apply/verify",
"acp_edit_file": "Edit a file through ACP with tracking",
"acp_delete_file": "Delete a file through ACP (requires approval)",
"acp_execute_command": "Execute a shell command through ACP",
"lsp_list_symbols": "List symbols in a file using LSP",
"lsp_find_definition": "Find where a symbol is defined",
"lsp_find_references": "Find all references to a symbol",
"lsp_get_diagnostics": "Get errors and warnings for a file",
"read_file": "Read content from a file (read-only)",
"list_files": "List files in a directory",
}
@@ -530,39 +530,21 @@ class CodeIntelligenceRouter:
)
try:
import subprocess
workspace = self.runtime.config.workspace
cmd = ["grep", "-rn", search_term]
if file_path:
cmd.append(file_path)
else:
cmd.append(workspace)
result = subprocess.run(
cmd,
capture_output=True,
text=True,
timeout=10
)
results = []
if result.stdout:
for line in result.stdout.strip().split('\n')[:50]: # Limit results
if line:
parts = line.split(':', 2)
if len(parts) >= 2:
results.append({
"file": parts[0],
"line": int(parts[1]),
"content": parts[2] if len(parts) > 2 else ""
})
search_path = file_path or workspace
# Prefer the ripgrep-backed core grep (gitignore-aware, result-capped,
# truncation-safe). Fall back to a plain ``grep -rn`` only when the
# core tool is unavailable or errors.
results = self._core_grep(search_term, search_path)
if results is None:
results = self._fallback_grep(search_term, search_path)
citations = [
{"file": r["file"], "line": r["line"], "type": "search_result"}
for r in results
]
return CodeQueryResult(
intent=CodeIntent.SEARCH_CODE,
success=True,
@@ -579,6 +561,88 @@ class CodeIntelligenceRouter:
error=str(e)
)
def _core_grep(self, search_term: str, search_path: Optional[str]) -> Optional[List[Dict[str, Any]]]:
"""Search using the ripgrep-backed core ``grep`` tool.
The core tool contains ``path`` under the process working directory. We
pass the caller-resolved ``search_path`` (the target file when one was
given, otherwise the configured workspace) so the search targets the
same tree as the ``grep -rn`` fallback instead of the process CWD. If
that path escapes the core tool's root it returns an ``Error:`` string,
which we treat as "fall back" so the subprocess path (which searches the
workspace directly) still returns the expected results.
Core ``grep`` returns a newline-joined ``path:line: matched line``
string. We parse it into ``{file, line, content}`` dicts.
Returns the parsed results, or ``None`` when the core tool is
unavailable/errors so the caller can fall back to ``grep -rn``.
"""
try:
from praisonaiagents.tools import grep as core_grep
except ImportError:
return None
try:
output = core_grep(search_term, path=search_path or ".")
except Exception as e:
logger.debug(f"Core grep failed, will fall back: {e}")
return None
if not isinstance(output, str):
return None
stripped = output.strip()
# Core grep signals no matches / errors with a short human message
# rather than raising; treat those as "fall back" so a real ``grep``
# error surfaces the same way it did before.
if not stripped or stripped == "No matches found.":
return []
if stripped.startswith("Error:"):
logger.debug(f"Core grep returned error, will fall back: {stripped}")
return None
results: List[Dict[str, Any]] = []
for line in stripped.split('\n'):
if not line or line.startswith("... "): # skip truncation hint
continue
parts = line.split(':', 2)
if len(parts) >= 2:
try:
line_no = int(parts[1])
except ValueError:
continue
results.append({
"file": parts[0],
"line": line_no,
"content": parts[2] if len(parts) > 2 else ""
})
return results
def _fallback_grep(self, search_term: str, search_path: str) -> List[Dict[str, Any]]:
"""Last-resort search using a plain ``grep -rn`` subprocess."""
import subprocess
result = subprocess.run(
["grep", "-rn", search_term, search_path],
capture_output=True,
text=True,
timeout=10
)
results: List[Dict[str, Any]] = []
if result.stdout:
for line in result.stdout.strip().split('\n')[:50]: # Limit results
if line:
parts = line.split(':', 2)
if len(parts) >= 2:
results.append({
"file": parts[0],
"line": int(parts[1]),
"content": parts[2] if len(parts) > 2 else ""
})
return results
def _extract_file_from_query(self, query: str) -> Optional[str]:
"""Extract file path from query."""
# Look for file patterns
Loaded 100 of 131 files, more files were not shown because too many files have changed in this diff. Show more