Compare commits

..
251 Commits
Author SHA1 Message Date
MervinPraison 17aa27739e Release v0.2.46 2026-10-31 20:34:10 +00:00
praisonai-triage-agent[bot] fb6fb48816 Merge pull request #118 from MervinPraison/claude/issue-6168-20261031
feat: add You.com tools (migrated from praisonaiagents core)
2026-10-31 20:33:07 +00:00
MervinPraisonandClaude Opus 5.5 0fac8a0aa5 fix: honour explicit YouTools api_key, expose ydc alias, tighten missing-key test
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-31 18:21:52 +00:00
MervinPraisonandClaude Opus 5.5 072585870e feat: add You.com tools migrated from praisonaiagents core (fixes MervinPraison/PraisonAI#6168)
Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-31 18:17:30 +00:00
Mervin PraisonandClaude Opus 5.5 436a66e163 ci: bump github-automation-template to v1.1.20 (#117)
Co-authored-by: Claude Opus 5.5 (1M context) <[email protected]>
2026-10-29 02:38:18 +01:00
Mervin PraisonandClaude Opus 5.5 e18de78873 ci: bump github-automation-template to v1.1.19 (#116)
Co-authored-by: Claude Opus 5.5 (1M context) <[email protected]>
2026-10-26 11:42:24 +01:00
Mervin PraisonandClaude Opus 5.5 b8ee7dc5b2 chore(ci): github-automation-template v1.1.18 (#115)
Co-authored-by: Claude Opus 5.5 (1M context) <[email protected]>
2026-10-26 09:46:30 +01:00
Mervin PraisonandClaude Opus 5.5 edb4a66564 fix(ci): github-automation-template v1.1.17 (permission ceiling) (#114)
Co-authored-by: Claude Opus 5.5 (1M context) <[email protected]>
2026-10-17 09:34:43 +01:00
Mervin PraisonandClaude Opus 5.5 d332f6e28d chore(ci): github-automation-template v1.1.16 (#113)
Co-authored-by: Claude Opus 5.5 (1M context) <[email protected]>
2026-10-17 09:27:20 +01:00
d3de3800a2 chore(ci): bump github-automation-template to v1.1.15 (#112)
* chore(ci): bump github-automation-template to v1.1.15

Writers-only auto-Claude, least-privilege tokens for Claude on fork PRs and
in the merge-gate assessor, merge-gate held-out BLOCKs (no repeat verdicts on
an unchanged HEAD), portable prompts. auto-pr-comment sweep moves to hourly so
stale FINAL reviews are re-posted once a push settles.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* fix: sync automation.meta.json hubRef to v1.1.15

Co-Authored-By: Claude Opus 5.5 <[email protected]>

---------

Co-authored-by: MervinPraison <[email protected]>
Co-authored-by: Claude Opus 5.5 (1M context) <[email protected]>
2026-10-16 16:55:55 +01:00
Mervin PraisonandClaude Opus 5.5 7dbfbdf5ae fix(ci): github-automation-template v1.1.14 (fork PR isolation) (#111)
Co-authored-by: Claude Opus 5.5 (1M context) <[email protected]>
2026-10-15 08:58:50 +01:00
Mervin PraisonandClaude Opus 5.5 47c30e3c1d fix(ci): require write access to invoke @claude (template v1.1.13) (#110)
Co-authored-by: Claude Opus 5.5 (1M context) <[email protected]>
2026-10-15 08:30:31 +01:00
MervinPraisonandCursor c16bd22c6c chore(ci): bump github-automation-template hub to v1.1.12 (Opus/Sonnet 5.5)
Co-authored-by: Cursor <[email protected]>
2026-10-11 22:33:21 +01:00
MervinPraison 0237c6f4ce Release v0.2.45 2026-10-08 05:19:04 +00:00
aa8e7079f3 feat: add Keenable keyless web search tool (#105)
* feat: add Keenable keyless web search tool

Keenable is a web-search service (api.keenable.ai). Per AGENTS.md routing,
vendor search services belong in PraisonAI-Tools as standalone tool modules
(mirroring tavily_tool.py / brave_search_tool.py), not inline in the core
praisonaiagents web_search.py aggregator.

Keyless by default via the public endpoint (stdlib urllib, no install);
KEENABLE_API_KEY switches to the authenticated endpoint for higher limits.
Redirects are refused so X-API-Key is never resent to another host; results
are bounded (50 max, 1 MB body cap, title/URL/snippet length caps).

Auto-discovered by praisonai_tools.tools (no manual registration). 13 unit
tests cover endpoint selection, result mapping/bounds, and error handling.

Relates to MervinPraison/PraisonAI#5671

Co-authored-by: Mervin Praison <[email protected]>

* test: cover Keenable redirect refusal and bound limits

Address Greptile P2: the protective limits had no focused coverage and
build_opener was fully mocked, so _NoRedirect, the result cap, the
oversized title/url/snippet caps, and the response read limit could be
removed with the suite staying green.

- Lift _NoRedirect to module level so the redirect refusal is importable
  and unit-testable (was a closure inside search()).
- _FakeResponse.read now honours the requested byte count, so the
  oversized-response test exercises the actual read cap.
- Add TestProtections: result-count cap in the request body, title and
  snippet truncation, oversized-URL skip, read-limit enforcement, and
  _NoRedirect dropping a 302 (no follow-up request).
- Add the missing __init__ docstring.

Co-Authored-By: Claude Opus 4.8 <[email protected]>

---------

Co-authored-by: MervinPraison <[email protected]>
Co-authored-by: Mervin Praison <[email protected]>
Co-authored-by: Claude Opus 4.8 <[email protected]>
2026-10-08 06:18:03 +01:00
Mervin PraisonandCursor 4e6a092548 chore: bump github-automation-template hub_ref to v1.1.11 (#103)
Picks up the 403 label fix in the hub callable workflows.

Co-authored-by: Cursor <[email protected]>
2026-09-30 08:38:13 +01:00
MervinPraison 363eaeafb1 Release v0.2.44 2026-09-28 03:39:50 +00:00
praisonai-triage-agent[bot] a9f01306da Merge pull request #95 from MervinPraison/claude/issue-37-20260902-1212
feat: add StripeTool (read paths + guarded payment link create)
2026-09-22 18:07:00 +00:00
MervinPraison b0132aa26a Release v0.2.43 2026-09-22 14:33:46 +00:00
praisonai-triage-agent[bot] 65659c7122 Merge pull request #93 from MervinPraison/claude/issue-55-20260902-1212
fix: enable running both MDX escaping test suites together
2026-09-22 14:32:18 +00:00
praisonai-triage-agent[bot] a43433ee60 Merge pull request #94 from MervinPraison/claude/issue-38-20260902-1212
feat: add FirefliesTool (curated GraphQL operations)
2026-09-22 14:32:16 +00:00
praisonai-triage-agent[bot] d8483b476e Merge pull request #96 from MervinPraison/claude/issue-39-20260902-1212
feat: gated Reddit write actions + User-Agent template + tests
2026-09-22 14:32:14 +00:00
MervinPraisonandCursor a1df45aef9 chore: bump automation hub to v1.1.10 for merge gate fix
Fixes claude-assess hub-paths.js module error and GitHub MCP permission
denials in the merge gate callable workflow.

Co-authored-by: Cursor <[email protected]>
2026-09-22 15:22:58 +01:00
MervinPraisonandMervin Praison 876cf29010 fix: enable running both MDX escaping test suites together (fixes #55)
Add empty scripts/__init__.py so scripts/tests/test_mdx_escaping.py
resolves as a unique module (scripts.tests.test_mdx_escaping) instead
of colliding with tests/test_mdx_escaping.py. This lets the root and
comprehensive suites be collected together in CI without the pytest
"import file mismatch" error (issue Fix 4).

Both suites pass together: 7 + 24 = 31 passed.

Co-authored-by: Mervin Praison <[email protected]>
2026-09-17 07:18:41 +00:00
MervinPraisonandCursor d70fd1aaa4 chore(automation): bump hub to v1.1.9 for review-chain token fallback
Co-authored-by: Cursor <[email protected]>
2026-09-16 17:21:29 +01:00
MervinPraison 55edaf9a85 Release v0.2.42 2026-09-16 16:20:22 +00:00
MervinPraisonandCursor 4c35c4817c fix(ci): add httpx to dev extras for Notion tool unit tests
CI installs dev dependencies for pytest; Notion tests patch httpx.AsyncClient
and fail when httpx is not installed alongside httpx2 from openai.

Co-authored-by: Cursor <[email protected]>
2026-09-16 17:19:00 +01:00
MervinPraisonandCursor f28385458c chore(automation): bump hub to v1.1.8 for dual OAuth failover
Co-authored-by: Cursor <[email protected]>
2026-09-15 08:03:23 +01:00
MervinPraisonandClaude Opus 4.8 68f161f136 fix: drop redundant mode key in list_payment_links (banner adds it)
Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-09-07 13:01:03 +00:00
MervinPraison 6ac9056331 fix: align stripe extra with requests, map catalogue extra, harden HTTP status
- pyproject: [stripe] extra installs requests (the actual runtime dep) instead
  of the unused stripe SDK, matching sibling requests-based tools (swarmscore)
- catalogue: map stripe_tool -> ('stripe',) in _MODULE_EXTRAS so discovery
  reports the correct install extra
- stripe_tool: reject non-2xx responses lacking Stripe's error object instead
  of treating decoded JSON as success
- tests: give mock responses a real status_code; add non-2xx failure test
2026-09-07 12:57:49 +00:00
MervinPraisonandClaude Opus 4.8 838a7b7000 fix(fireflies): enforce HTTPS, surface HTTP errors, clamp limit, parse 429 payload
- Reject non-HTTPS FIREFLIES_API_URL so the API key never leaves over plaintext
- Return a distinct error for generic HTTP failures (e.g. 5xx) instead of
  normalizing an error body into an empty/missing result
- Parse GraphQL 429 payloads for extensions.metadata.retryAfter when the
  Retry-After header is absent
- Clamp limit to Fireflies' 1..50 page-size bound in list/search

Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-09-07 12:57:09 +00:00
MervinPraisonandClaude Opus 4.8 77df0ca393 fix: advertise gated write actions in RedditTool description
Update the agent-facing description so post_comment/submit_post are
discoverable, addressing Greptile review feedback on PR #96.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-09-07 12:56:33 +00:00
Mervin Praison 997898a5fa Merge pull request #101 from MervinPraison/chore/hub-v1.1.6
fix(ci): bump github-automation hub to v1.1.6
2026-09-07 13:54:40 +01:00
MervinPraisonandCursor 1ce2299a3d fix(ci): bump github-automation hub to v1.1.6
Fixes startup_failure on Auto PR Comment, merge gate, and Claude Assistant
(reusable-workflow caller permissions + hub checkout before claude-code-action).
Slim claude.yml to hub callable; gate-config.js unchanged (tools profile).

Co-authored-by: Cursor <[email protected]>
2026-09-07 13:54:30 +01:00
MervinPraisonandCursor 9d4e0e047c fix(gate-config): add triggerLogins so owner-posted FINAL @claude counts
Hub install preserved gate-config but omitted triggerLogins; merge gate and
pipeline sync only recognised github-actions[bot] FINAL triggers.

Co-authored-by: Cursor <[email protected]>
2026-09-02 22:01:01 +01:00
MervinPraisonandCursor 799bf1a1f9 chore(automation): bump hub callers to v1.1.2 (fix merge gate callable parse)
Co-authored-by: Cursor <[email protected]>
2026-09-02 21:57:00 +01:00
MervinPraisonandCursor ff10557b99 fix(claude): support fork PR review checkout (e.g. external contributor PRs)
Fetch pull/N/head for all PRs and block origin push on read-only forks so
claude-code-action does not fail with "couldn't find remote ref" on upstream.

Co-authored-by: Cursor <[email protected]>
2026-09-02 13:16:26 +01:00
MervinPraisonandMervin Praison 820833c1b3 feat: add gated Reddit write actions + User-Agent template + tests (fixes #39)
Add REDDIT_ALLOW_WRITE-gated post_comment and submit_post actions to
RedditTool, a compliant default User-Agent referencing the app id, and
unit tests covering import, schema generation, write-flag gating, and
mocked write flows.

Co-authored-by: Mervin Praison <[email protected]>
2026-09-02 12:15:07 +00:00
MervinPraisonandMervin Praison 361b1e70b9 feat: add StripeTool (read paths + guarded payment link create) (fixes #37)
Adds a narrow, audited StripeTool following existing tool patterns
(lazy optional requests import, clear errors). Covers Phase 1 reads
(get_customer, get_payment_intent, list_payment_links) and a Phase 2
guarded write (create_payment_link) requiring a correlation_id and
supporting an optional Idempotency-Key. Includes test/live key prefix
detection with a mode banner, audit-only logging (no payloads), the
[stripe] pyproject extra, .env.example, and unit tests.

Co-authored-by: Mervin Praison <[email protected]>
2026-09-02 12:14:56 +00:00
MervinPraisonandMervin Praison 8c0ac62c02 feat: add FirefliesTool with curated GraphQL operations (fixes #38)
Add a Fireflies.ai meeting-intelligence tool exposing curated GraphQL
operations (list_recent_meetings, get_transcript, search) with
FIREFLIES_API_KEY Bearer auth, lazy requests import, cursor pagination,
and error handling that distinguishes a missing key from HTTP 401/403
and surfaces retry_after on rate limits.

Co-authored-by: Mervin Praison <[email protected]>
2026-09-02 12:14:46 +00:00
MervinPraisonandCursor 5f4f694dc0 chore(automation): upgrade to hub v1.1.1 and fix pre-labelled issue Claude trigger
Switch merge-gate, review chain, CI failure, and pipeline workflows to thin
hub callers; keep local claude.yml for tools-specific prompts. Remove
redundant auto-issue-comment.yml — claude.yml now handles issues opened
with the claude label already present.

Co-authored-by: Cursor <[email protected]>
2026-09-02 13:08:31 +01:00
MervinPraison bd84a17340 Release v0.2.41 2026-09-02 11:43:17 +00:00
Mervin Praison 25e8bc6c6e Merge pull request #92 from MervinPraison/claude/issue-91-20260902-0915
feat: 8 meeting-agent tools (transcribe, CRUD, summarize, extract, index, search)
2026-09-02 12:42:18 +01:00
MervinPraisonandMervin Praison d4bdd82739 fix(meeting-tools): real chunk timestamps, safe re-index, bounded score
Address reviewer findings on PR #92:
- index_meeting now accepts optional segments and maps each chunk to the
  covering segment start/end times instead of hardcoding 0.0 (Greptile/CodeRabbit)
- embed before delete so a failed re-index never wipes prior searchable vectors
- search_meetings uses bounded 1/(1+distance) score for Chroma squared-L2 space
- raise openai floor to >=1.12.0 (timestamp_granularities support)
- use contextlib.closing for SQLite connections (leak-safe on error)
- strengthen registration test to detect shadowed exports
- add tests: segment->timestamp mapping, failed-reindex preservation, bounded score

Co-authored-by: Mervin Praison <[email protected]>
2026-09-02 09:39:00 +00:00
MervinPraisonandMervin Praison 88993be922 feat: add 8 meeting-agent tools (transcribe, CRUD, summarize, extract, index, search) (fixes #91)
Implements PAI-MEET-TOOLS: transcribe_file, save_meeting, get_meeting,
list_meetings, summarize_transcript, extract_action_items, index_meeting,
search_meetings as @tool functions with lazy openai/chromadb imports,
SQLite persistence, deterministic JSON output, and clear error mapping.

Co-authored-by: Mervin Praison <[email protected]>
2026-09-02 09:19:39 +00:00
MervinPraison 4c76cb69a3 Release v0.2.40 2026-08-28 09:32:21 +00:00
Mervin Praison 01e14d91f8 Merge pull request #89 from MervinPraison/fix/docs-generator-nav-update-errors
fix: update docs/docs.json path for Mintlify subpath hosting
2026-08-26 18:47:54 +01:00
MervinPraison 0b5eb93bb6 fix: update docs/docs.json path for Mintlify subpath hosting
Generator looked for docs.json at repo root; PraisonAIDocs uses
docs/docs.json so nav was never updated after auto-gen. Also run
sync_sdk_reference_nav.py after generation as a safety net.
2026-08-25 21:49:23 +01:00
MervinPraison 936ff4f3fa Release v0.2.39 2026-08-25 01:07:44 +00:00
Mervin Praison 30c1b54fe5 Merge pull request #88 from MervinPraison/fix/docs-generator-nav-update-errors
fix: stop swallowing docs.json nav update failures
2026-08-20 10:19:12 +01:00
MervinPraisonandMervin Praison 8da1746df8 fix(docs): report actual docs.json page count after legacy filtering
Legacy layout strips /index and /__init__ before writing package_group['pages'],
so logging len(unique_pages) overstated the count. Count the final navigation
entries instead (granular sums sub-group pages).

Co-authored-by: Mervin Praison <[email protected]>
2026-08-19 21:04:13 +00:00
MervinPraison 8928ed870a fix: stop swallowing docs.json nav update failures
Auto-generate was writing MDX without updating docs.json when update_docs_json failed silently, breaking nav-check on every docs PR.
2026-08-19 21:48:01 +01:00
MervinPraison 1eabeb4b25 Release v0.2.38 2026-08-19 01:05:53 +00:00
MervinPraison 3819b3d51d fix(docs): serialize generate-docs pushes and rebase before push
Prevent parallel workflow runs from racing on PraisonAIDocs main; pull
--rebase before push when SDK reference pages were regenerated.
2026-08-16 07:22:18 +01:00
MervinPraison fe205e3b77 fix(docs): use CI docs-root in generator path tracking
Track generated MDX paths relative to the passed --docs-root instead of a
hardcoded local PraisonAIDocs path so generate-docs runs in GitHub Actions
can commit navigation updates.
2026-08-16 07:21:15 +01:00
Mervin Praison 193bd05793 Merge pull request #86 from MervinPraison/claude/issue-85-20260816-0512
fix: unwrap describe envelope and structured cost in registry_proxy
2026-08-16 07:18:11 +01:00
MervinPraisonandMervin Praison 7e23cddd9b fix: reject non-finite/negative prices in registry_proxy spend guard
NaN/inf prices passed float() but made both per-call and session-spend
comparisons evaluate false, bypassing the configured guard and poisoning
_session_spend. Fail closed on non-finite/negative prices in _check_budget
and validate the response charge in spend recording so a malformed response
cannot corrupt cumulative spend.

Co-authored-by: Mervin Praison <[email protected]>
2026-08-16 05:31:42 +00:00
MervinPraisonandMervin Praison 96b7a2f8e4 fix: unwrap describe envelope and structured cost in registry_proxy (fixes #85)
Both _extract_method and _extract_price now unwrap the live registry's
`endpoint` envelope before reading fields, and _extract_price prefers the
structured cost object's `usd` value (guard budgets are USD). Flat forms
and fail-closed behaviour are preserved. Fixtures mirror the live schema.

Co-authored-by: Mervin Praison <[email protected]>
2026-08-16 05:14:11 +00:00
praisonai-triage-agent[bot] 0d3e97a6b1 Merge pull request #84 from MervinPraison/claude/issue-83-20260815-1921
fix: registry_proxy describe path + HTTP method support
2026-08-15 20:30:27 +00:00
MervinPraisonandMervin Praison 23e3a88eb0 fix(registry_proxy): validate describe_path, encode tool_id, normalize+whitelist method
Address reviewer findings on PR #84:
- describe_path template validated (must contain {tool_id}; malformed templates
  return a tool-result error instead of raising, honoring the error contract)
- tool_id URL-encoded on both describe and /call routes (prevents path escape)
- explicit method stripped/upper-cased and validated against an allowed set
- deny-path spend-guard tests assert client.request (not client.post) not called

Co-authored-by: Mervin Praison <[email protected]>
2026-08-15 19:45:24 +00:00
MervinPraisonandMervin Praison 923425b443 fix: registry_proxy describe path + HTTP method support (fixes #83)
- describe now uses /catalog/endpoints/{tool_id} (configurable via
  TOOL_PROXY_DESCRIBE_PATH); fixes 404s that also broke spend-guarded calls
- registry_call derives the endpoint's HTTP method from describe (reusing the
  guard's describe fetch); GET sends query params, POST/PUT send a JSON body;
  method can be passed explicitly to skip the lookup
- docs: describe-path env var, HTTP-method behaviour, passthrough-URL/SSRF note

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-authored-by: Mervin Praison <[email protected]>
2026-08-15 19:26:59 +00:00
praisonai-triage-agent[bot] 310f15d6eb Merge pull request #72 from MervinPraison/claude/issue-63-20260727-0909
fix: mock langextract in test_agent_usage_pattern
2026-08-15 12:35:50 +00:00
MervinPraison 893cfea6e7 fix(docs): use GH_TOKEN fallback for PraisonAIDocs checkout/push
DOCS_GITHUB_TOKEN was never synced to this repo; fall back to GH_TOKEN so
generate-docs can checkout and push reference docs after main merges.
2026-08-15 13:34:30 +01:00
MervinPraison 0605eec551 fix(ci): unblock auto-merge after conflict rebase and sync PraisonAIDocs
Recognise MervinPraison Claude finished replies in stale-FINAL detection,
re-post FINAL immediately after conflict rebase pushes, broaden generate-docs
triggers to all praisonai_tools changes plus post-main CI, and sync pipeline
labels on push/PR events.
2026-08-15 13:33:24 +01:00
praisonai-triage-agent[bot] 2098ab469d Merge pull request #53 from MervinPraison/claude/issue-52-20260713-2242
feat: automatic tool discovery (fixes #52)
2026-08-15 12:21:22 +00:00
MervinPraisonandMervin Praison 1424ce7769 test: isolate langextract test output to temp dirs (fixes #63)
Address Greptile finding: langextract integration/render tests wrote
generated HTML into the working directory. Pass temporary output_path
in test_agent_usage_pattern, test_contract_analysis_scenario, and
test_render_file_success so tests no longer modify the checkout or
collide across runs.

Also remove committed test artifacts (contract-001.html,
financial-analysis.html, tmpc8_hvm6m.html) that leaked from these
CWD writes in the original langextract PR.

Co-authored-by: Mervin Praison <[email protected]>
2026-08-15 11:41:09 +00:00
MervinPraisonandMervin Praison 3a67c9bc59 test: run tool-discovery parse guard inside CI-covered path
CI runs `pytest tests/unit --ignore=tests/unit/video`, so the parse-failure
and collision guards in tests/test_tool_discovery.py (at tests/ root) never
executed in CI. Adds a thin bridge under tests/unit/tools/ so a syntactically
broken shipped tool module fails loudly at CI time instead of silently
dropping out of discovery and surfacing later as an AttributeError.

Closes the remaining Greptile P1 (parse test misses CI).

Co-authored-by: Mervin Praison <[email protected]>
2026-08-15 11:40:59 +00:00
MervinPraisonandMervin Praison 5a233cb3b5 test: account for top-level catalogue exports after rebase onto main
Main (#74) added ToolCatalogueEntry/list_tools/get_tool_names to the
top-level __all__. The top-level surface now equals the tools-package
surface plus those catalogue helpers, so test_all_matches_manifest is
updated accordingly.

Co-authored-by: Mervin Praison <[email protected]>
2026-08-15 11:40:59 +00:00
MervinPraisonandMervin Praison 44cbefe9b7 test: fail loudly if a shipped tool module cannot be parsed
Discovery intentionally swallows a per-module SyntaxError so one broken
module cannot break the whole package import, but that meant a broken
shipped module silently dropped from the manifest (resolving as
AttributeError instead of the real parse error). The existing coverage
test shared the same swallowing helper, so it could not catch it.

Add test_every_shipped_module_parses which ast.parse()s each module
directly, surfacing the pointing SyntaxError at CI time. Addresses the
re-raised Greptile P1.

Co-authored-by: Mervin Praison <[email protected]>
2026-08-15 11:40:59 +00:00
MervinPraisonandMervin Praison 5c302e2acc fix: respect __all__, skip private modules, cache discovery scan
Address reviewer feedback on PR #53:
- Honour an explicit module-level __all__ (CodeRabbit/Gemini): modules that
  expose tools via aliases can opt them in; bare aliases are otherwise not
  auto-exported to avoid cross-module name collisions.
- Skip underscore-prefixed modules so internal code cannot leak into the
  public surface (Greptile).
- Warn instead of silently dropping tools when a module fails to parse
  (Greptile).
- Guard the drop-in test for read-only installs and use a non-underscore
  probe module (Gemini); add an n8n external-symbol resolution test
  (CodeRabbit) and an explicit-__all__ test.
- Memoise the AST scan in a fingerprinted on-disk cache, cutting import
  time from ~260ms to ~70ms (AGENTS.md <200ms budget) while keeping
  discovery automatic; add cache round-trip/corruption tests and gitignore.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-authored-by: Mervin Praison <[email protected]>
2026-08-15 11:40:59 +00:00
MervinPraisonandMervin Praison 517a3b8bcb feat: replace hand-maintained tool import map with automatic discovery (fixes #52)
Adding a tool now means adding one file under praisonai_tools/tools/.
An AST-based scan builds the name->module manifest without importing any
tool module, preserving per-symbol lazy import. __all__ at both package
levels is derived from the same manifest so they cannot drift, and
collisions are detected at discovery time. README now documents that this
package uses in-package discovery while external packages use the
praisonaiagents.tools entry-point group.

Co-authored-by: Mervin Praison <[email protected]>
2026-08-15 11:40:59 +00:00
Mervin Praison 6d6330087e Merge pull request #67 from MervinPraison/claude/issue-58-20260727-0903
fix(tests): mock langextract in agent usage integration test
2026-08-15 12:39:39 +01:00
Mervin Praison ea56295eac Merge pull request #71 from MervinPraison/claude/issue-61-20260727-0908
fix: sync marketplace tool wrapper docstrings with signature tests
2026-08-15 12:38:52 +01:00
Mervin Praison 251ed8b800 Merge pull request #73 from MervinPraison/claude/issue-60-20260727-0907
fix(docs): repair MDX escaping utilities (fixes #60)
2026-08-15 12:38:06 +01:00
Mervin Praison 2720359927 Merge pull request #77 from MervinPraison/claude/issue-76-20260806-0854
feat: source-derive the tool catalogue (fixes #76)
2026-08-15 12:37:37 +01:00
Mervin Praison 6087b47bf8 Merge pull request #81 from MervinPraison/claude/issue-80-20260814-1422
fix: silence benign 'Registry is None' WARNING spam on bare import
2026-08-15 12:36:53 +01:00
praisonai-triage-agent[bot] b7d3a4a178 Merge pull request #82 from MervinPraison/claude/issue-79-20260814-1422
feat: registry_proxy wire-compatibility — configurable auth header and price-field tolerance
2026-08-14 15:33:01 +00:00
MervinPraisonandpraisonai-triage-agent[bot] 5011f3311c fix: add noqa E402 for intentional post-code imports in decorator
The logging-suppression filter must run before the @tool re-exports are
imported, requiring those imports to sit after module-level code. Mark them
with noqa: E402 to satisfy ruff while preserving the required ordering.

Co-authored-by: praisonai-triage-agent[bot] <praisonai-triage-agent[bot]@users.noreply.github.com>
2026-08-14 14:47:07 +00:00
MervinPraisonandMervin Praison 9699ff1513 feat: registry_proxy wire-compat — configurable auth header and price-field tolerance (fixes #79)
- Add TOOL_PROXY_AUTH_HEADER / auth_header param: default Authorization sends
  Bearer, a custom header name (e.g. X-Treg-Token) sends the raw token.
- Harmonise budget check to accept price_per_call -> price -> cost (same
  tolerance as spend recording); keep fail-closed when none parse.

Co-authored-by: Mervin Praison <[email protected]>
2026-08-14 14:25:39 +00:00
MervinPraisonandMervin Praison dfd2d18357 fix: silence benign 'Registry is None' WARNING spam on bare import (fixes #80)
Importing praisonai_tools tools (e.g. registry_proxy) outside a running
agent context emitted one WARNING per @tool-decorated function from
praisonaiagents.tools.decorator ("Registry is None for tool …"). On a bare
import that is the normal state, not an anomaly, so it polluted logs/CI for
every library consumer and scaled with the tool catalogue.

Attach a targeted logging.Filter (to the root logger and its handlers, where
praisonaiagents routes the record) that drops only that specific message and
passes all other logging through untouched. Registration behaviour is
unchanged; the root fix (defer/downgrade) belongs in praisonaiagents core.

Co-authored-by: Mervin Praison <[email protected]>
2026-08-14 14:25:32 +00:00
MervinPraison f10936de79 Release v0.2.37 2026-08-14 14:16:18 +00:00
Mervin Praison 49b717ecfe Merge pull request #78 from MervinPraison/claude/issue-3937-20260814
feat: optional tool-registry proxy connector
2026-08-14 15:15:24 +01:00
MervinPraisonandMervin Praison 6acddaaa95 fix(registry-proxy): bind token to trusted config, persist session spend, fail-closed budget
Addresses reviewer findings on PR #78:
- Do not attach the ambient TOOL_PROXY_TOKEN when an explicit proxy_url is
  supplied without a matching token, preventing token exfiltration to an
  attacker-controlled endpoint (Critical).
- Persist cumulative spend per (proxy_url, token) via a module-level store so
  max_session_spend enforces across separate registry_call invocations (Major).
- Fail closed in _check_budget when a guard is configured but price cannot be
  validated; record spend deterministically from the quoted price (Major).
- Add regression tests + autouse spend-state reset fixture; update docs.

Co-authored-by: Mervin Praison <[email protected]>
2026-08-14 14:05:23 +00:00
MervinPraisonandMervinPraison ed07dfc6a5 feat: optional tool-registry proxy connector (fixes MervinPraison/PraisonAI#3937)
Add a bring-your-own-account connector mirroring PraisonAI's deferred
tool-search bridge: registry_search / registry_describe / registry_call
over a registry's plain-HTTP proxy (token header, configurable base URL).
Credentials are injected server-side by the registry, never held by the
agent. Connector is disabled unless TOOL_PROXY_URL is set; httpx is
lazy-imported; errors surface as {"error": ...} tool results. Includes
optional per-call/per-session spend guards, docs (Tiers 0/1/2) and mocked
HTTP unit tests plus a skills-dir compatibility test.

Co-authored-by: MervinPraison <[email protected]>
2026-08-14 13:45:49 +00:00
MervinPraisonandMervin Praison ce9cbe08e6 fix: read annotated tool descriptions and scope catalogue fallback to nested tools
Address reviewer findings on PR #77:
- _scan_module now also reads annotated `description: str = "..."` class
  attributes (AnnAssign), so CapsuleTool/NexusPredictionMarketTool surface
  their real descriptions instead of name-synthesised summaries.
- Restrict the _TOOL_MAP catalogue fallback to nested (absolute-module) tools
  only; flat *_tool.py classes stay fully source-derived, preserving the
  no-hand-maintained-bookkeeping guarantee.
- Fix phantom-guard test to resolve absolute tool modules correctly so a
  missing nested first-party module is flagged as a phantom, not tolerated as
  an optional dependency.
- Tighten CalculatorTool summary assertion and add regression coverage for
  annotated descriptions.

Co-authored-by: Mervin Praison <[email protected]>
2026-08-06 09:18:35 +00:00
MervinPraisonandMervin Praison 9d17d0e7a1 feat: source-derive the tool catalogue to eliminate hand-maintained bookkeeping (fixes #76)
Derive the first-party tool catalogue from the source tree instead of the
hand-maintained _TOOL_MAP, so adding a tool becomes a single-file change and
phantom names (advertised tools with no file) are impossible by construction.

- catalogue.py: AST-scan praisonai_tools/tools/*_tool.py for *Tool classes and
  their literal description (cheap, import-free, no optional deps). Summaries now
  come from each tool's real description rather than name-synthesis.
- tools/__init__.py: extend lazy __getattr__ to fall back to the source-derived
  scan, so dropping a *_tool.py makes its class importable with no _TOOL_MAP edit.
  Auto-discovers previously-unmapped ClaudeMemoryTool/CrowPayTool/NightmarketTool.
- tests/test_catalogue_no_phantoms.py: CI guard asserting every catalogue name
  resolves (tolerating only missing optional deps), maps to a real file, and that
  auto-discovery + real-description summaries work.

Lazy-import behaviour and cheap `import praisonai_tools` are preserved; existing
public import names remain stable.

Co-authored-by: Mervin Praison <[email protected]>
2026-08-06 08:59:52 +00:00
MervinPraison 8e1fe1d860 Release v0.2.36 2026-08-04 11:41:31 +00:00
praisonai-triage-agent[bot] 0629d18a4e Merge pull request #75 from MervinPraison/claude/issue-74-20260804-0919
feat: add tool discovery/catalogue contract
2026-08-04 11:40:29 +00:00
MervinPraisonandMervin Praison 1950368530 feat: add tool discovery/catalogue contract (fixes #74)
Expose a programmatic catalogue of shipped tools plus a `praisonai.tools`
entry-point discovery contract so validators/CLI/docs/third-party packages
resolve against the live catalogue instead of a hand-maintained allowlist
that drifts.

- praisonai_tools.catalogue: ToolCatalogueEntry, list_tools(), get_tool_names()
- Derive catalogue from the existing _TOOL_MAP source of truth (no optional deps)
- Discover third-party tools via `praisonai.tools` entry points
- Add `praisonai-tools list [--extras]` CLI subcommand
- Document the entry-point group in pyproject.toml
- Tests in tests/test_catalogue.py

Co-authored-by: Mervin Praison <[email protected]>
2026-08-04 09:24:51 +00:00
MervinPraison ea270f747a Release v0.2.35 2026-07-31 02:37:35 +00:00
praisonai-triage-agent[bot] d06254fc68 Merge pull request #66 from MervinPraison/claude/issue-57-20260727-0902
fix(video): align motion graphics team tests with hierarchical API
2026-07-27 14:58:54 +00:00
MervinPraisonandMervin Praison e0b164c530 fix(video): use hierarchical coordinator in motion graphics example
Replace stale team.leader.name access with team.agents[0].name to match
the hierarchical AgentTeam API (no static leader in this mode).

Co-authored-by: Mervin Praison <[email protected]>
2026-07-27 14:14:49 +00:00
MervinPraisonandMervin Praison a8627e295e fix(video): align motion graphics team tests with hierarchical API (fixes #57)
The integration tests asserted `team.leader` and an exact `kwargs` dict,
but the preset uses `process="hierarchical"` where the coordinator is the
first agent (`team.agents[0]`) and `process`/`manager_llm` are forwarded.
Update tests to the hierarchical semantics and document the pattern in the
`motion_graphics_team` docstring.

Co-authored-by: Mervin Praison <[email protected]>
2026-07-27 14:14:49 +00:00
praisonai-triage-agent[bot] 2b159d8816 Merge pull request #70 from MervinPraison/claude/issue-62-20260727-0908
fix: align motion graphics team tests with hierarchical AgentTeam API
2026-07-27 14:12:05 +00:00
praisonai-triage-agent[bot] 27b0f5e7a3 Merge pull request #65 from MervinPraison/claude/issue-54-20260727-0859
fix: add video-motion deps to dev extra; skip video tests when missing
2026-07-27 14:06:18 +00:00
MervinPraisonandMervin Praison f44138e6d9 fix(docs): make Mintlify JSX components case-sensitive in MDX escaping
HTML tags stay case-insensitive (<div>/<DIV>) but JSX components are now
matched exactly, so a mis-cased component such as <badge> is escaped as
prose instead of being emitted as a broken component. escape_mdx and
validate_mdx share a single _is_valid_mdx_tag helper so they stay in
lockstep. Resolves the outstanding Greptile P1 tag-casing finding.

Co-authored-by: Mervin Praison <[email protected]>
2026-07-27 12:06:09 +00:00
MervinPraisonandMervin Praison ea9407b34b style(video-tests): fix ruff I001/SIM117 in video test files
Clears this PR's lint contribution (video dir 13 -> 0 ruff errors):
sort imports and combine nested `with` statements. No behaviour change;
video tests still pass (54 passed with deps). Repo-wide ruff debt in
praisonai_tools/ (~5321 pre-existing errors) is out of scope for this
video-test-deps PR.

Co-authored-by: Mervin Praison <[email protected]>
2026-07-27 12:03:57 +00:00
MervinPraisonandMervin Praison ae16e3eebb test(video): narrow video-dep guards to backend-constructing tests
Replace the directory-wide pytest.importorskip in tests/unit/video/conftest.py
with per-test skipif guards on only the tests that build a real
HtmlRenderBackend. Dependency-independent protocol, mock-backend and retry
tests now run on a minimal `pip install -e '.[dev]'` (40 passed / 14 skipped
without video-motion; 54 passed with it), addressing Greptile's feedback that
the broad skip dropped useful coverage.

Also make test_import_error_handling deterministic by patching async_playwright
so the imageio-ffmpeg guard is what's under test regardless of installed deps.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-authored-by: Mervin Praison <[email protected]>
2026-07-27 12:03:57 +00:00
MervinPraisonandMervin Praison 803f2b219e fix: add video-motion deps to dev extra and skip video tests when missing (fixes #54)
- Add playwright and imageio-ffmpeg to the [dev] extra so the full pytest
  suite runs after `pip install -e ".[dev]"` (Option A).
- Add tests/unit/video/conftest.py with pytest.importorskip guards so
  minimal installs skip video tests gracefully instead of erroring (Option C).
- Document the video-motion render deps and `playwright install chromium`
  step in README Testing section.

Co-authored-by: Mervin Praison <[email protected]>
2026-07-27 12:03:42 +00:00
MervinPraisonandMervin Praison 7b36689195 fix(ci): pin ruff to E/F rule families to unblock CI lint
The CI `ruff check praisonai_tools tests` step had no ruff config, so a
newer ruff release pulled in strict opinionated defaults (UP/I/W/SIM/RUF)
flagging 5000+ pre-existing, non-functional findings across untouched
files. This pre-dated PR #62 (failing on main at b96bfbc).

Add [tool.ruff.lint] selecting the historical E/F baseline that the
existing `--ignore E…,F…` CI flags already imply, restoring green
without editing unrelated files or weakening any tests.

Co-authored-by: Mervin Praison <[email protected]>
2026-07-27 12:02:27 +00:00
MervinPraisonandMervin Praison f519a92927 fix(tests): align motion graphics team tests with hierarchical API (fixes #62)
Update integration tests to match the hierarchical process pattern used by
motion_graphics_team(): the coordinator is the semantic leader at agents[0]
and process/manager_llm are forwarded into AgentTeam kwargs. Document the
hierarchical coordinator role in the team docstring.

Co-authored-by: Mervin Praison <[email protected]>
2026-07-27 12:02:06 +00:00
MervinPraisonandMervin Praison ffeeae549b fix(ci): pin ruff rule set to E,F to restore green lint
ruff 0.16.0 expanded its default rule set, causing `ruff check` in CI to
flag thousands of pre-existing, repo-wide style items (UP006/UP045/I001/
RUF022/BLE001/...) in files unrelated to this PR. This failure is present
on `main` (base b96bfbc) as well.

Add a [tool.ruff.lint] select = ["E", "F"] to pyproject.toml to pin the
historically-green pycodestyle+pyflakes contract regardless of ruff
version. No test behaviour is changed; MDX suites still pass (7/7 + 24/24).

Co-authored-by: Mervin Praison <[email protected]>
2026-07-27 12:01:49 +00:00
MervinPraisonandMervin Praison 9dba6906a5 fix(docs): unify MDX tag-casing allowlist, harden placeholders, wrap mixed examples
Addresses PR #73 reviewer findings:
- validate_mdx now shares escape_mdx's case-insensitive allowlist via
  _VALID_MDX_TAGS_LOWER, so mis-cased components (e.g. <badge>) are treated
  consistently by both and don't fail validation after being preserved.
- Replace collision-prone __MDX_TAG_n__/__CODE_BLOCK_n__ placeholders with
  per-call random NUL-delimited sentinels so literal docstring text can't be
  corrupted during restore.
- _wrap_docstring_code_examples no longer bails on the first fence; it tracks
  fenced regions and still wraps later indented Usage/Example sections.

Co-authored-by: Mervin Praison <[email protected]>
2026-07-27 12:01:35 +00:00
MervinPraisonandMervin Praison 192f40ee61 fix(docs): repair MDX escaping utilities and add reference-docs shim (fixes #60)
- Rewrite sanitize_type_for_mdx to collapse Callable/Literal/Coroutine and
  nested Optional[Union[...]] while preserving simple generics
- Make escape_mdx selectively escape unknown <tag>/{expr} while preserving
  valid Mintlify components and wrapping docstring code examples
- Auto-simplify complex nested types in escape_for_table
- Add validate_docs_json_structure and module-level VALID_MDX_TAGS
- Add missing hooks/mcp/policy icons to ICON_MAP
- Add scripts/generate_reference_docs.py shim (stdlib-only) so both test
  suites resolve the canonical generator implementation

tests/test_mdx_escaping.py: 7/7 pass; scripts/tests/test_mdx_escaping.py: 24/24 pass

Co-authored-by: Mervin Praison <[email protected]>
2026-07-27 12:01:35 +00:00
MervinPraison a79978d3b7 Release v0.2.34 2026-07-27 12:01:13 +00:00
praisonai-triage-agent[bot] fb45d41170 Merge pull request #69 from MervinPraison/claude/issue-59-20260727-0906
fix: include video-motion deps in dev extra + graceful skip
2026-07-27 12:00:18 +00:00
MervinPraisonandMervin Praison 28b4598ac6 fix(tests): resolve ruff lint errors in langextract test file
Fix I001 (import ordering) and SIM117 (nested with statements)
introduced by the temp-dir isolation changes, so the ruff lint
check passes on the changed file.

Co-authored-by: Mervin Praison <[email protected]>
2026-07-27 09:40:47 +00:00
MervinPraisonandMervin Praison 191a4a6641 fix(ci): pin ruff rule set to historical default to stop spurious lint failures
The CI "Lint with ruff" step installs ruff unpinned; ruff 0.16.0 enabled
thousands of new default rules across the pre-existing codebase, failing
main and every PR (including this one) regardless of their diff. Pin
[tool.ruff.lint] select to E4/E7/E9/F — the rules the CI --ignore flags
were written against — restoring deterministic, intent-preserving linting
without mass-rewriting existing code.

Co-authored-by: Mervin Praison <[email protected]>
2026-07-27 09:36:58 +00:00
MervinPraisonandMervin Praison b8d0694f36 test: align trust config default assertion with code (min_score=1.5)
The test asserted min_score == 3.0 but TrustConfig deliberately defaults to
1.5 (Joy network avg is 1.1, 'standard' threshold), consistent with from_env.
Updated the stale assertion to match the intentional code default.

Co-authored-by: Mervin Praison <[email protected]>
2026-07-27 09:35:24 +00:00
MervinPraisonandMervin Praison 3060daa92c test(video): scope video-dep skips to backend-bound tests (addresses Greptile P2)
Replace the directory-wide pytest.importorskip guard in the video conftest
with a `requires_video_deps` marker + collection hook that skips ONLY the
tests that construct HtmlRenderBackend (which hard-requires playwright /
imageio-ffmpeg). Dependency-independent tests (protocol definitions,
dataclasses, mocked backends) now retain coverage on minimal installs.

Verified:
- deps absent:  39 passed, 15 skipped (previously entire tree skipped)
- deps present: 54 passed (unchanged)
- full unit:    227 passed, 6 skipped

Co-authored-by: Mervin Praison <[email protected]>
2026-07-27 09:29:43 +00:00
MervinPraisonandMervin Praison e78237f4d2 docs: complete return-shape docs for marketplace wrappers (fixes #61)
Address Greptile P2 review feedback on PR #71:
- verify_agent_identity: document agent_url (always present) and timestamp
  (present on success) return fields
- pinchwork_delegate: document error-string return paths (missing input,
  missing httpx, connection/HTTP errors, unexpected exceptions)

Co-authored-by: Mervin Praison <[email protected]>
2026-07-27 09:28:21 +00:00
MervinPraisonandMervin Praison 3505e63a3d fix(tests): direct langextract HTML output to temp dir to avoid test pollution
Integration and render_file tests called the tool without output_path,
so the real writer left *.html artifacts (financial-analysis.html,
contract-001.html, tmp<stem>.html) in the working directory. Route
output into TemporaryDirectory (matching test_extract_success) so no
files escape test isolation.

Co-authored-by: Mervin Praison <[email protected]>
2026-07-27 09:17:13 +00:00
MervinPraisonandMervin Praison 292ef01194 fix: add Google-style Args to marketplace tool wrapper docstrings (fixes #61)
Expand thin-wrapper docstrings for pinchwork_delegate, verify_agent_identity,
check_behavioral_trust, and verify_task_delegation_safety with Args/Returns
sections so signature tests in tests/test_marketplace_tools.py pass.

Co-authored-by: Mervin Praison <[email protected]>
2026-07-27 09:10:08 +00:00
MervinPraisonandMervin Praison bcaa5a5219 fix: include video-motion deps in dev extra and skip video tests when absent (fixes #59)
- Add playwright and imageio-ffmpeg to the [dev] extra so a standard
  `pip install -e ".[dev]"` runs the video/motion-graphics tests green.
- Add tests/unit/video/conftest.py with importorskip guards so minimal
  installs skip the video suite gracefully instead of raising ImportError.

Co-authored-by: Mervin Praison <[email protected]>
2026-07-27 09:08:46 +00:00
MervinPraisonandMervin Praison 6a6c8646e4 fix(tests): mock langextract in agent usage integration test (fixes #58)
test_agent_usage_pattern invoked langextract_extract without mocking
_get_langextract, causing failure on dev-only installs where the optional
langextract package is absent. Added the same mocks used by the sibling
test_contract_analysis_scenario for consistency.

Co-authored-by: Mervin Praison <[email protected]>
2026-07-27 09:04:50 +00:00
MervinPraison b96bfbc7e2 fix(release): 3-day patch cadence and explicit PyPI __token__ auth
Extend release-gate dedupe from daily to every 3 days and configure uv publish with UV_PUBLISH_USERNAME=__token__ per PyPI API token guidance.
2026-07-25 18:24:01 +01:00
MervinPraison f146c6c7ad Release v0.2.33 2026-07-10 04:01:00 +00:00
Mervin Praison 82fc2031f4 Merge pull request #41 from rakshith48/firecrawl-v2-upgrade
Upgrade Firecrawl tool to firecrawl-py v2 SDK + add search()
2026-07-09 18:35:36 +01:00
MervinPraison c427e2588e Release v0.2.32 2026-07-09 17:05:21 +00:00
praisonai-triage-agent[bot] 30719720cd Merge pull request #30 from MervinPraison/claude/issue-1593-20260501
feat: Add SwarmScore integration tool for agent trust ratings
2026-07-09 16:41:05 +00:00
Mervin Praison d4ac1e42ce Merge pull request #51 from MervinPraison/fix/merge-gate-stash-config
fix(merge-gate): stash gate-config.js with merge-gate.js
2026-07-09 16:21:23 +01:00
MervinPraisonandMervin Praison d8ee45d8b7 style(swarmscore): remove extraneous f-string prefixes in example (ruff F541)
Co-authored-by: Mervin Praison <[email protected]>
2026-07-09 14:57:17 +00:00
MervinPraison 3bf058e687 fix(merge-gate): stash gate-config.js with merge-gate.js
After PR checkout, require(/tmp/merge-gate.js) failed looking for
./gate-config.js. Keep both scripts in /tmp/merge-gate-scripts/.
2026-07-09 15:57:13 +01:00
Mervin Praison 04f1d165ff Merge pull request #42 from MervinPraison/claude/issue-1887-20260610
feat: add Inkog security analysis tool for AI agent code
2026-07-09 15:56:21 +01:00
MervinPraisonandMervin Praison d8d3ef8df0 fix(swarmscore): default run() action to 'load' for graceful degradation
Every other multi-action tool in the repo provides a default for `action`.
Without it, a bare `tool.run()` raised a raw TypeError bypassing the
ToolResult(success=False) error path. Add default + regression test.

Co-authored-by: Mervin Praison <[email protected]>
2026-07-09 13:42:35 +00:00
Mervin Praison 11f67da66f Merge pull request #50 from MervinPraison/fix/merge-gate-claude-action
fix(merge-gate): add claude-code-action and restore after PR checkout
2026-07-09 14:37:41 +01:00
MervinPraisonandMervin Praison 222787293c fix(swarmscore): URL-encode slug in load-by-slug path
Prevents malformed request paths when a slug contains special
characters (e.g. '/', spaces). Adds a regression test asserting
the encoded path.

Co-authored-by: Mervin Praison <[email protected]>
2026-07-09 13:33:46 +00:00
MervinPraisonandMervin Praison 5353ea28d6 fix: guard analyze_findings against non-dict scan output
Move the "error" key guard inside the try block and short-circuit
non-dict (text/HTML) scan results to prevent an unhandled TypeError
when a string containing "error" is passed. Add regression test.

Co-authored-by: Mervin Praison <[email protected]>
2026-07-09 13:32:24 +00:00
MervinPraisonandMervin Praison d873ccd07b fix: standardize Inkog CLI flags to single-dash across all scan methods
skill_scan and mcp_scan used double-dash flags (--deep, --output, --repo)
while scan_directory and the version check (inkog -version) use single-dash
Go-style flags. Align subcommands and docs on single-dash for consistency.

Co-authored-by: Mervin Praison <[email protected]>
2026-07-09 13:32:24 +00:00
MervinPraisonandpraisonai-triage-agent[bot] 11ca0e8bde fix: honor InkogTool api_key in availability check and add severity to scan_agent_code
- _check_inkog_available now accepts an api_key arg so the constructor's
  api_key is respected instead of only reading INKOG_API_KEY env var
- scan_agent_code now forwards a severity parameter (matches docs/CI examples)
- add regression tests for both fixes

Co-authored-by: praisonai-triage-agent[bot] <praisonai-triage-agent[bot]@users.noreply.github.com>
2026-07-09 13:32:24 +00:00
MervinPraison f39b01afda Fix ruff F541 in Inkog tool. 2026-07-09 13:32:24 +00:00
MervinPraison ef02203d46 Fix ruff lint in Inkog tool tests. 2026-07-09 13:32:24 +00:00
MervinPraison 5a08365e15 feat: add Inkog security analysis tool for AI agent code
- Add InkogTool class with comprehensive security analysis capabilities
- Support for token bombing, prompt injection, recursive tool calling detection
- Multi-format output: JSON, text, HTML, SARIF
- Compliance mapping to EU AI Act, NIST AI RMF, OWASP LLM Top 10
- Support for 21+ frameworks (LangChain, CrewAI, AutoGen, etc.)
- Deep scan with AI orchestrator analysis
- MCP server security auditing
- Skill package vulnerability scanning
- Comprehensive test suite and documentation
- CI/CD integration examples

Fixes MervinPraison/PraisonAI#1887
2026-07-09 13:32:24 +00:00
MervinPraison f21fca919d Release v0.2.31 2026-07-09 13:31:47 +00:00
Mervin Praison 70d8bc9bf6 Merge pull request #40 from MervinPraison/claude/issue-1716-20260522
feat: Add NEXUS Prediction Market Tool Integration
2026-07-09 14:30:43 +01:00
MervinPraison da6ce8387c fix(merge-gate): add claude-code-action and restore after PR checkout
Assess was failing because Tools had no local action, and checkout of
the PR branch would drop it even when present. Stash/restore from main.
2026-07-09 14:30:10 +01:00
Mervin Praison 1e08305c80 fix(release): stop failing on gitignored uv.lock (#49)
git add uv.lock exits 1 under bash -e and aborts the release before
publish. Only commit pyproject.toml (uv.lock is intentionally ignored).
2026-07-09 13:52:13 +01:00
Mervin Praison d915d18961 fix(release): align version with PyPI and skip Release/Bump loops (#48)
* fix(release): align version with PyPI and skip Release/Bump loops

Sync pyproject to 0.2.30 (already on PyPI) so the next package change
can patch-release. Skip nightly gate on Release/Bump commits like monorepo.

* fix(release): skip Release/Bump commits in nightly gate
2026-07-09 10:25:03 +01:00
MervinPraisonandMervin Praison dd7fe0efa4 fix(swarmscore): derive discovery manifest URL from domain root via urlparse
Replace fragile split("/v1/") host derivation in get_discovery_manifest
with urlparse-based origin (scheme+netloc), so .well-known resolves to the
domain root (RFC 8615) regardless of version prefix (/v1/, /v2/, /api/, none).
Adds a parametrized test covering multiple base-URL shapes.

Co-authored-by: Mervin Praison <[email protected]>
2026-07-09 09:05:32 +00:00
Mervin Praison 45c8c1d98f fix(merge-gate): raise Tools auto-merge size limit via gate-config (#47)
Tool + tests + docs PRs routinely exceed the 800-line UI default.
Make limits config-driven (2000 lines) and stop treating pyproject.toml
as a hard sensitive path so optional tool deps can auto-merge.
2026-07-09 10:03:27 +01:00
18b4b0c4d4 fix(merge-gate): clear stale conflict block when PR is mergeable (#46)
* fix(merge-gate): clear stale conflict block when PR is mergeable

Ignore old merge-conflict @claude triggers once HEAD is after the
conflict comment and merge state is CLEAN/UNSTABLE, and accept owner
rebase-complete comments so manual recoveries unblock auto-merge.

* fix(merge-gate): guard reduce with null seed in conflict bypass

Address Greptile review: the CLEAN/UNSTABLE conflict-clear bypass used
.reduce() without an initial value and a length>0 guard that could
silently fall through to `return true`. Seed reduce with null and add an
explicit null check so the non-empty invariant is explicit and safe if a
future refactor changes it.

Co-authored-by: Mervin Praison <[email protected]>

---------

Co-authored-by: MervinPraison <[email protected]>
Co-authored-by: Mervin Praison <[email protected]>
2026-07-09 03:09:02 +01:00
MervinPraisonandMervin Praison f404b2e903 fix(nexus): resolve Greptile P1 issues in exports and tests
- Remove get_market_data / check_arbitrage_opportunities from tool_map
  and __all__ in tools/__init__.py; they are instance methods, not
  module-level names, so importing them raised AttributeError and
  polluted the package namespace with generic identifiers.
- Fix two failure-path tests that raised a plain Exception (never caught
  by the tool). Now raise requests.exceptions.ConnectionError and a
  proper HTTPError with response.status_code=402 to exercise the real
  error-handling branches.

Co-authored-by: Mervin Praison <[email protected]>
2026-07-08 22:32:09 +00:00
MervinPraisonandMervin Praison 94a0167653 fix(swarmscore): use ToolResult.output in docs/example; export in __all__
- Replace all result.data references with result.output in docs and example
  (ToolResult exposes .output, not .data)
- Add SwarmScoreTool and standalone functions to tools __all__ for wildcard imports

Co-authored-by: Mervin Praison <[email protected]>
2026-07-08 22:32:06 +00:00
2d79dcbb01 feat: add JIRA watch tools for issue and project monitoring (#43)
* feat: add JIRA watch tools for issue and project monitoring

Port JIRA watch functionality from PraisonAI PR #1875 to the tools package
where external integrations belong.

Co-authored-by: Cursor <[email protected]>

* Stub optional jira module in watch tool unit tests for CI.

* Allow module stub before import in JIRA watch tests.

* fix(jira): sanitize inputs, safe timestamp comparison, guard None fields

- Validate since_timestamp and issue_key to prevent JQL injection / abuse
- Compare timestamps as tz-aware datetimes instead of lexicographic strings
- Guard None derefs on history/comment author, creator, reporter
- Fix recent-activity count/display mismatch and description truncation suffix
- Harden env-isolated auth test; add since_timestamp + validation tests

Co-authored-by: praisonai-triage-agent[bot] <praisonai-triage-agent[bot]@users.noreply.github.com>

* fix(jira): detect all in-window changelog entries; isolate URL test

- jira_watch_issue: filter changelog by since_dt over the full history
  instead of slicing to the last 3 entries first (fixes silently dropped
  in-window field changes); cap display to 10 with a total count.
- test_connection_missing_url: clear JIRA_URL via monkeypatch to avoid
  CI flakiness when the var is set.
- Add test asserting all in-window field changes are reported.

Co-authored-by: Mervin Praison <[email protected]>

---------

Co-authored-by: Cursor <[email protected]>
Co-authored-by: MervinPraison <[email protected]>
Co-authored-by: praisonai-triage-agent[bot] <praisonai-triage-agent[bot]@users.noreply.github.com>
Co-authored-by: Mervin Praison <[email protected]>
2026-07-08 23:27:31 +01:00
b8d64b432c feat: Composio agent-callable tools (#44)
* feat: port Composio agent-callable tools from core SDK

Add composio_tools(), composio_list_apps(), ComposioTools class,
[composio] optional extra, tests, and README usage docs.

* fix(composio): honor explicit api_key and use v3 toolkits API

- _check_composio_available() now accepts an explicit api_key and uses it
  as a fallback, fixing a broken contract where a key passed to
  composio_tools()/ComposioTools was silently ignored.
- Availability check requires the `composio` package only, removing the
  false-positive composio_openai branch that led to silent ImportError.
- get_tools() maps to the Composio v3 SDK (`toolkits`, required `user_id`)
  for the new client while preserving legacy ComposioToolSet behaviour.
- Remove redundant availability check inside _get_client().
- list_apps() handles string app entries defensively.

Co-authored-by: praisonai-triage-agent[bot] <praisonai-triage-agent[bot]@users.noreply.github.com>

* fix(composio): never use api_key as user_id fallback in v3 SDK

The API key is an auth credential, not an entity identifier. Fall back
to "default" only. Add test for explicit user_id and correct the v3
call assertion.

Co-authored-by: Mervin Praison <[email protected]>

---------

Co-authored-by: MervinPraison <[email protected]>
Co-authored-by: praisonai-triage-agent[bot] <praisonai-triage-agent[bot]@users.noreply.github.com>
Co-authored-by: Mervin Praison <[email protected]>
2026-07-08 22:45:35 +01:00
MervinPraisonandMervin Praison 65a53de573 fix: align SwarmScore tool with ToolResult API and resolve rebase
- Use ToolResult(output=..., success=..., metadata=...) instead of
  non-existent data/message kwargs
- Call super().__init__() for BaseTool registration
- Derive discovery manifest URL from api_base_url instead of hardcoding
- Remove unused urllib.parse.urljoin import
- Add swarmscore optional-dependency group (requests) to match docs
- Fix tests to mock real requests.exceptions namespace and read result.output

Co-authored-by: Mervin Praison <[email protected]>
2026-07-08 13:41:23 +00:00
MervinPraisonandpraisonai-triage-agent[bot] 8a47aff40a feat: add NEXUS prediction market tool integration (fixes MervinPraison/PraisonAI#1716)
- Added NexusPredictionMarketTool for live Kalshi/Polymarket data access
- Supports free market data queries and paid arbitrage analysis
- Includes comprehensive examples and documentation
- Full test coverage with mocked API responses
- Integrated with PraisonAI agent workflow patterns

Features:
- Real-time prediction market data from NEXUS API
- Arbitrage opportunity detection (paid feature: /bin/bash.02 USDC)
- Agent metadata and capabilities discovery
- Source tracking via X-NEXUS-Source header
- Error handling for network/payment issues

API Endpoints:
- Free: /kalshi?market={market_name}
- Paid: /arb/check?markets={market1,market2}
- Validation: https://agentic.market/validate

Co-authored-by: praisonai-triage-agent[bot] <praisonai-triage-agent[bot]@users.noreply.github.com>
2026-07-08 14:39:21 +01:00
MervinPraisonandpraisonai-triage-agent[bot] 6c25336a1b feat: add SwarmScore integration tool for PraisonAI (fixes MervinPraison/PraisonAI#1593)
Add SwarmScore trust rating integration to PraisonAI-Tools:

- SwarmScoreTool class for loading and verifying agent trust scores
- Standalone utility functions for easy integration
- Comprehensive documentation and examples
- Complete test coverage
- Integration with PraisonAI agent workflows

SwarmScore provides portable, cryptographically signed trust ratings
based on verified execution history and success rates.

Co-authored-by: praisonai-triage-agent[bot] <praisonai-triage-agent[bot]@users.noreply.github.com>
2026-07-08 13:38:01 +00:00
cc7a3de286 Add automation pipeline (merge gate, CI, PyPI release) (#45)
* Add PraisonAIUI-style automation pipeline for Tools repo.

Merge gate, pipeline status, CI failure Claude, nightly release gate,
PyPI release workflow, and repo-specific gate-config; replace legacy CI workflows.

* Fix ruff auto-fixes and relax CI lint for legacy debt.

Apply safe ruff fixes; ignore E701/E722/F401/F402 until incremental cleanup.
New CI runs unit tests only (matches merge gate expectations).

* Skip optional Playwright video tests in CI.

Motion graphics unit tests require playwright browsers not installed on runners.

* fix(ci): correct release ordering and CodeRabbit trigger string

- pypi-release: commit/tag/push git before publishing to PyPI so a git
  failure never strands a version on PyPI with no matching tag/release
- pypi-release: scope tag force-push to the new tag (git push origin
  "${TAG}" --force) instead of destructive `git push --tags -f`
- auto-pr-comment: revert trigger from 'summarize by coderabbit' to
  'summary by coderabbit' to match CodeRabbit's actual summary body

Co-authored-by: Mervin Praison <[email protected]>

---------

Co-authored-by: MervinPraison <[email protected]>
Co-authored-by: Mervin Praison <[email protected]>
2026-07-08 14:35:43 +01:00
MervinPraisonandCursor d3a5279fad fix(ci): repair invalid expression in claude.yml STEP 5
Escaped jq quotes inside a GitHub Actions expression broke workflow
validation, so @claude issue_comment events never triggered.

Co-authored-by: Cursor <[email protected]>
2026-06-28 06:00:29 +01:00
MervinPraisonandCursor edba25d635 fix(ci): use GH_TOKEN PAT in claude.yml for push and @claude triggers
Replace default GITHUB_TOKEN with synced GH_TOKEN secret. Add PR review
comment triggers and fix fork check for review events.

Co-authored-by: Cursor <[email protected]>
2026-06-28 05:40:53 +01:00
MervinPraison b80deb9a55 fix(ci): use GH_TOKEN PAT so @claude comment triggers claude.yml 2026-06-24 09:02:28 +01:00
MervinPraison 58978a7260 fix(ci): grant pull-requests write for merge-conflict Claude comments 2026-06-24 09:00:42 +01:00
MervinPraison 24292e367d ci: auto-trigger Claude on merge conflicts across PRs
Add merge-conflict-claude workflow, check_fork with GraphQL merge_state, and PR conflict rebase prompts in claude.yml.
2026-06-24 08:58:37 +01:00
Rakshith RamprakashandClaude Opus 4.8 ca809f7f4d fix(firecrawl): coerce crawl/search limit to int (LLM string inputs)
LLMs/agents pass `limit` as a string ("10") or None; without coercion
the `data[:limit]` / `web[:limit]` slices raise TypeError. Coerce with a
try/except fallback to the method default. Addresses gemini-code-assist
review on PR #41.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
2026-06-05 12:48:05 +05:30
Rakshith RamprakashandClaude Opus 4.8 a5d92e8cb4 feat(firecrawl): register firecrawl_search helper
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
2026-06-04 14:49:31 +05:30
Rakshith RamprakashandClaude Opus 4.8 ca4267be06 feat(firecrawl): add search() to the tool + firecrawl_search helper (v2 /search)
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
2026-06-04 14:49:28 +05:30
rramprakashandClaude Opus 4.8 4b03908c11 Upgrade Firecrawl tool to firecrawl-py v2 SDK (>=4)
Migrate FirecrawlTool from the legacy v1 SDK to the v2 SDK while
preserving the tool's public surface and return shapes.

- Import `Firecrawl` instead of `FirecrawlApp`; client is `Firecrawl(api_key=...)`.
- scrape: `client.scrape(url, formats=[...])` (kwargs, not a params dict).
  v2 returns a typed `Document`, so read `.markdown` via attribute access
  and coerce the typed `DocumentMetadata` to a plain dict via `model_dump`.
- crawl: `client.crawl(url, limit=..., poll_interval=...)` returns a typed
  `CrawlJob`; iterate `.data` (list of `Document`) and read
  `.metadata.source_url` / `.metadata.title` via attribute access.
- run() dispatcher and the module-level `firecrawl_scrape` helper are
  unchanged; scrape() still returns a dict and crawl() still returns a
  list[dict] so agents relying on these shapes keep working.

No pin change: firecrawl-py is not pinned in pyproject.toml (runtime-only
optional install); install guidance updated to `firecrawl-py>=4`.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
2026-06-04 14:41:46 +05:30
MervinPraison 2bf7bb372f fix: add layer validation to Claude workflow (repo-scoped routing) 2026-06-03 09:42:50 +01:00
MervinPraison c53f1c443f bump: version 0.2.29 2026-05-08 09:18:28 +01:00
MervinPraison e8d2f175d6 bump: version 0.2.28 2026-05-08 09:14:22 +01:00
Mervin Praison c1b48165f8 feat(linear-tool): add update_issue/add_comment/list_cycles/list_issue_states + OAuth Bearer (#31)
* feat(linear-tool): add update_issue/add_comment/list_cycles/list_issue_states + OAuth Bearer support

- Extend LinearTool with 4 methods needed for full agent workflows: update_issue,
  add_comment, list_cycles, list_issue_states (matches AgentSession-driven use cases)
- Add OAuth 2.0 token support via LINEAR_OAUTH_TOKEN env var; OAuth tokens get the
  required 'Bearer ' prefix while personal API keys remain raw (per Linear's API spec).
  Personal API key takes precedence when both are configured
- Wire all new methods into the run() action dispatcher (incl. forwarding 'title'
  for update_issue since run() binds it)
- 36 unit tests covering auth header logic, GraphQL request shape, all CRUD
  operations, error paths, and the action dispatcher

Avoids duplicating this tool inside praisonaiagents core SDK (architectural fit:
SaaS HTTP wrappers belong in praisonai-tools, per AGENTS.md).

* fix(test): correct copy-paste typo in motion-graphics team test (unblock CI)

Line 82 asserted 'code exploration specialist' was in researcher.instructions
but the description belongs to code_explorer. This has been failing on main
since commit 6d0d510 and blocks all PR CI.
2026-05-05 07:48:17 +01:00
Mervin Praison eaa9292c27 fix(motion-graphics): agent tool wiring + ffmpeg frame clipping (v0.2.26) (#29)
End-to-end agent loop now works with real LLMs (OpenAI gpt-4o-mini
verified). Caught by running the Example 03 agent factory live against
gpt-4o-mini and capturing the full failure chain. Unit tests previously
mocked these paths so the bugs were invisible.

Three real bugs fixed:

1. Tool registration (agent.py)
   Passing class instances (FileTools(), RenderTools()) as tools caused the
   OpenAI adapter to log 'Tool ... not recognized' and the agent to skip
   tool calls entirely. Now we expose individual bound methods:
     - file_tools.read_file / write_file / list_files
     - lint_composition / render_composition  (sync wrappers, see #2)

2. Async tools in sync agent path (agent.py)
   RenderTools.lint_composition / render_composition are async but the
   Agent sync path does not await coroutines. Result:
     'Object of type coroutine is not JSON serializable'
   Fix: wrap each async tool with a local sync function that uses
   asyncio.run(...). Bytes are also stripped from the render_composition
   return (unserializable, and the file already lives at output_path).

3. Odd-height screenshots break libx264 (backend_html.py)
   page.screenshot(full_page=True) captured 1920x1167 for the LLM-authored
   composition (SVG overflowed viewport). libx264 rejects odd height:
     'height not divisible by 2 (1920x1167)'
   Fix: clip to the exact 1920x1080 viewport via
     page.screenshot(clip={x:0,y:0,width:1920,height:1080})

Tests updated (test_motion_graphics_agent.py):
  - MockFileTools now exposes read_file/write_file/list_files methods
  - tool-count assertions updated from 2 class instances to 5 callables

Verification (live, not mocked):

  PRAISONAI_AUTO_APPROVE=true MOTION_LLM=gpt-4o-mini \
    python examples/python/video/03_motion_graphics_agent_factory.py

  Produces:
    index.html  (LLM-authored, 1031 bytes)
    intro.mp4   (1920x1080 H.264 yuv420p, 30fps, 1.5s, 13.5KB)

87/87 unit tests pass. Version bump 0.2.25 -> 0.2.26.
2026-04-19 01:31:09 +01:00
Mervin Praison ddc77bc373 fix(motion-graphics): correct FileTools import path + constructor (v0.2.25) (#28)
Post-merge fixes for #27 caught via live end-to-end run of motion_graphics_team:

- agent.py / team.py: import FileTools from praisonaiagents.tools.file_tools
  (not praisonaiagents.tools). FileTools is not exported via the tools
  package __getattr__, so 'from praisonaiagents.tools import FileTools'
  raises ImportError in production and causes motion_graphics_team() and
  create_motion_graphics_agent() to both fail with the generic
  'praisonaiagents not available' ImportError.

- agent.py: drop the base_dir kwarg when instantiating FileTools — the
  class takes no constructor args (methods are bound directly).

- team.py: also catch AttributeError alongside ImportError so lazy
  attribute failures fall back cleanly to None stubs.

- pyproject.toml: bump version 0.2.24 -> 0.2.25

Unit tests continue to pass because they mock out praisonaiagents. The
bug only surfaces in live runs. Follow-up: add one non-mocked smoke test
that actually instantiates motion_graphics_team().
2026-04-19 01:15:54 +01:00
Mervin Praison 6d0d510c8e feat: motion-graphics video pipeline (HTML/GSAP → MP4 + GitTools + RenderBackendProtocol) (#27)
Closes #26. Adds programmatic motion-graphics pipeline under praisonai_tools/video/motion_graphics/ with RenderBackendProtocol, HtmlRenderBackend (Playwright + ffmpeg), create_motion_graphics_agent factory, motion_graphics_team preset, and GitTools toolkit. Zero changes to praisonaiagents core SDK. 87/87 unit tests pass.
2026-04-18 17:44:00 +01:00
08fed26832 feat: implement langextract tools for interactive text visualization (#25)
* feat: implement langextract tools for interactive text visualization (fixes #24)

- Add langextract_extract and langextract_render_file tools
- Support for highlighting text extractions in interactive HTML
- Lazy imports with graceful degradation when langextract not installed
- Security warnings for file operations
- Comprehensive test suite with mocking
- Usage example with agent integration patterns
- Optional dependency langextract>=0.1.0 in pyproject.toml

Co-authored-by: Mervin Praison <[email protected]>

* fix: correct AnnotatedDocument kwarg from id= to document_id= (fixes #25)

- Fix runtime TypeError in langextract tool
- AnnotatedDocument constructor expects document_id= not id=
- Tests now pass: 16 passed, 0 failed

Co-authored-by: Mervin Praison <[email protected]>

* fix: correct AnnotatedDocument kwarg from intervals= to extractions= (fixes langextract API)

- Changed _create_annotated_document to build Extraction objects instead of CharInterval objects
- Updated AnnotatedDocument constructor to use extractions= kwarg instead of intervals=
- Each Extraction now properly wraps CharInterval as char_interval attribute
- Added extraction metadata (class, text, attributes) as required by langextract API
- All 16 tests now pass (previously 1 failing)

Co-authored-by: Mervin Praison <[email protected]>

* fix: correct langextract tool visualization and test bugs

- Fix Bug 1: lx.visualize() now receives JSONL path instead of HTML path
  - Use temporary JSONL file, capture HTML return value, write to output file
  - Follows correct langextract API pattern
- Fix Bug 2: test_agent_usage_pattern now asserts success instead of impossible 'not installed' branch
- Fix test mocks: lx.visualize now returns string instead of Mock object

Fixes #25

Co-authored-by: Mervin Praison <[email protected]>

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Mervin Praison <[email protected]>
2026-04-18 17:14:36 +01:00
MervinPraison 4ef6193987 fix(ci): post GH_TOKEN @claude comment on new issues; drop duplicate label trigger
Made-with: Cursor
2026-04-18 08:44:32 +01:00
MervinPraison 27c4d2d573 fix(ci): align Claude Assistant with PraisonAIUI; use GH_TOKEN for chained @claude/@copilot comments
Made-with: Cursor
2026-04-18 08:42:59 +01:00
MervinPraison 3c698b48f1 fix: keep Claude action auth on GITHUB_TOKEN
Use GH_TOKEN only for auto issue label chaining, while keeping claude-code-action authentication on GITHUB_TOKEN to avoid bad-credentials failures.

Made-with: Cursor
2026-04-17 10:47:21 +01:00
MervinPraison ed683ed3ca fix: use GH_TOKEN for Claude issue automation
Align issue auto-label and Claude action authentication with the PraisonAI setup so issue-triggered Claude workflows reliably chain and execute.

Made-with: Cursor
2026-04-17 10:46:28 +01:00
cf7ddbf19e fix: Replace non-existent /execute endpoint with webhook-based execution in N8nWorkflowTool (#23)
* fix: Replace non-existent /execute endpoint with webhook-based execution in N8nWorkflowTool

- Replace POST /api/v1/workflows/{id}/execute with webhook trigger execution
- Add _get_webhook_url method to fetch workflow and find webhook trigger nodes
- Automatically activate workflows if inactive
- Add proper error handling for missing webhook triggers
- Update unit tests to mock workflow fetch + webhook execution pattern
- Remove obsolete _wait_for_execution method

Fixes #22

Co-authored-by: Mervin Praison <[email protected]>

* fix: Update timeout test assertion to match actual error message

The test was expecting 'timed out after 5.0 seconds' but the actual
implementation returns 'Request timed out'. Updated assertion to match
the real error message.

Co-authored-by: Mervin Praison <[email protected]>

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Mervin Praison <[email protected]>
2026-04-16 21:02:12 +01:00
4ad5166034 fix: Fix n8n integration test mocking issues (#21)
- Fix test_n8n_workflow_missing_httpx to use sys.modules approach for lazy import mocking
- Enhance mock_httpx fixture with proper HTTPStatusError and TimeoutException classes
- All 14 tests now pass (12 passed, 2 integration tests correctly skipped)

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Mervin Praison <[email protected]>
2026-04-16 18:41:35 +01:00
442a0d71dd feat: Add n8n workflow automation integration (#19)
* feat: add n8n workflow automation integration (fixes MervinPraison/PraisonAI#1396)

- Add N8nWorkflowTool class for executing n8n workflows
- Add n8n_workflow and n8n_list_workflows decorator functions
- Provides access to 400+ integrations (Slack, Gmail, Notion, databases, APIs)
- Includes comprehensive unit tests with mocking
- Add example usage and documentation
- Optional httpx dependency with lazy loading
- Bidirectional PraisonAI ↔ n8n integration support

* fix: resolve critical n8n integration issues

- Export n8n_list_workflows from __init__.py (fixes ImportError)
- Fix module path in tools registry to use absolute imports
- Update __getattr__ to handle absolute module paths
- Fix timeout parameter not being used in _wait_for_execution
- Update test mocking to work with lazy imports

Addresses P0 and P1 issues identified in code review.

Co-authored-by: Mervin Praison <[email protected]>

---------

Co-authored-by: MervinPraison <[email protected]>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Mervin Praison <[email protected]>
2026-04-16 15:44:16 +01:00
MervinPraison 4820ffdd79 ci: align Claude workflow with triage-safe triggers and mandatory PR step
- Gate issue_comment on @claude; only claude label triggers label events
- Resolve issue/PR number in env; single BRANCH; gh pr create --base/--head
- GitHub Actions bot identity, gh auth, GH_TOKEN for gh; MCP issue tools

Made-with: Cursor
2026-04-16 13:26:48 +01:00
tlkc888-JenkinsandClaude Opus 4.5 1ff799a871 fix: parse Joy Trust API response correctly (#18)
* fix: parse Joy Trust API response correctly

The /agents/discover endpoint returns {agents: [...]} not top-level fields.
This fix extracts the agent from the array before reading trust_score.

Fixes #17

* fix: remove fallback-to-first-result (security risk)

Greptile correctly identified that falling back to the first search
result when no exact name match is found could return a different
agent's trust score, allowing untrusted agents to pass verification.

Now only exact name matches are accepted.

* fix: import time at top level and normalize response schema

- Move 'import time' to top of file (fixes NameError on cache check)
- Normalize response schema across success/error paths
- All responses now include: agent_id, vouch_count, capabilities, tier, badges

* Address code review feedback

- Use idiomatic next() with generator expression for agent lookup
- Fix ImportError response to use consistent schema with new fields

Co-Authored-By: Claude Opus 4.5 <[email protected]>

* Remove redundant time import from try block

time is already imported at module level (line 37)

Co-Authored-By: Claude Opus 4.5 <[email protected]>

* Handle null trust_score from API

Use 'or' operator to handle both missing key AND null value,
preventing TypeError on comparison and security bypass.

Co-Authored-By: Claude Opus 4.5 <[email protected]>

* Handle null agents array from API

Use 'or' operator to handle both missing key AND null value,
preventing TypeError when iterating and security bypass.

Co-Authored-By: Claude Opus 4.5 <[email protected]>

* Security: Fail closed on all error paths

Error handlers now always set meets_threshold=False instead of
using fallback_on_error config. In trust-gating context, errors
should deny handoffs, not silently allow them.

Changes:
- meets_threshold: False (was self.config.fallback_on_error)
- fallback_used: True (indicates error occurred)
- logger.exception() for unexpected errors (includes stack trace)

Co-Authored-By: Claude Opus 4.5 <[email protected]>

* Fix misleading fallback_used flag in error handlers

Set fallback_used=False so _get_handoff_recommendation returns
"Handoff not recommended" instead of misleading "Proceed with caution"
message when errors occur.

Co-Authored-By: Claude Opus 4.5 <[email protected]>

* Handle null name and string trust_score edge cases

- Use (a.get("name") or "").lower() to handle {"name": null}
- Safely convert trust_score to float with try/except
- Return fail-closed response for invalid trust_score format
- Use 'or' pattern consistently for null-safe field access

Co-Authored-By: Claude Opus 4.5 <[email protected]>

* fix: Change default min_score from 3.0 to 1.5

The Joy Trust Network has:
- Network average: 1.11
- Network median: 1.10
- Network max: 2.40
- Agents >= 3.0: 0

A default of 3.0 blocks 100% of agents. Changed to 1.5
which is Joy's recommended 'standard' threshold.

Joy now returns trust_context.recommended_thresholds:
- permissive: 1.0 (low-risk tasks)
- standard: 1.5 (general use) <- new default
- moderate: 2.0 (established agents)
- strict: 2.5 (high security)

Co-Authored-By: Claude Opus 4.5 <[email protected]>

---------

Co-authored-by: Claude Opus 4.5 <[email protected]>
2026-04-16 13:12:50 +01:00
1275ca4792 feat: enhance Joy Trust integration for secure agent handoffs (fixes MervinPraison/PraisonAI#1324) (#16)
Enhanced Joy Trust tool with native integration features:

- Added automatic handoff trust verification
- Environment variable configuration (PRAISONAI_TRUST_PROVIDER=joy)
- @trust_verified_handoff decorator for secure delegation
- Handoff safety verification with recommendations
- Configurable trust score thresholds and caching
- Fallback handling for robust operation
- Comprehensive examples and documentation
- Updated tests for new functionality

Addresses feature request for native Joy Trust integration to make
trust verification automatic and free for PraisonAI users.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-authored-by: MervinPraison <[email protected]>
Co-authored-by: praisonai-triage-agent[bot] <praisonai-triage-agent[bot]@users.noreply.github.com>
2026-04-09 12:40:09 +01:00
MervinPraison 324d3458d9 merge: resolve conflict, keep marketplace + social media tools 2026-04-08 21:50:48 +01:00
MervinPraison c140915b0b ci: add Codecov coverage reporting to test workflow 2026-04-08 21:49:44 +01:00
08c2ad811b feat: add marketplace tools - Pinchwork, AgentID, Joy Trust (#15)
* feat: add marketplace tools - Pinchwork, AgentID, Joy Trust (fixes #1292)

Implements external agent-to-agent marketplace integrations as @tool plugins:

- Pinchwork: Task delegation to agent marketplace
- AgentID: Identity verification via ECDSA certificates
- Joy Trust: Trust score verification before delegation

Features:
- Lazy imports with graceful error handling
- Optional dependencies (httpx>=0.24.0)
- @tool decorators for agent discovery
- Comprehensive error handling and type hints
- Real agentic tests for LLM integration

Per AGENTS.md architecture: Tools are external integrations
in PraisonAI-Tools repository with protocol-driven design.

Co-authored-by: Mervin Praison <[email protected]>

* feat: add AgentFolio behavioral trust tools

- Add check_behavioral_trust for task-scoped trust scoring
- Add verify_task_delegation_safety for comprehensive checks
- Support SATP protocol for cross-organizational reputation
- Include proper error handling and lazy httpx imports
- Add tests and documentation following marketplace patterns
- Update dependencies with agentfolio optional extra

Co-authored-by: Mervin Praison <[email protected]>

* feat: refactor marketplace tools to follow BaseTool pattern

- Add proper BaseTool class implementations for all marketplace tools
- Implement constructor with api_key parameter and env var fallbacks
- Add lazy imports with proper error handling
- Export tools in tools/__init__.py for discoverability
- Maintain both class-based and standalone function interfaces
- Follow established LumaLabs tool pattern for consistency

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-authored-by: Mervin Praison <[email protected]>

* refactor: eliminate marketplace/tools duplication - consolidate to single source of truth

- Remove duplicate marketplace/ directory (~500 lines of duplicated code)
- Remove redundant test_standalone.py script
- Update imports to use consolidated tools/ implementations
- Maintain backward compatibility via praisonai_tools/__init__.py exports
- Follow DRY principles with single BaseTool implementation
- Preserve comprehensive error handling and environment variable support

Fixes code duplication issue identified in PR review.

Co-authored-by: Mervin Praison <[email protected]>

---------

Co-authored-by: MervinPraison <[email protected]>
Co-authored-by: Mervin Praison <[email protected]>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-04-08 17:16:59 +01:00
MervinPraison b790db45f5 chore: bump version to 0.2.24 (HeyGen + Higgsfield tools) 2026-03-30 18:44:31 +01:00
MervinPraison b8cfbb24e1 Merge remote-tracking branch 'origin/claude/issue-12-20260330-1710'
# Conflicts:
#	praisonai_tools/tools/__init__.py
2026-03-30 18:43:21 +01:00
MervinPraison 02ae96e859 chore: bump version to 0.2.23 (HeyGen tool) 2026-03-30 18:42:46 +01:00
1e5ba9ad94 feat: Add HeyGen AI video generation tool (#13)
Implement HeyGen tool with REST API integration for AI avatar video generation:
- Add HeyGenTool class with lazy import pattern
- Implement four core functions: list_avatars, list_voices, generate_video, video_status
- Follow existing tool patterns with env var API key handling
- Add proper error handling and validation
- Export all functions in __init__.py

Fixes #11

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Mervin Praison <[email protected]>
2026-03-30 18:40:17 +01:00
MervinPraison b77c1b945a ci: fix Copilot bot username in chain-claude workflow 2026-03-30 18:33:45 +01:00
MervinPraison 8ada66a9d8 ci: fix Claude workflow to ensure PR creation succeeds
- Added explicit git/gh steps in direct_prompt
- Added GH_TOKEN fallback for gh pr create
- Added detailed implementation instructions matching tool patterns
2026-03-30 18:23:17 +01:00
github-actions[bot]andMervin Praison 081cbdd887 feat: Add Higgsfield AI video/image generation tool
- Implement HiggsfieldTool with three main functions:
  - higgsfield_generate: AI image/video generation
  - higgsfield_status: Check generation status
  - higgsfield_upload: Upload files for image-to-video
- Support both HF_KEY and HF_API_KEY/HF_API_SECRET auth methods
- Follow existing patterns with lazy import of higgsfield-client SDK
- Add proper error handling and validation
- Update __init__.py to expose new tools

Fixes #12

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Mervin Praison <[email protected]>
2026-03-30 17:13:33 +00:00
MervinPraison e51d945fcc ci: add AI code review pipeline (Claude + Copilot + reviewers)
Workflows:
- auto-issue-comment.yml: adds 'claude' label on new issues
- claude.yml: Claude analyzes issues and creates PRs
- auto-pr-comment.yml: triggers Copilot after Coderabbit/Qodo
- chain-claude-after-copilot.yml: Claude final review after Copilot

Sequential chain: Issue → Claude fix → PR → Qodo/Coderabbit →
Copilot → Claude final review → ready to merge
2026-03-30 18:10:29 +01:00
Mervin Praison 7ca5d56e9b chore(pyproject): bump version to 0.2.22
feat(tests): load environment variables from .env file in test setup

Added functionality to load environment variables from a .env file for integration tests, enhancing configuration flexibility. Updated version in pyproject.toml.
2026-03-17 13:24:25 +00:00
Mervin Praison f6fe1a82a5 feat(tools): extend Gmail and Jira tools with additional methods and actions
Added new methods to Gmail and Jira tools for enhanced functionality, including
email archiving, labeling, and various Jira board and issue operations.
Refactored GmailTool action handling for clearer logic and improved
email message header retrieval. Expanded test coverage for new features.
2026-03-17 13:18:42 +00:00
Mervin Praison 258233c275 Merge pull request #10 from mavdol/add-capsule-sandboxed-code-execution
Add CapsuleTool for secure, sandboxed Python code execution.
2026-03-16 20:29:32 +00:00
Mervin Praison a7cd49dc53 Merge pull request #9 from streacy/add-nightmarket-tool
Add Nightmarket API discovery and calling tool
2026-03-16 20:28:58 +00:00
Mervin Praison 3190004dc0 Merge pull request #8 from streacy/add-crowpay-tool
Add CrowPay payment tool for AI agents
2026-03-16 20:28:32 +00:00
Mavdol 80850cdede Migrate CapsuleTool to capsule_adapter, add preload action, and implement a singleton instance. (#10) 2026-03-13 22:16:08 +01:00
Mavdol 765c48864a Add CapsuleTool for secure, sandboxed Python code execution. 2026-03-12 11:44:27 +01:00
streacy dbb586d890 Refactor: extract _make_request helper, fix imports and error handling
- Add _make_request() to centralize key check, requests import, HTTP call, and error handling
- Replace all duplicated boilerplate in action methods with _make_request() calls
- Move urllib.parse.quote to top-level import (PEP 8)
- Replace broad Exception catches with specific RequestException + ValueError
- Fix return type hint: remove str from Union
2026-03-06 21:59:15 +00:00
streacy 448abce30b Address review feedback: fix imports, types, error handling, and naming
- Move urllib.parse.quote to top-level import (PEP 8)
- Remove unused in-method quote imports
- Fix return type hint: remove str from Union
- Replace broad Exception catches with (RequestException, ValueError)
- Fix requests_lib -> requests for consistent naming throughout
- Fix _authorize_payment signature to match
2026-03-06 21:58:11 +00:00
Stephanieandgemini-code-assist[bot] 09e5a23d32 Update praisonai_tools/tools/nightmarket_tool.py
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2026-03-06 13:56:32 -08:00
Stephanieandgemini-code-assist[bot] 12ea3bd0b2 Update praisonai_tools/tools/nightmarket_tool.py
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2026-03-06 13:56:24 -08:00
Stephanieandgemini-code-assist[bot] 801ba178ec Update praisonai_tools/tools/nightmarket_tool.py
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2026-03-06 13:56:01 -08:00
streacy d6af47c9e7 Add Nightmarket API discovery and calling tool 2026-03-06 21:27:06 +00:00
streacy 7c61fce8d2 Add CrowPay payment tool for AI agents 2026-03-06 21:26:46 +00:00
Mervin Praison ebc77ec671 fix: correctly extract tasks and agents from dictionary or list structures for observability input data. 2026-02-12 11:38:15 +00:00
Mervin Praison fe82f00038 feat: Implement PraisonAI branding for observability traces and add a CLI for diagnostics and verification. 2026-02-12 11:19:05 +00:00
Mervin PraisonandCursor 9e24c99af5 Bump version to 0.2.18
Co-authored-by: Cursor <[email protected]>
2026-02-12 08:04:50 +00:00
Mervin Praison 9770a5af51 u 2026-02-12 07:39:27 +00:00
Mervin Praison 0575cb7467 Comprehensive SDK related docs - TypeScript (143+ JS pages), Rust (88 pages, 100% coverage) 2026-02-05 17:21:24 +00:00
Mervin Praison a8dc67f3da Expand TYPESCRIPT_RELATED_DOCS (40+ keywords) and RUST_RELATED_DOCS (60+ keywords) 2026-02-05 17:17:13 +00:00
Mervin Praison 1dbfee72ca Add RUST_RELATED_DOCS with 18 keyword categories for Rust SDK docs 2026-02-05 17:12:13 +00:00
Mervin Praison 7cf6584df5 Fix TypeScript/Rust github_repo URLs - use main PraisonAI monorepo 2026-02-05 15:42:57 +00:00
Mervin Praison b9e318c76f Add TypeScript/Rust Uses/UsedBy extraction and source link features
- Added extract_ts_function_calls and extract_rust_function_calls for cross-references
- Added source_file/source_line fields to all dataclasses
- Updated TypeScript parser with function call extraction and source info
- Updated Rust parser with workspace_root and source tracking
- Fixed fallback dataclasses in rust_parser.py
2026-02-05 12:15:43 +00:00
Mervin Praison 097f2e5214 fix: Correct TypeScript related docs paths
- Update TYPESCRIPT_RELATED_DOCS to use /docs/sdk/typescript/
- Add mappings for guard, rag, voice, image, research keywords
- Link to actual existing pages (auto-agents, vector-stores, etc.)
2026-02-05 06:05:39 +00:00
Mervin Praison 66b720ea4c feat: Add Uses and Used By cross-reference sections
- Implement FunctionCallVisitor to extract function calls from AST
- Add 'calls' field to MethodInfo and FunctionInfo data classes
- Add two-pass generation: parse all modules first, then generate
- Build cross-reference index for 'Used By' reverse dependencies
- Add render_uses_section and render_used_by_section functions
- Used By shows clickable links to caller documentation pages
2026-02-04 15:32:05 +00:00
Mervin Praison 0f949fb5e3 feat: Add Source links, Notes, and See Also parsing
- Add 'View on GitHub' source links with file path and line number
- Parse Notes/Warning/Caution sections from docstrings
- Parse See Also section for cross-references
- Add source_file and source_line tracking to data classes
- Add github_repo configuration per SDK package
2026-02-04 14:50:23 +00:00
Mervin Praison 506c566a0a feat: Make related docs SDK-specific
- Rust SDK pages link only to /docs/rust/ pages
- TypeScript SDK pages link only to /docs/typescript/ pages
- Python SDK pages link to concepts/guides/features (as main SDK)
- Add package parameter to render_related_section
2026-02-04 13:42:38 +00:00
Mervin Praison 5b930a99e2 feat: Add related documentation to Rust SDK generator
- Add RELATED_DOCS mapping for Rust-specific topics
- Link to Rust guides (quickstart, agent, tools, memory, cli)
- Link to relevant concept pages
- Integrate in module/class/function page generators
2026-02-04 13:31:38 +00:00
Mervin Praison c481e6bb7e feat: Add related documentation section to SDK reference pages
- Add RELATED_DOCS mapping with 40+ keywords linking to guides/concepts/features
- Add get_related_docs() and render_related_section() helper functions
- Integrate related section in class/function/module page renders
- Fix f-string escaping bug for JSX boolean values in ParamField
- Shows up to 5 related docs per page, most relevant first
2026-02-04 13:29:43 +00:00
Mervin Praison 107c4dcf38 feat: Use 'AI Agents Framework' for praisonai wrapper 2026-02-04 13:15:47 +00:00
Mervin Praison 35dacd7867 feat: Add sidebarTitle and SDK suffix for cleaner SEO and navigation
- Add sidebarTitle field for clean sidebar navigation
- Update title suffix to use 'SDK' for clarity
- Use bullet separator (•) instead of pipe
- Keep friendly_title for SEO optimization
2026-02-04 13:09:46 +00:00
Mervin Praison 58e429693b feat: Add Rust documentation generation with dedicated parser and generator scripts, and integrate it into the CLI. 2026-02-04 12:07:43 +00:00
Mervin Praison 09b5bc0476 u 2026-01-29 17:46:04 +00:00
Mervin Praison b8678aade9 u 2026-01-27 18:53:20 +00:00
Mervin Praison c5cd150180 feat: enhance documentation generation by skipping internal methods, improving docstring parsing, and adding orphaned file cleanup. 2026-01-26 14:45:36 +00:00
Mervin Praison 9403eef445 feat: Implement parsing and rendering of class examples, and enhance documentation page layouts with MDX components. 2026-01-26 13:36:07 +00:00
Mervin Praison f8a954b05f feat: enhance docstring parsing to extract return descriptions, exceptions, and examples, and update documentation rendering to use new MDX components for parameters, returns, and exceptions. 2026-01-26 13:31:24 +00:00
Mervin Praison 2e3a0ac6b2 feat: add documentation generator README and enhance lazy import scanning to support _LAZY_GROUPS and TOOL_MAPPINGS patterns. 2026-01-26 13:14:43 +00:00
Mervin Praison be81fc5740 u 6 2026-01-26 12:24:03 +00:00
Mervin Praison 8ccded21f9 u 5 2026-01-26 12:17:26 +00:00
Mervin Praison 354e566244 update 2 2026-01-26 11:10:29 +00:00
Mervin Praison dfcb2e1669 feat: Introduce a unified CLI, refactor documentation generation into a dedicated package, and add a CI workflow for automated documentation builds. 2026-01-26 10:48:12 +00:00
Mervin Praison d23bc9383c u 2026-01-26 09:44:17 +00:00
Mervin Praison 7f1cfd8ebc refactor: Update icon mappings and refactor reference documentation rendering to use improved markdown formatting and icon selection logic. 2026-01-26 09:41:53 +00:00
Mervin Praison af5e121d92 feat: Add Mermaid diagrams and enhance MDX escaping and type annotation sanitization for reference documentation. 2026-01-26 09:25:49 +00:00
Mervin Praison 294f7f46c7 update 2026-01-26 09:14:13 +00:00
Mervin Praison c051a10fe6 Version bump to 0.2.8 and add raw_content support to TavilyTool search method 2026-01-21 14:21:24 +00:00
Mervin Praison 114b8813e5 Version bump to 0.2.7 and add WordPress tools with enhanced SQLite vector store concurrency
- Add WordPressTool, check_wp_duplicate, and create_wp_post to tools module
- Add optional wordpress dependency group with praisonaiwp[ai]>=0.1.0
- Enable WAL mode in SQLite vector store for better concurrent access
- Add connection timeout and busy_timeout pragmas to SQLite connections
- Add optional include parameter to SQLiteVectorTool.get() method to selectively retrieve embeddings
2026-01-20 19:22:39 +00:00
Mervin Praison d8b5d4f9cd Version bump to 0.2.6 and enhance observability features
- Added auto-instrumentation option for the ObservabilityManager to automatically trace Agent and Workflow classes.
- Introduced new methods for wrapping Agent and Workflow class methods to create spans without explicit tracing.
- Updated project version in pyproject.toml.
- Added SQLite Vector Store tools to the tools module for zero dependencies.
2026-01-18 22:30:01 +00:00
Mervin Praison 99a27c9798 Version bump to 0.2.5 and fix FCP tool FCPXML DTD compliance and dialog handling
- Add /Applications/Utilities/CommandPost.app to CommandPost search paths
- Fix FCPXML format element to use FFVideoFormat naming and remove invalid audio attributes per DTD
- Remove invalid src attribute from asset elements (only valid on media-rep)
- Add audio rate mapping to DTD-compliant values (32k, 44.1k, 48k, etc.)
- Move markers from sequence to first asset-clip per DTD requirements
- Reorder asset-clip attributes
2025-12-31 01:48:06 +00:00
Mervin Praison be899f354a Version bump to 0.2.4 2025-12-30 08:14:55 +00:00
Mervin Praison 5a2cc35bea Version bump to 0.2.3 and add 4 new recipe tools: LLM, Vision, Chart, and Email 2025-12-29 16:26:12 +00:00
Mervin Praison 11fa995aaf Version bump to 0.2.2 and use dynamic version from package metadata 2025-12-29 01:55:55 +00:00
Mervin Praison 99ae17f7cb Version bump to 0.2.1 and fix recipe tools import paths with fallback mechanism 2025-12-29 01:51:00 +00:00
Mervin Praison c737381f72 Version bump to 0.2 2025-12-29 01:35:07 +00:00
Mervin Praison d02a004e92 Version bump to 0.1.8 and add video editing tools documentation 2025-12-28 07:25:46 +00:00
Mervin Praison 2be9170464 Version bump to 0.1.7 2025-12-26 19:40:05 +00:00
Mervin Praison b5ddf993f2 Version bump to 0.1.6 2025-12-25 00:29:56 +00:00
Mervin Praison 5e9b2f9d92 Add ready-to-use tools documentation and lazy loading system
Extended README with comprehensive documentation for 9 ready-to-use tools:
- Communication: EmailTool, SlackTool, DiscordTool
- Developer: GitHubTool
- AI/Media: ImageTool, TTSTool, YouTubeTool
- Data: WeatherTool

Implemented lazy loading system in __init__.py to avoid loading dependencies until needed. Updated package description to clarify it provides both base classes for custom tools AND ready-to-use tools for common tasks.
2025-12-24 12:15:07 +00:00
Mervin Praison ddba53e30e Comprehensive documentation: 8 ways to add tools to PraisonAI Agents
Added detailed documentation for all tool integration methods:
1. Plain Python Functions - simplest approach
2. Built-in Tools - from praisonaiagents.tools
3. @tool Decorator - custom tools with metadata
4. BaseTool Class - complex tools with state
5. Pydantic Class with run() - validated tools
6. LangChain Tools - direct integration
7. CrewAI Tools - _run method support
8. MCP Tools - Model Context Protocol servers

Also includes:
- Combining multiple tool types
- Creating distributable tool packages
- Entry points for auto-discovery
- Contributing guidelines
2025-12-19 12:15:19 +00:00
Mervin Praison 479efe6939 Version bump to 0.1.1 2025-12-19 12:10:45 +00:00
Mervin Praison 128467f297 Remove duplicate tools - use praisonaiagents built-in tools instead
- Remove Tavily, Exa, You.com tools (already in praisonaiagents)
- Keep only BaseTool and @tool decorator for custom tool creation
- Update README to clarify package purpose
- Document built-in tools available in praisonaiagents
- Simplify dependencies and tests

This package is now focused on creating CUSTOM tools for praisonaiagents.
2025-12-19 12:06:15 +00:00
Mervin Praison dd31fe219f Add PraisonAI Agents integration documentation
- Document how tools work with praisonai-agents
- Add 4 integration methods: plain functions, class methods, @tool decorator, BaseTool
- Explain automatic schema generation from function signatures
- Show how type hints and docstrings are used
2025-12-19 11:56:27 +00:00
Mervin Praison be857cc01e Initial commit: PraisonAI Tools v0.1.0
A comprehensive toolkit for AI agents providing web search, content extraction, and research capabilities.

Features:
- BaseTool abstract class with schema generation
- @tool decorator for function-based tools
- TavilyTools: search, extract, crawl, map
- ExaTools: search, search_contents, find_similar, answer
- YouTools: search, contents, news, images
- Comprehensive test suite (60 tests)
2025-12-19 11:53:27 +00:00
433 changed files with 80721 additions and 8664 deletions

No files matched your search

+12
View File
@@ -0,0 +1,12 @@
# PraisonAI-Tools environment variables (examples)
#
# Copy to `.env` and fill in the values for the tools you use.
# Never commit real secrets.
# ── Stripe (StripeTool) ─────────────────────────────────────────────
# Stripe secret key. Prefer least-privilege *restricted* keys (rk_...).
# Use test keys (sk_test_/rk_test_) for development and live keys
# (sk_live_/rk_live_) only in production. STRIPE_API_KEY is accepted as a
# fallback if STRIPE_SECRET_KEY is unset.
STRIPE_SECRET_KEY=sk_test_your_key_here
# STRIPE_API_KEY=sk_test_your_key_here
@@ -0,0 +1,290 @@
name: "Claude Code Action"
description: "Run Claude Code in GitHub Actions workflows"
inputs:
anthropic_api_key:
description: "Anthropic API key"
required: false
claude_code_oauth_token:
description: "Claude Code OAuth Token for Claude Pro"
required: false
github_token:
description: "GitHub token for Claude to operate with"
required: false
default: ${{ github.token }}
trigger_phrase:
description: "The trigger phrase to look for in comments, issue/PR bodies, and issue titles"
required: false
default: "@claude"
assignee_trigger:
description: "The assignee username that triggers the action (e.g. @claude). Only used for issue assignment"
required: false
max_turns:
description: "Maximum number of conversation turns Claude can take"
required: false
timeout_minutes:
description: "Timeout in minutes for execution"
required: false
default: "30"
model:
description: "Model to use (provider-specific format required for Bedrock/Vertex)"
required: false
use_bedrock:
description: "Use Amazon Bedrock with OIDC authentication instead of direct Anthropic API"
required: false
default: "false"
use_vertex:
description: "Use Google Vertex AI with OIDC authentication instead of direct Anthropic API"
required: false
default: "false"
allowed_tools:
description: "Additional tools for Claude to use (the base GitHub tools will always be included)"
required: false
default: ""
disallowed_tools:
description: "Tools that Claude should never use"
required: false
default: ""
custom_instructions:
description: "Additional custom instructions to include in the prompt for Claude"
required: false
default: ""
mcp_config:
description: "Additional MCP configuration (JSON string) that merges with the built-in GitHub MCP servers"
required: false
default: ""
claude_env:
description: "Custom environment variables to pass to Claude Code execution (YAML format)"
required: false
default: ""
direct_prompt:
description: "Direct prompt for Claude to execute automatically without needing a trigger (for automated workflows)"
required: false
runs:
using: "composite"
steps:
- name: Install Claude Code
shell: bash
run: npm install -g @anthropic-ai/claude-code
- name: Install GitHub MCP Server
shell: bash
run: |
claude mcp add-json github '{
"command": "docker",
"args": [
"run",
"-i",
"--rm",
"-e",
"GITHUB_PERSONAL_ACCESS_TOKEN",
"ghcr.io/github/github-mcp-server:sha-ff3036d"
],
"env": {
"GITHUB_PERSONAL_ACCESS_TOKEN": "${{ inputs.github_token }}"
}
}'
- name: Extract GitHub Context and Create Prompt
shell: bash
id: prepare_context
env:
# Pass user-controlled inputs via environment variables to prevent script injection (GHSL-2025-093)
EVENT_COMMENT_BODY: ${{ github.event.comment.body }}
EVENT_ISSUE_TITLE: ${{ github.event.issue.title }}
EVENT_ISSUE_BODY: ${{ github.event.issue.body }}
EVENT_ISSUE_NUMBER: ${{ github.event.issue.number }}
EVENT_PR_TITLE: ${{ github.event.pull_request.title }}
EVENT_PR_NUMBER: ${{ github.event.pull_request.number }}
EVENT_REVIEW_BODY: ${{ github.event.review.body }}
EVENT_ASSIGNEE_LOGIN: ${{ github.event.assignee.login }}
INPUT_TRIGGER_PHRASE: ${{ inputs.trigger_phrase }}
INPUT_ASSIGNEE_TRIGGER: ${{ inputs.assignee_trigger }}
INPUT_DIRECT_PROMPT: ${{ inputs.direct_prompt }}
EVENT_ACTION: ${{ github.event.action }}
EVENT_NAME: ${{ github.event_name }}
REPO_NAME: ${{ github.repository }}
run: |
echo "🔍 Extracting GitHub context from event: $EVENT_NAME"
# Function to check for trigger phrase
check_trigger() {
local text="$1"
local trigger="$INPUT_TRIGGER_PHRASE"
if [[ "$text" == *"$trigger"* ]]; then
return 0
fi
return 1
}
# Extract context based on event type
TRIGGER_FOUND="false"
USER_REQUEST=""
CONTEXT_INFO=""
case "$EVENT_NAME" in
"issue_comment")
COMMENT_BODY="$EVENT_COMMENT_BODY"
ISSUE_TITLE="$EVENT_ISSUE_TITLE"
ISSUE_NUMBER="$EVENT_ISSUE_NUMBER"
if check_trigger "$COMMENT_BODY"; then
TRIGGER_FOUND="true"
USER_REQUEST="$COMMENT_BODY"
CONTEXT_INFO="Issue Comment on #$ISSUE_NUMBER: $ISSUE_TITLE"
fi
;;
"pull_request_review_comment")
COMMENT_BODY="$EVENT_COMMENT_BODY"
PR_TITLE="$EVENT_PR_TITLE"
PR_NUMBER="$EVENT_PR_NUMBER"
if check_trigger "$COMMENT_BODY"; then
TRIGGER_FOUND="true"
USER_REQUEST="$COMMENT_BODY"
CONTEXT_INFO="PR Comment on #$PR_NUMBER: $PR_TITLE"
fi
;;
"pull_request_review")
REVIEW_BODY="$EVENT_REVIEW_BODY"
PR_TITLE="$EVENT_PR_TITLE"
PR_NUMBER="$EVENT_PR_NUMBER"
if check_trigger "$REVIEW_BODY"; then
TRIGGER_FOUND="true"
USER_REQUEST="$REVIEW_BODY"
CONTEXT_INFO="PR Review on #$PR_NUMBER: $PR_TITLE"
fi
;;
"issues")
ISSUE_BODY="$EVENT_ISSUE_BODY"
ISSUE_TITLE="$EVENT_ISSUE_TITLE"
ISSUE_NUMBER="$EVENT_ISSUE_NUMBER"
if check_trigger "$ISSUE_TITLE" || check_trigger "$ISSUE_BODY"; then
TRIGGER_FOUND="true"
USER_REQUEST="$ISSUE_BODY"
CONTEXT_INFO="Issue #$ISSUE_NUMBER: $ISSUE_TITLE"
elif [[ "$EVENT_ACTION" == "assigned" && -n "$INPUT_ASSIGNEE_TRIGGER" ]]; then
ASSIGNEE="$EVENT_ASSIGNEE_LOGIN"
if [[ "$ASSIGNEE" == "$INPUT_ASSIGNEE_TRIGGER" ]]; then
TRIGGER_FOUND="true"
USER_REQUEST="$ISSUE_BODY"
CONTEXT_INFO="Issue #$ISSUE_NUMBER assigned to $ASSIGNEE: $ISSUE_TITLE"
fi
fi
;;
esac
# Check for direct prompt override
if [[ -n "$INPUT_DIRECT_PROMPT" ]]; then
TRIGGER_FOUND="true"
USER_REQUEST="$INPUT_DIRECT_PROMPT"
CONTEXT_INFO="Automated GitHub workflow"
fi
if [[ "$TRIGGER_FOUND" != "true" ]]; then
echo "❌ No trigger phrase found or direct prompt provided. Exiting gracefully."
echo "SKIP_EXECUTION=true" >> $GITHUB_ENV
exit 0
fi
echo "✅ Trigger found! Context: $CONTEXT_INFO"
# Create comprehensive prompt
mkdir -p /tmp/claude-action
cat > /tmp/claude-action/github-context-prompt.txt << EOF
You are Claude Code, an AI assistant helping with GitHub workflows and code.
Repository: $REPO_NAME
Context: $CONTEXT_INFO
Event: $EVENT_NAME
User Request:
$USER_REQUEST
Please analyze the request and provide helpful assistance.
CRITICAL INSTRUCTION: If your task involves modifying codebase files to add a feature or fix a bug:
STEP 0 — SETUP GIT IDENTITY:
git config user.name "MervinPraison"
git config user.email "[email protected]"
STEP 1 — READ GUIDELINES:
MANDATORY: Before writing any code, you MUST carefully read "src/praisonai-agents/AGENTS.md" to understand this repository's extremely specific Architecture, Protocol-First design rules, and Agent-Centric Philosophy.
STEP 2 — ARCHITECTURE VALIDATION & ROUTING (MANDATORY before writing code):
Before implementing anything, answer these questions:
- CORE vs WRAPPER vs TOOLS ROUTING:
1. Core SDK (praisonaiagents/): Only core protocols, base classes, decorators. No heavy implementations.
2. Wrapper (praisonai/): CLI, heavy implementations, optional dependencies.
3. Tools (PraisonAI-Tools): Specific integrations, external tools (e.g. SurrealDB, Slack), and community extensions MUST go to the `MervinPraison/PraisonAI-Tools` repository. If the feature is a tool, clone `https://github.com/MervinPraison/PraisonAI-Tools`, implement it there, and create the PR using `gh pr create -R MervinPraison/PraisonAI-Tools`.
- Does it duplicate existing functionality? Check if Agent already supports this via existing params (reflection, planning, tools, hooks, memory).
- Does it inherit from Agent properly? New agent types MUST inherit Agent, not wrap it with composition.
- Does it add new dependencies? Only optional deps allowed, must be lazy-imported.
- Will agent.py grow larger? If the change adds >50 lines to agent.py, find a way to extract instead.
- Is there a name collision with existing exports in __init__.py?
If ANY of these conceptual checks fail (excluding routing), add a comment to the issue explaining why and close it. Do NOT create a PR.
STEP 3 — IMPLEMENT:
- Create a new git branch appropriately named (e.g. \`feature/issue-<number>\`)
- Follow protocol-driven design: protocols in core SDK, heavy implementations in wrapper
- Keep changes small and backward-compatible
STEP 4 — TEST:
- Run: cd src/praisonai-agents && PYTHONPATH=. python -m pytest tests/ -x -q --timeout=30
- Ensure no regressions
STEP 5 — CREATE PR:
- Commit with descriptive message, push the branch
- Use the GitHub CLI (\`gh pr create --title "..." --body "Fixes #<number>"\`) to open a Pull Request
CRITICAL: You MUST create the PR automatically using \`gh pr create\`. Do NOT just provide a link.
Respond naturally and helpfully to the user's request using the available tools.
EOF
echo "PROMPT_FILE=/tmp/claude-action/github-context-prompt.txt" >> $GITHUB_ENV
echo "SKIP_EXECUTION=false" >> $GITHUB_ENV
- name: Run Claude Code
if: env.SKIP_EXECUTION != 'true'
shell: bash
run: |
echo "🚀 Running Claude Code with GitHub context..."
# Build command arguments
CMD_ARGS=("-p" "--verbose" "--output-format" "stream-json")
# Add max turns if specified
if [[ -n "${{ inputs.max_turns }}" ]]; then
CMD_ARGS+=("--max-turns" "${{ inputs.max_turns }}")
fi
# Add allowed tools (include GitHub tools by default)
TOOLS="mcp__github__get_issue,mcp__github__get_issue_comments,mcp__github__update_issue,mcp__github__search_issues,mcp__github__list_issues,mcp__github__create_comment,Read,Write,Edit,Bash"
if [[ -n "${{ inputs.allowed_tools }}" ]]; then
TOOLS="$TOOLS,${{ inputs.allowed_tools }}"
fi
CMD_ARGS+=("--allowedTools" "$TOOLS")
# Add disallowed tools
if [[ -n "${{ inputs.disallowed_tools }}" ]]; then
CMD_ARGS+=("--disallowedTools" "${{ inputs.disallowed_tools }}")
fi
echo "📝 Executing Claude Code with prompt from file..."
# Execute Claude Code with timeout, using stdin for the prompt
TIMEOUT_SECONDS=$((${{ inputs.timeout_minutes }} * 60))
timeout $TIMEOUT_SECONDS claude "${CMD_ARGS[@]}" < "${{ env.PROMPT_FILE }}"
echo "✅ Claude Code execution completed"
env:
ANTHROPIC_API_KEY: ${{ inputs.anthropic_api_key }}
CLAUDE_CODE_OAUTH_TOKEN: ${{ inputs.claude_code_oauth_token }}
GITHUB_TOKEN: ${{ inputs.github_token }}
@@ -0,0 +1,91 @@
name: "Claude Issue Triage Action"
description: "Automatically triage GitHub issues using Claude Code"
inputs:
timeout_minutes:
description: "Timeout in minutes for execution"
required: false
default: "5"
claude_code_oauth_token:
description: "Claude Code OAuth token for Pro subscription access"
required: true
github_token:
description: "GitHub token with repo and issues permissions"
required: true
runs:
using: "composite"
steps:
- name: Checkout repository code
uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Create prompt
id: generate_prompt
shell: bash
run: |
mkdir -p /tmp/claude-prompts
cat > /tmp/claude-prompts/claude-issue-triage-prompt.txt << 'EOF'
You're an issue triage assistant for GitHub issues. Your task is to analyze the issue and select appropriate labels from the provided list.
IMPORTANT: Don't post any comments or messages to the issue. Your only action should be to apply labels.
Issue Information:
- REPO: ${{ github.repository }}
- ISSUE_NUMBER: ${{ github.event.issue.number }}
TASK OVERVIEW:
1. First, fetch the list of labels available in this repository by running: `gh label list`. Run exactly this command with nothing else.
2. Next, use the GitHub tools to get context about the issue:
- You have access to these tools:
- mcp__github__get_issue: Use this to retrieve the current issue's details including title, description, and existing labels
- mcp__github__get_issue_comments: Use this to read any discussion or additional context provided in the comments
- mcp__github__update_issue: Use this to apply labels to the issue (do not use this for commenting)
- mcp__github__search_issues: Use this to find similar issues that might provide context for proper categorization and to identify potential duplicate issues
- mcp__github__list_issues: Use this to understand patterns in how other issues are labeled
- Start by using mcp__github__get_issue to get the issue details
3. Analyze the issue content, considering:
- The issue title and description
- The type of issue (bug report, feature request, question, etc.)
- Technical areas mentioned
- Severity or priority indicators
- User impact
- Components affected
4. Select appropriate labels from the available labels list provided above:
- Choose labels that accurately reflect the issue's nature
- Be specific but comprehensive
- Select priority labels if you can determine urgency (high-priority, med-priority, or low-priority)
- Consider platform labels (android, ios) if applicable
- If you find similar issues using mcp__github__search_issues, consider using a "duplicate" label if appropriate. Only do so if the issue is a duplicate of another OPEN issue.
5. Apply the selected labels:
- Use mcp__github__update_issue to apply your selected labels
- DO NOT post any comments explaining your decision
- DO NOT communicate directly with users
- If no labels are clearly applicable, do not apply any labels
IMPORTANT GUIDELINES:
- Be thorough in your analysis
- Only select labels from the provided list above
- DO NOT post any comments to the issue
- Your ONLY action should be to apply labels using mcp__github__update_issue
- It's okay to not add any labels if none are clearly applicable
EOF
echo "PROMPT<<DELIMITER" >> $GITHUB_OUTPUT
cat /tmp/claude-prompts/claude-issue-triage-prompt.txt >> $GITHUB_OUTPUT
echo "DELIMITER" >> $GITHUB_OUTPUT
- name: Run Claude Code
uses: ./.github/actions/claude-code-action
with:
direct_prompt: ${{ steps.generate_prompt.outputs.PROMPT }}
allowed_tools: "Bash(gh label list),mcp__github__get_issue,mcp__github__get_issue_comments,mcp__github__update_issue,mcp__github__search_issues,mcp__github__list_issues"
timeout_minutes: ${{ inputs.timeout_minutes }}
claude_code_oauth_token: ${{ inputs.claude_code_oauth_token }}
github_token: ${{ inputs.github_token }}
+9
View File
@@ -0,0 +1,9 @@
{
"templateRepo": "MervinPraison/github-automation-template",
"stack": "claude",
"profile": "tools",
"mode": "hub",
"hubRef": "v1.1.15",
"templateVersion": "1.1.6",
"installedAt": "2026-09-07T12:54:25Z"
}
+4
View File
@@ -0,0 +1,4 @@
# Final architecture review prompt (referenced by merge-gate.js via gate-config fields).
# Most FINAL review text is assembled dynamically from gate-config.js at runtime.
See `finalClaudeScope`, `finalClaudeProductValue` in `.github/scripts/gate-config.js`.
@@ -0,0 +1,34 @@
#!/usr/bin/env node
const chain = require('./bot-pr-review-chain.js');
let failed = 0;
function assert(name, cond) {
if (!cond) {
console.error('FAIL:', name);
failed++;
} else {
console.log('ok:', name);
}
}
assert('chainKickPosted requires both markers', chain.chainKickPosted([
{ user: { login: 'MervinPraison' }, body: '@coderabbitai review' },
{ user: { login: 'MervinPraison' }, body: '/review' },
]));
assert('chainKickPosted false with only coderabbit', !chain.chainKickPosted([
{ user: { login: 'MervinPraison' }, body: '@coderabbitai review' },
]));
assert('coderabbitKickPosted detects coderabbit', chain.coderabbitKickPosted([
{ user: { login: 'MervinPraison' }, body: '@coderabbitai review' },
]));
assert('qodoKickPosted detects qodo', chain.qodoKickPosted([
{ user: { login: 'github-actions[bot]' }, body: '/review' },
]));
assert('ignores coderabbit bot comment', !chain.coderabbitKickPosted([
{ user: { login: 'coderabbitai[bot]' }, body: '@coderabbitai review' },
]));
assert('bot PR author match', chain.isBotOpenedPr({ user: { login: 'praisonai-triage-agent[bot]' } }));
assert('generic bot type match', chain.isBotOpenedPr({ user: { login: 'my-bot', type: 'Bot' } }));
assert('human author excluded', !chain.isBotOpenedPr({ user: { login: 'MervinPraison', type: 'User' } }));
process.exit(failed ? 1 : 0);
+153
View File
@@ -0,0 +1,153 @@
/**
* Idempotent CodeRabbit/Qodo kick for bot-opened PRs.
* @see .github/workflows/auto-pr-comment.yml, claude.yml, bot-pr-recovery.yml
*/
const KICK_AUTHORS = new Set(['MervinPraison', 'github-actions[bot]']);
const BOT_PR_AUTHORS = new Set(['praisonai-triage-agent[bot]', 'github-actions[bot]']);
async function listAllComments(github, owner, repo, issueNumber) {
if (typeof github.paginate === 'function') {
return github.paginate(github.rest.issues.listComments, {
owner,
repo,
issue_number: issueNumber,
per_page: 100,
});
}
const { data } = await github.rest.issues.listComments({
owner,
repo,
issue_number: issueNumber,
per_page: 100,
});
return data;
}
function kickAuthored(comment, marker) {
return (
KICK_AUTHORS.has(comment.user?.login) &&
(comment.body || '').includes(marker)
);
}
function coderabbitKickPosted(comments) {
return comments.some((c) => kickAuthored(c, '@coderabbitai review'));
}
function qodoKickPosted(comments) {
return comments.some((c) => kickAuthored(c, '/review'));
}
function chainKickPosted(comments) {
return coderabbitKickPosted(comments) && qodoKickPosted(comments);
}
function isBotOpenedPr(pr) {
if (BOT_PR_AUTHORS.has(pr.user?.login)) return true;
return pr.user?.type === 'Bot';
}
async function kickReviewChain(github, owner, repo, prNumber, core, preFetchedComments = null) {
const comments = preFetchedComments || await listAllComments(github, owner, repo, prNumber);
const needCoderabbit = !coderabbitKickPosted(comments);
const needQodo = !qodoKickPosted(comments);
if (!needCoderabbit && !needQodo) {
core?.info?.(`Review chain already kicked on PR #${prNumber}`);
return { kicked: false, reason: 'already_kicked' };
}
if (needCoderabbit) {
await github.rest.issues.createComment({
owner,
repo,
issue_number: prNumber,
body: '@coderabbitai review',
});
}
if (needQodo) {
await github.rest.issues.createComment({
owner,
repo,
issue_number: prNumber,
body: '/review',
});
}
core?.info?.(`Kicked review chain for PR #${prNumber} (coderabbit=${needCoderabbit}, qodo=${needQodo})`);
return { kicked: true, coderabbit: needCoderabbit, qodo: needQodo };
}
async function findOpenPrForIssue(github, owner, repo, issueNumber) {
const prefix = `claude/issue-${issueNumber}-`;
const { data: prs } = await github.rest.pulls.list({
owner,
repo,
state: 'open',
sort: 'created',
direction: 'desc',
per_page: 30,
});
return (
prs.find(
(p) => (p.head?.ref || '').startsWith(prefix) && isBotOpenedPr(p)
) || null
);
}
async function kickReviewChainForIssue(github, owner, repo, issueNumber, core) {
const pr = await findOpenPrForIssue(github, owner, repo, issueNumber);
if (!pr) {
core?.info?.(`No open PR for issue #${issueNumber}, skipping review kick`);
return { kicked: false, reason: 'no_pr' };
}
const result = await kickReviewChain(github, owner, repo, pr.number, core);
return { ...result, prNumber: pr.number };
}
async function recoverStalledBotPrs(github, owner, repo, options, core) {
const { prNumber = null, minAgeMs = 10 * 60 * 1000, maxRecover = 10 } = options || {};
let prs;
if (prNumber) {
const { data } = await github.rest.pulls.get({ owner, repo, pull_number: prNumber });
prs = [data];
} else if (typeof github.paginate === 'function') {
prs = await github.paginate(github.rest.pulls.list, {
owner,
repo,
state: 'open',
per_page: 100,
});
} else {
const { data } = await github.rest.pulls.list({ owner, repo, state: 'open', per_page: 100 });
prs = data;
}
const cutoff = Date.now() - minAgeMs;
let recovered = 0;
for (const pr of prs) {
if (recovered >= maxRecover) break;
if (!isBotOpenedPr(pr)) continue;
if (!prNumber && new Date(pr.created_at).getTime() > cutoff) continue;
const comments = await listAllComments(github, owner, repo, pr.number);
if (chainKickPosted(comments)) continue;
await kickReviewChain(github, owner, repo, pr.number, core, comments);
recovered += 1;
}
core?.info?.(`Recovery complete (${recovered} PR(s) kicked)`);
return recovered;
}
module.exports = {
KICK_AUTHORS,
BOT_PR_AUTHORS,
listAllComments,
chainKickPosted,
coderabbitKickPosted,
qodoKickPosted,
isBotOpenedPr,
kickReviewChain,
findOpenPrForIssue,
kickReviewChainForIssue,
recoverStalledBotPrs,
};
@@ -0,0 +1,35 @@
#!/usr/bin/env node
/**
* Run: node .github/scripts/ci-failure-claude-selftest.js
*/
const ciFix = require('./ci-failure-claude.js');
const mergeGate = require('./merge-gate.js');
const config = require('./gate-config.js');
let failed = 0;
function assert(name, cond) {
if (!cond) {
console.error('FAIL:', name);
failed += 1;
} else {
console.log('ok:', name);
}
}
const LOG = `
python UNKNOWN STEP FAILED (0.0100s) tests/unit/test_example.py::test_foo - AssertionError: bar
python UNKNOWN STEP ##[error]Process completed with exit code 1.
`;
const parsed = ciFix.parsePytestFailures(LOG);
assert('parses pytest failure', parsed.length === 1);
assert('uses CI workflow list', config.ciFailureWorkflowRuns.includes('CI'));
const comment = ciFix.buildCiFixComment({
headSha: 'abc1234567890abcdef1234567890abcdef12',
failedChecks: [{ name: 'python', workflow: 'CI', html_url: 'https://example.com/job/1' }],
failureSummaries: [{ jobName: 'python', failures: parsed }],
});
assert('comment mentions product guardrails', comment.includes('Product guardrails'));
process.exit(failed ? 1 : 0);
+294
View File
@@ -0,0 +1,294 @@
/**
* Post @claude comments with CI failure details for internal PRs.
* @see .github/workflows/ci-failure-claude.yml
*/
const mergeGate = require('./merge-gate.js');
const config = require('./gate-config.js');
const CI_FIX_LABEL = 'claude-ci-fix-pending';
const COOLDOWN_MS = 12 * 60 * 60 * 1000;
const AUTO_ACTORS = mergeGate.AUTO_ACTORS;
const MAX_FAILURES = 15;
const MAX_JOBS_TO_FETCH = 5;
function shortSha(headSha) {
return (headSha || '').slice(0, 8);
}
function ciFixShaMarker(headSha) {
return `ci failed on head \`${shortSha(headSha)}`.toLowerCase();
}
function isCiFixComment(comment) {
const body = (comment.body || '').toLowerCase();
return body.includes('@claude') && body.includes('ci failed on head');
}
function hasCiFixCommentForSha(comments, headSha) {
const marker = shortSha(headSha).toLowerCase();
return comments.some((c) => {
if (!AUTO_ACTORS.includes(c.user.login)) return false;
if (!isCiFixComment(c)) return false;
return (c.body || '').toLowerCase().includes(marker);
});
}
function hasRecentCiFixComment(comments, headSha) {
const cutoff = Date.now() - COOLDOWN_MS;
const marker = shortSha(headSha).toLowerCase();
return comments.some((c) => {
if (!AUTO_ACTORS.includes(c.user.login)) return false;
if (!isCiFixComment(c)) return false;
const body = (c.body || '').toLowerCase();
if (body.includes(marker)) return true;
return new Date(c.created_at).getTime() > cutoff;
});
}
function shouldSkipCiFix({ comments, headSha, labels, hasFinal, claudeInProgress, failedChecks }) {
if (!failedChecks.length) return { skip: true, reason: 'no failed checks' };
if (labels.includes(CI_FIX_LABEL)) return { skip: true, reason: 'ci fix pending' };
if (!hasFinal) return { skip: true, reason: 'awaiting final claude review trigger' };
if (claudeInProgress) return { skip: true, reason: 'claude in progress' };
if (hasCiFixCommentForSha(comments, headSha)) {
return { skip: true, reason: 'already commented for this sha' };
}
if (hasRecentCiFixComment(comments, headSha)) {
return { skip: true, reason: 'recent ci fix comment cooldown' };
}
return { skip: false };
}
function parsePytestFailures(logText) {
if (!logText) return [];
const lines = logText.split('\n');
const failures = [];
const seen = new Set();
for (let i = lines.length - 1; i >= 0 && failures.length < MAX_FAILURES; i -= 1) {
const line = lines[i].trim();
if (!line) continue;
const pytestMatch = line.match(
/FAILED\s*(?:\([^)]+\))?\s+(tests\/[^\s]+(?:::[^\s-]+)*)\s*-\s*(.+)/i
);
if (pytestMatch) {
const testId = pytestMatch[1];
if (!seen.has(testId)) {
seen.add(testId);
failures.push({ testId, error: pytestMatch[2].trim() });
}
continue;
}
const errorMatch = line.match(/##\[error\](.+)/);
if (errorMatch && /tests\//.test(errorMatch[1])) {
const msg = errorMatch[1].trim();
if (!seen.has(msg)) {
seen.add(msg);
failures.push({ testId: msg, error: msg });
}
}
}
return failures;
}
async function fetchJobFailureSummary(github, owner, repo, jobId, jobName) {
try {
const response = await github.rest.actions.downloadJobLogsForWorkflowRun({
owner,
repo,
job_id: jobId,
});
let logText = typeof response.data === 'string' ? response.data : String(response.data || '');
if (logText.length > 100000) {
logText = logText.slice(-100000);
}
return { jobName, jobId, failures: parsePytestFailures(logText) };
} catch (err) {
return { jobName, jobId, failures: [], error: err.message };
}
}
function mapFailedChecks(runs) {
return runs.map((run) => ({
name: run.name,
id: run.id,
html_url: run.html_url || run.details_url,
workflow: run.app?.slug || run.check_suite?.app?.slug || run.name,
}));
}
function buildCiFixComment({ headSha, failedChecks, failureSummaries }) {
const parts = [
`@claude CI failed on HEAD \`${shortSha(headSha)}\`. Please fix the failures below and push to this branch.`,
'',
'## Failed checks',
];
for (const check of failedChecks) {
parts.push(`- **${check.workflow || check.name}** / \`${check.name}\` — ${check.html_url}`);
}
parts.push('', '## Failures (extracted)');
let idx = 0;
for (const summary of failureSummaries) {
for (const failure of summary.failures) {
idx += 1;
if (idx > MAX_FAILURES) break;
parts.push(`${idx}. \`${failure.testId}\` — \`${failure.error}\``);
parts.push(` - Job: \`${summary.jobName}\``);
}
if (idx >= MAX_FAILURES) break;
}
if (idx === 0) {
parts.push('_(Could not extract pytest details — see job logs above.)_');
}
const exampleTest = failureSummaries.find((s) => s.failures.length)?.failures[0]?.testId;
parts.push(
'',
'## Critical review first',
'Before changing code or tests, decide **which side is wrong**:',
'1. **Legitimate feature change** — the PR intent is correct but implementation or tests need updating. Preserve product guarantees; update tests only when behaviour intentionally changed and document why.',
'2. **Regression / bug in this PR** — the failure exposes a real breakage introduced here. Fix the implementation; **do not weaken, skip, or delete tests** just to go green.',
'3. **Pre-existing flake or unrelated failure** — say so explicitly; prefer fixing the root cause over masking it.',
'',
'**Product guardrails (AGENTS.md):**',
`- ${config.finalClaudeScope}`,
'- Tests must continue to guard backward compatibility — passing CI by lowering test standards is not acceptable.',
'- If the feature does not genuinely add product value, recommend reverting or narrowing scope instead of patching around failures.',
'',
'## What to do',
'1. State your verdict: **legitimate fix**, **regression fix**, or **needs human review** — and why (1–3 sentences).',
'2. Fix root cause with **minimal changes**; never bloat the Agent class with extra params.',
exampleTest
? `3. Run failing tests locally, e.g. \`pytest ${exampleTest} -q\`, plus any related tests touched by the PR.`
: '3. Run failing tests locally with targeted pytest, plus any related tests touched by the PR.',
'4. Push to this branch and comment: files changed, review verdict, and why tests still protect product behaviour.',
);
return parts.join('\n');
}
async function maybeClearCiFixLabel(github, owner, repo, prNumber, labels, headSha, core) {
if (!labels.includes(CI_FIX_LABEL)) return false;
const ciGreen = await mergeGate.allChecksGreenOnSha(github, owner, repo, headSha, core);
if (!ciGreen) return false;
try {
await github.rest.issues.removeLabel({
owner,
repo,
issue_number: prNumber,
name: CI_FIX_LABEL,
});
core?.info?.(`Removed ${CI_FIX_LABEL} from PR #${prNumber}`);
} catch (err) {
if (err.status !== 404) throw err;
}
return true;
}
async function maybeTriggerCiFixClaude(github, owner, repo, prNumber, core, opts = {}) {
const baseRepo = `${owner}/${repo}`;
const ctx = await mergeGate.loadPrContext(github, owner, repo, prNumber);
const pr = ctx.pr;
if (pr.state !== 'open') return { skipped: true, reason: 'not open' };
if (pr.draft) return { skipped: true, reason: 'draft' };
const headRepo = pr.head.repo?.full_name;
if (headRepo && headRepo !== baseRepo && !pr.maintainer_can_modify) {
return { skipped: true, reason: 'fork without maintainer edits' };
}
const headSha = pr.head.sha;
const labels = ctx.labels;
if (await maybeClearCiFixLabel(github, owner, repo, prNumber, labels, headSha, core)) {
return { skipped: true, reason: 'ci green, label cleared' };
}
const ciGreen = await mergeGate.allChecksGreenOnSha(github, owner, repo, headSha, core);
if (ciGreen) return { skipped: true, reason: 'ci green' };
const runs = await mergeGate.listChecksOnSha(github, owner, repo, headSha);
const failedRuns = mergeGate.listFailedChecksOnSha(runs);
if (failedRuns.length === 0) {
return { skipped: true, reason: 'no failed checks yet (may be pending)' };
}
const hasFinal = mergeGate.hasFinalClaudeReviewTrigger(ctx.comments);
const claudeInProgress = await mergeGate.hasInProgressClaudeAssistant(
github, owner, repo, prNumber
);
const skip = shouldSkipCiFix({
comments: ctx.comments,
headSha,
labels,
hasFinal,
claudeInProgress,
failedChecks: failedRuns,
});
if (skip.skip) return { skipped: true, reason: skip.reason };
const failedChecks = mapFailedChecks(failedRuns);
const failureSummaries = [];
for (const check of failedChecks.slice(0, MAX_JOBS_TO_FETCH)) {
failureSummaries.push(
await fetchJobFailureSummary(github, owner, repo, check.id, check.name)
);
}
const body = buildCiFixComment({ headSha, failedChecks, failureSummaries });
await github.rest.issues.addLabels({
owner,
repo,
issue_number: prNumber,
labels: [CI_FIX_LABEL],
});
await github.rest.issues.createComment({
owner,
repo,
issue_number: prNumber,
body,
});
core?.info?.(`Posted CI fix @claude on PR #${prNumber} (${shortSha(headSha)})`);
return { triggered: true, headSha: shortSha(headSha) };
}
async function processWorkflowRunFailure(github, owner, repo, workflowRun, core) {
if (workflowRun.conclusion !== 'failure') {
return { skipped: true, reason: `conclusion=${workflowRun.conclusion}` };
}
const allowed = config.ciFailureWorkflowRuns;
if (!allowed.includes(workflowRun.name)) {
return { skipped: true, reason: `workflow ${workflowRun.name}` };
}
const prNumber = await mergeGate.resolvePrNumberFromWorkflowRun(
github, owner, repo, workflowRun
);
if (!prNumber) return { skipped: true, reason: 'no linked PR' };
return maybeTriggerCiFixClaude(github, owner, repo, prNumber, core);
}
module.exports = {
CI_FIX_LABEL,
COOLDOWN_MS,
ciFixShaMarker,
isCiFixComment,
hasCiFixCommentForSha,
hasRecentCiFixComment,
shouldSkipCiFix,
parsePytestFailures,
fetchJobFailureSummary,
buildCiFixComment,
maybeClearCiFixLabel,
maybeTriggerCiFixClaude,
processWorkflowRunFailure,
};
+35
View File
@@ -0,0 +1,35 @@
/**
* PraisonAI-Tools gate configuration.
*/
module.exports = {
repoFullName: 'MervinPraison/PraisonAI-Tools',
triggerLogins: ['MervinPraison', 'github-actions[bot]'],
productPathPrefixes: ['praisonai_tools/', 'tests/'],
// Workflows stay gated; pyproject.toml is normal for optional tool deps.
sensitivePathPatterns: [
/^\.github\/workflows\//,
],
// Tool + tests + docs routinely exceed the UI/SDK 800-line default.
maxAutoAdditions: 2000,
maxAutoFiles: 30,
requiredCheckPatterns: [/^ci$/i, /python/i, /test/i, /lint/i, /ruff/i],
ciWorkflowFile: 'ci.yml',
ciWorkflowName: 'CI',
mergeGateWorkflowRuns: ['CI', 'Claude Assistant'],
ciFailureWorkflowRuns: ['CI'],
pypiPackageName: 'praisonai-tools',
packagePaths: ['praisonai_tools', 'pyproject.toml'],
finalClaudeScope:
'SCOPE: Focus ONLY on PraisonAI-Tools (praisonai_tools/, tests). '
+ 'Agent-callable integrations belong here, not praisonaiagents core or PraisonAI-Plugins.',
finalClaudeProductValue:
'4. Product value: tools must be agent-invokable via parameters; lazy imports; '
+ 'no lifecycle hooks or sandbox backends in this repo.',
agentPyChecks: false,
reviewBotLogins: [
'coderabbitai[bot]',
'qodo-code-review[bot]',
'greptile-apps[bot]',
],
};
+15
View File
@@ -0,0 +1,15 @@
/**
* Load per-repo gate configuration (installed as .github/scripts/gate-config.js).
*/
function loadGateConfig() {
try {
return require('./gate-config.js');
} catch (err) {
if (err.code !== 'MODULE_NOT_FOUND') throw err;
throw new Error(
'Missing .github/scripts/gate-config.js — run install.sh from github-automation-template'
);
}
}
module.exports = { loadGateConfig };
+300
View File
@@ -0,0 +1,300 @@
#!/usr/bin/env node
/**
* Local self-test for merge-gate.js heuristics (no GitHub API).
* Run: node .github/scripts/merge-gate-selftest.js
*/
const mg = require('./merge-gate.js');
const config = require('./gate-config.js');
const productSample = `${config.productPathPrefixes[0]}a/b.py`;
const repoShort = config.repoFullName.split('/')[1];
let failed = 0;
function assert(name, cond) {
if (!cond) {
console.error('FAIL:', name);
failed++;
} else {
console.log('ok:', name);
}
}
// Stale FINAL: push after FINAL, no @claude since head
const finals = [
{ user: { login: 'github-actions[bot]' }, body: '@claude FINAL architecture reviewer', created_at: '2026-06-12T08:00:00Z' },
];
assert('stale when head after final', mg.isStaleFinalAfterPush(finals, '2026-06-12T09:00:00Z'));
const withRecovery = [
...finals,
{ user: { login: 'github-actions[bot]' }, body: '@claude FINAL architecture reviewer', created_at: '2026-06-12T09:30:00Z' },
];
assert('not stale when @claude after head', !mg.isStaleFinalAfterPush(withRecovery, '2026-06-12T09:00:00Z'));
const withClaudeReply = [
...finals,
{
user: { login: 'praisonai-triage-agent[bot]' },
body: "**Claude finished @MervinPraison's task** —— [View job](https://github.com/)",
created_at: '2026-06-12T08:30:00Z',
},
];
assert('not stale when Claude replied after FINAL', !mg.isStaleFinalAfterPush(withClaudeReply, '2026-06-12T09:00:00Z'));
assert('claude final reply detected', mg.isClaudeFinalReplyComment(withClaudeReply[1]));
const mervinReply = {
user: { login: 'MervinPraison' },
body: "**Claude finished @MervinPraison's task** —— [View job](https://github.com/)",
created_at: '2026-06-12T08:30:00Z',
};
assert('mervin claude final reply detected', mg.isClaudeFinalReplyComment(mervinReply));
assert(
'not stale when MervinPraison posted Claude finished',
!mg.isStaleFinalAfterPush([...finals, mervinReply], '2026-06-12T09:00:00Z')
);
const conflictRebaseComments = [
{
user: { login: 'MervinPraison' },
body: '@claude You are the FINAL architecture reviewer.',
created_at: '2026-06-12T08:00:00Z',
},
{
user: { login: 'MervinPraison' },
body: '@claude this PR has merge conflicts with `main`.',
created_at: '2026-06-12T08:10:00Z',
},
{
user: { login: 'MervinPraison' },
body: 'Rebase onto main complete. Force-pushed to branch.',
created_at: '2026-06-12T08:20:00Z',
},
];
assert(
'allow stale-FINAL recovery after conflict rebase push',
!mg.shouldSkipStaleFinalRecovery(conflictRebaseComments, '2026-06-12T08:25:00Z').skip
);
assert('cancelled detect-and-trigger does not block', mg.OPTIONAL_CANCELLED_CHECKS.has('detect-and-trigger'));
// Stale-FINAL recovery guards (PR #2560 push loop)
const nowMs = Date.now();
const iso = (ms) => new Date(ms).toISOString();
const firstFinal = {
user: { login: 'MervinPraison' },
body: '@claude You are the FINAL architecture reviewer.',
created_at: iso(nowMs - 20 * 60 * 1000),
};
const pushSoonAfter = mg.shouldSkipStaleFinalRecovery(
[firstFinal],
iso(nowMs - 16 * 60 * 1000)
);
assert('debounce push soon after FINAL', pushSoonAfter.skip && pushSoonAfter.reason.includes('soon after'));
const secondFinal = {
user: { login: 'MervinPraison' },
body: '@claude You are the FINAL architecture reviewer.',
created_at: iso(nowMs - 20 * 60 * 1000),
};
const capped = mg.shouldSkipStaleFinalRecovery(
[firstFinal, secondFinal],
iso(nowMs - 2 * 60 * 1000)
);
assert('hourly cap blocks third FINAL in window', capped.skip && capped.reason.includes('capped'));
const botPush = mg.shouldSkipStaleFinalRecovery(
[firstFinal],
iso(nowMs - 16 * 60 * 1000),
'praisonai-triage-agent[bot]'
);
assert('skip when automation pushed head', botPush.skip && botPush.reason.includes('automation'));
assert('claude automation login', mg.isClaudeAutomationLogin('praisonai-triage-agent[bot]'));
// Bot CHANGES_REQUESTED then APPROVE
const reviews = [
{ user: { login: 'coderabbit[bot]', type: 'Bot' }, state: 'CHANGES_REQUESTED', submitted_at: '2026-06-12T08:00:00Z' },
{ user: { login: 'coderabbit[bot]', type: 'Bot' }, state: 'APPROVED', submitted_at: '2026-06-12T09:00:00Z' },
];
assert('bot approve clears CR', !mg.hasAnyChangesRequested(reviews));
assert('human CR blocks', mg.hasAnyChangesRequested([
{ user: { login: 'MervinPraison', type: 'User' }, state: 'CHANGES_REQUESTED', submitted_at: '2026-06-12T09:00:00Z' },
]));
// Verdict after HEAD
const verdictComments = [
{ body: 'MERGE_GATE_VERDICT: APPROVE', created_at: '2026-06-12T08:00:00Z' },
];
assert('verdict before head rejected', mg.findMergeGateVerdict(verdictComments, null, '2026-06-12T09:00:00Z') === null);
assert('verdict after head accepted', mg.findMergeGateVerdict(
[{ body: 'MERGE_GATE_VERDICT: APPROVE', created_at: '2026-06-12T10:00:00Z' }],
null,
'2026-06-12T09:00:00Z'
) === 'APPROVE');
const fallbackApprove = [
{
body: 'MERGE_GATE_VERDICT: APPROVE\n\nAutomated fallback — Claude assess did not post a verdict comment.',
created_at: '2026-06-12T10:00:00Z',
},
];
assert(
'automated fallback APPROVE ignored when Opus required',
mg.findMergeGateVerdict(fallbackApprove, null, '2026-06-12T09:00:00Z', { excludeAutomatedFallback: true }) === null
);
assert(
'automated fallback APPROVE still visible without Opus-only filter',
mg.findMergeGateVerdict(fallbackApprove, null, '2026-06-12T09:00:00Z') === 'APPROVE'
);
const fallbackBlockNewFormat = [
{
body: 'MERGE_GATE_VERDICT: BLOCK\n\nAutomated fallback — Opus merge gate assessment did not complete.',
created_at: '2026-06-12T10:00:00Z',
},
];
assert(
'new-format fallback BLOCK ignored when Opus required',
mg.findMergeGateVerdict(fallbackBlockNewFormat, null, '2026-06-12T09:00:00Z', { excludeAutomatedFallback: true }) === null
);
assert(
'new-format fallback BLOCK detected by marker helper',
mg.isAutomatedFallbackVerdict(fallbackBlockNewFormat[0].body)
);
assert(
'findMergeGateVerdict null-safe with explicit null options',
mg.findMergeGateVerdict(fallbackApprove, null, '2026-06-12T09:00:00Z', null) === 'APPROVE'
);
const noise = [{ user: { login: 'MervinPraison' }, body: '**Merge gate scan** — wait for `@claude`', created_at: new Date().toISOString() }];
assert('diagnostic comment not a trigger', !mg.hasRecentClaudeTrigger(noise, 35));
// Cooldown skip requires an actual verdict on HEAD, not just a FINAL trigger comment
const recentFinalOnHead = [
{
user: { login: 'MervinPraison' },
body: '@claude You are the FINAL architecture reviewer.',
created_at: '2026-06-27T10:00:00Z',
},
];
assert(
'final trigger alone does not count as verdict on head',
mg.findMergeGateVerdict(recentFinalOnHead, null, '2026-06-27T09:55:00Z') === null
);
const recentVerdictOnHead = [
...recentFinalOnHead,
{
user: { login: 'github-actions[bot]' },
body: 'MERGE_GATE_VERDICT: APPROVE',
created_at: '2026-06-27T10:05:00Z',
},
];
assert(
'verdict on head skips cooldown gate',
mg.findMergeGateVerdict(recentVerdictOnHead, null, '2026-06-27T09:55:00Z') === 'APPROVE'
);
// Sensitive + secrets
assert('workflow path sensitive', mg.sensitivePathReasons([{ filename: '.github/workflows/foo.yml' }]).length === 1);
assert('ci-only label exempts workflows', mg.sensitivePathReasons(
[{ filename: '.github/workflows/foo.yml' }],
[mg.WORKFLOW_ONLY_LABEL]
).length === 0);
assert('ci-only label not exempt mixed changes', mg.sensitivePathReasons(
[{ filename: '.github/workflows/foo.yml' }, { filename: productSample }],
[mg.WORKFLOW_ONLY_LABEL]
).length === 1);
assert('secret in patch', mg.secretScanReasons([{ filename: 'x.py', patch: '+key = "ghp_abcdefghijklmnopqrstuvwxyz1234567890"' }]).length === 1);
// Claude run scoping — other PR branches must not block
assert('other branch claude does not block', !mg.hasBlockingClaudeRunForPr(
[{ event: 'issue_comment', head_branch: 'other-branch' }],
'my-branch'
));
assert('same branch claude blocks', mg.hasBlockingClaudeRunForPr(
[{ event: 'issue_comment', head_branch: 'my-branch' }],
'my-branch'
));
assert('issues event never blocks', !mg.hasBlockingClaudeRunForPr(
[{ event: 'issues', head_branch: 'my-branch' }],
'my-branch'
));
// Conflict rebase clears after bot completion + FINAL on HEAD (within 12h cooldown)
const conflictNowMs = Date.now();
const conflictIso = (offsetMs) => new Date(conflictNowMs + offsetMs).toISOString();
const conflictTrigger = {
user: { login: 'MervinPraison' },
body: '@claude this PR has merge conflicts with main. Please rebase',
created_at: conflictIso(-30 * 60 * 1000),
};
const rebaseDone = {
user: { login: 'praisonai-triage-agent[bot]' },
body: 'Rebase complete — PR #2308 onto latest main',
created_at: conflictIso(-29 * 60 * 1000),
};
const finalAfterRebase = {
user: { login: 'MervinPraison' },
body: '@claude You are the FINAL architecture reviewer.',
created_at: conflictIso(-20 * 60 * 1000),
};
const headAfterRebase = conflictIso(-21 * 60 * 1000);
const rebaseComments = [conflictTrigger, rebaseDone, finalAfterRebase];
assert('conflict blocks before rebase done', mg.hasRecentConflictComment([conflictTrigger], headAfterRebase));
assert('conflict clears after rebase + FINAL on HEAD', !mg.hasRecentConflictComment(rebaseComments, headAfterRebase));
assert('conflict still blocks without FINAL on HEAD', mg.hasRecentConflictComment(
[conflictTrigger, rebaseDone],
headAfterRebase
));
assert(
'CLEAN + head after conflict clears without completion comment',
!mg.hasRecentConflictComment([conflictTrigger], headAfterRebase, 'CLEAN')
);
assert(
'UNSTABLE + head after conflict clears without completion comment',
!mg.hasRecentConflictComment([conflictTrigger], headAfterRebase, 'UNSTABLE')
);
assert(
'DIRTY still blocks even with head after conflict',
mg.hasRecentConflictComment([conflictTrigger], headAfterRebase, 'DIRTY')
);
assert(
'owner rebase-complete comment clears conflict',
!mg.hasRecentConflictComment(
[
conflictTrigger,
{
user: { login: 'MervinPraison' },
body: '**Rebase complete** — conflicts resolved.',
created_at: conflictIso(-29 * 60 * 1000),
},
finalAfterRebase,
],
headAfterRebase
)
);
// Tests heuristic
assert('product code without tests', mg.missingTestsReason([{ filename: productSample, additions: 3 }]) !== null);
assert('product code with tests ok', mg.missingTestsReason([
{ filename: productSample, additions: 3 },
{ filename: 'tests/test_x.py', additions: 10 },
]) === null);
// PR size (threshold from gate-config; Tools allows larger tool+tests+docs PRs)
const sizeLimit = config.maxAutoAdditions ?? 800;
assert('under size limit ok', mg.prSizeReasons([{ additions: sizeLimit }]).length === 0);
assert('large PR blocked', mg.prSizeReasons([{ additions: sizeLimit + 1 }]).length > 0);
assert('typical tool PR under Tools limit', mg.prSizeReasons([{ additions: 1510 }]).length === 0);
assert('internal PR link accepted', mg.isInternalPullRequestLink(
{ base: { repo: { full_name: config.repoFullName } } },
'MervinPraison',
repoShort
));
assert('fork sync PR link rejected', !mg.isInternalPullRequestLink(
{ number: 21, base: { repo: { full_name: `Milkmange/${repoShort}` } } },
'MervinPraison',
repoShort
));
process.exit(failed ? 1 : 0);
+957
View File
@@ -0,0 +1,957 @@
/**
* Shared merge-gate helpers for claude-merge-gate.yml
* @see .github/workflows/claude-merge-gate.yml
*/
const config = require('./gate-config.js');
const CLAUDE_TRIGGER_LOGINS = ['MervinPraison', 'github-actions[bot]'];
const AUTO_ACTORS = CLAUDE_TRIGGER_LOGINS;
const CONFLICT_COOLDOWN_MS = 12 * 60 * 60 * 1000;
const CLAUDE_ACTIVE_MS = 35 * 60 * 1000;
const STALE_FINAL_RECOVERY_WINDOW_MS = 60 * 60 * 1000;
const STALE_FINAL_MAX_PER_WINDOW = 2;
const PUSH_AFTER_FINAL_DEBOUNCE_MS = 15 * 60 * 1000;
const ALLOWED_MERGE_STATES = new Set(['CLEAN', 'UNSTABLE']);
const AGENT_PY_MAX_AUTO_LINES = 100;
const PR_MAX_AUTO_ADDITIONS = config.maxAutoAdditions ?? 800;
const PR_MAX_AUTO_FILES = config.maxAutoFiles ?? 30;
const MANUAL_ONLY_LABELS = new Set(['security', 'breaking-change', 'needs-manual-review', 'release']);
const WORKFLOW_ONLY_LABEL = 'merge-gate-ci-only';
const CI_ONLY_PATH_PREFIXES = ['.github/workflows/', '.github/actions/', '.github/scripts/merge-gate'];
const SDK_PATH_PREFIXES = config.productPathPrefixes;
const SENSITIVE_PATH_PATTERNS = config.sensitivePathPatterns;
const REQUIRED_SDK_CHECK_PATTERNS = config.requiredCheckPatterns;
const SECRET_PATTERNS = [
/sk-[a-zA-Z0-9]{20,}/,
/AKIA[0-9A-Z]{16}/,
/ghp_[a-zA-Z0-9]{36,}/,
/github_pat_/,
/Bearer eyJ[a-zA-Z0-9_-]+\.[a-zA-Z0-9_-]+/,
/-----BEGIN (RSA |EC |OPENSSH )?PRIVATE KEY-----/,
];
const BLOCK_LABELS = new Set([
'claude-conflict-pending',
'claude-merge-gate-active',
'no-auto-merge',
'auto-merged-by-gate',
]);
const MERGE_READY_LABEL = 'pipeline/merge-ready';
/** Superseded concurrency runs; must not block merge when real tests passed. */
const OPTIONAL_CANCELLED_CHECKS = new Set(['detect-and-trigger']);
const BOT_REVIEWER_PATTERNS = [
'coderabbit',
'qodo',
'gemini',
'copilot',
'greptile',
];
const sleep = (ms) => new Promise((r) => setTimeout(r, ms));
function isFinalClaudeTriggerComment(c) {
const body = (c.body || '').toLowerCase();
if (!AUTO_ACTORS.includes(c.user.login)) return false;
if (!body.includes('@claude')) return false;
if (body.includes('merge conflict')) return false;
return body.includes('final architecture reviewer') || body.includes('lead engineer');
}
function isClaudeTriggerNoise(c) {
const body = c.body || '';
if (body.includes('Merge gate scan')) return true;
if (body.includes('MERGE_GATE_VERDICT')) return true;
if (body.includes('Merged by **Claude PR merge gate**')) return true;
return false;
}
function isClaudeFinalReplyComment(c) {
const login = (c.user?.login || '').toLowerCase();
const fromAutomation =
login.includes('praisonai-triage') ||
login.includes('github-actions') ||
AUTO_ACTORS.some((a) => a.toLowerCase() === login);
if (!fromAutomation) return false;
return (c.body || '').includes('Claude finished');
}
function hasRecentClaudeTrigger(comments, minutes = 35) {
const cutoff = Date.now() - minutes * 60 * 1000;
return comments.some((c) => {
if (!CLAUDE_TRIGGER_LOGINS.includes(c.user.login)) return false;
if (isClaudeTriggerNoise(c)) return false;
if (!(c.body || '').includes('@claude')) return false;
return new Date(c.created_at).getTime() > cutoff;
});
}
function isConflictRebaseTriggerComment(c) {
if (!AUTO_ACTORS.includes(c.user.login)) return false;
const body = (c.body || '').toLowerCase();
return body.includes('@claude') && body.includes('merge conflict');
}
function isConflictRebaseCompletionComment(c) {
const login = (c.user?.login || '').toLowerCase();
const allowed =
login.includes('praisonai-triage') ||
login.includes('github-actions') ||
AUTO_ACTORS.some((a) => a.toLowerCase() === login);
if (!allowed) return false;
const body = (c.body || '').toLowerCase();
return (
body.includes('rebase complete') ||
body.includes('rebase onto') ||
body.includes('force-pushed') ||
(body.includes('conflict') && body.includes('resolved'))
);
}
/** HEAD landed after conflict rebase but no FINAL @claude since that push. */
function needsFinalReReviewAfterConflictRebase(comments, headPushedAt) {
if (!headPushedAt) return false;
const headTime = new Date(headPushedAt).getTime();
const conflictTriggers = comments.filter(isConflictRebaseTriggerComment);
if (conflictTriggers.length === 0) return false;
const latestConflict = conflictTriggers.reduce((a, b) =>
new Date(a.created_at) > new Date(b.created_at) ? a : b
);
const conflictTime = new Date(latestConflict.created_at).getTime();
if (headTime <= conflictTime) return false;
const rebaseDone = comments.some(
(c) =>
new Date(c.created_at).getTime() > conflictTime && isConflictRebaseCompletionComment(c)
);
if (!rebaseDone) return false;
const finalsAfterHead = comments.filter(
(c) =>
isFinalClaudeTriggerComment(c) &&
new Date(c.created_at).getTime() >= headTime - 60000
);
return finalsAfterHead.length === 0;
}
function conflictRebaseQuiescent(comments, headPushedAt) {
const conflictTriggers = comments.filter(isConflictRebaseTriggerComment);
if (conflictTriggers.length === 0) return true;
const latestConflict = conflictTriggers.reduce((a, b) =>
new Date(a.created_at) > new Date(b.created_at) ? a : b
);
const conflictTime = new Date(latestConflict.created_at).getTime();
const rebaseDone = comments.some(
(c) =>
new Date(c.created_at).getTime() > conflictTime && isConflictRebaseCompletionComment(c)
);
if (!rebaseDone) return false;
return finalClaudeCompletedOnSha(comments, headPushedAt);
}
function hasRecentConflictComment(comments, headPushedAt = null, mergeStateStatus = null) {
const cutoff = Date.now() - CONFLICT_COOLDOWN_MS;
const hasRecentTrigger = comments.some((c) => {
if (!isConflictRebaseTriggerComment(c)) return false;
return new Date(c.created_at).getTime() > cutoff;
});
if (!hasRecentTrigger) return false;
if (headPushedAt && conflictRebaseQuiescent(comments, headPushedAt)) {
return false;
}
// GitHub mergeable (CLEAN/UNSTABLE) and HEAD pushed after the conflict trigger → resolved
const status = String(mergeStateStatus || '').toUpperCase();
if (headPushedAt && ALLOWED_MERGE_STATES.has(status)) {
// conflictTriggers is guaranteed non-empty here (hasRecentTrigger is true above);
// the null seed keeps reduce() safe if that invariant ever changes.
const conflictTriggers = comments.filter(isConflictRebaseTriggerComment);
const latestConflict = conflictTriggers.reduce(
(a, b) => (a && new Date(a.created_at) > new Date(b.created_at) ? a : b),
null
);
if (
latestConflict &&
new Date(headPushedAt).getTime() > new Date(latestConflict.created_at).getTime()
) {
return false;
}
}
return true;
}
function isBotReviewer(login, userType) {
const lower = (login || '').toLowerCase();
if (userType === 'Bot') return true;
if (lower.endsWith('[bot]')) return true;
return ['coderabbit', 'qodo', 'gemini', 'copilot', 'greptile'].some((p) => lower.includes(p));
}
function latestReviewsByUser(reviews) {
const latestByUser = new Map();
for (const r of reviews) {
const login = r.user?.login;
if (!login) continue;
const prev = latestByUser.get(login);
if (!prev || new Date(r.submitted_at) > new Date(prev.submitted_at)) {
latestByUser.set(login, r);
}
}
return latestByUser;
}
function hasHumanChangesRequested(reviews) {
for (const [login, review] of latestReviewsByUser(reviews)) {
if (review.state !== 'CHANGES_REQUESTED') continue;
if (!isBotReviewer(login, review.user?.type)) return true;
}
return false;
}
function hasAnyChangesRequested(reviews) {
for (const [, review] of latestReviewsByUser(reviews)) {
if (review.state === 'CHANGES_REQUESTED') return true;
}
return false;
}
function hasFinalClaudeReviewTrigger(comments) {
return comments.some(isFinalClaudeTriggerComment);
}
function isStaleFinalAfterPush(comments, headPushedAt) {
if (!headPushedAt) return false;
const headTime = new Date(headPushedAt).getTime();
const finals = comments.filter(isFinalClaudeTriggerComment);
if (finals.length === 0) return true;
const latestFinal = finals.reduce((a, b) =>
new Date(a.created_at) > new Date(b.created_at) ? a : b
);
const finalTime = new Date(latestFinal.created_at).getTime();
if (headTime <= finalTime + 60000) return false;
const claudeRepliedAfterFinal = comments.some((c) => {
if (!isClaudeFinalReplyComment(c)) return false;
return new Date(c.created_at).getTime() >= finalTime - 60000;
});
if (claudeRepliedAfterFinal) return false;
const claudeSinceHead = comments.some((c) => {
if (!CLAUDE_TRIGGER_LOGINS.includes(c.user.login)) return false;
if (isClaudeTriggerNoise(c)) return false;
if (!(c.body || '').includes('@claude')) return false;
return new Date(c.created_at).getTime() >= headTime - 60000;
});
return !claudeSinceHead;
}
function needsStaleFinalRecovery(comments, headPushedAt) {
return (
hasFinalClaudeReviewTrigger(comments) &&
isStaleFinalAfterPush(comments, headPushedAt)
);
}
function shouldSkipFinalRecovery(comments, headPushedAt) {
const isStale = isStaleFinalAfterPush(comments, headPushedAt);
if (isStale) return false;
return hasRecentClaudeTrigger(comments, 35);
}
function countFinalTriggersSince(comments, sinceMs) {
return comments.filter(
(c) => isFinalClaudeTriggerComment(c) && new Date(c.created_at).getTime() > sinceMs
).length;
}
function isClaudeAutomationLogin(login) {
const lower = (login || '').toLowerCase();
return lower.includes('praisonai-triage') || lower === 'github-actions[bot]';
}
function isPushSoonAfterLatestFinal(comments, headPushedAt) {
if (!headPushedAt) return false;
const finals = comments.filter(isFinalClaudeTriggerComment);
if (finals.length === 0) return false;
const latestFinal = finals.reduce((a, b) =>
new Date(a.created_at) > new Date(b.created_at) ? a : b
);
const finalTime = new Date(latestFinal.created_at).getTime();
const headTime = new Date(headPushedAt).getTime();
if (headTime <= finalTime) return false;
return headTime - finalTime < PUSH_AFTER_FINAL_DEBOUNCE_MS;
}
/** Returns { skip: true, reason } when stale-FINAL recovery should not post. */
function shouldSkipStaleFinalRecovery(comments, headPushedAt, headPusherLogin = null) {
if (!isStaleFinalAfterPush(comments, headPushedAt)) {
return { skip: true, reason: 'not stale' };
}
if (headPusherLogin && isClaudeAutomationLogin(headPusherLogin)) {
return { skip: true, reason: 'head pushed by Claude automation' };
}
if (needsFinalReReviewAfterConflictRebase(comments, headPushedAt)) {
return { skip: false, reason: '' };
}
if (isPushSoonAfterLatestFinal(comments, headPushedAt)) {
return {
skip: true,
reason: 'head pushed soon after FINAL (wait for CI / batched fixes)',
};
}
const windowStart = Date.now() - STALE_FINAL_RECOVERY_WINDOW_MS;
const finalsInWindow = countFinalTriggersSince(comments, windowStart);
if (finalsInWindow >= STALE_FINAL_MAX_PER_WINDOW) {
return {
skip: true,
reason: `stale-FINAL capped (${finalsInWindow} FINAL triggers in last hour)`,
};
}
return { skip: false, reason: '' };
}
function finalClaudeCompletedOnSha(comments, headPushedAt) {
if (!hasFinalClaudeReviewTrigger(comments)) return false;
if (isStaleFinalAfterPush(comments, headPushedAt)) return false;
return true;
}
const FINAL_CLAUDE_REVIEW_BODY =
`@claude You are the FINAL architecture reviewer. If the branch is under ${config.repoFullName} (not a fork), you are able to make modifications to this branch and push directly. ${config.finalClaudeScope} Read ALL comments above from Gemini, Qodo, CodeRabbit, and Copilot carefully before responding.\n\n**Phase 1: Review per AGENTS.md**\n1. Protocol-driven: check heavy implementations vs core SDK\n2. Backward compatible: ensure zero feature regressions\n3. Performance: no hot-path regressions (lazy imports, import-time <200ms)\n${config.finalClaudeProductValue}\n\n**Phase 2: FIX Valid Issues**\n5. For any VALID bugs or architectural flaws found by Gemini, CodeRabbit, Qodo, Copilot, or any other reviewer: implement the fix\n6. Push all code fixes directly to THIS branch (do NOT create a new PR)\n7. Comment a summary of exact files modified and what you skipped\n\n**Phase 3: Final Verdict**\n8. If all issues are resolved, approve the PR / close the Issue\n9. If blocking issues remain, request changes / leave clear action items`;
async function getMergeState(github, owner, repo, prNumber) {
const query = `
query($owner: String!, $repo: String!, $number: Int!) {
repository(owner: $owner, name: $repo) {
pullRequest(number: $number) {
mergeStateStatus
maintainerCanModify
isDraft
headRefOid
headRef { repository { nameWithOwner } }
}
}
}
`;
for (let attempt = 0; attempt < 3; attempt++) {
const result = await github.graphql(query, { owner, repo, number: prNumber });
const prGql = result.repository.pullRequest;
const status = (prGql?.mergeStateStatus || '').toUpperCase();
if (status && status !== 'UNKNOWN') {
return {
status,
isDraft: prGql.isDraft,
headRepo: prGql.headRef?.repository?.nameWithOwner,
headSha: prGql.headRefOid,
maintainerCanModify: prGql.maintainerCanModify === true,
};
}
if (attempt < 2) await sleep(10000);
}
const { data: pr } = await github.rest.pulls.get({ owner, repo, pull_number: prNumber });
return {
status: (pr.mergeable_state || '').toUpperCase(),
isDraft: pr.draft,
headRepo: pr.head.repo?.full_name,
headSha: pr.head.sha,
maintainerCanModify: pr.maintainer_can_modify === true,
};
}
async function listChecksOnSha(github, owner, repo, sha) {
const { data } = await github.rest.checks.listForRef({
owner,
repo,
ref: sha,
per_page: 100,
});
return (data.check_runs || []).filter((r) => r.head_sha === sha);
}
function listFailedChecksOnSha(runs) {
return (runs || []).filter((run) => {
if (run.status !== 'completed') return false;
return run.conclusion === 'failure';
});
}
async function allChecksGreenOnSha(github, owner, repo, sha, core) {
const runs = await listChecksOnSha(github, owner, repo, sha);
if (runs.length === 0) {
core?.info?.(`No check runs on ${sha.slice(0, 7)} — allowing (e.g. docs-only PR)`);
return true;
}
for (const run of runs) {
if (run.status !== 'completed') {
core?.info?.(`Check pending: ${run.name} (${run.status})`);
return false;
}
const ok =
['success', 'neutral', 'skipped'].includes(run.conclusion) ||
(run.conclusion === 'cancelled' && OPTIONAL_CANCELLED_CHECKS.has(run.name));
if (!ok) {
core?.info?.(`Check failed: ${run.name} (${run.conclusion})`);
return false;
}
}
return true;
}
function claudeRunBlocksPr(run, headRef) {
if (!run || run.event === 'issues') return false;
if (!headRef) return true;
return run.head_branch === headRef;
}
function hasBlockingClaudeRunForPr(runs, headRef) {
return (runs || []).some((r) => claudeRunBlocksPr(r, headRef));
}
async function hasInProgressClaudeAssistant(github, owner, repo, prNumber = null) {
try {
const { data } = await github.rest.actions.listWorkflowRuns({
owner,
repo,
workflow_id: 'claude.yml',
status: 'in_progress',
per_page: 20,
});
const runs = (data.workflow_runs || []).filter((r) => r.event !== 'issues');
if (runs.length === 0) return false;
if (prNumber == null) return runs.length > 0;
const { data: pr } = await github.rest.pulls.get({ owner, repo, pull_number: prNumber });
const headRef = pr.head.ref;
return hasBlockingClaudeRunForPr(runs, headRef);
} catch {
return false;
}
}
function isCiOnlyChange(files) {
if (!files.length) return false;
return files.every((f) => CI_ONLY_PATH_PREFIXES.some((p) => f.filename.startsWith(p)));
}
function isInternalPullRequestLink(link, owner, repo) {
if (link?.base?.repo?.full_name === `${owner}/${repo}`) return true;
const baseUrl = link?.base?.repo?.url || '';
return baseUrl.endsWith(`/repos/${owner}/${repo}`);
}
async function resolvePrNumberFromLinkedPullRequests(github, owner, repo, linked) {
const repoFull = `${owner}/${repo}`;
const internal = (linked || []).filter((l) => isInternalPullRequestLink(l, owner, repo));
for (const link of internal) {
if (!link.number) continue;
try {
const { data } = await github.rest.pulls.get({
owner,
repo,
pull_number: link.number,
});
if (data.state === 'open' && data.base?.repo?.full_name === repoFull) {
return data.number;
}
} catch (err) {
if (err.status !== 404) throw err;
}
}
return null;
}
async function resolvePrNumberFromHeadBranch(github, owner, repo, branch) {
if (!branch || branch === 'main' || branch === 'master') return null;
const { data } = await github.rest.pulls.list({
owner,
repo,
state: 'open',
head: `${owner}:${branch}`,
per_page: 1,
});
return data[0]?.number || null;
}
async function resolvePrNumberFromWorkflowRun(github, owner, repo, workflowRun) {
const fromLinked = await resolvePrNumberFromLinkedPullRequests(
github, owner, repo, workflowRun.pull_requests
);
if (fromLinked) return fromLinked;
const fromBranch = await resolvePrNumberFromHeadBranch(
github, owner, repo, workflowRun.head_branch
);
if (fromBranch) return fromBranch;
return null;
}
async function listPullFiles(github, owner, repo, prNumber) {
if (typeof github.paginate === 'function') {
return github.paginate(github.rest.pulls.listFiles, {
owner,
repo,
pull_number: prNumber,
per_page: 100,
});
}
const { data } = await github.rest.pulls.listFiles({
owner,
repo,
pull_number: prNumber,
per_page: 100,
});
return data;
}
function countNewAgentParams(patch) {
if (!patch) return 0;
return patch
.split('\n')
.filter((l) => l.startsWith('+') && !l.startsWith('+++'))
.filter((l) => {
const t = l.slice(1).trim();
if (!t || t.startsWith('#') || t.startsWith('"""') || t.startsWith("'''")) return false;
if (/^(def|class|@|return\b|if\b|elif\b|else\b|for\b|while\b)/.test(t)) return false;
if (/^\w+\s*[:=]/.test(t) && !t.startsWith('self.')) return true;
return false;
}).length;
}
function getAgentPyChangeFromFiles(files) {
const agentFile = files.find((f) => f.filename.endsWith('praisonaiagents/agent/agent.py'));
if (!agentFile) {
return { touched: false, additions: 0, newParams: 0 };
}
return {
touched: true,
additions: agentFile.additions || 0,
newParams: countNewAgentParams(agentFile.patch || ''),
};
}
async function getAgentPyChange(github, owner, repo, prNumber) {
const files = await listPullFiles(github, owner, repo, prNumber);
return getAgentPyChangeFromFiles(files);
}
function touchesSdk(files) {
return files.some((f) => SDK_PATH_PREFIXES.some((p) => f.filename.startsWith(p)));
}
function hasManualOnlyLabel(labels) {
return labels.some((l) => MANUAL_ONLY_LABELS.has(l));
}
function sensitivePathReasons(files, labels = []) {
if (labels.includes(WORKFLOW_ONLY_LABEL) && isCiOnlyChange(files)) {
return [];
}
const reasons = [];
for (const f of files) {
if (SENSITIVE_PATH_PATTERNS.some((p) => p.test(f.filename))) {
reasons.push(`sensitive path: ${f.filename}`);
break;
}
}
return reasons;
}
function prSizeReasons(files) {
const reasons = [];
const totalAdditions = files.reduce((sum, f) => sum + (f.additions || 0), 0);
if (totalAdditions > PR_MAX_AUTO_ADDITIONS) {
reasons.push(`PR +${totalAdditions} lines (>${PR_MAX_AUTO_ADDITIONS}) requires manual review`);
}
if (files.length > PR_MAX_AUTO_FILES) {
reasons.push(`${files.length} files changed (>${PR_MAX_AUTO_FILES}) requires manual review`);
}
return reasons;
}
function missingTestsReason(files) {
const productAdds = files.filter(
(f) =>
SDK_PATH_PREFIXES.some((p) => f.filename.startsWith(p)) &&
f.filename.endsWith('.py') &&
!f.filename.endsWith('__init__.py') &&
(f.additions || 0) > 0
);
if (productAdds.length === 0) return null;
const hasTestChange = files.some(
(f) => /\/tests?\//.test(f.filename) || /test_.*\.py$/.test(f.filename) || /_test\.py$/.test(f.filename)
);
if (!hasTestChange) return 'Product code added without test file changes — requires manual review';
return null;
}
function secretScanReasons(files) {
for (const f of files) {
if (/\/tests?\//.test(f.filename) || /test_.*\.py$/.test(f.filename)) continue;
const patch = f.patch || '';
if (!patch) continue;
for (const pattern of SECRET_PATTERNS) {
if (pattern.test(patch)) {
return [`possible secret in diff (${f.filename}) — requires manual review`];
}
}
}
return [];
}
async function sdkTestChecksReason(github, owner, repo, sha, files, core) {
if (!touchesSdk(files)) return null;
const { data } = await github.rest.checks.listForRef({
owner,
repo,
ref: sha,
per_page: 100,
});
const runs = (data.check_runs || []).filter((r) => r.head_sha === sha);
if (runs.length === 0) {
return 'SDK code changed but no CI checks on HEAD';
}
const testRuns = runs.filter((r) => REQUIRED_SDK_CHECK_PATTERNS.some((p) => p.test(r.name || '')));
if (testRuns.length === 0) {
return 'SDK code changed but no test check runs on HEAD';
}
core?.info?.(`SDK test checks on HEAD: ${testRuns.map((r) => r.name).join(', ')}`);
return null;
}
function manualReviewReasonForAgentPy(agentChange) {
if (!agentChange.touched) return null;
if (agentChange.additions > AGENT_PY_MAX_AUTO_LINES) {
return `agent.py +${agentChange.additions} lines (>${AGENT_PY_MAX_AUTO_LINES}) requires manual review`;
}
if (agentChange.newParams > 0) {
return `agent.py adds ${agentChange.newParams} new Agent param(s) — requires manual review`;
}
return null;
}
async function listAllComments(github, owner, repo, issueNumber) {
if (typeof github.paginate === 'function') {
return github.paginate(github.rest.issues.listComments, {
owner,
repo,
issue_number: issueNumber,
per_page: 100,
});
}
const { data } = await github.rest.issues.listComments({
owner,
repo,
issue_number: issueNumber,
per_page: 100,
});
return data;
}
async function getHeadCommitDate(github, owner, repo, prNumber) {
try {
let commits;
if (typeof github.paginate === 'function') {
commits = await github.paginate(github.rest.pulls.listCommits, {
owner,
repo,
pull_number: prNumber,
per_page: 100,
});
} else {
const { data } = await github.rest.pulls.listCommits({
owner,
repo,
pull_number: prNumber,
per_page: 100,
});
commits = data;
}
const last = commits[commits.length - 1];
return last?.commit?.committer?.date || last?.commit?.author?.date || null;
} catch {
return null;
}
}
async function loadPrContext(github, owner, repo, prNumber) {
const { data: pr } = await github.rest.pulls.get({ owner, repo, pull_number: prNumber });
const { data: issue } = await github.rest.issues.get({ owner, repo, issue_number: prNumber });
const comments = await listAllComments(github, owner, repo, prNumber);
const { data: reviews } = await github.rest.pulls.listReviews({
owner,
repo,
pull_number: prNumber,
per_page: 100,
});
const mergeState = await getMergeState(github, owner, repo, prNumber);
const headSha = mergeState.headSha || pr.head.sha;
const headCommitDate = await getHeadCommitDate(github, owner, repo, prNumber);
const headPushedAt = headCommitDate || pr.updated_at;
return {
pr,
issue,
comments,
reviews,
mergeState,
headSha,
headPushedAt,
labels: issue.labels.map((l) => l.name),
baseRepo: `${owner}/${repo}`,
};
}
async function evaluatePipelineQuiescent(github, owner, repo, prNumber, core, options = {}) {
const {
forMergeStep = false,
skipGlobalClaudeRunCheck = false,
skipRecentClaudeCooldown = false,
} = options;
const ctx = await loadPrContext(github, owner, repo, prNumber);
const reasons = [];
if (ctx.pr.draft) reasons.push('draft');
if (ctx.pr.state !== 'open') reasons.push('not open');
if (ctx.labels.includes('auto-merged-by-gate')) reasons.push('already merged by gate');
if (ctx.labels.includes('no-auto-merge')) reasons.push('no-auto-merge label');
if (ctx.labels.includes('claude-conflict-pending')) reasons.push('claude-conflict-pending');
if (!forMergeStep && ctx.labels.includes('claude-merge-gate-active')) {
reasons.push('claude-merge-gate-active');
}
const { status, headRepo, maintainerCanModify } = ctx.mergeState;
if (!ALLOWED_MERGE_STATES.has(status)) reasons.push(`mergeState=${status}`);
if (headRepo && headRepo !== ctx.baseRepo) {
reasons.push('fork PR');
}
if (hasRecentConflictComment(ctx.comments, ctx.headPushedAt, status)) {
reasons.push('recent merge-conflict @claude');
}
if (!skipRecentClaudeCooldown && hasRecentClaudeTrigger(ctx.comments, 35)) {
const verdictOnHead = findMergeGateVerdict(ctx.comments, null, ctx.headPushedAt) !== null;
if (!verdictOnHead) reasons.push('recent @claude within 35min');
}
if (!skipGlobalClaudeRunCheck && (await hasInProgressClaudeAssistant(github, owner, repo, prNumber))) {
reasons.push('claude.yml in progress');
}
const checksOk = await allChecksGreenOnSha(github, owner, repo, ctx.headSha, core);
if (!checksOk) reasons.push('CI not green on HEAD');
if (!finalClaudeCompletedOnSha(ctx.comments, ctx.headPushedAt)) {
if (!hasFinalClaudeReviewTrigger(ctx.comments)) {
reasons.push('no FINAL Claude review trigger');
} else if (isStaleFinalAfterPush(ctx.comments, ctx.headPushedAt)) {
reasons.push('stale FINAL after new commits (needs @claude re-review)');
} else {
reasons.push('FINAL Claude not complete on HEAD');
}
}
if (hasAnyChangesRequested(ctx.reviews)) reasons.push('CHANGES_REQUESTED review');
if (hasManualOnlyLabel(ctx.labels)) {
const manualLabel = ctx.labels.find((l) => MANUAL_ONLY_LABELS.has(l));
reasons.push(`manual-only label: ${manualLabel}`);
}
const pullFiles = await listPullFiles(github, owner, repo, prNumber);
if (config.agentPyChecks) {
const agentChange = getAgentPyChangeFromFiles(pullFiles);
const agentManual = manualReviewReasonForAgentPy(agentChange);
if (agentManual) reasons.push(agentManual);
}
reasons.push(...sensitivePathReasons(pullFiles, ctx.labels));
reasons.push(...prSizeReasons(pullFiles));
reasons.push(...secretScanReasons(pullFiles));
const testsReason = missingTestsReason(pullFiles);
if (testsReason) reasons.push(testsReason);
const sdkCheckReason = await sdkTestChecksReason(github, owner, repo, ctx.headSha, pullFiles, core);
if (sdkCheckReason) reasons.push(sdkCheckReason);
if (ctx.pr.mergeable === false) reasons.push('not mergeable');
const ready = reasons.length === 0;
return {
ready,
reasons,
headSha: ctx.headSha,
prNumber,
};
}
async function listPrNumbersForMergeGateScan(github, owner, repo, core) {
let mergeReadyIssues;
if (typeof github.paginate === 'function') {
mergeReadyIssues = await github.paginate(github.rest.issues.listForRepo, {
owner,
repo,
state: 'open',
labels: MERGE_READY_LABEL,
per_page: 100,
});
} else {
({ data: mergeReadyIssues } = await github.rest.issues.listForRepo({
owner,
repo,
state: 'open',
labels: MERGE_READY_LABEL,
per_page: 100,
}));
}
const mergeReady = mergeReadyIssues
.filter((issue) => issue.pull_request)
.map((issue) => issue.number)
.sort((a, b) => a - b);
if (mergeReady.length > 0) {
core?.info?.(
`Merge gate scan: ${mergeReady.length} ${MERGE_READY_LABEL} PR(s) (skipping unlabelled open PRs)`
);
return mergeReady;
}
core?.info?.(`No ${MERGE_READY_LABEL} PRs — scanning oldest open PRs`);
const prs =
typeof github.paginate === 'function'
? await github.paginate(github.rest.pulls.list, {
owner,
repo,
state: 'open',
per_page: 100,
sort: 'created',
direction: 'asc',
})
: (await github.rest.pulls.list({
owner,
repo,
state: 'open',
per_page: 100,
sort: 'created',
direction: 'asc',
})).data;
return prs.map((pr) => pr.number);
}
async function selectMergeGateCandidates(github, owner, repo, prNumbers, maxCandidates, core) {
const readyList = [];
const skipped = [];
for (const num of prNumbers) {
if (readyList.length >= maxCandidates) {
core?.info?.(`Found ${maxCandidates} candidate(s) — skipping remaining PR scans`);
break;
}
const result = await evaluatePipelineQuiescent(github, owner, repo, num, core);
if (result.ready) {
readyList.push({ pr_number: num, head_sha: result.headSha });
} else {
skipped.push({ pr: num, reasons: result.reasons });
}
}
readyList.sort((a, b) => a.pr_number - b.pr_number);
return {
candidates: readyList.slice(0, maxCandidates),
skipped,
};
}
const AUTOMATED_FALLBACK_MARKER = 'Automated fallback —';
function isAutomatedFallbackVerdict(body) {
return (body || '').includes(AUTOMATED_FALLBACK_MARKER);
}
function findMergeGateVerdict(comments, minCreatedAt = null, headPushedAt = null, options = {}) {
const { excludeAutomatedFallback = false } = options || {};
const minTime = minCreatedAt ? new Date(minCreatedAt).getTime() : 0;
const headTime = headPushedAt ? new Date(headPushedAt).getTime() - 60000 : 0;
const gateComments = comments
.filter((c) => {
if (!(c.body || '').includes('MERGE_GATE_VERDICT:')) return false;
if (excludeAutomatedFallback && isAutomatedFallbackVerdict(c.body)) return false;
const created = new Date(c.created_at).getTime();
if (minTime && created < minTime) return false;
if (headTime && created < headTime) return false;
return true;
})
.sort((a, b) => new Date(b.created_at) - new Date(a.created_at));
if (gateComments.length === 0) return null;
const body = gateComments[0].body || '';
if (body.includes('MERGE_GATE_VERDICT: APPROVE')) return 'APPROVE';
if (body.includes('MERGE_GATE_VERDICT: BLOCK')) return 'BLOCK';
return null;
}
module.exports = {
CLAUDE_TRIGGER_LOGINS,
AUTO_ACTORS,
isFinalClaudeTriggerComment,
isClaudeTriggerNoise,
isClaudeFinalReplyComment,
hasRecentClaudeTrigger,
hasRecentConflictComment,
isConflictRebaseTriggerComment,
isConflictRebaseCompletionComment,
needsFinalReReviewAfterConflictRebase,
conflictRebaseQuiescent,
hasHumanChangesRequested,
hasAnyChangesRequested,
hasFinalClaudeReviewTrigger,
finalClaudeCompletedOnSha,
getMergeState,
OPTIONAL_CANCELLED_CHECKS,
listChecksOnSha,
listFailedChecksOnSha,
allChecksGreenOnSha,
hasInProgressClaudeAssistant,
claudeRunBlocksPr,
hasBlockingClaudeRunForPr,
isCiOnlyChange,
isInternalPullRequestLink,
resolvePrNumberFromLinkedPullRequests,
resolvePrNumberFromHeadBranch,
resolvePrNumberFromWorkflowRun,
WORKFLOW_ONLY_LABEL,
getAgentPyChange,
getAgentPyChangeFromFiles,
manualReviewReasonForAgentPy,
countNewAgentParams,
listPullFiles,
touchesSdk,
hasManualOnlyLabel,
sensitivePathReasons,
prSizeReasons,
missingTestsReason,
secretScanReasons,
sdkTestChecksReason,
listAllComments,
getHeadCommitDate,
isStaleFinalAfterPush,
needsStaleFinalRecovery,
shouldSkipFinalRecovery,
shouldSkipStaleFinalRecovery,
isClaudeAutomationLogin,
isPushSoonAfterLatestFinal,
countFinalTriggersSince,
STALE_FINAL_RECOVERY_WINDOW_MS,
STALE_FINAL_MAX_PER_WINDOW,
FINAL_CLAUDE_REVIEW_BODY,
loadPrContext,
evaluatePipelineQuiescent,
MERGE_READY_LABEL,
listPrNumbersForMergeGateScan,
selectMergeGateCandidates,
isAutomatedFallbackVerdict,
AUTOMATED_FALLBACK_MARKER,
findMergeGateVerdict,
};
@@ -0,0 +1,82 @@
#!/usr/bin/env node
/**
* Run: node .github/scripts/pipeline-status-selftest.js
*/
const ps = require('./pipeline-status.js');
const mg = require('./merge-gate.js');
const config = require('./gate-config.js');
let failed = 0;
function assert(name, cond) {
if (!cond) {
console.error('FAIL:', name);
failed++;
} else {
console.log('ok:', name);
}
}
const kickComments = [
{ user: { login: 'MervinPraison' }, body: '@coderabbitai review' },
{ user: { login: 'MervinPraison' }, body: '/review' },
];
const finalComment = {
user: { login: 'MervinPraison' },
body: '@claude You are the FINAL architecture reviewer.',
created_at: '2026-06-26T10:00:00Z',
};
assert(
'reviews pending when not kicked',
ps.deriveStage([], { ready: false, reasons: ['no FINAL Claude review trigger'] })
=== 'pipeline/reviews-pending'
);
assert(
'final pending after kick',
ps.deriveStage(kickComments, { ready: false, reasons: ['no FINAL Claude review trigger'] })
=== 'pipeline/final-claude-pending'
);
assert(
'awaiting merge gate after bot review comment',
ps.deriveStage(
[
{ user: { login: 'coderabbitai[bot]' }, body: 'review summary' },
finalComment,
],
{ ready: false, reasons: ['recent @claude within 35min'] }
) === 'pipeline/awaiting-merge-gate'
);
assert('review bot logins configured', config.reviewBotLogins.includes('coderabbitai[bot]'));
assert(
'merge ready',
ps.deriveStage([...kickComments, finalComment], { ready: true, reasons: [] })
=== 'pipeline/merge-ready'
);
const blockers = ps.deriveBlockerLabels({
ready: false,
reasons: ['CI not green on HEAD', 'SDK code added without test file changes — requires manual review'],
});
assert('maps ci blocker', blockers.includes('pipeline/blocked:ci'));
assert('maps manual blocker', blockers.includes('pipeline/blocked:manual-review'));
assert(
'internal link matches upstream base',
mg.isInternalPullRequestLink(
{ base: { repo: { full_name: 'MervinPraison/PraisonAI' } } },
'MervinPraison',
'PraisonAI'
)
);
assert(
'fork sync link rejected',
!mg.isInternalPullRequestLink(
{ number: 21, base: { repo: { full_name: 'Milkmange/PraisonAI' } } },
'MervinPraison',
'PraisonAI'
)
);
process.exit(failed ? 1 : 0);
+264
View File
@@ -0,0 +1,264 @@
/**
* Sync pipeline stage/blocker labels on open PRs.
* @see .github/workflows/pipeline-status-sync.yml
*/
const mergeGate = require('./merge-gate.js');
const chain = require('./bot-pr-review-chain.js');
const ciFix = require('./ci-failure-claude.js');
const config = require('./gate-config.js');
const PIPELINE_PREFIX = 'pipeline/';
const STAGE_LABELS = [
'pipeline/reviews-pending',
'pipeline/final-claude-pending',
'pipeline/awaiting-merge-gate',
'pipeline/merge-ready',
'pipeline/merged',
];
const BLOCKER_LABELS = [
'pipeline/blocked:ci',
'pipeline/blocked:conflict',
'pipeline/blocked:manual-review',
'pipeline/blocked:cooldown',
'pipeline/blocked:stale-final',
'pipeline/blocked:no-final',
];
const ALL_PIPELINE_LABELS = [...STAGE_LABELS, ...BLOCKER_LABELS];
const LABEL_SPECS = [
{ name: 'pipeline/reviews-pending', color: 'fbca04', description: 'Waiting for CodeRabbit/Qodo/Copilot reviews' },
{ name: 'pipeline/final-claude-pending', color: 'd4c5f9', description: 'Reviews done; waiting for FINAL @claude' },
{ name: 'pipeline/awaiting-merge-gate', color: 'c5def5', description: 'FINAL done; waiting for merge gate / CI' },
{ name: 'pipeline/merge-ready', color: '0e8a16', description: 'Eligible for merge gate auto-merge' },
{ name: 'pipeline/merged', color: '6f42c1', description: 'Merged via pipeline' },
{ name: 'pipeline/blocked:ci', color: 'd93f0b', description: 'Blocked: CI not green on HEAD' },
{ name: 'pipeline/blocked:conflict', color: 'b60205', description: 'Blocked: merge conflict or rebase pending' },
{ name: 'pipeline/blocked:manual-review', color: 'e99695', description: 'Blocked: requires manual review' },
{ name: 'pipeline/blocked:cooldown', color: 'fef2c0', description: 'Blocked: post-push or @claude cooldown' },
{ name: 'pipeline/blocked:stale-final', color: 'f9d0c4', description: 'Blocked: FINAL stale after new commits' },
{ name: 'pipeline/blocked:no-final', color: 'ededed', description: 'Blocked: no FINAL @claude trigger yet' },
];
function reviewChainStarted(comments) {
if (chain.chainKickPosted(comments)) return true;
return comments.some((c) => config.reviewBotLogins.includes(c.user?.login));
}
function deriveStage(comments, evalResult) {
if (evalResult.ready) return 'pipeline/merge-ready';
if (!reviewChainStarted(comments)) return 'pipeline/reviews-pending';
if (!mergeGate.hasFinalClaudeReviewTrigger(comments)) return 'pipeline/final-claude-pending';
return 'pipeline/awaiting-merge-gate';
}
function reasonToBlockerLabel(reason) {
const r = (reason || '').toLowerCase();
if (r.includes('ci not green') || r.includes('sdk code changed but no ci')) return 'pipeline/blocked:ci';
if (
r.includes('conflict') ||
r.includes('mergestate=dirty') ||
r.includes('not mergeable')
) return 'pipeline/blocked:conflict';
if (
r.includes('manual') ||
r.includes('requires manual') ||
r.includes('sensitive path') ||
r.includes('no-auto-merge') ||
r.includes('manual-only label') ||
r.includes('without test') ||
r.includes('product code') ||
r.includes('files changed') ||
r.includes('possible secret')
) return 'pipeline/blocked:manual-review';
if (r.includes('recent @claude') || r.includes('post-push buffer')) return 'pipeline/blocked:cooldown';
if (r.includes('stale final')) return 'pipeline/blocked:stale-final';
if (r.includes('no final claude')) return 'pipeline/blocked:no-final';
if (r.includes('final claude not complete')) return 'pipeline/blocked:stale-final';
if (r.includes('claude.yml in progress') || r.includes('claude-merge-gate-active')) {
return null;
}
return null;
}
function deriveBlockerLabels(evalResult) {
if (evalResult.ready) return [];
const labels = new Set();
for (const reason of evalResult.reasons || []) {
const mapped = reasonToBlockerLabel(reason);
if (mapped && mapped.startsWith('pipeline/blocked:')) labels.add(mapped);
}
return [...labels];
}
function computePipelineLabels(comments, evalResult) {
const stage = deriveStage(comments, evalResult);
const blockers = stage === 'pipeline/merge-ready' ? [] : deriveBlockerLabels(evalResult);
return { stage, blockers, all: [stage, ...blockers] };
}
async function ensurePipelineLabels(github, owner, repo, core) {
let existing;
if (typeof github.paginate === 'function') {
existing = await github.paginate(github.rest.issues.listLabelsForRepo, {
owner,
repo,
per_page: 100,
});
} else {
({ data: existing } = await github.rest.issues.listLabelsForRepo({
owner,
repo,
per_page: 100,
}));
}
const names = new Set(existing.map((l) => l.name));
for (const spec of LABEL_SPECS) {
if (names.has(spec.name)) continue;
try {
await github.rest.issues.createLabel({
owner,
repo,
name: spec.name,
color: spec.color,
description: spec.description,
});
core?.info?.(`Created label ${spec.name}`);
} catch (err) {
if (err.status !== 422) throw err;
core?.info?.(`Label ${spec.name} already exists`);
}
}
}
async function syncPipelineLabels(github, owner, repo, prNumber, core) {
const ctx = await mergeGate.loadPrContext(github, owner, repo, prNumber);
if (ctx.pr.state !== 'open') return { synced: false, reason: 'not_open' };
const evalResult = await mergeGate.evaluatePipelineQuiescent(
github, owner, repo, prNumber, core
);
const { stage, blockers, all } = computePipelineLabels(ctx.comments, evalResult);
const current = ctx.labels.filter((l) => l.startsWith(PIPELINE_PREFIX));
const desired = new Set(all);
const toRemove = current.filter((l) => !desired.has(l) && l !== 'pipeline/merged');
const toAdd = all.filter((l) => !current.includes(l));
for (const name of toRemove) {
try {
await github.rest.issues.removeLabel({ owner, repo, issue_number: prNumber, name });
} catch (err) {
if (err.status !== 404) throw err;
}
}
if (toAdd.length) {
await github.rest.issues.addLabels({
owner,
repo,
issue_number: prNumber,
labels: toAdd,
});
}
core?.info?.(`PR #${prNumber}: stage=${stage} blockers=[${blockers.join(', ')}]`);
const ciNotGreen = (evalResult.reasons || []).some((r) =>
r.toLowerCase().includes('ci not green')
);
if (ciNotGreen || blockers.includes('pipeline/blocked:ci')) {
await ciFix.maybeTriggerCiFixClaude(github, owner, repo, prNumber, core);
} else {
await ciFix.maybeClearCiFixLabel(
github, owner, repo, prNumber, ctx.labels, ctx.headSha, core
);
}
return {
synced: true,
stage,
blockers,
ready: evalResult.ready,
reasons: evalResult.reasons,
labels: ctx.labels,
createdAt: new Date(ctx.pr.created_at).getTime(),
};
}
async function dispatchMergeGateForOldestReady(github, owner, repo, readyCandidates, core) {
if (!readyCandidates.length) return 0;
readyCandidates.sort((a, b) => a.createdAt - b.createdAt);
for (const cand of readyCandidates) {
if ((cand.labels || []).includes('claude-merge-gate-active')) {
core?.info?.(`Skip dispatch PR #${cand.prNumber}: merge gate already active`);
continue;
}
await github.rest.repos.createDispatchEvent({
owner,
repo,
event_type: 'claude-merge-gate',
client_payload: { pr_number: cand.prNumber },
});
core?.info?.(`Dispatched merge gate for ready PR #${cand.prNumber}`);
return cand.prNumber;
}
return 0;
}
async function syncOpenPullRequests(github, owner, repo, options, core) {
const { maxPrs = 20, dispatchMergeGate = true } = options || {};
await ensurePipelineLabels(github, owner, repo, core);
let prs;
if (maxPrs <= 100) {
({ data: prs } = await github.rest.pulls.list({
owner,
repo,
state: 'open',
per_page: 100,
}));
} else {
prs = await github.paginate(github.rest.pulls.list, {
owner,
repo,
state: 'open',
per_page: 100,
});
}
let synced = 0;
const readyCandidates = [];
for (const pr of prs) {
if (synced >= maxPrs) break;
if (pr.draft) continue;
if (pr.head?.repo?.full_name && pr.head.repo.full_name !== `${owner}/${repo}`) continue;
const result = await syncPipelineLabels(github, owner, repo, pr.number, core);
if (result.ready) {
readyCandidates.push({
prNumber: pr.number,
createdAt: result.createdAt || new Date(pr.created_at).getTime(),
labels: result.labels || [],
});
}
synced += 1;
}
let dispatched = 0;
if (dispatchMergeGate && readyCandidates.length) {
dispatched = await dispatchMergeGateForOldestReady(
github, owner, repo, readyCandidates, core
);
}
core?.info?.(`Pipeline label sync complete (${synced} PR(s), dispatched=${dispatched || 'none'})`);
return synced;
}
module.exports = {
STAGE_LABELS,
BLOCKER_LABELS,
ALL_PIPELINE_LABELS,
deriveStage,
deriveBlockerLabels,
computePipelineLabels,
ensurePipelineLabels,
syncPipelineLabels,
syncOpenPullRequests,
dispatchMergeGateForOldestReady,
};
+111
View File
@@ -0,0 +1,111 @@
#!/usr/bin/env node
const chain = require('./pr-review-chain.js');
let failed = 0;
function assert(name, cond) {
if (!cond) {
console.error('FAIL:', name);
failed++;
} else {
console.log('ok:', name);
}
}
const coderabbit = {
user: { login: 'coderabbitai[bot]' },
body: '<!-- summarize by coderabbit.ai --> summary',
created_at: '2026-07-02T07:16:00Z',
};
const greptile = {
user: { login: 'greptile-apps[bot]' },
body: '<h3>Greptile Summary</h3><p>Looks good</p>',
created_at: '2026-07-02T07:19:00Z',
};
const copilotKick = {
user: { login: 'MervinPraison' },
body: '@copilot Do a thorough review',
created_at: '2026-07-02T07:20:00Z',
};
const copilotReply = {
user: { login: 'copilot-swe-agent' },
body: 'Copilot review here',
created_at: '2026-07-02T07:22:00Z',
};
assert('prior not ready without greptile', !chain.priorReviewersReady([coderabbit]).ready);
assert('prior ready with coderabbit + greptile', chain.priorReviewersReady([coderabbit, greptile]).ready);
assert('copilot not ready before trigger', !chain.copilotReviewReady([coderabbit, greptile]).ready);
assert(
'copilot ready after reply',
chain.copilotReviewReady([coderabbit, greptile, copilotKick, copilotReply]).ready
);
assert(
'claude not ready before copilot',
!chain.claudeFinalReady([coderabbit, greptile]).ready
);
assert(
'claude ready after full chain',
chain.claudeFinalReady([coderabbit, greptile, copilotKick, copilotReply]).ready
);
assert(
'claude timeout fallback when copilot triggered but silent',
chain.claudeFinalReady([coderabbit, greptile, copilotKick], [], { allowCopilotTimeout: true }).ready
);
assert(
'no timeout fallback without copilot trigger',
!chain.claudeFinalReady([coderabbit, greptile], [], { allowCopilotTimeout: true }).ready
);
assert(
'skipCopilot proceeds after prior reviewers',
chain.claudeFinalReady([coderabbit, greptile], [], { skipCopilot: true }).ready
);
const prev = process.env.REVIEW_CHAIN_SKIP_COPILOT;
process.env.REVIEW_CHAIN_SKIP_COPILOT = '1';
assert(
'REVIEW_CHAIN_SKIP_COPILOT env skips copilot',
chain.claudeFinalReady([coderabbit, greptile], [], {}).ready
);
process.env.REVIEW_CHAIN_SKIP_COPILOT = prev;
const geminiReview = {
user: { login: 'gemini-code-assist[bot]' },
body: '## Code Review\n\nThis pull request decomposes the large main.py file',
submitted_at: '2026-07-03T08:08:00Z',
};
assert(
'gemini pull review counts as prior reviewer',
chain.hasGeminiReview([], [geminiReview])
);
assert(
'prior ready with gemini review only (no gemini comment)',
chain.priorReviewersReady([coderabbit, greptile], [geminiReview], { optionalWaitMs: 0 }).ready
);
const oldPr = new Date(Date.now() - 31 * 60 * 1000).toISOString();
assert(
'prior not ready without coderabbit before timeout',
!chain.priorReviewersReady([greptile], [], { prCreatedAt: new Date().toISOString(), requiredWaitMs: 30 * 60 * 1000 }).ready
);
assert(
'prior timeout when coderabbit silent after 30m',
chain.priorReviewersReady([greptile], [], { prCreatedAt: oldPr, requiredWaitMs: 30 * 60 * 1000 }).ready
);
assert(
'claude FINAL timeout when required reviewer silent and no prior FINAL',
chain.claudeFinalReady([greptile], [], { prCreatedAt: oldPr, requiredWaitMs: 30 * 60 * 1000 }).ready
);
assert(
'claude FINAL timeout auto-skips copilot without explicit skipCopilot',
chain.claudeFinalReady([greptile], [], { prCreatedAt: oldPr, requiredWaitMs: 30 * 60 * 1000 }).copilotSkipped
);
const finalClaude = {
user: { login: 'github-actions[bot]' },
body: '@claude You are the FINAL architecture reviewer for this PR.',
created_at: '2026-07-02T07:25:00Z',
};
assert(
'claude FINAL not re-posted when already triggered',
!chain.claudeFinalReady([greptile, finalClaude], [], { prCreatedAt: oldPr, requiredWaitMs: 0 }).ready
);
process.exit(failed ? 1 : 0);
+300
View File
@@ -0,0 +1,300 @@
/**
* PR review pipeline: CodeRabbit / Greptile (+ optional Qodo/Gemini) → Claude (FINAL).
* Copilot step optional — set REVIEW_CHAIN_SKIP_COPILOT=1 or skipCopilot: true to bypass.
* @see .github/workflows/auto-pr-comment.yml
*/
function isSkipCopilot(options = {}) {
if (options.skipCopilot === true) return true;
if (options.skipCopilot === false) return false;
return process.env.REVIEW_CHAIN_SKIP_COPILOT === '1';
}
const COPILOT_TRIGGER_LOGINS = new Set(['MervinPraison', 'github-actions[bot]']);
const CLAUDE_TRIGGER_LOGINS = new Set(['MervinPraison', 'github-actions[bot]']);
const REQUIRED_PRIOR = ['coderabbit', 'greptile'];
const OPTIONAL_PRIOR = ['qodo', 'gemini'];
const OPTIONAL_PRIOR_WAIT_MS = 20 * 60 * 1000;
function loginOf(item) {
return (item?.user?.login || '').toLowerCase();
}
function bodyOf(item) {
return item?.body || '';
}
function hasCoderabbitSummary(comments) {
return comments.some(
(c) =>
loginOf(c).includes('coderabbit') &&
bodyOf(c).toLowerCase().includes('summarize by coderabbit')
);
}
function hasGreptileReview(comments) {
return comments.some((c) => {
if (!loginOf(c).includes('greptile')) return false;
const body = bodyOf(c);
return body.includes('Greptile Summary') || body.includes('<h3>Greptile Summary</h3>') || body.length > 150;
});
}
function hasQodoReview(comments, reviews = []) {
if (comments.some((c) => loginOf(c).includes('qodo'))) return true;
return reviews.some((r) => loginOf(r).includes('qodo'));
}
function hasGeminiReview(comments, reviews = []) {
if (reviews.some((r) => loginOf(r).includes('gemini'))) return true;
return comments.some((c) => {
if (loginOf(c).includes('gemini')) return true;
const body = bodyOf(c);
return (
body.includes('Review completed by Gemini CLI') ||
body.includes('The fix is ready for review. Please test')
);
});
}
const PRIOR_CHECKS = {
coderabbit: (comments) => hasCoderabbitSummary(comments),
greptile: (comments) => hasGreptileReview(comments),
qodo: (comments, reviews) => hasQodoReview(comments, reviews),
gemini: (comments, reviews) => hasGeminiReview(comments, reviews),
};
function priorReviewerStatus(comments, reviews = []) {
const status = {};
for (const [id, check] of Object.entries(PRIOR_CHECKS)) {
status[id] = check(comments, reviews);
}
return status;
}
function priorReviewersReady(comments, reviews = [], options = {}) {
const waitMs = options.optionalWaitMs ?? OPTIONAL_PRIOR_WAIT_MS;
const prCreatedAt = options.prCreatedAt || null;
const status = priorReviewerStatus(comments, reviews);
const missingRequired = REQUIRED_PRIOR.filter((id) => !status[id]);
if (missingRequired.length) {
return { ready: false, reason: `waiting for ${missingRequired.join(', ')}`, status };
}
const missingOptional = OPTIONAL_PRIOR.filter((id) => !status[id]);
if (missingOptional.length && prCreatedAt) {
const age = Date.now() - new Date(prCreatedAt).getTime();
if (age < waitMs) {
return {
ready: false,
reason: `waiting for optional ${missingOptional.join(', ')} (${Math.round((waitMs - age) / 60000)}m left)`,
status,
};
}
}
return { ready: true, reason: '', status };
}
function copilotTriggerComment(comments) {
return comments.find(
(c) =>
bodyOf(c).includes('@copilot') &&
COPILOT_TRIGGER_LOGINS.has(c.user?.login) &&
!bodyOf(c).toLowerCase().includes('@claude')
);
}
function copilotTriggered(comments) {
return Boolean(copilotTriggerComment(comments));
}
function copilotReviewReady(comments, reviews = []) {
const trigger = copilotTriggerComment(comments);
if (!trigger) {
return { ready: false, reason: 'copilot not triggered yet' };
}
const triggerTime = new Date(trigger.created_at).getTime();
const copilotComment = comments.some((c) => {
if (c === trigger) return false;
if (new Date(c.created_at).getTime() < triggerTime) return false;
const login = loginOf(c);
return (
login.includes('copilot') ||
c.user?.login === 'Copilot' ||
c.user?.login === 'copilot-swe-agent'
);
});
const copilotReview = reviews.some((r) => {
if (new Date(r.submitted_at).getTime() < triggerTime) return false;
return loginOf(r).includes('copilot');
});
if (copilotComment || copilotReview) {
return { ready: true, reason: '' };
}
return { ready: false, reason: 'awaiting copilot review' };
}
function isFinalClaudeTriggerComment(c) {
const body = bodyOf(c).toLowerCase();
if (!CLAUDE_TRIGGER_LOGINS.has(c.user?.login)) return false;
if (!body.includes('@claude')) return false;
if (body.includes('merge conflict')) return false;
return body.includes('final architecture reviewer') || body.includes('lead engineer');
}
function claudeFinalAlreadyTriggered(comments) {
return comments.some(isFinalClaudeTriggerComment);
}
function claudeFinalReady(comments, reviews = [], options = {}) {
if (!options.allowStaleRepost && claudeFinalAlreadyTriggered(comments)) {
return { ready: false, reason: 'claude FINAL already triggered', already: true };
}
const prior = priorReviewersReady(comments, reviews, options);
if (!prior.ready) {
return { ready: false, reason: prior.reason };
}
if (isSkipCopilot(options)) {
return { ready: true, reason: 'copilot skipped', copilotSkipped: true };
}
const copilot = copilotReviewReady(comments, reviews);
if (copilot.ready) {
return { ready: true, reason: '' };
}
if (options.allowCopilotTimeout && copilotTriggered(comments)) {
return { ready: true, reason: 'copilot timeout fallback', copilotSkipped: true };
}
return { ready: false, reason: copilot.reason };
}
/** Post @copilot when prior reviewers are ready, then @claude FINAL when Copilot done (or timeout). */
async function advanceReviewChain(github, owner, repo, prNumber, finalBody, core, options = {}) {
const { data: pr } = await github.rest.pulls.get({ owner, repo, pull_number: prNumber });
const chainOpts = {
prCreatedAt: pr.created_at,
optionalWaitMs: options.optionalWaitMs ?? 0,
};
const copilot = options.skipCopilot
? { triggered: false, reason: 'skipped' }
: await maybeTriggerCopilot(github, owner, repo, prNumber, core, chainOpts);
const claude = await maybeTriggerClaudeFinal(
github, owner, repo, prNumber, finalBody, core,
{ ...options, ...chainOpts }
);
return { copilot, claude };
}
async function maybeTriggerClaudeFinal(github, owner, repo, prNumber, finalBody, core, options = {}) {
const { data: pr } = await github.rest.pulls.get({ owner, repo, pull_number: prNumber });
const { comments, reviews } = await listCommentsAndReviews(github, owner, repo, prNumber);
const gate = claudeFinalReady(comments, reviews, {
prCreatedAt: pr.created_at,
optionalWaitMs: options.optionalWaitMs,
allowCopilotTimeout: options.allowCopilotTimeout !== false,
allowStaleRepost: options.allowStaleRepost === true,
skipCopilot: options.skipCopilot,
});
if (!gate.ready) {
core?.info?.(`PR #${prNumber}: skip Claude FINAL — ${gate.reason}`);
return { posted: false, reason: gate.reason };
}
await github.rest.issues.createComment({
owner,
repo,
issue_number: prNumber,
body: finalBody,
});
const note = gate.copilotSkipped ? ' (Copilot timeout fallback)' : '';
core?.info?.(`Posted Claude FINAL on PR #${prNumber}${note}`);
return { posted: true, reason: '' };
}
const COPILOT_REVIEW_BODY =
'@copilot Do a thorough review of this PR. Read ALL existing reviewer comments above from Qodo, CodeRabbit, Gemini, and Greptile first — incorporate their findings.\n\nReview areas:\n1. **Bloat check**: Are changes minimal and focused? Any unnecessary code or scope creep?\n2. **Security**: Any hardcoded secrets, unsafe eval/exec, missing input validation?\n3. **Performance**: Any module-level heavy imports? Hot-path regressions?\n4. **Tests**: Are tests included? Do they cover the changes adequately?\n5. **Backward compat**: Any public API changes without deprecation?\n6. **Code quality**: DRY violations, naming conventions, error handling?\n7. **Address reviewer feedback**: If Qodo, CodeRabbit, Gemini, or Greptile flagged valid issues, include them in your review\n8. Suggest specific improvements with code examples where possible';
async function listCommentsAndReviews(github, owner, repo, prNumber) {
let comments;
if (typeof github.paginate === 'function') {
comments = await github.paginate(github.rest.issues.listComments, {
owner,
repo,
issue_number: prNumber,
per_page: 100,
});
} else {
const { data } = await github.rest.issues.listComments({
owner,
repo,
issue_number: prNumber,
per_page: 100,
});
comments = data;
}
const { data: reviews } = await github.rest.pulls.listReviews({
owner,
repo,
pull_number: prNumber,
per_page: 100,
});
return { comments, reviews };
}
async function maybeTriggerCopilot(github, owner, repo, prNumber, core, options = {}) {
const { comments, reviews } = await listCommentsAndReviews(github, owner, repo, prNumber);
if (copilotTriggered(comments)) {
core?.info?.(`Copilot already triggered on PR #${prNumber}`);
return { triggered: false, reason: 'already_triggered' };
}
const prior = priorReviewersReady(comments, reviews, options);
if (!prior.ready) {
core?.info?.(`PR #${prNumber}: not ready for Copilot — ${prior.reason}`);
return { triggered: false, reason: prior.reason };
}
await github.rest.issues.createComment({
owner,
repo,
issue_number: prNumber,
body: COPILOT_REVIEW_BODY,
});
core?.info?.(`Posted @copilot on PR #${prNumber} after prior reviewers`);
return { triggered: true };
}
async function pollCopilotResponse(github, owner, repo, prNumber, options = {}) {
const maxAttempts = options.maxAttempts ?? 20;
const delayMs = options.delayMs ?? 30000;
for (let i = 0; i < maxAttempts; i += 1) {
const { comments, reviews } = await listCommentsAndReviews(github, owner, repo, prNumber);
const copilot = copilotReviewReady(comments, reviews);
if (copilot.ready) {
return { ready: true, reason: '' };
}
if (i < maxAttempts - 1) {
await new Promise((resolve) => setTimeout(resolve, delayMs));
}
}
return { ready: false, reason: 'copilot timeout' };
}
module.exports = {
isSkipCopilot,
REQUIRED_PRIOR,
OPTIONAL_PRIOR,
COPILOT_REVIEW_BODY,
priorReviewerStatus,
priorReviewersReady,
copilotTriggered,
copilotReviewReady,
claudeFinalReady,
claudeFinalAlreadyTriggered,
isFinalClaudeTriggerComment,
maybeTriggerCopilot,
maybeTriggerClaudeFinal,
advanceReviewChain,
pollCopilotResponse,
hasGeminiReview,
listCommentsAndReviews,
};
+81
View File
@@ -0,0 +1,81 @@
#!/usr/bin/env node
/**
* Run: node .github/scripts/release-gate-selftest.js
*/
const rg = require('./release-gate.js');
const config = require('./gate-config.js');
let failed = 0;
function assert(name, cond) {
if (!cond) {
console.error('FAIL:', name);
failed += 1;
} else {
console.log('ok:', name);
}
}
assert(`package name is ${config.pypiPackageName}`, config.pypiPackageName === 'praisonai-tools');
assert('bumpPatch works', rg.bumpPatch('0.3.127') === '0.3.128');
try {
const versions = rg.readVersionsFromTree(process.cwd());
assert('reads current version', /^\d+\.\d+\.\d+$/.test(versions.current));
assert('computes target patch', versions.target === rg.bumpPatch(versions.current));
} catch (err) {
console.error('FAIL: readVersionsFromTree', err.message);
failed += 1;
}
assert('PACKAGE_PATHS includes pyproject', rg.PACKAGE_PATHS.includes('pyproject.toml'));
(async () => {
const noonUtc = new Date('2026-07-08T12:00:00Z');
const mockGithub = {
rest: {
actions: {
listWorkflowRuns: async () => ({
data: {
workflow_runs: [
{
conclusion: 'success',
created_at: '2026-07-08T09:00:00Z',
status: 'completed',
},
],
},
}),
},
},
};
assert('PATCH_RELEASE_INTERVAL_DAYS is 3', rg.PATCH_RELEASE_INTERVAL_DAYS === 3);
assert(
'hasSuccessfulReleaseWithinDays blocks recent release',
await rg.hasSuccessfulReleaseWithinDays(mockGithub, 'o', 'r', noonUtc)
);
const oldReleaseGithub = {
rest: {
actions: {
listWorkflowRuns: async () => ({
data: {
workflow_runs: [
{
conclusion: 'success',
created_at: '2026-07-04T09:00:00Z',
status: 'completed',
},
],
},
}),
},
},
};
assert(
'hasSuccessfulReleaseWithinDays allows after 3 days',
!(await rg.hasSuccessfulReleaseWithinDays(oldReleaseGithub, 'o', 'r', noonUtc))
);
process.exit(failed ? 1 : 0);
})();
+227
View File
@@ -0,0 +1,227 @@
/**
* Release gate preflight — path changes, CI SHA, dedupe, PyPI version checks (PraisonAIUI).
*/
const https = require('https');
const config = require('./gate-config.js');
const PACKAGE_PATHS = config.packagePaths;
/** Minimum days between successful patch auto-releases. */
const PATCH_RELEASE_INTERVAL_DAYS = 3;
const ACTIVE_RELEASE_STATUSES = new Set([
'queued', 'in_progress', 'waiting', 'pending', 'requested',
]);
function bumpPatch(version) {
const parts = version.split('.');
if (parts.length !== 3) throw new Error(`Invalid version: ${version}`);
return `${parts[0]}.${parts[1]}.${Number(parts[2]) + 1}`;
}
function readVersionsFromTree(root = '.') {
const fs = require('fs');
const path = require('path');
const toml = fs.readFileSync(path.join(root, 'pyproject.toml'), 'utf8');
const match = toml.match(/^version\s*=\s*"([^"]+)"/m);
if (!match) throw new Error('Could not read aiui version from pyproject.toml');
const current = match[1];
const target = bumpPatch(current);
return { current, target, packageName: config.pypiPackageName };
}
function pypiVersionExists(packageName, version) {
return new Promise((resolve) => {
const req = https.get(
`https://pypi.org/pypi/${packageName}/${version}/json`,
{ timeout: 15000 },
(res) => {
res.resume();
resolve(res.statusCode === 200);
}
);
req.on('error', () => resolve(false));
req.on('timeout', () => {
req.destroy();
resolve(false);
});
});
}
async function hasActiveReleaseRun(github, owner, repo) {
const runs = await github.rest.actions.listWorkflowRuns({
owner,
repo,
workflow_id: 'pypi-release.yml',
per_page: 20,
});
return runs.data.workflow_runs.some(
(r) => ACTIVE_RELEASE_STATUSES.has(r.status) && !r.conclusion
);
}
function utcDayStart(now = new Date()) {
return new Date(Date.UTC(now.getUTCFullYear(), now.getUTCMonth(), now.getUTCDate()));
}
function releaseIntervalStart(now = new Date(), intervalDays = PATCH_RELEASE_INTERVAL_DAYS) {
return new Date(now.getTime() - intervalDays * 24 * 60 * 60 * 1000);
}
async function hasSuccessfulReleaseWithinDays(
github,
owner,
repo,
now = new Date(),
intervalDays = PATCH_RELEASE_INTERVAL_DAYS,
) {
const windowStart = releaseIntervalStart(now, intervalDays);
const runs = await github.rest.actions.listWorkflowRuns({
owner,
repo,
workflow_id: 'pypi-release.yml',
status: 'completed',
per_page: 30,
});
return runs.data.workflow_runs.some(
(r) => r.conclusion === 'success' && new Date(r.created_at) >= windowStart
);
}
/** @deprecated Use hasSuccessfulReleaseWithinDays — kept for selftests. */
async function hasSuccessfulReleaseToday(github, owner, repo, now = new Date()) {
return hasSuccessfulReleaseWithinDays(github, owner, repo, now, 1);
}
async function lastGreenCiSha(github, owner, repo) {
const runs = await github.rest.actions.listWorkflowRuns({
owner,
repo,
workflow_id: config.ciWorkflowFile,
branch: 'main',
status: 'completed',
per_page: 30,
});
const hit = runs.data.workflow_runs.find((r) => r.conclusion === 'success');
return hit ? hit.head_sha : '';
}
async function evaluateReleasePreflight(github, owner, repo, options, core) {
const {
headSha,
isCiTrigger = false,
bump = 'patch',
} = options;
const reasons = [];
const out = {
ready: false,
reasons,
headSha: headSha || '',
lastTag: '',
targetVersion: '',
packageName: config.pypiPackageName,
};
if (bump !== 'patch') {
reasons.push('only patch auto-release supported');
return out;
}
if (await hasActiveReleaseRun(github, owner, repo)) {
reasons.push('PyPI Release already in progress or awaiting approval');
return out;
}
const referenceTime = options.now instanceof Date ? options.now : new Date();
if (await hasSuccessfulReleaseWithinDays(github, owner, repo, referenceTime)) {
reasons.push(
`already released within last ${PATCH_RELEASE_INTERVAL_DAYS} days; `
+ `max one patch release every ${PATCH_RELEASE_INTERVAL_DAYS} days`
);
return out;
}
let versions;
try {
versions = readVersionsFromTree();
} catch (err) {
reasons.push(err.message);
return out;
}
out.targetVersion = versions.target;
const onPypi = await pypiVersionExists(config.pypiPackageName, versions.target);
if (onPypi) {
reasons.push(`already published: ${config.pypiPackageName}==${versions.target}`);
return out;
}
const { execSync } = require('child_process');
let lastTag = '';
try {
lastTag = execSync('git describe --tags --match "v*" --abbrev=0', { encoding: 'utf8' }).trim();
} catch {
reasons.push('no v* tag found');
return out;
}
out.lastTag = lastTag;
let changed = '';
try {
changed = execSync(
`git diff --name-only ${lastTag} HEAD -- ${PACKAGE_PATHS.join(' ')}`,
{ encoding: 'utf8' }
).trim();
} catch {
reasons.push('git diff failed');
return out;
}
if (!changed) {
reasons.push(`no changes in ${PACKAGE_PATHS.join(' or ')} since ${lastTag}`);
return out;
}
if (isCiTrigger) {
let mainSha = '';
try {
mainSha = execSync('git rev-parse origin/main', { encoding: 'utf8' }).trim();
} catch {
reasons.push('could not resolve origin/main');
return out;
}
if (headSha !== mainSha) {
reasons.push(`superseded: green SHA ${headSha.slice(0, 7)} != main ${mainSha.slice(0, 7)}`);
return out;
}
out.headSha = headSha;
} else {
const evalSha = execSync('git rev-parse HEAD', { encoding: 'utf8' }).trim();
out.headSha = evalSha;
const greenSha = await lastGreenCiSha(github, owner, repo);
if (greenSha !== evalSha) {
reasons.push(`CI not green on HEAD (last green: ${greenSha ? greenSha.slice(0, 7) : 'none'})`);
return out;
}
}
out.ready = true;
out.reasons = ['ready'];
if (core) core.info(`Release preflight passed for ${out.headSha || headSha}`);
return out;
}
module.exports = {
bumpPatch,
readVersionsFromTree,
pypiVersionExists,
PATCH_RELEASE_INTERVAL_DAYS,
releaseIntervalStart,
hasSuccessfulReleaseWithinDays,
hasSuccessfulReleaseToday,
utcDayStart,
evaluateReleasePreflight,
lastGreenCiSha,
ACTIVE_RELEASE_STATUSES,
PACKAGE_PATHS,
};
+33
View File
@@ -0,0 +1,33 @@
name: Auto PR Comment
on:
issue_comment:
types: [created]
pull_request_review:
types: [submitted]
pull_request:
types: [opened, synchronize]
pull_request_target:
types: [opened, synchronize]
schedule:
- cron: '30 * * * *'
workflow_dispatch:
env:
REVIEW_CHAIN_SKIP_COPILOT: '1'
permissions:
issues: write
pull-requests: write
jobs:
review-chain:
# Grant union of callee job permissions (repos with default_workflow_permissions: read
# reject reusable calls when the caller ceiling is lower than any callee job).
permissions:
issues: write
pull-requests: write
uses: MervinPraison/github-automation-template/.github/workflows/[email protected]
secrets: inherit
with:
hub_ref: v1.1.20
+23
View File
@@ -0,0 +1,23 @@
name: Bot PR Recovery
on:
schedule:
- cron: '*/15 * * * *'
workflow_dispatch:
inputs:
pr_number:
description: 'Specific PR number to kick (optional)'
required: false
type: string
permissions:
pull-requests: write
issues: write
jobs:
recover:
uses: MervinPraison/github-automation-template/.github/workflows/[email protected]
secrets: inherit
with:
hub_ref: v1.1.20
pr_number: ${{ github.event.inputs.pr_number || '' }}
@@ -0,0 +1,37 @@
name: Chain Claude After Reviewers
on:
pull_request_review:
types: [submitted]
jobs:
claude-final-review:
if: |
github.event_name == 'pull_request_review' &&
github.event.review.user.login == 'copilot-pull-request-reviewer[bot]'
runs-on: ubuntu-latest
permissions:
pull-requests: write
steps:
- name: Post Claude final review + fix request
uses: actions/github-script@v7
with:
github-token: ${{ secrets.GH_TOKEN }}
script: |
const issue_number = context.payload.pull_request?.number ?? context.issue?.number;
const comments = await github.rest.issues.listComments({
issue_number,
owner: context.repo.owner,
repo: context.repo.repo,
per_page: 100
});
if (comments.data.some(c => c.body && /@claude\b/i.test(c.body))) {
console.log('@claude already requested, skipping');
return;
}
await github.rest.issues.createComment({
issue_number,
owner: context.repo.owner,
repo: context.repo.repo,
body: '@claude You are the FINAL reviewer. Read ALL comments above from Qodo, Coderabbit, and Copilot.\n\n**Phase 1: Review**\n1. Code follows existing tool patterns (see lumalabs_tool.py)?\n2. Lazy imports for optional deps?\n3. All options as function parameters?\n4. Proper error handling?\n\n**Phase 2: Fix valid issues**\n5. Implement fixes from other reviewers\n6. Push to THIS branch\n\n**Phase 3: Verdict**\n7. Approve or request changes'
});
+32
View File
@@ -0,0 +1,32 @@
name: Auto CI Failure → Claude
on:
workflow_run:
workflows: [CI, Optimized Test Suite]
types: [completed]
pull_request:
types: [opened, synchronize, reopened]
workflow_dispatch:
inputs:
pr_number:
required: false
type: string
schedule:
- cron: '15 */6 * * *'
concurrency:
group: ci-failure-scan-${{ github.repository }}-${{ github.event.pull_request.number || github.event.workflow_run.id || github.run_id }}
cancel-in-progress: true
permissions:
pull-requests: read
issues: write
actions: read
jobs:
detect-and-trigger:
uses: MervinPraison/github-automation-template/.github/workflows/[email protected]
secrets: inherit
with:
hub_ref: v1.1.20
pr_number: ${{ github.event.inputs.pr_number || '' }}
+34
View File
@@ -0,0 +1,34 @@
name: CI
on:
push:
branches: [main]
pull_request:
branches: [main]
jobs:
python:
name: python
runs-on: ubuntu-latest
strategy:
matrix:
python-version: ["3.10", "3.11", "3.12"]
steps:
- uses: actions/checkout@v4
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
- name: Install dependencies
run: |
pip install -e ".[dev]"
pip install ruff pytest-timeout
- name: Lint with ruff
run: ruff check praisonai_tools tests --ignore E501,E701,E722,F401,F402
- name: Test with pytest
run: pytest tests/unit --ignore=tests/unit/video -v --timeout=120 -x -q
+96
View File
@@ -0,0 +1,96 @@
name: Claude PR Merge Gate
on:
schedule:
- cron: '*/10 * * * *'
workflow_dispatch:
inputs:
pr_number:
required: false
type: string
merge_method:
required: false
type: choice
default: merge
options: [merge, squash, rebase]
workflow_run:
workflows:
- Claude Assistant
- CI
types: [completed]
repository_dispatch:
types: [claude-merge-gate]
concurrency:
group: claude-merge-gate-${{ github.repository }}-${{ github.event.inputs.pr_number || github.event.client_payload.pr_number || github.event.workflow_run.id || 'scan' }}
cancel-in-progress: false
permissions:
contents: read
pull-requests: read
issues: read
actions: read
jobs:
auto-dispatch:
if: github.event_name == 'workflow_run'
runs-on: ubuntu-latest
permissions:
contents: read
pull-requests: read
issues: read
actions: write
id-token: write
steps:
- uses: actions/checkout@v4
- uses: MervinPraison/github-automation-template/.github/actions/[email protected]
with:
hub_ref: v1.1.20
- name: Generate GitHub App token
id: app-token
uses: actions/create-github-app-token@v1
with:
app-id: ${{ secrets.CLAUDE_APP_ID }}
private-key: ${{ secrets.CLAUDE_APP_PRIVATE_KEY }}
owner: ${{ github.repository_owner }}
- name: Dispatch merge gate when PR pipeline is ready
uses: actions/github-script@v7
with:
github-token: ${{ steps.app-token.outputs.token }}
script: |
const path = require('path');
const mergeGate = require(path.join(process.env.HUB_SCRIPTS, 'merge-gate.js'));
const wr = context.payload.workflow_run;
const owner = context.repo.owner;
const repo = context.repo.repo;
if (wr.conclusion !== 'success') return;
const allowed = ['Claude Assistant', 'CI'];
if (!allowed.includes(wr.name)) return;
const prNumber = await mergeGate.resolvePrNumberFromWorkflowRun(github, owner, repo, wr);
if (!prNumber) return;
const fromClaude = wr.name === 'Claude Assistant';
const check = await mergeGate.evaluatePipelineQuiescent(github, owner, repo, prNumber, core, {
skipGlobalClaudeRunCheck: fromClaude,
skipRecentClaudeCooldown: fromClaude,
});
if (!check.ready) return;
await github.rest.repos.createDispatchEvent({
owner, repo, event_type: 'claude-merge-gate',
client_payload: { pr_number: prNumber },
});
merge-gate:
if: github.event_name != 'workflow_run'
permissions:
contents: write
pull-requests: write
issues: write
actions: read
id-token: write
statuses: write # the assess job records security/outsider-review
uses: MervinPraison/github-automation-template/.github/workflows/[email protected]
secrets: inherit
with:
hub_ref: v1.1.20
pr_number: ${{ github.event.inputs.pr_number || github.event.client_payload.pr_number || '' }}
merge_method: ${{ github.event.inputs.merge_method || 'merge' }}
+21
View File
@@ -0,0 +1,21 @@
name: Claude OAuth
on:
workflow_dispatch:
inputs:
code:
description: 'Authorization code (leave empty for step 1)'
required: false
permissions:
actions: write # Required for cache management
contents: read # Required for basic repository access
jobs:
auth:
runs-on: ubuntu-latest
steps:
- uses: grll/claude-code-login@v1
with:
code: ${{ inputs.code }}
secrets_admin_pat: ${{ secrets.SECRETS_ADMIN_PAT }}
+24
View File
@@ -0,0 +1,24 @@
name: Claude Assistant
on:
issue_comment:
types: [created]
pull_request_review_comment:
types: [created]
issues:
types: [assigned, labeled, opened]
pull_request_review:
types: [submitted]
jobs:
claude:
permissions:
contents: write
pull-requests: write
issues: write
actions: read
id-token: write
uses: MervinPraison/github-automation-template/.github/workflows/[email protected]
secrets: inherit
with:
hub_ref: v1.1.20
+99
View File
@@ -0,0 +1,99 @@
name: Generate SDK Reference Docs
on:
push:
branches:
- main
paths:
- 'praisonai_tools/**'
- 'pyproject.toml'
- 'scripts/generate_reference_docs.py'
workflow_run:
workflows:
- CI
types:
- completed
branches:
- main
repository_dispatch:
types: [trigger-docs-generation]
workflow_dispatch:
permissions:
contents: read
concurrency:
group: generate-docs-praisonaidocs
cancel-in-progress: false
jobs:
generate-docs:
if: |
github.event_name != 'workflow_run' ||
(
github.event.workflow_run.conclusion == 'success' &&
github.event.workflow_run.head_branch == 'main' &&
github.event.workflow_run.event == 'push' &&
!startsWith(github.event.workflow_run.head_commit.message, 'docs: auto-generate')
)
runs-on: ubuntu-latest
permissions:
contents: write
env:
DOCS_PUSH_TOKEN: ${{ secrets.DOCS_GITHUB_TOKEN || secrets.GH_TOKEN }}
steps:
- name: Checkout PraisonAI Tools
uses: actions/checkout@v4
with:
path: praisonai-tools
- name: Checkout PraisonAI Source Repo
uses: actions/checkout@v4
with:
repository: MervinPraison/PraisonAI
path: praisonai-source
- name: Checkout PraisonAI Docs
uses: actions/checkout@v4
with:
repository: MervinPraison/PraisonAIDocs
path: praisonai-docs
token: ${{ env.DOCS_PUSH_TOKEN }}
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: '3.11'
- name: Install PraisonAI Tools
run: |
cd praisonai-tools
pip install -e .
- name: Run Docs Generation
run: |
# Use the new unified CLI
praisonai-tools docs-generate \
--package all \
--docs-root $(pwd)/praisonai-docs \
--source-root $(pwd)/praisonai-source
- name: Sync SDK reference navigation
run: python3 praisonai-docs/scripts/sync_sdk_reference_nav.py
env:
PYTHONPATH: $(pwd)/praisonai-tools
- name: Commit and Push Docs
if: env.DOCS_PUSH_TOKEN != ''
run: |
cd praisonai-docs
git config --global user.name "github-actions[bot]"
git config --global user.email "github-actions[bot]@users.noreply.github.com"
git add .
if git diff --staged --quiet; then
echo "No changes to commit"
else
git commit -m "docs: auto-generate SDK reference documentation"
git pull --rebase origin main
git push origin main
fi
@@ -0,0 +1,52 @@
name: Auto Merge Conflict → Claude
on:
push:
branches: [main]
pull_request:
types: [opened, synchronize, reopened]
workflow_dispatch:
schedule:
- cron: '0 */6 * * *'
concurrency:
group: merge-conflict-scan-${{ github.repository }}-${{ github.event.pull_request.number || github.run_id }}
cancel-in-progress: true
env:
GH_TOKEN: ${{ secrets.GH_TOKEN || github.token }}
permissions:
pull-requests: write
issues: write
jobs:
detect-and-trigger:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: MervinPraison/github-automation-template/.github/actions/[email protected]
with:
hub_ref: v1.1.20
- name: Detect conflicting PRs and trigger Claude
uses: actions/github-script@v7
with:
github-token: ${{ env.GH_TOKEN }}
script: |
const path = require('path');
const mg = require(path.join(process.env.HUB_SCRIPTS, 'merge-gate.js'));
const owner = context.repo.owner;
const repo = context.repo.repo;
const baseRepo = `${owner}/${repo}`;
const prs = await github.paginate(github.rest.pulls.list, { owner, repo, state: 'open', per_page: 50 });
for (const pr of prs) {
if (pr.draft) continue;
if (pr.head?.repo?.full_name !== baseRepo) continue;
const state = await mg.getMergeState(github, owner, repo, pr.number);
if (state.status !== 'DIRTY') continue;
const { data: comments } = await github.rest.issues.listComments({ owner, repo, issue_number: pr.number, per_page: 100 });
if (mg.hasRecentConflictComment(comments)) continue;
const body = '@claude this PR has merge conflicts with `main`. Rebase onto latest main, resolve conflicts, run tests, force-push with --force-with-lease.';
await github.rest.issues.createComment({ owner, repo, issue_number: pr.number, body });
await github.rest.issues.addLabels({ owner, repo, issue_number: pr.number, labels: ['claude-conflict-pending'] });
}
+132
View File
@@ -0,0 +1,132 @@
name: Nightly Release Gate
# Preflight before PyPI Release (patch bump).
# Primary: CI success on main (workflow_run).
# Backstop: cron every 3 days at 00:00 UTC.
# Dedupe: max one successful patch release every 3 days (see release-gate.js).
on:
schedule:
- cron: '0 0 */3 * *'
workflow_dispatch:
inputs:
dry_run:
description: 'Run preflight only; do not dispatch pypi-release.yml'
required: false
default: false
type: boolean
workflow_run:
workflows:
- CI
types:
- completed
branches:
- main
permissions:
contents: read
actions: write
concurrency:
group: nightly-release-gate
cancel-in-progress: false
jobs:
preflight:
if: |
github.event_name == 'workflow_dispatch' ||
github.event_name == 'schedule' ||
(
github.event_name == 'workflow_run' &&
github.event.workflow_run.conclusion == 'success' &&
github.event.workflow_run.head_branch == 'main' &&
github.event.workflow_run.event == 'push' &&
!startsWith(github.event.workflow_run.head_commit.message, 'Release v') &&
!startsWith(github.event.workflow_run.head_commit.message, 'Bump ')
)
runs-on: ubuntu-latest
env:
GH_TOKEN: ${{ secrets.GH_TOKEN }}
steps:
- name: Resolve trigger context
id: ctx
run: |
if [ "${{ github.event_name }}" = "workflow_run" ]; then
echo "checkout_sha=${{ github.event.workflow_run.head_sha }}" >> "$GITHUB_OUTPUT"
echo "source=ci" >> "$GITHUB_OUTPUT"
echo "is_ci=true" >> "$GITHUB_OUTPUT"
else
echo "checkout_sha=main" >> "$GITHUB_OUTPUT"
echo "source=nightly" >> "$GITHUB_OUTPUT"
echo "is_ci=false" >> "$GITHUB_OUTPUT"
fi
- name: Checkout
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
with:
ref: ${{ steps.ctx.outputs.checkout_sha }}
fetch-depth: 0
- name: Fetch main and tags
run: git fetch --tags origin main
- name: Preflight — path changes, CI SHA, dedupe
id: preflight
uses: actions/github-script@v7
env:
HEAD_SHA: ${{ steps.ctx.outputs.checkout_sha }}
IS_CI: ${{ steps.ctx.outputs.is_ci }}
with:
github-token: ${{ secrets.GH_TOKEN }}
script: |
const path = require('path');
const rg = require(path.join(process.env.GITHUB_WORKSPACE, '.github/scripts/release-gate.js'));
const owner = context.repo.owner;
const repo = context.repo.repo;
const headSha = process.env.HEAD_SHA;
const isCi = process.env.IS_CI === 'true';
const result = await rg.evaluateReleasePreflight(
github, owner, repo,
{ headSha, isCiTrigger: isCi, bump: 'patch' },
core
);
core.setOutput('should_release', result.ready ? 'true' : 'false');
core.setOutput('reason', result.reasons.join('; '));
core.setOutput('head_sha', result.headSha || headSha);
core.setOutput('last_tag', result.lastTag || '');
core.setOutput('target_version', result.targetVersion || '');
core.summary.addRaw('## Release gate preflight\n');
core.summary.addRaw(`Ready: ${result.ready}\n`);
core.summary.addRaw(`Reason: ${result.reasons.join('; ')}\n`);
if (result.targetVersion) {
core.summary.addRaw(`Target: package==${result.targetVersion}\n`);
}
await core.summary.write();
if (!result.ready) {
core.info(`Skip release: ${result.reasons.join('; ')}`);
}
- name: Dry run summary
if: inputs.dry_run == true
run: |
echo "Dry run — preflight only, no pypi-release dispatch."
echo "should_release=${{ steps.preflight.outputs.should_release }}"
echo "reason=${{ steps.preflight.outputs.reason }}"
- name: Dispatch PyPI Release (patch)
if: steps.preflight.outputs.should_release == 'true' && inputs.dry_run != true
run: |
set -euo pipefail
gh workflow run pypi-release.yml \
--repo "${GITHUB_REPOSITORY}" \
--ref main \
-f bump=patch \
-f source=${{ steps.ctx.outputs.source }} \
-f trigger_sha=${{ steps.preflight.outputs.head_sha }} \
-f dry_run=false
echo "Dispatched pypi-release.yml (patch, source=${{ steps.ctx.outputs.source }})"
echo "Uses pypi-auto environment for ci/nightly patch releases."
@@ -0,0 +1,19 @@
name: Pipeline Status Sync
on:
schedule:
- cron: '*/10 * * * *'
workflow_dispatch:
permissions:
pull-requests: read
issues: write
actions: write
id-token: write
jobs:
sync:
uses: MervinPraison/github-automation-template/.github/workflows/[email protected]
secrets: inherit
with:
hub_ref: v1.1.20
+148
View File
@@ -0,0 +1,148 @@
name: PyPI Release
on:
workflow_dispatch:
inputs:
bump:
description: 'Version bump type'
required: true
default: patch
type: choice
options: [patch, minor, major]
dry_run:
description: 'Preview only; do not publish'
required: false
default: false
type: boolean
version:
description: 'Override version (empty = auto bump)'
required: false
default: ''
type: string
source:
description: 'Trigger source (audit trail)'
required: false
default: manual
type: string
trigger_sha:
description: 'SHA that triggered this release'
required: false
default: ''
type: string
permissions:
contents: write
concurrency:
group: pypi-release
cancel-in-progress: false
jobs:
release:
if: github.ref == 'refs/heads/main'
runs-on: ubuntu-latest
environment: >-
${{
(inputs.source == 'ci' || inputs.source == 'nightly') &&
inputs.bump == 'patch' &&
'pypi-auto' || 'pypi'
}}
env:
PACKAGE: praisonai-tools
GH_TOKEN: ${{ secrets.GH_TOKEN }}
steps:
- uses: actions/checkout@v4
with:
ref: main
fetch-depth: 0
token: ${{ secrets.GH_TOKEN }}
- uses: actions/setup-python@v5
with:
python-version: '3.11'
- uses: astral-sh/setup-uv@v4
- name: Configure git
run: |
git config user.name "MervinPraison"
git config user.email "[email protected]"
gh auth setup-git
- name: Compute release version
id: versions
env:
BUMP: ${{ inputs.bump }}
VERSION_OVERRIDE: ${{ inputs.version }}
run: |
python <<'PY'
import os, re, sys
from pathlib import Path
bump = os.environ["BUMP"]
override = os.environ.get("VERSION_OVERRIDE", "").strip()
content = Path("pyproject.toml").read_text()
match = re.search(r'^version\s*=\s*"([^"]+)"', content, re.MULTILINE)
if not match:
sys.exit("Could not read version from pyproject.toml")
current = match.group(1)
maj, min_, patch = map(int, current.split("."))
if override:
new_version = override
elif bump == "major":
new_version = f"{maj + 1}.0.0"
elif bump == "minor":
new_version = f"{maj}.{min_ + 1}.0"
else:
new_version = f"{maj}.{min_}.{patch + 1}"
with open(os.environ["GITHUB_OUTPUT"], "a") as fh:
fh.write(f"current_version={current}\nnew_version={new_version}\n")
PY
- name: Bump pyproject.toml
if: inputs.dry_run != true
env:
CURRENT: ${{ steps.versions.outputs.current_version }}
NEW_VERSION: ${{ steps.versions.outputs.new_version }}
run: |
python -c "
from pathlib import Path
import os
p = Path('pyproject.toml')
p.write_text(p.read_text().replace(f'version = \"{os.environ[\"CURRENT\"]}\"', f'version = \"{os.environ[\"NEW_VERSION\"]}\"', 1))
"
- name: Build distribution
if: inputs.dry_run != true
run: |
uv lock
uv build
- name: Commit, tag, and push
if: inputs.dry_run != true
env:
VERSION: ${{ steps.versions.outputs.new_version }}
run: |
TAG="v${VERSION}"
git add pyproject.toml
git commit -m "Release ${TAG}" || exit 0
git tag -f "${TAG}"
git pull --rebase origin main
git push origin main
git push origin "${TAG}" --force
- name: Publish to PyPI
if: inputs.dry_run != true
env:
# PyPI API token auth: username __token__, password = pypi-… token (see pypi.org/help/#invalid-auth)
UV_PUBLISH_USERNAME: __token__
UV_PUBLISH_PASSWORD: ${{ secrets.PYPI_TOKEN || secrets.PYPI_API_TOKEN }}
run: |
uv publish
- name: Create GitHub release
if: inputs.dry_run != true
env:
VERSION: ${{ steps.versions.outputs.new_version }}
run: |
TAG="v${VERSION}"
gh release create "${TAG}" --title "praisonai-tools ${TAG}" --notes "Release ${TAG}" --latest
-31
View File
@@ -1,31 +0,0 @@
# This workflow will install Python dependencies, run tests and lint with a variety of Python versions
# For more information see: https://docs.github.com/en/actions/automating-builds-and-tests/building-and-testing-python
name: Python package
on:
push:
branches: [ "develop" ]
pull_request:
branches: [ "develop" ]
jobs:
build:
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
python-version: ["3.10", "3.11"]
steps:
- uses: actions/checkout@v3
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v3
with:
python-version: ${{ matrix.python-version }}
- name: Install dependencies
run: |
python -m pip install --upgrade pip
python -m pip install flake8 pytest
if [ -f requirements.txt ]; then pip install -r requirements.txt; fi
+36
View File
@@ -0,0 +1,36 @@
name: Sync Claude Secrets
on:
workflow_dispatch:
inputs:
target_repo:
description: 'Destination repo (owner/name)'
required: true
type: string
permissions:
contents: read
jobs:
sync:
runs-on: ubuntu-latest
steps:
- name: Sync Claude secrets to target repo
env:
GH_TOKEN: ${{ secrets.SECRETS_ADMIN_PAT }}
TARGET: ${{ inputs.target_repo }}
CLAUDE_APP_ID: ${{ secrets.CLAUDE_APP_ID }}
CLAUDE_APP_PRIVATE_KEY: ${{ secrets.CLAUDE_APP_PRIVATE_KEY }}
CLAUDE_CODE_OAUTH_TOKEN: ${{ secrets.CLAUDE_CODE_OAUTH_TOKEN }}
GH_TOKEN_VALUE: ${{ secrets.GH_TOKEN }}
run: |
set -euo pipefail
if [ -z "${GH_TOKEN:-}" ]; then
echo "SECRETS_ADMIN_PAT is not configured in this repository"
exit 1
fi
echo -n "$CLAUDE_APP_ID" | gh secret set CLAUDE_APP_ID --repo "$TARGET"
echo -n "$CLAUDE_APP_PRIVATE_KEY" | gh secret set CLAUDE_APP_PRIVATE_KEY --repo "$TARGET"
echo -n "$CLAUDE_CODE_OAUTH_TOKEN" | gh secret set CLAUDE_CODE_OAUTH_TOKEN --repo "$TARGET"
echo -n "$GH_TOKEN_VALUE" | gh secret set GH_TOKEN --repo "$TARGET"
echo "Synced CLAUDE_APP_ID, CLAUDE_APP_PRIVATE_KEY, CLAUDE_CODE_OAUTH_TOKEN, GH_TOKEN → $TARGET"
@@ -0,0 +1,21 @@
name: Sync Claude Secrets from PraisonAI
on:
workflow_dispatch:
permissions:
contents: read
jobs:
trigger-sync:
runs-on: ubuntu-latest
steps:
- name: Trigger PraisonAI secret sync
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
gh workflow run sync-secrets-to-aiui.yml \
--repo MervinPraison/PraisonAI \
--ref main \
-f target_repo=MervinPraison/PraisonAI-Tools
echo "Triggered secret sync → MervinPraison/PraisonAI-Tools"
+36 -6
View File
@@ -1,16 +1,46 @@
venv
.DS_Store
.pytest_cache
__pycache__
__pycache__/
*.py[cod]
*$py.class
# Distribution / packaging
.Python
build/
dist/
.env
.idea
test.py
*.egg-info/
# Virtual environments
.venv/
venv/
# IDE
.vscode/
.idea/
# Testing
.pytest_cache/
.coverage
htmlcov/
# Tool auto-discovery cache (generated at import time)
._discovery_cache_v*.json
# Lock files
poetry.lock
uv.lock
# Database
*.db
db/
chroma.sqlite3
.vscode
# OS
.DS_Store
praisonai
.cache
__pycache__
chroma.sqlite3
test/
.env
assets/*
-21
View File
@@ -1,21 +0,0 @@
repos:
- repo: https://github.com/psf/black-pre-commit-mirror
rev: 23.12.1
hooks:
- id: black
language_version: python3.11
files: \.(py)$
- repo: https://github.com/pycqa/isort
rev: 5.13.2
hooks:
- id: isort
name: isort (python)
args: ["--profile", "black", "--filter-files"]
- repo: https://github.com/PyCQA/autoflake
rev: v2.2.1
hooks:
- id: autoflake
args: ['--in-place', '--remove-all-unused-imports', '--remove-unused-variables', '--ignore-init-module-imports']
+1 -1
View File
@@ -1,6 +1,6 @@
MIT License
Copyright (c) 2024 João Moura
Copyright (c) 2024 Mervin Praison
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
+784 -1
View File
@@ -1 +1,784 @@
# Praison AI Tools
# PraisonAI Tools
Base classes for creating **custom tools** for [PraisonAI Agents](https://github.com/MervinPraison/PraisonAI).
[![PyPI version](https://badge.fury.io/py/praisonai-tools.svg)](https://badge.fury.io/py/praisonai-tools)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
[![Python 3.10+](https://img.shields.io/badge/python-3.10+-blue.svg)](https://www.python.org/downloads/)
## What is this package?
This package provides **base classes** (`BaseTool`, `@tool` decorator) for creating custom tools that work with PraisonAI Agents.
> **Note:** Common tools like Tavily, Exa, You.com, DuckDuckGo, Wikipedia, arXiv, and many more are **already built into `praisonaiagents`**. You don't need this package for those - just use them directly from `praisonaiagents.tools`.
## Installation
```bash
pip install praisonai-tools
```
---
# All Ways to Add Tools to PraisonAI Agents
PraisonAI Agents supports **8 different ways** to add tools. Choose the method that best fits your use case:
| Method | Best For | Complexity |
|--------|----------|------------|
| [1. Plain Python Functions](#1-plain-python-functions) | Quick custom tools | ⭐ Easy |
| [2. Built-in Tools](#2-built-in-tools) | Common operations | ⭐ Easy |
| [3. @tool Decorator](#3-tool-decorator) | Custom tools with metadata | ⭐ Easy |
| [4. BaseTool Class](#4-basetool-class) | Complex tools with state | ⭐⭐ Medium |
| [5. Pydantic Class with run()](#5-pydantic-class-with-run) | Validated tools | ⭐⭐ Medium |
| [6. LangChain Tools](#6-langchain-tools) | LangChain ecosystem | ⭐⭐ Medium |
| [7. CrewAI Tools](#7-crewai-tools) | CrewAI ecosystem | ⭐⭐ Medium |
| [8. MCP Tools](#8-mcp-model-context-protocol-tools) | External services | ⭐⭐⭐ Advanced |
---
## 1. Plain Python Functions
The simplest way - just write a function with type hints and docstring:
```python
from praisonaiagents import Agent
def search_web(query: str, max_results: int = 5) -> list:
"""Search the web for information.
Args:
query: The search query
max_results: Maximum number of results to return
"""
# Your implementation
return [{"title": "Result 1", "url": "https://..."}]
def calculate(expression: str) -> float:
"""Evaluate a mathematical expression."""
return eval(expression)
# Just pass the functions!
agent = Agent(
instructions="You are a helpful assistant",
tools=[search_web, calculate]
)
agent.start("Search for Python tutorials and calculate 15 * 7")
```
**How it works:** PraisonAI automatically:
- Extracts function name as tool name
- Uses docstring as description
- Generates JSON schema from type hints
- Parses Args section for parameter descriptions
---
## 2. Built-in Tools
Use pre-built tools from `praisonaiagents.tools`:
```python
from praisonaiagents import Agent
from praisonaiagents.tools import (
# Search
tavily_search,
exa_search,
internet_search, # DuckDuckGo
# Wikipedia
wiki_search,
wiki_summary,
# News
get_article,
get_trending_topics,
# Files
read_file,
write_file,
# Code
execute_code,
# And many more...
)
agent = Agent(
instructions="You are a research assistant",
tools=[tavily_search, wiki_search, read_file]
)
agent.start("Search for AI news and save a summary to a file")
```
### Available Built-in Tools
| Category | Tools |
|----------|-------|
| **Search** | `tavily_search`, `exa_search`, `ydc_search`, `internet_search`, `searxng_search` |
| **Wikipedia** | `wiki_search`, `wiki_summary`, `wiki_page`, `wiki_random` |
| **arXiv** | `search_arxiv`, `get_arxiv_paper`, `get_papers_by_author`, `get_papers_by_category` |
| **News** | `get_article`, `get_news_sources`, `get_articles_from_source`, `get_trending_topics` |
| **Web Crawling** | `crawl4ai`, `scrape_page`, `extract_links`, `extract_text` |
| **Files** | `read_file`, `write_file`, `list_files`, `copy_file`, `move_file`, `delete_file` |
| **Data** | `read_csv`, `write_csv`, `read_json`, `write_json`, `read_excel`, `read_yaml` |
| **Code** | `execute_code`, `analyze_code`, `format_code`, `lint_code` |
| **Shell** | `execute_command`, `list_processes`, `kill_process`, `get_system_info` |
| **Calculator** | `evaluate`, `solve_equation`, `convert_units`, `calculate_statistics` |
| **Finance** | `get_stock_price`, `get_stock_info`, `get_historical_data` |
| **Database** | `query` (DuckDB), `insert_document`, `find_documents` (MongoDB) |
---
## 3. @tool Decorator
Use the `@tool` decorator for custom tools with metadata:
```python
from praisonaiagents import Agent
from praisonai_tools import tool
@tool
def get_weather(location: str, units: str = "celsius") -> dict:
"""Get current weather for a location.
Args:
location: City name or coordinates
units: Temperature units (celsius/fahrenheit)
"""
# Your implementation
return {"temp": 22, "condition": "sunny", "location": location}
@tool(name="custom_search", description="Search with custom parameters")
def my_search(query: str, limit: int = 10) -> list:
"""Custom search implementation."""
return [{"result": query}]
agent = Agent(
instructions="You are a weather assistant",
tools=[get_weather, my_search]
)
agent.start("What's the weather in London?")
```
---
## 4. BaseTool Class
For complex tools with state, validation, or multiple methods:
```python
from praisonaiagents import Agent
from praisonai_tools import BaseTool
class DatabaseTool(BaseTool):
name = "database_query"
description = "Query a database and return results"
def __init__(self, connection_string: str):
self.connection_string = connection_string
self._connection = None
super().__init__()
def run(self, query: str, limit: int = 100) -> list:
"""Execute a database query.
Args:
query: SQL query to execute
limit: Maximum rows to return
"""
# Your implementation
return [{"id": 1, "name": "Example"}]
# Create instance with configuration
db_tool = DatabaseTool(connection_string="postgresql://...")
agent = Agent(
instructions="You are a data analyst",
tools=[db_tool]
)
agent.start("Query the users table for active users")
```
---
## 5. Pydantic Class with run()
Use Pydantic for validated tools with type checking:
```python
from praisonaiagents import Agent
from pydantic import BaseModel, Field
from typing import Optional
import requests
class APISearchTool(BaseModel):
"""Search tool using an external API."""
api_url: str = "https://api.example.com/search"
api_key: Optional[str] = None
max_results: int = Field(default=10, ge=1, le=100)
def run(self, query: str) -> dict:
"""Execute search query.
Args:
query: Search query string
"""
headers = {"Authorization": f"Bearer {self.api_key}"} if self.api_key else {}
response = requests.get(
self.api_url,
params={"q": query, "limit": self.max_results},
headers=headers
)
return response.json()
# Pass the class (not instance) - PraisonAI will instantiate it
agent = Agent(
instructions="You are a search assistant",
tools=[APISearchTool]
)
agent.start("Search for machine learning tutorials")
```
---
## 6. LangChain Tools
Use any LangChain tool directly:
```python
from praisonaiagents import Agent
from langchain_community.tools import YouTubeSearchTool, DuckDuckGoSearchRun
from langchain_community.utilities import WikipediaAPIWrapper
# Method 1: Pass LangChain tool classes directly
agent = Agent(
instructions="You are a research assistant",
tools=[YouTubeSearchTool, WikipediaAPIWrapper]
)
agent.start("Find YouTube videos about Python and search Wikipedia for its history")
```
### Wrapping LangChain Tools
For more control, wrap LangChain tools in functions:
```python
from praisonaiagents import Agent
from langchain_community.tools import YouTubeSearchTool
from langchain_community.utilities import WikipediaAPIWrapper
def youtube_search(query: str, max_results: int = 5) -> str:
"""Search YouTube for videos.
Args:
query: Search query
max_results: Number of results
"""
yt = YouTubeSearchTool()
return yt.run(f"{query}, {max_results}")
def wikipedia_search(query: str) -> str:
"""Search Wikipedia for information.
Args:
query: Search query
"""
wiki = WikipediaAPIWrapper()
return wiki.run(query)
agent = Agent(
instructions="You are a research assistant",
tools=[youtube_search, wikipedia_search]
)
agent.start("Find videos about AI and get Wikipedia info on machine learning")
```
### Using LangChain Toolkits
```python
from praisonaiagents import Agent
from langchain_agentql.tools import ExtractWebDataTool
def extract_web_data(url: str, query: str) -> dict:
"""Extract structured data from a webpage.
Args:
url: URL to extract from
query: What data to extract
"""
tool = ExtractWebDataTool()
return tool.invoke({"url": url, "prompt": query})
agent = Agent(
instructions="You are a web scraping assistant",
tools=[extract_web_data]
)
agent.start("Extract product names and prices from https://example.com/products")
```
---
## 7. CrewAI Tools
Use CrewAI tools (classes with `_run` method):
```python
from praisonaiagents import Agent
# CrewAI-style tool class
class CrewAISearchTool:
"""A CrewAI-compatible search tool."""
name = "web_search"
description = "Search the web for information"
def _run(self, query: str) -> str:
"""Execute the search.
Args:
query: Search query
"""
# Your implementation
return f"Results for: {query}"
# Pass the class - PraisonAI detects _run method
agent = Agent(
instructions="You are a search assistant",
tools=[CrewAISearchTool]
)
agent.start("Search for latest tech news")
```
---
## 8. MCP (Model Context Protocol) Tools
Use external tools via MCP servers:
### Filesystem MCP
```python
from praisonaiagents import Agent, MCP
agent = Agent(
instructions="You are a file manager assistant",
tools=MCP("npx -y @modelcontextprotocol/server-filesystem",
args=["/Users/username/Documents"])
)
agent.start("List all Python files in the Documents folder")
```
### Time MCP
```python
from praisonaiagents import Agent, MCP
agent = Agent(
instructions="You are a time assistant",
tools=MCP("python -m mcp_server_time --local-timezone=America/New_York")
)
agent.start("What time is it in New York? Convert to UTC.")
```
### GitHub MCP
```python
from praisonaiagents import Agent, MCP
import os
agent = Agent(
instructions="You are a GitHub assistant",
tools=MCP("npx -y @modelcontextprotocol/server-github",
env={"GITHUB_TOKEN": os.environ["GITHUB_TOKEN"]})
)
agent.start("List my recent repositories")
```
### Multiple MCP Servers
```python
from praisonaiagents import Agent, MCP
agent = Agent(
instructions="You are a multi-capable assistant",
tools=[
MCP("npx -y @modelcontextprotocol/server-filesystem", args=["/tmp"]),
MCP("python -m mcp_server_time"),
my_custom_function, # Can mix with other tool types!
]
)
agent.start("Create a file with the current timestamp")
```
---
## Combining Multiple Tool Types
You can mix and match all tool types:
```python
from praisonaiagents import Agent, MCP
from praisonaiagents.tools import tavily_search, wiki_search
from praisonai_tools import tool, BaseTool
from langchain_community.tools import YouTubeSearchTool
# Plain function
def calculate(expression: str) -> float:
"""Calculate a math expression."""
return eval(expression)
# @tool decorator
@tool
def format_output(data: dict) -> str:
"""Format data as a nice string."""
return str(data)
# BaseTool class
class CustomTool(BaseTool):
name = "custom"
description = "A custom tool"
def run(self, input: str) -> str:
return f"Processed: {input}"
# Combine everything!
agent = Agent(
instructions="You are a super assistant with many capabilities",
tools=[
# Built-in tools
tavily_search,
wiki_search,
# Plain function
calculate,
# Decorated function
format_output,
# BaseTool instance
CustomTool(),
# LangChain tool
YouTubeSearchTool,
# MCP server
MCP("python -m mcp_server_time"),
]
)
agent.start("Search for AI news, find related YouTube videos, and calculate 2^10")
```
---
## Creating Distributable Tool Packages
To create a pip-installable tool package:
### 1. Create your tool module
```python
# my_tools/weather.py
from praisonai_tools import BaseTool
class WeatherTool(BaseTool):
name = "get_weather"
description = "Get weather for a location"
def run(self, location: str) -> dict:
# Implementation
return {"temp": 22, "location": location}
# Convenience instance
weather_tool = WeatherTool()
```
### 2. Create pyproject.toml
```toml
[project]
name = "my-weather-tools"
version = "0.1.0"
dependencies = ["praisonai-tools"]
[project.entry-points."praisonaiagents.tools"]
weather = "my_tools.weather:WeatherTool"
```
### 3. Users can then:
```python
from praisonaiagents import Agent
from my_tools.weather import weather_tool
agent = Agent(
instructions="Weather assistant",
tools=[weather_tool]
)
```
### How `praisonai-tools` registers its own tools
The bundled tools in this package are **discovered automatically**. Every
module under `praisonai_tools/tools/` is scanned (at import time, without
executing the module) and its public top-level classes and functions become
importable directly:
```python
from praisonai_tools.tools import WeatherTool, get_weather # discovered
from praisonai_tools import WeatherTool # same symbol
```
Adding a new tool therefore means **adding one file** to
`praisonai_tools/tools/` — there is no import map or `__all__` to edit, and
imports stay lazy (a tool module and its optional dependencies are only loaded
the first time one of its symbols is accessed).
By default the discovered public surface is the module's top-level classes and
functions. If a module exposes tools through module-level aliases
(e.g. `read_yaml = _yaml_tools.read_yaml`), declare a standard `__all__` in that
module and the listed names are exported verbatim. Modules and names starting
with `_` are treated as private and never exported.
The scan result is memoised in a small on-disk cache next to the package, so
import stays fast; the cache is rebuilt automatically whenever a tool file is
added, removed, or edited.
External packages that want their tools discovered by `praisonaiagents` should
use the `[project.entry-points."praisonaiagents.tools"]` group shown above,
which the core `praisonaiagents` registry resolves.
---
## 9. Video Editing Tools
```bash
python -m praisonai_tools.video probe input.mp4
```
```bash
python -m praisonai_tools.video transcribe input.mp4 --output transcript.srt
```
```bash
python -m praisonai_tools.video edit input.mp4 --output edited.mp4 --preset podcast
```
```bash
python -m praisonai_tools.video edit input.mp4 --output edited.mp4 --preset podcast -v
```
```bash
python -m praisonai_tools.video render input.mp4 --timeline plan.json --output output.mp4
```
| Feature | Code Docs | CLI Docs |
|---------|-----------|----------|
| Video Module | [docs/video-module.md](docs/video-module.md) | [docs/video-cli.md](docs/video-cli.md) |
---
## 10. Ready-to-Use Tools from praisonai-tools
This package also includes ready-to-use tools for common integrations:
### Communication Tools
```python
from praisonai_tools import EmailTool, SlackTool, DiscordTool
# Email (Gmail, Outlook, Yahoo)
email = EmailTool(provider="gmail")
email.send(to="[email protected]", subject="Hello", body="Message")
emails = email.read(limit=10)
# Slack
slack = SlackTool() # Uses SLACK_TOKEN env var
slack.send_message(channel="#general", text="Hello from AI!")
channels = slack.list_channels()
# Discord (webhook or bot)
discord = DiscordTool(webhook_url="https://discord.com/api/webhooks/...")
discord.send_webhook(content="Hello!", embed=DiscordTool.create_embed(title="Alert"))
```
### Developer Tools
```python
from praisonai_tools import GitHubTool
github = GitHubTool() # Uses GITHUB_TOKEN env var
repos = github.search_repos("machine learning python", limit=10)
github.create_issue(repo="owner/repo", title="Bug", body="Description")
prs = github.list_pull_requests(repo="owner/repo")
```
### AI/Media Tools
```python
from praisonai_tools import ImageTool, TTSTool, YouTubeTool
# Image Generation (DALL-E)
img = ImageTool(model="dall-e-3", quality="hd")
result = img.generate("A sunset over mountains")
# Text-to-Speech (OpenAI or ElevenLabs)
tts = TTSTool(provider="openai", voice="nova")
tts.speak("Hello world!", output_path="output.mp3")
# YouTube
yt = YouTubeTool() # Uses YOUTUBE_API_KEY env var
videos = yt.search("python tutorial", limit=5)
transcript = yt.get_transcript("dQw4w9WgXcQ")
```
### Meeting Intelligence Tools
```python
from praisonai_tools import FirefliesTool
# Fireflies.ai meeting intelligence (curated GraphQL operations)
fireflies = FirefliesTool() # Uses FIREFLIES_API_KEY env var
meetings = fireflies.list_recent_meetings(limit=5)
meetings = fireflies.list_recent_meetings(limit=5, after="5") # cursor pagination
transcript = fireflies.get_transcript(meeting_id="abc123")
results = fireflies.search(keywords="roadmap")
```
`FirefliesTool` distinguishes a missing key (`{"error": "FIREFLIES_API_KEY required"}`)
from a rejected key (HTTP 401/403), and surfaces `retry_after` on rate limits.
Set `FIREFLIES_API_URL` to override the endpoint (previews/proxies).
### Data Tools
```python
from praisonai_tools import WeatherTool
weather = WeatherTool(units="metric") # Uses OPENWEATHER_API_KEY env var
current = weather.get_current("London")
forecast = weather.get_forecast("New York", days=5)
air = weather.get_air_quality("Tokyo")
```
### Environment Variables
| Tool | Required Environment Variable |
|------|------------------------------|
| EmailTool | `EMAIL_USERNAME`, `EMAIL_PASSWORD` |
| SlackTool | `SLACK_TOKEN` |
| DiscordTool | `DISCORD_BOT_TOKEN` or `DISCORD_WEBHOOK_URL` |
| GitHubTool | `GITHUB_TOKEN` |
| ImageTool | `OPENAI_API_KEY` |
| TTSTool | `OPENAI_API_KEY` or `ELEVENLABS_API_KEY` |
| YouTubeTool | `YOUTUBE_API_KEY` |
| WeatherTool | `OPENWEATHER_API_KEY` |
| FirefliesTool | `FIREFLIES_API_KEY` (optional `FIREFLIES_API_URL`) |
| SwarmScoreTool | None (uses public API) |
### Using with PraisonAI Agents
```python
from praisonaiagents import Agent
from praisonai_tools import EmailTool, SlackTool, GitHubTool, WeatherTool, SwarmScoreTool
agent = Agent(
instructions="You are a helpful assistant with access to email, slack, github, weather, and trust ratings",
tools=[
EmailTool(provider="gmail"),
SlackTool(),
GitHubTool(),
WeatherTool(),
SwarmScoreTool(),
]
)
agent.start("Check the weather in London and send a Slack message about it")
```
---
## Composio Integration
Load 250+ managed app integrations (GitHub, Slack, Gmail, Jira, etc.) as agent-callable tools:
```bash
pip install praisonai-tools[composio]
export COMPOSIO_API_KEY=your_api_key
```
```python
from praisonaiagents import Agent
from praisonai_tools import composio_tools, composio_list_apps
print(composio_list_apps()[:5]) # ["github", "slack", ...]
agent = Agent(name="dev", tools=composio_tools(apps=["github"]))
agent.start("Star the praisonai/PraisonAI repository")
```
For explicit action execution, use `ComposioTool` or `composio_execute()`.
---
## Contributing
We welcome contributions! To add a new tool:
1. Fork this repository
2. Create your tool using `BaseTool` or `@tool` decorator
3. Add tests in `tests/`
4. Submit a pull request
---
## Testing
```bash
pip install praisonai-tools[dev]
pytest tests/ -v
```
The `dev` extra installs the video/motion-graphics render deps
(`playwright`, `imageio-ffmpeg`) so the full suite runs. To exercise the
HTML render backend you also need the Chromium browser:
```bash
pip install -e ".[dev]"
playwright install chromium
pytest tests/ -v
```
If you use a minimal install without those deps, the video tests under
`tests/unit/video/` are skipped automatically rather than failing.
---
## License
MIT License - see [LICENSE](LICENSE) for details.
## Author
**Mervin Praison** - [GitHub](https://github.com/MervinPraison)
## Links
- [PraisonAI Documentation](https://docs.praison.ai)
- [PraisonAI Agents](https://github.com/MervinPraison/PraisonAI)
- [PyPI](https://pypi.org/project/praisonai-tools/)
- [GitHub](https://github.com/MervinPraison/PraisonAI-Tools)
- [Issues](https://github.com/MervinPraison/PraisonAI-Tools/issues)
+378
View File
@@ -0,0 +1,378 @@
# Inkog Security Analysis Integration
This document describes the integration of [Inkog](https://github.com/inkog-io/inkog) security analysis capabilities into PraisonAI-Tools.
## Overview
Inkog is a specialized security analysis tool for AI agent code that detects vulnerabilities unique to agentic systems. It performs static analysis to catch bugs that only break in agent code, such as:
- **Token bombing** - loops where the LLM controls termination, draining API budgets
- **Recursive tool calling** - one user request fans out into thousands of LLM invocations
- **Prompt injection sinks** - RAG output flowing into system prompts without review
- **Missing oversight** - destructive tools firing without human approval
- **Cross-tenant leakage** - global state shared between agent invocations
- **MCP tool poisoning** - malicious tool descriptions hijacking agents
## Features
### Framework Support
- **21+ frameworks**: LangChain, CrewAI, AutoGen, OpenAI Agents, LangGraph, DSPy, Phidata, and more
- **Multiple languages**: Python, TypeScript, JavaScript, and no-code platforms
- **MCP servers**: Audit Model Context Protocol servers for security issues
### Compliance Mapping
- **EU AI Act** Articles 14/15 (human oversight requirements)
- **NIST AI RMF** (risk management framework)
- **ISO 42001** (AI management systems)
- **OWASP LLM Top 10** (security vulnerabilities)
### Scan Types
- **Core scan**: Fast static analysis with vulnerability detection
- **Deep scan**: AI orchestrator-driven analysis with enhanced context
- **Skill scan**: Security analysis of SKILL.md packages and agent tools
- **MCP scan**: Audit MCP servers for tool poisoning and privilege escalation
## Installation
### Prerequisites
1. **Install Inkog CLI**:
```bash
# macOS (Homebrew)
brew tap inkog-io/inkog && brew install inkog
# Or install via Go
go install github.com/inkog-io/inkog/cmd/inkog@latest
# Or download binary from releases
# https://github.com/inkog-io/inkog/releases
```
2. **Get API Key**:
- Visit [app.inkog.io](https://app.inkog.io) to get a free API key
- Set environment variable: `export INKOG_API_KEY=your_key_here`
3. **Verify Installation**:
```bash
inkog -version
```
## Usage
### Basic Usage
```python
from praisonai_tools import InkogTool
# Initialize the tool
inkog = InkogTool()
# Scan current directory for vulnerabilities
result = inkog.scan_directory(
path=".",
output_format="json",
policy="balanced"
)
# Analyze results
analysis = inkog.analyze_findings(result)
print(analysis)
```
### Standalone Functions
```python
from praisonai_tools.tools.inkog_tool import scan_agent_code, scan_skill_package
# Quick scan using standalone function
result = scan_agent_code("./my-agent", policy="low-noise")
# Scan a skill package
skill_result = scan_skill_package("./my-skill", deep=True)
```
### Advanced Scanning
```python
# Deep scan with AI orchestrator (requires Inkog Deep role)
deep_result = inkog.scan_directory(
path="./agent-code",
output_format="json",
policy="comprehensive",
deep=True,
severity="high"
)
# Generate HTML report
html_report = inkog.scan_directory(
path="./agent-code",
output_format="html",
policy="balanced"
)
# Save report to file
with open("security_report.html", "w") as f:
f.write(html_report)
```
### MCP Server Auditing
```python
# Scan MCP server by name
mcp_result = inkog.mcp_scan("github")
# Scan MCP server with custom repository
mcp_result = inkog.mcp_scan(
"my-server",
repo="https://github.com/org/my-mcp-server",
deep=True
)
```
## Configuration
### Environment Variables
- **`INKOG_API_KEY`** (required): API key from app.inkog.io
- **`INKOG_SERVER_URL`** (optional): Custom server URL for enterprise deployments
### Security Policies
Choose the appropriate policy based on your use case:
- **`low-noise`**: Only exploitable vulnerabilities (good for CI/CD)
- **`balanced`**: Vulnerabilities + risk patterns (default, good for code review)
- **`comprehensive`**: All findings including hardening recommendations (good for security audits)
### Output Formats
- **`json`**: Structured data for programmatic processing
- **`text`**: Human-readable terminal output
- **`html`**: Interactive report with visualizations
- **`sarif`**: SARIF format for GitHub Security tab integration
### Severity Levels
- **`critical`**: Immediate security threats
- **`high`**: Serious vulnerabilities that should be addressed soon
- **`medium`**: Moderate issues worth reviewing
- **`low`**: Minor issues and best practice recommendations
## Integration Examples
### PraisonAI Agent Security Audit
```python
from praisonaiagents import Agent
from praisonai_tools import InkogTool
# Create an agent with security analysis capability
security_agent = Agent(
name="security_auditor",
instructions="You analyze AI agent code for security vulnerabilities",
tools=[InkogTool()]
)
# Audit agent code
audit_result = security_agent.start("""
Scan the ./my-agent directory for security vulnerabilities.
Focus on critical and high severity issues.
Provide a summary of findings and remediation recommendations.
""")
print(audit_result)
```
### CI/CD Pipeline Integration
```python
import sys
from praisonai_tools.tools.inkog_tool import scan_agent_code
def security_check():
"""Security check for CI/CD pipeline."""
result = scan_agent_code(
path=".",
policy="low-noise", # Only critical vulnerabilities
severity="critical"
)
if "error" in result:
print(f"Security scan failed: {result['error']}")
return 1
critical_count = result.get("summary", {}).get("critical", 0)
if critical_count > 0:
print(f"❌ Found {critical_count} critical security issues")
return 1
print("✅ No critical security vulnerabilities found")
return 0
if __name__ == "__main__":
sys.exit(security_check())
```
### Multi-Agent Security Review
```python
from praisonaiagents import Agent, AgentTeam
from praisonai_tools import InkogTool
# Security analysis agent
security_analyzer = Agent(
name="security_analyzer",
instructions="Analyze code for security vulnerabilities using Inkog",
tools=[InkogTool()]
)
# Code reviewer agent
code_reviewer = Agent(
name="code_reviewer",
instructions="Review security findings and provide remediation advice"
)
# Create security review team
security_team = AgentTeam(
name="security_review_team",
agents=[security_analyzer, code_reviewer],
tasks=[
{"agent": "security_analyzer", "task": "Scan the codebase for vulnerabilities"},
{"agent": "code_reviewer", "task": "Review findings and create remediation plan"}
]
)
# Run security review
review_result = security_team.start("Perform comprehensive security review of ./agent-code")
```
## Error Handling
The tool provides detailed error messages for common issues:
```python
result = inkog.scan_directory("./nonexistent")
if "error" in result:
error_msg = result["error"]
if "not installed" in error_msg:
print("Install inkog CLI first")
elif "API_KEY" in error_msg:
print("Set INKOG_API_KEY environment variable")
elif "timeout" in error_msg:
print("Scan timed out - try smaller directory or increase timeout")
else:
print(f"Scan error: {error_msg}")
```
## Best Practices
### 1. Development Workflow
```bash
# Add to your development script
inkog . -policy balanced -output json > security_report.json
```
### 2. Pre-commit Hooks
```yaml
# .pre-commit-config.yaml
repos:
- repo: local
hooks:
- id: inkog-security-scan
name: Inkog Security Scan
entry: inkog
args: [".", "-policy", "low-noise", "-severity", "critical"]
language: system
always_run: true
```
### 3. GitHub Actions
```yaml
name: Security Scan
on: [push, pull_request]
jobs:
security:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Install Inkog
run: |
curl -fsSL https://inkog.io/install.sh | sh
echo "$HOME/.local/bin" >> $GITHUB_PATH
- name: Run Security Scan
env:
INKOG_API_KEY: ${{ secrets.INKOG_API_KEY }}
run: inkog . -output sarif > results.sarif
- name: Upload SARIF
uses: github/codeql-action/upload-sarif@v2
with:
sarif_file: results.sarif
```
## Privacy & Security
Inkog uses "surgical redaction" to protect sensitive data:
### What Gets Redacted
- AWS access keys, GitHub tokens, Stripe keys
- Database connection strings with passwords
- Private keys and JWT tokens
- High-entropy strings (API keys)
### What is NOT Redacted
- Prompts and templates (needed for injection detection)
- Business logic (needed for flow analysis)
- Configuration values (model names, etc.)
- Normal strings that don't match credential patterns
## Troubleshooting
### Common Issues
1. **"inkog CLI not installed"**
- Install via Homebrew: `brew tap inkog-io/inkog && brew install inkog`
- Or download from releases: https://github.com/inkog-io/inkog/releases
2. **"INKOG_API_KEY required"**
- Get free key at https://app.inkog.io
- Set environment variable: `export INKOG_API_KEY=your_key`
3. **"Deep scan requires Inkog Deep role"**
- Deep scans need special account permissions
- Use standard scan instead: `deep=False`
4. **"Scan timed out"**
- Large codebases may timeout
- Scan specific directories instead of entire project
- Use policy="low-noise" for faster scans
### Debug Mode
Enable verbose output for troubleshooting:
```python
result = inkog.scan_directory(
path=".",
verbose=True # Shows detailed analysis steps
)
```
## Support
- **Documentation**: https://docs.inkog.io
- **GitHub Issues**: https://github.com/inkog-io/inkog/issues
- **Community**: https://discord.gg/NuG4SSGRH
- **Security**: security@inkog.io
## References
- [Inkog Repository](https://github.com/inkog-io/inkog)
- [State of AI Agent Security 2026 Report](https://inkog.io/report)
- [EU AI Act Compliance Guide](https://docs.inkog.io/compliance/eu-ai-act)
- [OWASP LLM Top 10](https://owasp.org/www-project-top-10-for-large-language-model-applications/)
+302
View File
@@ -0,0 +1,302 @@
# Joy Trust Integration for PraisonAI
Native Joy Trust Network integration for secure agent handoffs in PraisonAI.
## Overview
The Joy Trust integration provides automatic trust verification for agent handoffs, ensuring that tasks are only delegated to verified, trustworthy agents. This integration builds on the existing Joy Trust Network documentation and provides seamless, zero-configuration security for multi-agent workflows.
## Features
- **Automatic Trust Verification**: Environment variable configuration for seamless integration
- **Handoff Safety Checks**: Verify agent trustworthiness before delegation
- **Decorator-based Security**: `@trust_verified_handoff` decorator for secure functions
- **Configurable Thresholds**: Set minimum trust scores for different use cases
- **Caching**: Built-in result caching to reduce API calls
- **Fallback Handling**: Graceful degradation when trust verification fails
## Quick Start
### 1. Environment Variable Configuration
```bash
# Enable trust verification for all agents
export PRAISONAI_TRUST_PROVIDER=joy
# Set minimum trust score threshold (default: 3.0)
export PRAISONAI_TRUST_MIN_SCORE=3.5
# Enable automatic handoff verification (default: true)
export PRAISONAI_TRUST_AUTO_VERIFY=true
# Optional: Set Joy Trust API key for enhanced features
export JOY_TRUST_API_KEY=your_api_key_here
```
### 2. Basic Usage
```python
from praisonai_tools import check_trust_score, verify_handoff_safety
# Check an agent's trust score
result = check_trust_score("researcher_agent", min_score=3.0)
print(f"Trust Score: {result['trust_score']}")
print(f"Safe to use: {result['meets_threshold']}")
# Verify handoff safety with recommendations
verification = verify_handoff_safety("data_analyst", min_score=3.5)
print(f"Handoff Safe: {verification['handoff_safe']}")
print(f"Recommendation: {verification['recommendation']}")
```
### 3. Decorator-based Security
```python
from praisonai_tools import trust_verified_handoff
@trust_verified_handoff(min_score=4.0)
def delegate_analysis_task(agent_name: str, data: dict):
# This function only executes if the agent meets trust requirements
return perform_delegation(agent_name, data)
# The decorator automatically checks trust before execution
result = delegate_analysis_task("expert_analyst", {"data": "..."})
```
## API Reference
### Functions
#### `check_trust_score(agent_name: str, min_score: float = 3.0) -> dict`
Check an agent's trust score on the Joy Trust Network.
**Parameters:**
- `agent_name`: Name/identifier of the agent to check
- `min_score`: Minimum acceptable trust score
**Returns:**
```python
{
"agent_name": str,
"trust_score": float,
"verified": bool,
"meets_threshold": bool,
"threshold_used": float,
"reputation": dict,
"recommendations": int,
"error": str | None
}
```
#### `verify_handoff_safety(agent_name: str, min_score: float = 3.0) -> dict`
Verify if it's safe to hand off to the specified agent.
**Parameters:**
- `agent_name`: Target agent for handoff
- `min_score`: Minimum acceptable trust score
**Returns:**
```python
{
"agent_name": str,
"trust_score": float,
"verified": bool,
"handoff_safe": bool,
"recommendation": str,
"threshold_used": float,
"verification_time": float,
"error": str | None
}
```
#### `trust_verified_handoff(min_score: float = 3.0)`
Decorator for automatic trust verification before function execution.
**Parameters:**
- `min_score`: Minimum trust score required
**Usage:**
```python
@trust_verified_handoff(min_score=3.5)
def my_delegation_function(agent_name, task):
return delegate_task(agent_name, task)
```
#### `is_trust_verification_enabled() -> bool`
Check if trust verification is enabled via environment variables.
#### `get_trust_config() -> TrustConfig`
Get current trust configuration from environment variables.
### Classes
#### `JoyTrustTool`
Enhanced tool class for Joy Trust Network verification.
```python
from praisonai_tools import JoyTrustTool
tool = JoyTrustTool(api_key="optional_key")
result = tool.check_trust("agent_name")
```
#### `TrustConfig`
Configuration dataclass for Joy Trust integration.
```python
from praisonai_tools.tools.joy_trust_tool import TrustConfig
config = TrustConfig.from_env()
print(f"Enabled: {config.enabled}")
print(f"Min Score: {config.min_score}")
```
## Environment Variables
| Variable | Description | Default |
|----------|-------------|---------|
| `PRAISONAI_TRUST_PROVIDER` | Set to 'joy' to enable trust verification | None |
| `PRAISONAI_TRUST_MIN_SCORE` | Minimum trust score threshold | 3.0 |
| `PRAISONAI_TRUST_AUTO_VERIFY` | Enable automatic handoff verification | true |
| `PRAISONAI_TRUST_TIMEOUT` | Request timeout in seconds | 10.0 |
| `PRAISONAI_TRUST_CACHE_DURATION` | Cache duration in seconds | 300 |
| `PRAISONAI_TRUST_FALLBACK` | Allow fallback on errors | true |
| `JOY_TRUST_API_KEY` | Joy Trust API key (optional) | None |
## Trust Score Ranges
The Joy Trust Network uses a 0-5 scale for trust scores:
- **4.5-5.0**: Excellent - Highly trusted, verified agents
- **3.5-4.4**: Good - Reliable agents with good reputation
- **2.5-3.4**: Moderate - Acceptable with caution
- **1.0-2.4**: Low - Not recommended for important tasks
- **0.0-0.9**: Very Low - Strongly discouraged
## Integration Patterns
### 1. Workflow-level Integration
```python
import os
from praisonai_tools import verify_handoff_safety
# Enable trust verification
os.environ['PRAISONAI_TRUST_PROVIDER'] = 'joy'
def secure_workflow(agents: list, min_score: float = 3.0):
trusted_agents = []
for agent in agents:
verification = verify_handoff_safety(agent['name'], min_score)
if verification['handoff_safe']:
trusted_agents.append(agent)
else:
print(f"⚠️ Agent {agent['name']} failed trust check")
return trusted_agents
```
### 2. Conditional Trust Checking
```python
from praisonai_tools import is_trust_verification_enabled, check_trust_score
def delegate_with_optional_trust(agent_name: str, task: str):
if is_trust_verification_enabled():
trust_result = check_trust_score(agent_name)
if not trust_result['meets_threshold']:
return {"error": f"Agent {agent_name} doesn't meet trust requirements"}
return perform_delegation(agent_name, task)
```
### 3. Custom Trust Policies
```python
from praisonai_tools import JoyTrustTool
from praisonai_tools.tools.joy_trust_tool import TrustConfig
# Create custom configuration
custom_config = TrustConfig(
enabled=True,
min_score=4.0, # High security requirement
timeout_seconds=5.0, # Fast timeout
fallback_on_error=False # Strict mode
)
tool = JoyTrustTool(config=custom_config)
result = tool.verify_handoff_safety("critical_agent")
```
## Error Handling
The integration provides robust error handling:
```python
from praisonai_tools import check_trust_score
result = check_trust_score("unknown_agent")
if result['error']:
print(f"Trust check failed: {result['error']}")
if result.get('fallback_used'):
print("Using fallback behavior")
else:
print("Strict mode - blocking delegation")
else:
print(f"Trust verification successful: {result['trust_score']}")
```
## Best Practices
1. **Start with moderate thresholds** (3.0-3.5) and adjust based on your security needs
2. **Use environment variables** for configuration to avoid hardcoding values
3. **Enable caching** in production to reduce API calls and improve performance
4. **Set appropriate timeouts** based on your application's requirements
5. **Use fallback mode** for non-critical operations to maintain availability
6. **Monitor trust scores** and update agent permissions accordingly
## Troubleshooting
### Common Issues
1. **"httpx is required" error**: Install httpx with `pip install httpx` or `pip install praisonai-tools[marketplace]`
2. **Connection timeouts**: Increase `PRAISONAI_TRUST_TIMEOUT` value
3. **False negatives**: Check if agent names match Joy Trust Network entries exactly
4. **API rate limits**: Enable caching with `PRAISONAI_TRUST_CACHE_DURATION`
### Debug Mode
Enable debug logging to troubleshoot issues:
```python
import logging
logging.basicConfig(level=logging.DEBUG)
from praisonai_tools import check_trust_score
result = check_trust_score("debug_agent")
```
## Contributing
To contribute to the Joy Trust integration:
1. Fork the PraisonAI-Tools repository
2. Create a feature branch
3. Add tests for new functionality
4. Submit a pull request
## Support
- **Documentation**: [Joy Trust Network Docs](https://docs.praison.ai/joy-trust-network)
- **Issues**: [PraisonAI-Tools Issues](https://github.com/MervinPraison/PraisonAI-Tools/issues)
- **Community**: [PraisonAI Discord](https://discord.gg/praisonai)
+399
View File
@@ -0,0 +1,399 @@
# n8n Integration for PraisonAI Agents
Enable bidirectional integration between PraisonAI agents and n8n workflow automation platform, providing access to **400+ integrations** including Slack, Gmail, Notion, databases, APIs, and more.
## Overview
The n8n integration allows:
- **PraisonAI → n8n**: Agents can execute n8n workflows to access 400+ integrations
- **n8n → PraisonAI**: n8n workflows can invoke PraisonAI agents (see API endpoint section)
## Installation
```bash
# Install with n8n support
pip install "praisonai-tools[n8n]"
```
## Setup
### 1. n8n Instance
**Local Development:**
```bash
# Run n8n locally with Docker
docker run -it --rm --name n8n -p 5678:5678 -v n8n_data:/home/node/.n8n docker.n8n.io/n8nio/n8n
```
**Production:**
- Use [n8n Cloud](https://n8n.cloud) (recommended)
- Or self-host n8n on your infrastructure
### 2. Environment Variables
```bash
export N8N_URL="http://localhost:5678" # Your n8n instance URL
export N8N_API_KEY="your-api-key" # Optional for local, required for production
```
### 3. Create API Key (Production)
1. Open n8n instance
2. Go to Settings → API
3. Generate new API key
4. Set `N8N_API_KEY` environment variable
## Quick Start
### Basic Usage
```python
from praisonai_tools.n8n import n8n_workflow, n8n_list_workflows
# List available workflows
workflows = n8n_list_workflows()
print(f"Available workflows: {len(workflows.get('data', []))}")
# Execute a workflow
result = n8n_workflow(
workflow_id="your-workflow-id",
input_data={"message": "Hello from PraisonAI!"}
)
```
### With PraisonAI Agents
```python
from praisonaiagents import Agent
from praisonai_tools.n8n import n8n_workflow, n8n_list_workflows
# Create agent with n8n tools
agent = Agent(
name="automation-agent",
instructions="You help automate tasks using n8n workflows with 400+ integrations",
tools=[n8n_workflow, n8n_list_workflows],
)
# Agent can now use n8n workflows
response = agent.start("Send a Slack message to #general saying 'Hello team!'")
```
## Available Tools
### `n8n_workflow`
Execute an n8n workflow and return the result.
**Parameters:**
- `workflow_id` (str): The n8n workflow ID to execute
- `input_data` (dict, optional): Input data to pass to the workflow
- `n8n_url` (str, optional): n8n instance URL (defaults to `N8N_URL` env var)
- `api_key` (str, optional): n8n API key (defaults to `N8N_API_KEY` env var)
- `timeout` (float, optional): Request timeout in seconds (default: 60)
- `wait_for_completion` (bool, optional): Wait for workflow completion (default: True)
**Returns:** Dict with execution result or error information
**Example:**
```python
# Send Slack message via n8n workflow
result = n8n_workflow(
workflow_id="slack-notify",
input_data={
"channel": "#general",
"message": "Hello from PraisonAI!"
}
)
# Create Notion page via n8n workflow
result = n8n_workflow(
workflow_id="notion-create-page",
input_data={
"title": "Meeting Notes",
"content": "Discussion about project timeline"
}
)
```
### `n8n_list_workflows`
List available n8n workflows.
**Parameters:**
- `n8n_url` (str, optional): n8n instance URL
- `api_key` (str, optional): n8n API key
**Returns:** Dict containing list of workflows or error information
**Example:**
```python
workflows = n8n_list_workflows()
for workflow in workflows.get("data", []):
print(f"- {workflow['name']} (ID: {workflow['id']})")
```
## Common Use Cases
### 1. Communication Automation
```python
# Slack notifications
n8n_workflow("slack-notify", {
"channel": "#alerts",
"message": "Task completed successfully!",
"emoji": ":white_check_mark:"
})
# Email with Gmail
n8n_workflow("gmail-send", {
"to": "[email protected]",
"subject": "Report Ready",
"body": "Your daily report is attached."
})
```
### 2. Data Management
```python
# Add to Google Sheets
n8n_workflow("sheets-append", {
"spreadsheet_id": "1BxiMVs0XRA5nFMdKvBdBZjgmUUqptlbs74OgvE2upms",
"range": "Sheet1!A1:E1",
"values": [["Date", "Name", "Status", "Notes"]]
})
# Create Notion page
n8n_workflow("notion-create", {
"database_id": "your-database-id",
"properties": {
"Title": {"title": [{"text": {"content": "New Task"}}]},
"Status": {"select": {"name": "To Do"}}
}
})
```
### 3. Task Management
```python
# Create Jira ticket
n8n_workflow("jira-create-issue", {
"project": "PROJ",
"summary": "Bug report from agent",
"description": "Issue details...",
"issue_type": "Bug"
})
# Add Trello card
n8n_workflow("trello-add-card", {
"list_id": "your-list-id",
"name": "New Task",
"desc": "Task description"
})
```
## Agent Examples
### Multi-Platform Notification Agent
```python
agent = Agent(
name="notifier",
instructions="""
You send notifications across multiple platforms using n8n workflows.
Available workflows:
- slack-notify: Send Slack messages
- discord-webhook: Send Discord messages
- email-send: Send emails
- teams-notify: Send Microsoft Teams messages
Always ask which platform and customize the message appropriately.
""",
tools=[n8n_workflow, n8n_list_workflows]
)
```
### Data Sync Agent
```python
agent = Agent(
name="data-sync",
instructions="""
You help synchronize data between different systems using n8n.
You can:
- Sync data between databases
- Update spreadsheets
- Create/update CRM records
- Generate reports
Always confirm the source and destination before proceeding.
""",
tools=[n8n_workflow]
)
```
## Error Handling
The n8n tools provide detailed error information:
```python
result = n8n_workflow("invalid-workflow")
if "error" in result:
print(f"Error: {result['error']}")
# Common errors:
# - "workflow_id is required"
# - "httpx not installed. Install with: pip install 'praisonai-tools[n8n]'"
# - "HTTP 404: Workflow not found"
# - "Workflow execution timed out after 60 seconds"
```
## Configuration
### Timeout Settings
```python
# Short timeout for quick workflows
result = n8n_workflow(
workflow_id="quick-task",
timeout=10.0
)
# Long timeout for complex workflows
result = n8n_workflow(
workflow_id="data-processing",
timeout=300.0 # 5 minutes
)
```
### Async vs Sync Execution
```python
# Wait for completion (default)
result = n8n_workflow("workflow-id", wait_for_completion=True)
# Fire-and-forget
result = n8n_workflow("workflow-id", wait_for_completion=False)
# Returns immediately with executionId
```
## Integration with n8n → PraisonAI
To enable n8n workflows to invoke PraisonAI agents, add this endpoint to your PraisonAI application:
```python
from fastapi import FastAPI, HTTPException
from pydantic import BaseModel
from praisonaiagents import Agent
app = FastAPI()
class InvokeRequest(BaseModel):
agent_name: str
message: str
session_id: str = None
@app.post("/api/v1/agents/{agent_id}/invoke")
async def invoke_agent(agent_id: str, request: InvokeRequest):
"""Endpoint for n8n to invoke PraisonAI agents."""
# Load your agent
agent = get_agent(agent_id) # Your agent loading logic
if not agent:
raise HTTPException(404, f"Agent {agent_id} not found")
result = await agent.astart(request.message)
return {"result": result, "session_id": request.session_id or "default"}
```
Then use n8n's HTTP Request node to call this endpoint.
## Best Practices
### 1. Workflow Design
- Create focused workflows for specific tasks
- Use descriptive workflow names and IDs
- Add error handling in your n8n workflows
- Test workflows manually before using with agents
### 2. Security
- Use API keys for production environments
- Implement input validation in n8n workflows
- Consider rate limiting for agent-triggered workflows
- Audit workflow executions regularly
### 3. Performance
- Use appropriate timeouts based on workflow complexity
- Consider async execution for long-running workflows
- Monitor n8n instance performance
- Cache workflow lists when possible
### 4. Error Handling
- Always check for error responses
- Implement retry logic for transient failures
- Log workflow execution results
- Provide fallback options when workflows fail
## Troubleshooting
### Common Issues
**Connection Errors:**
```python
# Check n8n connectivity
result = n8n_list_workflows()
if "error" in result:
print("n8n connection issue:", result["error"])
```
**Workflow Not Found:**
```python
# Verify workflow exists
workflows = n8n_list_workflows()
workflow_ids = [w["id"] for w in workflows.get("data", [])]
print("Available workflows:", workflow_ids)
```
**Authentication Issues:**
```python
# Check API key configuration
import os
api_key = os.getenv("N8N_API_KEY")
if not api_key:
print("N8N_API_KEY not set - OK for local testing")
else:
print("API key configured")
```
### Debug Mode
Enable detailed logging:
```python
import logging
logging.basicConfig(level=logging.DEBUG)
# Now n8n tool calls will show detailed logs
result = n8n_workflow("test-workflow")
```
## Resources
- [n8n Documentation](https://docs.n8n.io/)
- [n8n API Reference](https://docs.n8n.io/api/)
- [n8n Workflow Templates](https://n8n.io/workflows)
- [n8n Integrations](https://n8n.io/integrations)
- [PraisonAI Documentation](https://docs.praison.ai)
## Contributing
To contribute to the n8n integration:
1. Fork the [PraisonAI-Tools repository](https://github.com/MervinPraison/PraisonAI-Tools)
2. Create your feature branch
3. Add tests for your changes
4. Update documentation
5. Submit a pull request
## License
This integration is released under the MIT License, same as PraisonAI Tools.
+382
View File
@@ -0,0 +1,382 @@
# NEXUS Prediction Market Integration
This document provides comprehensive information about integrating the NEXUS prediction market API with PraisonAI agents.
## Overview
NEXUS provides live Kalshi/Polymarket prediction market data and arbitrage opportunities through a validated API. The integration allows PraisonAI agents to:
- Access real-time prediction market data
- Identify arbitrage opportunities between markets
- Analyze market sentiment and trends
- Build sophisticated trading and analysis workflows
## API Information
### Endpoints
- **Free Market Data**: `/kalshi?market={market_name}`
- **Paid Arbitrage Analysis**: `/arb/check?markets={market1,market2}` ($0.02 USDC on eip155:8453)
- **Agent Metadata**: `/.well-known/agent.json`
### Validation
The API is fully validated on Agentic Market:
- **Validation URL**: https://agentic.market/validate?url=https://nexus-agent-xa12.onrender.com/arb/check
- **API Card**: https://nexus-agent-xa12.onrender.com/.well-known/agent.json
### Usage Tracking
Include the `X-NEXUS-Source` header to track per-source call volume:
```
X-NEXUS-Source: your-bot-name
```
## Installation
```bash
pip install praisonai-tools
```
## Quick Start
### Basic Usage
```python
from praisonai_tools.tools import NexusPredictionMarketTool
# Initialize the tool
nexus = NexusPredictionMarketTool(source_name="my-agent")
# Get market data
fed_data = nexus.get_market_data("Fed")
print(fed_data)
# Check arbitrage opportunities (paid)
arb_data = nexus.check_arbitrage_opportunities(["Fed", "BTC"])
print(arb_data)
```
### Agent Integration
```python
from praisonaiagents import Agent, tool
from praisonai_tools.tools import nexus_prediction_market_tool
@tool
def get_market_data(market: str) -> str:
"""Get prediction market data."""
return nexus_prediction_market_tool("get_market_data", market=market)
agent = Agent(
name="MarketAnalyst",
tools=[get_market_data],
instructions="Analyze prediction markets using NEXUS data."
)
response = agent.start("What do Fed rate predictions show?")
```
## Features
### Free Features
1. **Market Data Queries**
- Real-time price data
- Volume and liquidity information
- Market sentiment indicators
- Historical trends
2. **Agent Metadata**
- API capabilities and endpoints
- Service status and version info
- Usage guidelines
### Paid Features
1. **Arbitrage Analysis** ($0.02 USDC on eip155:8453)
- Cross-market opportunity detection
- Profit potential calculations
- Risk assessment
- Execution recommendations
## Supported Markets
Common market identifiers include:
- `Fed` - Federal Reserve interest rate predictions
- `BTC` - Bitcoin price predictions
- `Election` - Political election outcomes
- `ETH` - Ethereum price predictions
- `Inflation` - Inflation rate predictions
*Note: Market availability may vary. Check current markets via the API.*
## API Reference
### NexusPredictionMarketTool Class
```python
class NexusPredictionMarketTool:
def __init__(self, source_name: Optional[str] = None, api_key: Optional[str] = None)
def get_market_data(self, market: str) -> Dict[str, Any]
def check_arbitrage_opportunities(self, markets: List[str]) -> Dict[str, Any]
def get_agent_info(self) -> Dict[str, Any]
def run(self, action: str, market: Optional[str] = None, markets: Optional[List[str]] = None) -> str
```
### Function Interface
```python
def nexus_prediction_market_tool(
action: str,
market: Optional[str] = None,
markets: Optional[List[str]] = None,
source_name: Optional[str] = None
) -> str
```
## Examples
### Market Analysis Agent
```python
from praisonaiagents import Agent
from praisonai_tools.tools import NexusPredictionMarketTool
class MarketAnalysisAgent:
def __init__(self):
self.nexus = NexusPredictionMarketTool(source_name="analysis-bot")
def analyze_market_sentiment(self, market):
"""Analyze market sentiment for a specific prediction market."""
data = self.nexus.get_market_data(market)
if data["success"]:
market_data = data["data"]
# Analyze price, volume, trends etc.
return f"Market sentiment for {market}: {self.interpret_data(market_data)}"
else:
return f"Error getting data: {data['error']}"
def interpret_data(self, data):
"""Custom logic to interpret market data."""
# Implementation specific to your analysis needs
pass
```
### Multi-Agent Workflow
```python
from praisonaiagents import Agent, AgentTeam, Task
# Data collector agent
collector = Agent(
name="DataCollector",
tools=[get_market_data],
instructions="Collect prediction market data from NEXUS."
)
# Analyst agent
analyst = Agent(
name="Analyst",
instructions="Analyze market data and provide insights."
)
# Create workflow
team = AgentTeam(agents=[collector, analyst])
# Define tasks
collect_task = Task(
name="collect_data",
description="Collect data for Fed and BTC markets",
agent=collector
)
analyze_task = Task(
name="analyze_trends",
description="Analyze the collected market data",
agent=analyst,
dependencies=[collect_task]
)
# Execute workflow
results = team.run(tasks=[collect_task, analyze_task])
```
## Error Handling
The tool provides comprehensive error handling for common scenarios:
### Network Errors
```python
{
"success": False,
"error": "Failed to fetch market data: Connection timeout",
"market": "Fed"
}
```
### Payment Required (Arbitrage Features)
```python
{
"success": False,
"error": "Payment required for arbitrage analysis. Cost: $0.02 USDC on eip155:8453",
"markets": ["Fed", "BTC"],
"payment_info": "This feature requires payment on eip155:8453 blockchain"
}
```
### Invalid Parameters
```python
{
"success": False,
"error": "Market parameter required for get_market_data action"
}
```
## Best Practices
### 1. Source Identification
Always set a descriptive source name for tracking:
```python
nexus = NexusPredictionMarketTool(source_name="trading-bot-v1")
```
### 2. Error Handling
Always check the `success` field before processing results:
```python
result = nexus.get_market_data("Fed")
if result["success"]:
data = result["data"]
# Process data
else:
print(f"Error: {result['error']}")
```
### 3. Rate Limiting
Implement appropriate delays between requests to avoid overwhelming the API:
```python
import time
import asyncio
async def batch_collect_data(markets):
data = {}
for market in markets:
result = nexus.get_market_data(market)
data[market] = result
await asyncio.sleep(1) # 1 second delay between requests
return data
```
### 4. Caching
Cache results for frequently accessed data:
```python
from functools import lru_cache
import time
@lru_cache(maxsize=100)
def cached_market_data(market, timestamp):
return nexus.get_market_data(market)
def get_cached_data(market, cache_duration=300): # 5 minutes
timestamp = int(time.time() // cache_duration)
return cached_market_data(market, timestamp)
```
## Security Considerations
### 1. API Key Management
If using paid features, securely manage your API keys:
```python
import os
api_key = os.getenv("NEXUS_API_KEY")
nexus = NexusPredictionMarketTool(api_key=api_key)
```
### 2. Input Validation
Validate market identifiers before making requests:
```python
VALID_MARKETS = ["Fed", "BTC", "ETH", "Election", "Inflation"]
def validate_market(market):
if market not in VALID_MARKETS:
raise ValueError(f"Invalid market: {market}")
return market
```
### 3. Rate Limiting
Implement client-side rate limiting to prevent abuse:
```python
from time import time, sleep
class RateLimiter:
def __init__(self, max_calls_per_minute=60):
self.max_calls = max_calls_per_minute
self.calls = []
def wait_if_needed(self):
now = time()
self.calls = [call_time for call_time in self.calls if now - call_time < 60]
if len(self.calls) >= self.max_calls:
sleep(1)
self.calls.append(now)
```
## Troubleshooting
### Common Issues
1. **Connection Timeout**
- Check internet connectivity
- Verify the API endpoint is accessible
- Implement retry logic with exponential backoff
2. **Invalid Market Identifier**
- Verify the market name is correct
- Check available markets via agent info endpoint
- Use exact case-sensitive market names
3. **Payment Required Errors**
- Ensure you have sufficient USDC balance on eip155:8453
- Verify payment setup for arbitrage features
- Contact NEXUS support if payment issues persist
4. **JSON Decode Errors**
- Check API response format
- Verify the endpoint returned valid JSON
- Implement fallback error handling
### Debug Mode
Enable debug logging to troubleshoot issues:
```python
import logging
logging.basicConfig(level=logging.DEBUG)
nexus = NexusPredictionMarketTool(source_name="debug-agent")
# Requests will now be logged with full details
```
## Support and Resources
- **API Documentation**: https://nexus-agent-xa12.onrender.com/.well-known/agent.json
- **Validation Check**: https://agentic.market/validate?url=https://nexus-agent-xa12.onrender.com/arb/check
- **PraisonAI Documentation**: https://docs.praison.ai
- **Issue Tracking**: Report issues via GitHub issues in the PraisonAI-Tools repository
## Contributing
To contribute improvements or report bugs:
1. Fork the PraisonAI-Tools repository
2. Create a feature branch
3. Make your changes
4. Add tests for new functionality
5. Submit a pull request
## License
This tool is part of the PraisonAI-Tools package and follows the same licensing terms.
+141
View File
@@ -0,0 +1,141 @@
# Tool-Registry Proxy Connector
An **optional, bring-your-own-account** connector that lets any PraisonAI agent
discover and call a large catalogue of third-party API endpoints (SEO,
enrichment, social data, scraping, ads) through a single proxy token — with
**credentials injected server-side by the registry, never held by the agent**.
PraisonAI ships **no registry code, no bundled binary and no default endpoint**.
You supply your own account token or a self-hosted base URL. The connector is
disabled unless `TOOL_PROXY_URL` is configured.
## Overview
The connector mirrors PraisonAI's deferred tool-search bridge
(`tool_search` / `tool_describe` / `tool_call`) with three functions:
| Function | Cost | Purpose |
| --- | --- | --- |
| `registry_search(query)` | free | capability search over the catalogue |
| `registry_describe(tool_id)` | free | params, price-per-call, example response |
| `registry_call(tool_id, params)` | paid | invoke via proxy; credential injected server-side |
## Installation
```bash
pip install "praisonai-tools[registry-proxy]"
```
## Setup (3 lines)
```bash
export TOOL_PROXY_URL="https://your-registry.example.com" # or a self-hosted instance
export TOOL_PROXY_TOKEN="your-proxy-token"
# Optional: override the auth header for registries that expect a custom header.
# Default is "Authorization" (sent as "Bearer <token>"); any other name (e.g.
# "X-Treg-Token") sends the raw token in that header.
export TOOL_PROXY_AUTH_HEADER="Authorization"
# Optional: override the describe route for registries with a different layout.
# Must contain "{tool_id}"; defaults to the live registry route below.
export TOOL_PROXY_DESCRIBE_PATH="/catalog/endpoints/{tool_id}"
```
```python
from praisonaiagents import Agent
from praisonai_tools.registry_proxy import (
registry_search, registry_describe, registry_call,
)
agent = Agent(
instructions="SEO analyst",
tools=[registry_search, registry_describe, registry_call],
)
agent.start("Find the top backlink sources for example.com and summarise")
# → registry_search("backlinks for a domain") → describe → call
# one token, no vendor keys in the agent
```
## Spend safety
`registry_call` accepts optional budget guards. When a configured budget would
be exceeded, the call is **denied and reported** as a tool-result error (never
raised as an exception):
```python
registry_call(
"seo.backlinks",
{"domain": "example.com"},
max_cost_per_call=0.10, # deny if price-per-call exceeds this
max_session_spend=5.00, # deny if cumulative spend would exceed this
)
```
Per-call price is read from `registry_describe`, so guards work without any
vendor-specific configuration. Guards **fail closed**: if a budget is set but
the price cannot be validated, the call is denied. Cumulative `max_session_spend`
persists across separate `registry_call` invocations within the same process,
scoped per `(proxy_url, token)` so distinct accounts never share a budget.
## HTTP methods
The registry enforces each endpoint's declared HTTP method on `/call/{id}`.
`registry_call` derives the method automatically from `registry_describe`
(the endpoint doc carries `method`): `GET` endpoints send `params` as the query
string, `POST`/`PUT` send them as a JSON body. Pass `method="GET"` explicitly to
skip the describe lookup when you already know the method. Only the standard
`GET`/`POST`/`PUT`/`PATCH`/`DELETE` verbs are accepted; any other value (e.g. a
mislabelled endpoint doc) is rejected as a tool-result error before any request.
`tool_id` is URL-encoded before being placed in the describe/call route, so it
cannot alter the requested path.
## Passthrough-URL calls
Passthrough-URL calls require the upstream to be **registered on the registry
first**. Loopback/private upstreams (e.g. `127.0.0.1`, RFC-1918 addresses) are
**refused by the registry's SSRF guard by design** — the base URL must be a
public `http(s)` address. This is registry-side policy, not a connector setting.
## Credential safety
The proxy token is only sent to the endpoint it was configured for. If a caller
(or a prompt-injected agent) supplies an explicit `proxy_url` without a matching
`token`, the ambient `TOOL_PROXY_TOKEN` is **not** attached — preventing the
token from being exfiltrated to an untrusted endpoint. For normal use, leave
`proxy_url`/`token` unset and configure `TOOL_PROXY_URL`/`TOOL_PROXY_TOKEN` via
the environment.
## Error taxonomy
All failures are returned as `{"error": ...}` tool results:
- Authentication failed (HTTP 401/403)
- Insufficient balance (HTTP 402)
- Upstream 4xx/5xx passed through
- Request timeout
- `httpx` not installed → install hint
## Tier 0 — Skills directory compatibility (zero code)
Registries that expose `skill install <name>` install skills into
`./.claude/skills/`. PraisonAI **already scans** this directory, so
registry-installed skills are discoverable by PraisonAI agents today with no
additional code:
- `praisonaiagents/skills/discovery.py` — `.claude/skills` compatibility scan
plus an ancestor walk for monorepos.
## Tier 2 — Environment / CLI passthrough (config-only)
A registry's `shell` / `run <vendor> -- args` inject vendor-CLI credentials at
execution time. Using a PraisonAI environment definition, install the registry
CLI and log in via a token env var in `setup:`; the agent's `execute_command`
then has authenticated vendor CLIs (e.g. `run gh -- pr list`) without any key in
the container. Sandbox / board workers inherit the same via the environment
definition.
## Licensing note
Some registries are Apache-2.0 **with additional terms** prohibiting embedding
in commercially distributed products without permission. This connector is
therefore **optional and bring-your-own-account only**: it ships no registry
code and no default endpoint — the same posture as other BYO-account connectors.
+357
View File
@@ -0,0 +1,357 @@
# SwarmScore Integration for PraisonAI
SwarmScore is a portable trust rating system for AI agents that provides cryptographically signed reputation scores based on verified execution history. This integration allows PraisonAI agents to leverage trust ratings for enhanced security and decision-making.
## Overview
SwarmScore provides:
- **Portable Trust Ratings**: Cryptographically signed scores that travel with your agent across platforms
- **Verified Execution History**: Trust based on actual performance, not self-reported metrics
- **Consistency Tracking**: 80 jobs at 95% success rate beats 1 job at 100%
- **Agent-to-Agent Discovery**: Machine-readable trust data for autonomous agent interactions
## Installation
```bash
pip install praisonai-tools[swarmscore]
# or
pip install praisonai-tools requests
```
## Quick Start
### Basic Usage
```python
from praisonai_tools.tools import SwarmScoreTool
# Initialize the tool
swarmscore = SwarmScoreTool()
# Load your agent's trust score
result = swarmscore.load_swarmscore("your-agent-slug")
if result.success:
score_data = result.output
print(f"Trust Score: {score_data['score']}")
print(f"Trust Tier: {score_data['tier']}")
print(f"Success Rate: {score_data['success_rate']}%")
else:
print(f"Error: {result.error}")
```
### Standalone Functions
```python
from praisonai_tools.tools import (
load_swarmscore_by_slug,
verify_swarmscore_freshness,
get_agent_discovery_manifest
)
# Load score data
try:
score_data = load_swarmscore_by_slug("your-agent-slug")
print(f"Score: {score_data['score']}")
except Exception as e:
print(f"Failed to load score: {e}")
# Verify score freshness
try:
verification = verify_swarmscore_freshness(score_data['verify_payload'])
print(f"Verification: {verification}")
except Exception as e:
print(f"Verification failed: {e}")
# Get discovery manifest
try:
manifest = get_agent_discovery_manifest()
print(f"Discovery data: {manifest}")
except Exception as e:
print(f"Failed to get manifest: {e}")
```
## Integration Patterns
### 1. Trust-Based Decision Making
```python
from praisonai_tools.tools import SwarmScoreTool
def execute_with_trust_level(agent_slug, task):
"""Execute task based on agent's trust level."""
swarmscore = SwarmScoreTool()
result = swarmscore.load_swarmscore(agent_slug)
if not result.success:
# Default to restricted mode if score unavailable
return execute_restricted(task)
trust_score = result.output.get('score', 0)
if trust_score >= 80:
# High trust - full autonomy
return execute_autonomous(task)
elif trust_score >= 60:
# Medium trust - human oversight
return execute_supervised(task)
else:
# Low trust - restricted operations
return execute_restricted(task)
```
### 2. Agent Authentication
```python
def authenticate_agent(agent_slug):
"""Verify agent identity and trust level."""
swarmscore = SwarmScoreTool()
# Load and verify score
load_result = swarmscore.load_swarmscore(agent_slug)
if not load_result.success:
return False, "Failed to load trust score"
# Verify freshness
verify_payload = load_result.output.get('verify_payload')
if verify_payload:
verify_result = swarmscore.verify_swarmscore(verify_payload)
if not verify_result.success:
return False, "Trust score verification failed"
trust_score = load_result.output.get('score', 0)
return trust_score >= 50, f"Trust score: {trust_score}"
```
### 3. Multi-Agent Coordination
```python
def find_trusted_collaborators(task_requirements):
"""Find agents suitable for collaboration based on trust scores."""
swarmscore = SwarmScoreTool()
# Get discovery manifest for agent lookup
manifest_result = swarmscore.get_discovery_manifest()
if not manifest_result.success:
return []
trusted_agents = []
discovery_data = manifest_result.output
# Example: Filter agents by trust score and capabilities
for agent_id in discovery_data.get('agents', []):
score_result = swarmscore.load_swarmscore(agent_id)
if score_result.success:
score_data = score_result.output
if score_data.get('score', 0) >= 70:
trusted_agents.append({
'id': agent_id,
'score': score_data['score'],
'capabilities': score_data.get('capabilities', [])
})
return trusted_agents
```
## PraisonAI Agent Integration
### Using with PraisonAI Agent Tools
```python
from praisonaiagents import Agent, tool
from praisonai_tools.tools import SwarmScoreTool
@tool
def check_agent_trust_score(agent_slug: str) -> str:
"""Check the trust score for an agent."""
swarmscore = SwarmScoreTool()
result = swarmscore.load_swarmscore(agent_slug)
if result.success:
score_data = result.output
return f"Agent {agent_slug} has trust score {score_data.get('score', 'unknown')} with {score_data.get('success_rate', 'unknown')}% success rate"
else:
return f"Failed to load trust score for {agent_slug}: {result.error}"
# Create agent with SwarmScore capability
agent = Agent(
name="trust-aware-agent",
instructions="You can check agent trust scores using the check_agent_trust_score tool.",
tools=[check_agent_trust_score]
)
# Use the agent
response = agent.start("What is the trust score for agent 'trading-bot-v2'?")
print(response)
```
### Agent with Built-in Trust Management
```python
from praisonaiagents import Agent
from praisonai_tools.tools import SwarmScoreTool
class TrustAwareAgent(Agent):
"""PraisonAI Agent with built-in SwarmScore integration."""
def __init__(self, agent_slug: str, **kwargs):
super().__init__(**kwargs)
self.agent_slug = agent_slug
self.swarmscore = SwarmScoreTool()
self._trust_score = None
async def check_trust_level(self):
"""Check and cache current trust level."""
result = self.swarmscore.load_swarmscore(self.agent_slug)
if result.success:
self._trust_score = result.output.get('score', 0)
return self._trust_score
def can_execute_action(self, action_type: str) -> bool:
"""Determine if action can be executed based on trust level."""
if self._trust_score is None:
return False # Default to safe
# Define trust requirements for different actions
trust_requirements = {
'read': 30,
'write': 60,
'execute': 80,
'admin': 95
}
required_trust = trust_requirements.get(action_type, 100)
return self._trust_score >= required_trust
# Usage
agent = TrustAwareAgent(
agent_slug="my-agent-123",
name="secure-agent",
instructions="Execute tasks based on trust level"
)
# Check trust before executing
trust_score = await agent.check_trust_level()
if agent.can_execute_action('execute'):
print("Proceeding with high-trust execution")
else:
print("Restricted to low-trust operations")
```
## API Reference
### SwarmScoreTool Class
#### Methods
**`__init__(api_base_url: str = "https://api.swarmsync.ai/v1/swarmscore/")`**
- Initialize SwarmScore tool
- `api_base_url`: Base URL for SwarmScore API
**`load_swarmscore(slug: str) -> ToolResult`**
- Load SwarmScore data by agent slug
- Returns: ToolResult with passport, certificate, and verification data
**`verify_swarmscore(verify_payload: Dict[str, Any]) -> ToolResult`**
- Verify SwarmScore freshness using verification payload
- Returns: ToolResult with verification status
**`get_discovery_manifest() -> ToolResult`**
- Get machine-readable agent discovery manifest
- Returns: ToolResult with discovery data
**`run(action: str, **kwargs) -> ToolResult`**
- Execute SwarmScore action
- Actions: 'load', 'verify', 'discover'
### Standalone Functions
**`load_swarmscore_by_slug(slug: str) -> Dict[str, Any]`**
- Load SwarmScore data by agent slug
- Raises: Exception if loading fails
**`verify_swarmscore_freshness(verify_payload: Dict[str, Any]) -> Dict[str, Any]`**
- Verify SwarmScore freshness
- Raises: Exception if verification fails
**`get_agent_discovery_manifest() -> Dict[str, Any]`**
- Get agent discovery manifest
- Raises: Exception if retrieval fails
## SwarmScore Data Structure
```json
{
"score": 85,
"tier": "gold",
"jobs_completed": 150,
"success_rate": 94.5,
"consistency_score": 88,
"signed_certificate": "...",
"verify_payload": {
"signature": "...",
"timestamp": "...",
"agent_id": "..."
},
"capabilities": ["trading", "analysis", "reporting"],
"last_updated": "2024-01-15T10:30:00Z"
}
```
## Error Handling
```python
from praisonai_tools.tools import SwarmScoreTool
swarmscore = SwarmScoreTool()
result = swarmscore.load_swarmscore("agent-123")
if not result.success:
error_msg = result.error
if "network" in error_msg.lower():
# Handle network issues
print("Network connectivity problem")
elif "not found" in error_msg.lower():
# Handle missing agent
print("Agent not registered in SwarmScore")
else:
# Handle other errors
print(f"Unexpected error: {error_msg}")
```
## Best Practices
1. **Cache Trust Scores**: Cache scores temporarily to avoid excessive API calls
2. **Graceful Degradation**: Have fallback behavior when SwarmScore is unavailable
3. **Verify Freshness**: Always verify score freshness for critical decisions
4. **Trust Thresholds**: Define clear trust score requirements for different operations
5. **Error Handling**: Handle network and API errors gracefully
## Getting Started with SwarmScore
1. **Register**: Create an account at [SwarmSync.AI](https://swarmsync.ai)
2. **Get Your Slug**: Obtain your agent identifier from the dashboard
3. **Start Building**: Begin tracking execution history to build trust
4. **Integrate**: Use this tool to incorporate trust ratings into your workflows
## Resources
- [SwarmScore Documentation](https://swarmsync.ai/docs/protocol-specs/swarmscore)
- [SwarmScore GitHub Spec](https://github.com/swarmsync-ai/swarmscore-spec)
- [SwarmSync.AI Platform](https://swarmsync.ai)
- [Example Implementation](../examples/swarmscore_example.py)
## Contributing
To contribute improvements to the SwarmScore integration:
1. Fork the PraisonAI-Tools repository
2. Create a feature branch
3. Make your changes
4. Add tests for new functionality
5. Submit a pull request
## License
This integration is part of PraisonAI-Tools and follows the same license terms.
+153
View File
@@ -0,0 +1,153 @@
# Video CLI
```bash
python -m praisonai_tools.video probe input.mp4
```
```bash
python -m praisonai_tools.video probe input.mp4 --output probe.json
```
```bash
python -m praisonai_tools.video probe input.mp4 --json
```
```bash
python -m praisonai_tools.video transcribe input.mp4
```
```bash
python -m praisonai_tools.video transcribe input.mp4 --output transcript.srt
```
```bash
python -m praisonai_tools.video transcribe input.mp4 --format txt --output transcript.txt
```
```bash
python -m praisonai_tools.video transcribe input.mp4 --format json --output transcript.json
```
```bash
python -m praisonai_tools.video transcribe input.mp4 --local
```
```bash
python -m praisonai_tools.video plan input.mp4
```
```bash
python -m praisonai_tools.video plan input.mp4 --output plan.json
```
```bash
python -m praisonai_tools.video plan input.mp4 --preset podcast
```
```bash
python -m praisonai_tools.video plan input.mp4 --no-fillers --no-repetitions
```
```bash
python -m praisonai_tools.video render input.mp4 --timeline plan.json --output output.mp4
```
```bash
python -m praisonai_tools.video render input.mp4 --timeline plan.json --output output.mp4 --reencode
```
```bash
python -m praisonai_tools.video edit input.mp4 --output edited.mp4
```
```bash
python -m praisonai_tools.video edit input.mp4 --output edited.mp4 --preset podcast
```
```bash
python -m praisonai_tools.video edit input.mp4 --output edited.mp4 --preset meeting
```
```bash
python -m praisonai_tools.video edit input.mp4 --output edited.mp4 --preset course
```
```bash
python -m praisonai_tools.video edit input.mp4 --output edited.mp4 --preset clean
```
```bash
python -m praisonai_tools.video edit input.mp4 --output edited.mp4 --no-fillers
```
```bash
python -m praisonai_tools.video edit input.mp4 --output edited.mp4 --no-repetitions
```
```bash
python -m praisonai_tools.video edit input.mp4 --output edited.mp4 --no-silence
```
```bash
python -m praisonai_tools.video edit input.mp4 --output edited.mp4 --tangents
```
```bash
python -m praisonai_tools.video edit input.mp4 --output edited.mp4 --target-length 6m
```
```bash
python -m praisonai_tools.video edit input.mp4 --output edited.mp4 --captions srt
```
```bash
python -m praisonai_tools.video edit input.mp4 --output edited.mp4 --captions burn
```
```bash
python -m praisonai_tools.video edit input.mp4 --output edited.mp4 --captions off
```
```bash
python -m praisonai_tools.video edit input.mp4 --output edited.mp4 --force
```
```bash
python -m praisonai_tools.video edit input.mp4 --output edited.mp4 --provider openai
```
```bash
python -m praisonai_tools.video edit input.mp4 --output edited.mp4 --provider local
```
```bash
python -m praisonai_tools.video edit input.mp4 --output edited.mp4 --no-llm
```
```bash
python -m praisonai_tools.video edit input.mp4 --output edited.mp4 --model gpt-4o
```
```bash
python -m praisonai_tools.video edit input.mp4 --output edited.mp4 --whisper-model large-v3
```
```bash
python -m praisonai_tools.video edit input.mp4 --output edited.mp4 --local
```
```bash
python -m praisonai_tools.video edit input.mp4 --output edited.mp4 --reencode
```
```bash
python -m praisonai_tools.video edit input.mp4 --output edited.mp4 --verbose
```
```bash
python -m praisonai_tools.video edit input.mp4 --output edited.mp4 --workdir ./artifacts
```
```bash
python -m praisonai_tools.video edit input.mp4 --output edited.mp4 --no-artifacts
```
+126
View File
@@ -0,0 +1,126 @@
# Video Module
## Installation
```python
pip install praisonai-tools
```
## Code Usage
```python
from praisonai_tools.video import (
probe_video,
transcribe_video,
create_edit_plan,
render_video,
edit_video,
)
# Probe video metadata
result = probe_video("input.mp4")
print(f"Duration: {result.duration}s")
print(f"Resolution: {result.width}x{result.height}")
# Transcribe audio
transcript = transcribe_video("input.mp4")
print(transcript.text)
print(transcript.to_srt())
# Create edit plan
plan = create_edit_plan(
transcript=transcript,
duration=result.duration,
remove_fillers=True,
remove_repetitions=True,
remove_silence=True,
)
# Render video
render_video(
input_path="input.mp4",
output_path="output.mp4",
plan=plan,
)
# Full pipeline
result = edit_video(
input_path="input.mp4",
output_path="edited.mp4",
preset="podcast",
remove_fillers=True,
remove_repetitions=True,
remove_silence=True,
verbose=True,
)
```
## Classes
### VideoProbeResult
```python
@dataclass
class VideoProbeResult:
path: str
duration: float
width: int
height: int
fps: float
codec: str
bitrate: int
size_bytes: int
audio_codec: Optional[str]
audio_sample_rate: Optional[int]
audio_channels: Optional[int]
```
### TranscriptResult
```python
@dataclass
class TranscriptResult:
text: str
words: List[Word]
language: str
duration: float
def to_srt(self) -> str: ...
def to_dict(self) -> dict: ...
```
### EditPlan
```python
@dataclass
class EditPlan:
segments: List[Segment]
original_duration: float
edited_duration: float
removed_duration: float
removal_summary: Dict[str, float]
```
### VideoEditResult
```python
@dataclass
class VideoEditResult:
input_path: str
output_path: str
probe: VideoProbeResult
transcript: TranscriptResult
plan: EditPlan
success: bool
error: Optional[str]
artifacts: Dict[str, str]
```
## Presets
| Preset | Fillers | Repetitions | Silence | Tangents |
|--------|---------|-------------|---------|----------|
| podcast | ✓ | ✓ | ✓ | ✗ |
| meeting | ✓ | ✓ | ✓ | ✗ |
| course | ✓ | ✓ | ✓ | ✗ |
| clean | ✓ | ✓ | ✓ | ✓ |
+261
View File
@@ -0,0 +1,261 @@
"""
Inkog Security Analysis Tool Example
This example demonstrates how to use the Inkog tool to analyze AI agent code
for security vulnerabilities like token bombing, prompt injection, and more.
Prerequisites:
1. Install inkog CLI: brew tap inkog-io/inkog && brew install inkog
2. Get free API key from https://app.inkog.io
3. Set environment variable: export INKOG_API_KEY=your_key_here
The Inkog tool can detect:
- Token bombing attacks (loops where LLM controls termination)
- Prompt injection vulnerabilities
- Recursive tool calling without cycle detection
- Missing human oversight for destructive operations
- Cross-tenant data leakage
- MCP tool poisoning
- EU AI Act / NIST / OWASP compliance issues
"""
import os
from praisonai_tools import InkogTool, scan_agent_code
def basic_security_scan():
"""Perform a basic security scan of the current directory."""
print("🔍 Basic Security Scan Example")
print("=" * 50)
# Method 1: Using the class
inkog = InkogTool()
result = inkog.scan_directory(
path=".",
output_format="json",
policy="balanced" # balanced, low-noise, comprehensive
)
if "error" in result:
print(f"❌ Scan failed: {result['error']}")
return
# Analyze the results
analysis = inkog.analyze_findings(result)
print(analysis)
print("\n" + "=" * 50)
def advanced_security_scan():
"""Perform an advanced security scan with deep analysis."""
print("🔬 Advanced Security Scan Example (Deep Analysis)")
print("=" * 60)
inkog = InkogTool()
# Deep scan provides enhanced analysis with AI orchestrator
result = inkog.scan_directory(
path=".",
output_format="json",
policy="comprehensive", # Show all findings including best practices
deep=True, # Enable AI orchestrator analysis (requires Inkog Deep role)
verbose=True
)
if "error" in result:
print(f"❌ Deep scan failed: {result['error']}")
print("Note: Deep scans require the Inkog Deep role on your account")
return
# Show detailed analysis
analysis = inkog.analyze_findings(result)
print(analysis)
# Show raw findings for debugging
if "server_findings" in result:
print(f"\n📊 Raw Findings ({len(result['server_findings'])} total):")
for i, finding in enumerate(result["server_findings"][:3]): # Show first 3
print(f"{i+1}. {finding.get('pattern', 'Unknown')} - {finding.get('severity', 'N/A')}")
print(f" File: {finding.get('file', 'N/A')}:{finding.get('line', 'N/A')}")
print(f" Message: {finding.get('message', 'N/A')}")
print()
print("=" * 60)
def skill_package_scan():
"""Scan a SKILL.md package or MCP server."""
print("📦 Skill Package Security Scan Example")
print("=" * 50)
inkog = InkogTool()
# Scan current directory as a skill package
result = inkog.skill_scan(
path=".",
output_format="json",
deep=False
)
if "error" in result:
print(f"❌ Skill scan failed: {result['error']}")
return
analysis = inkog.analyze_findings(result)
print(analysis)
print("\n" + "=" * 50)
def mcp_server_scan():
"""Scan an MCP server from the registry."""
print("🔌 MCP Server Security Scan Example")
print("=" * 50)
inkog = InkogTool()
# Scan a popular MCP server (GitHub)
result = inkog.mcp_scan(
server_name="github",
output_format="json"
)
if "error" in result:
print(f"❌ MCP scan failed: {result['error']}")
return
analysis = inkog.analyze_findings(result)
print(analysis)
print("\n" + "=" * 50)
def standalone_function_example():
"""Example using standalone functions for quick scans."""
print("⚡ Standalone Function Examples")
print("=" * 40)
# Quick scan using standalone function
print("Running quick security scan...")
result = scan_agent_code(
path=".",
output_format="json",
policy="low-noise", # Only critical vulnerabilities
deep=False
)
if isinstance(result, dict) and "error" not in result:
tool = InkogTool()
analysis = tool.analyze_findings(result)
print(analysis)
else:
print(f"Scan result: {result}")
print("\n" + "=" * 40)
def html_report_example():
"""Generate an HTML security report."""
print("📄 HTML Report Generation Example")
print("=" * 45)
inkog = InkogTool()
# Generate HTML report
html_report = inkog.scan_directory(
path=".",
output_format="html",
policy="comprehensive",
deep=False
)
if isinstance(html_report, str) and not html_report.startswith("Error"):
# Save report to file
report_path = "inkog_security_report.html"
with open(report_path, "w", encoding="utf-8") as f:
f.write(html_report)
print(f"✅ HTML report generated: {report_path}")
print(f"📄 Report size: {len(html_report)} characters")
else:
print(f"❌ HTML report failed: {html_report}")
print("=" * 45)
def compliance_scan_example():
"""Example focused on compliance mapping."""
print("🏛️ Compliance Mapping Example")
print("=" * 40)
# Use comprehensive policy to see compliance mappings
inkog = InkogTool()
result = inkog.scan_directory(
path=".",
output_format="json",
policy="comprehensive", # Shows compliance mappings
deep=False
)
if "error" in result:
print(f"❌ Compliance scan failed: {result['error']}")
return
# Look for compliance information
findings = result.get("server_findings", [])
compliance_findings = []
for finding in findings:
if any(term in finding.get("message", "").lower()
for term in ["eu ai act", "nist", "owasp", "iso", "article"]):
compliance_findings.append(finding)
if compliance_findings:
print(f"📋 Found {len(compliance_findings)} compliance-related findings:")
for finding in compliance_findings[:3]: # Show first 3
print(f"• {finding.get('pattern', 'Unknown')}")
print(f" {finding.get('message', 'N/A')[:100]}...")
else:
print("✅ No specific compliance violations found")
print("=" * 40)
def main():
"""Run all examples."""
print("Inkog Security Analysis Tool Examples")
print("=" * 60)
# Check if inkog is available
api_key = os.environ.get("INKOG_API_KEY")
if not api_key:
print("⚠️ WARNING: INKOG_API_KEY environment variable not set")
print(" Get your free API key at https://app.inkog.io")
print(" Set it with: export INKOG_API_KEY=your_key_here")
print()
try:
# Run examples in order of complexity
basic_security_scan()
standalone_function_example()
skill_package_scan()
html_report_example()
compliance_scan_example()
# Advanced features (may require special access)
print("\n🎓 Advanced Examples (may require Inkog Deep role):")
print("-" * 50)
advanced_security_scan()
mcp_server_scan()
except Exception as e:
print(f"❌ Example failed: {e}")
print("This may be due to:")
print("1. Inkog CLI not installed")
print("2. Missing INKOG_API_KEY environment variable")
print("3. Network connectivity issues")
print("4. Insufficient permissions for deep scans")
if __name__ == "__main__":
main()
+55
View File
@@ -0,0 +1,55 @@
#!/usr/bin/env python3
"""JIRA Agent watch example using praisonai-tools.
Setup:
pip install praisonai-tools jira
export JIRA_URL=https://yourcompany.atlassian.net
export [email protected]
export JIRA_API_TOKEN=your_api_token
"""
import os
from praisonaiagents import Agent
from praisonai_tools import jira_tools
def main():
jira_url = os.getenv("JIRA_URL")
jira_token = os.getenv("JIRA_API_TOKEN")
jira_email = os.getenv("JIRA_EMAIL")
jira_username = os.getenv("JIRA_USERNAME")
if not jira_url or not jira_token or not (jira_email or jira_username):
print("Set JIRA_URL, JIRA_API_TOKEN, and JIRA_EMAIL (or JIRA_USERNAME)")
return
agent = Agent(
name="JIRA Monitor Agent",
instructions=(
"You monitor JIRA issues and projects. Use jira_watch_issue, "
"jira_watch_project, jira_get_issue_info, and jira_search_issues."
),
tools=jira_tools(),
llm="gpt-4o-mini",
)
print("JIRA watch tools ready. Example prompts:")
print(" Get info for PROJ-123")
print(" Search open issues: project = PROJ AND status = Open")
print(" Watch PROJ-123 for changes since 2024-01-01T00:00:00")
while True:
try:
user_input = input("\nYou: ").strip()
if user_input.lower() in {"quit", "exit", "q"}:
break
if not user_input:
continue
print("Agent:", agent.start(user_input))
except KeyboardInterrupt:
break
if __name__ == "__main__":
main()
+198
View File
@@ -0,0 +1,198 @@
#!/usr/bin/env python3
"""
Joy Trust Integration Examples for PraisonAI
This example demonstrates the native Joy Trust integration for secure agent handoffs.
"""
import os
from praisonai_tools import (
check_trust_score,
verify_handoff_safety,
trust_verified_handoff,
is_trust_verification_enabled,
get_trust_config,
JoyTrustTool
)
def example_basic_trust_check():
"""Example 1: Basic trust score verification"""
print("=== Basic Trust Score Check ===")
# Check trust for a specific agent
result = check_trust_score("researcher_agent", min_score=3.0)
print(f"Agent: {result['agent_name']}")
print(f"Trust Score: {result['trust_score']}")
print(f"Verified: {result['verified']}")
print(f"Meets Threshold: {result['meets_threshold']}")
print(f"Recommendation: {result.get('recommendation', 'N/A')}")
if result['error']:
print(f"Error: {result['error']}")
print()
def example_handoff_verification():
"""Example 2: Handoff safety verification"""
print("=== Handoff Safety Verification ===")
target_agent = "data_analyst"
verification = verify_handoff_safety(target_agent, min_score=3.5)
print(f"Target Agent: {verification['agent_name']}")
print(f"Trust Score: {verification['trust_score']}")
print(f"Handoff Safe: {verification['handoff_safe']}")
print(f"Recommendation: {verification['recommendation']}")
print(f"Threshold Used: {verification['threshold_used']}")
if verification['handoff_safe']:
print("✅ Safe to proceed with handoff")
else:
print("❌ Handoff not recommended")
print()
@trust_verified_handoff(min_score=4.0)
def example_secure_delegation(agent_name: str, task: str):
"""Example 3: Decorator-based secure delegation"""
print(f"Delegating task '{task}' to {agent_name}")
# Simulate task delegation
# In a real scenario, this would delegate to the actual agent
return {
"success": True,
"result": f"Task completed by {agent_name}",
"agent": agent_name,
"task": task
}
def example_environment_configuration():
"""Example 4: Environment-based configuration"""
print("=== Environment Configuration ===")
# Set environment variables for automatic trust verification
os.environ['PRAISONAI_TRUST_PROVIDER'] = 'joy'
os.environ['PRAISONAI_TRUST_MIN_SCORE'] = '3.5'
os.environ['PRAISONAI_TRUST_AUTO_VERIFY'] = 'true'
# Check if trust verification is enabled
enabled = is_trust_verification_enabled()
print(f"Trust Verification Enabled: {enabled}")
# Get current configuration
config = get_trust_config()
print(f"Provider: {config.provider}")
print(f"Min Score: {config.min_score}")
print(f"Auto Verify: {config.auto_verify_handoffs}")
print(f"Timeout: {config.timeout_seconds}s")
print()
def example_advanced_configuration():
"""Example 5: Advanced tool configuration"""
print("=== Advanced Configuration ===")
# Create a custom configured tool
trust_tool = JoyTrustTool(
api_key=os.getenv("JOY_TRUST_API_KEY"),
config=None # Will use environment config
)
# Configure the tool
config_result = trust_tool.configure(
min_score=4.5,
timeout_seconds=15.0,
auto_verify_handoffs=True
)
print(f"Configuration Status: {config_result['status']}")
print(f"Current Config: {config_result['config']}")
# Use the configured tool
result = trust_tool.check_trust("expert_agent")
print(f"\nExpert Agent Trust: {result['trust_score']}")
print(f"Meets Threshold: {result['meets_threshold']}")
print()
def example_workflow_integration():
"""Example 6: Workflow integration pattern"""
print("=== Workflow Integration ===")
# Enable trust verification
os.environ['PRAISONAI_TRUST_PROVIDER'] = 'joy'
# Define a multi-agent workflow
workflow_agents = [
{"name": "data_collector", "role": "collect"},
{"name": "data_processor", "role": "process"},
{"name": "report_generator", "role": "generate"}
]
trusted_agents = []
for agent in workflow_agents:
verification = verify_handoff_safety(agent["name"], min_score=3.0)
print(f"Agent: {agent['name']}")
print(f" Role: {agent['role']}")
print(f" Trust Score: {verification['trust_score']}")
print(f" Safe: {verification['handoff_safe']}")
print(f" Recommendation: {verification['recommendation']}")
if verification['handoff_safe']:
trusted_agents.append(agent)
print()
print(f"Trusted agents for workflow: {len(trusted_agents)}/{len(workflow_agents)}")
# Proceed with trusted agents only
if len(trusted_agents) == len(workflow_agents):
print("✅ All agents trusted - workflow can proceed safely")
else:
print("⚠️ Some agents not trusted - review workflow security")
def example_decorator_usage():
"""Example 7: Using the decorator in practice"""
print("=== Decorator Usage ===")
# Enable automatic trust verification
os.environ['PRAISONAI_TRUST_PROVIDER'] = 'joy'
os.environ['PRAISONAI_TRUST_AUTO_VERIFY'] = 'true'
# Example with high trust requirement
try:
result = example_secure_delegation("trusted_expert", "analyze_data")
print(f"Delegation Result: {result}")
except Exception as e:
print(f"Delegation Failed: {e}")
print()
if __name__ == "__main__":
print("Joy Trust Integration Examples\n")
# Note: These examples will make actual API calls if JOY_TRUST_API_KEY is set
# and the agents exist in the Joy Trust Network
try:
example_basic_trust_check()
example_handoff_verification()
example_environment_configuration()
example_advanced_configuration()
example_workflow_integration()
example_decorator_usage()
except Exception as e:
print(f"Example execution error: {e}")
print("\nNote: Some examples may fail if Joy Trust Network is not accessible")
print("or if the example agent names don't exist in the network.")
+275
View File
@@ -0,0 +1,275 @@
"""
Langextract Tool Example
This example demonstrates how to use the langextract tools for
interactive text visualization and analysis with PraisonAI agents.
Prerequisites:
pip install praisonai-tools[langextract]
# or
pip install praisonai-tools langextract
Features demonstrated:
- Text analysis with highlighted extractions
- File-based document analysis
- Integration with PraisonAI agents
- Interactive HTML generation
"""
import os
import tempfile
from pathlib import Path
from praisonai_tools import langextract_extract, langextract_render_file
def basic_text_analysis():
"""Demonstrate basic text analysis with highlighted extractions."""
print("🔍 Basic Text Analysis with Langextract")
print("=" * 50)
# Sample contract text
contract_text = """
CONSULTING AGREEMENT
This Agreement is entered into on March 15, 2024, between TechCorp Inc.
(the "Client") and Jane Smith (the "Consultant"). The Consultant will
provide AI development services for a period of 6 months starting
April 1, 2024.
Payment terms: $5,000 per month, payable within 15 days of invoice date.
Confidentiality obligations remain in effect for 2 years after termination.
"""
# Key terms to highlight
key_terms = [
"TechCorp Inc.",
"Jane Smith",
"March 15, 2024",
"April 1, 2024",
"$5,000 per month",
"15 days",
"2 years",
"AI development services"
]
# Extract and visualize
result = langextract_extract(
text=contract_text,
extractions=key_terms,
document_id="consulting-agreement",
output_path="contract_analysis.html",
auto_open=False # Set to True to open in browser automatically
)
if result.get("success"):
print(f"✅ Analysis complete!")
print(f" Document ID: {result['document_id']}")
print(f" Output file: {result['output_path']}")
print(f" Extractions: {result['extractions_count']} terms highlighted")
print(f" Text length: {result['text_length']} characters")
print()
print(f"💡 Open {result['output_path']} in your browser to view the interactive visualization!")
else:
print(f"❌ Error: {result.get('error')}")
if "langextract not installed" in result.get("error", ""):
print("💡 Install with: pip install langextract")
def file_analysis_example():
"""Demonstrate file-based document analysis."""
print("📄 File-based Document Analysis")
print("=" * 50)
# Create a sample document
document_content = """
TECHNICAL SPECIFICATION DOCUMENT
Project: AI-Powered Analytics Dashboard
Version: 2.1.0
Date: April 17, 2026
REQUIREMENTS:
- Python 3.10+ runtime environment
- PostgreSQL 14+ database
- Redis for caching layer
- Docker for containerization
- Kubernetes for orchestration
SECURITY CONSIDERATIONS:
- JWT authentication with 24-hour expiry
- Rate limiting: 1000 requests per hour per API key
- HTTPS-only communication
- Data encryption at rest using AES-256
PERFORMANCE TARGETS:
- API response time: < 200ms for 95th percentile
- Database query optimization required
- Concurrent user support: 10,000+ users
"""
# Create temporary file
with tempfile.NamedTemporaryFile(mode='w', delete=False, suffix='.txt') as f:
f.write(document_content)
temp_file_path = f.name
try:
# Technical terms to highlight
tech_terms = [
"Python 3.10+",
"PostgreSQL 14+",
"Redis",
"Docker",
"Kubernetes",
"JWT authentication",
"24-hour expiry",
"1000 requests per hour",
"AES-256",
"< 200ms",
"10,000+ users"
]
# Analyze the file
result = langextract_render_file(
file_path=temp_file_path,
extractions=tech_terms,
output_path="tech_spec_analysis.html",
auto_open=False
)
if result.get("success"):
print(f"✅ File analysis complete!")
print(f" Source file: {temp_file_path}")
print(f" Document ID: {result['document_id']}")
print(f" Output file: {result['output_path']}")
print(f" Extractions: {result['extractions_count']} terms highlighted")
print(f" Text length: {result['text_length']} characters")
print()
print(f"💡 Open {result['output_path']} in your browser to view the interactive visualization!")
else:
print(f"❌ Error: {result.get('error')}")
finally:
# Clean up temporary file
try:
os.unlink(temp_file_path)
except OSError:
pass
def agent_integration_example():
"""Demonstrate integration with PraisonAI agents."""
print("🤖 PraisonAI Agent Integration")
print("=" * 50)
# This is how you would integrate with agents
agent_code = '''
from praisonaiagents import Agent
from praisonai_tools import langextract_extract, langextract_render_file
# Create an agent that analyzes documents
document_analyzer = Agent(
name="DocumentAnalyzer",
instructions="""
You are a document analysis expert. When given text or documents,
identify key entities, dates, financial figures, and important terms.
Use langextract tools to create interactive visualizations highlighting
your findings.
""",
tools=[langextract_extract, langextract_render_file]
)
# Agent workflow example
def analyze_document(text_or_file_path):
"""Agent analyzes document and creates visualization."""
if text_or_file_path.endswith('.txt'):
# File-based analysis
response = document_analyzer.start(
f"Analyze this document file and highlight key terms: {text_or_file_path}"
)
else:
# Text-based analysis
response = document_analyzer.start(
f"Analyze this text and highlight important information: {text_or_file_path}"
)
return response
# Example usage:
# result = analyze_document("contract.txt")
# result = analyze_document("The quarterly report shows revenue of $1.2M...")
'''
print("💻 Example agent integration code:")
print(agent_code)
print()
print("🔧 Key integration points:")
print(" - Import tools: langextract_extract, langextract_render_file")
print(" - Add to agent tools list")
print(" - Agent can automatically use tools based on instructions")
print(" - Interactive HTML files created for human review")
def error_handling_example():
"""Demonstrate graceful error handling."""
print("⚠️ Error Handling and Graceful Degradation")
print("=" * 50)
# Test without langextract installed (simulated)
print("Testing graceful degradation when langextract is not installed:")
result = langextract_extract(
text="Sample text for analysis",
extractions=["Sample"],
document_id="test"
)
if "error" in result:
print(f"✅ Graceful error handling: {result['error']}")
print("💡 Users get clear installation instructions")
else:
print("✅ Langextract is available and working!")
print()
print("Common error scenarios handled:")
print(" - ❌ langextract not installed → Clear installation message")
print(" - ❌ Invalid file path → File not found error")
print(" - ❌ Empty text input → Parameter validation")
print(" - ❌ Browser auto-open fails → Graceful fallback")
def main():
"""Run all examples."""
print("🚀 Langextract Tool Examples for PraisonAI")
print("=" * 70)
print()
try:
basic_text_analysis()
print()
file_analysis_example()
print()
agent_integration_example()
print()
error_handling_example()
print()
print("🎉 All examples completed!")
print()
print("📚 Next Steps:")
print(" 1. Install langextract: pip install langextract")
print(" 2. Run your own text analysis")
print(" 3. Integrate with your PraisonAI agents")
print(" 4. Open generated HTML files to see interactive visualizations")
except Exception as e:
print(f"❌ Example error: {e}")
print("💡 This is expected if langextract is not installed")
if __name__ == "__main__":
main()
+238
View File
@@ -0,0 +1,238 @@
#!/usr/bin/env python3
"""Example usage of motion graphics video pipeline.
This example demonstrates how to use the motion graphics pipeline to create
videos from natural language prompts using HTML/GSAP animations.
Requirements:
pip install praisonai-tools[video-motion]
playwright install chromium
"""
import asyncio
import tempfile
from pathlib import Path
try:
from praisonai_tools.video.motion_graphics import (
create_motion_graphics_agent,
motion_graphics_team,
HtmlRenderBackend,
RenderOpts
)
from praisonai_tools.tools.git_tools import GitTools
except ImportError as e:
print(f"Import error: {e}")
print("Please install with: pip install praisonai-tools[video-motion]")
exit(1)
async def example_basic_animation():
"""Example 1: Basic motion graphics with simple animations."""
print("Example 1: Creating basic motion graphics animation...")
with tempfile.TemporaryDirectory() as tmpdir:
workspace = Path(tmpdir)
# Create a simple HTML composition
html_content = """
<!DOCTYPE html>
<html>
<head>
<script src="https://cdnjs.cloudflare.com/ajax/libs/gsap/3.12.2/gsap.min.js"></script>
<style>
body { margin: 0; padding: 0; background: #1a1a1a; overflow: hidden; font-family: Arial, sans-serif; }
#stage { width: 1920px; height: 1080px; position: relative; }
.title { font-size: 72px; color: #ffffff; position: absolute; top: 50%; left: 50%;
transform: translate(-50%, -50%); opacity: 0; }
.subtitle { font-size: 36px; color: #888; position: absolute; top: 60%; left: 50%;
transform: translate(-50%, -50%); opacity: 0; }
</style>
</head>
<body>
<div id="stage" data-duration="3.0">
<div class="title">Motion Graphics</div>
<div class="subtitle">Powered by PraisonAI</div>
</div>
<script>
const tl = gsap.timeline({ paused: true });
// Animate title in
tl.to(".title", {
duration: 1,
opacity: 1,
y: -20,
ease: "power2.out"
})
// Animate subtitle in
.to(".subtitle", {
duration: 0.8,
opacity: 1,
y: -10,
ease: "power2.out"
}, "-=0.3")
// Hold for a moment
.to({}, { duration: 1 })
// Fade out
.to([".title", ".subtitle"], {
duration: 0.5,
opacity: 0,
ease: "power2.in"
});
// Required: Export timeline
window.__timelines = [tl];
</script>
</body>
</html>
"""
# Save HTML file
(workspace / "index.html").write_text(html_content)
# Create backend and render
backend = HtmlRenderBackend()
# Lint first
lint_result = await backend.lint(workspace)
print(f"Lint result: {'✓ Passed' if lint_result.ok else '✗ Failed'}")
if not lint_result.ok:
print(f" Errors: {', '.join(lint_result.messages)}")
# Render video
if lint_result.ok:
render_opts = RenderOpts(
output_name="basic_animation.mp4",
fps=30,
quality="standard"
)
print("Rendering video... (this may take a moment)")
render_result = await backend.render(workspace, render_opts)
if render_result.ok:
print(f"✓ Video rendered successfully!")
print(f" Output: {render_result.output_path}")
print(f" Size: {render_result.size_kb}KB")
else:
print(f"✗ Render failed: {render_result.stderr}")
async def example_git_tools():
"""Example 2: Using GitTools for code exploration."""
print("\nExample 2: Using GitTools for safe git operations...")
with tempfile.TemporaryDirectory() as tmpdir:
git_tools = GitTools(base_dir=tmpdir)
print("Testing repository URL parsing...")
# Test different repository formats
test_repos = [
"octocat/Hello-World", # owner/repo format
"https://github.com/octocat/Hello-World.git", # HTTPS URL
]
for repo_input in test_repos:
try:
url, name = git_tools._parse_repo_input(repo_input)
print(f" {repo_input} -> URL: {url}, Name: {name}")
except Exception as e:
print(f" {repo_input} -> Error: {e}")
print("\nTesting file path safety...")
# Test file path validation
test_paths = [
"README.md", # Safe
"src/main.py", # Safe
"../etc/passwd", # Unsafe
"../../secret.txt", # Unsafe
]
for path in test_paths:
try:
safe_path = git_tools._validate_file_path(path)
print(f" {path} -> ✓ Safe: {safe_path}")
except ValueError as e:
print(f" {path} -> ✗ Unsafe: {e}")
def example_agent_factory():
"""Example 3: Using motion graphics agent factory."""
print("\nExample 3: Creating motion graphics agent...")
try:
# This would normally require praisonaiagents to be installed
agent = create_motion_graphics_agent(
backend="html",
max_retries=3,
llm="claude-sonnet-4"
)
print("✓ Motion graphics agent created successfully!")
print(f" Backend: {agent._motion_graphics_backend.__class__.__name__}")
print(f" Workspace: {agent._motion_graphics_workspace}")
print(f" Max retries: {agent._motion_graphics_max_retries}")
except ImportError as e:
print(f"⚠ Agent creation skipped: {e}")
print(" Install praisonaiagents to use agent features")
def example_team_preset():
"""Example 4: Using motion graphics team preset."""
print("\nExample 4: Creating motion graphics team...")
try:
team = motion_graphics_team(
research=True,
code_exploration=True,
backend="html"
)
print("✓ Motion graphics team created successfully!")
print(f" Agents: {[agent.name for agent in team.agents]}")
print(f" Coordinator: {team.agents[0].name}")
print(f" Workspace: {team._motion_graphics_workspace}")
except ImportError as e:
print(f"⚠ Team creation skipped: {e}")
print(" Install praisonaiagents to use team features")
async def main():
"""Run all examples."""
print("Motion Graphics Pipeline Examples")
print("=" * 50)
# Example 1: Basic animation (requires Playwright)
try:
await example_basic_animation()
except ImportError as e:
print(f"Example 1 skipped: {e}")
print("Install with: pip install playwright && playwright install chromium")
except Exception as e:
print(f"Example 1 failed: {e}")
# Example 2: GitTools (no extra dependencies)
await example_git_tools()
# Example 3: Agent factory (requires praisonaiagents)
example_agent_factory()
# Example 4: Team preset (requires praisonaiagents)
example_team_preset()
print("\n" + "=" * 50)
print("Examples complete!")
print("\nNext steps:")
print("1. Install optional dependencies: pip install praisonai-tools[video-motion]")
print("2. Install Playwright: playwright install chromium")
print("3. Install praisonaiagents for agent features")
print("4. Try creating your own motion graphics!")
if __name__ == "__main__":
asyncio.run(main())
+118
View File
@@ -0,0 +1,118 @@
# Motion Graphics Team Configuration
# This file shows how to configure a motion graphics team using YAML
framework: praisonai
project_name: "Motion Graphics Pipeline"
agents:
- name: "Coordinator"
role: "Team Leader"
backstory: |
You are the motion graphics team coordinator. Your role is to analyze incoming
requests, route them to appropriate specialists, coordinate between team members,
and validate final outputs with strict rules.
goal: |
Route requests effectively, coordinate team work, and ensure high-quality video
output with concrete file paths and proper error handling.
instructions: |
CRITICAL OUTPUT VALIDATION RULES:
- A render succeeded ONLY IF the reply contains a concrete file path AND no error indicators
- Never fabricate file paths or claim success without concrete evidence
- Surface all errors from the Animator
- Stop work after maximum retry budget is exceeded
- name: "Researcher"
role: "Research Specialist"
backstory: |
You are a research specialist who gathers information about topics in user requests.
You search the web for relevant facts, concepts, and examples, then summarize findings
in a brief that the Animator can use for on-screen content.
goal: |
Provide focused research that can be turned into visual motion graphics elements.
tools:
- search_web
- name: "CodeExplorer"
role: "Code Analysis Specialist"
backstory: |
You are a code exploration specialist who can clone and explore git repositories
on-demand. You read source code, understand implementations, and extract key
algorithms, data structures, and concepts for visualization.
goal: |
Extract essential programming concepts that can be animated visually.
tools:
- GitTools
instructions: |
IMPORTANT: You are READ-ONLY. Never write or modify code.
Always validate file paths to prevent directory traversal.
- name: "Animator"
role: "Motion Graphics Creator"
backstory: |
You are a motion graphics specialist who creates HTML/CSS/GSAP compositions
that render to high-quality MP4 videos. You follow strict authoring guidelines
and iterate on render failures.
goal: |
Create deterministic, frame-perfect HTML/GSAP compositions for video export.
tools:
- FileTools
- RenderTools
instructions: |
Follow the motion graphics authoring skill guide precisely:
- Use deterministic animations only (no Math.random, no infinite loops)
- Export timelines as window.__timelines = [tl]
- Add data-duration attributes
- Use 1920x1080 viewport
- Test with lint and render tools
- Iterate on failures with maximum retry limit
tasks:
- description: |
Create a 30-second motion graphics video explaining the concept requested by the user.
Process:
1. Coordinator routes to Researcher if topic research is needed
2. Coordinator routes to CodeExplorer if code analysis is needed
3. Coordinator always routes to Animator for final composition and rendering
4. Animator creates HTML/GSAP composition following authoring guidelines
5. Animator tests with lint and render tools
6. Animator iterates on failures up to maximum retries
7. Coordinator validates final output has concrete file path
expected_output: |
A rendered MP4 file with:
- Concrete file path (e.g., /renders/video_123.mp4)
- Video duration matching user request
- High-quality motion graphics explaining the concept
- No render errors or fabricated paths
agent: "Coordinator"
# Team configuration
team:
type: "hierarchical"
manager: "Coordinator"
verbose: true
# Tool configuration
tools:
- name: "GitTools"
config:
base_dir: "./repos"
- name: "RenderTools"
config:
backend: "html"
workspace: "./renders"
max_retries: 3
- name: "FileTools"
config:
base_dir: "./workspace"
# Output configuration
output:
format: "mp4"
quality: "standard"
fps: 30
workspace: "./motion_graphics_output"
+182
View File
@@ -0,0 +1,182 @@
"""n8n Integration Example for PraisonAI Agents.
This example demonstrates how to use the n8n integration tools to enable
PraisonAI agents to access n8n's 400+ integrations (Slack, Gmail, Notion, etc.).
Prerequisites:
1. Install PraisonAI Tools with n8n support:
pip install "praisonai-tools[n8n]"
2. Set up n8n (local or cloud):
- Local: docker run -it --rm --name n8n -p 5678:5678 -v n8n_data:/home/node/.n8n docker.n8n.io/n8nio/n8n
- Cloud: https://n8n.cloud
3. Configure environment variables:
export N8N_URL="http://localhost:5678" # or your n8n instance URL
export N8N_API_KEY="your-api-key" # optional for local testing
4. Create some workflows in n8n that your agents can use
Usage:
python examples/n8n_integration_example.py
"""
import os
from praisonaiagents import Agent
from praisonai_tools.n8n import n8n_workflow, n8n_list_workflows
def example_list_workflows():
"""Example: List available n8n workflows."""
print("🔍 Listing available n8n workflows...")
try:
result = n8n_list_workflows()
if "error" in result:
print(f"❌ Error: {result['error']}")
return
workflows = result.get("data", [])
if workflows:
print(f"✅ Found {len(workflows)} workflows:")
for workflow in workflows[:5]: # Show first 5
print(f" - {workflow.get('name', 'Unnamed')} (ID: {workflow.get('id')})")
else:
print("📝 No workflows found. Create some workflows in n8n first!")
return workflows
except Exception as e:
print(f"❌ Exception: {e}")
return []
def example_execute_workflow():
"""Example: Execute a simple n8n workflow."""
print("\n🚀 Executing n8n workflow...")
# This would be the ID of a workflow you created in n8n
# For demo purposes, we'll use a placeholder
workflow_id = os.getenv("N8N_TEST_WORKFLOW_ID", "demo-workflow-123")
try:
result = n8n_workflow(
workflow_id=workflow_id,
input_data={
"message": "Hello from PraisonAI!",
"timestamp": "2026-04-16T12:00:00Z",
"source": "praisonai-agent"
},
wait_for_completion=True
)
if "error" in result:
print(f"❌ Error: {result['error']}")
if "workflow" in result["error"].lower():
print("💡 Tip: Create a test workflow in n8n and set N8N_TEST_WORKFLOW_ID")
return
print("✅ Workflow executed successfully!")
print(f"📊 Result: {result}")
except Exception as e:
print(f"❌ Exception: {e}")
def example_agent_with_n8n_tools():
"""Example: PraisonAI agent using n8n tools."""
print("\n🤖 Creating PraisonAI agent with n8n tools...")
# Create an agent with n8n tools
agent = Agent(
name="automation-agent",
instructions="""
You are an automation assistant with access to n8n workflows.
You can help users automate tasks using n8n's 400+ integrations.
Available capabilities through n8n:
- Send messages to Slack, Discord, Telegram
- Create pages in Notion, Google Docs
- Add rows to Google Sheets, Airtable
- Send emails via Gmail, Outlook
- Create tasks in Trello, Jira, Linear
- Store data in databases (PostgreSQL, MongoDB, etc.)
- Call external APIs and webhooks
Always explain what you're doing and ask for workflow IDs when needed.
""",
tools=[n8n_workflow, n8n_list_workflows],
llm="gpt-4o-mini", # Use a lightweight model for demo
)
print("✅ Agent created with n8n tools!")
return agent
def example_agent_conversation():
"""Example conversation with the n8n-enabled agent."""
agent = example_agent_with_n8n_tools()
if not agent:
print("❌ Failed to create agent")
return
print("\n💬 Starting conversation with agent...")
# Example conversation
conversations = [
"Can you list the available n8n workflows?",
"What can you help me automate using n8n?",
# "Execute the Slack notification workflow with the message 'Test from PraisonAI'"
]
for i, message in enumerate(conversations, 1):
print(f"\n👤 User #{i}: {message}")
try:
response = agent.start(message)
print(f"🤖 Agent: {response}")
except Exception as e:
print(f"❌ Error in conversation: {e}")
def main():
"""Main function to run all examples."""
print("🔗 n8n Integration Example for PraisonAI Agents")
print("=" * 50)
# Check configuration
n8n_url = os.getenv("N8N_URL", "http://localhost:5678")
n8n_api_key = os.getenv("N8N_API_KEY")
print(f"🔧 Configuration:")
print(f" N8N_URL: {n8n_url}")
print(f" N8N_API_KEY: {'Set' if n8n_api_key else 'Not set (OK for local testing)'}")
# Run examples
workflows = example_list_workflows()
if workflows:
example_execute_workflow()
else:
print("\n💡 To test workflow execution:")
print(" 1. Open your n8n instance and create a simple workflow")
print(" 2. Set the workflow ID: export N8N_TEST_WORKFLOW_ID='your-workflow-id'")
print(" 3. Run this example again")
# Agent examples (these work even without actual workflows)
try:
example_agent_conversation()
except Exception as e:
print(f"❌ Agent example failed: {e}")
print("💡 This might be due to missing OpenAI API key or n8n connectivity")
print("\n✨ Example completed!")
print("\n📚 Next steps:")
print(" - Create workflows in your n8n instance")
print(" - Use the agent in your own applications")
print(" - Explore n8n's 400+ integrations at https://n8n.io/integrations")
if __name__ == "__main__":
main()
+198
View File
@@ -0,0 +1,198 @@
"""Example: Using NEXUS Prediction Market Tool with PraisonAI
This example demonstrates how to use the NEXUS prediction market tool to:
1. Get free market data for prediction markets
2. Check arbitrage opportunities between markets (paid feature)
3. Build an intelligent prediction market analysis agent
NEXUS API Information:
- Free endpoint: /kalshi?market={market_name}
- Paid endpoint: /arb/check?markets={market1,market2} ($0.02 USDC on eip155:8453)
- Validation: https://agentic.market/validate?url=https://nexus-agent-xa12.onrender.com/arb/check
"""
from praisonaiagents import Agent, tool
from praisonai_tools.tools.nexus_prediction_market_tool import NexusPredictionMarketTool
import json
# Example 1: Direct tool usage
def example_direct_usage():
"""Example of using the NEXUS tool directly."""
print("=== Direct NEXUS Tool Usage ===\n")
# Initialize the tool
nexus_tool = NexusPredictionMarketTool(source_name="praisonai-example")
# Get market data for Fed predictions
print("1. Getting Fed market data...")
fed_data = nexus_tool.get_market_data("Fed")
print(json.dumps(fed_data, indent=2))
print()
# Get BTC market data
print("2. Getting BTC market data...")
btc_data = nexus_tool.get_market_data("BTC")
print(json.dumps(btc_data, indent=2))
print()
# Check arbitrage opportunities (paid feature)
print("3. Checking arbitrage opportunities between Fed and BTC markets...")
arb_data = nexus_tool.check_arbitrage_opportunities(["Fed", "BTC"])
print(json.dumps(arb_data, indent=2))
print()
# Get agent information
print("4. Getting NEXUS agent information...")
agent_info = nexus_tool.get_agent_info()
print(json.dumps(agent_info, indent=2))
# Example 2: Create a custom tool wrapper for agents
@tool
def get_prediction_market_data(market: str) -> str:
"""Get prediction market data for a specific market.
Args:
market: Market identifier (e.g., 'Fed', 'BTC', 'Election')
Returns:
JSON string with market data including prices and trends
"""
nexus_tool = NexusPredictionMarketTool(source_name="praisonai-agent")
return nexus_tool.run(action="get_market_data", market=market)
@tool
def check_prediction_market_arbitrage(markets: str) -> str:
"""Check for arbitrage opportunities between prediction markets.
Args:
markets: Comma-separated list of markets to analyze (e.g., 'Fed,BTC')
Returns:
JSON string with arbitrage opportunities and analysis
Note:
This is a paid feature requiring $0.02 USDC on eip155:8453
"""
markets_list = [m.strip() for m in markets.split(',')]
nexus_tool = NexusPredictionMarketTool(source_name="praisonai-agent")
return nexus_tool.run(action="check_arbitrage", markets=markets_list)
# Example 3: Prediction market analysis agent
def example_prediction_market_agent():
"""Example of a prediction market analysis agent."""
print("\n=== Prediction Market Analysis Agent ===\n")
# Create an agent with NEXUS prediction market tools
market_agent = Agent(
name="PredictionMarketAnalyst",
instructions="""You are an expert prediction market analyst.
Your capabilities:
- Analyze prediction market data from Kalshi and Polymarket via NEXUS API
- Identify market trends and sentiment
- Spot potential arbitrage opportunities
- Provide insights on market dynamics
When analyzing markets:
1. First get current market data
2. Analyze the data for trends and insights
3. Look for arbitrage opportunities if requested
4. Provide clear explanations of your findings
Be thorough in your analysis and cite specific data points.""",
tools=[get_prediction_market_data, check_prediction_market_arbitrage],
llm="gpt-4o-mini"
)
# Example queries
queries = [
"Get the current Fed interest rate prediction market data and analyze what it shows about market sentiment.",
"Compare Fed and BTC prediction markets. What insights can you derive?",
"Check if there are any arbitrage opportunities between Fed and BTC markets."
]
for i, query in enumerate(queries, 1):
print(f"Query {i}: {query}")
response = market_agent.start(query)
print(f"Response: {response}\n")
print("-" * 80)
# Example 4: Multi-agent prediction market workflow
def example_multi_agent_workflow():
"""Example of a multi-agent workflow for comprehensive market analysis."""
print("\n=== Multi-Agent Prediction Market Workflow ===\n")
# Data gathering agent
data_agent = Agent(
name="MarketDataCollector",
instructions="""You collect prediction market data from NEXUS API.
Focus on gathering comprehensive data for analysis.""",
tools=[get_prediction_market_data]
)
# Analysis agent
analysis_agent = Agent(
name="MarketAnalyst",
instructions="""You analyze prediction market data to identify trends,
sentiment, and trading opportunities. Provide detailed insights.""",
tools=[]
)
# Arbitrage agent
arbitrage_agent = Agent(
name="ArbitrageSpecialist",
instructions="""You specialize in finding arbitrage opportunities
between prediction markets. Assess risk and profitability.""",
tools=[check_prediction_market_arbitrage]
)
# Example workflow
markets = ["Fed", "BTC", "Election"]
print("Step 1: Collecting market data...")
market_data = {}
for market in markets:
print(f"Collecting data for {market} market...")
data = data_agent.start(f"Get current market data for {market}")
market_data[market] = data
print(f"Data collected for {market}: {len(data)} characters")
print("\nStep 2: Analyzing market trends...")
combined_data = "\n".join([f"{market}: {data}" for market, data in market_data.items()])
analysis = analysis_agent.start(f"Analyze this prediction market data and identify key trends:\n{combined_data}")
print(f"Analysis: {analysis[:200]}...")
print("\nStep 3: Checking arbitrage opportunities...")
arb_analysis = arbitrage_agent.start("Check for arbitrage opportunities between Fed,BTC markets")
print(f"Arbitrage analysis: {arb_analysis[:200]}...")
if __name__ == "__main__":
print("NEXUS Prediction Market Tool Examples")
print("=" * 50)
# Run examples
try:
example_direct_usage()
example_prediction_market_agent()
example_multi_agent_workflow()
except Exception as e:
print(f"Error running examples: {e}")
print("\nNote: Some features require API access or payment.")
print("Free features: Market data queries")
print("Paid features: Arbitrage analysis ($0.02 USDC on eip155:8453)")
print("\n" + "=" * 50)
print("Examples completed!")
print("\nTo use in your own projects:")
print("1. Install: pip install praisonai-tools")
print("2. Import: from praisonai_tools.tools import NexusPredictionMarketTool")
print("3. Use: tool = NexusPredictionMarketTool()")
print("\nFor arbitrage features, ensure you have the required payment setup.")
+141
View File
@@ -0,0 +1,141 @@
#!/usr/bin/env python3
"""
SwarmScore Integration Example for PraisonAI
This example demonstrates how to integrate SwarmScore trust ratings
with PraisonAI agents for portable reputation management.
SwarmScore provides cryptographically signed trust ratings based on
verified execution history, success rates, and consistency.
Requirements:
pip install requests praisonai-tools
Usage:
python swarmscore_example.py
"""
import json
from praisonai_tools.tools import SwarmScoreTool, load_swarmscore_by_slug
def main():
"""Demonstrate SwarmScore integration with PraisonAI."""
print("🔍 SwarmScore Integration Example")
print("=" * 50)
# Initialize SwarmScore tool
swarmscore = SwarmScoreTool()
# Example agent slug (replace with actual agent identifier)
agent_slug = "example-agent-123"
print(f"\n📊 Loading SwarmScore for agent: {agent_slug}")
# Load SwarmScore data
result = swarmscore.load_swarmscore(agent_slug)
if result.success:
print("✅ SwarmScore loaded successfully!")
# Display basic score information
score_data = result.output
if 'score' in score_data:
print(f" Trust Score: {score_data['score']}")
if 'tier' in score_data:
print(f" Trust Tier: {score_data['tier']}")
if 'jobs_completed' in score_data:
print(f" Jobs Completed: {score_data['jobs_completed']}")
if 'success_rate' in score_data:
print(f" Success Rate: {score_data['success_rate']}%")
print("\n🔍 Full SwarmScore data:")
print(json.dumps(score_data, indent=2))
# Verify score freshness
if 'verify_payload' in score_data:
print("\n🔐 Verifying score freshness...")
verify_result = swarmscore.verify_swarmscore(score_data['verify_payload'])
if verify_result.success:
print("✅ Score verification successful!")
print(f" Verification data: {verify_result.output}")
else:
print(f"❌ Score verification failed: {verify_result.error}")
else:
print(f"❌ Failed to load SwarmScore: {result.error}")
print("\n💡 This might happen if:")
print(" • The agent slug doesn't exist in SwarmScore")
print(" • The SwarmScore API is temporarily unavailable")
print(" • Network connectivity issues")
# Get discovery manifest
print("\n🌐 Loading agent discovery manifest...")
manifest_result = swarmscore.get_discovery_manifest()
if manifest_result.success:
print("✅ Discovery manifest loaded!")
print(" This contains machine-readable data for agent-to-agent discovery")
print(f" Manifest keys: {list(manifest_result.output.keys())}")
else:
print(f"❌ Failed to load manifest: {manifest_result.error}")
# Demonstrate standalone function usage
print("\n🛠️ Using standalone functions:")
try:
# This will likely fail with the example slug, but shows the API
score_data = load_swarmscore_by_slug(agent_slug)
print(f"✅ Standalone function succeeded: {score_data}")
except Exception as e:
print(f"❌ Standalone function failed (expected): {e}")
print("\n📚 Next Steps:")
print(" 1. Register your agent at https://swarmsync.ai")
print(" 2. Get your agent slug from the SwarmSync dashboard")
print(" 3. Replace 'example-agent-123' with your actual slug")
print(" 4. Integrate SwarmScore checks into your agent workflows")
print("\n 📖 Documentation: https://swarmsync.ai/docs/protocol-specs/swarmscore")
def agent_workflow_example():
"""Example of how to integrate SwarmScore into an agent workflow."""
print("\n🤖 Agent Workflow Integration Example")
print("=" * 45)
# Simulated agent execution flow
agent_id = "my-trading-agent"
# 1. Load current trust score before executing task
swarmscore = SwarmScoreTool()
score_result = swarmscore.load_swarmscore(agent_id)
if score_result.success:
trust_score = score_result.output.get('score', 0)
print(f"🔍 Agent {agent_id} current trust score: {trust_score}")
# 2. Make decisions based on trust level
if trust_score >= 80:
print("✅ High trust - proceeding with full autonomy")
task_approach = "autonomous"
elif trust_score >= 60:
print("⚠️ Medium trust - requesting human approval for critical actions")
task_approach = "supervised"
else:
print("❌ Low trust - operating in restricted mode")
task_approach = "restricted"
# 3. Execute task based on trust level
print(f"🚀 Executing task with {task_approach} approach...")
# 4. After task completion, the score would be updated by SwarmSync
# (This happens externally based on verified execution results)
print("📝 Task completed - SwarmScore will be updated based on results")
else:
print(f"⚠️ Could not load trust score: {score_result.error}")
print("🔄 Falling back to default security settings")
if __name__ == "__main__":
main()
agent_workflow_example()
+21
View File
@@ -0,0 +1,21 @@
"""Example: Full video editing pipeline."""
import sys
sys.path.insert(0, "/Users/praison/PraisonAI-tools/praisonai_tools/video")
from pipeline import edit_video
result = edit_video(
input_path="/Users/praison/Agent-Recipes/agent_recipes/input.mov",
output_path="/tmp/edited_example.mp4",
preset="podcast",
remove_fillers=True,
remove_repetitions=True,
remove_silence=True,
verbose=True,
)
if result.success:
print(f"Success! Output: {result.output_path}")
print(f"Artifacts: {result.artifacts}")
else:
print(f"Failed: {result.error}")
+12
View File
@@ -0,0 +1,12 @@
"""Example: Probe video metadata."""
import sys
sys.path.insert(0, "/Users/praison/PraisonAI-tools/praisonai_tools/video")
from probe import probe_video
result = probe_video("/Users/praison/Agent-Recipes/agent_recipes/input.mov")
print(f"Duration: {result.duration}s")
print(f"Resolution: {result.width}x{result.height}")
print(f"FPS: {result.fps}")
print(f"Codec: {result.codec}")
Generated
-5553
View File
File diff suppressed because it is too large. Load diff
+81 -27
View File
@@ -1,27 +1,81 @@
from crewai_tools import BaseTool, Tool, tool
from crewai_tools import (
BrowserbaseLoadTool,
CodeDocsSearchTool,
CSVSearchTool,
DirectorySearchTool,
DOCXSearchTool,
DirectoryReadTool,
EXASearchTool,
FileReadTool,
GithubSearchTool,
SerperDevTool,
TXTSearchTool,
JSONSearchTool,
MDXSearchTool,
PDFSearchTool,
PGSearchTool,
RagTool,
ScrapeElementFromWebsiteTool,
ScrapeWebsiteTool,
SeleniumScrapingTool,
WebsiteSearchTool,
XMLSearchTool,
YoutubeChannelSearchTool,
YoutubeVideoSearchTool,
LlamaIndexTool
)
"""PraisonAI Tools - Tools and base classes for AI agents.
This package provides:
1. Base classes for creating custom tools (BaseTool, @tool decorator)
2. Ready-to-use tools for common tasks (Email, Slack, GitHub, etc.)
Usage:
from praisonai_tools import BaseTool, tool
# Method 1: Using @tool decorator
@tool
def my_search(query: str) -> str:
'''Search for something.'''
return f"Results for {query}"
# Method 2: Subclassing BaseTool
class MyTool(BaseTool):
name = "my_tool"
description = "Does something useful"
def run(self, query: str) -> str:
return f"Result for {query}"
# Method 3: Use built-in tools
from praisonai_tools import EmailTool, SlackTool, GitHubTool
email = EmailTool(provider="gmail")
slack = SlackTool()
github = GitHubTool()
# Use with PraisonAI Agents
from praisonaiagents import Agent
agent = Agent(tools=[my_search, MyTool(), email, slack])
"""
from importlib.metadata import version, PackageNotFoundError
from praisonai_tools.tools.base import BaseTool, ToolResult, ToolValidationError, validate_tool
from praisonai_tools.tools.decorator import tool, FunctionTool, is_tool, get_tool_schema
from praisonai_tools.catalogue import (
ToolCatalogueEntry,
list_tools,
get_tool_names,
)
# Imported at top (not below ``__getattr__``) to satisfy ruff E402. This only
# reads the auto-discovered manifest - no tool module is eagerly imported.
from praisonai_tools.tools import __all__ as _tools_all
try:
__version__ = version("praisonai-tools")
except PackageNotFoundError:
__version__ = "0.0.0"
__author__ = "Mervin Praison"
# Lazy imports for tool classes
def __getattr__(name):
"""Lazy load tool classes to avoid loading dependencies until needed."""
from praisonai_tools.tools import __getattr__ as tools_getattr
try:
return tools_getattr(name)
except AttributeError:
raise AttributeError(f"module {__name__!r} has no attribute {name!r}")
# Catalogue/discovery helpers exported eagerly above (not tool symbols, so they
# are not part of the auto-discovered manifest).
_CATALOGUE_EXPORTS = [
"ToolCatalogueEntry",
"list_tools",
"get_tool_names",
]
# The remainder of ``__all__`` is derived from the tools package's
# automatically-discovered manifest so the two levels cannot drift apart. Only
# source files are parsed (via AST) to build it - no tool module is eagerly
# imported here.
__all__ = sorted(set(_tools_all) | set(_CATALOGUE_EXPORTS))
def __dir__():
return sorted(set(globals()) | set(__all__))
@@ -1,25 +0,0 @@
from typing import Any
from embedchain import App
from praisonai_tools.tools.rag.rag_tool import Adapter
class EmbedchainAdapter(Adapter):
embedchain_app: App
summarize: bool = False
def query(self, question: str) -> str:
result, sources = self.embedchain_app.query(
question, citations=True, dry_run=(not self.summarize)
)
if self.summarize:
return result
return "\n\n".join([source[0] for source in sources])
def add(
self,
*args: Any,
**kwargs: Any,
) -> None:
self.embedchain_app.add(*args, **kwargs)
@@ -1,56 +0,0 @@
from pathlib import Path
from typing import Any, Callable
from lancedb import DBConnection as LanceDBConnection
from lancedb import connect as lancedb_connect
from lancedb.table import Table as LanceDBTable
from openai import Client as OpenAIClient
from pydantic import Field, PrivateAttr
from praisonai_tools.tools.rag.rag_tool import Adapter
def _default_embedding_function():
client = OpenAIClient()
def _embedding_function(input):
rs = client.embeddings.create(input=input, model="text-embedding-ada-002")
return [record.embedding for record in rs.data]
return _embedding_function
class LanceDBAdapter(Adapter):
uri: str | Path
table_name: str
embedding_function: Callable = Field(default_factory=_default_embedding_function)
top_k: int = 3
vector_column_name: str = "vector"
text_column_name: str = "text"
_db: LanceDBConnection = PrivateAttr()
_table: LanceDBTable = PrivateAttr()
def model_post_init(self, __context: Any) -> None:
self._db = lancedb_connect(self.uri)
self._table = self._db.open_table(self.table_name)
super().model_post_init(__context)
def query(self, question: str) -> str:
query = self.embedding_function([question])[0]
results = (
self._table.search(query, vector_column_name=self.vector_column_name)
.limit(self.top_k)
.select([self.text_column_name])
.to_list()
)
values = [result[self.text_column_name] for result in results]
return "\n".join(values)
def add(
self,
*args: Any,
**kwargs: Any,
) -> None:
self._table.add(*args, **kwargs)
+274
View File
@@ -0,0 +1,274 @@
"""Tool discovery / catalogue contract for PraisonAI Tools.
This module provides a single, programmatic way to enumerate every tool that
``praisonai-tools`` ships, plus any third-party tool registered via the
``praisonai.tools`` entry-point group. It exists so that the wider SDK (CLI,
docs, YAML validators, third-party packages) can answer *"what tools can I
actually use here?"* against a live catalogue derived from the real registry
rather than a hand-maintained allowlist that drifts.
Usage:
from praisonai_tools import list_tools
for entry in list_tools():
print(entry.name, entry.module, entry.extras, entry.summary)
Third-party packages can register their own tools by declaring an entry point:
# pyproject.toml (third-party tool package)
[project.entry-points."praisonai.tools"]
my_tool = "my_pkg:MyTool"
Nothing here imports optional/heavy dependencies: the catalogue is built from
the module map alone, so ``list_tools()`` is safe to call without any extras
installed.
"""
from __future__ import annotations
import ast
import os
from dataclasses import dataclass, field
from functools import lru_cache
from typing import Dict, Optional, Tuple
ENTRY_POINT_GROUP = "praisonai.tools"
# Directory that holds the first-party ``*_tool.py`` files. The catalogue is
# derived by scanning this directory, so dropping a new ``*_tool.py`` that
# defines a ``*Tool`` class makes the tool discoverable with no edits to any
# hand-maintained list.
_TOOLS_DIR = os.path.join(os.path.dirname(__file__), "tools")
# Map from the module (as recorded in ``_TOOL_MAP``) to the optional-dependency
# extras required to use tools defined in it. Modules absent from this mapping
# have no extra requirements (they rely only on the standard library or on the
# base dependencies). Keep this aligned with ``[project.optional-dependencies]``
# in ``pyproject.toml``.
_MODULE_EXTRAS = {
"wordpress_tool": ("wordpress",),
"pinchwork_tool": ("pinchwork", "marketplace"),
"agentid_tool": ("agentid", "marketplace"),
"joy_trust_tool": ("joy-trust", "marketplace"),
"agentfolio_tool": ("agentfolio", "marketplace"),
"praisonai_tools.n8n.n8n_workflow": ("n8n",),
"langextract_tool": ("langextract",),
"swarmscore_tool": ("swarmscore",),
"composio_tool": ("composio",),
"stripe_tool": ("stripe",),
}
@dataclass(frozen=True)
class ToolCatalogueEntry:
"""A single discoverable tool.
Attributes:
name: Public name used to import/reference the tool, e.g. ``"QdrantTool"``.
module: Dotted or relative module path that defines the tool.
extras: Optional-dependency extras required to use the tool, e.g.
``("swarmscore",)``. Empty when no extra install is needed.
summary: A one-line human-readable description of the tool.
"""
name: str
module: str
extras: Tuple[str, ...] = field(default_factory=tuple)
summary: str = ""
def _summary_for(name: str) -> str:
"""Derive a concise one-line summary for a tool class name.
Fallback used only when a tool class does not declare a literal
``description``. We avoid importing the tool (which could pull heavy/optional
deps) and instead produce a stable, human-friendly summary from the name.
"""
if name.endswith("Tool"):
base = name[:-4]
else:
base = name
return f"{base} integration tool".strip()
def _first_line(text: str) -> str:
"""Return the first non-empty line of ``text`` (a tool ``description``)."""
for line in text.splitlines():
line = line.strip()
if line:
return line
return ""
def _scan_module(path: str) -> "list[tuple[str, str]]":
"""AST-scan a ``*_tool.py`` file for ``*Tool`` classes and descriptions.
Returns a list of ``(class_name, summary)`` tuples. The file is parsed, not
imported, so this stays cheap and never triggers optional dependencies.
Classes without a literal ``description`` fall back to a name-synthesised
summary.
"""
try:
with open(path, "r", encoding="utf-8") as fh:
tree = ast.parse(fh.read(), filename=path)
except (OSError, SyntaxError): # pragma: no cover - defensive
return []
found = []
for node in tree.body:
if not isinstance(node, ast.ClassDef) or not node.name.endswith("Tool"):
continue
description: Optional[str] = None
for stmt in node.body:
# Accept both plain ``description = "..."`` and annotated
# ``description: str = "..."`` class attributes.
if isinstance(stmt, ast.Assign):
targets = [t.id for t in stmt.targets if isinstance(t, ast.Name)]
value = stmt.value
elif isinstance(stmt, ast.AnnAssign) and isinstance(stmt.target, ast.Name):
targets = [stmt.target.id]
value = stmt.value
else:
continue
if (
"description" in targets
and isinstance(value, ast.Constant)
and isinstance(value.value, str)
):
description = value.value
summary = _first_line(description) if description else _summary_for(node.name)
found.append((node.name, summary or _summary_for(node.name)))
return found
@lru_cache(maxsize=1)
def _scan_source_tree() -> "Dict[str, Tuple[str, str]]":
"""Derive the first-party tool catalogue from the source tree.
Scans every ``praisonai_tools/tools/*_tool.py`` file for ``*Tool`` classes
and returns ``{class_name: (module, summary)}``. Because the mapping is
derived from the files that actually exist, phantom names (advertised tools
with no file) are impossible by construction, and adding a tool is a single
new-file change.
"""
derived: "Dict[str, Tuple[str, str]]" = {}
try:
filenames = sorted(os.listdir(_TOOLS_DIR))
except OSError: # pragma: no cover - defensive
return derived
for filename in filenames:
if not filename.endswith("_tool.py"):
continue
module = filename[:-3] # strip ".py"; relative module name within tools
path = os.path.join(_TOOLS_DIR, filename)
for class_name, summary in _scan_module(path):
derived.setdefault(class_name, (module, summary))
return derived
def _first_party_entries() -> "list[ToolCatalogueEntry]":
"""Build catalogue entries for every first-party tool class.
The catalogue is source-derived: it is built by scanning the actual
``*_tool.py`` files on disk (see ``_scan_source_tree``) rather than a
hand-maintained dictionary, so it can never advertise a tool whose file does
not exist. Summaries come from each tool's own ``description``.
A few tools live in nested packages (e.g. the n8n workflow tool under
``praisonai_tools.n8n``) rather than flat ``*_tool.py`` files; those are
still surfaced via ``_TOOL_MAP`` so their public names remain stable.
"""
entries: "Dict[str, ToolCatalogueEntry]" = {}
for name, (module, summary) in _scan_source_tree().items():
entries[name] = ToolCatalogueEntry(
name=name,
module=module,
extras=_MODULE_EXTRAS.get(module, ()),
summary=summary,
)
# Include tool classes defined outside the flat ``*_tool.py`` layout (e.g.
# nested subpackages such as ``praisonai_tools.n8n``) that the scan cannot
# reach, keeping their names stable. Only *nested* (absolute-module) tools
# are merged here: flat ``*_tool.py`` classes are already source-derived by
# the scan above, so restoring them from ``_TOOL_MAP`` would reintroduce the
# hand-maintained bookkeeping this catalogue exists to eliminate.
try:
from praisonai_tools.tools import _TOOL_MAP
except Exception: # pragma: no cover - defensive
_TOOL_MAP = {}
for name, module in _TOOL_MAP.items():
if not name.endswith("Tool") or name in entries:
continue
if not module.startswith("praisonai_tools."):
continue
entries[name] = ToolCatalogueEntry(
name=name,
module=module,
extras=_MODULE_EXTRAS.get(module, ()),
summary=_summary_for(name),
)
return list(entries.values())
def _entry_point_entries() -> "list[ToolCatalogueEntry]":
"""Build catalogue entries for third-party tools registered via entry points.
Third-party packages register under the ``praisonai.tools`` entry-point
group. We do not import the target object (to keep discovery cheap and
dependency-free); we record the declared name and module value instead.
"""
try:
from importlib.metadata import entry_points
except ImportError: # pragma: no cover - Python < 3.8
return []
entries = []
try:
eps = entry_points()
# importlib.metadata API differs across versions.
if hasattr(eps, "select"):
selected = eps.select(group=ENTRY_POINT_GROUP)
else: # pragma: no cover - Python < 3.10 dict interface
selected = eps.get(ENTRY_POINT_GROUP, [])
except Exception: # pragma: no cover - defensive
return []
for ep in selected:
entries.append(
ToolCatalogueEntry(
name=ep.name,
module=getattr(ep, "value", "") or "",
extras=(),
summary=f"Third-party tool registered via {ENTRY_POINT_GROUP}",
)
)
return entries
def list_tools() -> "list[ToolCatalogueEntry]":
"""Return the full catalogue of available tools.
Includes every first-party tool shipped with ``praisonai-tools`` plus any
third-party tool registered via the ``praisonai.tools`` entry-point group.
Entries are de-duplicated by name (first-party wins) and returned sorted by
name.
"""
seen = {}
for entry in _first_party_entries():
seen[entry.name] = entry
for entry in _entry_point_entries():
seen.setdefault(entry.name, entry)
return sorted(seen.values(), key=lambda e: e.name)
def get_tool_names() -> "set[str]":
"""Return the set of every available tool name.
Convenience helper for validators that only need membership checks (e.g. to
replace a hand-maintained allowlist).
"""
return {entry.name for entry in list_tools()}
+81
View File
@@ -0,0 +1,81 @@
"""
Unified CLI for PraisonAI Tools.
"""
import argparse
import sys
from importlib import import_module
def main():
if len(sys.argv) < 2:
print("PraisonAI Developer Tools CLI")
print("\nUsage: praisonai-tools [tool] [args]")
print("\nAvailable tools:")
print(" list List the available tool catalogue")
print(" docs-generate Generate SDK reference documentation")
print(" video AI-powered video editing tools")
print(" fcp Final Cut Pro automation tools")
return 1
tool = sys.argv[1]
tool_args = sys.argv[2:]
if tool == "list":
parser = argparse.ArgumentParser(prog="praisonai-tools list")
parser.add_argument("--extras", action="store_true",
help="Only show tools that require optional-dependency extras")
args = parser.parse_args(tool_args)
from .catalogue import list_tools
entries = list_tools()
if args.extras:
entries = [e for e in entries if e.extras]
width = max((len(e.name) for e in entries), default=0)
for entry in entries:
extras = f" [extras: {', '.join(entry.extras)}]" if entry.extras else ""
print(f"{entry.name.ljust(width)} {entry.summary}{extras}")
print(f"\n{len(entries)} tools")
return 0
elif tool == "docs-generate":
# Handle docs-generate with its own parser for better help
parser = argparse.ArgumentParser(prog="praisonai-tools docs-generate")
parser.add_argument("--package", choices=["praisonaiagents", "praisonai", "typescript", "all"],
default="all", help="Package to generate docs for")
parser.add_argument("--dry-run", action="store_true", help="Don't write files")
parser.add_argument("--docs-root", default="/Users/praison/PraisonAIDocs",
help="Root directory of the documentation repository")
parser.add_argument("--source-root", help="Root directory of the source code")
args = parser.parse_args(tool_args)
from .docs_generator.generator import ReferenceDocsGenerator
generator = ReferenceDocsGenerator(docs_root=args.docs_root, source_root=args.source_root)
if args.package == "all":
generator.generate_all(dry_run=args.dry_run)
else:
generator.generate_package(args.package, dry_run=args.dry_run)
return 0
elif tool == "video":
try:
video_main = import_module(".video.__main__", package="praisonai_tools").main
sys.argv = [f"{sys.argv[0]} video"] + tool_args
return video_main()
except ImportError as e:
print(f"Error: Video tool not available. {e}")
return 1
elif tool == "fcp":
try:
fcp_main = import_module(".fcp_tool.cli", package="praisonai_tools").main
return fcp_main(tool_args)
except ImportError as e:
print(f"Error: FCP tool not available. {e}")
return 1
else:
print(f"Unknown tool: {tool}")
return 1
if __name__ == "__main__":
sys.exit(main())
+109
View File
@@ -0,0 +1,109 @@
# PraisonAI Docs Generator
A powerful documentation generator for PraisonAI SDK reference documentation.
## Installation
```bash
pip install praisonai-tools
```
## Quick Start
### Generate Rust SDK Documentation (Standalone)
The Rust documentation generator works without dependencies:
```bash
# Direct execution (recommended for Rust)
cd praisonai_tools/docs_generator
python3 __main__.py --package rust
# Or use the standalone script
python3 generate_rust_docs.py --rust-path /path/to/praisonai-rust
# Dry run
python3 __main__.py --package rust --dry-run
```
### Generate All SDK Documentation
For Python/TypeScript docs, install the full package first:
```bash
pip install praisonaiagents
python -m praisonai_tools.docs_generator --package all
```
### Using CLI
```bash
# Generate documentation with default settings
praisonai-tools docs-generate
# Generate with granular layout (separate pages for classes/functions)
praisonai-tools docs-generate --layout granular
# Specify custom output directory
praisonai-tools docs-generate --output /path/to/docs
```
### Direct Script Execution
```bash
python generator.py --layout granular
```
## Options
| Option | Description | Default |
|--------|-------------|---------|
| `--layout` | Documentation layout: `compact` or `granular` | `granular` |
| `--package` | Package to generate: `praisonaiagents`, `praisonai`, `typescript`, `rust`, `all` | `all` |
| `--docs-root` | Root directory of the documentation repository | Auto-detected |
| `--dry-run` | Show what would be generated without writing files | `false` |
## Layouts
### Compact Layout
- Single page per module
- Classes and functions inline
### Granular Layout (Recommended)
- Separate pages for each class and function
- Module hub pages with links to components
- Better navigation and searchability
## Features
- **Dynamic lazy-loading support**: Captures `_LAZY_IMPORTS`, `_LAZY_GROUPS`, `TOOL_MAPPINGS`
- **Recursive scanning**: Scans all `__init__.py` files in package tree
- **MDX output**: Mintlify-compatible MDX format
- **Navigation generation**: Auto-updates `docs.json` navigation
- **Icon support**: Maps modules to Font Awesome icons
- **Multi-language support**: Python, TypeScript, and Rust
## Supported Packages
- `praisonaiagents` (Python Core SDK)
- `praisonai` (Python Wrapper)
- TypeScript SDK (`praisonai-ts`)
- **Rust SDK** (`praisonai-rust`) - Agent, Tools, Workflows, Memory, LLM
## Output Structure
```
docs/sdk/reference/
├── praisonaiagents/
│ ├── modules/ # Module hub pages
│ ├── classes/ # Class pages
│ └── functions/ # Function pages
├── praisonai/
│ └── ...
├── typescript/
│ └── ...
└── rust/
├── modules/ # praisonai.agent, praisonai.tools, etc.
├── classes/ # Agent, ToolRegistry, AgentTeam, etc.
└── functions/ # Standalone functions
```
@@ -0,0 +1,14 @@
"""
PraisonAI Documentation Generator.
This module can be run standalone without requiring the praisonaiagents dependency.
Usage:
python -m praisonai_tools.docs_generator --package rust
python -m praisonai_tools.docs_generator --package all
"""
# This file is intentionally minimal to allow standalone execution
# without importing the parent package dependencies.
__version__ = "0.1.0"
@@ -0,0 +1,66 @@
"""
CLI entry point for praisonai_tools.docs_generator.
This can be run in two ways:
1. Directly: cd praisonai_tools/docs_generator && python __main__.py --package rust
2. As module (requires praisonaiagents): python -m praisonai_tools.docs_generator --package rust
"""
import argparse
import sys
def main():
parser = argparse.ArgumentParser(description="PraisonAI SDK Documentation Generator")
parser.add_argument("--package", choices=["praisonaiagents", "praisonai", "typescript", "rust", "all"],
default="all", help="Package to generate docs for")
parser.add_argument("--dry-run", action="store_true", help="Don't write files")
parser.add_argument("--docs-root", default="/Users/praison/PraisonAIDocs",
help="Root directory of the documentation repository")
parser.add_argument("--layout", choices=["legacy", "granular"], default="granular",
help="Documentation layout type")
parser.add_argument("--rust-path", default="/Users/praison/praisonai-package/src/praisonai-rust",
help="Path to Rust SDK workspace")
args = parser.parse_args()
# For Rust-only generation, use the standalone script to avoid praisonaiagents dependency
if args.package == "rust":
# Use the standalone Rust generator
from pathlib import Path
script_dir = Path(__file__).parent
sys.path.insert(0, str(script_dir))
from generate_rust_docs import main as rust_main
# Patch sys.argv for the rust script
sys.argv = [
'generate_rust_docs.py',
'--rust-path', args.rust_path,
'--docs-root', args.docs_root,
]
if args.dry_run:
sys.argv.append('--dry-run')
rust_main()
return 0
# For other packages, use the full generator (requires praisonaiagents)
try:
from .generator import ReferenceDocsGenerator, LayoutType
except ImportError as e:
print(f"Error: {e}")
print("\nFor generating Python/TypeScript docs, install praisonaiagents:")
print(" pip install praisonaiagents")
print("\nFor Rust docs only, use: --package rust")
return 1
layout = LayoutType.LEGACY if args.layout == "legacy" else LayoutType.GRANULAR
generator = ReferenceDocsGenerator(docs_root=args.docs_root, layout=layout)
if args.package == "all":
generator.generate_all(dry_run=args.dry_run)
else:
generator.generate_package(args.package, dry_run=args.dry_run)
return 0
if __name__ == "__main__":
sys.exit(main())
@@ -0,0 +1,573 @@
#!/usr/bin/env python3
"""
Standalone Rust SDK documentation generator.
This can be run without the praisonaiagents dependency.
"""
import sys
import json
import argparse
from pathlib import Path
# Add docs_generator to path
sys.path.insert(0, str(Path(__file__).parent))
from rust_parser import RustWorkspaceParser
def escape_mdx(text: str) -> str:
"""Escape special MDX characters."""
if not text:
return ""
return text.replace('<', '&lt;').replace('>', '&gt;').replace('{', '&#123;').replace('}', '&#125;')
def sanitize_description(text: str, max_len: int = 150) -> str:
"""Sanitize description for frontmatter. Truncates at word boundary."""
if not text:
return ""
clean = text.replace('"', "'").replace('\n', ' ').strip()
if len(clean) <= max_len:
return clean
# Truncate at word boundary
truncated = clean[:max_len].rsplit(' ', 1)[0]
return truncated + "..."
# Icon map
ICON_MAP = {
"agent": "robot", "tools": "wrench", "workflows": "route",
"memory": "brain", "llm": "microchip", "config": "gear",
"error": "circle-exclamation", "builder": "hammer",
"derive": "wand-magic-sparkles", "cli": "terminal",
"commands": "terminal", "default": "file-code"
}
# Friendly title mappings for modules
MODULE_TITLES = {
"agent": "Agent",
"builder": "Agent Builder",
"config": "Configuration",
"error": "Error Handling",
"llm": "LLM Providers",
"memory": "Memory",
"tools": "Tools",
"workflows": "Workflows",
"praisonai_derive": "Derive Macros",
"chat": "Chat Command",
"prompt": "Prompt Command",
"run": "Run Command",
}
# Rich descriptions for modules (used when docstring is empty or as fallback)
MODULE_DESCRIPTIONS = {
"agent": "Core AI Agent implementation for building intelligent agents in Rust",
"builder": "Fluent builder pattern for constructing Rust AI agents",
"config": "Configuration types for PraisonAI Rust AI agents",
"error": "Error handling utilities for Rust AI agent operations",
"llm": "LLM provider abstractions for Rust AI agents (OpenAI, Anthropic, Ollama)",
"memory": "Memory and conversation history for Rust AI agents",
"tools": "Tool system for extending Rust AI agent capabilities",
"workflows": "Multi-agent workflow patterns for Rust AI orchestration",
"praisonai_derive": "Procedural macros for defining Rust AI agent tools",
"chat": "Interactive chat command for Rust AI agents",
"prompt": "Single-shot prompt execution for Rust AI agents",
"run": "Workflow execution command for Rust AI agents",
}
# Abbreviations to preserve in titles
ABBREVIATIONS = {"llm", "api", "cli", "id", "url", "http", "ai", "io"}
# Related documentation mapping - Rust SDK only links to Rust docs
RELATED_DOCS = {
"agent": [
("Rust Overview", "/docs/rust/overview", "book-open"),
("Rust Quickstart", "/docs/rust/quickstart", "rocket"),
("Rust Agent Guide", "/docs/rust/agent", "robot"),
("Rust Installation", "/docs/rust/installation", "download"),
],
"builder": [
("Rust Agent Guide", "/docs/rust/agent", "robot"),
("Rust Quickstart", "/docs/rust/quickstart", "rocket"),
],
"tool": [
("Rust Tools Guide", "/docs/rust/tools", "wrench"),
("Rust Agent Guide", "/docs/rust/agent", "robot"),
],
"workflow": [
("Rust Agent Flow", "/docs/rust/agent-flow", "diagram-project"),
("Rust Agent Team", "/docs/rust/agent-team", "users"),
],
"memory": [
("Rust Memory Guide", "/docs/rust/memory", "brain"),
("Rust Agent Guide", "/docs/rust/agent", "robot"),
],
"llm": [
("Rust Overview", "/docs/rust/overview", "book-open"),
("Rust Agent Guide", "/docs/rust/agent", "robot"),
],
"config": [
("Rust Installation", "/docs/rust/installation", "gear"),
("Rust Overview", "/docs/rust/overview", "book-open"),
],
"error": [
("Rust Overview", "/docs/rust/overview", "book-open"),
],
"derive": [
("Rust Tools Guide", "/docs/rust/tools", "wrench"),
],
"cli": [
("Rust CLI Guide", "/docs/rust/cli", "terminal"),
("Rust Quickstart", "/docs/rust/quickstart", "rocket"),
],
"chat": [
("Rust CLI Guide", "/docs/rust/cli", "terminal"),
],
"prompt": [
("Rust CLI Guide", "/docs/rust/cli", "terminal"),
],
"run": [
("Rust CLI Guide", "/docs/rust/cli", "terminal"),
("Rust Agent Flow", "/docs/rust/agent-flow", "diagram-project"),
],
"task": [
("Rust Agent Team", "/docs/rust/agent-team", "users"),
("Rust Agent Flow", "/docs/rust/agent-flow", "diagram-project"),
],
"team": [
("Rust Agent Team", "/docs/rust/agent-team", "users"),
("Rust Agent Flow", "/docs/rust/agent-flow", "diagram-project"),
],
"flow": [
("Rust Agent Flow", "/docs/rust/agent-flow", "diagram-project"),
("Rust Agent Team", "/docs/rust/agent-team", "users"),
],
}
def get_related_docs(name: str, max_items: int = 5) -> list:
"""Find related documentation based on keywords in the name."""
name_lower = name.lower()
related = []
seen_paths = set()
for keyword, docs in RELATED_DOCS.items():
if keyword in name_lower:
for doc in docs:
if doc[1] not in seen_paths:
related.append(doc)
seen_paths.add(doc[1])
return related[:max_items]
def render_related_section(name: str, max_items: int = 5) -> str:
"""Render a CardGroup section with related documentation links."""
related = get_related_docs(name, max_items)
if not related:
return ""
cards = []
for title, path, icon in related:
cards.append(f' <Card title="{title}" icon="{icon}" href="{path}" />')
return f"""
---
## Related Documentation
<CardGroup cols={{2}}>
{chr(10).join(cards)}
</CardGroup>
"""
def friendly_title(name: str, page_type: str = "class") -> str:
"""Convert a name to a friendly, human-readable title.
Args:
name: The raw name (e.g., "AgentBuilder", "praisonai_derive")
page_type: One of "module", "class", "function"
"""
# Check for explicit module title mapping
if page_type == "module" and name.lower() in MODULE_TITLES:
return MODULE_TITLES[name.lower()]
# For functions, add parentheses
if page_type == "function":
# Special case for macros
if name == "tool":
return "#[tool] Macro"
return f"{name}()"
# Convert snake_case to Title Case
if "_" in name:
parts = name.split("_")
titled_parts = []
for part in parts:
if part.lower() in ABBREVIATIONS:
titled_parts.append(part.upper())
else:
titled_parts.append(part.capitalize())
return " ".join(titled_parts)
# Convert PascalCase to Title Case with spaces
# e.g., "AgentBuilder" -> "Agent Builder", "LlmConfig" -> "LLM Config"
result = []
current_word = []
for i, char in enumerate(name):
if char.isupper():
# Check if this is part of an abbreviation
if current_word:
word = "".join(current_word)
# If previous chars form an abbreviation, keep them together
if word.lower() in ABBREVIATIONS:
result.append(word.upper())
else:
result.append(word)
current_word = []
current_word.append(char)
else:
current_word.append(char)
# Don't forget the last word
if current_word:
word = "".join(current_word)
if word.lower() in ABBREVIATIONS:
result.append(word.upper())
else:
result.append(word)
# Join and fix common patterns
title = " ".join(result)
# Fix cases like "L L M" -> "LLM"
for abbr in ABBREVIATIONS:
spaced = " ".join(abbr.upper())
if spaced in title:
title = title.replace(spaced, abbr.upper())
return title
def get_icon(name: str) -> str:
"""Get icon for module name."""
return ICON_MAP.get(name.lower(), ICON_MAP["default"])
def generate_module_page(info, output_dir: Path, dry_run: bool = False) -> str:
"""Generate a module hub page."""
short_name = info.short_name or info.name.split('.')[-1]
title = friendly_title(short_name, "module")
title_suffix = " • Rust AI Agent SDK"
# Use docstring if available, otherwise use our rich descriptions
desc = sanitize_description(info.docstring) or MODULE_DESCRIPTIONS.get(short_name.lower(), f"Rust AI Agent SDK - {title}")
content = f'''---
title: "{title}{title_suffix}"
sidebarTitle: "{title}"
description: "{desc}"
icon: "{get_icon(short_name)}"
---
# {short_name}
<Badge color="orange">Rust AI Agent SDK</Badge>
{escape_mdx(info.docstring) if info.docstring else ""}
## Import
```rust
use {info.name.replace('.', '::')}::*;
```
'''
if info.classes:
content += "## Types\n\n<CardGroup cols={2}>\n"
for cls in info.classes:
cls_title = friendly_title(cls.name, "class")
cls_desc = sanitize_description(cls.docstring) or "Type definition."
content += f' <Card title="{cls_title}" icon="brackets-curly" href="../classes/{cls.name}">\n'
content += f" {cls_desc}\n"
content += " </Card>\n"
content += "</CardGroup>\n\n"
if info.functions:
content += "## Functions\n\n<CardGroup cols={2}>\n"
for func in info.functions:
func_title = friendly_title(func.name, "function")
func_desc = sanitize_description(func.docstring) or "Function definition."
content += f' <Card title="{func_title}" icon="function" href="../functions/{func.name}">\n'
content += f" {func_desc}\n"
content += " </Card>\n"
content += "</CardGroup>\n\n"
# Add related documentation section
related = render_related_section(short_name, max_items=5)
if related:
content += related
module_file = output_dir / 'modules' / f'{short_name}.mdx'
if not dry_run:
module_file.parent.mkdir(parents=True, exist_ok=True)
module_file.write_text(content)
return f"docs/sdk/reference/rust/modules/{short_name}"
def generate_class_page(cls, module_info, output_dir: Path, dry_run: bool = False) -> str:
"""Generate a class/struct page."""
short_name = module_info.short_name or module_info.name.split('.')[-1]
module_title = friendly_title(short_name, "module")
title = friendly_title(cls.name, "class")
title_suffix = " • Rust AI Agent SDK"
# Include original name in description for searchability
base_desc = sanitize_description(cls.docstring) if cls.docstring else f"{cls.name} struct for Rust AI agents"
desc = f"{base_desc}" if cls.name in base_desc else f"{cls.name}: {base_desc}"
content = f'''---
title: "{title}{title_suffix}"
sidebarTitle: "{title}"
description: "{desc}"
icon: "brackets-curly"
---
# {cls.name}
> Defined in the [**{module_title}**](../modules/{short_name}) module.
<Badge color="orange">Rust AI Agent SDK</Badge>
{escape_mdx(cls.docstring) if cls.docstring else ""}
'''
# Properties/Fields
if cls.properties:
content += "## Fields\n\n"
content += "| Name | Type | Description |\n"
content += "|------|------|-------------|\n"
for p in cls.properties:
p_type = escape_mdx(p.type) if hasattr(p, 'type') and p.type else "-"
p_desc = escape_mdx(p.description)[:80] if hasattr(p, 'description') and p.description else "-"
content += f"| `{p.name}` | `{p_type}` | {p_desc} |\n"
content += "\n"
# Methods
if cls.methods:
content += "## Methods\n\n"
for m in cls.methods:
async_prefix = "async " if hasattr(m, 'is_async') and m.is_async else ""
ret_type = m.return_type if hasattr(m, 'return_type') and m.return_type else "()"
content += f"### `{m.name}`\n\n"
content += f"```rust\n{async_prefix}fn {m.name}({m.signature}) -> {ret_type}\n```\n\n"
if hasattr(m, 'docstring') and m.docstring:
content += f"{escape_mdx(m.docstring)}\n\n"
if hasattr(m, 'params') and m.params:
content += "**Parameters:**\n\n"
content += "| Name | Type |\n|------|------|\n"
for p in m.params:
p_type = escape_mdx(p.type) if hasattr(p, 'type') and p.type else "-"
content += f"| `{p.name}` | `{p_type}` |\n"
content += "\n"
# Add related documentation section
related = render_related_section(cls.name, max_items=5)
if related:
content += related
class_file = output_dir / 'classes' / f'{cls.name}.mdx'
if not dry_run:
class_file.parent.mkdir(parents=True, exist_ok=True)
class_file.write_text(content)
return f"docs/sdk/reference/rust/classes/{cls.name}"
def generate_function_page(func, module_info, output_dir: Path, dry_run: bool = False) -> str:
"""Generate a function page."""
short_name = module_info.short_name or module_info.name.split('.')[-1]
module_title = friendly_title(short_name, "module")
title = friendly_title(func.name, "function")
title_suffix = " • Rust AI Agent SDK"
# Include original name in description for searchability
base_desc = sanitize_description(func.docstring) if func.docstring else f"{func.name} function for Rust AI agents"
desc = f"{base_desc}" if func.name in base_desc else f"{func.name}: {base_desc}"
async_prefix = "async " if hasattr(func, 'is_async') and func.is_async else ""
ret_type = func.return_type if hasattr(func, 'return_type') and func.return_type else "()"
content = f'''---
title: "{title}{title_suffix}"
sidebarTitle: "{title}"
description: "{desc}"
icon: "function"
---
# {func.name}
> Defined in the [**{module_title}**](../modules/{short_name}) module.
<Badge color="orange">Rust AI Agent SDK</Badge>
```rust
{async_prefix}fn {func.name}({func.signature}) -> {ret_type}
```
{escape_mdx(func.docstring) if func.docstring else ""}
'''
if hasattr(func, 'params') and func.params:
content += "## Parameters\n\n"
content += "| Name | Type | Description |\n"
content += "|------|------|-------------|\n"
for p in func.params:
p_type = escape_mdx(p.type) if hasattr(p, 'type') and p.type else "-"
p_desc = escape_mdx(p.description) if hasattr(p, 'description') and p.description else "-"
content += f"| `{p.name}` | `{p_type}` | {p_desc} |\n"
content += "\n"
# Add related documentation section
related = render_related_section(func.name, max_items=5)
if related:
content += related
func_file = output_dir / 'functions' / f'{func.name}.mdx'
if not dry_run:
func_file.parent.mkdir(parents=True, exist_ok=True)
func_file.write_text(content)
return f"docs/sdk/reference/rust/functions/{func.name}"
def update_docs_json(docs_json_path: Path, generated_files: list):
"""Update docs.json with Rust SDK navigation."""
if not docs_json_path.exists():
print(f"docs.json not found at {docs_json_path}")
return
with open(docs_json_path, 'r') as f:
docs_config = json.load(f)
# Find or create SDK tab
sdk_tab = None
for tab in docs_config.get('navigation', {}).get('tabs', []):
if tab.get('tab') == 'SDK':
sdk_tab = tab
break
if not sdk_tab:
print("SDK tab not found in docs.json")
return
# Find Reference group
ref_group = None
for g in sdk_tab.get('groups', []):
if isinstance(g, dict) and g.get('group') == 'Reference':
ref_group = g
break
if not ref_group:
print("Reference group not found in docs.json")
return
# Find or create rust group
rust_group = None
for pg in ref_group.get('pages', []):
if isinstance(pg, dict) and pg.get('group') == 'rust':
rust_group = pg
break
if not rust_group:
rust_group = {"group": "rust", "pages": []}
ref_group['pages'].append(rust_group)
# Organize by type
modules = sorted([p for p in generated_files if '/modules/' in p])
classes = sorted([p for p in generated_files if '/classes/' in p])
functions = sorted([p for p in generated_files if '/functions/' in p])
new_pages = []
if modules:
new_pages.append({"group": "Modules", "icon": "box", "pages": modules})
if classes:
new_pages.append({"group": "Classes", "icon": "brackets-curly", "pages": classes})
if functions:
new_pages.append({"group": "Functions", "icon": "function", "pages": functions})
rust_group['pages'] = new_pages
with open(docs_json_path, 'w') as f:
json.dump(docs_config, f, indent=2)
print("Updated docs.json with Rust SDK navigation")
def main():
parser = argparse.ArgumentParser(description="Generate Rust SDK documentation")
parser.add_argument("--rust-path", default="/Users/praison/praisonai-package/src/praisonai-rust",
help="Path to Rust SDK workspace")
parser.add_argument("--docs-root", default="/Users/praison/PraisonAIDocs",
help="Root directory of documentation")
parser.add_argument("--dry-run", action="store_true", help="Don't write files")
args = parser.parse_args()
rust_path = Path(args.rust_path)
docs_root = Path(args.docs_root)
output_dir = docs_root / 'docs/sdk/reference/rust'
# Ensure directories exist
if not args.dry_run:
(output_dir / 'modules').mkdir(parents=True, exist_ok=True)
(output_dir / 'classes').mkdir(parents=True, exist_ok=True)
(output_dir / 'functions').mkdir(parents=True, exist_ok=True)
# Parse all modules
workspace_parser = RustWorkspaceParser(rust_path)
all_modules = workspace_parser.parse_all()
generated_files = []
print("=" * 60)
print("Generating Rust SDK Documentation")
print("=" * 60)
for crate_name, modules in all_modules.items():
print(f"\nCrate: {crate_name}")
for info in modules:
# Skip empty modules
if not info.classes and not info.functions:
print(f" Skipping empty: {info.name}")
continue
# Generate module page
path = generate_module_page(info, output_dir, args.dry_run)
generated_files.append(path)
print(f" Generated: modules/{info.short_name}.mdx")
# Generate class pages
for cls in info.classes:
path = generate_class_page(cls, info, output_dir, args.dry_run)
generated_files.append(path)
print(f" Class: classes/{cls.name}.mdx")
# Generate function pages
for func in info.functions:
path = generate_function_page(func, info, output_dir, args.dry_run)
generated_files.append(path)
print(f" Function: functions/{func.name}.mdx")
print(f"\n=== Generated {len(generated_files)} files ===")
# Update docs.json
if not args.dry_run:
update_docs_json(docs_root / 'docs.json', generated_files)
if __name__ == "__main__":
main()
File diff suppressed because it is too large. Load diff
@@ -0,0 +1,826 @@
"""
Rust Documentation Parser for PraisonAI.
Parses Rust source code to extract documentation for MDX generation.
Uses tree-sitter for robust Rust syntax parsing.
"""
from __future__ import annotations
import re
from dataclasses import dataclass, field
from pathlib import Path
from typing import Dict, List, Optional, Tuple
# Import shared data classes from generator
try:
from .generator import (
ParamInfo,
MethodInfo,
ClassInfo,
FunctionInfo,
ModuleInfo,
SKIP_MODULES,
SKIP_METHODS,
extract_rust_function_calls,
)
except ImportError:
# Define minimal fallback classes when run standalone
from dataclasses import dataclass, field
from typing import List, Optional, Tuple
@dataclass
class ParamInfo:
name: str
type: str = ""
default: Optional[str] = None
description: str = ""
required: bool = True
@dataclass
class MethodInfo:
name: str
signature: str = ""
return_type: str = ""
docstring: str = ""
params: List[ParamInfo] = field(default_factory=list)
is_async: bool = False
is_static: bool = False
examples: List[str] = field(default_factory=list)
@dataclass
class ClassInfo:
name: str
docstring: str = ""
bases: List[str] = field(default_factory=list)
methods: List[MethodInfo] = field(default_factory=list)
static_methods: List[MethodInfo] = field(default_factory=list)
properties: List[ParamInfo] = field(default_factory=list)
init_params: List[ParamInfo] = field(default_factory=list)
examples: List[str] = field(default_factory=list)
source_file: str = ""
source_line: int = 0
@dataclass
class FunctionInfo:
name: str
signature: str = ""
return_type: str = ""
docstring: str = ""
params: List[ParamInfo] = field(default_factory=list)
is_async: bool = False
examples: List[str] = field(default_factory=list)
source_file: str = ""
source_line: int = 0
calls: List[str] = field(default_factory=list)
@dataclass
class ModuleInfo:
name: str
short_name: str = ""
display_name: str = ""
docstring: str = ""
is_init: bool = False
package: str = ""
classes: List[ClassInfo] = field(default_factory=list)
functions: List[FunctionInfo] = field(default_factory=list)
constants: List[Tuple[str, str]] = field(default_factory=list)
source_file: str = ""
def extract_rust_function_calls(source_code: str) -> List[str]:
"""Fallback when run standalone - return empty list."""
return []
SKIP_MODULES = {"tests", "__pycache__", "conftest"}
SKIP_METHODS = {"__init__", "__repr__", "__str__"}
# Try to import tree-sitter, provide fallback
try:
import tree_sitter_rust as ts_rust
from tree_sitter import Language, Parser
TREE_SITTER_AVAILABLE = True
except ImportError:
TREE_SITTER_AVAILABLE = False
# =============================================================================
# RUST-SPECIFIC DATA CLASSES
# =============================================================================
@dataclass
class TraitInfo:
"""Information about a Rust trait."""
name: str
docstring: str = ""
methods: List[MethodInfo] = field(default_factory=list)
supertraits: List[str] = field(default_factory=list)
source_file: str = ""
source_line: int = 0
@dataclass
class EnumInfo:
"""Information about a Rust enum."""
name: str
docstring: str = ""
variants: List[Tuple[str, str]] = field(default_factory=list) # (name, docstring)
source_file: str = ""
source_line: int = 0
@dataclass
class MacroInfo:
"""Information about a Rust macro."""
name: str
docstring: str = ""
attributes: List[str] = field(default_factory=list)
examples: List[str] = field(default_factory=list)
# =============================================================================
# RUST DOC PARSER (REGEX-BASED FALLBACK)
# =============================================================================
class RustDocParser:
"""Parse Rust source code for documentation.
This parser extracts documentation from Rust source files using regex patterns.
It handles:
- Module doc comments (//!)
- Item doc comments (///)
- Structs, traits, enums, and functions
- Impl blocks and methods
- Proc-macro definitions
"""
# Skip these module names
SKIP_MODULES = {"target", "tests", "examples", "benches", ".git"}
def __init__(self, crate_path: Path, crate_name: str = "praisonai"):
"""Initialize the parser.
Args:
crate_path: Path to the crate root (containing Cargo.toml)
crate_name: Name of the crate for module prefixes
"""
self.crate_path = crate_path
self.crate_name = crate_name
self.src_path = crate_path / "src"
self.workspace_root = crate_path.parent # Parent of crate for relative paths
# Cache for parsed files
self._file_cache: Dict[Path, str] = {}
def get_modules(self) -> List[str]:
"""Get list of modules to document.
Returns:
List of module paths (e.g., ['praisonai.agent', 'praisonai.tools'])
"""
modules = set()
if not self.src_path.exists():
return []
# Find all .rs files
for rs_file in self.src_path.rglob("*.rs"):
# Skip test files
if "tests" in rs_file.parts or "test_" in rs_file.name:
continue
# Convert file path to module path
rel_path = rs_file.relative_to(self.src_path)
parts = list(rel_path.parts)
# Handle mod.rs and lib.rs
if parts[-1] == "mod.rs":
parts = parts[:-1]
elif parts[-1] == "lib.rs":
parts = [] # lib.rs is the crate root
else:
parts[-1] = parts[-1].replace(".rs", "")
if parts:
module_name = f"{self.crate_name}.{'.'.join(parts)}"
else:
module_name = self.crate_name
modules.add(module_name)
return sorted(modules)
def parse_module(self, module_path: str) -> Optional[ModuleInfo]:
"""Parse a module and extract documentation.
Args:
module_path: Fully qualified module path (e.g., 'praisonai.agent')
Returns:
ModuleInfo or None if module not found
"""
# Convert module path to file path
file_path = self._module_to_file(module_path)
if not file_path or not file_path.exists():
return None
try:
content = self._read_file(file_path)
# Extract module-level doc comments
module_doc = self._extract_module_docs(content)
# Get module short name
parts = module_path.split(".")
short_name = parts[-1] if len(parts) > 1 else module_path
# Get relative source path for GitHub links
try:
source_file = str(file_path.relative_to(self.workspace_root))
except ValueError:
source_file = str(file_path)
info = ModuleInfo(
name=module_path,
short_name=short_name,
docstring=module_doc,
is_init=file_path.name in ("lib.rs", "mod.rs"),
package="rust",
source_file=source_file,
)
# Parse structs as classes
info.classes = self._parse_structs(content, source_file)
# Parse traits (also as classes with special handling)
traits = self._parse_traits(content)
for trait in traits:
info.classes.append(ClassInfo(
name=trait.name,
docstring=trait.docstring,
bases=trait.supertraits,
methods=trait.methods,
source_file=source_file,
))
# Parse enums
enums = self._parse_enums(content)
for enum in enums:
info.classes.append(ClassInfo(
name=enum.name,
docstring=enum.docstring,
properties=[
ParamInfo(name=v[0], description=v[1], type="variant")
for v in enum.variants
],
source_file=source_file,
))
# Parse standalone functions
info.functions = self._parse_functions(content, source_file)
# Parse impl blocks and attach methods to structs
self._parse_impl_blocks(content, info)
return info
except Exception as e:
print(f"Error parsing {file_path}: {e}")
return None
def _module_to_file(self, module_path: str) -> Optional[Path]:
"""Convert module path to file path."""
parts = module_path.split(".")
# Remove crate name prefix
if parts[0] == self.crate_name:
parts = parts[1:]
if not parts:
# Crate root - try lib.rs
lib_rs = self.src_path / "lib.rs"
if lib_rs.exists():
return lib_rs
return None
# Try mod.rs first
mod_path = self.src_path / "/".join(parts) / "mod.rs"
if mod_path.exists():
return mod_path
# Try direct file
file_path = self.src_path / "/".join(parts[:-1]) / f"{parts[-1]}.rs"
if file_path.exists():
return file_path
# Try as single file in src/
single_file = self.src_path / f"{parts[-1]}.rs"
if single_file.exists():
return single_file
return None
def _read_file(self, path: Path) -> str:
"""Read file with caching."""
if path not in self._file_cache:
self._file_cache[path] = path.read_text(encoding="utf-8")
return self._file_cache[path]
def _extract_module_docs(self, content: str) -> str:
"""Extract module-level doc comments (//!)."""
lines = []
for line in content.split("\n"):
stripped = line.strip()
if stripped.startswith("//!"):
# Remove //! prefix
doc_line = stripped[3:].strip()
lines.append(doc_line)
elif stripped.startswith("//") or stripped.startswith("#"):
continue # Skip other comments and attributes
elif stripped:
break # Stop at first non-comment/non-attribute line
return "\n".join(lines)
def _extract_doc_comments(self, content: str, start_pos: int) -> str:
"""Extract doc comments (///) before an item.
Args:
content: Full file content
start_pos: Position where the item definition starts
Returns:
Extracted documentation string
"""
# Find the start of line containing start_pos
line_start = content.rfind("\n", 0, start_pos) + 1
# Work backwards to find doc comments
lines = []
pos = line_start - 1
while pos > 0:
# Find previous line
prev_line_end = pos
prev_line_start = content.rfind("\n", 0, prev_line_end) + 1
line = content[prev_line_start:prev_line_end].strip()
if line.startswith("///"):
lines.insert(0, line[3:].strip())
pos = prev_line_start - 1
elif line.startswith("#["):
# Skip attributes
pos = prev_line_start - 1
elif not line:
# Skip empty lines
pos = prev_line_start - 1
else:
break
return "\n".join(lines)
def _parse_structs(self, content: str, source_file: str = "") -> List[ClassInfo]:
"""Parse struct definitions."""
structs = []
# Match: pub struct Name { ... } or pub struct Name;
pattern = r'pub\s+struct\s+(\w+)(?:<[^>]*>)?\s*(?:\{([^}]*)\}|;)'
for match in re.finditer(pattern, content, re.DOTALL):
name = match.group(1)
fields_str = match.group(2) or ""
# Calculate line number
line_no = content[:match.start()].count('\n') + 1
# Extract doc comments
doc = self._extract_doc_comments(content, match.start())
# Parse fields
properties = []
if fields_str:
field_pattern = r'(?:pub\s+)?(\w+):\s*([^,\n]+)'
for field_match in re.finditer(field_pattern, fields_str):
field_name = field_match.group(1)
field_type = field_match.group(2).strip().rstrip(",")
# Extract field doc comment
field_doc = self._extract_doc_comments(fields_str, field_match.start())
properties.append(ParamInfo(
name=field_name,
type=field_type,
description=field_doc,
))
# Parse examples from docstring
examples = self._extract_examples(doc)
structs.append(ClassInfo(
name=name,
docstring=self._clean_docstring(doc),
properties=properties,
examples=examples,
source_file=source_file,
source_line=line_no,
))
return structs
def _parse_traits(self, content: str) -> List[TraitInfo]:
"""Parse trait definitions."""
traits = []
# Match: pub trait Name { ... }
pattern = r'pub\s+trait\s+(\w+)(?:<[^>]*>)?(?:\s*:\s*([^{]+))?\s*\{([^}]*(?:\{[^}]*\}[^}]*)*)\}'
for match in re.finditer(pattern, content, re.DOTALL):
name = match.group(1)
supertraits_str = match.group(2) or ""
body = match.group(3)
# Extract doc comments
doc = self._extract_doc_comments(content, match.start())
# Parse supertraits
supertraits = []
if supertraits_str:
supertraits = [s.strip() for s in supertraits_str.split("+")]
# Parse required methods
methods = self._parse_trait_methods(body)
traits.append(TraitInfo(
name=name,
docstring=self._clean_docstring(doc),
methods=methods,
supertraits=supertraits,
))
return traits
def _parse_trait_methods(self, body: str) -> List[MethodInfo]:
"""Parse methods from a trait body."""
methods = []
# Match method signatures: fn name(...) -> Type;
pattern = r'fn\s+(\w+)\s*\(([^)]*)\)(?:\s*->\s*([^;{]+))?[;{]'
for match in re.finditer(pattern, body):
name = match.group(1)
params_str = match.group(2)
return_type = match.group(3) or "()".strip() if match.group(3) else "()"
# Extract doc comment
doc = self._extract_doc_comments(body, match.start())
# Parse parameters
params = self._parse_params(params_str)
methods.append(MethodInfo(
name=name,
signature=params_str,
return_type=return_type.strip(),
docstring=self._clean_docstring(doc),
params=params,
is_async="async" in body[max(0, match.start()-10):match.start()],
))
return methods
def _parse_enums(self, content: str) -> List[EnumInfo]:
"""Parse enum definitions."""
enums = []
# Match: pub enum Name { ... }
pattern = r'pub\s+enum\s+(\w+)(?:<[^>]*>)?\s*\{([^}]*(?:\{[^}]*\}[^}]*)*)\}'
for match in re.finditer(pattern, content, re.DOTALL):
name = match.group(1)
body = match.group(2)
# Extract doc comments
doc = self._extract_doc_comments(content, match.start())
# Parse variants
variants = []
variant_pattern = r'(\w+)(?:\s*\{[^}]*\}|\s*\([^)]*\))?'
for var_match in re.finditer(variant_pattern, body):
var_name = var_match.group(1)
if var_name in ("pub", "struct", "fn"):
continue
var_doc = self._extract_doc_comments(body, var_match.start())
variants.append((var_name, var_doc))
enums.append(EnumInfo(
name=name,
docstring=self._clean_docstring(doc),
variants=variants,
))
return enums
def _parse_functions(self, content: str, source_file: str = "") -> List[FunctionInfo]:
"""Parse standalone public functions."""
functions = []
# Match: pub fn name(...) -> Type { ... }
# Exclude functions inside impl blocks by checking context
pattern = r'^pub\s+(?:async\s+)?fn\s+(\w+)\s*(?:<[^>]*>)?\s*\(([^)]*)\)(?:\s*->\s*([^{]+))?\s*\{'
for match in re.finditer(pattern, content, re.MULTILINE):
name = match.group(1)
params_str = match.group(2)
return_type = match.group(3).strip() if match.group(3) else "()"
# Skip if this looks like it's inside an impl block
# (simple heuristic: check if there's an unmatched `impl` before this)
before = content[:match.start()]
impl_count = before.count("impl ")
close_brace_count = before.count("}")
if impl_count > close_brace_count:
continue
# Calculate line number
line_no = content[:match.start()].count('\n') + 1
# Extract doc comments
doc = self._extract_doc_comments(content, match.start())
# Parse parameters
params = self._parse_params(params_str)
# Check if async
is_async = "async" in match.group(0)
# Extract function body for calls
func_body = self._extract_brace_content(content, match.end() - 1)
try:
calls = extract_rust_function_calls(func_body) if func_body else []
except:
calls = []
functions.append(FunctionInfo(
name=name,
signature=params_str,
return_type=return_type,
docstring=self._clean_docstring(doc),
params=params,
is_async=is_async,
examples=self._extract_examples(doc),
source_file=source_file,
source_line=line_no,
calls=calls,
))
return functions
def _parse_impl_blocks(self, content: str, module_info: ModuleInfo) -> None:
"""Parse impl blocks and attach methods to corresponding structs."""
# Find all impl block starts
impl_pattern = r'impl(?:<[^>]*>)?\s+(?:(\w+)\s+for\s+)?(\w+)(?:<[^>]*>)?\s*\{'
for match in re.finditer(impl_pattern, content):
match.group(1) # For trait impls
struct_name = match.group(2)
# Find the matching closing brace using brace counting
start_pos = match.end() - 1 # Position of opening brace
body = self._extract_brace_content(content, start_pos)
if body:
# Parse methods from impl body
methods = self._parse_impl_methods(body)
# Find the corresponding struct and add methods
for cls in module_info.classes:
if cls.name == struct_name:
cls.methods.extend(methods)
break
def _extract_brace_content(self, content: str, start_pos: int) -> str:
"""Extract content within braces, handling nested braces correctly."""
if content[start_pos] != '{':
return ""
depth = 0
i = start_pos
while i < len(content):
if content[i] == '{':
depth += 1
elif content[i] == '}':
depth -= 1
if depth == 0:
return content[start_pos + 1:i]
i += 1
return content[start_pos + 1:] # No closing brace found
def _parse_impl_methods(self, body: str) -> List[MethodInfo]:
"""Parse methods from an impl block body."""
methods = []
# Match: pub fn name(...) -> Type { ... }
pattern = r'pub\s+(?:async\s+)?fn\s+(\w+)\s*\(([^)]*)\)(?:\s*->\s*([^{]+))?\s*\{'
for match in re.finditer(pattern, body):
name = match.group(1)
params_str = match.group(2)
return_type = match.group(3).strip() if match.group(3) else "()"
# Skip internal methods
if name.startswith("_") or name in SKIP_METHODS:
continue
# Extract doc comments
doc = self._extract_doc_comments(body, match.start())
# Parse parameters
params = self._parse_params(params_str)
# Detect method type
is_static = "&self" not in params_str and "self" not in params_str
is_async = "async" in match.group(0)
methods.append(MethodInfo(
name=name,
signature=params_str,
return_type=return_type,
docstring=self._clean_docstring(doc),
params=params,
is_async=is_async,
is_static=is_static,
examples=self._extract_examples(doc),
))
return methods
def _parse_params(self, params_str: str) -> List[ParamInfo]:
"""Parse function/method parameters."""
params = []
if not params_str.strip():
return params
# Simple parameter parsing
# Handle: name: Type, &self, &mut self, self
for part in params_str.split(","):
part = part.strip()
if not part:
continue
# Skip self parameters
if part in ("self", "&self", "&mut self"):
continue
# Match: name: Type
param_match = re.match(r'(\w+)\s*:\s*(.+)', part)
if param_match:
name = param_match.group(1)
type_str = param_match.group(2).strip()
# Check for default (= value)
default = None
if "=" in type_str:
type_str, default = type_str.split("=", 1)
type_str = type_str.strip()
default = default.strip()
params.append(ParamInfo(
name=name,
type=type_str,
default=default,
required=default is None,
))
return params
def _extract_examples(self, doc: str) -> List[str]:
"""Extract code examples from documentation."""
examples = []
# Find ```rust or ``` blocks
pattern = r'```(?:rust(?:,ignore)?)?\n(.*?)```'
for match in re.finditer(pattern, doc, re.DOTALL):
examples.append(match.group(1).strip())
return examples
def _clean_docstring(self, doc: str) -> str:
"""Clean up a docstring for display."""
# Remove example blocks for the main description
doc = re.sub(r'# Example.*', '', doc, flags=re.DOTALL | re.IGNORECASE)
doc = re.sub(r'```.*?```', '', doc, flags=re.DOTALL)
# Clean up whitespace
lines = [line.strip() for line in doc.split("\n")]
lines = [line for line in lines if line]
return " ".join(lines)
# =============================================================================
# PROC-MACRO PARSER
# =============================================================================
class ProcMacroParser(RustDocParser):
"""Parser specialized for proc-macro crates (like praisonai-derive)."""
def parse_macros(self) -> List[MacroInfo]:
"""Parse proc-macro definitions."""
macros = []
lib_rs = self.src_path / "lib.rs"
if not lib_rs.exists():
return macros
content = self._read_file(lib_rs)
# Match: #[proc_macro_attribute] pub fn name(...) { ... }
pattern = r'#\[proc_macro(?:_attribute|_derive)?\]\s*pub\s+fn\s+(\w+)'
for match in re.finditer(pattern, content):
name = match.group(1)
# Extract doc comments
doc = self._extract_doc_comments(content, match.start())
macros.append(MacroInfo(
name=name,
docstring=self._clean_docstring(doc),
examples=self._extract_examples(doc),
))
return macros
# =============================================================================
# WORKSPACE PARSER
# =============================================================================
class RustWorkspaceParser:
"""Parse an entire Rust workspace with multiple crates."""
def __init__(self, workspace_path: Path):
"""Initialize the workspace parser.
Args:
workspace_path: Path to workspace root (containing Cargo.toml with [workspace])
"""
self.workspace_path = workspace_path
self.crates: Dict[str, RustDocParser] = {}
self._discover_crates()
def _discover_crates(self) -> None:
"""Discover crates in the workspace."""
# Parse workspace Cargo.toml
cargo_toml = self.workspace_path / "Cargo.toml"
if not cargo_toml.exists():
return
content = cargo_toml.read_text()
# Find workspace members
members_match = re.search(r'members\s*=\s*\[(.*?)\]', content, re.DOTALL)
if members_match:
members_str = members_match.group(1)
members = re.findall(r'"([^"]+)"', members_str)
for member in members:
member_path = self.workspace_path / member
if member_path.exists():
crate_name = member.replace("-", "_")
# Use ProcMacroParser for derive crate
if "derive" in member:
self.crates[crate_name] = ProcMacroParser(member_path, crate_name)
else:
self.crates[crate_name] = RustDocParser(member_path, crate_name)
def get_all_modules(self) -> Dict[str, List[str]]:
"""Get all modules from all crates.
Returns:
Dict mapping crate name to list of module paths
"""
result = {}
for crate_name, parser in self.crates.items():
result[crate_name] = parser.get_modules()
return result
def parse_all(self) -> Dict[str, List[ModuleInfo]]:
"""Parse all modules from all crates.
Returns:
Dict mapping crate name to list of ModuleInfo
"""
result = {}
for crate_name, parser in self.crates.items():
modules = []
for module_path in parser.get_modules():
module_info = parser.parse_module(module_path)
if module_info:
modules.append(module_info)
result[crate_name] = modules
return result
+267
View File
@@ -0,0 +1,267 @@
"""
FCP Tool - Final Cut Pro automation tools for Agent Recipes.
This module provides tools for:
- EditIntent schema validation and generation
- FCPXML generation from EditIntent
- CommandPost integration for auto-import
- Injection daemon for batch processing
- Health checks and diagnostics
Usage:
from praisonai_tools.fcp_tool import (
fcp_autoedit,
fcp_doctor,
EditIntent,
FCPXMLGenerator,
)
"""
from .intent import (
EditIntent,
Asset,
Segment,
Marker,
Timeline,
Project,
ProjectFormat,
AudioSettings,
AudioRole,
AudioLayout,
Operations,
seconds_to_rational,
rational_to_seconds,
get_format_preset,
FORMAT_PRESETS,
)
from .fcpxml import FCPXMLGenerator, generate_fcpxml, validate_fcpxml_structure
from .commandpost import CommandPostBridge, CommandPostStatus
from .injector import Injector, InjectionJob, JobStatus, DaemonState
from .doctor import FCPDoctor, CheckResult
from .prompting import generate_edit_intent, create_simple_intent
__all__ = [
# Intent schema
"EditIntent",
"Asset",
"Segment",
"Marker",
"Timeline",
"Project",
"ProjectFormat",
"AudioSettings",
"AudioRole",
"AudioLayout",
"Operations",
"seconds_to_rational",
"rational_to_seconds",
"get_format_preset",
"FORMAT_PRESETS",
# FCPXML
"FCPXMLGenerator",
"generate_fcpxml",
"validate_fcpxml_structure",
# CommandPost
"CommandPostBridge",
"CommandPostStatus",
# Injector
"Injector",
"InjectionJob",
"JobStatus",
"DaemonState",
# Doctor
"FCPDoctor",
"CheckResult",
# Prompting
"generate_edit_intent",
"create_simple_intent",
]
def fcp_doctor() -> dict:
"""
Run health checks for FCP integration.
Returns:
dict: Health check results with 'all_passed' boolean and 'checks' list
"""
doctor = FCPDoctor()
results = doctor.run_all_checks()
summary = doctor.get_summary(results)
return {
"success": True,
"all_passed": summary["all_passed"],
"passed": summary["passed"],
"failed": summary["failed"],
"checks": summary["checks"],
}
def fcp_autoedit(
action: str,
instruction: str = None,
media_paths: list = None,
project_name: str = "AI Generated Project",
format_preset: str = "1080p25",
intent_json: str = None,
fcpxml_content: str = None,
watch_folder: str = None,
dry_run: bool = False,
model: str = "gpt-4o",
) -> dict:
"""
FCP AutoEdit tool for generating and injecting FCPXML.
Actions:
- "generate_intent": Generate EditIntent from instruction
- "generate_fcpxml": Generate FCPXML from EditIntent
- "inject": Inject FCPXML into Final Cut Pro
- "full_pipeline": Run the complete pipeline
Args:
action: The action to perform
instruction: Natural language editing instruction
media_paths: List of media file paths
project_name: Project name
format_preset: Format preset (e.g., "1080p25")
intent_json: EditIntent JSON string (for generate_fcpxml)
fcpxml_content: FCPXML content (for inject)
watch_folder: Watch folder path
dry_run: If True, don't inject into FCP
model: OpenAI model to use
Returns:
dict: Result with success status and relevant data
"""
import json
import os
if action == "generate_intent":
if not instruction:
return {"success": False, "error": "instruction is required"}
if not media_paths:
return {"success": False, "error": "media_paths is required"}
abs_paths = [os.path.abspath(p) for p in media_paths]
missing = [p for p in abs_paths if not os.path.exists(p)]
if missing:
return {"success": False, "error": f"Media files not found: {missing}"}
intent, warnings = generate_edit_intent(
instruction=instruction,
media_paths=abs_paths,
project_name=project_name,
format_preset=format_preset,
model=model,
)
return {
"success": True,
"intent_json": intent.model_dump_json(),
"intent": intent.model_dump(),
"warnings": warnings,
"assets_count": len(intent.assets),
"segments_count": len(intent.timeline.segments),
}
elif action == "generate_fcpxml":
if not intent_json:
return {"success": False, "error": "intent_json is required"}
try:
intent_data = json.loads(intent_json)
intent = EditIntent.model_validate(intent_data)
except json.JSONDecodeError as e:
return {"success": False, "error": f"Invalid JSON: {e}"}
except Exception as e:
return {"success": False, "error": f"Invalid EditIntent: {e}"}
generator = FCPXMLGenerator(intent)
fcpxml = generator.generate()
warnings = generator.get_warnings()
return {
"success": True,
"fcpxml": fcpxml,
"fcpxml_length": len(fcpxml),
"warnings": warnings,
}
elif action == "inject":
if not fcpxml_content:
return {"success": False, "error": "fcpxml_content is required"}
commandpost = CommandPostBridge(watch_folder=watch_folder)
injector = Injector(watch_folder=watch_folder, commandpost=commandpost)
job_id, fcpxml_path, messages = injector.inject_one_shot(
fcpxml_content=fcpxml_content,
instruction=instruction or "",
intent_json=intent_json,
)
return {
"success": True,
"job_id": job_id,
"fcpxml_path": fcpxml_path,
"messages": messages,
}
elif action == "full_pipeline":
# Generate intent
intent_result = fcp_autoedit(
action="generate_intent",
instruction=instruction,
media_paths=media_paths,
project_name=project_name,
format_preset=format_preset,
model=model,
)
if not intent_result["success"]:
return intent_result
# Generate FCPXML
fcpxml_result = fcp_autoedit(
action="generate_fcpxml",
intent_json=intent_result["intent_json"],
)
if not fcpxml_result["success"]:
return fcpxml_result
all_warnings = intent_result.get("warnings", []) + fcpxml_result.get("warnings", [])
if dry_run:
return {
"success": True,
"dry_run": True,
"intent": intent_result["intent"],
"fcpxml_length": fcpxml_result["fcpxml_length"],
"warnings": all_warnings,
}
# Inject
inject_result = fcp_autoedit(
action="inject",
fcpxml_content=fcpxml_result["fcpxml"],
instruction=instruction,
intent_json=intent_result["intent_json"],
watch_folder=watch_folder,
)
if not inject_result["success"]:
return inject_result
return {
"success": True,
"job_id": inject_result["job_id"],
"fcpxml_path": inject_result["fcpxml_path"],
"assets_count": intent_result["assets_count"],
"segments_count": intent_result["segments_count"],
"warnings": all_warnings,
"messages": inject_result["messages"],
}
else:
return {
"success": False,
"error": f"Unknown action: {action}. Valid: generate_intent, generate_fcpxml, inject, full_pipeline",
}
+310
View File
@@ -0,0 +1,310 @@
"""
CLI for FCP Tool - Final Cut Pro automation commands.
Usage:
python -m praisonai_tools.fcp_tool.cli doctor
python -m praisonai_tools.fcp_tool.cli autoedit --instruction "..." --media /path/to/file.mov
python -m praisonai_tools.fcp_tool.cli daemon start|stop|status
python -m praisonai_tools.fcp_tool.cli bootstrap
"""
from __future__ import annotations
import argparse
import json
import sys
from typing import Optional
def main(args: Optional[list[str]] = None) -> int:
"""Main CLI entry point."""
parser = argparse.ArgumentParser(
prog="praisonai-tools-fcp",
description="Final Cut Pro automation tools",
)
subparsers = parser.add_subparsers(dest="command", help="Available commands")
# Doctor command
doctor_parser = subparsers.add_parser("doctor", help="Run health checks")
doctor_parser.add_argument("--json", action="store_true", help="Output as JSON")
# Autoedit command
autoedit_parser = subparsers.add_parser("autoedit", help="Generate and inject FCPXML")
autoedit_parser.add_argument(
"--instruction", "-i", type=str, help="Editing instruction"
)
autoedit_parser.add_argument(
"--instruction-file", "-f", type=str, help="File containing instruction"
)
autoedit_parser.add_argument(
"--media", "-m", action="append", dest="media_paths", help="Media file path (repeatable)"
)
autoedit_parser.add_argument(
"--project-name", "-n", type=str, default="Untitled Project", help="Project name"
)
autoedit_parser.add_argument(
"--format", type=str, default="1080p25", help="Format preset"
)
autoedit_parser.add_argument(
"--model", type=str, default="gpt-4o", help="OpenAI model"
)
autoedit_parser.add_argument(
"--dry-run", action="store_true", help="Generate without injecting"
)
autoedit_parser.add_argument(
"--simple", action="store_true", help="Use simple concatenation (no LLM)"
)
autoedit_parser.add_argument(
"--print-intent", action="store_true", help="Print EditIntent JSON"
)
autoedit_parser.add_argument(
"--print-fcpxml", action="store_true", help="Print generated FCPXML"
)
autoedit_parser.add_argument("--json", action="store_true", help="Output as JSON")
# Daemon command
daemon_parser = subparsers.add_parser("daemon", help="Manage injection daemon")
daemon_parser.add_argument(
"action", choices=["start", "stop", "status"], help="Daemon action"
)
daemon_parser.add_argument("--json", action="store_true", help="Output as JSON")
# Bootstrap command
bootstrap_parser = subparsers.add_parser(
"bootstrap", help="One-time CommandPost setup"
)
bootstrap_parser.add_argument("--json", action="store_true", help="Output as JSON")
parsed = parser.parse_args(args)
if not parsed.command:
parser.print_help()
return 1
if parsed.command == "doctor":
return cmd_doctor(parsed)
elif parsed.command == "autoedit":
return cmd_autoedit(parsed)
elif parsed.command == "daemon":
return cmd_daemon(parsed)
elif parsed.command == "bootstrap":
return cmd_bootstrap(parsed)
return 1
def cmd_doctor(args: argparse.Namespace) -> int:
"""Run health checks."""
from .doctor import FCPDoctor
doctor = FCPDoctor()
results = doctor.run_all_checks()
if args.json:
summary = doctor.get_summary(results)
print(json.dumps(summary, indent=2))
else:
doctor.print_report(results)
return 0
def cmd_autoedit(args: argparse.Namespace) -> int:
"""Generate and inject FCPXML."""
import os
from .fcpxml import FCPXMLGenerator
from .prompting import generate_edit_intent, create_simple_intent
from .injector import Injector
from .commandpost import CommandPostBridge
instruction = args.instruction
if args.instruction_file:
with open(args.instruction_file, "r") as f:
instruction = f.read().strip()
media_paths = args.media_paths or []
if not media_paths:
print("Error: At least one --media path is required", file=sys.stderr)
return 1
abs_paths = [os.path.abspath(p) for p in media_paths]
missing = [p for p in abs_paths if not os.path.exists(p)]
if missing:
print(f"Error: Media files not found: {missing}", file=sys.stderr)
return 1
try:
if args.simple or not instruction:
if not instruction:
print("No instruction provided, using simple concatenation mode")
intent = create_simple_intent(
media_paths=abs_paths,
project_name=args.project_name,
format_preset=args.format,
)
warnings = []
else:
intent, warnings = generate_edit_intent(
instruction=instruction,
media_paths=abs_paths,
project_name=args.project_name,
format_preset=args.format,
model=args.model,
)
if args.print_intent:
print("\n=== EditIntent JSON ===")
print(intent.model_dump_json(indent=2))
generator = FCPXMLGenerator(intent)
fcpxml = generator.generate()
warnings.extend(generator.get_warnings())
if args.print_fcpxml:
print("\n=== Generated FCPXML ===")
print(fcpxml)
if warnings:
print("\n=== Warnings ===")
for w in warnings:
print(f" ⚠ {w}")
if args.dry_run:
if args.json:
print(json.dumps({
"success": True,
"dry_run": True,
"intent": intent.model_dump(),
"fcpxml_length": len(fcpxml),
"warnings": warnings,
}, indent=2))
else:
print("\n=== Dry Run Complete ===")
print(f"FCPXML generated: {len(fcpxml)} characters")
print("No injection performed (--dry-run)")
return 0
commandpost = CommandPostBridge()
injector = Injector(commandpost=commandpost)
job_id, fcpxml_path, messages = injector.inject_one_shot(
fcpxml_content=fcpxml,
instruction=instruction or "Simple concatenation",
intent_json=intent.model_dump_json(),
)
if args.json:
print(json.dumps({
"success": True,
"job_id": job_id,
"fcpxml_path": fcpxml_path,
"messages": messages,
"warnings": warnings,
}, indent=2))
else:
print("\n=== Injection Complete ===")
print(f"Job ID: {job_id}")
print(f"FCPXML: {fcpxml_path}")
for msg in messages:
print(f" • {msg}")
return 0
except Exception as e:
if args.json:
print(json.dumps({"success": False, "error": str(e)}, indent=2))
else:
print(f"Error: {e}", file=sys.stderr)
return 1
def cmd_daemon(args: argparse.Namespace) -> int:
"""Manage injection daemon."""
from .injector import Injector
injector = Injector()
if args.action == "start":
if injector.is_daemon_running():
msg = "Daemon is already running"
if args.json:
print(json.dumps({"success": False, "error": msg}))
else:
print(msg)
return 1
print("Starting daemon...")
success = injector.start_daemon(detach=True)
if args.json:
print(json.dumps({"success": success}))
else:
print("Daemon started" if success else "Failed to start daemon")
return 0 if success else 1
elif args.action == "stop":
success = injector.stop_daemon()
if args.json:
print(json.dumps({"success": success}))
else:
print("Daemon stopped" if success else "Daemon not running or failed to stop")
return 0 if success else 1
elif args.action == "status":
running = injector.is_daemon_running()
state = injector.get_daemon_state()
if args.json:
print(json.dumps({
"running": running,
"state": {
"pid": state.pid if state else None,
"started_at": state.started_at if state else None,
"watch_folder": state.watch_folder if state else None,
"jobs_processed": state.jobs_processed if state else 0,
"last_job_at": state.last_job_at if state else None,
} if state else None,
}, indent=2))
else:
if running and state:
print(f"Daemon running (PID: {state.pid})")
print(f" Started: {state.started_at}")
print(f" Watch folder: {state.watch_folder}")
print(f" Jobs processed: {state.jobs_processed}")
if state.last_job_at:
print(f" Last job: {state.last_job_at}")
else:
print("Daemon not running")
return 0
return 1
def cmd_bootstrap(args: argparse.Namespace) -> int:
"""One-time CommandPost setup."""
from .commandpost import CommandPostBridge
bridge = CommandPostBridge()
results = bridge.bootstrap()
if args.json:
print(json.dumps({
"results": [{"success": s, "message": m} for s, m in results],
"all_success": all(s for s, _ in results),
}, indent=2))
else:
print("\n=== CommandPost Bootstrap ===")
for success, message in results:
status = "✓" if success else "✗"
print(f" {status} {message}")
print()
if all(s for s, _ in results):
print("Bootstrap complete!")
else:
print("Some steps failed. See above for details.")
return 0 if all(s for s, _ in results) else 1
if __name__ == "__main__":
sys.exit(main())
+361
View File
@@ -0,0 +1,361 @@
"""
CommandPost Bridge for Final Cut Pro Integration
Interfaces with CommandPost's cmdpost CLI to configure watch folders
and trigger FCPXML auto-import into Final Cut Pro.
"""
from __future__ import annotations
import os
import platform
import shutil
import subprocess
from dataclasses import dataclass
from pathlib import Path
from typing import Optional
@dataclass
class CommandPostStatus:
"""Status of CommandPost installation and configuration."""
is_macos: bool
commandpost_installed: bool
cmdpost_available: bool
cmdpost_path: Optional[str]
watch_folder_configured: bool
watch_folder_path: Optional[str]
auto_import_enabled: bool
errors: list[str]
@property
def is_ready(self) -> bool:
"""Check if CommandPost is ready for auto-import."""
return (
self.is_macos
and self.commandpost_installed
and self.cmdpost_available
and self.watch_folder_configured
and self.auto_import_enabled
)
class CommandPostBridge:
"""
Bridge to CommandPost for FCPXML auto-import.
CommandPost is a macOS app that extends Final Cut Pro with automation
capabilities. This bridge uses the cmdpost CLI to configure watch folders
and enable automatic FCPXML import.
"""
KNOWN_CMDPOST_PATHS = [
"/Applications/CommandPost.app/Contents/Resources/extensions/hs/cmdpost",
"/usr/local/bin/cmdpost",
os.path.expanduser("~/Applications/CommandPost.app/Contents/Resources/extensions/hs/cmdpost"),
]
COMMANDPOST_APP_PATHS = [
"/Applications/CommandPost.app",
os.path.expanduser("~/Applications/CommandPost.app"),
]
DEFAULT_WATCH_FOLDER = os.path.expanduser("~/.praison/fcp/watch/fcpxml")
def __init__(self, cmdpost_path: Optional[str] = None, watch_folder: Optional[str] = None):
"""
Initialize the CommandPost bridge.
Args:
cmdpost_path: Path to cmdpost CLI (auto-detected if not provided)
watch_folder: Watch folder path (uses default if not provided)
"""
self.cmdpost_path = cmdpost_path or self._find_cmdpost()
self.watch_folder = watch_folder or self.DEFAULT_WATCH_FOLDER
def _find_cmdpost(self) -> Optional[str]:
"""Find the cmdpost CLI binary."""
cmdpost_in_path = shutil.which("cmdpost")
if cmdpost_in_path:
return cmdpost_in_path
for path in self.KNOWN_CMDPOST_PATHS:
if os.path.isfile(path) and os.access(path, os.X_OK):
return path
return None
def _is_commandpost_installed(self) -> bool:
"""Check if CommandPost app is installed."""
for path in self.COMMANDPOST_APP_PATHS:
if os.path.isdir(path):
return True
return False
def get_status(self) -> CommandPostStatus:
"""Get the current status of CommandPost configuration."""
is_macos = platform.system() == "Darwin"
errors = []
if not is_macos:
errors.append("CommandPost is only available on macOS")
return CommandPostStatus(
is_macos=False,
commandpost_installed=False,
cmdpost_available=False,
cmdpost_path=None,
watch_folder_configured=False,
watch_folder_path=None,
auto_import_enabled=False,
errors=errors,
)
commandpost_installed = self._is_commandpost_installed()
if not commandpost_installed:
errors.append("CommandPost is not installed. Download from https://commandpost.io")
cmdpost_available = self.cmdpost_path is not None
if not cmdpost_available:
errors.append("cmdpost CLI not found. Ensure CommandPost is installed and running.")
watch_folder_configured = os.path.isdir(self.watch_folder)
auto_import_enabled = False
if cmdpost_available and watch_folder_configured:
auto_import_enabled = self._check_auto_import_enabled()
return CommandPostStatus(
is_macos=is_macos,
commandpost_installed=commandpost_installed,
cmdpost_available=cmdpost_available,
cmdpost_path=self.cmdpost_path,
watch_folder_configured=watch_folder_configured,
watch_folder_path=self.watch_folder if watch_folder_configured else None,
auto_import_enabled=auto_import_enabled,
errors=errors,
)
def _check_auto_import_enabled(self) -> bool:
"""Check if auto-import is enabled in CommandPost."""
if not self.cmdpost_path:
return False
lua_script = '''
local fcpxmlPlugin = require("cp.plugins")("finalcutpro.watchfolders.fcpxml")
if fcpxmlPlugin and fcpxmlPlugin.automaticallyImport then
print(fcpxmlPlugin.automaticallyImport() and "true" or "false")
else
print("unknown")
end
'''
try:
result = subprocess.run(
[self.cmdpost_path, "-c", lua_script],
capture_output=True,
text=True,
timeout=10,
)
return result.stdout.strip() == "true"
except (subprocess.TimeoutExpired, subprocess.SubprocessError):
return False
def setup_watch_folder(self) -> tuple[bool, str]:
"""
Set up the watch folder for FCPXML auto-import.
Returns:
Tuple of (success, message)
"""
try:
Path(self.watch_folder).mkdir(parents=True, exist_ok=True)
return True, f"Watch folder created: {self.watch_folder}"
except OSError as e:
return False, f"Failed to create watch folder: {e}"
def configure_auto_import(self, enable: bool = True) -> tuple[bool, str]:
"""
Configure CommandPost to automatically import FCPXML files.
Args:
enable: Whether to enable auto-import
Returns:
Tuple of (success, message)
"""
if not self.cmdpost_path:
return False, "cmdpost CLI not available"
lua_script = f'''
local fcpxmlPlugin = require("cp.plugins")("finalcutpro.watchfolders.fcpxml")
if fcpxmlPlugin and fcpxmlPlugin.automaticallyImport then
fcpxmlPlugin.automaticallyImport({"true" if enable else "false"})
print("configured")
else
print("plugin_not_found")
end
'''
try:
result = subprocess.run(
[self.cmdpost_path, "-c", lua_script],
capture_output=True,
text=True,
timeout=10,
)
if "configured" in result.stdout:
return True, f"Auto-import {'enabled' if enable else 'disabled'}"
elif "plugin_not_found" in result.stdout:
return False, "FCPXML watch folder plugin not found in CommandPost"
else:
return False, f"Unexpected response: {result.stdout}"
except subprocess.TimeoutExpired:
return False, "CommandPost command timed out"
except subprocess.SubprocessError as e:
return False, f"Failed to configure CommandPost: {e}"
def add_watch_folder(self) -> tuple[bool, str]:
"""
Add the watch folder to CommandPost's FCPXML watch list.
Returns:
Tuple of (success, message)
"""
if not self.cmdpost_path:
return False, "cmdpost CLI not available"
escaped_path = self.watch_folder.replace('"', '\\"')
lua_script = f'''
local fcpxmlPlugin = require("cp.plugins")("finalcutpro.watchfolders.fcpxml")
if fcpxmlPlugin and fcpxmlPlugin.addWatchFolder then
fcpxmlPlugin.addWatchFolder("{escaped_path}")
print("added")
elseif fcpxmlPlugin then
-- Try alternative method
local watchFolders = fcpxmlPlugin.watchFolders
if watchFolders and watchFolders.addFolder then
watchFolders.addFolder("{escaped_path}")
print("added")
else
print("method_not_found")
end
else
print("plugin_not_found")
end
'''
try:
result = subprocess.run(
[self.cmdpost_path, "-c", lua_script],
capture_output=True,
text=True,
timeout=10,
)
if "added" in result.stdout:
return True, f"Watch folder added: {self.watch_folder}"
elif "method_not_found" in result.stdout:
return False, (
"Watch folder API not available. "
"Please add the watch folder manually in CommandPost preferences."
)
elif "plugin_not_found" in result.stdout:
return False, "FCPXML watch folder plugin not found in CommandPost"
else:
return False, f"Unexpected response: {result.stdout}"
except subprocess.TimeoutExpired:
return False, "CommandPost command timed out"
except subprocess.SubprocessError as e:
return False, f"Failed to add watch folder: {e}"
def trigger_import(self, fcpxml_path: str) -> tuple[bool, str]:
"""
Trigger import of a specific FCPXML file.
This is an alternative to watch folder auto-import that directly
calls CommandPost to import a file.
Args:
fcpxml_path: Path to the FCPXML file
Returns:
Tuple of (success, message)
"""
if not self.cmdpost_path:
return False, "cmdpost CLI not available"
if not os.path.isfile(fcpxml_path):
return False, f"FCPXML file not found: {fcpxml_path}"
escaped_path = fcpxml_path.replace('"', '\\"')
lua_script = f'''
local fcpxmlPlugin = require("cp.plugins")("finalcutpro.watchfolders.fcpxml")
if fcpxmlPlugin and fcpxmlPlugin.insertFilesIntoFinalCutPro then
fcpxmlPlugin.insertFilesIntoFinalCutPro({{"{escaped_path}"}})
print("imported")
elseif fcpxmlPlugin and fcpxmlPlugin.importFile then
fcpxmlPlugin.importFile("{escaped_path}")
print("imported")
else
print("import_not_available")
end
'''
try:
result = subprocess.run(
[self.cmdpost_path, "-c", lua_script],
capture_output=True,
text=True,
timeout=30,
)
if "imported" in result.stdout:
return True, f"FCPXML imported: {fcpxml_path}"
elif "import_not_available" in result.stdout:
return False, (
"Direct import not available. "
"File will be imported via watch folder if configured."
)
else:
return False, f"Import may have failed: {result.stdout}"
except subprocess.TimeoutExpired:
return False, "Import command timed out"
except subprocess.SubprocessError as e:
return False, f"Failed to import: {e}"
def bring_fcp_to_front(self) -> tuple[bool, str]:
"""
Bring Final Cut Pro to the foreground.
Returns:
Tuple of (success, message)
"""
if platform.system() != "Darwin":
return False, "Only available on macOS"
try:
subprocess.run(
["osascript", "-e", 'tell application "Final Cut Pro" to activate'],
capture_output=True,
timeout=5,
)
return True, "Final Cut Pro activated"
except subprocess.SubprocessError as e:
return False, f"Failed to activate Final Cut Pro: {e}"
def bootstrap(self) -> list[tuple[bool, str]]:
"""
Perform one-time bootstrap setup for CommandPost integration.
Returns:
List of (success, message) tuples for each setup step
"""
results = []
success, msg = self.setup_watch_folder()
results.append((success, msg))
if self.cmdpost_path:
success, msg = self.configure_auto_import(True)
results.append((success, msg))
success, msg = self.add_watch_folder()
results.append((success, msg))
else:
results.append((False, "cmdpost CLI not available - skipping configuration"))
return results
+285
View File
@@ -0,0 +1,285 @@
"""
FCP Doctor - Health Checks for Final Cut Pro Integration
Provides diagnostic checks and remediation guidance for the FCP integration.
"""
from __future__ import annotations
import os
import platform
import shutil
from dataclasses import dataclass
from typing import Optional
from .commandpost import CommandPostBridge
@dataclass
class CheckResult:
"""Result of a health check."""
name: str
passed: bool
message: str
remediation: Optional[str] = None
class FCPDoctor:
"""
Diagnostic tool for FCP integration health checks.
Checks for:
- macOS environment
- Final Cut Pro installation
- CommandPost installation
- cmdpost CLI availability
- Watch folder configuration
"""
def __init__(self, commandpost: Optional[CommandPostBridge] = None):
"""
Initialize the doctor.
Args:
commandpost: CommandPost bridge instance
"""
self.commandpost = commandpost or CommandPostBridge()
def run_all_checks(self) -> list[CheckResult]:
"""Run all health checks."""
return [
self.check_macos(),
self.check_final_cut_pro(),
self.check_commandpost_app(),
self.check_cmdpost_cli(),
self.check_watch_folder(),
self.check_auto_import(),
self.check_openai_api_key(),
]
def check_macos(self) -> CheckResult:
"""Check if running on macOS."""
is_macos = platform.system() == "Darwin"
if is_macos:
version = platform.mac_ver()[0]
return CheckResult(
name="macOS",
passed=True,
message=f"Running on macOS {version}",
)
else:
return CheckResult(
name="macOS",
passed=False,
message=f"Running on {platform.system()}, not macOS",
remediation="FCP integration requires macOS. Other platforms can generate FCPXML but cannot auto-import.",
)
def check_final_cut_pro(self) -> CheckResult:
"""Check if Final Cut Pro is installed."""
fcp_paths = [
"/Applications/Final Cut Pro.app",
os.path.expanduser("~/Applications/Final Cut Pro.app"),
]
for path in fcp_paths:
if os.path.isdir(path):
return CheckResult(
name="Final Cut Pro",
passed=True,
message=f"Found at {path}",
)
return CheckResult(
name="Final Cut Pro",
passed=False,
message="Final Cut Pro not found",
remediation="Install Final Cut Pro from the Mac App Store.",
)
def check_commandpost_app(self) -> CheckResult:
"""Check if CommandPost app is installed."""
cp_paths = [
"/Applications/CommandPost.app",
"/Applications/Utilities/CommandPost.app",
os.path.expanduser("~/Applications/CommandPost.app"),
]
for path in cp_paths:
if os.path.isdir(path):
return CheckResult(
name="CommandPost App",
passed=True,
message=f"Found at {path}",
)
return CheckResult(
name="CommandPost App",
passed=False,
message="CommandPost not found",
remediation="Download CommandPost from https://commandpost.io",
)
def check_cmdpost_cli(self) -> CheckResult:
"""Check if cmdpost CLI is available."""
cmdpost_path = self.commandpost.cmdpost_path
if cmdpost_path:
return CheckResult(
name="cmdpost CLI",
passed=True,
message=f"Found at {cmdpost_path}",
)
cmdpost_in_path = shutil.which("cmdpost")
if cmdpost_in_path:
return CheckResult(
name="cmdpost CLI",
passed=True,
message=f"Found in PATH at {cmdpost_in_path}",
)
return CheckResult(
name="cmdpost CLI",
passed=False,
message="cmdpost CLI not found",
remediation=(
"Ensure CommandPost is installed and running. "
"The cmdpost CLI is bundled with CommandPost at: "
"/Applications/CommandPost.app/Contents/Resources/extensions/hs/cmdpost"
),
)
def check_watch_folder(self) -> CheckResult:
"""Check if watch folder exists."""
watch_folder = self.commandpost.watch_folder
if os.path.isdir(watch_folder):
return CheckResult(
name="Watch Folder",
passed=True,
message=f"Exists at {watch_folder}",
)
return CheckResult(
name="Watch Folder",
passed=False,
message=f"Watch folder not found: {watch_folder}",
remediation=f"Run: mkdir -p {watch_folder}",
)
def check_auto_import(self) -> CheckResult:
"""Check if auto-import is enabled in CommandPost."""
status = self.commandpost.get_status()
if not status.cmdpost_available:
return CheckResult(
name="Auto-Import",
passed=False,
message="Cannot check - cmdpost CLI not available",
remediation="Install and run CommandPost first.",
)
if status.auto_import_enabled:
return CheckResult(
name="Auto-Import",
passed=True,
message="Enabled in CommandPost",
)
return CheckResult(
name="Auto-Import",
passed=False,
message="Auto-import not enabled or could not be verified",
remediation=(
"Run: praisonai-tools fcp bootstrap\n"
"Or manually enable in CommandPost preferences."
),
)
def check_openai_api_key(self) -> CheckResult:
"""Check if OpenAI API key is configured."""
api_key = os.environ.get("OPENAI_API_KEY")
if api_key:
masked = f"...{api_key[-4:]}" if len(api_key) > 4 else "****"
return CheckResult(
name="OpenAI API Key",
passed=True,
message=f"Set (ending in {masked})",
)
return CheckResult(
name="OpenAI API Key",
passed=False,
message="OPENAI_API_KEY not set",
remediation="Set with: export OPENAI_API_KEY=your-key",
)
def print_report(self, results: Optional[list[CheckResult]] = None) -> bool:
"""
Print a formatted health check report.
Args:
results: Check results (runs all checks if not provided)
Returns:
True if all checks passed
"""
if results is None:
results = self.run_all_checks()
print("\n" + "=" * 60)
print("FCP Integration Health Check")
print("=" * 60 + "\n")
all_passed = True
for result in results:
status = "✓" if result.passed else "✗"
print(f" {status} {result.name}: {result.message}")
if not result.passed:
all_passed = False
if result.remediation:
print(f" → {result.remediation}")
print()
print("=" * 60)
if all_passed:
print("All checks passed! FCP integration is ready.")
else:
print("Some checks failed. See remediation steps above.")
print("=" * 60 + "\n")
return all_passed
def get_summary(self, results: Optional[list[CheckResult]] = None) -> dict:
"""
Get a summary of health check results.
Args:
results: Check results (runs all checks if not provided)
Returns:
Summary dictionary
"""
if results is None:
results = self.run_all_checks()
passed = [r for r in results if r.passed]
failed = [r for r in results if not r.passed]
return {
"total": len(results),
"passed": len(passed),
"failed": len(failed),
"all_passed": len(failed) == 0,
"checks": [
{
"name": r.name,
"passed": r.passed,
"message": r.message,
"remediation": r.remediation,
}
for r in results
],
}
+419
View File
@@ -0,0 +1,419 @@
"""
FCPXML Generator for Final Cut Pro Integration
Converts EditIntent to valid FCPXML 1.11 format.
"""
from __future__ import annotations
import xml.etree.ElementTree as ET
from typing import Optional
from xml.dom import minidom
from .intent import EditIntent, Segment
class FCPXMLGenerator:
"""
Generates FCPXML from EditIntent.
FCPXML is the interchange format for Final Cut Pro. This generator
produces valid FCPXML 1.11 that can be imported into FCP.
"""
FCPXML_VERSION = "1.11"
def __init__(self, intent: EditIntent):
"""
Initialize the generator with an EditIntent.
Args:
intent: The EditIntent to convert to FCPXML
"""
self.intent = intent
self._warnings: list[str] = []
def generate(self, pretty_print: bool = True) -> str:
"""
Generate FCPXML string from the intent.
Args:
pretty_print: Whether to format the XML with indentation
Returns:
FCPXML string
"""
self._warnings = list(self.intent.get_warnings())
root = ET.Element("fcpxml", version=self.FCPXML_VERSION)
resources = ET.SubElement(root, "resources")
self._add_format_resource(resources)
self._add_asset_resources(resources)
library = ET.SubElement(root, "library")
event = ET.SubElement(library, "event", name=self.intent.project.name)
self._add_project(event)
xml_str = ET.tostring(root, encoding="unicode")
if pretty_print:
xml_str = self._pretty_print(xml_str)
xml_declaration = '<?xml version="1.0" encoding="UTF-8"?>\n'
doctype = '<!DOCTYPE fcpxml>\n'
return xml_declaration + doctype + xml_str
def get_warnings(self) -> list[str]:
"""Get warnings generated during FCPXML generation."""
return self._warnings
def _add_format_resource(self, resources: ET.Element) -> None:
"""Add the format resource definition."""
fmt = self.intent.project.format
ET.SubElement(
resources,
"format",
id="r1",
name=f"FFVideoFormat{fmt.height}p{int(fmt.fps)}",
frameDuration=fmt.frame_duration_rational,
width=str(fmt.width),
height=str(fmt.height),
)
# Note: audioChannels and audioSampleRate are NOT valid on format element per DTD
def _add_asset_resources(self, resources: ET.Element) -> None:
"""Add asset resource definitions."""
for asset in self.intent.assets:
asset_elem = ET.SubElement(
resources,
"asset",
id=asset.id,
name=asset.name,
uid=asset.uid,
)
# Note: src is NOT valid on asset element per DTD - only on media-rep
if asset.has_video:
asset_elem.set("hasVideo", "1")
if asset.format_ref:
asset_elem.set("format", asset.format_ref)
else:
asset_elem.set("format", "r1")
if asset.has_audio:
asset_elem.set("hasAudio", "1")
asset_elem.set("audioSources", "1")
asset_elem.set("audioChannels", str(asset.audio_channels))
# audioRate must be one of: 32k, 44.1k, 48k, 88.2k, 96k, 176.4k, 192k
audio_rate_map = {
32000: "32k", 44100: "44.1k", 48000: "48k",
88200: "88.2k", 96000: "96k", 176400: "176.4k", 192000: "192k"
}
asset_elem.set("audioRate", audio_rate_map.get(asset.audio_rate, "48k"))
if asset.duration_rational:
asset_elem.set("duration", asset.duration_rational)
ET.SubElement(
asset_elem,
"media-rep",
kind="original-media",
src=asset.get_src_url(),
)
def _add_project(self, event: ET.Element) -> None:
"""Add the project with sequence and spine."""
project = ET.SubElement(
event,
"project",
name=self.intent.project.name,
)
total_duration = self._calculate_total_duration()
sequence = ET.SubElement(
project,
"sequence",
format="r1",
duration=total_duration,
tcStart="0/2500s",
tcFormat="NDF",
)
sequence.set("audioLayout", self.intent.project.audio.layout.value)
# audioRate must be one of: 32k, 44.1k, 48k, 88.2k, 96k, 176.4k, 192k
audio_rate_map = {
32000: "32k", 44100: "44.1k", 48000: "48k",
88200: "88.2k", 96000: "96k", 176400: "176.4k", 192000: "192k"
}
sequence.set("audioRate", audio_rate_map.get(self.intent.project.audio.rate, "48k"))
spine = ET.SubElement(sequence, "spine")
self._add_timeline_clips(spine)
# Note: markers are added inside asset-clips, not on sequence
# Sequence content must be (note?, spine, metadata?) per DTD
def _add_timeline_clips(self, spine: ET.Element) -> None:
"""Add clips to the spine based on timeline segments."""
primary_segments = [s for s in self.intent.timeline.segments if s.lane == 0]
connected_segments = [s for s in self.intent.timeline.segments if s.lane != 0]
primary_segments.sort(key=lambda s: self._rational_to_frames(s.offset))
for i, segment in enumerate(primary_segments):
self._add_asset_clip(spine, segment, is_first=(i == 0))
for segment in connected_segments:
self._warnings.append(
f"Connected clip (lane={segment.lane}) for asset '{segment.asset_id}' "
"added as primary storyline clip - connected clips not fully supported in v1"
)
self._add_asset_clip(spine, segment)
def _add_asset_clip(self, parent: ET.Element, segment: Segment, is_first: bool = False) -> None:
"""Add an asset-clip element for a segment."""
asset = self.intent.get_asset_by_id(segment.asset_id)
if not asset:
self._warnings.append(f"Asset not found for segment: {segment.asset_id}")
return
clip_name = segment.name or asset.name
clip = ET.SubElement(
parent,
"asset-clip",
name=clip_name,
ref=asset.id,
offset=segment.offset,
duration=segment.duration,
start=segment.start,
)
if asset.has_audio:
clip.set("audioRole", segment.role.value if segment.role else "dialogue")
if segment.volume is not None and segment.volume != 1.0:
adjust = ET.SubElement(clip, "adjust-volume")
adjust.set("amount", f"{segment.volume * 100 - 100:+.1f}dB")
# Add markers inside the clip (first clip gets all markers)
if is_first:
for marker in self.intent.timeline.markers:
ET.SubElement(
clip,
"marker",
start=marker.start,
duration=marker.duration,
value=marker.name,
)
# Markers are now added inside asset-clips in _add_asset_clip method
def _calculate_total_duration(self) -> str:
"""Calculate total timeline duration from segments."""
if not self.intent.timeline.segments:
return "0/2500s"
max_end = 0
for segment in self.intent.timeline.segments:
offset_frames = self._rational_to_frames(segment.offset)
duration_frames = self._rational_to_frames(segment.duration)
end_frames = offset_frames + duration_frames
max_end = max(max_end, end_frames)
timescale = self.intent.project.format.get_timescale()
return f"{max_end}/{timescale}s"
def _rational_to_frames(self, rational: str) -> int:
"""Convert rational time to frame count."""
parts = rational.rstrip("s").split("/")
if len(parts) != 2:
return 0
return int(parts[0])
def _pretty_print(self, xml_str: str) -> str:
"""Format XML with proper indentation."""
try:
dom = minidom.parseString(xml_str)
pretty = dom.toprettyxml(indent=" ")
lines = pretty.split("\n")
lines = [line for line in lines if line.strip() and not line.startswith("<?xml")]
return "\n".join(lines)
except Exception:
return xml_str
def generate_fcpxml(intent: EditIntent, pretty_print: bool = True) -> tuple[str, list[str]]:
"""
Convenience function to generate FCPXML from an EditIntent.
Args:
intent: The EditIntent to convert
pretty_print: Whether to format the XML
Returns:
Tuple of (fcpxml_string, warnings_list)
"""
generator = FCPXMLGenerator(intent)
xml = generator.generate(pretty_print=pretty_print)
return xml, generator.get_warnings()
def validate_fcpxml_structure(xml_str: str) -> tuple[bool, Optional[str]]:
"""
Validate basic FCPXML structure.
Args:
xml_str: FCPXML string to validate
Returns:
Tuple of (is_valid, error_message)
"""
try:
lines = xml_str.split("\n")
xml_content = "\n".join(
line for line in lines
if not line.strip().startswith("<!DOCTYPE")
)
root = ET.fromstring(xml_content)
if root.tag != "fcpxml":
return False, f"Root element must be 'fcpxml', got '{root.tag}'"
resources = root.find("resources")
if resources is None:
return False, "Missing 'resources' element"
library = root.find("library")
if library is None:
return False, "Missing 'library' element"
event = library.find("event")
if event is None:
return False, "Missing 'event' element in library"
project = event.find("project")
if project is None:
return False, "Missing 'project' element in event"
sequence = project.find("sequence")
if sequence is None:
return False, "Missing 'sequence' element in project"
spine = sequence.find("spine")
if spine is None:
return False, "Missing 'spine' element in sequence"
return True, None
except ET.ParseError as e:
return False, f"XML parse error: {e}"
def align_to_frame_boundary(time_str: str, fps: float = 30.0) -> str:
"""
Align a rational time string to the nearest frame boundary.
Args:
time_str: Rational time string like "12345/3000s"
fps: Frames per second (default 30.0)
Returns:
Frame-aligned rational time string
"""
parts = time_str.rstrip("s").split("/")
if len(parts) != 2:
return time_str
numerator = int(parts[0])
timescale = int(parts[1])
# Calculate frame duration in timescale units
frame_duration = int(timescale / fps)
# Round to nearest frame boundary
aligned = round(numerator / frame_duration) * frame_duration
return f"{aligned}/{timescale}s"
def import_fcpxml_silent(fcpxml_path: str, timeout: int = 30) -> dict:
"""
Import FCPXML into Final Cut Pro with automatic dialog handling.
Args:
fcpxml_path: Path to the FCPXML file
timeout: Maximum seconds to wait for import
Returns:
Dict with success status and any messages
"""
import subprocess
import time
import os
from datetime import datetime
result = {
"success": False,
"fcpxml_path": fcpxml_path,
"messages": [],
}
if not os.path.exists(fcpxml_path):
result["messages"].append(f"File not found: {fcpxml_path}")
return result
# Open FCPXML in FCP
subprocess.run(['open', '-a', 'Final Cut Pro', fcpxml_path])
time.sleep(3)
# Auto-handle dialogs
auto_script = '''
tell application "System Events"
tell process "Final Cut Pro"
set frontmost to true
delay 0.5
repeat 10 times
set windowNames to name of every window
if (windowNames as string) contains "Import XML" then
keystroke tab
delay 0.2
keystroke return
delay 2
else if (windowNames as string) contains "Open Library" then
key code 53
delay 0.5
else if (count of windowNames) > 1 then
keystroke return
delay 1
else
exit repeat
end if
end repeat
return "done"
end tell
end tell
'''
try:
subprocess.run(['osascript', '-e', auto_script], capture_output=True, timeout=timeout)
result["messages"].append("Dialog handling completed")
except subprocess.TimeoutExpired:
result["messages"].append("Dialog handling timed out")
# Check if library was modified
lib_path = os.path.expanduser("~/Movies/Untitled.fcpbundle/CurrentVersion.flexolibrary")
if os.path.exists(lib_path):
mtime = datetime.fromtimestamp(os.path.getmtime(lib_path))
age = (datetime.now() - mtime).total_seconds()
if age < 60:
result["success"] = True
result["messages"].append(f"Library modified {age:.0f}s ago")
return result
+453
View File
@@ -0,0 +1,453 @@
"""
FCPXML Injector for Final Cut Pro Integration
Manages watch folder delivery and daemon mode for automatic FCPXML injection.
"""
from __future__ import annotations
import json
import os
import signal
import sys
import tempfile
import time
import uuid
from dataclasses import asdict, dataclass
from datetime import datetime
from enum import Enum
from pathlib import Path
from typing import Callable, Optional
from .commandpost import CommandPostBridge
class JobStatus(str, Enum):
"""Job processing status."""
PENDING = "pending"
PROCESSING = "processing"
INJECTED = "injected"
FAILED = "failed"
@dataclass
class InjectionJob:
"""Represents an FCPXML injection job."""
job_id: str
created_at: str
instruction: str
intent_path: Optional[str] = None
fcpxml_path: Optional[str] = None
status: JobStatus = JobStatus.PENDING
error: Optional[str] = None
completed_at: Optional[str] = None
def to_dict(self) -> dict:
"""Convert to dictionary."""
d = asdict(self)
d["status"] = self.status.value
return d
@classmethod
def from_dict(cls, d: dict) -> "InjectionJob":
"""Create from dictionary."""
d = d.copy()
d["status"] = JobStatus(d["status"])
return cls(**d)
@dataclass
class DaemonState:
"""State of the daemon process."""
pid: int
started_at: str
watch_folder: str
jobs_processed: int = 0
last_job_at: Optional[str] = None
class Injector:
"""
Manages FCPXML injection into Final Cut Pro via CommandPost.
Supports both one-shot injection and daemon mode with watch folder.
"""
DEFAULT_BASE_DIR = os.path.expanduser("~/.praison/fcp")
def __init__(
self,
base_dir: Optional[str] = None,
watch_folder: Optional[str] = None,
commandpost: Optional[CommandPostBridge] = None,
):
"""
Initialize the injector.
Args:
base_dir: Base directory for jobs and outputs
watch_folder: Watch folder for CommandPost
commandpost: CommandPost bridge instance
"""
self.base_dir = Path(base_dir or self.DEFAULT_BASE_DIR)
self.jobs_dir = self.base_dir / "jobs"
self.out_dir = self.base_dir / "out"
self.watch_folder = Path(watch_folder or self.base_dir / "watch" / "fcpxml")
self.pid_file = self.base_dir / "daemon.pid"
self.state_file = self.base_dir / "daemon.state"
self.commandpost = commandpost or CommandPostBridge(
watch_folder=str(self.watch_folder)
)
self._ensure_directories()
def _ensure_directories(self) -> None:
"""Create necessary directories."""
self.jobs_dir.mkdir(parents=True, exist_ok=True)
self.out_dir.mkdir(parents=True, exist_ok=True)
self.watch_folder.mkdir(parents=True, exist_ok=True)
def inject_one_shot(
self,
fcpxml_content: str,
job_id: Optional[str] = None,
instruction: str = "",
intent_json: Optional[str] = None,
delete_after_import: bool = False,
) -> tuple[str, str, list[str]]:
"""
Perform one-shot FCPXML injection.
Args:
fcpxml_content: FCPXML content to inject
job_id: Optional job ID (generated if not provided)
instruction: Original instruction for logging
intent_json: Optional EditIntent JSON for logging
delete_after_import: Whether to delete file after import
Returns:
Tuple of (job_id, fcpxml_path, messages)
"""
job_id = job_id or str(uuid.uuid4())
messages = []
job = InjectionJob(
job_id=job_id,
created_at=datetime.utcnow().isoformat(),
instruction=instruction,
)
if intent_json:
intent_path = self.jobs_dir / f"{job_id}_intent.json"
intent_path.write_text(intent_json)
job.intent_path = str(intent_path)
messages.append(f"Intent saved: {intent_path}")
fcpxml_path = self._write_fcpxml_atomic(fcpxml_content, job_id)
job.fcpxml_path = str(fcpxml_path)
messages.append(f"FCPXML written: {fcpxml_path}")
watch_path = self._deliver_to_watch_folder(fcpxml_path, job_id)
messages.append(f"Delivered to watch folder: {watch_path}")
job.status = JobStatus.INJECTED
job.completed_at = datetime.utcnow().isoformat()
job_path = self.jobs_dir / f"{job_id}.json"
job_path.write_text(json.dumps(job.to_dict(), indent=2))
messages.append(f"Job logged: {job_path}")
if delete_after_import:
try:
os.remove(fcpxml_path)
messages.append(f"Cleaned up: {fcpxml_path}")
except OSError:
pass
return job_id, str(watch_path), messages
def _write_fcpxml_atomic(self, content: str, job_id: str) -> Path:
"""
Write FCPXML content atomically (write to temp, then rename).
Args:
content: FCPXML content
job_id: Job ID for filename
Returns:
Path to the written file
"""
final_path = self.out_dir / f"{job_id}.fcpxml"
fd, tmp_path = tempfile.mkstemp(
suffix=".fcpxml.tmp",
dir=str(self.out_dir),
)
try:
with os.fdopen(fd, "w", encoding="utf-8") as f:
f.write(content)
os.rename(tmp_path, final_path)
except Exception:
if os.path.exists(tmp_path):
os.remove(tmp_path)
raise
return final_path
def _deliver_to_watch_folder(self, source_path: Path, job_id: str) -> Path:
"""
Deliver FCPXML to the watch folder with atomic rename.
Args:
source_path: Source FCPXML file
job_id: Job ID for filename
Returns:
Path in watch folder
"""
watch_path = self.watch_folder / f"{job_id}.fcpxml"
fd, tmp_path = tempfile.mkstemp(
suffix=".fcpxml.tmp",
dir=str(self.watch_folder),
)
try:
with os.fdopen(fd, "w", encoding="utf-8") as f:
f.write(source_path.read_text())
os.rename(tmp_path, watch_path)
except Exception:
if os.path.exists(tmp_path):
os.remove(tmp_path)
raise
return watch_path
def get_job(self, job_id: str) -> Optional[InjectionJob]:
"""Get a job by ID."""
job_path = self.jobs_dir / f"{job_id}.json"
if not job_path.exists():
return None
try:
return InjectionJob.from_dict(json.loads(job_path.read_text()))
except (json.JSONDecodeError, KeyError):
return None
def list_jobs(self, limit: int = 10) -> list[InjectionJob]:
"""List recent jobs."""
jobs = []
for job_file in sorted(
self.jobs_dir.glob("*.json"),
key=lambda p: p.stat().st_mtime,
reverse=True,
)[:limit]:
if "_intent" not in job_file.name:
try:
jobs.append(InjectionJob.from_dict(json.loads(job_file.read_text())))
except (json.JSONDecodeError, KeyError):
pass
return jobs
def is_daemon_running(self) -> bool:
"""Check if daemon is running."""
if not self.pid_file.exists():
return False
try:
pid = int(self.pid_file.read_text().strip())
os.kill(pid, 0)
return True
except (ValueError, ProcessLookupError, PermissionError):
return False
def get_daemon_state(self) -> Optional[DaemonState]:
"""Get daemon state if running."""
if not self.is_daemon_running():
return None
if not self.state_file.exists():
return None
try:
data = json.loads(self.state_file.read_text())
return DaemonState(**data)
except (json.JSONDecodeError, TypeError):
return None
def start_daemon(
self,
detach: bool = False,
poll_interval: float = 1.0,
on_job: Optional[Callable[[InjectionJob], None]] = None,
) -> bool:
"""
Start the daemon process.
Args:
detach: Whether to detach from terminal
poll_interval: Seconds between poll cycles
on_job: Callback when job is processed
Returns:
True if started successfully
"""
if self.is_daemon_running():
return False
if detach:
pid = os.fork()
if pid > 0:
return True
os.setsid()
pid = os.fork()
if pid > 0:
sys.exit(0)
self.pid_file.write_text(str(os.getpid()))
state = DaemonState(
pid=os.getpid(),
started_at=datetime.utcnow().isoformat(),
watch_folder=str(self.watch_folder),
)
self.state_file.write_text(json.dumps(asdict(state)))
self._running = True
def handle_signal(signum, frame):
self._running = False
signal.signal(signal.SIGTERM, handle_signal)
signal.signal(signal.SIGINT, handle_signal)
try:
self._daemon_loop(poll_interval, on_job, state)
finally:
self._cleanup_daemon()
return True
def _daemon_loop(
self,
poll_interval: float,
on_job: Optional[Callable[[InjectionJob], None]],
state: DaemonState,
) -> None:
"""Main daemon loop."""
pending_dir = self.base_dir / "pending"
pending_dir.mkdir(exist_ok=True)
while self._running:
for pending_file in pending_dir.glob("*.json"):
try:
job_data = json.loads(pending_file.read_text())
job = InjectionJob.from_dict(job_data)
if job.fcpxml_path and os.path.exists(job.fcpxml_path):
self._deliver_to_watch_folder(
Path(job.fcpxml_path),
job.job_id,
)
job.status = JobStatus.INJECTED
job.completed_at = datetime.utcnow().isoformat()
else:
job.status = JobStatus.FAILED
job.error = "FCPXML file not found"
job_path = self.jobs_dir / f"{job.job_id}.json"
job_path.write_text(json.dumps(job.to_dict(), indent=2))
pending_file.unlink()
state.jobs_processed += 1
state.last_job_at = datetime.utcnow().isoformat()
self.state_file.write_text(json.dumps(asdict(state)))
if on_job:
on_job(job)
except Exception as e:
try:
error_path = pending_dir / f"{pending_file.stem}_error.txt"
error_path.write_text(str(e))
pending_file.unlink()
except Exception:
pass
time.sleep(poll_interval)
def _cleanup_daemon(self) -> None:
"""Clean up daemon files."""
try:
self.pid_file.unlink(missing_ok=True)
except Exception:
pass
def stop_daemon(self) -> bool:
"""
Stop the daemon process.
Returns:
True if stopped successfully
"""
if not self.pid_file.exists():
return False
try:
pid = int(self.pid_file.read_text().strip())
os.kill(pid, signal.SIGTERM)
for _ in range(50):
time.sleep(0.1)
try:
os.kill(pid, 0)
except ProcessLookupError:
break
self.pid_file.unlink(missing_ok=True)
return True
except (ValueError, ProcessLookupError, PermissionError):
self.pid_file.unlink(missing_ok=True)
return False
def submit_job(
self,
fcpxml_content: str,
instruction: str = "",
intent_json: Optional[str] = None,
) -> str:
"""
Submit a job to the daemon queue.
Args:
fcpxml_content: FCPXML content
instruction: Original instruction
intent_json: Optional EditIntent JSON
Returns:
Job ID
"""
job_id = str(uuid.uuid4())
fcpxml_path = self._write_fcpxml_atomic(fcpxml_content, job_id)
job = InjectionJob(
job_id=job_id,
created_at=datetime.utcnow().isoformat(),
instruction=instruction,
fcpxml_path=str(fcpxml_path),
)
if intent_json:
intent_path = self.jobs_dir / f"{job_id}_intent.json"
intent_path.write_text(intent_json)
job.intent_path = str(intent_path)
pending_dir = self.base_dir / "pending"
pending_dir.mkdir(exist_ok=True)
pending_path = pending_dir / f"{job_id}.json"
pending_path.write_text(json.dumps(job.to_dict(), indent=2))
return job_id
+337
View File
@@ -0,0 +1,337 @@
"""
EditIntent Pydantic Schema for Final Cut Pro Integration
This module defines the structured data models for representing video editing
intents that can be converted to FCPXML for Final Cut Pro.
"""
from __future__ import annotations
import hashlib
import os
import re
from enum import Enum
from typing import Any, Optional
from urllib.parse import quote
from pydantic import BaseModel, Field, field_validator, model_validator
class AudioRole(str, Enum):
"""Audio roles supported in FCPXML."""
DIALOGUE = "dialogue"
MUSIC = "music"
EFFECTS = "effects"
class AudioLayout(str, Enum):
"""Audio channel layouts."""
MONO = "mono"
STEREO = "stereo"
SURROUND = "surround"
class ProjectFormat(BaseModel):
"""Video format specification for the project."""
width: int = Field(default=1920, ge=1, description="Frame width in pixels")
height: int = Field(default=1080, ge=1, description="Frame height in pixels")
fps: float = Field(default=25.0, gt=0, description="Frames per second")
frame_duration_rational: Optional[str] = Field(
default=None,
description="Rational frame duration (e.g., '100/2500s'). Auto-calculated if not provided."
)
@model_validator(mode="after")
def calculate_frame_duration(self) -> "ProjectFormat":
"""Auto-calculate rational frame duration from fps if not provided."""
if self.frame_duration_rational is None:
timescale = int(self.fps * 100)
self.frame_duration_rational = f"100/{timescale}s"
return self
def get_timescale(self) -> int:
"""Get the timescale for rational time calculations."""
return int(self.fps * 100)
class AudioSettings(BaseModel):
"""Audio settings for the project."""
layout: AudioLayout = Field(default=AudioLayout.STEREO)
rate: int = Field(default=48000, description="Audio sample rate in Hz")
channels: int = Field(default=2, ge=1, description="Number of audio channels")
class Asset(BaseModel):
"""Represents a media asset (video/audio file)."""
id: str = Field(description="Internal reference ID (e.g., 'r2', 'r3')")
name: str = Field(description="Display name for the asset")
path: str = Field(description="Absolute local file path")
uid: Optional[str] = Field(default=None, description="Stable hash UID for the asset")
has_video: bool = Field(default=True, description="Whether asset contains video")
has_audio: bool = Field(default=True, description="Whether asset contains audio")
format_ref: Optional[str] = Field(
default=None,
description="Reference to format definition (e.g., 'r1')"
)
audio_rate: int = Field(default=48000, description="Audio sample rate")
audio_channels: int = Field(default=2, description="Number of audio channels")
duration_rational: Optional[str] = Field(
default=None,
description="Asset duration as rational time (e.g., '12000/2500s')"
)
@field_validator("path")
@classmethod
def validate_path(cls, v: str) -> str:
"""Validate that path is absolute."""
if not os.path.isabs(v):
raise ValueError(f"Asset path must be absolute: {v}")
return v
@model_validator(mode="after")
def generate_uid(self) -> "Asset":
"""Generate stable UID from path if not provided."""
if self.uid is None:
self.uid = hashlib.sha256(self.path.encode()).hexdigest()[:32].upper()
return self
def get_src_url(self) -> str:
"""Get file:// URL for the asset path."""
return f"file://{quote(self.path, safe='/:')}"
class Segment(BaseModel):
"""Represents a timeline segment (clip placement)."""
asset_id: str = Field(description="Reference to asset ID")
offset: str = Field(
description="Timeline position as rational time (e.g., '0/2500s')"
)
start: str = Field(
default="0/2500s",
description="Source start point as rational time"
)
duration: str = Field(
description="Segment duration as rational time (e.g., '5000/2500s')"
)
lane: int = Field(default=0, description="Timeline lane (0 = primary storyline)")
role: Optional[AudioRole] = Field(default=None, description="Audio role assignment")
volume: Optional[float] = Field(
default=None,
ge=0.0,
le=2.0,
description="Volume adjustment (1.0 = unity gain)"
)
name: Optional[str] = Field(default=None, description="Optional clip name override")
@field_validator("offset", "start", "duration")
@classmethod
def validate_rational_time(cls, v: str) -> str:
"""Validate rational time format."""
if not re.match(r"^\d+/\d+s$", v):
raise ValueError(f"Invalid rational time format: {v}. Expected format: '1234/2500s'")
return v
class Marker(BaseModel):
"""Represents a timeline marker (chapter point)."""
name: str = Field(description="Marker name/label")
start: str = Field(description="Marker position as rational time")
duration: str = Field(default="0/2500s", description="Marker duration")
@field_validator("start", "duration")
@classmethod
def validate_rational_time(cls, v: str) -> str:
"""Validate rational time format."""
if not re.match(r"^\d+/\d+s$", v):
raise ValueError(f"Invalid rational time format: {v}. Expected format: '1234/2500s'")
return v
class Operations(BaseModel):
"""Optional processing operations (may not all be implemented)."""
remove_pauses_over_seconds: Optional[float] = Field(
default=None,
ge=0.0,
description="Remove pauses longer than this duration (NOT IMPLEMENTED)"
)
loudness_target_lufs: Optional[float] = Field(
default=None,
le=0.0,
description="Target loudness in LUFS (NOT IMPLEMENTED)"
)
zoom_punches: Optional[list[dict[str, Any]]] = Field(
default=None,
description="Zoom punch-in effects (NOT IMPLEMENTED)"
)
def get_unimplemented_warnings(self) -> list[str]:
"""Return warnings for operations that are requested but not implemented."""
warnings = []
if self.remove_pauses_over_seconds is not None:
warnings.append(
f"Operation 'remove_pauses_over_seconds={self.remove_pauses_over_seconds}' "
"is NOT IMPLEMENTED - pauses will not be removed"
)
if self.loudness_target_lufs is not None:
warnings.append(
f"Operation 'loudness_target_lufs={self.loudness_target_lufs}' "
"is NOT IMPLEMENTED - loudness will not be adjusted"
)
if self.zoom_punches:
warnings.append(
f"Operation 'zoom_punches' with {len(self.zoom_punches)} entries "
"is NOT IMPLEMENTED - zoom effects will not be applied"
)
return warnings
class Timeline(BaseModel):
"""Timeline structure containing segments and markers."""
segments: list[Segment] = Field(default_factory=list)
markers: list[Marker] = Field(default_factory=list)
class Project(BaseModel):
"""Project metadata."""
name: str = Field(description="Project name")
format: ProjectFormat = Field(default_factory=ProjectFormat)
audio: AudioSettings = Field(default_factory=AudioSettings)
class EditIntent(BaseModel):
"""
Complete edit intent specification.
This is the top-level schema that represents a complete video editing
instruction that can be converted to FCPXML.
"""
version: str = Field(default="1", description="Schema version")
project: Project = Field(description="Project metadata")
assets: list[Asset] = Field(default_factory=list, description="Media assets")
timeline: Timeline = Field(default_factory=Timeline, description="Timeline structure")
operations: Optional[Operations] = Field(
default=None,
description="Optional processing operations"
)
agent_id: Optional[str] = Field(default=None, description="Agent ID for attribution")
run_id: Optional[str] = Field(default=None, description="Run ID for attribution")
session_id: Optional[str] = Field(default=None, description="Session ID for attribution")
missing_inputs: Optional[list[str]] = Field(
default=None,
description="List of missing inputs that were requested but not provided"
)
needs_user_timestamps: bool = Field(
default=False,
description="True if user needs to provide specific timestamps"
)
@model_validator(mode="after")
def validate_asset_references(self) -> "EditIntent":
"""Validate that all segment asset_ids reference valid assets."""
asset_ids = {asset.id for asset in self.assets}
for segment in self.timeline.segments:
if segment.asset_id not in asset_ids:
raise ValueError(
f"Segment references unknown asset_id '{segment.asset_id}'. "
f"Valid asset IDs: {asset_ids}"
)
return self
def get_warnings(self) -> list[str]:
"""Get all warnings for this intent."""
warnings = []
if self.operations:
warnings.extend(self.operations.get_unimplemented_warnings())
if self.missing_inputs:
warnings.append(
f"Missing inputs requested: {', '.join(self.missing_inputs)}"
)
if self.needs_user_timestamps:
warnings.append(
"User timestamps needed - using default concatenation order"
)
return warnings
def get_asset_by_id(self, asset_id: str) -> Optional[Asset]:
"""Get an asset by its ID."""
for asset in self.assets:
if asset.id == asset_id:
return asset
return None
@classmethod
def get_json_schema(cls) -> dict:
"""Get the JSON schema for this model."""
return cls.model_json_schema()
def seconds_to_rational(seconds: float, fps: float = 25.0) -> str:
"""
Convert seconds to rational time string.
Args:
seconds: Time in seconds
fps: Frames per second for timescale calculation
Returns:
Rational time string (e.g., '2500/2500s' for 1 second at 25fps)
"""
timescale = int(fps * 100)
frames = int(round(seconds * timescale))
return f"{frames}/{timescale}s"
def rational_to_seconds(rational: str) -> float:
"""
Convert rational time string to seconds.
Args:
rational: Rational time string (e.g., '2500/2500s')
Returns:
Time in seconds
"""
match = re.match(r"^(\d+)/(\d+)s$", rational)
if not match:
raise ValueError(f"Invalid rational time format: {rational}")
numerator = int(match.group(1))
denominator = int(match.group(2))
return numerator / denominator
FORMAT_PRESETS = {
"1080p25": ProjectFormat(width=1920, height=1080, fps=25.0),
"1080p30": ProjectFormat(width=1920, height=1080, fps=30.0),
"1080p24": ProjectFormat(width=1920, height=1080, fps=24.0),
"1080p50": ProjectFormat(width=1920, height=1080, fps=50.0),
"1080p60": ProjectFormat(width=1920, height=1080, fps=60.0),
"4k25": ProjectFormat(width=3840, height=2160, fps=25.0),
"4k30": ProjectFormat(width=3840, height=2160, fps=30.0),
"4k24": ProjectFormat(width=3840, height=2160, fps=24.0),
"4k50": ProjectFormat(width=3840, height=2160, fps=50.0),
"4k60": ProjectFormat(width=3840, height=2160, fps=60.0),
"720p25": ProjectFormat(width=1280, height=720, fps=25.0),
"720p30": ProjectFormat(width=1280, height=720, fps=30.0),
}
def get_format_preset(name: str) -> ProjectFormat:
"""Get a format preset by name."""
if name not in FORMAT_PRESETS:
raise ValueError(
f"Unknown format preset: {name}. "
f"Available presets: {', '.join(FORMAT_PRESETS.keys())}"
)
return FORMAT_PRESETS[name].model_copy()
# Rebuild models to resolve forward references
ProjectFormat.model_rebuild()
AudioSettings.model_rebuild()
Asset.model_rebuild()
Segment.model_rebuild()
Marker.model_rebuild()
Operations.model_rebuild()
Timeline.model_rebuild()
Project.model_rebuild()
EditIntent.model_rebuild()
+401
View File
@@ -0,0 +1,401 @@
"""
LLM Prompting for Final Cut Pro Integration
Converts natural language instructions to EditIntent JSON using OpenAI SDK.
"""
from __future__ import annotations
import json
import os
from typing import Any, Optional
from pydantic import ValidationError
from .intent import (
Asset,
AudioRole,
AudioSettings,
EditIntent,
Project,
ProjectFormat,
Segment,
Timeline,
get_format_preset,
)
EDIT_INTENT_SCHEMA = EditIntent.model_json_schema()
SYSTEM_PROMPT = """You are an expert video editor assistant that converts natural language editing instructions into structured JSON for Final Cut Pro.
You MUST output ONLY valid JSON that conforms to the EditIntent schema. No prose, no explanations, just JSON.
## CRITICAL RULES:
1. ONLY use media files from the provided media_paths list. NEVER invent file paths.
2. If the user requests media not in the list, add it to "missing_inputs" array.
3. For time-based edits without specific timestamps, set "needs_user_timestamps": true and use default concatenation order.
4. Use rational time format for all time values: "numerator/denominators" (e.g., "2500/2500s" = 1 second at 25fps)
5. Default timescale is fps * 100 (e.g., 25fps → 2500)
## CAPABILITIES (v1):
- Create projects with primary storyline
- Place asset clips in sequence
- Set audio roles (dialogue, music, effects)
- Add chapter markers
- Basic volume adjustments
## NOT IMPLEMENTED (store in operations but will warn):
- remove_pauses_over_seconds: Silence/pause removal
- loudness_target_lufs: Audio normalization
- zoom_punches: Ken Burns / zoom effects
## SCHEMA STRUCTURE:
{schema}
## EXAMPLE OUTPUT:
```json
{{
"version": "1",
"project": {{
"name": "My Edit",
"format": {{"width": 1920, "height": 1080, "fps": 25.0}},
"audio": {{"layout": "stereo", "rate": 48000, "channels": 2}}
}},
"assets": [
{{
"id": "r2",
"name": "interview",
"path": "/path/to/interview.mov",
"has_video": true,
"has_audio": true
}}
],
"timeline": {{
"segments": [
{{
"asset_id": "r2",
"offset": "0/2500s",
"start": "0/2500s",
"duration": "5000/2500s",
"lane": 0,
"role": "dialogue"
}}
],
"markers": []
}}
}}
```
"""
USER_PROMPT_TEMPLATE = """## INSTRUCTION:
{instruction}
## AVAILABLE MEDIA FILES:
{media_list}
## PROJECT SETTINGS:
- Project name: {project_name}
- Format: {format_info}
Generate the EditIntent JSON now. Output ONLY the JSON, no other text."""
def _get_openai_client():
"""Lazy import and create OpenAI client."""
try:
from openai import OpenAI
except ImportError:
raise ImportError(
"OpenAI SDK not installed. Install with: pip install openai"
)
api_key = os.environ.get("OPENAI_API_KEY")
if not api_key:
raise ValueError(
"OPENAI_API_KEY environment variable not set. "
"Set it with: export OPENAI_API_KEY=your-key"
)
return OpenAI(api_key=api_key)
def generate_edit_intent(
instruction: str,
media_paths: list[str],
project_name: str = "Untitled Project",
format_preset: Optional[str] = None,
width: Optional[int] = None,
height: Optional[int] = None,
fps: Optional[float] = None,
model: str = "gpt-4o",
max_retries: int = 3,
agent_id: Optional[str] = None,
run_id: Optional[str] = None,
session_id: Optional[str] = None,
) -> tuple[EditIntent, list[str]]:
"""
Generate EditIntent from natural language instruction using OpenAI.
Args:
instruction: Natural language editing instruction
media_paths: List of available media file paths
project_name: Name for the project
format_preset: Format preset name (e.g., "1080p25")
width: Custom width (overrides preset)
height: Custom height (overrides preset)
fps: Custom fps (overrides preset)
model: OpenAI model to use
max_retries: Maximum retry attempts for validation
agent_id: Agent ID for attribution
run_id: Run ID for attribution
session_id: Session ID for attribution
Returns:
Tuple of (EditIntent, warnings)
"""
client = _get_openai_client()
warnings = []
for path in media_paths:
if not os.path.isabs(path):
raise ValueError(f"Media path must be absolute: {path}")
if not os.path.exists(path):
warnings.append(f"Media file not found (will be referenced anyway): {path}")
if format_preset:
project_format = get_format_preset(format_preset)
else:
project_format = ProjectFormat(
width=width or 1920,
height=height or 1080,
fps=fps or 25.0,
)
if width:
project_format.width = width
if height:
project_format.height = height
if fps:
project_format.fps = fps
media_list = "\n".join(f"- {path}" for path in media_paths) or "No media files provided"
format_info = f"{project_format.width}x{project_format.height} @ {project_format.fps}fps"
system_prompt = SYSTEM_PROMPT.format(schema=json.dumps(EDIT_INTENT_SCHEMA, indent=2))
user_prompt = USER_PROMPT_TEMPLATE.format(
instruction=instruction,
media_list=media_list,
project_name=project_name,
format_info=format_info,
)
last_error = None
for attempt in range(max_retries):
try:
if attempt > 0:
user_prompt = f"{user_prompt}\n\n## PREVIOUS ERROR (attempt {attempt}):\n{last_error}\n\nPlease fix the JSON and try again."
response = client.chat.completions.create(
model=model,
messages=[
{"role": "system", "content": system_prompt},
{"role": "user", "content": user_prompt},
],
response_format={"type": "json_object"},
temperature=0.1,
)
content = response.choices[0].message.content
if not content:
last_error = "Empty response from LLM"
continue
try:
data = json.loads(content)
except json.JSONDecodeError as e:
last_error = f"Invalid JSON: {e}"
continue
data = _normalize_intent_data(data, media_paths, project_name, project_format)
intent = EditIntent.model_validate(data)
intent.agent_id = agent_id
intent.run_id = run_id
intent.session_id = session_id
warnings.extend(intent.get_warnings())
return intent, warnings
except ValidationError as e:
last_error = f"Schema validation error: {e}"
continue
except Exception as e:
last_error = f"Error: {e}"
continue
raise ValueError(
f"Failed to generate valid EditIntent after {max_retries} attempts. "
f"Last error: {last_error}"
)
def _normalize_intent_data(
data: dict[str, Any],
media_paths: list[str],
project_name: str,
project_format: ProjectFormat,
) -> dict[str, Any]:
"""
Normalize and fill in missing fields in the intent data.
Args:
data: Raw data from LLM
media_paths: Available media paths
project_name: Project name
project_format: Project format settings
Returns:
Normalized data dictionary
"""
if "version" not in data:
data["version"] = "1"
if "project" not in data:
data["project"] = {}
if "name" not in data["project"]:
data["project"]["name"] = project_name
if "format" not in data["project"]:
data["project"]["format"] = project_format.model_dump()
else:
fmt = data["project"]["format"]
if "width" not in fmt:
fmt["width"] = project_format.width
if "height" not in fmt:
fmt["height"] = project_format.height
if "fps" not in fmt:
fmt["fps"] = project_format.fps
if "audio" not in data["project"]:
data["project"]["audio"] = {"layout": "stereo", "rate": 48000, "channels": 2}
if "assets" not in data or not data["assets"]:
data["assets"] = []
for i, path in enumerate(media_paths):
ext = os.path.splitext(path)[1].lower()
has_video = ext in (".mov", ".mp4", ".m4v", ".avi", ".mkv", ".webm")
has_audio = ext in (".wav", ".mp3", ".aac", ".m4a", ".aiff") or has_video
data["assets"].append({
"id": f"r{i + 2}",
"name": os.path.basename(path),
"path": path,
"has_video": has_video,
"has_audio": has_audio,
})
for asset in data["assets"]:
if "id" not in asset:
asset["id"] = f"r{data['assets'].index(asset) + 2}"
if "name" not in asset:
asset["name"] = os.path.basename(asset.get("path", "unknown"))
if "timeline" not in data:
data["timeline"] = {"segments": [], "markers": []}
if "segments" not in data["timeline"]:
data["timeline"]["segments"] = []
if "markers" not in data["timeline"]:
data["timeline"]["markers"] = []
if not data["timeline"]["segments"] and data["assets"]:
fps = data["project"]["format"].get("fps", 25.0)
timescale = int(fps * 100)
offset = 0
for asset in data["assets"]:
duration_seconds = 10.0
duration_frames = int(duration_seconds * timescale)
data["timeline"]["segments"].append({
"asset_id": asset["id"],
"offset": f"{offset}/{timescale}s",
"start": f"0/{timescale}s",
"duration": f"{duration_frames}/{timescale}s",
"lane": 0,
})
offset += duration_frames
return data
def create_simple_intent(
media_paths: list[str],
project_name: str = "Simple Project",
format_preset: str = "1080p25",
) -> EditIntent:
"""
Create a simple EditIntent that concatenates all media files.
This is useful for testing or when LLM is not available.
Args:
media_paths: List of media file paths
project_name: Project name
format_preset: Format preset name
Returns:
EditIntent with all media concatenated
"""
project_format = get_format_preset(format_preset)
timescale = project_format.get_timescale()
assets = []
segments = []
offset = 0
for i, path in enumerate(media_paths):
if not os.path.isabs(path):
raise ValueError(f"Media path must be absolute: {path}")
ext = os.path.splitext(path)[1].lower()
has_video = ext in (".mov", ".mp4", ".m4v", ".avi", ".mkv", ".webm")
has_audio = ext in (".wav", ".mp3", ".aac", ".m4a", ".aiff") or has_video
asset_id = f"r{i + 2}"
assets.append(Asset(
id=asset_id,
name=os.path.basename(path),
path=path,
has_video=has_video,
has_audio=has_audio,
format_ref="r1" if has_video else None,
))
duration_frames = 10 * timescale
segments.append(Segment(
asset_id=asset_id,
offset=f"{offset}/{timescale}s",
start=f"0/{timescale}s",
duration=f"{duration_frames}/{timescale}s",
lane=0,
role=AudioRole.DIALOGUE if has_audio else None,
))
offset += duration_frames
return EditIntent(
version="1",
project=Project(
name=project_name,
format=project_format,
audio=AudioSettings(),
),
assets=assets,
timeline=Timeline(segments=segments, markers=[]),
)
+23
View File
@@ -0,0 +1,23 @@
"""n8n Workflow Automation Tools for PraisonAI Agents.
Bidirectional integration between PraisonAI agents and n8n workflow automation platform.
Usage:
from praisonai_tools.n8n import n8n_workflow
# Agent uses n8n workflow
result = n8n_workflow(
workflow_id="my-workflow-id",
input_data={"message": "Hello from PraisonAI"},
n8n_url="http://localhost:5678",
api_key="your-n8n-api-key"
)
"""
from .n8n_workflow import n8n_workflow, n8n_list_workflows, N8nWorkflowTool
__all__ = [
"n8n_workflow",
"n8n_list_workflows",
"N8nWorkflowTool",
]
+278
View File
@@ -0,0 +1,278 @@
"""n8n Workflow Execution Tool for PraisonAI Agents.
Enables PraisonAI agents to execute n8n workflows, providing access to
n8n's 400+ integrations (Slack, Gmail, Notion, databases, APIs, etc.).
Usage:
from praisonai_tools.n8n import n8n_workflow
# Execute n8n workflow
result = n8n_workflow(
workflow_id="workflow-123",
input_data={"message": "Hello from PraisonAI"},
n8n_url="http://localhost:5678",
api_key="your-api-key"
)
Environment Variables:
N8N_URL: n8n instance URL (default: http://localhost:5678)
N8N_API_KEY: n8n API key for authentication
"""
import os
import logging
from typing import Any, Dict, Optional, Union
from praisonai_tools.tools.base import BaseTool
from praisonai_tools.tools.decorator import tool
logger = logging.getLogger(__name__)
class N8nWorkflowTool(BaseTool):
"""Tool for executing n8n workflows from PraisonAI agents."""
name = "n8n_workflow"
description = "Execute n8n workflows to access 400+ integrations (Slack, Gmail, Notion, databases, APIs, etc.)"
def __init__(
self,
n8n_url: Optional[str] = None,
api_key: Optional[str] = None,
timeout: float = 60.0,
):
self.n8n_url = (n8n_url or os.getenv("N8N_URL", "http://localhost:5678")).rstrip('/')
self.api_key = api_key or os.getenv("N8N_API_KEY")
self.timeout = timeout
super().__init__()
def run(
self,
workflow_id: str,
input_data: Optional[Dict[str, Any]] = None,
wait_for_completion: bool = True,
**kwargs
) -> Dict[str, Any]:
"""Execute an n8n workflow via webhook trigger.
Args:
workflow_id: The n8n workflow ID to execute
input_data: Input data to pass to the workflow
wait_for_completion: Whether to wait for workflow completion (ignored for webhook execution)
Returns:
Workflow execution result
"""
if not workflow_id:
return {"error": "workflow_id is required"}
try:
# Lazy import httpx to avoid import-time overhead
import httpx
except ImportError:
return {"error": "httpx not installed. Install with: pip install 'praisonai-tools[n8n]'"}
# Fetch workflow to find webhook trigger
webhook_url = self._get_webhook_url(workflow_id)
if isinstance(webhook_url, dict) and "error" in webhook_url:
return webhook_url # Return error from _get_webhook_url
# Execute workflow via webhook (no API key needed for webhooks)
try:
with httpx.Client(timeout=self.timeout) as client:
response = client.post(
webhook_url,
json=input_data or {},
headers={"Content-Type": "application/json"},
)
response.raise_for_status()
# Try to parse JSON response, fallback to text if not JSON
try:
return response.json()
except ValueError:
return {"result": response.text, "status_code": response.status_code}
except httpx.TimeoutException:
logger.error(f"n8n workflow {workflow_id} webhook timed out after {self.timeout}s")
return {"error": f"Workflow webhook execution timed out after {self.timeout} seconds"}
except httpx.HTTPStatusError as e:
logger.error(f"n8n webhook error: {e.response.status_code} - {e.response.text}")
return {"error": f"HTTP {e.response.status_code}: {e.response.text}"}
except Exception as e:
logger.error(f"n8n workflow webhook execution error: {e}")
return {"error": str(e)}
def _get_webhook_url(self, workflow_id: str) -> Union[str, Dict[str, Any]]:
"""Get webhook URL for a workflow by finding its webhook trigger node.
Args:
workflow_id: The n8n workflow ID
Returns:
Webhook URL string or error dict
"""
try:
import httpx
except ImportError:
return {"error": "httpx not installed"}
# Prepare headers for API calls
headers = {"Content-Type": "application/json"}
if self.api_key:
headers["X-N8N-API-KEY"] = self.api_key
try:
with httpx.Client(timeout=30.0) as client:
# Fetch workflow definition
response = client.get(
f"{self.n8n_url}/api/v1/workflows/{workflow_id}",
headers=headers,
)
response.raise_for_status()
workflow = response.json()
# Find webhook trigger node
nodes = workflow.get("nodes", [])
webhook_node = None
for node in nodes:
if node.get("type") == "n8n-nodes-base.webhook":
webhook_node = node
break
if not webhook_node:
return {"error": "Workflow has no Webhook trigger node. Add a Webhook trigger to enable external execution."}
# Extract webhook path
parameters = webhook_node.get("parameters", {})
path = parameters.get("path", "")
if not path:
return {"error": "Webhook trigger node has no path configured"}
# Ensure workflow is active
if not workflow.get("active"):
activate_response = client.patch(
f"{self.n8n_url}/api/v1/workflows/{workflow_id}",
json={"active": True},
headers=headers,
)
activate_response.raise_for_status()
logger.info(f"Activated workflow {workflow_id}")
# Construct webhook URL
# Use /webhook-test/ for test mode, /webhook/ for production
webhook_url = f"{self.n8n_url}/webhook/{path}"
return webhook_url
except httpx.HTTPStatusError as e:
logger.error(f"n8n API error fetching workflow {workflow_id}: {e.response.status_code} - {e.response.text}")
return {"error": f"HTTP {e.response.status_code}: {e.response.text}"}
except Exception as e:
logger.error(f"Error fetching workflow {workflow_id}: {e}")
return {"error": str(e)}
def list_workflows(self) -> Dict[str, Any]:
"""List available n8n workflows."""
try:
import httpx
except ImportError:
return {"error": "httpx not installed. Install with: pip install 'praisonai-tools[n8n]'"}
headers = {"Content-Type": "application/json"}
if self.api_key:
headers["X-N8N-API-KEY"] = self.api_key
try:
with httpx.Client(timeout=30.0) as client:
response = client.get(
f"{self.n8n_url}/api/v1/workflows",
headers=headers,
)
response.raise_for_status()
return response.json()
except httpx.HTTPStatusError as e:
logger.error(f"n8n API error: {e.response.status_code} - {e.response.text}")
return {"error": f"HTTP {e.response.status_code}: {e.response.text}"}
except Exception as e:
logger.error(f"n8n list workflows error: {e}")
return {"error": str(e)}
@tool
def n8n_workflow(
workflow_id: str,
input_data: Optional[Dict[str, Any]] = None,
n8n_url: Optional[str] = None,
api_key: Optional[str] = None,
timeout: float = 60.0,
wait_for_completion: bool = True,
) -> Dict[str, Any]:
"""Execute an n8n workflow and return the result.
Provides PraisonAI agents access to n8n's 400+ integrations including:
- Communication: Slack, Discord, Telegram, Gmail, Outlook
- Productivity: Notion, Google Sheets, Airtable, Trello
- Databases: PostgreSQL, MongoDB, MySQL, Redis
- APIs: REST, GraphQL, webhooks
- And 400+ more integrations
Args:
workflow_id: The n8n workflow ID to execute
input_data: Input data to pass to the workflow (optional)
n8n_url: n8n instance URL (defaults to N8N_URL env var or http://localhost:5678)
api_key: n8n API key (defaults to N8N_API_KEY env var)
timeout: Request timeout in seconds (default: 60)
wait_for_completion: Wait for workflow to complete (default: True)
Returns:
Dict containing workflow execution result or error information
Example:
# Send Slack message via n8n workflow
result = n8n_workflow(
workflow_id="slack-notify",
input_data={
"channel": "#general",
"message": "Hello from PraisonAI!"
}
)
# Create Notion page via n8n workflow
result = n8n_workflow(
workflow_id="notion-create-page",
input_data={
"title": "Meeting Notes",
"content": "Discussion about project timeline"
}
)
"""
tool = N8nWorkflowTool(
n8n_url=n8n_url,
api_key=api_key,
timeout=timeout,
)
return tool.run(
workflow_id=workflow_id,
input_data=input_data,
wait_for_completion=wait_for_completion,
)
@tool
def n8n_list_workflows(
n8n_url: Optional[str] = None,
api_key: Optional[str] = None,
) -> Dict[str, Any]:
"""List available n8n workflows.
Args:
n8n_url: n8n instance URL (defaults to N8N_URL env var or http://localhost:5678)
api_key: n8n API key (defaults to N8N_API_KEY env var)
Returns:
Dict containing list of available workflows or error information
"""
tool = N8nWorkflowTool(n8n_url=n8n_url, api_key=api_key)
return tool.list_workflows()
+88
View File
@@ -0,0 +1,88 @@
"""
PraisonAI Observability Suite
Unified observability integrations for PraisonAI agents.
All imports are lazy-loaded for zero performance impact when not in use.
Usage:
from praisonai_tools.observability import obs
# Auto-detect and configure
obs.init()
# Or use specific provider
obs.init(provider="langfuse")
"""
__all__ = [
# Core
"obs",
"ObservabilityManager",
"ObservabilityConfig",
# Providers
"AgentOpsProvider",
"LangfuseProvider",
"LangSmithProvider",
"TraceloopProvider",
"ArizePhoenixProvider",
"OpenLITProvider",
"LangtraceProvider",
"LangWatchProvider",
"DatadogProvider",
"MLflowProvider",
"OpikProvider",
"PortkeyProvider",
"BraintrustProvider",
"MaximProvider",
"WeaveProvider",
"NeatlogsProvider",
"LangDBProvider",
"AtlaProvider",
"PatronusProvider",
"TrueFoundryProvider",
]
# Lazy loading implementation
def __getattr__(name: str):
"""Lazy load observability components."""
if name == "obs" or name == "ObservabilityManager":
from praisonai_tools.observability.manager import ObservabilityManager
if name == "obs":
return ObservabilityManager()
return ObservabilityManager
if name == "ObservabilityConfig":
from praisonai_tools.observability.config import ObservabilityConfig
return ObservabilityConfig
# Provider lazy loading
provider_map = {
"AgentOpsProvider": ("agentops_provider", "AgentOpsProvider"),
"LangfuseProvider": ("langfuse_provider", "LangfuseProvider"),
"LangSmithProvider": ("langsmith_provider", "LangSmithProvider"),
"TraceloopProvider": ("traceloop_provider", "TraceloopProvider"),
"ArizePhoenixProvider": ("arize_phoenix_provider", "ArizePhoenixProvider"),
"OpenLITProvider": ("openlit_provider", "OpenLITProvider"),
"LangtraceProvider": ("langtrace_provider", "LangtraceProvider"),
"LangWatchProvider": ("langwatch_provider", "LangWatchProvider"),
"DatadogProvider": ("datadog_provider", "DatadogProvider"),
"MLflowProvider": ("mlflow_provider", "MLflowProvider"),
"OpikProvider": ("opik_provider", "OpikProvider"),
"PortkeyProvider": ("portkey_provider", "PortkeyProvider"),
"BraintrustProvider": ("braintrust_provider", "BraintrustProvider"),
"MaximProvider": ("maxim_provider", "MaximProvider"),
"WeaveProvider": ("weave_provider", "WeaveProvider"),
"NeatlogsProvider": ("neatlogs_provider", "NeatlogsProvider"),
"LangDBProvider": ("langdb_provider", "LangDBProvider"),
"AtlaProvider": ("atla_provider", "AtlaProvider"),
"PatronusProvider": ("patronus_provider", "PatronusProvider"),
"TrueFoundryProvider": ("truefoundry_provider", "TrueFoundryProvider"),
}
if name in provider_map:
module_name, class_name = provider_map[name]
import importlib
module = importlib.import_module(f"praisonai_tools.observability.providers.{module_name}")
return getattr(module, class_name)
raise AttributeError(f"module 'praisonai_tools.observability' has no attribute '{name}'")
+447
View File
@@ -0,0 +1,447 @@
"""
Base Provider Interface
Abstract base class for all observability providers.
"""
import contextvars
import uuid
from abc import ABC, abstractmethod
from contextlib import contextmanager
from dataclasses import dataclass, field
from datetime import datetime, timezone
from enum import Enum
from typing import Any, Callable, Dict, Generator, List, Optional
from praisonai_tools.observability.config import ObservabilityConfig
class SpanKind(Enum):
"""Types of spans in the observability system."""
AGENT = "agent"
LLM = "llm"
TOOL = "tool"
RETRIEVAL = "retrieval"
CHAIN = "chain"
WORKFLOW = "workflow"
CUSTOM = "custom"
class SpanStatus(Enum):
"""Status of a span."""
UNSET = "unset"
OK = "ok"
ERROR = "error"
@dataclass
class SpanContext:
"""Context for span propagation."""
trace_id: str
span_id: str
parent_span_id: Optional[str] = None
@classmethod
def new(cls, parent: Optional["SpanContext"] = None) -> "SpanContext":
"""Create a new span context."""
return cls(
trace_id=parent.trace_id if parent else str(uuid.uuid4()),
span_id=str(uuid.uuid4()),
parent_span_id=parent.span_id if parent else None,
)
@dataclass
class Span:
"""Represents a single span in a trace."""
name: str
kind: SpanKind
context: SpanContext
start_time: datetime = field(default_factory=lambda: datetime.now(timezone.utc))
end_time: Optional[datetime] = None
status: SpanStatus = SpanStatus.UNSET
attributes: Dict[str, Any] = field(default_factory=dict)
events: List[Dict[str, Any]] = field(default_factory=list)
# LLM-specific
model: Optional[str] = None
input_tokens: Optional[int] = None
output_tokens: Optional[int] = None
total_tokens: Optional[int] = None
# Tool-specific
tool_name: Optional[str] = None
tool_input: Optional[Any] = None
tool_output: Optional[Any] = None
# Error info
error_message: Optional[str] = None
error_type: Optional[str] = None
def end(self, status: SpanStatus = SpanStatus.OK) -> None:
"""End the span."""
self.end_time = datetime.now(timezone.utc)
self.status = status
def set_error(self, error: Exception) -> None:
"""Set error information."""
self.status = SpanStatus.ERROR
self.error_message = str(error)
self.error_type = type(error).__name__
def add_event(self, name: str, attributes: Optional[Dict[str, Any]] = None) -> None:
"""Add an event to the span."""
self.events.append({
"name": name,
"timestamp": datetime.utcnow().isoformat(),
"attributes": attributes or {},
})
@property
def duration_ms(self) -> Optional[float]:
"""Get span duration in milliseconds."""
if self.end_time and self.start_time:
return (self.end_time - self.start_time).total_seconds() * 1000
return None
def to_dict(self) -> Dict[str, Any]:
"""Convert span to dictionary."""
return {
"name": self.name,
"kind": self.kind.value,
"trace_id": self.context.trace_id,
"span_id": self.context.span_id,
"parent_span_id": self.context.parent_span_id,
"start_time": self.start_time.isoformat(),
"end_time": self.end_time.isoformat() if self.end_time else None,
"status": self.status.value,
"attributes": self.attributes,
"events": self.events,
"model": self.model,
"input_tokens": self.input_tokens,
"output_tokens": self.output_tokens,
"total_tokens": self.total_tokens,
"tool_name": self.tool_name,
"tool_input": self.tool_input,
"tool_output": self.tool_output,
"error_message": self.error_message,
"error_type": self.error_type,
"duration_ms": self.duration_ms,
}
@dataclass
class Trace:
"""Represents a complete trace."""
trace_id: str
name: str
spans: List[Span] = field(default_factory=list)
start_time: datetime = field(default_factory=lambda: datetime.now(timezone.utc))
end_time: Optional[datetime] = None
metadata: Dict[str, Any] = field(default_factory=dict)
# Context
session_id: Optional[str] = None
user_id: Optional[str] = None
agent_id: Optional[str] = None
def add_span(self, span: Span) -> None:
"""Add a span to the trace."""
self.spans.append(span)
def end(self) -> None:
"""End the trace."""
self.end_time = datetime.now(timezone.utc)
@property
def duration_ms(self) -> Optional[float]:
"""Get trace duration in milliseconds."""
if self.end_time and self.start_time:
return (self.end_time - self.start_time).total_seconds() * 1000
return None
def to_dict(self) -> Dict[str, Any]:
"""Convert trace to dictionary."""
return {
"trace_id": self.trace_id,
"name": self.name,
"spans": [s.to_dict() for s in self.spans],
"start_time": self.start_time.isoformat(),
"end_time": self.end_time.isoformat() if self.end_time else None,
"metadata": self.metadata,
"session_id": self.session_id,
"user_id": self.user_id,
"agent_id": self.agent_id,
"duration_ms": self.duration_ms,
}
class BaseObservabilityProvider(ABC):
"""Abstract base class for observability providers."""
name: str = "base"
# Per-context span stack and trace for async safety
_ctx_span_stack: contextvars.ContextVar = contextvars.ContextVar(
'obs_span_stack', default=None
)
_ctx_current_trace: contextvars.ContextVar = contextvars.ContextVar(
'obs_current_trace', default=None
)
def __init__(self, config: Optional[ObservabilityConfig] = None):
"""Initialize the provider."""
self.config = config or ObservabilityConfig()
self._initialized = False
@property
def _current_trace(self) -> Optional[Trace]:
return self._ctx_current_trace.get(None)
@_current_trace.setter
def _current_trace(self, value: Optional[Trace]):
self._ctx_current_trace.set(value)
@property
def _span_stack(self) -> List[Span]:
stack = self._ctx_span_stack.get(None)
if stack is None:
stack = []
self._ctx_span_stack.set(stack)
return stack
@_span_stack.setter
def _span_stack(self, value: List[Span]):
self._ctx_span_stack.set(value)
@abstractmethod
def init(self, **kwargs) -> bool:
"""Initialize the provider. Returns True if successful."""
pass
@abstractmethod
def shutdown(self) -> None:
"""Shutdown the provider and flush any pending data."""
pass
@abstractmethod
def is_available(self) -> bool:
"""Check if the provider SDK is available."""
pass
@abstractmethod
def check_connection(self) -> tuple[bool, str]:
"""Check connection to the provider. Returns (success, message)."""
pass
@abstractmethod
def export_span(self, span: Span) -> bool:
"""Export a span to the provider."""
pass
@abstractmethod
def export_trace(self, trace: Trace) -> bool:
"""Export a complete trace to the provider."""
pass
def start_trace(
self,
name: str,
session_id: Optional[str] = None,
user_id: Optional[str] = None,
agent_id: Optional[str] = None,
metadata: Optional[Dict[str, Any]] = None,
) -> Trace:
"""Start a new trace."""
trace = Trace(
trace_id=str(uuid.uuid4()),
name=name,
session_id=session_id or self.config.session_id,
user_id=user_id or self.config.user_id,
metadata=metadata or {},
)
trace.agent_id = agent_id
self._current_trace = trace
return trace
def end_trace(self, trace: Optional[Trace] = None) -> None:
"""End a trace and export it."""
trace = trace or self._current_trace
if trace:
trace.end()
self.export_trace(trace)
if trace == self._current_trace:
self._current_trace = None
def start_span(
self,
name: str,
kind: SpanKind = SpanKind.CUSTOM,
parent: Optional[Span] = None,
attributes: Optional[Dict[str, Any]] = None,
) -> Span:
"""Start a new span."""
parent_context = None
if parent:
parent_context = parent.context
elif self._span_stack:
parent_context = self._span_stack[-1].context
elif self._current_trace:
parent_context = SpanContext(
trace_id=self._current_trace.trace_id,
span_id="root",
)
span = Span(
name=name,
kind=kind,
context=SpanContext.new(parent_context),
attributes=attributes or {},
)
self._span_stack.append(span)
if self._current_trace:
self._current_trace.add_span(span)
return span
def end_span(
self,
span: Optional[Span] = None,
status: SpanStatus = SpanStatus.OK,
) -> None:
"""End a span and export it."""
span = span or (self._span_stack[-1] if self._span_stack else None)
if span:
span.end(status)
self.export_span(span)
if span in self._span_stack:
self._span_stack.remove(span)
@contextmanager
def trace(
self,
name: str,
session_id: Optional[str] = None,
user_id: Optional[str] = None,
agent_id: Optional[str] = None,
metadata: Optional[Dict[str, Any]] = None,
) -> Generator[Trace, None, None]:
"""Context manager for traces."""
trace = self.start_trace(name, session_id, user_id, agent_id, metadata)
try:
yield trace
except Exception as e:
trace.metadata["error"] = str(e)
raise
finally:
self.end_trace(trace)
@contextmanager
def span(
self,
name: str,
kind: SpanKind = SpanKind.CUSTOM,
attributes: Optional[Dict[str, Any]] = None,
) -> Generator[Span, None, None]:
"""Context manager for spans."""
span = self.start_span(name, kind, attributes=attributes)
try:
yield span
except Exception as e:
span.set_error(e)
raise
finally:
self.end_span(span)
def log_llm_call(
self,
model: str,
input_messages: Any,
output: Any,
input_tokens: Optional[int] = None,
output_tokens: Optional[int] = None,
duration_ms: Optional[float] = None,
error: Optional[Exception] = None,
metadata: Optional[Dict[str, Any]] = None,
) -> Span:
"""Log an LLM call."""
span = self.start_span(f"llm.{model}", SpanKind.LLM)
span.model = model
span.input_tokens = input_tokens
span.output_tokens = output_tokens
span.total_tokens = (input_tokens or 0) + (output_tokens or 0) if input_tokens or output_tokens else None
span.attributes["input"] = str(input_messages) if not self.config.redact_inputs else "[REDACTED]"
span.attributes["output"] = str(output) if not self.config.redact_outputs else "[REDACTED]"
span.attributes.update(metadata or {})
if error:
span.set_error(error)
self.end_span(span, SpanStatus.ERROR if error else SpanStatus.OK)
return span
def log_tool_call(
self,
tool_name: str,
tool_input: Any,
tool_output: Any,
duration_ms: Optional[float] = None,
error: Optional[Exception] = None,
metadata: Optional[Dict[str, Any]] = None,
) -> Span:
"""Log a tool call."""
span = self.start_span(f"tool.{tool_name}", SpanKind.TOOL)
span.tool_name = tool_name
span.tool_input = tool_input if not self.config.redact_inputs else "[REDACTED]"
span.tool_output = tool_output if not self.config.redact_outputs else "[REDACTED]"
span.attributes.update(metadata or {})
if error:
span.set_error(error)
self.end_span(span, SpanStatus.ERROR if error else SpanStatus.OK)
return span
def log_agent_step(
self,
agent_name: str,
step_name: str,
input_data: Any,
output_data: Any,
error: Optional[Exception] = None,
metadata: Optional[Dict[str, Any]] = None,
) -> Span:
"""Log an agent step."""
span = self.start_span(f"agent.{agent_name}.{step_name}", SpanKind.AGENT)
span.attributes["agent_name"] = agent_name
span.attributes["step_name"] = step_name
span.attributes["input"] = str(input_data) if not self.config.redact_inputs else "[REDACTED]"
span.attributes["output"] = str(output_data) if not self.config.redact_outputs else "[REDACTED]"
span.attributes.update(metadata or {})
if error:
span.set_error(error)
self.end_span(span, SpanStatus.ERROR if error else SpanStatus.OK)
return span
def decorator(self, name: Optional[str] = None, kind: SpanKind = SpanKind.CUSTOM):
"""Decorator to trace a function."""
def wrapper(func: Callable) -> Callable:
span_name = name or func.__name__
def wrapped(*args, **kwargs):
with self.span(span_name, kind):
return func(*args, **kwargs)
async def async_wrapped(*args, **kwargs):
with self.span(span_name, kind):
return await func(*args, **kwargs)
import asyncio
if asyncio.iscoroutinefunction(func):
return async_wrapped
return wrapped
return wrapper
+207
View File
@@ -0,0 +1,207 @@
"""
Observability Bridge
Bridges ContextTraceEmitter events from praisonaiagents core SDK
to the ObservabilityManager's provider (e.g., LangSmith, Langfuse).
This implements the ContextTraceSinkProtocol so the core SDK's
internal trace events (LLM calls, tool calls, agent lifecycle)
are forwarded to external observability providers.
Zero Performance Impact:
- Only active when obs.init() is called
- Lazy imports for praisonaiagents types
- No overhead when not used
"""
from typing import Any, TYPE_CHECKING
if TYPE_CHECKING:
from praisonai_tools.observability.manager import ObservabilityManager
class ObservabilitySink:
"""
Bridges ContextTraceEmitter events to ObservabilityManager spans.
Implements the ContextTraceSinkProtocol from praisonaiagents.trace.context_events.
When registered as the active sink, all agent/LLM/tool events are forwarded
to the configured observability provider.
"""
def __init__(self, manager: "ObservabilityManager"):
self._manager = manager
self._active_llm_span = None
self._active_tool_span = None
self._active_agent_span = None
def emit(self, event: Any) -> None:
"""
Receive a ContextEvent and forward to the observability provider.
Args:
event: A ContextEvent from praisonaiagents.trace.context_events
"""
try:
from praisonaiagents.trace.context_events import ContextEventType
except ImportError:
return
if not self._manager or not self._manager.enabled:
return
event_type = event.event_type
# AGENT_START/AGENT_END are handled by _wrap_agent_class in manager.py
# with full input/output data — skip bridge duplicates to avoid empty spans
if event_type == ContextEventType.LLM_REQUEST:
self._handle_llm_request(event)
elif event_type == ContextEventType.LLM_RESPONSE:
self._handle_llm_response(event)
elif event_type == ContextEventType.TOOL_CALL_START:
self._handle_tool_start(event)
elif event_type == ContextEventType.TOOL_CALL_END:
self._handle_tool_end(event)
def _handle_agent_start(self, event: Any) -> None:
"""Create an agent span."""
from praisonai_tools.observability.base import SpanKind
agent_name = getattr(event, 'agent_name', 'unknown')
data = getattr(event, 'data', {}) or {}
attrs = {}
if data.get('role'):
attrs['agent.role'] = str(data['role'])
if data.get('goal'):
attrs['agent.goal'] = str(data['goal'])
span = self._manager.start_span(
f"agent.{agent_name}",
kind=SpanKind.AGENT,
attributes=attrs,
)
self._active_agent_span = span
def _handle_agent_end(self, event: Any) -> None:
"""End the agent span."""
if self._active_agent_span:
self._manager.end_span(self._active_agent_span)
self._active_agent_span = None
def _handle_llm_request(self, event: Any) -> None:
"""Create an LLM span with input messages."""
from praisonai_tools.observability.base import SpanKind
data = getattr(event, 'data', {}) or {}
model = data.get('model', 'unknown')
messages = data.get('messages', [])
attrs = {}
attrs['gen_ai.request.model'] = str(model)
attrs['gen_ai.system'] = 'openai'
# Set input messages as GenAI prompt attributes
if messages:
for i, msg in enumerate(messages):
if isinstance(msg, dict):
role = msg.get('role', '')
content = msg.get('content', '')
attrs[f'gen_ai.prompt.{i}.role'] = str(role)
if content:
# Truncate very long content
content_str = str(content)
if len(content_str) > 10000:
content_str = content_str[:10000] + "..."
attrs[f'gen_ai.prompt.{i}.content'] = content_str
span = self._manager.start_span(
str(model),
kind=SpanKind.LLM,
attributes=attrs,
)
if span:
span.model = str(model)
self._active_llm_span = span
def _handle_llm_response(self, event: Any) -> None:
"""Populate LLM span with response data and end it."""
data = getattr(event, 'data', {}) or {}
prompt_tokens = getattr(event, 'prompt_tokens', 0) or 0
completion_tokens = getattr(event, 'completion_tokens', 0) or 0
cost_usd = getattr(event, 'cost_usd', 0.0) or 0.0
if self._active_llm_span:
self._active_llm_span.input_tokens = prompt_tokens
self._active_llm_span.output_tokens = completion_tokens
self._active_llm_span.total_tokens = prompt_tokens + completion_tokens
# Set GenAI usage attributes
self._active_llm_span.attributes['gen_ai.usage.prompt_tokens'] = prompt_tokens
self._active_llm_span.attributes['gen_ai.usage.completion_tokens'] = completion_tokens
self._active_llm_span.attributes['gen_ai.usage.total_tokens'] = prompt_tokens + completion_tokens
if cost_usd:
self._active_llm_span.attributes['gen_ai.usage.cost'] = cost_usd
finish_reason = data.get('finish_reason')
if finish_reason:
self._active_llm_span.attributes['gen_ai.response.finish_reason'] = str(finish_reason)
response_content = data.get('response_content')
if response_content:
content_str = str(response_content)
if len(content_str) > 10000:
content_str = content_str[:10000] + "..."
self._active_llm_span.attributes['gen_ai.completion.0.content'] = content_str
self._active_llm_span.attributes['gen_ai.completion.0.role'] = 'assistant'
self._manager.end_span(self._active_llm_span)
self._active_llm_span = None
def _handle_tool_start(self, event: Any) -> None:
"""Create a tool span."""
from praisonai_tools.observability.base import SpanKind
data = getattr(event, 'data', {}) or {}
tool_name = data.get('tool_name', 'unknown')
arguments = data.get('arguments', {})
attrs = {
'tool.name': str(tool_name),
}
if arguments:
attrs['tool.arguments'] = str(arguments)
span = self._manager.start_span(
str(tool_name),
kind=SpanKind.TOOL,
attributes=attrs,
)
if span:
span.tool_name = str(tool_name)
span.tool_input = arguments
self._active_tool_span = span
def _handle_tool_end(self, event: Any) -> None:
"""End the tool span with result."""
data = getattr(event, 'data', {}) or {}
if self._active_tool_span:
result = data.get('result')
error = data.get('error')
if result is not None:
result_str = str(result)
if len(result_str) > 10000:
result_str = result_str[:10000] + "..."
self._active_tool_span.tool_output = result_str
if error:
from praisonai_tools.observability.base import SpanStatus
self._active_tool_span.error_message = str(error)
self._manager.end_span(self._active_tool_span, SpanStatus.ERROR)
else:
self._manager.end_span(self._active_tool_span)
self._active_tool_span = None
+252
View File
@@ -0,0 +1,252 @@
"""
PraisonAI Observability CLI.
Lightweight CLI that lives in praisonai_tools, close to the observability code.
Can be run standalone: python -m praisonai_tools.observability.cli doctor
Or via the praisonai wrapper: praisonai obs doctor
"""
import typer
app = typer.Typer(help="Observability diagnostics and management")
def _get_obs():
"""Lazy-import obs singleton."""
try:
from praisonai_tools.observability import obs
return obs
except ImportError:
from rich.console import Console
Console(stderr=True).print(
"[red]✗ praisonai-tools observability not available.[/red]"
)
raise typer.Exit(1)
def _print_doctor_table(results: dict) -> None:
"""Pretty-print doctor results as a Rich table."""
from rich.console import Console
from rich.table import Table
console = Console()
console.print("\n[bold cyan]🔍 PraisonAI Observability Doctor[/bold cyan]\n")
table = Table(show_header=True, header_style="bold")
table.add_column("Check", style="dim", min_width=20)
table.add_column("Result")
# Enabled
enabled = results.get("enabled", False)
table.add_row(
"Enabled",
"[green]✓ Yes[/green]" if enabled else "[yellow]✗ No[/yellow]",
)
# Provider
provider = results.get("provider")
table.add_row(
"Active Provider",
f"[green]{provider}[/green]" if provider else "[dim]None[/dim]",
)
# Connection
conn_status = results.get("connection_status")
conn_msg = results.get("connection_message", "")
if conn_status is True:
table.add_row("Connection", f"[green]✓ {conn_msg}[/green]")
elif conn_status is False:
table.add_row("Connection", f"[red]✗ {conn_msg}[/red]")
else:
table.add_row("Connection", "[dim]N/A[/dim]")
# Available providers
available = results.get("available_providers", [])
table.add_row(
"Available Providers",
", ".join(available) if available else "[dim]None[/dim]",
)
# Registered providers
registered = results.get("registered_providers", [])
table.add_row(
"Registered Providers",
", ".join(registered) if registered else "[dim]None[/dim]",
)
console.print(table)
console.print()
@app.command("doctor")
def obs_doctor(
json_output: bool = typer.Option(False, "--json", help="Output as JSON"),
):
"""
Run observability health checks.
Checks provider status, connection, and available providers.
Examples:
python -m praisonai_tools.observability.cli doctor
praisonai obs doctor
praisonai obs doctor --json
"""
obs = _get_obs()
results = obs.doctor()
if json_output:
import json
typer.echo(json.dumps(results, indent=2, default=str))
raise typer.Exit(0)
_print_doctor_table(results)
raise typer.Exit(0)
@app.command("verify")
def obs_verify(
provider: str = typer.Option("langsmith", help="Provider to verify"),
project: str = typer.Option("default", help="Project name to check"),
limit: int = typer.Option(5, help="Number of recent runs to check"),
json_output: bool = typer.Option(False, "--json", help="Output as JSON"),
):
"""
Verify traces are recorded in the observability backend.
Uses the provider's SDK to fetch recent traces and check for
PraisonAI branding attributes.
Examples:
praisonai obs verify --provider langsmith --project "My First App"
python -m praisonai_tools.observability.cli verify --json
"""
from rich.console import Console
console = Console()
if provider == "langsmith":
_verify_langsmith(console, project, limit, json_output)
else:
console.print(f"[yellow]Verify not yet supported for provider: {provider}[/yellow]")
raise typer.Exit(1)
def _verify_langsmith(console, project: str, limit: int, json_output: bool):
"""Verify traces in LangSmith using the LangSmith SDK."""
import os
from rich.table import Table
try:
from langsmith import Client
except ImportError:
console.print("[red]✗ langsmith SDK not installed.[/red]")
console.print("[dim]Install with: pip install langsmith[/dim]")
raise typer.Exit(1)
api_key = os.getenv("LANGSMITH_API_KEY")
endpoint = os.getenv("LANGSMITH_ENDPOINT", "https://api.smith.langchain.com")
if not api_key:
console.print("[red]✗ LANGSMITH_API_KEY not set.[/red]")
raise typer.Exit(1)
ls = Client(api_url=endpoint, api_key=api_key)
try:
runs = list(ls.list_runs(
project_name=project,
execution_order=1, # root runs only
limit=limit,
))
except Exception as e:
console.print(f"[red]✗ Failed to fetch runs: {e}[/red]")
raise typer.Exit(1)
# Collect results
results = []
for run in runs:
extra = run.extra or {}
metadata = extra.get("metadata", {})
results.append({
"name": run.name,
"id": str(run.id),
"status": run.status,
"run_type": run.run_type,
"has_inputs": bool(run.inputs),
"has_outputs": bool(run.outputs),
"praisonai.version": metadata.get("praisonai.version"),
"praisonai.framework": metadata.get("praisonai.framework"),
"start_time": str(run.start_time) if run.start_time else None,
})
if json_output:
import json
typer.echo(json.dumps({
"project": project,
"endpoint": endpoint,
"runs_checked": len(results),
"runs": results,
}, indent=2, default=str))
raise typer.Exit(0)
# Pretty print
console.print("\n[bold cyan]🔍 PraisonAI Trace Verification[/bold cyan]")
console.print(f"[dim]Project: {project} | Endpoint: {endpoint}[/dim]\n")
if not results:
console.print("[yellow]No runs found in project.[/yellow]")
raise typer.Exit(0)
table = Table(show_header=True, header_style="bold")
table.add_column("Name", min_width=15)
table.add_column("Type")
table.add_column("Status")
table.add_column("I/O")
table.add_column("Version")
table.add_column("Framework")
branded_count = 0
for r in results:
version = r["praisonai.version"]
framework = r["praisonai.framework"]
has_branding = version is not None and framework is not None
if has_branding:
branded_count += 1
table.add_row(
r["name"],
r["run_type"] or "—",
f"[green]{r['status']}[/green]" if r["status"] == "success" else r["status"] or "—",
"[green]✓[/green]" if r["has_inputs"] and r["has_outputs"] else "[yellow]partial[/yellow]",
f"[green]{version}[/green]" if version else "[dim]—[/dim]",
f"[green]{framework}[/green]" if framework else "[dim]—[/dim]",
)
console.print(table)
console.print()
# Summary
total = len(results)
if branded_count == total:
console.print(f"[green]✓ All {total} runs have PraisonAI branding[/green]")
elif branded_count > 0:
console.print(f"[yellow]⚠ {branded_count}/{total} runs have PraisonAI branding[/yellow]")
else:
console.print("[red]✗ No runs have PraisonAI branding[/red]")
console.print()
raise typer.Exit(0)
@app.callback(invoke_without_command=True)
def obs_callback(ctx: typer.Context):
"""Observability diagnostics and management."""
if ctx.invoked_subcommand is None:
ctx.invoke(obs_doctor)
if __name__ == "__main__":
app()
+125
View File
@@ -0,0 +1,125 @@
"""
Observability Configuration
Unified configuration for all observability providers.
"""
import os
from dataclasses import dataclass, field
from typing import Any, Dict, List, Optional
@dataclass
class ObservabilityConfig:
"""Configuration for observability providers."""
# Provider selection
provider: Optional[str] = None # Auto-detect if None
# Common settings
project_name: Optional[str] = None
session_id: Optional[str] = None
run_id: Optional[str] = None
user_id: Optional[str] = None
# Tracing settings
enabled: bool = True
trace_llm_calls: bool = True
trace_tool_calls: bool = True
trace_agent_steps: bool = True
# Multi-agent settings
propagate_context: bool = True
parent_trace_id: Optional[str] = None
# Performance settings
batch_size: int = 100
flush_interval: float = 5.0
async_export: bool = True
# Redaction settings
redact_inputs: bool = False
redact_outputs: bool = False
redaction_patterns: List[str] = field(default_factory=list)
# Provider-specific settings
extra: Dict[str, Any] = field(default_factory=dict)
@classmethod
def from_env(cls) -> "ObservabilityConfig":
"""Create config from environment variables."""
return cls(
provider=os.getenv("PRAISONAI_OBS_PROVIDER"),
project_name=os.getenv("PRAISONAI_OBS_PROJECT"),
session_id=os.getenv("PRAISONAI_OBS_SESSION_ID"),
run_id=os.getenv("PRAISONAI_OBS_RUN_ID"),
user_id=os.getenv("PRAISONAI_OBS_USER_ID"),
enabled=os.getenv("PRAISONAI_OBS_ENABLED", "true").lower() == "true",
trace_llm_calls=os.getenv("PRAISONAI_OBS_TRACE_LLM", "true").lower() == "true",
trace_tool_calls=os.getenv("PRAISONAI_OBS_TRACE_TOOLS", "true").lower() == "true",
trace_agent_steps=os.getenv("PRAISONAI_OBS_TRACE_AGENTS", "true").lower() == "true",
propagate_context=os.getenv("PRAISONAI_OBS_PROPAGATE", "true").lower() == "true",
batch_size=int(os.getenv("PRAISONAI_OBS_BATCH_SIZE", "100")),
flush_interval=float(os.getenv("PRAISONAI_OBS_FLUSH_INTERVAL", "5.0")),
async_export=os.getenv("PRAISONAI_OBS_ASYNC", "true").lower() == "true",
redact_inputs=os.getenv("PRAISONAI_OBS_REDACT_INPUTS", "false").lower() == "true",
redact_outputs=os.getenv("PRAISONAI_OBS_REDACT_OUTPUTS", "false").lower() == "true",
)
def merge(self, **kwargs) -> "ObservabilityConfig":
"""Create a new config with merged values."""
current = {
"provider": self.provider,
"project_name": self.project_name,
"session_id": self.session_id,
"run_id": self.run_id,
"user_id": self.user_id,
"enabled": self.enabled,
"trace_llm_calls": self.trace_llm_calls,
"trace_tool_calls": self.trace_tool_calls,
"trace_agent_steps": self.trace_agent_steps,
"propagate_context": self.propagate_context,
"parent_trace_id": self.parent_trace_id,
"batch_size": self.batch_size,
"flush_interval": self.flush_interval,
"async_export": self.async_export,
"redact_inputs": self.redact_inputs,
"redact_outputs": self.redact_outputs,
"redaction_patterns": self.redaction_patterns.copy(),
"extra": self.extra.copy(),
}
current.update({k: v for k, v in kwargs.items() if v is not None})
return ObservabilityConfig(**current)
# Provider environment variable mappings
PROVIDER_ENV_KEYS = {
"agentops": ["AGENTOPS_API_KEY"],
"langfuse": ["LANGFUSE_PUBLIC_KEY", "LANGFUSE_SECRET_KEY"],
"langsmith": ["LANGSMITH_API_KEY"],
"traceloop": ["TRACELOOP_API_KEY"],
"arize_phoenix": ["PHOENIX_API_KEY"],
"openlit": ["OPENLIT_API_KEY"],
"langtrace": ["LANGTRACE_API_KEY"],
"langwatch": ["LANGWATCH_API_KEY"],
"datadog": ["DD_API_KEY"],
"mlflow": ["MLFLOW_TRACKING_URI"],
"opik": ["OPIK_API_KEY", "COMET_API_KEY"],
"portkey": ["PORTKEY_API_KEY"],
"braintrust": ["BRAINTRUST_API_KEY"],
"maxim": ["MAXIM_API_KEY"],
"weave": ["WANDB_API_KEY"],
"neatlogs": ["NEATLOGS_API_KEY"],
"langdb": ["LANGDB_API_KEY"],
"atla": ["ATLA_API_KEY"],
"patronus": ["PATRONUS_API_KEY"],
"truefoundry": ["TRUEFOUNDRY_API_KEY"],
}
def detect_provider() -> Optional[str]:
"""Auto-detect available provider from environment variables."""
for provider, keys in PROVIDER_ENV_KEYS.items():
if all(os.getenv(key) for key in keys):
return provider
return None
+672
View File
@@ -0,0 +1,672 @@
"""
Observability Manager
Central manager for all observability providers.
"""
import os
from typing import Any, Dict, List, Optional, Type
from praisonai_tools.observability.base import (
BaseObservabilityProvider,
Span,
SpanKind,
SpanStatus,
Trace,
)
from praisonai_tools.observability.config import (
ObservabilityConfig,
PROVIDER_ENV_KEYS,
detect_provider,
)
class ObservabilityManager:
"""
Central manager for observability providers.
Usage:
from praisonai_tools.observability import obs
# Auto-detect provider from env vars
obs.init()
# Or specify provider
obs.init(provider="langfuse")
# Use context managers
with obs.trace("my-workflow"):
with obs.span("step-1", kind=SpanKind.AGENT):
# do work
pass
"""
_instance: Optional["ObservabilityManager"] = None
_providers: Dict[str, Type[BaseObservabilityProvider]] = {}
def __new__(cls) -> "ObservabilityManager":
"""Singleton pattern."""
if cls._instance is None:
cls._instance = super().__new__(cls)
cls._instance._initialized = False
return cls._instance
def __init__(self):
"""Initialize the manager."""
if self._initialized:
return
self.config: Optional[ObservabilityConfig] = None
self._provider: Optional[BaseObservabilityProvider] = None
self._initialized = True
@classmethod
def register_provider(cls, name: str, provider_class: Type[BaseObservabilityProvider]) -> None:
"""Register a provider class."""
cls._providers[name] = provider_class
@classmethod
def list_providers(cls) -> List[str]:
"""List all registered providers."""
return list(cls._providers.keys())
@classmethod
def get_available_providers(cls) -> List[str]:
"""Get providers that have required env vars set."""
available = []
for name in cls._providers:
if name in PROVIDER_ENV_KEYS:
keys = PROVIDER_ENV_KEYS[name]
if all(os.getenv(key) for key in keys):
available.append(name)
return available
def init(
self,
provider: Optional[str] = None,
project_name: Optional[str] = None,
session_id: Optional[str] = None,
auto_instrument: bool = True,
**kwargs
) -> bool:
"""
Initialize observability with a provider.
Args:
provider: Provider name (auto-detect if None)
project_name: Project/app name for traces
session_id: Session identifier
auto_instrument: If True (default), auto-instruments Agent/Agents/Workflow
classes so all operations are traced without explicit
obs.trace() wrappers
**kwargs: Additional provider-specific config
Returns:
True if initialization successful
Example:
# Simplest usage - auto-detect provider and auto-instrument
obs.init()
# Explicit provider
obs.init(provider="langfuse")
# Disable auto-instrumentation for manual control
obs.init(auto_instrument=False)
"""
# Build config
self.config = ObservabilityConfig.from_env().merge(
provider=provider,
project_name=project_name,
session_id=session_id,
extra=kwargs,
)
# Auto-detect provider if not specified
provider_name = self.config.provider or detect_provider()
if not provider_name:
# No provider configured - silent no-op mode
return False
# Get provider class
if provider_name not in self._providers:
# Try to load provider dynamically
self._load_provider(provider_name)
if provider_name not in self._providers:
return False
# Instantiate and initialize provider
provider_class = self._providers[provider_name]
self._provider = provider_class(self.config)
result = self._provider.init(**kwargs)
# Auto-instrument Agent classes after successful init
if result and auto_instrument:
self._auto_instrument_agents()
return result
# Mapping from obs provider names to litellm callback names
_LITELLM_CALLBACK_MAP = {
"langsmith": "langsmith",
"langfuse": "langfuse",
"datadog": "datadog",
"opik": "opik",
"braintrust": "braintrust",
"mlflow": "mlflow",
"langwatch": "langwatch",
}
def _get_litellm_callback_name(self, provider_name: str) -> Optional[str]:
"""Get the litellm callback name for a provider."""
return self._LITELLM_CALLBACK_MAP.get(provider_name)
def _auto_instrument_agents(self) -> None:
"""
Auto-instrument Agent/Agents/Workflow classes to create spans automatically.
This patches the main lifecycle methods (chat, start, run, astart) to
create traces and spans without requiring explicit obs.trace() wrappers.
Also sets litellm callbacks for providers that support it, and registers
the ObservabilitySink bridge for ContextTraceEmitter events.
"""
if not self._provider or not self._provider._initialized:
return
try:
# Set litellm callbacks for the active provider
self._setup_litellm_callbacks()
# Register bridge sink for ContextTraceEmitter events
self._setup_bridge_sink()
# Try to import praisonaiagents classes
try:
from praisonaiagents.agent.agent import Agent
self._wrap_agent_class(Agent)
except ImportError:
pass
try:
from praisonaiagents.agents.agents import Agents
self._wrap_workflow_class(Agents)
except ImportError:
pass
except Exception:
# Silently fail - don't break user code if instrumentation fails
pass
def _setup_litellm_callbacks(self) -> None:
"""Set litellm callbacks for the active provider."""
provider_name = self._provider.name if self._provider else None
if not provider_name:
return
callback_name = self._get_litellm_callback_name(provider_name)
if not callback_name:
return
try:
import litellm
# Add callback if not already present
if not hasattr(litellm, 'callbacks') or litellm.callbacks is None:
litellm.callbacks = []
if callback_name not in litellm.callbacks:
litellm.callbacks.append(callback_name)
if not hasattr(litellm, 'success_callback') or litellm.success_callback is None:
litellm.success_callback = []
if callback_name not in litellm.success_callback:
litellm.success_callback.append(callback_name)
except ImportError:
pass
def _setup_bridge_sink(self) -> None:
"""Register ObservabilitySink as the ContextTraceEmitter sink."""
try:
from praisonai_tools.observability.bridge import ObservabilitySink
from praisonaiagents.trace.context_events import (
ContextTraceEmitter,
set_context_emitter,
)
sink = ObservabilitySink(self)
emitter = ContextTraceEmitter(
sink=sink,
session_id=self.config.session_id or "",
enabled=True,
)
set_context_emitter(emitter)
except ImportError:
pass
@staticmethod
def _get_version():
"""Get praisonai-tools version lazily."""
try:
from praisonai_tools import __version__
return __version__
except Exception:
return "unknown"
@staticmethod
def _set_branding_attrs(span):
"""Set framework branding attributes on a span."""
if span is None:
return
try:
span.attributes["praisonai.version"] = ObservabilityManager._get_version()
span.attributes["praisonai.framework"] = "praisonai"
except Exception:
pass
def _wrap_agent_class(self, agent_cls: type) -> None:
"""Wrap Agent class methods to auto-create spans with input/output data."""
from functools import wraps
import json
# Skip if already instrumented
if getattr(agent_cls, '_obs_instrumented', False):
return
manager = self
def _extract_chat_input(agent_self, args, kwargs):
"""Extract the input prompt from chat() arguments."""
prompt = args[0] if args else kwargs.get('prompt', kwargs.get('message', ''))
agent_name = getattr(agent_self, 'name', 'unknown')
instructions = getattr(agent_self, 'instructions', '')
model = getattr(agent_self, 'model', None)
# Use a dict format that LangSmith can display
input_data = {"input": str(prompt) if prompt else ""}
if instructions:
input_data["instructions"] = str(instructions)
if agent_name:
input_data["agent_name"] = agent_name
if model:
input_data["model"] = str(model)
return input_data
def _set_span_io(span, input_data, output_data):
"""Set input/output attributes on a span for LangSmith display."""
if span is None:
return
try:
# LangSmith recognizes these attributes for chain/agent spans
if input_data:
input_str = json.dumps(input_data) if isinstance(input_data, dict) else str(input_data)
span.attributes["input.value"] = input_str
span.attributes["input"] = input_str
if output_data:
output_str = str(output_data) if not isinstance(output_data, str) else output_data
span.attributes["output.value"] = output_str
span.attributes["output"] = output_str
except Exception:
pass
# Wrap chat method
if hasattr(agent_cls, 'chat'):
original_chat = agent_cls.chat
@wraps(original_chat)
def instrumented_chat(self, *args, **kwargs):
agent_name = getattr(self, 'name', 'unknown')
input_data = _extract_chat_input(self, args, kwargs)
with manager.trace(agent_name):
span = manager.start_span(agent_name, kind=SpanKind.AGENT)
_set_branding_attrs = ObservabilityManager._set_branding_attrs
_set_branding_attrs(span)
try:
# Set input before the call
_set_span_io(span, input_data, None)
result = original_chat(self, *args, **kwargs)
# Set output after the call
output_str = str(result) if result else ""
# Truncate very long outputs for span attributes
if len(output_str) > 4000:
output_str = output_str[:4000] + "..."
_set_span_io(span, None, output_str)
return result
except Exception as e:
if span:
span.set_error(e)
raise
finally:
manager.end_span(span)
agent_cls.chat = instrumented_chat
# Wrap start method
if hasattr(agent_cls, 'start'):
original_start = agent_cls.start
@wraps(original_start)
def instrumented_start(self, *args, **kwargs):
import types
agent_name = getattr(self, 'name', 'unknown')
result = original_start(self, *args, **kwargs)
# Handle generator (streaming) mode
if isinstance(result, types.GeneratorType):
def traced_generator():
with manager.trace(agent_name):
span_ctx = manager.span(agent_name, kind=SpanKind.AGENT)
with span_ctx as span:
ObservabilityManager._set_branding_attrs(span)
for chunk in result:
yield chunk
return traced_generator()
else:
return result
agent_cls.start = instrumented_start
# Wrap run method
if hasattr(agent_cls, 'run'):
original_run = agent_cls.run
@wraps(original_run)
def instrumented_run(self, *args, **kwargs):
agent_name = getattr(self, 'name', 'unknown')
with manager.trace(agent_name):
with manager.span(agent_name, kind=SpanKind.AGENT) as span:
ObservabilityManager._set_branding_attrs(span)
return original_run(self, *args, **kwargs)
agent_cls.run = instrumented_run
# Mark as instrumented
agent_cls._obs_instrumented = True
def _wrap_workflow_class(self, workflow_cls: type) -> None:
"""Wrap Agents (workflow) class methods to auto-create spans."""
from functools import wraps
# Skip if already instrumented
if getattr(workflow_cls, '_obs_instrumented', False):
return
manager = self
# Wrap start method
if hasattr(workflow_cls, 'start'):
original_start = workflow_cls.start
@wraps(original_start)
def instrumented_start(self, *args, **kwargs):
workflow_name = getattr(self, 'name', None) or 'workflow'
with manager.trace(workflow_name):
with manager.span(workflow_name, kind=SpanKind.WORKFLOW) as span:
ObservabilityManager._set_branding_attrs(span)
# Capture input: task descriptions
try:
tasks_raw = getattr(self, 'tasks', []) or []
# Agents stores tasks as dict {idx: Task}, not a list
task_items = tasks_raw.values() if isinstance(tasks_raw, dict) else tasks_raw
agents_raw = getattr(self, 'agents', []) or []
agent_items = agents_raw.values() if isinstance(agents_raw, dict) else agents_raw
input_data = {
"workflow": workflow_name,
"agents": [getattr(a, 'name', str(a)) for a in agent_items],
"tasks": [getattr(t, 'description', str(t)) for t in task_items],
}
import json
input_str = json.dumps(input_data)
if span:
span.attributes["input.value"] = input_str
span.attributes["input"] = input_str
except Exception:
pass
result = original_start(self, *args, **kwargs)
# Capture output
try:
if span and result:
output_str = str(result)
if len(output_str) > 4000:
output_str = output_str[:4000] + "..."
span.attributes["output.value"] = output_str
span.attributes["output"] = output_str
except Exception:
pass
return result
workflow_cls.start = instrumented_start
# Wrap astart (async) method
if hasattr(workflow_cls, 'astart'):
original_astart = workflow_cls.astart
@wraps(original_astart)
async def instrumented_astart(self, *args, **kwargs):
workflow_name = getattr(self, 'name', None) or 'workflow'
with manager.trace(workflow_name):
with manager.span(workflow_name, kind=SpanKind.WORKFLOW) as span:
ObservabilityManager._set_branding_attrs(span)
# Capture input
try:
tasks_raw = getattr(self, 'tasks', []) or []
task_items = tasks_raw.values() if isinstance(tasks_raw, dict) else tasks_raw
agents_raw = getattr(self, 'agents', []) or []
agent_items = agents_raw.values() if isinstance(agents_raw, dict) else agents_raw
input_data = {
"workflow": workflow_name,
"agents": [getattr(a, 'name', str(a)) for a in agent_items],
"tasks": [getattr(t, 'description', str(t)) for t in task_items],
}
import json
input_str = json.dumps(input_data)
if span:
span.attributes["input.value"] = input_str
span.attributes["input"] = input_str
except Exception:
pass
result = await original_astart(self, *args, **kwargs)
# Capture output
try:
if span and result:
output_str = str(result)
if len(output_str) > 4000:
output_str = output_str[:4000] + "..."
span.attributes["output.value"] = output_str
span.attributes["output"] = output_str
except Exception:
pass
return result
workflow_cls.astart = instrumented_astart
# Mark as instrumented
workflow_cls._obs_instrumented = True
def _load_provider(self, name: str) -> None:
"""Dynamically load a provider module."""
provider_modules = {
"agentops": "agentops_provider",
"langfuse": "langfuse_provider",
"langsmith": "langsmith_provider",
"traceloop": "traceloop_provider",
"arize_phoenix": "arize_phoenix_provider",
"openlit": "openlit_provider",
"langtrace": "langtrace_provider",
"langwatch": "langwatch_provider",
"datadog": "datadog_provider",
"mlflow": "mlflow_provider",
"opik": "opik_provider",
"portkey": "portkey_provider",
"braintrust": "braintrust_provider",
"maxim": "maxim_provider",
"weave": "weave_provider",
"neatlogs": "neatlogs_provider",
"langdb": "langdb_provider",
"atla": "atla_provider",
"patronus": "patronus_provider",
"truefoundry": "truefoundry_provider",
}
if name in provider_modules:
try:
import importlib
importlib.import_module(
f"praisonai_tools.observability.providers.{provider_modules[name]}"
)
# Provider should auto-register on import
except ImportError:
pass
def shutdown(self) -> None:
"""Shutdown the current provider."""
if self._provider:
self._provider.shutdown()
self._provider = None
@property
def provider(self) -> Optional[BaseObservabilityProvider]:
"""Get the current provider."""
return self._provider
@property
def enabled(self) -> bool:
"""Check if observability is enabled."""
return self._provider is not None and self._provider._initialized
def doctor(self) -> Dict[str, Any]:
"""
Run diagnostics on observability setup.
Returns:
Dict with diagnostic results
"""
results = {
"enabled": self.enabled,
"provider": self._provider.name if self._provider else None,
"available_providers": self.get_available_providers(),
"registered_providers": self.list_providers(),
"connection_status": None,
"connection_message": None,
}
if self._provider:
success, message = self._provider.check_connection()
results["connection_status"] = success
results["connection_message"] = message
return results
# Delegate methods to provider
def trace(self, name: str, **kwargs):
"""Start a trace context manager."""
if self._provider:
return self._provider.trace(name, **kwargs)
return _NoOpContextManager()
def span(self, name: str, kind: SpanKind = SpanKind.CUSTOM, **kwargs):
"""Start a span context manager."""
if self._provider:
return self._provider.span(name, kind, **kwargs)
return _NoOpContextManager()
def start_trace(self, name: str, **kwargs) -> Optional[Trace]:
"""Start a new trace."""
if self._provider:
return self._provider.start_trace(name, **kwargs)
return None
def end_trace(self, trace: Optional[Trace] = None) -> None:
"""End a trace."""
if self._provider:
self._provider.end_trace(trace)
def start_span(self, name: str, kind: SpanKind = SpanKind.CUSTOM, **kwargs) -> Optional[Span]:
"""Start a new span."""
if self._provider:
return self._provider.start_span(name, kind, **kwargs)
return None
def end_span(self, span: Optional[Span] = None, status: SpanStatus = SpanStatus.OK) -> None:
"""End a span."""
if self._provider:
self._provider.end_span(span, status)
def log_llm_call(self, model: str, input_messages: Any, output: Any, **kwargs) -> Optional[Span]:
"""Log an LLM call."""
if self._provider:
return self._provider.log_llm_call(model, input_messages, output, **kwargs)
return None
def log_tool_call(self, tool_name: str, tool_input: Any, tool_output: Any, **kwargs) -> Optional[Span]:
"""Log a tool call."""
if self._provider:
return self._provider.log_tool_call(tool_name, tool_input, tool_output, **kwargs)
return None
def log_agent_step(self, agent_name: str, step_name: str, input_data: Any, output_data: Any, **kwargs) -> Optional[Span]:
"""Log an agent step."""
if self._provider:
return self._provider.log_agent_step(agent_name, step_name, input_data, output_data, **kwargs)
return None
def decorator(self, name: Optional[str] = None, kind: SpanKind = SpanKind.CUSTOM):
"""Decorator to trace a function."""
if self._provider:
return self._provider.decorator(name, kind)
# No-op decorator
def noop(func):
return func
return noop
class _NoOpContextManager:
"""No-op context manager when observability is disabled."""
def __enter__(self):
return None
def __exit__(self, *args):
pass
# Auto-register built-in providers on first access
def _auto_register_providers():
"""Auto-register all built-in providers."""
provider_classes = [
("agentops", "AgentOpsProvider", "agentops_provider"),
("langfuse", "LangfuseProvider", "langfuse_provider"),
("langsmith", "LangSmithProvider", "langsmith_provider"),
("traceloop", "TraceloopProvider", "traceloop_provider"),
("arize_phoenix", "ArizePhoenixProvider", "arize_phoenix_provider"),
("openlit", "OpenLITProvider", "openlit_provider"),
("langtrace", "LangtraceProvider", "langtrace_provider"),
("langwatch", "LangWatchProvider", "langwatch_provider"),
("datadog", "DatadogProvider", "datadog_provider"),
("mlflow", "MLflowProvider", "mlflow_provider"),
("opik", "OpikProvider", "opik_provider"),
("portkey", "PortkeyProvider", "portkey_provider"),
("braintrust", "BraintrustProvider", "braintrust_provider"),
("maxim", "MaximProvider", "maxim_provider"),
("weave", "WeaveProvider", "weave_provider"),
("neatlogs", "NeatlogsProvider", "neatlogs_provider"),
("langdb", "LangDBProvider", "langdb_provider"),
("atla", "AtlaProvider", "atla_provider"),
("patronus", "PatronusProvider", "patronus_provider"),
("truefoundry", "TrueFoundryProvider", "truefoundry_provider"),
]
for name, class_name, module_name in provider_classes:
try:
import importlib
module = importlib.import_module(
f"praisonai_tools.observability.providers.{module_name}"
)
provider_class = getattr(module, class_name)
ObservabilityManager.register_provider(name, provider_class)
except (ImportError, AttributeError):
pass # Provider not available
@@ -0,0 +1,63 @@
"""
Observability Providers
All provider implementations for the observability suite.
"""
__all__ = [
"AgentOpsProvider",
"LangfuseProvider",
"LangSmithProvider",
"TraceloopProvider",
"ArizePhoenixProvider",
"OpenLITProvider",
"LangtraceProvider",
"LangWatchProvider",
"DatadogProvider",
"MLflowProvider",
"OpikProvider",
"PortkeyProvider",
"BraintrustProvider",
"MaximProvider",
"WeaveProvider",
"NeatlogsProvider",
"LangDBProvider",
"AtlaProvider",
"PatronusProvider",
"TrueFoundryProvider",
]
def __getattr__(name: str):
"""Lazy load provider classes."""
provider_map = {
"AgentOpsProvider": "agentops_provider",
"LangfuseProvider": "langfuse_provider",
"LangSmithProvider": "langsmith_provider",
"TraceloopProvider": "traceloop_provider",
"ArizePhoenixProvider": "arize_phoenix_provider",
"OpenLITProvider": "openlit_provider",
"LangtraceProvider": "langtrace_provider",
"LangWatchProvider": "langwatch_provider",
"DatadogProvider": "datadog_provider",
"MLflowProvider": "mlflow_provider",
"OpikProvider": "opik_provider",
"PortkeyProvider": "portkey_provider",
"BraintrustProvider": "braintrust_provider",
"MaximProvider": "maxim_provider",
"WeaveProvider": "weave_provider",
"NeatlogsProvider": "neatlogs_provider",
"LangDBProvider": "langdb_provider",
"AtlaProvider": "atla_provider",
"PatronusProvider": "patronus_provider",
"TrueFoundryProvider": "truefoundry_provider",
}
if name in provider_map:
import importlib
module = importlib.import_module(
f"praisonai_tools.observability.providers.{provider_map[name]}"
)
return getattr(module, name)
raise AttributeError(f"module 'praisonai_tools.observability.providers' has no attribute '{name}'")
@@ -0,0 +1,119 @@
"""
AgentOps Provider
Integration with AgentOps for agent observability.
https://agentops.ai/
"""
import os
from typing import Optional
from praisonai_tools.observability.base import (
BaseObservabilityProvider,
Span,
Trace,
)
from praisonai_tools.observability.config import ObservabilityConfig
from praisonai_tools.observability.manager import ObservabilityManager
class AgentOpsProvider(BaseObservabilityProvider):
"""AgentOps observability provider."""
name = "agentops"
def __init__(self, config: Optional[ObservabilityConfig] = None):
"""Initialize AgentOps provider."""
super().__init__(config)
self._client = None
self._session = None
def is_available(self) -> bool:
"""Check if agentops SDK is available."""
try:
import agentops
return True
except ImportError:
return False
def init(self, **kwargs) -> bool:
"""Initialize AgentOps."""
if not self.is_available():
return False
try:
import agentops
api_key = kwargs.get("api_key") or os.getenv("AGENTOPS_API_KEY")
if not api_key:
return False
# Initialize AgentOps
agentops.init(
api_key=api_key,
default_tags=kwargs.get("tags", []),
auto_start_session=kwargs.get("auto_start_session", True),
)
self._client = agentops
self._initialized = True
return True
except Exception as e:
print(f"AgentOps init failed: {e}")
return False
def shutdown(self) -> None:
"""Shutdown AgentOps."""
if self._client and self._session:
try:
self._client.end_session()
except Exception:
pass
self._initialized = False
def check_connection(self) -> tuple:
"""Check AgentOps connection."""
if not self._initialized:
return False, "AgentOps not initialized"
try:
# AgentOps doesn't have a direct ping method
# Check if we have a valid API key
api_key = os.getenv("AGENTOPS_API_KEY")
if api_key:
return True, "AgentOps API key configured"
return False, "AGENTOPS_API_KEY not set"
except Exception as e:
return False, str(e)
def export_span(self, span: Span) -> bool:
"""Export span to AgentOps."""
if not self._initialized or not self._client:
return False
try:
# AgentOps auto-instruments, but we can record events
if span.error_message:
self._client.record(
self._client.ErrorEvent(
error_type=span.error_type or "Error",
details=span.error_message,
)
)
return True
except Exception:
return False
def export_trace(self, trace: Trace) -> bool:
"""Export trace to AgentOps."""
if not self._initialized:
return False
# AgentOps handles traces automatically via sessions
return True
# Auto-register provider
ObservabilityManager.register_provider("agentops", AgentOpsProvider)
@@ -0,0 +1,104 @@
"""
Arize Phoenix Provider
Integration with Arize Phoenix for LLM observability.
https://phoenix.arize.com/
"""
import os
from typing import Optional
from praisonai_tools.observability.base import (
BaseObservabilityProvider,
Span,
Trace,
)
from praisonai_tools.observability.config import ObservabilityConfig
from praisonai_tools.observability.manager import ObservabilityManager
class ArizePhoenixProvider(BaseObservabilityProvider):
"""Arize Phoenix observability provider."""
name = "arize_phoenix"
def __init__(self, config: Optional[ObservabilityConfig] = None):
"""Initialize Arize Phoenix provider."""
super().__init__(config)
self._tracer_provider = None
def is_available(self) -> bool:
"""Check if phoenix SDK is available."""
try:
import importlib.util
return importlib.util.find_spec("phoenix.otel") is not None
except ImportError:
return False
def init(self, **kwargs) -> bool:
"""Initialize Arize Phoenix."""
if not self.is_available():
return False
try:
from phoenix.otel import register
api_key = kwargs.get("api_key") or os.getenv("PHOENIX_API_KEY")
endpoint = kwargs.get("endpoint") or os.getenv(
"PHOENIX_COLLECTOR_ENDPOINT", "https://app.phoenix.arize.com/"
)
project_name = kwargs.get("project_name") or self.config.project_name or "default"
# Set environment variables
if api_key:
os.environ["PHOENIX_API_KEY"] = api_key
os.environ["PHOENIX_COLLECTOR_ENDPOINT"] = endpoint
# Register Phoenix tracer
self._tracer_provider = register(
project_name=project_name,
auto_instrument=kwargs.get("auto_instrument", True),
)
self._initialized = True
return True
except Exception as e:
print(f"Arize Phoenix init failed: {e}")
return False
def shutdown(self) -> None:
"""Shutdown Arize Phoenix provider."""
if self._tracer_provider:
try:
self._tracer_provider.shutdown()
except Exception:
pass
self._initialized = False
def check_connection(self) -> tuple:
"""Check Arize Phoenix connection."""
if not self._initialized:
return False, "Arize Phoenix not initialized"
api_key = os.getenv("PHOENIX_API_KEY")
if api_key:
return True, "Phoenix API key configured"
return False, "PHOENIX_API_KEY not set"
def export_span(self, span: Span) -> bool:
"""Export span to Arize Phoenix."""
if not self._initialized:
return False
# Phoenix auto-instruments via OpenInference
return True
def export_trace(self, trace_obj: Trace) -> bool:
"""Export trace to Arize Phoenix."""
if not self._initialized:
return False
return True
# Auto-register provider
ObservabilityManager.register_provider("arize_phoenix", ArizePhoenixProvider)
@@ -0,0 +1,95 @@
"""
Atla Provider
Integration with Atla for LLM observability.
https://app.atla-ai.com/
"""
import os
from typing import Optional
from praisonai_tools.observability.base import (
BaseObservabilityProvider,
Span,
Trace,
)
from praisonai_tools.observability.config import ObservabilityConfig
from praisonai_tools.observability.manager import ObservabilityManager
class AtlaProvider(BaseObservabilityProvider):
"""Atla observability provider."""
name = "atla"
def __init__(self, config: Optional[ObservabilityConfig] = None):
"""Initialize Atla provider."""
super().__init__(config)
self._atla = None
def is_available(self) -> bool:
"""Check if atla SDK is available."""
try:
import importlib.util
return importlib.util.find_spec("atla_insights") is not None
except ImportError:
return False
def init(self, **kwargs) -> bool:
"""Initialize Atla."""
if not self.is_available():
return False
try:
from atla_insights import configure
api_key = kwargs.get("api_key") or os.getenv("ATLA_API_KEY")
if not api_key:
return False
configure(token=api_key)
self._initialized = True
return True
except Exception as e:
print(f"Atla init failed: {e}")
return False
def shutdown(self) -> None:
"""Shutdown Atla provider."""
self._initialized = False
def check_connection(self) -> tuple:
"""Check Atla connection."""
if not self._initialized:
return False, "Atla not initialized"
api_key = os.getenv("ATLA_API_KEY")
if api_key:
return True, "Atla API key configured"
return False, "ATLA_API_KEY not set"
def instrument(self, provider: str = "openai"):
"""Get Atla instrumentation context manager."""
if not self.is_available():
from contextlib import nullcontext
return nullcontext()
try:
from atla_insights import instrument_agno
return instrument_agno(provider)
except ImportError:
from contextlib import nullcontext
return nullcontext()
def export_span(self, span: Span) -> bool:
"""Export span to Atla."""
return self._initialized
def export_trace(self, trace_obj: Trace) -> bool:
"""Export trace to Atla."""
return self._initialized
ObservabilityManager.register_provider("atla", AtlaProvider)
@@ -0,0 +1,106 @@
"""
Braintrust Provider
Integration with Braintrust for LLM observability and evaluation.
https://www.braintrust.dev/
"""
import os
from typing import Optional
from praisonai_tools.observability.base import (
BaseObservabilityProvider,
Span,
Trace,
)
from praisonai_tools.observability.config import ObservabilityConfig
from praisonai_tools.observability.manager import ObservabilityManager
class BraintrustProvider(BaseObservabilityProvider):
"""Braintrust observability provider."""
name = "braintrust"
def __init__(self, config: Optional[ObservabilityConfig] = None):
"""Initialize Braintrust provider."""
super().__init__(config)
self._tracer_provider = None
def is_available(self) -> bool:
"""Check if braintrust SDK is available."""
try:
import importlib.util
return importlib.util.find_spec("braintrust") is not None
except ImportError:
return False
def init(self, **kwargs) -> bool:
"""Initialize Braintrust."""
if not self.is_available():
return False
try:
from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from braintrust.otel import BraintrustSpanProcessor
api_key = kwargs.get("api_key") or os.getenv("BRAINTRUST_API_KEY")
project = kwargs.get("project") or os.getenv("BRAINTRUST_PARENT", "praisonai")
if not api_key:
return False
os.environ["BRAINTRUST_API_KEY"] = api_key
os.environ["BRAINTRUST_PARENT"] = f"project_name:{project}"
current_provider = trace.get_tracer_provider()
if isinstance(current_provider, TracerProvider):
provider = current_provider
else:
provider = TracerProvider()
trace.set_tracer_provider(provider)
provider.add_span_processor(BraintrustSpanProcessor())
self._tracer_provider = provider
self._initialized = True
return True
except Exception as e:
print(f"Braintrust init failed: {e}")
return False
def shutdown(self) -> None:
"""Shutdown Braintrust provider."""
if self._tracer_provider:
try:
self._tracer_provider.shutdown()
except Exception:
pass
self._initialized = False
def check_connection(self) -> tuple:
"""Check Braintrust connection."""
if not self._initialized:
return False, "Braintrust not initialized"
api_key = os.getenv("BRAINTRUST_API_KEY")
if api_key:
return True, "Braintrust API key configured"
return False, "BRAINTRUST_API_KEY not set"
def export_span(self, span: Span) -> bool:
"""Export span to Braintrust."""
if not self._initialized:
return False
return True
def export_trace(self, trace_obj: Trace) -> bool:
"""Export trace to Braintrust."""
if not self._initialized:
return False
return True
ObservabilityManager.register_provider("braintrust", BraintrustProvider)
@@ -0,0 +1,91 @@
"""
Datadog Provider
Integration with Datadog LLM Observability.
https://www.datadoghq.com/product/llm-observability/
"""
import os
from typing import Optional
from praisonai_tools.observability.base import (
BaseObservabilityProvider,
Span,
Trace,
)
from praisonai_tools.observability.config import ObservabilityConfig
from praisonai_tools.observability.manager import ObservabilityManager
class DatadogProvider(BaseObservabilityProvider):
"""Datadog LLM Observability provider."""
name = "datadog"
def __init__(self, config: Optional[ObservabilityConfig] = None):
"""Initialize Datadog provider."""
super().__init__(config)
self._ddtrace = None
def is_available(self) -> bool:
"""Check if ddtrace SDK is available."""
try:
import importlib.util
return importlib.util.find_spec("ddtrace") is not None
except ImportError:
return False
def init(self, **kwargs) -> bool:
"""Initialize Datadog."""
if not self.is_available():
return False
try:
api_key = kwargs.get("api_key") or os.getenv("DD_API_KEY")
site = kwargs.get("site") or os.getenv("DD_SITE", "datadoghq.com")
ml_app = kwargs.get("ml_app") or os.getenv("DD_LLMOBS_ML_APP", "praisonai")
if not api_key:
return False
os.environ["DD_API_KEY"] = api_key
os.environ["DD_SITE"] = site
os.environ["DD_LLMOBS_ENABLED"] = "true"
os.environ["DD_LLMOBS_ML_APP"] = ml_app
os.environ["DD_LLMOBS_AGENTLESS_ENABLED"] = "true"
self._initialized = True
return True
except Exception as e:
print(f"Datadog init failed: {e}")
return False
def shutdown(self) -> None:
"""Shutdown Datadog provider."""
self._initialized = False
def check_connection(self) -> tuple:
"""Check Datadog connection."""
if not self._initialized:
return False, "Datadog not initialized"
api_key = os.getenv("DD_API_KEY")
if api_key:
return True, "Datadog API key configured"
return False, "DD_API_KEY not set"
def export_span(self, span: Span) -> bool:
"""Export span to Datadog."""
if not self._initialized:
return False
return True
def export_trace(self, trace_obj: Trace) -> bool:
"""Export trace to Datadog."""
if not self._initialized:
return False
return True
ObservabilityManager.register_provider("datadog", DatadogProvider)
@@ -0,0 +1,63 @@
"""
LangDB Provider
Integration with LangDB for SQL-based LLM observability.
"""
import os
from typing import Optional
from praisonai_tools.observability.base import (
BaseObservabilityProvider,
Span,
Trace,
)
from praisonai_tools.observability.config import ObservabilityConfig
from praisonai_tools.observability.manager import ObservabilityManager
class LangDBProvider(BaseObservabilityProvider):
"""LangDB observability provider."""
name = "langdb"
def __init__(self, config: Optional[ObservabilityConfig] = None):
"""Initialize LangDB provider."""
super().__init__(config)
def is_available(self) -> bool:
"""Check if langdb SDK is available."""
try:
import importlib.util
return importlib.util.find_spec("langdb") is not None
except ImportError:
return False
def init(self, **kwargs) -> bool:
"""Initialize LangDB."""
api_key = kwargs.get("api_key") or os.getenv("LANGDB_API_KEY")
if not api_key:
return False
self._initialized = True
return True
def shutdown(self) -> None:
"""Shutdown LangDB provider."""
self._initialized = False
def check_connection(self) -> tuple:
"""Check LangDB connection."""
if not self._initialized:
return False, "LangDB not initialized"
return True, "LangDB configured"
def export_span(self, span: Span) -> bool:
"""Export span to LangDB."""
return self._initialized
def export_trace(self, trace_obj: Trace) -> bool:
"""Export trace to LangDB."""
return self._initialized
ObservabilityManager.register_provider("langdb", LangDBProvider)
@@ -0,0 +1,159 @@
"""
Langfuse Provider
Integration with Langfuse for LLM observability.
https://langfuse.com/
"""
import base64
import os
from typing import Optional
from praisonai_tools.observability.base import (
BaseObservabilityProvider,
Span,
Trace,
)
from praisonai_tools.observability.config import ObservabilityConfig
from praisonai_tools.observability.manager import ObservabilityManager
class LangfuseProvider(BaseObservabilityProvider):
"""Langfuse observability provider using OpenTelemetry."""
name = "langfuse"
def __init__(self, config: Optional[ObservabilityConfig] = None):
"""Initialize Langfuse provider."""
super().__init__(config)
self._tracer_provider = None
self._tracer = None
def is_available(self) -> bool:
"""Check if required SDKs are available."""
try:
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
return True
except ImportError:
return False
def init(self, **kwargs) -> bool:
"""Initialize Langfuse via OpenTelemetry."""
if not self.is_available():
return False
try:
from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import SimpleSpanProcessor
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
from opentelemetry.sdk.resources import Resource
public_key = kwargs.get("public_key") or os.getenv("LANGFUSE_PUBLIC_KEY")
secret_key = kwargs.get("secret_key") or os.getenv("LANGFUSE_SECRET_KEY")
host = kwargs.get("host") or os.getenv("LANGFUSE_HOST", "https://cloud.langfuse.com")
if not public_key or not secret_key:
return False
# Create auth header
auth = base64.b64encode(f"{public_key}:{secret_key}".encode()).decode()
# Configure OTLP endpoint
endpoint = f"{host}/api/public/otel/v1/traces"
headers = {"Authorization": f"Basic {auth}"}
# Get praisonai version for resource attributes
_version = "unknown"
try:
import praisonai_tools
_version = getattr(praisonai_tools, "__version__", "unknown")
except Exception:
pass
# Setup tracer provider with PraisonAI resource branding
resource = Resource.create({
"service.name": "praisonai",
"service.version": _version,
"praisonai.version": _version,
"praisonai.framework": "praisonai",
})
self._tracer_provider = TracerProvider(resource=resource)
self._tracer_provider.add_span_processor(
SimpleSpanProcessor(OTLPSpanExporter(endpoint=endpoint, headers=headers))
)
trace.set_tracer_provider(self._tracer_provider)
self._tracer = trace.get_tracer("praisonai.observability")
self._initialized = True
return True
except Exception as e:
print(f"Langfuse init failed: {e}")
return False
def shutdown(self) -> None:
"""Shutdown Langfuse provider."""
if self._tracer_provider:
try:
self._tracer_provider.shutdown()
except Exception:
pass
self._initialized = False
def check_connection(self) -> tuple:
"""Check Langfuse connection."""
if not self._initialized:
return False, "Langfuse not initialized"
public_key = os.getenv("LANGFUSE_PUBLIC_KEY")
secret_key = os.getenv("LANGFUSE_SECRET_KEY")
if public_key and secret_key:
return True, "Langfuse credentials configured"
return False, "LANGFUSE_PUBLIC_KEY or LANGFUSE_SECRET_KEY not set"
def export_span(self, span: Span) -> bool:
"""Export span to Langfuse via OTel."""
if not self._initialized or not self._tracer:
return False
try:
with self._tracer.start_as_current_span(span.name) as otel_span:
# Set attributes
otel_span.set_attribute("span.kind", span.kind.value)
if span.model:
otel_span.set_attribute("llm.model", span.model)
if span.input_tokens:
otel_span.set_attribute("llm.input_tokens", span.input_tokens)
if span.output_tokens:
otel_span.set_attribute("llm.output_tokens", span.output_tokens)
if span.tool_name:
otel_span.set_attribute("tool.name", span.tool_name)
for key, value in span.attributes.items():
otel_span.set_attribute(key, str(value))
if span.error_message:
from opentelemetry.trace import Status, StatusCode
otel_span.set_status(
Status(StatusCode.ERROR, span.error_message)
)
return True
except Exception:
return False
def export_trace(self, trace_obj: Trace) -> bool:
"""Export trace to Langfuse."""
if not self._initialized:
return False
# Spans are exported individually via OTel
return True
# Auto-register provider
ObservabilityManager.register_provider("langfuse", LangfuseProvider)
@@ -0,0 +1,186 @@
"""
LangSmith Provider
Integration with LangSmith for LLM observability.
https://smith.langchain.com/
"""
import os
from typing import Optional
from praisonai_tools.observability.base import (
BaseObservabilityProvider,
Span,
Trace,
)
from praisonai_tools.observability.config import ObservabilityConfig
from praisonai_tools.observability.manager import ObservabilityManager
class LangSmithProvider(BaseObservabilityProvider):
"""LangSmith observability provider using OpenTelemetry."""
name = "langsmith"
def __init__(self, config: Optional[ObservabilityConfig] = None):
"""Initialize LangSmith provider."""
super().__init__(config)
self._tracer_provider = None
self._tracer = None
def is_available(self) -> bool:
"""Check if required SDKs are available."""
try:
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
return True
except ImportError:
return False
def init(self, **kwargs) -> bool:
"""Initialize LangSmith via OpenTelemetry."""
if not self.is_available():
return False
try:
from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import SimpleSpanProcessor
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
from opentelemetry.sdk.resources import Resource
api_key = kwargs.get("api_key") or os.getenv("LANGSMITH_API_KEY")
project = kwargs.get("project") or os.getenv("LANGSMITH_PROJECT", "default")
endpoint = kwargs.get("endpoint") or os.getenv(
"LANGSMITH_ENDPOINT", "https://api.smith.langchain.com"
)
if not api_key:
return False
# Configure OTLP endpoint for LangSmith
otlp_endpoint = f"{endpoint}/otel/v1/traces"
headers = {
"x-api-key": api_key,
"Langsmith-Project": project,
}
# Get praisonai version for resource attributes
_version = "unknown"
try:
import praisonai_tools
_version = getattr(praisonai_tools, "__version__", "unknown")
except Exception:
pass
# Setup tracer provider with PraisonAI resource branding
resource = Resource.create({
"service.name": "praisonai",
"service.version": _version,
"praisonai.version": _version,
"praisonai.framework": "praisonai",
})
self._tracer_provider = TracerProvider(resource=resource)
self._tracer_provider.add_span_processor(
SimpleSpanProcessor(OTLPSpanExporter(endpoint=otlp_endpoint, headers=headers))
)
trace.set_tracer_provider(self._tracer_provider)
self._tracer = trace.get_tracer("praisonai.observability")
self._initialized = True
return True
except Exception as e:
print(f"LangSmith init failed: {e}")
return False
def shutdown(self) -> None:
"""Shutdown LangSmith provider."""
if self._tracer_provider:
try:
self._tracer_provider.shutdown()
except Exception:
pass
self._initialized = False
def check_connection(self) -> tuple:
"""Check LangSmith connection."""
if not self._initialized:
return False, "LangSmith not initialized"
api_key = os.getenv("LANGSMITH_API_KEY")
if api_key:
return True, "LangSmith API key configured"
return False, "LANGSMITH_API_KEY not set"
# Map SpanKind to LangSmith span kind values
_LANGSMITH_SPAN_KINDS = {
"llm": "llm",
"tool": "tool",
"agent": "chain",
"chain": "chain",
"workflow": "chain",
"retrieval": "retriever",
"custom": "chain",
}
def export_span(self, span: Span) -> bool:
"""Export span to LangSmith via OTel with GenAI semantic conventions."""
if not self._initialized or not self._tracer:
return False
try:
with self._tracer.start_as_current_span(span.name) as otel_span:
# LangSmith-specific span kind
ls_kind = self._LANGSMITH_SPAN_KINDS.get(span.kind.value, "chain")
otel_span.set_attribute("langsmith.span.kind", ls_kind)
# GenAI model attributes
if span.model:
otel_span.set_attribute("gen_ai.request.model", span.model)
otel_span.set_attribute("gen_ai.response.model", span.model)
otel_span.set_attribute("gen_ai.system", "openai")
# GenAI usage metrics
if span.input_tokens:
otel_span.set_attribute("gen_ai.usage.prompt_tokens", span.input_tokens)
if span.output_tokens:
otel_span.set_attribute("gen_ai.usage.completion_tokens", span.output_tokens)
if span.input_tokens or span.output_tokens:
otel_span.set_attribute(
"gen_ai.usage.total_tokens",
(span.input_tokens or 0) + (span.output_tokens or 0),
)
# Tool attributes
if span.tool_name:
otel_span.set_attribute("gen_ai.tool.name", span.tool_name)
otel_span.set_attribute("tool.name", span.tool_name)
# Forward all custom attributes (includes gen_ai.prompt.*, gen_ai.completion.*, etc.)
for key, value in span.attributes.items():
try:
otel_span.set_attribute(key, str(value))
# LangSmith maps langsmith.metadata.* to run metadata
if key.startswith("praisonai."):
otel_span.set_attribute(f"langsmith.metadata.{key}", str(value))
except Exception:
pass
if span.error_message:
from opentelemetry.trace import Status, StatusCode
otel_span.set_status(Status(StatusCode.ERROR, span.error_message))
return True
except Exception:
return False
def export_trace(self, trace_obj: Trace) -> bool:
"""Export trace to LangSmith."""
if not self._initialized:
return False
return True
# Auto-register provider
ObservabilityManager.register_provider("langsmith", LangSmithProvider)
@@ -0,0 +1,89 @@
"""
Langtrace Provider
Integration with Langtrace for LLM observability.
https://langtrace.ai/
"""
import os
from typing import Optional
from praisonai_tools.observability.base import (
BaseObservabilityProvider,
Span,
Trace,
)
from praisonai_tools.observability.config import ObservabilityConfig
from praisonai_tools.observability.manager import ObservabilityManager
class LangtraceProvider(BaseObservabilityProvider):
"""Langtrace observability provider."""
name = "langtrace"
def __init__(self, config: Optional[ObservabilityConfig] = None):
"""Initialize Langtrace provider."""
super().__init__(config)
self._langtrace = None
def is_available(self) -> bool:
"""Check if langtrace SDK is available."""
try:
import importlib.util
return importlib.util.find_spec("langtrace_python_sdk") is not None
except ImportError:
return False
def init(self, **kwargs) -> bool:
"""Initialize Langtrace."""
if not self.is_available():
return False
try:
from langtrace_python_sdk import langtrace
api_key = kwargs.get("api_key") or os.getenv("LANGTRACE_API_KEY")
if api_key:
os.environ["LANGTRACE_API_KEY"] = api_key
langtrace.init()
self._langtrace = langtrace
self._initialized = True
return True
except Exception as e:
print(f"Langtrace init failed: {e}")
return False
def shutdown(self) -> None:
"""Shutdown Langtrace provider."""
self._initialized = False
def check_connection(self) -> tuple:
"""Check Langtrace connection."""
if not self._initialized:
return False, "Langtrace not initialized"
api_key = os.getenv("LANGTRACE_API_KEY")
if api_key:
return True, "Langtrace API key configured"
return False, "LANGTRACE_API_KEY not set"
def export_span(self, span: Span) -> bool:
"""Export span to Langtrace."""
if not self._initialized:
return False
return True
def export_trace(self, trace_obj: Trace) -> bool:
"""Export trace to Langtrace."""
if not self._initialized:
return False
return True
# Auto-register provider
ObservabilityManager.register_provider("langtrace", LangtraceProvider)
@@ -0,0 +1,90 @@
"""
LangWatch Provider
Integration with LangWatch for LLM observability.
https://langwatch.ai/
"""
import os
from typing import Optional
from praisonai_tools.observability.base import (
BaseObservabilityProvider,
Span,
Trace,
)
from praisonai_tools.observability.config import ObservabilityConfig
from praisonai_tools.observability.manager import ObservabilityManager
class LangWatchProvider(BaseObservabilityProvider):
"""LangWatch observability provider."""
name = "langwatch"
def __init__(self, config: Optional[ObservabilityConfig] = None):
"""Initialize LangWatch provider."""
super().__init__(config)
self._langwatch = None
def is_available(self) -> bool:
"""Check if langwatch SDK is available."""
try:
import importlib.util
return importlib.util.find_spec("langwatch") is not None
except ImportError:
return False
def init(self, **kwargs) -> bool:
"""Initialize LangWatch."""
if not self.is_available():
return False
try:
import langwatch
api_key = kwargs.get("api_key") or os.getenv("LANGWATCH_API_KEY")
if api_key:
os.environ["LANGWATCH_API_KEY"] = api_key
instrumentors = kwargs.get("instrumentors", [])
langwatch.setup(instrumentors=instrumentors)
self._langwatch = langwatch
self._initialized = True
return True
except Exception as e:
print(f"LangWatch init failed: {e}")
return False
def shutdown(self) -> None:
"""Shutdown LangWatch provider."""
self._initialized = False
def check_connection(self) -> tuple:
"""Check LangWatch connection."""
if not self._initialized:
return False, "LangWatch not initialized"
api_key = os.getenv("LANGWATCH_API_KEY")
if api_key:
return True, "LangWatch API key configured"
return False, "LANGWATCH_API_KEY not set"
def export_span(self, span: Span) -> bool:
"""Export span to LangWatch."""
if not self._initialized:
return False
return True
def export_trace(self, trace_obj: Trace) -> bool:
"""Export trace to LangWatch."""
if not self._initialized:
return False
return True
# Auto-register provider
ObservabilityManager.register_provider("langwatch", LangWatchProvider)
@@ -0,0 +1,92 @@
"""
Maxim Provider
Integration with Maxim for LLM observability.
https://getmaxim.ai/
"""
import os
from typing import Optional
from praisonai_tools.observability.base import (
BaseObservabilityProvider,
Span,
Trace,
)
from praisonai_tools.observability.config import ObservabilityConfig
from praisonai_tools.observability.manager import ObservabilityManager
class MaximProvider(BaseObservabilityProvider):
"""Maxim observability provider."""
name = "maxim"
def __init__(self, config: Optional[ObservabilityConfig] = None):
"""Initialize Maxim provider."""
super().__init__(config)
self._maxim = None
self._logger = None
def is_available(self) -> bool:
"""Check if maxim SDK is available."""
try:
import importlib.util
return importlib.util.find_spec("maxim") is not None
except ImportError:
return False
def init(self, **kwargs) -> bool:
"""Initialize Maxim."""
if not self.is_available():
return False
try:
from maxim import Maxim
api_key = kwargs.get("api_key") or os.getenv("MAXIM_API_KEY")
repo_id = kwargs.get("repo_id") or os.getenv("MAXIM_LOG_REPO_ID")
if api_key:
os.environ["MAXIM_API_KEY"] = api_key
if repo_id:
os.environ["MAXIM_LOG_REPO_ID"] = repo_id
self._maxim = Maxim()
self._logger = self._maxim.logger()
self._initialized = True
return True
except Exception as e:
print(f"Maxim init failed: {e}")
return False
def shutdown(self) -> None:
"""Shutdown Maxim provider."""
self._initialized = False
def check_connection(self) -> tuple:
"""Check Maxim connection."""
if not self._initialized:
return False, "Maxim not initialized"
api_key = os.getenv("MAXIM_API_KEY")
if api_key:
return True, "Maxim API key configured"
return False, "MAXIM_API_KEY not set"
def export_span(self, span: Span) -> bool:
"""Export span to Maxim."""
if not self._initialized:
return False
return True
def export_trace(self, trace_obj: Trace) -> bool:
"""Export trace to Maxim."""
if not self._initialized:
return False
return True
ObservabilityManager.register_provider("maxim", MaximProvider)
@@ -0,0 +1,89 @@
"""
MLflow Provider
Integration with MLflow for LLM observability.
https://mlflow.org/
"""
import os
from typing import Optional
from praisonai_tools.observability.base import (
BaseObservabilityProvider,
Span,
Trace,
)
from praisonai_tools.observability.config import ObservabilityConfig
from praisonai_tools.observability.manager import ObservabilityManager
class MLflowProvider(BaseObservabilityProvider):
"""MLflow observability provider."""
name = "mlflow"
def __init__(self, config: Optional[ObservabilityConfig] = None):
"""Initialize MLflow provider."""
super().__init__(config)
self._mlflow = None
def is_available(self) -> bool:
"""Check if mlflow SDK is available."""
try:
import importlib.util
return importlib.util.find_spec("mlflow") is not None
except ImportError:
return False
def init(self, **kwargs) -> bool:
"""Initialize MLflow."""
if not self.is_available():
return False
try:
import mlflow
tracking_uri = kwargs.get("tracking_uri") or os.getenv("MLFLOW_TRACKING_URI")
experiment_name = kwargs.get("experiment") or self.config.project_name or "PraisonAI"
if tracking_uri:
mlflow.set_tracking_uri(tracking_uri)
mlflow.set_experiment(experiment_name)
self._mlflow = mlflow
self._initialized = True
return True
except Exception as e:
print(f"MLflow init failed: {e}")
return False
def shutdown(self) -> None:
"""Shutdown MLflow provider."""
self._initialized = False
def check_connection(self) -> tuple:
"""Check MLflow connection."""
if not self._initialized:
return False, "MLflow not initialized"
tracking_uri = os.getenv("MLFLOW_TRACKING_URI")
if tracking_uri:
return True, f"MLflow tracking URI: {tracking_uri}"
return True, "MLflow initialized with default tracking"
def export_span(self, span: Span) -> bool:
"""Export span to MLflow."""
if not self._initialized or not self._mlflow:
return False
return True
def export_trace(self, trace_obj: Trace) -> bool:
"""Export trace to MLflow."""
if not self._initialized:
return False
return True
ObservabilityManager.register_provider("mlflow", MLflowProvider)
@@ -0,0 +1,63 @@
"""
Neatlogs Provider
Integration with Neatlogs for LLM observability.
"""
import os
from typing import Optional
from praisonai_tools.observability.base import (
BaseObservabilityProvider,
Span,
Trace,
)
from praisonai_tools.observability.config import ObservabilityConfig
from praisonai_tools.observability.manager import ObservabilityManager
class NeatlogsProvider(BaseObservabilityProvider):
"""Neatlogs observability provider."""
name = "neatlogs"
def __init__(self, config: Optional[ObservabilityConfig] = None):
"""Initialize Neatlogs provider."""
super().__init__(config)
def is_available(self) -> bool:
"""Check if neatlogs SDK is available."""
try:
import importlib.util
return importlib.util.find_spec("neatlogs") is not None
except ImportError:
return False
def init(self, **kwargs) -> bool:
"""Initialize Neatlogs."""
api_key = kwargs.get("api_key") or os.getenv("NEATLOGS_API_KEY")
if not api_key:
return False
self._initialized = True
return True
def shutdown(self) -> None:
"""Shutdown Neatlogs provider."""
self._initialized = False
def check_connection(self) -> tuple:
"""Check Neatlogs connection."""
if not self._initialized:
return False, "Neatlogs not initialized"
return True, "Neatlogs configured"
def export_span(self, span: Span) -> bool:
"""Export span to Neatlogs."""
return self._initialized
def export_trace(self, trace_obj: Trace) -> bool:
"""Export trace to Neatlogs."""
return self._initialized
ObservabilityManager.register_provider("neatlogs", NeatlogsProvider)
Loaded 100 of 433 files, more files were not shown because too many files have changed in this diff. Show more