Compare commits

...
Author SHA1 Message Date
MervinPraisonandClaude Opus 4.8 ffcb9da9df docs: fix vanilla chat plugin path in BeautifulUI strategy
Correct the fallback reference to templates/frontend/plugins/views/chat.js
(the file lives under views/, not directly under plugins/).

Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-09-23 14:23:22 +00:00
MervinPraisonandClaude Opus 4.8 0746ef5e6c docs: add BeautifulUI integration strategy (fixes #292)
Adds the durable integration strategy for the BeautifulUI epic: vendored
source (no runtime npm dep), adapter layer behind @praisonaiui/react,
phased flag-gated rollout, selective token merge, and lucide-react for
nav icons. Documents risks and links the child issues (#293-#298).

Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-09-23 13:41:43 +00:00
MervinPraisonandCursor 3058886312 chore: bump automation hub to v1.1.10 for merge gate fix
Co-authored-by: Cursor <[email protected]>
2026-09-22 15:30:07 +01:00
Mervin Praison 70d8610733 Merge pull request #291 from MervinPraison/claude/issue-290-20260916
docs: add SECURITY.md with private disclosure policy
2026-09-17 10:37:03 +01:00
MervinPraison 0f60ec41f1 Release v0.3.146 2026-09-17 03:02:10 +00:00
MervinPraisonandClaude Opus 4.8 2797667e04 docs: add SECURITY.md with private disclosure policy (fixes #290)
Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-09-16 16:28:01 +00:00
Mervin Praison f23aacdec9 Merge pull request #289 from MervinPraison/claude/issue-286-20260915
fix: auto-load ~/.praisonai/.env on aiui run so env: channel tokens resolve
2026-09-16 14:43:31 +01:00
MervinPraison 17cd4d9f86 Release v0.3.145 2026-09-16 13:00:52 +00:00
Mervin Praison 43935e4e41 fix: forward extra kwargs through channels chat bridge (fixes #287)
fix: forward extra kwargs through channels chat bridge (fixes #287)
2026-09-16 13:58:08 +01:00
MervinPraisonandClaude Opus 4.8 8dc8b9f49f fix: auto-load ~/.praisonai/.env on aiui run so env: tokens resolve (fixes #286)
Channels configured with env:VAR token references silently resolved to
empty strings when the dashboard was started without the shell exporting
the tokens, so bots showed 'Connected' but never replied. aiui run now
loads the standard PraisonAI .env files (~/.praisonai/.env, then ./.env)
into the process environment at startup, matching the PraisonAI CLI
convention. Existing process vars are never overridden and parsing is
dependency-free (no python-dotenv).

Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-09-15 13:16:37 +00:00
MervinPraisonandClaude Opus 4.8 135b05435c fix: forward extra kwargs through channels chat bridge (fixes #287)
The chat bridge installed over bot._session.chat used a fixed
(agent, user_id, text) signature. When the bot internals invoked the
session with per-conversation keyword arguments (chat_id, platform,
thread_id, ...) the wrapper raised
`TypeError: _wrapped_chat() got an unexpected keyword argument 'chat_id'`
and the bot silently failed to reply across all platforms (Telegram,
Slack, Discord).

Add `*args, **kwargs` to `_wrapped_chat` and forward them untouched to
the original session chat so memory/session continuity is preserved.
Minimal, backward-compatible change; no new params or dependencies.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-09-15 13:11:51 +00:00
MervinPraisonandCursor be502d823f chore(automation): bump hub to v1.1.8 for dual OAuth failover
Uses pick-oauth-token pre-flight probe and TOKEN_B backup.

Co-authored-by: Cursor <[email protected]>
2026-09-15 07:49:29 +01:00
Mervin Praison 9be1266747 Merge pull request #285 from MervinPraison/fix/voice-doctor-mcp-probe-timeout
fix(voice-agent): warn on slow MCP doctor probe instead of failing
2026-09-14 17:45:00 +01:00
MervinPraisonandCursor 8ce13a9fec fix(voice-agent): warn on slow MCP doctor probe instead of failing
Use a 2s timeout for GET /api/mcp/servers and downgrade timeout-style
errors to warn so stdio MCP connect does not fail the whole doctor run.

Follow-up to #273 (example-only; no core SDK changes).

Co-authored-by: Cursor <[email protected]>
2026-09-14 14:51:08 +01:00
Mervin Praison e6a346aa22 Merge pull request #284 from MervinPraison/chore/automation-hub-v1.1.7
chore(automation): bump github-automation-template hub to v1.1.7
2026-09-14 14:22:51 +01:00
MervinPraisonandCursor b7b0cc2109 chore(automation): bump github-automation-template hub to v1.1.7
Fixes FINAL detection and triggerLogins fallback for merge gate auto-merge.

Co-authored-by: Cursor <[email protected]>
2026-09-14 14:22:39 +01:00
MervinPraison dc0d81fb55 Release v0.3.144 2026-09-14 13:19:10 +00:00
Mervin Praison 032f3832a7 Merge pull request #283 from MervinPraison/claude/issue-274-20260914
fix: wire chat run abort (populate _active_runs) — fixes #274
2026-09-14 14:17:05 +01:00
Mervin Praison 0c282f5268 Merge pull request #282 from MervinPraison/claude/issue-276-20260914
fix: prevent MCP subprocess leak on reconnect and failed connect
2026-09-14 14:17:02 +01:00
Mervin Praison 85f50ff9b8 Merge pull request #281 from MervinPraison/claude/issue-277-20260914
fix: make _start_channel_bot idempotent to prevent duplicate bots
2026-09-14 14:16:59 +01:00
Mervin Praison 6b79a346a7 Merge pull request #280 from MervinPraison/claude/issue-275-20260914
fix: preserve channel message metadata in SDKFileDataStore
2026-09-14 14:16:56 +01:00
Mervin Praison f2dec3bc81 Merge pull request #279 from MervinPraison/claude/issue-278-20260914
fix: bound @aiui.on_startup hooks with AIUI_STARTUP_TIMEOUT
2026-09-14 14:16:53 +01:00
Mervin Praison 169c0c2eb0 feat(voice-agent): Phases 1–4 — telephony, call analytics, doctor, and MCP manager (#272)
feat(voice-agent): Phases 1–4 — telephony, call analytics, doctor, and MCP manager
2026-09-14 14:15:23 +01:00
MervinPraisonandClaude Opus 4.8 1de8d054a8 fix: remove stale gateway task on channel bot restart (#277)
_start_channel_bot registers the task in gateway _channel_tasks, but
_stop_channel_bot only cleaned up _channel_bots. For the list-based
gateway registry this left the old cancelled task appended on every
restart, accumulating orphaned task references. Mirror start-time
registration by removing the task from _channel_tasks (dict pop / list
remove) on stop. Adds regression tests for both dict and list forms.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-09-14 10:21:01 +00:00
MervinPraisonandClaude Opus 4.8 9ac4414901 fix: register chat runs in _active_runs so abort works (fixes #274)
_run_and_broadcast never inserted its task into ChatManager._active_runs
and never emitted run_started, so the Stop button, POST /api/chat/abort,
and the chat_abort WS message always returned no_active_run while tokens
were still streaming.

- Register the running task under run_id at run start
- Broadcast run_started up-front so the frontend captures run_id and can
  enable the Stop button before the first token
- Deregister the run in a finally on completion, error, or abort

Existing CancelledError handler already broadcasts run_cancelled once the
task is actually cancelled.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-09-14 09:51:06 +00:00
MervinPraisonandClaude Opus 4.8 5168ea253a fix: disconnect MCP client on reconnect and failed connect to prevent subprocess leak (fixes #276)
Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-09-14 09:48:12 +00:00
MervinPraisonandClaude Opus 4.8 a739ba8a6c fix: make _start_channel_bot idempotent to prevent duplicate bots (fixes #277)
Add a guard at the top of ChannelsFeature._start_channel_bot that stops any
bot already running for the same channel_id before creating a new one. This
prevents duplicate bots (with duplicate event handlers) when a channel is
toggled off/on, re-added, or when autostart races with a manual start.

Also make _stop_channel_bot await the cancelled task so the previous bot is
fully torn down before a restart proceeds, eliminating zombie asyncio tasks.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-09-14 09:47:54 +00:00
MervinPraisonandClaude Opus 4.8 4f8ea88a5c fix: preserve channel message metadata in SDKFileDataStore (fixes #275)
SDKFileDataStore only copied toolCalls/elements into SDK message
metadata on write and only restored those two keys on read, so
channel/voice fields (platform, sender, icon, channel_id) were
silently dropped after reload — despite being persisted correctly by
the SQLAlchemy backend.

Write path now preserves all non-core keys (excluding None), and the
read path restores every persisted metadata key (skipping internal
sentinel keys, without clobbering role/content), matching the
SQLAlchemy backend. Backward compatible: existing toolCalls/elements
round-trips are unchanged and no migration is required.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-09-14 09:47:29 +00:00
MervinPraisonandClaude Opus 4.8 112253013a fix: add AIUI_STARTUP_TIMEOUT to bound startup hooks (fixes #278)
Startup hooks ran with an unbounded await while shutdown hooks were
already protected by AIUI_SHUTDOWN_TIMEOUT. A single hung startup hook
blocked the Starlette lifespan from reaching yield, so /health/live never
responded and the dashboard never loaded.

Mirror the shutdown pattern: wrap startup hooks in asyncio.wait_for with
AIUI_STARTUP_TIMEOUT (default 60s). On timeout, log the error and mark
startup completed so the server still reaches lifespan yield. Shutdown
behaviour is unchanged.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-09-14 09:47:16 +00:00
MervinPraisonandClaude Opus 4.8 ac88e1cab7 fix(mcp): close partially-entered exit stack on connect cancellation/timeout
Store the AsyncExitStack on self._exit_stack before entering the stdio/SSE
contexts. On an asyncio.wait_for timeout the inner connect() is cancelled with
CancelledError (a BaseException), which bypasses the except-Exception cleanup,
so self._exit_stack was never set and MCPClientManager.connect()'s timeout
branch called client.disconnect() as a no-op — leaking the stdio subprocess /
SSE streams. Assigning the stack up front lets disconnect() always close it.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-09-10 18:55:52 +00:00
MervinPraisonandClaude Opus 4.8 14ca101086 fix(mcp): catch asyncio.TimeoutError so connect timeout works on Python 3.10
asyncio.wait_for raises asyncio.TimeoutError, which is only aliased to
the builtin TimeoutError on Python 3.11+. On the 3.10 CI leg the bare
'except TimeoutError' missed it, so the timeout fell through to the
generic handler with an empty last_error and no client.disconnect()
cleanup. Catch asyncio.TimeoutError to restore the helpful message and
subprocess/stream cleanup across all supported Pythons.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-09-10 18:40:48 +00:00
Dhivya-Bharathy 30f046d0fb feat(voice-agent): Phases 2-4, call analytics, doctor, and MCP connect fix 2026-09-10 16:24:20 +05:30
Mervin Praison 597280a6c6 Merge pull request #265 from Dhivya-Bharathy/feat/meeting-agent-phase-2-recall
feat(meeting-agent): Phase 2 Recall.ai live bots (Part A)
2026-09-08 16:39:01 +01:00
MervinPraisonandClaude Opus 4.8 fe954cbd4a fix(meeting-agent): drop leftover hardcoded Recall workspace UUID from part_b test
The personal contributor workspace UUID (33a035c7-...) was still hardcoded
in the test_meeting_agent_part_b.py env fixture, even though the same value
was cleaned to a neutral placeholder everywhere else (config default,
.env.example, smoke_recall.py, and test_meeting_agent_recall.py). Aligned it
with the sibling test's 'test-workspace' placeholder. No assertion depends on
the value, so behavior is unchanged.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-09-08 09:43:30 +00:00
MervinPraisonandClaude Opus 4.8 ab9ef2544d fix(meeting-agent): point Part B tests at renamed meeting_integrations dir
The Part A branch renamed integrations/recall -> meeting_integrations/recall
to avoid a tests/unit/integrations module-name collision, but the Part B test
loaders (test_meeting_agent_part_b.py) still resolved the old integrations/recall
paths, raising FileNotFoundError on CI (py3.10/3.11/3.12). Repoint the 9 _load()
paths at meeting_integrations/recall to match the source layout and the already-
updated test_meeting_agent_recall.py.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-09-08 09:39:03 +00:00
MervinPraisonandCursor e15b3c0718 merge: resolve conflicts with main (Part B + voice-agent merged)
Reconcile meeting-agent Phase A with main after #266/#267: keep
meeting_integrations/ package rename, merge Part B live-transcript and
Calendar V2 routes, and preserve Part A fixes (atomic webhook idempotency,
background task refs, workspace ID placeholders, py3.10 compiler compat).
Update part_b tests to meeting_integrations paths.

Co-authored-by: Cursor <[email protected]>
2026-09-08 10:34:47 +01:00
MervinPraisonandClaude Opus 4.8 12b42e88be fix(meeting-agent): make Recall webhook idempotency check atomic
Replace the non-atomic SELECT-then-INSERT in mark_event_processed with an
atomic INSERT OR IGNORE + rowcount check. Under concurrent duplicate webhook
deliveries (Svix retries/fan-out) both callers could pass the SELECT and the
second INSERT would raise an unhandled IntegrityError on the primary key,
silently dropping that event's processing after a 200 was returned.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-09-07 15:29:26 +00:00
MervinPraisonandClaude Opus 4.8 5e80f11535 fix(meeting-agent): drop hardcoded personal Recall workspace ID from example
The RECALL_WORKSPACE_ID default was a real contributor workspace UUID
committed across the example (config default, README, .env.example, smoke
script, test). It is non-functional — the Recall API key already scopes
requests to a workspace and the value is only echoed back by the
config-status endpoint. Default it to an empty string and replace the
committed UUID with neutral placeholders.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-09-07 15:25:58 +00:00
MervinPraisonandClaude Opus 4.8 84ae1fd0ac fix(meeting-agent): retain webhook background task refs to prevent GC
asyncio only holds weak references to tasks created via create_task, so the
fire-and-forget Recall webhook processing task could be garbage-collected
mid-execution, silently dropping transcript/status updates. Keep strong refs
in a module-level set and discard on completion.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-09-07 15:21:44 +00:00
MervinPraisonandClaude Opus 4.8 b59af41d88 fix(compiler): use timezone.utc for Python 3.10 compatibility
datetime.UTC was added in Python 3.11, so importing it broke the
python (3.10) CI job with ImportError in _format_lastmod. Switch to
timezone.utc which is equivalent and available on all supported
Python versions (3.10+).

Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-09-07 15:17:41 +00:00
MervinPraisonandClaude Opus 4.8 8e5c4d40e2 fix(meeting-agent): resolve CI failures on Phase 2 Recall PR
- Rename example package integrations/ -> meeting_integrations/ to avoid a
  top-level module-name collision with tests/unit/integrations (imported as
  top-level 'integrations' because tests/unit has no __init__.py). The
  collision made 'meeting_integrations.recall' unresolvable once the test
  session imported the tests integrations package first, causing 8 errors +
  3 failures. Updated all absolute imports and test path references.
- Guard praisonai_tools-dependent tests with pytest.importorskip: the meeting
  tools live in the external PraisonAI-Tools package and are not installed in
  this repo's CI, so those tests skip cleanly instead of erroring.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-09-07 15:15:01 +00:00
80 changed files with 5001 additions and 350 deletions

No files matched your search

+2 -2
View File
@@ -3,7 +3,7 @@
"stack": "claude",
"profile": "sdk",
"mode": "hub",
"hubRef": "v1.1.6",
"templateVersion": "1.1.6",
"hubRef": "v1.1.8",
"templateVersion": "1.1.7",
"installedAt": "2026-09-07T10:02:11Z"
}
+4
View File
@@ -4,6 +4,10 @@
module.exports = {
repoFullName: 'MervinPraison/PraisonAIUI',
gitUser: 'MervinPraison',
gitEmail: '[email protected]',
triggerLogins: ['MervinPraison', 'github-actions[bot]'],
allowedTriageBots: ['praisonai-triage-agent[bot]'],
productPathPrefixes: ['src/praisonaiui/', 'src/frontend/', 'tests/'],
sensitivePathPatterns: [
/^\.github\/workflows\//,
+2 -2
View File
@@ -27,7 +27,7 @@ jobs:
permissions:
issues: write
pull-requests: write
uses: MervinPraison/github-automation-template/.github/workflows/[email protected].6
uses: MervinPraison/github-automation-template/.github/workflows/[email protected].10
secrets: inherit
with:
hub_ref: v1.1.6
hub_ref: v1.1.10
+2 -2
View File
@@ -16,8 +16,8 @@ permissions:
jobs:
recover:
uses: MervinPraison/github-automation-template/.github/workflows/[email protected].6
uses: MervinPraison/github-automation-template/.github/workflows/[email protected].10
secrets: inherit
with:
hub_ref: v1.1.6
hub_ref: v1.1.10
pr_number: ${{ github.event.inputs.pr_number || '' }}
+2 -2
View File
@@ -25,8 +25,8 @@ permissions:
jobs:
detect-and-trigger:
uses: MervinPraison/github-automation-template/.github/workflows/[email protected].6
uses: MervinPraison/github-automation-template/.github/workflows/[email protected].10
secrets: inherit
with:
hub_ref: v1.1.6
hub_ref: v1.1.10
pr_number: ${{ github.event.inputs.pr_number || '' }}
+4 -4
View File
@@ -43,9 +43,9 @@ jobs:
id-token: write
steps:
- uses: actions/checkout@v4
- uses: MervinPraison/github-automation-template/.github/actions/[email protected].6
- uses: MervinPraison/github-automation-template/.github/actions/[email protected].10
with:
hub_ref: v1.1.6
hub_ref: v1.1.10
- name: Generate GitHub App token
id: app-token
uses: actions/create-github-app-token@v1
@@ -87,9 +87,9 @@ jobs:
issues: write
actions: read
id-token: write
uses: MervinPraison/github-automation-template/.github/workflows/[email protected].6
uses: MervinPraison/github-automation-template/.github/workflows/[email protected].10
secrets: inherit
with:
hub_ref: v1.1.6
hub_ref: v1.1.10
pr_number: ${{ github.event.inputs.pr_number || github.event.client_payload.pr_number || '' }}
merge_method: ${{ github.event.inputs.merge_method || 'merge' }}
+2 -2
View File
@@ -18,7 +18,7 @@ jobs:
issues: write
actions: read
id-token: write
uses: MervinPraison/github-automation-template/.github/workflows/[email protected].6
uses: MervinPraison/github-automation-template/.github/workflows/[email protected].10
secrets: inherit
with:
hub_ref: v1.1.6
hub_ref: v1.1.10
+2 -2
View File
@@ -25,9 +25,9 @@ jobs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: MervinPraison/github-automation-template/.github/actions/[email protected].6
- uses: MervinPraison/github-automation-template/.github/actions/[email protected].10
with:
hub_ref: v1.1.6
hub_ref: v1.1.10
- name: Detect conflicting PRs and trigger Claude
uses: actions/github-script@v7
with:
+2 -2
View File
@@ -13,7 +13,7 @@ permissions:
jobs:
sync:
uses: MervinPraison/github-automation-template/.github/workflows/[email protected].6
uses: MervinPraison/github-automation-template/.github/workflows/[email protected].10
secrets: inherit
with:
hub_ref: v1.1.6
hub_ref: v1.1.10
+37
View File
@@ -0,0 +1,37 @@
# Security Policy
We take the security of PraisonAIUI seriously and appreciate responsible
disclosure of vulnerabilities.
## Supported Versions
Security fixes are applied to the latest released version of PraisonAIUI.
Please make sure you are on the most recent release before reporting.
## Reporting a Vulnerability
**Please do not open a public issue for security vulnerabilities.**
Report privately through GitHub's **Private Vulnerability Reporting**:
1. Go to the [Security tab](https://github.com/MervinPraison/PraisonAIUI/security)
of this repository.
2. Click **"Report a vulnerability"** to open a private advisory.
3. Include as much detail as you can:
- Affected component(s) and version.
- Steps to reproduce.
- Impact assessment.
- Any suggested fix or mitigation.
If you are unable to use private reporting, you may open a public issue that
contains **no technical specifics** and requests a private channel, and a
maintainer will follow up.
## What to Expect
- We aim to acknowledge new reports within a few business days.
- We will keep you informed as we investigate and work on a fix.
- Once a fix is released, we are happy to credit you in the advisory (unless
you prefer to remain anonymous).
Thank you for helping keep PraisonAIUI and its users safe.
+56
View File
@@ -0,0 +1,56 @@
# BeautifulUI Integration Strategy
> **Epic:** [#292](https://github.com/MervinPraison/PraisonAIUI/issues/292) — BeautifulUI integration strategy for AI-native dashboard UX.
> This is the durable, agreed strategy that the linked child issues execute against.
[BeautifulUI](https://www.beautifului.dev/) (MIT, Shane Levine / Turbo) is a **copy-paste / shadcn-registry** library of React primitives for AI-native interfaces: thinking traces, streaming answers with inline sources, HITL approval cards, tool chips, task rows, context chunks, diff/records tables, prompt bars, and a reference chat harness.
There is **no published npm package** named `beautiful-ui`. It ships only as source consumed via the shadcn registry or manual copy.
## 1. Strategic decision
BeautifulUI is adopted as **vendored source**, not a runtime dependency, and is exposed to dashboard plugins **only through the existing `@praisonaiui/react` package** — never by importing upstream file paths directly. Rollout is **phased behind a feature flag**, keeping the current vanilla dashboard as the fallback until parity is proven.
Upstream stack (Next.js 15 · React 19 · Tailwind v4 · TypeScript) is closely aligned with PraisonAIUI's `src/frontend` (Vite · React 19 · Tailwind v4 · shadcn/Radix), so primitives port with adapter work rather than a runtime bridge.
## 2. Pillars
| # | Pillar | Decision |
|---|--------|----------|
| 1 | **Vendoring model** | Add primitives into `src/frontend` via `npx shadcn add https://www.beautifului.dev/r/<component>.json`, or manual copy from [github.com/slev12397/beautiful-ui](https://github.com/slev12397/beautiful-ui). Pin via lockfile plus a recorded registry URL / git SHA. No `beautiful-ui` npm dependency in production builds. |
| 2 | **Adapter layer** | Map PraisonAIUI chat / approval / trace **protocol events** to stable internal props. Primitives sit behind `@praisonaiui/react` exports; dashboard plugins consume those exports, not upstream paths. See [#294](https://github.com/MervinPraison/PraisonAIUI/issues/294). |
| 3 | **Hybrid rollout** | Mount React **islands** (or expand `@praisonaiui/react`) for high-value surfaces first — chat streaming/thinking/tools, then approvals. Vanilla chat (`templates/frontend/plugins/views/chat.js`) stays as the flag-gated fallback until parity is proven. |
| 4 | **Token strategy** | Merge BeautifulUI foundation CSS **selectively** into the existing shadcn theme generation (`praisonaiui/themes.py`). One design system per page — avoid two competing token sets. |
| 5 | **Icon licence** | Upstream `SidebarNav` uses the paid `@central-icons-react`. Port nav patterns with **lucide-react** (already in PraisonAIUI) instead. |
## 3. Phasing
1. **Spike** — evaluate primitives in the dev environment, confirm React 19 / Tailwind v4 alignment. [#293](https://github.com/MervinPraison/PraisonAIUI/issues/293)
2. **Architecture** — adapter layer + upstream pin policy. [#294](https://github.com/MervinPraison/PraisonAIUI/issues/294)
3. **CI gate** — frontend build + BeautifulUI compatibility check (no frontend build gate exists today). [#295](https://github.com/MervinPraison/PraisonAIUI/issues/295)
4. **Migrations** — chat streaming / thinking / tool chips ([#296](https://github.com/MervinPraison/PraisonAIUI/issues/296)), then the approval card ([#297](https://github.com/MervinPraison/PraisonAIUI/issues/297)).
5. **Docs** — upgrade playbook + MIT attribution. [#298](https://github.com/MervinPraison/PraisonAIUI/issues/298)
## 4. Risks
| Risk | Mitigation |
|------|------------|
| No semver npm package | Vendor the source; pin to a recorded registry URL / git SHA in the lockfile. |
| Upstream breaking registry JSON | Vendoring decouples builds from live registry; re-sync is a deliberate, reviewed step. |
| Icon licence (`@central-icons-react` is paid) | Use lucide-react for ported nav patterns; never vendor the paid package. |
| Two competing design systems on one page | Selective CSS token merge into `themes.py`; adapter-only prop surface. |
| React / Tailwind drift | Upstream (React 19 / Tailwind v4) already matches `src/frontend`; the CI gate catches drift. |
## 5. Out of scope
- Replacing the entire dashboard shell in one PR.
- PyPI packaging of BeautifulUI.
- iframe embedding of beautifului.dev.
- Depending on any unpublished npm package for production builds.
## 6. Acceptance criteria status
- [x] Written integration strategy agreed (vendoring + adapter + phased rollout) — this document.
- [x] Risks documented — section 4.
- [x] Child issues filed and prioritised — [#293](https://github.com/MervinPraison/PraisonAIUI/issues/293)–[#298](https://github.com/MervinPraison/PraisonAIUI/issues/298).
- [x] No requirement to depend on unpublished npm packages for production builds — pillar 1, out of scope.
+3 -1
View File
@@ -13,7 +13,9 @@ MEETING_AGENT_PORT=8000
# ── Recall.ai (EU workspace: Praison) ───────────────────────────────────────
# Region must match dashboard: eu-central-1 (Frankfurt).
RECALL_REGION=eu-central-1
RECALL_WORKSPACE_ID=33a035c7-03c1-4ec9-b752-18cdfbbefc42
# Optional: your Recall workspace ID (shown in the config-status API only; the
# API key already scopes requests to a workspace).
RECALL_WORKSPACE_ID=
# REST API key from Recall dashboard (Developers → API Keys). Server-side only.
RECALL_API_KEY=
+3 -2
View File
@@ -17,9 +17,10 @@ Meeting Bots for live Zoom / Meet / Teams capture.
## Recall.ai workspace
This integration targets the **Praison** workspace in **EU (Frankfurt)**:
This integration targets your Recall workspace in **EU (Frankfurt)** by default:
- Workspace ID: `33a035c7-03c1-4ec9-b752-18cdfbbefc42`
- Workspace ID: set `RECALL_WORKSPACE_ID` to your own (optional — the API key
already scopes requests to a workspace)
- API region: `eu-central-1`
Follow the [Recall onboarding guide](https://docs.recall.ai/docs/agent-quickstarts) via Recall MCP (`recall://guides/onboarding`) for credential and webhook setup.
+5 -5
View File
@@ -102,7 +102,7 @@ _agent: Any | None = None
def _recall_schedule_tool():
from praisonai_tools.tools.decorator import tool
from integrations.recall.service import schedule_recall_bot as _schedule
from meeting_integrations.recall.service import schedule_recall_bot as _schedule
@tool
def schedule_recall_bot(meeting_url: str, title: str = "") -> dict:
@@ -115,7 +115,7 @@ def _recall_schedule_tool():
def _upcoming_meetings_tool():
from praisonai_tools.tools.decorator import tool
from integrations.recall.calendar_service import list_upcoming_calendar_meetings
from meeting_integrations.recall.calendar_service import list_upcoming_calendar_meetings
@tool
def get_upcoming_meetings(hours: int = 24) -> list[dict]:
@@ -194,8 +194,8 @@ def _sync_stale_recall_status(record: dict[str, Any]) -> dict[str, Any]:
if not os.getenv("RECALL_API_KEY"):
return record
try:
from integrations.recall.client import RecallClient
from integrations.recall.config import load_recall_settings
from meeting_integrations.recall.client import RecallClient
from meeting_integrations.recall.config import load_recall_settings
from pipeline import merge_meeting_metadata
bot = RecallClient(load_recall_settings()).get_bot(str(bot_id))
@@ -464,7 +464,7 @@ async def calendar_page():
status_note = "Set RECALL_API_KEY to load calendar status."
if os.getenv("RECALL_API_KEY"):
try:
from integrations.recall.calendar_service import list_upcoming_calendar_meetings
from meeting_integrations.recall.calendar_service import list_upcoming_calendar_meetings
upcoming = list_upcoming_calendar_meetings(hours=48)
status_note = f"{len(upcoming)} upcoming event(s) with video links (48h window)."
@@ -1,13 +0,0 @@
"""Recall.ai Meeting Bot integration for the meeting-agent example."""
from integrations.recall.client import RecallClient
from integrations.recall.config import RecallSettings, load_recall_settings
from integrations.recall.verify import VerificationError, verify_request_from_recall
__all__ = [
"RecallClient",
"RecallSettings",
"VerificationError",
"load_recall_settings",
"verify_request_from_recall",
]
@@ -0,0 +1,13 @@
"""Recall.ai Meeting Bot integration for the meeting-agent example."""
from meeting_integrations.recall.client import RecallClient
from meeting_integrations.recall.config import RecallSettings, load_recall_settings
from meeting_integrations.recall.verify import VerificationError, verify_request_from_recall
__all__ = [
"RecallClient",
"RecallSettings",
"VerificationError",
"load_recall_settings",
"verify_request_from_recall",
]
@@ -6,9 +6,9 @@ import logging
from datetime import datetime, timedelta, timezone
from typing import Any
from integrations.recall.client import RecallAPIError, RecallClient
from integrations.recall.config import RecallSettings
from integrations.recall.store import RecallStore
from meeting_integrations.recall.client import RecallAPIError, RecallClient
from meeting_integrations.recall.config import RecallSettings
from meeting_integrations.recall.store import RecallStore
logger = logging.getLogger(__name__)
@@ -64,7 +64,7 @@ def list_upcoming_calendar_meetings(
client: RecallClient | None = None,
) -> list[dict[str, Any]]:
"""List upcoming Recall calendar events with video links (for UI + agent tool)."""
from integrations.recall.config import load_recall_settings
from meeting_integrations.recall.config import load_recall_settings
settings = settings or load_recall_settings()
client = client or RecallClient(settings)
@@ -10,7 +10,7 @@ from typing import Any
import httpx
from integrations.recall.config import RecallSettings
from meeting_integrations.recall.config import RecallSettings
logger = logging.getLogger(__name__)
@@ -93,9 +93,7 @@ def load_recall_settings(*, require_public_url: bool = False) -> RecallSettings:
api_key=_require("RECALL_API_KEY"),
webhook_verification_secret=_require("RECALL_WEBHOOK_VERIFICATION_SECRET"),
public_api_base_url=public_url,
workspace_id=os.getenv(
"RECALL_WORKSPACE_ID", "33a035c7-03c1-4ec9-b752-18cdfbbefc42"
).strip(),
workspace_id=os.getenv("RECALL_WORKSPACE_ID", "").strip(),
bot_name=os.getenv("RECALL_BOT_NAME", "Praison Meeting Agent").strip()
or "Praison Meeting Agent",
calendar_auto_record=auto_record not in {"0", "false", "no", "off"},
@@ -5,10 +5,10 @@ from __future__ import annotations
import logging
from typing import Any
from integrations.recall.client import RecallAPIError, RecallClient
from integrations.recall.config import RecallSettings
from integrations.recall.store import RecallStore, webhook_event_key
from integrations.recall.transcript import transcript_data_to_line, transcript_download_to_text
from meeting_integrations.recall.client import RecallAPIError, RecallClient
from meeting_integrations.recall.config import RecallSettings
from meeting_integrations.recall.store import RecallStore, webhook_event_key
from meeting_integrations.recall.transcript import transcript_data_to_line, transcript_download_to_text
logger = logging.getLogger(__name__)
@@ -22,7 +22,7 @@ def _speaker_prefix(line: str) -> str:
def _upsert_live_transcript(meeting_id: str, line: str, *, partial: bool = False) -> str:
"""Merge one live caption line; partial updates replace the last same-speaker line."""
from praisonai_tools.tools.meeting_tools import get_meeting
from integrations.recall.live import publish_live_chunk
from meeting_integrations.recall.live import publish_live_chunk
from pipeline import _metadata_lock
# Hold the merge lock across the read too: streaming transcript events are
@@ -120,7 +120,7 @@ def process_recall_webhook(
if event_type == "bot.in_call_recording" and meeting_id:
_merge_metadata(meeting_id, {"live_status": "live", "status": "live", "error": ""})
from integrations.recall.live import set_live_status
from meeting_integrations.recall.live import set_live_status
set_live_status(meeting_id, "live")
return
@@ -139,7 +139,7 @@ def process_recall_webhook(
if event_type == "bot.call_ended" and meeting_id:
_merge_metadata(meeting_id, {"live_status": "ended"})
from integrations.recall.live import set_live_status
from meeting_integrations.recall.live import set_live_status
set_live_status(meeting_id, "ended")
return
@@ -6,8 +6,8 @@ import logging
import re
from typing import Any
from integrations.recall.client import RecallAPIError, RecallClient
from integrations.recall.config import RecallConfigError, RecallSettings, load_recall_settings
from meeting_integrations.recall.client import RecallAPIError, RecallClient
from meeting_integrations.recall.config import RecallConfigError, RecallSettings, load_recall_settings
logger = logging.getLogger(__name__)
@@ -9,7 +9,7 @@ from contextlib import closing
from pathlib import Path
from typing import Any
from integrations.recall.client import RecallClient
from meeting_integrations.recall.client import RecallClient
def _state_db_path() -> str:
@@ -100,42 +100,42 @@ class RecallStore:
@staticmethod
def extract_bot_id(payload: dict[str, Any]) -> str | None:
from integrations.recall.client import RecallClient
from meeting_integrations.recall.client import RecallClient
return RecallClient.extract_bot_id(payload)
@staticmethod
def extract_recording_id(payload: dict[str, Any]) -> str | None:
from integrations.recall.client import RecallClient
from meeting_integrations.recall.client import RecallClient
return RecallClient.extract_recording_id(payload)
@staticmethod
def extract_transcript_id(payload: dict[str, Any]) -> str | None:
from integrations.recall.client import RecallClient
from meeting_integrations.recall.client import RecallClient
return RecallClient.extract_transcript_id(payload)
@staticmethod
def mark_event_processed(event_key: str, event_type: str) -> bool:
"""Return False if this event was already processed."""
"""Claim an event for processing; return False if already processed.
Uses an atomic ``INSERT OR IGNORE`` so concurrent duplicate webhook
deliveries (Svix retries/fan-out) cannot both pass the check — the
second insert is ignored and reports zero changed rows.
"""
from datetime import datetime, timezone
with closing(_connect()) as conn, conn:
row = conn.execute(
"SELECT 1 FROM recall_webhook_events WHERE event_key = ?",
(event_key,),
).fetchone()
if row:
return False
conn.execute(
cursor = conn.execute(
"""
INSERT INTO recall_webhook_events (event_key, event_type, processed_at)
INSERT OR IGNORE INTO recall_webhook_events
(event_key, event_type, processed_at)
VALUES (?, ?, ?)
""",
(event_key, event_type, datetime.now(timezone.utc).isoformat()),
)
return True
return cursor.rowcount > 0
@staticmethod
def save_calendar_setup(setup_id: str, platform: str, state: dict[str, Any]) -> None:
@@ -11,18 +11,18 @@ from typing import Any
from starlette.requests import Request
from starlette.responses import JSONResponse, PlainTextResponse, Response, StreamingResponse
from integrations.recall.calendar import forward_calendar_callback
from integrations.recall.calendar_service import (
from meeting_integrations.recall.calendar import forward_calendar_callback
from meeting_integrations.recall.calendar_service import (
list_upcoming_calendar_meetings,
process_calendar_webhook,
sync_calendar_events,
)
from integrations.recall.config import RecallConfigError, load_recall_settings
from integrations.recall.live import snapshot_for_api, wait_for_update
from integrations.recall.processor import process_recall_webhook
from integrations.recall.service import cancel_recall_bot, schedule_recall_bot
from integrations.recall.store import RecallStore
from integrations.recall.verify import VerificationError, verify_request_from_recall
from meeting_integrations.recall.config import RecallConfigError, load_recall_settings
from meeting_integrations.recall.live import snapshot_for_api, wait_for_update
from meeting_integrations.recall.processor import process_recall_webhook
from meeting_integrations.recall.service import cancel_recall_bot, schedule_recall_bot
from meeting_integrations.recall.store import RecallStore
from meeting_integrations.recall.verify import VerificationError, verify_request_from_recall
logger = logging.getLogger(__name__)
@@ -208,7 +208,7 @@ async def api_recall_calendar_status(_request: Request) -> JSONResponse:
public_url = os.getenv("PUBLIC_API_BASE_URL", "").strip()
calendars: list[dict[str, Any]] = []
try:
from integrations.recall.client import RecallClient
from meeting_integrations.recall.client import RecallClient
settings = load_recall_settings()
calendars = RecallClient(settings).list_calendars(status="connected")
@@ -45,9 +45,9 @@ def _duration_seconds(started_at: str | None, completed_at: str | None) -> int |
def recover(meeting_id: str, bot_id: str | None = None) -> dict:
from integrations.recall.client import RecallClient
from integrations.recall.config import load_recall_settings
from integrations.recall.transcript import transcript_download_to_text
from meeting_integrations.recall.client import RecallClient
from meeting_integrations.recall.config import load_recall_settings
from meeting_integrations.recall.transcript import transcript_download_to_text
from pipeline import merge_meeting_metadata, run_post_transcript_pipeline
from praisonai_tools.tools.meeting_tools import get_meeting
@@ -39,12 +39,12 @@ def run_smoke() -> None:
os.environ.setdefault("RECALL_REGION", "eu-central-1")
os.environ.setdefault("RECALL_API_KEY", "smoke-test-key")
os.environ.setdefault("RECALL_WEBHOOK_VERIFICATION_SECRET", "whsec_" + base64.b64encode(b"smoke-secret-key-32-bytes-long!!").decode())
os.environ.setdefault("RECALL_WORKSPACE_ID", "33a035c7-03c1-4ec9-b752-18cdfbbefc42")
os.environ.setdefault("RECALL_WORKSPACE_ID", "smoke-workspace")
os.environ.setdefault("PUBLIC_API_BASE_URL", "https://smoke.example.test")
import app as meeting_app # noqa: WPS433
from integrations.recall.config import load_recall_settings
from integrations.recall.verify import verify_request_from_recall
from meeting_integrations.recall.config import load_recall_settings
from meeting_integrations.recall.verify import verify_request_from_recall
settings = load_recall_settings()
assert settings.api_key == "smoke-test-key"
+35
View File
@@ -4,13 +4,48 @@ OPENAI_API_KEY=
VOICE_AGENT_PORT=8001
# LLM for voice assistant (synced to provider on setup / start_dev)
VOICE_AGENT_MODEL=gpt-4o-mini
# Voice telephony provider
VOICE_API_BASE=
VOICE_PRIVATE_API_KEY=
VOICE_PUBLIC_API_KEY=
VOICE_SERVER_URL_SECRET=
VOICE_ASSISTANT_ID=
VOICE_PHONE_NUMBER_ID=
# Phase 2 — browser web talk (provider web SDK)
# VOICE_WEB_SDK_URL=https://esm.sh/@vapi-ai/[email protected]
# VOICE_WEB_SDK_GLOBAL=VoiceWebClient
# Dynamic assistant (assistant-request returns inline config + agent tools)
# VOICE_DYNAMIC_ASSISTANT=true
# VOICE_AGENT_EXTRA_TOOLS=meeting_search
# ElevenLabs Speech Engine — OpenAI tool calling on the sidecar LLM path
# VOICE_SPEECH_USE_TOOLS=true
# Cross-call memory — stable browser user id → PraisonAI session reload
# VOICE_MEMORY_ENABLED=true
# VOICE_MEMORY_HISTORY_LIMIT=20
# VOICE_MEMORY_USE_KNOWLEDGE=true
# OpenAI Realtime (browser speech-to-speech — higher audio token cost; keep demos short)
# REALTIME_MODEL=gpt-realtime-2.1-mini
# REALTIME_VOICE=marin
# REALTIME_REASONING_EFFORT=low
# REALTIME_INSTRUCTIONS=You are a helpful voice assistant. Keep replies to one or two short sentences.
# REALTIME_FIRST_MESSAGE=
# ElevenLabs Speech Engine (Phase 3 — browser voice, BYO LLM)
# ELEVENLABS_API_KEY=
# SPEECH_ENGINE_ID= # auto-written by setup_speech_engine.py
# SPEECH_ENGINE_PUBLIC_URL= # auto-written — dedicated tunnel for sidecar :8002
# SPEECH_ENGINE_PORT=8002
# SPEECH_ENGINE_FIRST_MESSAGE=Hello! How can I help you today?
# ELEVENLABS_VOICE_ID= # Praison/custom voice from ElevenLabs dashboard
# Public HTTPS URL for provider webhook (Cloudflare tunnel)
# cloudflared tunnel --url http://127.0.0.1:8001
PUBLIC_API_BASE_URL=
+67 -3
View File
@@ -34,16 +34,80 @@ Or run `py -3.13 setup_voice_provider.py` after `start_dev.ps1`.
| `/api/voice/calls` | POST | Outbound call `{ "customer_number": "+1..." }` |
| `/api/voice/calls` | GET | List stored calls |
| `/api/voice/calls/{id}` | GET | Call detail + transcript |
| `/api/voice/calls/{id}/live-transcript` | GET | SSE live transcript stream |
| `/api/voice/web-config` | GET | Browser talk widget config |
| `/webhooks/voice` | POST | Provider webhooks (tool-calls, transcript, end-of-call) |
| `/api/voice/config` | GET | Config status |
| `/api/voice/doctor` | GET | Voice stack health checks (.env, ports, APIs, tunnel) |
## Phase 2 features
- **Live transcript** — `GET /api/voice/calls/{id}/live-transcript` (SSE); **Call detail** page uses `plugin.js` + EventSource
- **Dynamic assistant** — set `VOICE_DYNAMIC_ASSISTANT=true` so `assistant-request` returns inline config with agent tool names
- **Agent tools** — `tool-calls` run through `integrations/voice/agent_runner.py` (same registry as dashboard chat)
- **Web talk** — **Web talk** page loads provider web SDK from `VOICE_WEB_SDK_URL` / `VOICE_WEB_SDK_GLOBAL`
- **Chat bridge** — final transcript lines and end-of-call summaries appear in **Chat** sidebar as `📞 Call …` sessions
Optional: `VOICE_AGENT_EXTRA_TOOLS=meeting_search` adds meeting search when `praisonai_tools` is installed.
## Cross-call memory (PraisonAI sessions)
Stable browser `voice-user-id` (localStorage) maps to `voice-user-{id}` PraisonAI session folder. Prior turns reload on new calls/restarts when `VOICE_MEMORY_ENABLED=true` (default).
| Route | Purpose |
|-------|---------|
| `/api/voice/memory/attach-user` | ElevenLabs — attach user before conversation starts |
| `/api/voice/memory/bind` | Bind `call_id` + `user_id` |
| `/api/voice/memory/resolve?user_id=` | Resolve Praison session id |
Knowledge base entries are injected into voice prompts when available.
## Call detail UI (PraisonAIUI components)
Call detail uses server-built components (`code_block`, `badge`, `key_value_list`, `tabs`) via `GET /api/voice/calls/{id}/ui`. Live SSE updates the transcript code block. Mic/WebRTC talk pages still use minimal `plugin.js`.
## End-of-call summary & analytics
When a session ends (ElevenLabs, GPT Realtime, or telephony webhook), `integrations/voice/call_finalize.py`:
- Generates a one-sentence summary (OpenAI) when the provider did not send one
- Stores **duration** and **turn count** in call metadata
- Posts the summary to the **Chat** sidebar via the chat bridge
**Voice calls** table shows Duration and Turns. **Config** page runs the voice doctor (`GET /api/voice/doctor` or `py -3.13 doctor.py`).
## MCP manager
The dashboard **MCP** page lists configured Model Context Protocol servers (`GET /api/mcp/servers`) and supports connect/disconnect from the UI.
## Phase 3 — ElevenLabs Speech Engine
Bring-your-own-LLM browser voice ([Speech Engine docs](https://elevenlabs.io/docs/overview/capabilities/speech-engine)):
```powershell
pip install -r requirements-voice.txt
# Add ELEVENLABS_API_KEY to .env
.\start_dev.ps1 -Restart
```
- **ElevenLabs talk** page — WebRTC client, token from `/api/voice/speech-engine/token`
- **Speech Engine sidecar** (`speech_engine_sidecar.py` on port **8002**) — official SDK WebSocket server
- **Dedicated tunnel** — `SPEECH_ENGINE_PUBLIC_URL` → `wss://…/ws` (separate from dashboard tunnel)
- **`setup_speech_engine.py`** — registers the public WebSocket URL with ElevenLabs
| Route | Purpose |
|-------|---------|
| `/api/voice/speech-engine/config` | Client config |
| `/api/voice/speech-engine/token` | Browser conversation token |
| `/ws` | Speech Engine WebSocket (ElevenLabs → your server) |
## Architecture
External platform handles voice pipeline (phone, STT, TTS, turn-taking). This app handles:
- **tool-calls** → PraisonAI-registered tools (`integrations/voice/tools.py`)
- **transcript** / **end-of-call-report** → SQLite store + dashboard pages
- **assistant-request** → returns `VOICE_ASSISTANT_ID`
- **tool-calls** → PraisonAI agent tools (`integrations/voice/agent_runner.py`)
- **transcript** / **end-of-call-report** → SQLite store, SSE hub, chat bridge
- **assistant-request** → default `VOICE_ASSISTANT_ID` or dynamic inline assistant
Same pattern as `examples/python/meeting-agent` (Recall webhooks).
+153 -76
View File
@@ -2,7 +2,7 @@
- Outbound calls → POST /api/voice/calls
- Provider webhooks → POST /webhooks/voice (tool-calls, transcript, end-of-call)
- Dashboard → Calls list + call detail + chat
- Dashboard → Calls list + live call detail + web talk + chat bridge
Run:
cd examples/python/voice-agent
@@ -20,17 +20,40 @@ import sys
from pathlib import Path
from typing import Any
from starlette.routing import Route
from voice_routes import (
api_voice_call_detail,
api_voice_config_status,
api_voice_create_call,
api_voice_list_calls,
webhook_voice,
)
import praisonaiui as aiui
from praisonaiui.server import create_app
from starlette.routing import Route, WebSocketRoute
from memory_routes import (
api_voice_memory_attach_user,
api_voice_memory_bind,
api_voice_memory_config,
api_voice_memory_for_call,
api_voice_memory_resolve,
)
from realtime_routes import (
api_realtime_config,
api_realtime_end,
api_realtime_new_call,
api_realtime_session,
api_realtime_transcript,
)
from speech_engine_routes import (
api_speech_engine_config,
api_speech_engine_token,
speech_engine_websocket,
)
from voice_routes import (
api_voice_call_detail,
api_voice_call_detail_ui,
api_voice_config_status,
api_voice_create_call,
api_voice_doctor,
api_voice_list_calls,
api_voice_live_transcript,
api_voice_web_config,
webhook_voice,
)
_EXAMPLE_DIR = Path(__file__).resolve().parent
if str(_EXAMPLE_DIR) not in sys.path:
@@ -47,68 +70,54 @@ def _load_local_env() -> None:
continue
key, _, value = line.partition("=")
key, value = key.strip(), value.strip()
if key and key not in os.environ:
if key:
os.environ[key] = value
_load_local_env()
aiui.set_pages(["chat", "voice-calls", "call-detail", "config"])
aiui.set_pages(
[
"chat",
"voice-calls",
"call-detail",
"web-talk",
"eleven-talk",
"realtime-talk",
"mcp",
"config",
]
)
aiui.set_style("dashboard")
_agent: Any | None = None
aiui.set_custom_js(_EXAMPLE_DIR / "plugin.js")
def get_agent():
global _agent
if _agent is not None:
return _agent
from integrations.voice.tools import echo_message, get_current_time, register_tool
from praisonaiagents import Agent
from integrations.voice.agent_runner import get_voice_agent
register_tool("get_current_time", get_current_time)
register_tool("echo_message", echo_message)
_agent = Agent(
name="Voice Assistant",
instructions=(
"You help with voice and phone agent demos. "
"Use tools when the caller asks for the time or wants a message echoed."
),
model="gpt-4o-mini",
tools=[get_current_time, echo_message],
)
from praisonaiagents.escalation.loop_guard import LoopGuard, LoopGuardConfig
max_turn_sec = float(os.getenv("VOICE_AGENT_LOOP_GUARD_MAX_SEC", "600"))
_agent._loop_guard = LoopGuard(
LoopGuardConfig(enabled=True, max_time_per_turn=max_turn_sec)
)
return _agent
return get_voice_agent()
def _demo_tool_reply(text: str) -> str | None:
"""Fast path for demo tool queries — no LLM required."""
from integrations.voice.tools import echo_message, get_current_time
aiui.register_agent("voice-assistant", get_agent())
lower = text.lower().strip()
if any(p in lower for p in ("what time", "current time", "tell me the time", "time is it")):
return str(get_current_time()["spoken"])
if lower.startswith("echo "):
return str(echo_message(text[5:])["spoken"])
if lower.startswith("repeat "):
return str(echo_message(text[7:])["spoken"])
return None
@aiui.on_app_startup
async def _start_voice_chat_bridge() -> None:
from integrations.voice.chat_bridge import start_chat_bridge_worker
start_chat_bridge_worker()
@aiui.reply
async def on_message(message: str):
"""Chat with the voice demo agent."""
from integrations.voice.tools import demo_tool_reply
text = str(message).strip()
if not text:
return
demo = _demo_tool_reply(text)
demo = demo_tool_reply(text)
if demo is not None:
await aiui.say(demo)
return
@@ -126,19 +135,25 @@ async def on_message(message: str):
def _list_calls() -> list[dict[str, Any]]:
from integrations.voice.call_finalize import analytics_for_record
from integrations.voice.store import VoiceCallStore
rows = VoiceCallStore.list_calls()
return [
{
"id": r["call_id"],
"status": r["status"],
"customer_number": r.get("customer_number") or "—",
"transcript_preview": (r.get("transcript") or "")[:120],
"updated_at": r.get("updated_at") or "",
}
for r in rows
]
out: list[dict[str, Any]] = []
for r in rows:
analytics = analytics_for_record(r)
out.append(
{
"id": r["call_id"],
"status": r["status"],
"customer_number": r.get("customer_number") or "—",
"transcript_preview": (r.get("transcript") or "")[:120],
"duration": analytics.get("duration") or "—",
"turn_count": analytics.get("turn_count") or 0,
"updated_at": r.get("updated_at") or "",
}
)
return out
def _pick_call(calls: list[dict[str, Any]]) -> dict[str, Any]:
@@ -149,19 +164,22 @@ def _pick_call(calls: list[dict[str, Any]]) -> dict[str, Any]:
@aiui.page("voice-calls", title="Voice calls", icon="📞", group="Voice", order=1)
async def voice_calls_page():
calls = _list_calls()
rows = [[c["id"], c["customer_number"], c["status"], c["updated_at"]] for c in calls]
rows = [
[c["id"], c["customer_number"], c["status"], c["duration"], str(c["turn_count"]), c["updated_at"]]
for c in calls
]
return aiui.layout(
[
aiui.text("Outbound and inbound voice calls via telephony provider"),
aiui.table(
headers=["Call ID", "Customer", "Status", "Updated"],
rows=rows or [["—", "—", "—", "—"]],
headers=["Call ID", "Customer", "Status", "Duration", "Turns", "Updated"],
rows=rows or [["—", "—", "—", "—", "—", "—"]],
),
aiui.alert(
"Start a call: POST /api/voice/calls with {\"customer_number\": \"+1...\"}. "
"Set provider webhook URL to {PUBLIC_API_BASE_URL}/webhooks/voice",
"Open **Call detail** for live transcript (SSE). "
"Voice sessions also appear in **Chat** sidebar as 📞 entries.",
variant="info",
title="Setup",
title="Phase 2",
),
]
)
@@ -169,23 +187,65 @@ async def voice_calls_page():
@aiui.page("call-detail", title="Call detail", icon="🎙️", group="Voice", order=2)
async def call_detail_page():
"""Server fallback — plugin.js registerView overrides with live SSE + UI components."""
from integrations.voice.call_detail_ui import build_call_detail_layout
from integrations.voice.store import VoiceCallStore
calls = _list_calls()
picked = _pick_call(calls)
record = VoiceCallStore.get_call(picked["id"]) if picked.get("id") and picked["id"] != "—" else None
transcript = (record or {}).get("transcript") or "(empty)"
summary = (record or {}).get("summary") or "(not available yet)"
status = (record or {}).get("status") or picked.get("status") or "unknown"
if record:
return build_call_detail_layout(record)
return aiui.layout(
[
aiui.text(f"Call {picked.get('id', '—')}"),
aiui.badge(status, variant="default"),
aiui.tabs(
[
{"label": "Transcript", "children": [aiui.code_block(transcript, language="text")]},
{"label": "Summary", "children": [aiui.text(summary)]},
]
aiui.alert(
"No calls yet. Start from ElevenLabs talk, GPT Realtime, or Web talk.",
variant="info",
title="Call detail",
),
]
)
@aiui.page("web-talk", title="Web talk", icon="🎤", group="Voice", order=3)
async def web_talk_page():
return aiui.layout(
[
aiui.text("Browser voice widget — client view loads from plugin.js"),
aiui.alert(
"Set VOICE_PUBLIC_API_KEY, VOICE_ASSISTANT_ID, VOICE_WEB_SDK_URL, and "
"VOICE_WEB_SDK_GLOBAL in .env.",
variant="info",
title="Web SDK",
),
]
)
@aiui.page("eleven-talk", title="ElevenLabs talk", icon="🗣️", group="Voice", order=4)
async def eleven_talk_page():
return aiui.layout(
[
aiui.text("Browser voice via ElevenLabs Speech Engine — loads from plugin.js"),
aiui.alert(
"Set ELEVENLABS_API_KEY, run setup_speech_engine.py after start_dev.ps1, "
"then click Start conversation.",
variant="info",
title="Speech Engine",
),
]
)
@aiui.page("realtime-talk", title="GPT Realtime", icon="⚡", group="Voice", order=5)
async def realtime_talk_page():
return aiui.layout(
[
aiui.text("Browser voice via OpenAI gpt-realtime-2.1-mini — loads from plugin.js"),
aiui.alert(
"Set OPENAI_API_KEY and REALTIME_MODEL=gpt-realtime-2.1-mini.",
variant="warning",
title="OpenAI Realtime",
),
]
)
@@ -196,8 +256,25 @@ app.routes[0:0] = [
Route("/api/voice/calls", api_voice_create_call, methods=["POST"]),
Route("/api/voice/calls", api_voice_list_calls, methods=["GET"]),
Route("/api/voice/calls/{call_id}", api_voice_call_detail, methods=["GET"]),
Route("/api/voice/calls/{call_id}/ui", api_voice_call_detail_ui, methods=["GET"]),
Route("/api/voice/calls/{call_id}/live-transcript", api_voice_live_transcript, methods=["GET"]),
Route("/api/voice/config", api_voice_config_status, methods=["GET"]),
Route("/api/voice/doctor", api_voice_doctor, methods=["GET"]),
Route("/api/voice/web-config", api_voice_web_config, methods=["GET"]),
Route("/api/voice/speech-engine/config", api_speech_engine_config, methods=["GET"]),
Route("/api/voice/speech-engine/token", api_speech_engine_token, methods=["GET"]),
Route("/api/voice/memory/config", api_voice_memory_config, methods=["GET"]),
Route("/api/voice/memory/bind", api_voice_memory_bind, methods=["POST"]),
Route("/api/voice/memory/attach-user", api_voice_memory_attach_user, methods=["POST"]),
Route("/api/voice/memory/resolve", api_voice_memory_resolve, methods=["GET"]),
Route("/api/voice/memory/for-call", api_voice_memory_for_call, methods=["GET"]),
Route("/api/voice/realtime/config", api_realtime_config, methods=["GET"]),
Route("/api/voice/realtime/new-call", api_realtime_new_call, methods=["POST"]),
Route("/api/voice/realtime/session", api_realtime_session, methods=["POST"]),
Route("/api/voice/realtime/transcript", api_realtime_transcript, methods=["POST"]),
Route("/api/voice/realtime/end", api_realtime_end, methods=["POST"]),
Route("/webhooks/voice", webhook_voice, methods=["POST"]),
WebSocketRoute("/ws", speech_engine_websocket),
]
if __name__ == "__main__":
+41
View File
@@ -0,0 +1,41 @@
"""CLI for voice-agent stack diagnostics."""
from __future__ import annotations
import json
import os
import sys
from pathlib import Path
_EXAMPLE_DIR = Path(__file__).resolve().parent
if str(_EXAMPLE_DIR) not in sys.path:
sys.path.insert(0, str(_EXAMPLE_DIR))
def _load_local_env() -> None:
env_path = _EXAMPLE_DIR / ".env"
if not env_path.is_file():
return
for raw in env_path.read_text(encoding="utf-8").splitlines():
line = raw.strip()
if not line or line.startswith("#") or "=" not in line:
continue
key, _, value = line.partition("=")
key, value = key.strip(), value.strip()
if key:
os.environ[key] = value
def main() -> int:
_load_local_env()
from integrations.voice.doctor import run_voice_doctor
app_port = int(os.getenv("VOICE_AGENT_PORT", "8001"))
sidecar_port = int(os.getenv("SPEECH_ENGINE_SIDECAR_PORT", "8002"))
report = run_voice_doctor(app_port=app_port, sidecar_port=sidecar_port)
print(json.dumps(report, indent=2))
return 1 if report["summary"]["failed"] else 0
if __name__ == "__main__":
raise SystemExit(main())
@@ -0,0 +1,171 @@
"""Run voice tool-calls through the PraisonAI agent tool registry."""
from __future__ import annotations
import json
import os
from typing import Any, Callable
from integrations.voice.tools import execute_tool, register_tool
_agent: Any | None = None
_session_agents: dict[str, Any] = {}
def _tool_name(tool: Any) -> str | None:
for attr in ("name", "__name__"):
value = getattr(tool, attr, None)
if isinstance(value, str) and value.strip():
return value.strip()
fn = getattr(tool, "func", None) or getattr(tool, "function", None)
if fn is not None:
return _tool_name(fn)
return None
def _build_voice_agent(*, session_id: str | None = None):
from praisonaiagents import Agent
from praisonaiagents.config.feature_configs import MemoryConfig
from praisonaiagents.escalation.loop_guard import LoopGuard, LoopGuardConfig
from integrations.voice.session_memory import memory_enabled
from integrations.voice.tools import echo_message, get_current_time
register_tool("get_current_time", get_current_time)
register_tool("echo_message", echo_message)
tools: list[Any] = [get_current_time, echo_message]
extra = os.getenv("VOICE_AGENT_EXTRA_TOOLS", "").strip()
if extra == "meeting_search":
try:
from praisonai_tools.tools.meeting_tools import search_meetings
tools.append(search_meetings)
register_tool("search_meetings", search_meetings)
except ImportError:
pass
memory_cfg = None
if memory_enabled() and session_id:
memory_cfg = MemoryConfig(
history=True,
session_id=session_id,
history_limit=int(os.getenv("VOICE_MEMORY_HISTORY_LIMIT", "20")),
)
agent = Agent(
name="Voice Assistant",
instructions=(
"You help with voice and phone agent demos. "
"Use tools when the caller asks for the time, wants a message echoed, "
"or asks about past meetings."
),
model=os.getenv("VOICE_AGENT_MODEL", "gpt-4o-mini"),
tools=tools,
memory=memory_cfg or False,
)
max_turn_sec = float(os.getenv("VOICE_AGENT_LOOP_GUARD_MAX_SEC", "600"))
agent._loop_guard = LoopGuard(LoopGuardConfig(enabled=True, max_time_per_turn=max_turn_sec))
for tool in tools:
name = _tool_name(tool)
if name:
register_tool(name, _callable_tool(tool))
return agent
def get_voice_agent(session_id: str | None = None):
"""Lazy-load the voice assistant (optionally bound to a PraisonAI session)."""
global _agent
if session_id:
cached = _session_agents.get(session_id)
if cached is not None:
return cached
agent = _build_voice_agent(session_id=session_id)
_session_agents[session_id] = agent
return agent
if _agent is not None:
return _agent
_agent = _build_voice_agent()
return _agent
def _callable_tool(tool: Any) -> Callable[..., Any]:
fn = getattr(tool, "func", None) or getattr(tool, "run", None) or tool
return fn # type: ignore[return-value]
def execute_voice_tool(
name: str,
parameters: dict[str, Any] | None,
*,
session_id: str | None = None,
) -> str:
"""Execute via agent tool registry, then fall back to static registry."""
params = parameters or {}
agent = get_voice_agent(session_id)
for tool in agent.tools or []:
tname = _tool_name(tool)
if tname != name:
continue
fn = _callable_tool(tool)
try:
result = fn(**params)
except TypeError:
try:
result = fn(params)
except Exception as exc: # noqa: BLE001
return json.dumps({"error": str(exc)})
except Exception as exc: # noqa: BLE001
return json.dumps({"error": str(exc)})
if isinstance(result, str):
return result
return json.dumps(result)
return execute_tool(name, params)
def list_registered_tool_names() -> list[str]:
agent = get_voice_agent()
names: list[str] = []
for tool in agent.tools or []:
tname = _tool_name(tool)
if tname:
names.append(tname)
return names
def build_assistant_request_response(settings: Any) -> dict[str, Any]:
"""Return provider assistant config for dynamic assistant-request events."""
if settings.default_assistant_id and not os.getenv("VOICE_DYNAMIC_ASSISTANT", "").lower() in (
"1",
"true",
"yes",
):
return {"assistantId": settings.default_assistant_id}
tool_names = list_registered_tool_names()
assistant: dict[str, Any] = {
"name": "Praison Voice Assistant",
"firstMessage": os.getenv("VOICE_FIRST_MESSAGE", "Hello! How can I help you today?"),
"model": {
"provider": os.getenv("VOICE_MODEL_PROVIDER", "openai"),
"model": os.getenv("VOICE_AGENT_MODEL", "gpt-4o-mini"),
"messages": [
{
"role": "system",
"content": (
"You are a helpful voice assistant. When asked for the time, call get_current_time. "
"When asked to repeat or echo text, call echo_message with the message parameter. "
"Available tools: "
+ ", ".join(tool_names or ["get_current_time", "echo_message"])
),
}
],
},
}
if settings.webhook_url and settings.server_url_secret:
assistant["server"] = {
"url": settings.webhook_url,
"headers": {"X-Voice-Webhook-Secret": settings.server_url_secret},
}
return {"assistant": assistant}
@@ -0,0 +1,57 @@
"""Build PraisonAIUI components for the call detail page."""
from __future__ import annotations
from typing import Any
import praisonaiui as aiui
def build_call_detail_layout(record: dict[str, Any]) -> dict[str, Any]:
"""Return dashboard component tree for call detail (code_block + metadata)."""
from integrations.voice.call_finalize import analytics_for_record
metadata = record.get("metadata") or {}
analytics = analytics_for_record(record)
transcript = (record.get("transcript") or "").strip() or "(empty — waiting for speech…)"
summary = (record.get("summary") or "").strip() or "(not available yet)"
status = str(record.get("status") or "unknown")
kv_items: list[dict[str, str]] = [
{"label": "Call ID", "value": str(record.get("call_id") or "—")},
{"label": "Status", "value": status},
{"label": "Customer", "value": str(record.get("customer_number") or "—")},
]
if metadata.get("user_id"):
kv_items.append({"label": "User ID", "value": str(metadata["user_id"])})
if metadata.get("praison_session_id"):
kv_items.append({"label": "Memory session", "value": str(metadata["praison_session_id"])})
kv_items.append({"label": "Duration", "value": str(analytics.get("duration") or "—")})
kv_items.append({"label": "Turns", "value": str(analytics.get("turn_count") or 0)})
return aiui.layout(
[
aiui.text("Live voice call transcript and metadata"),
aiui.badge(status, variant="default"),
aiui.definition_list(kv_items, title="Call metadata"),
aiui.alert(
"Transcript updates live via SSE. Final lines use PraisonAIUI code blocks.",
variant="info",
title="Live feed",
),
aiui.tabs(
[
{
"label": "Transcript",
"children": [
aiui.code_block(transcript, language="text"),
],
},
{
"label": "Summary",
"children": [aiui.text(summary)],
},
]
),
]
)
@@ -0,0 +1,126 @@
"""End-of-call summary, analytics, and chat bridge on session close."""
from __future__ import annotations
import asyncio
import logging
import os
from datetime import datetime, timezone
from typing import Any
logger = logging.getLogger(__name__)
def count_turns(transcript: str) -> int:
"""Count user/agent lines in stored transcript."""
turns = 0
for line in (transcript or "").splitlines():
text = line.strip()
if text.startswith("user:") or text.startswith("agent:") or text.startswith("assistant:"):
turns += 1
return turns
def duration_seconds(created_at: str | None, ended_at: str | None) -> int | None:
if not created_at or not ended_at:
return None
try:
start = datetime.fromisoformat(created_at.replace("Z", "+00:00"))
end = datetime.fromisoformat(ended_at.replace("Z", "+00:00"))
return max(0, int((end - start).total_seconds()))
except ValueError:
return None
def format_duration(seconds: int | None) -> str:
if seconds is None:
return "—"
if seconds < 60:
return f"{seconds}s"
minutes, rem = divmod(seconds, 60)
return f"{minutes}m {rem}s"
async def generate_call_summary(transcript: str) -> str:
"""One-sentence LLM summary for call detail + chat sidebar."""
text = (transcript or "").strip()
if not text:
return "Empty call — no conversation captured."
api_key = os.getenv("OPENAI_API_KEY", "").strip()
if not api_key:
lines = [ln.strip() for ln in text.splitlines() if ln.strip()]
preview = " · ".join(lines[:3])
return f"Call covered: {preview[:240]}"
from openai import AsyncOpenAI
client = AsyncOpenAI(api_key=api_key)
try:
response = await asyncio.wait_for(
client.chat.completions.create(
model=os.getenv("VOICE_AGENT_MODEL", "gpt-4o-mini"),
messages=[
{
"role": "system",
"content": (
"Summarize this voice call in one concise sentence for a dashboard. "
"Mention main topic and outcome only."
),
},
{"role": "user", "content": text[:6000]},
],
max_tokens=80,
),
timeout=15.0,
)
summary = (response.choices[0].message.content or "").strip()
return summary or "Call completed."
except Exception: # noqa: BLE001
logger.exception("Call summary generation failed")
return "Call completed — summary unavailable."
async def finalize_call(call_id: str, *, status: str) -> None:
"""Persist analytics, optional summary, and notify chat bridge."""
from integrations.voice.chat_bridge import enqueue_call_summary
from integrations.voice.store import VoiceCallStore
record = VoiceCallStore.get_call(call_id)
if not record:
return
transcript = (record.get("transcript") or "").strip()
metadata: dict[str, Any] = dict(record.get("metadata") or {})
ended_at = datetime.now(timezone.utc).isoformat()
metadata["ended_at"] = ended_at
metadata["turn_count"] = count_turns(transcript)
metadata["duration_sec"] = duration_seconds(record.get("created_at"), ended_at)
summary = (record.get("summary") or "").strip()
if not summary and transcript:
summary = await generate_call_summary(transcript)
VoiceCallStore.upsert_call(
call_id,
status=status,
summary=summary or None,
metadata=metadata,
)
enqueue_call_summary(call_id, summary or "(no summary)", status)
def analytics_for_record(record: dict[str, Any]) -> dict[str, Any]:
"""Derived analytics for API + dashboard tables."""
metadata = record.get("metadata") or {}
transcript = record.get("transcript") or ""
turns = metadata.get("turn_count")
if turns is None:
turns = count_turns(transcript)
duration_sec = metadata.get("duration_sec")
if duration_sec is None:
duration_sec = duration_seconds(record.get("created_at"), metadata.get("ended_at"))
return {
"turn_count": turns,
"duration_sec": duration_sec,
"duration": format_duration(duration_sec if isinstance(duration_sec, int) else None),
}
@@ -0,0 +1,150 @@
"""Bridge voice call events into Chat UI sessions (sidebar + history)."""
from __future__ import annotations
import asyncio
import logging
import queue
import threading
from typing import Any
logger = logging.getLogger(__name__)
_bridge_queue: queue.Queue[dict[str, Any]] = queue.Queue(maxsize=512)
_worker_lock = threading.Lock()
_worker_task: asyncio.Task[None] | None = None
def voice_session_id(call_id: str) -> str:
return f"voice-{call_id}"
def enqueue_transcript(call_id: str, role: str, text: str, *, final: bool) -> None:
if not call_id or not text.strip():
return
_enqueue(
{
"kind": "transcript",
"call_id": call_id,
"role": role,
"text": text.strip(),
"final": final,
}
)
def enqueue_call_summary(call_id: str, summary: str, status: str) -> None:
if not call_id:
return
_enqueue(
{
"kind": "summary",
"call_id": call_id,
"summary": summary.strip(),
"status": status,
}
)
def _enqueue(item: dict[str, Any]) -> None:
try:
_bridge_queue.put_nowait(item)
except queue.Full:
logger.warning("Voice chat bridge queue full; dropping event")
async def _ensure_session(call_id: str) -> str:
from praisonaiui.server import _datastore
session_id = voice_session_id(call_id)
existing = await _datastore.get_session(session_id)
if existing is None:
await _datastore.create_session(session_id)
await _datastore.update_session(
session_id,
platform="voice",
icon="📞",
title=f"📞 Call {call_id[:8]}",
)
return session_id
async def _process_item(item: dict[str, Any]) -> None:
from praisonaiui.features.chat import get_chat_manager
from praisonaiui.server import _datastore
call_id = str(item.get("call_id") or "")
if not call_id:
return
session_id = await _ensure_session(call_id)
mgr = get_chat_manager()
if item.get("kind") == "transcript":
role = str(item.get("role") or "unknown")
text = str(item.get("text") or "")
final = bool(item.get("final"))
if not final:
await mgr.broadcast(
session_id,
{
"type": "voice_transcript_partial",
"session_id": session_id,
"call_id": call_id,
"role": role,
"content": text,
},
)
return
content = f"**{role}:** {text}"
await _datastore.add_message(
session_id,
{"role": "user" if role == "user" else "assistant", "content": content, "platform": "voice"},
)
await mgr.broadcast(
session_id,
{
"type": "voice_transcript",
"session_id": session_id,
"call_id": call_id,
"role": role,
"content": text,
},
)
return
if item.get("kind") == "summary":
summary = str(item.get("summary") or "(call ended)")
status = str(item.get("status") or "ended")
content = f"**Call {status}**\n\n{summary}"
await _datastore.add_message(session_id, {"role": "assistant", "content": content, "platform": "voice"})
await mgr.broadcast(
session_id,
{
"type": "voice_call_ended",
"session_id": session_id,
"call_id": call_id,
"status": status,
"summary": summary,
},
)
async def run_chat_bridge_worker() -> None:
while True:
try:
item = await asyncio.to_thread(_bridge_queue.get, True, 1.0)
except queue.Empty:
continue
try:
await _process_item(item)
except Exception as exc: # noqa: BLE001
logger.debug("Voice chat bridge error: %s", exc)
def start_chat_bridge_worker() -> None:
global _worker_task
with _worker_lock:
if _worker_task is not None and not _worker_task.done():
return
loop = asyncio.get_running_loop()
_worker_task = loop.create_task(run_chat_bridge_worker())
@@ -18,6 +18,7 @@ class VoiceSettings:
public_api_base_url: str | None
default_assistant_id: str | None
default_phone_number_id: str | None
public_api_key: str | None
api_base: str
@property
@@ -61,5 +62,13 @@ def load_voice_settings(*, require_key: bool = True) -> VoiceSettings:
public_api_base_url=_env("PUBLIC_API_BASE_URL") or None,
default_assistant_id=_env("VOICE_ASSISTANT_ID") or None,
default_phone_number_id=_env("VOICE_PHONE_NUMBER_ID") or None,
public_api_key=_env("VOICE_PUBLIC_API_KEY") or None,
api_base=api_base.rstrip("/"),
)
def get_web_sdk_settings() -> tuple[str, str]:
"""Browser SDK URL + legacy global name (global unused for ESM imports)."""
url = _env("VOICE_WEB_SDK_URL") or "https://esm.sh/@vapi-ai/[email protected]"
global_name = _env("VOICE_WEB_SDK_GLOBAL") or "Vapi"
return url, global_name
@@ -0,0 +1,119 @@
"""Voice agent stack diagnostics (OpenClaw-style doctor)."""
from __future__ import annotations
import os
import socket
from typing import Any
from urllib.error import HTTPError, URLError
from urllib.request import Request, urlopen
def _tcp_open(host: str, port: int, timeout: float = 2.0) -> bool:
try:
with socket.create_connection((host, port), timeout=timeout):
return True
except OSError:
return False
_MCP_PROBE_TIMEOUT_SEC = 2.0
def _is_slow_mcp_probe(detail: str) -> bool:
lowered = detail.lower()
return any(token in lowered for token in ("timed out", "timeout", "temporarily unavailable"))
def _mcp_servers_probe(app_port: int) -> tuple[str, str]:
"""Probe MCP list endpoint; slow stdio connect must not fail the whole doctor run."""
status, detail = _http_status(
f"http://127.0.0.1:{app_port}/api/mcp/servers",
timeout=_MCP_PROBE_TIMEOUT_SEC,
)
if status == "fail" and _is_slow_mcp_probe(detail):
return "warn", detail
return status, detail
def _http_status(url: str, timeout: float = 5.0) -> tuple[str, str]:
try:
req = Request(url, headers={"User-Agent": "praisonai-voice-doctor"})
with urlopen(req, timeout=timeout) as resp:
code = resp.getcode()
if 200 <= code < 500:
return "pass", f"HTTP {code}"
return "warn", f"HTTP {code}"
except HTTPError as exc:
if 200 <= exc.code < 500:
return "pass", f"HTTP {exc.code}"
return "fail", f"HTTP {exc.code}"
except URLError as exc:
return "fail", str(exc.reason)
except Exception as exc: # noqa: BLE001
return "fail", str(exc)
def _env_check(name: str, *, required: bool = False) -> dict[str, Any]:
value = os.getenv(name, "").strip()
if value:
masked = value[:4] + "…" if len(value) > 8 else "set"
return {"name": name, "status": "pass", "detail": masked}
if required:
return {"name": name, "status": "fail", "detail": "missing"}
return {"name": name, "status": "warn", "detail": "not set"}
def run_voice_doctor(*, app_port: int = 8001, sidecar_port: int = 8002) -> dict[str, Any]:
"""Run local voice-agent health checks."""
checks: list[dict[str, Any]] = []
checks.append(_env_check("OPENAI_API_KEY", required=True))
checks.append(_env_check("ELEVENLABS_API_KEY"))
checks.append(_env_check("SPEECH_ENGINE_ID"))
checks.append(_env_check("SPEECH_ENGINE_PUBLIC_URL"))
checks.append(_env_check("PUBLIC_API_BASE_URL"))
checks.append(_env_check("VOICE_PRIVATE_API_KEY"))
if _tcp_open("127.0.0.1", app_port):
checks.append({"name": f"App port {app_port}", "status": "pass", "detail": "listening"})
status, detail = _http_status(f"http://127.0.0.1:{app_port}/api/voice/config")
checks.append({"name": "Voice config API", "status": status, "detail": detail})
status, detail = _http_status(f"http://127.0.0.1:{app_port}/api/voice/speech-engine/config")
checks.append({"name": "Speech Engine config", "status": status, "detail": detail})
status, detail = _http_status(f"http://127.0.0.1:{app_port}/api/voice/realtime/config")
checks.append({"name": "GPT Realtime config", "status": status, "detail": detail})
status, detail = _mcp_servers_probe(app_port)
checks.append({"name": "MCP servers API", "status": status, "detail": detail})
else:
checks.append(
{
"name": f"App port {app_port}",
"status": "fail",
"detail": "not listening — run .\\start_dev.ps1",
}
)
if _tcp_open("127.0.0.1", sidecar_port):
checks.append({"name": f"Speech sidecar {sidecar_port}", "status": "pass", "detail": "listening"})
else:
checks.append(
{
"name": f"Speech sidecar {sidecar_port}",
"status": "warn",
"detail": "not listening (ElevenLabs path needs sidecar)",
}
)
speech_url = os.getenv("SPEECH_ENGINE_PUBLIC_URL", "").strip().rstrip("/")
if speech_url:
status, detail = _http_status(speech_url)
checks.append({"name": "Speech Engine tunnel", "status": status, "detail": detail})
passed = sum(1 for c in checks if c["status"] == "pass")
warnings = sum(1 for c in checks if c["status"] == "warn")
failed = sum(1 for c in checks if c["status"] == "fail")
return {
"checks": checks,
"summary": {"passed": passed, "warnings": warnings, "failed": failed},
}
@@ -0,0 +1,51 @@
"""Live transcript pub/sub for voice calls (SSE)."""
from __future__ import annotations
import asyncio
import json
import queue
import threading
from collections import defaultdict
from typing import Any, AsyncIterator
class TranscriptHub:
"""Thread-safe hub broadcasting transcript events per call_id."""
def __init__(self) -> None:
self._queues: dict[str, list[queue.Queue[dict[str, Any]]]] = defaultdict(list)
self._lock = threading.Lock()
def publish(self, call_id: str, event: dict[str, Any]) -> None:
if not call_id:
return
with self._lock:
queues = list(self._queues.get(call_id, []))
for sub in queues:
try:
sub.put_nowait(event)
except queue.Full:
pass
async def subscribe(self, call_id: str) -> AsyncIterator[dict[str, Any]]:
sub: queue.Queue[dict[str, Any]] = queue.Queue(maxsize=256)
with self._lock:
self._queues[call_id].append(sub)
try:
while True:
try:
event = await asyncio.to_thread(sub.get, True, 1.0)
except queue.Empty:
continue
yield event
finally:
with self._lock:
if call_id in self._queues and sub in self._queues[call_id]:
self._queues[call_id].remove(sub)
def snapshot_sse(self, event: dict[str, Any]) -> str:
return f"data: {json.dumps(event, ensure_ascii=False)}\n\n"
transcript_hub = TranscriptHub()
@@ -2,13 +2,15 @@
from __future__ import annotations
import asyncio
import json
import logging
from typing import Any
from integrations.voice.agent_runner import build_assistant_request_response, execute_voice_tool
from integrations.voice.config import VoiceSettings
from integrations.voice.live import transcript_hub
from integrations.voice.store import VoiceCallStore, webhook_event_key
from integrations.voice.tools import execute_tool
logger = logging.getLogger(__name__)
@@ -35,6 +37,34 @@ def _customer_number(payload: dict[str, Any]) -> str | None:
return None
def _tool_parameters(item: dict[str, Any]) -> dict[str, Any]:
"""Extract tool params from provider payloads (parameters or arguments)."""
for key in ("parameters", "arguments"):
value = item.get(key)
if isinstance(value, dict):
return value
fn = item.get("function")
if isinstance(fn, dict):
for key in ("parameters", "arguments"):
value = fn.get(key)
if isinstance(value, dict):
return value
return {}
def _spoken_tool_result(raw: str) -> str:
"""Return plain spoken text for the voice pipeline (no newlines)."""
try:
data = json.loads(raw)
if isinstance(data, dict):
spoken = data.get("spoken") or data.get("message")
if spoken:
return str(spoken).replace("\n", " ").strip()
except json.JSONDecodeError:
pass
return str(raw).replace("\n", " ").strip()
def process_voice_webhook(
event_type: str, payload: dict[str, Any], *, settings: VoiceSettings
) -> dict[str, Any] | None:
@@ -49,21 +79,36 @@ def process_voice_webhook(
call_id = _call_id(payload)
if event_type == "assistant-request":
assistant_id = settings.default_assistant_id
if not assistant_id:
return {"error": "No assistant configured on server"}
return {"assistantId": assistant_id}
return build_assistant_request_response(settings)
if event_type == "tool-calls" and call_id:
tool_calls = msg.get("toolCallList") or []
if not tool_calls and isinstance(msg.get("toolWithToolCallList"), list):
for entry in msg["toolWithToolCallList"]:
if not isinstance(entry, dict):
continue
tool_call = entry.get("toolCall")
if isinstance(tool_call, dict):
tool_calls.append(
{
"id": tool_call.get("id"),
"name": entry.get("name") or (tool_call.get("function") or {}).get("name"),
"parameters": _tool_parameters(tool_call),
"arguments": _tool_parameters(tool_call),
}
)
results = []
for item in tool_calls:
if not isinstance(item, dict):
continue
tool_call_id = str(item.get("id") or "")
name = str(item.get("name") or "")
params = item.get("parameters") if isinstance(item.get("parameters"), dict) else {}
result = execute_tool(name, params)
if not name:
fn = item.get("function")
if isinstance(fn, dict):
name = str(fn.get("name") or "")
params = _tool_parameters(item)
result = _spoken_tool_result(execute_voice_tool(name, params))
results.append({"toolCallId": tool_call_id, "name": name, "result": result})
VoiceCallStore.upsert_call(call_id, status="in-progress", customer_number=_customer_number(payload))
return {"results": results}
@@ -76,8 +121,25 @@ def process_voice_webhook(
if event_type == "transcript" and call_id:
role = str(msg.get("role") or "unknown")
text = str(msg.get("transcript") or "").strip()
if text and str(msg.get("transcriptType") or "final") == "final":
VoiceCallStore.append_transcript_line(call_id, f"{role}: {text}")
kind = str(msg.get("transcriptType") or "final")
if text:
line = f"{role}: {text}"
if kind == "final":
VoiceCallStore.append_transcript_line(call_id, line)
else:
VoiceCallStore.upsert_call(
call_id,
status="in-progress",
customer_number=_customer_number(payload),
metadata={"partial_transcript": line},
)
transcript_hub.publish(
call_id,
{"type": "transcript", "role": role, "text": text, "transcriptType": kind},
)
from integrations.voice.chat_bridge import enqueue_transcript
enqueue_transcript(call_id, role, text, final=kind == "final")
return None
if event_type == "end-of-call-report" and call_id:
@@ -93,6 +155,9 @@ def process_voice_webhook(
summary=summary or None,
metadata={"artifact": artifact},
)
from integrations.voice.call_finalize import finalize_call
asyncio.run(finalize_call(call_id, status=f"ended ({ended})"))
return None
if event_type == "conversation-update" and call_id:
@@ -0,0 +1,180 @@
"""OpenAI Realtime API — browser WebRTC voice (gpt-realtime-2.1-mini)."""
from __future__ import annotations
import json
import logging
import os
import uuid
from dataclasses import dataclass
from typing import Any
import httpx
logger = logging.getLogger(__name__)
OPENAI_REALTIME_CALLS_URL = "https://api.openai.com/v1/realtime/calls"
OPENAI_CLIENT_SECRETS_URL = "https://api.openai.com/v1/realtime/client_secrets"
class RealtimeConfigError(Exception):
pass
@dataclass(frozen=True)
class RealtimeSettings:
api_key: str
model: str
voice: str
reasoning_effort: str
instructions: str
first_message: str | None
def load_realtime_settings() -> RealtimeSettings:
api_key = os.getenv("OPENAI_API_KEY", "").strip()
if not api_key:
raise RealtimeConfigError("Set OPENAI_API_KEY in .env")
first_message = os.getenv("REALTIME_FIRST_MESSAGE", "").strip() or None
return RealtimeSettings(
api_key=api_key,
model=os.getenv("REALTIME_MODEL", "gpt-realtime-2.1-mini").strip(),
voice=os.getenv("REALTIME_VOICE", "marin").strip(),
reasoning_effort=os.getenv("REALTIME_REASONING_EFFORT", "low").strip(),
instructions=os.getenv(
"REALTIME_INSTRUCTIONS",
"You are a helpful voice assistant for PraisonAI demos. "
"Keep every reply to one or two short sentences.",
).strip(),
first_message=first_message,
)
def realtime_enabled() -> bool:
try:
load_realtime_settings()
except RealtimeConfigError:
return False
return True
def build_session_config(
settings: RealtimeSettings | None = None,
*,
praison_session_id: str | None = None,
) -> dict[str, Any]:
from integrations.voice.session_memory import instructions_with_memory
cfg = settings or load_realtime_settings()
session: dict[str, Any] = {
"type": "realtime",
"model": cfg.model,
"instructions": instructions_with_memory(cfg.instructions, session_id=praison_session_id),
"audio": {"output": {"voice": cfg.voice}},
# Required for user transcript events in the browser data channel.
"input_audio_transcription": {
"model": os.getenv("REALTIME_TRANSCRIBE_MODEL", "gpt-4o-mini-transcribe"),
},
}
if cfg.reasoning_effort:
session["reasoning"] = {"effort": cfg.reasoning_effort}
return session
def new_call_id() -> str:
return f"rt-{uuid.uuid4().hex[:12]}"
async def create_webrtc_session(sdp_offer: str, *, call_id: str | None = None) -> str:
"""Relay browser SDP to OpenAI unified Realtime calls endpoint."""
from integrations.voice.session_memory import resolve_praison_session_id
settings = load_realtime_settings()
praison_session_id = resolve_praison_session_id(call_id)
session_json = json.dumps(build_session_config(settings, praison_session_id=praison_session_id))
files = {
"sdp": (None, sdp_offer, "application/sdp"),
"session": (None, session_json, "application/json"),
}
async with httpx.AsyncClient(timeout=30.0) as client:
response = await client.post(
OPENAI_REALTIME_CALLS_URL,
headers={"Authorization": f"Bearer {settings.api_key}"},
files=files,
)
if response.status_code >= 400:
detail = response.text[:500]
logger.error("OpenAI Realtime session failed (%s): %s", response.status_code, detail)
raise RealtimeConfigError(f"OpenAI Realtime session failed: HTTP {response.status_code}")
return response.text
async def mint_client_secret() -> str:
"""Mint ephemeral key (alternative to unified SDP relay)."""
settings = load_realtime_settings()
payload = {"session": build_session_config(settings)}
async with httpx.AsyncClient(timeout=30.0) as client:
response = await client.post(
OPENAI_CLIENT_SECRETS_URL,
headers={
"Authorization": f"Bearer {settings.api_key}",
"Content-Type": "application/json",
},
json=payload,
)
if response.status_code >= 400:
detail = response.text[:500]
raise RealtimeConfigError(f"OpenAI client secret failed: HTTP {response.status_code} — {detail}")
data = response.json()
token = data.get("value") or data.get("client_secret", {}).get("value")
if not token:
raise RealtimeConfigError("OpenAI client secret response missing value")
return str(token)
def persist_transcript_line(call_id: str, role: str, text: str) -> None:
from integrations.voice.live import transcript_hub
from integrations.voice.store import VoiceCallStore
text = text.strip()
if not call_id or not text:
return
record = VoiceCallStore.get_call(call_id) or {}
prefix = f"{role}: {text}"
existing = (record.get("transcript") or "").strip()
merged = f"{existing}\n{prefix}".strip() if existing else prefix
VoiceCallStore.upsert_call(
call_id,
status=record.get("status") or "in-progress",
customer_number="web-openai-realtime",
transcript=merged,
)
transcript_hub.publish(
call_id,
{
"type": "transcript",
"role": role,
"transcriptType": "final",
"text": text,
},
)
def mark_session_started(call_id: str) -> None:
from integrations.voice.store import VoiceCallStore
VoiceCallStore.upsert_call(
call_id,
status="in-progress",
customer_number="web-openai-realtime",
)
def mark_session_ended(call_id: str) -> None:
from integrations.voice.store import VoiceCallStore
VoiceCallStore.upsert_call(
call_id,
status="ended (customer-ended-call)",
customer_number="web-openai-realtime",
)
@@ -0,0 +1,251 @@
"""Cross-call voice memory via PraisonAI session reload by stable user ID."""
from __future__ import annotations
import logging
import os
from typing import Any
logger = logging.getLogger(__name__)
_CALL_USER: dict[str, str] = {}
def memory_enabled() -> bool:
return os.getenv("VOICE_MEMORY_ENABLED", "true").lower() in ("1", "true", "yes")
def voice_user_session_id(user_id: str) -> str:
clean = str(user_id or "").strip()
if not clean:
raise ValueError("user_id required")
return f"voice-user-{clean}"
def attach_pending_user(user_id: str) -> None:
from integrations.voice.store import VoiceCallStore
clean = str(user_id or "").strip()
if clean:
VoiceCallStore.set_pending_user(clean)
def consume_pending_user() -> str | None:
from integrations.voice.store import VoiceCallStore
return VoiceCallStore.consume_pending_user()
def bind_call_user(call_id: str, user_id: str) -> str:
"""Associate a provider call/conversation id with a stable browser user id."""
from integrations.voice.store import VoiceCallStore
call_id = str(call_id or "").strip()
user_id = str(user_id or "").strip()
if not call_id or not user_id:
raise ValueError("call_id and user_id required")
session_id = voice_user_session_id(user_id)
_CALL_USER[call_id] = user_id
VoiceCallStore.upsert_call(
call_id,
metadata={"user_id": user_id, "praison_session_id": session_id},
)
return session_id
def resolve_call_user(call_id: str | None) -> str | None:
if not call_id:
return None
if call_id in _CALL_USER:
return _CALL_USER[call_id]
from integrations.voice.store import VoiceCallStore
record = VoiceCallStore.get_call(call_id)
if not record:
return None
user_id = (record.get("metadata") or {}).get("user_id")
if user_id:
_CALL_USER[call_id] = str(user_id)
return str(user_id) if user_id else None
def resolve_praison_session_id(call_id: str | None) -> str | None:
user_id = resolve_call_user(call_id)
if not user_id:
return None
return voice_user_session_id(user_id)
def _history_limit() -> int:
try:
return max(1, int(os.getenv("VOICE_MEMORY_HISTORY_LIMIT", "20")))
except ValueError:
return 20
def load_session_history(session_id: str) -> list[dict[str, str]]:
"""Load prior turns from PraisonAI DefaultSessionStore."""
try:
from praisonaiagents.session.store import DefaultSessionStore
store = DefaultSessionStore()
return store.get_chat_history(session_id, max_messages=_history_limit())
except Exception: # noqa: BLE001
logger.debug("Failed to load session history for %s", session_id, exc_info=True)
return []
def append_session_turn(session_id: str, role: str, content: str) -> None:
"""Persist one voice turn into the PraisonAI session folder."""
text = str(content or "").strip()
if not session_id or not text:
return
try:
from praisonaiagents.session.store import DefaultSessionStore
store = DefaultSessionStore()
if role == "user":
store.add_user_message(session_id, text, metadata={"platform": "voice"})
else:
store.add_assistant_message(session_id, text, metadata={"platform": "voice"})
except Exception: # noqa: BLE001
logger.debug("Failed to append session turn for %s", session_id, exc_info=True)
async def ensure_ui_session(session_id: str, *, user_id: str | None = None, call_id: str | None = None) -> None:
"""Ensure PraisonAIUI datastore session exists for sidebar reload."""
try:
from praisonaiui.server import _datastore
except Exception: # noqa: BLE001
return
existing = await _datastore.get_session(session_id)
title = "Voice memory"
if call_id:
title = f"Voice user {user_id or call_id[:8]}"
if existing is None:
await _datastore.create_session(session_id)
await _datastore.update_session(
session_id,
platform="voice",
icon="🎙️",
title=title,
)
async def append_ui_message(session_id: str, role: str, content: str) -> None:
try:
from praisonaiui.server import _datastore
await _datastore.add_message(
session_id,
{"role": "user" if role == "user" else "assistant", "content": content, "platform": "voice"},
)
except Exception: # noqa: BLE001
logger.debug("Failed to append UI message for %s", session_id, exc_info=True)
def _apply_knowledge_augmentation(prompt: str) -> str:
from praisonaiui.features.knowledge import get_knowledge_manager
k_mgr = get_knowledge_manager()
if not k_mgr.list_all():
return prompt
results = k_mgr.search(prompt, limit=5)
lines = [r.get("text", "") for r in results if r.get("text")]
if not lines:
return prompt
block = "\n".join(lines)
return f"[Knowledge Context]\n{block}\n[/Knowledge Context]\n\n{prompt}"
def build_knowledge_augmented_prompt(prompt: str, *, timeout_s: float | None = 2.0) -> str:
"""Inject knowledge search results like dashboard chat run_agent().
Knowledge indexing can take tens of seconds on cold start; Speech Engine
disconnects if the LLM path blocks too long, so callers use a short timeout.
"""
if not os.getenv("VOICE_MEMORY_USE_KNOWLEDGE", "true").lower() in ("1", "true", "yes"):
return prompt
try:
if timeout_s is None:
return _apply_knowledge_augmentation(prompt)
from concurrent.futures import ThreadPoolExecutor, TimeoutError as FuturesTimeoutError
with ThreadPoolExecutor(max_workers=1) as pool:
future = pool.submit(_apply_knowledge_augmentation, prompt)
return future.result(timeout=timeout_s)
except FuturesTimeoutError:
logger.warning("Knowledge augmentation timed out after %.1fs; using plain prompt", timeout_s)
return prompt
except Exception: # noqa: BLE001
logger.debug("Knowledge augmentation skipped", exc_info=True)
return prompt
def history_context_block(session_id: str | None) -> str:
if not memory_enabled() or not session_id:
return ""
history = load_session_history(session_id)
if not history:
return ""
lines = []
for item in history[-_history_limit() :]:
role = item.get("role", "user")
content = str(item.get("content") or "").strip()
if content:
lines.append(f"{role}: {content}")
if not lines:
return ""
return "Previous conversation with this user:\n" + "\n".join(lines)
def openai_messages_with_memory(
transcript_lines: list[str],
*,
session_id: str | None,
base_instructions: str,
) -> list[dict[str, str]]:
messages: list[dict[str, str]] = [{"role": "system", "content": base_instructions}]
prior = history_context_block(session_id)
if prior:
messages.append({"role": "system", "content": prior})
for line in transcript_lines:
if line.startswith("user:"):
messages.append({"role": "user", "content": line[5:].strip()})
elif line.startswith("agent:"):
messages.append({"role": "assistant", "content": line[6:].strip()})
return messages
def bind_call_from_pending(call_id: str) -> str | None:
user_id = consume_pending_user()
if not user_id:
return None
return bind_call_user(call_id, user_id)
async def record_voice_turn(
call_id: str | None,
*,
user_text: str | None = None,
agent_text: str | None = None,
) -> None:
"""Save latest user/agent lines to Praison session + chat UI."""
session_id = resolve_praison_session_id(call_id)
if not session_id:
return
user_id = resolve_call_user(call_id)
await ensure_ui_session(session_id, user_id=user_id, call_id=call_id)
if user_text:
append_session_turn(session_id, "user", user_text)
await append_ui_message(session_id, "user", user_text)
if agent_text:
append_session_turn(session_id, "assistant", agent_text)
await append_ui_message(session_id, "assistant", agent_text)
def instructions_with_memory(base: str, *, session_id: str | None) -> str:
prior = history_context_block(session_id)
if not prior:
return base
return f"{base}\n\n{prior}\nUse prior conversation when the user refers to earlier calls."
@@ -0,0 +1,346 @@
"""ElevenLabs Speech Engine — WebSocket LLM bridge for browser voice."""
from __future__ import annotations
import asyncio
import logging
import os
from dataclasses import dataclass
from typing import Any
logger = logging.getLogger(__name__)
_engine_resource: Any | None = None
_engine_lock = asyncio.Lock()
class SpeechEngineConfigError(Exception):
pass
@dataclass(frozen=True)
class SpeechEngineSettings:
api_key: str
engine_id: str | None
public_ws_url: str | None
first_message: str
def _public_ws_url() -> str | None:
speech_base = os.getenv("SPEECH_ENGINE_PUBLIC_URL", "").strip().rstrip("/")
base = speech_base or os.getenv("PUBLIC_API_BASE_URL", "").strip().rstrip("/")
if not base:
return None
if base.startswith("https://"):
return "wss://" + base[len("https://") :] + "/ws"
if base.startswith("http://"):
return "ws://" + base[len("http://") :] + "/ws"
return None
def load_speech_engine_settings(*, require_engine_id: bool = False) -> SpeechEngineSettings:
api_key = os.getenv("ELEVENLABS_API_KEY", "").strip()
if not api_key:
raise SpeechEngineConfigError("Set ELEVENLABS_API_KEY in .env")
engine_id = os.getenv("SPEECH_ENGINE_ID", "").strip() or None
if require_engine_id and not engine_id:
raise SpeechEngineConfigError("Set SPEECH_ENGINE_ID in .env (run setup_speech_engine.py)")
return SpeechEngineSettings(
api_key=api_key,
engine_id=engine_id,
public_ws_url=_public_ws_url(),
first_message=os.getenv(
"SPEECH_ENGINE_FIRST_MESSAGE", "Hello! How can I help you today?"
).strip(),
)
def speech_engine_enabled() -> bool:
try:
settings = load_speech_engine_settings()
except SpeechEngineConfigError:
return False
return bool(settings.engine_id and settings.public_ws_url)
async def get_engine_resource():
"""Lazy-load the ElevenLabs SpeechEngineResource."""
global _engine_resource
if _engine_resource is not None:
return _engine_resource
async with _engine_lock:
if _engine_resource is not None:
return _engine_resource
settings = load_speech_engine_settings(require_engine_id=True)
try:
from elevenlabs import AsyncElevenLabs
except ImportError as exc:
raise SpeechEngineConfigError(
"Install Speech Engine SDK: pip install 'elevenlabs>=2.47.0'"
) from exc
if not hasattr(AsyncElevenLabs(api_key=settings.api_key), "speech_engine"):
raise SpeechEngineConfigError(
"elevenlabs package is too old. Run: pip install -U 'elevenlabs>=2.47.0'"
)
client = AsyncElevenLabs(api_key=settings.api_key)
_engine_resource = await client.speech_engine.get(settings.engine_id)
return _engine_resource
def _conversation_id(session: Any) -> str | None:
cid = getattr(session, "conversation_id", None)
return str(cid) if cid else None
def _transcript_display_text(transcript: list[Any], agent_reply: str | None = None) -> str:
text = _transcript_to_prompt(transcript)
if agent_reply:
agent_line = f"agent: {agent_reply.strip()}"
text = f"{text}\n{agent_line}".strip() if text else agent_line
return text
def _persist_conversation_turn(call_id: str, transcript: list[Any], agent_reply: str) -> None:
from integrations.voice.live import transcript_hub
from integrations.voice.store import VoiceCallStore
display = _transcript_display_text(transcript, agent_reply)
VoiceCallStore.upsert_call(
call_id,
status="in-progress",
customer_number="web-speech-engine",
transcript=display,
)
prompt = _transcript_to_prompt(transcript)
if prompt:
last_line = prompt.splitlines()[-1]
role, _, content = last_line.partition(": ")
transcript_hub.publish(
call_id,
{
"type": "transcript",
"role": role or "user",
"transcriptType": "final",
"text": content,
},
)
transcript_hub.publish(
call_id,
{
"type": "transcript",
"role": "assistant",
"transcriptType": "final",
"text": agent_reply,
},
)
def _transcript_to_prompt(transcript: list[Any]) -> str:
lines: list[str] = []
for message in transcript:
role = getattr(message, "role", None) or (message.get("role") if isinstance(message, dict) else "user")
content = getattr(message, "content", None) or (
message.get("content") if isinstance(message, dict) else ""
)
text = str(content or "").strip()
if text:
lines.append(f"{role}: {text}")
if not lines:
return ""
return "\n".join(lines)
def _try_demo_tool_reply(transcript: list[Any]) -> str | None:
from integrations.voice.tools import demo_tool_reply
user_text = _latest_user_text(transcript)
if not user_text:
return None
return demo_tool_reply(user_text)
async def _run_openai_reply(transcript: list[Any], *, session_id: str | None = None) -> str:
"""Voice LLM path with demo fast-path + OpenAI tool calling."""
import json
from openai import AsyncOpenAI
from integrations.voice.agent_runner import execute_voice_tool
from integrations.voice.session_memory import openai_messages_with_memory
from integrations.voice.tools import openai_tool_schemas, tool_result_for_voice
prompt = _transcript_to_prompt(transcript)
if not prompt:
return "I did not catch that. Could you repeat?"
fast = _try_demo_tool_reply(transcript)
if fast:
return fast
client = AsyncOpenAI(api_key=os.getenv("OPENAI_API_KEY", "").strip())
model = os.getenv("VOICE_AGENT_MODEL", "gpt-4o-mini")
base_instructions = (
"You are a helpful voice assistant for PraisonAI demos. "
"Keep answers concise (1-3 sentences) and conversational. "
"When the caller asks for the time, call get_current_time. "
"When they ask you to repeat or echo text, call echo_message."
)
messages = openai_messages_with_memory(
prompt.splitlines(),
session_id=session_id,
base_instructions=base_instructions,
)
use_tools = os.getenv("VOICE_SPEECH_USE_TOOLS", "true").lower() in ("1", "true", "yes")
create_kwargs: dict[str, Any] = {
"model": model,
"messages": messages,
"max_tokens": 200,
}
if use_tools:
create_kwargs["tools"] = openai_tool_schemas()
create_kwargs["tool_choice"] = "auto"
response = await client.chat.completions.create(**create_kwargs)
choice = response.choices[0].message
if use_tools and choice.tool_calls:
messages.append(choice.model_dump(exclude_none=True))
fallback = "Done."
for tool_call in choice.tool_calls:
fn = tool_call.function
name = fn.name if fn else ""
try:
args = json.loads(fn.arguments or "{}") if fn else {}
except json.JSONDecodeError:
args = {}
raw = execute_voice_tool(name, args, session_id=session_id)
fallback = tool_result_for_voice(raw)
messages.append(
{
"role": "tool",
"tool_call_id": tool_call.id,
"content": raw,
}
)
final = await client.chat.completions.create(
model=model,
messages=messages,
max_tokens=200,
)
text = (final.choices[0].message.content or "").strip()
return text or fallback
text = (choice.content or "").strip()
return text or "Sorry, I could not generate a response."
async def _run_agent_reply(transcript: list[Any], *, session_id: str | None = None) -> str:
prompt = _transcript_to_prompt(transcript)
if not prompt:
return "I did not catch that. Could you repeat?"
# Prefer fast OpenAI streaming for Speech Engine turn latency.
if os.getenv("OPENAI_API_KEY", "").strip():
try:
return await asyncio.wait_for(
_run_openai_reply(transcript, session_id=session_id),
timeout=20.0,
)
except TimeoutError:
return "Sorry, that took too long. Could you ask again with a shorter question?"
except Exception: # noqa: BLE001
logger.exception("OpenAI Speech Engine reply failed; falling back to agent")
from integrations.voice.agent_runner import get_voice_agent
agent = get_voice_agent(session_id)
reply = await asyncio.wait_for(asyncio.to_thread(agent.chat, prompt), timeout=25.0)
return str(reply).strip() or "Sorry, I could not generate a response."
def _latest_user_text(transcript: list[Any]) -> str:
prompt = _transcript_to_prompt(transcript)
if not prompt:
return ""
for line in reversed(prompt.splitlines()):
if line.startswith("user:"):
return line[5:].strip()
return ""
async def handle_user_transcript(transcript: list[Any], session: Any) -> None:
"""Run voice LLM on transcript and send text back to Speech Engine."""
from integrations.voice.session_memory import record_voice_turn, resolve_praison_session_id
call_id = _conversation_id(session)
praison_session_id = resolve_praison_session_id(call_id)
preview = _transcript_to_prompt(transcript)
print(f"[speech sidecar] transcript: {preview[:200]!r}", flush=True)
reply = ""
try:
reply = await _run_agent_reply(transcript, session_id=praison_session_id)
print(f"[speech sidecar] reply: {reply[:200]!r}", flush=True)
await session.send_response(reply)
except TimeoutError:
reply = "Sorry, that took too long. Please try a shorter question."
await session.send_response(reply)
except Exception: # noqa: BLE001
logger.exception("Speech Engine agent reply failed")
reply = "Sorry, something went wrong on my side."
await session.send_response(reply)
if call_id and reply:
_persist_conversation_turn(call_id, transcript, reply)
user_text = _latest_user_text(transcript)
await record_voice_turn(call_id, user_text=user_text or None, agent_text=reply)
async def handle_session_init(conversation_id: str, session: Any) -> None:
from integrations.voice.session_memory import bind_call_from_pending, ensure_ui_session, resolve_praison_session_id
from integrations.voice.store import VoiceCallStore
call_id = str(conversation_id)
bind_call_from_pending(call_id)
VoiceCallStore.upsert_call(
call_id,
status="in-progress",
customer_number="web-speech-engine",
)
session_id = resolve_praison_session_id(call_id)
if session_id:
await ensure_ui_session(session_id, call_id=call_id)
logger.info("Speech Engine session started: %s", conversation_id)
async def handle_session_close(session: Any) -> None:
from integrations.voice.call_finalize import finalize_call
conversation_id = _conversation_id(session)
if conversation_id:
await finalize_call(conversation_id, status="ended (customer-ended-call)")
logger.info("Speech Engine session ended: %s", conversation_id)
async def handle_session_disconnect(session: Any) -> None:
"""Fallback when WebSocket drops without a formal close message."""
await handle_session_close(session)
def get_webrtc_token() -> str:
"""Mint a browser conversation token (keeps API key server-side)."""
settings = load_speech_engine_settings(require_engine_id=True)
try:
from elevenlabs import ElevenLabs
except ImportError as exc:
raise SpeechEngineConfigError("Install elevenlabs: pip install elevenlabs") from exc
client = ElevenLabs(api_key=settings.api_key)
response = client.conversational_ai.conversations.get_webrtc_token(
agent_id=settings.engine_id,
)
token = getattr(response, "token", None) or (response.get("token") if isinstance(response, dict) else None)
if not token:
raise SpeechEngineConfigError("Could not mint Speech Engine conversation token")
return str(token)
@@ -43,6 +43,15 @@ def _connect() -> sqlite3.Connection:
)
"""
)
conn.execute(
"""
CREATE TABLE IF NOT EXISTS voice_memory_pending (
key TEXT PRIMARY KEY,
user_id TEXT NOT NULL,
updated_at TEXT NOT NULL
)
"""
)
return conn
@@ -146,14 +155,25 @@ class VoiceCallStore:
@staticmethod
def append_transcript_line(call_id: str, line: str) -> str:
with closing(_connect()) as conn:
now = datetime.now(timezone.utc).isoformat()
with closing(_connect()) as conn, conn:
row = conn.execute(
"SELECT transcript FROM voice_calls WHERE call_id = ?",
(call_id,),
).fetchone()
existing = row[0] if row else ""
full = f"{existing}\n{line}".strip() if existing else line
VoiceCallStore.upsert_call(call_id, transcript=full)
existing = row[0] if row else ""
full = f"{existing}\n{line}".strip() if existing else line
conn.execute(
"""
INSERT INTO voice_calls (
call_id, status, transcript, summary, metadata_json, created_at, updated_at
) VALUES (?, 'unknown', ?, '', '{}', ?, ?)
ON CONFLICT(call_id) DO UPDATE SET
transcript = excluded.transcript,
updated_at = excluded.updated_at
""",
(call_id, full, now, now),
)
return full
@staticmethod
@@ -161,7 +181,8 @@ class VoiceCallStore:
with closing(_connect()) as conn:
rows = conn.execute(
"""
SELECT call_id, status, customer_number, transcript, summary, metadata_json, updated_at
SELECT call_id, status, customer_number, transcript, summary, metadata_json,
created_at, updated_at
FROM voice_calls ORDER BY updated_at DESC LIMIT ?
""",
(max(1, limit),),
@@ -176,17 +197,43 @@ class VoiceCallStore:
"transcript": row[3],
"summary": row[4],
"metadata": json.loads(row[5] or "{}"),
"updated_at": row[6],
"created_at": row[6],
"updated_at": row[7],
}
)
return out
@staticmethod
def set_pending_user(user_id: str) -> None:
now = datetime.now(timezone.utc).isoformat()
with closing(_connect()) as conn, conn:
conn.execute(
"""
INSERT INTO voice_memory_pending (key, user_id, updated_at)
VALUES ('browser', ?, ?)
ON CONFLICT(key) DO UPDATE SET user_id = excluded.user_id, updated_at = excluded.updated_at
""",
(user_id, now),
)
@staticmethod
def consume_pending_user() -> str | None:
with closing(_connect()) as conn, conn:
row = conn.execute(
"SELECT user_id FROM voice_memory_pending WHERE key = 'browser'",
).fetchone()
if not row:
return None
conn.execute("DELETE FROM voice_memory_pending WHERE key = 'browser'")
return str(row[0])
@staticmethod
def get_call(call_id: str) -> dict[str, Any] | None:
with closing(_connect()) as conn:
row = conn.execute(
"""
SELECT call_id, status, customer_number, transcript, summary, metadata_json, updated_at
SELECT call_id, status, customer_number, transcript, summary, metadata_json,
created_at, updated_at
FROM voice_calls WHERE call_id = ?
""",
(call_id,),
@@ -200,5 +247,6 @@ class VoiceCallStore:
"transcript": row[3],
"summary": row[4],
"metadata": json.loads(row[5] or "{}"),
"updated_at": row[6],
"created_at": row[6],
"updated_at": row[7],
}
@@ -19,6 +19,65 @@ def echo_message(message: str = "") -> dict[str, Any]:
return {"message": text, "spoken": f"You said {text}"}
def demo_tool_reply(text: str) -> str | None:
"""Fast path for common voice tool queries — no LLM round-trip."""
lower = text.lower().strip()
if any(p in lower for p in ("what time", "current time", "tell me the time", "time is it")):
return str(get_current_time()["spoken"])
if lower.startswith("echo "):
return str(echo_message(text[5:])["spoken"])
if lower.startswith("repeat "):
return str(echo_message(text[7:])["spoken"])
return None
def openai_tool_schemas() -> list[dict[str, Any]]:
"""OpenAI function schemas for Speech Engine / voice LLM paths."""
return [
{
"type": "function",
"function": {
"name": "get_current_time",
"description": "Return the current UTC time when the caller asks for the time.",
"parameters": {"type": "object", "properties": {}, "required": []},
},
},
{
"type": "function",
"function": {
"name": "echo_message",
"description": "Repeat or echo back a phrase the caller wants repeated.",
"parameters": {
"type": "object",
"properties": {
"message": {
"type": "string",
"description": "The text to echo back to the caller",
}
},
"required": ["message"],
},
},
},
]
def tool_result_for_voice(raw_result: str) -> str:
"""Convert tool JSON output to speakable voice text."""
try:
data = json.loads(raw_result)
except json.JSONDecodeError:
return raw_result.strip() or "Done."
if isinstance(data, dict):
spoken = data.get("spoken")
if spoken:
return str(spoken)
err = data.get("error")
if err:
return f"Sorry, {err}"
return raw_result.strip() or "Done."
_TOOL_REGISTRY: dict[str, Callable[..., Any]] = {
"get_current_time": get_current_time,
"echo_message": echo_message,
@@ -0,0 +1,76 @@
"""HTTP routes for stable voice user sessions."""
from __future__ import annotations
from starlette.requests import Request
from starlette.responses import JSONResponse
from integrations.voice.session_memory import (
attach_pending_user,
bind_call_user,
memory_enabled,
resolve_praison_session_id,
voice_user_session_id,
)
async def api_voice_memory_config(_request: Request) -> JSONResponse:
return JSONResponse({"enabled": memory_enabled()})
async def api_voice_memory_bind(request: Request) -> JSONResponse:
try:
payload = await request.json()
except Exception: # noqa: BLE001
return JSONResponse({"error": "Invalid JSON"}, status_code=400)
call_id = str(payload.get("call_id") or "").strip()
user_id = str(payload.get("user_id") or "").strip()
if not call_id or not user_id:
return JSONResponse({"error": "call_id and user_id required"}, status_code=400)
try:
session_id = bind_call_user(call_id, user_id)
except ValueError as exc:
return JSONResponse({"error": str(exc)}, status_code=400)
return JSONResponse(
{
"ok": True,
"call_id": call_id,
"user_id": user_id,
"praison_session_id": session_id,
}
)
async def api_voice_memory_resolve(request: Request) -> JSONResponse:
user_id = request.query_params.get("user_id", "").strip()
if not user_id:
return JSONResponse({"error": "user_id required"}, status_code=400)
session_id = voice_user_session_id(user_id)
return JSONResponse({"user_id": user_id, "praison_session_id": session_id})
async def api_voice_memory_attach_user(request: Request) -> JSONResponse:
"""ElevenLabs: browser attaches user before conversation id exists."""
try:
payload = await request.json()
except Exception: # noqa: BLE001
return JSONResponse({"error": "Invalid JSON"}, status_code=400)
user_id = str(payload.get("user_id") or "").strip()
if not user_id:
return JSONResponse({"error": "user_id required"}, status_code=400)
attach_pending_user(user_id)
return JSONResponse(
{
"ok": True,
"user_id": user_id,
"praison_session_id": voice_user_session_id(user_id),
}
)
async def api_voice_memory_for_call(request: Request) -> JSONResponse:
call_id = request.query_params.get("call_id", "").strip()
if not call_id:
return JSONResponse({"error": "call_id required"}, status_code=400)
session_id = resolve_praison_session_id(call_id)
return JSONResponse({"call_id": call_id, "praison_session_id": session_id})
+781
View File
@@ -0,0 +1,781 @@
/**
* Voice Agent Phase 2 — live transcript + browser talk widget.
*
* custom.js can run before dashboard.js defines window.aiui, so we wait.
*/
(function () {
function whenReady(fn) {
if (window.aiui && typeof window.aiui.registerView === 'function') {
fn();
return;
}
var n = 0;
var t = setInterval(function () {
n += 1;
if (window.aiui && typeof window.aiui.registerView === 'function') {
clearInterval(t);
fn();
} else if (n > 150) {
clearInterval(t);
console.warn('[voice plugin] window.aiui never became ready');
}
}, 100);
}
function el(tag, style, text) {
var node = document.createElement(tag);
if (style) node.style.cssText = style;
if (text != null) node.textContent = text;
return node;
}
function currentCallId() {
return new URLSearchParams(window.location.search).get('call_id') || '';
}
function voiceUserId() {
var key = 'voice-user-id';
var id = localStorage.getItem(key);
if (!id) {
id = 'u-' + Math.random().toString(36).slice(2, 12);
localStorage.setItem(key, id);
}
return id;
}
async function postJson(url, body) {
var resp = await fetch(url, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(body || {}),
});
if (!resp.ok) throw new Error('HTTP ' + resp.status);
return resp.json();
}
async function attachVoiceUser() {
return postJson('/api/voice/memory/attach-user', { user_id: voiceUserId() });
}
function renderUiComponent(comp) {
if (window.aiui && typeof window.aiui.renderComponent === 'function') {
return window.aiui.renderComponent(comp);
}
var wrap = el('div');
if (comp.type === 'code_block') {
var pre = el(
'pre',
'margin:0;padding:16px;border-radius:10px;background:var(--db-card-bg,#1f2937);' +
'border:1px solid var(--db-border,#334155);white-space:pre-wrap;font-family:ui-monospace,monospace;font-size:13px'
);
pre.textContent = comp.code || '';
wrap.appendChild(pre);
wrap.className = 'db-code-block';
return wrap;
}
if (comp.type === 'text') {
wrap.textContent = comp.content || '';
return wrap;
}
wrap.textContent = JSON.stringify(comp);
return wrap;
}
function renderUiTree(data, container) {
if (!data) return;
if (data._components) {
data._components.forEach(function (c) {
container.appendChild(renderUiComponent(c));
});
return;
}
if (data.type === 'layout' && data.children) {
data.children.forEach(function (c) {
container.appendChild(renderUiComponent(c));
});
}
}
function findTranscriptCodeEl(root) {
var blocks = root.querySelectorAll('.db-code-block code');
if (!blocks.length) {
blocks = root.querySelectorAll('.db-code-block pre');
}
return blocks.length ? blocks[blocks.length - 1] : null;
}
async function fetchJson(url, retries) {
var attempts = retries == null ? 1 : retries;
var lastErr = null;
for (var i = 0; i < attempts; i += 1) {
try {
var resp = await fetch(url);
if (!resp.ok) throw new Error('HTTP ' + resp.status);
return resp.json();
} catch (err) {
lastErr = err;
if (i < attempts - 1) {
await new Promise(function (r) {
setTimeout(r, 1500);
});
}
}
}
throw lastErr || new Error('fetch failed');
}
async function resolveCallId() {
var fromQuery = currentCallId();
if (fromQuery) return fromQuery;
var data = await fetchJson('/api/voice/calls');
var calls = data.calls || [];
var live = calls.find(function (c) {
return String(c.status || '').toLowerCase().indexOf('progress') >= 0;
});
return (live && live.call_id) || (calls[0] && calls[0].call_id) || '';
}
async function loadWebSdk(cfg) {
var candidates = [];
var configured = cfg.sdk_url || '';
if (configured && configured.indexOf('dist/vapi.js') < 0) {
candidates.push(configured);
}
candidates.push('https://esm.sh/@vapi-ai/[email protected]');
var lastErr = null;
for (var i = 0; i < candidates.length; i++) {
try {
var mod = await import(/* webpackIgnore: true */ candidates[i]);
var Sdk = mod.default || mod.Vapi;
if (typeof Sdk === 'function') return Sdk;
lastErr = new Error('Module loaded but no Vapi class');
} catch (err) {
lastErr = err;
console.warn('[voice plugin] SDK load failed for', candidates[i], err);
}
}
throw new Error(
'Could not load voice web SDK' + (lastErr && lastErr.message ? ': ' + lastErr.message : '')
);
}
function registerVoiceViews() {
window.aiui.registerView(
'call-detail',
async function (container) {
container.innerHTML = '';
var wrap = el(
'div',
'padding:20px;display:flex;flex-direction:column;gap:16px;color:var(--db-text,#eee)'
);
var uiHost = el('div', 'display:flex;flex-direction:column;gap:12px');
var statusLine = el(
'div',
'font-size:12px;color:var(--db-text-dim,#94a3b8)',
'Loading PraisonAIUI components…'
);
wrap.appendChild(uiHost);
wrap.appendChild(statusLine);
container.appendChild(wrap);
var es = null;
var lines = [];
var transcriptEl = null;
function renderTranscript(fullText) {
if (!transcriptEl) return;
if (fullText) {
transcriptEl.textContent = fullText;
return;
}
transcriptEl.textContent = lines.length ? lines.join('\n') : '(empty)';
}
function appendLine(role, text) {
lines.push(role + ': ' + text);
renderTranscript('');
}
try {
var callId = await resolveCallId();
if (!callId) {
renderUiTree(
{
type: 'layout',
children: [
{
type: 'alert',
variant: 'info',
title: 'Call detail',
message: 'No calls yet — start from ElevenLabs talk or GPT Realtime.',
},
],
},
uiHost
);
statusLine.textContent = 'Idle';
return;
}
var uiData = await fetchJson('/api/voice/calls/' + encodeURIComponent(callId) + '/ui');
renderUiTree(uiData.layout, uiHost);
transcriptEl = findTranscriptCodeEl(uiHost);
var detail = await fetchJson('/api/voice/calls/' + encodeURIComponent(callId));
if (detail.transcript) {
lines = detail.transcript.split('\n').filter(Boolean);
renderTranscript(detail.transcript);
}
statusLine.textContent = 'Live feed connecting…';
es = new EventSource(
'/api/voice/calls/' + encodeURIComponent(callId) + '/live-transcript'
);
es.onmessage = function (ev) {
try {
var payload = JSON.parse(ev.data);
if (payload.type === 'snapshot' && payload.transcript) {
lines = payload.transcript.split('\n').filter(Boolean);
renderTranscript(payload.transcript);
statusLine.textContent = 'Live · snapshot loaded';
return;
}
if (payload.type === 'transcript') {
if (payload.transcriptType === 'final') {
appendLine(payload.role || 'unknown', payload.text || '');
} else if (transcriptEl) {
transcriptEl.textContent =
(lines.length ? lines.join('\n') + '\n' : '') +
(payload.role || 'unknown') +
': ' +
(payload.text || '') +
' …';
}
statusLine.textContent = 'Live · ' + (payload.transcriptType || 'update');
}
} catch (err) {
console.warn('[voice plugin] SSE parse error', err);
}
};
es.onerror = function () {
statusLine.textContent = 'Live feed disconnected (call may have ended)';
};
} catch (err) {
statusLine.textContent = 'Failed to load call: ' + err.message;
}
return function cleanup() {
if (es) es.close();
};
},
function () {}
);
window.aiui.registerView(
'web-talk',
async function (container) {
container.innerHTML = '';
var wrap = el('div', 'padding:24px;max-width:640px;color:var(--db-text,#eee)');
wrap.appendChild(el('h2', 'margin:0 0 12px 0', 'Web voice'));
wrap.appendChild(
el(
'p',
'margin:0 0 20px 0;line-height:1.6;color:var(--db-text-dim,#94a3b8)',
'Click Start talking, then allow the microphone when Chrome asks. The prompt does not appear until you click the button.'
)
);
var status = el('div', 'margin-bottom:16px;font-size:13px', 'Loading config…');
var btn = el(
'button',
'padding:12px 20px;border-radius:10px;border:none;background:var(--db-accent,#6366f1);' +
'color:#fff;font-size:15px;font-weight:600;cursor:pointer',
'Start talking'
);
btn.disabled = true;
wrap.appendChild(status);
wrap.appendChild(btn);
container.appendChild(wrap);
var cfg = null;
var client = null;
try {
cfg = await fetchJson('/api/voice/web-config');
if (!cfg.enabled) {
status.textContent =
cfg.error ||
'Web talk is not configured. Set VOICE_PUBLIC_API_KEY, VOICE_ASSISTANT_ID, VOICE_WEB_SDK_URL, VOICE_WEB_SDK_GLOBAL.';
btn.style.display = 'none';
return;
}
status.textContent = 'Ready — click Start talking, then allow microphone.';
btn.disabled = false;
} catch (err) {
status.textContent = 'Config error: ' + err.message;
return;
}
btn.addEventListener('click', async function () {
if (client && typeof client.stop === 'function' && btn.textContent === 'End call') {
client.stop();
client = null;
status.textContent = 'Call ended';
btn.textContent = 'Start talking';
return;
}
btn.disabled = true;
status.textContent = 'Loading web SDK…';
try {
var Sdk = await loadWebSdk(cfg);
client = new Sdk(cfg.public_key);
if (typeof client.start === 'function') {
await client.start(cfg.assistant_id);
} else if (typeof client.startAssistant === 'function') {
await client.startAssistant(cfg.assistant_id);
} else {
throw new Error('SDK has no start() method');
}
status.textContent = 'In call — allow microphone if Chrome asks. Speak now.';
btn.textContent = 'End call';
btn.disabled = false;
} catch (err) {
status.textContent = 'Start failed: ' + err.message;
btn.disabled = false;
}
});
}
);
window.aiui.registerView(
'eleven-talk',
async function (container) {
container.innerHTML = '';
var wrap = el('div', 'padding:24px;max-width:640px;color:var(--db-text,#eee)');
wrap.appendChild(el('h2', 'margin:0 0 12px 0', 'ElevenLabs voice'));
wrap.appendChild(
el(
'p',
'margin:0 0 20px 0;line-height:1.6;color:var(--db-text-dim,#94a3b8)',
'Speech Engine runs STT/TTS in the browser. Your PraisonAI agent handles replies on the server.'
)
);
var status = el('div', 'margin-bottom:16px;font-size:13px', 'Loading config…');
var btn = el(
'button',
'padding:12px 20px;border-radius:10px;border:none;background:var(--db-accent,#6366f1);' +
'color:#fff;font-size:15px;font-weight:600;cursor:pointer',
'Start conversation'
);
btn.disabled = true;
wrap.appendChild(status);
wrap.appendChild(btn);
container.appendChild(wrap);
var conversation = null;
var cfg = null;
try {
cfg = await fetchJson('/api/voice/speech-engine/config', 8);
if (!cfg.enabled) {
status.textContent =
cfg.error ||
'Speech Engine not ready. Run: pip install -U "elevenlabs>=2.47.0" then .\\start_dev.ps1 -Restart. ' +
'API key needs convai_write permission in ElevenLabs dashboard.';
btn.style.display = 'none';
return;
}
var toolList = (cfg.tools && cfg.tools.length ? cfg.tools.join(', ') : 'get_current_time, echo_message');
status.textContent =
'Ready — LLM ' +
(cfg.model || 'gpt-4o-mini') +
' · tools: ' +
toolList +
'. Click Start conversation, then allow microphone.';
btn.disabled = false;
} catch (err) {
status.textContent = 'Config error: ' + err.message;
return;
}
btn.addEventListener('click', async function () {
if (conversation && btn.textContent === 'End conversation') {
try {
await conversation.endSession();
} catch (e) {
/* ignore */
}
conversation = null;
status.textContent = 'Conversation ended';
btn.textContent = 'Start conversation';
return;
}
btn.disabled = true;
status.textContent = 'Loading ElevenLabs client…';
try {
var sdkUrl = cfg.client_sdk_url || 'https://esm.sh/@elevenlabs/client';
var mod = await import(/* webpackIgnore: true */ sdkUrl);
var Conversation = mod.Conversation;
if (!Conversation || typeof Conversation.startSession !== 'function') {
throw new Error('ElevenLabs client SDK missing Conversation.startSession');
}
var tokenResp = await fetchJson('/api/voice/speech-engine/token');
if (!tokenResp.token) throw new Error('No conversation token returned');
await navigator.mediaDevices.getUserMedia({ audio: true });
await attachVoiceUser();
var startOpts = {
conversationToken: tokenResp.token,
onConnect: function () {
status.textContent =
'Connected — speak, then pause ~1s so the agent knows you finished.';
btn.textContent = 'End conversation';
btn.disabled = false;
},
onDisconnect: function (details) {
var reason = details && details.reason ? details.reason : '';
if (reason === 'agent' || reason === 'unknown') {
reason =
'Speech Engine server did not respond in time. Restart .\\start_dev.ps1 -Restart and try again.';
} else if (!reason) {
reason =
'Connection closed (ElevenLabs could not reach your Speech Engine WebSocket).';
}
status.textContent = 'Disconnected — ' + reason;
btn.textContent = 'Start conversation';
btn.disabled = false;
conversation = null;
},
onError: function (error) {
status.textContent =
'Error: ' + (error && error.message ? error.message : String(error));
btn.disabled = false;
},
onStatusChange: function (state) {
if (state && state.status) {
status.textContent = 'Status: ' + state.status;
}
},
onModeChange: function (mode) {
if (!mode || !mode.mode) return;
if (mode.mode === 'listening') {
status.textContent = 'Listening… speak, then pause when done.';
} else if (mode.mode === 'speaking') {
status.textContent = 'Agent speaking…';
} else {
status.textContent = 'Mode: ' + mode.mode;
}
},
onMessage: function (msg) {
if (msg && msg.message) {
console.log('[eleven-talk]', msg.source || 'unknown', msg.message);
}
},
};
if (cfg.first_message) {
startOpts.overrides = {
agent: { firstMessage: cfg.first_message },
};
}
conversation = await Conversation.startSession(startOpts);
} catch (err) {
status.textContent = 'Start failed: ' + err.message;
btn.disabled = false;
}
});
}
);
window.aiui.registerView(
'realtime-talk',
async function (container) {
container.innerHTML = '';
var wrap = el('div', 'padding:24px;max-width:640px;color:var(--db-text,#eee)');
wrap.appendChild(el('h2', 'margin:0 0 12px 0', 'GPT Realtime voice'));
wrap.appendChild(
el(
'p',
'margin:0 0 20px 0;line-height:1.6;color:var(--db-text-dim,#94a3b8)',
'Speech-to-speech via OpenAI gpt-realtime-2.1-mini (WebRTC).'
)
);
var status = el('div', 'margin-bottom:16px;font-size:13px', 'Loading config…');
var btn = el(
'button',
'padding:12px 20px;border-radius:10px;border:none;background:var(--db-accent,#6366f1);' +
'color:#fff;font-size:15px;font-weight:600;cursor:pointer',
'Start conversation'
);
btn.disabled = true;
wrap.appendChild(status);
wrap.appendChild(btn);
container.appendChild(wrap);
var pc = null;
var dc = null;
var audioEl = null;
var localStream = null;
var callId = null;
var cfg = null;
async function saveTranscript(role, text) {
if (!callId || !text) return;
try {
await postJson('/api/voice/realtime/transcript', {
call_id: callId,
role: role,
text: text,
});
} catch (e) {
console.warn('[realtime-talk] transcript save failed', e);
}
}
function handleRealtimeEvent(event) {
if (!event || !event.type) return;
if (
event.type === 'conversation.item.input_audio_transcription.completed' &&
event.transcript
) {
saveTranscript('user', event.transcript);
status.textContent = 'You: ' + event.transcript;
}
if (event.type === 'response.output_audio_transcript.done' && event.transcript) {
saveTranscript('agent', event.transcript);
status.textContent = 'Agent: ' + event.transcript;
}
if (event.type === 'response.output_audio_transcript.delta' && event.delta) {
status.textContent = 'Agent speaking…';
}
if (event.type === 'input_audio_buffer.speech_started') {
status.textContent = 'Listening…';
}
}
async function endSession() {
if (dc) {
try {
dc.close();
} catch (e) {
/* ignore */
}
dc = null;
}
if (pc) {
try {
pc.close();
} catch (e) {
/* ignore */
}
pc = null;
}
if (localStream) {
localStream.getTracks().forEach(function (t) {
t.stop();
});
localStream = null;
}
if (audioEl && audioEl.parentNode) {
audioEl.parentNode.removeChild(audioEl);
audioEl = null;
}
if (callId) {
try {
await postJson('/api/voice/realtime/end', { call_id: callId });
} catch (e) {
/* ignore */
}
}
callId = null;
status.textContent = 'Conversation ended.';
btn.textContent = 'Start conversation';
btn.disabled = false;
}
try {
cfg = await fetchJson('/api/voice/realtime/config', 8);
if (!cfg.enabled) {
status.textContent =
cfg.error || 'Set OPENAI_API_KEY in .env, then restart .\\start_dev.ps1';
btn.style.display = 'none';
return;
}
status.textContent = 'Ready (' + (cfg.model || 'gpt-realtime-2.1-mini') + ').';
btn.disabled = false;
} catch (err) {
status.textContent = 'Config error: ' + err.message;
return;
}
btn.addEventListener('click', async function () {
if (pc && btn.textContent === 'End conversation') {
await endSession();
return;
}
btn.disabled = true;
status.textContent = 'Starting WebRTC session…';
try {
var callResp = await postJson('/api/voice/realtime/new-call', {
user_id: voiceUserId(),
});
callId = callResp.call_id;
localStream = await navigator.mediaDevices.getUserMedia({ audio: true });
pc = new RTCPeerConnection();
audioEl = document.createElement('audio');
audioEl.autoplay = true;
audioEl.style.display = 'none';
wrap.appendChild(audioEl);
pc.ontrack = function (e) {
if (audioEl) audioEl.srcObject = e.streams[0];
};
localStream.getTracks().forEach(function (track) {
pc.addTrack(track, localStream);
});
dc = pc.createDataChannel('oai-events');
dc.addEventListener('open', function () {
if (cfg.first_message) {
dc.send(
JSON.stringify({
type: 'response.create',
response: {
instructions:
'Greet the user by saying exactly: ' + cfg.first_message,
},
})
);
}
});
dc.addEventListener('message', function (e) {
try {
handleRealtimeEvent(JSON.parse(e.data));
} catch (err) {
console.warn('[realtime-talk] event parse failed', err);
}
});
var offer = await pc.createOffer();
await pc.setLocalDescription(offer);
var sdpResp = await fetch(
'/api/voice/realtime/session?call_id=' + encodeURIComponent(callId),
{
method: 'POST',
headers: { 'Content-Type': 'application/sdp' },
body: offer.sdp,
}
);
if (!sdpResp.ok) {
var errText = await sdpResp.text();
throw new Error(errText || 'SDP exchange failed');
}
var answerSdp = await sdpResp.text();
await pc.setRemoteDescription({ type: 'answer', sdp: answerSdp });
status.textContent = 'Connected — speak when ready.';
btn.textContent = 'End conversation';
btn.disabled = false;
} catch (err) {
status.textContent = 'Start failed: ' + err.message;
await endSession();
btn.disabled = false;
}
});
}
);
window.aiui.registerView(
'config',
async function (container) {
container.innerHTML = '';
var wrap = el('div', 'padding:24px;max-width:900px;color:var(--db-text,#eee)');
wrap.appendChild(el('h2', 'margin:0 0 8px 0', 'Voice stack doctor'));
wrap.appendChild(
el(
'p',
'margin:0 0 16px 0;font-size:13px;color:var(--db-text-dim,#94a3b8)',
'Checks .env keys, local ports, APIs, and Speech Engine tunnel reachability.'
)
);
var status = el('div', 'margin-bottom:16px;font-size:13px', 'Running checks…');
var tableHost = el('div', '');
wrap.appendChild(status);
wrap.appendChild(tableHost);
container.appendChild(wrap);
function statusColor(st) {
if (st === 'pass') return '#22c55e';
if (st === 'warn') return '#eab308';
return '#ef4444';
}
try {
var data = await fetchJson('/api/voice/doctor', 12);
var summary = data.summary || {};
status.textContent =
'Passed ' +
(summary.passed || 0) +
' · Warnings ' +
(summary.warnings || 0) +
' · Failed ' +
(summary.failed || 0);
var rows = (data.checks || []).map(function (c) {
return [
c.name || '—',
c.status || '—',
c.detail || '—',
];
});
renderUiTree(
{
type: 'layout',
children: [
{
type: 'table',
headers: ['Check', 'Status', 'Detail'],
rows: rows.length ? rows : [['—', '—', '—']],
},
],
},
tableHost
);
var badges = tableHost.querySelectorAll('td');
badges.forEach(function (cell) {
if (cell.textContent === 'pass' || cell.textContent === 'warn' || cell.textContent === 'fail') {
cell.style.color = statusColor(cell.textContent);
cell.style.fontWeight = '600';
}
});
} catch (err) {
status.textContent = 'Doctor failed: ' + err.message;
}
},
function () {}
);
console.log(
'[voice plugin] Registered call-detail + web-talk + eleven-talk + realtime-talk + config views'
);
var path = (window.location.pathname || '').replace(/^\//, '');
if (
(path === 'web-talk' ||
path === 'call-detail' ||
path === 'eleven-talk' ||
path === 'realtime-talk') &&
typeof window.aiui.selectPage === 'function'
) {
window.aiui.selectPage(path);
} else if (
path === 'web-talk' ||
path === 'call-detail' ||
path === 'eleven-talk' ||
path === 'realtime-talk'
) {
var n = 0;
var t = setInterval(function () {
n += 1;
if (typeof window.aiui.selectPage === 'function') {
clearInterval(t);
window.aiui.selectPage(path);
} else if (n > 50) {
clearInterval(t);
}
}, 100);
}
}
whenReady(registerVoiceViews);
})();
@@ -0,0 +1,115 @@
"""HTTP routes for OpenAI Realtime WebRTC voice."""
from __future__ import annotations
import logging
from starlette.requests import Request
from starlette.responses import JSONResponse, PlainTextResponse
from integrations.voice.realtime import (
RealtimeConfigError,
load_realtime_settings,
mark_session_started,
new_call_id,
persist_transcript_line,
realtime_enabled,
create_webrtc_session,
)
logger = logging.getLogger(__name__)
async def api_realtime_config(_request: Request) -> JSONResponse:
try:
settings = load_realtime_settings()
enabled = realtime_enabled()
error = None
except RealtimeConfigError as exc:
settings = None
enabled = False
error = str(exc)
return JSONResponse(
{
"enabled": enabled,
"error": error,
"model": settings.model if settings else None,
"voice": settings.voice if settings else None,
"reasoning_effort": settings.reasoning_effort if settings else None,
"first_message": settings.first_message if settings else None,
}
)
async def api_realtime_new_call(request: Request) -> JSONResponse:
from integrations.voice.session_memory import bind_call_user, voice_user_session_id
try:
payload = await request.json()
except Exception: # noqa: BLE001
payload = {}
call_id = new_call_id()
mark_session_started(call_id)
user_id = str(payload.get("user_id") or "").strip()
praison_session_id = None
if user_id:
praison_session_id = bind_call_user(call_id, user_id)
return JSONResponse(
{
"call_id": call_id,
"user_id": user_id or None,
"praison_session_id": praison_session_id or (voice_user_session_id(user_id) if user_id else None),
}
)
async def api_realtime_session(request: Request) -> PlainTextResponse | JSONResponse:
"""Unified WebRTC interface — browser SDP in, OpenAI SDP answer out."""
call_id = request.query_params.get("call_id", "").strip()
body = await request.body()
sdp_offer = body.decode("utf-8").strip()
if not sdp_offer:
return JSONResponse({"error": "Missing SDP offer body"}, status_code=400)
try:
sdp_answer = await create_webrtc_session(sdp_offer, call_id=call_id or None)
except RealtimeConfigError as exc:
return JSONResponse({"error": str(exc)}, status_code=503)
except Exception as exc: # noqa: BLE001
logger.exception("Realtime WebRTC session failed")
return JSONResponse({"error": str(exc)}, status_code=502)
if call_id:
mark_session_started(call_id)
return PlainTextResponse(sdp_answer, media_type="application/sdp")
async def api_realtime_transcript(request: Request) -> JSONResponse:
try:
payload = await request.json()
except Exception: # noqa: BLE001
return JSONResponse({"error": "Invalid JSON"}, status_code=400)
call_id = str(payload.get("call_id") or "").strip()
role = str(payload.get("role") or "user").strip()
text = str(payload.get("text") or "").strip()
if not call_id or not text:
return JSONResponse({"error": "call_id and text required"}, status_code=400)
from integrations.voice.session_memory import record_voice_turn
persist_transcript_line(call_id, role, text)
if role == "user":
await record_voice_turn(call_id, user_text=text)
else:
await record_voice_turn(call_id, agent_text=text)
return JSONResponse({"ok": True})
async def api_realtime_end(request: Request) -> JSONResponse:
try:
payload = await request.json()
except Exception: # noqa: BLE001
return JSONResponse({"error": "Invalid JSON"}, status_code=400)
call_id = str(payload.get("call_id") or "").strip()
if call_id:
from integrations.voice.call_finalize import finalize_call
await finalize_call(call_id, status="ended (customer-ended-call)")
return JSONResponse({"ok": True})
@@ -0,0 +1,3 @@
httpx
elevenlabs>=2.47.0
openai
@@ -0,0 +1,172 @@
"""Create or update ElevenLabs Speech Engine with the public WebSocket URL.
Usage:
cd examples/python/voice-agent
.\\start_dev.ps1 # sets PUBLIC_API_BASE_URL first
py -3.13 setup_speech_engine.py
"""
from __future__ import annotations
import asyncio
import os
from pathlib import Path
_ROOT = Path(__file__).resolve().parent
ENV_PATH = _ROOT / ".env"
def _load_env() -> None:
if not ENV_PATH.is_file():
return
for raw in ENV_PATH.read_text(encoding="utf-8").splitlines():
line = raw.strip()
if not line or line.startswith("#") or "=" not in line:
continue
key, _, value = line.partition("=")
key, value = key.strip(), value.strip()
if key:
os.environ[key] = value
def _write_env_value(key: str, value: str) -> None:
lines = ENV_PATH.read_text(encoding="utf-8").splitlines() if ENV_PATH.is_file() else []
updated = False
new_lines: list[str] = []
for line in lines:
if line.strip().startswith(f"{key}="):
new_lines.append(f"{key}={value}")
updated = True
else:
new_lines.append(line)
if not updated:
new_lines.append(f"{key}={value}")
ENV_PATH.write_text("\n".join(new_lines) + "\n", encoding="utf-8")
os.environ[key] = value
def _http_to_ws(base: str) -> str:
base = base.strip().rstrip("/")
if base.startswith("https://"):
return "wss://" + base[len("https://") :] + "/ws"
if base.startswith("http://"):
return "ws://" + base[len("http://") :] + "/ws"
raise RuntimeError(f"Unsupported URL: {base}")
def _public_ws_url() -> str:
# Prefer dedicated Speech Engine tunnel (sidecar on 8002).
speech_base = os.getenv("SPEECH_ENGINE_PUBLIC_URL", "").strip().rstrip("/")
if speech_base:
return _http_to_ws(speech_base)
base = os.getenv("PUBLIC_API_BASE_URL", "").strip().rstrip("/")
if not base:
raise RuntimeError(
"Set SPEECH_ENGINE_PUBLIC_URL or PUBLIC_API_BASE_URL in .env "
"(run .\\start_dev.ps1 first)"
)
return _http_to_ws(base)
async def _patch_overrides(client: object, engine_id: str) -> None:
"""Enable client first-message override (not exposed on speech_engine.update yet)."""
wrapper = getattr(client, "_client_wrapper", None)
if wrapper is None:
return
try:
await wrapper.httpx_client.request(
f"v1/speech-engine/{engine_id}",
method="PATCH",
json={"overrides": {"first_message": True}},
headers={"content-type": "application/json"},
)
print("Enabled first_message client override")
except Exception as exc: # noqa: BLE001
print(f"Note: could not enable first_message override ({exc})")
async def _ensure_engine() -> str:
api_key = os.getenv("ELEVENLABS_API_KEY", "").strip()
if not api_key:
raise RuntimeError("Set ELEVENLABS_API_KEY in .env")
try:
from elevenlabs import AsyncElevenLabs
except ImportError as exc:
raise RuntimeError(
"Install Speech Engine SDK: pip install 'elevenlabs>=2.47.0'"
) from exc
if not hasattr(AsyncElevenLabs(api_key="check"), "speech_engine"):
raise RuntimeError(
"elevenlabs package is too old for Speech Engine. Run: "
"pip install -U 'elevenlabs>=2.47.0'"
)
ws_url = _public_ws_url()
engine_id = os.getenv("SPEECH_ENGINE_ID", "").strip()
client = AsyncElevenLabs(api_key=api_key)
turn_config = {"turn_eagerness": "eager", "turn_timeout": 5.0}
voice_id = os.getenv("ELEVENLABS_VOICE_ID", "").strip() or "cjVigY5qzO86Huf0OWal"
tts_config = {"model_id": "eleven_flash_v2", "voice_id": voice_id}
if engine_id:
updated = await client.speech_engine.update(
speech_engine_id=engine_id,
speech_engine={"ws_url": ws_url},
turn=turn_config,
tts=tts_config,
)
resolved = getattr(updated, "speech_engine_id", None) or getattr(updated, "engine_id", None) or engine_id
print(f"Updated Speech Engine: {resolved}")
print(f"WebSocket URL: {ws_url}")
print(f"TTS voice_id: {voice_id}")
await _patch_overrides(client, str(resolved))
return str(resolved)
created = await client.speech_engine.create(
name="PraisonAI Voice",
speech_engine={"ws_url": ws_url},
turn=turn_config,
tts=tts_config,
overrides={"first_message": True},
)
resolved = getattr(created, "engine_id", None) or getattr(created, "id", None)
if not resolved:
raise RuntimeError("Speech Engine create did not return an engine_id")
print(f"Created Speech Engine: {resolved}")
print(f"WebSocket URL: {ws_url}")
print(f"TTS voice_id: {voice_id}")
return str(resolved)
async def main_async() -> int:
_load_env()
try:
engine_id = await _ensure_engine()
except RuntimeError as exc:
print(f"Setup failed: {exc}")
return 1
except Exception as exc: # noqa: BLE001
message = str(exc)
if "convai_write" in message or "missing_permissions" in message:
print("Setup failed: ElevenLabs API key lacks convai_write permission.")
print(
"In ElevenLabs dashboard: API Keys -> edit key -> ElevenAgents: enable Write."
)
else:
print(f"Setup failed: {exc}")
return 1
_write_env_value("SPEECH_ENGINE_ID", engine_id)
print("Speech Engine auto-setup complete.")
return 0
def main() -> int:
return asyncio.run(main_async())
if __name__ == "__main__":
raise SystemExit(main())
@@ -28,7 +28,7 @@ def _load_env() -> None:
continue
key, _, value = line.partition("=")
key, value = key.strip(), value.strip()
if key and key not in os.environ:
if key:
os.environ[key] = value
@@ -117,6 +117,119 @@ def ensure_server_secret() -> str:
return secret
def ensure_assistant_model(api_key: str, api_base: str, assistant_id: str) -> dict[str, Any]:
assistant = _request("GET", f"/assistant/{assistant_id}", api_key=api_key, api_base=api_base)
if not isinstance(assistant, dict):
raise RuntimeError("Could not load assistant")
model = dict(assistant.get("model") or {})
target = os.getenv("VOICE_AGENT_MODEL", "gpt-4o-mini")
if model.get("provider") != "openai" or model.get("model") != target:
model["provider"] = "openai"
model["model"] = target
patched = _request(
"PATCH",
f"/assistant/{assistant_id}",
api_key=api_key,
api_base=api_base,
body={"model": model},
)
if patched is None:
raise RuntimeError(f"Failed to set assistant model to {target}")
print(f"Set assistant model to openai/{target}")
return model
def ensure_agent_model_env() -> str:
"""Ensure VOICE_AGENT_MODEL is set in .env and return the target model."""
target = os.getenv("VOICE_AGENT_MODEL", "").strip() or "gpt-4o-mini"
if not os.getenv("VOICE_AGENT_MODEL", "").strip():
_write_env_value("VOICE_AGENT_MODEL", target)
print(f"Set VOICE_AGENT_MODEL={target} in .env")
return target
def ensure_openai_credential(api_key: str, api_base: str, openai_api_key: str) -> tuple[str, str]:
"""Create or reuse an OpenAI-compatible credential in the provider account."""
existing = _request("GET", "/credential", api_key=api_key, api_base=api_base)
if isinstance(existing, list):
preferred = sorted(
[item for item in existing if isinstance(item, dict) and item.get("id")],
key=lambda item: 0 if str(item.get("provider") or "") == "openai" else 1,
)
for item in preferred:
provider = str(item.get("provider") or "")
if provider in ("openai", "custom-llm"):
cred_id = str(item["id"])
print(f"Reusing credential ({provider}): {cred_id}")
updated = _request(
"PATCH",
f"/credential/{cred_id}",
api_key=api_key,
api_base=api_base,
body={
"provider": provider,
"apiKey": openai_api_key,
"name": "Praison Voice OpenAI",
},
)
if updated is None:
raise RuntimeError("Failed to update OpenAI credential")
return cred_id, provider
for provider in ("openai", "custom-llm"):
created = _request(
"POST",
"/credential",
api_key=api_key,
api_base=api_base,
body={
"provider": provider,
"apiKey": openai_api_key,
"name": "Praison Voice OpenAI",
},
)
if isinstance(created, dict) and created.get("id"):
cred_id = str(created["id"])
print(f"Created credential ({provider}): {cred_id}")
return cred_id, provider
raise RuntimeError(
"Failed to create OpenAI credential. Add the key manually in the provider dashboard "
"under Provider Keys / OpenAI, or enable gpt-4o-mini on your OpenAI project."
)
def ensure_openai_on_assistant(
api_key: str, api_base: str, assistant_id: str, openai_credential_id: str, *, provider: str
) -> None:
assistant = _request("GET", f"/assistant/{assistant_id}", api_key=api_key, api_base=api_base)
if not isinstance(assistant, dict):
raise RuntimeError("Could not load assistant for OpenAI setup")
target_model = os.getenv("VOICE_AGENT_MODEL", "gpt-4o-mini")
if provider == "custom-llm":
model = dict(assistant.get("model") or {})
model["model"] = target_model
model["provider"] = "custom-llm"
model["url"] = "https://api.openai.com/v1"
model["metadataSendMode"] = "off"
model.pop("credentialId", None)
else:
# Native OpenAI — drop custom-llm-only fields from prior config.
model = {"provider": "openai", "model": target_model}
patched = _request(
"PATCH",
f"/assistant/{assistant_id}",
api_key=api_key,
api_base=api_base,
body={"model": model},
)
if patched is None:
raise RuntimeError("Failed to attach OpenAI credential to assistant")
print(f"Configured assistant model ({provider}): {target_model}")
def ensure_tools(api_key: str, api_base: str, assistant_id: str) -> list[str]:
existing = _request("GET", "/tool", api_key=api_key, api_base=api_base)
by_name: dict[str, str] = {}
@@ -230,8 +343,18 @@ def main() -> int:
secret = ensure_server_secret()
webhook_url = f"{public_base}/webhooks/voice"
target_model = ensure_agent_model_env()
openai_api_key = os.getenv("OPENAI_API_KEY", "").strip()
try:
if openai_api_key:
cred_id, cred_provider = ensure_openai_credential(api_key, api_base, openai_api_key)
ensure_openai_on_assistant(
api_key, api_base, assistant_id, cred_id, provider=cred_provider
)
print(f"Assistant model target: {target_model}")
else:
print("OPENAI_API_KEY not set — skipping provider OpenAI credential setup")
ensure_tools(api_key, api_base, assistant_id)
ensure_webhook(api_key, api_base, assistant_id, webhook_url, secret)
except RuntimeError as exc:
@@ -0,0 +1,110 @@
"""HTTP + WebSocket routes for ElevenLabs Speech Engine."""
from __future__ import annotations
import logging
import os
from starlette.requests import Request
from starlette.responses import JSONResponse
from starlette.websockets import WebSocket, WebSocketDisconnect
from integrations.voice.speech_engine import (
SpeechEngineConfigError,
get_engine_resource,
get_webrtc_token,
handle_session_close,
handle_session_init,
handle_user_transcript,
load_speech_engine_settings,
speech_engine_enabled,
)
logger = logging.getLogger(__name__)
async def api_speech_engine_config(_request: Request) -> JSONResponse:
try:
settings = load_speech_engine_settings()
enabled = speech_engine_enabled()
error = None
except SpeechEngineConfigError as exc:
settings = None
enabled = False
error = str(exc)
tools: list[str] = []
if enabled:
try:
from integrations.voice.agent_runner import list_registered_tool_names
tools = list_registered_tool_names()
except Exception: # noqa: BLE001
tools = ["get_current_time", "echo_message"]
return JSONResponse(
{
"enabled": enabled,
"error": error,
"engine_id": settings.engine_id if settings else None,
"ws_url": settings.public_ws_url if settings else None,
"client_sdk_url": "https://esm.sh/@elevenlabs/client",
"first_message": settings.first_message if settings else None,
"model": os.getenv("VOICE_AGENT_MODEL", "gpt-4o-mini"),
"tools": tools,
}
)
async def api_speech_engine_token(_request: Request) -> JSONResponse:
try:
token = get_webrtc_token()
except SpeechEngineConfigError as exc:
return JSONResponse({"error": str(exc)}, status_code=503)
except Exception as exc: # noqa: BLE001
logger.exception("Speech Engine token mint failed")
return JSONResponse({"error": str(exc)}, status_code=502)
return JSONResponse({"token": token})
async def speech_engine_websocket(websocket: WebSocket) -> None:
try:
engine = await get_engine_resource()
except SpeechEngineConfigError as exc:
logger.warning("Speech Engine WS rejected: %s", exc)
return
headers = dict(websocket.headers)
if hasattr(engine, "verify_request") and not engine.verify_request(headers):
logger.warning(
"Speech Engine WS unauthorized (missing or invalid ElevenLabs JWT header)"
)
return
await websocket.accept()
logger.info("Speech Engine WebSocket accepted from ElevenLabs")
session = engine.create_session(websocket, debug=False)
async def on_transcript(transcript, sess): # noqa: ANN001
await handle_user_transcript(transcript, sess)
async def on_init(conversation_id, sess): # noqa: ANN001
await handle_session_init(conversation_id, sess)
async def on_close(sess): # noqa: ANN001
await handle_session_close(sess)
def on_error(err, sess=None): # noqa: ANN001
logger.error("Speech Engine session error: %s", err)
session.on("user_transcript", on_transcript)
session.on("init", on_init)
session.on("close", on_close)
session.on("error", on_error)
try:
await session.run()
except WebSocketDisconnect:
logger.info("Speech Engine WebSocket disconnected")
await handle_session_close(session)
except Exception: # noqa: BLE001
logger.exception("Speech Engine WebSocket session failed")
await handle_session_close(session)
@@ -0,0 +1,89 @@
"""Standalone ElevenLabs Speech Engine WebSocket server (port 8002).
ElevenLabs connects here for transcripts; the dashboard stays on 8001.
Official SDK path: engine.serve() — more reliable than in-app Starlette /ws.
"""
from __future__ import annotations
import asyncio
import os
from pathlib import Path
_ROOT = Path(__file__).resolve().parent
if str(_ROOT) not in __import__("sys").path:
__import__("sys").path.insert(0, str(_ROOT))
def _load_env() -> None:
env_path = _ROOT / ".env"
if not env_path.is_file():
return
for raw in env_path.read_text(encoding="utf-8").splitlines():
line = raw.strip()
if not line or line.startswith("#") or "=" not in line:
continue
key, _, value = line.partition("=")
key, value = key.strip(), value.strip()
if key:
os.environ[key] = value
async def _main() -> None:
_load_env()
api_key = os.getenv("ELEVENLABS_API_KEY", "").strip()
engine_id = os.getenv("SPEECH_ENGINE_ID", "").strip()
port = int(os.getenv("SPEECH_ENGINE_PORT", "8002"))
disable_auth = os.getenv("SPEECH_ENGINE_DISABLE_AUTH", "").lower() in (
"1",
"true",
"yes",
)
if not api_key or not engine_id:
raise SystemExit("Set ELEVENLABS_API_KEY and SPEECH_ENGINE_ID in .env")
from elevenlabs import AsyncElevenLabs
from integrations.voice.speech_engine import (
handle_session_close,
handle_session_disconnect,
handle_session_init,
handle_user_transcript,
)
client = AsyncElevenLabs(api_key=api_key)
engine = await client.speech_engine.get(engine_id)
async def on_transcript(transcript, session): # noqa: ANN001
await handle_user_transcript(transcript, session)
async def on_init(conversation_id, session): # noqa: ANN001
await handle_session_init(conversation_id, session)
async def on_close(session): # noqa: ANN001
await handle_session_close(session)
async def on_disconnect(session): # noqa: ANN001
await handle_session_disconnect(session)
async def on_error(err, session): # noqa: ANN001
print(f"[speech sidecar] error: {err}", flush=True)
debug = os.getenv("SPEECH_ENGINE_DEBUG", "true").lower() in ("1", "true", "yes")
print(f"[speech sidecar] listening on ws://127.0.0.1:{port}/ws (debug={debug})", flush=True)
await engine.serve(
port=port,
path="/ws",
debug=debug,
disable_auth=disable_auth,
on_init=on_init,
on_transcript=on_transcript,
on_close=on_close,
on_disconnect=on_disconnect,
on_error=on_error,
)
if __name__ == "__main__":
asyncio.run(_main())
+247 -40
View File
@@ -18,13 +18,14 @@ function Load-DotEnv {
$line = $_.Trim()
if (-not $line -or $line.StartsWith("#") -or $line -notmatch "=") { return }
$key, $value = $line -split "=", 2
if ($key -and -not [string]::IsNullOrWhiteSpace($key) -and -not (Get-Item "Env:$key" -ErrorAction SilentlyContinue)) {
$key = $key.Trim()
if ($key -and -not [string]::IsNullOrWhiteSpace($key)) {
Set-Item -Path "Env:$key" -Value $value.Trim()
}
}
}
function Test-PortListening([int]$p) {
function Test-AppHttpReady([int]$p) {
foreach ($path in @("/health/live", "/health", "/api/voice/config")) {
try {
$r = Invoke-WebRequest -Uri "http://127.0.0.1:$p$path" -UseBasicParsing -TimeoutSec 15
@@ -34,6 +35,33 @@ function Test-PortListening([int]$p) {
return $false
}
function Test-TcpPortListening([int]$p) {
try {
$conn = Get-NetTCPConnection -LocalPort $p -State Listen -ErrorAction SilentlyContinue | Select-Object -First 1
if ($conn) { return $true }
} catch { }
try {
$client = New-Object System.Net.Sockets.TcpClient
$async = $client.BeginConnect("127.0.0.1", $p, $null, $null)
$ok = $async.AsyncWaitHandle.WaitOne(500)
if ($ok -and $client.Connected) {
$client.Close()
return $true
}
$client.Close()
} catch { }
return $false
}
function Wait-PortReleased([int]$p, [int]$maxSec = 20) {
$deadline = (Get-Date).AddSeconds($maxSec)
while ((Get-Date) -lt $deadline) {
if (-not (Test-TcpPortListening $p)) { return $true }
Start-Sleep -Seconds 1
}
return -not (Test-TcpPortListening $p)
}
function Stop-PortProcess([int]$p) {
try {
$conns = Get-NetTCPConnection -LocalPort $p -State Listen -ErrorAction SilentlyContinue
@@ -69,10 +97,10 @@ function Start-VoiceApp([int]$p) {
-RedirectStandardOutput $appLog -RedirectStandardError $appErr
$deadline = (Get-Date).AddSeconds(120)
while ((Get-Date) -lt $deadline) {
if (Test-PortListening $p) { break }
if (Test-AppHttpReady $p) { break }
Start-Sleep -Seconds 3
}
if (-not (Test-PortListening $p)) {
if (-not (Test-AppHttpReady $p)) {
$errText = ""
if (Test-Path $appErr) { $errText = Get-Content $appErr -Raw -ErrorAction SilentlyContinue }
if ($errText -match "Uvicorn running") {
@@ -80,7 +108,7 @@ function Start-VoiceApp([int]$p) {
Start-Sleep -Seconds 10
}
}
if (-not (Test-PortListening $p)) {
if (-not (Test-AppHttpReady $p)) {
if (Test-Path $appErr) {
Write-Host "App startup log:" -ForegroundColor Red
Get-Content $appErr -Tail 20 | ForEach-Object { Write-Host $_ }
@@ -90,6 +118,164 @@ function Start-VoiceApp([int]$p) {
Write-Host "App ready: http://127.0.0.1:$p"
}
function Get-CloudflaredPidFile([string]$suffix) {
return Join-Path $Root ".cloudflared$suffix.pid"
}
function Stop-CloudflaredForSuffix([string]$suffix) {
$pidFile = Get-CloudflaredPidFile $suffix
try {
if (Test-Path $pidFile) {
$procId = [int](Get-Content $pidFile -Raw).Trim()
if ($procId) {
Stop-Process -Id $procId -Force -ErrorAction SilentlyContinue
}
Remove-Item $pidFile -Force -ErrorAction SilentlyContinue
}
Start-Sleep -Seconds 1
} catch { }
}
function Stop-CloudflaredTunnel {
Stop-CloudflaredForSuffix ""
}
function Test-TunnelUrl([string]$baseUrl, [int]$retries = 2) {
if (-not $baseUrl) { return $false }
for ($i = 0; $i -lt $retries; $i++) {
try {
$null = Invoke-WebRequest -Uri "$baseUrl/api/voice/config" -UseBasicParsing -TimeoutSec 8
return $true
} catch {
if ($i -lt ($retries - 1)) { Start-Sleep -Seconds 2 }
}
}
return $false
}
function Test-SpeechTunnelUrl([string]$baseUrl, [int]$retries = 12) {
if (-not $baseUrl) { return $false }
$paths = @("", "/ws")
for ($i = 0; $i -lt $retries; $i++) {
foreach ($path in $paths) {
$uri = if ($path) { "$baseUrl$path" } else { $baseUrl }
try {
$null = Invoke-WebRequest -Uri $uri -UseBasicParsing -TimeoutSec 10
return $true
} catch {
$code = $null
try {
if ($_.Exception.Response) {
$code = [int]$_.Exception.Response.StatusCode
}
} catch { }
# Sidecar returns 404 on / — that still means the tunnel is up.
if ($code -and $code -ge 200 -and $code -lt 500) {
return $true
}
}
}
if ($i -lt ($retries - 1)) {
Write-Host " Speech tunnel not ready yet (attempt $($i + 1)/$retries)..."
Start-Sleep -Seconds 3
}
}
return $false
}
function Update-EnvValue([string]$key, [string]$value) {
$envFile = Join-Path $Root ".env"
if (-not (Test-Path $envFile)) { return }
$lines = Get-Content $envFile
$updated = $false
$newLines = foreach ($line in $lines) {
if ($line -match "^\s*$key=") {
$updated = $true
"$key=$value"
} else { $line }
}
if (-not $updated) { $newLines += "$key=$value" }
Set-Content -Path $envFile -Value ($newLines -join "`n")
Set-Item -Path "Env:$key" -Value $value
}
function Start-SpeechSidecar([int]$p = 8002, [switch]$Force) {
if ((Test-TcpPortListening $p) -and -not $Force) {
Write-Host "Speech Engine sidecar already listening on port $p"
return
}
Write-Host "Starting Speech Engine sidecar on port $p..."
Stop-PortProcess $p
if (-not (Wait-PortReleased $p)) {
Write-Host "Port $p still in use — forcing stop again..." -ForegroundColor Yellow
Stop-PortProcess $p
Start-Sleep -Seconds 3
}
$log = Join-Path $Root ".speech-sidecar.log"
$logErr = "${log}.err"
Clear-LogFile $log
Clear-LogFile $logErr
$env:PYTHONUNBUFFERED = "1"
Start-Process -FilePath "py" -ArgumentList "-3.13", "-u", "speech_engine_sidecar.py" `
-WorkingDirectory $Root -WindowStyle Minimized `
-RedirectStandardOutput $log -RedirectStandardError $logErr
$deadline = (Get-Date).AddSeconds(90)
while ((Get-Date) -lt $deadline) {
if (Test-TcpPortListening $p) {
Write-Host "Speech Engine sidecar ready on port $p"
return
}
if (Test-Path $log) {
$tail = Get-Content $log -Tail 1 -ErrorAction SilentlyContinue
if ($tail) { Write-Host " $tail" }
}
Start-Sleep -Seconds 2
}
Write-Host "Sidecar logs:" -ForegroundColor Red
if (Test-Path $log) {
Get-Content $log -Tail 15 -ErrorAction SilentlyContinue | ForEach-Object { Write-Host $_ }
}
if (Test-Path $logErr) {
Get-Content $logErr -Tail 15 -ErrorAction SilentlyContinue | ForEach-Object { Write-Host $_ }
}
throw "Speech Engine sidecar did not start on port $p (ElevenLabs API fetch can take up to 90s — check .speech-sidecar.log)"
}
function Start-CloudflaredTunnel([int]$p, [string]$logSuffix = "") {
Write-Host "Starting Cloudflare tunnel for port $p..."
$log = Join-Path $Root ".tunnel$logSuffix.log"
$logErr = "${log}.err"
Stop-CloudflaredForSuffix $logSuffix
Clear-LogFile $log
Clear-LogFile $logErr
$proc = Start-Process -FilePath "cloudflared" -ArgumentList "tunnel", "--url", "http://127.0.0.1:$p" `
-RedirectStandardOutput $log -RedirectStandardError $logErr -WindowStyle Minimized -PassThru
if ($proc -and $proc.Id) {
Set-Content -Path (Get-CloudflaredPidFile $logSuffix) -Value $proc.Id
}
$deadline = (Get-Date).AddSeconds(90)
$newUrl = $null
while ((Get-Date) -lt $deadline -and -not $newUrl) {
Start-Sleep -Seconds 2
foreach ($path in @($logErr, $log)) {
if (-not (Test-Path $path)) { continue }
$match = Select-String -Path $path -Pattern "https://[a-z0-9]+(-[a-z0-9]+)+\.trycloudflare\.com" | Select-Object -First 1
if ($match) {
$newUrl = $match.Matches[0].Value
break
}
}
}
if (-not $newUrl) { throw "Could not read tunnel URL from $logErr" }
if ($logSuffix) {
Write-Host "Tunnel ready ($logSuffix): $newUrl"
} else {
Update-PublicUrl $newUrl
Write-Host "Tunnel ready: $newUrl"
}
return $newUrl
}
function Update-PublicUrl([string]$baseUrl) {
$envFile = Join-Path $Root ".env"
$content = Get-Content $envFile -Raw
@@ -147,7 +333,7 @@ Ensure-ServerSecret
$env:VOICE_AGENT_PORT = "$Port"
$needsRestart = $Restart.IsPresent
if (Test-PortListening $Port) {
if (Test-TcpPortListening $Port) {
if ($needsRestart) {
Stop-PortProcess $Port
} else {
@@ -166,42 +352,20 @@ if (Test-PortListening $Port) {
}
}
if ($needsRestart -or -not (Test-PortListening $Port)) {
Start-VoiceApp $Port
}
$public = $env:PUBLIC_API_BASE_URL
try {
$null = Invoke-WebRequest -Uri "$public/api/voice/config" -UseBasicParsing -TimeoutSec 5
$mainTunnelRecreated = $false
if (Test-TunnelUrl $public) {
Write-Host "Tunnel OK: $public"
} catch {
Write-Host "Starting Cloudflare tunnel..."
$log = Join-Path $Root ".tunnel.log"
$logErr = "${log}.err"
if (Test-Path $log) { Remove-Item $log -Force }
if (Test-Path $logErr) { Remove-Item $logErr -Force }
Start-Process -FilePath "cloudflared" -ArgumentList "tunnel", "--url", "http://127.0.0.1:$Port" `
-RedirectStandardOutput $log -RedirectStandardError $logErr -WindowStyle Minimized
$deadline = (Get-Date).AddSeconds(90)
$newUrl = $null
while ((Get-Date) -lt $deadline -and -not $newUrl) {
Start-Sleep -Seconds 2
foreach ($path in @($logErr, $log)) {
if (-not (Test-Path $path)) { continue }
$match = Select-String -Path $path -Pattern "https://[a-z0-9-]+\.trycloudflare\.com" | Select-Object -First 1
if ($match) {
$newUrl = $match.Matches[0].Value
break
}
}
}
if (-not $newUrl) { throw "Could not read tunnel URL from $logErr" }
Update-PublicUrl $newUrl
Write-Host "Tunnel ready: $newUrl"
if ($needsRestart -or -not (Test-PortListening $Port)) {
Stop-PortProcess $Port
Start-VoiceApp $Port
}
} else {
$null = Start-CloudflaredTunnel $Port
$mainTunnelRecreated = $true
}
if ($needsRestart -or -not (Test-TcpPortListening $Port)) {
Start-VoiceApp $Port
} elseif ($mainTunnelRecreated) {
Write-Host "Main tunnel URL changed — restarting app to reload .env"
Stop-PortProcess $Port
Start-VoiceApp $Port
}
$config = Invoke-RestMethod -Uri "http://127.0.0.1:$Port/api/voice/config"
@@ -218,4 +382,47 @@ if ($LASTEXITCODE -eq 0) {
Show-ProviderSetupSteps -webhookUrl $webhook -secret $env:VOICE_SERVER_URL_SECRET
}
if ($env:ELEVENLABS_API_KEY) {
Write-Host ""
$speechSidecarOk = Test-TcpPortListening 8002
$speechTunnelOk = Test-SpeechTunnelUrl $env:SPEECH_ENGINE_PUBLIC_URL
if (-not $speechTunnelOk) {
Write-Host "Speech tunnel down — starting sidecar + new Cloudflare tunnel..." -ForegroundColor Cyan
Start-SpeechSidecar 8002 -Force
$speechTunnel = Start-CloudflaredTunnel 8002 "-speech"
Update-EnvValue "SPEECH_ENGINE_PUBLIC_URL" $speechTunnel
Write-Host "Speech Engine tunnel: $speechTunnel"
Write-Host "Waiting for speech tunnel to accept connections..."
Start-Sleep -Seconds 5
Write-Host "Configuring ElevenLabs Speech Engine..." -ForegroundColor Cyan
py -3.13 setup_speech_engine.py
if ($LASTEXITCODE -ne 0) {
throw "Speech Engine setup failed — run: py -3.13 setup_speech_engine.py"
}
Write-Host "Speech Engine configured (WebSocket /ws + browser token)." -ForegroundColor Green
Load-DotEnv
Stop-PortProcess $Port
Start-VoiceApp $Port
} elseif ($Restart.IsPresent -or -not $speechSidecarOk) {
Write-Host "Restarting speech sidecar to reload code (keeping existing tunnel)..." -ForegroundColor Yellow
Start-SpeechSidecar 8002 -Force
} else {
Write-Host "Speech stack OK (sidecar :8002 + tunnel reachable)" -ForegroundColor Green
}
if (-not (Test-TcpPortListening 8002)) {
throw "Speech Engine sidecar not listening on 8002"
}
Load-DotEnv
if (-not (Test-SpeechTunnelUrl $env:SPEECH_ENGINE_PUBLIC_URL)) {
Write-Host ""
Write-Host "WARNING: Speech Engine tunnel not reachable yet: $($env:SPEECH_ENGINE_PUBLIC_URL)" -ForegroundColor Yellow
Write-Host "Local dashboard is up at http://127.0.0.1:$Port — wait ~30s, then retry ElevenLabs talk." -ForegroundColor Yellow
Write-Host "If it still fails, run: .\start_dev.ps1 -Restart" -ForegroundColor Yellow
} else {
Write-Host "Speech sidecar: port 8002 | ws_url wss://$($env:SPEECH_ENGINE_PUBLIC_URL -replace 'https://','')/ws"
}
}
Write-Host "Done. Dashboard: http://127.0.0.1:$Port"
Write-Host "ElevenLabs talk: http://127.0.0.1:$Port/eleven-talk"
Write-Host "Keep this window open — sidecar + tunnels run in the background."
+79 -9
View File
@@ -7,12 +7,13 @@ import json
import logging
from integrations.voice.client import VoiceClient
from integrations.voice.config import VoiceConfigError, load_voice_settings
from integrations.voice.config import VoiceConfigError, get_web_sdk_settings, load_voice_settings
from integrations.voice.live import transcript_hub
from integrations.voice.processor import process_voice_webhook
from integrations.voice.store import VoiceCallStore
from integrations.voice.verify import VerificationError, verify_webhook_request
from starlette.requests import Request
from starlette.responses import JSONResponse, PlainTextResponse, Response
from starlette.responses import JSONResponse, PlainTextResponse, Response, StreamingResponse
logger = logging.getLogger(__name__)
@@ -21,6 +22,14 @@ logger = logging.getLogger(__name__)
_background_tasks: set[asyncio.Task] = set()
def _schedule_webhook_processing(event_type: str, payload: dict, *, settings) -> None:
task = asyncio.create_task(
asyncio.to_thread(process_voice_webhook, event_type, payload, settings=settings)
)
_background_tasks.add(task)
task.add_done_callback(_background_tasks.discard)
async def api_voice_config_status(_request: Request) -> JSONResponse:
try:
settings = load_voice_settings()
@@ -36,6 +45,24 @@ async def api_voice_config_status(_request: Request) -> JSONResponse:
"error": error,
"webhook_url": settings.webhook_url if settings else None,
"default_assistant_id": settings.default_assistant_id if settings else None,
"web_talk_enabled": bool(settings.public_api_key and settings.default_assistant_id),
}
)
async def api_voice_web_config(_request: Request) -> JSONResponse:
try:
settings = load_voice_settings(require_key=False)
except VoiceConfigError as exc:
return JSONResponse({"enabled": False, "error": str(exc)})
sdk_url, sdk_global = get_web_sdk_settings()
return JSONResponse(
{
"enabled": bool(settings.public_api_key and settings.default_assistant_id and sdk_url),
"public_key": settings.public_api_key,
"assistant_id": settings.default_assistant_id,
"sdk_url": sdk_url or None,
"sdk_global": sdk_global or "Vapi",
}
)
@@ -69,15 +96,62 @@ async def api_voice_create_call(request: Request) -> JSONResponse:
async def api_voice_list_calls(_request: Request) -> JSONResponse:
return JSONResponse({"calls": VoiceCallStore.list_calls()})
from integrations.voice.call_finalize import analytics_for_record
calls = []
for record in VoiceCallStore.list_calls():
row = dict(record)
row["analytics"] = analytics_for_record(record)
calls.append(row)
return JSONResponse({"calls": calls})
async def api_voice_doctor(_request: Request) -> JSONResponse:
import os
from integrations.voice.doctor import run_voice_doctor
app_port = int(os.getenv("VOICE_AGENT_PORT", "8001"))
sidecar_port = int(os.getenv("SPEECH_ENGINE_SIDECAR_PORT", "8002"))
return JSONResponse(run_voice_doctor(app_port=app_port, sidecar_port=sidecar_port))
async def api_voice_call_detail(request: Request) -> JSONResponse:
from integrations.voice.call_finalize import analytics_for_record
call_id = request.path_params["call_id"]
record = VoiceCallStore.get_call(call_id)
if not record:
return JSONResponse({"error": "not found"}, status_code=404)
return JSONResponse(record)
out = dict(record)
out["analytics"] = analytics_for_record(record)
return JSONResponse(out)
async def api_voice_call_detail_ui(request: Request) -> JSONResponse:
from integrations.voice.call_detail_ui import build_call_detail_layout
call_id = request.path_params["call_id"]
record = VoiceCallStore.get_call(call_id)
if not record:
return JSONResponse({"error": "not found"}, status_code=404)
layout = build_call_detail_layout(record)
return JSONResponse({"call_id": call_id, "layout": layout})
async def api_voice_live_transcript(request: Request) -> StreamingResponse:
call_id = request.path_params["call_id"]
async def event_stream():
record = VoiceCallStore.get_call(call_id)
if record and record.get("transcript"):
yield transcript_hub.snapshot_sse(
{"type": "snapshot", "transcript": record["transcript"]}
)
async for event in transcript_hub.subscribe(call_id):
yield transcript_hub.snapshot_sse(event)
return StreamingResponse(event_stream(), media_type="text/event-stream")
async def webhook_voice(request: Request) -> Response:
@@ -105,9 +179,5 @@ async def webhook_voice(request: Request) -> Response:
result = await asyncio.to_thread(process_voice_webhook, event_type, payload, settings=settings)
return JSONResponse(result or {})
task = asyncio.create_task(
asyncio.to_thread(process_voice_webhook, event_type, payload, settings=settings)
)
_background_tasks.add(task)
task.add_done_callback(_background_tasks.discard)
_schedule_webhook_processing(event_type, payload, settings=settings)
return PlainTextResponse("ok", status_code=200)
+1
View File
@@ -129,6 +129,7 @@ nav:
- Feature Inventory: features/feature-inventory.md
- Gap Analysis: features/gap-analysis.md
- Integration Analysis: features/integration-analysis.md
- BeautifulUI Integration: features/beautifului-integration.md
- Standardization: features/standardization-verification.md
- Comparison (OpenClaw): features/comparison-openclaw.md
- What Makes It Different: features/differentiation.md
+1 -1
View File
@@ -1,6 +1,6 @@
[project]
name = "aiui"
version = "0.3.143"
version = "0.3.146"
description = "YAML-driven website generator - CLI and compiler"
readme = "README.md"
license = {text = "MIT"}
+1 -1
View File
@@ -1,3 +1,3 @@
"""Version information."""
__version__ = "0.3.143"
__version__ = "0.3.146"
+48
View File
@@ -44,6 +44,43 @@ def _icon(symbol: str, fallback: str) -> str:
return symbol if _supports_unicode() else fallback
def _load_env_files() -> int:
"""Load environment variables from the standard PraisonAI ``.env`` files.
Mirrors the PraisonAI CLI convention so ``env:VAR`` references (e.g. in
Channels config) resolve when the server was started without the shell
exporting the tokens. Reads ``~/.praisonai/.env`` first, then a ``.env`` in
the current working directory. Existing process env vars are never
overridden, and parsing is intentionally dependency-free (no python-dotenv).
Returns the number of variables newly set into ``os.environ``.
"""
import os
loaded = 0
for env_path in (Path.home() / ".praisonai" / ".env", Path.cwd() / ".env"):
try:
if not env_path.is_file():
continue
for line in env_path.read_text(encoding="utf-8").splitlines():
line = line.strip()
if not line or line.startswith("#") or "=" not in line:
continue
if line.startswith("export "):
line = line[len("export "):]
key, value = line.split("=", 1)
key = key.strip()
value = value.strip().strip('"').strip("'")
if key and key not in os.environ:
os.environ[key] = value
loaded += 1
except Exception:
# Never let env loading break startup — resolution falls back to
# the existing process environment.
continue
return loaded
app = typer.Typer(
name="aiui",
help="PraisonAIUI - YAML-driven website generator",
@@ -1380,6 +1417,17 @@ def run(
console.print(f"[red]Error:[/red] App file not found: {app_file}")
raise typer.Exit(code=1)
# Auto-load standard PraisonAI env files so `env:VAR` references (e.g. in
# Channels config) resolve even when the shell did not export the tokens.
# This matches the PraisonAI CLI convention and fixes the "Connected but
# bot never replies" class of issues caused by empty env-resolved tokens.
_loaded = _load_env_files()
if _loaded:
console.print(
f"[green]{_icon('✓', 'OK')}[/green] Loaded {_loaded} environment "
f"variable(s) from PraisonAI config"
)
is_yaml = app_file.suffix in (".yaml", ".yml")
if is_yaml:
+16 -11
View File
@@ -155,12 +155,14 @@ class SDKFileDataStore(BaseDataStore):
role = message.get("role", "user")
content = message.get("content", "")
# UI-specific fields go into metadata
metadata: dict[str, Any] = {}
if "toolCalls" in message:
metadata["toolCalls"] = message["toolCalls"]
if "elements" in message:
metadata["elements"] = message["elements"]
# All non-core fields (toolCalls, elements, platform, sender, icon,
# channel_id, …) are preserved in metadata so channel and voice
# context survives reload — matching the SQLAlchemy backend.
metadata: dict[str, Any] = {
k: v
for k, v in message.items()
if k not in ("role", "content") and v is not None
}
await asyncio.to_thread(
self._store.add_message,
@@ -290,10 +292,13 @@ class SDKFileDataStore(BaseDataStore):
"role": msg.role,
"content": msg.content,
}
# Restore UI-specific fields from metadata
if "toolCalls" in meta:
d["toolCalls"] = meta["toolCalls"]
if "elements" in meta:
d["elements"] = meta["elements"]
# Restore all persisted metadata (toolCalls, elements, platform,
# sender, icon, channel_id, …). Internal sentinel keys never
# appear here (the sentinel message is skipped above), but guard
# against them defensively without clobbering role/content.
for k, v in meta.items():
if k in (_META_KEY, _TITLE_KEY, _CUSTOM_META_KEY):
continue
d.setdefault(k, v)
messages.append(d)
return messages
+40 -3
View File
@@ -315,6 +315,13 @@ class ChannelsFeature(BaseFeatureProtocol):
Returns None on success, or an error string on failure.
"""
# Idempotent start (#277): stop any bot already running for this channel
# before creating a new one. Without this, a toggle off/on, a re-add, or
# an autostart+manual race leaves a duplicate bot with duplicate event
# handlers attached while _live_bots only tracks the latest task.
if channel_id in _live_bots:
await self._stop_channel_bot(channel_id)
platform = entry["platform"]
config = entry.get("config", {})
fallback_errors: List[str] = []
@@ -645,13 +652,24 @@ class ChannelsFeature(BaseFeatureProtocol):
if hasattr(bot, "_session") and hasattr(bot._session, "chat"):
_original_chat = bot._session.chat
async def _wrapped_chat(agent: Any, user_id: str, text: str) -> str:
async def _wrapped_chat(
agent: Any,
user_id: str,
text: str,
*args: Any,
**kwargs: Any,
) -> str:
"""Wrap agent chat to broadcast responses AND intermediate
tool/step events to the Chat UI.
Hooks into the agent's StreamEventEmitter (same pattern as
PraisonAIProvider._run_direct_mode) to capture tool_call,
reasoning, and content events during execution.
Any extra positional/keyword arguments (e.g. ``chat_id``,
``platform``, ``thread_id``) are forwarded untouched to the
wrapped session ``chat`` so per-conversation memory/session
continuity is preserved instead of raising ``TypeError``.
"""
from .chat import _enrich_tool_payload, get_chat_manager
@@ -824,7 +842,7 @@ class ChannelsFeature(BaseFeatureProtocol):
drain_task = asyncio.create_task(_drain_events())
try:
response = await _original_chat(agent, user_id, text)
response = await _original_chat(agent, user_id, text, *args, **kwargs)
finally:
# Signal drain to stop and clean up callback
try:
@@ -905,9 +923,15 @@ class ChannelsFeature(BaseFeatureProtocol):
try:
if hasattr(bot, "stop"):
await bot.stop()
# Cancel the task if still running
# Cancel the task and await it so the old bot is fully torn down
# before this returns — otherwise a restart (#277) can spawn a new
# bot while the previous task is still cancelling (a zombie).
if info and info.get("task") and not info["task"].done():
info["task"].cancel()
try:
await info["task"]
except asyncio.CancelledError:
pass
logger.info(f"Stopped bot for channel '{channel_id}'")
except Exception as e:
logger.error(f"Error stopping bot for '{channel_id}': {e}")
@@ -918,6 +942,19 @@ class ChannelsFeature(BaseFeatureProtocol):
gw = self._get_gateway()
if gw is not None:
getattr(gw, "_channel_bots", {}).pop(channel_id, None)
# Mirror start-time registration: drop the task from the gateway
# too, so a restart (#277) leaves no stale/cancelled task behind.
# The dict form is overwritten on restart, but the list form would
# otherwise accumulate a cancelled task on every restart.
channel_tasks = getattr(gw, "_channel_tasks", None)
task = info.get("task") if info else None
if isinstance(channel_tasks, dict):
channel_tasks.pop(channel_id, None)
elif isinstance(channel_tasks, list) and task is not None:
try:
channel_tasks.remove(task)
except ValueError:
pass
# Update local state
ch = _channels.get(channel_id)
+21
View File
@@ -462,6 +462,24 @@ async def _run_and_broadcast(
collected_tool_calls: Dict[str, Dict] = {} # Merge STARTED+COMPLETED per tool_call_id
collected_elements: List[Dict[str, Any]] = []
# Register this run so ``abort_run(run_id)`` can cancel it, and announce the
# run to WS clients up-front. Without this the ``_active_runs`` registry was
# never populated, so the Stop button and ``POST /api/chat/abort`` were
# silent no-ops that always returned ``no_active_run`` (issue #274). The
# ``run_started`` event also lets the frontend capture ``run_id`` and enable
# the Stop button immediately, before the first token arrives.
self_task = asyncio.current_task()
if self_task is not None:
mgr._active_runs[run_id] = self_task
started_event: Dict[str, Any] = {
"type": "run_started",
"session_id": session_id,
"run_id": run_id,
}
if agent_name:
started_event["agent_name"] = agent_name
await mgr.broadcast(session_id, started_event)
try:
# Pass image attachments to provider for SDK native multimodal handling
run_kwargs = {}
@@ -644,6 +662,9 @@ async def _run_and_broadcast(
"error": str(e),
},
)
finally:
# Always deregister the run, on normal completion, error, or abort.
mgr._active_runs.pop(run_id, None)
# Save assistant response
if full_response or collected_elements:
+21 -10
View File
@@ -168,7 +168,9 @@ async def run_startup_hooks() -> None:
"""Execute all registered startup hooks.
Called by the server during application startup.
Blocks until all hooks complete.
Has a timeout from AIUI_STARTUP_TIMEOUT (default 60s) so a hung hook
cannot block the server from reaching the lifespan ``yield`` — startup is
marked completed on timeout so ``/health/live`` still responds.
"""
if _lifecycle_state["startup_completed"]:
_log.warning("Startup hooks already executed")
@@ -177,15 +179,24 @@ async def run_startup_hooks() -> None:
_log.info(f"Running {len(_startup_hooks)} startup hooks")
start_time = time.perf_counter()
for hook in _startup_hooks:
try:
_log.debug(f"Executing startup hook: {getattr(hook, '__name__', str(hook))}")
result = hook()
if asyncio.iscoroutine(result):
await result
except Exception as e:
_log.error(f"Startup hook failed: {getattr(hook, '__name__', str(hook))}: {e}")
# Continue with other hooks - individual failures shouldn't break startup
# Get startup timeout from environment (mirrors AIUI_SHUTDOWN_TIMEOUT)
timeout = float(os.environ.get("AIUI_STARTUP_TIMEOUT", "60.0"))
async def _execute_hooks():
for hook in _startup_hooks:
try:
_log.debug(f"Executing startup hook: {getattr(hook, '__name__', str(hook))}")
result = hook()
if asyncio.iscoroutine(result):
await result
except Exception as e:
_log.error(f"Startup hook failed: {getattr(hook, '__name__', str(hook))}: {e}")
# Continue with other hooks - individual failures shouldn't break startup
try:
await asyncio.wait_for(_execute_hooks(), timeout=timeout)
except asyncio.TimeoutError:
_log.error(f"Startup hooks timed out after {timeout}s")
execution_time = time.perf_counter() - start_time
_lifecycle_state.update(
+70 -35
View File
@@ -9,6 +9,8 @@ Provides first-class MCP client functionality with:
from __future__ import annotations
import asyncio
import contextlib
import json
import logging
import subprocess
@@ -109,50 +111,52 @@ class StdioMCPClient:
self.args = args
self.session: Optional[ClientSession] = None
self._connected = False
self._session_manager = None
self._exit_stack: contextlib.AsyncExitStack | None = None
# For test backward compatibility
self.process: Optional[subprocess.Popen] = None
async def connect(self) -> bool:
"""Connect via stdio subprocess using official MCP SDK."""
# Store the stack on self before entering contexts so that a caller
# cancellation (e.g. asyncio.wait_for timeout, which raises the
# BaseException CancelledError past the except below) can still find
# and close the partially-entered stack via disconnect().
exit_stack = contextlib.AsyncExitStack()
self._exit_stack = exit_stack
try:
# Use official MCP stdio_client instead of raw subprocess
server_params = StdioServerParameters(command=self.command, args=self.args)
# Create a new session using stdio_client context manager
session_manager = stdio_client(server_params)
self.session = await session_manager.__aenter__()
# Initialize the MCP session
read_stream, write_stream = await exit_stack.enter_async_context(
stdio_client(server_params)
)
self.session = await exit_stack.enter_async_context(
ClientSession(read_stream, write_stream)
)
await self.session.initialize()
# Store the context manager for cleanup
self._session_manager = session_manager
self._connected = True
logger.info(f"Connected to MCP stdio server: {self.command}")
return True
except Exception as e:
logger.error(f"Failed to connect to MCP stdio server: {e}")
await exit_stack.aclose()
self._exit_stack = None
self._connected = False
self.session = None
return False
async def disconnect(self) -> None:
"""Properly disconnect the MCP session."""
if self.session and self._connected:
if self._exit_stack:
try:
# Properly exit the context manager
if hasattr(self, "_session_manager"):
await self._session_manager.__aexit__(None, None, None)
else:
await self.session.close()
await self._exit_stack.aclose()
except Exception as e:
logger.warning(f"Error during MCP session close: {e}")
finally:
self._exit_stack = None
self.session = None
self._connected = False
self.process = None # Reset for backward compatibility
self.process = None
async def list_tools(self) -> List[ToolInfo]:
"""List tools via proper MCP protocol."""
@@ -203,42 +207,45 @@ class SSEMCPClient:
self.headers = headers or {}
self.session: Optional[ClientSession] = None
self._connected = False
self._session_manager = None
self._exit_stack: contextlib.AsyncExitStack | None = None
async def connect(self) -> bool:
"""Connect via SSE using official MCP SDK."""
# Store the stack on self before entering contexts so a caller
# cancellation (e.g. asyncio.wait_for timeout) can still close the
# partially-entered stack via disconnect(). See StdioMCPClient.connect.
exit_stack = contextlib.AsyncExitStack()
self._exit_stack = exit_stack
try:
# Use official MCP sse_client context manager
session_manager = sse_client(self.url, headers=self.headers)
self.session = await session_manager.__aenter__()
# Initialize the MCP session
read_stream, write_stream = await exit_stack.enter_async_context(
sse_client(self.url, headers=self.headers)
)
self.session = await exit_stack.enter_async_context(
ClientSession(read_stream, write_stream)
)
await self.session.initialize()
# Store the context manager for cleanup
self._session_manager = session_manager
self._connected = True
logger.info(f"Connected to MCP SSE server at {self.url}")
return True
except Exception as e:
logger.error(f"Failed to connect to MCP SSE server: {e}")
await exit_stack.aclose()
self._exit_stack = None
self._connected = False
self.session = None
return False
async def disconnect(self) -> None:
"""Properly disconnect SSE."""
if self.session and self._connected:
if self._exit_stack:
try:
# Properly exit the context manager
if hasattr(self, "_session_manager") and self._session_manager:
await self._session_manager.__aexit__(None, None, None)
else:
await self.session.close()
await self._exit_stack.aclose()
except Exception as e:
logger.warning(f"Error during MCP SSE session close: {e}")
finally:
self._exit_stack = None
self.session = None
self._connected = False
@@ -354,6 +361,13 @@ class MCPClientManager:
connection_data=server_config,
)
# Disconnect any existing client for this name (reconnect) before
# replacing it, otherwise the previous stdio subprocess/streams are
# orphaned when the reference is overwritten.
previous_client = self._clients.pop(name, None)
if previous_client is not None:
await self._safe_disconnect(name, previous_client)
# Register server
_mcp_servers[name] = server
self._clients[name] = client
@@ -362,8 +376,7 @@ class MCPClientManager:
await self._notify_status_change(server)
try:
# Attempt connection
connected = await client.connect()
connected = await asyncio.wait_for(client.connect(), timeout=120.0)
if connected:
server.status = MCPStatus.CONNECTED
server.tools = await client.list_tools()
@@ -374,16 +387,38 @@ class MCPClientManager:
else:
server.status = MCPStatus.ERROR
server.last_error = "Connection failed"
await self._cleanup_failed_client(name, client)
except asyncio.TimeoutError:
logger.error("Timed out connecting MCP server %s", name)
server.status = MCPStatus.ERROR
server.last_error = "Connection timed out after 120s (check npx/Node and server args)"
await client.disconnect()
except Exception as e:
logger.exception(f"Failed to connect MCP server {name}")
server.status = MCPStatus.ERROR
server.last_error = str(e)
await self._cleanup_failed_client(name, client)
# Notify final status
await self._notify_status_change(server)
return server
async def _safe_disconnect(self, name: str, client: MCPClientProtocol) -> None:
"""Disconnect a client, logging (not raising) on failure."""
try:
await client.disconnect()
except Exception:
logger.exception(f"Error disconnecting MCP client {name}")
async def _cleanup_failed_client(self, name: str, client: MCPClientProtocol) -> None:
"""Release a client whose connection failed and drop it from the registry."""
await self._safe_disconnect(name, client)
# Only remove if this client is still the registered one (avoid evicting
# a newer client from a concurrent reconnect).
if self._clients.get(name) is client:
del self._clients[name]
async def disconnect_server(
self, name: str, session_context: Optional[Dict[str, Any]] = None
) -> bool:
@@ -252,6 +252,7 @@ window.aiui.registerSurfaceRenderer = function(surfaceId, renderFn) {
window.aiui.surfaces = SURFACE_REGISTRY;
window.aiui.version = '1';
window.aiui.components = COMPONENT_REGISTRY;
window.aiui.renderComponent = renderComponent;
window.aiui.registerSlot = function (name, renderFn) { SLOT_REGISTRY[name] = renderFn; };
window.aiui.slots = SLOT_REGISTRY;
window.aiui.sdk = aiuiSdk;
@@ -116,7 +116,7 @@ function openConnectForm(host, container) {
<h3 style="margin:0 0 16px;font-size:14px;font-weight:600">Connect MCP Server</h3>
<div class="db-form-group">
<label class="db-form-label">Name</label>
<input id="mcp-f-name" class="db-form-input" placeholder="my-server">
<input id="mcp-f-name" class="db-form-input" value="everything-demo" placeholder="my-server">
</div>
<div class="db-form-group">
<label class="db-form-label">Transport</label>
@@ -128,11 +128,11 @@ function openConnectForm(host, container) {
<div id="mcp-f-stdio">
<div class="db-form-group">
<label class="db-form-label">Command</label>
<input id="mcp-f-command" class="db-form-input" placeholder="npx">
<input id="mcp-f-command" class="db-form-input" value="npx" placeholder="npx" required>
</div>
<div class="db-form-group">
<label class="db-form-label">Arguments (space-separated)</label>
<input id="mcp-f-args" class="db-form-input" placeholder="-y @modelcontextprotocol/server-filesystem /tmp">
<input id="mcp-f-args" class="db-form-input" value="-y @modelcontextprotocol/server-everything" placeholder="-y @modelcontextprotocol/server-everything">
</div>
</div>
<div id="mcp-f-url" style="display:none">
+68
View File
@@ -144,6 +144,74 @@ async def test_auto_start_skips_disabled_channels():
assert mock_start.call_count == 0
@pytest.mark.asyncio
async def test_start_channel_bot_is_idempotent_stops_existing_first():
"""Regression for #277: starting a bot for a channel that already has a
live bot must stop the existing one first so no duplicate bot/task lingers.
"""
feature = ChannelsFeature()
gw = _GatewayDictTasks()
entry = {"id": "ch6", "platform": "discord", "config": {"bot_token": "x" * 30}}
with patch.object(feature, "_get_gateway", return_value=gw), patch.object(
feature, "_create_bot_direct", side_effect=lambda *a, **k: _FakeBot()
), patch.object(feature, "_attach_chat_bridge"):
await feature._start_channel_bot("ch6", entry)
first = channels_mod._live_bots["ch6"]
await feature._start_channel_bot("ch6", entry)
second = channels_mod._live_bots["ch6"]
# A new bot/task replaced the first, and only one entry is tracked.
assert second is not first
assert first["bot"] is not second["bot"]
assert list(channels_mod._live_bots.keys()).count("ch6") == 1
# The first task was cancelled by the pre-start _stop_channel_bot guard.
assert first["task"].cancelled() or first["task"].done()
# Gateway has exactly one (fresh) bot reference — no stale entry.
assert gw._channel_bots["ch6"] is second["bot"]
# ...and exactly one (fresh) task reference in the dict registry.
assert gw._channel_tasks["ch6"] is second["task"]
@pytest.mark.asyncio
async def test_restart_leaves_no_stale_task_in_list_gateway():
"""Regression for #277: with a list-based gateway _channel_tasks, a restart
must not accumulate the old cancelled task alongside the fresh one.
"""
feature = ChannelsFeature()
gw = _GatewayListTasks()
entry = {"id": "ch8", "platform": "discord", "config": {"bot_token": "x" * 30}}
with patch.object(feature, "_get_gateway", return_value=gw), patch.object(
feature, "_create_bot_direct", side_effect=lambda *a, **k: _FakeBot()
), patch.object(feature, "_attach_chat_bridge"):
await feature._start_channel_bot("ch8", entry)
await feature._start_channel_bot("ch8", entry)
current = channels_mod._live_bots["ch8"]
# Only the fresh task remains in the list — the cancelled one was removed.
assert gw._channel_tasks == [current["task"]]
@pytest.mark.asyncio
async def test_start_channel_bot_no_existing_does_not_stop():
"""Guard must be a no-op on first start (nothing in _live_bots)."""
feature = ChannelsFeature()
gw = _GatewayDictTasks()
entry = {"id": "ch7", "platform": "discord", "config": {"bot_token": "x" * 30}}
with patch.object(feature, "_get_gateway", return_value=gw), patch.object(
feature, "_create_bot_direct", return_value=_FakeBot()
), patch.object(feature, "_attach_chat_bridge"), patch.object(
feature, "_stop_channel_bot"
) as mock_stop:
error = await feature._start_channel_bot("ch7", entry)
assert error is None
mock_stop.assert_not_called()
@pytest.mark.asyncio
async def test_auto_start_failure_sets_not_running_and_error():
feature = ChannelsFeature()
+88
View File
@@ -0,0 +1,88 @@
"""Regression coverage for issue #287: the channels chat bridge dropped
extra keyword arguments (e.g. ``chat_id``, ``platform``) because the
``_wrapped_chat`` wrapper installed over ``bot._session.chat`` had a fixed
``(agent, user_id, text)`` signature.
When the bot internals invoked the session with per-conversation kwargs the
wrapper raised ``TypeError: _wrapped_chat() got an unexpected keyword
argument 'chat_id'`` and the bot silently failed to reply. The wrapper must
forward any extra positional/keyword arguments untouched to the original
session ``chat`` so memory/session continuity is preserved.
"""
from __future__ import annotations
from typing import Any
import pytest
from praisonaiui.features.channels import ChannelsFeature
class _FakeSession:
"""Minimal session whose ``chat`` records forwarded arguments."""
def __init__(self) -> None:
self.received_args: tuple = ()
self.received_kwargs: dict = {}
async def chat(self, agent: Any, user_id: str, text: str, *args: Any, **kwargs: Any) -> str:
self.received_args = args
self.received_kwargs = dict(kwargs)
return f"reply to {text}"
class _FakeBot:
def __init__(self) -> None:
self._session = _FakeSession()
@pytest.mark.asyncio
async def test_wrapped_chat_forwards_kwargs() -> None:
feature = ChannelsFeature()
bot = _FakeBot()
feature._attach_chat_bridge("chan-1", bot, "telegram")
# The bridge replaced the session chat with the wrapper.
assert bot._session.chat.__name__ == "_wrapped_chat"
reply = await bot._session.chat(
object(),
"user-42",
"Hi",
chat_id="987654321",
platform="telegram",
user_id_meta="987654321",
)
assert reply == "reply to Hi"
assert bot._session.received_kwargs["chat_id"] == "987654321"
assert bot._session.received_kwargs["platform"] == "telegram"
@pytest.mark.asyncio
async def test_wrapped_chat_forwards_extra_positional_args() -> None:
feature = ChannelsFeature()
bot = _FakeBot()
feature._attach_chat_bridge("chan-2", bot, "slack")
reply = await bot._session.chat(object(), "user-7", "Hello", "extra")
assert reply == "reply to Hello"
assert bot._session.received_args == ("extra",)
@pytest.mark.asyncio
async def test_wrapped_chat_still_works_without_kwargs() -> None:
"""No-kwargs path (web UI / existing callers) must keep working."""
feature = ChannelsFeature()
bot = _FakeBot()
feature._attach_chat_bridge("chan-3", bot, "discord")
reply = await bot._session.chat(object(), "user-9", "Ping")
assert reply == "reply to Ping"
assert bot._session.received_kwargs == {}
+145
View File
@@ -0,0 +1,145 @@
"""Tests for chat run abort / cancellation (issue #274).
Regression coverage for the wiring bug where ``ChatManager._active_runs`` was
never populated, so ``abort_run`` (Stop button / ``POST /api/chat/abort`` /
``chat_abort`` WS) always returned ``no_active_run`` even while a run was
streaming.
"""
from __future__ import annotations
import asyncio
import pytest
from praisonaiui.provider import RunEvent, RunEventType
# ── Unit: abort_run against the _active_runs registry ────────────────
@pytest.mark.asyncio
async def test_abort_cancels_active_run():
from praisonaiui.features.chat import ChatManager
mgr = ChatManager()
run_id = "test-run"
async def slow_run():
await asyncio.sleep(60)
task = asyncio.create_task(slow_run())
mgr._active_runs[run_id] = task
result = await mgr.abort_run(run_id)
assert result["status"] == "aborted"
assert result["run_id"] == run_id
# Give the event loop a tick to process the cancellation.
await asyncio.sleep(0)
assert task.cancelled() or task.done()
# Registry entry removed on abort.
assert run_id not in mgr._active_runs
@pytest.mark.asyncio
async def test_abort_unknown_run_returns_no_active_run():
from praisonaiui.features.chat import ChatManager
mgr = ChatManager()
result = await mgr.abort_run("does-not-exist")
assert result["status"] == "no_active_run"
assert result["run_id"] == "does-not-exist"
# ── Integration: _run_and_broadcast registers + aborts cleanly ───────
class _FakeWS:
"""Collects broadcast frames pushed by ChatManager.broadcast."""
def __init__(self) -> None:
self.messages: list[dict] = []
async def send_text(self, data: str) -> None:
import json
self.messages.append(json.loads(data))
class _SlowProvider:
"""Streams a couple of tokens then hangs, keeping the run active."""
async def run(self, content, *, session_id=None, agent_name=None, **kwargs):
yield RunEvent(type=RunEventType.RUN_CONTENT, token="hello ")
yield RunEvent(type=RunEventType.RUN_CONTENT, token="world")
# Simulate a long generation so the run stays registered.
await asyncio.sleep(60)
yield RunEvent(type=RunEventType.RUN_COMPLETED, content="hello world")
class _FakeDatastore:
async def add_message(self, *args, **kwargs):
return None
@pytest.mark.asyncio
async def test_run_and_broadcast_registers_and_aborts(monkeypatch):
import praisonaiui.server as server
from praisonaiui.features.chat import ChatManager, _run_and_broadcast, set_chat_manager
mgr = ChatManager()
set_chat_manager(mgr)
ws = _FakeWS()
mgr.add_ws_client("c1", ws)
monkeypatch.setattr(server, "get_provider", lambda: _SlowProvider())
monkeypatch.setattr(server, "_datastore", _FakeDatastore())
task = asyncio.create_task(_run_and_broadcast("hi", "sess-1", "Agent", None))
# Wait until the run is registered (populated by _run_and_broadcast).
for _ in range(100):
if mgr._active_runs:
break
await asyncio.sleep(0.01)
assert mgr._active_runs, "run was never registered in _active_runs"
run_id = next(iter(mgr._active_runs))
# Frontend contract: a run_started frame is broadcast with the run_id.
started = [m for m in ws.messages if m.get("type") == "run_started"]
assert started and started[0]["run_id"] == run_id
# Abort the active run.
result = await mgr.abort_run(run_id)
assert result["status"] == "aborted"
# Let the cancellation propagate and _run_and_broadcast finish.
await asyncio.sleep(0.05)
# Clients receive a run_cancelled frame, and the registry is cleaned up.
cancelled = [m for m in ws.messages if m.get("type") == "run_cancelled"]
assert cancelled and cancelled[0]["run_id"] == run_id
assert run_id not in mgr._active_runs
assert task.done()
@pytest.mark.asyncio
async def test_run_and_broadcast_cleans_up_on_normal_completion(monkeypatch):
import praisonaiui.server as server
from praisonaiui.features.chat import ChatManager, _run_and_broadcast, set_chat_manager
class _QuickProvider:
async def run(self, content, *, session_id=None, agent_name=None, **kwargs):
yield RunEvent(type=RunEventType.RUN_CONTENT, token="done")
yield RunEvent(type=RunEventType.RUN_COMPLETED, content="done")
mgr = ChatManager()
set_chat_manager(mgr)
monkeypatch.setattr(server, "get_provider", lambda: _QuickProvider())
monkeypatch.setattr(server, "_datastore", _FakeDatastore())
await _run_and_broadcast("hi", "sess-2", None, None)
# No leftover registry entries after a normal run.
assert mgr._active_runs == {}
+45
View File
@@ -117,6 +117,51 @@ async def test_elements_persisted_in_metadata(adapter):
assert messages[0]["elements"] == elements
# ── Channel metadata round-trip ──────────────────────────────────────
@pytest.mark.asyncio
async def test_channel_metadata_round_trip(adapter):
"""platform/sender/icon/channel_id survive a write→read cycle."""
sid = str(uuid.uuid4())
await adapter.create_session(sid)
await adapter.add_message(sid, {
"role": "user",
"content": "[Bob] hi",
"platform": "discord",
"sender": "Bob",
"icon": "game",
"channel_id": "ch-99",
})
messages = await adapter.get_messages(sid)
assert len(messages) == 1
assert messages[0]["platform"] == "discord"
assert messages[0]["sender"] == "Bob"
assert messages[0]["icon"] == "game"
assert messages[0]["channel_id"] == "ch-99"
# Core fields still intact
assert messages[0]["role"] == "user"
assert messages[0]["content"] == "[Bob] hi"
@pytest.mark.asyncio
async def test_none_metadata_values_dropped(adapter):
"""None-valued fields are not persisted (kept out of the payload)."""
sid = str(uuid.uuid4())
await adapter.create_session(sid)
await adapter.add_message(sid, {
"role": "assistant",
"content": "reply",
"platform": "voice",
"sender": None,
})
messages = await adapter.get_messages(sid)
assert messages[0]["platform"] == "voice"
assert "sender" not in messages[0]
# ── Title auto-generation ────────────────────────────────────────────
+68
View File
@@ -0,0 +1,68 @@
"""Unit tests for `_load_env_files` — auto-loading PraisonAI `.env` files.
Regression coverage for issue #286: `env:VAR` channel token references
resolved to empty strings because `aiui run` never loaded the standard
`~/.praisonai/.env` file, so bots showed "Connected" but never replied.
"""
import os
from praisonaiui.cli import _load_env_files
def _write_env(path, body):
path.parent.mkdir(parents=True, exist_ok=True)
path.write_text(body, encoding="utf-8")
def test_loads_home_praisonai_env(tmp_path, monkeypatch):
"""Variables in ~/.praisonai/.env are loaded into the process env."""
monkeypatch.setattr("pathlib.Path.home", lambda: tmp_path)
monkeypatch.chdir(tmp_path)
monkeypatch.delenv("TELEGRAM_BOT_TOKEN", raising=False)
_write_env(tmp_path / ".praisonai" / ".env", 'TELEGRAM_BOT_TOKEN="123:AAtoken"\n')
loaded = _load_env_files()
assert loaded >= 1
assert os.environ["TELEGRAM_BOT_TOKEN"] == "123:AAtoken"
def test_does_not_override_existing_env(tmp_path, monkeypatch):
"""An already-set process variable is never clobbered by the file."""
monkeypatch.setattr("pathlib.Path.home", lambda: tmp_path)
monkeypatch.chdir(tmp_path)
monkeypatch.setenv("SLACK_BOT_TOKEN", "xoxb-from-shell")
_write_env(tmp_path / ".praisonai" / ".env", "SLACK_BOT_TOKEN=xoxb-from-file\n")
_load_env_files()
assert os.environ["SLACK_BOT_TOKEN"] == "xoxb-from-shell"
def test_ignores_comments_blanks_and_export_prefix(tmp_path, monkeypatch):
"""Comments/blank lines are skipped and `export ` prefixes are stripped."""
monkeypatch.setattr("pathlib.Path.home", lambda: tmp_path)
monkeypatch.chdir(tmp_path)
monkeypatch.delenv("FOO", raising=False)
monkeypatch.delenv("BAR", raising=False)
_write_env(
tmp_path / ".praisonai" / ".env",
"# a comment\n\nexport FOO=foo\nBAR='bar'\n",
)
_load_env_files()
assert os.environ["FOO"] == "foo"
assert os.environ["BAR"] == "bar"
def test_missing_files_are_harmless(tmp_path, monkeypatch):
"""No env files present → returns 0 and does not raise."""
monkeypatch.setattr("pathlib.Path.home", lambda: tmp_path)
monkeypatch.chdir(tmp_path)
assert _load_env_files() == 0
+19
View File
@@ -246,6 +246,25 @@ class TestLifecycleHooks:
assert elapsed < 0.5
assert _lifecycle_state["shutdown_initiated"] is True
@pytest.mark.asyncio
async def test_startup_timeout(self):
"""Test startup timeout mechanism — a hung hook must not block forever."""
@on_app_startup
async def slow_hook():
await asyncio.sleep(2) # Simulate hung startup
# Set short timeout
with patch.dict(os.environ, {"AIUI_STARTUP_TIMEOUT": "0.1"}):
start_time = time.time()
await run_startup_hooks()
elapsed = time.time() - start_time
# Should timeout quickly and still mark startup completed so the
# server reaches lifespan yield and /health/live responds
assert elapsed < 0.5
assert _lifecycle_state["startup_completed"] is True
@pytest.mark.asyncio
async def test_startup_idempotent(self):
"""Test that startup hooks are only run once."""
+77 -20
View File
@@ -110,18 +110,27 @@ class TestStdioMCPClient:
"""Test successful stdio connection."""
client = StdioMCPClient("echo", ["test"])
with patch("praisonaiui.features.mcp.stdio_client") as mock_stdio_client:
# Mock the context manager
mock_session_manager = AsyncMock()
with (
patch("praisonaiui.features.mcp.stdio_client") as mock_stdio_client,
patch("praisonaiui.features.mcp.ClientSession") as mock_client_session,
patch("praisonaiui.features.mcp.contextlib.AsyncExitStack") as mock_stack_cls,
):
mock_read = MagicMock()
mock_write = MagicMock()
mock_session = AsyncMock()
mock_session_manager.__aenter__.return_value = mock_session
mock_stdio_client.return_value = mock_session_manager
mock_stack = AsyncMock()
mock_stack.enter_async_context = AsyncMock(
side_effect=[(mock_read, mock_write), mock_session]
)
mock_stack_cls.return_value = mock_stack
mock_stdio_client.return_value = MagicMock()
mock_client_session.return_value = mock_session
result = await client.connect()
assert result is True
assert client.session == mock_session
mock_stdio_client.assert_called_once()
assert mock_stack.enter_async_context.await_count == 2
mock_session.initialize.assert_called_once()
@pytest.mark.asyncio
@@ -138,16 +147,13 @@ class TestStdioMCPClient:
"""Test stdio disconnect."""
client = StdioMCPClient("echo", ["test"])
# Mock session and session manager
mock_session = AsyncMock()
mock_session_manager = AsyncMock()
client.session = mock_session
client._session_manager = mock_session_manager
mock_stack = AsyncMock()
client._exit_stack = mock_stack
client._connected = True
await client.disconnect()
mock_session_manager.__aexit__.assert_called_once_with(None, None, None)
mock_stack.aclose.assert_called_once()
assert client.session is None
assert client.process is None
@@ -156,18 +162,14 @@ class TestStdioMCPClient:
"""Test stdio disconnect with exception handling."""
client = StdioMCPClient("echo", ["test"])
# Mock session that throws exception during disconnect
mock_session = AsyncMock()
mock_session_manager = AsyncMock()
mock_session_manager.__aexit__.side_effect = Exception("Disconnect failed")
client.session = mock_session
client._session_manager = mock_session_manager
mock_stack = AsyncMock()
mock_stack.aclose.side_effect = Exception("Disconnect failed")
client._exit_stack = mock_stack
client._connected = True
# Should not raise exception
await client.disconnect()
mock_session_manager.__aexit__.assert_called_once()
mock_stack.aclose.assert_called_once()
assert client.session is None
assert client.process is None
@@ -347,6 +349,61 @@ class TestMCPClientManager:
with pytest.raises(ValueError, match="Invalid MCP server config"):
await manager.connect_server(config)
@pytest.mark.asyncio
async def test_reconnect_same_name_disconnects_old(self):
"""Reconnecting the same name disconnects the previous client (no leak)."""
manager = MCPClientManager()
old_client = AsyncMock()
old_client.disconnect = AsyncMock()
manager._clients["srv"] = old_client
config = {"name": "srv", "command": "echo", "args": []}
with patch("praisonaiui.features.mcp.StdioMCPClient") as mock_client_class:
new_client = AsyncMock()
new_client.connect.return_value = True
new_client.list_tools.return_value = []
mock_client_class.return_value = new_client
await manager.connect_server(config)
old_client.disconnect.assert_awaited_once()
assert manager._clients["srv"] is new_client
@pytest.mark.asyncio
async def test_connect_failure_removes_client(self):
"""A failed connect (returns False) removes the client from the registry."""
manager = MCPClientManager()
config = {"name": "srv", "command": "echo", "args": []}
with patch("praisonaiui.features.mcp.StdioMCPClient") as mock_client_class:
client = AsyncMock()
client.connect.return_value = False
mock_client_class.return_value = client
server = await manager.connect_server(config)
assert server.status == MCPStatus.ERROR
client.disconnect.assert_awaited_once()
assert "srv" not in manager._clients
@pytest.mark.asyncio
async def test_connect_exception_removes_client(self):
"""A connect that raises removes the client from the registry."""
manager = MCPClientManager()
config = {"name": "srv", "command": "echo", "args": []}
with patch("praisonaiui.features.mcp.StdioMCPClient") as mock_client_class:
client = AsyncMock()
client.connect.side_effect = Exception("boom")
mock_client_class.return_value = client
server = await manager.connect_server(config)
assert server.status == MCPStatus.ERROR
client.disconnect.assert_awaited_once()
assert "srv" not in manager._clients
@pytest.mark.asyncio
async def test_disconnect_server_success(self):
"""Test successful server disconnect."""
+11 -11
View File
@@ -27,7 +27,7 @@ def part_b_env(monkeypatch, tmp_path):
monkeypatch.setenv("RECALL_REGION", "eu-central-1")
monkeypatch.setenv("RECALL_API_KEY", "test-recall-key")
monkeypatch.setenv("RECALL_WEBHOOK_VERIFICATION_SECRET", "whsec_test")
monkeypatch.setenv("RECALL_WORKSPACE_ID", "33a035c7-03c1-4ec9-b752-18cdfbbefc42")
monkeypatch.setenv("RECALL_WORKSPACE_ID", "test-workspace")
monkeypatch.setenv("PUBLIC_API_BASE_URL", "https://dev.example.test")
monkeypatch.setenv("PRAISONAI_MEETINGS_DIR", str(tmp_path))
monkeypatch.setenv("PRAISONAI_MEETINGS_DB", str(tmp_path / "meetings.db"))
@@ -35,7 +35,7 @@ def part_b_env(monkeypatch, tmp_path):
def test_transcript_data_to_line():
tx = _load("part_b_transcript", _ROOT / "integrations" / "recall" / "transcript.py")
tx = _load("part_b_transcript", _ROOT / "meeting_integrations" / "recall" / "transcript.py")
payload = {
"data": {
"data": {
@@ -48,7 +48,7 @@ def test_transcript_data_to_line():
def test_live_hub_publish_and_snapshot():
live = _load("part_b_live", _ROOT / "integrations" / "recall" / "live.py")
live = _load("part_b_live", _ROOT / "meeting_integrations" / "recall" / "live.py")
live.publish_live_chunk("m-live", line="Alice: Hi", full_text="Alice: Hi", live_status="live")
snap = live.snapshot_for_api("m-live", {"metadata": {}})
assert snap["transcript"] == "Alice: Hi"
@@ -56,8 +56,8 @@ def test_live_hub_publish_and_snapshot():
def test_create_bot_includes_realtime_config():
client_mod = _load("part_b_client", _ROOT / "integrations" / "recall" / "client.py")
config_mod = _load("part_b_config", _ROOT / "integrations" / "recall" / "config.py")
client_mod = _load("part_b_client", _ROOT / "meeting_integrations" / "recall" / "client.py")
config_mod = _load("part_b_config", _ROOT / "meeting_integrations" / "recall" / "config.py")
settings = config_mod.load_recall_settings()
client = client_mod.RecallClient(settings)
client._request = MagicMock(return_value={"id": "bot-1"}) # noqa: SLF001
@@ -76,7 +76,7 @@ def test_create_bot_includes_realtime_config():
def test_calendar_eligibility():
cal = _load("part_b_calendar", _ROOT / "integrations" / "recall" / "calendar_service.py")
cal = _load("part_b_calendar", _ROOT / "meeting_integrations" / "recall" / "calendar_service.py")
assert cal.is_eligible_for_auto_record(
{
"is_deleted": False,
@@ -88,7 +88,7 @@ def test_calendar_eligibility():
def test_transcript_webhook_keys_are_unique_per_utterance():
store = _load("part_b_store", _ROOT / "integrations" / "recall" / "store.py")
store = _load("part_b_store", _ROOT / "meeting_integrations" / "recall" / "store.py")
bot = "bot-1"
p1 = {
"data": {
@@ -109,8 +109,8 @@ def test_transcript_webhook_keys_are_unique_per_utterance():
def test_upsert_live_transcript_replaces_partial():
pytest.importorskip("praisonai_tools") # external PraisonAI-Tools dep, optional in CI
processor = _load("part_b_processor2", _ROOT / "integrations" / "recall" / "processor.py")
with patch("integrations.recall.live.publish_live_chunk") as publish:
processor = _load("part_b_processor2", _ROOT / "meeting_integrations" / "recall" / "processor.py")
with patch("meeting_integrations.recall.live.publish_live_chunk") as publish:
with patch("praisonai_tools.tools.meeting_tools.get_meeting") as get_m:
get_m.__wrapped__ = MagicMock(
return_value={"metadata": {"transcript": "Alice: hel"}}
@@ -121,8 +121,8 @@ def test_upsert_live_transcript_replaces_partial():
def test_process_transcript_data_webhook():
processor = _load("part_b_processor", _ROOT / "integrations" / "recall" / "processor.py")
config_mod = _load("part_b_config2", _ROOT / "integrations" / "recall" / "config.py")
processor = _load("part_b_processor", _ROOT / "meeting_integrations" / "recall" / "processor.py")
config_mod = _load("part_b_config2", _ROOT / "meeting_integrations" / "recall" / "config.py")
settings = config_mod.load_recall_settings()
with patch.object(processor, "_append_live_transcript") as append:
+1 -2
View File
@@ -9,8 +9,7 @@ from unittest.mock import patch
import pytest
# The ingest pipeline calls the external PraisonAI-Tools ``meeting_tools`` at
# runtime; it is not a PraisonAIUI CI dependency, so skip when unavailable.
# Meeting tools live in the external PraisonAI-Tools package; skip when unavailable in CI.
pytest.importorskip("praisonai_tools")
_PIPELINE = (
+6 -6
View File
@@ -32,7 +32,7 @@ def recall_env(monkeypatch, tmp_path):
monkeypatch.setenv("RECALL_REGION", "eu-central-1")
monkeypatch.setenv("RECALL_API_KEY", "test-recall-key")
monkeypatch.setenv("RECALL_WEBHOOK_VERIFICATION_SECRET", _SECRET)
monkeypatch.setenv("RECALL_WORKSPACE_ID", "33a035c7-03c1-4ec9-b752-18cdfbbefc42")
monkeypatch.setenv("RECALL_WORKSPACE_ID", "test-workspace")
monkeypatch.setenv("PUBLIC_API_BASE_URL", "https://dev.example.test")
monkeypatch.setenv("PRAISONAI_MEETINGS_DIR", str(tmp_path))
monkeypatch.setenv("PRAISONAI_MEETINGS_DB", str(tmp_path / "meetings.db"))
@@ -40,7 +40,7 @@ def recall_env(monkeypatch, tmp_path):
def test_verify_request_accepts_valid_signature():
verify_mod = _load("recall_verify", _ROOT / "integrations" / "recall" / "verify.py")
verify_mod = _load("recall_verify", _ROOT / "meeting_integrations" / "recall" / "verify.py")
import hmac
import hashlib
@@ -63,7 +63,7 @@ def test_verify_request_accepts_valid_signature():
def test_verify_request_rejects_bad_signature():
verify_mod = _load("recall_verify", _ROOT / "integrations" / "recall" / "verify.py")
verify_mod = _load("recall_verify", _ROOT / "meeting_integrations" / "recall" / "verify.py")
with pytest.raises(verify_mod.VerificationError):
verify_mod.verify_request_from_recall(
secret=_SECRET,
@@ -78,7 +78,7 @@ def test_verify_request_rejects_bad_signature():
def test_schedule_bot_persists_intent_before_api():
pytest.importorskip("praisonai_tools") # external PraisonAI-Tools dep, optional in CI
service = _load("recall_service", _ROOT / "integrations" / "recall" / "service.py")
service = _load("recall_service", _ROOT / "meeting_integrations" / "recall" / "service.py")
client = MagicMock()
client.create_bot.return_value = {"id": "bot-abc"}
@@ -104,7 +104,7 @@ def test_webhook_rejects_invalid_signature():
def test_webhook_accepts_valid_signature_and_queues():
example = _load("meeting_agent_recall_app2", _ROOT / "app.py")
verify_mod = _load("recall_verify2", _ROOT / "integrations" / "recall" / "verify.py")
verify_mod = _load("recall_verify2", _ROOT / "meeting_integrations" / "recall" / "verify.py")
import hmac
import hashlib
@@ -136,7 +136,7 @@ def test_webhook_accepts_valid_signature_and_queues():
def test_transcript_download_to_text():
tx = _load("recall_transcript", _ROOT / "integrations" / "recall" / "transcript.py")
tx = _load("recall_transcript", _ROOT / "meeting_integrations" / "recall" / "transcript.py")
data = [
{"speaker": "Alice", "words": [{"text": "Hello"}, {"text": "team"}]},
{"speaker": "Bob", "text": "Hi there"},
+325 -15
View File
@@ -20,17 +20,10 @@ import pytest
_ROOT = Path(__file__).resolve().parents[2] / "examples" / "python" / "voice-agent"
_INTEGRATIONS_PREFIX = ("integrations", "integrations.")
@contextlib.contextmanager
def _voice_modules():
"""Import the example's ``integrations.voice`` package in isolation.
Saves and clears any already-imported ``integrations*`` modules, puts the
example directory first on ``sys.path``, then restores the previous state
on exit so unrelated tests keep their own ``integrations`` package.
"""
"""Import the example's ``integrations.voice`` package in isolation."""
saved = {
name: mod
for name, mod in list(sys.modules.items())
@@ -69,6 +62,40 @@ def test_execute_tool_get_current_time():
assert "utc" in raw
def test_demo_tool_reply_time_and_echo():
with _voice_modules() as imp:
tools = imp("integrations.voice.tools")
time_reply = tools.demo_tool_reply("What time is it?")
echo_reply = tools.demo_tool_reply("Echo hello world")
assert time_reply is not None
assert "UTC time" in time_reply
assert echo_reply is not None
assert "hello world" in echo_reply.lower()
def test_speech_engine_demo_fast_path():
with _voice_modules() as imp:
se = imp("integrations.voice.speech_engine")
class Msg:
def __init__(self, role, content):
self.role = role
self.content = content
reply = se._try_demo_tool_reply([Msg("user", "Tell me the time please")])
assert reply is not None
assert "UTC time" in reply
def test_openai_tool_schemas():
with _voice_modules() as imp:
tools = imp("integrations.voice.tools")
schemas = tools.openai_tool_schemas()
names = [s["function"]["name"] for s in schemas]
assert "get_current_time" in names
assert "echo_message" in names
def test_tool_calls_webhook_response():
with _voice_modules() as imp:
processor = imp("integrations.voice.processor")
@@ -88,6 +115,22 @@ def test_tool_calls_webhook_response():
assert "hello" in result["results"][0]["result"]
def test_tool_calls_accepts_arguments_field():
with _voice_modules() as imp:
processor = imp("integrations.voice.processor")
config = imp("integrations.voice.config")
settings = config.load_voice_settings()
payload = {
"message": {
"type": "tool-calls",
"call": {"id": "call-2"},
"toolCallList": [{"id": "tc-2", "name": "get_current_time", "arguments": {}}],
}
}
result = processor.process_voice_webhook("tool-calls", payload, settings=settings)
assert "time" in result["results"][0]["result"].lower()
def test_assistant_request_returns_default_assistant(monkeypatch):
monkeypatch.setenv("VOICE_ASSISTANT_ID", "asst-123")
with _voice_modules() as imp:
@@ -123,7 +166,6 @@ def test_transcript_appends_final_lines():
def test_transcript_preserves_live_status():
"""A transcript/conversation event must not reset an in-progress call to 'unknown'."""
with _voice_modules() as imp:
processor = imp("integrations.voice.processor")
config = imp("integrations.voice.config")
@@ -153,12 +195,6 @@ def test_transcript_preserves_live_status():
def test_status_updates_are_not_collapsed_by_idempotency_key():
"""Distinct status transitions must each be applied, not dropped as duplicates.
A large static field (``call``) sorts before ``status``; a truncated JSON key
would collapse successive updates to the same prefix and freeze the dashboard
status on the first value.
"""
with _voice_modules() as imp:
processor = imp("integrations.voice.processor")
config = imp("integrations.voice.config")
@@ -206,3 +242,277 @@ def test_create_call_request_body(monkeypatch):
body = client._request.call_args.kwargs["json_body"] # noqa: SLF001
assert body["assistantId"] == "asst-1"
assert body["customer"]["number"] == "+15551234567"
def test_transcript_hub_publish_sync():
with _voice_modules() as imp:
live = imp("integrations.voice.live")
hub = live.TranscriptHub()
hub.publish("call-x", {"type": "transcript", "text": "hello"})
def test_dynamic_assistant_request(monkeypatch):
monkeypatch.setenv("VOICE_ASSISTANT_ID", "asst-123")
monkeypatch.setenv("VOICE_DYNAMIC_ASSISTANT", "true")
with _voice_modules() as imp:
processor = imp("integrations.voice.processor")
config = imp("integrations.voice.config")
settings = config.load_voice_settings()
result = processor.process_voice_webhook(
"assistant-request",
{"message": {"type": "assistant-request", "call": {"id": "c1"}}},
settings=settings,
)
assert "assistant" in result
assert result["assistant"]["name"] == "Praison Voice Assistant"
def test_chat_bridge_session_id():
with _voice_modules() as imp:
bridge = imp("integrations.voice.chat_bridge")
assert bridge.voice_session_id("abc-123") == "voice-abc-123"
def test_speech_engine_settings_ws_url(monkeypatch):
monkeypatch.setenv("ELEVENLABS_API_KEY", "el-test-key")
monkeypatch.setenv("SPEECH_ENGINE_PUBLIC_URL", "https://speech-tunnel.example.test")
with _voice_modules() as imp:
se = imp("integrations.voice.speech_engine")
settings = se.load_speech_engine_settings()
assert settings.public_ws_url == "wss://speech-tunnel.example.test/ws"
def test_speech_engine_transcript_prompt():
with _voice_modules() as imp:
se = imp("integrations.voice.speech_engine")
class Msg:
def __init__(self, role, content):
self.role = role
self.content = content
prompt = se._transcript_to_prompt([Msg("user", "What time is it?")])
assert "user: What time is it?" in prompt
def test_speech_engine_transcript_display_includes_agent_reply():
with _voice_modules() as imp:
se = imp("integrations.voice.speech_engine")
class Msg:
def __init__(self, role, content):
self.role = role
self.content = content
text = se._transcript_display_text([Msg("user", "Hello")], "Hi there!")
assert text == "user: Hello\nagent: Hi there!"
def test_speech_engine_persist_turn(tmp_path, monkeypatch):
monkeypatch.setenv("PRAISONAI_VOICE_DIR", str(tmp_path))
with _voice_modules() as imp:
se = imp("integrations.voice.speech_engine")
store = imp("integrations.voice.store")
class Msg:
def __init__(self, role, content):
self.role = role
self.content = content
se._persist_conversation_turn("conv-test-1", [Msg("user", "What time is it?")], "It is noon UTC.")
record = store.VoiceCallStore.get_call("conv-test-1")
assert record is not None
assert "user: What time is it?" in record["transcript"]
assert "agent: It is noon UTC." in record["transcript"]
def test_transcript_publishes_to_hub():
with _voice_modules() as imp:
processor = imp("integrations.voice.processor")
config = imp("integrations.voice.config")
live = imp("integrations.voice.live")
settings = config.load_voice_settings()
payload = {
"message": {
"type": "transcript",
"call": {"id": "call-hub"},
"role": "assistant",
"transcriptType": "partial",
"transcript": "One moment",
}
}
processor.process_voice_webhook("transcript", payload, settings=settings)
live.transcript_hub.publish("call-hub", {"type": "ping"})
def test_realtime_settings_and_session_config(monkeypatch):
monkeypatch.setenv("OPENAI_API_KEY", "sk-test")
monkeypatch.setenv("REALTIME_MODEL", "gpt-realtime-2.1-mini")
monkeypatch.setenv("REALTIME_VOICE", "marin")
monkeypatch.setenv("REALTIME_REASONING_EFFORT", "low")
with _voice_modules() as imp:
rt = imp("integrations.voice.realtime")
settings = rt.load_realtime_settings()
session = rt.build_session_config(settings)
assert settings.model == "gpt-realtime-2.1-mini"
assert session["type"] == "realtime"
assert session["model"] == "gpt-realtime-2.1-mini"
assert session["reasoning"]["effort"] == "low"
assert session["audio"]["output"]["voice"] == "marin"
assert "input_audio_transcription" in session
def test_call_detail_ui_layout():
with _voice_modules() as imp:
ui = imp("integrations.voice.call_detail_ui")
layout = ui.build_call_detail_layout(
{
"call_id": "conv-test",
"status": "in-progress",
"customer_number": "web-speech-engine",
"transcript": "user: hello\nagent: hi",
"summary": "",
"metadata": {"user_id": "u-1", "praison_session_id": "voice-user-u-1"},
}
)
children = layout.get("_components") or []
types = [c.get("type") for c in children]
assert "badge" in types
assert "key_value_list" in types
assert "tabs" in types
def test_voice_memory_bind_and_history(tmp_path, monkeypatch):
monkeypatch.setenv("PRAISONAI_VOICE_DIR", str(tmp_path))
monkeypatch.setenv("VOICE_MEMORY_ENABLED", "true")
with _voice_modules() as imp:
sm = imp("integrations.voice.session_memory")
store = imp("integrations.voice.store")
session_id = sm.bind_call_user("call-a", "user-123")
assert session_id == "voice-user-user-123"
sm.append_session_turn(session_id, "user", "codeword is blue")
sm.append_session_turn(session_id, "assistant", "Got it.")
block = sm.history_context_block(session_id)
assert "codeword is blue" in block
assert sm.resolve_call_user("call-a") == "user-123"
store.VoiceCallStore.set_pending_user("user-pending")
bound = sm.bind_call_from_pending("call-b")
assert bound == "voice-user-user-pending"
assert sm.resolve_call_user("call-b") == "user-pending"
assert store.VoiceCallStore.consume_pending_user() is None
def test_realtime_persist_transcript(tmp_path, monkeypatch):
monkeypatch.setenv("PRAISONAI_VOICE_DIR", str(tmp_path))
with _voice_modules() as imp:
rt = imp("integrations.voice.realtime")
store = imp("integrations.voice.store")
rt.mark_session_started("rt-call-1")
rt.persist_transcript_line("rt-call-1", "user", "My name is Alex")
rt.persist_transcript_line("rt-call-1", "agent", "Hi Alex!")
record = store.VoiceCallStore.get_call("rt-call-1")
assert record is not None
assert "user: My name is Alex" in record["transcript"]
assert "agent: Hi Alex!" in record["transcript"]
def test_call_finalize_analytics():
with _voice_modules() as imp:
cf = imp("integrations.voice.call_finalize")
transcript = "user: hello\nagent: hi there\nuser: thanks"
assert cf.count_turns(transcript) == 3
assert cf.format_duration(45) == "45s"
assert cf.format_duration(125) == "2m 5s"
record = {
"transcript": transcript,
"created_at": "2026-01-01T12:00:00+00:00",
"metadata": {"ended_at": "2026-01-01T12:01:30+00:00"},
}
analytics = cf.analytics_for_record(record)
assert analytics["turn_count"] == 3
assert analytics["duration_sec"] == 90
assert analytics["duration"] == "1m 30s"
@pytest.mark.asyncio
async def test_finalize_call_persists_summary_and_analytics(tmp_path, monkeypatch):
monkeypatch.setenv("PRAISONAI_VOICE_DIR", str(tmp_path))
async def fake_summary(_transcript: str) -> str:
return "Demo call about scheduling."
enqueued: list[tuple] = []
def fake_enqueue(call_id, summary, status):
enqueued.append((call_id, summary, status))
with _voice_modules() as imp:
cf = imp("integrations.voice.call_finalize")
store = imp("integrations.voice.store")
chat_bridge = imp("integrations.voice.chat_bridge")
monkeypatch.setattr(cf, "generate_call_summary", fake_summary)
monkeypatch.setattr(chat_bridge, "enqueue_call_summary", fake_enqueue)
store.VoiceCallStore.upsert_call(
"call-final",
status="in-progress",
transcript="user: book a meeting\nagent: sure",
)
await cf.finalize_call("call-final", status="ended (customer-ended-call)")
record = store.VoiceCallStore.get_call("call-final")
assert record is not None
assert record["summary"] == "Demo call about scheduling."
assert record["metadata"]["turn_count"] == 2
assert record["metadata"]["duration_sec"] is not None
assert enqueued == [("call-final", "Demo call about scheduling.", "ended (customer-ended-call)")]
def test_voice_doctor_report(monkeypatch):
monkeypatch.setenv("OPENAI_API_KEY", "sk-test-key-12345678")
with _voice_modules() as imp:
doctor = imp("integrations.voice.doctor")
report = doctor.run_voice_doctor(app_port=59999, sidecar_port=59998)
assert "checks" in report
assert "summary" in report
names = [c["name"] for c in report["checks"]]
assert "OPENAI_API_KEY" in names
assert report["summary"]["failed"] >= 1
def test_voice_doctor_mcp_probe_timeout_is_warn(monkeypatch):
monkeypatch.setenv("OPENAI_API_KEY", "sk-test-key-12345678")
def fake_http_status(url: str, timeout: float = 5.0):
if "/api/mcp/servers" in url:
assert timeout == 2.0
return "fail", "timed out"
return "pass", "HTTP 200"
with _voice_modules() as imp:
doctor = imp("integrations.voice.doctor")
monkeypatch.setattr(doctor, "_tcp_open", lambda *args, **kwargs: True)
monkeypatch.setattr(doctor, "_http_status", fake_http_status)
report = doctor.run_voice_doctor(app_port=8001, sidecar_port=59998)
mcp = next(c for c in report["checks"] if c["name"] == "MCP servers API")
assert mcp["status"] == "warn"
assert mcp["detail"] == "timed out"
def test_call_detail_ui_includes_analytics():
with _voice_modules() as imp:
ui = imp("integrations.voice.call_detail_ui")
layout = ui.build_call_detail_layout(
{
"call_id": "conv-analytics",
"status": "ended (customer-ended-call)",
"customer_number": "web-speech-engine",
"transcript": "user: one\nagent: two",
"summary": "Short demo.",
"metadata": {"turn_count": 2, "duration_sec": 30},
"created_at": "2026-01-01T12:00:00+00:00",
}
)
kv = next(c for c in (layout.get("_components") or []) if c.get("type") == "key_value_list")
labels = [item.get("label") for item in kv.get("items") or []]
assert "Duration" in labels
assert "Turns" in labels
Generated
+1 -1
View File
@@ -207,7 +207,7 @@ wheels = [
[[package]]
name = "aiui"
version = "0.3.143"
version = "0.3.146"
source = { editable = "." }
dependencies = [
{ name = "pydantic" },