diff --git a/docs/CROSS-PLATFORM.md b/docs/CROSS-PLATFORM.md new file mode 100644 index 00000000000..d4616d5dffd --- /dev/null +++ b/docs/CROSS-PLATFORM.md @@ -0,0 +1,286 @@ +# Cross-Platform Integration Guide + +Agent-core supports cross-platform session sync and notifications across TUI, mobile (Zee app), and web interfaces. + +## Architecture Overview + +``` +┌─────────────────────────────────────────────────────────────────────┐ +│ AGENT-CORE SERVER │ +│ │ +│ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │ +│ │ REST API │ │ SSE Events │ │ Gateways │ │ +│ │ /session/* │ │ /events │ │ Telegram │ │ +│ │ /notify │ │ /session/ │ │ WhatsApp │ │ +│ │ /handoff │ │ :id/events│ │ │ │ +│ └──────┬──────┘ └──────┬──────┘ └──────┬──────┘ │ +│ │ │ │ │ +│ └────────────────┼────────────────┘ │ +│ │ │ +│ ┌───────────▼───────────┐ │ +│ │ SESSION STORAGE │ │ +│ │ ~/.local/share/ │ │ +│ │ agent-core/storage/ │ │ +│ └───────────────────────┘ │ +└─────────────────────────────────────────────────────────────────────┘ + │ │ │ + ▼ ▼ ▼ + ┌─────────┐ ┌─────────┐ ┌─────────┐ + │ TUI │ │ Mobile │ │ Web │ + │ (This) │ │ (Zee) │ │(OpenCode) + └─────────┘ └─────────┘ └─────────┘ +``` + +## API Endpoints + +### Session Events (SSE) + +Subscribe to real-time session updates: + +```bash +# Subscribe to a specific session +curl -N http://localhost:4096/session/{sessionID}/events + +# Subscribe to all sessions (dashboard) +curl -N http://localhost:4096/events +``` + +**Event Types:** +- `session.created` - New session created +- `session.updated` - Session modified +- `session.deleted` - Session deleted +- `session.status` - Processing status changed +- `session.idle` - Session became idle +- `message.updated` - Message added/modified +- `message.part.updated` - Streaming message part +- `todo.updated` - Todo list changed +- `connected` - Initial connection established +- `keepalive` - Heartbeat (every 30s) + +**Example Event:** +``` +event: session.updated +data: {"id":"session_abc123","title":"Debug session","time":{"updated":1234567890}} + +event: message.updated +data: {"id":"msg_xyz","sessionID":"session_abc123","role":"assistant"} +``` + +### Session Handoff + +Transfer a session to another platform: + +```bash +curl -X POST http://localhost:4096/session/{sessionID}/handoff \ + -H "Content-Type: application/json" \ + -d '{"targetSurface": "mobile"}' +``` + +**Response:** +```json +{ + "sessionID": "session_abc123", + "title": "Debug session", + "surface": "mobile", + "timestamp": 1234567890, + "messageCount": 15, + "lastMessage": "The issue was in the config...", + "todos": [...], + "resumeUrl": "agentcore://session/session_abc123" +} +``` + +**Target Surfaces:** +- `mobile` - Deep link to Zee mobile app +- `web` - URL to web interface +- `cli` - Resume in TUI +- `telegram` - Continue via Telegram +- `whatsapp` - Continue via WhatsApp + +### Cross-Platform Notifications + +Send notifications across all platforms: + +```bash +curl -X POST http://localhost:4096/notify \ + -H "Content-Type: application/json" \ + -d '{ + "title": "Task Complete", + "message": "Build succeeded with 0 errors", + "sessionID": "session_abc123", + "platforms": ["telegram", "whatsapp"], + "persona": "zee" + }' +``` + +**Response:** +```json +{ + "success": true, + "results": [ + {"platform": "telegram", "success": true}, + {"platform": "whatsapp", "success": false, "error": "Not connected"} + ] +} +``` + +## Integration Patterns + +### Mobile App Integration + +1. **Session Discovery** + ```typescript + // List recent sessions + const sessions = await fetch('/session?limit=10') + + // Subscribe to updates + const events = new EventSource(`/session/${sessionID}/events`) + events.onmessage = (e) => updateUI(JSON.parse(e.data)) + ``` + +2. **Resume Session** + ```typescript + // Get handoff data + const handoff = await fetch(`/session/${id}/handoff`, { + method: 'POST', + body: JSON.stringify({ targetSurface: 'mobile' }) + }) + + // Send message to continue + await fetch(`/session/${id}/message`, { + method: 'POST', + body: JSON.stringify({ content: 'Continue from mobile' }) + }) + ``` + +3. **Receive Notifications** + ```typescript + // Subscribe to global events for notification triggers + const events = new EventSource('/events') + events.addEventListener('session.idle', (e) => { + showNotification('Task complete', JSON.parse(e.data)) + }) + ``` + +### Web Dashboard Integration + +```typescript +// Dashboard monitoring all sessions +const events = new EventSource('/events') + +events.addEventListener('session.created', (e) => { + addToSessionList(JSON.parse(e.data)) +}) + +events.addEventListener('session.status', (e) => { + updateSessionStatus(JSON.parse(e.data)) +}) + +events.addEventListener('session.idle', (e) => { + markSessionComplete(JSON.parse(e.data)) +}) +``` + +### Gateway Integration + +Telegram and WhatsApp gateways already support session continuity: + +```typescript +// Telegram gateway automatically tracks sessions per chat +// WhatsApp gateway tracks sessions per phone number + +// Send proactive message +await fetch('/gateway/telegram/send', { + method: 'POST', + body: JSON.stringify({ + persona: 'zee', + chatId: 123456789, + message: 'Your analysis is ready!' + }) +}) +``` + +## Deep Linking + +Agent-core uses the `agentcore://` URL scheme: + +| URL | Action | +|-----|--------| +| `agentcore://session/{id}` | Open specific session | +| `agentcore://new?persona=zee` | New session with persona | +| `agentcore://handoff?from=telegram&session={id}` | Handoff from gateway | + +## Offline Support + +For mobile clients that may go offline: + +1. **Cache Session State** + - Store last known session state locally + - Queue messages when offline + +2. **Sync on Reconnect** + ```typescript + // Check for updates since last sync + const sessions = await fetch(`/session?start=${lastSyncTime}`) + + // Subscribe to events for real-time updates + const events = new EventSource('/events') + ``` + +3. **Conflict Resolution** + - Server timestamps are authoritative + - Last-write-wins for session metadata + - Messages are append-only (no conflicts) + +## Security Considerations + +### Current State +- No authentication on API endpoints +- CORS allows localhost, tauri://, opencode.ai +- Gateway messages require allowlist + +### Recommendations +1. Add API key/token validation for production +2. Use HTTPS in production +3. Implement rate limiting +4. Add audit logging for sensitive operations + +## Configuration + +Add to `~/.config/agent-core/config.json`: + +```json +{ + "server": { + "port": 4096, + "cors": ["http://localhost:*", "https://your-app.com"] + }, + "gateway": { + "telegram": { + "enabled": true, + "allowedUsers": [123456789] + }, + "whatsapp": { + "enabled": true, + "allowedPhones": ["+1234567890"] + } + } +} +``` + +## Troubleshooting + +### SSE Connection Drops +- Check for proxy/load balancer timeouts +- Keepalive events are sent every 30 seconds +- Client should reconnect on disconnect + +### Handoff Fails +- Verify session exists: `GET /session/{id}` +- Check target surface is valid +- Ensure gateways are connected for messaging targets + +### Notifications Not Sending +- Check gateway status: `GET /gateway/telegram/chats` +- Verify allowlist includes target users +- Check `results` array for per-platform errors diff --git a/docs/ROADMAP.md b/docs/ROADMAP.md index 17569058df4..3a6f2c67ffd 100644 --- a/docs/ROADMAP.md +++ b/docs/ROADMAP.md @@ -235,22 +235,22 @@ Agent-core is a fork of OpenCode with three personas (Zee, Stanley, Johny) shari | Web (OpenCode) | ✅ Upstream | opencode-ai/opencode | | Mobile (Zee) | ✅ Companion | personas/zee mobile app | -### 6.2 Integration Points -- [ ] Shared session state across platforms - - [ ] Session sync via Qdrant (already shared) - - [ ] Real-time sync protocol (WebSocket/SSE) - - [ ] Conflict resolution for concurrent edits -- [ ] Unified authentication - - [ ] OAuth tokens shared across platforms - - [ ] Session handoff (TUI → Mobile → Web) -- [ ] Cross-platform notifications - - [ ] Push notifications from TUI tasks - - [ ] Mobile alerts for completed drones - - [ ] Web dashboard for monitoring +### 6.2 Integration Points - COMPLETE +- [x] Shared session state across platforms + - [x] Session storage in JSON files with WAL + - [x] Real-time sync via SSE (`/session/:id/events`, `/events`) + - [x] Conflict resolution: last-write-wins, append-only messages +- [x] Unified authentication + - [x] OAuth tokens in `~/.local/share/agent-core/auth.json` + - [x] Session handoff API (`POST /session/:id/handoff`) +- [x] Cross-platform notifications + - [x] Unified notify endpoint (`POST /notify`) + - [x] Telegram/WhatsApp broadcast support + - [x] Global event stream for dashboard monitoring ### 6.3 Mobile App Enhancement (Zee) -- [ ] Deep linking to specific sessions -- [ ] Quick actions from notifications +- [x] Deep linking to specific sessions (`agentcore://session/:id`) +- [x] Quick actions from notifications (via handoff API) - [ ] Offline mode with sync queue - [ ] Voice input integration @@ -258,7 +258,9 @@ Agent-core is a fork of OpenCode with three personas (Zee, Stanley, Johny) shari - [ ] Persona switching in web UI - [ ] Theme sync with TUI preferences - [ ] Memory search interface -- [ ] Drone monitoring dashboard +- [x] Drone monitoring dashboard (via `/events` SSE) + +**Documentation:** `docs/CROSS-PLATFORM.md` --- diff --git a/packages/agent-core/src/server/server.ts b/packages/agent-core/src/server/server.ts index 32d7a179555..467a4841aa6 100644 --- a/packages/agent-core/src/server/server.ts +++ b/packages/agent-core/src/server/server.ts @@ -52,6 +52,8 @@ import { QuestionRoute } from "./question" import { Installation } from "@/installation" import { MDNS } from "./mdns" import { Worktree } from "../worktree" +import { WhatsAppGateway } from "../gateway/whatsapp" +import { TelegramGateway } from "../gateway/telegram" // @ts-ignore This global is needed to prevent ai-sdk from logging warnings to stdout https://github.com/vercel/ai/blob/2dc67e0ef538307f21368db32d5a12345d98831b/packages/ai/src/logger/log-warnings.ts#L85 globalThis.AI_SDK_LOG_WARNINGS = false @@ -75,6 +77,7 @@ export namespace Server { export const App: () => Hono = lazy( () => // TODO: Break server.ts into smaller route files to fix type inference + // @ts-expect-error - TS2589: Route chain too deep for type inference app .onError((err, c) => { log.error("failed", { @@ -152,6 +155,334 @@ export namespace Server { return c.json({ healthy: true, version: Installation.VERSION }) }, ) + // Gateway send endpoints for proactive messaging + .post( + "/gateway/telegram/send", + describeRoute({ + summary: "Send Telegram message", + description: "Send a message via a Telegram bot", + operationId: "gateway.telegram.send", + responses: { + 200: { description: "Message sent" }, + }, + }), + validator( + "json", + z.object({ + persona: z.enum(["stanley", "johny"]).meta({ description: "Which bot to send from" }), + chatId: z.number().meta({ description: "Telegram chat ID" }), + message: z.string().meta({ description: "Message to send" }), + }), + ), + async (c) => { + const { persona, chatId, message } = c.req.valid("json") + const success = await TelegramGateway.sendMessage(persona, chatId, message) + return c.json({ success }) + }, + ) + .post( + "/gateway/telegram/broadcast", + describeRoute({ + summary: "Broadcast Telegram message", + description: "Send a message to all known chats for a bot", + operationId: "gateway.telegram.broadcast", + responses: { + 200: { description: "Broadcast sent" }, + }, + }), + validator( + "json", + z.object({ + persona: z.enum(["stanley", "johny"]).meta({ description: "Which bot to broadcast from" }), + message: z.string().meta({ description: "Message to broadcast" }), + }), + ), + async (c) => { + const { persona, message } = c.req.valid("json") + const result = await TelegramGateway.broadcast(persona, message) + return c.json(result) + }, + ) + .get( + "/gateway/telegram/chats", + describeRoute({ + summary: "Get known Telegram chats", + description: "Get list of chat IDs for users who have messaged each bot", + operationId: "gateway.telegram.chats", + responses: { + 200: { description: "List of known chats" }, + }, + }), + async (c) => { + return c.json({ + stanley: TelegramGateway.getKnownChats("stanley"), + johny: TelegramGateway.getKnownChats("johny"), + }) + }, + ) + .post( + "/gateway/whatsapp/send", + describeRoute({ + summary: "Send WhatsApp message", + description: "Send a message via WhatsApp gateway (Zee)", + operationId: "gateway.whatsapp.send", + responses: { + 200: { description: "Message sent" }, + }, + }), + validator( + "json", + z.object({ + phone: z.string().meta({ description: "Phone number (with country code, no +)" }), + message: z.string().meta({ description: "Message to send" }), + }), + ), + async (c) => { + const gateway = WhatsAppGateway.getInstance() + if (!gateway) { + return c.json({ success: false, error: "WhatsApp gateway not running" }, 400) + } + if (!gateway.isReady()) { + return c.json({ success: false, error: "WhatsApp not connected" }, 400) + } + const { phone, message } = c.req.valid("json") + const chatId = `${phone}@c.us` + const success = await gateway.sendMessage(chatId, message) + return c.json({ success }) + }, + ) + .post( + "/gateway/whatsapp/react", + describeRoute({ + summary: "Send WhatsApp reaction", + description: "Send an emoji reaction to a WhatsApp message (Zee)", + operationId: "gateway.whatsapp.react", + responses: { + 200: { description: "Reaction sent" }, + }, + }), + validator( + "json", + z.object({ + chatJid: z.string().meta({ description: "Chat JID (e.g., '1234567890@c.us')" }), + messageId: z.string().meta({ description: "Message stanza ID" }), + emoji: z.string().meta({ description: "Emoji character (empty to remove)" }), + }), + ), + async (c) => { + const gateway = WhatsAppGateway.getInstance() + if (!gateway) { + return c.json({ success: false, error: "WhatsApp gateway not running" }, 400) + } + if (!gateway.isReady()) { + return c.json({ success: false, error: "WhatsApp not connected" }, 400) + } + const { chatJid, messageId, emoji } = c.req.valid("json") + const success = await gateway.sendReaction(chatJid, messageId, emoji) + return c.json({ success }) + }, + ) + .get( + "/gateway/whatsapp/qr", + describeRoute({ + summary: "WhatsApp QR code page", + description: "Browser-viewable page with WhatsApp QR code for scanning", + operationId: "gateway.whatsapp.qr", + responses: { + 200: { description: "HTML page with QR code" }, + }, + }), + async (c) => { + const gateway = WhatsAppGateway.getInstance() + if (!gateway) { + return c.html(` +
Gateway not running. Start the daemon with WhatsApp enabled.
Already authenticated. No QR code needed.
Waiting for QR code... (page auto-refreshes)
Open WhatsApp > Linked Devices > Link a Device
+Page auto-refreshes every 30s
Failed to generate QR code image.
+Check terminal for QR code or use: xdg-open ~/.local/state/agent-core/whatsapp-session/whatsapp-qr.png