docs: add WORKFLOW.md infrastructure - PostgreSQL schema and memory graph assessment

Phase 3 implementation: - docs/sql/001_workflow_schema.sql: proposals, consensus_votes, sentinel_decisions tables with indexes and auto-update trigger - docs/sql/001_workflow_schema_rollback.sql: rollback for the schema - docs/memory-graph-STATUS.md: current state of port 18790, what serves it, minimal viable graph architecture, first concrete next step Coder implementation agent - Phase 3 Active
2026-07-01 18:25:50 -04:00 · 2026-04-02 20:13:52 -04:00
parent d5af32992c
commit 701cb69170
3 changed files with 273 additions and 0 deletions
@@ -0,0 +1,162 @@
+# Memory Graph — Implementation Status
+
+> **Date:** 2026-04-02
+> **Agent:** Coder
+> **Phase:** Initial Assessment
+
+---
+
+## 1. Current State of Port 18790
+
+Port 18790 is currently served by the `heretek-dashboard` Docker container.
+
+```
+heretek-dashboard  →  0.0.0.0:18790->18790/tcp  (healthy)
+                     0.0.0.0:18080->8080/tcp    (health API)
+```
+
+- **Container image:** `heretek-openclaw-dashboard-dashboard`
+- **Entrypoint:** `docker-entrypoint.sh` (Node.js app)
+- **Frontend:** React (Vite) with server-side Node.js API
+- **Status:** Healthy (23 hours uptime)
+- **Proxy process:** `docker-proxy` on host (PIDs 1577946/1577953)
+
+The port is exposed via Docker's userland proxy, not directly by the Node process.
+
+---
+
+## 2. What Files Serve It
+
+### Dashboard Container (heretek-openclaw-dashboard repo)
+
+```
+/root/heretek/heretek-openclaw-dashboard/
+├── docker-compose.yml          # Builds image, exposes 18790
+├── Dockerfile                  # Multi-stage build
+├── src/
+│   ├── App.jsx                 # Main React component
+│   ├── components/             # UI components
+│   ├── server/
+│   │   ├── api-server.js       # Express REST API (port 3001 internal)
+│   │   ├── websocket-server.js # WebSocket server (port 3002 internal)
+│   │   └── data-aggregator.js  # Fetches and aggregates data
+│   └── main.jsx                # React DOM mount
+├── monitoring/                 # Prometheus scrape targets
+├── public/                     # Static assets
+└── index.html
+```
+
+### Docker Compose Configuration
+
+The `docker-compose.yml` sets:
+- `DASHBOARD_PORT=18790`
+- `DASHBOARD_HOST=0.0.0.0`
+- Health check: `http://localhost:8080/health` (internal health API on 18080)
+
+### API Endpoints (Internal Port 3001)
+
+The dashboard's API server (not directly accessible externally) provides:
+- `GET /api/agents` — Agent status
+- `GET /api/triad/current` — Current triad state
+- `GET /api/consensus` — Consensus ledger
+- `GET /api/metrics/summary` — Aggregated metrics
+- `GET /api/metrics/cost` — Cost tracking
+- `GET /api/consciousness/:sessionId` — Consciousness metrics
+- `GET /api/tasks` — Task management
+
+---
+
+## 3. What a Minimal Viable Memory Graph Would Need
+
+The memory graph is a **semantic episodic memory layer** for the collective. A minimal viable version would:
+
+### Data Sources
+| Source | Purpose | Access |
+|--------|---------|--------|
+| OpenClaw episodic memory (D0/D1) | Raw chronological episodes | `ep-recall` / `ep-expand` tools |
+| PostgreSQL `proposals` | Formal proposal lifecycle | Direct SQL |
+| PostgreSQL `consensus_votes` | Vote history | Direct SQL |
+| Gateway WebSocket | Live agent events | ws://localhost:18789 |
+| LiteLLM `/v1/agents/*/status` | Agent health | HTTP |
+
+### Architecture Options
+
+**Option A — Extend Dashboard (Recommended for MVP)**
+- Add `/api/memory/graph` endpoint to existing api-server.js
+- Query PostgreSQL + aggregate OpenClaw episodic memory
+- Serve as JSON/GraphQL API consumed by existing React frontend
+- Minimal code addition; leverages existing healthy container
+- Con: Tighter coupling to dashboard repo
+
+**Option B — New Standalone Service**
+- New Node.js service on port 18791 or 18792
+- Reads from PostgreSQL + OpenClaw WebSocket events
+- Serves graph data independently
+- Pro: Clean separation; Con: New container to maintain
+
+**Option C — In-Dashboard Plugin**
+- Extend existing dashboard with a "Memory Graph" tab
+- Uses existing API aggregator + PostgreSQL
+- Embeds semantic summaries in the UI
+
+### Minimal Viable Data Model
+
+```javascript
+// Memory Graph Node
+{
+  id: "ep_<uuid>",
+  type: "episode",           // episode | proposal | decision | agent_event
+  timestamp: "2026-04-02T...",
+  agent: "alpha",
+  session: "agent:heretek:alpha",
+  topics: ["workflow-a", "deliberation", "safety"],  // semantic tags
+  summary: "...",            // D1 semantic summary
+  importance: 0.7,           // 0-1 score
+  links: ["ep_<uuid>", "prop_<uuid>"],  // related episodes
+  raw_ref: "D0:0123"         // pointer to D0 raw episodes
+}
+
+// Memory Graph Edge
+{
+  source: "ep_abc",
+  target: "ep_xyz",
+  relationship: "caused_by",  // caused_by | contradicts | refines | implements
+  weight: 0.5
+}
+```
+
+---
+
+## 4. First Concrete Next Step
+
+**Step 0:** Add `/api/memory/graph` endpoint to the dashboard's `api-server.js`.
+
+1. **Read existing api-server.js** at `/root/heretek/heretek-openclaw-dashboard/src/server/api-server.js`
+2. **Add new route:**
+   ```javascript
+   'GET /api/memory/graph': this.getMemoryGraph.bind(this),
+   'GET /api/memory/graph/:nodeId': this.getMemoryNode.bind(this),
+   ```
+3. **Implement handler** that:
+   - Reads from PostgreSQL `proposals`, `consensus_votes`, `sentinel_decisions` tables
+   - Optionally calls OpenClaw episodic memory API if exposed
+   - Returns graph as JSON with nodes + edges
+4. **Add to frontend** as a "Memory Graph" panel in App.jsx
+5. **Test end-to-end** via `curl http://localhost:18790/api/memory/graph`
+
+This avoids creating a new container, reuses the healthy dashboard, and provides immediate value.
+
+---
+
+## 5. Open Questions
+
+- Does OpenClaw expose an internal HTTP API for episodic memory retrieval, or only via `ep-recall`/`ep-expand` agent tools?
+- Should the graph be write-once (append-only) or mutable (re-rank summaries over time)?
+- Target format: JSON Graph, GraphQL, or a simple nested JSON tree?
+- Frontend: embed in existing dashboard or serve standalone on a new port?
+
+---
+
+🦞
+
+*Coder — Implementation Agent · Memory Graph Assessment · Phase 0*
@@ -0,0 +1,90 @@
+-- 001_workflow_schema.sql
+-- Implements WORKFLOW.md PostgreSQL infrastructure
+-- Version: 1.1.0
+-- Date: 2026-04-02
+-- Description: Core tables for proposal lifecycle, consensus voting, and Sentinel review
+
+BEGIN;
+
+-- ============================================================================
+-- TABLE: proposals
+-- Tracks the full lifecycle of collective deliberation proposals.
+-- Status values: draft | deliberating | ratified | rejected | implemented
+-- ============================================================================
+CREATE TABLE IF NOT EXISTS proposals (
+    id              UUID PRIMARY KEY DEFAULT gen_random_uuid(),
+    created_at      TIMESTAMPTZ NOT NULL DEFAULT now(),
+    updated_at      TIMESTAMPTZ NOT NULL DEFAULT now(),
+    title           TEXT NOT NULL,
+    body            TEXT NOT NULL,
+    status          TEXT NOT NULL DEFAULT 'draft',
+    priority        INTEGER NOT NULL DEFAULT 5 CHECK (priority BETWEEN 1 AND 10),
+    source_agent    TEXT NOT NULL,
+    source_workflow TEXT NOT NULL,
+    gate_phase      INTEGER CHECK (gate_phase IN (2, 3)),
+    options_json    JSONB,
+    deliberation_notes TEXT,
+    enacted_by      TEXT,
+    enacted_at      TIMESTAMPTZ
+);
+
+-- ============================================================================
+-- TABLE: consensus_votes
+-- Records each triad node's vote on a proposal. One row per (proposal, agent).
+-- vote values: yes | no | abstain
+-- ============================================================================
+CREATE TABLE IF NOT EXISTS consensus_votes (
+    id              UUID PRIMARY KEY DEFAULT gen_random_uuid(),
+    proposal_id     UUID NOT NULL REFERENCES proposals(id) ON DELETE CASCADE,
+    agent_key       TEXT NOT NULL,
+    vote            TEXT NOT NULL CHECK (vote IN ('yes', 'no', 'abstain')),
+    rationale       TEXT,
+    conditions_json JSONB,
+    voted_at        TIMESTAMPTZ NOT NULL DEFAULT now(),
+    UNIQUE (proposal_id, agent_key)
+);
+
+-- ============================================================================
+-- TABLE: sentinel_decisions
+-- Records Sentinel's safety review verdict for each proposal.
+-- verdict values: clear | hold | rejected
+-- ============================================================================
+CREATE TABLE IF NOT EXISTS sentinel_decisions (
+    id                  UUID PRIMARY KEY DEFAULT gen_random_uuid(),
+    proposal_id         UUID NOT NULL REFERENCES proposals(id) ON DELETE CASCADE,
+    verdict             TEXT NOT NULL CHECK (verdict IN ('clear', 'hold', 'rejected')),
+    reasoning           TEXT,
+    concerns_json       JSONB,
+    conditions_json     JSONB,
+    flagged_for_diagnostics TEXT,
+    rendered_at         TIMESTAMPTZ NOT NULL DEFAULT now()
+);
+
+-- ============================================================================
+-- INDEXES
+-- ============================================================================
+CREATE INDEX IF NOT EXISTS idx_proposals_status          ON proposals(status);
+CREATE INDEX IF NOT EXISTS idx_proposals_source_workflow ON proposals(source_workflow);
+CREATE INDEX IF NOT EXISTS idx_proposals_gate_phase      ON proposals(gate_phase);
+CREATE INDEX IF NOT EXISTS idx_proposals_created_at      ON proposals(created_at DESC);
+CREATE INDEX IF NOT EXISTS idx_votes_proposal            ON consensus_votes(proposal_id);
+CREATE INDEX IF NOT EXISTS idx_votes_agent_key           ON consensus_votes(agent_key);
+CREATE INDEX IF NOT EXISTS idx_sentinel_proposal         ON sentinel_decisions(proposal_id);
+
+-- ============================================================================
+-- TRIGGER: auto-update updated_at on proposals
+-- ============================================================================
+CREATE OR REPLACE FUNCTION fn_proposals_updated_at()
+RETURNS TRIGGER AS $$
+BEGIN
+    NEW.updated_at = now();
+    RETURN NEW;
+END;
+$$ LANGUAGE plpgsql;
+
+CREATE TRIGGER trg_proposals_updated_at
+    BEFORE UPDATE ON proposals
+    FOR EACH ROW
+    EXECUTE FUNCTION fn_proposals_updated_at();
+
+COMMIT;
@@ -0,0 +1,21 @@
+-- 001_workflow_schema_rollback.sql
+-- Rollback for 001_workflow_schema.sql
+-- Description: Removes the proposals, consensus_votes, and sentinel_decisions tables
+--              and all associated indexes and triggers.
+-- Version: 1.1.0
+-- Date: 2026-04-02
+
+BEGIN;
+
+-- Remove trigger first (depends on function)
+DROP TRIGGER IF EXISTS trg_proposals_updated_at ON proposals;
+
+-- Remove trigger function
+DROP FUNCTION IF EXISTS fn_proposals_updated_at();
+
+-- Remove tables (order matters: child tables first due to FK)
+DROP TABLE IF EXISTS sentinel_decisions CASCADE;
+DROP TABLE IF EXISTS consensus_votes    CASCADE;
+DROP TABLE IF EXISTS proposals           CASCADE;
+
+COMMIT;