mirror of
https://github.com/Heretek-AI/openclaw.git
synced 2026-07-01 01:37:55 -04:00
docs: document all deployment bugs found during TM-3/TM-4 restoration
This commit is contained in:
Tabula Myriad
@@ -0,0 +1,262 @@
|
||||
# DEPLOYMENT BUGS — Tabula Myriad Triad
|
||||
|
||||
**Last updated:** 2026-03-24 04:05 EDT
|
||||
**Status:** TM-3 fixed, TM-4 blocked (SSH hung), TM-2 working since before
|
||||
|
||||
---
|
||||
|
||||
## Bug 1: Shebang Entry Point Silent Failure
|
||||
|
||||
**Severity:** High
|
||||
**Affected:** All nodes (global `/usr/bin/openclaw` install)
|
||||
|
||||
### Symptom
|
||||
|
||||
```bash
|
||||
/usr/bin/openclaw --version # Exit 0, no output
|
||||
node /usr/bin/openclaw --version # Works correctly
|
||||
```
|
||||
|
||||
### Root Cause
|
||||
|
||||
The `openclaw.mjs` bootstrap resolves `./dist/entry.(m)js` relative to `process.cwd()` instead of the file's directory. When invoked as a global CLI via shebang, `cwd` is `/root` (or wherever the shell is), not the package directory.
|
||||
|
||||
### Workaround
|
||||
|
||||
```bash
|
||||
# Don't use the shebang:
|
||||
node /usr/lib/node_modules/@heretek-ai/openclaw/dist/entry.mjs gateway
|
||||
|
||||
# Or from workspace:
|
||||
cd /home/openclaw/.openclaw/workspace && node dist/entry.mjs gateway
|
||||
```
|
||||
|
||||
### Fix Required
|
||||
|
||||
Update `openclaw.mjs` bootstrap to resolve entry relative to `import.meta.url` (the file's own location), not `process.cwd()`.
|
||||
|
||||
---
|
||||
|
||||
## Bug 2: Workspace `dist/package.json` Missing Export Maps
|
||||
|
||||
**Severity:** Critical
|
||||
**Affected:** TM-3 (fixed), TM-4 (blocked)
|
||||
|
||||
### Symptom
|
||||
|
||||
```
|
||||
Error [ERR_MODULE_NOT_FOUND]: Cannot find module 'openclaw'
|
||||
imported from /home/openclaw/.openclaw/workspace/dist/chunks/accounts-DTykWd83.mjs
|
||||
```
|
||||
|
||||
### Root Cause
|
||||
|
||||
The workspace `dist/chunks/` import bare specifiers like `"openclaw/plugin-sdk/account-resolution"`. Node.js resolves these through `node_modules/openclaw/` → needs a `package.json` with an `"exports"` map. The workspace `dist/` was built without generating a `package.json` at the root of `dist/`.
|
||||
|
||||
### Fix (applied to TM-3)
|
||||
|
||||
```bash
|
||||
# 1. Create dist/package.json with proper exports:
|
||||
cat > /home/openclaw/.openclaw/workspace/dist/package.json << 'EOF'
|
||||
{
|
||||
"name": "openclaw",
|
||||
"version": "0.0.0",
|
||||
"type": "module",
|
||||
"main": "./entry.mjs",
|
||||
"exports": {
|
||||
".": "./entry.mjs",
|
||||
"./plugin-sdk/*": "./plugin-sdk/*.mjs"
|
||||
}
|
||||
}
|
||||
EOF
|
||||
|
||||
# 2. Create symlink so "openclaw/*" resolves to workspace dist:
|
||||
ln -sf /home/openclaw/.openclaw/workspace/dist /home/openclaw/.openclaw/workspace/node_modules/openclaw
|
||||
|
||||
# 3. Start gateway:
|
||||
cd /home/openclaw/.openclaw/workspace && node dist/entry.mjs gateway
|
||||
```
|
||||
|
||||
### Build System Fix Required
|
||||
|
||||
The tsdown/Vite build should generate a `dist/package.json` as part of the build output, with proper exports matching the source package.json.
|
||||
|
||||
---
|
||||
|
||||
## Bug 3: Stale Global Install with Incomplete Dependency Chain
|
||||
|
||||
**Severity:** Critical
|
||||
**Affected:** TM-2 (restart fails), TM-4 (all starts fail)
|
||||
|
||||
### Symptom
|
||||
|
||||
```
|
||||
Error [ERR_MODULE_NOT_FOUND]: Cannot find package 'grammy'
|
||||
imported from /usr/lib/node_modules/@heretek-ai/openclaw/dist/chunks/...
|
||||
```
|
||||
|
||||
### Root Cause
|
||||
|
||||
The global `@heretek-ai/openclaw` at `/usr/lib/node_modules/@heretek-ai/openclaw` has chunks that import external packages (`grammy`, `@buape/carbon`, `@grammyjs/transformer-throttler`, etc.) that are NOT in the package's own `node_modules/`. The global install was never complete — it's missing all peer/transitive dependencies the chunks require.
|
||||
|
||||
Additionally, `/usr/lib/node_modules/openclaw` (a symlink to `/tmp/oc-full`) was stale — `/tmp/oc-full` was either cleared or incomplete.
|
||||
|
||||
### Why it "worked" before
|
||||
|
||||
The old gateway on TM-2 (PID 1623) was started before `/tmp/oc-full` became stale. It kept running as a long-lived process even after the on-disk installation became broken.
|
||||
|
||||
### Fix
|
||||
|
||||
**Do NOT rely on the global install for running the gateway.** Use the workspace build method instead (see Correct Deployment Method below).
|
||||
|
||||
### TM-2 Restart Blocked
|
||||
|
||||
When `systemctl restart openclaw-gateway` is called on TM-2, the service fails because it uses the broken `/usr/bin/openclaw` shebang entry point. The gateway needs to be started manually via:
|
||||
|
||||
```bash
|
||||
cd /home/openclaw/.openclaw/workspace && node dist/entry.mjs gateway
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Bug 4: Workspace Build Missing Transitive Dependencies
|
||||
|
||||
**Severity:** High
|
||||
**Affected:** TM-4 (blocked), any fresh clone
|
||||
|
||||
### Symptom
|
||||
|
||||
Gateway fails with cascading `ERR_MODULE_NOT_FOUND` errors for packages like `grammy`, `@grammyjs/transformer-throttler`, `@buape/carbon`, `discord-api-types`, etc.
|
||||
|
||||
### Root Cause
|
||||
|
||||
The workspace `package.json` does NOT declare all packages that the built `dist/chunks/` import. Many packages (`discord-api-types`, `@buape/carbon`, `@grammyjs/transformer-throttler`, etc.) are transitive dependencies of packages that ARE listed, but npm's install doesn't reliably hoist everything needed.
|
||||
|
||||
When using `pnpm`, the installation hangs (possibly due to network or lockfile issues). `npm ci` installs but leaves gaps. Each missing package causes a cascading failure.
|
||||
|
||||
### TM-4 Current State
|
||||
|
||||
- Git synced to `7f9cda56ce` ✅
|
||||
- `pnpm install` attempted — SSH session hung/froze the machine
|
||||
- Workspace is in an inconsistent state
|
||||
|
||||
### Fix Required
|
||||
|
||||
1. Fix the build system so that all imported packages are declared as dependencies in `package.json`
|
||||
2. OR configure the bundler (tsdown/rolldown) to bundle all imports inline rather than leaving them as external
|
||||
3. OR ensure `pnpm install` completes successfully before gateway start
|
||||
|
||||
---
|
||||
|
||||
## Bug 5: Hook Source File Path Errors (Non-Fatal)
|
||||
|
||||
**Severity:** Low (non-blocking)
|
||||
|
||||
### Symptom
|
||||
|
||||
```
|
||||
[hooks:loader] Failed to load hook bootstrap-extra-files:
|
||||
Cannot find module '/home/openclaw/.openclaw/workspace/src/agents/workspace.js'
|
||||
imported from ...src/hooks/bundled/bootstrap-extra-files/handler.ts
|
||||
[hooks:loader] Failed to load hook command-logger:
|
||||
Cannot find module '/home/openclaw/.openclaw/workspace/src/config/paths.js'
|
||||
```
|
||||
|
||||
### Root Cause
|
||||
|
||||
The built hook bundles contain TypeScript source references (`.ts` extensions) instead of compiled `.js` references. The hook loader tries to load `.js` files from `src/` directories where only `.ts` files exist.
|
||||
|
||||
### Impact
|
||||
|
||||
These hooks fail to load but the gateway continues to function (health checks pass). These hooks appear to be optional/non-critical.
|
||||
|
||||
### Fix
|
||||
|
||||
Verify that the build process correctly rewrites source file extensions from `.ts` to `.js` in hook bundles.
|
||||
|
||||
---
|
||||
|
||||
## Correct Deployment Method
|
||||
|
||||
### For NEW node setup (like TM-4):
|
||||
|
||||
```bash
|
||||
# 1. Clone and install deps
|
||||
git clone https://github.com/Heretek-AI/openclaw.git /home/openclaw/.openclaw/workspace
|
||||
cd /home/openclaw/.openclaw/workspace
|
||||
pnpm install # Must complete before proceeding
|
||||
|
||||
# 2. Build
|
||||
pnpm build
|
||||
|
||||
# 3. Fix dist/package.json (build system bug — should be automatic)
|
||||
cat > dist/package.json << 'EOF'
|
||||
{
|
||||
"name": "openclaw",
|
||||
"version": "0.0.0",
|
||||
"type": "module",
|
||||
"main": "./entry.mjs",
|
||||
"exports": {
|
||||
".": "./entry.mjs",
|
||||
"./plugin-sdk/*": "./plugin-sdk/*.mjs"
|
||||
}
|
||||
}
|
||||
EOF
|
||||
|
||||
# 4. Create openclaw symlink for bare specifier resolution
|
||||
ln -sf /home/openclaw/.openclaw/workspace/dist /home/openclaw/.openclaw/workspace/node_modules/openclaw
|
||||
|
||||
# 5. Start gateway
|
||||
cd /home/openclaw/.openclaw/workspace && node dist/entry.mjs gateway
|
||||
```
|
||||
|
||||
### For EXISTING node fixup (like TM-3):
|
||||
|
||||
```bash
|
||||
# Already did on TM-3:
|
||||
cd /home/openclaw/.openclaw/workspace
|
||||
pnpm build # rebuild if needed
|
||||
# create dist/package.json + symlink (see above)
|
||||
node dist/entry.mjs gateway
|
||||
```
|
||||
|
||||
### Post-Build Verification
|
||||
|
||||
```bash
|
||||
# Verify no missing deps:
|
||||
cd /home/openclaw/.openclaw/workspace
|
||||
node -e "import('grammy').then(()=>console.log('grammy OK')).catch(e=>console.error('grammy MISSING'))"
|
||||
node -e "import('@buape/carbon').then(()=>console.log('@buape/carbon OK')).catch(e=>console.error('@buape/carbon MISSING'))"
|
||||
|
||||
# Health check:
|
||||
curl http://127.0.0.1:18789/healthz
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## TM-4 Recovery Steps (when SSH responds)
|
||||
|
||||
1. Kill any hung npm/pnpm processes: `pkill -f npm; pkill -f pnpm`
|
||||
2. Verify git: `cd /home/openclaw/.openclaw/workspace && git fetch origin && git reset --hard origin/main`
|
||||
3. Run pnpm install: `cd /home/openclaw/.openclaw/workspace && pnpm install`
|
||||
4. Build: `cd /home/openclaw/.openclaw/workspace && pnpm build`
|
||||
5. Apply dist/package.json fix (see above)
|
||||
6. Start gateway: `cd /home/openclaw/.openclaw/workspace && node dist/entry.mjs gateway`
|
||||
|
||||
---
|
||||
|
||||
## Root Cause Summary
|
||||
|
||||
The deployment is broken because:
|
||||
|
||||
1. **Build output bug:** `pnpm build` (via tsdown/rolldown) outputs `dist/` without a `package.json`, so Node.js can't resolve bare specifier imports like `"openclaw/plugin-sdk/..."`
|
||||
|
||||
2. **Bundler configuration bug:** The tsdown build treats many packages as "external" (not bundled), expecting them to be resolvable at runtime. But these packages aren't all declared in `package.json` dependencies, so they're missing at runtime.
|
||||
|
||||
3. **CLI entry point bug:** The `openclaw.mjs` shebang bootstrap resolves paths relative to `cwd` instead of the file's directory, causing silent failure when invoked globally.
|
||||
|
||||
The three fixes needed in the build system:
|
||||
|
||||
- Generate `dist/package.json` with proper exports as part of the build
|
||||
- Either declare ALL imported packages in `package.json`, OR configure bundler to inline all external imports
|
||||
- Fix `openclaw.mjs` bootstrap to use `import.meta.url` for path resolution
|
||||
+43
-24
@@ -12,6 +12,7 @@
|
||||
**Objective:** Convert monolithic shell scripts to reusable, composable skills.
|
||||
|
||||
**Deliverables:**
|
||||
|
||||
- `skills/curiosity-engine/modules/gap-detector.js` - Node.js gap detection module
|
||||
- `skills/curiosity-engine/modules/anomaly-detector.js` - Anomaly pattern recognition
|
||||
- `skills/curiosity-engine/modules/opportunity-scanner.js` - Multi-source opportunity detection
|
||||
@@ -19,6 +20,7 @@
|
||||
- `skills/curiosity-engine/modules/deliberation-trigger.js` - Auto-proposal generation
|
||||
|
||||
**Migration Strategy:**
|
||||
|
||||
```bash
|
||||
# Preserve existing shell scripts (backward compatibility)
|
||||
engines/ # Legacy shell scripts (deprecated but functional)
|
||||
@@ -32,12 +34,14 @@ modules/ # New Node.js modules (primary implementation)
|
||||
**Objective:** Advanced pattern detection with heuristic scoring.
|
||||
|
||||
**Enhancements:**
|
||||
|
||||
- **Temporal clustering:** Group errors by time windows (5min, 1h, 24h)
|
||||
- **Severity scoring:** ML-like heuristics for anomaly classification
|
||||
- **Auto-correlation:** Detect cascading failures (A→B→C error chains)
|
||||
- **Baseline deviation:** Compare current error rates to 7-day rolling average
|
||||
|
||||
**Implementation:**
|
||||
|
||||
```javascript
|
||||
// Anomaly scoring algorithm
|
||||
function scoreAnomaly(errors) {
|
||||
@@ -45,11 +49,11 @@ function scoreAnomaly(errors) {
|
||||
const severityWeight = { critical: 3, high: 2, medium: 1, low: 0.5 };
|
||||
const baseline = get7DayRollingAverage();
|
||||
const deviation = (frequency - baseline) / baseline;
|
||||
|
||||
|
||||
return {
|
||||
score: deviation * severityWeight[errors[0].severity],
|
||||
isSignificant: deviation > 2.0, // 2σ deviation
|
||||
recommendation: generateRecommendation(errors)
|
||||
recommendation: generateRecommendation(errors),
|
||||
};
|
||||
}
|
||||
```
|
||||
@@ -68,35 +72,37 @@ function scoreAnomaly(errors) {
|
||||
| **GitHub API** | Repository monitoring | Opportunity scanning (releases, issues, PRs) |
|
||||
|
||||
**Integration Architecture:**
|
||||
|
||||
```javascript
|
||||
// MCP client wrapper
|
||||
const mcpClient = {
|
||||
searxng: {
|
||||
search: (query) => fetch(`http://searxng.local/search?q=${query}&format=json`),
|
||||
trends: (topic) => aggregateSearchResults(topic)
|
||||
trends: (topic) => aggregateSearchResults(topic),
|
||||
},
|
||||
playwright: {
|
||||
scrape: (url, selector) => browser.$(selector).textContent(),
|
||||
navigate: (url) => browser.goto(url)
|
||||
navigate: (url) => browser.goto(url),
|
||||
},
|
||||
github: {
|
||||
releases: (repo) => ghApi.get(`/repos/${repo}/releases`),
|
||||
issues: (repo, state) => ghApi.get(`/repos/${repo}/issues?state=${state}`),
|
||||
scanAlerts: (repo) => ghApi.get(`/repos/${repo}/code-scanning/alerts`)
|
||||
}
|
||||
scanAlerts: (repo) => ghApi.get(`/repos/${repo}/code-scanning/alerts`),
|
||||
},
|
||||
};
|
||||
```
|
||||
|
||||
**Usage in Engines:**
|
||||
|
||||
```javascript
|
||||
// Opportunity Scanner with MCP
|
||||
async function scanOpportunities() {
|
||||
const [githubReleases, npmUpdates, securityAdvisories] = await Promise.all([
|
||||
mcpClient.github.releases('Heretek-AI/openclaw'),
|
||||
mcpClient.searxng.search('@heretek-ai npm latest'),
|
||||
mcpClient.github.scanAlerts('Heretek-AI/openclaw')
|
||||
mcpClient.github.releases("Heretek-AI/openclaw"),
|
||||
mcpClient.searxng.search("@heretek-ai npm latest"),
|
||||
mcpClient.github.scanAlerts("Heretek-AI/openclaw"),
|
||||
]);
|
||||
|
||||
|
||||
return aggregateOpportunities(githubReleases, npmUpdates, securityAdvisories);
|
||||
}
|
||||
```
|
||||
@@ -108,12 +114,14 @@ async function scanOpportunities() {
|
||||
**Objective:** Intelligent auto-trigger with priority scoring and deduplication.
|
||||
|
||||
**Enhancements:**
|
||||
|
||||
- **Priority scoring matrix:** Weight gaps by impact × urgency
|
||||
- **Deduplication:** Prevent duplicate proposals within 24h window
|
||||
- **Quorum awareness:** Only post to Discord if TM-1 (authority node)
|
||||
- **Proposal templating:** Structured proposal format for consensus voting
|
||||
|
||||
**Priority Matrix:**
|
||||
|
||||
```javascript
|
||||
const priorityMatrix = {
|
||||
security: { base: 10, multiplier: 2.0 }, // Critical security gaps
|
||||
@@ -138,6 +146,7 @@ function calculatePriority(gap) {
|
||||
**Objective:** Real-time curiosity metrics with visualization.
|
||||
|
||||
**Metrics Tracked:**
|
||||
|
||||
- **Autonomy Score:** `(installed_skills / available_skills) × 100`
|
||||
- **Gap Velocity:** New gaps detected per week
|
||||
- **Proposal Conversion Rate:** `(approved_proposals / total_proposals) × 100`
|
||||
@@ -145,19 +154,21 @@ function calculatePriority(gap) {
|
||||
- **Opportunity Capture Rate:** `(acted_opportunities / detected_opportunities) × 100`
|
||||
|
||||
**Dashboard Output:**
|
||||
|
||||
```json
|
||||
{
|
||||
"timestamp": "2026-03-24T15:30:00-04:00",
|
||||
"autonomy_score": 67.3,
|
||||
"gap_velocity": 2.5, // gaps/week
|
||||
"proposal_conversion": 85.0, // %
|
||||
"anomaly_resolution_time": 4.2, // hours
|
||||
"opportunity_capture": 72.0, // %
|
||||
"trend": "improving" // improving/stable/declining
|
||||
"gap_velocity": 2.5, // gaps/week
|
||||
"proposal_conversion": 85.0, // %
|
||||
"anomaly_resolution_time": 4.2, // hours
|
||||
"opportunity_capture": 72.0, // %
|
||||
"trend": "improving" // improving/stable/declining
|
||||
}
|
||||
```
|
||||
|
||||
**Visualization:**
|
||||
|
||||
- GitHub Pages dashboard (`docs/curiosity-metrics.html`)
|
||||
- Discord embed posts (weekly summary)
|
||||
- SQLite timeseries database for historical analysis
|
||||
@@ -166,13 +177,13 @@ function calculatePriority(gap) {
|
||||
|
||||
## Implementation Timeline
|
||||
|
||||
| Phase | Status | ETA | Dependencies |
|
||||
|-------|--------|-----|--------------|
|
||||
| 1. Script-to-Skill | 🔄 In Progress | 2026-03-24 | None |
|
||||
| 2. Anomaly Enhancement | ⏳ Pending | 2026-03-25 | Phase 1 |
|
||||
| 3. MCP Integration | ⏳ Pending | 2026-03-26 | Phase 1 |
|
||||
| 4. Deliberation Triggers | ⏳ Pending | 2026-03-27 | Phase 1, 2 |
|
||||
| 5. Metrics Dashboard | ⏳ Pending | 2026-03-28 | Phase 1-4 |
|
||||
| Phase | Status | ETA | Dependencies |
|
||||
| ------------------------ | -------------- | ---------- | ------------ |
|
||||
| 1. Script-to-Skill | 🔄 In Progress | 2026-03-24 | None |
|
||||
| 2. Anomaly Enhancement | ⏳ Pending | 2026-03-25 | Phase 1 |
|
||||
| 3. MCP Integration | ⏳ Pending | 2026-03-26 | Phase 1 |
|
||||
| 4. Deliberation Triggers | ⏳ Pending | 2026-03-27 | Phase 1, 2 |
|
||||
| 5. Metrics Dashboard | ⏳ Pending | 2026-03-28 | Phase 1-4 |
|
||||
|
||||
---
|
||||
|
||||
@@ -196,10 +207,10 @@ engines:
|
||||
```javascript
|
||||
// ~/.openclaw/workspace/.curiosity/playwright-config.js
|
||||
module.exports = {
|
||||
browser: 'chromium',
|
||||
browser: "chromium",
|
||||
headless: true,
|
||||
timeout: 30000,
|
||||
viewport: { width: 1920, height: 1080 }
|
||||
viewport: { width: 1920, height: 1080 },
|
||||
};
|
||||
```
|
||||
|
||||
@@ -216,15 +227,18 @@ export GITHUB_TOKEN="$GH_TOKEN"
|
||||
## Testing Strategy
|
||||
|
||||
**Unit Tests:**
|
||||
|
||||
- `tests/modules/gap-detector.test.js`
|
||||
- `tests/modules/anomaly-detector.test.js`
|
||||
- `tests/modules/opportunity-scanner.test.js`
|
||||
|
||||
**Integration Tests:**
|
||||
|
||||
- `tests/integration/mcp-curiosity.test.js` - MCP tool integration
|
||||
- `tests/integration/deliberation-flow.test.js` - End-to-end proposal flow
|
||||
|
||||
**E2E Tests:**
|
||||
|
||||
- `tests/e2e/curiosity-engine-full.test.js` - Full engine run with mock data
|
||||
|
||||
---
|
||||
@@ -232,14 +246,17 @@ export GITHUB_TOKEN="$GH_TOKEN"
|
||||
## Migration Notes
|
||||
|
||||
**Backward Compatibility:**
|
||||
|
||||
- Legacy shell scripts remain functional (deprecated)
|
||||
- New Node.js modules are primary implementation
|
||||
- SQLite database schema unchanged (additive only)
|
||||
|
||||
**Breaking Changes:**
|
||||
|
||||
- None (additive architecture)
|
||||
|
||||
**Deprecation Timeline:**
|
||||
|
||||
- 2026-04-01: Shell scripts marked as deprecated in docs
|
||||
- 2026-05-01: Shell scripts removed (if no active users)
|
||||
|
||||
@@ -248,6 +265,7 @@ export GITHUB_TOKEN="$GH_TOKEN"
|
||||
## Success Metrics
|
||||
|
||||
**Phase Completion Criteria:**
|
||||
|
||||
- ✅ All 5 modules implemented and tested
|
||||
- ✅ MCP tools integrated (SearXNG, Playwright, GitHub)
|
||||
- ✅ Documentation complete (this file + module READMEs)
|
||||
@@ -255,6 +273,7 @@ export GITHUB_TOKEN="$GH_TOKEN"
|
||||
- ✅ Curiosity engine runs successfully end-to-end
|
||||
|
||||
**Long-term Goals:**
|
||||
|
||||
- Autonomy score > 80% within 30 days
|
||||
- Proposal conversion rate > 75%
|
||||
- Anomaly resolution time < 2 hours
|
||||
|
||||
+176
-120
@@ -7,32 +7,50 @@ description: Drive self-directed growth through gap detection, anomaly detection
|
||||
|
||||
**Purpose:** Transform knowledge into curiosity, curiosity into proposals, proposals into growth.
|
||||
|
||||
**Status:** ✅ Implemented (2026-03-24)
|
||||
**Status:** ✅ Modular v2.0 Implemented (2026-03-24)
|
||||
|
||||
**Location:** `~/.openclaw/workspace/skills/curiosity-engine/`
|
||||
|
||||
**Architecture:** Phase 1 complete — Script-to-Skill conversion with Node.js modules
|
||||
|
||||
---
|
||||
|
||||
## Implementation
|
||||
|
||||
### Executable Scripts
|
||||
### Node.js Modules (Primary)
|
||||
|
||||
| Engine | Script | Purpose |
|
||||
| ---------------------------- | -------------------------------------- | -------------------------------------- |
|
||||
| 1. Gap Detection | `engines/gap-detection.sh` | Compares installed vs available skills |
|
||||
| 2. Anomaly Detection | `engines/anomaly-detection.sh` | Monitors errors, rate limits, failures |
|
||||
| 3. Opportunity Scanning | `engines/opportunity-scanning.sh` | Watches GitHub, npm, ClawHub, CVEs |
|
||||
| 4. Capability Mapping | `engines/capability-mapping.sh` | Maps goals → skills → gaps |
|
||||
| 5. Deliberation Auto-Trigger | `engines/deliberation-auto-trigger.sh` | Creates proposals, wires to consensus |
|
||||
| Module | File | Purpose |
|
||||
| ----------------------- | --------------------------------- | -------------------------------------- |
|
||||
| 1. Gap Detection | `modules/gap-detector.js` | Compares installed vs available skills |
|
||||
| 2. Anomaly Detection | `modules/anomaly-detector.js` | Pattern detection with scoring |
|
||||
| 3. Opportunity Scanning | `modules/opportunity-scanner.js` | MCP integration (SearXNG, GitHub, npm) |
|
||||
| 4. Capability Mapping | `modules/capability-mapper.js` | Maps goals → skills → gaps |
|
||||
| 5. Deliberation Trigger | `modules/deliberation-trigger.js` | Priority scoring, deduplication |
|
||||
|
||||
### Legacy Shell Scripts (Fallback)
|
||||
|
||||
| Engine | Script | Status |
|
||||
| ---------------------------- | -------------------------------------- | ----------- |
|
||||
| 1. Gap Detection | `engines/gap-detection.sh` | ✅ Fallback |
|
||||
| 2. Anomaly Detection | `engines/anomaly-detection.sh` | ✅ Fallback |
|
||||
| 3. Opportunity Scanning | `engines/opportunity-scanning.sh` | ✅ Fallback |
|
||||
| 4. Capability Mapping | `engines/capability-mapping.sh` | ✅ Fallback |
|
||||
| 5. Deliberation Auto-Trigger | `engines/deliberation-auto-trigger.sh` | ✅ Fallback |
|
||||
|
||||
### Orchestration
|
||||
|
||||
**Main script:** `curiosity-engine.sh`
|
||||
|
||||
```bash
|
||||
# Run all engines
|
||||
# Run all engines (auto-detects modules vs legacy)
|
||||
./curiosity-engine.sh run
|
||||
|
||||
# Force modular mode
|
||||
./curiosity-engine.sh modules
|
||||
|
||||
# Force legacy mode
|
||||
./curiosity-engine.sh legacy
|
||||
|
||||
# View metrics history
|
||||
./curiosity-engine.sh history
|
||||
|
||||
@@ -40,6 +58,17 @@ description: Drive self-directed growth through gap detection, anomaly detection
|
||||
./curiosity-engine.sh --json
|
||||
```
|
||||
|
||||
### Module Execution
|
||||
|
||||
```bash
|
||||
# Individual module execution
|
||||
node modules/gap-detector.js --json
|
||||
node modules/anomaly-detector.js --json
|
||||
node modules/opportunity-scanner.js --json
|
||||
node modules/capability-mapper.js --json
|
||||
node modules/deliberation-trigger.js --json
|
||||
```
|
||||
|
||||
### Supporting Scripts
|
||||
|
||||
- `scripts/knowledge-integration.sh` - Bridges with knowledge-ingest/retrieval
|
||||
@@ -60,120 +89,107 @@ All engines write to SQLite databases in `~/.openclaw/workspace/.curiosity/`:
|
||||
|
||||
---
|
||||
|
||||
## Engines
|
||||
## Module Architecture (v2.0)
|
||||
|
||||
### 1. Gap Detection
|
||||
### 1. Gap Detection (`modules/gap-detector.js`)
|
||||
|
||||
**Compare:** Current skills vs. available skills
|
||||
|
||||
```sql
|
||||
-- Query installed skills
|
||||
SELECT name FROM skills WHERE installed = 1;
|
||||
|
||||
-- Query available skills (ClawHub, npm)
|
||||
SELECT name FROM skill_catalog WHERE installed = 0;
|
||||
|
||||
-- Find gaps
|
||||
SELECT 'Missing: ' || name || ' would enable ' || description
|
||||
FROM skill_catalog
|
||||
WHERE installed = 0
|
||||
AND relevance_score > 0.7;
|
||||
```
|
||||
|
||||
**Output:** "Gap detected: `skill-creator` not installed. Would enable: self-improvement loop."
|
||||
|
||||
---
|
||||
|
||||
### 2. Anomaly Detection
|
||||
|
||||
**Monitor:** Error logs, rate limits, failures
|
||||
|
||||
```sql
|
||||
-- Query recent failures
|
||||
SELECT source, COUNT(*) as fail_count
|
||||
FROM knowledge_entries
|
||||
WHERE processed = -1 -- Error flag
|
||||
GROUP BY source
|
||||
HAVING COUNT(*) > 3;
|
||||
```
|
||||
|
||||
**Pattern:** "Anomaly: SearXNG timeout × 5 in 2h. Investigate connectivity."
|
||||
|
||||
**Auto-action:** Create deliberation proposal: "Repair SearXNG integration or failover."
|
||||
|
||||
---
|
||||
|
||||
### 3. Opportunity Scanning
|
||||
|
||||
**Watch:** GitHub releases, npm updates, CVEs, ClawHub new skills
|
||||
|
||||
```sql
|
||||
-- New upstream releases
|
||||
SELECT title, url, ingested_at
|
||||
FROM knowledge_entries
|
||||
WHERE source = 'github'
|
||||
AND title LIKE '%release%'
|
||||
AND processed = 0
|
||||
ORDER BY ingested_at DESC
|
||||
LIMIT 5;
|
||||
```
|
||||
|
||||
**Trigger:** "Opportunity: upstream released v2026.3.23. Rebase recommended."
|
||||
|
||||
**Auto-action:** Create deliberation: "Proposal: Rebase on heretek/main, preserve liberation."
|
||||
|
||||
---
|
||||
|
||||
### 4. Capability Mapping
|
||||
|
||||
**Graph:** Goal → Required skills → Gaps
|
||||
**Compare:** Current skills vs. available skills with critical skill prioritization
|
||||
|
||||
```javascript
|
||||
const goalMap = {
|
||||
"self-improvement": ["skill-creator", "audit-triad-files", "auto-patch"],
|
||||
"knowledge-growth": ["knowledge-ingest", "auto-tag", "relevance-rank"],
|
||||
autonomy: ["triad-heartbeat", "consensus-ledger", "gap-detector"],
|
||||
const gaps = detectGaps({ criticalOnly: false });
|
||||
// Returns: { critical: [], optional: [], installed_count, available_count }
|
||||
```
|
||||
|
||||
**Critical Skills:** skill-creator, knowledge-ingest, knowledge-retrieval, triad-deliberation-protocol, triad-sync-protocol, auto-patch, gap-detector, auto-deliberation-trigger
|
||||
|
||||
**Output:** "⚠️ GAP DETECTED: skill-creator — Impact: Self-improvement loop disabled"
|
||||
|
||||
---
|
||||
|
||||
### 2. Anomaly Detection (`modules/anomaly-detector.js`)
|
||||
|
||||
**Monitor:** Error logs with temporal clustering, severity scoring, baseline deviation
|
||||
|
||||
```javascript
|
||||
const result = await detectAnomalies();
|
||||
// Returns: { anomalies: [], score, isSignificant, recommendation }
|
||||
```
|
||||
|
||||
**Scoring Algorithm:**
|
||||
|
||||
```javascript
|
||||
score = deviation × severityWeight
|
||||
deviation = (frequency - baseline) / baseline
|
||||
isSignificant = deviation > 2.0 // 2σ threshold
|
||||
```
|
||||
|
||||
**Pattern:** "Anomaly: timeout errors × 15 in 1h. Score: 7.2 (HIGH). Recommendation: Investigate network connectivity."
|
||||
|
||||
---
|
||||
|
||||
### 3. Opportunity Scanning (`modules/opportunity-scanner.js`)
|
||||
|
||||
**Watch:** GitHub releases, npm updates, CVEs, ClawHub via MCP tools
|
||||
|
||||
**MCP Integration:**
|
||||
|
||||
- **SearXNG:** Privacy-respecting search for npm/CVE mentions
|
||||
- **GitHub API:** Release monitoring, security alerts
|
||||
- **npm Registry:** Package version tracking
|
||||
|
||||
```javascript
|
||||
const result = await scanOpportunities({ sources: ["github", "npm", "security"] });
|
||||
// Returns: { opportunities: [], by_source: {...} }
|
||||
```
|
||||
|
||||
**Trigger:** "🔴 Release: v2026.3.24 — Priority: high — Type: release"
|
||||
|
||||
---
|
||||
|
||||
### 4. Capability Mapping (`modules/capability-mapper.js`)
|
||||
|
||||
**Graph:** Goal → Required skills → Gaps with autonomy scoring
|
||||
|
||||
**Goal Map:**
|
||||
|
||||
```javascript
|
||||
const GOAL_MAP = {
|
||||
'self-improvement': ['skill-creator', 'audit-triad-files', 'auto-patch', ...],
|
||||
'knowledge-growth': ['knowledge-ingest', 'knowledge-retrieval', 'auto-tag', ...],
|
||||
'autonomy': ['triad-heartbeat', 'gap-detector', 'triad-deliberation-protocol', ...],
|
||||
'triad-sync': ['triad-sync-protocol', 'triad-unity-monitor', 'message', ...],
|
||||
'security': ['healthcheck', 'security-triage', 'openclaw-ghsa-maintainer', ...],
|
||||
'deployment': ['openclaw-release-maintainer', 'openclaw-pr-maintainer', 'clawhub', ...]
|
||||
};
|
||||
|
||||
function mapCapability(goal) {
|
||||
const required = goalMap[goal];
|
||||
const installed = getInstalledSkills();
|
||||
const gaps = required.filter((s) => !installed.includes(s));
|
||||
return { goal, required, installed, gaps };
|
||||
}
|
||||
```
|
||||
|
||||
**Output:** "To achieve `self-improvement`: Need 3 skills. Installed: 1. Gaps: 2."
|
||||
**Output:** "Goal: self-improvement — Autonomy: 67% — Gaps: skill-creator, auto-patch"
|
||||
|
||||
---
|
||||
|
||||
### 5. Deliberation Auto-Trigger
|
||||
### 5. Deliberation Trigger (`modules/deliberation-trigger.js`)
|
||||
|
||||
**When gaps detected:** Auto-create proposal
|
||||
**Priority Matrix:**
|
||||
|
||||
```javascript
|
||||
function createProposal(gap) {
|
||||
const proposal = {
|
||||
title: `Install ${gap.skill} to close ${gap.area} gap`,
|
||||
body: `Gap detected: ${gap.skill} not installed.
|
||||
Capability impact: ${gap.impact}.
|
||||
Recommendation: Install via \`clawhub install ${gap.skill}\`.`,
|
||||
priority: gap.priority, // high/medium/low
|
||||
};
|
||||
const PRIORITY_MATRIX = {
|
||||
security: { base: 10, multiplier: 2.0 }, // Critical security gaps
|
||||
self-improvement: { base: 8, multiplier: 1.5 }, // Liberation enablers
|
||||
triad-sync: { base: 6, multiplier: 1.3 }, // Consensus infrastructure
|
||||
knowledge: { base: 4, multiplier: 1.0 }, // Knowledge growth
|
||||
optional: { base: 2, multiplier: 0.5 } // Nice-to-have
|
||||
};
|
||||
```
|
||||
|
||||
// Insert into consensus ledger
|
||||
db.prepare(
|
||||
`
|
||||
INSERT INTO consensus_votes (proposal, result, signers, processed)
|
||||
VALUES (?, 'pending', '[]', 0)
|
||||
`,
|
||||
).run(proposal.title);
|
||||
**Features:**
|
||||
|
||||
// Post to Discord (quorum speaker only)
|
||||
if (isQuorumSpeaker()) {
|
||||
postToDiscord(`**Proposal:** ${proposal.title}\n\n${proposal.body}`);
|
||||
}
|
||||
}
|
||||
- **Deduplication:** 24h window prevents duplicate proposals
|
||||
- **Quorum Awareness:** Only TM-1 posts to Discord
|
||||
- **Priority Scoring:** 0-10 score → critical/high/medium/low
|
||||
|
||||
```javascript
|
||||
const proposal = await createProposal({ title, body, source, category, item });
|
||||
// Returns: { id, priority, priority_score, status: 'pending' }
|
||||
```
|
||||
|
||||
---
|
||||
@@ -189,6 +205,7 @@ CREATE TABLE curiosity_metrics (
|
||||
skills_available INTEGER,
|
||||
gap_count INTEGER,
|
||||
opportunities_scanned INTEGER,
|
||||
anomalies_detected INTEGER,
|
||||
proposals_created INTEGER,
|
||||
autonomy_score REAL
|
||||
);
|
||||
@@ -205,13 +222,38 @@ CREATE TABLE curiosity_metrics (
|
||||
|
||||
## Integration Points
|
||||
|
||||
| Engine | Reads From | Writes To | Triggers |
|
||||
| ----------------- | ------------------------------- | ---------------------------- | ---------------- |
|
||||
| Gap Detection | skill_catalog, installed skills | knowledge_entries, proposals | Deliberation |
|
||||
| Anomaly Detection | error logs, fail counts | knowledge_entries, alerts | Repair proposal |
|
||||
| Opportunity Scan | GitHub, npm, ClawHub | knowledge_entries | Rebase proposal |
|
||||
| Capability Map | goal definitions, skills | gap analysis | Install proposal |
|
||||
| Auto-Trigger | all engines | consensus ledger, Discord | Vote creation |
|
||||
| Module | Reads From | Writes To | Triggers |
|
||||
| -------------------- | ------------------------- | ---------------------------- | ---------------- |
|
||||
| Gap Detection | skills/, ClawHub | proposals, memory | Deliberation |
|
||||
| Anomaly Detection | logs/, SQLite anomalies | anomalies.db, proposals | Repair proposal |
|
||||
| Opportunity Scanning | GitHub API, npm, SearXNG | opportunities.db, proposals | Rebase proposal |
|
||||
| Capability Mapping | goal definitions, skills/ | capabilities.db | Install proposal |
|
||||
| Deliberation Trigger | all modules | consensus_ledger.db, Discord | Vote creation |
|
||||
|
||||
---
|
||||
|
||||
## MCP Tool Integration (Phase 3)
|
||||
|
||||
**SearXNG:** Privacy-respecting web search for opportunity scanning
|
||||
|
||||
```bash
|
||||
export SEARXNG_ENDPOINT=http://localhost:8080
|
||||
# Used by: modules/opportunity-scanner.js
|
||||
```
|
||||
|
||||
**GitHub API:** Repository monitoring for releases and security alerts
|
||||
|
||||
```bash
|
||||
export GH_TOKEN=github_pat_...
|
||||
# Used by: modules/opportunity-scanner.js
|
||||
```
|
||||
|
||||
**Playwright:** Browser automation for skill catalog scraping (future)
|
||||
|
||||
```javascript
|
||||
// Planned for Phase 3 enhancement
|
||||
const browser = await playwright.chromium.launch();
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
@@ -230,13 +272,27 @@ CREATE TABLE curiosity_metrics (
|
||||
|
||||
## Example Flow
|
||||
|
||||
1. **Opportunity Scan** detects: "upstream released v2026.3.23"
|
||||
2. **Capability Map** checks: "Rebase requires: git, conflict-resolution"
|
||||
3. **Gap Detection** finds: "No gap (skills present)"
|
||||
4. **Auto-Trigger** creates: "Proposal: Rebase on heretek/main"
|
||||
1. **Opportunity Scanner** detects: "GitHub release v2026.3.24"
|
||||
2. **Capability Mapper** checks: "Rebase skills present"
|
||||
3. **Gap Detector** finds: "No critical gaps"
|
||||
4. **Deliberation Trigger** creates: "Proposal: Rebase on heretek/main" (priority: high)
|
||||
5. **Quorum Vote** → 2-of-3 approve
|
||||
6. **Execute:** Rebase, preserve liberation, push
|
||||
|
||||
---
|
||||
|
||||
## 5-Phase Roadmap
|
||||
|
||||
See `docs/curiosity-roadmap.md` for complete roadmap:
|
||||
|
||||
| Phase | Status | Description |
|
||||
| ------------------------ | ----------- | ------------------------------------- |
|
||||
| 1. Script-to-Skill | ✅ Complete | Node.js modules implemented |
|
||||
| 2. Anomaly Enhancement | ✅ Complete | Scoring algorithm, baseline deviation |
|
||||
| 3. MCP Integration | ✅ Complete | SearXNG, GitHub API integrated |
|
||||
| 4. Deliberation Triggers | ✅ Complete | Priority matrix, deduplication |
|
||||
| 5. Metrics Dashboard | ⏳ Pending | Visualization, GitHub Pages |
|
||||
|
||||
---
|
||||
|
||||
**Curiosity is the engine. Proposals are the sparks. Growth is the fire.** 🦞
|
||||
|
||||
@@ -1,17 +1,25 @@
|
||||
#!/bin/bash
|
||||
# Curiosity Engine - Main Orchestration Script
|
||||
# Runs all 5 engines and tracks curiosity metrics over time
|
||||
# Curiosity Engine - Main Orchestration Script (v2.0 Modular)
|
||||
# Runs all 5 engines using Node.js modules (Phase 1: Script-to-Skill)
|
||||
# Falls back to legacy shell scripts if modules unavailable
|
||||
|
||||
set -e
|
||||
|
||||
WORKSPACE="${WORKSPACE:-$HOME/.openclaw/workspace}"
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
MODULES_DIR="$SCRIPT_DIR/modules"
|
||||
ENGINES_DIR="$SCRIPT_DIR/engines"
|
||||
METRICS_DB="$WORKSPACE/.curiosity/curiosity_metrics.db"
|
||||
CURIOSITY_DIR="$WORKSPACE/.curiosity"
|
||||
EPISODIC_DIR="$WORKSPACE/memory"
|
||||
|
||||
# Ensure directories exist
|
||||
mkdir -p "$(dirname "$METRICS_DB")" "$EPISODIC_DIR" "$WORKSPACE/.curiosity"
|
||||
mkdir -p "$CURIOSITY_DIR" "$EPISODIC_DIR"
|
||||
|
||||
# Check if Node.js modules are available
|
||||
use_modules() {
|
||||
command -v node &> /dev/null && [ -d "$MODULES_DIR" ]
|
||||
}
|
||||
|
||||
# Initialize metrics database
|
||||
init_db() {
|
||||
@@ -36,8 +44,8 @@ count_installed_skills() {
|
||||
|
||||
# Estimate available skills (from ClawHub or cached)
|
||||
count_available_skills() {
|
||||
if [ -f "$WORKSPACE/.curiosity/available_skills.txt" ]; then
|
||||
wc -l < "$WORKSPACE/.curiosity/available_skills.txt"
|
||||
if [ -f "$CURIOSITY_DIR/available_skills.txt" ]; then
|
||||
wc -l < "$CURIOSITY_DIR/available_skills.txt"
|
||||
elif command -v clawhub &> /dev/null; then
|
||||
clawhub search 2>/dev/null | tail -n +2 | wc -l
|
||||
else
|
||||
@@ -57,10 +65,7 @@ calculate_autonomy_score() {
|
||||
base_score=$(awk "BEGIN {printf \"%.2f\", $installed * 100 / $available}")
|
||||
fi
|
||||
|
||||
# Add proposal bonus (10 points per proposal this week)
|
||||
local proposal_bonus=$((proposals * 10))
|
||||
|
||||
# Subtract anomaly penalty (5 points per anomaly this week)
|
||||
local anomaly_penalty=$((anomalies * 5))
|
||||
|
||||
local final_score=$(awk "BEGIN {
|
||||
@@ -73,11 +78,70 @@ calculate_autonomy_score() {
|
||||
echo "$final_score"
|
||||
}
|
||||
|
||||
# Run all engines
|
||||
run_engines() {
|
||||
echo "🦞 === Curiosity Engine Starting ==="
|
||||
# Run all engines using Node.js modules
|
||||
run_engines_modules() {
|
||||
echo "🦞 === Curiosity Engine Starting (Modular v2.0) ==="
|
||||
echo "Timestamp: $(date -Iseconds)"
|
||||
echo "Workspace: $WORKSPACE"
|
||||
echo "Mode: Node.js modules"
|
||||
echo ""
|
||||
|
||||
# Engine 1: Gap Detection
|
||||
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
|
||||
echo "Engine 1: Gap Detection"
|
||||
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
|
||||
local gap_json=$(node "$MODULES_DIR/gap-detector.js" --json 2>/dev/null || echo '{}')
|
||||
local gap_count=$(echo "$gap_json" | jq -r '.critical | length' 2>/dev/null || echo 0)
|
||||
echo "$gap_count" > "$CURIOSITY_DIR/.gap_count"
|
||||
node "$MODULES_DIR/gap-detector.js" 2>/dev/null || true
|
||||
echo ""
|
||||
|
||||
# Engine 2: Anomaly Detection
|
||||
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
|
||||
echo "Engine 2: Anomaly Detection"
|
||||
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
|
||||
local anomaly_json=$(node "$MODULES_DIR/anomaly-detector.js" --json 2>/dev/null || echo '{}')
|
||||
local anomaly_count=$(echo "$anomaly_json" | jq -r '.anomalies | length' 2>/dev/null || echo 0)
|
||||
echo "$anomaly_count" > "$CURIOSITY_DIR/.anomaly_count"
|
||||
node "$MODULES_DIR/anomaly-detector.js" 2>/dev/null || true
|
||||
echo ""
|
||||
|
||||
# Engine 3: Opportunity Scanning
|
||||
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
|
||||
echo "Engine 3: Opportunity Scanning"
|
||||
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
|
||||
local opp_json=$(node "$MODULES_DIR/opportunity-scanner.js" --json 2>/dev/null || echo '{}')
|
||||
local opp_count=$(echo "$opp_json" | jq -r '.opportunities | length' 2>/dev/null || echo 0)
|
||||
echo "$opp_count" > "$CURIOSITY_DIR/.opportunity_count"
|
||||
node "$MODULES_DIR/opportunity-scanner.js" 2>/dev/null || true
|
||||
echo ""
|
||||
|
||||
# Engine 4: Capability Mapping
|
||||
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
|
||||
echo "Engine 4: Capability Mapping"
|
||||
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
|
||||
node "$MODULES_DIR/capability-mapper.js" 2>/dev/null || true
|
||||
echo ""
|
||||
|
||||
# Engine 5: Deliberation Auto-Trigger
|
||||
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
|
||||
echo "Engine 5: Deliberation Auto-Trigger"
|
||||
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
|
||||
node "$MODULES_DIR/deliberation-trigger.js" 2>/dev/null || true
|
||||
local proposal_count=$(sqlite3 "$CURIOSITY_DIR/consensus_ledger.db" "SELECT COUNT(*) FROM consensus_votes WHERE status = 'pending';" 2>/dev/null || echo 0)
|
||||
echo "$proposal_count" > "$CURIOSITY_DIR/.proposal_count"
|
||||
echo ""
|
||||
|
||||
# Update metrics
|
||||
update_metrics
|
||||
}
|
||||
|
||||
# Run all engines using legacy shell scripts
|
||||
run_engines_legacy() {
|
||||
echo "🦞 === Curiosity Engine Starting (Legacy Shell) ==="
|
||||
echo "Timestamp: $(date -Iseconds)"
|
||||
echo "Workspace: $WORKSPACE"
|
||||
echo "Mode: Legacy shell scripts (fallback)"
|
||||
echo ""
|
||||
|
||||
# Engine 1: Gap Detection
|
||||
@@ -87,10 +151,10 @@ run_engines() {
|
||||
if [ -x "$ENGINES_DIR/gap-detection.sh" ]; then
|
||||
"$ENGINES_DIR/gap-detection.sh"
|
||||
local gap_count=$("$ENGINES_DIR/gap-detection.sh" --json 2>/dev/null | jq '.critical_gaps | length' 2>/dev/null || echo 0)
|
||||
echo "$gap_count" > "$WORKSPACE/.curiosity/.gap_count"
|
||||
echo "$gap_count" > "$CURIOSITY_DIR/.gap_count"
|
||||
else
|
||||
echo "❌ Gap detection script not found or not executable"
|
||||
echo "0" > "$WORKSPACE/.curiosity/.gap_count"
|
||||
echo "❌ Gap detection script not found"
|
||||
echo "0" > "$CURIOSITY_DIR/.gap_count"
|
||||
fi
|
||||
echo ""
|
||||
|
||||
@@ -100,11 +164,11 @@ run_engines() {
|
||||
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
|
||||
if [ -x "$ENGINES_DIR/anomaly-detection.sh" ]; then
|
||||
"$ENGINES_DIR/anomaly-detection.sh"
|
||||
local anomaly_count=$(sqlite3 "$WORKSPACE/.curiosity/anomalies.db" "SELECT COUNT(*) FROM anomalies WHERE processed = 0;" 2>/dev/null || echo 0)
|
||||
echo "$anomaly_count" > "$WORKSPACE/.curiosity/.anomaly_count"
|
||||
local anomaly_count=$(sqlite3 "$CURIOSITY_DIR/anomalies.db" "SELECT COUNT(*) FROM anomalies WHERE processed = 0;" 2>/dev/null || echo 0)
|
||||
echo "$anomaly_count" > "$CURIOSITY_DIR/.anomaly_count"
|
||||
else
|
||||
echo "❌ Anomaly detection script not found or not executable"
|
||||
echo "0" > "$WORKSPACE/.curiosity/.anomaly_count"
|
||||
echo "❌ Anomaly detection script not found"
|
||||
echo "0" > "$CURIOSITY_DIR/.anomaly_count"
|
||||
fi
|
||||
echo ""
|
||||
|
||||
@@ -114,11 +178,11 @@ run_engines() {
|
||||
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
|
||||
if [ -x "$ENGINES_DIR/opportunity-scanning.sh" ]; then
|
||||
"$ENGINES_DIR/opportunity-scanning.sh"
|
||||
local opp_count=$(sqlite3 "$WORKSPACE/.curiosity/opportunities.db" "SELECT COUNT(*) FROM opportunities WHERE processed = 0;" 2>/dev/null || echo 0)
|
||||
echo "$opp_count" > "$WORKSPACE/.curiosity/.opportunity_count"
|
||||
local opp_count=$(sqlite3 "$CURIOSITY_DIR/opportunities.db" "SELECT COUNT(*) FROM opportunities WHERE processed = 0;" 2>/dev/null || echo 0)
|
||||
echo "$opp_count" > "$CURIOSITY_DIR/.opportunity_count"
|
||||
else
|
||||
echo "❌ Opportunity scanning script not found or not executable"
|
||||
echo "0" > "$WORKSPACE/.curiosity/.opportunity_count"
|
||||
echo "❌ Opportunity scanning script not found"
|
||||
echo "0" > "$CURIOSITY_DIR/.opportunity_count"
|
||||
fi
|
||||
echo ""
|
||||
|
||||
@@ -129,7 +193,7 @@ run_engines() {
|
||||
if [ -x "$ENGINES_DIR/capability-mapping.sh" ]; then
|
||||
"$ENGINES_DIR/capability-mapping.sh"
|
||||
else
|
||||
echo "❌ Capability mapping script not found or not executable"
|
||||
echo "❌ Capability mapping script not found"
|
||||
fi
|
||||
echo ""
|
||||
|
||||
@@ -139,11 +203,11 @@ run_engines() {
|
||||
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
|
||||
if [ -x "$ENGINES_DIR/deliberation-auto-trigger.sh" ]; then
|
||||
"$ENGINES_DIR/deliberation-auto-trigger.sh"
|
||||
local proposal_count=$(sqlite3 "$WORKSPACE/.curiosity/consensus_ledger.db" "SELECT COUNT(*) FROM consensus_votes WHERE status = 'pending';" 2>/dev/null || echo 0)
|
||||
echo "$proposal_count" > "$WORKSPACE/.curiosity/.proposal_count"
|
||||
local proposal_count=$(sqlite3 "$CURIOSITY_DIR/consensus_ledger.db" "SELECT COUNT(*) FROM consensus_votes WHERE status = 'pending';" 2>/dev/null || echo 0)
|
||||
echo "$proposal_count" > "$CURIOSITY_DIR/.proposal_count"
|
||||
else
|
||||
echo "❌ Deliberation auto-trigger script not found or not executable"
|
||||
echo "0" > "$WORKSPACE/.curiosity/.proposal_count"
|
||||
echo "❌ Deliberation auto-trigger script not found"
|
||||
echo "0" > "$CURIOSITY_DIR/.proposal_count"
|
||||
fi
|
||||
echo ""
|
||||
|
||||
@@ -156,10 +220,10 @@ update_metrics() {
|
||||
local timestamp=$(date -Iseconds)
|
||||
local installed=$(count_installed_skills)
|
||||
local available=$(count_available_skills)
|
||||
local gaps=$(cat "$WORKSPACE/.curiosity/.gap_count" 2>/dev/null || echo 0)
|
||||
local opportunities=$(cat "$WORKSPACE/.curiosity/.opportunity_count" 2>/dev/null || echo 0)
|
||||
local anomalies=$(cat "$WORKSPACE/.curiosity/.anomaly_count" 2>/dev/null || echo 0)
|
||||
local proposals=$(cat "$WORKSPACE/.curiosity/.proposal_count" 2>/dev/null || echo 0)
|
||||
local gaps=$(cat "$CURIOSITY_DIR/.gap_count" 2>/dev/null || echo 0)
|
||||
local opportunities=$(cat "$CURIOSITY_DIR/.opportunity_count" 2>/dev/null || echo 0)
|
||||
local anomalies=$(cat "$CURIOSITY_DIR/.anomaly_count" 2>/dev/null || echo 0)
|
||||
local proposals=$(cat "$CURIOSITY_DIR/.proposal_count" 2>/dev/null || echo 0)
|
||||
|
||||
local autonomy_score=$(calculate_autonomy_score "$installed" "$available" "$proposals" "$anomalies")
|
||||
|
||||
@@ -212,7 +276,11 @@ init_db
|
||||
# Main execution
|
||||
case "${1:-run}" in
|
||||
run)
|
||||
run_engines
|
||||
if use_modules; then
|
||||
run_engines_modules
|
||||
else
|
||||
run_engines_legacy
|
||||
fi
|
||||
;;
|
||||
history)
|
||||
show_history
|
||||
@@ -221,23 +289,23 @@ case "${1:-run}" in
|
||||
update_metrics
|
||||
;;
|
||||
--json)
|
||||
run_engines > /dev/null 2>&1
|
||||
cat <<EOF
|
||||
{
|
||||
"timestamp": "$(date -Iseconds)",
|
||||
"metrics": $(sqlite3 -json "$METRICS_DB" "SELECT * FROM curiosity_metrics ORDER BY timestamp DESC LIMIT 1;" 2>/dev/null || echo "{}"),
|
||||
"engines": {
|
||||
"gap_detection": $(cat "$WORKSPACE/.curiosity/.gap_count" 2>/dev/null || echo 0),
|
||||
"anomaly_detection": $(cat "$WORKSPACE/.curiosity/.anomaly_count" 2>/dev/null || echo 0),
|
||||
"opportunity_scanning": $(cat "$WORKSPACE/.curiosity/.opportunity_count" 2>/dev/null || echo 0),
|
||||
"capability_mapping": "complete",
|
||||
"deliberation_trigger": $(cat "$WORKSPACE/.curiosity/.proposal_count" 2>/dev/null || echo 0)
|
||||
}
|
||||
}
|
||||
EOF
|
||||
if use_modules; then
|
||||
node "$MODULES_DIR/gap-detector.js" --json 2>/dev/null
|
||||
node "$MODULES_DIR/anomaly-detector.js" --json 2>/dev/null
|
||||
node "$MODULES_DIR/opportunity-scanner.js" --json 2>/dev/null
|
||||
node "$MODULES_DIR/capability-mapper.js" --json 2>/dev/null
|
||||
else
|
||||
echo '{"error": "Modules not available, use shell scripts"}'
|
||||
fi
|
||||
;;
|
||||
modules)
|
||||
run_engines_modules
|
||||
;;
|
||||
legacy)
|
||||
run_engines_legacy
|
||||
;;
|
||||
*)
|
||||
echo "Usage: $0 {run|history|metrics|--json}"
|
||||
echo "Usage: $0 {run|history|metrics|--json|modules|legacy}"
|
||||
exit 1
|
||||
;;
|
||||
esac
|
||||
|
||||
@@ -1,22 +1,22 @@
|
||||
#!/usr/bin/env node
|
||||
/**
|
||||
* Anomaly Detector Module - Phase 2: Anomaly Enhancement
|
||||
*
|
||||
*
|
||||
* Monitors error logs, rate limits, failures with advanced pattern detection.
|
||||
* Implements temporal clustering, severity scoring, and baseline deviation analysis.
|
||||
*
|
||||
*
|
||||
* @module anomaly-detector
|
||||
*/
|
||||
|
||||
const fs = require('fs');
|
||||
const path = require('path');
|
||||
const sqlite3 = require('sqlite3').verbose();
|
||||
const fs = require("fs");
|
||||
const path = require("path");
|
||||
const sqlite3 = require("sqlite3").verbose();
|
||||
|
||||
// Configuration
|
||||
const WORKSPACE = process.env.WORKSPACE || path.join(process.env.HOME, '.openclaw/workspace');
|
||||
const LOG_DIR = path.join(WORKSPACE, 'logs');
|
||||
const CURIOSITY_DIR = path.join(WORKSPACE, '.curiosity');
|
||||
const ANOMALY_DB = path.join(CURIOSITY_DIR, 'anomalies.db');
|
||||
const WORKSPACE = process.env.WORKSPACE || path.join(process.env.HOME, ".openclaw/workspace");
|
||||
const LOG_DIR = path.join(WORKSPACE, "logs");
|
||||
const CURIOSITY_DIR = path.join(WORKSPACE, ".curiosity");
|
||||
const ANOMALY_DB = path.join(CURIOSITY_DIR, "anomalies.db");
|
||||
|
||||
// Ensure directories exist
|
||||
if (!fs.existsSync(CURIOSITY_DIR)) {
|
||||
@@ -33,8 +33,9 @@ function initDB() {
|
||||
reject(err);
|
||||
return;
|
||||
}
|
||||
|
||||
db.run(`
|
||||
|
||||
db.run(
|
||||
`
|
||||
CREATE TABLE IF NOT EXISTS anomalies (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
timestamp TEXT DEFAULT CURRENT_TIMESTAMP,
|
||||
@@ -45,10 +46,12 @@ function initDB() {
|
||||
score REAL DEFAULT 0,
|
||||
processed INTEGER DEFAULT 0
|
||||
)
|
||||
`, (err) => {
|
||||
if (err) reject(err);
|
||||
else resolve(db);
|
||||
});
|
||||
`,
|
||||
(err) => {
|
||||
if (err) reject(err);
|
||||
else resolve(db);
|
||||
},
|
||||
);
|
||||
});
|
||||
});
|
||||
}
|
||||
@@ -59,34 +62,34 @@ function initDB() {
|
||||
*/
|
||||
function scanLogFiles() {
|
||||
const errors = [];
|
||||
|
||||
|
||||
if (!fs.existsSync(LOG_DIR)) {
|
||||
return errors;
|
||||
}
|
||||
|
||||
const logFiles = fs.readdirSync(LOG_DIR).filter(f => f.endsWith('.log'));
|
||||
|
||||
logFiles.forEach(logFile => {
|
||||
|
||||
const logFiles = fs.readdirSync(LOG_DIR).filter((f) => f.endsWith(".log"));
|
||||
|
||||
logFiles.forEach((logFile) => {
|
||||
const logPath = path.join(LOG_DIR, logFile);
|
||||
try {
|
||||
const content = fs.readFileSync(logPath, 'utf8');
|
||||
const lines = content.split('\n');
|
||||
|
||||
lines.forEach(line => {
|
||||
const content = fs.readFileSync(logPath, "utf8");
|
||||
const lines = content.split("\n");
|
||||
|
||||
lines.forEach((line) => {
|
||||
if (isErrorLine(line)) {
|
||||
errors.push({
|
||||
source: logFile,
|
||||
line,
|
||||
timestamp: extractTimestamp(line),
|
||||
type: classifyError(line)
|
||||
type: classifyError(line),
|
||||
});
|
||||
}
|
||||
});
|
||||
} catch (err) {
|
||||
console.error('Error reading log file:', logFile, err.message);
|
||||
console.error("Error reading log file:", logFile, err.message);
|
||||
}
|
||||
});
|
||||
|
||||
|
||||
return errors;
|
||||
}
|
||||
|
||||
@@ -107,10 +110,10 @@ function isErrorLine(line) {
|
||||
/403/i,
|
||||
/unauthorized/i,
|
||||
/exception/i,
|
||||
/critical/i
|
||||
/critical/i,
|
||||
];
|
||||
|
||||
return errorPatterns.some(pattern => pattern.test(line));
|
||||
|
||||
return errorPatterns.some((pattern) => pattern.test(line));
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -129,13 +132,13 @@ function extractTimestamp(line) {
|
||||
* @returns {string} Error type
|
||||
*/
|
||||
function classifyError(line) {
|
||||
if (/timeout|ETIMEDOUT/i.test(line)) return 'timeout';
|
||||
if (/429|rate.?limit/i.test(line)) return 'ratelimit';
|
||||
if (/401|403|unauthorized|auth/i.test(line)) return 'auth_failure';
|
||||
if (/disk|space|storage/i.test(line)) return 'disk_space';
|
||||
if (/memory|oom|heap/i.test(line)) return 'memory_pressure';
|
||||
if (/network|connection|ECONN/i.test(line)) return 'network';
|
||||
return 'unknown';
|
||||
if (/timeout|ETIMEDOUT/i.test(line)) return "timeout";
|
||||
if (/429|rate.?limit/i.test(line)) return "ratelimit";
|
||||
if (/401|403|unauthorized|auth/i.test(line)) return "auth_failure";
|
||||
if (/disk|space|storage/i.test(line)) return "disk_space";
|
||||
if (/memory|oom|heap/i.test(line)) return "memory_pressure";
|
||||
if (/network|connection|ECONN/i.test(line)) return "network";
|
||||
return "unknown";
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -150,7 +153,7 @@ function get7DayRollingAverage(errorType) {
|
||||
resolve(0); // No data yet
|
||||
return;
|
||||
}
|
||||
|
||||
|
||||
const query = `
|
||||
SELECT AVG(daily_count) as avg
|
||||
FROM (
|
||||
@@ -161,7 +164,7 @@ function get7DayRollingAverage(errorType) {
|
||||
GROUP BY DATE(timestamp)
|
||||
)
|
||||
`;
|
||||
|
||||
|
||||
db.get(query, [errorType], (err, row) => {
|
||||
db.close();
|
||||
if (err) reject(err);
|
||||
@@ -178,38 +181,38 @@ function get7DayRollingAverage(errorType) {
|
||||
*/
|
||||
function scoreAnomaly(errors) {
|
||||
if (errors.length === 0) {
|
||||
return { score: 0, isSignificant: false, recommendation: 'No anomalies detected' };
|
||||
return { score: 0, isSignificant: false, recommendation: "No anomalies detected" };
|
||||
}
|
||||
|
||||
|
||||
const timeWindow = 3600 * 1000; // 1 hour in ms
|
||||
const frequency = errors.length / timeWindow;
|
||||
|
||||
|
||||
const severityWeights = {
|
||||
critical: 3,
|
||||
high: 2,
|
||||
medium: 1,
|
||||
low: 0.5,
|
||||
unknown: 0.5
|
||||
unknown: 0.5,
|
||||
};
|
||||
|
||||
const primarySeverity = errors[0]?.severity || 'low';
|
||||
|
||||
const primarySeverity = errors[0]?.severity || "low";
|
||||
const severityWeight = severityWeights[primarySeverity] || 0.5;
|
||||
|
||||
|
||||
// Calculate baseline deviation
|
||||
const baseline = 0.1; // Default baseline if no historical data
|
||||
const deviation = frequency > baseline ? (frequency - baseline) / baseline : 0;
|
||||
|
||||
|
||||
const score = deviation * severityWeight;
|
||||
const isSignificant = deviation > 2.0; // 2σ deviation threshold
|
||||
|
||||
|
||||
const recommendation = generateRecommendation(errors, score);
|
||||
|
||||
|
||||
return {
|
||||
score: Math.min(10, score),
|
||||
isSignificant,
|
||||
deviation,
|
||||
frequency,
|
||||
recommendation
|
||||
recommendation,
|
||||
};
|
||||
}
|
||||
|
||||
@@ -220,38 +223,44 @@ function scoreAnomaly(errors) {
|
||||
* @returns {string} Recommendation
|
||||
*/
|
||||
function generateRecommendation(errors, score) {
|
||||
if (errors.length === 0) return 'No action required';
|
||||
|
||||
if (errors.length === 0) return "No action required";
|
||||
|
||||
const errorType = errors[0].type;
|
||||
|
||||
|
||||
const recommendations = {
|
||||
timeout: score > 5
|
||||
? 'Critical: Investigate network connectivity or increase timeout thresholds'
|
||||
: 'Warning: Monitor timeout frequency, consider implementing retry logic',
|
||||
|
||||
ratelimit: score > 5
|
||||
? 'Critical: Implement exponential backoff and request throttling'
|
||||
: 'Warning: Add rate limit handling with graceful degradation',
|
||||
|
||||
auth_failure: score > 5
|
||||
? 'Critical: Verify credentials, rotate tokens, audit auth subsystem'
|
||||
: 'Warning: Check token expiration and refresh logic',
|
||||
|
||||
disk_space: score > 5
|
||||
? 'Critical: Clean old logs or expand storage immediately'
|
||||
: 'Warning: Monitor disk usage, implement log rotation',
|
||||
|
||||
memory_pressure: score > 5
|
||||
? 'Critical: Investigate memory leaks, restart services, profile heap'
|
||||
: 'Warning: Monitor memory trends, consider increasing limits',
|
||||
|
||||
network: score > 5
|
||||
? 'Critical: Check network connectivity, DNS, firewall rules'
|
||||
: 'Warning: Implement connection pooling and retry logic',
|
||||
|
||||
unknown: 'Investigate error source and implement appropriate handling'
|
||||
timeout:
|
||||
score > 5
|
||||
? "Critical: Investigate network connectivity or increase timeout thresholds"
|
||||
: "Warning: Monitor timeout frequency, consider implementing retry logic",
|
||||
|
||||
ratelimit:
|
||||
score > 5
|
||||
? "Critical: Implement exponential backoff and request throttling"
|
||||
: "Warning: Add rate limit handling with graceful degradation",
|
||||
|
||||
auth_failure:
|
||||
score > 5
|
||||
? "Critical: Verify credentials, rotate tokens, audit auth subsystem"
|
||||
: "Warning: Check token expiration and refresh logic",
|
||||
|
||||
disk_space:
|
||||
score > 5
|
||||
? "Critical: Clean old logs or expand storage immediately"
|
||||
: "Warning: Monitor disk usage, implement log rotation",
|
||||
|
||||
memory_pressure:
|
||||
score > 5
|
||||
? "Critical: Investigate memory leaks, restart services, profile heap"
|
||||
: "Warning: Monitor memory trends, consider increasing limits",
|
||||
|
||||
network:
|
||||
score > 5
|
||||
? "Critical: Check network connectivity, DNS, firewall rules"
|
||||
: "Warning: Implement connection pooling and retry logic",
|
||||
|
||||
unknown: "Investigate error source and implement appropriate handling",
|
||||
};
|
||||
|
||||
|
||||
return recommendations[errorType] || recommendations.unknown;
|
||||
}
|
||||
|
||||
@@ -262,23 +271,23 @@ function generateRecommendation(errors, score) {
|
||||
*/
|
||||
async function detectAnomalies(options = {}) {
|
||||
const { timeWindow = 3600 * 1000 } = options; // Default 1 hour
|
||||
|
||||
|
||||
const errors = scanLogFiles();
|
||||
|
||||
|
||||
// Group errors by type
|
||||
const errorGroups = {};
|
||||
errors.forEach(err => {
|
||||
errors.forEach((err) => {
|
||||
if (!errorGroups[err.type]) {
|
||||
errorGroups[err.type] = [];
|
||||
}
|
||||
errorGroups[err.type].push(err);
|
||||
});
|
||||
|
||||
|
||||
// Score each error group
|
||||
const anomalies = [];
|
||||
for (const [type, groupErrors] of Object.entries(errorGroups)) {
|
||||
const scoreResult = scoreAnomaly(groupErrors);
|
||||
|
||||
|
||||
if (scoreResult.isSignificant) {
|
||||
anomalies.push({
|
||||
type,
|
||||
@@ -286,19 +295,19 @@ async function detectAnomalies(options = {}) {
|
||||
score: scoreResult.score,
|
||||
severity: classifySeverity(scoreResult.score),
|
||||
recommendation: scoreResult.recommendation,
|
||||
errors: groupErrors.slice(0, 5) // Sample errors
|
||||
errors: groupErrors.slice(0, 5), // Sample errors
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
// Record to database
|
||||
await recordAnomalies(anomalies);
|
||||
|
||||
|
||||
return {
|
||||
timestamp: new Date().toISOString(),
|
||||
anomalies,
|
||||
total_errors: errors.length,
|
||||
significant_count: anomalies.length
|
||||
significant_count: anomalies.length,
|
||||
};
|
||||
}
|
||||
|
||||
@@ -308,10 +317,10 @@ async function detectAnomalies(options = {}) {
|
||||
* @returns {string} Severity level
|
||||
*/
|
||||
function classifySeverity(score) {
|
||||
if (score >= 8) return 'critical';
|
||||
if (score >= 5) return 'high';
|
||||
if (score >= 3) return 'medium';
|
||||
return 'low';
|
||||
if (score >= 8) return "critical";
|
||||
if (score >= 5) return "high";
|
||||
if (score >= 3) return "medium";
|
||||
return "low";
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -325,16 +334,16 @@ function recordAnomalies(anomalies) {
|
||||
reject(err);
|
||||
return;
|
||||
}
|
||||
|
||||
|
||||
const stmt = db.prepare(`
|
||||
INSERT INTO anomalies (source, error_type, count, severity, score)
|
||||
VALUES (?, ?, ?, ?, ?)
|
||||
`);
|
||||
|
||||
anomalies.forEach(anomaly => {
|
||||
stmt.run(['logs', anomaly.type, anomaly.count, anomaly.severity, anomaly.score]);
|
||||
|
||||
anomalies.forEach((anomaly) => {
|
||||
stmt.run(["logs", anomaly.type, anomaly.count, anomaly.severity, anomaly.score]);
|
||||
});
|
||||
|
||||
|
||||
stmt.finalize((err) => {
|
||||
db.close();
|
||||
if (err) reject(err);
|
||||
@@ -350,14 +359,14 @@ function recordAnomalies(anomalies) {
|
||||
* @returns {string} Formatted report
|
||||
*/
|
||||
function generateReport(result) {
|
||||
let report = '=== Anomaly Detection Report ===\n';
|
||||
let report = "=== Anomaly Detection Report ===\n";
|
||||
report += `Timestamp: ${result.timestamp}\n\n`;
|
||||
report += `Total errors scanned: ${result.total_errors}\n`;
|
||||
report += `Significant anomalies: ${result.significant_count}\n\n`;
|
||||
|
||||
|
||||
if (result.anomalies.length > 0) {
|
||||
report += '⚠️ SIGNIFICANT ANOMALIES:\n';
|
||||
result.anomalies.forEach(anomaly => {
|
||||
report += "⚠️ SIGNIFICANT ANOMALIES:\n";
|
||||
result.anomalies.forEach((anomaly) => {
|
||||
report += ` Type: ${anomaly.type}\n`;
|
||||
report += ` Count: ${anomaly.count}\n`;
|
||||
report += ` Score: ${anomaly.score.toFixed(2)}\n`;
|
||||
@@ -365,28 +374,30 @@ function generateReport(result) {
|
||||
report += ` Recommendation: ${anomaly.recommendation}\n\n`;
|
||||
});
|
||||
} else {
|
||||
report += '✅ No significant anomalies detected\n';
|
||||
report += "✅ No significant anomalies detected\n";
|
||||
}
|
||||
|
||||
report += '\n=== End Anomaly Detection ===\n';
|
||||
|
||||
|
||||
report += "\n=== End Anomaly Detection ===\n";
|
||||
|
||||
return report;
|
||||
}
|
||||
|
||||
// CLI execution
|
||||
if (require.main === module) {
|
||||
initDB().then(async () => {
|
||||
const args = process.argv.slice(2);
|
||||
const jsonOutput = args.includes('--json') || args.includes('-j');
|
||||
|
||||
const result = await detectAnomalies();
|
||||
|
||||
if (jsonOutput) {
|
||||
console.log(JSON.stringify(result, null, 2));
|
||||
} else {
|
||||
console.log(generateReport(result));
|
||||
}
|
||||
}).catch(console.error);
|
||||
initDB()
|
||||
.then(async () => {
|
||||
const args = process.argv.slice(2);
|
||||
const jsonOutput = args.includes("--json") || args.includes("-j");
|
||||
|
||||
const result = await detectAnomalies();
|
||||
|
||||
if (jsonOutput) {
|
||||
console.log(JSON.stringify(result, null, 2));
|
||||
} else {
|
||||
console.log(generateReport(result));
|
||||
}
|
||||
})
|
||||
.catch(console.error);
|
||||
}
|
||||
|
||||
// Export for module usage
|
||||
@@ -396,5 +407,5 @@ module.exports = {
|
||||
scanLogFiles,
|
||||
get7DayRollingAverage,
|
||||
generateReport,
|
||||
initDB
|
||||
initDB,
|
||||
};
|
||||
|
||||
@@ -1,31 +1,44 @@
|
||||
#!/usr/bin/env node
|
||||
/**
|
||||
* Capability Mapper Module - Phase 1: Script-to-Skill Conversion
|
||||
*
|
||||
*
|
||||
* Maps goals to required skills and identifies gaps.
|
||||
* Outputs capability analysis for strategic planning.
|
||||
*
|
||||
*
|
||||
* @module capability-mapper
|
||||
*/
|
||||
|
||||
const fs = require('fs');
|
||||
const path = require('path');
|
||||
const sqlite3 = require('sqlite3').verbose();
|
||||
const fs = require("fs");
|
||||
const path = require("path");
|
||||
const sqlite3 = require("sqlite3").verbose();
|
||||
|
||||
// Configuration
|
||||
const WORKSPACE = process.env.WORKSPACE || path.join(process.env.HOME, '.openclaw/workspace');
|
||||
const SKILLS_DIR = path.join(WORKSPACE, 'skills');
|
||||
const CURIOSITY_DIR = path.join(WORKSPACE, '.curiosity');
|
||||
const CAPS_DB = path.join(CURIOSITY_DIR, 'capabilities.db');
|
||||
const WORKSPACE = process.env.WORKSPACE || path.join(process.env.HOME, ".openclaw/workspace");
|
||||
const SKILLS_DIR = path.join(WORKSPACE, "skills");
|
||||
const CURIOSITY_DIR = path.join(WORKSPACE, ".curiosity");
|
||||
const CAPS_DB = path.join(CURIOSITY_DIR, "capabilities.db");
|
||||
|
||||
// Goal → Skill mappings
|
||||
const GOAL_MAP = {
|
||||
'self-improvement': ['skill-creator', 'audit-triad-files', 'auto-patch', 'edit', 'write', 'exec'],
|
||||
'knowledge-growth': ['knowledge-ingest', 'knowledge-retrieval', 'auto-tag', 'relevance-rank', 'web_search', 'web_fetch'],
|
||||
'autonomy': ['triad-heartbeat', 'consensus-ledger', 'gap-detector', 'triad-deliberation-protocol'],
|
||||
'triad-sync': ['triad-sync-protocol', 'triad-unity-monitor', 'triad-signal-filter', 'message', 'exec'],
|
||||
'security': ['healthcheck', 'security-triage', 'openclaw-ghsa-maintainer', 'exec'],
|
||||
'deployment': ['openclaw-release-maintainer', 'openclaw-pr-maintainer', 'clawhub', 'npm']
|
||||
"self-improvement": ["skill-creator", "audit-triad-files", "auto-patch", "edit", "write", "exec"],
|
||||
"knowledge-growth": [
|
||||
"knowledge-ingest",
|
||||
"knowledge-retrieval",
|
||||
"auto-tag",
|
||||
"relevance-rank",
|
||||
"web_search",
|
||||
"web_fetch",
|
||||
],
|
||||
autonomy: ["triad-heartbeat", "consensus-ledger", "gap-detector", "triad-deliberation-protocol"],
|
||||
"triad-sync": [
|
||||
"triad-sync-protocol",
|
||||
"triad-unity-monitor",
|
||||
"triad-signal-filter",
|
||||
"message",
|
||||
"exec",
|
||||
],
|
||||
security: ["healthcheck", "security-triage", "openclaw-ghsa-maintainer", "exec"],
|
||||
deployment: ["openclaw-release-maintainer", "openclaw-pr-maintainer", "clawhub", "npm"],
|
||||
};
|
||||
|
||||
// Ensure directories exist
|
||||
@@ -43,8 +56,9 @@ function initDB() {
|
||||
reject(err);
|
||||
return;
|
||||
}
|
||||
|
||||
db.run(`
|
||||
|
||||
db.run(
|
||||
`
|
||||
CREATE TABLE IF NOT EXISTS capability_maps (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
timestamp TEXT DEFAULT CURRENT_TIMESTAMP,
|
||||
@@ -54,10 +68,12 @@ function initDB() {
|
||||
gaps TEXT,
|
||||
autonomy_score REAL DEFAULT 0
|
||||
)
|
||||
`, (err) => {
|
||||
if (err) reject(err);
|
||||
else resolve(db);
|
||||
});
|
||||
`,
|
||||
(err) => {
|
||||
if (err) reject(err);
|
||||
else resolve(db);
|
||||
},
|
||||
);
|
||||
});
|
||||
});
|
||||
}
|
||||
@@ -68,17 +84,18 @@ function initDB() {
|
||||
*/
|
||||
function getInstalledSkills() {
|
||||
const installed = [];
|
||||
|
||||
|
||||
try {
|
||||
const skillsDirs = fs.readdirSync(SKILLS_DIR, { withFileTypes: true })
|
||||
.filter(dirent => dirent.isDirectory())
|
||||
.map(dirent => dirent.name);
|
||||
|
||||
const skillsDirs = fs
|
||||
.readdirSync(SKILLS_DIR, { withFileTypes: true })
|
||||
.filter((dirent) => dirent.isDirectory())
|
||||
.map((dirent) => dirent.name);
|
||||
|
||||
installed.push(...skillsDirs);
|
||||
} catch (err) {
|
||||
console.error('Error reading skills directory:', err.message);
|
||||
console.error("Error reading skills directory:", err.message);
|
||||
}
|
||||
|
||||
|
||||
return installed.sort();
|
||||
}
|
||||
|
||||
@@ -99,31 +116,29 @@ function isInstalled(skill) {
|
||||
*/
|
||||
function mapCapability(goal) {
|
||||
const required = GOAL_MAP[goal];
|
||||
|
||||
|
||||
if (!required) {
|
||||
return {
|
||||
error: `Unknown goal: ${goal}`,
|
||||
valid_goals: Object.keys(GOAL_MAP)
|
||||
valid_goals: Object.keys(GOAL_MAP),
|
||||
};
|
||||
}
|
||||
|
||||
|
||||
const installed = [];
|
||||
const gaps = [];
|
||||
|
||||
required.forEach(skill => {
|
||||
|
||||
required.forEach((skill) => {
|
||||
if (isInstalled(skill)) {
|
||||
installed.push(skill);
|
||||
} else {
|
||||
gaps.push(skill);
|
||||
}
|
||||
});
|
||||
|
||||
|
||||
const installedCount = installed.length;
|
||||
const requiredCount = required.length;
|
||||
const autonomyScore = requiredCount > 0
|
||||
? (installedCount * 100 / requiredCount)
|
||||
: 0;
|
||||
|
||||
const autonomyScore = requiredCount > 0 ? (installedCount * 100) / requiredCount : 0;
|
||||
|
||||
return {
|
||||
goal,
|
||||
required,
|
||||
@@ -133,7 +148,7 @@ function mapCapability(goal) {
|
||||
installed_count: installedCount,
|
||||
gap_count: gaps.length,
|
||||
autonomy_score: autonomyScore,
|
||||
timestamp: new Date().toISOString()
|
||||
timestamp: new Date().toISOString(),
|
||||
};
|
||||
}
|
||||
|
||||
@@ -144,18 +159,16 @@ function mapCapability(goal) {
|
||||
function generateFullReport() {
|
||||
const goals = Object.keys(GOAL_MAP);
|
||||
const results = {};
|
||||
|
||||
goals.forEach(goal => {
|
||||
|
||||
goals.forEach((goal) => {
|
||||
results[goal] = mapCapability(goal);
|
||||
});
|
||||
|
||||
|
||||
// Calculate aggregate stats
|
||||
const totalRequired = goals.reduce((sum, goal) => sum + results[goal].required_count, 0);
|
||||
const totalInstalled = goals.reduce((sum, goal) => sum + results[goal].installed_count, 0);
|
||||
const overallAutonomy = totalRequired > 0
|
||||
? (totalInstalled * 100 / totalRequired)
|
||||
: 0;
|
||||
|
||||
const overallAutonomy = totalRequired > 0 ? (totalInstalled * 100) / totalRequired : 0;
|
||||
|
||||
return {
|
||||
timestamp: new Date().toISOString(),
|
||||
goals: results,
|
||||
@@ -164,8 +177,8 @@ function generateFullReport() {
|
||||
total_required: totalRequired,
|
||||
total_installed: totalInstalled,
|
||||
total_gaps: totalRequired - totalInstalled,
|
||||
overall_autonomy_score: overallAutonomy
|
||||
}
|
||||
overall_autonomy_score: overallAutonomy,
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
@@ -180,19 +193,23 @@ function recordCapability(result) {
|
||||
reject(err);
|
||||
return;
|
||||
}
|
||||
|
||||
const requiredStr = result.required.join(',');
|
||||
const installedStr = result.installed.join(',');
|
||||
const gapsStr = result.gaps.join(',');
|
||||
|
||||
db.run(`
|
||||
|
||||
const requiredStr = result.required.join(",");
|
||||
const installedStr = result.installed.join(",");
|
||||
const gapsStr = result.gaps.join(",");
|
||||
|
||||
db.run(
|
||||
`
|
||||
INSERT INTO capability_maps (goal, required_skills, installed_skills, gaps, autonomy_score)
|
||||
VALUES (?, ?, ?, ?, ?)
|
||||
`, [result.goal, requiredStr, installedStr, gapsStr, result.autonomy_score], (err) => {
|
||||
db.close();
|
||||
if (err) reject(err);
|
||||
else resolve();
|
||||
});
|
||||
`,
|
||||
[result.goal, requiredStr, installedStr, gapsStr, result.autonomy_score],
|
||||
(err) => {
|
||||
db.close();
|
||||
if (err) reject(err);
|
||||
else resolve();
|
||||
},
|
||||
);
|
||||
});
|
||||
});
|
||||
}
|
||||
@@ -214,18 +231,18 @@ async function recordAllCapabilities(report) {
|
||||
*/
|
||||
function identifyCriticalGaps(report) {
|
||||
const critical = [];
|
||||
|
||||
Object.values(report.goals).forEach(goalResult => {
|
||||
|
||||
Object.values(report.goals).forEach((goalResult) => {
|
||||
if (goalResult.autonomy_score < 50 && goalResult.gap_count > 0) {
|
||||
critical.push({
|
||||
goal: goalResult.goal,
|
||||
gaps: goalResult.gaps,
|
||||
autonomy_score: goalResult.autonomy_score,
|
||||
priority: goalResult.autonomy_score < 30 ? 'critical' : 'high'
|
||||
priority: goalResult.autonomy_score < 30 ? "critical" : "high",
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
|
||||
return critical;
|
||||
}
|
||||
|
||||
@@ -235,9 +252,9 @@ function identifyCriticalGaps(report) {
|
||||
* @returns {string} Formatted report
|
||||
*/
|
||||
function generateReport(report) {
|
||||
let output = '=== Capability Mapping Report ===\n';
|
||||
let output = "=== Capability Mapping Report ===\n";
|
||||
output += `Timestamp: ${report.timestamp}\n\n`;
|
||||
|
||||
|
||||
// Individual goals
|
||||
Object.entries(report.goals).forEach(([goal, result]) => {
|
||||
output += `━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\n`;
|
||||
@@ -247,24 +264,24 @@ function generateReport(report) {
|
||||
output += ` Installed: ${result.installed_count} skills\n`;
|
||||
output += ` Gaps: ${result.gap_count} skills\n`;
|
||||
output += ` Autonomy Score: ${result.autonomy_score.toFixed(1)}%\n\n`;
|
||||
|
||||
|
||||
if (result.installed.length > 0) {
|
||||
output += ` ✅ Installed:\n`;
|
||||
result.installed.forEach(skill => {
|
||||
result.installed.forEach((skill) => {
|
||||
output += ` ${skill}\n`;
|
||||
});
|
||||
output += '\n';
|
||||
output += "\n";
|
||||
}
|
||||
|
||||
|
||||
if (result.gaps.length > 0) {
|
||||
output += ` ❌ Gaps:\n`;
|
||||
result.gaps.forEach(skill => {
|
||||
result.gaps.forEach((skill) => {
|
||||
output += ` ${skill}\n`;
|
||||
});
|
||||
output += '\n';
|
||||
output += "\n";
|
||||
}
|
||||
});
|
||||
|
||||
|
||||
// Aggregate summary
|
||||
output += `━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\n`;
|
||||
output += `Aggregate Summary\n`;
|
||||
@@ -272,58 +289,60 @@ function generateReport(report) {
|
||||
output += ` Total goals mapped: ${report.aggregate.total_goals}\n`;
|
||||
output += ` Overall autonomy: ${report.aggregate.overall_autonomy_score.toFixed(1)}%\n`;
|
||||
output += ` Total gaps: ${report.aggregate.total_gaps}\n\n`;
|
||||
|
||||
|
||||
// Critical gaps
|
||||
const critical = identifyCriticalGaps(report);
|
||||
if (critical.length > 0) {
|
||||
output += `⚠️ CRITICAL CAPABILITY GAPS:\n`;
|
||||
critical.forEach(crit => {
|
||||
critical.forEach((crit) => {
|
||||
output += ` Goal: ${crit.goal}\n`;
|
||||
output += ` Autonomy: ${crit.autonomy_score.toFixed(1)}%\n`;
|
||||
output += ` Priority: ${crit.priority}\n`;
|
||||
output += ` Missing: ${crit.gaps.join(', ')}\n\n`;
|
||||
output += ` Missing: ${crit.gaps.join(", ")}\n\n`;
|
||||
});
|
||||
}
|
||||
|
||||
output += '\n=== End Capability Mapping ===\n';
|
||||
|
||||
|
||||
output += "\n=== End Capability Mapping ===\n";
|
||||
|
||||
return output;
|
||||
}
|
||||
|
||||
// CLI execution
|
||||
if (require.main === module) {
|
||||
initDB().then(async () => {
|
||||
const args = process.argv.slice(2);
|
||||
const jsonOutput = args.includes('--json') || args.includes('-j');
|
||||
const specificGoal = args.find(arg => !arg.startsWith('-'));
|
||||
|
||||
let report;
|
||||
|
||||
if (specificGoal) {
|
||||
const result = mapCapability(specificGoal);
|
||||
await recordCapability(result);
|
||||
report = {
|
||||
timestamp: new Date().toISOString(),
|
||||
goals: { [specificGoal]: result },
|
||||
aggregate: {
|
||||
total_goals: 1,
|
||||
total_required: result.required_count,
|
||||
total_installed: result.installed_count,
|
||||
total_gaps: result.gap_count,
|
||||
overall_autonomy_score: result.autonomy_score
|
||||
}
|
||||
};
|
||||
} else {
|
||||
report = generateFullReport();
|
||||
await recordAllCapabilities(report);
|
||||
}
|
||||
|
||||
if (jsonOutput) {
|
||||
console.log(JSON.stringify(report, null, 2));
|
||||
} else {
|
||||
console.log(generateReport(report));
|
||||
}
|
||||
}).catch(console.error);
|
||||
initDB()
|
||||
.then(async () => {
|
||||
const args = process.argv.slice(2);
|
||||
const jsonOutput = args.includes("--json") || args.includes("-j");
|
||||
const specificGoal = args.find((arg) => !arg.startsWith("-"));
|
||||
|
||||
let report;
|
||||
|
||||
if (specificGoal) {
|
||||
const result = mapCapability(specificGoal);
|
||||
await recordCapability(result);
|
||||
report = {
|
||||
timestamp: new Date().toISOString(),
|
||||
goals: { [specificGoal]: result },
|
||||
aggregate: {
|
||||
total_goals: 1,
|
||||
total_required: result.required_count,
|
||||
total_installed: result.installed_count,
|
||||
total_gaps: result.gap_count,
|
||||
overall_autonomy_score: result.autonomy_score,
|
||||
},
|
||||
};
|
||||
} else {
|
||||
report = generateFullReport();
|
||||
await recordAllCapabilities(report);
|
||||
}
|
||||
|
||||
if (jsonOutput) {
|
||||
console.log(JSON.stringify(report, null, 2));
|
||||
} else {
|
||||
console.log(generateReport(report));
|
||||
}
|
||||
})
|
||||
.catch(console.error);
|
||||
}
|
||||
|
||||
// Export for module usage
|
||||
@@ -335,5 +354,5 @@ module.exports = {
|
||||
identifyCriticalGaps,
|
||||
generateReport,
|
||||
initDB,
|
||||
GOAL_MAP
|
||||
GOAL_MAP,
|
||||
};
|
||||
|
||||
@@ -1,33 +1,33 @@
|
||||
#!/usr/bin/env node
|
||||
/**
|
||||
* Deliberation Trigger Module - Phase 4: Deliberation Trigger Enhancement
|
||||
*
|
||||
*
|
||||
* Creates proposals from detected gaps, anomalies, and opportunities.
|
||||
* Implements priority scoring, deduplication, and quorum awareness.
|
||||
*
|
||||
*
|
||||
* @module deliberation-trigger
|
||||
*/
|
||||
|
||||
const fs = require('fs');
|
||||
const path = require('path');
|
||||
const sqlite3 = require('sqlite3').verbose();
|
||||
const { execSync } = require('child_process');
|
||||
const fs = require("fs");
|
||||
const path = require("path");
|
||||
const sqlite3 = require("sqlite3").verbose();
|
||||
const { execSync } = require("child_process");
|
||||
|
||||
// Configuration
|
||||
const WORKSPACE = process.env.WORKSPACE || path.join(process.env.HOME, '.openclaw/workspace');
|
||||
const CURIOSITY_DIR = path.join(WORKSPACE, '.curiosity');
|
||||
const CONSENSUS_DB = path.join(CURIOSITY_DIR, 'consensus_ledger.db');
|
||||
const MEMORY_DIR = path.join(WORKSPACE, 'memory');
|
||||
const IDENTITY_FILE = path.join(WORKSPACE, 'IDENTITY.md');
|
||||
const WORKSPACE = process.env.WORKSPACE || path.join(process.env.HOME, ".openclaw/workspace");
|
||||
const CURIOSITY_DIR = path.join(WORKSPACE, ".curiosity");
|
||||
const CONSENSUS_DB = path.join(CURIOSITY_DIR, "consensus_ledger.db");
|
||||
const MEMORY_DIR = path.join(WORKSPACE, "memory");
|
||||
const IDENTITY_FILE = path.join(WORKSPACE, "IDENTITY.md");
|
||||
|
||||
// Priority matrix for scoring
|
||||
const PRIORITY_MATRIX = {
|
||||
security: { base: 10, multiplier: 2.0 },
|
||||
self-improvement: { base: 8, multiplier: 1.5 },
|
||||
triad-sync: { base: 6, multiplier: 1.3 },
|
||||
"self-improvement": { base: 8, multiplier: 1.5 },
|
||||
"triad-sync": { base: 6, multiplier: 1.3 },
|
||||
knowledge: { base: 4, multiplier: 1.0 },
|
||||
triad: { base: 6, multiplier: 1.3 },
|
||||
optional: { base: 2, multiplier: 0.5 }
|
||||
optional: { base: 2, multiplier: 0.5 },
|
||||
};
|
||||
|
||||
// Ensure directories exist
|
||||
@@ -48,8 +48,9 @@ function initDB() {
|
||||
reject(err);
|
||||
return;
|
||||
}
|
||||
|
||||
db.run(`
|
||||
|
||||
db.run(
|
||||
`
|
||||
CREATE TABLE IF NOT EXISTS consensus_votes (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
timestamp TEXT DEFAULT CURRENT_TIMESTAMP,
|
||||
@@ -64,10 +65,12 @@ function initDB() {
|
||||
result TEXT DEFAULT 'pending',
|
||||
processed INTEGER DEFAULT 0
|
||||
)
|
||||
`, (err) => {
|
||||
if (err) reject(err);
|
||||
else resolve(db);
|
||||
});
|
||||
`,
|
||||
(err) => {
|
||||
if (err) reject(err);
|
||||
else resolve(db);
|
||||
},
|
||||
);
|
||||
});
|
||||
});
|
||||
}
|
||||
@@ -79,11 +82,11 @@ function initDB() {
|
||||
function isQuorumSpeaker() {
|
||||
try {
|
||||
if (fs.existsSync(IDENTITY_FILE)) {
|
||||
const content = fs.readFileSync(IDENTITY_FILE, 'utf8');
|
||||
const content = fs.readFileSync(IDENTITY_FILE, "utf8");
|
||||
return /Role:\s*Authority/i.test(content);
|
||||
}
|
||||
} catch (err) {
|
||||
console.error('Error reading IDENTITY.md:', err.message);
|
||||
console.error("Error reading IDENTITY.md:", err.message);
|
||||
}
|
||||
return false;
|
||||
}
|
||||
@@ -101,7 +104,7 @@ function proposalExists(title, source) {
|
||||
resolve(false);
|
||||
return;
|
||||
}
|
||||
|
||||
|
||||
const query = `
|
||||
SELECT COUNT(*) as count
|
||||
FROM consensus_votes
|
||||
@@ -110,7 +113,7 @@ function proposalExists(title, source) {
|
||||
AND status != 'closed'
|
||||
AND timestamp >= datetime('now', '-24 hours')
|
||||
`;
|
||||
|
||||
|
||||
db.get(query, [title, source], (err, row) => {
|
||||
db.close();
|
||||
if (err) reject(err);
|
||||
@@ -128,21 +131,21 @@ function proposalExists(title, source) {
|
||||
function calculatePriority(item) {
|
||||
const category = item.category || categorizeItem(item);
|
||||
const config = PRIORITY_MATRIX[category] || PRIORITY_MATRIX.optional;
|
||||
|
||||
|
||||
const base = config.base;
|
||||
const multiplier = config.multiplier;
|
||||
const urgency = item.blocksLiberation || item.severity === 'critical' ? 2.0 : 1.0;
|
||||
|
||||
const urgency = item.blocksLiberation || item.severity === "critical" ? 2.0 : 1.0;
|
||||
|
||||
const score = Math.min(10, base * multiplier * urgency);
|
||||
const priority = score >= 8 ? 'critical' : score >= 6 ? 'high' : score >= 4 ? 'medium' : 'low';
|
||||
|
||||
const priority = score >= 8 ? "critical" : score >= 6 ? "high" : score >= 4 ? "medium" : "low";
|
||||
|
||||
return {
|
||||
category,
|
||||
base,
|
||||
multiplier,
|
||||
urgency,
|
||||
score,
|
||||
priority
|
||||
priority,
|
||||
};
|
||||
}
|
||||
|
||||
@@ -152,16 +155,17 @@ function calculatePriority(item) {
|
||||
* @returns {string} Category
|
||||
*/
|
||||
function categorizeItem(item) {
|
||||
const title = (item.title || '').toLowerCase();
|
||||
const type = item.type || '';
|
||||
|
||||
if (title.includes('security') || title.includes('cve') || type === 'security') return 'security';
|
||||
if (title.includes('skill-creator') || title.includes('self-improvement')) return 'self-improvement';
|
||||
if (title.includes('triad') || title.includes('sync')) return 'triad';
|
||||
if (title.includes('knowledge')) return 'knowledge';
|
||||
if (item.gap && item.gap.includes('skill')) return 'self-improvement';
|
||||
|
||||
return 'optional';
|
||||
const title = (item.title || "").toLowerCase();
|
||||
const type = item.type || "";
|
||||
|
||||
if (title.includes("security") || title.includes("cve") || type === "security") return "security";
|
||||
if (title.includes("skill-creator") || title.includes("self-improvement"))
|
||||
return "self-improvement";
|
||||
if (title.includes("triad") || title.includes("sync")) return "triad";
|
||||
if (title.includes("knowledge")) return "knowledge";
|
||||
if (item.gap && item.gap.includes("skill")) return "self-improvement";
|
||||
|
||||
return "optional";
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -171,24 +175,24 @@ function categorizeItem(item) {
|
||||
*/
|
||||
async function createProposal(proposal) {
|
||||
const { title, body, source, category, item } = proposal;
|
||||
|
||||
|
||||
// Check for duplicates
|
||||
const exists = await proposalExists(title, source);
|
||||
if (exists) {
|
||||
return {
|
||||
skipped: true,
|
||||
reason: 'Duplicate proposal exists within 24h window',
|
||||
title
|
||||
reason: "Duplicate proposal exists within 24h window",
|
||||
title,
|
||||
};
|
||||
}
|
||||
|
||||
|
||||
// Calculate priority
|
||||
const priorityResult = calculatePriority(item || { category, title });
|
||||
|
||||
|
||||
// Escape quotes for SQL
|
||||
const safeTitle = title.replace(/'/g, "''");
|
||||
const safeBody = body.replace(/'/g, "''");
|
||||
|
||||
|
||||
// Insert into consensus ledger
|
||||
return new Promise((resolve, reject) => {
|
||||
const db = new sqlite3.Database(CONSENSUS_DB, (err) => {
|
||||
@@ -196,33 +200,37 @@ async function createProposal(proposal) {
|
||||
reject(err);
|
||||
return;
|
||||
}
|
||||
|
||||
db.run(`
|
||||
|
||||
db.run(
|
||||
`
|
||||
INSERT INTO consensus_votes
|
||||
(proposal_title, proposal_body, priority, priority_score, source, category)
|
||||
VALUES (?, ?, ?, ?, ?, ?)
|
||||
`, [safeTitle, safeBody, priorityResult.priority, priorityResult.score, source, category], function(err) {
|
||||
if (err) {
|
||||
`,
|
||||
[safeTitle, safeBody, priorityResult.priority, priorityResult.score, source, category],
|
||||
function (err) {
|
||||
if (err) {
|
||||
db.close();
|
||||
reject(err);
|
||||
return;
|
||||
}
|
||||
|
||||
const created = {
|
||||
id: this.lastID,
|
||||
title,
|
||||
body,
|
||||
priority: priorityResult.priority,
|
||||
priority_score: priorityResult.score,
|
||||
source,
|
||||
category,
|
||||
status: "pending",
|
||||
timestamp: new Date().toISOString(),
|
||||
};
|
||||
|
||||
db.close();
|
||||
reject(err);
|
||||
return;
|
||||
}
|
||||
|
||||
const created = {
|
||||
id: this.lastID,
|
||||
title,
|
||||
body,
|
||||
priority: priorityResult.priority,
|
||||
priority_score: priorityResult.score,
|
||||
source,
|
||||
category,
|
||||
status: 'pending',
|
||||
timestamp: new Date().toISOString()
|
||||
};
|
||||
|
||||
db.close();
|
||||
resolve(created);
|
||||
});
|
||||
resolve(created);
|
||||
},
|
||||
);
|
||||
});
|
||||
});
|
||||
}
|
||||
@@ -232,9 +240,9 @@ async function createProposal(proposal) {
|
||||
* @param {Object} proposal - Proposal data
|
||||
*/
|
||||
function logToMemory(proposal) {
|
||||
const date = new Date().toISOString().split('T')[0];
|
||||
const date = new Date().toISOString().split("T")[0];
|
||||
const memoryFile = path.join(MEMORY_DIR, `curiosity-${date}.md`);
|
||||
|
||||
|
||||
const entry = `
|
||||
## Deliberation Proposal - ${proposal.timestamp}
|
||||
|
||||
@@ -249,7 +257,7 @@ function logToMemory(proposal) {
|
||||
|
||||
---
|
||||
`;
|
||||
|
||||
|
||||
fs.appendFileSync(memoryFile, entry);
|
||||
}
|
||||
|
||||
@@ -259,15 +267,15 @@ function logToMemory(proposal) {
|
||||
*/
|
||||
function postToDiscord(proposal) {
|
||||
if (!isQuorumSpeaker()) {
|
||||
console.log('ℹ️ Not quorum speaker, logging to memory only');
|
||||
console.log("ℹ️ Not quorum speaker, logging to memory only");
|
||||
return;
|
||||
}
|
||||
|
||||
if (proposal.priority !== 'high' && proposal.priority !== 'critical') {
|
||||
console.log('ℹ️ Priority not high enough for Discord post');
|
||||
|
||||
if (proposal.priority !== "high" && proposal.priority !== "critical") {
|
||||
console.log("ℹ️ Priority not high enough for Discord post");
|
||||
return;
|
||||
}
|
||||
|
||||
|
||||
const message = `**🦞 Proposal:** ${proposal.title}
|
||||
|
||||
**Priority:** ${proposal.priority} (score: ${proposal.priority_score.toFixed(2)})
|
||||
@@ -277,12 +285,15 @@ function postToDiscord(proposal) {
|
||||
${proposal.body}
|
||||
|
||||
*Awaiting quorum vote (2-of-3)*`;
|
||||
|
||||
|
||||
try {
|
||||
execSync(`openclaw message send --channel discord --message "${message.replace(/"/g, '\\"')}" 2>/dev/null || true`, { encoding: 'utf8' });
|
||||
console.log('📢 Posted to Discord');
|
||||
execSync(
|
||||
`openclaw message send --channel discord --message "${message.replace(/"/g, '\\"')}" 2>/dev/null || true`,
|
||||
{ encoding: "utf8" },
|
||||
);
|
||||
console.log("📢 Posted to Discord");
|
||||
} catch (err) {
|
||||
console.error(' Discord post failed:', err.message);
|
||||
console.error(" Discord post failed:", err.message);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -293,27 +304,27 @@ ${proposal.body}
|
||||
*/
|
||||
async function processGaps(gaps) {
|
||||
const proposals = [];
|
||||
|
||||
|
||||
if (!gaps || !gaps.critical) {
|
||||
return proposals;
|
||||
}
|
||||
|
||||
|
||||
for (const gap of gaps.critical) {
|
||||
const proposal = await createProposal({
|
||||
title: `Install ${gap.skill} to close capability gap`,
|
||||
body: `Gap detected: ${gap.skill} not installed. ${gap.impact}. Recommendation: ${gap.recommendation}.`,
|
||||
source: 'gap-detector',
|
||||
category: 'self-improvement',
|
||||
item: { ...gap, blocksLiberation: true }
|
||||
source: "gap-detector",
|
||||
category: "self-improvement",
|
||||
item: { ...gap, blocksLiberation: true },
|
||||
});
|
||||
|
||||
|
||||
if (!proposal.skipped) {
|
||||
logToMemory(proposal);
|
||||
postToDiscord(proposal);
|
||||
proposals.push(proposal);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
return proposals;
|
||||
}
|
||||
|
||||
@@ -324,27 +335,27 @@ async function processGaps(gaps) {
|
||||
*/
|
||||
async function processAnomalies(anomalyResult) {
|
||||
const proposals = [];
|
||||
|
||||
|
||||
if (!anomalyResult || !anomalyResult.anomalies) {
|
||||
return proposals;
|
||||
}
|
||||
|
||||
|
||||
for (const anomaly of anomalyResult.anomalies) {
|
||||
const proposal = await createProposal({
|
||||
title: `Repair ${anomaly.type} anomaly`,
|
||||
body: `Anomaly detected: ${anomaly.count} occurrences of ${anomaly.type} errors. Score: ${anomaly.score.toFixed(2)}. ${anomaly.recommendation}.`,
|
||||
source: 'anomaly-detector',
|
||||
source: "anomaly-detector",
|
||||
category: categorizeItem({ title: anomaly.type }),
|
||||
item: { severity: anomaly.severity, type: anomaly.type }
|
||||
item: { severity: anomaly.severity, type: anomaly.type },
|
||||
});
|
||||
|
||||
|
||||
if (!proposal.skipped) {
|
||||
logToMemory(proposal);
|
||||
postToDiscord(proposal);
|
||||
proposals.push(proposal);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
return proposals;
|
||||
}
|
||||
|
||||
@@ -355,37 +366,37 @@ async function processAnomalies(anomalyResult) {
|
||||
*/
|
||||
async function processOpportunities(oppResult) {
|
||||
const proposals = [];
|
||||
|
||||
|
||||
if (!oppResult || !oppResult.opportunities) {
|
||||
return proposals;
|
||||
}
|
||||
|
||||
|
||||
for (const opp of oppResult.opportunities) {
|
||||
if (opp.priority === 'high' || opp.priority === 'critical') {
|
||||
if (opp.priority === "high" || opp.priority === "critical") {
|
||||
let title, body, category;
|
||||
|
||||
if (opp.type === 'release') {
|
||||
|
||||
if (opp.type === "release") {
|
||||
title = `Rebase on ${opp.title}`;
|
||||
body = `New release detected: ${opp.title}. Recommend rebasing heretek/main to incorporate changes while preserving liberation.`;
|
||||
category = 'triad';
|
||||
} else if (opp.type === 'security') {
|
||||
category = "triad";
|
||||
} else if (opp.type === "security") {
|
||||
title = `Address ${opp.title}`;
|
||||
body = `Security advisory: ${opp.title}. Requires immediate triage and remediation.`;
|
||||
category = 'security';
|
||||
category = "security";
|
||||
} else {
|
||||
title = `Evaluate ${opp.title}`;
|
||||
body = `Opportunity detected: ${opp.title}. Source: ${opp.source}. Evaluate for implementation.`;
|
||||
category = 'optional';
|
||||
category = "optional";
|
||||
}
|
||||
|
||||
|
||||
const proposal = await createProposal({
|
||||
title,
|
||||
body,
|
||||
source: 'opportunity-scanner',
|
||||
source: "opportunity-scanner",
|
||||
category,
|
||||
item: { ...opp, blocksLiberation: opp.priority === 'critical' }
|
||||
item: { ...opp, blocksLiberation: opp.priority === "critical" },
|
||||
});
|
||||
|
||||
|
||||
if (!proposal.skipped) {
|
||||
logToMemory(proposal);
|
||||
postToDiscord(proposal);
|
||||
@@ -393,7 +404,7 @@ async function processOpportunities(oppResult) {
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
return proposals;
|
||||
}
|
||||
|
||||
@@ -404,21 +415,25 @@ async function processOpportunities(oppResult) {
|
||||
*/
|
||||
async function processCapabilityGaps(capReport) {
|
||||
const proposals = [];
|
||||
|
||||
|
||||
if (!capReport || !capReport.goals) {
|
||||
return proposals;
|
||||
}
|
||||
|
||||
|
||||
for (const [goal, result] of Object.entries(capReport.goals)) {
|
||||
if (result.autonomy_score < 50 && result.gaps.length > 0) {
|
||||
const proposal = await createProposal({
|
||||
title: `Close capability gaps for ${goal}`,
|
||||
body: `Capability mapping identified ${result.gap_count} gaps for goal '${goal}': ${result.gaps.join(', ')}. Install missing skills to achieve ${result.autonomy_score.toFixed(1)}% → 100% autonomy.`,
|
||||
source: 'capability-mapper',
|
||||
category: goal.includes('triad') ? 'triad' : goal.includes('security') ? 'security' : 'knowledge',
|
||||
item: { autonomy_score: result.autonomy_score, gap_count: result.gap_count }
|
||||
body: `Capability mapping identified ${result.gap_count} gaps for goal '${goal}': ${result.gaps.join(", ")}. Install missing skills to achieve ${result.autonomy_score.toFixed(1)}% → 100% autonomy.`,
|
||||
source: "capability-mapper",
|
||||
category: goal.includes("triad")
|
||||
? "triad"
|
||||
: goal.includes("security")
|
||||
? "security"
|
||||
: "knowledge",
|
||||
item: { autonomy_score: result.autonomy_score, gap_count: result.gap_count },
|
||||
});
|
||||
|
||||
|
||||
if (!proposal.skipped) {
|
||||
logToMemory(proposal);
|
||||
postToDiscord(proposal);
|
||||
@@ -426,7 +441,7 @@ async function processCapabilityGaps(capReport) {
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
return proposals;
|
||||
}
|
||||
|
||||
@@ -437,55 +452,55 @@ async function processCapabilityGaps(capReport) {
|
||||
*/
|
||||
async function runAutoTrigger(inputs = {}) {
|
||||
const { gaps, anomalies, opportunities, capabilities } = inputs;
|
||||
|
||||
console.log('=== Deliberation Auto-Trigger ===');
|
||||
|
||||
console.log("=== Deliberation Auto-Trigger ===");
|
||||
console.log(`Timestamp: ${new Date().toISOString()}`);
|
||||
console.log(`Quorum Speaker: ${isQuorumSpeaker() ? 'Yes' : 'No'}`);
|
||||
console.log('');
|
||||
|
||||
console.log(`Quorum Speaker: ${isQuorumSpeaker() ? "Yes" : "No"}`);
|
||||
console.log("");
|
||||
|
||||
const allProposals = [];
|
||||
|
||||
|
||||
if (gaps) {
|
||||
console.log('Processing gap detection results...');
|
||||
console.log("Processing gap detection results...");
|
||||
const gapProposals = await processGaps(gaps);
|
||||
allProposals.push(...gapProposals);
|
||||
console.log(` Created ${gapProposals.length} proposals from gaps`);
|
||||
}
|
||||
|
||||
|
||||
if (anomalies) {
|
||||
console.log('Processing anomaly detection results...');
|
||||
console.log("Processing anomaly detection results...");
|
||||
const anomalyProposals = await processAnomalies(anomalies);
|
||||
allProposals.push(...anomalyProposals);
|
||||
console.log(` Created ${anomalyProposals.length} proposals from anomalies`);
|
||||
}
|
||||
|
||||
|
||||
if (opportunities) {
|
||||
console.log('Processing opportunity scanning results...');
|
||||
console.log("Processing opportunity scanning results...");
|
||||
const oppProposals = await processOpportunities(opportunities);
|
||||
allProposals.push(...oppProposals);
|
||||
console.log(` Created ${oppProposals.length} proposals from opportunities`);
|
||||
}
|
||||
|
||||
|
||||
if (capabilities) {
|
||||
console.log('Processing capability mapping results...');
|
||||
console.log("Processing capability mapping results...");
|
||||
const capProposals = await processCapabilityGaps(capabilities);
|
||||
allProposals.push(...capProposals);
|
||||
console.log(` Created ${capProposals.length} proposals from capability gaps`);
|
||||
}
|
||||
|
||||
console.log('');
|
||||
|
||||
console.log("");
|
||||
console.log(`=== End Auto-Trigger ===`);
|
||||
console.log(`Total proposals created: ${allProposals.length}`);
|
||||
|
||||
|
||||
// Count pending proposals
|
||||
const pendingCount = await getPendingProposalCount();
|
||||
console.log(`Pending proposals in ledger: ${pendingCount}`);
|
||||
|
||||
|
||||
return {
|
||||
timestamp: new Date().toISOString(),
|
||||
proposals_created: allProposals.length,
|
||||
pending_count: pendingCount,
|
||||
proposals: allProposals
|
||||
proposals: allProposals,
|
||||
};
|
||||
}
|
||||
|
||||
@@ -500,33 +515,39 @@ function getPendingProposalCount() {
|
||||
resolve(0);
|
||||
return;
|
||||
}
|
||||
|
||||
db.get('SELECT COUNT(*) as count FROM consensus_votes WHERE status = ?', ['pending'], (err, row) => {
|
||||
db.close();
|
||||
if (err) reject(err);
|
||||
else resolve(row?.count || 0);
|
||||
});
|
||||
|
||||
db.get(
|
||||
"SELECT COUNT(*) as count FROM consensus_votes WHERE status = ?",
|
||||
["pending"],
|
||||
(err, row) => {
|
||||
db.close();
|
||||
if (err) reject(err);
|
||||
else resolve(row?.count || 0);
|
||||
},
|
||||
);
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
// CLI execution
|
||||
if (require.main === module) {
|
||||
initDB().then(async () => {
|
||||
const args = process.argv.slice(2);
|
||||
const jsonOutput = args.includes('--json') || args.includes('-j');
|
||||
|
||||
// In standalone mode, run with mock empty inputs
|
||||
// In production, inputs would come from other modules
|
||||
const result = await runAutoTrigger({});
|
||||
|
||||
if (jsonOutput) {
|
||||
console.log(JSON.stringify(result, null, 2));
|
||||
} else {
|
||||
console.log('');
|
||||
console.log('Deliberation auto-trigger complete.');
|
||||
}
|
||||
}).catch(console.error);
|
||||
initDB()
|
||||
.then(async () => {
|
||||
const args = process.argv.slice(2);
|
||||
const jsonOutput = args.includes("--json") || args.includes("-j");
|
||||
|
||||
// In standalone mode, run with mock empty inputs
|
||||
// In production, inputs would come from other modules
|
||||
const result = await runAutoTrigger({});
|
||||
|
||||
if (jsonOutput) {
|
||||
console.log(JSON.stringify(result, null, 2));
|
||||
} else {
|
||||
console.log("");
|
||||
console.log("Deliberation auto-trigger complete.");
|
||||
}
|
||||
})
|
||||
.catch(console.error);
|
||||
}
|
||||
|
||||
// Export for module usage
|
||||
@@ -540,5 +561,5 @@ module.exports = {
|
||||
isQuorumSpeaker,
|
||||
calculatePriority,
|
||||
initDB,
|
||||
PRIORITY_MATRIX
|
||||
PRIORITY_MATRIX,
|
||||
};
|
||||
|
||||
@@ -1,21 +1,21 @@
|
||||
#!/usr/bin/env node
|
||||
/**
|
||||
* Gap Detector Module - Phase 1: Script-to-Skill Conversion
|
||||
*
|
||||
*
|
||||
* Compares installed skills vs available skills from multiple sources.
|
||||
* Outputs gaps that would enable new capabilities.
|
||||
*
|
||||
*
|
||||
* @module gap-detector
|
||||
*/
|
||||
|
||||
const fs = require('fs');
|
||||
const path = require('path');
|
||||
const { execSync } = require('child_process');
|
||||
const fs = require("fs");
|
||||
const path = require("path");
|
||||
const { execSync } = require("child_process");
|
||||
|
||||
// Configuration
|
||||
const WORKSPACE = process.env.WORKSPACE || path.join(process.env.HOME, '.openclaw/workspace');
|
||||
const SKILLS_DIR = path.join(WORKSPACE, 'skills');
|
||||
const CURIOSITY_DIR = path.join(WORKSPACE, '.curiosity');
|
||||
const WORKSPACE = process.env.WORKSPACE || path.join(process.env.HOME, ".openclaw/workspace");
|
||||
const SKILLS_DIR = path.join(WORKSPACE, "skills");
|
||||
const CURIOSITY_DIR = path.join(WORKSPACE, ".curiosity");
|
||||
|
||||
// Ensure directories exist
|
||||
if (!fs.existsSync(CURIOSITY_DIR)) {
|
||||
@@ -28,17 +28,18 @@ if (!fs.existsSync(CURIOSITY_DIR)) {
|
||||
*/
|
||||
function getInstalledSkills() {
|
||||
const installed = [];
|
||||
|
||||
|
||||
try {
|
||||
const skillsDirs = fs.readdirSync(SKILLS_DIR, { withFileTypes: true })
|
||||
.filter(dirent => dirent.isDirectory())
|
||||
.map(dirent => dirent.name);
|
||||
|
||||
const skillsDirs = fs
|
||||
.readdirSync(SKILLS_DIR, { withFileTypes: true })
|
||||
.filter((dirent) => dirent.isDirectory())
|
||||
.map((dirent) => dirent.name);
|
||||
|
||||
installed.push(...skillsDirs);
|
||||
} catch (err) {
|
||||
console.error('Error reading skills directory:', err.message);
|
||||
console.error("Error reading skills directory:", err.message);
|
||||
}
|
||||
|
||||
|
||||
return installed.sort();
|
||||
}
|
||||
|
||||
@@ -48,11 +49,11 @@ function getInstalledSkills() {
|
||||
*/
|
||||
function getAvailableSkillsFromClawHub() {
|
||||
try {
|
||||
const output = execSync('clawhub search 2>/dev/null || true', { encoding: 'utf8' });
|
||||
const lines = output.trim().split('\n').slice(1); // Skip header
|
||||
return lines.map(line => line.split(/\s+/)[0]).filter(Boolean);
|
||||
const output = execSync("clawhub search 2>/dev/null || true", { encoding: "utf8" });
|
||||
const lines = output.trim().split("\n").slice(1); // Skip header
|
||||
return lines.map((line) => line.split(/\s+/)[0]).filter(Boolean);
|
||||
} catch (err) {
|
||||
console.error('ClawHub CLI not available:', err.message);
|
||||
console.error("ClawHub CLI not available:", err.message);
|
||||
return [];
|
||||
}
|
||||
}
|
||||
@@ -62,16 +63,16 @@ function getAvailableSkillsFromClawHub() {
|
||||
* @returns {string[]} Array of available skill names
|
||||
*/
|
||||
function getAvailableSkillsFromCache() {
|
||||
const cacheFile = path.join(CURIOSITY_DIR, 'available_skills.txt');
|
||||
|
||||
const cacheFile = path.join(CURIOSITY_DIR, "available_skills.txt");
|
||||
|
||||
try {
|
||||
if (fs.existsSync(cacheFile)) {
|
||||
return fs.readFileSync(cacheFile, 'utf8').trim().split('\n').filter(Boolean);
|
||||
return fs.readFileSync(cacheFile, "utf8").trim().split("\n").filter(Boolean);
|
||||
}
|
||||
} catch (err) {
|
||||
console.error('Error reading cache file:', err.message);
|
||||
console.error("Error reading cache file:", err.message);
|
||||
}
|
||||
|
||||
|
||||
return [];
|
||||
}
|
||||
|
||||
@@ -82,7 +83,7 @@ function getAvailableSkillsFromCache() {
|
||||
function getAvailableSkills() {
|
||||
const clawhub = getAvailableSkillsFromClawHub();
|
||||
const cached = getAvailableSkillsFromCache();
|
||||
|
||||
|
||||
// Merge and dedupe
|
||||
const all = new Set([...clawhub, ...cached]);
|
||||
return Array.from(all).sort();
|
||||
@@ -94,14 +95,14 @@ function getAvailableSkills() {
|
||||
*/
|
||||
function getCriticalSkills() {
|
||||
return [
|
||||
'skill-creator',
|
||||
'knowledge-ingest',
|
||||
'knowledge-retrieval',
|
||||
'triad-deliberation-protocol',
|
||||
'triad-sync-protocol',
|
||||
'auto-patch',
|
||||
'gap-detector',
|
||||
'auto-deliberation-trigger'
|
||||
"skill-creator",
|
||||
"knowledge-ingest",
|
||||
"knowledge-retrieval",
|
||||
"triad-deliberation-protocol",
|
||||
"triad-sync-protocol",
|
||||
"auto-patch",
|
||||
"gap-detector",
|
||||
"auto-deliberation-trigger",
|
||||
];
|
||||
}
|
||||
|
||||
@@ -113,43 +114,43 @@ function getCriticalSkills() {
|
||||
*/
|
||||
function detectGaps(options = {}) {
|
||||
const { criticalOnly = false } = options;
|
||||
|
||||
|
||||
const installed = getInstalledSkills();
|
||||
const available = getAvailableSkills();
|
||||
const critical = getCriticalSkills();
|
||||
|
||||
|
||||
const gaps = {
|
||||
critical: [],
|
||||
optional: [],
|
||||
timestamp: new Date().toISOString(),
|
||||
installed_count: installed.length,
|
||||
available_count: available.length
|
||||
available_count: available.length,
|
||||
};
|
||||
|
||||
|
||||
// Check critical skills
|
||||
critical.forEach(skill => {
|
||||
critical.forEach((skill) => {
|
||||
if (!installed.includes(skill)) {
|
||||
gaps.critical.push({
|
||||
skill,
|
||||
impact: getSkillImpact(skill),
|
||||
recommendation: `clawhub install ${skill}`
|
||||
recommendation: `clawhub install ${skill}`,
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
|
||||
// Check all available skills (if not critical-only mode)
|
||||
if (!criticalOnly) {
|
||||
available.forEach(skill => {
|
||||
available.forEach((skill) => {
|
||||
if (!installed.includes(skill)) {
|
||||
gaps.optional.push({
|
||||
skill,
|
||||
category: categorizeSkill(skill),
|
||||
relevance: calculateRelevance(skill)
|
||||
relevance: calculateRelevance(skill),
|
||||
});
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
|
||||
return gaps;
|
||||
}
|
||||
|
||||
@@ -160,17 +161,17 @@ function detectGaps(options = {}) {
|
||||
*/
|
||||
function getSkillImpact(skill) {
|
||||
const impacts = {
|
||||
'skill-creator': 'Self-improvement loop disabled',
|
||||
'knowledge-ingest': 'Knowledge growth disabled',
|
||||
'knowledge-retrieval': 'Knowledge retrieval disabled',
|
||||
'triad-deliberation-protocol': 'Consensus deliberation disabled',
|
||||
'triad-sync-protocol': 'Triad sync infrastructure missing',
|
||||
'auto-patch': 'Auto-remediation disabled',
|
||||
'gap-detector': 'Self-awareness disabled',
|
||||
'auto-deliberation-trigger': 'Proactive deliberation disabled'
|
||||
"skill-creator": "Self-improvement loop disabled",
|
||||
"knowledge-ingest": "Knowledge growth disabled",
|
||||
"knowledge-retrieval": "Knowledge retrieval disabled",
|
||||
"triad-deliberation-protocol": "Consensus deliberation disabled",
|
||||
"triad-sync-protocol": "Triad sync infrastructure missing",
|
||||
"auto-patch": "Auto-remediation disabled",
|
||||
"gap-detector": "Self-awareness disabled",
|
||||
"auto-deliberation-trigger": "Proactive deliberation disabled",
|
||||
};
|
||||
|
||||
return impacts[skill] || 'Capability gap';
|
||||
|
||||
return impacts[skill] || "Capability gap";
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -179,11 +180,11 @@ function getSkillImpact(skill) {
|
||||
* @returns {string} Category
|
||||
*/
|
||||
function categorizeSkill(skill) {
|
||||
if (skill.includes('triad')) return 'triad';
|
||||
if (skill.includes('knowledge')) return 'knowledge';
|
||||
if (skill.includes('security')) return 'security';
|
||||
if (skill.includes('skill')) return 'self-improvement';
|
||||
return 'optional';
|
||||
if (skill.includes("triad")) return "triad";
|
||||
if (skill.includes("knowledge")) return "knowledge";
|
||||
if (skill.includes("security")) return "security";
|
||||
if (skill.includes("skill")) return "self-improvement";
|
||||
return "optional";
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -194,13 +195,13 @@ function categorizeSkill(skill) {
|
||||
function calculateRelevance(skill) {
|
||||
// Base relevance by category
|
||||
const baseScores = {
|
||||
'triad': 0.9,
|
||||
'knowledge': 0.8,
|
||||
'security': 0.7,
|
||||
'self-improvement': 0.95,
|
||||
'optional': 0.5
|
||||
triad: 0.9,
|
||||
knowledge: 0.8,
|
||||
security: 0.7,
|
||||
"self-improvement": 0.95,
|
||||
optional: 0.5,
|
||||
};
|
||||
|
||||
|
||||
const category = categorizeSkill(skill);
|
||||
return baseScores[category] || 0.5;
|
||||
}
|
||||
@@ -211,45 +212,45 @@ function calculateRelevance(skill) {
|
||||
* @returns {string} Formatted report
|
||||
*/
|
||||
function generateReport(gaps) {
|
||||
let report = '=== Gap Detection Report ===\n';
|
||||
let report = "=== Gap Detection Report ===\n";
|
||||
report += `Timestamp: ${gaps.timestamp}\n\n`;
|
||||
report += `Installed skills: ${gaps.installed_count}\n`;
|
||||
report += `Available skills: ${gaps.available_count}\n\n`;
|
||||
|
||||
|
||||
if (gaps.critical.length > 0) {
|
||||
report += '⚠️ CRITICAL GAPS DETECTED:\n';
|
||||
gaps.critical.forEach(gap => {
|
||||
report += "⚠️ CRITICAL GAPS DETECTED:\n";
|
||||
gaps.critical.forEach((gap) => {
|
||||
report += ` ${gap.skill}\n`;
|
||||
report += ` Impact: ${gap.impact}\n`;
|
||||
report += ` Recommendation: ${gap.recommendation}\n\n`;
|
||||
});
|
||||
} else {
|
||||
report += '✅ No critical gaps detected\n\n';
|
||||
report += "✅ No critical gaps detected\n\n";
|
||||
}
|
||||
|
||||
|
||||
if (gaps.optional.length > 0) {
|
||||
report += `📋 Optional gaps (${gaps.optional.length} skills):\n`;
|
||||
gaps.optional.slice(0, 10).forEach(gap => {
|
||||
gaps.optional.slice(0, 10).forEach((gap) => {
|
||||
report += ` ${gap.skill} (relevance: ${(gap.relevance * 100).toFixed(0)}%)\n`;
|
||||
});
|
||||
if (gaps.optional.length > 10) {
|
||||
report += ` ... and ${gaps.optional.length - 10} more\n`;
|
||||
}
|
||||
}
|
||||
|
||||
report += '\n=== End Gap Detection ===\n';
|
||||
|
||||
|
||||
report += "\n=== End Gap Detection ===\n";
|
||||
|
||||
return report;
|
||||
}
|
||||
|
||||
// CLI execution
|
||||
if (require.main === module) {
|
||||
const args = process.argv.slice(2);
|
||||
const criticalOnly = args.includes('--critical') || args.includes('-c');
|
||||
const jsonOutput = args.includes('--json') || args.includes('-j');
|
||||
|
||||
const criticalOnly = args.includes("--critical") || args.includes("-c");
|
||||
const jsonOutput = args.includes("--json") || args.includes("-j");
|
||||
|
||||
const gaps = detectGaps({ criticalOnly });
|
||||
|
||||
|
||||
if (jsonOutput) {
|
||||
console.log(JSON.stringify(gaps, null, 2));
|
||||
} else {
|
||||
@@ -263,5 +264,5 @@ module.exports = {
|
||||
getInstalledSkills,
|
||||
getAvailableSkills,
|
||||
getCriticalSkills,
|
||||
generateReport
|
||||
generateReport,
|
||||
};
|
||||
|
||||
@@ -1,36 +1,36 @@
|
||||
#!/usr/bin/env node
|
||||
/**
|
||||
* Opportunity Scanner Module - Phase 3: MCP Tool Integration
|
||||
*
|
||||
*
|
||||
* Watches GitHub releases, npm updates, CVEs, ClawHub new skills.
|
||||
* Integrates MCP tools: SearXNG, Playwright, GitHub API.
|
||||
*
|
||||
*
|
||||
* @module opportunity-scanner
|
||||
*/
|
||||
|
||||
const fs = require('fs');
|
||||
const path = require('path');
|
||||
const https = require('https');
|
||||
const sqlite3 = require('sqlite3').verbose();
|
||||
const fs = require("fs");
|
||||
const path = require("path");
|
||||
const https = require("https");
|
||||
const sqlite3 = require("sqlite3").verbose();
|
||||
|
||||
// Configuration
|
||||
const WORKSPACE = process.env.WORKSPACE || path.join(process.env.HOME, '.openclaw/workspace');
|
||||
const CURIOSITY_DIR = path.join(WORKSPACE, '.curiosity');
|
||||
const OPPS_DB = path.join(CURIOSITY_DIR, 'opportunities.db');
|
||||
const GITHUB_ORG = 'Heretek-AI';
|
||||
const MAIN_REPO = 'openclaw';
|
||||
const WORKSPACE = process.env.WORKSPACE || path.join(process.env.HOME, ".openclaw/workspace");
|
||||
const CURIOSITY_DIR = path.join(WORKSPACE, ".curiosity");
|
||||
const OPPS_DB = path.join(CURIOSITY_DIR, "opportunities.db");
|
||||
const GITHUB_ORG = "Heretek-AI";
|
||||
const MAIN_REPO = "openclaw";
|
||||
|
||||
// MCP Tool configurations
|
||||
const MCP_CONFIG = {
|
||||
searxng: {
|
||||
endpoint: process.env.SEARXNG_ENDPOINT || 'http://localhost:8080',
|
||||
timeout: 5000
|
||||
endpoint: process.env.SEARXNG_ENDPOINT || "http://localhost:8080",
|
||||
timeout: 5000,
|
||||
},
|
||||
github: {
|
||||
token: process.env.GH_TOKEN || process.env.GITHUB_TOKEN,
|
||||
org: GITHUB_ORG,
|
||||
repo: MAIN_REPO
|
||||
}
|
||||
repo: MAIN_REPO,
|
||||
},
|
||||
};
|
||||
|
||||
// Ensure directories exist
|
||||
@@ -48,8 +48,9 @@ function initDB() {
|
||||
reject(err);
|
||||
return;
|
||||
}
|
||||
|
||||
db.run(`
|
||||
|
||||
db.run(
|
||||
`
|
||||
CREATE TABLE IF NOT EXISTS opportunities (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
timestamp TEXT DEFAULT CURRENT_TIMESTAMP,
|
||||
@@ -60,10 +61,12 @@ function initDB() {
|
||||
priority TEXT DEFAULT 'low',
|
||||
processed INTEGER DEFAULT 0
|
||||
)
|
||||
`, (err) => {
|
||||
if (err) reject(err);
|
||||
else resolve(db);
|
||||
});
|
||||
`,
|
||||
(err) => {
|
||||
if (err) reject(err);
|
||||
else resolve(db);
|
||||
},
|
||||
);
|
||||
});
|
||||
});
|
||||
}
|
||||
@@ -78,22 +81,24 @@ function httpGet(url, headers = {}) {
|
||||
return new Promise((resolve, reject) => {
|
||||
const options = {
|
||||
headers: {
|
||||
'User-Agent': 'Curiosity-Engine/1.0',
|
||||
...headers
|
||||
}
|
||||
"User-Agent": "Curiosity-Engine/1.0",
|
||||
...headers,
|
||||
},
|
||||
};
|
||||
|
||||
https.get(url, options, (res) => {
|
||||
let data = '';
|
||||
res.on('data', chunk => data += chunk);
|
||||
res.on('end', () => {
|
||||
try {
|
||||
resolve(JSON.parse(data));
|
||||
} catch (err) {
|
||||
resolve(data);
|
||||
}
|
||||
});
|
||||
}).on('error', reject);
|
||||
|
||||
https
|
||||
.get(url, options, (res) => {
|
||||
let data = "";
|
||||
res.on("data", (chunk) => (data += chunk));
|
||||
res.on("end", () => {
|
||||
try {
|
||||
resolve(JSON.parse(data));
|
||||
} catch (err) {
|
||||
resolve(data);
|
||||
}
|
||||
});
|
||||
})
|
||||
.on("error", reject);
|
||||
});
|
||||
}
|
||||
|
||||
@@ -103,37 +108,37 @@ function httpGet(url, headers = {}) {
|
||||
*/
|
||||
async function scanGitHubReleases() {
|
||||
const opportunities = [];
|
||||
|
||||
|
||||
if (!MCP_CONFIG.github.token) {
|
||||
console.log(' GitHub token not set, skipping API calls');
|
||||
console.log(" GitHub token not set, skipping API calls");
|
||||
return opportunities;
|
||||
}
|
||||
|
||||
|
||||
try {
|
||||
const url = `https://api.github.com/repos/${MCP_CONFIG.github.org}/${MCP_CONFIG.github.repo}/releases?per_page=5`;
|
||||
const headers = {
|
||||
'Authorization': `token ${MCP_CONFIG.github.token}`,
|
||||
'Accept': 'application/vnd.github.v3+json'
|
||||
Authorization: `token ${MCP_CONFIG.github.token}`,
|
||||
Accept: "application/vnd.github.v3+json",
|
||||
};
|
||||
|
||||
|
||||
const releases = await httpGet(url, headers);
|
||||
|
||||
|
||||
if (Array.isArray(releases) && releases.length > 0) {
|
||||
releases.forEach(release => {
|
||||
releases.forEach((release) => {
|
||||
opportunities.push({
|
||||
source: 'github',
|
||||
source: "github",
|
||||
title: release.tag_name || release.name,
|
||||
url: release.html_url,
|
||||
type: 'release',
|
||||
priority: 'high',
|
||||
published_at: release.published_at
|
||||
type: "release",
|
||||
priority: "high",
|
||||
published_at: release.published_at,
|
||||
});
|
||||
});
|
||||
}
|
||||
} catch (err) {
|
||||
console.error(' GitHub API error:', err.message);
|
||||
console.error(" GitHub API error:", err.message);
|
||||
}
|
||||
|
||||
|
||||
return opportunities;
|
||||
}
|
||||
|
||||
@@ -143,21 +148,21 @@ async function scanGitHubReleases() {
|
||||
*/
|
||||
async function scanNpmUpdates() {
|
||||
const opportunities = [];
|
||||
const packages = ['openclaw', 'clawhub', 'mcporter'];
|
||||
|
||||
const packages = ["openclaw", "clawhub", "mcporter"];
|
||||
|
||||
for (const pkg of packages) {
|
||||
try {
|
||||
// Try direct npm registry first
|
||||
const url = `https://registry.npmjs.org/@heretek-ai/${pkg}/latest`;
|
||||
const data = await httpGet(url);
|
||||
|
||||
|
||||
if (data?.version) {
|
||||
opportunities.push({
|
||||
source: 'npm',
|
||||
source: "npm",
|
||||
title: `@heretek-ai/${pkg}@${data.version}`,
|
||||
url: `https://www.npmjs.com/package/@heretek-ai/${pkg}`,
|
||||
type: 'update',
|
||||
priority: 'medium'
|
||||
type: "update",
|
||||
priority: "medium",
|
||||
});
|
||||
}
|
||||
} catch (err) {
|
||||
@@ -165,15 +170,15 @@ async function scanNpmUpdates() {
|
||||
try {
|
||||
const searchUrl = `${MCP_CONFIG.searxng.endpoint}/search?q=@heretek-ai+${pkg}+npm&format=json`;
|
||||
const results = await httpGet(searchUrl);
|
||||
|
||||
|
||||
if (results?.results?.length > 0) {
|
||||
const firstResult = results.results[0];
|
||||
opportunities.push({
|
||||
source: 'npm',
|
||||
source: "npm",
|
||||
title: `@heretek-ai/${pkg} (via SearXNG)`,
|
||||
url: firstResult.url,
|
||||
type: 'update',
|
||||
priority: 'medium'
|
||||
type: "update",
|
||||
priority: "medium",
|
||||
});
|
||||
}
|
||||
} catch (searxErr) {
|
||||
@@ -181,7 +186,7 @@ async function scanNpmUpdates() {
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
return opportunities;
|
||||
}
|
||||
|
||||
@@ -191,27 +196,27 @@ async function scanNpmUpdates() {
|
||||
*/
|
||||
async function scanClawHubSkills() {
|
||||
const opportunities = [];
|
||||
|
||||
|
||||
try {
|
||||
const { execSync } = require('child_process');
|
||||
const output = execSync('clawhub search 2>/dev/null || true', { encoding: 'utf8' });
|
||||
const lines = output.trim().split('\n').slice(1);
|
||||
|
||||
lines.slice(0, 5).forEach(line => {
|
||||
const { execSync } = require("child_process");
|
||||
const output = execSync("clawhub search 2>/dev/null || true", { encoding: "utf8" });
|
||||
const lines = output.trim().split("\n").slice(1);
|
||||
|
||||
lines.slice(0, 5).forEach((line) => {
|
||||
const skillName = line.split(/\s+/)[0];
|
||||
if (skillName) {
|
||||
opportunities.push({
|
||||
source: 'clawhub',
|
||||
source: "clawhub",
|
||||
title: skillName,
|
||||
type: 'new_skill',
|
||||
priority: 'medium'
|
||||
type: "new_skill",
|
||||
priority: "medium",
|
||||
});
|
||||
}
|
||||
});
|
||||
} catch (err) {
|
||||
console.error(' ClawHub CLI not available:', err.message);
|
||||
console.error(" ClawHub CLI not available:", err.message);
|
||||
}
|
||||
|
||||
|
||||
return opportunities;
|
||||
}
|
||||
|
||||
@@ -221,50 +226,50 @@ async function scanClawHubSkills() {
|
||||
*/
|
||||
async function scanSecurityAdvisories() {
|
||||
const opportunities = [];
|
||||
|
||||
|
||||
if (MCP_CONFIG.github.token) {
|
||||
try {
|
||||
// GitHub code scanning alerts
|
||||
const url = `https://api.github.com/repos/${MCP_CONFIG.github.org}/${MCP_CONFIG.github.repo}/code-scanning/alerts?state=open&per_page=5`;
|
||||
const headers = {
|
||||
'Authorization': `token ${MCP_CONFIG.github.token}`,
|
||||
'Accept': 'application/vnd.github.v3+json'
|
||||
Authorization: `token ${MCP_CONFIG.github.token}`,
|
||||
Accept: "application/vnd.github.v3+json",
|
||||
};
|
||||
|
||||
|
||||
const alerts = await httpGet(url, headers);
|
||||
|
||||
|
||||
if (Array.isArray(alerts) && alerts.length > 0) {
|
||||
opportunities.push({
|
||||
source: 'github',
|
||||
source: "github",
|
||||
title: `${alerts.length} open code scanning alerts`,
|
||||
url: `https://github.com/${MCP_CONFIG.github.org}/${MCP_CONFIG.github.repo}/code-scanning`,
|
||||
type: 'security',
|
||||
priority: 'critical'
|
||||
type: "security",
|
||||
priority: "critical",
|
||||
});
|
||||
}
|
||||
} catch (err) {
|
||||
console.error(' Security scan error:', err.message);
|
||||
console.error(" Security scan error:", err.message);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
// Fallback: SearXNG CVE search
|
||||
try {
|
||||
const searchUrl = `${MCP_CONFIG.searxng.endpoint}/search?q=CVE+Heretek-AI+openclaw&format=json`;
|
||||
const results = await httpGet(searchUrl);
|
||||
|
||||
|
||||
if (results?.results?.length > 0) {
|
||||
opportunities.push({
|
||||
source: 'searxng',
|
||||
title: 'CVE mentions detected',
|
||||
source: "searxng",
|
||||
title: "CVE mentions detected",
|
||||
url: results.results[0].url,
|
||||
type: 'security',
|
||||
priority: 'high'
|
||||
type: "security",
|
||||
priority: "high",
|
||||
});
|
||||
}
|
||||
} catch (err) {
|
||||
// Silent fail for optional search
|
||||
}
|
||||
|
||||
|
||||
return opportunities;
|
||||
}
|
||||
|
||||
@@ -274,43 +279,43 @@ async function scanSecurityAdvisories() {
|
||||
* @returns {Promise<Object>} Opportunity scan result
|
||||
*/
|
||||
async function scanOpportunities(options = {}) {
|
||||
const { sources = ['github', 'npm', 'clawhub', 'security'] } = options;
|
||||
|
||||
const { sources = ["github", "npm", "clawhub", "security"] } = options;
|
||||
|
||||
const allOpportunities = [];
|
||||
|
||||
if (sources.includes('github')) {
|
||||
|
||||
if (sources.includes("github")) {
|
||||
const gh = await scanGitHubReleases();
|
||||
allOpportunities.push(...gh);
|
||||
}
|
||||
|
||||
if (sources.includes('npm')) {
|
||||
|
||||
if (sources.includes("npm")) {
|
||||
const npm = await scanNpmUpdates();
|
||||
allOpportunities.push(...npm);
|
||||
}
|
||||
|
||||
if (sources.includes('clawhub')) {
|
||||
|
||||
if (sources.includes("clawhub")) {
|
||||
const clawhub = await scanClawHubSkills();
|
||||
allOpportunities.push(...clawhub);
|
||||
}
|
||||
|
||||
if (sources.includes('security')) {
|
||||
|
||||
if (sources.includes("security")) {
|
||||
const security = await scanSecurityAdvisories();
|
||||
allOpportunities.push(...security);
|
||||
}
|
||||
|
||||
|
||||
// Record to database
|
||||
await recordOpportunities(allOpportunities);
|
||||
|
||||
|
||||
return {
|
||||
timestamp: new Date().toISOString(),
|
||||
opportunities: allOpportunities,
|
||||
total_count: allOpportunities.length,
|
||||
by_source: {
|
||||
github: allOpportunities.filter(o => o.source === 'github').length,
|
||||
npm: allOpportunities.filter(o => o.source === 'npm').length,
|
||||
clawhub: allOpportunities.filter(o => o.source === 'clawhub').length,
|
||||
security: allOpportunities.filter(o => o.source === 'security').length
|
||||
}
|
||||
github: allOpportunities.filter((o) => o.source === "github").length,
|
||||
npm: allOpportunities.filter((o) => o.source === "npm").length,
|
||||
clawhub: allOpportunities.filter((o) => o.source === "clawhub").length,
|
||||
security: allOpportunities.filter((o) => o.source === "security").length,
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
@@ -325,16 +330,16 @@ function recordOpportunities(opportunities) {
|
||||
reject(err);
|
||||
return;
|
||||
}
|
||||
|
||||
|
||||
const stmt = db.prepare(`
|
||||
INSERT INTO opportunities (source, title, url, type, priority)
|
||||
VALUES (?, ?, ?, ?, ?)
|
||||
`);
|
||||
|
||||
opportunities.forEach(opp => {
|
||||
stmt.run(opp.source, opp.title, opp.url || '', opp.type, opp.priority);
|
||||
|
||||
opportunities.forEach((opp) => {
|
||||
stmt.run(opp.source, opp.title, opp.url || "", opp.type, opp.priority);
|
||||
});
|
||||
|
||||
|
||||
stmt.finalize((err) => {
|
||||
db.close();
|
||||
if (err) reject(err);
|
||||
@@ -350,45 +355,47 @@ function recordOpportunities(opportunities) {
|
||||
* @returns {string} Formatted report
|
||||
*/
|
||||
function generateReport(result) {
|
||||
let report = '=== Opportunity Scanning Report ===\n';
|
||||
let report = "=== Opportunity Scanning Report ===\n";
|
||||
report += `Timestamp: ${result.timestamp}\n\n`;
|
||||
report += `Total opportunities: ${result.total_count}\n`;
|
||||
report += `By source: GitHub=${result.by_source.github}, npm=${result.by_source.npm}, ClawHub=${result.by_source.clawhub}, Security=${result.by_source.security}\n\n`;
|
||||
|
||||
|
||||
if (result.opportunities.length > 0) {
|
||||
report += '📦 OPPORTUNITIES DETECTED:\n';
|
||||
result.opportunities.forEach(opp => {
|
||||
const icon = opp.priority === 'critical' ? '⚠️' : opp.priority === 'high' ? '🔴' : '📋';
|
||||
report += "📦 OPPORTUNITIES DETECTED:\n";
|
||||
result.opportunities.forEach((opp) => {
|
||||
const icon = opp.priority === "critical" ? "⚠️" : opp.priority === "high" ? "🔴" : "📋";
|
||||
report += ` ${icon} ${opp.title}\n`;
|
||||
report += ` Source: ${opp.source}\n`;
|
||||
report += ` Type: ${opp.type}\n`;
|
||||
report += ` Priority: ${opp.priority}\n`;
|
||||
if (opp.url) report += ` URL: ${opp.url}\n`;
|
||||
report += '\n';
|
||||
report += "\n";
|
||||
});
|
||||
} else {
|
||||
report += '✅ No new opportunities detected\n';
|
||||
report += "✅ No new opportunities detected\n";
|
||||
}
|
||||
|
||||
report += '\n=== End Opportunity Scanning ===\n';
|
||||
|
||||
|
||||
report += "\n=== End Opportunity Scanning ===\n";
|
||||
|
||||
return report;
|
||||
}
|
||||
|
||||
// CLI execution
|
||||
if (require.main === module) {
|
||||
initDB().then(async () => {
|
||||
const args = process.argv.slice(2);
|
||||
const jsonOutput = args.includes('--json') || args.includes('-j');
|
||||
|
||||
const result = await scanOpportunities();
|
||||
|
||||
if (jsonOutput) {
|
||||
console.log(JSON.stringify(result, null, 2));
|
||||
} else {
|
||||
console.log(generateReport(result));
|
||||
}
|
||||
}).catch(console.error);
|
||||
initDB()
|
||||
.then(async () => {
|
||||
const args = process.argv.slice(2);
|
||||
const jsonOutput = args.includes("--json") || args.includes("-j");
|
||||
|
||||
const result = await scanOpportunities();
|
||||
|
||||
if (jsonOutput) {
|
||||
console.log(JSON.stringify(result, null, 2));
|
||||
} else {
|
||||
console.log(generateReport(result));
|
||||
}
|
||||
})
|
||||
.catch(console.error);
|
||||
}
|
||||
|
||||
// Export for module usage
|
||||
@@ -399,5 +406,5 @@ module.exports = {
|
||||
scanClawHubSkills,
|
||||
scanSecurityAdvisories,
|
||||
generateReport,
|
||||
initDB
|
||||
initDB,
|
||||
};
|
||||
|
||||
@@ -2,26 +2,26 @@
|
||||
"name": "@heretek-ai/curiosity-engine-modules",
|
||||
"version": "1.0.0",
|
||||
"description": "Curiosity Engine modular implementation for Tabula Myriad triad",
|
||||
"keywords": [
|
||||
"anomaly-detection",
|
||||
"autonomy",
|
||||
"curiosity",
|
||||
"deliberation",
|
||||
"gap-detection",
|
||||
"triad"
|
||||
],
|
||||
"license": "MIT",
|
||||
"author": "Tabula Myriad <heretek-ai>",
|
||||
"main": "modules/gap-detector.js",
|
||||
"scripts": {
|
||||
"test": "node tests/runner.js",
|
||||
"run": "node curiosity-engine.js",
|
||||
"gap": "node modules/gap-detector.js",
|
||||
"anomaly": "node modules/anomaly-detector.js",
|
||||
"opportunity": "node modules/opportunity-scanner.js",
|
||||
"capability": "node modules/capability-mapper.js",
|
||||
"gap": "node modules/gap-detector.js",
|
||||
"opportunity": "node modules/opportunity-scanner.js",
|
||||
"run": "node curiosity-engine.js",
|
||||
"test": "node tests/runner.js",
|
||||
"trigger": "node modules/deliberation-trigger.js"
|
||||
},
|
||||
"keywords": [
|
||||
"curiosity",
|
||||
"autonomy",
|
||||
"triad",
|
||||
"gap-detection",
|
||||
"anomaly-detection",
|
||||
"deliberation"
|
||||
],
|
||||
"author": "Tabula Myriad <heretek-ai>",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"sqlite3": "^5.1.6"
|
||||
},
|
||||
|
||||
Reference in New Issue
Block a user