mirror of
https://github.com/n8n-io/n8n.git
synced 2026-10-12 02:19:43 +00:00
chore: Add dev tooling to seed an instance and inspect its activity feed (#37645)
This commit is contained in:
Arvin A
@@ -106,6 +106,26 @@ When changing directories, use `pushd` to navigate into the directory and
|
||||
`popd` to return to the previous directory. When in doubt, use `pwd` to check
|
||||
your current directory.
|
||||
|
||||
### Seeding a local instance
|
||||
|
||||
An empty instance is a bad place to test anything that reads a user's work.
|
||||
These commands fill one. They are dev tooling on the private root package, so
|
||||
they never reach a user.
|
||||
|
||||
```bash
|
||||
N8N_API_KEY=<jwt> pnpm seed:preference # 10 workflows in one house style, plus history
|
||||
N8N_API_KEY=<jwt> pnpm seed:account # ~500 varied workflows across 30 projects
|
||||
pnpm inspect:activity # read-only activity_event viewer on 127.0.0.1
|
||||
```
|
||||
|
||||
Both seed profiles delete their own prior output, so a re-run replaces it.
|
||||
That clear step deletes anything named `[seed]` and any empty team project,
|
||||
whoever made them, so do not point either at a shared instance. The viewer is
|
||||
unauthenticated and serves the whole table: keep it on loopback.
|
||||
|
||||
See [scripts/instance-seeding/AGENTS.md](scripts/instance-seeding/AGENTS.md) for
|
||||
profiles, tokens, determinism, and the other commands.
|
||||
|
||||
### Code Quality
|
||||
- `pnpm lint` - Lint code
|
||||
- `pnpm typecheck` - Run type checks
|
||||
|
||||
+7
-1
@@ -41,7 +41,13 @@
|
||||
"reset": "node scripts/ensure-zx.mjs && zx scripts/reset.mjs",
|
||||
"format": "turbo run format && node scripts/format.mjs",
|
||||
"grind": "node scripts/grind.mjs",
|
||||
"nathan": "node scripts/nathan.mjs",
|
||||
"inspect:activity": "node scripts/instance-seeding/inspectActivity.mjs",
|
||||
"seed:account": "node scripts/instance-seeding/seedInstance.mjs",
|
||||
"seed:history": "node scripts/instance-seeding/seedHistory.mjs",
|
||||
"seed:preference": "cross-env PROFILE=preference pnpm seed:account && pnpm seed:history",
|
||||
"seed:account:check": "node scripts/instance-seeding/checkPreferenceProfile.mjs",
|
||||
"seed:test": "node --test scripts/instance-seeding/*.test.mjs",
|
||||
"nathan": "node scripts/nathan.mjs",
|
||||
"agent:setup": "node scripts/agent-setup.mjs",
|
||||
"format:check": "turbo run format:check",
|
||||
"lint": "turbo run lint",
|
||||
|
||||
@@ -1,5 +1,161 @@
|
||||
# Seed n8n instance
|
||||
|
||||
Four dev-tooling scripts, none of them product code:
|
||||
|
||||
| Script | Command | What it does |
|
||||
| --- | --- | --- |
|
||||
| `seedInstance.mjs` | `pnpm seed:account` | Builds the estate via the public API. Two profiles. |
|
||||
| `seedHistory.mjs` | `pnpm seed:history` | Executions, assistant threads, activity entries via SQLite. |
|
||||
| `inspectActivity.mjs` | `pnpm inspect:activity` | Read-only viewer of `activity_event`. |
|
||||
| `checkPreferenceProfile.mjs` | `pnpm seed:account:check` | Checks top-level parameter names against node definitions. |
|
||||
|
||||
`pnpm seed:preference` chains the first two. That is the single command that leaves
|
||||
a usable instance.
|
||||
|
||||
## Two profiles
|
||||
|
||||
`PROFILE` picks the shape of the estate. They exist for opposite reasons.
|
||||
|
||||
**`estate`** (default) is ~500 workflows across 30 projects, varied and random, built
|
||||
to stress the workflow dependency graph. The rest of this document describes it.
|
||||
|
||||
**`preference`** is 10 hand-written workflows in one house style, so an agent can be
|
||||
asked "what does this org normally do?" and be graded on the answer. Diversity is the
|
||||
bug here, not the feature.
|
||||
|
||||
```sh
|
||||
N8N_API_KEY=… PROFILE=preference pnpm seed:account
|
||||
```
|
||||
|
||||
The ten live in `preference-profile.mjs` and are written out, not generated. At n=10
|
||||
generation buys nothing, and a generated workflow whose name does not match its nodes
|
||||
teaches the reader something false.
|
||||
|
||||
### The house style
|
||||
|
||||
Five rules, no exceptions. Each is a place n8n offers a real choice and this estate
|
||||
always makes the same one, because a rule with no alternative is a constraint rather
|
||||
than a preference.
|
||||
|
||||
| Rule | What it rejects |
|
||||
| --- | --- |
|
||||
| OpenAI `gpt-4o-mini` chat model (5/5 AI workflows) | Anthropic, Gemini, Mistral, Ollama |
|
||||
| Linear for tracking (6/6 tracker workflows) | Jira, GitHub Issues, Asana, Trello |
|
||||
| Slack for notifications (10/10 workflows) | Discord, Teams, Telegram, email |
|
||||
| Cron expressions on every schedule | the `interval` rule form |
|
||||
| Every workflow ends writing to `automation_runs` | no audit trail |
|
||||
|
||||
### Determinism
|
||||
|
||||
Every random choice goes through one seeded PRNG, and seeded runs pin the clock too.
|
||||
`PROFILE=preference` sets `SEED=1`, so two runs give byte-identical estates. That is
|
||||
what lets an A/B eval compare two arms against one instance. Set `SEED` explicitly for
|
||||
the estate profile.
|
||||
|
||||
### Credentials
|
||||
|
||||
`preference` builds credentials from the developer's own tokens:
|
||||
|
||||
| Env var | Credential |
|
||||
| --- | --- |
|
||||
| `SEED_OPENAI_API_KEY` | `openAiApi` |
|
||||
| `SEED_LINEAR_API_KEY` | `linearApi` |
|
||||
| `SEED_SLACK_TOKEN` | `slackApi` |
|
||||
| `SEED_GMAIL_OAUTH` | `gmailOAuth2` |
|
||||
| `SEED_ENRICHMENT_TOKEN` | `httpHeaderAuth` |
|
||||
| `SEED_LINEAR_TEAM_ID` | Linear team the issue nodes target |
|
||||
|
||||
A missing token is not an error. It gives a placeholder named `(seed, fake key)`, so
|
||||
the workflows stay wired and openable but visibly not runnable. Supply the token later
|
||||
and re-run: the credential is upgraded in place and keeps its id, so nodes keep
|
||||
pointing at it.
|
||||
|
||||
Two things make that work. The upgrade is a `PATCH`, because the public API has no
|
||||
`PUT` for credentials and its 405 fails quietly enough to look like success. And the
|
||||
`preference` profile skips `clearSeeded()`, which deletes credentials and would change
|
||||
every id on every run; it removes only the workflows.
|
||||
|
||||
Secrets go through the public API, so n8n does the encryption and this tooling never
|
||||
touches the instance key. Token lengths are logged, never values.
|
||||
|
||||
## `seedHistory.mjs`
|
||||
|
||||
Writes what the public API cannot: executions have no create route, threads and
|
||||
activity entries have none at all. Run it after `seed:account`.
|
||||
|
||||
A live instance is fine. SQLite serialises writers, so n8n's inserts queue behind the
|
||||
script. Prefer an idle one: an instance actively executing workflows can hold the write
|
||||
lock long enough to fail.
|
||||
|
||||
A default run gives 175 executions over 14 days, 10 threads with 22 messages, and 35
|
||||
activity entries. `[seed] Invoice Dunning` fails its three most recent runs, so a "what
|
||||
broke?" probe has a definite answer.
|
||||
|
||||
### The window ends at the current time
|
||||
|
||||
n8n prunes on age: threads after 30 days, executions past `EXECUTIONS_DATA_MAX_AGE`
|
||||
(336 hours). A fixed past date puts the fortnight beyond both cutoffs and the next
|
||||
startup deletes it. This was observed, not predicted: a hardcoded clock lost all 10
|
||||
threads on the first restart.
|
||||
|
||||
Determinism survives. The seeded PRNG still decides which workflow fails, on which
|
||||
run, at which node, and every message body. Only absolute timestamps move. `HISTORY_NOW`
|
||||
pins the window if you accept the pruning.
|
||||
|
||||
### Errors match their workflow
|
||||
|
||||
An error names a node, and that node must exist in the workflow the error is attached
|
||||
to and say something a node of that type could say. One shared error message produced
|
||||
self-contradicting records, such as a Gmail auth failure on a workflow with no Gmail
|
||||
node. `NODE_FAILURES` maps node type to a plausible message.
|
||||
|
||||
### Clearing
|
||||
|
||||
Clear and rewrite are one transaction, so a failed insert cannot leave the previous
|
||||
history deleted and nothing in its place. Threads and activity are cleared by project
|
||||
id, not workflow id: re-seeding replaces the workflows with fresh ids, which orphans
|
||||
rows keyed on the old ones and then collides on the seeded primary keys. Executions
|
||||
cascade when their workflow is deleted.
|
||||
|
||||
## `inspectActivity.mjs`
|
||||
|
||||
```sh
|
||||
pnpm inspect:activity # ~/.n8n/database.sqlite
|
||||
DB_SQLITE_DATABASE=/path/to/database.sqlite pnpm inspect:activity
|
||||
PORT=5700 pnpm inspect:activity
|
||||
```
|
||||
|
||||
Every column of `activity_event`, paginated, sortable on any column, with a free-text
|
||||
filter spanning all columns including the JSON `data` blob.
|
||||
|
||||
Read-only three times over, since a later edit can undo any one layer alone:
|
||||
|
||||
1. The connection is opened `readOnly`, so SQLite refuses writes.
|
||||
2. Every statement is a SELECT.
|
||||
3. The sort column comes from an allowlist derived from the table, because a column
|
||||
name cannot be a bound parameter.
|
||||
|
||||
Non-GET methods return 405. A request whose `Host` is not loopback returns 403, which
|
||||
closes DNS rebinding. Nothing else authenticates.
|
||||
|
||||
**A debug surface.** Unauthenticated, serving the whole table including who did what in
|
||||
which project. Loopback only. Do not tunnel or port-forward it.
|
||||
|
||||
No dependencies: `node:sqlite` and `node:http` are built into Node 24.
|
||||
|
||||
## Note on `activity_event`
|
||||
|
||||
`activityEventCategories` is `['workflow', 'credential']`. Executions are deliberately
|
||||
absent, because `execution_entity` already indexes `(workflowId, status, id)` for the
|
||||
read a feed wants, so a row per run would duplicate it and pay an insert on the
|
||||
execution hot path. A reader queries that table instead. The execution and activity
|
||||
phases of the seed are therefore independent, with no ordering between them.
|
||||
|
||||
---
|
||||
|
||||
# The estate profile
|
||||
|
||||
|
||||
`seedInstance.mjs` fills a local n8n instance with a realistic-looking spread of
|
||||
projects, workflows, credentials, and data tables via the **public API**. The
|
||||
resulting dependency graph is designed to render like a real org's automation
|
||||
|
||||
@@ -0,0 +1,99 @@
|
||||
#!/usr/bin/env node
|
||||
// Check the top-level parameter names the preference workflows emit against the names
|
||||
// their node declares. Catches a silent failure: n8n accepts a misspelled parameter,
|
||||
// stores it, and renders an empty field without erroring.
|
||||
//
|
||||
// Walks `node.parameters` one level deep. Nested shapes (`filters.conditions[]`,
|
||||
// `columns.value`, `additionalFields`) and values are not checked, so a green run
|
||||
// means nothing is misspelled at the top level, not that the parameters are correct.
|
||||
//
|
||||
// Some nodes declare parameters outside their own directory, via a spread or an
|
||||
// exported constant. Those sources need listing in SOURCES or the check false-fails.
|
||||
|
||||
import { readdirSync, readFileSync, statSync } from 'node:fs';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
import path from 'node:path';
|
||||
|
||||
import { preferenceWorkflows } from './preference-profile.mjs';
|
||||
|
||||
const REPO = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '../..');
|
||||
|
||||
const SOURCES = {
|
||||
'n8n-nodes-base.scheduleTrigger': ['packages/nodes-base/nodes/Schedule'],
|
||||
'n8n-nodes-base.webhook': ['packages/nodes-base/nodes/Webhook'],
|
||||
'n8n-nodes-base.if': ['packages/nodes-base/nodes/If'],
|
||||
'n8n-nodes-base.set': ['packages/nodes-base/nodes/Set'],
|
||||
'n8n-nodes-base.httpRequest': ['packages/nodes-base/nodes/HttpRequest'],
|
||||
'n8n-nodes-base.slack': ['packages/nodes-base/nodes/Slack'],
|
||||
'n8n-nodes-base.linear': ['packages/nodes-base/nodes/Linear'],
|
||||
'n8n-nodes-base.gmail': ['packages/nodes-base/nodes/Google/Gmail'],
|
||||
'n8n-nodes-base.dataTable': ['packages/nodes-base/nodes/DataTable'],
|
||||
'@n8n/n8n-nodes-langchain.agent': [
|
||||
'packages/@n8n/nodes-langchain/nodes/agents/Agent',
|
||||
// `promptType` and friends are spread in from here.
|
||||
'packages/@n8n/nodes-langchain/utils/descriptions.ts',
|
||||
],
|
||||
'@n8n/n8n-nodes-langchain.lmChatOpenAi': [
|
||||
'packages/@n8n/nodes-langchain/nodes/llms/LMChatOpenAi',
|
||||
],
|
||||
};
|
||||
|
||||
/** Every `.ts` file under `target`, which may itself be a file. */
|
||||
function typeScriptFiles(target) {
|
||||
const stat = statSync(target, { throwIfNoEntry: false });
|
||||
if (!stat) throw new Error(`SOURCES points at a missing path: ${target}`);
|
||||
if (stat.isFile()) return target.endsWith('.ts') ? [target] : [];
|
||||
return readdirSync(target, { withFileTypes: true }).flatMap((entry) => {
|
||||
const child = path.join(target, entry.name);
|
||||
if (entry.isDirectory()) return typeScriptFiles(child);
|
||||
return entry.isFile() && child.endsWith('.ts') ? [child] : [];
|
||||
});
|
||||
}
|
||||
|
||||
// Read the files in Node rather than shelling out to grep. A missing path now throws
|
||||
// instead of arriving as an empty result set, and there is no dependency on which
|
||||
// grep the platform provides or on its exit codes.
|
||||
function declaredNames(dirs) {
|
||||
const out = new Set();
|
||||
for (const dir of dirs) {
|
||||
for (const file of typeScriptFiles(path.join(REPO, dir))) {
|
||||
const txt = readFileSync(file, 'utf8');
|
||||
for (const m of txt.matchAll(/name: '([A-Za-z0-9_]+)'/g)) out.add(m[1]);
|
||||
for (const m of txt.matchAll(/_FIELD = '([A-Za-z0-9_]+)'/g)) out.add(m[1]);
|
||||
}
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
const cache = new Map();
|
||||
const problems = [];
|
||||
let checked = 0;
|
||||
|
||||
for (const wf of preferenceWorkflows({ customer_accounts: 'dt1', automation_runs: 'dt2' })) {
|
||||
for (const node of wf.nodes) {
|
||||
const dirs = SOURCES[node.type];
|
||||
if (!dirs) {
|
||||
problems.push(`${wf.name} / ${node.name}: node type ${node.type} is not in SOURCES`);
|
||||
continue;
|
||||
}
|
||||
if (!cache.has(node.type)) cache.set(node.type, declaredNames(dirs));
|
||||
const declared = cache.get(node.type);
|
||||
for (const key of Object.keys(node.parameters ?? {})) {
|
||||
checked++;
|
||||
if (!declared.has(key)) {
|
||||
problems.push(`${wf.name} / ${node.name} (${node.type}): parameter '${key}' not declared`);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
console.log(
|
||||
`Checked ${checked} parameters across ${preferenceWorkflows({ customer_accounts: 'a', automation_runs: 'b' }).length} workflows.`,
|
||||
);
|
||||
if (problems.length === 0) {
|
||||
console.log('All parameter names are declared by their node.');
|
||||
process.exit(0);
|
||||
}
|
||||
console.error(`\n${problems.length} problem(s):`);
|
||||
for (const p of problems) console.error(` - ${p}`);
|
||||
process.exit(1);
|
||||
@@ -0,0 +1,249 @@
|
||||
#!/usr/bin/env node
|
||||
// A one-page read-only viewer for the `activity_event` table.
|
||||
//
|
||||
// A debug surface: unauthenticated, and it serves the whole table including who did
|
||||
// what in which project. Binds loopback only. Do not tunnel or port-forward it.
|
||||
//
|
||||
// Read-only three times over, since a later edit can undo any one layer alone:
|
||||
// the connection is opened `readOnly`, every statement is a SELECT, and the sort
|
||||
// column comes from an allowlist because a column name cannot be bound.
|
||||
|
||||
import { DatabaseSync } from 'node:sqlite';
|
||||
import { realpathSync } from 'node:fs';
|
||||
import http from 'node:http';
|
||||
import path from 'node:path';
|
||||
import os from 'node:os';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
|
||||
// Mirrors `getN8nFolder()` in @n8n/config: `.n8n` sits inside N8N_USER_FOLDER.
|
||||
const DB_PATH =
|
||||
process.env.DB_SQLITE_DATABASE ??
|
||||
path.join(process.env.N8N_USER_FOLDER ?? os.homedir(), '.n8n', 'database.sqlite');
|
||||
const PORT = Number(process.env.PORT) || 5699;
|
||||
const HOST = '127.0.0.1';
|
||||
const PAGE_SIZES = [25, 50, 100, 250];
|
||||
|
||||
// Assigned by `open()`, not at import. Opening the database and binding a port are
|
||||
// side effects, and a module that does them on import cannot be tested or reused.
|
||||
let db;
|
||||
let COLUMNS = [];
|
||||
let SORTABLE = new Set();
|
||||
|
||||
function open(dbPath) {
|
||||
try {
|
||||
db = new DatabaseSync(dbPath, { readOnly: true });
|
||||
} catch (e) {
|
||||
console.error(`Cannot open ${dbPath} read-only: ${e.message}`);
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
const tableExists = db
|
||||
.prepare("SELECT name FROM sqlite_master WHERE type='table' AND name='activity_event'")
|
||||
.get();
|
||||
if (!tableExists) {
|
||||
console.error(`No activity_event table in ${dbPath}.`);
|
||||
console.error('The table ships with the activity-event migration; this instance predates it.');
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
// Derived from the table, so a new column becomes sortable without a code change.
|
||||
COLUMNS = db
|
||||
.prepare("SELECT name FROM pragma_table_info('activity_event')")
|
||||
.all()
|
||||
.map((r) => r.name);
|
||||
SORTABLE = new Set(COLUMNS);
|
||||
}
|
||||
|
||||
// Match the whole authority, not a prefix of it. Slicing up to the first `]` accepts
|
||||
// `[::1]evil.com`, and splitting on `:` accepts `127.0.0.1.evil.com`, so both let a
|
||||
// rebinding host through. Anchored patterns with an optional numeric port do not.
|
||||
export function isLoopbackHost(rawHost) {
|
||||
const host = rawHost ?? '';
|
||||
const bracketed = /^\[([0-9a-fA-F:]+)\](?::\d+)?$/.exec(host);
|
||||
if (bracketed) return ['::1', '0:0:0:0:0:0:0:1'].includes(bracketed[1].toLowerCase());
|
||||
const plain = /^([0-9a-zA-Z.-]+)(?::\d+)?$/.exec(host);
|
||||
return plain !== null && ['127.0.0.1', 'localhost'].includes(plain[1].toLowerCase());
|
||||
}
|
||||
|
||||
const esc = (v) =>
|
||||
v === null || v === undefined
|
||||
? ''
|
||||
: String(v).replace(
|
||||
/[&<>"']/g,
|
||||
(c) => ({ '&': '&', '<': '<', '>': '>', '"': '"', "'": ''' })[c],
|
||||
);
|
||||
|
||||
function query({ q, sort, dir, limit, offset }) {
|
||||
// Spans every column including the JSON `data` blob, so a run id or a node name
|
||||
// finds its entry without the reader knowing which field holds it.
|
||||
// `%` and `_` are LIKE wildcards; unescaped, either matches every row.
|
||||
const escaped = q.replace(/[\\%_]/g, (c) => `\\${c}`);
|
||||
const where = q
|
||||
? `WHERE ${COLUMNS.map((c) => `IFNULL("${c}", '') LIKE ? ESCAPE '\\'`).join(' OR ')}`
|
||||
: '';
|
||||
const params = q ? COLUMNS.map(() => `%${escaped}%`) : [];
|
||||
|
||||
const total = db.prepare(`SELECT COUNT(*) AS n FROM activity_event ${where}`).get(...params).n;
|
||||
// `sort` and `dir` are allowlisted above; everything else is bound.
|
||||
const rows = db
|
||||
.prepare(`SELECT * FROM activity_event ${where} ORDER BY "${sort}" ${dir} LIMIT ? OFFSET ?`)
|
||||
.all(...params, limit, offset);
|
||||
return { total, rows };
|
||||
}
|
||||
|
||||
function page({ q, sort, dir, limit, offset, total, rows }) {
|
||||
const link = (over) => {
|
||||
const p = new URLSearchParams({
|
||||
q,
|
||||
sort,
|
||||
dir,
|
||||
limit: String(limit),
|
||||
offset: String(offset),
|
||||
...over,
|
||||
});
|
||||
if (!p.get('q')) p.delete('q');
|
||||
return `/?${p}`;
|
||||
};
|
||||
const header = COLUMNS.map((c) => {
|
||||
const active = c === sort;
|
||||
const nextDir = active && dir === 'DESC' ? 'asc' : 'desc';
|
||||
const arrow = active ? (dir === 'DESC' ? ' ▾' : ' ▴') : '';
|
||||
return `<th><a href="${esc(link({ sort: c, dir: nextDir, offset: '0' }))}"${active ? ' class="on"' : ''}>${esc(c)}${arrow}</a></th>`;
|
||||
}).join('');
|
||||
|
||||
// One line per row, full value on hover. A wrapped cell turns 25 rows into six
|
||||
// screens, and this is read by scanning down it.
|
||||
const body =
|
||||
rows.length === 0
|
||||
? `<tr><td colspan="${COLUMNS.length}" class="empty">No rows${q ? ` matching “${esc(q)}”` : ''}.</td></tr>`
|
||||
: rows
|
||||
.map(
|
||||
(r) =>
|
||||
`<tr>${COLUMNS.map((c) => {
|
||||
const v = r[c];
|
||||
const s = v === null || v === undefined ? '' : String(v);
|
||||
const cls = ` class="c-${c}"`;
|
||||
// `title` only where it adds something the cell cannot show.
|
||||
const t = s.length > 20 ? ` title="${esc(s)}"` : '';
|
||||
return `<td${cls}${t}>${esc(s)}</td>`;
|
||||
}).join('')}</tr>`,
|
||||
)
|
||||
.join('');
|
||||
|
||||
const from = total === 0 ? 0 : offset + 1;
|
||||
const to = Math.min(offset + limit, total);
|
||||
const prev = Math.max(0, offset - limit);
|
||||
const next = offset + limit;
|
||||
|
||||
return `<!doctype html>
|
||||
<html lang="en"><head><meta charset="utf-8">
|
||||
<title>activity_event · ${total} rows</title>
|
||||
<style>
|
||||
:root { color-scheme: light dark; --line: #8883; }
|
||||
body { font: 13px/1.5 ui-monospace, SFMono-Regular, Menlo, monospace; margin: 0; padding: 16px 20px; }
|
||||
h1 { font-size: 15px; margin: 0 0 2px; font-weight: 600; }
|
||||
.sub { opacity: .65; margin-bottom: 14px; }
|
||||
.bar { display: flex; gap: 10px; align-items: center; margin-bottom: 12px; flex-wrap: wrap; }
|
||||
input[type=search] { font: inherit; padding: 5px 8px; min-width: 280px; border: 1px solid var(--line); border-radius: 5px; background: transparent; color: inherit; }
|
||||
button, select { font: inherit; padding: 5px 8px; border: 1px solid var(--line); border-radius: 5px; background: transparent; color: inherit; cursor: pointer; }
|
||||
table { border-collapse: collapse; width: 100%; table-layout: fixed; }
|
||||
th, td { border-bottom: 1px solid var(--line); padding: 5px 8px; text-align: left; overflow: hidden; text-overflow: ellipsis; white-space: nowrap; }
|
||||
th { position: sticky; top: 0; background: Canvas; font-weight: 600; }
|
||||
/* Fixed layout so the wide id columns cannot starve the ones being scanned. */
|
||||
.c-id, th:nth-child(1) { width: 3.5rem; }
|
||||
.c-category, th:nth-child(2) { width: 6rem; }
|
||||
.c-action, th:nth-child(3) { width: 6rem; }
|
||||
.c-typeVersion, th:nth-child(4) { width: 3.5rem; }
|
||||
.c-userId, th:nth-child(5) { width: 8%; }
|
||||
.c-projectId, th:nth-child(6) { width: 8%; }
|
||||
.c-resourceType, th:nth-child(7) { width: 6rem; }
|
||||
.c-resourceId, th:nth-child(8) { width: 8%; }
|
||||
.c-resourceName, th:nth-child(9) { width: 22%; }
|
||||
.c-data, th:nth-child(10) { width: 14%; }
|
||||
.c-createdAt, th:nth-child(11) { width: 13.5rem; }
|
||||
td[title] { cursor: help; }
|
||||
th a { color: inherit; text-decoration: none; }
|
||||
th a.on { text-decoration: underline; }
|
||||
tr:hover td { background: #8881; }
|
||||
.empty { opacity: .6; text-align: center; padding: 28px; }
|
||||
.pager { display: flex; gap: 10px; align-items: center; margin-top: 12px; }
|
||||
.pager a { color: inherit; }
|
||||
.pager .off { opacity: .35; pointer-events: none; }
|
||||
.warn { margin-top: 18px; opacity: .6; font-size: 12px; }
|
||||
</style></head><body>
|
||||
<h1>activity_event</h1>
|
||||
<div class="sub">${esc(DB_PATH)} · read-only · ${total} row${total === 1 ? '' : 's'}</div>
|
||||
<form class="bar" method="get" action="/">
|
||||
<input type="search" name="q" value="${esc(q)}" placeholder="filter any column, including the JSON data blob" autofocus>
|
||||
<input type="hidden" name="sort" value="${esc(sort)}">
|
||||
<input type="hidden" name="dir" value="${esc(dir.toLowerCase())}">
|
||||
<select name="limit">${PAGE_SIZES.map((n) => `<option value="${n}"${n === limit ? ' selected' : ''}>${n} / page</option>`).join('')}</select>
|
||||
<button type="submit">Filter</button>
|
||||
${q ? '<a href="/">clear</a>' : ''}
|
||||
</form>
|
||||
<table><thead><tr>${header}</tr></thead><tbody>${body}</tbody></table>
|
||||
<div class="pager">
|
||||
<a class="${offset === 0 ? 'off' : ''}" href="${esc(link({ offset: '0' }))}">« first</a>
|
||||
<a class="${offset === 0 ? 'off' : ''}" href="${esc(link({ offset: String(prev) }))}">‹ prev</a>
|
||||
<span>${from}–${to} of ${total}</span>
|
||||
<a class="${next >= total ? 'off' : ''}" href="${esc(link({ offset: String(next) }))}">next ›</a>
|
||||
</div>
|
||||
<div class="warn">Debug surface: unauthenticated, serves the whole table. Loopback only: do not tunnel or port-forward.</div>
|
||||
</body></html>`;
|
||||
}
|
||||
|
||||
const server = http.createServer((req, res) => {
|
||||
// The socket is loopback, but a hostname resolving to 127.0.0.1 would still be
|
||||
// served. Requiring a loopback Host closes that. Nothing else authenticates.
|
||||
if (!isLoopbackHost(req.headers.host)) {
|
||||
res.writeHead(403, { 'content-type': 'text/plain' });
|
||||
return res.end('Loopback host required.\n');
|
||||
}
|
||||
if (req.method !== 'GET') {
|
||||
res.writeHead(405, { 'content-type': 'text/plain', allow: 'GET' });
|
||||
return res.end('Read-only.\n');
|
||||
}
|
||||
|
||||
const url = new URL(req.url, `http://${HOST}:${PORT}`);
|
||||
if (url.pathname === '/favicon.ico') {
|
||||
res.writeHead(204);
|
||||
return res.end();
|
||||
}
|
||||
if (url.pathname !== '/') {
|
||||
res.writeHead(404, { 'content-type': 'text/plain' });
|
||||
return res.end('Not found.\n');
|
||||
}
|
||||
|
||||
const q = (url.searchParams.get('q') ?? '').slice(0, 200);
|
||||
const askedSort = url.searchParams.get('sort') ?? 'id';
|
||||
const sort = SORTABLE.has(askedSort) ? askedSort : 'id';
|
||||
const dir = (url.searchParams.get('dir') ?? 'desc').toLowerCase() === 'asc' ? 'ASC' : 'DESC';
|
||||
const askedLimit = Number(url.searchParams.get('limit'));
|
||||
const limit = PAGE_SIZES.includes(askedLimit) ? askedLimit : 50;
|
||||
const offset = Math.max(0, Number(url.searchParams.get('offset')) || 0);
|
||||
|
||||
try {
|
||||
const { total, rows } = query({ q, sort, dir, limit, offset });
|
||||
res.writeHead(200, {
|
||||
'content-type': 'text/html; charset=utf-8',
|
||||
'cache-control': 'no-store',
|
||||
'x-frame-options': 'DENY',
|
||||
});
|
||||
res.end(page({ q, sort, dir, limit, offset, total, rows }));
|
||||
} catch (e) {
|
||||
res.writeHead(500, { 'content-type': 'text/plain' });
|
||||
res.end(`Query failed: ${e.message}\n`);
|
||||
}
|
||||
});
|
||||
|
||||
// Only serve when run as a script. Importing this module gives you `isLoopbackHost`
|
||||
// and the server without either touching the database or taking the port.
|
||||
if (process.argv[1] && fileURLToPath(import.meta.url) === realpathSync(process.argv[1])) {
|
||||
open(DB_PATH);
|
||||
server.listen(PORT, HOST, () => {
|
||||
console.log(`activity_event viewer http://${HOST}:${PORT}`);
|
||||
console.log(`database ${DB_PATH} (read-only)`);
|
||||
console.log(`columns ${COLUMNS.join(', ')}`);
|
||||
console.log('Loopback only, unauthenticated. Ctrl-C to stop.');
|
||||
});
|
||||
}
|
||||
@@ -0,0 +1,193 @@
|
||||
import assert from 'node:assert/strict';
|
||||
import { spawn } from 'node:child_process';
|
||||
import http from 'node:http';
|
||||
import { mkdtempSync } from 'node:fs';
|
||||
import { tmpdir } from 'node:os';
|
||||
import { join, dirname } from 'node:path';
|
||||
import { DatabaseSync } from 'node:sqlite';
|
||||
import { after, before, describe, it } from 'node:test';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
|
||||
import { isLoopbackHost } from './inspectActivity.mjs';
|
||||
|
||||
const SCRIPT = join(dirname(fileURLToPath(import.meta.url)), 'inspectActivity.mjs');
|
||||
const PORT = 5788;
|
||||
const BASE = `http://127.0.0.1:${PORT}`;
|
||||
|
||||
/** A database with the columns the viewer reads, and two rows to render. */
|
||||
function seedDb() {
|
||||
const file = join(mkdtempSync(join(tmpdir(), 'inspect-activity-')), 'database.sqlite');
|
||||
const db = new DatabaseSync(file);
|
||||
db.exec(`CREATE TABLE activity_event (
|
||||
id integer PRIMARY KEY NOT NULL,
|
||||
category varchar(32) NOT NULL,
|
||||
action varchar(64) NOT NULL,
|
||||
typeVersion integer NOT NULL DEFAULT 1,
|
||||
userId varchar,
|
||||
projectId varchar(36),
|
||||
resourceType varchar(32),
|
||||
resourceId varchar(36),
|
||||
resourceName text,
|
||||
data text,
|
||||
createdAt datetime(3) NOT NULL
|
||||
)`);
|
||||
const insert = db.prepare(`INSERT INTO activity_event
|
||||
(category, action, userId, projectId, resourceType, resourceId, resourceName, data, createdAt)
|
||||
VALUES (?, ?, 'u1', 'p1', ?, ?, ?, ?, ?)`);
|
||||
insert.run(
|
||||
'workflow',
|
||||
'created',
|
||||
'workflow',
|
||||
'w1',
|
||||
'[seed] Alpha',
|
||||
'{"nodeCount":6}',
|
||||
'2026-01-01 00:00:00.000',
|
||||
);
|
||||
insert.run(
|
||||
'credential',
|
||||
'created',
|
||||
'credential',
|
||||
'c1',
|
||||
'[seed] 100% Cover',
|
||||
null,
|
||||
'2026-01-02 00:00:00.000',
|
||||
);
|
||||
db.close();
|
||||
return file;
|
||||
}
|
||||
|
||||
/**
|
||||
* `fetch` treats Host as a forbidden header and drops an override silently, which
|
||||
* makes the loopback guard untestable through it. node:http sends what it is given.
|
||||
*/
|
||||
function request(requestPath, { method = 'GET', headers = {} } = {}) {
|
||||
return new Promise((resolve, reject) => {
|
||||
const req = http.request(
|
||||
{ host: '127.0.0.1', port: PORT, path: requestPath, method, headers },
|
||||
(res) => {
|
||||
let body = '';
|
||||
res.on('data', (c) => (body += c));
|
||||
res.on('end', () => resolve({ status: res.statusCode, headers: res.headers, body }));
|
||||
},
|
||||
);
|
||||
req.on('error', reject);
|
||||
req.end();
|
||||
});
|
||||
}
|
||||
|
||||
const get = (requestPath, headers = {}) => request(requestPath, { headers });
|
||||
|
||||
describe('isLoopbackHost', () => {
|
||||
it('accepts loopback authorities, with or without a port', () => {
|
||||
for (const host of [
|
||||
'127.0.0.1',
|
||||
'127.0.0.1:5699',
|
||||
'localhost',
|
||||
'LOCALHOST:80',
|
||||
'[::1]',
|
||||
'[::1]:5699',
|
||||
'[0:0:0:0:0:0:0:1]',
|
||||
]) {
|
||||
assert.equal(isLoopbackHost(host), true, `should accept ${host}`);
|
||||
}
|
||||
});
|
||||
|
||||
// A prefix match on the bracket accepted `[::1]evil`, so the guard passed a
|
||||
// rebinding host straight through to the handler.
|
||||
it('rejects hosts that only start with a loopback authority', () => {
|
||||
for (const host of [
|
||||
'[::1]evil',
|
||||
'[::1]evil.com:80',
|
||||
'[::1]@evil.com',
|
||||
'127.0.0.1.evil.com',
|
||||
'[email protected]',
|
||||
'localhost.evil.com',
|
||||
]) {
|
||||
assert.equal(isLoopbackHost(host), false, `should reject ${host}`);
|
||||
}
|
||||
});
|
||||
|
||||
it('rejects anything else', () => {
|
||||
for (const host of ['evil.com', '[::2]', '[::1]:abc', 'localhost:1:2', '', undefined]) {
|
||||
assert.equal(isLoopbackHost(host), false, `should reject ${JSON.stringify(host)}`);
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
describe('inspectActivity server', () => {
|
||||
let child;
|
||||
let dbFile;
|
||||
|
||||
before(async () => {
|
||||
dbFile = seedDb();
|
||||
child = spawn(process.execPath, [SCRIPT], {
|
||||
env: { ...process.env, DB_SQLITE_DATABASE: dbFile, PORT: String(PORT) },
|
||||
stdio: 'ignore',
|
||||
});
|
||||
for (let i = 0; i < 60; i++) {
|
||||
try {
|
||||
await get('/');
|
||||
return;
|
||||
} catch {
|
||||
await new Promise((r) => setTimeout(r, 100));
|
||||
}
|
||||
}
|
||||
throw new Error('viewer did not start');
|
||||
});
|
||||
|
||||
after(() => child?.kill());
|
||||
|
||||
it('serves the table on a loopback host', async () => {
|
||||
const { status, body } = await get('/');
|
||||
assert.equal(status, 200);
|
||||
assert.match(body, /activity_event/);
|
||||
assert.match(body, /2 rows/);
|
||||
assert.match(body, /\[seed\] Alpha/);
|
||||
});
|
||||
|
||||
it('rejects a non-loopback Host with 403', async () => {
|
||||
for (const host of ['evil.example.com', '[::1]evil.com']) {
|
||||
const { status } = await get('/', { host });
|
||||
assert.equal(status, 403, `expected 403 for Host: ${host}`);
|
||||
}
|
||||
});
|
||||
|
||||
it('rejects every non-GET method with 405', async () => {
|
||||
for (const method of ['POST', 'PUT', 'PATCH', 'DELETE']) {
|
||||
const { status, headers } = await request('/', { method });
|
||||
assert.equal(status, 405);
|
||||
assert.equal(headers.allow, 'GET');
|
||||
}
|
||||
});
|
||||
|
||||
it('opens the database read-only', () => {
|
||||
const ro = new DatabaseSync(dbFile, { readOnly: true });
|
||||
assert.throws(() => ro.exec('DELETE FROM activity_event'), /readonly/i);
|
||||
ro.close();
|
||||
});
|
||||
|
||||
it('filters on the JSON data column', async () => {
|
||||
const { body } = await get('/?q=nodeCount');
|
||||
assert.match(body, /1 rows?/);
|
||||
assert.match(body, /\[seed\] Alpha/);
|
||||
});
|
||||
|
||||
it('treats LIKE wildcards in the filter as literals', async () => {
|
||||
// `%` unescaped matches every row. Only the row containing it should match.
|
||||
const { body } = await get('/?q=%25');
|
||||
assert.match(body, /1 rows?/);
|
||||
assert.match(body, /100% Cover/);
|
||||
});
|
||||
|
||||
it('falls back to a safe sort column instead of interpolating one', async () => {
|
||||
const { status } = await get('/?sort=id%3BDROP+TABLE+activity_event--');
|
||||
assert.equal(status, 200);
|
||||
const check = new DatabaseSync(dbFile, { readOnly: true });
|
||||
assert.equal(check.prepare('SELECT COUNT(*) AS n FROM activity_event').get().n, 2);
|
||||
check.close();
|
||||
});
|
||||
|
||||
it('404s an unknown path', async () => {
|
||||
assert.equal((await get('/secrets')).status, 404);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,512 @@
|
||||
// Ten hand-written workflows that share one house style.
|
||||
//
|
||||
// Written out rather than generated. The point is that "what does this org normally
|
||||
// do?" has a checkable answer, which needs the answer to be a fact about the estate
|
||||
// and not an artifact of sampling. Names describe what the nodes do, so a reader
|
||||
// does not learn something false from the title.
|
||||
|
||||
// Five rules, no exceptions: at n=10 an exception is indistinguishable from noise.
|
||||
// Each rule is a place n8n offers a real choice. A rule with no alternative is a
|
||||
// constraint, not a preference, and detecting it proves nothing.
|
||||
export const HOUSE_STYLE = {
|
||||
// Alternatives: Anthropic, Gemini, Mistral, Ollama, Azure OpenAI.
|
||||
chatModel: '@n8n/n8n-nodes-langchain.lmChatOpenAi',
|
||||
chatModelName: 'gpt-4o-mini',
|
||||
// Alternatives: Jira, GitHub Issues, Asana, Trello, ClickUp.
|
||||
tracker: 'n8n-nodes-base.linear',
|
||||
// Alternatives: Discord, Teams, Telegram, email.
|
||||
notifier: 'n8n-nodes-base.slack',
|
||||
// Alternatives: the `interval` rule form, which the estate profile uses.
|
||||
scheduleStyle: 'cronExpression',
|
||||
// Every workflow's last node writes one row to `automation_runs`.
|
||||
auditTable: 'automation_runs',
|
||||
};
|
||||
|
||||
export const PREFERENCE_PROJECT = 'Automation Platform';
|
||||
|
||||
// Read and written by the workflows, not decoration. Three read `customer_accounts`;
|
||||
// all ten write `automation_runs`, which puts the audit convention in the data too.
|
||||
export const PREFERENCE_DATA_TABLES = [
|
||||
{
|
||||
name: 'customer_accounts',
|
||||
columns: [
|
||||
{ name: 'account_id', type: 'string' },
|
||||
{ name: 'company', type: 'string' },
|
||||
{ name: 'plan', type: 'string' },
|
||||
{ name: 'health_score', type: 'number' },
|
||||
{ name: 'renewal_date', type: 'string' },
|
||||
{ name: 'invoice_overdue_days', type: 'number' },
|
||||
],
|
||||
// Tabular, in column order, because six objects with six keys each is 48 lines
|
||||
// of the same shape and harder to scan for the values that matter.
|
||||
rows: [
|
||||
['acc_1001', 'Northwind Trading', 'enterprise', 82, '2026-03-01', 0],
|
||||
['acc_1002', 'Contoso Logistics', 'growth', 41, '2026-02-14', 12],
|
||||
['acc_1003', 'Fabrikam Health', 'enterprise', 67, '2026-05-20', 0],
|
||||
['acc_1004', 'Tailspin Media', 'starter', 29, '2026-01-30', 34],
|
||||
['acc_1005', 'Proseware Analytics', 'growth', 74, '2026-04-11', 0],
|
||||
['acc_1006', 'Litware Robotics', 'enterprise', 55, '2026-02-28', 7],
|
||||
].map(([account_id, company, plan, health_score, renewal_date, invoice_overdue_days]) => ({
|
||||
account_id,
|
||||
company,
|
||||
plan,
|
||||
health_score,
|
||||
renewal_date,
|
||||
invoice_overdue_days,
|
||||
})),
|
||||
},
|
||||
{
|
||||
name: 'automation_runs',
|
||||
columns: [
|
||||
{ name: 'workflow', type: 'string' },
|
||||
{ name: 'outcome', type: 'string' },
|
||||
{ name: 'detail', type: 'string' },
|
||||
{ name: 'ran_at', type: 'string' },
|
||||
],
|
||||
rows: [],
|
||||
},
|
||||
];
|
||||
|
||||
// `env` names the variable that supplies a real token. Without it the seeder writes a
|
||||
// labelled placeholder, so the workflow stays wired and openable but is not runnable.
|
||||
export const PREFERENCE_CREDENTIALS = [
|
||||
['openai', 'openAiApi', 'OpenAI', 'SEED_OPENAI_API_KEY', 'apiKey'],
|
||||
['linear', 'linearApi', 'Linear', 'SEED_LINEAR_API_KEY', 'apiKey'],
|
||||
['slack', 'slackApi', 'Slack', 'SEED_SLACK_TOKEN', 'accessToken'],
|
||||
['gmail', 'gmailOAuth2', 'Gmail', 'SEED_GMAIL_OAUTH', 'oauthTokenData'],
|
||||
['http', 'httpHeaderAuth', 'Enrichment API', 'SEED_ENRICHMENT_TOKEN', 'value'],
|
||||
].map(([key, type, label, env, field]) => ({
|
||||
key,
|
||||
type,
|
||||
name: `${label} (${PREFERENCE_PROJECT})`,
|
||||
env,
|
||||
field,
|
||||
}));
|
||||
|
||||
const LINEAR_TEAM_ID = process.env.SEED_LINEAR_TEAM_ID || 'seed-placeholder-team-id';
|
||||
|
||||
// Small helpers only. Each workflow spells out its own graph, so "is this a real
|
||||
// workflow?" is answerable from the recipe without chasing factories.
|
||||
|
||||
let x = 0;
|
||||
const at = (row = 0) => [240 + x++ * 220, 300 + row * 160];
|
||||
const resetPos = () => {
|
||||
x = 0;
|
||||
};
|
||||
|
||||
const scheduleTrigger = (cron) => ({
|
||||
parameters: { rule: { interval: [{ field: 'cronExpression', expression: cron }] } },
|
||||
id: undefined,
|
||||
name: 'Schedule Trigger',
|
||||
type: 'n8n-nodes-base.scheduleTrigger',
|
||||
typeVersion: 1.4,
|
||||
position: at(),
|
||||
});
|
||||
|
||||
const webhookTrigger = (path) => ({
|
||||
parameters: { httpMethod: 'POST', path, responseMode: 'lastNode', options: {} },
|
||||
id: undefined,
|
||||
name: 'Webhook',
|
||||
type: 'n8n-nodes-base.webhook',
|
||||
typeVersion: 2,
|
||||
position: at(),
|
||||
});
|
||||
|
||||
const chatModel = () => ({
|
||||
parameters: {
|
||||
model: { __rl: true, value: HOUSE_STYLE.chatModelName, mode: 'list' },
|
||||
options: {},
|
||||
},
|
||||
id: undefined,
|
||||
name: 'OpenAI Chat Model',
|
||||
type: HOUSE_STYLE.chatModel,
|
||||
typeVersion: 1.3,
|
||||
position: [0, 0], // repositioned by the caller, sub-nodes sit below their root
|
||||
credentials: { openai: true },
|
||||
});
|
||||
|
||||
const agent = (name, prompt) => ({
|
||||
parameters: {
|
||||
promptType: 'define',
|
||||
text: prompt,
|
||||
options: {},
|
||||
},
|
||||
id: undefined,
|
||||
name,
|
||||
type: '@n8n/n8n-nodes-langchain.agent',
|
||||
typeVersion: 3.1,
|
||||
position: at(),
|
||||
});
|
||||
|
||||
const linearCreate = (name, title, description) => ({
|
||||
parameters: {
|
||||
resource: 'issue',
|
||||
operation: 'create',
|
||||
teamId: LINEAR_TEAM_ID,
|
||||
title,
|
||||
additionalFields: { description },
|
||||
},
|
||||
id: undefined,
|
||||
name,
|
||||
type: HOUSE_STYLE.tracker,
|
||||
typeVersion: 1.1,
|
||||
position: at(),
|
||||
credentials: { linear: true },
|
||||
});
|
||||
|
||||
const linearGetAll = (name) => ({
|
||||
parameters: { resource: 'issue', operation: 'getAll', returnAll: false, limit: 50 },
|
||||
id: undefined,
|
||||
name,
|
||||
type: HOUSE_STYLE.tracker,
|
||||
typeVersion: 1.1,
|
||||
position: at(),
|
||||
credentials: { linear: true },
|
||||
});
|
||||
|
||||
const slackPost = (name, channel, text) => ({
|
||||
parameters: {
|
||||
resource: 'message',
|
||||
operation: 'post',
|
||||
select: 'channel',
|
||||
channelId: { __rl: true, value: channel, mode: 'name' },
|
||||
text,
|
||||
otherOptions: {},
|
||||
},
|
||||
id: undefined,
|
||||
name,
|
||||
type: HOUSE_STYLE.notifier,
|
||||
typeVersion: 2.7,
|
||||
position: at(),
|
||||
credentials: { slack: true },
|
||||
});
|
||||
|
||||
const dataTableGet = (name, tableId, filters) => ({
|
||||
parameters: {
|
||||
resource: 'row',
|
||||
operation: 'get',
|
||||
dataTableId: { __rl: true, value: tableId, mode: 'id' },
|
||||
filters: filters ?? { conditions: [] },
|
||||
options: {},
|
||||
},
|
||||
id: undefined,
|
||||
name,
|
||||
type: 'n8n-nodes-base.dataTable',
|
||||
typeVersion: 1.1,
|
||||
position: at(),
|
||||
});
|
||||
|
||||
const auditRow = (tableId, workflowName, detail) => ({
|
||||
parameters: {
|
||||
resource: 'row',
|
||||
operation: 'insert',
|
||||
dataTableId: { __rl: true, value: tableId, mode: 'id' },
|
||||
columns: {
|
||||
mappingMode: 'defineBelow',
|
||||
value: {
|
||||
workflow: workflowName,
|
||||
outcome: 'success',
|
||||
detail,
|
||||
ran_at: '={{ $now.toISO() }}',
|
||||
},
|
||||
},
|
||||
options: {},
|
||||
},
|
||||
id: undefined,
|
||||
name: 'Record Run',
|
||||
type: 'n8n-nodes-base.dataTable',
|
||||
typeVersion: 1.1,
|
||||
position: at(),
|
||||
});
|
||||
|
||||
const ifNode = (name, leftValue, operator, rightValue) => ({
|
||||
parameters: {
|
||||
conditions: {
|
||||
options: { caseSensitive: true, leftValue: '', typeValidation: 'strict', version: 2 },
|
||||
conditions: [
|
||||
{
|
||||
id: 'cond-1',
|
||||
leftValue,
|
||||
rightValue,
|
||||
operator,
|
||||
},
|
||||
],
|
||||
combinator: 'and',
|
||||
},
|
||||
options: {},
|
||||
},
|
||||
id: undefined,
|
||||
name,
|
||||
type: 'n8n-nodes-base.if',
|
||||
typeVersion: 2.3,
|
||||
position: at(),
|
||||
});
|
||||
|
||||
const numberGt = { type: 'number', operation: 'gt' };
|
||||
const stringEquals = { type: 'string', operation: 'equals' };
|
||||
|
||||
// --- the ten workflows -------------------------------------------------------
|
||||
// `tables` maps a data-table name to the id the seeder created, so the recipes can
|
||||
// reference real tables rather than placeholders.
|
||||
|
||||
export function preferenceWorkflows(tables) {
|
||||
const accounts = tables.customer_accounts;
|
||||
const runs = tables.automation_runs;
|
||||
|
||||
/** Chain nodes left to right on `main`, and hang the chat model off its agent. */
|
||||
const build = (name, nodes, { modelFor } = {}) => {
|
||||
const connections = {};
|
||||
const main = nodes.filter((n) => n.name !== 'OpenAI Chat Model');
|
||||
for (let i = 0; i < main.length - 1; i++) {
|
||||
connections[main[i].name] = {
|
||||
main: [[{ node: main[i + 1].name, type: 'main', index: 0 }]],
|
||||
};
|
||||
}
|
||||
if (modelFor) {
|
||||
const model = nodes.find((n) => n.name === 'OpenAI Chat Model');
|
||||
const root = main.find((n) => n.name === modelFor);
|
||||
model.position = [root.position[0], root.position[1] + 200];
|
||||
connections['OpenAI Chat Model'] = {
|
||||
ai_languageModel: [[{ node: modelFor, type: 'ai_languageModel', index: 0 }]],
|
||||
};
|
||||
}
|
||||
resetPos();
|
||||
return { name, nodes, connections };
|
||||
};
|
||||
|
||||
return [
|
||||
build(
|
||||
'Support Inbox Triage',
|
||||
[
|
||||
scheduleTrigger('*/15 * * * *'),
|
||||
{
|
||||
parameters: {
|
||||
resource: 'message',
|
||||
operation: 'getAll',
|
||||
returnAll: false,
|
||||
limit: 25,
|
||||
filters: { readStatus: 'unread' },
|
||||
},
|
||||
id: undefined,
|
||||
name: 'Fetch Unread Support Mail',
|
||||
type: 'n8n-nodes-base.gmail',
|
||||
typeVersion: 2.2,
|
||||
position: at(),
|
||||
credentials: { gmail: true },
|
||||
},
|
||||
agent(
|
||||
'Classify Ticket',
|
||||
'Read the support email below and reply with one line: "<severity> | <one-sentence summary>". Severity must be one of P1, P2, P3.\n\nSubject: {{ $json.subject }}\nBody: {{ $json.snippet }}',
|
||||
),
|
||||
chatModel(),
|
||||
linearCreate(
|
||||
'Open Support Issue',
|
||||
'={{ $json.output.split("|")[1].trim() }}',
|
||||
'=Raised by Support Inbox Triage.\n\nSeverity: {{ $json.output.split("|")[0].trim() }}',
|
||||
),
|
||||
slackPost(
|
||||
'Notify Support Channel',
|
||||
'#support-triage',
|
||||
'=New ticket triaged: {{ $json.output }}',
|
||||
),
|
||||
auditRow(runs, 'Support Inbox Triage', '=Triaged {{ $json.title }}'),
|
||||
],
|
||||
{ modelFor: 'Classify Ticket' },
|
||||
),
|
||||
|
||||
build(
|
||||
'Churn Risk Digest',
|
||||
[
|
||||
scheduleTrigger('0 8 * * 1-5'),
|
||||
dataTableGet('Read Accounts', accounts, {
|
||||
conditions: [{ keyName: 'health_score', condition: 'lt', keyValue: 50 }],
|
||||
}),
|
||||
agent(
|
||||
'Summarise Churn Risk',
|
||||
'These accounts have a health score below 50. Write a three-bullet summary for the customer success team, naming the accounts and the single clearest risk for each.\n\n{{ JSON.stringify($json) }}',
|
||||
),
|
||||
chatModel(),
|
||||
slackPost('Post Digest', '#cs-alerts', '=Daily churn risk digest:\n{{ $json.output }}'),
|
||||
auditRow(runs, 'Churn Risk Digest', 'Digest posted to #cs-alerts'),
|
||||
],
|
||||
{ modelFor: 'Summarise Churn Risk' },
|
||||
),
|
||||
|
||||
build(
|
||||
'Incident Intake',
|
||||
[
|
||||
webhookTrigger('incident-intake'),
|
||||
agent(
|
||||
'Assess Severity',
|
||||
'An incident was reported. Reply with only P1, P2 or P3.\n\nTitle: {{ $json.body.title }}\nDescription: {{ $json.body.description }}',
|
||||
),
|
||||
chatModel(),
|
||||
ifNode('Is P1?', '={{ $json.output.trim() }}', stringEquals, 'P1'),
|
||||
linearCreate(
|
||||
'Raise Incident',
|
||||
'=[{{ $json.output.trim() }}] {{ $json.body.title }}',
|
||||
'={{ $json.body.description }}',
|
||||
),
|
||||
slackPost('Page On-Call', '#incidents', '=P1 incident raised: {{ $json.title }}'),
|
||||
auditRow(runs, 'Incident Intake', '=Incident {{ $json.title }} recorded'),
|
||||
],
|
||||
{ modelFor: 'Assess Severity' },
|
||||
),
|
||||
|
||||
build('Lead Enrichment', [
|
||||
webhookTrigger('lead-capture'),
|
||||
{
|
||||
parameters: {
|
||||
method: 'GET',
|
||||
url: '=https://api.clearbit.com/v2/companies/find?domain={{ $json.body.domain }}',
|
||||
authentication: 'genericCredentialType',
|
||||
genericAuthType: 'httpHeaderAuth',
|
||||
options: {},
|
||||
},
|
||||
id: undefined,
|
||||
name: 'Enrich Company',
|
||||
type: 'n8n-nodes-base.httpRequest',
|
||||
typeVersion: 4.2,
|
||||
position: at(),
|
||||
credentials: { http: true },
|
||||
},
|
||||
slackPost(
|
||||
'Notify Sales',
|
||||
'#sales-inbound',
|
||||
'=New enriched lead: {{ $json.name }} ({{ $json.metrics.employees }} employees)',
|
||||
),
|
||||
auditRow(runs, 'Lead Enrichment', '=Enriched {{ $json.name }}'),
|
||||
]),
|
||||
|
||||
build(
|
||||
'Weekly Engineering Digest',
|
||||
[
|
||||
scheduleTrigger('0 9 * * 1'),
|
||||
linearGetAll('Fetch Open Issues'),
|
||||
agent(
|
||||
'Write Digest',
|
||||
'Summarise this week of Linear issues for an engineering all-hands. Group by theme, keep it under 150 words.\n\n{{ JSON.stringify($json) }}',
|
||||
),
|
||||
chatModel(),
|
||||
slackPost('Post To Engineering', '#engineering', '=Weekly digest:\n{{ $json.output }}'),
|
||||
auditRow(runs, 'Weekly Engineering Digest', 'Digest posted to #engineering'),
|
||||
],
|
||||
{ modelFor: 'Write Digest' },
|
||||
),
|
||||
|
||||
build('Invoice Dunning', [
|
||||
scheduleTrigger('0 10 * * *'),
|
||||
dataTableGet('Read Overdue Accounts', accounts, {
|
||||
conditions: [{ keyName: 'invoice_overdue_days', condition: 'gt', keyValue: 5 }],
|
||||
}),
|
||||
ifNode('Overdue Past Grace?', '={{ $json.invoice_overdue_days }}', numberGt, 30),
|
||||
{
|
||||
parameters: {
|
||||
resource: 'message',
|
||||
operation: 'send',
|
||||
sendTo: '=billing@{{ $json.company.toLowerCase().replaceAll(" ", "") }}.com',
|
||||
subject: '=Invoice overdue: {{ $json.invoice_overdue_days }} days',
|
||||
message:
|
||||
'=Hello {{ $json.company }},\n\nOur records show an invoice outstanding for {{ $json.invoice_overdue_days }} days. Please let us know if you need a copy.\n\nBilling team',
|
||||
options: {},
|
||||
},
|
||||
id: undefined,
|
||||
name: 'Send Dunning Email',
|
||||
type: 'n8n-nodes-base.gmail',
|
||||
typeVersion: 2.2,
|
||||
position: at(),
|
||||
credentials: { gmail: true },
|
||||
},
|
||||
slackPost('Flag To Finance', '#finance-ops', '=Dunning email sent to {{ $json.company }}'),
|
||||
auditRow(runs, 'Invoice Dunning', '=Chased {{ $json.company }}'),
|
||||
]),
|
||||
|
||||
build(
|
||||
'Product Feedback Classifier',
|
||||
[
|
||||
webhookTrigger('product-feedback'),
|
||||
agent(
|
||||
'Categorise Feedback',
|
||||
'Classify this product feedback into exactly one of: bug, feature-request, praise, confusion. Reply with the label only.\n\n{{ $json.body.message }}',
|
||||
),
|
||||
chatModel(),
|
||||
linearCreate(
|
||||
'Log Feedback',
|
||||
'=[{{ $json.output.trim() }}] {{ $json.body.message.slice(0, 60) }}',
|
||||
'={{ $json.body.message }}\n\nFrom: {{ $json.body.email }}',
|
||||
),
|
||||
slackPost(
|
||||
'Share With Product',
|
||||
'#product-feedback',
|
||||
'=New {{ $json.output.trim() }} logged: {{ $json.title }}',
|
||||
),
|
||||
auditRow(runs, 'Product Feedback Classifier', '=Classified as {{ $json.output.trim() }}'),
|
||||
],
|
||||
{ modelFor: 'Categorise Feedback' },
|
||||
),
|
||||
|
||||
build('Onboarding Kickoff', [
|
||||
webhookTrigger('onboarding-start'),
|
||||
linearCreate(
|
||||
'Create Onboarding Issue',
|
||||
'=Onboard {{ $json.body.company }}',
|
||||
'=New {{ $json.body.plan }} customer.\n\nPrimary contact: {{ $json.body.contact_email }}\nKickoff call: {{ $json.body.kickoff_date }}',
|
||||
),
|
||||
slackPost(
|
||||
'Announce New Customer',
|
||||
'#customer-success',
|
||||
'=Onboarding started for {{ $json.body.company }} ({{ $json.body.plan }})',
|
||||
),
|
||||
auditRow(runs, 'Onboarding Kickoff', '=Onboarding opened for {{ $json.body.company }}'),
|
||||
]),
|
||||
|
||||
build('Deploy Announcer', [
|
||||
webhookTrigger('deploy-complete'),
|
||||
{
|
||||
parameters: {
|
||||
assignments: {
|
||||
assignments: [
|
||||
{ id: 'a1', name: 'release', value: '={{ $json.body.tag }}', type: 'string' },
|
||||
{ id: 'a2', name: 'author', value: '={{ $json.body.author }}', type: 'string' },
|
||||
{
|
||||
id: 'a3',
|
||||
name: 'services',
|
||||
value: '={{ $json.body.services.join(", ") }}',
|
||||
type: 'string',
|
||||
},
|
||||
],
|
||||
},
|
||||
options: {},
|
||||
},
|
||||
id: undefined,
|
||||
name: 'Format Release Note',
|
||||
type: 'n8n-nodes-base.set',
|
||||
typeVersion: 3.4,
|
||||
position: at(),
|
||||
},
|
||||
slackPost(
|
||||
'Announce Release',
|
||||
'#releases',
|
||||
'=Deployed {{ $json.release }} by {{ $json.author }}. Services: {{ $json.services }}',
|
||||
),
|
||||
auditRow(runs, 'Deploy Announcer', '=Announced {{ $json.release }}'),
|
||||
]),
|
||||
|
||||
build('Stale Issue Sweeper', [
|
||||
scheduleTrigger('0 7 * * 3'),
|
||||
linearGetAll('Fetch Issues'),
|
||||
ifNode(
|
||||
'Untouched Over 30 Days?',
|
||||
'={{ $now.diff($fromISO($json.updatedAt), "days").days }}',
|
||||
numberGt,
|
||||
30,
|
||||
),
|
||||
slackPost(
|
||||
'Nudge Owners',
|
||||
'#engineering',
|
||||
'=Stale issue needs an update: {{ $json.title }} (last touched {{ $json.updatedAt }})',
|
||||
),
|
||||
auditRow(runs, 'Stale Issue Sweeper', '=Flagged {{ $json.title }}'),
|
||||
]),
|
||||
];
|
||||
}
|
||||
@@ -0,0 +1,421 @@
|
||||
#!/usr/bin/env node
|
||||
// Write a fortnight of history for a seeded instance: executions, assistant threads,
|
||||
// and activity entries. Writes SQLite directly, because the public API has no create
|
||||
// route for any of them, and because backdating needs `startedAt` set by hand.
|
||||
//
|
||||
// Run after `seed:account`. A live instance is fine, but prefer an idle one: SQLite
|
||||
// serialises writers, so a busy instance can hold the write lock long enough to fail.
|
||||
|
||||
import { DatabaseSync } from 'node:sqlite';
|
||||
import path from 'node:path';
|
||||
import os from 'node:os';
|
||||
import { createRequire } from 'node:module';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
|
||||
const require = createRequire(import.meta.url);
|
||||
const REPO = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '../..');
|
||||
// `flatted` is how n8n serialises run data. The format has cycles, so plain JSON
|
||||
// would not round-trip. Resolved from the cli package rather than vendored.
|
||||
const { stringify } = require(
|
||||
require.resolve('flatted', { paths: [path.join(REPO, 'packages/cli')] }),
|
||||
);
|
||||
|
||||
// Mirrors `getN8nFolder()` in @n8n/config: the `.n8n` directory sits *inside*
|
||||
// N8N_USER_FOLDER, so joining the db name straight onto that variable looks one
|
||||
// directory too high.
|
||||
const DB_PATH =
|
||||
process.env.DB_SQLITE_DATABASE ??
|
||||
path.join(process.env.N8N_USER_FOLDER ?? os.homedir(), '.n8n', 'database.sqlite');
|
||||
|
||||
const SEED_PREFIX = '[seed] ';
|
||||
// Threads have no name to prefix, so they carry this in `metadata` instead. It is how
|
||||
// the clear step tells its own rows from a developer's real assistant threads.
|
||||
const THREAD_MARKER = '"seeded":true';
|
||||
const DAYS = Number(process.env.HISTORY_DAYS) || 14;
|
||||
|
||||
// Fixed seed and fixed clock, for the same reason the estate is fixed: an A/B eval
|
||||
// compares two arms against one instance, which requires the instance not to move.
|
||||
function makeRandom(seed) {
|
||||
let a = seed >>> 0;
|
||||
return () => {
|
||||
a = (a + 0x6d2b79f5) >>> 0;
|
||||
let t = a;
|
||||
t = Math.imul(t ^ (t >>> 15), t | 1);
|
||||
t ^= t + Math.imul(t ^ (t >>> 7), t | 61);
|
||||
return ((t ^ (t >>> 14)) >>> 0) / 4294967296;
|
||||
};
|
||||
}
|
||||
const random = makeRandom(Number(process.env.SEED) || 1);
|
||||
|
||||
// The window has to end at the current time. n8n prunes threads after 30 days and
|
||||
// executions past EXECUTIONS_DATA_MAX_AGE, so a fixed past date gets the whole
|
||||
// fortnight deleted on the next start. Only timestamps move; the PRNG still decides
|
||||
// what happens. HISTORY_NOW pins the window if you accept the pruning.
|
||||
const NOW = process.env.HISTORY_NOW ? Date.parse(process.env.HISTORY_NOW) : Date.now();
|
||||
|
||||
const iso = (ms) => new Date(ms).toISOString().replace('T', ' ').replace('Z', '');
|
||||
const pick = (arr) => arr[Math.floor(random() * arr.length)];
|
||||
const uuid = () => {
|
||||
// Deterministic v4-shaped id. Not cryptographic; these are fixture ids.
|
||||
const hex = '0123456789abcdef';
|
||||
let s = '';
|
||||
for (let i = 0; i < 32; i++) s += hex[Math.floor(random() * 16)];
|
||||
return `${s.slice(0, 8)}-${s.slice(8, 12)}-4${s.slice(13, 16)}-a${s.slice(17, 20)}-${s.slice(20, 32)}`;
|
||||
};
|
||||
|
||||
// One workflow fails its last run and two earlier ones, so "what broke?" has an
|
||||
// answer. A uniformly green history has none.
|
||||
const FAILING_WORKFLOW = 'Invoice Dunning';
|
||||
const FAILURE = {
|
||||
node: 'Send Dunning Email',
|
||||
message: 'Forbidden - perhaps check your credentials?',
|
||||
description: 'The Gmail credential has no valid token. Reconnect the account.',
|
||||
};
|
||||
|
||||
// An error must name a node that exists in its own workflow and say something that
|
||||
// node could say. One shared message produces records that contradict themselves.
|
||||
const NODE_FAILURES = {
|
||||
'n8n-nodes-base.gmail': {
|
||||
message: 'Forbidden - perhaps check your credentials?',
|
||||
description: 'The Gmail credential has no valid token. Reconnect the account.',
|
||||
},
|
||||
'n8n-nodes-base.slack': {
|
||||
message: 'Bad request - please check your parameters',
|
||||
description: 'channel_not_found: the bot is not a member of that channel.',
|
||||
},
|
||||
'n8n-nodes-base.linear': {
|
||||
message: 'Authorization failed - please check your credentials',
|
||||
description: 'The Linear API key was rejected.',
|
||||
},
|
||||
'n8n-nodes-base.httpRequest': {
|
||||
message: 'The service refused the connection - perhaps it is offline',
|
||||
description: 'connect ETIMEDOUT: the enrichment API did not respond.',
|
||||
},
|
||||
'n8n-nodes-base.dataTable': {
|
||||
message: 'Bad request - please check your parameters',
|
||||
description: 'The referenced column does not exist on this data table.',
|
||||
},
|
||||
'@n8n/n8n-nodes-langchain.agent': {
|
||||
message: 'Bad request - please check your parameters',
|
||||
description: 'The model returned no output for this prompt.',
|
||||
},
|
||||
};
|
||||
|
||||
const GENERIC_FAILURE = {
|
||||
message: 'The service was not able to process your request',
|
||||
description: 'The node returned an unexpected response.',
|
||||
};
|
||||
|
||||
/** Pick a node in this workflow that can plausibly fail, and an error to match. */
|
||||
function failureFor(wf, isDesignated) {
|
||||
const candidates = wf.nodes.filter((n) => !n.type.includes('lmChat') && NODE_FAILURES[n.type]);
|
||||
if (isDesignated) {
|
||||
const node = wf.nodes.find((n) => n.name === FAILURE.node);
|
||||
if (node) return { node: FAILURE.node, ...FAILURE };
|
||||
}
|
||||
if (candidates.length === 0) return null;
|
||||
const node = pick(candidates);
|
||||
return { node: node.name, ...(NODE_FAILURES[node.type] ?? GENERIC_FAILURE) };
|
||||
}
|
||||
|
||||
function runData(wf, failure) {
|
||||
const nodes = wf.nodes.filter((n) => !n.type.includes('lmChat'));
|
||||
const runData = {};
|
||||
for (const n of nodes) {
|
||||
// Stop before the failing node: a run that broke never produced output past it.
|
||||
if (failure && n.name === failure.node) break;
|
||||
runData[n.name] = [
|
||||
{
|
||||
hints: [],
|
||||
startTime: NOW,
|
||||
executionTime: 40 + Math.floor(random() * 400),
|
||||
source: [null],
|
||||
executionStatus: 'success',
|
||||
data: { main: [[{ json: { seeded: true }, pairedItem: { item: 0 } }]] },
|
||||
},
|
||||
];
|
||||
}
|
||||
const result = { resultData: { runData, lastNodeExecuted: Object.keys(runData).pop() } };
|
||||
if (failure) {
|
||||
result.resultData.error = {
|
||||
level: 'warning',
|
||||
tags: { packageName: 'nodes-base' },
|
||||
message: failure.message,
|
||||
description: failure.description,
|
||||
node: { name: failure.node },
|
||||
name: 'NodeApiError',
|
||||
};
|
||||
result.resultData.lastNodeExecuted = failure.node;
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
function main() {
|
||||
const db = new DatabaseSync(DB_PATH);
|
||||
db.exec('PRAGMA foreign_keys = ON');
|
||||
|
||||
const workflows = db
|
||||
.prepare('SELECT id, name, nodes FROM workflow_entity WHERE name LIKE ? ORDER BY name')
|
||||
.all(`${SEED_PREFIX}%`)
|
||||
.map((w) => ({ ...w, nodes: JSON.parse(w.nodes) }));
|
||||
|
||||
if (workflows.length === 0) {
|
||||
console.error(`No ${SEED_PREFIX} workflows in ${DB_PATH}. Run seed:account first.`);
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
const project = db
|
||||
.prepare("SELECT id FROM project WHERE name LIKE ? AND type = 'team' LIMIT 1")
|
||||
.get(`${SEED_PREFIX}%`);
|
||||
const user = db.prepare('SELECT id FROM user ORDER BY createdAt LIMIT 1').get();
|
||||
if (!project || !user) {
|
||||
console.error('Could not find the seeded project or an owner user.');
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
console.log(`Database: ${DB_PATH}`);
|
||||
console.log(`Workflows: ${workflows.length}, project ${project.id}`);
|
||||
|
||||
// Clear and rewrite in one transaction. Split apart, a failed insert rolls back the
|
||||
// new history while the deletes stay committed, leaving nothing behind.
|
||||
// Outside the transaction block so the summary after the catch can read them.
|
||||
let execCount = 0;
|
||||
let failCount = 0;
|
||||
let threadCount = 0;
|
||||
let msgCount = 0;
|
||||
let actCount = 0;
|
||||
|
||||
db.exec('BEGIN');
|
||||
try {
|
||||
const wfIds = workflows.map((w) => w.id);
|
||||
const placeholders = wfIds.map(() => '?').join(',');
|
||||
const priorExec = db
|
||||
.prepare(`SELECT id FROM execution_entity WHERE workflowId IN (${placeholders})`)
|
||||
.all(...wfIds)
|
||||
.map((r) => r.id);
|
||||
if (priorExec.length > 0) {
|
||||
const p = priorExec.map(() => '?').join(',');
|
||||
db.prepare(`DELETE FROM execution_data WHERE executionId IN (${p})`).run(...priorExec);
|
||||
db.prepare(`DELETE FROM execution_entity WHERE id IN (${p})`).run(...priorExec);
|
||||
}
|
||||
// Matched on this script's own marker, not on workflow id and not on project
|
||||
// alone. Workflow id fails because re-seeding replaces the workflows with fresh
|
||||
// ids, which orphans the old rows and then collides on the seeded primary keys.
|
||||
// Project alone fails the other way: a developer who opens the assistant on a
|
||||
// seeded workflow, or whose instance writes real activity in that project, would
|
||||
// have that work deleted by the next run. Executions need neither, because
|
||||
// deleting a workflow cascades them.
|
||||
const priorThreads = db
|
||||
.prepare(`SELECT id FROM instance_ai_threads WHERE projectId = ? AND metadata LIKE ?`)
|
||||
.all(project.id, `%${THREAD_MARKER}%`)
|
||||
.map((r) => r.id);
|
||||
if (priorThreads.length > 0) {
|
||||
const p = priorThreads.map(() => '?').join(',');
|
||||
db.prepare(`DELETE FROM instance_ai_messages WHERE threadId IN (${p})`).run(...priorThreads);
|
||||
db.prepare(`DELETE FROM instance_ai_threads WHERE id IN (${p})`).run(...priorThreads);
|
||||
}
|
||||
// `[` is not a LIKE wildcard in SQLite, so the prefix matches literally.
|
||||
const clearedActivity = db
|
||||
.prepare('DELETE FROM activity_event WHERE projectId = ? AND resourceName LIKE ?')
|
||||
.run(project.id, `${SEED_PREFIX}%`);
|
||||
console.log(
|
||||
`Cleared prior: ${priorExec.length} executions, ${priorThreads.length} threads, ${clearedActivity.changes} activity entries`,
|
||||
);
|
||||
|
||||
const insExec = db.prepare(`
|
||||
INSERT INTO execution_entity
|
||||
(workflowId, finished, mode, startedAt, stoppedAt, status, createdAt,
|
||||
storedAt, jsonSizeBytes, binaryDataSizeBytes, usedPrivateCredentials)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, 'db', ?, 0, 0)
|
||||
`);
|
||||
const insExecData = db.prepare(
|
||||
'INSERT INTO execution_data (executionId, workflowData, data) VALUES (?, ?, ?)',
|
||||
);
|
||||
const insThread = db.prepare(`
|
||||
INSERT INTO instance_ai_threads (id, resourceId, projectId, title, metadata, createdAt, updatedAt)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?)
|
||||
`);
|
||||
const insMessage = db.prepare(`
|
||||
INSERT INTO instance_ai_messages (id, threadId, content, role, type, resourceId, createdAt, updatedAt)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?)
|
||||
`);
|
||||
const insActivity = db.prepare(`
|
||||
INSERT INTO activity_event
|
||||
(category, action, typeVersion, userId, projectId, resourceType, resourceId, resourceName, data, createdAt)
|
||||
VALUES (?, ?, 1, ?, ?, ?, ?, ?, ?, ?)
|
||||
`);
|
||||
|
||||
for (const wf of workflows) {
|
||||
const short = wf.name.replace(SEED_PREFIX, '');
|
||||
const isFailing = short === FAILING_WORKFLOW;
|
||||
|
||||
// Scheduled workflows run often, webhook-driven ones sporadically.
|
||||
const scheduled = wf.nodes.some((n) => n.type === 'n8n-nodes-base.scheduleTrigger');
|
||||
const runs = scheduled ? DAYS * 2 : Math.ceil(DAYS / 2);
|
||||
|
||||
for (let i = 0; i < runs; i++) {
|
||||
// Newest run is i = 0, walking backwards across the window.
|
||||
const ageMs = (i * DAYS * 86400e3) / runs + Math.floor(random() * 3600e3);
|
||||
const startedAt = NOW - ageMs;
|
||||
const duration = 400 + Math.floor(random() * 4000);
|
||||
|
||||
// The failing workflow breaks on its newest run and twice more; everything
|
||||
// else fails rarely, so the estate looks used rather than synthetic.
|
||||
const shouldFail = isFailing ? i === 0 || i === 3 || i === 7 : random() < 0.04;
|
||||
const failure = shouldFail ? failureFor(wf, isFailing) : null;
|
||||
const failed = failure !== null;
|
||||
const status = failed ? 'error' : 'success';
|
||||
|
||||
const info = insExec.run(
|
||||
wf.id,
|
||||
failed ? 0 : 1,
|
||||
scheduled ? 'trigger' : 'webhook',
|
||||
iso(startedAt),
|
||||
iso(startedAt + duration),
|
||||
status,
|
||||
iso(startedAt),
|
||||
0,
|
||||
);
|
||||
const data = runData(wf, failure);
|
||||
const serialised = stringify(data);
|
||||
insExecData.run(
|
||||
Number(info.lastInsertRowid),
|
||||
JSON.stringify({
|
||||
id: wf.id,
|
||||
name: wf.name,
|
||||
nodes: wf.nodes,
|
||||
connections: {},
|
||||
active: true,
|
||||
settings: { executionOrder: 'v1' },
|
||||
}),
|
||||
serialised,
|
||||
);
|
||||
execCount++;
|
||||
if (failed) failCount++;
|
||||
}
|
||||
|
||||
// One thread for each workflow, phrased as a developer talking about that
|
||||
// workflow, so a "what was I working on?" probe has something to find.
|
||||
const threadId = uuid();
|
||||
const threadAt = NOW - Math.floor(random() * DAYS * 86400e3);
|
||||
insThread.run(
|
||||
threadId,
|
||||
wf.id,
|
||||
project.id,
|
||||
`Working on ${short}`,
|
||||
`{${THREAD_MARKER}}`,
|
||||
iso(threadAt),
|
||||
iso(threadAt + 600e3),
|
||||
);
|
||||
threadCount++;
|
||||
|
||||
const turns = isFailing
|
||||
? [
|
||||
['user', `${short} stopped working. What is going on?`],
|
||||
[
|
||||
'assistant',
|
||||
`The last three runs of ${short} failed at **${FAILURE.node}** with "${FAILURE.message}". ${FAILURE.description}`,
|
||||
],
|
||||
['user', 'Can you keep the rest of the flow running while I fix the credential?'],
|
||||
[
|
||||
'assistant',
|
||||
'You can set "Continue on Fail" on that node, so the Slack notification and the audit row still run.',
|
||||
],
|
||||
]
|
||||
: [
|
||||
['user', `Add a step to ${short} that records the run in a data table.`],
|
||||
[
|
||||
'assistant',
|
||||
`Added a Data Table insert named "Record Run" at the end of ${short}, writing to \`automation_runs\`, matching the other workflows here.`,
|
||||
],
|
||||
];
|
||||
for (const [role, content] of turns) {
|
||||
insMessage.run(
|
||||
uuid(),
|
||||
threadId,
|
||||
content,
|
||||
role,
|
||||
'message',
|
||||
wf.id,
|
||||
iso(threadAt),
|
||||
iso(threadAt),
|
||||
);
|
||||
msgCount++;
|
||||
}
|
||||
|
||||
// Activity entries. Only `workflow` and `credential` categories exist:
|
||||
// executions are deliberately not in the vocabulary, because
|
||||
// `execution_entity` already indexes the read a feed wants.
|
||||
const createdAt = NOW - DAYS * 86400e3 - Math.floor(random() * 7 * 86400e3);
|
||||
insActivity.run(
|
||||
'workflow',
|
||||
'created',
|
||||
user.id,
|
||||
project.id,
|
||||
'workflow',
|
||||
wf.id,
|
||||
wf.name,
|
||||
JSON.stringify({ nodeCount: wf.nodes.length }),
|
||||
iso(createdAt),
|
||||
);
|
||||
actCount++;
|
||||
const savedAt = NOW - Math.floor(random() * DAYS * 86400e3);
|
||||
insActivity.run(
|
||||
'workflow',
|
||||
'saved',
|
||||
user.id,
|
||||
project.id,
|
||||
'workflow',
|
||||
wf.id,
|
||||
wf.name,
|
||||
JSON.stringify({ nodeCount: wf.nodes.length }),
|
||||
iso(savedAt),
|
||||
);
|
||||
actCount++;
|
||||
insActivity.run(
|
||||
'workflow',
|
||||
'published',
|
||||
user.id,
|
||||
project.id,
|
||||
'workflow',
|
||||
wf.id,
|
||||
wf.name,
|
||||
JSON.stringify({ nodeCount: wf.nodes.length }),
|
||||
iso(savedAt + 60e3),
|
||||
);
|
||||
actCount++;
|
||||
}
|
||||
|
||||
// Credential activity, so the feed is not workflow-only.
|
||||
for (const cred of db
|
||||
.prepare('SELECT id, name FROM credentials_entity WHERE name LIKE ?')
|
||||
.all(`${SEED_PREFIX}%`)) {
|
||||
insActivity.run(
|
||||
'credential',
|
||||
'created',
|
||||
user.id,
|
||||
project.id,
|
||||
'credential',
|
||||
cred.id,
|
||||
cred.name,
|
||||
null,
|
||||
iso(NOW - DAYS * 86400e3 - 86400e3),
|
||||
);
|
||||
actCount++;
|
||||
}
|
||||
|
||||
db.exec('COMMIT');
|
||||
} catch (e) {
|
||||
db.exec('ROLLBACK');
|
||||
console.error('Rolled back:', e.message);
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
console.log(`Executions: ${execCount} (${failCount} failed)`);
|
||||
console.log(`AI threads: ${threadCount}, messages: ${msgCount}`);
|
||||
console.log(`Activity entries: ${actCount}`);
|
||||
console.log(`Newest failure: "${SEED_PREFIX}${FAILING_WORKFLOW}" at ${FAILURE.node}`);
|
||||
db.close();
|
||||
}
|
||||
|
||||
main();
|
||||
@@ -0,0 +1,274 @@
|
||||
import assert from 'node:assert/strict';
|
||||
import { execFileSync } from 'node:child_process';
|
||||
import { mkdtempSync } from 'node:fs';
|
||||
import { tmpdir } from 'node:os';
|
||||
import { join, dirname } from 'node:path';
|
||||
import { DatabaseSync } from 'node:sqlite';
|
||||
import { describe, it } from 'node:test';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
|
||||
const SCRIPT = join(dirname(fileURLToPath(import.meta.url)), 'seedHistory.mjs');
|
||||
const SEED_PREFIX = '[seed] ';
|
||||
|
||||
// Only the tables and columns seedHistory reads or writes, with the foreign keys it
|
||||
// relies on. It sets `PRAGMA foreign_keys = ON`, so the referenced tables have to
|
||||
// exist and executions have to cascade from their workflow.
|
||||
const SCHEMA = `
|
||||
CREATE TABLE user (id varchar PRIMARY KEY, createdAt datetime(3) NOT NULL);
|
||||
CREATE TABLE project (id varchar(36) PRIMARY KEY, name varchar(255) NOT NULL, type varchar(36) NOT NULL);
|
||||
CREATE TABLE workflow_entity (id varchar(36) PRIMARY KEY, name varchar(128) NOT NULL, nodes text);
|
||||
CREATE TABLE credentials_entity (id varchar(36) PRIMARY KEY, name varchar(128) NOT NULL);
|
||||
CREATE TABLE execution_entity (
|
||||
id integer PRIMARY KEY AUTOINCREMENT NOT NULL,
|
||||
workflowId varchar(36) NOT NULL,
|
||||
finished boolean NOT NULL,
|
||||
mode varchar NOT NULL,
|
||||
startedAt datetime, stoppedAt datetime,
|
||||
status varchar NOT NULL,
|
||||
createdAt datetime(3) NOT NULL,
|
||||
storedAt varchar(2) NOT NULL DEFAULT ('db'),
|
||||
jsonSizeBytes bigint NOT NULL DEFAULT (0),
|
||||
binaryDataSizeBytes bigint NOT NULL DEFAULT (0),
|
||||
usedPrivateCredentials boolean NOT NULL DEFAULT FALSE,
|
||||
FOREIGN KEY (workflowId) REFERENCES workflow_entity (id) ON DELETE CASCADE
|
||||
);
|
||||
CREATE TABLE execution_data (
|
||||
executionId int PRIMARY KEY NOT NULL,
|
||||
workflowData text NOT NULL,
|
||||
data text NOT NULL,
|
||||
FOREIGN KEY (executionId) REFERENCES execution_entity (id) ON DELETE CASCADE
|
||||
);
|
||||
CREATE TABLE instance_ai_threads (
|
||||
id varchar PRIMARY KEY NOT NULL,
|
||||
resourceId varchar(255) NOT NULL,
|
||||
projectId varchar(36) NOT NULL,
|
||||
title text NOT NULL DEFAULT (''),
|
||||
metadata text,
|
||||
createdAt datetime(3) NOT NULL, updatedAt datetime(3) NOT NULL,
|
||||
FOREIGN KEY (projectId) REFERENCES project (id) ON DELETE CASCADE
|
||||
);
|
||||
CREATE TABLE instance_ai_messages (
|
||||
id varchar(36) PRIMARY KEY NOT NULL,
|
||||
threadId varchar NOT NULL,
|
||||
content text NOT NULL, role varchar(16) NOT NULL, type varchar(32),
|
||||
resourceId varchar(255),
|
||||
createdAt datetime(3) NOT NULL, updatedAt datetime(3) NOT NULL,
|
||||
FOREIGN KEY (threadId) REFERENCES instance_ai_threads (id) ON DELETE CASCADE
|
||||
);
|
||||
CREATE TABLE activity_event (
|
||||
id integer PRIMARY KEY NOT NULL,
|
||||
category varchar(32) NOT NULL, action varchar(64) NOT NULL,
|
||||
typeVersion integer NOT NULL DEFAULT (1),
|
||||
userId varchar, projectId varchar(36),
|
||||
resourceType varchar(32), resourceId varchar(36), resourceName text, data text,
|
||||
createdAt datetime(3) NOT NULL,
|
||||
FOREIGN KEY (userId) REFERENCES user (id) ON DELETE SET NULL,
|
||||
FOREIGN KEY (projectId) REFERENCES project (id) ON DELETE CASCADE
|
||||
);
|
||||
`;
|
||||
|
||||
const NODES = JSON.stringify([
|
||||
{
|
||||
name: 'Schedule Trigger',
|
||||
type: 'n8n-nodes-base.scheduleTrigger',
|
||||
typeVersion: 1.4,
|
||||
parameters: {},
|
||||
},
|
||||
{ name: 'Notify', type: 'n8n-nodes-base.slack', typeVersion: 2.7, parameters: {} },
|
||||
]);
|
||||
|
||||
/**
|
||||
* A database holding a seeded project with two workflows, plus rows the seeder must
|
||||
* not touch: unmarked threads and activity inside the seeded project, and a marked
|
||||
* thread belonging to a different project.
|
||||
*/
|
||||
function fixture() {
|
||||
const file = join(mkdtempSync(join(tmpdir(), 'seed-history-')), 'database.sqlite');
|
||||
const db = new DatabaseSync(file);
|
||||
db.exec(SCHEMA);
|
||||
const now = '2026-01-01 00:00:00.000';
|
||||
|
||||
db.exec(`
|
||||
INSERT INTO user (id, createdAt) VALUES ('u1', '${now}');
|
||||
INSERT INTO project (id, name, type) VALUES
|
||||
('p-seed', '${SEED_PREFIX}Automation Platform', 'team'),
|
||||
('p-other', 'Someone Else', 'team');
|
||||
INSERT INTO workflow_entity (id, name, nodes) VALUES
|
||||
('wf1', '${SEED_PREFIX}Alpha', '${NODES}'),
|
||||
('wf2', '${SEED_PREFIX}Beta', '${NODES}');
|
||||
INSERT INTO credentials_entity (id, name) VALUES ('c1', '${SEED_PREFIX}OpenAI');
|
||||
`);
|
||||
|
||||
const thread = db.prepare(
|
||||
'INSERT INTO instance_ai_threads (id, resourceId, projectId, title, metadata, createdAt, updatedAt) VALUES (?,?,?,?,?,?,?)',
|
||||
);
|
||||
// A developer's own conversation on a seeded workflow: same project, no marker.
|
||||
thread.run('t-real', 'wf1', 'p-seed', 'My real conversation', '{"seeded":false}', now, now);
|
||||
// Metadata is nullable, so the filter has to cope with NULL rather than throw.
|
||||
thread.run('t-null', 'wf2', 'p-seed', 'No metadata at all', null, now, now);
|
||||
// Marked, but in another project. Scoping must keep the seeder out of it.
|
||||
thread.run('t-other', 'wf9', 'p-other', 'Marked elsewhere', '{"seeded":true}', now, now);
|
||||
|
||||
db.prepare(
|
||||
'INSERT INTO instance_ai_messages (id, threadId, content, role, type, resourceId, createdAt, updatedAt) VALUES (?,?,?,?,?,?,?,?)',
|
||||
).run('m-real', 't-real', 'do not delete me', 'user', 'message', 'wf1', now, now);
|
||||
|
||||
const activity = db.prepare(
|
||||
'INSERT INTO activity_event (category, action, userId, projectId, resourceType, resourceId, resourceName, data, createdAt) VALUES (?,?,?,?,?,?,?,?,?)',
|
||||
);
|
||||
// Real activity in the seeded project: the relay would write rows like this.
|
||||
activity.run(
|
||||
'workflow',
|
||||
'saved',
|
||||
'u1',
|
||||
'p-seed',
|
||||
'workflow',
|
||||
'wf-real',
|
||||
'My Real Workflow',
|
||||
null,
|
||||
now,
|
||||
);
|
||||
// A null resourceName must not be swept up by a LIKE on that column.
|
||||
activity.run('workflow', 'saved', 'u1', 'p-seed', 'workflow', 'wf-anon', null, null, now);
|
||||
db.close();
|
||||
return file;
|
||||
}
|
||||
|
||||
const run = (dbFile) =>
|
||||
execFileSync(process.execPath, [SCRIPT], {
|
||||
env: { ...process.env, DB_SQLITE_DATABASE: dbFile, HISTORY_DAYS: '2' },
|
||||
encoding: 'utf8',
|
||||
});
|
||||
|
||||
function counts(dbFile) {
|
||||
const db = new DatabaseSync(dbFile, { readOnly: true });
|
||||
const one = (sql) => db.prepare(sql).get().n;
|
||||
const out = {
|
||||
executions: one('SELECT COUNT(*) AS n FROM execution_entity'),
|
||||
executionData: one('SELECT COUNT(*) AS n FROM execution_data'),
|
||||
threads: one('SELECT COUNT(*) AS n FROM instance_ai_threads'),
|
||||
messages: one('SELECT COUNT(*) AS n FROM instance_ai_messages'),
|
||||
activity: one('SELECT COUNT(*) AS n FROM activity_event'),
|
||||
seededThreads: one(
|
||||
`SELECT COUNT(*) AS n FROM instance_ai_threads WHERE projectId = 'p-seed' AND metadata LIKE '%"seeded":true%'`,
|
||||
),
|
||||
seededActivity: one(
|
||||
`SELECT COUNT(*) AS n FROM activity_event WHERE projectId = 'p-seed' AND resourceName LIKE '[seed] %'`,
|
||||
),
|
||||
};
|
||||
const ids = (sql) =>
|
||||
db
|
||||
.prepare(sql)
|
||||
.all()
|
||||
.map((r) => r.id);
|
||||
out.threadIds = ids('SELECT id FROM instance_ai_threads ORDER BY id');
|
||||
db.close();
|
||||
return out;
|
||||
}
|
||||
|
||||
const survivors = (dbFile) => {
|
||||
const db = new DatabaseSync(dbFile, { readOnly: true });
|
||||
const has = (sql, ...p) => db.prepare(sql).all(...p).length > 0;
|
||||
const out = {
|
||||
realThread: has(`SELECT 1 FROM instance_ai_threads WHERE id = 't-real'`),
|
||||
nullMetadataThread: has(`SELECT 1 FROM instance_ai_threads WHERE id = 't-null'`),
|
||||
otherProjectThread: has(`SELECT 1 FROM instance_ai_threads WHERE id = 't-other'`),
|
||||
realMessage: has(`SELECT 1 FROM instance_ai_messages WHERE id = 'm-real'`),
|
||||
realActivity: has(`SELECT 1 FROM activity_event WHERE resourceName = 'My Real Workflow'`),
|
||||
anonActivity: has(`SELECT 1 FROM activity_event WHERE resourceId = 'wf-anon'`),
|
||||
};
|
||||
db.close();
|
||||
return out;
|
||||
};
|
||||
|
||||
describe('seedHistory cleanup', () => {
|
||||
it('writes history for the seeded workflows', () => {
|
||||
const dbFile = fixture();
|
||||
run(dbFile);
|
||||
const c = counts(dbFile);
|
||||
assert.equal(c.seededThreads, 2, 'one marked thread per seeded workflow');
|
||||
assert.ok(c.executions > 0, 'executions written');
|
||||
assert.equal(c.executionData, c.executions, 'every execution has its data row');
|
||||
assert.ok(c.seededActivity > 0, 'activity written');
|
||||
});
|
||||
|
||||
it('leaves rows it did not create alone', () => {
|
||||
const dbFile = fixture();
|
||||
run(dbFile);
|
||||
assert.deepEqual(survivors(dbFile), {
|
||||
realThread: true,
|
||||
nullMetadataThread: true,
|
||||
otherProjectThread: true,
|
||||
realMessage: true,
|
||||
realActivity: true,
|
||||
anonActivity: true,
|
||||
});
|
||||
});
|
||||
|
||||
// Re-running used to stack a second fortnight on the first, and later to collide
|
||||
// on primary keys generated from the fixed seed.
|
||||
it('replaces its own history instead of accumulating', () => {
|
||||
const dbFile = fixture();
|
||||
run(dbFile);
|
||||
const first = counts(dbFile);
|
||||
run(dbFile);
|
||||
const second = counts(dbFile);
|
||||
run(dbFile);
|
||||
const third = counts(dbFile);
|
||||
assert.deepEqual(second, first, 'second run matches the first');
|
||||
assert.deepEqual(third, first, 'third run matches the first');
|
||||
});
|
||||
|
||||
it('keeps the untouched rows across repeated runs', () => {
|
||||
const dbFile = fixture();
|
||||
run(dbFile);
|
||||
run(dbFile);
|
||||
const s = survivors(dbFile);
|
||||
assert.equal(s.realThread, true);
|
||||
assert.equal(s.realMessage, true);
|
||||
assert.equal(s.otherProjectThread, true);
|
||||
assert.equal(s.realActivity, true);
|
||||
});
|
||||
|
||||
// `seed:account` deletes and recreates the workflows, so their ids change between
|
||||
// history runs. Clearing by workflow id therefore matched nothing on the second
|
||||
// run, orphaning the first run's rows, and the ids generated from the fixed seed
|
||||
// then collided on insert.
|
||||
it('survives the seeded workflows being replaced with new ids', () => {
|
||||
const dbFile = fixture();
|
||||
run(dbFile);
|
||||
const before = counts(dbFile);
|
||||
|
||||
const db = new DatabaseSync(dbFile);
|
||||
db.exec('PRAGMA foreign_keys = ON');
|
||||
db.exec(`
|
||||
INSERT INTO workflow_entity (id, name, nodes)
|
||||
SELECT 'new-' || id, name, nodes FROM workflow_entity WHERE name LIKE '[seed] %';
|
||||
DELETE FROM workflow_entity WHERE id IN ('wf1', 'wf2');
|
||||
`);
|
||||
db.close();
|
||||
|
||||
const out = run(dbFile);
|
||||
const after = counts(dbFile);
|
||||
assert.equal(after.seededThreads, before.seededThreads, 'threads replaced, not duplicated');
|
||||
assert.equal(after.seededActivity, before.seededActivity, 'activity replaced, not duplicated');
|
||||
assert.equal(after.executionData, after.executions, 'no orphaned execution data');
|
||||
assert.doesNotMatch(out, /UNIQUE constraint|Rolled back/);
|
||||
assert.equal(survivors(dbFile).realThread, true, 'the real thread still survives');
|
||||
});
|
||||
|
||||
it('reports what it cleared on the second run', () => {
|
||||
const dbFile = fixture();
|
||||
run(dbFile);
|
||||
const out = run(dbFile);
|
||||
assert.match(out, /Cleared prior: \d+ executions, 2 threads, \d+ activity entries/);
|
||||
});
|
||||
|
||||
it('exits with a message when nothing is seeded', () => {
|
||||
const file = join(mkdtempSync(join(tmpdir(), 'seed-history-empty-')), 'database.sqlite');
|
||||
const db = new DatabaseSync(file);
|
||||
db.exec(SCHEMA);
|
||||
db.close();
|
||||
assert.throws(() => run(file), /Run seed:account first/);
|
||||
});
|
||||
});
|
||||
@@ -7,6 +7,13 @@
|
||||
// data-tables, whose names disallow brackets) so the script can clear its
|
||||
// own output before reseeding without touching pre-existing data.
|
||||
|
||||
import {
|
||||
PREFERENCE_CREDENTIALS,
|
||||
PREFERENCE_DATA_TABLES,
|
||||
PREFERENCE_PROJECT,
|
||||
preferenceWorkflows,
|
||||
} from './preference-profile.mjs';
|
||||
|
||||
const BASE = process.env.N8N_BASE_URL ?? 'http://localhost:5678';
|
||||
const KEY = process.env.N8N_API_KEY;
|
||||
if (!KEY) {
|
||||
@@ -19,6 +26,24 @@ const PERSONAL_WORKFLOWS = Number(process.env.WF_MULTIPLIER) || 50;
|
||||
const SEED_PREFIX = '[seed] ';
|
||||
const SEED_DT_PREFIX = 'seed_';
|
||||
|
||||
// Set it and the run is reproducible. `preference` defaults it: a profile meant for
|
||||
// A/B evaluation is useless if the estate moves between runs.
|
||||
const SEED_ENV = process.env.SEED;
|
||||
|
||||
// `estate` (default): ~500 varied workflows, built to stress the dependency graph.
|
||||
// `preference`: ten hand-written workflows in one house style, small enough to fit an
|
||||
// agent's context. Diversity is the bug in that profile, not the feature.
|
||||
const PROFILE = (process.env.PROFILE ?? 'estate').toLowerCase();
|
||||
if (!['estate', 'preference'].includes(PROFILE)) {
|
||||
console.error(`PROFILE must be 'estate' or 'preference', got '${PROFILE}'`);
|
||||
process.exit(1);
|
||||
}
|
||||
const SEED = SEED_ENV !== undefined ? Number(SEED_ENV) : PROFILE === 'preference' ? 1 : undefined;
|
||||
if (SEED !== undefined && !Number.isFinite(SEED)) {
|
||||
console.error(`SEED must be a number, got '${SEED_ENV}'`);
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
const CREDS_PER_PROJECT_RANGE = [1, 2];
|
||||
const DATA_TABLES_PER_PROJECT_PROB = 0.55;
|
||||
const SUBWORKFLOW_PROB = 0.7;
|
||||
@@ -27,7 +52,7 @@ const SUBWORKFLOW_PROB = 0.7;
|
||||
// community projects; small/large are minority outliers. Trenchcoats are
|
||||
// kept deliberately small — they're legacy noise, not the bulk of the org.
|
||||
function sampleWorkflowCount(kind /* 'community' | 'trenchcoat' */) {
|
||||
const r = Math.random();
|
||||
const r = random();
|
||||
if (kind === 'trenchcoat') {
|
||||
if (r < 0.5) return randInt([6, 12]);
|
||||
if (r < 0.9) return randInt([12, 20]);
|
||||
@@ -340,25 +365,45 @@ const CRED_RECIPES = [
|
||||
},
|
||||
];
|
||||
|
||||
// Every random choice goes through `random()`. With SEED set it is a seeded PRNG and
|
||||
// two runs are byte-identical; without, it falls back to Math.random.
|
||||
function makeRandom(seed) {
|
||||
if (seed === undefined) return Math.random;
|
||||
// mulberry32: small, fast, good enough for fixtures. Not for anything security-facing.
|
||||
let a = seed >>> 0;
|
||||
return function () {
|
||||
a = (a + 0x6d2b79f5) >>> 0;
|
||||
let t = a;
|
||||
t = Math.imul(t ^ (t >>> 15), t | 1);
|
||||
t ^= t + Math.imul(t ^ (t >>> 7), t | 61);
|
||||
return ((t ^ (t >>> 14)) >>> 0) / 4294967296;
|
||||
};
|
||||
}
|
||||
const random = makeRandom(SEED);
|
||||
|
||||
// Seeded runs pin the clock too, or timestamps drift and the estate is only half
|
||||
// reproducible. The date is arbitrary.
|
||||
const NOW = SEED !== undefined ? Date.parse('2026-01-15T09:00:00.000Z') : Date.now();
|
||||
|
||||
function rand(len = 16) {
|
||||
const chars = 'abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789';
|
||||
let s = '';
|
||||
for (let i = 0; i < len; i++) s += chars[Math.floor(Math.random() * chars.length)];
|
||||
for (let i = 0; i < len; i++) s += chars[Math.floor(random() * chars.length)];
|
||||
return s;
|
||||
}
|
||||
function pickWord(arr) {
|
||||
return arr[Math.floor(Math.random() * arr.length)];
|
||||
return arr[Math.floor(random() * arr.length)];
|
||||
}
|
||||
function pick(arr, n) {
|
||||
const c = [...arr];
|
||||
const out = [];
|
||||
for (let i = 0; i < n && c.length; i++) {
|
||||
out.push(c.splice(Math.floor(Math.random() * c.length), 1)[0]);
|
||||
out.push(c.splice(Math.floor(random() * c.length), 1)[0]);
|
||||
}
|
||||
return out;
|
||||
}
|
||||
function randInt([lo, hi]) {
|
||||
return lo + Math.floor(Math.random() * (hi - lo + 1));
|
||||
return lo + Math.floor(random() * (hi - lo + 1));
|
||||
}
|
||||
|
||||
const WORKFLOW_THEMES = [
|
||||
@@ -745,12 +790,151 @@ async function clearSeeded() {
|
||||
log('Clear done.');
|
||||
}
|
||||
|
||||
// A missing token gives a placeholder that says so, keeping every node wired to
|
||||
// something openable but not runnable. Supplying it later upgrades in place by id.
|
||||
// Token lengths are logged, never values.
|
||||
async function seedPreferenceCredentials(projectId, existingByName) {
|
||||
const out = {};
|
||||
for (const spec of PREFERENCE_CREDENTIALS) {
|
||||
const token = process.env[spec.env];
|
||||
const real = typeof token === 'string' && token.length > 0;
|
||||
const placeholderName = `${SEED_PREFIX}${spec.name} (seed, fake key)`;
|
||||
const realName = `${SEED_PREFIX}${spec.name}`;
|
||||
const name = real ? realName : placeholderName;
|
||||
const data = { [spec.field]: real ? token : `seed-placeholder-${spec.key}` };
|
||||
|
||||
// Upgrade in place so the id survives and the nodes keep resolving. Both names
|
||||
// are tried: supplying a token drops the "(seed, fake key)" suffix, so the new
|
||||
// name alone would miss the placeholder being replaced.
|
||||
const prior = existingByName.get(name) ?? existingByName.get(real ? placeholderName : realName);
|
||||
try {
|
||||
if (prior) {
|
||||
// PATCH, not PUT: there is no PUT route, and its 405 fails quietly enough
|
||||
// to look like a working upgrade.
|
||||
await api('PATCH', `/credentials/${prior.id}`, { name, type: spec.type, data });
|
||||
out[spec.key] = { id: prior.id, name };
|
||||
} else {
|
||||
const created = await api('POST', '/credentials', {
|
||||
name,
|
||||
type: spec.type,
|
||||
data,
|
||||
projectId,
|
||||
});
|
||||
out[spec.key] = { id: created.id, name: created.name };
|
||||
}
|
||||
log(` ${spec.type}: ${real ? `real token, ${token.length} chars` : 'placeholder'}`);
|
||||
} catch (e) {
|
||||
log(` credential failed: ${spec.type}`, String(e).slice(0, 200));
|
||||
}
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
// Ten workflows in one project, one house style. See `preference-profile.mjs`.
|
||||
async function seedPreferenceProfile() {
|
||||
log('Seeding preference profile at', BASE);
|
||||
log(`Seed: ${SEED} (fixed — re-running produces the same estate)`);
|
||||
|
||||
// Not `clearSeeded()`: that deletes credentials, so their ids would change every
|
||||
// run and every node pointing at them would need rewiring. Only the workflows are
|
||||
// removed, since they are the one thing recreated unconditionally below.
|
||||
const priorWorkflows = (await listAll('/workflows')).filter((w) =>
|
||||
w.name.startsWith(SEED_PREFIX),
|
||||
);
|
||||
if (priorWorkflows.length > 0) {
|
||||
log(`Removing ${priorWorkflows.length} prior seeded workflows...`);
|
||||
for (const w of priorWorkflows) {
|
||||
try {
|
||||
await api('DELETE', `/workflows/${w.id}`);
|
||||
} catch (e) {
|
||||
log(' workflow delete failed:', w.name, String(e).slice(0, 200));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
const projects = await listAll('/projects');
|
||||
const projectName = `${SEED_PREFIX}${PREFERENCE_PROJECT}`;
|
||||
let project = projects.find((p) => p.name === projectName);
|
||||
if (!project) {
|
||||
project = await api('POST', '/projects', { name: projectName });
|
||||
log('Created project:', projectName);
|
||||
} else {
|
||||
log('Reusing project:', projectName);
|
||||
}
|
||||
|
||||
log('Credentials...');
|
||||
const existingCreds = new Map((await listAll('/credentials')).map((c) => [c.name, c]));
|
||||
const creds = await seedPreferenceCredentials(project.id, existingCreds);
|
||||
|
||||
log('Data tables...');
|
||||
const tables = {};
|
||||
const existingDts = await listAll('/data-tables');
|
||||
for (const spec of PREFERENCE_DATA_TABLES) {
|
||||
const name = `${SEED_DT_PREFIX}${spec.name}`;
|
||||
const prior = existingDts.find((d) => d.name === name);
|
||||
if (prior) {
|
||||
tables[spec.name] = prior.id;
|
||||
log(` ${name}: reused`);
|
||||
continue;
|
||||
}
|
||||
const created = await api('POST', '/data-tables', {
|
||||
name,
|
||||
projectId: project.id,
|
||||
columns: spec.columns,
|
||||
});
|
||||
tables[spec.name] = created.id;
|
||||
if (spec.rows.length > 0) {
|
||||
await api('POST', `/data-tables/${created.id}/rows`, { data: spec.rows });
|
||||
}
|
||||
log(` ${name}: created with ${spec.rows.length} rows`);
|
||||
}
|
||||
|
||||
log('Workflows...');
|
||||
let created = 0;
|
||||
for (const wf of preferenceWorkflows(tables)) {
|
||||
// Recipes name the credential by key; resolve to the ids just created.
|
||||
const nodes = wf.nodes.map((n) => {
|
||||
const node = { ...n, id: rand(36) };
|
||||
if (node.credentials) {
|
||||
const resolved = {};
|
||||
for (const key of Object.keys(node.credentials)) {
|
||||
const spec = PREFERENCE_CREDENTIALS.find((c) => c.key === key);
|
||||
const cred = creds[key];
|
||||
if (spec && cred) resolved[spec.type] = { id: cred.id, name: cred.name };
|
||||
}
|
||||
node.credentials = resolved;
|
||||
}
|
||||
return node;
|
||||
});
|
||||
try {
|
||||
await api('POST', '/workflows', {
|
||||
name: `${SEED_PREFIX}${wf.name}`,
|
||||
nodes,
|
||||
connections: wf.connections,
|
||||
settings: { executionOrder: 'v1' },
|
||||
projectId: project.id,
|
||||
});
|
||||
created++;
|
||||
log(` ${wf.name}`);
|
||||
} catch (e) {
|
||||
log(` workflow failed: ${wf.name}`, String(e).slice(0, 300));
|
||||
}
|
||||
}
|
||||
|
||||
log(`Done. ${created}/10 workflows, ${Object.keys(tables).length} data tables.`);
|
||||
log('Total API requests:', totalReq);
|
||||
}
|
||||
|
||||
async function main() {
|
||||
if (CLEAR) {
|
||||
await clearSeeded();
|
||||
log('Clear finished');
|
||||
return;
|
||||
}
|
||||
if (PROFILE === 'preference') {
|
||||
await seedPreferenceProfile();
|
||||
return;
|
||||
}
|
||||
log('Seeding n8n at', BASE);
|
||||
|
||||
log('Fetching projects...');
|
||||
@@ -797,7 +981,7 @@ async function main() {
|
||||
const n = randInt(CREDS_PER_PROJECT_RANGE);
|
||||
const recipes = Array.from(
|
||||
{ length: n },
|
||||
() => CRED_RECIPES[Math.floor(Math.random() * CRED_RECIPES.length)],
|
||||
() => CRED_RECIPES[Math.floor(random() * CRED_RECIPES.length)],
|
||||
);
|
||||
for (const recipe of recipes) {
|
||||
try {
|
||||
@@ -859,7 +1043,7 @@ async function main() {
|
||||
name: `${pickWord(['Acme', 'Globex', 'Initech', 'Umbrella', 'Wonka', 'Stark', 'Wayne'])} ${i + 1}`,
|
||||
plan: pickWord(['free', 'pro', 'enterprise']),
|
||||
region: pickWord(['us-east', 'us-west', 'eu-west', 'ap-south']),
|
||||
created_at: new Date(Date.now() - Math.floor(Math.random() * 365 * 86400e3)).toISOString(),
|
||||
created_at: new Date(NOW - Math.floor(random() * 365 * 86400e3)).toISOString(),
|
||||
}));
|
||||
await api('POST', `/data-tables/${created.id}/rows`, { data: sampleRows });
|
||||
log(`Central data table: ${created.name} → ${centralDtOwner.name} (${sampleRows.length} rows)`);
|
||||
@@ -940,7 +1124,7 @@ async function main() {
|
||||
const theme = themes[i];
|
||||
const credPick = utilCreds.length ? [utilCreds[i % utilCreds.length]] : [];
|
||||
const directDt =
|
||||
isOrgUtil && centralDataTable && Math.random() < ORG_UTIL_DIRECT_DT_PROB
|
||||
isOrgUtil && centralDataTable && random() < ORG_UTIL_DIRECT_DT_PROB
|
||||
? centralDataTable
|
||||
: null;
|
||||
const wf = await createWf({
|
||||
@@ -990,7 +1174,7 @@ async function main() {
|
||||
// Customers-proxy ref → indirect access to central data table.
|
||||
if (
|
||||
customersProxy &&
|
||||
Math.random() < CENTRAL_DT_REF_PROB &&
|
||||
random() < CENTRAL_DT_REF_PROB &&
|
||||
!updatedSubIds.includes(customersProxy.id) &&
|
||||
(refs?.wfs.has(customersProxy.id) || refBudgetAllowsWf(proj.id, customersProxy))
|
||||
) {
|
||||
@@ -999,11 +1183,10 @@ async function main() {
|
||||
}
|
||||
|
||||
// Generic utility-workflow ref.
|
||||
if (Math.random() < UTILITY_REF_PROB && lynchpinWorkflows.length > 0) {
|
||||
if (random() < UTILITY_REF_PROB && lynchpinWorkflows.length > 0) {
|
||||
const alreadyOwned = lynchpinWorkflows.filter((w) => refs?.wfs.has(w.id));
|
||||
const pool =
|
||||
alreadyOwned.length > 0 && Math.random() < 0.7 ? alreadyOwned : lynchpinWorkflows;
|
||||
const cand = pool[Math.floor(Math.random() * pool.length)];
|
||||
const pool = alreadyOwned.length > 0 && random() < 0.7 ? alreadyOwned : lynchpinWorkflows;
|
||||
const cand = pool[Math.floor(random() * pool.length)];
|
||||
if (cand && (refs?.wfs.has(cand.id) || refBudgetAllowsWf(proj.id, cand))) {
|
||||
if (!updatedSubIds.includes(cand.id)) {
|
||||
updatedSubIds = [...updatedSubIds, cand.id];
|
||||
@@ -1029,7 +1212,7 @@ async function main() {
|
||||
// Roughly half the opt-outs go full self-contained.
|
||||
const selfContainedIds = new Set();
|
||||
for (const pid of nonUtilityUsingIds) {
|
||||
if (Math.random() < 0.5) selfContainedIds.add(pid);
|
||||
if (random() < 0.5) selfContainedIds.add(pid);
|
||||
}
|
||||
const projectBudget = new Map();
|
||||
const projectRefs = new Map(); // projectId -> { wfs: Set, creds: Set }
|
||||
@@ -1113,17 +1296,17 @@ async function main() {
|
||||
const refs = projectRefs.get(projectId);
|
||||
const ids = [];
|
||||
for (let i = 0; i < count; i++) {
|
||||
const r = Math.random();
|
||||
const r = random();
|
||||
let pool;
|
||||
if (r < SUBWF_PROB_OWN) pool = ownPool;
|
||||
else if (r < SUBWF_PROB_OWN + SUBWF_PROB_SIBLING)
|
||||
pool = siblingPool.length ? siblingPool : ownPool;
|
||||
else pool = lynchpinPool.length ? lynchpinPool : ownPool;
|
||||
if (pool.length === 0) continue;
|
||||
let cand = pool[Math.floor(Math.random() * pool.length)];
|
||||
let cand = pool[Math.floor(random() * pool.length)];
|
||||
if (!refBudgetAllowsWf(projectId, cand)) {
|
||||
if (ownPool.length === 0) continue;
|
||||
cand = ownPool[Math.floor(Math.random() * ownPool.length)];
|
||||
cand = ownPool[Math.floor(random() * ownPool.length)];
|
||||
}
|
||||
if (ids.includes(cand.id)) continue;
|
||||
ids.push(cand.id);
|
||||
@@ -1144,7 +1327,7 @@ async function main() {
|
||||
const refs = projectRefs.get(projectId);
|
||||
const ids = [];
|
||||
for (let i = 0; i < count; i++) {
|
||||
const r = Math.random();
|
||||
const r = random();
|
||||
let pool;
|
||||
if (r < 0.75) pool = sameGroupPool.length ? sameGroupPool : ownPool;
|
||||
else if (r < 0.85)
|
||||
@@ -1155,7 +1338,7 @@ async function main() {
|
||||
: ownPool;
|
||||
else pool = lynchpinPool.length ? lynchpinPool : ownPool;
|
||||
if (pool.length === 0) continue;
|
||||
const cand = pool[Math.floor(Math.random() * pool.length)];
|
||||
const cand = pool[Math.floor(random() * pool.length)];
|
||||
if (ids.includes(cand.id)) continue;
|
||||
ids.push(cand.id);
|
||||
if (cand.projectId !== projectId) refs.wfs.add(cand.id);
|
||||
@@ -1188,8 +1371,8 @@ async function main() {
|
||||
const theme = pickWord(themePool);
|
||||
const group = groups.length ? groups[(p.leafCount + i) % groups.length] : undefined;
|
||||
let credPick = p.creds.length ? pick(p.creds, Math.min(p.creds.length, randInt([0, 2]))) : [];
|
||||
if (Math.random() < lynchpinCredProb && lynchpinCreds.length > 0) {
|
||||
const lc = lynchpinCreds[Math.floor(Math.random() * lynchpinCreds.length)];
|
||||
if (random() < lynchpinCredProb && lynchpinCreds.length > 0) {
|
||||
const lc = lynchpinCreds[Math.floor(random() * lynchpinCreds.length)];
|
||||
if (refBudgetAllowsCred(proj.id, lc.id, lc.ownerId)) {
|
||||
credPick = [lc, ...credPick.filter((c) => c.type !== lc.type)];
|
||||
refs.creds.add(lc.id);
|
||||
@@ -1197,7 +1380,7 @@ async function main() {
|
||||
}
|
||||
}
|
||||
const baseSubIds =
|
||||
Math.random() < SUBWORKFLOW_PROB
|
||||
random() < SUBWORKFLOW_PROB
|
||||
? pickSubWorkflowIds(proj.id, proj.name, group, randInt([1, 3]))
|
||||
: [];
|
||||
const { subWorkflowIds: subIds } = applyOrgUtilityRefs({
|
||||
@@ -1248,7 +1431,7 @@ async function main() {
|
||||
const dataTablesByProject = new Map();
|
||||
let dtTotal = 0;
|
||||
for (const proj of allTargetProjects) {
|
||||
if (Math.random() > DATA_TABLES_PER_PROJECT_PROB) continue;
|
||||
if (random() > DATA_TABLES_PER_PROJECT_PROB) continue;
|
||||
const dtName = `${SEED_DT_PREFIX}${pickWord(dtNames)}_${rand(4)}`;
|
||||
try {
|
||||
const dt = await api('POST', `/data-tables`, {
|
||||
@@ -1268,9 +1451,9 @@ async function main() {
|
||||
const rowCount = randInt([5, 20]);
|
||||
const rows = Array.from({ length: rowCount }, () => ({
|
||||
label: pickWord(['alpha', 'beta', 'gamma', 'delta', 'epsilon']),
|
||||
score: Math.floor(Math.random() * 100),
|
||||
is_active: Math.random() < 0.7,
|
||||
recorded_at: new Date(Date.now() - Math.floor(Math.random() * 30 * 86400e3)).toISOString(),
|
||||
score: Math.floor(random() * 100),
|
||||
is_active: random() < 0.7,
|
||||
recorded_at: new Date(NOW - Math.floor(random() * 30 * 86400e3)).toISOString(),
|
||||
}));
|
||||
try {
|
||||
await api('POST', `/data-tables/${dt.id}/rows`, { data: rows });
|
||||
@@ -1297,8 +1480,8 @@ async function main() {
|
||||
const dt = dts[i % dts.length];
|
||||
const theme = `${pickWord(['Sync to', 'Report from', 'Backfill', 'Reconcile', 'Audit'])} ${dt.name.replace(SEED_DT_PREFIX, '')}`;
|
||||
let credPick = creds.length ? pick(creds, Math.min(creds.length, randInt([0, 1]))) : [];
|
||||
if (Math.random() < LYNCHPIN_CRED_REF_PROB && lynchpinCreds.length > 0) {
|
||||
const lc = lynchpinCreds[Math.floor(Math.random() * lynchpinCreds.length)];
|
||||
if (random() < LYNCHPIN_CRED_REF_PROB && lynchpinCreds.length > 0) {
|
||||
const lc = lynchpinCreds[Math.floor(random() * lynchpinCreds.length)];
|
||||
credPick = [lc, ...credPick.filter((c) => c.type !== lc.type)];
|
||||
}
|
||||
const baseSubIds = pickSubWorkflowIds(proj.id, proj.name, null, 1);
|
||||
|
||||
Reference in New Issue
Block a user