mirror of
https://github.com/n8n-io/n8n.git
synced 2026-10-11 22:50:06 +00:00
refactor: Group quality engineering workspaces by capability (#40007)
This commit is contained in:
@@ -17,7 +17,7 @@ clear, when to stay silent, what never to say — lives once in
|
|||||||
| `backend/` | Backend | `cli`, `@n8n/db`, `core`, `workflow`, node packages |
|
| `backend/` | Backend | `cli`, `@n8n/db`, `core`, `workflow`, node packages |
|
||||||
| `db-migrations/` | DB migrations | `@n8n/db` migrations + their tests in `cli` |
|
| `db-migrations/` | DB migrations | `@n8n/db` migrations + their tests in `cli` |
|
||||||
| `frontend/` | Frontend | `packages/frontend` |
|
| `frontend/` | Frontend | `packages/frontend` |
|
||||||
| `qa-dx/` | QA & DX | `.github`, `docker`, `scripts`, `patches`, `packages/testing`, the lint/test/TS config packages, baselines |
|
| `qa-dx/` | QA & DX | `.github`, `docker`, `scripts`, `patches`, `packages/quality`, the lint/test/TS config packages, baselines |
|
||||||
| `testing/` | Backend + Frontend | any package with a test suite |
|
| `testing/` | Backend + Frontend | any package with a test suite |
|
||||||
|
|
||||||
A directory maps to one agent unless, like `testing/`, the policy is identical
|
A directory maps to one agent unless, like `testing/`, the policy is identical
|
||||||
|
|||||||
@@ -13,7 +13,7 @@ Flag a diff that **adds** entries to any of these, and ask for the fix instead:
|
|||||||
|------|---------|
|
|------|---------|
|
||||||
| `.code-health-baseline.json` | `@n8n/code-health` violations |
|
| `.code-health-baseline.json` | `@n8n/code-health` violations |
|
||||||
| `.boundaries-baseline.json` | `turbo boundaries` issue count |
|
| `.boundaries-baseline.json` | `turbo boundaries` issue count |
|
||||||
| `packages/testing/playwright/.janitor-baseline.json` | Playwright janitor findings |
|
| `packages/quality/testing/playwright/.janitor-baseline.json` | Playwright janitor findings |
|
||||||
| `packages/cli/eslint.config.mjs` | the `misplaced-n8n-typeorm-import` and public-API allowlists, each captioned "NEVER add to this list" |
|
| `packages/cli/eslint.config.mjs` | the `misplaced-n8n-typeorm-import` and public-API allowlists, each captioned "NEVER add to this list" |
|
||||||
| `.code-health-baseline.json` (`lint-config-layering`) | package-wide rule downgrades left in package ESLint configs |
|
| `.code-health-baseline.json` (`lint-config-layering`) | package-wide rule downgrades left in package ESLint configs |
|
||||||
|
|
||||||
|
|||||||
@@ -35,7 +35,7 @@ fed from a `needs:` list. Flag:
|
|||||||
Jobs gated on a `ci-filter` filter only run when a matching file changed, so a
|
Jobs gated on a `ci-filter` filter only run when a matching file changed, so a
|
||||||
filter that is too narrow means the test quietly stops covering new code. When
|
filter that is too narrow means the test quietly stops covering new code. When
|
||||||
a PR adds a directory that an existing filter was meant to cover — a new package
|
a PR adds a directory that an existing filter was meant to cover — a new package
|
||||||
under `packages/testing/`, a new script under `.github/scripts/` — check the
|
under `packages/quality/`, a new script under `.github/scripts/` — check the
|
||||||
filter still matches it.
|
filter still matches it.
|
||||||
|
|
||||||
## Least privilege
|
## Least privilege
|
||||||
|
|||||||
@@ -32,7 +32,7 @@ Based on the affected area, pick the test layer and pattern:
|
|||||||
| CLI / API | Vitest integration | setupTestServer + supertest | `packages/cli/test/integration/` |
|
| CLI / API | Vitest integration | setupTestServer + supertest | `packages/cli/test/integration/` |
|
||||||
| Config | Vitest unit | GlobalConfig + Container | `packages/@n8n/config/src/configs/__tests__/` |
|
| Config | Vitest unit | GlobalConfig + Container | `packages/@n8n/config/src/configs/__tests__/` |
|
||||||
| Editor UI | Vitest | Vue Test Utils + Pinia | `packages/frontend/editor-ui/src/**/__tests__/` |
|
| Editor UI | Vitest | Vue Test Utils + Pinia | `packages/frontend/editor-ui/src/**/__tests__/` |
|
||||||
| E2E / Canvas | Playwright | Test containers + composables | `packages/testing/playwright/` |
|
| E2E / Canvas | Playwright | Test containers + composables | `packages/quality/testing/playwright/` |
|
||||||
|
|
||||||
## Step 3: Locate Source Files
|
## Step 3: Locate Source Files
|
||||||
|
|
||||||
|
|||||||
@@ -8,7 +8,7 @@ exclude:
|
|||||||
# Machine-generated instance-AI expectation traces. Contain ephemeral
|
# Machine-generated instance-AI expectation traces. Contain ephemeral
|
||||||
# credential reference IDs (nanoids) that re-trip the secrets scanner
|
# credential reference IDs (nanoids) that re-trip the secrets scanner
|
||||||
# on every regeneration. No secret material.
|
# on every regeneration. No secret material.
|
||||||
- packages/testing/playwright/expectations
|
- packages/quality/testing/playwright/expectations
|
||||||
|
|
||||||
# Test code repo-wide. Fixtures routinely contain synthetic PEM blocks,
|
# Test code repo-wide. Fixtures routinely contain synthetic PEM blocks,
|
||||||
# basic-auth strings, and deliberately fake keys (including the secret
|
# basic-auth strings, and deliberately fake keys (including the secret
|
||||||
|
|||||||
+17
-17
@@ -435,12 +435,12 @@
|
|||||||
"hash": "4a6882cb8d62"
|
"hash": "4a6882cb8d62"
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
"packages/@n8n/benchmark/package.json": [
|
"packages/quality/efficiency/scale/benchmark/package.json": [
|
||||||
{
|
{
|
||||||
"rule": "catalog-violations",
|
"rule": "catalog-violations",
|
||||||
"line": 38,
|
"line": 38,
|
||||||
"message": "@oclif/core appears in 4 packages with 2 different versions — add to pnpm-workspace.yaml catalog",
|
"message": "@oclif/core appears in 4 packages with 2 different versions — add to pnpm-workspace.yaml catalog",
|
||||||
"hash": "67f9d81d9528"
|
"hash": "e07bb1da47d7"
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
"packages/@n8n/cli/package.json": [
|
"packages/@n8n/cli/package.json": [
|
||||||
@@ -2043,48 +2043,48 @@
|
|||||||
"hash": "0f94a71d5ae8"
|
"hash": "0f94a71d5ae8"
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
"packages/testing/playwright/eslint.config.mjs": [
|
"packages/quality/testing/playwright/eslint.config.mjs": [
|
||||||
{
|
{
|
||||||
"rule": "lint-config-layering",
|
"rule": "lint-config-layering",
|
||||||
"line": 21,
|
"line": 21,
|
||||||
"message": "packages/testing/playwright turns '@typescript-eslint/no-unused-expressions' down to 'off' for the whole package.",
|
"message": "packages/quality/testing/playwright turns '@typescript-eslint/no-unused-expressions' down to 'off' for the whole package.",
|
||||||
"hash": "e2f93b5b591a"
|
"hash": "c789a142996b"
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
"rule": "lint-config-layering",
|
"rule": "lint-config-layering",
|
||||||
"line": 22,
|
"line": 22,
|
||||||
"message": "packages/testing/playwright turns '@typescript-eslint/no-use-before-define' down to 'off' for the whole package.",
|
"message": "packages/quality/testing/playwright turns '@typescript-eslint/no-use-before-define' down to 'off' for the whole package.",
|
||||||
"hash": "75d3ec1677ba"
|
"hash": "fcc3a69d8a2e"
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
"rule": "lint-config-layering",
|
"rule": "lint-config-layering",
|
||||||
"line": 23,
|
"line": 23,
|
||||||
"message": "packages/testing/playwright turns '@typescript-eslint/promise-function-async' down to 'off' for the whole package.",
|
"message": "packages/quality/testing/playwright turns '@typescript-eslint/promise-function-async' down to 'off' for the whole package.",
|
||||||
"hash": "35dbe9e44ada"
|
"hash": "5ab4c080de15"
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
"rule": "lint-config-layering",
|
"rule": "lint-config-layering",
|
||||||
"line": 24,
|
"line": 24,
|
||||||
"message": "packages/testing/playwright turns 'playwright/expect-expect' down to 'warn' for the whole package.",
|
"message": "packages/quality/testing/playwright turns 'playwright/expect-expect' down to 'warn' for the whole package.",
|
||||||
"hash": "e8b5f4b0f0dd"
|
"hash": "3a584940f6c7"
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
"rule": "lint-config-layering",
|
"rule": "lint-config-layering",
|
||||||
"line": 25,
|
"line": 25,
|
||||||
"message": "packages/testing/playwright turns 'playwright/max-nested-describe' down to 'warn' for the whole package.",
|
"message": "packages/quality/testing/playwright turns 'playwright/max-nested-describe' down to 'warn' for the whole package.",
|
||||||
"hash": "77d82ee8a802"
|
"hash": "8ea7fd515ed8"
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
"rule": "lint-config-layering",
|
"rule": "lint-config-layering",
|
||||||
"line": 27,
|
"line": 27,
|
||||||
"message": "packages/testing/playwright turns 'playwright/no-skipped-test' down to 'warn' for the whole package.",
|
"message": "packages/quality/testing/playwright turns 'playwright/no-skipped-test' down to 'warn' for the whole package.",
|
||||||
"hash": "14eb9eecd1b4"
|
"hash": "6ce1d38312f9"
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
"rule": "lint-config-layering",
|
"rule": "lint-config-layering",
|
||||||
"line": 78,
|
"line": 78,
|
||||||
"message": "packages/testing/playwright turns 'unicorn/filename-case' down to 'off' for the whole package.",
|
"message": "packages/quality/testing/playwright turns 'unicorn/filename-case' down to 'off' for the whole package.",
|
||||||
"hash": "de86b9645930"
|
"hash": "949214800425"
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
"packages/frontend/@n8n/design-system/eslint.config.mjs": [
|
"packages/frontend/@n8n/design-system/eslint.config.mjs": [
|
||||||
|
|||||||
+6
-4
@@ -26,7 +26,7 @@
|
|||||||
!docker/images/runners
|
!docker/images/runners
|
||||||
!docker/images/runners/n8n-task-runners.json
|
!docker/images/runners/n8n-task-runners.json
|
||||||
|
|
||||||
# === benchmark image (packages/@n8n/benchmark/Dockerfile) ===
|
# === benchmark image (packages/quality/efficiency/scale/benchmark/Dockerfile) ===
|
||||||
!package.json
|
!package.json
|
||||||
!pnpm-lock.yaml
|
!pnpm-lock.yaml
|
||||||
!pnpm-workspace.yaml
|
!pnpm-workspace.yaml
|
||||||
@@ -34,8 +34,10 @@
|
|||||||
!patches/**
|
!patches/**
|
||||||
!scripts
|
!scripts
|
||||||
!scripts/**
|
!scripts/**
|
||||||
!packages/@n8n/benchmark
|
!packages/quality
|
||||||
!packages/@n8n/benchmark/**
|
!packages/quality/efficiency
|
||||||
|
!packages/quality/efficiency/scale
|
||||||
|
!packages/quality/efficiency/scale/benchmark
|
||||||
|
!packages/quality/efficiency/scale/benchmark/**
|
||||||
!packages/@n8n/typescript-config
|
!packages/@n8n/typescript-config
|
||||||
!packages/@n8n/typescript-config/**
|
!packages/@n8n/typescript-config/**
|
||||||
|
|
||||||
|
|||||||
@@ -81,11 +81,11 @@ return 'blacksmith';
|
|||||||
|
|
||||||
| Telemetry | Source | Metrics |
|
| Telemetry | Source | Metrics |
|
||||||
|-----------|--------|---------|
|
|-----------|--------|---------|
|
||||||
| Playwright perf/benchmark | `packages/testing/playwright/reporters/metrics-reporter.ts` | Any metric attached via `attachMetric()` |
|
| Playwright perf/benchmark | `packages/quality/testing/playwright/reporters/metrics-reporter.ts` | Any metric attached via `attachMetric()` |
|
||||||
| Accessibility buckets | `packages/testing/playwright/reporters/a11y-reporter.ts` | Per-bucket axe score, violated rules, violating elements |
|
| Accessibility buckets | `packages/quality/testing/playwright/reporters/a11y-reporter.ts` | Per-bucket axe score, violated rules, violating elements |
|
||||||
| Build stats | `.github/scripts/send-build-stats.mjs` | Per-package build duration, cache hit/miss, run total |
|
| Build stats | `.github/scripts/send-build-stats.mjs` | Per-package build duration, cache hit/miss, run total |
|
||||||
| Docker stats | `.github/scripts/send-docker-stats.mjs` | Image size per platform, docker build duration |
|
| Docker stats | `.github/scripts/send-docker-stats.mjs` | Image size per platform, docker build duration |
|
||||||
| Container stack | `packages/testing/containers/telemetry.ts` | E2E stack startup times per service |
|
| Container stack | `packages/quality/environments/containers/telemetry.ts` | E2E stack startup times per service |
|
||||||
|
|
||||||
## Secrets
|
## Secrets
|
||||||
|
|
||||||
|
|||||||
@@ -178,7 +178,7 @@ These only run if specific files changed:
|
|||||||
| `docker/images/n8n-base/Dockerfile` | `build-base-image.yml` | any |
|
| `docker/images/n8n-base/Dockerfile` | `build-base-image.yml` | any |
|
||||||
| `**/package.json`, `**/turbo.json` | `build-windows.yml` | master |
|
| `**/package.json`, `**/turbo.json` | `build-windows.yml` | master |
|
||||||
| `packages/@n8n/ai-workflow-builder.ee/evaluations/programmatic/python/**` | `test-evals-python.yml` | any |
|
| `packages/@n8n/ai-workflow-builder.ee/evaluations/programmatic/python/**` | `test-evals-python.yml` | any |
|
||||||
| `packages/@n8n/benchmark/**` | `build-benchmark-image.yml` | master |
|
| `packages/quality/efficiency/scale/benchmark/**` | `build-benchmark-image.yml` | master |
|
||||||
| `packages/cli/src/public-api/**/*.yml`, `packages/cli/src/public-api/**/*.yaml`, `packages/cli/src/public-api/**/*.css`, `packages/cli/src/public-api/v1/openapi-gen/**/*.ts`, `packages/cli/scripts/build.mjs`, `packages/cli/package.json` | `util-publish-api-schema.yml` | master |
|
| `packages/cli/src/public-api/**/*.yml`, `packages/cli/src/public-api/**/*.yaml`, `packages/cli/src/public-api/**/*.css`, `packages/cli/src/public-api/v1/openapi-gen/**/*.ts`, `packages/cli/scripts/build.mjs`, `packages/cli/package.json` | `util-publish-api-schema.yml` | master |
|
||||||
| `packages/@n8n/instance-ai/src/**`, `packages/@n8n/instance-ai/skills/**`, `packages/@n8n/instance-ai/knowledge-base/**`, `packages/@n8n/instance-ai/evaluations/**`, `packages/cli/src/modules/instance-ai/**`, `packages/core/src/execution-engine/eval-mock-helpers.ts`, `packages/@n8n/agents/src/**` | `ci-instance-ai-evals.yml` | on PR `opened` / `reopened` / `ready_for_review` |
|
| `packages/@n8n/instance-ai/src/**`, `packages/@n8n/instance-ai/skills/**`, `packages/@n8n/instance-ai/knowledge-base/**`, `packages/@n8n/instance-ai/evaluations/**`, `packages/cli/src/modules/instance-ai/**`, `packages/core/src/execution-engine/eval-mock-helpers.ts`, `packages/@n8n/agents/src/**` | `ci-instance-ai-evals.yml` | on PR `opened` / `reopened` / `ready_for_review` |
|
||||||
| `docker/get-n8n.sh`, `docker/get-n8n-compose.yml`, `docker/test-get-n8n.sh` | `test-get-n8n.yml` | any |
|
| `docker/get-n8n.sh`, `docker/get-n8n-compose.yml`, `docker/test-get-n8n.sh` | `test-get-n8n.yml` | any |
|
||||||
|
|||||||
@@ -242,7 +242,7 @@ describe('runtime filter', () => {
|
|||||||
'!**/CHANGELOG.md',
|
'!**/CHANGELOG.md',
|
||||||
'!**/*.test.ts',
|
'!**/*.test.ts',
|
||||||
'!**/*.spec.ts',
|
'!**/*.spec.ts',
|
||||||
'!packages/testing/playwright/**',
|
'!packages/quality/testing/playwright/**',
|
||||||
'!packages/frontend/@n8n/storybook/**',
|
'!packages/frontend/@n8n/storybook/**',
|
||||||
'!scripts/agent-setup.mjs',
|
'!scripts/agent-setup.mjs',
|
||||||
'!scripts/backend-module/**',
|
'!scripts/backend-module/**',
|
||||||
@@ -265,7 +265,7 @@ describe('runtime filter', () => {
|
|||||||
|
|
||||||
it('does not trigger on playwright tests', () => {
|
it('does not trigger on playwright tests', () => {
|
||||||
assert.equal(
|
assert.equal(
|
||||||
evaluateFilter(['packages/testing/playwright/tests/x.spec.ts'], runtimePatterns),
|
evaluateFilter(['packages/quality/testing/playwright/tests/x.spec.ts'], runtimePatterns),
|
||||||
false,
|
false,
|
||||||
);
|
);
|
||||||
});
|
});
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
services:
|
services:
|
||||||
postgres:
|
postgres:
|
||||||
image: ${TEST_IMAGE_POSTGRES:?set it from packages/testing/containers/postgres-versions.json}
|
image: ${TEST_IMAGE_POSTGRES:?set it from packages/quality/environments/containers/postgres-versions.json}
|
||||||
restart: always
|
restart: always
|
||||||
environment:
|
environment:
|
||||||
- POSTGRES_DB=n8n
|
- POSTGRES_DB=n8n
|
||||||
|
|||||||
@@ -7,7 +7,7 @@ import { fileURLToPath } from 'node:url';
|
|||||||
|
|
||||||
const REPO_ROOT = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '../..');
|
const REPO_ROOT = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '../..');
|
||||||
|
|
||||||
export const POSTGRES_VERSIONS_PATH = 'packages/testing/containers/postgres-versions.json';
|
export const POSTGRES_VERSIONS_PATH = 'packages/quality/environments/containers/postgres-versions.json';
|
||||||
|
|
||||||
/** The matrix wall is bounded by its slowest leg, so a larger runner buys nothing. */
|
/** The matrix wall is bounded by its slowest leg, so a larger runner buys nothing. */
|
||||||
const RUNNER = 'blacksmith-4vcpu-ubuntu-2204';
|
const RUNNER = 'blacksmith-4vcpu-ubuntu-2204';
|
||||||
|
|||||||
@@ -30,9 +30,9 @@ describe('matchesGlob', () => {
|
|||||||
});
|
});
|
||||||
|
|
||||||
it('matches a prefix pattern', () => {
|
it('matches a prefix pattern', () => {
|
||||||
assert.equal(matchesGlob('packages/testing/playwright/a.ts', 'packages/testing/**'), true);
|
assert.equal(matchesGlob('packages/quality/testing/playwright/a.ts', 'packages/quality/**'), true);
|
||||||
assert.equal(matchesGlob('packages/testing', 'packages/testing/**'), false);
|
assert.equal(matchesGlob('packages/quality', 'packages/quality/**'), false);
|
||||||
assert.equal(matchesGlob('packages/cli/a.ts', 'packages/testing/**'), false);
|
assert.equal(matchesGlob('packages/cli/a.ts', 'packages/quality/**'), false);
|
||||||
});
|
});
|
||||||
|
|
||||||
it('matches a literal path only exactly', () => {
|
it('matches a literal path only exactly', () => {
|
||||||
@@ -55,8 +55,8 @@ describe('matchesGlob', () => {
|
|||||||
'packages/cli/__snapshots__/a.ts.snap',
|
'packages/cli/__snapshots__/a.ts.snap',
|
||||||
'packages/nodes-base/nodes/Foo/test/fixtures/a.json',
|
'packages/nodes-base/nodes/Foo/test/fixtures/a.json',
|
||||||
'packages/frontend/editor-ui/src/__mocks__/a.ts',
|
'packages/frontend/editor-ui/src/__mocks__/a.ts',
|
||||||
'packages/testing/playwright/a.ts',
|
'packages/quality/testing/playwright/a.ts',
|
||||||
'packages/testing',
|
'packages/quality',
|
||||||
'.github/scripts/glob.test.mjs',
|
'.github/scripts/glob.test.mjs',
|
||||||
'.github/workflows/ci.yml',
|
'.github/workflows/ci.yml',
|
||||||
'a/.hidden/b.spec.js',
|
'a/.hidden/b.spec.js',
|
||||||
|
|||||||
@@ -217,7 +217,7 @@ describe('computeLineStats', () => {
|
|||||||
['*.snap extension', 'src/foo.snap'],
|
['*.snap extension', 'src/foo.snap'],
|
||||||
['fixtures/ directory', 'src/fixtures/data.json'],
|
['fixtures/ directory', 'src/fixtures/data.json'],
|
||||||
['__mocks__/ directory', 'src/__mocks__/axios.ts'],
|
['__mocks__/ directory', 'src/__mocks__/axios.ts'],
|
||||||
['packages/testing/**', 'packages/testing/playwright/spec.ts'],
|
['packages/quality/**', 'packages/quality/testing/playwright/spec.ts'],
|
||||||
['dot-directory *.test.mjs', '.github/scripts/owners/owners.test.mjs'],
|
['dot-directory *.test.mjs', '.github/scripts/owners/owners.test.mjs'],
|
||||||
]) {
|
]) {
|
||||||
it(`classifies ${label} as testFiles`, () => {
|
it(`classifies ${label} as testFiles`, () => {
|
||||||
|
|||||||
@@ -36,8 +36,11 @@ export const TEST_PATTERNS = [
|
|||||||
// Fixtures and mocks
|
// Fixtures and mocks
|
||||||
'**/fixtures/**',
|
'**/fixtures/**',
|
||||||
'**/__mocks__/**',
|
'**/__mocks__/**',
|
||||||
// Dedicated testing package
|
// Former packages/testing workspaces
|
||||||
'packages/testing/**',
|
'packages/quality/policy/**',
|
||||||
|
'packages/quality/testing/**',
|
||||||
|
'packages/quality/environments/**',
|
||||||
|
'packages/quality/efficiency/microbenchmarks/**',
|
||||||
];
|
];
|
||||||
|
|
||||||
export const MISC_PATTERNS = [
|
export const MISC_PATTERNS = [
|
||||||
|
|||||||
@@ -203,14 +203,21 @@ describe('countFilteredAdditions', () => {
|
|||||||
assert.equal(countFilteredAdditions(files, EXCLUDE_PATTERNS), 50);
|
assert.equal(countFilteredAdditions(files, EXCLUDE_PATTERNS), 50);
|
||||||
});
|
});
|
||||||
|
|
||||||
it('applies EXCLUDE_PATTERNS to packages/testing and pnpm-lock.yaml', () => {
|
it('excludes former testing workspaces but counts benchmark CLI source', () => {
|
||||||
const files = [
|
const files = [
|
||||||
{ filename: 'packages/cli/src/service.ts', additions: 50 },
|
{ filename: 'packages/cli/src/service.ts', additions: 50 },
|
||||||
{ filename: 'packages/testing/playwright/tests/workflow.spec.ts', additions: 100 },
|
{ filename: 'packages/quality/testing/playwright/tests/workflow.spec.ts', additions: 100 },
|
||||||
{ filename: 'packages/testing/playwright/pages/CanvasPage.ts', additions: 100 },
|
{ filename: 'packages/quality/testing/playwright/pages/CanvasPage.ts', additions: 100 },
|
||||||
|
{ filename: 'packages/quality/policy/code-health/src/cli.ts', additions: 100 },
|
||||||
|
{ filename: 'packages/quality/environments/containers/service-stack.ts', additions: 100 },
|
||||||
|
{
|
||||||
|
filename: 'packages/quality/efficiency/microbenchmarks/benchmarks/bench-options.ts',
|
||||||
|
additions: 100,
|
||||||
|
},
|
||||||
|
{ filename: 'packages/quality/efficiency/scale/benchmark/src/commands/list.ts', additions: 100 },
|
||||||
{ filename: 'pnpm-lock.yaml', additions: 500 },
|
{ filename: 'pnpm-lock.yaml', additions: 500 },
|
||||||
];
|
];
|
||||||
assert.equal(countFilteredAdditions(files, EXCLUDE_PATTERNS), 50);
|
assert.equal(countFilteredAdditions(files, EXCLUDE_PATTERNS), 150);
|
||||||
});
|
});
|
||||||
|
|
||||||
it('applies EXCLUDE_PATTERNS to markdown files', () => {
|
it('applies EXCLUDE_PATTERNS to markdown files', () => {
|
||||||
|
|||||||
@@ -6,7 +6,7 @@ on:
|
|||||||
branches:
|
branches:
|
||||||
- master
|
- master
|
||||||
paths:
|
paths:
|
||||||
- 'packages/@n8n/benchmark/**'
|
- 'packages/quality/efficiency/scale/benchmark/**'
|
||||||
- 'pnpm-lock.yaml'
|
- 'pnpm-lock.yaml'
|
||||||
- 'pnpm-workspace.yaml'
|
- 'pnpm-workspace.yaml'
|
||||||
- '.github/workflows/build-benchmark-image.yml'
|
- '.github/workflows/build-benchmark-image.yml'
|
||||||
@@ -33,7 +33,7 @@ jobs:
|
|||||||
DOCKER_BUILD_SUMMARY: false
|
DOCKER_BUILD_SUMMARY: false
|
||||||
with:
|
with:
|
||||||
context: .
|
context: .
|
||||||
file: ./packages/@n8n/benchmark/Dockerfile
|
file: ./packages/quality/efficiency/scale/benchmark/Dockerfile
|
||||||
platforms: linux/amd64
|
platforms: linux/amd64
|
||||||
provenance: false
|
provenance: false
|
||||||
push: true
|
push: true
|
||||||
|
|||||||
@@ -99,8 +99,8 @@ jobs:
|
|||||||
mode: filter
|
mode: filter
|
||||||
filters: |
|
filters: |
|
||||||
janitor:
|
janitor:
|
||||||
packages/testing/playwright/**
|
packages/quality/testing/playwright/**
|
||||||
packages/testing/janitor/**
|
packages/quality/testing/janitor/**
|
||||||
|
|
||||||
check-static-analysis:
|
check-static-analysis:
|
||||||
name: Static Analysis
|
name: Static Analysis
|
||||||
|
|||||||
@@ -79,7 +79,7 @@ jobs:
|
|||||||
!**/CHANGELOG.md
|
!**/CHANGELOG.md
|
||||||
!**/*.test.ts
|
!**/*.test.ts
|
||||||
!**/*.spec.ts
|
!**/*.spec.ts
|
||||||
!packages/testing/playwright/**
|
!packages/quality/testing/playwright/**
|
||||||
!packages/frontend/@n8n/storybook/**
|
!packages/frontend/@n8n/storybook/**
|
||||||
!.devcontainer/**
|
!.devcontainer/**
|
||||||
!scripts/agent-setup.mjs
|
!scripts/agent-setup.mjs
|
||||||
@@ -92,7 +92,7 @@ jobs:
|
|||||||
unit:
|
unit:
|
||||||
**
|
**
|
||||||
!packages/@n8n/task-runner-python/**
|
!packages/@n8n/task-runner-python/**
|
||||||
!packages/testing/playwright/**
|
!packages/quality/testing/playwright/**
|
||||||
!.github/**
|
!.github/**
|
||||||
.github/workflows/test-unit-reusable.yml
|
.github/workflows/test-unit-reusable.yml
|
||||||
!**/*.md
|
!**/*.md
|
||||||
@@ -111,8 +111,8 @@ jobs:
|
|||||||
.github/workflows/prepare-docker-reusable.yml
|
.github/workflows/prepare-docker-reusable.yml
|
||||||
.github/actions/build-n8n-docker/**
|
.github/actions/build-n8n-docker/**
|
||||||
.github/actions/load-n8n-docker/**
|
.github/actions/load-n8n-docker/**
|
||||||
packages/testing/playwright/**
|
packages/quality/testing/playwright/**
|
||||||
packages/testing/containers/**
|
packages/quality/environments/containers/**
|
||||||
dev-server-smoke:
|
dev-server-smoke:
|
||||||
packages/frontend/editor-ui/vite.config.mts
|
packages/frontend/editor-ui/vite.config.mts
|
||||||
# If this job becomes too slow, use a smaller glob. Do not delete the assertion.
|
# If this job becomes too slow, use a smaller glob. Do not delete the assertion.
|
||||||
@@ -120,10 +120,10 @@ jobs:
|
|||||||
pnpm-workspace.yaml
|
pnpm-workspace.yaml
|
||||||
packages/@n8n/*/package.json
|
packages/@n8n/*/package.json
|
||||||
packages/frontend/@n8n/frontend-vite-config/**
|
packages/frontend/@n8n/frontend-vite-config/**
|
||||||
packages/testing/playwright/tests/dev-server-smoke/**
|
packages/quality/testing/playwright/tests/dev-server-smoke/**
|
||||||
packages/testing/playwright/playwright.config.ts
|
packages/quality/testing/playwright/playwright.config.ts
|
||||||
packages/testing/playwright/playwright-projects.ts
|
packages/quality/testing/playwright/playwright-projects.ts
|
||||||
packages/testing/playwright/package.json
|
packages/quality/testing/playwright/package.json
|
||||||
.github/workflows/test-dev-server-smoke-reusable.yml
|
.github/workflows/test-dev-server-smoke-reusable.yml
|
||||||
frontend-declarations:
|
frontend-declarations:
|
||||||
packages/frontend/@n8n/design-system/**
|
packages/frontend/@n8n/design-system/**
|
||||||
@@ -146,9 +146,9 @@ jobs:
|
|||||||
cubic.yaml
|
cubic.yaml
|
||||||
.agents/review-rules/**
|
.agents/review-rules/**
|
||||||
e2e-performance:
|
e2e-performance:
|
||||||
packages/testing/playwright/tests/performance/**
|
packages/quality/testing/playwright/tests/performance/**
|
||||||
packages/testing/playwright/utils/performance-helper.ts
|
packages/quality/testing/playwright/utils/performance-helper.ts
|
||||||
packages/testing/containers/**
|
packages/quality/environments/containers/**
|
||||||
.github/workflows/test-e2e-performance-reusable.yml
|
.github/workflows/test-e2e-performance-reusable.yml
|
||||||
# Whole directories, not the `engine-*` files alone. Engine v2
|
# Whole directories, not the `engine-*` files alone. Engine v2
|
||||||
# routing lives in ordinary control plane code: `workflow-runner.ts`
|
# routing lives in ordinary control plane code: `workflow-runner.ts`
|
||||||
@@ -169,8 +169,8 @@ jobs:
|
|||||||
packages/cli/src/webhooks/**
|
packages/cli/src/webhooks/**
|
||||||
packages/cli/src/workflows/triggers/**
|
packages/cli/src/workflows/triggers/**
|
||||||
packages/cli/src/workflow-runner.ts
|
packages/cli/src/workflow-runner.ts
|
||||||
packages/testing/containers/**
|
packages/quality/environments/containers/**
|
||||||
packages/testing/playwright/**
|
packages/quality/testing/playwright/**
|
||||||
instance-ai-workflow-eval:
|
instance-ai-workflow-eval:
|
||||||
packages/@n8n/instance-ai/src/**
|
packages/@n8n/instance-ai/src/**
|
||||||
packages/@n8n/instance-ai/evaluations/**
|
packages/@n8n/instance-ai/evaluations/**
|
||||||
@@ -193,10 +193,10 @@ jobs:
|
|||||||
packages/cli/src/**/*.integration.test.ts
|
packages/cli/src/**/*.integration.test.ts
|
||||||
packages/cli/src/modules/instance-registry/**
|
packages/cli/src/modules/instance-registry/**
|
||||||
packages/cli/vitest.*
|
packages/cli/vitest.*
|
||||||
packages/testing/containers/services/postgres.ts
|
packages/quality/environments/containers/services/postgres.ts
|
||||||
packages/testing/containers/services/redis.ts
|
packages/quality/environments/containers/services/redis.ts
|
||||||
packages/testing/containers/postgres-versions.json
|
packages/quality/environments/containers/postgres-versions.json
|
||||||
packages/testing/containers/test-containers.ts
|
packages/quality/environments/containers/test-containers.ts
|
||||||
.github/workflows/test-db-reusable.yml
|
.github/workflows/test-db-reusable.yml
|
||||||
.github/scripts/db-test-matrix.mjs
|
.github/scripts/db-test-matrix.mjs
|
||||||
docs/generated/**
|
docs/generated/**
|
||||||
@@ -214,10 +214,10 @@ jobs:
|
|||||||
packages/cli/vitest.config.base.ts
|
packages/cli/vitest.config.base.ts
|
||||||
packages/cli/vitest.config.migration.ts
|
packages/cli/vitest.config.migration.ts
|
||||||
packages/cli/vitest.config.migration.testcontainers.ts
|
packages/cli/vitest.config.migration.testcontainers.ts
|
||||||
packages/testing/containers/services/postgres.ts
|
packages/quality/environments/containers/services/postgres.ts
|
||||||
packages/testing/containers/services/redis.ts
|
packages/quality/environments/containers/services/redis.ts
|
||||||
packages/testing/containers/postgres-versions.json
|
packages/quality/environments/containers/postgres-versions.json
|
||||||
packages/testing/containers/test-containers.ts
|
packages/quality/environments/containers/test-containers.ts
|
||||||
.github/workflows/test-db-reusable.yml
|
.github/workflows/test-db-reusable.yml
|
||||||
.github/scripts/db-test-matrix.mjs
|
.github/scripts/db-test-matrix.mjs
|
||||||
|
|
||||||
@@ -247,7 +247,7 @@ jobs:
|
|||||||
env:
|
env:
|
||||||
CHANGED_FILES: ${{ steps.ci-filter.outputs.changed-files }}
|
CHANGED_FILES: ${{ steps.ci-filter.outputs.changed-files }}
|
||||||
run: |
|
run: |
|
||||||
PACKAGES=$(node packages/testing/janitor/dist/cli.js affected-packages | tr '\n' ' ' | sed 's/ *$//')
|
PACKAGES=$(node packages/quality/testing/janitor/dist/cli.js affected-packages | tr '\n' ' ' | sed 's/ *$//')
|
||||||
echo "Affected packages: $PACKAGES"
|
echo "Affected packages: $PACKAGES"
|
||||||
echo "list=$PACKAGES" >> "$GITHUB_OUTPUT"
|
echo "list=$PACKAGES" >> "$GITHUB_OUTPUT"
|
||||||
|
|
||||||
@@ -393,7 +393,7 @@ jobs:
|
|||||||
PLAYWRIGHT_PROJECT: ${{ github.event.pull_request.head.repo.fork == true && 'sqlite:e2e' || 'multi-main:e2e' }}
|
PLAYWRIGHT_PROJECT: ${{ github.event.pull_request.head.repo.fork == true && 'sqlite:e2e' || 'multi-main:e2e' }}
|
||||||
run: |
|
run: |
|
||||||
FILES_CSV=$(echo "$CHANGED_FILES" | tr '\n' ',' | sed 's/,$//')
|
FILES_CSV=$(echo "$CHANGED_FILES" | tr '\n' ',' | sed 's/,$//')
|
||||||
MATRIX=$(node packages/testing/playwright/scripts/distribute-tests.mjs --matrix 20 --orchestrate --impact "--project=$PLAYWRIGHT_PROJECT" "--files=$FILES_CSV" "--base=$MERGE_BASE")
|
MATRIX=$(node packages/quality/testing/playwright/scripts/distribute-tests.mjs --matrix 20 --orchestrate --impact "--project=$PLAYWRIGHT_PROJECT" "--files=$FILES_CSV" "--base=$MERGE_BASE")
|
||||||
echo "matrix=$MATRIX" >> "$GITHUB_OUTPUT"
|
echo "matrix=$MATRIX" >> "$GITHUB_OUTPUT"
|
||||||
echo "skip-tests=$(node -e "process.stdout.write(JSON.parse(process.argv[1])[0]?.skip === true ? 'true' : 'false')" "$MATRIX")" >> "$GITHUB_OUTPUT"
|
echo "skip-tests=$(node -e "process.stdout.write(JSON.parse(process.argv[1])[0]?.skip === true ? 'true' : 'false')" "$MATRIX")" >> "$GITHUB_OUTPUT"
|
||||||
|
|
||||||
|
|||||||
@@ -3,7 +3,7 @@ name: 'CI: Terraform'
|
|||||||
on:
|
on:
|
||||||
pull_request:
|
pull_request:
|
||||||
paths:
|
paths:
|
||||||
- packages/@n8n/benchmark/infra/**
|
- packages/quality/efficiency/scale/benchmark/infra/**
|
||||||
- .github/workflows/ci-terraform.yml
|
- .github/workflows/ci-terraform.yml
|
||||||
|
|
||||||
# The job only reads the repo and runs terraform. It writes nothing.
|
# The job only reads the repo and runs terraform. It writes nothing.
|
||||||
@@ -16,7 +16,7 @@ jobs:
|
|||||||
runs-on: ubuntu-latest
|
runs-on: ubuntu-latest
|
||||||
defaults:
|
defaults:
|
||||||
run:
|
run:
|
||||||
working-directory: packages/@n8n/benchmark/infra
|
working-directory: packages/quality/efficiency/scale/benchmark/infra
|
||||||
steps:
|
steps:
|
||||||
- name: Check out project
|
- name: Check out project
|
||||||
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
||||||
|
|||||||
@@ -29,10 +29,10 @@ on:
|
|||||||
# verify themselves (both modes) before the merge.
|
# verify themselves (both modes) before the merge.
|
||||||
pull_request:
|
pull_request:
|
||||||
paths:
|
paths:
|
||||||
- 'packages/testing/playwright/tests/infrastructure/encryption/**'
|
- 'packages/quality/testing/playwright/tests/infrastructure/encryption/**'
|
||||||
- 'packages/testing/containers/services/n8n.ts'
|
- 'packages/quality/environments/containers/services/n8n.ts'
|
||||||
- 'packages/testing/containers/services/types.ts'
|
- 'packages/quality/environments/containers/services/types.ts'
|
||||||
- 'packages/testing/containers/stack.ts'
|
- 'packages/quality/environments/containers/stack.ts'
|
||||||
- '.github/workflows/encryption-upgrade-test.yml'
|
- '.github/workflows/encryption-upgrade-test.yml'
|
||||||
|
|
||||||
permissions:
|
permissions:
|
||||||
|
|||||||
@@ -36,4 +36,4 @@ jobs:
|
|||||||
|
|
||||||
- name: Destroy cloud env
|
- name: Destroy cloud env
|
||||||
run: pnpm destroy-cloud-env
|
run: pnpm destroy-cloud-env
|
||||||
working-directory: packages/@n8n/benchmark
|
working-directory: packages/quality/efficiency/scale/benchmark
|
||||||
|
|||||||
@@ -62,11 +62,11 @@ jobs:
|
|||||||
|
|
||||||
- name: Destroy any existing environment
|
- name: Destroy any existing environment
|
||||||
run: pnpm destroy-cloud-env
|
run: pnpm destroy-cloud-env
|
||||||
working-directory: packages/@n8n/benchmark
|
working-directory: packages/quality/efficiency/scale/benchmark
|
||||||
|
|
||||||
- name: Provision the environment
|
- name: Provision the environment
|
||||||
run: pnpm provision-cloud-env ${{ env.DEBUG }}
|
run: pnpm provision-cloud-env ${{ env.DEBUG }}
|
||||||
working-directory: packages/@n8n/benchmark
|
working-directory: packages/quality/efficiency/scale/benchmark
|
||||||
|
|
||||||
- name: Run the benchmark
|
- name: Run the benchmark
|
||||||
id: benchmark
|
id: benchmark
|
||||||
@@ -81,7 +81,7 @@ jobs:
|
|||||||
--n8nTag ${{ env.N8N_TAG }} \
|
--n8nTag ${{ env.N8N_TAG }} \
|
||||||
--benchmarkTag ${{ env.N8N_BENCHMARK_TAG }} \
|
--benchmarkTag ${{ env.N8N_BENCHMARK_TAG }} \
|
||||||
${{ env.DEBUG }}
|
${{ env.DEBUG }}
|
||||||
working-directory: packages/@n8n/benchmark
|
working-directory: packages/quality/efficiency/scale/benchmark
|
||||||
|
|
||||||
# We need to login again because the access token expires
|
# We need to login again because the access token expires
|
||||||
- name: Azure login
|
- name: Azure login
|
||||||
@@ -95,7 +95,7 @@ jobs:
|
|||||||
- name: Destroy the environment
|
- name: Destroy the environment
|
||||||
if: always()
|
if: always()
|
||||||
run: pnpm destroy-cloud-env ${{ env.DEBUG }}
|
run: pnpm destroy-cloud-env ${{ env.DEBUG }}
|
||||||
working-directory: packages/@n8n/benchmark
|
working-directory: packages/quality/efficiency/scale/benchmark
|
||||||
|
|
||||||
- name: Fail `build` job if `benchmark` step failed
|
- name: Fail `build` job if `benchmark` step failed
|
||||||
if: steps.benchmark.outcome == 'failure'
|
if: steps.benchmark.outcome == 'failure'
|
||||||
|
|||||||
@@ -10,7 +10,7 @@ on:
|
|||||||
|
|
||||||
env:
|
env:
|
||||||
NODE_OPTIONS: '--max-old-space-size=6144'
|
NODE_OPTIONS: '--max-old-space-size=6144'
|
||||||
PLAYWRIGHT_BROWSERS_PATH: packages/testing/playwright/.playwright-browsers
|
PLAYWRIGHT_BROWSERS_PATH: packages/quality/testing/playwright/.playwright-browsers
|
||||||
|
|
||||||
jobs:
|
jobs:
|
||||||
smoke:
|
smoke:
|
||||||
@@ -53,6 +53,6 @@ jobs:
|
|||||||
with:
|
with:
|
||||||
name: dev-server-smoke-report
|
name: dev-server-smoke-report
|
||||||
path: |
|
path: |
|
||||||
packages/testing/playwright/test-results/
|
packages/quality/testing/playwright/test-results/
|
||||||
packages/testing/playwright/playwright-report/
|
packages/quality/testing/playwright/playwright-report/
|
||||||
retention-days: 7
|
retention-days: 7
|
||||||
|
|||||||
@@ -53,7 +53,7 @@ jobs:
|
|||||||
- name: Generate matrix (up to 20 shards, duration-weighted)
|
- name: Generate matrix (up to 20 shards, duration-weighted)
|
||||||
id: gen
|
id: gen
|
||||||
run: |
|
run: |
|
||||||
MATRIX=$(node packages/testing/playwright/scripts/distribute-tests.mjs --matrix 20 --orchestrate --project=coverage)
|
MATRIX=$(node packages/quality/testing/playwright/scripts/distribute-tests.mjs --matrix 20 --orchestrate --project=coverage)
|
||||||
echo "matrix=$MATRIX" >> "$GITHUB_OUTPUT"
|
echo "matrix=$MATRIX" >> "$GITHUB_OUTPUT"
|
||||||
|
|
||||||
e2e:
|
e2e:
|
||||||
@@ -215,7 +215,7 @@ jobs:
|
|||||||
# Merges all lcovs into one lcov.info (SF: paths normalised to repo-root-
|
# Merges all lcovs into one lcov.info (SF: paths normalised to repo-root-
|
||||||
# relative) and builds the per-spec E2E impact map for PR selection.
|
# relative) and builds the per-spec E2E impact map for PR selection.
|
||||||
- name: Aggregate coverage + build impact map
|
- name: Aggregate coverage + build impact map
|
||||||
working-directory: packages/testing/playwright
|
working-directory: packages/quality/testing/playwright
|
||||||
run: |
|
run: |
|
||||||
node scripts/aggregate-coverage.mjs \
|
node scripts/aggregate-coverage.mjs \
|
||||||
--shards=/tmp/shards \
|
--shards=/tmp/shards \
|
||||||
@@ -227,7 +227,7 @@ jobs:
|
|||||||
uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0
|
uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0
|
||||||
with:
|
with:
|
||||||
name: coverage-report
|
name: coverage-report
|
||||||
path: packages/testing/playwright/coverage/
|
path: packages/quality/testing/playwright/coverage/
|
||||||
retention-days: 14
|
retention-days: 14
|
||||||
|
|
||||||
- name: Upload Combined Coverage to Codecov
|
- name: Upload Combined Coverage to Codecov
|
||||||
@@ -235,7 +235,7 @@ jobs:
|
|||||||
uses: codecov/codecov-action@671740ac38dd9b0130fbe1cec585b89eea48d3de # v5.5.2
|
uses: codecov/codecov-action@671740ac38dd9b0130fbe1cec585b89eea48d3de # v5.5.2
|
||||||
with:
|
with:
|
||||||
token: ${{ secrets.CODECOV_TOKEN }}
|
token: ${{ secrets.CODECOV_TOKEN }}
|
||||||
files: packages/testing/playwright/coverage/lcov.info
|
files: packages/quality/testing/playwright/coverage/lcov.info
|
||||||
flags: nightly-full
|
flags: nightly-full
|
||||||
name: nightly-full-coverage
|
name: nightly-full-coverage
|
||||||
fail_ci_if_error: false
|
fail_ci_if_error: false
|
||||||
@@ -256,7 +256,7 @@ jobs:
|
|||||||
env:
|
env:
|
||||||
CODECOV_API_TOKEN: ${{ secrets.CODECOV_API_TOKEN }}
|
CODECOV_API_TOKEN: ${{ secrets.CODECOV_API_TOKEN }}
|
||||||
run: |
|
run: |
|
||||||
node packages/testing/playwright/scripts/coverage-analysis.mjs \
|
node packages/quality/testing/playwright/scripts/coverage-analysis.mjs \
|
||||||
--md --top=15 --out-json=coverage-gaps.json >> "$GITHUB_STEP_SUMMARY"
|
--md --top=15 --out-json=coverage-gaps.json >> "$GITHUB_STEP_SUMMARY"
|
||||||
|
|
||||||
- name: Upload Coverage Gap Report
|
- name: Upload Coverage Gap Report
|
||||||
|
|||||||
@@ -5,8 +5,8 @@ on:
|
|||||||
pull_request:
|
pull_request:
|
||||||
paths:
|
paths:
|
||||||
- '.github/workflows/test-e2e-helm.yml'
|
- '.github/workflows/test-e2e-helm.yml'
|
||||||
- 'packages/testing/containers/helm-stack.ts'
|
- 'packages/quality/environments/containers/helm-stack.ts'
|
||||||
- 'packages/testing/containers/helm-start-stack.ts'
|
- 'packages/quality/environments/containers/helm-start-stack.ts'
|
||||||
workflow_dispatch:
|
workflow_dispatch:
|
||||||
inputs:
|
inputs:
|
||||||
branch:
|
branch:
|
||||||
@@ -83,7 +83,7 @@ jobs:
|
|||||||
build-command: 'pnpm build'
|
build-command: 'pnpm build'
|
||||||
|
|
||||||
- name: Install Browsers
|
- name: Install Browsers
|
||||||
working-directory: packages/testing/playwright
|
working-directory: packages/quality/testing/playwright
|
||||||
run: pnpm exec playwright install chromium
|
run: pnpm exec playwright install chromium
|
||||||
|
|
||||||
- name: Install Helm
|
- name: Install Helm
|
||||||
@@ -115,7 +115,7 @@ jobs:
|
|||||||
echo "KUBECONFIG=${KUBECONFIG_PATH}" >> "$GITHUB_ENV"
|
echo "KUBECONFIG=${KUBECONFIG_PATH}" >> "$GITHUB_ENV"
|
||||||
|
|
||||||
- name: Run Building Blocks E2E Tests
|
- name: Run Building Blocks E2E Tests
|
||||||
working-directory: packages/testing/playwright
|
working-directory: packages/quality/testing/playwright
|
||||||
run: |
|
run: |
|
||||||
N8N_BASE_URL="${{ env.N8N_BASE_URL }}" \
|
N8N_BASE_URL="${{ env.N8N_BASE_URL }}" \
|
||||||
RESET_E2E_DB=true \
|
RESET_E2E_DB=true \
|
||||||
@@ -150,8 +150,8 @@ jobs:
|
|||||||
with:
|
with:
|
||||||
name: helm-e2e-report-${{ inputs.mode || 'queue' }}
|
name: helm-e2e-report-${{ inputs.mode || 'queue' }}
|
||||||
path: |
|
path: |
|
||||||
packages/testing/playwright/test-results/
|
packages/quality/testing/playwright/test-results/
|
||||||
packages/testing/playwright/playwright-report/
|
packages/quality/testing/playwright/playwright-report/
|
||||||
retention-days: 7
|
retention-days: 7
|
||||||
|
|
||||||
cleanup-ghcr:
|
cleanup-ghcr:
|
||||||
|
|||||||
@@ -74,7 +74,7 @@ jobs:
|
|||||||
# The aggregator auto-detects n8n version and commit SHA from the repo
|
# The aggregator auto-detects n8n version and commit SHA from the repo
|
||||||
# (anchored to its own file location), so no path/version glue is needed.
|
# (anchored to its own file location), so no path/version glue is needed.
|
||||||
- name: Aggregate sizing matrix
|
- name: Aggregate sizing matrix
|
||||||
working-directory: packages/testing/playwright
|
working-directory: packages/quality/testing/playwright
|
||||||
run: |
|
run: |
|
||||||
pnpm exec tsx scripts/sizing-matrix-aggregate.ts \
|
pnpm exec tsx scripts/sizing-matrix-aggregate.ts \
|
||||||
--input /tmp/benchmark-reports \
|
--input /tmp/benchmark-reports \
|
||||||
@@ -86,8 +86,8 @@ jobs:
|
|||||||
with:
|
with:
|
||||||
name: sizing-matrix
|
name: sizing-matrix
|
||||||
path: |
|
path: |
|
||||||
packages/testing/playwright/sizing-matrix.json
|
packages/quality/testing/playwright/sizing-matrix.json
|
||||||
packages/testing/playwright/sizing-matrix.md
|
packages/quality/testing/playwright/sizing-matrix.md
|
||||||
retention-days: 14
|
retention-days: 14
|
||||||
if-no-files-found: error
|
if-no-files-found: error
|
||||||
|
|
||||||
|
|||||||
@@ -77,7 +77,7 @@ jobs:
|
|||||||
- name: Generate matrix (up to 20 shards, duration-weighted)
|
- name: Generate matrix (up to 20 shards, duration-weighted)
|
||||||
id: gen
|
id: gen
|
||||||
run: |
|
run: |
|
||||||
MATRIX=$(node packages/testing/playwright/scripts/distribute-tests.mjs --matrix 20 --orchestrate --project=sqlite:e2e)
|
MATRIX=$(node packages/quality/testing/playwright/scripts/distribute-tests.mjs --matrix 20 --orchestrate --project=sqlite:e2e)
|
||||||
echo "matrix=$MATRIX" >> "$GITHUB_OUTPUT"
|
echo "matrix=$MATRIX" >> "$GITHUB_OUTPUT"
|
||||||
|
|
||||||
e2e:
|
e2e:
|
||||||
|
|||||||
@@ -37,7 +37,7 @@ jobs:
|
|||||||
- name: Checkout
|
- name: Checkout
|
||||||
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
||||||
with:
|
with:
|
||||||
sparse-checkout: packages/testing/playwright/scripts/canvas-perf-sentinels.mjs
|
sparse-checkout: packages/quality/testing/playwright/scripts/canvas-perf-sentinels.mjs
|
||||||
sparse-checkout-cone-mode: false
|
sparse-checkout-cone-mode: false
|
||||||
|
|
||||||
- name: Download performance shard artifacts
|
- name: Download performance shard artifacts
|
||||||
@@ -54,4 +54,4 @@ jobs:
|
|||||||
echo "::warning::No test-results.json found in performance artifacts — sentinel skipped."
|
echo "::warning::No test-results.json found in performance artifacts — sentinel skipped."
|
||||||
exit 0
|
exit 0
|
||||||
fi
|
fi
|
||||||
node packages/testing/playwright/scripts/canvas-perf-sentinels.mjs "${RESULTS_PATHS[@]}"
|
node packages/quality/testing/playwright/scripts/canvas-perf-sentinels.mjs "${RESULTS_PATHS[@]}"
|
||||||
|
|||||||
@@ -69,7 +69,7 @@ env:
|
|||||||
# Browser cache location - must match install-browsers script.
|
# Browser cache location - must match install-browsers script.
|
||||||
# Absolute so nested pnpm invocations (e.g. coverage shard runner) don't
|
# Absolute so nested pnpm invocations (e.g. coverage shard runner) don't
|
||||||
# re-resolve a relative path against the package's INIT_CWD and double it.
|
# re-resolve a relative path against the package's INIT_CWD and double it.
|
||||||
PLAYWRIGHT_BROWSERS_PATH: ${{ github.workspace }}/packages/testing/playwright/.playwright-browsers
|
PLAYWRIGHT_BROWSERS_PATH: ${{ github.workspace }}/packages/quality/testing/playwright/.playwright-browsers
|
||||||
# docker-artifact loads the image locally as n8nio/n8n:local; unused in local mode.
|
# docker-artifact loads the image locally as n8nio/n8n:local; unused in local mode.
|
||||||
TEST_IMAGE_N8N: 'n8nio/n8n:local'
|
TEST_IMAGE_N8N: 'n8nio/n8n:local'
|
||||||
N8N_SKIP_LICENSES: 'true'
|
N8N_SKIP_LICENSES: 'true'
|
||||||
@@ -123,7 +123,7 @@ jobs:
|
|||||||
GITHUB_RUN_ID: ${{ github.run_id }}
|
GITHUB_RUN_ID: ${{ github.run_id }}
|
||||||
GITHUB_RUN_ATTEMPT: ${{ github.run_attempt }}
|
GITHUB_RUN_ATTEMPT: ${{ github.run_attempt }}
|
||||||
run: |
|
run: |
|
||||||
FILTERED=$(echo "$CANDIDATES" | node packages/testing/janitor/dist/cli.js filter-shard | tr '\n' ' ' | sed 's/ *$//')
|
FILTERED=$(echo "$CANDIDATES" | node packages/quality/testing/janitor/dist/cli.js filter-shard | tr '\n' ' ' | sed 's/ *$//')
|
||||||
echo "specs=$FILTERED" >> "$GITHUB_OUTPUT"
|
echo "specs=$FILTERED" >> "$GITHUB_OUTPUT"
|
||||||
|
|
||||||
- name: Resolve shard args
|
- name: Resolve shard args
|
||||||
@@ -195,13 +195,13 @@ jobs:
|
|||||||
with:
|
with:
|
||||||
name: ${{ inputs.artifact-prefix }}-shard-${{ matrix.shard }}
|
name: ${{ inputs.artifact-prefix }}-shard-${{ matrix.shard }}
|
||||||
path: |
|
path: |
|
||||||
packages/testing/playwright/test-results/
|
packages/quality/testing/playwright/test-results/
|
||||||
packages/testing/playwright/test-results.json
|
packages/quality/testing/playwright/test-results.json
|
||||||
packages/testing/playwright/playwright-report/
|
packages/quality/testing/playwright/playwright-report/
|
||||||
packages/testing/playwright/coverage/lcov.info
|
packages/quality/testing/playwright/coverage/lcov.info
|
||||||
packages/testing/playwright/coverage/by-spec/
|
packages/quality/testing/playwright/coverage/by-spec/
|
||||||
packages/testing/playwright/coverage/index.html
|
packages/quality/testing/playwright/coverage/index.html
|
||||||
packages/testing/playwright/coverage/assets/
|
packages/quality/testing/playwright/coverage/assets/
|
||||||
retention-days: 1
|
retention-days: 1
|
||||||
if-no-files-found: ignore
|
if-no-files-found: ignore
|
||||||
|
|
||||||
@@ -214,7 +214,7 @@ jobs:
|
|||||||
uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0
|
uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0
|
||||||
with:
|
with:
|
||||||
name: ${{ inputs.artifact-prefix }}-a11y-report-shard-${{ matrix.shard }}
|
name: ${{ inputs.artifact-prefix }}-a11y-report-shard-${{ matrix.shard }}
|
||||||
path: packages/testing/playwright/a11y-report/
|
path: packages/quality/testing/playwright/a11y-report/
|
||||||
retention-days: 7
|
retention-days: 7
|
||||||
if-no-files-found: ignore
|
if-no-files-found: ignore
|
||||||
|
|
||||||
|
|||||||
@@ -90,13 +90,13 @@ jobs:
|
|||||||
env:
|
env:
|
||||||
BASE_BRANCH: ${{ inputs.base-branch }}
|
BASE_BRANCH: ${{ inputs.base-branch }}
|
||||||
run: |
|
run: |
|
||||||
if [ -f packages/testing/code-health/package.json ]; then
|
if [ -f packages/quality/policy/code-health/package.json ]; then
|
||||||
echo 'present=true' >> "$GITHUB_OUTPUT"
|
echo 'present=true' >> "$GITHUB_OUTPUT"
|
||||||
elif [ -n "$BASE_BRANCH" ]; then
|
elif [ -n "$BASE_BRANCH" ]; then
|
||||||
echo 'present=false' >> "$GITHUB_OUTPUT"
|
echo 'present=false' >> "$GITHUB_OUTPUT"
|
||||||
echo "::notice::packages/testing/code-health is absent on \"$BASE_BRANCH\"; nothing to verify."
|
echo "::notice::packages/quality/policy/code-health is absent on \"$BASE_BRANCH\"; nothing to verify."
|
||||||
else
|
else
|
||||||
echo '::error::packages/testing/code-health is missing (moved or renamed?).'
|
echo '::error::packages/quality/policy/code-health is missing (moved or renamed?).'
|
||||||
exit 1
|
exit 1
|
||||||
fi
|
fi
|
||||||
|
|
||||||
@@ -119,10 +119,10 @@ jobs:
|
|||||||
# REPORT_ONLY is a single flag or empty. Quoting it would pass an empty argument, which
|
# REPORT_ONLY is a single flag or empty. Quoting it would pass an empty argument, which
|
||||||
# the CLI reads as an (unknown) package name and rejects.
|
# the CLI reads as an (unknown) package name and rejects.
|
||||||
# shellcheck disable=SC2086
|
# shellcheck disable=SC2086
|
||||||
pnpm --dir packages/testing/code-health exec tsx src/cli.ts verify-npm-install --changed="$BASE_REF" $REPORT_ONLY
|
pnpm --dir packages/quality/policy/code-health exec tsx src/cli.ts verify-npm-install --changed="$BASE_REF" $REPORT_ONLY
|
||||||
|
|
||||||
- name: Verify npm-install closure (all packages)
|
- name: Verify npm-install closure (all packages)
|
||||||
if: steps.verifier.outputs.present == 'true' && inputs.scope == 'all'
|
if: steps.verifier.outputs.present == 'true' && inputs.scope == 'all'
|
||||||
run: |
|
run: |
|
||||||
# shellcheck disable=SC2086 # see the scoped step above
|
# shellcheck disable=SC2086 # see the scoped step above
|
||||||
pnpm --dir packages/testing/code-health exec tsx src/cli.ts verify-npm-install --all $REPORT_ONLY
|
pnpm --dir packages/quality/policy/code-health exec tsx src/cli.ts verify-npm-install --all $REPORT_ONLY
|
||||||
|
|||||||
@@ -71,7 +71,7 @@ jobs:
|
|||||||
- name: Resolve Postgres image
|
- name: Resolve Postgres image
|
||||||
if: fromJSON(steps.paths-filter.outputs.results).db
|
if: fromJSON(steps.paths-filter.outputs.results).db
|
||||||
run: |
|
run: |
|
||||||
IMAGE=$(node -p "require('./packages/testing/containers/postgres-versions.json').primary")
|
IMAGE=$(node -p "require('./packages/quality/environments/containers/postgres-versions.json').primary")
|
||||||
echo "TEST_IMAGE_POSTGRES=$IMAGE" >> "$GITHUB_ENV"
|
echo "TEST_IMAGE_POSTGRES=$IMAGE" >> "$GITHUB_ENV"
|
||||||
|
|
||||||
- name: Start Postgres
|
- name: Start Postgres
|
||||||
|
|||||||
+9
-9
@@ -46,17 +46,17 @@ sbom-source.cdx.json
|
|||||||
junit.xml
|
junit.xml
|
||||||
test-results.json
|
test-results.json
|
||||||
*.0x
|
*.0x
|
||||||
packages/testing/playwright/playwright-report
|
packages/quality/testing/playwright/playwright-report
|
||||||
packages/testing/playwright/eval-report
|
packages/quality/testing/playwright/eval-report
|
||||||
packages/testing/playwright/a11y-report
|
packages/quality/testing/playwright/a11y-report
|
||||||
packages/testing/playwright/test-results
|
packages/quality/testing/playwright/test-results
|
||||||
packages/testing/playwright/results.xml
|
packages/quality/testing/playwright/results.xml
|
||||||
packages/testing/playwright/eval-results.json
|
packages/quality/testing/playwright/eval-results.json
|
||||||
packages/@n8n/instance-ai/eval-results.json
|
packages/@n8n/instance-ai/eval-results.json
|
||||||
packages/@n8n/instance-ai/.eval-output/
|
packages/@n8n/instance-ai/.eval-output/
|
||||||
packages/@n8n/instance-ai/eval-pr-comment.md
|
packages/@n8n/instance-ai/eval-pr-comment.md
|
||||||
packages/testing/playwright/.playwright-browsers
|
packages/quality/testing/playwright/.playwright-browsers
|
||||||
packages/testing/playwright/.playwright-cli
|
packages/quality/testing/playwright/.playwright-cli
|
||||||
test-results/
|
test-results/
|
||||||
compiled_app_output
|
compiled_app_output
|
||||||
trivy_report*
|
trivy_report*
|
||||||
@@ -65,7 +65,7 @@ docker-build-manifest.json
|
|||||||
packages/cli/src/modules/my-feature
|
packages/cli/src/modules/my-feature
|
||||||
.secrets
|
.secrets
|
||||||
act-event.json
|
act-event.json
|
||||||
packages/testing/**/.cursor/rules/
|
packages/quality/**/.cursor/rules/
|
||||||
.venv
|
.venv
|
||||||
.ruff_cache
|
.ruff_cache
|
||||||
__pycache__
|
__pycache__
|
||||||
|
|||||||
@@ -310,14 +310,14 @@ extends one of those layers):
|
|||||||
- Run that package's `pnpm lint` and `pnpm typecheck` before committing code.
|
- Run that package's `pnpm lint` and `pnpm typecheck` before committing code.
|
||||||
Build first when shared types or cross-package dependencies change.
|
Build first when shared types or cross-package dependencies change.
|
||||||
- Use Vitest for unit tests. Use
|
- Use Vitest for unit tests. Use
|
||||||
[Playwright](packages/testing/playwright/AGENTS.md) when a test needs its
|
[Playwright](packages/quality/testing/playwright/AGENTS.md) when a test needs its
|
||||||
browser, fixtures, or managed containers.
|
browser, fixtures, or managed containers.
|
||||||
- For Vitest packages with `@n8n/di` decorators, use
|
- For Vitest packages with `@n8n/di` decorators, use
|
||||||
`createVitestConfigWithDecorators` from `@n8n/vitest-config/node-decorators`.
|
`createVitestConfigWithDecorators` from `@n8n/vitest-config/node-decorators`.
|
||||||
- Check import and mock side effects before running tests. Keep tests out of
|
- Check import and mock side effects before running tests. Keep tests out of
|
||||||
user-owned directories. Set `N8N_USER_FOLDER` to a test-owned directory before
|
user-owned directories. Set `N8N_USER_FOLDER` to a test-owned directory before
|
||||||
importing n8n settings. Clean up only paths that the test created.
|
importing n8n settings. Clean up only paths that the test created.
|
||||||
- CI runs [`@n8n/code-health`](packages/testing/code-health/README.md) static
|
- CI runs [`@n8n/code-health`](packages/quality/policy/code-health/README.md) static
|
||||||
analysis on PRs. It checks monorepo rules, including dependency hygiene and
|
analysis on PRs. It checks monorepo rules, including dependency hygiene and
|
||||||
encryption-boundary coverage.
|
encryption-boundary coverage.
|
||||||
|
|
||||||
@@ -331,7 +331,7 @@ extends one of those layers):
|
|||||||
| Start a Codespace backend and share its port | `pnpm dev:up` |
|
| Start a Codespace backend and share its port | `pnpm dev:up` |
|
||||||
|
|
||||||
The root `pnpm dev` command does not start a server. See the
|
The root `pnpm dev` command does not start a server. See the
|
||||||
[Playwright guide](packages/testing/playwright/README.md) and the
|
[Playwright guide](packages/quality/testing/playwright/README.md) and the
|
||||||
[Codespaces guide](.devcontainer/codespaces/README.md) for details.
|
[Codespaces guide](.devcontainer/codespaces/README.md) for details.
|
||||||
|
|
||||||
### Common Development Tasks
|
### Common Development Tasks
|
||||||
|
|||||||
+2
-2
@@ -432,7 +432,7 @@ Each instance gets its own port, and the webhook URL matches the main URL. Multi
|
|||||||
>- This does not work with Podman out of the box. You need to [configure Testcontainers for Podman](https://podman-desktop.io/tutorial/testcontainers-with-podman) first.
|
>- This does not work with Podman out of the box. You need to [configure Testcontainers for Podman](https://podman-desktop.io/tutorial/testcontainers-with-podman) first.
|
||||||
>- Alternatively, you can use colima and set DOCKER_HOST and TESTCONTAINERS_DOCKER_SOCKET_OVERRIDE environment variables as described [here](https://node.testcontainers.org/supported-container-runtimes/#colima)
|
>- Alternatively, you can use colima and set DOCKER_HOST and TESTCONTAINERS_DOCKER_SOCKET_OVERRIDE environment variables as described [here](https://node.testcontainers.org/supported-container-runtimes/#colima)
|
||||||
|
|
||||||
Refer to [packages/testing/containers/README.md](packages/testing/containers/README.md) for more information.
|
Refer to [packages/quality/environments/containers/README.md](packages/quality/environments/containers/README.md) for more information.
|
||||||
|
|
||||||
### Work locally with syslog
|
### Work locally with syslog
|
||||||
|
|
||||||
@@ -661,7 +661,7 @@ E2E tests can be started via one of the following commands:
|
|||||||
- `pnpm --filter=n8n-playwright test:local --ui` - Run tests in interactive UI mode (useful for debugging)
|
- `pnpm --filter=n8n-playwright test:local --ui` - Run tests in interactive UI mode (useful for debugging)
|
||||||
- `pnpm --filter=n8n-playwright test:local --grep="test-name"` - Run specific tests matching pattern
|
- `pnpm --filter=n8n-playwright test:local --grep="test-name"` - Run specific tests matching pattern
|
||||||
|
|
||||||
See `packages/testing/playwright/README.md` for more test commands and `packages/testing/playwright/CONTRIBUTING.md` for writing guidelines.
|
See `packages/quality/testing/playwright/README.md` for more test commands and `packages/quality/testing/playwright/CONTRIBUTING.md` for writing guidelines.
|
||||||
|
|
||||||
## Create custom nodes
|
## Create custom nodes
|
||||||
|
|
||||||
|
|||||||
@@ -254,8 +254,7 @@ docker/get-n8n.sh @n8n-io/qa-dx required
|
|||||||
|
|
||||||
# QA
|
# QA
|
||||||
|
|
||||||
packages/testing/ @n8n-io/qa-dx
|
packages/quality/ @n8n-io/qa-dx
|
||||||
packages/@n8n/benchmark/ @n8n-io/qa-dx
|
|
||||||
packages/@n8n/vitest-config/ @n8n-io/qa-dx
|
packages/@n8n/vitest-config/ @n8n-io/qa-dx
|
||||||
|
|
||||||
# Encryption boundary (IAM) — the encryption area and its guardrail rules.
|
# Encryption boundary (IAM) — the encryption area and its guardrail rules.
|
||||||
|
|||||||
@@ -18,6 +18,7 @@
|
|||||||
"**/cl100k_base.json",
|
"**/cl100k_base.json",
|
||||||
"**/o200k_base.json",
|
"**/o200k_base.json",
|
||||||
"**/*.generated.ts",
|
"**/*.generated.ts",
|
||||||
|
"**/benchmark/scripts/mock-api/**",
|
||||||
"**/expectations/**"
|
"**/expectations/**"
|
||||||
]
|
]
|
||||||
},
|
},
|
||||||
|
|||||||
+1
-1
@@ -150,5 +150,5 @@ ignore:
|
|||||||
- (?s:.*/migrations/(postgresdb|sqlite|mysqldb)/.*\.ts.*)\Z
|
- (?s:.*/migrations/(postgresdb|sqlite|mysqldb)/.*\.ts.*)\Z
|
||||||
- (?s:.*/scripts/.*)\Z
|
- (?s:.*/scripts/.*)\Z
|
||||||
# Test/build infrastructure packages
|
# Test/build infrastructure packages
|
||||||
- (?s:.*/packages/testing/.*)\Z
|
- (?s:.*/packages/quality/.*)\Z
|
||||||
- (?s:.*/packages/@n8n/(benchmark|eslint-config|eslint-plugin-community-nodes|stylelint-config|typescript-config|vitest-config)/.*)\Z
|
- (?s:.*/packages/@n8n/(benchmark|eslint-config|eslint-plugin-community-nodes|stylelint-config|typescript-config|vitest-config)/.*)\Z
|
||||||
|
|||||||
+1
-1
@@ -202,7 +202,7 @@ reviews:
|
|||||||
- docker/**
|
- docker/**
|
||||||
- scripts/**
|
- scripts/**
|
||||||
- patches/**
|
- patches/**
|
||||||
- packages/testing/**
|
- packages/quality/**
|
||||||
- packages/@n8n/eslint-config/**
|
- packages/@n8n/eslint-config/**
|
||||||
- packages/@n8n/stylelint-config/**
|
- packages/@n8n/stylelint-config/**
|
||||||
- packages/@n8n/vitest-config/**
|
- packages/@n8n/vitest-config/**
|
||||||
|
|||||||
@@ -147,7 +147,7 @@ const config: KnipConfig = {
|
|||||||
// Only named in JSDoc type imports; @stryker-mutator/core provides it.
|
// Only named in JSDoc type imports; @stryker-mutator/core provides it.
|
||||||
ignoreDependencies: ['@stryker-mutator/api'],
|
ignoreDependencies: ['@stryker-mutator/api'],
|
||||||
}),
|
}),
|
||||||
'packages/@n8n/benchmark': pkg({
|
'packages/quality/efficiency/scale/benchmark': pkg({
|
||||||
// The k6 runtime provides its own modules; only @types/k6 is installed.
|
// The k6 runtime provides its own modules; only @types/k6 is installed.
|
||||||
ignoreDependencies: ['k6'],
|
ignoreDependencies: ['k6'],
|
||||||
}),
|
}),
|
||||||
@@ -204,7 +204,7 @@ const config: KnipConfig = {
|
|||||||
// Spawned as a binary through execFile.
|
// Spawned as a binary through execFile.
|
||||||
ignoreDependencies: ['agent-browser'],
|
ignoreDependencies: ['agent-browser'],
|
||||||
}),
|
}),
|
||||||
'packages/testing/playwright': pkg({
|
'packages/quality/testing/playwright': pkg({
|
||||||
ignoreDependencies: [
|
ignoreDependencies: [
|
||||||
// The e2e suite runs against the built app; the edge orders the turbo build.
|
// The e2e suite runs against the built app; the edge orders the turbo build.
|
||||||
'n8n',
|
'n8n',
|
||||||
|
|||||||
+1
-1
@@ -87,7 +87,7 @@ pre-commit:
|
|||||||
- merge
|
- merge
|
||||||
- rebase
|
- rebase
|
||||||
playwright_janitor:
|
playwright_janitor:
|
||||||
glob: 'packages/testing/playwright/**/*.ts'
|
glob: 'packages/quality/testing/playwright/**/*.ts'
|
||||||
run: |
|
run: |
|
||||||
files=$(echo {staged_files} | tr ' ' ',')
|
files=$(echo {staged_files} | tr ' ' ',')
|
||||||
pnpm --filter=n8n-playwright janitor --files="$files"
|
pnpm --filter=n8n-playwright janitor --files="$files"
|
||||||
|
|||||||
@@ -1,7 +0,0 @@
|
|||||||
{
|
|
||||||
"$schema": "../node_modules/@biomejs/biome/configuration_schema.json",
|
|
||||||
"extends": ["../../../biome.jsonc"],
|
|
||||||
"files": {
|
|
||||||
"ignore": ["scripts/mock-api/**"]
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -33,7 +33,7 @@ const TBLS_IMAGE =
|
|||||||
|
|
||||||
const POSTGRES_VERSIONS_PATH = resolve(
|
const POSTGRES_VERSIONS_PATH = resolve(
|
||||||
REPO_ROOT,
|
REPO_ROOT,
|
||||||
'packages/testing/containers/postgres-versions.json',
|
'packages/quality/environments/containers/postgres-versions.json',
|
||||||
);
|
);
|
||||||
|
|
||||||
function primaryPostgresImage() {
|
function primaryPostgresImage() {
|
||||||
|
|||||||
@@ -26,7 +26,7 @@ export const backendNetworkBoundaryConfig = tseslint.config({
|
|||||||
allow: [
|
allow: [
|
||||||
// The factory itself: this is where the guarded client is built.
|
// The factory itself: this is where the guarded client is built.
|
||||||
'packages/@n8n/backend-network/',
|
'packages/@n8n/backend-network/',
|
||||||
'packages/@n8n/benchmark/',
|
'packages/quality/efficiency/scale/benchmark/',
|
||||||
],
|
],
|
||||||
},
|
},
|
||||||
],
|
],
|
||||||
|
|||||||
@@ -25,7 +25,7 @@ const TEST_FILE_PATTERN = /(\.(test|spec)\.ts$)|([\\/]__tests__[\\/])|([\\/]test
|
|||||||
* so bare or self-naming directives of those forms swallow this rule's report,
|
* so bare or self-naming directives of those forms swallow this rule's report,
|
||||||
* and a package-level `'off'` override is invisible to lint altogether. The
|
* and a package-level `'off'` override is invisible to lint altogether. The
|
||||||
* authoritative check is the code-health rule `encryption-boundary`
|
* authoritative check is the code-health rule `encryption-boundary`
|
||||||
* (packages/testing/code-health): it scans the source out of band for every
|
* (packages/quality/policy/code-health): it scans the source out of band for every
|
||||||
* directive form and verifies that each package that can reach the primitives
|
* directive form and verifies that each package that can reach the primitives
|
||||||
* composes the boundary config at error severity.
|
* composes the boundary config at error severity.
|
||||||
*/
|
*/
|
||||||
|
|||||||
@@ -7,7 +7,7 @@ import { ExpressionEvaluator } from '../evaluator/expression-evaluator';
|
|||||||
// can a real bridge be built from inside the synchronous evaluate() path?
|
// can a real bridge be built from inside the synchronous evaluate() path?
|
||||||
// isolated-vm uses its sync APIs directly; QuickJS relies on the WASM module
|
// isolated-vm uses its sync APIs directly; QuickJS relies on the WASM module
|
||||||
// cached by pool warmup (an earlier async initialize). The cold start's cost
|
// cached by pool warmup (an earlier async initialize). The cold start's cost
|
||||||
// is tracked in packages/testing/performance (cold-start.bench.ts).
|
// is tracked in packages/quality/efficiency/microbenchmarks (cold-start.bench.ts).
|
||||||
|
|
||||||
describe(`Lazy acquisition: synchronous cold start (${engineName})`, () => {
|
describe(`Lazy acquisition: synchronous cold start (${engineName})`, () => {
|
||||||
it('builds a working bridge synchronously when the pool is exhausted', async () => {
|
it('builds a working bridge synchronously when the pool is exhausted', async () => {
|
||||||
|
|||||||
@@ -29,12 +29,12 @@ Read these docs before starting any implementation:
|
|||||||
|
|
||||||
## E2E Testing
|
## E2E Testing
|
||||||
|
|
||||||
Tests live in `packages/testing/playwright/tests/e2e/instance-ai/`.
|
Tests live in `packages/quality/testing/playwright/tests/e2e/instance-ai/`.
|
||||||
|
|
||||||
### Local-build mode (no docker, no recording — hits real Anthropic API)
|
### Local-build mode (no docker, no recording — hits real Anthropic API)
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
cd packages/testing/playwright
|
cd packages/quality/testing/playwright
|
||||||
export ANTHROPIC_API_KEY=sk-ant-...
|
export ANTHROPIC_API_KEY=sk-ant-...
|
||||||
pnpm test:local:instance-ai # full suite
|
pnpm test:local:instance-ai # full suite
|
||||||
pnpm test:local:instance-ai --grep "preview" # single test
|
pnpm test:local:instance-ai --grep "preview" # single test
|
||||||
@@ -52,7 +52,7 @@ mode with a real key. This captures LLM traffic + tool traces into
|
|||||||
|
|
||||||
```bash
|
```bash
|
||||||
pnpm build:docker # from repo root — build the local n8n image first
|
pnpm build:docker # from repo root — build the local n8n image first
|
||||||
cd packages/testing/playwright
|
cd packages/quality/testing/playwright
|
||||||
ANTHROPIC_API_KEY=sk-ant-... pnpm test:container:sqlite tests/e2e/instance-ai --workers 1
|
ANTHROPIC_API_KEY=sk-ant-... pnpm test:container:sqlite tests/e2e/instance-ai --workers 1
|
||||||
```
|
```
|
||||||
|
|
||||||
|
|||||||
@@ -263,8 +263,8 @@ N8N_INSTANCE_AI_SEARXNG_URL=http://searxng:8080
|
|||||||
INSTANCE_AI_BRAVE_SEARCH_API_KEY=BSA-xxx
|
INSTANCE_AI_BRAVE_SEARCH_API_KEY=BSA-xxx
|
||||||
|
|
||||||
# With sandbox (n8n sandbox service)
|
# With sandbox (n8n sandbox service)
|
||||||
# CI can start it with:
|
# For local development, start it with:
|
||||||
# pnpm tsx packages/testing/containers/start-sandbox.ts --network n8n-eval-net
|
# pnpm --filter n8n-containers services --services sandbox --network n8n-eval-net
|
||||||
N8N_INSTANCE_AI_SANDBOX_ENABLED=true
|
N8N_INSTANCE_AI_SANDBOX_ENABLED=true
|
||||||
N8N_INSTANCE_AI_SANDBOX_PROVIDER=n8n-sandbox
|
N8N_INSTANCE_AI_SANDBOX_PROVIDER=n8n-sandbox
|
||||||
N8N_SANDBOX_SERVICE_URL=https://sandbox.example.com
|
N8N_SANDBOX_SERVICE_URL=https://sandbox.example.com
|
||||||
|
|||||||
@@ -255,13 +255,13 @@ volatile request body.
|
|||||||
|
|
||||||
| File | Purpose |
|
| File | Purpose |
|
||||||
|------|---------|
|
|------|---------|
|
||||||
| `packages/testing/playwright/tests/e2e/instance-ai/fixtures.ts` | Test fixtures — proxy setup, recording/replay orchestration |
|
| `packages/quality/testing/playwright/tests/e2e/instance-ai/fixtures.ts` | Test fixtures — proxy setup, recording/replay orchestration |
|
||||||
| `packages/@n8n/instance-ai/src/tracing/trace-replay.ts` | `TraceIndex`, `IdRemapper`, `TraceWriter`, JSONL parsing |
|
| `packages/@n8n/instance-ai/src/tracing/trace-replay.ts` | `TraceIndex`, `IdRemapper`, `TraceWriter`, JSONL parsing |
|
||||||
| `packages/@n8n/instance-ai/src/tracing/langsmith-tracing.ts` | Tool wrapping — `replayWrapTools`, `recordWrapTools` |
|
| `packages/@n8n/instance-ai/src/tracing/langsmith-tracing.ts` | Tool wrapping — `replayWrapTools`, `recordWrapTools` |
|
||||||
| `packages/@n8n/instance-ai/src/types.ts` | `InstanceAiTraceContext`, `TraceReplayMode` |
|
| `packages/@n8n/instance-ai/src/types.ts` | `InstanceAiTraceContext`, `TraceReplayMode` |
|
||||||
| `packages/cli/src/modules/instance-ai/instance-ai.service.ts` | Trace mode initialization, shared state management |
|
| `packages/cli/src/modules/instance-ai/instance-ai.service.ts` | Trace mode initialization, shared state management |
|
||||||
| `packages/cli/src/modules/instance-ai/instance-ai.controller.ts` | Test-only REST endpoints for trace delivery |
|
| `packages/cli/src/modules/instance-ai/instance-ai.controller.ts` | Test-only REST endpoints for trace delivery |
|
||||||
| `packages/testing/containers/services/proxy.ts` | `ProxyServer` class (MockServer client) |
|
| `packages/quality/environments/containers/services/proxy.ts` | `ProxyServer` class (MockServer client) |
|
||||||
|
|
||||||
### Test-Only Endpoints
|
### Test-Only Endpoints
|
||||||
|
|
||||||
@@ -323,7 +323,7 @@ stack entirely. Tests hit the real Anthropic API directly. This mode does
|
|||||||
**not** record proxy expectations.
|
**not** record proxy expectations.
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
cd packages/testing/playwright
|
cd packages/quality/testing/playwright
|
||||||
export ANTHROPIC_API_KEY=sk-ant-...
|
export ANTHROPIC_API_KEY=sk-ant-...
|
||||||
pnpm test:local:instance-ai # full suite
|
pnpm test:local:instance-ai # full suite
|
||||||
pnpm test:local:instance-ai --grep "preview" # single test
|
pnpm test:local:instance-ai --grep "preview" # single test
|
||||||
@@ -351,7 +351,7 @@ pnpm test:local:instance-ai --grep "preview" --headed
|
|||||||
vars over the generic `test:local:isolated` runner, which provides random free
|
vars over the generic `test:local:isolated` runner, which provides random free
|
||||||
ports, a throwaway `N8N_USER_FOLDER` (so `~/.n8n` is never touched), and
|
ports, a throwaway `N8N_USER_FOLDER` (so `~/.n8n` is never touched), and
|
||||||
process-group cleanup. See the
|
process-group cleanup. See the
|
||||||
[Playwright README](../../../testing/playwright/README.md) for full details on
|
[Playwright README](../../../quality/testing/playwright/README.md) for full details on
|
||||||
`test:local:isolated`.
|
`test:local:isolated`.
|
||||||
|
|
||||||
> **Cost note:** Each run makes real Anthropic calls. Scope with `--grep` or
|
> **Cost note:** Each run makes real Anthropic calls. Scope with `--grep` or
|
||||||
|
|||||||
@@ -5,7 +5,7 @@ export const backendNetworkBoundaryConfig = defineConfig({
|
|||||||
'n8n-local-rules/no-uncentralized-http': [
|
'n8n-local-rules/no-uncentralized-http': [
|
||||||
'error',
|
'error',
|
||||||
{
|
{
|
||||||
allow: ['packages/@n8n/backend-network/', 'packages/@n8n/benchmark/'],
|
allow: ['packages/@n8n/backend-network/', 'packages/quality/efficiency/scale/benchmark/'],
|
||||||
},
|
},
|
||||||
],
|
],
|
||||||
},
|
},
|
||||||
|
|||||||
@@ -3,7 +3,7 @@ import type { App, ComponentPublicInstance } from 'vue';
|
|||||||
/**
|
/**
|
||||||
* localStorage key that switches on app-wide component re-render counting.
|
* localStorage key that switches on app-wide component re-render counting.
|
||||||
*
|
*
|
||||||
* Set by the canvas performance benchmark (packages/testing/playwright) before
|
* Set by the canvas performance benchmark (packages/quality/testing/playwright) before
|
||||||
* the SPA boots. Real users never set it, so the global mixin below is never
|
* the SPA boots. Real users never set it, so the global mixin below is never
|
||||||
* installed in normal use and the feature carries no runtime cost for them.
|
* installed in normal use and the feature carries no runtime cost for them.
|
||||||
*/
|
*/
|
||||||
|
|||||||
@@ -0,0 +1,19 @@
|
|||||||
|
# Quality engineering instructions
|
||||||
|
|
||||||
|
Read the repository [AGENTS.md](../../AGENTS.md) first. Use the [quality engineering index](README.md) to find the tool for your task.
|
||||||
|
|
||||||
|
| Area | Responsibility | Current location |
|
||||||
|
| --- | --- | --- |
|
||||||
|
| Policy | Repository rules and static analysis | [`rules-engine`](policy/rules-engine/), [`code-health`](policy/code-health/) |
|
||||||
|
| Testing | Test selection, architecture checks, and test orchestration | [`test-impact`](testing/test-impact/), [`janitor`](testing/janitor/), [`playwright`](testing/playwright/) |
|
||||||
|
| Environments | Test services and deployment stacks | [`containers`](environments/containers/) |
|
||||||
|
| Efficiency | Speed, memory, load, latency, throughput, capacity, and resource use | [Microbenchmarks](efficiency/microbenchmarks/), [Playwright performance tests](testing/playwright/tests/performance/), [infrastructure benchmarks](testing/playwright/tests/infrastructure/benchmarks/), [benchmark CLI](efficiency/scale/benchmark/) |
|
||||||
|
|
||||||
|
## Working rules
|
||||||
|
|
||||||
|
- Read the instructions in the owning package before you change its files.
|
||||||
|
- Keep benchmark methods and results comparable when you move files. Do not change behavior during a path migration.
|
||||||
|
- Keep existing package names and `pnpm --filter` commands when you move workspaces.
|
||||||
|
- Update workspace discovery, CI paths, test-impact rules, ownership, documentation, and the lockfile when you move a package. Verify that each moved workspace still runs its intended checks.
|
||||||
|
- Prefer local efficiency measurements that do not require an n8n Docker image rebuild. State clearly when a benchmark does require an image.
|
||||||
|
- Put task instructions in focused README files. Keep inherited `AGENTS.md` files short and limited to rules for their area.
|
||||||
@@ -0,0 +1,12 @@
|
|||||||
|
# Quality engineering
|
||||||
|
|
||||||
|
Each area owns workspaces. A test stays with its runner, even when it measures another quality concern.
|
||||||
|
|
||||||
|
| Area | Owns | Start here |
|
||||||
|
| --- | --- | --- |
|
||||||
|
| Policy | Repository rules and static checks | [Policy](policy/README.md) |
|
||||||
|
| Testing | Test selection, architecture checks, and the Playwright runner | [Testing](testing/README.md) |
|
||||||
|
| Environments | Services and deployment stacks for tests | [Environments](environments/README.md) |
|
||||||
|
| Efficiency | Standalone microbenchmark and load-test tools | [Efficiency](efficiency/README.md) |
|
||||||
|
|
||||||
|
Use the [Playwright suite map](testing/playwright/README.md#choose-a-suite) to find browser, accessibility, memory, and infrastructure tests. Their specs remain in Playwright because they use its fixtures and projects.
|
||||||
@@ -0,0 +1,8 @@
|
|||||||
|
# Efficiency instructions
|
||||||
|
|
||||||
|
Read the parent [quality instructions](../AGENTS.md) before you change a benchmark.
|
||||||
|
|
||||||
|
- Compare results with the same machine, workload, and process profile.
|
||||||
|
- Do not change benchmark methods during a path migration.
|
||||||
|
- Keep local measurements possible without an n8n Docker image rebuild. State when a benchmark needs an image.
|
||||||
|
- Put task steps and result interpretation in the owning benchmark's README.
|
||||||
@@ -0,0 +1,10 @@
|
|||||||
|
# Efficiency tooling
|
||||||
|
|
||||||
|
This area owns tools that measure speed and scale outside the Playwright runner.
|
||||||
|
|
||||||
|
| Tool | Use it to | Guide |
|
||||||
|
| --- | --- | --- |
|
||||||
|
| `@n8n/performance` | Compare code paths without building an n8n Docker image | [Microbenchmarks](microbenchmarks/README.md) |
|
||||||
|
| `@n8n/n8n-benchmark` | Run load scenarios against an n8n instance | [Benchmark CLI](scale/benchmark/README.md) |
|
||||||
|
|
||||||
|
For idle memory, canvas performance, throughput, and local soak tests, use the [Playwright suite map](../testing/playwright/README.md#choose-a-suite). These specs need Playwright fixtures, projects, metrics, or managed containers.
|
||||||
+1
-1
@@ -1,5 +1,5 @@
|
|||||||
{
|
{
|
||||||
"extends": "../../../tsconfig.json",
|
"extends": "../../../../tsconfig.json",
|
||||||
"compilerOptions": {
|
"compilerOptions": {
|
||||||
"module": "esnext",
|
"module": "esnext",
|
||||||
"moduleResolution": "bundler",
|
"moduleResolution": "bundler",
|
||||||
+9
-9
@@ -38,7 +38,7 @@ WORKDIR /app
|
|||||||
COPY --chown=node:node ./pnpm-lock.yaml /app/pnpm-lock.yaml
|
COPY --chown=node:node ./pnpm-lock.yaml /app/pnpm-lock.yaml
|
||||||
COPY --chown=node:node ./pnpm-workspace.yaml /app/pnpm-workspace.yaml
|
COPY --chown=node:node ./pnpm-workspace.yaml /app/pnpm-workspace.yaml
|
||||||
COPY --chown=node:node ./package.json /app/package.json
|
COPY --chown=node:node ./package.json /app/package.json
|
||||||
COPY --chown=node:node ./packages/@n8n/benchmark/package.json /app/packages/@n8n/benchmark/package.json
|
COPY --chown=node:node ./packages/quality/efficiency/scale/benchmark/package.json /app/packages/quality/efficiency/scale/benchmark/package.json
|
||||||
COPY --chown=node:node ./patches /app/patches
|
COPY --chown=node:node ./patches /app/patches
|
||||||
COPY --chown=node:node ./scripts /app/scripts
|
COPY --chown=node:node ./scripts /app/scripts
|
||||||
|
|
||||||
@@ -47,15 +47,15 @@ RUN pnpm install --frozen-lockfile
|
|||||||
|
|
||||||
# TS config files
|
# TS config files
|
||||||
COPY --chown=node:node ./packages/@n8n/typescript-config /app/packages/@n8n/typescript-config
|
COPY --chown=node:node ./packages/@n8n/typescript-config /app/packages/@n8n/typescript-config
|
||||||
COPY --chown=node:node ./packages/@n8n/benchmark/tsconfig.json /app/packages/@n8n/benchmark/tsconfig.json
|
COPY --chown=node:node ./packages/quality/efficiency/scale/benchmark/tsconfig.json /app/packages/quality/efficiency/scale/benchmark/tsconfig.json
|
||||||
COPY --chown=node:node ./packages/@n8n/benchmark/tsconfig.build.json /app/packages/@n8n/benchmark/tsconfig.build.json
|
COPY --chown=node:node ./packages/quality/efficiency/scale/benchmark/tsconfig.build.json /app/packages/quality/efficiency/scale/benchmark/tsconfig.build.json
|
||||||
|
|
||||||
# Source files
|
# Source files
|
||||||
COPY --chown=node:node ./packages/@n8n/benchmark/src /app/packages/@n8n/benchmark/src
|
COPY --chown=node:node ./packages/quality/efficiency/scale/benchmark/src /app/packages/quality/efficiency/scale/benchmark/src
|
||||||
COPY --chown=node:node ./packages/@n8n/benchmark/bin /app/packages/@n8n/benchmark/bin
|
COPY --chown=node:node ./packages/quality/efficiency/scale/benchmark/bin /app/packages/quality/efficiency/scale/benchmark/bin
|
||||||
COPY --chown=node:node ./packages/@n8n/benchmark/scenarios /app/packages/@n8n/benchmark/scenarios
|
COPY --chown=node:node ./packages/quality/efficiency/scale/benchmark/scenarios /app/packages/quality/efficiency/scale/benchmark/scenarios
|
||||||
|
|
||||||
WORKDIR /app/packages/@n8n/benchmark
|
WORKDIR /app/packages/quality/efficiency/scale/benchmark
|
||||||
RUN pnpm build
|
RUN pnpm build
|
||||||
|
|
||||||
#
|
#
|
||||||
@@ -64,7 +64,7 @@ FROM base AS runner
|
|||||||
|
|
||||||
COPY --from=builder /app /app
|
COPY --from=builder /app /app
|
||||||
|
|
||||||
WORKDIR /app/packages/@n8n/benchmark
|
WORKDIR /app/packages/quality/efficiency/scale/benchmark
|
||||||
USER node
|
USER node
|
||||||
|
|
||||||
ENTRYPOINT [ "/app/packages/@n8n/benchmark/bin/n8n-benchmark" ]
|
ENTRYPOINT [ "/app/packages/quality/efficiency/scale/benchmark/bin/n8n-benchmark" ]
|
||||||
+15
-11
@@ -2,10 +2,12 @@
|
|||||||
|
|
||||||
Tool for executing benchmarks against an n8n instance.
|
Tool for executing benchmarks against an n8n instance.
|
||||||
|
|
||||||
|
Run the `pnpm --filter @n8n/n8n-benchmark` commands below from the repository root. pnpm runs each script in the benchmark package directory.
|
||||||
|
|
||||||
## Directory structure
|
## Directory structure
|
||||||
|
|
||||||
```text
|
```text
|
||||||
packages/@n8n/benchmark
|
packages/quality/efficiency/scale/benchmark
|
||||||
├── scenarios Benchmark scenarios
|
├── scenarios Benchmark scenarios
|
||||||
├── src Source code for the n8n-benchmark cli
|
├── src Source code for the n8n-benchmark cli
|
||||||
├── Dockerfile Dockerfile for the n8n-benchmark cli
|
├── Dockerfile Dockerfile for the n8n-benchmark cli
|
||||||
@@ -50,17 +52,17 @@ docker run -v ./scenarios:/scenarios ghcr.io/n8n-io/n8n-benchmark:latest run \
|
|||||||
|
|
||||||
The benchmark suite consists of [benchmark scenarios](#benchmark-scenarios) and different [n8n setups](#n8n-setups).
|
The benchmark suite consists of [benchmark scenarios](#benchmark-scenarios) and different [n8n setups](#n8n-setups).
|
||||||
|
|
||||||
### locally
|
### Locally
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
pnpm benchmark-locally
|
pnpm --filter @n8n/n8n-benchmark benchmark-locally
|
||||||
```
|
```
|
||||||
|
|
||||||
You can filter to a specific scenario and setup:
|
You can filter to a specific scenario and setup:
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
# Run only the http-node scenario with the sqlite setup
|
# Run only the http-node scenario with the sqlite setup
|
||||||
pnpm benchmark-locally --runDir /tmp/n8n-data --scenarioFilter http-node sqlite
|
pnpm --filter @n8n/n8n-benchmark benchmark-locally --runDir /tmp/n8n-data --scenarioFilter http-node sqlite
|
||||||
```
|
```
|
||||||
|
|
||||||
### In the cloud
|
### In the cloud
|
||||||
@@ -71,9 +73,9 @@ Terraform code that creates it.
|
|||||||
Create the environment, run the benchmarks, then delete the environment:
|
Create the environment, run the benchmarks, then delete the environment:
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
pnpm provision-cloud-env
|
pnpm --filter @n8n/n8n-benchmark provision-cloud-env
|
||||||
pnpm benchmark-in-cloud
|
pnpm --filter @n8n/n8n-benchmark benchmark-in-cloud
|
||||||
pnpm destroy-cloud-env
|
pnpm --filter @n8n/n8n-benchmark destroy-cloud-env
|
||||||
```
|
```
|
||||||
|
|
||||||
## Running the `n8n-benchmark` cli
|
## Running the `n8n-benchmark` cli
|
||||||
@@ -87,7 +89,7 @@ Build the Docker image:
|
|||||||
```sh
|
```sh
|
||||||
# Must be run in the repository root
|
# Must be run in the repository root
|
||||||
# k6 doesn't have an arm64 build available for linux, we need to build against amd64
|
# k6 doesn't have an arm64 build available for linux, we need to build against amd64
|
||||||
docker build --platform linux/amd64 -t n8n-benchmark -f packages/@n8n/benchmark/Dockerfile .
|
docker build --platform linux/amd64 -t n8n-benchmark -f packages/quality/efficiency/scale/benchmark/Dockerfile .
|
||||||
```
|
```
|
||||||
|
|
||||||
Run the image
|
Run the image
|
||||||
@@ -108,11 +110,13 @@ Requirements:
|
|||||||
- [k6](https://grafana.com/docs/k6/latest/set-up/install-k6/)
|
- [k6](https://grafana.com/docs/k6/latest/set-up/install-k6/)
|
||||||
- Node.js v24 or higher
|
- Node.js v24 or higher
|
||||||
|
|
||||||
|
Set `N8N_USER_EMAIL` and `N8N_USER_PASSWORD` in your environment before you run the CLI.
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
pnpm build
|
pnpm --filter @n8n/n8n-benchmark build > benchmark-build.log 2>&1
|
||||||
|
|
||||||
# Run tests against http://localhost:5678 with specified email and password
|
# Run tests against http://localhost:5678 with specified email and password
|
||||||
N8N_USER_EMAIL=user@n8n.io N8N_USER_PASSWORD=password ./bin/n8n-benchmark run
|
pnpm --filter @n8n/n8n-benchmark exec ./bin/n8n-benchmark run
|
||||||
```
|
```
|
||||||
|
|
||||||
## Benchmark scenarios
|
## Benchmark scenarios
|
||||||
@@ -127,4 +131,4 @@ Available scenarios are located in [`./scenarios`](./scenarios/).
|
|||||||
|
|
||||||
## n8n setups
|
## n8n setups
|
||||||
|
|
||||||
A n8n setup defines a single n8n runtime configuration using Docker compose. Different n8n setups are located in [`./scripts/n8nSetups`](./scripts/n8nSetups).
|
An n8n setup defines one n8n runtime configuration with Docker Compose. The setups are in [`scripts/n8n-setups`](scripts/n8n-setups/).
|
||||||
@@ -0,0 +1,7 @@
|
|||||||
|
{
|
||||||
|
"$schema": "../../../../../node_modules/@biomejs/biome/configuration_schema.json",
|
||||||
|
"extends": ["../../../../../biome.jsonc"],
|
||||||
|
"files": {
|
||||||
|
"ignore": ["scripts/mock-api/**"]
|
||||||
|
}
|
||||||
|
}
|
||||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user