mirror of
https://github.com/n8n-io/n8n.git
synced 2026-10-11 22:50:06 +00:00
refactor: Group quality engineering workspaces by capability (#40007)
This commit is contained in:
@@ -17,7 +17,7 @@ clear, when to stay silent, what never to say — lives once in
|
||||
| `backend/` | Backend | `cli`, `@n8n/db`, `core`, `workflow`, node packages |
|
||||
| `db-migrations/` | DB migrations | `@n8n/db` migrations + their tests in `cli` |
|
||||
| `frontend/` | Frontend | `packages/frontend` |
|
||||
| `qa-dx/` | QA & DX | `.github`, `docker`, `scripts`, `patches`, `packages/testing`, the lint/test/TS config packages, baselines |
|
||||
| `qa-dx/` | QA & DX | `.github`, `docker`, `scripts`, `patches`, `packages/quality`, the lint/test/TS config packages, baselines |
|
||||
| `testing/` | Backend + Frontend | any package with a test suite |
|
||||
|
||||
A directory maps to one agent unless, like `testing/`, the policy is identical
|
||||
|
||||
@@ -13,7 +13,7 @@ Flag a diff that **adds** entries to any of these, and ask for the fix instead:
|
||||
|------|---------|
|
||||
| `.code-health-baseline.json` | `@n8n/code-health` violations |
|
||||
| `.boundaries-baseline.json` | `turbo boundaries` issue count |
|
||||
| `packages/testing/playwright/.janitor-baseline.json` | Playwright janitor findings |
|
||||
| `packages/quality/testing/playwright/.janitor-baseline.json` | Playwright janitor findings |
|
||||
| `packages/cli/eslint.config.mjs` | the `misplaced-n8n-typeorm-import` and public-API allowlists, each captioned "NEVER add to this list" |
|
||||
| `.code-health-baseline.json` (`lint-config-layering`) | package-wide rule downgrades left in package ESLint configs |
|
||||
|
||||
|
||||
@@ -35,7 +35,7 @@ fed from a `needs:` list. Flag:
|
||||
Jobs gated on a `ci-filter` filter only run when a matching file changed, so a
|
||||
filter that is too narrow means the test quietly stops covering new code. When
|
||||
a PR adds a directory that an existing filter was meant to cover — a new package
|
||||
under `packages/testing/`, a new script under `.github/scripts/` — check the
|
||||
under `packages/quality/`, a new script under `.github/scripts/` — check the
|
||||
filter still matches it.
|
||||
|
||||
## Least privilege
|
||||
|
||||
@@ -32,7 +32,7 @@ Based on the affected area, pick the test layer and pattern:
|
||||
| CLI / API | Vitest integration | setupTestServer + supertest | `packages/cli/test/integration/` |
|
||||
| Config | Vitest unit | GlobalConfig + Container | `packages/@n8n/config/src/configs/__tests__/` |
|
||||
| Editor UI | Vitest | Vue Test Utils + Pinia | `packages/frontend/editor-ui/src/**/__tests__/` |
|
||||
| E2E / Canvas | Playwright | Test containers + composables | `packages/testing/playwright/` |
|
||||
| E2E / Canvas | Playwright | Test containers + composables | `packages/quality/testing/playwright/` |
|
||||
|
||||
## Step 3: Locate Source Files
|
||||
|
||||
|
||||
@@ -8,7 +8,7 @@ exclude:
|
||||
# Machine-generated instance-AI expectation traces. Contain ephemeral
|
||||
# credential reference IDs (nanoids) that re-trip the secrets scanner
|
||||
# on every regeneration. No secret material.
|
||||
- packages/testing/playwright/expectations
|
||||
- packages/quality/testing/playwright/expectations
|
||||
|
||||
# Test code repo-wide. Fixtures routinely contain synthetic PEM blocks,
|
||||
# basic-auth strings, and deliberately fake keys (including the secret
|
||||
|
||||
+17
-17
@@ -435,12 +435,12 @@
|
||||
"hash": "4a6882cb8d62"
|
||||
}
|
||||
],
|
||||
"packages/@n8n/benchmark/package.json": [
|
||||
"packages/quality/efficiency/scale/benchmark/package.json": [
|
||||
{
|
||||
"rule": "catalog-violations",
|
||||
"line": 38,
|
||||
"message": "@oclif/core appears in 4 packages with 2 different versions — add to pnpm-workspace.yaml catalog",
|
||||
"hash": "67f9d81d9528"
|
||||
"hash": "e07bb1da47d7"
|
||||
}
|
||||
],
|
||||
"packages/@n8n/cli/package.json": [
|
||||
@@ -2043,48 +2043,48 @@
|
||||
"hash": "0f94a71d5ae8"
|
||||
}
|
||||
],
|
||||
"packages/testing/playwright/eslint.config.mjs": [
|
||||
"packages/quality/testing/playwright/eslint.config.mjs": [
|
||||
{
|
||||
"rule": "lint-config-layering",
|
||||
"line": 21,
|
||||
"message": "packages/testing/playwright turns '@typescript-eslint/no-unused-expressions' down to 'off' for the whole package.",
|
||||
"hash": "e2f93b5b591a"
|
||||
"message": "packages/quality/testing/playwright turns '@typescript-eslint/no-unused-expressions' down to 'off' for the whole package.",
|
||||
"hash": "c789a142996b"
|
||||
},
|
||||
{
|
||||
"rule": "lint-config-layering",
|
||||
"line": 22,
|
||||
"message": "packages/testing/playwright turns '@typescript-eslint/no-use-before-define' down to 'off' for the whole package.",
|
||||
"hash": "75d3ec1677ba"
|
||||
"message": "packages/quality/testing/playwright turns '@typescript-eslint/no-use-before-define' down to 'off' for the whole package.",
|
||||
"hash": "fcc3a69d8a2e"
|
||||
},
|
||||
{
|
||||
"rule": "lint-config-layering",
|
||||
"line": 23,
|
||||
"message": "packages/testing/playwright turns '@typescript-eslint/promise-function-async' down to 'off' for the whole package.",
|
||||
"hash": "35dbe9e44ada"
|
||||
"message": "packages/quality/testing/playwright turns '@typescript-eslint/promise-function-async' down to 'off' for the whole package.",
|
||||
"hash": "5ab4c080de15"
|
||||
},
|
||||
{
|
||||
"rule": "lint-config-layering",
|
||||
"line": 24,
|
||||
"message": "packages/testing/playwright turns 'playwright/expect-expect' down to 'warn' for the whole package.",
|
||||
"hash": "e8b5f4b0f0dd"
|
||||
"message": "packages/quality/testing/playwright turns 'playwright/expect-expect' down to 'warn' for the whole package.",
|
||||
"hash": "3a584940f6c7"
|
||||
},
|
||||
{
|
||||
"rule": "lint-config-layering",
|
||||
"line": 25,
|
||||
"message": "packages/testing/playwright turns 'playwright/max-nested-describe' down to 'warn' for the whole package.",
|
||||
"hash": "77d82ee8a802"
|
||||
"message": "packages/quality/testing/playwright turns 'playwright/max-nested-describe' down to 'warn' for the whole package.",
|
||||
"hash": "8ea7fd515ed8"
|
||||
},
|
||||
{
|
||||
"rule": "lint-config-layering",
|
||||
"line": 27,
|
||||
"message": "packages/testing/playwright turns 'playwright/no-skipped-test' down to 'warn' for the whole package.",
|
||||
"hash": "14eb9eecd1b4"
|
||||
"message": "packages/quality/testing/playwright turns 'playwright/no-skipped-test' down to 'warn' for the whole package.",
|
||||
"hash": "6ce1d38312f9"
|
||||
},
|
||||
{
|
||||
"rule": "lint-config-layering",
|
||||
"line": 78,
|
||||
"message": "packages/testing/playwright turns 'unicorn/filename-case' down to 'off' for the whole package.",
|
||||
"hash": "de86b9645930"
|
||||
"message": "packages/quality/testing/playwright turns 'unicorn/filename-case' down to 'off' for the whole package.",
|
||||
"hash": "949214800425"
|
||||
}
|
||||
],
|
||||
"packages/frontend/@n8n/design-system/eslint.config.mjs": [
|
||||
|
||||
+6
-4
@@ -26,7 +26,7 @@
|
||||
!docker/images/runners
|
||||
!docker/images/runners/n8n-task-runners.json
|
||||
|
||||
# === benchmark image (packages/@n8n/benchmark/Dockerfile) ===
|
||||
# === benchmark image (packages/quality/efficiency/scale/benchmark/Dockerfile) ===
|
||||
!package.json
|
||||
!pnpm-lock.yaml
|
||||
!pnpm-workspace.yaml
|
||||
@@ -34,8 +34,10 @@
|
||||
!patches/**
|
||||
!scripts
|
||||
!scripts/**
|
||||
!packages/@n8n/benchmark
|
||||
!packages/@n8n/benchmark/**
|
||||
!packages/quality
|
||||
!packages/quality/efficiency
|
||||
!packages/quality/efficiency/scale
|
||||
!packages/quality/efficiency/scale/benchmark
|
||||
!packages/quality/efficiency/scale/benchmark/**
|
||||
!packages/@n8n/typescript-config
|
||||
!packages/@n8n/typescript-config/**
|
||||
|
||||
|
||||
@@ -81,11 +81,11 @@ return 'blacksmith';
|
||||
|
||||
| Telemetry | Source | Metrics |
|
||||
|-----------|--------|---------|
|
||||
| Playwright perf/benchmark | `packages/testing/playwright/reporters/metrics-reporter.ts` | Any metric attached via `attachMetric()` |
|
||||
| Accessibility buckets | `packages/testing/playwright/reporters/a11y-reporter.ts` | Per-bucket axe score, violated rules, violating elements |
|
||||
| Playwright perf/benchmark | `packages/quality/testing/playwright/reporters/metrics-reporter.ts` | Any metric attached via `attachMetric()` |
|
||||
| Accessibility buckets | `packages/quality/testing/playwright/reporters/a11y-reporter.ts` | Per-bucket axe score, violated rules, violating elements |
|
||||
| Build stats | `.github/scripts/send-build-stats.mjs` | Per-package build duration, cache hit/miss, run total |
|
||||
| Docker stats | `.github/scripts/send-docker-stats.mjs` | Image size per platform, docker build duration |
|
||||
| Container stack | `packages/testing/containers/telemetry.ts` | E2E stack startup times per service |
|
||||
| Container stack | `packages/quality/environments/containers/telemetry.ts` | E2E stack startup times per service |
|
||||
|
||||
## Secrets
|
||||
|
||||
|
||||
@@ -178,7 +178,7 @@ These only run if specific files changed:
|
||||
| `docker/images/n8n-base/Dockerfile` | `build-base-image.yml` | any |
|
||||
| `**/package.json`, `**/turbo.json` | `build-windows.yml` | master |
|
||||
| `packages/@n8n/ai-workflow-builder.ee/evaluations/programmatic/python/**` | `test-evals-python.yml` | any |
|
||||
| `packages/@n8n/benchmark/**` | `build-benchmark-image.yml` | master |
|
||||
| `packages/quality/efficiency/scale/benchmark/**` | `build-benchmark-image.yml` | master |
|
||||
| `packages/cli/src/public-api/**/*.yml`, `packages/cli/src/public-api/**/*.yaml`, `packages/cli/src/public-api/**/*.css`, `packages/cli/src/public-api/v1/openapi-gen/**/*.ts`, `packages/cli/scripts/build.mjs`, `packages/cli/package.json` | `util-publish-api-schema.yml` | master |
|
||||
| `packages/@n8n/instance-ai/src/**`, `packages/@n8n/instance-ai/skills/**`, `packages/@n8n/instance-ai/knowledge-base/**`, `packages/@n8n/instance-ai/evaluations/**`, `packages/cli/src/modules/instance-ai/**`, `packages/core/src/execution-engine/eval-mock-helpers.ts`, `packages/@n8n/agents/src/**` | `ci-instance-ai-evals.yml` | on PR `opened` / `reopened` / `ready_for_review` |
|
||||
| `docker/get-n8n.sh`, `docker/get-n8n-compose.yml`, `docker/test-get-n8n.sh` | `test-get-n8n.yml` | any |
|
||||
|
||||
@@ -242,7 +242,7 @@ describe('runtime filter', () => {
|
||||
'!**/CHANGELOG.md',
|
||||
'!**/*.test.ts',
|
||||
'!**/*.spec.ts',
|
||||
'!packages/testing/playwright/**',
|
||||
'!packages/quality/testing/playwright/**',
|
||||
'!packages/frontend/@n8n/storybook/**',
|
||||
'!scripts/agent-setup.mjs',
|
||||
'!scripts/backend-module/**',
|
||||
@@ -265,7 +265,7 @@ describe('runtime filter', () => {
|
||||
|
||||
it('does not trigger on playwright tests', () => {
|
||||
assert.equal(
|
||||
evaluateFilter(['packages/testing/playwright/tests/x.spec.ts'], runtimePatterns),
|
||||
evaluateFilter(['packages/quality/testing/playwright/tests/x.spec.ts'], runtimePatterns),
|
||||
false,
|
||||
);
|
||||
});
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
services:
|
||||
postgres:
|
||||
image: ${TEST_IMAGE_POSTGRES:?set it from packages/testing/containers/postgres-versions.json}
|
||||
image: ${TEST_IMAGE_POSTGRES:?set it from packages/quality/environments/containers/postgres-versions.json}
|
||||
restart: always
|
||||
environment:
|
||||
- POSTGRES_DB=n8n
|
||||
|
||||
@@ -7,7 +7,7 @@ import { fileURLToPath } from 'node:url';
|
||||
|
||||
const REPO_ROOT = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '../..');
|
||||
|
||||
export const POSTGRES_VERSIONS_PATH = 'packages/testing/containers/postgres-versions.json';
|
||||
export const POSTGRES_VERSIONS_PATH = 'packages/quality/environments/containers/postgres-versions.json';
|
||||
|
||||
/** The matrix wall is bounded by its slowest leg, so a larger runner buys nothing. */
|
||||
const RUNNER = 'blacksmith-4vcpu-ubuntu-2204';
|
||||
|
||||
@@ -30,9 +30,9 @@ describe('matchesGlob', () => {
|
||||
});
|
||||
|
||||
it('matches a prefix pattern', () => {
|
||||
assert.equal(matchesGlob('packages/testing/playwright/a.ts', 'packages/testing/**'), true);
|
||||
assert.equal(matchesGlob('packages/testing', 'packages/testing/**'), false);
|
||||
assert.equal(matchesGlob('packages/cli/a.ts', 'packages/testing/**'), false);
|
||||
assert.equal(matchesGlob('packages/quality/testing/playwright/a.ts', 'packages/quality/**'), true);
|
||||
assert.equal(matchesGlob('packages/quality', 'packages/quality/**'), false);
|
||||
assert.equal(matchesGlob('packages/cli/a.ts', 'packages/quality/**'), false);
|
||||
});
|
||||
|
||||
it('matches a literal path only exactly', () => {
|
||||
@@ -55,8 +55,8 @@ describe('matchesGlob', () => {
|
||||
'packages/cli/__snapshots__/a.ts.snap',
|
||||
'packages/nodes-base/nodes/Foo/test/fixtures/a.json',
|
||||
'packages/frontend/editor-ui/src/__mocks__/a.ts',
|
||||
'packages/testing/playwright/a.ts',
|
||||
'packages/testing',
|
||||
'packages/quality/testing/playwright/a.ts',
|
||||
'packages/quality',
|
||||
'.github/scripts/glob.test.mjs',
|
||||
'.github/workflows/ci.yml',
|
||||
'a/.hidden/b.spec.js',
|
||||
|
||||
@@ -217,7 +217,7 @@ describe('computeLineStats', () => {
|
||||
['*.snap extension', 'src/foo.snap'],
|
||||
['fixtures/ directory', 'src/fixtures/data.json'],
|
||||
['__mocks__/ directory', 'src/__mocks__/axios.ts'],
|
||||
['packages/testing/**', 'packages/testing/playwright/spec.ts'],
|
||||
['packages/quality/**', 'packages/quality/testing/playwright/spec.ts'],
|
||||
['dot-directory *.test.mjs', '.github/scripts/owners/owners.test.mjs'],
|
||||
]) {
|
||||
it(`classifies ${label} as testFiles`, () => {
|
||||
|
||||
@@ -36,8 +36,11 @@ export const TEST_PATTERNS = [
|
||||
// Fixtures and mocks
|
||||
'**/fixtures/**',
|
||||
'**/__mocks__/**',
|
||||
// Dedicated testing package
|
||||
'packages/testing/**',
|
||||
// Former packages/testing workspaces
|
||||
'packages/quality/policy/**',
|
||||
'packages/quality/testing/**',
|
||||
'packages/quality/environments/**',
|
||||
'packages/quality/efficiency/microbenchmarks/**',
|
||||
];
|
||||
|
||||
export const MISC_PATTERNS = [
|
||||
|
||||
@@ -203,14 +203,21 @@ describe('countFilteredAdditions', () => {
|
||||
assert.equal(countFilteredAdditions(files, EXCLUDE_PATTERNS), 50);
|
||||
});
|
||||
|
||||
it('applies EXCLUDE_PATTERNS to packages/testing and pnpm-lock.yaml', () => {
|
||||
it('excludes former testing workspaces but counts benchmark CLI source', () => {
|
||||
const files = [
|
||||
{ filename: 'packages/cli/src/service.ts', additions: 50 },
|
||||
{ filename: 'packages/testing/playwright/tests/workflow.spec.ts', additions: 100 },
|
||||
{ filename: 'packages/testing/playwright/pages/CanvasPage.ts', additions: 100 },
|
||||
{ filename: 'packages/quality/testing/playwright/tests/workflow.spec.ts', additions: 100 },
|
||||
{ filename: 'packages/quality/testing/playwright/pages/CanvasPage.ts', additions: 100 },
|
||||
{ filename: 'packages/quality/policy/code-health/src/cli.ts', additions: 100 },
|
||||
{ filename: 'packages/quality/environments/containers/service-stack.ts', additions: 100 },
|
||||
{
|
||||
filename: 'packages/quality/efficiency/microbenchmarks/benchmarks/bench-options.ts',
|
||||
additions: 100,
|
||||
},
|
||||
{ filename: 'packages/quality/efficiency/scale/benchmark/src/commands/list.ts', additions: 100 },
|
||||
{ filename: 'pnpm-lock.yaml', additions: 500 },
|
||||
];
|
||||
assert.equal(countFilteredAdditions(files, EXCLUDE_PATTERNS), 50);
|
||||
assert.equal(countFilteredAdditions(files, EXCLUDE_PATTERNS), 150);
|
||||
});
|
||||
|
||||
it('applies EXCLUDE_PATTERNS to markdown files', () => {
|
||||
|
||||
@@ -6,7 +6,7 @@ on:
|
||||
branches:
|
||||
- master
|
||||
paths:
|
||||
- 'packages/@n8n/benchmark/**'
|
||||
- 'packages/quality/efficiency/scale/benchmark/**'
|
||||
- 'pnpm-lock.yaml'
|
||||
- 'pnpm-workspace.yaml'
|
||||
- '.github/workflows/build-benchmark-image.yml'
|
||||
@@ -33,7 +33,7 @@ jobs:
|
||||
DOCKER_BUILD_SUMMARY: false
|
||||
with:
|
||||
context: .
|
||||
file: ./packages/@n8n/benchmark/Dockerfile
|
||||
file: ./packages/quality/efficiency/scale/benchmark/Dockerfile
|
||||
platforms: linux/amd64
|
||||
provenance: false
|
||||
push: true
|
||||
|
||||
@@ -99,8 +99,8 @@ jobs:
|
||||
mode: filter
|
||||
filters: |
|
||||
janitor:
|
||||
packages/testing/playwright/**
|
||||
packages/testing/janitor/**
|
||||
packages/quality/testing/playwright/**
|
||||
packages/quality/testing/janitor/**
|
||||
|
||||
check-static-analysis:
|
||||
name: Static Analysis
|
||||
|
||||
@@ -79,7 +79,7 @@ jobs:
|
||||
!**/CHANGELOG.md
|
||||
!**/*.test.ts
|
||||
!**/*.spec.ts
|
||||
!packages/testing/playwright/**
|
||||
!packages/quality/testing/playwright/**
|
||||
!packages/frontend/@n8n/storybook/**
|
||||
!.devcontainer/**
|
||||
!scripts/agent-setup.mjs
|
||||
@@ -92,7 +92,7 @@ jobs:
|
||||
unit:
|
||||
**
|
||||
!packages/@n8n/task-runner-python/**
|
||||
!packages/testing/playwright/**
|
||||
!packages/quality/testing/playwright/**
|
||||
!.github/**
|
||||
.github/workflows/test-unit-reusable.yml
|
||||
!**/*.md
|
||||
@@ -111,8 +111,8 @@ jobs:
|
||||
.github/workflows/prepare-docker-reusable.yml
|
||||
.github/actions/build-n8n-docker/**
|
||||
.github/actions/load-n8n-docker/**
|
||||
packages/testing/playwright/**
|
||||
packages/testing/containers/**
|
||||
packages/quality/testing/playwright/**
|
||||
packages/quality/environments/containers/**
|
||||
dev-server-smoke:
|
||||
packages/frontend/editor-ui/vite.config.mts
|
||||
# If this job becomes too slow, use a smaller glob. Do not delete the assertion.
|
||||
@@ -120,10 +120,10 @@ jobs:
|
||||
pnpm-workspace.yaml
|
||||
packages/@n8n/*/package.json
|
||||
packages/frontend/@n8n/frontend-vite-config/**
|
||||
packages/testing/playwright/tests/dev-server-smoke/**
|
||||
packages/testing/playwright/playwright.config.ts
|
||||
packages/testing/playwright/playwright-projects.ts
|
||||
packages/testing/playwright/package.json
|
||||
packages/quality/testing/playwright/tests/dev-server-smoke/**
|
||||
packages/quality/testing/playwright/playwright.config.ts
|
||||
packages/quality/testing/playwright/playwright-projects.ts
|
||||
packages/quality/testing/playwright/package.json
|
||||
.github/workflows/test-dev-server-smoke-reusable.yml
|
||||
frontend-declarations:
|
||||
packages/frontend/@n8n/design-system/**
|
||||
@@ -146,9 +146,9 @@ jobs:
|
||||
cubic.yaml
|
||||
.agents/review-rules/**
|
||||
e2e-performance:
|
||||
packages/testing/playwright/tests/performance/**
|
||||
packages/testing/playwright/utils/performance-helper.ts
|
||||
packages/testing/containers/**
|
||||
packages/quality/testing/playwright/tests/performance/**
|
||||
packages/quality/testing/playwright/utils/performance-helper.ts
|
||||
packages/quality/environments/containers/**
|
||||
.github/workflows/test-e2e-performance-reusable.yml
|
||||
# Whole directories, not the `engine-*` files alone. Engine v2
|
||||
# routing lives in ordinary control plane code: `workflow-runner.ts`
|
||||
@@ -169,8 +169,8 @@ jobs:
|
||||
packages/cli/src/webhooks/**
|
||||
packages/cli/src/workflows/triggers/**
|
||||
packages/cli/src/workflow-runner.ts
|
||||
packages/testing/containers/**
|
||||
packages/testing/playwright/**
|
||||
packages/quality/environments/containers/**
|
||||
packages/quality/testing/playwright/**
|
||||
instance-ai-workflow-eval:
|
||||
packages/@n8n/instance-ai/src/**
|
||||
packages/@n8n/instance-ai/evaluations/**
|
||||
@@ -193,10 +193,10 @@ jobs:
|
||||
packages/cli/src/**/*.integration.test.ts
|
||||
packages/cli/src/modules/instance-registry/**
|
||||
packages/cli/vitest.*
|
||||
packages/testing/containers/services/postgres.ts
|
||||
packages/testing/containers/services/redis.ts
|
||||
packages/testing/containers/postgres-versions.json
|
||||
packages/testing/containers/test-containers.ts
|
||||
packages/quality/environments/containers/services/postgres.ts
|
||||
packages/quality/environments/containers/services/redis.ts
|
||||
packages/quality/environments/containers/postgres-versions.json
|
||||
packages/quality/environments/containers/test-containers.ts
|
||||
.github/workflows/test-db-reusable.yml
|
||||
.github/scripts/db-test-matrix.mjs
|
||||
docs/generated/**
|
||||
@@ -214,10 +214,10 @@ jobs:
|
||||
packages/cli/vitest.config.base.ts
|
||||
packages/cli/vitest.config.migration.ts
|
||||
packages/cli/vitest.config.migration.testcontainers.ts
|
||||
packages/testing/containers/services/postgres.ts
|
||||
packages/testing/containers/services/redis.ts
|
||||
packages/testing/containers/postgres-versions.json
|
||||
packages/testing/containers/test-containers.ts
|
||||
packages/quality/environments/containers/services/postgres.ts
|
||||
packages/quality/environments/containers/services/redis.ts
|
||||
packages/quality/environments/containers/postgres-versions.json
|
||||
packages/quality/environments/containers/test-containers.ts
|
||||
.github/workflows/test-db-reusable.yml
|
||||
.github/scripts/db-test-matrix.mjs
|
||||
|
||||
@@ -247,7 +247,7 @@ jobs:
|
||||
env:
|
||||
CHANGED_FILES: ${{ steps.ci-filter.outputs.changed-files }}
|
||||
run: |
|
||||
PACKAGES=$(node packages/testing/janitor/dist/cli.js affected-packages | tr '\n' ' ' | sed 's/ *$//')
|
||||
PACKAGES=$(node packages/quality/testing/janitor/dist/cli.js affected-packages | tr '\n' ' ' | sed 's/ *$//')
|
||||
echo "Affected packages: $PACKAGES"
|
||||
echo "list=$PACKAGES" >> "$GITHUB_OUTPUT"
|
||||
|
||||
@@ -393,7 +393,7 @@ jobs:
|
||||
PLAYWRIGHT_PROJECT: ${{ github.event.pull_request.head.repo.fork == true && 'sqlite:e2e' || 'multi-main:e2e' }}
|
||||
run: |
|
||||
FILES_CSV=$(echo "$CHANGED_FILES" | tr '\n' ',' | sed 's/,$//')
|
||||
MATRIX=$(node packages/testing/playwright/scripts/distribute-tests.mjs --matrix 20 --orchestrate --impact "--project=$PLAYWRIGHT_PROJECT" "--files=$FILES_CSV" "--base=$MERGE_BASE")
|
||||
MATRIX=$(node packages/quality/testing/playwright/scripts/distribute-tests.mjs --matrix 20 --orchestrate --impact "--project=$PLAYWRIGHT_PROJECT" "--files=$FILES_CSV" "--base=$MERGE_BASE")
|
||||
echo "matrix=$MATRIX" >> "$GITHUB_OUTPUT"
|
||||
echo "skip-tests=$(node -e "process.stdout.write(JSON.parse(process.argv[1])[0]?.skip === true ? 'true' : 'false')" "$MATRIX")" >> "$GITHUB_OUTPUT"
|
||||
|
||||
|
||||
@@ -3,7 +3,7 @@ name: 'CI: Terraform'
|
||||
on:
|
||||
pull_request:
|
||||
paths:
|
||||
- packages/@n8n/benchmark/infra/**
|
||||
- packages/quality/efficiency/scale/benchmark/infra/**
|
||||
- .github/workflows/ci-terraform.yml
|
||||
|
||||
# The job only reads the repo and runs terraform. It writes nothing.
|
||||
@@ -16,7 +16,7 @@ jobs:
|
||||
runs-on: ubuntu-latest
|
||||
defaults:
|
||||
run:
|
||||
working-directory: packages/@n8n/benchmark/infra
|
||||
working-directory: packages/quality/efficiency/scale/benchmark/infra
|
||||
steps:
|
||||
- name: Check out project
|
||||
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
||||
|
||||
@@ -29,10 +29,10 @@ on:
|
||||
# verify themselves (both modes) before the merge.
|
||||
pull_request:
|
||||
paths:
|
||||
- 'packages/testing/playwright/tests/infrastructure/encryption/**'
|
||||
- 'packages/testing/containers/services/n8n.ts'
|
||||
- 'packages/testing/containers/services/types.ts'
|
||||
- 'packages/testing/containers/stack.ts'
|
||||
- 'packages/quality/testing/playwright/tests/infrastructure/encryption/**'
|
||||
- 'packages/quality/environments/containers/services/n8n.ts'
|
||||
- 'packages/quality/environments/containers/services/types.ts'
|
||||
- 'packages/quality/environments/containers/stack.ts'
|
||||
- '.github/workflows/encryption-upgrade-test.yml'
|
||||
|
||||
permissions:
|
||||
|
||||
@@ -36,4 +36,4 @@ jobs:
|
||||
|
||||
- name: Destroy cloud env
|
||||
run: pnpm destroy-cloud-env
|
||||
working-directory: packages/@n8n/benchmark
|
||||
working-directory: packages/quality/efficiency/scale/benchmark
|
||||
|
||||
@@ -62,11 +62,11 @@ jobs:
|
||||
|
||||
- name: Destroy any existing environment
|
||||
run: pnpm destroy-cloud-env
|
||||
working-directory: packages/@n8n/benchmark
|
||||
working-directory: packages/quality/efficiency/scale/benchmark
|
||||
|
||||
- name: Provision the environment
|
||||
run: pnpm provision-cloud-env ${{ env.DEBUG }}
|
||||
working-directory: packages/@n8n/benchmark
|
||||
working-directory: packages/quality/efficiency/scale/benchmark
|
||||
|
||||
- name: Run the benchmark
|
||||
id: benchmark
|
||||
@@ -81,7 +81,7 @@ jobs:
|
||||
--n8nTag ${{ env.N8N_TAG }} \
|
||||
--benchmarkTag ${{ env.N8N_BENCHMARK_TAG }} \
|
||||
${{ env.DEBUG }}
|
||||
working-directory: packages/@n8n/benchmark
|
||||
working-directory: packages/quality/efficiency/scale/benchmark
|
||||
|
||||
# We need to login again because the access token expires
|
||||
- name: Azure login
|
||||
@@ -95,7 +95,7 @@ jobs:
|
||||
- name: Destroy the environment
|
||||
if: always()
|
||||
run: pnpm destroy-cloud-env ${{ env.DEBUG }}
|
||||
working-directory: packages/@n8n/benchmark
|
||||
working-directory: packages/quality/efficiency/scale/benchmark
|
||||
|
||||
- name: Fail `build` job if `benchmark` step failed
|
||||
if: steps.benchmark.outcome == 'failure'
|
||||
|
||||
@@ -10,7 +10,7 @@ on:
|
||||
|
||||
env:
|
||||
NODE_OPTIONS: '--max-old-space-size=6144'
|
||||
PLAYWRIGHT_BROWSERS_PATH: packages/testing/playwright/.playwright-browsers
|
||||
PLAYWRIGHT_BROWSERS_PATH: packages/quality/testing/playwright/.playwright-browsers
|
||||
|
||||
jobs:
|
||||
smoke:
|
||||
@@ -53,6 +53,6 @@ jobs:
|
||||
with:
|
||||
name: dev-server-smoke-report
|
||||
path: |
|
||||
packages/testing/playwright/test-results/
|
||||
packages/testing/playwright/playwright-report/
|
||||
packages/quality/testing/playwright/test-results/
|
||||
packages/quality/testing/playwright/playwright-report/
|
||||
retention-days: 7
|
||||
|
||||
@@ -53,7 +53,7 @@ jobs:
|
||||
- name: Generate matrix (up to 20 shards, duration-weighted)
|
||||
id: gen
|
||||
run: |
|
||||
MATRIX=$(node packages/testing/playwright/scripts/distribute-tests.mjs --matrix 20 --orchestrate --project=coverage)
|
||||
MATRIX=$(node packages/quality/testing/playwright/scripts/distribute-tests.mjs --matrix 20 --orchestrate --project=coverage)
|
||||
echo "matrix=$MATRIX" >> "$GITHUB_OUTPUT"
|
||||
|
||||
e2e:
|
||||
@@ -215,7 +215,7 @@ jobs:
|
||||
# Merges all lcovs into one lcov.info (SF: paths normalised to repo-root-
|
||||
# relative) and builds the per-spec E2E impact map for PR selection.
|
||||
- name: Aggregate coverage + build impact map
|
||||
working-directory: packages/testing/playwright
|
||||
working-directory: packages/quality/testing/playwright
|
||||
run: |
|
||||
node scripts/aggregate-coverage.mjs \
|
||||
--shards=/tmp/shards \
|
||||
@@ -227,7 +227,7 @@ jobs:
|
||||
uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0
|
||||
with:
|
||||
name: coverage-report
|
||||
path: packages/testing/playwright/coverage/
|
||||
path: packages/quality/testing/playwright/coverage/
|
||||
retention-days: 14
|
||||
|
||||
- name: Upload Combined Coverage to Codecov
|
||||
@@ -235,7 +235,7 @@ jobs:
|
||||
uses: codecov/codecov-action@671740ac38dd9b0130fbe1cec585b89eea48d3de # v5.5.2
|
||||
with:
|
||||
token: ${{ secrets.CODECOV_TOKEN }}
|
||||
files: packages/testing/playwright/coverage/lcov.info
|
||||
files: packages/quality/testing/playwright/coverage/lcov.info
|
||||
flags: nightly-full
|
||||
name: nightly-full-coverage
|
||||
fail_ci_if_error: false
|
||||
@@ -256,7 +256,7 @@ jobs:
|
||||
env:
|
||||
CODECOV_API_TOKEN: ${{ secrets.CODECOV_API_TOKEN }}
|
||||
run: |
|
||||
node packages/testing/playwright/scripts/coverage-analysis.mjs \
|
||||
node packages/quality/testing/playwright/scripts/coverage-analysis.mjs \
|
||||
--md --top=15 --out-json=coverage-gaps.json >> "$GITHUB_STEP_SUMMARY"
|
||||
|
||||
- name: Upload Coverage Gap Report
|
||||
|
||||
@@ -5,8 +5,8 @@ on:
|
||||
pull_request:
|
||||
paths:
|
||||
- '.github/workflows/test-e2e-helm.yml'
|
||||
- 'packages/testing/containers/helm-stack.ts'
|
||||
- 'packages/testing/containers/helm-start-stack.ts'
|
||||
- 'packages/quality/environments/containers/helm-stack.ts'
|
||||
- 'packages/quality/environments/containers/helm-start-stack.ts'
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
branch:
|
||||
@@ -83,7 +83,7 @@ jobs:
|
||||
build-command: 'pnpm build'
|
||||
|
||||
- name: Install Browsers
|
||||
working-directory: packages/testing/playwright
|
||||
working-directory: packages/quality/testing/playwright
|
||||
run: pnpm exec playwright install chromium
|
||||
|
||||
- name: Install Helm
|
||||
@@ -115,7 +115,7 @@ jobs:
|
||||
echo "KUBECONFIG=${KUBECONFIG_PATH}" >> "$GITHUB_ENV"
|
||||
|
||||
- name: Run Building Blocks E2E Tests
|
||||
working-directory: packages/testing/playwright
|
||||
working-directory: packages/quality/testing/playwright
|
||||
run: |
|
||||
N8N_BASE_URL="${{ env.N8N_BASE_URL }}" \
|
||||
RESET_E2E_DB=true \
|
||||
@@ -150,8 +150,8 @@ jobs:
|
||||
with:
|
||||
name: helm-e2e-report-${{ inputs.mode || 'queue' }}
|
||||
path: |
|
||||
packages/testing/playwright/test-results/
|
||||
packages/testing/playwright/playwright-report/
|
||||
packages/quality/testing/playwright/test-results/
|
||||
packages/quality/testing/playwright/playwright-report/
|
||||
retention-days: 7
|
||||
|
||||
cleanup-ghcr:
|
||||
|
||||
@@ -74,7 +74,7 @@ jobs:
|
||||
# The aggregator auto-detects n8n version and commit SHA from the repo
|
||||
# (anchored to its own file location), so no path/version glue is needed.
|
||||
- name: Aggregate sizing matrix
|
||||
working-directory: packages/testing/playwright
|
||||
working-directory: packages/quality/testing/playwright
|
||||
run: |
|
||||
pnpm exec tsx scripts/sizing-matrix-aggregate.ts \
|
||||
--input /tmp/benchmark-reports \
|
||||
@@ -86,8 +86,8 @@ jobs:
|
||||
with:
|
||||
name: sizing-matrix
|
||||
path: |
|
||||
packages/testing/playwright/sizing-matrix.json
|
||||
packages/testing/playwright/sizing-matrix.md
|
||||
packages/quality/testing/playwright/sizing-matrix.json
|
||||
packages/quality/testing/playwright/sizing-matrix.md
|
||||
retention-days: 14
|
||||
if-no-files-found: error
|
||||
|
||||
|
||||
@@ -77,7 +77,7 @@ jobs:
|
||||
- name: Generate matrix (up to 20 shards, duration-weighted)
|
||||
id: gen
|
||||
run: |
|
||||
MATRIX=$(node packages/testing/playwright/scripts/distribute-tests.mjs --matrix 20 --orchestrate --project=sqlite:e2e)
|
||||
MATRIX=$(node packages/quality/testing/playwright/scripts/distribute-tests.mjs --matrix 20 --orchestrate --project=sqlite:e2e)
|
||||
echo "matrix=$MATRIX" >> "$GITHUB_OUTPUT"
|
||||
|
||||
e2e:
|
||||
|
||||
@@ -37,7 +37,7 @@ jobs:
|
||||
- name: Checkout
|
||||
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
||||
with:
|
||||
sparse-checkout: packages/testing/playwright/scripts/canvas-perf-sentinels.mjs
|
||||
sparse-checkout: packages/quality/testing/playwright/scripts/canvas-perf-sentinels.mjs
|
||||
sparse-checkout-cone-mode: false
|
||||
|
||||
- name: Download performance shard artifacts
|
||||
@@ -54,4 +54,4 @@ jobs:
|
||||
echo "::warning::No test-results.json found in performance artifacts — sentinel skipped."
|
||||
exit 0
|
||||
fi
|
||||
node packages/testing/playwright/scripts/canvas-perf-sentinels.mjs "${RESULTS_PATHS[@]}"
|
||||
node packages/quality/testing/playwright/scripts/canvas-perf-sentinels.mjs "${RESULTS_PATHS[@]}"
|
||||
|
||||
@@ -69,7 +69,7 @@ env:
|
||||
# Browser cache location - must match install-browsers script.
|
||||
# Absolute so nested pnpm invocations (e.g. coverage shard runner) don't
|
||||
# re-resolve a relative path against the package's INIT_CWD and double it.
|
||||
PLAYWRIGHT_BROWSERS_PATH: ${{ github.workspace }}/packages/testing/playwright/.playwright-browsers
|
||||
PLAYWRIGHT_BROWSERS_PATH: ${{ github.workspace }}/packages/quality/testing/playwright/.playwright-browsers
|
||||
# docker-artifact loads the image locally as n8nio/n8n:local; unused in local mode.
|
||||
TEST_IMAGE_N8N: 'n8nio/n8n:local'
|
||||
N8N_SKIP_LICENSES: 'true'
|
||||
@@ -123,7 +123,7 @@ jobs:
|
||||
GITHUB_RUN_ID: ${{ github.run_id }}
|
||||
GITHUB_RUN_ATTEMPT: ${{ github.run_attempt }}
|
||||
run: |
|
||||
FILTERED=$(echo "$CANDIDATES" | node packages/testing/janitor/dist/cli.js filter-shard | tr '\n' ' ' | sed 's/ *$//')
|
||||
FILTERED=$(echo "$CANDIDATES" | node packages/quality/testing/janitor/dist/cli.js filter-shard | tr '\n' ' ' | sed 's/ *$//')
|
||||
echo "specs=$FILTERED" >> "$GITHUB_OUTPUT"
|
||||
|
||||
- name: Resolve shard args
|
||||
@@ -195,13 +195,13 @@ jobs:
|
||||
with:
|
||||
name: ${{ inputs.artifact-prefix }}-shard-${{ matrix.shard }}
|
||||
path: |
|
||||
packages/testing/playwright/test-results/
|
||||
packages/testing/playwright/test-results.json
|
||||
packages/testing/playwright/playwright-report/
|
||||
packages/testing/playwright/coverage/lcov.info
|
||||
packages/testing/playwright/coverage/by-spec/
|
||||
packages/testing/playwright/coverage/index.html
|
||||
packages/testing/playwright/coverage/assets/
|
||||
packages/quality/testing/playwright/test-results/
|
||||
packages/quality/testing/playwright/test-results.json
|
||||
packages/quality/testing/playwright/playwright-report/
|
||||
packages/quality/testing/playwright/coverage/lcov.info
|
||||
packages/quality/testing/playwright/coverage/by-spec/
|
||||
packages/quality/testing/playwright/coverage/index.html
|
||||
packages/quality/testing/playwright/coverage/assets/
|
||||
retention-days: 1
|
||||
if-no-files-found: ignore
|
||||
|
||||
@@ -214,7 +214,7 @@ jobs:
|
||||
uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0
|
||||
with:
|
||||
name: ${{ inputs.artifact-prefix }}-a11y-report-shard-${{ matrix.shard }}
|
||||
path: packages/testing/playwright/a11y-report/
|
||||
path: packages/quality/testing/playwright/a11y-report/
|
||||
retention-days: 7
|
||||
if-no-files-found: ignore
|
||||
|
||||
|
||||
@@ -90,13 +90,13 @@ jobs:
|
||||
env:
|
||||
BASE_BRANCH: ${{ inputs.base-branch }}
|
||||
run: |
|
||||
if [ -f packages/testing/code-health/package.json ]; then
|
||||
if [ -f packages/quality/policy/code-health/package.json ]; then
|
||||
echo 'present=true' >> "$GITHUB_OUTPUT"
|
||||
elif [ -n "$BASE_BRANCH" ]; then
|
||||
echo 'present=false' >> "$GITHUB_OUTPUT"
|
||||
echo "::notice::packages/testing/code-health is absent on \"$BASE_BRANCH\"; nothing to verify."
|
||||
echo "::notice::packages/quality/policy/code-health is absent on \"$BASE_BRANCH\"; nothing to verify."
|
||||
else
|
||||
echo '::error::packages/testing/code-health is missing (moved or renamed?).'
|
||||
echo '::error::packages/quality/policy/code-health is missing (moved or renamed?).'
|
||||
exit 1
|
||||
fi
|
||||
|
||||
@@ -119,10 +119,10 @@ jobs:
|
||||
# REPORT_ONLY is a single flag or empty. Quoting it would pass an empty argument, which
|
||||
# the CLI reads as an (unknown) package name and rejects.
|
||||
# shellcheck disable=SC2086
|
||||
pnpm --dir packages/testing/code-health exec tsx src/cli.ts verify-npm-install --changed="$BASE_REF" $REPORT_ONLY
|
||||
pnpm --dir packages/quality/policy/code-health exec tsx src/cli.ts verify-npm-install --changed="$BASE_REF" $REPORT_ONLY
|
||||
|
||||
- name: Verify npm-install closure (all packages)
|
||||
if: steps.verifier.outputs.present == 'true' && inputs.scope == 'all'
|
||||
run: |
|
||||
# shellcheck disable=SC2086 # see the scoped step above
|
||||
pnpm --dir packages/testing/code-health exec tsx src/cli.ts verify-npm-install --all $REPORT_ONLY
|
||||
pnpm --dir packages/quality/policy/code-health exec tsx src/cli.ts verify-npm-install --all $REPORT_ONLY
|
||||
|
||||
@@ -71,7 +71,7 @@ jobs:
|
||||
- name: Resolve Postgres image
|
||||
if: fromJSON(steps.paths-filter.outputs.results).db
|
||||
run: |
|
||||
IMAGE=$(node -p "require('./packages/testing/containers/postgres-versions.json').primary")
|
||||
IMAGE=$(node -p "require('./packages/quality/environments/containers/postgres-versions.json').primary")
|
||||
echo "TEST_IMAGE_POSTGRES=$IMAGE" >> "$GITHUB_ENV"
|
||||
|
||||
- name: Start Postgres
|
||||
|
||||
+9
-9
@@ -46,17 +46,17 @@ sbom-source.cdx.json
|
||||
junit.xml
|
||||
test-results.json
|
||||
*.0x
|
||||
packages/testing/playwright/playwright-report
|
||||
packages/testing/playwright/eval-report
|
||||
packages/testing/playwright/a11y-report
|
||||
packages/testing/playwright/test-results
|
||||
packages/testing/playwright/results.xml
|
||||
packages/testing/playwright/eval-results.json
|
||||
packages/quality/testing/playwright/playwright-report
|
||||
packages/quality/testing/playwright/eval-report
|
||||
packages/quality/testing/playwright/a11y-report
|
||||
packages/quality/testing/playwright/test-results
|
||||
packages/quality/testing/playwright/results.xml
|
||||
packages/quality/testing/playwright/eval-results.json
|
||||
packages/@n8n/instance-ai/eval-results.json
|
||||
packages/@n8n/instance-ai/.eval-output/
|
||||
packages/@n8n/instance-ai/eval-pr-comment.md
|
||||
packages/testing/playwright/.playwright-browsers
|
||||
packages/testing/playwright/.playwright-cli
|
||||
packages/quality/testing/playwright/.playwright-browsers
|
||||
packages/quality/testing/playwright/.playwright-cli
|
||||
test-results/
|
||||
compiled_app_output
|
||||
trivy_report*
|
||||
@@ -65,7 +65,7 @@ docker-build-manifest.json
|
||||
packages/cli/src/modules/my-feature
|
||||
.secrets
|
||||
act-event.json
|
||||
packages/testing/**/.cursor/rules/
|
||||
packages/quality/**/.cursor/rules/
|
||||
.venv
|
||||
.ruff_cache
|
||||
__pycache__
|
||||
|
||||
@@ -310,14 +310,14 @@ extends one of those layers):
|
||||
- Run that package's `pnpm lint` and `pnpm typecheck` before committing code.
|
||||
Build first when shared types or cross-package dependencies change.
|
||||
- Use Vitest for unit tests. Use
|
||||
[Playwright](packages/testing/playwright/AGENTS.md) when a test needs its
|
||||
[Playwright](packages/quality/testing/playwright/AGENTS.md) when a test needs its
|
||||
browser, fixtures, or managed containers.
|
||||
- For Vitest packages with `@n8n/di` decorators, use
|
||||
`createVitestConfigWithDecorators` from `@n8n/vitest-config/node-decorators`.
|
||||
- Check import and mock side effects before running tests. Keep tests out of
|
||||
user-owned directories. Set `N8N_USER_FOLDER` to a test-owned directory before
|
||||
importing n8n settings. Clean up only paths that the test created.
|
||||
- CI runs [`@n8n/code-health`](packages/testing/code-health/README.md) static
|
||||
- CI runs [`@n8n/code-health`](packages/quality/policy/code-health/README.md) static
|
||||
analysis on PRs. It checks monorepo rules, including dependency hygiene and
|
||||
encryption-boundary coverage.
|
||||
|
||||
@@ -331,7 +331,7 @@ extends one of those layers):
|
||||
| Start a Codespace backend and share its port | `pnpm dev:up` |
|
||||
|
||||
The root `pnpm dev` command does not start a server. See the
|
||||
[Playwright guide](packages/testing/playwright/README.md) and the
|
||||
[Playwright guide](packages/quality/testing/playwright/README.md) and the
|
||||
[Codespaces guide](.devcontainer/codespaces/README.md) for details.
|
||||
|
||||
### Common Development Tasks
|
||||
|
||||
+2
-2
@@ -432,7 +432,7 @@ Each instance gets its own port, and the webhook URL matches the main URL. Multi
|
||||
>- This does not work with Podman out of the box. You need to [configure Testcontainers for Podman](https://podman-desktop.io/tutorial/testcontainers-with-podman) first.
|
||||
>- Alternatively, you can use colima and set DOCKER_HOST and TESTCONTAINERS_DOCKER_SOCKET_OVERRIDE environment variables as described [here](https://node.testcontainers.org/supported-container-runtimes/#colima)
|
||||
|
||||
Refer to [packages/testing/containers/README.md](packages/testing/containers/README.md) for more information.
|
||||
Refer to [packages/quality/environments/containers/README.md](packages/quality/environments/containers/README.md) for more information.
|
||||
|
||||
### Work locally with syslog
|
||||
|
||||
@@ -661,7 +661,7 @@ E2E tests can be started via one of the following commands:
|
||||
- `pnpm --filter=n8n-playwright test:local --ui` - Run tests in interactive UI mode (useful for debugging)
|
||||
- `pnpm --filter=n8n-playwright test:local --grep="test-name"` - Run specific tests matching pattern
|
||||
|
||||
See `packages/testing/playwright/README.md` for more test commands and `packages/testing/playwright/CONTRIBUTING.md` for writing guidelines.
|
||||
See `packages/quality/testing/playwright/README.md` for more test commands and `packages/quality/testing/playwright/CONTRIBUTING.md` for writing guidelines.
|
||||
|
||||
## Create custom nodes
|
||||
|
||||
|
||||
@@ -254,8 +254,7 @@ docker/get-n8n.sh @n8n-io/qa-dx required
|
||||
|
||||
# QA
|
||||
|
||||
packages/testing/ @n8n-io/qa-dx
|
||||
packages/@n8n/benchmark/ @n8n-io/qa-dx
|
||||
packages/quality/ @n8n-io/qa-dx
|
||||
packages/@n8n/vitest-config/ @n8n-io/qa-dx
|
||||
|
||||
# Encryption boundary (IAM) — the encryption area and its guardrail rules.
|
||||
|
||||
@@ -18,6 +18,7 @@
|
||||
"**/cl100k_base.json",
|
||||
"**/o200k_base.json",
|
||||
"**/*.generated.ts",
|
||||
"**/benchmark/scripts/mock-api/**",
|
||||
"**/expectations/**"
|
||||
]
|
||||
},
|
||||
|
||||
+1
-1
@@ -150,5 +150,5 @@ ignore:
|
||||
- (?s:.*/migrations/(postgresdb|sqlite|mysqldb)/.*\.ts.*)\Z
|
||||
- (?s:.*/scripts/.*)\Z
|
||||
# Test/build infrastructure packages
|
||||
- (?s:.*/packages/testing/.*)\Z
|
||||
- (?s:.*/packages/quality/.*)\Z
|
||||
- (?s:.*/packages/@n8n/(benchmark|eslint-config|eslint-plugin-community-nodes|stylelint-config|typescript-config|vitest-config)/.*)\Z
|
||||
|
||||
+1
-1
@@ -202,7 +202,7 @@ reviews:
|
||||
- docker/**
|
||||
- scripts/**
|
||||
- patches/**
|
||||
- packages/testing/**
|
||||
- packages/quality/**
|
||||
- packages/@n8n/eslint-config/**
|
||||
- packages/@n8n/stylelint-config/**
|
||||
- packages/@n8n/vitest-config/**
|
||||
|
||||
@@ -147,7 +147,7 @@ const config: KnipConfig = {
|
||||
// Only named in JSDoc type imports; @stryker-mutator/core provides it.
|
||||
ignoreDependencies: ['@stryker-mutator/api'],
|
||||
}),
|
||||
'packages/@n8n/benchmark': pkg({
|
||||
'packages/quality/efficiency/scale/benchmark': pkg({
|
||||
// The k6 runtime provides its own modules; only @types/k6 is installed.
|
||||
ignoreDependencies: ['k6'],
|
||||
}),
|
||||
@@ -204,7 +204,7 @@ const config: KnipConfig = {
|
||||
// Spawned as a binary through execFile.
|
||||
ignoreDependencies: ['agent-browser'],
|
||||
}),
|
||||
'packages/testing/playwright': pkg({
|
||||
'packages/quality/testing/playwright': pkg({
|
||||
ignoreDependencies: [
|
||||
// The e2e suite runs against the built app; the edge orders the turbo build.
|
||||
'n8n',
|
||||
|
||||
+1
-1
@@ -87,7 +87,7 @@ pre-commit:
|
||||
- merge
|
||||
- rebase
|
||||
playwright_janitor:
|
||||
glob: 'packages/testing/playwright/**/*.ts'
|
||||
glob: 'packages/quality/testing/playwright/**/*.ts'
|
||||
run: |
|
||||
files=$(echo {staged_files} | tr ' ' ',')
|
||||
pnpm --filter=n8n-playwright janitor --files="$files"
|
||||
|
||||
@@ -1,7 +0,0 @@
|
||||
{
|
||||
"$schema": "../node_modules/@biomejs/biome/configuration_schema.json",
|
||||
"extends": ["../../../biome.jsonc"],
|
||||
"files": {
|
||||
"ignore": ["scripts/mock-api/**"]
|
||||
}
|
||||
}
|
||||
@@ -33,7 +33,7 @@ const TBLS_IMAGE =
|
||||
|
||||
const POSTGRES_VERSIONS_PATH = resolve(
|
||||
REPO_ROOT,
|
||||
'packages/testing/containers/postgres-versions.json',
|
||||
'packages/quality/environments/containers/postgres-versions.json',
|
||||
);
|
||||
|
||||
function primaryPostgresImage() {
|
||||
|
||||
@@ -26,7 +26,7 @@ export const backendNetworkBoundaryConfig = tseslint.config({
|
||||
allow: [
|
||||
// The factory itself: this is where the guarded client is built.
|
||||
'packages/@n8n/backend-network/',
|
||||
'packages/@n8n/benchmark/',
|
||||
'packages/quality/efficiency/scale/benchmark/',
|
||||
],
|
||||
},
|
||||
],
|
||||
|
||||
@@ -25,7 +25,7 @@ const TEST_FILE_PATTERN = /(\.(test|spec)\.ts$)|([\\/]__tests__[\\/])|([\\/]test
|
||||
* so bare or self-naming directives of those forms swallow this rule's report,
|
||||
* and a package-level `'off'` override is invisible to lint altogether. The
|
||||
* authoritative check is the code-health rule `encryption-boundary`
|
||||
* (packages/testing/code-health): it scans the source out of band for every
|
||||
* (packages/quality/policy/code-health): it scans the source out of band for every
|
||||
* directive form and verifies that each package that can reach the primitives
|
||||
* composes the boundary config at error severity.
|
||||
*/
|
||||
|
||||
@@ -7,7 +7,7 @@ import { ExpressionEvaluator } from '../evaluator/expression-evaluator';
|
||||
// can a real bridge be built from inside the synchronous evaluate() path?
|
||||
// isolated-vm uses its sync APIs directly; QuickJS relies on the WASM module
|
||||
// cached by pool warmup (an earlier async initialize). The cold start's cost
|
||||
// is tracked in packages/testing/performance (cold-start.bench.ts).
|
||||
// is tracked in packages/quality/efficiency/microbenchmarks (cold-start.bench.ts).
|
||||
|
||||
describe(`Lazy acquisition: synchronous cold start (${engineName})`, () => {
|
||||
it('builds a working bridge synchronously when the pool is exhausted', async () => {
|
||||
|
||||
@@ -29,12 +29,12 @@ Read these docs before starting any implementation:
|
||||
|
||||
## E2E Testing
|
||||
|
||||
Tests live in `packages/testing/playwright/tests/e2e/instance-ai/`.
|
||||
Tests live in `packages/quality/testing/playwright/tests/e2e/instance-ai/`.
|
||||
|
||||
### Local-build mode (no docker, no recording — hits real Anthropic API)
|
||||
|
||||
```bash
|
||||
cd packages/testing/playwright
|
||||
cd packages/quality/testing/playwright
|
||||
export ANTHROPIC_API_KEY=sk-ant-...
|
||||
pnpm test:local:instance-ai # full suite
|
||||
pnpm test:local:instance-ai --grep "preview" # single test
|
||||
@@ -52,7 +52,7 @@ mode with a real key. This captures LLM traffic + tool traces into
|
||||
|
||||
```bash
|
||||
pnpm build:docker # from repo root — build the local n8n image first
|
||||
cd packages/testing/playwright
|
||||
cd packages/quality/testing/playwright
|
||||
ANTHROPIC_API_KEY=sk-ant-... pnpm test:container:sqlite tests/e2e/instance-ai --workers 1
|
||||
```
|
||||
|
||||
|
||||
@@ -263,8 +263,8 @@ N8N_INSTANCE_AI_SEARXNG_URL=http://searxng:8080
|
||||
INSTANCE_AI_BRAVE_SEARCH_API_KEY=BSA-xxx
|
||||
|
||||
# With sandbox (n8n sandbox service)
|
||||
# CI can start it with:
|
||||
# pnpm tsx packages/testing/containers/start-sandbox.ts --network n8n-eval-net
|
||||
# For local development, start it with:
|
||||
# pnpm --filter n8n-containers services --services sandbox --network n8n-eval-net
|
||||
N8N_INSTANCE_AI_SANDBOX_ENABLED=true
|
||||
N8N_INSTANCE_AI_SANDBOX_PROVIDER=n8n-sandbox
|
||||
N8N_SANDBOX_SERVICE_URL=https://sandbox.example.com
|
||||
|
||||
@@ -255,13 +255,13 @@ volatile request body.
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `packages/testing/playwright/tests/e2e/instance-ai/fixtures.ts` | Test fixtures — proxy setup, recording/replay orchestration |
|
||||
| `packages/quality/testing/playwright/tests/e2e/instance-ai/fixtures.ts` | Test fixtures — proxy setup, recording/replay orchestration |
|
||||
| `packages/@n8n/instance-ai/src/tracing/trace-replay.ts` | `TraceIndex`, `IdRemapper`, `TraceWriter`, JSONL parsing |
|
||||
| `packages/@n8n/instance-ai/src/tracing/langsmith-tracing.ts` | Tool wrapping — `replayWrapTools`, `recordWrapTools` |
|
||||
| `packages/@n8n/instance-ai/src/types.ts` | `InstanceAiTraceContext`, `TraceReplayMode` |
|
||||
| `packages/cli/src/modules/instance-ai/instance-ai.service.ts` | Trace mode initialization, shared state management |
|
||||
| `packages/cli/src/modules/instance-ai/instance-ai.controller.ts` | Test-only REST endpoints for trace delivery |
|
||||
| `packages/testing/containers/services/proxy.ts` | `ProxyServer` class (MockServer client) |
|
||||
| `packages/quality/environments/containers/services/proxy.ts` | `ProxyServer` class (MockServer client) |
|
||||
|
||||
### Test-Only Endpoints
|
||||
|
||||
@@ -323,7 +323,7 @@ stack entirely. Tests hit the real Anthropic API directly. This mode does
|
||||
**not** record proxy expectations.
|
||||
|
||||
```bash
|
||||
cd packages/testing/playwright
|
||||
cd packages/quality/testing/playwright
|
||||
export ANTHROPIC_API_KEY=sk-ant-...
|
||||
pnpm test:local:instance-ai # full suite
|
||||
pnpm test:local:instance-ai --grep "preview" # single test
|
||||
@@ -351,7 +351,7 @@ pnpm test:local:instance-ai --grep "preview" --headed
|
||||
vars over the generic `test:local:isolated` runner, which provides random free
|
||||
ports, a throwaway `N8N_USER_FOLDER` (so `~/.n8n` is never touched), and
|
||||
process-group cleanup. See the
|
||||
[Playwright README](../../../testing/playwright/README.md) for full details on
|
||||
[Playwright README](../../../quality/testing/playwright/README.md) for full details on
|
||||
`test:local:isolated`.
|
||||
|
||||
> **Cost note:** Each run makes real Anthropic calls. Scope with `--grep` or
|
||||
|
||||
@@ -5,7 +5,7 @@ export const backendNetworkBoundaryConfig = defineConfig({
|
||||
'n8n-local-rules/no-uncentralized-http': [
|
||||
'error',
|
||||
{
|
||||
allow: ['packages/@n8n/backend-network/', 'packages/@n8n/benchmark/'],
|
||||
allow: ['packages/@n8n/backend-network/', 'packages/quality/efficiency/scale/benchmark/'],
|
||||
},
|
||||
],
|
||||
},
|
||||
|
||||
@@ -3,7 +3,7 @@ import type { App, ComponentPublicInstance } from 'vue';
|
||||
/**
|
||||
* localStorage key that switches on app-wide component re-render counting.
|
||||
*
|
||||
* Set by the canvas performance benchmark (packages/testing/playwright) before
|
||||
* Set by the canvas performance benchmark (packages/quality/testing/playwright) before
|
||||
* the SPA boots. Real users never set it, so the global mixin below is never
|
||||
* installed in normal use and the feature carries no runtime cost for them.
|
||||
*/
|
||||
|
||||
@@ -0,0 +1,19 @@
|
||||
# Quality engineering instructions
|
||||
|
||||
Read the repository [AGENTS.md](../../AGENTS.md) first. Use the [quality engineering index](README.md) to find the tool for your task.
|
||||
|
||||
| Area | Responsibility | Current location |
|
||||
| --- | --- | --- |
|
||||
| Policy | Repository rules and static analysis | [`rules-engine`](policy/rules-engine/), [`code-health`](policy/code-health/) |
|
||||
| Testing | Test selection, architecture checks, and test orchestration | [`test-impact`](testing/test-impact/), [`janitor`](testing/janitor/), [`playwright`](testing/playwright/) |
|
||||
| Environments | Test services and deployment stacks | [`containers`](environments/containers/) |
|
||||
| Efficiency | Speed, memory, load, latency, throughput, capacity, and resource use | [Microbenchmarks](efficiency/microbenchmarks/), [Playwright performance tests](testing/playwright/tests/performance/), [infrastructure benchmarks](testing/playwright/tests/infrastructure/benchmarks/), [benchmark CLI](efficiency/scale/benchmark/) |
|
||||
|
||||
## Working rules
|
||||
|
||||
- Read the instructions in the owning package before you change its files.
|
||||
- Keep benchmark methods and results comparable when you move files. Do not change behavior during a path migration.
|
||||
- Keep existing package names and `pnpm --filter` commands when you move workspaces.
|
||||
- Update workspace discovery, CI paths, test-impact rules, ownership, documentation, and the lockfile when you move a package. Verify that each moved workspace still runs its intended checks.
|
||||
- Prefer local efficiency measurements that do not require an n8n Docker image rebuild. State clearly when a benchmark does require an image.
|
||||
- Put task instructions in focused README files. Keep inherited `AGENTS.md` files short and limited to rules for their area.
|
||||
@@ -0,0 +1,12 @@
|
||||
# Quality engineering
|
||||
|
||||
Each area owns workspaces. A test stays with its runner, even when it measures another quality concern.
|
||||
|
||||
| Area | Owns | Start here |
|
||||
| --- | --- | --- |
|
||||
| Policy | Repository rules and static checks | [Policy](policy/README.md) |
|
||||
| Testing | Test selection, architecture checks, and the Playwright runner | [Testing](testing/README.md) |
|
||||
| Environments | Services and deployment stacks for tests | [Environments](environments/README.md) |
|
||||
| Efficiency | Standalone microbenchmark and load-test tools | [Efficiency](efficiency/README.md) |
|
||||
|
||||
Use the [Playwright suite map](testing/playwright/README.md#choose-a-suite) to find browser, accessibility, memory, and infrastructure tests. Their specs remain in Playwright because they use its fixtures and projects.
|
||||
@@ -0,0 +1,8 @@
|
||||
# Efficiency instructions
|
||||
|
||||
Read the parent [quality instructions](../AGENTS.md) before you change a benchmark.
|
||||
|
||||
- Compare results with the same machine, workload, and process profile.
|
||||
- Do not change benchmark methods during a path migration.
|
||||
- Keep local measurements possible without an n8n Docker image rebuild. State when a benchmark needs an image.
|
||||
- Put task steps and result interpretation in the owning benchmark's README.
|
||||
@@ -0,0 +1,10 @@
|
||||
# Efficiency tooling
|
||||
|
||||
This area owns tools that measure speed and scale outside the Playwright runner.
|
||||
|
||||
| Tool | Use it to | Guide |
|
||||
| --- | --- | --- |
|
||||
| `@n8n/performance` | Compare code paths without building an n8n Docker image | [Microbenchmarks](microbenchmarks/README.md) |
|
||||
| `@n8n/n8n-benchmark` | Run load scenarios against an n8n instance | [Benchmark CLI](scale/benchmark/README.md) |
|
||||
|
||||
For idle memory, canvas performance, throughput, and local soak tests, use the [Playwright suite map](../testing/playwright/README.md#choose-a-suite). These specs need Playwright fixtures, projects, metrics, or managed containers.
|
||||
+1
-1
@@ -1,5 +1,5 @@
|
||||
{
|
||||
"extends": "../../../tsconfig.json",
|
||||
"extends": "../../../../tsconfig.json",
|
||||
"compilerOptions": {
|
||||
"module": "esnext",
|
||||
"moduleResolution": "bundler",
|
||||
+9
-9
@@ -38,7 +38,7 @@ WORKDIR /app
|
||||
COPY --chown=node:node ./pnpm-lock.yaml /app/pnpm-lock.yaml
|
||||
COPY --chown=node:node ./pnpm-workspace.yaml /app/pnpm-workspace.yaml
|
||||
COPY --chown=node:node ./package.json /app/package.json
|
||||
COPY --chown=node:node ./packages/@n8n/benchmark/package.json /app/packages/@n8n/benchmark/package.json
|
||||
COPY --chown=node:node ./packages/quality/efficiency/scale/benchmark/package.json /app/packages/quality/efficiency/scale/benchmark/package.json
|
||||
COPY --chown=node:node ./patches /app/patches
|
||||
COPY --chown=node:node ./scripts /app/scripts
|
||||
|
||||
@@ -47,15 +47,15 @@ RUN pnpm install --frozen-lockfile
|
||||
|
||||
# TS config files
|
||||
COPY --chown=node:node ./packages/@n8n/typescript-config /app/packages/@n8n/typescript-config
|
||||
COPY --chown=node:node ./packages/@n8n/benchmark/tsconfig.json /app/packages/@n8n/benchmark/tsconfig.json
|
||||
COPY --chown=node:node ./packages/@n8n/benchmark/tsconfig.build.json /app/packages/@n8n/benchmark/tsconfig.build.json
|
||||
COPY --chown=node:node ./packages/quality/efficiency/scale/benchmark/tsconfig.json /app/packages/quality/efficiency/scale/benchmark/tsconfig.json
|
||||
COPY --chown=node:node ./packages/quality/efficiency/scale/benchmark/tsconfig.build.json /app/packages/quality/efficiency/scale/benchmark/tsconfig.build.json
|
||||
|
||||
# Source files
|
||||
COPY --chown=node:node ./packages/@n8n/benchmark/src /app/packages/@n8n/benchmark/src
|
||||
COPY --chown=node:node ./packages/@n8n/benchmark/bin /app/packages/@n8n/benchmark/bin
|
||||
COPY --chown=node:node ./packages/@n8n/benchmark/scenarios /app/packages/@n8n/benchmark/scenarios
|
||||
COPY --chown=node:node ./packages/quality/efficiency/scale/benchmark/src /app/packages/quality/efficiency/scale/benchmark/src
|
||||
COPY --chown=node:node ./packages/quality/efficiency/scale/benchmark/bin /app/packages/quality/efficiency/scale/benchmark/bin
|
||||
COPY --chown=node:node ./packages/quality/efficiency/scale/benchmark/scenarios /app/packages/quality/efficiency/scale/benchmark/scenarios
|
||||
|
||||
WORKDIR /app/packages/@n8n/benchmark
|
||||
WORKDIR /app/packages/quality/efficiency/scale/benchmark
|
||||
RUN pnpm build
|
||||
|
||||
#
|
||||
@@ -64,7 +64,7 @@ FROM base AS runner
|
||||
|
||||
COPY --from=builder /app /app
|
||||
|
||||
WORKDIR /app/packages/@n8n/benchmark
|
||||
WORKDIR /app/packages/quality/efficiency/scale/benchmark
|
||||
USER node
|
||||
|
||||
ENTRYPOINT [ "/app/packages/@n8n/benchmark/bin/n8n-benchmark" ]
|
||||
ENTRYPOINT [ "/app/packages/quality/efficiency/scale/benchmark/bin/n8n-benchmark" ]
|
||||
+15
-11
@@ -2,10 +2,12 @@
|
||||
|
||||
Tool for executing benchmarks against an n8n instance.
|
||||
|
||||
Run the `pnpm --filter @n8n/n8n-benchmark` commands below from the repository root. pnpm runs each script in the benchmark package directory.
|
||||
|
||||
## Directory structure
|
||||
|
||||
```text
|
||||
packages/@n8n/benchmark
|
||||
packages/quality/efficiency/scale/benchmark
|
||||
├── scenarios Benchmark scenarios
|
||||
├── src Source code for the n8n-benchmark cli
|
||||
├── Dockerfile Dockerfile for the n8n-benchmark cli
|
||||
@@ -50,17 +52,17 @@ docker run -v ./scenarios:/scenarios ghcr.io/n8n-io/n8n-benchmark:latest run \
|
||||
|
||||
The benchmark suite consists of [benchmark scenarios](#benchmark-scenarios) and different [n8n setups](#n8n-setups).
|
||||
|
||||
### locally
|
||||
### Locally
|
||||
|
||||
```sh
|
||||
pnpm benchmark-locally
|
||||
pnpm --filter @n8n/n8n-benchmark benchmark-locally
|
||||
```
|
||||
|
||||
You can filter to a specific scenario and setup:
|
||||
|
||||
```sh
|
||||
# Run only the http-node scenario with the sqlite setup
|
||||
pnpm benchmark-locally --runDir /tmp/n8n-data --scenarioFilter http-node sqlite
|
||||
pnpm --filter @n8n/n8n-benchmark benchmark-locally --runDir /tmp/n8n-data --scenarioFilter http-node sqlite
|
||||
```
|
||||
|
||||
### In the cloud
|
||||
@@ -71,9 +73,9 @@ Terraform code that creates it.
|
||||
Create the environment, run the benchmarks, then delete the environment:
|
||||
|
||||
```sh
|
||||
pnpm provision-cloud-env
|
||||
pnpm benchmark-in-cloud
|
||||
pnpm destroy-cloud-env
|
||||
pnpm --filter @n8n/n8n-benchmark provision-cloud-env
|
||||
pnpm --filter @n8n/n8n-benchmark benchmark-in-cloud
|
||||
pnpm --filter @n8n/n8n-benchmark destroy-cloud-env
|
||||
```
|
||||
|
||||
## Running the `n8n-benchmark` cli
|
||||
@@ -87,7 +89,7 @@ Build the Docker image:
|
||||
```sh
|
||||
# Must be run in the repository root
|
||||
# k6 doesn't have an arm64 build available for linux, we need to build against amd64
|
||||
docker build --platform linux/amd64 -t n8n-benchmark -f packages/@n8n/benchmark/Dockerfile .
|
||||
docker build --platform linux/amd64 -t n8n-benchmark -f packages/quality/efficiency/scale/benchmark/Dockerfile .
|
||||
```
|
||||
|
||||
Run the image
|
||||
@@ -108,11 +110,13 @@ Requirements:
|
||||
- [k6](https://grafana.com/docs/k6/latest/set-up/install-k6/)
|
||||
- Node.js v24 or higher
|
||||
|
||||
Set `N8N_USER_EMAIL` and `N8N_USER_PASSWORD` in your environment before you run the CLI.
|
||||
|
||||
```sh
|
||||
pnpm build
|
||||
pnpm --filter @n8n/n8n-benchmark build > benchmark-build.log 2>&1
|
||||
|
||||
# Run tests against http://localhost:5678 with specified email and password
|
||||
N8N_USER_EMAIL=user@n8n.io N8N_USER_PASSWORD=password ./bin/n8n-benchmark run
|
||||
pnpm --filter @n8n/n8n-benchmark exec ./bin/n8n-benchmark run
|
||||
```
|
||||
|
||||
## Benchmark scenarios
|
||||
@@ -127,4 +131,4 @@ Available scenarios are located in [`./scenarios`](./scenarios/).
|
||||
|
||||
## n8n setups
|
||||
|
||||
A n8n setup defines a single n8n runtime configuration using Docker compose. Different n8n setups are located in [`./scripts/n8nSetups`](./scripts/n8nSetups).
|
||||
An n8n setup defines one n8n runtime configuration with Docker Compose. The setups are in [`scripts/n8n-setups`](scripts/n8n-setups/).
|
||||
@@ -0,0 +1,7 @@
|
||||
{
|
||||
"$schema": "../../../../../node_modules/@biomejs/biome/configuration_schema.json",
|
||||
"extends": ["../../../../../biome.jsonc"],
|
||||
"files": {
|
||||
"ignore": ["scripts/mock-api/**"]
|
||||
}
|
||||
}
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user