ci: Set up v3 branch sync and nightly build workflows (no-changelog) (#33678)

Co-authored-by: Claude Opus 4.8 (1M context) <[email protected]>
This commit is contained in:
Matsu
2026-07-09 12:49:39 +03:00
committed by GitHub
co-authored by Claude Opus 4.8
parent 31e75b9022
commit 885c250b12
13 changed files with 952 additions and 8 deletions
+1
View File
@@ -9,6 +9,7 @@ This folder contains n8n's GitHub Actions infrastructure.
| File/Folder | Purpose |
|-------------|---------|
| `WORKFLOWS.md` | Complete CI/CD documentation |
| `DEVELOPING_V3.md` | How to develop v3 features (master + 3.x branch model, opt-in flags) |
| `workflows/` | GitHub Actions workflows |
| `actions/` | Reusable composite actions |
| `scripts/` | Release & Docker automation |
+161
View File
@@ -0,0 +1,161 @@
# Developing v3 features
n8n is preparing a **v3 major release** (~October 2026). v3 is an *operational*
release: breaking changes, removals, and legacy cleanup — not a big-bang feature
launch. From July until release we keep **two long-lived branches alive at once**,
and this guide explains how to develop against them without friction.
> **TL;DR**
> - Normal feature work → land on **`master`**, behind an **opt-in feature flag**.
> - Breaking changes → a separate PR targeting **`3.x`** directly. **Never on `master`.**
> - `master` is synced into `3.x` **daily**, automatically.
## The branch model
`3.x` is not a divergent fork — it is simply **"whatever is on `master`, plus the
breaking-change commits"**. Their histories stay in lockstep via a daily sync, so
merging `3.x` back into `master` at release time is painless.
```mermaid
flowchart LR
subgraph master["master (v2 line)"]
F["Feature work<br/>(behind opt-in flags)"]
D["Deprecation notices"]
end
subgraph threex["3.x (v3 line)"]
B["Breaking-change PRs"]
end
F --> master
D --> master
master -- "daily sync (util-sync-master-to-3x.yml)" --> threex
B --> threex
threex -. "merged to master at v3 release" .-> master
```
| You want to… | Where it goes | How |
|--------------|---------------|-----|
| Ship a new feature/behavior | `master` | Behind an **opt-in flag** (see below) |
| Warn engineers a function is going away | `master` | Add a **deprecation notice** so usage drops before v3 |
| Create a migration | `master` | Create a non-destructive migration in master. After the release of v3, you can create the destructive part of migration if needed |
| Remove/change something in a breaking way | `3.x` | A **separate PR targeting `3.x`** directly |
## Developing a normal feature on `master` (behind an opt-in flag)
Land new implementations on `master` disabled by default, so they ride the daily
sync into `3.x` and can be trialed without affecting v2 users. n8n uses **PostHog**
for flags, evaluated server-side and bootstrapped to the frontend.
### Frontend (editor-ui)
1. Register the experiment in
[`packages/frontend/editor-ui/src/app/constants/experiments.ts`](../packages/frontend/editor-ui/src/app/constants/experiments.ts)
with `createExperiment`, using the next numeric index prefix:
```ts
export const MY_V3_FEATURE_EXPERIMENT = createExperiment('0XX_my_v3_feature');
```
Add its name to `EXPERIMENTS_TO_TRACK` if it should emit exposure telemetry.
2. Gate the code via the PostHog store — for a boolean opt-in flag use
`isFeatureEnabled`:
```ts
const posthog = usePostHogStore();
if (posthog.isFeatureEnabled(MY_V3_FEATURE_EXPERIMENT.name)) {
// new v3 behavior
}
```
3. Put per-experiment code in its own folder under
`packages/frontend/editor-ui/src/experiments/<name>/`.
The **`n8n:experiments` skill** ([`.agents/skills/experiments/`](../.agents/skills/experiments/SKILL.md))
is the authoritative, step-by-step procedure — including creating the disabled
PostHog flags in Staging/Production first.
### Backend (cli / config)
A backend opt-in flag is three small pieces (worked example: the
`084_eval_collections` flag):
1. **Flag key** constant in `@n8n/api-types`
(e.g. `EVAL_COLLECTIONS_FLAG = '084_eval_collections'` in
`packages/@n8n/api-types/src/schemas/eval-collections.schema.ts`).
2. **Env toggle** — an `@Env('N8N_...')` boolean defaulting to `false` in a
`@n8n/config` config class
(e.g. `N8N_EVAL_COLLECTIONS_ENABLED` in
`packages/@n8n/config/src/configs/evaluation.config.ts`).
3. **Override wiring** in `PostHogClient.applyEnvOverrides()`
([`packages/cli/src/posthog/index.ts`](../packages/cli/src/posthog/index.ts)) —
force-enable the flag when the env toggle is on:
```ts
if (this.globalConfig.evaluation.collectionsEnabled) {
overrides[EVAL_COLLECTIONS_FLAG] = true;
}
```
The override is **force-enable only**; `false` defers to PostHog.
Evaluated flags flow to the frontend through the login / current-user response,
so a single flag key can gate both backend and frontend behavior.
### Testing behind a flag
Override flags locally without touching PostHog:
- **Browser:** `window.featureFlags.override('0XX_my_v3_feature', true)`.
- **Playwright:** set the storage override in `TestRequirements`:
```ts
test.use({ requirements: {
storage: { N8N_EXPERIMENT_OVERRIDES: JSON.stringify({ '0XX_my_v3_feature': true }) },
} });
```
## Introducing a breaking change (on `3.x`)
Breaking changes go **only on `3.x`**, via a PR that targets `3.x` directly
(branch off the latest `3.x`, open the PR against `3.x`). Do not land breaking
changes on `master` — the sync guarantees `master` stays releasable as v2.
- Track the change in the [v3 breaking-changes tracker](https://www.notion.so/n8n/1a75b6e0c94f802caca3ce378d0d8046)
and the [Release v3 Linear project](https://linear.app/n8n/project/release-v3-7d7032bebbec/activity).
- Follow the `BREAKING CHANGE:` PR-title convention (see
[`pull_request_title_conventions.md`](./pull_request_title_conventions.md)).
**Deprecations land on `master`.** If you plan to remove a function/class in v3,
add a deprecation notice on `master` first so other engineers reduce usage ahead
of the breaking removal on `3.x`.
## How the daily sync works
[`util-sync-master-to-3x.yml`](./workflows/util-sync-master-to-3x.yml) runs daily
and merges `master` into `3.x`:
1. **Fast-forward** when `3.x` hasn't diverged.
2. **Three-way merge** when it has.
3. On a **merge conflict** it opens a **draft conflict PR** (labeled
`automation:v3-sync`) and posts to the **`#alerts-v3-sync`** Slack channel.
**Syncs pause until that PR is resolved and merged** — so conflicts never pile
up silently.
**Who gets pinged.** The conflict is attributed to the authors of the `3.x`
breaking commits touching the conflicted files (computed by
`.github/scripts/sync-conflict-owners.mjs`: the `master..HEAD` commits per
conflicted file, mapped to GitHub accounts). Those authors are **requested as
reviewers** on the conflict PR and listed in the `#alerts-v3-sync` message. So if you
authored the breaking commit that caused a conflict, you'll be nudged to resolve
it: fix the conflict markers on the `sync/master-to-3x` branch and merge the PR;
the next daily run then resumes normally.
## Trialing v3
`3.x` publishes nightly Docker images (see
[`build-v3-nightly.yml`](./workflows/build-v3-nightly.yml)):
```bash
docker pull n8nio/n8n:v3-nightly # latest v3 nightly
docker pull n8nio/n8n:v3-nightly-20260625 # a specific build date
```
Use these to trial v3 in docker/kubernetes before release. Do **not** use them in
production.
## See also
- [`.github/WORKFLOWS.md`](./WORKFLOWS.md) — full CI/CD + release lifecycle.
- Root [`AGENTS.md`](../AGENTS.md) — general repo guidance.
- [Branching strategy & releases (Notion)](https://www.notion.so/n8n/Major-Release-v3-Branching-strategy-and-releases-38a5b6e0c94f800881deeb11e515f543).
+17 -1
View File
@@ -410,12 +410,28 @@ Push to master/1.x
| Daily 01:30, 02:30, 03:30 | `test-benchmark-nightly.yml` | Performance benchmarks |
| Daily 04:00 | `test-e2e-vm-expressions-nightly.yml`| VM expression E2E |
| Daily 05:00 | `test-benchmark-destroy-nightly.yml`| Cleanup benchmark env |
| Daily 06:00 | `util-sync-master-to-3x.yml` | Sync master → 3.x (v3) |
| Daily 08:00 | `build-v3-nightly.yml` | Nightly v3 Docker images |
| Monday 00:00 | `util-update-node-popularity.yml` | Node usage stats |
| Monday 02:00 | `test-e2e-coverage-weekly.yml` | Weekly E2E coverage |
| Saturday 22:00 | `test-evals-ai.yml` | AI workflow evals |
---
## v3 development (master + 3.x)
During the v3 release window, `master` carries normal feature work (behind opt-in
flags) and the long-lived `3.x` branch carries breaking changes. `master` is
synced into `3.x` daily by `util-sync-master-to-3x.yml` (conflicts open a draft PR
labeled `automation:v3-sync`, request the breaking-commit authors as reviewers via
`sync-conflict-owners.mjs`, post to `#alerts-v3-sync`, and pause further syncs).
`build-v3-nightly.yml` publishes `n8nio/n8n:v3-nightly[-<date>]` images from `3.x`
by calling `docker-build-push.yml` with `ref: 3.x` + `date_tag`.
See **[`DEVELOPING_V3.md`](./DEVELOPING_V3.md)** for the full model.
---
## Custom Actions
Composite actions in `.github/actions/`:
@@ -455,7 +471,7 @@ Workflows with `workflow_call` trigger:
| `test-linting-reusable.yml` | `ref`, `nodeVersion` | ESLint |
| `test-e2e-reusable.yml` | `branch`, `test-mode`, `shards`, `runner` | Core E2E executor |
| `test-workflows-callable.yml` | `git_ref`, `compare_schemas` | Workflow tests |
| `docker-build-push.yml` | `n8n_version`, `release_type`, `push_enabled` | Docker build |
| `docker-build-push.yml` | `n8n_version`, `release_type`, `push_enabled`, `ref`, `date_tag` | Docker build |
| `sec-ci-reusable.yml` | `ref` | Security orchestrator |
| `sec-poutine-reusable.yml` | `ref` | Poutine scanner |
| `security-trivy-scan-callable.yml` | `image_ref` | Trivy scan |
+22 -3
View File
@@ -9,7 +9,7 @@ class TagGenerator {
this.githubOutput = process.env.GITHUB_OUTPUT || null;
}
generate({ image, version, platform, includeDockerHub = false, sha = '' }) {
generate({ image, version, platform, includeDockerHub = false, sha = '', date = '' }) {
let imageName = image;
let versionSuffix = '';
@@ -42,6 +42,20 @@ class TagGenerator {
tags.shaPrimaryTag = shaGhcr[0].replace(/-amd64$|-arm64$/, '');
}
// Generate additional date-based tags (e.g. v3-nightly-20260625) for nightly builds
if (date) {
const dateVersion = `${version}-${date}`;
const datePlatformTag = `${dateVersion}${versionSuffix}${platformSuffix}`;
const dateGhcr = [`ghcr.io/${this.githubOwner}/${imageName}:${datePlatformTag}`];
const dateDocker = includeDockerHub
? [`${this.dockerUsername}/${imageName}:${datePlatformTag}`]
: [];
tags.all = [...tags.all, ...dateGhcr, ...dateDocker];
tags.ghcr = [...tags.ghcr, ...dateGhcr];
tags.docker = [...tags.docker, ...dateDocker];
tags.datePrimaryTag = dateGhcr[0].replace(/-amd64$|-arm64$/, '');
}
return tags;
}
@@ -58,18 +72,21 @@ class TagGenerator {
if (tags.shaPrimaryTag) {
outputs.push(`${prefixStr}sha_primary_tag=${tags.shaPrimaryTag}`);
}
if (tags.datePrimaryTag) {
outputs.push(`${prefixStr}date_primary_tag=${tags.datePrimaryTag}`);
}
appendFileSync(this.githubOutput, outputs.join('\n') + '\n');
} else {
console.log(JSON.stringify(tags, null, 2));
}
}
generateAll({ version, platform, includeDockerHub = false, sha = '' }) {
generateAll({ version, platform, includeDockerHub = false, sha = '', date = '' }) {
const images = ['n8n', 'runners', 'runners-distroless'];
const results = {};
for (const image of images) {
const tags = this.generate({ image, version, platform, includeDockerHub, sha });
const tags = this.generate({ image, version, platform, includeDockerHub, sha, date });
const prefix = image.replace('-distroless', '_distroless');
results[prefix] = tags;
@@ -105,6 +122,7 @@ if (import.meta.url === `file://${process.argv[1]}`) {
platform: getArg('platform'),
includeDockerHub: hasFlag('include-docker'),
sha: getArg('sha') || '',
date: getArg('date') || '',
});
if (!generator.githubOutput) {
console.log(JSON.stringify(results, null, 2));
@@ -121,6 +139,7 @@ if (import.meta.url === `file://${process.argv[1]}`) {
platform: getArg('platform'),
includeDockerHub: hasFlag('include-docker'),
sha: getArg('sha') || '',
date: getArg('date') || '',
});
generator.output(tags);
}
+139
View File
@@ -0,0 +1,139 @@
#!/usr/bin/env node
/**
* Attributes a master→3.x sync conflict to the authors of the breaking commits.
*
* Since 3.x = master + breaking commits, the commits that diverged 3.x are exactly
* `<base>..HEAD` (where <base> is the fetched master SHA). Scoped to the conflicted
* files, those are the commits responsible for the conflict; their GitHub authors are
* who to nudge. Must run while the merge is unresolved — HEAD still at the pre-merge
* 3.x tip and unmerged paths present in the index.
*
* git log gives only the author name/email, and ~2/3 of n8n authors commit with a
* non-noreply email that carries no GitHub username. So the conflicted files → breaking
* SHAs analysis is done locally, and a SINGLE GraphQL call maps those few SHAs to GitHub
* logins. Bot- and unlinked-account commits resolve to a null user and are skipped.
*
* Emits a JSON object to stdout: { ownersCsv, slack, body }.
*
* Usage:
* node .github/scripts/sync-conflict-owners.mjs --base <masterSha> --sync-branch <name>
*
* Env: GITHUB_REPOSITORY (owner/repo), GH_TOKEN or GITHUB_TOKEN (for the GraphQL API).
* Requires Node 18+ (global fetch).
*/
import { execFileSync } from 'node:child_process';
import { parseArgs } from 'node:util';
export function runGit(args) {
return execFileSync('git', args, { encoding: 'utf8' }).trim();
}
// Files with unresolved conflicts in the current (in-progress) merge.
export function conflictedFiles(git = runGit) {
const out = git(['diff', '--name-only', '--diff-filter=U']);
return out ? out.split('\n').filter(Boolean) : [];
}
// Unique SHAs of the 3.x-only (breaking) commits that touched the given files.
export function breakingShas(base, files, git = runGit) {
const shas = new Set();
for (const file of files) {
const out = git(['log', `${base}..HEAD`, '--format=%H', '--', file]);
for (const sha of out.split('\n').filter(Boolean)) shas.add(sha);
}
return [...shas];
}
// Resolve commit SHAs to GitHub logins in one GraphQL call. Commits whose author has
// no linked account (unverified email, bots) resolve to a null user and are dropped.
export async function resolveLogins(repo, shas, token, fetchFn = fetch) {
if (shas.length === 0) return [];
const [owner, name] = repo.split('/');
const aliases = shas
.map((sha, i) => `c${i}: object(oid: "${sha}") { ... on Commit { author { user { login } } } }`)
.join('\n');
const query = `query($owner: String!, $name: String!) { repository(owner: $owner, name: $name) { ${aliases} } }`;
const res = await fetchFn('https://api.github.com/graphql', {
method: 'POST',
headers: {
Authorization: `Bearer ${token}`,
'Content-Type': 'application/json',
'User-Agent': 'n8n-sync-conflict-owners',
},
body: JSON.stringify({ query, variables: { owner, name } }),
});
if (!res.ok) throw new Error(`GitHub GraphQL request failed: ${res.status}`);
const json = await res.json();
if (json.errors) throw new Error(`GitHub GraphQL error: ${JSON.stringify(json.errors)}`);
const repository = json.data?.repository ?? {};
const logins = new Set();
for (const node of Object.values(repository)) {
const login = node?.author?.user?.login;
if (login) logins.add(login);
}
return [...logins].sort();
}
// Build the conflict-PR body, reviewer CSV, and Slack owner line.
export function buildOutputs({ syncBranch, files, owners }) {
const filesMd = files.map((f) => `- \`${f}\``).join('\n') || '_none detected_';
const ownersMd = owners.length
? owners.map((o) => `- @${o}`).join('\n')
: '_Could not auto-attribute — review the conflicted files manually._';
const slack = owners.length
? `Likely owners (GitHub): ${owners.map((o) => `@${o}`).join(' ')}`
: 'Could not auto-attribute owners.';
const body = [
`Automated \`master\`→\`3.x\` sync hit a merge conflict. Resolve the conflict markers on \`${syncBranch}\`, then merge this PR. **Daily syncs are paused until it is merged.**`,
'',
'### Conflicted files',
filesMd,
'',
'### Likely owners',
'Authors of the 3.x commits touching the conflicted files, requested as reviewers:',
ownersMd,
].join('\n');
return { ownersCsv: owners.join(','), slack, body };
}
async function main() {
const { values } = parseArgs({
options: {
base: { type: 'string' },
'sync-branch': { type: 'string', default: 'sync/master-to-3x' },
},
});
const base = values.base;
const syncBranch = values['sync-branch'];
const repo = process.env.GITHUB_REPOSITORY;
const token = process.env.GH_TOKEN || process.env.GITHUB_TOKEN;
if (!base) throw new Error('--base <masterSha> is required');
if (!repo) throw new Error('GITHUB_REPOSITORY env var is required');
if (!token) throw new Error('GH_TOKEN / GITHUB_TOKEN env var is required');
const files = conflictedFiles();
const shas = breakingShas(base, files);
// Degrade gracefully: a transient API failure should still open the PR
// (unattributed) rather than fail the whole sync.
let owners = [];
try {
owners = await resolveLogins(repo, shas, token);
} catch (error) {
console.error(`warning: could not resolve owners: ${error.message}`);
}
process.stdout.write(JSON.stringify(buildOutputs({ syncBranch, files, owners })));
}
if (import.meta.url === `file://${process.argv[1]}`) {
main().catch((error) => {
console.error(`Error: ${error.message}`);
process.exit(1);
});
}
@@ -0,0 +1,81 @@
import assert from 'node:assert/strict';
import { test } from 'node:test';
import { breakingShas, resolveLogins, buildOutputs } from './sync-conflict-owners.mjs';
test('breakingShas collects unique SHAs across the conflicted files only', () => {
const calls = [];
const git = (args) => {
calls.push(args);
// args: ['log', 'BASE..HEAD', '--format=%H', '--', <file>]
const file = args.at(-1);
if (file === 'a.ts') return 'sha1\nsha2\n';
if (file === 'b.ts') return 'sha2\nsha3'; // sha2 shared -> deduped
return '';
};
const shas = breakingShas('BASE', ['a.ts', 'b.ts'], git);
assert.deepEqual(shas, ['sha1', 'sha2', 'sha3']);
assert.equal(calls.length, 2);
assert.deepEqual(calls[0], ['log', 'BASE..HEAD', '--format=%H', '--', 'a.ts']);
});
test('resolveLogins maps SHAs to logins in one call, dropping unlinked/bot authors', async () => {
let calls = 0;
const fetchFn = async (url, opts) => {
calls++;
assert.equal(url, 'https://api.github.com/graphql');
const query = JSON.parse(opts.body).query;
assert.match(query, /c0: object\(oid: "sha1"\)/);
assert.match(query, /c2: object\(oid: "sha3"\)/);
return {
ok: true,
json: async () => ({
data: {
repository: {
c0: { author: { user: { login: 'bob' } } },
c1: { author: { user: { login: 'alice' } } },
c2: { author: { user: null } }, // unlinked / bot -> dropped
},
},
}),
};
};
const owners = await resolveLogins('n8n-io/n8n', ['sha1', 'sha2', 'sha3'], 't', fetchFn);
assert.equal(calls, 1); // single batched request
assert.deepEqual(owners, ['alice', 'bob']); // sorted, deduped, null dropped
});
test('resolveLogins makes no request when there are no SHAs', async () => {
let calls = 0;
const fetchFn = async () => {
calls++;
return { ok: true, json: async () => ({ data: { repository: {} } }) };
};
assert.deepEqual(await resolveLogins('r', [], 't', fetchFn), []);
assert.equal(calls, 0);
});
test('resolveLogins throws on API/GraphQL errors (caller degrades gracefully)', async () => {
const httpError = async () => ({ ok: false, status: 502, json: async () => ({}) });
const gqlError = async () => ({ ok: true, json: async () => ({ errors: [{ message: 'bad' }] }) });
await assert.rejects(resolveLogins('r', ['s'], 't', httpError), /502/);
await assert.rejects(resolveLogins('r', ['s'], 't', gqlError), /GraphQL error/);
});
test('buildOutputs formats reviewers, slack line, and PR body with owners', () => {
const out = buildOutputs({ syncBranch: 'sync/master-to-3x', files: ['packages/cli/x.ts'], owners: ['alice', 'bob'] });
assert.equal(out.ownersCsv, 'alice,bob');
assert.equal(out.slack, 'Likely owners (GitHub): @alice @bob');
assert.match(out.body, /### Conflicted files/);
assert.match(out.body, /- `packages\/cli\/x\.ts`/);
assert.match(out.body, /- @alice/);
assert.match(out.body, /- @bob/);
assert.match(out.body, /Daily syncs are paused until it is merged/);
});
test('buildOutputs degrades gracefully when nothing could be attributed', () => {
const out = buildOutputs({ syncBranch: 'sync/master-to-3x', files: ['x.ts'], owners: [] });
assert.equal(out.ownersCsv, '');
assert.equal(out.slack, 'Could not auto-attribute owners.');
assert.match(out.body, /Could not auto-attribute/);
});
+171
View File
@@ -0,0 +1,171 @@
#!/usr/bin/env node
/**
* Syncs the master branch into the long-lived 3.x branch.
*
* During the v3 development window, master carries normal feature work (behind
* opt-in flags) and 3.x is "master + breaking-change commits". This script
* fast-forwards 3.x when possible, falls back to a three-way merge when 3.x has
* diverged, and — when a merge conflict occurs — opens a draft conflict PR,
* attributing it to the authors of the breaking commits. No further syncs run
* until that PR is resolved and merged (see the halt gate below).
*
* Runs from a checkout of the 3.x branch (fetch-depth 0). Assumes credentials
* are NOT persisted by checkout — pushes go through an explicit token URL.
*
* On conflict, emits `conflict_pr` and `conflict_owners` to $GITHUB_OUTPUT so a
* downstream job can post to Slack.
*
* Env: GH_TOKEN (installation token with contents/pull-requests/issues write),
* GITHUB_REPOSITORY (owner/repo, auto-provided by Actions).
* Requires Node 18+ (global fetch) and the `gh` CLI on PATH.
*
* See .github/DEVELOPING_V3.md for the full v3 development model.
*/
import { execFileSync } from 'node:child_process';
import { appendFileSync } from 'node:fs';
import { conflictedFiles, breakingShas, resolveLogins, buildOutputs } from './sync-conflict-owners.mjs';
export const TARGET_BRANCH = '3.x';
export const SYNC_BRANCH = 'sync/master-to-3x';
export const CONFLICT_LABEL = 'automation:v3-sync';
const BOT_NAME = 'n8n-assistant[bot]';
const BOT_EMAIL = 'n8n-assistant[bot]@users.noreply.github.com';
// Real command runners. Each takes an args array and returns trimmed stdout,
// throwing on a non-zero exit (mirrors `set -e`). Injectable for tests.
const runGit = (args, opts = {}) => execFileSync('git', args, { encoding: 'utf8', ...opts }).trim();
const runGh = (args, opts = {}) => execFileSync('gh', args, { encoding: 'utf8', ...opts }).trim();
// True when a previous conflict PR is still open — the halt gate.
export function hasOpenConflictPr(gh, label = CONFLICT_LABEL) {
const out = gh(['pr', 'list', '--state', 'open', '--label', label, '--json', 'number']);
return JSON.parse(out || '[]').length > 0;
}
// Attempt the merge. `git merge` fast-forwards when 3.x has not diverged and
// does a three-way merge otherwise; a non-zero exit means a conflict.
export function tryMerge(git, masterSha, log = console.log) {
try {
git(['merge', '--no-edit', masterSha]);
return true;
} catch (error) {
// Surface the merge output (conflict summary) before falling back.
if (error.stdout) log(String(error.stdout).trim());
return false;
}
}
// Append key=value lines to $GITHUB_OUTPUT (no-op when running outside Actions).
export function writeGithubOutput(obj, env = process.env) {
const path = env.GITHUB_OUTPUT;
if (!path) return;
const lines = Object.entries(obj)
.map(([k, v]) => `${k}=${v ?? ''}`)
.join('\n');
appendFileSync(path, lines + '\n', 'utf8');
}
/**
* Record the conflicted state on the sync branch and open a draft PR, attributing
* it to the authors of the 3.x breaking commits touching the conflicted files.
* Runs while the merge is unresolved (HEAD still at the pre-merge 3.x tip,
* unmerged paths present) — before committing below.
*
* @returns {Promise<{ prUrl: string, ownersSlack: string }>}
*/
export async function openConflictPr({ git, gh, repo, token, masterSha, pushUrl, fetchFn = fetch, log = console.log }) {
const files = conflictedFiles(git);
const shas = breakingShas(masterSha, files, git);
// Degrade gracefully: a transient API failure should still open the PR
// (unattributed) rather than fail the whole sync.
let owners = [];
try {
owners = await resolveLogins(repo, shas, token, fetchFn);
} catch (error) {
log(`warning: could not resolve owners: ${error.message}`);
}
const { ownersCsv, slack, body } = buildOutputs({ syncBranch: SYNC_BRANCH, files, owners });
// Record the conflicted state (with markers) on the PR branch so it can be
// resolved in review, mirroring the backport conflict convention.
git(['add', '-A']);
git(['commit', '--no-edit']);
git(['push', '--force', pushUrl, `HEAD:refs/heads/${SYNC_BRANCH}`]);
// Ensure the label exists (idempotent), then open the draft conflict PR.
gh(['label', 'create', CONFLICT_LABEL, '--color', 'B60205', '--description', 'master→3.x sync conflict', '--force']);
const prUrl = gh([
'pr', 'create', '--draft',
'--base', TARGET_BRANCH,
'--head', SYNC_BRANCH,
'--label', CONFLICT_LABEL,
'--title', 'chore: Resolve master→3.x sync conflict',
'--body', body,
]);
// Request owners as reviewers (best-effort: the API rejects the PR author
// and non-collaborators, so a failure here must not fail the sync).
if (ownersCsv) {
try {
gh(['pr', 'edit', prUrl, '--add-reviewer', ownersCsv]);
} catch {
log(`::warning::could not request some reviewers: ${ownersCsv}`);
}
}
return { prUrl, ownersSlack: slack };
}
export async function sync({
git = runGit,
gh = runGh,
env = process.env,
fetchFn = fetch,
log = console.log,
} = {}) {
const token = env.GH_TOKEN || env.GITHUB_TOKEN;
const repo = env.GITHUB_REPOSITORY;
if (!token) throw new Error('GH_TOKEN / GITHUB_TOKEN env var is required');
if (!repo) throw new Error('GITHUB_REPOSITORY env var is required');
// Authenticated push URL (credentials are not persisted by checkout).
const pushUrl = `https://x-access-token:${token}@github.com/${repo}.git`;
// Halt gate: if a previous conflict PR is still open, do nothing until it is
// resolved and merged.
if (hasOpenConflictPr(gh)) {
log(`An open '${CONFLICT_LABEL}' conflict PR exists; skipping sync until it is resolved and merged.`);
return;
}
git(['config', 'user.name', BOT_NAME]);
git(['config', 'user.email', BOT_EMAIL]);
git(['fetch', 'origin', 'master']);
// Pin to the fetched SHA — a command-line refspec doesn't reliably update the
// origin/master tracking ref, so FETCH_HEAD is the unambiguous target.
const masterSha = git(['rev-parse', 'FETCH_HEAD']);
if (tryMerge(git, masterSha, log)) {
git(['push', pushUrl, `HEAD:${TARGET_BRANCH}`]);
log('Synced master into 3.x.');
return;
}
log('Merge conflict encountered — attributing owners and opening a conflict PR.');
const { prUrl, ownersSlack } = await openConflictPr({ git, gh, repo, token, masterSha, pushUrl, fetchFn, log });
writeGithubOutput({ conflict_pr: prUrl, conflict_owners: ownersSlack }, env);
}
// Only run when executed directly, not when imported by tests.
if (import.meta.url === `file://${process.argv[1]}`) {
sync().catch((error) => {
console.error(`Error: ${error.message}`);
process.exit(1);
});
}
+142
View File
@@ -0,0 +1,142 @@
import assert from 'node:assert/strict';
import { test } from 'node:test';
import {
hasOpenConflictPr,
tryMerge,
openConflictPr,
sync,
CONFLICT_LABEL,
SYNC_BRANCH,
TARGET_BRANCH,
} from './sync-master-to-3x.mjs';
// A git/gh stub: routes calls by a matcher, records every invocation.
function makeStub(routes = []) {
const calls = [];
const fn = (args) => {
calls.push(args);
for (const [match, result] of routes) {
if (match(args)) return typeof result === 'function' ? result(args) : result;
}
return '';
};
fn.calls = calls;
return fn;
}
const okFetch = (logins) => async () => ({
ok: true,
json: async () => ({ data: { repository: Object.fromEntries(logins.map((l, i) => [`c${i}`, { author: { user: { login: l } } }])) } }),
});
test('hasOpenConflictPr reflects the open-PR count from gh', () => {
const empty = makeStub([[() => true, '[]']]);
assert.equal(hasOpenConflictPr(empty), false);
assert.deepEqual(empty.calls[0], ['pr', 'list', '--state', 'open', '--label', CONFLICT_LABEL, '--json', 'number']);
const one = makeStub([[() => true, JSON.stringify([{ number: 42 }])]]);
assert.equal(hasOpenConflictPr(one), true);
});
test('tryMerge returns true when the merge succeeds', () => {
const git = makeStub([[(a) => a[0] === 'merge', '']]);
assert.equal(tryMerge(git, 'MASTER', () => {}), true);
assert.deepEqual(git.calls[0], ['merge', '--no-edit', 'MASTER']);
});
test('tryMerge returns false and logs merge output on conflict', () => {
const logged = [];
const git = () => {
const err = new Error('merge failed');
err.stdout = 'CONFLICT (content): x.ts\n';
throw err;
};
assert.equal(tryMerge(git, 'MASTER', (m) => logged.push(m)), false);
assert.match(logged.join('\n'), /CONFLICT/);
});
test('sync fast-forwards and pushes to 3.x on a clean merge', async () => {
const git = makeStub([
[(a) => a[0] === 'rev-parse', 'MASTERSHA'],
[(a) => a[0] === 'merge', ''],
]);
const gh = makeStub([[(a) => a[0] === 'pr' && a[1] === 'list', '[]']]);
const env = { GH_TOKEN: 'tok', GITHUB_REPOSITORY: 'n8n-io/n8n' };
await sync({ git, gh, env, log: () => {} });
const push = git.calls.find((a) => a[0] === 'push');
assert.ok(push, 'expected a push');
assert.equal(push[1], 'https://x-access-token:[email protected]/n8n-io/n8n.git');
assert.equal(push[2], `HEAD:${TARGET_BRANCH}`);
// No PR created on a clean merge.
assert.equal(gh.calls.some((a) => a[0] === 'pr' && a[1] === 'create'), false);
});
test('sync halts (no fetch/merge) when a conflict PR is already open', async () => {
const git = makeStub();
const gh = makeStub([[(a) => a[0] === 'pr' && a[1] === 'list', JSON.stringify([{ number: 7 }])]]);
await sync({ git, gh, env: { GH_TOKEN: 't', GITHUB_REPOSITORY: 'n8n-io/n8n' }, log: () => {} });
assert.equal(git.calls.length, 0, 'must not touch git while halted');
});
test('sync opens a conflict PR and writes outputs on merge conflict', async () => {
const git = makeStub([
[(a) => a[0] === 'rev-parse', 'MASTERSHA'],
[(a) => a[0] === 'merge', () => { const e = new Error('conflict'); e.status = 1; throw e; }],
[(a) => a[0] === 'diff', 'packages/cli/x.ts'],
[(a) => a[0] === 'log', 'sha1'],
]);
const gh = makeStub([
[(a) => a[0] === 'pr' && a[1] === 'list', '[]'],
[(a) => a[0] === 'pr' && a[1] === 'create', 'https://github.com/n8n-io/n8n/pull/99'],
]);
// No GITHUB_OUTPUT set → writeGithubOutput no-ops; assert on the gh/git calls instead.
const env = { GH_TOKEN: 'tok', GITHUB_REPOSITORY: 'n8n-io/n8n' };
await sync({ git, gh, env, fetchFn: okFetch(['alice']), log: () => {} });
const create = gh.calls.find((a) => a[0] === 'pr' && a[1] === 'create');
assert.ok(create, 'expected a PR to be created');
assert.ok(create.includes('--draft'));
assert.equal(create[create.indexOf('--base') + 1], TARGET_BRANCH);
assert.equal(create[create.indexOf('--head') + 1], SYNC_BRANCH);
// Owner requested as reviewer.
const edit = gh.calls.find((a) => a[0] === 'pr' && a[1] === 'edit');
assert.ok(edit, 'expected reviewers to be requested');
assert.equal(edit[edit.indexOf('--add-reviewer') + 1], 'alice');
// Conflicted state force-pushed to the sync branch.
const push = git.calls.find((a) => a[0] === 'push' && a.includes('--force'));
assert.ok(push, 'expected a force push to the sync branch');
assert.equal(push.at(-1), `HEAD:refs/heads/${SYNC_BRANCH}`);
});
test('openConflictPr degrades gracefully when owner resolution fails', async () => {
const git = makeStub([
[(a) => a[0] === 'diff', 'x.ts'],
[(a) => a[0] === 'log', 'sha1'],
]);
const gh = makeStub([[(a) => a[0] === 'pr' && a[1] === 'create', 'https://github.com/n8n-io/n8n/pull/1']]);
const failingFetch = async () => ({ ok: false, status: 500, json: async () => ({}) });
const { prUrl, ownersSlack } = await openConflictPr({
git,
gh,
repo: 'n8n-io/n8n',
token: 't',
masterSha: 'MASTER',
pushUrl: 'https://push',
fetchFn: failingFetch,
log: () => {},
});
assert.equal(prUrl, 'https://github.com/n8n-io/n8n/pull/1');
assert.equal(ownersSlack, 'Could not auto-attribute owners.');
// No reviewer request when there are no owners.
assert.equal(gh.calls.some((a) => a[0] === 'pr' && a[1] === 'edit'), false);
});
+46
View File
@@ -0,0 +1,46 @@
# Builds and publishes nightly Docker images for the v3 (3.x) branch.
#
# Produces n8nio/n8n:v3-nightly and a timestamped n8nio/n8n:v3-nightly-<date>
# (plus the matching runners images) so v3 can be trialed in docker/kubernetes
# before release. Reuses the shared docker-build-push.yml pipeline, pointing it
# at the 3.x branch via the `ref` input.
#
# Lives on master so the schedule fires (scheduled runs only trigger on the
# default branch), but builds the 3.x branch's code.
name: 'Build: v3 Nightly Docker Images'
on:
schedule:
# 08:00 UTC — after the master→3.x sync (06:00) so the image reflects the latest sync
- cron: '0 8 * * *'
workflow_dispatch:
jobs:
prepare:
name: Compute date tag
if: github.repository == 'n8n-io/n8n'
runs-on: ubuntu-latest
permissions: {}
outputs:
date_tag: ${{ steps.date.outputs.date }}
steps:
- name: Compute date tag
id: date
run: echo "date=$(date -u +%Y%m%d)" >> "$GITHUB_OUTPUT"
build:
name: Build and push v3-nightly images
needs: prepare
# No permissions/secrets caps here on purpose: this mirrors the proven
# release-publish.yml caller so docker-build-push.yml runs with the permissions
# (its own per-job blocks + repo defaults) and org secrets it already relies on.
# Capping here risks starving its sub-jobs (SLSA/attestation/scan/push).
uses: ./.github/workflows/docker-build-push.yml # zizmor: ignore[excessive-permissions,secrets-inherit]
with:
ref: '3.x'
n8n_version: 'v3-nightly'
release_type: 'nightly'
date_tag: ${{ needs.prepare.outputs.date_tag }}
push_enabled: true
secrets: inherit
+3 -1
View File
@@ -41,5 +41,7 @@ jobs:
env:
GITHUB_TOKEN: ${{ github.token }}
STALE_DAYS: ${{ inputs.days || '100' }}
EXCLUDE_BRANCHES: ${{ inputs.exclude || '' }}
# Long-lived major branches are also protected by their deletion rulesets;
# this static keep-list is a safety net for scheduled runs (which pass no exclude input).
EXCLUDE_BRANCHES: ${{ inputs.exclude || '1.x,3.x' }}
run: node .github/scripts/stale/clean-stale-branches.mjs --days="$STALE_DAYS" ${{ !inputs.dry-run && '--execute' || '--dry-run' }}
+61 -3
View File
@@ -29,6 +29,16 @@ on:
required: false
type: boolean
default: true
ref:
description: 'Git ref (branch/tag/sha) to build from. Empty checks out the triggering ref.'
required: false
type: string
default: ''
date_tag:
description: 'Optional date suffix for an extra <version>-<date> tag (e.g. 20260625)'
required: false
type: string
default: ''
workflow_dispatch:
inputs:
@@ -52,9 +62,18 @@ jobs:
push_enabled: ${{ steps.context.outputs.push_enabled }}
push_to_docker: ${{ steps.context.outputs.push_to_docker }}
build_matrix: ${{ steps.context.outputs.build_matrix }}
short_sha: ${{ steps.sha.outputs.short_sha }}
steps:
- name: Checkout code
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
ref: ${{ inputs.ref }}
- name: Resolve built commit SHA
id: sha
# From the checked-out working tree (honours `ref`), not GITHUB_SHA — which
# is the triggering ref (e.g. master) and would mislabel a ref-override build.
run: echo "short_sha=$(git rev-parse --short=7 HEAD)" >> "$GITHUB_OUTPUT"
- name: Determine build context
id: context
@@ -87,10 +106,14 @@ jobs:
n8n_sha_manifest_tag: ${{ steps.determine-tags.outputs.n8n_sha_primary_tag }}
runners_sha_manifest_tag: ${{ steps.determine-tags.outputs.runners_sha_primary_tag }}
runners_distroless_sha_manifest_tag: ${{ steps.determine-tags.outputs.runners_distroless_sha_primary_tag }}
n8n_date_manifest_tag: ${{ steps.determine-tags.outputs.n8n_date_primary_tag }}
runners_date_manifest_tag: ${{ steps.determine-tags.outputs.runners_date_primary_tag }}
runners_distroless_date_manifest_tag: ${{ steps.determine-tags.outputs.runners_distroless_date_primary_tag }}
steps:
- name: Checkout code
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
ref: ${{ inputs.ref }}
fetch-depth: 0
- name: Setup and Build
@@ -104,12 +127,21 @@ jobs:
- name: Determine Docker tags for all images
id: determine-tags
env:
DATE_TAG: ${{ inputs.date_tag }}
SHORT_SHA: ${{ needs.determine-build-context.outputs.short_sha }}
run: |
# Build the optional --date flag in shell (not via a template expansion) so
# the caller-supplied input never expands directly into the command line.
DATE_ARGS=()
if [ -n "${DATE_TAG:-}" ]; then DATE_ARGS=(--date "$DATE_TAG"); fi
node .github/scripts/docker/docker-tags.mjs \
--all \
--version "${{ needs.determine-build-context.outputs.n8n_version }}" \
--platform "${{ matrix.docker_platform }}" \
--sha "${GITHUB_SHA::7}" \
--sha "$SHORT_SHA" \
"${DATE_ARGS[@]}" \
${{ needs.determine-build-context.outputs.push_to_docker == 'true' && '--include-docker' || '' }}
echo "=== Generated Docker Tags ==="
@@ -204,6 +236,10 @@ jobs:
dockerhub-password: ${{ secrets.DOCKER_PASSWORD }}
- name: Create GHCR multi-arch manifests
env:
N8N_DATE_MANIFEST_TAG: ${{ needs.build-and-push-docker.outputs.n8n_date_manifest_tag }}
RUNNERS_DATE_MANIFEST_TAG: ${{ needs.build-and-push-docker.outputs.runners_date_manifest_tag }}
RUNNERS_DISTROLESS_DATE_MANIFEST_TAG: ${{ needs.build-and-push-docker.outputs.runners_distroless_date_manifest_tag }}
run: |
RELEASE_TYPE="${{ needs.determine-build-context.outputs.release_type }}"
@@ -242,10 +278,19 @@ jobs:
create_manifest "runners (sha)" "${{ needs.build-and-push-docker.outputs.runners_sha_manifest_tag }}"
create_manifest "runners-distroless (sha)" "${{ needs.build-and-push-docker.outputs.runners_distroless_sha_manifest_tag }}"
# Create date-tagged manifests. The *_DATE_MANIFEST_TAG vars are empty unless the
# date_tag input was set (e.g. nightly), and create_manifest skips empty tags —
# so these calls are no-ops on non-dated builds.
create_manifest "n8n (date)" "$N8N_DATE_MANIFEST_TAG"
create_manifest "runners (date)" "$RUNNERS_DATE_MANIFEST_TAG"
create_manifest "runners-distroless (date)" "$RUNNERS_DISTROLESS_DATE_MANIFEST_TAG"
- name: Create Docker Hub manifests
if: needs.determine-build-context.outputs.push_to_docker == 'true'
env:
DOCKER_USERNAME: ${{ secrets.DOCKER_USERNAME }}
DATE_TAG: ${{ inputs.date_tag }}
SHORT_SHA: ${{ needs.determine-build-context.outputs.short_sha }}
run: |
VERSION="${{ needs.determine-build-context.outputs.n8n_version }}"
DOCKER_BASE="$DOCKER_USERNAME"
@@ -257,8 +302,6 @@ jobs:
["runners-distroless"]="${VERSION}-distroless"
)
SHORT_SHA="${GITHUB_SHA::7}"
for image in "${!images[@]}"; do
TAG_SUFFIX="${images[$image]}"
IMAGE_NAME="${image//-distroless/}" # Remove -distroless from image name
@@ -282,6 +325,21 @@ jobs:
--tag "${DOCKER_BASE}/${IMAGE_NAME}:${SHA_SUFFIX}" \
"${DOCKER_BASE}/${IMAGE_NAME}:${SHA_SUFFIX}-amd64" \
"${DOCKER_BASE}/${IMAGE_NAME}:${SHA_SUFFIX}-arm64"
# Create date-tagged manifest when a date suffix was provided (e.g. v3-nightly-20260625)
# Mirrors the SHA suffix placement: <version>-<date> and <version>-<date>-distroless
if [[ -n "$DATE_TAG" ]]; then
if [[ "$image" == *"-distroless"* ]]; then
DATE_SUFFIX="${VERSION}-${DATE_TAG}-distroless"
else
DATE_SUFFIX="${TAG_SUFFIX}-${DATE_TAG}"
fi
echo "Creating Docker Hub date manifest for $image: ${DATE_SUFFIX}"
docker buildx imagetools create \
--tag "${DOCKER_BASE}/${IMAGE_NAME}:${DATE_SUFFIX}" \
"${DOCKER_BASE}/${IMAGE_NAME}:${DATE_SUFFIX}-amd64" \
"${DOCKER_BASE}/${IMAGE_NAME}:${DATE_SUFFIX}-arm64"
fi
done
- name: Get manifest digests for attestation
@@ -0,0 +1,105 @@
# Syncs the master branch into the long-lived 3.x branch daily.
#
# During the v3 development window, master carries normal feature work (behind
# opt-in flags) and 3.x is "master + breaking-change commits". This workflow
# fast-forwards 3.x when possible, falls back to a three-way merge when 3.x has
# diverged, and — when a merge conflict occurs — opens a draft conflict PR and
# posts to #alerts-v3-sync. No further syncs run until that PR is resolved and merged.
#
# See .github/DEVELOPING_V3.md for the full v3 development model.
name: 'Util: Sync master to 3.x'
on:
schedule:
- cron: '0 6 * * *'
workflow_dispatch:
# Serialize syncs — never run two at once.
concurrency:
group: sync-master-to-3x
cancel-in-progress: false
# Least privilege by default; each job opts into exactly what it needs.
permissions: {}
jobs:
sync:
name: Sync master into 3.x
if: github.repository == 'n8n-io/n8n'
runs-on: ubuntu-latest
permissions:
contents: write
pull-requests: write
outputs:
conflict_pr: ${{ steps.sync.outputs.conflict_pr }}
conflict_owners: ${{ steps.sync.outputs.conflict_owners }}
steps:
- name: Generate GitHub App Token
id: app-token
uses: actions/create-github-app-token@29824e69f54612133e76f7eaac726eef6c875baf # v2.2.1
with:
app-id: ${{ secrets.N8N_ASSISTANT_APP_ID }}
private-key: ${{ secrets.N8N_ASSISTANT_PRIVATE_KEY }}
# Scope the installation token to only what the sync needs.
permission-contents: write # push to 3.x / the conflict branch
permission-pull-requests: write # open the conflict PR
permission-issues: write # create the conflict label
- name: Checkout 3.x
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
ref: 3.x
fetch-depth: 0
persist-credentials: false # we push with an explicit token URL instead
- name: Sync master into 3.x
id: sync
env:
GH_TOKEN: ${{ steps.app-token.outputs.token }}
run: node .github/scripts/sync-master-to-3x.mjs
notify-conflict:
name: Notify Slack about conflict PR
needs: [sync]
if: ${{ needs.sync.outputs.conflict_pr != '' }}
runs-on: ubuntu-latest
permissions:
contents: read # checkout the slack scripts
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
sparse-checkout: .github/scripts/slack
sparse-checkout-cone-mode: false
persist-credentials: false
- name: Notify Slack
env:
SLACK_TOKEN: ${{ secrets.QBOT_SLACK_TOKEN }}
PR_URL: ${{ needs.sync.outputs.conflict_pr }}
OWNERS_TEXT: ${{ needs.sync.outputs.conflict_owners }}
run: |
node .github/scripts/slack/notify.mjs \
--channel '#alerts-v3-sync' \
--text "<${PR_URL}|master→3.x sync hit a conflict — resolve this PR>. Daily syncs are paused until it is merged. ${OWNERS_TEXT}"
notify-on-failure:
name: Notify Slack on failure
needs: [sync]
if: ${{ always() && needs.sync.result == 'failure' }}
runs-on: ubuntu-latest
permissions:
contents: read # checkout the slack scripts
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
sparse-checkout: .github/scripts/slack
sparse-checkout-cone-mode: false
persist-credentials: false
- name: Notify Slack
env:
SLACK_TOKEN: ${{ secrets.QBOT_SLACK_TOKEN }}
RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
run: |
node .github/scripts/slack/notify.mjs \
--channel '#alerts-v3-sync' \
--text "<${RUN_URL}|master→3.x sync workflow failed unexpectedly>"
+3
View File
@@ -22,6 +22,9 @@ frontend, and extensible node-based workflow engine.
suggested by Linear, **unless it is a security fix** (see Security Fix
Hygiene below)
- Use mermaid diagrams in MD files when you need to visualise something
- **Developing v3 features:** land normal feature work on `master` behind an
opt-in flag; introduce breaking changes only on the `3.x` branch. See
[.github/DEVELOPING_V3.md](.github/DEVELOPING_V3.md).
## Agent Skills and Claude Code Plugin