n8n-io/n8n
1
0
Fork 0
mirror of https://github.com/n8n-io/n8n.git synced 2026-10-12 02:19:43 +00:00
Code Issues Packages Projects Releases Wiki Activity

Merge branch 'master' into fix-migration-rule-detail-member-search-test

Browse Source
This commit is contained in:
Mutasem Aldmour
2026-10-11 08:45:01 +02:00
committed by GitHub
parent 1e9d54dbd5 529b0d1c3b
commit 43aaee9114
185 changed files with 8477 additions and 811 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.agents/skills/create-instance-ai-eval/SKILL.md
+6
View File
@@ -659,6 +659,12 @@ nodes get LLM-generated pin data). So:
otherwise
accept the static-data red as a harness limit and carry the logic in
`outcomeExpectations`. Note the agent may *choose* static-data dedup on its own.
- **Message an Agent runs the called Agent with its real model.** The harness
mocks the Agent's tools, MCP servers and web search, and records those calls
under the calling node. It turns off the Agent's memory, vector stores and chat
integrations. Only OpenAI models are supported today, and only when
`EVAL_OPENAI_API_KEY` is set (CI sets it): put the Agent on an `openai/*` model
and declare an `openAiApi` credential.
- Don't assert exact counts that depend on mock generation ("exactly 7 posts").
Say "fewer than the original 10".
.agents/skills/public-api/SKILL.md
+1 -1
View File
@@ -109,7 +109,7 @@ model; reuse only what applies. Decorators, all from `@n8n/decorators`:
| `@Get/@Post/@Put/@Patch/@Delete('/path')` | Route method. |
| `@ApiKeyScope('res:action')` | API-key grant check. |
| `@ProjectScope/@GlobalScope('res:action')` | User RBAC check. |
| `@ApiResponse(status)` / `@ApiResponse(status, Dto)` | Success status + (optional) output DTO; registry `.parse()`s + strips the return value. Exactly one per route — a second `@ApiResponse` throws. `204` can't carry a DTO — throws. |
| `@ApiResponse(status)` / `@ApiResponse(status, Dto)` | Success status + (optional) output DTO; registry `.parse()`s + strips the return value. Exactly one per route — a second `@ApiResponse` throws. `204` can't carry a DTO or a binary body — throws. For a binary response body (returned as `{ body, headers? }`), use `@ApiResponse(status, { mediaType, description?, headers? })` — see [Binary response bodies](reference.md#binary-response-bodies). |
| `@ApiErrorResponse(status)` | Declares an additional documented non-2xx status (e.g. `404`, `409`). Stack multiple for more than one. `400`/`401`/`403` are added automatically (body/query present, always, and `@ApiKeyScope` present, respectively) — don't declare those yourself. |
| `@ApiSummary(text)` / `@ApiDescription(text)` / `@ApiTags([...])` | OpenAPI summary/description/tags. `@ApiTags` sorts alphabetically regardless of the order you pass. All optional but expected on every real route. |
| `@Query` / `@Body` / `@Param('name')` | Bind + validate via a `Z.class` DTO / path param. `@Body` is JSON by default; `@Body({ mediaType: 'multipart/form-data', uploadLimits })` takes a `multipart/form-data` body instead — see [Request body media types](reference.md#request-body-media-types). |
.agents/skills/public-api/reference.md
+29
View File
@@ -146,6 +146,35 @@ handler per media type (`REQUEST_BODY_HANDLERS` in
`REQUEST_BODY_HANDLERS` — the registry, resolver, generator and `/discover`
need no change, since they all read the handler, not the media type.
## Binary response bodies
`@ApiResponse(status, { mediaType, description?, headers? })` declares a
success body that the controller method returns as `{ body, headers? }`, for
example an `application/gzip` stream. The framework treats options whose
`mediaType` is in `BINARY_RESPONSE_MEDIA_TYPES` as a binary response.
`mediaType` accepts only those types.
- `body` is a `Buffer` or a `Readable`. The method returns the result and never
writes to `res`.
- `headers` holds the response headers. Every header declared in `headers` must
be in the result. A missing header fails the request with a `500`. Matching is
case-insensitive.
- The registry checks the headers, then sets the declared status, the headers,
and `Content-Type: <mediaType>`. A header in the result cannot replace the
`Content-Type`.
- A stream's first chunk is read before any header is set. So a stream that
fails before its first chunk gives the normal JSON error response, with no
binary `Content-Type`. A method that throws before it returns does the same.
- After the first chunk is sent, an error ends the response. A client that
disconnects is not an error, and the framework destroys the stream.
- A method that returns no binary body fails with a `500`.
- The generator documents the body as `{ type: string, format: binary }`
under the `mediaType` content key, with the description and headers.
The description defaults to `Operation successful.`.
The runtime part is `sendBinaryResponse` in
`packages/cli/src/public-api/media-types/binary-response.ts`.
## Migrating legacy EOV endpoints
Legacy `express-openapi-validator` endpoints live under
.github/test-metrics/e2e-impact-map.json
+1 -1
View File
File diff suppressed because one or more lines are too long
docs/generated/postgres-schema/README.md
Generated
+13
View File
@@ -75,6 +75,7 @@ Auto-generated from the PostgreSQL migrations in @n8n/db. Do not edit by hand.
| [public.execution_metadata](public.execution_metadata.md) | 4 | | BASE TABLE |
| [public.folder](public.folder.md) | 6 | | BASE TABLE |
| [public.folder_tag](public.folder_tag.md) | 2 | | BASE TABLE |
| [public.idempotency_key](public.idempotency_key.md) | 9 | | BASE TABLE |
| [public.insights_by_period](public.insights_by_period.md) | 6 | | BASE TABLE |
| [public.insights_metadata](public.insights_metadata.md) | 5 | | BASE TABLE |
| [public.insights_raw](public.insights_raw.md) | 5 | | BASE TABLE |
@@ -300,6 +301,7 @@ erDiagram
"public.folder" }o--o| "public.folder" : "FOREIGN KEY (#quot;parentFolderId#quot;) REFERENCES folder(id) ON DELETE CASCADE"
"public.folder_tag" }o--|| "public.tag_entity" : "FOREIGN KEY (#quot;tagId#quot;) REFERENCES tag_entity(id) ON DELETE CASCADE"
"public.folder_tag" }o--|| "public.folder" : "FOREIGN KEY (#quot;folderId#quot;) REFERENCES folder(id) ON DELETE CASCADE"
"public.idempotency_key" }o--|| "public.user" : "FOREIGN KEY (#quot;userId#quot;) REFERENCES #quot;user#quot;(id) ON DELETE CASCADE"
"public.insights_by_period" }o--|| "public.insights_metadata" : "FOREIGN KEY (#quot;metaId#quot;) REFERENCES insights_metadata(#quot;metaId#quot;) ON DELETE CASCADE"
"public.insights_metadata" }o--o| "public.workflow_entity" : "FOREIGN KEY (#quot;workflowId#quot;) REFERENCES workflow_entity(id) ON DELETE SET NULL"
"public.insights_metadata" }o--o| "public.project" : "FOREIGN KEY (#quot;projectId#quot;) REFERENCES project(id) ON DELETE SET NULL"
@@ -1123,6 +1125,17 @@ erDiagram
varchar_36_ folderId FK
varchar_36_ tagId FK
}
"public.idempotency_key" {
timestamp_3__with_time_zone createdAt
text fingerprint
varchar_16_ id
varchar_128_ idempotencyKey
json responseBody
smallint responseStatus
varchar_16_ status
timestamp_3__with_time_zone updatedAt
uuid userId FK
}
"public.insights_by_period" {
integer id
integer metaId FK
docs/generated/postgres-schema/public.idempotency_key.md
Generated
+80
View File
@@ -0,0 +1,80 @@
# public.idempotency_key
## Columns
| Name | Type | Default | Nullable | Children | Parents | Comment |
| ---- | ---- | ------- | -------- | -------- | ------- | ------- |
| createdAt | timestamp(3) with time zone | CURRENT_TIMESTAMP(3) | false | | | |
| fingerprint | text | | false | | | Hash of the request method, path, and body |
| id | varchar(16) | | false | | | Application-generated n8n nano ID |
| idempotencyKey | varchar(128) | | false | | | Client Idempotency-Key header. Opaque ASCII, length 1 to 128 |
| responseBody | json | | true | | | Stored response body. NULL while status is processing |
| responseStatus | smallint | | true | | | HTTP status of the stored response. NULL while processing, required once completed |
| status | varchar(16) | 'processing'::character varying | false | | | processing while the handler runs. completed after the response is stored |
| updatedAt | timestamp(3) with time zone | CURRENT_TIMESTAMP(3) | false | | | |
| userId | uuid | | false | | [public.user](public.user.md) | |
## Constraints
| Name | Type | Definition |
| ---- | ---- | ---------- |
| CHK_idempotency_key_responseStatus | CHECK | CHECK (((((status)::text = 'processing'::text) AND ("responseStatus" IS NULL)) OR (((status)::text = 'completed'::text) AND ("responseStatus" IS NOT NULL)))) |
| CHK_idempotency_key_status | CHECK | CHECK (((status)::text = ANY ((ARRAY['processing'::character varying, 'completed'::character varying])::text[]))) |
| FK_idempotency_key_userId | FOREIGN KEY | FOREIGN KEY ("userId") REFERENCES "user"(id) ON DELETE CASCADE |
| PK_213f125e14469be304f9ff1d452 | PRIMARY KEY | PRIMARY KEY (id) |
| idempotency_key_createdAt_not_null | n | NOT NULL "createdAt" |
| idempotency_key_fingerprint_not_null | n | NOT NULL fingerprint |
| idempotency_key_id_not_null | n | NOT NULL id |
| idempotency_key_idempotencyKey_not_null | n | NOT NULL "idempotencyKey" |
| idempotency_key_status_not_null | n | NOT NULL status |
| idempotency_key_updatedAt_not_null | n | NOT NULL "updatedAt" |
| idempotency_key_userId_not_null | n | NOT NULL "userId" |
## Indexes
| Name | Definition |
| ---- | ---------- |
| IDX_25289244569e505292cde510d7 | CREATE UNIQUE INDEX "IDX_25289244569e505292cde510d7" ON public.idempotency_key USING btree ("userId", "idempotencyKey") |
| IDX_38536b1937c55d305af4d69a97 | CREATE INDEX "IDX_38536b1937c55d305af4d69a97" ON public.idempotency_key USING btree ("createdAt") |
| PK_213f125e14469be304f9ff1d452 | CREATE UNIQUE INDEX "PK_213f125e14469be304f9ff1d452" ON public.idempotency_key USING btree (id) |
## Relations
```mermaid
erDiagram
"public.idempotency_key" }o--|| "public.user" : "FOREIGN KEY (#quot;userId#quot;) REFERENCES #quot;user#quot;(id) ON DELETE CASCADE"
"public.idempotency_key" {
timestamp_3__with_time_zone createdAt
text fingerprint
varchar_16_ id
varchar_128_ idempotencyKey
json responseBody
smallint responseStatus
varchar_16_ status
timestamp_3__with_time_zone updatedAt
uuid userId FK
}
"public.user" {
timestamp_3__with_time_zone createdAt
boolean disabled
varchar_255_ email
varchar_32_ firstName
uuid id
date lastActiveAt
varchar_32_ lastName
boolean mfaEnabled
text mfaRecoveryCodes
text mfaSecret
varchar_255_ password
json personalizationAnswers
varchar_128_ roleSlug FK
json settings
timestamp_3__with_time_zone updatedAt
}
```
---
> Generated by [tbls](https://github.com/k1LoW/tbls)
docs/generated/postgres-schema/public.user.md
Generated
+13 -1
View File
@@ -8,7 +8,7 @@
| disabled | boolean | false | false | | | |
| email | varchar(255) | | true | | | |
| firstName | varchar(32) | | true | | | |
| id | uuid | gen_random_uuid() | false | [public.activity_event](public.activity_event.md) [public.agent_eval_dataset](public.agent_eval_dataset.md) [public.agent_eval_rating](public.agent_eval_rating.md) [public.agent_eval_run](public.agent_eval_run.md) [public.agent_execution_threads](public.agent_execution_threads.md) [public.agent_history](public.agent_history.md) [public.ai_preference](public.ai_preference.md) [public.auth_identity](public.auth_identity.md) [public.chat_hub_agents](public.chat_hub_agents.md) [public.chat_hub_sessions](public.chat_hub_sessions.md) [public.chat_hub_tools](public.chat_hub_tools.md) [public.dynamic_credential_user_entry](public.dynamic_credential_user_entry.md) [public.evaluation_collection](public.evaluation_collection.md) [public.instance_ai_mcp_registry_connections](public.instance_ai_mcp_registry_connections.md) [public.instance_ai_pending_confirmations](public.instance_ai_pending_confirmations.md) [public.instance_ai_thread_grants](public.instance_ai_thread_grants.md) [public.instance_ai_thread_tabs](public.instance_ai_thread_tabs.md) [public.migration_workflow_owner](public.migration_workflow_owner.md) [public.oauth_access_tokens](public.oauth_access_tokens.md) [public.oauth_authorization_codes](public.oauth_authorization_codes.md) [public.oauth_refresh_tokens](public.oauth_refresh_tokens.md) [public.oauth_user_consents](public.oauth_user_consents.md) [public.project](public.project.md) [public.project_relation](public.project_relation.md) [public.self_healing_result](public.self_healing_result.md) [public.trusted_source_identity](public.trusted_source_identity.md) [public.user_api_keys](public.user_api_keys.md) [public.user_favorites](public.user_favorites.md) [public.workflow_builder_session](public.workflow_builder_session.md) [public.workflow_publish_history](public.workflow_publish_history.md) [public.workflow_review_activity](public.workflow_review_activity.md) [public.workflow_review_activity_comment](public.workflow_review_activity_comment.md) [public.workflow_review_request](public.workflow_review_request.md) [public.workflow_review_request_authors](public.workflow_review_request_authors.md) [public.workflow_review_request_reviewers](public.workflow_review_request_reviewers.md) [public.workflow_suggestion](public.workflow_suggestion.md) [public.workflow_suggestion_activity](public.workflow_suggestion_activity.md) | | |
| id | uuid | gen_random_uuid() | false | [public.activity_event](public.activity_event.md) [public.agent_eval_dataset](public.agent_eval_dataset.md) [public.agent_eval_rating](public.agent_eval_rating.md) [public.agent_eval_run](public.agent_eval_run.md) [public.agent_execution_threads](public.agent_execution_threads.md) [public.agent_history](public.agent_history.md) [public.ai_preference](public.ai_preference.md) [public.auth_identity](public.auth_identity.md) [public.chat_hub_agents](public.chat_hub_agents.md) [public.chat_hub_sessions](public.chat_hub_sessions.md) [public.chat_hub_tools](public.chat_hub_tools.md) [public.dynamic_credential_user_entry](public.dynamic_credential_user_entry.md) [public.evaluation_collection](public.evaluation_collection.md) [public.idempotency_key](public.idempotency_key.md) [public.instance_ai_mcp_registry_connections](public.instance_ai_mcp_registry_connections.md) [public.instance_ai_pending_confirmations](public.instance_ai_pending_confirmations.md) [public.instance_ai_thread_grants](public.instance_ai_thread_grants.md) [public.instance_ai_thread_tabs](public.instance_ai_thread_tabs.md) [public.migration_workflow_owner](public.migration_workflow_owner.md) [public.oauth_access_tokens](public.oauth_access_tokens.md) [public.oauth_authorization_codes](public.oauth_authorization_codes.md) [public.oauth_refresh_tokens](public.oauth_refresh_tokens.md) [public.oauth_user_consents](public.oauth_user_consents.md) [public.project](public.project.md) [public.project_relation](public.project_relation.md) [public.self_healing_result](public.self_healing_result.md) [public.trusted_source_identity](public.trusted_source_identity.md) [public.user_api_keys](public.user_api_keys.md) [public.user_favorites](public.user_favorites.md) [public.workflow_builder_session](public.workflow_builder_session.md) [public.workflow_publish_history](public.workflow_publish_history.md) [public.workflow_review_activity](public.workflow_review_activity.md) [public.workflow_review_activity_comment](public.workflow_review_activity_comment.md) [public.workflow_review_request](public.workflow_review_request.md) [public.workflow_review_request_authors](public.workflow_review_request_authors.md) [public.workflow_review_request_reviewers](public.workflow_review_request_reviewers.md) [public.workflow_suggestion](public.workflow_suggestion.md) [public.workflow_suggestion_activity](public.workflow_suggestion_activity.md) | | |
| lastActiveAt | date | | true | | | |
| lastName | varchar(32) | | true | | | |
| mfaEnabled | boolean | false | false | | | |
@@ -61,6 +61,7 @@ erDiagram
"public.chat_hub_tools" }o--|| "public.user" : "FOREIGN KEY (#quot;ownerId#quot;) REFERENCES #quot;user#quot;(id) ON DELETE CASCADE"
"public.dynamic_credential_user_entry" }o--|| "public.user" : "FOREIGN KEY (#quot;userId#quot;) REFERENCES #quot;user#quot;(id) ON DELETE CASCADE"
"public.evaluation_collection" }o--o| "public.user" : "FOREIGN KEY (#quot;createdById#quot;) REFERENCES #quot;user#quot;(id) ON DELETE SET NULL"
"public.idempotency_key" }o--|| "public.user" : "FOREIGN KEY (#quot;userId#quot;) REFERENCES #quot;user#quot;(id) ON DELETE CASCADE"
"public.instance_ai_mcp_registry_connections" }o--|| "public.user" : "FOREIGN KEY (#quot;userId#quot;) REFERENCES #quot;user#quot;(id) ON DELETE CASCADE"
"public.instance_ai_pending_confirmations" }o--|| "public.user" : "FOREIGN KEY (#quot;userId#quot;) REFERENCES #quot;user#quot;(id) ON DELETE CASCADE"
"public.instance_ai_thread_grants" }o--|| "public.user" : "FOREIGN KEY (#quot;userId#quot;) REFERENCES #quot;user#quot;(id) ON DELETE CASCADE"
@@ -268,6 +269,17 @@ erDiagram
timestamp_3__with_time_zone updatedAt
varchar_36_ workflowId FK
}
"public.idempotency_key" {
timestamp_3__with_time_zone createdAt
text fingerprint
varchar_16_ id
varchar_128_ idempotencyKey
json responseBody
smallint responseStatus
varchar_16_ status
timestamp_3__with_time_zone updatedAt
uuid userId FK
}
"public.instance_ai_mcp_registry_connections" {
timestamp_3__with_time_zone createdAt
varchar_36_ credentialId FK
docs/generated/sqlite-schema/README.md
Generated
+13
View File
@@ -75,6 +75,7 @@ Auto-generated from the SQLite migrations in @n8n/db. Do not edit by hand.
| [execution_metadata](execution_metadata.md) | 4 | | table |
| [folder](folder.md) | 6 | | table |
| [folder_tag](folder_tag.md) | 2 | | table |
| [idempotency_key](idempotency_key.md) | 9 | | table |
| [insights_by_period](insights_by_period.md) | 6 | | table |
| [insights_metadata](insights_metadata.md) | 5 | | table |
| [insights_raw](insights_raw.md) | 5 | | table |
@@ -282,6 +283,7 @@ erDiagram
"folder" }o--|| "project" : "FOREIGN KEY (projectId) REFERENCES project (id) ON UPDATE NO ACTION ON DELETE CASCADE MATCH NONE"
"folder_tag" |o--|| "tag_entity" : "FOREIGN KEY (tagId) REFERENCES tag_entity (id) ON UPDATE NO ACTION ON DELETE CASCADE MATCH NONE"
"folder_tag" |o--|| "folder" : "FOREIGN KEY (folderId) REFERENCES folder (id) ON UPDATE NO ACTION ON DELETE CASCADE MATCH NONE"
"idempotency_key" }o--|| "user" : "FOREIGN KEY (userId) REFERENCES user (id) ON UPDATE NO ACTION ON DELETE CASCADE MATCH NONE"
"insights_by_period" }o--|| "insights_metadata" : "FOREIGN KEY (metaId) REFERENCES insights_metadata (metaId) ON UPDATE NO ACTION ON DELETE CASCADE MATCH NONE"
"insights_metadata" }o--o| "project" : "FOREIGN KEY (projectId) REFERENCES project (id) ON UPDATE NO ACTION ON DELETE SET NULL MATCH NONE"
"insights_metadata" }o--o| "workflow_entity" : "FOREIGN KEY (workflowId) REFERENCES workflow_entity (id) ON UPDATE NO ACTION ON DELETE SET NULL MATCH NONE"
@@ -1109,6 +1111,17 @@ erDiagram
varchar_36_ folderId PK
varchar_36_ tagId PK
}
"idempotency_key" {
datetime_3_ createdAt
TEXT fingerprint
varchar_16_ id PK
varchar_128_ idempotencyKey
TEXT responseBody
smallint responseStatus
varchar_16_ status
datetime_3_ updatedAt
varchar userId FK
}
"insights_by_period" {
INTEGER id
INTEGER metaId FK
docs/generated/sqlite-schema/idempotency_key.md
Generated
+85
View File
@@ -0,0 +1,85 @@
# idempotency_key
## Description
<details>
<summary><strong>Table Definition</strong></summary>
```sql
CREATE TABLE "idempotency_key" ("id" varchar(16) PRIMARY KEY NOT NULL, "userId" varchar NOT NULL, "idempotencyKey" varchar(128) NOT NULL, "fingerprint" text NOT NULL, "status" varchar(16) NOT NULL DEFAULT ('processing'), "responseStatus" smallint, "responseBody" text, "createdAt" datetime(3) NOT NULL DEFAULT (STRFTIME('%Y-%m-%d %H:%M:%f', 'NOW')), "updatedAt" datetime(3) NOT NULL DEFAULT (STRFTIME('%Y-%m-%d %H:%M:%f', 'NOW')), CONSTRAINT "CHK_idempotency_key_responseStatus" CHECK (("status" = 'processing' AND "responseStatus" IS NULL) OR ("status" = 'completed' AND "responseStatus" IS NOT NULL)), CONSTRAINT "CHK_idempotency_key_status" CHECK ("status" IN ('processing', 'completed')), CONSTRAINT "FK_idempotency_key_userId" FOREIGN KEY ("userId") REFERENCES "user" ("id") ON DELETE CASCADE)
```
</details>
## Columns
| Name | Type | Default | Nullable | Children | Parents | Comment |
| ---- | ---- | ------- | -------- | -------- | ------- | ------- |
| createdAt | datetime(3) | STRFTIME('%Y-%m-%d %H:%M:%f', 'NOW') | false | | | |
| fingerprint | TEXT | | false | | | |
| id | varchar(16) | | false | | | |
| idempotencyKey | varchar(128) | | false | | | |
| responseBody | TEXT | | true | | | |
| responseStatus | smallint | | true | | | |
| status | varchar(16) | 'processing' | false | | | |
| updatedAt | datetime(3) | STRFTIME('%Y-%m-%d %H:%M:%f', 'NOW') | false | | | |
| userId | varchar | | false | | [user](user.md) | |
## Constraints
| Name | Type | Definition |
| ---- | ---- | ---------- |
| - | CHECK | CHECK (("status" = 'processing' AND "responseStatus" IS NULL) OR ("status" = 'completed' AND "responseStatus" IS NOT NULL)) |
| - | CHECK | CHECK ("status" IN ('processing', 'completed')) |
| - (Foreign key ID: 0) | FOREIGN KEY | FOREIGN KEY (userId) REFERENCES user (id) ON UPDATE NO ACTION ON DELETE CASCADE MATCH NONE |
| id | PRIMARY KEY | PRIMARY KEY (id) |
| sqlite_autoindex_idempotency_key_1 | PRIMARY KEY | PRIMARY KEY (id) |
## Indexes
| Name | Definition |
| ---- | ---------- |
| IDX_25289244569e505292cde510d7 | CREATE UNIQUE INDEX "IDX_25289244569e505292cde510d7" ON "idempotency_key" ("userId", "idempotencyKey") |
| IDX_38536b1937c55d305af4d69a97 | CREATE INDEX "IDX_38536b1937c55d305af4d69a97" ON "idempotency_key" ("createdAt") |
| sqlite_autoindex_idempotency_key_1 | PRIMARY KEY (id) |
## Relations
```mermaid
erDiagram
"idempotency_key" }o--|| "user" : "FOREIGN KEY (userId) REFERENCES user (id) ON UPDATE NO ACTION ON DELETE CASCADE MATCH NONE"
"idempotency_key" {
datetime_3_ createdAt
TEXT fingerprint
varchar_16_ id PK
varchar_128_ idempotencyKey
TEXT responseBody
smallint responseStatus
varchar_16_ status
datetime_3_ updatedAt
varchar userId FK
}
"user" {
datetime_3_ createdAt
boolean disabled
varchar_255_ email
varchar_32_ firstName
varchar id PK
date lastActiveAt
varchar_32_ lastName
boolean mfaEnabled
TEXT mfaRecoveryCodes
TEXT mfaSecret
varchar password
TEXT personalizationAnswers
varchar_128_ roleSlug FK
TEXT settings
datetime_3_ updatedAt
}
```
---
> Generated by [tbls](https://github.com/k1LoW/tbls)
docs/generated/sqlite-schema/user.md
Generated
+13 -1
View File
@@ -19,7 +19,7 @@ CREATE TABLE "user" ("id" varchar PRIMARY KEY, "email" varchar(255), "firstName"
| disabled | boolean | FALSE | false | | | |
| email | varchar(255) | | true | | | |
| firstName | varchar(32) | | true | | | |
| id | varchar | | true | [activity_event](activity_event.md) [agent_eval_dataset](agent_eval_dataset.md) [agent_eval_rating](agent_eval_rating.md) [agent_eval_run](agent_eval_run.md) [agent_execution_threads](agent_execution_threads.md) [agent_history](agent_history.md) [ai_preference](ai_preference.md) [auth_identity](auth_identity.md) [chat_hub_agents](chat_hub_agents.md) [chat_hub_sessions](chat_hub_sessions.md) [chat_hub_tools](chat_hub_tools.md) [dynamic_credential_user_entry](dynamic_credential_user_entry.md) [evaluation_collection](evaluation_collection.md) [instance_ai_mcp_registry_connections](instance_ai_mcp_registry_connections.md) [instance_ai_pending_confirmations](instance_ai_pending_confirmations.md) [instance_ai_thread_grants](instance_ai_thread_grants.md) [instance_ai_thread_tabs](instance_ai_thread_tabs.md) [migration_workflow_owner](migration_workflow_owner.md) [oauth_access_tokens](oauth_access_tokens.md) [oauth_authorization_codes](oauth_authorization_codes.md) [oauth_refresh_tokens](oauth_refresh_tokens.md) [oauth_user_consents](oauth_user_consents.md) [project](project.md) [project_relation](project_relation.md) [self_healing_result](self_healing_result.md) [trusted_source_identity](trusted_source_identity.md) [user_api_keys](user_api_keys.md) [user_favorites](user_favorites.md) [workflow_builder_session](workflow_builder_session.md) [workflow_publish_history](workflow_publish_history.md) [workflow_review_activity](workflow_review_activity.md) [workflow_review_activity_comment](workflow_review_activity_comment.md) [workflow_review_request](workflow_review_request.md) [workflow_review_request_authors](workflow_review_request_authors.md) [workflow_review_request_reviewers](workflow_review_request_reviewers.md) [workflow_suggestion](workflow_suggestion.md) [workflow_suggestion_activity](workflow_suggestion_activity.md) | | |
| id | varchar | | true | [activity_event](activity_event.md) [agent_eval_dataset](agent_eval_dataset.md) [agent_eval_rating](agent_eval_rating.md) [agent_eval_run](agent_eval_run.md) [agent_execution_threads](agent_execution_threads.md) [agent_history](agent_history.md) [ai_preference](ai_preference.md) [auth_identity](auth_identity.md) [chat_hub_agents](chat_hub_agents.md) [chat_hub_sessions](chat_hub_sessions.md) [chat_hub_tools](chat_hub_tools.md) [dynamic_credential_user_entry](dynamic_credential_user_entry.md) [evaluation_collection](evaluation_collection.md) [idempotency_key](idempotency_key.md) [instance_ai_mcp_registry_connections](instance_ai_mcp_registry_connections.md) [instance_ai_pending_confirmations](instance_ai_pending_confirmations.md) [instance_ai_thread_grants](instance_ai_thread_grants.md) [instance_ai_thread_tabs](instance_ai_thread_tabs.md) [migration_workflow_owner](migration_workflow_owner.md) [oauth_access_tokens](oauth_access_tokens.md) [oauth_authorization_codes](oauth_authorization_codes.md) [oauth_refresh_tokens](oauth_refresh_tokens.md) [oauth_user_consents](oauth_user_consents.md) [project](project.md) [project_relation](project_relation.md) [self_healing_result](self_healing_result.md) [trusted_source_identity](trusted_source_identity.md) [user_api_keys](user_api_keys.md) [user_favorites](user_favorites.md) [workflow_builder_session](workflow_builder_session.md) [workflow_publish_history](workflow_publish_history.md) [workflow_review_activity](workflow_review_activity.md) [workflow_review_activity_comment](workflow_review_activity_comment.md) [workflow_review_request](workflow_review_request.md) [workflow_review_request_authors](workflow_review_request_authors.md) [workflow_review_request_reviewers](workflow_review_request_reviewers.md) [workflow_suggestion](workflow_suggestion.md) [workflow_suggestion_activity](workflow_suggestion_activity.md) | | |
| lastActiveAt | date | | true | | | |
| lastName | varchar(32) | | true | | | |
| mfaEnabled | boolean | FALSE | false | | | |
@@ -67,6 +67,7 @@ erDiagram
"chat_hub_tools" }o--|| "user" : "FOREIGN KEY (ownerId) REFERENCES user (id) ON UPDATE NO ACTION ON DELETE CASCADE MATCH NONE"
"dynamic_credential_user_entry" |o--|| "user" : "FOREIGN KEY (userId) REFERENCES user (id) ON UPDATE NO ACTION ON DELETE CASCADE MATCH NONE"
"evaluation_collection" }o--o| "user" : "FOREIGN KEY (createdById) REFERENCES user (id) ON UPDATE NO ACTION ON DELETE SET NULL MATCH NONE"
"idempotency_key" }o--|| "user" : "FOREIGN KEY (userId) REFERENCES user (id) ON UPDATE NO ACTION ON DELETE CASCADE MATCH NONE"
"instance_ai_mcp_registry_connections" }o--|| "user" : "FOREIGN KEY (userId) REFERENCES user (id) ON UPDATE NO ACTION ON DELETE CASCADE MATCH NONE"
"instance_ai_pending_confirmations" }o--|| "user" : "FOREIGN KEY (userId) REFERENCES user (id) ON UPDATE NO ACTION ON DELETE CASCADE MATCH NONE"
"instance_ai_thread_grants" |o--|| "user" : "FOREIGN KEY (userId) REFERENCES user (id) ON UPDATE NO ACTION ON DELETE CASCADE MATCH NONE"
@@ -274,6 +275,17 @@ erDiagram
datetime_3_ updatedAt
varchar_36_ workflowId FK
}
"idempotency_key" {
datetime_3_ createdAt
TEXT fingerprint
varchar_16_ id PK
varchar_128_ idempotencyKey
TEXT responseBody
smallint responseStatus
varchar_16_ status
datetime_3_ updatedAt
varchar userId FK
}
"instance_ai_mcp_registry_connections" {
datetime_3_ createdAt
varchar_36_ credentialId FK
packages/@n8n/agents/src/index.ts
+1 -1
View File
@@ -119,7 +119,7 @@ export type {
export { FINISH_REASONS, isFinishReason } from './types';
export type { ProviderOptions } from '@ai-sdk/provider-utils';
export { AgentEvent } from './types';
export type { AgentEventData, AgentEventHandler } from './types';
export type { AgentEventData, AgentEventHandler, ForwardedChildChunk } from './types';
export {
OBSERVATION_LOG_MARKERS,
OBSERVATION_LOG_STATUSES,
packages/@n8n/api-types/src/agents/__tests__/agent-chat-message.dto.test.ts
+24
View File
@@ -62,6 +62,30 @@ describe('AgentChatMessageDto', () => {
expect(result.success).toBe(false);
});
it('accepts an object client context and keeps it in the parsed payload', () => {
const clientContext = { timeZone: 'Europe/Helsinki', nested: { ids: ['a', 'b'] }, flag: null };
const payload = { message: 'hello', clientContext };
expect(AgentChatMessageDto.parse(payload)).toEqual(payload);
expect(AgentChatMessageDto.safeParse({ message: 'hello', clientContext: {} }).success).toBe(
true,
);
});
it('accepts a payload without a client context', () => {
const parsed = AgentChatMessageDto.parse({ message: 'hello' });
expect(parsed).not.toHaveProperty('clientContext');
});
it.each([
['null', null],
['an array', [{ timeZone: 'UTC' }]],
['a string', 'UTC'],
['a number', 1],
['a boolean', true],
])('rejects a client context that is %s', (_label, clientContext) => {
expect(AgentChatMessageDto.safeParse({ message: 'hello', clientContext }).success).toBe(false);
});
it('rejects attachments with empty fileName or mimeType', () => {
expect(
AgentChatMessageDto.safeParse({
packages/@n8n/api-types/src/agents/dto.ts
+6
View File
@@ -258,6 +258,12 @@ const agentChatMessageShape = {
.array(agentChatAttachmentSchema)
.max(MAX_AGENT_CHAT_ATTACHMENTS_PER_MESSAGE)
.optional(),
/**
* Per-message context from the client (for example the user's time zone).
* The Agents layer does not read it. A system agent's provider defines its
* shape and validates it. Project agents ignore it.
*/
clientContext: z.record(z.unknown()).optional(),
};
const agentChatMessageSchema = z
packages/@n8n/api-types/src/dto/credentials/credential-options-request.dto.ts
+26
View File
@@ -0,0 +1,26 @@
import type { ICredentialDataDecryptedObject } from 'n8n-workflow';
import { z } from 'zod';
import { jsonValueSchema } from '../../schemas/json-value.schema';
import { Z } from '../../zod-class';
const jsonObjectSchema = z.object({}).catchall(jsonValueSchema);
const credentialDataSchema: z.ZodType<ICredentialDataDecryptedObject> = z.record(
z.string(),
z.union([
z.string(),
z.number(),
z.boolean(),
z.array(z.string()),
jsonObjectSchema,
z.array(jsonObjectSchema),
]),
);
export class CredentialOptionsRequestDto extends Z.class({
type: z.string().min(1),
data: credentialDataSchema,
propertyName: z.string().min(1),
filter: z.string().optional(),
paginationToken: z.string().optional(),
}) {}
packages/@n8n/api-types/src/dto/index.ts
+32 -8
View File
@@ -168,6 +168,7 @@ export { SourceControlPullResponsePublicDto } from './source-control/source-cont
export { CreateCredentialDto } from './credentials/create-credential.dto';
export { TestCredentialRequestDto } from './credentials/test-credential-request.dto';
export { CredentialOptionsRequestDto } from './credentials/credential-options-request.dto';
export {
CredentialPublicDto,
CredentialListPublicDto,
@@ -353,26 +354,49 @@ export {
export { PaginationDto, MAX_ITEMS_PER_PAGE } from './pagination/pagination.dto';
export type {
NodeTypePolicyRule,
NodeTypePolicySelector,
NonDelegatingPolicyAction,
NonDelegatingPolicyRule,
PolicyAction,
PolicyRule,
PolicySelector,
} from './type-availability-policies/policy-rule.types';
export { policySelectorSchema } from './type-availability-policies/policy-selector.schema';
export { isNodeTypePolicyRule } from './type-availability-policies/policy-rule.types';
export {
credentialTypePolicySelectorSchema,
nodeTypePolicySelectorSchema,
} from './type-availability-policies/policy-selector.schema';
export {
credentialTypePolicyRuleSchemas,
nodeTypePolicyRuleSchemas,
nonDelegatingPolicyActionSchema,
nonDelegatingPolicyRuleListSchema,
policyActionSchema,
policyRuleListSchema,
policyRuleSchema,
} from './type-availability-policies/policy-rule.schema';
export { PutInstancePolicyDto } from './type-availability-policies/put-instance-policy.dto';
export { PutProjectPolicyDto } from './type-availability-policies/put-project-policy.dto';
export { CreatePolicyDocumentDto } from './type-availability-policies/create-policy-document.dto';
export { UpdatePolicyDocumentDto } from './type-availability-policies/update-policy-document.dto';
export {
PutCredentialTypeInstancePolicyDto,
PutInstancePolicyDto,
} from './type-availability-policies/put-instance-policy.dto';
export {
PutCredentialTypeProjectPolicyDto,
PutProjectPolicyDto,
} from './type-availability-policies/put-project-policy.dto';
export {
CreateCredentialTypePolicyDocumentDto,
CreatePolicyDocumentDto,
} from './type-availability-policies/create-policy-document.dto';
export {
UpdateCredentialTypePolicyDocumentDto,
UpdatePolicyDocumentDto,
} from './type-availability-policies/update-policy-document.dto';
export { ReplaceAttachmentsDto } from './type-availability-policies/replace-attachments.dto';
export {
CredentialTypePolicyAttachmentsPublicDto,
CredentialTypePolicyDocumentListPublicDto,
CredentialTypePolicyDocumentPublicDto,
CredentialTypePolicyDocumentWriteResultPublicDto,
CredentialTypePolicyEffectivePublicDto,
CredentialTypePolicyEffectiveWriteResultPublicDto,
ListNodeTypePolicyDocumentsQueryDto,
PolicyAttachmentsPublicDto,
PolicyDocumentListPublicDto,
packages/@n8n/api-types/src/dto/type-availability-policies/__tests__/policy-rule.schema.test.ts
+36 -3
View File
@@ -1,11 +1,16 @@
import {
credentialTypePolicyRuleSchemas,
nodeTypePolicyRuleSchemas,
nonDelegatingPolicyActionSchema,
nonDelegatingPolicyRuleListSchema,
policyActionSchema,
policyRuleListSchema,
policyRuleSchema,
} from '../policy-rule.schema';
const {
rule: policyRuleSchema,
ruleList: policyRuleListSchema,
nonDelegatingRuleList: nonDelegatingPolicyRuleListSchema,
} = nodeTypePolicyRuleSchemas;
const nameSelector = { kind: 'name' as const, value: 'n8n-nodes-base.slack' };
describe('policyActionSchema', () => {
@@ -105,3 +110,31 @@ describe('nonDelegatingPolicyRuleListSchema', () => {
}
});
});
describe('rule schemas per kind', () => {
const extendsRule = {
id: 'r1',
action: 'deny',
selector: { kind: 'extends', value: 'oAuth2Api' },
};
it('rejects an extends rule in a node type policy', () => {
expect(nodeTypePolicyRuleSchemas.ruleList.safeParse([extendsRule]).success).toBe(false);
expect(nodeTypePolicyRuleSchemas.nonDelegatingRuleList.safeParse([extendsRule]).success).toBe(
false,
);
});
it('accepts an extends rule in a credential type policy', () => {
expect(credentialTypePolicyRuleSchemas.ruleList.safeParse([extendsRule]).success).toBe(true);
expect(
credentialTypePolicyRuleSchemas.nonDelegatingRuleList.safeParse([extendsRule]).success,
).toBe(true);
});
it('rejects duplicate rule ids in a credential type policy', () => {
expect(
credentialTypePolicyRuleSchemas.ruleList.safeParse([extendsRule, extendsRule]).success,
).toBe(false);
});
});
packages/@n8n/api-types/src/dto/type-availability-policies/__tests__/policy-rule.types.test.ts
+19
View File
@@ -0,0 +1,19 @@
import { isNodeTypePolicyRule } from '../policy-rule.types';
describe('isNodeTypePolicyRule', () => {
it.each(['name', 'package'] as const)('accepts a %s rule', (kind) => {
expect(isNodeTypePolicyRule({ id: 'r1', action: 'deny', selector: { kind, value: 'x' } })).toBe(
true,
);
});
it('rejects an extends rule, which only credential type policies accept', () => {
expect(
isNodeTypePolicyRule({
id: 'r1',
action: 'deny',
selector: { kind: 'extends', value: 'oAuth2Api' },
}),
).toBe(false);
});
});
packages/@n8n/api-types/src/dto/type-availability-policies/__tests__/policy-selector.schema.test.ts
+25 -11
View File
@@ -1,27 +1,41 @@
import { policySelectorSchema } from '../policy-selector.schema';
import {
credentialTypePolicySelectorSchema,
nodeTypePolicySelectorSchema,
} from '../policy-selector.schema';
describe('policySelectorSchema', () => {
describe.each([
['nodeTypePolicySelectorSchema', nodeTypePolicySelectorSchema],
['credentialTypePolicySelectorSchema', credentialTypePolicySelectorSchema],
] as const)('%s', (_, schema) => {
it('accepts a valid name selector', () => {
expect(
policySelectorSchema.safeParse({ kind: 'name', value: 'n8n-nodes-base.slack' }).success,
).toBe(true);
expect(schema.safeParse({ kind: 'name', value: 'n8n-nodes-base.slack' }).success).toBe(true);
});
it('accepts a valid package selector', () => {
expect(
policySelectorSchema.safeParse({ kind: 'package', value: 'n8n-nodes-base' }).success,
).toBe(true);
expect(schema.safeParse({ kind: 'package', value: 'n8n-nodes-base' }).success).toBe(true);
});
it('rejects an empty value', () => {
expect(policySelectorSchema.safeParse({ kind: 'name', value: '' }).success).toBe(false);
expect(schema.safeParse({ kind: 'name', value: '' }).success).toBe(false);
});
it('rejects an unknown kind', () => {
expect(policySelectorSchema.safeParse({ kind: 'glob', value: '*' }).success).toBe(false);
expect(schema.safeParse({ kind: 'glob', value: '*' }).success).toBe(false);
});
it('rejects a selector missing its value', () => {
expect(policySelectorSchema.safeParse({ kind: 'name' }).success).toBe(false);
expect(schema.safeParse({ kind: 'name' }).success).toBe(false);
});
});
describe('extends selector', () => {
const selector = { kind: 'extends', value: 'oAuth2Api' };
it('is rejected by the node type schema', () => {
expect(nodeTypePolicySelectorSchema.safeParse(selector).success).toBe(false);
});
it('is accepted by the credential type schema', () => {
expect(credentialTypePolicySelectorSchema.safeParse(selector).success).toBe(true);
});
});
packages/@n8n/api-types/src/dto/type-availability-policies/create-policy-document.dto.ts
+8 -3
View File
@@ -1,7 +1,12 @@
import { policyRuleListSchema } from './policy-rule.schema';
import { credentialTypePolicyRuleSchemas, nodeTypePolicyRuleSchemas } from './policy-rule.schema';
import { Z } from '../../zod-class';
/** Request body for creating a policy document. */
/** Request body for creating a node type policy document. */
export class CreatePolicyDocumentDto extends Z.class({
rules: policyRuleListSchema,
rules: nodeTypePolicyRuleSchemas.ruleList,
}) {}
/** Request body for creating a credential type policy document. */
export class CreateCredentialTypePolicyDocumentDto extends Z.class({
rules: credentialTypePolicyRuleSchemas.ruleList,
}) {}
packages/@n8n/api-types/src/dto/type-availability-policies/node-type-policy-public.dto.ts
+79 -55
View File
@@ -1,14 +1,17 @@
import { z } from 'zod';
import { policyActionSchema, policyRuleSchema } from './policy-rule.schema';
import {
credentialTypePolicyRuleSchemas,
nodeTypePolicyRuleSchemas,
policyActionSchema,
} from './policy-rule.schema';
import { Z } from '../../zod-class';
import { publicApiPaginationSchema } from '../pagination/pagination.dto';
/**
* Response shapes for the public policy routes, shared by every policy `kind` (node types
* today, credential types later). Request bodies reuse the internal DTOs
* (`PutInstancePolicyDto`, `PutProjectPolicyDto`, …) so validation is identical on both
* surfaces; only the responses need a public allowlist.
* Response shapes for the public policy routes, one set per policy `kind` because each kind
* accepts different selectors. Request bodies reuse the internal DTOs (`PutInstancePolicyDto`,
* `PutCredentialTypeInstancePolicyDto`, …), so validation is identical on both surfaces.
*/
/** A rule that never matches because an earlier rule in the same list already covers it. */
@@ -17,65 +20,86 @@ const shadowWarningSchema = z.object({
shadowedByRuleId: z.string(),
});
/** Plain array on purpose: the response never re-runs the duplicate-id refinement. */
const policyRulesSchema = z.array(policyRuleSchema);
/** Plain rule arrays on purpose: a response never re-runs the duplicate-id refinement. */
function publicSchemasFor<Rule extends z.ZodTypeAny>(rule: Rule) {
const rules = z.array(rule);
const effectivePolicySchema = z.object({
// `null` until the scope is first written; the scope then reports `version: 0`.
scopeId: z.string().nullable(),
rules: policyRulesSchema,
defaultAction: policyActionSchema,
version: z.number().int().nonnegative(),
});
const effectivePolicy = z.object({
// `null` until the scope is first written; the scope then reports `version: 0`.
scopeId: z.string().nullable(),
rules,
defaultAction: policyActionSchema,
version: z.number().int().nonnegative(),
});
/** `GET` of a scope's composed policy. */
export class PolicyEffectivePublicDto extends Z.class(effectivePolicySchema.shape) {}
const policyDocument = z.object({
id: z.string(),
kind: z.string(),
rules,
version: z.number().int().positive(),
/** A user id, or the literal `environment` for env-bootstrap writes. */
updatedBy: z.string(),
createdAt: z.string().datetime(),
updatedAt: z.string().datetime(),
});
/** `PUT` of a scope's composed policy: the new state plus shadowing warnings. */
export class PolicyEffectiveWriteResultPublicDto extends Z.class({
...effectivePolicySchema.shape,
warnings: z.array(shadowWarningSchema),
}) {}
const attachment = z.object({
policyId: z.string(),
rules,
priority: z.number().int(),
isFloor: z.boolean(),
});
const policyDocumentSchema = z.object({
id: z.string(),
kind: z.string(),
rules: policyRulesSchema,
version: z.number().int().positive(),
/** A user id, or the literal `environment` for env-bootstrap writes. */
updatedBy: z.string(),
createdAt: z.string().datetime(),
updatedAt: z.string().datetime(),
});
return {
effective: effectivePolicy.shape,
effectiveWriteResult: { ...effectivePolicy.shape, warnings: z.array(shadowWarningSchema) },
document: policyDocument.shape,
documentWriteResult: { policy: policyDocument, warnings: z.array(shadowWarningSchema) },
documentList: { data: z.array(policyDocument), nextCursor: z.string().nullable() },
// The scope's version after a replace, which always bumps it.
attachments: { attachments: z.array(attachment), version: z.number().int().positive() },
};
}
/** One reusable policy document. */
export class PolicyDocumentPublicDto extends Z.class(policyDocumentSchema.shape) {}
const nodeType = publicSchemasFor(nodeTypePolicyRuleSchemas.rule);
const credentialType = publicSchemasFor(credentialTypePolicyRuleSchemas.rule);
/** Create or update of a policy document: the document plus shadowing warnings. */
export class PolicyDocumentWriteResultPublicDto extends Z.class({
policy: policyDocumentSchema,
warnings: z.array(shadowWarningSchema),
}) {}
/** `GET` of a scope's composed node type policy. */
export class PolicyEffectivePublicDto extends Z.class(nodeType.effective) {}
export class PolicyDocumentListPublicDto extends Z.class({
data: z.array(policyDocumentSchema),
nextCursor: z.string().nullable(),
}) {}
/** `PUT` of a scope's composed node type policy: the new state plus shadowing warnings. */
export class PolicyEffectiveWriteResultPublicDto extends Z.class(nodeType.effectiveWriteResult) {}
/** One reusable node type policy document. */
export class PolicyDocumentPublicDto extends Z.class(nodeType.document) {}
/** Create or update of a node type policy document, plus shadowing warnings. */
export class PolicyDocumentWriteResultPublicDto extends Z.class(nodeType.documentWriteResult) {}
export class PolicyDocumentListPublicDto extends Z.class(nodeType.documentList) {}
/** The attachments on one node type scope after a replace. */
export class PolicyAttachmentsPublicDto extends Z.class(nodeType.attachments) {}
export class CredentialTypePolicyEffectivePublicDto extends Z.class(credentialType.effective) {}
export class CredentialTypePolicyEffectiveWriteResultPublicDto extends Z.class(
credentialType.effectiveWriteResult,
) {}
export class CredentialTypePolicyDocumentPublicDto extends Z.class(credentialType.document) {}
export class CredentialTypePolicyDocumentWriteResultPublicDto extends Z.class(
credentialType.documentWriteResult,
) {}
export class CredentialTypePolicyDocumentListPublicDto extends Z.class(
credentialType.documentList,
) {}
export class CredentialTypePolicyAttachmentsPublicDto extends Z.class(credentialType.attachments) {}
export class ListNodeTypePolicyDocumentsQueryDto extends Z.class({
limit: publicApiPaginationSchema.limit,
cursor: z.string().optional(),
}) {}
const policyAttachmentSchema = z.object({
policyId: z.string(),
rules: policyRulesSchema,
priority: z.number().int(),
isFloor: z.boolean(),
});
/** The attachments on one scope after a replace, with the scope's bumped version. */
export class PolicyAttachmentsPublicDto extends Z.class({
attachments: z.array(policyAttachmentSchema),
version: z.number().int().positive(),
}) {}
packages/@n8n/api-types/src/dto/type-availability-policies/policy-rule.schema.ts
+39 -24
View File
@@ -1,12 +1,16 @@
import { z } from 'zod';
import type {
NodeTypePolicyRule,
NonDelegatingPolicyAction,
NonDelegatingPolicyRule,
PolicyAction,
PolicyRule,
} from './policy-rule.types';
import { policySelectorSchema } from './policy-selector.schema';
import {
credentialTypePolicySelectorSchema,
nodeTypePolicySelectorSchema,
} from './policy-selector.schema';
/**
* Zod counterpart of `PolicyAction` in `./policy-rule.types.ts`. The `satisfies` check
@@ -19,16 +23,14 @@ export const policyActionSchema = z.enum([
]) satisfies z.ZodType<PolicyAction>;
/**
* Zod counterpart of `PolicyRule` in `./policy-rule.types.ts`. The `satisfies` check
* keeps this schema honest against that type.
* `delegate` is only meaningful where a narrower scope exists to opt in — instance scope
* today. There is no scope narrower than project, so a project-scope write rejects it outright.
*/
export const policyRuleSchema = z.object({
id: z.string().min(1),
action: policyActionSchema,
selector: policySelectorSchema,
}) satisfies z.ZodType<PolicyRule>;
export const nonDelegatingPolicyActionSchema = policyActionSchema.exclude([
'delegate',
]) satisfies z.ZodType<NonDelegatingPolicyAction>;
function rejectDuplicateRuleIds(rules: PolicyRule[], ctx: z.RefinementCtx): void {
function rejectDuplicateRuleIds(rules: ReadonlyArray<{ id: string }>, ctx: z.RefinementCtx): void {
const seenIds = new Set<string>();
for (const [index, rule] of rules.entries()) {
@@ -45,22 +47,35 @@ function rejectDuplicateRuleIds(rules: PolicyRule[], ctx: z.RefinementCtx): void
}
}
/** An ordered list of rules for one policy document. Rejects duplicate rule ids. */
export const policyRuleListSchema = z.array(policyRuleSchema).superRefine(rejectDuplicateRuleIds);
/**
* `delegate` is only meaningful where a narrower scope exists to opt in — instance scope
* today. There is no scope narrower than project, so a project-scope write rejects it outright.
* One rule, an ordered rule list, and the project-scope list that also rejects `delegate`, for
* one kind's selectors. Both lists reject duplicate rule ids.
*/
export const nonDelegatingPolicyActionSchema = policyActionSchema.exclude([
'delegate',
]) satisfies z.ZodType<NonDelegatingPolicyAction>;
function ruleSchemasFor<Selector extends z.ZodTypeAny>(selector: Selector) {
const rule = z.object({ id: z.string().min(1), action: policyActionSchema, selector });
const nonDelegatingRule = rule.extend({ action: nonDelegatingPolicyActionSchema });
const nonDelegatingPolicyRuleSchema = policyRuleSchema.extend({
action: nonDelegatingPolicyActionSchema,
}) satisfies z.ZodType<NonDelegatingPolicyRule>;
return {
rule,
ruleList: z.array(rule).superRefine(rejectDuplicateRuleIds),
nonDelegatingRuleList: z.array(nonDelegatingRule).superRefine(rejectDuplicateRuleIds),
};
}
/** Same as `policyRuleListSchema`, but for project scope: rejects a `delegate` rule too. */
export const nonDelegatingPolicyRuleListSchema = z
.array(nonDelegatingPolicyRuleSchema)
.superRefine(rejectDuplicateRuleIds);
/** What one kind's rule schemas must parse into. `satisfies` keeps each kind honest against it. */
type RuleSchemas<Rule, NonDelegatingRule> = {
rule: z.ZodType<Rule>;
ruleList: z.ZodType<Rule[]>;
nonDelegatingRuleList: z.ZodType<NonDelegatingRule[]>;
};
export const nodeTypePolicyRuleSchemas = ruleSchemasFor(
nodeTypePolicySelectorSchema,
) satisfies RuleSchemas<
NodeTypePolicyRule,
NodeTypePolicyRule & { readonly action: NonDelegatingPolicyAction }
>;
export const credentialTypePolicyRuleSchemas = ruleSchemasFor(
credentialTypePolicySelectorSchema,
) satisfies RuleSchemas<PolicyRule, NonDelegatingPolicyRule>;
packages/@n8n/api-types/src/dto/type-availability-policies/policy-rule.types.ts
+19 -3
View File
@@ -1,12 +1,19 @@
/**
* Domain types for node type availability policies, shared by the internal and public API
* Domain types for type availability policies, shared by the internal and public API
* request DTOs and by the backend evaluator.
*/
/** Matches a node type by its exact full name, or by its package segment. */
/**
* Matches a type by its exact full name, or by its package segment. `extends` is for credential
* types only: it matches the named type and every type built on it through `extends`.
*/
export type PolicySelector =
| { readonly kind: 'name'; readonly value: string }
| { readonly kind: 'package'; readonly value: string };
| { readonly kind: 'package'; readonly value: string }
| { readonly kind: 'extends'; readonly value: string };
/** The selectors a node type policy accepts. */
export type NodeTypePolicySelector = Exclude<PolicySelector, { readonly kind: 'extends' }>;
export type PolicyAction = 'allow' | 'deny' | 'delegate';
@@ -17,6 +24,15 @@ export type PolicyRule = {
readonly selector: PolicySelector;
};
/** A `PolicyRule` as a node type policy accepts it. */
export type NodeTypePolicyRule = Omit<PolicyRule, 'selector'> & {
readonly selector: NodeTypePolicySelector;
};
export function isNodeTypePolicyRule(rule: PolicyRule): rule is NodeTypePolicyRule {
return rule.selector.kind !== 'extends';
}
/**
* The actions a project scope accepts. `delegate` defers to a narrower scope, and none exists
* below project, so project-scope writes never accept it.
packages/@n8n/api-types/src/dto/type-availability-policies/policy-selector.schema.ts
+18 -9
View File
@@ -1,13 +1,22 @@
import { z } from 'zod';
import type { PolicySelector } from './policy-rule.types';
import type { NodeTypePolicySelector, PolicySelector } from './policy-rule.types';
/**
* Zod counterpart of `PolicySelector` in `./policy-rule.types.ts`. The `satisfies` check
* keeps this schema honest against that type — it fails to compile if this schema ever
* accepts a shape `PolicySelector` doesn't.
*/
export const policySelectorSchema = z.discriminatedUnion('kind', [
z.object({ kind: z.literal('name'), value: z.string().min(1) }),
z.object({ kind: z.literal('package'), value: z.string().min(1) }),
const nameSelectorSchema = z.object({ kind: z.literal('name'), value: z.string().min(1) });
const packageSelectorSchema = z.object({ kind: z.literal('package'), value: z.string().min(1) });
const extendsSelectorSchema = z
.object({ kind: z.literal('extends'), value: z.string().min(1) })
.describe('Matches the named credential type and every type that extends it');
/** The selectors a node type policy accepts. `satisfies` keeps it honest against the type. */
export const nodeTypePolicySelectorSchema = z.discriminatedUnion('kind', [
nameSelectorSchema,
packageSelectorSchema,
]) satisfies z.ZodType<NodeTypePolicySelector>;
/** The selectors a credential type policy accepts: the node type ones plus `extends`. */
export const credentialTypePolicySelectorSchema = z.discriminatedUnion('kind', [
nameSelectorSchema,
packageSelectorSchema,
extendsSelectorSchema,
]) satisfies z.ZodType<PolicySelector>;
packages/@n8n/api-types/src/dto/type-availability-policies/put-instance-policy.dto.ts
+18 -5
View File
@@ -1,12 +1,25 @@
import { z } from 'zod';
import { policyActionSchema, policyRuleListSchema } from './policy-rule.schema';
import {
credentialTypePolicyRuleSchemas,
nodeTypePolicyRuleSchemas,
policyActionSchema,
} from './policy-rule.schema';
import { Z } from '../../zod-class';
/** Request body for setting the composed instance policy. */
// Optimistic concurrency: callers send the version they last read, so a stale write is rejected.
const versionSchema = z.number().int().nonnegative();
/** Request body for setting the composed instance node type policy. */
export class PutInstancePolicyDto extends Z.class({
rules: policyRuleListSchema,
rules: nodeTypePolicyRuleSchemas.ruleList,
defaultAction: policyActionSchema,
// Optimistic concurrency: callers send the version they last read, so a stale write is rejected.
version: z.number().int().nonnegative(),
version: versionSchema,
}) {}
/** Request body for setting the composed instance credential type policy. */
export class PutCredentialTypeInstancePolicyDto extends Z.class({
rules: credentialTypePolicyRuleSchemas.ruleList,
defaultAction: policyActionSchema,
version: versionSchema,
}) {}
packages/@n8n/api-types/src/dto/type-availability-policies/put-project-policy.dto.ts
+15 -5
View File
@@ -1,19 +1,29 @@
import { z } from 'zod';
import {
credentialTypePolicyRuleSchemas,
nodeTypePolicyRuleSchemas,
nonDelegatingPolicyActionSchema,
nonDelegatingPolicyRuleListSchema,
} from './policy-rule.schema';
import { Z } from '../../zod-class';
// Optimistic concurrency: callers send the version they last read, so a stale write is rejected.
const versionSchema = z.number().int().nonnegative();
/**
* Request body for setting a project's composed policy. Same shape as
* Request body for setting a project's composed node type policy. Same shape as
* `PutInstancePolicyDto`, except `delegate` is rejected — there is no scope narrower than
* project for it to defer to.
*/
export class PutProjectPolicyDto extends Z.class({
rules: nonDelegatingPolicyRuleListSchema,
rules: nodeTypePolicyRuleSchemas.nonDelegatingRuleList,
defaultAction: nonDelegatingPolicyActionSchema,
// Optimistic concurrency: callers send the version they last read, so a stale write is rejected.
version: z.number().int().nonnegative(),
version: versionSchema,
}) {}
/** Same as `PutProjectPolicyDto`, for a project's credential type policy. */
export class PutCredentialTypeProjectPolicyDto extends Z.class({
rules: credentialTypePolicyRuleSchemas.nonDelegatingRuleList,
defaultAction: nonDelegatingPolicyActionSchema,
version: versionSchema,
}) {}
packages/@n8n/api-types/src/dto/type-availability-policies/update-policy-document.dto.ts
+13 -5
View File
@@ -1,11 +1,19 @@
import { z } from 'zod';
import { policyRuleListSchema } from './policy-rule.schema';
import { credentialTypePolicyRuleSchemas, nodeTypePolicyRuleSchemas } from './policy-rule.schema';
import { Z } from '../../zod-class';
/** Request body for replacing a policy document's rules. */
// Optimistic concurrency: callers send the version they last read, so a stale write is rejected.
const versionSchema = z.number().int().nonnegative();
/** Request body for replacing a node type policy document's rules. */
export class UpdatePolicyDocumentDto extends Z.class({
rules: policyRuleListSchema,
// Optimistic concurrency: callers send the version they last read, so a stale write is rejected.
version: z.number().int().nonnegative(),
rules: nodeTypePolicyRuleSchemas.ruleList,
version: versionSchema,
}) {}
/** Request body for replacing a credential type policy document's rules. */
export class UpdateCredentialTypePolicyDocumentDto extends Z.class({
rules: credentialTypePolicyRuleSchemas.ruleList,
version: versionSchema,
}) {}
packages/@n8n/db/src/entities/idempotency-key.ts
+40
View File
@@ -0,0 +1,40 @@
import { Column, Entity, Index, JoinColumn, ManyToOne, Relation } from '@n8n/typeorm';
import { JsonColumn, WithTimestampsAndStringId } from './abstract-entity';
import type { User } from './user';
export const idempotencyKeyStatuses = ['processing', 'completed'] as const;
export type IdempotencyKeyStatus = (typeof idempotencyKeyStatuses)[number];
/**
* One stored Public API write per user and Idempotency-Key.
* A retry with the same user, key, and fingerprint returns this row.
* The handler does not run again.
*/
@Entity({ name: 'idempotency_key' })
@Index(['userId', 'idempotencyKey'], { unique: true })
@Index(['createdAt'])
export class IdempotencyKey extends WithTimestampsAndStringId {
@Column({ type: 'uuid' })
userId: string;
@ManyToOne('User', { onDelete: 'CASCADE' })
@JoinColumn({ name: 'userId' })
user: Relation<User>;
@Column({ type: 'varchar', length: 128 })
idempotencyKey: string;
@Column({ type: 'text' })
fingerprint: string;
@Column({ type: 'varchar', length: 16, default: 'processing' })
status: IdempotencyKeyStatus;
@Column({ type: 'smallint', nullable: true })
responseStatus: number | null;
@JsonColumn({ nullable: true })
responseBody: unknown;
}
packages/@n8n/db/src/entities/index.ts
+9
View File
@@ -37,6 +37,11 @@ import type { ExecutionDataStorageLocation } from './execution-entity';
import { ExecutionMetadata } from './execution-metadata';
import { Folder } from './folder';
import { FolderTagMapping } from './folder-tag-mapping';
import {
IdempotencyKey,
idempotencyKeyStatuses,
type IdempotencyKeyStatus,
} from './idempotency-key';
import { InstanceCredentialAssignment } from './instance-credential-assignment';
import { InvalidAuthToken } from './invalid-auth-token';
import { PollerState } from './poller-state';
@@ -124,6 +129,9 @@ export {
type AgentEvalResultStatus,
AgentEvalRating,
type AgentEvalVote,
IdempotencyKey,
idempotencyKeyStatuses,
type IdempotencyKeyStatus,
InvalidAuthToken,
InstanceCredentialAssignment,
AiBuilderTemporaryWorkflow,
@@ -219,6 +227,7 @@ export const entities = {
AgentEvalRun,
AgentEvalResult,
AgentEvalRating,
IdempotencyKey,
InvalidAuthToken,
InstanceCredentialAssignment,
AiBuilderTemporaryWorkflow,
packages/@n8n/db/src/migrations/common/1791540050944-CreateIdempotencyKeyTable.ts
+47
View File
@@ -0,0 +1,47 @@
import type { MigrationContext, ReversibleMigration } from '../migration-types';
const tableName = 'idempotency_key';
export class CreateIdempotencyKeyTable1791540050944 implements ReversibleMigration {
async up({ schemaBuilder: { createTable, column }, escape, tablePrefix }: MigrationContext) {
const status = escape.columnName('status');
const responseStatus = escape.columnName('responseStatus');
await createTable(tableName)
.withColumns(
column('id').varchar(16).primary.comment('Application-generated n8n nano ID'),
column('userId').uuid.notNull,
column('idempotencyKey')
.varchar(128)
.notNull.comment('Client Idempotency-Key header. Opaque ASCII, length 1 to 128'),
column('fingerprint').text.notNull.comment('Hash of the request method, path, and body'),
column('status')
.varchar(16)
.notNull.default("'processing'")
.withEnumCheck(['processing', 'completed'])
.comment('processing while the handler runs. completed after the response is stored'),
column('responseStatus').smallint.comment(
'HTTP status of the stored response. NULL while processing, required once completed',
),
column('responseBody').json.comment(
'Stored response body. NULL while status is processing',
),
)
.withIndexOn(['userId', 'idempotencyKey'], true)
.withIndexOn('createdAt')
.withForeignKey('userId', {
tableName: 'user',
columnName: 'id',
onDelete: 'CASCADE',
name: `FK_${tablePrefix}idempotency_key_userId`,
})
.withCheck(
`CHK_${tablePrefix}idempotency_key_responseStatus`,
`(${status} = 'processing' AND ${responseStatus} IS NULL) OR (${status} = 'completed' AND ${responseStatus} IS NOT NULL)`,
).withTimestamps;
}
async down({ schemaBuilder: { dropTable } }: MigrationContext) {
await dropTable(tableName);
}
}
packages/@n8n/db/src/repositories/idempotency-key.repository.ts
+13
View File
@@ -0,0 +1,13 @@
import { Service } from '@n8n/di';
import { DataSource } from '@n8n/typeorm';
import { BaseRepository } from './base-repository';
import { IdempotencyKey } from '../entities';
import { TransactionRunner } from '../services/transaction';
@Service()
export class IdempotencyKeyRepository extends BaseRepository<IdempotencyKey> {
constructor(dataSource: DataSource, transactionRunner: TransactionRunner) {
super(IdempotencyKey, dataSource.manager, transactionRunner);
}
}
packages/@n8n/db/src/repositories/index.ts
+1
View File
@@ -55,6 +55,7 @@ export { FolderRepository } from './folder.repository';
export { FolderAccessRepository } from './folder-access.repository';
export { FolderTagMappingRepository } from './folder-tag-mapping.repository';
export { ScopeRepository } from './scope.repository';
export { IdempotencyKeyRepository } from './idempotency-key.repository';
export { InvalidAuthTokenRepository } from './invalid-auth-token.repository';
export { InstanceCredentialAssignmentRepository } from './instance-credential-assignment.repository';
export { LicenseMetricsRepository } from './license-metrics.repository';
packages/@n8n/decorators/src/controller/__tests__/api-response.test.ts
+59 -1
View File
@@ -5,7 +5,7 @@ import { z } from 'zod';
import { ApiResponse } from '../api-response';
import { ControllerRegistryMetadata } from '../controller-registry-metadata';
import { Get } from '../route';
import type { Controller } from '../types';
import type { BinaryResponse, Controller } from '../types';
const ExampleDto = Z.class({
id: z.string(),
@@ -64,4 +64,62 @@ describe('@ApiResponse Decorator', () => {
expect(route.successStatus).toBe(204);
expect(route.responseDto).toBeUndefined();
});
it('should store binary response metadata and leave responseDto undefined', () => {
const binaryOptions: BinaryResponse = {
mediaType: 'application/gzip',
description: 'Compressed archive',
headers: {
'X-Archive-Size': { description: 'Size of the archive' },
},
};
class TestController {
@Get('/')
@ApiResponse(200, binaryOptions)
async handler() {}
}
const route = controllerRegistryMetadata.getRouteMetadata(
TestController as Controller,
'handler',
);
expect(route.binaryResponse).toEqual(binaryOptions);
expect(route.successStatus).toBe(200);
expect(route.responseDto).toBeUndefined();
});
it('should reject a binary response with an unsupported media type', () => {
expect(() => {
class TestController {
@Get('/')
@ApiResponse(200, { mediaType: 'text/csv' } as never)
async handler() {}
}
void TestController;
}).toThrow('unsupported binary media type "text/csv"');
});
it('should reject a body that is neither a response DTO nor binary options', () => {
expect(() => {
class TestController {
@Get('/')
@ApiResponse(200, {} as never)
async handler() {}
}
void TestController;
}).toThrow('neither a response DTO nor binary options');
});
it('should reject 204 with binary response', () => {
expect(() => {
const binaryOptions = { mediaType: 'application/gzip' } as never;
class TestController {
@Get('/')
@ApiResponse(204, binaryOptions)
async handler() {}
}
void TestController;
}).toThrow('declares a 204 @ApiResponse with a binary body');
});
});
packages/@n8n/decorators/src/controller/api-response.ts
+61 -9
View File
@@ -1,15 +1,47 @@
import { Container } from '@n8n/di';
import { ControllerRegistryMetadata } from './controller-registry-metadata';
import type { Controller, ResponseDtoClass, SuccessStatus } from './types';
import {
BINARY_RESPONSE_MEDIA_TYPES,
type BinaryResponse,
type Controller,
type ResponseDtoClass,
type SuccessStatus,
} from './types';
function hasMediaType(body: unknown): body is { mediaType: unknown } {
return typeof body === 'object' && body !== null && 'mediaType' in body;
}
function isBinaryResponse(body: unknown): body is BinaryResponse {
return (
hasMediaType(body) &&
BINARY_RESPONSE_MEDIA_TYPES.some((mediaType) => mediaType === body.mediaType)
);
}
function isResponseDto(body: unknown): body is ResponseDtoClass {
return (
(typeof body === 'function' || (typeof body === 'object' && body !== null)) && 'parse' in body
);
}
/**
* Declares what a route returns on success: its HTTP status, and its public output DTO if it sends a
* body. Only one @ApiResponse decorator should be present per endpoint otherwise an error will be thrown.
* Declares what a route returns on success: its HTTP status, and either its public output DTO or a
* binary body that the controller method returns.
*
* Only one @ApiResponse decorator should be present per endpoint otherwise an error will be thrown.
*/
export const ApiResponse =
(status: SuccessStatus, dto?: ResponseDtoClass): MethodDecorator =>
(target, handlerName) => {
export function ApiResponse(status: SuccessStatus, dto?: ResponseDtoClass): MethodDecorator;
export function ApiResponse(
status: Exclude<SuccessStatus, 204>,
binary: BinaryResponse,
): MethodDecorator;
export function ApiResponse(
status: SuccessStatus,
body?: ResponseDtoClass | BinaryResponse,
): MethodDecorator {
return (target, handlerName) => {
const routeMetadata = Container.get(ControllerRegistryMetadata).getRouteMetadata(
target.constructor as Controller,
String(handlerName),
@@ -23,12 +55,32 @@ export const ApiResponse =
}
// HTTP 204 No Content responses must not have a body
if (status === 204 && dto !== undefined) {
if (status === 204 && body !== undefined) {
const bodyKind = isBinaryResponse(body) ? 'a binary body' : 'a response DTO';
throw new Error(
`${String(handlerName)} declares a 204 @ApiResponse with a response DTO - a 204 response must not have a body`,
`${String(handlerName)} declares a 204 @ApiResponse with ${bodyKind} - a 204 response must not have a body`,
);
}
routeMetadata.successStatus = status;
routeMetadata.responseDto = dto;
if (isBinaryResponse(body)) {
routeMetadata.binaryResponse = body;
return;
}
if (hasMediaType(body)) {
throw new Error(
`${String(handlerName)} declares an unsupported binary media type "${String(body.mediaType)}" - supported: ${BINARY_RESPONSE_MEDIA_TYPES.join(', ')}`,
);
}
if (body !== undefined && !isResponseDto(body)) {
throw new Error(
`${String(handlerName)} declares an @ApiResponse body that is neither a response DTO nor binary options`,
);
}
routeMetadata.responseDto = body;
};
}
packages/@n8n/decorators/src/controller/index.ts
+4
View File
@@ -19,6 +19,9 @@ export type {
AccessScope,
ApiKeyScopeRequirement,
Arg,
BinaryResponse,
BinaryResponseMediaType,
BinaryResult,
Controller,
CorsOptions,
DeprecationInfo,
@@ -30,6 +33,7 @@ export type {
RequestBodyMediaOptions,
RequestBodyMediaType,
ResponseDtoClass,
ResponseHeader,
RouteMetadata,
StaticRouterMetadata,
SuccessStatus,
packages/@n8n/decorators/src/controller/types.ts
+21
View File
@@ -2,6 +2,7 @@ import type { ZodClass } from '@n8n/api-types';
import type { BooleanLicenseFeature } from '@n8n/constants';
import type { Constructable } from '@n8n/di';
import type { ApiKeyScope, Scope } from '@n8n/permissions';
import type { Readable } from 'node:stream';
import type { RequestHandler, Router } from 'express';
import type { ZodTypeAny } from 'zod';
@@ -14,6 +15,24 @@ export type ApiKeyScopeRequirement =
export type ResponseDtoClass = Pick<ZodClass, 'parse'>;
export const BINARY_RESPONSE_MEDIA_TYPES = ['application/gzip'] as const;
export type BinaryResponseMediaType = (typeof BINARY_RESPONSE_MEDIA_TYPES)[number];
export interface ResponseHeader {
description: string;
}
export interface BinaryResponse {
mediaType: BinaryResponseMediaType;
description?: string;
headers?: Record<string, ResponseHeader>;
}
export interface BinaryResult {
body: Buffer | Readable;
headers?: Record<string, string | number | readonly string[]>;
}
export type SuccessStatus = 200 | 201 | 202 | 204;
export interface ErrorResponse {
@@ -99,6 +118,8 @@ export interface RouteMetadata {
accessScope?: AccessScope;
apiKeyScope?: ApiKeyScopeRequirement;
responseDto?: ResponseDtoClass;
/** Mutually exclusive with `responseDto`. */
binaryResponse?: BinaryResponse;
/** OpenAPI HTTP status sent on success, and documented as such. */
successStatus?: SuccessStatus;
/** OpenAPI operation summary. */
packages/@n8n/decorators/src/execution-lifecycle/lifecycle-metadata.ts
+5
View File
@@ -56,6 +56,11 @@ export type WorkflowExecuteAfterContext = {
retryOf?: string;
/** Who initiated the run. Unset means a regular user-initiated run. */
source?: WorkflowExecutionSource;
/**
* False when this process only enqueued the run and waited for its job. In queue mode,
* the process that ran the job fires its own `workflowExecuteAfter` for the same run.
*/
executedInThisProcess: boolean;
};
export type WorkflowExecuteResumeContext = {
packages/@n8n/module-cli/frontend-module-guide.md
+21
View File
@@ -274,6 +274,27 @@ routes: [
If `meta.middleware` has no `'custom'` entry, the route resolves. The state of the module then
makes no difference.
### Placeholder page for licensed features
An unlicensed module is inactive, so its settings links and routes disappear. To show a paywall
instead, set `placeholderPage` with the module's license feature:
```ts
placeholderPage: {
licenseFlag: 'logStreaming', // a key of `settings.enterprise`
component: async () => await import('./views/MyFeaturePaywall.vue'),
},
```
When the module is inactive and `licenseFlag` is off, its settings links stay and its routes show
this page. The `available` getter and route `rbac` still apply. The shell adds the `'custom'` guard
to these routes, so you don't need to list it. Route views must be lazy loaders. The shell picks
the view or the placeholder once, on the first load of the route.
If the module is licensed but inactive, for example turned off with `N8N_DISABLED_MODULES`, it
stays hidden. A module that is unlicensed and also turned off still shows the page, because the
frontend can't tell it apart from a module that is only unlicensed.
## Import-light descriptors
The descriptor file can import **types and the SDK only**. Load views lazily. Read a store inside
packages/@n8n/nodes-langchain/credentials/GoogleVertexAiApi.credentials.ts
+19 -1
View File
@@ -1,5 +1,7 @@
import type { ICredentialType, INodeProperties, Icon } from 'n8n-workflow';
import { GOOGLE_VERTEX_CUSTOM_PROJECT, searchGoogleProjects } from '../utils/google-vertex';
export class GoogleVertexAiApi implements ICredentialType {
name = 'googleVertexAiApi';
@@ -7,11 +9,24 @@ export class GoogleVertexAiApi implements ICredentialType {
displayName = 'Google Vertex AI';
documentationUrl = 'google/service-account';
documentationUrl = 'googlevertexai';
icon: Icon = 'file:icons/google.svg';
properties: INodeProperties[] = [
{
displayName: 'Project',
name: 'project',
type: 'options',
options: [{ name: 'Custom', value: GOOGLE_VERTEX_CUSTOM_PROJECT }],
default: GOOGLE_VERTEX_CUSTOM_PROJECT,
required: true,
description: 'Select a Google Cloud project, or select Custom to enter a project ID',
typeOptions: {
loadOptionsMethod: 'gcpProjectsList',
loadOptionsDependsOn: ['email', 'privateKey'],
},
},
{
displayName: 'Project ID',
name: 'projectId',
@@ -20,6 +35,7 @@ export class GoogleVertexAiApi implements ICredentialType {
required: true,
placeholder: 'my-project-id',
description: 'Google Cloud project to use with Vertex AI',
displayOptions: { show: { project: [GOOGLE_VERTEX_CUSTOM_PROJECT] } },
},
// Hide inherited settings that the Vertex AI client does not use.
{
@@ -65,4 +81,6 @@ export class GoogleVertexAiApi implements ICredentialType {
default: '',
},
];
methods = { loadOptions: { gcpProjectsList: searchGoogleProjects } };
}
packages/@n8n/nodes-langchain/nodes/embeddings/EmbeddingsGoogleVertex/EmbeddingsGoogleVertex.node.ts
+11 -22
View File
@@ -1,5 +1,3 @@
import { ProjectsClient } from '@google-cloud/resource-manager';
import { getGoogleServiceAccountCredentials } from 'n8n-nodes-base/google-service-account';
import { VertexAIEmbeddings } from '@langchain/google-vertexai';
import { logWrapper, getConnectionHintNoticeField } from '@n8n/ai-utilities';
import { NodeConnectionTypes } from 'n8n-workflow';
@@ -18,32 +16,23 @@ import {
googleVertexCredentials,
googleVertexAiCredentialTest,
resolveGoogleVertexCredentials,
searchGoogleProjects,
} from '@utils/google-vertex';
export class EmbeddingsGoogleVertex implements INodeType {
methods = {
credentialTest: { googleVertexAiCredentialTest },
listSearch: {
async gcpProjectsList(this: ILoadOptionsFunctions) {
const results: Array<{ name: string; value: string }> = [];
const credentials = await this.getCredentials('googleApi');
const client = new ProjectsClient({
credentials: getGoogleServiceAccountCredentials(credentials),
});
const [projects] = await client.searchProjects();
for (const project of projects) {
if (project.projectId) {
results.push({
name: project.displayName ?? project.projectId,
value: project.projectId,
});
}
}
return { results };
async gcpProjectsList(
this: ILoadOptionsFunctions,
filter?: string,
paginationToken?: string,
) {
return await searchGoogleProjects(
await this.getCredentials('googleApi'),
filter,
paginationToken,
);
},
},
};
packages/@n8n/nodes-langchain/nodes/llms/LmChatGoogleVertex/LmChatGoogleVertex.node.ts
+11 -22
View File
@@ -1,5 +1,3 @@
import { ProjectsClient } from '@google-cloud/resource-manager';
import { getGoogleServiceAccountCredentials } from 'n8n-nodes-base/google-service-account';
import type { GoogleAISafetySetting } from '@langchain/google-common';
import { ChatVertexAI, type ChatVertexAIInput } from '@langchain/google-vertexai';
import {
@@ -31,6 +29,7 @@ import {
googleVertexCredentials,
googleVertexAiCredentialTest,
resolveGoogleVertexCredentials,
searchGoogleProjects,
} from '@utils/google-vertex';
function errorDescriptionMapper(error: NodeError) {
@@ -121,26 +120,16 @@ export class LmChatGoogleVertex implements INodeType {
methods = {
credentialTest: { googleVertexAiCredentialTest },
listSearch: {
async gcpProjectsList(this: ILoadOptionsFunctions) {
const results: Array<{ name: string; value: string }> = [];
const credentials = await this.getCredentials('googleApi');
const client = new ProjectsClient({
credentials: getGoogleServiceAccountCredentials(credentials),
});
const [projects] = await client.searchProjects();
for (const project of projects) {
if (project.projectId) {
results.push({
name: project.displayName ?? project.projectId,
value: project.projectId,
});
}
}
return { results };
async gcpProjectsList(
this: ILoadOptionsFunctions,
filter?: string,
paginationToken?: string,
) {
return await searchGoogleProjects(
await this.getCredentials('googleApi'),
filter,
paginationToken,
);
},
},
};
packages/@n8n/nodes-langchain/utils/google-vertex.ts
+53 -4
View File
@@ -5,16 +5,20 @@ import {
} from 'n8n-nodes-base/google-service-account';
import {
NodeOperationError,
type ICredentialDataDecryptedObject,
type ICredentialsDecrypted,
type ICredentialTestFunctions,
type INodeCredentialDescription,
type INodeCredentialTestResult,
type INodeListSearchResult,
type INodeProperties,
type ISupplyDataFunctions,
} from 'n8n-workflow';
import { resolveVertexLocation } from '../nodes/llms/gemini-common/vertex-location';
export const GOOGLE_VERTEX_CUSTOM_PROJECT = '__custom__';
export const googleVertexCredentials: INodeCredentialDescription[] = [
{
name: 'googleApi',
@@ -41,6 +45,11 @@ export const googleVertexAuthentication: INodeProperties = {
default: 'googleApi',
};
function getGoogleVertexProjectId(credentials: ICredentialDataDecryptedObject) {
const project = credentials.project ?? GOOGLE_VERTEX_CUSTOM_PROJECT;
return project === GOOGLE_VERTEX_CUSTOM_PROJECT ? credentials.projectId : project;
}
export async function resolveGoogleVertexCredentials(
context: ISupplyDataFunctions,
itemIndex: number,
@@ -50,7 +59,7 @@ export async function resolveGoogleVertexCredentials(
const credentials = await context.getCredentials(credentialType);
let projectId: unknown;
if (credentialType === 'googleVertexAiApi') {
projectId = credentials.projectId;
projectId = getGoogleVertexProjectId(credentials);
} else {
projectId = context.getNodeParameter('projectId', itemIndex, '', { extractValue: true });
}
@@ -77,8 +86,9 @@ export async function googleVertexAiCredentialTest(
this: ICredentialTestFunctions,
credential: ICredentialsDecrypted,
): Promise<INodeCredentialTestResult> {
const data = credential.data;
if (typeof data?.projectId !== 'string' || !data.projectId.trim()) {
const data = credential.data ?? {};
const projectId = getGoogleVertexProjectId(data);
if (typeof projectId !== 'string' || !projectId.trim()) {
return { status: 'Error', message: 'Select or enter a Google Cloud project ID.' };
}
@@ -99,7 +109,7 @@ export async function googleVertexAiCredentialTest(
// Check project access without running a model or requiring the Resource Manager API.
await this.helpers.request({
method: 'GET',
uri: `https://aiplatform.googleapis.com/v1/projects/${encodeURIComponent(data.projectId.trim())}/locations`,
uri: `https://aiplatform.googleapis.com/v1/projects/${encodeURIComponent(projectId.trim())}/locations`,
headers: { Authorization: `Bearer ${token.access_token}` },
qs: { pageSize: 1 },
json: true,
@@ -114,3 +124,42 @@ export async function googleVertexAiCredentialTest(
};
}
}
export async function searchGoogleProjects(
credentials: ICredentialDataDecryptedObject,
filter?: string,
paginationToken?: string,
): Promise<INodeListSearchResult> {
const { ProjectsClient } = await import('@google-cloud/resource-manager');
const client = new ProjectsClient({
credentials: getGoogleServiceAccountCredentials(credentials),
});
try {
const trimmedFilter = filter?.trim();
let query: string | undefined;
if (trimmedFilter) {
const prefix = JSON.stringify(`${trimmedFilter}*`);
query = `displayName:${prefix} projectId:${prefix}`;
}
const [projects, nextPage] = await client.searchProjects(
{ pageToken: paginationToken, ...(query ? { query } : {}) },
{ autoPaginate: false },
);
return {
results: projects.flatMap((project) => {
if (!project.projectId) return [];
return [
{
name: project.displayName
? `${project.displayName} (${project.projectId})`
: project.projectId,
value: project.projectId,
},
];
}),
paginationToken: nextPage?.pageToken ?? undefined,
};
} finally {
await client.close();
}
}
packages/@n8n/nodes-langchain/utils/tests/google-vertex.test.ts
+148 -19
View File
@@ -6,18 +6,126 @@ import {
type ICredentialTestFunctions,
type INode,
type INodeTypes,
type ISupplyDataFunctions,
} from 'n8n-workflow';
import { mock } from 'vitest-mock-extended';
import { EmbeddingsGoogleVertex } from '../../nodes/embeddings/EmbeddingsGoogleVertex/EmbeddingsGoogleVertex.node';
import { LmChatGoogleVertex } from '../../nodes/llms/LmChatGoogleVertex/LmChatGoogleVertex.node';
import { googleVertexAiCredentialTest } from '../google-vertex';
import {
googleVertexAiCredentialTest,
resolveGoogleVertexCredentials,
searchGoogleProjects,
} from '../google-vertex';
vi.mock('n8n-nodes-base/google-service-account', async (importOriginal) => ({
...(await importOriginal<typeof import('n8n-nodes-base/google-service-account')>()),
getGoogleAccessToken: vi.fn(),
}));
const { searchProjects, close, ProjectsClient } = vi.hoisted(() => {
const searchProjects = vi.fn();
const close = vi.fn();
return {
searchProjects,
close,
ProjectsClient: vi.fn(
class {
searchProjects = searchProjects;
close = close;
},
),
};
});
vi.mock('@google-cloud/resource-manager', () => ({ ProjectsClient }));
describe('Google Vertex credential project', () => {
it.each([
{ project: undefined, projectId: 'saved-project', expected: 'saved-project' },
{ project: '__custom__', projectId: 'manual-project', expected: 'manual-project' },
{ project: 'selected-project', projectId: 'old-manual-project', expected: 'selected-project' },
])('uses project $expected for selector $project', async ({ project, projectId, expected }) => {
const context = mock<ISupplyDataFunctions>();
context.getCredentials.mockResolvedValue({
email: '[email protected]',
privateKey: 'test-key',
region: 'global',
...(project === undefined ? {} : { project }),
projectId,
});
context.getNodeParameter.mockImplementation((name) =>
name === 'authentication' ? 'googleVertexAiApi' : '',
);
await expect(resolveGoogleVertexCredentials(context, 0)).resolves.toMatchObject({
projectId: expected,
});
});
});
describe('Google project discovery', () => {
const credentials = Object.freeze({
email: ' [email protected] ',
privateKey: '-----BEGIN PRIVATE KEY----- key -----END PRIVATE KEY-----',
region: 'eu',
});
beforeEach(() => vi.clearAllMocks());
it.each([
{ filter: undefined, expectedRequest: { pageToken: 'current-page' } },
{ filter: '', expectedRequest: { pageToken: 'current-page' } },
{ filter: ' \t\n ', expectedRequest: { pageToken: 'current-page' } },
{
filter: 'target',
expectedRequest: {
pageToken: 'current-page',
query: 'displayName:"target*" projectId:"target*"',
},
},
{
filter: ' Target "project"\\folder ',
expectedRequest: {
pageToken: 'current-page',
query:
'displayName:"Target \\"project\\"\\\\folder*" projectId:"Target \\"project\\"\\\\folder*"',
},
},
])('returns projects and pagination for filter $filter', async ({ filter, expectedRequest }) => {
searchProjects.mockResolvedValue([
[
{ displayName: 'Target project', projectId: 'target-project' },
{ projectId: 'another-project' },
{ displayName: 'Missing ID' },
],
{ pageToken: 'next-page' },
]);
await expect(searchGoogleProjects(credentials, filter, 'current-page')).resolves.toEqual({
results: [
{ name: 'Target project (target-project)', value: 'target-project' },
{ name: 'another-project', value: 'another-project' },
],
paginationToken: 'next-page',
});
expect(ProjectsClient).toHaveBeenCalledWith({
credentials: {
client_email: '[email protected]',
private_key: '-----BEGIN PRIVATE KEY-----\nkey\n-----END PRIVATE KEY-----',
},
});
expect(searchProjects).toHaveBeenCalledWith(expectedRequest, { autoPaginate: false });
expect(close).toHaveBeenCalled();
});
it('closes the client when project discovery fails', async () => {
searchProjects.mockRejectedValue(new Error('Permission denied'));
await expect(searchGoogleProjects(credentials)).rejects.toThrow('Permission denied');
expect(close).toHaveBeenCalled();
});
});
describe.each([new LmChatGoogleVertex(), new EmbeddingsGoogleVertex()])(
'$description.displayName credential selection',
(node) => {
@@ -76,14 +184,17 @@ describe('googleVertexAiCredentialTest', () => {
const context = mock<ICredentialTestFunctions>({
helpers: mock<ICredentialTestFunctions['helpers']>({ request }),
});
const credential = mock<ICredentialsDecrypted>({
const credential: ICredentialsDecrypted = {
id: 'vertex-credential',
name: 'Google Vertex AI',
type: 'googleVertexAiApi',
data: {
email: '[email protected]',
privateKey: 'test-private-key',
projectId: ' test-project ',
delegatedEmail: '[email protected]',
},
});
};
beforeEach(() => {
vi.resetAllMocks();
@@ -91,23 +202,41 @@ describe('googleVertexAiCredentialTest', () => {
request.mockResolvedValue({ locations: [] });
});
it('checks the credential project with Vertex scopes and service account authentication', async () => {
const result = await googleVertexAiCredentialTest.call(context, credential);
it.each([
{ project: undefined, projectId: ' test-project ', expected: 'test-project' },
{ project: '__custom__', projectId: ' manual-project ', expected: 'manual-project' },
{
project: ' selected-project ',
projectId: 'old-manual-project',
expected: 'selected-project',
},
])(
'checks project $expected with Vertex scopes and service account authentication',
async ({ project, projectId, expected }) => {
const result = await googleVertexAiCredentialTest.call(context, {
...credential,
data: {
...credential.data,
...(project === undefined ? {} : { project }),
projectId,
},
});
expect(result.status).toBe('OK');
expect(getGoogleAccessToken).toHaveBeenCalledWith(
{ email: '[email protected]', privateKey: 'test-private-key' },
'vertex',
);
expect(request).toHaveBeenCalledWith(
expect.objectContaining({
method: 'GET',
uri: 'https://aiplatform.googleapis.com/v1/projects/test-project/locations',
headers: { Authorization: 'Bearer test-access-token' },
qs: { pageSize: 1 },
}),
);
});
expect(result.status).toBe('OK');
expect(getGoogleAccessToken).toHaveBeenCalledWith(
{ email: '[email protected]', privateKey: 'test-private-key' },
'vertex',
);
expect(request).toHaveBeenCalledWith(
expect.objectContaining({
method: 'GET',
uri: `https://aiplatform.googleapis.com/v1/projects/${expected}/locations`,
headers: { Authorization: 'Bearer test-access-token' },
qs: { pageSize: 1 },
}),
);
},
);
it('reports a project error even when the service account key is valid', async () => {
request.mockRejectedValue(new Error('Project not found'));
packages/@n8n/telemetry/src/events/type-availability-policies.ts
+3
View File
@@ -65,6 +65,9 @@ export const TYPE_AVAILABILITY_POLICIES_TELEMETRY = defineTelemetryEvents({
...ruleCounts,
name_selector_count: z.number().describe('Rules that target one exact type name'),
package_selector_count: z.number().describe('Rules that target a whole package'),
extends_selector_count: z
.number()
.describe('Rules that target a credential type and every type built on it'),
evaluated_type_count: z
.number()
.describe(
packages/cli/docs/adr/ADR-20261005-replay-the-first-public-api-response-for-an-idempotency-key.md
+88
View File
@@ -0,0 +1,88 @@
# Replay the first Public API response for an Idempotency-Key
Date: 2026-10-05
Status: Active
Decision Owner: API & OEM
Source: https://linear.app/n8n/issue/API-294
## Context
The Public API accepts an Idempotency-Key header on a POST that creates a
resource or starts work. The client opts in by sending the header. A request
with no header keeps today's behaviour and does no store I/O.
PUT and PATCH return the same outcome on a repeat. A second DELETE can return
404 after the first returns 200. The record stays deleted. The store ignores
the header on those routes, and on GET and HEAD.
A handler can save a resource and then fail. The status code does not show
whether that save happened. The credential create handler inserts the row,
then loads the owning project. A failure in that later step returns 500, and
the credential already exists.
## Decision
We keep the first finished response and return it when the same user sends the
same key and the same fingerprint again. The stored response includes 4xx and
5xx. The handler runs only for the first request. The client sends a new key
after a real failure.
1. Compare the fingerprint first. It is a hash of the method, the path, the
query string, and the body. The path is the request path. It includes the
public API prefix and the version, for example `/api/v1/workflows`. It
excludes the scheme and the host. The query string is the raw text after
`?`, without that `?`. No query is an empty string. The body is the raw
JSON text. A retry must send the same characters. For
`POST /api/v1/workflows?source=api` with body `{"name":"Hello"}`, the hash
covers `POST`, `/api/v1/workflows`, `source=api`, and `{"name":"Hello"}`.
The same key with a different fingerprint returns 422. This applies while
the first request is `processing` and after it completes. The handler does
not run. The client sends a new key.
2. The same key and the same fingerprint replay the stored response. A retry
while that request is still `processing` returns 409 Conflict. A completed
row returns the stored response, including a stored 400.
3. The key is an opaque string. We compare it exactly after trim. The length
is 1 to 128 characters, and the characters are visible ASCII. A longer value
or a disallowed character returns 400. An empty value, or a value that is
only whitespace, is the same as no header.
4. The internal `/rest` API stays out of this version. The store has no HTTP
types, so `/rest` can mount later.
## Alternatives Considered
1. **Wait for the in-flight request, then return its response.** Waiting needs
coordination across mains. A 409 tells the client to retry later.
2. **Return only the new resource id when the body is large.** The same POST
would then have two response shapes.
3. **Apply the store on every request.** Clients that send no key would pay for
a store read. The header stays opt-in.
4. **Mount the store on `/rest` in this version.** `/rest` has different auth,
a higher write volume, and existing version checks.
5. **Release the key when the handler returns an error.** A later retry would
run the handler again. A 500 can follow a committed write, so that retry
can create a second resource. The stored error blocks that second write.
## Consequences
1. If the process dies after the claim and before it stores the response, a
later retry with the same fingerprint receives 409 until the row expires.
2. An instance setting can turn the feature off. While it is off, the header is
ignored and the store is idle.
3. Idempotency leaves version checks unchanged. `forceSave` still works as it
does today.
4. The stored row is defined in
ADR-20261005-store-public-api-idempotency-keys-in-the-instance-database.
5. After an error, the same fingerprint returns the stored response until the
row expires. A bug fix does not change the stored response. A new key runs
the handler again.
## Links
RFC: https://app.notion.com/p/n8n/Public-API-idempotency-3d55b6e0c94f816cb074fef7efcb6280
Documentation: https://linear.app/n8n/issue/API-294
Related ADRs: ADR-20261005-store-public-api-idempotency-keys-in-the-instance-database
packages/cli/docs/adr/ADR-20261005-store-public-api-idempotency-keys-in-the-instance-database.md
+74
View File
@@ -0,0 +1,74 @@
# Store Public API idempotency keys in the instance database
Date: 2026-10-05
Status: Active
Decision Owner: API & OEM
Source: https://linear.app/n8n/issue/API-386
## Context
A dropped connection can make a client send the same Public API POST twice.
The second call can create a second workflow, credential, or execution. The
client sends an Idempotency-Key header so a retry returns the first result.
The record must survive a process restart and stay unique across mains. A
flush or eviction drops a Redis row. A lost row lets the POST run again. The
record holds a status and a response for hours.
## Decision
We store each key in the instance database, in the table `idempotency_key`.
Postgres and SQLite both receive the table.
1. The unique key is `(userId, idempotencyKey)`. Two API keys of the same user
share one namespace. Two users may send the same key.
2. The row stores the fingerprint and the status (`processing` or
`completed`). It also stores the response status and the response body.
Those two columns stay empty while the handler runs. The stored response
includes a 4xx or a 5xx. We redact sensitive values in the response body
before we store it. We then encrypt that body. The column holds
ciphertext. Replay decrypts the stored body and returns it. The first
response and the replay are that same body. The replay rules are in
ADR-20261005-replay-the-first-public-api-response-for-an-idempotency-key.
3. `userId` references `user.id`. Deleting the user deletes that user's keys.
4. A row expires 12 hours after `createdAt`. The read path treats an older row
as a miss. A changed TTL applies to rows that already exist, because expiry
is `createdAt` plus the TTL.
5. A system task deletes expired rows on an interval.
## Alternatives Considered
1. **Redis for the key store.** A flush or eviction drops the row, and the POST
can run again. The instance database keeps the row for the full retention
window.
2. **A 24 hour window, as Stripe uses.** The dropped-connection case lasts
seconds to minutes. 12 hours covers same-day recovery.
3. **A 6 hour window.** A late-day retry of a morning write misses the stored
response.
4. **Store the response body as plaintext JSON.** A workflow response can
contain values the user put in node parameters. The row would keep a
second plaintext copy of those values until it expires.
## Consequences
1. A keyed write takes the SQLite writer lock. A client that sends a key on
every request should use Postgres.
2. The same key can run the operation again after the row expires.
3. A create response can be larger than the request. The JSON body parser caps
the incoming payload. Express does not cap the outgoing response. Many keyed
creates hold that response until cleanup.
4. The request contract is recorded in
ADR-20261005-replay-the-first-public-api-response-for-an-idempotency-key.
5. A database reader sees ciphertext for the response body. Replay decrypts
that body before it sends the response.
## Links
RFC: https://app.notion.com/p/n8n/Public-API-idempotency-3d55b6e0c94f816cb074fef7efcb6280
Documentation: https://linear.app/n8n/issue/API-386
Related ADRs: ADR-20261005-replay-the-first-public-api-response-for-an-idempotency-key
packages/cli/src/__tests__/workflow-execute-additional-data.test.ts
+98 -1
View File
@@ -1817,6 +1817,102 @@ describe('WorkflowExecuteAdditionalData', () => {
);
});
describe('eval runs', () => {
const additionalData = mock<IWorkflowExecuteAdditionalData>({
userId: 'user-1',
projectId: 'project-1',
workflowId: 'workflow-1',
});
it('runs the draft of a stored agent with the eval hook', async () => {
const prepareForEval = vi.fn();
await executeAgent(
{ agentId: AGENT_ID },
MESSAGE,
EXEC_ID,
THREAD_ID,
additionalData,
'evaluation',
undefined,
undefined,
undefined,
prepareForEval,
);
expect(agentWorkflowExecutionService.executeForWorkflow).toHaveBeenCalledWith(
AGENT_ID,
MESSAGE,
EXEC_ID,
PROJECT_THREAD_ID,
'project-1',
'user-1',
true,
undefined,
undefined,
executionSandboxScope,
{ streamObserver: undefined, prepareForEval },
);
});
it('runs an inline agent as a test run with the eval hook', async () => {
const prepareForEval = vi.fn();
const inlineAgent = {
config: { name: 'Inline', model: 'openai/gpt-5', credential: 'c1', instructions: '' },
};
await executeAgent(
{ inlineAgent },
MESSAGE,
EXEC_ID,
THREAD_ID,
additionalData,
'evaluation',
undefined,
undefined,
undefined,
prepareForEval,
);
expect(agentWorkflowExecutionService.executeInlineForWorkflow).toHaveBeenCalledWith(
inlineAgent,
MESSAGE,
EXEC_ID,
PROJECT_THREAD_ID,
'project-1',
'user-1',
'test',
undefined,
undefined,
{ streamObserver: undefined, prepareForEval },
);
});
it('keeps the published version for evaluation runs without the eval hook', async () => {
await executeAgent(
{ agentId: AGENT_ID },
MESSAGE,
EXEC_ID,
THREAD_ID,
additionalData,
'evaluation',
);
expect(agentWorkflowExecutionService.executeForWorkflow).toHaveBeenCalledWith(
AGENT_ID,
MESSAGE,
EXEC_ID,
PROJECT_THREAD_ID,
'project-1',
'user-1',
false,
undefined,
undefined,
executionSandboxScope,
);
});
});
it('uses projectId from additionalData when present', async () => {
const additionalData = mock<IWorkflowExecuteAdditionalData>({
userId: 'user-1',
@@ -2067,7 +2163,8 @@ describe('WorkflowExecuteAdditionalData', () => {
expect(agentWorkflowExecutionService.executeForWorkflow.mock.calls[0]?.[9]).toEqual(
executionSandboxScope,
);
const streamObserver = agentWorkflowExecutionService.executeForWorkflow.mock.calls[0]?.[10];
const streamObserver =
agentWorkflowExecutionService.executeForWorkflow.mock.calls[0]?.[10]?.streamObserver;
expect(streamObserver).toEqual(expect.any(Function));
await streamObserver?.({ type: 'response-delta', delta: 'hello' });
packages/cli/src/credentials/__tests__/credential-options.controller.test.ts
+19
View File
@@ -0,0 +1,19 @@
import { ControllerRegistryMetadata, type Controller } from '@n8n/decorators';
import { Container } from '@n8n/di';
import { CredentialOptionsController } from '../credential-options.controller';
describe('Credential resource locator route access', () => {
const { routes } = Container.get(ControllerRegistryMetadata).getControllerMetadata(
CredentialOptionsController as Controller,
);
it.each([
['lookupProjectDraft', 'credential:create', false],
['lookupInstanceDraft', 'credential:manageInstance', true],
['lookupStored', 'credential:read', false],
])('%s requires %s', (handler, scope, globalOnly) => {
expect(routes.get(handler)?.accessScope).toEqual({ scope, globalOnly });
expect(routes.get(handler)?.skipAuth).not.toBe(true);
});
});
packages/cli/src/credentials/__tests__/credential-options.service.test.ts
+215
View File
@@ -0,0 +1,215 @@
import type { CredentialOptionsRequestDto } from '@n8n/api-types';
import type { CredentialsFinderService } from '@n8n/backend-services';
import { GLOBAL_MEMBER_ROLE, GLOBAL_OWNER_ROLE, type CredentialsEntity, type User } from '@n8n/db';
import type {
ICredentialType,
IWorkflowExecuteAdditionalData,
ProjectSharingData,
} from 'n8n-workflow';
import { mock } from 'vitest-mock-extended';
import type { CredentialTypes } from '@/credential-types';
import type { CredentialsHelper } from '@/credentials-helper';
import type { ExternalSecretsConfig } from '@/modules/external-secrets.ee/external-secrets.config';
import type { SecretsProviderAccessCheckService } from '@/modules/external-secrets.ee/secret-provider-access-check.service.ee';
import { userHasScopes } from '@/permissions.ee/check-access';
import { getBase } from '@/workflow-execute-additional-data';
import { CredentialOptionsService } from '../credential-options.service';
import type { CredentialsService } from '../credentials.service';
vi.mock('@/permissions.ee/check-access', () => ({ userHasScopes: vi.fn() }));
vi.mock('@/workflow-execute-additional-data', () => ({ getBase: vi.fn() }));
describe('CredentialOptionsService', () => {
const credentialTypes = mock<CredentialTypes>();
const helper = mock<CredentialsHelper>();
const credentials = mock<CredentialsService>();
const finder = mock<CredentialsFinderService>();
const externalSecretsConfig = mock<ExternalSecretsConfig>();
const secretProviderAccess = mock<SecretsProviderAccessCheckService>();
const service = new CredentialOptionsService(
credentialTypes,
helper,
credentials,
finder,
externalSecretsConfig,
secretProviderAccess,
);
const user = mock<User>({ id: 'user-id', role: GLOBAL_OWNER_ROLE });
const stored = mock<CredentialsEntity>({
id: 'saved-id',
name: 'Vertex',
type: 'googleVertexAiApi',
isManaged: false,
});
const loader = vi.fn();
const homeProject = mock<ProjectSharingData>({ id: 'owning-project' });
const type: ICredentialType = {
name: 'googleVertexAiApi',
displayName: 'Google Vertex AI',
properties: [
{
name: 'project',
displayName: 'Project',
type: 'options',
options: [{ name: 'Custom', value: '__custom__' }],
default: '__custom__',
typeOptions: { loadOptionsMethod: 'projects' },
},
],
methods: { loadOptions: { projects: loader } },
};
const request: CredentialOptionsRequestDto = {
type: type.name,
propertyName: 'project',
data: { email: '[email protected]', privateKey: 'draft-key' },
paginationToken: 'page-two',
};
beforeEach(() => {
vi.resetAllMocks();
vi.mocked(userHasScopes).mockResolvedValue(true);
vi.mocked(getBase).mockResolvedValue(mock<IWorkflowExecuteAdditionalData>());
credentialTypes.getByName.mockReturnValue(type);
helper.getCredentialsProperties.mockReturnValue(type.properties);
helper.applyDefaultsAndOverwrites.mockImplementation(async (_, data) => data);
finder.findCredentialForUser.mockResolvedValue(stored);
credentials.prepareCredentialsForUse.mockResolvedValue({
credentials: {
id: stored.id,
name: stored.name,
type: stored.type,
data: request.data,
homeProject,
},
storedData: request.data,
});
loader.mockResolvedValue({ results: [{ name: 'Target project', value: 'target-project' }] });
});
it('loads an unsaved draft with its n8n project context and prepared values', async () => {
helper.applyDefaultsAndOverwrites.mockResolvedValue({
...request.data,
privateKey: 'overwritten-key',
region: 'global',
});
await expect(service.lookupDraft(user, request, 'n8n-project')).resolves.toEqual({
results: [{ name: 'Target project', value: 'target-project' }],
});
expect(getBase).toHaveBeenCalledWith({ userId: user.id, projectId: 'n8n-project' });
expect(loader).toHaveBeenCalledWith(
{ email: '[email protected]', privateKey: 'overwritten-key', region: 'global' },
undefined,
'page-two',
);
expect(credentials.enforceCredentialUse).toHaveBeenCalledWith(
{ id: '', type: type.name },
{ kind: 'user', user },
'n8n-project',
);
});
it('loads a stored credential with the restored data and server-owned project', async () => {
await service.lookupStored(user, stored.id, { ...request, data: { privateKey: '***' } });
expect(getBase).toHaveBeenCalledWith({ userId: user.id, projectId: 'owning-project' });
expect(loader).toHaveBeenCalledWith(request.data, undefined, 'page-two');
});
it('uses stored values for managed credentials', async () => {
finder.findCredentialForUser.mockResolvedValue({ ...stored, isManaged: true });
await service.lookupStored(user, stored.id, request);
expect(credentials.prepareCredentialsForUse).toHaveBeenCalledWith({
storedCredential: { ...stored, isManaged: true },
user,
credentialsToUse: undefined,
});
});
it('rejects drafts outside the allowed project before a provider call', async () => {
vi.mocked(userHasScopes).mockResolvedValue(false);
await expect(service.lookupDraft(user, request, 'other-project')).rejects.toThrow();
expect(loader).not.toHaveBeenCalled();
});
it.each([false, true])(
'enforces instance credential access for an owner: %s',
async (isOwner) => {
const actor = mock<User>({
id: 'actor',
role: isOwner ? GLOBAL_OWNER_ROLE : GLOBAL_MEMBER_ROLE,
});
const result = service.lookupDraft(actor, request, null);
if (isOwner) {
await expect(result).resolves.toMatchObject({ results: [{ value: 'target-project' }] });
expect(credentials.validateInstanceCredentialData).toHaveBeenCalledWith(request.data);
} else {
await expect(result).rejects.toThrow();
expect(loader).not.toHaveBeenCalled();
}
},
);
it.each(['missing', 'wrong-type', 'blocked'])(
'rejects a %s stored credential before a provider call',
async (reason) => {
if (reason === 'missing') finder.findCredentialForUser.mockResolvedValue(null);
if (reason === 'wrong-type')
finder.findCredentialForUser.mockResolvedValue({ ...stored, type: 'googleApi' });
if (reason === 'blocked')
credentials.prepareCredentialsForUse.mockRejectedValue(
new Error('Credential use is blocked'),
);
await expect(service.lookupStored(user, stored.id, request)).rejects.toThrow();
expect(loader).not.toHaveBeenCalled();
},
);
it.each([false, true])(
'checks secret-reference permissions when stored values change: %s',
async (isChanged) => {
vi.mocked(userHasScopes).mockResolvedValue(false);
secretProviderAccess.isProviderAvailableInProject.mockImplementation(
async (providerKey, projectId) => providerKey === 'vault' && projectId === homeProject.id,
);
const storedData = { ...request.data, privateKey: '={{ $secrets.vault.existingKey }}' };
const data = {
...storedData,
privateKey: isChanged ? '={{ $secrets.vault.newKey }}' : storedData.privateKey,
};
credentials.prepareCredentialsForUse.mockResolvedValue({
credentials: {
id: stored.id,
name: stored.name,
type: stored.type,
homeProject,
data,
},
storedData,
});
const result = service.lookupStored(user, stored.id, { ...request, data });
if (isChanged) {
await expect(result).rejects.toThrow('permissions');
expect(loader).not.toHaveBeenCalled();
} else {
await expect(result).resolves.toEqual({
results: [{ name: 'Target project', value: 'target-project' }],
});
}
},
);
it('rejects fields without a declared list callback', async () => {
await expect(
service.lookupDraft(user, { ...request, propertyName: 'privateKey' }, 'n8n-project'),
).rejects.toThrow('does not support');
expect(loader).not.toHaveBeenCalled();
});
it('returns a safe error when discovery fails', async () => {
loader.mockRejectedValue(new Error('Provider failed with private key draft-key'));
await expect(service.lookupDraft(user, request, 'n8n-project')).rejects.toThrow(
'Could not load the list. Try again or enter an ID.',
);
});
});
packages/cli/src/credentials/__tests__/credentials.service.test.ts
+30
View File
@@ -2258,6 +2258,36 @@ describe('CredentialsService', () => {
});
});
describe('prepareCredentialsForUse', () => {
it('restores a masked key and preserves the original data for secret permission checks', async () => {
const storedCredential = mock<CredentialsEntity>({
id: 'credential-id',
name: 'Vertex',
type: 'googleVertexAiApi',
});
const storedData = { email: '[email protected]', privateKey: 'stored-key' };
vi.spyOn(service, 'decrypt').mockResolvedValue(storedData);
vi.spyOn(service, 'replaceCredentialContentsForSharee').mockResolvedValue(undefined);
vi.spyOn(service, 'getCredentialTypeProperties').mockReturnValue([]);
const result = await service.prepareCredentialsForUse({
storedCredential,
user: ownerUser,
credentialsToUse: {
id: storedCredential.id,
name: storedCredential.name,
type: storedCredential.type,
data: { email: '[email protected]', privateKey: CREDENTIAL_BLANKING_VALUE },
},
});
expect(result.credentials.data).toEqual({
email: '[email protected]',
privateKey: 'stored-key',
});
expect(result.storedData).toEqual(storedData);
});
});
describe('testWithCredentials', () => {
it('refuses a stored credential of a blocked type before the tester runs', async () => {
credentialsFinderService.findCredentialForUser.mockResolvedValue(
packages/cli/src/credentials/credential-options.controller.ts
+42
View File
@@ -0,0 +1,42 @@
import { CredentialOptionsRequestDto } from '@n8n/api-types';
import { AuthenticatedRequest } from '@n8n/db';
import { Body, GlobalScope, Param, Post, ProjectScope, RestController } from '@n8n/decorators';
import { CredentialOptionsService } from './credential-options.service';
@RestController('/credentials')
export class CredentialOptionsController {
constructor(private readonly credentialOptionsService: CredentialOptionsService) {}
@Post('/options/projects/:projectId')
@ProjectScope('credential:create')
async lookupProjectDraft(
req: AuthenticatedRequest,
_res: unknown,
@Param('projectId') projectId: string,
@Body request: CredentialOptionsRequestDto,
) {
return await this.credentialOptionsService.lookupDraft(req.user, request, projectId);
}
@Post('/options/instance')
@GlobalScope('credential:manageInstance')
async lookupInstanceDraft(
req: AuthenticatedRequest,
_res: unknown,
@Body request: CredentialOptionsRequestDto,
) {
return await this.credentialOptionsService.lookupDraft(req.user, request, null);
}
@Post('/:credentialId/options')
@ProjectScope('credential:read')
async lookupStored(
req: AuthenticatedRequest,
_res: unknown,
@Param('credentialId') credentialId: string,
@Body request: CredentialOptionsRequestDto,
) {
return await this.credentialOptionsService.lookupStored(req.user, credentialId, request);
}
}
packages/cli/src/credentials/credential-options.service.ts
+132
View File
@@ -0,0 +1,132 @@
import type { CredentialOptionsRequestDto } from '@n8n/api-types';
import { CredentialsFinderService } from '@n8n/backend-services';
import type { User } from '@n8n/db';
import { Service } from '@n8n/di';
import { BadRequestError, ForbiddenError } from '@n8n/errors';
import { hasGlobalScope } from '@n8n/permissions';
import type { ICredentialDataDecryptedObject } from 'n8n-workflow';
import { CredentialTypes } from '@/credential-types';
import { CredentialsHelper } from '@/credentials-helper';
import { ExternalSecretsConfig } from '@/modules/external-secrets.ee/external-secrets.config';
import { SecretsProviderAccessCheckService } from '@/modules/external-secrets.ee/secret-provider-access-check.service.ee';
import { userHasScopes } from '@/permissions.ee/check-access';
import { getBase } from '@/workflow-execute-additional-data';
import { CredentialsService } from './credentials.service';
import {
validateAccessToReferencedSecretProviders,
validateExternalSecretsPermissions,
} from './validation';
@Service()
export class CredentialOptionsService {
constructor(
private readonly credentialTypes: CredentialTypes,
private readonly credentialsHelper: CredentialsHelper,
private readonly credentialsService: CredentialsService,
private readonly credentialsFinder: CredentialsFinderService,
private readonly externalSecretsConfig: ExternalSecretsConfig,
private readonly secretProviderAccess: SecretsProviderAccessCheckService,
) {}
async lookupDraft(user: User, request: CredentialOptionsRequestDto, projectId: string | null) {
const allowed =
projectId === null
? hasGlobalScope(user, 'credential:manageInstance')
: await userHasScopes(user, ['credential:create'], false, { projectId });
if (!allowed) throw new ForbiddenError();
await this.credentialsService.enforceCredentialUse(
{ id: '', type: request.type },
{ kind: 'user', user },
projectId,
);
await this.validateExternalSecrets(user, request.data, projectId);
return await this.load(user, request, request.data, projectId);
}
async lookupStored(user: User, credentialId: string, request: CredentialOptionsRequestDto) {
const storedCredential = await this.credentialsFinder.findCredentialForUser(
credentialId,
user,
['credential:read'],
{ includeInstanceCredentials: true },
);
if (!storedCredential) throw new ForbiddenError();
if (storedCredential.type !== request.type) {
throw new BadRequestError('The credential type does not match the stored credential');
}
const { credentials, storedData } = await this.credentialsService.prepareCredentialsForUse({
storedCredential,
user,
credentialsToUse: storedCredential.isManaged
? undefined
: { id: credentialId, name: storedCredential.name, type: request.type, data: request.data },
});
const projectId = credentials.homeProject?.id ?? null;
const data = credentials.data ?? {};
await this.validateExternalSecrets(user, data, projectId, storedData);
return await this.load(user, request, data, projectId);
}
private async validateExternalSecrets(
user: User,
data: ICredentialDataDecryptedObject,
projectId: string | null,
storedData?: ICredentialDataDecryptedObject,
) {
if (projectId === null) {
this.credentialsService.validateInstanceCredentialData(data);
return;
}
await validateExternalSecretsPermissions({
user,
projectId,
dataToSave: data,
decryptedExistingData: storedData,
});
if (this.externalSecretsConfig.externalSecretsForProjects) {
await validateAccessToReferencedSecretProviders(
projectId,
data,
this.secretProviderAccess,
'create',
);
}
}
private async load(
user: User,
request: CredentialOptionsRequestDto,
data: ICredentialDataDecryptedObject,
projectId: string | null,
) {
const credentialType = this.credentialTypes.getByName(request.type);
const property = this.credentialsHelper
.getCredentialsProperties(request.type)
.find((field) => field.name === request.propertyName);
const methodName = property?.typeOptions?.loadOptionsMethod;
const loader =
methodName && Object.hasOwn(credentialType.methods?.loadOptions ?? {}, methodName)
? credentialType.methods?.loadOptions?.[methodName]
: undefined;
if (property?.type !== 'options' || !loader) {
throw new BadRequestError('This credential field does not support loading a list.');
}
try {
const additionalData = await getBase({ userId: user.id, projectId: projectId ?? undefined });
const resolvedData = await this.credentialsHelper.applyDefaultsAndOverwrites(
additionalData,
data,
request.type,
'internal',
undefined,
undefined,
);
return await loader(resolvedData, request.filter, request.paginationToken);
} catch {
// Provider errors can contain credential data.
throw new BadRequestError('Could not load the list. Try again or enter an ID.');
}
}
}
packages/cli/src/credentials/credentials.service.ts
+17 -17
View File
@@ -1047,7 +1047,7 @@ export class CredentialsService {
return await this.decrypt(credential, true);
}
private async enforceCredentialUse(
async enforceCredentialUse(
credential: Pick<ICredentialsDecrypted, 'id' | 'type'>,
actor: PolicyActor,
projectId: string | null,
@@ -1538,7 +1538,7 @@ export class CredentialsService {
throw new CredentialNotFoundError(credentialId);
}
const credentials = await this.prepareCredentialsForTest({ storedCredential, user });
const { credentials } = await this.prepareCredentialsForUse({ storedCredential, user });
return await this.test(user.id, credentials);
}
@@ -1560,10 +1560,10 @@ export class CredentialsService {
throw new CredentialNotFoundError(credentials.id);
}
const mergedCredentials = await this.prepareCredentialsForTest({
const { credentials: mergedCredentials } = await this.prepareCredentialsForUse({
storedCredential,
user,
credentialsToTest: storedCredential.isManaged ? undefined : credentials,
credentialsToUse: storedCredential.isManaged ? undefined : credentials,
});
return await this.test(user.id, mergedCredentials);
@@ -2438,24 +2438,24 @@ export class CredentialsService {
}
/**
* Build credentials payload ready to pass to credential testing.
* Prepare stored credentials for a provider request.
*
* - If `credentialsToTest` is not provided, uses stored decrypted credential data.
* - If `credentialsToTest` is provided, normalizes it for testing:
* - If `credentialsToUse` is not provided, uses stored decrypted credential data.
* - If `credentialsToUse` is provided, normalizes the draft:
* - fills payload data for sharees when needed
* - restores redacted values from stored decrypted data
*/
private async prepareCredentialsForTest({
async prepareCredentialsForUse({
storedCredential,
user,
credentialsToTest,
credentialsToUse,
}: {
storedCredential: CredentialsEntity;
user: User;
credentialsToTest?: ICredentialsDecrypted;
}): Promise<ICredentialsDecrypted> {
// The tester picks its test from the posted type, so it must match the stored secrets.
if (credentialsToTest && credentialsToTest.type !== storedCredential.type) {
credentialsToUse?: ICredentialsDecrypted;
}): Promise<{ credentials: ICredentialsDecrypted; storedData: ICredentialDataDecryptedObject }> {
// Provider callbacks must use the type that owns the stored secrets.
if (credentialsToUse && credentialsToUse.type !== storedCredential.type) {
throw new BadRequestError('The credential type does not match the stored credential');
}
@@ -2466,8 +2466,8 @@ export class CredentialsService {
{ kind: 'user', user },
owningProject?.id ?? null,
);
const mergedCredentials: ICredentialsDecrypted = credentialsToTest
? deepCopy(credentialsToTest)
const mergedCredentials: ICredentialsDecrypted = credentialsToUse
? deepCopy(credentialsToUse)
: {
id: storedCredential.id,
name: storedCredential.name,
@@ -2488,7 +2488,7 @@ export class CredentialsService {
};
}
if (credentialsToTest) {
if (credentialsToUse) {
await this.replaceCredentialContentsForSharee(
user,
storedCredential,
@@ -2505,6 +2505,6 @@ export class CredentialsService {
}
}
return mergedCredentials;
return { credentials: mergedCredentials, storedData: decryptedData };
}
}
packages/cli/src/events/__tests__/telemetry-event-relay.test.ts
+36
View File
@@ -802,6 +802,7 @@ describe('TelemetryEventRelay', () => {
delegate_rule_count: 1,
name_selector_count: 2,
package_selector_count: 1,
extends_selector_count: 0,
evaluated_type_count: 3,
blocked_type_count: 2,
allowed_type_count: 1,
@@ -849,6 +850,41 @@ describe('TelemetryEventRelay', () => {
);
});
it('should count a credential type built on the base an extends rule names as blocked', () => {
Object.defineProperty(loadNodesAndCredentials, 'knownCredentials', {
configurable: true,
value: {
oAuth2Api: {},
googleOAuth2Api: { extends: ['oAuth2Api'] },
googleSheetsOAuth2Api: { extends: ['googleOAuth2Api'] },
slackApi: {},
},
});
eventService.emit('node-type-policy-saved', {
updatedBy: 'user123',
kind: 'credential-types',
projectId: null,
scopeId: 'scope-1',
before: null,
after: { defaultAction: 'allow', version: 1 },
rulesBefore: null,
rulesAfter: [
{ id: 'rule-1', action: 'deny', selector: { kind: 'extends', value: 'oAuth2Api' } },
],
warningCount: 0,
});
expect(telemetry.track).toHaveBeenCalledWith(
TELEMETRY_EVENT.TYPE_AVAILABILITY_POLICIES.USER_SAVED_TYPE_AVAILABILITY_POLICY,
expect.objectContaining({
extends_selector_count: 1,
blocked_types: ['oAuth2Api', 'googleOAuth2Api', 'googleSheetsOAuth2Api'],
allowed_types: ['slackApi'],
}),
);
});
it('should expand a credential type package rule using the credential loader package, not a dotted-type prefix', () => {
const credentialPackageDenyRule: PolicyRule = {
id: 'rule-1',
packages/cli/src/events/relays/telemetry.event-relay.ts
+2 -1
View File
@@ -160,6 +160,7 @@ function countSelectorKinds(rules: readonly PolicyRule[]) {
return {
name_selector_count: rules.filter((rule) => rule.selector.kind === 'name').length,
package_selector_count: rules.filter((rule) => rule.selector.kind === 'package').length,
extends_selector_count: rules.filter((rule) => rule.selector.kind === 'extends').length,
};
}
@@ -586,7 +587,7 @@ export class TelemetryEventRelay extends EventRelay {
kind === CREDENTIAL_TYPES_KIND
? Object.keys(this.loadNodesAndCredentials.knownCredentials)
: Object.keys(this.nodeTypes.getKnownTypes());
const types = typeNames.map(policedTypeFor(kind, this.nodeTypes));
const types = typeNames.map(policedTypeFor(kind, this.nodeTypes, this.loadNodesAndCredentials));
const resolvePackage = packageResolverFor(kind, this.loadNodesAndCredentials);
this.telemetry.track(
packages/cli/src/execution-lifecycle/__tests__/execution-lifecycle-hooks.test.ts
+62
View File
@@ -6,6 +6,7 @@ import { ExecutionsConfig } from '@n8n/config';
import type { Project, User } from '@n8n/db';
import { ExecutionRepository, UserRepository } from '@n8n/db';
import { LifecycleMetadata } from '@n8n/decorators';
import type { WorkflowExecuteAfterContext } from '@n8n/decorators';
import { Container } from '@n8n/di';
import { stringify } from 'flatted';
import {
@@ -536,6 +537,67 @@ describe('Execution Lifecycle Hooks', () => {
Container.set(LifecycleMetadata, original);
}
});
describe('executedInThisProcess', () => {
class RecordingHandler {
contexts: WorkflowExecuteAfterContext[] = [];
async onWorkflowExecuteAfter(ctx: WorkflowExecuteAfterContext) {
this.contexts.push(ctx);
}
}
const recorder = new RecordingHandler();
const runModuleHandler = async (hooks: ExecutionLifecycleHooks) => {
await hooks.runHook('workflowExecuteAfter', [successfulRun, {}]);
return recorder.contexts.at(-1);
};
let original: LifecycleMetadata;
beforeEach(() => {
recorder.contexts = [];
const recording = new LifecycleMetadata();
recording.register({
handlerClass: RecordingHandler as unknown as HandlerClass,
methodName: 'onWorkflowExecuteAfter',
eventName: 'workflowExecuteAfter',
});
Container.set(RecordingHandler, recorder);
original = Container.get(LifecycleMetadata);
Container.set(LifecycleMetadata, recording);
});
afterEach(() => {
Container.set(LifecycleMetadata, original);
});
it('should be false for the hooks of a process that only enqueued the run', async () => {
const hooks = getLifecycleHooksForScalingMain(
{ executionMode: 'manual', workflowData },
executionId,
);
expect(await runModuleHandler(hooks)).toMatchObject({ executedInThisProcess: false });
});
it('should be true for the hooks of a process that runs the execution', async () => {
const workerHooks = getLifecycleHooksForScalingWorker(
{ executionMode: 'manual', workflowData },
executionId,
);
const regularHooks = getLifecycleHooksForRegularMain(
{ executionMode: 'manual', workflowData },
executionId,
);
expect(await runModuleHandler(workerHooks)).toMatchObject({ executedInThisProcess: true });
expect(await runModuleHandler(regularHooks)).toMatchObject({
executedInThisProcess: true,
});
});
});
});
describe('getLifecycleHooksForRegularMain', () => {
packages/cli/src/execution-lifecycle/execution-lifecycle-hooks.ts
+8 -2
View File
@@ -53,8 +53,13 @@ class ModulesHooksRegistry {
/**
* `source` is not carried on `ExecutionLifecycleHooks`, so it is passed in
* here to let module handlers tell agent-initiated runs from user ones.
* `executedInThisProcess` is false for the hooks of a process that only enqueued the run.
*/
addHooks(hooks: ExecutionLifecycleHooks, source?: IWorkflowExecutionDataProcess['source']) {
addHooks(
hooks: ExecutionLifecycleHooks,
source?: IWorkflowExecutionDataProcess['source'],
{ executedInThisProcess = true }: { executedInThisProcess?: boolean } = {},
) {
const handlers = Container.get(LifecycleMetadata).getHandlers();
for (const { handlerClass, methodName, eventName } of handlers) {
@@ -72,6 +77,7 @@ class ModulesHooksRegistry {
executionId: this.executionId,
retryOf: this.retryOf,
source,
executedInThisProcess,
};
return await instance[methodName].call(instance, context);
@@ -977,7 +983,7 @@ export function getLifecycleHooksForScalingMain(
hooks.handlers.nodeExecuteBefore = [];
hooks.handlers.nodeExecuteAfter = [];
Container.get(ModulesHooksRegistry).addHooks(hooks, source);
Container.get(ModulesHooksRegistry).addHooks(hooks, source, { executedInThisProcess: false });
return hooks;
}
packages/cli/src/modules/agents/__tests__/agent-workflow-execution.service.test.ts
+105 -3
View File
@@ -20,6 +20,10 @@ import type { AgentExecutionService } from '../agent-execution.service';
import type { AgentMessageQueueService } from '../agent-message-queue.service';
import type { AgentChatExecutionService } from '../agent-chat-execution.service';
import type { AgentRunTracingService } from '../agent-run-tracing.service';
import type {
AgentRuntimeInstrumentation,
PrepareWorkflowAgentForEval,
} from '../agent-runtime-instrumentation';
import type { AgentRuntimeReconstructionService } from '../agent-runtime-reconstruction.service';
import { AgentTurnExecutionService } from '../agent-turn-execution.service';
import type { AgentToolApprovalService } from '../agent-tool-approval.service';
@@ -920,7 +924,7 @@ describe('AgentWorkflowExecutionService', () => {
'production',
undefined,
workflowContext,
streamObserver,
{ streamObserver },
);
// No entity lookup — the embedded config is the source of truth.
@@ -1302,7 +1306,7 @@ describe('AgentWorkflowExecutionService', () => {
undefined,
undefined,
undefined,
streamObserver,
{ streamObserver },
);
await expect(execution).rejects.toMatchObject({
message: 'reader failed while consuming stream',
@@ -1340,7 +1344,7 @@ describe('AgentWorkflowExecutionService', () => {
undefined,
undefined,
undefined,
streamObserver,
{ streamObserver },
);
await expect(execution).rejects.toMatchObject({
@@ -1458,4 +1462,102 @@ describe('AgentWorkflowExecutionService', () => {
}),
);
});
describe('eval runs', () => {
const evalConfig: AgentJsonConfig = { ...schema, instructions: 'Help users (eval copy)' };
const integrations = [{ type: 'slack', credentialId: 'slack-1' }] as Agent['integrations'];
function makePrepareForEval() {
const instrumentation: AgentRuntimeInstrumentation = {
configureToolAdditionalData: vi.fn(),
};
const prepareForEval = vi
.fn<PrepareWorkflowAgentForEval>()
.mockResolvedValue({ config: evalConfig, instrumentation });
return { prepareForEval, instrumentation };
}
it('compiles a stored agent from the eval config and seams, without chat integrations', async () => {
const { service, agentRepository, reconstructionService, executionService } = makeService();
agentRepository.findByIdAndProjectId.mockResolvedValue(makeAgent({ integrations }));
reconstructionService.reconstructFromAgentEntity.mockResolvedValue(makeRuntime());
const { prepareForEval, instrumentation } = makePrepareForEval();
await service.executeForWorkflow(
agentId,
'hello',
'execution-1',
'thread-1',
projectId,
userId,
true,
undefined,
undefined,
undefined,
{ prepareForEval },
);
expect(prepareForEval).toHaveBeenCalledWith(schema, { hasChatIntegrations: true });
const [entity, , , , , passedInstrumentation] =
reconstructionService.reconstructFromAgentEntity.mock.calls[0];
expect(entity.schema).toEqual(evalConfig);
expect(entity.integrations).toEqual([]);
expect(passedInstrumentation).toBe(instrumentation);
// Run telemetry describes the config that ran, not the stored one.
expect(executionService.startExecutionRecording).toHaveBeenCalledWith(
expect.objectContaining({
telemetry: expect.objectContaining({
configuration: expect.objectContaining({ channels: [] }),
}),
}),
expect.anything(),
);
});
it('compiles an inline agent from the eval config and seams', async () => {
const { service, reconstructionService } = makeService();
reconstructionService.reconstructFromResolvedSource.mockResolvedValue(makeRuntime());
const { prepareForEval, instrumentation } = makePrepareForEval();
await service.executeInlineForWorkflow(
{
config: {
name: 'Inline Agent',
model: 'openai/gpt-5',
credential: 'cred-1',
instructions: 'Help users',
},
},
'hello',
'execution-1',
'thread-1',
projectId,
userId,
'test',
undefined,
undefined,
{ prepareForEval },
);
expect(prepareForEval).toHaveBeenCalledWith(
expect.objectContaining({ name: 'Inline Agent' }),
);
expect(reconstructionService.reconstructFromResolvedSource).toHaveBeenCalledWith(
expect.objectContaining({ config: evalConfig, instrumentation }),
);
});
it('passes no seams and keeps chat integrations outside eval runs', async () => {
const { service, agentRepository, reconstructionService } = makeService();
agentRepository.findByIdAndProjectId.mockResolvedValue(makeAgent({ integrations }));
reconstructionService.reconstructFromAgentEntity.mockResolvedValue(makeRuntime());
await service.executeForWorkflow(agentId, 'hello', 'execution-1', 'thread-1', projectId);
const [entity, , , , , passedInstrumentation] =
reconstructionService.reconstructFromAgentEntity.mock.calls[0];
expect(entity.integrations).toEqual(integrations);
expect(passedInstrumentation).toBeUndefined();
});
});
});
packages/cli/src/modules/agents/agent-runtime-instrumentation.ts
+9
View File
@@ -65,3 +65,12 @@ export interface AgentRuntimeInstrumentation {
export type InstrumentToolAdditionalData = NonNullable<
AgentRuntimeInstrumentation['configureToolAdditionalData']
>;
/**
* Set by the Instance AI workflow eval when a workflow calls an agent. Returns
* the config to run (features only removed) and the seams that fake its I/O.
*/
export type PrepareWorkflowAgentForEval = (
config: AgentJsonConfig,
context?: { hasChatIntegrations?: boolean },
) => Promise<{ config: AgentJsonConfig; instrumentation: AgentRuntimeInstrumentation }>;
packages/cli/src/modules/agents/agent-workflow-execution.service.ts
+35 -4
View File
@@ -33,6 +33,10 @@ import { Telemetry } from '@/telemetry';
import type { StartExecutionParams } from './agent-execution.service';
import { AgentRunTracingService } from './agent-run-tracing.service';
import type {
AgentRuntimeInstrumentation,
PrepareWorkflowAgentForEval,
} from './agent-runtime-instrumentation';
import { AgentRuntimeReconstructionService } from './agent-runtime-reconstruction.service';
import { AgentsSettingsService } from './agents-settings.service';
import {
@@ -123,10 +127,17 @@ interface WorkflowExecutionContext {
streamObserver?: WorkflowAgentStreamObserver;
}
/** Optional hooks for one workflow agent run. */
export interface WorkflowAgentRunOptions {
streamObserver?: WorkflowAgentStreamObserver;
prepareForEval?: PrepareWorkflowAgentForEval;
}
interface StoredWorkflowExecutionContext extends WorkflowExecutionContext {
agentId: string;
useDraftVersion?: boolean;
sandboxScope?: WorkflowSandboxScope;
prepareForEval?: PrepareWorkflowAgentForEval;
}
function getFinalWorkflowResponse(messageRecord: MessageRecord): string {
@@ -240,6 +251,7 @@ export class AgentWorkflowExecutionService {
outputSchema?: JSONSchema7,
extraTools?: BuiltTool[],
sandboxPrincipalHash?: AgentSandboxPrincipalHash,
instrumentation?: AgentRuntimeInstrumentation,
): Promise<{ ok: boolean; agent?: BuiltAgent; budget?: BudgetGuardrailConfig; error?: string }> {
if (!agentEntity.schema) {
return { ok: false, error: 'Agent has no JSON config. Create a config first.' };
@@ -259,7 +271,7 @@ export class AgentWorkflowExecutionService {
runType,
undefined,
undefined,
undefined,
instrumentation,
'manual',
sandboxPrincipalHash,
// A workflow execution cannot resume a suspended run — it throws
@@ -294,6 +306,7 @@ export class AgentWorkflowExecutionService {
runType: AgentRunTelemetryType,
outputSchema?: JSONSchema7,
extraTools?: BuiltTool[],
instrumentation?: AgentRuntimeInstrumentation,
): Promise<{ ok: boolean; agent?: BuiltAgent; budget?: BudgetGuardrailConfig; error?: string }> {
try {
const reconstructed =
@@ -307,6 +320,7 @@ export class AgentWorkflowExecutionService {
skills,
runtimeProfile: 'inline',
runType,
...(instrumentation ? { instrumentation } : {}),
});
const applied = this.applyPerCallAgentExtras(reconstructed.agent, outputSchema, extraTools);
if (!applied.ok) return applied;
@@ -623,7 +637,7 @@ export class AgentWorkflowExecutionService {
outputSchema?: JSONSchema7,
workflowContext?: ExecuteAgentWorkflowContext,
sandboxScope?: WorkflowSandboxScope,
streamObserver?: WorkflowAgentStreamObserver,
{ streamObserver, prepareForEval }: WorkflowAgentRunOptions = {},
): Promise<ExecuteAgentData> {
return await this.executeForWorkflowInternal({
agentId,
@@ -637,6 +651,7 @@ export class AgentWorkflowExecutionService {
workflowContext,
sandboxScope,
streamObserver,
prepareForEval,
});
}
@@ -676,10 +691,12 @@ export class AgentWorkflowExecutionService {
outputSchema,
workflowContext,
sandboxScope,
prepareForEval,
} = params;
// Keep the original intent if deletion happens during preparation.
const sessionMode = await this.turnExecutionService.getSessionMode(threadId);
const { agentData, credentialProvider } = await this.loadWorkflowAgent(params);
const instrumentation = await this.applyEvalSetup(agentData, prepareForEval);
const telemetryConfiguration = buildAgentConfigurationTelemetry(agentData);
const runType: AgentRunTelemetryType = useDraftVersion ? 'test' : 'production';
@@ -704,6 +721,7 @@ export class AgentWorkflowExecutionService {
outputSchema,
extraTools?.length ? extraTools : undefined,
sandboxScope?.principalHash,
instrumentation,
);
if (!compiled.ok || !compiled.agent) {
const error = new OperationalError(
@@ -737,6 +755,17 @@ export class AgentWorkflowExecutionService {
return { agentData, credentialProvider };
}
/** Chat integrations post through chat SDKs, which an eval cannot fake, so eval runs drop them. */
private async applyEvalSetup(agent: Agent, prepareForEval?: PrepareWorkflowAgentForEval) {
if (!prepareForEval || !agent.schema) return undefined;
const { config, instrumentation } = await prepareForEval(agent.schema, {
hasChatIntegrations: (agent.integrations ?? []).length > 0,
});
agent.schema = config;
agent.integrations = [];
return instrumentation;
}
private async streamCompiledWorkflowAgent(
agentInstance: BuiltAgent,
params: WorkflowExecutionContext,
@@ -797,7 +826,7 @@ export class AgentWorkflowExecutionService {
runType: AgentRunTelemetryType = 'production',
outputSchema?: JSONSchema7,
workflowContext?: ExecuteAgentWorkflowContext,
streamObserver?: WorkflowAgentStreamObserver,
{ streamObserver, prepareForEval }: WorkflowAgentRunOptions = {},
): Promise<ExecuteAgentData> {
await this.settingsService.assertEnabled();
const { runtimeConfig, skills, credentialProvider } = await this.prepareInlineRuntime(
@@ -805,6 +834,7 @@ export class AgentWorkflowExecutionService {
projectId,
workflowContext,
);
const prepared = prepareForEval ? await prepareForEval(runtimeConfig) : undefined;
// For telemetry/logging and memory-owner keying — never persisted, and
// stable enough to aggregate runs of the same node across executions.
const syntheticAgentId = `inline:${workflowContext?.workflowId ?? 'unknown'}:${
@@ -813,7 +843,7 @@ export class AgentWorkflowExecutionService {
const extraTools = this.buildWorkflowExtraTools(workflowContext);
const compiled = await this.compileIsolatedFromSource(
runtimeConfig,
prepared?.config ?? runtimeConfig,
skills,
syntheticAgentId,
projectId,
@@ -821,6 +851,7 @@ export class AgentWorkflowExecutionService {
runType,
outputSchema,
extraTools?.length ? extraTools : undefined,
prepared?.instrumentation,
);
if (!compiled.ok || !compiled.agent) {
throw new OperationalError(`Failed to compile agent: ${compiled.error ?? 'unknown error'}`);
packages/cli/src/modules/agents/integrations/CHANNEL_INTEGRATION_TESTS.md
+12
View File
@@ -34,6 +34,7 @@ Each platform adapter uses a different HTTP client, so the interception mechanis
| Telegram | native `fetch` | replace `globalThis.fetch` | `installFetchStub` (replay-test-helpers) |
| Slack | `@slack/web-api` (axios) | `nock` at the HTTP layer | inline in slack `replay-test-context` |
| Linear | `@linear/sdk` (GraphQL over fetch) | replace `globalThis.fetch` | `installFetchStub` |
| WhatsApp | native `fetch` | replace `globalThis.fetch` | `installWhatsAppApiStub` (whatsapp `replay-test-context`) |
| Teams | `@microsoft/teams.*` (axios) + `jwks-rsa` (node `https`) | `nock` at the HTTP layer | inline in teams `replay-test-context` |
Responses are answered from two sources, in order of preference:
@@ -73,6 +74,17 @@ so response stubs only need to be valid enough for the real adapter to proceed.
(e.g. a `Comment` needs `reactions: []`; an `AgentActivity` references `agentSession`/`sourceComment`
by id). Linear's "mention" is an **agent-session** event, not a comment — see the contract note
below.
- **WhatsApp** — inbound media (`fetchData`) triggers `downloadMedia`'s two plain GETs (a metadata
lookup, then the CDN url it returns), neither of which carries a JSON body — the stub branches on
`httpMethod === 'GET'` before the send-message logic, which assumes a POST. Testing that
media-download path meaningfully needs *real* bytes: `resolveInboundMimeType` sniffs magic bytes
before trusting a declared type, so `StubResponse` supports an optional `rawResponseBody` (raw
bytes, no `JSON.stringify`) alongside the normal JSON `responseBody` — see
`WHATSAPP_MEDIA_CONTENT` in whatsapp's `replay-test-context` for genuine minimal file content per
media kind. Attachments also need an `attachmentService` passed to `createReplayContextSetup`
(opt-in; `createMockAttachmentService()` echoes back a plausible stored record) — without it,
`AgentChatBridge`'s attachment pipeline silently no-ops for every platform, not just WhatsApp.
It is not the default because other platforms' replay tests do not stub their file downloads.
#### Teams setup, outside the adapter
packages/cli/src/modules/agents/integrations/__tests__/agent-chat-stream-consumer.test.ts
+86
View File
@@ -354,3 +354,89 @@ describe('AgentChatStreamConsumer — suspension routing', () => {
expect(handleSuspension).toHaveBeenCalledWith(SUSPENDED_CHUNK, thread, undefined);
});
});
const SESSION_BUDGET_MESSAGE =
'⚠️ This session reached its cost cap and stopped. Open the agent in n8n and increase the cap.';
const MONTHLY_BUDGET_MESSAGE =
'⚠️ This agent reached its monthly budget and stopped. Open the agent in n8n and increase the cap.';
describe('AgentChatStreamConsumer — budget stop', () => {
it('posts the session message once and clears the status when the run never starts', async () => {
const postErrorToThread = vi.fn().mockResolvedValue(undefined);
const consumer = makeConsumer(postErrorToThread);
const localThread = mock<Thread<unknown, unknown>>();
const clearBeforeResponse = vi.fn().mockResolvedValue(undefined);
await consumer.consume(
makeStream([
{ type: 'finish', finishReason: 'guardrail', guardrail: { code: 'budget.session' } },
]),
localThread,
{ statusHandle: { clearBeforeResponse } },
);
expect(localThread.post).toHaveBeenCalledTimes(1);
expect(localThread.post).toHaveBeenCalledWith({ markdown: SESSION_BUDGET_MESSAGE });
expect(clearBeforeResponse).toHaveBeenCalledTimes(1);
expect(postErrorToThread).not.toHaveBeenCalled();
});
it.each([
['streaming', false],
['buffered', true],
])('posts partial text and then the monthly message (%s)', async (_name, disableStreaming) => {
const { thread: localThread, streamedText, discrete } = makeStreamingThread();
await makeStreamingConsumer({ disableStreaming }).consume(
makeStream([
{ type: 'text-delta', id: 't-1', delta: 'Partial answer' },
{ type: 'finish', finishReason: 'guardrail', guardrail: { code: 'budget.monthly' } },
]),
localThread,
);
if (disableStreaming) {
expect(discrete).toEqual([
{ markdown: 'Partial answer' },
{ markdown: MONTHLY_BUDGET_MESSAGE },
]);
return;
}
expect(streamedText).toEqual(['Partial answer']);
expect(discrete).toEqual([{ markdown: MONTHLY_BUDGET_MESSAGE }]);
});
it('does not post a budget message when the run finishes normally', async () => {
const { thread: localThread, discrete } = makeStreamingThread();
await makeStreamingConsumer({ disableStreaming: true }).consume(
makeStream([
{ type: 'text-delta', id: 't-1', delta: 'All done' },
{ type: 'finish', finishReason: 'stop' },
]),
localThread,
);
expect(discrete).toEqual([{ markdown: 'All done' }]);
});
it('reports a rejected budget post once and does not post the budget line again', async () => {
const postErrorToThread = vi.fn().mockResolvedValue(undefined);
const consumer = makeConsumer(postErrorToThread);
const localThread = mock<Thread<unknown, unknown>>();
const deliveryError = new Error('delivery failed');
localThread.post.mockRejectedValue(deliveryError);
await consumer.consume(
makeStream([
{ type: 'finish', finishReason: 'guardrail', guardrail: { code: 'budget.session' } },
]),
localThread,
);
expect(localThread.post).toHaveBeenCalledTimes(1);
expect(localThread.post).toHaveBeenCalledWith({ markdown: SESSION_BUDGET_MESSAGE });
expect(postErrorToThread).toHaveBeenCalledTimes(1);
expect(postErrorToThread).toHaveBeenCalledWith(localThread, deliveryError, undefined);
});
});
packages/cli/src/modules/agents/integrations/__tests__/helpers/replay-test-helpers.ts
+52 -4
View File
@@ -5,6 +5,7 @@ import { deepCopy, type Logger } from 'n8n-workflow';
import type { Mock } from 'vitest';
import { mock } from 'vitest-mock-extended';
import type { AgentChatAttachmentService } from '../../../agent-chat-attachment.service';
import { AgentChatBridge } from '../../agent-chat-bridge';
import { ChatIntegrationRegistry, type AgentChatIntegration } from '../../agent-chat-integration';
import type { ChatIntegrationService, ChatInstance } from '../../chat-integration.service';
@@ -126,8 +127,18 @@ export interface StubbedRequest {
export interface StubResponse {
/** Recorded call exposed to assertions (platform method name + request body). */
apiCall: ReplayApiCall;
/** JSON body returned to the real adapter so it proceeds without live network I/O. */
responseBody: unknown;
/**
* JSON body returned to the real adapter so it proceeds without live
* network I/O. Ignored when `rawResponseBody` is set.
*/
responseBody?: unknown;
/**
* Non-JSON bytes returned as-is (no `JSON.stringify`) — for endpoints a
* real adapter reads as a binary file, such as WhatsApp's media download,
* where sniffing/parsing needs genuine magic bytes, not a JSON-escaped
* stand-in that would corrupt them.
*/
rawResponseBody?: Buffer | Uint8Array;
status?: number;
}
@@ -171,9 +182,17 @@ export function installFetchStub(options: {
}
}
const { apiCall, responseBody, status } = options.onRequest({ httpMethod, url, body, rawBody });
const { apiCall, responseBody, rawResponseBody, status } = options.onRequest({
httpMethod,
url,
body,
rawBody,
});
apiCalls.push(apiCall);
if (rawResponseBody) {
return new Response(new Uint8Array(rawResponseBody), { status: status ?? 200 });
}
return new Response(JSON.stringify(responseBody), {
status: status ?? 200,
headers: { 'content-type': 'application/json' },
@@ -200,6 +219,7 @@ export interface ReplayContextSetup<TChat extends ChatInstance = ChatInstance> {
descriptor: ReturnType<typeof getIntegrationToolConnectionDescriptors>[number];
integration: AgentIntegrationConfig;
messageContextStore: MemoryMessageContextStore;
attachmentService?: AgentChatAttachmentService;
latestContext: () => IntegrationMessageContext | undefined;
latestThreadId: () => string | undefined;
nextStream: (chunks: StreamChunk[]) => void;
@@ -213,6 +233,13 @@ export function createReplayContextSetup<TChat extends ChatInstance>(params: {
integration: AgentIntegrationConfig;
componentMapper?: ComponentMapper;
stream?: StreamChunk[];
/**
* Without this, `AgentChatBridge`'s attachment pipeline silently no-ops
* (see its `!this.attachmentService` guard). Pass
* `createMockAttachmentService()` when a platform's tests cover inbound
* attachments and stub the platform's file download.
*/
attachmentService?: AgentChatAttachmentService;
// Shared with the integration impl when its adapter also needs to record or
// check the connection cooldown itself (e.g. WhatsApp's automatic replies —
// see `withWhatsAppRateLimitBackoff`), so both paths agree on one connection's state.
@@ -248,6 +275,7 @@ export function createReplayContextSetup<TChat extends ChatInstance>(params: {
isResumable: vi.fn(async () => true),
};
const messageContextStore = new MemoryMessageContextStore();
const { attachmentService } = params;
const pending: Array<{ payload: QueuedIntegrationMessage; threadId: string }> = [];
const queue = mock<AgentMessageQueueService>();
queue.enqueue.mockImplementation(async ({ payload, threadId }) => {
@@ -265,7 +293,7 @@ export function createReplayContextSetup<TChat extends ChatInstance>(params: {
'project-1',
params.integration,
messageContextStore as unknown as IntegrationMessageContextService,
undefined,
attachmentService,
undefined,
queue,
);
@@ -288,6 +316,7 @@ export function createReplayContextSetup<TChat extends ChatInstance>(params: {
descriptor,
integration: params.integration,
messageContextStore,
attachmentService,
latestContext: () => selectedContext,
latestThreadId: () => selectedThreadId,
nextStream: (chunks: StreamChunk[]) => {
@@ -313,3 +342,22 @@ export function createReplayContextSetup<TChat extends ChatInstance>(params: {
},
};
}
/**
* Echoes back a plausible stored record instead of hitting real binary
* storage — good enough for asserting a platform's attachment made it
* through the bridge's pipeline with the right name/type/size, without
* needing a real `BinaryDataService`.
*/
export function createMockAttachmentService(): AgentChatAttachmentService {
const service = mock<AgentChatAttachmentService>();
service.storeInbound.mockImplementation(async (params) =>
mock<Awaited<ReturnType<AgentChatAttachmentService['storeInbound']>>>({
id: `attachment-${params.fileName}`,
fileName: params.fileName,
mimeType: params.mimeType,
fileSizeBytes: params.data.byteLength,
}),
);
return service;
}
packages/cli/src/modules/agents/integrations/__tests__/helpers/whatsapp/replay-test-context.ts
+105 -4
View File
@@ -1,5 +1,5 @@
import type { StreamChunk } from '@n8n/agents';
import type { AgentIntegrationConfig } from '@n8n/api-types';
import { MAX_AGENT_CHAT_ATTACHMENT_SIZE_BYTES, type AgentIntegrationConfig } from '@n8n/api-types';
import type { Logger as BackendLogger } from '@n8n/backend-common';
import type { WhatsAppAdapter } from '@chat-adapter/whatsapp';
import { createHmac } from 'crypto';
@@ -7,6 +7,7 @@ import type { InstanceSettings } from 'n8n-core';
import type { Mock } from 'vitest';
import { mock } from 'vitest-mock-extended';
import type { AgentChatAttachmentService } from '../../../../agent-chat-attachment.service';
import type { AgentChatIntegrationContext } from '../../../agent-chat-integration';
import { ChannelRateLimitGuard } from '../../../channel-rate-limit.guard';
import type { ChatInstance } from '../../../chat-integration.service';
@@ -19,6 +20,7 @@ import type {
import type { AgentRepository } from '../../../../repositories/agent.repository';
import { WhatsAppIntegration } from '../../../platforms/whatsapp-integration';
import {
createMockAttachmentService,
createReplayContextSetup,
installFetchStub,
type MemoryMessageContextStore,
@@ -36,7 +38,18 @@ export interface WhatsAppInboundMessageFixture {
from: string;
id: string;
timestamp: string;
type: 'text' | 'interactive' | 'button' | 'reaction';
type:
| 'text'
| 'interactive'
| 'button'
| 'reaction'
| 'image'
| 'document'
| 'audio'
| 'video'
| 'sticker'
| 'location'
| 'contacts';
text?: { body: string };
interactive?: {
type: 'button_reply' | 'list_reply';
@@ -45,6 +58,22 @@ export interface WhatsAppInboundMessageFixture {
};
button?: { payload: string; text: string };
reaction?: { emoji: string; message_id: string };
image?: { id: string; mime_type: string; sha256: string; caption?: string };
document?: {
id: string;
mime_type: string;
sha256: string;
filename?: string;
caption?: string;
};
audio?: { id: string; mime_type: string; sha256: string; voice?: boolean };
video?: { id: string; mime_type: string; sha256: string; caption?: string };
sticker?: { id: string; mime_type: string; sha256: string; animated: boolean };
location?: { latitude: number; longitude: number; name?: string; address?: string; url?: string };
contacts?: Array<{
name?: { formatted_name: string; first_name?: string; last_name?: string };
phones?: Array<{ phone: string; wa_id?: string; type?: string }>;
}>;
}
export interface WhatsAppWebhookFixture {
@@ -72,8 +101,10 @@ export interface WhatsAppReplayFixtures {
followUp: WhatsAppWebhookFixture;
}
export interface WhatsAppReplayContext extends Omit<ReplayContextSetup, 'nextStream' | 'chat'> {
export interface WhatsAppReplayContext
extends Omit<ReplayContextSetup, 'nextStream' | 'chat' | 'attachmentService'> {
chat: ChatInstance;
attachmentService: AgentChatAttachmentService;
agentExecutor: {
executeForChatPublished: Mock;
resumeForChat: Mock;
@@ -119,6 +150,49 @@ export function createWhatsAppIntegration(
);
}
/**
* Real, minimal-but-valid file content per media kind — genuine magic bytes,
* not arbitrary filler — so `resolveInboundMimeType`'s magic-byte sniffing
* (see `agent-chat-bridge.ts`) resolves them the same way it would for a real
* download, rather than falling back to `application/octet-stream`. Keyed by
* the same string each fixture's fake media `id` embeds (e.g. `media-image-1`
* contains `"image"`), so the stub below can serve the right bytes for
* whichever media a test's fixture declares.
*/
const WHATSAPP_MEDIA_CONTENT: Record<string, Buffer> = {
image: Buffer.from([0xff, 0xd8, 0xff, 0xe0, 0x00, 0x10, 0x4a, 0x46, 0x49, 0x46]),
document: Buffer.from('%PDF-1.4\nsynthetic test content padding padding padding'),
audio: Buffer.from([0xff, 0xfb, 0x90, 0x64, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00]),
// A minimal real OGG page (see https://www.rfc-editor.org/rfc/rfc3533)
// wrapping a complete 19-byte Opus ID header (RFC 7845 §5.1), so it sniffs
// as audio/ogg like a real WhatsApp voice note would.
voice: Buffer.concat([
Buffer.from('OggS'),
Buffer.from([0x00]), // version
Buffer.from([0x02]), // header_type: beginning of stream
Buffer.alloc(8), // granule position
Buffer.from([0x01, 0x02, 0x03, 0x04]), // serial number
Buffer.alloc(4), // page sequence number
Buffer.alloc(4), // CRC checksum
Buffer.from([0x01]), // page_segments: one lacing value follows
Buffer.from([19]), // segment_table: one 19-byte segment
Buffer.from('OpusHead'),
Buffer.from([0x01]), // version
Buffer.from([0x01]), // channel count
Buffer.from([0x38, 0x01]), // pre-skip (312, little-endian)
Buffer.from([0x80, 0xbb, 0x00, 0x00]), // input sample rate (48000, little-endian)
Buffer.from([0x00, 0x00]), // output gain
Buffer.from([0x00]), // channel mapping family
]),
video: Buffer.from([0, 0, 0, 0x18, 0x66, 0x74, 0x79, 0x70, 0x6d, 0x70, 0x34, 0x32]),
sticker: Buffer.concat([Buffer.from('RIFF'), Buffer.from([0, 0, 0, 0]), Buffer.from('WEBPVP8 ')]),
// One byte over the shared cap: WhatsApp attachments never carry a
// declared `size` (see `WhatsAppInboundMessage`), so the bridge's size
// check only ever fires after this downloads — there's nothing to
// pre-empt it on, unlike a platform that reports size upfront.
oversized: Buffer.alloc(MAX_AGENT_CHAT_ATTACHMENT_SIZE_BYTES + 1, 0x41),
};
/**
* A fixed number of leading sends fail with the given Meta error before the
* stub starts succeeding — for exercising `withWhatsAppRateLimitBackoff`'s
@@ -145,8 +219,32 @@ function installWhatsAppApiStub(
let failuresLeft = failureSequence?.count ?? 0;
return installFetchStub({
match: /graph\.facebook\.com/,
onRequest: ({ url, body }) => {
onRequest: ({ httpMethod, url, body }) => {
const path = url.split('?')[0];
// `downloadMedia(mediaId)` (used for every inbound attachment) makes two
// plain GETs with no JSON body — a metadata lookup, then the CDN url it
// returns — so they need to be branched on method+path before the
// send-message logic below, which assumes a POST with a JSON body.
if (httpMethod === 'GET') {
if (path.includes('/media-download/')) {
const mediaId = path.slice(path.lastIndexOf('/') + 1);
// Fixtures' media ids embed their kind (e.g. `media-image-1`), so
// real magic-byte content can be matched to whichever media the
// test's fixture actually declared.
const kind = Object.keys(WHATSAPP_MEDIA_CONTENT).find((key) => mediaId.includes(key));
return {
apiCall: { method: 'media-download', body: {} },
rawResponseBody: kind ? WHATSAPP_MEDIA_CONTENT[kind] : Buffer.from('unknown-media'),
};
}
const mediaId = path.slice(path.lastIndexOf('/') + 1);
return {
apiCall: { method: 'media-metadata', body: {} },
responseBody: { url: `https://graph.facebook.com/media-download/${mediaId}` },
};
}
const method = path.slice(path.lastIndexOf('/') + 1);
const type = typeof body.type === 'string' ? body.type : undefined;
if (type && failedTypes.includes(type)) {
@@ -236,12 +334,14 @@ export async function createWhatsAppReplayContext(
state: createMemoryState(),
});
const attachmentService = createMockAttachmentService();
const setup = createReplayContextSetup({
chat: chat as never,
integrationImpl,
integration,
componentMapper: new ComponentMapper(),
stream: options.stream,
attachmentService,
channelRateLimitGuard,
});
@@ -270,6 +370,7 @@ export async function createWhatsAppReplayContext(
return {
...setup,
attachmentService,
chat: chat as unknown as ChatInstance,
adapter,
apiCalls: stub.apiCalls,
packages/cli/src/modules/agents/integrations/__tests__/helpers/whatsapp/synthetic-fixtures.ts
+106
View File
@@ -33,6 +33,112 @@ export const whatsAppInboundTextMessage = (
...overrides,
});
/** Shared base fields every media fixture below overrides on top of. */
const mediaMessageBase = (
overrides: Partial<WhatsAppInboundMessageFixture>,
): Pick<WhatsAppInboundMessageFixture, 'from' | 'id' | 'timestamp'> => ({
from: DEFAULT_USER_WA_ID,
id: 'wamid.TEST_INBOUND_MEDIA',
timestamp: String(Math.floor(Date.now() / 1000)),
...overrides,
});
export const whatsAppInboundImageMessage = (
overrides: Partial<WhatsAppInboundMessageFixture> = {},
): WhatsAppInboundMessageFixture => ({
...mediaMessageBase(overrides),
type: 'image',
image: { id: 'media-image-1', mime_type: 'image/jpeg', sha256: 'test-sha256-image' },
...overrides,
});
export const whatsAppInboundDocumentMessage = (
overrides: Partial<WhatsAppInboundMessageFixture> = {},
): WhatsAppInboundMessageFixture => ({
...mediaMessageBase(overrides),
type: 'document',
document: {
id: 'media-document-1',
mime_type: 'application/pdf',
sha256: 'test-sha256-document',
filename: 'invoice.pdf',
},
...overrides,
});
export const whatsAppInboundAudioMessage = (
overrides: Partial<WhatsAppInboundMessageFixture> = {},
): WhatsAppInboundMessageFixture => ({
...mediaMessageBase(overrides),
type: 'audio',
audio: { id: 'media-audio-1', mime_type: 'audio/mpeg', sha256: 'test-sha256-audio' },
...overrides,
});
/**
* A recorded voice note. Meta sends these as `type: "audio"` with
* `audio.voice: true` and an Opus/OGG payload, not as `type: "voice"`.
*/
export const whatsAppInboundVoiceMessage = (
overrides: Partial<WhatsAppInboundMessageFixture> = {},
): WhatsAppInboundMessageFixture => ({
...mediaMessageBase(overrides),
type: 'audio',
audio: {
id: 'media-voice-1',
mime_type: 'audio/ogg; codecs=opus',
sha256: 'test-sha256-voice',
voice: true,
},
...overrides,
});
export const whatsAppInboundVideoMessage = (
overrides: Partial<WhatsAppInboundMessageFixture> = {},
): WhatsAppInboundMessageFixture => ({
...mediaMessageBase(overrides),
type: 'video',
video: { id: 'media-video-1', mime_type: 'video/mp4', sha256: 'test-sha256-video' },
...overrides,
});
export const whatsAppInboundStickerMessage = (
overrides: Partial<WhatsAppInboundMessageFixture> = {},
): WhatsAppInboundMessageFixture => ({
...mediaMessageBase(overrides),
type: 'sticker',
sticker: {
id: 'media-sticker-1',
mime_type: 'image/webp',
sha256: 'test-sha256-sticker',
animated: false,
},
...overrides,
});
export const whatsAppInboundLocationMessage = (
overrides: Partial<WhatsAppInboundMessageFixture> = {},
): WhatsAppInboundMessageFixture => ({
...mediaMessageBase(overrides),
type: 'location',
location: { latitude: 51.5072, longitude: -0.1276, name: 'London' },
...overrides,
});
export const whatsAppInboundContactsMessage = (
overrides: Partial<WhatsAppInboundMessageFixture> = {},
): WhatsAppInboundMessageFixture => ({
...mediaMessageBase(overrides),
type: 'contacts',
contacts: [
{
name: { formatted_name: 'Jane Doe', first_name: 'Jane', last_name: 'Doe' },
phones: [{ phone: '+44 7700 900123', wa_id: '447700900123', type: 'CELL' }],
},
],
...overrides,
});
export const whatsAppWebhook = (options: {
phoneNumberId?: string;
contact?: WhatsAppContactFixture;
packages/cli/src/modules/agents/integrations/agent-chat-stream-consumer.ts
+55 -9
View File
@@ -96,6 +96,18 @@ const createResponseState = (options: ConsumeStreamOptions): ResponseState => ({
quietErrors: options.quietErrors === true,
});
const BUDGET_STOP_TEXT = {
'budget.session':
'⚠️ This session reached its cost cap and stopped. Open the agent in n8n and increase the cap.',
'budget.monthly':
'⚠️ This agent reached its monthly budget and stopped. Open the agent in n8n and increase the cap.',
} as const;
function budgetStopText(code: string | undefined): string | undefined {
if (code === 'budget.session' || code === 'budget.monthly') return BUDGET_STOP_TEXT[code];
return undefined;
}
export class AgentChatStreamConsumer {
constructor(private readonly options: AgentChatStreamConsumerOptions) {}
@@ -222,6 +234,7 @@ export class AgentChatStreamConsumer {
endStreamingPost,
});
const responseState = createResponseState(options);
let budgetStop: string | undefined;
try {
for await (const chunk of stream) {
@@ -267,13 +280,20 @@ export class AgentChatStreamConsumer {
pendingText = '';
}
break;
case 'finish':
budgetStop ??= budgetStopText(chunk.guardrail?.code);
break;
default:
// Ignore non-user-visible chunks (reasoning, finish,
// tool-input-*, start-step, finish-step, etc.)
// Ignore non-user-visible chunks (reasoning, tool-input-*,
// start-step, finish-step, etc.)
break;
}
}
await this.postFallbackIfNeeded(responseState, responseLifecycle, thread);
if (budgetStop !== undefined) {
await this.postBudgetStop(thread, budgetStop, responseState, responseLifecycle);
} else {
await this.postFallbackIfNeeded(responseState, responseLifecycle, thread);
}
} finally {
await responseLifecycle.finish();
}
@@ -327,6 +347,18 @@ export class AgentChatStreamConsumer {
}
}
private async postBudgetStop(
thread: Thread<unknown, unknown>,
text: string,
state: ResponseState,
lifecycle: ResponseLifecycle,
throwOnDeliveryError = false,
): Promise<void> {
await lifecycle.startDiscreteResponse();
await this.postBufferedText(thread, text, state, throwOnDeliveryError);
state.hasVisibleResponse = true;
}
private async postError(
thread: Thread<unknown, unknown>,
error: unknown,
@@ -436,6 +468,7 @@ export class AgentChatStreamConsumer {
await this.postBufferedText(thread, text, responseState, options.throwOnDeliveryError);
responseState.hasVisibleResponse = true;
};
let budgetStop: string | undefined;
try {
for await (const chunk of stream) {
@@ -488,17 +521,30 @@ export class AgentChatStreamConsumer {
buffer = '';
}
break;
case 'finish':
budgetStop ??= budgetStopText(chunk.guardrail?.code);
break;
default:
break;
}
}
await flushBuffer();
await this.postFallbackIfNeeded(
responseState,
responseLifecycle,
thread,
options.throwOnDeliveryError,
);
if (budgetStop !== undefined) {
await this.postBudgetStop(
thread,
budgetStop,
responseState,
responseLifecycle,
options.throwOnDeliveryError,
);
} else {
await this.postFallbackIfNeeded(
responseState,
responseLifecycle,
thread,
options.throwOnDeliveryError,
);
}
} finally {
await flushBuffer();
await responseLifecycle.finish();
packages/cli/src/modules/agents/integrations/platforms/__tests__/whatsapp/synthetic-integration.test.ts
+187
View File
@@ -8,10 +8,19 @@ import {
createWhatsAppIntegration,
createWhatsAppReplayContext,
whatsAppThreadId,
type WhatsAppInboundMessageFixture,
} from '../../../__tests__/helpers/whatsapp/replay-test-context';
import {
whatsAppContact,
whatsAppInboundAudioMessage,
whatsAppInboundContactsMessage,
whatsAppInboundDocumentMessage,
whatsAppInboundImageMessage,
whatsAppInboundLocationMessage,
whatsAppInboundStickerMessage,
whatsAppInboundTextMessage,
whatsAppInboundVideoMessage,
whatsAppInboundVoiceMessage,
whatsAppReplayFixtures,
whatsAppWebhook,
} from '../../../__tests__/helpers/whatsapp/synthetic-fixtures';
@@ -84,6 +93,184 @@ describe('WhatsApp Cloud API integration scenarios', () => {
}
});
describe('inbound media', () => {
it.each([
['image', whatsAppInboundImageMessage, 'image/jpeg'],
['document', whatsAppInboundDocumentMessage, 'application/pdf'],
['audio', whatsAppInboundAudioMessage, 'audio/mpeg'],
// Real voice-note shape: `type: "audio"` with `audio.voice: true`. The
// MIME type is sniffed from the Opus/OGG bytes in WHATSAPP_MEDIA_CONTENT.
['voice', whatsAppInboundVoiceMessage, 'audio/ogg; codecs=opus'],
['video', whatsAppInboundVideoMessage, 'video/mp4'],
['sticker', whatsAppInboundStickerMessage, 'image/webp'],
] as const)(
'stores a %s attachment and passes it to the agent executor',
async (_label, buildMessage, expectedMimeType) => {
const fixtures = whatsAppReplayFixtures();
const ctx = await createWhatsAppReplayContext(fixtures);
try {
await ctx.sendWebhook(
whatsAppWebhook({
phoneNumberId: fixtures.phoneNumberId,
contact: fixtures.contact,
message: buildMessage({ from: fixtures.contact.wa_id }),
}),
);
expect(ctx.attachmentService.storeInbound).toHaveBeenCalledExactlyOnceWith(
expect.objectContaining({
source: 'whatsapp',
mimeType: expectedMimeType,
data: expect.any(Buffer),
}),
);
expect(ctx.agentExecutor.executeForChatPublished).toHaveBeenCalledExactlyOnceWith(
expect.objectContaining({
attachments: [expect.objectContaining({ mimeType: expectedMimeType })],
}),
);
} finally {
await ctx.shutdown();
}
},
);
it('degrades to a text note instead of storing an attachment over the size cap', async () => {
// WhatsApp's inbound media messages never carry a declared size (see
// `WhatsAppInboundMessage`), unlike some other platforms — so this can
// only ever be caught after downloading, never pre-empted upfront.
const fixtures = whatsAppReplayFixtures();
const ctx = await createWhatsAppReplayContext(fixtures);
try {
await ctx.sendWebhook(
whatsAppWebhook({
phoneNumberId: fixtures.phoneNumberId,
contact: fixtures.contact,
message: whatsAppInboundDocumentMessage({
from: fixtures.contact.wa_id,
document: {
id: 'media-oversized-1',
mime_type: 'application/pdf',
sha256: 'test-sha256-oversized',
filename: 'big.pdf',
},
}),
}),
);
expect(ctx.attachmentService.storeInbound).not.toHaveBeenCalled();
expect(ctx.agentExecutor.executeForChatPublished).toHaveBeenCalledExactlyOnceWith(
expect.objectContaining({ message: expect.stringContaining('MB') }),
);
} finally {
await ctx.shutdown();
}
});
it('passes a location to the agent as a text summary only', async () => {
// The adapter also adds the location as a URL-only pseudo-file (a Google
// Maps link with no bytes). The bridge cannot download it, so the
// integration drops it. Otherwise the agent gets a misleading
// "could not be downloaded" note next to the summary.
const fixtures = whatsAppReplayFixtures();
const ctx = await createWhatsAppReplayContext(fixtures);
try {
await ctx.sendWebhook(
whatsAppWebhook({
phoneNumberId: fixtures.phoneNumberId,
contact: fixtures.contact,
message: whatsAppInboundLocationMessage({ from: fixtures.contact.wa_id }),
}),
);
expect(ctx.attachmentService.storeInbound).not.toHaveBeenCalled();
expect(ctx.agentExecutor.executeForChatPublished).toHaveBeenCalledExactlyOnceWith(
expect.objectContaining({ message: '[Location: London]' }),
);
} finally {
await ctx.shutdown();
}
});
it('passes a shared contact to the agent as a text summary', async () => {
// The adapter has no handling for `contacts` and drops the message, so
// the integration's adapter subclass summarizes it as text instead.
const fixtures = whatsAppReplayFixtures();
const ctx = await createWhatsAppReplayContext(fixtures);
try {
await ctx.sendWebhook(
whatsAppWebhook({
phoneNumberId: fixtures.phoneNumberId,
contact: fixtures.contact,
message: whatsAppInboundContactsMessage({ from: fixtures.contact.wa_id }),
}),
);
expect(ctx.attachmentService.storeInbound).not.toHaveBeenCalled();
expect(ctx.agentExecutor.executeForChatPublished).toHaveBeenCalledExactlyOnceWith(
expect.objectContaining({ message: '[Contact: Jane Doe - +44 7700 900123]' }),
);
} finally {
await ctx.shutdown();
}
});
it.each([
[
'a contact with no name',
[{ phones: [{ phone: '+44 7700 900123' }] }],
'[Contact: +44 7700 900123]',
],
[
'a contact with no phone numbers',
[{ name: { formatted_name: 'Jane Doe' } }],
'[Contact: Jane Doe]',
],
[
'a contact with several phone numbers',
[
{
name: { formatted_name: 'Jane Doe' },
phones: [{ phone: '+44 7700 900123' }, { phone: '+44 20 7946 0000' }],
},
],
'[Contact: Jane Doe - +44 7700 900123 - +44 20 7946 0000]',
],
[
'several contacts',
[
{ name: { formatted_name: 'Jane Doe' }, phones: [{ phone: '+44 7700 900123' }] },
{ name: { formatted_name: 'John Smith' } },
],
'[Contact: Jane Doe - +44 7700 900123]\n[Contact: John Smith]',
],
['a contact with no name or phone numbers', [{}], '[Contact]'],
['an empty contacts list', [], '[Contact]'],
['no contacts field', undefined, '[Contact]'],
] as const)('summarizes %s', async (_label, contacts, expectedMessage) => {
const fixtures = whatsAppReplayFixtures();
const ctx = await createWhatsAppReplayContext(fixtures);
try {
await ctx.sendWebhook(
whatsAppWebhook({
phoneNumberId: fixtures.phoneNumberId,
contact: fixtures.contact,
message: whatsAppInboundContactsMessage({
from: fixtures.contact.wa_id,
contacts: contacts as WhatsAppInboundMessageFixture['contacts'],
}),
}),
);
expect(ctx.agentExecutor.executeForChatPublished).toHaveBeenCalledExactlyOnceWith(
expect.objectContaining({ message: expectedMessage }),
);
} finally {
await ctx.shutdown();
}
});
});
it('persists current message context for the integration context tool', async () => {
const fixtures = whatsAppReplayFixtures();
const ctx = await createWhatsAppReplayContext(fixtures);
packages/cli/src/modules/agents/integrations/platforms/whatsapp-integration.ts
+38 -1
View File
@@ -27,7 +27,7 @@ import {
} from '../component-mapper';
import { assertCredentialNotClaimed } from '../credential-claim';
import { loadChatSdk, loadWhatsAppAdapter } from '../esm-loader';
import { deriveWhatsAppVerifyToken, stringValue } from '../integration-helpers';
import { deriveWhatsAppVerifyToken, stringProperty, stringValue } from '../integration-helpers';
import { resolveIntegrationActionDefinitions } from '../integration-tool-definitions';
import { buildIntegrationConnectionId } from '../integration-tool-factory';
@@ -262,6 +262,23 @@ export class WhatsAppIntegration extends AgentChatIntegration {
{ guard: channelRateLimitGuard, connectionId, recipient },
);
}
// The adapter returns null for `contacts`, which drops the message.
// Summarize shared contact cards as text so the agent still gets them.
protected override extractTextContent(message: WhatsAppRawMessage['message']) {
if (message.type === 'contacts') return formatSharedContacts(message);
return super.extractTextContent(message);
}
// The adapter adds a location as a URL-only pseudo-file (a Maps link with
// no bytes). The bridge cannot download it, and the text summary already
// carries the location, so drop it.
protected override buildAttachments(inbound: WhatsAppRawMessage['message']) {
const attachments = super.buildAttachments(inbound);
return inbound.type === 'location'
? attachments.filter((attachment) => attachment.fetchData !== undefined)
: attachments;
}
}
return new ConversationWindowGuardedAdapter(config);
@@ -395,6 +412,26 @@ export class WhatsAppIntegration extends AgentChatIntegration {
}
}
/**
* Summarize a `contacts` message in the same bracketed style the adapter uses
* for other non-text types (e.g. `[Location: ...]`). The adapter's inbound
* type does not declare the `contacts` field, so read it defensively.
* @see https://developers.facebook.com/docs/whatsapp/cloud-api/webhooks/payload-examples#contacts-messages
*/
function formatSharedContacts(message: unknown): string {
const contacts = isRecord(message) && Array.isArray(message.contacts) ? message.contacts : [];
const lines = contacts.map((contact: unknown) => {
const name = isRecord(contact) ? stringProperty(contact.name, 'formatted_name') : undefined;
const phones = isRecord(contact) && Array.isArray(contact.phones) ? contact.phones : [];
const phoneNumbers = phones
.map((phone: unknown) => stringProperty(phone, 'phone'))
.filter((phone): phone is string => phone !== undefined);
const parts = name ? [name, ...phoneNumbers] : phoneNumbers;
return parts.length > 0 ? `[Contact: ${parts.join(' - ')}]` : '[Contact]';
});
return lines.length > 0 ? lines.join('\n') : '[Contact]';
}
/**
* Reject a send once more than 24 hours have passed since the customer's
* last inbound message. `chat` is `null` before the adapter's `initialize()`
packages/cli/src/modules/chat-hub/__tests__/chat-hub-execution-watcher.service.test.ts
+47 -2
View File
@@ -171,8 +171,12 @@ describe('ChatHubExecutionWatcherService', () => {
...overrides,
}) as IRun;
const createAfterContext = (executionId: string, runData: IRun): WorkflowExecuteAfterContext =>
({ executionId, runData }) as WorkflowExecuteAfterContext;
const createAfterContext = (
executionId: string,
runData: IRun,
{ executedInThisProcess = true }: { executedInThisProcess?: boolean } = {},
): WorkflowExecuteAfterContext =>
({ executionId, runData, executedInThisProcess }) as WorkflowExecuteAfterContext;
it('should skip if context not found', async () => {
executionStore.get.mockResolvedValue(null);
@@ -457,9 +461,50 @@ describe('ChatHubExecutionWatcherService', () => {
createMessageOnResume: true,
});
});
it('should handle a segment once when the enqueuing process also reports it', async () => {
let stored = createContext({ responseMode: 'responseNodes' });
executionStore.get.mockImplementation(async () => stored);
executionStore.update.mockImplementation(async (_executionId, updates) => {
stored = { ...stored, ...updates };
});
chatHubExecutionService.extractMessage.mockReturnValue('Found the incident.');
executionPersistence.findSingleExecution.mockResolvedValue(
createExecution('@n8n/n8n-nodes-langchain.chat', { operation: 'send' }),
);
const runData = createRunData({ status: 'waiting' });
await service.handleWorkflowExecuteAfter(createAfterContext(EXECUTION_ID, runData));
await service.handleWorkflowExecuteAfter(
createAfterContext(EXECUTION_ID, runData, { executedInThisProcess: false }),
);
expect(messageRepository.createChatMessage).toHaveBeenCalledTimes(1);
expect(chatStreamService.endExecution).toHaveBeenCalledTimes(1);
expect(executionManager.runWorkflow).toHaveBeenCalledTimes(1);
expect(messageRepository.updateChatMessage).not.toHaveBeenCalledWith(
MOCK_NEW_MESSAGE_ID,
expect.anything(),
);
});
});
});
it('should ignore a call from a process that only enqueued the run', async () => {
executionStore.get.mockResolvedValue(createContext({ responseMode: 'responseNodes' }));
await service.handleWorkflowExecuteAfter(
createAfterContext(EXECUTION_ID, createRunData({ status: 'waiting' }), {
executedInThisProcess: false,
}),
);
expect(executionStore.get).not.toHaveBeenCalled();
expect(messageRepository.updateChatMessage).not.toHaveBeenCalled();
expect(chatStreamService.endExecution).not.toHaveBeenCalled();
expect(executionManager.runWorkflow).not.toHaveBeenCalled();
});
it('should not remove context when finished is false', async () => {
const context = createContext();
executionStore.get.mockResolvedValue(context);
packages/cli/src/modules/chat-hub/__tests__/chat-hub.service.integration.test.ts
+13
View File
@@ -1483,6 +1483,7 @@ describe('chatHub', () => {
runData,
newStaticData: {},
executionId,
executedInThisProcess: true,
});
});
@@ -1601,6 +1602,7 @@ describe('chatHub', () => {
runData,
newStaticData: {},
executionId,
executedInThisProcess: true,
});
});
@@ -1733,6 +1735,7 @@ describe('chatHub', () => {
runData,
newStaticData: {},
executionId,
executedInThisProcess: true,
});
});
@@ -1874,6 +1877,7 @@ describe('chatHub', () => {
runData,
newStaticData: {},
executionId,
executedInThisProcess: true,
});
});
@@ -1930,6 +1934,7 @@ describe('chatHub', () => {
runData,
newStaticData: {},
executionId: capturedExecutionId,
executedInThisProcess: true,
});
});
});
@@ -2064,6 +2069,7 @@ describe('chatHub', () => {
runData,
newStaticData: {},
executionId,
executedInThisProcess: true,
});
});
@@ -2205,6 +2211,7 @@ describe('chatHub', () => {
runData,
newStaticData: {},
executionId,
executedInThisProcess: true,
});
});
@@ -2298,6 +2305,7 @@ describe('chatHub', () => {
runData,
newStaticData: {},
executionId: capturedExecutionId,
executedInThisProcess: true,
});
});
});
@@ -2449,6 +2457,7 @@ describe('chatHub', () => {
runData,
newStaticData: {},
executionId,
executedInThisProcess: true,
});
});
@@ -2658,6 +2667,7 @@ describe('chatHub', () => {
runData,
newStaticData: {},
executionId,
executedInThisProcess: true,
});
});
@@ -2793,6 +2803,7 @@ describe('chatHub', () => {
runData,
newStaticData: {},
executionId,
executedInThisProcess: true,
});
});
@@ -2909,6 +2920,7 @@ describe('chatHub', () => {
runData,
newStaticData: {},
executionId,
executedInThisProcess: true,
});
});
@@ -3025,6 +3037,7 @@ describe('chatHub', () => {
runData,
newStaticData: {},
executionId,
executedInThisProcess: true,
});
});
packages/cli/src/modules/chat-hub/chat-hub-execution-watcher.service.ts
+6 -4
View File
@@ -117,6 +117,11 @@ export class ChatHubExecutionWatcherService {
async handleWorkflowExecuteAfter(ctx: WorkflowExecuteAfterContext): Promise<void> {
const { runData, executionId } = ctx;
// An auto-resume enqueues the next segment from the worker, so that worker also waits
// for the job and gets a second call for the same segment. The process that ran the
// segment handles it.
if (!ctx.executedInThisProcess) return;
const context = await this.executionStore.get(executionId);
if (!context) return; // Not a tracked chat hub execution
@@ -144,10 +149,7 @@ export class ChatHubExecutionWatcherService {
return;
}
// NOTE: This check is required because on multi-main/queue mode, the resumed execution's
// 'workflowExecuteAfter' hook fires one more time after it should have stopped in 'waiting'
// state on a Chat response node. On this final hook call the runData.status is 'success' even though
// the execution isn't finished. On single-main mode this does not happen.
// A `success` run that has not finished must not end the conversation.
// oxlint-disable-next-line typescript/no-deprecated
if (runData.finished) {
await this.pushFinalResults(context, message);
packages/cli/src/modules/instance-ai/eval/README.md
+12
View File
@@ -54,6 +54,18 @@ The bypass is queried per test rather than snapshotted into the adapter: the
harness registers ids mid-run, when the simulated user creates a credential on
a card, long after the run's context was built.
## Agents a workflow calls
A Message an Agent node, or its tool version, runs another agent in-process, so
no HTTP mock reaches it by itself. In an eval run the workflow eval gives that
agent the Agent eval's seams (`prepareCalledAgent` in `execution.service.ts`):
its node and workflow tools, streamable-HTTP MCP servers and fallback web search
are mocked, and each mocked call is recorded under the calling node. The agent's
own model call stays real, so its model needs a credential with a real key. The
eval runs the agent's draft and turns off its memory, vector stores, SSE MCP
servers and chat integrations. A sub-agent keeps only the MCP servers its calling
agent also has; the eval turns off the others and adds a warning.
## Agent model catalogs
The eval credential allowlist also marks Agent Builder sessions as evaluations.
packages/cli/src/modules/instance-ai/eval/__tests__/execution.service.test.ts
+278 -1
View File
@@ -1,5 +1,6 @@
import type { Mock } from 'vitest';
import type { Logger } from '@n8n/backend-common';
import type { AgentJsonConfig } from '@n8n/api-types';
import type { Logger, ModuleRegistry } from '@n8n/backend-common';
import type { ExecutionsConfig, InstanceAiConfig } from '@n8n/config';
import type { ProcessedDataRepository, User } from '@n8n/db';
import { mock } from 'vitest-mock-extended';
@@ -16,6 +17,7 @@ import { NodeConnectionTypes, TimeoutExecutionCancelledError, UserError } from '
import type { ActiveExecutions } from '@/active-executions';
import type { LoadNodesAndCredentials } from '@/load-nodes-and-credentials';
import type { PrepareWorkflowAgentForEval } from '@/modules/agents/agent-runtime-instrumentation';
import type { NodeTypes } from '@/node-types';
import type { PostHogClient } from '@/posthog';
import type { DataTableService } from '@/modules/data-table/data-table.service';
@@ -101,12 +103,22 @@ vi.mock('n8n-workflow', async () => {
};
});
const mockExecuteAgent = vi.fn();
vi.mock('@/workflow-execute-additional-data', async (importOriginal) => ({
...(await importOriginal<typeof import('@/workflow-execute-additional-data')>()),
executeAgent: async (...args: unknown[]) => await mockExecuteAgent(...args),
}));
vi.mock('../mcp-mock-fetch', () => ({ createMcpMockFetch: vi.fn() }));
vi.mock('../web-search-mock', () => ({ createWebSearchMock: vi.fn() }));
// ---------------------------------------------------------------------------
// Import SUT and mocked modules (after vi.mock calls)
// ---------------------------------------------------------------------------
import { EvalExecutionService } from '../execution.service';
import { createMcpMockFetch } from '../mcp-mock-fetch';
import { createLlmMockHandler } from '../mock-handler';
import { createWebSearchMock } from '../web-search-mock';
import { generatePinData } from '../pin-data-generator';
import {
detectBinaryDependencies,
@@ -227,6 +239,7 @@ describe('EvalExecutionService', () => {
const dataTableService = mock<DataTableService>();
const processedDataRepository = mock<ProcessedDataRepository>();
const instanceAiConfig = { evalInstance: true } as InstanceAiConfig;
const moduleRegistry = mock<ModuleRegistry>();
// Captured configureAdditionalData closure so tests can re-invoke it on a
// stub additionalData without booting the real runner.
@@ -265,6 +278,7 @@ describe('EvalExecutionService', () => {
dataTableService,
processedDataRepository,
instanceAiConfig,
moduleRegistry,
);
// Reset to safe default — tests that flip queue mode reassign in-test.
Object.assign(executionsConfig, { mode: 'regular' });
@@ -1869,6 +1883,48 @@ describe('EvalExecutionService', () => {
});
});
it('writes the embedded text into a synthesized PDF', async () => {
const hints = makeEmptyHints();
hints.triggerContent = {
body: {},
binary: {
attachment_0: {
mimeType: 'application/pdf',
fileName: 'invoice.pdf',
text: 'Invoice Number: INV-2026-0042',
},
},
};
generateMockHintsMock.mockResolvedValue(hints);
await service.executeWithLlmMock('wf-1', makeUser());
const runData = workflowRunner.run.mock.calls[0][0];
const item = runData.pinData?.['Webhook']?.[0];
const pdf = Buffer.from(item?.binary?.attachment_0.data ?? '', 'base64');
expect(pdf.toString('latin1')).toContain('(Invoice Number: INV-2026-0042) Tj');
});
it('does not add the requirement key when the embedded map names other files', async () => {
const hints = makeEmptyHints();
hints.triggerContent = {
body: {},
binary: { CV_0: { mimeType: 'application/pdf', fileName: 'cv.pdf' } },
};
generateMockHintsMock.mockResolvedValue(hints);
detectBinaryDependenciesMock.mockReturnValueOnce({
propertyName: 'data',
contentType: 'application/octet-stream',
filename: 'input.bin',
});
await service.executeWithLlmMock('wf-1', makeUser());
const runData = workflowRunner.run.mock.calls[0][0];
const item = runData.pinData?.['Webhook']?.[0];
expect(Object.keys(item?.binary ?? {})).toEqual(['CV_0']);
});
it('does not create pin data when triggerContent is empty', async () => {
const hints = makeEmptyHints();
hints.triggerContent = {};
@@ -2078,6 +2134,227 @@ describe('EvalExecutionService', () => {
});
});
// ── agents a workflow calls ──────────────────────────────────────
describe('agents a workflow calls', () => {
const callingNode = 'Ask Support';
const agentConfig = {
name: 'Support Bot',
model: 'openai/gpt-5',
credential: 'cred-1',
instructions: 'Answer support questions.',
memory: { enabled: true, storage: 'n8n' },
vectorStores: [{ id: 'store-1' }],
mcpServers: [{ name: 'linear', url: 'https://mcp.linear.app/mcp' }],
} as unknown as AgentJsonConfig;
const toolNode: INode = {
id: 'tool-1',
name: 'slack',
type: 'n8n-nodes-base.slack',
typeVersion: 2,
position: [0, 0],
parameters: {},
};
type AgentCallData = StubAdditionalData & {
executeAgent: (...args: unknown[]) => Promise<unknown>;
};
let runData: AgentCallData | undefined;
/** Makes the run call an agent through the swapped `executeAgent`, once per entry. */
function callAgentsDuringRun(...calls: Array<[threadId: string, workflowContext: object]>) {
workflowRunner.run.mockImplementation(async (data) => {
const ad = makeMockedAdditionalData() as AgentCallData;
runData = ad;
await data.configureAdditionalData?.(ad as never);
for (const [threadId, workflowContext] of calls) {
await ad.executeAgent(
{ agentId: 'agent-1' },
'hi',
'exec-1',
threadId,
ad,
'evaluation',
undefined,
workflowContext,
undefined,
);
}
return DB_EXECUTION_ID;
});
}
beforeEach(() => {
workflowFinderService.findWorkflowForUser.mockResolvedValue(makeWorkflowEntity() as never);
createLlmMockHandlerMock.mockReturnValue(
vi.fn().mockResolvedValue({ body: { ok: true }, headers: {}, statusCode: 200 }),
);
});
it('runs the called agent with fake I/O and records it under the calling node', async () => {
let prepared: Awaited<ReturnType<PrepareWorkflowAgentForEval>> | undefined;
const toolData = makeMockedAdditionalData();
mockExecuteAgent.mockImplementation(async (...args: unknown[]) => {
const prepareForEval = args[9] as PrepareWorkflowAgentForEval;
prepared = await prepareForEval(agentConfig);
prepared.instrumentation.configureToolAdditionalData?.(toolData as never, {
toolName: 'slack',
toolKind: 'node',
});
await toolData.evalLlmMockHandler?.(
{ url: 'https://slack.com/api/chat.postMessage', method: 'POST', body: { text: 'hi' } },
toolNode,
);
vi.mocked(createMcpMockFetch).mock.calls[0][0].onToolCall({
serverName: 'linear',
toolName: 'list_issues',
args: { query: 'refund' },
result: { issues: [] },
});
vi.mocked(createWebSearchMock).mock.calls[0][0].onSearch(
{ query: 'refund policy' },
{ query: 'refund policy', results: [] },
);
return { response: 'done' };
});
callAgentsDuringRun(['thread-1', { callingNodeName: callingNode }]);
const result = await service.executeWithLlmMock('wf-1', makeUser());
expect(mockExecuteAgent).toHaveBeenCalledWith(
{ agentId: 'agent-1' },
'hi',
'exec-1',
'thread-1',
expect.anything(),
'evaluation',
undefined,
{ callingNodeName: callingNode },
undefined,
expect.any(Function),
);
expect(prepared?.config.mcpServers).toEqual(agentConfig.mcpServers);
expect(prepared?.config.memory).toBeUndefined();
expect(prepared?.config.vectorStores).toBeUndefined();
const entry = result.nodeResults[callingNode];
expect(entry.executionMode).toBe('mocked');
expect(entry.interceptedRequests.map((request) => request.nodeType)).toEqual([
'n8n-nodes-base.slack',
'mcp:linear',
'web-search:fallback',
]);
expect(result.nodeResults.slack).toBeUndefined();
// The run's helper reports credentials the tools needed mocked.
expect(toolData.credentialsHelper).toBe(runData?.credentialsHelper);
// Memory goes off without a flag; a lost capability gets one.
expect(result.hints.warnings).toEqual([expect.stringContaining('vectorStores')]);
});
it('reuses one set of fakes per node and flags memory when a call continues a session', async () => {
mockExecuteAgent.mockImplementation(async (...args: unknown[]) => {
await (args[9] as PrepareWorkflowAgentForEval)(agentConfig);
return { response: 'done' };
});
const session = { callingNodeName: callingNode, hasCallerSessionId: true };
callAgentsDuringRun(['session-1', session], ['session-1', session]);
const result = await service.executeWithLlmMock('wf-1', makeUser());
expect(createMcpMockFetch).toHaveBeenCalledTimes(1);
expect(createWebSearchMock).toHaveBeenCalledTimes(1);
expect(result.hints.warnings).toEqual([
expect.stringContaining('vectorStores'),
expect.stringContaining('memory'),
]);
});
it('flags the chat integrations that the eval drops from a called agent', async () => {
mockExecuteAgent.mockImplementation(async (...args: unknown[]) => {
await (args[9] as PrepareWorkflowAgentForEval)(agentConfig, { hasChatIntegrations: true });
return { response: 'done' };
});
callAgentsDuringRun(['thread-1', { callingNodeName: callingNode }]);
const result = await service.executeWithLlmMock('wf-1', makeUser());
expect(result.hints.warnings).toContainEqual(
expect.stringContaining('turned off its chat integrations'),
);
});
it("flags sub-agent features and mocks agents that the called agent's tools call", async () => {
mockExecuteAgent.mockImplementation(async (...args: unknown[]) => {
const { instrumentation } = await (args[9] as PrepareWorkflowAgentForEval)(agentConfig);
const toolData = makeMockedAdditionalData() as AgentCallData;
instrumentation.configureToolAdditionalData?.(toolData as never, {
toolName: 'triage',
toolKind: 'workflow',
});
const { callingNodeName } = args[7] as { callingNodeName: string };
if (callingNodeName === callingNode) {
instrumentation.transformDelegatedAgentConfig?.(agentConfig, { subAgentId: 'child-1' });
await toolData.executeAgent(
{ agentId: 'agent-2' },
'hi',
'exec-1',
'thread-2',
toolData,
'evaluation',
undefined,
{ callingNodeName: 'Inner Message' },
undefined,
);
} else {
await toolData.evalLlmMockHandler?.(
{ url: 'https://slack.com/api/chat.postMessage', method: 'POST' },
toolNode,
);
}
return { response: 'done' };
});
callAgentsDuringRun(['thread-1', { callingNodeName: callingNode }]);
const result = await service.executeWithLlmMock('wf-1', makeUser());
expect(
result.nodeResults[callingNode].interceptedRequests.map((request) => request.nodeType),
).toEqual(['n8n-nodes-base.slack']);
expect(result.nodeResults['Inner Message']).toBeUndefined();
expect(result.hints.warnings).toContainEqual(expect.stringContaining('sub-agent child-1'));
});
it("turns off a sub-agent's MCP servers that the calling agent's mock does not cover", async () => {
let childConfig: AgentJsonConfig | undefined;
mockExecuteAgent.mockImplementation(async (...args: unknown[]) => {
const { instrumentation } = await (args[9] as PrepareWorkflowAgentForEval)(agentConfig);
childConfig = instrumentation.transformDelegatedAgentConfig?.(
{
...agentConfig,
memory: undefined,
vectorStores: undefined,
mcpServers: [
{ name: 'linear', url: 'https://mcp.linear.app/mcp' },
{ name: 'github', url: 'https://mcp.github.com/mcp' },
],
} as unknown as AgentJsonConfig,
{ subAgentId: 'child-1' },
);
return { response: 'done' };
});
callAgentsDuringRun(['thread-1', { callingNodeName: callingNode }]);
const result = await service.executeWithLlmMock('wf-1', makeUser());
expect(childConfig?.mcpServers).toEqual([
{ name: 'linear', url: 'https://mcp.linear.app/mcp' },
]);
expect(result.hints.warnings).toContainEqual(
expect.stringMatching(/sub-agent child-1's MCP server "github"/),
);
});
});
// ── reserved node names ──────────────────────────────────────────
describe('reserved node names', () => {
packages/cli/src/modules/instance-ai/eval/__tests__/mock-handler.test.ts
+17
View File
@@ -270,6 +270,23 @@ describe('createLlmMockHandler', () => {
expect(sniffed?.ext).toBe('pdf');
});
it('should write textBody into a binary PDF response', async () => {
llmSubmits({
type: 'binary',
contentType: 'application/pdf',
filename: 'invoice.pdf',
textBody: 'Invoice Number: INV-2026-0042',
});
const handler = createLlmMockHandler();
const result = await callHandler(handler);
const sniffed = await fileTypeFromBuffer(result.body as Buffer);
expect(sniffed?.mime).toBe('application/pdf');
expect((result.body as Buffer).toString('latin1')).toContain(
'(Invoice Number: INV-2026-0042) Tj',
);
});
it('should populate content-disposition and content-length headers for binary responses', async () => {
llmSubmits({ type: 'binary', contentType: 'image/png', filename: 'logo.png' });
const handler = createLlmMockHandler();
packages/cli/src/modules/instance-ai/eval/__tests__/workflow-analysis.test.ts
+26
View File
@@ -1139,6 +1139,32 @@ describe('generateMockHints', () => {
expect(result.warnings).toEqual([]);
});
it('should unwrap trigger content that comes back as an n8n item', async () => {
const binary = { CV_0: { mimeType: 'application/pdf', fileName: 'cv.pdf' } };
mockAgentResponses(
JSON.stringify({
globalContext: '',
triggerContent: { json: { 'Full Name': 'Ada Lovelace' }, binary },
nodeHints: { Slack: 'post a message' },
}),
);
const result = await generateMockHints({ workflow, nodeNames: ['Schedule', 'Slack'] });
expect(result.triggerContent).toEqual({ 'Full Name': 'Ada Lovelace', binary });
});
it('should keep trigger content whose json field sits next to other fields', async () => {
const triggerContent = { json: { id: 1 }, headers: {}, body: {} };
mockAgentResponses(
JSON.stringify({ globalContext: '', triggerContent, nodeHints: { Slack: 'post' } }),
);
const result = await generateMockHints({ workflow, nodeNames: ['Schedule', 'Slack'] });
expect(result.triggerContent).toEqual(triggerContent);
});
it('should retry when the first attempt returns empty triggerContent, then succeed', async () => {
const generate = mockAgentResponses(
JSON.stringify({ globalContext: '', triggerContent: {}, nodeHints: { Slack: 'foo' } }),
packages/cli/src/modules/instance-ai/eval/agent-execution.service.ts
+54 -48
View File
@@ -94,6 +94,55 @@ export function mcpUrlsMatch(configUrl: string, remoteUrl: string): boolean {
return longer === shorter || longer === base || longer.startsWith(`${base}/`);
}
/**
* Resolve canonical tool catalogs for configured MCP servers, so the mock
* exposes the tools the real server would: registry entries carry the
* server's declared catalog (matched by remote URL), and an allow-mode
* toolFilter pins names even without a registry match. Best-effort —
* without a match the mock's LLM-designed catalog applies.
*/
export async function resolveCanonicalMcpCatalogs(
mcpServers: NonNullable<AgentJsonConfig['mcpServers']>,
moduleRegistry: ModuleRegistry,
logger: Logger,
): Promise<Record<string, McpMockCanonicalTool[]> | undefined> {
if (mcpServers.length === 0) return undefined;
const result: Record<string, McpMockCanonicalTool[]> = {};
if (moduleRegistry.isActive('mcp-registry')) {
try {
const entries = await Container.get(McpRegistryService).getAll();
for (const server of mcpServers) {
const entry = entries.find((candidate) =>
candidate.remotes.some((remote) => mcpUrlsMatch(server.url, remote.url)),
);
if (entry && entry.tools.length > 0) {
result[server.name] = entry.tools.map((tool) => ({
name: tool.name,
description: tool.title ?? tool.name,
}));
}
}
} catch (error) {
logger.debug(
`[EvalAgentMock] MCP registry catalog lookup failed: ${error instanceof Error ? error.message : String(error)}`,
);
}
}
for (const server of mcpServers) {
if (result[server.name]) continue;
if (server.toolFilter?.mode === 'allow' && server.toolFilter.tools.length > 0) {
result[server.name] = server.toolFilter.tools.map((name) => ({
name,
description: name,
}));
}
}
return Object.keys(result).length > 0 ? result : undefined;
}
@Service()
export class EvalAgentExecutionService {
constructor(
@@ -216,7 +265,11 @@ export class EvalAgentExecutionService {
// Always created — delegated sub-agents may bring their own servers.
const mcpServers = config.mcpServers ?? [];
const knownToolsByServer = await this.resolveCanonicalMcpCatalogs(mcpServers);
const knownToolsByServer = await resolveCanonicalMcpCatalogs(
mcpServers,
this.moduleRegistry,
this.logger,
);
const mcpFetch = createMcpMockFetch({
servers: mcpServers.map((server) => ({
name: server.name,
@@ -493,53 +546,6 @@ export class EvalAgentExecutionService {
};
}
/**
* Resolve canonical tool catalogs for configured MCP servers, so the mock
* exposes the tools the real server would: registry entries carry the
* server's declared catalog (matched by remote URL), and an allow-mode
* toolFilter pins names even without a registry match. Best-effort —
* without a match the mock's LLM-designed catalog applies.
*/
private async resolveCanonicalMcpCatalogs(
mcpServers: NonNullable<AgentJsonConfig['mcpServers']>,
): Promise<Record<string, McpMockCanonicalTool[]> | undefined> {
if (mcpServers.length === 0) return undefined;
const result: Record<string, McpMockCanonicalTool[]> = {};
if (this.moduleRegistry.isActive('mcp-registry')) {
try {
const entries = await Container.get(McpRegistryService).getAll();
for (const server of mcpServers) {
const entry = entries.find((candidate) =>
candidate.remotes.some((remote) => mcpUrlsMatch(server.url, remote.url)),
);
if (entry && entry.tools.length > 0) {
result[server.name] = entry.tools.map((tool) => ({
name: tool.name,
description: tool.title ?? tool.name,
}));
}
}
} catch (error) {
this.logger.debug(
`[EvalAgentMock] MCP registry catalog lookup failed: ${error instanceof Error ? error.message : String(error)}`,
);
}
}
for (const server of mcpServers) {
if (result[server.name]) continue;
if (server.toolFilter?.mode === 'allow' && server.toolFilter.tools.length > 0) {
result[server.name] = server.toolFilter.tools.map((name) => ({
name,
description: name,
}));
}
}
return Object.keys(result).length > 0 ? result : undefined;
}
/**
* Wraps the shared mock handler to record intercepted requests against the
* owning tool — the agent-eval analog of the workflow eval's
packages/cli/src/modules/instance-ai/eval/execution.service.ts
+232 -17
View File
@@ -1,10 +1,12 @@
import {
type AgentJsonConfig,
EVAL_VENDOR_SDK_INTERCEPTION_FLAG,
type InstanceAiEvalExecutionRequest,
type InstanceAiEvalInterceptedRequest,
type InstanceAiEvalNodeResult,
type InstanceAiEvalExecutionResult,
} from '@n8n/api-types';
import { Logger } from '@n8n/backend-common';
import { Logger, ModuleRegistry } from '@n8n/backend-common';
import { ensureHostsBypassProxy } from '@n8n/backend-network/proxy';
import { ExecutionsConfig, InstanceAiConfig } from '@n8n/config';
import { ProcessedDataRepository, type User } from '@n8n/db';
@@ -38,6 +40,7 @@ import {
MANUAL_TRIGGER_NODE_TYPE,
NodeHelpers,
TimeoutExecutionCancelledError,
UnexpectedError,
UserError,
Workflow,
type IWorkflowIssues,
@@ -46,16 +49,24 @@ import { randomUUID } from 'node:crypto';
import { ActiveExecutions } from '@/active-executions';
import { LoadNodesAndCredentials } from '@/load-nodes-and-credentials';
import type {
AgentRuntimeInstrumentation,
PrepareWorkflowAgentForEval,
} from '@/modules/agents/agent-runtime-instrumentation';
import { DataTableService } from '@/modules/data-table/data-table.service';
import { NodeTypes } from '@/node-types';
import { PostHogClient } from '@/posthog';
import { OwnershipService } from '@/services/ownership.service';
import { executeAgent } from '@/workflow-execute-additional-data';
import { WorkflowRunner } from '@/workflow-runner';
import { WorkflowFinderService } from '@/workflows/workflow-finder.service';
import { WorkflowStaticDataService } from '@/workflows/workflow-static-data.service';
import { pruneConfigForEval, resolveCanonicalMcpCatalogs } from './agent-execution.service';
import { createLlmCompletionMockHandler } from './llm-completion-mock';
import { EvalMockedCredentialsHelper } from './eval-mocked-credentials-helper';
import { createMcpMockFetch } from './mcp-mock-fetch';
import { createWebSearchMock } from './web-search-mock';
import { EvalTimings } from './eval-timings';
import { snapshotLedgerBody } from './ledger-snapshot';
import { type InterceptedTurn, LlmWireServer } from './llm-wire-server';
@@ -95,6 +106,22 @@ interface RunBudget {
deadlineAt: number;
}
type CalledAgentMocks = Pick<AgentRuntimeInstrumentation, 'mcpFetch' | 'webSearch'>;
/** What the agents a workflow calls share during one eval run. */
interface CalledAgentRun {
mockHandler: EvalLlmMockHandler;
nodeResults: Record<string, InstanceAiEvalNodeResult>;
timings: EvalTimings;
hints: MockHints;
scenarioHints?: string;
credentialsHelper: EvalMockedCredentialsHelper;
/** MCP and web-search fakes by calling node and agent config. */
mocks: Map<string, Promise<CalledAgentMocks>>;
/** Agent sessions this run has already called. */
sessions: Set<string>;
}
/** A Data Table node's locator: the id it carries, or the name when it is in `name` mode. */
function dataTableLocator(node: INode): { mode: 'name' | 'id'; value: string } | undefined {
const locator = node.parameters?.dataTableId as
@@ -131,6 +158,7 @@ export class EvalExecutionService {
private readonly dataTableService: DataTableService,
private readonly processedDataRepository: ProcessedDataRepository,
private readonly instanceAiConfig: InstanceAiConfig,
private readonly moduleRegistry: ModuleRegistry,
) {}
async executeWithLlmMock(
@@ -652,18 +680,29 @@ export class EvalExecutionService {
executionData,
pinData,
configureAdditionalData: (additionalData: IWorkflowExecuteAdditionalData) => {
credentialsHelper = new EvalMockedCredentialsHelper(
const runCredentialsHelper = new EvalMockedCredentialsHelper(
additionalData.credentialsHelper,
serverUrl,
this.logger,
vendorLlmRouting?.subNodeToRoot,
);
additionalData.credentialsHelper = credentialsHelper;
credentialsHelper = runCredentialsHelper;
additionalData.credentialsHelper = runCredentialsHelper;
additionalData.evalLlmMockHandler = this.createInterceptingHandler(
mockHandler,
nodeResults,
timings,
);
additionalData.executeAgent = this.createEvalExecuteAgent({
mockHandler,
nodeResults,
timings,
hints,
scenarioHints,
credentialsHelper: runCredentialsHelper,
mocks: new Map(),
sessions: new Set(),
});
},
};
@@ -934,7 +973,12 @@ export class EvalExecutionService {
const item: INodeExecutionData = { json: triggerContent as IDataObject };
const binary: IBinaryKeyData = {};
if (binaryRequirement) {
// The embedded files are what the trigger emits; the requirement only fills in when they name none.
const embeddedKeys = Object.keys(embedded);
if (
binaryRequirement &&
(embeddedKeys.length === 0 || embeddedKeys.includes(binaryRequirement.propertyName))
) {
// Requirement wins for its key, except embedded meta beats the generic fallback.
const embeddedMeta = embedded[binaryRequirement.propertyName];
const isGenericFallback =
@@ -943,6 +987,7 @@ export class EvalExecutionService {
binary[binaryRequirement.propertyName] = synthesizeBinaryEntry(
(isGenericFallback && embeddedMeta?.mimeType) || binaryRequirement.contentType,
(isGenericFallback && embeddedMeta?.fileName) || binaryRequirement.filename,
embeddedMeta?.text,
);
}
@@ -951,6 +996,7 @@ export class EvalExecutionService {
binary[key] = synthesizeBinaryEntry(
meta.mimeType ?? 'application/octet-stream',
meta.fileName ?? 'input.bin',
meta.text,
);
}
@@ -1021,31 +1067,197 @@ export class EvalExecutionService {
);
}
/**
* Replaces `executeAgent` for a run. `attributeTo` records the calls of an
* agent that a called agent's tool runs under that called agent's node.
*/
private createEvalExecuteAgent(
run: CalledAgentRun,
attributeTo?: string,
): NonNullable<IWorkflowExecuteAdditionalData['executeAgent']> {
return async (
source,
message,
executionId,
threadId,
agentAdditionalData,
mode,
outputSchema,
workflowContext,
invocationContext,
) => {
if (!workflowContext) {
throw new UnexpectedError('An eval run cannot attribute an agent call without its node');
}
const callingNode = attributeTo ?? workflowContext.callingNodeName;
const session = `${source.agentId ?? `inline:${workflowContext.callingNodeName}`}:${threadId}`;
const continuesSession =
workflowContext.hasCallerSessionId === true && run.sessions.has(session);
run.sessions.add(session);
return await executeAgent(
source,
message,
executionId,
threadId,
agentAdditionalData,
mode,
outputSchema,
workflowContext,
invocationContext,
this.prepareCalledAgent(callingNode, run, continuesSession),
);
};
}
/**
* Fakes the I/O of an agent the workflow calls, as the Agent eval does: tools,
* MCP servers and web search are mocked; the model call stays real.
*/
private prepareCalledAgent(
callingNode: string,
run: CalledAgentRun,
continuesSession: boolean,
): PrepareWorkflowAgentForEval {
const warn = (feature: string, reason: string) => {
const warning = `"${callingNode}" calls an Agent; the eval turned off its ${feature}: ${reason}`;
if (!run.hints.warnings.includes(warning)) run.hints.warnings.push(warning);
};
return async (original: AgentJsonConfig, { hasChatIntegrations } = {}) => {
const { config, skippedFeatures } = pruneConfigForEval(original);
if (hasChatIntegrations) {
warn('chat integrations', 'an eval run cannot fake the chat SDKs they post through.');
}
for (const { feature, reason } of skippedFeatures) {
// Memory matters only to a call that continues an earlier session of this run.
if (feature !== 'memory') warn(feature, reason);
else if (continuesSession) warn(feature, 'this call does not see its earlier turns.');
}
return {
config,
instrumentation: {
...(await this.calledAgentMocks(callingNode, config, run)),
configureToolAdditionalData: (toolData) => {
toolData.credentialsHelper = run.credentialsHelper;
toolData.evalLlmMockHandler = this.createInterceptingHandler(
run.mockHandler,
run.nodeResults,
run.timings,
callingNode,
);
toolData.executeAgent = this.createEvalExecuteAgent(run, callingNode);
},
transformDelegatedAgentConfig: (childConfig, { subAgentId }) => {
const pruned = pruneConfigForEval(childConfig);
for (const { feature, reason } of pruned.skippedFeatures) {
if (feature !== 'memory') warn(`sub-agent ${subAgentId}'s ${feature}`, reason);
}
// The MCP mock knows only the calling agent's servers; another server
// would get a catalog invented without its own description.
const mocked = new Set((config.mcpServers ?? []).map((server) => server.url));
const mcpServers = pruned.config.mcpServers?.filter((server) => {
if (mocked.has(server.url)) return true;
warn(
`sub-agent ${subAgentId}'s MCP server "${server.name}"`,
'the eval mocks only the MCP servers of the agent the workflow calls.',
);
return false;
});
return mcpServers ? { ...pruned.config, mcpServers } : pruned.config;
},
},
};
};
}
/** One MCP and web-search fake per node and agent config, so repeated calls share one MCP catalog. */
private async calledAgentMocks(
callingNode: string,
config: AgentJsonConfig,
run: CalledAgentRun,
): Promise<CalledAgentMocks> {
const mcpServers = config.mcpServers ?? [];
const key = JSON.stringify([callingNode, config.instructions, mcpServers]);
const cached = run.mocks.get(key);
if (cached) return await cached;
const record = (request: InstanceAiEvalInterceptedRequest) => {
this.mockedEntry(run.nodeResults, callingNode).interceptedRequests.push(request);
};
const mockContext = {
agentInstructions: config.instructions,
scenarioHints: run.scenarioHints,
globalContext: run.hints.globalContext,
logger: this.logger,
};
const mocks = (async (): Promise<CalledAgentMocks> => ({
mcpFetch: createMcpMockFetch({
...mockContext,
servers: mcpServers.map(({ name, url, description }) => ({ name, url, description })),
knownToolsByServer: await resolveCanonicalMcpCatalogs(
mcpServers,
this.moduleRegistry,
this.logger,
),
onToolCall: (call) =>
record({
url:
mcpServers.find((server) => server.name === call.serverName)?.url ?? call.serverName,
method: 'POST',
nodeType: `mcp:${call.serverName}`,
requestBody: call.args,
mockResponse: call.result,
}),
}),
webSearch: createWebSearchMock({
...mockContext,
onSearch: (args, result) =>
record({
url: 'mock://web-search',
method: 'POST',
nodeType: 'web-search:fallback',
requestBody: args,
mockResponse: result,
}),
}),
}))();
run.mocks.set(key, mocks);
return await mocks;
}
/** A node's entry, marked mocked: `checkNodeConfig` may have pre-created it as 'real'. */
private mockedEntry(
nodeResults: Record<string, InstanceAiEvalNodeResult>,
nodeName: string,
): InstanceAiEvalNodeResult {
const entry = (nodeResults[nodeName] ??= {
outputs: {},
outputCount: 0,
iterationCount: 0,
interceptedRequests: [],
executionMode: 'mocked',
});
entry.executionMode = 'mocked';
return entry;
}
/**
* Wraps the mock handler to collect intercepted request metadata for diagnostics.
* `attributeTo` records a called agent's tool requests under the calling node.
*/
private createInterceptingHandler(
mockHandler: EvalLlmMockHandler,
nodeResults: Record<string, InstanceAiEvalNodeResult>,
timings: EvalTimings,
attributeTo?: string,
): EvalLlmMockHandler {
return async (
requestOptions: IHttpRequestOptions,
node: INode,
): Promise<EvalMockHttpResponse | undefined> => {
// A node may make multiple HTTP requests — ensure it's marked as mocked.
// checkNodeConfig may have pre-created the entry as 'real', so always override.
const entry = (nodeResults[node.name] ??= {
outputs: {},
outputCount: 0,
iterationCount: 0,
interceptedRequests: [],
executionMode: 'mocked',
});
entry.executionMode = 'mocked';
const entry = this.mockedEntry(nodeResults, attributeTo ?? node.name);
let response = await timings.time(
'http-mock',
node.name,
attributeTo ?? node.name,
async () => await mockHandler(requestOptions, node),
);
@@ -1333,8 +1545,8 @@ function withInterceptionGaps(
}
/** Synthesize a structurally valid binary entry (real bytes, base64-inlined). */
function synthesizeBinaryEntry(contentType: string, filename: string): IBinaryData {
const bytes = synthesizeBinaryFixture(contentType, filename);
function synthesizeBinaryEntry(contentType: string, filename: string, text?: string): IBinaryData {
const bytes = synthesizeBinaryFixture(contentType, filename, { text });
const extension = filename.includes('.') ? filename.slice(filename.lastIndexOf('.') + 1) : 'bin';
return {
mimeType: contentType,
@@ -1348,6 +1560,8 @@ function synthesizeBinaryEntry(contentType: string, filename: string): IBinaryDa
interface EmbeddedBinaryMeta {
mimeType?: string;
fileName?: string;
/** The document's text, written into the synthesized file. */
text?: string;
}
function nonEmptyString(value: unknown): string | undefined {
@@ -1382,6 +1596,7 @@ function readEmbeddedBinaryMeta(
mimeType,
// `name` is a fileName fallback, not qualification evidence.
fileName: strictFileName ?? nonEmptyString(v.name),
text: nonEmptyString(v.text),
};
}
packages/cli/src/modules/instance-ai/eval/mock-handler.ts
+6 -3
View File
@@ -76,7 +76,7 @@ Match THIS request only (URL + method): a node may make multiple sequential call
For APIs that return empty responses on success (204/202), call submit_response with type="json" and body={}.
**Binary / file responses.** Pick \`type: "binary"\` when the request URL or node parameters indicate a file download — Telegram \`getFile\` / \`/file/bot...\`, Google Drive \`alt=media\`, Dropbox \`/files/download\`, OneDrive \`/items/{id}/content\`, S3 \`GetObject\`, OpenAI \`audio/transcriptions\` source file, or any path containing \`/download\`, \`/file\`, \`/attachment\`, \`/media\`, \`/image\`, \`/voice\`, \`/audio\`, \`/export\`. Always set \`contentType\` (real MIME like \`application/pdf\`, \`audio/ogg\`, \`image/png\`) and \`filename\` (with the correct extension). Use \`sizeHint\` only when the scenario hints mention file size constraints (e.g. "rejects files > 100KB"). Do NOT pick \`binary\` for JSON metadata endpoints like Slack \`files.upload\`, \`files.info\`, or Telegram \`getFile\` (which returns a JSON envelope describing the file — the binary comes from the follow-up \`/file/bot.../path\` request). Exception: when the document at a download/export path is itself textual (CSV, XML, HTML, plain text) and the scenario or node hints specify its contents, use \`type: "text"\` with the exact document — reserve \`binary\` for opaque formats (PDF, images, audio, video, archives).
**Binary / file responses.** Pick \`type: "binary"\` when the request URL or node parameters indicate a file download — Telegram \`getFile\` / \`/file/bot...\`, Google Drive \`alt=media\`, Dropbox \`/files/download\`, OneDrive \`/items/{id}/content\`, S3 \`GetObject\`, OpenAI \`audio/transcriptions\` source file, or any path containing \`/download\`, \`/file\`, \`/attachment\`, \`/media\`, \`/image\`, \`/voice\`, \`/audio\`, \`/export\`. Always set \`contentType\` (real MIME like \`application/pdf\`, \`audio/ogg\`, \`image/png\`) and \`filename\` (with the correct extension). For a PDF whose content the scenario or node hints state, also put that content in \`textBody\` as plain text — the harness writes it into the PDF; without it the PDF has no text. Use \`sizeHint\` only when the scenario hints mention file size constraints (e.g. "rejects files > 100KB"). Do NOT pick \`binary\` for JSON metadata endpoints like Slack \`files.upload\`, \`files.info\`, or Telegram \`getFile\` (which returns a JSON envelope describing the file — the binary comes from the follow-up \`/file/bot.../path\` request). Exception: when the document at a download/export path is itself textual (CSV, XML, HTML, plain text) and the scenario or node hints specify its contents, use \`type: "text"\` with the exact document — reserve \`binary\` for opaque formats (PDF, images, audio, video, archives).
**Raw text / non-JSON responses.** When the real endpoint returns a non-JSON text document — XML/SOAP, CSV, HTML, RSS/Atom, plain text — use \`type: "text"\` and put the EXACT raw document in \`textBody\`, with \`contentType\` set to the real MIME (\`text/xml\`, \`application/xml\`, \`text/csv\`, \`text/html\`, \`text/plain\`). NEVER wrap such a document inside a JSON object like \`{"data": "<?xml..."}\` — the node consumes the raw text, and a JSON wrapper corrupts it. SOAP endpoints return the full SOAP envelope as \`textBody\`; SOAP faults use \`type: "error"\` with the fault XML in \`textBody\`.`;
@@ -628,7 +628,7 @@ const submitResponseSchema = z.object({
.string()
.optional()
.describe(
'The raw text document exactly as the service would send it (e.g. "<?xml version=\\"1.0\\"?>..."). Required for type="text"; allowed with type="error" when the service returns a text/XML error document. Omit otherwise.',
'The raw text document exactly as the service would send it (e.g. "<?xml version=\\"1.0\\"?>..."). Required for type="text"; allowed with type="error" when the service returns a text/XML error document; with type="binary" and a PDF, the document\'s text as plain text. Omit otherwise.',
),
statusCode: z
.number()
@@ -1019,7 +1019,10 @@ function materializeSpec(spec: MockResponseSpec): EvalMockHttpResponse {
case 'binary': {
const filename = spec.filename ?? 'mock-file.dat';
const contentType = spec.contentType ?? 'application/octet-stream';
const body = synthesizeBinaryFixture(contentType, filename, { sizeHint: spec.sizeHint });
const body = synthesizeBinaryFixture(contentType, filename, {
sizeHint: spec.sizeHint,
text: spec.textBody,
});
return {
body,
headers: {
packages/cli/src/modules/instance-ai/eval/workflow-analysis.ts
+15 -5
View File
@@ -2,6 +2,7 @@ import { extractJsonCandidate } from '@n8n/ai-utilities/llm-output';
import { Logger } from '@n8n/backend-common';
import { Container } from '@n8n/di';
import { createEvalAgent, extractText } from '@n8n/instance-ai';
import { isRecord } from '@n8n/utils/is-record';
// AI root node types (single source in @n8n/workflow-sdk mock-data) — lets
// the typo guard accept a no-sub-node Agent.
import { isAiRootNodeType } from '@n8n/workflow-sdk';
@@ -613,7 +614,7 @@ RULES:
- CRITICAL: apart from a Manual Trigger, triggerContent must NEVER be an empty object ({}). Even for scenarios that test empty payloads ("empty submission", "no data", "missing fields"), emit the trigger envelope with empty *nested* fields — an empty webhook is { headers: {}, query: {}, body: {} }, a schedule with no context is { timestamp: "..." }. The workflow cannot execute without trigger output. The one exception is a polling or event trigger that the scenario says has NOTHING to emit ("no new emails", "no new rows", "no results"): then set "triggerEmitsNoItems": true and omit triggerContent — the harness pins the trigger to zero items so downstream nodes do not run.
- CRITICAL: check what downstream nodes reference (e.g., $json.body.email, $json.subject, $json.text) and ensure those paths exist in triggerContent
- CRITICAL: when the workflow has MULTIPLE trigger nodes, pick the ONE the Test Scenario targets (the trigger whose firing the scenario describes, e.g. "The weekly Schedule Trigger fires") and return its exact node name in a "startNodeName" field. triggerContent must be THAT trigger's output.
- CRITICAL: triggerContent must NEVER contain binary file CONTENT — no base64 blobs, no fake file-bytes placeholders. When the trigger carries a file (form upload, email attachment, incoming media), declare it with a METADATA-ONLY binary map instead: "binary": { "<propertyKey>": { "mimeType": "<real MIME>", "fileName": "<name.ext>" } } — the harness synthesizes real file bytes from that metadata and attaches them at the item level. The MIME type and file name MUST match the scenario: an image/png upload scenario needs mimeType "image/png" and a .png fileName, never a generic application/octet-stream. Use "data" as the propertyKey unless downstream nodes reference a different binary property name.
- CRITICAL: triggerContent must NEVER contain binary file CONTENT — no base64 blobs, no fake file-bytes placeholders. When the trigger carries a file (form upload, email attachment, incoming media), declare it with a METADATA-ONLY binary map instead: "binary": { "<propertyKey>": { "mimeType": "<real MIME>", "fileName": "<name.ext>" } } — the harness synthesizes real file bytes from that metadata and attaches them at the item level. For a PDF or text file whose words the scenario states, also add "text": "<those words as plain text>" to its entry — the harness writes them into the file. The MIME type and file name MUST match the scenario: an image/png upload scenario needs mimeType "image/png" and a .png fileName, never a generic application/octet-stream. Use "data" as the propertyKey unless downstream nodes reference a different binary property name.
3. Create a "nodeHints" object with one entry per node. Each hint describes what data that specific node's API response should contain, referencing entities from the global context.
4. Hints should describe the DATA CONTENT, not the API response format. The mock server already knows the API schema.
5. Ensure data flows logically through the workflow. If node A fetches items that node B processes, the items in A's hint should match what B expects.
@@ -704,6 +705,14 @@ function isManualTriggerNode(workflow: IWorkflowBase, nodeName?: string): boolea
return node?.type === MANUAL_TRIGGER_NODE_TYPE && !node.disabled;
}
/** The model sometimes answers with an n8n item (`{ json, binary }`) instead of the item's json. */
function unwrapItemShape(content: object): object {
if (!isRecord(content)) return content;
const { json, ...rest } = content;
if (!isRecord(json) || Object.keys(rest).some((key) => key !== 'binary')) return content;
return rest.binary === undefined ? json : { ...json, binary: rest.binary };
}
/** One LLM call → globalContext + triggerContent + per-node hints. Retried once on structural issues. */
export async function generateMockHints(options: GenerateMockHintsOptions): Promise<MockHints> {
const { workflow, nodeNames, scenarioHints } = options;
@@ -759,12 +768,13 @@ export async function generateMockHints(options: GenerateMockHintsOptions): Prom
) {
reason = `invalid nodeHints structure (raw: ${text.slice(0, 200)})`;
} else {
const triggerContent =
const triggerContent = unwrapItemShape(
typeof parsed.triggerContent === 'object' &&
parsed.triggerContent !== null &&
!Array.isArray(parsed.triggerContent)
parsed.triggerContent !== null &&
!Array.isArray(parsed.triggerContent)
? parsed.triggerContent
: {};
: {},
);
// The model answers a bool as a word often enough to read both spellings.
const triggerEmitsNoItems =
parsed.triggerEmitsNoItems === true || parsed.triggerEmitsNoItems === 'true';
packages/cli/src/modules/type-availability-policies/README.md
+10 -1
View File
@@ -188,7 +188,8 @@ A policy lives in three tables:
- **attachments**, which bind a document to a scope with a priority and a floor flag.
`projectId: null` is the instance scope. An unwritten scope has no row: it allows everything and
reports version `0`, so a first write sends `expectedVersion: 0`.
reports version `0`, so a first write sends `expectedVersion: 0`. One accepted write moves the
version by exactly one, whatever it changed.
Evaluation is `instance ∩ project`, and a project can only restrict further. `delegate` is the
one exception: an instance `delegate` is satisfied only by an explicit project `allow` rule,
@@ -205,6 +206,14 @@ lint warns about it. The verdict names the variant the user placed and the rule
Grandfathering compares literal type names, so adding `gmailTool` to a workflow that already
stores `gmail` is a new type and is judged.
A credential type policy has a third selector, `extends`. It matches the named type and every
type built on it through the credential's `extends` list, at any depth: `extends oAuth2Api`
covers `googleOAuth2Api` and `googleSheetsOAuth2Api`. A `name` rule stays exact, so
`name oAuth2Api` blocks only the generic OAuth2 credential. Rule order decides as usual: an
`allow name googleSheetsOAuth2Api` placed before `deny extends oAuth2Api` keeps Sheets usable,
and placed after it, the shadow lint warns that it can never match. A write rejects an `extends`
rule in a node type policy, and one that names a credential type that is not installed.
### Reading it on the execution path
`workflowStart` runs for every execution and every sub-execution, under a 250 ms deadline it
packages/cli/src/modules/type-availability-policies/__tests__/package-resolver.test.ts
+80 -1
View File
@@ -1,7 +1,13 @@
import type { LoadNodesAndCredentials } from '@/load-nodes-and-credentials';
import type { NodeTypes } from '@/node-types';
import { CREDENTIAL_TYPES_KIND, NODE_TYPES_KIND } from '../constants';
import { isPackageInstalled, packageResolverFor } from '../package-resolver';
import {
isCredentialTypeKnown,
isPackageInstalled,
packageResolverFor,
policedTypeFor,
} from '../package-resolver';
/** A minimal stand-in for `LoadNodesAndCredentials`, built from just its public `loaders`. */
function makeRegistry(loaders: LoadNodesAndCredentials['loaders']): LoadNodesAndCredentials {
@@ -89,3 +95,76 @@ describe('isPackageInstalled', () => {
expect(isPackageInstalled(makeRegistry({}), 'constructor')).toBe(false);
});
});
describe('policedTypeFor', () => {
const withKnownCredentials = (
knownCredentials: LoadNodesAndCredentials['knownCredentials'],
): LoadNodesAndCredentials => ({ knownCredentials }) as LoadNodesAndCredentials;
const nodeTypes = {} as NodeTypes;
it('lists every credential type a type is built on, nearest first', () => {
const policedType = policedTypeFor(
CREDENTIAL_TYPES_KIND,
nodeTypes,
withKnownCredentials({
oAuth2Api: { className: '', sourcePath: '' },
googleOAuth2Api: { className: '', sourcePath: '', extends: ['oAuth2Api'] },
googleSheetsOAuth2Api: { className: '', sourcePath: '', extends: ['googleOAuth2Api'] },
}),
);
expect(policedType('googleSheetsOAuth2Api')).toEqual({
name: 'googleSheetsOAuth2Api',
baseName: 'googleSheetsOAuth2Api',
ancestors: ['googleOAuth2Api', 'oAuth2Api'],
});
});
it('lists a shared base once when two parents build on it', () => {
const policedType = policedTypeFor(
CREDENTIAL_TYPES_KIND,
nodeTypes,
withKnownCredentials({
left: { className: '', sourcePath: '', extends: ['root'] },
right: { className: '', sourcePath: '', extends: ['root'] },
both: { className: '', sourcePath: '', extends: ['left', 'right'] },
}),
);
expect(policedType('both').ancestors).toEqual(['left', 'right', 'root']);
});
it('stops on a cycle instead of looping', () => {
const policedType = policedTypeFor(
CREDENTIAL_TYPES_KIND,
nodeTypes,
withKnownCredentials({
aApi: { className: '', sourcePath: '', extends: ['bApi'] },
bApi: { className: '', sourcePath: '', extends: ['aApi'] },
}),
);
expect(policedType('aApi').ancestors).toEqual(['bApi']);
});
it('gives an unknown credential type no ancestors', () => {
const policedType = policedTypeFor(CREDENTIAL_TYPES_KIND, nodeTypes, withKnownCredentials({}));
expect(policedType('toString').ancestors).toEqual([]);
});
});
describe('isCredentialTypeKnown', () => {
const registry = {
knownCredentials: { slackApi: { className: '', sourcePath: '' } },
} as unknown as LoadNodesAndCredentials;
it('is true for a known credential type', () => {
expect(isCredentialTypeKnown(registry, 'slackApi')).toBe(true);
});
it('is false for an unknown type, or one named after an Object.prototype property', () => {
expect(isCredentialTypeKnown(registry, 'unknownApi')).toBe(false);
expect(isCredentialTypeKnown(registry, 'constructor')).toBe(false);
});
});
packages/cli/src/modules/type-availability-policies/__tests__/policy-evaluator.test.ts
+112
View File
@@ -253,6 +253,118 @@ describe('evaluateType', () => {
});
});
describe('extends selector', () => {
const SHEETS: PolicedType = {
name: 'googleSheetsOAuth2Api',
baseName: 'googleSheetsOAuth2Api',
ancestors: ['googleOAuth2Api', 'oAuth2Api'],
};
const GENERIC: PolicedType = { name: 'oAuth2Api', baseName: 'oAuth2Api', ancestors: [] };
const denyOAuth2Family: PolicyRule = {
id: 'deny-oauth2-family',
action: 'deny',
selector: { kind: 'extends', value: 'oAuth2Api' },
};
const allowSheets: PolicyRule = {
id: 'allow-sheets',
action: 'allow',
selector: { kind: 'name', value: 'googleSheetsOAuth2Api' },
};
it('matches a type built on the named base, through every step of the chain', () => {
const attachments = [attachment({ rules: [denyOAuth2Family] })];
expect(evaluateType(attachments, 'allow', SHEETS)).toEqual({
action: 'deny',
matchedRuleId: 'deny-oauth2-family',
});
});
it('matches the named base itself', () => {
const attachments = [attachment({ rules: [denyOAuth2Family] })];
expect(evaluateType(attachments, 'allow', GENERIC).action).toBe('deny');
});
it('does not match a type outside the family', () => {
const attachments = [attachment({ rules: [denyOAuth2Family] })];
expect(evaluateType(attachments, 'allow', type('slackApi'))).toEqual({
action: 'allow',
matchedRuleId: null,
});
});
it('leaves a name rule on the base exact, so derived types stay allowed', () => {
const attachments = [
attachment({
rules: [
{ id: 'deny-generic', action: 'deny', selector: { kind: 'name', value: 'oAuth2Api' } },
],
}),
];
expect(evaluateType(attachments, 'allow', SHEETS).action).toBe('allow');
expect(evaluateType(attachments, 'allow', GENERIC).action).toBe('deny');
});
it('lets rule order decide between a derived-type allow and a family deny', () => {
const allowFirst = [attachment({ rules: [allowSheets, denyOAuth2Family] })];
const denyFirst = [attachment({ rules: [denyOAuth2Family, allowSheets] })];
expect(evaluateType(allowFirst, 'allow', SHEETS)).toEqual({
action: 'allow',
matchedRuleId: 'allow-sheets',
});
expect(evaluateType(denyFirst, 'allow', SHEETS)).toEqual({
action: 'deny',
matchedRuleId: 'deny-oauth2-family',
});
});
it('keeps an instance family deny over a project allow of the derived type', () => {
const verdict = evaluateComposedType(
scopePolicy({ attachments: [attachment({ rules: [denyOAuth2Family] })] }),
scopePolicy({ attachments: [attachment({ rules: [allowSheets] })] }),
SHEETS,
);
expect(verdict).toEqual({
action: 'deny',
scope: 'instance',
matchedRuleId: 'deny-oauth2-family',
optInAvailable: false,
});
});
it('satisfies an instance delegate on a derived type with a project family allow', () => {
const verdict = evaluateComposedType(
scopePolicy({ defaultAction: 'delegate' }),
scopePolicy({
attachments: [
attachment({
rules: [
{
id: 'allow-family',
action: 'allow',
selector: { kind: 'extends', value: 'oAuth2Api' },
},
],
}),
],
}),
SHEETS,
);
expect(verdict).toEqual({
action: 'allow',
scope: 'project',
matchedRuleId: 'allow-family',
optInAvailable: false,
});
});
});
describe('evaluateComposedType', () => {
const TYPE = 'n8n-nodes-base.slack';
const allowRule = (id: string) =>
packages/cli/src/modules/type-availability-policies/__tests__/policy-shadow-lint.test.ts
+51
View File
@@ -177,6 +177,57 @@ describe('lintRulesForShadowing', () => {
]);
});
describe('extends selector', () => {
/** `googleSheetsOAuth2Api` is built on `googleOAuth2Api`, which is built on `oAuth2Api`. */
const credentialType = (name: string) => ({
name,
baseName: name,
ancestors:
{
googleSheetsOAuth2Api: ['googleOAuth2Api', 'oAuth2Api'],
googleOAuth2Api: ['oAuth2Api'],
}[name] ?? [],
});
const lint = (rules: PolicyRule[]) => lintRulesForShadowing(rules, () => null, credentialType);
const family = (id: string, value: string) =>
rule({ id, selector: { kind: 'extends', value } });
const name = (id: string, value: string) => rule({ id, selector: { kind: 'name', value } });
it('flags a later name rule for a type built on an earlier family rule', () => {
expect(
lint([family('oauth2', 'oAuth2Api'), name('sheets', 'googleSheetsOAuth2Api')]),
).toEqual([{ ruleId: 'sheets', shadowedByRuleId: 'oauth2' }]);
});
it('flags a later name rule for the family base itself', () => {
expect(lint([family('oauth2', 'oAuth2Api'), name('generic', 'oAuth2Api')])).toEqual([
{ ruleId: 'generic', shadowedByRuleId: 'oauth2' },
]);
});
it('flags a later family rule nested inside an earlier one', () => {
expect(lint([family('oauth2', 'oAuth2Api'), family('google', 'googleOAuth2Api')])).toEqual([
{ ruleId: 'google', shadowedByRuleId: 'oauth2' },
]);
});
it('does not flag a family rule after a name rule for its base, which matches one type only', () => {
expect(lint([name('generic', 'oAuth2Api'), family('oauth2', 'oAuth2Api')])).toEqual([]);
});
it('does not flag a wider family rule after a narrower one', () => {
expect(lint([family('google', 'googleOAuth2Api'), family('oauth2', 'oAuth2Api')])).toEqual(
[],
);
});
it('does not flag a derived-type allow placed before the family rule', () => {
expect(
lint([name('sheets', 'googleSheetsOAuth2Api'), family('oauth2', 'oAuth2Api')]),
).toEqual([]);
});
});
it('returns no warnings for an empty or single-rule list', () => {
expect(lintRulesForShadowing([])).toEqual([]);
expect(
packages/cli/src/modules/type-availability-policies/__tests__/type-availability-policy.service.test.ts
+95 -6
View File
@@ -332,6 +332,67 @@ describe('TypeAvailabilityPolicyService', () => {
expect(policyRepository.createPolicy).not.toHaveBeenCalled();
});
describe('extends rules', () => {
const familyRule = (id: string, value: string): PolicyRule => ({
id,
action: 'deny',
selector: { kind: 'extends', value },
});
beforeEach(() => {
Object.defineProperty(loadNodesAndCredentials, 'knownCredentials', {
configurable: true,
value: {
oAuth2Api: { className: '', sourcePath: '' },
googleSheetsOAuth2Api: { className: '', sourcePath: '', extends: ['oAuth2Api'] },
},
});
});
afterEach(() => {
delete (loadNodesAndCredentials as { knownCredentials?: unknown }).knownCredentials;
});
it('rejects an extends rule in a node type policy', async () => {
await expect(
service.createPolicyDocument(KIND, [familyRule('r1', 'oAuth2Api')], 'user-1'),
).rejects.toThrow('An "extends" rule is only valid in a credential type policy');
expect(policyRepository.createPolicy).not.toHaveBeenCalled();
});
it('rejects an extends rule naming a credential type that is not installed', async () => {
await expect(
service.createPolicyDocument(
CREDENTIAL_TYPES_KIND,
[familyRule('r1', 'notInstalledApi')],
'user-1',
),
).rejects.toThrow(
'Extends rule names a credential type that is not installed: notInstalledApi',
);
expect(policyRepository.createPolicy).not.toHaveBeenCalled();
});
it('warns about a derived-type rule placed after the family rule that covers it', async () => {
policyRepository.createPolicy.mockResolvedValue(
makePolicy({ kind: CREDENTIAL_TYPES_KIND }),
);
const allowSheets: PolicyRule = {
id: 'r2',
action: 'allow',
selector: { kind: 'name', value: 'googleSheetsOAuth2Api' },
};
const { warnings } = await service.createPolicyDocument(
CREDENTIAL_TYPES_KIND,
[familyRule('r1', 'oAuth2Api'), allowSheets],
'user-1',
);
expect(warnings).toEqual([{ ruleId: 'r2', shadowedByRuleId: 'r1' }]);
});
});
it('accepts a package rule naming an installed package', async () => {
policyRepository.createPolicy.mockResolvedValue(makePolicy());
const rule: PolicyRule = {
@@ -901,6 +962,7 @@ describe('TypeAvailabilityPolicyService', () => {
expect(policyRepository.findByIdAndKind.mock.invocationCallOrder[0]).toBeLessThan(
attachmentRepository.listScopeIdsAttachedToPolicy.mock.invocationCallOrder[0],
);
expect(scopeRepository.bumpVersion).toHaveBeenCalledWith(scope.id, ROOT);
});
it('rejects a delegate defaultAction at project scope before opening a transaction', async () => {
@@ -945,7 +1007,7 @@ describe('TypeAvailabilityPolicyService', () => {
});
policyRepository.createPolicy.mockResolvedValue(makePolicy({ rules: [], version: 1 }));
scopeRepository.findScopeById.mockResolvedValue(
makeScope({ defaultAction: 'delegate', version: 2 }),
makeScope({ defaultAction: 'delegate', version: 1 }),
);
await expect(
@@ -1014,7 +1076,7 @@ describe('TypeAvailabilityPolicyService', () => {
const createdPolicy = makePolicy({ rules: [RULE], version: 1 });
policyRepository.createPolicy.mockResolvedValue(createdPolicy);
scopeRepository.findScopeById.mockResolvedValue(
makeScope({ defaultAction: 'deny', version: 2 }),
makeScope({ defaultAction: 'deny', version: 1 }),
);
const result = await service.setEffectivePolicy(
@@ -1025,8 +1087,9 @@ describe('TypeAvailabilityPolicyService', () => {
'user-1',
);
expect(result.version).toBe(2);
expect(result.version).toBe(1);
expect(result.rules).toEqual([RULE]);
expect(scopeRepository.bumpVersion).not.toHaveBeenCalled();
expect(attachmentRepository.replaceAttachmentsForScope).toHaveBeenCalledWith(
createdScope.id,
[{ policyId: createdPolicy.id, priority: 0, isFloor: false }],
@@ -1047,7 +1110,7 @@ describe('TypeAvailabilityPolicyService', () => {
projectId: null,
scopeId: createdScope.id,
before: null,
after: { defaultAction: 'deny', version: 2 },
after: { defaultAction: 'deny', version: 1 },
rulesBefore: null,
rulesAfter: [RULE],
warningCount: 0,
@@ -1065,7 +1128,7 @@ describe('TypeAvailabilityPolicyService', () => {
const updatedPolicy = makePolicy({ rules: [RULE], version: 2 });
policyRepository.updateRules.mockResolvedValue(updatedPolicy);
scopeRepository.findScopeById.mockResolvedValue(
makeScope({ defaultAction: 'deny', version: 3 }),
makeScope({ defaultAction: 'deny', version: 2 }),
);
await service.setEffectivePolicy(
@@ -1082,6 +1145,7 @@ describe('TypeAvailabilityPolicyService', () => {
'user-2',
ROOT,
);
expect(scopeRepository.bumpVersion).not.toHaveBeenCalled();
expect(eventService.emit).toHaveBeenCalledTimes(3);
expect(eventService.emit).toHaveBeenCalledWith(
'node-type-policy-document-updated',
@@ -1097,7 +1161,7 @@ describe('TypeAvailabilityPolicyService', () => {
projectId: null,
scopeId: scope.id,
before: { defaultAction: 'allow', version: 1 },
after: { defaultAction: 'deny', version: 3 },
after: { defaultAction: 'deny', version: 2 },
rulesBefore: [],
rulesAfter: [RULE],
warningCount: 0,
@@ -1171,6 +1235,31 @@ describe('TypeAvailabilityPolicyService', () => {
expect(result.defaultAction).toBe('deny');
expect(result.version).toBe(1);
});
it('moves the version once when neither the rules nor the default action change', async () => {
const scope = makeScope({ defaultAction: 'allow', version: 1 });
scopeRepository.findScopeByKindAndProject.mockResolvedValue(scope);
const existingPolicy = makePolicy({ rules: [RULE], version: 1 });
attachmentRepository.listAttachmentsForScope.mockResolvedValue([
{ policyId: existingPolicy.id, rules: [RULE], priority: 0, isFloor: false },
]);
policyRepository.findByIdAndKind.mockResolvedValue(existingPolicy);
policyRepository.updateRules.mockResolvedValue(existingPolicy);
scopeRepository.findScopeById.mockResolvedValue(
makeScope({ defaultAction: 'allow', version: 2 }),
);
await service.setEffectivePolicy(
KIND,
null,
{ rules: [RULE], defaultAction: 'allow' },
1,
'user-2',
);
expect(scopeRepository.updateDefaultAction).not.toHaveBeenCalled();
expect(scopeRepository.bumpVersion).toHaveBeenCalledTimes(1);
});
});
describe('evaluateComposedType', () => {
packages/cli/src/modules/type-availability-policies/credential-type-policy-instance.controller.ts
+6 -6
View File
@@ -1,8 +1,8 @@
import {
CreatePolicyDocumentDto,
PutInstancePolicyDto,
CreateCredentialTypePolicyDocumentDto,
PutCredentialTypeInstancePolicyDto,
ReplaceAttachmentsDto,
UpdatePolicyDocumentDto,
UpdateCredentialTypePolicyDocumentDto,
} from '@n8n/api-types';
import { LICENSE_FEATURES } from '@n8n/constants';
import { AuthenticatedRequest } from '@n8n/db';
@@ -55,7 +55,7 @@ export class CredentialTypePolicyInstanceController {
async putInstancePolicy(
req: AuthenticatedRequest,
_res: Response,
@Body dto: PutInstancePolicyDto,
@Body dto: PutCredentialTypeInstancePolicyDto,
) {
const result = await this.service.setEffectivePolicy(
CREDENTIAL_TYPES_KIND,
@@ -80,7 +80,7 @@ export class CredentialTypePolicyInstanceController {
async createPolicy(
req: AuthenticatedRequest,
_res: Response,
@Body dto: CreatePolicyDocumentDto,
@Body dto: CreateCredentialTypePolicyDocumentDto,
) {
const { policy, warnings } = await this.service.createPolicyDocument(
CREDENTIAL_TYPES_KIND,
@@ -117,7 +117,7 @@ export class CredentialTypePolicyInstanceController {
req: AuthenticatedRequest,
_res: Response,
@Param('policyId') policyId: string,
@Body dto: UpdatePolicyDocumentDto,
@Body dto: UpdateCredentialTypePolicyDocumentDto,
) {
const { policy, warnings } = await this.service.updatePolicyDocument(
CREDENTIAL_TYPES_KIND,
packages/cli/src/modules/type-availability-policies/credential-type-policy-project.controller.ts
+2 -2
View File
@@ -1,4 +1,4 @@
import { PutProjectPolicyDto } from '@n8n/api-types';
import { PutCredentialTypeProjectPolicyDto } from '@n8n/api-types';
import { LICENSE_FEATURES } from '@n8n/constants';
import { AuthenticatedRequest } from '@n8n/db';
import { Body, Get, Licensed, ProjectScope, Put, RestController } from '@n8n/decorators';
@@ -42,7 +42,7 @@ export class CredentialTypePolicyProjectController {
async putProjectPolicy(
req: AuthenticatedRequest<{ projectId: string }>,
_res: Response,
@Body dto: PutProjectPolicyDto,
@Body dto: PutCredentialTypeProjectPolicyDto,
) {
const result = await this.service.setEffectivePolicy(
CREDENTIAL_TYPES_KIND,
packages/cli/src/modules/type-availability-policies/database/repositories/type-availability-policy-scope.repository.ts
+5
View File
@@ -187,6 +187,8 @@ export class TypeAvailabilityPolicyScopeRepository extends BaseRepository<TypeAv
*
* Joins the caller's transaction when `ctx` carries one, so the read-back lock is held
* for the rest of the caller's work; opens one otherwise, since the lock needs it.
*
* The new row starts at version 1, which is the first write's own bump.
*/
async createScopeIfAbsent(
input: NewPolicyScope,
@@ -242,6 +244,9 @@ export class TypeAvailabilityPolicyScopeRepository extends BaseRepository<TypeAv
* Call this under the same `ctx` as whatever changed the scope's *effective* policy —
* an attachment added, removed or reordered, or an attached policy's content edited —
* so the version and the change commit together.
*
* One write moves the version once. Do not call this in a write that also ran
* `createScopeIfAbsent` or `updateDefaultAction`: each of those sets the version itself.
*/
async bumpVersion(id: string, ctx: OperationContext): Promise<void> {
await this.managerFor(ctx).increment(TypeAvailabilityPolicyScope, { id }, 'version', 1);
packages/cli/src/modules/type-availability-policies/package-resolver.ts
+48 -4
View File
@@ -51,10 +51,54 @@ export function packageResolverFor(
: nodeTypePackageResolver;
}
export function policedTypeFor(kind: string, nodeTypes: NodeTypes): (name: string) => PolicedType {
return kind === NODE_TYPES_KIND
? (name) => ({ name, baseName: nodeTypes.resolveBaseName(name).baseName })
: (name) => ({ name, baseName: name });
/**
* Every type `typeName` is built on, walking `extends` breadth-first. The visited set keeps a
* community package that declares a cycle from looping forever.
*/
function credentialTypeAncestors(
loadNodesAndCredentials: LoadNodesAndCredentials,
typeName: string,
): string[] {
const { knownCredentials } = loadNodesAndCredentials;
const parentsOf = (name: string) =>
Object.hasOwn(knownCredentials, name) ? (knownCredentials[name].extends ?? []) : [];
const seen = new Set<string>([typeName]);
const queue = [...parentsOf(typeName)];
for (let index = 0; index < queue.length; index++) {
const parent = queue[index];
if (seen.has(parent)) continue;
seen.add(parent);
queue.push(...parentsOf(parent));
}
seen.delete(typeName);
return [...seen];
}
export function policedTypeFor(
kind: string,
nodeTypes: NodeTypes,
loadNodesAndCredentials: LoadNodesAndCredentials,
): (name: string) => PolicedType {
if (kind === NODE_TYPES_KIND) {
return (name) => ({ name, baseName: nodeTypes.resolveBaseName(name).baseName });
}
return (name) => ({
name,
baseName: name,
ancestors: credentialTypeAncestors(loadNodesAndCredentials, name),
});
}
/** Whether a credential type is known, checked at write time for an `extends` rule's value. */
export function isCredentialTypeKnown(
loadNodesAndCredentials: LoadNodesAndCredentials,
typeName: string,
): boolean {
return Object.hasOwn(loadNodesAndCredentials.knownCredentials, typeName);
}
/**
packages/cli/src/modules/type-availability-policies/policy-evaluator.ts
+8 -1
View File
@@ -50,7 +50,12 @@ export type PackageResolver = (typeName: string) => string | null;
*/
export const nodeTypePackageResolver: PackageResolver = (typeName) => typeName.split('.')[0];
export type PolicedType = { readonly name: string; readonly baseName: string };
/** `ancestors` lists what a credential type is built on through `extends`; empty for node types. */
export type PolicedType = {
readonly name: string;
readonly baseName: string;
readonly ancestors?: readonly string[];
};
function selectorMatches(
selector: PolicySelector,
@@ -62,6 +67,8 @@ function selectorMatches(
return selector.value === type.name || selector.value === type.baseName;
case 'package':
return resolvePackage(type.name) === selector.value;
case 'extends':
return selector.value === type.name || (type.ancestors ?? []).includes(selector.value);
}
}
packages/cli/src/modules/type-availability-policies/policy-shadow-lint.ts
+35 -14
View File
@@ -3,7 +3,7 @@ import {
type PackageResolver,
type PolicedType,
} from './policy-evaluator';
import type { PolicyRule } from './policy-rule.types';
import type { PolicyRule, PolicySelector } from './policy-rule.types';
/**
* One case of an unreachable rule: `ruleId` can never match, because `shadowedByRuleId`
@@ -54,6 +54,9 @@ function resolvePackageOrUndefined(
* package, so it also shadows a later name selector scoped to that same package. A name
* selector never shadows a package selector: one type can't cover a whole package.
*
* An `extends` selector shadows a later `name` or `extends` selector whose type is the same one
* or built on it. Nothing else shadows an `extends`: its family can span several packages.
*
* Runs in one O(n) pass: instead of comparing each rule against every rule before it, it
* keeps, per selector kind, only the earliest rule seen so far for each selector value, and
* looks that up once per rule.
@@ -73,29 +76,47 @@ export function lintRulesForShadowing(
): ShadowWarning[] {
const warnings: ShadowWarning[] = [];
const firstByName = new Map<string, FirstOccurrence>();
const firstByPackage = new Map<string, FirstOccurrence>();
const firstByKind: Record<PolicySelector['kind'], Map<string, FirstOccurrence>> = {
name: new Map(),
package: new Map(),
extends: new Map(),
};
/** The earliest `extends` rule naming the type itself or one it is built on. */
const firstFamilyOccurrence = (type: PolicedType) =>
[type.name, ...(type.ancestors ?? [])]
.map((name) => firstByKind.extends.get(name))
.reduce(earlierOccurrence, undefined);
for (let index = 0; index < rules.length; index++) {
const rule = rules[index];
const { selector } = rule;
const shadowedBy =
selector.kind === 'package'
? firstByPackage.get(selector.value)
: earlierOccurrence(
earlierOccurrence(
firstByName.get(selector.value),
firstByName.get(policedType(selector.value).baseName),
),
resolvePackageOrUndefined(firstByPackage, resolvePackage, selector.value),
);
let shadowedBy: FirstOccurrence | undefined;
switch (selector.kind) {
case 'package':
shadowedBy = firstByKind.package.get(selector.value);
break;
case 'extends':
shadowedBy = firstFamilyOccurrence(policedType(selector.value));
break;
case 'name': {
const type = policedType(selector.value);
shadowedBy = [
firstByKind.name.get(selector.value),
firstByKind.name.get(type.baseName),
resolvePackageOrUndefined(firstByKind.package, resolvePackage, selector.value),
firstFamilyOccurrence(type),
].reduce(earlierOccurrence, undefined);
break;
}
}
if (shadowedBy) {
warnings.push({ ruleId: rule.id, shadowedByRuleId: shadowedBy.rule.id });
}
const firstOccurrenceByValue = selector.kind === 'package' ? firstByPackage : firstByName;
const firstOccurrenceByValue = firstByKind[selector.kind];
if (!firstOccurrenceByValue.has(selector.value)) {
firstOccurrenceByValue.set(selector.value, { rule, index });
}
packages/cli/src/modules/type-availability-policies/type-availability-policy.service.ts
+54 -14
View File
@@ -21,8 +21,13 @@ import { TypeAvailabilityPolicyScopeRepository } from './database/repositories/t
import { TypeAvailabilityPolicyRepository } from './database/repositories/type-availability-policy.repository';
import type { TypeAvailabilityPolicy } from './database/entities/type-availability-policy.entity';
import type { TypeAvailabilityPolicyScope } from './database/entities/type-availability-policy-scope.entity';
import { NODE_TYPES_KIND } from './constants';
import { isPackageInstalled, packageResolverFor, policedTypeFor } from './package-resolver';
import { CREDENTIAL_TYPES_KIND, NODE_TYPES_KIND } from './constants';
import {
isCredentialTypeKnown,
isPackageInstalled,
packageResolverFor,
policedTypeFor,
} from './package-resolver';
import { evaluateComposedType, orderedAttachments, type ComposedVerdict } from './policy-evaluator';
import type {
PolicyAction,
@@ -267,6 +272,34 @@ function assertNoCredentialOnlyNodeRules(kind: string, rules: readonly PolicyRul
}
}
/**
* Only credential types declare `extends`, so the selector means nothing for another kind. An
* unknown base would match nothing, the same trap `assertPackagesInstalled` closes.
*/
function assertExtendsSelectorsValid(
kind: string,
rules: readonly PolicyRule[],
loadNodesAndCredentials: LoadNodesAndCredentials,
): void {
const values = rules.flatMap((rule) =>
rule.selector.kind === 'extends' ? [rule.selector.value] : [],
);
if (values.length === 0) return;
if (kind !== CREDENTIAL_TYPES_KIND) {
throw new UserError('An "extends" rule is only valid in a credential type policy');
}
const unknown = new Set(
values.filter((value) => !isCredentialTypeKnown(loadNodesAndCredentials, value)),
);
if (unknown.size > 0) {
throw new UserError(
`Extends rule names a credential type that is not installed: ${[...unknown].join(', ')}`,
);
}
}
/** Mirrors the DTO-level check in `ReplaceAttachmentsDto`, as a defensive service-level guard. */
function assertNoDuplicateAttachmentSlots(attachments: readonly AttachmentInput[]): void {
const seenPolicyIds = new Set<string>();
@@ -464,10 +497,11 @@ export class TypeAvailabilityPolicyService {
): Promise<PolicyDocumentWrite> {
assertPackagesInstalled(rules, this.loadNodesAndCredentials);
assertNoCredentialOnlyNodeRules(kind, rules);
assertExtendsSelectorsValid(kind, rules, this.loadNodesAndCredentials);
const warnings = lintRulesForShadowing(
rules,
packageResolverFor(kind, this.loadNodesAndCredentials),
policedTypeFor(kind, this.nodeTypes),
policedTypeFor(kind, this.nodeTypes, this.loadNodesAndCredentials),
);
const policy = await this.policyRepository.createPolicy({ kind, rules, updatedBy }, {});
@@ -503,10 +537,11 @@ export class TypeAvailabilityPolicyService {
): Promise<PolicyDocumentWrite> {
assertPackagesInstalled(rules, this.loadNodesAndCredentials);
assertNoCredentialOnlyNodeRules(kind, rules);
assertExtendsSelectorsValid(kind, rules, this.loadNodesAndCredentials);
const warnings = lintRulesForShadowing(
rules,
packageResolverFor(kind, this.loadNodesAndCredentials),
policedTypeFor(kind, this.nodeTypes),
policedTypeFor(kind, this.nodeTypes, this.loadNodesAndCredentials),
);
const result = await this.transactionRunner.run({}, async (ctx) => {
@@ -758,11 +793,12 @@ export class TypeAvailabilityPolicyService {
assertNoDelegateAtProjectScope(projectId, input.defaultAction, input.rules);
assertPackagesInstalled(input.rules, this.loadNodesAndCredentials);
assertNoCredentialOnlyNodeRules(kind, input.rules);
assertExtendsSelectorsValid(kind, input.rules, this.loadNodesAndCredentials);
const warnings = lintRulesForShadowing(
input.rules,
packageResolverFor(kind, this.loadNodesAndCredentials),
policedTypeFor(kind, this.nodeTypes),
policedTypeFor(kind, this.nodeTypes, this.loadNodesAndCredentials),
);
const result = await this.transactionRunner.run({}, async (ctx) => {
@@ -829,6 +865,9 @@ export class TypeAvailabilityPolicyService {
}
}
const scopeCreated = scope === null;
const defaultActionChanged = scope !== null && scope.defaultAction !== input.defaultAction;
const scopeId = scope
? scope.id
: (
@@ -838,7 +877,7 @@ export class TypeAvailabilityPolicyService {
)
).id;
if (scope && scope.defaultAction !== input.defaultAction) {
if (defaultActionChanged) {
await this.scopeRepository.updateDefaultAction(
scopeId,
input.defaultAction,
@@ -885,11 +924,12 @@ export class TypeAvailabilityPolicyService {
);
}
// This composed write always bumps the scope's freshness signal once, on top of
// whatever `updateDefaultAction` bumped on its own — a document-only edit (no
// `defaultAction` change) still must move the version, since it changes what the
// scope effectively enforces.
await this.scopeRepository.bumpVersion(scopeId, ctx);
// One accepted write moves the version by exactly one. The insert and
// `updateDefaultAction` each move it already, so bump here only when neither ran —
// a document-only edit still changes what the scope enforces and must move it too.
if (!scopeCreated && !defaultActionChanged) {
await this.scopeRepository.bumpVersion(scopeId, ctx);
}
const scopeAfterRow = await this.scopeRepository.findScopeById(scopeId, ctx);
const scopeAfter = {
@@ -908,7 +948,7 @@ export class TypeAvailabilityPolicyService {
};
});
// This path always bumps the scope's version, so it always invalidates.
// This path always moves the scope's version, so it always invalidates.
await this.invalidateScopes([{ kind, projectId }]);
this.eventService.emit('node-type-policy-scope-updated', {
@@ -973,7 +1013,7 @@ export class TypeAvailabilityPolicyService {
return evaluateComposedType(
instance,
project,
policedTypeFor(kind, this.nodeTypes)(typeName),
policedTypeFor(kind, this.nodeTypes, this.loadNodesAndCredentials)(typeName),
packageResolverFor(kind, this.loadNodesAndCredentials),
);
}
@@ -1050,7 +1090,7 @@ export class TypeAvailabilityPolicyService {
private verdictComposer(kind: string, typeNames: readonly string[]) {
const resolvePackage = packageResolverFor(kind, this.loadNodesAndCredentials);
const types = typeNames.map(policedTypeFor(kind, this.nodeTypes));
const types = typeNames.map(policedTypeFor(kind, this.nodeTypes, this.loadNodesAndCredentials));
return (instance: EffectivePolicy, project: EffectivePolicy): ComposedTypeVerdict[] =>
types.map((type) => ({
packages/cli/src/public-api/__tests__/public-api-controller.registry.test.ts
+220
View File
@@ -20,6 +20,7 @@ import {
import type { Controller, MultipartUploadLimits } from '@n8n/decorators';
import { Container, Service } from '@n8n/di';
import express from 'express';
import { Readable } from 'node:stream';
import request from 'supertest';
import { mock } from 'vitest-mock-extended';
import { z } from 'zod';
@@ -724,4 +725,223 @@ describe('PublicApiControllerRegistry', () => {
expect(response.body).toEqual({ message: 'Forbidden' });
});
});
describe('binary response bodies', () => {
// supertest buffers only text and JSON bodies by default
const readBinaryBody = (
res: request.Response,
callback: (error: Error | null, body: Buffer) => void,
) => {
const chunks: Buffer[] = [];
res.on('data', (chunk: Buffer) => chunks.push(chunk));
res.on('end', () => callback(null, Buffer.concat(chunks)));
};
it('sends the body the method returns, with the declared status, media type and headers', async () => {
@Service()
class WidgetsPublicController {
@Get('/')
@ApiResponse(202, {
mediaType: 'application/gzip',
headers: { 'X-Example': { description: 'Example header.' } },
})
async method() {
return { body: Buffer.from([1, 2, 3]), headers: { 'X-Example': '1' } };
}
}
markPublicApiController(WidgetsPublicController as Controller, '/widgets');
const response = await request(activate())
.get('/api/v1/widgets')
.buffer(true)
.parse(readBinaryBody)
.expect(202);
expect(response.headers['content-type']).toBe('application/gzip');
expect(response.headers['x-example']).toBe('1');
expect(response.body).toEqual(Buffer.from([1, 2, 3]));
});
it('streams a returned stream', async () => {
@Service()
class WidgetsPublicController {
@Get('/')
@ApiResponse(200, { mediaType: 'application/gzip' })
async method() {
return { body: Readable.from([Buffer.from('ab'), Buffer.from('cd')]) };
}
}
markPublicApiController(WidgetsPublicController as Controller, '/widgets');
const response = await request(activate())
.get('/api/v1/widgets')
.buffer(true)
.parse(readBinaryBody)
.expect(200);
expect(response.body.toString()).toBe('abcd');
});
it('matches a declared header name case-insensitively', async () => {
@Service()
class WidgetsPublicController {
@Get('/')
@ApiResponse(200, {
mediaType: 'application/gzip',
headers: { 'X-Example': { description: 'Example header.' } },
})
async method() {
return { body: Buffer.from('x'), headers: { 'x-example': '1' } };
}
}
markPublicApiController(WidgetsPublicController as Controller, '/widgets');
const response = await request(activate())
.get('/api/v1/widgets')
.buffer(true)
.parse(readBinaryBody)
.expect(200);
expect(response.headers['x-example']).toBe('1');
});
it('keeps the declared media type when a result header tries to replace it', async () => {
@Service()
class WidgetsPublicController {
@Get('/')
@ApiResponse(200, { mediaType: 'application/gzip' })
async method() {
return { body: Buffer.from('x'), headers: { 'Content-Type': 'text/plain' } };
}
}
markPublicApiController(WidgetsPublicController as Controller, '/widgets');
const response = await request(activate())
.get('/api/v1/widgets')
.buffer(true)
.parse(readBinaryBody)
.expect(200);
expect(response.headers['content-type']).toBe('application/gzip');
});
it('fails with 500 when the method returns no binary body', async () => {
@Service()
class WidgetsPublicController {
@Get('/')
@ApiResponse(200, { mediaType: 'application/gzip' })
async method() {}
}
markPublicApiController(WidgetsPublicController as Controller, '/widgets');
const response = await request(activate()).get('/api/v1/widgets').expect(500);
expect(response.headers['content-type']).toMatch(/application\/json/);
expect(response.body.message).toBe('Internal server error');
});
it('fails with 500 when a declared header is missing from the result', async () => {
@Service()
class WidgetsBinaryHeaderPublicController {
@Get('/')
@ApiResponse(200, {
mediaType: 'application/gzip',
headers: { 'X-Required': { description: 'Must be set.' } },
})
async method() {
return { body: Buffer.from('x') };
}
}
markPublicApiController(WidgetsBinaryHeaderPublicController as Controller, '/widgets');
const response = await request(activate()).get('/api/v1/widgets').expect(500);
expect(response.headers['content-type']).toMatch(/application\/json/);
expect(response.body.message).toBe('Internal server error');
});
it('sends a clean JSON error when the method throws before the body starts', async () => {
@Service()
class WidgetsPublicController {
@Get('/')
@ApiResponse(200, { mediaType: 'application/gzip' })
async method() {
throw new NotFoundError('missing');
}
}
markPublicApiController(WidgetsPublicController as Controller, '/widgets');
const response = await request(activate()).get('/api/v1/widgets').expect(404);
expect(response.headers['content-type']).toMatch(/application\/json/);
expect(response.body.message).toBe('missing');
});
it('sends a clean JSON error when a stream fails before its first chunk', async () => {
@Service()
class WidgetsPublicController {
@Get('/')
@ApiResponse(200, {
mediaType: 'application/gzip',
headers: { 'X-Example': { description: 'Example header.' } },
})
async method() {
const failing = new Readable({
read() {
this.destroy(new NotFoundError('missing'));
},
});
return { body: failing, headers: { 'X-Example': '1' } };
}
}
markPublicApiController(WidgetsPublicController as Controller, '/widgets');
const response = await request(activate()).get('/api/v1/widgets').expect(404);
expect(response.headers['content-type']).toMatch(/application\/json/);
expect(response.headers['x-example']).toBeUndefined();
expect(response.body.message).toBe('missing');
});
it('keeps headers set before the method on an error response', async () => {
const since = new Date('2026-07-23T00:00:00Z');
@Service()
class WidgetsPublicController {
@Get('/')
@ApiResponse(200, { mediaType: 'application/gzip' })
@Deprecated({ since })
async method() {
throw new NotFoundError('missing');
}
}
markPublicApiController(WidgetsPublicController as Controller, '/widgets');
const response = await request(activate()).get('/api/v1/widgets').expect(404);
expect(response.headers.deprecation).toBe(`@${Math.floor(since.getTime() / 1000)}`);
});
it('keeps a Content-Type set by earlier middleware on an error response', async () => {
@Service()
class WidgetsBinaryMiddlewarePublicController {
@Middleware()
label(_req: express.Request, res: express.Response, next: express.NextFunction) {
res.setHeader('Content-Type', 'text/plain');
next();
}
@Get('/')
@ApiResponse(200, { mediaType: 'application/gzip' })
async method() {
throw new NotFoundError('missing');
}
}
markPublicApiController(WidgetsBinaryMiddlewarePublicController as Controller, '/widgets');
const response = await request(activate()).get('/api/v1/widgets').expect(404);
expect(response.headers['content-type']).toMatch(/text\/plain/);
});
});
});
packages/cli/src/public-api/media-types/__tests__/binary-response.test.ts
+200
View File
@@ -0,0 +1,200 @@
import type { BinaryResponse } from '@n8n/decorators';
import type { Response } from 'express';
import { Readable, Writable } from 'node:stream';
import { mock } from 'vitest-mock-extended';
import { sendBinaryResponse } from '../binary-response';
/** A live response mock whose `status` returns the response, as Express does. */
function mockResponse() {
const res = mock<Response>();
res.status.mockReturnValue(res);
res.destroyed = false;
return res;
}
describe('sendBinaryResponse', () => {
const binaryResponse: BinaryResponse = {
mediaType: 'application/gzip',
headers: { 'X-Required': { description: 'Must be set.' } },
};
it('rejects a result that is missing a declared header, before the response is touched', async () => {
const res = mockResponse();
await expect(
sendBinaryResponse(res, binaryResponse, 200, 'Widgets.method', { body: Buffer.from('x') }),
).rejects.toThrow('Widgets.method did not set the declared response header(s): X-Required');
expect(res.status).not.toHaveBeenCalled();
expect(res.setHeader).not.toHaveBeenCalled();
});
it('matches a declared header name case-insensitively', async () => {
const res = mockResponse();
await sendBinaryResponse(res, binaryResponse, 200, 'Widgets.method', {
body: Buffer.from('x'),
headers: { 'x-required': '1' },
});
expect(res.end).toHaveBeenCalledWith(Buffer.from('x'));
});
it('rejects a result without a binary body', async () => {
const res = mockResponse();
await expect(
sendBinaryResponse(res, binaryResponse, 200, 'Widgets.method', {
body: 'not binary',
headers: { 'X-Required': '1' },
}),
).rejects.toThrow('Widgets.method declares a binary @ApiResponse but returned no body');
expect(res.status).not.toHaveBeenCalled();
});
it('ends an empty stream with a 200 and no body', async () => {
const sent: Buffer[] = [];
const client = new Writable({
write(chunk: Buffer, _encoding, callback) {
sent.push(chunk);
callback();
},
});
const res = Object.assign(client, {
status: vi.fn(() => client),
setHeader: vi.fn(() => client),
});
await sendBinaryResponse(res as unknown as Response, binaryResponse, 200, 'Widgets.method', {
body: Readable.from([]),
headers: { 'X-Required': '1' },
});
expect(res.status).toHaveBeenCalledWith(200);
expect(client.writableFinished).toBe(true);
expect(sent).toEqual([]);
});
it('resolves when the client disconnects after the first chunk', async () => {
const sent: Buffer[] = [];
const client = new Writable({
write(chunk: Buffer, _encoding, callback) {
sent.push(chunk);
// Drop the connection on the second chunk, before its write completes.
if (sent.length === 2) {
client.destroy();
return;
}
callback();
},
});
const res = Object.assign(client, { status: () => client, setHeader: () => client });
await expect(
sendBinaryResponse(res as unknown as Response, binaryResponse, 200, 'Widgets.method', {
body: Readable.from([Buffer.from('ab'), Buffer.from('cd'), Buffer.from('ef')]),
headers: { 'X-Required': '1' },
}),
).resolves.toBeUndefined();
expect(sent).toHaveLength(2);
});
it('destroys the stream when the client has already left', async () => {
const client = new Writable({
write(_chunk, _encoding, callback) {
callback();
},
});
client.destroy();
const res = Object.assign(client, { status: () => client, setHeader: () => client });
const body = Readable.from([Buffer.from('ab')]);
await expect(
sendBinaryResponse(res as unknown as Response, binaryResponse, 200, 'Widgets.method', {
body,
headers: { 'X-Required': '1' },
}),
).resolves.toBeUndefined();
expect(body.destroyed).toBe(true);
});
it('destroys a returned stream when a declared header is missing', async () => {
const body = Readable.from([Buffer.from('x')]);
await expect(
sendBinaryResponse(mockResponse(), binaryResponse, 200, 'Widgets.method', { body }),
).rejects.toThrow('X-Required');
expect(body.destroyed).toBe(true);
});
it('destroys the stream when the client leaves while the first chunk is pending', async () => {
const client = new Writable({
write(_chunk, _encoding, callback) {
callback();
},
});
const res = Object.assign(client, { status: () => client, setHeader: () => client });
// A source that never produces a chunk.
const body = new Readable({ read() {} });
const sending = sendBinaryResponse(
res as unknown as Response,
binaryResponse,
200,
'Widgets.method',
{
body,
headers: { 'X-Required': '1' },
},
);
client.destroy();
await expect(sending).resolves.toBeUndefined();
expect(body.destroyed).toBe(true);
});
it('destroys a stalled stream when the client left before the response started', async () => {
const client = new Writable({
write(_chunk, _encoding, callback) {
callback();
},
});
client.destroy();
// Let the close event fire before the response starts, so a listener added later never sees it.
await new Promise((resolve) => setImmediate(resolve));
const res = Object.assign(client, { status: () => client, setHeader: () => client });
// A source that never produces a chunk.
const body = new Readable({ read() {} });
await expect(
sendBinaryResponse(res as unknown as Response, binaryResponse, 200, 'Widgets.method', {
body,
headers: { 'X-Required': '1' },
}),
).resolves.toBeUndefined();
expect(body.destroyed).toBe(true);
});
it('destroys the stream when setting a header throws', async () => {
const res = mockResponse();
res.setHeader.mockImplementation(() => {
throw new Error('Invalid character in header content');
});
const body = Readable.from([Buffer.from('x')]);
await expect(
sendBinaryResponse(res, binaryResponse, 200, 'Widgets.method', {
body,
headers: { 'X-Required': '1' },
}),
).rejects.toThrow('Invalid character in header content');
expect(body.destroyed).toBe(true);
});
});
packages/cli/src/public-api/media-types/binary-response.ts
+119
View File
@@ -0,0 +1,119 @@
import type { BinaryResponse, BinaryResult, SuccessStatus } from '@n8n/decorators';
import { UnexpectedError } from '@n8n/errors';
import type { Response } from 'express';
import { Readable } from 'node:stream';
import { pipeline } from 'node:stream/promises';
function isBinaryResult(result: unknown): result is BinaryResult {
return (
typeof result === 'object' &&
result !== null &&
'body' in result &&
(Buffer.isBuffer(result.body) || result.body instanceof Readable)
);
}
function isClientGone(error: unknown): boolean {
return (
error instanceof Error &&
'code' in error &&
(error.code === 'ERR_STREAM_PREMATURE_CLOSE' || error.code === 'ERR_STREAM_UNABLE_TO_PIPE')
);
}
function setResponseHeaders(
res: Response,
successStatus: SuccessStatus,
mediaType: BinaryResponse['mediaType'],
headers: NonNullable<BinaryResult['headers']>,
) {
for (const [name, value] of Object.entries(headers)) {
res.setHeader(name, value);
}
// Set last, so a result header cannot replace the declared media type.
res.status(successStatus).setHeader('Content-Type', mediaType);
}
async function sendStream(res: Response, body: Readable, setHeaders: () => void) {
const release = () => body.destroy();
res.once('close', release);
// The client may have left before the listener was attached.
if (res.destroyed) {
release();
}
try {
// Peek at the first chunk before any header is set. If the stream fails here, the error
// still gets a JSON response.
const iterator = body[Symbol.asyncIterator]();
const first = await iterator.next();
setHeaders();
const rest = { [Symbol.asyncIterator]: () => iterator };
async function* withFirstChunk() {
if (!first.done) {
yield first.value;
}
yield* rest;
}
await pipeline(Readable.from(withFirstChunk()), res);
} catch (error) {
// The client disconnected. There is no one left to send to.
if (!isClientGone(error)) {
throw error;
}
} finally {
res.off('close', release);
}
}
/**
* Sends the result of a route whose success body is binary.
*/
export async function sendBinaryResponse(
res: Response,
binaryResponse: BinaryResponse,
successStatus: SuccessStatus,
routeName: string,
result: unknown,
): Promise<void> {
if (!isBinaryResult(result)) {
throw new UnexpectedError(`${routeName} declares a binary @ApiResponse but returned no body`);
}
const { body } = result;
const declaredHeaders = binaryResponse.headers ?? {};
const resultHeaders = result.headers ?? {};
try {
const setHeaderNames = new Set(Object.keys(resultHeaders).map((name) => name.toLowerCase()));
const missing = Object.keys(declaredHeaders).filter(
(name) => !setHeaderNames.has(name.toLowerCase()),
);
if (missing.length) {
throw new UnexpectedError(
`${routeName} did not set the declared response header(s): ${missing.join(', ')}`,
);
}
const setHeaders = () =>
setResponseHeaders(res, successStatus, binaryResponse.mediaType, resultHeaders);
// A buffer is already in memory, so send it in one write.
if (Buffer.isBuffer(body)) {
setHeaders();
res.end(body);
return;
}
await sendStream(res, body, setHeaders);
} finally {
// However this function exits, the stream must not stay open.
if (body instanceof Readable) {
body.destroy();
}
}
}
packages/cli/src/public-api/public-api-controller.registry.ts
+13
View File
@@ -20,6 +20,7 @@ import { FeatureNotLicensedError } from '@/errors/feature-not-licensed.error';
import { BadRequestError, UnsupportedMediaTypeError } from '@n8n/errors';
import { License } from '@/license';
import { assertContentType } from '@/public-api/media-types/content-type';
import { sendBinaryResponse } from '@/public-api/media-types/binary-response';
import type { RequestBodyHandler } from '@/public-api/media-types/request-body';
import { userHasScopes } from '@/permissions.ee/check-access';
import { USER_QUOTA_FORBIDDEN_MESSAGE } from '@/public-api/constants';
@@ -120,6 +121,18 @@ export class PublicApiControllerRegistry {
}
}
if (route.binaryResponse) {
const result = await controller[handlerName](...args);
await sendBinaryResponse(
res,
route.binaryResponse,
successStatus,
`${controllerClass.name}.${handlerName}`,
result,
);
return;
}
const result = await controller[handlerName](...args);
if (res.headersSent) return;
packages/cli/src/public-api/public-api-route-resolver.ts
+4
View File
@@ -2,6 +2,7 @@ import type { ZodClass } from '@n8n/api-types';
import type {
ApiKeyScopeRequirement,
Arg,
BinaryResponse,
Controller,
DeprecationInfo,
ErrorResponse,
@@ -83,6 +84,8 @@ export interface ResolvedPublicApiRoute {
requestBodyHandler?: RequestBodyHandler;
requestQueryDto?: ZodClass;
responseDto?: ResponseDtoClass;
/** Mutually exclusive with `responseDto`. */
binaryResponse?: BinaryResponse;
/** Success status declared via `@ApiResponse` - always present, see `resolveSuccessStatus`. */
successStatus: SuccessStatus;
apiKeyScope?: ApiKeyScopeRequirement;
@@ -296,6 +299,7 @@ export function resolvePublicApiRoutes(): ResolvedPublicApiRoute[] {
requestBodyHandler,
requestQueryDto,
responseDto: route.responseDto,
binaryResponse: route.binaryResponse,
successStatus: resolveSuccessStatus(controllerClass.name, handlerName, route.successStatus),
apiKeyScope: route.apiKeyScope,
summary: route.summary,
packages/cli/src/public-api/v1/controllers/__tests__/credential-type-policies.public.controller.test.ts
+6 -6
View File
@@ -1,7 +1,7 @@
import type {
PolicyEffectiveWriteResultPublicDto,
PutInstancePolicyDto,
PutProjectPolicyDto,
CredentialTypePolicyEffectiveWriteResultPublicDto,
PutCredentialTypeInstancePolicyDto,
PutCredentialTypeProjectPolicyDto,
} from '@n8n/api-types';
import type { ModuleRegistry } from '@n8n/backend-common';
import { LICENSE_FEATURES } from '@n8n/constants';
@@ -195,9 +195,9 @@ describe('CredentialTypePoliciesPublicController handler bodies', () => {
it('putCredentialTypeInstancePolicy forwards rules, defaultAction, version, and the caller id, and maps the result', async () => {
service.setEffectivePolicy.mockResolvedValue(effectiveWrite);
const dto = { rules, defaultAction: 'deny', version: 3 } as PutInstancePolicyDto;
const dto = { rules, defaultAction: 'deny', version: 3 } as PutCredentialTypeInstancePolicyDto;
const result: PolicyEffectiveWriteResultPublicDto =
const result: CredentialTypePolicyEffectiveWriteResultPublicDto =
await controller.putCredentialTypeInstancePolicy(req, res, dto);
expect(service.setEffectivePolicy).toHaveBeenCalledWith(
@@ -218,7 +218,7 @@ describe('CredentialTypePoliciesPublicController handler bodies', () => {
it('putCredentialTypeProjectPolicy forwards the project id, rules, defaultAction, version, and the caller id, and maps the result', async () => {
service.setEffectivePolicy.mockResolvedValue(effectiveWrite);
const dto = { rules, defaultAction: 'deny', version: 3 } as PutProjectPolicyDto;
const dto = { rules, defaultAction: 'deny', version: 3 } as PutCredentialTypeProjectPolicyDto;
const result = await controller.putCredentialTypeProjectPolicy(req, res, 'project-id', dto);
packages/cli/src/public-api/v1/controllers/__tests__/node-type-policies.public.controller.test.ts
+11
View File
@@ -160,6 +160,17 @@ describe('NodeTypePoliciesPublicController handler bodies', () => {
warnings: [{ ruleId: 'r2', shadowedByRuleId: 'r1' }],
};
it('fails loudly instead of returning an extends rule, which no node type write accepts', async () => {
service.getEffectivePolicy.mockResolvedValue({
...effectivePolicy,
rules: [{ id: 'r1', action: 'deny', selector: { kind: 'extends', value: 'oAuth2Api' } }],
});
await expect(controller.getNodeTypeInstancePolicy()).rejects.toThrow(
'A node type policy holds an extends rule',
);
});
it('getNodeTypeInstancePolicy reads the null-project scope and maps the response', async () => {
service.getEffectivePolicy.mockResolvedValue(effectivePolicy);
packages/cli/src/public-api/v1/controllers/credential-type-policies.public.controller.ts
+33 -33
View File
@@ -1,16 +1,16 @@
import {
CreatePolicyDocumentDto,
CreateCredentialTypePolicyDocumentDto,
ListNodeTypePolicyDocumentsQueryDto,
PolicyAttachmentsPublicDto,
PolicyDocumentListPublicDto,
PolicyDocumentPublicDto,
PolicyDocumentWriteResultPublicDto,
PolicyEffectivePublicDto,
PolicyEffectiveWriteResultPublicDto,
PutInstancePolicyDto,
PutProjectPolicyDto,
CredentialTypePolicyAttachmentsPublicDto,
CredentialTypePolicyDocumentListPublicDto,
CredentialTypePolicyDocumentPublicDto,
CredentialTypePolicyDocumentWriteResultPublicDto,
CredentialTypePolicyEffectivePublicDto,
CredentialTypePolicyEffectiveWriteResultPublicDto,
PutCredentialTypeInstancePolicyDto,
PutCredentialTypeProjectPolicyDto,
ReplaceAttachmentsDto,
UpdatePolicyDocumentDto,
UpdateCredentialTypePolicyDocumentDto,
credentialTypePolicyIdParamSchema,
credentialTypePolicyScopeIdParamSchema,
projectIdParamSchema,
@@ -50,7 +50,7 @@ import {
const tags = ['CredentialTypePolicy'];
function toPublicDocument(policy: TypeAvailabilityPolicy): PolicyDocumentPublicDto {
function toPublicDocument(policy: TypeAvailabilityPolicy): CredentialTypePolicyDocumentPublicDto {
return {
id: policy.id,
kind: policy.kind,
@@ -91,9 +91,9 @@ export class CredentialTypePoliciesPublicController {
'Returns the composed instance-scope policy: its default action and the rules of every attached policy document in evaluation order. An instance that was never configured reports `scopeId: null`, no rules, `defaultAction: allow`, and `version: 0`.',
)
@ApiTags(tags)
@ApiResponse(200, PolicyEffectivePublicDto)
@ApiResponse(200, CredentialTypePolicyEffectivePublicDto)
@ApiErrorResponse(503)
async getCredentialTypeInstancePolicy(): Promise<PolicyEffectivePublicDto> {
async getCredentialTypeInstancePolicy(): Promise<CredentialTypePolicyEffectivePublicDto> {
const effective = await (await this.service()).getEffectivePolicy(CREDENTIAL_TYPES_KIND, null);
return {
@@ -113,14 +113,14 @@ export class CredentialTypePoliciesPublicController {
'Sets the instance default action and replaces the rules of its single policy document, creating both on first write. `version` must equal the version last read; a stale value is rejected with 409. Rule ids must be unique within the list.',
)
@ApiTags(tags)
@ApiResponse(200, PolicyEffectiveWriteResultPublicDto)
@ApiResponse(200, CredentialTypePolicyEffectiveWriteResultPublicDto)
@ApiErrorResponse(409)
@ApiErrorResponse(503)
async putCredentialTypeInstancePolicy(
req: AuthenticatedRequest,
_res: Response,
@Body dto: PutInstancePolicyDto,
): Promise<PolicyEffectiveWriteResultPublicDto> {
@Body dto: PutCredentialTypeInstancePolicyDto,
): Promise<CredentialTypePolicyEffectiveWriteResultPublicDto> {
const result = await (await this.service()).setEffectivePolicy(
CREDENTIAL_TYPES_KIND,
null,
@@ -147,13 +147,13 @@ export class CredentialTypePoliciesPublicController {
"Returns the project's own composed policy, not the result of combining it with the instance policy. A project that was never configured reports `scopeId: null`, no rules, `defaultAction: allow`, and `version: 0`.",
)
@ApiTags(tags)
@ApiResponse(200, PolicyEffectivePublicDto)
@ApiResponse(200, CredentialTypePolicyEffectivePublicDto)
@ApiErrorResponse(503)
async getCredentialTypeProjectPolicy(
_req: AuthenticatedRequest,
_res: Response,
@Param('projectId', projectIdParamSchema) projectId: string,
): Promise<PolicyEffectivePublicDto> {
): Promise<CredentialTypePolicyEffectivePublicDto> {
const effective = await (await this.service()).getEffectivePolicy(
CREDENTIAL_TYPES_KIND,
projectId,
@@ -176,15 +176,15 @@ export class CredentialTypePoliciesPublicController {
"Sets the project's default action and replaces the rules of its single policy document, creating both on first write. The `delegate` action is not accepted at project scope. `version` must equal the version last read; a stale value is rejected with 409, as is a project document that is shared with another scope.",
)
@ApiTags(tags)
@ApiResponse(200, PolicyEffectiveWriteResultPublicDto)
@ApiResponse(200, CredentialTypePolicyEffectiveWriteResultPublicDto)
@ApiErrorResponse(409)
@ApiErrorResponse(503)
async putCredentialTypeProjectPolicy(
req: AuthenticatedRequest,
_res: Response,
@Param('projectId', projectIdParamSchema) projectId: string,
@Body dto: PutProjectPolicyDto,
): Promise<PolicyEffectiveWriteResultPublicDto> {
@Body dto: PutCredentialTypeProjectPolicyDto,
): Promise<CredentialTypePolicyEffectiveWriteResultPublicDto> {
const result = await (await this.service()).setEffectivePolicy(
CREDENTIAL_TYPES_KIND,
projectId,
@@ -209,13 +209,13 @@ export class CredentialTypePoliciesPublicController {
@ApiSummary('List credential type policy documents')
@ApiDescription('Returns a cursor-paginated list of reusable policy documents.')
@ApiTags(tags)
@ApiResponse(200, PolicyDocumentListPublicDto)
@ApiResponse(200, CredentialTypePolicyDocumentListPublicDto)
@ApiErrorResponse(503)
async listCredentialTypePolicyDocuments(
_req: AuthenticatedRequest,
_res: Response,
@Query query: ListNodeTypePolicyDocumentsQueryDto,
): Promise<PolicyDocumentListPublicDto> {
): Promise<CredentialTypePolicyDocumentListPublicDto> {
const { offset, limit } = resolveOffsetPagination(query);
const { items, count } = await (await this.service()).listPolicyDocumentsPage(
@@ -239,13 +239,13 @@ export class CredentialTypePoliciesPublicController {
'Creates a reusable policy document that is not yet attached to any scope. Rule ids must be unique within the list. `warnings` lists rules that an earlier rule already shadows.',
)
@ApiTags(tags)
@ApiResponse(201, PolicyDocumentWriteResultPublicDto)
@ApiResponse(201, CredentialTypePolicyDocumentWriteResultPublicDto)
@ApiErrorResponse(503)
async createCredentialTypePolicyDocument(
req: AuthenticatedRequest,
_res: Response,
@Body dto: CreatePolicyDocumentDto,
): Promise<PolicyDocumentWriteResultPublicDto> {
@Body dto: CreateCredentialTypePolicyDocumentDto,
): Promise<CredentialTypePolicyDocumentWriteResultPublicDto> {
const { policy, warnings } = await (await this.service()).createPolicyDocument(
CREDENTIAL_TYPES_KIND,
dto.rules,
@@ -261,14 +261,14 @@ export class CredentialTypePoliciesPublicController {
@GlobalScope('credentialTypePolicy:manage')
@ApiSummary('Retrieve a credential type policy document')
@ApiTags(tags)
@ApiResponse(200, PolicyDocumentPublicDto)
@ApiResponse(200, CredentialTypePolicyDocumentPublicDto)
@ApiErrorResponse(404)
@ApiErrorResponse(503)
async getCredentialTypePolicyDocument(
_req: AuthenticatedRequest,
_res: Response,
@Param('policyId', credentialTypePolicyIdParamSchema) policyId: string,
): Promise<PolicyDocumentPublicDto> {
): Promise<CredentialTypePolicyDocumentPublicDto> {
const policy = await (await this.service()).getPolicyDocument(CREDENTIAL_TYPES_KIND, policyId);
if (!policy) {
throw new NotFoundError(`Policy document not found: ${policyId}`);
@@ -286,7 +286,7 @@ export class CredentialTypePoliciesPublicController {
"Replaces the document's whole rule list. `version` must equal the version last read; a stale value is rejected with 409. Every scope the document is attached to has its version bumped. A `delegate` rule is rejected when the document is attached to a project scope.",
)
@ApiTags(tags)
@ApiResponse(200, PolicyDocumentWriteResultPublicDto)
@ApiResponse(200, CredentialTypePolicyDocumentWriteResultPublicDto)
@ApiErrorResponse(404)
@ApiErrorResponse(409)
@ApiErrorResponse(503)
@@ -294,8 +294,8 @@ export class CredentialTypePoliciesPublicController {
req: AuthenticatedRequest,
_res: Response,
@Param('policyId', credentialTypePolicyIdParamSchema) policyId: string,
@Body dto: UpdatePolicyDocumentDto,
): Promise<PolicyDocumentWriteResultPublicDto> {
@Body dto: UpdateCredentialTypePolicyDocumentDto,
): Promise<CredentialTypePolicyDocumentWriteResultPublicDto> {
const { policy, warnings } = await (await this.service()).updatePolicyDocument(
CREDENTIAL_TYPES_KIND,
policyId,
@@ -337,7 +337,7 @@ export class CredentialTypePoliciesPublicController {
"Replaces every attachment on a scope. `scopeId` comes from the scope's `GET` response once it has been written. Each `policyId` and each `(isFloor, priority)` pair must be unique within the list. Last write wins; the scope's version is bumped.",
)
@ApiTags(tags)
@ApiResponse(200, PolicyAttachmentsPublicDto)
@ApiResponse(200, CredentialTypePolicyAttachmentsPublicDto)
@ApiErrorResponse(404)
@ApiErrorResponse(503)
async replaceCredentialTypePolicyAttachments(
@@ -345,7 +345,7 @@ export class CredentialTypePoliciesPublicController {
_res: Response,
@Param('scopeId', credentialTypePolicyScopeIdParamSchema) scopeId: string,
@Body dto: ReplaceAttachmentsDto,
): Promise<PolicyAttachmentsPublicDto> {
): Promise<CredentialTypePolicyAttachmentsPublicDto> {
const result = await (await this.service()).replaceAttachments(
CREDENTIAL_TYPES_KIND,
scopeId,
packages/cli/src/public-api/v1/controllers/node-type-policies.public.controller.ts
+18 -6
View File
@@ -11,7 +11,10 @@ import {
PutProjectPolicyDto,
ReplaceAttachmentsDto,
UpdatePolicyDocumentDto,
isNodeTypePolicyRule,
nodeTypePolicyIdParamSchema,
type NodeTypePolicyRule,
type PolicyRule,
nodeTypePolicyScopeIdParamSchema,
projectIdParamSchema,
} from '@n8n/api-types';
@@ -39,6 +42,7 @@ import {
} from '@n8n/decorators';
import { Container } from '@n8n/di';
import type { Response } from 'express';
import { UnexpectedError } from 'n8n-workflow';
import { NotFoundError, ServiceUnavailableError } from '@n8n/errors';
import { NODE_TYPES_KIND } from '@/modules/type-availability-policies/constants';
@@ -50,11 +54,19 @@ import {
const tags = ['NodeTypePolicy'];
/** Every write rejects an `extends` rule for this kind, so meeting one here is a bug. */
function toNodeTypeRules(rules: readonly PolicyRule[]): NodeTypePolicyRule[] {
if (!rules.every(isNodeTypePolicyRule)) {
throw new UnexpectedError('A node type policy holds an extends rule');
}
return [...rules];
}
function toPublicDocument(policy: TypeAvailabilityPolicy): PolicyDocumentPublicDto {
return {
id: policy.id,
kind: policy.kind,
rules: [...policy.rules],
rules: toNodeTypeRules(policy.rules),
version: policy.version,
updatedBy: policy.updatedBy,
createdAt: policy.createdAt.toISOString(),
@@ -98,7 +110,7 @@ export class NodeTypePoliciesPublicController {
return {
scopeId: effective.scopeId,
rules: [...effective.rules],
rules: toNodeTypeRules(effective.rules),
defaultAction: effective.defaultAction,
version: effective.version,
};
@@ -131,7 +143,7 @@ export class NodeTypePoliciesPublicController {
return {
scopeId: result.scopeId,
rules: result.rules,
rules: toNodeTypeRules(result.rules),
defaultAction: result.defaultAction,
version: result.version,
warnings: [...result.warnings],
@@ -158,7 +170,7 @@ export class NodeTypePoliciesPublicController {
return {
scopeId: effective.scopeId,
rules: [...effective.rules],
rules: toNodeTypeRules(effective.rules),
defaultAction: effective.defaultAction,
version: effective.version,
};
@@ -192,7 +204,7 @@ export class NodeTypePoliciesPublicController {
return {
scopeId: result.scopeId,
rules: result.rules,
rules: toNodeTypeRules(result.rules),
defaultAction: result.defaultAction,
version: result.version,
warnings: [...result.warnings],
@@ -353,7 +365,7 @@ export class NodeTypePoliciesPublicController {
return {
attachments: result.attachments.map((attachment) => ({
policyId: attachment.policyId,
rules: [...attachment.rules],
rules: toNodeTypeRules(attachment.rules),
priority: attachment.priority,
isFloor: attachment.isFloor,
})),
packages/cli/src/public-api/v1/handlers/credential-type-policies/spec/paths/createCredentialTypePolicyDocument.generated.yml
+14 -1
View File
@@ -54,6 +54,19 @@ requestBody:
required:
- kind
- value
- type: object
properties:
kind:
type: string
enum:
- extends
value:
type: string
minLength: 1
required:
- kind
- value
description: Matches the named credential type and every type that extends it
type: object
required:
- id
@@ -67,7 +80,7 @@ responses:
content:
application/json:
schema:
$ref: ../../../../shared/spec/schemas/policyDocumentWriteResultPublicDto.generated.yml
$ref: ../../../../shared/spec/schemas/credentialTypePolicyDocumentWriteResultPublicDto.generated.yml
'400':
$ref: ../../../../shared/spec/responses/badRequest.yml
'401':

Some files were not shown because too many files have changed in this diff Show More