mirror of
https://github.com/paperclipai/paperclip.git
synced 2026-11-07 06:16:06 -10:00
Keep personal MCP connections functional while idle tenants sleep (#19427)
## Thinking Path > - Paperclip manages work for AI agents. > - Personal MCP connections should remain enabled while an idle tenant sleeps. > - Unfinished OAuth, transfer and delivery work must retain its owner and deadline. > - Cloud can receive bounded metadata and incoming work while the tenant remains the company-scoped authorizer. > - Current identity, revision and delivery acknowledgments establish when ownership has transferred. > - This change preserves useful wake, retries and revocation while making settled ownership inspectable without writes. ## Linked Issues or Issue Description Refs: #19377 and #19750. Hosted companion: [paperclip-cloud#1217](https://github.com/paperclipai/paperclip-cloud/pull/1217). **What happened?** Enabled personal MCP grants and subscriptions could keep an otherwise idle tenant awake. Ignoring those records would lose discovery, authorization, delivery and revocation obligations. **Expected behavior** Unconsumed authorization codes, device requests, file handoffs and delivery work retain their original deadlines. Settled connections remain enabled and become quiet after the current external owner acknowledges responsibility. Incoming useful work resumes normal authorization and delivery; revocation still takes effect for the current company. **Steps to reproduce** Complete normal PKCE/device consent and company selection, synchronize client/device and revocation state, retain the connection while idle, then exercise useful tools/events and revocation. Sleep eligibility requires current incoming/deadline ownership throughout. ## What Changed - Add bounded signed client/device metadata and revocation synchronization with exact revision, identity and ACK checks, startup recovery and finite legacy-device cutover. - Preserve company selection, authorization and operation identity. Register grant work before its INSERT in the same transaction; retain rollback, expiry and retry behavior. - Require current managed identity/resource before hosted publication. Receipt-cleared re-enable must obtain a fresh source ACK before device creation. - Reuse read-only settings and inspection budgets; connect MCP owners to normal startup, drain and shutdown. - Bound metadata pages and the actual signed request body while preserving the default limits of other operations. - Generate migration 0440 after the merged Legacy migration. Preserve all 438 existing executable migrations and the journal prefix; retain the normal five newest snapshots. - Preserve the merged transaction-owner forwarding and regression coverage from #19750 through normal ancestry integration. ## Verification - Current own Shared/SDK, database, eval-kernel and Runner builds passed, together with all seven normal application TypeScript programs. Missing generated package exports/workspace links were restored through normal setup; the original setup failures were retained. No source, compiler option, test assertion or timeout was weakened. - Normal migration generation changed only the five intended MCP tables; its SQL is byte-identical to the reviewed MCP delta. Fresh reference databases and both retained master/Muse histories upgraded through 0440, preserving historical hashes, row fields, schema and idempotence. Direct assertions cover the Legacy null default and MCP null/zero defaults. Two selected history cases passed; the other 21 database cases were unselected. - All six current MCP lifecycle cases passed with zero skips: normal PKCE approval intent, ACK recovery after outage, mutation during publication, disable/re-enable with original expiry, company-deletion rollback/commit and key rotation. - All 81 checks in five current whole files passed with zero skips: OpenAPI 34, read-only settings 2, diagnostic catalog 25, PostgreSQL diagnostics 7 and dependency scheduling 13. PostgreSQL fixtures ran normal migrations before test imports. - Earlier canonical proof at App `cd1c8c6` / Cloud `9f67531` includes real 60-second rate and three 15-second quiet windows, useful work and three revokes. Later paired proof at App `43af514` / Cloud `914eb0f` plus its reviewed candidate includes normal DCR, PostgreSQL metadata, a 144,017-byte signed publication, actual ACK, cap/default-413 controls and a real 15-second quiet window. The relevant producer/signing/wire inputs remain byte-identical in this integration; shared changes are qualified above. These remain source-scoped observations, not a fresh full canonical run on the integrated tree. - All known test owners, process groups and OS sockets closed; an independent final PostgreSQL audit found no remaining test sessions. One dependency-test trace recorded 46 opens and 45 close events. Its cause is unknown; this is not claimed as proof of graceful closure. - Fresh exact-head remote checks passed: 59 successful and 4 skipped contexts, with a complete current 5/5 Greptile review and all seven review threads resolved. Targeted local checks do not constitute a new whole-repository test run. ## Risks App and Cloud must agree on identity, revision, namespace, deadlines and revocation. Partial rollout, unavailable authority or missing ACK must retain awake debt. Shared startup/shutdown and settings changed since the earlier canonical profile, so historical observations carry only within their stated source scope. No deployed Railway/Neon suspension, Cloud alarm, savings or complete hosted Dot/Muse acceptance is claimed. ## Model Used OpenAI `gpt-6.1-sol` produced code and verification, with a Codex coordinator reviewing and integrating the changes. The coordinator's exact runtime model ID and context-window sizes were not exposed. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <[email protected]> Co-authored-by: Codex <[email protected]>
This commit is contained in:
74 files changed
+3533
-82
No files matched your search
@@ -94,6 +94,8 @@ Every row below currently blocks on **any retained row**, including completed-lo
|
||||
| `browser_use_runs` | [browser_use_runs](../../packages/db/src/schema/browser_use.ts); [browser-use.ts](../../server/src/services/browser-use.ts) | Any retained row, with no terminal exception. [browser-use.test.ts](../../server/src/__tests__/browser-use.test.ts) supplies domain fixtures. | History/resource: verify exact completion/authority/resource and dependent effects before narrowing. |
|
||||
| `browser_use_browsers` | [browser_use_browsers](../../packages/db/src/schema/browser_use.ts); [browser-use.ts](../../server/src/services/browser-use.ts) | Any retained row, with no terminal exception. [browser-use-work.test.ts](../../server/src/__tests__/browser-use-work.test.ts) supplies domain fixtures. | History/resource: verify exact completion/authority/resource and dependent effects before narrowing. |
|
||||
| `company_secret_proposals` | [company_secret_proposals](../../packages/db/src/schema/company_secret_proposals.ts); [secret-proposals.ts](../../server/src/services/secret-proposals.ts) | Any retained row, with no terminal exception. [secret-proposals-routes.test.ts](../../server/src/__tests__/secret-proposals-routes.test.ts) supplies domain fixtures. | History/deadline: verify exact completion/authority/resource and dependent effects before narrowing. |
|
||||
| `mcp_file_tickets` | [public-mcp/file-transfers.ts](../../server/src/services/public-mcp/file-transfers.ts); [idle-transfer-schedule.ts](../../server/src/services/public-mcp/idle-transfer-schedule.ts) | Unexpired reusable file tickets hold unfinished transfers; expiry history is passive. | Incoming-work/deadline: preserve current grant, resource and exact expiry. |
|
||||
| `mcp_authorization_handoffs` | [public-mcp/oauth.ts](../../server/src/services/public-mcp/oauth.ts); [idle-authorization-handoffs.ts](../../server/src/services/public-mcp/idle-authorization-handoffs.ts) | Approved unconsumed code/device handoffs remain work through actual expiry; undecided anonymous requests do not hold sleep. | Incoming-work/deadline: preserve current owner, code consumption and actual expiry. |
|
||||
| `mcp_oauth_grants` | [mcp_oauth_grants](../../packages/db/src/schema/public_mcp.ts); [public-mcp/oauth.ts](../../server/src/services/public-mcp/oauth.ts) | Any retained row, with no terminal exception. [public-mcp.test.ts](../../server/src/__tests__/public-mcp.test.ts) supplies domain fixtures. | Incoming-work: verify exact completion/authority/resource and dependent effects before narrowing. |
|
||||
| `mcp_mutation_receipts` | [mcp_mutation_receipts](../../packages/db/src/schema/public_mcp.ts); [public-mcp/capabilities.ts](../../server/src/services/public-mcp/capabilities.ts) | Any retained row, with no terminal exception. [public-mcp.test.ts](../../server/src/__tests__/public-mcp.test.ts) supplies domain fixtures. | Completed-history: verify exact completion/authority/resource and dependent effects before narrowing. |
|
||||
| `mcp_event_subscriptions` | [mcp_event_subscriptions](../../packages/db/src/schema/public_mcp.ts); [public-mcp/events.ts](../../server/src/services/public-mcp/events.ts) | Any retained row, with no terminal exception. [public-mcp.test.ts](../../server/src/__tests__/public-mcp.test.ts) supplies domain fixtures. | Incoming-work/deadline: verify exact completion/authority/resource and dependent effects before narrowing. |
|
||||
|
||||
@@ -22,10 +22,28 @@ import {
|
||||
} from "./test-embedded-postgres.js";
|
||||
|
||||
const cleanups: Array<() => Promise<void>> = [];
|
||||
const embeddedPostgresSupport = await getEmbeddedPostgresTestSupport();
|
||||
const historyExternalVariables = {
|
||||
master0346: ["PAPERCLIP_DB_HISTORY_MASTER_REFERENCE_URL", "PAPERCLIP_DB_HISTORY_MASTER_TARGET_URL"],
|
||||
muse0348: ["PAPERCLIP_DB_HISTORY_MUSE_REFERENCE_URL", "PAPERCLIP_DB_HISTORY_MUSE_TARGET_URL"],
|
||||
} as const;
|
||||
const anyHistoryExternal = Object.values(historyExternalVariables).flat().some(name => process.env[name]);
|
||||
const historyExternal = anyHistoryExternal ? Object.fromEntries(Object.entries(historyExternalVariables).map(([baseline, names]) => {
|
||||
const values = names.map(name => {
|
||||
const raw = process.env[name];
|
||||
if (!raw) throw new Error(`Supply all explicit retained-history fixture URLs: missing ${name}`);
|
||||
const url = new URL(raw);
|
||||
if (!["postgres:", "postgresql:"].includes(url.protocol) || !/^[A-Za-z0-9_]+_test$/.test(decodeURIComponent(url.pathname.slice(1)))
|
||||
|| url.search || url.hash) throw new Error(`${name} requires an explicit dedicated *_test database without overrides`);
|
||||
return raw;
|
||||
});
|
||||
return [baseline, { reference: values[0]!, target: values[1]! }];
|
||||
})) : undefined;
|
||||
if (historyExternal && new Set(Object.values(historyExternal).flatMap(pair => [new URL(pair.reference).pathname, new URL(pair.target).pathname])).size !== 4) throw new Error("Retained-history fixture databases must be distinct");
|
||||
const embeddedPostgresSupport: Awaited<ReturnType<typeof getEmbeddedPostgresTestSupport>> = historyExternal ? { supported: true } : await getEmbeddedPostgresTestSupport();
|
||||
const describeEmbeddedPostgres = embeddedPostgresSupport.supported ? describe : describe.skip;
|
||||
|
||||
async function createTempDatabase(): Promise<string> {
|
||||
if (historyExternal) throw new Error("Explicit retained-history mode permits only its two selected migration cases");
|
||||
const db = await startEmbeddedPostgresTestDatabase("paperclip-db-client-");
|
||||
cleanups.push(db.cleanup);
|
||||
return db.connectionString;
|
||||
@@ -275,12 +293,16 @@ describeEmbeddedPostgres("merged Muse migration histories", () => {
|
||||
}
|
||||
|
||||
it.each(["fresh", "master0346", "muse0348"] as const)("migrates %s through the normal loader without rewriting historical receipts", async baseline => {
|
||||
const referenceUrl = await createTempDatabase();
|
||||
const externalPair = historyExternal?.[baseline];
|
||||
const referenceUrl = externalPair?.reference ?? await createTempDatabase();
|
||||
if (externalPair) await applyPendingMigrations(referenceUrl);
|
||||
const adminUrl = new URL(referenceUrl), databaseName = `merged_muse_${baseline.toLowerCase()}`;
|
||||
adminUrl.pathname = "/postgres";
|
||||
await ensurePostgresDatabase(adminUrl.toString(), databaseName);
|
||||
adminUrl.pathname = `/${databaseName}`;
|
||||
const connectionString = adminUrl.toString();
|
||||
if (!externalPair) {
|
||||
adminUrl.pathname = "/postgres";
|
||||
await ensurePostgresDatabase(adminUrl.toString(), databaseName);
|
||||
adminUrl.pathname = `/${databaseName}`;
|
||||
}
|
||||
const connectionString = externalPair?.target ?? adminUrl.toString();
|
||||
const sql = postgres(connectionString, { max: 1, onnotice: () => {} });
|
||||
const reference = postgres(referenceUrl, { max: 1, onnotice: () => {} });
|
||||
try {
|
||||
@@ -303,7 +325,7 @@ describeEmbeddedPostgres("merged Muse migration histories", () => {
|
||||
const retainedRows = baseline === "fresh" ? null : await seedRetainedRows(sql, baseline === "muse0348");
|
||||
const beforeRows = await retainedRows?.();
|
||||
const expectedRetainedRows = beforeRows
|
||||
? { ...beforeRows, run: { ...beforeRows.run, legacy_process_settlement: null } }
|
||||
? { ...beforeRows, run: { ...beforeRows.run, legacy_process_settlement: null }, grant: { ...beforeRows.grant, revocation_index_receipt: null, revocation_revision: 0 } }
|
||||
: beforeRows;
|
||||
const state = await inspectMigrations(connectionString);
|
||||
expect(state.status).toBe("needsMigrations");
|
||||
@@ -330,7 +352,14 @@ describeEmbeddedPostgres("merged Muse migration histories", () => {
|
||||
const currentHash = await migrationHash(entry.file);
|
||||
expect(afterHistory.filter(row => row.hash === currentHash)).toHaveLength(1);
|
||||
}
|
||||
if (retainedRows) expect(retainedColumns(await retainedRows(), expectedRetainedRows!)).toEqual(expectedRetainedRows);
|
||||
if (retainedRows) {
|
||||
const afterRows = await retainedRows();
|
||||
expect(retainedColumns(afterRows, expectedRetainedRows!)).toEqual(expectedRetainedRows);
|
||||
expect(afterRows).toMatchObject({
|
||||
run: { legacy_process_settlement: null },
|
||||
grant: { revocation_index_receipt: null, revocation_revision: 0 },
|
||||
});
|
||||
}
|
||||
const indexes = await sql`
|
||||
SELECT c.relname, i.indisvalid, i.indisready, pg_get_indexdef(c.oid) AS definition
|
||||
FROM pg_index i JOIN pg_class c ON c.oid = i.indexrelid JOIN pg_namespace n ON n.oid = c.relnamespace
|
||||
@@ -347,7 +376,14 @@ describeEmbeddedPostgres("merged Muse migration histories", () => {
|
||||
expect((await inspectMigrations(connectionString)).status).toBe("upToDate");
|
||||
expect(await historyRows(sql)).toEqual(afterHistory);
|
||||
expect(await schemaShape(sql)).toEqual(finalShape);
|
||||
if (retainedRows) expect(retainedColumns(await retainedRows(), expectedRetainedRows!)).toEqual(expectedRetainedRows);
|
||||
if (retainedRows) {
|
||||
const afterRows = await retainedRows();
|
||||
expect(retainedColumns(afterRows, expectedRetainedRows!)).toEqual(expectedRetainedRows);
|
||||
expect(afterRows).toMatchObject({
|
||||
run: { legacy_process_settlement: null },
|
||||
grant: { revocation_index_receipt: null, revocation_revision: 0 },
|
||||
});
|
||||
}
|
||||
} finally {
|
||||
await Promise.all([sql.end(), reference.end()]);
|
||||
}
|
||||
|
||||
@@ -0,0 +1,28 @@
|
||||
CREATE TABLE "mcp_oauth_client_changes" (
|
||||
"client_id" text PRIMARY KEY NOT NULL,
|
||||
"revision" integer NOT NULL,
|
||||
"operation" text NOT NULL,
|
||||
CONSTRAINT "mcp_client_change_revision" CHECK ("mcp_oauth_client_changes"."revision" >= 1),
|
||||
CONSTRAINT "mcp_client_change_operation" CHECK ("mcp_oauth_client_changes"."operation" IN ('upsert','delete'))
|
||||
);
|
||||
--> statement-breakpoint
|
||||
CREATE TABLE "mcp_oauth_client_mirror_state" (
|
||||
"id" text PRIMARY KEY NOT NULL,
|
||||
"enabled" boolean DEFAULT false NOT NULL,
|
||||
"remote_responsibility" boolean DEFAULT false NOT NULL,
|
||||
"device_cutover_stack_id" text,
|
||||
"device_cutover_at" timestamp with time zone,
|
||||
"revision" integer DEFAULT 0 NOT NULL,
|
||||
"receipt" jsonb,
|
||||
CONSTRAINT "mcp_client_mirror_singleton" CHECK ("mcp_oauth_client_mirror_state"."id" = 'public'),
|
||||
CONSTRAINT "mcp_client_mirror_revision" CHECK ("mcp_oauth_client_mirror_state"."revision" >= 0),
|
||||
CONSTRAINT "mcp_device_cutover_identity" CHECK (("mcp_oauth_client_mirror_state"."device_cutover_at" IS NULL AND "mcp_oauth_client_mirror_state"."device_cutover_stack_id" IS NULL) OR ("mcp_oauth_client_mirror_state"."device_cutover_at" IS NOT NULL AND "mcp_oauth_client_mirror_state"."device_cutover_stack_id" IS NOT NULL))
|
||||
);
|
||||
--> statement-breakpoint
|
||||
ALTER TABLE "mcp_oauth_device_requests" ADD COLUMN "ingress_owner" text DEFAULT 'tenant' NOT NULL;--> statement-breakpoint
|
||||
ALTER TABLE "mcp_oauth_grants" ADD COLUMN "revocation_revision" integer DEFAULT 0 NOT NULL;--> statement-breakpoint
|
||||
ALTER TABLE "mcp_oauth_grants" ADD COLUMN "revocation_index_receipt" jsonb;--> statement-breakpoint
|
||||
ALTER TABLE "mcp_oauth_tokens" ADD COLUMN "revocation_revision" integer DEFAULT 0 NOT NULL;--> statement-breakpoint
|
||||
CREATE INDEX "mcp_client_changes_revision_idx" ON "mcp_oauth_client_changes" USING btree ("revision","client_id");--> statement-breakpoint
|
||||
CREATE INDEX "mcp_oauth_tokens_revocation_revision_idx" ON "mcp_oauth_tokens" USING btree ("grant_id","revocation_revision","id");--> statement-breakpoint
|
||||
ALTER TABLE "mcp_oauth_device_requests" ADD CONSTRAINT "mcp_device_ingress_owner" CHECK ("mcp_oauth_device_requests"."ingress_owner" IN ('tenant','cloud'));
|
||||
+489
-3
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"id": "dedfce21-7ff9-4079-b5da-4a74a856a251",
|
||||
"prevId": "5b40e9ac-12aa-4757-aac7-36218c7500a8",
|
||||
"id": "0c990bdc-6df0-44ca-8f21-28124f7b39e7",
|
||||
"prevId": "d290d707-7d29-43a0-a697-06eecbdb743d",
|
||||
"version": "7",
|
||||
"dialect": "postgresql",
|
||||
"tables": {
|
||||
@@ -28984,6 +28984,22 @@
|
||||
"method": "btree",
|
||||
"with": {}
|
||||
},
|
||||
"environments_managed_plugin_environment_idx": {
|
||||
"name": "environments_managed_plugin_environment_idx",
|
||||
"columns": [
|
||||
{
|
||||
"expression": "(\"metadata\" ->> 'managedPluginEnvironment')",
|
||||
"asc": true,
|
||||
"isExpression": true,
|
||||
"nulls": "last"
|
||||
}
|
||||
],
|
||||
"isUnique": true,
|
||||
"where": "jsonb_typeof(\"environments\".\"metadata\" -> 'managedPluginEnvironment') = 'string'",
|
||||
"concurrently": false,
|
||||
"method": "btree",
|
||||
"with": {}
|
||||
},
|
||||
"environments_name_idx": {
|
||||
"name": "environments_name_idx",
|
||||
"columns": [
|
||||
@@ -34864,6 +34880,12 @@
|
||||
"primaryKey": false,
|
||||
"notNull": false
|
||||
},
|
||||
"legacy_process_settlement": {
|
||||
"name": "legacy_process_settlement",
|
||||
"type": "jsonb",
|
||||
"primaryKey": false,
|
||||
"notNull": false
|
||||
},
|
||||
"last_output_at": {
|
||||
"name": "last_output_at",
|
||||
"type": "timestamp with time zone",
|
||||
@@ -35075,6 +35097,28 @@
|
||||
"method": "btree",
|
||||
"with": {}
|
||||
},
|
||||
"heartbeat_runs_tree_hold_suppression_idx": {
|
||||
"name": "heartbeat_runs_tree_hold_suppression_idx",
|
||||
"columns": [
|
||||
{
|
||||
"expression": "company_id",
|
||||
"isExpression": false,
|
||||
"asc": true,
|
||||
"nulls": "last"
|
||||
},
|
||||
{
|
||||
"expression": "(\"result_json\"->'treeHoldSuppression'->>'holdId')",
|
||||
"asc": true,
|
||||
"isExpression": true,
|
||||
"nulls": "last"
|
||||
}
|
||||
],
|
||||
"isUnique": false,
|
||||
"where": "\"heartbeat_runs\".\"status\" = 'cancelled' and \"heartbeat_runs\".\"result_json\" ? 'treeHoldSuppression'",
|
||||
"concurrently": false,
|
||||
"method": "btree",
|
||||
"with": {}
|
||||
},
|
||||
"heartbeat_runs_cost_accounting_pending_idx": {
|
||||
"name": "heartbeat_runs_cost_accounting_pending_idx",
|
||||
"columns": [
|
||||
@@ -44419,6 +44463,256 @@
|
||||
"checkConstraints": {},
|
||||
"isRLSEnabled": false
|
||||
},
|
||||
"public.native_provider_artifact_references": {
|
||||
"name": "native_provider_artifact_references",
|
||||
"schema": "",
|
||||
"columns": {
|
||||
"id": {
|
||||
"name": "id",
|
||||
"type": "uuid",
|
||||
"primaryKey": true,
|
||||
"notNull": true,
|
||||
"default": "gen_random_uuid()"
|
||||
},
|
||||
"company_id": {
|
||||
"name": "company_id",
|
||||
"type": "uuid",
|
||||
"primaryKey": false,
|
||||
"notNull": true
|
||||
},
|
||||
"artifact_id": {
|
||||
"name": "artifact_id",
|
||||
"type": "uuid",
|
||||
"primaryKey": false,
|
||||
"notNull": true
|
||||
},
|
||||
"run_id": {
|
||||
"name": "run_id",
|
||||
"type": "uuid",
|
||||
"primaryKey": false,
|
||||
"notNull": true
|
||||
},
|
||||
"backup_id": {
|
||||
"name": "backup_id",
|
||||
"type": "uuid",
|
||||
"primaryKey": false,
|
||||
"notNull": false
|
||||
},
|
||||
"created_at": {
|
||||
"name": "created_at",
|
||||
"type": "timestamp with time zone",
|
||||
"primaryKey": false,
|
||||
"notNull": true,
|
||||
"default": "now()"
|
||||
}
|
||||
},
|
||||
"indexes": {
|
||||
"native_provider_artifact_references_artifact_idx": {
|
||||
"name": "native_provider_artifact_references_artifact_idx",
|
||||
"columns": [
|
||||
{
|
||||
"expression": "company_id",
|
||||
"isExpression": false,
|
||||
"asc": true,
|
||||
"nulls": "last"
|
||||
},
|
||||
{
|
||||
"expression": "artifact_id",
|
||||
"isExpression": false,
|
||||
"asc": true,
|
||||
"nulls": "last"
|
||||
}
|
||||
],
|
||||
"isUnique": false,
|
||||
"concurrently": false,
|
||||
"method": "btree",
|
||||
"with": {}
|
||||
},
|
||||
"native_provider_artifact_references_run_idx": {
|
||||
"name": "native_provider_artifact_references_run_idx",
|
||||
"columns": [
|
||||
{
|
||||
"expression": "company_id",
|
||||
"isExpression": false,
|
||||
"asc": true,
|
||||
"nulls": "last"
|
||||
},
|
||||
{
|
||||
"expression": "run_id",
|
||||
"isExpression": false,
|
||||
"asc": true,
|
||||
"nulls": "last"
|
||||
},
|
||||
{
|
||||
"expression": "artifact_id",
|
||||
"isExpression": false,
|
||||
"asc": true,
|
||||
"nulls": "last"
|
||||
}
|
||||
],
|
||||
"isUnique": true,
|
||||
"where": "\"native_provider_artifact_references\".\"backup_id\" is null",
|
||||
"concurrently": false,
|
||||
"method": "btree",
|
||||
"with": {}
|
||||
},
|
||||
"native_provider_artifact_references_backup_idx": {
|
||||
"name": "native_provider_artifact_references_backup_idx",
|
||||
"columns": [
|
||||
{
|
||||
"expression": "company_id",
|
||||
"isExpression": false,
|
||||
"asc": true,
|
||||
"nulls": "last"
|
||||
},
|
||||
{
|
||||
"expression": "artifact_id",
|
||||
"isExpression": false,
|
||||
"asc": true,
|
||||
"nulls": "last"
|
||||
},
|
||||
{
|
||||
"expression": "backup_id",
|
||||
"isExpression": false,
|
||||
"asc": true,
|
||||
"nulls": "last"
|
||||
}
|
||||
],
|
||||
"isUnique": true,
|
||||
"where": "\"native_provider_artifact_references\".\"backup_id\" is not null",
|
||||
"concurrently": false,
|
||||
"method": "btree",
|
||||
"with": {}
|
||||
}
|
||||
},
|
||||
"foreignKeys": {
|
||||
"native_provider_artifact_references_artifact_id_native_provider_artifacts_id_fk": {
|
||||
"name": "native_provider_artifact_references_artifact_id_native_provider_artifacts_id_fk",
|
||||
"tableFrom": "native_provider_artifact_references",
|
||||
"tableTo": "native_provider_artifacts",
|
||||
"columnsFrom": [
|
||||
"artifact_id"
|
||||
],
|
||||
"columnsTo": [
|
||||
"id"
|
||||
],
|
||||
"onDelete": "restrict",
|
||||
"onUpdate": "no action"
|
||||
},
|
||||
"native_provider_artifact_references_backup_id_native_harness_backups_id_fk": {
|
||||
"name": "native_provider_artifact_references_backup_id_native_harness_backups_id_fk",
|
||||
"tableFrom": "native_provider_artifact_references",
|
||||
"tableTo": "native_harness_backups",
|
||||
"columnsFrom": [
|
||||
"backup_id"
|
||||
],
|
||||
"columnsTo": [
|
||||
"id"
|
||||
],
|
||||
"onDelete": "cascade",
|
||||
"onUpdate": "no action"
|
||||
}
|
||||
},
|
||||
"compositePrimaryKeys": {},
|
||||
"uniqueConstraints": {},
|
||||
"policies": {},
|
||||
"checkConstraints": {},
|
||||
"isRLSEnabled": false
|
||||
},
|
||||
"public.native_provider_artifacts": {
|
||||
"name": "native_provider_artifacts",
|
||||
"schema": "",
|
||||
"columns": {
|
||||
"id": {
|
||||
"name": "id",
|
||||
"type": "uuid",
|
||||
"primaryKey": true,
|
||||
"notNull": true,
|
||||
"default": "gen_random_uuid()"
|
||||
},
|
||||
"company_id": {
|
||||
"name": "company_id",
|
||||
"type": "uuid",
|
||||
"primaryKey": false,
|
||||
"notNull": true
|
||||
},
|
||||
"closure_digest": {
|
||||
"name": "closure_digest",
|
||||
"type": "text",
|
||||
"primaryKey": false,
|
||||
"notNull": true
|
||||
},
|
||||
"status": {
|
||||
"name": "status",
|
||||
"type": "text",
|
||||
"primaryKey": false,
|
||||
"notNull": true,
|
||||
"default": "'pending'"
|
||||
},
|
||||
"upload_token": {
|
||||
"name": "upload_token",
|
||||
"type": "uuid",
|
||||
"primaryKey": false,
|
||||
"notNull": false
|
||||
},
|
||||
"descriptor": {
|
||||
"name": "descriptor",
|
||||
"type": "jsonb",
|
||||
"primaryKey": false,
|
||||
"notNull": true
|
||||
},
|
||||
"reference": {
|
||||
"name": "reference",
|
||||
"type": "jsonb",
|
||||
"primaryKey": false,
|
||||
"notNull": false
|
||||
},
|
||||
"created_at": {
|
||||
"name": "created_at",
|
||||
"type": "timestamp with time zone",
|
||||
"primaryKey": false,
|
||||
"notNull": true,
|
||||
"default": "now()"
|
||||
},
|
||||
"updated_at": {
|
||||
"name": "updated_at",
|
||||
"type": "timestamp with time zone",
|
||||
"primaryKey": false,
|
||||
"notNull": true,
|
||||
"default": "now()"
|
||||
}
|
||||
},
|
||||
"indexes": {
|
||||
"native_provider_artifacts_company_digest_idx": {
|
||||
"name": "native_provider_artifacts_company_digest_idx",
|
||||
"columns": [
|
||||
{
|
||||
"expression": "company_id",
|
||||
"isExpression": false,
|
||||
"asc": true,
|
||||
"nulls": "last"
|
||||
},
|
||||
{
|
||||
"expression": "closure_digest",
|
||||
"isExpression": false,
|
||||
"asc": true,
|
||||
"nulls": "last"
|
||||
}
|
||||
],
|
||||
"isUnique": true,
|
||||
"where": "\"native_provider_artifacts\".\"status\" = 'ready'",
|
||||
"concurrently": false,
|
||||
"method": "btree",
|
||||
"with": {}
|
||||
}
|
||||
},
|
||||
"foreignKeys": {},
|
||||
"compositePrimaryKeys": {},
|
||||
"uniqueConstraints": {},
|
||||
"policies": {},
|
||||
"checkConstraints": {},
|
||||
"isRLSEnabled": false
|
||||
},
|
||||
"public.native_run_finalizations": {
|
||||
"name": "native_run_finalizations",
|
||||
"schema": "",
|
||||
@@ -64037,6 +64331,139 @@
|
||||
"checkConstraints": {},
|
||||
"isRLSEnabled": false
|
||||
},
|
||||
"public.mcp_oauth_client_changes": {
|
||||
"name": "mcp_oauth_client_changes",
|
||||
"schema": "",
|
||||
"columns": {
|
||||
"client_id": {
|
||||
"name": "client_id",
|
||||
"type": "text",
|
||||
"primaryKey": true,
|
||||
"notNull": true
|
||||
},
|
||||
"revision": {
|
||||
"name": "revision",
|
||||
"type": "integer",
|
||||
"primaryKey": false,
|
||||
"notNull": true
|
||||
},
|
||||
"operation": {
|
||||
"name": "operation",
|
||||
"type": "text",
|
||||
"primaryKey": false,
|
||||
"notNull": true
|
||||
}
|
||||
},
|
||||
"indexes": {
|
||||
"mcp_client_changes_revision_idx": {
|
||||
"name": "mcp_client_changes_revision_idx",
|
||||
"columns": [
|
||||
{
|
||||
"expression": "revision",
|
||||
"isExpression": false,
|
||||
"asc": true,
|
||||
"nulls": "last"
|
||||
},
|
||||
{
|
||||
"expression": "client_id",
|
||||
"isExpression": false,
|
||||
"asc": true,
|
||||
"nulls": "last"
|
||||
}
|
||||
],
|
||||
"isUnique": false,
|
||||
"concurrently": false,
|
||||
"method": "btree",
|
||||
"with": {}
|
||||
}
|
||||
},
|
||||
"foreignKeys": {},
|
||||
"compositePrimaryKeys": {},
|
||||
"uniqueConstraints": {},
|
||||
"policies": {},
|
||||
"checkConstraints": {
|
||||
"mcp_client_change_revision": {
|
||||
"name": "mcp_client_change_revision",
|
||||
"value": "\"mcp_oauth_client_changes\".\"revision\" >= 1"
|
||||
},
|
||||
"mcp_client_change_operation": {
|
||||
"name": "mcp_client_change_operation",
|
||||
"value": "\"mcp_oauth_client_changes\".\"operation\" IN ('upsert','delete')"
|
||||
}
|
||||
},
|
||||
"isRLSEnabled": false
|
||||
},
|
||||
"public.mcp_oauth_client_mirror_state": {
|
||||
"name": "mcp_oauth_client_mirror_state",
|
||||
"schema": "",
|
||||
"columns": {
|
||||
"id": {
|
||||
"name": "id",
|
||||
"type": "text",
|
||||
"primaryKey": true,
|
||||
"notNull": true
|
||||
},
|
||||
"enabled": {
|
||||
"name": "enabled",
|
||||
"type": "boolean",
|
||||
"primaryKey": false,
|
||||
"notNull": true,
|
||||
"default": false
|
||||
},
|
||||
"remote_responsibility": {
|
||||
"name": "remote_responsibility",
|
||||
"type": "boolean",
|
||||
"primaryKey": false,
|
||||
"notNull": true,
|
||||
"default": false
|
||||
},
|
||||
"device_cutover_stack_id": {
|
||||
"name": "device_cutover_stack_id",
|
||||
"type": "text",
|
||||
"primaryKey": false,
|
||||
"notNull": false
|
||||
},
|
||||
"device_cutover_at": {
|
||||
"name": "device_cutover_at",
|
||||
"type": "timestamp with time zone",
|
||||
"primaryKey": false,
|
||||
"notNull": false
|
||||
},
|
||||
"revision": {
|
||||
"name": "revision",
|
||||
"type": "integer",
|
||||
"primaryKey": false,
|
||||
"notNull": true,
|
||||
"default": 0
|
||||
},
|
||||
"receipt": {
|
||||
"name": "receipt",
|
||||
"type": "jsonb",
|
||||
"primaryKey": false,
|
||||
"notNull": false
|
||||
}
|
||||
},
|
||||
"indexes": {},
|
||||
"foreignKeys": {},
|
||||
"compositePrimaryKeys": {},
|
||||
"uniqueConstraints": {},
|
||||
"policies": {},
|
||||
"checkConstraints": {
|
||||
"mcp_client_mirror_singleton": {
|
||||
"name": "mcp_client_mirror_singleton",
|
||||
"value": "\"mcp_oauth_client_mirror_state\".\"id\" = 'public'"
|
||||
},
|
||||
"mcp_client_mirror_revision": {
|
||||
"name": "mcp_client_mirror_revision",
|
||||
"value": "\"mcp_oauth_client_mirror_state\".\"revision\" >= 0"
|
||||
},
|
||||
"mcp_device_cutover_identity": {
|
||||
"name": "mcp_device_cutover_identity",
|
||||
"value": "(\"mcp_oauth_client_mirror_state\".\"device_cutover_at\" IS NULL AND \"mcp_oauth_client_mirror_state\".\"device_cutover_stack_id\" IS NULL) OR (\"mcp_oauth_client_mirror_state\".\"device_cutover_at\" IS NOT NULL AND \"mcp_oauth_client_mirror_state\".\"device_cutover_stack_id\" IS NOT NULL)"
|
||||
}
|
||||
},
|
||||
"isRLSEnabled": false
|
||||
},
|
||||
"public.mcp_oauth_clients": {
|
||||
"name": "mcp_oauth_clients",
|
||||
"schema": "",
|
||||
@@ -64105,6 +64532,13 @@
|
||||
"primaryKey": false,
|
||||
"notNull": true
|
||||
},
|
||||
"ingress_owner": {
|
||||
"name": "ingress_owner",
|
||||
"type": "text",
|
||||
"primaryKey": false,
|
||||
"notNull": true,
|
||||
"default": "'tenant'"
|
||||
},
|
||||
"device_code_hash": {
|
||||
"name": "device_code_hash",
|
||||
"type": "text",
|
||||
@@ -64260,7 +64694,12 @@
|
||||
"compositePrimaryKeys": {},
|
||||
"uniqueConstraints": {},
|
||||
"policies": {},
|
||||
"checkConstraints": {},
|
||||
"checkConstraints": {
|
||||
"mcp_device_ingress_owner": {
|
||||
"name": "mcp_device_ingress_owner",
|
||||
"value": "\"mcp_oauth_device_requests\".\"ingress_owner\" IN ('tenant','cloud')"
|
||||
}
|
||||
},
|
||||
"isRLSEnabled": false
|
||||
},
|
||||
"public.mcp_oauth_grants": {
|
||||
@@ -64323,6 +64762,19 @@
|
||||
"primaryKey": false,
|
||||
"notNull": false
|
||||
},
|
||||
"revocation_revision": {
|
||||
"name": "revocation_revision",
|
||||
"type": "integer",
|
||||
"primaryKey": false,
|
||||
"notNull": true,
|
||||
"default": 0
|
||||
},
|
||||
"revocation_index_receipt": {
|
||||
"name": "revocation_index_receipt",
|
||||
"type": "jsonb",
|
||||
"primaryKey": false,
|
||||
"notNull": false
|
||||
},
|
||||
"created_at": {
|
||||
"name": "created_at",
|
||||
"type": "timestamp with time zone",
|
||||
@@ -64653,6 +65105,13 @@
|
||||
"primaryKey": false,
|
||||
"notNull": false
|
||||
},
|
||||
"revocation_revision": {
|
||||
"name": "revocation_revision",
|
||||
"type": "integer",
|
||||
"primaryKey": false,
|
||||
"notNull": true,
|
||||
"default": 0
|
||||
},
|
||||
"expires_at": {
|
||||
"name": "expires_at",
|
||||
"type": "timestamp with time zone",
|
||||
@@ -64698,6 +65157,33 @@
|
||||
"method": "btree",
|
||||
"with": {}
|
||||
},
|
||||
"mcp_oauth_tokens_revocation_revision_idx": {
|
||||
"name": "mcp_oauth_tokens_revocation_revision_idx",
|
||||
"columns": [
|
||||
{
|
||||
"expression": "grant_id",
|
||||
"isExpression": false,
|
||||
"asc": true,
|
||||
"nulls": "last"
|
||||
},
|
||||
{
|
||||
"expression": "revocation_revision",
|
||||
"isExpression": false,
|
||||
"asc": true,
|
||||
"nulls": "last"
|
||||
},
|
||||
{
|
||||
"expression": "id",
|
||||
"isExpression": false,
|
||||
"asc": true,
|
||||
"nulls": "last"
|
||||
}
|
||||
],
|
||||
"isUnique": false,
|
||||
"concurrently": false,
|
||||
"method": "btree",
|
||||
"with": {}
|
||||
},
|
||||
"mcp_oauth_tokens_expiry_idx": {
|
||||
"name": "mcp_oauth_tokens_expiry_idx",
|
||||
"columns": [
|
||||
+7
@@ -3067,6 +3067,13 @@
|
||||
"when": 1794024254666,
|
||||
"tag": "0439_cultured_impossible_man",
|
||||
"breakpoints": true
|
||||
},
|
||||
{
|
||||
"idx": 440,
|
||||
"version": "7",
|
||||
"when": 1794035686391,
|
||||
"tag": "0440_flippant_the_leader",
|
||||
"breakpoints": true
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -33,6 +33,12 @@ export const mcpOauthGrants = pgTable("mcp_oauth_grants", {
|
||||
agentId: uuid("agent_id").references(() => agents.id, { onDelete: "cascade" }),
|
||||
scopes: jsonb("scopes").$type<string[]>().notNull(),
|
||||
revokedAt: timestamp("revoked_at", { withTimezone: true }),
|
||||
|
||||
|
||||
revocationRevision: integer("revocation_revision").notNull().default(0),
|
||||
revocationIndexReceipt: jsonb("revocation_index_receipt").$type<{
|
||||
version: 1; authorityId: string; keyId: string; revision: number; generation: string;
|
||||
}>(),
|
||||
createdAt: timestamp("created_at", { withTimezone: true }).notNull().defaultNow(),
|
||||
}, (t) => [
|
||||
index("mcp_oauth_grants_user_company_idx").on(t.userId, t.companyId),
|
||||
@@ -65,18 +71,21 @@ export const mcpOauthTokens = pgTable("mcp_oauth_tokens", {
|
||||
tokenHash: text("token_hash").notNull(),
|
||||
kind: text("kind").$type<"access" | "refresh">().notNull(),
|
||||
usedAt: timestamp("used_at", { withTimezone: true }),
|
||||
revocationRevision: integer("revocation_revision").notNull().default(0),
|
||||
// Null only for renewable agent connection refresh tokens; access tokens expire.
|
||||
expiresAt: timestamp("expires_at", { withTimezone: true }),
|
||||
createdAt: timestamp("created_at", { withTimezone: true }).notNull().defaultNow(),
|
||||
}, (t) => [
|
||||
uniqueIndex("mcp_oauth_tokens_hash_uq").on(t.tokenHash),
|
||||
index("mcp_oauth_tokens_grant_idx").on(t.grantId),
|
||||
index("mcp_oauth_tokens_revocation_revision_idx").on(t.grantId, t.revocationRevision, t.id),
|
||||
index("mcp_oauth_tokens_expiry_idx").on(t.expiresAt),
|
||||
]);
|
||||
|
||||
export const mcpOauthDeviceRequests = pgTable("mcp_oauth_device_requests", {
|
||||
id: uuid("id").primaryKey().defaultRandom(),
|
||||
clientId: text("client_id").notNull().references(() => mcpOauthClients.id, { onDelete: "cascade" }),
|
||||
ingressOwner:text("ingress_owner").$type<"tenant"|"cloud">().notNull().default("tenant"),
|
||||
deviceCodeHash: text("device_code_hash").notNull(),
|
||||
userCodeHash: text("user_code_hash").notNull(),
|
||||
resource: text("resource").notNull(),
|
||||
@@ -90,6 +99,7 @@ export const mcpOauthDeviceRequests = pgTable("mcp_oauth_device_requests", {
|
||||
expiresAt: timestamp("expires_at", { withTimezone: true }).notNull(),
|
||||
createdAt: timestamp("created_at", { withTimezone: true }).notNull().defaultNow(),
|
||||
}, (t) => [
|
||||
check("mcp_device_ingress_owner",sql`${t.ingressOwner} IN ('tenant','cloud')`),
|
||||
uniqueIndex("mcp_oauth_device_code_uq").on(t.deviceCodeHash),
|
||||
uniqueIndex("mcp_oauth_user_code_uq").on(t.userCodeHash),
|
||||
index("mcp_oauth_device_expiry_idx").on(t.expiresAt),
|
||||
@@ -207,3 +217,18 @@ export const mcpFileTickets = pgTable("mcp_file_tickets", {
|
||||
expiresAt: timestamp("expires_at", { withTimezone: true }).notNull(),
|
||||
createdAt: timestamp("created_at", { withTimezone: true }).notNull().defaultNow(),
|
||||
}, (t) => [index("mcp_file_ticket_expiry_idx").on(t.expiresAt), index("mcp_file_ticket_grant_idx").on(t.grantId)]);
|
||||
|
||||
export const mcpOauthClientMirrorState = pgTable("mcp_oauth_client_mirror_state", {
|
||||
id: text("id").primaryKey(),
|
||||
enabled:boolean("enabled").notNull().default(false),
|
||||
remoteResponsibility:boolean("remote_responsibility").notNull().default(false),
|
||||
deviceCutoverStackId:text("device_cutover_stack_id"),
|
||||
deviceCutoverAt:timestamp("device_cutover_at",{withTimezone:true}),
|
||||
revision: integer("revision").notNull().default(0),
|
||||
receipt: jsonb("receipt").$type<{ version: 1; instanceId: string; origin: string; revision: number; enabled:boolean; frontDoor?:"tenant-personal-v1";authorizationOrigin?:string; deviceStatus?:{keyId:string;generation:string;count:number}; generation: string }>(),
|
||||
}, t => [check("mcp_client_mirror_singleton",sql`${t.id} = 'public'`),check("mcp_client_mirror_revision",sql`${t.revision} >= 0`),check("mcp_device_cutover_identity",sql`(${t.deviceCutoverAt} IS NULL AND ${t.deviceCutoverStackId} IS NULL) OR (${t.deviceCutoverAt} IS NOT NULL AND ${t.deviceCutoverStackId} IS NOT NULL)`)]);
|
||||
export const mcpOauthClientChanges = pgTable("mcp_oauth_client_changes", {
|
||||
clientId: text("client_id").primaryKey(),
|
||||
revision: integer("revision").notNull(),
|
||||
operation: text("operation").$type<"upsert"|"delete">().notNull(),
|
||||
}, t => [index("mcp_client_changes_revision_idx").on(t.revision,t.clientId),check("mcp_client_change_revision",sql`${t.revision} >= 1`),check("mcp_client_change_operation",sql`${t.operation} IN ('upsert','delete')`)]);
|
||||
@@ -128,11 +128,7 @@ describeEmbeddedPostgres("heartbeat dependency-aware queued run selection", () =
|
||||
}
|
||||
await new Promise((resolve) => setTimeout(resolve, 50));
|
||||
}
|
||||
const runIds = await db
|
||||
.select({ id: heartbeatRuns.id })
|
||||
.from(heartbeatRuns)
|
||||
.then((runs) => runs.map((run) => run.id));
|
||||
await Promise.all(runIds.map((runId) => heartbeat.waitForRunExecutionDrain(runId)));
|
||||
await heartbeat.drainActiveRunExecutions();
|
||||
mockAdapterExecute.mockReset();
|
||||
mockAdapterExecute.mockImplementation(async () => ({
|
||||
exitCode: 0,
|
||||
|
||||
@@ -147,8 +147,9 @@ const DELETE_METHOD = "DELETE" as never;
|
||||
return { ...f, ...p, ...c, endpoint, conversation, principal, authority };
|
||||
}
|
||||
|
||||
it("checks all 32 families on the real migrated schema and keeps an empty instance eligible", async () => {
|
||||
expect(new Set(completedHistoryChecks.map(c => c.reason)).size).toBe(32);
|
||||
it("checks all 34 families on the real migrated schema and keeps an empty instance eligible", async () => {
|
||||
expect(new Set(completedHistoryChecks.map(c => c.reason)).size).toBe(34);
|
||||
expect(completedHistoryChecks.map(c => c.reason)).toEqual(expect.arrayContaining(["mcp_file_tickets", "mcp_authorization_handoffs"]));
|
||||
for (const check of completedHistoryChecks) expect(await blocked(check.reason), check.reason).toBe(false);
|
||||
expect(await read()).toEqual({ version: 1, backgroundWork: "none" });
|
||||
});
|
||||
|
||||
@@ -64,7 +64,11 @@ if (!support.supported) console.warn(`Skipping idle sleep diagnostics Postgres t
|
||||
expect(report.diagnostics?.checks).toContainEqual({ reason: "budget_evaluation_work", state: "none" });
|
||||
expect(report.diagnostics?.checks.filter(check => !check.reason.includes(".") ||
|
||||
["agents.execution_or_timer", "agents.lifecycle", "agents.company_hold", "heartbeat_runs.budget_cancellation"].includes(check.reason)))
|
||||
.toHaveLength(73);
|
||||
.toHaveLength(75);
|
||||
expect(report.diagnostics?.checks).toEqual(expect.arrayContaining([
|
||||
{ reason: "mcp_file_tickets", state: "none" },
|
||||
{ reason: "mcp_authorization_handoffs", state: "none" },
|
||||
]));
|
||||
expect(await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id, f.runId))).toHaveLength(1);
|
||||
await db.transaction(tx => queueBudgetEvaluation(tx as unknown as Db, { companyId: f.companyId, agentId: f.agentId }));
|
||||
const pending = await read();
|
||||
|
||||
@@ -40,7 +40,8 @@ describe("owned idle sleep diagnostics", () => {
|
||||
expect(result.diagnostics?.complete).toBe(true);
|
||||
expect(result.diagnostics?.checks.filter(check => check.state === "present").map(check => check.reason))
|
||||
.toEqual(["agent_api_keys", "budget_evaluation_work", "chat_deliveries", "heartbeat_runs", "muse_mailbox_items"]);
|
||||
expect(f.keys()).toHaveLength(73);
|
||||
expect(f.keys()).toHaveLength(75);
|
||||
expect(f.keys()).toEqual(expect.arrayContaining(["mcp_file_tickets", "mcp_authorization_handoffs"]));
|
||||
expect(new Set(f.keys()).size).toBe(f.keys().length);
|
||||
const inventory = readFileSync(new URL("../../../doc/plans/2026-11-02-sleep-blocker-inventory.md", import.meta.url), "utf8");
|
||||
const documented = [...inventory.matchAll(/^\| `([a-z0-9_.]+)` \|/gm)].map(match => match[1]);
|
||||
|
||||
@@ -8,6 +8,7 @@ import { MUSE_ASSET_VERSION, MUSE_PUBLIC_ROUTES } from "@paperclipai/shared";
|
||||
import { COMPANY_IMPORT_TRANSFERS_ROUTE_PATH } from "@paperclipai/shared/company-import-transfer";
|
||||
import { errorHandler } from "../middleware/index.js";
|
||||
import { buildOpenApiSpec, openApiRoutes } from "../routes/openapi.js";
|
||||
import { mcpDiscoveryDocuments } from "../services/public-mcp/discovery.js";
|
||||
|
||||
const __dirname = path.dirname(fileURLToPath(import.meta.url));
|
||||
const ROUTES_DIR = path.resolve(__dirname, "../routes");
|
||||
@@ -133,6 +134,9 @@ const explicitOpenApiOperationCoverageExclusions = new Set([
|
||||
// browser connection-management operations remain documented in the board API.
|
||||
// Markdown rendering of the separately documented assistant setup page.
|
||||
"GET /api/mcp/setup.md",
|
||||
"GET /.well-known/oauth-protected-resource",
|
||||
"GET /.well-known/oauth-protected-resource/mcp/paperclip",
|
||||
"GET /.well-known/oauth-protected-resource/mcp/runner",
|
||||
"GET /.well-known/oauth-authorization-server",
|
||||
"POST /mcp/oauth/register",
|
||||
"POST /mcp/oauth/device_authorization",
|
||||
@@ -256,9 +260,13 @@ function loadActualRoutes() {
|
||||
|
||||
if (file === "public-mcp.ts") {
|
||||
// The shared gateway mounts these protocol paths for each OAuth resource.
|
||||
if (source.includes("router.get(metadataPath,")) {
|
||||
excludedRoutes.add("GET /.well-known/oauth-authorization-server");
|
||||
excludedRoutes.add("GET /.well-known/oauth-authorization-server/mcp/runner/oauth");
|
||||
if (source.includes("mcpDiscoveryDocuments({") && source.includes("router.get(entry.path,")) {
|
||||
for (const entry of mcpDiscoveryDocuments({ origin: "https://example.test",
|
||||
authorizationOrigin: "https://example.test", profiles: ["personal-mcp", "dot-mcp"] })) {
|
||||
const operation = `GET ${entry.path}`;
|
||||
if (explicitOpenApiOperationCoverageExclusions.has(operation)) excludedRoutes.add(operation);
|
||||
else routes.add(operation);
|
||||
}
|
||||
}
|
||||
for (const match of source.matchAll(/router\.(get|post)\(oauthPath \+ "([^"]+)"/g)) {
|
||||
for (const oauthPath of ["/mcp/oauth", "/mcp/runner/oauth"]) {
|
||||
|
||||
@@ -1,4 +1,8 @@
|
||||
import type { IdleDeadlineOwners } from "./services/idle-deadline-types.js";
|
||||
import { startMcpClientMirrorSync } from "./services/public-mcp/client-mirror-sync.js";
|
||||
import { registerIdleMcpOwners } from "./services/public-mcp/idle-sleep-proof.js";
|
||||
import { startMcpRevocationIndexSync } from "./services/public-mcp/revocation-index-sync.js";
|
||||
import { startMcpIngressHandoffExpiry } from "./services/public-mcp/ingress-handoff-expiry.js";
|
||||
import { nativeRunMcpRoutes } from "./routes/native-run-mcp.js";
|
||||
import { connectionEventWakeRoutes } from "./routes/connection-event-wake.js";
|
||||
import { registerGitHubReviewProjectionWork } from "./services/chat-channels/github/review-projection-work.js";
|
||||
@@ -1231,6 +1235,10 @@ export async function createApp(
|
||||
nextRunAt: () => agentLifecycle.nextRunAt(),
|
||||
});
|
||||
agentLifecycle.setWake(agentLifecycleDelivery.wake);
|
||||
const clientMirror = startMcpClientMirrorSync(db, deliveryWork, mcpConfig?.origin,{authorizationOrigin:mcpConfig?.authorizationOrigin});
|
||||
const unregisterMcpOwners = registerIdleMcpOwners(db, { config: mcpConfig, personal: publicMcpEvents, dot: dotMcpEvents, clientMirrorInstanceId:clientMirror.instanceId });
|
||||
startMcpRevocationIndexSync(db, deliveryWork);
|
||||
startMcpIngressHandoffExpiry(db, deliveryWork);
|
||||
publicMcpEvents?.start(deliveryWork);
|
||||
dotMcpEvents?.start(deliveryWork);
|
||||
if (publicMcpOAuth) grokDispatcher.start(deliveryWork);
|
||||
@@ -1366,6 +1374,7 @@ export async function createApp(
|
||||
if (appServicesShutdown) return appServicesShutdown;
|
||||
appServicesShutdown = finishServiceShutdown(() => scheduler.stop(), async () => {
|
||||
await agentLifecycle.stop();
|
||||
unregisterMcpOwners();
|
||||
await publicMcpEvents?.stop();
|
||||
await dotMcpEvents?.stop();
|
||||
jobCoordinator.stop();
|
||||
|
||||
@@ -0,0 +1,23 @@
|
||||
import { sql, type SQL } from "drizzle-orm";
|
||||
import type { IdleMcpWakeProof } from "../../services/public-mcp/idle-wake-proof.js";
|
||||
import { idleExternalAgentHistoryChecks } from "./idle-history-proof.js";
|
||||
|
||||
|
||||
|
||||
export function idleExternalAgentIncomingWorkQuery(reason: string, proof?: IdleMcpWakeProof): SQL | undefined {
|
||||
if (!proof?.dotBindingIds.length) return;
|
||||
const ids = sql.join(proof.dotBindingIds.map(id => sql`${id}::uuid`), sql`, `);
|
||||
const check = idleExternalAgentHistoryChecks.find(check => check.reason === reason);
|
||||
if (!check) return;
|
||||
if (reason === "dot_agent_bindings") return sql`${sql.raw(check.query)} AND b.id NOT IN (${ids})`;
|
||||
if (reason === "dot_mailbox_items") {
|
||||
|
||||
|
||||
|
||||
return sql`${sql.raw(check.query)} AND NOT (m.binding_id IN (${ids}) AND m.assignment_id IS NULL
|
||||
AND m.kind = 'readiness_challenge' AND EXISTS (SELECT 1 FROM dot_agent_bindings binding
|
||||
WHERE binding.id = m.binding_id AND binding.company_id = m.company_id
|
||||
AND m.binding_generation > 0 AND m.binding_generation <= binding.generation
|
||||
AND binding.challenge_hash IS NULL AND binding.pairing_code_hash IS NULL))`;
|
||||
}
|
||||
}
|
||||
@@ -6,3 +6,5 @@ export { dotRunnerBroker, createDotRunnerMcpTools } from "./dot.js";
|
||||
export { notifyExternalAgentSettlement, pendingMuseTerminalBindings, type ExternalAgentSettlementScope } from "./muse-terminal-settlement.js";
|
||||
|
||||
export { idleExternalAgentHistoryChecks } from "./idle-history-proof.js";
|
||||
|
||||
export { idleExternalAgentIncomingWorkQuery } from "./idle-incoming-proof.js";
|
||||
@@ -1,3 +1,6 @@
|
||||
import {cloudEventAuthoritySchema} from "../services/public-mcp/cloud-event-authority.js";
|
||||
import { verifyMcpDeviceMintContext } from "../services/public-mcp/device-mint-authority.js";
|
||||
import { mcpDiscoveryDocuments } from "../services/public-mcp/discovery.js";
|
||||
import express, { Router, type ErrorRequestHandler, type RequestHandler } from "express";
|
||||
import { Server } from "@modelcontextprotocol/sdk/server/index.js";
|
||||
import { StreamableHTTPServerTransport } from "@modelcontextprotocol/sdk/server/streamableHttp.js";
|
||||
@@ -6,9 +9,9 @@ import { z } from "zod";
|
||||
import { type PublicMcpEvents } from "../services/public-mcp/events.js";
|
||||
import type { PublicMcpToolExtension } from "../services/public-mcp/dot-runner.js";
|
||||
import { McpEventError } from "../services/public-mcp/event-webhooks.js";
|
||||
import { DOT_RUNNER_MCP_PATH, DOT_RUNNER_MCP_SCOPES, PUBLIC_MCP_SCOPES, mcpConsentSchema, mcpInvitation, mcpSetupUrl } from "@paperclipai/shared";
|
||||
import { DOT_RUNNER_MCP_PATH, mcpConsentSchema, mcpInvitation, mcpSetupUrl } from "@paperclipai/shared";
|
||||
import { renderMcpSetup, mcpSetupMarkdown } from "../services/public-mcp/setup.js";
|
||||
import { DEVICE_GRANT, McpOAuthError, type PublicMcpOAuth } from "../services/public-mcp/oauth.js";
|
||||
import { McpOAuthError, type PublicMcpOAuth } from "../services/public-mcp/oauth.js";
|
||||
import { McpApiError, McpCapabilityError, publicMcpCapabilities, type createPublicMcpExecutor } from "../services/public-mcp/capabilities.js";
|
||||
|
||||
const oauthErrors: ErrorRequestHandler = (error, _req, res, next) => {
|
||||
@@ -43,10 +46,7 @@ export function publicMcpIngressRoutes(oauth: PublicMcpOAuth, execute: ReturnTyp
|
||||
const endpoint = new URL(resource).pathname;
|
||||
const agentConnection = endpoint === DOT_RUNNER_MCP_PATH;
|
||||
const oauthPath = agentConnection ? endpoint + "/oauth" : "/mcp/oauth";
|
||||
const issuer = agentConnection ? origin + oauthPath : origin;
|
||||
const scopes = agentConnection ? DOT_RUNNER_MCP_SCOPES : PUBLIC_MCP_SCOPES;
|
||||
const metadataPath = "/.well-known/oauth-authorization-server" + (agentConnection ? oauthPath : "");
|
||||
const prefix = origin + oauthPath;
|
||||
const resourceMetadata = origin + "/.well-known/oauth-protected-resource" + endpoint;
|
||||
if (!agentConnection) router.get(["/mcp/setup", "/mcp/setup.md"], async (req, res) => {
|
||||
await oauth.assertEnabled();
|
||||
@@ -70,23 +70,11 @@ export function publicMcpIngressRoutes(oauth: PublicMcpOAuth, execute: ReturnTyp
|
||||
await oauth.assertEnabled();
|
||||
next();
|
||||
});
|
||||
router.get(agentConnection ? ["/.well-known/oauth-protected-resource" + endpoint] : ["/.well-known/oauth-protected-resource", "/.well-known/oauth-protected-resource" + endpoint], (_req, res) => res.json({
|
||||
resource, authorization_servers: [issuer], scopes_supported: agentConnection ? DOT_RUNNER_MCP_SCOPES : ["paperclip:read", "paperclip:write", "paperclip:configure"], bearer_methods_supported: ["header"],
|
||||
resource_name: agentConnection ? "Paperclip agent" : "Paperclip",
|
||||
}));
|
||||
router.get(metadataPath, (_req, res) => res.json({
|
||||
issuer, authorization_endpoint: (oauth.config.authorizationOrigin ?? origin) + oauthPath + "/authorize", token_endpoint: prefix + "/token",
|
||||
registration_endpoint: prefix + "/register", revocation_endpoint: prefix + "/revoke",
|
||||
response_types_supported: ["code"], grant_types_supported: ["authorization_code", "refresh_token", DEVICE_GRANT],
|
||||
device_authorization_endpoint: prefix + "/device_authorization",
|
||||
token_endpoint_auth_methods_supported: ["none"], revocation_endpoint_auth_methods_supported: ["none"],
|
||||
code_challenge_methods_supported: ["S256"], scopes_supported: scopes,
|
||||
authorization_response_iss_parameter_supported: true,
|
||||
client_id_metadata_document_supported: true,
|
||||
}));
|
||||
for (const entry of mcpDiscoveryDocuments({ origin, authorizationOrigin: oauth.config.authorizationOrigin ?? origin,
|
||||
profiles: [agentConnection ? "dot-mcp" : "personal-mcp"] })) router.get(entry.path, (_req,res) => res.json(entry.document));
|
||||
router.use(oauthPath, authRateLimit(), express.urlencoded({ extended: false, limit: "16kb" }));
|
||||
router.post(oauthPath + "/register", async (req, res) => res.status(201).json(await oauth.register(req.body, req.ip ?? req.socket.remoteAddress ?? "unknown")));
|
||||
router.post(oauthPath + "/device_authorization", async (req, res) => res.json(await oauth.deviceAuthorize(req.body, req.ip ?? req.socket.remoteAddress ?? "unknown")));
|
||||
router.post(oauthPath + "/device_authorization", async (req, res) => res.json(await oauth.deviceAuthorize(req.body, req.ip ?? req.socket.remoteAddress ?? "unknown",verifyMcpDeviceMintContext(req,oauth.config.resource))));
|
||||
router.get(oauthPath + "/authorize", async (req, res) => res.redirect(303, await oauth.authorize(req.query, req.ip ?? req.socket.remoteAddress ?? "unknown")));
|
||||
router.post(oauthPath + "/token", async (req, res) => res.json(await oauth.token(req.body ?? {})));
|
||||
router.post(oauthPath + "/revoke", async (req, res) => {
|
||||
@@ -166,8 +154,9 @@ export function publicMcpIngressRoutes(oauth: PublicMcpOAuth, execute: ReturnTyp
|
||||
case "events/subscribe": {
|
||||
if (!events) { fail(-32601, "Events are unavailable."); return; }
|
||||
// The broker supplies its original access proof only for hosted subscriptions.
|
||||
// It is validated against the fixed Cloud origin before being retained encrypted.
|
||||
const authority = z.object({ token: z.string().max(24000), expiresAt: z.number() }).safeParse(meta["ai.paperclip/cloudAuthority"]);
|
||||
// It is validated against the selected verified issuer before being retained encrypted.
|
||||
const rawAuthority=meta["ai.paperclip/cloudAuthority"],authority = cloudEventAuthoritySchema.safeParse(rawAuthority);
|
||||
if(rawAuthority!==undefined&&!authority.success)throw new McpEventError(-32602,"Invalid hosted subscription authority.");
|
||||
result = await events.subscribe(principal!, params, authority.success ? authority.data : undefined); break;
|
||||
}
|
||||
case "events/unsubscribe":
|
||||
|
||||
@@ -55,7 +55,7 @@ export function isValidLocalEncryptedMasterKey(raw: string): boolean {
|
||||
return decodeMasterKey(raw) !== null;
|
||||
}
|
||||
|
||||
function loadOrCreateMasterKey(allowCreate = true, env: LocalEncryptedKeyEnv = process.env): Buffer {
|
||||
function loadOrCreateMasterKey(allowCreate = true, env: LocalEncryptedKeyEnv = process.env, options:{readOnly?:boolean}={}): Buffer {
|
||||
const envKeyRaw = env.PAPERCLIP_SECRETS_MASTER_KEY;
|
||||
if (envKeyRaw && envKeyRaw.trim().length > 0) {
|
||||
const fromEnv = decodeMasterKey(envKeyRaw);
|
||||
@@ -69,7 +69,9 @@ function loadOrCreateMasterKey(allowCreate = true, env: LocalEncryptedKeyEnv = p
|
||||
|
||||
const keyPath = resolveMasterKeyFilePath(env);
|
||||
if (existsSync(keyPath)) {
|
||||
enforceKeyFilePermissionsBestEffort(keyPath);
|
||||
if(options.readOnly){
|
||||
if((statSync(keyPath).mode&0o077)!==0)throw new Error("Secrets master key permissions are unsafe for inspection");
|
||||
}else enforceKeyFilePermissionsBestEffort(keyPath);
|
||||
const raw = readFileSync(keyPath, "utf8");
|
||||
const decoded = decodeMasterKey(raw);
|
||||
if (!decoded) {
|
||||
@@ -78,7 +80,7 @@ function loadOrCreateMasterKey(allowCreate = true, env: LocalEncryptedKeyEnv = p
|
||||
return decoded;
|
||||
}
|
||||
|
||||
if (!allowCreate) throw new Error(`Secrets master key file is missing: ${keyPath}`);
|
||||
if (!allowCreate||options.readOnly) throw new Error(`Secrets master key file is missing: ${keyPath}`);
|
||||
const dir = path.dirname(keyPath);
|
||||
mkdirSync(dir, { recursive: true });
|
||||
const generated = randomBytes(32);
|
||||
@@ -99,8 +101,8 @@ function loadOrCreateMasterKey(allowCreate = true, env: LocalEncryptedKeyEnv = p
|
||||
}
|
||||
|
||||
/** Derive a purpose-specific instance secret without exposing or reusing the vault key. */
|
||||
export function deriveLocalEncryptedInstanceSecret(purpose: string, env: LocalEncryptedKeyEnv = process.env, options: { allowCreate?: boolean } = {}): string {
|
||||
return Buffer.from(hkdfSync("sha256", loadOrCreateMasterKey(options.allowCreate ?? true, env), Buffer.alloc(0),
|
||||
export function deriveLocalEncryptedInstanceSecret(purpose: string, env: LocalEncryptedKeyEnv = process.env, options: { allowCreate?: boolean;readOnly?:boolean } = {}): string {
|
||||
return Buffer.from(hkdfSync("sha256", loadOrCreateMasterKey(options.readOnly?false:options.allowCreate ?? true, env,{readOnly:options.readOnly}), Buffer.alloc(0),
|
||||
`paperclip:instance-secret:v1\0${purpose}`, 32)).toString("base64url");
|
||||
}
|
||||
|
||||
@@ -278,10 +280,11 @@ function resolveLocalEncryptedValue<T>(operation: () => T): T {
|
||||
}
|
||||
|
||||
/** Reuse one resolved key within a single batch; reload it for the next operation. */
|
||||
export function createLocalEncryptedCipher(mode: "read" | "write") {
|
||||
export function createLocalEncryptedCipher(mode: "read" | "write",options:{readOnly?:boolean}={}) {
|
||||
if(mode==="write"&&options.readOnly)throw badRequest("Read-only inspection cannot create an encryption writer");
|
||||
const masterKey = mode === "write"
|
||||
? loadOrCreateMasterKey()
|
||||
: resolveLocalEncryptedValue(() => loadOrCreateMasterKey(false));
|
||||
: resolveLocalEncryptedValue(() => loadOrCreateMasterKey(false,process.env,options));
|
||||
return {
|
||||
encrypt(value: string): StoredSecretVersionMaterial {
|
||||
return encryptValue(masterKey, value);
|
||||
|
||||
@@ -454,6 +454,7 @@ export const CLOUD_CONTROL_ACTIONS = [
|
||||
"lifecycle:read",
|
||||
"lifecycle:unarchive-primary",
|
||||
"connector-events:wake",
|
||||
"mcp-device:mint",
|
||||
] as const;
|
||||
export type CloudControlAction = (typeof CLOUD_CONTROL_ACTIONS)[number];
|
||||
|
||||
|
||||
@@ -1,3 +1,4 @@
|
||||
import { mcpCompanyDeletion } from "./public-mcp/company-deletion.js";
|
||||
import { notifyStatusCardEvaluation } from "./status-card-notifications.js";
|
||||
import { notifyExternalObjectRefresh } from "./external-object-refresh-notifications.js";
|
||||
import { notifyIssueRecovery } from "./issue-recovery-notifications.js";
|
||||
@@ -75,6 +76,7 @@ export async function deleteCompany(db: Db, id: string) {
|
||||
const [existing] = await tx.select({ id: companies.id }).from(companies)
|
||||
.where(eq(companies.id, id)).for("no key update");
|
||||
if (!existing) return null;
|
||||
await mcpCompanyDeletion.deleteCompanyData(tx, id);
|
||||
await notifyExternalObjectRefresh(tx, id);
|
||||
await notifyIssueRecovery(tx, { companyId: id });
|
||||
await notifyChatCompanyConfiguration(tx, id);
|
||||
|
||||
@@ -2,6 +2,9 @@ import { signalDatabaseWork, subscribeDatabaseWork } from "@paperclipai/db/work-
|
||||
|
||||
/** Topics for existing durable queues; register intent before their writes. */
|
||||
export const DELIVERY_QUEUES = {
|
||||
mcpClientMirror: "mcp-client-mirror",
|
||||
mcpIngressHandoffs: "mcp-ingress-handoffs",
|
||||
mcpRevocationIndex: "mcp-revocation-index",
|
||||
nativeRuntimeResponses: "native-runtime-responses",
|
||||
nativeCancellation: "native-cancellation",
|
||||
deploymentNotifications: "deployment-notifications",
|
||||
|
||||
@@ -3,8 +3,9 @@ export interface IdleIncomingWakeCapability {
|
||||
version: 1;
|
||||
ownerId: string;
|
||||
generation: string;
|
||||
routes: ("tenant-api-http-v2" | "github-broker-events-v1")[];
|
||||
routes: ("tenant-api-http-v2" | "github-broker-events-v1" | "mcp-oauth-http-v1")[];
|
||||
apiKeyProof?: { version: 1; challenge: string };
|
||||
mcpOAuthProof?: { version: 1; challenge: string };
|
||||
githubBroker?: { instanceId: string; bindingsFingerprint: string };
|
||||
}
|
||||
export interface IdleCapabilities {
|
||||
@@ -40,6 +41,7 @@ export function parseIdleCapabilities(value: unknown): IdleCapabilities {
|
||||
"routes",
|
||||
"githubBroker",
|
||||
"apiKeyProof",
|
||||
"mcpOAuthProof",
|
||||
]) ||
|
||||
v.version !== 1 ||
|
||||
typeof v.ownerId !== "string" ||
|
||||
@@ -49,9 +51,9 @@ export function parseIdleCapabilities(value: unknown): IdleCapabilities {
|
||||
!digest(v.generation) ||
|
||||
!Array.isArray(v.routes) ||
|
||||
v.routes.length < 1 ||
|
||||
v.routes.length > 2 ||
|
||||
v.routes.length > 3 ||
|
||||
v.routes.some(
|
||||
(r) => !["tenant-api-http-v2", "github-broker-events-v1"].includes(r),
|
||||
(r) => !["tenant-api-http-v2", "github-broker-events-v1", "mcp-oauth-http-v1"].includes(r),
|
||||
) ||
|
||||
new Set(v.routes).size !== v.routes.length ||
|
||||
[...v.routes].sort().join() !== v.routes.join()
|
||||
@@ -62,6 +64,11 @@ export function parseIdleCapabilities(value: unknown): IdleCapabilities {
|
||||
if (!object(proof) || Object.keys(proof).length !== 2 || !exact(proof, ["version", "challenge"])
|
||||
|| proof.version !== 1 || !digest(proof.challenge)) throw new Error("Invalid API wake challenge");
|
||||
} else if (v.apiKeyProof !== undefined) throw new Error("Unexpected API wake challenge");
|
||||
if (v.routes.includes("mcp-oauth-http-v1")) {
|
||||
const proof = v.mcpOAuthProof;
|
||||
if (!object(proof) || Object.keys(proof).length !== 2 || !exact(proof, ["version", "challenge"])
|
||||
|| proof.version !== 1 || !digest(proof.challenge)) throw new Error("Invalid MCP OAuth wake challenge");
|
||||
} else if (v.mcpOAuthProof !== undefined) throw new Error("Unexpected MCP OAuth wake challenge");
|
||||
if (v.routes.includes("github-broker-events-v1")) {
|
||||
const b = v.githubBroker;
|
||||
if (
|
||||
@@ -80,6 +87,7 @@ export function parseIdleCapabilities(value: unknown): IdleCapabilities {
|
||||
if (parsed.incomingWake) {
|
||||
Object.freeze(parsed.incomingWake.routes);
|
||||
if (parsed.incomingWake.apiKeyProof) Object.freeze(parsed.incomingWake.apiKeyProof);
|
||||
if (parsed.incomingWake.mcpOAuthProof) Object.freeze(parsed.incomingWake.mcpOAuthProof);
|
||||
if (parsed.incomingWake.githubBroker)
|
||||
Object.freeze(parsed.incomingWake.githubBroker);
|
||||
Object.freeze(parsed.incomingWake);
|
||||
|
||||
@@ -1,3 +1,6 @@
|
||||
import { readIdleMcpSleepProof, type IdleMcpSleepProof } from "./public-mcp/idle-sleep-proof.js";
|
||||
import { idleMcpWorkQuery } from "./public-mcp/idle-work-query.js";
|
||||
import { idleExternalAgentIncomingWorkQuery } from "../modules/external-agents/index.js";
|
||||
import { apiWakeFingerprint, createApiWakeAdmission, type ApiWakeAdmission } from "./idle-api-wake-admission.js";
|
||||
import { createHash } from "node:crypto";
|
||||
import { and, eq, inArray, sql, type SQL } from "drizzle-orm";
|
||||
@@ -19,6 +22,7 @@ import { hasConnectionEventWakeReceiver } from "./connection-event-wake.js";
|
||||
export interface IdleIncomingWakeProof {
|
||||
capability: IdleIncomingWakeCapability;
|
||||
apiWakeAdmission?: ApiWakeAdmission;
|
||||
mcp?: IdleMcpSleepProof;
|
||||
connectionIds: string[];
|
||||
endpointIds: string[];
|
||||
/** A broker enrollment cannot discharge these credential continuity obligations. */
|
||||
@@ -49,6 +53,7 @@ export async function readIdleIncomingWakeProof(
|
||||
ownerId: string;
|
||||
receiverOwner: object;
|
||||
now?: () => number;
|
||||
inspectionNow?: number;
|
||||
},
|
||||
): Promise<IdleIncomingWakeProof | undefined> {
|
||||
const capability = input.capabilities?.incomingWake;
|
||||
@@ -69,7 +74,11 @@ export async function readIdleIncomingWakeProof(
|
||||
throw new Error("Incoming wake inspection budget exhausted");
|
||||
return Math.max(1, Math.floor(ms));
|
||||
};
|
||||
let queries = 0;
|
||||
const inspectionNow = input.inspectionNow ?? Date.now();
|
||||
const beforeRead = async () => {
|
||||
|
||||
if ((queries += 2) > 64) throw new Error("Incoming wake query budget exhausted");
|
||||
// One shared owner budget, not five seconds multiplied by the row count.
|
||||
await tx.execute(
|
||||
sql`SELECT set_config('statement_timeout', ${`${remaining()}ms`}, true)`,
|
||||
@@ -93,6 +102,13 @@ export async function readIdleIncomingWakeProof(
|
||||
expiresAt: row.expiresAt === null ? null : new Date(row.expiresAt).toISOString() })),
|
||||
});
|
||||
}
|
||||
if (capability.routes.includes("mcp-oauth-http-v1")) {
|
||||
const stackId = process.env.PAPERCLIP_CLOUD_STACK_ID;
|
||||
if (!stackId || !capability.mcpOAuthProof) throw new Error("MCP wake challenge is missing");
|
||||
proof.mcp = await readIdleMcpSleepProof(tx, input.receiverOwner, { stackId, ownerId: input.ownerId,
|
||||
capabilityGeneration: capability.generation, challenge: capability.mcpOAuthProof.challenge, now: inspectionNow, budget: { beforeRead, check: () => { remaining(); } } });
|
||||
remaining();
|
||||
}
|
||||
if (!capability.routes.includes("github-broker-events-v1")) {
|
||||
await tx.execute(sql`SET LOCAL statement_timeout = '5s'`);
|
||||
remaining();
|
||||
@@ -280,6 +296,10 @@ export function incomingWakeWorkQuery(
|
||||
proof?: IdleIncomingWakeProof,
|
||||
continuityConnectionIds: readonly string[] = [],
|
||||
): SQL | undefined {
|
||||
const mcp = proof?.mcp;
|
||||
const mcpQuery = idleMcpWorkQuery(reason, { ingress: mcp?.ingress, subscriptions: mcp?.subscriptions, transfers: mcp?.transfers, handoffs: mcp?.handoffs })
|
||||
?? idleExternalAgentIncomingWorkQuery(reason, mcp?.ingress);
|
||||
if (mcpQuery) return mcpQuery;
|
||||
const connectionIds = [
|
||||
...(proof?.connectionIds ?? []),
|
||||
...(proof?.requiredContinuityConnectionIds.filter((id) =>
|
||||
|
||||
@@ -1,3 +1,5 @@
|
||||
import { idleMcpWorkQuery } from "./public-mcp/idle-work-query.js";
|
||||
import type { McpWakeAdmission } from "./public-mcp/wake-admission.js";
|
||||
import { pendingBudgetEvaluationWorkQuery } from "./budget-evaluation-work.js";
|
||||
import { createIdleDeadlineInspectionBudget, readIdleDeadlineOwners, deadlineOwnerWorkReplacement, readIdleDeadlineProof, deadlineWorkReplacement, type IdleDeadlineProof } from "./idle-deadline-proof.js";
|
||||
import type { IdleDeadlineOwners, DomainScheduledWake, IdleDeadlineReceipt } from "./idle-deadline-types.js";
|
||||
@@ -52,6 +54,7 @@ export interface IdleSleepSafety {
|
||||
incomingWake?: IdleIncomingWakeCapability;
|
||||
deadlineProof?: IdleDeadlineReceipt;
|
||||
apiWakeAdmission?: ApiWakeAdmission;
|
||||
mcpWakeAdmission?: McpWakeAdmission;
|
||||
}
|
||||
|
||||
export type InspectIdlePlugins = (hold: { ownerId: string; expiresAt: number }) => Promise<{
|
||||
@@ -320,7 +323,8 @@ export async function readIdleSleepSafety(
|
||||
const wake = deadlineProof ? deadlineProof.scheduledWake : scheduledWake;
|
||||
return { version: backgroundWork === "none" && (incomingProof || deadlineProof) ? 3 : backgroundWork === "none" && wake ? 2 : 1, backgroundWork,
|
||||
...(backgroundWork === "none" && incomingProof ? { incomingWake: incomingProof.capability,
|
||||
...(incomingProof.apiWakeAdmission ? { apiWakeAdmission: incomingProof.apiWakeAdmission } : {}) } : {}),
|
||||
...(incomingProof.apiWakeAdmission ? { apiWakeAdmission: incomingProof.apiWakeAdmission } : {}),
|
||||
...(incomingProof.mcp ? { mcpWakeAdmission: incomingProof.mcp.ingress.admission } : {}) } : {}),
|
||||
...(backgroundWork === "none" && deadlineProof?.generation ? { deadlineProof: {
|
||||
catalog: "domain_deadlines_v1" as const, generation: deadlineProof.generation } } : {}),
|
||||
...(backgroundWork === "none" && wake ? { scheduledWake: wake } : {}),
|
||||
@@ -396,7 +400,7 @@ export async function readIdleSleepSafety(
|
||||
await tx.execute(sql`SET LOCAL statement_timeout = '5s'`);
|
||||
if (options.incomingWake?.incomingWake) {
|
||||
if (!ownerId || ownerId !== before.ownerId) return undefined;
|
||||
incomingProof = await readIdleIncomingWakeProof(tx, { capabilities: options.incomingWake, ownerId, receiverOwner: db });
|
||||
incomingProof = await readIdleIncomingWakeProof(tx, { capabilities: options.incomingWake, ownerId, receiverOwner: db, inspectionNow: now() });
|
||||
}
|
||||
if (deadlines) {
|
||||
if (!owned() || !scheduledRoutines || !scheduledIssueMonitors) return undefined;
|
||||
@@ -419,7 +423,7 @@ export async function readIdleSleepSafety(
|
||||
deadlineProof = await readIdleDeadlineProof(tx, { ownerId: ownerId!, expiresAt: before.expiresAt!.getTime() }, {
|
||||
now: now(), verifiedCapability: options.deadlineCapability,
|
||||
quietPluginIds: quietPlugins, quietPluginGeneration, quietPluginCatalog,
|
||||
scheduledWake, passiveGeneration: issue.generation, owners, inspectionBudget,
|
||||
scheduledWake, passiveGeneration: issue.generation, owners, inspectionBudget, mcp: incomingProof?.mcp?.schedule,
|
||||
});
|
||||
}
|
||||
diagnostic?.record("deadlines.catalog", deadlineProof.blocked ? "present" : "none");
|
||||
@@ -431,7 +435,10 @@ export async function readIdleSleepSafety(
|
||||
&& (!scheduledIssueMonitors || check.reason !== "issues"))
|
||||
.map(check => {
|
||||
const replacement = deadlineProof?.blocked === false ? deadlineWorkReplacement(check.reason) : undefined;
|
||||
return { ...check, query: incomingWakeWorkQuery(check.reason, incomingProof,
|
||||
const mcp = incomingProof?.mcp;
|
||||
const mcpRetryQuery = mcp && deadlineProof?.blocked === false && deadlineProof.mcp === mcp.schedule
|
||||
? idleMcpWorkQuery(check.reason, { ingress: mcp.ingress, subscriptions: mcp.subscriptions, retries: mcp.retries }) : undefined;
|
||||
return { ...check, query: mcpRetryQuery ?? incomingWakeWorkQuery(check.reason, incomingProof,
|
||||
deadlineProof?.githubContinuity?.connectionIds) ?? deadlineOwnerWorkReplacement(deadlineProof, check.reason) ?? (replacement ? sql.raw(replacement) : check.query) };
|
||||
});
|
||||
checks.push(workCheck("agents.company_hold", sql`SELECT 1 FROM agents JOIN companies ON companies.id = agents.company_id
|
||||
|
||||
@@ -0,0 +1,27 @@
|
||||
import { describe, expect, it, vi } from "vitest";
|
||||
import type { Db } from "@paperclipai/db";
|
||||
import { instanceSettingsService } from "./instance-settings.js";
|
||||
function fixture(rows: Array<{ experimental: unknown }> = []) {
|
||||
const mutation = vi.fn(() => { throw new Error("read-only settings inspection mutated state"); });
|
||||
const select = vi.fn(() => ({ from: () => ({ where: async () => rows }) }));
|
||||
return { db: { select, insert: mutation, update: mutation, delete: mutation, execute: mutation } as unknown as Db, mutation, select };
|
||||
}
|
||||
describe("read-only experimental settings view", () => {
|
||||
it("returns disabled public-MCP defaults without creating a missing singleton", async () => {
|
||||
const f = fixture();
|
||||
const value = await instanceSettingsService(f.db,{ runtimeEnv: {} }).getExperimentalReadOnly();
|
||||
expect(value.enablePublicMcp).toBe(false);
|
||||
expect(f.select).toHaveBeenCalledTimes(1);
|
||||
expect(f.mutation).not.toHaveBeenCalled();
|
||||
});
|
||||
it("preserves the existing managed overlay on missing and stored rows", async () => {
|
||||
const runtimeEnv = { PAPERCLIP_MANAGED_CONFIG: JSON.stringify({ v: 1, mode: "cloud", catalogVersion: "test",
|
||||
features: { enablePublicMcp: true, enableOpenAiDot: true }, plugins: { autoInstall: [] } }) };
|
||||
for (const rows of [[], [{ experimental: { enablePublicMcp: false } }]]) {
|
||||
const f = fixture(rows);
|
||||
const value = await instanceSettingsService(f.db,{ runtimeEnv }).getExperimentalReadOnly();
|
||||
expect(value.enablePublicMcp).toBe(true); expect(value.enableOpenAiDot).toBe(true);
|
||||
expect(f.mutation).not.toHaveBeenCalled();
|
||||
}
|
||||
});
|
||||
});
|
||||
@@ -1,3 +1,4 @@
|
||||
import { prepareMcpClientMirrorPolicy } from "./public-mcp/client-mirror-mutations.js";
|
||||
import { notifyLiveRunRecovery, notifyLiveResourceRecovery } from "./live-run-recovery-notifications.js";
|
||||
import { notifyDeferredExecutionEligibility } from "./deferred-execution-notifications.js";
|
||||
import { notifyRunDispatch } from "./run-dispatch-notifications.js";
|
||||
@@ -650,6 +651,10 @@ export function instanceSettingsService(db: Db, options: InstanceSettingsService
|
||||
const [updated] = await db.transaction(async tx => {
|
||||
if (patch.enableStatusCards !== undefined) await notifyStatusCardEvaluation(tx);
|
||||
if (chatBecameEnabled) await notifyDeferredExecutionEligibility(tx);
|
||||
if(toExperimentalView(current.experimental).enablePublicMcp!==toExperimentalView(nextExperimental).enablePublicMcp)await prepareMcpClientMirrorPolicy(tx as unknown as Db,toExperimentalView(nextExperimental).enablePublicMcp);
|
||||
await notifyDeliveryWork(tx, DELIVERY_QUEUES.mcpIngressHandoffs);
|
||||
await notifyDeliveryWork(tx, DELIVERY_QUEUES.mcpClientMirror);
|
||||
await notifyDeliveryWork(tx, DELIVERY_QUEUES.mcpRevocationIndex, "*");
|
||||
await notifyDeliveryWork(tx, DELIVERY_QUEUES.mcpEvents, "*");
|
||||
await notifyDeliveryWork(tx, DELIVERY_QUEUES.dotEvents, "*");
|
||||
await notifyDeliveryWork(tx, DELIVERY_QUEUES.grokDispatch);
|
||||
|
||||
@@ -1,3 +1,4 @@
|
||||
import { assertMcpClientMirrorEnvelope } from "./public-mcp/client-mirror-wire.js";
|
||||
import { providerRetryAt } from "./provider-retry-after.js";
|
||||
import { SLACK_MANAGER_SCOPES, ASANA_CONNECTOR_SCOPES, isAsanaConnectorProfileId, type AsanaConnectorProfileId } from "@paperclipai/shared";
|
||||
import {
|
||||
@@ -35,7 +36,7 @@ export const GMAIL_CONNECTOR_SCOPES = [
|
||||
export { GOOGLE_WORKSPACE_CONNECTOR_PROFILES };
|
||||
|
||||
export type PaperclipCloudConnectorEnvironment = "development" | "test" | "staging" | "production";
|
||||
export type PaperclipCloudConnectorOperation = "status" | "session" | "claim" | "refresh" | "revoke" | "webhook-bind" | "event-lease" | "event-ack" | "github-app" | "slack-events";
|
||||
export type PaperclipCloudConnectorOperation = "status" | "session" | "claim" | "refresh" | "revoke" | "webhook-bind" | "event-lease" | "event-ack" | "github-app" | "slack-events" | "mcp-client-mirror" | "mcp-revocation-index";
|
||||
export type PaperclipCloudConnectorProfileId = GoogleWorkspaceConnectorProfileId | GitHubConnectorProfileId | AsanaConnectorProfileId | "slack.manager" | "github.bot";
|
||||
export type PaperclipCloudConnectorProvider = "google" | "github" | "asana" | "slack";
|
||||
|
||||
@@ -124,6 +125,8 @@ type ConnectorResponse = {
|
||||
};
|
||||
|
||||
const ENDPOINTS: Record<PaperclipCloudConnectorOperation, string> = {
|
||||
"mcp-client-mirror": "/v1/connector/mcp-client-mirror",
|
||||
"mcp-revocation-index": "/v1/connector/mcp-revocation-index",
|
||||
"slack-events": "/v1/connector/slack-events",
|
||||
"github-app": "/v1/connector/github-apps",
|
||||
status: "/v1/connector/instance-status",
|
||||
@@ -324,12 +327,16 @@ export function createPaperclipCloudConnector(input: {
|
||||
request: signRequest(payload, signingKey),
|
||||
...(secret ? { [secret.field]: secret.value } : {}),
|
||||
};
|
||||
const encodedBody = JSON.stringify(body);
|
||||
if (operation === "mcp-client-mirror" && secret?.field === "binding") {
|
||||
assertMcpClientMirrorEnvelope(body.request, secret.value, encodedBody);
|
||||
}
|
||||
let response: Response;
|
||||
try {
|
||||
response = await request(endpoint, {
|
||||
method: "POST",
|
||||
headers: { "content-type": "application/json", accept: "application/json" },
|
||||
body: JSON.stringify(body),
|
||||
body: encodedBody,
|
||||
signal: AbortSignal.timeout(15_000),
|
||||
});
|
||||
} catch {
|
||||
@@ -391,6 +398,14 @@ export function createPaperclipCloudConnector(input: {
|
||||
}
|
||||
|
||||
return {
|
||||
async syncMcpClientMirror(serialized: string): Promise<unknown> {
|
||||
const response = await call("mcp-client-mirror", {subject:"instance-client-metadata",companyId:"instance-client-metadata"}, {field:"binding",value:serialized});
|
||||
return (response as ConnectorResponse & {mcpClientMirror?:unknown}).mcpClientMirror;
|
||||
},
|
||||
async syncMcpRevocationIndex(values: { subject: string; companyId: string; serialized: string }): Promise<unknown> {
|
||||
const response = await call("mcp-revocation-index", values, { field: "binding", value: values.serialized });
|
||||
return (response as ConnectorResponse & { mcpRevocationIndex?: unknown }).mcpRevocationIndex;
|
||||
},
|
||||
eventWakeVersion: () => eventWakeVersion,
|
||||
async slackEventsAvailable() {
|
||||
const response = await call("status", { subject: "instance-capabilities", companyId: "instance-capabilities" });
|
||||
|
||||
@@ -0,0 +1,195 @@
|
||||
import assert from "node:assert/strict";
|
||||
import {randomBytes,randomUUID} from "node:crypto";
|
||||
import {once} from "node:events";
|
||||
import {mkdtemp,rm} from "node:fs/promises";
|
||||
import {createServer,request,type Server} from "node:http";
|
||||
import type {AddressInfo} from "node:net";
|
||||
import {tmpdir} from "node:os";
|
||||
import path from "node:path";
|
||||
import express from "express";
|
||||
import {eq,sql} from "drizzle-orm";
|
||||
import {applyPendingMigrations,createDb,closeRegisteredClients,closeLivenessDb,closeCheckpointDb,closeFileStorageDb,companies,authUsers,companyMemberships,issues,mcpOauthGrants,mcpOauthClients,mcpOauthDeviceRequests,mcpOauthClientChanges,mcpOauthClientMirrorState,mcpEventDeliveries,mcpEventSubscriptions,mcpMutationReceipts,activityLog} from "@paperclipai/db";
|
||||
import {resolveExternalTestDatabaseUrl} from "../../lib/dedicated-test-database-url.js";
|
||||
import {publicMcpIngressRoutes,publicMcpManagementRoutes} from "../../routes/public-mcp.js";
|
||||
import {issueRoutes} from "../../routes/issues.js";
|
||||
import {activityRoutes} from "../../routes/activity.js";
|
||||
import {errorHandler} from "../../middleware/error-handler.js";
|
||||
import {createPublicMcpOAuth} from "./oauth.js";
|
||||
import {createMcpApiDispatch,createPublicMcpExecutor} from "./capabilities.js";
|
||||
import {createPublicMcpEvents} from "./events.js";
|
||||
import {startMcpClientMirrorSync} from "./client-mirror-sync.js";
|
||||
import {readIdleMcpClientMirror} from "./idle-client-mirror.js";
|
||||
import type {McpClientMirrorRequest} from "./client-mirror-contract.js";
|
||||
import {startMcpRevocationIndexSync} from "./revocation-index-sync.js";
|
||||
import {startMcpIngressHandoffExpiry} from "./ingress-handoff-expiry.js";
|
||||
import {readIdleMcpSleepProof,registerIdleMcpOwners} from "./idle-sleep-proof.js";
|
||||
import {createDeliveryWorkCoordinator} from "../delivery-work-coordinator.js";
|
||||
import {createPaperclipCloudConnector,type PaperclipCloudConnectorConfig} from "../paperclip-cloud-connector.js";
|
||||
import {initializeCloudRuntimeIdentity,applyCloudRuntimeIdentityAssertion} from "../cloud-runtime-identity.js";
|
||||
import {initializeCloudRuntimeConfig} from "../cloud-runtime-config.js";
|
||||
import {instanceSettingsService} from "../instance-settings.js";
|
||||
import {idleWorkSnapshot} from "../task-admission.js";
|
||||
import {createStorageService} from "../../storage/service.js";
|
||||
import {createLocalDiskStorageProvider} from "../../storage/local-disk-provider.js";
|
||||
|
||||
const databaseUrl=resolveExternalTestDatabaseUrl("PAPERCLIP_MCP_CANONICAL_TEST_DATABASE_URL");
|
||||
if(!databaseUrl||!process.send)throw new Error("Canonical MCP fixture requires IPC and an explicit dedicated *_test database");
|
||||
type Input={origin:string;stackId:string;companyId:string;otherCompanyId:string;userId:string;taskId:string;otherTaskId:string;browserSecret:string;cloudOrigin:string;cloudLoopback:string;callbackUrl:string;callbackLoopback:string;runtimeJwks:string;runtimeAssertion:string;connector:PaperclipCloudConnectorConfig};
|
||||
let db:ReturnType<typeof createDb>|undefined,server:Server|undefined,home:string|undefined,draining=false,queryCount=0,callbackPosts=0,unregister:(()=>void)|undefined;
|
||||
let delivery:ReturnType<typeof createDeliveryWorkCoordinator>|undefined,events:ReturnType<typeof createPublicMcpEvents>|undefined,configuration:Input|undefined;
|
||||
const errors:string[]=[];
|
||||
let registerMirrorClient:((name:string)=>Promise<unknown>)|undefined;
|
||||
let mirrorExperiment:{mode:"mutation"|"loss-stable"|"loss-advanced"|"loss-repeat"|"loss-partial";completedRevision:number|null;partialRevision:number|null;recoveryBeginDropped:boolean;retryBegin:{revision:number;baseRevision:number|null}|null;nextBegin:{revision:number;baseRevision:number|null}|null;release:(()=>void)|null}|undefined;
|
||||
async function mirrorExperimentSnapshot(){
|
||||
assert.ok(db&&configuration);
|
||||
const [row]=await db.select().from(mcpOauthClientMirrorState).where(eq(mcpOauthClientMirrorState.id,"public"));
|
||||
const [journal]=await db.select({count:sql<number>`count(*)::int`,maximum:sql<number|null>`max(revision)`}).from(mcpOauthClientChanges);
|
||||
const proof=await readIdleMcpClientMirror(db,{budget:{beforeRead:async()=>{},check:()=>{}},enabled:true,origin:configuration.origin,instanceId:configuration.connector.instanceId,managedStackId:configuration.stackId});
|
||||
return{sourceRevision:row?.revision,receiptRevision:row?.receipt?.revision??null,cutoverPresent:Boolean(row?.deviceCutoverAt),cutoverAtMs:row?.deviceCutoverAt?.getTime()??null,journalCount:journal.count,journalMaximum:journal.maximum,blocked:proof.blocked,completedRevision:mirrorExperiment?.completedRevision??null,partialRevision:mirrorExperiment?.partialRevision??null,retryBegin:mirrorExperiment?.retryBegin??null,nextBegin:mirrorExperiment?.nextBegin??null,errors:[...errors]};
|
||||
}
|
||||
|
||||
async function mirrorExperimentRecords(){
|
||||
assert.ok(db&&mirrorExperiment);
|
||||
const clients=await db.select({id:mcpOauthClients.id,name:mcpOauthClients.name,redirectUris:mcpOauthClients.redirectUris,grantTypes:mcpOauthClients.grantTypes}).from(mcpOauthClients).orderBy(sql`${mcpOauthClients.id} COLLATE "C"`).limit(3);
|
||||
const devices=await db.select({status:mcpOauthDeviceRequests.status}).from(mcpOauthDeviceRequests).limit(1);
|
||||
assert.ok(clients.length<=2,"loss fixture uses only its two normal registered clients");
|
||||
assert.equal(devices.length,0,"this client mutation fixture preserves the actual empty device set");
|
||||
return{clients,deviceCount:devices.length};
|
||||
}
|
||||
async function mirrorExperimentQuiet(){
|
||||
assert.ok(events&&delivery&&mirrorExperiment&&mirrorExperiment.mode!=="mutation");
|
||||
const expected=mirrorExperiment.mode==="loss-repeat"?["controlled mirror complete response loss","controlled full mirror begin response loss"]:mirrorExperiment.mode==="loss-partial"?["controlled mirror append response loss"]:["controlled mirror complete response loss"];
|
||||
await settled();assert.deepEqual(errors,expected,"retain the exact expected controlled response-loss errors");
|
||||
assert.equal(delivery.nextWakeAt()===null||delivery.nextWakeAt()!>Date.now()+15_000,true);
|
||||
const before={queryCount,callbackPosts,revision:events.reconciliationSnapshot().revision,errorCount:errors.length},started=Date.now();
|
||||
await new Promise(resolve=>setTimeout(resolve,15_000));
|
||||
assert.equal(queryCount,before.queryCount,"tenant primary SQL remains quiet after response-loss recovery");
|
||||
assert.equal(callbackPosts,before.callbackPosts);assert.equal(events.reconciliationSnapshot().revision,before.revision);
|
||||
assert.deepEqual(errors,expected,"no unexpected or repeated owner error after response-loss recovery");
|
||||
return{started,ended:Date.now(),before,after:{queryCount,callbackPosts,revision:events.reconciliationSnapshot().revision,errorCount:errors.length}};
|
||||
}
|
||||
|
||||
async function mappedFetch(input:Parameters<typeof fetch>[0],init?:RequestInit){
|
||||
assert.ok(configuration);const original=new Request(input,init),url=new URL(original.url),headers=new Headers(original.headers);
|
||||
let destination:string;
|
||||
if(url.origin===configuration.cloudOrigin||url.origin===configuration.origin){headers.set("host",url.host);destination=configuration.cloudLoopback+url.pathname+url.search;}
|
||||
else if(url.href===configuration.callbackUrl){callbackPosts++;headers.set("host",url.host);destination=configuration.callbackLoopback;}
|
||||
else throw new Error("Canonical fixture attempted an unmapped transport");
|
||||
const body=original.body?Buffer.from(await original.arrayBuffer()):undefined;
|
||||
original.signal.throwIfAborted();
|
||||
return new Promise<Response>((resolve,reject)=>{
|
||||
const req=request(destination,{method:original.method,headers:Object.fromEntries(headers),signal:original.signal},res=>{
|
||||
const chunks:Buffer[]=[];let length=0;
|
||||
res.on("data",(part:Buffer)=>{length+=part.length;if(length>1_048_576){res.destroy(new Error("Canonical mapped response exceeds transport bound"));return;}chunks.push(part);});
|
||||
res.on("error",reject);res.on("end",()=>{const responseHeaders=new Headers();for(const [name,value] of Object.entries(res.headers))if(value!==undefined)for(const item of Array.isArray(value)?value:[value])responseHeaders.append(name,item);
|
||||
const status=res.statusCode!;resolve(new Response([204,205,304].includes(status)?null:Buffer.concat(chunks),{status,headers:responseHeaders}));});
|
||||
});
|
||||
req.on("error",reject);req.end(body);
|
||||
});
|
||||
}
|
||||
async function initialize(input:Input){
|
||||
assert.equal(process.versions.node.split(".")[0],"24","Canonical Tenant fixture requires actual Node24 before initialization or claims");
|
||||
assert.equal(db,undefined);configuration=input;home=await mkdtemp(path.join(tmpdir(),"canonical-mcp-tenant-"));
|
||||
Object.assign(process.env,{PAPERCLIP_HOME:home,PAPERCLIP_PUBLIC_URL:input.origin,PAPERCLIP_CLOUD_STACK_ID:input.stackId,PAPERCLIP_CLOUD_RUNTIME_IDENTITY_JWKS:input.runtimeJwks,PAPERCLIP_CLOUD_TENANT_SERVER_TOKEN:randomBytes(24).toString("base64url"),PAPERCLIP_CLOUD_CONNECTOR_INSTANCE_ID:input.connector.instanceId,PAPERCLIP_CLOUD_CONNECTOR_ENVIRONMENT:input.connector.environment,PAPERCLIP_CLOUD_CONNECTOR_SIGN_PRIVATE_KEY:input.connector.signPrivateKey,PAPERCLIP_CLOUD_CONNECTOR_SEAL_PRIVATE_KEY:input.connector.sealPrivateKey,PAPERCLIP_CLOUD_CONNECTOR_BASE_URL:input.connector.baseUrl,PAPERCLIP_SECRETS_MASTER_KEY:randomBytes(32).toString("base64")});
|
||||
await applyPendingMigrations(databaseUrl!);db=createDb(databaseUrl!,{maxConnections:2,applicationName:"sleep2-mcp-canonical-tenant"});
|
||||
(db.$client.options as unknown as {debug:(...args:unknown[])=>void}).debug=()=>{queryCount++;};
|
||||
await db.insert(companies).values([{id:input.companyId,name:"Selected company",issuePrefix:"SELECTED"},{id:input.otherCompanyId,name:"Primary company",issuePrefix:"PRIMARY"}]);
|
||||
await db.insert(authUsers).values({id:input.userId,name:"Canonical operator",email:`${input.userId}@example.test`,createdAt:new Date(),updatedAt:new Date()});
|
||||
await db.insert(companyMemberships).values([input.companyId,input.otherCompanyId].map(companyId=>({companyId,principalId:input.userId,principalType:"user" as const,membershipRole:"owner" as const,status:"active" as const})));
|
||||
await db.insert(issues).values([{id:input.taskId,companyId:input.companyId,title:"Canonical monitored task",status:"todo"},{id:input.otherTaskId,companyId:input.otherCompanyId,title:"Outside connection",status:"todo"}]);
|
||||
await instanceSettingsService(db).updateExperimental({enablePublicMcp:true,enableOpenAiDot:false,enableGrokBot:false});
|
||||
await initializeCloudRuntimeConfig(db);
|
||||
await initializeCloudRuntimeIdentity(db);await applyCloudRuntimeIdentityAssertion({db,compactJws:input.runtimeAssertion});
|
||||
const oauth=createPublicMcpOAuth(db,{origin:input.origin,resource:input.origin+"/mcp/paperclip"}),api=express.Router();
|
||||
api.use(issueRoutes(db,createStorageService(createLocalDiskStorageProvider(path.join(home,"storage")))));
|
||||
api.use(activityRoutes(db));
|
||||
const dispatch=createMcpApiDispatch(api),app=express();app.use(express.json({limit:"1mb"}));
|
||||
app.use((req,_res,next)=>{req.actor=req.headers["x-canonical-browser"]===input.browserSecret?{type:"board",source:"session",userId:input.userId,companyIds:[input.companyId,input.otherCompanyId]}:{type:"none",source:"none"};next();});
|
||||
events=createPublicMcpEvents(db,oauth,dispatch,{fetch:mappedFetch,cloudOrigin:input.cloudOrigin,isBackgroundWorkEnabled:()=>!draining});
|
||||
app.use(publicMcpIngressRoutes(oauth,createPublicMcpExecutor(db,oauth,dispatch),events));app.use("/api",publicMcpManagementRoutes(oauth));app.use(errorHandler);
|
||||
server=createServer(app);server.listen(0,"127.0.0.1");await once(server,"listening");
|
||||
delivery=createDeliveryWorkCoordinator({owner:db,canRun:()=>!draining,canReconcile:()=>true,onError:error=>errors.push(error instanceof Error?error.message:String(error))});
|
||||
await events.start(delivery).ready;
|
||||
const connector=createPaperclipCloudConnector({config:input.connector,request:mappedFetch});
|
||||
registerMirrorClient=name=>oauth.register({client_name:name,redirect_uris:["https://mirror-client.example.test/callback"],grant_types:["authorization_code","refresh_token"],response_types:["code"],token_endpoint_auth_method:"none"},"mirror-experiment");
|
||||
const publishMirror=async(value:McpClientMirrorRequest)=>{
|
||||
const experiment=mirrorExperiment;
|
||||
if(experiment&&value.operation==="begin"&&(experiment.completedRevision!==null||experiment.partialRevision!==null)&&experiment.nextBegin===null){
|
||||
experiment.nextBegin={revision:value.revision,baseRevision:value.baseRevision};
|
||||
await new Promise<void>(resolve=>{experiment.release=resolve;});
|
||||
}
|
||||
if(experiment?.mode==="loss-repeat"&&experiment.recoveryBeginDropped&&value.operation==="begin"&&experiment.retryBegin===null){
|
||||
experiment.retryBegin={revision:value.revision,baseRevision:value.baseRevision};
|
||||
await new Promise<void>(resolve=>{experiment.release=resolve;});
|
||||
}
|
||||
const response=await connector.syncMcpClientMirror(JSON.stringify(value));
|
||||
if(experiment?.mode==="loss-partial"&&value.operation==="append"&&experiment.partialRevision===null){
|
||||
experiment.partialRevision=value.revision;throw new Error("controlled mirror append response loss");
|
||||
}
|
||||
if(experiment?.mode==="loss-repeat"&&value.operation==="begin"&&experiment.completedRevision!==null&&!experiment.recoveryBeginDropped){
|
||||
experiment.recoveryBeginDropped=true;throw new Error("controlled full mirror begin response loss");
|
||||
}
|
||||
if(experiment&&experiment.mode!=="loss-partial"&&value.operation==="complete"&&experiment.completedRevision===null){
|
||||
experiment.completedRevision=value.revision;
|
||||
if(experiment.mode==="mutation"||experiment.mode==="loss-advanced")await registerMirrorClient!("Client committed after remote mirror complete");
|
||||
if(experiment.mode!=="mutation")throw new Error("controlled mirror complete response loss");
|
||||
}
|
||||
return response;
|
||||
};
|
||||
await startMcpClientMirrorSync(db,delivery,input.origin,{instanceId:input.connector.instanceId,publish:publishMirror}).worker.ready;
|
||||
await startMcpRevocationIndexSync(db,delivery,{client:{publish:value=>connector.syncMcpRevocationIndex({subject:value.operatorId,companyId:value.companyId,serialized:JSON.stringify(value)})}}).ready;
|
||||
await startMcpIngressHandoffExpiry(db,delivery).ready;
|
||||
unregister=registerIdleMcpOwners(db,{config:oauth.config,personal:events,dot:null,clientMirrorInstanceId:input.connector.instanceId});
|
||||
await settled();
|
||||
return{origin:`http://127.0.0.1:${(server.address() as AddressInfo).port}`,pid:process.pid,node:process.version,poolMaximums:{primary:2,dedicated:1,liveness:2,checkpoint:3,fileStorage:2},queryCount};
|
||||
}
|
||||
async function settled(){
|
||||
assert.ok(db&&events);const end=Date.now()+15_000;
|
||||
while(Date.now()<end){
|
||||
const [mirror]=await db.select().from(mcpOauthClientMirrorState),grants=await db.select().from(mcpOauthGrants);
|
||||
if(events.reconciliationSnapshot().reconciled&&idleWorkSnapshot().active===0&&mirror?.receipt?.revision===mirror?.revision&&grants.every(g=>g.revokedAt!==null||g.revocationIndexReceipt?.revision===g.revocationRevision))return{revision:mirror.revision,errors:[...errors]};
|
||||
await new Promise(resolve=>setTimeout(resolve,25));
|
||||
}
|
||||
throw new Error("Actual MCP owners did not settle within the fixture observation bound: "+errors.join("; "));
|
||||
}
|
||||
async function snapshot(){
|
||||
assert.ok(db);return{issues:await db.select({id:issues.id,status:issues.status}).from(issues),deliveries:await db.select().from(mcpEventDeliveries),subscriptions:await db.select({id:mcpEventSubscriptions.id,stoppedAt:mcpEventSubscriptions.stoppedAt}).from(mcpEventSubscriptions),receipts:await db.select().from(mcpMutationReceipts),activity:await db.select({action:activityLog.action,entityId:activityLog.entityId}).from(activityLog),queryCount,callbackPosts,errors:[...errors]};
|
||||
}
|
||||
async function proof(){
|
||||
assert.ok(db&&configuration);await settled();draining=true;
|
||||
return db.transaction(async tx=>{
|
||||
const started=Date.now();let reads=0;const context={stackId:configuration!.stackId,ownerId:randomUUID(),capabilityGeneration:"a".repeat(64),challenge:"c".repeat(64),now:Date.now(),budget:{beforeRead:async()=>{if(++reads>64||Date.now()-started>=2000)throw new Error("Canonical idle inspection exceeded shared budget");await tx.execute(sql`SET LOCAL statement_timeout = '2s'`);},check:()=>{if(Date.now()-started>=2000)throw new Error("Canonical idle inspection timed out");}}};
|
||||
const value=await readIdleMcpSleepProof(tx,db!,context);return{...value,readCount:reads};
|
||||
},{isolationLevel:"repeatable read",accessMode:"read only"});
|
||||
}
|
||||
async function quiet(milliseconds:number){
|
||||
assert.ok(events&&delivery);assert.ok(milliseconds===15_000||milliseconds===60_000);await settled();assert.equal(delivery.nextWakeAt()===null||delivery.nextWakeAt()!>Date.now()+milliseconds,true);
|
||||
const before={queryCount,callbackPosts,revision:events.reconciliationSnapshot().revision},started=Date.now();await new Promise(resolve=>setTimeout(resolve,milliseconds));
|
||||
assert.equal(queryCount,before.queryCount,"tenant primary SQL remains quiet between useful operations");assert.equal(callbackPosts,before.callbackPosts);assert.equal(events.reconciliationSnapshot().revision,before.revision);assert.equal(errors.length,0,errors.join(";"));
|
||||
return{started,ended:Date.now(),before,after:{queryCount,callbackPosts},scope:"MCP owners and primary driver; auxiliary sockets counted separately"};
|
||||
}
|
||||
async function close(){
|
||||
mirrorExperiment?.release?.();
|
||||
unregister?.();await events?.stop();await delivery?.stop();if(server){server.closeAllConnections();await new Promise<void>(resolve=>server!.close(()=>resolve()));}
|
||||
if(db){await closeCheckpointDb(db,1);await closeLivenessDb(db,1);await closeFileStorageDb(db,1);await db.$client.end({timeout:1});await closeRegisteredClients(databaseUrl!,{scope:"database"});}
|
||||
if(home)await rm(home,{recursive:true,force:true});return{closed:true,pid:process.pid};
|
||||
}
|
||||
let chain=Promise.resolve();
|
||||
process.on("message",(message:{id:number;operation:string;input?:unknown})=>{chain=chain.then(async()=>{
|
||||
try{
|
||||
let result:unknown;
|
||||
switch(message.operation){case"initialize":result=await initialize(message.input as Input);break;case"mirror-experiment-arm":{
|
||||
assert.ok(registerMirrorClient);assert.equal(mirrorExperiment,undefined);
|
||||
assert.ok(message.input==="mutation"||message.input==="loss-stable"||message.input==="loss-advanced"||message.input==="loss-repeat"||message.input==="loss-partial");
|
||||
mirrorExperiment={mode:message.input,completedRevision:null,partialRevision:null,recoveryBeginDropped:false,retryBegin:null,nextBegin:null,release:null};
|
||||
await registerMirrorClient("Client triggering actual mirror publication");result={armed:true};break;
|
||||
}case"mirror-experiment-snapshot":result=await mirrorExperimentSnapshot();break;
|
||||
case"mirror-experiment-records":result=await mirrorExperimentRecords();break;
|
||||
case"mirror-experiment-quiet":result=await mirrorExperimentQuiet();break;
|
||||
case"mirror-experiment-release":mirrorExperiment?.release?.();result={released:true};break;
|
||||
case"settled":result=await settled();break;case"snapshot":result=await snapshot();break;case"proof":result=await proof();break;case"quiet":result=await quiet(Number(message.input));break;case"resume":draining=false;result={resumed:true};break;case"close":result=await close();break;default:throw new Error("Unknown canonical fixture operation");}
|
||||
process.send?.({id:message.id,result});if(message.operation==="close")process.disconnect();
|
||||
}catch(error){process.send?.({id:message.id,error:error instanceof Error?error.stack:String(error)});}
|
||||
});});
|
||||
|
||||
process.send?.({fixtureReady:true,node:process.version,pid:process.pid});
|
||||
@@ -0,0 +1,16 @@
|
||||
import {describe,expect,it} from "vitest";
|
||||
import {sameMcpClientMirrorReceipt} from "./client-mirror-sync.js";
|
||||
import type {McpClientMirrorReceipt} from "./client-mirror-contract.js";
|
||||
|
||||
const make=():McpClientMirrorReceipt=>({version:1,instanceId:"mirror-control",origin:"https://mirror-control.example.test",revision:4,enabled:true,frontDoor:"tenant-personal-v1",deviceStatus:{keyId:"a".repeat(64),generation:"b".repeat(64),count:0},generation:"c".repeat(64)});
|
||||
describe("same-revision MCP mirror receipt identity",()=>{
|
||||
it("accepts property reordering and equivalent default issuer without mutating either receipt",()=>{
|
||||
const prior=make(),saved=structuredClone(prior),same={...prior,deviceStatus:{count:0,generation:prior.deviceStatus!.generation,keyId:prior.deviceStatus!.keyId},authorizationOrigin:prior.origin};
|
||||
expect(sameMcpClientMirrorReceipt(prior,same)).toBe(true);expect(prior).toEqual(saved);
|
||||
});
|
||||
it("rejects every conflicting immutable field even with the same opaque generation value",()=>{
|
||||
const prior=make(),saved=structuredClone(prior);
|
||||
const alternatives:McpClientMirrorReceipt[]=[{...prior,instanceId:"replacement"},{...prior,origin:"https://foreign.example.test"},{...prior,revision:5},{...prior,enabled:false},{...prior,frontDoor:undefined},{...prior,authorizationOrigin:"https://foreign.example.test"},{...prior,deviceStatus:undefined},{...prior,deviceStatus:{...prior.deviceStatus!,keyId:"d".repeat(64)}},{...prior,deviceStatus:{...prior.deviceStatus!,generation:"e".repeat(64)}},{...prior,deviceStatus:{...prior.deviceStatus!,count:1}},{...prior,generation:"f".repeat(64)}];
|
||||
for(const changed of alternatives){expect(sameMcpClientMirrorReceipt(prior,changed)).toBe(false);expect(prior).toEqual(saved);}
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,41 @@
|
||||
import { mcpClientMirrorPageFits } from "./client-mirror-wire.js";
|
||||
import { mcpDeviceStatusReferenceSchema,mcpDeviceStatusEntrySchema } from "./device-status-contract.js";
|
||||
import { createHash } from "node:crypto";
|
||||
import { z } from "zod";
|
||||
import { validMcpRedirect } from "./client-metadata.js";
|
||||
|
||||
const revision = z.number().int().min(0).max(Number.MAX_SAFE_INTEGER);
|
||||
const origin = z.string().url().max(2048).refine(value => { const url = new URL(value); return url.protocol === "https:" && url.origin === value; });
|
||||
const identifier = z.string().min(1).max(2048);
|
||||
export const mcpMirroredClientSchema = z.object({ id: identifier, name: z.string().min(1).max(100),
|
||||
redirectUris: z.array(z.string().max(2048).refine(validMcpRedirect)).max(10),
|
||||
grantTypes: z.array(z.enum(["authorization_code","refresh_token","urn:ietf:params:oauth:grant-type:device_code"])).transform(values=>[...new Set(values)]),
|
||||
}).strict();
|
||||
const change = z.discriminatedUnion("operation", [z.object({ operation: z.literal("upsert"), client: mcpMirroredClientSchema }).strict(),
|
||||
z.object({ operation: z.literal("delete"), clientId: identifier }).strict()]);
|
||||
const identity = z.object({ version: z.literal(1), instanceId: z.string().min(1).max(200), origin, revision,authorizationOrigin:origin.optional(),
|
||||
deviceStatus:mcpDeviceStatusReferenceSchema.optional(),baseRevision: revision.nullable(), expectedRecords: revision });
|
||||
export const mcpClientMirrorRequestSchema = z.discriminatedUnion("operation", [
|
||||
identity.extend({ operation: z.literal("begin"),deviceKey:z.string().regex(/^[A-Za-z0-9_-]{43}$/).optional() }).strict(),
|
||||
identity.extend({ operation: z.literal("disable") }).strict(),
|
||||
identity.extend({ operation: z.literal("append"), sequence: revision, changes: z.array(change).min(1).max(200) }).strict(),
|
||||
identity.extend({ operation: z.literal("append-device-status"),sequence:revision,entries:z.array(mcpDeviceStatusEntrySchema).min(1).max(200) }).strict(),
|
||||
identity.extend({ operation: z.literal("complete") }).strict(),
|
||||
]).refine(mcpClientMirrorPageFits,"MCP client mirror page is too large");
|
||||
export const mcpClientMirrorReceiptSchema = z.object({ version: z.literal(1), instanceId: z.string().min(1).max(200), origin, revision,authorizationOrigin:origin.optional(),
|
||||
deviceStatus:mcpDeviceStatusReferenceSchema.optional(),frontDoor:z.literal("tenant-personal-v1").optional(),enabled:z.boolean(),generation: z.string().regex(/^[a-f0-9]{64}$/) }).strict();
|
||||
export const mcpClientMirrorResponseSchema = z.discriminatedUnion("complete", [
|
||||
z.object({ complete: z.literal(true), receipt: mcpClientMirrorReceiptSchema }).strict(),
|
||||
z.object({ complete: z.literal(false), nextSequence: revision, receivedRecords: revision,nextDeviceSequence:revision.optional() }).strict(),
|
||||
]);
|
||||
export type McpMirroredClient = z.infer<typeof mcpMirroredClientSchema>;
|
||||
export type McpClientMirrorChange = z.infer<typeof change>;
|
||||
export type McpClientMirrorRequest = z.infer<typeof mcpClientMirrorRequestSchema>;
|
||||
export type McpClientMirrorReceipt = z.infer<typeof mcpClientMirrorReceiptSchema>;
|
||||
export type McpClientMirrorResponse = z.infer<typeof mcpClientMirrorResponseSchema>;
|
||||
export function mcpClientMirrorGeneration(value: Pick<McpClientMirrorRequest,"instanceId"|"origin"|"revision">&{enabled?:boolean;frontDoor?:"tenant-personal-v1";authorizationOrigin?:string;deviceStatus?:z.infer<typeof mcpDeviceStatusReferenceSchema>}) {
|
||||
return createHash("sha256").update(JSON.stringify(["mcp-client-mirror-v1",value.instanceId,value.origin,value.revision,value.enabled??true,value.frontDoor??null,value.deviceStatus??null,value.authorizationOrigin??value.origin])).digest("hex");
|
||||
}
|
||||
export function mcpClientMirrorPageHash(changes: McpClientMirrorChange[]) {
|
||||
return createHash("sha256").update(JSON.stringify(changes)).digest("hex");
|
||||
}
|
||||
@@ -0,0 +1,66 @@
|
||||
import { mcpClientMirrorClientPages } from "./client-mirror-pages.js";
|
||||
import assert from "node:assert/strict";
|
||||
import { createHash, generateKeyPairSync, verify } from "node:crypto";
|
||||
import { test } from "vitest";
|
||||
import { createPaperclipCloudConnector } from "../paperclip-cloud-connector.js";
|
||||
import { mcpClientMirrorRequestSchema } from "./client-mirror-contract.js";
|
||||
import { MCP_MIRROR_MAX_BINDING_BYTES, MCP_MIRROR_MAX_ENVELOPE_BYTES } from "./client-mirror-wire.js";
|
||||
|
||||
function owner(instanceId="unit-mirror-instance") {
|
||||
const signing=generateKeyPairSync("ed25519"),sealing=generateKeyPairSync("x25519");
|
||||
const captured:string[]=[];
|
||||
const connector=createPaperclipCloudConnector({config:{baseUrl:"https://my.paperclip.app",instanceId,environment:"production",
|
||||
signPrivateKey:signing.privateKey.export({format:"pem",type:"pkcs8"}).toString(),
|
||||
sealPrivateKey:sealing.privateKey.export({format:"pem",type:"pkcs8"}).toString()},
|
||||
request:async(_url,init)=>{captured.push(String(init?.body));return new Response(JSON.stringify({mcpClientMirror:{complete:false,nextSequence:0,receivedRecords:0}}),{status:200,headers:{"content-type":"application/json"}});}});
|
||||
return{connector,captured,signing};
|
||||
}
|
||||
|
||||
test("unit controlled transport captures the actual signed and escaped MCP singleton body",async()=>{
|
||||
const f=owner(),origin="https://tenant.example.test",redirect="https://example.test/"+"\u0000".repeat(2048-21);
|
||||
const value=mcpClientMirrorRequestSchema.parse({version:1,instanceId:"unit-mirror-instance",origin,revision:1,baseRevision:null,expectedRecords:1,
|
||||
operation:"append",sequence:0,changes:[{operation:"upsert",client:{id:"client",name:"client",redirectUris:Array(10).fill(redirect),grantTypes:["authorization_code"]}}]});
|
||||
const binding=JSON.stringify(value);
|
||||
await f.connector.syncMcpClientMirror(binding);
|
||||
assert.equal(f.captured.length,1);
|
||||
const encoded=f.captured[0]!;
|
||||
assert.ok(Buffer.byteLength(encoded)>64000);
|
||||
assert.ok(Buffer.byteLength(encoded)<=MCP_MIRROR_MAX_ENVELOPE_BYTES);
|
||||
const body=JSON.parse(encoded);
|
||||
assert.equal(body.binding,binding);
|
||||
const [header,payload,signature]=body.request.split(".");
|
||||
assert.equal(verify(null,Buffer.from(header+"."+payload),f.signing.publicKey,Buffer.from(signature,"base64url")),true);
|
||||
const claims=JSON.parse(Buffer.from(payload,"base64url").toString("utf8"));
|
||||
assert.equal(claims.op,"mcp-client-mirror");
|
||||
assert.equal(claims.iss,"unit-mirror-instance");
|
||||
assert.equal(claims.sub,"instance-client-metadata");
|
||||
assert.equal(claims.cid,"instance-client-metadata");
|
||||
assert.equal(claims.aud,"https://my.paperclip.app/v1/connector/mcp-client-mirror");
|
||||
assert.equal(claims.exp-claims.iat,60);
|
||||
assert.equal(claims.sh,createHash("sha256").update(binding).digest("base64url"));
|
||||
});
|
||||
|
||||
test("unit controlled transport refuses oversized MCP binding before invoking request",async()=>{
|
||||
const f=owner();
|
||||
await assert.rejects(f.connector.syncMcpClientMirror(JSON.stringify({padding:"x".repeat(MCP_MIRROR_MAX_BINDING_BYTES+1)})),/bounded wire contract/);
|
||||
assert.deepEqual(f.captured,[]);
|
||||
});
|
||||
|
||||
test("unit controlled transport refuses a JWS beyond the unchanged 12000 character boundary",async()=>{
|
||||
const f=owner("i".repeat(12000));
|
||||
await assert.rejects(f.connector.syncMcpClientMirror("{}"),/bounded wire contract/);
|
||||
assert.deepEqual(f.captured,[]);
|
||||
});
|
||||
|
||||
test("unit actual producer pages traverse the real signer once each with exact lossless bindings",async()=>{
|
||||
const f=owner(),origin="https://tenant.example.test";
|
||||
const identity={version:1 as const,instanceId:"unit-mirror-instance",origin,revision:1,baseRevision:null,expectedRecords:205};
|
||||
const changes=Array.from({length:205},(_,index)=>({operation:"delete" as const,clientId:"client-"+index}));
|
||||
const pages=mcpClientMirrorClientPages(identity,0,changes);
|
||||
for(const page of pages)await f.connector.syncMcpClientMirror(JSON.stringify(page));
|
||||
assert.equal(f.captured.length,pages.length);
|
||||
const sent=f.captured.map(encoded=>JSON.parse(JSON.parse(encoded).binding));
|
||||
assert.deepEqual(sent,pages);
|
||||
assert.deepEqual(sent.flatMap(page=>page.changes),changes);
|
||||
for(const body of f.captured)assert.ok(Buffer.byteLength(body)<=MCP_MIRROR_MAX_ENVELOPE_BYTES);
|
||||
});
|
||||
@@ -0,0 +1,201 @@
|
||||
import {createHash,randomBytes,randomUUID} from "node:crypto";
|
||||
import {appendFileSync} from "node:fs";
|
||||
import {mkdtemp,rm} from "node:fs/promises";
|
||||
import {tmpdir} from "node:os";
|
||||
import path from "node:path";
|
||||
import type {Request} from "express";
|
||||
import {afterAll,afterEach,beforeAll,beforeEach,describe,expect,it,vi} from "vitest";
|
||||
import {eq,sql} from "drizzle-orm";
|
||||
import {PgDatabase} from "drizzle-orm/pg-core";
|
||||
import {subscribeDatabaseWork} from "@paperclipai/db/work-signals";
|
||||
import {DELIVERY_QUEUES} from "../delivery-work-notifications.js";
|
||||
import {applyPendingMigrations,createDb,companies,authUsers,companyMemberships,mcpOauthClients as clients,mcpOauthClientMirrorState as state,mcpOauthClientChanges as changes,mcpOauthDeviceRequests as devices,mcpOauthGrants as grants} from "@paperclipai/db";
|
||||
import {resolveExternalTestDatabaseUrl} from "../../__tests__/helpers/external-test-database.js";
|
||||
import {createDeliveryWorkCoordinator} from "../delivery-work-coordinator.js";
|
||||
import {instanceSettingsService} from "../instance-settings.js";
|
||||
import {deleteCompany} from "../company-deletion.js";
|
||||
import {createPublicMcpOAuth,DEVICE_GRANT} from "./oauth.js";
|
||||
import {startMcpRevocationIndexSync} from "./revocation-index-sync.js";
|
||||
import {startMcpClientMirrorSync} from "./client-mirror-sync.js";
|
||||
import {mcpClientMirrorGeneration,type McpClientMirrorRequest,type McpClientMirrorReceipt} from "./client-mirror-contract.js";
|
||||
import {readIdleMcpClientMirror} from "./idle-client-mirror.js";
|
||||
import {readIdleMcpDeviceStatus} from "./idle-device-status.js";
|
||||
import {deriveLocalEncryptedInstanceSecret} from "../../secrets/local-encrypted-provider.js";
|
||||
import {mcpDeviceStatusFingerprint} from "./device-status-contract.js";
|
||||
import {readIdleMcpWakeProof} from "./idle-wake-proof.js";
|
||||
|
||||
const managed=vi.hoisted(()=>({identity:null as null|{stackId:string;canonicalOrigin:string}}));
|
||||
vi.mock("../cloud-runtime-identity.js",async importOriginal=>({...await importOriginal<typeof import("../cloud-runtime-identity.js")>(),getCloudRuntimeIdentity:()=>managed.identity}));
|
||||
const configured=resolveExternalTestDatabaseUrl("PAPERCLIP_MCP_SLEEP_TEST_DATABASE_URL"),origin="https://mirror.example.test",instanceId="mirror-instance",stackId="mirror-stack";
|
||||
const budget={beforeRead:async()=>{},check:()=>{}};
|
||||
describe.skipIf(!configured)("MCP mirror actual transaction lifecycle",()=>{
|
||||
let db:ReturnType<typeof createDb>,coordinator:ReturnType<typeof createDeliveryWorkCoordinator>|undefined;
|
||||
let home:string,companyId:string,userId:string,remoteAvailable:boolean,failures:number,ack:McpClientMirrorReceipt|undefined;
|
||||
const sent:McpClientMirrorRequest[]=[];
|
||||
const errors:unknown[]=[];
|
||||
let onAppend:(()=>Promise<void>)|undefined;
|
||||
beforeAll(async()=>{await applyPendingMigrations(configured!);db=createDb(configured!,{maxConnections:2});},90_000);
|
||||
beforeEach(async()=>{
|
||||
await db.execute(sql`TRUNCATE TABLE companies,mcp_oauth_clients,mcp_oauth_client_mirror_state,mcp_oauth_client_changes CASCADE`);
|
||||
companyId=randomUUID();userId=randomUUID();remoteAvailable=false;failures=0;ack=undefined;sent.length=0;errors.length=0;onAppend=undefined;
|
||||
home=await mkdtemp(path.join(tmpdir(),"mcp-mirror-lifecycle-"));vi.stubEnv("PAPERCLIP_HOME",home);
|
||||
managed.identity={stackId,canonicalOrigin:origin};vi.stubEnv("PAPERCLIP_SECRETS_MASTER_KEY",randomBytes(32).toString("base64"));
|
||||
await db.insert(companies).values({id:companyId,name:"Mirror lifecycle",issuePrefix:"MIRROR"});
|
||||
await db.insert(authUsers).values({id:userId,name:"Mirror operator",email:`${userId}@example.test`,createdAt:new Date(),updatedAt:new Date()});
|
||||
await db.insert(companyMemberships).values({companyId,principalId:userId,principalType:"user",membershipRole:"owner",status:"active"});
|
||||
await instanceSettingsService(db).updateExperimental({enablePublicMcp:true,enableOpenAiDot:false});
|
||||
});
|
||||
afterEach(async()=>{await coordinator?.stop();coordinator=undefined;managed.identity=null;await db.execute(sql`TRUNCATE TABLE companies,mcp_oauth_clients,mcp_oauth_client_mirror_state,mcp_oauth_client_changes CASCADE`);await db.delete(authUsers).where(eq(authUsers.id,userId));await rm(home,{recursive:true,force:true});vi.unstubAllEnvs();});
|
||||
afterAll(async()=>{await db?.$client.end({timeout:1});});
|
||||
async function publish(value:McpClientMirrorRequest){
|
||||
sent.push(structuredClone(value));if(value.operation==="append"&&onAppend){const append=onAppend;onAppend=undefined;await append();}if(failures>0){failures--;throw new Error("controlled mirror outage");}
|
||||
if(value.operation==="disable"||value.operation==="complete"){
|
||||
const enabled=value.operation!=="disable",frontDoor=enabled&&remoteAvailable&&value.deviceStatus&&(!value.authorizationOrigin||value.authorizationOrigin===value.origin)?"tenant-personal-v1" as const:undefined;
|
||||
ack={version:1,instanceId:value.instanceId,origin:value.origin,revision:value.revision,enabled,...(enabled&&value.deviceStatus?{deviceStatus:value.deviceStatus}:{}),...(enabled&&value.authorizationOrigin?{authorizationOrigin:value.authorizationOrigin}:{}),...(frontDoor?{frontDoor}:{}),generation:mcpClientMirrorGeneration({...value,enabled,frontDoor})};
|
||||
return{complete:true,receipt:ack};
|
||||
}
|
||||
return{complete:false,nextSequence:0,receivedRecords:0,nextDeviceSequence:0};
|
||||
}
|
||||
async function start(mirrorInstance=instanceId){
|
||||
await coordinator?.stop();coordinator=createDeliveryWorkCoordinator({owner:db,canRun:()=>true,onError:error=>errors.push(error)});
|
||||
const handle=startMcpClientMirrorSync(db,coordinator,origin,{instanceId:mirrorInstance,publish});await handle.worker.ready;return handle.worker;
|
||||
}
|
||||
const current=async()=>(await db.select().from(state).where(eq(state.id,"public")))[0];
|
||||
async function acknowledged(){await vi.waitFor(async()=>{const row=await current();expect(row).toBeDefined();expect(row?.receipt).toBeTruthy();expect(row?.receipt?.revision).toBe(row?.revision);},{timeout:10_000});return(await current())!;}
|
||||
const inspect=()=>readIdleMcpClientMirror(db,{budget,enabled:true,origin,instanceId,managedStackId:stackId});
|
||||
async function device(){
|
||||
const oauth=createPublicMcpOAuth(db,{origin,resource:origin+"/mcp/paperclip"});
|
||||
const client=await oauth.register({client_name:"Legacy device",redirect_uris:["https://client.example.test/callback"],grant_types:["authorization_code","refresh_token",DEVICE_GRANT],response_types:["code"],token_endpoint_auth_method:"none"});
|
||||
const authorization=await oauth.deviceAuthorize({client_id:client.client_id,resource:oauth.config.resource,scope:"paperclip:read offline_access"});
|
||||
return{oauth,client,authorization};
|
||||
}
|
||||
it("indexes a normally approved abandoned authorization code after an empty index startup",async()=>{
|
||||
const oauth=createPublicMcpOAuth(db,{origin,resource:origin+"/mcp/paperclip"});
|
||||
const client=await oauth.register({client_name:"Abandoned approved client",redirect_uris:["https://client.example.test/callback"],grant_types:["authorization_code"]});
|
||||
await start();
|
||||
const published:string[]=[];
|
||||
const index=startMcpRevocationIndexSync(db,coordinator!,{key:Buffer.alloc(32,9).toString("base64url"),client:{publish:async value=>{
|
||||
if(value.operation==="complete"){published.push(value.authorityId);return{complete:true,receipt:{version:1,authorityId:value.authorityId,keyId:value.keyId,revision:value.revision,generation:createHash("sha256").update(value.authorityId).digest("hex")}};}
|
||||
return{complete:false,baseRevision:-1};
|
||||
}}});await index.ready;expect(published).toEqual([]);
|
||||
const verifier="v".repeat(43),location=await oauth.authorize({response_type:"code",client_id:client.client_id,redirect_uri:"https://client.example.test/callback",resource:oauth.config.resource,scope:"paperclip:read",code_challenge:createHash("sha256").update(verifier).digest("base64url"),code_challenge_method:"S256"});
|
||||
const actor:Request["actor"]={type:"board",source:"session",userId,companyIds:[companyId]};
|
||||
const grantInsert=vi.spyOn(PgDatabase.prototype,"insert"),intents:Array<{hints:string[];grantInserts:number}>=[];
|
||||
const unsubscribe=subscribeDatabaseWork(db,DELIVERY_QUEUES.mcpRevocationIndex,(event,hints)=>{if(event==="intent"&&hints?.size)intents.push({hints:[...hints],grantInserts:grantInsert.mock.calls.filter(([table])=>table===grants).length});});
|
||||
try{
|
||||
const approved=await oauth.consent(new URL(location).pathname.split("/").at(-1)!,actor,{decision:"approve",companyId,allowWrites:false});
|
||||
expect(new URL(approved.redirectUrl).searchParams.get("code")).toBeTruthy();
|
||||
const [approvedGrant]=await db.select().from(grants);
|
||||
expect(intents,JSON.stringify({expectedGrantInserts:0,observed:intents})).toEqual([{hints:[approvedGrant.id],grantInserts:0}]);
|
||||
expect(grantInsert.mock.calls.filter(([table])=>table===grants)).toHaveLength(1);
|
||||
}finally{unsubscribe();grantInsert.mockRestore();}
|
||||
await vi.waitFor(async()=>{const [grant]=await db.select().from(grants);expect(grant.revocationIndexReceipt?.revision).toBe(0);},{timeout:10_000});
|
||||
expect(published).toHaveLength(1);expect(await db.select().from(grants)).toHaveLength(1);
|
||||
});
|
||||
it("refreshes old partial capability exactly once at owner startup and waits for the current ACK after an outage",async()=>{
|
||||
await device();const [originalClient]=await db.select().from(clients),[originalDevice]=await db.select().from(devices);
|
||||
await start();const old=await acknowledged();expect(old.receipt?.frontDoor).toBeUndefined();expect((await inspect()).blocked).toBe(true);
|
||||
const outageStart=sent.length;
|
||||
remoteAvailable=true;failures=100;const worker=await start();const pending=(await current())!;
|
||||
expect(pending.revision).toBe(old.revision+1);expect(pending.receipt?.revision).toBe(old.revision);expect(pending.deviceCutoverAt).toBeNull();expect((await inspect()).blocked).toBe(true);
|
||||
expect(sent.slice(outageStart).map(value=>({operation:value.operation,revision:value.revision}))).toEqual([{operation:"begin",revision:pending.revision}]);
|
||||
const recoveryStart=sent.length;
|
||||
failures=0;worker.wake();const next=await acknowledged();expect(next.revision).toBe(pending.revision+1);expect(next.receipt?.revision).toBe(next.revision);expect(next.receipt?.frontDoor).toBe("tenant-personal-v1");expect(next.deviceCutoverAt).toBeInstanceOf(Date);expect((await inspect()).blocked).toBe(false);
|
||||
const recovery=sent.slice(recoveryStart);
|
||||
expect(recovery.map(value=>value.operation)).toEqual(["begin","append","append-device-status","complete"]);
|
||||
for(const value of recovery){expect(value.revision).toBe(pending.revision+1);expect(value.baseRevision).toBeNull();expect(value.expectedRecords).toBe(1);expect(value.instanceId).toBe(instanceId);expect(value.origin).toBe(origin);}
|
||||
const clientPage=recovery.find(value=>value.operation==="append"),devicePage=recovery.find(value=>value.operation==="append-device-status");
|
||||
expect(clientPage).toMatchObject({operation:"append",sequence:0,changes:[{operation:"upsert",client:{id:originalClient.id,name:originalClient.name,redirectUris:originalClient.redirectUris,grantTypes:originalClient.grantTypes}}]});
|
||||
if(clientPage?.operation!=="append"||devicePage?.operation!=="append-device-status")throw new Error("Missing full mirror recovery pages");
|
||||
expect(clientPage.changes).toEqual([{operation:"upsert",client:{id:originalClient.id,name:originalClient.name,redirectUris:originalClient.redirectUris,grantTypes:originalClient.grantTypes}}]);
|
||||
const key=deriveLocalEncryptedInstanceSecret(`mcp-device-status:v1:${instanceId}`);
|
||||
expect(devicePage.sequence).toBe(0);expect(devicePage.entries).toEqual([{requestId:originalDevice.id,clientId:originalDevice.clientId,resource:originalDevice.resource,ingressOwner:originalDevice.ingressOwner,status:originalDevice.status,expiresAt:originalDevice.expiresAt.getTime(),fingerprint:mcpDeviceStatusFingerprint(key,originalDevice.deviceCodeHash,originalDevice.clientId,originalDevice.resource)}]);
|
||||
expect((await db.select().from(devices))[0].expiresAt).toEqual(originalDevice.expiresAt);expect(await db.select().from(changes)).toEqual([]);expect(errors.length).toBeGreaterThanOrEqual(1);
|
||||
});
|
||||
it("a normal client mutation supersedes an in-flight page without acknowledging or logging the old revision",async()=>{
|
||||
const oauth=createPublicMcpOAuth(db,{origin,resource:origin+"/mcp/paperclip"});
|
||||
const register=(name:string)=>oauth.register({client_name:name,redirect_uris:["https://client.example.test/callback"],grant_types:["authorization_code","refresh_token"],response_types:["code"],token_endpoint_auth_method:"none"});
|
||||
await register("First client");remoteAvailable=true;let superseded:number|undefined;
|
||||
onAppend=async()=>{superseded=(await current())!.revision;await register("Concurrent client");};
|
||||
await start();expect(superseded).toBeDefined();expect((await inspect()).blocked).toBe(true);
|
||||
expect((await current())?.receipt).toBeNull();expect((await current())?.deviceCutoverAt).toBeNull();
|
||||
expect(sent.some(value=>value.operation==="complete"&&value.revision===superseded)).toBe(false);expect(errors).toEqual([]);
|
||||
const ready=await acknowledged();expect(ready.revision).toBeGreaterThan(superseded!);expect(ready.receipt?.revision).toBe(ready.revision);expect(ready.deviceCutoverAt).toBeTruthy();
|
||||
expect((await inspect()).blocked).toBe(false);expect(errors).toEqual([]);expect(await db.select().from(changes)).toEqual([]);
|
||||
});
|
||||
it("factory loss, disable and re-enable preserve cutover and demand a new source ACK without resetting legacy expiry",async()=>{
|
||||
await device();remoteAvailable=true;await start();const ready=await acknowledged(),expiry=(await db.select().from(devices))[0].expiresAt;
|
||||
remoteAvailable=false;await start();const unavailable=await acknowledged();expect(unavailable.revision).toBe(ready.revision+1);expect(unavailable.receipt?.frontDoor).toBeUndefined();expect((await inspect()).blocked).toBe(true);
|
||||
expect(unavailable.deviceCutoverAt).toEqual(ready.deviceCutoverAt);expect((await db.select().from(devices))[0].expiresAt).toEqual(expiry);
|
||||
await instanceSettingsService(db).updateExperimental({enablePublicMcp:false});await vi.waitFor(async()=>expect((await current())?.remoteResponsibility).toBe(false));
|
||||
const disabled=await current();expect(disabled?.receipt?.enabled).toBe(false);
|
||||
remoteAvailable=true;await instanceSettingsService(db).updateExperimental({enablePublicMcp:true});const enabled=await acknowledged();
|
||||
expect(enabled.revision).toBeGreaterThan(disabled!.revision);expect(enabled.receipt?.enabled).toBe(true);expect(enabled.receipt?.frontDoor).toBe("tenant-personal-v1");expect(enabled.deviceCutoverAt).toEqual(ready.deviceCutoverAt);
|
||||
failures=100;const replacement=await start("replacement-instance");
|
||||
expect((await readIdleMcpClientMirror(db,{budget,enabled:true,origin,instanceId:"replacement-instance",managedStackId:stackId})).blocked).toBe(true);
|
||||
expect((await current())!.receipt?.instanceId).toBe(instanceId);failures=0;replacement.wake();const replaced=await acknowledged();expect(replaced.receipt?.instanceId).toBe("replacement-instance");expect(replaced.deviceCutoverAt).toEqual(ready.deviceCutoverAt);
|
||||
const before=sent.length;managed.identity=null;await start();expect(sent).toHaveLength(before);
|
||||
});
|
||||
it("actual company deletion rolls back mirror intent and invalidates source status and token proof on commit",async()=>{
|
||||
const f=await device();const actor:Request["actor"]={type:"board",source:"session",userId,companyIds:[companyId]};
|
||||
await f.oauth.consentDevice(f.authorization.user_code,actor,{decision:"approve",companyId,allowWrites:false});
|
||||
const approved=(await db.select().from(devices))[0];expect(approved.grantId).toBeTruthy();
|
||||
await new Promise(resolve=>setTimeout(resolve,5001));
|
||||
const tokens=await f.oauth.token({grant_type:DEVICE_GRANT,client_id:f.client.client_id,resource:f.oauth.config.resource,device_code:f.authorization.device_code});expect(tokens.access_token).toMatch(/^pcmcp_at_/);
|
||||
remoteAvailable=true;await start();
|
||||
await startMcpRevocationIndexSync(db,coordinator!,{key:Buffer.alloc(32,9).toString("base64url"),client:{publish:async value=>value.operation==="complete"?{complete:true,receipt:{version:1,authorityId:value.authorityId,keyId:value.keyId,revision:value.revision,generation:createHash("sha256").update(`${value.authorityId}:${value.revision}`).digest("hex")}}:{complete:false,baseRevision:-1}}}).ready;
|
||||
const before=await acknowledged(),receipt=before.receipt!;
|
||||
await coordinator!.stop();coordinator=undefined;
|
||||
const proofOwnerId=randomUUID(),proofNow=Date.now();
|
||||
const inspectTokens=()=>readIdleMcpWakeProof(db,{stackId,ownerId:proofOwnerId,capabilityGeneration:"a".repeat(64),challenge:"c".repeat(64),personalResource:origin+"/mcp/paperclip",dotResource:origin+"/mcp/runner",personalEnabled:true,dotEnabled:false,clientMirror:receipt,now:proofNow,budget});
|
||||
const existing=await inspectTokens();expect(existing.grantIds).toContain(approved.grantId);expect(existing.admission.revocationAuthorities).toHaveLength(1);expect(existing.admission.lanes.length).toBeGreaterThan(0);
|
||||
expect((await readIdleMcpDeviceStatus(db,{enabled:true,receipt,budget})).blocked).toBe(false);
|
||||
const retained={company:await db.select().from(companies).where(eq(companies.id,companyId)),clients:await db.select().from(clients),devices:await db.select().from(devices),grants:await db.select().from(grants),changes:await db.select().from(changes),state:await current(),status:await readIdleMcpDeviceStatus(db,{enabled:true,receipt,budget})};
|
||||
const suffix=randomUUID().replaceAll("-",""),barrier=`mcp_delete_barrier_${suffix}`,constraint=`mcp_delete_fk_${suffix}`;
|
||||
await db.execute(sql`CREATE TABLE ${sql.identifier(barrier)} (company_id uuid NOT NULL, CONSTRAINT ${sql.identifier(constraint)} FOREIGN KEY (company_id) REFERENCES companies(id) ON DELETE RESTRICT)`);
|
||||
try{
|
||||
await db.execute(sql`INSERT INTO ${sql.identifier(barrier)} (company_id) VALUES (${companyId})`);
|
||||
await expect(deleteCompany(db,companyId)).rejects.toMatchObject({cause:{code:"23503",constraint_name:constraint}});
|
||||
expect(await db.select().from(companies).where(eq(companies.id,companyId))).toEqual(retained.company);expect(await db.select().from(clients)).toEqual(retained.clients);expect(await db.select().from(devices)).toEqual(retained.devices);expect(await db.select().from(grants)).toEqual(retained.grants);expect(await db.select().from(changes)).toEqual(retained.changes);expect(await current()).toEqual(retained.state);
|
||||
expect(await inspectTokens()).toEqual(existing);expect(await readIdleMcpDeviceStatus(db,{enabled:true,receipt,budget})).toEqual(retained.status);
|
||||
expect((await current())?.revision).toBe(before.revision);expect((await db.select().from(devices))[0].id).toBe(approved.id);expect((await readIdleMcpDeviceStatus(db,{enabled:true,receipt,budget})).blocked).toBe(false);
|
||||
}finally{await db.execute(sql`DROP TABLE ${sql.identifier(barrier)}`);}
|
||||
await deleteCompany(db,companyId);
|
||||
expect(await db.select().from(devices)).toEqual([]);expect(await db.select().from(grants)).toEqual([]);expect((await current())!.revision).toBeGreaterThan(before.revision);
|
||||
expect((await readIdleMcpDeviceStatus(db,{enabled:true,receipt,budget})).blocked).toBe(true);
|
||||
await start();const after=await acknowledged();expect(after.receipt?.deviceStatus?.count).toBe(0);expect(after.receipt?.deviceStatus?.generation).not.toBe(receipt.deviceStatus?.generation);
|
||||
const proof=await readIdleMcpWakeProof(db,{stackId,ownerId:randomUUID(),capabilityGeneration:"a".repeat(64),challenge:"c".repeat(64),personalResource:origin+"/mcp/paperclip",dotResource:origin+"/mcp/runner",personalEnabled:true,dotEnabled:false,clientMirror:after.receipt!,now:Date.now(),budget});
|
||||
expect(proof.grantIds).toEqual([]);expect(proof.admission.revocationAuthorities).toEqual([]);expect(await db.select().from(changes)).toEqual([]);
|
||||
},30_000);
|
||||
it("rotated local key invalidates old status and only the current bounded source publication restores it",async()=>{
|
||||
const packetStart=sent.length;
|
||||
let beforeRevision:number|undefined,afterRevision:number|undefined;
|
||||
try{
|
||||
await device();remoteAvailable=true;await start();const before=await acknowledged();beforeRevision=before.revision;
|
||||
// Replace the key context only after the previous publication owner has joined.
|
||||
await coordinator!.stop();coordinator=undefined;
|
||||
vi.stubEnv("PAPERCLIP_SECRETS_MASTER_KEY",randomBytes(32).toString("base64"));expect((await readIdleMcpDeviceStatus(db,{enabled:true,receipt:before.receipt!,budget})).blocked).toBe(true);
|
||||
await start();const after=await acknowledged();afterRevision=after.revision;expect(after.revision).toBe(before.revision+1);expect(after.receipt?.deviceStatus?.keyId).not.toBe(before.receipt?.deviceStatus?.keyId);expect((await readIdleMcpDeviceStatus(db,{enabled:true,receipt:after.receipt!,budget})).blocked).toBe(false);
|
||||
}finally{
|
||||
const category=(error:unknown)=>{
|
||||
if(!(error instanceof Error))return "non-error";
|
||||
if(error.message==="MCP client mirror changed during publication")return "publication-changed";
|
||||
if(error.message==="MCP client mirror was not acknowledged")return "ack-invalid";
|
||||
if(error.message==="MCP client mirror ACK generation changed")return "ack-generation-changed";
|
||||
if(error.message==="MCP client mirror ACK revision regressed")return "ack-revision-regressed";
|
||||
if(error.message==="MCP client mirror ACK base changed")return "ack-base-changed";
|
||||
if(error.message==="MCP client mirror ACK issuer changed")return "ack-issuer-changed";
|
||||
if(error.message==="MCP client mirror current authority changed")return "authority-changed";
|
||||
return error.name==="ZodError"?"schema-invalid":"other-error";
|
||||
};
|
||||
const diagnosticPath=process.env.PAPERCLIP_MCP_KEY_DIAGNOSTIC_PATH;
|
||||
if(diagnosticPath){
|
||||
try{
|
||||
const cache=process.env.XDG_CACHE_HOME;
|
||||
if(!cache||!path.isAbsolute(cache)||!path.isAbsolute(diagnosticPath)||diagnosticPath!==path.join(path.dirname(cache),"key-rotation-diagnostic.jsonl"))throw new Error("Invalid owned key diagnostic path");
|
||||
appendFileSync(diagnosticPath,JSON.stringify({event:"mcp-key-rotation-ram-diagnostic",beforeRevision:beforeRevision??null,afterRevision:afterRevision??null,packets:sent.slice(packetStart).map(value=>({operation:value.operation,revision:value.revision,baseRevision:value.baseRevision,expectedRecords:value.expectedRecords,recordCount:value.operation==="append"?value.changes.length:value.operation==="append-device-status"?value.entries.length:null})),errorCategories:errors.map(category)})+"\n",{encoding:"utf8",mode:0o600});
|
||||
}catch{process.stderr.write("MCP key diagnostic artifact write failed\n");}
|
||||
}
|
||||
}
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,25 @@
|
||||
import { eq, sql } from "drizzle-orm";
|
||||
import { mcpOauthClientMirrorState as state, mcpOauthClientChanges as changes, type Db } from "@paperclipai/db";
|
||||
import { DELIVERY_QUEUES, notifyDeliveryWork } from "../delivery-work-notifications.js";
|
||||
|
||||
const revisions = new WeakMap<object, number>();
|
||||
export async function prepareMcpClientMutation(tx: Db) {
|
||||
const existing = revisions.get(tx);
|
||||
if (existing !== undefined) return existing;
|
||||
await notifyDeliveryWork(tx, DELIVERY_QUEUES.mcpClientMirror);
|
||||
await tx.insert(state).values({id:"public",revision:0}).onConflictDoNothing();
|
||||
const [row] = await tx.update(state).set({revision:sql`${state.revision} + 1`}).where(eq(state.id,"public")).returning({revision:state.revision});
|
||||
if (!row || !Number.isSafeInteger(row.revision) || row.revision < 1) throw new Error("Invalid MCP client mirror revision");
|
||||
revisions.set(tx,row.revision);
|
||||
return row.revision;
|
||||
}
|
||||
export async function recordMcpClientChange(tx: Db, clientId: string, operation: "upsert"|"delete") {
|
||||
const revision = revisions.get(tx);
|
||||
if (revision === undefined) throw new Error("MCP client mutation was not prepared");
|
||||
await tx.insert(changes).values({clientId,revision,operation}).onConflictDoUpdate({target:changes.clientId,set:{revision,operation}});
|
||||
}
|
||||
|
||||
export async function prepareMcpClientMirrorPolicy(tx:Db,enabled:boolean){
|
||||
await prepareMcpClientMutation(tx);
|
||||
await tx.update(state).set({enabled,receipt:null}).where(eq(state.id,"public"));
|
||||
}
|
||||
@@ -0,0 +1,35 @@
|
||||
import type { McpClientMirrorChange, McpClientMirrorRequest } from "./client-mirror-contract.js";
|
||||
import type { McpDeviceStatusEntry } from "./device-status-contract.js";
|
||||
import { mcpClientMirrorPageFits } from "./client-mirror-wire.js";
|
||||
|
||||
type ClientPage = Extract<McpClientMirrorRequest, { operation: "append" }>;
|
||||
type DevicePage = Extract<McpClientMirrorRequest, { operation: "append-device-status" }>;
|
||||
type Identity = Omit<ClientPage, "operation" | "sequence" | "changes">;
|
||||
|
||||
export function mcpClientMirrorClientPages(identity: Identity, sequence: number, changes: McpClientMirrorChange[]): ClientPage[] {
|
||||
const pages: ClientPage[]=[];
|
||||
let page: McpClientMirrorChange[]=[];
|
||||
const request=():ClientPage=>({...identity,operation:"append",sequence:sequence+pages.length,changes:page});
|
||||
for(const change of changes){
|
||||
const candidate={...request(),changes:[...page,change]};
|
||||
if(page.length&&(page.length===200||!mcpClientMirrorPageFits(candidate))){pages.push(request());page=[];}
|
||||
page.push(change);
|
||||
if(!mcpClientMirrorPageFits(request()))throw new Error("MCP mirrored client exceeds the bounded single-record contract");
|
||||
}
|
||||
if(page.length)pages.push(request());
|
||||
return pages;
|
||||
}
|
||||
|
||||
export function mcpClientMirrorDevicePages(identity: Identity, entries: McpDeviceStatusEntry[]): DevicePage[] {
|
||||
const pages: DevicePage[]=[];
|
||||
let page: McpDeviceStatusEntry[]=[];
|
||||
const request=():DevicePage=>({...identity,operation:"append-device-status",sequence:pages.length,entries:page});
|
||||
for(const entry of entries){
|
||||
const candidate={...request(),entries:[...page,entry]};
|
||||
if(page.length&&(page.length===200||!mcpClientMirrorPageFits(candidate))){pages.push(request());page=[];}
|
||||
page.push(entry);
|
||||
if(!mcpClientMirrorPageFits(request()))throw new Error("MCP device status exceeds the bounded single-record contract");
|
||||
}
|
||||
if(page.length)pages.push(request());
|
||||
return pages;
|
||||
}
|
||||
@@ -0,0 +1,169 @@
|
||||
import { mcpClientMirrorClientPages, mcpClientMirrorDevicePages } from "./client-mirror-pages.js";
|
||||
import { getCloudRuntimeIdentity } from "../cloud-runtime-identity.js";
|
||||
import { mcpDeviceStatusEntrySchema,mcpDeviceStatusFingerprint,mcpDeviceStatusGeneration,mcpDeviceStatusKeyId } from "./device-status-contract.js";
|
||||
import { deriveLocalEncryptedInstanceSecret } from "../../secrets/local-encrypted-provider.js";
|
||||
import { prepareMcpClientMutation } from "./client-mirror-mutations.js";
|
||||
import { installMcpDeviceCutover } from "./device-cutover.js";
|
||||
import { prepareMcpClientMirrorPolicy } from "./client-mirror-mutations.js";
|
||||
import { and, eq, gt, lte, sql } from "drizzle-orm";
|
||||
import { mcpOauthClients as clients,mcpOauthDeviceRequests as devices, mcpOauthClientMirrorState as state, mcpOauthClientChanges as changes, type Db } from "@paperclipai/db";
|
||||
import { instanceSettingsService } from "../instance-settings.js";
|
||||
import { createPaperclipCloudConnector, paperclipCloudConnectorConfigFromEnv } from "../paperclip-cloud-connector.js";
|
||||
import { DELIVERY_QUEUES } from "../delivery-work-notifications.js";
|
||||
import type { createDeliveryWorkCoordinator } from "../delivery-work-coordinator.js";
|
||||
import { mcpClientMirrorGeneration, mcpClientMirrorReceiptSchema, mcpClientMirrorResponseSchema, mcpMirroredClientSchema,
|
||||
type McpClientMirrorChange, type McpClientMirrorRequest, type McpClientMirrorReceipt } from "./client-mirror-contract.js";
|
||||
|
||||
export function sameMcpClientMirrorReceipt(left:McpClientMirrorReceipt,right:McpClientMirrorReceipt){
|
||||
return left.version===right.version&&left.instanceId===right.instanceId&&left.origin===right.origin
|
||||
&&left.revision===right.revision&&left.enabled===right.enabled&&left.generation===right.generation
|
||||
&&left.frontDoor===right.frontDoor&&(left.authorizationOrigin??left.origin)===(right.authorizationOrigin??right.origin)
|
||||
&&left.deviceStatus?.keyId===right.deviceStatus?.keyId&&left.deviceStatus?.generation===right.deviceStatus?.generation
|
||||
&&left.deviceStatus?.count===right.deviceStatus?.count;
|
||||
}
|
||||
|
||||
export function startMcpClientMirrorSync(db: Db, coordinator: Pick<ReturnType<typeof createDeliveryWorkCoordinator>,"register">,
|
||||
origin: string | undefined, input: {authorizationOrigin?:string;instanceId?:string;publish?:(request:McpClientMirrorRequest)=>Promise<unknown>} = {}) {
|
||||
const config = paperclipCloudConnectorConfigFromEnv();
|
||||
const connector = config ? createPaperclipCloudConnector({config}) : undefined;
|
||||
const instanceId = input.instanceId ?? config?.instanceId;
|
||||
const publish = input.publish ?? (connector ? (request:McpClientMirrorRequest) => connector.syncMcpClientMirror(JSON.stringify(request)) : undefined);
|
||||
let pending = true,refreshPublished=true,fullRecovery=false,reserveFullRevision=false;
|
||||
let boundStackId=getCloudRuntimeIdentity()?.stackId;
|
||||
const worker=coordinator.register(DELIVERY_QUEUES.mcpClientMirror, {
|
||||
retryMs:5_000,pausePassiveMaintenance:true,onSettled:()=>{pending=true;},
|
||||
async run(signal) {
|
||||
pending=false;
|
||||
const managedIdentity=getCloudRuntimeIdentity();
|
||||
if(!managedIdentity)return;
|
||||
if (!origin || !instanceId || !publish) return;
|
||||
let enabled=false,publicationAttempted=false;
|
||||
try {
|
||||
enabled=(await instanceSettingsService(db).getExperimentalReadOnly()).enablePublicMcp;
|
||||
const assertCurrentAuthority=()=>{
|
||||
const identity=getCloudRuntimeIdentity(),currentConfig=paperclipCloudConnectorConfigFromEnv();
|
||||
if(!identity||identity.stackId!==managedIdentity.stackId||boundStackId!==undefined&&identity.stackId!==boundStackId
|
||||
||identity.canonicalOrigin!==origin||config&&(!currentConfig||currentConfig.instanceId!==instanceId
|
||||
||currentConfig.baseUrl!==config.baseUrl||currentConfig.environment!==config.environment
|
||||
||currentConfig.signPrivateKey!==config.signPrivateKey||currentConfig.sealPrivateKey!==config.sealPrivateKey))throw new Error("MCP client mirror current authority changed");
|
||||
boundStackId??=identity.stackId;
|
||||
};
|
||||
if(enabled)assertCurrentAuthority();
|
||||
const publishEnabled=(request:McpClientMirrorRequest)=>{assertCurrentAuthority();publicationAttempted=true;return publish(request);};
|
||||
const captured=await db.transaction(async tx=>{
|
||||
await tx.execute(sql`SELECT pg_advisory_xact_lock(736721042)`);
|
||||
if(enabled)await tx.insert(state).values({id:"public",revision:0}).onConflictDoNothing();
|
||||
const [row]=await tx.select().from(state).where(eq(state.id,"public")).for("update");
|
||||
if(!row)return;
|
||||
if(row.enabled===enabled){
|
||||
if(!enabled||!reserveFullRevision&&(!refreshPublished||row.receipt===null))return row;
|
||||
await prepareMcpClientMutation(tx as unknown as Db);
|
||||
return(await tx.select().from(state).where(eq(state.id,"public")))[0];
|
||||
}
|
||||
await prepareMcpClientMirrorPolicy(tx as unknown as Db,enabled);
|
||||
return(await tx.select().from(state).where(eq(state.id,"public")))[0];
|
||||
});
|
||||
refreshPublished=false;
|
||||
if(enabled&&captured&&fullRecovery)reserveFullRevision=false;
|
||||
if(!captured&&!enabled)return;
|
||||
if (!captured || !Number.isSafeInteger(captured.revision) || captured.revision < 0) throw new Error("Invalid MCP client mirror state");
|
||||
const generation = mcpClientMirrorGeneration({instanceId,origin,revision:captured.revision,enabled});
|
||||
const old = mcpClientMirrorReceiptSchema.safeParse(captured.receipt);
|
||||
const baseRevision = !fullRecovery && old.success && old.data.instanceId===instanceId && old.data.origin===origin && old.data.revision<=captured.revision
|
||||
&& old.data.generation===mcpClientMirrorGeneration(old.data) ? old.data.revision : null;
|
||||
if (!enabled&&baseRevision===captured.revision&&old.success&&old.data.enabled===false) return;
|
||||
if(!enabled){
|
||||
if(!captured.remoteResponsibility)return;
|
||||
const result=mcpClientMirrorResponseSchema.parse(await publish({version:1,instanceId,origin,revision:captured.revision,baseRevision:null,expectedRecords:0,operation:"disable"}));
|
||||
if(!result.complete||result.receipt.enabled||result.receipt.instanceId!==instanceId||result.receipt.origin!==origin||result.receipt.revision!==captured.revision||result.receipt.generation!==generation)throw new Error("MCP client mirror disable was not acknowledged");
|
||||
const [ack]=await db.update(state).set({receipt:result.receipt,remoteResponsibility:false}).where(and(eq(state.id,"public"),eq(state.revision,captured.revision))).returning({id:state.id});
|
||||
if(!ack)pending=true;
|
||||
return;
|
||||
}
|
||||
const deviceKey=deriveLocalEncryptedInstanceSecret(`mcp-device-status:v1:${instanceId}`),keyId=mcpDeviceStatusKeyId(deviceKey);
|
||||
const deviceRows=await db.select({requestId:devices.id,deviceCodeHash:devices.deviceCodeHash,clientId:devices.clientId,resource:devices.resource,ingressOwner:devices.ingressOwner,status:devices.status,expiresAt:devices.expiresAt}).from(devices).where(eq(devices.resource,origin+"/mcp/paperclip")).orderBy(devices.id).limit(1001);
|
||||
if(deviceRows.length>1000)throw new Error("MCP retained device set exceeds canonical1000 admission bound");
|
||||
const entries=deviceRows.map(row=>mcpDeviceStatusEntrySchema.parse({requestId:row.requestId,clientId:row.clientId,resource:row.resource,ingressOwner:row.ingressOwner,status:row.status,expiresAt:row.expiresAt.getTime(),fingerprint:mcpDeviceStatusFingerprint(deviceKey,row.deviceCodeHash,row.clientId,row.resource)}));
|
||||
const deviceStatus={keyId,generation:mcpDeviceStatusGeneration({instanceId,origin,revision:captured.revision,keyId},entries),count:entries.length};
|
||||
if(baseRevision===captured.revision&&old.success){
|
||||
if(JSON.stringify(old.data.deviceStatus)===JSON.stringify(deviceStatus)&&(old.data.authorizationOrigin??origin)===(input.authorizationOrigin??origin))return;
|
||||
await db.transaction(async tx=>{
|
||||
await tx.execute(sql`SELECT pg_advisory_xact_lock(736721042)`);
|
||||
const [current]=await tx.select({revision:state.revision}).from(state).where(eq(state.id,"public")).for("update");
|
||||
if(current?.revision===captured.revision)await prepareMcpClientMutation(tx as unknown as Db);
|
||||
});
|
||||
pending=true;return;
|
||||
}
|
||||
const [count] = baseRevision===null ? await db.select({total:sql<number>`count(*)::int`}).from(clients)
|
||||
: await db.select({total:sql<number>`count(*)::int`}).from(changes).where(and(gt(changes.revision,baseRevision),lte(changes.revision,captured.revision)));
|
||||
if (!count || !Number.isSafeInteger(count.total) || count.total<0) throw new Error("Invalid MCP client mirror count");
|
||||
const identity = {version:1 as const,instanceId,origin,revision:captured.revision,baseRevision,expectedRecords:count.total,deviceStatus,...(input.authorizationOrigin?{authorizationOrigin:input.authorizationOrigin}:{})};
|
||||
const [owned]=await db.update(state).set({remoteResponsibility:true}).where(and(eq(state.id,"public"),eq(state.revision,captured.revision))).returning({id:state.id});
|
||||
if(!owned){pending=true;return;}
|
||||
let result = mcpClientMirrorResponseSchema.parse(await publishEnabled({...identity,operation:"begin",deviceKey}));
|
||||
if (!result.complete) {
|
||||
const resumeSequence=result.nextSequence,resumeDeviceSequence=result.nextDeviceSequence??0;
|
||||
let cursor:string|undefined, sequence=0, sent=0;
|
||||
while (true) {
|
||||
signal.throwIfAborted();
|
||||
const rows: {clientId:string;change:McpClientMirrorChange}[] = baseRevision===null
|
||||
? (await db.select({id:clients.id,name:clients.name,redirectUris:clients.redirectUris,grantTypes:clients.grantTypes}).from(clients)
|
||||
.where(cursor ? sql`${clients.id} COLLATE "C" > ${cursor}` : undefined).orderBy(sql`${clients.id} COLLATE "C"`).limit(200))
|
||||
.map(row=>({clientId:row.id,change:{operation:"upsert" as const,client:mcpMirroredClientSchema.parse(row)}}))
|
||||
: (await db.select({change:changes,client:{id:clients.id,name:clients.name,redirectUris:clients.redirectUris,grantTypes:clients.grantTypes}}).from(changes)
|
||||
.leftJoin(clients,eq(clients.id,changes.clientId)).where(and(gt(changes.revision,baseRevision),lte(changes.revision,captured.revision),cursor?sql`${changes.clientId} COLLATE "C" > ${cursor}`:undefined))
|
||||
.orderBy(sql`${changes.clientId} COLLATE "C"`).limit(200)).map(({change:row,client})=>{
|
||||
if(row.operation==="delete")return{clientId:row.clientId,change:{operation:"delete" as const,clientId:row.clientId}};
|
||||
if(!client)throw new Error("MCP client changed during publication");
|
||||
return{clientId:row.clientId,change:{operation:"upsert" as const,client:mcpMirroredClientSchema.parse(client)}};
|
||||
});
|
||||
if(!rows.length)break;
|
||||
for(const page of mcpClientMirrorClientPages(identity,sequence,rows.map(row=>row.change))){
|
||||
if(sequence>=resumeSequence)result=mcpClientMirrorResponseSchema.parse(await publishEnabled(page));
|
||||
sequence++;sent+=page.changes.length;
|
||||
}
|
||||
cursor=rows.at(-1)!.clientId;
|
||||
if(rows.length<200)break;
|
||||
}
|
||||
for(const page of mcpClientMirrorDevicePages(identity,entries)){
|
||||
if(page.sequence>=resumeDeviceSequence)result=mcpClientMirrorResponseSchema.parse(await publishEnabled(page));
|
||||
}
|
||||
signal.throwIfAborted();
|
||||
const [current]=await db.select({revision:state.revision}).from(state).where(eq(state.id,"public"));
|
||||
if(current&&Number.isSafeInteger(current.revision)&¤t.revision>captured.revision){pending=true;return;}
|
||||
if(current?.revision!==captured.revision || sent!==count.total)throw new Error("MCP client mirror changed during publication");
|
||||
result=mcpClientMirrorResponseSchema.parse(await publishEnabled({...identity,operation:"complete"}));
|
||||
}
|
||||
if(!result.complete||!result.receipt.enabled||JSON.stringify(result.receipt.deviceStatus)!==JSON.stringify(deviceStatus)||result.receipt.instanceId!==instanceId||result.receipt.origin!==origin||result.receipt.revision!==captured.revision||result.receipt.generation!==mcpClientMirrorGeneration(result.receipt))throw new Error("MCP client mirror was not acknowledged");
|
||||
const receipt=result.receipt;
|
||||
let retained=false;
|
||||
await db.transaction(async tx=>{
|
||||
await tx.execute(sql`SELECT pg_advisory_xact_lock(736721042)`);
|
||||
const [current]=await tx.select().from(state).where(eq(state.id,"public")).for("update");
|
||||
assertCurrentAuthority();
|
||||
if(!current||!current.enabled){pending=true;return;}
|
||||
if(!Number.isSafeInteger(current.revision)||current.revision<receipt.revision)throw new Error("MCP client mirror ACK revision regressed");
|
||||
if((receipt.authorizationOrigin??origin)!==(input.authorizationOrigin??origin))throw new Error("MCP client mirror ACK issuer changed");
|
||||
const prior=mcpClientMirrorReceiptSchema.safeParse(current.receipt);
|
||||
if(current.receipt!==null&&(!prior.success||prior.data.generation!==mcpClientMirrorGeneration(prior.data)))throw new Error("MCP client mirror ACK base changed");
|
||||
if(current.revision>receipt.revision&&(!prior.success||!prior.data.enabled||prior.data.instanceId!==instanceId||prior.data.origin!==origin)){
|
||||
pending=true;return;
|
||||
}
|
||||
if(prior.success&&prior.data.revision>receipt.revision){pending=true;return;}
|
||||
if(prior.success&&prior.data.revision===receipt.revision&&!sameMcpClientMirrorReceipt(prior.data,receipt))throw new Error("MCP client mirror ACK generation changed");
|
||||
await tx.update(state).set({receipt}).where(eq(state.id,"public"));
|
||||
retained=true;
|
||||
if(current.revision!==receipt.revision){pending=true;return;}
|
||||
await installMcpDeviceCutover(tx as unknown as Db,receipt);
|
||||
await tx.delete(changes).where(lte(changes.revision,receipt.revision));
|
||||
});
|
||||
if(retained){fullRecovery=false;reserveFullRevision=false;}
|
||||
} catch(error){
|
||||
if(enabled&&publicationAttempted&&!fullRecovery){fullRecovery=true;reserveFullRevision=true;}
|
||||
pending=true;throw error;
|
||||
}
|
||||
},
|
||||
hasPending:async()=>false,nextRunAt:()=>pending?Date.now()+5_000:null,
|
||||
});
|
||||
return {worker,instanceId};
|
||||
}
|
||||
@@ -0,0 +1,91 @@
|
||||
import { mcpClientMirrorClientPages, mcpClientMirrorDevicePages } from "./client-mirror-pages.js";
|
||||
import assert from "node:assert/strict";
|
||||
import { test } from "vitest";
|
||||
import { mcpClientMirrorRequestSchema } from "./client-mirror-contract.js";
|
||||
import { assertMcpClientMirrorEnvelope, mcpClientMirrorPageFits, mcpClientMirrorRequestByteLimit,
|
||||
MCP_MIRROR_NORMAL_PAGE_BYTES, MCP_MIRROR_SINGLE_RECORD_BYTES,
|
||||
MCP_MIRROR_MAX_BINDING_BYTES, MCP_MIRROR_MAX_ENVELOPE_BYTES, MCP_MIRROR_MAX_JWS_CHARACTERS,
|
||||
} from "./client-mirror-wire.js";
|
||||
|
||||
const origin="https://tenant.example.test";
|
||||
const identity={version:1 as const,instanceId:"i".repeat(200),origin,authorizationOrigin:origin,revision:Number.MAX_SAFE_INTEGER,
|
||||
baseRevision:Number.MAX_SAFE_INTEGER,expectedRecords:Number.MAX_SAFE_INTEGER,
|
||||
deviceStatus:{keyId:"f".repeat(64),generation:"f".repeat(64),count:1000}};
|
||||
const redirect="https://example.test/"+"\u0000".repeat(2048-21);
|
||||
const client={id:"c".repeat(2048),name:"\u0000".repeat(100),redirectUris:Array(10).fill(redirect),grantTypes:["authorization_code","refresh_token","urn:ietf:params:oauth:grant-type:device_code"]};
|
||||
const upsert={operation:"upsert",client};
|
||||
const device={requestId:"00000000-0000-4000-8000-000000000000",fingerprint:"f".repeat(64),clientId:"\u0000".repeat(2048),resource:redirect,ingressOwner:"tenant" as const,status:"consumed" as const,expiresAt:Number.MAX_SAFE_INTEGER};
|
||||
const size=(value:unknown)=>Buffer.byteLength(JSON.stringify(value),"utf8");
|
||||
|
||||
test("derived singleton bounds cover strict upsert, delete and device variants without dropping metadata",()=>{
|
||||
const variants=[
|
||||
{...identity,operation:"append",sequence:Number.MAX_SAFE_INTEGER,changes:[upsert]},
|
||||
{...identity,operation:"append",sequence:Number.MAX_SAFE_INTEGER,changes:[{operation:"delete",clientId:"\u0000".repeat(2048)}]},
|
||||
{...identity,operation:"append-device-status",sequence:Number.MAX_SAFE_INTEGER,entries:[device]},
|
||||
];
|
||||
assert.ok(size(variants[0])>MCP_MIRROR_NORMAL_PAGE_BYTES);
|
||||
for(const value of variants){
|
||||
const parsed=mcpClientMirrorRequestSchema.parse(value);
|
||||
assert.deepEqual(parsed,value);
|
||||
assert.equal(mcpClientMirrorPageFits(value),true);
|
||||
assert.ok(size(value)<=mcpClientMirrorRequestByteLimit(value));
|
||||
}
|
||||
assert.ok(MCP_MIRROR_MAX_BINDING_BYTES>=Math.max(...Object.values(MCP_MIRROR_SINGLE_RECORD_BYTES)));
|
||||
});
|
||||
|
||||
test("singleton exception cannot admit oversized multi-record, invalid, unknown or count-overflow data",()=>{
|
||||
assert.equal(mcpClientMirrorRequestSchema.safeParse({...identity,operation:"append",sequence:0,changes:[upsert,upsert]}).success,false);
|
||||
assert.equal(mcpClientMirrorRequestSchema.safeParse({...identity,operation:"append",sequence:0,changes:[{operation:"upsert",client:{...client,name:"n".repeat(101)}}]}).success,false);
|
||||
assert.equal(mcpClientMirrorRequestSchema.safeParse({...identity,operation:"append",sequence:0,changes:[{...upsert,extra:true}]}).success,false);
|
||||
assert.equal(mcpClientMirrorRequestSchema.safeParse({...identity,operation:"append-device-status",sequence:0,entries:[{...device,clientId:"c".repeat(2049)}]}).success,false);
|
||||
assert.equal(mcpClientMirrorRequestSchema.safeParse({...identity,operation:"append-device-status",sequence:0,entries:Array(201).fill(device)}).success,false);
|
||||
for(const change of [
|
||||
{operation:"delete",clientId:"c".repeat(2049)},
|
||||
{operation:"upsert",client:{...client,id:"c".repeat(2049)}},
|
||||
{operation:"upsert",client:{...client,redirectUris:["https://example.test/"+"a".repeat(2049-21)]}},
|
||||
{operation:"upsert",client:{...client,redirectUris:Array(11).fill("https://example.test/")}},
|
||||
])assert.equal(mcpClientMirrorRequestSchema.safeParse({...identity,operation:"append",sequence:0,changes:[change]}).success,false);
|
||||
assert.equal(mcpClientMirrorRequestSchema.safeParse({...identity,instanceId:"i".repeat(201),operation:"complete"}).success,false);
|
||||
assert.equal(mcpClientMirrorRequestSchema.safeParse({...identity,operation:"append-device-status",sequence:0,entries:[{...device,resource:"https://example.test/"+"a".repeat(2049-21)}]}).success,false);
|
||||
for(const field of ["revision","baseRevision","expectedRecords","sequence"])assert.equal(mcpClientMirrorRequestSchema.safeParse({...identity,operation:"append",sequence:0,changes:[{operation:"delete",clientId:"client"}],[field]:-1}).success,false);
|
||||
assert.equal(mcpClientMirrorRequestSchema.safeParse({...identity,operation:"append-device-status",sequence:0,entries:[{...device,expiresAt:-1}]}).success,false);
|
||||
for(const operation of ["begin","complete","disable"])assert.equal(mcpClientMirrorRequestByteLimit({...identity,operation}),MCP_MIRROR_NORMAL_PAGE_BYTES);
|
||||
});
|
||||
|
||||
test("normal pages keep the byte-and-count contract for escaped and nonASCII metadata",()=>{
|
||||
for(const text of ['"\\\n',"漢字🙂","\ud800"]){
|
||||
const change={operation:"upsert",client:{id:"c",name:text,redirectUris:["https://example.test/callback"],grantTypes:["authorization_code"]}};
|
||||
const candidate={...identity,operation:"append",sequence:0,changes:[change,change]};
|
||||
assert.equal(mcpClientMirrorPageFits(candidate),true);
|
||||
assert.deepEqual(mcpClientMirrorRequestSchema.parse(candidate),candidate);
|
||||
}
|
||||
});
|
||||
|
||||
test("one exact encoded-envelope check enforces outerUTF8 and unchanged JWS bound",()=>{
|
||||
const binding=JSON.stringify({...identity,operation:"append",sequence:0,changes:[upsert]});
|
||||
const request="a".repeat(MCP_MIRROR_MAX_JWS_CHARACTERS);
|
||||
const body=JSON.stringify({request,binding});
|
||||
assert.ok(Buffer.byteLength(body)>64000);
|
||||
assert.ok(Buffer.byteLength(body)<=MCP_MIRROR_MAX_ENVELOPE_BYTES);
|
||||
assert.doesNotThrow(()=>assertMcpClientMirrorEnvelope(request,binding,body));
|
||||
assert.throws(()=>assertMcpClientMirrorEnvelope(request+"a",binding,body));
|
||||
assert.throws(()=>assertMcpClientMirrorEnvelope(request,"x".repeat(MCP_MIRROR_MAX_BINDING_BYTES+1),body));
|
||||
assert.throws(()=>assertMcpClientMirrorEnvelope(request,binding,"x".repeat(MCP_MIRROR_MAX_ENVELOPE_BYTES+1)));
|
||||
});
|
||||
|
||||
test("actual domain packers preserve every record, page sequence and normal200/49152 limits",()=>{
|
||||
const clients=Array.from({length:205},(_,index)=>({operation:"delete" as const,clientId:"client-"+index}));
|
||||
const pages=mcpClientMirrorClientPages(identity,7,clients);
|
||||
assert.deepEqual(pages.map(page=>page.sequence),[7,8]);
|
||||
assert.deepEqual(pages.flatMap(page=>page.changes),clients);
|
||||
const entries=Array.from({length:1000},(_,index)=>({...device,requestId:`00000000-0000-4000-8000-${String(index).padStart(12,"0")}`,clientId:"legacy-client",resource:origin+"/mcp/paperclip"}));
|
||||
const devicePages=mcpClientMirrorDevicePages(identity,entries);
|
||||
assert.deepEqual(devicePages.flatMap(page=>page.entries),entries);
|
||||
assert.deepEqual(devicePages.map(page=>page.sequence),devicePages.map((_,index)=>index));
|
||||
for(const page of [...pages,...devicePages]){
|
||||
const count=page.operation==="append"?page.changes.length:page.entries.length;
|
||||
assert.ok(count>0&&count<=200);
|
||||
assert.ok(size(page)<=MCP_MIRROR_NORMAL_PAGE_BYTES||count===1);
|
||||
mcpClientMirrorRequestSchema.parse(page);
|
||||
}
|
||||
});
|
||||
@@ -0,0 +1,46 @@
|
||||
const maxInteger = Number.MAX_SAFE_INTEGER;
|
||||
const digest = "f".repeat(64);
|
||||
const identity = {
|
||||
version: 1, instanceId: "", origin: "", revision: maxInteger,
|
||||
baseRevision: maxInteger, expectedRecords: maxInteger, authorizationOrigin: "",
|
||||
deviceStatus: { keyId: digest, generation: digest, count: 1000 },
|
||||
};
|
||||
const bytes = (value: unknown) => Buffer.byteLength(JSON.stringify(value), "utf8");
|
||||
const bound = (value: unknown, variableUtf16Units: number) => bytes(value) + 6 * variableUtf16Units;
|
||||
const identityUnits = 200 + 2048 + 2048;
|
||||
export const MCP_MIRROR_NORMAL_PAGE_BYTES = 49_152;
|
||||
export const MCP_MIRROR_MAX_JWS_CHARACTERS = 12_000;
|
||||
export const MCP_MIRROR_SINGLE_RECORD_BYTES = Object.freeze({
|
||||
upsert: bound({ ...identity, operation: "append", sequence: maxInteger, changes: [{
|
||||
operation: "upsert", client: { id: "", name: "", redirectUris: Array(10).fill(""),
|
||||
grantTypes: Array(3).fill("urn:ietf:params:oauth:grant-type:device_code") },
|
||||
}] }, identityUnits + 2048 + 100 + 10 * 2048),
|
||||
delete: bound({ ...identity, operation: "append", sequence: maxInteger,
|
||||
changes: [{ operation: "delete", clientId: "" }] }, identityUnits + 2048),
|
||||
device: bound({ ...identity, operation: "append-device-status", sequence: maxInteger, entries: [{
|
||||
requestId: "00000000-0000-4000-8000-000000000000", fingerprint: digest,
|
||||
clientId: "", resource: "", ingressOwner: "tenant", status: "consumed", expiresAt: maxInteger,
|
||||
}] }, identityUnits + 2048 + 2048),
|
||||
});
|
||||
export const MCP_MIRROR_MAX_BINDING_BYTES = Math.max(MCP_MIRROR_NORMAL_PAGE_BYTES, ...Object.values(MCP_MIRROR_SINGLE_RECORD_BYTES));
|
||||
export const MCP_MIRROR_MAX_ENVELOPE_BYTES = 2 * MCP_MIRROR_MAX_BINDING_BYTES
|
||||
+ MCP_MIRROR_MAX_JWS_CHARACTERS + bytes({ request: "", binding: "" });
|
||||
type Page = { operation: string; sequence?: number; changes?: { operation: string }[]; entries?: unknown[] };
|
||||
export function mcpClientMirrorRequestByteLimit(value: Page): number {
|
||||
if (value.operation === "append" && value.changes?.length === 1) {
|
||||
if (value.changes[0]?.operation === "upsert") return Math.max(MCP_MIRROR_NORMAL_PAGE_BYTES, MCP_MIRROR_SINGLE_RECORD_BYTES.upsert);
|
||||
if (value.changes[0]?.operation === "delete") return Math.max(MCP_MIRROR_NORMAL_PAGE_BYTES, MCP_MIRROR_SINGLE_RECORD_BYTES.delete);
|
||||
}
|
||||
if (value.operation === "append-device-status" && value.entries?.length === 1) return Math.max(MCP_MIRROR_NORMAL_PAGE_BYTES, MCP_MIRROR_SINGLE_RECORD_BYTES.device);
|
||||
return MCP_MIRROR_NORMAL_PAGE_BYTES;
|
||||
}
|
||||
export function mcpClientMirrorPageFits(value: Page): boolean {
|
||||
return bytes(value) <= mcpClientMirrorRequestByteLimit(value);
|
||||
}
|
||||
export function assertMcpClientMirrorEnvelope(request: string, binding: string, encodedBody: string): void {
|
||||
if (request.length > MCP_MIRROR_MAX_JWS_CHARACTERS
|
||||
|| Buffer.byteLength(binding, "utf8") > MCP_MIRROR_MAX_BINDING_BYTES
|
||||
|| Buffer.byteLength(encodedBody, "utf8") > MCP_MIRROR_MAX_ENVELOPE_BYTES) {
|
||||
throw new Error("MCP client mirror signed envelope exceeds its bounded wire contract");
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,37 @@
|
||||
import {beforeEach,describe,expect,it,vi} from "vitest";
|
||||
import type {Db} from "@paperclipai/db";
|
||||
const current=vi.hoisted(()=>({identity:null as null|{stackId:string;canonicalOrigin:string},instanceId:null as string|null,enabled:true,settingsReads:0}));
|
||||
vi.mock("../cloud-runtime-identity.js",()=>({getCloudRuntimeIdentity:()=>current.identity}));
|
||||
vi.mock("../paperclip-cloud-connector.js",()=>({paperclipCloudConnectorConfigFromEnv:()=>current.instanceId?{instanceId:current.instanceId}:null}));
|
||||
vi.mock("../instance-settings.js",()=>({instanceSettingsService:()=>({getExperimentalReadOnly:async()=>{current.settingsReads++;return{enablePublicMcp:current.enabled};}})}));
|
||||
import {cloudEventAuthoritySchema,cloudEventValidationOrigin} from "./cloud-event-authority.js";
|
||||
import {mcpClientMirrorGeneration} from "./client-mirror-contract.js";
|
||||
const origin="https://tenant.example.test",central="https://my.example.test",config={origin,resource:origin+"/mcp/paperclip"},authority={token:"opaque-wrapper",expiresAt:Date.now()+60_000,transport:"tenant-personal-v1" as const};
|
||||
function fixture(patch:Record<string,unknown>={}){
|
||||
const receipt={version:1 as const,instanceId:"instance",origin,revision:1,enabled:true,frontDoor:"tenant-personal-v1" as const,deviceStatus:{keyId:"a".repeat(64),generation:"b".repeat(64),count:0},generation:""};receipt.generation=mcpClientMirrorGeneration(receipt);
|
||||
const row={revision:1,enabled:true,remoteResponsibility:true,receipt,deviceCutoverAt:new Date(1),deviceCutoverStackId:"stack",...patch};let reads=0;
|
||||
return{db:{select:()=>{reads++;return{from:()=>({where:async()=>[row]})};}}as unknown as Db,reads:()=>reads};
|
||||
}
|
||||
describe("hosted event authority transport selection",()=>{
|
||||
beforeEach(()=>{current.identity={stackId:"stack",canonicalOrigin:origin};current.instanceId="instance";current.enabled=true;current.settingsReads=0;});
|
||||
it("absent marker preserves the existing fixed central origin without tenant metadata reads",async()=>{
|
||||
const f=fixture();expect(await cloudEventValidationOrigin(f.db,config,central,{token:"central-wrapper",expiresAt:authority.expiresAt})).toBe(central);expect(f.reads()).toBe(0);expect(current.settingsReads).toBe(0);
|
||||
});
|
||||
it("current complete mirror derives the exact verified tenant endpoint",async()=>{
|
||||
const f=fixture();expect(await cloudEventValidationOrigin(f.db,config,central,authority)).toBe(origin);expect(f.reads()).toBe(1);expect(current.settingsReads).toBe(1);
|
||||
});
|
||||
it("unknown marker or caller URL never selects a validation endpoint",async()=>{
|
||||
for(const value of [{...authority,transport:"unknown"},{...authority,transport:null},{...authority,url:"https://attacker.example"}])expect(cloudEventAuthoritySchema.safeParse(value).success).toBe(false);
|
||||
const f=fixture();await expect(cloudEventValidationOrigin(f.db,config,central,{...authority,transport:"unknown"}as unknown as typeof authority)).rejects.toThrow("Invalid hosted");expect(f.reads()).toBe(0);
|
||||
});
|
||||
it("missing managed authority, connector or central enrollment fails closed",async()=>{
|
||||
const f=fixture();current.identity=null;await expect(cloudEventValidationOrigin(f.db,config,central,authority)).rejects.toThrow();current.identity={stackId:"stack",canonicalOrigin:origin};current.instanceId=null;await expect(cloudEventValidationOrigin(f.db,config,central,authority)).rejects.toThrow();current.instanceId="instance";await expect(cloudEventValidationOrigin(f.db,config,undefined,authority)).rejects.toThrow();expect(f.reads()).toBe(0);
|
||||
});
|
||||
it("pending, disabled, missing cutover and replaced instance receipts cannot use the tenant transport",async()=>{
|
||||
for(const patch of [{revision:2},{enabled:false},{remoteResponsibility:false},{deviceCutoverAt:null},{receipt:null}])await expect(cloudEventValidationOrigin(fixture(patch).db,config,central,authority)).rejects.toThrow();
|
||||
current.instanceId="replacement";await expect(cloudEventValidationOrigin(fixture().db,config,central,authority)).rejects.toThrow();current.instanceId="instance";current.enabled=false;await expect(cloudEventValidationOrigin(fixture().db,config,central,authority)).rejects.toThrow();
|
||||
});
|
||||
it("foreign issuer, authorization origin and Dot resource never reuse a personal marker",async()=>{
|
||||
for(const value of [{...config,origin:"https://old.example.test"},{...config,authorizationOrigin:"https://foreign.example.test"},{...config,resource:origin+"/mcp/runner"}])await expect(cloudEventValidationOrigin(fixture().db,value,central,authority)).rejects.toThrow();
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,27 @@
|
||||
import {z} from "zod";
|
||||
import type {Db} from "@paperclipai/db";
|
||||
import {getCloudRuntimeIdentity} from "../cloud-runtime-identity.js";
|
||||
import {paperclipCloudConnectorConfigFromEnv} from "../paperclip-cloud-connector.js";
|
||||
import {instanceSettingsService} from "../instance-settings.js";
|
||||
import {readIdleMcpClientMirror} from "./idle-client-mirror.js";
|
||||
import {McpEventError} from "./event-webhooks.js";
|
||||
import type {PublicMcpConfig} from "./oauth.js";
|
||||
|
||||
export const cloudEventAuthoritySchema=z.object({token:z.string().min(1).max(24000),expiresAt:z.number().finite(),transport:z.literal("tenant-personal-v1").optional()}).strict();
|
||||
export type CloudEventAuthority=z.infer<typeof cloudEventAuthoritySchema>;
|
||||
export function currentTenantEventOrigin(config:PublicMcpConfig){
|
||||
const identity=getCloudRuntimeIdentity();
|
||||
if(!identity||config.origin!==identity.canonicalOrigin||config.resource!==identity.canonicalOrigin+"/mcp/paperclip"||config.authorizationOrigin&&config.authorizationOrigin!==identity.canonicalOrigin)throw new McpEventError(-32602,"The hosted subscription authority changed.");
|
||||
return identity;
|
||||
}
|
||||
export async function cloudEventValidationOrigin(db:Db,config:PublicMcpConfig,centralOrigin:string|undefined,authority:CloudEventAuthority){
|
||||
const parsed=cloudEventAuthoritySchema.safeParse(authority);
|
||||
if(!parsed.success)throw new McpEventError(-32602,"Invalid hosted subscription authority.");
|
||||
if(parsed.data.transport===undefined)return centralOrigin;
|
||||
const identity=currentTenantEventOrigin(config),connector=paperclipCloudConnectorConfigFromEnv();
|
||||
if(!centralOrigin||!connector)throw new McpEventError(-32602,"The hosted subscription authority is unavailable.");
|
||||
const feature=await instanceSettingsService(db).getExperimentalReadOnly();
|
||||
const proof=await readIdleMcpClientMirror(db,{enabled:feature.enablePublicMcp,origin:identity.canonicalOrigin,authorizationOrigin:config.authorizationOrigin,instanceId:connector.instanceId,managedStackId:identity.stackId,budget:{beforeRead:async()=>{},check:()=>{}}});
|
||||
if(proof.blocked)throw new McpEventError(-32602,"The hosted subscription authority is being reconciled.");
|
||||
return identity.canonicalOrigin;
|
||||
}
|
||||
@@ -0,0 +1,11 @@
|
||||
import {eq,sql} from "drizzle-orm";
|
||||
import {mcpOauthDeviceRequests as devices,mcpOauthGrants as grants,type Db} from "@paperclipai/db";
|
||||
import type {CompanyDeletionParticipant} from "../../lib/company-deletion.js";
|
||||
import {prepareMcpClientMutation} from "./client-mirror-mutations.js";
|
||||
|
||||
export const mcpCompanyDeletion:CompanyDeletionParticipant={async deleteCompanyData(runner,companyId){
|
||||
const tx=runner as unknown as Db;
|
||||
await tx.execute(sql`SELECT pg_advisory_xact_lock(736721042)`);
|
||||
const [affected]=await tx.select({id:devices.id}).from(devices).innerJoin(grants,eq(devices.grantId,grants.id)).where(eq(grants.companyId,companyId)).limit(1);
|
||||
if(affected)await prepareMcpClientMutation(tx);
|
||||
}};
|
||||
@@ -0,0 +1,65 @@
|
||||
import {beforeEach,describe,it,expect,vi} from "vitest";
|
||||
import type {Db} from "@paperclipai/db";
|
||||
const current=vi.hoisted(()=>({identity:null as null|{stackId:string;canonicalOrigin:string}}));
|
||||
vi.mock("../cloud-runtime-identity.js",()=>({getCloudRuntimeIdentity:()=>current.identity}));
|
||||
vi.mock("../delivery-work-notifications.js",async importOriginal=>({...await importOriginal<typeof import("../delivery-work-notifications.js")>(),notifyDeliveryWork:vi.fn(async()=>{})}));
|
||||
import {mcpClientMirrorGeneration} from "./client-mirror-contract.js";
|
||||
import {requiresMcpDeviceBroker,currentMcpDeviceCutoverStack,installMcpDeviceCutover} from "./device-cutover.js";
|
||||
|
||||
const resource="https://tenant.example.test/mcp/paperclip";
|
||||
const identity={version:1 as const,instanceId:"instance",origin:"https://tenant.example.test",revision:1};
|
||||
const receipt={...identity,enabled:true,frontDoor:"tenant-personal-v1" as const,deviceStatus:{keyId:"a".repeat(64),generation:"b".repeat(64),count:0},generation:""};receipt.generation=mcpClientMirrorGeneration(receipt);
|
||||
function fixture(row:unknown){let reads=0;const tx={select:()=>{reads++;return{from:()=>({where:async()=>row?[row]:[]})};}} as unknown as Pick<Db,"select">;return{tx,reads:()=>reads};}
|
||||
describe("one-time managed device cutover",()=>{
|
||||
beforeEach(()=>{current.identity=null;});
|
||||
it("persisted managed cutover is inert after legitimate selfhost/no-Cloud startup",async()=>{
|
||||
const f=fixture({stackId:"old-stack",at:new Date(1)});
|
||||
expect(await requiresMcpDeviceBroker(f.tx,resource)).toBe(false);expect(f.reads()).toBe(0);
|
||||
});
|
||||
it("current verified managed identity gates raw mint and keeps original cutover after restart/re-enable",async()=>{
|
||||
current.identity={stackId:"stack",canonicalOrigin:"https://tenant.example.test"};const at=new Date(1),f=fixture({stackId:"stack",at,receipt});
|
||||
expect(await requiresMcpDeviceBroker(f.tx,resource)).toBe(true);expect(await requiresMcpDeviceBroker(f.tx,resource)).toBe(true);expect(at.getTime()).toBe(1);
|
||||
expect(currentMcpDeviceCutoverStack(resource)).toBe("stack");
|
||||
});
|
||||
it("same-stack cutover keeps unsigned mint fenced while re-enable has cleared its receipt",async()=>{
|
||||
current.identity={stackId:"stack",canonicalOrigin:identity.origin};const at=new Date(1),f=fixture({stackId:"stack",at,receipt:null});
|
||||
expect(await requiresMcpDeviceBroker(f.tx,resource)).toBe(true);expect(at.getTime()).toBe(1);
|
||||
expect(await requiresMcpDeviceBroker(f.tx,identity.origin+"/mcp/runner")).toBe(false);
|
||||
});
|
||||
it("a missing receipt cannot transfer the cutover fence across current stack or inspection identity",async()=>{
|
||||
current.identity={stackId:"replacement",canonicalOrigin:identity.origin};
|
||||
expect(await requiresMcpDeviceBroker(fixture({stackId:"stack",at:new Date(1),receipt:null}).tx,resource)).toBe(false);
|
||||
current.identity={stackId:"stack",canonicalOrigin:identity.origin};
|
||||
const tx={select:()=>({from:()=>({where:async()=>{current.identity=null;return[{stackId:"stack",at:new Date(1),receipt:null}];}})})} as unknown as Pick<Db,"select">;
|
||||
expect(await requiresMcpDeviceBroker(tx,resource)).toBe(false);
|
||||
current.identity={stackId:"stack",canonicalOrigin:identity.origin};
|
||||
expect(await requiresMcpDeviceBroker(fixture({stackId:"stack",at:new Date(1),receipt:{}}).tx,resource)).toBe(false);
|
||||
});
|
||||
it("a replacement current managed stack cannot inherit old persisted cutover authority",async()=>{
|
||||
current.identity={stackId:"replacement-stack",canonicalOrigin:"https://tenant.example.test"};
|
||||
expect(await requiresMcpDeviceBroker(fixture({stackId:"old-stack",at:new Date(1),receipt}).tx,resource)).toBe(false);
|
||||
expect(currentMcpDeviceCutoverStack(resource)).toBe("replacement-stack");
|
||||
});
|
||||
it("never-cut-over and other resource/origin paths preserve their original owner",async()=>{
|
||||
current.identity={stackId:"stack",canonicalOrigin:"https://tenant.example.test"};const f=fixture({stackId:null,at:null});
|
||||
expect(await requiresMcpDeviceBroker(f.tx,resource)).toBe(false);
|
||||
expect(await requiresMcpDeviceBroker(f.tx,"https://tenant.example.test/mcp/runner")).toBe(false);
|
||||
expect(await requiresMcpDeviceBroker(f.tx,"https://old.example.test/mcp/paperclip")).toBe(false);expect(f.reads()).toBe(1);
|
||||
});
|
||||
it("same-stack origin replacement and identity rotation during inspection cannot inherit cutover",async()=>{
|
||||
current.identity={stackId:"stack",canonicalOrigin:"https://replacement.example.test"};
|
||||
expect(await requiresMcpDeviceBroker(fixture({stackId:"stack",at:new Date(1),receipt}).tx,"https://replacement.example.test/mcp/paperclip")).toBe(false);
|
||||
current.identity={stackId:"stack",canonicalOrigin:identity.origin};
|
||||
const tx={select:()=>({from:()=>({where:async()=>{current.identity={stackId:"replacement",canonicalOrigin:identity.origin};return[{stackId:"stack",at:new Date(1),receipt}];}})})} as unknown as Pick<Db,"select">;
|
||||
expect(await requiresMcpDeviceBroker(tx,resource)).toBe(false);
|
||||
});
|
||||
|
||||
it("only an acknowledged current namespace refreshes cutover owner while retaining original time",async()=>{
|
||||
current.identity={stackId:"replacement",canonicalOrigin:identity.origin};const at=new Date(1),updates:unknown[]=[];
|
||||
const tx={select:()=>({from:()=>({where:()=>({for:async()=>[{revision:1,deviceCutoverStackId:"old-stack",deviceCutoverAt:at}]})})}),update:()=>({set:(value:unknown)=>{updates.push(value);return{where:async()=>[]};}})} as unknown as Db;
|
||||
await installMcpDeviceCutover(tx,receipt);expect(updates).toEqual([{deviceCutoverStackId:"replacement",deviceCutoverAt:at}]);
|
||||
updates.length=0;await installMcpDeviceCutover(tx,{...receipt,generation:"0".repeat(64)});expect(updates).toEqual([]);
|
||||
await installMcpDeviceCutover(tx,{...receipt,origin:"https://foreign.example.test"});expect(updates).toEqual([]);
|
||||
});
|
||||
|
||||
});
|
||||
@@ -0,0 +1,31 @@
|
||||
import { DELIVERY_QUEUES,notifyDeliveryWork } from "../delivery-work-notifications.js";
|
||||
import { eq,sql } from "drizzle-orm";
|
||||
import { mcpOauthClientMirrorState as state,type Db } from "@paperclipai/db";
|
||||
import { getCloudRuntimeIdentity } from "../cloud-runtime-identity.js";
|
||||
import { mcpClientMirrorGeneration,mcpClientMirrorReceiptSchema,type McpClientMirrorReceipt } from "./client-mirror-contract.js";
|
||||
|
||||
export function currentMcpDeviceCutoverStack(resource:string){
|
||||
const identity=getCloudRuntimeIdentity();return identity&&resource===identity.canonicalOrigin+"/mcp/paperclip"?identity.stackId:undefined;
|
||||
}
|
||||
export async function installMcpDeviceCutover(tx:Db,receipt:McpClientMirrorReceipt){
|
||||
if(receipt.generation!==mcpClientMirrorGeneration(receipt)||!receipt.enabled||!receipt.deviceStatus||receipt.frontDoor!=="tenant-personal-v1"||receipt.authorizationOrigin&&receipt.authorizationOrigin!==receipt.origin)return;
|
||||
const stackId=currentMcpDeviceCutoverStack(receipt.origin+"/mcp/paperclip");if(!stackId)return;
|
||||
const [row]=await tx.select().from(state).where(eq(state.id,"public")).for("update");
|
||||
if(!row||row.revision!==receipt.revision||currentMcpDeviceCutoverStack(receipt.origin+"/mcp/paperclip")!==stackId
|
||||
||row.deviceCutoverAt!==null&&row.deviceCutoverStackId===stackId)return;
|
||||
await notifyDeliveryWork(tx,DELIVERY_QUEUES.mcpIngressHandoffs);
|
||||
await tx.update(state).set({deviceCutoverStackId:stackId,deviceCutoverAt:row.deviceCutoverAt??sql`clock_timestamp()`}).where(eq(state.id,"public"));
|
||||
}
|
||||
export async function requiresMcpDeviceBroker(tx:Pick<Db,"select">,resource:string){
|
||||
const stackId=currentMcpDeviceCutoverStack(resource);if(!stackId)return false;
|
||||
const[row]=await tx.select({stackId:state.deviceCutoverStackId,at:state.deviceCutoverAt,receipt:state.receipt}).from(state).where(eq(state.id,"public"));
|
||||
if(row?.stackId!==stackId||!(row.at instanceof Date)||!Number.isFinite(row.at.getTime())||currentMcpDeviceCutoverStack(resource)!==stackId)return false;
|
||||
if(row.receipt===null)return true;
|
||||
const receipt=mcpClientMirrorReceiptSchema.safeParse(row.receipt);
|
||||
return receipt.success&&receipt.data.origin+"/mcp/paperclip"===resource
|
||||
&&receipt.data.generation===mcpClientMirrorGeneration(receipt.data);
|
||||
}
|
||||
|
||||
export function managedMcpDeviceScope(){
|
||||
const identity=getCloudRuntimeIdentity();return identity?{stackId:identity.stackId,resource:identity.canonicalOrigin+"/mcp/paperclip"}:undefined;
|
||||
}
|
||||
@@ -0,0 +1,23 @@
|
||||
import { unauthorized } from "../../errors.js";
|
||||
import type { Request } from "express";
|
||||
import { CLOUD_CONTROL_HEADER,verifyCloudControlAssertion,getCloudRuntimeIdentity } from "../cloud-runtime-identity.js";
|
||||
|
||||
export interface McpDeviceMintContext {readonly resource:string;readonly stackId:string;readonly expiresAt:number}
|
||||
const verified=new WeakSet<object>();
|
||||
export function verifyMcpDeviceMintContext(req:Pick<Request,"get">,resource:string):McpDeviceMintContext|undefined{
|
||||
const compactJws=req.get(CLOUD_CONTROL_HEADER)?.trim();if(!compactJws)return;
|
||||
if(new URL(resource).pathname!=="/mcp/paperclip")throw unauthorized("MCP device mint context has another resource");
|
||||
let claims:ReturnType<typeof verifyCloudControlAssertion>;
|
||||
try{claims=verifyCloudControlAssertion({compactJws,expectedAction:"mcp-device:mint"});}catch{throw unauthorized("Invalid MCP device mint assertion");}
|
||||
const context=Object.freeze({resource,stackId:claims.sub,expiresAt:claims.exp*1000});verified.add(context);return context;
|
||||
}
|
||||
export function consumeMcpDeviceMintContext(context:McpDeviceMintContext|undefined,resource:string){
|
||||
if(!context)return;
|
||||
if(context.resource!==resource||!verified.delete(context))throw unauthorized("MCP device mint context is unavailable");
|
||||
return context;
|
||||
}
|
||||
|
||||
export function assertCurrentMcpDeviceMint(context:McpDeviceMintContext,resource:string){
|
||||
const identity=getCloudRuntimeIdentity();
|
||||
if(!identity||context.stackId!==identity.stackId||resource!==identity.canonicalOrigin+"/mcp/paperclip"||context.expiresAt<=Date.now())throw unauthorized("MCP device mint identity changed");
|
||||
}
|
||||
@@ -0,0 +1,22 @@
|
||||
import {createHash,createHmac} from "node:crypto";
|
||||
import {z} from "zod";
|
||||
|
||||
const digest=z.string().regex(/^[a-f0-9]{64}$/);
|
||||
export const mcpDeviceStatusEntrySchema=z.object({requestId:z.uuid(),fingerprint:digest,clientId:z.string().min(1).max(2048),resource:z.string().url().max(2048),ingressOwner:z.enum(["tenant","cloud"]),status:z.enum(["pending","approved","denied","consumed"]),expiresAt:z.number().int().min(0).max(Number.MAX_SAFE_INTEGER)}).strict();
|
||||
export const mcpDeviceStatusReferenceSchema=z.object({keyId:digest,generation:digest,count:z.number().int().min(0).max(1000)}).strict();
|
||||
export type McpDeviceStatusEntry=z.infer<typeof mcpDeviceStatusEntrySchema>;
|
||||
export type McpDeviceStatusReference=z.infer<typeof mcpDeviceStatusReferenceSchema>;
|
||||
export function mcpDeviceStatusKeyId(key:string){
|
||||
if(Buffer.from(key,"base64url").length!==32)throw new Error("Invalid MCP device status key");
|
||||
return createHash("sha256").update("mcp-device-status-key-v1:"+key).digest("hex");
|
||||
}
|
||||
export function mcpDeviceStatusFingerprint(key:string,hash:string,clientId:string,resource:string){
|
||||
if(!/^[a-f0-9]{64}$/.test(hash))throw new Error("Invalid MCP device status credential hash");mcpDeviceStatusKeyId(key);
|
||||
return createHmac("sha256",Buffer.from(key,"base64url")).update(JSON.stringify(["mcp-device-status-v1",hash,clientId,resource])).digest("hex");
|
||||
}
|
||||
export function mcpDeviceStatusGeneration(input:{instanceId:string;origin:string;revision:number;keyId:string},entries:McpDeviceStatusEntry[]){
|
||||
if(entries.length>1000)throw new Error("MCP device status set exceeds canonical1000 admission bound");
|
||||
const parsed=entries.map(entry=>mcpDeviceStatusEntrySchema.parse(entry)).sort((a,b)=>Buffer.compare(Buffer.from(a.requestId),Buffer.from(b.requestId)));
|
||||
if(new Set(parsed.map(entry=>entry.requestId)).size!==parsed.length||new Set(parsed.map(entry=>entry.fingerprint)).size!==parsed.length||parsed.some(entry=>entry.resource!==input.origin+"/mcp/paperclip"))throw new Error("MCP device status identity changed");
|
||||
return createHash("sha256").update(JSON.stringify(["mcp-device-status-snapshot-v1",input,parsed])).digest("hex");
|
||||
}
|
||||
@@ -0,0 +1,23 @@
|
||||
import {describe,it,expect} from "vitest";
|
||||
import {mkdtempSync,chmodSync,writeFileSync,statSync,readdirSync,rmSync} from "node:fs";
|
||||
import os from "node:os";
|
||||
import path from "node:path";
|
||||
import {deriveLocalEncryptedInstanceSecret} from "../../secrets/local-encrypted-provider.js";
|
||||
|
||||
function fixture(){const directory=mkdtempSync(path.join(os.tmpdir(),"mcp-status-readonly-key-")),file=path.join(directory,"master.key");return{directory,file,env:{PAPERCLIP_SECRETS_MASTER_KEY_FILE:file},cleanup:()=>rmSync(directory,{recursive:true,force:true})};}
|
||||
describe("device status inspection key access",()=>{
|
||||
it("missing key is refused without creation even if allowCreate is also supplied",()=>{
|
||||
const f=fixture();try{expect(()=>deriveLocalEncryptedInstanceSecret("mcp-device-status:v1:instance",f.env,{readOnly:true,allowCreate:true})).toThrow("missing");expect(readdirSync(f.directory)).toEqual([]);}finally{f.cleanup();}
|
||||
});
|
||||
it("unsafe file permissions are refused without chmod",()=>{
|
||||
const f=fixture();try{writeFileSync(f.file,Buffer.alloc(32,7).toString("base64"));chmodSync(f.file,0o644);expect(()=>deriveLocalEncryptedInstanceSecret("mcp-device-status:v1:instance",f.env,{readOnly:true})).toThrow("permissions are unsafe");expect(statSync(f.file).mode&0o777).toBe(0o644);}finally{f.cleanup();}
|
||||
});
|
||||
it("safe key reads and environment keys are pure",()=>{
|
||||
const f=fixture();try{writeFileSync(f.file,Buffer.alloc(32,7).toString("base64"));chmodSync(f.file,0o600);const before=statSync(f.file);expect(deriveLocalEncryptedInstanceSecret("mcp-device-status:v1:instance",f.env,{readOnly:true})).toHaveLength(43);const after=statSync(f.file);expect(after.mode).toBe(before.mode);expect(after.mtimeMs).toBe(before.mtimeMs);
|
||||
const missing=path.join(f.directory,"missing.key");expect(deriveLocalEncryptedInstanceSecret("mcp-device-status:v1:instance",{PAPERCLIP_SECRETS_MASTER_KEY:Buffer.alloc(32,8).toString("base64"),PAPERCLIP_SECRETS_MASTER_KEY_FILE:missing},{readOnly:true})).toHaveLength(43);expect(readdirSync(f.directory)).toEqual(["master.key"]);
|
||||
}finally{f.cleanup();}
|
||||
});
|
||||
it("ordinary read/derive still repairs unsafe permissions",()=>{
|
||||
const f=fixture();try{writeFileSync(f.file,Buffer.alloc(32,7).toString("base64"));chmodSync(f.file,0o644);expect(deriveLocalEncryptedInstanceSecret("ordinary-owner",f.env,{allowCreate:false})).toHaveLength(43);expect(statSync(f.file).mode&0o777).toBe(0o600);}finally{f.cleanup();}
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,25 @@
|
||||
import { describe, expect, it } from "vitest";
|
||||
import { mcpDiscoveryDocuments, parseMcpDiscovery } from "./discovery.js";
|
||||
import { createMcpWakeAdmission, parseMcpWakeAdmission } from "./wake-admission.js";
|
||||
import { DOT_RUNNER_MCP_SCOPES, PUBLIC_MCP_SCOPES } from "@paperclipai/shared";
|
||||
const snapshot = { origin: "https://tenant.example.test", authorizationOrigin: "https://browser.example.test", profiles: ["personal-mcp","dot-mcp"] as ("personal-mcp"|"dot-mcp")[] };
|
||||
describe("MCP owner discovery snapshot",()=>{
|
||||
it("preserves dedicated issuers, real scope contracts and configured browser authorization origin",()=>{
|
||||
const documents = mcpDiscoveryDocuments(snapshot);
|
||||
expect(documents).toHaveLength(5);
|
||||
const personal = documents.find(entry=>entry.path==="/.well-known/oauth-authorization-server")!.document;
|
||||
expect(personal).toMatchObject({issuer:snapshot.origin,authorization_endpoint:snapshot.authorizationOrigin+"/mcp/oauth/authorize",token_endpoint:snapshot.origin+"/mcp/oauth/token",scopes_supported:[...PUBLIC_MCP_SCOPES],client_id_metadata_document_supported:true});
|
||||
const dot = documents.find(entry=>entry.path==="/.well-known/oauth-authorization-server/mcp/runner/oauth")!.document;
|
||||
expect(dot).toMatchObject({issuer:snapshot.origin+"/mcp/runner/oauth",authorization_endpoint:snapshot.authorizationOrigin+"/mcp/runner/oauth/authorize",scopes_supported:[...DOT_RUNNER_MCP_SCOPES]});
|
||||
expect(documents.find(entry=>entry.path==="/.well-known/oauth-protected-resource/mcp/paperclip")!.document.scopes_supported).toEqual(["paperclip:read","paperclip:write","paperclip:configure"]);
|
||||
expect(mcpDiscoveryDocuments({...snapshot,profiles:["personal-mcp"]}).some(entry=>entry.path.includes("runner"))).toBe(false);
|
||||
});
|
||||
it("binds public snapshot changes to the private cycle generation and rejects unfamiliar shapes",()=>{
|
||||
const input = { stackId:"stack",ownerId:"00000000-0000-4000-8000-000000000001",capabilityGeneration:"a".repeat(64),challenge:"c".repeat(64),tokens:[] };
|
||||
const old = createMcpWakeAdmission(input), current = createMcpWakeAdmission({...input,discovery:snapshot});
|
||||
expect(parseMcpWakeAdmission(old)).toEqual(old);
|
||||
expect(current.generation).not.toBe(old.generation);
|
||||
expect(()=>parseMcpWakeAdmission({...current,discovery:{...snapshot,authorizationOrigin:"https://changed.example.test"}})).toThrow();
|
||||
for(const change of [{profiles:["muse"]},{profiles:["dot-mcp","personal-mcp"]},{origin:"https://tenant.example.test/path"},{enabled:true},{profiles:[]}])expect(()=>parseMcpDiscovery({...snapshot,...change})).toThrow();
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,39 @@
|
||||
export type McpDiscoveryProfile = "personal-mcp" | "dot-mcp";
|
||||
export interface McpDiscovery {
|
||||
origin: string;
|
||||
authorizationOrigin: string;
|
||||
profiles: McpDiscoveryProfile[];
|
||||
}
|
||||
const profiles = ["personal-mcp","dot-mcp"] as const;
|
||||
function origin(value: unknown): string {
|
||||
if (typeof value !== "string" || value.length > 2048) throw new Error("Invalid MCP discovery origin");
|
||||
const url = new URL(value);
|
||||
if (url.origin !== value || url.protocol !== "https:" && !(url.protocol === "http:" && ["localhost","127.0.0.1","[::1]"].includes(url.hostname))) throw new Error("Invalid MCP discovery origin");
|
||||
return value;
|
||||
}
|
||||
export function parseMcpDiscovery(value: unknown): McpDiscovery {
|
||||
if (!value || typeof value !== "object" || Array.isArray(value)) throw new Error("Invalid MCP discovery snapshot");
|
||||
const row = value as Record<string,unknown>;
|
||||
const selected=row.profiles;
|
||||
if (Object.keys(row).sort().join() !== "authorizationOrigin,origin,profiles" || !Array.isArray(selected) || !selected.length || selected.length > 2
|
||||
|| selected.some((profile,index) => !profiles.includes(profile) || index > 0 && profiles.indexOf(profile) <= profiles.indexOf(selected[index-1]))) throw new Error("Invalid MCP discovery profiles");
|
||||
return { origin: origin(row.origin), authorizationOrigin: origin(row.authorizationOrigin), profiles: selected as McpDiscoveryProfile[] };
|
||||
}
|
||||
export function mcpDiscoveryDocuments(raw: McpDiscovery): Array<{ path: string; document: Record<string,unknown> }> {
|
||||
const input = parseMcpDiscovery(raw), documents: Array<{ path: string; document: Record<string,unknown> }> = [];
|
||||
for (const profile of input.profiles) {
|
||||
const dot = profile === "dot-mcp", endpoint = dot ? "/mcp/runner" : "/mcp/paperclip", oauthPath = dot ? "/mcp/runner/oauth" : "/mcp/oauth";
|
||||
const issuer = dot ? input.origin + oauthPath : input.origin, prefix = input.origin + oauthPath;
|
||||
const scopes = dot ? ["paperclip:agent","offline_access"] : ["paperclip:read","paperclip:write","paperclip:configure","offline_access"];
|
||||
const resource = { resource: input.origin + endpoint, authorization_servers: [issuer], scopes_supported: dot ? scopes : scopes.slice(0,3), bearer_methods_supported: ["header"], resource_name: dot ? "Paperclip agent" : "Paperclip" };
|
||||
if (!dot) documents.push({ path: "/.well-known/oauth-protected-resource", document: resource });
|
||||
documents.push({ path: "/.well-known/oauth-protected-resource" + endpoint, document: resource });
|
||||
documents.push({ path: "/.well-known/oauth-authorization-server" + (dot ? oauthPath : ""), document: {
|
||||
issuer, authorization_endpoint: input.authorizationOrigin + oauthPath + "/authorize", token_endpoint: prefix + "/token", registration_endpoint: prefix + "/register", revocation_endpoint: prefix + "/revoke",
|
||||
response_types_supported: ["code"], grant_types_supported: ["authorization_code","refresh_token","urn:ietf:params:oauth:grant-type:device_code"], device_authorization_endpoint: prefix + "/device_authorization",
|
||||
token_endpoint_auth_methods_supported: ["none"], revocation_endpoint_auth_methods_supported: ["none"], code_challenge_methods_supported: ["S256"], scopes_supported: scopes,
|
||||
authorization_response_iss_parameter_supported: true, client_id_metadata_document_supported: true,
|
||||
} });
|
||||
}
|
||||
return documents;
|
||||
}
|
||||
@@ -1,18 +1,41 @@
|
||||
import { randomBytes, randomUUID } from "node:crypto";
|
||||
import { createHash, randomBytes, randomUUID } from "node:crypto";
|
||||
import { mkdtemp, rm } from "node:fs/promises";
|
||||
import { tmpdir } from "node:os";
|
||||
import path from "node:path";
|
||||
import express, { type Request } from "express";
|
||||
import request from "supertest";
|
||||
import { afterAll, afterEach, beforeAll, beforeEach, describe, expect, it, vi } from "vitest";
|
||||
import { eq, sql } from "drizzle-orm";
|
||||
import { applyPendingMigrations, createDb, companies, authUsers, companyMemberships, mcpOauthClients, mcpOauthGrants, issues, agents, dotAgentBindings, dotMailboxItems, dotRunnerAssignments, heartbeatRuns, mcpEventDeliveries, mcpEventSubscriptions } from "@paperclipai/db";
|
||||
import { applyPendingMigrations, createDb, companies, authUsers, companyMemberships, instanceSettings, mcpOauthClients, mcpOauthGrants, mcpOauthRequests, mcpOauthDeviceRequests, issues, agents, dotAgentBindings, dotMailboxItems, dotRunnerAssignments, heartbeatRuns, mcpEventDeliveries, mcpEventSubscriptions, mcpOauthTokens, mcpFileTickets } from "@paperclipai/db";
|
||||
import { resolveExternalTestDatabaseUrl } from "../../__tests__/helpers/external-test-database.js";
|
||||
import { getEmbeddedPostgresTestSupport, startEmbeddedPostgresTestDatabase } from "../../__tests__/helpers/embedded-postgres.js";
|
||||
import { createPublicMcpEvents } from "./events.js";
|
||||
import { createPublicMcpOAuth } from "./oauth.js";
|
||||
import { createPublicMcpOAuth, DEVICE_GRANT } from "./oauth.js";
|
||||
import { instanceSettingsService } from "../instance-settings.js";
|
||||
import { createDeliveryWorkCoordinator } from "../delivery-work-coordinator.js";
|
||||
import { logActivity } from "../activity-log.js";
|
||||
import { insertIssueComment } from "../issue-comment-writes.js";
|
||||
import { dotRunnerBroker } from "../dot-runner-broker.js";
|
||||
import { readIdleMcpWakeProof } from "./idle-wake-proof.js";
|
||||
import { readIdleMcpSubscriptionProof } from "./idle-subscription-proof.js";
|
||||
import { idleMcpWorkQuery } from "./idle-work-query.js";
|
||||
import { startMcpRevocationIndexSync } from "./revocation-index-sync.js";
|
||||
import { parseMcpRevocationIndexRequest } from "./revocation-index-contract.js";
|
||||
import { readIdleMcpSleepProof } from "./idle-sleep-proof.js";
|
||||
import { registerIdleMcpOwners } from "./idle-sleep-proof.js";
|
||||
import { createPublicMcpTransfers } from "./file-transfers.js";
|
||||
import { startMcpIngressHandoffExpiry } from "./ingress-handoff-expiry.js";
|
||||
import { readIdleMcpAuthorizationHandoffs } from "./idle-authorization-handoffs.js";
|
||||
import { createStorageService } from "../../storage/service.js";
|
||||
import { createLocalDiskStorageProvider } from "../../storage/local-disk-provider.js";
|
||||
import { idleMcpHistoryChecks } from "./idle-history-proof.js";
|
||||
import { idleWorkSnapshot } from "../task-admission.js";
|
||||
|
||||
vi.mock("../cloud-runtime-identity.js", async importOriginal => ({
|
||||
...await importOriginal<typeof import("../cloud-runtime-identity.js")>(),
|
||||
getCloudRuntimeIdentity: () => ({ stackId: "fixture-stack", canonicalOrigin: "https://paperclip.example" }),
|
||||
}));
|
||||
|
||||
const externalUrl = resolveExternalTestDatabaseUrl("PAPERCLIP_MCP_SLEEP_TEST_DATABASE_URL");
|
||||
const support = externalUrl ? { supported: true } : await getEmbeddedPostgresTestSupport();
|
||||
|
||||
@@ -31,7 +54,7 @@ describe.skipIf(!support.supported)("MCP event committed work", () => {
|
||||
vi.stubEnv("PAPERCLIP_SECRETS_MASTER_KEY", randomBytes(32).toString("base64"));
|
||||
if (externalUrl) await applyPendingMigrations(externalUrl);
|
||||
else temporary = await startEmbeddedPostgresTestDatabase("mcp-sleep-");
|
||||
db = createDb(externalUrl ?? temporary.connectionString, { maxConnections: 6 });
|
||||
db = createDb(externalUrl ?? temporary.connectionString, { maxConnections: 4 });
|
||||
}, 90000);
|
||||
beforeEach(async () => {
|
||||
await db.execute(sql`TRUNCATE TABLE companies CASCADE`);
|
||||
@@ -94,7 +117,7 @@ describe.skipIf(!support.supported)("MCP event committed work", () => {
|
||||
await settle();
|
||||
};
|
||||
const restart = async () => { await service.stop(); await coordinator.stop(); service = makeService(); await start(); };
|
||||
return { input, principal, start, restart, bindingId: bindingId!, agentId: agentId!, subscribe: () => service.subscribe(principal, input) };
|
||||
return { input, principal, oauth, observed, start, restart, bindingId: bindingId!, agentId: agentId!, subscribe: () => service.subscribe(principal, input) };
|
||||
}
|
||||
async function settle() {
|
||||
await vi.advanceTimersByTimeAsync(0);
|
||||
@@ -112,15 +135,229 @@ describe.skipIf(!support.supported)("MCP event committed work", () => {
|
||||
}
|
||||
async function delivered(count: number) { await vi.waitFor(() => expect(received).toHaveLength(count), { timeout: 8_000 }); await settle(); }
|
||||
|
||||
it("inspects missing settings and absent MCP owners in the real read-only safety transaction", async () => {
|
||||
await db.delete(instanceSettings);
|
||||
const network = vi.spyOn(globalThis, "fetch").mockRejectedValue(new Error("inspection must not call a provider"));
|
||||
try {
|
||||
const proof = await db.transaction(async tx => readIdleMcpSleepProof(tx, {}, {
|
||||
stackId: "fixture-stack", ownerId: "00000000-0000-4000-8000-000000000001",
|
||||
capabilityGeneration: "a".repeat(64), challenge: "c".repeat(64), now: Date.now(),
|
||||
budget: { beforeRead: async () => { await tx.execute(sql`SET LOCAL statement_timeout = '2s'`); }, check: () => {} },
|
||||
}), { isolationLevel: "repeatable read", accessMode: "read only" });
|
||||
expect(proof.ingress.grantIds).toEqual([]);
|
||||
expect(proof.subscriptions.blocked).toBe(false);
|
||||
expect(proof.retries).toMatchObject({ blocked: false, deadlines: [] });
|
||||
expect(await db.select().from(instanceSettings)).toEqual([]);
|
||||
expect(network).not.toHaveBeenCalled();
|
||||
} finally { network.mockRestore(); }
|
||||
});
|
||||
|
||||
it("is quiet empty, subscribed without events, and with delivered history", async () => {
|
||||
const f = await fixture(); await f.start(); await quiet();
|
||||
expect(service.reconciliationSnapshot().reconciled).toBe(true);
|
||||
vi.setSystemTime(vi.getRealSystemTime());
|
||||
await f.subscribe(); await settle(); await quiet();
|
||||
const before = service.reconciliationSnapshot();
|
||||
await activity();
|
||||
expect(service.reconciliationSnapshot().reconciled).toBe(false);
|
||||
await delivered(1); await quiet(600_000);
|
||||
expect(service.reconciliationSnapshot().reconciled).toBe(true);
|
||||
expect(service.reconciliationSnapshot().revision).not.toBe(before.revision);
|
||||
expect(coordinator.nextWakeAt()).toBeNull(); expect(errors).toEqual([]);
|
||||
});
|
||||
|
||||
it("certifies an enabled settled subscription only with its exact ingress and producer proof", async () => {
|
||||
const f = await fixture(); await f.subscribe(); await f.start();
|
||||
await db.insert(mcpOauthTokens).values({ grantId: f.principal.grant.id,
|
||||
tokenHash: createHash("sha256").update("pcmcp_at_" + "a".repeat(43)).digest("hex"),
|
||||
kind: "access", expiresAt: new Date(Date.now() + 900_000) });
|
||||
|
||||
|
||||
await db.insert(mcpOauthTokens).values(Array.from({ length: 1005 }, (_, index) => ({
|
||||
grantId: f.principal.grant.id, tokenHash: createHash("sha256").update(`rotated-${index}`).digest("hex"),
|
||||
kind: "refresh" as const, usedAt: new Date(), expiresAt: null,
|
||||
})));
|
||||
const publications: Array<{ operation: string; entries?: unknown[] }> = [];
|
||||
const indexed = new Set<string>();
|
||||
let remoteRevision = -1;
|
||||
const index = startMcpRevocationIndexSync(f.observed, coordinator, { key: Buffer.alloc(32,9).toString("base64url"), client: {
|
||||
publish: async raw => {
|
||||
const request = parseMcpRevocationIndexRequest(raw); publications.push(request);
|
||||
if (request.operation === "append") for (const entry of request.entries) indexed.add(entry.fingerprint);
|
||||
if (request.operation === "complete" || request.revision === remoteRevision) {
|
||||
remoteRevision = request.revision;
|
||||
return { complete: true, receipt: { version: 1, authorityId: request.authorityId, keyId: request.keyId,
|
||||
revision: remoteRevision, generation: createHash("sha256").update(JSON.stringify([...indexed].sort())).digest("hex") } };
|
||||
}
|
||||
return { complete: false, baseRevision: remoteRevision };
|
||||
},
|
||||
} });
|
||||
await index.ready; await settle();
|
||||
expect(indexed.size).toBe(1006);
|
||||
expect(publications.filter(row => row.operation === "append").map(row => row.entries!.length)).toEqual([200,200,200,200,200,6]);
|
||||
await activity(); await delivered(1); await quiet();
|
||||
const ingress = await readIdleMcpWakeProof(db, { stackId: "fixture-stack", ownerId: "00000000-0000-4000-8000-000000000001",
|
||||
capabilityGeneration: "a".repeat(64), challenge: "c".repeat(64), personalResource: f.principal.grant.resource,
|
||||
dotResource: "https://paperclip.example/mcp/runner", personalEnabled: true, dotEnabled: true, now: Date.now() });
|
||||
expect(ingress.grantIds).toEqual([f.principal.grant.id]);
|
||||
const proof = await readIdleMcpSubscriptionProof(db, { now: Date.now(), ingress,
|
||||
personal: service.reconciliationSnapshot(), dot: { revision: "absent", reconciled: false }, inspectDestination: service.inspectIdleDestination });
|
||||
expect(proof.blocked).toBe(false);
|
||||
for (const reason of ["mcp_oauth_grants", "mcp_event_subscriptions"] as const) {
|
||||
const baseline = idleMcpHistoryChecks.find(check => check.reason === reason)!;
|
||||
expect((await db.execute(sql.raw(baseline.query))).length).toBe(1);
|
||||
const query = idleMcpWorkQuery(reason, { ingress, subscriptions: proof });
|
||||
expect(query).toBeDefined();
|
||||
expect((await db.execute(query!)).length).toBe(0);
|
||||
expect(idleMcpWorkQuery(reason, {})).toBeUndefined();
|
||||
}
|
||||
expect((await readIdleMcpSubscriptionProof(db, { now: Date.now(), ingress,
|
||||
personal: { ...service.reconciliationSnapshot(), reconciled: false }, dot: { revision: "absent", reconciled: false },
|
||||
inspectDestination: service.inspectIdleDestination })).blocked).toBe(true);
|
||||
const [subscription] = await db.select().from(mcpEventSubscriptions);
|
||||
expect(subscription.stoppedAt).toBeNull();
|
||||
expect((await db.select().from(mcpEventDeliveries))[0].outcome).toBe("delivered");
|
||||
});
|
||||
|
||||
it("publishes real OAuth rotation only after commit and indexes only the new token revision", async () => {
|
||||
const f = await fixture(); await f.start();
|
||||
await db.update(mcpOauthGrants).set({ scopes: ["paperclip:read", "offline_access"] }).where(eq(mcpOauthGrants.id, f.principal.grant.id));
|
||||
const refresh = "pcmcp_rt_" + randomBytes(32).toString("base64url");
|
||||
await db.insert(mcpOauthTokens).values({ grantId: f.principal.grant.id, kind: "refresh",
|
||||
tokenHash: createHash("sha256").update(refresh).digest("hex"), expiresAt: new Date(Date.now() + 30 * 24 * 60 * 60_000) });
|
||||
let publishedRevision = -1;
|
||||
const pages: number[] = [], publications: string[] = [];
|
||||
const index = startMcpRevocationIndexSync(f.observed, coordinator, { key: Buffer.alloc(32,9).toString("base64url"), client: {
|
||||
publish: async raw => {
|
||||
const request = parseMcpRevocationIndexRequest(raw); publications.push(request.operation);
|
||||
if (request.operation === "append") pages.push(request.entries.length);
|
||||
if (request.operation === "complete" || request.revision === publishedRevision) {
|
||||
publishedRevision = request.revision;
|
||||
return { complete: true, receipt: { version: 1, authorityId: request.authorityId, keyId: request.keyId,
|
||||
revision: publishedRevision, generation: createHash("sha256").update(String(publishedRevision)).digest("hex") } };
|
||||
}
|
||||
return { complete: false, baseRevision: publishedRevision };
|
||||
},
|
||||
} });
|
||||
await index.ready; await settle(); expect(pages).toEqual([1]);
|
||||
const request = { grant_type: "refresh_token", client_id: f.principal.grant.clientId, resource: f.principal.grant.resource, refresh_token: refresh };
|
||||
const before = [...publications];
|
||||
await expect(db.transaction(async tx => {
|
||||
await createPublicMcpOAuth(tx as unknown as typeof db, f.oauth.config).token(request);
|
||||
throw new Error("rollback rotation");
|
||||
})).rejects.toThrow("rollback rotation");
|
||||
await settle(); await quiet();
|
||||
expect(publications).toEqual(before);
|
||||
expect((await db.select().from(mcpOauthTokens))).toHaveLength(1);
|
||||
expect((await db.select().from(mcpOauthGrants))[0].revocationRevision).toBe(0);
|
||||
expect((await db.select().from(mcpOauthTokens))[0].expiresAt!.getTime()).toBeGreaterThan(Date.now());
|
||||
const rotated = await f.oauth.token(request); expect(rotated.refresh_token).toMatch(/^pcmcp_rt_/);
|
||||
await vi.waitFor(async () => expect((await db.select().from(mcpOauthGrants))[0].revocationIndexReceipt?.revision).toBe(1));
|
||||
expect(pages).toEqual([1,2]);
|
||||
const rows = await db.select().from(mcpOauthTokens);
|
||||
expect(rows.filter(row => row.revocationRevision === 1)).toHaveLength(2);
|
||||
expect(rows.find(row => row.tokenHash === createHash("sha256").update(refresh).digest("hex"))?.usedAt).not.toBeNull();
|
||||
await quiet(); expect(errors).toEqual([]);
|
||||
});
|
||||
|
||||
it("keeps real minted reusable file tickets awake through upload/download and becomes quiet after expiry", async () => {
|
||||
const f = await fixture(); await f.start();
|
||||
await db.update(mcpOauthGrants).set({ scopes: ["paperclip:read", "paperclip:write"] }).where(eq(mcpOauthGrants.id,f.principal.grant.id));
|
||||
const principal = await f.oauth.authorizeGrant(f.principal.grant.id);
|
||||
const index = startMcpRevocationIndexSync(f.observed,coordinator,{ key: Buffer.alloc(32,9).toString("base64url"), client: {
|
||||
publish: async value => {
|
||||
const row = parseMcpRevocationIndexRequest(value);
|
||||
return row.operation === "complete" ? { complete: true, receipt: { version: 1, authorityId: row.authorityId, keyId: row.keyId, revision: row.revision, generation: "d".repeat(64) } }
|
||||
: { complete: false, baseRevision: -1 };
|
||||
},
|
||||
} });
|
||||
await index.ready; await startMcpIngressHandoffExpiry(f.observed,coordinator).ready; await settle(); await quiet();
|
||||
const directory = await mkdtemp(path.join(tmpdir(),"paperclip-mcp-ticket-test-"));
|
||||
const unregister = registerIdleMcpOwners(db,{ config: f.oauth.config, personal: service, dot: null });
|
||||
try {
|
||||
const storage = createStorageService(createLocalDiskStorageProvider(directory));
|
||||
const transfers = createPublicMcpTransfers(f.observed,f.oauth,async (_principal,method,url) => {
|
||||
expect(method).toBe("GET"); expect(url).toBe(`/issues/${taskId}`);
|
||||
return (await db.select().from(issues).where(eq(issues.id,taskId)))[0];
|
||||
},storage);
|
||||
const app = express(); app.use(transfers.router);
|
||||
const bytes = Buffer.from("real ticket lifecycle");
|
||||
const uploaded = await transfers.getUploadUrl(principal,{ companyId, taskId, requestId: randomUUID(), filename: "ticket.txt", contentType: "text/plain",
|
||||
byteSize: bytes.length, sha256: createHash("sha256").update(bytes).digest("hex") });
|
||||
expect("url" in uploaded).toBe(true); if (!("url" in uploaded) || typeof uploaded.url !== "string") throw new Error("missing minted upload ticket");
|
||||
const budget = { beforeRead: async () => {}, check: () => {} };
|
||||
const inspect = () => readIdleMcpSleepProof(db,db,{ stackId: "fixture-stack", ownerId: "00000000-0000-4000-8000-000000000001",
|
||||
capabilityGeneration: "a".repeat(64), challenge: "c".repeat(64), now: Date.now(), budget });
|
||||
const live = await inspect(); expect(live.ingress.grantIds).toEqual([principal.grant.id]);
|
||||
expect(live.transfers.blocked).toBe(true); expect(live.schedule.blocked).toBe(true);
|
||||
expect((await db.execute(idleMcpWorkQuery("mcp_file_tickets",{ ingress: live.ingress, transfers: live.transfers })!)).length).toBe(1);
|
||||
const uploadUrl = new URL(uploaded.url);
|
||||
const result = await request(app).put(uploadUrl.pathname+uploadUrl.search).set("Content-Type","text/plain").send(bytes);
|
||||
expect(result.status).toBe(200);
|
||||
expect((await request(app).put(uploadUrl.pathname+uploadUrl.search).set("Content-Type","text/plain").send(bytes)).status).toBe(200);
|
||||
const download = await transfers.getDownloadUrl(principal,result.body.attachment.id), downloadUrl = new URL(download.url);
|
||||
expect((await request(app).get(downloadUrl.pathname+downloadUrl.search)).status).toBe(200);
|
||||
expect((await inspect()).transfers.blocked).toBe(true);
|
||||
expect(coordinator.nextWakeAt()).not.toBeNull();
|
||||
await vi.advanceTimersByTimeAsync(610_000); await settle();
|
||||
const expired = await inspect(); expect(expired.transfers.blocked).toBe(false);
|
||||
expect(expired.frontDoor.blocked).toBe(true);
|
||||
expect(expired.deviceStatus).toEqual({blocked:true,state:"missing-device-status-proof",deadlines:[]});
|
||||
expect(expired.schedule.blocked).toBe(true);expect(expired.schedule.deadlines).toEqual([]);
|
||||
expect((await db.execute(idleMcpWorkQuery("mcp_file_tickets",{ ingress: expired.ingress, transfers: expired.transfers })!)).length).toBe(0);
|
||||
expect(await db.select().from(mcpFileTickets)).toHaveLength(2);
|
||||
expect((await db.execute(sql.raw(idleMcpHistoryChecks.find(check => check.reason === "mcp_attachment_uploads")!.query))).length).toBe(0);
|
||||
expect(coordinator.nextWakeAt()).toBeNull(); await quiet(); expect(errors).toEqual([]);
|
||||
} finally { unregister(); await rm(directory,{recursive:true,force:true}); }
|
||||
},30_000);
|
||||
|
||||
it.each(["code","device"] as const)("keeps approved %s handoff awake after empty index backfill until the real exchange consumes it", async kind => {
|
||||
const f = await fixture(); await f.start(); await startMcpIngressHandoffExpiry(f.observed,coordinator).ready;
|
||||
const budget = { beforeRead: async () => {}, check: () => {} };
|
||||
const inspectHandoffs = () => readIdleMcpAuthorizationHandoffs(db,{now:Date.now(),budget});
|
||||
const client = await f.oauth.register({client_name:"Handoff",redirect_uris:["https://receiver.example/oauth"],token_endpoint_auth_method:"none",
|
||||
grant_types:["authorization_code","refresh_token",DEVICE_GRANT],response_types:["code"]});
|
||||
const actor: Request["actor"] = {type:"board",source:"session",userId,companyIds:[companyId]};
|
||||
let exchange!: () => ReturnType<typeof f.oauth.token>, grantId: string;
|
||||
if (kind === "code") {
|
||||
const verifier = "v".repeat(64);
|
||||
const start = await f.oauth.authorize({client_id:client.client_id,redirect_uri:"https://receiver.example/oauth",resource:f.oauth.config.resource,
|
||||
response_type:"code",code_challenge:createHash("sha256").update(verifier).digest("base64url"),code_challenge_method:"S256",scope:"paperclip:read offline_access",company_id:companyId});
|
||||
const id = new URL(start).pathname.split("/").at(-1)!;
|
||||
expect((await inspectHandoffs()).blocked).toBe(false);
|
||||
const approved = await f.oauth.consent(id,actor,{decision:"approve",companyId,allowWrites:false});
|
||||
grantId = (await db.select().from(mcpOauthRequests).where(eq(mcpOauthRequests.id,id)))[0].grantId!;
|
||||
exchange = () => f.oauth.token({grant_type:"authorization_code",client_id:client.client_id,resource:f.oauth.config.resource,redirect_uri:"https://receiver.example/oauth",
|
||||
code:new URL(approved.redirectUrl).searchParams.get("code"),code_verifier:verifier});
|
||||
} else {
|
||||
const start = await f.oauth.deviceAuthorize({client_id:client.client_id,resource:f.oauth.config.resource,scope:"paperclip:read offline_access",company_id:companyId});
|
||||
expect((await inspectHandoffs()).blocked).toBe(false);
|
||||
await f.oauth.consentDevice(start.user_code,actor,{decision:"approve",companyId,allowWrites:false});
|
||||
grantId = (await db.select().from(mcpOauthDeviceRequests))[0].grantId!;
|
||||
await vi.advanceTimersByTimeAsync(5001);
|
||||
exchange = () => f.oauth.token({grant_type:DEVICE_GRANT,client_id:client.client_id,resource:f.oauth.config.resource,device_code:start.device_code});
|
||||
}
|
||||
const published = new Map<string,number>();
|
||||
await startMcpRevocationIndexSync(f.observed,coordinator,{key:Buffer.alloc(32,9).toString("base64url"),client:{publish:async value=>{
|
||||
const row = parseMcpRevocationIndexRequest(value);
|
||||
if(row.operation==="complete") {published.set(row.authorityId,row.revision);return {complete:true,receipt:{version:1,authorityId:row.authorityId,keyId:row.keyId,revision:row.revision,generation:createHash("sha256").update(`${row.authorityId}:${row.revision}`).digest("hex")}};}
|
||||
return {complete:false,baseRevision:published.get(row.authorityId)??-1};
|
||||
}}}).ready;
|
||||
const unregister = registerIdleMcpOwners(db,{config:f.oauth.config,personal:service,dot:null});
|
||||
try {
|
||||
await settle();
|
||||
const live = await readIdleMcpSleepProof(db,db,{stackId:"fixture-stack",ownerId:"00000000-0000-4000-8000-000000000001",capabilityGeneration:"a".repeat(64),challenge:"c".repeat(64),now:Date.now(),budget});
|
||||
expect(live.ingress.grantIds).toContain(grantId);
|
||||
expect(live.handoffs.blocked).toBe(true); expect(live.schedule.blocked).toBe(true);
|
||||
expect((await db.execute(idleMcpWorkQuery("mcp_authorization_handoffs",{ingress:live.ingress,handoffs:live.handoffs})!)).length).toBe(1);
|
||||
expect(coordinator.nextWakeAt()).not.toBeNull();
|
||||
const tokens = await exchange(); expect(tokens.access_token).toMatch(/^pcmcp_at_/);
|
||||
await settle(); expect((await inspectHandoffs()).blocked).toBe(false);
|
||||
await vi.waitFor(async () => expect((await db.select().from(mcpOauthGrants).where(eq(mcpOauthGrants.id,grantId)))[0].revocationIndexReceipt?.revision).toBe(1));
|
||||
expect(coordinator.nextWakeAt()).toBeNull(); await quiet(); expect(errors).toEqual([]);
|
||||
} finally { unregister(); }
|
||||
},30_000);
|
||||
|
||||
it("waits for outer commit and rolls back nested producer notifications", async () => {
|
||||
const f = await fixture(); await f.subscribe(); await f.start();
|
||||
let release!: () => void, written!: () => void;
|
||||
@@ -137,6 +374,7 @@ describe.skipIf(!support.supported)("MCP event committed work", () => {
|
||||
it("retains first writes and failed sweeps through drain without passive SQL", async () => {
|
||||
const f = await fixture(); await f.subscribe(); await f.start();
|
||||
draining = true; await activity(); await settle(); await quiet(10_000); expect(received).toEqual([]);
|
||||
expect(service.reconciliationSnapshot().reconciled).toBe(false);
|
||||
draining = false; await vi.advanceTimersByTimeAsync(2_000); await delivered(1);
|
||||
failRead = true; await activity(); await vi.waitFor(() => expect(errors).toHaveLength(1));
|
||||
draining = true; await vi.advanceTimersByTimeAsync(2_000); await settle(); await quiet(10_000);
|
||||
|
||||
@@ -1,13 +1,15 @@
|
||||
import { createHash } from "node:crypto";
|
||||
import {cloudEventAuthoritySchema,cloudEventValidationOrigin,currentTenantEventOrigin,type CloudEventAuthority} from "./cloud-event-authority.js";
|
||||
import { createHash, randomUUID } from "node:crypto";
|
||||
import { and, asc, count, inArray, eq, gt, gte, isNotNull, isNull, lt, lte, or, sql } from "drizzle-orm";
|
||||
import { z } from "zod";
|
||||
import { companies, activityLog, issueComments, heartbeatRuns, dotMailboxItems, dotAgentBindings, dotRunnerAssignments, mcpEventAdmissions as admissions, mcpEventDeliveries as deliveries, mcpEventSubscriptions as subscriptions, type Db } from "@paperclipai/db";
|
||||
import { dotRunnerBroker } from "../dot-runner-broker.js";
|
||||
import { ISSUE_STATUSES } from "@paperclipai/shared";
|
||||
import { localEncryptedProvider } from "../../secrets/local-encrypted-provider.js";
|
||||
import { createLocalEncryptedCipher, localEncryptedProvider } from "../../secrets/local-encrypted-provider.js";
|
||||
import { logActivity } from "../activity-log.js";
|
||||
import type { createDeliveryWorkCoordinator } from "../delivery-work-coordinator.js";
|
||||
import { DELIVERY_QUEUES, notifyDeliveryWork } from "../delivery-work-notifications.js";
|
||||
import type { DeadlineReconciliationSnapshot } from "../deadline-reconciliation.js";
|
||||
import { type ApiDispatch } from "./capabilities.js";
|
||||
import { PublicMcpDisabledError, type McpPrincipal, type PublicMcpOAuth } from "./oauth.js";
|
||||
import { boundedJson, callbackUrl, eventFetch, McpEventError, postEvent, signingKey, verifyCallback, type EventFetch } from "./event-webhooks.js";
|
||||
@@ -23,7 +25,7 @@ const hour = 3600_000;
|
||||
const lifetime = 24 * hour;
|
||||
const rotationMs = 5 * 60_000;
|
||||
type Subscription = typeof subscriptions.$inferSelect;
|
||||
export type CloudEventAuthority = { token: string; expiresAt: number };
|
||||
export type {CloudEventAuthority} from "./cloud-event-authority.js";
|
||||
type Destination = { url: string; secret: string; previousSecret?: string; previousUntil?: number; cloud?: CloudEventAuthority };
|
||||
|
||||
export const publicMcpEventDefinitions = names.slice(0, 3).map((name, index) => ({
|
||||
@@ -55,6 +57,10 @@ export function createPublicMcpEvents(db: Db, oauth: PublicMcpOAuth, api: ApiDis
|
||||
let moreSources = false;
|
||||
let nextPage: string | null = null;
|
||||
let enabled = false;
|
||||
const ownerGeneration = randomUUID();
|
||||
let ownerRevision = 0;
|
||||
let ownerReconciled = false;
|
||||
const invalidate = () => { ownerRevision++; ownerReconciled = false; };
|
||||
const now = options.now ?? Date.now;
|
||||
// Dedicated tenant Dot grants are authenticated locally, including background
|
||||
// deliveries. They never use the personal connection's Cloud broker proof.
|
||||
@@ -72,9 +78,12 @@ export function createPublicMcpEvents(db: Db, oauth: PublicMcpOAuth, api: ApiDis
|
||||
await api(principal, "GET", `/issues/${args.taskId}`);
|
||||
}
|
||||
async function authorizeCloud(principal: McpPrincipal, authority?: CloudEventAuthority) {
|
||||
if (!cloudOrigin) return;
|
||||
if(authority!==undefined&&!cloudEventAuthoritySchema.safeParse(authority).success)throw new McpEventError(-32602,"Invalid hosted subscription authority.");
|
||||
if (!cloudOrigin&&authority?.transport===undefined) return;
|
||||
if (!authority || !Number.isFinite(authority.expiresAt) || authority.expiresAt <= now() || authority.token.length > 24000) throw new McpEventError(-32602, "Refresh the hosted connection before subscribing.");
|
||||
const response = await fetcher(cloudOrigin + "/mcp/paperclip", {
|
||||
const validationOrigin=await cloudEventValidationOrigin(db,oauth.config,cloudOrigin,authority);
|
||||
if(!validationOrigin)throw new McpEventError(-32602,"The hosted subscription authority is unavailable.");
|
||||
const response = await fetcher(validationOrigin + "/mcp/paperclip", {
|
||||
method: "POST", redirect: "error", signal: AbortSignal.timeout(10_000),
|
||||
headers: { Authorization: `Bearer ${authority.token}`, "Content-Type": "application/json", Accept: "application/json, text/event-stream" },
|
||||
body: JSON.stringify({ jsonrpc: "2.0", id: 1, method: "tools/call", params: { name: "paperclip_connection", arguments: {} } }),
|
||||
@@ -383,12 +392,15 @@ export function createPublicMcpEvents(db: Db, oauth: PublicMcpOAuth, api: ApiDis
|
||||
return coordinator.register(queue, {
|
||||
retryMs: 2_000,
|
||||
pausePassiveMaintenance: true,
|
||||
onStop: invalidate,
|
||||
onSettled: hints => {
|
||||
invalidate();
|
||||
requested++;
|
||||
if (!hints || hints.has("*")) requestedIds = null;
|
||||
else if (requestedIds) for (const id of hints) requestedIds.add(id);
|
||||
},
|
||||
async run() {
|
||||
invalidate();
|
||||
if (stopped || options.isBackgroundWorkEnabled?.() === false) return;
|
||||
const generation = requested;
|
||||
if (!scan && completed !== requested) {
|
||||
@@ -417,19 +429,45 @@ export function createPublicMcpEvents(db: Db, oauth: PublicMcpOAuth, api: ApiDis
|
||||
completed = generation; scan = null;
|
||||
if (completed === requested) requestedIds = new Set();
|
||||
}
|
||||
ownerReconciled = completed === requested && !scan && !moreSources && !more;
|
||||
},
|
||||
hasPending: async () => false,
|
||||
nextRunAt: () => stopped ? null : completed !== requested
|
||||
? Math.min(nextDeadline ?? Infinity, Date.now() + 2_000) : nextDeadline,
|
||||
});
|
||||
}
|
||||
return { subscribe, unsubscribe, tick,
|
||||
function createIdleDestinationInspector() {
|
||||
let cipher: ReturnType<typeof createLocalEncryptedCipher> | undefined;
|
||||
return async (s: Subscription): Promise<boolean> => {
|
||||
try {
|
||||
if (JSON.stringify(s.deliveryMaterial).length > 65_536) return false;
|
||||
|
||||
|
||||
cipher ??= createLocalEncryptedCipher("read",{readOnly:true});
|
||||
const destination: Destination = JSON.parse(cipher.decrypt(s.deliveryMaterial));
|
||||
callbackUrl(destination.url); signingKey(destination.secret);
|
||||
if (destination.previousSecret) signingKey(destination.previousSecret);
|
||||
if (destination.previousUntil !== undefined && !Number.isFinite(destination.previousUntil)) return false;
|
||||
|
||||
|
||||
if(destination.cloud?.transport!==undefined){
|
||||
if(!cloudEventAuthoritySchema.safeParse(destination.cloud).success||!cloudOrigin)return false;
|
||||
currentTenantEventOrigin(oauth.config);
|
||||
}
|
||||
return !cloudOrigin || s.expiresAt.getTime() <= now() || !!destination.cloud && cloudEventAuthoritySchema.safeParse(destination.cloud).success
|
||||
&& destination.cloud.expiresAt > now();
|
||||
} catch { return false; }
|
||||
};
|
||||
}
|
||||
const inspectIdleDestination = (s: Subscription) => createIdleDestinationInspector()(s);
|
||||
return { subscribe, unsubscribe, tick, inspectIdleDestination, createIdleDestinationInspector,
|
||||
definitions: options.enableDotRunner ? [{ name: names[4], description: "The bound Dot mailbox changed. Drain paperclip_dot_inbox, read and accept current work. Duplicate/out-of-order events do not imply new assignments. Readiness challenges require paperclip_dot_confirm_event.", delivery: ["webhook"],
|
||||
inputSchema: z.toJSONSchema(z.object({ companyId: z.uuid(), bindingId: z.uuid() }).strict()),
|
||||
payloadSchema: z.toJSONSchema(z.object({ companyId: z.uuid(), bindingId: z.uuid(), bindingGeneration: z.number().int(), mailboxItemId: z.number().int(), kind: z.string() }).passthrough()),
|
||||
}] : [...publicMcpEventDefinitions, ...(options.enableDotPrototype ? [dotEventDefinition] : [])],
|
||||
start,
|
||||
async stop() { stopped = true; await running?.catch(() => {}); },
|
||||
reconciliationSnapshot: (): DeadlineReconciliationSnapshot => ({ revision: `${ownerGeneration}:${ownerRevision}`, reconciled: ownerReconciled && !stopped && !running }),
|
||||
async stop() { invalidate(); stopped = true; await running?.catch(() => {}); },
|
||||
};
|
||||
}
|
||||
export type PublicMcpEvents = ReturnType<typeof createPublicMcpEvents>;
|
||||
@@ -11,6 +11,7 @@ import { authorizationService } from "../authorization.js";
|
||||
import { issueService } from "../issues.js";
|
||||
import { prepareExternalDeliverablePublication, readExternalDeliverablePublication } from "../native-runtime/external-deliverable-publication.js";
|
||||
import { logActivity } from "../activity-log.js";
|
||||
import { DELIVERY_QUEUES, notifyDeliveryWork } from "../delivery-work-notifications.js";
|
||||
import { hashMcpSecret, McpOAuthError, type PublicMcpOAuth, type McpPrincipal } from "./oauth.js";
|
||||
import { McpCapabilityError, company, task, requestId, pick, object, stable, type ApiDispatch, type Capability } from "./contracts.js";
|
||||
|
||||
@@ -121,6 +122,7 @@ export function createPublicMcpTransfers(db: Db, oauth: PublicMcpOAuth, api: Api
|
||||
const token = randomBytes(32).toString("base64url");
|
||||
const expiresAt = new Date(Date.now() + lifetime);
|
||||
await db.transaction(async tx => {
|
||||
await notifyDeliveryWork(tx, DELIVERY_QUEUES.mcpIngressHandoffs, p.grant.id);
|
||||
await tx.execute(sql`select pg_advisory_xact_lock(hashtextextended(${p.grant.id}, 0))`);
|
||||
const [count] = await tx.select({ n: sql<number>`count(*)::int` }).from(mcpFileTickets).where(and(eq(mcpFileTickets.grantId, p.grant.id), gt(mcpFileTickets.expiresAt, new Date())));
|
||||
if (count.n >= 100) throw new McpCapabilityError("Too many active file links. Wait for earlier links to expire.");
|
||||
|
||||
@@ -0,0 +1,23 @@
|
||||
import type { Db } from "@paperclipai/db";
|
||||
import { describe, expect, it, vi } from "vitest";
|
||||
import { readIdleMcpAuthorizationHandoffs } from "./idle-authorization-handoffs.js";
|
||||
function fixture(results:unknown[][]) {
|
||||
const select=vi.fn(()=>({from:()=>({where:()=>({orderBy:()=>({limit:async()=>results.shift()??[]})})})}));
|
||||
return {select,tx:{select} as unknown as Pick<Db,"select">};
|
||||
}
|
||||
const code={id:"request",grantId:"grant",decidedAt:new Date(0),expiresAt:new Date(10_000),codeValid:true};
|
||||
const device={id:"device",grantId:"grant",status:"approved",expiresAt:new Date(20_000)};
|
||||
describe("approved MCP precredential handoff proof",()=>{
|
||||
it("retains both exact finite expiries and charges every actual read",async()=>{
|
||||
const beforeRead=vi.fn(async()=>{}),f=fixture([[code],[device]]);
|
||||
const value=await readIdleMcpAuthorizationHandoffs(f.tx,{now:1,budget:{beforeRead,check:()=>{}}});
|
||||
expect(value.blocked).toBe(true);expect(value.deadlines).toEqual([10_000,20_000]);expect(beforeRead).toHaveBeenCalledTimes(f.select.mock.calls.length);
|
||||
});
|
||||
it("fails closed for malformed/uncertain authority and incomplete coverage",async()=>{
|
||||
for(const results of [[[{...code,codeValid:false}],[]],[[{...code,grantId:null}],[]],[[{...code,decidedAt:null}],[]],[[{...code,expiresAt:new Date(NaN)}],[]],[[],[{...device,status:"unknown"}]], [Array.from({length:1001},()=>code),[]]]) {
|
||||
const value=await readIdleMcpAuthorizationHandoffs(fixture(results).tx,{now:1,budget:{beforeRead:async()=>{},check:()=>{}}});
|
||||
expect(value.blocked).toBe(true);expect(value.deadlines).toEqual([]);
|
||||
}
|
||||
const empty=await readIdleMcpAuthorizationHandoffs(fixture([[],[]]).tx,{now:1,budget:{beforeRead:async()=>{},check:()=>{}}});expect(empty).toMatchObject({blocked:false,deadlines:[]});
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,32 @@
|
||||
import { and, asc, gt, isNotNull, isNull, or, sql } from "drizzle-orm";
|
||||
import { mcpOauthRequests, mcpOauthDeviceRequests, type Db } from "@paperclipai/db";
|
||||
import type { IdleDeadlineSnapshot } from "../idle-deadline-types.js";
|
||||
import type { IdleMcpInspectionBudget } from "./idle-wake-proof.js";
|
||||
|
||||
export function pendingMcpCodePredicate(now: number) {
|
||||
return and(sql`${mcpOauthRequests.resource} ~ '/mcp/(paperclip|runner)$'`,isNotNull(mcpOauthRequests.codeHash),isNull(mcpOauthRequests.consumedAt),gt(mcpOauthRequests.expiresAt,new Date(now)));
|
||||
}
|
||||
export function pendingMcpDevicePredicate(now: number) {
|
||||
return and(sql`${mcpOauthDeviceRequests.resource} ~ '/mcp/(paperclip|runner)$'`,gt(mcpOauthDeviceRequests.expiresAt,new Date(now)),
|
||||
or(sql`${mcpOauthDeviceRequests.status} NOT IN ('pending','denied','consumed')`,and(sql`${mcpOauthDeviceRequests.status} = 'pending'`,isNotNull(mcpOauthDeviceRequests.grantId))));
|
||||
}
|
||||
export async function readIdleMcpAuthorizationHandoffs(tx: Pick<Db,"select">,input: {now:number;budget:IdleMcpInspectionBudget}): Promise<IdleDeadlineSnapshot> {
|
||||
await input.budget.beforeRead();
|
||||
const codes = await tx.select({ id:mcpOauthRequests.id,grantId:mcpOauthRequests.grantId,decidedAt:mcpOauthRequests.decidedAt,expiresAt:mcpOauthRequests.expiresAt,
|
||||
codeValid:sql<boolean>`${mcpOauthRequests.codeHash} ~ '^[a-f0-9]{64}$'` }).from(mcpOauthRequests)
|
||||
.where(pendingMcpCodePredicate(input.now)).orderBy(asc(mcpOauthRequests.expiresAt)).limit(1001);
|
||||
input.budget.check();await input.budget.beforeRead();
|
||||
const devices = await tx.select({id:mcpOauthDeviceRequests.id,grantId:mcpOauthDeviceRequests.grantId,status:mcpOauthDeviceRequests.status,expiresAt:mcpOauthDeviceRequests.expiresAt}).from(mcpOauthDeviceRequests)
|
||||
.where(pendingMcpDevicePredicate(input.now)).orderBy(asc(mcpOauthDeviceRequests.expiresAt)).limit(1001);
|
||||
input.budget.check();
|
||||
const rows = [...codes.map(row=>({id:row.id,grantId:row.grantId,expiresAt:row.expiresAt,valid:row.codeValid===true&&row.decidedAt!==null})),
|
||||
...devices.map(row=>({id:row.id,grantId:row.grantId,expiresAt:row.expiresAt,valid:row.status==="approved"}))];
|
||||
if (!Number.isFinite(input.now)||rows.length>1000) return {blocked:true,state:["incomplete-authorization-handoff"],deadlines:[]};
|
||||
const state:unknown[]=[],deadlines:number[]=[];
|
||||
for(const row of rows){
|
||||
input.budget.check();const at=row.expiresAt.getTime();
|
||||
if(!row.valid||!row.grantId||!Number.isFinite(at)||at<=input.now)return {blocked:true,state:["uncertain-authorization-handoff"],deadlines:[]};
|
||||
state.push([row.id,row.grantId,row.expiresAt.toISOString()]);deadlines.push(at);
|
||||
}
|
||||
return {blocked:rows.length>0,state,deadlines};
|
||||
}
|
||||
@@ -0,0 +1,70 @@
|
||||
import {describe,it,expect} from "vitest";
|
||||
import type {Db} from "@paperclipai/db";
|
||||
import {readIdleMcpClientMirror} from "./idle-client-mirror.js";
|
||||
import {mcpClientMirrorGeneration} from "./client-mirror-contract.js";
|
||||
|
||||
const identity={version:1 as const,instanceId:"instance-1",origin:"https://tenant.example.test",revision:7};
|
||||
const receipt={...identity,enabled:true,generation:mcpClientMirrorGeneration(identity)};
|
||||
const complete={...receipt,frontDoor:"tenant-personal-v1" as const,deviceStatus:{keyId:"a".repeat(64),generation:"b".repeat(64),count:0}};
|
||||
complete.generation=mcpClientMirrorGeneration(complete);
|
||||
const rowFor=(value=receipt)=>({revision:7,enabled:true,remoteResponsibility:true,receipt:value,deviceCutoverAt:new Date(1),deviceCutoverStackId:"stack"});
|
||||
function fixture(row:unknown){
|
||||
let queries=0,reads=0,checks=0;
|
||||
const tx={select:()=>{queries++;return{from:()=>({where:async()=>row===undefined?[]:[row]})};}} as unknown as Pick<Db,"select">;
|
||||
return{tx,context:{enabled:true,origin:identity.origin,instanceId:identity.instanceId,managedStackId:"stack",budget:{beforeRead:async()=>{reads++;},check:()=>{checks++;}}},counts:()=>({queries,reads,checks})};
|
||||
}
|
||||
describe("MCP client mirror inspection",()=>{
|
||||
it("unconfigured selfhost adds no old Cloud responsibility read",async()=>{
|
||||
const f=fixture(undefined);
|
||||
expect((await readIdleMcpClientMirror(f.tx,{...f.context,enabled:false,managedStackId:undefined})).blocked).toBe(false);
|
||||
expect((await readIdleMcpClientMirror(f.tx,{...f.context,managedStackId:undefined})).blocked).toBe(true);
|
||||
expect(f.counts()).toEqual({queries:0,reads:0,checks:0});
|
||||
});
|
||||
it("missing, partial, future, malformed and foreign receipts fail closed without mutations",async()=>{
|
||||
for(const row of [undefined,{...rowFor(),receipt:null},{...rowFor(),revision:8},{...rowFor(),revision:6},{...rowFor(),receipt:{...receipt,instanceId:"old-instance"}},{...rowFor(),receipt:{...receipt,generation:"0".repeat(64)}},{...rowFor(),revision:NaN}]){
|
||||
const f=fixture(row);expect((await readIdleMcpClientMirror(f.tx,f.context)).blocked).toBe(true);
|
||||
expect(f.counts()).toEqual({queries:1,reads:1,checks:1});
|
||||
}
|
||||
});
|
||||
it("old metadata-only acknowledgment cannot certify the advertised front door",async()=>{
|
||||
const f=fixture(rowFor()),result=await readIdleMcpClientMirror(f.tx,f.context);
|
||||
expect(result.blocked).toBe(true);expect(result.receipt).toEqual(receipt);expect((result.state as unknown[])[0]).toBe("incomplete-front-door");
|
||||
});
|
||||
it("only actual capability, status ref and durable cutover permit the personal profile",async()=>{
|
||||
const f=fixture(rowFor(complete)),result=await readIdleMcpClientMirror(f.tx,f.context);
|
||||
expect(result.blocked).toBe(false);expect(result.receipt).toEqual(complete);expect(result.deadlines).toEqual([]);
|
||||
for(const patch of [{deviceCutoverAt:null},{deviceCutoverStackId:null},{deviceCutoverStackId:"foreign-stack"}]){const missing=fixture({...rowFor(complete),...patch});expect((await readIdleMcpClientMirror(missing.tx,missing.context)).blocked).toBe(true);}
|
||||
expect((await readIdleMcpClientMirror(f.tx,{...f.context,unsupportedProfiles:true})).blocked).toBe(true);
|
||||
expect(f.counts()).toEqual({queries:2,reads:2,checks:2});
|
||||
});
|
||||
it("disabled flag retains published responsibility until exact durable disable ACK",async()=>{
|
||||
const pending=fixture({...rowFor(),revision:8,enabled:false,receipt:null});
|
||||
expect((await readIdleMcpClientMirror(pending.tx,{...pending.context,enabled:false})).blocked).toBe(true);
|
||||
const disabled={...receipt,enabled:false,revision:8,generation:mcpClientMirrorGeneration({...receipt,enabled:false,revision:8})};
|
||||
const acknowledged=fixture({...rowFor(),revision:8,enabled:false,remoteResponsibility:false,receipt:disabled});
|
||||
expect((await readIdleMcpClientMirror(acknowledged.tx,{...acknowledged.context,enabled:false})).blocked).toBe(false);
|
||||
const late=fixture({...rowFor(),revision:9,receipt:disabled});expect((await readIdleMcpClientMirror(late.tx,late.context)).blocked).toBe(true);
|
||||
});
|
||||
it("missing connector/origin cannot erase managed published teardown debt",async()=>{
|
||||
for(const missing of [{instanceId:undefined},{origin:undefined},{origin:undefined,instanceId:undefined}]){
|
||||
const f=fixture({...rowFor(),revision:8,enabled:false,receipt:null});expect((await readIdleMcpClientMirror(f.tx,{...f.context,...missing,enabled:false})).blocked).toBe(true);
|
||||
expect(f.counts()).toEqual({queries:1,reads:1,checks:1});
|
||||
}
|
||||
for(const row of [undefined,{revision:1,enabled:true,remoteResponsibility:false,receipt:null}]){const never=fixture(row);expect((await readIdleMcpClientMirror(never.tx,{...never.context,enabled:false,origin:undefined,instanceId:undefined})).blocked).toBe(false);}
|
||||
});
|
||||
it("relocated selfhost keeps old persisted history without acquiring old cleanup authority",async()=>{
|
||||
const f=fixture({...rowFor(),revision:8,enabled:false,receipt:null});expect((await readIdleMcpClientMirror(f.tx,{...f.context,enabled:false,managedStackId:undefined})).blocked).toBe(false);
|
||||
expect(f.counts()).toEqual({queries:0,reads:0,checks:0});
|
||||
});
|
||||
it("strict retired namespace ACK remains absent after managed instance and origin replacement",async()=>{
|
||||
const disabled={...receipt,enabled:false,revision:8,generation:mcpClientMirrorGeneration({...receipt,enabled:false,revision:8})};
|
||||
const row={...rowFor(),revision:9,enabled:false,remoteResponsibility:false,receipt:disabled};
|
||||
const f=fixture(row),next={...f.context,enabled:false,instanceId:"replacement-instance",origin:"https://replacement.example.test",managedStackId:"replacement-stack"};
|
||||
expect(await readIdleMcpClientMirror(f.tx,next)).toEqual({blocked:false,state:"absent",deadlines:[]});
|
||||
for(const patch of [{remoteResponsibility:true},{enabled:true},{revision:7},{receipt:{...disabled,generation:"0".repeat(64)}},{receipt:{...disabled,enabled:true}}]){
|
||||
const pending=fixture({...row,...patch});expect((await readIdleMcpClientMirror(pending.tx,next)).blocked).toBe(true);
|
||||
expect((await readIdleMcpClientMirror(pending.tx,{...next,origin:undefined,instanceId:undefined})).blocked).toBe(true);
|
||||
}
|
||||
});
|
||||
|
||||
});
|
||||
@@ -0,0 +1,26 @@
|
||||
import { eq } from "drizzle-orm";
|
||||
import { mcpOauthClientMirrorState as state,type Db } from "@paperclipai/db";
|
||||
import type { IdleMcpInspectionBudget } from "./idle-wake-proof.js";
|
||||
import { mcpClientMirrorGeneration,mcpClientMirrorReceiptSchema,type McpClientMirrorReceipt } from "./client-mirror-contract.js";
|
||||
import type { IdleDeadlineSnapshot } from "../idle-deadline-types.js";
|
||||
|
||||
export async function readIdleMcpClientMirror(tx:Pick<Db,"select">,context:{budget:IdleMcpInspectionBudget;enabled:boolean;origin?:string;instanceId?:string;managedStackId?:string;authorizationOrigin?:string;unsupportedProfiles?:boolean}):Promise<IdleDeadlineSnapshot&{receipt?:McpClientMirrorReceipt}>{
|
||||
if(!context.managedStackId)return{blocked:context.enabled,state:context.enabled?"unconfigured-front-door":"absent",deadlines:[]};
|
||||
await context.budget.beforeRead();
|
||||
const[row]=await tx.select().from(state).where(eq(state.id,"public"));context.budget.check();
|
||||
const receipt=mcpClientMirrorReceiptSchema.safeParse(row?.receipt);
|
||||
const retired = !row?.remoteResponsibility && row?.enabled === false && Number.isSafeInteger(row.revision) && row.revision >= 0
|
||||
&& receipt.success && !receipt.data.enabled && receipt.data.revision <= row.revision
|
||||
&& receipt.data.generation === mcpClientMirrorGeneration(receipt.data);
|
||||
if(!context.origin||!context.instanceId){
|
||||
const debt=Boolean(row?.remoteResponsibility||row?.receipt!==null&&row?.receipt!==undefined&&!retired);
|
||||
return{blocked:context.enabled||debt,state:debt?["unavailable-published-front-door",row?.revision??null]:context.enabled?"unconfigured-front-door":"absent",deadlines:[]};
|
||||
}
|
||||
if(!context.enabled){
|
||||
if(!row||!row.remoteResponsibility&&(row.receipt===null||retired))return{blocked:false,state:"absent",deadlines:[]};
|
||||
return{blocked:true,state:["pending-front-door-disable",row.revision],deadlines:[]};
|
||||
}
|
||||
if(!row||!row.enabled||!row.remoteResponsibility||!Number.isSafeInteger(row.revision)||row.revision<0||!receipt.success||!receipt.data.enabled||receipt.data.revision!==row.revision||receipt.data.instanceId!==context.instanceId||receipt.data.origin!==context.origin||(receipt.data.authorizationOrigin??receipt.data.origin)!==(context.authorizationOrigin??context.origin)||receipt.data.generation!==mcpClientMirrorGeneration(receipt.data))return{blocked:true,state:["pending-client-mirror",row?.revision??null],deadlines:[]};
|
||||
const ready=receipt.data.frontDoor==="tenant-personal-v1"&&receipt.data.deviceStatus!==undefined&&row.deviceCutoverAt instanceof Date&&Number.isFinite(row.deviceCutoverAt.getTime())&&row.deviceCutoverStackId===context.managedStackId&&(receipt.data.authorizationOrigin??receipt.data.origin)===receipt.data.origin&&!context.unsupportedProfiles;
|
||||
return{blocked:!ready,state:[ready?"tenant-facade-v1":"incomplete-front-door",receipt.data,row.deviceCutoverStackId,row.deviceCutoverAt?.toISOString()??null],receipt:receipt.data,deadlines:[]};
|
||||
}
|
||||
@@ -0,0 +1,24 @@
|
||||
import { and,asc,eq,exists,gt,isNotNull,ne,notInArray } from "drizzle-orm";
|
||||
import { mcpOauthClientMirrorState as state,mcpOauthDeviceRequests as devices,type Db } from "@paperclipai/db";
|
||||
import { managedMcpDeviceScope } from "./device-cutover.js";
|
||||
import type { IdleMcpInspectionBudget } from "./idle-wake-proof.js";
|
||||
import type { IdleDeadlineSnapshot } from "../idle-deadline-types.js";
|
||||
|
||||
export function pendingLegacyMcpDevicePredicate(tx:Pick<Db,"select">,now:number){
|
||||
const scope=managedMcpDeviceScope();if(!scope)return;
|
||||
return and(eq(devices.resource,scope.resource),ne(devices.ingressOwner,"cloud"),gt(devices.expiresAt,new Date(now)),
|
||||
notInArray(devices.status,["denied","consumed"]),exists(tx.select({id:state.id}).from(state).where(and(eq(state.id,"public"),isNotNull(state.deviceCutoverStackId),isNotNull(state.deviceCutoverAt)))));
|
||||
}
|
||||
export async function readIdleMcpDeviceCutover(tx:Pick<Db,"select">,context:{now:number;budget:IdleMcpInspectionBudget}):Promise<IdleDeadlineSnapshot>{
|
||||
const predicate=pendingLegacyMcpDevicePredicate(tx,context.now);if(!predicate)return{blocked:false,state:"absent",deadlines:[]};
|
||||
await context.budget.beforeRead();
|
||||
const rows=await tx.select({id:devices.id,status:devices.status,ingressOwner:devices.ingressOwner,expiresAt:devices.expiresAt}).from(devices).where(predicate).orderBy(asc(devices.expiresAt)).limit(1001);
|
||||
context.budget.check();
|
||||
if(rows.length>1000||!Number.isFinite(context.now))return{blocked:true,state:"uncertain-legacy-device-drain",deadlines:[]};
|
||||
const deadlines:number[]=[],snapshot:unknown[]=[];
|
||||
for(const row of rows){context.budget.check();const at=row.expiresAt.getTime();
|
||||
if(row.ingressOwner!=="tenant"||!["pending","approved"].includes(row.status)||!Number.isFinite(at)||at<=context.now)return{blocked:true,state:"uncertain-legacy-device-drain",deadlines:[]};
|
||||
deadlines.push(at);snapshot.push([row.id,row.status,row.expiresAt.toISOString()]);
|
||||
}
|
||||
return{blocked:rows.length>0,state:snapshot,deadlines};
|
||||
}
|
||||
@@ -0,0 +1,33 @@
|
||||
import {describe,it,expect} from "vitest";
|
||||
import {createHash} from "node:crypto";
|
||||
import type {Db} from "@paperclipai/db";
|
||||
import {readIdleMcpDeviceStatus} from "./idle-device-status.js";
|
||||
import {mcpDeviceStatusFingerprint,mcpDeviceStatusGeneration,mcpDeviceStatusKeyId} from "./device-status-contract.js";
|
||||
import type {McpClientMirrorReceipt} from "./client-mirror-contract.js";
|
||||
|
||||
const key=Buffer.alloc(32,9).toString("base64url"),instanceId="instance",origin="https://tenant.example.test",revision=7;
|
||||
const row={requestId:"00000000-0000-4000-8000-000000000001",clientId:"client",resource:origin+"/mcp/paperclip",ingressOwner:"tenant" as const,status:"pending" as const,expiresAt:new Date(1_800_000_000_000),deviceCodeHash:createHash("sha256").update("private-device").digest("hex")};
|
||||
const entry={requestId:row.requestId,clientId:row.clientId,resource:row.resource,ingressOwner:row.ingressOwner,status:row.status,expiresAt:row.expiresAt.getTime(),fingerprint:mcpDeviceStatusFingerprint(key,row.deviceCodeHash,row.clientId,row.resource)};
|
||||
const reference={keyId:mcpDeviceStatusKeyId(key),generation:mcpDeviceStatusGeneration({instanceId,origin,revision,keyId:mcpDeviceStatusKeyId(key)},[entry]),count:1};
|
||||
const receipt:McpClientMirrorReceipt={version:1,instanceId,origin,revision,enabled:true,generation:"a".repeat(64),deviceStatus:reference};
|
||||
function fixture(rows:unknown[]){let reads=0,checks=0,keyReads=0;const tx={select:()=>({from:()=>({where:()=>({orderBy:()=>({limit:async()=>rows})})})})} as unknown as Pick<Db,"select">;
|
||||
return{tx,input:{enabled:true,receipt,key:()=>{keyReads++;return key;},budget:{beforeRead:async()=>{reads++;},check:()=>{checks++;}}},counts:()=>({reads,checks,keyReads})};}
|
||||
describe("bounded current device status proof",()=>{
|
||||
it("absent feature performs no reads or key access",async()=>{
|
||||
const f=fixture([row]);expect((await readIdleMcpDeviceStatus(f.tx,{...f.input,enabled:false})).blocked).toBe(false);expect(f.counts()).toEqual({reads:0,checks:0,keyReads:0});
|
||||
});
|
||||
it("exact source identities and original expiry match their acknowledged projection",async()=>{
|
||||
const f=fixture([row]);expect((await readIdleMcpDeviceStatus(f.tx,f.input)).blocked).toBe(false);expect(f.counts().reads).toBe(1);expect(f.counts().keyReads).toBe(1);
|
||||
});
|
||||
it("deletion, status change and different expiry cannot reuse an old ACK",async()=>{
|
||||
for(const rows of [[],[{...row,status:"denied"}],[{...row,expiresAt:new Date(row.expiresAt.getTime()+1)}]]){
|
||||
const f=fixture(rows);expect((await readIdleMcpDeviceStatus(f.tx,f.input)).blocked).toBe(true);
|
||||
}
|
||||
});
|
||||
it("missing reference, unavailable/rotated key and canonical overflow fail closed",async()=>{
|
||||
const f=fixture([row]);expect((await readIdleMcpDeviceStatus(f.tx,{...f.input,receipt:undefined})).blocked).toBe(true);
|
||||
expect((await readIdleMcpDeviceStatus(f.tx,{...f.input,key:()=>{throw new Error("No local key");}})).blocked).toBe(true);
|
||||
expect((await readIdleMcpDeviceStatus(f.tx,{...f.input,key:()=>Buffer.alloc(32,8).toString("base64url")})).blocked).toBe(true);
|
||||
const overflow=fixture(Array.from({length:1001},()=>row));expect((await readIdleMcpDeviceStatus(overflow.tx,overflow.input)).blocked).toBe(true);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,24 @@
|
||||
import {eq} from "drizzle-orm";
|
||||
import {mcpOauthDeviceRequests as devices,type Db} from "@paperclipai/db";
|
||||
import {deriveLocalEncryptedInstanceSecret} from "../../secrets/local-encrypted-provider.js";
|
||||
import {mcpDeviceStatusEntrySchema,mcpDeviceStatusFingerprint,mcpDeviceStatusGeneration,mcpDeviceStatusKeyId} from "./device-status-contract.js";
|
||||
import type {McpClientMirrorReceipt} from "./client-mirror-contract.js";
|
||||
import type {IdleMcpInspectionBudget} from "./idle-wake-proof.js";
|
||||
import type {IdleDeadlineSnapshot} from "../idle-deadline-types.js";
|
||||
|
||||
export async function readIdleMcpDeviceStatus(tx:Pick<Db,"select">,input:{enabled:boolean;receipt?:McpClientMirrorReceipt;budget:IdleMcpInspectionBudget;key?:()=>string}):Promise<IdleDeadlineSnapshot>{
|
||||
if(!input.enabled)return{blocked:false,state:"absent",deadlines:[]};
|
||||
const receipt=input.receipt;if(!receipt?.deviceStatus)return{blocked:true,state:"missing-device-status-proof",deadlines:[]};
|
||||
input.budget.check();
|
||||
let key:string;
|
||||
try{key=input.key?input.key():deriveLocalEncryptedInstanceSecret(`mcp-device-status:v1:${receipt.instanceId}`,process.env,{readOnly:true});}
|
||||
catch{return{blocked:true,state:"device-status-key-unavailable",deadlines:[]};}
|
||||
if(mcpDeviceStatusKeyId(key)!==receipt.deviceStatus.keyId)return{blocked:true,state:"device-status-key-replaced",deadlines:[]};
|
||||
await input.budget.beforeRead();
|
||||
const rows=await tx.select({requestId:devices.id,deviceCodeHash:devices.deviceCodeHash,clientId:devices.clientId,resource:devices.resource,ingressOwner:devices.ingressOwner,status:devices.status,expiresAt:devices.expiresAt}).from(devices).where(eq(devices.resource,receipt.origin+"/mcp/paperclip")).orderBy(devices.id).limit(1001);
|
||||
input.budget.check();
|
||||
if(rows.length>1000)return{blocked:true,state:"device-status-overflow",deadlines:[]};
|
||||
const entries=rows.map(row=>{input.budget.check();return mcpDeviceStatusEntrySchema.parse({requestId:row.requestId,clientId:row.clientId,resource:row.resource,ingressOwner:row.ingressOwner,status:row.status,expiresAt:row.expiresAt.getTime(),fingerprint:mcpDeviceStatusFingerprint(key,row.deviceCodeHash,row.clientId,row.resource)});});
|
||||
const generation=mcpDeviceStatusGeneration({instanceId:receipt.instanceId,origin:receipt.origin,revision:receipt.revision,keyId:receipt.deviceStatus.keyId},entries);
|
||||
return{blocked:generation!==receipt.deviceStatus.generation||entries.length!==receipt.deviceStatus.count,state:[receipt.deviceStatus,generation],deadlines:[]};
|
||||
}
|
||||
@@ -0,0 +1,31 @@
|
||||
import { describe, expect, it } from "vitest";
|
||||
import { projectIdleMcpEventSchedule, type IdleMcpEventDeadlineRow } from "./idle-event-schedule.js";
|
||||
const input = { now: 1000, coveredGrantIds: ["grant"], personal: { revision: "personal:1", reconciled: true }, dot: { revision: "dot:1", reconciled: true } };
|
||||
const row: IdleMcpEventDeadlineRow = { id: "delivery", subscriptionId: "subscription", companyId: "company", grantId: "grant", taskId: "task", bindingId: null,
|
||||
subscriptionExpiresAt: new Date(10_000), subscriptionStoppedAt: null, nextAttemptAt: new Date(2000), attempts: 1, outcome: "http_503", sourceMatches: true };
|
||||
describe("MCP durable retry deadline proof", () => {
|
||||
it("represents future retries after successful reconciliation without advancing their deadline", () => {
|
||||
const result = projectIdleMcpEventSchedule([row], input);
|
||||
expect(result.blocked).toBe(false);
|
||||
expect(result.deadlines).toEqual([2000]);
|
||||
expect(projectIdleMcpEventSchedule([row], { ...input, now: 1500 }).deadlines).toEqual([2000]);
|
||||
expect(projectIdleMcpEventSchedule([], input).deadlines).toEqual([]);
|
||||
});
|
||||
it("retains due, in-flight, orphaned, exhausted, stopped, foreign and unfamiliar delivery work", () => {
|
||||
for (const change of [{ nextAttemptAt: new Date(1000) }, { nextAttemptAt: new Date(NaN) }, { attempts: 0 }, { attempts: 6 }, { attempts: 1.5 },
|
||||
{ outcome: null }, { outcome: "delivered" }, { outcome: "http_404" }, { subscriptionStoppedAt: new Date() }, { subscriptionExpiresAt: new Date(1000) },
|
||||
{ sourceMatches: false }, { grantId: "foreign" }, { taskId: null, bindingId: null }, { bindingId: "binding" }]) {
|
||||
expect(projectIdleMcpEventSchedule([{ ...row, ...change }], input).blocked).toBe(true);
|
||||
}
|
||||
expect(projectIdleMcpEventSchedule([row], { ...input, personal: { revision: "changed", reconciled: false } }).blocked).toBe(true);
|
||||
expect(projectIdleMcpEventSchedule([row], { ...input, coveredGrantIds: [] }).blocked).toBe(true);
|
||||
expect(projectIdleMcpEventSchedule(Array.from({ length: 10_001 }, () => row), input).blocked).toBe(true);
|
||||
});
|
||||
it("checks the Dot owner independently and changes proof state on a new reconciliation", () => {
|
||||
const dot = { ...row, taskId: null, bindingId: "binding", outcome: "network_error" };
|
||||
expect(projectIdleMcpEventSchedule([dot], input).blocked).toBe(false);
|
||||
expect(projectIdleMcpEventSchedule([dot], { ...input, dot: { revision: "dot:2", reconciled: false } }).blocked).toBe(true);
|
||||
expect(projectIdleMcpEventSchedule([row], { ...input, personal: { revision: "personal:2", reconciled: true } }).state)
|
||||
.not.toEqual(projectIdleMcpEventSchedule([row], input).state);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,77 @@
|
||||
import type { IdleMcpInspectionBudget } from "./idle-wake-proof.js";
|
||||
import { asc, eq, isNull, sql } from "drizzle-orm";
|
||||
import { mcpEventDeliveries as deliveries, mcpEventSubscriptions as subscriptions, type Db } from "@paperclipai/db";
|
||||
import type { DeadlineReconciliationSnapshot } from "../deadline-reconciliation.js";
|
||||
import { idleDeadlineRowLimit, isFutureIdleDeadline, type IdleDeadlineSnapshot } from "../idle-deadline-types.js";
|
||||
|
||||
export interface IdleMcpEventDeadlineRow {
|
||||
id: string;
|
||||
subscriptionId: string;
|
||||
companyId: string;
|
||||
grantId: string;
|
||||
taskId: string | null;
|
||||
bindingId: string | null;
|
||||
subscriptionExpiresAt: Date;
|
||||
subscriptionStoppedAt: Date | null;
|
||||
nextAttemptAt: Date;
|
||||
attempts: number;
|
||||
outcome: string | null;
|
||||
sourceMatches: boolean;
|
||||
}
|
||||
export interface IdleMcpEventScheduleProof extends IdleDeadlineSnapshot {
|
||||
deliveryIds: string[];
|
||||
}
|
||||
export interface IdleMcpEventScheduleInput {
|
||||
budget?: IdleMcpInspectionBudget;
|
||||
now: number;
|
||||
coveredGrantIds: readonly string[];
|
||||
personal: DeadlineReconciliationSnapshot;
|
||||
dot: DeadlineReconciliationSnapshot;
|
||||
}
|
||||
|
||||
|
||||
|
||||
export function projectIdleMcpEventSchedule(rows: IdleMcpEventDeadlineRow[], input: IdleMcpEventScheduleInput): IdleMcpEventScheduleProof {
|
||||
input.budget?.check();
|
||||
const state: unknown[] = [], deadlines: number[] = [], deliveryIds: string[] = [];
|
||||
const blocked = () => ({ blocked: true, state, deadlines: [], deliveryIds: [] });
|
||||
if (!Number.isFinite(input.now) || rows.length > idleDeadlineRowLimit) return blocked();
|
||||
const grants = new Set(input.coveredGrantIds);
|
||||
for (const row of rows) {
|
||||
input.budget?.check();
|
||||
const owner = row.taskId !== null && row.bindingId === null ? input.personal
|
||||
: row.bindingId !== null && row.taskId === null ? input.dot : undefined;
|
||||
if (!owner?.reconciled || !owner.revision || !grants.has(row.grantId) || row.sourceMatches !== true
|
||||
|| row.subscriptionStoppedAt !== null || row.subscriptionExpiresAt.getTime() <= input.now
|
||||
|| !Number.isSafeInteger(row.attempts) || row.attempts < 1 || row.attempts >= 6
|
||||
|| !(row.outcome === "network_error" || row.outcome === "http_408" || row.outcome === "http_429" || /^http_5[0-9][0-9]$/.test(row.outcome ?? ""))
|
||||
|| !isFutureIdleDeadline(row.nextAttemptAt.getTime(), input.now)) return blocked();
|
||||
state.push([row.id, row.subscriptionId, row.companyId, row.grantId, row.taskId, row.bindingId,
|
||||
row.subscriptionExpiresAt.toISOString(), row.nextAttemptAt.toISOString(), row.attempts, row.outcome, owner.revision]);
|
||||
deadlines.push(row.nextAttemptAt.getTime());
|
||||
deliveryIds.push(row.id);
|
||||
}
|
||||
return { blocked: false, state: [input.personal, input.dot, state], deadlines, deliveryIds };
|
||||
}
|
||||
|
||||
export async function readIdleMcpEventSchedule(tx: Pick<Db, "select">, input: IdleMcpEventScheduleInput): Promise<IdleMcpEventScheduleProof> {
|
||||
await input.budget?.beforeRead();
|
||||
const rows = await tx.select({ id: deliveries.id, subscriptionId: subscriptions.id,
|
||||
companyId: subscriptions.companyId, grantId: subscriptions.grantId, taskId: subscriptions.taskId, bindingId: subscriptions.bindingId,
|
||||
subscriptionExpiresAt: subscriptions.expiresAt, subscriptionStoppedAt: subscriptions.stoppedAt,
|
||||
nextAttemptAt: deliveries.nextAttemptAt, attempts: deliveries.attempts, outcome: deliveries.outcome,
|
||||
sourceMatches: sql<boolean>`((
|
||||
${deliveries.activityId} IS NOT NULL AND ${deliveries.mailboxItemId} IS NULL
|
||||
AND ${subscriptions.taskId} IS NOT NULL AND ${subscriptions.bindingId} IS NULL
|
||||
AND EXISTS (SELECT 1 FROM activity_log a WHERE a.id = ${deliveries.activityId}
|
||||
AND a.company_id = ${subscriptions.companyId} AND a.entity_type = 'issue' AND a.entity_id = ${subscriptions.taskId}::text)
|
||||
) OR (
|
||||
${deliveries.mailboxItemId} IS NOT NULL AND ${deliveries.activityId} IS NULL
|
||||
AND ${subscriptions.bindingId} IS NOT NULL AND ${subscriptions.taskId} IS NULL
|
||||
AND EXISTS (SELECT 1 FROM dot_mailbox_items m WHERE m.id = ${deliveries.mailboxItemId}
|
||||
AND m.company_id = ${subscriptions.companyId} AND m.binding_id = ${subscriptions.bindingId})
|
||||
))`,
|
||||
}).from(deliveries).innerJoin(subscriptions, eq(subscriptions.id, deliveries.subscriptionId))
|
||||
.where(isNull(deliveries.finishedAt)).orderBy(asc(deliveries.id)).limit(idleDeadlineRowLimit + 1);
|
||||
return projectIdleMcpEventSchedule(rows, input);
|
||||
}
|
||||
@@ -1,5 +1,11 @@
|
||||
/** Completion receipts are retained for idempotency. Fixed aliases only. */
|
||||
export const idleMcpHistoryChecks = [
|
||||
{ reason: "mcp_file_tickets", query: `SELECT 1 FROM mcp_file_tickets t WHERE t.expires_at > statement_timestamp() OR t.expires_at IS NULL` },
|
||||
{ reason: "mcp_authorization_handoffs", query: `SELECT 1 FROM mcp_oauth_requests r WHERE r.resource ~ '/mcp/(paperclip|runner)$'
|
||||
AND r.code_hash IS NOT NULL AND r.consumed_at IS NULL AND (r.expires_at > statement_timestamp() OR r.expires_at IS NULL)
|
||||
UNION ALL SELECT 1 FROM mcp_oauth_device_requests d WHERE d.resource ~ '/mcp/(paperclip|runner)$'
|
||||
AND (d.status NOT IN ('pending','denied','consumed') OR (d.status = 'pending' AND d.grant_id IS NOT NULL))
|
||||
AND (d.expires_at > statement_timestamp() OR d.expires_at IS NULL)` },
|
||||
{ reason: "mcp_oauth_grants", query: `SELECT 1 FROM mcp_oauth_grants g WHERE
|
||||
(g.revoked_at IS NOT NULL AND EXISTS (SELECT 1 FROM companies c WHERE c.id = g.company_id)) IS NOT TRUE` },
|
||||
{ reason: "mcp_mutation_receipts", query: `SELECT 1 FROM mcp_mutation_receipts m WHERE (
|
||||
|
||||
@@ -0,0 +1,46 @@
|
||||
import type { Db } from "@paperclipai/db";
|
||||
import { describe, expect, it, vi } from "vitest";
|
||||
import { readIdleMcpSleepProof } from "./idle-sleep-proof.js";
|
||||
|
||||
const context = { stackId: "stack", ownerId: "00000000-0000-4000-8000-000000000001",
|
||||
capabilityGeneration: "a".repeat(64), challenge: "c".repeat(64), now: 1234 };
|
||||
function emptyReadOnlyTransaction() {
|
||||
const mutation = vi.fn(() => { throw new Error("read-only inspection mutated state"); });
|
||||
const select = vi.fn(() => {
|
||||
const chain: unknown = new Proxy({}, { get(_target, name) {
|
||||
if (name === "then") return Promise.resolve([]).then.bind(Promise.resolve([]));
|
||||
return () => chain;
|
||||
} });
|
||||
return chain;
|
||||
});
|
||||
return { tx: { select, insert: mutation, update: mutation, delete: mutation, execute: mutation } as unknown as Db, select, mutation };
|
||||
}
|
||||
describe("MCP owner read-only inspection", () => {
|
||||
it("inspects missing settings and absent owners without creating configuration or secrets", async () => {
|
||||
const fixture = emptyReadOnlyTransaction(), beforeRead = vi.fn(async () => {}), check = vi.fn();
|
||||
const proof = await readIdleMcpSleepProof(fixture.tx, {}, { ...context, budget: { beforeRead, check } });
|
||||
expect(proof.ingress.grantIds).toEqual([]);
|
||||
expect(proof.subscriptions).toMatchObject({ blocked: false, subscriptionIds: [] });
|
||||
expect(proof.retries).toMatchObject({ blocked: false, deliveryIds: [], deadlines: [] });
|
||||
expect(fixture.mutation).not.toHaveBeenCalled();
|
||||
expect(fixture.select).toHaveBeenCalledTimes(8);
|
||||
expect(beforeRead).toHaveBeenCalledTimes(fixture.select.mock.calls.length);
|
||||
expect(check).toHaveBeenCalled();
|
||||
});
|
||||
it("stops before issuing the read that exhausts the shared query budget", async () => {
|
||||
const fixture = emptyReadOnlyTransaction(); let remainingReads = 2;
|
||||
await expect(readIdleMcpSleepProof(fixture.tx, {}, { ...context, budget: {
|
||||
beforeRead: async () => { if (remainingReads-- <= 0) throw new Error("query budget exhausted"); }, check: () => {},
|
||||
} })).rejects.toThrow("query budget exhausted");
|
||||
expect(fixture.select).toHaveBeenCalledTimes(2);
|
||||
expect(fixture.mutation).not.toHaveBeenCalled();
|
||||
});
|
||||
it("checks elapsed time after a completed read before more owner work", async () => {
|
||||
const fixture = emptyReadOnlyTransaction();
|
||||
await expect(readIdleMcpSleepProof(fixture.tx, {}, { ...context, budget: {
|
||||
beforeRead: async () => {}, check: () => { throw new Error("inspection deadline exhausted"); },
|
||||
} })).rejects.toThrow("inspection deadline exhausted");
|
||||
expect(fixture.select).toHaveBeenCalledTimes(2);
|
||||
expect(fixture.mutation).not.toHaveBeenCalled();
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,62 @@
|
||||
import { readIdleMcpDeviceStatus } from "./idle-device-status.js";
|
||||
import { getCloudRuntimeIdentity } from "../cloud-runtime-identity.js";
|
||||
import { readIdleMcpDeviceCutover } from "./idle-device-cutover.js";
|
||||
import { readIdleMcpClientMirror } from "./idle-client-mirror.js";
|
||||
import { createHash } from "node:crypto";
|
||||
import type { Db } from "@paperclipai/db";
|
||||
import { instanceSettingsService } from "../instance-settings.js";
|
||||
import type { PublicMcpConfig } from "./oauth.js";
|
||||
import type { PublicMcpEvents } from "./events.js";
|
||||
import { readIdleMcpWakeProof, type IdleMcpWakeProof, type IdleMcpInspectionBudget } from "./idle-wake-proof.js";
|
||||
import { readIdleMcpSubscriptionProof, type IdleMcpSubscriptionProof } from "./idle-subscription-proof.js";
|
||||
import { readIdleMcpEventSchedule, type IdleMcpEventScheduleProof } from "./idle-event-schedule.js";
|
||||
import { readIdleMcpTransferSchedule } from "./idle-transfer-schedule.js";
|
||||
import { readIdleMcpAuthorizationHandoffs } from "./idle-authorization-handoffs.js";
|
||||
import type { IdleDeadlineSnapshot } from "../idle-deadline-types.js";
|
||||
|
||||
const owners = new WeakMap<object, { config: PublicMcpConfig | null; personal: PublicMcpEvents | null; dot: PublicMcpEvents | null; clientMirrorInstanceId?:string }>();
|
||||
export function registerIdleMcpOwners(owner: object, services: { config: PublicMcpConfig | null; personal: PublicMcpEvents | null; dot: PublicMcpEvents | null; clientMirrorInstanceId?:string }) {
|
||||
owners.set(owner, services);
|
||||
return () => { if (owners.get(owner) === services) owners.delete(owner); };
|
||||
}
|
||||
export interface IdleMcpSleepProof {
|
||||
ingress: IdleMcpWakeProof;
|
||||
subscriptions: IdleMcpSubscriptionProof;
|
||||
retries: IdleMcpEventScheduleProof;
|
||||
frontDoor: IdleDeadlineSnapshot;
|
||||
legacyDevices:IdleDeadlineSnapshot;
|
||||
deviceStatus:IdleDeadlineSnapshot;
|
||||
transfers: IdleDeadlineSnapshot;
|
||||
handoffs: IdleDeadlineSnapshot;
|
||||
schedule: IdleDeadlineSnapshot;
|
||||
generation: string;
|
||||
}
|
||||
export async function readIdleMcpSleepProof(tx: Pick<Db, "select">, owner: object, context: {
|
||||
stackId: string; ownerId: string; capabilityGeneration: string; challenge: string; now: number; budget: IdleMcpInspectionBudget;
|
||||
}) : Promise<IdleMcpSleepProof> {
|
||||
const services = owners.get(owner);
|
||||
await context.budget.beforeRead();
|
||||
const feature = await instanceSettingsService(tx as Db).getExperimentalReadOnly();
|
||||
const frontDoor = await readIdleMcpClientMirror(tx,{budget:context.budget,enabled:feature.enablePublicMcp,origin:services?.config?.origin,instanceId:services?.clientMirrorInstanceId,managedStackId:getCloudRuntimeIdentity()?.stackId,authorizationOrigin:services?.config?.authorizationOrigin,unsupportedProfiles:feature.enableOpenAiDot||feature.enableGrokBot&&feature.enableNativeRunner});
|
||||
const ingress = await readIdleMcpWakeProof(tx, { ...context,
|
||||
clientMirror:frontDoor.receipt,
|
||||
personalResource: services?.config?.resource ?? "", dotResource: services?.config ? services.config.origin + "/mcp/runner" : "",
|
||||
discovery: services?.config && feature.enablePublicMcp ? { origin: services.config.origin, authorizationOrigin: services.config.authorizationOrigin ?? services.config.origin,
|
||||
profiles: ["personal-mcp", ...(feature.enableOpenAiDot || feature.enableGrokBot && feature.enableNativeRunner ? ["dot-mcp" as const] : [])] } : undefined,
|
||||
personalEnabled: feature.enablePublicMcp, dotEnabled: feature.enablePublicMcp && feature.enableOpenAiDot });
|
||||
const absent = { revision: "absent", reconciled: false };
|
||||
const personal = services?.personal?.reconciliationSnapshot() ?? absent, dot = services?.dot?.reconciliationSnapshot() ?? absent;
|
||||
const personalDestination = services?.personal?.createIdleDestinationInspector();
|
||||
const dotDestination = services?.dot?.createIdleDestinationInspector();
|
||||
const subscriptions = await readIdleMcpSubscriptionProof(tx, { now: context.now, ingress, personal, dot, budget: context.budget,
|
||||
inspectDestination: row => (row.bindingId ? dotDestination : personalDestination)?.(row) ?? Promise.resolve(false) });
|
||||
const retries = await readIdleMcpEventSchedule(tx, { now: context.now, coveredGrantIds: ingress.grantIds, personal, dot, budget: context.budget });
|
||||
const deviceStatus=await readIdleMcpDeviceStatus(tx,{enabled:feature.enablePublicMcp,receipt:frontDoor.receipt,budget:context.budget});
|
||||
const legacyDevices=feature.enablePublicMcp?await readIdleMcpDeviceCutover(tx,context):{blocked:false,state:"absent",deadlines:[]};
|
||||
const transfers = await readIdleMcpTransferSchedule(tx, context);
|
||||
const handoffs = await readIdleMcpAuthorizationHandoffs(tx, context);
|
||||
const schedule = { blocked: frontDoor.blocked || deviceStatus.blocked || legacyDevices.blocked || retries.blocked || transfers.blocked || handoffs.blocked, state: [frontDoor.state,deviceStatus.state,legacyDevices.state,retries.state,transfers.state,handoffs.state], deadlines: [...legacyDevices.deadlines,...retries.deadlines,...transfers.deadlines,...handoffs.deadlines] };
|
||||
context.budget.check();
|
||||
return { ingress, subscriptions, retries, frontDoor, deviceStatus,legacyDevices, transfers, handoffs, schedule,
|
||||
generation: createHash("sha256").update(JSON.stringify([ingress.generation, subscriptions.generation, schedule])).digest("hex") };
|
||||
}
|
||||
@@ -0,0 +1,44 @@
|
||||
import { describe, expect, it } from "vitest";
|
||||
import { projectIdleMcpSubscriptionProof, type IdleMcpSubscriptionProofInput } from "./idle-subscription-proof.js";
|
||||
import { createMcpWakeAdmission } from "./wake-admission.js";
|
||||
import { mcpEventSubscriptions } from "@paperclipai/db";
|
||||
const context = { stackId: "stack", ownerId: "00000000-0000-4000-8000-000000000001", capabilityGeneration: "a".repeat(64), challenge: "c".repeat(64) };
|
||||
const row: typeof mcpEventSubscriptions.$inferSelect = { id: "sub", companyId: "company", grantId: "grant", name: "paperclip.task.status_changed", taskId: "task", bindingId: null,
|
||||
arguments: { companyId: "company", taskId: "task" }, deliveryMaterial: {}, verifiedAt: new Date(0), startsAt: new Date(0), expiresAt: new Date(10_000), stoppedAt: null, scannedAt: new Date(1000) };
|
||||
const input: IdleMcpSubscriptionProofInput = { now: 2000,
|
||||
ingress: { admission: createMcpWakeAdmission({ ...context, tokens: [] }), generation: "d".repeat(64), grantIds: ["grant"], dotBindingIds: ["binding"] },
|
||||
personal: { revision: "personal:1", reconciled: true }, dot: { revision: "dot:1", reconciled: true }, inspectDestination: async () => true };
|
||||
describe("configured-idle MCP subscription proof", () => {
|
||||
it("retains enabled known subscription state with explicit ingress and reconciled producer", async () => {
|
||||
const proof = await projectIdleMcpSubscriptionProof([{ subscription: row, taskMatches: true }], input);
|
||||
expect(proof.blocked).toBe(false);
|
||||
expect(proof.subscriptionIds).toEqual(["sub"]);
|
||||
expect(row.stoppedAt).toBeNull();
|
||||
expect((await projectIdleMcpSubscriptionProof([{ subscription: { ...row, expiresAt: new Date(1500) }, taskMatches: true }], input)).blocked).toBe(false);
|
||||
expect((await projectIdleMcpSubscriptionProof([{ subscription: row, taskMatches: true }], { ...input, personal: { revision: "personal:2", reconciled: true } })).generation).not.toBe(proof.generation);
|
||||
});
|
||||
it("keeps unknown/foreign/malformed/uncertain subscriptions blocking", async () => {
|
||||
for (const change of [{ name: "unknown.event" }, { grantId: "foreign" }, { taskId: null }, { arguments: { companyId: "foreign", taskId: "task" } },
|
||||
{ arguments: { companyId: "company", taskId: "task", injected: true } }, { arguments: { companyId: "company", taskId: "task", statuses: ["unknown"] } },
|
||||
{ expiresAt: new Date(NaN) }, { verifiedAt: new Date(3000) }, { scannedAt: new Date(-1) }]) {
|
||||
expect((await projectIdleMcpSubscriptionProof([{ subscription: { ...row, ...change }, taskMatches: true }], input)).blocked).toBe(true);
|
||||
}
|
||||
expect((await projectIdleMcpSubscriptionProof([{ subscription: row, taskMatches: false }], input)).blocked).toBe(true);
|
||||
expect((await projectIdleMcpSubscriptionProof([{ subscription: row, taskMatches: true }], { ...input, inspectDestination: async () => false })).blocked).toBe(true);
|
||||
expect((await projectIdleMcpSubscriptionProof([{ subscription: row, taskMatches: true }], { ...input, personal: { revision: "dirty", reconciled: false } })).blocked).toBe(true);
|
||||
});
|
||||
it("requires the exact certified Dot binding and matching Dot reconciliation", async () => {
|
||||
const dot = { ...row, name: "paperclip.dot.mailbox_updated", taskId: null, bindingId: "binding", arguments: { companyId: "company", bindingId: "binding" } };
|
||||
expect((await projectIdleMcpSubscriptionProof([{ subscription: dot, taskMatches: false }], input)).blocked).toBe(false);
|
||||
expect((await projectIdleMcpSubscriptionProof([{ subscription: { ...dot, bindingId: "other" }, taskMatches: false }], input)).blocked).toBe(true);
|
||||
expect((await projectIdleMcpSubscriptionProof([], { ...input, personal: { revision: "missing", reconciled: false } })).blocked).toBe(false);
|
||||
});
|
||||
it("checks the shared inspection deadline between local decryptions", async () => {
|
||||
let expired = false, decryptions = 0;
|
||||
await expect(projectIdleMcpSubscriptionProof([{ subscription: row, taskMatches: true }, { subscription: { ...row, id: "next" }, taskMatches: true }], {
|
||||
...input, budget: { beforeRead: async () => {}, check: () => { if (expired) throw new Error("inspection deadline exhausted"); } },
|
||||
inspectDestination: async () => { decryptions++; expired = true; return true; },
|
||||
})).rejects.toThrow("inspection deadline exhausted");
|
||||
expect(decryptions).toBe(1);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,65 @@
|
||||
import { createHash } from "node:crypto";
|
||||
import { asc, isNull, sql } from "drizzle-orm";
|
||||
import { mcpEventSubscriptions, type Db } from "@paperclipai/db";
|
||||
import { ISSUE_STATUSES } from "@paperclipai/shared";
|
||||
import type { DeadlineReconciliationSnapshot } from "../deadline-reconciliation.js";
|
||||
import type { IdleMcpInspectionBudget, IdleMcpWakeProof } from "./idle-wake-proof.js";
|
||||
|
||||
type Subscription = typeof mcpEventSubscriptions.$inferSelect;
|
||||
export interface IdleMcpSubscriptionProof {
|
||||
blocked: boolean;
|
||||
subscriptionIds: string[];
|
||||
generation: string;
|
||||
}
|
||||
export interface IdleMcpSubscriptionProofInput {
|
||||
budget?: IdleMcpInspectionBudget;
|
||||
now: number;
|
||||
ingress: IdleMcpWakeProof;
|
||||
personal: DeadlineReconciliationSnapshot;
|
||||
dot: DeadlineReconciliationSnapshot;
|
||||
|
||||
|
||||
inspectDestination: (subscription: Subscription) => Promise<boolean>;
|
||||
}
|
||||
|
||||
|
||||
|
||||
export async function readIdleMcpSubscriptionProof(tx: Pick<Db, "select">, input: IdleMcpSubscriptionProofInput): Promise<IdleMcpSubscriptionProof> {
|
||||
await input.budget?.beforeRead();
|
||||
const rows = await tx.select({ subscription: mcpEventSubscriptions,
|
||||
taskMatches: sql<boolean>`EXISTS (SELECT 1 FROM issues i WHERE i.id = ${mcpEventSubscriptions.taskId}
|
||||
AND i.company_id = ${mcpEventSubscriptions.companyId})`,
|
||||
}).from(mcpEventSubscriptions).where(isNull(mcpEventSubscriptions.stoppedAt)).orderBy(asc(mcpEventSubscriptions.id)).limit(1001);
|
||||
return projectIdleMcpSubscriptionProof(rows, input);
|
||||
}
|
||||
|
||||
export async function projectIdleMcpSubscriptionProof(rows: { subscription: Subscription; taskMatches: boolean }[], input: IdleMcpSubscriptionProofInput): Promise<IdleMcpSubscriptionProof> {
|
||||
input.budget?.check();
|
||||
const state: unknown[] = [], subscriptionIds: string[] = [];
|
||||
const result = (blocked: boolean): IdleMcpSubscriptionProof => ({ blocked, subscriptionIds: blocked ? [] : subscriptionIds,
|
||||
generation: createHash("sha256").update(JSON.stringify([input.ingress.generation, input.personal, input.dot, state])).digest("hex") });
|
||||
if (!Number.isFinite(input.now) || rows.length > 1000) return result(true);
|
||||
const grants = new Set(input.ingress.grantIds), bindings = new Set(input.ingress.dotBindingIds);
|
||||
for (const { subscription: row, taskMatches } of rows) {
|
||||
input.budget?.check();
|
||||
const args = row.arguments;
|
||||
const task = row.taskId !== null && row.bindingId === null && taskMatches === true
|
||||
&& ["paperclip.task.status_changed", "paperclip.task.comment_created", "paperclip.task.document_updated"].includes(row.name);
|
||||
const dot = row.bindingId !== null && row.taskId === null && bindings.has(row.bindingId) && row.name === "paperclip.dot.mailbox_updated";
|
||||
const owner = task ? input.personal : dot ? input.dot : undefined;
|
||||
if (!grants.has(row.grantId) || !owner?.reconciled || !owner.revision || !args || typeof args !== "object" || Array.isArray(args)
|
||||
|| args.companyId !== row.companyId || args[task ? "taskId" : "bindingId"] !== (task ? row.taskId : row.bindingId)
|
||||
|| Object.keys(args).some(key => !["companyId", task ? "taskId" : "bindingId", ...(row.name === "paperclip.task.status_changed" ? ["statuses"] : [])].includes(key))
|
||||
|| (args.statuses !== undefined && (!Array.isArray(args.statuses) || !args.statuses.length || args.statuses.some(status => !ISSUE_STATUSES.includes(status as typeof ISSUE_STATUSES[number]))))
|
||||
|| !Number.isFinite(row.startsAt.getTime()) || !Number.isFinite(row.expiresAt.getTime()) || row.expiresAt <= row.startsAt
|
||||
|| !Number.isFinite(row.verifiedAt.getTime()) || row.verifiedAt.getTime() > input.now
|
||||
|| !Number.isFinite(row.scannedAt.getTime()) || (row.expiresAt.getTime() > input.now && row.scannedAt < row.verifiedAt)
|
||||
|| !await input.inspectDestination(row)) return result(true);
|
||||
input.budget?.check();
|
||||
state.push([row.id, row.companyId, row.grantId, row.name, row.taskId, row.bindingId, row.arguments,
|
||||
row.verifiedAt, row.startsAt, row.expiresAt, row.scannedAt,
|
||||
createHash("sha256").update(JSON.stringify(row.deliveryMaterial)).digest("hex")]);
|
||||
subscriptionIds.push(row.id);
|
||||
}
|
||||
return result(false);
|
||||
}
|
||||
@@ -0,0 +1,24 @@
|
||||
import type { Db, mcpFileTickets } from "@paperclipai/db";
|
||||
import { describe, expect, it, vi } from "vitest";
|
||||
import { readIdleMcpTransferSchedule } from "./idle-transfer-schedule.js";
|
||||
type Ticket = Pick<typeof mcpFileTickets.$inferSelect,"companyId"|"grantId"|"uploadId"|"attachmentId"|"workSessionId"|"expiresAt">;
|
||||
const row: Ticket = {companyId:"company",grantId:"grant",uploadId:"upload",attachmentId:null,workSessionId:null,expiresAt:new Date(10_000)};
|
||||
function fixture(rows:Ticket[]) {
|
||||
const select = vi.fn(()=>({from:()=>({where:()=>({orderBy:()=>({limit:async()=>rows})})})}));
|
||||
return {select,tx:{select} as unknown as Pick<Db,"select">};
|
||||
}
|
||||
const budget = {beforeRead:async()=>{},check:()=>{}};
|
||||
describe("live MCP transfer boundary",()=>{
|
||||
it("keeps reusable live tickets blocked and reports the actual bounded expiry",async()=>{
|
||||
const value = await readIdleMcpTransferSchedule(fixture([row]).tx,{now:0,budget});
|
||||
expect(value.blocked).toBe(true);expect(value.deadlines).toEqual([10_000]);
|
||||
expect((await readIdleMcpTransferSchedule(fixture([]).tx,{now:0,budget})).blocked).toBe(false);
|
||||
});
|
||||
it("retains malformed, uncertain and overflowing authority as blockers",async()=>{
|
||||
for(const change of [{expiresAt:new Date(NaN)},{expiresAt:new Date(0)},{companyId:""},{uploadId:null},{attachmentId:"attachment"}]) {
|
||||
const value=await readIdleMcpTransferSchedule(fixture([{...row,...change}]).tx,{now:0,budget});expect(value.blocked).toBe(true);expect(value.deadlines).toEqual([]);
|
||||
}
|
||||
const overflow=await readIdleMcpTransferSchedule(fixture(Array.from({length:1001},()=>row)).tx,{now:0,budget});expect(overflow.blocked).toBe(true);expect(overflow.deadlines).toEqual([]);
|
||||
const f=fixture([]);await expect(readIdleMcpTransferSchedule(f.tx,{now:0,budget:{beforeRead:async()=>{throw new Error("budget exhausted");},check:()=>{}}})).rejects.toThrow("budget exhausted");expect(f.select).not.toHaveBeenCalled();
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,25 @@
|
||||
import { asc, gt } from "drizzle-orm";
|
||||
import { mcpFileTickets, type Db } from "@paperclipai/db";
|
||||
import type { IdleDeadlineSnapshot } from "../idle-deadline-types.js";
|
||||
import type { IdleMcpInspectionBudget } from "./idle-wake-proof.js";
|
||||
|
||||
export async function readIdleMcpTransferSchedule(tx: Pick<Db,"select">, input: { now: number; budget: IdleMcpInspectionBudget }): Promise<IdleDeadlineSnapshot> {
|
||||
await input.budget.beforeRead();
|
||||
const rows = await tx.select({ companyId: mcpFileTickets.companyId, grantId: mcpFileTickets.grantId,
|
||||
uploadId: mcpFileTickets.uploadId, attachmentId: mcpFileTickets.attachmentId, workSessionId: mcpFileTickets.workSessionId,
|
||||
expiresAt: mcpFileTickets.expiresAt }).from(mcpFileTickets)
|
||||
.where(gt(mcpFileTickets.expiresAt,new Date(input.now))).orderBy(asc(mcpFileTickets.expiresAt)).limit(1001);
|
||||
input.budget.check();
|
||||
if (!Number.isFinite(input.now) || rows.length > 1000) return { blocked: true, state: ["incomplete-transfer-inspection"], deadlines: [] };
|
||||
const deadlines: number[] = [], state: unknown[] = [];
|
||||
for (const row of rows) {
|
||||
input.budget.check();
|
||||
const expiresAt = row.expiresAt.getTime();
|
||||
if (!Number.isFinite(expiresAt) || expiresAt <= input.now || !row.companyId || !row.grantId || Boolean(row.uploadId) === Boolean(row.attachmentId)) {
|
||||
return { blocked: true, state: ["uncertain-transfer"], deadlines: [] };
|
||||
}
|
||||
deadlines.push(expiresAt);
|
||||
state.push([row.companyId,row.grantId,row.uploadId,row.attachmentId,row.workSessionId,row.expiresAt.toISOString()]);
|
||||
}
|
||||
return { blocked: rows.length > 0, state, deadlines };
|
||||
}
|
||||
@@ -0,0 +1,37 @@
|
||||
import { createHash } from "node:crypto";
|
||||
import { describe, expect, it } from "vitest";
|
||||
import { projectIdleMcpWakeProof, type IdleMcpCredentialRow } from "./idle-wake-proof.js";
|
||||
import { authorizeMcpWakeAdmission } from "./wake-admission.js";
|
||||
const token = "pcmcp_rt_" + "x".repeat(43);
|
||||
const input = { stackId: "stack", ownerId: "00000000-0000-4000-8000-000000000001", capabilityGeneration: "a".repeat(64), challenge: "c".repeat(64),
|
||||
personalResource: "https://tenant.example/mcp/paperclip", dotResource: "https://tenant.example/mcp/runner", personalEnabled: true, dotEnabled: true, now: 0 };
|
||||
const row: IdleMcpCredentialRow = { grantId: "grant", companyId: "company", clientId: "pcmcp_client_test", resource: input.dotResource, purpose: "agent", scopes: ["paperclip:agent", "offline_access"],
|
||||
tokenHash: createHash("sha256").update(token).digest("hex"), tokenKind: "refresh", tokenExpiresAt: null, tokenUsedAt: null,
|
||||
dotBindingId: "binding", dotProvider: "openai_dot", dotStatus: "connected", dotGeneration: 1, dotPairingHash: null, dotChallengeHash: null,
|
||||
agentStatus: "active", membershipRole: "owner" };
|
||||
describe("MCP credential owner projection", () => {
|
||||
it("retains Dot renewable refresh but binds its actual client and tenant scope", () => {
|
||||
const proof = projectIdleMcpWakeProof([row], input);
|
||||
expect(proof.grantIds).toEqual(["grant"]);
|
||||
expect(proof.dotBindingIds).toEqual(["binding"]);
|
||||
const request = { stackId: "stack", token, clientId: row.clientId, profile: "dot-mcp" as const, tokenKind: "refresh" as const, now: 0 };
|
||||
expect(authorizeMcpWakeAdmission(proof.admission, request)).toBeDefined();
|
||||
expect(authorizeMcpWakeAdmission(proof.admission, { ...request, clientId: "other" })).toBeUndefined();
|
||||
expect(JSON.stringify(proof.admission)).not.toContain(row.tokenHash);
|
||||
});
|
||||
it("preserves inactive, unfamiliar, pairing and changed-authority obligations", () => {
|
||||
for (const change of [{ tokenUsedAt: new Date() }, { dotProvider: "grok_bot" }, { dotStatus: "pairing" }, { dotGeneration: 0 },
|
||||
{ dotPairingHash: "pending" }, { dotChallengeHash: "pending" }, { membershipRole: "viewer" }, { agentStatus: "terminated" },
|
||||
{ resource: "https://other.example/mcp/runner" }, { tokenExpiresAt: new Date(0) }]) {
|
||||
expect(projectIdleMcpWakeProof([{ ...row, ...change }], input).grantIds).toEqual([]);
|
||||
}
|
||||
expect(projectIdleMcpWakeProof([row], { ...input, dotEnabled: false }).grantIds).toEqual([]);
|
||||
});
|
||||
it("does not export an unbounded personal/access token, used refresh, or overlarge projection", () => {
|
||||
expect(projectIdleMcpWakeProof([{ ...row, tokenKind: "access" }], input).grantIds).toEqual([]);
|
||||
const personal = { ...row, resource: input.personalResource, purpose: "personal", scopes: ["paperclip:read"], tokenExpiresAt: new Date(1000) };
|
||||
expect(projectIdleMcpWakeProof([personal], input).grantIds).toEqual(["grant"]);
|
||||
expect(projectIdleMcpWakeProof([personal], input).dotBindingIds).toEqual([]);
|
||||
expect(() => projectIdleMcpWakeProof(Array.from({ length: 1001 }, () => row), input)).toThrow();
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,163 @@
|
||||
import type { McpClientMirrorReceipt } from "./client-mirror-contract.js";
|
||||
import type { McpDiscovery } from "./discovery.js";
|
||||
import { createHash } from "node:crypto";
|
||||
import { PUBLIC_MCP_SCOPES, DOT_RUNNER_MCP_SCOPES } from "@paperclipai/shared";
|
||||
import { and, asc, eq, gt, isNull, or, sql } from "drizzle-orm";
|
||||
import {
|
||||
agents, authUsers, companies, companyMemberships, dotAgentBindings,
|
||||
mcpOauthGrants, mcpOauthTokens, type Db,
|
||||
} from "@paperclipai/db";
|
||||
import { mcpRevocationAuthorityId, parseMcpRevocationIndexReceipt, type McpRevocationIndexReceipt } from "./revocation-index-contract.js";
|
||||
import { createMcpWakeAdmission, type McpWakeAdmission, type McpWakeProfile, type McpWakeTokenKind } from "./wake-admission.js";
|
||||
|
||||
export interface IdleMcpWakeProof {
|
||||
|
||||
admission: McpWakeAdmission;
|
||||
generation: string;
|
||||
grantIds: string[];
|
||||
dotBindingIds: string[];
|
||||
}
|
||||
export interface IdleMcpCredentialRow {
|
||||
grantId: string;
|
||||
companyId: string;
|
||||
clientId: string;
|
||||
resource: string;
|
||||
purpose: string;
|
||||
scopes: string[];
|
||||
tokenHash: string;
|
||||
tokenKind: string;
|
||||
tokenExpiresAt: Date | null;
|
||||
tokenUsedAt: Date | null;
|
||||
dotBindingId: string | null;
|
||||
dotProvider: string | null;
|
||||
dotStatus: string | null;
|
||||
dotGeneration: number | null;
|
||||
dotPairingHash: string | null;
|
||||
dotChallengeHash: string | null;
|
||||
agentStatus: string | null;
|
||||
membershipRole: string | null;
|
||||
}
|
||||
export interface IdleMcpInspectionBudget { beforeRead(): Promise<void>; check(): void }
|
||||
export interface IdleMcpWakeProofInput {
|
||||
budget?: IdleMcpInspectionBudget;
|
||||
clientMirror?: McpClientMirrorReceipt;
|
||||
discovery?: McpDiscovery;
|
||||
stackId: string;
|
||||
ownerId: string;
|
||||
capabilityGeneration: string;
|
||||
challenge: string;
|
||||
personalResource: string;
|
||||
dotResource: string;
|
||||
personalEnabled: boolean;
|
||||
dotEnabled: boolean;
|
||||
now: number;
|
||||
}
|
||||
|
||||
|
||||
export function projectIdleMcpWakeProof(rows: IdleMcpCredentialRow[], input: IdleMcpWakeProofInput): IdleMcpWakeProof {
|
||||
input.budget?.check();
|
||||
if (rows.length > 1000 || !Number.isFinite(input.now)) throw new Error("MCP wake inspection is incomplete");
|
||||
const tokens: { profile: McpWakeProfile; tokenKind: McpWakeTokenKind; tokenHash: string; expiresAt: string | null; clientId: string }[] = [];
|
||||
const grantIds = new Set<string>(), dotBindingIds = new Set<string>();
|
||||
for (const row of rows) {
|
||||
input.budget?.check();
|
||||
const personal = input.personalEnabled && row.resource === input.personalResource && row.purpose === "personal"
|
||||
&& row.scopes.includes("paperclip:read") && row.scopes.every(scope => (PUBLIC_MCP_SCOPES as readonly string[]).includes(scope));
|
||||
const dot = input.dotEnabled && row.resource === input.dotResource && row.purpose === "agent"
|
||||
&& row.scopes.includes("paperclip:agent") && row.scopes.every(scope => (DOT_RUNNER_MCP_SCOPES as readonly string[]).includes(scope)) && row.membershipRole !== "viewer"
|
||||
&& row.dotProvider === "openai_dot" && ["connected", "ready"].includes(row.dotStatus ?? "")
|
||||
&& row.dotBindingId !== null && row.dotGeneration !== null && Number.isSafeInteger(row.dotGeneration) && row.dotGeneration > 0
|
||||
&& row.dotPairingHash === null && row.dotChallengeHash === null
|
||||
&& row.agentStatus !== null && !["terminated", "pending_approval"].includes(row.agentStatus);
|
||||
if (!personal && !dot) continue;
|
||||
if (row.tokenKind !== "access" && row.tokenKind !== "refresh") throw new Error("Unknown MCP credential kind");
|
||||
if (row.tokenKind === "refresh" && row.tokenUsedAt !== null) continue;
|
||||
if (row.tokenExpiresAt === null ? !dot || row.tokenKind !== "refresh" : row.tokenExpiresAt.getTime() <= input.now) continue;
|
||||
tokens.push({ profile: personal ? "personal-mcp" : "dot-mcp", tokenKind: row.tokenKind,
|
||||
tokenHash: row.tokenHash, expiresAt: row.tokenExpiresAt?.toISOString() ?? null, clientId: row.clientId });
|
||||
grantIds.add(row.grantId);
|
||||
if (dot) dotBindingIds.add(row.dotBindingId!);
|
||||
}
|
||||
const admission = createMcpWakeAdmission({ stackId: input.stackId, ownerId: input.ownerId,
|
||||
capabilityGeneration: input.capabilityGeneration, challenge: input.challenge, tokens, discovery: input.discovery, clientMirror: input.clientMirror,...(!input.personalEnabled&&!input.dotEnabled?{absent:true as const}:{}) });
|
||||
return { admission, generation: createHash("sha256").update(JSON.stringify([admission.generation, rows.map(({ tokenHash: _private, ...row }) => row)])).digest("hex"),
|
||||
grantIds: [...grantIds].sort(), dotBindingIds: [...dotBindingIds].sort() };
|
||||
}
|
||||
|
||||
|
||||
|
||||
|
||||
export async function readIdleMcpWakeProof(tx: Pick<Db, "select">, input: IdleMcpWakeProofInput): Promise<IdleMcpWakeProof> {
|
||||
await input.budget?.beforeRead();
|
||||
const rows = await tx.select({
|
||||
grantId: mcpOauthGrants.id, companyId: mcpOauthGrants.companyId, clientId: mcpOauthGrants.clientId,
|
||||
resource: mcpOauthGrants.resource, purpose: mcpOauthGrants.purpose, scopes: mcpOauthGrants.scopes,
|
||||
tokenHash: mcpOauthTokens.tokenHash, tokenKind: mcpOauthTokens.kind,
|
||||
tokenExpiresAt: mcpOauthTokens.expiresAt, tokenUsedAt: mcpOauthTokens.usedAt,
|
||||
dotBindingId: dotAgentBindings.id, dotProvider: dotAgentBindings.providerKind,
|
||||
dotStatus: dotAgentBindings.status, dotGeneration: dotAgentBindings.generation,
|
||||
dotPairingHash: dotAgentBindings.pairingCodeHash, dotChallengeHash: dotAgentBindings.challengeHash,
|
||||
agentStatus: agents.status, membershipRole: companyMemberships.membershipRole,
|
||||
}).from(mcpOauthTokens)
|
||||
.innerJoin(mcpOauthGrants, and(eq(mcpOauthGrants.id, mcpOauthTokens.grantId), isNull(mcpOauthGrants.revokedAt)))
|
||||
.innerJoin(companies, and(eq(companies.id, mcpOauthGrants.companyId), sql`${companies.status} <> 'archived'`))
|
||||
.innerJoin(authUsers, eq(authUsers.id, mcpOauthGrants.userId))
|
||||
.innerJoin(companyMemberships, and(eq(companyMemberships.companyId, mcpOauthGrants.companyId),
|
||||
eq(companyMemberships.principalId, mcpOauthGrants.userId), eq(companyMemberships.principalType, "user"), eq(companyMemberships.status, "active")))
|
||||
.leftJoin(agents, and(eq(agents.id, mcpOauthGrants.agentId), eq(agents.companyId, mcpOauthGrants.companyId)))
|
||||
.leftJoin(dotAgentBindings, and(eq(dotAgentBindings.grantId, mcpOauthGrants.id),
|
||||
eq(dotAgentBindings.companyId, mcpOauthGrants.companyId), eq(dotAgentBindings.agentId, mcpOauthGrants.agentId),
|
||||
eq(dotAgentBindings.operatorId, mcpOauthGrants.userId), isNull(dotAgentBindings.revokedAt)))
|
||||
.where(and(or(gt(mcpOauthTokens.expiresAt, new Date(input.now)), isNull(mcpOauthTokens.expiresAt)),
|
||||
|
||||
|
||||
sql`(${mcpOauthTokens.kind} <> 'refresh' OR ${mcpOauthTokens.usedAt} IS NULL)`))
|
||||
.orderBy(asc(mcpOauthTokens.id)).limit(1001);
|
||||
input.budget?.check();
|
||||
const projected = projectIdleMcpWakeProof(rows, input);
|
||||
await input.budget?.beforeRead();
|
||||
const authorities = await tx.select({ grant: mcpOauthGrants,
|
||||
dotBindingId: dotAgentBindings.id, dotProvider: dotAgentBindings.providerKind, dotStatus: dotAgentBindings.status,
|
||||
dotGeneration: dotAgentBindings.generation, dotPairingHash: dotAgentBindings.pairingCodeHash,
|
||||
dotChallengeHash: dotAgentBindings.challengeHash, agentStatus: agents.status,
|
||||
membershipRole: companyMemberships.membershipRole,
|
||||
}).from(mcpOauthGrants)
|
||||
.innerJoin(companies, and(eq(companies.id, mcpOauthGrants.companyId), sql`${companies.status} <> 'archived'`))
|
||||
.innerJoin(authUsers, eq(authUsers.id, mcpOauthGrants.userId))
|
||||
.innerJoin(companyMemberships, and(eq(companyMemberships.companyId, mcpOauthGrants.companyId),
|
||||
eq(companyMemberships.principalId, mcpOauthGrants.userId), eq(companyMemberships.principalType, "user"), eq(companyMemberships.status, "active")))
|
||||
.leftJoin(agents, and(eq(agents.id, mcpOauthGrants.agentId), eq(agents.companyId, mcpOauthGrants.companyId)))
|
||||
.leftJoin(dotAgentBindings, and(eq(dotAgentBindings.grantId, mcpOauthGrants.id), eq(dotAgentBindings.companyId, mcpOauthGrants.companyId),
|
||||
eq(dotAgentBindings.agentId, mcpOauthGrants.agentId), eq(dotAgentBindings.operatorId, mcpOauthGrants.userId), isNull(dotAgentBindings.revokedAt)))
|
||||
.where(isNull(mcpOauthGrants.revokedAt)).orderBy(asc(mcpOauthGrants.id)).limit(1001);
|
||||
if (authorities.length > 1000) throw new Error("Too many active MCP grant authorities");
|
||||
const grantIds: string[] = [], dotBindingIds: string[] = [], revocationAuthorities: McpRevocationIndexReceipt[] = [];
|
||||
for (const row of authorities) {
|
||||
input.budget?.check();
|
||||
const grant = row.grant;
|
||||
const personal = input.personalEnabled && grant.purpose === "personal" && grant.resource === input.personalResource
|
||||
&& grant.scopes.includes("paperclip:read") && grant.scopes.every(scope => (PUBLIC_MCP_SCOPES as readonly string[]).includes(scope));
|
||||
const dot = input.dotEnabled && grant.purpose === "agent" && grant.resource === input.dotResource
|
||||
&& grant.scopes.includes("paperclip:agent") && grant.scopes.every(scope => (DOT_RUNNER_MCP_SCOPES as readonly string[]).includes(scope))
|
||||
&& row.membershipRole !== "viewer" && row.dotProvider === "openai_dot" && ["connected", "ready"].includes(row.dotStatus ?? "")
|
||||
&& row.dotBindingId !== null && row.dotGeneration !== null && Number.isSafeInteger(row.dotGeneration) && row.dotGeneration > 0
|
||||
&& row.dotPairingHash === null && row.dotChallengeHash === null && row.agentStatus !== null && !["terminated", "pending_approval"].includes(row.agentStatus);
|
||||
if ((!personal && !dot) || !grant.revocationIndexReceipt) continue;
|
||||
const receipt = parseMcpRevocationIndexReceipt(grant.revocationIndexReceipt);
|
||||
if (receipt.revision !== grant.revocationRevision || receipt.authorityId !== mcpRevocationAuthorityId(grant.id, receipt.keyId)) continue;
|
||||
grantIds.push(grant.id); if (dot) dotBindingIds.push(row.dotBindingId!); revocationAuthorities.push(receipt);
|
||||
}
|
||||
if (new Set(revocationAuthorities.map(receipt => receipt.keyId)).size > 1) throw new Error("MCP revocation key rotation is incomplete");
|
||||
const allowed = new Set(grantIds);
|
||||
const tokensForAdmission = rows.filter(row => allowed.has(row.grantId));
|
||||
const rebuilt = createMcpWakeAdmission({ stackId: input.stackId, ownerId: input.ownerId, capabilityGeneration: input.capabilityGeneration,
|
||||
challenge: input.challenge, revocationAuthorities, discovery: input.discovery, clientMirror: input.clientMirror,...(!input.personalEnabled&&!input.dotEnabled?{absent:true as const}:{}),
|
||||
tokens: tokensForAdmission.filter(row => !(row.tokenKind === "refresh" && row.tokenUsedAt !== null)).map(row => ({
|
||||
profile: row.resource === input.personalResource ? "personal-mcp" as const : "dot-mcp" as const,
|
||||
tokenKind: row.tokenKind as "access" | "refresh", tokenHash: row.tokenHash, expiresAt: row.tokenExpiresAt?.toISOString() ?? null, clientId: row.clientId,
|
||||
})) });
|
||||
input.budget?.check();
|
||||
return { admission: rebuilt, grantIds: grantIds.sort(), dotBindingIds: dotBindingIds.sort(),
|
||||
generation: createHash("sha256").update(JSON.stringify([projected.generation, authorities.map(({ grant, ...authority }) => [grant.id,grant.revocationRevision,grant.revocationIndexReceipt,authority]),revocationAuthorities])).digest("hex") };
|
||||
|
||||
}
|
||||
@@ -0,0 +1,31 @@
|
||||
import { sql, type SQL } from "drizzle-orm";
|
||||
import { idleMcpHistoryChecks } from "./idle-history-proof.js";
|
||||
import type { IdleMcpWakeProof } from "./idle-wake-proof.js";
|
||||
import type { IdleMcpSubscriptionProof } from "./idle-subscription-proof.js";
|
||||
import type { IdleMcpEventScheduleProof } from "./idle-event-schedule.js";
|
||||
import type { IdleDeadlineSnapshot } from "../idle-deadline-types.js";
|
||||
|
||||
|
||||
|
||||
|
||||
export function idleMcpWorkQuery(reason: string, input: {
|
||||
ingress?: IdleMcpWakeProof;
|
||||
subscriptions?: IdleMcpSubscriptionProof;
|
||||
retries?: IdleMcpEventScheduleProof;
|
||||
transfers?: IdleDeadlineSnapshot;
|
||||
handoffs?: IdleDeadlineSnapshot;
|
||||
}): SQL | undefined {
|
||||
const check = idleMcpHistoryChecks.find(check => check.reason === reason);
|
||||
if (!check || !input.ingress) return;
|
||||
if (reason === "mcp_file_tickets" && input.transfers) return sql`SELECT 1 WHERE ${input.transfers.blocked}`;
|
||||
if (reason === "mcp_authorization_handoffs" && input.handoffs) return sql`SELECT 1 WHERE ${input.handoffs.blocked}`;
|
||||
if (reason === "mcp_oauth_grants" && input.ingress.grantIds.length) {
|
||||
return sql`${sql.raw(check.query)} AND g.id NOT IN (${sql.join(input.ingress.grantIds.map(id => sql`${id}::uuid`), sql`, `)})`;
|
||||
}
|
||||
if (reason === "mcp_event_subscriptions" && input.subscriptions?.blocked === false && input.subscriptions.subscriptionIds.length) {
|
||||
return sql`${sql.raw(check.query)} AND s.id NOT IN (${sql.join(input.subscriptions.subscriptionIds.map(id => sql`${id}`), sql`, `)})`;
|
||||
}
|
||||
if (reason === "mcp_event_deliveries" && input.retries?.blocked === false && input.retries.deliveryIds.length) {
|
||||
return sql`${sql.raw(check.query)} AND d.id NOT IN (${sql.join(input.retries.deliveryIds.map(id => sql`${id}::uuid`), sql`, `)})`;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,39 @@
|
||||
import { instanceSettingsService } from "../instance-settings.js";
|
||||
import { managedMcpDeviceScope } from "./device-cutover.js";
|
||||
import { pendingLegacyMcpDevicePredicate } from "./idle-device-cutover.js";
|
||||
import { pendingMcpCodePredicate, pendingMcpDevicePredicate } from "./idle-authorization-handoffs.js";
|
||||
import { asc, gt } from "drizzle-orm";
|
||||
import { mcpFileTickets, mcpOauthRequests, mcpOauthDeviceRequests, type Db } from "@paperclipai/db";
|
||||
import { DELIVERY_QUEUES } from "../delivery-work-notifications.js";
|
||||
import type { createDeliveryWorkCoordinator } from "../delivery-work-coordinator.js";
|
||||
|
||||
export function startMcpIngressHandoffExpiry(db: Db, coordinator: Pick<ReturnType<typeof createDeliveryWorkCoordinator>,"register">) {
|
||||
let requested = true, nextDeadline: number | null = null, revision = 0;
|
||||
return coordinator.register(DELIVERY_QUEUES.mcpIngressHandoffs, {
|
||||
retryMs: 5000, pausePassiveMaintenance: true,
|
||||
onSettled: () => { revision++; requested = true; },
|
||||
async run(signal) {
|
||||
if (!requested && (nextDeadline === null || nextDeadline > Date.now())) return;
|
||||
requested = true;
|
||||
const inspectedRevision = revision;
|
||||
signal.throwIfAborted();
|
||||
const [next] = await db.select({ expiresAt: mcpFileTickets.expiresAt }).from(mcpFileTickets)
|
||||
.where(gt(mcpFileTickets.expiresAt,new Date())).orderBy(asc(mcpFileTickets.expiresAt)).limit(1);
|
||||
signal.throwIfAborted();
|
||||
const [code] = await db.select({expiresAt:mcpOauthRequests.expiresAt}).from(mcpOauthRequests).where(pendingMcpCodePredicate(Date.now())).orderBy(asc(mcpOauthRequests.expiresAt)).limit(1);
|
||||
signal.throwIfAborted();
|
||||
const [device] = await db.select({expiresAt:mcpOauthDeviceRequests.expiresAt}).from(mcpOauthDeviceRequests).where(pendingMcpDevicePredicate(Date.now())).orderBy(asc(mcpOauthDeviceRequests.expiresAt)).limit(1);
|
||||
signal.throwIfAborted();
|
||||
const legacyEnabled=managedMcpDeviceScope()?(await instanceSettingsService(db).getExperimentalReadOnly()).enablePublicMcp:false;
|
||||
const legacyPredicate=legacyEnabled?pendingLegacyMcpDevicePredicate(db,Date.now()):undefined;
|
||||
const [legacy]=legacyPredicate?await db.select({expiresAt:mcpOauthDeviceRequests.expiresAt}).from(mcpOauthDeviceRequests).where(legacyPredicate).orderBy(asc(mcpOauthDeviceRequests.expiresAt)).limit(1):[];
|
||||
signal.throwIfAborted();
|
||||
const candidates = [next,code,device,legacy].filter(row=>row!==undefined).map(row=>row.expiresAt.getTime());
|
||||
const at = candidates.length ? Math.min(...candidates) : undefined;
|
||||
if (at !== undefined && !Number.isFinite(at)) throw new Error("MCP ingress handoff expiry is uncertain");
|
||||
nextDeadline = at ?? null; requested = inspectedRevision !== revision;
|
||||
},
|
||||
hasPending: async () => false,
|
||||
nextRunAt: () => requested ? Date.now() + 5000 : nextDeadline,
|
||||
});
|
||||
}
|
||||
@@ -1,8 +1,11 @@
|
||||
import { consumeMcpDeviceMintContext,assertCurrentMcpDeviceMint,type McpDeviceMintContext } from "./device-mint-authority.js";
|
||||
import { requiresMcpDeviceBroker } from "./device-cutover.js";
|
||||
import { prepareMcpClientMutation, recordMcpClientChange } from "./client-mirror-mutations.js";
|
||||
import { DELIVERY_QUEUES, notifyDeliveryWork } from "../delivery-work-notifications.js";
|
||||
import { authorizeExternalBotOperator, externalBotEnabled } from "../external-bot-authority.js";
|
||||
import { updateAgentConfigurationInTransaction } from "../agent-configuration-transaction.js";
|
||||
import { instanceSettingsService } from "../instance-settings.js";
|
||||
import { createHash, randomBytes } from "node:crypto";
|
||||
import { createHash, randomBytes, randomUUID } from "node:crypto";
|
||||
import { canConfigureAgentConnection } from "../../modules/agent-lifecycle/index.js";
|
||||
import { and, eq, gt, inArray, isNull, lt, lte, notExists, sql } from "drizzle-orm";
|
||||
import { z } from "zod";
|
||||
@@ -153,11 +156,15 @@ export function createPublicMcpOAuth(db: Db, config: PublicMcpConfig, options: {
|
||||
}
|
||||
|
||||
async function issueTokens(tx: Db, grant: typeof mcpOauthGrants.$inferSelect) {
|
||||
await notifyDeliveryWork(tx, DELIVERY_QUEUES.mcpRevocationIndex, grant.id);
|
||||
const [indexed] = await tx.update(mcpOauthGrants).set({ revocationRevision: sql`${mcpOauthGrants.revocationRevision} + 1` })
|
||||
.where(eq(mcpOauthGrants.id, grant.id)).returning({ revision: mcpOauthGrants.revocationRevision });
|
||||
if (!indexed) throw invalidGrant();
|
||||
const access = secret("pcmcp_at_");
|
||||
const refresh = grant.scopes.includes("offline_access") ? secret("pcmcp_rt_") : null;
|
||||
await tx.insert(mcpOauthTokens).values([
|
||||
{ grantId: grant.id, tokenHash: hashMcpSecret(access), kind: "access", expiresAt: new Date(Date.now() + accessLifetime) },
|
||||
...(refresh ? [{ grantId: grant.id, tokenHash: hashMcpSecret(refresh), kind: "refresh" as const,
|
||||
{ grantId: grant.id, revocationRevision: indexed.revision, tokenHash: hashMcpSecret(access), kind: "access", expiresAt: new Date(Date.now() + accessLifetime) },
|
||||
...(refresh ? [{ grantId: grant.id, revocationRevision: indexed.revision, tokenHash: hashMcpSecret(refresh), kind: "refresh" as const,
|
||||
expiresAt: dotAgentConnection && grant.purpose === "agent" ? null : new Date(Date.now() + refreshLifetime) }] : []),
|
||||
]);
|
||||
if (museAgentConnection) await tx.update(museAgentBindings).set({ workerActivityAt: new Date() })
|
||||
@@ -207,24 +214,27 @@ export function createPublicMcpOAuth(db: Db, config: PublicMcpConfig, options: {
|
||||
if (!client.id.startsWith("https://")) return;
|
||||
const sourceHash = hashMcpSecret(config.resource + ":" + source);
|
||||
// Caller holds the shared registration lock and has admitted this request.
|
||||
await prepareMcpClientMutation(tx);
|
||||
await pruneClients(tx);
|
||||
const [existing] = await tx.select().from(mcpOauthClients).where(eq(mcpOauthClients.id, client.id));
|
||||
if (!existing) await admitClient(tx, sourceHash);
|
||||
const value = { name: client.name, redirectUris: client.redirectUris, grantTypes: client.grantTypes };
|
||||
await tx.insert(mcpOauthClients).values({ id: client.id, registrationSourceHash: sourceHash, ...value })
|
||||
.onConflictDoUpdate({ target: mcpOauthClients.id, set: value });
|
||||
await recordMcpClientChange(tx,client.id,"upsert");
|
||||
}
|
||||
|
||||
async function pruneClients(tx: Db) {
|
||||
const now = new Date();
|
||||
await tx.delete(mcpOauthRequests).where(lt(mcpOauthRequests.expiresAt, now));
|
||||
await tx.delete(mcpOauthDeviceRequests).where(lt(mcpOauthDeviceRequests.expiresAt, now));
|
||||
await tx.delete(mcpOauthClients).where(and(
|
||||
const deleted = await tx.delete(mcpOauthClients).where(and(
|
||||
lt(mcpOauthClients.createdAt, new Date(now.getTime() - 60 * minute)),
|
||||
notExists(tx.select({ id: mcpOauthGrants.id }).from(mcpOauthGrants).where(eq(mcpOauthGrants.clientId, mcpOauthClients.id))),
|
||||
notExists(tx.select({ id: mcpOauthRequests.id }).from(mcpOauthRequests).where(eq(mcpOauthRequests.clientId, mcpOauthClients.id))),
|
||||
notExists(tx.select({ id: mcpOauthDeviceRequests.id }).from(mcpOauthDeviceRequests).where(eq(mcpOauthDeviceRequests.clientId, mcpOauthClients.id))),
|
||||
));
|
||||
)).returning({id:mcpOauthClients.id});
|
||||
for (const row of deleted) await recordMcpClientChange(tx,row.id,"delete");
|
||||
}
|
||||
|
||||
async function admitClient(tx: Db, sourceHash: string) {
|
||||
@@ -263,7 +273,10 @@ export function createPublicMcpOAuth(db: Db, config: PublicMcpConfig, options: {
|
||||
if ((input.allowWrites || input.allowConfiguration) && membership.membershipRole === "viewer") throw new McpOAuthError("access_denied", "Viewer access is read-only.", 403);
|
||||
const scopes = request.scopes.filter(s => (s !== "paperclip:write" || input.allowWrites)
|
||||
&& (s !== "paperclip:configure" || input.allowConfiguration === true));
|
||||
const [grant] = await tx.insert(mcpOauthGrants).values({ companyId: input.companyId!, userId: actor.userId, clientId: request.clientId, resource: request.resource, scopes, purpose: agentConnection ? "agent" : "personal" }).returning();
|
||||
const grantId=randomUUID();
|
||||
await notifyDeliveryWork(tx, DELIVERY_QUEUES.mcpIngressHandoffs);
|
||||
await notifyDeliveryWork(tx, DELIVERY_QUEUES.mcpRevocationIndex, grantId);
|
||||
const [grant] = await tx.insert(mcpOauthGrants).values({ id:grantId, companyId: input.companyId!, userId: actor.userId, clientId: request.clientId, resource: request.resource, scopes, purpose: agentConnection ? "agent" : "personal" }).returning();
|
||||
await tx.insert(activityLog).values({ companyId: grant!.companyId, actorType: "user", actorId: actor.userId,
|
||||
action: "mcp.connection_authorized", entityType: "mcp_connection", entityId: grant!.id, details: { clientId: grant!.clientId, scopes } });
|
||||
return grant!;
|
||||
@@ -273,6 +286,7 @@ export function createPublicMcpOAuth(db: Db, config: PublicMcpConfig, options: {
|
||||
async function deviceToken(input: Record<string, unknown>, clientId: string) {
|
||||
if (typeof input.device_code !== "string") throw invalidGrant();
|
||||
const outcome = await db.transaction(async (tx) => {
|
||||
await tx.execute(sql`SELECT pg_advisory_xact_lock(736721042)`);
|
||||
const [row] = await tx.select().from(mcpOauthDeviceRequests).where(eq(mcpOauthDeviceRequests.deviceCodeHash, hashMcpSecret(input.device_code as string))).for("update");
|
||||
if (!row || row.clientId !== clientId || row.resource !== config.resource || row.status === "consumed") return { error: "invalid_grant" };
|
||||
const now = new Date();
|
||||
@@ -283,11 +297,13 @@ export function createPublicMcpOAuth(db: Db, config: PublicMcpConfig, options: {
|
||||
await tx.update(mcpOauthDeviceRequests).set({ intervalSeconds, nextPollAt: new Date(now.getTime() + intervalSeconds * 1000) }).where(eq(mcpOauthDeviceRequests.id, row.id));
|
||||
return { error: "slow_down" };
|
||||
}
|
||||
if (row.status === "approved") await notifyDeliveryWork(tx, DELIVERY_QUEUES.mcpIngressHandoffs);
|
||||
await tx.update(mcpOauthDeviceRequests).set({ nextPollAt: new Date(now.getTime() + row.intervalSeconds * 1000) }).where(eq(mcpOauthDeviceRequests.id, row.id));
|
||||
if (row.status !== "approved" || !row.grantId) return { error: "authorization_pending" };
|
||||
const [grant] = await tx.select().from(mcpOauthGrants).where(eq(mcpOauthGrants.id, row.grantId)).for("update");
|
||||
if (!grant) return { error: "invalid_grant" };
|
||||
await actorForGrant(grant, tx as unknown as Db);
|
||||
await prepareMcpClientMutation(tx as unknown as Db);
|
||||
await tx.update(mcpOauthDeviceRequests).set({ status: "consumed" }).where(eq(mcpOauthDeviceRequests.id, row.id));
|
||||
return { tokens: await issueTokens(tx as unknown as Db, grant) };
|
||||
});
|
||||
@@ -375,6 +391,7 @@ export function createPublicMcpOAuth(db: Db, config: PublicMcpConfig, options: {
|
||||
throw error;
|
||||
}
|
||||
decideMuseCodeExchange(observation,[],row);
|
||||
if (!museAgentConnection) await notifyDeliveryWork(tx, DELIVERY_QUEUES.mcpIngressHandoffs);
|
||||
await tx.update(mcpOauthRequests).set({consumedAt:new Date()}).where(eq(mcpOauthRequests.id,row.id));
|
||||
const result=await issueTokens(tx as unknown as Db,grant);
|
||||
if(observation)await recordMuseCodeSuccess(tx as unknown as Db,observation,row,grant);
|
||||
@@ -412,7 +429,7 @@ export function createPublicMcpOAuth(db: Db, config: PublicMcpConfig, options: {
|
||||
const [row] = await db.select({ resource: mcpOauthDeviceRequests.resource }).from(mcpOauthDeviceRequests).where(eq(mcpOauthDeviceRequests.userCodeHash, deviceHash(userCode)));
|
||||
return row?.resource === config.resource;
|
||||
},
|
||||
async deviceAuthorize(input: unknown, source = "unknown") {
|
||||
async deviceAuthorize(input: unknown, source = "unknown", context?:McpDeviceMintContext) {
|
||||
if (museAgentConnection) throw new McpOAuthError("unsupported_grant_type", "Muse requires public-client PKCE authorization.");
|
||||
await assertEnabled();
|
||||
const p = z.object({ client_id: z.string().min(1).max(2048), resource: z.literal(config.resource), scope: z.string().max(200).default(requiredScope), company_id: z.uuid().optional() }).safeParse(input);
|
||||
@@ -421,6 +438,7 @@ export function createPublicMcpOAuth(db: Db, config: PublicMcpConfig, options: {
|
||||
if (!scopes.includes(requiredScope) || scopes.some(s => !(scopesSupported as readonly string[]).includes(s))) throw new McpOAuthError("invalid_scope", "Unsupported Paperclip scope.");
|
||||
const client = await resolveClient(p.data.client_id, source);
|
||||
if (!client?.grantTypes.includes(DEVICE_GRANT)) throw new McpOAuthError("unauthorized_client", "Register a device authorization client.");
|
||||
const mintAuthority=consumeMcpDeviceMintContext(context,config.resource);
|
||||
const deviceCode = secret("pcmcp_device_");
|
||||
const alphabet = "BCDFGHJKLMNPQRSTVWXYZ23456789";
|
||||
const chars = Array.from(randomBytes(8), byte => alphabet[byte % alphabet.length]).join("");
|
||||
@@ -428,13 +446,16 @@ export function createPublicMcpOAuth(db: Db, config: PublicMcpConfig, options: {
|
||||
const sourceHash = hashMcpSecret(config.resource + ":" + source);
|
||||
await db.transaction(async tx => {
|
||||
await tx.execute(sql`SELECT pg_advisory_xact_lock(736721042)`);
|
||||
if(mintAuthority)assertCurrentMcpDeviceMint(mintAuthority,config.resource);
|
||||
if(await requiresMcpDeviceBroker(tx as unknown as Db,config.resource)&&!mintAuthority)throw new McpOAuthError("invalid_request","Use the canonical Paperclip device authorization endpoint.");
|
||||
await prepareMcpClientMutation(tx as unknown as Db);
|
||||
await tx.execute(sql`SELECT pg_advisory_xact_lock(736721043)`);
|
||||
const now = new Date();
|
||||
await tx.delete(mcpOauthDeviceRequests).where(lt(mcpOauthDeviceRequests.expiresAt, now));
|
||||
const [count] = await tx.select({ total: sql<number>`count(*)::int`, source: sql<number>`(count(*) FILTER (WHERE ${mcpOauthDeviceRequests.sourceHash} = ${sourceHash}))::int` }).from(mcpOauthDeviceRequests);
|
||||
if (!count || count.total >= 1000 || count.source >= 10) throw new McpOAuthError("temporarily_unavailable", "Too many device connection attempts. Try again later.", 429);
|
||||
await retainMetadataClient(tx as unknown as Db, client, source);
|
||||
await tx.insert(mcpOauthDeviceRequests).values({ clientId: client.id, deviceCodeHash: hashMcpSecret(deviceCode), userCodeHash: deviceHash(userCode),
|
||||
await tx.insert(mcpOauthDeviceRequests).values({ clientId: client.id, ingressOwner:mintAuthority?"cloud":"tenant",deviceCodeHash: hashMcpSecret(deviceCode), userCodeHash: deviceHash(userCode),
|
||||
resource: config.resource, scopes, requestedCompanyId: p.data.company_id ?? null, sourceHash, expiresAt: new Date(now.getTime() + 10 * minute) });
|
||||
});
|
||||
return { device_code: deviceCode, user_code: userCode, verification_uri: authorizationOrigin + "/mcp-device", verification_uri_complete: authorizationOrigin + "/mcp-device?user_code=" + userCode, expires_in: 600, interval: 5 };
|
||||
@@ -453,8 +474,10 @@ export function createPublicMcpOAuth(db: Db, config: PublicMcpConfig, options: {
|
||||
await assertEnabled();
|
||||
if (actor.type !== "board" || !actor.userId || !["session", "cloud_tenant"].includes(actor.source ?? "")) throw new McpOAuthError("access_denied", "Sign in to approve a connection.", 401);
|
||||
return db.transaction(async tx => {
|
||||
await tx.execute(sql`SELECT pg_advisory_xact_lock(736721042)`);
|
||||
const [row] = await tx.select().from(mcpOauthDeviceRequests).where(eq(mcpOauthDeviceRequests.userCodeHash, deviceHash(userCode))).for("update");
|
||||
if (!row || row.resource !== config.resource || row.status !== "pending" || row.expiresAt <= new Date()) throw invalidGrant();
|
||||
await prepareMcpClientMutation(tx as unknown as Db);
|
||||
const grant = input.decision === "approve" ? await approveGrant(tx as unknown as Db, actor, row, input) : null;
|
||||
await tx.update(mcpOauthDeviceRequests).set({ status: grant ? "approved" : "denied", grantId: grant?.id ?? null }).where(eq(mcpOauthDeviceRequests.id, row.id));
|
||||
return { status: grant ? "approved" as const : "denied" as const };
|
||||
@@ -470,9 +493,11 @@ export function createPublicMcpOAuth(db: Db, config: PublicMcpConfig, options: {
|
||||
await db.transaction(async (tx) => {
|
||||
// Bound public DCR across replicas, not only per-IP in each process.
|
||||
await tx.execute(sql`SELECT pg_advisory_xact_lock(736721042)`);
|
||||
await prepareMcpClientMutation(tx as unknown as Db);
|
||||
await pruneClients(tx as unknown as Db);
|
||||
await admitClient(tx as unknown as Db, client.registrationSourceHash);
|
||||
await tx.insert(mcpOauthClients).values(client);
|
||||
await recordMcpClientChange(tx as unknown as Db,client.id,"upsert");
|
||||
});
|
||||
return { ...parsed.data, client_id: client.id, client_id_issued_at: Math.floor(Date.now() / 1000) };
|
||||
},
|
||||
|
||||
@@ -0,0 +1,65 @@
|
||||
import { createHash, createHmac } from "node:crypto";
|
||||
|
||||
export interface McpRevocationIndexReceipt {
|
||||
version: 1;
|
||||
authorityId: string;
|
||||
keyId: string;
|
||||
revision: number;
|
||||
generation: string;
|
||||
}
|
||||
export interface McpRevocationIndexIdentity {
|
||||
authorityId: string;
|
||||
companyId: string;
|
||||
operatorId: string;
|
||||
clientId: string;
|
||||
resource: string;
|
||||
keyId: string;
|
||||
revision: number;
|
||||
}
|
||||
export type McpRevocationIndexRequest = McpRevocationIndexIdentity & (
|
||||
| { operation: "begin"; key: string }
|
||||
| { operation: "append"; entries: { fingerprint: string; tokenKind: "access" | "refresh" }[] }
|
||||
| { operation: "complete" }
|
||||
);
|
||||
export type McpRevocationIndexResponse = { complete: false; baseRevision: number } | { complete: true; receipt: McpRevocationIndexReceipt };
|
||||
const digest = (value: unknown): value is string => typeof value === "string" && /^[a-f0-9]{64}$/.test(value);
|
||||
const uuid = (value: unknown): value is string => typeof value === "string" && /^[a-f0-9]{8}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{12}$/.test(value);
|
||||
const object = (value: unknown): value is Record<string, unknown> => Boolean(value && typeof value === "object" && !Array.isArray(value));
|
||||
export function mcpRevocationAuthorityId(grantId: string, keyId: string): string {
|
||||
const hex = createHash("sha256").update(JSON.stringify(["paperclip-mcp-revoke-authority-v1", grantId, keyId])).digest("hex");
|
||||
return `${hex.slice(0,8)}-${hex.slice(8,12)}-4${hex.slice(13,16)}-a${hex.slice(17,20)}-${hex.slice(20,32)}`;
|
||||
}
|
||||
export function mcpRevocationKeyId(key: string): string {
|
||||
if (!/^[A-Za-z0-9_-]{43}$/.test(key) || Buffer.from(key, "base64url").length !== 32) throw new Error("Invalid MCP revocation key");
|
||||
return createHash("sha256").update(Buffer.from(key, "base64url")).digest("hex");
|
||||
}
|
||||
|
||||
|
||||
export function mcpRevocationFingerprint(key: string, tokenHash: string, clientId: string, resource: string): string {
|
||||
mcpRevocationKeyId(key);
|
||||
if (!digest(tokenHash)) throw new Error("Invalid MCP revocation token hash");
|
||||
return createHmac("sha256", Buffer.from(key, "base64url"))
|
||||
.update(JSON.stringify(["paperclip-dot-revoke-index-v1", tokenHash, clientId, resource])).digest("hex");
|
||||
}
|
||||
export function parseMcpRevocationIndexReceipt(value: unknown): McpRevocationIndexReceipt {
|
||||
if (!object(value) || Object.keys(value).sort().join() !== "authorityId,generation,keyId,revision,version"
|
||||
|| value.version !== 1 || !uuid(value.authorityId) || !digest(value.keyId) || !digest(value.generation)
|
||||
|| !Number.isSafeInteger(value.revision) || Number(value.revision) < 0) throw new Error("Invalid MCP revocation receipt");
|
||||
return { version: 1, authorityId: value.authorityId, keyId: value.keyId, revision: Number(value.revision), generation: value.generation };
|
||||
}
|
||||
export function parseMcpRevocationIndexRequest(value: unknown): McpRevocationIndexRequest {
|
||||
if (!object(value) || !uuid(value.authorityId) || !uuid(value.companyId)
|
||||
|| typeof value.operatorId !== "string" || !value.operatorId || value.operatorId.length > 200
|
||||
|| typeof value.clientId !== "string" || !value.clientId || value.clientId.length > 2048
|
||||
|| typeof value.resource !== "string" || value.resource.length > 2048
|
||||
|| !digest(value.keyId) || !Number.isSafeInteger(value.revision) || Number(value.revision) < 0) throw new Error("Invalid MCP revocation identity");
|
||||
const url = new URL(value.resource);
|
||||
if (url.protocol !== "https:" || !["/mcp/runner", "/mcp/paperclip"].includes(url.pathname) || url.search || url.hash || url.username || url.password) throw new Error("Invalid MCP revocation resource");
|
||||
const common = ["authorityId", "companyId", "operatorId", "clientId", "resource", "keyId", "revision", "operation"];
|
||||
const fields = value.operation === "begin" ? ["key"] : value.operation === "append" ? ["entries"] : value.operation === "complete" ? [] : undefined;
|
||||
if (!fields || Object.keys(value).some(key => ![...common, ...fields].includes(key))) throw new Error("Unknown MCP revocation operation");
|
||||
if (value.operation === "begin" && (typeof value.key !== "string" || mcpRevocationKeyId(value.key) !== value.keyId)) throw new Error("MCP revocation key changed");
|
||||
if (value.operation === "append" && (!Array.isArray(value.entries) || !value.entries.length || value.entries.length > 200 || value.entries.some(entry =>
|
||||
!object(entry) || Object.keys(entry).sort().join() !== "fingerprint,tokenKind" || !digest(entry.fingerprint) || !["access", "refresh"].includes(String(entry.tokenKind))))) throw new Error("Invalid MCP revocation page");
|
||||
return value as unknown as McpRevocationIndexRequest;
|
||||
}
|
||||
@@ -0,0 +1,34 @@
|
||||
import { beforeEach, expect, it, vi } from "vitest";
|
||||
import { PgDialect } from "drizzle-orm/pg-core";
|
||||
import type { Db } from "@paperclipai/db";
|
||||
import { startMcpRevocationIndexSync } from "./revocation-index-sync.js";
|
||||
|
||||
const managed=vi.hoisted(()=>({value:null as {stackId:string;canonicalOrigin:string}|null}));
|
||||
vi.mock("../cloud-runtime-identity.js",()=>({getCloudRuntimeIdentity:()=>managed.value}));
|
||||
vi.mock("../paperclip-cloud-connector.js",()=>({paperclipCloudConnectorConfigFromEnv:()=>undefined,createPaperclipCloudConnector:vi.fn()}));
|
||||
beforeEach(()=>{managed.value=null;});
|
||||
function unitOwner(db:Db){
|
||||
const register=vi.fn(),publish=vi.fn();
|
||||
startMcpRevocationIndexSync(db,{register},{key:Buffer.alloc(32,9).toString("base64url"),client:{publish}});
|
||||
return{publish,run:()=>register.mock.calls[0]![1].run(new AbortController().signal)};
|
||||
}
|
||||
it("unit unmanaged index owner reads and publishes no local grant metadata",async()=>{
|
||||
const select=vi.fn(()=>{throw new Error("No unmanaged DB read permitted");});
|
||||
const owner=unitOwner({select} as unknown as Db);await owner.run();
|
||||
expect(select).not.toHaveBeenCalled();expect(owner.publish).not.toHaveBeenCalled();
|
||||
});
|
||||
it("unit current managed index selection uses only its two exact canonical resources",async()=>{
|
||||
managed.value={stackId:"managed-stack",canonicalOrigin:"https://managed.example.test"};
|
||||
const conditions:unknown[]=[];let reads=0;
|
||||
const db={select:()=>({from:()=>({where:(condition:unknown)=>{conditions.push(condition);return{limit:async()=>++reads===1?[{id:"grant"}]:[],orderBy:()=>({limit:async()=>[]})};}})})};
|
||||
const owner=unitOwner(db as unknown as Db);await owner.run();
|
||||
expect(conditions).toHaveLength(2);
|
||||
for(const condition of conditions){const query=new PgDialect().sqlToQuery(condition as Parameters<PgDialect["sqlToQuery"]>[0]);expect(query.params).toContain("https://managed.example.test/mcp/paperclip");expect(query.params).toContain("https://managed.example.test/mcp/runner");expect(query.sql).not.toContain(" ~ ");}
|
||||
expect(owner.publish).not.toHaveBeenCalled();
|
||||
});
|
||||
it("unit identity loss during a local read refuses the outbound publication",async()=>{
|
||||
managed.value={stackId:"managed-stack",canonicalOrigin:"https://managed.example.test"};
|
||||
const grant={id:"grant",companyId:"company",userId:"operator",clientId:"client",resource:"https://managed.example.test/mcp/paperclip",revocationRevision:0,revocationIndexReceipt:null};
|
||||
const db={select:()=>({from:()=>({where:()=>({limit:async()=>[{id:grant.id}],orderBy:()=>({limit:async()=>{managed.value=null;return[{grant}];}})})})})};
|
||||
const owner=unitOwner(db as unknown as Db);await expect(owner.run()).rejects.toThrow("current authority changed");expect(owner.publish).not.toHaveBeenCalled();
|
||||
});
|
||||
@@ -0,0 +1,95 @@
|
||||
import { getCloudRuntimeIdentity } from "../cloud-runtime-identity.js";
|
||||
import { and, asc, eq, gt, inArray, isNull, lte, or, sql } from "drizzle-orm";
|
||||
import { mcpOauthGrants as grants, mcpOauthTokens as tokens, type Db } from "@paperclipai/db";
|
||||
import { createPaperclipCloudConnector, paperclipCloudConnectorConfigFromEnv } from "../paperclip-cloud-connector.js";
|
||||
import { deriveLocalEncryptedInstanceSecret } from "../../secrets/local-encrypted-provider.js";
|
||||
import { DELIVERY_QUEUES } from "../delivery-work-notifications.js";
|
||||
import type { createDeliveryWorkCoordinator } from "../delivery-work-coordinator.js";
|
||||
import { mcpRevocationAuthorityId, mcpRevocationFingerprint, mcpRevocationKeyId, parseMcpRevocationIndexReceipt,
|
||||
type McpRevocationIndexIdentity, type McpRevocationIndexRequest, type McpRevocationIndexResponse } from "./revocation-index-contract.js";
|
||||
|
||||
function response(value: unknown): McpRevocationIndexResponse {
|
||||
if (!value || typeof value !== "object" || Array.isArray(value)) throw new Error("Invalid MCP index response");
|
||||
const row = value as Record<string, unknown>;
|
||||
if (row.complete === true && Object.keys(row).sort().join() === "complete,receipt") return { complete: true, receipt: parseMcpRevocationIndexReceipt(row.receipt) };
|
||||
if (row.complete === false && Object.keys(row).sort().join() === "baseRevision,complete" && Number.isSafeInteger(row.baseRevision) && Number(row.baseRevision) >= -1) return { complete: false, baseRevision: Number(row.baseRevision) };
|
||||
throw new Error("Invalid MCP index response");
|
||||
}
|
||||
export function startMcpRevocationIndexSync(db: Db, coordinator: Pick<ReturnType<typeof createDeliveryWorkCoordinator>, "register">,
|
||||
input: { client?: { publish: (request: McpRevocationIndexRequest) => Promise<unknown> }; key?: string } = {}) {
|
||||
const config = paperclipCloudConnectorConfigFromEnv();
|
||||
const connector = config ? createPaperclipCloudConnector({ config }) : undefined;
|
||||
const client = input.client ?? (connector ? { publish: (request: McpRevocationIndexRequest) => connector.syncMcpRevocationIndex({
|
||||
subject: request.operatorId, companyId: request.companyId, serialized: JSON.stringify(request) }) } : undefined);
|
||||
const requested = new Set<string>(["*"]);
|
||||
return coordinator.register(DELIVERY_QUEUES.mcpRevocationIndex, {
|
||||
retryMs: 5_000, pausePassiveMaintenance: true,
|
||||
onSettled: hints => { for (const hint of hints ?? ["*"]) requested.add(hint); },
|
||||
async run(signal) {
|
||||
|
||||
|
||||
const managed = getCloudRuntimeIdentity();
|
||||
if (!client || !managed) { requested.clear(); return; }
|
||||
const resources = [managed.canonicalOrigin + "/mcp/paperclip", managed.canonicalOrigin + "/mcp/runner"];
|
||||
const assertCurrentAuthority = () => {
|
||||
const current = getCloudRuntimeIdentity(), currentConfig = paperclipCloudConnectorConfigFromEnv();
|
||||
if (!current || current.stackId !== managed.stackId || current.canonicalOrigin !== managed.canonicalOrigin
|
||||
|| config && (!currentConfig || currentConfig.instanceId !== config.instanceId || currentConfig.baseUrl !== config.baseUrl
|
||||
|| currentConfig.environment !== config.environment || currentConfig.signPrivateKey !== config.signPrivateKey
|
||||
|| currentConfig.sealPrivateKey !== config.sealPrivateKey)) throw new Error("MCP revocation index current authority changed");
|
||||
};
|
||||
const publish = (request: McpRevocationIndexRequest) => { assertCurrentAuthority(); return client.publish(request); };
|
||||
const hints = [...requested]; requested.clear();
|
||||
try {
|
||||
const [configured] = await db.select({ id: grants.id }).from(grants)
|
||||
.where(and(isNull(grants.revokedAt), inArray(grants.resource, resources))).limit(1);
|
||||
if (!configured) return;
|
||||
const key = input.key ?? deriveLocalEncryptedInstanceSecret(`mcp-revocation-index:v1:${config!.instanceId}`);
|
||||
const keyId = mcpRevocationKeyId(key);
|
||||
const pending = await db.select({ grant: grants }).from(grants)
|
||||
.where(and(isNull(grants.revokedAt), inArray(grants.resource, resources),
|
||||
sql`(COALESCE(${grants.revocationIndexReceipt}->'revision', 'null'::jsonb) <> to_jsonb(${grants.revocationRevision}) OR COALESCE(${grants.revocationIndexReceipt}->>'keyId', '') <> ${keyId})`))
|
||||
.orderBy(asc(grants.id)).limit(20);
|
||||
if (!pending.length) return;
|
||||
for (const { grant } of pending) {
|
||||
signal.throwIfAborted();
|
||||
if (grant.revocationIndexReceipt && grant.revocationIndexReceipt.keyId !== keyId) {
|
||||
|
||||
|
||||
await db.update(grants).set({ revocationIndexReceipt: null }).where(and(eq(grants.id, grant.id), eq(grants.revocationRevision, grant.revocationRevision)));
|
||||
}
|
||||
const identity: McpRevocationIndexIdentity = { authorityId: mcpRevocationAuthorityId(grant.id, keyId), companyId: grant.companyId, operatorId: grant.userId,
|
||||
clientId: grant.clientId, resource: grant.resource, revision: grant.revocationRevision, keyId };
|
||||
let result = response(await publish({ ...identity, operation: "begin", key }));
|
||||
if (!result.complete) {
|
||||
const baseRevision = result.baseRevision;
|
||||
if (baseRevision > grant.revocationRevision) throw new Error("MCP index base revision is in the future");
|
||||
let cursor: { revision: number; id: string } | undefined;
|
||||
while (!signal.aborted) {
|
||||
const page = await db.select({ id: tokens.id, tokenHash: tokens.tokenHash, kind: tokens.kind, revision: tokens.revocationRevision }).from(tokens)
|
||||
.where(and(eq(tokens.grantId, grant.id), gt(tokens.revocationRevision, baseRevision), lte(tokens.revocationRevision, grant.revocationRevision),
|
||||
cursor ? or(gt(tokens.revocationRevision, cursor.revision), and(eq(tokens.revocationRevision, cursor.revision), gt(tokens.id, cursor.id))) : undefined))
|
||||
.orderBy(asc(tokens.revocationRevision), asc(tokens.id)).limit(200);
|
||||
if (page.some(token => token.kind !== "access" && token.kind !== "refresh")) throw new Error("Unknown MCP token kind cannot be indexed");
|
||||
if (page.length) result = response(await publish({ ...identity, operation: "append", entries: page.map(token => ({
|
||||
tokenKind: token.kind, fingerprint: mcpRevocationFingerprint(key, token.tokenHash, grant.clientId, grant.resource),
|
||||
})) }));
|
||||
if (page.length < 200) break;
|
||||
const last = page.at(-1)!; cursor = { revision: last.revision, id: last.id };
|
||||
}
|
||||
signal.throwIfAborted();
|
||||
result = response(await publish({ ...identity, operation: "complete" }));
|
||||
}
|
||||
if (!result.complete || result.receipt.authorityId !== identity.authorityId || result.receipt.keyId !== keyId || result.receipt.revision !== grant.revocationRevision) throw new Error("MCP revocation publication was not acknowledged");
|
||||
|
||||
|
||||
assertCurrentAuthority();
|
||||
await db.update(grants).set({ revocationIndexReceipt: result.receipt }).where(and(eq(grants.id, grant.id), eq(grants.revocationRevision, grant.revocationRevision), isNull(grants.revokedAt)));
|
||||
}
|
||||
if (pending.length === 20) requested.add("*");
|
||||
} catch (error) { for (const hint of hints) requested.add(hint); throw error; }
|
||||
},
|
||||
hasPending: async () => false,
|
||||
nextRunAt: () => requested.size ? Date.now() + 5_000 : null,
|
||||
});
|
||||
}
|
||||
@@ -0,0 +1,46 @@
|
||||
import { createHash } from "node:crypto";
|
||||
import { describe, expect, it } from "vitest";
|
||||
import { authorizeMcpWakeAdmission, createMcpWakeAdmission, matchesMcpWakeClaim, parseMcpWakeAdmission } from "./wake-admission.js";
|
||||
const context = { stackId: "mcp-vector-stack", ownerId: "00000000-0000-4000-8000-000000000001", capabilityGeneration: "a".repeat(64), challenge: "c".repeat(64) };
|
||||
const token = "pcmcp_at_" + "x".repeat(43);
|
||||
const tokenHash = createHash("sha256").update(token).digest("hex");
|
||||
const future = "2099-01-01T00:00:00.000Z";
|
||||
const make = () => createMcpWakeAdmission({ ...context, tokens: [{ profile: "personal-mcp", tokenKind: "access", tokenHash, expiresAt: future }] });
|
||||
const request = { stackId: context.stackId, token, now: Date.parse("2098-01-01"), profile: "personal-mcp" as const, tokenKind: "access" as const };
|
||||
|
||||
describe("MCP OAuth sleep admission", () => {
|
||||
it("admits only the bound stack, lane and token before expiry without exporting credentials", () => {
|
||||
const receipt = make();
|
||||
const claim = authorizeMcpWakeAdmission(receipt, request)!;
|
||||
expect(claim).toBeDefined();
|
||||
expect(matchesMcpWakeClaim(receipt, claim, request.now)).toBe(true);
|
||||
expect(matchesMcpWakeClaim(receipt, claim, Date.parse(future))).toBe(false);
|
||||
for (const changed of [{ stackId: "foreign" }, { token: "unknown" }, { profile: "dot-mcp" as const },
|
||||
{ tokenKind: "refresh" as const }, { now: Date.parse(future) }, { now: NaN }, { operation: "revoke" as const }]) {
|
||||
expect(authorizeMcpWakeAdmission(receipt, { ...request, ...changed })).toBeUndefined();
|
||||
}
|
||||
expect(JSON.stringify(receipt)).not.toContain(token);
|
||||
expect(JSON.stringify(receipt)).not.toContain(tokenHash);
|
||||
expect(parseMcpWakeAdmission(JSON.parse(JSON.stringify(receipt)))).toEqual(receipt);
|
||||
const rotated = createMcpWakeAdmission({ ...context, challenge: "d".repeat(64), tokens: [{ profile: "personal-mcp", tokenKind: "access", tokenHash, expiresAt: future }] });
|
||||
expect(rotated.lanes[0].keys[0].fingerprint).not.toBe(claim.fingerprint);
|
||||
expect(matchesMcpWakeClaim(rotated, claim, request.now)).toBe(false);
|
||||
});
|
||||
it("binds all four lanes and permits timeless inactivity only for renewable Dot refresh", () => {
|
||||
const tokens = (["personal-mcp", "dot-mcp"] as const).flatMap(profile => (["access", "refresh"] as const).map(tokenKind =>
|
||||
({ profile, tokenKind, tokenHash, clientId: "test-client", expiresAt: profile === "dot-mcp" && tokenKind === "refresh" ? null : future })));
|
||||
const receipt = createMcpWakeAdmission({ ...context, tokens });
|
||||
expect(new Set(receipt.lanes.map(lane => lane.keys[0].fingerprint)).size).toBe(4);
|
||||
expect(authorizeMcpWakeAdmission(receipt, { ...request, profile: "dot-mcp", tokenKind: "refresh", clientId: "test-client", now: Date.parse("2199-01-01") })).toBeDefined();
|
||||
expect(authorizeMcpWakeAdmission(receipt, { ...request, profile: "dot-mcp", tokenKind: "refresh", clientId: "wrong-client" })).toBeUndefined();
|
||||
expect(() => createMcpWakeAdmission({ ...context, tokens: [{ ...tokens[0], expiresAt: null }] })).toThrow();
|
||||
expect(() => createMcpWakeAdmission({ ...context, tokens: Array.from({ length: 1001 }, () => tokens[0]) })).toThrow();
|
||||
});
|
||||
it("fails closed on unknown, altered, duplicate or foreign receipts and empty old-host enrollment", () => {
|
||||
const receipt = make();
|
||||
for (const altered of [undefined, { ...receipt, extra: true }, { ...receipt, generation: "b".repeat(64) },
|
||||
{ ...receipt, ownerId: context.ownerId.replace(/1$/, "2") }, { ...receipt, lanes: [...receipt.lanes, ...receipt.lanes] },
|
||||
{ ...receipt, lanes: [{ ...receipt.lanes[0], tokenKind: "refresh" }] }]) expect(() => parseMcpWakeAdmission(altered)).toThrow();
|
||||
expect(authorizeMcpWakeAdmission(createMcpWakeAdmission({ ...context, tokens: [] }), request)).toBeUndefined();
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,130 @@
|
||||
import { mcpClientMirrorReceiptSchema, type McpClientMirrorReceipt } from "./client-mirror-contract.js";
|
||||
import { parseMcpDiscovery, type McpDiscovery } from "./discovery.js";
|
||||
import { parseMcpRevocationIndexReceipt, type McpRevocationIndexReceipt } from "./revocation-index-contract.js";
|
||||
import { createHash } from "node:crypto";
|
||||
import {
|
||||
apiWakeFingerprint, createApiWakeAdmission,
|
||||
} from "../idle-api-wake-admission.js";
|
||||
|
||||
export const mcpWakeProfiles = ["personal-mcp", "dot-mcp"] as const;
|
||||
export type McpWakeProfile = typeof mcpWakeProfiles[number];
|
||||
export type McpWakeTokenKind = "access" | "refresh";
|
||||
export interface McpWakeClaim {
|
||||
generation: string;
|
||||
fingerprint: string;
|
||||
profile: McpWakeProfile;
|
||||
tokenKind: McpWakeTokenKind;
|
||||
}
|
||||
export interface McpWakeAdmission {
|
||||
version: 1;
|
||||
absent?:true;
|
||||
stackId: string;
|
||||
ownerId: string;
|
||||
capabilityGeneration: string;
|
||||
challenge: string;
|
||||
generation: string;
|
||||
revocationAuthorities: McpRevocationIndexReceipt[];
|
||||
clientMirror?: McpClientMirrorReceipt;
|
||||
discovery?: McpDiscovery;
|
||||
lanes: { profile: McpWakeProfile; tokenKind: McpWakeTokenKind; keys: { fingerprint: string; expiresAt: string | null }[] }[];
|
||||
}
|
||||
type Context = Pick<McpWakeAdmission, "stackId" | "ownerId" | "capabilityGeneration" | "challenge">;
|
||||
const object = (v: unknown): v is Record<string, unknown> => Boolean(v && typeof v === "object" && !Array.isArray(v));
|
||||
const exact = (v: Record<string, unknown>, keys: string[]) => Object.keys(v).length === keys.length && Object.keys(v).every(key => keys.includes(key));
|
||||
const digest = (value: unknown): value is string => typeof value === "string" && /^[a-f0-9]{64}$/.test(value);
|
||||
const hash = (value: unknown) => createHash("sha256").update(JSON.stringify(value)).digest("hex");
|
||||
const laneChallenge = (context: Context, profile: McpWakeProfile, kind: McpWakeTokenKind) =>
|
||||
hash(["paperclip-mcp-oauth-wake-v1", context.stackId, context.ownerId, context.challenge, profile, kind]);
|
||||
const generation = (value: Omit<McpWakeAdmission, "generation">) =>
|
||||
hash(["paperclip-mcp-oauth-wake-receipt-v1", value.version, value.stackId, value.ownerId,
|
||||
value.capabilityGeneration, value.challenge, value.lanes, value.revocationAuthorities, ...(value.discovery ? [value.discovery] : []), ...(value.clientMirror ? [value.clientMirror] : []), ...(value.absent ? ["absent"] : [])]);
|
||||
|
||||
|
||||
|
||||
export function createMcpWakeAdmission(input: Context & {
|
||||
absent?:true;
|
||||
revocationAuthorities?: McpRevocationIndexReceipt[];
|
||||
clientMirror?: McpClientMirrorReceipt;
|
||||
discovery?: McpDiscovery;
|
||||
tokens: { profile: McpWakeProfile; tokenKind: McpWakeTokenKind; tokenHash: string; expiresAt: string | null; clientId?: string }[];
|
||||
}): McpWakeAdmission {
|
||||
const { tokens, revocationAuthorities = [], discovery, clientMirror, absent, ...context } = input;
|
||||
if (tokens.length > 1000) throw new Error("Too many MCP wake credentials");
|
||||
const lanes: McpWakeAdmission["lanes"] = [];
|
||||
for (const profile of mcpWakeProfiles) for (const tokenKind of ["access", "refresh"] as const) {
|
||||
const selected = tokens.filter(token => token.profile === profile && token.tokenKind === tokenKind);
|
||||
if (!selected.length) continue;
|
||||
if (selected.some(token => token.expiresAt === null && !(profile === "dot-mcp" && tokenKind === "refresh"))) {
|
||||
throw new Error("Only Dot refresh credentials can have unbounded inactivity");
|
||||
}
|
||||
const challenge = laneChallenge(context, profile, tokenKind);
|
||||
|
||||
const keys = selected.map(token => {
|
||||
if (!digest(token.tokenHash)) throw new Error("Invalid MCP wake credential hash");
|
||||
if (tokenKind === "refresh" && (!token.clientId || token.clientId.length > 2048)) throw new Error("MCP refresh client is required");
|
||||
const keyHash = tokenKind === "refresh" ? hash(["paperclip-mcp-refresh-client-v1", token.tokenHash, token.clientId]) : token.tokenHash;
|
||||
return { fingerprint: apiWakeFingerprint({ ...context, challenge, keyHash }), expiresAt: token.expiresAt };
|
||||
});
|
||||
lanes.push({ profile, tokenKind, keys: createApiWakeAdmission({ ...context, challenge, keys }).keys });
|
||||
}
|
||||
const receipt = { version: 1 as const, ...context,...(absent?{absent}:{}), lanes, revocationAuthorities: [...revocationAuthorities].sort((a,b) => a.authorityId.localeCompare(b.authorityId)), ...(discovery ? { discovery: parseMcpDiscovery(discovery) } : {}), ...(clientMirror ? {clientMirror:mcpClientMirrorReceiptSchema.parse(clientMirror)} : {}) };
|
||||
return parseMcpWakeAdmission({ ...receipt, generation: generation(receipt) });
|
||||
}
|
||||
|
||||
export function parseMcpWakeAdmission(value: unknown): McpWakeAdmission {
|
||||
if (!object(value) || !exact(value, ["version", "stackId", "ownerId", "capabilityGeneration", "challenge", "generation", "lanes", "revocationAuthorities", ...(object(value) && "discovery" in value ? ["discovery"] : []), ...(object(value) && "clientMirror" in value ? ["clientMirror"] : []), ...(object(value) && "absent" in value ? ["absent"] : [])])
|
||||
|| value.version !== 1 || !digest(value.challenge) || !digest(value.capabilityGeneration) || !digest(value.generation)
|
||||
|| !Array.isArray(value.lanes) || value.lanes.length > 4 || !Array.isArray(value.revocationAuthorities) || value.revocationAuthorities.length > 1000) throw new Error("Invalid MCP wake receipt");
|
||||
const context = { stackId: value.stackId, ownerId: value.ownerId, capabilityGeneration: value.capabilityGeneration, challenge: value.challenge } as Context;
|
||||
|
||||
createApiWakeAdmission({ ...context, keys: [] });
|
||||
let previous = "", total = 0;
|
||||
const lanes = value.lanes.map(lane => {
|
||||
if (!object(lane) || !exact(lane, ["profile", "tokenKind", "keys"])
|
||||
|| !mcpWakeProfiles.includes(lane.profile as McpWakeProfile) || !["access", "refresh"].includes(String(lane.tokenKind))) throw new Error("Invalid MCP wake lane");
|
||||
const profile = lane.profile as McpWakeProfile, tokenKind = lane.tokenKind as McpWakeTokenKind;
|
||||
const order = `${mcpWakeProfiles.indexOf(profile)}:${tokenKind}`;
|
||||
if (order <= previous) throw new Error("MCP wake lanes must be canonical and unique");
|
||||
previous = order;
|
||||
if (!Array.isArray(lane.keys)) throw new Error("Invalid MCP wake credentials");
|
||||
const keys = createApiWakeAdmission({ ...context, challenge: laneChallenge(context, profile, tokenKind),
|
||||
keys: lane.keys as McpWakeAdmission["lanes"][number]["keys"] }).keys;
|
||||
total += keys.length;
|
||||
if (!keys.length || total > 1000 || JSON.stringify(keys) !== JSON.stringify(lane.keys)
|
||||
|| keys.some(key => key.expiresAt === null && !(profile === "dot-mcp" && tokenKind === "refresh"))) throw new Error("MCP wake lane changed");
|
||||
return { profile, tokenKind, keys };
|
||||
});
|
||||
let previousAuthority = "";
|
||||
const revocationAuthorities = value.revocationAuthorities.map(authority => {
|
||||
const parsed = parseMcpRevocationIndexReceipt(authority);
|
||||
if (parsed.authorityId <= previousAuthority) throw new Error("MCP revocation authorities must be canonical and unique");
|
||||
previousAuthority = parsed.authorityId; return parsed;
|
||||
});
|
||||
if("absent" in value&&(value.absent!==true||lanes.length||revocationAuthorities.length||"discovery"in value||"clientMirror"in value))throw new Error("Absent MCP admission contains authority");
|
||||
const parsed = { version: 1 as const,...("absent"in value?{absent:true as const}:{}), stackId: context.stackId, ownerId: context.ownerId,
|
||||
capabilityGeneration: context.capabilityGeneration, challenge: context.challenge, generation: value.generation, lanes, revocationAuthorities, ...("discovery" in value ? { discovery: parseMcpDiscovery(value.discovery) } : {}), ...("clientMirror" in value ? {clientMirror:mcpClientMirrorReceiptSchema.parse(value.clientMirror)} : {}) };
|
||||
if (generation(parsed) !== parsed.generation) throw new Error("MCP wake receipt generation changed");
|
||||
return parsed;
|
||||
}
|
||||
|
||||
|
||||
export function authorizeMcpWakeAdmission(receipt: McpWakeAdmission, input: {
|
||||
stackId: string; token: string; now: number; profile: McpWakeProfile; tokenKind: McpWakeTokenKind; clientId?: string; operation?: "use" | "revoke";
|
||||
}): McpWakeClaim | undefined {
|
||||
if (input.operation === "revoke" || !Number.isFinite(input.now) || receipt.stackId !== input.stackId) return;
|
||||
const lane = receipt.lanes.find(lane => lane.profile === input.profile && lane.tokenKind === input.tokenKind);
|
||||
if (!lane || !input.token || input.token.length > 8192 || /\s/.test(input.token)) return;
|
||||
const tokenHash = createHash("sha256").update(input.token).digest("hex");
|
||||
if (input.tokenKind === "refresh" && (!input.clientId || input.clientId.length > 2048)) return;
|
||||
const keyHash = input.tokenKind === "refresh" ? hash(["paperclip-mcp-refresh-client-v1", tokenHash, input.clientId]) : tokenHash;
|
||||
const fingerprint = apiWakeFingerprint({ ...receipt, challenge: laneChallenge(receipt, input.profile, input.tokenKind), keyHash });
|
||||
return lane.keys.some(key => key.fingerprint === fingerprint && (key.expiresAt === null || Date.parse(key.expiresAt) > input.now))
|
||||
? { generation: receipt.generation, fingerprint, profile: input.profile, tokenKind: input.tokenKind } : undefined;
|
||||
}
|
||||
|
||||
|
||||
export function matchesMcpWakeClaim(receipt: McpWakeAdmission, claim: McpWakeClaim, now: number): boolean {
|
||||
return Number.isFinite(now) && receipt.generation === claim.generation && receipt.lanes.some(lane =>
|
||||
lane.profile === claim.profile && lane.tokenKind === claim.tokenKind && lane.keys.some(key =>
|
||||
key.fingerprint === claim.fingerprint && (key.expiresAt === null || Date.parse(key.expiresAt) > now)));
|
||||
}
|
||||
Reference in new issue
Block a user